From 56feb484120d9040ca5314ba43061e5d1654d384 Mon Sep 17 00:00:00 2001 From: Andrew Innes Date: Sat, 30 Jul 2022 15:29:41 +0800 Subject: [PATCH] Squashed commit of the following: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit commit e8cf3a4f7662f2d1c13684ce52b73ab0d9a12266 Author: Alek P Date: Thu Jul 28 18:52:46 2022 -0400 Implement a new type of zfs receive: corrective receive (-c) This type of recv is used to heal corrupted data when a replica of the data already exists (in the form of a send file for example). With the provided send stream, corrective receive will read from disk blocks described by the WRITE records. When any of the reads come back with ECKSUM we use the data from the corresponding WRITE record to rewrite the corrupted block. Reviewed-by: Paul Dagnelie Reviewed-by: Brian Behlendorf Reviewed-by: Paul Zuchowski Signed-off-by: Alek Pinchuk Closes #9372 commit 5fae33e04771e255f3dba57263fd06eb68bd38b5 Author: Tino Reichardt Date: Thu Jul 28 23:19:41 2022 +0200 FreeBSD compile fix The file module/os/freebsd/zfs/zfs_ioctl_compat.c fails compiling because of this error: 'static' is not at beginning of declaration This commit fixes the three places within that file. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Closes #13702 commit 34aa0f0487705671c81262adb7646a90d15c5a12 Author: Brian Behlendorf Date: Tue Jul 26 14:39:23 2022 -0700 ZTS: Fix io_uring support check Not all Linux distribution kernels enable io_uring support by default. Update the run time check to verify that the booted kernel was built with CONFIG_IO_URING=y. Reviewed-by: Tony Hutter Reviewed-by: Tony Nguyen Co-authored-by: George Melikov Signed-off-by: Brian Behlendorf Closes #13648 Closes #13685 commit 3a1ce4914172ce4c1e39123cd31b1e5245765a5e Author: Ameer Hamza <106930537+ixhamza@users.noreply.github.com> Date: Tue Jul 26 02:04:46 2022 +0500 Add createtxg sort support for simple snapshot iterator - When iterating snapshots with name only, e.g., "-o name -s name", libzfs uses simple snapshot iterator and results are displayed in alphabetic order. This PR adds support for faster version of createtxg sort by avoiding nvlist parsing for properties. Flags "-o name -s createtxg" will enable createtxg sort while using simple snapshot iterator. - Added support to read createtxg property directly from zfs handle for filesystem, volume and snapshot types instead of parsing nvlist. Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Signed-off-by: Ameer Hamza Closes #13577 commit 8792dd24cd9599cf506d45bcaed3af78c8cd888d Author: Brian Behlendorf Date: Mon Jul 25 09:52:42 2022 -0700 ZTS: Fix occasional inherit_001_pos.ksh failure The mountpoint may still be busy when the `zfs unmount -a` command is run causing an unexpected failure. Retry the unmount a couple of times since it should not remain busy for long. 19:10:50.29 NOTE: Reading state from .../inheritance/state021.cfg 19:10:50.32 cannot unmount '/TESTPOOL': pool or dataset is busy 19:10:50.32 ERROR: zfs unmount -a exited 1 Reviewed-by: George Melikov Signed-off-by: Brian Behlendorf Closes #13686 commit bf61a507a276866d691a2b56866302bc42145af3 Author: Christian Schwarz Date: Thu Jul 21 02:16:29 2022 +0200 zdb: dump spill block pointer if present Output will look like so: $ sudo zdb -dddd -vv testpool/fs 2 Dataset testpool/fs [ZPL], ID 260, cr_txg 8, 25K, 7 objects, rootbp DVA[0]=<0:1800be00:200> DVA[1]=<0:1c00be00:200> [L0 DMU objset] fletcher4 lz4 unencrypted LE contiguous unique double size=1000L/200P birth=16L/16P fill=7 cksum=d03b396cd:489ca835517:d4b04a4d0a62:1b413aac454d53 Object lvl iblk dblk dsize dnsize lsize %full type 2 1 128K 512 1K 512 512 0.00 ZFS plain file (K=inherit) (Z=inherit=lz4) 192 bonus System attributes dnode flags: USED_BYTES USERUSED_ACCOUNTED USEROBJUSED_ACCOUNTED SPILL_BLKPTR dnode maxblkid: 0 path /testfile uid 0 gid 0 atime Fri Jul 15 12:36:35 2022 mtime Fri Jul 15 12:36:35 2022 ctime Fri Jul 15 12:36:51 2022 crtime Fri Jul 15 12:36:35 2022 gen 10 mode 100600 size 0 parent 34 links 1 pflags 840800000004 SA xattrs: 248 bytes, 2 entries security.selinux = nutanix_u:object_r:unlabeled_t:s0\000 user.foo = xbLQJjyVvEVPGGuRHV/gjkFFO1MdehKnLjjd36ZaoMVaUqtqFoMMYT5Ya9yywHApJNoK/1hNJfO3\012XCJWv9/QUTKamoWW9xVDE7yi8zn166RNw5QUhf84cZ3JNLnw6oN Spill block: 0:10005c00:200 0:14005c00:200 200L/200P F=1 B=16/16 cksum=1cdfac47a4:910c5caa557:195d0493dfe5a:332b6fde6ad547 Indirect blocks: Reviewed-by: Brian Behlendorf Reviewed-by: Allan Jude Signed-off-by: Christian Schwarz Closes #13640 commit fb087146de0118108e3b44222d2052415dcb1f7f Author: ixhamza <106930537+ixhamza@users.noreply.github.com> Date: Thu Jul 21 05:14:06 2022 +0500 Add support for per dataset zil stats and use wmsum counters ZIL kstats are reported in an inclusive way, i.e., same counters are shared to capture all the activities happening in zil. Added support to report zil stats for every datset individually by combining them with already exposed dataset kstats. Wmsum uses per cpu counters and provide less overhead as compared to atomic operations. Updated zil kstats to replace wmsum counters to avoid atomic operations. Reviewed-by: Christian Schwarz Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Signed-off-by: Ameer Hamza Closes #13636 commit 33dba8c79224ce33dc661d545ab1d17fc3d84a0c Author: Alexander Motin Date: Wed Jul 20 20:02:36 2022 -0400 Fix scrub resume from newly created hole It may happen that scan bookmark points to a block that was turned into a part of a big hole. In such case dsl_scan_visitbp() may skip it and dsl_scan_check_resume() will not be called for it. As result new scan suspend won't be possible until the end of the object, that may take hours if the object is a multi-terabyte ZVOL on a slow HDD pool, stretching TXG to all that time, creating all sorts of problems. This patch changes the resume condition to any greater or equal block, so even if we miss the bookmarked block, the next one we find will delete the bookmark, allowing new suspend. Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Closes #13643 commit 97fd1ea42a59b85ca29c91056ceef56c12cfae0b Author: Tino Reichardt Date: Thu Jul 21 02:01:32 2022 +0200 Fix memory allocation for the checksum benchmark Allocation via kmem_cache_alloc() is limited to less then 4m for some architectures. This commit limits the benchmarks with the linear abd cache to 1m on all architectures and adds 4m + 16m benchmarks via non-linear abd_alloc(). Reviewed-by: Brian Behlendorf Reviewed-by: Alexander Motin Co-authored-by: Sebastian Gottschall Signed-off-by: Tino Reichardt Closes #13669 Closes #13670 commit f371cc18f81168c74314b77480862b6c516e15d5 Author: ixhamza <106930537+ixhamza@users.noreply.github.com> Date: Thu Jul 14 22:38:16 2022 +0500 Expose ZFS dataset case sensitivity setting via sb_opts Makes the case sensitivity setting visible on Linux in /proc/mounts. Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Signed-off-by: Ameer Hamza Closes #13607 commit 9fe2f262aa24e3eda716787005cd127642aed22b Author: Tony Hutter Date: Thu Jul 14 10:19:37 2022 -0700 zed: Look for NVMe DEVPATH if no ID_BUS We tried replacing an NVMe drive using autoreplace, only to see zed reject it with: zed[27955]: zed_udev_monitor: /dev/nvme5n1 no devid source This happened because ZED saw that ID_BUS was not set by udev for the NVMe drive, and thus didn't think it was "real drive". This commit allows NVMe drives to be autoreplaced even if ID_BUS is not set. Reviewed-by: Don Brady Reviewed-by: Brian Behlendorf Signed-off-by: Tony Hutter Closes #13512 Closes #13646 commit 1d3ba0bf01020f5459b1c28db3979129088924c0 Author: Tino Reichardt Date: Mon Jul 11 23:16:13 2022 +0200 Replace dead opensolaris.org license link The commit replaces all findings of the link: http://www.opensolaris.org/os/licensing with this one: https://opensource.org/licenses/CDDL-1.0 Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Closes #13619 commit e4ab3f40df994178c5fc629c2ad07c505f5a76eb Author: Tony Hutter Date: Mon Jul 11 13:35:19 2022 -0700 zed: Ignore false 'atari' partitions in autoreplace libudev will sometimes falsely identify an 'atari' partition on a blank disk, preventing it from being used in an autoreplace. This seems to be a known issue. The workaround is to just ignore the fake partition and continue with the autoreplace. Reviewed-by: Brian Behlendorf Signed-off-by: Tony Hutter Closes #13497 Closes #13632 commit 677ca1e825af80f60569f84803304ccf0092728b Author: Tony Hutter Date: Mon Jul 11 11:35:01 2022 -0700 rpm: Silence "unversioned Obsoletes" warnings on EL 9 Get rid of RPM warnings on AlmaLinux 9: "It's not recommended to have unversioned Obsoletes" Reviewed-by: Brian Behlendorf Signed-off-by: Tony Hutter Closes #13584 Closes #13638 commit e6489be3470ecc1088b69a17b63a6d45d57f1c16 Author: Brian Behlendorf Date: Mon Jul 11 11:29:12 2022 -0700 Linux: Align MODULE_LICENSE macro text Specify the lua and zstd license text in the manor in which the kernel MODULE_LICENSE macro requires it. The now duplicate entries were merged and a comment added to make it clear what they apply to. Reviewed-by: Christian Schwarz Signed-off-by: Brian Behlendorf Closes #13641 commit cb01da68057dcb9e612e8d2e97d058c46c3574af Author: Finix1979 Date: Fri Jul 8 02:43:58 2022 +0800 Call nvlist_free before return Fixes a small kernel memory leak which would occur if a pool failed to import because the `DMU_POOL_VDEV_ZAP_MAP` key can't be read from a presumably damaged MOS config. In the case of a missing key there was no leak. Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: Finix1979 Closes #13629 commit 74230a5bc1be6e5e84a5f41b26f6f65a155078f0 Author: Alexander Motin Date: Tue Jul 5 19:27:29 2022 -0400 Avoid memory copy when verifying raidz/draid parity Before this change for every valid parity column raidz_parity_verify() allocated new buffer and copied there existing data, then recalculated the parity and compared the result with the copy. This patch removes the memory copy, simply swapping original buffer pointers with newly allocated empty ones for parity recalculation and comparison. Original buffers with potentially incorrect parity data are then just freed, while new recalculated ones are used for repair. On a pool of 12 4-wide raidz vdevs, storing 1.5TB of 16MB blocks, this change reduces memory traffic during scrub by 17% and total unhalted CPU time by 25%. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Closes #13613 commit 1ac7d194e5ca31d5284e410a87ad9f9669a7f5b5 Author: Alexander Motin Date: Tue Jul 5 19:26:20 2022 -0400 Avoid memory copies during mirror scrub Issuing several scrub reads for a block we may use the parent ZIO buffer for one of child ZIOs. If that read complete successfully, then we won't need to copy the data explicitly. If block has only one copy (typical for root vdev, which is also a mirror inside), then we never need to copy -- succeed or fail as-is. Previous code also copied data from buffer of every successfully completed child ZIO, but that just does not make any sense. On healthy N-wide mirror this saves all N+1 (or even more in case of ditto blocks) memory copies for each scrubbed block, allowing CPU to focus mostly on check-summing. For other vdev types it should save one memory copy per block copy at root vdev. Reviewed-by: Brian Behlendorf Reviewed-by: Mark Maybee Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Closes #13606 commit 6fca6195cdf3079a6e11444e42d7ce4aca10c6a1 Author: наб Date: Thu Jun 30 20:31:09 2022 +0200 Re-fix -Wwrite-strings on FreeBSD Follow up fix for a926aab902ac5c680f4766568d19674b80fb58bb. Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13348 Closes #13610 commit eefe83eaa68f7cb4a49c580dd940d3688e42c849 Author: Toyam Cox Date: Thu Jun 30 13:47:58 2022 -0400 dracut: fix boot on non-zfs-root systems Simply prevent overwriting root until it needs to be overwritten. Dracut could change this value before this module is called, but won't change the kernel command line. Reviewed-by: Andrew J. Hesford Signed-off-by: Toyam Cox Closes #13592 commit 5a4dd3a262106c1506bdc154d9474efa6510f33a Author: gregory-lee-bartholomew Date: Thu Jun 30 12:43:27 2022 -0500 contrib: dracut: README.md Change zfs-snapshot-bootfs.service to zfs-rollback-bootfs.service in cmdline point 4 of README.md. Reviewed-by: Brian Behlendorf Signed-off-by: Gregory Bartholomew Closes #13609 commit 34e5423f83202653bd6b153577187f6ed943c157 Author: George Amanakis Date: Thu Jun 30 02:06:16 2022 +0200 Fix dnode byteswapping If a dnode has a spill pointer, and we use DN_SLOTS_TO_BONUSLEN() then we will possibly include the spill pointer in the len calculation and it will be byteswapped. Then dnode_byteswap() will carry on and swap the spill pointer again. Fix this by using DN_MAX_BONUS_LEN() instead. Reviewed-by: Brian Behlendorf Signed-off-by: George Amanakis Closes #13002 Closes #13015 commit 2d434e8ae4139ce14a1b058839a144bf952e79ea Author: gregory-lee-bartholomew Date: Wed Jun 29 18:56:04 2022 -0500 contrib: dracut: zfs-{rollback,snapshot}-bootfs: explicit snapname fix Due to a missing semicolon on the ExecStart line, it wasn't possible to specify the snapshot name on the bootfs.{rollback,snapshot} kernel parameters if the boot dataset name was obtained from the root=zfs:... kernel parameter. Reviewed-by: Ahelenia Ziemiańska Reviewed-by: Brian Behlendorf Signed-off-by: Gregory Bartholomew Closes #13585 commit 07f2793e869196fcbcd5057d9ada377674262fe3 Author: Brian Behlendorf Date: Wed Jun 29 15:33:38 2022 -0700 dracut: fix typo in mount-zfs.sh.in Format the `zpool get` command correctly. The -o option must be followed by "all" or the requested field name. Reviewed-by: Ahelenia Ziemiańska Reviewed-by: George Melikov Signed-off-by: Brian Behlendorf Closes #13602 commit 60e389ca10085acfa7cd35f79ab4465d968a942f Author: наб Date: Tue Jun 28 23:31:55 2022 +0200 module: lua: ldo: fix pragma name /home/nabijaczleweli/store/code/zfs/module/lua/ldo.c:175:32: warning: unknown option after ‘#pragma GCC diagnostic’ kind [-Wpragmas] 175 | #pragma GCC diagnostic ignored "-Winfinite-recursion"a | ^~~~~~~~~~~~~~~~~~~~~~ Fixes: a6e8113fed8a508ffda13cf1c4d8da99a4e8133a ("Silence -Winfinite-recursion warning in luaD_throw()") Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13348 commit 404601aca07dc44273c1ab33e09067423f97c591 Author: наб Date: Tue Jun 28 23:27:44 2022 +0200 linux: libzfs: util: don't fallthrough to to end-of-switch lib/libzfs/os/linux/libzfs_util_os.c:262:3: error: fallthrough annotation does not directly precede switch label zfs_fallthrough; ^ ./lib/libspl/include/sys/feature_tests.h:34:26: note: expanded from macro 'zfs_fallthrough' Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13348 commit a2f6bff976e43c58d8460c288bac13868c614447 Author: наб Date: Sat May 28 15:19:05 2022 +0200 tests: modernise zdb_decompress Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13348 commit dd66857d92d86643bda57b92fdd58f016bd1725e Author: наб Date: Tue Apr 19 20:49:30 2022 +0200 Remaining {=> const} char|void *tag Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13348 commit a926aab902ac5c680f4766568d19674b80fb58bb Author: наб Date: Tue Apr 19 20:38:30 2022 +0200 Enable -Wwrite-strings Also, fix leak from ztest_global_vars_to_zdb_args() Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13348 commit e7d90362e5d5f873e1272519da96780cf00a0e28 Author: gaoyanping Date: Thu Jun 30 04:38:46 2022 +0800 Fix znode group permission different from acl mask Zp->z_mode is set at the same time inode->i_mode is being changed. This has the effect of keeping both in sync without relying on zfs_znode_update_vfs. Reviewed-by: Brian Behlendorf Signed-off-by: yanping.gao Closes #13581 commit 325096545a47aa01cd5966301ba1f7a6e5eff349 Author: Kristof Provost Date: Tue Jun 28 23:11:38 2022 +0200 FreeBSD: only define B_FALSE/B_TRUE if NEED_SOLARIS_BOOLEAN is not set If NEED_SOLARIS_BOOLEAN is defined we define an enum boolean_t, which defines B_TRUE/B_FALSE as well. If we have both the define and the enum things don't build (because that translates to 'enum { 0, 1 } boolean_t'). While here also remove an incorrect '#else'. With it in place we only parse a section if the include guard is triggered. So we'd only use that code if this file is included twice. This is clearly unintended, and also means we don't get the 'boolean_t' definition. Fix this. Reviewed-by: Warner Losh Reviewed-by: Ryan Moeller Signed-off-by: Kristof Provost Sponsored-By: Rubicon Communications, LLC ("Netgate") Closes #13596 commit 827322991f25785ff032ad3cef84e12b1c609759 Author: Alexander Motin Date: Tue Jun 28 14:23:31 2022 -0400 Fix and disable blocks statistics during scrub Block statistics calculation during scrub I/O issue in case of sorted scrub accounted ditto blocks several times. Embedded blocks on other side were not accounted at all. This change moves the accounting from issue to scan stage, that fixes both problems and also allows to avoid pool-wide locking and the lock contention it created. Since this statistics is quite specific and is not even exposed now anywhere, disable its calculation by default to not waste CPU time. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Closes #13579 commit 43569ee374208e827409ec1ce4cf169d7a9a3095 Author: Brian Behlendorf Date: Mon Jun 20 23:36:21 2022 +0000 Fix objtool: missing int3 after ret warning Resolve straight-line speculation warnings reported by objtool for x86_64 assembly on Linux when CONFIG_SLS is set. See the following LWN article for the complete details. https://lwn.net/Articles/877845/ Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Signed-off-by: Brian Behlendorf Closes #13528 Closes #13575 commit 8aceded193f58da9a3837fb9d828dab05ad9e82f Author: Brian Behlendorf Date: Mon Jun 20 22:27:55 2022 +0000 Fix -Wformat-overflow warning in zfs_project_handle_dir() Switch to using asprintf() to satisfy the compiler and resolve the potential format-overflow warning. Not the conditional before the sprintf() would have prevented this regardless. cmd/zfs/zfs_project.c: In function ‘zfs_project_handle_dir’: cmd/zfs/zfs_project.c:241:38: error: ‘/’ directive writing 1 byte into a region of size between 0 and 4352 [-Werror=format-overflow=] cmd/zfs/zfs_project.c:241:17: note: ‘sprintf’ output between 2 and 4609 bytes into a destination of size 4352 Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Signed-off-by: Brian Behlendorf Closes #13528 Closes #13575 commit f11431a31776734722e14bd6186b93a25823a0ee Author: Brian Behlendorf Date: Mon Jun 20 21:54:42 2022 +0000 Fix -Wformat-truncation warning in upgrade_set_callback() Extend the buffer slightly resolve the warning. cmd/zfs/zfs_main.c: In function ‘upgrade_set_callback’: cmd/zfs/zfs_main.c:2446:22: error: ‘%llu’ directive output may be truncated writing between 1 and 20 bytes into a region of size 16 [-Werror=format-truncation=] cmd/zfs/zfs_main.c:2445:24: note: ‘snprintf’ output between 2 and 21 bytes into a destination of size 16 Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Signed-off-by: Brian Behlendorf Closes #13528 Closes #13575 commit c175f5ebb2f4910e2d4f38f794e3973e53baa94e Author: Brian Behlendorf Date: Mon Jun 20 21:35:38 2022 +0000 Fix -Wuse-after-free warning in dbuf_destroy() Move the use of the db pointer after it is freed. It's only used as a tag so a dereference would never occur, but there's no reason we can't invert the order to resolve the warning. module/zfs/dbuf.c: In function 'dbuf_destroy': module/zfs/dbuf.c:2953:17: error: pointer 'db' may be used after 'free' [-Werror=use-after-free] Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Signed-off-by: Brian Behlendorf Closes #13528 Closes #13575 commit 9619bcdefb0dff38e36f8c89d9e2980112105cbb Author: Brian Behlendorf Date: Mon Jun 20 21:32:03 2022 +0000 Fix -Wuse-after-free warning in dbuf_issue_final_prefetch_done() Move the use of the private pointer after it is freed. It's only used as a tag so a dereference would never occur, but there's no harm in inverting the order to resolve the warning. module/zfs/dbuf.c: In function 'dbuf_issue_final_prefetch_done': module/zfs/dbuf.c:3204:17: error: pointer 'private' may be used after 'free' [-Werror=use-after-free] Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Signed-off-by: Brian Behlendorf Closes #13528 Closes #13575 commit ff7e405f83fbfcd763c4b7ed8b68258227765731 Author: Brian Behlendorf Date: Mon Jun 20 21:13:26 2022 +0000 Fix -Wattribute-warning in dsl layer The memcpy(), memmove(), and memset() functions have been annotated to perform bounds checking when using FORTIFY_SOURCE. A warning is now generted when writing beyond the end of the specified field. Alternately, the new struct_group() macro could be used to create an anonymous union member for use by memcpy(). However, since this is the only place the macro would be helpful it's preferable to restructure the code slights to avoid the need for additional compatibility code when the macro does not exist. https://lore.kernel.org/lkml/20211118183807.1283332-1-keescook@chromium.org/T/ Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Signed-off-by: Brian Behlendorf Closes #13528 Closes #13575 commit 18df6afdfc63b7c27cbb2b6152d76c40196e9dbb Author: Brian Behlendorf Date: Mon Jun 20 19:37:38 2022 +0000 Fix -Wattribute-warning in edonr The wrong union memory was being accessed in EdonRInit resulting in a write beyond size of field compiler warning. Reference the correct member to resolve the warning. The warning was correct and this in case the mistake was harmless. In function ‘fortify_memcpy_chk’, inlined from ‘EdonRInit’ at zfs/module/icp/algs/edonr/edonr.c:494:3: ./include/linux/fortify-string.h:344:25: error: call to ‘__write_overflow_field’ declared with attribute warning: detected write beyond size of field (1st parameter); maybe use struct_group()? [-Werror=attribute-warning] Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Signed-off-by: Brian Behlendorf Closes #13528 Closes #13575 commit b0f7dd276c930129fef8575e15a36ec659e31cd2 Author: Brian Behlendorf Date: Mon Jun 20 22:36:38 2022 +0000 Fix -Wattribute-warning in zfs_log_xvattr() Restructure the code in zfs_log_xvattr() to use a lr_attr_end structure when accessing lr_attr_t elements located after the variable sized array. This makes the code more understandable and resolves the accessing beyond the end of the field warnings. Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Signed-off-by: Brian Behlendorf Closes #13528 Closes #13575 commit a6e8113fed8a508ffda13cf1c4d8da99a4e8133a Author: Brian Behlendorf Date: Mon Jun 20 19:53:58 2022 +0000 Silence -Winfinite-recursion warning in luaD_throw() This code should be kept inline with the upstream lua version as much as possible. Therefore, we simply want to silence the warning. This check was enabled by default as part of -Wall in gcc 12.1. Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Signed-off-by: Brian Behlendorf Closes #13528 Closes #13575 commit 80a650b7bb04bce3aef5e4cfd1d966e3599dafd4 Author: George Amanakis Date: Mon Jun 27 23:17:25 2022 +0200 Avoid panic with recordsize > 128k, raw sending and no large_blocks The current codebase does not support raw sending buffers with block size > 128kB when large_blocks is not active. This can happen in the codepath dsl_dataset_sync()->dmu_objset_sync()->zio_nowait() which calls back dmu_objset_write_done()->dsl_dataset_block_born(). If dsl_dataset_sync() completes its run before dsl_dataset_block_born() is called, we will end up not activating some of the necessary flags, while having blocks based on those flags written in the filesystem. A subsequent send will then panic. Fix this by directly deciding in dmu_objset_sync() whether these flags need to be activated later by dsl_dataset_sync(). Instead of panicking due to a NULL pointer dereference in dmu_dump_write() in case of a send, print out an error message. Also during scrub verify there are no contradicting filesystem flags. Reviewed-by: Paul Dagnelie Signed-off-by: George Amanakis Closes #12275 Closes #12438 commit 1cd72b9c1348bbc71310fb6c3d49670362faf306 Author: Alexander Motin Date: Mon Jun 27 14:08:21 2022 -0400 Avoid two 64-bit divisions per scanned block Change math to make it like the ARC, using multiplications instead. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Closes #13591 commit c0bf952c846100750f526c2a32ebec17694a201b Author: Alexander Motin Date: Fri Jun 24 16:55:58 2022 -0400 Several B-tree optimizations - Introduce first element offset within a leaf. It allows to reduce by ~50% average memmove() size when adding/removing elements. If the added/removed element is in the first half of the leaf, we may shift elements before it and adjust the bth_first instead of moving more elements after it. - Use memcpy() instead of memmove() when we know there is no overlap. - Switch from uint64_t to uint32_t. It does not limit anything, but 32-bit arches should appreciate it greatly in hot paths. - Store leaf capacity in struct btree to avoid 64-bit divisions. - Adjust zfs_btree_insert_into_leaf() to always result in balanced leaves after splitting, no matter where the new element was inserted. Not that we care about it much, but it should also allow B-trees with as little as two elements per leaf instead of 4 previously. When scrubbing pool of 12 SSDs, storing 1.5TB of 4KB zvol blocks this reduces amount of time spent in memmove() inside the scan thread from 13.7% to 5.7% and total scrub time by ~15 seconds out of 9 minutes. It should also reduce spacemaps load time, but I haven't measured it. Reviewed-by: Paul Dagnelie Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Closes #13582 commit ccf89b39fe7f30dd53aec69e04de3f2728c7387c Author: Alan Somers Date: Fri Jun 24 14:28:42 2022 -0600 Add a "zstream decompress" subcommand It can be used to repair a ZFS file system corrupted by ZFS bug #12762. Use it like this: zfs send -c | \ zstream decompress ,[,] ... | \ zfs recv Reviewed-by: Ahelenia Ziemiańska Reviewed-by: Brian Behlendorf Reviewed-by: Allan Jude Signed-off-by: Alan Somers Sponsored-by: Axcient Workaround for #12762 Closes #13256 commit 1c0c729ab4165cd828fbeab404353b45b3836360 Author: Alexander Motin Date: Fri Jun 24 12:50:37 2022 -0400 Several sorted scrub optimizations - Reduce size and comparison complexity of q_exts_by_size B-tree. Previous code used two 64-bit divisions and many other operations to compare two B-tree elements. It created enormous overhead. This implementation moves the math to the upper level and stores the score in the B-tree elements themselves. Since all that we need to store in that B-tree is the extent score and offset, those can fit into single 8 byte value instead of 24 bytes of q_exts_by_addr element and can be compared with single operation. - Better decouple secondary tree logic from main range_tree by moving rt_btree_ops and related functions into dsl_scan.c as ext_size_ops. Those functions are very small to worry about the code duplication and range_tree does not need to know details such as rt_btree_compare. - Instead of accounting number of pending bytes per pool, that needs atomic on global variable per block, account the number of non-empty per-vdev queues, that change much more rarely. - When extent scan is interrupted by TXG end, continue it in the next TXG instead of selecting next best extent. It allows to avoid leaving one truncated (and so likely not the best any more) extent each TXG. On top of some other optimizations this saves about 1.5 minutes out of 10 to scrub pool of 12 SSDs, storing 1.5TB of 4KB zvol blocks. Reviewed-by: Paul Dagnelie Reviewed-by: Brian Behlendorf Reviewed-by: Tom Caputi Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Closes #13576 commit 83691bebf062d13e64a93ff80bf727cd1c162bb2 Author: Toomas Soome Date: Fri Jun 24 19:48:10 2022 +0300 Use macros for quotes and such Use Dq,Pq/Po/Pc macros. illumos dumpadm is now in section 8. Reviewed-by: Brian Behlendorf Signed-off-by: Toomas Soome Closes #13586 commit ad8b9f940c1e39a38af61934737c1e4cf8ab5c08 Author: Brian Behlendorf Date: Thu Jun 23 10:36:28 2022 -0700 Scrub mirror children without BPs When scrubbing a raidz/draid pool, which contains a replacing or sparing mirror with multiple online children, only one child will be read. This is not normally a serious concern because the DTL records are used to determine where a good copy of the data is. As long as the data can be read from one child the mirror vdev will use it to repair gaps in any of its children. Furthermore, even if the data which was read is corrupt the raidz code will detect this and issue its own repair I/O to correct the damage in the mirror vdev. However, in the scenario where the DTL is wrong due to silent data corruption (say due to overwriting one child) and the scrub happens to read from a child with good data, then the other damaged mirror child will not be detected nor repaired. While this is possible for both raidz and draid vdevs, it's most pronounced when using draid. This is because by default the zed will sequentially rebuild a draid pool to a distributed spare, and the distributed spare half of the mirror is always preferred since it delivers better performance. This means the damaged half of the mirror will go undetected even after scrubbing. For system administrations this behavior is non-intuitive and in a worst case scenario could result in the only good copy of the data being unknowingly detached from the mirror. This change resolves the issue by reading all replacing/sparing mirror children when scrubbing. When the BP isn't available for verification, then compare the data buffers from each child. They must all be identical, if not there's silent damage and an error is returned to prompt the top-level vdev to issue a repair I/O to rewrite the data on all of the mirror children. Since we can't tell which child was wrong a checksum error is logged against the replacing or sparing mirror vdev. Reviewed-by: Mark Maybee Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13555 commit deb1213098e2dc10e6eee5e5c57bb40584e096a6 Author: Tino Reichardt Date: Tue Jun 21 23:32:09 2022 +0200 Fix memory allocation issue for BLAKE3 context The kmem_alloc(sizeof (*ctx), KM_NOSLEEP) call on FreeBSD can't be used in this code segment. Work around this by pre-allocating a percpu context array for later use. Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Closes #13568 commit b17663f571bfa1ef5e77d3c72f1610bacfc0c6ad Author: Matthew Thode Date: Tue Jun 21 12:37:20 2022 -0500 Remove install of zfs-load-module.service for dracut The zfs-load-module.service service is not currently provided by the OpenZFS repository so we cannot safely assume it exists. Reviewed-by: Brian Behlendorf Signed-off-by: Matthew Thode Closes #13574 commit d51f4ea5f9575847a271f2c99253b072a6ede07e Author: Alexander Motin Date: Fri Jun 17 18:38:51 2022 -0400 FreeBSD: Improve crypto_dispatch() handling Handle crypto_dispatch() return values same as crp->crp_etype errors. On FreeBSD 12 many drivers returned same errors both ways, and lack of proper handling for the first ended up in assertion panic later. It was changed in FreeBSD 13, but there is no reason to not be safe. While there, skip waiting for completion, including locking and wakeup() call, for sessions on synchronous crypto drivers, such as typical aesni and software. Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Closes #13563 commit f60973998593c37f55beea05e74528c1992b7849 Author: Andrew Date: Fri Jun 17 13:44:49 2022 -0500 expose snapshot count via stat(2) of .zfs/snapshot (#13559) Increase nlinks in stat results of ./zfs/snapshot based on snapshot count. This provides quick and efficient method for administrators to get snapshot counts without having to use libzfs or list the snapdir contents. Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Andrew Walker Closes #13559 commit 10891b37fa8f2cbf71ec529fc3808113d94d52ef Author: ixhamza <106930537+ixhamza@users.noreply.github.com> Date: Thu Jun 16 02:26:12 2022 +0500 libzfs: Prevent overridding of error code zfs_send_cb_impl fails to report error for some flags. Use second error variable for send_conclusion_record. Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Ameer Hamza Closes #13558 commit dd8671459f59484b1c20fc1b5e1c3acaa2a290c1 Author: Alexander Motin Date: Wed Jun 15 17:25:08 2022 -0400 Reduce ZIO io_lock contention on sorted scrub During sorted scrub multiple threads (one per vdev) are issuing many ZIOs same time, all using the same scn->scn_zio_root ZIO as parent. It causes huge lock contention on the single global lock on that ZIO. Improve it by introducing per-queue null ZIOs, children to that one, and using them instead as proxy. For 12 SSD pool storing 1.5TB of 4KB blocks on 80-core system this dramatically reduces lock contention and reduces scrub time from 21 minutes down to 12.5, while actual read stages (not scan) are about 3x faster, reaching 100K blocks per second per vdev. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Closes #13553 commit bc00d2c711c5bd930fe63deae3946ed3fb2463b4 Author: crass Date: Wed Jun 15 16:22:52 2022 -0500 Add support for ARCH=um for x86 sub-architectures When building modules (as well as the kernel) with ARCH=um, the options -Dsetjmp=kernel_setjmp and -Dlongjmp=kernel_longjmp are passed to the C preprocessor for C files. This causes the setjmp and longjmp used in module/lua/ldo.c to be kernel_setjmp and kernel_longjmp respectively in the object file. However, the setjmp and longjmp that is intended to be called is defined in an architecture dependent assembly file under the directory module/lua/setjmp. Since it is an assembly and not a C file, the preprocessor define is not given and the names do not change. This becomes an issue when modpost is trying to create the Module.symvers and sees no defined symbol for kernel_setjmp and kernel_longjmp. To fix this, if the macro CONFIG_UML is defined, then setjmp and longjmp macros are undefined. When building with ARCH=um for x86 sub-architectures, CONFIG_X86 is not defined. Instead, CONFIG_UML_X86 is defined. Despite this, the UML x86 sub-architecture can use the same object files as the x86 architectures because the x86 sub-architecture UML kernel is running with the same instruction set as CONFIG_X86. So the modules/Kbuild build file is updated to add the same object files that CONFIG_X86 would add when CONFIG_UML_X86 is defined. Reviewed-by: Brian Behlendorf Signed-off-by: Glenn Washburn Closes #13547 commit 988431966639d791ac269011d136e85f3602df75 Author: Damian Szuberski Date: Thu Jun 16 07:20:28 2022 +1000 Fix clang 13 compilation errors ``` os/linux/zfs/zvol_os.c:1111:3: error: ignoring return value of function declared with 'warn_unused_result' attribute [-Werror,-Wunused-result] add_disk(zv->zv_zso->zvo_disk); ^~~~~~~~ ~~~~~~~~~~~~~~~~~~~~ zpl_xattr.c:1579:1: warning: no previous prototype for function 'zpl_posix_acl_release_impl' [-Wmissing-prototypes] ``` Reviewed-by: Brian Behlendorf Signed-off-by: szubersk Closes #13551 commit 4ff7a8fa2f1e66c4a10ecd4fd74c9381db24fb02 Author: Allan Jude Date: Tue Jun 14 14:27:53 2022 -0400 Replace ZPROP_INVAL with ZPROP_USERPROP where it means a user property Reviewed-by: Brian Behlendorf Signed-off-by: Allan Jude Sponsored-by: Klara Inc. Closes #12676 commit 9e605cf155040f76f3971f056a4410ac8f500fe3 Author: Ryan Moeller Date: Mon Jun 13 20:30:34 2022 +0000 spl: Use a clearer name for the user namespace fd This fd has nothing to do with cleanup, that's just the name of the field in zfs_cmd_t that was used to pass it to the kernel. Call it what it is, an fd for a user namespace. Reviewed-by: Brian Behlendorf Reviewed-by: Allan Jude Signed-off-by: Ryan Moeller Closes #13554 commit def1a401f44abce71196949feaecbd66ebcddf0b Author: Ryan Moeller Date: Mon Jun 13 20:24:23 2022 +0000 libzfs: zfs_userns: Don't leak the namespace fd zfs_userns opens a file descriptor for the kernel to look up a namespace, but does not close it. Close the fd when we're done with it. Reviewed-by: Brian Behlendorf Reviewed-by: Allan Jude Signed-off-by: Ryan Moeller Closes #13554 commit 482505fd4219a4c3a91f2857199436df27ad687e Author: Julian Brunner Date: Sat Jun 11 03:22:14 2022 +0200 Add weekly and monthly systemd timers for trimming On machines using systemd, trim timers can be enabled on a per-pool basis. Weekly and monthly timer units are provided. Timers can be enabled as follows: systemctl enable zfs-trim-weekly@rpool.timer --now systemctl enable zfs-trim-monthly@datapool.timer --now Each timer will pull in zfs-trim@${poolname}.service, which is not schedule-specific. The manpage zpool-trim has been updated accordingly. Reviewed-by: Brian Behlendorf Signed-off-by: Julian Brunner Closes #13544 commit 87b46d63b283d9e2c3b10945f37233d1dedef02a Author: Alexander Motin Date: Fri Jun 10 13:01:46 2022 -0400 Improve sorted scan memory accounting Since we use two B-trees q_exts_by_size and q_exts_by_addr, we should count 2x sizeof (range_seg_gap_t) per node. And since average B-tree memory efficiency is about 75%, we should increase it to 3x. Previous code under-counted up to 30% of the memory usage. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Closes #13537 commit 4ed5e25074ffec266df38556d9b3a928c5e0dee9 Author: Will Andrews Date: Sun Feb 21 10:19:43 2021 -0600 Add Linux namespace delegation support This allows ZFS datasets to be delegated to a user/mount namespace Within that namespace, only the delegated datasets are visible Works very similarly to Zones/Jailes on other ZFS OSes As a user: ``` $ unshare -Um $ zfs list no datasets available $ echo $$ 1234 ``` As root: ``` # zfs list NAME ZONED MOUNTPOINT containers off /containers containers/host off /containers/host containers/host/child off /containers/host/child containers/host/child/gchild off /containers/host/child/gchild containers/unpriv on /unpriv containers/unpriv/child on /unpriv/child containers/unpriv/child/gchild on /unpriv/child/gchild # zfs zone /proc/1234/ns/user containers/unpriv ``` Back to the user namespace: ``` $ zfs list NAME USED AVAIL REFER MOUNTPOINT containers 129M 47.8G 24K /containers containers/unpriv 128M 47.8G 24K /unpriv containers/unpriv/child 128M 47.8G 128M /unpriv/child ``` Reviewed-by: Brian Behlendorf Signed-off-by: Will Andrews Signed-off-by: Allan Jude Signed-off-by: Mateusz Piotrowski Co-authored-by: Allan Jude Co-authored-by: Mateusz Piotrowski Sponsored-by: Buddy Closes #12263 commit a1aa8f14c864b6851649f9c3e74e9f12e6518edd Author: Allan Jude Date: Fri Jul 2 19:16:58 2021 +0000 Revert parts of 938cfeb0f27303721081223816d4f251ffeb1767 When read and writing the UID/GID, we always want the value relative to the root user namespace, the kernel will take care of remapping this to the user namespace for us. Calling from_kuid(user_ns, uid) with a unmapped uid will return -1 as that uid is outside of the scope of that namespace, and will result in the files inside the namespace all being owned by 'nobody' and not being allowed to call chmod or chown on them. Reviewed-by: Brian Behlendorf Signed-off-by: Allan Jude Closes #12263 commit fc5200aa9b345972fc1b99869b03a373090b84c7 Author: Alexander Motin Date: Thu Jun 9 18:27:36 2022 -0400 AVL: Remove obsolete branching optimizations Modern Clang and GCC can successfully implement simple conditions without branching with math and flag operations. Use of arrays for translation no longer helps as much as it was 14+ years ago. Disassemble of the code generated by Clang 13.0.0 on FreeBSD 13.1, Clang 14.0.4 on FreeBSD 14 and GCC 10.2.1 on Debian 11 with this change still shows no branching instructions. Profiling of CPU-bound scan stage of sorted scrub shows reproducible reduction of time spent inside avl_find() from 6.52% to 4.58%. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Closes #13540 commit cdeb98a116a3b31558d6d7cb861b0cbd992a27cc Author: Ryan Moeller Date: Wed Jun 8 16:32:38 2022 +0000 libzfs: Rename msg bufs to errbuf for consistency `libzfs_pool.c` uses the name `msg` where everywhere else in libzfs uses `errbuf` for the error message buffer. Use the name consistent with the rest of libzfs and use ERRBUFLEN instead of 1024. Reviewed-by: Brian Behlendorf Signed-off-by: Ryan Moeller Closes #13539 commit d68a2bfa994d3d064924e8c303a9a7e8e6e02682 Author: Ryan Moeller Date: Wed Jun 8 13:08:10 2022 +0000 libzfs: Define the defecto standard errbuf size Every errbuf array in libzfs is 1024 chars. Define ERRBUFLEN in a shared header, and use it. Reviewed-by: Brian Behlendorf Signed-off-by: Ryan Moeller Closes #13539 commit 6f73d02168ea4d4c27e95d3f23df7221c7321e07 Author: Tony Hutter Date: Thu Jun 9 07:10:38 2022 -0700 zvol: Support blk-mq for better performance Add support for the kernel's block multiqueue (blk-mq) interface in the zvol block driver. blk-mq creates multiple request queues on different CPUs rather than having a single request queue. This can improve zvol performance with multithreaded reads/writes. This implementation uses the blk-mq interfaces on 4.13 or newer kernels. Building against older kernels will fall back to the older BIO interfaces. Note that you must set the `zvol_use_blk_mq` module param to enable the blk-mq API. It is disabled by default. In addition, this commit lets the zvol blk-mq layer process whole `struct request` IOs at a time, rather than breaking them down into their individual BIOs. This reduces dbuf lock contention and overhead versus the legacy zvol submit_bio() codepath. sequential dd to one zvol, 8k volblocksize, no O_DIRECT: legacy submit_bio() 292MB/s write 453MB/s read this commit 453MB/s write 885MB/s read It also introduces a new `zvol_blk_mq_chunks_per_thread` module parameter. This parameter represents how many volblocksize'd chunks to process per each zvol thread. It can be used to tune your zvols for better read vs write performance (higher values favor write, lower favor read). Reviewed-by: Brian Behlendorf Reviewed-by: Ahelenia Ziemiańska Reviewed-by: Tony Nguyen Signed-off-by: Tony Hutter Closes #13148 Issue #12483 commit 985c33b132f6c23a69bd808e008ae0f46131a31e Author: Tino Reichardt Date: Thu Jun 9 00:55:57 2022 +0200 Introduce BLAKE3 checksums as an OpenZFS feature This commit adds BLAKE3 checksums to OpenZFS, it has similar performance to Edon-R, but without the caveats around the latter. Homepage of BLAKE3: https://github.com/BLAKE3-team/BLAKE3 Wikipedia: https://en.wikipedia.org/wiki/BLAKE_(hash_function)#BLAKE3 Short description of Wikipedia: BLAKE3 is a cryptographic hash function based on Bao and BLAKE2, created by Jack O'Connor, Jean-Philippe Aumasson, Samuel Neves, and Zooko Wilcox-O'Hearn. It was announced on January 9, 2020, at Real World Crypto. BLAKE3 is a single algorithm with many desirable features (parallelism, XOF, KDF, PRF and MAC), in contrast to BLAKE and BLAKE2, which are algorithm families with multiple variants. BLAKE3 has a binary tree structure, so it supports a practically unlimited degree of parallelism (both SIMD and multithreading) given enough input. The official Rust and C implementations are dual-licensed as public domain (CC0) and the Apache License. Along with adding the BLAKE3 hash into the OpenZFS infrastructure a new benchmarking file called chksum_bench was introduced. When read it reports the speed of the available checksum functions. On Linux: cat /proc/spl/kstat/zfs/chksum_bench On FreeBSD: sysctl kstat.zfs.misc.chksum_bench This is an example output of an i3-1005G1 test system with Debian 11: implementation 1k 4k 16k 64k 256k 1m 4m edonr-generic 1196 1602 1761 1749 1762 1759 1751 skein-generic 546 591 608 615 619 612 616 sha256-generic 240 300 316 314 304 285 276 sha512-generic 353 441 467 476 472 467 426 blake3-generic 308 313 313 313 312 313 312 blake3-sse2 402 1289 1423 1446 1432 1458 1413 blake3-sse41 427 1470 1625 1704 1679 1607 1629 blake3-avx2 428 1920 3095 3343 3356 3318 3204 blake3-avx512 473 2687 4905 5836 5844 5643 5374 Output on Debian 5.10.0-10-amd64 system: (Ryzen 7 5800X) implementation 1k 4k 16k 64k 256k 1m 4m edonr-generic 1840 2458 2665 2719 2711 2723 2693 skein-generic 870 966 996 992 1003 1005 1009 sha256-generic 415 442 453 455 457 457 457 sha512-generic 608 690 711 718 719 720 721 blake3-generic 301 313 311 309 309 310 310 blake3-sse2 343 1865 2124 2188 2180 2181 2186 blake3-sse41 364 2091 2396 2509 2463 2482 2488 blake3-avx2 365 2590 4399 4971 4915 4802 4764 Output on Debian 5.10.0-9-powerpc64le system: (POWER 9) implementation 1k 4k 16k 64k 256k 1m 4m edonr-generic 1213 1703 1889 1918 1957 1902 1907 skein-generic 434 492 520 522 511 525 525 sha256-generic 167 183 187 188 188 187 188 sha512-generic 186 216 222 221 225 224 224 blake3-generic 153 152 154 153 151 153 153 blake3-sse2 391 1170 1366 1406 1428 1426 1414 blake3-sse41 352 1049 1212 1174 1262 1258 1259 Output on Debian 5.10.0-11-arm64 system: (Pi400) implementation 1k 4k 16k 64k 256k 1m 4m edonr-generic 487 603 629 639 643 641 641 skein-generic 271 299 303 308 309 309 307 sha256-generic 117 127 128 130 130 129 130 sha512-generic 145 165 170 172 173 174 175 blake3-generic 81 29 71 89 89 89 89 blake3-sse2 112 323 368 379 380 371 374 blake3-sse41 101 315 357 368 369 364 360 Structurally, the new code is mainly split into these parts: - 1x cross platform generic c variant: blake3_generic.c - 4x assembly for X86-64 (SSE2, SSE4.1, AVX2, AVX512) - 2x assembly for ARMv8 (NEON converted from SSE2) - 2x assembly for PPC64-LE (POWER8 converted from SSE2) - one file for switching between the implementations Note the PPC64 assembly requires the VSX instruction set and the kfpu_begin() / kfpu_end() calls on PowerPC were updated accordingly. Reviewed-by: Felix Dörre Reviewed-by: Ahelenia Ziemiańska Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Co-authored-by: Rich Ercolani Closes #10058 Closes #12918 commit b9d98453f9387c413f91d1d9cdb0cba8e04dbd95 Author: Brian Behlendorf Date: Tue May 31 16:42:49 2022 -0700 autoconf: AC_MSG_CHECKING consistency Make the wording more consistent for the kernel AC_MSG_CHECKING output (e.g. "checking whether ...".). Additionally, group some of the VFS interface checks with the others. No functional change. Reviewed-by: Tony Hutter Reviewed-by: Attila Fülöp Signed-off-by: Brian Behlendorf Closes #13529 commit 4c6526208db0d3d5abf44664e74d1e28156a3db7 Author: Brian Behlendorf Date: Tue May 31 16:30:59 2022 -0700 Linux 5.19 compat: asm/fpu/internal.h As of the Linux 5.19 kernel the asm/fpu/internal.h header was entirely removed. It has been effectively empty since the 5.16 kernel and provides no required functionality. Reviewed-by: Tony Hutter Reviewed-by: Attila Fülöp Signed-off-by: Brian Behlendorf Closes #13529 commit 42cf2ad0e4e2adfa232f42e4254693467a4cc08c Author: Alexander Motin Date: Wed Jun 1 12:54:35 2022 -0400 Remove wrong assertion in log spacemap It is typical, but not generally true that if log summary has more blocks it must also have unflushed metaslabs. Normally with metaslabs flushed in order it works, but there are known exceptions, such as device removal or metaslab being loaded during its flush attempt. Before 600a02b8844 if spa_flush_metaslabs() hit loading metaslab it usually stopped (unless memlimit is also exceeded), but now it may flush more metaslabs, just skipping that particular one. This increased chances of assertion to fire when the skipped metaslab is flushed on next iteration if all other metaslabs in that summary entry are already flushed out of order. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Closes #13486 Closes #13513 commit bc8192cd5b14cf4182bd2b19a6a8ed4f0bbed12b Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Tue May 31 18:41:33 2022 -0400 Corrected parameters for zstd early abort That'll teach me to try and recall them from the definition. Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #13519 commit 2310dba9ebf6259515b63fda3202199831669271 Author: Allan Jude Date: Tue May 31 18:37:46 2022 -0400 Fix typo in zil_commit() comment block Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: Allan Jude Closes #13518 commit a70e613070a8ca96f8214ba1ff61549cbbad0a2f Author: Brian Behlendorf Date: Tue May 31 14:38:00 2022 -0700 Linux 5.18 compat: META Update the META file to reflect compatibility with the 5.18 kernel. Reviewed-by: George Melikov Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13527 commit 91350681b8c8b3f0a9b04e6ab3b8931406e87355 Author: Brian Behlendorf Date: Fri May 27 15:56:05 2022 -0700 Linux 5.19 compat: zap_flags_t conflict As of the Linux 5.19 kernel an identically named zap_flags_t typedef is declared in the include/linux/mm_types.h linux header. Sadly, the inclusion of this header cannot be easily avoided. To resolve the conflict a #define is used to remap the name in the OpenZFS sources when building against the Linux kernel. Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13515 commit d41e864181e4544eca08332b31f85318a3b0e3b3 Author: Brian Behlendorf Date: Fri May 27 21:31:03 2022 +0000 Linux 5.19 compat: bdev_start_io_acct() / bdev_end_io_acct() As of the Linux 5.19 kernel the disk_*_io_acct() helper functions have been replaced by the bdev_*_io_acct() functions. Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13515 commit c2c2e7bb8b7c269904777b61f4b0a678f1ffb9a3 Author: Brian Behlendorf Date: Fri May 27 20:44:43 2022 +0000 Linux 5.19 compat: aops->read_folio() As of the Linux 5.19 kernel the readpage() address space operation has been replaced by read_folio(). Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13515 commit a12a5cb5b821f24f26d388094cdac79deb0e879f Author: Brian Behlendorf Date: Fri May 27 19:40:22 2022 +0000 Linux 5.19 compat: blkdev_issue_secure_erase() Linux 5.19 commit torvalds/linux@44abff2c0 splits the secure erase functionality from the blkdev_issue_discard() function. The blkdev_issue_secure_erase() must now be issued to issue a secure erase. Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13515 commit e2c31f2bc7d190fbd8fc5c13bac23daffc5d7b56 Author: Brian Behlendorf Date: Fri May 27 18:20:04 2022 +0000 Linux 5.19 compat: bdev_max_secure_erase_sectors() Linux 5.19 commit torvalds/linux@44abff2c0 removed the blk_queue_secure_erase() helper function. The preferred interface is to now use the bdev_max_secure_erase_sectors() function to check for discard support. Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13515 commit 5e4aedaca7cee981ed21ac856fd27b4682bb7888 Author: Brian Behlendorf Date: Fri May 27 17:51:55 2022 +0000 Linux 5.19 compat: bdev_max_discard_sectors() Linux 5.19 commit torvalds/linux@70200574cc removed the blk_queue_discard() helper function. The preferred interface is to now use the bdev_max_discard_sectors() function to check for discard support. Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13515 commit 5f264996f4dc2d5279afe96698688a20c281c473 Author: Brian Behlendorf Date: Fri May 27 20:28:51 2022 +0000 Linux 5.18 compat: bio_alloc() As for the Linux 5.18 kernel bio_alloc() expects a block_device struct as an argument. This removes the need for the bio_set_dev() compatibility code for 5.18 and newer kernels. Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13515 commit 152d6fda54e61042a70059c95c44b364aea0bbd8 Author: Kevin Jin <33590050+jxdking@users.noreply.github.com> Date: Thu May 26 12:36:14 2022 -0400 Fix inflated quiesce time caused by lwb_tx during zil_commit() In current zil_commit() process, transaction lwb_tx is assigned in zil_lwb_write_issue(), and is committed in zil_lwb_flush_vdevs_done(). Thus, during lwb write out process, the txg is held in open or quiesing state, until zil_lwb_flush_vdevs_done() is called. If the zil's zio latency is high, it will cause txg_sync_thread() to starve. The goal here is to defer waiting for zil_lwb_flush_vdevs_done to the 'syncing' txg state. That is, in zil_sync(). In this patch, it achieves the goal without holding transaction. A new function zil_lwb_flush_wait_all() is introduced. It waits for the completion of all the zil_lwb_flush_vdevs_done() by given txg. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Reviewed-by: Prakash Surya Signed-off-by: jxdking Closes #12321 commit d98a67a53a180bd88ec8d9aeea75d92e1c9968b5 Author: Brian Behlendorf Date: Thu May 26 09:24:50 2022 -0700 Replace EXTRA_DIST with dist_noinst_DATA The EXTRA_DIST variable is ignored when used in the FALSE conditional of a Makefile.am. This results in the `make dist` target omitting these files from the generated tarball unless CONFIG_USER is defined. This issue can be avoided by switching to use the dist_noinst_DATA variable which is handled as expected by autoconf. This change also adds support for --with-config=dist as an alias for --with-config=srpm and updates the GitHub workflows to use it. Reviewed-by: Ahelenia Ziemiańska Signed-off-by: Brian Behlendorf Closes #13459 Closes #13505 commit b62829295e9529d1c321816a1027fac5afc7d6f5 Author: Ryan Moeller Date: Wed May 25 20:26:59 2022 -0400 Silence unused-but-set-variable warning This was breaking the kmod port build on FreeBSD with Clang 13. Use the same trick as we do for ASSERT() to make DNODE_VERIFY() use its parameter at compile time without actually using it at run time in non-debug builds. Reviewed-by: Brian Behlendorf Signed-off-by: Ryan Moeller Closes #13507 commit 6aa8c21a2ad29ddd4564cdfd4c99048c891b717a Author: Alexander Motin Date: Wed May 25 13:12:52 2022 -0400 More speculative prefetcher improvements - Make prefetch distance adaptive: up to 4MB prefetch doubles for every, hit same as before, but after that it grows by 1/8 every time the prefetch read does not complete in time to satisfy the demand. My tests show that 4MB is sufficient for wide NVMe pool to saturate single reader thread at 2.5GB/s, while new 64MB maximum allows the same thread to reach 1.5GB/s on wide HDD pool. Further distance increase may increase speed even more, but less dramatic and with higher latency. - Allow early reuse of inactive prefetch streams: streams that never saw hits can be reused immediately if there is a demand, while others can be reused after 1s of inactivity, starting with the oldest. After 2s of inactivity streams are deleted to free resources same as before. This allows by several times increase strided read performance on HDD pool in presence of simultaneous random reads, previously filling the zfetch_max_streams limit for seconds and so blocking most of prefetch. - Always issue intermediate indirect block reads with SYNC priority. Each of those reads if delayed for longer may delay up to 1024 other block prefetches, that may be not good for wide pools. Reviewed-by: Allan Jude Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Closes #13452 commit 1d89b989c15acdc9d70878253d68162c3c5c5836 Author: наб Date: Wed May 25 18:29:47 2022 +0200 automake: don't install /e/d/zfs or /e/z/zfs-functions +x _SCRIPTS means it's made +x when installing; _DATA is made -x. Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13496 Closes #13503 commit 7829b465a7f736e9458257fcb9dcefd8eb882ee0 Author: Paul Dagnelie Date: Wed May 25 09:25:13 2022 -0700 Cancel in-progress rebuilds when we finish removal This issue was discovered by zloop runs. When a mirror or other redundant top-level vdev has a disk failure, and the disk is replaced, the rebuild process occurs. A removal can happen while this is in progress. If the removal completes before the rebuild does, the removal process will try to free the vdev that is still in use. Reviewed-by: Brian Behlendorf Signed-off-by: Paul Dagnelie Closes #13498 commit b37093a188d94279e5e2faaf09e6ff754873b0a2 Author: Umer Saleem Date: Wed May 25 21:22:11 2022 +0500 rpm: Keep debug symbols if configured with '--enable-debuginfo' Do not strip debug information from packages if '--enable-debuginfo' is configured. Reviewed-by: Brian Behlendorf Signed-off-by: Umer Saleem Closes #13500 commit 61ef68727b0b3c53e27d6e503947f6c5efd1318c Author: Brian Behlendorf Date: Wed May 25 09:20:17 2022 -0700 Standardize RHEL version check in packages This is a follow up to 3c356622994 which standardizes how the RHEL version check is done. This simpler "0%{?rhel}" check is used elsewhere in the packages so we do the same here. Reviewed-by: Neal Gompa Reviewed-by: Rich Ercolani Signed-off-by: Brian Behlendorf Closes #13501 commit 3bbc26097e53c472084d978dd37343b07ed17e3c Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Wed May 25 12:18:49 2022 -0400 Unbreak zstd build on sparc64 It turns out that wrapping the atomic macro in () breaks build on Linux/SPARC64. Oops. Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #13506 commit 82aa4f6f858549ba51d8afa207b179e4a3403d95 Author: Brian Behlendorf Date: Wed May 25 09:13:51 2022 -0700 Switch sed -E to -r for better portability GNU sed 4.1.2 does not support the -E flag and this version is used by some cross-compiling tool chains. Switch -E to -r which is understood. Reviewed-by: Ahelenia Ziemiańska Signed-off-by: Brian Behlendorf Closes #13502 commit 4dc1c8a0b84678a98e1c541493988b348ea0e644 Author: Neal Gompa (ニール・ゴンパ) Date: Tue May 24 17:07:01 2022 -0400 rpm: Use the correct version-release information in dependencies This tightly links the subpackages together and ensures that everything is upgraded together. Reviewed-by: Tony Hutter Reviewed-by: Brian Behlendorf Signed-off-by: Neal Gompa Closes #13489 commit 84d0a03f3e38b1c4c9361a4b4ec613a2f46248b3 Author: Alexander Motin Date: Tue May 24 12:46:35 2022 -0400 Refactor Log Size Limit Original Log Size Limit implementation blocked all writes in case of limit reached until the TXG is committed and the log is freed. It caused huge delays and following speed spikes in application writes. This implementation instead smoothly throttles writes, using exactly the same mechanism as used for dirty data. Reviewed-by: Brian Behlendorf Reviewed-by: jxdking Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Issue #12284 Closes #13476 commit f375b23c026aec00cc9527470084191b5071d9b2 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Tue May 24 12:43:22 2022 -0400 Tiered early abort, zstd edition It turns out that "do LZ4 and zstd-1 both fail" is a great heuristic for "don't even bother trying higher zstd tiers". By way of illustration: $ cat /incompress | mbuffer | zfs recv -o compression=zstd-12 evenfaster/lowcomp_1M_zstd12_normal summary: 39.8 GiByte in 3min 40.2sec - average of 185 MiB/s $ echo 3 | sudo tee /sys/module/zzstd/parameters/zstd_lz4_pass 3 $ cat /incompress | mbuffer -m 4G | zfs recv -o compression=zstd-12 evenfaster/lowcomp_1M_zstd12_patched summary: 39.8 GiByte in 48.6sec - average of 839 MiB/s $ sudo zfs list -p -o name,used,lused,ratio evenfaster/lowcomp_1M_zstd12_normal evenfaster/lowcomp_1M_zstd12_patched NAME USED LUSED RATIO evenfaster/lowcomp_1M_zstd12_normal 39549931520 42721221632 1.08 evenfaster/lowcomp_1M_zstd12_patched 39626399744 42721217536 1.07 $ python3 -c "print(39626399744 - 39549931520)" 76468224 $ I'll take 76 MB out of 42 GB for > 4x speedup. Reviewed-by: Allan Jude Reviewed-by: Brian Behlendorf Reviewed-by: George Melikov Reviewed-by: Kjeld Schouten Reviewed-by: Ahelenia Ziemiańska Signed-off-by: Rich Ercolani Closes #13244 commit 2e05765006913b0c381fdbbbf0370b35c0e61be4 Author: Ryan Moeller Date: Tue May 24 12:40:20 2022 -0400 FreeBSD: libspl: Add locking around statfs globals Makes getmntent and getmntany thread-safe for external consumers of libzfs zpool_disable_datasets, zfs_iter_mounted, libzfs_mnttab_update, libzfs_mnttab_find. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Ryan Moeller Closes #13484 commit 3c356622994f1837f42a0d4bcd6558a3b3749521 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Tue May 24 12:39:32 2022 -0400 Modified ncompress requirement in RPM to exclude RHEL9 The bug this was working around is no longer present. Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #13480 Closes #13490 commit cf70c0f8ae01c24699767b5ecfbe1882c009d53a Author: Brian Behlendorf Date: Tue May 24 09:36:07 2022 -0700 zed: Take no action on scrub/resilver checksum errors When scrubbing/resilvering a pool it can be counter productive to cancel the scan and kick of a replace operation to a hot spare when encountering checksum errors. In this case, the best course of action is to allow the scrub/resilver to complete as quickly as possible and to keep the vdevs fully online if possible. Realistically, this is less of an issue for a RAIDZ since a traditional resilver must be used and checksums will be verified. However, this is not the case for a mirror or dRAID pool which is sequentially resilvered and checksum verification is deferred until after the replace operation completes. Regardless, we apply this policy to all pool types since it's a good idea for all vdevs. Degrading additional vdevs has the potential to make a bad situation worse. Note the checksum errors will still be reported as both an event and by `zpool status`. This change only prevents the ZED from proactively taking any action. Reviewed-by: Tony Hutter Reviewed-by: Tony Nguyen Signed-off-by: Brian Behlendorf Closes #13499 commit 2cd0f98f4aae7110a48cb0623e1e3d66b9f58785 Author: Brian Behlendorf Date: Fri May 20 10:36:14 2022 -0700 Verify BPs in spa_load_verify_cb() and dsl_scan_visitbp() We want `zpool import` to be highly robust and never panic, even when encountering corrupt metadata. This is already handled in the arc_read() code path, which covers most cases, but spa_load_verify_cb() relies on zio_read() and is responsible for verifying the block pointer. During import it is also possible to encounter blocks pointers which contain ZIO_COMPRESS_INHERIT and ZIO_CHECKSUM_INHERIT values. Relax the verification function slightly to allow this. Futhermore, extend dsl_scan_recurse() to verify the block pointer contents of level zero blocks which are not of type DMU_OT_DNODE or DMU_OT_OBJSET. This is handled by arc_read() in the other cases. Reviewed-by: Paul Dagnelie Signed-off-by: Brian Behlendorf Closes #13124 Closes #13360 commit 03df6bad94504b80929b332a3e5fbfbc8bdae04b Author: Mark Johnston Date: Fri May 20 13:32:49 2022 -0400 zdb: Fix handling of nul termination in symlink targets The SA attribute containing the symlink target does not include a nul terminator, so when printing the target zdb would sometimes include garbage at the end of the string. Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: Mark Johnston Closes #13482 commit e02d84d3a5158a942c3f09d11c973e5aabb6fa40 Author: наб Date: Thu May 19 00:56:38 2022 +0200 linux: libshare: smb: don't swallow net(1) errors Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13191 Closes #13470 commit 2b4f2fc93c4298415c5bc70554514c3e8892e5d5 Author: наб Date: Fri Apr 15 00:00:02 2022 +0200 libzfs: return (allocated) strings instead of filling buffers This also expands the zfs version output from 127 characters to However Many Are Actually Set Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13330 commit 38f4d99f769efbb472ab3c35558700163e4f023c Author: наб Date: Thu Apr 14 23:30:41 2022 +0200 linux: libzfs: simplify module-loaded check The short-path is now one access() call, we always modprobe zfs (ZFS_MODULE_LOADING which doesn't use the libzfs boolean parsing is gone), and we use a simple inotify IN_CREATE loop with a timerfd timeout rather than 10ms kernel-style polling There's one substantial difference: ZFS_MODULE_TIMEOUT=-1 now means "never give up", rather than "wait 10 minutes" Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13330 commit 89e81bc6adf58ec442ddaa23e629bf888bc55488 Author: наб Date: Tue May 10 23:28:02 2022 +0200 Remove final K&R definitions Clang trunk now warns -Wstrict-prototypes on this, and they're removed in C2x Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13447 commit 6b575417e2bd500906ede0c82c0d29780ee26e9f Author: наб Date: Tue May 10 23:25:43 2022 +0200 libspl/include: remove unused/empty headers Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13447 commit 2f713390d18ff3ae8ad4fdb8e97f9fdfbf867bc7 Author: наб Date: Tue May 10 22:11:30 2022 +0200 kmodtool: cleanup Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13447 commit 7062a956f79b264084d71e542ce74bc5144f504d Author: наб Date: Tue May 10 22:10:57 2022 +0200 rpm: don't spec obsolete_name/version anymore Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13447 commit 7506f5af922895afe52377e2ea1e9813e3be8111 Author: наб Date: Tue May 10 22:05:20 2022 +0200 Add make regen-tests to regenerate the test bundle Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13447 commit 08b32c6fa9dd68d024447979e7c9b711b9e60c56 Author: heeplr <32984777+heeplr@users.noreply.github.com> Date: Wed May 18 19:27:53 2022 +0200 zed: support subject as header in zed_notify_email() Some minimal MUAs don't support passing the subjects as cmdline option. This commit checks if "@SUBJECT@" is missing in ZED_EMAIL_OPTS and then prepends a subject header to the notification message. Also set a default for ${subject}. Reviewed-by: Ahelenia Ziemia<84>ska Reviewed-by: Tony Hutter Signed-off-by: Daniel Hiepler Closes #13440 commit 00ac77464ef15c56b2cffb049518b78545552a9d Author: Andrew Date: Wed May 18 12:25:33 2022 -0500 Expose zpool guids through kstats There are times when end-users may wish to have a fast and convenient method to get zpool guid without having to use libzfs. This commit exposes the zpool guid via kstats in similar manner to the zpool state. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Andrew Walker Closes #13466 commit c0cf6ed6792e545fd614c2a88cb53756db7e03f8 Author: Coleman Kane Date: Tue May 17 16:07:39 2022 -0400 Fix compiler warnings about zero-length arrays in inline bitops The compiler appears to be expanding the unused NULL pointer into a zero-length array via the inline bitops code. When -Werror=array-bounds is used, this causes a build failure. Recommended solution is allocate temporary structures, fill with zeros (to avoid uninitialized data use warnings), and pass the pointer to those to the inline calls. Reviewed-by: Brian Behlendorf Signed-off-by: Coleman Kane Closes #13463 Closes #13465 commit 276b08cb0077d7f6c4fe6b735e72740e8e72d572 Author: наб Date: Tue May 3 20:13:22 2022 +0200 linux: libzutil: zfs_strip_path: only strip known prefixes This mirrors FreeBSD: # zpool create -o cachefile= testpsko media/testpsko # zpool create -o cachefile= testpsko2 $PWD/testpsko2 $ ./zpool list -v NAME SIZE ALLOC FREE filling 25.5T 6.85T 18.6T mirror-0 3.64T 500G 3.15T ata-HGST_HUS726T4TALE6L4_V6K2L4RR - - - ata-HGST_HUS726T4TALE6L4_V6K2MHYR - - - raidz1-1 21.8T 6.36T 15.5T ata-HGST_HUS728T8TALE6L4_VDKT237K - - - ata-HGST_HUS728T8TALE6L4_VDGY075D - - - ata-HGST_HUS728T8TALE6L4_VDKVRRJK - - - cache - - - nvme0n1p4 63.0G 12.8G 50.2G tarta-boot 240M 50.0M 190M mirror-0 240M 50.0M 190M tarta-boot - - - tarta-boot-nvme - - - tarta-zoot 55.5G 6.96G 48.5G mirror-0 55.5G 6.96G 48.5G tarta-zoot - - - tarta-zoot-nvme - - - testpsko 39.5G 744K 39.5G media/testpsko1 39.5G 744K 39.5G testpsko2 39.5G 130K 39.5G /home/nabijaczleweli/store/code/zfs/testpsko2 39.5G 130K 39.5G Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13413 Closes #9771 commit 5ac80603bdafef9317b3a815137181667cef8947 Author: наб Date: Tue May 3 19:56:12 2022 +0200 libzfs: constify zfs_strip_partition(), zfs_strip_path() Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13413 commit ca3b105cd99198e59fa6f0b74943319ad7eedcde Author: наб Date: Tue May 3 19:38:15 2022 +0200 libzfs: pool: zpool_vdev_name: use libzfs_envvar_is_set Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13413 commit e9072c76f832dc649d27cdb7cd739b3e73fbed37 Author: наб Date: Tue May 3 19:33:31 2022 +0200 zpool: max_width: monomorphise subtype iteration Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13413 commit de8216451840add3be18df0d9fa423a30e35e8a6 Author: наб Date: Sat May 7 19:54:29 2022 +0200 linux: spl: generic: ddi_strto*: match solaris ddi_strto*(9) Recognise initial whitespace, + in both cases, and - also in unsigneds Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13434 commit 354a1bfb8e77ebcba5482139931144e88ec74bbc Author: наб Date: Sat May 7 19:23:28 2022 +0200 linux: spl: generic: ddi_strtou##type: elide unused flag Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13434 commit c25b281378e504d69ad0f2212f233f0c6b016b33 Author: наб Date: Sat May 7 19:18:41 2022 +0200 Remove hw_serial, ddi_strtoul() Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13434 commit 0c11a2738d3778a359de708911643a6b2cc15fa6 Author: Mateusz Piotrowski Date: Thu May 12 18:34:24 2022 +0200 Fix typos in zfs-bookmark examples Reviewed-by: George Melikov Reviewed-by: Brian Behlendorf Signed-off-by: Mateusz Piotrowski <0mp@FreeBSD.org> Closes #13456 commit 53600767ee8c9e3c25fff3fbb61de89a2952a4ef Author: наб Date: Sun Apr 17 15:03:03 2022 +0200 linux: libshare/nfs: don't do anything unless exportfs is available Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 Closes #13324 commit 086af23e683c62d24055b46a6b93de78825a5219 Author: наб Date: Sun Apr 17 15:00:15 2022 +0200 linux: libshare/smb: cache smb_available Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 commit 1ec9218faa2f0d11b66ec1eb9d202692b1eec2cd Author: наб Date: Fri Mar 18 18:40:13 2022 +0100 libzfs: zfs_unshare: minor cleanup Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 commit 88d5580e511540b996dfb802a7ee4912980fe1e7 Author: наб Date: Sun Mar 6 01:39:54 2022 +0100 tests: add zfs_unshare_008_pos checking whitespace escaping Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 commit 9b06aa634a80d6820ffb1453ccb3e4572be303d4 Author: наб Date: Mon Feb 28 20:42:22 2022 +0100 libshare/nfs: escape mount points when needed Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 Closes #13153 commit a31fcd4bad22d20a22cdd8e9f7e388a99508a8d7 Author: наб Date: Mon Feb 28 17:44:06 2022 +0100 linux: libshare/nfs: bsearch() over valid keys Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 commit 5b14feec0646ae3a09e1713388bb7b3ef745a421 Author: наб Date: Mon Feb 28 16:55:16 2022 +0100 libzfs: mount: zfs_unshare: don't reallocate mountpoint Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 commit b4d9a82f6244df1b500a2988cf60849866fb4265 Author: наб Date: Mon Feb 28 16:52:07 2022 +0100 Replace libzfs sharing _nfs() and _smb() APIs with protocol lists With the additional benefit of removing all the _all() functions and treating a NULL list as "all" ‒ the remaining all function is for all /datasets/, which is consistent with the rest of the API Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 commit 471e9a108e51e92ef645223efcde59c8ab1b9db7 Author: наб Date: Mon Feb 28 15:46:25 2022 +0100 Publish libshare protocols, use enum-based API Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 commit 21d976a62119d3442a0b83693b96f74ac6d0ff5b Author: наб Date: Mon Feb 28 15:00:49 2022 +0100 libshare: delineate obsolete errors Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 commit 63ce6dd9886a6bf7b070fb353780b1b25f693c90 Author: наб Date: Mon Feb 28 14:50:28 2022 +0100 libshare: use AVL tree with static data, pass all data in arguments This makes it so we don't leak a consistent 64 bytes anymore, makes the searches simpler and faster, removes /all allocations/ from the driver (quite trivially, since they were absolutely needless), and makes libshare thread-safe (except, maybe, linux/smb, but that only does pointer-width loads/stores so it's also mostly fine, except for leaking smb_shares) Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 commit 4ccbb23971f6e8b41073509fb4934f1dd1ea1e7e Author: наб Date: Mon Feb 28 13:37:06 2022 +0100 libshare: interface: {=> const} char * Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 commit 2faf05612f9172f4c02245cdfcca6b62a3e5f527 Author: наб Date: Mon Feb 28 13:13:10 2022 +0100 libshare/smb: cleanup Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 commit 566e4a58b7e0952243517fa9ed8b4104975ad884 Author: наб Date: Mon Feb 28 12:57:47 2022 +0100 libshare/nfs: destaticify nfs_lock_fd Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 commit ee668b83316c9020abc4929010fbb18ea29b1ae5 Author: наб Date: Mon Feb 28 12:55:07 2022 +0100 freebsd: libshare/nfs: write directly in translate_opts() This renders it thread-safe Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 commit 5f0c1c4ebd3ed1148a57ecdfeafce9b3adadc0d8 Author: наб Date: Mon Feb 28 12:40:14 2022 +0100 freebsd: libshare/nfs: constify static const data Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13165 commit 45588be2851b16add95883e3a7f0f6bdb9393f97 Author: Brian Behlendorf Date: Thu May 12 09:12:32 2022 -0700 Add missing AC_MSG_RESULT(no) to configure When the HAVE_IOPS_MKDIR_USERNS check fails output result as required. Reviewed-by: George Melikov Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13454 commit 196e7c76174b50df4ab54f0b41a58e5659e15a48 Author: Brian Behlendorf Date: Thu May 12 09:11:29 2022 -0700 ztest: reduce runtile of zloop.sh in CI The zloop.sh script is primarily designed to randomly stress the DMU and SPA layers. This can result in some unrealistic (or even impossible) scenarios being tested which then fail. Since the longer we run zloop.sh the more likely this is to occur this commit reduces the runtime. The intention being that normally this will result in a clean CI run unless the PR does introduce serious breaking change. Reviewed-by: George Melikov Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13453 commit bd88c036e61811ced1d8d3dfa89d590b9aa133a6 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Wed May 11 16:26:55 2022 -0400 Added a workaround for Linux KASAN builds Linux passes -Wframe-larger-than=1024, which breaks our build in a number of places with -Werror. Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #13450 commit 5a142533f8b117f6ac3ad7490ae4ac2cef3d78e9 Author: наб Date: Wed May 11 19:58:19 2022 +0200 udev: zvol_id: simplify/modernise zero-alloc, sensibler errors, don't close (or free) before exit. Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13337 commit 1f5bc12893eefb0e3a247c2266c94a6f7f2b3364 Author: наб Date: Tue May 3 16:58:40 2022 +0200 ztest: O_CLOEXEC ztest_fd_rand Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13411 commit 888914486e26d81c8dbfd7a9d8091c05f9fc98ba Author: наб Date: Tue May 3 16:55:14 2022 +0200 ztest: take -B ./path/to/ztest, LD_LIBRARY_PATH=./path/lib:$L_L_P This changes the behaviour of -B from the illumos one which would, in the example in the manual, take just ./chroots/lenny; this, however, is more versatile, and scales much better for systems with ZFS in /usr/local, for example Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13411 Closes #1770 commit 951a3889d19925fb75bd0bfaac0dce6767e2e0a4 Author: наб Date: Tue May 3 15:40:34 2022 +0200 tests: many_fds: simplify, modernise Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13411 commit 510ee280c0b6b025eca0a786a180c14f21aee715 Author: наб Date: Tue May 3 14:52:53 2022 +0200 Remove enable_extended_FILE_stdio() Even on Illumos it's only available in the 32-bit programming environment, and, quoth enable_extended_FILE_stdio(3C): > Historically, 32-bit Solaris applications have been limited to using > only the file descriptors 0 through 255 with the standard I/O > functions (see stdio(3C)) in the C library. The extended FILE > facility allows well-behaved 32-bit applications to use any > valid file descriptor with the standard I/O functions. where "well-behaved" means that it > does not directly access any fields in the FILE structure pointed > to by the FILE pointer associated with any standard I/O stream, And the stdio/flush.c implementation reads: /* * if this is not an internal extended FILE then check * if _file is being changed from underneath us. * It should not be because if * it is then then we lose our ability to guard against * silent data corruption. */ if (!iop->__xf_nocheck && bad_fd > -1 && iop->_magic != bad_fd) { (void) fprintf(stderr, "Application violated extended FILE safety mechanism.\n" "Please read the man page for extendedFILE.\nAborting\n"); abort(); } This appears to be an insane workaround for broken implementation with exposed FILE internals and _file being an u8, both only on non-LP64; it's shimmed out on all LP64 targets in Illumos, and we shim it out as well: just get rid of it This appears to've been originally fixed in illumos-gate a5f69788de7ac07553de47f7fec8c05a9a94c105 ("PSARC 2006/162 Extended FILE space for 32-bit Solaris processes", "1085341 32-bit stdio routines should support file descriptors >255"), which also bears extendedFILE and enable_extended_FILE_stdio(3C): - unsigned char _file; /* UNIX System file descriptor */ + unsigned char _magic; /* Old home of the file descriptor */ + /* Only fileno(3C) can retrieve the value now */ and +/* + * Macros to aid the extended fd FILE work. + * This helps isolate the changes to only the 32-bit code + * since 64-bit Solaris is not affected by this. + */ +#ifdef _LP64 +#define GET_FD(iop) ((iop)->_file) +#define SET_FILE(iop, fd) ((iop)->_file = (fd)) +#else +#define GET_FD(iop) \ + (((iop)->__extendedfd) ? _file_get(iop) : (iop)->_magic) +#define SET_FILE(iop, fd) (iop)->_magic = (fd); (iop)->__extendedfd = 0 +#endif Also remove the 1k setrlimit(NOFILE) calls: that's the default on Linux, with 64k on Illumos and 171k on FreeBSD Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13411 commit e0911f7b7f1ebd9aa00c198b87618b0308fe8c85 Author: szubersk Date: Sat May 7 00:53:42 2022 +0000 autoconf: Fail when __copy_from_user_inatomic is a non-GPL symbol A followup to 849c14e04844a2f0e1f7e42886c2cef083563f35 Fix https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1009242 Reviewed-by: Brian Behlendorf Signed-off-by: szubersk Closes #13389 commit f567d67fdae5819ec25048a8b5d648fc14e6defe Author: Brian Atkinson Date: Wed May 11 11:38:16 2022 -0400 Adding ZTS test for O_APPEND Commit 63b18e4 fixed an issue in zpl_aio_write() to make sure that kiocb->ki_pos was updated correctly when opening a file with O_APPEND. Adding a test to verify O_APPEND functionality with lseek can make sure that all other distros/kernel versions also have the correct behavior. Also moved the threadappends_001_pos test into this append test directory in functional ZTS directory. This way the two append tests are together for organization purposes. Reviewed-by: Brian Behlendorf Signed-off-by: Brian Atkinson Closes #13424 commit 3ed04d66aa440ee3e2c38d639a26f11d568e387d Author: наб Date: Tue May 3 13:17:50 2022 +0200 Remove constrained path on clean Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit e8ca72439327d1f249bef65bc48f9c7520470818 Author: наб Date: Tue May 3 12:22:30 2022 +0200 ztest: fix in-tree detection for automatic zdb path Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit c6a5d7d9970a26f10e0404bebeeda5e1acc30b3a Author: наб Date: Tue May 3 12:15:46 2022 +0200 ztest: use $ZDB instead of $ZDB_PATH for zdb Which actually gets zdb as set in common.sh Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 1a58941275c50b5f2a51c82f41cb522f0b47852a Author: наб Date: Sat Apr 30 01:21:16 2022 +0200 scripts: zpool.sh: cleanup Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit c982359460268cca466ce5fae3e6ef6a6c3e7c12 Author: наб Date: Mon Apr 25 23:27:03 2022 +0200 cppcheck: explicitly exclude kernel code from userspace checks Thus extracting the final shred of utility Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit c1851bf08176d2b387ba8257c2dd2b58516ec249 Author: наб Date: Tue Apr 12 00:56:32 2022 +0200 Makefile: respect V=1 for checks Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 779ac93e9600fb50d7901bbddc6e5c6ed970845e Author: наб Date: Mon Apr 11 23:41:14 2022 +0200 autogen.sh: paper over automake <1.14's lack of %reldir% support Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 612b8dff5bc3d827efb864a199a62bda1a419254 Author: наб Date: Sun Apr 10 23:10:02 2022 +0200 contrib: bash_completion.d: install, fix dist dist diff: -zfs-2.1.99/contrib/bash_completion.d/zfs install diff: +destdir/usr/local/etc/bash_completion.d +destdir/usr/local/etc/bash_completion.d/zfs Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 0a9aaa7f0cc16f1771f492cce665bc5c5eb8e735 Author: наб Date: Sun Apr 10 22:47:56 2022 +0200 cmd: move single-file binaries up, extract udev programs to udev/ Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit eaf94bda6cc31eb11822be5c782b8b43279212ee Author: наб Date: Sun Apr 10 19:00:47 2022 +0200 Replace config/config.awk with simple sed invocation Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit ea04cc4a22a865199ab6ef1c4215117d5b1bae64 Author: наб Date: Sun Apr 10 18:24:48 2022 +0200 autoconf: use include directives instead of recursing down test data We drop /multiple/ seconds off the generation, a dozen off a clean rebuild, 185 files, and trivialise the distribution, which can now be trivially generated via the provided snippets Dist diff: -zfs-2.1.99/tests/zfs-tests/tests/functional/pam/utilities.kshlib +zfs-2.1.99/tests/zfs-tests/tests/functional/pam/utilities.kshlib.in Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 0425d58852a4b834b1813e0feb1a1ba6d5b72de8 Author: наб Date: Sun Apr 10 02:46:19 2022 +0200 autoconf: use include directives instead of recursing down tests (mostly) Only down to tests/zfs-tests/tests, but pull out C programs into the main Makefile ‒ this means we get correct dependency tracking for all programs (and parallelise across them) dist diff: -zfs-2.1.99/tests/zfs-tests/tests/stress/ -zfs-2.1.99/tests/zfs-tests/tests/stress/Makefile.am -zfs-2.1.99/tests/zfs-tests/tests/stress/Makefile.in Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 48f4379974d5b28bba9c22fcf3bab6ceb2844ff7 Author: наб Date: Sun Apr 10 01:19:15 2022 +0200 autoconf: use include directives instead of recursing down etc dist diff: -zfs-2.1.99/etc/systemd/system/50-zfs.preset.in +zfs-2.1.99/etc/systemd/system/50-zfs.preset Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 50d2c9e4fd2a891cad9530da39b39a173307e1f7 Author: наб Date: Sun Apr 10 00:16:59 2022 +0200 autoconf: use include directives instead of recursing down contrib Also make the pyzfs build actually out-of-tree and quiet by default Reviewed-by: Brian Behlendorf Co-authored-by: Rapptz Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 674a9f3727f6e54fb3a391d4d6257a96f3a81a6e Author: наб Date: Sat Apr 9 20:05:18 2022 +0200 autoconf: use include directives instead of recursing down udev Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 2820719800cde2576baa12a43e4b9124a1906e40 Author: наб Date: Sat Apr 9 19:44:40 2022 +0200 autoconf: use include directives instead of recursing down rpm Also simplify .gitignore Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 93a8c85e7bf5acfc3d2d9c6fade77c921c4ae396 Author: наб Date: Sat Apr 9 19:20:52 2022 +0200 Move test-runner.1 into top-level man/ dist delta: +zfs-2.1.99/man/man1/test-runner.1 -zfs-2.1.99/tests/test-runner/man/ -zfs-2.1.99/tests/test-runner/man/test-runner.1 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 0f6c4fd00ef7ae320860f792b17f103166511507 Author: наб Date: Sat Apr 9 19:19:09 2022 +0200 autoconf: use include directives instead of recursing down man Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 0a8b1fc6251af5d16d9f87ce696ec6f04edaba31 Author: наб Date: Sat Apr 9 18:35:53 2022 +0200 autoconf: use include directives instead of recursing down scripts Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 09a7ad38a5407b5ac6c2a75f679504367655e471 Author: наб Date: Sat Apr 9 14:37:22 2022 +0200 autoconf: single-step includes Still descend, but only once: we get a lot of mileage out of nodist_ Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 5cdca5b1da7884c54260db473f892354bc79951b Author: наб Date: Sat Apr 9 04:09:55 2022 +0200 autoconf: use include directives instead of recursing down cmd No installation diff, dist lost -zfs-2.1.99/cmd/fsck_zfs/fsck.zfs which was distributed erroneously, since it's generated Also clean gitrev on clean Also add -e 'any possible bashisms' to default checkbashisms flags, and fully parallelise it and shellcheck, and it works out-of-tree, too Also align the Release in the dist META file correctly Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 3ff81c460114e0b3f69a9993d4777ec2c590fb16 Author: наб Date: Fri Apr 8 03:58:14 2022 +0200 cmd: zvol_id: don't build with -fno-stack-protector anymore #569 was opened in 2012 and closed in 2015; if the issue was still there we'd presumably've seen it? Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit c8970f52ed53ed62262b6246c7f791018ee96d2b Author: наб Date: Fri Apr 8 01:07:08 2022 +0200 autoconf: use include directives instead of recursing down lib As a bonus, this also adds zfs-mount-generator (previously undescended down) and libzstd (not included) to CppCheck As a bonus bonus, abigail rules work out-of-tree, too Against current trunk: $ diff -U0 ./destdir.listing ~/store/code/zfs/destdir.listing -destdir/usr/local/include/libspl/sscanf.h $ diff --color -U0 ./zfs-2.1.99.tar.gz.listing ../oot/zfs-2.1.99.tar.gz.listing | grep -v @@ | grep -v /Makefile -zfs-2.1.99/config/Abigail.am -zfs-2.1.99/lib/libspl/include/util/ -zfs-2.1.99/lib/libspl/include/util/sscanf.h $ diff --color -U0 ./zfs-2.1.99.tar.gz.listing ../oot/zfs-2.1.99.tar.gz.listing | grep -v @@ | grep /Makefile -zfs-2.1.99/lib/libavl/Makefile.in -zfs-2.1.99/lib/libefi/Makefile.in -zfs-2.1.99/lib/libicp/Makefile.in -zfs-2.1.99/lib/libnvpair/Makefile.in -zfs-2.1.99/lib/libshare/Makefile.in -zfs-2.1.99/lib/libspl/include/Makefile.in -zfs-2.1.99/lib/libspl/include/os/freebsd/Makefile.am -zfs-2.1.99/lib/libspl/include/os/freebsd/Makefile.in -zfs-2.1.99/lib/libspl/include/os/freebsd/sys/Makefile.am -zfs-2.1.99/lib/libspl/include/os/freebsd/sys/Makefile.in -zfs-2.1.99/lib/libspl/include/os/linux/Makefile.am -zfs-2.1.99/lib/libspl/include/os/linux/Makefile.in -zfs-2.1.99/lib/libspl/include/os/linux/sys/Makefile.am -zfs-2.1.99/lib/libspl/include/os/linux/sys/Makefile.in -zfs-2.1.99/lib/libspl/include/os/Makefile.am -zfs-2.1.99/lib/libspl/include/os/Makefile.in -zfs-2.1.99/lib/libspl/include/rpc/Makefile.am -zfs-2.1.99/lib/libspl/include/rpc/Makefile.in -zfs-2.1.99/lib/libspl/include/sys/dktp/Makefile.am -zfs-2.1.99/lib/libspl/include/sys/dktp/Makefile.in -zfs-2.1.99/lib/libspl/include/sys/Makefile.am -zfs-2.1.99/lib/libspl/include/sys/Makefile.in -zfs-2.1.99/lib/libspl/include/util/Makefile.am -zfs-2.1.99/lib/libspl/include/util/Makefile.in -zfs-2.1.99/lib/libspl/Makefile.in -zfs-2.1.99/lib/libtpool/Makefile.in -zfs-2.1.99/lib/libunicode/Makefile.in -zfs-2.1.99/lib/libuutil/Makefile.in -zfs-2.1.99/lib/libzfsbootenv/Makefile.in -zfs-2.1.99/lib/libzfs_core/Makefile.in -zfs-2.1.99/lib/libzfs/Makefile.in -zfs-2.1.99/lib/libzpool/Makefile.in -zfs-2.1.99/lib/libzstd/Makefile.in -zfs-2.1.99/lib/libzutil/Makefile.in -zfs-2.1.99/lib/Makefile.in Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 6fc34371e1df4d74dee6e50ffd9d9fae234b55ba Author: наб Date: Thu Apr 7 22:49:22 2022 +0200 libzfs: pool: fix false-positives -Wmaybe-uninitialised As noted by gcc (Debian 10.2.1-6) 10.2.1 20210110 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit be91239efaec7729d4eac334f94395aeb4699092 Author: наб Date: Thu Apr 7 02:22:37 2022 +0200 module: Makefile: cppcheck: zfs_config.h lives in builddir Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit ae43120abc7f8ac952530c48d413d4dc89ab94cc Author: наб Date: Thu Apr 7 00:41:46 2022 +0200 config: user: drop ZONENAME, avoid lying about being Linux-only Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 0fb1bf6586dc1d24d384d64cc2b32afb9e165332 Author: наб Date: Wed Apr 6 03:12:33 2022 +0200 libspl: include: remove [util/]sscanf.h Unused, empty, installs in a weird location Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 841fd303b852e241b347490748184cc5aeea8b7b Author: наб Date: Wed Apr 6 01:34:25 2022 +0200 copy-builtin: add hooks with sed/>> The order in fs/Makefile doesn't matter, the order in fs/Kconfig is preserved (ext2 is included as the first thing in the first if BUILD block, and only once), but I don't think it matters much either, realistically Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit c6f923fba4bebce43ee414c1e8a7dda91f9fe2b1 Author: наб Date: Wed Apr 6 00:36:09 2022 +0200 autogen.sh: explicitly build its containing directory No changes in currently-accepted usages (no-argument), but allows /src/path/autogen.sh && /src/path/configure for simpler out-of-tree builds Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit ea532b0e6ee61370ad552533016c48c858240671 Author: наб Date: Wed Apr 6 00:20:16 2022 +0200 Remove compat spl-x.y.z from the distribution This was added in 93ce2b4ca5a40c41ac945cd3aaf4a4a22bb751e1 ("Update build system and packaging"), which merged the SPL and ZFS trees, and included in 0.8.0; "the next major release" was 2.0.0 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit 5fc332ae4b723a39cffca9386bc40a3a5398924e Author: наб Date: Wed Apr 6 00:15:45 2022 +0200 Makefile: zfs-tests lives in srcdir; zfs-tests: accept common.sh path This fixes out-of-tree builds Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13316 commit ecaccf764a8ccbece8bb411f56b81ef5858fe710 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Tue May 10 13:14:07 2022 -0400 Workaround broken VDEV_UPATH Sometimes, for reasons I haven't looked into yet, VDEV_UPATH gets set to /dev/(null), breaking all these scripts. It'd be nice to have a fallback case to avoid total failure. Reviewed-by: Tony Hutter Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #13436 commit a30927f763834d3b16b4945ff8d5638ede69d545 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Mon May 9 19:33:55 2022 -0400 Add workaround for broken Linux pipes Linux has an unresolved hang if you resize a pipe with bytes in it. Since there's no obvious way to detect this happening, added a workaround to disable resizing the pipe buffer if you set an environment variable. Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #13309 commit a18d13c200b1d3e059d9435a557093148a10ace6 Author: hping Date: Tue May 10 07:30:16 2022 +0800 abd_os: remove redundant refcount creation for abd_children Refcount creation for abd_zero_scatter->abd_children is redundant in abd_alloc_zero_scatter, as it has been done in abd_init_struct. In addition, abd_children is undefined when ZFS_DEBUG is disabled, the reference of abd_children in abd_alloc_zero_scatter breaks build of libzpool when ZFS_DEBUG is disabled. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Ping Huang Closes #13429 commit d1fd6dbf6d51ec0784cd987ead43504ebc02dd41 Author: наб Date: Tue May 10 01:03:17 2022 +0200 man: zpool-import.8: -d -or -c Reviewed-by: George Melikov Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13437 commit 493b6e56077842de05a91563760c2055b3ddba79 Author: Aidan Harris Date: Fri May 6 18:57:37 2022 +0000 Fix functions without a prototype clang-15 emits the following error message for functions without a prototype: fs/zfs/os/linux/spl/spl-kmem-cache.c:1423:27: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes] Reviewed-by: Brian Behlendorf Signed-off-by: Aidan Harris Closes #13421 commit e77d59ebb2fc4394f5e7ddc85d849ec8c4f0a0d2 Author: наб Date: Thu May 5 18:45:12 2022 +0200 zpool: guard vs_noalloc and vs_pspace with VDEV_STAT_VALID() Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13412 commit 7bf06f7262b52b6d8ac5725e072f46ad450485be Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Wed May 4 14:59:30 2022 -0400 Corrected edge case in uncompressed ARC->L2ARC handling I genuinely don't know why this didn't come up before, but adding the LZ4 early abort pointed out this flaw, in which we're allocating a buffer of one size, and then telling the compressor that we're handing it buffers of a different size, which may be Very Different - say, allocating 512b and then telling it the inputs are 128k. Reviewed-by: Brian Behlendorf Reviewed-by: George Amanakis Signed-off-by: Rich Ercolani Closes #13375 commit 81b8b2d004442c3e7f196d61c2ae8eed02712b6c Author: Mateusz Guzik Date: Wed May 4 20:46:37 2022 +0200 FreeBSD: use zero_region instead of allocating a dedicated page Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Mateusz Guzik Closes #13406 commit a894ae75b817631e49cf8a23f3f9b432bd31296e Author: WHR Date: Tue May 3 20:07:04 2022 +0800 Fix incorrect use of unit prefix names in man pages Reviewed-by: George Melikov Reviewed-by: Brian Behlendorf Reviewed-by: Damian Szuberski Signed-off-by: WHR Closes #13363 commit c55b2932878c444e1bc627a7071787a0e5023d63 Author: Alexander Motin Date: Wed May 4 14:33:42 2022 -0400 Improve mg_aliquot math When calculating mg_aliquot alike to #12046 use number of unique data disks in the vdev, not the total number of children vdev. Increase default value of the tunable from 512KB to 1MB to compensate. Before this change each disk in striped pool was getting 512KB of sequential data, in 2-wide mirror -- 1MB, in 3-wide RAIDZ1 -- 768KB. After this change in all the cases each disk should get 1MB. Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Closes #13388 commit 34dbc618f50cfcd392f90af80c140398c38cbcd1 Author: Brian Behlendorf Date: Wed May 4 11:17:29 2022 -0700 Reduce dbuf_find() lock contention Holding a dbuf is a common operation which can become highly contended in dbuf_find() when acquiring the dbuf hash mutex. This is particularly true on Linux when reading/writing volumes since by default up to 32 threads from the zvol_taskq may be taking a hold of the same dbuf. This should also be observable on FreeBSD as long as there are enough processes accessing the volume concurrently. This is further aggregrated by the fact that only the block id will be unique when calculating the dbuf hash for a single volume. The objset id, object id, and level will be the same for data blocks. This has been observed to result in a somehwat less than uniform hash distribution and a longer than expected max hash chain depth (~20) on a large memory system (256 GB) using volumes. This commit improves the siutation by switching the hash mutex to an rwlock to allow concurrent lookups, and increasing DBUF_RWLOCKS from 2048 to 8192 to further reduce the odds of a hash collision. Reviewed-by: Tony Hutter Reviewed-by: Alexander Motin Signed-off-by: Brian Behlendorf Closes #13405 commit 411f4a018d7a5672701975b4fd8a697ddff16f33 Author: Shaan Nobee Date: Wed May 4 00:23:26 2022 +0400 Speed up WB_SYNC_NONE when a WB_SYNC_ALL occurs simultaneously Page writebacks with WB_SYNC_NONE can take several seconds to complete since they wait for the transaction group to close before being committed. This is usually not a problem since the caller does not need to wait. However, if we're simultaneously doing a writeback with WB_SYNC_ALL (e.g via msync), the latter can block for several seconds (up to zfs_txg_timeout) due to the active WB_SYNC_NONE writeback since it needs to wait for the transaction to complete and the PG_writeback bit to be cleared. This commit deals with 2 cases: - No page writeback is active. A WB_SYNC_ALL page writeback starts and even completes. But when it's about to check if the PG_writeback bit has been cleared, another writeback with WB_SYNC_NONE starts. The sync page writeback ends up waiting for the non-sync page writeback to complete. - A page writeback with WB_SYNC_NONE is already active when a WB_SYNC_ALL writeback starts. The WB_SYNC_ALL writeback ends up waiting for the WB_SYNC_NONE writeback. The fix works by carefully keeping track of active sync/non-sync writebacks and committing when beneficial. Reviewed-by: Brian Behlendorf Signed-off-by: Shaan Nobee Closes #12662 Closes #12790 commit a64d757aa4d4796af540ebe2a098e82c94ccbfcf Author: Pawel Jakub Dawidek Date: Mon May 2 16:26:28 2022 -0700 FreeBSD: Clean up the use of ioflags - Prefer O_* flags over F* flags that mostly mirror O_* flags anyway, but O_* flags seem to be preferred. - Simplify the code as all the F*SYNC flags were defined as FFSYNC flag. - Don't define FRSYNC flag, so we don't generate unnecessary ZIL commits. - Remove EXCL define, FreeBSD ignores the excl argument for zfs_create() anyway. Reviewed-by: Allan Jude Reviewed-by: Brian Behlendorf Signed-off-by: Pawel Jakub Dawidek Closes #13400 commit 159c6fd1540239120f9872d7f4cf8d340fc21c44 Author: Jitendra Patidar <53164267+jsai20@users.noreply.github.com> Date: Mon May 2 23:31:26 2022 +0530 Add missing replay entry in zvol_replay_vector for TX_SETSAXATTR Commit 361a7e8 (log xattr=sa create/remove/update to ZIL) introduced a TX_SETSAXATTR, but missed to add a corresponding entry in zvol_replay_vector. Adding a missing replay entry in zvol_replay_vector. Reviewed-by: Christian Schwarz Reviewed-by: Brian Behlendorf Signed-off-by: Jitendra Patidar Closes #13396 Closes #13395 commit 1bf3abc6343bc5b14c13bc402ffcdd4bbd6c7f40 Author: Brian Behlendorf Date: Fri Apr 29 14:21:11 2022 -0700 Silence unused-but-set-variable warnings Clang 13.0.0 added support for `Wunused-but-set-parameter` and `-Wunused-but-set-variable` which correctly detects two unused variables in zstd resulting in a build failure. This commit annotates these instances accordingly. https://releases.llvm.org/13.0.1/tools/clang/docs/ReleaseNotes.html#id6 In FSE_createCTable(), malloc() is intentionally defined as NULL when compiled in the kernel so the variable is unused. zstd/lib/compress/fse_compress.c:307:12: error: variable 'size' set but not used [-Werror,-Wunused-but-set-variable] Additionally, in ZSTD_seqDecompressedSize() the assert is compiled out similarly resulting in an unused variable. zstd/lib/compress/zstd_compress_superblock.c:412:12: error: variable 'litLengthSum' set but not used [-Werror,-Wunused-but-set-variable] Reviewed-by: Rich Ercolani Signed-off-by: Brian Behlendorf Closes #13382 commit f2330bd1568489ae1fb16d975a5a9bcfe12ed219 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Thu Apr 28 18:12:24 2022 -0400 Default zfs_max_recordsize to 16M Increase the default allowed maximum recordsize from 1M to 16M. As described in the zfs(4) man page, there are significant costs which need to be considered before using very large blocks. However, there are scenarios where they make good sense and it should no longer be necessary to artificially restrict their use behind a module option. Note that for 32-bit platforms we continue to leave this restriction in place due to the limited virtual address space available (256-512MB). On these systems only a handful of blocks could be cached at any one time severely impacting performance and potentially stability. Reviewed-by: Brian Behlendorf Reviewed-by: Alexander Motin Signed-off-by: Rich Ercolani Closes #12830 Closes #13302 commit 63b18e4097cd5268cb197327ee8494ffa3dca151 Author: Brian Behlendorf Date: Wed Apr 27 12:56:17 2022 -0700 Fix O_APPEND for Linux 3.15 and older kernels When using a Linux kernel which predates the iov_iter interface the O_APPEND flag should be applied in zpl_aio_write() via the call to generic_write_checks(). The updated pos variable was incorrectly ignored resulting in the current offset being used. This issue should only realistically impact the RHEL/CentOS 7.x kernels which are based on Linux 3.10. Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13370 Closes #13377 commit 7dde17e8609313d41c8956b3fec6db3a695bf852 Author: Satadru Pramanik Date: Wed Apr 27 15:54:17 2022 -0400 Linux 5.18 compat: replace __set_page_dirty_nobuffers Replace __set_page_dirty_nobuffers with filemap_dirty_folio. Upstream-commit: 6b1f86f8e9c7f9de7ca1cb987b2cf25e99b1ae3a ("Merge tag 'folio-5.18b' of git://git.infradead.org/users/willy/pagecache ") Reviewed-by: Brian Behlendorf Reviewed-by: Tony Hutter Authored-by: Satadru Pramanik Signed-off-by: Satadru Pramanik Closes #13325 Closes #13380 commit 2a70a090729dade59967bbfe6a4984c18613e0ee Author: наб Date: Sat Apr 23 19:21:09 2022 +0200 zfs: holds: dequadratify Before: 15 0m0.177s 30 0m0.653s 45 0m1.289s 60 0m2.129s 75 0m3.264s 90 0m4.397s 100 0m5.996s 117 0m8.552s After: 30 0m0.053s 117 0m0.125s Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13372 Closes #13373 commit 55d36e47c814209304c232e0facd1331a0c52f64 Author: наб Date: Sat Apr 23 19:19:41 2022 +0200 zfs: holds: general cleanup Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13373 commit 849c14e04844a2f0e1f7e42886c2cef083563f35 Author: Damian Szuberski Date: Wed Apr 27 03:52:40 2022 +1000 PPC get_user workaround Linux 5.12 PPC 5.12 get_user() and __copy_from_user_inatomic() inline helpers very indirectly include a reference to the GPL'd array mmu_feature_keys[] and fails to build. Workaround this by using copy_from_user() and throwing EFAULT for any calls to __copy_from_user_inatomic(). This is a workaround until a fix for Linux commit 7613f5a66becfd0e43a0f34de8518695888f5458 "powerpc/64s/kuap: Use mmu_has_feature()" is fully addressed. Reviewed-by: Brian Behlendorf Authored-by: Colin Ian King Signed-off-by: szubersk Closes #11958 Closes #12590 Closes #13367 commit a0dfd98a25b57411b5b31e69356a92b194b690a9 Author: Damian Szuberski Date: Wed Apr 27 03:47:09 2022 +1000 autoconf: Pretend `CONFIG_MODULES` is always on - Unconditionally inject `CONFIG_MODULES` make variable and `#define CONFIG_MODULES` to Kbuild in `ZFS_LINUX_COMPILE` autoconf function to emulate loadable kernel modules support. This allows OpenZFS to perform Linux checks despite `CONFIG_MODULES=n` in the actual Linux config. - Add `ZFS_AC_KERNEL_CONFIG_MODULES` check which encompasses the logic from `ZFS_AC_KERNEL_TEST_MODULE` with additional diagnostic messages to the user - Removed `ZFS_AC_KERNEL_TEST_MODULE` as it merely duplicates every check in `ZFS_AC_KERNEL_CONFIG_DEFINED` - Moved `ZFS_AC_MODULE_SYMVERS` after `ZFS_AC_KERNEL_CONFIG_DEFINED` so the user has a chance to see the proper diagnostic from the steps before. A workaround for Linux's ``` commit 3e3005df73b535cb849cf4ec8075d6aa3c460f68 Author: Masahiro Yamada Date: Wed Mar 31 22:38:03 2021 +0900 kbuild: unify modules(_install) for in-tree and external modules If you attempt to build or install modules ('make modules(_install)' with CONFIG_MODULES disabled, you will get a clear error message, but nothing for external module builds. Factor out the modules and modules_install rules into the common part, so you will get the same error message when you try to build external modules with CONFIG_MODULES=n. Signed-off-by: Masahiro Yamada ``` Reviewed-by: Brian Behlendorf Signed-off-by: szubersk Closes #10832 Closes #13361 commit 600a02b8844edb16b88b9bb179d1fbd7a169037d Author: Alexander Motin Date: Tue Apr 26 13:44:21 2022 -0400 Improve log spacemap load time Previous flushing algorithm limited only total number of log blocks to the minimum of 256K and 4x number of metaslabs in the pool. As result, system with 1500 disks with 1000 metaslabs each, touching several new metaslabs each TXG could grow spacemap log to huge size without much benefits. We've observed one of such systems importing pool for about 45 minutes. This patch improves the situation from five sides: - By limiting maximum period for each metaslab to be flushed to 1000 TXGs, that effectively limits maximum number of per-TXG spacemap logs to load to the same number. - By making flushing more smooth via accounting number of metaslabs that were touched after the last flush and actually need another flush, not just ms_unflushed_txg bump. - By applying zfs_unflushed_log_block_pct to the number of metaslabs that were touched after the last flush, not all metaslabs in the pool. - By aggressively prefetching per-TXG spacemap logs up to 16 TXGs in advance, making log spacemap load process for wide HDD pool CPU-bound, accelerating it by many times. - By reducing zfs_unflushed_log_block_max from 256K to 128K, reducing single-threaded by nature log processing time from ~10 to ~5 minutes. As further optimization we could skip bumping ms_unflushed_txg for metaslabs not touched since the last flush, but that would be an incompatible change, requiring new pool feature. Reviewed-by: Matthew Ahrens Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored-By: iXsystems, Inc. Closes #12789 commit 0409d3327371cef8a8c5886cb7530ded6f5f1091 Author: George Amanakis Date: Tue Apr 26 02:25:42 2022 +0200 Improve zpool status output, list all affected datasets Currently, determining which datasets are affected by corruption is a manual process. The primary difficulty in reporting the list of affected snapshots is that since the error was initially found, the snapshot where the error originally occurred in, may have been deleted. To solve this issue, we add the ID of the head dataset of the original snapshot which the error was detected in, to the stored error report. Then any time a filesystem is deleted, the errors associated with it are deleted as well. Any time a clone promote occurs, we modify reports associated with the original head to refer to the new head. The stored error reports are identified by this head ID, the birth time of the block which the error occurred in, as well as some information about the error itself are also stored. Once this information is stored, we can find the set of datasets affected by an error by walking back the list of snapshots in the given head until we find one with the appropriate birth txg, and then traverse through the snapshots of the clone family, terminating a branch if the block was replaced in a given snapshot. Then we report this information back to libzfs, and to the zpool status command, where it is displayed as follows: pool: test state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A scan: scrub repaired 0B in 00:00:00 with 800 errors on Fri Dec 3 08:27:57 2021 config: NAME STATE READ WRITE CKSUM test ONLINE 0 0 0 sdb ONLINE 0 0 1.58K errors: Permanent errors have been detected in the following files: test@1:/test.0.0 /test/test.0.0 /test/1clone/test.0.0 A new feature flag is introduced to mark the presence of this change, as well as promotion and backwards compatibility logic. This is an updated version of #9175. Rebase required fixing the tests, updating the ABI of libzfs, updating the man pages, fixing bugs, fixing the error returns, and updating the old on-disk error logs to the new format when activating the feature. Reviewed-by: Matthew Ahrens Reviewed-by: Brian Behlendorf Reviewed-by: Mark Maybee Reviewed-by: Tony Hutter Co-authored-by: TulsiJain Signed-off-by: George Amanakis Closes #9175 Closes #12812 commit 95146fd7baba8f70baea35b9b6bfac277431b940 Author: наб Date: Fri Apr 22 20:59:51 2022 +0200 tests: cli_user: zfs_001_neg: print the problematic lines Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13352 commit 6625262c7099feaf741ff435dd91a574331635a7 Author: наб Date: Thu Mar 24 21:07:07 2022 +0100 man: zfs-send.8: fix -X synopses and description Also clean up the horrendously verbose -X handling in zfs_main() Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13352 commit 7eba3891e9d0c211a1650ecfdfc78c654be055f8 Author: Richard Laager Date: Fri Apr 22 17:37:03 2022 -0500 zvol_wait: Ignore locked zvols "When an encrypted zvol is locked the zfs-volume-wait service does not start. The /sbin/zvol_wait should not wait for links when the volume has property keystatus=unavailable." -- https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1888405 Reviewed-by: Tony Hutter Reviewed-by: Damian Szuberski Thanks: James Dingwall Signed-off-by: Richard Laager Closes #10662 commit 0cdda2edb3f312d56518934811960ad57564c1bb Author: наб Date: Thu Apr 21 16:27:15 2022 +0200 Linux 5.18 compat: kobj_type.default_attrs replaced with default_groups Upstream-commit: cdb4f26a63c391317e335e6e683a614358e70aeb ("kobject: kobj_type: remove default_attrs") Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13357 commit 520a7c8a85e12a92ef88f99ad623a5dc1422b0c1 Author: наб Date: Thu Apr 21 14:27:51 2022 +0200 linux: module: zfs: sysfs: constify types and attrs Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13357 commit c8871ff784731f56bab58efe6077fff38fa74c5a Author: наб Date: Thu Apr 21 13:32:06 2022 +0200 scripts: zfs.sh: explicitly unload all modules via rmmod modprobe -r only works for depmodded modules, but this also means we have to re-iterate legacy modules, and in the right order Reviewed-by: Brian Atkinson Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13356 commit bee24d69295b9d31eed85f5eac7f313250aebf56 Author: наб Date: Thu Apr 21 13:29:12 2022 +0200 scripts: zfs.sh: explicitly ignore unloaded modules when unloading Reviewed-by: Brian Atkinson Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13356 commit 2f31b585e77780b887c3cbf01c0b5498c42fe302 Author: Damian Szuberski Date: Thu Apr 21 18:37:11 2022 +0200 Strengthen Linux kernel capabilities detection - Add `CONFIG_BLOCK` Linux config requirement to `ZFS_AC_KERNEL_CONFIG_DEFINED`. OpenZFS won't compile without that block device support due to large amount of functional dependencies on it. - Remove dependency on `groups_alloc()` in `ZFS_AC_KERNEL_SRC_GROUP_INFO_GID` to circumvent the missing stub in Linux 4.X kernel headers. Reviewed-by: Brian Behlendorf Signed-off-by: szubersk Closes #13351 commit 47a02e39721bd226646dbdfe3063563a4a5e9749 Author: наб Date: Tue Apr 5 02:56:57 2022 +0200 contrib: dracut: remove getargbool polyfill It was originally released in dracut 008 in February 2011; we can probably drop it now Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13291 commit e3fc330d6cc9445821fa44c699b7e66e2f6f209d Author: наб Date: Tue Apr 5 01:14:49 2022 +0200 Add dracut.zfs.7 Thorough documentation with a dracut.bootup(7)-style flowchart, dracut.cmdline(7)-style cmdline listing, and per-file docs like the old README Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13291 commit 1cc9cc2f89eb67722109bbb5a2d9814b5e77d5db Author: наб Date: Tue Apr 5 01:09:11 2022 +0200 contrib: dracut: zfs-needshutdown: don't list Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13291 commit 6ebdb0b20de444f69404c265352a27e1b4e49b93 Author: наб Date: Tue Apr 5 00:19:38 2022 +0200 contrib: dracut: zfs-{rollback,snapshot}-bootfs: order after key loading This fixes at least one race I got with an encrypted root Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13291 commit 30c6dce7f7d3808b544ac1c3dbcb4d32c9831c60 Author: наб Date: Mon Apr 4 23:39:18 2022 +0200 contrib: dracut: don't require essentials to be under the same encroot Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13291 commit eaf1e060453ad6a335b708c5e724092741d6d1d3 Author: наб Date: Mon Apr 4 23:24:29 2022 +0200 contrib: dracut: inline single-use import_pool, move single-use ask_for_password Also don't set ROOTFS_MOUNTED; the final mention was removed in dracut 011 from July 2011 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13291 commit dac0b0785a795efe3bbb5367af35e2c4fbe32040 Author: наб Date: Mon Apr 4 23:16:59 2022 +0200 contrib: dracut: zfs-lib: remove find_bootfs Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13291 commit 5d31169d7c8810baf87f1dc778c97a7024c8c205 Author: наб Date: Mon Apr 4 22:57:35 2022 +0200 contrib: dracut: zfs-lib: simplify ask_for_password The only user is mount-zfs.sh (non-systemd systems), so reduce it to what it needs Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13291 commit fec2c613a49192942200d26bd20bf434649687b7 Author: наб Date: Mon Apr 4 22:52:43 2022 +0200 contrib; dracut: flatten zfs-load-key, simplify zfs-env-bootfs Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13291 commit 245529d85fb807bfc4525b3b1858896d2860995b Author: наб Date: Mon Apr 4 22:45:58 2022 +0200 contrib; dracut: centralise root= parsing, actually support root=s So far, everything parsed root= manually, which meant that while zfs-parse.sh was updated, and supposedly supported + -> ' ' conversion, it meant nothing Instead, centralise parsing, and allow: root= root=zfs root=zfs: root=zfs:AUTO root=ZFS=data/set root=zfs:data/set root=zfs:ZFS=data/set (as a side-effect; allowed but undocumented) rootfstype=zfs AND root=data/set <=> root=data/set rootfstype=zfs AND root= <=> root=zfs:AUTO So rootfstype=zfs /also/ behaves as expected, and + decoding works Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13291 commit 2c74617bcf76b305937097278b614443d47681a0 Author: наб Date: Mon Apr 4 20:59:53 2022 +0200 contrib: dracut: parse-zfs: stop pretending we support FILESYSTEM= It was added in the original ae26d0465a ("Add dracut support") commit in 2011, and was then broken a bit later with the advent of dracut-zfs-generator, or maybe earlier as part of other churn Either way, it's broken, and has been in 2.0+ as well, and no-one complained. Stop pretending we support it at all Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13291 commit 47636f5661abf202c725f1975939878e7156d3c8 Author: наб Date: Mon Apr 4 18:38:07 2022 +0200 contrib: dracut: parse-zfs: drop initqueue-finished for i/f The switch was released in dracut 009 in March 2011, we can safely get rid of the compatibility hook Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13291 commit b657f2c59220dba92607aa253ed2a6a1f6d9bec8 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Wed Apr 20 19:07:03 2022 -0400 Corrected oversight in ZERO_RANGE behavior It turns out, no, in fact, ZERO_RANGE and PUNCH_HOLE do have differing semantics in some ways - in particular, one requires KEEP_SIZE, and the other does not. Also added a zero-range test to catch this, corrected a flaw that made the punch-hole test succeed vacuously, and a typo in file_write. Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #13329 Closes #13338 commit 9209ea69bc03e7e9f678b2294da7a0317b5c9c5b Author: Alexander Motin Date: Wed Apr 20 19:05:38 2022 -0400 FreeBSD: Fix translation from ABD to physical pages In hypothetical case of non-linear ABD with single segment, multiple to page size but not aligned to it, vdev_geom_fill_unmap_cb() could fill one page less into bio_ma array. I am not sure it is exploitable, but better to be safe than sorry. Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Reported-by: Mark Johnston Signed-off-by: Alexander Motin Closes #13345 commit e37e7dd6a6f5ae170dd93f2b7843e6dfb7b2fb89 Author: наб Date: Thu Mar 24 20:14:25 2022 +0100 man: ... -> … again zfs-program.8 is left, but that's literal Lua syntax Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13255 commit b1fb4e1ba4986b4a6393e9b238a4f611c00c9bbf Author: Damian Szuberski Date: Wed Apr 20 22:43:42 2022 +0200 rpm -> deb doesn't fail when optional packages are missing Reviewed-by: Brian Behlendorf Signed-off-by: szubersk Closes #13331 Closes #13336 commit 92295af8004b07a58e0860d66dd565143486a757 Author: наб Date: Sat Apr 16 16:07:04 2022 +0200 Document zfs inherit -S's interaction with noninheritable properties Reviewed-by: Brian Behlendorf Reviewed-by: Damian Szuberski Signed-off-by: Ahelenia Ziemiańska Closes #11894 Closes #13335 commit 115a92ca6a8cfd242346e089a96fdcb09922a39a Author: Low-power Date: Thu Apr 21 04:35:44 2022 +0800 zpool_history_unpack: return correct errno on nvlist_unpack failure Reviewed-by: Brian Behlendorf Reviewed-by: Damian Szuberski Signed-off-by: WHR Closes #13321 commit 9b80d9e6f92b07290585d1e91e6da1e9223badba Author: наб Date: Mon Apr 4 13:16:31 2022 +0200 linux: module: uninstall legacy modules on (un)installation This can be reverted once we're sure nobody's using them anymore (post-3.0 release?) Reviewed-by: Tony Hutter Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13274 commit 469b8481aa8ee9448faf8093bf11d7f36dbe894d Author: наб Date: Mon Apr 4 13:08:59 2022 +0200 scripts: zfs.sh: unload zfs with dependencies Reviewed-by: Tony Hutter Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13274 commit d18da59d30dac5a9c7e0ea605af2a49f3fa4383e Author: наб Date: Mon Apr 4 12:55:41 2022 +0200 scripts: zfs.sh: make usage make sense We don't pass the arguments as arguments Reviewed-by: Tony Hutter Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13274 commit f6f505c4d62deef41ef2b130f465b3eb12b9a795 Author: наб Date: Mon Apr 4 12:48:47 2022 +0200 scripts: zfs.sh: remove cat Reviewed-by: Tony Hutter Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13274 commit ad9e767657c3d3c0ec9e09a17f6732bcbf915401 Author: наб Date: Thu Mar 31 17:20:50 2022 +0200 linux: module: weld all but spl.ko into zfs.ko Originally it was thought it would be useful to split up the kmods by functionality. This would allow external consumers to only load what was needed. However, in practice we've never had a case where this functionality would be needed, and conversely managing multiple kmods can be awkward. Therefore, this change merges all but the spl.ko kmod in to a single zfs.ko kmod. Reviewed-by: Tony Hutter Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13274 commit 310ab9d2610493f27fa624b295293a503828d48d Author: Allan Jude Date: Wed Apr 20 16:16:25 2022 -0400 Improve the inline descriptions of the ARC module parameters These are displayed as the descriptions of the sysctl's on FreeBSD Reviewed-by: George Melikov Reviewed-by: Brian Behlendorf Signed-off-by: Allan Jude Closes #13334 commit 4d462729132be6b9a3c30af288fa1e85361c0f4a Author: Paul Dagnelie Date: Tue Apr 19 13:55:04 2022 -0700 Fix style checking error introduced by zfs-mount changes Reviewed-by: Matthew Ahrens Reviewed-by: Brian Behlendorf Reviewed-by: Tony Nguyen Signed-off-by: Paul Dagnelie Closes #13347 commit 026f126b83c09b7aaaf9e10f57871283c9794a68 Author: Brian Behlendorf Date: Tue Apr 19 10:38:04 2022 -0700 Linux 5.17 compat: GENHD_FL_EXT_DEVT / GENHD_FL_NO_PART_SCAN As of the 5.17 kernel the GENHD_FL_EXT_DEVT flag has been removed and the GENHD_FL_NO_PART_SCAN flag renamed GENHD_FL_NO_PART. Update zvol_alloc() to set GENHD_FL_NO_PART for the newer kernels which is sufficient. The behavior for prior kernels remains unchanged. 1ebe2e5f ("block: remove GENHD_FL_EXT_DEVT") 46e7eac6 ("block: rename GENHD_FL_NO_PART_SCAN to GENHD_FL_NO_PART") Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13294 Closes #13297 commit ec4d860eb0771b8f9204716b96e4e6ff6c692a73 Author: omni <79493359+omnivagant@users.noreply.github.com> Date: Fri Apr 15 21:26:44 2022 +0000 init.d/zfs-mount: Don't fsck or mount/umount fstab entries This is better handled by existing OS toolset. Reviewed-by: Brian Behlendorf Reviewed-by: Damian Szuberski Signed-off-by: omni Issue #7374 Closes #12780 commit 4dced31b98228f2fedbbe32de60ffcf8cc8f4fb3 Author: Low-power Date: Sat Apr 16 05:16:07 2022 +0800 Fix 'zpool history' sometimes fails without reporting any error The corresponding function 'zpool_get_history' in libzfs would printing an error messages only when the ioctl call failed. Add missing error reporting, specifically memory allocation failures and error from 'zpool_history_unpack'. Also avoid possibly reading of uninitialized 'err' variable in case the requested offset pasts EOF. Reviewed-by: Brian Behlendorf Reviewed-by: Damian Szuberski Signed-off-by: WHR Issue #13322 Closes #13320 commit 0dd34a1955b06cdea460f2f1711b72536a5a511b Author: наб Date: Thu Apr 7 14:00:38 2022 +0200 libzfs: import: zpool_clear_label: bool for boolean status Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13304 commit 74e4bfbcff3636182b76a18d6d08efd3ad0ad165 Author: наб Date: Thu Apr 7 04:37:16 2022 +0200 libzfs: import: zpool_clear_label: don't allocate another time for L2ARC header Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13304 commit a4e0cee1780cbd8f2cb9a263a0ed8d91dbe68b4a Author: наб Date: Thu Apr 7 04:32:27 2022 +0200 libzfs: import: zpool_clear_label: actually fail if clearing l2arc header fails Found with -Wunused-but-set-variable on Clang trunk Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13304 commit 16c3290bbe291f3c560cc695b4a9b11a67bbfad3 Author: наб Date: Thu Apr 7 04:26:30 2022 +0200 module: zfs: vdev_removal: remove unused num_indirect Found with -Wunused-but-set-variable on Clang trunk Fixes: a1d477c24c ("OpenZFS 7614, 9064 - zfs device evacuation/removal") Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13304 commit 63cb3413ea702818d421e0bbee583c9749a296af Author: наб Date: Thu Apr 7 04:25:05 2022 +0200 tests: cmd: draid: remove unused and undocumented -v Found with -Wunused-but-set-variable on Clang trunk Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13304 commit 4620342a32bb616fd45f0b6af07deb31b8b38ee0 Author: szubersk Date: Mon Apr 11 14:49:08 2022 +0200 zfs(8): remove errors reported by mandoc 1.14.6 Reviewed-by: Brian Behlendorf Reviewed-by: Ahelenia Ziemiańska Signed-off-by: szubersk Closes #13315 commit 4f0be2bdb001876c48af47fccdb77b8f4682608e Author: szubersk Date: Mon Apr 11 15:14:38 2022 +0200 zfs(8): add requirements towards fs names Provide explicit requirements towards file system naming convention in OpenZFS man pages. Reviewed-by: Brian Behlendorf Reviewed-by: Ahelenia Ziemiańska Signed-off-by: szubersk Mitigates #13310 Closes #13315 commit 3d149db1f3a6dddeeb650a03c559415e8e9fdc80 Author: Brian Behlendorf Date: Mon Apr 11 17:26:01 2022 -0700 ZTS: Retry auto_spare_multiple.ksh The auto_spare_multiple.ksh test may incorrectly fail for a similar reason as the auto_spare_shared.ksh test. Add it to known list of exceptions which should be retried to prevent failures in the CI. Reviewed-by: George Melikov Signed-off-by: Brian Behlendorf Closes #13318 commit 5846f71182688a06cd625fbb635d0e6031798231 Author: Brian Behlendorf Date: Mon Apr 11 16:18:51 2022 -0700 ZTS: Retry redundancy_draid_spare[1,3].ksh The redundancy_draid_spare1.ksh and redundancy_draid_spare3.ksh test cases are a little to strict for the sequential resilver case. While unlikely it is possible that a handful of correctable checksum errors will be reported resulting in a test failure. Update the zts-report.py script to allow this the test case to be retried if requested. Reviewed-by: George Melikov Signed-off-by: Brian Behlendorf Closes #13318 commit 7dcb8ed23d76c13ae96eb9718bd7eb186aa89178 Author: Mark Johnston Date: Thu Apr 7 17:13:18 2022 -0400 FreeBSD: Return Mach error codes from VOP_(GET|PUT)PAGES FreeBSD's memory management system uses its own error numbers and gets confused when these VOPs return EIO. Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Reported-by: Peter Holm Signed-off-by: Mark Johnston Closes #13311 commit e9084d071226b2b8b5c44d9bd781b9632dc68227 Author: Mark Johnston Date: Thu Apr 7 17:11:00 2022 -0400 FreeBSD: Parameterize ZFS_ENTER/ZFS_VERIFY_VP with an error code For legacy reasons, a couple of VOPs have to return error numbers that don't come from the usual errno namespace. To handle the cases where ZFS_ENTER or ZFS_VERIFY_ZP fail, we need to be able to override the default error return value of EIO. Extend the macros to permit this. Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: Mark Johnston Closes #13311 commit 35d81a75a8c13e011e19fd12cf553d9c5849386e Author: Damian Szuberski Date: Tue Apr 12 00:51:23 2022 +0200 initramfs: use `mount.zfs` instead of `mount` A followup to d7a67402a85252e163aa8a9b69e7eda499db8c61 For `mount -t zfs -o opts ds mp` command line some implementations of `mount(8)`, e. g. Busybox in Debian work as follows: ``` newfstatat(AT_FDCWD, "ds", 0x7fff826f4ab0, 0) = -1 mount("ds", "mp", "zfs", MS_SILENT, NULL) = 0 ``` The logic above skips completely `mount.zfs` and prevents us from reading filesystem properties and applying mount options. For comparison, the coreutils `mount(8)` implementation does: ``` openat(AT_FDCWD, "/proc/filesystems", O_RDONLY|O_CLOEXEC) = 3 // figure out that zfs is a `nodev` filesystem and look for a helper newfstatat(AT_FDCWD, "/sbin/mount.zfs" ...) = 0 execve("/sbin/mount.zfs" ...) = 0 ``` Using `mount.zfs` in initramfs would help circumvent deficiencies of some of `mount(8)` implementations. `mount -t zfs` translates to `mount.zfs` invocation, except for cases when explicitly disabled by `-i`. Reviewed-by: Brian Behlendorf Signed-off-by: szubersk Closes #13305 commit ed715283de8e65a30d777e9576399ab75014b6fe Author: наб Date: Tue Apr 5 22:50:55 2022 +0200 cmd: zed: rc: drop "should be owned by root and 0600" It doesn't matter, 0600 are Weird Permissions, and it's even weirder to spec them for no reason ‒ it's perfectly fine if it's the usual 0:0 644, or literally anything else, so long as unprivileged users can't edit it (which (a) 644 accomplishes and (b) is at the administrator's discretion, it's not unheard of to have adm users and having it be 664 in that case is just as good; it's not our place to say) Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12544 Closes #13276 commit 4d972ab5aefb2499609f6327d0cfa5ab52856205 Author: Jorgen Lundman Date: Wed Apr 6 05:02:17 2022 +0900 Prefer ATTR_ in shared codebase over AT_ An earlier commit introduces AT_MODE into the shared kernel sources, instead of the preferred existing ATTR_MODE use. Reviewed-by: Brian Behlendorf Reviewed-by: Igor Kozhukhov Signed-off-by: Jorgen lundman Closes #13293 commit e3e4c30b0a56553af643cfa5204f4dd69e3b8529 Author: наб Date: Mon Apr 4 15:19:12 2022 +0200 libzfs: sendrecv: use common progress thread killer Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13284 commit 1217fd1ff8fc312f2167589c0e8dff969cde7ce1 Author: наб Date: Mon Apr 4 14:25:01 2022 +0200 libzfs: sendrecv: always cancel progress thread in zfs_send_one() This is in line with all the other uses of the progress thread Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #11560 Closes #13284 commit 3c69b539fe5acef100a2baad2084db6ade51f51b Author: наб Date: Mon Apr 4 14:07:36 2022 +0200 Forbid asctime{,_r}(), gmtime(), localtime() ctime() is only used in binary main threads, which is fine Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13284 commit 9185303b32b370762f12d405b6752166bb5ced47 Author: наб Date: Mon Apr 4 14:07:26 2022 +0200 libspl: zed: event: use localtime_r() Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13284 commit ae3683ce11153126ae0d03e89dccef96e21ccdf4 Author: наб Date: Mon Apr 4 13:59:48 2022 +0200 libspl: print_timestamp: use localtime_r() Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13284 commit 8ab279484f9987d26188782149dbed03c11ed1cd Author: наб Date: Mon Apr 4 13:56:16 2022 +0200 libzfs: sendrecv: send_progress_thread: use localtime_r() Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13284 commit d5036491e6d76a957e04e3e74c6fb50352ba94b0 Author: наб Date: Mon Apr 4 13:55:49 2022 +0200 zdb: standardise on ctime() Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13284 commit 95babbe5733f1851514ab8145dbb243d056f5366 Author: наб Date: Mon Apr 4 16:12:34 2022 +0200 scripts: cstyle: remove unused -h Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13285 commit 14ed1f242494b0120bfbfa75502339353e855d75 Author: наб Date: Mon Apr 4 16:10:52 2022 +0200 cstyle.1: note -g, $CI being equivalent to -g Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13285 commit 93f3a3517c6ade39942d8d6996476734ae8b98f3 Author: наб Date: Mon Apr 4 16:07:53 2022 +0200 scripts: cstyle: remove unused -C Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13285 commit 9292cf761e949cb52287d98da895a61c49b98561 Author: наб Date: Tue Apr 5 18:34:46 2022 +0200 Fix string/index variables being unflexed by default I got the status backward (B_FALSE for fixed, rather than B_TRUE for flex); before: $ zfs get mountpoint tarta-zoot -r NAME PROPERTY VALUE SOURCE tarta-zoot mountpoint / local tarta-zoot/PAGEFILE.SYS mountpoint - - tarta-zoot/etc mountpoint /etc inherited from tarta-zoot tarta-zoot/home mountpoint /home inherited from tarta-zoot tarta-zoot/home/xspon mountpoint /home/xspon inherited from tarta-zoot tarta-zoot/home/nabijaczleweli mountpoint /home/nabijaczleweli inherited from tarta-zoot tarta-zoot/home/nabijaczleweli/tftp mountpoint /home/nabijaczleweli/tftp inherited from tarta-zoot tarta-zoot/home/root mountpoint /root local after: $ zfs get mountpoint tarta-zoot -r NAME PROPERTY VALUE SOURCE tarta-zoot mountpoint / local tarta-zoot/PAGEFILE.SYS mountpoint - - tarta-zoot/etc mountpoint /etc inherited from tarta-zoot tarta-zoot/home mountpoint /home inherited from tarta-zoot tarta-zoot/home/xspon mountpoint /home/xspon inherited from tarta-zoot tarta-zoot/home/nabijaczleweli mountpoint /home/nabijaczleweli inherited from tarta-zoot tarta-zoot/home/nabijaczleweli/tftp mountpoint /home/nabijaczleweli/tftp inherited from tarta-zoot tarta-zoot/home/root mountpoint /root local Fixes: be8e1d81bf2b85f6f76851b7013ac54e9bf488f0 ("Flex non-pretty-printed properties and raw-/pretty-print remaining ones") Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13286 commit 7d524c068d62c740b7557f0ee5d3fe5edcd2fb08 Author: Riccardo Schirone Date: Sat Apr 2 01:15:25 2022 +0200 Linux 5.18 compat: use address_space_operations->readahead ->readpages was removed and replaced by ->readahead. Define zpl_readahead for kernels that don't have ->readpages. Reviewed-by: Brian Behlendorf Signed-off-by: Riccardo Schirone Closes #13278 commit 036e846abc84f823fa8cac806a469d964e703666 Author: Riccardo Schirone Date: Fri Apr 1 23:47:36 2022 +0200 Linux 5.18 compat: blkg_tryget is moved to private headers Reviewed-by: Brian Behlendorf Signed-off-by: Riccardo Schirone Closes #13278 commit b61507ec1d529bae45148ce3dd0c66bf9bc4d384 Author: Ryan Moeller Date: Sat Apr 2 15:10:55 2022 -0400 FreeBSD: Use NDFREE_PNBUF if available NDF_ONLY_PNBUF has been removed from FreeBSD in favor of NDFREE_PNBUF. Reviewed-by: Brian Behlendorf Signed-off-by: Ryan Moeller Closes #13277 commit 533aef2b3c1e65d44526a0776275f21bdcc7cd91 Author: наб Date: Sat Mar 26 11:25:05 2022 +0100 tests: zts-report: add #13215 to known list for FreeBSD Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit e78b7a73feaa255dff8981da794b45702797c948 Author: наб Date: Sat Mar 26 11:24:40 2022 +0100 tests: zts-report: issue numbers are numbers Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 6ef2151c80b6e68e03ff2272bdb50505309b2b2e Author: наб Date: Sat Mar 26 11:19:17 2022 +0100 tests: clean out more temporary files What remains is a bunch of anonymous untraceable /tmp/tmp.XXXXXXXXXX files and bak.root.receive.staff1.3835 from an error branch, testdir.1, testdir.3, and testroot454470 (with children) in testroot Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 22ddc7f5ff9c773bc85de5a61033f4967a96d3aa Author: наб Date: Thu Mar 24 14:47:22 2022 +0100 scripts: zfs-tests: fix setup script detection when ran with -t /absolute Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit fbe811b054fecd2ac9f7c3659616ffa1802aadf0 Author: наб Date: Thu Mar 24 13:15:00 2022 +0100 tests: remove unused functions As found by git -C tests/ grep ^function | grep -vFe '.lua:' -e '.zcp:' | while IFS=":$IFS" read -r _ _ fn _; do [ $(git -C tests/ grep -wF $fn | head -2 | wc -l) -eq 1 ] && echo $fn; done after all rounds this comes out to, sorted: check_slog_state chgusr_exec cksum_files cleanup_pools compare_modes count_ACE dataset_set_defaultproperties ds_is_snapshot get_ACE get_group get_min get_mode get_owner get_rand_checksum get_rand_checksum_any get_rand_large_recsize get_rand_recsize get_user_group getitem indirect_vdev_mapping_size is_dilos log_noresult log_notinuse log_other log_timed_out log_uninitiated log_warning num_jobs_by_cpu plus_sign_check_l plus_sign_check_v record_cksum rwx_node seconds_mmp_waits_for_activity set_cur_usr setup_mirrors setup_raidzs showshares_smb zfs_zones_setup This, of course, doesn't catch recursive ones, or ones that log with their own function name as a prefix, but Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 5c9f744b1a4ea8ab3e0a93442ea30ab62b88b504 Author: наб Date: Tue Mar 22 22:18:48 2022 +0100 tests: include: use already-set $UNAME instead of shelling out to uname each time Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit ff0fc5af12bc4c702664d1e9e48d158ba42ebc2e Author: наб Date: Wed Mar 23 02:21:56 2022 +0100 tests: pam: use absolute path to module .so This is a valid configuration and both (a) skips the tests if it's unbuilt/not installed and (b) makes it work even if installed outside the system directory (like in /u/l/l/s instead of /l/s) Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 7b6c4cbb1f4c308490b701d4ba181310d7d4d290 Author: наб Date: Mon Mar 21 00:56:58 2022 +0100 tests: zts-report: simplify Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit abccb4b58448e3dc73a81c77ce12f9c586416517 Author: наб Date: Sun Mar 20 23:42:54 2022 +0100 tests: zts-report: prune unused reasons Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 6b2616435b863f106df5098aca1d00742005feae Author: наб Date: Sun Mar 20 23:42:40 2022 +0100 tests: zts-report: only known-SKIP zfs_unshare_002_pos on Linux Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 56692e77e2178e9af977ca8bbf99a664df002f3f Author: наб Date: Tue Mar 22 15:50:46 2022 +0100 linux: libshare: smb: fix more than one smb_is_share_active() call This also fixes zfs_unshare_006_pos, which exposed this Fixes: 2f71caf2d926249920d1b9162550c56715cc6461 ("Allow zfs unshare -a") Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 34abca3e2c51f34f94ec4b4b91974792951e863a Author: наб Date: Thu Mar 17 21:27:11 2022 +0100 tests: zfs_003_neg: handle failures correctly Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 61f1502246f3c0ff8737e519807cbc8f9ffa053b Author: наб Date: Wed Mar 23 20:28:57 2022 +0100 tests: zfs_share_005: don't fail open Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 7b875ee6019ddf1d5eda98ac611403db2e98f670 Author: наб Date: Wed Mar 16 20:37:38 2022 +0100 module: zstd: zfs_zstd: staticify zstd_ksp Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit b1b5843ae4c8298a11791257d95b220ed8993b39 Author: наб Date: Wed Mar 23 20:45:25 2022 +0100 tests: lua_core: use herewords for single-line programs Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 598fed7ecdade29e569067ba49d5556a78f1adc3 Author: наб Date: Wed Mar 16 14:27:04 2022 +0100 tests: revert back to original coredump patterns on Linux, too Otherwise, they leak past the tests and contaminate the running system, breaking coredumps entirely Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Co-authored-by: Yannick Le Pennec Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 912d2aa7d7c76dc021c67a090a472a82c6431b52 Author: наб Date: Tue Mar 22 15:56:45 2022 +0100 tests: zdb_args_pos.ksh: fix indentation Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 20093de25c22fad1aa21b9c1317e5291908cd303 Author: наб Date: Tue Mar 22 20:09:35 2022 +0100 tests: move C test helpers into test cmd Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 1948f6dbbfc3cd77781c576ba129ac886d73eaac Author: наб Date: Tue Mar 15 01:54:10 2022 +0100 tests: echo-with-arguments review Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit b1579689a9e863ecd178e0a363fd9629e281ad27 Author: наб Date: Wed Mar 23 21:07:33 2022 +0100 tests: rsend/send-c_props: make it chooch Original error: 23:47:40.59 SUCCESS: eval zfs receive -dFv testpool2 < /mnt/testroot/backdir-rsend/pool-final-p 23:47:40.61 1,23d0 23:47:40.61 < type filesystem - 23:47:40.61 < origin POOL@psnap - 23:47:40.61 < volblocksize - - 23:47:40.61 < acltype nfsv4 inherited from POOL 23:47:40.61 < dnodesize legacy inherited from POOL 23:47:40.61 < atime off local 23:47:40.61 < canmount off local 23:47:40.61 < checksum off local 23:47:40.61 < compression off local 23:47:40.61 < copies 3 local 23:47:40.61 < devices off local 23:47:40.61 < exec off local 23:47:40.61 < quota none default 23:47:40.61 < readonly on local 23:47:40.61 < recordsize 128K local 23:47:40.61 < reservation none default 23:47:40.61 < setuid off local 23:47:40.61 < snapdir hidden local 23:47:40.61 < version 5 - 23:47:40.61 < volsize - - 23:47:40.61 < xattr off local 23:47:40.61 < mountpoint /PREFIX inherited from POOL 23:47:40.61 < jailed on local 23:47:40.62 cannot open 'testpool2/pclone': dataset does not exist 23:47:40.62 ERROR: cmp_ds_prop testpool/pclone testpool2/pclone exited 1 So: (a) actually send all the datasets in -p mode and (b) drop origin for clones sent with -p: 00:38:05.46 SUCCESS: eval zfs receive -dFv testpool2 < /mnt/testroot/backdir-rsend/pool-final-p 00:38:05.48 2c2 00:38:05.48 < origin POOL@psnap 00:38:05.48 --- 00:38:05.48 > origin POOL 00:38:05.49 ERROR: cmp_ds_prop testpool/pclone testpool2/pclone nosource exited 1 Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13250 Closes #13259 commit a5addbbb7d2f40a6403b7e78f241448783529d20 Author: наб Date: Tue Mar 15 01:00:03 2022 +0100 tests: rsend.kshlib: cmp_ds_prop: anonymise "inherited from" sourcces Fixes rsend_012_pos: 00:04:59.68 SUCCESS: cmp_ds_prop testpool/testfs/vol testpool2/testfs/vol nosource 00:04:59.70 4c4 00:04:59.70 < acltype posix inherited from testpool 00:04:59.70 --- 00:04:59.70 > acltype posix inherited from testpool2 00:04:59.70 11,12c11,12 00:04:59.70 < devices off inherited from testpool 00:04:59.70 < exec on inherited from testpool 00:04:59.70 --- 00:04:59.70 > devices off inherited from testpool2 00:04:59.70 > exec on inherited from testpool2 00:04:59.70 17c17 00:04:59.70 < setuid off inherited from testpool 00:04:59.70 --- 00:04:59.70 > setuid off inherited from testpool2 00:04:59.70 ERROR: cmp_ds_prop testpool@final testpool2@final exited 1 Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13250 Closes #13259 commit b2c5291b7e2b49e6b482c1be898e992bf0ccc37f Author: наб Date: Mon Mar 14 23:47:38 2022 +0100 tests: prune cat (ab)uses Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit f7cc8dddf76f164be221c39b7687b1cb99af55e5 Author: наб Date: Wed Mar 23 21:08:55 2022 +0100 tests: rsend_012_pos: backup/restore in one invocation Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 9da14f39810a5c4eae4ede8440a1e22cf119a949 Author: наб Date: Mon Mar 14 23:01:57 2022 +0100 tests: rsend.kshlib: cmp_ds_prop: allow skipping source This fixes rsend_012_pos: 20:28:50.50 SUCCESS: eval zfs receive -d -F testpool2 < /mnt/testroot/backdir-rsend/pool-final-R 20:28:50.53 4,6c4,6 20:28:50.53 < acltype off local 20:28:50.53 < dnodesize 4k local 20:28:50.53 < atime off local 20:28:50.53 --- 20:28:50.53 > acltype off received 20:28:50.53 > dnodesize 4k received 20:28:50.53 > atime off received 20:28:50.53 8,13c8,13 20:28:50.53 < checksum sha256 local 20:28:50.53 < compression off local 20:28:50.53 < copies 2 local 20:28:50.53 < devices on local 20:28:50.53 < exec on local 20:28:50.53 < quota 1G local 20:28:50.53 --- 20:28:50.53 > checksum sha256 received 20:28:50.53 > compression off received 20:28:50.53 > copies 2 received 20:28:50.53 > devices on received 20:28:50.53 > exec on received 20:28:50.53 > quota 1G received 20:28:50.53 15c15 20:28:50.53 < recordsize 128K local 20:28:50.53 --- 20:28:50.53 > recordsize 128K received 20:28:50.53 17,18c17,18 20:28:50.53 < setuid off local 20:28:50.53 < snapdir visible local 20:28:50.53 --- 20:28:50.53 > setuid off received 20:28:50.53 > snapdir visible received 20:28:50.53 ERROR: cmp_ds_prop testpool testpool2 exited 1 Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13250 Closes #13259 commit 91933eb9770066388047cb8f4b8a63ed6b1d2944 Author: наб Date: Mon Mar 14 22:50:56 2022 +0100 tests: rsend.kshlib: cmp_ds_prop: mask out differences in origin pool This fixes rsend_011_pos: 20:28:26.94 SUCCESS: cmp_ds_prop testpool/testfs/fs1/fs2 testpool2/testfs/fs1/fs2 20:28:26.96 2c2 20:28:26.96 < origin testpool@psnap - 20:28:26.96 --- 20:28:26.96 > origin testpool2@psnap - 20:28:26.97 ERROR: cmp_ds_prop testpool/pclone testpool2/pclone exited 1 Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13250 Closes #13259 commit 964d41806f0e84a9cefb0c01591c9b598deda780 Author: наб Date: Wed Mar 23 22:01:06 2022 +0100 tests: review all wc(1) invocations Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 23914a3b9146c05ddf985b134472b6c885cefdfe Author: наб Date: Wed Mar 23 01:52:39 2022 +0100 tests: review every instance of $? Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 6586085673db2e2d7e66652855d449b4abaab467 Author: наб Date: Wed Mar 23 17:54:07 2022 +0100 tests: include: math: simplify bc conditions, review $? Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit caccfc870fe17ce9bc040ed559771cd81e3c2a38 Author: наб Date: Mon Mar 14 01:41:03 2022 +0100 tests: clean out unused/single-use/useless commands from the list Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit df6e0f009265c92b1c6e30be66400f1e37b70f6f Author: наб Date: Sun Mar 20 02:22:43 2022 +0100 zed: functions: zed_log_err: forward to zed_log_msg Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit be866839f0144eba5bc1168c8b60c2cf809af239 Author: наб Date: Sun Mar 20 02:22:00 2022 +0100 zed: lets: all-debug: printenv -> env Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit ee798fee458cc16424e30508d5da4a8fd03233c6 Author: наб Date: Mon Mar 14 01:39:03 2022 +0100 scripts: zfs-tests.sh: cleanup, optimise The single-stage losetup doesn't work on busybox, but then most of the testsuite doesn't Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 0990228ab8477829c6bd9b1fd5cb60d740f61574 Author: наб Date: Tue Mar 22 22:26:00 2022 +0100 tests: mixed_create_failure: explicitly note the error Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Issue: #13215 Closes #13259 commit 50a33b8c696e400e33e190b25579fc5abad98b82 Author: наб Date: Sat Mar 12 17:52:38 2022 +0100 tests: logapi: don't cat excessively This also fixes line welding in test error output Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit e294acdba879bf86e7253beeaaa5f098b92157d7 Author: наб Date: Tue Mar 22 20:09:47 2022 +0100 tests: cmd: don't recurse This confers an >10x speedup on t/z-t/cmd builds (12s -> 1.1s), gets rid of 23 redundant identical automake specs and gitignores, and groups the binaries with their common headers Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit caeab993ec77ce15bcb25484017ff18f18a6550b Author: наб Date: Sat Mar 12 04:30:22 2022 +0100 tests: {read,write}_dos_attributes: despaghettify Also: actually accept all the flags in write_d_a Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit d30577c9dd811688f2609ad532b011b99bceb485 Author: наб Date: Sat Mar 12 00:26:46 2022 +0100 fgrep -> grep -F Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit f63c9dc70a40f930b09b586e6bc53cabf90fa3a1 Author: наб Date: Sat Mar 12 00:25:47 2022 +0100 egrep -> grep -E Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 270d0a5a16869e2becf301ae4548ae1cefeed2fb Author: наб Date: Sat Mar 12 00:13:19 2022 +0100 tests: nawk -> awk Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 75746e9a40e15e7994fb99799f0b02912155f179 Author: наб Date: Fri Mar 11 23:54:08 2022 +0100 tests: review every awk(1) invocation Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 72f3516094cd772053c4367cb8549fd697a6c107 Author: наб Date: Sat Mar 12 00:32:13 2022 +0100 tests: zfs_rollback_commit: talkative failures Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit d7976a073e65087148fa721887997cc612da9920 Author: наб Date: Thu Mar 10 19:40:35 2022 +0100 tests: README: note non-default mountd requirement Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit d5cc930b8fcf90851ce3fbfa7120445fc6f85c4b Author: наб Date: Wed Mar 9 14:17:11 2022 +0100 tests: README: note that -d *must* be 777 for the deleg tests Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 41ebf403754ee24e83d56be582b0209e386cad1c Author: наб Date: Wed Mar 9 23:54:26 2022 +0100 tests: vdev_zaps: cleanup library Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 25aeffadb2e0df2703e5b970f4059cde71adbd2d Author: наб Date: Wed Mar 9 23:54:07 2022 +0100 tests: vdev_zaps_007: actually test the new pool Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 592cf7f1e23280ab3f02b649fdcda75b72d16729 Author: наб Date: Wed Mar 9 13:52:14 2022 +0100 tests: get rid of which Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 62c5ccdf929fe3683625bf577175a1b59bc82f32 Author: наб Date: Wed Mar 9 13:39:34 2022 +0100 tests: don't >-redirect without eval Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 053dac9e7de190e3a991327b904c7d1103abb13c Author: наб Date: Wed Mar 9 12:48:23 2022 +0100 tests: vdev_zaps_007: log_must with > must eval Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit b9b763111c6e1ab4e58449057c94cf8ad777657f Author: наб Date: Wed Mar 9 00:56:32 2022 +0100 tests: redacted_send: explicitly assume instant /dev on non-Linux The users spew udevadm ENOENTs on FreeBSD Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 9423c932d49afc715f8422b94321fd656842fa41 Author: наб Date: Wed Mar 9 00:51:07 2022 +0100 tests: replace sum(1) with cksum(1) Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit c2fcc55d972fb49d24bfb626cc20b954040ecc5a Author: наб Date: Wed Mar 9 00:30:52 2022 +0100 tests: snapshot_00[15]_pos: use cksum instead of sum -r This only worked by accident on FreeBSD, where sum doesn't take flags POSIX guarantees us cksum (Ethernet CRC) ‒ use that instead Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 7aa7e6bd0ad71d75a800d7892bb3d055748c3680 Author: наб Date: Wed Mar 9 00:18:26 2022 +0100 tests: zfs_create_nomount: undefined local -> typeset Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit f806d678462ca5fa492f38c6913ff5f3a8e5dadb Author: наб Date: Wed Mar 9 00:16:55 2022 +0100 tests: mixed_create_failure: undefined log_err -> log_fail Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Issue #13215 Closes #13259 commit 2d9da5e1c88d46118b1c98bfa57bfa3c4f0f2a23 Author: наб Date: Tue Mar 8 13:36:03 2022 +0100 tests: don't fail if no fio or python3.sysctl Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit bd328a588b7a3fc053a256d39ac8145eb1dd80e5 Author: наб Date: Wed Mar 23 14:23:51 2022 +0100 tests: nonspecific cleanup Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 33c319eb1efe601e450b23763d7d23f70b19885b Author: наб Date: Wed Mar 23 14:23:51 2022 +0100 tests: zfs_share_concurrent_shares: don't use log_musts in subprocesses This thoroughly destroys logapi and races to the log files horribly Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 3886e7081ada7e4275ff20f7e42c19d3645af212 Author: наб Date: Fri Mar 11 01:42:32 2022 +0100 tests: zfs_unshare_006: log_unsupported iff usershares are actually off Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit a3dcc0aa0c0b7873265d0b014ea6133bd8e40591 Author: наб Date: Tue Mar 8 22:50:45 2022 +0100 zfs, zpool: safe_malloc() duplicate argv Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 84a3eab6aa44dedaec6d3f5e6da9ccb4b9942907 Author: наб Date: Tue Mar 8 22:08:55 2022 +0100 zfs: simplify usage_prop_cb values Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 5b0e75caef66cda8e5c14dcd9f7ab689a6d605ad Author: наб Date: Sun Mar 6 01:14:12 2022 +0100 tests: don't use share/unshare exportfs aliases, support FreeBSD NFS Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit c22c87203682bb983220cde0be4da0d5995fd502 Author: наб Date: Sat Mar 5 19:31:01 2022 +0100 tests: don't always skip zfs_unshare tests on FreeBSD Previously, they'd all be skipped on FreeBSD where share is showmount Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit 261f10b717e2e0ce78c8176fb1793aa712e16ead Author: наб Date: Fri Mar 4 00:12:39 2022 +0100 tests: zfs_unshare_001_pos: print which filesystem failed Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit bf228f3de038ddb1b8498119c95a3e1766ad884e Author: наб Date: Fri Mar 4 00:09:08 2022 +0100 tests: standardise on no-arg uname with *) case for illumos Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Reviewed-by: Ryan Moeller Signed-off-by: Ahelenia Ziemiańska Closes #13259 commit eebfd28e9d712bcfb5a6d20d59be208645474066 Author: Andrew Date: Fri Apr 1 11:53:54 2022 -0500 Linux optimize access checks when ACL is trivial Bypass check of ZFS aces if the ACL is trivial. When an ACL is trivial its permissions are represented by the mode without any loss of information. In this case, it is safe to convert the access request into equivalent mode and then pass desired mask and inode to generic_permission(). This has the added benefit of also checking whether entries in a POSIX ACL on the file grant the desired access. This commit also skips the ACL check on looking up the xattr dir since such restrictions don't exist in Linux kernel and it makes xattr lookup behavior inconsistent between SA and file-based xattrs. We also don't want to perform a POSIX ACL check while looking up the POSIX ACL if for some reason it is located in the xattr dir rather than an SA. Reviewed-by: Brian Behlendorf Co-authored-by: Ryan Moeller Signed-off-by: Andrew Walker Closes #13237 commit 6a2dda8f05d9fb3c5b7d81c8c6762cd43be07dd7 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Thu Mar 31 13:09:18 2022 -0400 Ask libtool to stop hiding some errors For #13083, curiously, it did not print the actual error, just that the compile failed with "Error 1". In theory, this flag should cause it to report errors twice sometimes. In practice, I'm pretty okay with reporting some twice if it avoids reporting some never. Reviewed-by: Brian Behlendorf Reviewed-by: Damian Szuberski Signed-off-by: Rich Ercolani Closes #13086 commit 4d04e41e4def222e454d05d77c3a93057c58f471 Author: hpingfs <101400146+hpingfs@users.noreply.github.com> Date: Thu Mar 31 06:40:31 2022 +0800 zfs_ctldir: fix incorrect argument type of rw_destroy The argument type of rw_destroy is (krwlock_t *) while currently krwlock_t is passed in zfs_ctldir.c. This error is hidden because rw_destroy is defined as ((void) 0) in linux. But anyway, this mismatch should be fixed. Reviewed-by: Brian Behlendorf Signed-off-by: Ping Huang Closes #13272 commit abdcef47d2ef9f38092bcfd3e4a8ec5499244826 Author: hpingfs <101400146+hpingfs@users.noreply.github.com> Date: Thu Mar 31 06:39:55 2022 +0800 zvol_os: suppress compiler warning for zvol_open_timeout_ms When HAVE_BLKDEV_GET_ERESTARTSYS is defined, compiler will complain "defined but not used" warning for zvol_open_timeout_ms. Reviewed-by: Brian Behlendorf Signed-off-by: Ping Huang Closes #13270 commit 7dc782e5c557cb0bbcd509cc92c22a3b5bf4c500 Author: наб Date: Thu Mar 31 00:37:28 2022 +0200 cstyle: remove unused -o Remove handling for allowing doxygen- and embedding in splint(?)-style comments. This functionality is unused by OpenZFS. Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13264 commit 3cfbeb4e906df75ac3377c0bf952f16c37ea93a4 Author: наб Date: Wed Mar 16 19:56:46 2022 +0100 zfs: main: don't NULL-check infallible safe_malloc() Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13229 commit 18dbf5c8c34b1a412480dcfed2bdbefbafd80d80 Author: наб Date: Wed Mar 16 19:51:28 2022 +0100 libzfs: don't NULL-check infallible allocations Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13229 commit bc3f12bfac152a0c28951cec92340ba14f9ccee9 Author: наб Date: Mon Mar 28 19:24:22 2022 +0200 config: user: check for And always zpool_read_label_slow() on non-conformant libcs Reviewed-by: Brian Behlendorf Co-authored-by: José Luis Salvador Rufo Signed-off-by: Ahelenia Ziemiańska Closes #13207 Closes #13254 commit b61595ff86fb708aa6e22818f36cea3d346f8241 Author: наб Date: Wed Mar 16 19:01:08 2022 +0100 man: zfs-rename.8: import examples from zfs.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 4d4fa5215dedaa0e6f53c61a288656362345fa79 Author: наб Date: Wed Mar 16 18:53:59 2022 +0100 man: zfs-snapshot.8: import examples from zfs.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit bc2fe49a4f7c8970d2a9175b45146a7368fcdfc2 Author: наб Date: Wed Mar 16 18:50:53 2022 +0100 man: zfs-promote.8: import examples from zfs.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit d842ca3ff274581682678bd8bd6dc3f709c2b460 Author: наб Date: Wed Mar 16 18:49:31 2022 +0100 man: zfs-create.8: import examples from zfs.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 3b887e485ccc70b34baf02bd229f1808aa01ef14 Author: наб Date: Wed Mar 16 18:48:01 2022 +0100 man: zfs-destroy.8: import examples from zfs.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 9bda775bd952b1863abda57429f9c2044ae078b9 Author: наб Date: Wed Mar 16 18:44:25 2022 +0100 man: zfs-rollback.8: import examples from zfs.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit c6fd08797f69b39fc06419f824d94f9663b8943d Author: наб Date: Wed Mar 16 18:43:33 2022 +0100 man: zfs-clone.8: import examples from zfs.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 52f3aebdb2b4b457812fbac572df67461d7162bf Author: наб Date: Wed Mar 16 18:33:41 2022 +0100 man: zfs-list.8: import examples from zfs.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 2057b47b91a4c7937209f23be85608848521f284 Author: наб Date: Wed Mar 16 18:31:42 2022 +0100 man: zfs-send.8, zfs-receive.8: import examples from zfs.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit b7b3f199351ab6f0fe26461be8f5d756c404249c Author: наб Date: Wed Mar 16 18:27:44 2022 +0100 man: zfs-diff.8: import examples from zfs.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 1e164f89e582b4697e4c83bf29fa75e187e423e5 Author: наб Date: Wed Mar 16 18:25:36 2022 +0100 man: zfs-bookmark.8: import examples from zfs.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit c522339fd7dfcb9c79abf97fa2016b0cd93c88ad Author: наб Date: Wed Mar 16 18:23:19 2022 +0100 man: zfs-set.8: import examples from zfs.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 575e20602c78006225cf30a101318b0434eeb0bd Author: наб Date: Wed Mar 16 18:20:22 2022 +0100 man: zfs-allow.8: import examples from zfs.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 69b06a77b6ed57b0a519ff3b312f6cf6a8c5cad7 Author: наб Date: Wed Mar 16 18:04:41 2022 +0100 man: zpool-iostat.8: import examples from zpool.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit b36069ab8862502760bc6edcd16c45bceb9aadc8 Author: наб Date: Wed Mar 16 18:04:32 2022 +0100 man: zpool-remove.8: import examples from zpool.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit d8dd89acd1b05788243037888cdd5091105927de Author: наб Date: Wed Mar 16 18:04:23 2022 +0100 man: zpool-upgrade.8: import examples from zpool.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 8ccb3fd6b1b68ea889a9d2181969a6c7d7eabf45 Author: наб Date: Wed Mar 16 18:03:56 2022 +0100 man: zpool-import.8: import examples from zpool.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 4849523af089b4533035696028d3d9789fc98cf6 Author: наб Date: Wed Mar 16 18:03:47 2022 +0100 man: zpool-export.8: import examples from zpool.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 4b660ffe7bf37d9df71295156e63054064126c4e Author: наб Date: Wed Mar 16 18:03:27 2022 +0100 man: zpool-destroy.8: import examples from zpool.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 74181ca44ca0dcf36bb3a151d76d5929515f1931 Author: наб Date: Wed Mar 16 18:03:13 2022 +0100 man: zpool-list.8: import examples from zpool.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit c34eb6f7a1ff9d1e0b22e937d629e8097d34f587 Author: наб Date: Wed Mar 16 18:02:42 2022 +0100 man: zpool-add.8: import examples from zpool.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 9cdb06753966f8419260114a081e8a083a680351 Author: наб Date: Wed Mar 16 17:47:06 2022 +0100 man: zpool-create.8: import examples from zpool.8 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 Closes #13140 commit a2550c9df9a9b1346f61fb1652c1c716fc492d44 Author: наб Date: Wed Mar 16 17:46:32 2022 +0100 man: Examples: use subsections instead of lists Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 1ebe4c482dbfecd0f4005f079d9557cb59f50f7b Author: наб Date: Wed Mar 16 17:37:18 2022 +0100 man: zpool.8: Examples: unmirrored -> non-redundant Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 6571873f1b3866f244f4f45b680dad6d6a73ee1b Author: наб Date: Wed Mar 16 17:36:03 2022 +0100 man: zpool.8: Examples: use Pa for disks and scripts Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13228 commit 7db823bd613507679aa2bf2829538210b25f8471 Author: наб Date: Mon Mar 28 19:03:13 2022 +0200 module: zfs: dsl_bookmark: silence false-positive maybe-uninitialised Reviewed-by: Brian Behlendorf Reviewed-by: Damian Szuberski Signed-off-by: Ahelenia Ziemiańska Closes #13247 Closes #13258 commit 063c83a853f51fe98e3fcda827929ade21d104cc Author: Jeremy Visser Date: Tue Mar 29 03:55:57 2022 +1100 zfs-dkms rpm: simplify scriptlets, fix uninstall Two problems led to unexpected behaviour of the scriptlets: 1) Newer DKMS versions change the formatting of "dkms status": (old) zfs, 2.1.2, 5.14.10-300.fc35.x86_64, x86_64: installed (new) zfs/2.1.2, 5.14.10-300.fc35.x86_64, x86_64: installed Which broke a conditional determining whether to uninstall. 2) zfs_config.h not packaged properly, but was attempted to be read in the %preun scriptlet: CONFIG_H="/var/lib/dkms/zfs/2.1.2/*/*/zfs_config.h" Which broke the uninstallation of the module, which left behind a dangling symlink, which broke DKMS entirely with this error: Error! Could not locate dkms.conf file. File: /var/lib/dkms/zfs/2.1.1/source/dkms.conf does not exist. This change attempts to simplify life by: * Avoiding parsing anything (less prone to future breakage) * Uses %posttrans instead of %post for module installation, because %post happens before %preun, while %posttrans happens afterwards * Unconditionally reinstall module on upgrade, which is less efficient but the trade-off is that it's more reliable Alternative approaches could involve fixing the existing parsing bugs or improving the logic, but this comes at the cost of complexity and possible future bugs. Reviewed-by: Tony Hutter Reviewed-by: Brian Behlendorf Signed-off-by: Jeremy Visser Closes #10463 Closes #13182 commit 86690775b0a6615003eebb73961400fa94f72a8b Author: наб Date: Thu Mar 24 16:33:40 2022 +0100 Linux 5.18 compat: replace genhd.h with blkdev.h includes blkdev.h includes genhd.h since dawn of upstream git, so this is globally safe Upstream-commit: 322cbb50de711814c42fb088f6d31901502c711a ("block: remove genhd.h") Reviewed-by: Tony Hutter Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13251 commit d1325b4fa2ddc42beb88d00bf85041507341fdd0 Author: наб Date: Thu Mar 24 15:22:53 2022 +0100 Linux 5.18 compat: 4-argument bio_alloc() bio_alloc(gfp_t gfp_mask, unsigned short nr_iovecs) became bio_alloc(struct block_device *bdev, unsigned short nr_vecs, unsigned int opf, gfp_t gfp_mask) passing NULL/0 continues previous behaviour Upstream-commit: 07888c665b405b1cd3577ddebfeb74f4717a84c4 ("block: pass a block_device and opf to bio_alloc") Reviewed-by: Tony Hutter Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13251 commit 427fad7f247d8ef0975e6038587d954bd771fa63 Author: George Melikov Date: Thu Mar 24 17:01:26 2022 +0300 CI: Log test name to /dev/kmsg in ZTS Reviewed-by: Brian Behlendorf Reviewed-by: John Kennedy Signed-off-by: George Melikov Closes #13249 commit 6322a77ce7e661facac3335721dde9eff9ceb634 Author: наб Date: Wed Mar 16 02:06:27 2022 +0100 linux: libzutil: zfs_path_order: don't strdup ZPOOL_IMPORT_PATH Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13223 commit 8a2ed86001583280b4d84d1f37919ea8a34ddb79 Author: наб Date: Tue Mar 15 23:23:53 2022 +0100 libzutil: zfs_resolve_shortname: don't strdup() ZPOOL_IMPORT_PATH Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13223 commit 018937884f93e0f4ae7f560c352a138ec6b69cb4 Author: George Melikov Date: Wed Mar 23 18:53:19 2022 +0300 zpoolconcepts.7: fix comma typo Reviewed-by: Brian Behlendorf Signed-off-by: George Melikov Closes #13239 commit 706ba5dc1d08044ef875ec5eb3882da34bf0265c Author: Brian Behlendorf Date: Wed Mar 23 08:52:30 2022 -0700 Linux 5.17 compat: META Update the META file to reflect compatibility with the 5.17 kernel. Reviewed-by: George Melikov Signed-off-by: Brian Behlendorf Closes #13243 commit 460748d4aeb1ed3c78efdd562fcf3d2eaa9ff536 Author: Brian Behlendorf Date: Wed Mar 23 08:51:00 2022 -0700 Switch from _Noreturn to __attribute__((noreturn)) Parts of the Linux kernel build system struggle with _Noreturn. This results in the following warnings when building on RHEL 8.5, and likely other environments. Switch to using the __attribute__((noreturn)). warning: objtool: dbuf_free_range()+0x2b8: return with modified stack frame warning: objtool: dbuf_free_range()+0x0: stack state mismatch: cfa1=7+40 cfa2=7+8 ... WARNING: EXPORT symbol "arc_buf_size" [zfs.ko] version generation failed, symbol will not be versioned. WARNING: EXPORT symbol "spa_open" [zfs.ko] version generation failed, symbol will not be versioned. ... Additionally, __thread_exit() has been renamed spl_thread_exit() and made a static inline function. This was needed because the kernel will generate a warning for symbols which are __attribute__((noreturn)) and then exported with EXPORT_SYMBOL. While we could continue to use _Noreturn in user space I've also switched it to __attribute__((noreturn)) purely for consistency throughout the code base. Reviewed-by: Ryan Moeller Reviewed-by: Brian Atkinson Signed-off-by: Brian Behlendorf Closes #13238 commit b73505c7e056417cba73dcf51b49062c84fd2b59 Author: Tony Hutter Date: Wed Mar 23 08:15:02 2022 -0700 ZTS: Log test name to /dev/kmsg on Linux Add a -K option to the test suite to log each test name to /dev/kmsg (on Linux), so if there's a kernel warning we'll be able to match it up to a particular test. Reviewed-by: John Kennedy Signed-off-by: Tony Hutter Closes #13227 commit 6b444cb9711a8c0a089612dc3ac67e45c4a4e108 Author: Brian Behlendorf Date: Sat Mar 19 12:46:33 2022 -0700 Linux 5.16 compat: restore FSR and FSAVE Commit 3b52ccd introduced a flaw where FSR and FSAVE are not restored when using a Linux 5.16 kernel. These instructions are only used when XSAVE is not supported by the processor meaning only some systems will encounter this issue. Reviewed-by: Tony Hutter Reviewed-by: Attila Fülöp Signed-off-by: Brian Behlendorf Closes #13210 Closes #13236 commit 565089f592426f4a8f0e3df1027f947700799fa6 Author: Sean Eric Fagan Date: Fri Mar 18 17:02:12 2022 -0700 Allow zfs send to exclude datasets Add support for a -exclude/-X option to `zfs send` to allow dataset hierarchies to be excluded. Snapshots can be excluded using a channel program; however, this can result in failures with 'zfs send -R'; this option allows them to be excluded. Fortunately, this required a change only to cmd/zfs/zfs_main.c, using the already-existing callback argument to zfs_send() that is currently unused. Reviewed-by: Paul Dagnelie Reviewed-by: Christian Schwarz Reviewed-by: Ahelenia Ziemiańska Reviewed-by: Brian Behlendorf Co-authored-by: Sean Eric Fagan Signed-off-by: Sean Eric Fagan Closes #13158 commit 3ce3d305324ba5ffddfcbc90a996a2b3a8424109 Author: ColMelvin Date: Fri Mar 18 18:54:50 2022 -0500 RPM: Split out pam_zfs_key into separate package Create a separate `pam_zfs_key` package for the PAM module components, an optional addition to the deliverables, in much the same way as the Python bindings are released as a separate `python#-pyzfs` package. This makes it clear when the PAM module is shipped with the package, since it's now in its own package. Reviewed-by: Brian Behlendorf Reviewed-by: Tony Hutter Signed-off-by: Chris Lindee Closes: #13026 commit 045aeabce6e5df9c88412c479ddd073087b2d83d Author: наб Date: Sat Mar 19 00:53:05 2022 +0100 module: zfs: arc: hdr_full_crypt_dest: drop unevaulated-only variable This explodes as -Wunused-variable on GCC 8.5.0, despite it being used, just not in an evaluated context Reviewed-by: Matthew Ahrens Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13195 commit 4629dfe963ac6f2691689a358ce2669c6421b547 Author: наб Date: Wed Mar 16 17:23:27 2022 +0100 config: always-arch: don't subst TARGET_CPU Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13225 commit bd0955cd8e005136ea2e58da58d39e4ff2f3b517 Author: наб Date: Wed Mar 16 17:19:56 2022 +0100 config: always-arch: prune unused TARGET_CPU_*, add TARGET_CPU catch-all This fixes (harmless) error spew from configuring on, e.g., armv6l Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13193 Closes #13225 commit 515710a1e1a11c7d1aec28adac0c486eae4519f1 Author: Tony Hutter Date: Fri Mar 18 14:06:40 2022 -0700 zed: Fix mpath autoreplace on Centos 7 A prior commit included a udev check for MPATH_DEVICE_READY to determine if a path was multipath when doing an autoreplace: f2f6c18 zed: Misc multipath autoreplace fixes However, MPATH_DEVICE_READY is not provided by the older version of udev that's on Centos 7 (it is on Centos 8). This patch instead looks for 'mpath-' in the UUID, which works on both Centos 7 and 8. Reviewed-by: Brian Behlendorf Signed-off-by: Tony Hutter Closes #13222 commit d42979c6ef1ec10b041c3394d969643f8862f7c3 Author: Ryan Moeller Date: Fri Mar 18 08:47:57 2022 -0400 Fix ACL checks for NFS kernel server This PR changes ZFS ACL checks to evaluate fsuid / fsgid rather than euid / egid to avoid accidentally granting elevated permissions to NFS clients. Reviewed-by: Serapheim Dimitropoulos Reviewed-by: Brian Behlendorf Co-authored-by: Andrew Walker Signed-off-by: Ryan Moeller Closes #13221 commit a5920d24c04b64a96b4bd6be43a591a29f278b16 Author: Mateusz Guzik Date: Thu Mar 17 18:30:10 2022 +0100 FreeBSD: add missing replay check to an assert in zfs_xvattr_set Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Mateusz Guzik Closes #13219 commit bee314a798b9495bfa8368952461301fb7080a61 Author: Kyle Evans Date: Thu Mar 17 12:14:00 2022 -0500 module: freebsd: avoid a taking a destroyed lock in zfs_zevent bits At shutdown time, we drain all of the zevents and set the ZEVENT_SHUTDOWN flag. On FreeBSD, we may end up calling zfs_zevent_destroy() after the zevent_lock has been destroyed while the sysevent thread is winding down; we observe ESHUTDOWN, then back out. Events have already been drained, so just inline the kmem_free call in sysevent_worker() to avoid the race, and document the assumption that zfs_zevent_destroy doesn't do anything else useful at that point. This fixes a panic that can occur at module unload time. Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Kyle Evans Closes #13220 commit 6ef00196db1cc6bd189eeb72df26d494a2aee889 Author: наб Date: Wed Mar 16 00:10:10 2022 +0100 module: zstd: check we don't leak symbols; regenerate symbol map Reviewed-by: Brian Behlendorf Co-authored-by: Rich Ercolani Signed-off-by: Ahelenia Ziemiańska Closes #12988 Closes #13209 commit 1c41d8941cb5a76d71930d2af976c376c05ed318 Author: наб Date: Thu Mar 10 20:14:28 2022 +0100 tests: validate getsubopt(3) expulsion Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit 77cdc63d0959971f7e13cb81dad348074b4c3ff6 Author: наб Date: Sun Jan 23 01:48:05 2022 +0100 zfs-list.8: mention -S after -s it calls back to Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit 40f09cb0f441ffcafb421e83bdc10ce939efe2d0 Author: наб Date: Sat Jan 22 23:44:33 2022 +0100 zfs: list: only accept whole type for -t, not tp[=whatever] Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit 539d16c35ebb7728c2121e53c895cac62889f44f Author: наб Date: Sun Jan 23 00:12:27 2022 +0100 libzfs: tokenise consistently with zfs and zpool Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit 46952300210647faeef391e2df2a59ec6a185591 Author: наб Date: Sat Jan 22 23:41:36 2022 +0100 zfs: get: only accept whole type for -t, not tp[=whatever] Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit 7867f430b4c15b94fccac9cd4829b811bbf7383b Author: наб Date: Sat Jan 22 23:33:44 2022 +0100 zfs: get: only accept whole source for -s, not src[=whatever] Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit 7c17e82cbe42d01511e86465cb9bc20d56f03325 Author: наб Date: Sat Jan 22 23:24:14 2022 +0100 zfs: get: only accept whole column for -o, not col[=whatever] Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit c79787845dfc9386d89e821963506d9c90c36677 Author: наб Date: Sat Jan 22 23:17:49 2022 +0100 zpool: get: there's one fewer column than in zfs get The current code allows -o name,property,value,source,name Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit 15aca3ad5976a426b416645e6274a22bdd94c756 Author: наб Date: Sat Jan 22 23:12:54 2022 +0100 zfs: wait: only accept whole activity for -t, not act[=whatever] Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit 66cd170db332f914d81c94c3515f3c0481bc34ba Author: наб Date: Sat Jan 22 14:24:57 2022 +0100 zpool: get: only accept whole columns for -o, not col[=whatever] Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit 675508f608fbf10297b0060d4ee5e88d86facb90 Author: наб Date: Sat Jan 22 00:38:24 2022 +0100 zpool: wait: only accept whole columns for -t, not col[=whatever] Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit c21819026f199869f515e9b8b74ec5cf43a324f1 Author: наб Date: Sat Jan 22 03:41:47 2022 +0100 Replace FMD_B_{TRUE,FALSE} with B_{TRUE,FALSE} Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit a9e2b22efb5f028d4b662bc0f6dc84e5461d48ee Author: наб Date: Sat Jan 22 02:39:09 2022 +0100 Integrate carcass of libspl/i/s/vtoc.h into i/s/efi_partition.h Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit d465fc584425c618f8baa13a31b70ffabe053484 Author: наб Date: Sat Jan 22 01:56:46 2022 +0100 Forbid b{copy,zero,cmp}(). Don't include for Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit 861166b02701dfc8f63a105bd32758e806c84fd7 Author: наб Date: Fri Feb 25 14:26:54 2022 +0100 Remove bcopy(), bzero(), bcmp() bcopy() has a confusing argument order and is actually a move, not a copy; they're all deprecated since POSIX.1-2001 and removed in -2008, and we shim them out to mem*() on Linux anyway Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit 1d77d62f5a77cab85d4b98ecf72a9838f70d6bf1 Author: наб Date: Fri Jan 21 21:28:19 2022 +0100 libspl: include: sys/vtoc.h: reduce to absolute barest minimum Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit 26bbce81739dd24f31d1f23ae352c7b7b108737a Author: наб Date: Wed Feb 23 02:44:53 2022 +0100 tests: replace explicit $? || log_fail with log_must Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit 85c2cce51cce1f5f01b5b3a50f5997ce0b24a189 Author: наб Date: Wed Feb 23 02:39:04 2022 +0100 tests: zfs_002_pos: simplify ZFS_ABORT tests Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit e09762c6c20f866502767fc113d65ab5efcdf9b6 Author: наб Date: Sun Feb 20 02:08:59 2022 +0100 tests: zfs_set_common: check_prop_inherit: print faulty values Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12996 commit 2feba9a6d8829605cbfe8f1f2ef56ddfdfb8d4ea Author: Brian Behlendorf Date: Mon Mar 14 16:39:07 2022 -0700 Linux 5.16 compat: META Update the META file to reflect compatibility with the 5.16 kernel. Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13212 Closes #13218 commit 8bb9ecf4bf927980ff569917e137d37fe62d1f27 Author: Ryan Moeller Date: Mon Mar 14 18:44:56 2022 -0400 libzfs: Convert to fnvpair functions Improves readability. No functional change intended. Reviewed-by: Brian Behlendorf Signed-off-by: Ryan Moeller Closes #13197 commit becc717f6143caf80bad6879b6c6402538ecae89 Author: Brian Atkinson Date: Mon Mar 14 13:37:39 2022 -0600 Adding ZERO_PAGE detection On some architectures ZERO_PAGE is unavailable because it references a GPL exported symbol of empty_zero_page. Originally e08b993 removed the call to PAGE_ZERO(0) for assignment to the abd_zero_page. However, a simple check can be done to avoid a kernel allocation and free for the abd_zero_page if ZERO_PAGE is available. Reviewed-by: Brian Behlendorf Signed-off-by: Brian Atkinson Closes #13199 commit dad2b19fffda43894a005a042055e268fe6b32e3 Author: наб Date: Sun Mar 13 21:18:17 2022 +0100 module: zfs: zio_inject: zio_match_handler: don't << -1 Caught by UBSAN: ZI_NO_DVA is passed explicitly in zio_handle_decrypt_injection() and can be an ENOENT from zio_match_dva() Reviewed-by: Brian Behlendorf Reviewed-by: Damian Szuberski Signed-off-by: Ahelenia Ziemiańska Closes #13146 Closes #13190 commit 56a0699e5e06242a7a134ad080aadfa13486bb59 Author: Brian Behlendorf Date: Sun Mar 13 13:16:49 2022 -0700 ZTS: Fix send_partial_dataset.ksh The send_partial_dataset test verifies that partial send streams can be resumed. This test may occasionally fail with a "token is corrupt" error if the `mess_send_file` truncates a send stream below the size of the DRR_BEGIN record. Update this function to set a minimum size to ensure there is at least an intact DDR_BEGIN record which allows for the receiving dataset to be created. Reviewed-by: George Melikov Reviewed-by: Rich Ercolani Signed-off-by: Brian Behlendorf Closes #13177 commit ebcf12f763130d17daeac7f0065adc924eff3847 Author: Harry Sintonen Date: Sun Mar 13 22:15:40 2022 +0200 get_key_material_https: removed bogus free() call The get_key_material_https() function error code path had a bogus free() call, either resulting in double-free or free() of undefined pointer. Reviewed-by: Brian Behlendorf Reviewed-by: Ahelenia Ziemiańska Co-authored-by: Harry Sintonen Signed-off-by: Harry Sintonen Closes #13198 commit 76bcffb7dc7ef620d186caacab266532fb293fde Author: Ryan Moeller Date: Fri Mar 11 11:52:49 2022 -0500 libzfs: FreeBSD doesn't resize partitions for you This code can be failure prone on FreeBSD, where zfsd will pass a guid as the vdev path to online. The guid causes zfs_resolve_shortname to fail because it expects a path. We can just skip the whole ordeal. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Ryan Moeller Closes #12083 commit 1282274f3355bcd11bdae911d9279ef1892f7061 Author: Akash B Date: Wed Mar 9 05:50:41 2022 +0530 Add physical device size to SIZE column in 'zpool list -v' Add physical device size/capacity only for physical devices in 'zpool list -v' instead of displaying "-" in the SIZE column. This would make it easier to see the individual device capacity and to determine which spares are large enough to replace which devices. Reviewed-by: Brian Behlendorf Reviewed-by: Tony Hutter Reviewed-by: Dipak Ghosh Signed-off-by: Akash B Closes #12561 Closes #13106 commit ce7a5dbf4b37a0ba28dd6911e1a17f039fdd4f4e Author: Attila Fülöp Date: Wed Mar 9 01:19:15 2022 +0100 Linux x86 SIMD: factor out unneeded kernel dependencies Cleanup the kernel SIMD code by removing kernel dependencies. - Replace XSTATE_XSAVE with our own XSAVE implementation for all kernels not exporting kernel_fpu{begin,end}(), see #13059 - Replace union fpregs_state by a uint8_t * buffer and get the size of the buffer from the hardware via the CPUID instruction - Replace kernels xgetbv() by our own implementation which was already there for userspace. Reviewed-by: Tony Hutter Reviewed-by: Brian Behlendorf Signed-off-by: Attila Fülöp Closes #13102 commit a86e089415679cf1b98eb424a159bb36aa2c19e3 Author: наб Date: Mon Feb 21 19:51:48 2022 +0100 libzfs_core: lzc_send_wraper: maximise pipe buffer for existing pipes This reduces scheduler overhead by letting the reader consume bigger chunks (64k => 128k at full throttle) Reviewed-by: Brian Behlendorf Reviewed-by: Paul Dagnelie Reviewed-by: Rich Ercolani Signed-off-by: Ahelenia Ziemiańska Closes #13133 commit 7eb179be18af2d693e087770f0d960092b83e8ab Author: Rich Ercolani Date: Thu Dec 16 23:19:22 2021 -0500 ZTS: /dev/null: accept no substitutes Instead of writing to "devnull" and rming it later, just > /dev/null to not have to cleanup later. Reviewed-by: Brian Behlendorf Reviewed-by: Paul Dagnelie Co-authored-by: Rich Ercolani Signed-off-by: Ahelenia Ziemiańska Closes #13133 commit dd899641eeca595de07c0ae4a8021fa1ca2a9703 Author: наб Date: Mon Feb 21 02:43:49 2022 +0100 Revert "ZTS: Avoid piping send directly to /dev/null" This reverts commit 1a79f7e86021c5de33d3518dd9a0f14f924ee345. Reviewed-by: Brian Behlendorf Reviewed-by: Paul Dagnelie Reviewed-by: Rich Ercolani Signed-off-by: Ahelenia Ziemiańska Closes #13133 commit d1a5e9594cb119a4741c5a7ba13abfe59e0575e1 Author: наб Date: Sun Feb 20 14:41:10 2022 +0100 Revert "Added error for writing to /dev/ on Linux" This reverts commit 860051f1d1ef7ee995188b852d8da36bce85b1dc. Reviewed-by: Brian Behlendorf Reviewed-by: Paul Dagnelie Reviewed-by: Rich Ercolani Signed-off-by: Ahelenia Ziemiańska Closes #13133 commit 3a909fe33efa17f09f83da25442b6a4b2aa2b27d Author: наб Date: Mon Feb 21 02:31:32 2022 +0100 libzfs, libzfs_core: send: always write to pipe By introducing lzc_send_wrapper() and routing all ZFS_IOC_SEND* users through it, we fix a Linux 5.10-introduced bug (see comment) This is all /transparent/ to the users API, ABI, and usage-wise, and disabled on FreeBSD and if the output is already a pipe, and transparently nestable (i.e. zfs_send_one() is wrapped, but so is lzc_send_redacted() it calls to ‒ this wouldn't be strictly necessary if ZFS_IOC_SEND_PROGRESS wasn't strictly denominational w.r.t. the descriptor the send is happening on) Reviewed-by: Brian Behlendorf Reviewed-by: Paul Dagnelie Co-authored-by: Rich Ercolani Signed-off-by: Ahelenia Ziemiańska Closes #11445 Closes #13133 commit fbbea09db965d5ed7fe554db0fe33e80b98dadd5 Author: наб Date: Sun Feb 20 20:17:22 2022 +0100 libzfs_core: simplify max_pipe_buffer(), return new max We do still check and only optionally set because all F_SETPIPE_SZ directives reallocate the pipe buffer Reviewed-by: Brian Behlendorf Reviewed-by: Rich Ercolani Reviewed-by: Paul Dagnelie Signed-off-by: Ahelenia Ziemiańska Closes #13133 commit 8a3d77358aa8a6ad327b0837a3e6bd269bee6fb4 Author: наб Date: Sun Feb 20 15:04:49 2022 +0100 libzfs: migrate single-use libzfs_set_pipe_max() to libzfs_core Notably, this also means that the pipe is expanded before each dataset is received, so updates to /p/s/f/pipe-max-size are reflected for each new dataset Reviewed-by: Brian Behlendorf Reviewed-by: Rich Ercolani Reviewed-by: Paul Dagnelie Signed-off-by: Ahelenia Ziemiańska Closes #13133 commit 08eb2309ce256d555a3a46d3887ca3d9bb9ead92 Author: Brian Atkinson Date: Mon Mar 7 16:22:01 2022 -0700 Cleaning up a couple of ZTS tests setup scripts With the zfs_destroy ZTS test case the setup script needed to call default_setup_noexit so compression could be turned off. Also, added log_must to setting compression off in the reservation setup script for turning off compression. Reviewed-by: Brian Behlendorf Reviewed-by: Rich Ercolani Signed-off-by: Brian Atkinson Closes #13173 commit 1b609d4b035930a044bf9133b4c9dcf172b3c101 Author: Brian Atkinson Date: Thu Mar 3 18:18:07 2022 -0700 Added noexit variant for Raidz setup in ZTS tests The regular default_raidz_setup function in the ZFS test suite called log_pass after creating the zpool. However, with compression now being on by default 56fa4aa, there is no way to turn compression off in the setup.ksh scripts when creating a raidz VDEV. The addition of the function default_raidz_setup_noexit allows for a raidz VDEV to be created, additional zfs property settings to be applied and for the setup.ksh script itself to call log_pass. With the addition of default_raidz_setup_noexit some stray log_pass calls were removed from any setup.ksh scripts that call default_raidz_setup. Reviewed-by: Brian Behlendorf Reviewed-by: Rich Ercolani Signed-off-by: Brian Atkinson Closes #13173 commit 6df43169b30985ce7d9df11d25093dec26829d2c Author: Brian Behlendorf Date: Tue Mar 8 09:16:35 2022 -0800 Fix ENOSPC when unlinking multiple files from full pool When unlinking multiple files from a pool at 100% capacity, it was possible for ENOSPC to be returned after the first unlink. e.g. rm -f /mnt/fs/test1.0.0 /mnt/fs/test1.1.0 /mnt/fs/test1.2.0 rm: cannot remove '/mnt/fs/test1.1.0': No space left on device rm: cannot remove '/mnt/fs/test1.2.0': No space left on device After waiting for the pending deferred frees from the first unlink to be processed the remaining files can then be unlinked. This is caused by the quota limit in dsl_dir_tempreserve_impl() being temporarily decreased to the allocatable pool capacity less any deferred free space. This is resolved using the existing mechanism of returning ERESTART when over quota as long as we know enough space will shortly be available after processing the pending deferred frees. Reviewed-by: Alexander Motin Reviewed-by: Ryan Moeller Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13172 commit 39a4daf742a4bd34b34f85e004895385e4c46c1a Author: Umer Saleem Date: Tue Mar 8 06:52:03 2022 +0500 Expose additional file level attributes ZFS allows to update and retrieve additional file level attributes for FreeBSD. This commit allows additional file level attributes to be updated and retrieved for Linux. These include the flags stored in the upper half of z_pflags only. Two new IOCTLs have been added for this purpose. ZFS_IOC_GETDOSFLAGS can be used to retrieve the attributes, while ZFS_IOC_SETDOSFLAGS can be used to update the attributes. Attributes that are allowed to be updated include ZFS_IMMUTABLE, ZFS_APPENDONLY, ZFS_NOUNLINK, ZFS_ARCHIVE, ZFS_NODUMP, ZFS_SYSTEM, ZFS_HIDDEN, ZFS_READONLY, ZFS_REPARSE, ZFS_OFFLINE and ZFS_SPARSE. Flags can be or'd together while calling ZFS_IOC_SETDOSFLAGS. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: Umer Saleem Closes #13118 commit 9955b9ba2efafeeb88ccb4c4ab5707ef2974fb47 Author: Windel Bouwman Date: Tue Mar 8 02:49:34 2022 +0100 Handle aarch64 defines seperate from arm aarch64 is a different architecture than arm. Some compilers might choke when both __arm__ and __aarch64__ are defined. This change separates the checks for arm and for aarch64 in the isa_defs.h header files. Reviewed-by: Brian Behlendorf Signed-off-by: Windel Bouwman Closes #10335 Closes #13151 commit db7f1a91def6bbaf72dd3e9ad31255efb0bf81ab Author: Alejandro Colomar Date: Sat Mar 5 01:25:22 2022 +0100 Use _Noreturn (C11; GNU89) properly A function that returns with no value is a different thing from a function that doesn't return at all. Those are two orthogonal concepts, commonly confused. pthread_create(3) expects a pointer to a start routine that has a very precise prototype: void *(*start_routine)(void *); However, other thread functions, such as kernel ones, expect: void (*start_routine)(void *); Providing a different one is incorrect, and has only been working because the ABIs happen to produce a compatible function. We should use '_Noreturn void', since it's the natural type, and then provide a '_Noreturn void *' wrapper for pthread functions. For consistency, replace most cases of __NORETURN or __attribute__((noreturn)) by _Noreturn. _Noreturn is understood by -std=gnu89, so it should be safe to use everywhere. Ref: https://github.com/openzfs/zfs/pull/13110#discussion_r808450136 Ref: https://software.codidact.com/posts/285972 Reviewed-by: Brian Behlendorf Co-authored-by: Ahelenia Ziemiańska Signed-off-by: Alejandro Colomar Closes #13120 commit 06b805067833902613de1871fa01e29fc80c8247 Author: наб Date: Sun Feb 20 04:06:02 2022 +0100 zpool: main: list: don't pay for printf Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13125 commit cdb3eb5aa4d396e94dd985244a62c80bac689cfe Author: наб Date: Sun Feb 20 04:32:07 2022 +0100 zpool: main: list: -v: update dash spacing for special vdevs Before: nabijaczleweli@tarta:~/store/code/zfs$ /sbin/zpool list -v file NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT file 118G 150K 118G - - 0% 0% 1.00x ONLINE - /mnt/filling/store/nabijaczleweli/code/zfs/file 39.5G 0 39.5G - - 0% 0.00% - ONLINE dedup - - - - - - - - - /mnt/filling/store/nabijaczleweli/code/zfs/adedup 39.5G 0 39.5G - - 0% 0.00% - ONLINE special - - - - - - - - - /mnt/filling/store/nabijaczleweli/code/zfs/aspecial 39.5G 150K 39.5G - - 0% 0.00% - ONLINE logs - - - - - - - - - /mnt/filling/store/nabijaczleweli/code/zfs/alog 39.5G 0 39.5G - - 0% 0.00% - ONLINE cache - - - - - - - - - /mnt/filling/store/nabijaczleweli/code/zfs/acache 40.0G 1.50K 40.0G - - 0% 0.00% - ONLINE spare - - - - - - - - - /mnt/filling/store/nabijaczleweli/code/zfs/aspare - - - - - - - - AVAIL After: nabijaczleweli@tarta:~/store/code/zfs$ cmd/zpool/zpool list -v file NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT file 118G 150K 118G - - 0% 0% 1.00x ONLINE - /mnt/filling/store/nabijaczleweli/code/zfs/file 39.5G 0 39.5G - - 0% 0.00% - ONLINE dedup - - - - - - - - - /mnt/filling/store/nabijaczleweli/code/zfs/adedup 39.5G 0 39.5G - - 0% 0.00% - ONLINE special - - - - - - - - - /mnt/filling/store/nabijaczleweli/code/zfs/aspecial 39.5G 150K 39.5G - - 0% 0.00% - ONLINE logs - - - - - - - - - /mnt/filling/store/nabijaczleweli/code/zfs/alog 39.5G 0 39.5G - - 0% 0.00% - ONLINE cache - - - - - - - - - /mnt/filling/store/nabijaczleweli/code/zfs/acache 40.0G 1.50K 40.0G - - 0% 0.00% - ONLINE spare - - - - - - - - - /mnt/filling/store/nabijaczleweli/code/zfs/aspare - - - - - - - - AVAIL Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13123 Closes #13125 commit cf5a9d8c397f8394405c47e2909429eded4c78a7 Author: наб Date: Sun Feb 20 04:16:21 2022 +0100 zpool: main: use ARRAY_SIZE(class_name) instead of 3 Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13125 commit f4826a2d4676e0b4133e7bd05ee051b7842d6478 Author: наб Date: Sun Feb 20 04:06:43 2022 +0100 libzfs: util: don't check for allocation errors from infallible zfs_*() Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13125 commit be8e1d81bf2b85f6f76851b7013ac54e9bf488f0 Author: наб Date: Sun Feb 20 03:07:25 2022 +0100 Flex non-pretty-printed properties and raw-/pretty-print remaining ones Before: nabijaczleweli@tarta:~/store/code/zfs$ /sbin/zpool list -Td -o name,size,alloc,free,ckpoint,expandsz,guid,load_guid,frag,cap,dedup,health,altroot,guid,dedupditto,load_guid,maxblocksize,maxdnodesize 2>/dev/null Sun 20 Feb 03:57:44 CET 2022 NAME SIZE ALLOC FREE CKPOINT EXPANDSZ GUID LOAD_GUID FRAG CAP DEDUP HEALTH ALTROOT GUID DEDUPDITTO LOAD_GUID MAXBLOCKSIZE MAXDNODESIZE filling 25.5T 6.52T 18.9T - 64M 11512889483096932869 11656109927366648364 1% 25% 1.00x ONLINE - 11512889483096932869 0 11656109927366648364 1048576 16384 tarta-boot 240M 50.6M 189M - - 2372068846917849656 7752280792179633787 12% 21% 1.00x ONLINE - 2372068846917849656 0 7752280792179633787 1048576 512 tarta-zoot 55.5G 6.42G 49.1G - - 12971868889665384604 8622632123393589527 17% 11% 1.00x ONLINE - 12971868889665384604 0 8622632123393589527 1048576 16384 nabijaczleweli@tarta:~/store/code/zfs$ /sbin/zfs list -o name,guid,keyguid,ivsetguid,createtxg,objsetid,pbkdf2iters,refratio -r tarta-zoot NAME GUID KEYGUID IVSETGUID CREATETXG OBJSETID PBKDF2ITERS REFRATIO tarta-zoot 1110930838977259561 659P - 1 54 0 1.03x tarta-zoot/PAGEFILE.SYS 2202570496672997800 3.20E - 2163 1539 0 1.07x tarta-zoot/dupa 16941280502417785695 9.81E - 2274707 1322 1000000000000 1.00x tarta-zoot/etc 17029963068508333530 12.9E - 3663 1087 0 1.52x tarta-zoot/home 3508163802370032575 8.50E - 3664 294 0 1.00x tarta-zoot/home/misio 7283672744014848555 13.0E - 3665 302 0 2.28x tarta-zoot/home/nabijaczleweli 12286744508078616303 5.15E - 3666 200 0 2.05x tarta-zoot/home/nabijaczleweli/tftp 13551632689932817643 5.16E - 3667 1095 0 1.00x tarta-zoot/home/root 5203106193060067946 15.4E - 3668 698 0 2.86x tarta-zoot/home/shared-config 8866040021005142194 14.5E - 3670 2069 0 1.20x tarta-zoot/home/tymek 9472751824283011822 4.56E - 3671 1202 0 1.32x tarta-zoot/oldboot 10460192444135730377 13.8E - 2268398 1232 0 1.01x tarta-zoot/opt 9945621324983170410 5.84E - 3672 1210 0 1.00x tarta-zoot/opt/icecc 13178238931846132425 9.04E - 3673 1103 0 2.83x tarta-zoot/opt/swtpm 10172962421514870859 4.13E - 825669 145132 0 1.87x tarta-zoot/srv 217179989022738337 3.90E - 3674 2469 0 1.00x tarta-zoot/usr 12214213243060765090 15.0E - 3675 2477 0 2.58x tarta-zoot/usr/local 7542700368693813134 941P - 3676 2484 0 2.33x tarta-zoot/var 13414177124447929530 10.2E - 3677 2492 0 1.57x tarta-zoot/var/lib 6969944550407159241 5.28E - 3678 2499 0 2.34x tarta-zoot/var/tmp 6399468088048343912 1.34E - 3679 1218 0 3.95x After: nabijaczleweli@tarta:~/store/code/zfs$ cmd/zpool/zpool list -Td -o name,size,alloc,free,ckpoint,expandsz,guid,load_guid,frag,cap,dedup,health,altroot,guid,dedupditto,load_guid,maxblocksize,maxdnodesize 2>/dev/null Sun 20 Feb 03:57:42 CET 2022 NAME SIZE ALLOC FREE CKPOINT EXPANDSZ GUID LOAD_GUID FRAG CAP DEDUP HEALTH ALTROOT GUID DEDUPDITTO LOAD_GUID MAXBLOCKSIZE MAXDNODESIZE filling 25.5T 6.52T 18.9T - 64M 11512889483096932869 11656109927366648364 1% 25% 1.00x ONLINE - 11512889483096932869 0 11656109927366648364 1M 16K tarta-boot 240M 50.6M 189M - - 2372068846917849656 7752280792179633787 12% 21% 1.00x ONLINE - 2372068846917849656 0 7752280792179633787 1M 512 tarta-zoot 55.5G 6.42G 49.1G - - 12971868889665384604 8622632123393589527 17% 11% 1.00x ONLINE - 12971868889665384604 0 8622632123393589527 1M 16K nabijaczleweli@tarta:~/store/code/zfs$ cmd/zfs/zfs list -o name,guid,keyguid,ivsetguid,createtxg,objsetid,pbkdf2iters,refratio -r tarta-zoot NAME GUID KEYGUID IVSETGUID CREATETXG OBJSETID PBKDF2ITERS REFRATIO tarta-zoot 1110930838977259561 741529699813639505 - 1 54 0 1.03x tarta-zoot/PAGEFILE.SYS 2202570496672997800 3689529982640017884 - 2163 1539 0 1.07x tarta-zoot/dupa 16941280502417785695 11312442953423259518 - 2274707 1322 1000000000000 1.00x tarta-zoot/etc 17029963068508333530 14852574366795347233 - 3663 1087 0 1.52x tarta-zoot/home 3508163802370032575 9802810070759776956 - 3664 294 0 1.00x tarta-zoot/home/misio 7283672744014848555 14983161489316798151 - 3665 302 0 2.28x tarta-zoot/home/nabijaczleweli 12286744508078616303 5937870537299886218 - 3666 200 0 2.05x tarta-zoot/home/nabijaczleweli/tftp 13551632689932817643 5950522828900813054 - 3667 1095 0 1.00x tarta-zoot/home/root 5203106193060067946 17718025091255443518 - 3668 698 0 2.86x tarta-zoot/home/shared-config 8866040021005142194 16716354482778968577 - 3670 2069 0 1.20x tarta-zoot/home/tymek 9472751824283011822 5251854710505749954 - 3671 1202 0 1.32x tarta-zoot/oldboot 10460192444135730377 15894065034622168157 - 2268398 1232 0 1.01x tarta-zoot/opt 9945621324983170410 6737735639539098405 - 3672 1210 0 1.00x tarta-zoot/opt/icecc 13178238931846132425 10425145983015238428 - 3673 1103 0 2.83x tarta-zoot/opt/swtpm 10172962421514870859 4764783754852521469 - 825669 145132 0 1.87x tarta-zoot/srv 217179989022738337 4492810461439647259 - 3674 2469 0 1.00x tarta-zoot/usr 12214213243060765090 17306702395865262834 - 3675 2477 0 2.58x tarta-zoot/usr/local 7542700368693813134 1059954157997659784 - 3676 2484 0 2.33x tarta-zoot/var 13414177124447929530 11764397504176937123 - 3677 2492 0 1.57x tarta-zoot/var/lib 6969944550407159241 6084753728494937404 - 3678 2499 0 2.34x tarta-zoot/var/tmp 6399468088048343912 1548692824635344277 - 3679 1218 0 3.95x Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13122 Closes #13125 commit a9a89755faa7cff2a7e7779986ac18fcfabfd3bd Author: наб Date: Sun Feb 20 02:58:09 2022 +0100 module: zcommon: zprop: common: zprop_width: namespace exceptions Before this, /all/ numerical properties 1 (ZFS_PROP_CREATION, ZPOOL_PROP_SIZE, VDEV_PROP_CAPACITY) would be non-fixed and /all/ numerical properties 5 (ZFS_PROP_COMPRESSRATIO, ZPOOL_PROP_HEALTH, VDEV_PROP_PSIZE) would be 8-wide Realistically, this doesn't appear to be much of a problem Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13125 commit 6d8a00ff1f6ebca32cd7fa5e706e904871872144 Author: наб Date: Tue Feb 15 18:17:02 2022 +0100 contrib/dracut: README: note rootfstype=zfs Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13093 commit 6a41310c7099ca4532f2d8134bba37261f72410e Author: наб Date: Sat Feb 12 14:08:02 2022 +0100 contrib/dracut: zfs-lib: export_all: replace with inline zpool export -a 07a3312f170ac56cb480b0df9fdf4c83f116b59b, which introduced this in October of 2014, didn't have zpool export -a available; we do Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13093 commit 93669c381249e0d42bb3eb21336861c1a759b6a6 Author: наб Date: Sat Feb 12 14:05:29 2022 +0100 contrib/dracut: export-zfs: simplify Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13093 commit 497bf14a4a10666adc760513cb119db4402c30df Author: наб Date: Tue Feb 15 18:34:25 2022 +0100 zdb.8: cleanup Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13093 commit 56fa4aa96eb3875f254e93eaef646ea20ba187f9 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Thu Mar 3 13:43:38 2022 -0500 Default to ON for compression A simple change, but so many tests break with it, and those are the majority of this. Reviewed-by: George Melikov Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #13078 commit 29a0ffe7951367faeea58e348ee93e037acf56f8 Author: Brian Behlendorf Date: Wed Mar 2 11:03:53 2022 -0800 ZTS: Fix import_devices_missing.ksh Related to commit 90b77a036. Retry the `zpool export` if the pool is "busy" indicating there is a process accessing the mount point. This can happen after an import, allowing it to be retried will avoid spurious test failures. Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #13169 commit fe2ea67ddd6359253a772c68e4bebafbdd2505da Author: Rich Ercolani Date: Tue Mar 1 08:36:32 2022 -0500 Re-apply 6ba2e72b, silence lint Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #12978 commit e2206359955de08a43373eff761af278bfd96d3f Author: Rich Ercolani Date: Tue Mar 1 08:34:29 2022 -0500 Re-apply a78f19d3 Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #12978 commit 234e9605c1f2cdf64f13df0697aa5017f416656a Author: Rich Ercolani Date: Tue Mar 1 08:32:52 2022 -0500 Explode zstd 1.4.5 into separate upstream files It's much nicer to import from upstream this way, and compiles faster too. Everything in lib/ is unmodified 1.4.5. Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #12978 commit 669683c4cbcd9c20258ce6641c6c777f29aeb94d Author: Aleksa Sarai Date: Wed Mar 2 05:05:32 2022 +1100 ZTS: switch to rsync for directory diffs While "diff -r" is the most straightforward way of comparing directory trees for differences, it has two major issues: * File metadata is not compared, which means that subtle bugs may be missed even if a test is written that exercises the buggy behaviour. * diff(1) doesn't know how to compare special files -- it assumes they are always different, which means that a test using diff(1) on special files will always fail (resulting in such tests not being added). rsync can be used in a very similar manner to diff (with the -ni flags), but has the additional benefit of being able to detect and resolve many more differences between directory trees. In addition, rsync has a standard set of features and flags while diffs feature set depends on whether you're using GNU or BSD binutils. Note that for several of the test cases we expect that file timestamps will not match. For example, the ctime for a file creation or modify event is stored in the intent log but not the mtime. Thus when replaying the log the correct ctime is set but the current mtime is used. This is the expected behavior, so to prevent these tests from failing, there's a replay_directory_diff function which ignores those kinds of changes. Reviewed-by: Brian Behlendorf Signed-off-by: Aleksa Sarai Closes #12588 commit 19229e5f1ee3b7ff4709e63f64c559518f2fd82e Author: Brian Behlendorf Date: Tue Mar 1 08:47:30 2022 -0800 ZTS: Modify receive-o-x_props_override.ksh exception As previously noted in #12272 the receive-o-x_props_override.ksh test reliably fails on FreeBSD. Since we don't expect this test to pass move the exception from the "maybe" to "known" section. This way we don't retry the FAILED test when it is not expected to pass. Reviewed-by: George Melikov Reviewed-by: Ryan Moeller Signed-off-by: Brian Behlendorf Closes #13167 commit a6bf7ea16a57478b8b953a3947a721a63b7abb4c Author: Brian Behlendorf Date: Tue Mar 1 08:46:00 2022 -0800 ZTS: Move largest_pool_001_pos.ksh to Linux runfile On FreeBSD pools are not allowed to be created using vdevs which are backed by ZFS volumes. This configuration is not recommended for any supported platform, nevertheless the largest_pool_001_pos.ksh test case makes use of it as a convenience. This causes the test case to fail reliably on FreeBSD. The layout is still tolerated on Linux so only perform this test on Linux. Reviewed-by: Igor Kozhukhov Reviewed by: George Melikov Reviewed-by: Ryan Moeller Signed-off-by: Brian Behlendorf Closes #13166 commit b3ab2908554b246b8c6e1adf70d43ba66f47fcdd Author: Savyasachee Jha Date: Tue Mar 1 04:05:25 2022 +0530 dracut: skip zfsexpandknoweldge when zfs_devs is present in dracut PR 1711 (https://github.com/dracutdevs/dracut/pull/1711) adds a zfs_devs function to dracut to detect the physical devices backing zfs pools. If this function exists in the version of dracut this module is being called from, then it does not need to run. Reviewed-by: Brian Behlendorf Reviewed-by: Ahelenia Ziemiańska Signed-off-by: Savyasachee Jha Closes #13121 commit 9e532d17f36a7483b58d9542562982cfc4ab098b Author: наб Date: Fri Feb 25 20:50:09 2022 +0100 config: gcc != cc Reviewed-by: Brian Behlendorf Reviewed-by: Rich Ercolani Signed-off-by: Ahelenia Ziemiańska Closes #13145 Closes #13152 commit b08153f1c1dc55711cbfe8a93d5d8579359a447a Author: наб Date: Fri Feb 25 17:50:12 2022 +0000 freebsd: libzfs: zmount: void-cast unused assert(3) variables Reviewed-by: Brian Behlendorf Reviewed-by: Rich Ercolani Signed-off-by: Ahelenia Ziemiańska Closes #13145 Closes #13152 commit 1ea68b6d38d8e20e53e813b6a9d5d68f4c7cd838 Author: наб Date: Fri Feb 25 17:49:29 2022 +0000 config: check for -Wno-cast-function-type Reviewed-by: Brian Behlendorf Reviewed-by: Rich Ercolani Signed-off-by: Ahelenia Ziemiańska Closes #13145 Closes #13152 commit 82226e4f44baa3f7c3101caaaf941927aa318e46 Author: Paul Dagnelie Date: Sat Feb 26 11:24:27 2022 -0800 Fix erroneous zstreamdump warning Reviewed-by: Brian Behlendorf Reviewed-by: George Amanakis Signed-off-by: Paul Dagnelie Closes #13154 commit ce91f973ec7cddeb825b900e8114886fc5ec9952 Author: Brian Behlendorf Date: Sat Feb 26 11:20:32 2022 -0800 ztest: Fix ASSERT in ztest_objset_destroy_cb() The dsl_destroy_snapshot() call in ztest_objset_destroy_cb() may encounter a runtime error when the pool is out of space. This is similar to the error handling for the dsl_destroy_head() case, but since dsl_destroy_snapshot() is implemented as a channel program ECHRNG is returned instead of ENOSPC. ECHRNG may also be returned instead of EBUSY if there is a hold on the snapshot. Reviewed by: George Melikov Signed-off-by: Brian Behlendorf Closes #13155 commit 4f453dcc1f062825ad70343783d0c2d616bae07a Author: наб Date: Sat Feb 26 20:19:05 2022 +0100 Fix FreeBSD reporting on reruns Turns out, when your test-suite fails on FreeBSD the rerun logic would fail as follows: Results Summary PASS 1358 FAIL 7 SKIP 47 Running Time: 04:00:02 Percent passed: 96.2% Log directory: /var/tmp/test_results/20220225T092538 mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory ... This change resolves a flaw from the original commit, 2320e6eb4 ("Add zfs-test facility to automatically rerun failing tests") Reviewed by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13156 commit f2f6c18f17425b64f9c4a5352085caac769a3601 Author: Tony Hutter Date: Thu Feb 24 11:43:39 2022 -0800 zed: Misc multipath autoreplace fixes We recently had a case where our operators replaced a bad multipathed disk, only to see it fail to autoreplace. The zed logs showed that the multipath replacement disk did not pass the 'is_dm' test in zfs_process_add() even though it should have. is_dm is set if there exists a sysfs entry for to the underlying /dev/sd* paths for the multipath disk. It's possible this path didn't exist due to a race condition where the sysfs paths weren't created at the time the udev event came in to zed, but this was never verified. This patch updates the check to look for udev properties that indicate if the new autoreplace disk is an empty multipath disk, rather than looking for the underlying sysfs entries. It also adds in additional logging, and fixes a bug where zed allowed you to use an already zfs-formatted disk from another pool as a multipath auto-replacement disk. Furthermore, while testing this patch, I also ran across a case where a force-faulted disk did not have a ZPOOL_CONFIG_PHYS_PATH entry in its config. This prevented it from being autoreplaced. I added additional logic to derive the PHYS_PATH from the PATH if the PATH was a /dev/disk/by-vdev/ path. For example, if PATH was /dev/disk/by-vdev/L28, then PHYS_PATH would be L28. This is safe since by-vdev paths represent physical locations and do not change between boots. Reviewed-by: Brian Behlendorf Signed-off-by: Tony Hutter Closes #13023 commit e25bcf906a425892a834f08157683e709a307a14 Author: Damian Szuberski Date: Thu Feb 24 19:33:48 2022 +0100 Fix directory detection in `dkms.mkconf` Fix `zfs-dkms` installation on Debian-derived distributions by aligning the directory detection logic to #13096. Reviewed-by: Brian Behlendorf Signed-off-by: szubersk Closes #11449 Closes #13141 commit 78fad47cb30c428f1f1b5a6373b94aa6e8ef24fa Author: Damian Szuberski Date: Thu Feb 24 19:21:13 2022 +0100 Add Linux kmemleak support to ZTS - Kmemleak `clear` is invoked right before every test case run. - Kmemleak `scan` is requested right after each test case is finished. - Kmemleak instrumentation is not used for setup/cleanup/pretest/posttest/failsafe stages to shorten the test case execution time. - Kmemleak periodic scan is disabled (`scan=0`) before the test suite run to avoid interfering with the on-demand scan results. - There are unavoidable potential false positives coming from kernel areas other than OpenZFS module. - The ZTS with kmemleak enabled duration is increased by ~50%. Example run ``` Running Time: 07:12:13 Percent passed: 98.3% unreferenced object 0xffff9da82aea5410 (size 80): comm "kworker/u32:10", pid 942206, jiffies 4296749716 (age 2615.516s) hex dump (first 32 bytes): 00 30 30 00 00 00 00 00 ff 8f 30 00 00 00 00 00 .00.......0..... 51 e6 77 05 a8 9d ff ff 00 00 00 00 00 00 00 00 Q.w............. backtrace: [<000000005cf1fea2>] alloc_extent_state+0x1d/0xb0 [btrfs] [<0000000083f78ae5>] set_extent_bit+0x2ff/0x670 [btrfs] [<00000000de29249e>] lock_extent_bits+0x6b/0xa0 [btrfs] [<00000000b241f424>] lock_and_cleanup_extent_if_need+0xaf/0x1c0 [btrfs] [<0000000093ca72b5>] btrfs_buffered_write+0x297/0x7d0 [btrfs] [<000000002c2938c8>] btrfs_file_write_iter+0x127/0x390 [btrfs] [<00000000b888f720>] do_iter_readv_writev+0x152/0x1b0 [<00000000320f0bcc>] do_iter_write+0x7c/0x1c0 [<000000000b5a8fe0>] lo_write_bvec+0x62/0x150 [loop] [<000000009aa03c73>] loop_process_work+0x250/0xbd0 [loop] [<00000000c7487d8a>] process_one_work+0x1f1/0x390 [<000000000b236831>] worker_thread+0x53/0x3e0 [<0000000023cb3e57>] kthread+0x127/0x150 [<000000002d48676a>] ret_from_fork+0x22/0x30 ``` Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: szubersk Closes #13084 commit 4d14a285b341b7031c0d23b7dd6b5ad63de8f0d4 Author: Attila Fülöp Date: Thu Feb 24 18:23:41 2022 +0100 Linux 5.11 compat: x86 SIMD: fix kernel_fpu_{begin,end}() detection Linux 5.11 changed kernel_fpu_begin() to an inlined function and moved the functionality to kernel_fpu_begin_mask(). This breaks the existing detection mechanism since it checks if kernel_fpu_begin is an exported kernel symbol, which isn't the case for an inlined function. To avoid assumptions about internal implementation, replace ZFS_LINUX_TEST_RESULT_SYMBOL in favor of ZFS_LINUX_TEST_RESULT which already makes sure kernel_fpu_{begin,end}() is usable by us. Reviewed-by: Tony Hutter Reviewed-by: Brian Behlendorf Signed-off-by: Attila Fülöp Closes #13147 commit 361a7e821178e105c8e1206ead4479de83c2a617 Author: Jitendra Patidar <53164267+jsai20@users.noreply.github.com> Date: Wed Feb 23 02:36:43 2022 +0530 log xattr=sa create/remove/update to ZIL As such, there are no specific synchronous semantics defined for the xattrs. But for xattr=on, it does log to ZIL and zil_commit() is done, if sync=always is set on dataset. This provides sync semantics for xattr=on with sync=always set on dataset. For the xattr=sa implementation, it doesn't log to ZIL, so, even with sync=always, xattrs are not guaranteed to be synced before xattr call returns to caller. So, xattr can be lost if system crash happens, before txg carrying xattr transaction is synced. This change adds xattr=sa logging to ZIL on xattr create/remove/update and xattrs are synced to ZIL (zil_commit() done) for sync=always. This makes xattr=sa behavior similar to xattr=on. Implementation notes: The actual logging is fairly straight-forward and does not warrant additional explanation. However, it has been 14 years since we last added new TX types to the ZIL [1], hence this is the first time we do it after the introduction of zpool features. Therefore, here is an overview of the feature activation and deactivation workflow: 1. The feature must be enabled. Otherwise, we don't log the new record type. This ensures compatibility with older software. 2. The feature is activated per-dataset, since the ZIL is per-dataset. 3. If the feature is enabled and dataset is not for zvol, any append to the ZIL chain will activate the feature for the dataset. Likewise for starting a new ZIL chain. 4. A dataset that doesn't have a ZIL chain has the feature deactivated. We ensure (3) by activating on the first zil_commit() after the feature was enabled. Since activating the features requires waiting for txg sync, the first zil_commit() after enabling the feature will be slower than usual. The downside is that this is really a conservative approximation: even if we never append a 'TX_SETSAXATTR' to the ZIL chain, we pay the penalty for feature activation. The upside is that the user is in control of when we pay the penalty, i.e., upon enabling the feature. We ensure (4) by hooking into zil_sync(), where ZIL destroy actually happens. One more piece on feature activation, since it's spread across multiple functions: zil_commit() zil_process_commit_list() if lwb == NULL // first zil_commit since zil_open zil_create() if no log block pointer in ZIL header: if feature enabled and not active: // CASE 1 enable, COALESCE txg wait with dmu_tx that allocated the log block else // log block was allocated earlier than this zil_open if feature enabled and not active: // CASE 2 enable, EXPLICIT txg wait else // already have an in-DRAM LWB if feature enabled and not active: // this happens when we enable the feature after zil_create // CASE 3 enable, EXPLICIT txg wait [1] https://github.com/illumos/illumos-gate/commit/da6c28aaf62fa55f0fdb8004aa40f88f23bf53f0 Reviewed-by: Matthew Ahrens Reviewed-by: Christian Schwarz Reviewed-by: Ahelenia Ziemiańska Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Jitendra Patidar Closes #8768 Closes #9078 commit ccdcc1dbe8b2b741194cdbc5b81bdb8b58cc7142 Author: Krzysztof Piecuch <3964215+pikrzysztof@users.noreply.github.com> Date: Tue Feb 22 20:59:11 2022 +0000 systemd: read initconfdir Systemd units do not read @initconfdir@ but refer to variables defined there, also a minor fixup in zfs-scrub service file. Reviewed-by: Ahelenia Ziemiańska Reviewed-by: George Melikov Reviewed-by: Brian Behlendorf Reviewed-by: Damian Szuberski Signed-off-by: Krzysztof Piecuch Closes #12946 commit 7454ca413cb7464c9dc055fe6b9727c177a3a73a Author: наб Date: Fri Feb 18 12:33:47 2022 +0100 zpoo-features.7: raidz -> RAID-Z near dRAID Reviewed by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13116 commit 1f4376eb019534223756a5ee5d23be9e76d17aac Author: наб Date: Fri Feb 18 12:31:20 2022 +0100 zpool-features.7: never-return-enabled consistency Reviewed by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13116 commit 58a48481328b5db1e9aeb965fd7db16f33647ac7 Author: наб Date: Fri Feb 18 12:27:32 2022 +0100 zpool-features.7: zfs sendstreams Reviewed by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13116 commit 6f763af2707b7f929f48cc6a3408ac5352c69266 Author: наб Date: Fri Feb 18 12:16:41 2022 +0100 zpool-features.7: spurious line break in enabled_txg Reviewed by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13116 commit 2d232ca806c3ece65c1b8915aa7ea6fa0d4e11e7 Author: наб Date: Thu Feb 17 21:36:30 2022 +0100 man: full stop at EOL Reviewed by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13116 commit 6985944e2ffcc91920fc90b8bae423521b42805c Author: наб Date: Thu Feb 17 21:30:22 2022 +0100 man: -based when -based Reviewed by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13116 commit a737b415d6f1de0d648253a5caf67f65fffa7d41 Author: наб Date: Thu Feb 17 21:26:43 2022 +0100 man: IO -> I/O; I/Os -> I/O operations again Reviewed by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13116 commit aafb89016bc3ebba43e2063102a476b3d82c0bf0 Author: наб Date: Thu Feb 17 21:10:00 2022 +0100 man: final RAIDZ -> RAID-Z Reviewed by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13116 commit 6bf6f5196b7a86939d6f32ff5d80095dd10bd88f Author: наб Date: Thu Feb 17 21:09:16 2022 +0100 man: final VDEV -> vdev Reviewed by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13116 commit a5b3fab341b375ecfaea173ba46abe0b6ff218c4 Author: Brian Behlendorf Date: Sun Feb 20 19:21:31 2022 -0800 ZTS: Retry in import_rewind_config_changed.ksh As explained by the disclaimer in the test case, "This test can fail since nothing guarantees that old MOS blocks aren't overwritten." This behavior is expected and correct, but results in a flaky test case which is problematic for the CI. The best we can do to resolve this is to retry the sub-test which failed when the MOS blocks have clearly been overwritten. When testing failures were rare enough that a single retry should normally be sufficient. However, we allow up to five for good measure. Reviewed by: George Melikov Signed-off-by: Brian Behlendorf Closes #13119 commit 806739f991aa0d3fc5a36989fff8d8ab8fddf78f Author: Damian Szuberski Date: Mon Feb 21 04:20:00 2022 +0100 Correct compilation errors reported by GCC 10/11 New `zfs_type_t` value `ZFS_TYPE_INVALID` is introduced. Variable initialization is now possible to make GCC happy. Reviewed by: Brian Behlendorf Signed-off-by: szubersk Closes #12167 Closes #13103 commit e41013078a90d5790c737eec573ec376ca21a234 Author: Ryan Moeller Date: Fri Feb 18 16:09:03 2022 -0500 libzfs: Fail making a dataset handle gracefully When a dataset is in the process of being received it gets marked as inconsistent and should not be used. We should check for this when opening a dataset handle in libzfs and return with an appropriate error set, rather than hitting an abort because of the incomplete data. zfs_open() passes errno to zfs_standard_error() after observing make_dataset_handle() fail, which ends up aborting if errno is 0. Set errno before returning where we know it has not been set already. Reviewed-by: Brian Behlendorf Signed-off-by: Ryan Moeller Closes #13077 commit a014378dd010fb4f9bf5ab003bd27526ff898952 Author: Damian Szuberski Date: Fri Feb 18 19:43:11 2022 +0000 spl: make 'spl_panic_halt' working for all cases The default behavior where the serious ZFS errors cause FS thread to stuck is very bad for some production scenario. In some production scenarios (Linux), it is recommended to make real kernel PANIC, where system can be rebooted by watchdog or kernel itself. This patch enables coherent handling of spl_panic_halt parameter. Reviewed by: Brian Behlendorf Authored-by: Wojciech Nizinski Signed-off-by: szubersk Closes #12120 Closes #13109 commit 15b982492a1fa8a3e75efd06b263121eda70e3a1 Author: наб Date: Wed Feb 16 03:02:00 2022 +0100 cstyle: forbid ARGSUSED Reviewed-by: Alejandro Colomar Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13110 commit a2995e76412d7621318c47ba4f0ed9d58566df2d Author: наб Date: Wed Feb 16 03:01:30 2022 +0100 config: add -Wextra (sans sign-compare and missing-field-initializers) Reviewed-by: Alejandro Colomar Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13110 commit 642827ecdae3834a6f62c6f1c0ff78b5c5b0c8ec Author: наб Date: Wed Feb 16 21:17:18 2022 +0100 module: zfs: zcp_get: fix uninitialised warning Reviewed-by: Alejandro Colomar Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13110 commit b7c42ce5b28041b45c3dff6cc9ecf55bd83b6087 Author: наб Date: Wed Feb 16 16:35:00 2022 +0100 raidz_test: silence unsigned >=0 warnings Reviewed-by: Alejandro Colomar Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13110 commit a215c3e83445c230ced10bc5ed13874c3eaaa20f Author: наб Date: Wed Feb 16 16:23:57 2022 +0100 libzpool: kernel: silence unrelated-function-pointer cast warning Reviewed-by: Alejandro Colomar Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13110 commit d8552689d1bb49dfff93f3970a0250938ba7dc3d Author: наб Date: Wed Feb 16 16:10:57 2022 +0100 libnvpair: json: suppress wchar_t >=0 warnings Reviewed-by: Alejandro Colomar Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13110 commit 4e1f1035d033d43acc39bafd6bfd45493b7e00f4 Author: наб Date: Wed Feb 16 16:08:20 2022 +0100 config: prune unused -Wno-bool-compare checks Reviewed-by: Alejandro Colomar Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13110 commit 72154bd6c9b352fe9f2d9a48c4d45211edd56823 Author: наб Date: Wed Feb 16 16:06:08 2022 +0100 libtpool: -Wno-clobbered Also remove -Wno-unused-but-set-variable Upstream-bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61118 Reviewed-by: Alejandro Colomar Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13110 commit 0ea6510aa006dc88f91fa5f666bfe8943eda4a24 Author: наб Date: Wed Feb 16 15:29:35 2022 +0100 module: icp: remove useless assert Which produces a warning since uints are, by definition, >=0 Reviewed-by: Alejandro Colomar Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13110 commit 5cf3c24fd8ada1949959deb8d511eeda9bc54e5b Author: наб Date: Wed Feb 16 15:29:01 2022 +0100 libzfs: sendrecv: fix NULL arithmetic UB Reviewed-by: Alejandro Colomar Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13110 commit 46c7a80280a5d36a9bc1b167ceadc316196965df Author: наб Date: Wed Feb 16 13:09:27 2022 +0100 userspace: mark arguments used Reviewed-by: Alejandro Colomar Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13110 commit ef70eff1982228862ef9ad70f21b222cde06eb5e Author: наб Date: Wed Feb 16 02:38:43 2022 +0100 module: mark arguments used Reviewed-by: Alejandro Colomar Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13110 commit 51c747de4330b1008593183de66a6b117185638a Author: наб Date: Wed Feb 16 02:07:56 2022 +0100 linux: module/zfs: vnops: make null_xattr static Reviewed-by: Alejandro Colomar Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13110 commit 7901b62685e91cb5b6d28346e6ffb7e6456326bb Author: Brian Behlendorf Date: Thu Feb 17 12:09:06 2022 -0800 ZTS: Fix vdev_zaps_004_pos.ksh When attaching a vdev to a mirror wait for the resilver to complete before invoking `zdb` to inspect the pool. This ensures the pool is essentially idle which allows `zdb` to open the imported pool reliably. Reviewed-by: John Kennedy Signed-off-by: Brian Behlendorf Closes #13112 Closes #6935 commit 5eae5a88b258cf955b46a99da07d9d5acad8bacd Author: Savyasachee Jha Date: Mon Feb 14 22:28:37 2022 +0530 Multiple dracut module install script cleanups - Replaced intances of `dracut_install` with `inst_simple` - Removed calls to `test -x mark_hostonly` because the function is an inbuilt dracut function - Removed redundant installation of `systemd-ask-password` and `systemd-tty-ask-password-agent` because they are already installed by the systemd module. There is no need to install them again - Removed multiple calls to the `mark_hostonly` function because the `inst_simple` has a command-line switch for it - Cleaned up the installation of the `zpool.cache`, `vdev_id.conf` and `hostid` files to make the logic easier to follow - Cleaned up and simplified the systemd service installation logic by invoking systemctl instead of creating symlinks manually - Replaced various hard-coded paths with dracut equivalents to better conform with expected dracut behaviour - Removed redundant call to `mkdir` (`inst_simple` creates the parent directory if it does not exist on the destination initrd) Reviewed-by: Ahelenia Ziemiańska Reviewed-by: Andrew J. Hesford Signed-off-by: Savyasachee Jha Closes #13010 commit ed66eafd0592bcc270dc61f64f4d91cf7ea91e30 Author: Savyasachee Jha Date: Mon Feb 14 18:15:16 2022 +0530 Remove absolute paths to udev rules and binaries for dracut Since dracut functions can locate both udev rules and binaries, there is no point in keeping absolute paths in the module setup script. It also breaks the --sysroot option in dracut. This commit removes mentions to absolute paths for binaries and udev rules. Reviewed-by: Ahelenia Ziemiańska Reviewed-by: Andrew J. Hesford Signed-off-by: Savyasachee Jha Closes #13010 commit 7f5a01bdd3eb70a8618eaf640f02482272fdd950 Author: Savyasachee Jha Date: Sun Feb 6 09:41:23 2022 +0530 Make dracut fail if essential files cannot be installed Dracut will now fail in initramfs generation if essential files cannot be installed. Reviewed-by: Ahelenia Ziemiańska Reviewed-by: Andrew J. Hesford Signed-off-by: Savyasachee Jha Closes #13010 commit 44b0380a3550562f34e06d109f20eb2174848068 Author: Savyasachee Jha Date: Tue Jan 25 03:22:48 2022 +0000 Make better use of dracut functions when building initramfs Setting up the module involves multiple redundant calls to a bunch of dracut functions wheich can be combined into one. Additionally, the mass of code required to load libgcc_s.so* can be replaced with one dracut function. This has the additional effect of removing errors involving the non-installation of libgcc_s.so* which are seen on debian bullseye when using version 2.1.2-1~bpo11+1 from the backports repository. The systemd binaries are separated out into their own `dracut_install` function call so they do not get pulled in when dracut does not load the systemd module. Reviewed-by: Ahelenia Ziemiańska Reviewed-by: Andrew J. Hesford Signed-off-by: Savyasachee Jha Closes #13010 commit 72f3c8b13019b8882f4536411e42734fd3b4a4d7 Author: Damian Szuberski Date: Thu Feb 17 00:17:16 2022 +0100 Fix Linux kernel directories detection Most modern Linux distributions have separate locations for bare source and prebuilt ("build") files. Additionally, there are `source` and `build` symlinks in `/lib/modules/$(KERNEL_VERSION)` pointing to them. The order of directory search is now: - `configure` command line values if both `--with-linux` and `--with-linux-obj` were defined - If only `--with-linux` was defined, `--with-linux-obj` is assumed to have the same value as `--with-linux` - If neither `--with-linux` nor `--with-linux-obj` were defined autodetection is used: - `/lib/modules/$(uname -r)/{source,build}` respectively, if exist - The first directory in `/lib/modules` with the highest version number according to `sort -V` which contains `source` and `build` symlinks/directories - The first directory matching `/usr/src/kernels/*` and `/usr/src/linux-*` with the highest version number according to `sort -V`. Here the source and prebuilt directories are assumed to be the same. Reviewed-by: Brian Behlendorf Signed-off-by: szubersk Closes #9935 Closes #13096 commit 52a36bd41a3baac19bcb9efc7f3ae4088758bf7f Author: George Amanakis Date: Wed Feb 16 20:52:02 2022 +0100 Enable encrypted raw sending to pools with greater ashift Raw sending from pool1/encrypted with ashift=9 to pool2/encrypted with ashift=12 results to failure when mounting pool2/encrypted (Input/Output error). Notably, the opposite, raw sending from a greater ashift to a lower one does not fail. This happens because zio_compress_write() falsely checks only ZIO_FLAG_RAW_COMPRESS and not ZIO_FLAG_RAW_ENCRYPT which is also set in encrypted raw send streams. In this case it rounds up the psize and if not equal to the zio->io_size it modifies the block by zeroing out the extra bytes. Because this happens in a SA attr. registration object (type=46), the decryption fails upon mounting the filesystem, and zpool status falsely reports an error. Fix this by checking both ZIO_FLAG_RAW_COMPRESS and ZIO_FLAG_RAW_ENCRYPT before deciding whether to zero-pad a block. Reviewed-by: Brian Behlendorf Signed-off-by: George Amanakis Closes #13067 Closes #13074 commit ba6005175eee22ec7fb9f4226690f70ce35a45ae Author: Damian Szuberski Date: Wed Feb 16 20:48:01 2022 +0100 `checkabi`/`storeabi` relevant only to x86_64 The stored ABI files are for the x86_64 architecture. Reviewed-by: Brian Behlendorf Reviewed-by: George Melikov Signed-off-by: szubersk Closes #11345 Closes #13104 commit 0df22afea22f5ed59be1e36d4b716eb832a319fd Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Wed Feb 16 14:40:25 2022 -0500 Colorize the Github test output Let's color our workflow test output for better readability. Reviewed by: George Melikov Reviewed-by: Brian Behlendorf Reviewed-by: Ahelenia Ziemiańska Signed-off-by: Rich Ercolani Closes #13000 commit 67de71e6445dae13520bb87c8fc5fb993cb66f30 Author: наб Date: Wed Feb 16 01:42:30 2022 +0100 libzfs: calculate receive times with sub-second precision Provide two digits of precision when reporting send/receive times. Tiny snapshots may take significantly less than a second and rounding up to a full second can introduce a significant error. Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #13047 commit 5c0061345b824eebe7a6578528f873ffcaae1cdd Author: Ryan Moeller Date: Tue Feb 15 19:35:30 2022 -0500 Cross-platform xattr user namespace compatibility ZFS on Linux originally implemented xattr namespaces in a way that is incompatible with other operating systems. On illumos, xattrs do not have namespaces. Every xattr name is visible. FreeBSD has two universally defined namespaces: EXTATTR_NAMESPACE_USER and EXTATTR_NAMESPACE_SYSTEM. The system namespace is used for protected FreeBSD-specific attributes such as MAC labels and pnfs state. These attributes have the namespace string "freebsd:system:" prefixed to the name in the encoding scheme used by ZFS. The user namespace is used for general purpose user attributes and obeys normal access control mechanisms. These attributes have no namespace string prefixed, so xattrs written on illumos are accessible in the user namespace on FreeBSD, and xattrs written to the user namespace on FreeBSD are accessible by the same name on illumos. Linux has several xattr namespaces. On Linux, ZFS encodes the namespace in the xattr name for every namespace, including the user namespace. As a consequence, an xattr in the user namespace with the name "foo" is stored by ZFS with the name "user.foo" and therefore appears on FreeBSD and illumos to have the name "user.foo" rather than "foo". Conversely, none of the xattrs written on FreeBSD or illumos are accessible on Linux unless the name happens to be prefixed with one of the Linux xattr namespaces, in which case the namespace is stripped from the name. This makes xattrs entirely incompatible between Linux and other platforms. We want to make the encoding of user namespace xattrs compatible across platforms. A critical requirement of this compatibility is for xattrs from existing pools from FreeBSD and illumos to be accessible by the same names in the user namespace on Linux. It is also necessary that existing pools with xattrs written by Linux retain access to those xattrs by the same names on Linux. Making user namespace xattrs from Linux accessible by the correct names on other platforms is important. The handling of other namespaces is not required to be consistent. Add a fallback mechanism for listing and getting xattrs to treat xattrs as being in the user namespace if they do not match a known prefix. Do not allow setting or getting xattrs with a name that is prefixed with one of the namespace names used by ZFS on supported platforms. Allow choosing between legacy illumos and FreeBSD compatibility and legacy Linux compatibility with a new tunable. This facilitates replication and migration of pools between hosts with different compatibility needs. The tunable controls whether or not to prefix the namespace to the name. If the xattr is already present with the alternate prefix, remove it so only the new version persists. By default the platform's existing convention is used. Reviewed-by: Christian Schwarz Reviewed-by: Ahelenia Ziemiańska Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Ryan Moeller Closes #11919 commit 666749806da7475dd0e02ab3d418bad99c74a3ea Author: наб Date: Wed Feb 2 23:11:34 2022 +0100 module: icp: remove provider stats These were all folded into a single kstat at /proc/spl/kstat/kcf/NONAME_provider_stats with no way to know which one it actually was, and only the AES and SHA (so not Skein) ones were ever updated Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit bf8663868752e1f3a423307b00cdb589d561310f Author: наб Date: Fri Jan 7 01:32:14 2022 +0100 module: icp: enforce KCF_{OPS_CLASSSIZE,MAXMECHTAB} Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit e013057492c43cec6eb4b55bc06581c7113ed4f6 Author: наб Date: Mon Dec 27 03:02:17 2021 +0100 module: icp: remove unused pd_{remove_cv,hash_limit} Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit de0ec5e7dfe9adc893440cf114cb719c043fe5f5 Author: наб Date: Mon Dec 27 02:53:32 2021 +0100 module: icp: remove vestigia of crypto sessions Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit cf497e18df6adff6f95bc89018d0582b5e5b7f67 Author: наб Date: Mon Dec 27 02:39:55 2021 +0100 module: icp: remove unused (and mostly faked) cm_{{min,max}_key_length,mech_flags} Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 11320b4cdf54b725d90ec158d29d7a18068b75c0 Author: наб Date: Mon Dec 27 02:32:37 2021 +0100 module: icp: remove unused crypto_provider_handle_t Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit f5e7d918a7cde95f2c2ff6c1645ba082961e06cc Author: наб Date: Sat Dec 25 04:37:22 2021 +0100 module: icp: remove pre-set entries from mechtabs They don't do anything except clogging up the AVL tree Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit df7b54f1d983ca41e7b2add09664b1da128f3424 Author: наб Date: Sat Dec 25 04:34:29 2021 +0100 module: icp: rip out insane crypto_req_handle_t mechanism, inline KM_SLEEP Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 15ec0863963a12f698e0318d6cac76c48b9770d5 Author: наб Date: Sat Dec 25 04:00:34 2021 +0100 include: crypto: clean out api.h Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 42dbc2025a24f4a945021938794111e4cb502a34 Author: наб Date: Sat Dec 25 03:50:01 2021 +0100 module: icp: remove unused headers. Migrate {ops => sched}_impl Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 1949be46c317610b3d7900dcbc9c0059120c7207 Author: наб Date: Sat Dec 25 03:44:06 2021 +0100 include: crypto: clean out unused SYSCALL32 and flags Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit f43748f6e1997e823167bace2b75d5e9ea05256b Author: наб Date: Sat Dec 25 03:40:59 2021 +0100 module: icp: remove algorithm name defines used only in the default mechtab Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit d223af9bbc0bbb449deda4b91b0ec051a6670e2d Author: наб Date: Sat Dec 25 03:37:53 2021 +0100 include: crypto: remove unused algorithm name defines Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 64e82cea137d6d60d724e0c632e5cad9fd7d5f75 Author: наб Date: Sat Dec 25 03:33:19 2021 +0100 module: icp: remove set-but-unused cd_miscdata Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 739afd9475494ef8443a7f8e251bf2aaff895f35 Author: наб Date: Sat Dec 25 03:23:07 2021 +0100 module: icp: fold away all key formats except CRYPTO_KEY_RAW It's the only one actually used Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 1018e81e30f030c9cf8dbc52508088ce1983e36e Author: наб Date: Sat Dec 25 02:50:25 2021 +0100 module: icp: remove unused CRYPTO_* error codes Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 7eacb8711277f360bc584f6def11e4a89e3df8b4 Author: наб Date: Sat Dec 25 01:07:58 2021 +0100 module: icp: rip out modhash. Replace the one user with AVL Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 1cb6fa2cb8f4ed9aa1b9496a4b16a30288e18215 Author: наб Date: Sat Dec 25 01:51:28 2021 +0100 module: icp: remove unused me_mutex It only needs to be locked if dynamic changes can occur. They can't. Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit cb6e9c3f5f5f95d567159ad0e5464e398c3dacc2 Author: наб Date: Sat Dec 25 01:32:06 2021 +0100 module: icp: remove unused me_threshold Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 7f90cf30437f44377c575cb7f385a1ad8f1282f9 Author: наб Date: Sat Dec 25 00:26:17 2021 +0100 module: icp: remove unused struct crypto_ctx::cc_{session,flags,opstate} Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit a288428d83d926bc5d95c48eb06388f31763ba62 Author: наб Date: Sat Dec 25 00:10:57 2021 +0100 module: icp: remove unused gswq, kcfpool, [as]req_cache, reqid_table, obsolete kstat Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 1c17d2940cd374751007291a1c0555ae08e4e017 Author: наб Date: Fri Dec 24 17:43:28 2021 +0100 module: icp: remove unused notification framework Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 3fd5ead75e5de0279b4aadbc87c2151a27af4fe9 Author: наб Date: Fri Dec 24 17:34:19 2021 +0100 module: icp: remove unused kcf_op_{group,type}, req_params, ... Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit f3c3a6d47e0296723cf6a478283e6dfc3eb1cd96 Author: наб Date: Fri Dec 24 17:04:32 2021 +0100 module: icp: remove unused p[di]_flags Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit d77702035ae9db8db62f40a8ab35ce9d37751e4a Author: наб Date: Fri Dec 24 16:43:18 2021 +0100 module: icp: remove unused CRYPTO_{NOTIFY_OPDONE,SKIP_REQID,RESTRICTED} Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit eb1e09b7ec5c388172fac7d505cd58af7fd9f4fa Author: наб Date: Fri Dec 24 16:26:11 2021 +0100 module: icp: remove unused CRYPTO_ALWAYS_QUEUE Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 65a613b70d3f151ddd3ef43c93b7f889434b7204 Author: наб Date: Fri Dec 24 16:26:25 2021 +0100 module: icp: remove unused kcf_digest.c Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 255bc38e6f221dbde792900007a625829aa14b75 Author: наб Date: Fri Dec 24 15:39:47 2021 +0100 module: icp: drop software provider generation numbers We register all providers at once, before anything happens Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit bf3fffe70de5dd905d315ab022a373296f5fd94b Author: наб Date: Thu Dec 23 20:11:01 2021 +0100 module: icp: remove unused kcf_mac operations Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 2c2f955aaebaeacc167970068a994ab014dd2e0d Author: наб Date: Thu Dec 23 20:06:22 2021 +0100 module: icp: remove unused kcf_cipher operations Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 710657f51d65b8fb1f389bf6a973e67a8243502f Author: наб Date: Thu Dec 23 19:51:00 2021 +0100 module: icp: remove other provider types Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 167ced3fb10ce9c2336828cb041420d96de8cf67 Author: наб Date: Thu Dec 23 18:44:07 2021 +0100 module: icp: use original mechanisms Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit bcee18d4e00b050f011d28df7ea9e74e95687061 Author: наб Date: Thu Dec 23 18:34:23 2021 +0100 module: icp: use original description Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit b0502ab09721aec6867b991c437b9d63df08ff3d Author: наб Date: Thu Dec 23 18:27:44 2021 +0100 module: icp: guarantee the ops vector is persistent Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit d59a7fae403f8d91b3512a559ec89432c87051a7 Author: наб Date: Thu Dec 23 03:22:27 2021 +0100 module: icp: have a static 8 providers This is currently twice the amount we actually have (sha[12], skein, aes), and 512 * sizeof(void*) = 4096: 128x more than we need and a waste of most of a page in the kernel address space Plus, there's no need to actually allocate it dynamically: it's always got a static size. Put it in .data Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 464700ae0293a3bb5d84d35eb7fc771ec22f3fad Author: наб Date: Wed Dec 22 23:29:25 2021 +0100 module: icp: spi: crypto_ops_t: remove unused op types Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit f5896e2bdf9d8824befe8660c7fe1f77ff773e3b Author: наб Date: Wed Dec 22 22:09:28 2021 +0100 module: icp: spi: flatten struct crypto_ops, crypto_provider_info Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 959b9d63927a18d711c8d49c9b904d0f3c7a1fa4 Author: наб Date: Wed Dec 22 22:03:00 2021 +0100 module: icp: spi: remove crypto_control_ops_t Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 9cdf015d0a1713c213f1bd5cbda956995a79c74c Author: наб Date: Wed Dec 22 21:55:37 2021 +0100 module: icp: spi: remove crypto_{provider,op}_notification() Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12901 commit 4759342a5eeabe168e0f658ba53e858566073085 Author: Jorgen Lundman Date: Wed Feb 16 08:54:25 2022 +0900 Add spa _os() hooks Add hooks for when spa is created, exported, activated and deactivated. Used by macOS to attach iokit, and lock kext as busy (to stop unloads). Userland, Linux, and, FreeBSD have empty stubs. Reviewed-by: Brian Behlendorf Signed-off-by: Jorgen Lundman Closes #12801 --- .github/workflows/build-dependencies.txt | 1 + .github/workflows/checkstyle.yaml | 4 +- .github/workflows/zfs-tests-functional.yml | 6 +- .github/workflows/zfs-tests-sanity.yml | 6 +- .github/workflows/zloop.yml | 7 +- .gitignore | 90 +- AUTHORS | 1 + META | 2 +- Makefile.am | 194 +- autogen.sh | 62 +- cmd/Makefile.am | 126 +- cmd/{arc_summary/arc_summary3 => arc_summary} | 0 cmd/arc_summary/.gitignore | 1 - cmd/arc_summary/Makefile.am | 8 - cmd/{arcstat => }/arcstat.in | 2 +- cmd/arcstat/.gitignore | 1 - cmd/arcstat/Makefile.am | 5 - cmd/{dbufstat => }/dbufstat.in | 2 +- cmd/dbufstat/.gitignore | 1 - cmd/dbufstat/Makefile.am | 5 - cmd/{fsck_zfs => }/fsck.zfs.in | 4 +- cmd/fsck_zfs/.gitignore | 1 - cmd/fsck_zfs/Makefile.am | 7 - cmd/{mount_zfs => }/mount_zfs.c | 15 +- cmd/mount_zfs/.gitignore | 1 - cmd/mount_zfs/Makefile.am | 22 - cmd/raidz_test/.gitignore | 1 - cmd/raidz_test/Makefile.am | 24 +- cmd/raidz_test/raidz_bench.c | 4 +- cmd/raidz_test/raidz_test.c | 18 +- cmd/raidz_test/raidz_test.h | 28 +- cmd/vdev_id/Makefile.am | 3 - cmd/zdb/.gitignore | 1 - cmd/zdb/Makefile.am | 22 +- cmd/zdb/zdb.c | 117 +- cmd/zdb/zdb.h | 2 +- cmd/zdb/zdb_il.c | 27 +- cmd/zed/Makefile.am | 87 +- cmd/zed/agents/fmd_api.c | 19 +- cmd/zed/agents/fmd_api.h | 9 +- cmd/zed/agents/fmd_serd.c | 14 +- cmd/zed/agents/fmd_serd.h | 2 +- cmd/zed/agents/zfs_diagnosis.c | 23 +- cmd/zed/agents/zfs_mod.c | 185 +- cmd/zed/agents/zfs_retire.c | 2 +- cmd/zed/zed.d/Makefile.am | 69 +- cmd/zed/zed.d/all-debug.sh | 2 +- cmd/zed/zed.d/zed-functions.sh | 22 +- cmd/zed/zed.d/zed.rc | 5 +- cmd/zed/zed_disk_event.c | 52 +- cmd/zed/zed_event.c | 10 +- cmd/zfs/.gitignore | 1 - cmd/zfs/Makefile.am | 27 +- cmd/zfs/zfs_iter.c | 32 +- cmd/zfs/zfs_iter.h | 3 +- cmd/zfs/zfs_main.c | 706 +- cmd/zfs/zfs_project.c | 18 +- cmd/zfs/zfs_projectutil.h | 2 +- cmd/zfs/zfs_util.h | 4 +- cmd/{zfs_ids_to_path => }/zfs_ids_to_path.c | 2 +- cmd/zfs_ids_to_path/.gitignore | 1 - cmd/zfs_ids_to_path/Makefile.am | 11 - cmd/{zgenhostid => }/zgenhostid.c | 2 +- cmd/zgenhostid/.gitignore | 1 - cmd/zgenhostid/Makefile.am | 7 - cmd/{zhack => }/zhack.c | 15 +- cmd/zhack/.gitignore | 1 - cmd/zhack/Makefile.am | 16 - cmd/zinject/.gitignore | 1 - cmd/zinject/Makefile.am | 19 +- cmd/zinject/translate.c | 5 +- cmd/zinject/zinject.c | 5 +- cmd/zinject/zinject.h | 2 +- cmd/zpool/.gitignore | 1 - cmd/zpool/Makefile.am | 190 +- cmd/zpool/os/freebsd/zpool_vdev_os.c | 2 +- cmd/zpool/os/linux/zpool_vdev_os.c | 3 +- cmd/zpool/zpool.d/iostat | 16 +- cmd/zpool/zpool.d/lsblk | 16 +- cmd/zpool/zpool.d/smart | 12 +- cmd/zpool/zpool_iter.c | 9 +- cmd/zpool/zpool_main.c | 280 +- cmd/zpool/zpool_util.c | 4 +- cmd/zpool/zpool_util.h | 2 +- cmd/zpool/zpool_vdev.c | 4 +- cmd/zpool_influxdb/.gitignore | 1 - cmd/zpool_influxdb/Makefile.am | 15 +- cmd/zpool_influxdb/zpool_influxdb.c | 67 +- cmd/zstream/.gitignore | 1 - cmd/zstream/Makefile.am | 26 +- cmd/zstream/zstream.c | 6 +- cmd/zstream/zstream.h | 4 + cmd/zstream/zstream_decompress.c | 359 + cmd/zstream/zstream_dump.c | 13 +- cmd/zstream/zstream_redup.c | 10 +- cmd/zstream/zstream_token.c | 4 +- cmd/{ztest => }/ztest.c | 366 +- cmd/ztest/.gitignore | 1 - cmd/ztest/Makefile.am | 25 - cmd/zvol_id/.gitignore | 1 - cmd/zvol_id/Makefile.am | 12 - cmd/zvol_id/zvol_id_main.c | 129 - cmd/{zvol_wait => }/zvol_wait | 10 +- cmd/zvol_wait/Makefile.am | 4 - config/Abigail.am | 33 - config/CppCheck.am | 16 +- config/Rules.am | 53 +- config/Shellcheck.am | 40 +- config/Substfiles.am | 48 +- config/always-arch.m4 | 12 +- config/always-compiler-options.m4 | 68 +- config/always-cppcheck.m4 | 3 + config/always-system.m4 | 4 + config/config.awk | 15 - config/deb.am | 7 +- config/kernel-add-disk.m4 | 8 +- config/kernel-bio.m4 | 39 +- config/kernel-blk-queue.m4 | 136 +- config/kernel-blkdev.m4 | 53 + config/kernel-block-device-operations.m4 | 17 +- config/kernel-config-defined.m4 | 88 +- config/kernel-copy-from-user-inatomic.m4 | 29 + config/kernel-fpu.m4 | 116 +- config/kernel-generic_io_acct.m4 | 83 +- config/kernel-genhd-flags.m4 | 58 + config/kernel-get-disk-ro.m4 | 4 +- config/kernel-global_page_state.m4 | 2 +- config/kernel-group-info.m4 | 4 +- config/kernel-inode-permission.m4 | 29 + config/kernel-mkdir.m4 | 2 + config/kernel-pagemap-folio_wait_bit.m4 | 2 +- config/kernel-readpages.m4 | 25 + config/kernel-revalidate-disk-size.m4 | 4 +- config/kernel-shrink.m4 | 10 +- config/kernel-sysfs.m4 | 37 + config/kernel-user-ns-inum.m4 | 23 + config/kernel-vfs-filemap_dirty_folio.m4 | 30 + config/kernel-vfs-iov_iter.m4 | 2 + config/kernel-vfs-read_folio.m4 | 32 + config/kernel-vfs-set_page_dirty.m4 | 2 +- config/kernel-zero_page.m4 | 27 + config/kernel.m4 | 164 +- config/user-aio.h.m4 | 7 + config/user-libaio.m4 | 2 +- config/user-systemd.m4 | 7 +- config/user-sysvinit.m4 | 7 +- config/user.m4 | 12 +- config/zfs-build.m4 | 19 +- configure.ac | 352 +- contrib/Makefile.am | 15 +- contrib/bash_completion.d/Makefile.am | 13 +- contrib/bpftrace/Makefile.am | 8 +- contrib/dracut/{90zfs => }/.gitignore | 0 .../dracut/02zfsexpandknowledge/.gitignore | 1 - .../dracut/02zfsexpandknowledge/Makefile.am | 8 - .../02zfsexpandknowledge/module-setup.sh.in | 6 + contrib/dracut/90zfs/Makefile.am | 24 - contrib/dracut/90zfs/export-zfs.sh.in | 18 +- contrib/dracut/90zfs/module-setup.sh.in | 163 +- contrib/dracut/90zfs/mount-zfs.sh.in | 132 +- contrib/dracut/90zfs/parse-zfs.sh.in | 67 +- .../dracut/90zfs/zfs-env-bootfs.service.in | 2 +- contrib/dracut/90zfs/zfs-generator.sh.in | 30 +- contrib/dracut/90zfs/zfs-lib.sh.in | 197 +- contrib/dracut/90zfs/zfs-load-key.sh.in | 105 +- contrib/dracut/90zfs/zfs-needshutdown.sh.in | 2 +- .../90zfs/zfs-rollback-bootfs.service.in | 6 +- .../90zfs/zfs-snapshot-bootfs.service.in | 6 +- contrib/dracut/Makefile.am | 29 +- contrib/dracut/README.md | 18 +- contrib/initramfs/Makefile.am | 42 +- contrib/initramfs/conf-hooks.d/Makefile.am | 4 - contrib/initramfs/conf.d/Makefile.am | 4 - contrib/initramfs/hooks/Makefile.am | 10 - contrib/initramfs/scripts/Makefile.am | 11 - .../initramfs/scripts/local-top/Makefile.am | 7 - contrib/initramfs/scripts/zfs | 6 +- contrib/pam_zfs_key/Makefile.am | 25 +- contrib/pam_zfs_key/pam_zfs_key.c | 3 +- contrib/pyzfs/Makefile.am | 32 +- contrib/pyzfs/libzfs_core/__init__.py | 2 + contrib/pyzfs/libzfs_core/_constants.py | 1 + .../pyzfs/libzfs_core/_error_translation.py | 2 + contrib/pyzfs/libzfs_core/_libzfs_core.py | 129 + .../pyzfs/libzfs_core/bindings/libzfs_core.py | 4 + .../libzfs_core/test/test_libzfs_core.py | 26 +- contrib/pyzfs/setup.py.in | 8 +- contrib/zcp/Makefile.am | 2 +- copy-builtin | 28 +- etc/Makefile.am | 99 +- etc/default/Makefile.am | 8 - etc/init.d/Makefile.am | 10 - etc/init.d/zfs-mount.in | 86 +- etc/init.d/zfs-zed.in | 3 +- etc/modules-load.d/.gitignore | 1 - etc/modules-load.d/Makefile.am | 2 - etc/sudoers.d/Makefile.am | 5 - etc/systemd/Makefile.am | 4 - etc/systemd/system-generators/Makefile.am | 14 - .../system-generators/zfs-mount-generator.c | 15 +- etc/systemd/system/.gitignore | 1 - .../{50-zfs.preset.in => 50-zfs.preset} | 0 etc/systemd/system/Makefile.am | 24 - .../system/zfs-import-cache.service.in | 1 + etc/systemd/system/zfs-import-scan.service.in | 1 + etc/systemd/system/zfs-mount.service.in | 1 + etc/systemd/system/zfs-scrub@.service.in | 3 +- etc/systemd/system/zfs-share.service.in | 1 + etc/systemd/system/zfs-trim-monthly@.timer.in | 12 + etc/systemd/system/zfs-trim-weekly@.timer.in | 12 + etc/systemd/system/zfs-trim@.service.in | 15 + etc/systemd/system/zfs-volume-wait.service.in | 1 + etc/systemd/system/zfs-zed.service.in | 1 + etc/zfs/Makefile.am | 18 - include/Makefile.am | 182 +- include/libnvpair.h | 2 +- include/libuutil.h | 18 +- include/libuutil_common.h | 2 +- include/libuutil_impl.h | 2 +- include/libzfs.h | 73 +- include/libzfs_core.h | 9 +- include/libzutil.h | 10 +- include/os/Makefile.am | 6 - include/os/freebsd/Makefile.am | 91 +- include/os/freebsd/linux/Makefile.am | 5 - include/os/freebsd/spl/Makefile.am | 1 - include/os/freebsd/spl/acl/Makefile.am | 4 - include/os/freebsd/spl/acl/acl_common.h | 2 +- include/os/freebsd/spl/rpc/Makefile.am | 4 - include/os/freebsd/spl/sys/Makefile.am | 75 - include/os/freebsd/spl/sys/acl.h | 2 +- include/os/freebsd/spl/sys/acl_impl.h | 2 +- include/os/freebsd/spl/sys/byteorder.h | 2 +- include/os/freebsd/spl/sys/callb.h | 4 +- include/os/freebsd/spl/sys/ccompile.h | 7 +- include/os/freebsd/spl/sys/cmn_err.h | 2 +- include/os/freebsd/spl/sys/cred.h | 138 +- include/os/freebsd/spl/sys/dkio.h | 2 +- include/os/freebsd/spl/sys/extdirent.h | 2 +- include/os/freebsd/spl/sys/idmap.h | 2 +- include/os/freebsd/spl/sys/isa_defs.h | 2 +- include/os/freebsd/spl/sys/kidmap.h | 41 - include/os/freebsd/spl/sys/kmem.h | 2 +- include/os/freebsd/spl/sys/list.h | 2 +- include/os/freebsd/spl/sys/list_impl.h | 2 +- include/os/freebsd/spl/sys/misc.h | 2 - include/os/freebsd/spl/sys/mod_os.h | 5 - include/os/freebsd/spl/sys/processor.h | 2 +- include/os/freebsd/spl/sys/procfs_list.h | 2 +- include/os/freebsd/spl/sys/sid.h | 25 - include/os/freebsd/spl/sys/strings.h | 1 - include/os/freebsd/spl/sys/sunddi.h | 1 - include/os/freebsd/spl/sys/sysmacros.h | 2 +- include/os/freebsd/spl/sys/taskq.h | 2 +- include/os/freebsd/spl/sys/types.h | 9 +- include/os/freebsd/spl/sys/uuid.h | 2 +- include/os/freebsd/spl/sys/vnode.h | 10 - include/os/freebsd/spl/sys/vnode_impl.h | 2 +- include/os/freebsd/spl/sys/zmod.h | 2 +- include/os/freebsd/zfs/Makefile.am | 1 - include/os/freebsd/zfs/sys/Makefile.am | 15 - include/os/freebsd/zfs/sys/freebsd_crypto.h | 7 +- include/os/freebsd/zfs/sys/sha2.h | 5 +- include/os/freebsd/zfs/sys/vdev_os.h | 2 +- include/os/freebsd/zfs/sys/zfs_ctldir.h | 2 +- include/os/freebsd/zfs/sys/zfs_dir.h | 2 +- include/os/freebsd/zfs/sys/zfs_ioctl_compat.h | 2 +- include/os/freebsd/zfs/sys/zfs_vfsops_os.h | 2 +- include/os/freebsd/zfs/sys/zfs_znode_impl.h | 34 +- include/os/linux/Makefile.am | 112 +- include/os/linux/kernel/Makefile.am | 1 - include/os/linux/kernel/linux/Makefile.am | 22 - include/os/linux/kernel/linux/blkdev_compat.h | 158 +- .../os/linux/kernel/linux/compiler_compat.h | 2 +- include/os/linux/kernel/linux/dcache_compat.h | 2 +- include/os/linux/kernel/linux/kmap_compat.h | 2 +- include/os/linux/kernel/linux/mod_compat.h | 9 +- include/os/linux/kernel/linux/percpu_compat.h | 2 +- include/os/linux/kernel/linux/simd.h | 2 +- include/os/linux/kernel/linux/simd_aarch64.h | 2 +- include/os/linux/kernel/linux/simd_powerpc.h | 36 +- include/os/linux/kernel/linux/simd_x86.h | 247 +- .../os/linux/kernel/linux/utsname_compat.h | 2 +- include/os/linux/kernel/linux/vfs_compat.h | 18 +- include/os/linux/kernel/linux/xattr_compat.h | 2 +- include/os/linux/spl/Makefile.am | 1 - include/os/linux/spl/rpc/Makefile.am | 7 - include/os/linux/spl/sys/Makefile.am | 64 - include/os/linux/spl/sys/cred.h | 5 - include/os/linux/spl/sys/errno.h | 2 +- include/os/linux/spl/sys/isa_defs.h | 23 +- include/os/linux/spl/sys/kmem_cache.h | 2 +- include/os/linux/spl/sys/procfs_list.h | 2 +- include/os/linux/spl/sys/string.h | 1 + include/os/linux/spl/sys/strings.h | 30 - include/os/linux/spl/sys/sunddi.h | 1 - include/os/linux/spl/sys/taskq.h | 2 - include/os/linux/spl/sys/thread.h | 10 +- include/os/linux/spl/sys/trace.h | 2 +- include/os/linux/spl/sys/trace_spl.h | 2 +- include/os/linux/spl/sys/trace_taskq.h | 2 +- include/os/linux/spl/sys/uio.h | 39 +- include/os/linux/spl/sys/zone.h | 31 +- include/os/linux/zfs/Makefile.am | 1 - include/os/linux/zfs/sys/Makefile.am | 31 - include/os/linux/zfs/sys/policy.h | 2 +- include/os/linux/zfs/sys/sha2.h | 5 +- include/os/linux/zfs/sys/trace_acl.h | 16 +- include/os/linux/zfs/sys/trace_arc.h | 2 +- include/os/linux/zfs/sys/trace_common.h | 2 +- include/os/linux/zfs/sys/trace_dbgmsg.h | 2 +- include/os/linux/zfs/sys/trace_dbuf.h | 2 +- include/os/linux/zfs/sys/trace_dmu.h | 2 +- include/os/linux/zfs/sys/trace_dnode.h | 2 +- include/os/linux/zfs/sys/trace_multilist.h | 2 +- include/os/linux/zfs/sys/trace_rrwlock.h | 2 +- include/os/linux/zfs/sys/trace_txg.h | 2 +- include/os/linux/zfs/sys/trace_vdev.h | 2 +- include/os/linux/zfs/sys/trace_zfs.h | 2 +- include/os/linux/zfs/sys/trace_zil.h | 2 +- include/os/linux/zfs/sys/trace_zio.h | 2 +- include/os/linux/zfs/sys/trace_zrlock.h | 2 +- include/os/linux/zfs/sys/zfs_context_os.h | 7 +- include/os/linux/zfs/sys/zfs_ctldir.h | 2 +- include/os/linux/zfs/sys/zfs_dir.h | 2 +- include/os/linux/zfs/sys/zfs_vfsops_os.h | 2 +- include/os/linux/zfs/sys/zfs_vnops_os.h | 4 +- include/os/linux/zfs/sys/zfs_znode_impl.h | 2 +- include/os/linux/zfs/sys/zpl.h | 2 +- include/sys/Makefile.am | 151 - include/sys/abd.h | 2 +- include/sys/abd_impl.h | 2 +- include/sys/arc.h | 16 +- include/sys/arc_impl.h | 2 +- include/sys/avl.h | 2 +- include/sys/avl_impl.h | 2 +- include/sys/bitops.h | 2 +- include/sys/blake3.h | 125 + include/sys/bplist.h | 2 +- include/sys/bpobj.h | 2 +- include/sys/bptree.h | 2 +- include/sys/btree.h | 12 +- include/sys/crypto/Makefile.am | 16 - include/sys/crypto/api.h | 369 +- include/sys/crypto/common.h | 410 +- include/sys/crypto/icp.h | 2 +- include/sys/dataset_kstats.h | 10 +- include/sys/dbuf.h | 32 +- include/sys/ddt.h | 2 +- include/sys/dmu.h | 49 +- include/sys/dmu_impl.h | 4 +- include/sys/dmu_objset.h | 18 +- include/sys/dmu_recv.h | 6 +- include/sys/dmu_redact.h | 2 +- include/sys/dmu_send.h | 2 +- include/sys/dmu_traverse.h | 2 +- include/sys/dmu_tx.h | 4 +- include/sys/dmu_zfetch.h | 18 +- include/sys/dnode.h | 18 +- include/sys/dsl_bookmark.h | 12 +- include/sys/dsl_crypt.h | 15 +- include/sys/dsl_dataset.h | 36 +- include/sys/dsl_deadlist.h | 2 +- include/sys/dsl_deleg.h | 2 +- include/sys/dsl_destroy.h | 2 +- include/sys/dsl_dir.h | 10 +- include/sys/dsl_pool.h | 16 +- include/sys/dsl_prop.h | 2 +- include/sys/dsl_scan.h | 4 +- include/sys/dsl_synctask.h | 2 +- include/sys/dsl_userhold.h | 2 +- include/sys/efi_partition.h | 22 +- include/sys/fm/Makefile.am | 17 - include/sys/fm/fs/Makefile.am | 14 - include/sys/fm/fs/zfs.h | 2 +- include/sys/fm/protocol.h | 2 +- include/sys/fm/util.h | 2 +- include/sys/fs/Makefile.am | 14 - include/sys/fs/zfs.h | 76 +- include/sys/lua/Makefile.am | 17 - include/sys/metaslab.h | 9 +- include/sys/metaslab_impl.h | 3 +- include/sys/mntent.h | 5 +- include/sys/mod.h | 5 - include/sys/nvpair.h | 2 +- include/sys/nvpair_impl.h | 2 +- include/sys/pathname.h | 2 +- include/sys/qat.h | 2 +- include/sys/range_tree.h | 17 +- include/sys/rrwlock.h | 16 +- include/sys/sa.h | 8 +- include/sys/sa_impl.h | 2 +- include/sys/spa.h | 40 +- include/sys/spa_boot.h | 2 +- include/sys/spa_checkpoint.h | 2 +- include/sys/spa_checksum.h | 2 +- include/sys/spa_impl.h | 5 +- include/sys/spa_log_spacemap.h | 11 +- include/sys/space_map.h | 2 +- include/sys/space_reftree.h | 2 +- include/sys/sysevent.h | 2 +- include/sys/sysevent/Makefile.am | 15 - include/sys/sysevent/dev.h | 2 +- include/sys/sysevent/eventdefs.h | 2 +- include/sys/txg.h | 2 +- include/sys/txg_impl.h | 2 +- include/sys/u8_textprep.h | 2 +- include/sys/u8_textprep_data.h | 2 +- include/sys/uberblock.h | 2 +- include/sys/uberblock_impl.h | 2 +- include/sys/uio_impl.h | 2 +- include/sys/unique.h | 2 +- include/sys/uuid.h | 2 +- include/sys/vdev.h | 4 +- include/sys/vdev_disk.h | 2 +- include/sys/vdev_draid.h | 2 +- include/sys/vdev_file.h | 2 +- include/sys/vdev_impl.h | 2 +- include/sys/vdev_initialize.h | 2 +- include/sys/vdev_raidz.h | 2 +- include/sys/vdev_raidz_impl.h | 2 +- include/sys/vdev_rebuild.h | 2 +- include/sys/vdev_trim.h | 2 +- include/sys/xvattr.h | 6 +- include/sys/zap.h | 9 +- include/sys/zap_impl.h | 13 +- include/sys/zap_leaf.h | 2 +- include/sys/zcp.h | 2 +- include/sys/zcp_iter.h | 2 +- include/sys/zfeature.h | 2 +- include/sys/zfs_acl.h | 2 +- .../sys/stropts.h => include/sys/zfs_chksum.h | 31 +- include/sys/zfs_context.h | 14 +- include/sys/zfs_debug.h | 2 +- include/sys/zfs_delay.h | 2 +- include/sys/zfs_file.h | 2 +- include/sys/zfs_fuid.h | 6 +- include/sys/zfs_ioctl.h | 5 +- include/sys/zfs_ioctl_impl.h | 2 +- include/sys/zfs_onexit.h | 2 +- include/sys/zfs_project.h | 2 +- include/sys/zfs_quota.h | 2 +- include/sys/zfs_racct.h | 2 +- include/sys/zfs_refcount.h | 2 +- include/sys/zfs_rlock.h | 2 +- include/sys/zfs_sa.h | 4 +- include/sys/zfs_stat.h | 2 +- include/sys/zfs_sysfs.h | 2 +- include/sys/zfs_vfsops.h | 2 +- include/sys/zfs_vnops.h | 2 +- include/sys/zfs_znode.h | 8 +- include/sys/zil.h | 69 +- include/sys/zil_impl.h | 13 +- include/sys/zio.h | 28 +- include/sys/zio_checksum.h | 18 +- include/sys/zio_compress.h | 6 +- include/sys/zio_crypt.h | 2 +- include/sys/zio_impl.h | 2 +- include/sys/zrlock.h | 2 +- include/sys/zstd/Makefile.am | 18 - include/sys/zstd/zstd.h | 4 + include/sys/zvol.h | 2 +- include/sys/zvol_impl.h | 2 +- include/thread_pool.h | 3 +- include/zfeature_common.h | 5 +- include/zfs_comutil.h | 4 +- include/zfs_deleg.h | 4 +- include/zfs_fletcher.h | 2 +- include/zfs_namecheck.h | 2 +- include/zfs_prop.h | 16 +- lib/Makefile.am | 105 +- lib/libavl/Makefile.am | 18 +- lib/libefi/Makefile.am | 18 +- lib/libefi/rdwr_efi.c | 18 +- lib/libicp/Makefile.am | 127 +- lib/libnvpair/Makefile.am | 45 +- lib/libnvpair/libnvpair.abi | 2 +- lib/libnvpair/libnvpair.c | 2 +- lib/libnvpair/libnvpair_json.c | 6 +- lib/libnvpair/nvpair_alloc_system.c | 2 +- lib/libshare/Makefile.am | 39 +- lib/libshare/libshare.c | 336 +- lib/libshare/libshare_impl.h | 43 +- lib/libshare/nfs.c | 79 +- lib/libshare/nfs.h | 5 +- lib/libshare/os/freebsd/nfs.c | 71 +- lib/libshare/os/freebsd/smb.c | 60 +- lib/libshare/os/linux/nfs.c | 130 +- lib/libshare/os/linux/smb.c | 156 +- lib/libshare/smb.h | 6 +- lib/libspl/Makefile.am | 62 +- lib/libspl/assert.c | 2 +- lib/libspl/atomic.c | 2 +- lib/libspl/getexecname.c | 2 +- lib/libspl/include/Makefile.am | 112 +- lib/libspl/include/assert.h | 2 +- lib/libspl/include/atomic.h | 2 +- lib/libspl/include/libdevinfo.h | 30 - lib/libspl/include/libgen.h | 2 +- lib/libspl/include/libshare.h | 41 +- lib/libspl/include/limits.h | 45 - lib/libspl/include/locale.h | 35 - lib/libspl/include/os/Makefile.am | 7 - lib/libspl/include/os/freebsd/Makefile.am | 5 - lib/libspl/include/os/freebsd/sys/Makefile.am | 12 - lib/libspl/include/os/freebsd/sys/byteorder.h | 4 +- lib/libspl/include/os/freebsd/sys/file.h | 10 +- lib/libspl/include/os/freebsd/sys/mnttab.h | 6 +- lib/libspl/include/os/freebsd/sys/mount.h | 2 +- lib/libspl/include/os/freebsd/sys/param.h | 2 +- lib/libspl/include/os/freebsd/sys/stat.h | 2 +- .../include/os/freebsd/sys/zfs_context_os.h | 1 - lib/libspl/include/os/linux/Makefile.am | 1 - lib/libspl/include/os/linux/sys/Makefile.am | 10 - lib/libspl/include/os/linux/sys/byteorder.h | 4 +- lib/libspl/include/os/linux/sys/errno.h | 2 +- lib/libspl/include/os/linux/sys/mnttab.h | 4 +- lib/libspl/include/os/linux/sys/mount.h | 2 +- lib/libspl/include/os/linux/sys/param.h | 2 +- lib/libspl/include/os/linux/sys/stat.h | 2 +- lib/libspl/include/os/linux/sys/sysmacros.h | 2 +- .../include/os/linux/sys/zfs_context_os.h | 2 +- lib/libspl/include/rpc/Makefile.am | 3 - lib/libspl/include/rpc/xdr.h | 2 +- lib/libspl/include/statcommon.h | 2 +- lib/libspl/include/stdio.h | 34 - lib/libspl/include/stdlib.h | 2 +- lib/libspl/include/string.h | 2 +- lib/libspl/include/stropts.h | 25 - lib/libspl/include/sys/Makefile.am | 48 - lib/libspl/include/sys/acl.h | 2 +- lib/libspl/include/sys/acl_impl.h | 2 +- lib/libspl/include/sys/callb.h | 2 +- lib/libspl/include/sys/cmn_err.h | 2 +- lib/libspl/include/sys/cred.h | 2 +- lib/libspl/include/sys/debug.h | 2 +- lib/libspl/include/sys/dkio.h | 2 +- lib/libspl/include/sys/dklabel.h | 2 +- lib/libspl/include/sys/dktp/Makefile.am | 4 - lib/libspl/include/sys/dktp/fdisk.h | 2 +- lib/libspl/include/sys/feature_tests.h | 3 +- lib/libspl/include/sys/int_limits.h | 30 - lib/libspl/include/sys/int_types.h | 32 - lib/libspl/include/sys/inttypes.h | 2 +- lib/libspl/include/sys/isa_defs.h | 27 +- lib/libspl/include/sys/kmem.h | 2 +- lib/libspl/include/sys/kstat.h | 47 +- lib/libspl/include/sys/list.h | 2 +- lib/libspl/include/sys/list_impl.h | 2 +- lib/libspl/include/sys/mhd.h | 2 +- lib/libspl/include/sys/mkdev.h | 2 +- lib/libspl/include/sys/policy.h | 2 +- lib/libspl/include/sys/poll.h | 2 +- lib/libspl/include/sys/priv.h | 2 +- lib/libspl/include/sys/processor.h | 2 +- lib/libspl/include/sys/sha2.h | 5 +- lib/libspl/include/sys/simd.h | 21 +- lib/libspl/include/sys/stack.h | 2 +- lib/libspl/include/sys/stdtypes.h | 2 +- lib/libspl/include/sys/string.h | 1 + lib/libspl/include/sys/strings.h | 33 - lib/libspl/include/sys/sunddi.h | 2 +- lib/libspl/include/sys/systeminfo.h | 2 +- lib/libspl/include/sys/time.h | 2 +- lib/libspl/include/sys/types.h | 6 +- lib/libspl/include/sys/types32.h | 2 +- lib/libspl/include/sys/tzfile.h | 164 - lib/libspl/include/sys/uio.h | 2 +- lib/libspl/include/sys/va_list.h | 32 - lib/libspl/include/sys/varargs.h | 30 - lib/libspl/include/sys/vnode.h | 2 +- lib/libspl/include/sys/vtoc.h | 350 - lib/libspl/include/sys/zone.h | 2 +- lib/libspl/include/thread.h | 30 - lib/libspl/include/tzfile.h | 32 - lib/libspl/include/ucred.h | 32 - lib/libspl/include/umem.h | 4 +- lib/libspl/include/unistd.h | 2 +- lib/libspl/include/util/Makefile.am | 3 - lib/libspl/include/util/sscanf.h | 30 - lib/libspl/include/zone.h | 14 +- lib/libspl/libspl_impl.h | 2 +- lib/libspl/list.c | 2 +- lib/libspl/mkdirp.c | 2 +- lib/libspl/os/freebsd/getexecname.c | 2 +- lib/libspl/os/freebsd/gethostid.c | 2 +- lib/libspl/os/freebsd/getmntany.c | 2 +- lib/libspl/os/freebsd/mnttab.c | 18 +- lib/libspl/os/linux/getexecname.c | 2 +- lib/libspl/os/linux/gethostid.c | 2 +- lib/libspl/os/linux/getmntany.c | 2 +- lib/libspl/os/linux/zone.c | 34 +- lib/libspl/page.c | 2 +- lib/libspl/strlcat.c | 2 +- lib/libspl/strlcpy.c | 2 +- lib/libspl/timestamp.c | 7 +- lib/libtpool/Makefile.am | 20 +- lib/libtpool/thread_pool.c | 2 +- lib/libtpool/thread_pool_impl.h | 2 +- lib/libunicode/Makefile.am | 18 +- lib/libuutil/Makefile.am | 34 +- lib/libuutil/libuutil.abi | 195 +- lib/libuutil/uu_alloc.c | 2 +- lib/libuutil/uu_avl.c | 2 +- lib/libuutil/uu_ident.c | 2 +- lib/libuutil/uu_list.c | 2 +- lib/libuutil/uu_misc.c | 2 +- lib/libuutil/uu_pname.c | 8 +- lib/libuutil/uu_string.c | 2 +- lib/libzfs/Makefile.am | 124 +- lib/libzfs/libzfs.abi | 5940 ++-- lib/libzfs/libzfs_changelist.c | 66 +- lib/libzfs/libzfs_config.c | 38 +- lib/libzfs/libzfs_crypto.c | 22 +- lib/libzfs/libzfs_dataset.c | 228 +- lib/libzfs/libzfs_diff.c | 10 +- lib/libzfs/libzfs_impl.h | 49 +- lib/libzfs/libzfs_import.c | 100 +- lib/libzfs/libzfs_iter.c | 23 +- lib/libzfs/libzfs_mount.c | 370 +- lib/libzfs/libzfs_pool.c | 670 +- lib/libzfs/libzfs_sendrecv.c | 416 +- lib/libzfs/libzfs_status.c | 49 +- lib/libzfs/libzfs_util.c | 168 +- lib/libzfs/os/freebsd/libzfs_compat.c | 58 +- lib/libzfs/os/freebsd/libzfs_zmount.c | 19 +- lib/libzfs/os/linux/libzfs_mount_os.c | 41 +- lib/libzfs/os/linux/libzfs_pool_os.c | 18 +- lib/libzfs/os/linux/libzfs_sendrecv_os.c | 52 - lib/libzfs/os/linux/libzfs_util_os.c | 258 +- lib/libzfs_core/Makefile.am | 43 +- lib/libzfs_core/libzfs_core.abi | 217 +- lib/libzfs_core/libzfs_core.c | 261 +- .../os/freebsd/libzfs_core_ioctl.c | 2 +- lib/libzfs_core/os/linux/libzfs_core_ioctl.c | 2 +- lib/libzfsbootenv/Makefile.am | 38 +- lib/libzfsbootenv/libzfsbootenv.abi | 37 +- lib/libzpool/Makefile.am | 405 +- lib/libzpool/kernel.c | 72 +- lib/libzpool/taskq.c | 10 +- lib/libzpool/util.c | 7 +- lib/libzstd/Makefile.am | 51 +- lib/libzutil/Makefile.am | 53 +- .../os/freebsd/zutil_device_path_os.c | 8 +- lib/libzutil/os/freebsd/zutil_import_os.c | 2 +- lib/libzutil/os/linux/zutil_device_path_os.c | 48 +- lib/libzutil/os/linux/zutil_import_os.c | 43 +- lib/libzutil/zutil_device_path.c | 41 +- lib/libzutil/zutil_import.c | 10 +- lib/libzutil/zutil_import.h | 2 +- lib/libzutil/zutil_nicenum.c | 2 +- lib/libzutil/zutil_pool.c | 7 +- man/Makefile.am | 209 +- man/man1/cstyle.1 | 30 +- man/man1/raidz_test.1 | 4 +- .../man => man/man1}/test-runner.1 | 2 + man/man1/zhack.1 | 2 +- man/man1/ztest.1 | 14 +- man/man4/spl.4 | 2 +- man/man4/zfs.4 | 359 +- man/man7/dracut.zfs.7 | 278 + man/man7/vdevprops.7 | 2 +- man/man7/zfsconcepts.7 | 6 +- man/man7/zfsprops.7 | 71 +- man/man7/zpool-features.7 | 190 +- man/man7/zpoolconcepts.7 | 10 +- man/man7/zpoolprops.7 | 4 +- man/man8/fsck.zfs.8 | 2 +- man/man8/mount.zfs.8 | 2 +- man/man8/zdb.8 | 53 +- man/man8/zed.8.in | 2 +- man/man8/zfs-allow.8 | 134 +- man/man8/zfs-bookmark.8 | 12 +- man/man8/zfs-clone.8 | 30 +- man/man8/zfs-create.8 | 37 +- man/man8/zfs-destroy.8 | 52 +- man/man8/zfs-diff.8 | 23 +- man/man8/zfs-hold.8 | 2 +- man/man8/zfs-jail.8 | 2 +- man/man8/zfs-list.8 | 33 +- man/man8/zfs-load-key.8 | 2 +- man/man8/zfs-mount.8 | 2 +- man/man8/zfs-program.8 | 11 +- man/man8/zfs-project.8 | 2 +- man/man8/zfs-promote.8 | 24 +- man/man8/zfs-receive.8 | 70 +- man/man8/zfs-rename.8 | 41 +- man/man8/zfs-rollback.8 | 15 +- man/man8/zfs-send.8 | 70 +- man/man8/zfs-set.8 | 135 +- man/man8/zfs-share.8 | 2 +- man/man8/zfs-snapshot.8 | 62 +- man/man8/zfs-unzone.8 | 1 + man/man8/zfs-upgrade.8 | 2 +- man/man8/zfs-userspace.8 | 2 +- man/man8/zfs-wait.8 | 2 +- man/man8/zfs-zone.8 | 116 + man/man8/zfs.8 | 136 +- man/man8/zfs_ids_to_path.8 | 2 +- man/man8/zgenhostid.8 | 2 +- man/man8/zinject.8 | 10 +- man/man8/zpool-add.8 | 27 +- man/man8/zpool-attach.8 | 2 +- man/man8/zpool-checkpoint.8 | 2 +- man/man8/zpool-clear.8 | 2 +- man/man8/zpool-create.8 | 36 +- man/man8/zpool-destroy.8 | 13 +- man/man8/zpool-detach.8 | 2 +- man/man8/zpool-events.8 | 2 +- man/man8/zpool-export.8 | 13 +- man/man8/zpool-get.8 | 2 +- man/man8/zpool-history.8 | 2 +- man/man8/zpool-import.8 | 31 +- man/man8/zpool-initialize.8 | 2 +- man/man8/zpool-iostat.8 | 44 +- man/man8/zpool-labelclear.8 | 5 +- man/man8/zpool-list.8 | 38 +- man/man8/zpool-offline.8 | 2 +- man/man8/zpool-reguid.8 | 2 +- man/man8/zpool-remove.8 | 48 +- man/man8/zpool-reopen.8 | 2 +- man/man8/zpool-replace.8 | 2 +- man/man8/zpool-resilver.8 | 2 +- man/man8/zpool-scrub.8 | 16 +- man/man8/zpool-split.8 | 2 +- man/man8/zpool-status.8 | 41 +- man/man8/zpool-sync.8 | 2 +- man/man8/zpool-trim.8 | 23 +- man/man8/zpool-upgrade.8 | 15 +- man/man8/zpool-wait.8 | 2 +- man/man8/zpool.8 | 111 +- man/man8/zstream.8 | 57 +- module/Kbuild.in | 459 +- module/Makefile.bsd | 83 +- module/Makefile.in | 62 +- module/avl/Makefile.in | 10 - module/avl/avl.c | 48 +- module/icp/Makefile.in | 93 - module/icp/algs/aes/aes_impl.c | 16 +- module/icp/algs/aes/aes_impl_aesni.c | 2 +- module/icp/algs/aes/aes_impl_generic.c | 2 +- module/icp/algs/aes/aes_impl_x86-64.c | 2 +- module/icp/algs/aes/aes_modes.c | 2 +- module/icp/algs/blake3/blake3.c | 732 + module/icp/algs/blake3/blake3_generic.c | 202 + module/icp/algs/blake3/blake3_impl.c | 284 + module/icp/algs/blake3/blake3_impl.h | 213 + module/icp/algs/blake3/blake3_x86-64.c | 248 + module/icp/algs/edonr/edonr.c | 47 +- module/icp/algs/modes/cbc.c | 49 +- module/icp/algs/modes/ccm.c | 104 +- module/icp/algs/modes/ctr.c | 23 +- module/icp/algs/modes/ecb.c | 16 +- module/icp/algs/modes/gcm.c | 106 +- module/icp/algs/modes/gcm_generic.c | 2 +- module/icp/algs/modes/gcm_pclmulqdq.c | 2 +- module/icp/algs/modes/modes.c | 4 +- module/icp/algs/sha2/sha2.c | 18 +- module/icp/algs/skein/skein.c | 136 +- module/icp/algs/skein/skein_impl.h | 2 +- module/icp/algs/skein/skein_port.h | 4 +- module/icp/api/kcf_cipher.c | 795 +- module/icp/api/kcf_ctxops.c | 12 +- module/icp/api/kcf_digest.c | 491 - module/icp/api/kcf_mac.c | 418 +- module/icp/api/kcf_miscapi.c | 127 - .../icp/asm-aarch64/blake3/b3_aarch64_sse2.S | 2450 ++ .../icp/asm-aarch64/blake3/b3_aarch64_sse41.S | 2463 ++ module/icp/asm-ppc64/blake3/b3_ppc64le_sse2.S | 2823 ++ .../icp/asm-ppc64/blake3/b3_ppc64le_sse41.S | 3064 ++ module/icp/asm-x86_64/aes/aes_aesni.S | 24 +- module/icp/asm-x86_64/aes/aes_amd64.S | 4 +- module/icp/asm-x86_64/aes/aestab2.h | 2 +- module/icp/asm-x86_64/blake3/blake3_avx2.S | 1845 + module/icp/asm-x86_64/blake3/blake3_avx512.S | 2618 ++ module/icp/asm-x86_64/blake3/blake3_sse2.S | 2323 ++ module/icp/asm-x86_64/blake3/blake3_sse41.S | 2058 ++ .../icp/asm-x86_64/modes/aesni-gcm-x86_64.S | 6 +- module/icp/asm-x86_64/modes/gcm_pclmulqdq.S | 4 +- module/icp/asm-x86_64/sha2/sha256_impl.S | 2 +- module/icp/asm-x86_64/sha2/sha512_impl.S | 2 +- module/icp/core/kcf_callprov.c | 1447 +- module/icp/core/kcf_mech_tabs.c | 543 +- module/icp/core/kcf_prov_lib.c | 74 +- module/icp/core/kcf_prov_tabs.c | 381 +- module/icp/core/kcf_sched.c | 1620 +- module/icp/illumos-crypto.c | 14 +- module/icp/include/aes/aes_impl.h | 9 +- module/icp/include/modes/gcm_impl.h | 2 +- module/icp/include/modes/modes.h | 8 +- module/icp/include/sha2/sha2_consts.h | 2 +- module/icp/include/sha2/sha2_impl.h | 2 +- module/icp/include/sys/asm_linkage.h | 2 +- module/icp/include/sys/bitmap.h | 183 - module/icp/include/sys/crypto/elfsign.h | 137 - module/icp/include/sys/crypto/impl.h | 1059 +- module/icp/include/sys/crypto/ops_impl.h | 630 - module/icp/include/sys/crypto/sched_impl.h | 424 +- module/icp/include/sys/crypto/spi.h | 574 +- module/icp/include/sys/ia32/asm_linkage.h | 8 +- module/icp/include/sys/ia32/stack.h | 2 +- module/icp/include/sys/ia32/trap.h | 2 +- module/icp/include/sys/modhash.h | 147 - module/icp/include/sys/modhash_impl.h | 108 - module/icp/include/sys/stack.h | 2 +- module/icp/include/sys/trap.h | 2 +- module/icp/io/aes.c | 279 +- module/icp/io/sha2_mod.c | 240 +- module/icp/io/skein_mod.c | 171 +- module/icp/os/modhash.c | 927 - module/icp/spi/kcf_spi.c | 745 +- module/lua/Makefile.in | 39 - module/lua/lapi.c | 23 - module/lua/ldo.c | 13 +- module/lua/setjmp/setjmp_i386.S | 2 +- module/lua/setjmp/setjmp_x86_64.S | 12 +- module/nvpair/Makefile.in | 13 - module/nvpair/fnvpair.c | 2 +- module/nvpair/nvpair.c | 77 +- module/nvpair/nvpair_alloc_fixed.c | 2 +- module/nvpair/nvpair_alloc_spl.c | 2 +- module/os/freebsd/spl/acl_common.c | 5 +- module/os/freebsd/spl/callb.c | 4 +- module/os/freebsd/spl/list.c | 2 +- module/os/freebsd/spl/sha256c.c | 4 +- module/os/freebsd/spl/sha512c.c | 8 +- module/os/freebsd/spl/spl_acl.c | 8 +- module/os/freebsd/spl/spl_cmn_err.c | 2 +- module/os/freebsd/spl/spl_kmem.c | 2 +- module/os/freebsd/spl/spl_misc.c | 4 - module/os/freebsd/spl/spl_string.c | 2 +- module/os/freebsd/spl/spl_sunddi.c | 13 - module/os/freebsd/spl/spl_sysevent.c | 12 +- module/os/freebsd/spl/spl_uio.c | 2 +- module/os/freebsd/spl/spl_vfs.c | 2 +- module/os/freebsd/spl/spl_zlib.c | 13 +- module/os/freebsd/spl/spl_zone.c | 4 +- module/os/freebsd/zfs/abd_os.c | 10 +- module/os/freebsd/zfs/arc_os.c | 2 +- module/os/freebsd/zfs/crypto_os.c | 87 +- module/os/freebsd/zfs/dmu_os.c | 12 +- module/os/freebsd/zfs/hkdf.c | 4 +- module/os/freebsd/zfs/spa_os.c | 26 +- module/os/freebsd/zfs/vdev_file.c | 2 +- module/os/freebsd/zfs/vdev_geom.c | 8 +- module/os/freebsd/zfs/vdev_label_os.c | 2 +- module/os/freebsd/zfs/zfs_acl.c | 42 +- module/os/freebsd/zfs/zfs_ctldir.c | 8 +- module/os/freebsd/zfs/zfs_debug.c | 2 +- module/os/freebsd/zfs/zfs_dir.c | 2 +- module/os/freebsd/zfs/zfs_ioctl_compat.c | 6 +- module/os/freebsd/zfs/zfs_vfsops.c | 17 +- module/os/freebsd/zfs/zfs_vnops_os.c | 374 +- module/os/freebsd/zfs/zfs_znode.c | 22 +- module/os/freebsd/zfs/zio_crypt.c | 155 +- module/os/freebsd/zfs/zvol_os.c | 27 +- module/os/linux/spl/Makefile.in | 17 - module/os/linux/spl/spl-cred.c | 42 +- module/os/linux/spl/spl-err.c | 3 + module/os/linux/spl/spl-generic.c | 67 +- module/os/linux/spl/spl-kmem-cache.c | 4 +- module/os/linux/spl/spl-procfs-list.c | 2 +- module/os/linux/spl/spl-taskq.c | 7 - module/os/linux/spl/spl-thread.c | 9 - module/os/linux/spl/spl-trace.c | 2 +- module/os/linux/spl/spl-zone.c | 424 + module/os/linux/zfs/Makefile.in | 38 - module/os/linux/zfs/abd_os.c | 18 +- module/os/linux/zfs/arc_os.c | 2 +- module/os/linux/zfs/mmp_os.c | 2 +- module/os/linux/zfs/policy.c | 14 +- module/os/linux/zfs/qat.c | 2 +- module/os/linux/zfs/qat_compress.c | 2 +- module/os/linux/zfs/qat_crypt.c | 12 +- module/os/linux/zfs/spa_misc_os.c | 26 +- module/os/linux/zfs/trace.c | 2 +- module/os/linux/zfs/vdev_disk.c | 109 +- module/os/linux/zfs/vdev_file.c | 2 +- module/os/linux/zfs/zfs_acl.c | 104 +- module/os/linux/zfs/zfs_ctldir.c | 6 +- module/os/linux/zfs/zfs_debug.c | 2 +- module/os/linux/zfs/zfs_dir.c | 6 +- module/os/linux/zfs/zfs_file_os.c | 2 +- module/os/linux/zfs/zfs_ioctl_os.c | 113 +- module/os/linux/zfs/zfs_sysfs.c | 51 +- module/os/linux/zfs/zfs_uio.c | 172 +- module/os/linux/zfs/zfs_vfsops.c | 41 +- module/os/linux/zfs/zfs_vnops_os.c | 92 +- module/os/linux/zfs/zfs_znode.c | 26 +- module/os/linux/zfs/zio_crypt.c | 178 +- module/os/linux/zfs/zpl_ctldir.c | 50 +- module/os/linux/zfs/zpl_export.c | 2 +- module/os/linux/zfs/zpl_file.c | 236 +- module/os/linux/zfs/zpl_inode.c | 6 +- module/os/linux/zfs/zpl_super.c | 15 +- module/os/linux/zfs/zpl_xattr.c | 176 +- module/os/linux/zfs/zvol_os.c | 670 +- module/spl/Makefile.in | 13 - module/unicode/Makefile.in | 11 - module/unicode/u8_textprep.c | 25 +- module/unicode/uconv.c | 2 +- module/zcommon/Makefile.in | 28 - module/zcommon/zfeature_common.c | 32 +- module/zcommon/zfs_comutil.c | 10 +- module/zcommon/zfs_deleg.c | 4 +- module/zcommon/zfs_fletcher.c | 6 +- module/zcommon/zfs_fletcher_aarch64_neon.c | 4 +- module/zcommon/zfs_fletcher_avx512.c | 6 +- module/zcommon/zfs_fletcher_intel.c | 4 +- module/zcommon/zfs_fletcher_sse.c | 4 +- module/zcommon/zfs_fletcher_superscalar.c | 4 +- module/zcommon/zfs_fletcher_superscalar4.c | 4 +- module/zcommon/zfs_namecheck.c | 2 +- module/zcommon/zfs_prop.c | 124 +- module/zcommon/zpool_prop.c | 128 +- module/zcommon/zprop_common.c | 37 +- module/zfs/Makefile.in | 158 - module/zfs/abd.c | 2 +- module/zfs/aggsum.c | 2 +- module/zfs/arc.c | 193 +- module/zfs/blake3_zfs.c | 117 + module/zfs/blkptr.c | 2 +- module/zfs/bplist.c | 2 +- module/zfs/bpobj.c | 8 +- module/zfs/bptree.c | 2 +- module/zfs/btree.c | 772 +- module/zfs/dataset_kstats.c | 51 +- module/zfs/dbuf.c | 117 +- module/zfs/dbuf_stats.c | 6 +- module/zfs/ddt.c | 32 +- module/zfs/ddt_zap.c | 2 +- module/zfs/dmu.c | 44 +- module/zfs/dmu_diff.c | 2 +- module/zfs/dmu_object.c | 6 +- module/zfs/dmu_objset.c | 36 +- module/zfs/dmu_recv.c | 433 +- module/zfs/dmu_redact.c | 8 +- module/zfs/dmu_send.c | 44 +- module/zfs/dmu_traverse.c | 6 +- module/zfs/dmu_tx.c | 61 +- module/zfs/dmu_zfetch.c | 187 +- module/zfs/dnode.c | 79 +- module/zfs/dnode_sync.c | 16 +- module/zfs/dsl_bookmark.c | 41 +- module/zfs/dsl_crypt.c | 64 +- module/zfs/dsl_dataset.c | 170 +- module/zfs/dsl_deadlist.c | 4 +- module/zfs/dsl_deleg.c | 2 +- module/zfs/dsl_destroy.c | 7 +- module/zfs/dsl_dir.c | 32 +- module/zfs/dsl_pool.c | 41 +- module/zfs/dsl_prop.c | 12 +- module/zfs/dsl_scan.c | 436 +- module/zfs/dsl_synctask.c | 2 +- module/zfs/dsl_userhold.c | 6 +- module/zfs/edonr_zfs.c | 9 +- module/zfs/fm.c | 2 +- module/zfs/gzip.c | 7 +- module/zfs/hkdf.c | 16 +- module/zfs/lzjb.c | 2 +- module/zfs/metaslab.c | 167 +- module/zfs/mmp.c | 11 +- module/zfs/pathname.c | 2 +- module/zfs/range_tree.c | 87 +- module/zfs/refcount.c | 2 +- module/zfs/rrwlock.c | 24 +- module/zfs/sa.c | 27 +- module/zfs/sha256.c | 2 +- module/zfs/skein_zfs.c | 14 +- module/zfs/spa.c | 72 +- module/zfs/spa_boot.c | 2 +- module/zfs/spa_checkpoint.c | 4 +- module/zfs/spa_config.c | 4 +- module/zfs/spa_errlog.c | 1020 +- module/zfs/spa_history.c | 3 +- module/zfs/spa_log_spacemap.c | 239 +- module/zfs/spa_misc.c | 25 +- module/zfs/spa_stats.c | 50 +- module/zfs/space_map.c | 9 +- module/zfs/space_reftree.c | 2 +- module/zfs/txg.c | 14 +- module/zfs/uberblock.c | 2 +- module/zfs/unique.c | 2 +- module/zfs/vdev.c | 26 +- module/zfs/vdev_cache.c | 2 +- module/zfs/vdev_draid.c | 4 +- module/zfs/vdev_indirect.c | 6 +- module/zfs/vdev_indirect_births.c | 2 +- module/zfs/vdev_indirect_mapping.c | 4 +- module/zfs/vdev_initialize.c | 4 +- module/zfs/vdev_label.c | 4 +- module/zfs/vdev_mirror.c | 137 +- module/zfs/vdev_missing.c | 2 +- module/zfs/vdev_queue.c | 2 +- module/zfs/vdev_raidz.c | 29 +- module/zfs/vdev_raidz_math.c | 4 +- module/zfs/vdev_raidz_math_aarch64_neon.c | 2 +- .../zfs/vdev_raidz_math_aarch64_neon_common.h | 2 +- module/zfs/vdev_raidz_math_aarch64_neonx2.c | 2 +- module/zfs/vdev_raidz_math_avx2.c | 2 +- module/zfs/vdev_raidz_math_avx512bw.c | 2 +- module/zfs/vdev_raidz_math_avx512f.c | 2 +- module/zfs/vdev_raidz_math_impl.h | 2 +- module/zfs/vdev_raidz_math_powerpc_altivec.c | 2 +- .../vdev_raidz_math_powerpc_altivec_common.h | 2 +- module/zfs/vdev_raidz_math_scalar.c | 2 +- module/zfs/vdev_raidz_math_sse2.c | 2 +- module/zfs/vdev_raidz_math_ssse3.c | 2 +- module/zfs/vdev_rebuild.c | 18 +- module/zfs/vdev_removal.c | 20 +- module/zfs/vdev_root.c | 2 +- module/zfs/vdev_trim.c | 20 +- module/zfs/zap.c | 14 +- module/zfs/zap_leaf.c | 11 +- module/zfs/zap_micro.c | 28 +- module/zfs/zcp.c | 2 +- module/zfs/zcp_get.c | 12 +- module/zfs/zcp_synctask.c | 2 +- module/zfs/zfeature.c | 9 +- module/zfs/zfs_byteswap.c | 15 +- module/zfs/zfs_chksum.c | 334 + module/zfs/zfs_fm.c | 4 +- module/zfs/zfs_fuid.c | 18 +- module/zfs/zfs_ioctl.c | 144 +- module/zfs/zfs_log.c | 121 +- module/zfs/zfs_onexit.c | 2 +- module/zfs/zfs_quota.c | 2 +- module/zfs/zfs_ratelimit.c | 2 +- module/zfs/zfs_replay.c | 102 +- module/zfs/zfs_rlock.c | 2 +- module/zfs/zfs_sa.c | 42 +- module/zfs/zfs_vnops.c | 22 +- module/zfs/zil.c | 345 +- module/zfs/zio.c | 61 +- module/zfs/zio_checksum.c | 12 +- module/zfs/zio_compress.c | 6 +- module/zfs/zio_inject.c | 12 +- module/zfs/zle.c | 2 +- module/zfs/zrlock.c | 2 +- module/zfs/zthr.c | 2 +- module/zfs/zvol.c | 9 +- module/zstd/Makefile.in | 39 - module/zstd/README.md | 43 +- module/zstd/include/string.h | 1 + module/zstd/include/zstd_compat_wrapper.h | 611 +- module/zstd/lib/common/bitstream.h | 454 + module/zstd/lib/common/compiler.h | 175 + module/zstd/lib/common/cpu.h | 215 + module/zstd/lib/common/debug.h | 114 + module/zstd/lib/common/entropy_common.c | 216 + module/zstd/lib/common/error_private.c | 55 + module/zstd/lib/common/error_private.h | 80 + module/zstd/lib/common/fse.h | 688 + module/zstd/lib/common/fse_decompress.c | 286 + module/zstd/lib/common/huf.h | 340 + module/zstd/lib/common/mem.h | 450 + module/zstd/lib/common/pool.c | 344 + module/zstd/lib/common/pool.h | 84 + module/zstd/lib/common/xxhash.c | 864 + module/zstd/lib/common/xxhash.h | 285 + module/zstd/lib/common/zstd_common.c | 83 + module/zstd/lib/{ => common}/zstd_errors.h | 0 module/zstd/lib/common/zstd_internal.h | 446 + module/zstd/lib/compress/fse_compress.c | 698 + module/zstd/lib/compress/hist.c | 183 + module/zstd/lib/compress/hist.h | 75 + module/zstd/lib/compress/huf_compress.c | 798 + module/zstd/lib/compress/zstd_compress.c | 4278 +++ .../lib/compress/zstd_compress_internal.h | 1125 + .../lib/compress/zstd_compress_literals.c | 158 + .../lib/compress/zstd_compress_literals.h | 29 + .../lib/compress/zstd_compress_sequences.c | 419 + .../lib/compress/zstd_compress_sequences.h | 54 + .../lib/compress/zstd_compress_superblock.c | 845 + .../lib/compress/zstd_compress_superblock.h | 32 + module/zstd/lib/compress/zstd_cwksp.h | 525 + module/zstd/lib/compress/zstd_double_fast.c | 521 + module/zstd/lib/compress/zstd_double_fast.h | 38 + module/zstd/lib/compress/zstd_fast.c | 496 + module/zstd/lib/compress/zstd_fast.h | 37 + module/zstd/lib/compress/zstd_lazy.c | 1138 + module/zstd/lib/compress/zstd_lazy.h | 67 + module/zstd/lib/compress/zstd_ldm.c | 619 + module/zstd/lib/compress/zstd_ldm.h | 110 + module/zstd/lib/compress/zstd_opt.c | 1200 + module/zstd/lib/compress/zstd_opt.h | 56 + module/zstd/lib/decompress/huf_decompress.c | 1248 + module/zstd/lib/decompress/zstd_ddict.c | 244 + module/zstd/lib/decompress/zstd_ddict.h | 44 + module/zstd/lib/decompress/zstd_decompress.c | 1885 ++ .../lib/decompress/zstd_decompress_block.c | 1432 + .../lib/decompress/zstd_decompress_block.h | 59 + .../lib/decompress/zstd_decompress_internal.h | 189 + module/zstd/lib/zstd.c | 27821 ---------------- module/zstd/lib/zstd.h | 37 +- module/zstd/zfs_zstd.c | 142 +- rpm/.gitignore | 1 + rpm/Makefile.am | 9 +- rpm/generic/.gitignore | 3 - rpm/generic/Makefile.am | 1 - rpm/generic/zfs-dkms.spec.in | 49 +- rpm/generic/zfs-kmod.spec.in | 8 +- rpm/generic/zfs.spec.in | 85 +- rpm/redhat/.gitignore | 3 - rpm/redhat/Makefile.am | 1 - rpm/redhat/zfs-kmod.spec.in | 8 +- scripts/Makefile.am | 143 +- scripts/cstyle.pl | 80 +- scripts/dkms.mkconf | 62 +- scripts/kmodtool | 74 +- scripts/zfs-helpers.sh | 49 +- scripts/zfs-tests-color.sh | 27 + scripts/zfs-tests.sh | 177 +- scripts/zfs.sh | 120 +- scripts/zfs2zol-patch.sed | 2 +- scripts/zimport.sh | 4 +- scripts/zloop.sh | 22 +- tests/Makefile.am | 32 +- tests/README.md | 4 + tests/runfiles/Makefile.am | 9 - tests/runfiles/common.run | 57 +- tests/runfiles/freebsd.run | 4 - tests/runfiles/linux.run | 20 +- tests/runfiles/sanity.run | 10 +- tests/test-runner/Makefile.am | 1 - tests/test-runner/bin/Makefile.am | 8 - tests/test-runner/bin/test-runner.py.in | 69 +- tests/test-runner/bin/zts-report.py.in | 137 +- tests/test-runner/include/Makefile.am | 5 - tests/test-runner/include/logapi.shlib | 146 +- tests/test-runner/man/Makefile.am | 4 - tests/zfs-tests/.gitignore | 1 + tests/zfs-tests/Makefile.am | 49 +- tests/zfs-tests/callbacks/Makefile.am | 6 - tests/zfs-tests/cmd/.gitignore | 48 + tests/zfs-tests/cmd/Makefile.am | 164 +- tests/zfs-tests/cmd/{badsend => }/badsend.c | 2 +- tests/zfs-tests/cmd/badsend/.gitignore | 1 - tests/zfs-tests/cmd/badsend/Makefile.am | 11 - .../cmd/{btree_test => }/btree_test.c | 0 tests/zfs-tests/cmd/btree_test/.gitignore | 1 - tests/zfs-tests/cmd/btree_test/Makefile.am | 32 - tests/zfs-tests/cmd/checksum/blake3_test.c | 575 + .../functional => cmd}/checksum/edonr_test.c | 7 +- .../functional => cmd}/checksum/sha2_test.c | 7 +- .../functional => cmd}/checksum/skein_test.c | 9 +- .../cmd/{chg_usr_exec => }/chg_usr_exec.c | 2 +- tests/zfs-tests/cmd/chg_usr_exec/.gitignore | 1 - tests/zfs-tests/cmd/chg_usr_exec/Makefile.am | 6 - .../functional/cp_files => cmd}/cp_files.c | 0 .../{tests/functional/ctime => cmd}/ctime.c | 9 +- .../cmd/{devname2devid => }/devname2devid.c | 2 +- tests/zfs-tests/cmd/devname2devid/.gitignore | 1 - tests/zfs-tests/cmd/devname2devid/Makefile.am | 10 - .../cmd/{dir_rd_update => }/dir_rd_update.c | 4 +- tests/zfs-tests/cmd/dir_rd_update/.gitignore | 1 - tests/zfs-tests/cmd/dir_rd_update/Makefile.am | 6 - .../acl/off => cmd}/dosmode_readonly_write.c | 11 + tests/zfs-tests/cmd/{draid => }/draid.c | 55 +- tests/zfs-tests/cmd/draid/.gitignore | 1 - tests/zfs-tests/cmd/draid/Makefile.am | 15 - .../cli_root/zpool_events => cmd}/ereports.c | 2 +- tests/zfs-tests/cmd/file/file_append.c | 206 + .../cmd/{file_check => file}/file_check.c | 4 +- tests/zfs-tests/cmd/{ => file}/file_common.h | 4 +- .../cmd/{file_trunc => file}/file_trunc.c | 2 +- .../cmd/{file_write => file}/file_write.c | 6 +- .../cmd/{largest_file => file}/largest_file.c | 4 +- .../{randfree_file => file}/randfree_file.c | 4 +- .../{randwritecomp => file}/randwritecomp.c | 19 +- tests/zfs-tests/cmd/file_check/.gitignore | 1 - tests/zfs-tests/cmd/file_check/Makefile.am | 6 - tests/zfs-tests/cmd/file_trunc/.gitignore | 1 - tests/zfs-tests/cmd/file_trunc/Makefile.am | 6 - tests/zfs-tests/cmd/file_write/.gitignore | 1 - tests/zfs-tests/cmd/file_write/Makefile.am | 6 - tests/zfs-tests/cmd/{get_diff => }/get_diff.c | 5 +- tests/zfs-tests/cmd/get_diff/.gitignore | 1 - tests/zfs-tests/cmd/get_diff/Makefile.am | 6 - .../cmd/{getversion => }/getversion.c | 0 tests/zfs-tests/cmd/getversion/.gitignore | 1 - tests/zfs-tests/cmd/getversion/Makefile.am | 6 - tests/zfs-tests/cmd/largest_file/.gitignore | 1 - tests/zfs-tests/cmd/largest_file/Makefile.am | 6 - .../libzfs_input_check.c | 16 +- .../cmd/libzfs_input_check/.gitignore | 1 - .../cmd/libzfs_input_check/Makefile.am | 17 - .../cmd/linux_dos_attributes/dos_attributes.h | 82 + .../read_dos_attributes.c | 60 + .../write_dos_attributes.c | 95 + tests/zfs-tests/cmd/{mkbusy => }/mkbusy.c | 6 +- tests/zfs-tests/cmd/mkbusy/.gitignore | 1 - tests/zfs-tests/cmd/mkbusy/Makefile.am | 6 - tests/zfs-tests/cmd/{mkfile => }/mkfile.c | 17 +- tests/zfs-tests/cmd/mkfile/.gitignore | 1 - tests/zfs-tests/cmd/mkfile/Makefile.am | 8 - tests/zfs-tests/cmd/{mkfiles => }/mkfiles.c | 18 +- tests/zfs-tests/cmd/mkfiles/.gitignore | 1 - tests/zfs-tests/cmd/mkfiles/Makefile.am | 6 - tests/zfs-tests/cmd/{mktree => }/mktree.c | 4 +- tests/zfs-tests/cmd/mktree/.gitignore | 1 - tests/zfs-tests/cmd/mktree/Makefile.am | 6 - .../zfs-tests/cmd/{mmap_exec => }/mmap_exec.c | 2 +- tests/zfs-tests/cmd/mmap_exec/.gitignore | 1 - tests/zfs-tests/cmd/mmap_exec/Makefile.am | 6 - .../cmd/{mmap_libaio => }/mmap_libaio.c | 2 +- tests/zfs-tests/cmd/mmap_libaio/.gitignore | 1 - tests/zfs-tests/cmd/mmap_libaio/Makefile.am | 10 - .../zfs-tests/cmd/{mmap_seek => }/mmap_seek.c | 2 +- tests/zfs-tests/cmd/mmap_seek/.gitignore | 1 - tests/zfs-tests/cmd/mmap_seek/Makefile.am | 6 - tests/zfs-tests/cmd/mmap_sync.c | 152 + .../zfs-tests/cmd/{mmapwrite => }/mmapwrite.c | 2 +- tests/zfs-tests/cmd/mmapwrite/.gitignore | 1 - tests/zfs-tests/cmd/mmapwrite/Makefile.am | 7 - .../cmd/{nvlist_to_lua => }/nvlist_to_lua.c | 13 +- tests/zfs-tests/cmd/nvlist_to_lua/.gitignore | 1 - tests/zfs-tests/cmd/nvlist_to_lua/Makefile.am | 10 - tests/zfs-tests/cmd/randfree_file/.gitignore | 1 - tests/zfs-tests/cmd/randfree_file/Makefile.am | 6 - tests/zfs-tests/cmd/randwritecomp/.gitignore | 1 - tests/zfs-tests/cmd/randwritecomp/Makefile.am | 9 - tests/zfs-tests/cmd/{readmmap => }/readmmap.c | 4 +- tests/zfs-tests/cmd/readmmap/.gitignore | 1 - tests/zfs-tests/cmd/readmmap/Makefile.am | 6 - .../cmd/{rename_dir => }/rename_dir.c | 17 +- tests/zfs-tests/cmd/rename_dir/.gitignore | 1 - tests/zfs-tests/cmd/rename_dir/Makefile.am | 6 - .../rm_lnkcnt_zero_file.c | 3 +- .../cmd/rm_lnkcnt_zero_file/.gitignore | 1 - .../cmd/rm_lnkcnt_zero_file/Makefile.am | 7 - .../cmd/{send_doall => }/send_doall.c | 2 +- tests/zfs-tests/cmd/send_doall/.gitignore | 1 - tests/zfs-tests/cmd/send_doall/Makefile.am | 11 - .../zfs-tests/cmd/{stride_dd => }/stride_dd.c | 2 +- tests/zfs-tests/cmd/stride_dd/.gitignore | 1 - tests/zfs-tests/cmd/stride_dd/Makefile.am | 7 - .../suid => cmd}/suid_write_to_file.c | 2 +- .../cmd/{threadsappend => }/threadsappend.c | 2 +- tests/zfs-tests/cmd/threadsappend/.gitignore | 1 - tests/zfs-tests/cmd/threadsappend/Makefile.am | 7 - .../truncate => cmd}/truncate_test.c | 0 .../cmd/{user_ns_exec => }/user_ns_exec.c | 2 +- tests/zfs-tests/cmd/user_ns_exec/.gitignore | 1 - tests/zfs-tests/cmd/user_ns_exec/Makefile.am | 6 - .../zfs-tests/cmd/{xattrtest => }/xattrtest.c | 20 +- tests/zfs-tests/cmd/xattrtest/.gitignore | 1 - tests/zfs-tests/cmd/xattrtest/Makefile.am | 6 - .../events => cmd}/zed_fd_spill-zedlet.c | 0 .../socket.c => cmd/zfs_diff-socket.c} | 0 tests/zfs-tests/include/Makefile.am | 14 - tests/zfs-tests/include/blkdev.shlib | 119 +- tests/zfs-tests/include/commands.cfg | 54 +- tests/zfs-tests/include/default.cfg.in | 8 +- tests/zfs-tests/include/libtest.shlib | 1444 +- tests/zfs-tests/include/math.shlib | 11 +- tests/zfs-tests/include/properties.shlib | 32 +- tests/zfs-tests/include/tunables.cfg | 3 + tests/zfs-tests/include/zpool_script.shlib | 8 +- tests/zfs-tests/tests/Makefile.am | 1993 +- tests/zfs-tests/tests/functional/Makefile.am | 94 - .../tests/functional/acl/Makefile.am | 6 - tests/zfs-tests/tests/functional/acl/acl.cfg | 2 +- .../tests/functional/acl/acl_common.kshlib | 387 +- .../tests/functional/acl/off/.gitignore | 1 - .../tests/functional/acl/off/Makefile.am | 16 - .../tests/functional/acl/off/cleanup.ksh | 2 +- .../tests/functional/acl/off/dosmode.ksh | 100 +- .../tests/functional/acl/off/posixmode.ksh | 2 +- .../tests/functional/acl/off/setup.ksh | 2 +- .../tests/functional/acl/posix-sa/Makefile.am | 8 - .../tests/functional/acl/posix-sa/cleanup.ksh | 2 +- .../tests/functional/acl/posix-sa/setup.ksh | 2 +- .../tests/functional/acl/posix/Makefile.am | 8 - .../tests/functional/acl/posix/cleanup.ksh | 2 +- .../functional/acl/posix/posix_001_pos.ksh | 70 +- .../functional/acl/posix/posix_002_pos.ksh | 13 +- .../functional/acl/posix/posix_003_pos.ksh | 14 +- .../functional/acl/posix/posix_004_pos.ksh | 2 +- .../tests/functional/acl/posix/setup.ksh | 2 +- .../tests/functional/alloc_class/Makefile.am | 21 - .../alloc_class/alloc_class_004_pos.ksh | 3 +- .../alloc_class/alloc_class_005_pos.ksh | 4 +- .../alloc_class/alloc_class_010_pos.ksh | 2 +- .../alloc_class/alloc_class_011_neg.ksh | 2 +- .../tests/functional/append/cleanup.ksh | 29 + .../tests/functional/append/file_append.ksh | 73 + .../tests/functional/append/setup.ksh | 31 + .../threadsappend_001_pos.ksh | 2 +- .../tests/functional/arc/Makefile.am | 8 - .../tests/functional/arc/cleanup.ksh | 2 +- .../functional/arc/dbufstats_001_pos.ksh | 5 +- .../functional/arc/dbufstats_002_pos.ksh | 2 +- .../functional/arc/dbufstats_003_pos.ksh | 2 +- .../zfs-tests/tests/functional/arc/setup.ksh | 4 +- .../tests/functional/atime/Makefile.am | 14 - .../tests/functional/atime/atime.cfg | 2 +- .../tests/functional/atime/atime_001_pos.ksh | 2 +- .../tests/functional/atime/atime_002_neg.ksh | 2 +- .../tests/functional/atime/atime_003_pos.ksh | 2 +- .../functional/atime/atime_common.kshlib | 2 +- .../tests/functional/atime/cleanup.ksh | 2 +- .../tests/functional/atime/root_atime_off.ksh | 2 +- .../tests/functional/atime/root_atime_on.ksh | 2 +- .../functional/atime/root_relatime_on.ksh | 2 +- .../tests/functional/atime/setup.ksh | 2 +- .../tests/functional/bootfs/Makefile.am | 12 - .../functional/bootfs/bootfs_001_pos.ksh | 8 +- .../functional/bootfs/bootfs_002_neg.ksh | 8 +- .../functional/bootfs/bootfs_003_pos.ksh | 10 +- .../functional/bootfs/bootfs_004_neg.ksh | 8 +- .../functional/bootfs/bootfs_005_neg.ksh | 2 +- .../functional/bootfs/bootfs_006_pos.ksh | 12 +- .../functional/bootfs/bootfs_007_pos.ksh | 2 +- .../functional/bootfs/bootfs_008_pos.ksh | 2 +- .../tests/functional/bootfs/cleanup.ksh | 2 +- .../tests/functional/bootfs/setup.ksh | 2 +- .../tests/functional/btree/Makefile.am | 20 - .../tests/functional/btree/btree_negative.ksh | 7 +- .../tests/functional/cache/Makefile.am | 20 - .../tests/functional/cache/cache.cfg | 2 +- .../tests/functional/cache/cache.kshlib | 10 +- .../tests/functional/cache/cache_001_pos.ksh | 2 +- .../tests/functional/cache/cache_002_pos.ksh | 2 +- .../tests/functional/cache/cache_003_pos.ksh | 2 +- .../tests/functional/cache/cache_004_neg.ksh | 2 +- .../tests/functional/cache/cache_005_neg.ksh | 2 +- .../tests/functional/cache/cache_006_pos.ksh | 2 +- .../tests/functional/cache/cache_007_neg.ksh | 2 +- .../tests/functional/cache/cache_008_neg.ksh | 2 +- .../tests/functional/cache/cache_009_pos.ksh | 2 +- .../tests/functional/cache/cache_010_pos.ksh | 2 +- .../tests/functional/cache/cache_011_pos.ksh | 2 +- .../tests/functional/cache/cache_012_pos.ksh | 2 + .../tests/functional/cache/cleanup.ksh | 2 +- .../tests/functional/cache/setup.ksh | 2 +- .../tests/functional/cachefile/Makefile.am | 12 - .../tests/functional/cachefile/cachefile.cfg | 4 +- .../functional/cachefile/cachefile.kshlib | 2 +- .../cachefile/cachefile_001_pos.ksh | 2 +- .../cachefile/cachefile_002_pos.ksh | 2 +- .../cachefile/cachefile_003_pos.ksh | 2 +- .../cachefile/cachefile_004_pos.ksh | 2 +- .../tests/functional/cachefile/cleanup.ksh | 2 +- .../tests/functional/cachefile/setup.ksh | 2 +- .../tests/functional/casenorm/Makefile.am | 25 - .../tests/functional/casenorm/casenorm.kshlib | 14 +- .../casenorm/mixed_create_failure.ksh | 22 +- .../functional/channel_program/Makefile.am | 6 - .../channel_program/channel_common.kshlib | 35 +- .../channel_program/lua_core/Makefile.am | 46 - .../channel_program/lua_core/tst.exists.ksh | 4 +- .../lua_core/tst.integer_illegal.ksh | 4 +- .../lua_core/tst.integer_overflow.ksh | 4 +- .../lua_core/tst.language_functions_neg.ksh | 4 +- .../lua_core/tst.language_functions_pos.ksh | 4 +- .../lua_core/tst.return_large.ksh | 3 +- .../lua_core/tst.return_nvlist_neg.ksh | 8 +- .../lua_core/tst.return_nvlist_pos.ksh | 8 +- .../channel_program/lua_core/tst.timeout.ksh | 14 +- .../channel_program/synctask_core/Makefile.am | 53 - .../synctask_core/tst.terminate_by_signal.ksh | 13 +- .../tests/functional/chattr/Makefile.am | 6 - .../functional/chattr/chattr_001_pos.ksh | 2 +- .../functional/chattr/chattr_002_neg.ksh | 2 +- .../tests/functional/chattr/cleanup.ksh | 2 +- .../tests/functional/chattr/setup.ksh | 6 +- .../tests/functional/checksum/.gitignore | 4 - .../tests/functional/checksum/Makefile.am | 31 - .../tests/functional/checksum/cleanup.ksh | 2 +- .../tests/functional/checksum/default.cfg | 4 +- .../functional/checksum/filetest_001_pos.ksh | 7 +- .../functional/checksum/filetest_002_pos.ksh | 2 +- .../run_blake3_test.ksh} | 16 +- .../functional/checksum/run_edonr_test.ksh | 2 +- .../functional/checksum/run_sha2_test.ksh | 2 +- .../functional/checksum/run_skein_test.ksh | 2 +- .../tests/functional/checksum/setup.ksh | 2 +- .../tests/functional/clean_mirror/Makefile.am | 12 - .../clean_mirror/clean_mirror_001_pos.ksh | 2 +- .../clean_mirror/clean_mirror_002_pos.ksh | 2 +- .../clean_mirror/clean_mirror_003_pos.ksh | 2 +- .../clean_mirror/clean_mirror_004_pos.ksh | 2 +- .../clean_mirror/clean_mirror_common.kshlib | 2 +- .../tests/functional/clean_mirror/cleanup.ksh | 2 +- .../tests/functional/clean_mirror/default.cfg | 2 +- .../tests/functional/clean_mirror/setup.ksh | 2 +- .../tests/functional/cli_root/Makefile.am | 67 - .../functional/cli_root/cli_common.kshlib | 16 +- .../tests/functional/cli_root/zdb/Makefile.am | 20 - .../functional/cli_root/zdb/zdb_003_pos.ksh | 4 +- .../functional/cli_root/zdb/zdb_004_pos.ksh | 6 +- .../functional/cli_root/zdb/zdb_args_neg.ksh | 2 +- .../functional/cli_root/zdb/zdb_args_pos.ksh | 10 +- .../cli_root/zdb/zdb_block_size_histogram.ksh | 9 +- .../functional/cli_root/zdb/zdb_checksum.ksh | 6 +- .../cli_root/zdb/zdb_decompress.ksh | 24 +- .../cli_root/zdb/zdb_decompress_zstd.ksh | 30 +- .../cli_root/zdb/zdb_display_block.ksh | 14 +- .../cli_root/zdb/zdb_object_range_neg.ksh | 2 +- .../cli_root/zdb/zdb_object_range_pos.ksh | 9 +- .../functional/cli_root/zdb/zdb_objset_id.ksh | 56 +- .../functional/cli_root/zdb/zdb_recover_2.ksh | 2 +- .../tests/functional/cli_root/zfs/Makefile.am | 7 - .../tests/functional/cli_root/zfs/cleanup.ksh | 2 +- .../tests/functional/cli_root/zfs/setup.ksh | 2 +- .../functional/cli_root/zfs/zfs_001_neg.ksh | 2 +- .../functional/cli_root/zfs/zfs_002_pos.ksh | 50 +- .../functional/cli_root/zfs/zfs_003_neg.ksh | 16 +- .../cli_root/zfs_bookmark/Makefile.am | 5 - .../cli_root/zfs_bookmark/cleanup.ksh | 2 +- .../cli_root/zfs_bookmark/setup.ksh | 2 +- .../zfs_bookmark/zfs_bookmark_cliargs.ksh | 4 +- .../cli_root/zfs_change-key/Makefile.am | 12 - .../cli_root/zfs_change-key/cleanup.ksh | 2 +- .../cli_root/zfs_change-key/setup.ksh | 2 +- .../functional/cli_root/zfs_clone/Makefile.am | 17 - .../functional/cli_root/zfs_clone/cleanup.ksh | 2 +- .../functional/cli_root/zfs_clone/setup.ksh | 2 +- .../cli_root/zfs_clone/zfs_clone_001_neg.ksh | 2 +- .../cli_root/zfs_clone/zfs_clone_002_pos.ksh | 2 +- .../cli_root/zfs_clone/zfs_clone_003_pos.ksh | 3 +- .../cli_root/zfs_clone/zfs_clone_004_pos.ksh | 2 +- .../cli_root/zfs_clone/zfs_clone_005_pos.ksh | 2 +- .../cli_root/zfs_clone/zfs_clone_006_pos.ksh | 2 +- .../cli_root/zfs_clone/zfs_clone_007_pos.ksh | 5 +- .../cli_root/zfs_clone/zfs_clone_008_neg.ksh | 2 +- .../cli_root/zfs_clone/zfs_clone_009_neg.ksh | 2 +- .../cli_root/zfs_clone/zfs_clone_010_pos.ksh | 2 +- .../cli_root/zfs_copies/Makefile.am | 14 - .../cli_root/zfs_copies/cleanup.ksh | 2 +- .../functional/cli_root/zfs_copies/setup.ksh | 2 +- .../cli_root/zfs_copies/zfs_copies.cfg | 2 +- .../cli_root/zfs_copies/zfs_copies.kshlib | 2 +- .../zfs_copies/zfs_copies_001_pos.ksh | 2 +- .../zfs_copies/zfs_copies_002_pos.ksh | 10 +- .../zfs_copies/zfs_copies_003_pos.ksh | 2 +- .../zfs_copies/zfs_copies_004_neg.ksh | 2 +- .../zfs_copies/zfs_copies_005_neg.ksh | 2 +- .../zfs_copies/zfs_copies_006_pos.ksh | 2 +- .../cli_root/zfs_create/Makefile.am | 28 - .../cli_root/zfs_create/cleanup.ksh | 2 +- .../cli_root/zfs_create/properties.kshlib | 2 +- .../functional/cli_root/zfs_create/setup.ksh | 2 +- .../cli_root/zfs_create/zfs_create.cfg | 2 +- .../zfs_create/zfs_create_001_pos.ksh | 2 +- .../zfs_create/zfs_create_002_pos.ksh | 5 +- .../zfs_create/zfs_create_003_pos.ksh | 2 +- .../zfs_create/zfs_create_004_pos.ksh | 17 +- .../zfs_create/zfs_create_005_pos.ksh | 19 +- .../zfs_create/zfs_create_006_pos.ksh | 2 +- .../zfs_create/zfs_create_007_pos.ksh | 2 +- .../zfs_create/zfs_create_008_neg.ksh | 4 +- .../zfs_create/zfs_create_009_neg.ksh | 7 +- .../zfs_create/zfs_create_010_neg.ksh | 7 +- .../zfs_create/zfs_create_011_pos.ksh | 2 +- .../zfs_create/zfs_create_012_pos.ksh | 5 +- .../zfs_create/zfs_create_013_pos.ksh | 5 +- .../zfs_create/zfs_create_014_pos.ksh | 2 +- .../zfs_create/zfs_create_common.kshlib | 15 +- .../cli_root/zfs_create/zfs_create_dryrun.ksh | 7 +- .../zfs_create/zfs_create_nomount.ksh | 2 +- .../zfs_create/zfs_create_verbose.ksh | 7 +- .../cli_root/zfs_destroy/Makefile.am | 30 - .../cli_root/zfs_destroy/cleanup.ksh | 2 +- .../functional/cli_root/zfs_destroy/setup.ksh | 6 +- ...fs_clone_livelist_condense_and_disable.ksh | 6 + .../zfs_clone_livelist_condense_races.ksh | 6 + .../zfs_destroy/zfs_clone_livelist_dedup.ksh | 7 + .../cli_root/zfs_destroy/zfs_destroy.cfg | 2 +- .../zfs_destroy/zfs_destroy_001_pos.ksh | 2 +- .../zfs_destroy/zfs_destroy_002_pos.ksh | 2 +- .../zfs_destroy/zfs_destroy_003_pos.ksh | 2 +- .../zfs_destroy/zfs_destroy_004_pos.ksh | 5 +- .../zfs_destroy/zfs_destroy_005_neg.ksh | 2 +- .../zfs_destroy/zfs_destroy_006_neg.ksh | 2 +- .../zfs_destroy/zfs_destroy_007_neg.ksh | 2 +- .../zfs_destroy/zfs_destroy_008_pos.ksh | 2 +- .../zfs_destroy/zfs_destroy_009_pos.ksh | 2 +- .../zfs_destroy/zfs_destroy_010_pos.ksh | 2 +- .../zfs_destroy_clone_livelist.ksh | 6 + .../zfs_destroy/zfs_destroy_common.kshlib | 2 +- .../zfs_destroy_dev_removal_condense.ksh | 2 +- .../functional/cli_root/zfs_diff/.gitignore | 1 - .../functional/cli_root/zfs_diff/Makefile.am | 18 - .../cli_root/zfs_diff/zfs_diff_changes.ksh | 2 +- .../cli_root/zfs_diff/zfs_diff_timestamp.ksh | 4 +- .../cli_root/zfs_diff/zfs_diff_types.ksh | 10 +- .../functional/cli_root/zfs_get/Makefile.am | 18 - .../functional/cli_root/zfs_get/cleanup.ksh | 2 +- .../functional/cli_root/zfs_get/setup.ksh | 2 +- .../cli_root/zfs_get/zfs_get_001_pos.ksh | 23 +- .../cli_root/zfs_get/zfs_get_002_pos.ksh | 8 +- .../cli_root/zfs_get/zfs_get_003_pos.ksh | 7 +- .../cli_root/zfs_get/zfs_get_004_pos.ksh | 11 +- .../cli_root/zfs_get/zfs_get_005_neg.ksh | 16 +- .../cli_root/zfs_get/zfs_get_006_neg.ksh | 5 +- .../cli_root/zfs_get/zfs_get_007_neg.ksh | 4 +- .../cli_root/zfs_get/zfs_get_008_pos.ksh | 9 +- .../cli_root/zfs_get/zfs_get_009_pos.ksh | 13 +- .../cli_root/zfs_get/zfs_get_010_neg.ksh | 2 +- .../cli_root/zfs_get/zfs_get_common.kshlib | 2 +- .../cli_root/zfs_get/zfs_get_list_d.kshlib | 2 +- .../cli_root/zfs_ids_to_path/Makefile.am | 5 - .../cli_root/zfs_ids_to_path/cleanup.ksh | 2 +- .../cli_root/zfs_ids_to_path/setup.ksh | 2 +- .../zfs_ids_to_path_001_pos.ksh | 2 +- .../cli_root/zfs_inherit/Makefile.am | 8 - .../cli_root/zfs_inherit/cleanup.ksh | 2 +- .../functional/cli_root/zfs_inherit/setup.ksh | 2 +- .../zfs_inherit/zfs_inherit_001_neg.ksh | 11 +- .../zfs_inherit/zfs_inherit_002_neg.ksh | 2 +- .../zfs_inherit/zfs_inherit_003_pos.ksh | 2 +- .../functional/cli_root/zfs_jail/Makefile.am | 6 - .../functional/cli_root/zfs_jail/cleanup.ksh | 2 +- .../functional/cli_root/zfs_jail/setup.ksh | 2 +- .../cli_root/zfs_jail/zfs_jail_001_pos.ksh | 2 +- .../cli_root/zfs_load-key/Makefile.am | 18 - .../cli_root/zfs_load-key/cleanup.ksh | 2 +- .../cli_root/zfs_load-key/setup.ksh | 2 +- .../cli_root/zfs_load-key/zfs_load-key.ksh | 3 +- .../zfs_load-key/zfs_load-key_all.ksh | 3 +- .../zfs_load-key/zfs_load-key_common.kshlib | 3 +- .../functional/cli_root/zfs_mount/Makefile.am | 29 - .../functional/cli_root/zfs_mount/cleanup.ksh | 2 +- .../functional/cli_root/zfs_mount/setup.ksh | 2 +- .../cli_root/zfs_mount/zfs_mount.cfg | 2 +- .../cli_root/zfs_mount/zfs_mount.kshlib | 7 +- .../cli_root/zfs_mount/zfs_mount_001_pos.ksh | 2 +- .../cli_root/zfs_mount/zfs_mount_002_pos.ksh | 2 +- .../cli_root/zfs_mount/zfs_mount_003_pos.ksh | 2 +- .../cli_root/zfs_mount/zfs_mount_004_pos.ksh | 2 +- .../cli_root/zfs_mount/zfs_mount_005_pos.ksh | 13 +- .../cli_root/zfs_mount/zfs_mount_006_pos.ksh | 2 +- .../cli_root/zfs_mount/zfs_mount_007_pos.ksh | 7 +- .../cli_root/zfs_mount/zfs_mount_008_pos.ksh | 2 +- .../cli_root/zfs_mount/zfs_mount_009_neg.ksh | 2 +- .../cli_root/zfs_mount/zfs_mount_010_neg.ksh | 2 +- .../cli_root/zfs_mount/zfs_mount_011_neg.ksh | 2 +- .../zfs_mount/zfs_mount_all_001_pos.ksh | 2 +- .../cli_root/zfs_mount/zfs_mount_remount.ksh | 2 +- .../cli_root/zfs_mount/zfs_multi_mount.ksh | 4 +- .../cli_root/zfs_program/Makefile.am | 5 - .../cli_root/zfs_program/cleanup.ksh | 2 +- .../functional/cli_root/zfs_program/setup.ksh | 2 +- .../cli_root/zfs_promote/Makefile.am | 16 - .../cli_root/zfs_promote/cleanup.ksh | 2 +- .../functional/cli_root/zfs_promote/setup.ksh | 2 +- .../cli_root/zfs_promote/zfs_promote.cfg | 2 +- .../zfs_promote/zfs_promote_001_pos.ksh | 2 +- .../zfs_promote/zfs_promote_002_pos.ksh | 2 +- .../zfs_promote/zfs_promote_003_pos.ksh | 2 +- .../zfs_promote/zfs_promote_004_pos.ksh | 2 +- .../zfs_promote/zfs_promote_005_pos.ksh | 2 +- .../zfs_promote/zfs_promote_006_neg.ksh | 2 +- .../zfs_promote/zfs_promote_007_neg.ksh | 2 +- .../zfs_promote/zfs_promote_008_pos.ksh | 2 +- .../cli_root/zfs_property/Makefile.am | 5 - .../cli_root/zfs_property/cleanup.ksh | 2 +- .../cli_root/zfs_property/setup.ksh | 2 +- .../zfs_written_property_001_pos.ksh | 5 +- .../cli_root/zfs_receive/Makefile.am | 34 - .../cli_root/zfs_receive/cleanup.ksh | 2 +- .../zfs_receive/receive-o-x_props_aliases.ksh | 2 +- .../receive-o-x_props_override.ksh | 2 +- .../functional/cli_root/zfs_receive/setup.ksh | 2 +- .../zfs_receive_-wR-encrypted-mix.ksh | 2 +- .../zfs_receive/zfs_receive_001_pos.ksh | 7 +- .../zfs_receive/zfs_receive_002_pos.ksh | 2 +- .../zfs_receive/zfs_receive_003_pos.ksh | 4 +- .../zfs_receive/zfs_receive_004_neg.ksh | 2 +- .../zfs_receive/zfs_receive_005_neg.ksh | 6 +- .../zfs_receive/zfs_receive_006_pos.ksh | 4 +- .../zfs_receive/zfs_receive_007_neg.ksh | 4 +- .../zfs_receive/zfs_receive_008_pos.ksh | 5 +- .../zfs_receive/zfs_receive_009_neg.ksh | 9 +- .../zfs_receive/zfs_receive_010_pos.ksh | 19 +- .../zfs_receive/zfs_receive_013_pos.ksh | 2 +- .../zfs_receive_compressed_corrective.ksh | 193 + .../zfs_receive/zfs_receive_corrective.ksh | 192 + .../zfs_receive/zfs_receive_from_zstd.ksh | 28 +- .../zfs_receive/zfs_receive_new_props.ksh | 4 +- .../cli_root/zfs_rename/Makefile.am | 26 - .../cli_root/zfs_rename/cleanup.ksh | 10 +- .../functional/cli_root/zfs_rename/setup.ksh | 14 +- .../cli_root/zfs_rename/zfs_rename.cfg | 2 +- .../cli_root/zfs_rename/zfs_rename.kshlib | 9 +- .../zfs_rename/zfs_rename_001_pos.ksh | 2 +- .../zfs_rename/zfs_rename_002_pos.ksh | 2 +- .../zfs_rename/zfs_rename_003_pos.ksh | 2 +- .../zfs_rename/zfs_rename_004_neg.ksh | 2 +- .../zfs_rename/zfs_rename_005_neg.ksh | 2 +- .../zfs_rename/zfs_rename_006_pos.ksh | 2 +- .../zfs_rename/zfs_rename_007_pos.ksh | 2 +- .../zfs_rename/zfs_rename_008_pos.ksh | 2 +- .../zfs_rename/zfs_rename_009_neg.ksh | 2 +- .../zfs_rename/zfs_rename_010_neg.ksh | 2 +- .../zfs_rename/zfs_rename_011_pos.ksh | 2 +- .../zfs_rename/zfs_rename_012_neg.ksh | 2 +- .../zfs_rename/zfs_rename_013_pos.ksh | 2 +- .../zfs_rename/zfs_rename_014_neg.ksh | 8 +- .../zfs_rename/zfs_rename_nounmount.ksh | 11 +- .../cli_root/zfs_reservation/Makefile.am | 6 - .../cli_root/zfs_reservation/cleanup.ksh | 2 +- .../cli_root/zfs_reservation/setup.ksh | 2 +- .../zfs_reservation_001_pos.ksh | 2 +- .../zfs_reservation_002_pos.ksh | 14 +- .../cli_root/zfs_rollback/Makefile.am | 12 - .../cli_root/zfs_rollback/cleanup.ksh | 2 +- .../cli_root/zfs_rollback/setup.ksh | 2 +- .../cli_root/zfs_rollback/zfs_rollback.cfg | 2 +- .../zfs_rollback/zfs_rollback_001_pos.ksh | 2 +- .../zfs_rollback/zfs_rollback_002_pos.ksh | 2 +- .../zfs_rollback/zfs_rollback_003_neg.ksh | 2 +- .../zfs_rollback/zfs_rollback_004_neg.ksh | 2 +- .../zfs_rollback/zfs_rollback_common.kshlib | 23 +- .../functional/cli_root/zfs_send/Makefile.am | 20 - .../functional/cli_root/zfs_send/cleanup.ksh | 2 +- .../functional/cli_root/zfs_send/setup.ksh | 2 +- .../cli_root/zfs_send/zfs_send-b.ksh | 10 +- .../functional/cli_root/zfs_send/zfs_send.cfg | 2 +- .../cli_root/zfs_send/zfs_send_001_pos.ksh | 10 +- .../cli_root/zfs_send/zfs_send_002_pos.ksh | 10 +- .../cli_root/zfs_send/zfs_send_003_pos.ksh | 4 +- .../cli_root/zfs_send/zfs_send_004_neg.ksh | 4 +- .../cli_root/zfs_send/zfs_send_005_pos.ksh | 4 +- .../cli_root/zfs_send/zfs_send_006_pos.ksh | 4 +- .../cli_root/zfs_send/zfs_send_007_pos.ksh | 11 +- .../cli_root/zfs_send/zfs_send_encrypted.ksh | 10 +- .../zfs_send/zfs_send_encrypted_unloaded.ksh | 2 +- .../cli_root/zfs_send/zfs_send_raw.ksh | 14 +- .../zfs_send/zfs_send_skip_missing.ksh | 7 +- .../functional/cli_root/zfs_set/Makefile.am | 35 - .../cli_root/zfs_set/cache_001_pos.ksh | 2 +- .../cli_root/zfs_set/cache_002_neg.ksh | 2 +- .../cli_root/zfs_set/canmount_001_pos.ksh | 12 +- .../cli_root/zfs_set/canmount_002_pos.ksh | 12 +- .../cli_root/zfs_set/canmount_003_pos.ksh | 6 +- .../cli_root/zfs_set/canmount_004_pos.ksh | 2 +- .../cli_root/zfs_set/checksum_001_pos.ksh | 4 +- .../functional/cli_root/zfs_set/cleanup.ksh | 2 +- .../cli_root/zfs_set/compression_001_pos.ksh | 2 +- .../cli_root/zfs_set/mountpoint_001_pos.ksh | 6 +- .../cli_root/zfs_set/mountpoint_002_pos.ksh | 6 +- .../cli_root/zfs_set/mountpoint_003_pos.ksh | 11 +- .../cli_root/zfs_set/onoffs_001_pos.ksh | 2 +- .../zfs_set/property_alias_001_pos.ksh | 10 +- .../cli_root/zfs_set/readonly_001_pos.ksh | 2 +- .../cli_root/zfs_set/reservation_001_neg.ksh | 12 +- .../cli_root/zfs_set/ro_props_001_pos.ksh | 9 +- .../functional/cli_root/zfs_set/setup.ksh | 2 +- .../cli_root/zfs_set/share_mount_001_neg.ksh | 2 +- .../cli_root/zfs_set/snapdir_001_pos.ksh | 27 +- .../zfs_set/user_property_001_pos.ksh | 2 +- .../zfs_set/user_property_002_pos.ksh | 2 +- .../zfs_set/user_property_003_neg.ksh | 2 +- .../zfs_set/user_property_004_pos.ksh | 25 +- .../cli_root/zfs_set/version_001_neg.ksh | 8 +- .../cli_root/zfs_set/zfs_set_001_neg.ksh | 8 +- .../cli_root/zfs_set/zfs_set_002_neg.ksh | 2 +- .../cli_root/zfs_set/zfs_set_003_neg.ksh | 2 +- .../cli_root/zfs_set/zfs_set_common.kshlib | 77 +- .../zfs_set/zfs_set_feature_activation.ksh | 2 +- .../functional/cli_root/zfs_share/Makefile.am | 21 - .../functional/cli_root/zfs_share/cleanup.ksh | 2 +- .../functional/cli_root/zfs_share/setup.ksh | 2 +- .../cli_root/zfs_share/zfs_share.cfg | 2 +- .../cli_root/zfs_share/zfs_share_001_pos.ksh | 2 +- .../cli_root/zfs_share/zfs_share_002_pos.ksh | 2 +- .../cli_root/zfs_share/zfs_share_003_pos.ksh | 2 +- .../cli_root/zfs_share/zfs_share_004_pos.ksh | 2 +- .../cli_root/zfs_share/zfs_share_005_pos.ksh | 11 +- .../cli_root/zfs_share/zfs_share_006_pos.ksh | 2 +- .../cli_root/zfs_share/zfs_share_007_neg.ksh | 7 +- .../cli_root/zfs_share/zfs_share_008_neg.ksh | 2 +- .../cli_root/zfs_share/zfs_share_009_neg.ksh | 5 +- .../cli_root/zfs_share/zfs_share_010_neg.ksh | 2 +- .../cli_root/zfs_share/zfs_share_011_pos.ksh | 16 +- .../cli_root/zfs_share/zfs_share_012_pos.ksh | 2 +- .../cli_root/zfs_share/zfs_share_013_pos.ksh | 26 +- .../zfs_share/zfs_share_concurrent_shares.ksh | 135 +- .../cli_root/zfs_snapshot/Makefile.am | 16 - .../cli_root/zfs_snapshot/cleanup.ksh | 2 +- .../cli_root/zfs_snapshot/setup.ksh | 2 +- .../cli_root/zfs_snapshot/zfs_snapshot.cfg | 2 +- .../zfs_snapshot/zfs_snapshot_001_neg.ksh | 2 +- .../zfs_snapshot/zfs_snapshot_002_neg.ksh | 2 +- .../zfs_snapshot/zfs_snapshot_003_neg.ksh | 2 +- .../zfs_snapshot/zfs_snapshot_004_neg.ksh | 8 +- .../zfs_snapshot/zfs_snapshot_005_neg.ksh | 6 +- .../zfs_snapshot/zfs_snapshot_006_pos.ksh | 2 +- .../zfs_snapshot/zfs_snapshot_007_neg.ksh | 9 +- .../zfs_snapshot/zfs_snapshot_008_neg.ksh | 2 +- .../zfs_snapshot/zfs_snapshot_009_pos.ksh | 15 +- .../functional/cli_root/zfs_sysfs/Makefile.am | 10 - .../functional/cli_root/zfs_sysfs/cleanup.ksh | 2 +- .../functional/cli_root/zfs_sysfs/setup.ksh | 2 +- .../zfs_sysfs/zfeature_set_unsupported.ksh | 2 +- .../zfs_sysfs/zfs_get_unsupported.ksh | 2 +- .../zfs_sysfs/zfs_set_unsupported.ksh | 2 +- .../cli_root/zfs_sysfs/zfs_sysfs_live.ksh | 2 +- .../zfs_sysfs/zpool_get_unsupported.ksh | 2 +- .../zfs_sysfs/zpool_set_unsupported.ksh | 2 +- .../cli_root/zfs_unload-key/Makefile.am | 7 - .../cli_root/zfs_unload-key/cleanup.ksh | 2 +- .../cli_root/zfs_unload-key/setup.ksh | 2 +- .../zfs_unload-key/zfs_unload-key_all.ksh | 3 +- .../cli_root/zfs_unmount/Makefile.am | 20 - .../cli_root/zfs_unmount/cleanup.ksh | 2 +- .../functional/cli_root/zfs_unmount/setup.ksh | 2 +- .../cli_root/zfs_unmount/zfs_unmount.cfg | 2 +- .../cli_root/zfs_unmount/zfs_unmount.kshlib | 4 +- .../zfs_unmount/zfs_unmount_001_pos.ksh | 2 +- .../zfs_unmount/zfs_unmount_002_pos.ksh | 2 +- .../zfs_unmount/zfs_unmount_003_pos.ksh | 2 +- .../zfs_unmount/zfs_unmount_004_pos.ksh | 2 +- .../zfs_unmount/zfs_unmount_005_pos.ksh | 2 +- .../zfs_unmount/zfs_unmount_006_pos.ksh | 3 +- .../zfs_unmount/zfs_unmount_007_neg.ksh | 2 +- .../zfs_unmount/zfs_unmount_008_neg.ksh | 2 +- .../zfs_unmount/zfs_unmount_009_pos.ksh | 2 +- .../zfs_unmount/zfs_unmount_all_001_pos.ksh | 2 +- .../zfs_unmount/zfs_unmount_unload_keys.ksh | 2 +- .../cli_root/zfs_unshare/Makefile.am | 11 - .../cli_root/zfs_unshare/cleanup.ksh | 2 +- .../functional/cli_root/zfs_unshare/setup.ksh | 40 +- .../zfs_unshare/zfs_unshare_001_pos.ksh | 6 +- .../zfs_unshare/zfs_unshare_002_pos.ksh | 5 +- .../zfs_unshare/zfs_unshare_003_pos.ksh | 2 +- .../zfs_unshare/zfs_unshare_004_neg.ksh | 2 +- .../zfs_unshare/zfs_unshare_005_neg.ksh | 2 +- .../zfs_unshare/zfs_unshare_006_pos.ksh | 25 +- .../zfs_unshare/zfs_unshare_007_pos.ksh | 2 +- .../zfs_unshare/zfs_unshare_008_pos.ksh | 58 + .../cli_root/zfs_upgrade/Makefile.am | 14 - .../cli_root/zfs_upgrade/cleanup.ksh | 2 +- .../functional/cli_root/zfs_upgrade/setup.ksh | 2 +- .../cli_root/zfs_upgrade/zfs_upgrade.kshlib | 2 +- .../zfs_upgrade/zfs_upgrade_001_pos.ksh | 22 +- .../zfs_upgrade/zfs_upgrade_002_pos.ksh | 8 +- .../zfs_upgrade/zfs_upgrade_003_pos.ksh | 2 +- .../zfs_upgrade/zfs_upgrade_004_pos.ksh | 2 +- .../zfs_upgrade/zfs_upgrade_005_pos.ksh | 2 +- .../zfs_upgrade/zfs_upgrade_006_neg.ksh | 2 +- .../zfs_upgrade/zfs_upgrade_007_neg.ksh | 2 +- .../functional/cli_root/zfs_wait/Makefile.am | 8 - .../cli_root/zfs_wait/zfs_wait_getsubopt.ksh | 14 + .../functional/cli_root/zhack/Makefile.am | 3 - .../functional/cli_root/zpool/Makefile.am | 8 - .../functional/cli_root/zpool/cleanup.ksh | 2 +- .../tests/functional/cli_root/zpool/setup.ksh | 2 +- .../cli_root/zpool/zpool_001_neg.ksh | 2 +- .../cli_root/zpool/zpool_002_pos.ksh | 41 +- .../cli_root/zpool/zpool_003_pos.ksh | 41 +- .../cli_root/zpool/zpool_colors.ksh | 16 +- .../functional/cli_root/zpool_add/Makefile.am | 22 - .../cli_root/zpool_add/add-o_ashift.ksh | 18 +- .../zpool_add/add_nested_replacing_spare.ksh | 6 +- .../cli_root/zpool_add/add_prop_ashift.ksh | 18 +- .../functional/cli_root/zpool_add/cleanup.ksh | 2 +- .../functional/cli_root/zpool_add/setup.ksh | 2 +- .../cli_root/zpool_add/zpool_add.cfg | 2 +- .../cli_root/zpool_add/zpool_add.kshlib | 14 +- .../cli_root/zpool_add/zpool_add_001_pos.ksh | 2 +- .../cli_root/zpool_add/zpool_add_002_pos.ksh | 2 +- .../cli_root/zpool_add/zpool_add_003_pos.ksh | 8 +- .../cli_root/zpool_add/zpool_add_004_pos.ksh | 2 +- .../cli_root/zpool_add/zpool_add_005_pos.ksh | 2 +- .../cli_root/zpool_add/zpool_add_006_pos.ksh | 2 +- .../cli_root/zpool_add/zpool_add_007_neg.ksh | 2 +- .../cli_root/zpool_add/zpool_add_008_neg.ksh | 2 +- .../cli_root/zpool_add/zpool_add_009_neg.ksh | 2 +- .../cli_root/zpool_add/zpool_add_010_pos.ksh | 2 +- .../zpool_add/zpool_add_dryrun_output.ksh | 2 +- .../cli_root/zpool_attach/Makefile.am | 6 - .../cli_root/zpool_attach/attach-o_ashift.ksh | 17 +- .../cli_root/zpool_attach/cleanup.ksh | 2 +- .../cli_root/zpool_attach/setup.ksh | 2 +- .../zpool_attach/zpool_attach_001_neg.ksh | 2 +- .../cli_root/zpool_clear/Makefile.am | 11 - .../cli_root/zpool_clear/cleanup.ksh | 2 +- .../functional/cli_root/zpool_clear/setup.ksh | 2 +- .../cli_root/zpool_clear/zpool_clear.cfg | 2 +- .../zpool_clear/zpool_clear_001_pos.ksh | 101 +- .../zpool_clear/zpool_clear_002_neg.ksh | 2 +- .../zpool_clear/zpool_clear_003_neg.ksh | 2 +- .../zpool_clear/zpool_clear_readonly.ksh | 2 +- .../cli_root/zpool_create/Makefile.am | 50 - .../cli_root/zpool_create/cleanup.ksh | 2 +- .../cli_root/zpool_create/create-o_ashift.ksh | 14 +- .../cli_root/zpool_create/setup.ksh | 2 +- .../cli_root/zpool_create/zpool_create.cfg | 2 +- .../cli_root/zpool_create/zpool_create.shlib | 9 +- .../zpool_create/zpool_create_001_pos.ksh | 2 +- .../zpool_create/zpool_create_002_pos.ksh | 2 +- .../zpool_create/zpool_create_003_pos.ksh | 2 +- .../zpool_create/zpool_create_004_pos.ksh | 2 +- .../zpool_create/zpool_create_005_pos.ksh | 6 +- .../zpool_create/zpool_create_006_pos.ksh | 2 +- .../zpool_create/zpool_create_007_neg.ksh | 2 +- .../zpool_create/zpool_create_008_pos.ksh | 2 +- .../zpool_create/zpool_create_009_neg.ksh | 2 +- .../zpool_create/zpool_create_010_neg.ksh | 2 +- .../zpool_create/zpool_create_011_neg.ksh | 2 +- .../zpool_create/zpool_create_012_neg.ksh | 9 +- .../zpool_create/zpool_create_014_neg.ksh | 2 +- .../zpool_create/zpool_create_015_neg.ksh | 2 +- .../zpool_create/zpool_create_016_pos.ksh | 10 +- .../zpool_create/zpool_create_017_neg.ksh | 2 +- .../zpool_create/zpool_create_018_pos.ksh | 2 +- .../zpool_create/zpool_create_019_pos.ksh | 2 +- .../zpool_create/zpool_create_020_pos.ksh | 17 +- .../zpool_create/zpool_create_021_pos.ksh | 18 +- .../zpool_create/zpool_create_022_pos.ksh | 17 +- .../zpool_create/zpool_create_023_neg.ksh | 4 +- .../zpool_create/zpool_create_024_pos.ksh | 27 +- .../zpool_create_draid_001_pos.ksh | 2 +- .../zpool_create_draid_002_pos.ksh | 2 +- .../zpool_create_draid_003_pos.ksh | 2 +- .../zpool_create_draid_004_pos.ksh | 2 +- .../zpool_create_dryrun_output.ksh | 2 +- .../zpool_create_features_001_pos.ksh | 2 +- .../zpool_create_features_002_pos.ksh | 2 +- .../zpool_create_features_003_pos.ksh | 2 +- .../zpool_create_features_004_neg.ksh | 2 +- .../zpool_create_features_005_pos.ksh | 2 +- .../zpool_create_features_006_pos.ksh | 2 +- .../zpool_create_features_007_pos.ksh | 2 +- .../zpool_create_features_008_pos.ksh | 2 +- .../zpool_create_features_009_pos.ksh | 2 +- .../zpool_create/zpool_create_tempname.ksh | 6 +- .../cli_root/zpool_destroy/Makefile.am | 8 - .../cli_root/zpool_destroy/zpool_destroy.cfg | 4 +- .../zpool_destroy/zpool_destroy_001_pos.ksh | 2 +- .../zpool_destroy/zpool_destroy_002_pos.ksh | 2 +- .../zpool_destroy/zpool_destroy_003_neg.ksh | 2 +- .../cli_root/zpool_detach/Makefile.am | 5 - .../cli_root/zpool_detach/cleanup.ksh | 2 +- .../cli_root/zpool_detach/setup.ksh | 2 +- .../zpool_detach/zpool_detach_001_neg.ksh | 2 +- .../cli_root/zpool_events/.gitignore | 1 - .../cli_root/zpool_events/Makefile.am | 26 - .../zpool_events_clear_retained.ksh | 13 +- .../zpool_events/zpool_events_cliargs.ksh | 2 +- .../zpool_events/zpool_events_duplicates.ksh | 11 +- .../zpool_events/zpool_events_errors.ksh | 6 +- .../zpool_events/zpool_events_follow.ksh | 4 +- .../zpool_events/zpool_events_poolname.ksh | 13 +- .../cli_root/zpool_expand/Makefile.am | 12 - .../cli_root/zpool_expand/cleanup.ksh | 2 +- .../cli_root/zpool_expand/setup.ksh | 2 +- .../cli_root/zpool_expand/zpool_expand.cfg | 2 +- .../zpool_expand/zpool_expand_001_pos.ksh | 23 +- .../zpool_expand/zpool_expand_002_pos.ksh | 31 +- .../zpool_expand/zpool_expand_003_neg.ksh | 31 +- .../zpool_expand/zpool_expand_004_pos.ksh | 2 +- .../zpool_expand/zpool_expand_005_pos.ksh | 2 +- .../cli_root/zpool_export/Makefile.am | 12 - .../cli_root/zpool_export/cleanup.ksh | 2 +- .../cli_root/zpool_export/setup.ksh | 2 +- .../cli_root/zpool_export/zpool_export.cfg | 8 +- .../cli_root/zpool_export/zpool_export.kshlib | 2 +- .../zpool_export/zpool_export_001_pos.ksh | 2 +- .../zpool_export/zpool_export_002_pos.ksh | 19 +- .../zpool_export/zpool_export_003_neg.ksh | 2 +- .../zpool_export/zpool_export_004_pos.ksh | 2 +- .../functional/cli_root/zpool_get/Makefile.am | 12 - .../functional/cli_root/zpool_get/cleanup.ksh | 2 +- .../functional/cli_root/zpool_get/setup.ksh | 2 +- .../cli_root/zpool_get/zpool_get.cfg | 7 +- .../cli_root/zpool_get/zpool_get_001_pos.ksh | 2 +- .../cli_root/zpool_get/zpool_get_002_pos.ksh | 17 +- .../cli_root/zpool_get/zpool_get_003_pos.ksh | 12 +- .../cli_root/zpool_get/zpool_get_004_neg.ksh | 5 +- .../cli_root/zpool_get/zpool_get_005_pos.ksh | 11 +- .../cli_root/zpool_get/zpool_get_parsable.cfg | 2 +- .../cli_root/zpool_history/Makefile.am | 6 - .../cli_root/zpool_history/cleanup.ksh | 2 +- .../cli_root/zpool_history/setup.ksh | 2 +- .../zpool_history/zpool_history_001_neg.ksh | 2 +- .../zpool_history/zpool_history_002_pos.ksh | 2 +- .../cli_root/zpool_import/Makefile.am | 51 - .../zpool_import/blockfiles/Makefile.am | 5 - .../cli_root/zpool_import/cleanup.ksh | 2 +- .../import_cachefile_shared_device.ksh | 8 +- .../zpool_import/import_devices_missing.ksh | 4 +- .../import_rewind_config_changed.ksh | 54 +- .../cli_root/zpool_import/setup.ksh | 2 +- .../cli_root/zpool_import/zpool_import.cfg | 2 +- .../cli_root/zpool_import/zpool_import.kshlib | 61 +- .../zpool_import/zpool_import_001_pos.ksh | 11 +- .../zpool_import/zpool_import_002_pos.ksh | 9 +- .../zpool_import/zpool_import_003_pos.ksh | 2 +- .../zpool_import/zpool_import_004_pos.ksh | 2 +- .../zpool_import/zpool_import_005_pos.ksh | 2 +- .../zpool_import/zpool_import_006_pos.ksh | 2 +- .../zpool_import/zpool_import_007_pos.ksh | 2 +- .../zpool_import/zpool_import_008_pos.ksh | 2 +- .../zpool_import/zpool_import_009_neg.ksh | 2 +- .../zpool_import/zpool_import_010_pos.ksh | 2 +- .../zpool_import/zpool_import_011_neg.ksh | 2 +- .../zpool_import/zpool_import_012_pos.ksh | 9 +- .../zpool_import/zpool_import_013_neg.ksh | 2 +- .../zpool_import/zpool_import_014_pos.ksh | 2 +- .../zpool_import/zpool_import_016_pos.ksh | 2 +- .../zpool_import/zpool_import_017_pos.ksh | 2 +- .../zpool_import/zpool_import_all_001_pos.ksh | 13 +- .../zpool_import/zpool_import_errata3.ksh | 8 +- .../zpool_import/zpool_import_errata4.ksh | 20 +- .../zpool_import_features_001_pos.ksh | 2 +- .../zpool_import_features_002_neg.ksh | 8 +- .../zpool_import_features_003_pos.ksh | 2 +- .../zpool_import_missing_001_pos.ksh | 10 +- .../zpool_import_missing_002_pos.ksh | 2 +- .../zpool_import_missing_003_pos.ksh | 15 +- .../zpool_import_rename_001_pos.ksh | 9 +- .../cli_root/zpool_initialize/Makefile.am | 18 - .../cli_root/zpool_initialize/cleanup.ksh | 2 +- ...ol_initialize_attach_detach_add_remove.ksh | 2 +- .../zpool_initialize_import_export.ksh | 2 +- ...nitialize_offline_export_import_online.ksh | 2 +- .../zpool_initialize_online_offline.ksh | 2 +- .../zpool_initialize_split.ksh | 2 +- .../zpool_initialize_start_and_cancel_neg.ksh | 2 +- .../zpool_initialize_start_and_cancel_pos.ksh | 2 +- .../zpool_initialize_suspend_resume.ksh | 2 +- .../zpool_initialize_unsupported_vdevs.ksh | 2 +- .../zpool_initialize_verify_checksums.ksh | 2 +- .../zpool_initialize_verify_initialized.ksh | 9 +- .../cli_root/zpool_labelclear/Makefile.am | 9 - .../cli_root/zpool_offline/Makefile.am | 7 - .../cli_root/zpool_offline/cleanup.ksh | 2 +- .../cli_root/zpool_offline/setup.ksh | 2 +- .../zpool_offline/zpool_offline_001_pos.ksh | 27 +- .../zpool_offline/zpool_offline_002_neg.ksh | 2 +- .../zpool_offline/zpool_offline_003_pos.ksh | 7 +- .../cli_root/zpool_online/Makefile.am | 6 - .../cli_root/zpool_online/cleanup.ksh | 2 +- .../cli_root/zpool_online/setup.ksh | 2 +- .../zpool_online/zpool_online_001_pos.ksh | 22 +- .../zpool_online/zpool_online_002_neg.ksh | 2 +- .../cli_root/zpool_remove/Makefile.am | 10 - .../cli_root/zpool_remove/cleanup.ksh | 2 +- .../cli_root/zpool_remove/setup.ksh | 2 +- .../cli_root/zpool_remove/zpool_remove.cfg | 2 +- .../zpool_remove/zpool_remove_001_neg.ksh | 2 +- .../zpool_remove/zpool_remove_002_pos.ksh | 2 +- .../zpool_remove/zpool_remove_003_pos.ksh | 2 +- .../cli_root/zpool_reopen/Makefile.am | 15 - .../cli_root/zpool_reopen/zpool_reopen.cfg | 7 +- .../cli_root/zpool_reopen/zpool_reopen.shlib | 13 +- .../cli_root/zpool_replace/Makefile.am | 7 - .../cli_root/zpool_replace/cleanup.ksh | 2 +- .../zpool_replace/replace-o_ashift.ksh | 16 +- .../zpool_replace/replace_prop_ashift.ksh | 16 +- .../cli_root/zpool_replace/setup.ksh | 2 +- .../zpool_replace/zpool_replace_001_neg.ksh | 2 +- .../cli_root/zpool_resilver/Makefile.am | 9 - .../cli_root/zpool_resilver/cleanup.ksh | 2 +- .../cli_root/zpool_resilver/setup.ksh | 2 +- .../zpool_resilver/zpool_resilver.cfg | 7 +- .../zpool_resilver_bad_args.ksh | 2 +- .../zpool_resilver/zpool_resilver_restart.ksh | 2 +- .../cli_root/zpool_scrub/Makefile.am | 16 - .../cli_root/zpool_scrub/cleanup.ksh | 2 +- .../functional/cli_root/zpool_scrub/setup.ksh | 2 +- .../cli_root/zpool_scrub/zpool_scrub.cfg | 6 +- .../zpool_scrub/zpool_scrub_001_neg.ksh | 2 +- .../zpool_scrub/zpool_scrub_002_pos.ksh | 2 +- .../zpool_scrub/zpool_scrub_003_pos.ksh | 2 +- .../zpool_scrub/zpool_scrub_004_pos.ksh | 2 +- .../zpool_scrub/zpool_scrub_005_pos.ksh | 2 +- .../zpool_scrub_offline_device.ksh | 2 +- .../zpool_scrub_print_repairing.ksh | 2 +- .../functional/cli_root/zpool_set/Makefile.am | 9 - .../functional/cli_root/zpool_set/cleanup.ksh | 2 +- .../functional/cli_root/zpool_set/setup.ksh | 2 +- .../cli_root/zpool_set/zpool_set_001_pos.ksh | 4 +- .../cli_root/zpool_set/zpool_set_002_neg.ksh | 2 +- .../cli_root/zpool_set/zpool_set_003_neg.ksh | 2 +- .../cli_root/zpool_set/zpool_set_ashift.ksh | 2 +- .../cli_root/zpool_split/Makefile.am | 19 - .../cli_root/zpool_split/zpool_split.cfg | 2 +- .../zpool_split/zpool_split_devices.ksh | 2 +- .../zpool_split/zpool_split_dryrun_output.ksh | 2 +- .../zpool_split/zpool_split_indirect.ksh | 4 +- .../zpool_split/zpool_split_props.ksh | 5 +- .../zpool_split/zpool_split_vdevs.ksh | 6 +- .../zpool_split/zpool_split_wholedisk.ksh | 2 +- .../cli_root/zpool_status/Makefile.am | 7 - .../cli_root/zpool_status/cleanup.ksh | 2 +- .../cli_root/zpool_status/setup.ksh | 2 +- .../zpool_status/zpool_status_001_pos.ksh | 2 +- .../zpool_status/zpool_status_002_pos.ksh | 2 +- .../zpool_status/zpool_status_003_pos.ksh | 70 + .../zpool_status/zpool_status_004_pos.ksh | 81 + .../zpool_status_features_001_pos.ksh | 2 +- .../cli_root/zpool_sync/Makefile.am | 6 - .../cli_root/zpool_sync/cleanup.ksh | 2 +- .../functional/cli_root/zpool_sync/setup.ksh | 2 +- .../cli_root/zpool_trim/Makefile.am | 25 - .../zpool_trim/zpool_trim_partial.ksh | 2 +- .../cli_root/zpool_upgrade/Makefile.am | 20 - .../zpool_upgrade/blockfiles/Makefile.am | 54 - .../cli_root/zpool_upgrade/cleanup.ksh | 2 +- .../cli_root/zpool_upgrade/setup.ksh | 2 +- .../cli_root/zpool_upgrade/zpool_upgrade.cfg | 2 +- .../zpool_upgrade/zpool_upgrade.kshlib | 5 +- .../zpool_upgrade/zpool_upgrade_001_pos.ksh | 2 +- .../zpool_upgrade/zpool_upgrade_002_pos.ksh | 2 +- .../zpool_upgrade/zpool_upgrade_003_pos.ksh | 2 +- .../zpool_upgrade/zpool_upgrade_004_pos.ksh | 2 +- .../zpool_upgrade/zpool_upgrade_005_neg.ksh | 2 +- .../zpool_upgrade/zpool_upgrade_006_neg.ksh | 2 +- .../zpool_upgrade/zpool_upgrade_007_pos.ksh | 2 +- .../zpool_upgrade/zpool_upgrade_008_pos.ksh | 4 +- .../zpool_upgrade/zpool_upgrade_009_neg.ksh | 2 +- .../zpool_upgrade_features_001_pos.ksh | 2 +- .../cli_root/zpool_wait/Makefile.am | 22 - .../cli_root/zpool_wait/scan/Makefile.am | 11 - .../cli_root/zpool_wait/zpool_wait.kshlib | 2 +- .../cli_root/zpool_wait/zpool_wait_usage.ksh | 2 + .../tests/functional/cli_user/Makefile.am | 6 - .../functional/cli_user/misc/Makefile.am | 52 - .../cli_user/misc/arc_summary_001_pos.ksh | 29 +- .../cli_user/misc/arc_summary_002_neg.ksh | 6 +- .../cli_user/misc/arcstat_001_pos.ksh | 7 +- .../functional/cli_user/misc/cleanup.ksh | 2 +- .../tests/functional/cli_user/misc/misc.cfg | 2 +- .../tests/functional/cli_user/misc/setup.ksh | 2 +- .../functional/cli_user/misc/zdb_001_neg.ksh | 14 +- .../functional/cli_user/misc/zfs_001_neg.ksh | 4 +- .../cli_user/misc/zfs_allow_001_neg.ksh | 13 +- .../cli_user/misc/zfs_clone_001_neg.ksh | 2 +- .../cli_user/misc/zfs_create_001_neg.ksh | 2 +- .../cli_user/misc/zfs_destroy_001_neg.ksh | 2 +- .../cli_user/misc/zfs_get_001_neg.ksh | 2 +- .../cli_user/misc/zfs_inherit_001_neg.ksh | 2 +- .../cli_user/misc/zfs_mount_001_neg.ksh | 2 +- .../cli_user/misc/zfs_promote_001_neg.ksh | 2 +- .../cli_user/misc/zfs_receive_001_neg.ksh | 2 +- .../cli_user/misc/zfs_rename_001_neg.ksh | 2 +- .../cli_user/misc/zfs_rollback_001_neg.ksh | 2 +- .../cli_user/misc/zfs_send_001_neg.ksh | 2 +- .../cli_user/misc/zfs_set_001_neg.ksh | 2 +- .../cli_user/misc/zfs_share_001_neg.ksh | 13 +- .../cli_user/misc/zfs_snapshot_001_neg.ksh | 2 +- .../cli_user/misc/zfs_unallow_001_neg.ksh | 11 +- .../cli_user/misc/zfs_unmount_001_neg.ksh | 2 +- .../cli_user/misc/zfs_unshare_001_neg.ksh | 14 +- .../cli_user/misc/zfs_upgrade_001_neg.ksh | 2 +- .../cli_user/misc/zpool_001_neg.ksh | 4 +- .../cli_user/misc/zpool_add_001_neg.ksh | 2 +- .../cli_user/misc/zpool_attach_001_neg.ksh | 2 +- .../cli_user/misc/zpool_clear_001_neg.ksh | 2 +- .../cli_user/misc/zpool_create_001_neg.ksh | 2 +- .../cli_user/misc/zpool_destroy_001_neg.ksh | 2 +- .../cli_user/misc/zpool_detach_001_neg.ksh | 2 +- .../cli_user/misc/zpool_export_001_neg.ksh | 2 +- .../cli_user/misc/zpool_get_001_neg.ksh | 6 +- .../cli_user/misc/zpool_history_001_neg.ksh | 2 +- .../cli_user/misc/zpool_import_001_neg.ksh | 2 +- .../cli_user/misc/zpool_import_002_neg.ksh | 2 +- .../cli_user/misc/zpool_offline_001_neg.ksh | 2 +- .../cli_user/misc/zpool_online_001_neg.ksh | 2 +- .../cli_user/misc/zpool_remove_001_neg.ksh | 2 +- .../cli_user/misc/zpool_replace_001_neg.ksh | 2 +- .../cli_user/misc/zpool_scrub_001_neg.ksh | 2 +- .../cli_user/misc/zpool_set_001_neg.ksh | 4 +- .../cli_user/misc/zpool_status_001_neg.ksh | 2 +- .../cli_user/misc/zpool_upgrade_001_neg.ksh | 2 +- .../functional/cli_user/zfs_list/Makefile.am | 14 - .../functional/cli_user/zfs_list/cleanup.ksh | 2 +- .../functional/cli_user/zfs_list/setup.ksh | 2 +- .../functional/cli_user/zfs_list/zfs_list.cfg | 2 +- .../cli_user/zfs_list/zfs_list.kshlib | 4 +- .../cli_user/zfs_list/zfs_list_001_pos.ksh | 2 +- .../cli_user/zfs_list/zfs_list_002_pos.ksh | 2 +- .../cli_user/zfs_list/zfs_list_003_pos.ksh | 7 +- .../cli_user/zfs_list/zfs_list_004_neg.ksh | 2 +- .../cli_user/zfs_list/zfs_list_005_neg.ksh | 14 + .../cli_user/zfs_list/zfs_list_007_pos.ksh | 11 +- .../cli_user/zfs_list/zfs_list_008_neg.ksh | 2 +- .../cli_user/zpool_iostat/Makefile.am | 12 - .../cli_user/zpool_iostat/cleanup.ksh | 2 +- .../cli_user/zpool_iostat/setup.ksh | 2 +- .../zpool_iostat/zpool_iostat_-c_disable.ksh | 2 +- .../zpool_iostat/zpool_iostat_-c_homedir.ksh | 2 +- .../zpool_iostat_-c_searchpath.ksh | 2 +- .../zpool_iostat/zpool_iostat_001_neg.ksh | 2 +- .../zpool_iostat/zpool_iostat_002_pos.ksh | 4 +- .../zpool_iostat/zpool_iostat_003_neg.ksh | 2 +- .../zpool_iostat/zpool_iostat_004_pos.ksh | 2 +- .../zpool_iostat/zpool_iostat_005_pos.ksh | 2 +- .../cli_user/zpool_list/Makefile.am | 6 - .../cli_user/zpool_list/cleanup.ksh | 2 +- .../functional/cli_user/zpool_list/setup.ksh | 2 +- .../zpool_list/zpool_list_001_pos.ksh | 2 +- .../zpool_list/zpool_list_002_neg.ksh | 2 +- .../cli_user/zpool_status/Makefile.am | 8 - .../cli_user/zpool_status/cleanup.ksh | 2 +- .../cli_user/zpool_status/setup.ksh | 2 +- .../zpool_status/zpool_status_-c_disable.ksh | 2 +- .../zpool_status/zpool_status_-c_homedir.ksh | 2 +- .../zpool_status_-c_searchpath.ksh | 2 +- .../zpool_status/zpool_status_003_pos.ksh | 2 +- .../tests/functional/compression/Makefile.am | 17 - .../tests/functional/compression/cleanup.ksh | 2 +- .../tests/functional/compression/compress.cfg | 2 +- .../compression/compress_001_pos.ksh | 6 +- .../compression/compress_002_pos.ksh | 6 +- .../compression/compress_003_pos.ksh | 2 +- .../compression/compress_004_pos.ksh | 2 +- .../compression/compress_zstd_bswap.ksh | 2 +- .../compression/l2arc_compressed_arc.ksh | 2 + .../l2arc_compressed_arc_disabled.ksh | 2 + .../compression/l2arc_encrypted.ksh | 2 + .../l2arc_encrypted_no_compressed_arc.ksh | 2 + .../tests/functional/compression/setup.ksh | 2 +- .../tests/functional/cp_files/.gitignore | 1 - .../tests/functional/cp_files/Makefile.am | 13 - .../tests/functional/cp_files/cleanup.ksh | 2 +- .../functional/cp_files/cp_files_001_pos.ksh | 9 +- .../tests/functional/cp_files/setup.ksh | 2 +- .../tests/functional/crtime/Makefile.am | 5 - .../tests/functional/crtime/cleanup.ksh | 2 +- .../functional/crtime/crtime_001_pos.ksh | 2 +- .../tests/functional/crtime/setup.ksh | 2 +- .../tests/functional/ctime/.gitignore | 1 - .../tests/functional/ctime/Makefile.am | 13 - .../tests/functional/ctime/cleanup.ksh | 2 +- .../tests/functional/ctime/ctime_001_pos.ksh | 4 +- .../tests/functional/ctime/setup.ksh | 2 +- .../tests/functional/deadman/Makefile.am | 8 - .../tests/functional/deadman/deadman.cfg | 2 +- .../functional/deadman/deadman_ratelimit.ksh | 4 +- .../tests/functional/deadman/deadman_sync.ksh | 2 +- .../tests/functional/deadman/deadman_zio.ksh | 2 +- .../tests/functional/delegate/Makefile.am | 28 - .../tests/functional/delegate/cleanup.ksh | 2 +- .../tests/functional/delegate/delegate.cfg | 2 +- .../delegate/delegate_common.kshlib | 73 +- .../tests/functional/delegate/setup.ksh | 6 +- .../functional/delegate/zfs_allow_001_pos.ksh | 8 +- .../functional/delegate/zfs_allow_002_pos.ksh | 2 +- .../functional/delegate/zfs_allow_003_pos.ksh | 2 +- .../functional/delegate/zfs_allow_004_pos.ksh | 2 +- .../functional/delegate/zfs_allow_005_pos.ksh | 2 +- .../functional/delegate/zfs_allow_006_pos.ksh | 2 +- .../functional/delegate/zfs_allow_007_pos.ksh | 2 +- .../functional/delegate/zfs_allow_008_pos.ksh | 2 +- .../functional/delegate/zfs_allow_009_neg.ksh | 2 +- .../functional/delegate/zfs_allow_010_pos.ksh | 2 +- .../functional/delegate/zfs_allow_011_neg.ksh | 2 +- .../functional/delegate/zfs_allow_012_neg.ksh | 2 +- .../delegate/zfs_unallow_001_pos.ksh | 2 +- .../delegate/zfs_unallow_002_pos.ksh | 2 +- .../delegate/zfs_unallow_003_pos.ksh | 2 +- .../delegate/zfs_unallow_004_pos.ksh | 2 +- .../delegate/zfs_unallow_005_pos.ksh | 2 +- .../delegate/zfs_unallow_006_pos.ksh | 2 +- .../delegate/zfs_unallow_007_neg.ksh | 2 +- .../delegate/zfs_unallow_008_neg.ksh | 2 +- .../tests/functional/devices/Makefile.am | 11 - .../tests/functional/devices/cleanup.ksh | 2 +- .../tests/functional/devices/devices.cfg | 2 +- .../functional/devices/devices_001_pos.ksh | 2 +- .../functional/devices/devices_002_neg.ksh | 2 +- .../functional/devices/devices_003_pos.ksh | 2 +- .../functional/devices/devices_common.kshlib | 2 +- .../tests/functional/devices/setup.ksh | 2 +- .../cleanup.ksh | 2 +- .../dos_attributes/read_dos_attrs_001.ksh | 60 + .../setup.ksh | 5 +- .../dos_attributes/write_dos_attrs_001.ksh | 61 + .../tests/functional/events/.gitignore | 1 - .../tests/functional/events/Makefile.am | 18 - .../tests/functional/events/cleanup.ksh | 4 +- .../tests/functional/events/events.cfg | 4 +- .../functional/events/events_001_pos.ksh | 2 +- .../functional/events/events_002_pos.ksh | 22 +- .../functional/events/events_common.kshlib | 23 +- .../tests/functional/events/setup.ksh | 2 +- .../tests/functional/events/zed_fd_spill.ksh | 11 +- .../tests/functional/events/zed_rc_filter.ksh | 2 +- .../tests/functional/exec/Makefile.am | 9 - .../tests/functional/exec/cleanup.ksh | 2 +- .../tests/functional/exec/exec_001_pos.ksh | 2 +- .../tests/functional/exec/exec_002_neg.ksh | 22 +- .../zfs-tests/tests/functional/exec/setup.ksh | 2 +- .../tests/functional/fallocate/Makefile.am | 6 - .../tests/functional/fallocate/cleanup.ksh | 2 +- .../fallocate/fallocate_prealloc.ksh | 2 +- .../fallocate/fallocate_punch-hole.ksh | 37 +- .../fallocate/fallocate_zero-range.ksh | 119 + .../tests/functional/fallocate/setup.ksh | 7 +- .../tests/functional/fault/Makefile.am | 20 - .../functional/fault/auto_offline_001_pos.ksh | 9 +- .../functional/fault/auto_online_001_pos.ksh | 17 +- .../functional/fault/auto_online_002_pos.ksh | 2 +- .../functional/fault/auto_replace_001_pos.ksh | 6 +- .../functional/fault/auto_spare_shared.ksh | 5 +- .../tests/functional/fault/cleanup.ksh | 2 +- .../tests/functional/fault/decrypt_fault.ksh | 1 + .../tests/functional/fault/fault.cfg | 9 +- .../tests/functional/fault/setup.ksh | 2 +- .../functional/fault/zpool_status_-s.ksh | 15 +- .../tests/functional/features/Makefile.am | 3 - .../features/async_destroy/Makefile.am | 5 - .../async_destroy/async_destroy_001_pos.ksh | 2 +- .../features/async_destroy/cleanup.ksh | 2 +- .../features/async_destroy/setup.ksh | 2 +- .../features/large_dnode/Makefile.am | 13 - .../features/large_dnode/cleanup.ksh | 2 +- .../large_dnode/large_dnode_001_pos.ksh | 4 +- .../large_dnode/large_dnode_002_pos.ksh | 2 +- .../large_dnode/large_dnode_003_pos.ksh | 2 +- .../large_dnode/large_dnode_004_neg.ksh | 2 +- .../large_dnode/large_dnode_005_pos.ksh | 8 +- .../large_dnode/large_dnode_006_pos.ksh | 2 +- .../large_dnode/large_dnode_007_neg.ksh | 2 +- .../large_dnode/large_dnode_008_pos.ksh | 2 +- .../large_dnode/large_dnode_009_pos.ksh | 4 +- .../functional/features/large_dnode/setup.ksh | 2 +- .../tests/functional/grow/Makefile.am | 7 - .../zfs-tests/tests/functional/grow/grow.cfg | 2 +- .../functional/grow/grow_pool_001_pos.ksh | 2 +- .../functional/grow/grow_replicas_001_pos.ksh | 2 +- .../tests/functional/history/Makefile.am | 23 - .../tests/functional/history/cleanup.ksh | 2 +- .../tests/functional/history/history.cfg | 2 +- .../functional/history/history_001_pos.ksh | 3 +- .../functional/history/history_002_pos.ksh | 2 +- .../functional/history/history_003_pos.ksh | 11 +- .../functional/history/history_004_pos.ksh | 6 +- .../functional/history/history_005_neg.ksh | 10 +- .../functional/history/history_006_neg.ksh | 10 +- .../functional/history/history_007_pos.ksh | 22 +- .../functional/history/history_008_pos.ksh | 5 +- .../functional/history/history_009_pos.ksh | 2 +- .../functional/history/history_010_pos.ksh | 8 +- .../functional/history/history_common.kshlib | 88 +- .../tests/functional/history/setup.ksh | 2 +- .../tests/functional/hkdf/Makefile.am | 17 - .../tests/functional/hkdf/hkdf_test.c | 35 +- .../tests/functional/hkdf/run_hkdf_test.ksh | 30 - .../zfs-tests/tests/functional/hkdf/setup.ksh | 22 - .../tests/functional/inheritance/Makefile.am | 57 - .../functional/inheritance/README.config | 2 +- .../tests/functional/inheritance/README.state | 2 +- .../tests/functional/inheritance/cleanup.ksh | 2 +- .../functional/inheritance/config001.cfg | 2 +- .../functional/inheritance/config002.cfg | 2 +- .../functional/inheritance/config003.cfg | 2 +- .../functional/inheritance/config004.cfg | 2 +- .../functional/inheritance/config005.cfg | 2 +- .../functional/inheritance/config006.cfg | 2 +- .../functional/inheritance/config007.cfg | 2 +- .../functional/inheritance/config008.cfg | 2 +- .../functional/inheritance/config009.cfg | 2 +- .../functional/inheritance/config010.cfg | 2 +- .../functional/inheritance/config011.cfg | 2 +- .../functional/inheritance/config012.cfg | 2 +- .../functional/inheritance/config013.cfg | 2 +- .../functional/inheritance/config014.cfg | 2 +- .../functional/inheritance/config015.cfg | 2 +- .../functional/inheritance/config016.cfg | 2 +- .../functional/inheritance/config017.cfg | 2 +- .../functional/inheritance/config018.cfg | 2 +- .../functional/inheritance/config019.cfg | 2 +- .../functional/inheritance/config020.cfg | 2 +- .../functional/inheritance/config021.cfg | 2 +- .../functional/inheritance/config022.cfg | 2 +- .../functional/inheritance/config023.cfg | 2 +- .../functional/inheritance/config024.cfg | 2 +- .../functional/inheritance/inherit.kshlib | 16 +- .../inheritance/inherit_001_pos.ksh | 18 +- .../tests/functional/inheritance/state001.cfg | 2 +- .../tests/functional/inheritance/state002.cfg | 2 +- .../tests/functional/inheritance/state003.cfg | 2 +- .../tests/functional/inheritance/state004.cfg | 2 +- .../tests/functional/inheritance/state005.cfg | 2 +- .../tests/functional/inheritance/state006.cfg | 2 +- .../tests/functional/inheritance/state007.cfg | 2 +- .../tests/functional/inheritance/state008.cfg | 2 +- .../tests/functional/inheritance/state009.cfg | 2 +- .../tests/functional/inheritance/state010.cfg | 2 +- .../tests/functional/inheritance/state011.cfg | 2 +- .../tests/functional/inheritance/state012.cfg | 2 +- .../tests/functional/inheritance/state013.cfg | 2 +- .../tests/functional/inheritance/state014.cfg | 2 +- .../tests/functional/inheritance/state015.cfg | 2 +- .../tests/functional/inheritance/state016.cfg | 2 +- .../tests/functional/inheritance/state017.cfg | 2 +- .../tests/functional/inheritance/state018.cfg | 2 +- .../tests/functional/inheritance/state019.cfg | 2 +- .../tests/functional/inheritance/state020.cfg | 2 +- .../tests/functional/inheritance/state021.cfg | 2 +- .../tests/functional/inheritance/state022.cfg | 2 +- .../tests/functional/inheritance/state023.cfg | 2 +- .../tests/functional/inheritance/state024.cfg | 2 +- .../tests/functional/inuse/Makefile.am | 14 - .../tests/functional/inuse/inuse.cfg | 2 +- .../tests/functional/inuse/inuse_001_pos.ksh | 10 +- .../tests/functional/inuse/inuse_003_pos.ksh | 26 +- .../tests/functional/inuse/inuse_004_pos.ksh | 20 +- .../tests/functional/inuse/inuse_005_pos.ksh | 2 +- .../tests/functional/inuse/inuse_006_pos.ksh | 4 +- .../tests/functional/inuse/inuse_007_pos.ksh | 4 +- .../tests/functional/inuse/inuse_008_pos.ksh | 2 +- .../tests/functional/inuse/inuse_009_pos.ksh | 2 +- .../tests/functional/inuse/setup.ksh | 2 +- .../zfs-tests/tests/functional/io/Makefile.am | 13 - .../zfs-tests/tests/functional/io/cleanup.ksh | 2 +- .../tests/functional/io/io_uring.ksh | 6 +- .../zfs-tests/tests/functional/io/libaio.ksh | 4 +- tests/zfs-tests/tests/functional/io/mmap.ksh | 4 +- .../tests/functional/io/posixaio.ksh | 4 +- tests/zfs-tests/tests/functional/io/psync.ksh | 4 +- tests/zfs-tests/tests/functional/io/setup.ksh | 2 +- tests/zfs-tests/tests/functional/io/sync.ksh | 4 +- .../tests/functional/l2arc/Makefile.am | 15 - .../functional/l2arc/l2arc_arcstats_pos.ksh | 2 + .../functional/l2arc/l2arc_l2miss_pos.ksh | 2 + .../functional/l2arc/l2arc_mfuonly_pos.ksh | 2 + .../l2arc/persist_l2arc_001_pos.ksh | 5 +- .../l2arc/persist_l2arc_002_pos.ksh | 5 +- .../l2arc/persist_l2arc_003_neg.ksh | 2 + .../l2arc/persist_l2arc_004_pos.ksh | 5 +- .../l2arc/persist_l2arc_005_pos.ksh | 5 +- .../tests/functional/large_files/Makefile.am | 6 - .../tests/functional/large_files/cleanup.ksh | 2 +- .../large_files/large_files_001_pos.ksh | 2 +- .../large_files/large_files_002_pos.ksh | 2 +- .../tests/functional/large_files/setup.ksh | 2 +- .../tests/functional/largest_pool/Makefile.am | 6 - .../functional/largest_pool/largest_pool.cfg | 2 +- .../largest_pool/largest_pool_001_pos.ksh | 11 +- .../tests/functional/libzfs/Makefile.am | 17 - .../tests/functional/libzfs/cleanup.ksh | 2 +- .../tests/functional/libzfs/many_fds.c | 56 +- .../tests/functional/libzfs/setup.ksh | 2 +- .../tests/functional/limits/Makefile.am | 9 - .../tests/functional/link_count/Makefile.am | 6 - .../tests/functional/link_count/cleanup.ksh | 2 +- .../functional/link_count/link_count_001.ksh | 2 +- .../tests/functional/link_count/setup.ksh | 2 +- .../tests/functional/log_spacemap/Makefile.am | 2 - .../log_spacemap/log_spacemap_import_logs.ksh | 7 +- .../tests/functional/migration/Makefile.am | 20 - .../tests/functional/migration/cleanup.ksh | 8 +- .../tests/functional/migration/migration.cfg | 9 +- .../functional/migration/migration.kshlib | 60 +- .../migration/migration_001_pos.ksh | 13 +- .../migration/migration_002_pos.ksh | 21 +- .../migration/migration_003_pos.ksh | 21 +- .../migration/migration_004_pos.ksh | 21 +- .../migration/migration_005_pos.ksh | 29 +- .../migration/migration_006_pos.ksh | 21 +- .../migration/migration_007_pos.ksh | 13 +- .../migration/migration_008_pos.ksh | 21 +- .../migration/migration_009_pos.ksh | 21 +- .../migration/migration_010_pos.ksh | 13 +- .../migration/migration_011_pos.ksh | 21 +- .../migration/migration_012_pos.ksh | 21 +- .../tests/functional/migration/setup.ksh | 5 +- .../tests/functional/mmap/Makefile.am | 11 - .../tests/functional/mmap/cleanup.ksh | 2 +- .../zfs-tests/tests/functional/mmap/mmap.cfg | 2 +- .../functional/mmap/mmap_libaio_001_pos.ksh | 6 +- .../functional/mmap/mmap_read_001_pos.ksh | 2 +- .../functional/mmap/mmap_seek_001_pos.ksh | 2 +- .../functional/mmap/mmap_sync_001_pos.ksh | 63 + .../functional/mmap/mmap_write_001_pos.ksh | 2 +- .../zfs-tests/tests/functional/mmap/setup.ksh | 2 +- .../tests/functional/mmp/Makefile.am | 21 - .../zfs-tests/tests/functional/mmp/mmp.kshlib | 58 +- .../tests/functional/mmp/mmp_on_off.ksh | 6 +- .../tests/functional/mmp/mmp_on_thread.ksh | 4 +- .../functional/mmp/mmp_write_distribution.ksh | 10 +- .../tests/functional/mount/Makefile.am | 7 - .../tests/functional/mount/cleanup.ksh | 2 +- .../tests/functional/mount/setup.ksh | 2 +- .../tests/functional/mount/umount_001.ksh | 2 +- .../mount/umount_unlinked_drain.ksh | 4 +- .../tests/functional/mount/umountall_001.ksh | 9 +- .../tests/functional/mv_files/Makefile.am | 11 - .../tests/functional/mv_files/cleanup.ksh | 9 +- .../tests/functional/mv_files/mv_files.cfg | 4 +- .../functional/mv_files/mv_files_001_pos.ksh | 8 +- .../functional/mv_files/mv_files_002_pos.ksh | 9 +- .../mv_files/mv_files_common.kshlib | 6 +- .../tests/functional/mv_files/setup.ksh | 2 +- .../tests/functional/nestedfs/Makefile.am | 5 - .../tests/functional/nestedfs/cleanup.ksh | 2 +- .../functional/nestedfs/nestedfs_001_pos.ksh | 2 +- .../tests/functional/nestedfs/setup.ksh | 2 +- .../tests/functional/no_space/Makefile.am | 11 - .../tests/functional/no_space/cleanup.ksh | 2 +- .../tests/functional/no_space/enospc.cfg | 2 +- .../functional/no_space/enospc_001_pos.ksh | 2 +- .../functional/no_space/enospc_002_pos.ksh | 4 +- .../functional/no_space/enospc_003_pos.ksh | 2 +- .../tests/functional/no_space/enospc_df.ksh | 4 +- .../tests/functional/no_space/enospc_rm.ksh | 62 + .../tests/functional/no_space/setup.ksh | 2 +- .../tests/functional/nopwrite/Makefile.am | 15 - .../functional/online_offline/Makefile.am | 10 - .../functional/online_offline/cleanup.ksh | 2 +- .../online_offline/online_offline.cfg | 4 +- .../online_offline/online_offline_001_pos.ksh | 17 +- .../online_offline/online_offline_002_neg.ksh | 8 +- .../online_offline/online_offline_003_neg.ksh | 2 +- .../tests/functional/online_offline/setup.ksh | 4 +- .../zfs-tests/tests/functional/pam/.gitignore | 1 + .../tests/functional/pam/Makefile.am | 8 - .../tests/functional/pam/cleanup.ksh | 8 +- .../tests/functional/pam/pam_basic.ksh | 2 +- .../tests/functional/pam/pam_nounmount.ksh | 2 +- .../functional/pam/pam_short_password.ksh | 2 +- .../zfs-tests/tests/functional/pam/setup.ksh | 7 +- .../{utilities.kshlib => utilities.kshlib.in} | 17 +- .../functional/pool_checkpoint/Makefile.am | 26 - .../pool_checkpoint/checkpoint_lun_expsz.ksh | 12 +- .../pool_checkpoint/checkpoint_zdb.ksh | 42 +- .../tests/functional/pool_names/Makefile.am | 4 - .../pool_names/pool_names_001_pos.ksh | 2 +- .../pool_names/pool_names_002_neg.ksh | 2 +- .../tests/functional/poolversion/Makefile.am | 6 - .../tests/functional/poolversion/cleanup.ksh | 2 +- .../poolversion/poolversion_001_pos.ksh | 5 +- .../poolversion/poolversion_002_pos.ksh | 6 +- .../tests/functional/poolversion/setup.ksh | 2 +- .../tests/functional/privilege/Makefile.am | 6 - .../tests/functional/privilege/cleanup.ksh | 2 +- .../privilege/privilege_001_pos.ksh | 2 +- .../privilege/privilege_002_pos.ksh | 2 +- .../tests/functional/privilege/setup.ksh | 5 +- .../tests/functional/procfs/Makefile.am | 8 - .../tests/functional/procfs/cleanup.ksh | 2 +- .../tests/functional/procfs/pool_state.ksh | 4 +- .../functional/procfs/procfs_list_basic.ksh | 8 +- .../procfs/procfs_list_concurrent_readers.ksh | 7 +- .../procfs/procfs_list_stale_read.ksh | 6 +- .../tests/functional/procfs/setup.ksh | 2 +- .../tests/functional/projectquota/Makefile.am | 27 - .../tests/functional/projectquota/cleanup.ksh | 2 +- .../projectquota/projectid_001_pos.ksh | 2 +- .../projectquota/projectid_002_pos.ksh | 2 +- .../projectquota/projectid_003_pos.ksh | 2 +- .../functional/projectquota/projectquota.cfg | 2 +- .../projectquota/projectquota_001_pos.ksh | 2 +- .../projectquota/projectquota_002_pos.ksh | 2 +- .../projectquota/projectquota_003_pos.ksh | 12 +- .../projectquota/projectquota_004_neg.ksh | 2 +- .../projectquota/projectquota_005_pos.ksh | 2 +- .../projectquota/projectquota_006_pos.ksh | 2 +- .../projectquota/projectquota_007_pos.ksh | 2 +- .../projectquota/projectquota_008_pos.ksh | 2 +- .../projectquota/projectquota_009_pos.ksh | 2 +- .../projectquota/projectquota_common.kshlib | 17 +- .../projectquota/projectspace_001_pos.ksh | 2 +- .../projectquota/projectspace_002_pos.ksh | 2 +- .../projectquota/projectspace_003_pos.ksh | 2 +- .../projectquota/projectspace_004_pos.ksh | 6 +- .../projectquota/projecttree_001_pos.ksh | 2 +- .../projectquota/projecttree_002_pos.ksh | 2 +- .../projectquota/projecttree_003_neg.ksh | 2 +- .../tests/functional/projectquota/setup.ksh | 8 +- .../tests/functional/pyzfs/Makefile.am | 7 - .../functional/pyzfs/pyzfs_unittest.ksh.in | 15 +- .../tests/functional/quota/Makefile.am | 14 - .../tests/functional/quota/cleanup.ksh | 2 +- .../tests/functional/quota/quota.cfg | 2 +- .../tests/functional/quota/quota.kshlib | 6 +- .../tests/functional/quota/quota_001_pos.ksh | 2 +- .../tests/functional/quota/quota_002_pos.ksh | 2 +- .../tests/functional/quota/quota_003_pos.ksh | 2 +- .../tests/functional/quota/quota_004_pos.ksh | 2 +- .../tests/functional/quota/quota_005_pos.ksh | 2 +- .../tests/functional/quota/quota_006_neg.ksh | 2 +- .../tests/functional/quota/setup.ksh | 6 +- .../tests/functional/raidz/Makefile.am | 8 - .../tests/functional/raidz/cleanup.ksh | 2 +- .../tests/functional/raidz/raidz_001_neg.ksh | 2 +- .../tests/functional/raidz/raidz_002_pos.ksh | 2 +- .../tests/functional/raidz/raidz_003_pos.ksh | 2 +- .../tests/functional/raidz/raidz_004_pos.ksh | 2 +- .../tests/functional/raidz/setup.ksh | 2 +- .../functional/redacted_send/Makefile.am | 26 - .../functional/redacted_send/cleanup.ksh | 2 +- .../functional/redacted_send/redacted.cfg | 8 +- .../functional/redacted_send/redacted.kshlib | 8 +- .../redacted_send/redacted_embedded.ksh | 18 +- .../redacted_send/redacted_incrementals.ksh | 14 +- .../redacted_send/redacted_largeblocks.ksh | 2 +- .../redacted_send/redacted_negative.ksh | 8 +- .../redacted_send/redacted_props.ksh | 4 +- .../redacted_send/redacted_resume.ksh | 6 +- .../redacted_send/redacted_size.ksh | 21 +- .../tests/functional/redacted_send/setup.ksh | 2 +- .../tests/functional/redundancy/Makefile.am | 22 - .../tests/functional/redundancy/cleanup.ksh | 2 +- .../functional/redundancy/redundancy.cfg | 2 +- .../functional/redundancy/redundancy.kshlib | 34 +- .../redundancy/redundancy_draid.ksh | 8 +- .../redundancy/redundancy_draid1.ksh | 2 +- .../redundancy/redundancy_draid2.ksh | 2 +- .../redundancy/redundancy_draid3.ksh | 2 +- ...aged.ksh => redundancy_draid_damaged1.ksh} | 21 +- .../redundancy/redundancy_draid_damaged2.ksh | 157 + .../redundancy/redundancy_draid_spare3.ksh | 4 - .../redundancy/redundancy_mirror.ksh | 2 +- .../redundancy/redundancy_raidz.ksh | 8 +- .../redundancy/redundancy_raidz1.ksh | 2 +- .../redundancy/redundancy_raidz2.ksh | 2 +- .../redundancy/redundancy_raidz3.ksh | 2 +- .../redundancy/redundancy_stripe.ksh | 2 +- .../tests/functional/redundancy/setup.ksh | 2 +- .../tests/functional/refquota/Makefile.am | 12 - .../tests/functional/refquota/cleanup.ksh | 2 +- .../functional/refquota/refquota_001_pos.ksh | 2 +- .../functional/refquota/refquota_002_pos.ksh | 2 +- .../functional/refquota/refquota_003_pos.ksh | 2 +- .../functional/refquota/refquota_004_pos.ksh | 2 +- .../functional/refquota/refquota_005_pos.ksh | 2 +- .../functional/refquota/refquota_006_neg.ksh | 2 +- .../tests/functional/refquota/setup.ksh | 6 +- .../tests/functional/refreserv/Makefile.am | 14 - .../tests/functional/refreserv/cleanup.ksh | 2 +- .../tests/functional/refreserv/refreserv.cfg | 2 +- .../refreserv/refreserv_001_pos.ksh | 2 +- .../refreserv/refreserv_002_pos.ksh | 5 +- .../refreserv/refreserv_003_pos.ksh | 2 +- .../refreserv/refreserv_004_pos.ksh | 2 +- .../refreserv/refreserv_005_pos.ksh | 2 +- .../functional/refreserv/refreserv_raidz.ksh | 2 +- .../tests/functional/refreserv/setup.ksh | 6 +- .../tests/functional/removal/Makefile.am | 38 - .../tests/functional/removal/removal.kshlib | 7 - .../removal/removal_with_errors.ksh | 1 + .../removal/removal_with_export.ksh | 1 + .../functional/removal/removal_with_send.ksh | 2 +- .../removal/remove_attach_mirror.ksh | 2 + .../removal/remove_mirror_sanity.ksh | 4 +- .../tests/functional/rename_dirs/Makefile.am | 5 - .../tests/functional/rename_dirs/cleanup.ksh | 2 +- .../rename_dirs/rename_dirs_001_pos.ksh | 7 +- .../tests/functional/rename_dirs/setup.ksh | 2 +- .../tests/functional/replacement/Makefile.am | 21 - .../functional/replacement/attach_rebuild.ksh | 12 +- .../replacement/attach_resilver.ksh | 12 +- .../tests/functional/replacement/cleanup.ksh | 2 +- .../tests/functional/replacement/detach.ksh | 12 +- .../replacement/rebuild_disabled_feature.ksh | 3 +- .../replacement/replace_rebuild.ksh | 7 +- .../replacement/replace_resilver.ksh | 7 +- .../functional/replacement/replacement.cfg | 2 +- .../replacement/resilver_restart_001.ksh | 2 +- .../replacement/resilver_restart_002.ksh | 10 +- .../tests/functional/replacement/setup.ksh | 4 +- .../tests/functional/reservation/Makefile.am | 30 - .../tests/functional/reservation/cleanup.ksh | 2 +- .../functional/reservation/reservation.cfg | 2 +- .../functional/reservation/reservation.shlib | 20 +- .../reservation/reservation_001_pos.ksh | 10 +- .../reservation/reservation_002_pos.ksh | 2 +- .../reservation/reservation_003_pos.ksh | 2 +- .../reservation/reservation_004_pos.ksh | 2 +- .../reservation/reservation_005_pos.ksh | 2 +- .../reservation/reservation_006_pos.ksh | 2 +- .../reservation/reservation_007_pos.ksh | 2 +- .../reservation/reservation_008_pos.ksh | 2 +- .../reservation/reservation_009_pos.ksh | 2 +- .../reservation/reservation_010_pos.ksh | 2 +- .../reservation/reservation_011_pos.ksh | 2 +- .../reservation/reservation_012_pos.ksh | 2 +- .../reservation/reservation_013_pos.ksh | 4 +- .../reservation/reservation_014_pos.ksh | 2 +- .../reservation/reservation_015_pos.ksh | 2 +- .../reservation/reservation_016_pos.ksh | 2 +- .../reservation/reservation_017_pos.ksh | 2 +- .../reservation/reservation_018_pos.ksh | 2 +- .../tests/functional/reservation/setup.ksh | 3 +- .../tests/functional/rootpool/Makefile.am | 7 - .../tests/functional/rootpool/cleanup.ksh | 2 +- .../functional/rootpool/rootpool_002_neg.ksh | 4 +- .../functional/rootpool/rootpool_003_neg.ksh | 2 +- .../functional/rootpool/rootpool_007_pos.ksh | 2 +- .../tests/functional/rootpool/setup.ksh | 4 +- .../tests/functional/rsend/Makefile.am | 67 - .../tests/functional/rsend/cleanup.ksh | 2 +- .../tests/functional/rsend/recv_dedup.ksh | 3 +- .../tests/functional/rsend/rsend.cfg | 7 +- .../tests/functional/rsend/rsend.kshlib | 135 +- .../tests/functional/rsend/rsend_001_pos.ksh | 2 +- .../tests/functional/rsend/rsend_002_pos.ksh | 2 +- .../tests/functional/rsend/rsend_003_pos.ksh | 2 +- .../tests/functional/rsend/rsend_004_pos.ksh | 2 +- .../tests/functional/rsend/rsend_005_pos.ksh | 2 +- .../tests/functional/rsend/rsend_006_pos.ksh | 2 +- .../tests/functional/rsend/rsend_007_pos.ksh | 2 +- .../tests/functional/rsend/rsend_008_pos.ksh | 2 +- .../tests/functional/rsend/rsend_009_pos.ksh | 6 +- .../tests/functional/rsend/rsend_010_pos.ksh | 2 +- .../tests/functional/rsend/rsend_011_pos.ksh | 20 +- .../tests/functional/rsend/rsend_012_pos.ksh | 46 +- .../tests/functional/rsend/rsend_013_pos.ksh | 2 +- .../tests/functional/rsend/rsend_016_neg.ksh | 11 +- .../tests/functional/rsend/rsend_025_pos.ksh | 90 + .../tests/functional/rsend/rsend_026_neg.ksh | 58 + .../tests/functional/rsend/rsend_027_pos.ksh | 92 + .../tests/functional/rsend/rsend_028_neg.ksh | 58 + .../tests/functional/rsend/rsend_029_neg.ksh | 58 + .../tests/functional/rsend/send-c_props.ksh | 19 +- .../rsend/send-c_stream_size_estimate.ksh | 28 +- .../functional/rsend/send-c_zstreamdump.ksh | 6 +- .../rsend/send-cpL_varied_recsize.ksh | 2 +- .../tests/functional/rsend/send_doall.ksh | 12 +- .../rsend/send_encrypted_truncated_files.ksh | 4 +- .../functional/rsend/send_hole_birth.ksh | 2 +- .../tests/functional/rsend/send_invalid.ksh | 2 +- .../functional/rsend/send_partial_dataset.ksh | 4 +- .../functional/rsend/send_raw_ashift.ksh | 193 + .../tests/functional/rsend/setup.ksh | 2 +- .../tests/functional/scrub_mirror/Makefile.am | 12 - .../tests/functional/scrub_mirror/cleanup.ksh | 2 +- .../tests/functional/scrub_mirror/default.cfg | 2 +- .../scrub_mirror/scrub_mirror_001_pos.ksh | 2 +- .../scrub_mirror/scrub_mirror_002_pos.ksh | 2 +- .../scrub_mirror/scrub_mirror_003_pos.ksh | 2 +- .../scrub_mirror/scrub_mirror_004_pos.ksh | 2 +- .../scrub_mirror/scrub_mirror_common.kshlib | 2 +- .../tests/functional/scrub_mirror/setup.ksh | 2 +- .../tests/functional/simd/Makefile.am | 2 - .../tests/functional/simd/simd_supported.ksh | 12 +- .../tests/functional/slog/Makefile.am | 26 - .../tests/functional/slog/cleanup.ksh | 2 +- .../zfs-tests/tests/functional/slog/setup.ksh | 2 +- .../zfs-tests/tests/functional/slog/slog.cfg | 2 +- .../tests/functional/slog/slog.kshlib | 4 +- .../tests/functional/slog/slog_001_pos.ksh | 2 +- .../tests/functional/slog/slog_002_pos.ksh | 2 +- .../tests/functional/slog/slog_003_pos.ksh | 2 +- .../tests/functional/slog/slog_004_pos.ksh | 2 +- .../tests/functional/slog/slog_005_pos.ksh | 2 +- .../tests/functional/slog/slog_006_pos.ksh | 2 +- .../tests/functional/slog/slog_007_pos.ksh | 2 +- .../tests/functional/slog/slog_008_neg.ksh | 2 +- .../tests/functional/slog/slog_009_neg.ksh | 2 +- .../tests/functional/slog/slog_010_neg.ksh | 2 +- .../tests/functional/slog/slog_011_neg.ksh | 2 +- .../tests/functional/slog/slog_012_neg.ksh | 2 +- .../tests/functional/slog/slog_013_pos.ksh | 2 +- .../tests/functional/slog/slog_014_pos.ksh | 8 +- .../tests/functional/slog/slog_015_neg.ksh | 2 + .../tests/functional/slog/slog_016_pos.ksh | 157 + .../functional/slog/slog_replay_fs_001.ksh | 8 +- .../functional/slog/slog_replay_fs_002.ksh | 8 +- .../functional/slog/slog_replay_volume.ksh | 2 +- .../tests/functional/snapshot/Makefile.am | 28 - .../tests/functional/snapshot/cleanup.ksh | 2 +- .../functional/snapshot/clone_001_pos.ksh | 10 +- .../functional/snapshot/rollback_001_pos.ksh | 18 +- .../functional/snapshot/rollback_002_pos.ksh | 24 +- .../functional/snapshot/rollback_003_pos.ksh | 6 +- .../tests/functional/snapshot/setup.ksh | 2 +- .../tests/functional/snapshot/snapshot.cfg | 2 +- .../functional/snapshot/snapshot_001_pos.ksh | 21 +- .../functional/snapshot/snapshot_002_pos.ksh | 60 +- .../functional/snapshot/snapshot_003_pos.ksh | 23 +- .../functional/snapshot/snapshot_004_pos.ksh | 16 +- .../functional/snapshot/snapshot_005_pos.ksh | 23 +- .../functional/snapshot/snapshot_006_pos.ksh | 61 +- .../functional/snapshot/snapshot_007_pos.ksh | 27 +- .../functional/snapshot/snapshot_008_pos.ksh | 13 +- .../functional/snapshot/snapshot_009_pos.ksh | 2 +- .../functional/snapshot/snapshot_010_pos.ksh | 2 +- .../functional/snapshot/snapshot_011_pos.ksh | 15 +- .../functional/snapshot/snapshot_012_pos.ksh | 2 +- .../functional/snapshot/snapshot_013_pos.ksh | 10 +- .../functional/snapshot/snapshot_014_pos.ksh | 8 +- .../functional/snapshot/snapshot_015_pos.ksh | 2 +- .../functional/snapshot/snapshot_016_pos.ksh | 2 +- .../functional/snapshot/snapshot_017_pos.ksh | 2 +- .../tests/functional/snapused/Makefile.am | 12 - .../tests/functional/snapused/cleanup.ksh | 2 +- .../tests/functional/snapused/setup.ksh | 2 +- .../tests/functional/snapused/snapused.kshlib | 2 +- .../functional/snapused/snapused_001_pos.ksh | 2 +- .../functional/snapused/snapused_002_pos.ksh | 2 +- .../functional/snapused/snapused_003_pos.ksh | 2 +- .../functional/snapused/snapused_004_pos.ksh | 2 +- .../functional/snapused/snapused_005_pos.ksh | 2 +- .../tests/functional/sparse/Makefile.am | 8 - .../tests/functional/sparse/cleanup.ksh | 2 +- .../tests/functional/sparse/setup.ksh | 2 +- .../tests/functional/sparse/sparse.cfg | 4 +- .../functional/sparse/sparse_001_pos.ksh | 2 +- .../tests/functional/stat/Makefile.am | 8 - .../tests/functional/stat/cleanup.ksh | 2 +- .../zfs-tests/tests/functional/stat/setup.ksh | 2 +- .../tests/functional/stat/stat_001_pos.ksh | 2 +- .../tests/functional/suid/.gitignore | 1 - .../tests/functional/suid/Makefile.am | 17 - .../tests/functional/suid/cleanup.ksh | 2 +- .../zfs-tests/tests/functional/suid/setup.ksh | 2 +- .../functional/suid/suid_write_to_none.ksh | 4 +- .../functional/suid/suid_write_to_sgid.ksh | 4 +- .../functional/suid/suid_write_to_suid.ksh | 4 +- .../suid/suid_write_to_suid_sgid.ksh | 4 +- .../functional/suid/suid_write_zil_replay.ksh | 18 +- .../tests/functional/threadsappend/.gitignore | 1 - .../functional/threadsappend/Makefile.am | 8 - .../tests/functional/tmpfile/Makefile.am | 16 - .../tests/functional/tmpfile/cleanup.ksh | 2 +- .../tests/functional/tmpfile/setup.ksh | 2 +- .../functional/tmpfile/tmpfile_001_pos.c | 81 +- .../functional/tmpfile/tmpfile_002_pos.c | 70 +- .../functional/tmpfile/tmpfile_003_pos.c | 33 +- .../functional/tmpfile/tmpfile_stat_mode.c | 52 +- .../tests/functional/trim/Makefile.am | 12 - .../zfs-tests/tests/functional/trim/trim.cfg | 2 +- .../tests/functional/trim/trim.kshlib | 13 +- .../tests/functional/trim/trim_l2arc.ksh | 6 +- .../tests/functional/truncate/.gitignore | 1 - .../tests/functional/truncate/Makefile.am | 18 - .../tests/functional/truncate/cleanup.ksh | 2 +- .../tests/functional/truncate/setup.ksh | 2 +- .../tests/functional/truncate/truncate.cfg | 4 +- .../functional/truncate/truncate_001_pos.ksh | 2 +- .../functional/truncate/truncate_002_pos.ksh | 2 +- .../truncate/truncate_timestamps.ksh | 4 +- .../tests/functional/upgrade/Makefile.am | 10 - .../tests/functional/upgrade/cleanup.ksh | 2 +- .../tests/functional/upgrade/setup.ksh | 2 +- .../functional/upgrade/upgrade_common.kshlib | 2 +- .../upgrade/upgrade_projectquota_001_pos.ksh | 4 +- .../upgrade/upgrade_userobj_001_pos.ksh | 2 +- .../functional/user_namespace/Makefile.am | 9 - .../functional/user_namespace/cleanup.ksh | 2 +- .../tests/functional/user_namespace/setup.ksh | 2 +- .../user_namespace/user_namespace.cfg | 2 +- .../user_namespace/user_namespace_001.ksh | 11 +- .../user_namespace/user_namespace_002.ksh | 115 + .../user_namespace/user_namespace_003.ksh | 97 + .../user_namespace/user_namespace_004.ksh | 67 + .../user_namespace_common.kshlib | 2 +- .../tests/functional/userquota/Makefile.am | 29 - .../tests/functional/userquota/cleanup.ksh | 2 +- .../userquota/groupspace_001_pos.ksh | 2 +- .../userquota/groupspace_002_pos.ksh | 2 +- .../userquota/groupspace_003_pos.ksh | 2 +- .../tests/functional/userquota/setup.ksh | 10 +- .../tests/functional/userquota/userquota.cfg | 2 +- .../userquota/userquota_001_pos.ksh | 2 +- .../userquota/userquota_002_pos.ksh | 2 +- .../userquota/userquota_003_pos.ksh | 2 +- .../userquota/userquota_004_pos.ksh | 12 +- .../userquota/userquota_005_neg.ksh | 2 +- .../userquota/userquota_006_pos.ksh | 2 +- .../userquota/userquota_007_pos.ksh | 2 +- .../userquota/userquota_008_pos.ksh | 2 +- .../userquota/userquota_009_pos.ksh | 2 +- .../userquota/userquota_010_pos.ksh | 2 +- .../userquota/userquota_011_pos.ksh | 2 +- .../userquota/userquota_012_neg.ksh | 2 +- .../userquota/userquota_013_pos.ksh | 2 +- .../userquota/userquota_common.kshlib | 22 +- .../userquota/userspace_001_pos.ksh | 2 +- .../userquota/userspace_002_pos.ksh | 2 +- .../userquota/userspace_003_pos.ksh | 2 +- .../userquota/userspace_encrypted.ksh | 3 +- .../userquota/userspace_send_encrypted.ksh | 2 +- .../tests/functional/vdev_zaps/Makefile.am | 14 - .../functional/vdev_zaps/vdev_zaps.kshlib | 44 +- .../vdev_zaps/vdev_zaps_001_pos.ksh | 2 +- .../vdev_zaps/vdev_zaps_002_pos.ksh | 2 +- .../vdev_zaps/vdev_zaps_003_pos.ksh | 2 +- .../vdev_zaps/vdev_zaps_004_pos.ksh | 9 +- .../vdev_zaps/vdev_zaps_005_pos.ksh | 4 +- .../vdev_zaps/vdev_zaps_006_pos.ksh | 2 +- .../vdev_zaps/vdev_zaps_007_pos.ksh | 6 +- .../tests/functional/write_dirs/Makefile.am | 6 - .../tests/functional/write_dirs/cleanup.ksh | 2 +- .../tests/functional/write_dirs/setup.ksh | 6 +- .../write_dirs/write_dirs_001_pos.ksh | 2 +- .../write_dirs/write_dirs_002_pos.ksh | 2 +- .../tests/functional/xattr/Makefile.am | 21 - .../tests/functional/xattr/cleanup.ksh | 6 +- .../tests/functional/xattr/setup.ksh | 9 +- .../tests/functional/xattr/xattr.cfg | 2 +- .../tests/functional/xattr/xattr_001_pos.ksh | 2 +- .../tests/functional/xattr/xattr_002_neg.ksh | 2 +- .../tests/functional/xattr/xattr_003_neg.ksh | 6 +- .../tests/functional/xattr/xattr_004_pos.ksh | 4 +- .../tests/functional/xattr/xattr_005_pos.ksh | 2 +- .../tests/functional/xattr/xattr_006_pos.ksh | 2 +- .../tests/functional/xattr/xattr_007_neg.ksh | 2 +- .../tests/functional/xattr/xattr_008_pos.ksh | 2 +- .../tests/functional/xattr/xattr_009_neg.ksh | 2 +- .../tests/functional/xattr/xattr_010_neg.ksh | 4 +- .../tests/functional/xattr/xattr_011_pos.ksh | 10 +- .../tests/functional/xattr/xattr_012_pos.ksh | 23 +- .../tests/functional/xattr/xattr_013_pos.ksh | 8 +- .../functional/xattr/xattr_common.kshlib | 20 +- .../tests/functional/xattr/xattr_compat.ksh | 92 + .../functional/zpool_influxdb/Makefile.am | 5 - .../zpool_influxdb/zpool_influxdb.ksh | 11 +- .../tests/functional/zvol/Makefile.am | 10 - .../zfs-tests/tests/functional/zvol/zvol.cfg | 2 +- .../functional/zvol/zvol_ENOSPC/Makefile.am | 8 - .../functional/zvol/zvol_ENOSPC/cleanup.ksh | 2 +- .../functional/zvol/zvol_ENOSPC/setup.ksh | 2 +- .../zvol/zvol_ENOSPC/zvol_ENOSPC.cfg | 4 +- .../zvol/zvol_ENOSPC/zvol_ENOSPC_001_pos.ksh | 2 +- .../functional/zvol/zvol_cli/Makefile.am | 10 - .../functional/zvol/zvol_cli/cleanup.ksh | 2 +- .../tests/functional/zvol/zvol_cli/setup.ksh | 2 +- .../functional/zvol/zvol_cli/zvol_cli.cfg | 4 +- .../zvol/zvol_cli/zvol_cli_001_pos.ksh | 2 +- .../zvol/zvol_cli/zvol_cli_002_pos.ksh | 2 +- .../zvol/zvol_cli/zvol_cli_003_neg.ksh | 2 +- .../tests/functional/zvol/zvol_common.shlib | 22 +- .../functional/zvol/zvol_misc/Makefile.am | 19 - .../functional/zvol/zvol_misc/cleanup.ksh | 2 +- .../tests/functional/zvol/zvol_misc/setup.ksh | 2 +- .../zvol/zvol_misc/zvol_misc_001_neg.ksh | 2 +- .../zvol/zvol_misc/zvol_misc_002_pos.ksh | 17 +- .../zvol/zvol_misc/zvol_misc_003_neg.ksh | 2 +- .../zvol/zvol_misc/zvol_misc_004_pos.ksh | 7 +- .../zvol/zvol_misc/zvol_misc_005_neg.ksh | 7 +- .../zvol/zvol_misc/zvol_misc_006_pos.ksh | 2 +- .../zvol/zvol_misc/zvol_misc_common.kshlib | 7 +- .../zvol/zvol_misc/zvol_misc_fua.ksh | 96 + .../zvol/zvol_misc/zvol_misc_snapdev.ksh | 2 +- .../zvol/zvol_misc/zvol_misc_trim.ksh | 136 + .../zvol/zvol_misc/zvol_misc_volmode.ksh | 2 +- .../zvol/zvol_misc/zvol_misc_zil.ksh | 2 +- .../functional/zvol/zvol_stress/cleanup.ksh} | 37 +- .../functional/zvol/zvol_stress/setup.ksh | 36 + .../zvol/zvol_stress/zvol_stress.ksh | 169 + .../functional/zvol/zvol_swap/Makefile.am | 13 - .../functional/zvol/zvol_swap/cleanup.ksh | 6 +- .../tests/functional/zvol/zvol_swap/setup.ksh | 6 +- .../functional/zvol/zvol_swap/zvol_swap.cfg | 8 +- .../zvol/zvol_swap/zvol_swap_001_pos.ksh | 2 +- .../zvol/zvol_swap/zvol_swap_002_pos.ksh | 2 +- .../zvol/zvol_swap/zvol_swap_003_pos.ksh | 4 +- .../zvol/zvol_swap/zvol_swap_004_pos.ksh | 2 +- .../zvol/zvol_swap/zvol_swap_005_pos.ksh | 2 +- .../zvol/zvol_swap/zvol_swap_006_pos.ksh | 2 +- tests/zfs-tests/tests/perf/Makefile.am | 9 - tests/zfs-tests/tests/perf/fio/Makefile.am | 10 - tests/zfs-tests/tests/perf/perf.shlib | 141 +- .../tests/perf/regression/Makefile.am | 13 - .../tests/perf/regression/random_reads.ksh | 2 + .../perf/regression/random_readwrite.ksh | 2 + .../regression/random_readwrite_fixed.ksh | 2 + .../tests/perf/regression/random_writes.ksh | 2 + .../perf/regression/random_writes_zil.ksh | 2 + .../sequential_reads_arc_cached.ksh | 2 + .../sequential_reads_arc_cached_clone.ksh | 2 + .../sequential_reads_dbuf_cached.ksh | 2 + .../perf/regression/sequential_writes.ksh | 2 + .../zfs-tests/tests/perf/scripts/Makefile.am | 2 - .../tests/perf/scripts/prefetch_io.sh | 40 +- tests/zfs-tests/tests/stress/Makefile.am | 1 - udev/.gitignore | 1 + udev/Makefile.am | 19 +- udev/rules.d/Makefile.am | 8 - {cmd/vdev_id => udev}/vdev_id | 0 udev/zvol_id.c | 86 + 2873 files changed, 78964 insertions(+), 67188 deletions(-) rename cmd/{arc_summary/arc_summary3 => arc_summary} (100%) delete mode 100644 cmd/arc_summary/.gitignore delete mode 100644 cmd/arc_summary/Makefile.am rename cmd/{arcstat => }/arcstat.in (99%) delete mode 100644 cmd/arcstat/.gitignore delete mode 100644 cmd/arcstat/Makefile.am rename cmd/{dbufstat => }/dbufstat.in (99%) delete mode 100644 cmd/dbufstat/.gitignore delete mode 100644 cmd/dbufstat/Makefile.am rename cmd/{fsck_zfs => }/fsck.zfs.in (93%) delete mode 100644 cmd/fsck_zfs/.gitignore delete mode 100644 cmd/fsck_zfs/Makefile.am rename cmd/{mount_zfs => }/mount_zfs.c (97%) delete mode 100644 cmd/mount_zfs/.gitignore delete mode 100644 cmd/mount_zfs/Makefile.am delete mode 100644 cmd/raidz_test/.gitignore delete mode 100644 cmd/vdev_id/Makefile.am delete mode 100644 cmd/zdb/.gitignore delete mode 100644 cmd/zfs/.gitignore rename cmd/{zfs_ids_to_path => }/zfs_ids_to_path.c (98%) delete mode 100644 cmd/zfs_ids_to_path/.gitignore delete mode 100644 cmd/zfs_ids_to_path/Makefile.am rename cmd/{zgenhostid => }/zgenhostid.c (98%) delete mode 100644 cmd/zgenhostid/.gitignore delete mode 100644 cmd/zgenhostid/Makefile.am rename cmd/{zhack => }/zhack.c (97%) delete mode 100644 cmd/zhack/.gitignore delete mode 100644 cmd/zhack/Makefile.am delete mode 100644 cmd/zinject/.gitignore delete mode 100644 cmd/zpool/.gitignore delete mode 100644 cmd/zpool_influxdb/.gitignore delete mode 100644 cmd/zstream/.gitignore create mode 100644 cmd/zstream/zstream_decompress.c rename cmd/{ztest => }/ztest.c (96%) delete mode 100644 cmd/ztest/.gitignore delete mode 100644 cmd/ztest/Makefile.am delete mode 100644 cmd/zvol_id/.gitignore delete mode 100644 cmd/zvol_id/Makefile.am delete mode 100644 cmd/zvol_id/zvol_id_main.c rename cmd/{zvol_wait => }/zvol_wait (87%) delete mode 100644 cmd/zvol_wait/Makefile.am delete mode 100644 config/Abigail.am delete mode 100644 config/config.awk create mode 100644 config/kernel-copy-from-user-inatomic.m4 create mode 100644 config/kernel-genhd-flags.m4 create mode 100644 config/kernel-inode-permission.m4 create mode 100644 config/kernel-readpages.m4 create mode 100644 config/kernel-sysfs.m4 create mode 100644 config/kernel-user-ns-inum.m4 create mode 100644 config/kernel-vfs-filemap_dirty_folio.m4 create mode 100644 config/kernel-vfs-read_folio.m4 create mode 100644 config/kernel-zero_page.m4 create mode 100644 config/user-aio.h.m4 rename contrib/dracut/{90zfs => }/.gitignore (100%) delete mode 100644 contrib/dracut/02zfsexpandknowledge/.gitignore delete mode 100644 contrib/dracut/02zfsexpandknowledge/Makefile.am delete mode 100644 contrib/dracut/90zfs/Makefile.am delete mode 100644 contrib/initramfs/conf-hooks.d/Makefile.am delete mode 100644 contrib/initramfs/conf.d/Makefile.am delete mode 100644 contrib/initramfs/hooks/Makefile.am delete mode 100644 contrib/initramfs/scripts/Makefile.am delete mode 100644 contrib/initramfs/scripts/local-top/Makefile.am delete mode 100644 etc/default/Makefile.am delete mode 100644 etc/init.d/Makefile.am delete mode 100644 etc/modules-load.d/.gitignore delete mode 100644 etc/modules-load.d/Makefile.am delete mode 100644 etc/sudoers.d/Makefile.am delete mode 100644 etc/systemd/Makefile.am delete mode 100644 etc/systemd/system-generators/Makefile.am rename etc/systemd/system/{50-zfs.preset.in => 50-zfs.preset} (100%) delete mode 100644 etc/systemd/system/Makefile.am create mode 100644 etc/systemd/system/zfs-trim-monthly@.timer.in create mode 100644 etc/systemd/system/zfs-trim-weekly@.timer.in create mode 100644 etc/systemd/system/zfs-trim@.service.in delete mode 100644 etc/zfs/Makefile.am delete mode 100644 include/os/Makefile.am delete mode 100644 include/os/freebsd/linux/Makefile.am delete mode 100644 include/os/freebsd/spl/Makefile.am delete mode 100644 include/os/freebsd/spl/acl/Makefile.am delete mode 100644 include/os/freebsd/spl/rpc/Makefile.am delete mode 100644 include/os/freebsd/spl/sys/Makefile.am delete mode 100644 include/os/freebsd/spl/sys/kidmap.h delete mode 100644 include/os/freebsd/spl/sys/strings.h delete mode 100644 include/os/freebsd/zfs/Makefile.am delete mode 100644 include/os/freebsd/zfs/sys/Makefile.am delete mode 100644 include/os/linux/kernel/Makefile.am delete mode 100644 include/os/linux/kernel/linux/Makefile.am delete mode 100644 include/os/linux/spl/Makefile.am delete mode 100644 include/os/linux/spl/rpc/Makefile.am delete mode 100644 include/os/linux/spl/sys/Makefile.am create mode 100644 include/os/linux/spl/sys/string.h delete mode 100644 include/os/linux/spl/sys/strings.h delete mode 100644 include/os/linux/zfs/Makefile.am delete mode 100644 include/os/linux/zfs/sys/Makefile.am delete mode 100644 include/sys/Makefile.am create mode 100644 include/sys/blake3.h delete mode 100644 include/sys/crypto/Makefile.am delete mode 100644 include/sys/fm/Makefile.am delete mode 100644 include/sys/fm/fs/Makefile.am delete mode 100644 include/sys/fs/Makefile.am delete mode 100644 include/sys/lua/Makefile.am delete mode 100644 include/sys/sysevent/Makefile.am rename lib/libspl/include/sys/stropts.h => include/sys/zfs_chksum.h (63%) delete mode 100644 include/sys/zstd/Makefile.am delete mode 100644 lib/libspl/include/libdevinfo.h delete mode 100644 lib/libspl/include/limits.h delete mode 100644 lib/libspl/include/locale.h delete mode 100644 lib/libspl/include/os/Makefile.am delete mode 100644 lib/libspl/include/os/freebsd/Makefile.am delete mode 100644 lib/libspl/include/os/freebsd/sys/Makefile.am delete mode 100644 lib/libspl/include/os/linux/Makefile.am delete mode 100644 lib/libspl/include/os/linux/sys/Makefile.am delete mode 100644 lib/libspl/include/rpc/Makefile.am delete mode 100644 lib/libspl/include/stdio.h delete mode 100644 lib/libspl/include/stropts.h delete mode 100644 lib/libspl/include/sys/Makefile.am delete mode 100644 lib/libspl/include/sys/dktp/Makefile.am delete mode 100644 lib/libspl/include/sys/int_limits.h delete mode 100644 lib/libspl/include/sys/int_types.h create mode 100644 lib/libspl/include/sys/string.h delete mode 100644 lib/libspl/include/sys/strings.h delete mode 100644 lib/libspl/include/sys/tzfile.h delete mode 100644 lib/libspl/include/sys/va_list.h delete mode 100644 lib/libspl/include/sys/varargs.h delete mode 100644 lib/libspl/include/sys/vtoc.h delete mode 100644 lib/libspl/include/thread.h delete mode 100644 lib/libspl/include/tzfile.h delete mode 100644 lib/libspl/include/ucred.h delete mode 100644 lib/libspl/include/util/Makefile.am delete mode 100644 lib/libspl/include/util/sscanf.h delete mode 100644 lib/libzfs/os/linux/libzfs_sendrecv_os.c rename {tests/test-runner/man => man/man1}/test-runner.1 (99%) create mode 100644 man/man7/dracut.zfs.7 create mode 120000 man/man8/zfs-unzone.8 create mode 100644 man/man8/zfs-zone.8 delete mode 100644 module/avl/Makefile.in delete mode 100644 module/icp/Makefile.in create mode 100644 module/icp/algs/blake3/blake3.c create mode 100644 module/icp/algs/blake3/blake3_generic.c create mode 100644 module/icp/algs/blake3/blake3_impl.c create mode 100644 module/icp/algs/blake3/blake3_impl.h create mode 100644 module/icp/algs/blake3/blake3_x86-64.c delete mode 100644 module/icp/api/kcf_digest.c delete mode 100644 module/icp/api/kcf_miscapi.c create mode 100644 module/icp/asm-aarch64/blake3/b3_aarch64_sse2.S create mode 100644 module/icp/asm-aarch64/blake3/b3_aarch64_sse41.S create mode 100644 module/icp/asm-ppc64/blake3/b3_ppc64le_sse2.S create mode 100644 module/icp/asm-ppc64/blake3/b3_ppc64le_sse41.S create mode 100644 module/icp/asm-x86_64/blake3/blake3_avx2.S create mode 100644 module/icp/asm-x86_64/blake3/blake3_avx512.S create mode 100644 module/icp/asm-x86_64/blake3/blake3_sse2.S create mode 100644 module/icp/asm-x86_64/blake3/blake3_sse41.S delete mode 100644 module/icp/include/sys/bitmap.h delete mode 100644 module/icp/include/sys/crypto/elfsign.h delete mode 100644 module/icp/include/sys/crypto/ops_impl.h delete mode 100644 module/icp/include/sys/modhash.h delete mode 100644 module/icp/include/sys/modhash_impl.h delete mode 100644 module/icp/os/modhash.c delete mode 100644 module/lua/Makefile.in delete mode 100644 module/nvpair/Makefile.in delete mode 100644 module/os/linux/spl/Makefile.in create mode 100644 module/os/linux/spl/spl-zone.c delete mode 100644 module/os/linux/zfs/Makefile.in delete mode 100644 module/spl/Makefile.in delete mode 100644 module/unicode/Makefile.in delete mode 100644 module/zcommon/Makefile.in delete mode 100644 module/zfs/Makefile.in create mode 100644 module/zfs/blake3_zfs.c create mode 100644 module/zfs/zfs_chksum.c delete mode 100644 module/zstd/Makefile.in create mode 100644 module/zstd/lib/common/bitstream.h create mode 100644 module/zstd/lib/common/compiler.h create mode 100644 module/zstd/lib/common/cpu.h create mode 100644 module/zstd/lib/common/debug.h create mode 100644 module/zstd/lib/common/entropy_common.c create mode 100644 module/zstd/lib/common/error_private.c create mode 100644 module/zstd/lib/common/error_private.h create mode 100644 module/zstd/lib/common/fse.h create mode 100644 module/zstd/lib/common/fse_decompress.c create mode 100644 module/zstd/lib/common/huf.h create mode 100644 module/zstd/lib/common/mem.h create mode 100644 module/zstd/lib/common/pool.c create mode 100644 module/zstd/lib/common/pool.h create mode 100644 module/zstd/lib/common/xxhash.c create mode 100644 module/zstd/lib/common/xxhash.h create mode 100644 module/zstd/lib/common/zstd_common.c rename module/zstd/lib/{ => common}/zstd_errors.h (100%) create mode 100644 module/zstd/lib/common/zstd_internal.h create mode 100644 module/zstd/lib/compress/fse_compress.c create mode 100644 module/zstd/lib/compress/hist.c create mode 100644 module/zstd/lib/compress/hist.h create mode 100644 module/zstd/lib/compress/huf_compress.c create mode 100644 module/zstd/lib/compress/zstd_compress.c create mode 100644 module/zstd/lib/compress/zstd_compress_internal.h create mode 100644 module/zstd/lib/compress/zstd_compress_literals.c create mode 100644 module/zstd/lib/compress/zstd_compress_literals.h create mode 100644 module/zstd/lib/compress/zstd_compress_sequences.c create mode 100644 module/zstd/lib/compress/zstd_compress_sequences.h create mode 100644 module/zstd/lib/compress/zstd_compress_superblock.c create mode 100644 module/zstd/lib/compress/zstd_compress_superblock.h create mode 100644 module/zstd/lib/compress/zstd_cwksp.h create mode 100644 module/zstd/lib/compress/zstd_double_fast.c create mode 100644 module/zstd/lib/compress/zstd_double_fast.h create mode 100644 module/zstd/lib/compress/zstd_fast.c create mode 100644 module/zstd/lib/compress/zstd_fast.h create mode 100644 module/zstd/lib/compress/zstd_lazy.c create mode 100644 module/zstd/lib/compress/zstd_lazy.h create mode 100644 module/zstd/lib/compress/zstd_ldm.c create mode 100644 module/zstd/lib/compress/zstd_ldm.h create mode 100644 module/zstd/lib/compress/zstd_opt.c create mode 100644 module/zstd/lib/compress/zstd_opt.h create mode 100644 module/zstd/lib/decompress/huf_decompress.c create mode 100644 module/zstd/lib/decompress/zstd_ddict.c create mode 100644 module/zstd/lib/decompress/zstd_ddict.h create mode 100644 module/zstd/lib/decompress/zstd_decompress.c create mode 100644 module/zstd/lib/decompress/zstd_decompress_block.c create mode 100644 module/zstd/lib/decompress/zstd_decompress_block.h create mode 100644 module/zstd/lib/decompress/zstd_decompress_internal.h delete mode 100644 module/zstd/lib/zstd.c create mode 100644 rpm/.gitignore delete mode 100644 rpm/generic/.gitignore delete mode 100644 rpm/generic/Makefile.am delete mode 100644 rpm/redhat/.gitignore delete mode 100644 rpm/redhat/Makefile.am create mode 100755 scripts/zfs-tests-color.sh delete mode 100644 tests/runfiles/Makefile.am delete mode 100644 tests/test-runner/Makefile.am delete mode 100644 tests/test-runner/bin/Makefile.am delete mode 100644 tests/test-runner/include/Makefile.am delete mode 100644 tests/test-runner/man/Makefile.am create mode 100644 tests/zfs-tests/.gitignore delete mode 100644 tests/zfs-tests/callbacks/Makefile.am create mode 100644 tests/zfs-tests/cmd/.gitignore rename tests/zfs-tests/cmd/{badsend => }/badsend.c (98%) delete mode 100644 tests/zfs-tests/cmd/badsend/.gitignore delete mode 100644 tests/zfs-tests/cmd/badsend/Makefile.am rename tests/zfs-tests/cmd/{btree_test => }/btree_test.c (100%) delete mode 100644 tests/zfs-tests/cmd/btree_test/.gitignore delete mode 100644 tests/zfs-tests/cmd/btree_test/Makefile.am create mode 100644 tests/zfs-tests/cmd/checksum/blake3_test.c rename tests/zfs-tests/{tests/functional => cmd}/checksum/edonr_test.c (98%) rename tests/zfs-tests/{tests/functional => cmd}/checksum/sha2_test.c (98%) rename tests/zfs-tests/{tests/functional => cmd}/checksum/skein_test.c (98%) rename tests/zfs-tests/cmd/{chg_usr_exec => }/chg_usr_exec.c (97%) delete mode 100644 tests/zfs-tests/cmd/chg_usr_exec/.gitignore delete mode 100644 tests/zfs-tests/cmd/chg_usr_exec/Makefile.am rename tests/zfs-tests/{tests/functional/cp_files => cmd}/cp_files.c (100%) rename tests/zfs-tests/{tests/functional/ctime => cmd}/ctime.c (98%) rename tests/zfs-tests/cmd/{devname2devid => }/devname2devid.c (98%) delete mode 100644 tests/zfs-tests/cmd/devname2devid/.gitignore delete mode 100644 tests/zfs-tests/cmd/devname2devid/Makefile.am rename tests/zfs-tests/cmd/{dir_rd_update => }/dir_rd_update.c (97%) delete mode 100644 tests/zfs-tests/cmd/dir_rd_update/.gitignore delete mode 100644 tests/zfs-tests/cmd/dir_rd_update/Makefile.am rename tests/zfs-tests/{tests/functional/acl/off => cmd}/dosmode_readonly_write.c (90%) rename tests/zfs-tests/cmd/{draid => }/draid.c (97%) delete mode 100644 tests/zfs-tests/cmd/draid/.gitignore delete mode 100644 tests/zfs-tests/cmd/draid/Makefile.am rename tests/zfs-tests/{tests/functional/cli_root/zpool_events => cmd}/ereports.c (98%) create mode 100644 tests/zfs-tests/cmd/file/file_append.c rename tests/zfs-tests/cmd/{file_check => file}/file_check.c (96%) rename tests/zfs-tests/cmd/{ => file}/file_common.h (95%) rename tests/zfs-tests/cmd/{file_trunc => file}/file_trunc.c (98%) rename tests/zfs-tests/cmd/{file_write => file}/file_write.c (97%) rename tests/zfs-tests/cmd/{largest_file => file}/largest_file.c (97%) rename tests/zfs-tests/cmd/{randfree_file => file}/randfree_file.c (97%) rename tests/zfs-tests/cmd/{randwritecomp => file}/randwritecomp.c (90%) delete mode 100644 tests/zfs-tests/cmd/file_check/.gitignore delete mode 100644 tests/zfs-tests/cmd/file_check/Makefile.am delete mode 100644 tests/zfs-tests/cmd/file_trunc/.gitignore delete mode 100644 tests/zfs-tests/cmd/file_trunc/Makefile.am delete mode 100644 tests/zfs-tests/cmd/file_write/.gitignore delete mode 100644 tests/zfs-tests/cmd/file_write/Makefile.am rename tests/zfs-tests/cmd/{get_diff => }/get_diff.c (95%) delete mode 100644 tests/zfs-tests/cmd/get_diff/.gitignore delete mode 100644 tests/zfs-tests/cmd/get_diff/Makefile.am rename tests/zfs-tests/cmd/{getversion => }/getversion.c (100%) delete mode 100644 tests/zfs-tests/cmd/getversion/.gitignore delete mode 100644 tests/zfs-tests/cmd/getversion/Makefile.am delete mode 100644 tests/zfs-tests/cmd/largest_file/.gitignore delete mode 100644 tests/zfs-tests/cmd/largest_file/Makefile.am rename tests/zfs-tests/cmd/{libzfs_input_check => }/libzfs_input_check.c (98%) delete mode 100644 tests/zfs-tests/cmd/libzfs_input_check/.gitignore delete mode 100644 tests/zfs-tests/cmd/libzfs_input_check/Makefile.am create mode 100644 tests/zfs-tests/cmd/linux_dos_attributes/dos_attributes.h create mode 100644 tests/zfs-tests/cmd/linux_dos_attributes/read_dos_attributes.c create mode 100644 tests/zfs-tests/cmd/linux_dos_attributes/write_dos_attributes.c rename tests/zfs-tests/cmd/{mkbusy => }/mkbusy.c (97%) delete mode 100644 tests/zfs-tests/cmd/mkbusy/.gitignore delete mode 100644 tests/zfs-tests/cmd/mkbusy/Makefile.am rename tests/zfs-tests/cmd/{mkfile => }/mkfile.c (98%) delete mode 100644 tests/zfs-tests/cmd/mkfile/.gitignore delete mode 100644 tests/zfs-tests/cmd/mkfile/Makefile.am rename tests/zfs-tests/cmd/{mkfiles => }/mkfiles.c (81%) delete mode 100644 tests/zfs-tests/cmd/mkfiles/.gitignore delete mode 100644 tests/zfs-tests/cmd/mkfiles/Makefile.am rename tests/zfs-tests/cmd/{mktree => }/mktree.c (97%) delete mode 100644 tests/zfs-tests/cmd/mktree/.gitignore delete mode 100644 tests/zfs-tests/cmd/mktree/Makefile.am rename tests/zfs-tests/cmd/{mmap_exec => }/mmap_exec.c (97%) delete mode 100644 tests/zfs-tests/cmd/mmap_exec/.gitignore delete mode 100644 tests/zfs-tests/cmd/mmap_exec/Makefile.am rename tests/zfs-tests/cmd/{mmap_libaio => }/mmap_libaio.c (97%) delete mode 100644 tests/zfs-tests/cmd/mmap_libaio/.gitignore delete mode 100644 tests/zfs-tests/cmd/mmap_libaio/Makefile.am rename tests/zfs-tests/cmd/{mmap_seek => }/mmap_seek.c (98%) delete mode 100644 tests/zfs-tests/cmd/mmap_seek/.gitignore delete mode 100644 tests/zfs-tests/cmd/mmap_seek/Makefile.am create mode 100644 tests/zfs-tests/cmd/mmap_sync.c rename tests/zfs-tests/cmd/{mmapwrite => }/mmapwrite.c (98%) delete mode 100644 tests/zfs-tests/cmd/mmapwrite/.gitignore delete mode 100644 tests/zfs-tests/cmd/mmapwrite/Makefile.am rename tests/zfs-tests/cmd/{nvlist_to_lua => }/nvlist_to_lua.c (96%) delete mode 100644 tests/zfs-tests/cmd/nvlist_to_lua/.gitignore delete mode 100644 tests/zfs-tests/cmd/nvlist_to_lua/Makefile.am delete mode 100644 tests/zfs-tests/cmd/randfree_file/.gitignore delete mode 100644 tests/zfs-tests/cmd/randfree_file/Makefile.am delete mode 100644 tests/zfs-tests/cmd/randwritecomp/.gitignore delete mode 100644 tests/zfs-tests/cmd/randwritecomp/Makefile.am rename tests/zfs-tests/cmd/{readmmap => }/readmmap.c (97%) delete mode 100644 tests/zfs-tests/cmd/readmmap/.gitignore delete mode 100644 tests/zfs-tests/cmd/readmmap/Makefile.am rename tests/zfs-tests/cmd/{rename_dir => }/rename_dir.c (92%) delete mode 100644 tests/zfs-tests/cmd/rename_dir/.gitignore delete mode 100644 tests/zfs-tests/cmd/rename_dir/Makefile.am rename tests/zfs-tests/cmd/{rm_lnkcnt_zero_file => }/rm_lnkcnt_zero_file.c (97%) delete mode 100644 tests/zfs-tests/cmd/rm_lnkcnt_zero_file/.gitignore delete mode 100644 tests/zfs-tests/cmd/rm_lnkcnt_zero_file/Makefile.am rename tests/zfs-tests/cmd/{send_doall => }/send_doall.c (97%) delete mode 100644 tests/zfs-tests/cmd/send_doall/.gitignore delete mode 100644 tests/zfs-tests/cmd/send_doall/Makefile.am rename tests/zfs-tests/cmd/{stride_dd => }/stride_dd.c (99%) delete mode 100644 tests/zfs-tests/cmd/stride_dd/.gitignore delete mode 100644 tests/zfs-tests/cmd/stride_dd/Makefile.am rename tests/zfs-tests/{tests/functional/suid => cmd}/suid_write_to_file.c (98%) rename tests/zfs-tests/cmd/{threadsappend => }/threadsappend.c (98%) delete mode 100644 tests/zfs-tests/cmd/threadsappend/.gitignore delete mode 100644 tests/zfs-tests/cmd/threadsappend/Makefile.am rename tests/zfs-tests/{tests/functional/truncate => cmd}/truncate_test.c (100%) rename tests/zfs-tests/cmd/{user_ns_exec => }/user_ns_exec.c (98%) delete mode 100644 tests/zfs-tests/cmd/user_ns_exec/.gitignore delete mode 100644 tests/zfs-tests/cmd/user_ns_exec/Makefile.am rename tests/zfs-tests/cmd/{xattrtest => }/xattrtest.c (97%) delete mode 100644 tests/zfs-tests/cmd/xattrtest/.gitignore delete mode 100644 tests/zfs-tests/cmd/xattrtest/Makefile.am rename tests/zfs-tests/{tests/functional/events => cmd}/zed_fd_spill-zedlet.c (100%) rename tests/zfs-tests/{tests/functional/cli_root/zfs_diff/socket.c => cmd/zfs_diff-socket.c} (100%) delete mode 100644 tests/zfs-tests/include/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/acl/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/acl/off/.gitignore delete mode 100644 tests/zfs-tests/tests/functional/acl/off/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/acl/posix-sa/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/acl/posix/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/alloc_class/Makefile.am create mode 100755 tests/zfs-tests/tests/functional/append/cleanup.ksh create mode 100755 tests/zfs-tests/tests/functional/append/file_append.ksh create mode 100755 tests/zfs-tests/tests/functional/append/setup.ksh rename tests/zfs-tests/tests/functional/{threadsappend => append}/threadsappend_001_pos.ksh (97%) delete mode 100644 tests/zfs-tests/tests/functional/arc/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/atime/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/bootfs/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/btree/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cache/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cachefile/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/casenorm/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/channel_program/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/channel_program/lua_core/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/channel_program/synctask_core/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/chattr/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/checksum/.gitignore delete mode 100644 tests/zfs-tests/tests/functional/checksum/Makefile.am rename tests/zfs-tests/tests/functional/{hkdf/cleanup.ksh => checksum/run_blake3_test.ksh} (61%) delete mode 100644 tests/zfs-tests/tests/functional/clean_mirror/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zdb/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_change-key/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_clone/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_copies/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_create/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_destroy/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_diff/.gitignore delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_diff/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_get/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_inherit/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_jail/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_load-key/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_mount/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_program/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_promote/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_property/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_receive/Makefile.am create mode 100755 tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_compressed_corrective.ksh create mode 100755 tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_corrective.ksh delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_rename/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_reservation/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_rollback/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_send/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_set/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_share/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_unmount/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_unshare/Makefile.am mode change 100755 => 120000 tests/zfs-tests/tests/functional/cli_root/zfs_unshare/setup.ksh create mode 100755 tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_008_pos.ksh delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zfs_wait/Makefile.am create mode 100755 tests/zfs-tests/tests/functional/cli_root/zfs_wait/zfs_wait_getsubopt.ksh delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zhack/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_add/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_attach/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_clear/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_create/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_destroy/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_detach/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_events/.gitignore delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_events/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_expand/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_export/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_get/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_history/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_import/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_import/blockfiles/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_initialize/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_labelclear/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_offline/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_online/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_remove/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_reopen/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_replace/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_resilver/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_scrub/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_set/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_split/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_status/Makefile.am create mode 100755 tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_003_pos.ksh create mode 100755 tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_004_pos.ksh delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_sync/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_trim/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/blockfiles/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_wait/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_root/zpool_wait/scan/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_user/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_user/misc/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_user/zfs_list/Makefile.am create mode 100755 tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_005_neg.ksh delete mode 100644 tests/zfs-tests/tests/functional/cli_user/zpool_iostat/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_user/zpool_list/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cli_user/zpool_status/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/compression/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/cp_files/.gitignore delete mode 100644 tests/zfs-tests/tests/functional/cp_files/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/crtime/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/ctime/.gitignore delete mode 100644 tests/zfs-tests/tests/functional/ctime/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/deadman/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/delegate/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/devices/Makefile.am rename tests/zfs-tests/tests/functional/{threadsappend => dos_attributes}/cleanup.ksh (95%) create mode 100755 tests/zfs-tests/tests/functional/dos_attributes/read_dos_attrs_001.ksh rename tests/zfs-tests/tests/functional/{threadsappend => dos_attributes}/setup.ksh (93%) create mode 100755 tests/zfs-tests/tests/functional/dos_attributes/write_dos_attrs_001.ksh delete mode 100644 tests/zfs-tests/tests/functional/events/.gitignore delete mode 100644 tests/zfs-tests/tests/functional/events/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/exec/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/fallocate/Makefile.am create mode 100755 tests/zfs-tests/tests/functional/fallocate/fallocate_zero-range.ksh delete mode 100644 tests/zfs-tests/tests/functional/fault/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/features/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/features/async_destroy/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/features/large_dnode/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/grow/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/history/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/hkdf/Makefile.am delete mode 100755 tests/zfs-tests/tests/functional/hkdf/run_hkdf_test.ksh delete mode 100755 tests/zfs-tests/tests/functional/hkdf/setup.ksh delete mode 100644 tests/zfs-tests/tests/functional/inheritance/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/inuse/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/io/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/l2arc/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/large_files/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/largest_pool/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/libzfs/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/limits/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/link_count/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/log_spacemap/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/migration/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/mmap/Makefile.am create mode 100755 tests/zfs-tests/tests/functional/mmap/mmap_sync_001_pos.ksh delete mode 100644 tests/zfs-tests/tests/functional/mmp/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/mount/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/mv_files/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/nestedfs/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/no_space/Makefile.am create mode 100755 tests/zfs-tests/tests/functional/no_space/enospc_rm.ksh delete mode 100644 tests/zfs-tests/tests/functional/nopwrite/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/online_offline/Makefile.am create mode 100644 tests/zfs-tests/tests/functional/pam/.gitignore delete mode 100644 tests/zfs-tests/tests/functional/pam/Makefile.am rename tests/zfs-tests/tests/functional/pam/{utilities.kshlib => utilities.kshlib.in} (67%) delete mode 100644 tests/zfs-tests/tests/functional/pool_checkpoint/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/pool_names/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/poolversion/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/privilege/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/procfs/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/projectquota/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/pyzfs/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/quota/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/raidz/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/redacted_send/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/redundancy/Makefile.am rename tests/zfs-tests/tests/functional/redundancy/{redundancy_draid_damaged.ksh => redundancy_draid_damaged1.ksh} (83%) create mode 100755 tests/zfs-tests/tests/functional/redundancy/redundancy_draid_damaged2.ksh delete mode 100644 tests/zfs-tests/tests/functional/refquota/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/refreserv/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/removal/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/rename_dirs/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/replacement/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/reservation/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/rootpool/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/rsend/Makefile.am create mode 100755 tests/zfs-tests/tests/functional/rsend/rsend_025_pos.ksh create mode 100755 tests/zfs-tests/tests/functional/rsend/rsend_026_neg.ksh create mode 100755 tests/zfs-tests/tests/functional/rsend/rsend_027_pos.ksh create mode 100755 tests/zfs-tests/tests/functional/rsend/rsend_028_neg.ksh create mode 100755 tests/zfs-tests/tests/functional/rsend/rsend_029_neg.ksh create mode 100755 tests/zfs-tests/tests/functional/rsend/send_raw_ashift.ksh delete mode 100644 tests/zfs-tests/tests/functional/scrub_mirror/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/simd/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/slog/Makefile.am create mode 100755 tests/zfs-tests/tests/functional/slog/slog_016_pos.ksh delete mode 100644 tests/zfs-tests/tests/functional/snapshot/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/snapused/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/sparse/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/stat/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/suid/.gitignore delete mode 100644 tests/zfs-tests/tests/functional/suid/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/threadsappend/.gitignore delete mode 100644 tests/zfs-tests/tests/functional/threadsappend/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/tmpfile/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/trim/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/truncate/.gitignore delete mode 100644 tests/zfs-tests/tests/functional/truncate/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/upgrade/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/user_namespace/Makefile.am create mode 100755 tests/zfs-tests/tests/functional/user_namespace/user_namespace_002.ksh create mode 100755 tests/zfs-tests/tests/functional/user_namespace/user_namespace_003.ksh create mode 100755 tests/zfs-tests/tests/functional/user_namespace/user_namespace_004.ksh delete mode 100644 tests/zfs-tests/tests/functional/userquota/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/vdev_zaps/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/write_dirs/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/xattr/Makefile.am create mode 100755 tests/zfs-tests/tests/functional/xattr/xattr_compat.ksh delete mode 100644 tests/zfs-tests/tests/functional/zpool_influxdb/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/zvol/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/zvol/zvol_ENOSPC/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/zvol/zvol_cli/Makefile.am delete mode 100644 tests/zfs-tests/tests/functional/zvol/zvol_misc/Makefile.am create mode 100755 tests/zfs-tests/tests/functional/zvol/zvol_misc/zvol_misc_fua.ksh create mode 100755 tests/zfs-tests/tests/functional/zvol/zvol_misc/zvol_misc_trim.ksh rename tests/{test-runner/include/stf.shlib => zfs-tests/tests/functional/zvol/zvol_stress/cleanup.ksh} (50%) mode change 100644 => 100755 create mode 100755 tests/zfs-tests/tests/functional/zvol/zvol_stress/setup.ksh create mode 100755 tests/zfs-tests/tests/functional/zvol/zvol_stress/zvol_stress.ksh delete mode 100644 tests/zfs-tests/tests/functional/zvol/zvol_swap/Makefile.am delete mode 100644 tests/zfs-tests/tests/perf/Makefile.am delete mode 100644 tests/zfs-tests/tests/perf/fio/Makefile.am delete mode 100644 tests/zfs-tests/tests/perf/regression/Makefile.am delete mode 100644 tests/zfs-tests/tests/perf/scripts/Makefile.am delete mode 100644 tests/zfs-tests/tests/stress/Makefile.am create mode 100644 udev/.gitignore delete mode 100644 udev/rules.d/Makefile.am rename {cmd/vdev_id => udev}/vdev_id (100%) create mode 100644 udev/zvol_id.c diff --git a/.github/workflows/build-dependencies.txt b/.github/workflows/build-dependencies.txt index 1dc745d5f642..e591399001f5 100644 --- a/.github/workflows/build-dependencies.txt +++ b/.github/workflows/build-dependencies.txt @@ -38,6 +38,7 @@ python3-dev python3-packaging python3-setuptools rng-tools +rsync samba sysstat uuid-dev diff --git a/.github/workflows/checkstyle.yaml b/.github/workflows/checkstyle.yaml index abc0ff11916a..8dafdcf07fed 100644 --- a/.github/workflows/checkstyle.yaml +++ b/.github/workflows/checkstyle.yaml @@ -40,11 +40,11 @@ jobs: - name: CheckABI id: CheckABI run: | - docker run -v $(pwd):/source ghcr.io/openzfs/libabigail make -j$(nproc) --no-print-directory --silent checkabi + docker run -v $PWD:/source ghcr.io/openzfs/libabigail make -j$(nproc) --no-print-directory --silent checkabi - name: StoreABI if: failure() && steps.CheckABI.outcome == 'failure' run: | - docker run -v $(pwd):/source ghcr.io/openzfs/libabigail make -j$(nproc) --no-print-directory --silent storeabi + docker run -v $PWD:/source ghcr.io/openzfs/libabigail make -j$(nproc) --no-print-directory --silent storeabi - name: Prepare artifacts if: failure() && steps.CheckABI.outcome == 'failure' run: | diff --git a/.github/workflows/zfs-tests-functional.yml b/.github/workflows/zfs-tests-functional.yml index 40d13788a96b..328cb97f10e4 100644 --- a/.github/workflows/zfs-tests-functional.yml +++ b/.github/workflows/zfs-tests-functional.yml @@ -28,7 +28,7 @@ jobs: ./autogen.sh - name: Configure run: | - ./configure --enable-debug --enable-debuginfo --enable-asan --enable-ubsan + ./configure --enable-debug --enable-debuginfo --enable-asan --enable-ubsan --with-config=dist - name: Make run: | make -j$(nproc) --no-print-directory --silent pkg-utils pkg-kmod @@ -59,7 +59,9 @@ jobs: df -h / - name: Tests run: | - /usr/share/zfs/zfs-tests.sh -vR -s 3G + set -o pipefail + /usr/share/zfs/zfs-tests.sh -vKR -s 3G | scripts/zfs-tests-color.sh + shell: bash timeout-minutes: 330 - name: Prepare artifacts if: failure() diff --git a/.github/workflows/zfs-tests-sanity.yml b/.github/workflows/zfs-tests-sanity.yml index ea538ce714f0..4c15cecf58d8 100644 --- a/.github/workflows/zfs-tests-sanity.yml +++ b/.github/workflows/zfs-tests-sanity.yml @@ -24,7 +24,7 @@ jobs: ./autogen.sh - name: Configure run: | - ./configure --enable-debug --enable-debuginfo --enable-asan --enable-ubsan + ./configure --enable-debug --enable-debuginfo --enable-asan --enable-ubsan --with-config=dist - name: Make run: | make -j$(nproc) --no-print-directory --silent pkg-utils pkg-kmod @@ -55,7 +55,9 @@ jobs: df -h / - name: Tests run: | - /usr/share/zfs/zfs-tests.sh -vR -s 3G -r sanity + set -o pipefail + /usr/share/zfs/zfs-tests.sh -vKR -s 3G -r sanity | scripts/zfs-tests-color.sh + shell: bash timeout-minutes: 330 - name: Prepare artifacts if: failure() diff --git a/.github/workflows/zloop.yml b/.github/workflows/zloop.yml index 8eb2a1d9bb0f..64fe96a3ab6b 100644 --- a/.github/workflows/zloop.yml +++ b/.github/workflows/zloop.yml @@ -23,7 +23,7 @@ jobs: ./autogen.sh - name: Configure run: | - ./configure --enable-debug --enable-debuginfo --enable-asan --enable-ubsan + ./configure --enable-debug --enable-debuginfo --enable-asan --enable-ubsan --with-config=dist - name: Make run: | make -j$(nproc) --no-print-directory --silent pkg-utils pkg-kmod @@ -38,8 +38,9 @@ jobs: - name: Tests run: | sudo mkdir -p $TEST_DIR - # run for 20 minutes to have a total runner time of 30 minutes - sudo /usr/share/zfs/zloop.sh -t 1200 -l -m1 -- -T 120 -P 60 + # run for 10 minutes or at most 2 iterations for a maximum runner + # time of 20 minutes. + sudo /usr/share/zfs/zloop.sh -t 600 -I 2 -l -m1 -- -T 120 -P 60 - name: Prepare artifacts if: failure() run: | diff --git a/.gitignore b/.gitignore index 056bbb8f08c9..8d91dd9466c5 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,7 @@ # -# N.B. -# This is the toplevel .gitignore file. +# This is the top-level .gitignore file: +# ignore everything except a list of allowed files. +# # This is not the place for entries that are specific to # a subdirectory. Instead add those files to the # .gitignore file in that subdirectory. @@ -10,6 +11,56 @@ # command after changing this file, to see if there are # any tracked files which get ignored after the change. +* + +!.github +!cmd +!config +!contrib +!etc +!include +!lib +!man +!module +!rpm +!scripts +!tests +!udev + +!.github/** +!cmd/** +!config/** +!contrib/** +!etc/** +!include/** +!lib/** +!man/** +!module/** +!rpm/** +!scripts/** +!tests/** +!udev/** + +!.editorconfig +!.gitignore +!.gitmodules +!AUTHORS +!autogen.sh +!CODE_OF_CONDUCT.md +!configure.ac +!copy-builtin +!COPYRIGHT +!LICENSE +!Makefile.am +!META +!NEWS +!NOTICE +!README.md +!RELEASES.md +!TEST +!zfs.release.in + + # # Normal rules # @@ -31,40 +82,7 @@ modules.order Makefile Makefile.in - -# -# Top level generated files specific to this top level dir -# -/bin -/build -/configure -/config.log -/config.status -/libtool -/zfs_config.h -/zfs_config.h.in -/zfs.release -/stamp-h1 -/aclocal.m4 -/autom4te.cache - -# -# Top level generic files -# -!.gitignore -tags -TAGS -current -cscope.* -*.rpm -*.deb -*.tar.gz *.patch *.orig -*.log *.tmp -venv - -*.so -*.so.debug -*.so.full +*.log diff --git a/AUTHORS b/AUTHORS index aab8bf29c99f..86083ba87715 100644 --- a/AUTHORS +++ b/AUTHORS @@ -285,6 +285,7 @@ CONTRIBUTORS: Tim Connors Tim Crawford Tim Haley + Tino Reichardt Tobin Harding Tom Caputi Tom Matthews diff --git a/META b/META index aab6e2abe567..f16f7972575f 100644 --- a/META +++ b/META @@ -6,5 +6,5 @@ Release: 1 Release-Tags: relext License: CDDL Author: OpenZFS -Linux-Maximum: 5.15 +Linux-Maximum: 5.18 Linux-Minimum: 3.10 diff --git a/Makefile.am b/Makefile.am index c8d1d08aa2c4..54d300e7d40b 100644 --- a/Makefile.am +++ b/Makefile.am @@ -1,68 +1,82 @@ +CLEANFILES = +dist_noinst_DATA = +INSTALL_DATA_HOOKS = +ALL_LOCAL = +CLEAN_LOCAL = +CHECKS = shellcheck checkbashisms + +include $(top_srcdir)/config/Rules.am +include $(top_srcdir)/config/CppCheck.am include $(top_srcdir)/config/Shellcheck.am +include $(top_srcdir)/config/Substfiles.am ACLOCAL_AMFLAGS = -I config SUBDIRS = include if BUILD_LINUX -SUBDIRS += rpm +include $(srcdir)/%D%/rpm/Makefile.am endif if CONFIG_USER -SUBDIRS += man scripts lib tests cmd etc contrib +include $(srcdir)/%D%/cmd/Makefile.am +include $(srcdir)/%D%/contrib/Makefile.am +include $(srcdir)/%D%/etc/Makefile.am +include $(srcdir)/%D%/lib/Makefile.am +include $(srcdir)/%D%/man/Makefile.am +include $(srcdir)/%D%/scripts/Makefile.am +include $(srcdir)/%D%/tests/Makefile.am if BUILD_LINUX -SUBDIRS += udev +include $(srcdir)/%D%/udev/Makefile.am endif endif +CPPCHECKDIRS += module if CONFIG_KERNEL SUBDIRS += module extradir = $(prefix)/src/zfs-$(VERSION) extra_HEADERS = zfs.release.in zfs_config.h.in - -if BUILD_LINUX -kerneldir = $(prefix)/src/zfs-$(VERSION)/$(LINUX_VERSION) -nodist_kernel_HEADERS = zfs.release zfs_config.h module/$(LINUX_SYMBOLS) -endif endif -AUTOMAKE_OPTIONS = foreign -EXTRA_DIST = autogen.sh copy-builtin -EXTRA_DIST += config/config.awk config/rpm.am config/deb.am config/tgz.am -EXTRA_DIST += AUTHORS CODE_OF_CONDUCT.md COPYRIGHT LICENSE META NEWS NOTICE -EXTRA_DIST += README.md RELEASES.md -EXTRA_DIST += module/lua/README.zfs module/os/linux/spl/README.md +dist_noinst_DATA += autogen.sh copy-builtin +dist_noinst_DATA += AUTHORS CODE_OF_CONDUCT.md COPYRIGHT LICENSE META NEWS NOTICE +dist_noinst_DATA += README.md RELEASES.md +dist_noinst_DATA += module/lua/README.zfs module/os/linux/spl/README.md # Include all the extra licensing information for modules -EXTRA_DIST += module/icp/algs/skein/THIRDPARTYLICENSE -EXTRA_DIST += module/icp/algs/skein/THIRDPARTYLICENSE.descrip -EXTRA_DIST += module/icp/asm-x86_64/aes/THIRDPARTYLICENSE.gladman -EXTRA_DIST += module/icp/asm-x86_64/aes/THIRDPARTYLICENSE.gladman.descrip -EXTRA_DIST += module/icp/asm-x86_64/aes/THIRDPARTYLICENSE.openssl -EXTRA_DIST += module/icp/asm-x86_64/aes/THIRDPARTYLICENSE.openssl.descrip -EXTRA_DIST += module/icp/asm-x86_64/modes/THIRDPARTYLICENSE.cryptogams -EXTRA_DIST += module/icp/asm-x86_64/modes/THIRDPARTYLICENSE.cryptogams.descrip -EXTRA_DIST += module/icp/asm-x86_64/modes/THIRDPARTYLICENSE.openssl -EXTRA_DIST += module/icp/asm-x86_64/modes/THIRDPARTYLICENSE.openssl.descrip -EXTRA_DIST += module/os/linux/spl/THIRDPARTYLICENSE.gplv2 -EXTRA_DIST += module/os/linux/spl/THIRDPARTYLICENSE.gplv2.descrip -EXTRA_DIST += module/zfs/THIRDPARTYLICENSE.cityhash -EXTRA_DIST += module/zfs/THIRDPARTYLICENSE.cityhash.descrip +dist_noinst_DATA += module/icp/algs/skein/THIRDPARTYLICENSE +dist_noinst_DATA += module/icp/algs/skein/THIRDPARTYLICENSE.descrip +dist_noinst_DATA += module/icp/asm-x86_64/aes/THIRDPARTYLICENSE.gladman +dist_noinst_DATA += module/icp/asm-x86_64/aes/THIRDPARTYLICENSE.gladman.descrip +dist_noinst_DATA += module/icp/asm-x86_64/aes/THIRDPARTYLICENSE.openssl +dist_noinst_DATA += module/icp/asm-x86_64/aes/THIRDPARTYLICENSE.openssl.descrip +dist_noinst_DATA += module/icp/asm-x86_64/modes/THIRDPARTYLICENSE.cryptogams +dist_noinst_DATA += module/icp/asm-x86_64/modes/THIRDPARTYLICENSE.cryptogams.descrip +dist_noinst_DATA += module/icp/asm-x86_64/modes/THIRDPARTYLICENSE.openssl +dist_noinst_DATA += module/icp/asm-x86_64/modes/THIRDPARTYLICENSE.openssl.descrip +dist_noinst_DATA += module/os/linux/spl/THIRDPARTYLICENSE.gplv2 +dist_noinst_DATA += module/os/linux/spl/THIRDPARTYLICENSE.gplv2.descrip +dist_noinst_DATA += module/zfs/THIRDPARTYLICENSE.cityhash +dist_noinst_DATA += module/zfs/THIRDPARTYLICENSE.cityhash.descrip @CODE_COVERAGE_RULES@ GITREV = include/zfs_gitrev.h - -PHONY = gitrev +CLEANFILES += $(GITREV) +PHONY += gitrev gitrev: $(AM_V_GEN)$(top_srcdir)/scripts/make_gitrev.sh $(GITREV) all: gitrev -# Double-colon rules are allowed; there are multiple independent definitions. -maintainer-clean-local:: +PHONY += install-data-hook $(INSTALL_DATA_HOOKS) +install-data-hook: $(INSTALL_DATA_HOOKS) + +PHONY += maintainer-clean-local +maintainer-clean-local: -$(RM) $(GITREV) -distclean-local:: +PHONY += distclean-local +distclean-local: -$(RM) -R autom4te*.cache build -find . \( -name SCCS -o -name BitKeeper -o -name .svn -o -name CVS \ -o -name .pc -o -name .hg -o -name .git \) -prune -o \ @@ -74,43 +88,27 @@ distclean-local:: -o -name '*.gcno' \) \ -type f -delete -all-local: - -[ -x ${top_builddir}/scripts/zfs-tests.sh ] && \ - ${top_builddir}/scripts/zfs-tests.sh -c +PHONY += $(CLEAN_LOCAL) +clean-local: $(CLEAN_LOCAL) + +PHONY += $(ALL_LOCAL) +all-local: $(ALL_LOCAL) dist-hook: - $(AM_V_GEN)$(top_srcdir)/scripts/make_gitrev.sh -D $(distdir) $(GITREV) - $(SED) ${ac_inplace} -e 's/Release:[[:print:]]*/Release: $(RELEASE)/' \ - $(distdir)/META + $(top_srcdir)/scripts/make_gitrev.sh -D $(distdir) $(GITREV) + $(SED) $(ac_inplace) 's/\(Release:[[:space:]]*\).*/\1$(RELEASE)/' $(distdir)/META -if BUILD_LINUX -# For compatibility, create a matching spl-x.y.z directly which contains -# symlinks to the updated header and object file locations. These -# compatibility links will be removed in the next major release. -if CONFIG_KERNEL -install-data-hook: - rm -rf $(DESTDIR)$(prefix)/src/spl-$(VERSION) && \ - mkdir $(DESTDIR)$(prefix)/src/spl-$(VERSION) && \ - cd $(DESTDIR)$(prefix)/src/spl-$(VERSION) && \ - ln -s ../zfs-$(VERSION)/include/spl include && \ - ln -s ../zfs-$(VERSION)/$(LINUX_VERSION) $(LINUX_VERSION) && \ - ln -s ../zfs-$(VERSION)/zfs_config.h.in spl_config.h.in && \ - ln -s ../zfs-$(VERSION)/zfs.release.in spl.release.in && \ - cd $(DESTDIR)$(prefix)/src/zfs-$(VERSION)/$(LINUX_VERSION) && \ - ln -fs zfs_config.h spl_config.h && \ - ln -fs zfs.release spl.release -endif -endif +PHONY += codecheck $(CHECKS) +codecheck: $(CHECKS) -PHONY += codecheck -codecheck: cstyle shellcheck checkbashisms flake8 mancheck testscheck vcscheck +SHELLCHECKSCRIPTS += autogen.sh PHONY += checkstyle checkstyle: codecheck commitcheck PHONY += commitcheck commitcheck: - @if git rev-parse --git-dir > /dev/null 2>&1; then \ + $(AM_V_at)if git rev-parse --git-dir > /dev/null 2>&1; then \ ${top_srcdir}/scripts/commitcheck.sh; \ fi @@ -119,12 +117,13 @@ cstyle_line = -print0 | parallel -X0 ${top_srcdir}/scripts/cstyle.pl -cpP {} else cstyle_line = -exec ${top_srcdir}/scripts/cstyle.pl -cpP {} + endif -PHONY += cstyle +CHECKS += cstyle cstyle: - @find ${top_srcdir} -name build -prune \ + $(AM_V_at)find $(top_srcdir) -name build -prune \ -o -type f -name '*.[hc]' \ ! -name 'zfs_config.*' ! -name '*.mod.c' \ ! -name 'opt_global.h' ! -name '*_if*.h' \ + ! -name 'zstd_compat_wrapper.h' \ ! -path './module/zstd/lib/*' \ ! -path './include/sys/lua/*' \ ! -path './module/lua/l*.[ch]' \ @@ -132,78 +131,49 @@ cstyle: $(cstyle_line) filter_executable = -exec test -x '{}' \; -print - -SHELLCHECKDIRS = cmd contrib etc scripts tests -SHELLCHECKSCRIPTS = autogen.sh - -PHONY += checkabi storeabi - -checklibabiversion: - libabiversion=`abidw -v | $(SED) 's/[^0-9]//g'`; \ - if test $$libabiversion -lt "200"; then \ - /bin/echo -e "\n" \ - "*** Please use libabigail 2.0.0 version or newer;\n" \ - "*** otherwise results are not consistent!\n" \ - "(or see https://github.com/openzfs/libabigail-docker )\n"; \ - exit 1; \ - fi; - -checkabi: checklibabiversion lib - $(MAKE) -C lib checkabi - -storeabi: checklibabiversion lib - $(MAKE) -C lib storeabi - -PHONY += mancheck -mancheck: - ${top_srcdir}/scripts/mancheck.sh ${top_srcdir}/man ${top_srcdir}/tests/test-runner/man - -PHONY += testscheck +CHECKS += testscheck testscheck: - @[ $$(find ${top_srcdir}/tests/zfs-tests -type f \ - \( -name '*.ksh' -not ${filter_executable} \) -o \ - \( -name '*.kshlib' ${filter_executable} \) -o \ - \( -name '*.shlib' ${filter_executable} \) -o \ - \( -name '*.cfg' ${filter_executable} \) | \ + $(AM_V_at)[ $$(find $(top_srcdir)/tests/zfs-tests -type f \ + \( -name '*.ksh' -not $(filter_executable) \) -o \ + \( -name '*.kshlib' $(filter_executable) \) -o \ + \( -name '*.shlib' $(filter_executable) \) -o \ + \( -name '*.cfg' $(filter_executable) \) | \ tee /dev/stderr | wc -l) -eq 0 ] -PHONY += vcscheck +CHECKS += vcscheck vcscheck: - @if git rev-parse --git-dir > /dev/null 2>&1; then \ + $(AM_V_at)if git rev-parse --git-dir > /dev/null 2>&1; then \ git ls-files . --exclude-standard --others | \ awk '{c++; print} END {if(c>0) exit 1}' ; \ fi +CHECKS += zstdcheck +zstdcheck: + @$(MAKE) -C module check-zstd-symbols + PHONY += lint lint: cppcheck paxcheck -CPPCHECKDIRS = cmd lib module -PHONY += cppcheck -cppcheck: $(CPPCHECKDIRS) - @if test -n "$(CPPCHECK)"; then \ - set -e ; for dir in $(CPPCHECKDIRS) ; do \ - $(MAKE) -C $$dir cppcheck ; \ - done \ - else \ - echo "skipping cppcheck because cppcheck is not installed"; \ - fi - PHONY += paxcheck paxcheck: - @if type scanelf > /dev/null 2>&1; then \ - ${top_srcdir}/scripts/paxcheck.sh ${top_builddir}; \ + $(AM_V_at)if type scanelf > /dev/null 2>&1; then \ + $(top_srcdir)/scripts/paxcheck.sh $(top_builddir); \ else \ echo "skipping paxcheck because scanelf is not installed"; \ fi -PHONY += flake8 +CHECKS += flake8 flake8: - @if type flake8 > /dev/null 2>&1; then \ - flake8 ${top_srcdir}; \ + $(AM_V_at)if type flake8 > /dev/null 2>&1; then \ + flake8 $(top_srcdir); \ else \ echo "skipping flake8 because flake8 is not installed"; \ fi +PHONY += regen-tests +regen-tests: + @$(MAKE) -C tests/zfs-tests/tests regen + PHONY += ctags ctags: $(RM) tags diff --git a/autogen.sh b/autogen.sh index 488e913b2bf4..c817090183f1 100755 --- a/autogen.sh +++ b/autogen.sh @@ -1,4 +1,62 @@ #!/bin/sh +[ "${0%/*}" = "$0" ] || cd "${0%/*}" || exit -autoreconf -fiv || exit 1 -rm -Rf autom4te.cache +# %reldir%/%canon_reldir% (%D%/%C%) only appeared in automake 1.14, but RHEL/CentOS 7 has 1.13.4 +# This is an (overly) simplistic preprocessor that papers around this for the duration of the generation step, +# and can be removed once support for CentOS 7 is dropped +automake --version | awk '{print $NF; exit}' | ( + IFS=. read -r AM_MAJ AM_MIN _ + [ "$AM_MAJ" -gt 1 ] || [ "$AM_MIN" -ge 14 ] +) || { + process_root() { + root="$1"; shift + + grep -q '%[CD]%' "$root/Makefile.am" || return + find "$root" -name Makefile.am "$@" | while read -r dir; do + dir="${dir%/Makefile.am}" + grep -q '%[CD]%' "$dir/Makefile.am" || continue + + reldir="${dir#$root}" + reldir="${reldir#/}" + + canon_reldir="$(printf '%s' "$reldir" | tr -C 'a-zA-Z0-9@_' '_')" + + reldir_slash="$reldir/" + canon_reldir_slash="${canon_reldir}_" + [ -z "$reldir" ] && reldir_slash= + [ -z "$reldir" ] && canon_reldir_slash= + + echo "$dir/Makefile.am" >&3 + sed -i~ -e "s:%D%/:$reldir_slash:g" -e "s:%D%:$reldir:g" \ + -e "s:%C%_:$canon_reldir_slash:g" -e "s:%C%:$canon_reldir:g" "$dir/Makefile.am" + done 3>>"$substituted_files" + } + + rollback() { + while read -r f; do + mv "$f~" "$f" + done < "$substituted_files" + rm -f "$substituted_files" + } + + + echo "Automake <1.14; papering over missing %reldir%/%canon_reldir% support" >&2 + + substituted_files="$(mktemp)" + trap rollback EXIT + + roots="$(sed '/Makefile$/!d;/module/d;s:^\s*:./:;s:/Makefile::;/^\.$/d' configure.ac)" + + IFS=" +" + for root in $roots; do + root="${root#./}" + process_root "$root" + done + + set -f + # shellcheck disable=SC2086,SC2046 + process_root . $(printf '!\n-path\n%s/*\n' $roots) +} + +autoreconf -fiv && rm -rf autom4te.cache diff --git a/cmd/Makefile.am b/cmd/Makefile.am index 68f1e892d3f4..65de980da308 100644 --- a/cmd/Makefile.am +++ b/cmd/Makefile.am @@ -1,27 +1,115 @@ -include $(top_srcdir)/config/Shellcheck.am +bin_SCRIPTS = +bin_PROGRAMS = +sbin_SCRIPTS = +sbin_PROGRAMS = +dist_bin_SCRIPTS = +zfsexec_PROGRAMS = +mounthelper_PROGRAMS = -SUBDIRS = zfs zpool zdb zhack zinject zstream ztest -SUBDIRS += fsck_zfs vdev_id raidz_test zfs_ids_to_path -SUBDIRS += zpool_influxdb -CPPCHECKDIRS = zfs zpool zdb zhack zinject zstream ztest -CPPCHECKDIRS += raidz_test zfs_ids_to_path zpool_influxdb +sbin_SCRIPTS += fsck.zfs +SHELLCHECKSCRIPTS += fsck.zfs +CLEANFILES += fsck.zfs +dist_noinst_DATA += %D%/fsck.zfs.in +$(call SUBST,fsck.zfs,%D%/) -# TODO: #12084: SHELLCHECKDIRS += vdev_id -SHELLCHECKDIRS = fsck_zfs zed zpool zvol_wait -if USING_PYTHON -SUBDIRS += arcstat arc_summary dbufstat -endif +sbin_PROGRAMS += zfs_ids_to_path +CPPCHECKTARGETS += zfs_ids_to_path + +zfs_ids_to_path_SOURCES = \ + %D%/zfs_ids_to_path.c + +zfs_ids_to_path_LDADD = \ + libzfs.la + + +zhack_CPPFLAGS = $(AM_CPPFLAGS) $(FORCEDEBUG_CPPFLAGS) + +sbin_PROGRAMS += zhack +CPPCHECKTARGETS += zhack + +zhack_SOURCES = \ + %D%/zhack.c + +zhack_LDADD = \ + libzpool.la \ + libzfs_core.la \ + libnvpair.la + + +ztest_CFLAGS = $(AM_CFLAGS) $(KERNEL_CFLAGS) +# Get rid of compiler warning for unchecked truncating snprintfs on gcc 7.1.1 +ztest_CFLAGS += $(NO_FORMAT_TRUNCATION) +ztest_CPPFLAGS = $(AM_CPPFLAGS) $(FORCEDEBUG_CPPFLAGS) + +sbin_PROGRAMS += ztest +CPPCHECKTARGETS += ztest + +ztest_SOURCES = \ + %D%/ztest.c + +ztest_LDADD = \ + libzpool.la \ + libzfs_core.la \ + libnvpair.la + +ztest_LDADD += -lm +ztest_LDFLAGS = -pthread + + +include $(srcdir)/%D%/raidz_test/Makefile.am +include $(srcdir)/%D%/zdb/Makefile.am +include $(srcdir)/%D%/zfs/Makefile.am +include $(srcdir)/%D%/zinject/Makefile.am +include $(srcdir)/%D%/zpool/Makefile.am +include $(srcdir)/%D%/zpool_influxdb/Makefile.am +include $(srcdir)/%D%/zstream/Makefile.am + if BUILD_LINUX -SUBDIRS += mount_zfs zed zgenhostid zvol_id zvol_wait -CPPCHECKDIRS += mount_zfs zed zgenhostid zvol_id -SHELLCHECKDIRS += zed +mounthelper_PROGRAMS += mount.zfs +CPPCHECKTARGETS += mount.zfs + +mount_zfs_SOURCES = \ + %D%/mount_zfs.c + +mount_zfs_LDADD = \ + libzfs.la \ + libzfs_core.la \ + libnvpair.la + +mount_zfs_LDADD += $(LTLIBINTL) + +CPPCHECKTARGETS += raidz_test + + +sbin_PROGRAMS += zgenhostid +CPPCHECKTARGETS += zgenhostid + +zgenhostid_SOURCES = \ + %D%/zgenhostid.c + + +dist_bin_SCRIPTS += %D%/zvol_wait +SHELLCHECKSCRIPTS += %D%/zvol_wait + + +include $(srcdir)/%D%/zed/Makefile.am +endif + + +if USING_PYTHON +bin_SCRIPTS += arc_summary arcstat dbufstat +CLEANFILES += arc_summary arcstat dbufstat +dist_noinst_DATA += %D%/arc_summary %D%/arcstat.in %D%/dbufstat.in + +$(call SUBST,arcstat,%D%/) +$(call SUBST,dbufstat,%D%/) +arc_summary: %D%/arc_summary + $(AM_V_at)cp $< $@ endif -PHONY = cppcheck -cppcheck: $(CPPCHECKDIRS) - set -e ; for dir in $(CPPCHECKDIRS) ; do \ - $(MAKE) -C $$dir cppcheck ; \ - done + +PHONY += cmd +cmd: $(bin_SCRIPTS) $(bin_PROGRAMS) $(sbin_SCRIPTS) $(sbin_PROGRAMS) $(dist_bin_SCRIPTS) $(zfsexec_PROGRAMS) $(mounthelper_PROGRAMS) diff --git a/cmd/arc_summary/arc_summary3 b/cmd/arc_summary similarity index 100% rename from cmd/arc_summary/arc_summary3 rename to cmd/arc_summary diff --git a/cmd/arc_summary/.gitignore b/cmd/arc_summary/.gitignore deleted file mode 100644 index 50ba15f034e2..000000000000 --- a/cmd/arc_summary/.gitignore +++ /dev/null @@ -1 +0,0 @@ -arc_summary diff --git a/cmd/arc_summary/Makefile.am b/cmd/arc_summary/Makefile.am deleted file mode 100644 index f419f07e0eda..000000000000 --- a/cmd/arc_summary/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -bin_SCRIPTS = arc_summary - -CLEANFILES = arc_summary -EXTRA_DIST = arc_summary3 -SCRIPT = arc_summary3 - -arc_summary: $(SCRIPT) - cp $< $@ diff --git a/cmd/arcstat/arcstat.in b/cmd/arcstat.in similarity index 99% rename from cmd/arcstat/arcstat.in rename to cmd/arcstat.in index 9327f644f544..b1b6655f7f26 100755 --- a/cmd/arcstat/arcstat.in +++ b/cmd/arcstat.in @@ -29,7 +29,7 @@ # with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/cmd/arcstat/.gitignore b/cmd/arcstat/.gitignore deleted file mode 100644 index 6d6cd1ab75fc..000000000000 --- a/cmd/arcstat/.gitignore +++ /dev/null @@ -1 +0,0 @@ -arcstat diff --git a/cmd/arcstat/Makefile.am b/cmd/arcstat/Makefile.am deleted file mode 100644 index d1ba989a0cd8..000000000000 --- a/cmd/arcstat/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -include $(top_srcdir)/config/Substfiles.am - -bin_SCRIPTS = arcstat - -SUBSTFILES += $(bin_SCRIPTS) diff --git a/cmd/dbufstat/dbufstat.in b/cmd/dbufstat.in similarity index 99% rename from cmd/dbufstat/dbufstat.in rename to cmd/dbufstat.in index b716a0c9749b..08c22864e5d8 100755 --- a/cmd/dbufstat/dbufstat.in +++ b/cmd/dbufstat.in @@ -12,7 +12,7 @@ # with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/cmd/dbufstat/.gitignore b/cmd/dbufstat/.gitignore deleted file mode 100644 index 2c2e913cef70..000000000000 --- a/cmd/dbufstat/.gitignore +++ /dev/null @@ -1 +0,0 @@ -dbufstat diff --git a/cmd/dbufstat/Makefile.am b/cmd/dbufstat/Makefile.am deleted file mode 100644 index e672a01a4227..000000000000 --- a/cmd/dbufstat/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -include $(top_srcdir)/config/Substfiles.am - -bin_SCRIPTS = dbufstat - -SUBSTFILES += $(bin_SCRIPTS) diff --git a/cmd/fsck_zfs/fsck.zfs.in b/cmd/fsck.zfs.in similarity index 93% rename from cmd/fsck_zfs/fsck.zfs.in rename to cmd/fsck.zfs.in index 37096902cb94..f0d4d2ec38ca 100755 --- a/cmd/fsck_zfs/fsck.zfs.in +++ b/cmd/fsck.zfs.in @@ -7,13 +7,13 @@ # see fsck.zfs(8) # -if [ "$#" = "0" ]; then +if [ $# -eq 0 ]; then echo "Usage: $0 [options] dataset…" >&2 exit 16 fi ret=0 -for dataset in "$@"; do +for dataset; do case "$dataset" in -*) continue diff --git a/cmd/fsck_zfs/.gitignore b/cmd/fsck_zfs/.gitignore deleted file mode 100644 index 0edf0309e94a..000000000000 --- a/cmd/fsck_zfs/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/fsck.zfs diff --git a/cmd/fsck_zfs/Makefile.am b/cmd/fsck_zfs/Makefile.am deleted file mode 100644 index ec955c7c7ff9..000000000000 --- a/cmd/fsck_zfs/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -include $(top_srcdir)/config/Substfiles.am -include $(top_srcdir)/config/Shellcheck.am - -dist_sbin_SCRIPTS = fsck.zfs - -SUBSTFILES += $(dist_sbin_SCRIPTS) - diff --git a/cmd/mount_zfs/mount_zfs.c b/cmd/mount_zfs.c similarity index 97% rename from cmd/mount_zfs/mount_zfs.c rename to cmd/mount_zfs.c index 669ed88f91c4..bc2d366bf25d 100644 --- a/cmd/mount_zfs/mount_zfs.c +++ b/cmd/mount_zfs.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -108,20 +108,21 @@ mtab_is_writeable(void) } static int -mtab_update(char *dataset, char *mntpoint, char *type, char *mntopts) +mtab_update(const char *dataset, const char *mntpoint, const char *type, + const char *mntopts) { struct mntent mnt; FILE *fp; int error; - mnt.mnt_fsname = dataset; - mnt.mnt_dir = mntpoint; - mnt.mnt_type = type; - mnt.mnt_opts = mntopts ? mntopts : ""; + mnt.mnt_fsname = (char *)dataset; + mnt.mnt_dir = (char *)mntpoint; + mnt.mnt_type = (char *)type; + mnt.mnt_opts = (char *)(mntopts ?: ""); mnt.mnt_freq = 0; mnt.mnt_passno = 0; - fp = setmntent("/etc/mtab", "a+"); + fp = setmntent("/etc/mtab", "a+e"); if (!fp) { (void) fprintf(stderr, gettext( "filesystem '%s' was mounted, but /etc/mtab " diff --git a/cmd/mount_zfs/.gitignore b/cmd/mount_zfs/.gitignore deleted file mode 100644 index cd9254bde3da..000000000000 --- a/cmd/mount_zfs/.gitignore +++ /dev/null @@ -1 +0,0 @@ -mount.zfs diff --git a/cmd/mount_zfs/Makefile.am b/cmd/mount_zfs/Makefile.am deleted file mode 100644 index 3957602d27ad..000000000000 --- a/cmd/mount_zfs/Makefile.am +++ /dev/null @@ -1,22 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -# -# Ignore the prefix for the mount helper. It must be installed in /sbin/ -# because this path is hardcoded in the mount(8) for security reasons. -# However, if needed, the configure option --with-mounthelperdir= can be used -# to override the default install location. -# -sbindir=$(mounthelperdir) -sbin_PROGRAMS = mount.zfs - -mount_zfs_SOURCES = \ - mount_zfs.c - -mount_zfs_LDADD = \ - $(abs_top_builddir)/lib/libzfs/libzfs.la \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la - -mount_zfs_LDADD += $(LTLIBINTL) - -include $(top_srcdir)/config/CppCheck.am diff --git a/cmd/raidz_test/.gitignore b/cmd/raidz_test/.gitignore deleted file mode 100644 index f8b83d9cce03..000000000000 --- a/cmd/raidz_test/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/raidz_test diff --git a/cmd/raidz_test/Makefile.am b/cmd/raidz_test/Makefile.am index 983ff25dc92a..3b8b60568323 100644 --- a/cmd/raidz_test/Makefile.am +++ b/cmd/raidz_test/Makefile.am @@ -1,22 +1,16 @@ -include $(top_srcdir)/config/Rules.am +raidz_test_CFLAGS = $(AM_CFLAGS) $(KERNEL_CFLAGS) +raidz_test_CPPFLAGS = $(AM_CPPFLAGS) $(FORCEDEBUG_CPPFLAGS) -# Includes kernel code, generate warnings for large stack frames -AM_CFLAGS += $(FRAME_LARGER_THAN) - -# Unconditionally enable ASSERTs -AM_CPPFLAGS += -DDEBUG -UNDEBUG -DZFS_DEBUG - -bin_PROGRAMS = raidz_test +bin_PROGRAMS += raidz_test +CPPCHECKTARGETS += raidz_test raidz_test_SOURCES = \ - raidz_test.h \ - raidz_test.c \ - raidz_bench.c + %D%/raidz_bench.c \ + %D%/raidz_test.c \ + %D%/raidz_test.h raidz_test_LDADD = \ - $(abs_top_builddir)/lib/libzpool/libzpool.la \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la + libzpool.la \ + libzfs_core.la raidz_test_LDADD += -lm - -include $(top_srcdir)/config/CppCheck.am diff --git a/cmd/raidz_test/raidz_bench.c b/cmd/raidz_test/raidz_bench.c index f44d6fbde707..8be08558b36d 100644 --- a/cmd/raidz_test/raidz_bench.c +++ b/cmd/raidz_test/raidz_bench.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -63,7 +63,7 @@ bench_fini_raidz_maps(void) { /* tear down golden zio */ raidz_free(zio_bench.io_abd, max_data_size); - bzero(&zio_bench, sizeof (zio_t)); + memset(&zio_bench, 0, sizeof (zio_t)); } static inline void diff --git a/cmd/raidz_test/raidz_test.c b/cmd/raidz_test/raidz_test.c index b177105ee63b..1ece55960d33 100644 --- a/cmd/raidz_test/raidz_test.c +++ b/cmd/raidz_test/raidz_test.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -69,14 +69,15 @@ static void sig_handler(int signo) static void print_opts(raidz_test_opts_t *opts, boolean_t force) { - char *verbose; + const char *verbose; switch (opts->rto_v) { - case 0: + case D_ALL: verbose = "no"; break; - case 1: + case D_INFO: verbose = "info"; break; + case D_DEBUG: default: verbose = "debug"; break; @@ -119,7 +120,7 @@ static void usage(boolean_t requested) "\t[-B benchmark all raidz implementations]\n" "\t[-e use expanded raidz map (default: %s)]\n" "\t[-r expanded raidz map reflow offset (default: %llx)]\n" - "\t[-v increase verbosity (default: %zu)]\n" + "\t[-v increase verbosity (default: %d)]\n" "\t[-h (print help)]\n" "\t[-T test the test, see if failure would be detected]\n" "\t[-D debug (attach gdb on SIGSEGV)]\n" @@ -131,7 +132,7 @@ static void usage(boolean_t requested) rto_opts.rto_sweep ? "yes" : "no", /* -S */ rto_opts.rto_expand ? "yes" : "no", /* -e */ (u_longlong_t)o->rto_expand_offset, /* -r */ - o->rto_v); /* -d */ + o->rto_v); /* -v */ exit(requested ? 0 : 1); } @@ -140,10 +141,9 @@ static void process_options(int argc, char **argv) { size_t value; int opt; - raidz_test_opts_t *o = &rto_opts; - bcopy(&rto_opts_defaults, o, sizeof (*o)); + memcpy(o, &rto_opts_defaults, sizeof (*o)); while ((opt = getopt(argc, argv, "TDBSvha:er:o:d:s:t:")) != -1) { value = 0; @@ -839,7 +839,7 @@ static kcondvar_t sem_cv; static int max_free_slots; static int free_slots; -static void +static __attribute__((noreturn)) void sweep_thread(void *arg) { int err = 0; diff --git a/cmd/raidz_test/raidz_test.h b/cmd/raidz_test/raidz_test.h index 0f7f4cee3eb6..163929defc73 100644 --- a/cmd/raidz_test/raidz_test.h +++ b/cmd/raidz_test/raidz_test.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -28,7 +28,7 @@ #include -static const char *raidz_impl_names[] = { +static const char *const raidz_impl_names[] = { "original", "scalar", "sse2", @@ -42,12 +42,18 @@ static const char *raidz_impl_names[] = { NULL }; +enum raidz_verbosity { + D_ALL, + D_INFO, + D_DEBUG, +}; + typedef struct raidz_test_opts { size_t rto_ashift; uint64_t rto_offset; size_t rto_dcols; size_t rto_dsize; - size_t rto_v; + enum raidz_verbosity rto_v; size_t rto_sweep; size_t rto_sweep_timeout; size_t rto_benchmark; @@ -68,7 +74,7 @@ static const raidz_test_opts_t rto_opts_defaults = { .rto_offset = 1ULL << 0, .rto_dcols = 8, .rto_dsize = 1<<19, - .rto_v = 0, + .rto_v = D_ALL, .rto_sweep = 0, .rto_benchmark = 0, .rto_expand = 0, @@ -86,23 +92,19 @@ static inline size_t ilog2(size_t a) } -#define D_ALL 0 -#define D_INFO 1 -#define D_DEBUG 2 - -#define LOG(lvl, a...) \ +#define LOG(lvl, ...) \ { \ if (rto_opts.rto_v >= lvl) \ - (void) fprintf(stdout, a); \ + (void) fprintf(stdout, __VA_ARGS__); \ } \ -#define LOG_OPT(lvl, opt, a...) \ +#define LOG_OPT(lvl, opt, ...) \ { \ if (opt->rto_v >= lvl) \ - (void) fprintf(stdout, a); \ + (void) fprintf(stdout, __VA_ARGS__); \ } \ -#define ERR(a...) (void) fprintf(stderr, a) +#define ERR(...) (void) fprintf(stderr, __VA_ARGS__) #define DBLSEP "================\n" diff --git a/cmd/vdev_id/Makefile.am b/cmd/vdev_id/Makefile.am deleted file mode 100644 index 4071c6d5ed6b..000000000000 --- a/cmd/vdev_id/Makefile.am +++ /dev/null @@ -1,3 +0,0 @@ -include $(top_srcdir)/config/Shellcheck.am - -dist_udev_SCRIPTS = vdev_id diff --git a/cmd/zdb/.gitignore b/cmd/zdb/.gitignore deleted file mode 100644 index f64a3fc5a160..000000000000 --- a/cmd/zdb/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/zdb diff --git a/cmd/zdb/Makefile.am b/cmd/zdb/Makefile.am index c5858c298053..b80f38b3fd57 100644 --- a/cmd/zdb/Makefile.am +++ b/cmd/zdb/Makefile.am @@ -1,18 +1,14 @@ -include $(top_srcdir)/config/Rules.am +zdb_CPPFLAGS = $(AM_CPPFLAGS) $(FORCEDEBUG_CPPFLAGS) -# Unconditionally enable debugging for zdb -AM_CPPFLAGS += -DDEBUG -UNDEBUG -DZFS_DEBUG - -sbin_PROGRAMS = zdb +sbin_PROGRAMS += zdb +CPPCHECKTARGETS += zdb zdb_SOURCES = \ - zdb.c \ - zdb_il.c \ - zdb.h + %D%/zdb.c \ + %D%/zdb.h \ + %D%/zdb_il.c zdb_LDADD = \ - $(abs_top_builddir)/lib/libzpool/libzpool.la \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la - -include $(top_srcdir)/config/CppCheck.am + libzpool.la \ + libzfs_core.la \ + libnvpair.la diff --git a/cmd/zdb/zdb.c b/cmd/zdb/zdb.c index 3fbe7510693a..fdf569691cb2 100644 --- a/cmd/zdb/zdb.c +++ b/cmd/zdb/zdb.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -102,7 +102,7 @@ #define ZDB_MAP_OBJECT_ID(obj) (obj) #endif -static char * +static const char * zdb_ot_name(dmu_object_type_t type) { if (type < DMU_OT_NUMTYPES) @@ -2035,11 +2035,8 @@ dump_ddt(ddt_t *ddt, enum ddt_type type, enum ddt_class class) static void dump_all_ddts(spa_t *spa) { - ddt_histogram_t ddh_total; - ddt_stat_t dds_total; - - bzero(&ddh_total, sizeof (ddh_total)); - bzero(&dds_total, sizeof (dds_total)); + ddt_histogram_t ddh_total = {{{0}}}; + ddt_stat_t dds_total = {0}; for (enum zio_checksum c = 0; c < ZIO_CHECKSUM_FUNCTIONS; c++) { ddt_t *ddt = spa->spa_ddt[c]; @@ -2925,7 +2922,7 @@ dsl_deadlist_entry_dump(void *arg, dsl_deadlist_entry_t *dle) } static void -dump_blkptr_list(dsl_deadlist_t *dl, char *name) +dump_blkptr_list(dsl_deadlist_t *dl, const char *name) { char bytes[32]; char comp[32]; @@ -2966,7 +2963,7 @@ dump_blkptr_list(dsl_deadlist_t *dl, char *name) if (dump_opt['d'] < 4) return; - (void) printf("\n"); + (void) putchar('\n'); dsl_deadlist_iterate(dl, dsl_deadlist_entry_dump, NULL); } @@ -3027,7 +3024,7 @@ static objset_t *sa_os = NULL; static sa_attr_type_t *sa_attr_table = NULL; static int -open_objset(const char *path, void *tag, objset_t **osp) +open_objset(const char *path, const void *tag, objset_t **osp) { int err; uint64_t sa_attrs = 0; @@ -3071,7 +3068,7 @@ open_objset(const char *path, void *tag, objset_t **osp) } static void -close_objset(objset_t *os, void *tag) +close_objset(objset_t *os, const void *tag) { VERIFY3P(os, ==, sa_os); if (os->os_sa != NULL) @@ -3101,9 +3098,8 @@ static void print_idstr(uint64_t id, const char *id_type) { if (FUID_INDEX(id)) { - char *domain; - - domain = zfs_fuid_idx_domain(&idx_tree, FUID_INDEX(id)); + const char *domain = + zfs_fuid_idx_domain(&idx_tree, FUID_INDEX(id)); (void) printf("\t%s %llx [%s-%d]\n", id_type, (u_longlong_t)id, domain, (int)FUID_RID(id)); } else { @@ -3197,13 +3193,18 @@ dump_znode_symlink(sa_handle_t *hdl) { int sa_symlink_size = 0; char linktarget[MAXPATHLEN]; - linktarget[0] = '\0'; int error; error = sa_size(hdl, sa_attr_table[ZPL_SYMLINK], &sa_symlink_size); if (error || sa_symlink_size == 0) { return; } + if (sa_symlink_size >= sizeof (linktarget)) { + (void) printf("symlink size %d is too large\n", + sa_symlink_size); + return; + } + linktarget[sa_symlink_size] = '\0'; if (sa_lookup(hdl, sa_attr_table[ZPL_SYMLINK], &linktarget, sa_symlink_size) == 0) (void) printf("\ttarget %s\n", linktarget); @@ -3564,8 +3565,15 @@ dump_object(objset_t *os, uint64_t object, int verbosity, *print_header = B_TRUE; } - if (verbosity >= 5) + if (verbosity >= 5) { + if (dn->dn_phys->dn_flags & DNODE_FLAG_SPILL_BLKPTR) { + char blkbuf[BP_SPRINTF_LEN]; + snprintf_blkptr_compact(blkbuf, sizeof (blkbuf), + DN_SPILL_BLKPTR(dn->dn_phys), B_FALSE); + (void) printf("\nSpill block: %s\n", blkbuf); + } dump_indirect(dn); + } if (verbosity >= 5) { /* @@ -3641,7 +3649,7 @@ count_ds_mos_objects(dsl_dataset_t *ds) } } -static const char *objset_types[DMU_OST_NUMTYPES] = { +static const char *const objset_types[DMU_OST_NUMTYPES] = { "NONE", "META", "ZPL", "ZVOL", "OTHER", "ANY" }; /* @@ -3651,7 +3659,7 @@ static const char *objset_types[DMU_OST_NUMTYPES] = { * pointer to point to a descriptive error message. */ static int -parse_object_range(char *range, zopt_object_range_t *zor, char **msg) +parse_object_range(char *range, zopt_object_range_t *zor, const char **msg) { uint64_t flags = 0; char *p, *s, *dup, *flagstr, *tmp = NULL; @@ -3929,7 +3937,7 @@ dump_uberblock(uberblock_t *ub, const char *header, const char *footer) (void) printf("\ttxg = %llu\n", (u_longlong_t)ub->ub_txg); (void) printf("\tguid_sum = %llu\n", (u_longlong_t)ub->ub_guid_sum); (void) printf("\ttimestamp = %llu UTC = %s", - (u_longlong_t)ub->ub_timestamp, asctime(localtime(×tamp))); + (u_longlong_t)ub->ub_timestamp, ctime(×tamp)); (void) printf("\tmmp_magic = %016llx\n", (u_longlong_t)ub->ub_mmp_magic); @@ -4234,13 +4242,13 @@ first_label(cksum_record_t *rec) } static void -print_label_numbers(char *prefix, cksum_record_t *rec) +print_label_numbers(const char *prefix, const cksum_record_t *rec) { - printf("%s", prefix); + fputs(prefix, stdout); for (int i = 0; i < VDEV_LABELS; i++) if (rec->labels[i] == B_TRUE) printf("%d ", i); - printf("\n"); + putchar('\n'); } #define MAX_UBERBLOCK_COUNT (VDEV_UBERBLOCK_RING >> UBERBLOCK_SHIFT) @@ -4360,7 +4368,7 @@ dump_l2arc_log_blocks(int fd, l2arc_dev_hdr_phys_t l2dhdr, if (!dump_opt['q']) print_l2arc_log_blocks(); - bcopy((&l2dhdr)->dh_start_lbps, lbps, sizeof (lbps)); + memcpy(lbps, l2dhdr.dh_start_lbps, sizeof (lbps)); dev.l2ad_evict = l2dhdr.dh_evict; dev.l2ad_start = l2dhdr.dh_start; @@ -4460,12 +4468,9 @@ dump_l2arc_log_blocks(int fd, l2arc_dev_hdr_phys_t l2dhdr, static int dump_l2arc_header(int fd) { - l2arc_dev_hdr_phys_t l2dhdr, rebuild; + l2arc_dev_hdr_phys_t l2dhdr = {0}, rebuild = {0}; int error = B_FALSE; - bzero(&l2dhdr, sizeof (l2dhdr)); - bzero(&rebuild, sizeof (rebuild)); - if (pread64(fd, &l2dhdr, sizeof (l2dhdr), VDEV_LABEL_START_SIZE) != sizeof (l2dhdr)) { error = B_TRUE; @@ -4820,7 +4825,7 @@ static int dump_label(const char *dev) { char path[MAXPATHLEN]; - zdb_label_t labels[VDEV_LABELS]; + zdb_label_t labels[VDEV_LABELS] = {{{{0}}}}; uint64_t psize, ashift, l2cache; struct stat64 statbuf; boolean_t config_found = B_FALSE; @@ -4831,8 +4836,6 @@ dump_label(const char *dev) void *node, *cookie; int fd; - bzero(labels, sizeof (labels)); - /* * Check if we were given absolute path and use it as is. * Otherwise if the provided vdev name doesn't point to a file, @@ -5129,7 +5132,7 @@ same_metaslab(spa_t *spa, uint64_t vdev, uint64_t off1, uint64_t off2) * Used to simplify reporting of the histogram data. */ typedef struct one_histo { - char *name; + const char *name; uint64_t *count; uint64_t *len; uint64_t cumulative; @@ -5746,14 +5749,13 @@ zdb_load_obsolete_counts(vdev_t *vd) static void zdb_ddt_leak_init(spa_t *spa, zdb_cb_t *zcb) { - ddt_bookmark_t ddb; + ddt_bookmark_t ddb = {0}; ddt_entry_t dde; int error; int p; ASSERT(!dump_opt['L']); - bzero(&ddb, sizeof (ddb)); while ((error = ddt_walk(spa, &ddb, &dde)) == 0) { blkptr_t blk; ddt_phys_t *ddp = dde.dde_phys; @@ -6413,7 +6415,7 @@ deleted_livelists_dump_mos(spa_t *spa) static int dump_block_stats(spa_t *spa) { - zdb_cb_t zcb; + zdb_cb_t zcb = {{{{0}}}}; zdb_blkstats_t *zb, *tzb; uint64_t norm_alloc, norm_space, total_alloc, total_found; int flags = TRAVERSE_PRE | TRAVERSE_PREFETCH_METADATA | @@ -6422,7 +6424,6 @@ dump_block_stats(spa_t *spa) int e, c, err; bp_embedded_type_t i; - bzero(&zcb, sizeof (zcb)); (void) printf("\nTraversing all blocks %s%s%s%s%s...\n\n", (dump_opt['c'] || !dump_opt['L']) ? "to verify " : "", (dump_opt['c'] == 1) ? "metadata " : "", @@ -6442,7 +6443,6 @@ dump_block_stats(spa_t *spa) * pool claiming each block we discover, but we skip opening any space * maps. */ - bzero(&zcb, sizeof (zdb_cb_t)); zdb_leak_init(spa, &zcb); /* @@ -6815,11 +6815,9 @@ dump_simulated_ddt(spa_t *spa) avl_tree_t t; void *cookie = NULL; zdb_ddt_entry_t *zdde; - ddt_histogram_t ddh_total; - ddt_stat_t dds_total; + ddt_histogram_t ddh_total = {{{0}}}; + ddt_stat_t dds_total = {0}; - bzero(&ddh_total, sizeof (ddh_total)); - bzero(&dds_total, sizeof (dds_total)); avl_create(&t, ddt_entry_compare, sizeof (zdb_ddt_entry_t), offsetof(zdb_ddt_entry_t, zdde_node)); @@ -7654,8 +7652,7 @@ dump_log_spacemap_obsolete_stats(spa_t *spa) if (!spa_feature_is_active(spa, SPA_FEATURE_LOG_SPACEMAP)) return; - log_sm_obsolete_stats_arg_t lsos; - bzero(&lsos, sizeof (lsos)); + log_sm_obsolete_stats_arg_t lsos = {0}; (void) printf("Log Space Map Obsolete Entry Statistics:\n"); @@ -8050,7 +8047,7 @@ zdb_decompress_block(abd_t *pabd, void *buf, void *lbuf, uint64_t lsize, lbuf, psize, lsize, NULL) == 0 && zio_decompress_data(*cfuncp, pabd, lbuf2, psize, lsize, NULL) == 0 && - bcmp(lbuf, lbuf2, lsize) == 0) + memcmp(lbuf, lbuf2, lsize) == 0) break; } if (*cfuncp != 0) @@ -8102,32 +8099,33 @@ zdb_read_block(char *thing, spa_t *spa) vdev_t *vd; abd_t *pabd; void *lbuf, *buf; - char *s, *p, *dup, *vdev, *flagstr, *sizes, *tmp = NULL; + char *s, *p, *dup, *flagstr, *sizes, *tmp = NULL; + const char *vdev, *errmsg = NULL; int i, error; boolean_t borrowed = B_FALSE, found = B_FALSE; dup = strdup(thing); s = strtok_r(dup, ":", &tmp); - vdev = s ? s : ""; + vdev = s ?: ""; s = strtok_r(NULL, ":", &tmp); offset = strtoull(s ? s : "", NULL, 16); sizes = strtok_r(NULL, ":", &tmp); s = strtok_r(NULL, ":", &tmp); - flagstr = strdup(s ? s : ""); + flagstr = strdup(s ?: ""); - s = NULL; - tmp = NULL; if (!zdb_parse_block_sizes(sizes, &lsize, &psize)) - s = "invalid size(s)"; + errmsg = "invalid size(s)"; if (!IS_P2ALIGNED(psize, DEV_BSIZE) || !IS_P2ALIGNED(lsize, DEV_BSIZE)) - s = "size must be a multiple of sector size"; + errmsg = "size must be a multiple of sector size"; if (!IS_P2ALIGNED(offset, DEV_BSIZE)) - s = "offset must be a multiple of sector size"; - if (s) { - (void) printf("Invalid block specifier: %s - %s\n", thing, s); + errmsg = "offset must be a multiple of sector size"; + if (errmsg) { + (void) printf("Invalid block specifier: %s - %s\n", + thing, errmsg); goto done; } + tmp = NULL; for (s = strtok_r(flagstr, ":", &tmp); s != NULL; s = strtok_r(NULL, ":", &tmp)) { @@ -8368,12 +8366,11 @@ zdb_read_block(char *thing, spa_t *spa) static void zdb_embedded_block(char *thing) { - blkptr_t bp; + blkptr_t bp = {{{{0}}}}; unsigned long long *words = (void *)&bp; char *buf; int err; - bzero(&bp, sizeof (bp)); err = sscanf(thing, "%llx:%llx:%llx:%llx:%llx:%llx:%llx:%llx:" "%llx:%llx:%llx:%llx:%llx:%llx:%llx:%llx", words + 0, words + 1, words + 2, words + 3, @@ -8420,7 +8417,6 @@ int main(int argc, char **argv) { int c; - struct rlimit rl = { 1024, 1024 }; spa_t *spa = NULL; objset_t *os = NULL; int dump_all = 1; @@ -8439,9 +8435,6 @@ main(int argc, char **argv) boolean_t target_is_spa = B_TRUE, dataset_lookup = B_FALSE; nvlist_t *cfg = NULL; - (void) setrlimit(RLIMIT_NOFILE, &rl); - (void) enable_extended_FILE_stdio(-1, -1); - dprintf_setup(&argc, argv); /* @@ -8566,7 +8559,7 @@ main(int argc, char **argv) } else { char **tmp = umem_alloc((nsearch + 1) * sizeof (char *), UMEM_NOFAIL); - bcopy(searchdirs, tmp, nsearch * + memcpy(tmp, searchdirs, nsearch * sizeof (char *)); umem_free(searchdirs, nsearch * sizeof (char *)); @@ -8944,13 +8937,13 @@ main(int argc, char **argv) sizeof (zopt_object_range_t)); for (unsigned i = 0; i < zopt_object_args; i++) { int err; - char *msg = NULL; + const char *msg = NULL; err = parse_object_range(argv[i], &zopt_object_ranges[i], &msg); if (err != 0) fatal("Bad object or range: '%s': %s\n", - argv[i], msg ? msg : ""); + argv[i], msg ?: ""); } } else if (argc > 0 && dump_opt['m']) { zopt_metaslab_args = argc; diff --git a/cmd/zdb/zdb.h b/cmd/zdb/zdb.h index 49579811efbb..a7d453c8c0d7 100644 --- a/cmd/zdb/zdb.h +++ b/cmd/zdb/zdb.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/cmd/zdb/zdb_il.c b/cmd/zdb/zdb_il.c index d6f588d8316e..4de7619e4bde 100644 --- a/cmd/zdb/zdb_il.c +++ b/cmd/zdb/zdb_il.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -265,6 +265,29 @@ zil_prt_rec_setattr(zilog_t *zilog, int txtype, const void *arg) } } +static void +zil_prt_rec_setsaxattr(zilog_t *zilog, int txtype, const void *arg) +{ + (void) zilog, (void) txtype; + const lr_setsaxattr_t *lr = arg; + + char *name = (char *)(lr + 1); + (void) printf("%sfoid %llu\n", tab_prefix, + (u_longlong_t)lr->lr_foid); + + (void) printf("%sXAT_NAME %s\n", tab_prefix, name); + if (lr->lr_size == 0) { + (void) printf("%sXAT_VALUE NULL\n", tab_prefix); + } else { + (void) printf("%sXAT_VALUE ", tab_prefix); + char *val = name + (strlen(name) + 1); + for (int i = 0; i < lr->lr_size; i++) { + (void) printf("%c", *val); + val++; + } + } +} + static void zil_prt_rec_acl(zilog_t *zilog, int txtype, const void *arg) { @@ -304,6 +327,8 @@ static zil_rec_info_t zil_rec_info[TX_MAX_TYPE] = { {.zri_print = zil_prt_rec_create, .zri_name = "TX_MKDIR_ATTR "}, {.zri_print = zil_prt_rec_create, .zri_name = "TX_MKDIR_ACL_ATTR "}, {.zri_print = zil_prt_rec_write, .zri_name = "TX_WRITE2 "}, + {.zri_print = zil_prt_rec_setsaxattr, + .zri_name = "TX_SETSAXATTR "}, }; static int diff --git a/cmd/zed/Makefile.am b/cmd/zed/Makefile.am index 7b662994d1c6..c437ff51dd2b 100644 --- a/cmd/zed/Makefile.am +++ b/cmd/zed/Makefile.am @@ -1,53 +1,46 @@ -include $(top_srcdir)/config/Rules.am -include $(top_srcdir)/config/Shellcheck.am - -AM_CFLAGS += $(LIBUDEV_CFLAGS) $(LIBUUID_CFLAGS) - -SUBDIRS = zed.d -SHELLCHECKDIRS = $(SUBDIRS) - -sbin_PROGRAMS = zed - -ZED_SRC = \ - zed.c \ - zed.h \ - zed_conf.c \ - zed_conf.h \ - zed_disk_event.c \ - zed_disk_event.h \ - zed_event.c \ - zed_event.h \ - zed_exec.c \ - zed_exec.h \ - zed_file.c \ - zed_file.h \ - zed_log.c \ - zed_log.h \ - zed_strings.c \ - zed_strings.h - -FMA_SRC = \ - agents/zfs_agents.c \ - agents/zfs_agents.h \ - agents/zfs_diagnosis.c \ - agents/zfs_mod.c \ - agents/zfs_retire.c \ - agents/fmd_api.c \ - agents/fmd_api.h \ - agents/fmd_serd.c \ - agents/fmd_serd.h - -zed_SOURCES = $(ZED_SRC) $(FMA_SRC) +include $(srcdir)/%D%/zed.d/Makefile.am + +zed_CFLAGS = $(AM_CFLAGS) +zed_CFLAGS += $(LIBUDEV_CFLAGS) $(LIBUUID_CFLAGS) + +sbin_PROGRAMS += zed +CPPCHECKTARGETS += zed + +zed_SOURCES = \ + %D%/zed.c \ + %D%/zed.h \ + %D%/zed_conf.c \ + %D%/zed_conf.h \ + %D%/zed_disk_event.c \ + %D%/zed_disk_event.h \ + %D%/zed_event.c \ + %D%/zed_event.h \ + %D%/zed_exec.c \ + %D%/zed_exec.h \ + %D%/zed_file.c \ + %D%/zed_file.h \ + %D%/zed_log.c \ + %D%/zed_log.h \ + %D%/zed_strings.c \ + %D%/zed_strings.h \ + \ + %D%/agents/fmd_api.c \ + %D%/agents/fmd_api.h \ + %D%/agents/fmd_serd.c \ + %D%/agents/fmd_serd.h \ + %D%/agents/zfs_agents.c \ + %D%/agents/zfs_agents.h \ + %D%/agents/zfs_diagnosis.c \ + %D%/agents/zfs_mod.c \ + %D%/agents/zfs_retire.c zed_LDADD = \ - $(abs_top_builddir)/lib/libzfs/libzfs.la \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la \ - $(abs_top_builddir)/lib/libuutil/libuutil.la + libzfs.la \ + libzfs_core.la \ + libnvpair.la \ + libuutil.la zed_LDADD += -lrt $(LIBATOMIC_LIBS) $(LIBUDEV_LIBS) $(LIBUUID_LIBS) zed_LDFLAGS = -pthread -EXTRA_DIST = agents/README.md - -include $(top_srcdir)/config/CppCheck.am +dist_noinst_DATA += %D%/agents/README.md diff --git a/cmd/zed/agents/fmd_api.c b/cmd/zed/agents/fmd_api.c index 4095901dff0d..9e46e831d517 100644 --- a/cmd/zed/agents/fmd_api.c +++ b/cmd/zed/agents/fmd_api.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -38,7 +38,7 @@ #include #include #include -#include +#include #include #include "fmd_api.h" @@ -342,11 +342,11 @@ fmd_case_uuresolved(fmd_hdl_t *hdl, const char *uuid) fmd_hdl_debug(hdl, "case resolved by uuid (%s)", uuid); } -int +boolean_t fmd_case_solved(fmd_hdl_t *hdl, fmd_case_t *cp) { (void) hdl; - return ((cp->ci_state >= FMD_CASE_SOLVED) ? FMD_B_TRUE : FMD_B_FALSE); + return (cp->ci_state >= FMD_CASE_SOLVED); } void @@ -389,7 +389,8 @@ zed_log_fault(nvlist_t *nvl, const char *uuid, const char *code) static const char * fmd_fault_mkcode(nvlist_t *fault) { - char *class, *code = "-"; + char *class; + const char *code = "-"; /* * Note: message codes come from: openzfs/usr/src/cmd/fm/dicts/ZFS.po @@ -485,7 +486,7 @@ fmd_buf_read(fmd_hdl_t *hdl, fmd_case_t *cp, assert(cp->ci_bufptr != NULL); assert(size <= cp->ci_bufsiz); - bcopy(cp->ci_bufptr, buf, size); + memcpy(buf, cp->ci_bufptr, size); } void @@ -497,7 +498,7 @@ fmd_buf_write(fmd_hdl_t *hdl, fmd_case_t *cp, assert(cp->ci_bufptr != NULL); assert(cp->ci_bufsiz >= size); - bcopy(buf, cp->ci_bufptr, size); + memcpy(cp->ci_bufptr, buf, size); } /* SERD Engines */ @@ -560,7 +561,7 @@ fmd_serd_record(fmd_hdl_t *hdl, const char *name, fmd_event_t *ep) if ((sgp = fmd_serd_eng_lookup(&mp->mod_serds, name)) == NULL) { zed_log_msg(LOG_ERR, "failed to add record to SERD engine '%s'", name); - return (FMD_B_FALSE); + return (0); } err = fmd_serd_eng_record(sgp, ep->ev_hrt); @@ -581,7 +582,7 @@ _timer_notify(union sigval sv) fmd_hdl_debug(hdl, "timer fired (%p)", ftp->ft_tid); /* disarm the timer */ - bzero(&its, sizeof (struct itimerspec)); + memset(&its, 0, sizeof (struct itimerspec)); timer_settime(ftp->ft_tid, 0, &its, NULL); /* Note that the fmdo_timeout can remove this timer */ diff --git a/cmd/zed/agents/fmd_api.h b/cmd/zed/agents/fmd_api.h index 4f06fb244b7b..b940d0d395ec 100644 --- a/cmd/zed/agents/fmd_api.h +++ b/cmd/zed/agents/fmd_api.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -72,10 +72,6 @@ typedef struct fmd_case { } fmd_case_t; -#define FMD_B_FALSE 0 /* false value for booleans as int */ -#define FMD_B_TRUE 1 /* true value for booleans as int */ - - #define FMD_CASE_UNSOLVED 0 /* case is not yet solved (waiting) */ #define FMD_CASE_SOLVED 1 /* case is solved (suspects added) */ #define FMD_CASE_CLOSE_WAIT 2 /* case is executing fmdo_close() */ @@ -176,8 +172,7 @@ extern int fmd_case_uuclosed(fmd_hdl_t *, const char *); extern int fmd_case_uuisresolved(fmd_hdl_t *, const char *); extern void fmd_case_uuresolved(fmd_hdl_t *, const char *); -extern int fmd_case_solved(fmd_hdl_t *, fmd_case_t *); -extern int fmd_case_closed(fmd_hdl_t *, fmd_case_t *); +extern boolean_t fmd_case_solved(fmd_hdl_t *, fmd_case_t *); extern void fmd_case_add_ereport(fmd_hdl_t *, fmd_case_t *, fmd_event_t *); extern void fmd_case_add_serd(fmd_hdl_t *, fmd_case_t *, const char *); diff --git a/cmd/zed/agents/fmd_serd.c b/cmd/zed/agents/fmd_serd.c index d4ec37fb7691..763ecb9589e7 100644 --- a/cmd/zed/agents/fmd_serd.c +++ b/cmd/zed/agents/fmd_serd.c @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -29,7 +29,7 @@ #include #include #include -#include +#include #include #include @@ -74,7 +74,7 @@ fmd_serd_eng_alloc(const char *name, uint64_t n, hrtime_t t) fmd_serd_eng_t *sgp; sgp = malloc(sizeof (fmd_serd_eng_t)); - bzero(sgp, sizeof (fmd_serd_eng_t)); + memset(sgp, 0, sizeof (fmd_serd_eng_t)); sgp->sg_name = strdup(name); sgp->sg_flags = FMD_SERD_DIRTY; @@ -139,7 +139,7 @@ fmd_serd_hash_destroy(fmd_serd_hash_t *shp) } free(shp->sh_hash); - bzero(shp, sizeof (fmd_serd_hash_t)); + memset(shp, 0, sizeof (fmd_serd_hash_t)); } void @@ -234,7 +234,7 @@ fmd_serd_eng_record(fmd_serd_eng_t *sgp, hrtime_t hrt) if (sgp->sg_flags & FMD_SERD_FIRED) { serd_log_msg(" SERD Engine: record %s already fired!", sgp->sg_name); - return (FMD_B_FALSE); + return (B_FALSE); } while (sgp->sg_count >= sgp->sg_n) @@ -259,11 +259,11 @@ fmd_serd_eng_record(fmd_serd_eng_t *sgp, hrtime_t hrt) fmd_event_delta(oep->se_hrt, sep->se_hrt) <= sgp->sg_t) { sgp->sg_flags |= FMD_SERD_FIRED | FMD_SERD_DIRTY; serd_log_msg(" SERD Engine: fired %s", sgp->sg_name); - return (FMD_B_TRUE); + return (B_TRUE); } sgp->sg_flags |= FMD_SERD_DIRTY; - return (FMD_B_FALSE); + return (B_FALSE); } int diff --git a/cmd/zed/agents/fmd_serd.h b/cmd/zed/agents/fmd_serd.h index c35c9acc7785..25b6888e61f2 100644 --- a/cmd/zed/agents/fmd_serd.h +++ b/cmd/zed/agents/fmd_serd.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/cmd/zed/agents/zfs_diagnosis.c b/cmd/zed/agents/zfs_diagnosis.c index 3e45a957f703..0250682f9d46 100644 --- a/cmd/zed/agents/zfs_diagnosis.c +++ b/cmd/zed/agents/zfs_diagnosis.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -27,7 +27,6 @@ #include #include -#include #include #include #include @@ -35,6 +34,7 @@ #include #include #include +#include #include "zfs_agents.h" #include "fmd_api.h" @@ -771,6 +771,8 @@ zfs_fm_recv(fmd_hdl_t *hdl, fmd_event_t *ep, nvlist_t *nvl, const char *class) ZFS_MAKE_EREPORT(FM_EREPORT_ZFS_PROBE_FAILURE))) { char *failmode = NULL; boolean_t checkremove = B_FALSE; + uint32_t pri = 0; + int32_t flags = 0; /* * If this is a checksum or I/O error, then toss it into the @@ -793,6 +795,23 @@ zfs_fm_recv(fmd_hdl_t *hdl, fmd_event_t *ep, nvlist_t *nvl, const char *class) checkremove = B_TRUE; } else if (fmd_nvl_class_match(hdl, nvl, ZFS_MAKE_EREPORT(FM_EREPORT_ZFS_CHECKSUM))) { + /* + * We ignore ereports for checksum errors generated by + * scrub/resilver I/O to avoid potentially further + * degrading the pool while it's being repaired. + */ + if (((nvlist_lookup_uint32(nvl, + FM_EREPORT_PAYLOAD_ZFS_ZIO_PRIORITY, &pri) == 0) && + (pri == ZIO_PRIORITY_SCRUB || + pri == ZIO_PRIORITY_REBUILD)) || + ((nvlist_lookup_int32(nvl, + FM_EREPORT_PAYLOAD_ZFS_ZIO_FLAGS, &flags) == 0) && + (flags & (ZIO_FLAG_SCRUB | ZIO_FLAG_RESILVER)))) { + fmd_hdl_debug(hdl, "ignoring '%s' for " + "scrub/resilver I/O", class); + return; + } + if (zcp->zc_data.zc_serd_checksum[0] == '\0') { zfs_serd_name(zcp->zc_data.zc_serd_checksum, pool_guid, vdev_guid, "checksum"); diff --git a/cmd/zed/agents/zfs_mod.c b/cmd/zed/agents/zfs_mod.c index c62a976c2e3f..d75854f2875a 100644 --- a/cmd/zed/agents/zfs_mod.c +++ b/cmd/zed/agents/zfs_mod.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -183,14 +183,14 @@ zfs_process_add(zpool_handle_t *zhp, nvlist_t *vdev, boolean_t labeled) nvlist_t *nvroot, *newvd; pendingdev_t *device; uint64_t wholedisk = 0ULL; - uint64_t offline = 0ULL; + uint64_t offline = 0ULL, faulted = 0ULL; uint64_t guid = 0ULL; char *physpath = NULL, *new_devid = NULL, *enc_sysfs_path = NULL; char rawpath[PATH_MAX], fullpath[PATH_MAX]; char devpath[PATH_MAX]; int ret; - boolean_t is_dm = B_FALSE; boolean_t is_sd = B_FALSE; + boolean_t is_mpath_wholedisk = B_FALSE; uint_t c; vdev_stat_t *vs; @@ -211,15 +211,73 @@ zfs_process_add(zpool_handle_t *zhp, nvlist_t *vdev, boolean_t labeled) &enc_sysfs_path); (void) nvlist_lookup_uint64(vdev, ZPOOL_CONFIG_WHOLE_DISK, &wholedisk); (void) nvlist_lookup_uint64(vdev, ZPOOL_CONFIG_OFFLINE, &offline); + (void) nvlist_lookup_uint64(vdev, ZPOOL_CONFIG_FAULTED, &faulted); + (void) nvlist_lookup_uint64(vdev, ZPOOL_CONFIG_GUID, &guid); - if (offline) - return; /* don't intervene if it was taken offline */ + /* + * Special case: + * + * We've seen times where a disk won't have a ZPOOL_CONFIG_PHYS_PATH + * entry in their config. For example, on this force-faulted disk: + * + * children[0]: + * type: 'disk' + * id: 0 + * guid: 14309659774640089719 + * path: '/dev/disk/by-vdev/L28' + * whole_disk: 0 + * DTL: 654 + * create_txg: 4 + * com.delphix:vdev_zap_leaf: 1161 + * faulted: 1 + * aux_state: 'external' + * children[1]: + * type: 'disk' + * id: 1 + * guid: 16002508084177980912 + * path: '/dev/disk/by-vdev/L29' + * devid: 'dm-uuid-mpath-35000c500a61d68a3' + * phys_path: 'L29' + * vdev_enc_sysfs_path: '/sys/class/enclosure/0:0:1:0/SLOT 30 32' + * whole_disk: 0 + * DTL: 1028 + * create_txg: 4 + * com.delphix:vdev_zap_leaf: 131 + * + * If the disk's path is a /dev/disk/by-vdev/ path, then we can infer + * the ZPOOL_CONFIG_PHYS_PATH from the by-vdev disk name. + */ + if (physpath == NULL && path != NULL) { + /* If path begins with "/dev/disk/by-vdev/" ... */ + if (strncmp(path, DEV_BYVDEV_PATH, + strlen(DEV_BYVDEV_PATH)) == 0) { + /* Set physpath to the char after "/dev/disk/by-vdev" */ + physpath = &path[strlen(DEV_BYVDEV_PATH)]; + } + } + + /* + * We don't want to autoreplace offlined disks. However, we do want to + * replace force-faulted disks (`zpool offline -f`). Force-faulted + * disks have both offline=1 and faulted=1 in the nvlist. + */ + if (offline && !faulted) { + zed_log_msg(LOG_INFO, "%s: %s is offline, skip autoreplace", + __func__, path); + return; + } - is_dm = zfs_dev_is_dm(path); + is_mpath_wholedisk = is_mpath_whole_disk(path); zed_log_msg(LOG_INFO, "zfs_process_add: pool '%s' vdev '%s', phys '%s'" - " wholedisk %d, %s dm (guid %llu)", zpool_get_name(zhp), path, - physpath ? physpath : "NULL", wholedisk, is_dm ? "is" : "not", + " %s blank disk, %s mpath blank disk, %s labeled, enc sysfs '%s', " + "(guid %llu)", + zpool_get_name(zhp), path, + physpath ? physpath : "NULL", + wholedisk ? "is" : "not", + is_mpath_wholedisk? "is" : "not", + labeled ? "is" : "not", + enc_sysfs_path, (long long unsigned int)guid); /* @@ -253,8 +311,9 @@ zfs_process_add(zpool_handle_t *zhp, nvlist_t *vdev, boolean_t labeled) ZFS_ONLINE_CHECKREMOVE | ZFS_ONLINE_UNSPARE, &newstate) == 0 && (newstate == VDEV_STATE_HEALTHY || newstate == VDEV_STATE_DEGRADED)) { - zed_log_msg(LOG_INFO, " zpool_vdev_online: vdev %s is %s", - fullpath, (newstate == VDEV_STATE_HEALTHY) ? + zed_log_msg(LOG_INFO, + " zpool_vdev_online: vdev '%s' ('%s') is " + "%s", fullpath, physpath, (newstate == VDEV_STATE_HEALTHY) ? "HEALTHY" : "DEGRADED"); return; } @@ -271,11 +330,12 @@ zfs_process_add(zpool_handle_t *zhp, nvlist_t *vdev, boolean_t labeled) * vdev online to trigger a FMA fault by posting an ereport. */ if (!zpool_get_prop_int(zhp, ZPOOL_PROP_AUTOREPLACE, NULL) || - !(wholedisk || is_dm) || (physpath == NULL)) { + !(wholedisk || is_mpath_wholedisk) || (physpath == NULL)) { (void) zpool_vdev_online(zhp, fullpath, ZFS_ONLINE_FORCEFAULT, &newstate); zed_log_msg(LOG_INFO, "Pool's autoreplace is not enabled or " - "not a whole disk for '%s'", fullpath); + "not a blank disk for '%s' ('%s')", fullpath, + physpath); return; } @@ -287,7 +347,7 @@ zfs_process_add(zpool_handle_t *zhp, nvlist_t *vdev, boolean_t labeled) (void) snprintf(rawpath, sizeof (rawpath), "%s%s", is_sd ? DEV_BYVDEV_PATH : DEV_BYPATH_PATH, physpath); - if (realpath(rawpath, devpath) == NULL && !is_dm) { + if (realpath(rawpath, devpath) == NULL && !is_mpath_wholedisk) { zed_log_msg(LOG_INFO, " realpath: %s failed (%s)", rawpath, strerror(errno)); @@ -303,12 +363,14 @@ zfs_process_add(zpool_handle_t *zhp, nvlist_t *vdev, boolean_t labeled) if ((vs->vs_state != VDEV_STATE_DEGRADED) && (vs->vs_state != VDEV_STATE_FAULTED) && (vs->vs_state != VDEV_STATE_CANT_OPEN)) { + zed_log_msg(LOG_INFO, " not autoreplacing since disk isn't in " + "a bad state (currently %d)", vs->vs_state); return; } nvlist_lookup_string(vdev, "new_devid", &new_devid); - if (is_dm) { + if (is_mpath_wholedisk) { /* Don't label device mapper or multipath disks. */ } else if (!labeled) { /* @@ -522,8 +584,11 @@ zfs_iter_vdev(zpool_handle_t *zhp, nvlist_t *nvl, void *data) * the dp->dd_compare value. */ if (nvlist_lookup_string(nvl, dp->dd_prop, &path) != 0 || - strcmp(dp->dd_compare, path) != 0) + strcmp(dp->dd_compare, path) != 0) { + zed_log_msg(LOG_INFO, " %s: no match (%s != vdev %s)", + __func__, dp->dd_compare, path); return; + } zed_log_msg(LOG_INFO, " zfs_iter_vdev: matched %s on %s", dp->dd_prop, path); @@ -571,6 +636,8 @@ zfs_iter_pool(zpool_handle_t *zhp, void *data) ZPOOL_CONFIG_VDEV_TREE, &nvl); zfs_iter_vdev(zhp, nvl, data); } + } else { + zed_log_msg(LOG_INFO, "%s: no config\n", __func__); } /* @@ -619,6 +686,72 @@ devphys_iter(const char *physical, const char *devid, zfs_process_func_t func, return (data.dd_found); } +/* + * Given a device identifier, find any vdevs with a matching by-vdev + * path. Normally we shouldn't need this as the comparison would be + * made earlier in the devphys_iter(). For example, if we were replacing + * /dev/disk/by-vdev/L28, normally devphys_iter() would match the + * ZPOOL_CONFIG_PHYS_PATH of "L28" from the old disk config to "L28" + * of the new disk config. However, we've seen cases where + * ZPOOL_CONFIG_PHYS_PATH was not in the config for the old disk. Here's + * an example of a real 2-disk mirror pool where one disk was force + * faulted: + * + * com.delphix:vdev_zap_top: 129 + * children[0]: + * type: 'disk' + * id: 0 + * guid: 14309659774640089719 + * path: '/dev/disk/by-vdev/L28' + * whole_disk: 0 + * DTL: 654 + * create_txg: 4 + * com.delphix:vdev_zap_leaf: 1161 + * faulted: 1 + * aux_state: 'external' + * children[1]: + * type: 'disk' + * id: 1 + * guid: 16002508084177980912 + * path: '/dev/disk/by-vdev/L29' + * devid: 'dm-uuid-mpath-35000c500a61d68a3' + * phys_path: 'L29' + * vdev_enc_sysfs_path: '/sys/class/enclosure/0:0:1:0/SLOT 30 32' + * whole_disk: 0 + * DTL: 1028 + * create_txg: 4 + * com.delphix:vdev_zap_leaf: 131 + * + * So in the case above, the only thing we could compare is the path. + * + * We can do this because we assume by-vdev paths are authoritative as physical + * paths. We could not assume this for normal paths like /dev/sda since the + * physical location /dev/sda points to could change over time. + */ +static boolean_t +by_vdev_path_iter(const char *by_vdev_path, const char *devid, + zfs_process_func_t func, boolean_t is_slice) +{ + dev_data_t data = { 0 }; + + data.dd_compare = by_vdev_path; + data.dd_func = func; + data.dd_prop = ZPOOL_CONFIG_PATH; + data.dd_found = B_FALSE; + data.dd_islabeled = is_slice; + data.dd_new_devid = devid; + + if (strncmp(by_vdev_path, DEV_BYVDEV_PATH, + strlen(DEV_BYVDEV_PATH)) != 0) { + /* by_vdev_path doesn't start with "/dev/disk/by-vdev/" */ + return (B_FALSE); + } + + (void) zpool_iter(g_zfshdl, zfs_iter_pool, &data); + + return (data.dd_found); +} + /* * Given a device identifier, find any vdevs with a matching devid. * On Linux we can match devid directly which is always a whole disk. @@ -683,15 +816,17 @@ guid_iter(uint64_t pool_guid, uint64_t vdev_guid, const char *devid, static int zfs_deliver_add(nvlist_t *nvl) { - char *devpath = NULL, *devid; + char *devpath = NULL, *devid = NULL; uint64_t pool_guid = 0, vdev_guid = 0; boolean_t is_slice; /* * Expecting a devid string and an optional physical location and guid */ - if (nvlist_lookup_string(nvl, DEV_IDENTIFIER, &devid) != 0) + if (nvlist_lookup_string(nvl, DEV_IDENTIFIER, &devid) != 0) { + zed_log_msg(LOG_INFO, "%s: no dev identifier\n", __func__); return (-1); + } (void) nvlist_lookup_string(nvl, DEV_PHYS_PATH, &devpath); (void) nvlist_lookup_uint64(nvl, ZFS_EV_POOL_GUID, &pool_guid); @@ -707,6 +842,8 @@ zfs_deliver_add(nvlist_t *nvl) * 1. ZPOOL_CONFIG_DEVID (identifies the unique disk) * 2. ZPOOL_CONFIG_PHYS_PATH (identifies disk physical location). * 3. ZPOOL_CONFIG_GUID (identifies unique vdev). + * 4. ZPOOL_CONFIG_PATH for /dev/disk/by-vdev devices only (since + * by-vdev paths represent physical paths). */ if (devid_iter(devid, zfs_process_add, is_slice)) return (0); @@ -717,6 +854,16 @@ zfs_deliver_add(nvlist_t *nvl) (void) guid_iter(pool_guid, vdev_guid, devid, zfs_process_add, is_slice); + if (devpath != NULL) { + /* Can we match a /dev/disk/by-vdev/ path? */ + char by_vdev_path[MAXPATHLEN]; + snprintf(by_vdev_path, sizeof (by_vdev_path), + "/dev/disk/by-vdev/%s", devpath); + if (by_vdev_path_iter(by_vdev_path, devid, zfs_process_add, + is_slice)) + return (0); + } + return (0); } @@ -926,7 +1073,7 @@ zfs_enum_pools(void *arg) * For now, each agent has its own libzfs instance */ int -zfs_slm_init() +zfs_slm_init(void) { if ((g_zfshdl = libzfs_init()) == NULL) return (-1); @@ -952,7 +1099,7 @@ zfs_slm_init() } void -zfs_slm_fini() +zfs_slm_fini(void) { unavailpool_t *pool; pendingdev_t *device; diff --git a/cmd/zed/agents/zfs_retire.c b/cmd/zed/agents/zfs_retire.c index 8f12bb82e1e4..a9e8baaa2c54 100644 --- a/cmd/zed/agents/zfs_retire.c +++ b/cmd/zed/agents/zfs_retire.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/cmd/zed/zed.d/Makefile.am b/cmd/zed/zed.d/Makefile.am index 3c0f5c3dd1b2..c65b43fb027e 100644 --- a/cmd/zed/zed.d/Makefile.am +++ b/cmd/zed/zed.d/Makefile.am @@ -1,36 +1,27 @@ -include $(top_srcdir)/config/Rules.am -include $(top_srcdir)/config/Substfiles.am -include $(top_srcdir)/config/Shellcheck.am - -EXTRA_DIST += README - zedconfdir = $(sysconfdir)/zfs/zed.d - dist_zedconf_DATA = \ - zed-functions.sh \ - zed.rc - -SHELLCHECKSCRIPTS = zed-functions.sh zed.rc -SHELLCHECK_SHELL = sh + %D%/zed-functions.sh \ + %D%/zed.rc zedexecdir = $(zfsexecdir)/zed.d dist_zedexec_SCRIPTS = \ - all-debug.sh \ - all-syslog.sh \ - data-notify.sh \ - generic-notify.sh \ - resilver_finish-notify.sh \ - scrub_finish-notify.sh \ - statechange-led.sh \ - statechange-notify.sh \ - vdev_clear-led.sh \ - vdev_attach-led.sh \ - pool_import-led.sh \ - resilver_finish-start-scrub.sh \ - trim_finish-notify.sh - -nodist_zedexec_SCRIPTS = history_event-zfs-list-cacher.sh + %D%/all-debug.sh \ + %D%/all-syslog.sh \ + %D%/data-notify.sh \ + %D%/generic-notify.sh \ + %D%/pool_import-led.sh \ + %D%/resilver_finish-notify.sh \ + %D%/resilver_finish-start-scrub.sh \ + %D%/scrub_finish-notify.sh \ + %D%/statechange-led.sh \ + %D%/statechange-notify.sh \ + %D%/trim_finish-notify.sh \ + %D%/vdev_attach-led.sh \ + %D%/vdev_clear-led.sh + +nodist_zedexec_SCRIPTS = \ + %D%/history_event-zfs-list-cacher.sh SUBSTFILES += $(nodist_zedexec_SCRIPTS) @@ -38,23 +29,27 @@ zedconfdefaults = \ all-syslog.sh \ data-notify.sh \ history_event-zfs-list-cacher.sh \ + pool_import-led.sh \ resilver_finish-notify.sh \ + resilver_finish-start-scrub.sh \ scrub_finish-notify.sh \ statechange-led.sh \ statechange-notify.sh \ - vdev_clear-led.sh \ vdev_attach-led.sh \ - pool_import-led.sh \ - resilver_finish-start-scrub.sh + vdev_clear-led.sh + +dist_noinst_DATA += %D%/README -install-data-hook: +INSTALL_DATA_HOOKS += zed-install-data-hook +zed-install-data-hook: $(MKDIR_P) "$(DESTDIR)$(zedconfdir)" - for f in $(zedconfdefaults); do \ - test -f "$(DESTDIR)$(zedconfdir)/$${f}" -o \ - -L "$(DESTDIR)$(zedconfdir)/$${f}" || \ - ln -s "$(zedexecdir)/$${f}" "$(DESTDIR)$(zedconfdir)"; \ + set -x; for f in $(zedconfdefaults); do \ + [ -f "$(DESTDIR)$(zedconfdir)/$${f}" ] ||\ + [ -L "$(DESTDIR)$(zedconfdir)/$${f}" ] || \ + $(LN_S) "$(zedexecdir)/$${f}" "$(DESTDIR)$(zedconfdir)"; \ done - chmod 0600 "$(DESTDIR)$(zedconfdir)/zed.rc" +SHELLCHECKSCRIPTS += $(dist_zedconf_DATA) $(dist_zedexec_SCRIPTS) $(nodist_zedexec_SCRIPTS) +$(call SHELLCHECK_OPTS,$(dist_zedconf_DATA) $(dist_zedexec_SCRIPTS) $(nodist_zedexec_SCRIPTS)): SHELLCHECK_SHELL = sh # False positive: 1>&"${ZED_FLOCK_FD}" looks suspiciously similar to a >&filename bash extension -CHECKBASHISMS_IGNORE = -e 'should be >word 2>&1' -e '&"$${ZED_FLOCK_FD}"' +$(call SHELLCHECK_OPTS,$(dist_zedconf_DATA) $(dist_zedexec_SCRIPTS) $(nodist_zedexec_SCRIPTS)): CHECKBASHISMS_IGNORE = -e 'should be >word 2>&1' -e '&"$${ZED_FLOCK_FD}"' diff --git a/cmd/zed/zed.d/all-debug.sh b/cmd/zed/zed.d/all-debug.sh index ba19b96b082f..9794cf26ce76 100755 --- a/cmd/zed/zed.d/all-debug.sh +++ b/cmd/zed/zed.d/all-debug.sh @@ -15,7 +15,7 @@ zed_exit_if_ignoring_this_event zed_lock "${ZED_DEBUG_LOG}" { - printenv | sort + env | sort echo } 1>&"${ZED_FLOCK_FD}" zed_unlock "${ZED_DEBUG_LOG}" diff --git a/cmd/zed/zed.d/zed-functions.sh b/cmd/zed/zed.d/zed-functions.sh index e6d3ce9ecc49..49b6b54029aa 100644 --- a/cmd/zed/zed.d/zed-functions.sh +++ b/cmd/zed/zed.d/zed-functions.sh @@ -76,8 +76,7 @@ zed_log_msg() # zed_log_err() { - logger -p "${ZED_SYSLOG_PRIORITY}" -t "${ZED_SYSLOG_TAG}" -- "error:" \ - "${0##*/}:""${ZEVENT_EID:+" eid=${ZEVENT_EID}:"}" "$@" + zed_log_msg "error: ${0##*/}:""${ZEVENT_EID:+" eid=${ZEVENT_EID}:"}" "$@" } @@ -224,6 +223,8 @@ zed_notify() # ZED_EMAIL_OPTS. This undergoes the following keyword substitutions: # - @ADDRESS@ is replaced with the space-delimited recipient email address(es) # - @SUBJECT@ is replaced with the notification subject +# If @SUBJECT@ was omited here, a "Subject: ..." header will be added to notification +# # # Arguments # subject: notification subject @@ -241,7 +242,7 @@ zed_notify() # zed_notify_email() { - local subject="$1" + local subject="${1:-"ZED notification"}" local pathname="${2:-"/dev/null"}" : "${ZED_EMAIL_PROG:="mail"}" @@ -262,12 +263,23 @@ zed_notify_email() return 1 fi - ZED_EMAIL_OPTS="$(echo "${ZED_EMAIL_OPTS}" \ + # construct cmdline options + ZED_EMAIL_OPTS_PARSED="$(echo "${ZED_EMAIL_OPTS}" \ | sed -e "s/@ADDRESS@/${ZED_EMAIL_ADDR}/g" \ -e "s/@SUBJECT@/${subject}/g")" + # pipe message to email prog # shellcheck disable=SC2086,SC2248 - eval ${ZED_EMAIL_PROG} ${ZED_EMAIL_OPTS} < "${pathname}" >/dev/null 2>&1 + { + # no subject passed as option? + if [ "${ZED_EMAIL_OPTS%@SUBJECT@*}" = "${ZED_EMAIL_OPTS}" ] ; then + # inject subject header + printf "Subject: %s\n" "${subject}" + fi + # output message + cat "${pathname}" + } | + eval ${ZED_EMAIL_PROG} ${ZED_EMAIL_OPTS_PARSED} >/dev/null 2>&1 rv=$? if [ "${rv}" -ne 0 ]; then zed_log_err "${ZED_EMAIL_PROG##*/} exit=${rv}" diff --git a/cmd/zed/zed.d/zed.rc b/cmd/zed/zed.d/zed.rc index a8fdc86c3431..c55a70c79f75 100644 --- a/cmd/zed/zed.d/zed.rc +++ b/cmd/zed/zed.d/zed.rc @@ -1,7 +1,5 @@ ## -# zed.rc -# -# This file should be owned by root and permissioned 0600. +# zed.rc – ZEDLET configuration. ## # shellcheck disable=SC2034 @@ -31,6 +29,7 @@ ZED_EMAIL_ADDR="root" # The string @SUBJECT@ will be replaced with the notification subject; # this should be protected with quotes to prevent word-splitting. # Email will only be sent if ZED_EMAIL_ADDR is defined. +# If @SUBJECT@ was omited here, a "Subject: ..." header will be added to notification # #ZED_EMAIL_OPTS="-s '@SUBJECT@' @ADDRESS@" diff --git a/cmd/zed/zed_disk_event.c b/cmd/zed/zed_disk_event.c index 94e24236063c..8845c5b2d009 100644 --- a/cmd/zed/zed_disk_event.c +++ b/cmd/zed/zed_disk_event.c @@ -169,7 +169,7 @@ zed_udev_monitor(void *arg) while (1) { struct udev_device *dev; const char *action, *type, *part, *sectors; - const char *bus, *uuid; + const char *bus, *uuid, *devpath; const char *class, *subclass; nvlist_t *nvl; boolean_t is_zfs = B_FALSE; @@ -208,6 +208,12 @@ zed_udev_monitor(void *arg) * if this is a disk and it is partitioned, then the * zfs label will reside in a DEVTYPE=partition and * we can skip passing this event + * + * Special case: Blank disks are sometimes reported with + * an erroneous 'atari' partition, and should not be + * excluded from being used as an autoreplace disk: + * + * https://github.com/openzfs/zfs/issues/13497 */ type = udev_device_get_property_value(dev, "DEVTYPE"); part = udev_device_get_property_value(dev, @@ -215,9 +221,23 @@ zed_udev_monitor(void *arg) if (type != NULL && type[0] != '\0' && strcmp(type, "disk") == 0 && part != NULL && part[0] != '\0') { - /* skip and wait for partition event */ - udev_device_unref(dev); - continue; + const char *devname = + udev_device_get_property_value(dev, "DEVNAME"); + + if (strcmp(part, "atari") == 0) { + zed_log_msg(LOG_INFO, + "%s: %s is reporting an atari partition, " + "but we're going to assume it's a false " + "positive and still use it (issue #13497)", + __func__, devname); + } else { + zed_log_msg(LOG_INFO, + "%s: skip %s since it has a %s partition " + "already", __func__, devname, part); + /* skip and wait for partition event */ + udev_device_unref(dev); + continue; + } } /* @@ -229,6 +249,11 @@ zed_udev_monitor(void *arg) sectors = udev_device_get_sysattr_value(dev, "size"); if (sectors != NULL && strtoull(sectors, NULL, 10) < MINIMUM_SECTORS) { + zed_log_msg(LOG_INFO, + "%s: %s sectors %s < %llu (minimum)", + __func__, + udev_device_get_property_value(dev, "DEVNAME"), + sectors, MINIMUM_SECTORS); udev_device_unref(dev); continue; } @@ -238,10 +263,19 @@ zed_udev_monitor(void *arg) * device id string is required in the message schema * for matching with vdevs. Preflight here for expected * udev information. + * + * Special case: + * NVMe devices don't have ID_BUS set (at least on RHEL 7-8), + * but they are valid for autoreplace. Add a special case for + * them by searching for "/nvme/" in the udev DEVPATH: + * + * DEVPATH=/devices/pci0000:00/0000:00:1e.0/nvme/nvme2/nvme2n1 */ bus = udev_device_get_property_value(dev, "ID_BUS"); uuid = udev_device_get_property_value(dev, "DM_UUID"); - if (!is_zfs && (bus == NULL && uuid == NULL)) { + devpath = udev_device_get_devpath(dev); + if (!is_zfs && (bus == NULL && uuid == NULL && + strstr(devpath, "/nvme/") == NULL)) { zed_log_msg(LOG_INFO, "zed_udev_monitor: %s no devid " "source", udev_device_get_devnode(dev)); udev_device_unref(dev); @@ -352,7 +386,7 @@ zed_udev_monitor(void *arg) } int -zed_disk_event_init() +zed_disk_event_init(void) { int fd, fflags; @@ -388,7 +422,7 @@ zed_disk_event_init() } void -zed_disk_event_fini() +zed_disk_event_fini(void) { /* cancel monitor thread at recvmsg() */ (void) pthread_cancel(g_mon_tid); @@ -406,13 +440,13 @@ zed_disk_event_fini() #include "zed_disk_event.h" int -zed_disk_event_init() +zed_disk_event_init(void) { return (0); } void -zed_disk_event_fini() +zed_disk_event_fini(void) { } diff --git a/cmd/zed/zed_event.c b/cmd/zed/zed_event.c index 9eaad0e92fbb..d5cafdaff061 100644 --- a/cmd/zed/zed_event.c +++ b/cmd/zed/zed_event.c @@ -884,21 +884,21 @@ _zed_event_get_subclass(const char *class) static void _zed_event_add_time_strings(uint64_t eid, zed_strings_t *zsp, int64_t etime[]) { - struct tm *stp; + struct tm stp; char buf[32]; assert(zsp != NULL); assert(etime != NULL); _zed_event_add_var(eid, zsp, ZEVENT_VAR_PREFIX, "TIME_SECS", - "%lld", (long long int) etime[0]); + "%" PRId64, etime[0]); _zed_event_add_var(eid, zsp, ZEVENT_VAR_PREFIX, "TIME_NSECS", - "%lld", (long long int) etime[1]); + "%" PRId64, etime[1]); - if (!(stp = localtime((const time_t *) &etime[0]))) { + if (!localtime_r((const time_t *) &etime[0], &stp)) { zed_log_msg(LOG_WARNING, "Failed to add %s%s for eid=%llu: %s", ZEVENT_VAR_PREFIX, "TIME_STRING", eid, "localtime error"); - } else if (!strftime(buf, sizeof (buf), "%Y-%m-%d %H:%M:%S%z", stp)) { + } else if (!strftime(buf, sizeof (buf), "%Y-%m-%d %H:%M:%S%z", &stp)) { zed_log_msg(LOG_WARNING, "Failed to add %s%s for eid=%llu: %s", ZEVENT_VAR_PREFIX, "TIME_STRING", eid, "strftime error"); } else { diff --git a/cmd/zfs/.gitignore b/cmd/zfs/.gitignore deleted file mode 100644 index 0fd9cc63af2a..000000000000 --- a/cmd/zfs/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/zfs diff --git a/cmd/zfs/Makefile.am b/cmd/zfs/Makefile.am index 1ead457f0f29..8a3c13a1fcfe 100644 --- a/cmd/zfs/Makefile.am +++ b/cmd/zfs/Makefile.am @@ -1,25 +1,22 @@ -include $(top_srcdir)/config/Rules.am - -sbin_PROGRAMS = zfs +sbin_PROGRAMS += zfs +CPPCHECKTARGETS += zfs zfs_SOURCES = \ - zfs_iter.c \ - zfs_iter.h \ - zfs_main.c \ - zfs_util.h \ - zfs_project.c \ - zfs_projectutil.h + %D%/zfs_iter.c \ + %D%/zfs_iter.h \ + %D%/zfs_main.c \ + %D%/zfs_project.c \ + %D%/zfs_projectutil.h \ + %D%/zfs_util.h zfs_LDADD = \ - $(abs_top_builddir)/lib/libzfs/libzfs.la \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la \ - $(abs_top_builddir)/lib/libuutil/libuutil.la + libzfs.la \ + libzfs_core.la \ + libnvpair.la \ + libuutil.la zfs_LDADD += $(LTLIBINTL) if BUILD_FREEBSD zfs_LDADD += -lgeom -ljail endif - -include $(top_srcdir)/config/CppCheck.am diff --git a/cmd/zfs/zfs_iter.c b/cmd/zfs/zfs_iter.c index 69b802f77a78..a0a80d481648 100644 --- a/cmd/zfs/zfs_iter.c +++ b/cmd/zfs/zfs_iter.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -32,7 +32,6 @@ #include #include #include -#include #include @@ -175,7 +174,7 @@ zfs_add_sort_column(zfs_sort_column_t **sc, const char *name, zfs_sort_column_t *col; zfs_prop_t prop; - if ((prop = zfs_name_to_prop(name)) == ZPROP_INVAL && + if ((prop = zfs_name_to_prop(name)) == ZPROP_USERPROP && !zfs_prop_user(name)) return (-1); @@ -183,7 +182,7 @@ zfs_add_sort_column(zfs_sort_column_t **sc, const char *name, col->sc_prop = prop; col->sc_reverse = reverse; - if (prop == ZPROP_INVAL) { + if (prop == ZPROP_USERPROP) { col->sc_user_prop = safe_malloc(strlen(name) + 1); (void) strcpy(col->sc_user_prop, name); } @@ -219,6 +218,13 @@ zfs_sort_only_by_name(const zfs_sort_column_t *sc) sc->sc_prop == ZFS_PROP_NAME); } +int +zfs_sort_only_by_createtxg(const zfs_sort_column_t *sc) +{ + return (sc != NULL && sc->sc_next == NULL && + sc->sc_prop == ZFS_PROP_CREATETXG); +} + static int zfs_compare(const void *larg, const void *rarg) { @@ -302,7 +308,7 @@ zfs_sort(const void *larg, const void *rarg, void *data) for (psc = sc; psc != NULL; psc = psc->sc_next) { char lbuf[ZFS_MAXPROPLEN], rbuf[ZFS_MAXPROPLEN]; char *lstr, *rstr; - uint64_t lnum, rnum; + uint64_t lnum = 0, rnum = 0; boolean_t lvalid, rvalid; int ret = 0; @@ -312,7 +318,7 @@ zfs_sort(const void *larg, const void *rarg, void *data) * Otherwise, we compare 'lnum' and 'rnum'. */ lstr = rstr = NULL; - if (psc->sc_prop == ZPROP_INVAL) { + if (psc->sc_prop == ZPROP_USERPROP) { nvlist_t *luser, *ruser; nvlist_t *lval, *rval; @@ -353,11 +359,9 @@ zfs_sort(const void *larg, const void *rarg, void *data) zfs_get_type(r), B_FALSE); if (lvalid) - (void) zfs_prop_get_numeric(l, psc->sc_prop, - &lnum, NULL, NULL, 0); + lnum = zfs_prop_get_int(l, psc->sc_prop); if (rvalid) - (void) zfs_prop_get_numeric(r, psc->sc_prop, - &rnum, NULL, NULL, 0); + rnum = zfs_prop_get_int(r, psc->sc_prop); } if (!lvalid && !rvalid) @@ -453,23 +457,21 @@ zfs_for_each(int argc, char **argv, int flags, zfs_type_t types, cb.cb_flags |= ZFS_ITER_RECURSE; ret = zfs_iter_root(g_zfs, zfs_callback, &cb); } else { - int i; - zfs_handle_t *zhp; - zfs_type_t argtype; + zfs_handle_t *zhp = NULL; + zfs_type_t argtype = types; /* * If we're recursive, then we always allow filesystems as * arguments. If we also are interested in snapshots or * bookmarks, then we can take volumes as well. */ - argtype = types; if (flags & ZFS_ITER_RECURSE) { argtype |= ZFS_TYPE_FILESYSTEM; if (types & (ZFS_TYPE_SNAPSHOT | ZFS_TYPE_BOOKMARK)) argtype |= ZFS_TYPE_VOLUME; } - for (i = 0; i < argc; i++) { + for (int i = 0; i < argc; i++) { if (flags & ZFS_ITER_ARGS_CAN_BE_PATHS) { zhp = zfs_path_to_zhandle(g_zfs, argv[i], argtype); diff --git a/cmd/zfs/zfs_iter.h b/cmd/zfs/zfs_iter.h index 2697fbdca1df..effb22ded3fc 100644 --- a/cmd/zfs/zfs_iter.h +++ b/cmd/zfs/zfs_iter.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -53,6 +53,7 @@ int zfs_for_each(int, char **, int options, zfs_type_t, int zfs_add_sort_column(zfs_sort_column_t **, const char *, boolean_t); void zfs_free_sort_columns(zfs_sort_column_t *); int zfs_sort_only_by_name(const zfs_sort_column_t *); +int zfs_sort_only_by_createtxg(const zfs_sort_column_t *); #ifdef __cplusplus } diff --git a/cmd/zfs/zfs_main.c b/cmd/zfs/zfs_main.c index 84f69f4e261b..f1d686753c25 100644 --- a/cmd/zfs/zfs_main.c +++ b/cmd/zfs/zfs_main.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -46,7 +46,7 @@ #include #include #include -#include +#include #include #include #include @@ -127,6 +127,11 @@ static int zfs_do_jail(int argc, char **argv); static int zfs_do_unjail(int argc, char **argv); #endif +#ifdef __linux__ +static int zfs_do_zone(int argc, char **argv); +static int zfs_do_unzone(int argc, char **argv); +#endif + /* * Enable a reasonable set of defaults for libumem debugging on DEBUG builds. */ @@ -184,6 +189,8 @@ typedef enum { HELP_JAIL, HELP_UNJAIL, HELP_WAIT, + HELP_ZONE, + HELP_UNZONE, } zfs_help_t; typedef struct zfs_command { @@ -254,6 +261,11 @@ static zfs_command_t command_table[] = { { "jail", zfs_do_jail, HELP_JAIL }, { "unjail", zfs_do_unjail, HELP_UNJAIL }, #endif + +#ifdef __linux__ + { "zone", zfs_do_zone, HELP_ZONE }, + { "unzone", zfs_do_unzone, HELP_UNZONE }, +#endif }; #define NCOMMAND (sizeof (command_table) / sizeof (command_table[0])) @@ -315,8 +327,9 @@ get_usage(zfs_help_t idx) case HELP_ROLLBACK: return (gettext("\trollback [-rRf] \n")); case HELP_SEND: - return (gettext("\tsend [-DnPpRvLecwhb] [-[i|I] snapshot] " - "\n" + return (gettext("\tsend [-DLPbcehnpsvw] " + "[-i|-I snapshot]\n" + "\t [-R [-X dataset[,dataset]...]] \n" "\tsend [-DnvPLecw] [-i snapshot|bookmark] " "\n" "\tsend [-DnPpvLec] [-i bookmark|snapshot] " @@ -414,6 +427,10 @@ get_usage(zfs_help_t idx) return (gettext("\tunjail \n")); case HELP_WAIT: return (gettext("\twait [-t ] \n")); + case HELP_ZONE: + return (gettext("\tzone \n")); + case HELP_UNZONE: + return (gettext("\tunzone \n")); default: __builtin_unreachable(); } @@ -454,7 +471,7 @@ safe_realloc(void *data, size_t size) } static char * -safe_strdup(char *str) +safe_strdup(const char *str) { char *dupstr = strdup(str); @@ -485,10 +502,7 @@ usage_prop_cb(int prop, void *cb) else (void) fprintf(fp, " NO "); - if (zfs_prop_values(prop) == NULL) - (void) fprintf(fp, "-\n"); - else - (void) fprintf(fp, "%s\n", zfs_prop_values(prop)); + (void) fprintf(fp, "%s\n", zfs_prop_values(prop) ?: "-"); return (ZPROP_CONT); } @@ -498,7 +512,7 @@ usage_prop_cb(int prop, void *cb) * that command. Otherwise, iterate over the entire command table and display * a complete usage message. */ -static void +static __attribute__((noreturn)) void usage(boolean_t requested) { int i; @@ -676,7 +690,8 @@ parse_depth(char *opt, int *flags) #define PROGRESS_DELAY 2 /* seconds */ -static char *pt_reverse = "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b"; +static const char *pt_reverse = + "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b"; static time_t pt_begin; static char *pt_header = NULL; static boolean_t pt_shown; @@ -689,7 +704,7 @@ start_progress_timer(void) } static void -set_progress_header(char *header) +set_progress_header(const char *header) { assert(pt_header == NULL); pt_header = safe_strdup(header); @@ -700,7 +715,7 @@ set_progress_header(char *header) } static void -update_progress(char *update) +update_progress(const char *update) { if (!pt_shown && time(NULL) > pt_begin) { int len = strlen(update); @@ -718,42 +733,16 @@ update_progress(char *update) } static void -finish_progress(char *done) +finish_progress(const char *done) { if (pt_shown) { - (void) printf("%s\n", done); + (void) puts(done); (void) fflush(stdout); } free(pt_header); pt_header = NULL; } -/* This function checks if the passed fd refers to /dev/null or /dev/zero */ -#ifdef __linux__ -static boolean_t -is_dev_nullzero(int fd) -{ - struct stat st; - fstat(fd, &st); - return (major(st.st_rdev) == 1 && (minor(st.st_rdev) == 3 /* null */ || - minor(st.st_rdev) == 5 /* zero */)); -} -#endif - -static void -note_dev_error(int err, int fd) -{ -#ifdef __linux__ - if (err == EINVAL && is_dev_nullzero(fd)) { - (void) fprintf(stderr, - gettext("Error: Writing directly to /dev/{null,zero} files" - " on certain kernels is not currently implemented.\n" - "(As a workaround, " - "try \"zfs send [...] | cat > /dev/null\")\n")); - } -#endif -} - static int zfs_mount_and_share(libzfs_handle_t *hdl, const char *dataset, zfs_type_t type) { @@ -792,12 +781,12 @@ zfs_mount_and_share(libzfs_handle_t *hdl, const char *dataset, zfs_type_t type) (void) fprintf(stderr, gettext("filesystem " "successfully created, but not mounted\n")); ret = 1; - } else if (zfs_share(zhp) != 0) { + } else if (zfs_share(zhp, NULL) != 0) { (void) fprintf(stderr, gettext("filesystem " "successfully created, but not shared\n")); ret = 1; } - zfs_commit_all_shares(); + zfs_commit_shares(NULL); } zfs_close(zhp); @@ -1915,8 +1904,8 @@ get_callback(zfs_handle_t *zhp, void *data) nvlist_t *user_props = zfs_get_user_props(zhp); zprop_list_t *pl = cbp->cb_proplist; nvlist_t *propval; - char *strval; - char *sourceval; + const char *strval; + const char *sourceval; boolean_t received = is_recvd_column(cbp); for (; pl != NULL; pl = pl->pl_next) { @@ -1929,7 +1918,7 @@ get_callback(zfs_handle_t *zhp, void *data) pl == cbp->cb_proplist) continue; - if (pl->pl_prop != ZPROP_INVAL) { + if (pl->pl_prop != ZPROP_USERPROP) { if (zfs_prop_get(zhp, pl->pl_prop, buf, sizeof (buf), &sourcetype, source, sizeof (source), @@ -1985,10 +1974,10 @@ get_callback(zfs_handle_t *zhp, void *data) sourcetype = ZPROP_SRC_NONE; strval = "-"; } else { - verify(nvlist_lookup_string(propval, - ZPROP_VALUE, &strval) == 0); - verify(nvlist_lookup_string(propval, - ZPROP_SOURCE, &sourceval) == 0); + strval = fnvlist_lookup_string(propval, + ZPROP_VALUE); + sourceval = fnvlist_lookup_string(propval, + ZPROP_SOURCE); if (strcmp(sourceval, zfs_get_name(zhp)) == 0) { @@ -2023,7 +2012,7 @@ zfs_do_get(int argc, char **argv) zprop_get_cbdata_t cb = { 0 }; int i, c, flags = ZFS_ITER_ARGS_CAN_BE_PATHS; int types = ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK; - char *value, *fields; + char *fields; int ret = 0; int limit = 0; zprop_list_t fake_name = { 0 }; @@ -2063,12 +2052,18 @@ zfs_do_get(int argc, char **argv) * Process the set of columns to display. We zero out * the structure to give us a blank slate. */ - bzero(&cb.cb_columns, sizeof (cb.cb_columns)); + memset(&cb.cb_columns, 0, sizeof (cb.cb_columns)); + i = 0; - while (*optarg != '\0') { - static char *col_subopts[] = - { "name", "property", "value", "received", - "source", "all", NULL }; + for (char *tok; (tok = strsep(&optarg, ",")); ) { + static const char *const col_subopts[] = + { "name", "property", "value", + "received", "source", "all" }; + static const zfs_get_column_t col_subopt_col[] = + { GET_COL_NAME, GET_COL_PROPERTY, GET_COL_VALUE, + GET_COL_RECVD, GET_COL_SOURCE }; + static const int col_subopt_flags[] = + { 0, 0, 0, ZFS_ITER_RECVD_PROPS, 0 }; if (i == ZFS_GET_NCOLS) { (void) fprintf(stderr, gettext("too " @@ -2077,25 +2072,16 @@ zfs_do_get(int argc, char **argv) usage(B_FALSE); } - switch (getsubopt(&optarg, col_subopts, - &value)) { - case 0: - cb.cb_columns[i++] = GET_COL_NAME; - break; - case 1: - cb.cb_columns[i++] = GET_COL_PROPERTY; - break; - case 2: - cb.cb_columns[i++] = GET_COL_VALUE; - break; - case 3: - cb.cb_columns[i++] = GET_COL_RECVD; - flags |= ZFS_ITER_RECVD_PROPS; - break; - case 4: - cb.cb_columns[i++] = GET_COL_SOURCE; - break; - case 5: + for (c = 0; c < ARRAY_SIZE(col_subopts); ++c) + if (strcmp(tok, col_subopts[c]) == 0) + goto found; + + (void) fprintf(stderr, + gettext("invalid column name '%s'\n"), tok); + usage(B_FALSE); + +found: + if (c >= 5) { if (i > 0) { (void) fprintf(stderr, gettext("\"all\" conflicts " @@ -2103,94 +2089,70 @@ zfs_do_get(int argc, char **argv) "given to -o option\n")); usage(B_FALSE); } - cb.cb_columns[0] = GET_COL_NAME; - cb.cb_columns[1] = GET_COL_PROPERTY; - cb.cb_columns[2] = GET_COL_VALUE; - cb.cb_columns[3] = GET_COL_RECVD; - cb.cb_columns[4] = GET_COL_SOURCE; + + memcpy(cb.cb_columns, col_subopt_col, + sizeof (col_subopt_col)); flags |= ZFS_ITER_RECVD_PROPS; i = ZFS_GET_NCOLS; - break; - default: - (void) fprintf(stderr, - gettext("invalid column name " - "'%s'\n"), value); - usage(B_FALSE); + } else { + cb.cb_columns[i++] = col_subopt_col[c]; + flags |= col_subopt_flags[c]; } } break; case 's': cb.cb_sources = 0; - while (*optarg != '\0') { - static char *source_subopts[] = { - "local", "default", "inherited", - "received", "temporary", "none", - NULL }; - - switch (getsubopt(&optarg, source_subopts, - &value)) { - case 0: - cb.cb_sources |= ZPROP_SRC_LOCAL; - break; - case 1: - cb.cb_sources |= ZPROP_SRC_DEFAULT; - break; - case 2: - cb.cb_sources |= ZPROP_SRC_INHERITED; - break; - case 3: - cb.cb_sources |= ZPROP_SRC_RECEIVED; - break; - case 4: - cb.cb_sources |= ZPROP_SRC_TEMPORARY; - break; - case 5: - cb.cb_sources |= ZPROP_SRC_NONE; - break; - default: - (void) fprintf(stderr, - gettext("invalid source " - "'%s'\n"), value); - usage(B_FALSE); - } + + for (char *tok; (tok = strsep(&optarg, ",")); ) { + static const char *const source_opt[] = { + "local", "default", + "inherited", "received", + "temporary", "none" }; + static const int source_flg[] = { + ZPROP_SRC_LOCAL, ZPROP_SRC_DEFAULT, + ZPROP_SRC_INHERITED, ZPROP_SRC_RECEIVED, + ZPROP_SRC_TEMPORARY, ZPROP_SRC_NONE }; + + for (i = 0; i < ARRAY_SIZE(source_opt); ++i) + if (strcmp(tok, source_opt[i]) == 0) { + cb.cb_sources |= source_flg[i]; + goto found2; + } + + (void) fprintf(stderr, + gettext("invalid source '%s'\n"), tok); + usage(B_FALSE); +found2:; } break; case 't': types = 0; flags &= ~ZFS_ITER_PROP_LISTSNAPS; - while (*optarg != '\0') { - static char *type_subopts[] = { "filesystem", - "volume", "snapshot", "snap", "bookmark", - "all", NULL }; - - switch (getsubopt(&optarg, type_subopts, - &value)) { - case 0: - types |= ZFS_TYPE_FILESYSTEM; - break; - case 1: - types |= ZFS_TYPE_VOLUME; - break; - case 2: - case 3: - types |= ZFS_TYPE_SNAPSHOT; - break; - case 4: - types |= ZFS_TYPE_BOOKMARK; - break; - case 5: - types = ZFS_TYPE_DATASET | - ZFS_TYPE_BOOKMARK; - break; - default: - (void) fprintf(stderr, - gettext("invalid type '%s'\n"), - value); - usage(B_FALSE); - } + for (char *tok; (tok = strsep(&optarg, ",")); ) { + static const char *const type_opts[] = { + "filesystem", "volume", + "snapshot", "snap", + "bookmark", + "all" }; + static const int type_types[] = { + ZFS_TYPE_FILESYSTEM, ZFS_TYPE_VOLUME, + ZFS_TYPE_SNAPSHOT, ZFS_TYPE_SNAPSHOT, + ZFS_TYPE_BOOKMARK, + ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK }; + + for (i = 0; i < ARRAY_SIZE(type_opts); ++i) + if (strcmp(tok, type_opts[i]) == 0) { + types |= type_types[i]; + goto found3; + } + + (void) fprintf(stderr, + gettext("invalid type '%s'\n"), tok); + usage(B_FALSE); +found3:; } break; @@ -2346,7 +2308,7 @@ zfs_do_inherit(int argc, char **argv) argc--; argv++; - if ((prop = zfs_name_to_prop(propname)) != ZPROP_INVAL) { + if ((prop = zfs_name_to_prop(propname)) != ZPROP_USERPROP) { if (zfs_prop_readonly(prop)) { (void) fprintf(stderr, gettext( "%s property is read-only\n"), @@ -2480,7 +2442,7 @@ upgrade_set_callback(zfs_handle_t *zhp, void *data) /* upgrade */ if (version < cb->cb_version) { - char verstr[16]; + char verstr[24]; (void) snprintf(verstr, sizeof (verstr), "%llu", (u_longlong_t)cb->cb_version); if (cb->cb_lastfs[0] && !same_pool(zhp, cb->cb_lastfs)) { @@ -2657,9 +2619,9 @@ enum us_field_types { USFIELD_OBJUSED, USFIELD_OBJQUOTA }; -static char *us_field_hdr[] = { "TYPE", "NAME", "USED", "QUOTA", +static const char *const us_field_hdr[] = { "TYPE", "NAME", "USED", "QUOTA", "OBJUSED", "OBJQUOTA" }; -static char *us_field_names[] = { "type", "name", "used", "quota", +static const char *const us_field_names[] = { "type", "name", "used", "quota", "objused", "objquota" }; #define USFIELD_LAST (sizeof (us_field_names) / sizeof (char *)) @@ -2679,8 +2641,8 @@ static int us_type_bits[] = { USTYPE_SMB_USR, USTYPE_ALL }; -static char *us_type_names[] = { "posixgroup", "posixuser", "smbgroup", - "smbuser", "all" }; +static const char *const us_type_names[] = { "posixgroup", "posixuser", + "smbgroup", "smbuser", "all" }; typedef struct us_node { nvlist_t *usn_nvl; @@ -2708,11 +2670,9 @@ typedef struct { } us_sort_info_t; static int -us_field_index(char *field) +us_field_index(const char *field) { - int i; - - for (i = 0; i < USFIELD_LAST; i++) { + for (int i = 0; i < USFIELD_LAST; i++) { if (strcmp(field, us_field_names[i]) == 0) return (i); } @@ -2734,8 +2694,8 @@ us_compare(const void *larg, const void *rarg, void *unused) boolean_t lvb, rvb; for (; sortcol != NULL; sortcol = sortcol->sc_next) { - char *lvstr = ""; - char *rvstr = ""; + char *lvstr = (char *)""; + char *rvstr = (char *)""; uint32_t lv32 = 0; uint32_t rv32 = 0; uint64_t lv64 = 0; @@ -2857,7 +2817,7 @@ userspace_cb(void *arg, const char *domain, uid_t rid, uint64_t space) us_cbdata_t *cb = (us_cbdata_t *)arg; zfs_userquota_prop_t prop = cb->cb_prop; char *name = NULL; - char *propname; + const char *propname; char sizebuf[32]; us_node_t *node; uu_avl_pool_t *avl_pool = cb->cb_avl_pool; @@ -3057,26 +3017,25 @@ print_us_node(boolean_t scripted, boolean_t parsable, int *fields, int types, while ((field = fields[cfield]) != USFIELD_LAST) { nvpair_t *nvp = NULL; data_type_t type; - uint32_t val32; - uint64_t val64; - char *strval = "-"; + uint32_t val32 = -1; + uint64_t val64 = -1; + const char *strval = "-"; - while ((nvp = nvlist_next_nvpair(nvl, nvp)) != NULL) { + while ((nvp = nvlist_next_nvpair(nvl, nvp)) != NULL) if (strcmp(nvpair_name(nvp), us_field_names[field]) == 0) break; - } type = nvp == NULL ? DATA_TYPE_UNKNOWN : nvpair_type(nvp); switch (type) { case DATA_TYPE_UINT32: - (void) nvpair_value_uint32(nvp, &val32); + val32 = fnvpair_value_uint32(nvp); break; case DATA_TYPE_UINT64: - (void) nvpair_value_uint64(nvp, &val64); + val64 = fnvpair_value_uint64(nvp); break; case DATA_TYPE_STRING: - (void) nvpair_value_string(nvp, &strval); + strval = fnvpair_value_string(nvp); break; case DATA_TYPE_UNKNOWN: break; @@ -3087,7 +3046,7 @@ print_us_node(boolean_t scripted, boolean_t parsable, int *fields, int types, switch (field) { case USFIELD_TYPE: if (type == DATA_TYPE_UINT32) - strval = (char *)us_type2str(val32); + strval = us_type2str(val32); break; case USFIELD_NAME: if (type == DATA_TYPE_UINT64) { @@ -3134,12 +3093,12 @@ print_us_node(boolean_t scripted, boolean_t parsable, int *fields, int types, if (!first) { if (scripted) - (void) printf("\t"); + (void) putchar('\t'); else - (void) printf(" "); + (void) fputs(" ", stdout); } if (scripted) - (void) printf("%s", strval); + (void) fputs(strval, stdout); else if (field == USFIELD_TYPE || field == USFIELD_NAME) (void) printf("%-*s", (int)width[field], strval); else @@ -3149,7 +3108,7 @@ print_us_node(boolean_t scripted, boolean_t parsable, int *fields, int types, cfield++; } - (void) printf("\n"); + (void) putchar('\n'); } static void @@ -3482,7 +3441,7 @@ print_header(list_cbdata_t *cb) } right_justify = B_FALSE; - if (pl->pl_prop != ZPROP_INVAL) { + if (pl->pl_prop != ZPROP_USERPROP) { header = zfs_prop_column_name(pl->pl_prop); right_justify = zfs_prop_align_right(pl->pl_prop); } else { @@ -3515,15 +3474,15 @@ print_dataset(zfs_handle_t *zhp, list_cbdata_t *cb) char property[ZFS_MAXPROPLEN]; nvlist_t *userprops = zfs_get_user_props(zhp); nvlist_t *propval; - char *propstr; + const char *propstr; boolean_t right_justify; for (; pl != NULL; pl = pl->pl_next) { if (!first) { if (cb->cb_scripted) - (void) printf("\t"); + (void) putchar('\t'); else - (void) printf(" "); + (void) fputs(" ", stdout); } else { first = B_FALSE; } @@ -3533,7 +3492,7 @@ print_dataset(zfs_handle_t *zhp, list_cbdata_t *cb) sizeof (property)); propstr = property; right_justify = zfs_prop_align_right(pl->pl_prop); - } else if (pl->pl_prop != ZPROP_INVAL) { + } else if (pl->pl_prop != ZPROP_USERPROP) { if (zfs_prop_get(zhp, pl->pl_prop, property, sizeof (property), NULL, NULL, 0, cb->cb_literal) != 0) @@ -3560,8 +3519,8 @@ print_dataset(zfs_handle_t *zhp, list_cbdata_t *cb) pl->pl_user_prop, &propval) != 0) propstr = "-"; else - verify(nvlist_lookup_string(propval, - ZPROP_VALUE, &propstr) == 0); + propstr = fnvlist_lookup_string(propval, + ZPROP_VALUE); right_justify = B_FALSE; } @@ -3571,14 +3530,14 @@ print_dataset(zfs_handle_t *zhp, list_cbdata_t *cb) * format specifier. */ if (cb->cb_scripted || (pl->pl_next == NULL && !right_justify)) - (void) printf("%s", propstr); + (void) fputs(propstr, stdout); else if (right_justify) (void) printf("%*s", (int)pl->pl_width, propstr); else (void) printf("%-*s", (int)pl->pl_width, propstr); } - (void) printf("\n"); + (void) putchar('\n'); } /* @@ -3604,13 +3563,12 @@ static int zfs_do_list(int argc, char **argv) { int c; - static char default_fields[] = + char default_fields[] = "name,used,available,referenced,mountpoint"; int types = ZFS_TYPE_DATASET; boolean_t types_specified = B_FALSE; - char *fields = NULL; + char *fields = default_fields; list_cbdata_t cb = { 0 }; - char *value; int limit = 0; int ret = 0; zfs_sort_column_t *sortcol = NULL; @@ -3655,36 +3613,29 @@ zfs_do_list(int argc, char **argv) types = 0; types_specified = B_TRUE; flags &= ~ZFS_ITER_PROP_LISTSNAPS; - while (*optarg != '\0') { - static char *type_subopts[] = { "filesystem", - "volume", "snapshot", "snap", "bookmark", - "all", NULL }; - - switch (getsubopt(&optarg, type_subopts, - &value)) { - case 0: - types |= ZFS_TYPE_FILESYSTEM; - break; - case 1: - types |= ZFS_TYPE_VOLUME; - break; - case 2: - case 3: - types |= ZFS_TYPE_SNAPSHOT; - break; - case 4: - types |= ZFS_TYPE_BOOKMARK; - break; - case 5: - types = ZFS_TYPE_DATASET | - ZFS_TYPE_BOOKMARK; - break; - default: - (void) fprintf(stderr, - gettext("invalid type '%s'\n"), - value); - usage(B_FALSE); - } + + for (char *tok; (tok = strsep(&optarg, ",")); ) { + static const char *const type_subopts[] = { + "filesystem", "volume", + "snapshot", "snap", + "bookmark", + "all" }; + static const int type_types[] = { + ZFS_TYPE_FILESYSTEM, ZFS_TYPE_VOLUME, + ZFS_TYPE_SNAPSHOT, ZFS_TYPE_SNAPSHOT, + ZFS_TYPE_BOOKMARK, + ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK }; + + for (c = 0; c < ARRAY_SIZE(type_subopts); ++c) + if (strcmp(tok, type_subopts[c]) == 0) { + types |= type_types[c]; + goto found3; + } + + (void) fprintf(stderr, + gettext("invalid type '%s'\n"), tok); + usage(B_FALSE); +found3:; } break; case ':': @@ -3702,15 +3653,15 @@ zfs_do_list(int argc, char **argv) argc -= optind; argv += optind; - if (fields == NULL) - fields = default_fields; - /* - * If we are only going to list snapshot names and sort by name, - * then we can use faster version. + * If we are only going to list snapshot names and sort by name or + * by createtxg, then we can use faster version. */ - if (strcmp(fields, "name") == 0 && zfs_sort_only_by_name(sortcol)) + if (strcmp(fields, "name") == 0 && + (zfs_sort_only_by_name(sortcol) || + zfs_sort_only_by_createtxg(sortcol))) { flags |= ZFS_ITER_SIMPLE; + } /* * If "-o space" and no types were specified, don't display snapshots. @@ -4381,6 +4332,31 @@ zfs_do_snapshot(int argc, char **argv) return (-1); } +/* + * Array of prefixes to exclude – + * a linear search, even if executed for each dataset, + * is plenty good enough. + */ +typedef struct zfs_send_exclude_arg { + size_t count; + const char **list; +} zfs_send_exclude_arg_t; + +static boolean_t +zfs_do_send_exclude(zfs_handle_t *zhp, void *context) +{ + zfs_send_exclude_arg_t *excludes = context; + const char *name = zfs_get_name(zhp); + + for (size_t i = 0; i < excludes->count; ++i) { + size_t len = strlen(excludes->list[i]); + if (strncmp(name, excludes->list[i], len) == 0 && + memchr("/@", name[len], sizeof ("/@"))) + return (B_FALSE); + } + + return (B_TRUE); +} /* * Send a backup stream to stdout. @@ -4397,10 +4373,11 @@ zfs_do_send(int argc, char **argv) int c, err; nvlist_t *dbgnv = NULL; char *redactbook = NULL; + zfs_send_exclude_arg_t excludes = { 0 }; struct option long_options[] = { {"replicate", no_argument, NULL, 'R'}, - {"skip-missing", no_argument, NULL, 's'}, + {"skip-missing", no_argument, NULL, 's'}, {"redact", required_argument, NULL, 'd'}, {"props", no_argument, NULL, 'p'}, {"parsable", no_argument, NULL, 'P'}, @@ -4415,13 +4392,28 @@ zfs_do_send(int argc, char **argv) {"backup", no_argument, NULL, 'b'}, {"holds", no_argument, NULL, 'h'}, {"saved", no_argument, NULL, 'S'}, + {"exclude", required_argument, NULL, 'X'}, {0, 0, 0, 0} }; /* check options */ - while ((c = getopt_long(argc, argv, ":i:I:RsDpvnPLeht:cwbd:S", + while ((c = getopt_long(argc, argv, ":i:I:RsDpvnPLeht:cwbd:SX:", long_options, NULL)) != -1) { switch (c) { + case 'X': + for (char *ds; (ds = strsep(&optarg, ",")) != NULL; ) { + if (!zfs_name_valid(ds, ZFS_TYPE_DATASET) || + strchr(ds, '/') == NULL) { + (void) fprintf(stderr, gettext("-X %s: " + "not a valid non-root dataset name" + ".\n"), ds); + usage(B_FALSE); + } + excludes.list = safe_realloc(excludes.list, + sizeof (char *) * (excludes.count + 1)); + excludes.list[excludes.count++] = ds; + } + break; case 'i': if (fromname) usage(B_FALSE); @@ -4531,6 +4523,13 @@ zfs_do_send(int argc, char **argv) if (flags.parsable && flags.verbosity == 0) flags.verbosity = 1; + if (excludes.count > 0 && !flags.replicate) { + (void) fprintf(stderr, gettext("Cannot specify " + "dataset exclusion (-X) on a non-recursive " + "send.\n")); + return (1); + } + argc -= optind; argv += optind; @@ -4595,16 +4594,11 @@ zfs_do_send(int argc, char **argv) err = zfs_send_saved(zhp, &flags, STDOUT_FILENO, resume_token); - if (err != 0) - note_dev_error(errno, STDOUT_FILENO); zfs_close(zhp); return (err != 0); } else if (resume_token != NULL) { - err = zfs_send_resume(g_zfs, &flags, STDOUT_FILENO, - resume_token); - if (err != 0) - note_dev_error(errno, STDOUT_FILENO); - return (err); + return (zfs_send_resume(g_zfs, &flags, STDOUT_FILENO, + resume_token)); } if (flags.skipmissing && !flags.replicate) { @@ -4655,8 +4649,6 @@ zfs_do_send(int argc, char **argv) err = zfs_send_one(zhp, fromname, STDOUT_FILENO, &flags, redactbook); zfs_close(zhp); - if (err != 0) - note_dev_error(errno, STDOUT_FILENO); return (err != 0); } @@ -4719,8 +4711,9 @@ zfs_do_send(int argc, char **argv) if (flags.replicate && fromname == NULL) flags.doall = B_TRUE; - err = zfs_send(zhp, fromname, toname, &flags, STDOUT_FILENO, NULL, 0, - flags.verbosity >= 3 ? &dbgnv : NULL); + err = zfs_send(zhp, fromname, toname, &flags, STDOUT_FILENO, + excludes.count > 0 ? zfs_do_send_exclude : NULL, + &excludes, flags.verbosity >= 3 ? &dbgnv : NULL); if (flags.verbosity >= 3 && dbgnv != NULL) { /* @@ -4732,9 +4725,9 @@ zfs_do_send(int argc, char **argv) dump_nvlist(dbgnv, 0); nvlist_free(dbgnv); } - zfs_close(zhp); - note_dev_error(errno, STDOUT_FILENO); + zfs_close(zhp); + free(excludes.list); return (err != 0); } @@ -4753,7 +4746,7 @@ zfs_do_receive(int argc, char **argv) nomem(); /* check options */ - while ((c = getopt(argc, argv, ":o:x:dehMnuvFsA")) != -1) { + while ((c = getopt(argc, argv, ":o:x:dehMnuvFsAc")) != -1) { switch (c) { case 'o': if (!parseprop(props, optarg)) { @@ -4809,6 +4802,9 @@ zfs_do_receive(int argc, char **argv) case 'A': abort_resumable = B_TRUE; break; + case 'c': + flags.heal = B_TRUE; + break; case ':': (void) fprintf(stderr, gettext("missing argument for " "'%c' option\n"), optopt); @@ -5138,7 +5134,7 @@ deleg_perm_compare(const void *larg, const void *rarg, void *unused) static inline void fs_perm_set_init(fs_perm_set_t *fspset) { - bzero(fspset, sizeof (fs_perm_set_t)); + memset(fspset, 0, sizeof (fs_perm_set_t)); if ((fspset->fsps_list_pool = uu_list_pool_create("fsps_list_pool", sizeof (fs_perm_node_t), offsetof(fs_perm_node_t, fspn_list_node), @@ -5205,7 +5201,7 @@ who_perm_init(who_perm_t *who_perm, fs_perm_t *fsperm, uu_avl_pool_t *pool; pool = fsperm->fsp_set->fsps_deleg_perm_avl_pool; - bzero(who_perm, sizeof (who_perm_t)); + memset(who_perm, 0, sizeof (who_perm_t)); if ((who_perm->who_deleg_perm_avl = uu_avl_create(pool, NULL, UU_DEFAULT)) == NULL) @@ -5239,7 +5235,7 @@ fs_perm_init(fs_perm_t *fsperm, fs_perm_set_t *fspset, const char *fsname) uu_avl_pool_t *nset_pool = fspset->fsps_named_set_avl_pool; uu_avl_pool_t *who_pool = fspset->fsps_who_perm_avl_pool; - bzero(fsperm, sizeof (fs_perm_t)); + memset(fsperm, 0, sizeof (fs_perm_t)); if ((fsperm->fsp_sc_avl = uu_avl_create(nset_pool, NULL, UU_DEFAULT)) == NULL) @@ -5458,8 +5454,6 @@ parse_fs_perm_set(fs_perm_set_t *fspset, nvlist_t *nvl) data_type_t type = nvpair_type(nvp); fs_perm_t *fsperm = NULL; fs_perm_node_t *node = safe_malloc(sizeof (fs_perm_node_t)); - if (node == NULL) - nomem(); fsperm = &node->fspn_fsperm; @@ -6429,7 +6423,7 @@ print_holds(boolean_t scripted, int nwidth, int tagwidth, nvlist_t *nvl) { int i; nvpair_t *nvp = NULL; - char *hdr_cols[] = { "NAME", "TAG", "TIMESTAMP" }; + const char *const hdr_cols[] = { "NAME", "TAG", "TIMESTAMP" }; const char *col; if (!scripted) { @@ -6450,7 +6444,7 @@ print_holds(boolean_t scripted, int nwidth, int tagwidth, nvlist_t *nvl) (void) nvpair_value_nvlist(nvp, &nvl2); while ((nvp2 = nvlist_next_nvpair(nvl2, nvp2)) != NULL) { char tsbuf[DATETIME_BUF_LEN]; - char *tagname = nvpair_name(nvp2); + const char *tagname = nvpair_name(nvp2); uint64_t val = 0; time_t time; struct tm t; @@ -6521,13 +6515,10 @@ holds_callback(zfs_handle_t *zhp, void *data) static int zfs_do_holds(int argc, char **argv) { - int errors = 0; int c; - int i; + boolean_t errors = B_FALSE; boolean_t scripted = B_FALSE; boolean_t recursive = B_FALSE; - const char *opts = "rH"; - nvlist_t *nvl; int types = ZFS_TYPE_SNAPSHOT; holds_cbdata_t cb = { 0 }; @@ -6537,7 +6528,7 @@ zfs_do_holds(int argc, char **argv) int flags = 0; /* check options */ - while ((c = getopt(argc, argv, opts)) != -1) { + while ((c = getopt(argc, argv, "rH")) != -1) { switch (c) { case 'r': recursive = B_TRUE; @@ -6564,10 +6555,9 @@ zfs_do_holds(int argc, char **argv) if (argc < 1) usage(B_FALSE); - if (nvlist_alloc(&nvl, NV_UNIQUE_NAME, 0) != 0) - nomem(); + nvlist_t *nvl = fnvlist_alloc(); - for (i = 0; i < argc; ++i) { + for (int i = 0; i < argc; ++i) { char *snapshot = argv[i]; const char *delim; const char *snapname; @@ -6576,7 +6566,7 @@ zfs_do_holds(int argc, char **argv) if (delim == NULL) { (void) fprintf(stderr, gettext("'%s' is not a snapshot\n"), snapshot); - ++errors; + errors = B_TRUE; continue; } snapname = delim + 1; @@ -6590,10 +6580,10 @@ zfs_do_holds(int argc, char **argv) /* * 1. collect holds data, set format options */ - ret = zfs_for_each(argc, argv, flags, types, NULL, NULL, limit, + ret = zfs_for_each(1, argv + i, flags, types, NULL, NULL, limit, holds_callback, &cb); if (ret != 0) - ++errors; + errors = B_TRUE; } /* @@ -6606,7 +6596,7 @@ zfs_do_holds(int argc, char **argv) nvlist_free(nvl); - return (0 != errors); + return (errors); } #define CHECK_SPINNER 30 @@ -6621,7 +6611,7 @@ typedef struct get_all_state { static int get_one_dataset(zfs_handle_t *zhp, void *data) { - static char *spin[] = { "-", "\\", "|", "/" }; + static const char *const spin[] = { "-", "\\", "|", "/" }; static int spinval = 0; static int spincheck = 0; static time_t last_spin_time = (time_t)0; @@ -6688,7 +6678,7 @@ typedef struct share_mount_state { boolean_t sm_verbose; int sm_flags; char *sm_options; - char *sm_proto; /* only valid for OP_SHARE */ + enum sa_protocol sm_proto; /* only valid for OP_SHARE */ pthread_mutex_t sm_lock; /* protects the remaining fields */ uint_t sm_total; /* number of filesystems to process */ uint_t sm_done; /* number of filesystems processed */ @@ -6699,7 +6689,7 @@ typedef struct share_mount_state { * Share or mount a dataset. */ static int -share_mount_one(zfs_handle_t *zhp, int op, int flags, char *protocol, +share_mount_one(zfs_handle_t *zhp, int op, int flags, enum sa_protocol protocol, boolean_t explicit, const char *options) { char mountpoint[ZFS_MAXPROPLEN]; @@ -6817,7 +6807,7 @@ share_mount_one(zfs_handle_t *zhp, int op, int flags, char *protocol, return (0); if (op == OP_SHARE && !zfs_is_mounted(zhp, NULL)) { /* also purge it from existing exports */ - zfs_unshareall_bypath(zhp, mountpoint); + zfs_unshare(zhp, mountpoint, NULL); return (0); } } @@ -6875,10 +6865,11 @@ share_mount_one(zfs_handle_t *zhp, int op, int flags, char *protocol, * filesystem. */ switch (op) { - case OP_SHARE: - - shared_nfs = zfs_is_shared_nfs(zhp, NULL); - shared_smb = zfs_is_shared_smb(zhp, NULL); + case OP_SHARE: { + enum sa_protocol prot[] = {SA_PROTOCOL_NFS, SA_NO_PROTOCOL}; + shared_nfs = zfs_is_shared(zhp, NULL, prot); + *prot = SA_PROTOCOL_SMB; + shared_smb = zfs_is_shared(zhp, NULL, prot); if ((shared_nfs && shared_smb) || (shared_nfs && strcmp(shareopts, "on") == 0 && @@ -6898,30 +6889,15 @@ share_mount_one(zfs_handle_t *zhp, int op, int flags, char *protocol, zfs_mount(zhp, NULL, flags) != 0) return (1); - if (protocol == NULL) { - if (zfs_shareall(zhp) != 0) - return (1); - } else if (strcmp(protocol, "nfs") == 0) { - if (zfs_share_nfs(zhp)) - return (1); - } else if (strcmp(protocol, "smb") == 0) { - if (zfs_share_smb(zhp)) - return (1); - } else { - (void) fprintf(stderr, gettext("cannot share " - "'%s': invalid share type '%s' " - "specified\n"), - zfs_get_name(zhp), protocol); + *prot = protocol; + if (zfs_share(zhp, protocol == SA_NO_PROTOCOL ? NULL : prot)) return (1); - } + } break; case OP_MOUNT: - if (options == NULL) - mnt.mnt_mntopts = ""; - else - mnt.mnt_mntopts = (char *)options; + mnt.mnt_mntopts = (char *)(options ?: ""); if (!hasmntopt(&mnt, MNTOPT_REMOUNT) && zfs_is_mounted(zhp, NULL)) { @@ -7012,6 +6988,22 @@ append_options(char *mntopts, char *newopts) (void) strcpy(&mntopts[len], newopts); } +static enum sa_protocol +sa_protocol_decode(const char *protocol) +{ + for (enum sa_protocol i = 0; i < ARRAY_SIZE(sa_protocol_names); ++i) + if (strcmp(protocol, sa_protocol_names[i]) == 0) + return (i); + + (void) fputs(gettext("share type must be one of: "), stderr); + for (enum sa_protocol i = 0; + i < ARRAY_SIZE(sa_protocol_names); ++i) + (void) fprintf(stderr, "%s%s", + i != 0 ? ", " : "", sa_protocol_names[i]); + (void) fputc('\n', stderr); + usage(B_FALSE); +} + static int share_mount(int op, int argc, char **argv) { @@ -7070,16 +7062,10 @@ share_mount(int op, int argc, char **argv) /* check number of arguments */ if (do_all) { - char *protocol = NULL; + enum sa_protocol protocol = SA_NO_PROTOCOL; if (op == OP_SHARE && argc > 0) { - if (strcmp(argv[0], "nfs") != 0 && - strcmp(argv[0], "smb") != 0) { - (void) fprintf(stderr, gettext("share type " - "must be 'nfs' or 'smb'\n")); - usage(B_FALSE); - } - protocol = argv[0]; + protocol = sa_protocol_decode(argv[0]); argc--; argv++; } @@ -7116,7 +7102,7 @@ share_mount(int op, int argc, char **argv) zfs_foreach_mountpoint(g_zfs, cb.cb_handles, cb.cb_used, share_mount_one_cb, &share_mount_state, op == OP_MOUNT && !(flags & MS_CRYPT)); - zfs_commit_all_shares(); + zfs_commit_shares(NULL); ret = share_mount_state.sm_status; @@ -7168,9 +7154,9 @@ share_mount(int op, int argc, char **argv) ZFS_TYPE_FILESYSTEM)) == NULL) { ret = 1; } else { - ret = share_mount_one(zhp, op, flags, NULL, B_TRUE, - options); - zfs_commit_all_shares(); + ret = share_mount_one(zhp, op, flags, SA_NO_PROTOCOL, + B_TRUE, options); + zfs_commit_shares(NULL); zfs_close(zhp); } } @@ -7180,7 +7166,7 @@ share_mount(int op, int argc, char **argv) } /* - * zfs mount -a [nfs] + * zfs mount -a * zfs mount filesystem * * Mount all filesystems, or mount the given filesystem. @@ -7289,12 +7275,12 @@ unshare_unmount_path(int op, char *path, int flags, boolean_t is_manual) "'%s': legacy share\n"), path); (void) fprintf(stderr, gettext("use exportfs(8) " "or smbcontrol(1) to unshare this filesystem\n")); - } else if (!zfs_is_shared(zhp)) { + } else if (!zfs_is_shared(zhp, NULL, NULL)) { (void) fprintf(stderr, gettext("cannot unshare '%s': " "not currently shared\n"), path); } else { - ret = zfs_unshareall_bypath(zhp, path); - zfs_commit_all_shares(); + ret = zfs_unshare(zhp, path, NULL); + zfs_commit_shares(NULL); } } else { char mtpt_prop[ZFS_MAXPROPLEN]; @@ -7384,16 +7370,12 @@ unshare_unmount(int op, int argc, char **argv) unshare_unmount_node_t *node; uu_avl_index_t idx; uu_avl_walk_t *walk; - char *protocol = NULL; + enum sa_protocol *protocol = NULL, + single_protocol[] = {SA_NO_PROTOCOL, SA_NO_PROTOCOL}; if (op == OP_SHARE && argc > 0) { - if (strcmp(argv[0], "nfs") != 0 && - strcmp(argv[0], "smb") != 0) { - (void) fprintf(stderr, gettext("share type " - "must be 'nfs' or 'smb'\n")); - usage(B_FALSE); - } - protocol = argv[0]; + *single_protocol = sa_protocol_decode(argv[0]); + protocol = single_protocol; argc--; argv++; } @@ -7500,7 +7482,7 @@ unshare_unmount(int op, int argc, char **argv) uu_avl_remove(tree, node); switch (op) { case OP_SHARE: - if (zfs_unshareall_bytype(node->un_zhp, + if (zfs_unshare(node->un_zhp, node->un_mountp, protocol) != 0) ret = 1; break; @@ -7573,12 +7555,12 @@ unshare_unmount(int op, int argc, char **argv) "exports(5) or smb.conf(5) to unshare " "this filesystem\n")); ret = 1; - } else if (!zfs_is_shared(zhp)) { + } else if (!zfs_is_shared(zhp, NULL, NULL)) { (void) fprintf(stderr, gettext("cannot " "unshare '%s': not currently " "shared\n"), zfs_get_name(zhp)); ret = 1; - } else if (zfs_unshareall(zhp) != 0) { + } else if (zfs_unshareall(zhp, NULL) != 0) { ret = 1; } break; @@ -7635,7 +7617,7 @@ zfs_do_unshare(int argc, char **argv) } static int -find_command_idx(char *command, int *idx) +find_command_idx(const char *command, int *idx) { int i; @@ -7900,7 +7882,6 @@ static int zfs_do_channel_program(int argc, char **argv) { int ret, fd, c; - char *progbuf, *filename, *poolname; size_t progsize, progread; nvlist_t *outnvl = NULL; uint64_t instrlimit = ZCP_DEFAULT_INSTRLIMIT; @@ -7957,8 +7938,8 @@ zfs_do_channel_program(int argc, char **argv) goto usage; } - poolname = argv[0]; - filename = argv[1]; + const char *poolname = argv[0]; + const char *filename = argv[1]; if (strcmp(filename, "-") == 0) { fd = 0; filename = "standard input"; @@ -7983,7 +7964,7 @@ zfs_do_channel_program(int argc, char **argv) */ progread = 0; progsize = 1024; - progbuf = safe_malloc(progsize); + char *progbuf = safe_malloc(progsize); do { ret = read(fd, progbuf + progread, progsize - progread); progread += ret; @@ -8029,14 +8010,17 @@ zfs_do_channel_program(int argc, char **argv) * exists. Otherwise, generate an appropriate error message, * falling back on strerror() for an unexpected return code. */ - char *errstring = NULL; + const char *errstring = NULL; const char *msg = gettext("Channel program execution failed"); uint64_t instructions = 0; if (outnvl != NULL && nvlist_exists(outnvl, ZCP_RET_ERROR)) { + char *es = NULL; (void) nvlist_lookup_string(outnvl, - ZCP_RET_ERROR, &errstring); - if (errstring == NULL) + ZCP_RET_ERROR, &es); + if (es == NULL) errstring = strerror(ret); + else + errstring = es; if (ret == ETIME) { (void) nvlist_lookup_uint64(outnvl, ZCP_ARG_INSTRLIMIT, &instructions); @@ -8537,27 +8521,25 @@ zfs_do_wait(int argc, char **argv) while ((c = getopt(argc, argv, "t:")) != -1) { switch (c) { case 't': - { - static char *col_subopts[] = { "deleteq", NULL }; - char *value; - /* Reset activities array */ - bzero(&enabled, sizeof (enabled)); - while (*optarg != '\0') { - int activity = getsubopt(&optarg, col_subopts, - &value); + memset(&enabled, 0, sizeof (enabled)); - if (activity < 0) { - (void) fprintf(stderr, - gettext("invalid activity '%s'\n"), - value); - usage(B_FALSE); - } + for (char *tok; (tok = strsep(&optarg, ",")); ) { + static const char *const col_subopts[ + ZFS_WAIT_NUM_ACTIVITIES] = { "deleteq" }; - enabled[activity] = B_TRUE; + for (i = 0; i < ARRAY_SIZE(col_subopts); ++i) + if (strcmp(tok, col_subopts[i]) == 0) { + enabled[i] = B_TRUE; + goto found; + } + + (void) fprintf(stderr, + gettext("invalid activity '%s'\n"), tok); + usage(B_FALSE); +found:; } break; - } case '?': (void) fprintf(stderr, gettext("invalid option '%c'\n"), optopt); @@ -8614,11 +8596,7 @@ static int zfs_do_version(int argc, char **argv) { (void) argc, (void) argv; - - if (zfs_version_print() == -1) - return (1); - - return (0); + return (zfs_version_print() != 0); } int @@ -8626,7 +8604,7 @@ main(int argc, char **argv) { int ret = 0; int i = 0; - char *cmdname; + const char *cmdname; char **newargv; (void) setlocale(LC_ALL, ""); @@ -8689,7 +8667,7 @@ main(int argc, char **argv) * Many commands modify input strings for string parsing reasons. * We create a copy to protect the original argv. */ - newargv = malloc((argc + 1) * sizeof (newargv[0])); + newargv = safe_malloc((argc + 1) * sizeof (newargv[0])); for (i = 0; i < argc; i++) newargv[i] = strdup(argv[i]); newargv[argc] = NULL; @@ -8733,6 +8711,50 @@ main(int argc, char **argv) return (ret); } +/* + * zfs zone nsfile filesystem + * + * Add or delete the given dataset to/from the namespace. + */ +#ifdef __linux__ +static int +zfs_do_zone_impl(int argc, char **argv, boolean_t attach) +{ + zfs_handle_t *zhp; + int ret; + + if (argc < 3) { + (void) fprintf(stderr, gettext("missing argument(s)\n")); + usage(B_FALSE); + } + if (argc > 3) { + (void) fprintf(stderr, gettext("too many arguments\n")); + usage(B_FALSE); + } + + zhp = zfs_open(g_zfs, argv[2], ZFS_TYPE_FILESYSTEM); + if (zhp == NULL) + return (1); + + ret = (zfs_userns(zhp, argv[1], attach) != 0); + + zfs_close(zhp); + return (ret); +} + +static int +zfs_do_zone(int argc, char **argv) +{ + return (zfs_do_zone_impl(argc, argv, B_TRUE)); +} + +static int +zfs_do_unzone(int argc, char **argv) +{ + return (zfs_do_zone_impl(argc, argv, B_FALSE)); +} +#endif + #ifdef __FreeBSD__ #include #include diff --git a/cmd/zfs/zfs_project.c b/cmd/zfs/zfs_project.c index 341cc005de48..8a010e103bef 100644 --- a/cmd/zfs/zfs_project.c +++ b/cmd/zfs/zfs_project.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -27,7 +27,7 @@ #include #include #include -#include +#include #include #include #include @@ -207,7 +207,6 @@ static int zfs_project_handle_dir(const char *name, zfs_project_control_t *zpc, list_t *head) { - char fullname[PATH_MAX]; struct dirent *ent; DIR *dir; int ret = 0; @@ -227,21 +226,28 @@ zfs_project_handle_dir(const char *name, zfs_project_control_t *zpc, zpc->zpc_ignore_noent = B_TRUE; errno = 0; while (!ret && (ent = readdir(dir)) != NULL) { + char *fullname; + /* skip "." and ".." */ if (strcmp(ent->d_name, ".") == 0 || strcmp(ent->d_name, "..") == 0) continue; - if (strlen(ent->d_name) + strlen(name) >= - sizeof (fullname) + 1) { + if (strlen(ent->d_name) + strlen(name) + 1 >= PATH_MAX) { errno = ENAMETOOLONG; break; } - sprintf(fullname, "%s/%s", name, ent->d_name); + if (asprintf(&fullname, "%s/%s", name, ent->d_name) == -1) { + errno = ENOMEM; + break; + } + ret = zfs_project_handle_one(fullname, zpc); if (!ret && zpc->zpc_recursive && ent->d_type == DT_DIR) zfs_project_item_alloc(head, fullname); + + free(fullname); } if (errno && !ret) { diff --git a/cmd/zfs/zfs_projectutil.h b/cmd/zfs/zfs_projectutil.h index 1792a3383a03..e4038d885cdd 100644 --- a/cmd/zfs/zfs_projectutil.h +++ b/cmd/zfs/zfs_projectutil.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/cmd/zfs/zfs_util.h b/cmd/zfs/zfs_util.h index a56af59adb15..07ade72dd28c 100644 --- a/cmd/zfs/zfs_util.h +++ b/cmd/zfs/zfs_util.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -31,7 +31,7 @@ extern "C" { #endif -void * safe_malloc(size_t size); +void *safe_malloc(size_t size); void nomem(void); extern libzfs_handle_t *g_zfs; diff --git a/cmd/zfs_ids_to_path/zfs_ids_to_path.c b/cmd/zfs_ids_to_path.c similarity index 98% rename from cmd/zfs_ids_to_path/zfs_ids_to_path.c rename to cmd/zfs_ids_to_path.c index 1d3bb6b29ee1..4db7e949ead0 100644 --- a/cmd/zfs_ids_to_path/zfs_ids_to_path.c +++ b/cmd/zfs_ids_to_path.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/cmd/zfs_ids_to_path/.gitignore b/cmd/zfs_ids_to_path/.gitignore deleted file mode 100644 index f95f853e48c2..000000000000 --- a/cmd/zfs_ids_to_path/.gitignore +++ /dev/null @@ -1 +0,0 @@ -zfs_ids_to_path diff --git a/cmd/zfs_ids_to_path/Makefile.am b/cmd/zfs_ids_to_path/Makefile.am deleted file mode 100644 index 549426764026..000000000000 --- a/cmd/zfs_ids_to_path/Makefile.am +++ /dev/null @@ -1,11 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -sbin_PROGRAMS = zfs_ids_to_path - -zfs_ids_to_path_SOURCES = \ - zfs_ids_to_path.c - -zfs_ids_to_path_LDADD = \ - $(abs_top_builddir)/lib/libzfs/libzfs.la - -include $(top_srcdir)/config/CppCheck.am diff --git a/cmd/zgenhostid/zgenhostid.c b/cmd/zgenhostid.c similarity index 98% rename from cmd/zgenhostid/zgenhostid.c rename to cmd/zgenhostid.c index 853931c6ad6e..04433b0d973c 100644 --- a/cmd/zgenhostid/zgenhostid.c +++ b/cmd/zgenhostid.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/cmd/zgenhostid/.gitignore b/cmd/zgenhostid/.gitignore deleted file mode 100644 index 072246c735ba..000000000000 --- a/cmd/zgenhostid/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/zgenhostid diff --git a/cmd/zgenhostid/Makefile.am b/cmd/zgenhostid/Makefile.am deleted file mode 100644 index 4526a90a1d51..000000000000 --- a/cmd/zgenhostid/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -sbin_PROGRAMS = zgenhostid - -zgenhostid_SOURCES = zgenhostid.c - -include $(top_srcdir)/config/CppCheck.am diff --git a/cmd/zhack/zhack.c b/cmd/zhack.c similarity index 97% rename from cmd/zhack/zhack.c rename to cmd/zhack.c index 73ce888c0b1d..75d9c93b3649 100644 --- a/cmd/zhack/zhack.c +++ b/cmd/zhack.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -87,8 +87,8 @@ usage(void) } -static __attribute__((noreturn)) __attribute__((format(printf, 3, 4))) void -fatal(spa_t *spa, void *tag, const char *fmt, ...) +static __attribute__((format(printf, 3, 4))) __attribute__((noreturn)) void +fatal(spa_t *spa, const void *tag, const char *fmt, ...) { va_list ap; @@ -166,7 +166,7 @@ zhack_import(char *target, boolean_t readonly) } static void -zhack_spa_open(char *target, boolean_t readonly, void *tag, spa_t **spa) +zhack_spa_open(char *target, boolean_t readonly, const void *tag, spa_t **spa) { int err; @@ -484,15 +484,12 @@ zhack_repair_label_cksum(int argc, char **argv) zio_checksum_info_t *ci = &zio_checksum_table[ZIO_CHECKSUM_LABEL]; const char *cfg_keys[] = { ZPOOL_CONFIG_VERSION, ZPOOL_CONFIG_POOL_STATE, ZPOOL_CONFIG_GUID }; - boolean_t labels_repaired[VDEV_LABELS]; + boolean_t labels_repaired[VDEV_LABELS] = {0}; boolean_t repaired = B_FALSE; - vdev_label_t labels[VDEV_LABELS]; + vdev_label_t labels[VDEV_LABELS] = {{{0}}}; struct stat st; int fd; - bzero(labels_repaired, sizeof (labels_repaired)); - bzero(labels, sizeof (labels)); - abd_init(); argc -= 1; diff --git a/cmd/zhack/.gitignore b/cmd/zhack/.gitignore deleted file mode 100644 index 763a18898b88..000000000000 --- a/cmd/zhack/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/zhack diff --git a/cmd/zhack/Makefile.am b/cmd/zhack/Makefile.am deleted file mode 100644 index 23f03ffd8243..000000000000 --- a/cmd/zhack/Makefile.am +++ /dev/null @@ -1,16 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -# Unconditionally enable debugging for zhack -AM_CPPFLAGS += -DDEBUG -UNDEBUG -DZFS_DEBUG - -sbin_PROGRAMS = zhack - -zhack_SOURCES = \ - zhack.c - -zhack_LDADD = \ - $(abs_top_builddir)/lib/libzpool/libzpool.la \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la - -include $(top_srcdir)/config/CppCheck.am diff --git a/cmd/zinject/.gitignore b/cmd/zinject/.gitignore deleted file mode 100644 index bded8400996c..000000000000 --- a/cmd/zinject/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/zinject diff --git a/cmd/zinject/Makefile.am b/cmd/zinject/Makefile.am index 40f382c66191..c90f73fc0165 100644 --- a/cmd/zinject/Makefile.am +++ b/cmd/zinject/Makefile.am @@ -1,15 +1,12 @@ -include $(top_srcdir)/config/Rules.am - -sbin_PROGRAMS = zinject +sbin_PROGRAMS += zinject +CPPCHECKTARGETS += zinject zinject_SOURCES = \ - translate.c \ - zinject.c \ - zinject.h + %D%/translate.c \ + %D%/zinject.c \ + %D%/zinject.h zinject_LDADD = \ - $(abs_top_builddir)/lib/libzfs/libzfs.la \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la - -include $(top_srcdir)/config/CppCheck.am + libzfs.la \ + libzfs_core.la \ + libnvpair.la diff --git a/cmd/zinject/translate.c b/cmd/zinject/translate.c index 4939c0b85b5f..cd4264bdc084 100644 --- a/cmd/zinject/translate.c +++ b/cmd/zinject/translate.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -31,7 +31,7 @@ #include #include #include -#include +#include #include #include #include @@ -186,6 +186,7 @@ initialize_range(err_type_t type, int level, char *range, switch (type) { default: break; + case TYPE_DATA: break; diff --git a/cmd/zinject/zinject.c b/cmd/zinject/zinject.c index 10d5dbe90044..f1262ed772de 100644 --- a/cmd/zinject/zinject.c +++ b/cmd/zinject/zinject.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -144,6 +144,7 @@ #include #include #include +#include #include #include @@ -159,7 +160,7 @@ libzfs_handle_t *g_zfs; int zfs_fd; -static const char *errtable[TYPE_INVAL] = { +static const char *const errtable[TYPE_INVAL] = { "data", "dnode", "mos", diff --git a/cmd/zinject/zinject.h b/cmd/zinject/zinject.h index 46fdcad8b31f..73176a24195a 100644 --- a/cmd/zinject/zinject.h +++ b/cmd/zinject/zinject.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/cmd/zpool/.gitignore b/cmd/zpool/.gitignore deleted file mode 100644 index 8ea518af78e5..000000000000 --- a/cmd/zpool/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/zpool diff --git a/cmd/zpool/Makefile.am b/cmd/zpool/Makefile.am index 7ea7c5fc3f3e..3c7c8a9aebe2 100644 --- a/cmd/zpool/Makefile.am +++ b/cmd/zpool/Makefile.am @@ -1,34 +1,35 @@ -include $(top_srcdir)/config/Rules.am -include $(top_srcdir)/config/Shellcheck.am +zpool_CFLAGS = $(AM_CFLAGS) +zpool_CFLAGS += $(LIBBLKID_CFLAGS) $(LIBUUID_CFLAGS) -AM_CFLAGS += $(LIBBLKID_CFLAGS) $(LIBUUID_CFLAGS) +zpool_CPPFLAGS = $(AM_CPPFLAGS) +zpool_CPPFLAGS += -I$(srcdir)/%D% -DEFAULT_INCLUDES += -I$(srcdir) - - -sbin_PROGRAMS = zpool +sbin_PROGRAMS += zpool +CPPCHECKTARGETS += zpool zpool_SOURCES = \ - zpool_iter.c \ - zpool_main.c \ - zpool_util.c \ - zpool_util.h \ - zpool_vdev.c + %D%/zpool_iter.c \ + %D%/zpool_main.c \ + %D%/zpool_util.c \ + %D%/zpool_util.h \ + %D%/zpool_vdev.c if BUILD_FREEBSD -zpool_SOURCES += os/freebsd/zpool_vdev_os.c +zpool_SOURCES += \ + %D%/os/freebsd/zpool_vdev_os.c endif if BUILD_LINUX -zpool_SOURCES += os/linux/zpool_vdev_os.c +zpool_SOURCES += \ + %D%/os/linux/zpool_vdev_os.c endif zpool_LDADD = \ - $(abs_top_builddir)/lib/libzfs/libzfs.la \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la \ - $(abs_top_builddir)/lib/libuutil/libuutil.la \ - $(abs_top_builddir)/lib/libzutil/libzutil.la + libzfs.la \ + libzfs_core.la \ + libnvpair.la \ + libuutil.la \ + libzutil.la zpool_LDADD += $(LTLIBINTL) @@ -37,56 +38,53 @@ zpool_LDADD += -lgeom endif zpool_LDADD += -lm $(LIBBLKID_LIBS) $(LIBUUID_LIBS) -include $(top_srcdir)/config/CppCheck.am +dist_noinst_DATA += %D%/zpool.d/README -zpoolconfdir = $(sysconfdir)/zfs/zpool.d +SHELLCHECKSCRIPTS += $(dist_zpoolexec_SCRIPTS) zpoolexecdir = $(zfsexecdir)/zpool.d - -EXTRA_DIST = zpool.d/README compatibility.d - dist_zpoolexec_SCRIPTS = \ - zpool.d/dm-deps \ - zpool.d/enc \ - zpool.d/encdev \ - zpool.d/fault_led \ - zpool.d/iostat \ - zpool.d/iostat-1s \ - zpool.d/iostat-10s \ - zpool.d/label \ - zpool.d/locate_led \ - zpool.d/lsblk \ - zpool.d/media \ - zpool.d/model \ - zpool.d/serial \ - zpool.d/ses \ - zpool.d/size \ - zpool.d/slot \ - zpool.d/smart \ - zpool.d/smartx \ - zpool.d/temp \ - zpool.d/health \ - zpool.d/r_proc \ - zpool.d/w_proc \ - zpool.d/r_ucor \ - zpool.d/w_ucor \ - zpool.d/nonmed \ - zpool.d/defect \ - zpool.d/hours_on \ - zpool.d/realloc \ - zpool.d/rep_ucor \ - zpool.d/cmd_to \ - zpool.d/pend_sec \ - zpool.d/off_ucor \ - zpool.d/ata_err \ - zpool.d/nvme_err \ - zpool.d/pwr_cyc \ - zpool.d/upath \ - zpool.d/vendor \ - zpool.d/smart_test \ - zpool.d/test_type \ - zpool.d/test_status \ - zpool.d/test_progress \ - zpool.d/test_ended + %D%/zpool.d/ata_err \ + %D%/zpool.d/cmd_to \ + %D%/zpool.d/defect \ + %D%/zpool.d/dm-deps \ + %D%/zpool.d/enc \ + %D%/zpool.d/encdev \ + %D%/zpool.d/fault_led \ + %D%/zpool.d/health \ + %D%/zpool.d/hours_on \ + %D%/zpool.d/iostat \ + %D%/zpool.d/iostat-10s \ + %D%/zpool.d/iostat-1s \ + %D%/zpool.d/label \ + %D%/zpool.d/locate_led \ + %D%/zpool.d/lsblk \ + %D%/zpool.d/media \ + %D%/zpool.d/model \ + %D%/zpool.d/nonmed \ + %D%/zpool.d/nvme_err \ + %D%/zpool.d/off_ucor \ + %D%/zpool.d/pend_sec \ + %D%/zpool.d/pwr_cyc \ + %D%/zpool.d/r_proc \ + %D%/zpool.d/r_ucor \ + %D%/zpool.d/realloc \ + %D%/zpool.d/rep_ucor \ + %D%/zpool.d/serial \ + %D%/zpool.d/ses \ + %D%/zpool.d/size \ + %D%/zpool.d/slot \ + %D%/zpool.d/smart \ + %D%/zpool.d/smart_test \ + %D%/zpool.d/smartx \ + %D%/zpool.d/temp \ + %D%/zpool.d/test_ended \ + %D%/zpool.d/test_progress \ + %D%/zpool.d/test_status \ + %D%/zpool.d/test_type \ + %D%/zpool.d/upath \ + %D%/zpool.d/vendor \ + %D%/zpool.d/w_proc \ + %D%/zpool.d/w_ucor zpoolconfdefaults = \ dm-deps \ @@ -133,29 +131,28 @@ zpoolconfdefaults = \ test_ended zpoolcompatdir = $(pkgdatadir)/compatibility.d - dist_zpoolcompat_DATA = \ - compatibility.d/compat-2018 \ - compatibility.d/compat-2019 \ - compatibility.d/compat-2020 \ - compatibility.d/compat-2021 \ - compatibility.d/freebsd-11.0 \ - compatibility.d/freebsd-11.2 \ - compatibility.d/freebsd-11.3 \ - compatibility.d/freenas-9.10.2 \ - compatibility.d/grub2 \ - compatibility.d/openzfsonosx-1.7.0 \ - compatibility.d/openzfsonosx-1.8.1 \ - compatibility.d/openzfsonosx-1.9.3 \ - compatibility.d/openzfs-2.0-freebsd \ - compatibility.d/openzfs-2.0-linux \ - compatibility.d/openzfs-2.1-freebsd \ - compatibility.d/openzfs-2.1-linux \ - compatibility.d/zol-0.6.1 \ - compatibility.d/zol-0.6.4 \ - compatibility.d/zol-0.6.5 \ - compatibility.d/zol-0.7 \ - compatibility.d/zol-0.8 + %D%/compatibility.d/compat-2018 \ + %D%/compatibility.d/compat-2019 \ + %D%/compatibility.d/compat-2020 \ + %D%/compatibility.d/compat-2021 \ + %D%/compatibility.d/freebsd-11.0 \ + %D%/compatibility.d/freebsd-11.2 \ + %D%/compatibility.d/freebsd-11.3 \ + %D%/compatibility.d/freenas-9.10.2 \ + %D%/compatibility.d/grub2 \ + %D%/compatibility.d/openzfs-2.0-freebsd \ + %D%/compatibility.d/openzfs-2.0-linux \ + %D%/compatibility.d/openzfs-2.1-freebsd \ + %D%/compatibility.d/openzfs-2.1-linux \ + %D%/compatibility.d/openzfsonosx-1.7.0 \ + %D%/compatibility.d/openzfsonosx-1.8.1 \ + %D%/compatibility.d/openzfsonosx-1.9.3 \ + %D%/compatibility.d/zol-0.6.1 \ + %D%/compatibility.d/zol-0.6.4 \ + %D%/compatibility.d/zol-0.6.5 \ + %D%/compatibility.d/zol-0.7 \ + %D%/compatibility.d/zol-0.8 # canonical <- alias symbolic link pairs # eg: "2018" is a link to "compat-2018" @@ -178,13 +175,16 @@ zpoolcompatlinks = \ "zol-0.7 ubuntu-18.04" \ "zol-0.8 ubuntu-20.04" -install-data-hook: +zpoolconfdir = $(sysconfdir)/zfs/zpool.d +INSTALL_DATA_HOOKS += zpool-install-data-hook +zpool-install-data-hook: $(MKDIR_P) "$(DESTDIR)$(zpoolconfdir)" - for f in $(zpoolconfdefaults); do \ - test -f "$(DESTDIR)$(zpoolconfdir)/$${f}" -o \ - -L "$(DESTDIR)$(zpoolconfdir)/$${f}" || \ - ln -s "$(zpoolexecdir)/$${f}" "$(DESTDIR)$(zpoolconfdir)"; \ + set -x; for f in $(zpoolconfdefaults); do \ + [ -f "$(DESTDIR)$(zpoolconfdir)/$${f}" ] || \ + [ -L "$(DESTDIR)$(zpoolconfdir)/$${f}" ] || \ + $(LN_S) "$(zpoolexecdir)/$${f}" "$(DESTDIR)$(zpoolconfdir)"; \ done - for l in $(zpoolcompatlinks); do \ - (cd "$(DESTDIR)$(zpoolcompatdir)"; ln -sf $${l} ); \ + set -x; printf '%s\n' $(zpoolcompatlinks) | \ + while read -r canon alias; do \ + $(LN_S) -f "$${canon}" "$(DESTDIR)$(zpoolcompatdir)/$${alias}"; \ done diff --git a/cmd/zpool/os/freebsd/zpool_vdev_os.c b/cmd/zpool/os/freebsd/zpool_vdev_os.c index 8ebd3e60d5d6..231ca97f1f6f 100644 --- a/cmd/zpool/os/freebsd/zpool_vdev_os.c +++ b/cmd/zpool/os/freebsd/zpool_vdev_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/cmd/zpool/os/linux/zpool_vdev_os.c b/cmd/zpool/os/linux/zpool_vdev_os.c index 9d2c21c2a87b..7f4486e062fe 100644 --- a/cmd/zpool/os/linux/zpool_vdev_os.c +++ b/cmd/zpool/os/linux/zpool_vdev_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -81,7 +81,6 @@ #include #include #include -#include #include #include #include diff --git a/cmd/zpool/zpool.d/iostat b/cmd/zpool/zpool.d/iostat index 95c459a3f0bf..2f8d79af8926 100755 --- a/cmd/zpool/zpool.d/iostat +++ b/cmd/zpool/zpool.d/iostat @@ -15,6 +15,15 @@ if [ "$1" = "-h" ] ; then exit fi +# Sometimes, UPATH ends up /dev/(null). +# That should be corrected, but for now... +# shellcheck disable=SC2154 +if [ ! -b "$VDEV_UPATH" ]; then + somepath="${VDEV_PATH}" +else + somepath="${VDEV_UPATH}" +fi + if [ "$script" = "iostat-1s" ] ; then # Do a single one-second sample interval=1 @@ -27,8 +36,7 @@ elif [ "$script" = "iostat-10s" ] ; then brief="yes" fi -# shellcheck disable=SC2154 -if [ -f "$VDEV_UPATH" ] ; then +if [ -f "$somepath" ] ; then # We're a file-based vdev, iostat doesn't work on us. Do nothing. exit fi @@ -37,13 +45,13 @@ if [ "$(uname)" = "FreeBSD" ]; then out=$(iostat -dKx \ ${interval:+"-w $interval"} \ ${interval:+"-c 1"} \ - "$VDEV_UPATH" | tail -n 2) + "$somepath" | tail -n 2) else out=$(iostat -kx \ ${brief:+"-y"} \ ${interval:+"$interval"} \ ${interval:+"1"} \ - "$VDEV_UPATH" | grep -v '^$' | tail -n 2) + "$somepath" | grep -v '^$' | tail -n 2) fi diff --git a/cmd/zpool/zpool.d/lsblk b/cmd/zpool/zpool.d/lsblk index 1ed1464431aa..293effd48ac5 100755 --- a/cmd/zpool/zpool.d/lsblk +++ b/cmd/zpool/zpool.d/lsblk @@ -61,21 +61,29 @@ else list=$(echo "$script" | tr '[:upper:]' '[:lower:]') fi +# Sometimes, UPATH ends up /dev/(null). +# That should be corrected, but for now... +# shellcheck disable=SC2154 +if [ ! -b "$VDEV_UPATH" ]; then + somepath="${VDEV_PATH}" +else + somepath="${VDEV_UPATH}" +fi + # Older versions of lsblk don't support all these values (like SERIAL). for i in $list ; do # Special case: Looking up the size of a file-based vdev can't # be done with lsblk. - # shellcheck disable=SC2154 - if [ "$i" = "size" ] && [ -f "$VDEV_UPATH" ] ; then - size=$(du -h --apparent-size "$VDEV_UPATH" | cut -f 1) + if [ "$i" = "size" ] && [ -f "$somepath" ] ; then + size=$(du -h --apparent-size "$somepath" | cut -f 1) echo "size=$size" continue fi val="" - if val=$(eval "lsblk -dl -n -o $i $VDEV_UPATH 2>/dev/null") ; then + if val=$(eval "lsblk -dl -n -o $i $somepath 2>/dev/null") ; then # Remove leading/trailing whitespace from value val=$(echo "$val" | sed -e 's/^[[:space:]]*//' \ -e 's/[[:space:]]*$//') diff --git a/cmd/zpool/zpool.d/smart b/cmd/zpool/zpool.d/smart index 032491988c18..8ad3e107f091 100755 --- a/cmd/zpool/zpool.d/smart +++ b/cmd/zpool/zpool.d/smart @@ -69,8 +69,16 @@ if [ "$1" = "-h" ] ; then exit fi +# Sometimes, UPATH ends up /dev/(null). +# That should be corrected, but for now... # shellcheck disable=SC2154 -if [ -b "$VDEV_UPATH" ] && PATH="/usr/sbin:$PATH" command -v smartctl > /dev/null || [ -n "$samples" ] ; then +if [ ! -b "$VDEV_UPATH" ]; then + somepath="${VDEV_PATH}" +else + somepath="${VDEV_UPATH}" +fi + +if [ -b "$somepath" ] && PATH="/usr/sbin:$PATH" command -v smartctl > /dev/null || [ -n "$samples" ] ; then if [ -n "$samples" ] ; then # cat a smartctl output text file instead of running smartctl # on a vdev (only used for developer testing). @@ -78,7 +86,7 @@ if [ -b "$VDEV_UPATH" ] && PATH="/usr/sbin:$PATH" command -v smartctl > /dev/nul echo "file=$file" raw_out=$(cat "$samples/$file") else - raw_out=$(sudo smartctl -a "$VDEV_UPATH") + raw_out=$(sudo smartctl -a "$somepath") fi # What kind of drive are we? Look for the right line in smartctl: diff --git a/cmd/zpool/zpool_iter.c b/cmd/zpool/zpool_iter.c index 65e53f8e0e2a..772742ea03bd 100644 --- a/cmd/zpool/zpool_iter.c +++ b/cmd/zpool/zpool_iter.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -32,7 +32,7 @@ #include #include #include -#include +#include #include #include @@ -442,9 +442,8 @@ static void vdev_run_cmd(vdev_cmd_data_t *data, char *cmd) { int rc; - char *argv[2] = {cmd, 0}; - char *env[5] = {"PATH=/bin:/sbin:/usr/bin:/usr/sbin", NULL, NULL, NULL, - NULL}; + char *argv[2] = {cmd}; + char *env[5] = {(char *)"PATH=/bin:/sbin:/usr/bin:/usr/sbin"}; char **lines = NULL; int lines_cnt = 0; int i; diff --git a/cmd/zpool/zpool_main.c b/cmd/zpool/zpool_main.c index 43ab9c4cbb7b..13a51691fa78 100644 --- a/cmd/zpool/zpool_main.c +++ b/cmd/zpool/zpool_main.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -50,7 +50,6 @@ #include #include #include -#include #include #include #include @@ -519,7 +518,7 @@ print_vdev_prop_cb(int prop, void *cb) * that command. Otherwise, iterate over the entire command table and display * a complete usage message. */ -static void +static __attribute__((noreturn)) void usage(boolean_t requested) { FILE *fp = requested ? stdout : stderr; @@ -712,7 +711,7 @@ print_vdev_tree(zpool_handle_t *zhp, const char *name, nvlist_t *nv, int indent, for (c = 0; c < children; c++) { uint64_t is_log = B_FALSE, is_hole = B_FALSE; - char *class = ""; + char *class = (char *)""; (void) nvlist_lookup_uint64(child[c], ZPOOL_CONFIG_IS_HOLE, &is_hole); @@ -724,7 +723,7 @@ print_vdev_tree(zpool_handle_t *zhp, const char *name, nvlist_t *nv, int indent, (void) nvlist_lookup_uint64(child[c], ZPOOL_CONFIG_IS_LOG, &is_log); if (is_log) - class = VDEV_ALLOC_BIAS_LOG; + class = (char *)VDEV_ALLOC_BIAS_LOG; (void) nvlist_lookup_string(child[c], ZPOOL_CONFIG_ALLOCATION_BIAS, &class); if (strcmp(match, class) != 0) @@ -805,7 +804,7 @@ prop_list_contains_feature(nvlist_t *proplist) * Add a property pair (name, string-value) into a property nvlist. */ static int -add_prop_list(const char *propname, char *propval, nvlist_t **props, +add_prop_list(const char *propname, const char *propval, nvlist_t **props, boolean_t poolprop) { zpool_prop_t prop = ZPOOL_PROP_INVAL; @@ -906,7 +905,8 @@ add_prop_list(const char *propname, char *propval, nvlist_t **props, * Set a default property pair (name, string-value) in a property nvlist */ static int -add_prop_list_default(const char *propname, char *propval, nvlist_t **props) +add_prop_list_default(const char *propname, const char *propval, + nvlist_t **props) { char *pval; @@ -1791,8 +1791,8 @@ zpool_do_create(int argc, char **argv) tname ? tname : poolname, ZFS_TYPE_FILESYSTEM); if (pool != NULL) { if (zfs_mount(pool, NULL, 0) == 0) { - ret = zfs_shareall(pool); - zfs_commit_all_shares(); + ret = zfs_share(pool, NULL); + zfs_commit_shares(NULL); } zfs_close(pool); } @@ -1988,40 +1988,21 @@ static int max_width(zpool_handle_t *zhp, nvlist_t *nv, int depth, int max, int name_flags) { - char *name; - nvlist_t **child; - uint_t c, children; - int ret; - - name = zpool_vdev_name(g_zfs, zhp, nv, name_flags); - if (strlen(name) + depth > max) - max = strlen(name) + depth; + static const char *const subtypes[] = + {ZPOOL_CONFIG_SPARES, ZPOOL_CONFIG_L2CACHE, ZPOOL_CONFIG_CHILDREN}; + char *name = zpool_vdev_name(g_zfs, zhp, nv, name_flags); + max = MAX(strlen(name) + depth, max); free(name); - if (nvlist_lookup_nvlist_array(nv, ZPOOL_CONFIG_SPARES, - &child, &children) == 0) { - for (c = 0; c < children; c++) - if ((ret = max_width(zhp, child[c], depth + 2, - max, name_flags)) > max) - max = ret; - } - - if (nvlist_lookup_nvlist_array(nv, ZPOOL_CONFIG_L2CACHE, - &child, &children) == 0) { - for (c = 0; c < children; c++) - if ((ret = max_width(zhp, child[c], depth + 2, - max, name_flags)) > max) - max = ret; - } - - if (nvlist_lookup_nvlist_array(nv, ZPOOL_CONFIG_CHILDREN, - &child, &children) == 0) { - for (c = 0; c < children; c++) - if ((ret = max_width(zhp, child[c], depth + 2, - max, name_flags)) > max) - max = ret; - } + nvlist_t **child; + uint_t children; + for (size_t i = 0; i < ARRAY_SIZE(subtypes); ++i) + if (nvlist_lookup_nvlist_array(nv, subtypes[i], + &child, &children) == 0) + for (uint_t c = 0; c < children; ++c) + max = MAX(max_width(zhp, child[c], depth + 2, + max, name_flags), max); return (max); } @@ -2089,15 +2070,13 @@ typedef struct status_cbdata { } status_cbdata_t; /* Return 1 if string is NULL, empty, or whitespace; return 0 otherwise. */ -static int -is_blank_str(char *str) +static boolean_t +is_blank_str(const char *str) { - while (str != NULL && *str != '\0') { + for (; str != NULL && *str != '\0'; ++str) if (!isblank(*str)) - return (0); - str++; - } - return (1); + return (B_FALSE); + return (B_TRUE); } /* Print command output lines for specific vdev in a specific pool */ @@ -2106,7 +2085,7 @@ zpool_print_cmd(vdev_cmd_data_list_t *vcdl, const char *pool, char *path) { vdev_cmd_data_t *data; int i, j; - char *val; + const char *val; for (i = 0; i < vcdl->count; i++) { if ((strcmp(vcdl->data[i].path, path) != 0) || @@ -2137,17 +2116,17 @@ zpool_print_cmd(vdev_cmd_data_list_t *vcdl, const char *pool, char *path) printf("%*s", vcdl->uniq_cols_width[j], val); if (j < vcdl->uniq_cols_cnt - 1) - printf(" "); + fputs(" ", stdout); } /* Print out any values that aren't in a column at the end */ for (j = data->cols_cnt; j < data->lines_cnt; j++) { /* Did we have any columns? If so print a spacer. */ if (vcdl->uniq_cols_cnt > 0) - printf(" "); + fputs(" ", stdout); val = data->lines[j]; - printf("%s", val ? val : ""); + fputs(val ?: "", stdout); } break; } @@ -2260,7 +2239,7 @@ print_status_trim(vdev_stat_t *vs, boolean_t verbose) * Return the color associated with a health string. This includes returning * NULL for no color change. */ -static char * +static const char * health_str_to_color(const char *health) { if (strcmp(health, gettext("FAULTED")) == 0 || @@ -2296,7 +2275,7 @@ print_status_config(zpool_handle_t *zhp, status_cbdata_t *cb, const char *name, const char *state; char *type; char *path = NULL; - char *rcolor = NULL, *wcolor = NULL, *ccolor = NULL; + const char *rcolor = NULL, *wcolor = NULL, *ccolor = NULL; if (nvlist_lookup_nvlist_array(nv, ZPOOL_CONFIG_CHILDREN, &child, &children) != 0) @@ -2338,13 +2317,13 @@ print_status_config(zpool_handle_t *zhp, status_cbdata_t *cb, const char *name, ccolor = ANSI_RED; if (cb->cb_literal) { - printf(" "); + fputc(' ', stdout); printf_color(rcolor, "%5llu", (u_longlong_t)vs->vs_read_errors); - printf(" "); + fputc(' ', stdout); printf_color(wcolor, "%5llu", (u_longlong_t)vs->vs_write_errors); - printf(" "); + fputc(' ', stdout); printf_color(ccolor, "%5llu", (u_longlong_t)vs->vs_checksum_errors); } else { @@ -2352,11 +2331,11 @@ print_status_config(zpool_handle_t *zhp, status_cbdata_t *cb, const char *name, zfs_nicenum(vs->vs_write_errors, wbuf, sizeof (wbuf)); zfs_nicenum(vs->vs_checksum_errors, cbuf, sizeof (cbuf)); - printf(" "); + fputc(' ', stdout); printf_color(rcolor, "%5s", rbuf); - printf(" "); + fputc(' ', stdout); printf_color(wcolor, "%5s", wbuf); - printf(" "); + fputc(' ', stdout); printf_color(ccolor, "%5s", cbuf); } if (cb->cb_print_slow_ios) { @@ -2473,7 +2452,7 @@ print_status_config(zpool_handle_t *zhp, status_cbdata_t *cb, const char *name, if (vs->vs_scan_removing != 0) { (void) printf(gettext(" (removing)")); - } else if (vs->vs_noalloc != 0) { + } else if (VDEV_STAT_VALID(vs_noalloc, vsc) && vs->vs_noalloc != 0) { (void) printf(gettext(" (non-allocating)")); } @@ -2691,7 +2670,7 @@ print_class_vdevs(zpool_handle_t *zhp, status_cbdata_t *cb, nvlist_t *nv, &is_log); if (is_log) { - bias = VDEV_ALLOC_CLASS_LOGS; + bias = (char *)VDEV_ALLOC_CLASS_LOGS; } else { (void) nvlist_lookup_string(child[c], ZPOOL_CONFIG_ALLOCATION_BIAS, &bias); @@ -2731,8 +2710,8 @@ show_import(nvlist_t *config, boolean_t report_error) char *name; uint64_t guid; uint64_t hostid = 0; - char *msgid; - char *hostname = "unknown"; + const char *msgid; + const char *hostname = "unknown"; nvlist_t *nvroot, *nvinfo; zpool_status_t reason; zpool_errata_t errata; @@ -3272,7 +3251,7 @@ import_pools(nvlist_t *pools, nvlist_t *props, char *mntopts, int flags, if (first) first = B_FALSE; else if (!do_all) - (void) putchar('\n'); + (void) fputc('\n', stdout); if (do_all) { err |= do_import(config, NULL, mntopts, @@ -4688,7 +4667,7 @@ print_iostat_default(vdev_stat_t *vs, iostat_cbdata_t *cb, double scale) format, column_width, cb->cb_scripted); } -static const char *class_name[] = { +static const char *const class_name[] = { VDEV_ALLOC_BIAS_DEDUP, VDEV_ALLOC_BIAS_SPECIAL, VDEV_ALLOC_CLASS_LOGS @@ -4857,7 +4836,7 @@ print_vdev_stats(zpool_handle_t *zhp, const char *name, nvlist_t *oldnv, /* * print all other top-level devices */ - for (uint_t n = 0; n < 3; n++) { + for (uint_t n = 0; n < ARRAY_SIZE(class_name); n++) { boolean_t printed = B_FALSE; for (c = 0; c < children; c++) { @@ -4868,7 +4847,7 @@ print_vdev_stats(zpool_handle_t *zhp, const char *name, nvlist_t *oldnv, (void) nvlist_lookup_uint64(newchild[c], ZPOOL_CONFIG_IS_LOG, &islog); if (islog) { - bias = VDEV_ALLOC_CLASS_LOGS; + bias = (char *)VDEV_ALLOC_CLASS_LOGS; } else { (void) nvlist_lookup_string(newchild[c], ZPOOL_CONFIG_ALLOCATION_BIAS, &bias); @@ -5022,7 +5001,7 @@ get_namewidth(zpool_handle_t *zhp, int min_width, int flags, boolean_t verbose) if ((config = zpool_get_config(zhp, NULL)) != NULL) { verify(nvlist_lookup_nvlist(config, ZPOOL_CONFIG_VDEV_TREE, &nvroot) == 0); - unsigned int poolname_len = strlen(zpool_get_name(zhp)); + size_t poolname_len = strlen(zpool_get_name(zhp)); if (verbose == B_FALSE) { width = MAX(poolname_len, min_width); } else { @@ -5402,7 +5381,7 @@ terminal_height(void) static void print_zpool_script_help(char *name, char *path) { - char *argv[] = {path, "-h", NULL}; + char *argv[] = {path, (char *)"-h", NULL}; char **lines = NULL; int lines_cnt = 0; int rc; @@ -5454,7 +5433,7 @@ print_zpool_dir_scripts(char *dirpath) * Print out help text for all zpool status/iostat -c scripts. */ static void -print_zpool_script_list(char *subcommand) +print_zpool_script_list(const char *subcommand) { char *dir, *sp, *tmp; @@ -5961,12 +5940,12 @@ print_header(list_cbdata_t *cb) } if (!first) - (void) printf(" "); + (void) fputs(" ", stdout); else first = B_FALSE; right_justify = B_FALSE; - if (pl->pl_prop != ZPROP_INVAL) { + if (pl->pl_prop != ZPROP_USERPROP) { header = zpool_prop_column_name(pl->pl_prop); right_justify = zpool_prop_align_right(pl->pl_prop); } else { @@ -5979,14 +5958,14 @@ print_header(list_cbdata_t *cb) } if (pl->pl_next == NULL && !right_justify) - (void) printf("%s", header); + (void) fputs(header, stdout); else if (right_justify) (void) printf("%*s", (int)width, header); else (void) printf("%-*s", (int)width, header); } - (void) printf("\n"); + (void) fputc('\n', stdout); } /* @@ -5999,7 +5978,7 @@ print_pool(zpool_handle_t *zhp, list_cbdata_t *cb) zprop_list_t *pl = cb->cb_proplist; boolean_t first = B_TRUE; char property[ZPOOL_MAXPROPLEN]; - char *propstr; + const char *propstr; boolean_t right_justify; size_t width; @@ -6016,15 +5995,15 @@ print_pool(zpool_handle_t *zhp, list_cbdata_t *cb) if (!first) { if (cb->cb_scripted) - (void) printf("\t"); + (void) fputc('\t', stdout); else - (void) printf(" "); + (void) fputs(" ", stdout); } else { first = B_FALSE; } right_justify = B_FALSE; - if (pl->pl_prop != ZPROP_INVAL) { + if (pl->pl_prop != ZPROP_USERPROP) { if (zpool_get_prop(zhp, pl->pl_prop, property, sizeof (property), NULL, cb->cb_literal) != 0) propstr = "-"; @@ -6048,14 +6027,14 @@ print_pool(zpool_handle_t *zhp, list_cbdata_t *cb) * format specifier. */ if (cb->cb_scripted || (pl->pl_next == NULL && !right_justify)) - (void) printf("%s", propstr); + (void) fputs(propstr, stdout); else if (right_justify) (void) printf("%*s", (int)width, propstr); else (void) printf("%-*s", (int)width, propstr); } - (void) printf("\n"); + (void) fputc('\n', stdout); } static void @@ -6067,6 +6046,7 @@ print_one_column(zpool_prop_t prop, uint64_t value, const char *str, size_t width = zprop_width(prop, &fixed, ZFS_TYPE_POOL); switch (prop) { + case ZPOOL_PROP_SIZE: case ZPOOL_PROP_EXPANDSZ: case ZPOOL_PROP_CHECKPOINT: case ZPOOL_PROP_DEDUPRATIO: @@ -6128,8 +6108,8 @@ print_list_stats(zpool_handle_t *zhp, const char *name, nvlist_t *nv, char *vname; boolean_t scripted = cb->cb_scripted; uint64_t islog = B_FALSE; - char *dashes = "%-*s - - - - " - "- - - - -\n"; + const char *dashes = "%-*s - - - - " + "- - - - -\n"; verify(nvlist_lookup_uint64_array(nv, ZPOOL_CONFIG_VDEV_STATS, (uint64_t **)&vs, &c) == 0); @@ -6162,8 +6142,12 @@ print_list_stats(zpool_handle_t *zhp, const char *name, nvlist_t *nv, * 'toplevel' boolean value is passed to the print_one_column() * to indicate that the value is valid. */ - print_one_column(ZPOOL_PROP_SIZE, vs->vs_space, NULL, scripted, - toplevel, format); + if (VDEV_STAT_VALID(vs_pspace, c) && vs->vs_pspace) + print_one_column(ZPOOL_PROP_SIZE, vs->vs_pspace, NULL, + scripted, B_TRUE, format); + else + print_one_column(ZPOOL_PROP_SIZE, vs->vs_space, NULL, + scripted, toplevel, format); print_one_column(ZPOOL_PROP_ALLOCATED, vs->vs_alloc, NULL, scripted, toplevel, format); print_one_column(ZPOOL_PROP_FREE, vs->vs_space - vs->vs_alloc, @@ -6191,7 +6175,7 @@ print_list_stats(zpool_handle_t *zhp, const char *name, nvlist_t *nv, } print_one_column(ZPOOL_PROP_HEALTH, 0, state, scripted, B_TRUE, format); - (void) printf("\n"); + (void) fputc('\n', stdout); } if (nvlist_lookup_nvlist_array(nv, ZPOOL_CONFIG_CHILDREN, @@ -6220,7 +6204,7 @@ print_list_stats(zpool_handle_t *zhp, const char *name, nvlist_t *nv, } /* list the classes: 'logs', 'dedup', and 'special' */ - for (uint_t n = 0; n < 3; n++) { + for (uint_t n = 0; n < ARRAY_SIZE(class_name); n++) { boolean_t printed = B_FALSE; for (c = 0; c < children; c++) { @@ -6229,7 +6213,7 @@ print_list_stats(zpool_handle_t *zhp, const char *name, nvlist_t *nv, if (nvlist_lookup_uint64(child[c], ZPOOL_CONFIG_IS_LOG, &islog) == 0 && islog) { - bias = VDEV_ALLOC_CLASS_LOGS; + bias = (char *)VDEV_ALLOC_CLASS_LOGS; } else { (void) nvlist_lookup_string(child[c], ZPOOL_CONFIG_ALLOCATION_BIAS, &bias); @@ -8133,7 +8117,7 @@ status_callback(zpool_handle_t *zhp, void *data) { status_cbdata_t *cbp = data; nvlist_t *config, *nvroot; - char *msgid; + const char *msgid; zpool_status_t reason; zpool_errata_t errata; const char *health; @@ -8178,12 +8162,12 @@ status_callback(zpool_handle_t *zhp, void *data) printf(" "); printf_color(ANSI_BOLD, gettext("pool:")); printf(" %s\n", zpool_get_name(zhp)); - printf(" "); + fputc(' ', stdout); printf_color(ANSI_BOLD, gettext("state: ")); printf_color(health_str_to_color(health), "%s", health); - printf("\n"); + fputc('\n', stdout); switch (reason) { case ZPOOL_STATUS_MISSING_DEV_R: @@ -10065,7 +10049,6 @@ zpool_do_get(int argc, char **argv) zprop_list_t fake_name = { 0 }; int ret; int c, i; - char *value; char *propstr = NULL; cb.cb_first = B_TRUE; @@ -10092,35 +10075,34 @@ zpool_do_get(int argc, char **argv) cb.cb_scripted = B_TRUE; break; case 'o': - bzero(&cb.cb_columns, sizeof (cb.cb_columns)); + memset(&cb.cb_columns, 0, sizeof (cb.cb_columns)); i = 0; - while (*optarg != '\0') { - static char *col_subopts[] = + + for (char *tok; (tok = strsep(&optarg, ",")); ) { + static const char *const col_opts[] = { "name", "property", "value", "source", - "all", NULL }; + "all" }; + static const zfs_get_column_t col_cols[] = + { GET_COL_NAME, GET_COL_PROPERTY, GET_COL_VALUE, + GET_COL_SOURCE }; - if (i == ZFS_GET_NCOLS) { + if (i == ZFS_GET_NCOLS - 1) { (void) fprintf(stderr, gettext("too " "many fields given to -o " "option\n")); usage(B_FALSE); } - switch (getsubopt(&optarg, col_subopts, - &value)) { - case 0: - cb.cb_columns[i++] = GET_COL_NAME; - break; - case 1: - cb.cb_columns[i++] = GET_COL_PROPERTY; - break; - case 2: - cb.cb_columns[i++] = GET_COL_VALUE; - break; - case 3: - cb.cb_columns[i++] = GET_COL_SOURCE; - break; - case 4: + for (c = 0; c < ARRAY_SIZE(col_opts); ++c) + if (strcmp(tok, col_opts[c]) == 0) + goto found; + + (void) fprintf(stderr, + gettext("invalid column name '%s'\n"), tok); + usage(B_FALSE); + +found: + if (c >= 4) { if (i > 0) { (void) fprintf(stderr, gettext("\"all\" conflicts " @@ -10128,18 +10110,12 @@ zpool_do_get(int argc, char **argv) "given to -o option\n")); usage(B_FALSE); } - cb.cb_columns[0] = GET_COL_NAME; - cb.cb_columns[1] = GET_COL_PROPERTY; - cb.cb_columns[2] = GET_COL_VALUE; - cb.cb_columns[3] = GET_COL_SOURCE; - i = ZFS_GET_NCOLS; - break; - default: - (void) fprintf(stderr, - gettext("invalid column name " - "'%s'\n"), value); - usage(B_FALSE); - } + + memcpy(cb.cb_columns, col_cols, + sizeof (col_cols)); + i = ZFS_GET_NCOLS - 1; + } else + cb.cb_columns[i++] = col_cols[c]; } break; case '?': @@ -10508,8 +10484,8 @@ print_wait_status_row(wait_data_t *wd, zpool_handle_t *zhp, int row) pool_checkpoint_stat_t *pcs = NULL; pool_scan_stat_t *pss = NULL; pool_removal_stat_t *prs = NULL; - char *headers[] = {"DISCARD", "FREE", "INITIALIZE", "REPLACE", - "REMOVE", "RESILVER", "SCRUB", "TRIM"}; + const char *const headers[] = {"DISCARD", "FREE", "INITIALIZE", + "REPLACE", "REMOVE", "RESILVER", "SCRUB", "TRIM"}; int col_widths[ZPOOL_WAIT_NUM_ACTIVITIES]; /* Calculate the width of each column */ @@ -10531,7 +10507,7 @@ print_wait_status_row(wait_data_t *wd, zpool_handle_t *zhp, int row) if (wd->wd_enabled[i]) (void) printf("%*s", col_widths[i], headers[i]); } - (void) printf("\n"); + (void) fputc('\n', stdout); } /* Bytes of work remaining in each activity */ @@ -10667,9 +10643,7 @@ int zpool_do_wait(int argc, char **argv) { boolean_t verbose = B_FALSE; - int c; - char *value; - int i; + int c, i; unsigned long count; pthread_t status_thr; int error = 0; @@ -10703,28 +10677,26 @@ zpool_do_wait(int argc, char **argv) get_timestamp_arg(*optarg); break; case 't': - { - static char *col_subopts[] = { "discard", "free", - "initialize", "replace", "remove", "resilver", - "scrub", "trim", NULL }; - /* Reset activities array */ - bzero(&wd.wd_enabled, sizeof (wd.wd_enabled)); - while (*optarg != '\0') { - int activity = getsubopt(&optarg, col_subopts, - &value); - - if (activity < 0) { - (void) fprintf(stderr, - gettext("invalid activity '%s'\n"), - value); - usage(B_FALSE); - } + memset(&wd.wd_enabled, 0, sizeof (wd.wd_enabled)); - wd.wd_enabled[activity] = B_TRUE; + for (char *tok; (tok = strsep(&optarg, ",")); ) { + static const char *const col_opts[] = { + "discard", "free", "initialize", "replace", + "remove", "resilver", "scrub", "trim" }; + + for (i = 0; i < ARRAY_SIZE(col_opts); ++i) + if (strcmp(tok, col_opts[i]) == 0) { + wd.wd_enabled[i] = B_TRUE; + goto found; + } + + (void) fprintf(stderr, + gettext("invalid activity '%s'\n"), tok); + usage(B_FALSE); +found:; } break; - } case '?': (void) fprintf(stderr, gettext("invalid option '%c'\n"), optopt); @@ -10822,11 +10794,9 @@ zpool_do_wait(int argc, char **argv) } static int -find_command_idx(char *command, int *idx) +find_command_idx(const char *command, int *idx) { - int i; - - for (i = 0; i < NCOMMAND; i++) { + for (int i = 0; i < NCOMMAND; ++i) { if (command_table[i].name == NULL) continue; @@ -10845,11 +10815,7 @@ static int zpool_do_version(int argc, char **argv) { (void) argc, (void) argv; - - if (zfs_version_print() == -1) - return (1); - - return (0); + return (zfs_version_print() != 0); } /* @@ -10932,7 +10898,7 @@ main(int argc, char **argv) * Many commands modify input strings for string parsing reasons. * We create a copy to protect the original argv. */ - newargv = malloc((argc + 1) * sizeof (newargv[0])); + newargv = safe_malloc((argc + 1) * sizeof (newargv[0])); for (i = 0; i < argc; i++) newargv[i] = strdup(argv[i]); newargv[argc] = NULL; diff --git a/cmd/zpool/zpool_util.c b/cmd/zpool/zpool_util.c index 1c64c83d8ff1..e7ff739e5b49 100644 --- a/cmd/zpool/zpool_util.c +++ b/cmd/zpool/zpool_util.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -28,7 +28,7 @@ #include #include #include -#include +#include #include #include "zpool_util.h" diff --git a/cmd/zpool/zpool_util.h b/cmd/zpool/zpool_util.h index 583f48cca82a..b35dea0cd449 100644 --- a/cmd/zpool/zpool_util.h +++ b/cmd/zpool/zpool_util.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/cmd/zpool/zpool_vdev.c b/cmd/zpool/zpool_vdev.c index 0653a09faea2..b9a0af118827 100644 --- a/cmd/zpool/zpool_vdev.c +++ b/cmd/zpool/zpool_vdev.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -274,7 +274,7 @@ make_leaf_vdev(nvlist_t *props, const char *arg, boolean_t is_primary) char path[MAXPATHLEN]; struct stat64 statbuf; nvlist_t *vdev = NULL; - char *type = NULL; + const char *type = NULL; boolean_t wholedisk = B_FALSE; uint64_t ashift = 0; int err; diff --git a/cmd/zpool_influxdb/.gitignore b/cmd/zpool_influxdb/.gitignore deleted file mode 100644 index bd765d188278..000000000000 --- a/cmd/zpool_influxdb/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/zpool_influxdb diff --git a/cmd/zpool_influxdb/Makefile.am b/cmd/zpool_influxdb/Makefile.am index a59217570b9d..b237532ce24e 100644 --- a/cmd/zpool_influxdb/Makefile.am +++ b/cmd/zpool_influxdb/Makefile.am @@ -1,13 +1,10 @@ -include $(top_srcdir)/config/Rules.am - -zfsexec_PROGRAMS = zpool_influxdb +zfsexec_PROGRAMS += zpool_influxdb +CPPCHECKTARGETS += zpool_influxdb zpool_influxdb_SOURCES = \ - zpool_influxdb.c + %D%/zpool_influxdb.c zpool_influxdb_LDADD = \ - $(top_builddir)/lib/libspl/libspl.la \ - $(top_builddir)/lib/libnvpair/libnvpair.la \ - $(top_builddir)/lib/libzfs/libzfs.la - -include $(top_srcdir)/config/CppCheck.am + libspl.la \ + libnvpair.la \ + libzfs.la diff --git a/cmd/zpool_influxdb/zpool_influxdb.c b/cmd/zpool_influxdb/zpool_influxdb.c index 2d6859efb44d..251d588d832a 100644 --- a/cmd/zpool_influxdb/zpool_influxdb.c +++ b/cmd/zpool_influxdb/zpool_influxdb.c @@ -90,7 +90,7 @@ char metric_data_type = 'u'; uint64_t metric_value_mask = UINT64_MAX; uint64_t timestamp = 0; int complained_about_sync = 0; -char *tags = ""; +const char *tags = ""; typedef int (*stat_printer_f)(nvlist_t *, const char *, const char *); @@ -131,7 +131,7 @@ escape_string(const char *s) * print key=value where value is a uint64_t */ static void -print_kv(char *key, uint64_t value) +print_kv(const char *key, uint64_t value) { printf("%s=%llu%c", key, (u_longlong_t)value & metric_value_mask, metric_data_type); @@ -152,9 +152,9 @@ print_scan_status(nvlist_t *nvroot, const char *pool_name) uint64_t remaining_time; pool_scan_stat_t *ps = NULL; double pct_done; - char *state[DSS_NUM_STATES] = { + const char *const state[DSS_NUM_STATES] = { "none", "scanning", "finished", "canceled"}; - char *func; + const char *func; (void) nvlist_lookup_uint64_array(nvroot, ZPOOL_CONFIG_SCAN_STATS, @@ -262,23 +262,21 @@ static char * get_vdev_name(nvlist_t *nvroot, const char *parent_name) { static char vdev_name[256]; - char *vdev_type = NULL; uint64_t vdev_id = 0; - if (nvlist_lookup_string(nvroot, ZPOOL_CONFIG_TYPE, - &vdev_type) != 0) { - vdev_type = "unknown"; - } + char *vdev_type = (char *)"unknown"; + nvlist_lookup_string(nvroot, ZPOOL_CONFIG_TYPE, &vdev_type); + if (nvlist_lookup_uint64( - nvroot, ZPOOL_CONFIG_ID, &vdev_id) != 0) { + nvroot, ZPOOL_CONFIG_ID, &vdev_id) != 0) vdev_id = UINT64_MAX; - } + if (parent_name == NULL) { (void) snprintf(vdev_name, sizeof (vdev_name), "%s", vdev_type); } else { (void) snprintf(vdev_name, sizeof (vdev_name), - "%s/%s-%llu", + "%.220s/%s-%llu", parent_name, vdev_type, (u_longlong_t)vdev_id); } return (vdev_name); @@ -298,22 +296,15 @@ static char * get_vdev_desc(nvlist_t *nvroot, const char *parent_name) { static char vdev_desc[2 * MAXPATHLEN]; - char *vdev_type = NULL; - uint64_t vdev_id = 0; char vdev_value[MAXPATHLEN]; - char *vdev_path = NULL; char *s, *t; - if (nvlist_lookup_string(nvroot, ZPOOL_CONFIG_TYPE, &vdev_type) != 0) { - vdev_type = "unknown"; - } - if (nvlist_lookup_uint64(nvroot, ZPOOL_CONFIG_ID, &vdev_id) != 0) { - vdev_id = UINT64_MAX; - } - if (nvlist_lookup_string( - nvroot, ZPOOL_CONFIG_PATH, &vdev_path) != 0) { - vdev_path = NULL; - } + char *vdev_type = (char *)"unknown"; + uint64_t vdev_id = UINT64_MAX; + char *vdev_path = NULL; + nvlist_lookup_string(nvroot, ZPOOL_CONFIG_TYPE, &vdev_type); + nvlist_lookup_uint64(nvroot, ZPOOL_CONFIG_ID, &vdev_id); + nvlist_lookup_string(nvroot, ZPOOL_CONFIG_PATH, &vdev_path); if (parent_name == NULL) { s = escape_string(vdev_type); @@ -393,8 +384,8 @@ print_vdev_latency_stats(nvlist_t *nvroot, const char *pool_name, /* short_names become part of the metric name and are influxdb-ready */ struct lat_lookup { - char *name; - char *short_name; + const char *name; + const char *short_name; uint64_t sum; uint64_t *array; }; @@ -487,8 +478,8 @@ print_vdev_size_stats(nvlist_t *nvroot, const char *pool_name, /* short_names become the field name */ struct size_lookup { - char *name; - char *short_name; + const char *name; + const char *short_name; uint64_t sum; uint64_t *array; }; @@ -579,8 +570,8 @@ print_queue_stats(nvlist_t *nvroot, const char *pool_name, /* short_names are used for the field name */ struct queue_lookup { - char *name; - char *short_name; + const char *name; + const char *short_name; }; struct queue_lookup queue_type[] = { {ZPOOL_CONFIG_VDEV_SYNC_R_ACTIVE_QUEUE, "sync_r_active"}, @@ -632,8 +623,8 @@ print_top_level_vdev_stats(nvlist_t *nvroot, const char *pool_name) /* short_names become part of the metric name */ struct queue_lookup { - char *name; - char *short_name; + const char *name; + const char *short_name; }; struct queue_lookup queue_type[] = { {ZPOOL_CONFIG_VDEV_SYNC_R_ACTIVE_QUEUE, "sync_r_active_queue"}, @@ -789,7 +780,7 @@ main(int argc, char *argv[]) { int opt; int ret = 8; - char *line = NULL; + char *line = NULL, *ttags = NULL; size_t len, tagslen = 0; struct option long_options[] = { {"execd", no_argument, NULL, 'e'}, @@ -817,15 +808,17 @@ main(int argc, char *argv[]) sum_histogram_buckets = 1; break; case 't': + free(ttags); tagslen = strlen(optarg) + 2; - tags = calloc(1, tagslen); - if (tags == NULL) { + ttags = calloc(1, tagslen); + if (ttags == NULL) { fprintf(stderr, "error: cannot allocate memory " "for tags\n"); exit(1); } - (void) snprintf(tags, tagslen, ",%s", optarg); + (void) snprintf(ttags, tagslen, ",%s", optarg); + tags = ttags; break; default: usage(argv[0]); diff --git a/cmd/zstream/.gitignore b/cmd/zstream/.gitignore deleted file mode 100644 index fd1240d55c4b..000000000000 --- a/cmd/zstream/.gitignore +++ /dev/null @@ -1 +0,0 @@ -zstream diff --git a/cmd/zstream/Makefile.am b/cmd/zstream/Makefile.am index 8e813027fa3d..9ae33179e5d6 100644 --- a/cmd/zstream/Makefile.am +++ b/cmd/zstream/Makefile.am @@ -1,20 +1,20 @@ -include $(top_srcdir)/config/Rules.am - -sbin_PROGRAMS = zstream +sbin_PROGRAMS += zstream +CPPCHECKTARGETS += zstream zstream_SOURCES = \ - zstream.c \ - zstream.h \ - zstream_dump.c \ - zstream_redup.c \ - zstream_token.c + %D%/zstream.c \ + %D%/zstream.h \ + %D%/zstream_decompress.c \ + %D%/zstream_dump.c \ + %D%/zstream_redup.c \ + %D%/zstream_token.c zstream_LDADD = \ - $(abs_top_builddir)/lib/libzfs/libzfs.la \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la - -include $(top_srcdir)/config/CppCheck.am + libzfs.la \ + libzfs_core.la \ + libzpool.la \ + libnvpair.la +PHONY += install-exec-hook install-exec-hook: cd $(DESTDIR)$(sbindir) && $(LN_S) -f zstream zstreamdump diff --git a/cmd/zstream/zstream.c b/cmd/zstream/zstream.c index 523ae0689710..eeceba2475ca 100644 --- a/cmd/zstream/zstream.c +++ b/cmd/zstream/zstream.c @@ -23,7 +23,7 @@ #include #include #include -#include +#include #include #include #include @@ -40,6 +40,8 @@ zstream_usage(void) "\tzstream dump [-vCd] FILE\n" "\t... | zstream dump [-vCd]\n" "\n" + "\tzstream decompress [-v] [OBJECT,OFFSET[,TYPE]] ...\n" + "\n" "\tzstream token resume_token\n" "\n" "\tzstream redup [-v] FILE | ...\n"); @@ -61,6 +63,8 @@ main(int argc, char *argv[]) if (strcmp(subcommand, "dump") == 0) { return (zstream_do_dump(argc - 1, argv + 1)); + } else if (strcmp(subcommand, "decompress") == 0) { + return (zstream_do_decompress(argc - 1, argv + 1)); } else if (strcmp(subcommand, "token") == 0) { return (zstream_do_token(argc - 1, argv + 1)); } else if (strcmp(subcommand, "redup") == 0) { diff --git a/cmd/zstream/zstream.h b/cmd/zstream/zstream.h index 319fecb2876b..931d4e13fec0 100644 --- a/cmd/zstream/zstream.h +++ b/cmd/zstream/zstream.h @@ -24,8 +24,12 @@ extern "C" { #endif +extern void *safe_calloc(size_t n); +extern int sfread(void *buf, size_t size, FILE *fp); +extern void *safe_malloc(size_t size); extern int zstream_do_redup(int, char *[]); extern int zstream_do_dump(int, char *[]); +extern int zstream_do_decompress(int argc, char *argv[]); extern int zstream_do_token(int, char *[]); extern void zstream_usage(void); diff --git a/cmd/zstream/zstream_decompress.c b/cmd/zstream/zstream_decompress.c new file mode 100644 index 000000000000..31d4b2d36982 --- /dev/null +++ b/cmd/zstream/zstream_decompress.c @@ -0,0 +1,359 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2022 Axcient. All rights reserved. + * Use is subject to license terms. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include "zfs_fletcher.h" +#include "zstream.h" + +static int +dump_record(dmu_replay_record_t *drr, void *payload, int payload_len, + zio_cksum_t *zc, int outfd) +{ + assert(offsetof(dmu_replay_record_t, drr_u.drr_checksum.drr_checksum) + == sizeof (dmu_replay_record_t) - sizeof (zio_cksum_t)); + fletcher_4_incremental_native(drr, + offsetof(dmu_replay_record_t, drr_u.drr_checksum.drr_checksum), zc); + if (drr->drr_type != DRR_BEGIN) { + assert(ZIO_CHECKSUM_IS_ZERO(&drr->drr_u. + drr_checksum.drr_checksum)); + drr->drr_u.drr_checksum.drr_checksum = *zc; + } + fletcher_4_incremental_native(&drr->drr_u.drr_checksum.drr_checksum, + sizeof (zio_cksum_t), zc); + if (write(outfd, drr, sizeof (*drr)) == -1) + return (errno); + if (payload_len != 0) { + fletcher_4_incremental_native(payload, payload_len, zc); + if (write(outfd, payload, payload_len) == -1) + return (errno); + } + return (0); +} + +int +zstream_do_decompress(int argc, char *argv[]) +{ + const int KEYSIZE = 64; + int bufsz = SPA_MAXBLOCKSIZE; + char *buf = safe_malloc(bufsz); + dmu_replay_record_t thedrr; + dmu_replay_record_t *drr = &thedrr; + zio_cksum_t stream_cksum; + int c; + boolean_t verbose = B_FALSE; + + while ((c = getopt(argc, argv, "v")) != -1) { + switch (c) { + case 'v': + verbose = B_TRUE; + break; + case '?': + (void) fprintf(stderr, "invalid option '%c'\n", + optopt); + zstream_usage(); + break; + } + } + + argc -= optind; + argv += optind; + + if (argc < 0) + zstream_usage(); + + if (hcreate(argc) == 0) + errx(1, "hcreate"); + for (int i = 0; i < argc; i++) { + uint64_t object, offset; + char *obj_str; + char *offset_str; + char *key; + char *end; + enum zio_compress type = ZIO_COMPRESS_LZ4; + + obj_str = strsep(&argv[i], ","); + if (argv[i] == NULL) { + zstream_usage(); + exit(2); + } + errno = 0; + object = strtoull(obj_str, &end, 0); + if (errno || *end != '\0') + errx(1, "invalid value for object"); + offset_str = strsep(&argv[i], ","); + offset = strtoull(offset_str, &end, 0); + if (errno || *end != '\0') + errx(1, "invalid value for offset"); + if (argv[i]) { + if (0 == strcmp("lz4", argv[i])) + type = ZIO_COMPRESS_LZ4; + else if (0 == strcmp("lzjb", argv[i])) + type = ZIO_COMPRESS_LZJB; + else if (0 == strcmp("gzip", argv[i])) + type = ZIO_COMPRESS_GZIP_1; + else if (0 == strcmp("zle", argv[i])) + type = ZIO_COMPRESS_ZLE; + else if (0 == strcmp("zstd", argv[i])) + type = ZIO_COMPRESS_ZSTD; + else { + fprintf(stderr, "Invalid compression type %s.\n" + "Supported types are lz4, lzjb, gzip, zle, " + "and zstd\n", + argv[i]); + exit(2); + } + } + + if (asprintf(&key, "%llu,%llu", (u_longlong_t)object, + (u_longlong_t)offset) < 0) { + err(1, "asprintf"); + } + ENTRY e = {.key = key}; + ENTRY *p; + + p = hsearch(e, ENTER); + if (p == NULL) + errx(1, "hsearch"); + p->data = (void*)type; + } + + if (isatty(STDIN_FILENO)) { + (void) fprintf(stderr, + "Error: The send stream is a binary format " + "and can not be read from a\n" + "terminal. Standard input must be redirected.\n"); + exit(1); + } + + fletcher_4_init(); + while (sfread(drr, sizeof (*drr), stdin) != 0) { + struct drr_write *drrw; + uint64_t payload_size = 0; + + /* + * We need to regenerate the checksum. + */ + if (drr->drr_type != DRR_BEGIN) { + memset(&drr->drr_u.drr_checksum.drr_checksum, 0, + sizeof (drr->drr_u.drr_checksum.drr_checksum)); + } + + switch (drr->drr_type) { + case DRR_BEGIN: + { + ZIO_SET_CHECKSUM(&stream_cksum, 0, 0, 0, 0); + + int sz = drr->drr_payloadlen; + if (sz != 0) { + if (sz > bufsz) { + buf = realloc(buf, sz); + if (buf == NULL) + err(1, "realloc"); + bufsz = sz; + } + (void) sfread(buf, sz, stdin); + } + payload_size = sz; + break; + } + case DRR_END: + { + struct drr_end *drre = &drr->drr_u.drr_end; + /* + * Use the recalculated checksum, unless this is + * the END record of a stream package, which has + * no checksum. + */ + if (!ZIO_CHECKSUM_IS_ZERO(&drre->drr_checksum)) + drre->drr_checksum = stream_cksum; + break; + } + + case DRR_OBJECT: + { + struct drr_object *drro = &drr->drr_u.drr_object; + + if (drro->drr_bonuslen > 0) { + payload_size = DRR_OBJECT_PAYLOAD_SIZE(drro); + (void) sfread(buf, payload_size, stdin); + } + break; + } + + case DRR_SPILL: + { + struct drr_spill *drrs = &drr->drr_u.drr_spill; + payload_size = DRR_SPILL_PAYLOAD_SIZE(drrs); + (void) sfread(buf, payload_size, stdin); + break; + } + + case DRR_WRITE_BYREF: + fprintf(stderr, + "Deduplicated streams are not supported\n"); + exit(1); + break; + + case DRR_WRITE: + { + drrw = &thedrr.drr_u.drr_write; + payload_size = DRR_WRITE_PAYLOAD_SIZE(drrw); + ENTRY *p; + char key[KEYSIZE]; + + snprintf(key, KEYSIZE, "%llu,%llu", + (u_longlong_t)drrw->drr_object, + (u_longlong_t)drrw->drr_offset); + ENTRY e = {.key = key}; + + p = hsearch(e, FIND); + if (p != NULL) { + zio_decompress_func_t *xfunc = NULL; + switch ((enum zio_compress)(intptr_t)p->data) { + case ZIO_COMPRESS_LZJB: + xfunc = lzjb_decompress; + break; + case ZIO_COMPRESS_GZIP_1: + xfunc = gzip_decompress; + break; + case ZIO_COMPRESS_ZLE: + xfunc = zle_decompress; + break; + case ZIO_COMPRESS_LZ4: + xfunc = lz4_decompress_zfs; + break; + case ZIO_COMPRESS_ZSTD: + xfunc = zfs_zstd_decompress; + break; + default: + assert(B_FALSE); + } + assert(xfunc != NULL); + + + /* + * Read and decompress the block + */ + char *lzbuf = safe_calloc(payload_size); + (void) sfread(lzbuf, payload_size, stdin); + if (0 != xfunc(lzbuf, buf, + payload_size, payload_size, 0)) { + /* + * The block must not be compressed, + * possibly because it gets written + * multiple times in this stream. + */ + warnx("decompression failed for " + "ino %llu offset %llu", + (u_longlong_t)drrw->drr_object, + (u_longlong_t)drrw->drr_offset); + memcpy(buf, lzbuf, payload_size); + } else if (verbose) { + fprintf(stderr, "successfully " + "decompressed ino %llu " + "offset %llu\n", + (u_longlong_t)drrw->drr_object, + (u_longlong_t)drrw->drr_offset); + } + free(lzbuf); + } else { + /* + * Read the contents of the block unaltered + */ + (void) sfread(buf, payload_size, stdin); + } + break; + } + + case DRR_WRITE_EMBEDDED: + { + struct drr_write_embedded *drrwe = + &drr->drr_u.drr_write_embedded; + payload_size = + P2ROUNDUP((uint64_t)drrwe->drr_psize, 8); + (void) sfread(buf, payload_size, stdin); + break; + } + + case DRR_FREEOBJECTS: + case DRR_FREE: + case DRR_OBJECT_RANGE: + break; + + default: + (void) fprintf(stderr, "INVALID record type 0x%x\n", + drr->drr_type); + /* should never happen, so assert */ + assert(B_FALSE); + } + + if (feof(stdout)) { + fprintf(stderr, "Error: unexpected end-of-file\n"); + exit(1); + } + if (ferror(stdout)) { + fprintf(stderr, "Error while reading file: %s\n", + strerror(errno)); + exit(1); + } + + /* + * We need to recalculate the checksum, and it needs to be + * initially zero to do that. BEGIN records don't have + * a checksum. + */ + if (drr->drr_type != DRR_BEGIN) { + memset(&drr->drr_u.drr_checksum.drr_checksum, 0, + sizeof (drr->drr_u.drr_checksum.drr_checksum)); + } + if (dump_record(drr, buf, payload_size, + &stream_cksum, STDOUT_FILENO) != 0) + break; + if (drr->drr_type == DRR_END) { + /* + * Typically the END record is either the last + * thing in the stream, or it is followed + * by a BEGIN record (which also zeros the checksum). + * However, a stream package ends with two END + * records. The last END record's checksum starts + * from zero. + */ + ZIO_SET_CHECKSUM(&stream_cksum, 0, 0, 0, 0); + } + } + free(buf); + fletcher_4_fini(); + hdestroy(); + + return (0); +} diff --git a/cmd/zstream/zstream_dump.c b/cmd/zstream/zstream_dump.c index af69a1494f28..0e3954678509 100644 --- a/cmd/zstream/zstream_dump.c +++ b/cmd/zstream/zstream_dump.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -34,7 +34,7 @@ #include #include #include -#include +#include #include #include @@ -59,7 +59,7 @@ FILE *send_stream = 0; boolean_t do_byteswap = B_FALSE; boolean_t do_cksum = B_TRUE; -static void * +void * safe_malloc(size_t size) { void *rv = malloc(size); @@ -297,6 +297,7 @@ zstream_do_dump(int argc, char *argv[]) fletcher_4_init(); while (read_hdr(drr, &zc)) { + uint64_t featureflags = 0; /* * If this is the first DMU record being processed, check for @@ -362,6 +363,9 @@ zstream_do_dump(int argc, char *argv[]) BSWAP_64(drrb->drr_fromguid); } + featureflags = + DMU_GET_FEATUREFLAGS(drrb->drr_versioninfo); + (void) printf("BEGIN record\n"); (void) printf("\thdrtype = %lld\n", DMU_GET_STREAM_HDRTYPE(drrb->drr_versioninfo)); @@ -461,7 +465,8 @@ zstream_do_dump(int argc, char *argv[]) BSWAP_64(drro->drr_maxblkid); } - if (drro->drr_bonuslen > drro->drr_raw_bonuslen) { + if (featureflags & DMU_BACKUP_FEATURE_RAW && + drro->drr_bonuslen > drro->drr_raw_bonuslen) { (void) fprintf(stderr, "Warning: Object %llu has bonuslen = " "%u > raw_bonuslen = %u\n\n", diff --git a/cmd/zstream/zstream_redup.c b/cmd/zstream/zstream_redup.c index 474527e76ea7..5807fabcecb5 100644 --- a/cmd/zstream/zstream_redup.c +++ b/cmd/zstream/zstream_redup.c @@ -27,7 +27,7 @@ #include #include #include -#include +#include #include #include #include @@ -65,7 +65,7 @@ highbit64(uint64_t i) return (NBBY * sizeof (uint64_t) - __builtin_clzll(i)); } -static void * +void * safe_calloc(size_t n) { void *rv = calloc(1, n); @@ -81,7 +81,7 @@ safe_calloc(size_t n) /* * Safe version of fread(), exits on error. */ -static int +int sfread(void *buf, size_t size, FILE *fp) { int rv = fread(buf, size, 1, fp); @@ -229,7 +229,7 @@ zfs_redup_stream(int infd, int outfd, boolean_t verbose) * We need to regenerate the checksum. */ if (drr->drr_type != DRR_BEGIN) { - bzero(&drr->drr_u.drr_checksum.drr_checksum, + memset(&drr->drr_u.drr_checksum.drr_checksum, 0, sizeof (drr->drr_u.drr_checksum.drr_checksum)); } @@ -380,7 +380,7 @@ zfs_redup_stream(int infd, int outfd, boolean_t verbose) * a checksum. */ if (drr->drr_type != DRR_BEGIN) { - bzero(&drr->drr_u.drr_checksum.drr_checksum, + memset(&drr->drr_u.drr_checksum.drr_checksum, 0, sizeof (drr->drr_u.drr_checksum.drr_checksum)); } if (dump_record(drr, buf, payload_size, diff --git a/cmd/zstream/zstream_token.c b/cmd/zstream/zstream_token.c index 36a76a4bb851..23cc56dce37c 100644 --- a/cmd/zstream/zstream_token.c +++ b/cmd/zstream/zstream_token.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -34,7 +34,7 @@ #include #include #include -#include +#include #include #include diff --git a/cmd/ztest/ztest.c b/cmd/ztest.c similarity index 96% rename from cmd/ztest/ztest.c rename to cmd/ztest.c index ed60d065c8a7..31b9990a1fcf 100644 --- a/cmd/ztest/ztest.c +++ b/cmd/ztest.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -121,6 +121,7 @@ #include #include #include +#include #include #include #include @@ -263,7 +264,7 @@ extern unsigned long zfs_reconstruct_indirect_damage_fraction; static ztest_shared_opts_t *ztest_shared_opts; static ztest_shared_opts_t ztest_opts; -static char *ztest_wkeydata = "abcdefghijklmnopqrstuvwxyz012345"; +static const char *const ztest_wkeydata = "abcdefghijklmnopqrstuvwxyz012345"; typedef struct ztest_shared_ds { uint64_t zd_seq; @@ -417,6 +418,7 @@ ztest_func_t ztest_device_removal; ztest_func_t ztest_spa_checkpoint_create_discard; ztest_func_t ztest_initialize; ztest_func_t ztest_trim; +ztest_func_t ztest_blake3; ztest_func_t ztest_fletcher; ztest_func_t ztest_fletcher_incr; ztest_func_t ztest_verify_dnode_bt; @@ -470,6 +472,7 @@ ztest_info_t ztest_info[] = { ZTI_INIT(ztest_spa_checkpoint_create_discard, 1, &zopt_rarely), ZTI_INIT(ztest_initialize, 1, &zopt_sometimes), ZTI_INIT(ztest_trim, 1, &zopt_sometimes), + ZTI_INIT(ztest_blake3, 1, &zopt_rarely), ZTI_INIT(ztest_fletcher, 1, &zopt_rarely), ZTI_INIT(ztest_fletcher_incr, 1, &zopt_rarely), ZTI_INIT(ztest_verify_dnode_bt, 1, &zopt_sometimes), @@ -558,7 +561,7 @@ enum ztest_object { ZTEST_OBJECTS }; -static void usage(boolean_t) __NORETURN; +static __attribute__((noreturn)) void usage(boolean_t requested); static int ztest_scrub_impl(spa_t *spa); /* @@ -620,10 +623,10 @@ static void sig_handler(int signo) #define FATAL_MSG_SZ 1024 -char *fatal_msg; +static const char *fatal_msg; -static __attribute__((noreturn)) __attribute__((format(printf, 2, 3))) void -fatal(int do_perror, char *message, ...) +static __attribute__((format(printf, 2, 3))) __attribute__((noreturn)) void +fatal(int do_perror, const char *message, ...) { va_list args; int save_errno = errno; @@ -721,7 +724,7 @@ typedef struct ztest_option { const char *long_opt_param; const char *comment; unsigned int default_int; - char *default_str; + const char *default_str; } ztest_option_t; /* @@ -831,7 +834,7 @@ fini_options(void) short_opts = NULL; } -static void +static __attribute__((noreturn)) void usage(boolean_t requested) { char option[80]; @@ -929,10 +932,9 @@ process_options(int argc, char **argv) int opt; uint64_t value; - char altdir[MAXNAMELEN] = { 0 }; - char raid_kind[8] = { "random" }; + const char *raid_kind = "random"; - bcopy(&ztest_opts_defaults, zo, sizeof (*zo)); + memcpy(zo, &ztest_opts_defaults, sizeof (*zo)); init_options(); @@ -978,7 +980,7 @@ process_options(int argc, char **argv) zo->zo_raid_parity = MIN(MAX(value, 1), 3); break; case 'K': - (void) strlcpy(raid_kind, optarg, sizeof (raid_kind)); + raid_kind = optarg; break; case 'D': zo->zo_draid_data = MAX(1, value); @@ -1037,7 +1039,8 @@ process_options(int argc, char **argv) zo->zo_maxloops = MAX(1, value); break; case 'B': - (void) strlcpy(altdir, optarg, sizeof (altdir)); + (void) strlcpy(zo->zo_alt_ztest, optarg, + sizeof (zo->zo_alt_ztest)); break; case 'C': ztest_parse_name_value(optarg, zo); @@ -1076,8 +1079,7 @@ process_options(int argc, char **argv) /* When raid choice is 'random' add a draid pool 50% of the time */ if (strcmp(raid_kind, "random") == 0) { - (void) strlcpy(raid_kind, (ztest_random(2) == 0) ? - "draid" : "raidz", sizeof (raid_kind)); + raid_kind = (ztest_random(2) == 0) ? "draid" : "raidz"; if (ztest_opts.zo_verbose >= 3) (void) printf("choosing RAID type '%s'\n", raid_kind); @@ -1127,51 +1129,28 @@ process_options(int argc, char **argv) (zo->zo_vdevs > 0 ? zo->zo_time * NANOSEC / zo->zo_vdevs : UINT64_MAX >> 2); - if (strlen(altdir) > 0) { - char *cmd; - char *realaltdir; - char *bin; - char *ztest; - char *isa; - int isalen; - - cmd = umem_alloc(MAXPATHLEN, UMEM_NOFAIL); - realaltdir = umem_alloc(MAXPATHLEN, UMEM_NOFAIL); - - VERIFY3P(NULL, !=, realpath(getexecname(), cmd)); - if (0 != access(altdir, F_OK)) { - ztest_dump_core = B_FALSE; - fatal(B_TRUE, "invalid alternate ztest path: %s", - altdir); - } - VERIFY3P(NULL, !=, realpath(altdir, realaltdir)); - - /* - * 'cmd' should be of the form "/usr/bin//ztest". - * We want to extract to determine if we should use - * 32 or 64 bit binaries. - */ - bin = strstr(cmd, "/usr/bin/"); - ztest = strstr(bin, "/ztest"); - isa = bin + 9; - isalen = ztest - isa; - (void) snprintf(zo->zo_alt_ztest, sizeof (zo->zo_alt_ztest), - "%s/usr/bin/%.*s/ztest", realaltdir, isalen, isa); - (void) snprintf(zo->zo_alt_libpath, sizeof (zo->zo_alt_libpath), - "%s/usr/lib/%.*s", realaltdir, isalen, isa); - - if (0 != access(zo->zo_alt_ztest, X_OK)) { - ztest_dump_core = B_FALSE; - fatal(B_TRUE, "invalid alternate ztest: %s", - zo->zo_alt_ztest); - } else if (0 != access(zo->zo_alt_libpath, X_OK)) { - ztest_dump_core = B_FALSE; - fatal(B_TRUE, "invalid alternate lib directory %s", - zo->zo_alt_libpath); - } + if (*zo->zo_alt_ztest) { + const char *invalid_what = "ztest"; + char *val = zo->zo_alt_ztest; + if (0 != access(val, X_OK) || + (strrchr(val, '/') == NULL && (errno = EINVAL))) + goto invalid; + + int dirlen = strrchr(val, '/') - val; + strncpy(zo->zo_alt_libpath, val, dirlen); + invalid_what = "library path", val = zo->zo_alt_libpath; + if (strrchr(val, '/') == NULL && (errno = EINVAL)) + goto invalid; + *strrchr(val, '/') = '\0'; + strlcat(val, "/lib", sizeof (zo->zo_alt_libpath)); + + if (0 != access(zo->zo_alt_libpath, X_OK)) + goto invalid; + return; - umem_free(cmd, MAXPATHLEN); - umem_free(realaltdir, MAXPATHLEN); +invalid: + ztest_dump_core = B_FALSE; + fatal(B_TRUE, "invalid alternate %s %s", invalid_what, val); } } @@ -1182,14 +1161,14 @@ ztest_kill(ztest_shared_t *zs) zs->zs_space = metaslab_class_get_space(spa_normal_class(ztest_spa)); /* - * Before we kill off ztest, make sure that the config is updated. + * Before we kill ourselves, make sure that the config is updated. * See comment above spa_write_cachefile(). */ mutex_enter(&spa_namespace_lock); spa_write_cachefile(ztest_spa, B_FALSE, B_FALSE); mutex_exit(&spa_namespace_lock); - (void) kill(getpid(), SIGKILL); + (void) raise(SIGKILL); } static void @@ -1221,30 +1200,31 @@ ztest_is_draid_spare(const char *name) } static nvlist_t * -make_vdev_file(char *path, char *aux, char *pool, size_t size, uint64_t ashift) +make_vdev_file(const char *path, const char *aux, const char *pool, + size_t size, uint64_t ashift) { - char *pathbuf; + char *pathbuf = NULL; uint64_t vdev; nvlist_t *file; boolean_t draid_spare = B_FALSE; - pathbuf = umem_alloc(MAXPATHLEN, UMEM_NOFAIL); if (ashift == 0) ashift = ztest_get_ashift(); if (path == NULL) { + pathbuf = umem_alloc(MAXPATHLEN, UMEM_NOFAIL); path = pathbuf; if (aux != NULL) { vdev = ztest_shared->zs_vdev_aux; - (void) snprintf(path, MAXPATHLEN, + (void) snprintf(pathbuf, MAXPATHLEN, ztest_aux_template, ztest_opts.zo_dir, pool == NULL ? ztest_opts.zo_pool : pool, aux, vdev); } else { vdev = ztest_shared->zs_vdev_next_leaf++; - (void) snprintf(path, MAXPATHLEN, + (void) snprintf(pathbuf, MAXPATHLEN, ztest_dev_template, ztest_opts.zo_dir, pool == NULL ? ztest_opts.zo_pool : pool, vdev); } @@ -1272,7 +1252,7 @@ make_vdev_file(char *path, char *aux, char *pool, size_t size, uint64_t ashift) } static nvlist_t * -make_vdev_raid(char *path, char *aux, char *pool, size_t size, +make_vdev_raid(const char *path, const char *aux, const char *pool, size_t size, uint64_t ashift, int r) { nvlist_t *raid, **child; @@ -1323,8 +1303,8 @@ make_vdev_raid(char *path, char *aux, char *pool, size_t size, } static nvlist_t * -make_vdev_mirror(char *path, char *aux, char *pool, size_t size, - uint64_t ashift, int r, int m) +make_vdev_mirror(const char *path, const char *aux, const char *pool, + size_t size, uint64_t ashift, int r, int m) { nvlist_t *mirror, **child; int c; @@ -1351,8 +1331,8 @@ make_vdev_mirror(char *path, char *aux, char *pool, size_t size, } static nvlist_t * -make_vdev_root(char *path, char *aux, char *pool, size_t size, uint64_t ashift, - const char *class, int r, int m, int t) +make_vdev_root(const char *path, const char *aux, const char *pool, size_t size, + uint64_t ashift, const char *class, int r, int m, int t) { nvlist_t *root, **child; int c; @@ -1557,7 +1537,7 @@ ztest_spa_prop_set_uint64(zpool_prop_t prop, uint64_t value) static int ztest_dmu_objset_own(const char *name, dmu_objset_type_t type, - boolean_t readonly, boolean_t decrypt, void *tag, objset_t **osp) + boolean_t readonly, boolean_t decrypt, const void *tag, objset_t **osp) { int err; char *cp = NULL; @@ -1887,7 +1867,7 @@ ztest_log_create(ztest_ds_t *zd, dmu_tx_t *tx, lr_create_t *lr) return; itx = zil_itx_create(TX_CREATE, sizeof (*lr) + namesize); - bcopy(&lr->lr_common + 1, &itx->itx_lr + 1, + memcpy(&itx->itx_lr + 1, &lr->lr_common + 1, sizeof (*lr) + namesize - sizeof (lr_t)); zil_itx_assign(zd->zd_zilog, itx, tx); @@ -1904,7 +1884,7 @@ ztest_log_remove(ztest_ds_t *zd, dmu_tx_t *tx, lr_remove_t *lr, uint64_t object) return; itx = zil_itx_create(TX_REMOVE, sizeof (*lr) + namesize); - bcopy(&lr->lr_common + 1, &itx->itx_lr + 1, + memcpy(&itx->itx_lr + 1, &lr->lr_common + 1, sizeof (*lr) + namesize - sizeof (lr_t)); itx->itx_oid = object; @@ -1937,7 +1917,7 @@ ztest_log_write(ztest_ds_t *zd, dmu_tx_t *tx, lr_write_t *lr) itx->itx_wr_state = write_state; itx->itx_sync = (ztest_random(8) == 0); - bcopy(&lr->lr_common + 1, &itx->itx_lr + 1, + memcpy(&itx->itx_lr + 1, &lr->lr_common + 1, sizeof (*lr) - sizeof (lr_t)); zil_itx_assign(zd->zd_zilog, itx, tx); @@ -1952,7 +1932,7 @@ ztest_log_truncate(ztest_ds_t *zd, dmu_tx_t *tx, lr_truncate_t *lr) return; itx = zil_itx_create(TX_TRUNCATE, sizeof (*lr)); - bcopy(&lr->lr_common + 1, &itx->itx_lr + 1, + memcpy(&itx->itx_lr + 1, &lr->lr_common + 1, sizeof (*lr) - sizeof (lr_t)); itx->itx_sync = B_FALSE; @@ -1968,7 +1948,7 @@ ztest_log_setattr(ztest_ds_t *zd, dmu_tx_t *tx, lr_setattr_t *lr) return; itx = zil_itx_create(TX_SETATTR, sizeof (*lr)); - bcopy(&lr->lr_common + 1, &itx->itx_lr + 1, + memcpy(&itx->itx_lr + 1, &lr->lr_common + 1, sizeof (*lr) - sizeof (lr_t)); itx->itx_sync = B_FALSE; @@ -2233,7 +2213,7 @@ ztest_replay_write(void *arg1, void *arg2, boolean_t byteswap) if (abuf == NULL) { dmu_write(os, lr->lr_foid, offset, length, data, tx); } else { - bcopy(data, abuf->b_data, length); + memcpy(abuf->b_data, data, length); dmu_assign_arcbuf_by_dbuf(db, offset, abuf, tx); } @@ -2386,6 +2366,7 @@ zil_replay_func_t *ztest_replay_vector[TX_MAX_TYPE] = { NULL, /* TX_MKDIR_ATTR */ NULL, /* TX_MKDIR_ACL_ATTR */ NULL, /* TX_WRITE2 */ + NULL, /* TX_SETSAXATTR */ }; /* @@ -2505,7 +2486,7 @@ ztest_lr_alloc(size_t lrsize, char *name) lr = umem_zalloc(lrsize + namesize, UMEM_NOFAIL); if (name) - bcopy(name, lr + lrsize, namesize); + memcpy(lr + lrsize, name, namesize); return (lr); } @@ -2662,7 +2643,7 @@ ztest_write(ztest_ds_t *zd, uint64_t object, uint64_t offset, uint64_t size, lr->lr_blkoff = 0; BP_ZERO(&lr->lr_blkptr); - bcopy(data, lr + 1, size); + memcpy(lr + 1, data, size); error = ztest_replay_write(zd, lr, B_FALSE); @@ -2786,7 +2767,7 @@ ztest_io(ztest_ds_t *zd, uint64_t object, uint64_t offset) break; case ZTEST_IO_WRITE_ZEROES: - bzero(data, blocksize); + memset(data, 0, blocksize); (void) ztest_write(zd, object, offset, blocksize, data); break; @@ -2829,7 +2810,7 @@ ztest_io(ztest_ds_t *zd, uint64_t object, uint64_t offset) * Initialize an object description template. */ static void -ztest_od_init(ztest_od_t *od, uint64_t id, char *tag, uint64_t index, +ztest_od_init(ztest_od_t *od, uint64_t id, const char *tag, uint64_t index, dmu_object_type_t type, uint64_t blocksize, uint64_t dnodesize, uint64_t gen) { @@ -2927,7 +2908,7 @@ ztest_zil_remount(ztest_ds_t *zd, uint64_t id) zil_close(zd->zd_zilog); /* zfsvfs_setup() */ - VERIFY3P(zil_open(os, ztest_get_data), ==, zd->zd_zilog); + VERIFY3P(zil_open(os, ztest_get_data, NULL), ==, zd->zd_zilog); zil_replay(os, zd, ztest_replay_vector); (void) pthread_rwlock_unlock(&zd->zd_zilog_lock); @@ -3391,7 +3372,7 @@ ztest_vdev_aux_add_remove(ztest_ds_t *zd, uint64_t id) spa_t *spa = ztest_spa; vdev_t *rvd = spa->spa_root_vdev; spa_aux_vdev_t *sav; - char *aux; + const char *aux; char *path; uint64_t guid = 0; int error, ignore_err = 0; @@ -4272,7 +4253,15 @@ ztest_objset_destroy_cb(const char *name, void *arg) * Destroy the dataset. */ if (strchr(name, '@') != NULL) { - VERIFY0(dsl_destroy_snapshot(name, B_TRUE)); + error = dsl_destroy_snapshot(name, B_TRUE); + if (error != ECHRNG) { + /* + * The program was executed, but encountered a runtime + * error, such as insufficient slop, or a hold on the + * dataset. + */ + ASSERT0(error); + } } else { error = dsl_destroy_head(name); if (error == ENOSPC) { @@ -4389,7 +4378,7 @@ ztest_dmu_objset_create_destroy(ztest_ds_t *zd, uint64_t id) /* * Open the intent log for it. */ - zilog = zil_open(os, ztest_get_data); + zilog = zil_open(os, ztest_get_data, NULL); /* * Put some objects in there, do a little I/O to them, @@ -4835,16 +4824,16 @@ ztest_dmu_read_write(ztest_ds_t *zd, uint64_t id) "got %"PRIx64", wanted %"PRIx64"+%"PRIx64"", pack->bw_index, n, i); - if (bcmp(pack, bigH, sizeof (bufwad_t)) != 0) + if (memcmp(pack, bigH, sizeof (bufwad_t)) != 0) fatal(B_FALSE, "pack/bigH mismatch in %p/%p", pack, bigH); - if (bcmp(pack, bigT, sizeof (bufwad_t)) != 0) + if (memcmp(pack, bigT, sizeof (bufwad_t)) != 0) fatal(B_FALSE, "pack/bigT mismatch in %p/%p", pack, bigT); if (freeit) { - bzero(pack, sizeof (bufwad_t)); + memset(pack, 0, sizeof (bufwad_t)); } else { pack->bw_index = n + i; pack->bw_txg = txg; @@ -4890,8 +4879,8 @@ ztest_dmu_read_write(ztest_ds_t *zd, uint64_t id) VERIFY0(dmu_read(os, bigobj, bigoff, bigsize, bigcheck, DMU_READ_PREFETCH)); - ASSERT0(bcmp(packbuf, packcheck, packsize)); - ASSERT0(bcmp(bigbuf, bigcheck, bigsize)); + ASSERT0(memcmp(packbuf, packcheck, packsize)); + ASSERT0(memcmp(bigbuf, bigcheck, bigsize)); umem_free(packcheck, packsize); umem_free(bigcheck, bigsize); @@ -4938,11 +4927,11 @@ compare_and_update_pbbufs(uint64_t s, bufwad_t *packbuf, bufwad_t *bigbuf, "got %"PRIx64", wanted %"PRIx64"+%"PRIx64"", pack->bw_index, n, i); - if (bcmp(pack, bigH, sizeof (bufwad_t)) != 0) + if (memcmp(pack, bigH, sizeof (bufwad_t)) != 0) fatal(B_FALSE, "pack/bigH mismatch in %p/%p", pack, bigH); - if (bcmp(pack, bigT, sizeof (bufwad_t)) != 0) + if (memcmp(pack, bigT, sizeof (bufwad_t)) != 0) fatal(B_FALSE, "pack/bigT mismatch in %p/%p", pack, bigT); @@ -5130,15 +5119,16 @@ ztest_dmu_read_write_zcopy(ztest_ds_t *zd, uint64_t id) for (off = bigoff, j = 0; j < s; j++, off += chunksize) { dmu_buf_t *dbt; if (i != 5 || chunksize < (SPA_MINBLOCKSIZE * 2)) { - bcopy((caddr_t)bigbuf + (off - bigoff), - bigbuf_arcbufs[j]->b_data, chunksize); + memcpy(bigbuf_arcbufs[j]->b_data, + (caddr_t)bigbuf + (off - bigoff), + chunksize); } else { - bcopy((caddr_t)bigbuf + (off - bigoff), - bigbuf_arcbufs[2 * j]->b_data, + memcpy(bigbuf_arcbufs[2 * j]->b_data, + (caddr_t)bigbuf + (off - bigoff), chunksize / 2); - bcopy((caddr_t)bigbuf + (off - bigoff) + + memcpy(bigbuf_arcbufs[2 * j + 1]->b_data, + (caddr_t)bigbuf + (off - bigoff) + chunksize / 2, - bigbuf_arcbufs[2 * j + 1]->b_data, chunksize / 2); } @@ -5174,8 +5164,8 @@ ztest_dmu_read_write_zcopy(ztest_ds_t *zd, uint64_t id) VERIFY0(dmu_read(os, bigobj, bigoff, bigsize, bigcheck, DMU_READ_PREFETCH)); - ASSERT0(bcmp(packbuf, packcheck, packsize)); - ASSERT0(bcmp(bigbuf, bigcheck, bigsize)); + ASSERT0(memcmp(packbuf, packcheck, packsize)); + ASSERT0(memcmp(bigbuf, bigcheck, bigsize)); umem_free(packcheck, packsize); umem_free(bigcheck, bigsize); @@ -5282,7 +5272,7 @@ ztest_zap(ztest_ds_t *zd, uint64_t id) dmu_tx_t *tx; char propname[100], txgname[100]; int error; - char *hc[2] = { "s.acl.h", ".s.open.h.hyLZlg" }; + const char *const hc[2] = { "s.acl.h", ".s.open.h.hyLZlg" }; od = umem_alloc(sizeof (ztest_od_t), UMEM_NOFAIL); ztest_od_init(od, id, FTAG, 0, DMU_OT_ZAP_OTHER, 0, 0, 0); @@ -5328,7 +5318,7 @@ ztest_zap(ztest_ds_t *zd, uint64_t id) prop = ztest_random(ZTEST_ZAP_MAX_PROPS); (void) sprintf(propname, "prop_%"PRIu64"", prop); (void) sprintf(txgname, "txg_%"PRIu64"", prop); - bzero(value, sizeof (value)); + memset(value, 0, sizeof (value)); last_txg = 0; /* @@ -5520,11 +5510,11 @@ ztest_zap_parallel(ztest_ds_t *zd, uint64_t id) umem_free(od, sizeof (ztest_od_t)); return; } - bcopy(name, string_value, namelen); + memcpy(string_value, name, namelen); } else { tx = NULL; txg = 0; - bzero(string_value, namelen); + memset(string_value, 0, namelen); } switch (i) { @@ -5543,7 +5533,7 @@ ztest_zap_parallel(ztest_ds_t *zd, uint64_t id) error = zap_lookup(os, object, name, wsize, wc, data); if (error == 0) { if (data == string_value && - bcmp(name, data, namelen) != 0) + memcmp(name, data, namelen) != 0) fatal(B_FALSE, "name '%s' != val '%s' len %d", name, (char *)data, namelen); } else { @@ -6387,6 +6377,92 @@ ztest_reguid(ztest_ds_t *zd, uint64_t id) VERIFY3U(load, ==, spa_load_guid(spa)); } +void +ztest_blake3(ztest_ds_t *zd, uint64_t id) +{ + (void) zd, (void) id; + hrtime_t end = gethrtime() + NANOSEC; + zio_cksum_salt_t salt; + void *salt_ptr = &salt.zcs_bytes; + struct abd *abd_data, *abd_meta; + void *buf, *templ; + int i, *ptr; + uint32_t size; + BLAKE3_CTX ctx; + + size = ztest_random_blocksize(); + buf = umem_alloc(size, UMEM_NOFAIL); + abd_data = abd_alloc(size, B_FALSE); + abd_meta = abd_alloc(size, B_TRUE); + + for (i = 0, ptr = buf; i < size / sizeof (*ptr); i++, ptr++) + *ptr = ztest_random(UINT_MAX); + memset(salt_ptr, 'A', 32); + + abd_copy_from_buf_off(abd_data, buf, 0, size); + abd_copy_from_buf_off(abd_meta, buf, 0, size); + + while (gethrtime() <= end) { + int run_count = 100; + zio_cksum_t zc_ref1, zc_ref2; + zio_cksum_t zc_res1, zc_res2; + + void *ref1 = &zc_ref1; + void *ref2 = &zc_ref2; + void *res1 = &zc_res1; + void *res2 = &zc_res2; + + /* BLAKE3_KEY_LEN = 32 */ + VERIFY0(blake3_set_impl_name("generic")); + templ = abd_checksum_blake3_tmpl_init(&salt); + Blake3_InitKeyed(&ctx, salt_ptr); + Blake3_Update(&ctx, buf, size); + Blake3_Final(&ctx, ref1); + zc_ref2 = zc_ref1; + ZIO_CHECKSUM_BSWAP(&zc_ref2); + abd_checksum_blake3_tmpl_free(templ); + + VERIFY0(blake3_set_impl_name("cycle")); + while (run_count-- > 0) { + + /* Test current implementation */ + Blake3_InitKeyed(&ctx, salt_ptr); + Blake3_Update(&ctx, buf, size); + Blake3_Final(&ctx, res1); + zc_res2 = zc_res1; + ZIO_CHECKSUM_BSWAP(&zc_res2); + + VERIFY0(memcmp(ref1, res1, 32)); + VERIFY0(memcmp(ref2, res2, 32)); + + /* Test ABD - data */ + templ = abd_checksum_blake3_tmpl_init(&salt); + abd_checksum_blake3_native(abd_data, size, + templ, &zc_res1); + abd_checksum_blake3_byteswap(abd_data, size, + templ, &zc_res2); + + VERIFY0(memcmp(ref1, res1, 32)); + VERIFY0(memcmp(ref2, res2, 32)); + + /* Test ABD - metadata */ + abd_checksum_blake3_native(abd_meta, size, + templ, &zc_res1); + abd_checksum_blake3_byteswap(abd_meta, size, + templ, &zc_res2); + abd_checksum_blake3_tmpl_free(templ); + + VERIFY0(memcmp(ref1, res1, 32)); + VERIFY0(memcmp(ref2, res2, 32)); + + } + } + + abd_free(abd_data); + abd_free(abd_meta); + umem_free(buf, size); +} + void ztest_fletcher(ztest_ds_t *zd, uint64_t id) { @@ -6427,8 +6503,8 @@ ztest_fletcher(ztest_ds_t *zd, uint64_t id) fletcher_4_byteswap(buf, size, NULL, &zc_byteswap); fletcher_4_native(buf, size, NULL, &zc); - VERIFY0(bcmp(&zc, &zc_ref, sizeof (zc))); - VERIFY0(bcmp(&zc_byteswap, &zc_ref_byteswap, + VERIFY0(memcmp(&zc, &zc_ref, sizeof (zc))); + VERIFY0(memcmp(&zc_byteswap, &zc_ref_byteswap, sizeof (zc_byteswap))); /* Test ABD - data */ @@ -6436,8 +6512,8 @@ ztest_fletcher(ztest_ds_t *zd, uint64_t id) &zc_byteswap); abd_fletcher_4_native(abd_data, size, NULL, &zc); - VERIFY0(bcmp(&zc, &zc_ref, sizeof (zc))); - VERIFY0(bcmp(&zc_byteswap, &zc_ref_byteswap, + VERIFY0(memcmp(&zc, &zc_ref, sizeof (zc))); + VERIFY0(memcmp(&zc_byteswap, &zc_ref_byteswap, sizeof (zc_byteswap))); /* Test ABD - metadata */ @@ -6445,8 +6521,8 @@ ztest_fletcher(ztest_ds_t *zd, uint64_t id) &zc_byteswap); abd_fletcher_4_native(abd_meta, size, NULL, &zc); - VERIFY0(bcmp(&zc, &zc_ref, sizeof (zc))); - VERIFY0(bcmp(&zc_byteswap, &zc_ref_byteswap, + VERIFY0(memcmp(&zc, &zc_ref, sizeof (zc))); + VERIFY0(memcmp(&zc_byteswap, &zc_ref_byteswap, sizeof (zc_byteswap))); } @@ -6561,11 +6637,8 @@ ztest_global_vars_to_zdb_args(void) char **args = calloc(2*ztest_opts.zo_gvars_count + 1, sizeof (char *)); char **cur = args; for (size_t i = 0; i < ztest_opts.zo_gvars_count; i++) { - char *kv = ztest_opts.zo_gvars[i]; - *cur = "-o"; - cur++; - *cur = strdup(kv); - cur++; + *cur++ = (char *)"-o"; + *cur++ = ztest_opts.zo_gvars[i]; } ASSERT3P(cur, ==, &args[2*ztest_opts.zo_gvars_count]); *cur = NULL; @@ -6616,22 +6689,22 @@ ztest_get_zdb_bin(char *bin, int len) { char *zdb_path; /* - * Try to use ZDB_PATH and in-tree zdb path. If not successful, just + * Try to use $ZDB and in-tree zdb path. If not successful, just * let popen to search through PATH. */ - if ((zdb_path = getenv("ZDB_PATH"))) { + if ((zdb_path = getenv("ZDB"))) { strlcpy(bin, zdb_path, len); /* In env */ if (!ztest_check_path(bin)) { ztest_dump_core = 0; - fatal(B_TRUE, "invalid ZDB_PATH '%s'", bin); + fatal(B_TRUE, "invalid ZDB '%s'", bin); } return; } VERIFY3P(realpath(getexecname(), bin), !=, NULL); - if (strstr(bin, "/ztest/")) { - strstr(bin, "/ztest/")[0] = '\0'; /* In-tree */ - strcat(bin, "/zdb/zdb"); + if (strstr(bin, ".libs/ztest")) { + strstr(bin, ".libs/ztest")[0] = '\0'; /* In-tree */ + strcat(bin, "zdb"); if (ztest_check_path(bin)) return; } @@ -6816,7 +6889,7 @@ ztest_trim(ztest_ds_t *zd, uint64_t id) * Verify pool integrity by running zdb. */ static void -ztest_run_zdb(char *pool) +ztest_run_zdb(const char *pool) { int status; char *bin; @@ -6848,7 +6921,7 @@ ztest_run_zdb(char *pool) free(set_gvars_args_joined); if (ztest_opts.zo_verbose >= 5) - (void) printf("Executing %s\n", strstr(zdb, "zdb ")); + (void) printf("Executing %s\n", zdb); fp = popen(zdb, "r"); @@ -6874,12 +6947,12 @@ ztest_run_zdb(char *pool) } static void -ztest_walk_pool_directory(char *header) +ztest_walk_pool_directory(const char *header) { spa_t *spa = NULL; if (ztest_opts.zo_verbose >= 6) - (void) printf("%s\n", header); + (void) puts(header); mutex_enter(&spa_namespace_lock); while ((spa = spa_next(spa)) != NULL) @@ -6984,7 +7057,7 @@ ztest_resume(spa_t *spa) (void) zio_resume(spa); } -static void +static __attribute__((noreturn)) void ztest_resume_thread(void *arg) { spa_t *spa = arg; @@ -7010,7 +7083,7 @@ ztest_resume_thread(void *arg) thread_exit(); } -static void +static __attribute__((noreturn)) void ztest_deadman_thread(void *arg) { ztest_shared_t *zs = arg; @@ -7088,7 +7161,7 @@ ztest_execute(int test, ztest_info_t *zi, uint64_t id) (double)functime / NANOSEC, zi->zi_funcname); } -static void +static __attribute__((noreturn)) void ztest_thread(void *arg) { int rand; @@ -7131,7 +7204,7 @@ ztest_thread(void *arg) } static void -ztest_dataset_name(char *dsname, char *pool, int d) +ztest_dataset_name(char *dsname, const char *pool, int d) { (void) snprintf(dsname, ZFS_MAX_DATASET_NAME_LEN, "%s/ds_%d", pool, d); } @@ -7231,7 +7304,7 @@ ztest_dataset_open(int d) zilog->zl_parse_lr_count, zilog->zl_replaying_seq); - zilog = zil_open(os, ztest_get_data); + zilog = zil_open(os, ztest_get_data, NULL); if (zilog->zl_replaying_seq != 0 && zilog->zl_replaying_seq < committed_seq) @@ -7878,21 +7951,24 @@ exec_child(char *cmd, char *libpath, boolean_t ignorekill, int *statusp) fatal(B_TRUE, "fork failed"); if (pid == 0) { /* child */ - char *emptyargv[2] = { cmd, NULL }; char fd_data_str[12]; - struct rlimit rl = { 1024, 1024 }; - (void) setrlimit(RLIMIT_NOFILE, &rl); - - (void) close(ztest_fd_rand); VERIFY3S(11, >=, snprintf(fd_data_str, 12, "%d", ztest_fd_data)); VERIFY0(setenv("ZTEST_FD_DATA", fd_data_str, 1)); - (void) enable_extended_FILE_stdio(-1, -1); - if (libpath != NULL) - VERIFY0(setenv("LD_LIBRARY_PATH", libpath, 1)); - (void) execv(cmd, emptyargv); + if (libpath != NULL) { + const char *curlp = getenv("LD_LIBRARY_PATH"); + if (curlp == NULL) + VERIFY0(setenv("LD_LIBRARY_PATH", libpath, 1)); + else { + char *newlp = NULL; + VERIFY3S(-1, !=, + asprintf(&newlp, "%s:%s", libpath, curlp)); + VERIFY0(setenv("LD_LIBRARY_PATH", newlp, 1)); + } + } + (void) execl(cmd, cmd, (char *)NULL); ztest_dump_core = B_FALSE; fatal(B_TRUE, "exec failed: %s", cmd); } @@ -7951,7 +8027,7 @@ ztest_run_init(void) * Create and initialize our storage pool. */ for (i = 1; i <= ztest_opts.zo_init; i++) { - bzero(zs, sizeof (ztest_shared_t)); + memset(zs, 0, sizeof (*zs)); if (ztest_opts.zo_verbose >= 3 && ztest_opts.zo_init != 1) { (void) printf("ztest_init(), pass %d\n", i); @@ -8023,7 +8099,7 @@ main(int argc, char **argv) * ztest from needlessly depleting the system entropy pool. */ random_path = "/dev/urandom"; - ztest_fd_rand = open(random_path, O_RDONLY); + ztest_fd_rand = open(random_path, O_RDONLY | O_CLOEXEC); ASSERT3S(ztest_fd_rand, >=, 0); if (!fd_data_str) { @@ -8032,12 +8108,12 @@ main(int argc, char **argv) setup_data_fd(); setup_hdr(); setup_data(); - bcopy(&ztest_opts, ztest_shared_opts, + memcpy(ztest_shared_opts, &ztest_opts, sizeof (*ztest_shared_opts)); } else { ztest_fd_data = atoi(fd_data_str); setup_data(); - bcopy(ztest_shared_opts, &ztest_opts, sizeof (ztest_opts)); + memcpy(&ztest_opts, ztest_shared_opts, sizeof (ztest_opts)); } ASSERT3U(ztest_opts.zo_datasets, ==, ztest_shared_hdr->zh_ds_count); diff --git a/cmd/ztest/.gitignore b/cmd/ztest/.gitignore deleted file mode 100644 index d3d498dae693..000000000000 --- a/cmd/ztest/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/ztest diff --git a/cmd/ztest/Makefile.am b/cmd/ztest/Makefile.am deleted file mode 100644 index d5e335e6d27e..000000000000 --- a/cmd/ztest/Makefile.am +++ /dev/null @@ -1,25 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -# Get rid of compiler warning for unchecked truncating snprintfs on gcc 7.1.1 -AM_CFLAGS += $(NO_FORMAT_TRUNCATION) - -# Includes kernel code, generate warnings for large stack frames -AM_CFLAGS += $(FRAME_LARGER_THAN) - -# Unconditionally enable ASSERTs -AM_CPPFLAGS += -DDEBUG -UNDEBUG -DZFS_DEBUG - -sbin_PROGRAMS = ztest - -ztest_SOURCES = \ - ztest.c - -ztest_LDADD = \ - $(abs_top_builddir)/lib/libzpool/libzpool.la \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la - -ztest_LDADD += -lm -ztest_LDFLAGS = -pthread - -include $(top_srcdir)/config/CppCheck.am diff --git a/cmd/zvol_id/.gitignore b/cmd/zvol_id/.gitignore deleted file mode 100644 index 8b757a2d6781..000000000000 --- a/cmd/zvol_id/.gitignore +++ /dev/null @@ -1 +0,0 @@ -zvol_id diff --git a/cmd/zvol_id/Makefile.am b/cmd/zvol_id/Makefile.am deleted file mode 100644 index bb7e31a0590f..000000000000 --- a/cmd/zvol_id/Makefile.am +++ /dev/null @@ -1,12 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -# Disable GCC stack protection for zvol_id. This is a kludge and should be -# removed once https://github.com/openzfs/zfs/issues/569 is resolved. -AM_CFLAGS += -fno-stack-protector - -udev_PROGRAMS = zvol_id - -zvol_id_SOURCES = \ - zvol_id_main.c - -include $(top_srcdir)/config/CppCheck.am diff --git a/cmd/zvol_id/zvol_id_main.c b/cmd/zvol_id/zvol_id_main.c deleted file mode 100644 index 929a1a6e794d..000000000000 --- a/cmd/zvol_id/zvol_id_main.c +++ /dev/null @@ -1,129 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License (the "License"). - * You may not use this file except in compliance with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright (c) 2011, Fajar A. Nugraha. All rights reserved. - * Use is subject to license terms. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#if defined(ZFS_ASAN_ENABLED) -/* - * zvol_id is invoked by udev with the help of ptrace() - * making sanitized binary with leak detection croak - * because of tracing mechanisms collision - */ -extern const char *__asan_default_options(void); - -const char *__asan_default_options(void) { - return ("abort_on_error=true:halt_on_error=true:" - "allocator_may_return_null=true:disable_coredump=false:" - "detect_stack_use_after_return=true:detect_leaks=false"); -} -#endif - -static int -ioctl_get_msg(char *var, int fd) -{ - int ret; - char msg[ZFS_MAX_DATASET_NAME_LEN]; - - ret = ioctl(fd, BLKZNAME, msg); - if (ret < 0) { - return (ret); - } - - snprintf(var, ZFS_MAX_DATASET_NAME_LEN, "%s", msg); - return (ret); -} - -int -main(int argc, char **argv) -{ - int fd = -1, ret = 0, status = EXIT_FAILURE; - char zvol_name[ZFS_MAX_DATASET_NAME_LEN]; - char *zvol_name_part = NULL; - char *dev_name; - struct stat64 statbuf; - int dev_minor, dev_part; - int i; - - if (argc < 2) { - fprintf(stderr, "Usage: %s /dev/zvol_device_node\n", argv[0]); - goto fail; - } - - dev_name = argv[1]; - ret = stat64(dev_name, &statbuf); - if (ret != 0) { - fprintf(stderr, "Unable to access device file: %s\n", dev_name); - goto fail; - } - - dev_minor = minor(statbuf.st_rdev); - dev_part = dev_minor % ZVOL_MINORS; - - fd = open(dev_name, O_RDONLY); - if (fd < 0) { - fprintf(stderr, "Unable to open device file: %s\n", dev_name); - goto fail; - } - - ret = ioctl_get_msg(zvol_name, fd); - if (ret < 0) { - fprintf(stderr, "ioctl_get_msg failed: %s\n", strerror(errno)); - goto fail; - } - if (dev_part > 0) - ret = asprintf(&zvol_name_part, "%s-part%d", zvol_name, - dev_part); - else - ret = asprintf(&zvol_name_part, "%s", zvol_name); - - if (ret == -1 || zvol_name_part == NULL) - goto fail; - - for (i = 0; i < strlen(zvol_name_part); i++) { - if (isblank(zvol_name_part[i])) - zvol_name_part[i] = '+'; - } - - printf("%s\n", zvol_name_part); - status = EXIT_SUCCESS; - -fail: - if (zvol_name_part) - free(zvol_name_part); - if (fd >= 0) - close(fd); - - return (status); -} diff --git a/cmd/zvol_wait/zvol_wait b/cmd/zvol_wait similarity index 87% rename from cmd/zvol_wait/zvol_wait rename to cmd/zvol_wait index 2aa929b0ca2b..f1fa42e27dc9 100755 --- a/cmd/zvol_wait/zvol_wait +++ b/cmd/zvol_wait @@ -28,15 +28,17 @@ filter_out_deleted_zvols() { list_zvols() { read -r default_volmode < /sys/module/zfs/parameters/zvol_volmode zfs list -t volume -H -o \ - name,volmode,receive_resume_token,redact_snaps | - while IFS=" " read -r name volmode token redacted; do # IFS=\t here! + name,volmode,receive_resume_token,redact_snaps,keystatus | + while IFS=" " read -r name volmode token redacted keystatus; do # IFS=\t here! - # /dev links are not created for zvols with volmode = "none" - # or for redacted zvols. + # /dev links are not created for zvols with volmode = "none", + # redacted zvols, or encrypted zvols for which the key has not + # been loaded. [ "$volmode" = "none" ] && continue [ "$volmode" = "default" ] && [ "$default_volmode" = "3" ] && continue [ "$redacted" = "-" ] || continue + [ "$keystatus" = "unavailable" ] && continue # We also ignore partially received zvols if it is # not an incremental receive, as those won't even have a block diff --git a/cmd/zvol_wait/Makefile.am b/cmd/zvol_wait/Makefile.am deleted file mode 100644 index e8b546a60659..000000000000 --- a/cmd/zvol_wait/Makefile.am +++ /dev/null @@ -1,4 +0,0 @@ -include $(top_srcdir)/config/Shellcheck.am - -dist_bin_SCRIPTS = zvol_wait - diff --git a/config/Abigail.am b/config/Abigail.am deleted file mode 100644 index 94687b90eef2..000000000000 --- a/config/Abigail.am +++ /dev/null @@ -1,33 +0,0 @@ -# -# When performing an ABI check the following options are applied: -# -# --no-unreferenced-symbols: Exclude symbols which are not referenced by -# any debug information. Without this _init() and _fini() are incorrectly -# reported on CentOS7 for libuutil.so. -# -# --headers-dir1: Limit ABI checks to public OpenZFS headers, otherwise -# changes in public system headers are also reported. -# -# --suppressions: Honor a suppressions file for each library to provide -# a mechanism for suppressing harmless warnings. -# - -PHONY += checkabi storeabi - -checkabi: - for lib in $(lib_LTLIBRARIES) ; do \ - abidiff --no-unreferenced-symbols \ - --headers-dir1 ../../include \ - --suppressions $${lib%.la}.suppr \ - $${lib%.la}.abi .libs/$${lib%.la}.so ; \ - done - -storeabi: - cd .libs ; \ - for lib in $(lib_LTLIBRARIES) ; do \ - abidw --no-show-locs \ - --no-corpus-path \ - --no-comp-dir-path \ - --type-id-style hash \ - $${lib%.la}.so > ../$${lib%.la}.abi ; \ - done diff --git a/config/CppCheck.am b/config/CppCheck.am index e53013bd01cc..89a067d814ff 100644 --- a/config/CppCheck.am +++ b/config/CppCheck.am @@ -1,5 +1,5 @@ # -# Default rules for running cppcheck against the user space components. +# cppcheck for userspace – nodist_*_SOURCES are kernel code and cppcheck goes crazy on them. # PHONY += cppcheck @@ -7,5 +7,15 @@ PHONY += cppcheck CPPCHECKFLAGS = --std=c99 --quiet --max-configs=1 --error-exitcode=2 CPPCHECKFLAGS += --inline-suppr -U_KERNEL -cppcheck: - $(CPPCHECK) -j$(CPU_COUNT) $(CPPCHECKFLAGS) $(DEFAULT_INCLUDES) $(SOURCES) +CPPCHECKDIRS = +CPPCHECKTARGETS = + +cppcheck-recursive-%: + $(MAKE) -C $(subst cppcheck-recursive-,,$@) cppcheck + +_CTGT = $(subst cppcheck-for-,,$@) +cppcheck-for-%: + @[ -n "$($(_CTGT)_SOURCES)$(dist_$(_CTGT)_SOURCES)" ] + $(CPPCHECK) -j$(CPU_COUNT) $(CPPCHECKFLAGS) $(patsubst -U%,,$(patsubst -D%,,$(filter-out $(AM_CPPFLAGS_NOCHECK),$(or $($(_CTGT)_CPPFLAGS),$(AM_CPPFLAGS))))) $($(_CTGT)_SOURCES) $(dist_$(_CTGT)_SOURCES) + +cppcheck: $(addprefix cppcheck-for-,$(subst -,_,$(subst .,_,$(subst /,_,$(CPPCHECKTARGETS))))) $(addprefix cppcheck-recursive-,$(CPPCHECKDIRS)) diff --git a/config/Rules.am b/config/Rules.am index 3146da6ee1d2..7162b771869d 100644 --- a/config/Rules.am +++ b/config/Rules.am @@ -4,26 +4,17 @@ # PHONY = -DEFAULT_INCLUDES = \ +AM_CPPFLAGS = \ -include $(top_builddir)/zfs_config.h \ -I$(top_builddir)/include \ -I$(top_srcdir)/include \ -I$(top_srcdir)/module/icp/include \ - -I$(top_srcdir)/lib/libspl/include - -if BUILD_LINUX -DEFAULT_INCLUDES += \ - -I$(top_srcdir)/lib/libspl/include/os/linux -endif - -if BUILD_FREEBSD -DEFAULT_INCLUDES += \ - -I$(top_srcdir)/lib/libspl/include/os/freebsd -endif + -I$(top_srcdir)/lib/libspl/include \ + -I$(top_srcdir)/lib/libspl/include/os/@ac_system_l@ AM_LIBTOOLFLAGS = --silent -AM_CFLAGS = -std=gnu99 -Wall -Wstrict-prototypes -Wmissing-prototypes +AM_CFLAGS = -std=gnu99 -Wall -Wextra -Wstrict-prototypes -Wmissing-prototypes -Wwrite-strings -Wno-sign-compare -Wno-missing-field-initializers AM_CFLAGS += -fno-strict-aliasing AM_CFLAGS += $(NO_OMIT_FRAME_POINTER) AM_CFLAGS += $(IMPLICIT_FALLTHROUGH) @@ -37,7 +28,7 @@ AM_CFLAGS += -include $(top_srcdir)/include/os/freebsd/spl/sys/ccompile.h AM_CFLAGS += -I/usr/include -I/usr/local/include endif -AM_CPPFLAGS = -D_GNU_SOURCE +AM_CPPFLAGS += -D_GNU_SOURCE AM_CPPFLAGS += -D_REENTRANT AM_CPPFLAGS += -D_FILE_OFFSET_BITS=64 AM_CPPFLAGS += -D_LARGEFILE64_SOURCE @@ -48,16 +39,20 @@ AM_CPPFLAGS += -DSYSCONFDIR=\"$(sysconfdir)\" AM_CPPFLAGS += -DPKGDATADIR=\"$(pkgdatadir)\" AM_CPPFLAGS += $(DEBUG_CPPFLAGS) AM_CPPFLAGS += $(CODE_COVERAGE_CPPFLAGS) -if BUILD_LINUX -AM_CPPFLAGS += -DTEXT_DOMAIN=\"zfs-linux-user\" -endif -if BUILD_FREEBSD -AM_CPPFLAGS += -DTEXT_DOMAIN=\"zfs-freebsd-user\" -endif -AM_CPPFLAGS += -D"strtok(...)=strtok(__VA_ARGS__) __attribute__((deprecated(\"Use strtok_r(3) instead!\")))" -AM_CPPFLAGS += -D"__xpg_basename(...)=__xpg_basename(__VA_ARGS__) __attribute__((deprecated(\"basename(3) is underspecified. Use zfs_basename() instead!\")))" -AM_CPPFLAGS += -D"basename(...)=basename(__VA_ARGS__) __attribute__((deprecated(\"basename(3) is underspecified. Use zfs_basename() instead!\")))" -AM_CPPFLAGS += -D"dirname(...)=dirname(__VA_ARGS__) __attribute__((deprecated(\"dirname(3) is underspecified. Use zfs_dirnamelen() instead!\")))" +AM_CPPFLAGS += -DTEXT_DOMAIN=\"zfs-@ac_system_l@-user\" + +AM_CPPFLAGS_NOCHECK = -D"strtok(...)=strtok(__VA_ARGS__) __attribute__((deprecated(\"Use strtok_r(3) instead!\")))" +AM_CPPFLAGS_NOCHECK += -D"__xpg_basename(...)=__xpg_basename(__VA_ARGS__) __attribute__((deprecated(\"basename(3) is underspecified. Use zfs_basename() instead!\")))" +AM_CPPFLAGS_NOCHECK += -D"basename(...)=basename(__VA_ARGS__) __attribute__((deprecated(\"basename(3) is underspecified. Use zfs_basename() instead!\")))" +AM_CPPFLAGS_NOCHECK += -D"dirname(...)=dirname(__VA_ARGS__) __attribute__((deprecated(\"dirname(3) is underspecified. Use zfs_dirnamelen() instead!\")))" +AM_CPPFLAGS_NOCHECK += -D"bcopy(...)=__attribute__((deprecated(\"bcopy(3) is deprecated. Use memcpy(3)/memmove(3) instead!\"))) bcopy(__VA_ARGS__)" +AM_CPPFLAGS_NOCHECK += -D"bcmp(...)=__attribute__((deprecated(\"bcmp(3) is deprecated. Use memcmp(3) instead!\"))) bcmp(__VA_ARGS__)" +AM_CPPFLAGS_NOCHECK += -D"bzero(...)=__attribute__((deprecated(\"bzero(3) is deprecated. Use memset(3) instead!\"))) bzero(__VA_ARGS__)" +AM_CPPFLAGS_NOCHECK += -D"asctime(...)=__attribute__((deprecated(\"Use strftime(3) instead!\"))) asctime(__VA_ARGS__)" +AM_CPPFLAGS_NOCHECK += -D"asctime_r(...)=__attribute__((deprecated(\"Use strftime(3) instead!\"))) asctime_r(__VA_ARGS__)" +AM_CPPFLAGS_NOCHECK += -D"gmtime(...)=__attribute__((deprecated(\"gmtime(3) isn't thread-safe. Use gmtime_r(3) instead!\"))) gmtime(__VA_ARGS__)" +AM_CPPFLAGS_NOCHECK += -D"localtime(...)=__attribute__((deprecated(\"localtime(3) isn't thread-safe. Use localtime_r(3) instead!\"))) localtime(__VA_ARGS__)" +AM_CPPFLAGS += $(AM_CPPFLAGS_NOCHECK) if ASAN_ENABLED AM_CPPFLAGS += -DZFS_ASAN_ENABLED @@ -76,3 +71,13 @@ AM_LDFLAGS += -fstack-protector-strong -shared AM_LDFLAGS += -Wl,-x -Wl,--fatal-warnings -Wl,--warn-shared-textrel AM_LDFLAGS += -lm endif + + +# If a target includes kernel code, generate warnings for large stack frames +KERNEL_CFLAGS = $(FRAME_LARGER_THAN) + +# See https://debbugs.gnu.org/cgi/bugreport.cgi?bug=54020 +LIBRARY_CFLAGS = -no-suppress + +# Forcibly enable asserts/debugging for libzpool &al. +FORCEDEBUG_CPPFLAGS = -DDEBUG -UNDEBUG -DZFS_DEBUG diff --git a/config/Shellcheck.am b/config/Shellcheck.am index fb0bdedd3be3..878919575321 100644 --- a/config/Shellcheck.am +++ b/config/Shellcheck.am @@ -1,30 +1,40 @@ -.PHONY: shellcheck -shellcheck: $(SCRIPTS) $(SHELLCHECKSCRIPTS) - -# ShellCheck exclusions +# Global ShellCheck exclusions: # # ShellCheck can't follow non-constant source. Use a directive to specify location. [SC1090] # Not following: a was not specified as input (see shellcheck -x). [SC1091] # Prefer putting braces around variable references even when not strictly required. [SC2250] # In POSIX sh, 'local' is undefined. [SC2039] # older ShellCheck versions # In POSIX sh, 'local' is undefined. [SC3043] # newer ShellCheck versions + +SHELLCHECKSCRIPTS = + +JUST_SHELLCHECK_OPTS = $(addprefix shellcheck-here-,$(subst /,^,$(1))) +JUST_CHECKBASHISMS_OPTS = $(addprefix checkbashisms-here-,$(subst /,^,$(1))) +SHELLCHECK_OPTS = $(call JUST_SHELLCHECK_OPTS,$(1)) $(call JUST_CHECKBASHISMS_OPTS,$(1)) + +PHONY += shellcheck + +_STGT = $(subst ^,/,$(subst shellcheck-here-,,$@)) +shellcheck-here-%: if HAVE_SHELLCHECK - [ -z "$(SCRIPTS)$(SHELLCHECKSCRIPTS)" ] && exit; shellcheck --format=gcc --enable=all --exclude=SC1090,SC1091,SC2039,SC2250,SC3043 $$([ -n "$(SHELLCHECK_SHELL)" ] && echo "--shell=$(SHELLCHECK_SHELL)") $(SHELLCHECK_OPTS) $(SCRIPTS) $(SHELLCHECKSCRIPTS) + shellcheck --format=gcc --enable=all --exclude=SC1090,SC1091,SC2039,SC2250,SC3043 $$([ -n "$(SHELLCHECK_SHELL)" ] && echo "--shell=$(SHELLCHECK_SHELL)") "$$([ -e "$(_STGT)" ] || echo "$(srcdir)/")$(_STGT)" else - @[ -z "$(SCRIPTS)$(SHELLCHECKSCRIPTS)" ] && exit; echo "skipping shellcheck of" $(SCRIPTS) $(SHELLCHECKSCRIPTS) "because shellcheck is not installed" + @echo "skipping shellcheck of" $(_STGT) "because shellcheck is not installed" endif - @set -e; for dir in $(SHELLCHECKDIRS); do $(MAKE) -C $$dir shellcheck; done + +shellcheck: $(SHELLCHECKSCRIPTS) $(call JUST_SHELLCHECK_OPTS,$(SHELLCHECKSCRIPTS)) +PHONY += checkbashisms + # command -v *is* specified by POSIX and every shell in existence supports it -.PHONY: checkbashisms -checkbashisms: $(SCRIPTS) $(SHELLCHECKSCRIPTS) +_BTGT = $(subst ^,/,$(subst checkbashisms-here-,,$@)) +checkbashisms-here-%: if HAVE_CHECKBASHISMS - [ -z "$(SCRIPTS)$(SHELLCHECKSCRIPTS)" ] && exit; ! if [ -z "$(SHELLCHECK_SHELL)" ]; then \ - checkbashisms -npx $(SCRIPTS) $(SHELLCHECKSCRIPTS); else \ - for f in $(SCRIPTS) $(SHELLCHECKSCRIPTS); do echo $$f >&3; { echo '#!/bin/$(SHELLCHECK_SHELL)'; cat $$f; } | checkbashisms -npx; done; \ - fi 3>&2 2>&1 | grep -vFe "'command' with option other than -p" -e 'command -v' $(CHECKBASHISMS_IGNORE) >&2 + ! { [ -n "$(SHELLCHECK_SHELL)" ] && echo '#!/bin/$(SHELLCHECK_SHELL)'; cat "$$([ -e "$(_BTGT)" ] || echo "$(srcdir)/")$(_BTGT)"; } | \ + checkbashisms -npx 2>&1 | grep -vFe "'command' with option other than -p" -e 'command -v' -e 'any possible bashisms' $(CHECKBASHISMS_IGNORE) >&2 else - @[ -z "$(SCRIPTS)$(SHELLCHECKSCRIPTS)" ] && exit; echo "skipping checkbashisms of" $(SCRIPTS) $(SHELLCHECKSCRIPTS) "because checkbashisms is not installed" + @echo "skipping checkbashisms of" $(_BTGT) "because checkbashisms is not installed" endif - @set -e; for dir in $(SHELLCHECKDIRS); do $(MAKE) -C $$dir checkbashisms; done + +checkbashisms: $(SHELLCHECKSCRIPTS) $(call JUST_CHECKBASHISMS_OPTS,$(SHELLCHECKSCRIPTS)) diff --git a/config/Substfiles.am b/config/Substfiles.am index b051438fb9e3..38e870b2f501 100644 --- a/config/Substfiles.am +++ b/config/Substfiles.am @@ -1,38 +1,46 @@ subst_sed_cmd = \ + -e 's|@abs_top_srcdir[@]|$(abs_top_srcdir)|g' \ -e 's|@bindir[@]|$(bindir)|g' \ - -e 's|@sbindir[@]|$(sbindir)|g' \ -e 's|@datadir[@]|$(datadir)|g' \ - -e 's|@sysconfdir[@]|$(sysconfdir)|g' \ - -e 's|@runstatedir[@]|$(runstatedir)|g' \ -e 's|@initconfdir[@]|$(initconfdir)|g' \ -e 's|@initdir[@]|$(initdir)|g' \ -e 's|@mounthelperdir[@]|$(mounthelperdir)|g' \ + -e 's|@pammoduledir[@]|$(pammoduledir)|g' \ + -e 's|@runstatedir[@]|$(runstatedir)|g' \ + -e 's|@sbindir[@]|$(sbindir)|g' \ + -e 's|@sysconfdir[@]|$(sysconfdir)|g' \ -e 's|@systemdgeneratordir[@]|$(systemdgeneratordir)|g' \ -e 's|@systemdunitdir[@]|$(systemdunitdir)|g' \ -e 's|@udevdir[@]|$(udevdir)|g' \ -e 's|@udevruledir[@]|$(udevruledir)|g' \ -e 's|@zfsexecdir[@]|$(zfsexecdir)|g' \ - -e 's|@PYTHON[@]|$(PYTHON)|g' \ - -e 's|@PYTHON_SHEBANG[@]|$(PYTHON_SHEBANG)|g' \ + \ + -e 's|@ASAN_ENABLED[@]|$(ASAN_ENABLED)|g' \ -e 's|@DEFAULT_INIT_NFS_SERVER[@]|$(DEFAULT_INIT_NFS_SERVER)|g' \ -e 's|@DEFAULT_INIT_SHELL[@]|$(DEFAULT_INIT_SHELL)|g' \ -e 's|@LIBFETCH_DYNAMIC[@]|$(LIBFETCH_DYNAMIC)|g' \ -e 's|@LIBFETCH_SONAME[@]|$(LIBFETCH_SONAME)|g' \ - -e 's|@ASAN_ENABLED[@]|$(ASAN_ENABLED)|g' \ - -e 's|@UBSAN_ENABLED[@]|$(UBSAN_ENABLED)|g' - -SUBSTFILES = -CLEANFILES = $(SUBSTFILES) -EXTRA_DIST = $(SUBSTFILES:=.in) + -e 's|@PYTHON[@]|$(PYTHON)|g' \ + -e 's|@PYTHON_SHEBANG[@]|$(PYTHON_SHEBANG)|g' \ + -e 's|@UBSAN_ENABLED[@]|$(UBSAN_ENABLED)|g' \ + -e 's|@VERSION[@]|$(VERSION)|g' -$(SUBSTFILES):%:%.in Makefile - $(AM_V_GEN)set -e; \ - $(MKDIR_P) $$(dirname $@); \ - $(RM) $@~; \ - $(SED) $(subst_sed_cmd) $< >$@~; \ - if grep -E '@[a-zA-Z0-9_]+@' $@~ >&2; then \ +define SUBST +$(1) : $(2)$(1).in Makefile; + $$(AM_V_GEN)set -e; \ + $$(MKDIR_P) $$(@D); \ + $$(RM) $$@~; \ + $$(SED) $$(subst_sed_cmd) $$< >$$@~; \ + if grep -E '@[a-zA-Z0-9_]+@' $$@~ >&2; then \ echo "Undefined substitution" >&2; \ exit 1; \ - else test $$? -eq 1; fi; \ - test -x $< && chmod +x $@~; \ - mv -f $@~ $@ + fi; \ + [ -x $$< ] && chmod +x $$@~; \ + mv -f $$@~ $$@ +endef + +SUBSTFILES = +CLEANFILES += $(SUBSTFILES) +dist_noinst_DATA += $(SUBSTFILES:=.in) + +$(call SUBST,%,) diff --git a/config/always-arch.m4 b/config/always-arch.m4 index 25e8c963a4b4..f7090a4826ba 100644 --- a/config/always-arch.m4 +++ b/config/always-arch.m4 @@ -2,10 +2,6 @@ dnl # dnl # Set the target cpu architecture. This allows the dnl # following syntax to be used in a Makefile.am. dnl # -dnl # ifeq ($(TARGET_CPU),x86_64) -dnl # ... -dnl # endif -dnl # dnl # if TARGET_CPU_POWERPC dnl # ... dnl # else @@ -29,13 +25,13 @@ AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_ARCH], [ sparc64) TARGET_CPU=sparc64 ;; + *) + TARGET_CPU=$target_cpu + ;; esac - AC_SUBST(TARGET_CPU) - - AM_CONDITIONAL([TARGET_CPU_I386], test $TARGET_CPU = i386) + AM_CONDITIONAL([TARGET_CPU_AARCH64], test $TARGET_CPU = aarch64) AM_CONDITIONAL([TARGET_CPU_X86_64], test $TARGET_CPU = x86_64) AM_CONDITIONAL([TARGET_CPU_POWERPC], test $TARGET_CPU = powerpc) - AM_CONDITIONAL([TARGET_CPU_AARCH64], test $TARGET_CPU = aarch64) AM_CONDITIONAL([TARGET_CPU_SPARC64], test $TARGET_CPU = sparc64) ]) diff --git a/config/always-compiler-options.m4 b/config/always-compiler-options.m4 index 82b351c6e079..a67319691523 100644 --- a/config/always-compiler-options.m4 +++ b/config/always-compiler-options.m4 @@ -1,5 +1,5 @@ dnl # -dnl # Enabled -fsanitize=address if supported by gcc. +dnl # Enabled -fsanitize=address if supported by $CC. dnl # dnl # LDFLAGS needs -fsanitize=address at all times so libraries compiled with dnl # it will be linked successfully. CFLAGS will vary by binary being built. @@ -46,7 +46,7 @@ AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CC_ASAN], [ ]) dnl # -dnl # Enabled -fsanitize=undefined if supported by gcc. +dnl # Enabled -fsanitize=undefined if supported by cc. dnl # dnl # LDFLAGS needs -fsanitize=undefined at all times so libraries compiled with dnl # it will be linked successfully. CFLAGS will vary by binary being built. @@ -93,7 +93,7 @@ AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CC_UBSAN], [ ]) dnl # -dnl # Check if gcc supports -Wframe-larger-than= option. +dnl # Check if cc supports -Wframe-larger-than= option. dnl # AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CC_FRAME_LARGER_THAN], [ AC_MSG_CHECKING([whether $CC supports -Wframe-larger-than=]) @@ -114,7 +114,7 @@ AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CC_FRAME_LARGER_THAN], [ ]) dnl # -dnl # Check if gcc supports -Wno-format-truncation option. +dnl # Check if cc supports -Wno-format-truncation option. dnl # AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CC_NO_FORMAT_TRUNCATION], [ AC_MSG_CHECKING([whether $CC supports -Wno-format-truncation]) @@ -135,7 +135,7 @@ AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CC_NO_FORMAT_TRUNCATION], [ ]) dnl # -dnl # Check if gcc supports -Wno-format-truncation option. +dnl # Check if cc supports -Wno-format-zero-length option. dnl # AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CC_NO_FORMAT_ZERO_LENGTH], [ AC_MSG_CHECKING([whether $CC supports -Wno-format-zero-length]) @@ -155,84 +155,80 @@ AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CC_NO_FORMAT_ZERO_LENGTH], [ AC_SUBST([NO_FORMAT_ZERO_LENGTH]) ]) - dnl # -dnl # Check if gcc supports -Wno-bool-compare option. +dnl # Check if cc supports -Wno-clobbered option. dnl # -dnl # We actually invoke gcc with the -Wbool-compare option +dnl # We actually invoke it with the -Wclobbered option dnl # and infer the 'no-' version does or doesn't exist based upon dnl # the results. This is required because when checking any of dnl # no- prefixed options gcc always returns success. dnl # -AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CC_NO_BOOL_COMPARE], [ - AC_MSG_CHECKING([whether $CC supports -Wno-bool-compare]) +AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CC_NO_CLOBBERED], [ + AC_MSG_CHECKING([whether $CC supports -Wno-clobbered]) saved_flags="$CFLAGS" - CFLAGS="$CFLAGS -Werror -Wbool-compare" + CFLAGS="$CFLAGS -Werror -Wclobbered" AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [])], [ - NO_BOOL_COMPARE=-Wno-bool-compare + NO_CLOBBERED=-Wno-clobbered AC_MSG_RESULT([yes]) ], [ - NO_BOOL_COMPARE= + NO_CLOBBERED= AC_MSG_RESULT([no]) ]) CFLAGS="$saved_flags" - AC_SUBST([NO_BOOL_COMPARE]) + AC_SUBST([NO_CLOBBERED]) ]) dnl # -dnl # Check if gcc supports -Wno-unused-but-set-variable option. -dnl # -dnl # We actually invoke gcc with the -Wunused-but-set-variable option -dnl # and infer the 'no-' version does or doesn't exist based upon -dnl # the results. This is required because when checking any of -dnl # no- prefixed options gcc always returns success. +dnl # Check if cc supports -Wimplicit-fallthrough option. dnl # -AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CC_NO_UNUSED_BUT_SET_VARIABLE], [ - AC_MSG_CHECKING([whether $CC supports -Wno-unused-but-set-variable]) +AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CC_IMPLICIT_FALLTHROUGH], [ + AC_MSG_CHECKING([whether $CC supports -Wimplicit-fallthrough]) saved_flags="$CFLAGS" - CFLAGS="$CFLAGS -Werror -Wunused-but-set-variable" + CFLAGS="$CFLAGS -Werror -Wimplicit-fallthrough" AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [])], [ - NO_UNUSED_BUT_SET_VARIABLE=-Wno-unused-but-set-variable + IMPLICIT_FALLTHROUGH=-Wimplicit-fallthrough + AC_DEFINE([HAVE_IMPLICIT_FALLTHROUGH], 1, + [Define if compiler supports -Wimplicit-fallthrough]) AC_MSG_RESULT([yes]) ], [ - NO_UNUSED_BUT_SET_VARIABLE= + IMPLICIT_FALLTHROUGH= AC_MSG_RESULT([no]) ]) CFLAGS="$saved_flags" - AC_SUBST([NO_UNUSED_BUT_SET_VARIABLE]) + AC_SUBST([IMPLICIT_FALLTHROUGH]) ]) dnl # -dnl # Check if gcc supports -Wimplicit-fallthrough option. +dnl # Check if cc supports -Winfinite-recursion option. dnl # -AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CC_IMPLICIT_FALLTHROUGH], [ - AC_MSG_CHECKING([whether $CC supports -Wimplicit-fallthrough]) +AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CC_INFINITE_RECURSION], [ + AC_MSG_CHECKING([whether $CC supports -Winfinite-recursion]) saved_flags="$CFLAGS" - CFLAGS="$CFLAGS -Werror -Wimplicit-fallthrough" + CFLAGS="$CFLAGS -Werror -Winfinite-recursion" AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [])], [ - IMPLICIT_FALLTHROUGH=-Wimplicit-fallthrough - AC_DEFINE([HAVE_IMPLICIT_FALLTHROUGH], 1, - [Define if compiler supports -Wimplicit-fallthrough]) + INFINITE_RECURSION=-Winfinite-recursion + AC_DEFINE([HAVE_INFINITE_RECURSION], 1, + [Define if compiler supports -Winfinite-recursion]) AC_MSG_RESULT([yes]) ], [ - IMPLICIT_FALLTHROUGH= + INFINITE_RECURSION= AC_MSG_RESULT([no]) ]) CFLAGS="$saved_flags" - AC_SUBST([IMPLICIT_FALLTHROUGH]) + AC_SUBST([INFINITE_RECURSION]) ]) dnl # -dnl # Check if gcc supports -fno-omit-frame-pointer option. +dnl # Check if cc supports -fno-omit-frame-pointer option. dnl # AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CC_NO_OMIT_FRAME_POINTER], [ AC_MSG_CHECKING([whether $CC supports -fno-omit-frame-pointer]) diff --git a/config/always-cppcheck.m4 b/config/always-cppcheck.m4 index c7c134a3e8cd..fa5a3398d923 100644 --- a/config/always-cppcheck.m4 +++ b/config/always-cppcheck.m4 @@ -3,4 +3,7 @@ dnl # Check if cppcheck is available. dnl # AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_CPPCHECK], [ AC_CHECK_PROG([CPPCHECK], [cppcheck], [cppcheck]) + if test -z "$CPPCHECK"; then + CPPCHECK='@printf "skipping cppcheck because cppcheck is not installed\n"' + fi ]) diff --git a/config/always-system.m4 b/config/always-system.m4 index 3225a52af8ae..3a3d4212f8b0 100644 --- a/config/always-system.m4 +++ b/config/always-system.m4 @@ -8,18 +8,22 @@ AC_DEFUN([ZFS_AC_CONFIG_ALWAYS_SYSTEM], [ AC_DEFINE([SYSTEM_LINUX], [1], [True if ZFS is to be compiled for a Linux system]) ac_system="Linux" + ac_system_l="linux" ;; *freebsd*) AC_DEFINE([SYSTEM_FREEBSD], [1], [True if ZFS is to be compiled for a FreeBSD system]) ac_system="FreeBSD" + ac_system_l="freebsd" ;; *) ac_system="unknown" + ac_system_l="unknown" ;; esac AC_MSG_RESULT([$ac_system]) AC_SUBST([ac_system]) + AC_SUBST([ac_system_l]) AM_CONDITIONAL([BUILD_LINUX], [test "x$ac_system" = "xLinux"]) AM_CONDITIONAL([BUILD_FREEBSD], [test "x$ac_system" = "xFreeBSD"]) diff --git a/config/config.awk b/config/config.awk deleted file mode 100644 index cc4b7cc265cd..000000000000 --- a/config/config.awk +++ /dev/null @@ -1,15 +0,0 @@ -# Remove default preprocessor define's from config.h -# PACKAGE -# PACKAGE_BUGREPORT -# PACKAGE_NAME -# PACKAGE_STRING -# PACKAGE_TARNAME -# PACKAGE_VERSION -# STDC_HEADERS -# VERSION - -BEGIN { RS = "" ; FS = "\n" } \ - !/.#define PACKAGE./ && \ - !/.#define VERSION./ && \ - !/.#define STDC_HEADERS./ \ - { print $0"\n" } diff --git a/config/deb.am b/config/deb.am index cdbdbecb356f..0033dd7591ff 100644 --- a/config/deb.am +++ b/config/deb.am @@ -61,7 +61,8 @@ deb-utils: deb-local rpm-utils-initramfs pkg7=$${name}-test-$${version}.$${arch}.rpm; \ pkg8=$${name}-dracut-$${version}.noarch.rpm; \ pkg9=$${name}-initramfs-$${version}.$${arch}.rpm; \ - pkg10=`ls python*-pyzfs-$${version}* | tail -1`; \ + pkg10=`ls python3-pyzfs-$${version}.noarch.rpm 2>/dev/null`; \ + pkg11=`ls pam_zfs_key-$${version}.$${arch}.rpm 2>/dev/null`; \ ## Arguments need to be passed to dh_shlibdeps. Alien provides no mechanism ## to do this, so we install a shim onto the path which calls the real ## dh_shlibdeps with the required arguments. @@ -77,10 +78,10 @@ deb-utils: deb-local rpm-utils-initramfs env PATH=$${path_prepend}:$${PATH} \ fakeroot $(ALIEN) --bump=0 --scripts --to-deb --target=$$debarch \ $$pkg1 $$pkg2 $$pkg3 $$pkg4 $$pkg5 $$pkg6 $$pkg7 \ - $$pkg8 $$pkg9 $$pkg10 || exit 1; \ + $$pkg8 $$pkg9 $$pkg10 $$pkg11 || exit 1; \ $(RM) $${path_prepend}/dh_shlibdeps; \ rmdir $${path_prepend}; \ $(RM) $$pkg1 $$pkg2 $$pkg3 $$pkg4 $$pkg5 $$pkg6 $$pkg7 \ - $$pkg8 $$pkg9 $$pkg10; + $$pkg8 $$pkg9 $$pkg10 $$pkg11; deb: deb-kmod deb-dkms deb-utils diff --git a/config/kernel-add-disk.m4 b/config/kernel-add-disk.m4 index 5d1779eb4328..86d81ea325b9 100644 --- a/config/kernel-add-disk.m4 +++ b/config/kernel-add-disk.m4 @@ -3,16 +3,14 @@ dnl # 5.16 API change dnl # add_disk grew a must-check return code dnl # AC_DEFUN([ZFS_AC_KERNEL_SRC_ADD_DISK], [ - ZFS_LINUX_TEST_SRC([add_disk_ret], [ - #include + #include ], [ struct gendisk *disk = NULL; - int err = add_disk(disk); - err = err; + int error __attribute__ ((unused)) = add_disk(disk); ]) - ]) + AC_DEFUN([ZFS_AC_KERNEL_ADD_DISK], [ AC_MSG_CHECKING([whether add_disk() returns int]) ZFS_LINUX_TEST_RESULT([add_disk_ret], diff --git a/config/kernel-bio.m4 b/config/kernel-bio.m4 index d088d7023cb0..18620ca5b7e4 100644 --- a/config/kernel-bio.m4 +++ b/config/kernel-bio.m4 @@ -464,7 +464,7 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_BLK_CGROUP_HEADER], [ ]) AC_DEFUN([ZFS_AC_KERNEL_BLK_CGROUP_HEADER], [ - AC_MSG_CHECKING([for existence of linux/blk-cgroup.h]) + AC_MSG_CHECKING([whether linux/blk-cgroup.h exists]) ZFS_LINUX_TEST_RESULT([blk_cgroup_header],[ AC_MSG_RESULT(yes) AC_DEFINE(HAVE_LINUX_BLK_CGROUP_HEADER, 1, @@ -474,6 +474,41 @@ AC_DEFUN([ZFS_AC_KERNEL_BLK_CGROUP_HEADER], [ ]) ]) +dnl # +dnl # Linux 5.18 API +dnl # +dnl # In 07888c665b405b1cd3577ddebfeb74f4717a84c4 ("block: pass a block_device and opf to bio_alloc") +dnl # bio_alloc(gfp_t gfp_mask, unsigned short nr_iovecs) +dnl # became +dnl # bio_alloc(struct block_device *bdev, unsigned short nr_vecs, unsigned int opf, gfp_t gfp_mask) +dnl # however +dnl # > NULL/0 can be passed, both for the +dnl # > passthrough case on a raw request_queue and to temporarily avoid +dnl # > refactoring some nasty code. +dnl # +AC_DEFUN([ZFS_AC_KERNEL_SRC_BIO_ALLOC_4ARG], [ + ZFS_LINUX_TEST_SRC([bio_alloc_4arg], [ + #include + ],[ + gfp_t gfp_mask = 0; + unsigned short nr_iovecs = 0; + struct block_device *bdev = NULL; + unsigned int opf = 0; + + struct bio *__attribute__((unused)) allocated = bio_alloc(bdev, nr_iovecs, opf, gfp_mask); + ]) +]) + +AC_DEFUN([ZFS_AC_KERNEL_BIO_ALLOC_4ARG], [ + AC_MSG_CHECKING([whether bio_alloc() wants 4 args]) + ZFS_LINUX_TEST_RESULT([bio_alloc_4arg],[ + AC_MSG_RESULT(yes) + AC_DEFINE([HAVE_BIO_ALLOC_4ARG], 1, [bio_alloc() takes 4 arguments]) + ],[ + AC_MSG_RESULT(no) + ]) +]) + AC_DEFUN([ZFS_AC_KERNEL_SRC_BIO], [ ZFS_AC_KERNEL_SRC_REQ ZFS_AC_KERNEL_SRC_BIO_OPS @@ -488,6 +523,7 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_BIO], [ ZFS_AC_KERNEL_SRC_BDEV_SUBMIT_BIO_RETURNS_VOID ZFS_AC_KERNEL_SRC_BIO_SET_DEV_MACRO ZFS_AC_KERNEL_SRC_BLK_CGROUP_HEADER + ZFS_AC_KERNEL_SRC_BIO_ALLOC_4ARG ]) AC_DEFUN([ZFS_AC_KERNEL_BIO], [ @@ -512,4 +548,5 @@ AC_DEFUN([ZFS_AC_KERNEL_BIO], [ ZFS_AC_KERNEL_BIO_BDEV_DISK ZFS_AC_KERNEL_BDEV_SUBMIT_BIO_RETURNS_VOID ZFS_AC_KERNEL_BLK_CGROUP_HEADER + ZFS_AC_KERNEL_BIO_ALLOC_4ARG ]) diff --git a/config/kernel-blk-queue.m4 b/config/kernel-blk-queue.m4 index ff5d2d370e98..29b0a28290ab 100644 --- a/config/kernel-blk-queue.m4 +++ b/config/kernel-blk-queue.m4 @@ -74,6 +74,8 @@ AC_DEFUN([ZFS_AC_KERNEL_BLK_QUEUE_UPDATE_READAHEAD], [ AC_DEFINE(HAVE_BLK_QUEUE_UPDATE_READAHEAD, 1, [blk_queue_update_readahead() exists]) ],[ + AC_MSG_RESULT(no) + AC_MSG_CHECKING([whether disk_update_readahead() exists]) ZFS_LINUX_TEST_RESULT([disk_update_readahead], [ AC_MSG_RESULT(yes) @@ -86,69 +88,111 @@ AC_DEFUN([ZFS_AC_KERNEL_BLK_QUEUE_UPDATE_READAHEAD], [ ]) dnl # -dnl # 2.6.32 API, -dnl # blk_queue_discard() +dnl # 5.19: bdev_max_discard_sectors() available +dnl # 2.6.32: blk_queue_discard() available dnl # AC_DEFUN([ZFS_AC_KERNEL_SRC_BLK_QUEUE_DISCARD], [ + ZFS_LINUX_TEST_SRC([bdev_max_discard_sectors], [ + #include + ],[ + struct block_device *bdev __attribute__ ((unused)) = NULL; + unsigned int error __attribute__ ((unused)); + + error = bdev_max_discard_sectors(bdev); + ]) + ZFS_LINUX_TEST_SRC([blk_queue_discard], [ #include ],[ - struct request_queue *q __attribute__ ((unused)) = NULL; + struct request_queue r; + struct request_queue *q = &r; int value __attribute__ ((unused)); + memset(q, 0, sizeof(r)); value = blk_queue_discard(q); ]) ]) AC_DEFUN([ZFS_AC_KERNEL_BLK_QUEUE_DISCARD], [ - AC_MSG_CHECKING([whether blk_queue_discard() is available]) - ZFS_LINUX_TEST_RESULT([blk_queue_discard], [ + AC_MSG_CHECKING([whether bdev_max_discard_sectors() is available]) + ZFS_LINUX_TEST_RESULT([bdev_max_discard_sectors], [ AC_MSG_RESULT(yes) + AC_DEFINE(HAVE_BDEV_MAX_DISCARD_SECTORS, 1, + [bdev_max_discard_sectors() is available]) ],[ - ZFS_LINUX_TEST_ERROR([blk_queue_discard]) + AC_MSG_RESULT(no) + + AC_MSG_CHECKING([whether blk_queue_discard() is available]) + ZFS_LINUX_TEST_RESULT([blk_queue_discard], [ + AC_MSG_RESULT(yes) + AC_DEFINE(HAVE_BLK_QUEUE_DISCARD, 1, + [blk_queue_discard() is available]) + ],[ + ZFS_LINUX_TEST_ERROR([blk_queue_discard]) + ]) ]) ]) dnl # -dnl # 4.8 API, -dnl # blk_queue_secure_erase() -dnl # -dnl # 2.6.36 - 4.7 API, -dnl # blk_queue_secdiscard() +dnl # 5.19: bdev_max_secure_erase_sectors() available +dnl # 4.8: blk_queue_secure_erase() available +dnl # 2.6.36: blk_queue_secdiscard() available dnl # AC_DEFUN([ZFS_AC_KERNEL_SRC_BLK_QUEUE_SECURE_ERASE], [ + ZFS_LINUX_TEST_SRC([bdev_max_secure_erase_sectors], [ + #include + ],[ + struct block_device *bdev __attribute__ ((unused)) = NULL; + unsigned int error __attribute__ ((unused)); + + error = bdev_max_secure_erase_sectors(bdev); + ]) + ZFS_LINUX_TEST_SRC([blk_queue_secure_erase], [ #include ],[ - struct request_queue *q __attribute__ ((unused)) = NULL; + struct request_queue r; + struct request_queue *q = &r; int value __attribute__ ((unused)); + memset(q, 0, sizeof(r)); value = blk_queue_secure_erase(q); ]) ZFS_LINUX_TEST_SRC([blk_queue_secdiscard], [ #include ],[ - struct request_queue *q __attribute__ ((unused)) = NULL; + struct request_queue r; + struct request_queue *q = &r; int value __attribute__ ((unused)); + memset(q, 0, sizeof(r)); value = blk_queue_secdiscard(q); ]) ]) AC_DEFUN([ZFS_AC_KERNEL_BLK_QUEUE_SECURE_ERASE], [ - AC_MSG_CHECKING([whether blk_queue_secure_erase() is available]) - ZFS_LINUX_TEST_RESULT([blk_queue_secure_erase], [ + AC_MSG_CHECKING([whether bdev_max_secure_erase_sectors() is available]) + ZFS_LINUX_TEST_RESULT([bdev_max_secure_erase_sectors], [ AC_MSG_RESULT(yes) - AC_DEFINE(HAVE_BLK_QUEUE_SECURE_ERASE, 1, - [blk_queue_secure_erase() is available]) + AC_DEFINE(HAVE_BDEV_MAX_SECURE_ERASE_SECTORS, 1, + [bdev_max_secure_erase_sectors() is available]) ],[ AC_MSG_RESULT(no) - AC_MSG_CHECKING([whether blk_queue_secdiscard() is available]) - ZFS_LINUX_TEST_RESULT([blk_queue_secdiscard], [ + AC_MSG_CHECKING([whether blk_queue_secure_erase() is available]) + ZFS_LINUX_TEST_RESULT([blk_queue_secure_erase], [ AC_MSG_RESULT(yes) - AC_DEFINE(HAVE_BLK_QUEUE_SECDISCARD, 1, - [blk_queue_secdiscard() is available]) + AC_DEFINE(HAVE_BLK_QUEUE_SECURE_ERASE, 1, + [blk_queue_secure_erase() is available]) ],[ - ZFS_LINUX_TEST_ERROR([blk_queue_secure_erase]) + AC_MSG_RESULT(no) + + AC_MSG_CHECKING([whether blk_queue_secdiscard() is available]) + ZFS_LINUX_TEST_RESULT([blk_queue_secdiscard], [ + AC_MSG_RESULT(yes) + AC_DEFINE(HAVE_BLK_QUEUE_SECDISCARD, 1, + [blk_queue_secdiscard() is available]) + ],[ + ZFS_LINUX_TEST_ERROR([blk_queue_secure_erase]) + ]) ]) ]) ]) @@ -215,17 +259,17 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_BLK_QUEUE_FLUSH], [ ZFS_LINUX_TEST_SRC([blk_queue_flush], [ #include ], [ - struct request_queue *q = NULL; + struct request_queue *q __attribute__ ((unused)) = NULL; (void) blk_queue_flush(q, REQ_FLUSH); - ], [$NO_UNUSED_BUT_SET_VARIABLE], [ZFS_META_LICENSE]) + ], [], [ZFS_META_LICENSE]) ZFS_LINUX_TEST_SRC([blk_queue_write_cache], [ #include #include ], [ - struct request_queue *q = NULL; + struct request_queue *q __attribute__ ((unused)) = NULL; blk_queue_write_cache(q, true, true); - ], [$NO_UNUSED_BUT_SET_VARIABLE], [ZFS_META_LICENSE]) + ], [], [ZFS_META_LICENSE]) ]) AC_DEFUN([ZFS_AC_KERNEL_BLK_QUEUE_FLUSH], [ @@ -278,9 +322,9 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_BLK_QUEUE_MAX_HW_SECTORS], [ ZFS_LINUX_TEST_SRC([blk_queue_max_hw_sectors], [ #include ], [ - struct request_queue *q = NULL; + struct request_queue *q __attribute__ ((unused)) = NULL; (void) blk_queue_max_hw_sectors(q, BLK_SAFE_MAX_SECTORS); - ], [$NO_UNUSED_BUT_SET_VARIABLE]) + ], []) ]) AC_DEFUN([ZFS_AC_KERNEL_BLK_QUEUE_MAX_HW_SECTORS], [ @@ -301,9 +345,9 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_BLK_QUEUE_MAX_SEGMENTS], [ ZFS_LINUX_TEST_SRC([blk_queue_max_segments], [ #include ], [ - struct request_queue *q = NULL; + struct request_queue *q __attribute__ ((unused)) = NULL; (void) blk_queue_max_segments(q, BLK_MAX_SEGMENTS); - ], [$NO_UNUSED_BUT_SET_VARIABLE]) + ], []) ]) AC_DEFUN([ZFS_AC_KERNEL_BLK_QUEUE_MAX_SEGMENTS], [ @@ -315,6 +359,36 @@ AC_DEFUN([ZFS_AC_KERNEL_BLK_QUEUE_MAX_SEGMENTS], [ ]) ]) +dnl # +dnl # See if kernel supports block multi-queue and blk_status_t. +dnl # blk_status_t represents the new status codes introduced in the 4.13 +dnl # kernel patch: +dnl # +dnl # block: introduce new block status code type +dnl # +dnl # We do not currently support the "old" block multi-queue interfaces from +dnl # prior kernels. +dnl # +AC_DEFUN([ZFS_AC_KERNEL_SRC_BLK_MQ], [ + ZFS_LINUX_TEST_SRC([blk_mq], [ + #include + ], [ + struct blk_mq_tag_set tag_set __attribute__ ((unused)) = {0}; + (void) blk_mq_alloc_tag_set(&tag_set); + return BLK_STS_OK; + ], []) +]) + +AC_DEFUN([ZFS_AC_KERNEL_BLK_MQ], [ + AC_MSG_CHECKING([whether block multiqueue with blk_status_t is available]) + ZFS_LINUX_TEST_RESULT([blk_mq], [ + AC_MSG_RESULT(yes) + AC_DEFINE(HAVE_BLK_MQ, 1, [block multiqueue is available]) + ], [ + AC_MSG_RESULT(no) + ]) +]) + AC_DEFUN([ZFS_AC_KERNEL_SRC_BLK_QUEUE], [ ZFS_AC_KERNEL_SRC_BLK_QUEUE_PLUG ZFS_AC_KERNEL_SRC_BLK_QUEUE_BDI @@ -326,6 +400,7 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_BLK_QUEUE], [ ZFS_AC_KERNEL_SRC_BLK_QUEUE_FLUSH ZFS_AC_KERNEL_SRC_BLK_QUEUE_MAX_HW_SECTORS ZFS_AC_KERNEL_SRC_BLK_QUEUE_MAX_SEGMENTS + ZFS_AC_KERNEL_SRC_BLK_MQ ]) AC_DEFUN([ZFS_AC_KERNEL_BLK_QUEUE], [ @@ -339,4 +414,5 @@ AC_DEFUN([ZFS_AC_KERNEL_BLK_QUEUE], [ ZFS_AC_KERNEL_BLK_QUEUE_FLUSH ZFS_AC_KERNEL_BLK_QUEUE_MAX_HW_SECTORS ZFS_AC_KERNEL_BLK_QUEUE_MAX_SEGMENTS + ZFS_AC_KERNEL_BLK_MQ ]) diff --git a/config/kernel-blkdev.m4 b/config/kernel-blkdev.m4 index 9c60e5dd4210..fb7b1a458638 100644 --- a/config/kernel-blkdev.m4 +++ b/config/kernel-blkdev.m4 @@ -294,6 +294,57 @@ AC_DEFUN([ZFS_AC_KERNEL_BLKDEV_BDEV_WHOLE], [ ]) ]) +dnl # +dnl # 5.19 API: blkdev_issue_secure_erase() +dnl # 3.10 API: blkdev_issue_discard(..., BLKDEV_DISCARD_SECURE) +dnl # +AC_DEFUN([ZFS_AC_KERNEL_SRC_BLKDEV_ISSUE_SECURE_ERASE], [ + ZFS_LINUX_TEST_SRC([blkdev_issue_secure_erase], [ + #include + ],[ + struct block_device *bdev = NULL; + sector_t sector = 0; + sector_t nr_sects = 0; + int error __attribute__ ((unused)); + + error = blkdev_issue_secure_erase(bdev, + sector, nr_sects, GFP_KERNEL); + ]) + + ZFS_LINUX_TEST_SRC([blkdev_issue_discard_flags], [ + #include + ],[ + struct block_device *bdev = NULL; + sector_t sector = 0; + sector_t nr_sects = 0; + unsigned long flags = 0; + int error __attribute__ ((unused)); + + error = blkdev_issue_discard(bdev, + sector, nr_sects, GFP_KERNEL, flags); + ]) +]) + +AC_DEFUN([ZFS_AC_KERNEL_BLKDEV_ISSUE_SECURE_ERASE], [ + AC_MSG_CHECKING([whether blkdev_issue_secure_erase() is available]) + ZFS_LINUX_TEST_RESULT([blkdev_issue_secure_erase], [ + AC_MSG_RESULT(yes) + AC_DEFINE(HAVE_BLKDEV_ISSUE_SECURE_ERASE, 1, + [blkdev_issue_secure_erase() is available]) + ],[ + AC_MSG_RESULT(no) + + AC_MSG_CHECKING([whether blkdev_issue_discard() is available]) + ZFS_LINUX_TEST_RESULT([blkdev_issue_discard_flags], [ + AC_MSG_RESULT(yes) + AC_DEFINE(HAVE_BLKDEV_ISSUE_DISCARD, 1, + [blkdev_issue_discard() is available]) + ],[ + ZFS_LINUX_TEST_ERROR([blkdev_issue_discard()]) + ]) + ]) +]) + dnl # dnl # 5.13 API change dnl # blkdev_get_by_path() no longer handles ERESTARTSYS @@ -326,6 +377,7 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_BLKDEV], [ ZFS_AC_KERNEL_SRC_BLKDEV_CHECK_DISK_CHANGE ZFS_AC_KERNEL_SRC_BLKDEV_BDEV_CHECK_MEDIA_CHANGE ZFS_AC_KERNEL_SRC_BLKDEV_BDEV_WHOLE + ZFS_AC_KERNEL_SRC_BLKDEV_ISSUE_SECURE_ERASE ]) AC_DEFUN([ZFS_AC_KERNEL_BLKDEV], [ @@ -340,4 +392,5 @@ AC_DEFUN([ZFS_AC_KERNEL_BLKDEV], [ ZFS_AC_KERNEL_BLKDEV_BDEV_CHECK_MEDIA_CHANGE ZFS_AC_KERNEL_BLKDEV_BDEV_WHOLE ZFS_AC_KERNEL_BLKDEV_GET_ERESTARTSYS + ZFS_AC_KERNEL_BLKDEV_ISSUE_SECURE_ERASE ]) diff --git a/config/kernel-block-device-operations.m4 b/config/kernel-block-device-operations.m4 index a48618185bfb..84e39dc8a2f6 100644 --- a/config/kernel-block-device-operations.m4 +++ b/config/kernel-block-device-operations.m4 @@ -6,13 +6,16 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_BLOCK_DEVICE_OPERATIONS_CHECK_EVENTS], [ #include unsigned int blk_check_events(struct gendisk *disk, - unsigned int clearing) { return (0); } + unsigned int clearing) { + (void) disk, (void) clearing; + return (0); + } static const struct block_device_operations bops __attribute__ ((unused)) = { .check_events = blk_check_events, }; - ], [], [$NO_UNUSED_BUT_SET_VARIABLE]) + ], [], []) ]) AC_DEFUN([ZFS_AC_KERNEL_BLOCK_DEVICE_OPERATIONS_CHECK_EVENTS], [ @@ -31,7 +34,10 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_BLOCK_DEVICE_OPERATIONS_RELEASE_VOID], [ ZFS_LINUX_TEST_SRC([block_device_operations_release_void], [ #include - void blk_release(struct gendisk *g, fmode_t mode) { return; } + void blk_release(struct gendisk *g, fmode_t mode) { + (void) g, (void) mode; + return; + } static const struct block_device_operations bops __attribute__ ((unused)) = { @@ -40,7 +46,7 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_BLOCK_DEVICE_OPERATIONS_RELEASE_VOID], [ .ioctl = NULL, .compat_ioctl = NULL, }; - ], [], [$NO_UNUSED_BUT_SET_VARIABLE]) + ], [], []) ]) AC_DEFUN([ZFS_AC_KERNEL_BLOCK_DEVICE_OPERATIONS_RELEASE_VOID], [ @@ -61,6 +67,7 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_BLOCK_DEVICE_OPERATIONS_REVALIDATE_DISK], [ #include int blk_revalidate_disk(struct gendisk *disk) { + (void) disk; return(0); } @@ -68,7 +75,7 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_BLOCK_DEVICE_OPERATIONS_REVALIDATE_DISK], [ bops __attribute__ ((unused)) = { .revalidate_disk = blk_revalidate_disk, }; - ], [], [$NO_UNUSED_BUT_SET_VARIABLE]) + ], [], []) ]) AC_DEFUN([ZFS_AC_KERNEL_BLOCK_DEVICE_OPERATIONS_REVALIDATE_DISK], [ diff --git a/config/kernel-config-defined.m4 b/config/kernel-config-defined.m4 index c7d18b49b14e..54837d728341 100644 --- a/config/kernel-config-defined.m4 +++ b/config/kernel-config-defined.m4 @@ -19,19 +19,48 @@ AC_DEFUN([ZFS_AC_KERNEL_CONFIG_DEFINED], [ ]) ]) + ZFS_AC_KERNEL_SRC_CONFIG_MODULES + ZFS_AC_KERNEL_SRC_CONFIG_BLOCK ZFS_AC_KERNEL_SRC_CONFIG_DEBUG_LOCK_ALLOC ZFS_AC_KERNEL_SRC_CONFIG_TRIM_UNUSED_KSYMS - ZFS_AC_KERNEL_SRC_CONFIG_ZLIB_INFLATE ZFS_AC_KERNEL_SRC_CONFIG_ZLIB_DEFLATE + ZFS_AC_KERNEL_SRC_CONFIG_ZLIB_INFLATE AC_MSG_CHECKING([for kernel config option compatibility]) ZFS_LINUX_TEST_COMPILE_ALL([config]) AC_MSG_RESULT([done]) + ZFS_AC_KERNEL_CONFIG_MODULES + ZFS_AC_KERNEL_CONFIG_BLOCK ZFS_AC_KERNEL_CONFIG_DEBUG_LOCK_ALLOC ZFS_AC_KERNEL_CONFIG_TRIM_UNUSED_KSYMS - ZFS_AC_KERNEL_CONFIG_ZLIB_INFLATE ZFS_AC_KERNEL_CONFIG_ZLIB_DEFLATE + ZFS_AC_KERNEL_CONFIG_ZLIB_INFLATE +]) + +dnl # +dnl # Check CONFIG_BLOCK +dnl # +dnl # Verify the kernel has CONFIG_BLOCK support enabled. +dnl # +AC_DEFUN([ZFS_AC_KERNEL_SRC_CONFIG_BLOCK], [ + ZFS_LINUX_TEST_SRC([config_block], [ + #if !defined(CONFIG_BLOCK) + #error CONFIG_BLOCK not defined + #endif + ],[]) +]) + +AC_DEFUN([ZFS_AC_KERNEL_CONFIG_BLOCK], [ + AC_MSG_CHECKING([whether CONFIG_BLOCK is defined]) + ZFS_LINUX_TEST_RESULT([config_block], [ + AC_MSG_RESULT([yes]) + ],[ + AC_MSG_RESULT([no]) + AC_MSG_ERROR([ + *** This kernel does not include the required block device support. + *** Rebuild the kernel with CONFIG_BLOCK=y set.]) + ]) ]) dnl # @@ -72,6 +101,61 @@ AC_DEFUN([ZFS_AC_KERNEL_CONFIG_DEBUG_LOCK_ALLOC], [ ]) ]) +dnl # +dnl # Check CONFIG_MODULES +dnl # +dnl # Verify the kernel has CONFIG_MODULES support enabled. +dnl # +AC_DEFUN([ZFS_AC_KERNEL_SRC_CONFIG_MODULES], [ + ZFS_LINUX_TEST_SRC([config_modules], [ + #if !defined(CONFIG_MODULES) + #error CONFIG_MODULES not defined + #endif + ],[]) +]) + +AC_DEFUN([ZFS_AC_KERNEL_CONFIG_MODULES], [ + AC_MSG_CHECKING([whether CONFIG_MODULES is defined]) + AS_IF([test "x$enable_linux_builtin" != xyes], [ + ZFS_LINUX_TEST_RESULT([config_modules], [ + AC_MSG_RESULT([yes]) + ],[ + AC_MSG_RESULT([no]) + AC_MSG_ERROR([ + *** This kernel does not include the required loadable module + *** support! + *** + *** To build OpenZFS as a loadable Linux kernel module + *** enable loadable module support by setting + *** `CONFIG_MODULES=y` in the kernel configuration and run + *** `make modules_prepare` in the Linux source tree. + *** + *** If you don't intend to enable loadable kernel module + *** support, please compile OpenZFS as a Linux kernel built-in. + *** + *** Prepare the Linux source tree by running `make prepare`, + *** use the OpenZFS `--enable-linux-builtin` configure option, + *** copy the OpenZFS sources into the Linux source tree using + *** `./copy-builtin `, + *** set `CONFIG_ZFS=y` in the kernel configuration and compile + *** kernel as usual. + ]) + ]) + ], [ + ZFS_LINUX_TRY_COMPILE([], [], [ + AC_MSG_RESULT([not needed]) + ],[ + AC_MSG_RESULT([error]) + AC_MSG_ERROR([ + *** This kernel is unable to compile object files. + *** + *** Please make sure you prepared the Linux source tree + *** by running `make prepare` there. + ]) + ]) + ]) +]) + dnl # dnl # Check CONFIG_TRIM_UNUSED_KSYMS dnl # diff --git a/config/kernel-copy-from-user-inatomic.m4 b/config/kernel-copy-from-user-inatomic.m4 new file mode 100644 index 000000000000..fec354b2f38e --- /dev/null +++ b/config/kernel-copy-from-user-inatomic.m4 @@ -0,0 +1,29 @@ +dnl # +dnl # On certain architectures `__copy_from_user_inatomic` +dnl # is a GPL exported variable and cannot be used by OpenZFS. +dnl # + +dnl # +dnl # Checking if `__copy_from_user_inatomic` is available. +dnl # +AC_DEFUN([ZFS_AC_KERNEL_SRC___COPY_FROM_USER_INATOMIC], [ + ZFS_LINUX_TEST_SRC([__copy_from_user_inatomic], [ + #include + ], [ + int result __attribute__ ((unused)) = __copy_from_user_inatomic(NULL, NULL, 0); + ], [], [ZFS_META_LICENSE]) +]) + +AC_DEFUN([ZFS_AC_KERNEL___COPY_FROM_USER_INATOMIC], [ + AC_MSG_CHECKING([whether __copy_from_user_inatomic is available]) + ZFS_LINUX_TEST_RESULT([__copy_from_user_inatomic_license], [ + AC_MSG_RESULT(yes) + ], [ + AC_MSG_RESULT(no) + AC_MSG_ERROR([ + *** The `__copy_from_user_inatomic()` Linux kernel function is + *** incompatible with the CDDL license and will prevent the module + *** linking stage from succeeding. OpenZFS cannot be compiled. + ]) + ]) +]) diff --git a/config/kernel-fpu.m4 b/config/kernel-fpu.m4 index 06280239ddf0..c6efebd8cf61 100644 --- a/config/kernel-fpu.m4 +++ b/config/kernel-fpu.m4 @@ -2,11 +2,11 @@ dnl # dnl # Handle differences in kernel FPU code. dnl # dnl # Kernel -dnl # 5.16: XCR code put into asm/fpu/xcr.h -dnl # HAVE_KERNEL_FPU_XCR_HEADER +dnl # 5.19: The asm/fpu/internal.h header was removed, it has been +dnl # effectively empty since the 5.16 kernel. dnl # -dnl # XSTATE_XSAVE and XSTATE_XRESTORE aren't accessible any more -dnl # HAVE_KERNEL_FPU_XSAVE_INTERNAL +dnl # 5.11: kernel_fpu_begin() is an inlined function now, so don't check +dnl # for it inside the kernel symbols. dnl # dnl # 5.0: Wrappers have been introduced to save/restore the FPU state. dnl # This change was made to the 4.19.38 and 4.14.120 LTS kernels. @@ -30,22 +30,22 @@ AC_DEFUN([ZFS_AC_KERNEL_FPU_HEADER], [ ],[ AC_DEFINE(HAVE_KERNEL_FPU_API_HEADER, 1, [kernel has asm/fpu/api.h]) - AC_MSG_RESULT(asm/fpu/api.h) - AC_MSG_CHECKING([whether fpu/xcr header is available]) + ZFS_LINUX_TRY_COMPILE([ #include - #include + #include ],[ ],[ - AC_DEFINE(HAVE_KERNEL_FPU_XCR_HEADER, 1, - [kernel has asm/fpu/xcr.h]) - AC_MSG_RESULT(asm/fpu/xcr.h) + AC_DEFINE(HAVE_KERNEL_FPU_INTERNAL_HEADER, 1, + [kernel has asm/fpu/internal.h]) + AC_MSG_RESULT([asm/fpu/api.h asm/fpu/internal.h]) ],[ - AC_MSG_RESULT(no asm/fpu/xcr.h) + AC_MSG_RESULT([asm/fpu/api.h]) ]) ],[ - AC_MSG_RESULT(i387.h & xcr.h) + AC_MSG_RESULT([i387.h]) ]) + ]) AC_DEFUN([ZFS_AC_KERNEL_SRC_FPU], [ @@ -53,9 +53,11 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_FPU], [ #include #ifdef HAVE_KERNEL_FPU_API_HEADER #include + #ifdef HAVE_KERNEL_FPU_INTERNAL_HEADER + #include + #endif #else #include - #include #endif ], [ kernel_fpu_begin(); @@ -66,80 +68,17 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_FPU], [ #include #ifdef HAVE_KERNEL_FPU_API_HEADER #include + #ifdef HAVE_KERNEL_FPU_INTERNAL_HEADER + #include + #endif #else #include - #include #endif ], [ __kernel_fpu_begin(); __kernel_fpu_end(); ], [], [ZFS_META_LICENSE]) - ZFS_LINUX_TEST_SRC([fpu_internal], [ - #if defined(__x86_64) || defined(__x86_64__) || \ - defined(__i386) || defined(__i386__) - #if !defined(__x86) - #define __x86 - #endif - #endif - - #if !defined(__x86) - #error Unsupported architecture - #endif - - #include - #ifdef HAVE_KERNEL_FPU_API_HEADER - #include - #include - #else - #include - #include - #endif - - #if !defined(XSTATE_XSAVE) - #error XSTATE_XSAVE not defined - #endif - - #if !defined(XSTATE_XRESTORE) - #error XSTATE_XRESTORE not defined - #endif - ],[ - struct fpu *fpu = ¤t->thread.fpu; - union fpregs_state *st = &fpu->state; - struct fregs_state *fr __attribute__ ((unused)) = &st->fsave; - struct fxregs_state *fxr __attribute__ ((unused)) = &st->fxsave; - struct xregs_state *xr __attribute__ ((unused)) = &st->xsave; - ]) - - ZFS_LINUX_TEST_SRC([fpu_xsave_internal], [ - #include - #if defined(__x86_64) || defined(__x86_64__) || \ - defined(__i386) || defined(__i386__) - #if !defined(__x86) - #define __x86 - #endif - #endif - - #if !defined(__x86) - #error Unsupported architecture - #endif - - #include - #ifdef HAVE_KERNEL_FPU_API_HEADER - #include - #include - #else - #include - #include - #endif - - ],[ - struct fpu *fpu = ¤t->thread.fpu; - union fpregs_state *st = &fpu->fpstate->regs; - struct fregs_state *fr __attribute__ ((unused)) = &st->fsave; - struct fxregs_state *fxr __attribute__ ((unused)) = &st->fxsave; - struct xregs_state *xr __attribute__ ((unused)) = &st->xsave; - ]) ]) AC_DEFUN([ZFS_AC_KERNEL_FPU], [ @@ -147,8 +86,7 @@ AC_DEFUN([ZFS_AC_KERNEL_FPU], [ dnl # Legacy kernel dnl # AC_MSG_CHECKING([whether kernel fpu is available]) - ZFS_LINUX_TEST_RESULT_SYMBOL([kernel_fpu_license], - [kernel_fpu_begin], [arch/x86/kernel/fpu/core.c], [ + ZFS_LINUX_TEST_RESULT([kernel_fpu_license], [ AC_MSG_RESULT(kernel_fpu_*) AC_DEFINE(HAVE_KERNEL_FPU, 1, [kernel has kernel_fpu_* functions]) @@ -167,19 +105,9 @@ AC_DEFUN([ZFS_AC_KERNEL_FPU], [ AC_DEFINE(KERNEL_EXPORTS_X86_FPU, 1, [kernel exports FPU functions]) ],[ - ZFS_LINUX_TEST_RESULT([fpu_internal], [ - AC_MSG_RESULT(internal) - AC_DEFINE(HAVE_KERNEL_FPU_INTERNAL, 1, - [kernel fpu internal]) - ],[ - ZFS_LINUX_TEST_RESULT([fpu_xsave_internal], [ - AC_MSG_RESULT(internal with internal XSAVE) - AC_DEFINE(HAVE_KERNEL_FPU_XSAVE_INTERNAL, 1, - [kernel fpu and XSAVE internal]) - ],[ - AC_MSG_RESULT(unavailable) - ]) - ]) + AC_MSG_RESULT(internal) + AC_DEFINE(HAVE_KERNEL_FPU_INTERNAL, 1, + [kernel fpu internal]) ]) ]) ]) diff --git a/config/kernel-generic_io_acct.m4 b/config/kernel-generic_io_acct.m4 index 0f4381db4c5e..a8a448c6fe96 100644 --- a/config/kernel-generic_io_acct.m4 +++ b/config/kernel-generic_io_acct.m4 @@ -2,6 +2,19 @@ dnl # dnl # Check for generic io accounting interface. dnl # AC_DEFUN([ZFS_AC_KERNEL_SRC_GENERIC_IO_ACCT], [ + ZFS_LINUX_TEST_SRC([bdev_io_acct], [ + #include + ], [ + struct block_device *bdev = NULL; + struct bio *bio = NULL; + unsigned long passed_time = 0; + unsigned long start_time; + + start_time = bdev_start_io_acct(bdev, bio_sectors(bio), + bio_op(bio), passed_time); + bdev_end_io_acct(bdev, bio_op(bio), start_time); + ]) + ZFS_LINUX_TEST_SRC([disk_io_acct], [ #include ], [ @@ -50,61 +63,75 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_GENERIC_IO_ACCT], [ AC_DEFUN([ZFS_AC_KERNEL_GENERIC_IO_ACCT], [ dnl # - dnl # 5.12 API, + dnl # 5.19 API, dnl # - dnl # bio_start_io_acct() and bio_end_io_acct() became GPL-exported - dnl # so use disk_start_io_acct() and disk_end_io_acct() instead + dnl # disk_start_io_acct() and disk_end_io_acct() have been replaced by + dnl # bdev_start_io_acct() and bdev_end_io_acct(). dnl # - AC_MSG_CHECKING([whether generic disk_*_io_acct() are available]) - ZFS_LINUX_TEST_RESULT([disk_io_acct], [ + AC_MSG_CHECKING([whether generic bdev_*_io_acct() are available]) + ZFS_LINUX_TEST_RESULT([bdev_io_acct], [ AC_MSG_RESULT(yes) - AC_DEFINE(HAVE_DISK_IO_ACCT, 1, [disk_*_io_acct() available]) + AC_DEFINE(HAVE_BDEV_IO_ACCT, 1, [bdev_*_io_acct() available]) ], [ AC_MSG_RESULT(no) dnl # - dnl # 5.7 API, + dnl # 5.12 API, dnl # - dnl # Added bio_start_io_acct() and bio_end_io_acct() helpers. + dnl # bio_start_io_acct() and bio_end_io_acct() became GPL-exported + dnl # so use disk_start_io_acct() and disk_end_io_acct() instead dnl # - AC_MSG_CHECKING([whether generic bio_*_io_acct() are available]) - ZFS_LINUX_TEST_RESULT([bio_io_acct], [ + AC_MSG_CHECKING([whether generic disk_*_io_acct() are available]) + ZFS_LINUX_TEST_RESULT([disk_io_acct], [ AC_MSG_RESULT(yes) - AC_DEFINE(HAVE_BIO_IO_ACCT, 1, [bio_*_io_acct() available]) + AC_DEFINE(HAVE_DISK_IO_ACCT, 1, [disk_*_io_acct() available]) ], [ AC_MSG_RESULT(no) dnl # - dnl # 4.14 API, + dnl # 5.7 API, dnl # - dnl # generic_start_io_acct/generic_end_io_acct now require - dnl # request_queue to be provided. No functional changes, - dnl # but preparation for inflight accounting. + dnl # Added bio_start_io_acct() and bio_end_io_acct() helpers. dnl # - AC_MSG_CHECKING([whether generic_*_io_acct wants 4 args]) - ZFS_LINUX_TEST_RESULT_SYMBOL([generic_acct_4args], - [generic_start_io_acct], [block/bio.c], [ + AC_MSG_CHECKING([whether generic bio_*_io_acct() are available]) + ZFS_LINUX_TEST_RESULT([bio_io_acct], [ AC_MSG_RESULT(yes) - AC_DEFINE(HAVE_GENERIC_IO_ACCT_4ARG, 1, - [generic_*_io_acct() 4 arg available]) + AC_DEFINE(HAVE_BIO_IO_ACCT, 1, [bio_*_io_acct() available]) ], [ AC_MSG_RESULT(no) dnl # - dnl # 3.19 API addition + dnl # 4.14 API, dnl # - dnl # torvalds/linux@394ffa50 allows us to increment - dnl # iostat counters without generic_make_request(). + dnl # generic_start_io_acct/generic_end_io_acct now require + dnl # request_queue to be provided. No functional changes, + dnl # but preparation for inflight accounting. dnl # - AC_MSG_CHECKING( - [whether generic_*_io_acct wants 3 args]) - ZFS_LINUX_TEST_RESULT_SYMBOL([generic_acct_3args], + AC_MSG_CHECKING([whether generic_*_io_acct wants 4 args]) + ZFS_LINUX_TEST_RESULT_SYMBOL([generic_acct_4args], [generic_start_io_acct], [block/bio.c], [ AC_MSG_RESULT(yes) - AC_DEFINE(HAVE_GENERIC_IO_ACCT_3ARG, 1, - [generic_*_io_acct() 3 arg available]) + AC_DEFINE(HAVE_GENERIC_IO_ACCT_4ARG, 1, + [generic_*_io_acct() 4 arg available]) ], [ AC_MSG_RESULT(no) + + dnl # + dnl # 3.19 API addition + dnl # + dnl # torvalds/linux@394ffa50 allows us to increment + dnl # iostat counters without generic_make_request(). + dnl # + AC_MSG_CHECKING( + [whether generic_*_io_acct wants 3 args]) + ZFS_LINUX_TEST_RESULT_SYMBOL([generic_acct_3args], + [generic_start_io_acct], [block/bio.c], [ + AC_MSG_RESULT(yes) + AC_DEFINE(HAVE_GENERIC_IO_ACCT_3ARG, 1, + [generic_*_io_acct() 3 arg available]) + ], [ + AC_MSG_RESULT(no) + ]) ]) ]) ]) diff --git a/config/kernel-genhd-flags.m4 b/config/kernel-genhd-flags.m4 new file mode 100644 index 000000000000..af6a8a086bc9 --- /dev/null +++ b/config/kernel-genhd-flags.m4 @@ -0,0 +1,58 @@ +dnl # +dnl # 5.17 API change, +dnl # +dnl # GENHD_FL_EXT_DEVT flag removed +dnl # GENHD_FL_NO_PART_SCAN renamed GENHD_FL_NO_PART +dnl # +AC_DEFUN([ZFS_AC_KERNEL_SRC_GENHD_FLAGS], [ + + ZFS_LINUX_TEST_SRC([genhd_fl_ext_devt], [ + #include + ], [ + int flags __attribute__ ((unused)) = GENHD_FL_EXT_DEVT; + ]) + + ZFS_LINUX_TEST_SRC([genhd_fl_no_part], [ + #include + ], [ + int flags __attribute__ ((unused)) = GENHD_FL_NO_PART; + ]) + + ZFS_LINUX_TEST_SRC([genhd_fl_no_part_scan], [ + #include + ], [ + int flags __attribute__ ((unused)) = GENHD_FL_NO_PART_SCAN; + ]) +]) + +AC_DEFUN([ZFS_AC_KERNEL_GENHD_FLAGS], [ + + AC_MSG_CHECKING([whether GENHD_FL_EXT_DEVT flag is available]) + ZFS_LINUX_TEST_RESULT([genhd_fl_ext_devt], [ + AC_MSG_RESULT(yes) + AC_DEFINE(ZFS_GENHD_FL_EXT_DEVT, GENHD_FL_EXT_DEVT, + [GENHD_FL_EXT_DEVT flag is available]) + ], [ + AC_MSG_RESULT(no) + AC_DEFINE(ZFS_GENHD_FL_EXT_DEVT, 0, + [GENHD_FL_EXT_DEVT flag is not available]) + ]) + + AC_MSG_CHECKING([whether GENHD_FL_NO_PART flag is available]) + ZFS_LINUX_TEST_RESULT([genhd_fl_no_part], [ + AC_MSG_RESULT(yes) + AC_DEFINE(ZFS_GENHD_FL_NO_PART, GENHD_FL_NO_PART, + [GENHD_FL_NO_PART flag is available]) + ], [ + AC_MSG_RESULT(no) + + AC_MSG_CHECKING([whether GENHD_FL_NO_PART_SCAN flag is available]) + ZFS_LINUX_TEST_RESULT([genhd_fl_no_part_scan], [ + AC_MSG_RESULT(yes) + AC_DEFINE(ZFS_GENHD_FL_NO_PART, GENHD_FL_NO_PART_SCAN, + [GENHD_FL_NO_PART_SCAN flag is available]) + ], [ + ZFS_LINUX_TEST_ERROR([GENHD_FL_NO_PART|GENHD_FL_NO_PART_SCAN]) + ]) + ]) +]) diff --git a/config/kernel-get-disk-ro.m4 b/config/kernel-get-disk-ro.m4 index 8a379c7669fa..acfcb69acc10 100644 --- a/config/kernel-get-disk-ro.m4 +++ b/config/kernel-get-disk-ro.m4 @@ -5,9 +5,9 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_GET_DISK_RO], [ ZFS_LINUX_TEST_SRC([get_disk_ro], [ #include ],[ - struct gendisk *disk = NULL; + struct gendisk *disk __attribute__ ((unused)) = NULL; (void) get_disk_ro(disk); - ], [$NO_UNUSED_BUT_SET_VARIABLE]) + ], []) ]) AC_DEFUN([ZFS_AC_KERNEL_GET_DISK_RO], [ diff --git a/config/kernel-global_page_state.m4 b/config/kernel-global_page_state.m4 index badb5e5d2e23..76f2bba202a1 100644 --- a/config/kernel-global_page_state.m4 +++ b/config/kernel-global_page_state.m4 @@ -55,7 +55,7 @@ dnl # AC_DEFUN([ZFS_AC_KERNEL_ENUM_MEMBER], [ AC_MSG_CHECKING([whether enum $2 contains $1]) AS_IF([AC_TRY_COMMAND( - "${srcdir}/scripts/enum-extract.pl" "$2" "$3" | egrep -qx $1)],[ + "${srcdir}/scripts/enum-extract.pl" "$2" "$3" | grep -Eqx $1)],[ AC_MSG_RESULT([yes]) AC_DEFINE(m4_join([_], [ZFS_ENUM], m4_toupper($2), $1), 1, [enum $2 contains $1]) diff --git a/config/kernel-group-info.m4 b/config/kernel-group-info.m4 index 0fee1d36d50d..6941d62da017 100644 --- a/config/kernel-group-info.m4 +++ b/config/kernel-group-info.m4 @@ -6,8 +6,8 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_GROUP_INFO_GID], [ ZFS_LINUX_TEST_SRC([group_info_gid], [ #include ],[ - struct group_info *gi = groups_alloc(1); - gi->gid[0] = KGIDT_INIT(0); + struct group_info gi __attribute__ ((unused)) = {}; + gi.gid[0] = KGIDT_INIT(0); ]) ]) diff --git a/config/kernel-inode-permission.m4 b/config/kernel-inode-permission.m4 new file mode 100644 index 000000000000..ba9ff5d43d4d --- /dev/null +++ b/config/kernel-inode-permission.m4 @@ -0,0 +1,29 @@ +AC_DEFUN([ZFS_AC_KERNEL_SRC_PERMISSION], [ + dnl # + dnl # 5.12 API change that added the struct user_namespace* arg + dnl # to the front of this function type's arg list. + dnl # + ZFS_LINUX_TEST_SRC([permission_userns], [ + #include + #include + + int inode_permission(struct user_namespace *userns, + struct inode *inode, int mask) { return 0; } + + static const struct inode_operations + iops __attribute__ ((unused)) = { + .permission = inode_permission, + }; + ],[]) +]) + +AC_DEFUN([ZFS_AC_KERNEL_PERMISSION], [ + AC_MSG_CHECKING([whether iops->permission() takes struct user_namespace*]) + ZFS_LINUX_TEST_RESULT([permission_userns], [ + AC_MSG_RESULT(yes) + AC_DEFINE(HAVE_IOPS_PERMISSION_USERNS, 1, + [iops->permission() takes struct user_namespace*]) + ],[ + AC_MSG_RESULT(no) + ]) +]) diff --git a/config/kernel-mkdir.m4 b/config/kernel-mkdir.m4 index a162bcd880ff..6667ed04fa4c 100644 --- a/config/kernel-mkdir.m4 +++ b/config/kernel-mkdir.m4 @@ -53,6 +53,8 @@ AC_DEFUN([ZFS_AC_KERNEL_MKDIR], [ AC_DEFINE(HAVE_IOPS_MKDIR_USERNS, 1, [iops->mkdir() takes struct user_namespace*]) ],[ + AC_MSG_RESULT(no) + AC_MSG_CHECKING([whether iops->mkdir() takes umode_t]) ZFS_LINUX_TEST_RESULT([inode_operations_mkdir], [ AC_MSG_RESULT(yes) diff --git a/config/kernel-pagemap-folio_wait_bit.m4 b/config/kernel-pagemap-folio_wait_bit.m4 index e0aaa4a57411..12d8841f51e6 100644 --- a/config/kernel-pagemap-folio_wait_bit.m4 +++ b/config/kernel-pagemap-folio_wait_bit.m4 @@ -15,7 +15,7 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_PAGEMAP_FOLIO_WAIT_BIT], [ ]) AC_DEFUN([ZFS_AC_KERNEL_PAGEMAP_FOLIO_WAIT_BIT], [ - AC_MSG_CHECKING([folio_wait_bit() exists]) + AC_MSG_CHECKING([whether folio_wait_bit() exists]) ZFS_LINUX_TEST_RESULT([pagemap_has_folio_wait_bit], [ AC_MSG_RESULT([yes]) AC_DEFINE(HAVE_PAGEMAP_FOLIO_WAIT_BIT, 1, diff --git a/config/kernel-readpages.m4 b/config/kernel-readpages.m4 new file mode 100644 index 000000000000..be65a0d5e4b4 --- /dev/null +++ b/config/kernel-readpages.m4 @@ -0,0 +1,25 @@ +dnl # +dnl # Linux 5.18 removes address_space_operations ->readpages in favour of +dnl # ->readahead +dnl # +AC_DEFUN([ZFS_AC_KERNEL_SRC_VFS_READPAGES], [ + ZFS_LINUX_TEST_SRC([vfs_has_readpages], [ + #include + + static const struct address_space_operations + aops __attribute__ ((unused)) = { + .readpages = NULL, + }; + ],[]) +]) + +AC_DEFUN([ZFS_AC_KERNEL_VFS_READPAGES], [ + AC_MSG_CHECKING([whether aops->readpages exists]) + ZFS_LINUX_TEST_RESULT([vfs_has_readpages], [ + AC_MSG_RESULT([yes]) + AC_DEFINE(HAVE_VFS_READPAGES, 1, + [address_space_operations->readpages exists]) + ],[ + AC_MSG_RESULT([no]) + ]) +]) diff --git a/config/kernel-revalidate-disk-size.m4 b/config/kernel-revalidate-disk-size.m4 index a7d0cb3cdab4..13cb92a174e3 100644 --- a/config/kernel-revalidate-disk-size.m4 +++ b/config/kernel-revalidate-disk-size.m4 @@ -8,14 +8,14 @@ dnl # AC_DEFUN([ZFS_AC_KERNEL_SRC_REVALIDATE_DISK], [ ZFS_LINUX_TEST_SRC([revalidate_disk_size], [ - #include + #include ], [ struct gendisk *disk = NULL; (void) revalidate_disk_size(disk, false); ]) ZFS_LINUX_TEST_SRC([revalidate_disk], [ - #include + #include ], [ struct gendisk *disk = NULL; (void) revalidate_disk(disk); diff --git a/config/kernel-shrink.m4 b/config/kernel-shrink.m4 index a40c86d5c57f..8cf0f2761bde 100644 --- a/config/kernel-shrink.m4 +++ b/config/kernel-shrink.m4 @@ -84,7 +84,7 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_SHRINKER_CALLBACK], [ AC_DEFUN([ZFS_AC_KERNEL_SHRINKER_CALLBACK],[ dnl # dnl # 3.0 - 3.11 API change - dnl # ->shrink(struct shrinker *, struct shrink_control *sc) + dnl # cs->shrink(struct shrinker *, struct shrink_control *sc) dnl # AC_MSG_CHECKING([whether new 2-argument shrinker exists]) ZFS_LINUX_TEST_RESULT([shrinker_cb_shrink_control], [ @@ -96,14 +96,14 @@ AC_DEFUN([ZFS_AC_KERNEL_SHRINKER_CALLBACK],[ dnl # dnl # 3.12 API change, - dnl # ->shrink() is logically split in to - dnl # ->count_objects() and ->scan_objects() + dnl # cs->shrink() is logically split in to + dnl # cs->count_objects() and cs->scan_objects() dnl # - AC_MSG_CHECKING([whether ->count_objects callback exists]) + AC_MSG_CHECKING([whether cs->count_objects callback exists]) ZFS_LINUX_TEST_RESULT([shrinker_cb_shrink_control_split], [ AC_MSG_RESULT(yes) AC_DEFINE(HAVE_SPLIT_SHRINKER_CALLBACK, 1, - [->count_objects exists]) + [cs->count_objects exists]) ],[ ZFS_LINUX_TEST_ERROR([shrinker]) ]) diff --git a/config/kernel-sysfs.m4 b/config/kernel-sysfs.m4 new file mode 100644 index 000000000000..bbc77c8fc5c0 --- /dev/null +++ b/config/kernel-sysfs.m4 @@ -0,0 +1,37 @@ +dnl # +dnl # Linux 5.2/5.18 API +dnl # +dnl # In cdb4f26a63c391317e335e6e683a614358e70aeb ("kobject: kobj_type: remove default_attrs") +dnl # struct kobj_type.default_attrs +dnl # was finally removed in favour of +dnl # struct kobj_type.default_groups +dnl # +dnl # This was added in aa30f47cf666111f6bbfd15f290a27e8a7b9d854 ("kobject: Add support for default attribute groups to kobj_type"), +dnl # if both are present (5.2-5.17), we prefer default_groups; they're otherwise equivalent +dnl # +AC_DEFUN([ZFS_AC_KERNEL_SRC_SYSFS_DEFAULT_GROUPS], [ + ZFS_LINUX_TEST_SRC([sysfs_default_groups], [ + #include + ],[ + struct kobj_type __attribute__ ((unused)) kt = { + .default_groups = (const struct attribute_group **)NULL }; + ]) +]) + +AC_DEFUN([ZFS_AC_KERNEL_SYSFS_DEFAULT_GROUPS], [ + AC_MSG_CHECKING([whether struct kobj_type.default_groups exists]) + ZFS_LINUX_TEST_RESULT([sysfs_default_groups],[ + AC_MSG_RESULT(yes) + AC_DEFINE([HAVE_SYSFS_DEFAULT_GROUPS], 1, [struct kobj_type has default_groups]) + ],[ + AC_MSG_RESULT(no) + ]) +]) + +AC_DEFUN([ZFS_AC_KERNEL_SRC_SYSFS], [ + ZFS_AC_KERNEL_SRC_SYSFS_DEFAULT_GROUPS +]) + +AC_DEFUN([ZFS_AC_KERNEL_SYSFS], [ + ZFS_AC_KERNEL_SYSFS_DEFAULT_GROUPS +]) diff --git a/config/kernel-user-ns-inum.m4 b/config/kernel-user-ns-inum.m4 new file mode 100644 index 000000000000..2207a4aa6921 --- /dev/null +++ b/config/kernel-user-ns-inum.m4 @@ -0,0 +1,23 @@ +dnl # +dnl # 3.18 API change +dnl # struct user_namespace inum moved from .proc_inum to .ns.inum. +dnl # +AC_DEFUN([ZFS_AC_KERNEL_SRC_USER_NS_COMMON_INUM], [ + ZFS_LINUX_TEST_SRC([user_ns_common_inum], [ + #include + ], [ + struct user_namespace uns; + uns.ns.inum = 0; + ]) +]) + +AC_DEFUN([ZFS_AC_KERNEL_USER_NS_COMMON_INUM], [ + AC_MSG_CHECKING([whether user_namespace->ns.inum exists]) + ZFS_LINUX_TEST_RESULT([user_ns_common_inum], [ + AC_MSG_RESULT(yes) + AC_DEFINE(HAVE_USER_NS_COMMON_INUM, 1, + [user_namespace->ns.inum exists]) + ],[ + AC_MSG_RESULT(no) + ]) +]) diff --git a/config/kernel-vfs-filemap_dirty_folio.m4 b/config/kernel-vfs-filemap_dirty_folio.m4 new file mode 100644 index 000000000000..729ca670da03 --- /dev/null +++ b/config/kernel-vfs-filemap_dirty_folio.m4 @@ -0,0 +1,30 @@ +dnl # +dnl # Linux 5.18 uses filemap_dirty_folio in lieu of +dnl # ___set_page_dirty_nobuffers +dnl # +AC_DEFUN([ZFS_AC_KERNEL_SRC_VFS_FILEMAP_DIRTY_FOLIO], [ + ZFS_LINUX_TEST_SRC([vfs_has_filemap_dirty_folio], [ + #include + #include + + static const struct address_space_operations + aops __attribute__ ((unused)) = { + .dirty_folio = filemap_dirty_folio, + }; + ],[]) +]) + +AC_DEFUN([ZFS_AC_KERNEL_VFS_FILEMAP_DIRTY_FOLIO], [ + dnl # + dnl # Linux 5.18 uses filemap_dirty_folio in lieu of + dnl # ___set_page_dirty_nobuffers + dnl # + AC_MSG_CHECKING([whether filemap_dirty_folio exists]) + ZFS_LINUX_TEST_RESULT([vfs_has_filemap_dirty_folio], [ + AC_MSG_RESULT([yes]) + AC_DEFINE(HAVE_VFS_FILEMAP_DIRTY_FOLIO, 1, + [filemap_dirty_folio exists]) + ],[ + AC_MSG_RESULT([no]) + ]) +]) diff --git a/config/kernel-vfs-iov_iter.m4 b/config/kernel-vfs-iov_iter.m4 index 57f78745a24b..e0617faab02c 100644 --- a/config/kernel-vfs-iov_iter.m4 +++ b/config/kernel-vfs-iov_iter.m4 @@ -134,6 +134,8 @@ AC_DEFUN([ZFS_AC_KERNEL_VFS_IOV_ITER], [ AC_DEFINE(HAVE_IOV_ITER_FAULT_IN_READABLE, 1, [iov_iter_fault_in_readable() is available]) ],[ + AC_MSG_RESULT(no) + AC_MSG_CHECKING([whether fault_in_iov_iter_readable() is available]) ZFS_LINUX_TEST_RESULT([fault_in_iov_iter_readable], [ AC_MSG_RESULT(yes) diff --git a/config/kernel-vfs-read_folio.m4 b/config/kernel-vfs-read_folio.m4 new file mode 100644 index 000000000000..9ca0faff218d --- /dev/null +++ b/config/kernel-vfs-read_folio.m4 @@ -0,0 +1,32 @@ +dnl # +dnl # Linux 5.19 uses read_folio in lieu of readpage +dnl # +AC_DEFUN([ZFS_AC_KERNEL_SRC_VFS_READ_FOLIO], [ + ZFS_LINUX_TEST_SRC([vfs_has_read_folio], [ + #include + + static int + test_read_folio(struct file *file, struct folio *folio) { + (void) file; (void) folio; + return (0); + } + + static const struct address_space_operations + aops __attribute__ ((unused)) = { + .read_folio = test_read_folio, + }; + ],[]) +]) + +AC_DEFUN([ZFS_AC_KERNEL_VFS_READ_FOLIO], [ + dnl # + dnl # Linux 5.19 uses read_folio in lieu of readpage + dnl # + AC_MSG_CHECKING([whether read_folio exists]) + ZFS_LINUX_TEST_RESULT([vfs_has_read_folio], [ + AC_MSG_RESULT([yes]) + AC_DEFINE(HAVE_VFS_READ_FOLIO, 1, [read_folio exists]) + ],[ + AC_MSG_RESULT([no]) + ]) +]) diff --git a/config/kernel-vfs-set_page_dirty.m4 b/config/kernel-vfs-set_page_dirty.m4 index a9d252e4e01e..90cb28f3682c 100644 --- a/config/kernel-vfs-set_page_dirty.m4 +++ b/config/kernel-vfs-set_page_dirty.m4 @@ -23,7 +23,7 @@ AC_DEFUN([ZFS_AC_KERNEL_VFS_SET_PAGE_DIRTY_NOBUFFERS], [ dnl # Linux 5.14 change requires set_page_dirty() to be assigned dnl # in address_space_operations() dnl # - AC_MSG_CHECKING([__set_page_dirty_nobuffers exists]) + AC_MSG_CHECKING([whether __set_page_dirty_nobuffers exists]) ZFS_LINUX_TEST_RESULT([vfs_has_set_page_dirty_nobuffers], [ AC_MSG_RESULT([yes]) AC_DEFINE(HAVE_VFS_SET_PAGE_DIRTY_NOBUFFERS, 1, diff --git a/config/kernel-zero_page.m4 b/config/kernel-zero_page.m4 new file mode 100644 index 000000000000..1461781acb41 --- /dev/null +++ b/config/kernel-zero_page.m4 @@ -0,0 +1,27 @@ +dnl # +dnl # ZERO_PAGE() is an alias for emtpy_zero_page. On certain architectures +dnl # this is a GPL exported variable. +dnl # + +dnl # +dnl # Checking if ZERO_PAGE is exported GPL-only +dnl # +AC_DEFUN([ZFS_AC_KERNEL_SRC_ZERO_PAGE], [ + ZFS_LINUX_TEST_SRC([zero_page], [ + #include + ], [ + struct page *p __attribute__ ((unused)); + p = ZERO_PAGE(0); + ], [], [ZFS_META_LICENSE]) +]) + +AC_DEFUN([ZFS_AC_KERNEL_ZERO_PAGE], [ + AC_MSG_CHECKING([whether ZERO_PAGE() is GPL-only]) + ZFS_LINUX_TEST_RESULT([zero_page_license], [ + AC_MSG_RESULT(no) + ], [ + AC_MSG_RESULT(yes) + AC_DEFINE(HAVE_ZERO_PAGE_GPL_ONLY, 1, + [ZERO_PAGE() is GPL-only]) + ]) +]) diff --git a/config/kernel.m4 b/config/kernel.m4 index b918f045341b..1f274cbe4f30 100644 --- a/config/kernel.m4 +++ b/config/kernel.m4 @@ -8,8 +8,8 @@ AC_DEFUN([ZFS_AC_CONFIG_KERNEL], [ ZFS_AC_QAT dnl # Sanity checks for module building and CONFIG_* defines - ZFS_AC_KERNEL_TEST_MODULE ZFS_AC_KERNEL_CONFIG_DEFINED + ZFS_AC_MODULE_SYMVERS dnl # Sequential ZFS_LINUX_TRY_COMPILE tests ZFS_AC_KERNEL_FPU_HEADER @@ -61,6 +61,7 @@ AC_DEFUN([ZFS_AC_KERNEL_TEST_SRC], [ ZFS_AC_KERNEL_SRC_BIO ZFS_AC_KERNEL_SRC_BLKDEV ZFS_AC_KERNEL_SRC_BLK_QUEUE + ZFS_AC_KERNEL_SRC_GENHD_FLAGS ZFS_AC_KERNEL_SRC_REVALIDATE_DISK ZFS_AC_KERNEL_SRC_GET_DISK_RO ZFS_AC_KERNEL_SRC_GENERIC_READLINK_GLOBAL @@ -82,6 +83,7 @@ AC_DEFUN([ZFS_AC_KERNEL_TEST_SRC], [ ZFS_AC_KERNEL_SRC_MKDIR ZFS_AC_KERNEL_SRC_LOOKUP_FLAGS ZFS_AC_KERNEL_SRC_CREATE + ZFS_AC_KERNEL_SRC_PERMISSION ZFS_AC_KERNEL_SRC_GET_LINK ZFS_AC_KERNEL_SRC_PUT_LINK ZFS_AC_KERNEL_SRC_TMPFILE @@ -99,10 +101,14 @@ AC_DEFUN([ZFS_AC_KERNEL_TEST_SRC], [ ZFS_AC_KERNEL_SRC_SET_NLINK ZFS_AC_KERNEL_SRC_SGET ZFS_AC_KERNEL_SRC_LSEEK_EXECUTE + ZFS_AC_KERNEL_SRC_VFS_FILEMAP_DIRTY_FOLIO + ZFS_AC_KERNEL_SRC_VFS_READ_FOLIO ZFS_AC_KERNEL_SRC_VFS_GETATTR ZFS_AC_KERNEL_SRC_VFS_FSYNC_2ARGS ZFS_AC_KERNEL_SRC_VFS_ITERATE ZFS_AC_KERNEL_SRC_VFS_DIRECT_IO + ZFS_AC_KERNEL_SRC_VFS_READPAGES + ZFS_AC_KERNEL_SRC_VFS_SET_PAGE_DIRTY_NOBUFFERS ZFS_AC_KERNEL_SRC_VFS_RW_ITERATE ZFS_AC_KERNEL_SRC_VFS_GENERIC_WRITE_CHECKS ZFS_AC_KERNEL_SRC_VFS_IOV_ITER @@ -131,12 +137,15 @@ AC_DEFUN([ZFS_AC_KERNEL_TEST_SRC], [ ZFS_AC_KERNEL_SRC_BIO_MAX_SEGS ZFS_AC_KERNEL_SRC_SIGNAL_STOP ZFS_AC_KERNEL_SRC_SIGINFO + ZFS_AC_KERNEL_SRC_SYSFS ZFS_AC_KERNEL_SRC_SET_SPECIAL_STATE - ZFS_AC_KERNEL_SRC_VFS_SET_PAGE_DIRTY_NOBUFFERS ZFS_AC_KERNEL_SRC_STANDALONE_LINUX_STDARG ZFS_AC_KERNEL_SRC_PAGEMAP_FOLIO_WAIT_BIT ZFS_AC_KERNEL_SRC_ADD_DISK ZFS_AC_KERNEL_SRC_KTHREAD + ZFS_AC_KERNEL_SRC_ZERO_PAGE + ZFS_AC_KERNEL_SRC___COPY_FROM_USER_INATOMIC + ZFS_AC_KERNEL_SRC_USER_NS_COMMON_INUM AC_MSG_CHECKING([for available kernel interfaces]) ZFS_LINUX_TEST_COMPILE_ALL([kabi]) @@ -171,6 +180,7 @@ AC_DEFUN([ZFS_AC_KERNEL_TEST_RESULT], [ ZFS_AC_KERNEL_BIO ZFS_AC_KERNEL_BLKDEV ZFS_AC_KERNEL_BLK_QUEUE + ZFS_AC_KERNEL_GENHD_FLAGS ZFS_AC_KERNEL_REVALIDATE_DISK ZFS_AC_KERNEL_GET_DISK_RO ZFS_AC_KERNEL_GENERIC_READLINK_GLOBAL @@ -192,6 +202,7 @@ AC_DEFUN([ZFS_AC_KERNEL_TEST_RESULT], [ ZFS_AC_KERNEL_MKDIR ZFS_AC_KERNEL_LOOKUP_FLAGS ZFS_AC_KERNEL_CREATE + ZFS_AC_KERNEL_PERMISSION ZFS_AC_KERNEL_GET_LINK ZFS_AC_KERNEL_PUT_LINK ZFS_AC_KERNEL_TMPFILE @@ -209,10 +220,14 @@ AC_DEFUN([ZFS_AC_KERNEL_TEST_RESULT], [ ZFS_AC_KERNEL_SET_NLINK ZFS_AC_KERNEL_SGET ZFS_AC_KERNEL_LSEEK_EXECUTE + ZFS_AC_KERNEL_VFS_FILEMAP_DIRTY_FOLIO + ZFS_AC_KERNEL_VFS_READ_FOLIO ZFS_AC_KERNEL_VFS_GETATTR ZFS_AC_KERNEL_VFS_FSYNC_2ARGS ZFS_AC_KERNEL_VFS_ITERATE ZFS_AC_KERNEL_VFS_DIRECT_IO + ZFS_AC_KERNEL_VFS_READPAGES + ZFS_AC_KERNEL_VFS_SET_PAGE_DIRTY_NOBUFFERS ZFS_AC_KERNEL_VFS_RW_ITERATE ZFS_AC_KERNEL_VFS_GENERIC_WRITE_CHECKS ZFS_AC_KERNEL_VFS_IOV_ITER @@ -241,12 +256,15 @@ AC_DEFUN([ZFS_AC_KERNEL_TEST_RESULT], [ ZFS_AC_KERNEL_BIO_MAX_SEGS ZFS_AC_KERNEL_SIGNAL_STOP ZFS_AC_KERNEL_SIGINFO + ZFS_AC_KERNEL_SYSFS ZFS_AC_KERNEL_SET_SPECIAL_STATE - ZFS_AC_KERNEL_VFS_SET_PAGE_DIRTY_NOBUFFERS ZFS_AC_KERNEL_STANDALONE_LINUX_STDARG ZFS_AC_KERNEL_PAGEMAP_FOLIO_WAIT_BIT ZFS_AC_KERNEL_ADD_DISK ZFS_AC_KERNEL_KTHREAD + ZFS_AC_KERNEL_ZERO_PAGE + ZFS_AC_KERNEL___COPY_FROM_USER_INATOMIC + ZFS_AC_KERNEL_USER_NS_COMMON_INUM ]) dnl # @@ -280,6 +298,35 @@ AC_DEFUN([ZFS_AC_MODULE_SYMVERS], [ dnl # dnl # Detect the kernel to be built against dnl # +dnl # Most modern Linux distributions have separate locations for bare +dnl # source (source) and prebuilt (build) files. Additionally, there are +dnl # `source` and `build` symlinks in `/lib/modules/$(KERNEL_VERSION)` +dnl # pointing to them. The directory search order is now: +dnl # +dnl # - `configure` command line values if both `--with-linux` and +dnl # `--with-linux-obj` were defined +dnl # +dnl # - If only `--with-linux` was defined, `--with-linux-obj` is assumed +dnl # to have the same value as `--with-linux` +dnl # +dnl # - If neither `--with-linux` nor `--with-linux-obj` were defined +dnl # autodetection is used: +dnl # +dnl # - `/lib/modules/$(uname -r)/{source,build}` respectively, if exist. +dnl # +dnl # - If only `/lib/modules/$(uname -r)/build` exists, it is assumed +dnl # to be both source and build directory. +dnl # +dnl # - The first directory in `/lib/modules` with the highest version +dnl # number according to `sort -V` which contains both `source` and +dnl # `build` symlinks/directories. If module directory contains only +dnl # `build` component, it is assumed to be both source and build +dnl # directory. +dnl # +dnl # - Last resort: the first directory matching `/usr/src/kernels/*` +dnl # and `/usr/src/linux-*` with the highest version number according +dnl # to `sort -V` is assumed to be both source and build directory. +dnl # AC_DEFUN([ZFS_AC_KERNEL], [ AC_ARG_WITH([linux], AS_HELP_STRING([--with-linux=PATH], @@ -291,25 +338,52 @@ AC_DEFUN([ZFS_AC_KERNEL], [ [Path to kernel build objects]), [kernelbuild="$withval"]) - AC_MSG_CHECKING([kernel source directory]) - AS_IF([test -z "$kernelsrc"], [ - AS_IF([test -e "/lib/modules/$(uname -r)/source"], [ - headersdir="/lib/modules/$(uname -r)/source" - sourcelink=$(readlink -f "$headersdir") + AC_MSG_CHECKING([kernel source and build directories]) + AS_IF([test -n "$kernelsrc" && test -z "$kernelbuild"], [ + kernelbuild="$kernelsrc" + ], [test -z "$kernelsrc"], [ + AS_IF([test -e "/lib/modules/$(uname -r)/source" && \ + test -e "/lib/modules/$(uname -r)/build"], [ + src="/lib/modules/$(uname -r)/source" + build="/lib/modules/$(uname -r)/build" ], [test -e "/lib/modules/$(uname -r)/build"], [ - headersdir="/lib/modules/$(uname -r)/build" - sourcelink=$(readlink -f "$headersdir") + build="/lib/modules/$(uname -r)/build" + src="$build" ], [ - sourcelink=$(ls -1d /usr/src/kernels/* \ - /usr/src/linux-* \ - 2>/dev/null | grep -v obj | tail -1) + src= + + for d in $(ls -1d /lib/modules/* 2>/dev/null | sort -Vr); do + if test -e "$d/source" && test -e "$d/build"; then + src="$d/source" + build="$d/build" + break + fi + + if test -e "$d/build"; then + src="$d/build" + build="$d/build" + break + fi + done + + # the least reliable method + if test -z "$src"; then + src=$(ls -1d /usr/src/kernels/* /usr/src/linux-* \ + 2>/dev/null | grep -v obj | sort -Vr | head -1) + build="$src" + fi ]) - AS_IF([test -n "$sourcelink" && test -e ${sourcelink}], [ - kernelsrc=`readlink -f ${sourcelink}` + AS_IF([test -n "$src" && test -e "$src"], [ + kernelsrc=$(readlink -e "$src") ], [ kernelsrc="[Not found]" ]) + AS_IF([test -n "$build" && test -e "$build"], [ + kernelbuild=$(readlink -e "$build") + ], [ + kernelbuild="[Not found]" + ]) ], [ AS_IF([test "$kernelsrc" = "NONE"], [ kernsrcver=NONE @@ -317,39 +391,28 @@ AC_DEFUN([ZFS_AC_KERNEL], [ withlinux=yes ]) + AC_MSG_RESULT([done]) + AC_MSG_CHECKING([kernel source directory]) AC_MSG_RESULT([$kernelsrc]) - AS_IF([test ! -d "$kernelsrc"], [ + AC_MSG_CHECKING([kernel build directory]) + AC_MSG_RESULT([$kernelbuild]) + AS_IF([test ! -d "$kernelsrc" || test ! -d "$kernelbuild"], [ AC_MSG_ERROR([ *** Please make sure the kernel devel package for your distribution *** is installed and then try again. If that fails, you can specify the - *** location of the kernel source with the '--with-linux=PATH' option.]) - ]) - - AC_MSG_CHECKING([kernel build directory]) - AS_IF([test -z "$kernelbuild"], [ - AS_IF([test x$withlinux != xyes -a -e "/lib/modules/$(uname -r)/build"], [ - kernelbuild=`readlink -f /lib/modules/$(uname -r)/build` - ], [test -d ${kernelsrc}-obj/${target_cpu}/${target_cpu}], [ - kernelbuild=${kernelsrc}-obj/${target_cpu}/${target_cpu} - ], [test -d ${kernelsrc}-obj/${target_cpu}/default], [ - kernelbuild=${kernelsrc}-obj/${target_cpu}/default - ], [test -d `dirname ${kernelsrc}`/build-${target_cpu}], [ - kernelbuild=`dirname ${kernelsrc}`/build-${target_cpu} - ], [ - kernelbuild=${kernelsrc} - ]) + *** location of the kernel source and build with the '--with-linux=PATH' and + *** '--with-linux-obj=PATH' options respectively.]) ]) - AC_MSG_RESULT([$kernelbuild]) AC_MSG_CHECKING([kernel source version]) utsrelease1=$kernelbuild/include/linux/version.h utsrelease2=$kernelbuild/include/linux/utsrelease.h utsrelease3=$kernelbuild/include/generated/utsrelease.h - AS_IF([test -r $utsrelease1 && fgrep -q UTS_RELEASE $utsrelease1], [ + AS_IF([test -r $utsrelease1 && grep -qF UTS_RELEASE $utsrelease1], [ utsrelease=$utsrelease1 - ], [test -r $utsrelease2 && fgrep -q UTS_RELEASE $utsrelease2], [ + ], [test -r $utsrelease2 && grep -qF UTS_RELEASE $utsrelease2], [ utsrelease=$utsrelease2 - ], [test -r $utsrelease3 && fgrep -q UTS_RELEASE $utsrelease3], [ + ], [test -r $utsrelease3 && grep -qF UTS_RELEASE $utsrelease3], [ utsrelease=$utsrelease3 ]) @@ -390,8 +453,6 @@ AC_DEFUN([ZFS_AC_KERNEL], [ AC_SUBST(LINUX) AC_SUBST(LINUX_OBJ) AC_SUBST(LINUX_VERSION) - - ZFS_AC_MODULE_SYMVERS ]) dnl # @@ -486,27 +547,6 @@ AC_DEFUN([ZFS_AC_QAT], [ ]) ]) -dnl # -dnl # Basic toolchain sanity check. -dnl # -AC_DEFUN([ZFS_AC_KERNEL_TEST_MODULE], [ - AC_MSG_CHECKING([whether modules can be built]) - ZFS_LINUX_TRY_COMPILE([], [], [ - AC_MSG_RESULT([yes]) - ],[ - AC_MSG_RESULT([no]) - if test "x$enable_linux_builtin" != xyes; then - AC_MSG_ERROR([ - *** Unable to build an empty module. - ]) - else - AC_MSG_ERROR([ - *** Unable to build an empty module. - *** Please run 'make scripts' inside the kernel source tree.]) - fi - ]) -]) - dnl # dnl # ZFS_LINUX_CONFTEST_H dnl # @@ -609,8 +649,10 @@ AC_DEFUN([ZFS_LINUX_COMPILE], [ build kernel modules with LLVM/CLANG toolchain]) AC_TRY_COMMAND([ KBUILD_MODPOST_NOFINAL="$5" KBUILD_MODPOST_WARN="$6" - make modules -k -j$TEST_JOBS ${KERNEL_CC:+CC=$KERNEL_CC} ${KERNEL_LD:+LD=$KERNEL_LD} ${KERNEL_LLVM:+LLVM=$KERNEL_LLVM} -C $LINUX_OBJ $ARCH_UM - M=$PWD/$1 >$1/build.log 2>&1]) + make modules -k -j$TEST_JOBS ${KERNEL_CC:+CC=$KERNEL_CC} + ${KERNEL_LD:+LD=$KERNEL_LD} ${KERNEL_LLVM:+LLVM=$KERNEL_LLVM} + CONFIG_MODULES=y CFLAGS_MODULE=-DCONFIG_MODULES + -C $LINUX_OBJ $ARCH_UM M=$PWD/$1 >$1/build.log 2>&1]) AS_IF([AC_TRY_COMMAND([$2])], [$3], [$4]) ]) diff --git a/config/user-aio.h.m4 b/config/user-aio.h.m4 new file mode 100644 index 000000000000..152c0946722f --- /dev/null +++ b/config/user-aio.h.m4 @@ -0,0 +1,7 @@ +dnl # +dnl # POSIX specifies as part of realtime extensions, +dnl # and is missing from at least uClibc – force fallbacks there +dnl # +AC_DEFUN([ZFS_AC_CONFIG_USER_AIO_H], [ + ZFS_AC_FIND_SYSTEM_LIBRARY(AIO_H, [], [aio.h], [], [rt], [lio_listio]) +]) diff --git a/config/user-libaio.m4 b/config/user-libaio.m4 index 95c144d76b4f..8009bd11b3e6 100644 --- a/config/user-libaio.m4 +++ b/config/user-libaio.m4 @@ -1,5 +1,5 @@ dnl # -dnl # Check for libaio - only used for libaiot test cases. +dnl # Check for libaio - only used for mmap_libaio test cases. dnl # AC_DEFUN([ZFS_AC_CONFIG_USER_LIBAIO], [ ZFS_AC_FIND_SYSTEM_LIBRARY(LIBAIO, [], [libaio.h], [], [aio], [], [user_libaio=yes], [user_libaio=no]) diff --git a/config/user-systemd.m4 b/config/user-systemd.m4 index 63f02ad2a82b..e4fd0b57c2b7 100644 --- a/config/user-systemd.m4 +++ b/config/user-systemd.m4 @@ -35,16 +35,15 @@ AC_DEFUN([ZFS_AC_CONFIG_USER_SYSTEMD], [ AC_MSG_RESULT([$enable_systemd]) AS_IF([test "x$enable_systemd" = xyes], [ - ZFS_INIT_SYSTEMD=systemd - ZFS_MODULE_LOAD=modules-load.d DEFINE_SYSTEMD='--with systemd --define "_unitdir $(systemdunitdir)" --define "_presetdir $(systemdpresetdir)" --define "_generatordir $(systemdgeneratordir)"' modulesloaddir=$systemdmodulesloaddir ],[ DEFINE_SYSTEMD='--without systemd' ]) - AC_SUBST(ZFS_INIT_SYSTEMD) - AC_SUBST(ZFS_MODULE_LOAD) + ZFS_INIT_SYSTEMD=$enable_systemd + ZFS_WANT_MODULES_LOAD_D=$enable_systemd + AC_SUBST(DEFINE_SYSTEMD) AC_SUBST(systemdunitdir) AC_SUBST(systemdpresetdir) diff --git a/config/user-sysvinit.m4 b/config/user-sysvinit.m4 index b6b63f1cfa36..cf3c8129f0d2 100644 --- a/config/user-sysvinit.m4 +++ b/config/user-sysvinit.m4 @@ -2,10 +2,7 @@ AC_DEFUN([ZFS_AC_CONFIG_USER_SYSVINIT], [ AC_ARG_ENABLE(sysvinit, AS_HELP_STRING([--enable-sysvinit], [install SysV init scripts [default: yes]]), - [],enable_sysvinit=yes) + [], enable_sysvinit=yes) - AS_IF([test "x$enable_sysvinit" = xyes], - [ZFS_INIT_SYSV=init.d]) - - AC_SUBST(ZFS_INIT_SYSV) + ZFS_INIT_SYSV=$enable_sysvinit ]) diff --git a/config/user.m4 b/config/user.m4 index 670820b37715..a7241f44f1fd 100644 --- a/config/user.m4 +++ b/config/user.m4 @@ -23,25 +23,15 @@ AC_DEFUN([ZFS_AC_CONFIG_USER], [ ZFS_AC_CONFIG_USER_LIBAIO ZFS_AC_CONFIG_USER_LIBATOMIC ZFS_AC_CONFIG_USER_LIBFETCH + ZFS_AC_CONFIG_USER_AIO_H ZFS_AC_CONFIG_USER_CLOCK_GETTIME ZFS_AC_CONFIG_USER_PAM ZFS_AC_CONFIG_USER_RUNSTATEDIR ZFS_AC_CONFIG_USER_MAKEDEV_IN_SYSMACROS ZFS_AC_CONFIG_USER_MAKEDEV_IN_MKDEV ZFS_AC_CONFIG_USER_ZFSEXEC - ZFS_AC_TEST_FRAMEWORK AC_CHECK_FUNCS([issetugid mlockall strlcat strlcpy]) -]) - -dnl # -dnl # Setup the environment for the ZFS Test Suite. Currently only -dnl # Linux style systems are supported but this infrastructure can -dnl # be extended to support other platforms if needed. -dnl # -AC_DEFUN([ZFS_AC_TEST_FRAMEWORK], [ - ZONENAME="echo global" - AC_SUBST(ZONENAME) AC_SUBST(RM) ]) diff --git a/config/zfs-build.m4 b/config/zfs-build.m4 index d75867c65d66..d14a6bb7ac9f 100644 --- a/config/zfs-build.m4 +++ b/config/zfs-build.m4 @@ -173,7 +173,7 @@ AC_DEFUN([ZFS_AC_DEBUG_KMEM_TRACKING], [ ]) AC_DEFUN([ZFS_AC_DEBUG_INVARIANTS_DETECT_FREEBSD], [ - AS_IF([sysctl -n kern.conftxt | fgrep -qx $'options\tINVARIANTS'], + AS_IF([sysctl -n kern.conftxt | grep -Fqx $'options\tINVARIANTS'], [enable_invariants="yes"], [enable_invariants="no"]) ]) @@ -209,8 +209,8 @@ AC_DEFUN([ZFS_AC_CONFIG_ALWAYS], [ AX_COUNT_CPUS([]) AC_SUBST(CPU_COUNT) - ZFS_AC_CONFIG_ALWAYS_CC_NO_UNUSED_BUT_SET_VARIABLE - ZFS_AC_CONFIG_ALWAYS_CC_NO_BOOL_COMPARE + ZFS_AC_CONFIG_ALWAYS_CC_NO_CLOBBERED + ZFS_AC_CONFIG_ALWAYS_CC_INFINITE_RECURSION ZFS_AC_CONFIG_ALWAYS_CC_IMPLICIT_FALLTHROUGH ZFS_AC_CONFIG_ALWAYS_CC_FRAME_LARGER_THAN ZFS_AC_CONFIG_ALWAYS_CC_NO_FORMAT_TRUNCATION @@ -260,11 +260,16 @@ AC_DEFUN([ZFS_AC_CONFIG], [ AC_SUBST(TEST_JOBS) ]) + ZFS_INIT_SYSV= + ZFS_INIT_SYSTEMD= + ZFS_WANT_MODULES_LOAD_D= + case "$ZFS_CONFIG" in kernel) ZFS_AC_CONFIG_KERNEL ;; user) ZFS_AC_CONFIG_USER ;; all) ZFS_AC_CONFIG_USER ZFS_AC_CONFIG_KERNEL ;; + dist) ;; srpm) ;; *) AC_MSG_RESULT([Error!]) @@ -272,6 +277,10 @@ AC_DEFUN([ZFS_AC_CONFIG], [ user kernel|user|all|srpm]) ;; esac + AM_CONDITIONAL([INIT_SYSV], [test "x$ZFS_INIT_SYSV" = "xyes"]) + AM_CONDITIONAL([INIT_SYSTEMD], [test "x$ZFS_INIT_SYSTEMD" = "xyes"]) + AM_CONDITIONAL([WANT_MODULES_LOAD_D], [test "x$ZFS_WANT_MODULES_LOAD_D" = "xyes"]) + AM_CONDITIONAL([CONFIG_USER], [test "$ZFS_CONFIG" = user -o "$ZFS_CONFIG" = all]) AM_CONDITIONAL([CONFIG_KERNEL], @@ -326,6 +335,10 @@ AC_DEFUN([ZFS_AC_RPM], [ RPM_DEFINE_COMMON=${RPM_DEFINE_COMMON}' --define "$(ASAN_ZFS) 1"' RPM_DEFINE_COMMON=${RPM_DEFINE_COMMON}' --define "$(UBSAN_ZFS) 1"' + AS_IF([test "x$enable_debuginfo" = xyes], [ + RPM_DEFINE_COMMON=${RPM_DEFINE_COMMON}' --define "__strip /bin/true"' + ]) + RPM_DEFINE_UTIL=' --define "_initconfdir $(initconfdir)"' dnl # Make the next three RPM_DEFINE_UTIL additions conditional, since diff --git a/configure.ac b/configure.ac index 7037c06b225f..5cb25b32ae2c 100644 --- a/configure.ac +++ b/configure.ac @@ -1,5 +1,5 @@ /* - * This file is part of the ZFS Linux port. + * This file is part of OpenZFS. * * Copyright (c) 2009 Lawrence Livermore National Security, LLC. * Produced at Lawrence Livermore National Laboratory @@ -17,7 +17,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -30,8 +30,8 @@ * CDDL HEADER END */ -AC_INIT(m4_esyscmd(grep ^Name: META | cut -d ':' -f 2 | tr -d ' \n'), - m4_esyscmd(grep ^Version: META | cut -d ':' -f 2 | tr -d ' \n')) +AC_INIT(m4_esyscmd(awk '/^Name:/ {printf $2}' META), + m4_esyscmd(awk '/^Version:/ {printf $2}' META)) AC_LANG(C) ZFS_AC_META AC_CONFIG_AUX_DIR([config]) @@ -39,11 +39,11 @@ AC_CONFIG_MACRO_DIR([config]) AC_CANONICAL_TARGET AM_MAINTAINER_MODE m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) -AM_INIT_AUTOMAKE([subdir-objects]) +AM_INIT_AUTOMAKE([subdir-objects foreign]) +# Remove default macros from config.h: +# PACKAGE, PACKAGE_{BUGREPORT,NAME,STRING,TARNAME,VERSION}, STDC_HEADERS, VERSION AC_CONFIG_HEADERS([zfs_config.h], [ - (mv zfs_config.h zfs_config.h.tmp && - awk -f ${ac_srcdir}/config/config.awk zfs_config.h.tmp >zfs_config.h && - rm zfs_config.h.tmp) || exit 1]) + sed -nri~ -e '/^$/be' -e 'N;N;/#define (PACKAGE|VERSION|STDC_HEADERS)/d' -e ':e' -e 'p' zfs_config.h && rm zfs_config.h~ || exit]) LT_INIT AC_PROG_INSTALL @@ -66,353 +66,19 @@ ZFS_AC_DEBUG_INVARIANTS AC_CONFIG_FILES([ Makefile - cmd/Makefile - cmd/arc_summary/Makefile - cmd/arcstat/Makefile - cmd/dbufstat/Makefile - cmd/fsck_zfs/Makefile - cmd/mount_zfs/Makefile - cmd/raidz_test/Makefile - cmd/vdev_id/Makefile - cmd/zdb/Makefile - cmd/zed/Makefile - cmd/zed/zed.d/Makefile - cmd/zfs/Makefile - cmd/zfs_ids_to_path/Makefile - cmd/zgenhostid/Makefile - cmd/zhack/Makefile - cmd/zinject/Makefile - cmd/zpool/Makefile - cmd/zstream/Makefile - cmd/ztest/Makefile - cmd/zvol_id/Makefile - cmd/zvol_wait/Makefile - cmd/zpool_influxdb/Makefile - contrib/Makefile - contrib/bash_completion.d/Makefile - contrib/bpftrace/Makefile - contrib/dracut/02zfsexpandknowledge/Makefile - contrib/dracut/90zfs/Makefile - contrib/dracut/Makefile - contrib/initramfs/Makefile - contrib/initramfs/conf.d/Makefile - contrib/initramfs/conf-hooks.d/Makefile - contrib/initramfs/hooks/Makefile - contrib/initramfs/scripts/Makefile - contrib/initramfs/scripts/local-top/Makefile - contrib/pam_zfs_key/Makefile - contrib/pyzfs/Makefile - contrib/pyzfs/setup.py - contrib/zcp/Makefile - etc/Makefile - etc/default/Makefile - etc/init.d/Makefile - etc/modules-load.d/Makefile - etc/sudoers.d/Makefile - etc/systemd/Makefile - etc/systemd/system-generators/Makefile - etc/systemd/system/Makefile - etc/zfs/Makefile include/Makefile - include/os/Makefile - include/os/freebsd/Makefile - include/os/freebsd/linux/Makefile - include/os/freebsd/spl/Makefile - include/os/freebsd/spl/acl/Makefile - include/os/freebsd/spl/rpc/Makefile - include/os/freebsd/spl/sys/Makefile - include/os/freebsd/zfs/Makefile - include/os/freebsd/zfs/sys/Makefile - include/os/linux/Makefile - include/os/linux/kernel/Makefile - include/os/linux/kernel/linux/Makefile - include/os/linux/spl/Makefile - include/os/linux/spl/rpc/Makefile - include/os/linux/spl/sys/Makefile - include/os/linux/zfs/Makefile - include/os/linux/zfs/sys/Makefile - include/sys/Makefile - include/sys/crypto/Makefile - include/sys/fm/Makefile - include/sys/fm/fs/Makefile - include/sys/fs/Makefile - include/sys/lua/Makefile - include/sys/sysevent/Makefile - include/sys/zstd/Makefile - lib/Makefile - lib/libavl/Makefile - lib/libefi/Makefile - lib/libicp/Makefile - lib/libnvpair/Makefile - lib/libshare/Makefile - lib/libspl/Makefile - lib/libspl/include/Makefile - lib/libspl/include/os/Makefile - lib/libspl/include/os/freebsd/Makefile - lib/libspl/include/os/freebsd/sys/Makefile - lib/libspl/include/os/linux/Makefile - lib/libspl/include/os/linux/sys/Makefile - lib/libspl/include/rpc/Makefile - lib/libspl/include/sys/Makefile - lib/libspl/include/sys/dktp/Makefile - lib/libspl/include/util/Makefile - lib/libtpool/Makefile - lib/libunicode/Makefile - lib/libuutil/Makefile - lib/libzfs/Makefile lib/libzfs/libzfs.pc - lib/libzfsbootenv/Makefile - lib/libzfsbootenv/libzfsbootenv.pc - lib/libzfs_core/Makefile lib/libzfs_core/libzfs_core.pc - lib/libzpool/Makefile - lib/libzstd/Makefile - lib/libzutil/Makefile - man/Makefile + lib/libzfsbootenv/libzfsbootenv.pc module/Kbuild module/Makefile - module/avl/Makefile - module/icp/Makefile - module/lua/Makefile - module/nvpair/Makefile - module/os/linux/spl/Makefile - module/os/linux/zfs/Makefile - module/spl/Makefile - module/unicode/Makefile - module/zcommon/Makefile - module/zfs/Makefile - module/zstd/Makefile - rpm/Makefile - rpm/generic/Makefile rpm/generic/zfs-dkms.spec rpm/generic/zfs-kmod.spec rpm/generic/zfs.spec - rpm/redhat/Makefile rpm/redhat/zfs-dkms.spec rpm/redhat/zfs-kmod.spec rpm/redhat/zfs.spec - scripts/Makefile - tests/Makefile - tests/runfiles/Makefile - tests/test-runner/Makefile - tests/test-runner/bin/Makefile - tests/test-runner/include/Makefile - tests/test-runner/man/Makefile - tests/zfs-tests/Makefile - tests/zfs-tests/callbacks/Makefile - tests/zfs-tests/cmd/Makefile - tests/zfs-tests/cmd/badsend/Makefile - tests/zfs-tests/cmd/btree_test/Makefile - tests/zfs-tests/cmd/chg_usr_exec/Makefile - tests/zfs-tests/cmd/devname2devid/Makefile - tests/zfs-tests/cmd/draid/Makefile - tests/zfs-tests/cmd/dir_rd_update/Makefile - tests/zfs-tests/cmd/file_check/Makefile - tests/zfs-tests/cmd/file_trunc/Makefile - tests/zfs-tests/cmd/file_write/Makefile - tests/zfs-tests/cmd/get_diff/Makefile - tests/zfs-tests/cmd/getversion/Makefile - tests/zfs-tests/cmd/largest_file/Makefile - tests/zfs-tests/cmd/libzfs_input_check/Makefile - tests/zfs-tests/cmd/mkbusy/Makefile - tests/zfs-tests/cmd/mkfile/Makefile - tests/zfs-tests/cmd/mkfiles/Makefile - tests/zfs-tests/cmd/mktree/Makefile - tests/zfs-tests/cmd/mmap_exec/Makefile - tests/zfs-tests/cmd/mmap_libaio/Makefile - tests/zfs-tests/cmd/mmap_seek/Makefile - tests/zfs-tests/cmd/mmapwrite/Makefile - tests/zfs-tests/cmd/nvlist_to_lua/Makefile - tests/zfs-tests/cmd/randfree_file/Makefile - tests/zfs-tests/cmd/randwritecomp/Makefile - tests/zfs-tests/cmd/readmmap/Makefile - tests/zfs-tests/cmd/rename_dir/Makefile - tests/zfs-tests/cmd/rm_lnkcnt_zero_file/Makefile - tests/zfs-tests/cmd/send_doall/Makefile - tests/zfs-tests/cmd/stride_dd/Makefile - tests/zfs-tests/cmd/threadsappend/Makefile - tests/zfs-tests/cmd/user_ns_exec/Makefile - tests/zfs-tests/cmd/xattrtest/Makefile - tests/zfs-tests/include/Makefile tests/zfs-tests/tests/Makefile - tests/zfs-tests/tests/functional/Makefile - tests/zfs-tests/tests/functional/acl/Makefile - tests/zfs-tests/tests/functional/acl/off/Makefile - tests/zfs-tests/tests/functional/acl/posix/Makefile - tests/zfs-tests/tests/functional/acl/posix-sa/Makefile - tests/zfs-tests/tests/functional/alloc_class/Makefile - tests/zfs-tests/tests/functional/arc/Makefile - tests/zfs-tests/tests/functional/atime/Makefile - tests/zfs-tests/tests/functional/bootfs/Makefile - tests/zfs-tests/tests/functional/btree/Makefile - tests/zfs-tests/tests/functional/cache/Makefile - tests/zfs-tests/tests/functional/cachefile/Makefile - tests/zfs-tests/tests/functional/casenorm/Makefile - tests/zfs-tests/tests/functional/channel_program/Makefile - tests/zfs-tests/tests/functional/channel_program/lua_core/Makefile - tests/zfs-tests/tests/functional/channel_program/synctask_core/Makefile - tests/zfs-tests/tests/functional/chattr/Makefile - tests/zfs-tests/tests/functional/checksum/Makefile - tests/zfs-tests/tests/functional/clean_mirror/Makefile - tests/zfs-tests/tests/functional/cli_root/Makefile - tests/zfs-tests/tests/functional/cli_root/zdb/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_change-key/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_clone/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_copies/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_create/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_destroy/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_diff/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_get/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_inherit/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_jail/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_load-key/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_mount/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_program/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_promote/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_property/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_receive/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_rename/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_reservation/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_rollback/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_send/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_set/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_share/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_unmount/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_unshare/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/Makefile - tests/zfs-tests/tests/functional/cli_root/zfs_wait/Makefile - tests/zfs-tests/tests/functional/cli_root/zhack/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_add/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_attach/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_clear/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_create/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_destroy/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_detach/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_events/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_expand/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_export/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_get/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_history/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_import/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_import/blockfiles/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_initialize/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_labelclear/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_offline/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_online/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_remove/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_reopen/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_replace/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_resilver/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_scrub/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_set/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_split/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_status/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_sync/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_trim/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/blockfiles/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_wait/Makefile - tests/zfs-tests/tests/functional/cli_root/zpool_wait/scan/Makefile - tests/zfs-tests/tests/functional/cli_user/Makefile - tests/zfs-tests/tests/functional/cli_user/misc/Makefile - tests/zfs-tests/tests/functional/cli_user/zfs_list/Makefile - tests/zfs-tests/tests/functional/cli_user/zpool_iostat/Makefile - tests/zfs-tests/tests/functional/cli_user/zpool_list/Makefile - tests/zfs-tests/tests/functional/cli_user/zpool_status/Makefile - tests/zfs-tests/tests/functional/compression/Makefile - tests/zfs-tests/tests/functional/cp_files/Makefile - tests/zfs-tests/tests/functional/crtime/Makefile - tests/zfs-tests/tests/functional/ctime/Makefile - tests/zfs-tests/tests/functional/deadman/Makefile - tests/zfs-tests/tests/functional/delegate/Makefile - tests/zfs-tests/tests/functional/devices/Makefile - tests/zfs-tests/tests/functional/events/Makefile - tests/zfs-tests/tests/functional/exec/Makefile - tests/zfs-tests/tests/functional/fallocate/Makefile - tests/zfs-tests/tests/functional/fault/Makefile - tests/zfs-tests/tests/functional/features/Makefile - tests/zfs-tests/tests/functional/features/async_destroy/Makefile - tests/zfs-tests/tests/functional/features/large_dnode/Makefile - tests/zfs-tests/tests/functional/grow/Makefile - tests/zfs-tests/tests/functional/history/Makefile - tests/zfs-tests/tests/functional/hkdf/Makefile - tests/zfs-tests/tests/functional/inheritance/Makefile - tests/zfs-tests/tests/functional/inuse/Makefile - tests/zfs-tests/tests/functional/io/Makefile - tests/zfs-tests/tests/functional/l2arc/Makefile - tests/zfs-tests/tests/functional/large_files/Makefile - tests/zfs-tests/tests/functional/largest_pool/Makefile - tests/zfs-tests/tests/functional/libzfs/Makefile - tests/zfs-tests/tests/functional/limits/Makefile - tests/zfs-tests/tests/functional/link_count/Makefile - tests/zfs-tests/tests/functional/log_spacemap/Makefile - tests/zfs-tests/tests/functional/migration/Makefile - tests/zfs-tests/tests/functional/mmap/Makefile - tests/zfs-tests/tests/functional/mmp/Makefile - tests/zfs-tests/tests/functional/mount/Makefile - tests/zfs-tests/tests/functional/mv_files/Makefile - tests/zfs-tests/tests/functional/nestedfs/Makefile - tests/zfs-tests/tests/functional/no_space/Makefile - tests/zfs-tests/tests/functional/nopwrite/Makefile - tests/zfs-tests/tests/functional/online_offline/Makefile - tests/zfs-tests/tests/functional/pam/Makefile - tests/zfs-tests/tests/functional/pool_checkpoint/Makefile - tests/zfs-tests/tests/functional/pool_names/Makefile - tests/zfs-tests/tests/functional/poolversion/Makefile - tests/zfs-tests/tests/functional/privilege/Makefile - tests/zfs-tests/tests/functional/procfs/Makefile - tests/zfs-tests/tests/functional/projectquota/Makefile - tests/zfs-tests/tests/functional/pyzfs/Makefile - tests/zfs-tests/tests/functional/quota/Makefile - tests/zfs-tests/tests/functional/raidz/Makefile - tests/zfs-tests/tests/functional/redacted_send/Makefile - tests/zfs-tests/tests/functional/redundancy/Makefile - tests/zfs-tests/tests/functional/refquota/Makefile - tests/zfs-tests/tests/functional/refreserv/Makefile - tests/zfs-tests/tests/functional/removal/Makefile - tests/zfs-tests/tests/functional/rename_dirs/Makefile - tests/zfs-tests/tests/functional/replacement/Makefile - tests/zfs-tests/tests/functional/reservation/Makefile - tests/zfs-tests/tests/functional/rootpool/Makefile - tests/zfs-tests/tests/functional/rsend/Makefile - tests/zfs-tests/tests/functional/scrub_mirror/Makefile - tests/zfs-tests/tests/functional/simd/Makefile - tests/zfs-tests/tests/functional/slog/Makefile - tests/zfs-tests/tests/functional/snapshot/Makefile - tests/zfs-tests/tests/functional/snapused/Makefile - tests/zfs-tests/tests/functional/sparse/Makefile - tests/zfs-tests/tests/functional/stat/Makefile - tests/zfs-tests/tests/functional/suid/Makefile - tests/zfs-tests/tests/functional/threadsappend/Makefile - tests/zfs-tests/tests/functional/tmpfile/Makefile - tests/zfs-tests/tests/functional/trim/Makefile - tests/zfs-tests/tests/functional/truncate/Makefile - tests/zfs-tests/tests/functional/upgrade/Makefile - tests/zfs-tests/tests/functional/user_namespace/Makefile - tests/zfs-tests/tests/functional/userquota/Makefile - tests/zfs-tests/tests/functional/vdev_zaps/Makefile - tests/zfs-tests/tests/functional/write_dirs/Makefile - tests/zfs-tests/tests/functional/xattr/Makefile - tests/zfs-tests/tests/functional/zpool_influxdb/Makefile - tests/zfs-tests/tests/functional/zvol/Makefile - tests/zfs-tests/tests/functional/zvol/zvol_ENOSPC/Makefile - tests/zfs-tests/tests/functional/zvol/zvol_cli/Makefile - tests/zfs-tests/tests/functional/zvol/zvol_misc/Makefile - tests/zfs-tests/tests/functional/zvol/zvol_swap/Makefile - tests/zfs-tests/tests/perf/Makefile - tests/zfs-tests/tests/perf/fio/Makefile - tests/zfs-tests/tests/perf/regression/Makefile - tests/zfs-tests/tests/perf/scripts/Makefile - tests/zfs-tests/tests/stress/Makefile - udev/Makefile - udev/rules.d/Makefile zfs.release ]) diff --git a/contrib/Makefile.am b/contrib/Makefile.am index 5ec13ece5327..fa51bd941d27 100644 --- a/contrib/Makefile.am +++ b/contrib/Makefile.am @@ -1,12 +1,13 @@ -include $(top_srcdir)/config/Shellcheck.am +include $(srcdir)/%D%/bash_completion.d/Makefile.am +include $(srcdir)/%D%/pyzfs/Makefile.am +include $(srcdir)/%D%/zcp/Makefile.am -SUBDIRS = bash_completion.d pyzfs zcp if BUILD_LINUX -SUBDIRS += bpftrace dracut initramfs +include $(srcdir)/%D%/bpftrace/Makefile.am +include $(srcdir)/%D%/dracut/Makefile.am +include $(srcdir)/%D%/initramfs/Makefile.am endif + if PAM_ZFS_ENABLED -SUBDIRS += pam_zfs_key +include $(srcdir)/%D%/pam_zfs_key/Makefile.am endif -DIST_SUBDIRS = bash_completion.d bpftrace dracut initramfs pam_zfs_key pyzfs zcp - -SHELLCHECKDIRS = bash_completion.d bpftrace dracut initramfs diff --git a/contrib/bash_completion.d/Makefile.am b/contrib/bash_completion.d/Makefile.am index eee617802bbe..dc4b610c42b8 100644 --- a/contrib/bash_completion.d/Makefile.am +++ b/contrib/bash_completion.d/Makefile.am @@ -1,12 +1,7 @@ -include $(top_srcdir)/config/Substfiles.am -include $(top_srcdir)/config/Shellcheck.am - bashcompletiondir = $(sysconfdir)/bash_completion.d -noinst_DATA = zfs - -EXTRA_DIST += $(noinst_DATA) -SUBSTFILES += $(noinst_DATA) +nodist_bashcompletion_DATA = %D%/zfs +SUBSTFILES += $(nodist_bashcompletion_DATA) -SHELLCHECKSCRIPTS = $(noinst_DATA) -SHELLCHECK_SHELL = bash +SHELLCHECKSCRIPTS += $(nodist_bashcompletion_DATA) +$(call SHELLCHECK_OPTS,$(nodist_bashcompletion_DATA)): SHELLCHECK_SHELL = bash diff --git a/contrib/bpftrace/Makefile.am b/contrib/bpftrace/Makefile.am index 05e4f1c507ce..4f649cf5433e 100644 --- a/contrib/bpftrace/Makefile.am +++ b/contrib/bpftrace/Makefile.am @@ -1,7 +1,3 @@ -include $(top_srcdir)/config/Shellcheck.am +dist_noinst_DATA += %D%/taskqlatency.bt %D%/zfs-trace.sh -EXTRA_DIST = \ - taskqlatency.bt \ - zfs-trace.sh - -SHELLCHECKSCRIPTS = zfs-trace.sh +SHELLCHECKSCRIPTS += %D%/zfs-trace.sh diff --git a/contrib/dracut/90zfs/.gitignore b/contrib/dracut/.gitignore similarity index 100% rename from contrib/dracut/90zfs/.gitignore rename to contrib/dracut/.gitignore diff --git a/contrib/dracut/02zfsexpandknowledge/.gitignore b/contrib/dracut/02zfsexpandknowledge/.gitignore deleted file mode 100644 index 7fb6b964f058..000000000000 --- a/contrib/dracut/02zfsexpandknowledge/.gitignore +++ /dev/null @@ -1 +0,0 @@ -module-setup.sh diff --git a/contrib/dracut/02zfsexpandknowledge/Makefile.am b/contrib/dracut/02zfsexpandknowledge/Makefile.am deleted file mode 100644 index b1bbb6bd3aac..000000000000 --- a/contrib/dracut/02zfsexpandknowledge/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -include $(top_srcdir)/config/Substfiles.am -include $(top_srcdir)/config/Shellcheck.am - -pkgdracutdir = $(dracutdir)/modules.d/02zfsexpandknowledge -pkgdracut_SCRIPTS = \ - module-setup.sh - -SUBSTFILES += $(pkgdracut_SCRIPTS) diff --git a/contrib/dracut/02zfsexpandknowledge/module-setup.sh.in b/contrib/dracut/02zfsexpandknowledge/module-setup.sh.in index a161fbf6f113..df8df3181fce 100755 --- a/contrib/dracut/02zfsexpandknowledge/module-setup.sh.in +++ b/contrib/dracut/02zfsexpandknowledge/module-setup.sh.in @@ -57,6 +57,12 @@ array_contains () { } check() { + # https://github.com/dracutdevs/dracut/pull/1711 provides a zfs_devs + # function to detect the physical devices backing zfs pools. If this + # function exists in the version of dracut this module is being called + # from, then it does not need to run. + type zfs_devs >/dev/null 2>&1 && return 1 + local mp local dev local blockdevs diff --git a/contrib/dracut/90zfs/Makefile.am b/contrib/dracut/90zfs/Makefile.am deleted file mode 100644 index 3f7050300994..000000000000 --- a/contrib/dracut/90zfs/Makefile.am +++ /dev/null @@ -1,24 +0,0 @@ -include $(top_srcdir)/config/Substfiles.am -include $(top_srcdir)/config/Shellcheck.am - -pkgdracutdir = $(dracutdir)/modules.d/90zfs -pkgdracut_SCRIPTS = \ - export-zfs.sh \ - module-setup.sh \ - mount-zfs.sh \ - parse-zfs.sh \ - zfs-generator.sh \ - zfs-load-key.sh \ - zfs-needshutdown.sh \ - zfs-lib.sh \ - import-opts-generator.sh - -pkgdracut_DATA = \ - zfs-env-bootfs.service \ - zfs-snapshot-bootfs.service \ - zfs-rollback-bootfs.service - -SUBSTFILES += $(pkgdracut_SCRIPTS) $(pkgdracut_DATA) - -# Provided by /bin/sleep, and, again, every implementation of that supports this -CHECKBASHISMS_IGNORE = -e 'sleep only takes one integer' -e 'sleep 0.' diff --git a/contrib/dracut/90zfs/export-zfs.sh.in b/contrib/dracut/90zfs/export-zfs.sh.in index 9ad72bd6f5cc..cfe059479436 100755 --- a/contrib/dracut/90zfs/export-zfs.sh.in +++ b/contrib/dracut/90zfs/export-zfs.sh.in @@ -1,30 +1,22 @@ #!/bin/sh -. /lib/dracut-zfs-lib.sh - _do_zpool_export() { - ret=0 - errs="" - final="${1}" - info "ZFS: Exporting ZFS storage pools..." - errs=$(export_all -F 2>&1) + errs=$(zpool export -aF 2>&1) ret=$? - [ -z "${errs}" ] || echo "${errs}" | vwarn - if [ "x${ret}" != "x0" ]; then + echo "${errs}" | vwarn + if [ "${ret}" -ne 0 ]; then info "ZFS: There was a problem exporting pools." fi - if [ "x${final}" != "x" ]; then + if [ -n "$1" ]; then info "ZFS: pool list" zpool list 2>&1 | vinfo fi - return "${ret}" + return "$ret" } if command -v zpool >/dev/null; then _do_zpool_export "${1}" -else - : fi diff --git a/contrib/dracut/90zfs/module-setup.sh.in b/contrib/dracut/90zfs/module-setup.sh.in index 1eaff331eab4..81d7d2abe496 100755 --- a/contrib/dracut/90zfs/module-setup.sh.in +++ b/contrib/dracut/90zfs/module-setup.sh.in @@ -6,138 +6,95 @@ check() { [ "${1}" = "-d" ] && return 0 # Verify the zfs tool chain - for tool in "@sbindir@/zgenhostid" "@sbindir@/zpool" "@sbindir@/zfs" "@mounthelperdir@/mount.zfs" ; do - test -x "$tool" || return 1 + for tool in "zgenhostid" "zpool" "zfs" "mount.zfs"; do + command -v "${tool}" >/dev/null || return 1 done - - return 0 } depends() { echo udev-rules - return 0 } installkernel() { - instmods zfs - instmods zcommon - instmods znvpair - instmods zavl - instmods zunicode - instmods zlua - instmods icp - instmods spl - instmods zlib_deflate - instmods zlib_inflate + instmods -c zfs } install() { - inst_rules @udevruledir@/90-zfs.rules - inst_rules @udevruledir@/69-vdev.rules - inst_rules @udevruledir@/60-zvol.rules - dracut_install hostid - dracut_install grep - dracut_install @sbindir@/zgenhostid - dracut_install @sbindir@/zfs - dracut_install @sbindir@/zpool - # Workaround for https://github.com/openzfs/zfs/issues/4749 by - # ensuring libgcc_s.so(.1) is included - if ldd @sbindir@/zpool | grep -qF 'libgcc_s.so'; then - # Dracut will have already tracked and included it - :; - elif command -v gcc-config >/dev/null 2>&1; then - # On systems with gcc-config (Gentoo, Funtoo, etc.): - # Use the current profile to resolve the appropriate path - s="$(gcc-config -c)" - dracut_install "/usr/lib/gcc/${s%-*}/${s##*-}/libgcc_s.so"* - elif [ "$(echo /usr/lib/libgcc_s.so*)" != "/usr/lib/libgcc_s.so*" ]; then - # Try a simple path first - dracut_install /usr/lib/libgcc_s.so* - elif [ "$(echo /lib*/libgcc_s.so*)" != "/lib*/libgcc_s.so*" ]; then - # SUSE - dracut_install /lib*/libgcc_s.so* - else - # Fallback: Guess the path and include all matches - dracut_install /usr/lib*/gcc/**/libgcc_s.so* - fi - # shellcheck disable=SC2050 - if [ @LIBFETCH_DYNAMIC@ -gt 0 ]; then - for d in $libdirs; do - [ -e "$d/@LIBFETCH_SONAME@" ] && dracut_install "$d/@LIBFETCH_SONAME@" - done + inst_rules 90-zfs.rules 69-vdev.rules 60-zvol.rules + + inst_multiple \ + zgenhostid \ + zfs \ + zpool \ + mount.zfs \ + hostid \ + grep \ + awk \ + tr \ + cut \ + head || + { dfatal "Failed to install essential binaries"; exit 1; } + + # Adapted from https://github.com/zbm-dev/zfsbootmenu + if ! ldd "$(command -v zpool)" | grep -qF 'libgcc_s.so'; then + # On systems with gcc-config (Gentoo, Funtoo, etc.), use it to find libgcc_s + if command -v gcc-config >/dev/null; then + inst_simple "/usr/lib/gcc/$(s=$(gcc-config -c); echo "${s%-*}/${s##*-}")/libgcc_s.so.1" || + { dfatal "Unable to install libgcc_s.so"; exit 1; } + # Otherwise, use dracut's library installation function to find the right one + elif ! inst_libdir_file "libgcc_s.so*"; then + # If all else fails, just try looking for some gcc arch directory + inst_simple /usr/lib/gcc/*/*/libgcc_s.so* || + { dfatal "Unable to install libgcc_s.so"; exit 1; } + fi fi - dracut_install @mounthelperdir@/mount.zfs - dracut_install @udevdir@/vdev_id - dracut_install awk - dracut_install cut - dracut_install tr - dracut_install head - dracut_install @udevdir@/zvol_id + inst_hook cmdline 95 "${moddir}/parse-zfs.sh" - if [ -n "$systemdutildir" ] ; then - inst_script "${moddir}/zfs-generator.sh" "$systemdutildir"/system-generators/dracut-zfs-generator + if [ -n "${systemdutildir}" ]; then + inst_script "${moddir}/zfs-generator.sh" "${systemdutildir}/system-generators/dracut-zfs-generator" fi inst_hook pre-mount 90 "${moddir}/zfs-load-key.sh" inst_hook mount 98 "${moddir}/mount-zfs.sh" inst_hook cleanup 99 "${moddir}/zfs-needshutdown.sh" inst_hook shutdown 20 "${moddir}/export-zfs.sh" - inst_simple "${moddir}/zfs-lib.sh" "/lib/dracut-zfs-lib.sh" - if [ -e @sysconfdir@/zfs/zpool.cache ]; then - inst @sysconfdir@/zfs/zpool.cache - type mark_hostonly >/dev/null 2>&1 && mark_hostonly @sysconfdir@/zfs/zpool.cache - fi + inst_script "${moddir}/zfs-lib.sh" "/lib/dracut-zfs-lib.sh" - if [ -e @sysconfdir@/zfs/vdev_id.conf ]; then - inst @sysconfdir@/zfs/vdev_id.conf - type mark_hostonly >/dev/null 2>&1 && mark_hostonly @sysconfdir@/zfs/vdev_id.conf - fi + # -H ensures they are marked host-only + # -o ensures there is no error upon absence of these files + inst_multiple -o -H \ + "@sysconfdir@/zfs/zpool.cache" \ + "@sysconfdir@/zfs/vdev_id.conf" # Synchronize initramfs and system hostid - if [ -f @sysconfdir@/hostid ]; then - inst @sysconfdir@/hostid - type mark_hostonly >/dev/null 2>&1 && mark_hostonly @sysconfdir@/hostid - elif HOSTID="$(hostid 2>/dev/null)" && [ "${HOSTID}" != "00000000" ]; then - zgenhostid -o "${initdir}@sysconfdir@/hostid" "${HOSTID}" - type mark_hostonly >/dev/null 2>&1 && mark_hostonly @sysconfdir@/hostid + if ! inst_simple -H @sysconfdir@/hostid; then + if HOSTID="$(hostid 2>/dev/null)" && [ "${HOSTID}" != "00000000" ]; then + zgenhostid -o "${initdir}@sysconfdir@/hostid" "${HOSTID}" + mark_hostonly @sysconfdir@/hostid + fi fi if dracut_module_included "systemd"; then - mkdir -p "${initdir}/$systemdsystemunitdir/zfs-import.target.wants" - for _service in "zfs-import-scan.service" "zfs-import-cache.service" ; do - dracut_install "@systemdunitdir@/$_service" - if ! [ -L "${initdir}/$systemdsystemunitdir/zfs-import.target.wants/$_service" ]; then - ln -sf "../$_service" "${initdir}/$systemdsystemunitdir/zfs-import.target.wants/$_service" - type mark_hostonly >/dev/null 2>&1 && mark_hostonly "@systemdunitdir@/$_service" - fi - done + inst_simple "${systemdsystemunitdir}/zfs-import.target" + systemctl -q --root "${initdir}" add-wants initrd.target zfs-import.target - inst "${moddir}"/zfs-env-bootfs.service "${systemdsystemunitdir}"/zfs-env-bootfs.service - ln -s ../zfs-env-bootfs.service "${initdir}/${systemdsystemunitdir}/zfs-import.target.wants"/zfs-env-bootfs.service - type mark_hostonly >/dev/null 2>&1 && mark_hostonly @systemdunitdir@/zfs-env-bootfs.service + inst_simple "${moddir}/zfs-env-bootfs.service" "${systemdsystemunitdir}/zfs-env-bootfs.service" + systemctl -q --root "${initdir}" add-wants zfs-import.target zfs-env-bootfs.service - dracut_install systemd-ask-password - dracut_install systemd-tty-ask-password-agent - - mkdir -p "${initdir}/$systemdsystemunitdir/initrd.target.wants" - dracut_install @systemdunitdir@/zfs-import.target - if ! [ -L "${initdir}/$systemdsystemunitdir/initrd.target.wants"/zfs-import.target ]; then - ln -s ../zfs-import.target "${initdir}/$systemdsystemunitdir/initrd.target.wants"/zfs-import.target - type mark_hostonly >/dev/null 2>&1 && mark_hostonly @systemdunitdir@/zfs-import.target - fi + for _service in \ + "zfs-import-scan.service" \ + "zfs-import-cache.service"; do + inst_simple "${systemdsystemunitdir}/${_service}" + systemctl -q --root "${initdir}" add-wants zfs-import.target "${_service}" + done - for _service in zfs-snapshot-bootfs.service zfs-rollback-bootfs.service ; do - inst "${moddir}/$_service" "${systemdsystemunitdir}/$_service" - if ! [ -L "${initdir}/$systemdsystemunitdir/initrd.target.wants/$_service" ]; then - ln -s "../$_service" "${initdir}/$systemdsystemunitdir/initrd.target.wants/$_service" - fi + for _service in \ + "zfs-snapshot-bootfs.service" \ + "zfs-rollback-bootfs.service"; do + inst_simple "${moddir}/${_service}" "${systemdsystemunitdir}/${_service}" + systemctl -q --root "${initdir}" add-wants initrd.target "${_service}" done - # There isn't a pkg-config variable for this, - # and dracut doesn't automatically resolve anything this'd be next to - local systemdsystemenvironmentgeneratordir - systemdsystemenvironmentgeneratordir="$(pkg-config --variable=prefix systemd || echo "/usr")/lib/systemd/system-environment-generators" - mkdir -p "${initdir}/${systemdsystemenvironmentgeneratordir}" - inst "${moddir}"/import-opts-generator.sh "${systemdsystemenvironmentgeneratordir}"/zfs-import-opts.sh + inst_simple "${moddir}/import-opts-generator.sh" "${systemdutildir}/system-environment-generators/zfs-import-opts.sh" fi } diff --git a/contrib/dracut/90zfs/mount-zfs.sh.in b/contrib/dracut/90zfs/mount-zfs.sh.in index 68e3f0e0d60b..b0eb614a62b6 100755 --- a/contrib/dracut/90zfs/mount-zfs.sh.in +++ b/contrib/dracut/90zfs/mount-zfs.sh.in @@ -3,48 +3,73 @@ . /lib/dracut-zfs-lib.sh -ZFS_DATASET="" -ZFS_POOL="" - -case "${root}" in - zfs:*) ;; - *) return ;; -esac +decode_root_args || return 0 GENERATOR_FILE=/run/systemd/generator/sysroot.mount GENERATOR_EXTENSION=/run/systemd/generator/sysroot.mount.d/zfs-enhancement.conf -if [ -e "$GENERATOR_FILE" ] && [ -e "$GENERATOR_EXTENSION" ] ; then - # If the ZFS sysroot.mount flag exists, the initial RAM disk configured - # it to mount ZFS on root. In that case, we bail early. This flag - # file gets created by the zfs-generator program upon successful run. - info "ZFS: There is a sysroot.mount and zfs-generator has extended it." - info "ZFS: Delegating root mount to sysroot.mount." - # Let us tell the initrd to run on shutdown. - # We have a shutdown hook to run - # because we imported the pool. +if [ -e "$GENERATOR_FILE" ] && [ -e "$GENERATOR_EXTENSION" ]; then + # We're under systemd and dracut-zfs-generator ran to completion. + info "ZFS: Delegating root mount to sysroot.mount at al." + # We now prevent Dracut from running this thing again. - for zfsmounthook in "$hookdir"/mount/*zfs* ; do - if [ -f "$zfsmounthook" ] ; then - rm -f "$zfsmounthook" - fi - done + rm -f "$hookdir"/mount/*zfs* return fi + info "ZFS: No sysroot.mount exists or zfs-generator did not extend it." info "ZFS: Mounting root with the traditional mount-zfs.sh instead." +# ask_for_password tries prompt cmd +# +# Wraps around plymouth ask-for-password and adds fallback to tty password ask +# if plymouth is not present. +ask_for_password() { + tries="$1" + prompt="$2" + cmd="$3" + + { + flock -s 9 + + # Prompt for password with plymouth, if installed and running. + if plymouth --ping 2>/dev/null; then + plymouth ask-for-password \ + --prompt "$prompt" --number-of-tries="$tries" | \ + eval "$cmd" + ret=$? + else + i=1 + while [ "$i" -le "$tries" ]; do + printf "%s [%i/%i]:" "$prompt" "$i" "$tries" >&2 + eval "$cmd" && ret=0 && break + ret=$? + i=$((i+1)) + printf '\n' >&2 + done + unset i + fi + } 9>/.console_lock + + [ "$ret" -ne 0 ] && echo "Wrong password" >&2 + return "$ret" +} + + # Delay until all required block devices are present. modprobe zfs 2>/dev/null udevadm settle +ZFS_DATASET= +ZFS_POOL= + if [ "${root}" = "zfs:AUTO" ] ; then - if ! ZFS_DATASET="$(find_bootfs)" ; then + if ! ZFS_DATASET="$(zpool get -Ho value bootfs | grep -m1 -vFx -)"; then # shellcheck disable=SC2086 zpool import -N -a ${ZPOOL_IMPORT_OPTS} - if ! ZFS_DATASET="$(find_bootfs)" ; then + if ! ZFS_DATASET="$(zpool get -Ho value bootfs | grep -m1 -vFx -)"; then warn "ZFS: No bootfs attribute found in importable pools." - export_all -F + zpool export -aF rootok=0 return 1 @@ -53,34 +78,43 @@ if [ "${root}" = "zfs:AUTO" ] ; then info "ZFS: Using ${ZFS_DATASET} as root." fi -ZFS_DATASET="${ZFS_DATASET:-${root#zfs:}}" +ZFS_DATASET="${ZFS_DATASET:-${root}}" ZFS_POOL="${ZFS_DATASET%%/*}" -if import_pool "${ZFS_POOL}" ; then - # Load keys if we can or if we need to - if [ "$(zpool list -H -o feature@encryption "${ZFS_POOL}")" = 'active' ]; then - # if the root dataset has encryption enabled - ENCRYPTIONROOT="$(zfs get -H -o value encryptionroot "${ZFS_DATASET}")" - if ! [ "${ENCRYPTIONROOT}" = "-" ]; then - KEYSTATUS="$(zfs get -H -o value keystatus "${ENCRYPTIONROOT}")" - # if the key needs to be loaded - if [ "$KEYSTATUS" = "unavailable" ]; then - # decrypt them - ask_for_password \ - --tries 5 \ - --prompt "Encrypted ZFS password for ${ENCRYPTIONROOT}: " \ - --cmd "zfs load-key '${ENCRYPTIONROOT}'" - fi + +if ! zpool get -Ho value name "${ZFS_POOL}" > /dev/null 2>&1; then + info "ZFS: Importing pool ${ZFS_POOL}..." + # shellcheck disable=SC2086 + if ! zpool import -N ${ZPOOL_IMPORT_OPTS} "${ZFS_POOL}"; then + warn "ZFS: Unable to import pool ${ZFS_POOL}" + rootok=0 + return 1 + fi +fi + +# Load keys if we can or if we need to +# TODO: for_relevant_root_children like in zfs-load-key.sh.in +if [ "$(zpool get -Ho value feature@encryption "${ZFS_POOL}")" = 'active' ]; then + # if the root dataset has encryption enabled + ENCRYPTIONROOT="$(zfs get -Ho value encryptionroot "${ZFS_DATASET}")" + if ! [ "${ENCRYPTIONROOT}" = "-" ]; then + KEYSTATUS="$(zfs get -Ho value keystatus "${ENCRYPTIONROOT}")" + # if the key needs to be loaded + if [ "$KEYSTATUS" = "unavailable" ]; then + # decrypt them + ask_for_password \ + 5 \ + "Encrypted ZFS password for ${ENCRYPTIONROOT}: " \ + "zfs load-key '${ENCRYPTIONROOT}'" fi fi - # Let us tell the initrd to run on shutdown. - # We have a shutdown hook to run - # because we imported the pool. - info "ZFS: Mounting dataset ${ZFS_DATASET}..." - if mount_dataset "${ZFS_DATASET}" ; then - ROOTFS_MOUNTED=yes - return 0 - fi fi -rootok=0 +# Let us tell the initrd to run on shutdown. +# We have a shutdown hook to run +# because we imported the pool. +info "ZFS: Mounting dataset ${ZFS_DATASET}..." +if ! mount_dataset "${ZFS_DATASET}"; then + rootok=0 + return 1 +fi diff --git a/contrib/dracut/90zfs/parse-zfs.sh.in b/contrib/dracut/90zfs/parse-zfs.sh.in index 724c5e2c6dff..f7d1f1c5da9f 100755 --- a/contrib/dracut/90zfs/parse-zfs.sh.in +++ b/contrib/dracut/90zfs/parse-zfs.sh.in @@ -1,7 +1,8 @@ #!/bin/sh # shellcheck disable=SC2034,SC2154 -. /lib/dracut-lib.sh +# shellcheck source=zfs-lib.sh.in +. /lib/dracut-zfs-lib.sh # Let the command line override our host id. spl_hostid=$(getarg spl_hostid=) @@ -15,52 +16,20 @@ else warn "ZFS: Pools may not import correctly." fi -wait_for_zfs=0 -case "${root}" in - ""|zfs|zfs:) - # We'll take root unset, root=zfs, or root=zfs: - # No root set, so we want to read the bootfs attribute. We - # can't do that until udev settles so we'll set dummy values - # and hope for the best later on. - root="zfs:AUTO" - rootok=1 - wait_for_zfs=1 - - info "ZFS: Enabling autodetection of bootfs after udev settles." - ;; - - ZFS=*|zfs:*|FILESYSTEM=*) - # root is explicit ZFS root. Parse it now. We can handle - # a root=... param in any of the following formats: - # root=ZFS=rpool/ROOT - # root=zfs:rpool/ROOT - # root=zfs:FILESYSTEM=rpool/ROOT - # root=FILESYSTEM=rpool/ROOT - # root=ZFS=pool+with+space/ROOT+WITH+SPACE (translates to root=ZFS=pool with space/ROOT WITH SPACE) - - # Strip down to just the pool/fs - root="${root#zfs:}" - root="${root#FILESYSTEM=}" - root="zfs:${root#ZFS=}" - # switch + with spaces because kernel cmdline does not allow us to quote parameters - root=$(echo "$root" | tr '+' ' ') - rootok=1 - wait_for_zfs=1 - - info "ZFS: Set ${root} as bootfs." - ;; - - *) - info "ZFS: no ZFS-on-root" -esac - -# Make sure Dracut is happy that we have a root and will wait for ZFS -# modules to settle before mounting. -if [ "${wait_for_zfs}" -eq 1 ]; then - ln -s /dev/null /dev/root 2>/dev/null - initqueuedir="${hookdir}/initqueue/finished" - test -d "${initqueuedir}" || { - initqueuedir="${hookdir}/initqueue-finished" - } - echo '[ -e /dev/zfs ]' > "${initqueuedir}/zfs.sh" +if decode_root_args; then + if [ "$root" = "zfs:AUTO" ]; then + info "ZFS: Boot dataset autodetected from bootfs=." + else + info "ZFS: Boot dataset is ${root}." + fi + + rootok=1 + # Make sure Dracut is happy that we have a root and will wait for ZFS + # modules to settle before mounting. + if [ -n "${wait_for_zfs}" ]; then + ln -s null /dev/root + echo '[ -e /dev/zfs ]' > "${hookdir}/initqueue/finished/zfs.sh" + fi +else + info "ZFS: no ZFS-on-root." fi diff --git a/contrib/dracut/90zfs/zfs-env-bootfs.service.in b/contrib/dracut/90zfs/zfs-env-bootfs.service.in index e143cb5ec1ed..34c88037cac2 100644 --- a/contrib/dracut/90zfs/zfs-env-bootfs.service.in +++ b/contrib/dracut/90zfs/zfs-env-bootfs.service.in @@ -8,7 +8,7 @@ Before=zfs-import.target [Service] Type=oneshot -ExecStart=/bin/sh -c "exec systemctl set-environment BOOTFS=$(@sbindir@/zpool list -H -o bootfs | grep -m1 -v '^-$')" +ExecStart=/bin/sh -c "exec systemctl set-environment BOOTFS=$(@sbindir@/zpool list -H -o bootfs | grep -m1 -vFx -)" [Install] WantedBy=zfs-import.target diff --git a/contrib/dracut/90zfs/zfs-generator.sh.in b/contrib/dracut/90zfs/zfs-generator.sh.in index e50b9530c4f0..56f7ca9785ba 100755 --- a/contrib/dracut/90zfs/zfs-generator.sh.in +++ b/contrib/dracut/90zfs/zfs-generator.sh.in @@ -1,5 +1,5 @@ #!/bin/sh -# shellcheck disable=SC2016,SC1004 +# shellcheck disable=SC2016,SC1004,SC2154 grep -wq debug /proc/cmdline && debug=1 [ -n "$debug" ] && echo "zfs-generator: starting" >> /dev/kmsg @@ -10,37 +10,17 @@ GENERATOR_DIR="$1" exit 1 } -[ -f /lib/dracut-lib.sh ] && dracutlib=/lib/dracut-lib.sh -[ -f /usr/lib/dracut/modules.d/99base/dracut-lib.sh ] && dracutlib=/usr/lib/dracut/modules.d/99base/dracut-lib.sh -command -v getarg >/dev/null 2>&1 || { - [ -n "$debug" ] && echo "zfs-generator: loading Dracut library from $dracutlib" >> /dev/kmsg - . "$dracutlib" -} - +# shellcheck source=zfs-lib.sh.in . /lib/dracut-zfs-lib.sh +decode_root_args || exit 0 -[ -z "$root" ] && root=$(getarg root=) -[ -z "$rootfstype" ] && rootfstype=$(getarg rootfstype=) -[ -z "$rootflags" ] && rootflags=$(getarg rootflags=) - -# If root is not ZFS= or zfs: or rootfstype is not zfs -# then we are not supposed to handle it. -[ "${root##zfs:}" = "${root}" ] && - [ "${root##ZFS=}" = "${root}" ] && - [ "$rootfstype" != "zfs" ] && - exit 0 - +[ -z "${rootflags}" ] && rootflags=$(getarg rootflags=) case ",${rootflags}," in *,zfsutil,*) ;; ,,) rootflags=zfsutil ;; *) rootflags="zfsutil,${rootflags}" ;; esac -if [ "${root}" != "zfs:AUTO" ]; then - root="${root##zfs:}" - root="${root##ZFS=}" -fi - [ -n "$debug" ] && echo "zfs-generator: writing extension for sysroot.mount to $GENERATOR_DIR/sysroot.mount.d/zfs-enhancement.conf" >> /dev/kmsg @@ -89,7 +69,7 @@ else _zfs_generator_cb() { dset="${1}" mpnt="${2}" - unit="sysroot$(echo "$mpnt" | tr '/' '-').mount" + unit="$(systemd-escape --suffix=mount -p "/sysroot${mpnt}")" { echo "[Unit]" diff --git a/contrib/dracut/90zfs/zfs-lib.sh.in b/contrib/dracut/90zfs/zfs-lib.sh.in index d7c3e96c1213..3a43e514d6f9 100755 --- a/contrib/dracut/90zfs/zfs-lib.sh.in +++ b/contrib/dracut/90zfs/zfs-lib.sh.in @@ -1,74 +1,16 @@ #!/bin/sh +# shellcheck disable=SC2034 -command -v getarg >/dev/null || . /lib/dracut-lib.sh -command -v getargbool >/dev/null || { - # Compatibility with older Dracut versions. - # With apologies to the Dracut developers. - getargbool() { - _default="$1"; shift - ! _b=$(getarg "$@") && [ -z "$_b" ] && _b="$_default" - if [ -n "$_b" ]; then - [ "$_b" = "0" ] && return 1 - [ "$_b" = "no" ] && return 1 - [ "$_b" = "off" ] && return 1 - fi - return 0 - } -} +command -v getarg >/dev/null || . /lib/dracut-lib.sh || . /usr/lib/dracut/modules.d/99base/dracut-lib.sh -OLDIFS="${IFS}" -NEWLINE=" -" TAB=" " -ZPOOL_IMPORT_OPTS="" -if getargbool 0 zfs_force -y zfs.force -y zfsforce ; then +ZPOOL_IMPORT_OPTS= +if getargbool 0 zfs_force -y zfs.force -y zfsforce; then warn "ZFS: Will force-import pools if necessary." - ZPOOL_IMPORT_OPTS="${ZPOOL_IMPORT_OPTS} -f" + ZPOOL_IMPORT_OPTS=-f fi -# find_bootfs -# returns the first dataset with the bootfs attribute. -find_bootfs() { - IFS="${NEWLINE}" - for dataset in $(zpool list -H -o bootfs); do - case "${dataset}" in - "" | "-") - continue - ;; - "no pools available") - IFS="${OLDIFS}" - return 1 - ;; - *) - IFS="${OLDIFS}" - echo "${dataset}" - return 0 - ;; - esac - done - - IFS="${OLDIFS}" - return 1 -} - -# import_pool POOL -# imports the given zfs pool if it isn't imported already. -import_pool() { - pool="${1}" - - if ! zpool list -H "${pool}" > /dev/null 2>&1; then - info "ZFS: Importing pool ${pool}..." - # shellcheck disable=SC2086 - if ! zpool import -N ${ZPOOL_IMPORT_OPTS} "${pool}" ; then - warn "ZFS: Unable to import pool ${pool}" - return 1 - fi - fi - - return 0 -} - _mount_dataset_cb() { # shellcheck disable=SC2154 mount -o zfsutil -t zfs "${1}" "${NEWROOT}${2}" @@ -122,88 +64,57 @@ for_relevant_root_children() { ) } -# export_all OPTS -# exports all imported zfs pools. -export_all() { - ret=0 - - IFS="${NEWLINE}" - for pool in $(zpool list -H -o name) ; do - if zpool list -H "${pool}" > /dev/null 2>&1; then - zpool export "${pool}" "$@" || ret=$? - fi - done - IFS="${OLDIFS}" - - return "${ret}" -} - -# ask_for_password +# Parse root=, rootfstype=, return them decoded and normalised to zfs:AUTO for auto, plain dset for explicit +# +# True if ZFS-on-root, false if we shouldn't +# +# Supported values: +# root= +# root=zfs +# root=zfs: +# root=zfs:AUTO +# +# root=ZFS=data/set +# root=zfs:data/set +# root=zfs:ZFS=data/set (as a side-effect; allowed but undocumented) # -# Wraps around plymouth ask-for-password and adds fallback to tty password ask -# if plymouth is not present. +# rootfstype=zfs AND root=data/set <=> root=data/set +# rootfstype=zfs AND root= <=> root=zfs:AUTO # -# --cmd command -# Command to execute. Required. -# --prompt prompt -# Password prompt. Note that function already adds ':' at the end. -# Recommended. -# --tries n -# How many times repeat command on its failure. Default is 3. -# --ply-[cmd|prompt|tries] -# Command/prompt/tries specific for plymouth password ask only. -# --tty-[cmd|prompt|tries] -# Command/prompt/tries specific for tty password ask only. -# --tty-echo-off -# Turn off input echo before tty command is executed and turn on after. -# It's useful when password is read from stdin. -ask_for_password() { - ply_tries=3 - tty_tries=3 - while [ "$#" -gt 0 ]; do - case "$1" in - --cmd) ply_cmd="$2"; tty_cmd="$2"; shift;; - --ply-cmd) ply_cmd="$2"; shift;; - --tty-cmd) tty_cmd="$2"; shift;; - --prompt) ply_prompt="$2"; tty_prompt="$2"; shift;; - --ply-prompt) ply_prompt="$2"; shift;; - --tty-prompt) tty_prompt="$2"; shift;; - --tries) ply_tries="$2"; tty_tries="$2"; shift;; - --ply-tries) ply_tries="$2"; shift;; - --tty-tries) tty_tries="$2"; shift;; - --tty-echo-off) tty_echo_off=yes;; - *) echo "ask_for_password(): wrong opt '$1'" >&2;; +# '+'es in explicit dataset decoded to ' 's. +decode_root_args() { + if [ -n "$rootfstype" ]; then + [ "$rootfstype" = zfs ] + return + fi + + xroot=$(getarg root=) + rootfstype=$(getarg rootfstype=) + + # shellcheck disable=SC2249 + case "$xroot" in + ""|zfs|zfs:|zfs:AUTO) + root=zfs:AUTO + rootfstype=zfs + return 0 + ;; + + ZFS=*|zfs:*) + root="${xroot#zfs:}" + root="${root#ZFS=}" + root=$(echo "$root" | tr '+' ' ') + rootfstype=zfs + return 0 + ;; + esac + + if [ "$rootfstype" = "zfs" ]; then + case "$xroot" in + "") root=zfs:AUTO ;; + *) root=$(echo "$xroot" | tr '+' ' ') ;; esac - shift - done - - { flock -s 9; - # Prompt for password with plymouth, if installed and running. - if plymouth --ping 2>/dev/null; then - plymouth ask-for-password \ - --prompt "$ply_prompt" --number-of-tries="$ply_tries" | \ - eval "$ply_cmd" - ret=$? - else - if [ "$tty_echo_off" = yes ]; then - stty_orig="$(stty -g)" - stty -echo - fi - - i=1 - while [ "$i" -le "$tty_tries" ]; do - [ -n "$tty_prompt" ] && \ - printf "%s [%i/%i]:" "$tty_prompt" "$i" "$tty_tries" >&2 - eval "$tty_cmd" && ret=0 && break - ret=$? - i=$((i+1)) - [ -n "$tty_prompt" ] && printf '\n' >&2 - done - unset i - [ "$tty_echo_off" = yes ] && stty "$stty_orig" - fi - } 9>/.console_lock + return 0 + fi - [ "$ret" -ne 0 ] && echo "Wrong password" >&2 - return "$ret" + return 1 } diff --git a/contrib/dracut/90zfs/zfs-load-key.sh.in b/contrib/dracut/90zfs/zfs-load-key.sh.in index c974b3d9ec4c..d916f43b4e95 100755 --- a/contrib/dracut/90zfs/zfs-load-key.sh.in +++ b/contrib/dracut/90zfs/zfs-load-key.sh.in @@ -4,70 +4,61 @@ # only run this on systemd systems, we handle the decrypt in mount-zfs.sh in the mount hook otherwise [ -e /bin/systemctl ] || [ -e /usr/bin/systemctl ] || return 0 -# This script only gets executed on systemd systems, see mount-zfs.sh for non-systemd systems +# shellcheck source=zfs-lib.sh.in +. /lib/dracut-zfs-lib.sh -# import the libs now that we know the pool imported -[ -f /lib/dracut-lib.sh ] && dracutlib=/lib/dracut-lib.sh -[ -f /usr/lib/dracut/modules.d/99base/dracut-lib.sh ] && dracutlib=/usr/lib/dracut/modules.d/99base/dracut-lib.sh -# shellcheck source=./lib-zfs.sh.in -. "$dracutlib" - -# load the kernel command line vars -[ -z "$root" ] && root="$(getarg root=)" -# If root is not ZFS= or zfs: or rootfstype is not zfs then we are not supposed to handle it. -[ "${root##zfs:}" = "${root}" ] && [ "${root##ZFS=}" = "${root}" ] && [ "$rootfstype" != "zfs" ] && exit 0 +decode_root_args || return 0 # There is a race between the zpool import and the pre-mount hooks, so we wait for a pool to be imported -while [ "$(zpool list -H)" = "" ]; do - systemctl is-failed --quiet zfs-import-cache.service zfs-import-scan.service && exit 1 +while ! systemctl is-active --quiet zfs-import.target; do + systemctl is-failed --quiet zfs-import-cache.service zfs-import-scan.service && return 1 sleep 0.1s done -# run this after import as zfs-import-cache/scan service is confirmed good -# we do not overwrite the ${root} variable, but create a new one, BOOTFS, to hold the dataset -if [ "${root}" = "zfs:AUTO" ] ; then - BOOTFS="$(zpool list -H -o bootfs | awk '$1 != "-" {print; exit}')" -else - BOOTFS="${root##zfs:}" - BOOTFS="${BOOTFS##ZFS=}" +BOOTFS="$root" +if [ "$BOOTFS" = "zfs:AUTO" ]; then + BOOTFS="$(zpool get -Ho value bootfs | grep -m1 -vFx -)" fi -# if pool encryption is active and the zfs command understands '-o encryption' -if [ "$(zpool list -H -o feature@encryption "${BOOTFS%%/*}")" = 'active' ]; then - # if the root dataset has encryption enabled - ENCRYPTIONROOT="$(zfs get -H -o value encryptionroot "${BOOTFS}")" - if ! [ "${ENCRYPTIONROOT}" = "-" ]; then - KEYSTATUS="$(zfs get -H -o value keystatus "${ENCRYPTIONROOT}")" - # continue only if the key needs to be loaded - [ "$KEYSTATUS" = "unavailable" ] || exit 0 +[ "$(zpool get -Ho value feature@encryption "${BOOTFS%%/*}")" = 'active' ] || return 0 + +_load_key_cb() { + dataset="$1" + + ENCRYPTIONROOT="$(zfs get -Ho value encryptionroot "${dataset}")" + [ "${ENCRYPTIONROOT}" = "-" ] && return 0 - KEYLOCATION="$(zfs get -H -o value keylocation "${ENCRYPTIONROOT}")" - case "${KEYLOCATION%%://*}" in - prompt) - for _ in 1 2 3; do - systemd-ask-password --no-tty "Encrypted ZFS password for ${BOOTFS}" | zfs load-key "${ENCRYPTIONROOT}" && break + [ "$(zfs get -Ho value keystatus "${ENCRYPTIONROOT}")" = "unavailable" ] || return 0 + + KEYLOCATION="$(zfs get -Ho value keylocation "${ENCRYPTIONROOT}")" + case "${KEYLOCATION%%://*}" in + prompt) + for _ in 1 2 3; do + systemd-ask-password --no-tty "Encrypted ZFS password for ${dataset}" | zfs load-key "${ENCRYPTIONROOT}" && break + done + ;; + http*) + systemctl start network-online.target + zfs load-key "${ENCRYPTIONROOT}" + ;; + file) + KEYFILE="${KEYLOCATION#file://}" + [ -r "${KEYFILE}" ] || udevadm settle + [ -r "${KEYFILE}" ] || { + info "ZFS: Waiting for key ${KEYFILE} for ${ENCRYPTIONROOT}..." + for _ in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20; do + sleep 0.5s + [ -r "${KEYFILE}" ] && break done - ;; - http*) - systemctl start network-online.target - zfs load-key "${ENCRYPTIONROOT}" - ;; - file) - KEYFILE="${KEYLOCATION#file://}" - [ -r "${KEYFILE}" ] || udevadm settle - [ -r "${KEYFILE}" ] || { - info "Waiting for key ${KEYFILE} for ${ENCRYPTIONROOT}..." - for _ in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20; do - sleep 0.5s - [ -r "${KEYFILE}" ] && break - done - } - [ -r "${KEYFILE}" ] || warn "Key ${KEYFILE} for ${ENCRYPTIONROOT} hasn't appeared. Trying anyway." - zfs load-key "${ENCRYPTIONROOT}" - ;; - *) - zfs load-key "${ENCRYPTIONROOT}" - ;; - esac - fi -fi + } + [ -r "${KEYFILE}" ] || warn "ZFS: Key ${KEYFILE} for ${ENCRYPTIONROOT} hasn't appeared. Trying anyway." + zfs load-key "${ENCRYPTIONROOT}" + ;; + *) + zfs load-key "${ENCRYPTIONROOT}" + ;; + esac +} + +_load_key_cb "$BOOTFS" +for_relevant_root_children "$BOOTFS" _load_key_cb diff --git a/contrib/dracut/90zfs/zfs-needshutdown.sh.in b/contrib/dracut/90zfs/zfs-needshutdown.sh.in index dd6de30c2704..7fb825bc95a2 100755 --- a/contrib/dracut/90zfs/zfs-needshutdown.sh.in +++ b/contrib/dracut/90zfs/zfs-needshutdown.sh.in @@ -2,7 +2,7 @@ command -v getarg >/dev/null 2>&1 || . /lib/dracut-lib.sh -if zpool list 2>&1 | grep -q 'no pools available' ; then +if [ -z "$(zpool get -Ho value name)" ]; then info "ZFS: No active pools, no need to export anything." else info "ZFS: There is an active pool, will export it." diff --git a/contrib/dracut/90zfs/zfs-rollback-bootfs.service.in b/contrib/dracut/90zfs/zfs-rollback-bootfs.service.in index 477b64f2b750..a29cf3a3dd81 100644 --- a/contrib/dracut/90zfs/zfs-rollback-bootfs.service.in +++ b/contrib/dracut/90zfs/zfs-rollback-bootfs.service.in @@ -1,14 +1,12 @@ [Unit] Description=Rollback bootfs just before it is mounted Requisite=zfs-import.target -After=zfs-import.target zfs-snapshot-bootfs.service +After=zfs-import.target dracut-pre-mount.service zfs-snapshot-bootfs.service Before=dracut-mount.service DefaultDependencies=no ConditionKernelCommandLine=bootfs.rollback [Service] -# ${BOOTFS} should have been set by zfs-env-bootfs.service Type=oneshot -ExecStartPre=/bin/test -n ${BOOTFS} -ExecStart=/bin/sh -c '. /lib/dracut-lib.sh; SNAPNAME="$(getarg bootfs.rollback)"; exec @sbindir@/zfs rollback -Rf "${BOOTFS}@${SNAPNAME:-%v}"' +ExecStart=/bin/sh -c '. /lib/dracut-zfs-lib.sh; decode_root_args || exit; [ "$root" = "zfs:AUTO" ] && root="$BOOTFS"; SNAPNAME="$(getarg bootfs.rollback)"; exec @sbindir@/zfs rollback -Rf "$root@${SNAPNAME:-%v}"' RemainAfterExit=yes diff --git a/contrib/dracut/90zfs/zfs-snapshot-bootfs.service.in b/contrib/dracut/90zfs/zfs-snapshot-bootfs.service.in index 8eae04adfb99..befd163b6536 100644 --- a/contrib/dracut/90zfs/zfs-snapshot-bootfs.service.in +++ b/contrib/dracut/90zfs/zfs-snapshot-bootfs.service.in @@ -1,14 +1,12 @@ [Unit] Description=Snapshot bootfs just before it is mounted Requisite=zfs-import.target -After=zfs-import.target +After=zfs-import.target dracut-pre-mount.service Before=dracut-mount.service DefaultDependencies=no ConditionKernelCommandLine=bootfs.snapshot [Service] -# ${BOOTFS} should have been set by zfs-env-bootfs.service Type=oneshot -ExecStartPre=/bin/test -n ${BOOTFS} -ExecStart=-/bin/sh -c '. /lib/dracut-lib.sh; SNAPNAME="$(getarg bootfs.snapshot)"; exec @sbindir@/zfs snapshot "${BOOTFS}@${SNAPNAME:-%v}"' +ExecStart=/bin/sh -c '. /lib/dracut-zfs-lib.sh; decode_root_args || exit; [ "$root" = "zfs:AUTO" ] && root="$BOOTFS"; SNAPNAME="$(getarg bootfs.snapshot)"; exec @sbindir@/zfs snapshot "$root@${SNAPNAME:-%v}"' RemainAfterExit=yes diff --git a/contrib/dracut/Makefile.am b/contrib/dracut/Makefile.am index 09805277ffb0..73ca52b66316 100644 --- a/contrib/dracut/Makefile.am +++ b/contrib/dracut/Makefile.am @@ -1,6 +1,27 @@ -include $(top_srcdir)/config/Shellcheck.am +pkgdracut_02dir = $(dracutdir)/modules.d/02zfsexpandknowledge +pkgdracut_02_SCRIPTS = \ + %D%/02zfsexpandknowledge/module-setup.sh -SUBDIRS = 02zfsexpandknowledge 90zfs -SHELLCHECKDIRS = $(SUBDIRS) +pkgdracut_90dir = $(dracutdir)/modules.d/90zfs +pkgdracut_90_SCRIPTS = \ + %D%/90zfs/export-zfs.sh \ + %D%/90zfs/import-opts-generator.sh \ + %D%/90zfs/module-setup.sh \ + %D%/90zfs/mount-zfs.sh \ + %D%/90zfs/parse-zfs.sh \ + %D%/90zfs/zfs-generator.sh \ + %D%/90zfs/zfs-lib.sh \ + %D%/90zfs/zfs-load-key.sh \ + %D%/90zfs/zfs-needshutdown.sh -EXTRA_DIST = README.md +pkgdracut_90_DATA = \ + %D%/90zfs/zfs-env-bootfs.service \ + %D%/90zfs/zfs-rollback-bootfs.service \ + %D%/90zfs/zfs-snapshot-bootfs.service + +SUBSTFILES += $(pkgdracut_02_SCRIPTS) $(pkgdracut_90_SCRIPTS) $(pkgdracut_90_DATA) +SHELLCHECKSCRIPTS += $(pkgdracut_02_SCRIPTS) $(pkgdracut_90_SCRIPTS) +# Provided by /bin/sleep, and, again, every implementation of that supports this +$(call SHELLCHECK_OPTS,$(pkgdracut_90_SCRIPTS)): CHECKBASHISMS_IGNORE = -e 'sleep only takes one integer' -e 'sleep 0.' + +dist_noinst_DATA += %D%/README.md diff --git a/contrib/dracut/README.md b/contrib/dracut/README.md index 906648db6314..42d86b34e440 100644 --- a/contrib/dracut/README.md +++ b/contrib/dracut/README.md @@ -15,18 +15,22 @@ Encrypted datasets have keys loaded automatically or prompted for. If the root dataset contains children with `mountpoint=`s of `/etc`, `/bin`, `/lib*`, or `/usr`, they're mounted too. +For complete documentation, see `dracut.zfs(7)`. + ## cmdline -1. `root=` | Root dataset is… | Pools imported | - -------------------|----------------------------------------------------------|----------------| - *(empty)* | the first `bootfs=` after `zpool import -aN` | all | - `zfs:AUTO` | *(as above, but overriding other autoselection methods)* | all | - `ZFS=pool/dataset` | `pool/dataset` | `pool` | - `zfs:pool/dataset` | *(as above)* | `pool` | +1. `root=` | Root dataset is… | + ---------------------------|----------------------------------------------------------| + *(empty)* | the first `bootfs=` after `zpool import -aN` | + `zfs:AUTO`, `zfs:`, `zfs` | *(as above, but overriding other autoselection methods)* | + `ZFS=pool/dataset` | `pool/dataset` | + `zfs:pool/dataset` | *(as above)* | All `+`es are replaced with spaces (i.e. to boot from `root pool/data set`, pass `root=zfs:root+pool/data+set`). The dataset can be at any depth, including being the pool's root dataset (i.e. `root=zfs:pool`). + `rootfstype=zfs` is equivalent to `root=zfs:AUTO`, `rootfstype=zfs root=pool/dataset` is equivalent to `root=zfs:pool/dataset`. + 2. `spl_hostid`: passed to `zgenhostid -f`, useful to override the `/etc/hostid` file baked into the initrd. 3. `bootfs.snapshot`, `bootfs.snapshot=snapshot-name`: enables `zfs-snapshot-bootfs.service`, @@ -34,7 +38,7 @@ If the root dataset contains children with `mountpoint=`s of `/etc`, `/bin`, `/l after pool import but before the rootfs is mounted. Failure to create the snapshot is noted, but booting continues. -4. `bootfs.rollback`, `bootfs.rollback=snapshot-name`: enables `zfs-snapshot-bootfs.service`, +4. `bootfs.rollback`, `bootfs.rollback=snapshot-name`: enables `zfs-rollback-bootfs.service`, which `-Rf` rolls back to `$root_dataset@$(uname -r)` (or, in the second form, `$root_dataset@snapshot-name`) after pool import but before the rootfs is mounted. Failure to roll back will fall down to the rescue shell. diff --git a/contrib/initramfs/Makefile.am b/contrib/initramfs/Makefile.am index 57e8f5c3a485..a583341ea2b1 100644 --- a/contrib/initramfs/Makefile.am +++ b/contrib/initramfs/Makefile.am @@ -1,11 +1,39 @@ -include $(top_srcdir)/config/Shellcheck.am +i_tdir = /usr/share/initramfs-tools +dist_i_t_SCRIPTS = \ + %D%/zfsunlock -initrddir = /usr/share/initramfs-tools -dist_initrd_SCRIPTS = \ - zfsunlock +i_t_confhooks_ddir = $(i_tdir)/conf-hooks.d +dist_i_t_confhooks_d_DATA = \ + %D%/conf-hooks.d/zfs -SUBDIRS = conf.d conf-hooks.d hooks scripts -SHELLCHECKDIRS = hooks scripts -EXTRA_DIST = README.md +i_t_conf_ddir = $(i_tdir)/conf.d +dist_i_t_conf_d_DATA = \ + %D%/conf.d/zfs + + +i_t_hooksdir = $(i_tdir)/hooks +i_t_hooks_SCRIPTS = \ + %D%/hooks/zfs \ + %D%/hooks/zfsunlock + +SUBSTFILES += $(i_t_hooks_SCRIPTS) + + +i_t_scriptsdir = $(i_tdir)/scripts +dist_i_t_scripts_SCRIPTS = \ + %D%/scripts/zfs + + +i_t_scripts_localtopdir = $(i_t_scriptsdir)/local-top +dist_i_t_scripts_localtop_SCRIPTS = \ + %D%/scripts/local-top/zfs + + +i_t_check_scripts = $(dist_i_t_SCRIPTS) $(i_t_hooks_SCRIPTS) $(dist_i_t_scripts_SCRIPTS) $(dist_i_t_scripts_localtop_SCRIPTS) +SHELLCHECKSCRIPTS += $(i_t_check_scripts) +$(call SHELLCHECK_OPTS,$(i_t_check_scripts)): SHELLCHECK_SHELL = sh + + +dist_noinst_DATA += %D%/README.md diff --git a/contrib/initramfs/conf-hooks.d/Makefile.am b/contrib/initramfs/conf-hooks.d/Makefile.am deleted file mode 100644 index f84ba5cc7e37..000000000000 --- a/contrib/initramfs/conf-hooks.d/Makefile.am +++ /dev/null @@ -1,4 +0,0 @@ -confhooksddir = /usr/share/initramfs-tools/conf-hooks.d - -dist_confhooksd_DATA = \ - zfs diff --git a/contrib/initramfs/conf.d/Makefile.am b/contrib/initramfs/conf.d/Makefile.am deleted file mode 100644 index 5ef27e0aa1ce..000000000000 --- a/contrib/initramfs/conf.d/Makefile.am +++ /dev/null @@ -1,4 +0,0 @@ -confddir = /usr/share/initramfs-tools/conf.d - -dist_confd_DATA = \ - zfs diff --git a/contrib/initramfs/hooks/Makefile.am b/contrib/initramfs/hooks/Makefile.am deleted file mode 100644 index 0cd1aafcd359..000000000000 --- a/contrib/initramfs/hooks/Makefile.am +++ /dev/null @@ -1,10 +0,0 @@ -include $(top_srcdir)/config/Substfiles.am -include $(top_srcdir)/config/Shellcheck.am - -hooksdir = /usr/share/initramfs-tools/hooks - -hooks_SCRIPTS = \ - zfs \ - zfsunlock - -SUBSTFILES += $(hooks_SCRIPTS) diff --git a/contrib/initramfs/scripts/Makefile.am b/contrib/initramfs/scripts/Makefile.am deleted file mode 100644 index 444a5f374bfe..000000000000 --- a/contrib/initramfs/scripts/Makefile.am +++ /dev/null @@ -1,11 +0,0 @@ -include $(top_srcdir)/config/Shellcheck.am - -scriptsdir = /usr/share/initramfs-tools/scripts - -dist_scripts_SCRIPTS = \ - zfs - -SUBDIRS = local-top - -SHELLCHECKDIRS = $(SUBDIRS) -SHELLCHECK_SHELL = sh diff --git a/contrib/initramfs/scripts/local-top/Makefile.am b/contrib/initramfs/scripts/local-top/Makefile.am deleted file mode 100644 index f3dc23129f09..000000000000 --- a/contrib/initramfs/scripts/local-top/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -include $(top_srcdir)/config/Shellcheck.am - -localtopdir = /usr/share/initramfs-tools/scripts/local-top - -dist_localtop_SCRIPTS = \ - zfs - diff --git a/contrib/initramfs/scripts/zfs b/contrib/initramfs/scripts/zfs index 85b00f9da86d..1c8c496a7ff4 100644 --- a/contrib/initramfs/scripts/zfs +++ b/contrib/initramfs/scripts/zfs @@ -328,7 +328,7 @@ mount_fs() # Need the _original_ datasets mountpoint! mountpoint=$(get_fs_value "$fs" mountpoint) - ZFS_CMD="mount -o zfsutil -t zfs" + ZFS_CMD="mount.zfs -o zfsutil" if [ "$mountpoint" = "legacy" ] || [ "$mountpoint" = "none" ]; then # Can't use the mountpoint property. Might be one of our # clones. Check the 'org.zol:mountpoint' property set in @@ -351,7 +351,7 @@ mount_fs() # If it's not a legacy filesystem, it can only be a # native one... if [ "$mountpoint" = "legacy" ]; then - ZFS_CMD="mount -t zfs" + ZFS_CMD="mount.zfs" fi fi @@ -919,7 +919,7 @@ mountroot() echo " not specified on the kernel command line." echo "" echo "Manually mount the root filesystem on $rootmnt and then exit." - echo "Hint: Try: mount -o zfsutil -t zfs ${ZFS_RPOOL-rpool}/ROOT/system $rootmnt" + echo "Hint: Try: mount.zfs -o zfsutil ${ZFS_RPOOL-rpool}/ROOT/system $rootmnt" shell fi diff --git a/contrib/pam_zfs_key/Makefile.am b/contrib/pam_zfs_key/Makefile.am index f0f2550afccb..aaa608b7da2b 100644 --- a/contrib/pam_zfs_key/Makefile.am +++ b/contrib/pam_zfs_key/Makefile.am @@ -1,19 +1,18 @@ -include $(top_srcdir)/config/Rules.am +%C%_pam_zfs_key_la_CFLAGS = $(AM_CFLAGS) +%C%_pam_zfs_key_la_CFLAGS += $(LIBCRYPTO_CFLAGS) -AM_CFLAGS += $(LIBCRYPTO_CFLAGS) +pammodule_LTLIBRARIES = %D%/pam_zfs_key.la -pammodule_LTLIBRARIES=pam_zfs_key.la +%C%_pam_zfs_key_la_SOURCES = %D%/pam_zfs_key.c -pam_zfs_key_la_SOURCES = pam_zfs_key.c +%C%_pam_zfs_key_la_LIBADD = \ + libnvpair.la \ + libuutil.la \ + libzfs.la \ + libzfs_core.la -pam_zfs_key_la_LIBADD = \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la \ - $(abs_top_builddir)/lib/libuutil/libuutil.la \ - $(abs_top_builddir)/lib/libzfs/libzfs.la \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la +%C%_pam_zfs_key_la_LIBADD += -lpam $(LIBCRYPTO_LIBS) -pam_zfs_key_la_LDFLAGS = -version-info 1:0:0 -avoid-version -module -shared +%C%_pam_zfs_key_la_LDFLAGS = -version-info 1:0:0 -avoid-version -module -shared -pam_zfs_key_la_LIBADD += -lpam $(LIBCRYPTO_LIBS) - -dist_pamconfigs_DATA = zfs_key +dist_pamconfigs_DATA = %D%/zfs_key diff --git a/contrib/pam_zfs_key/pam_zfs_key.c b/contrib/pam_zfs_key/pam_zfs_key.c index 9702189f6702..6f95d468074f 100644 --- a/contrib/pam_zfs_key/pam_zfs_key.c +++ b/contrib/pam_zfs_key/pam_zfs_key.c @@ -47,6 +47,7 @@ static void pam_syslog(pam_handle_t *pamh, int loglevel, const char *fmt, ...) { + (void) pamh; va_list args; va_start(args, fmt); vsyslog(loglevel, fmt, args); @@ -151,7 +152,7 @@ alloc_pw_string(const char *source) static void pw_free(pw_password_t *pw) { - bzero(pw->value, pw->len); + memset(pw->value, 0, pw->len); if (try_lock(munlock, pw->value, pw->len) == 0) { (void) munmap(pw->value, pw->len); } diff --git a/contrib/pyzfs/Makefile.am b/contrib/pyzfs/Makefile.am index fa1bb32ce2eb..06d9a09d7f1e 100644 --- a/contrib/pyzfs/Makefile.am +++ b/contrib/pyzfs/Makefile.am @@ -1,10 +1,11 @@ -EXTRA_DIST = libzfs_core setup.py.in README LICENSE docs +dist_noinst_DATA += %D%/libzfs_core %D%/README %D%/LICENSE %D%/docs +SUBSTFILES += %D%/setup.py if PYZFS_ENABLED -all: - -all-local: - $(PYTHON) setup.py build +ALL_LOCAL += pyzfs-all-local +pyzfs_V_1 = -v +pyzfs-all-local: %D%/setup.py + cd %D% && $(PYTHON) setup.py -q $(pyzfs_V_$(V)) egg_info -e . build # # On Debian (Ubuntu, and other downstream distros) the install location of @@ -24,17 +25,14 @@ all-local: # files are later created by manually loading the Python modules. # install-exec-local: - $(PYTHON) $(builddir)/setup.py install \ - --prefix $(prefix) \ - --root $(DESTDIR)/ \ - --install-lib $(pythonsitedir) \ - --single-version-externally-managed \ - --verbose - -clean: clean-local - rm -rf build/ pyzfs.egg-info/ - -clean-local: + cd %D% && $(PYTHON) setup.py egg_info -e . install \ + --prefix $(prefix) \ + --root $(DESTDIR)/ \ + --install-lib $(pythonsitedir) \ + --single-version-externally-managed \ + --verbose -check-local: all +CLEAN_LOCAL += pyzfs-clean-local +pyzfs-clean-local: + -$(RM) -r %D%/build/ %D%/pyzfs.egg-info/ endif diff --git a/contrib/pyzfs/libzfs_core/__init__.py b/contrib/pyzfs/libzfs_core/__init__.py index 25ea3e495b02..a80f94b524ec 100644 --- a/contrib/pyzfs/libzfs_core/__init__.py +++ b/contrib/pyzfs/libzfs_core/__init__.py @@ -72,6 +72,7 @@ lzc_receive_resumable, lzc_receive_with_cmdprops, lzc_receive_with_header, + lzc_receive_with_heal, lzc_release, lzc_reopen, lzc_rollback, @@ -127,6 +128,7 @@ 'lzc_receive_resumable', 'lzc_receive_with_cmdprops', 'lzc_receive_with_header', + 'lzc_receive_with_heal', 'lzc_release', 'lzc_reopen', 'lzc_rollback', diff --git a/contrib/pyzfs/libzfs_core/_constants.py b/contrib/pyzfs/libzfs_core/_constants.py index 3273652f758a..7ee2ef87df3e 100644 --- a/contrib/pyzfs/libzfs_core/_constants.py +++ b/contrib/pyzfs/libzfs_core/_constants.py @@ -100,6 +100,7 @@ def enum(*sequential, **named): 'ZFS_ERR_REBUILD_IN_PROGRESS', 'ZFS_ERR_BADPROP', 'ZFS_ERR_VDEV_NOTSUP', + 'ZFS_ERR_NOT_USER_NAMESPACE', ], {} ) diff --git a/contrib/pyzfs/libzfs_core/_error_translation.py b/contrib/pyzfs/libzfs_core/_error_translation.py index f494461f63b2..26676db398c5 100644 --- a/contrib/pyzfs/libzfs_core/_error_translation.py +++ b/contrib/pyzfs/libzfs_core/_error_translation.py @@ -469,6 +469,8 @@ def _map(ret, name): raise lzc_exc.ReadOnlyPool(_pool_name(snapname)) if ret == errno.EAGAIN: raise lzc_exc.SuspendedPool(_pool_name(snapname)) + if ret == errno.EACCES: + raise lzc_exc.EncryptionKeyNotLoaded() if ret == ECKSUM: raise lzc_exc.BadStream() if ret == ZFS_ERR_WRONG_PARENT: diff --git a/contrib/pyzfs/libzfs_core/_libzfs_core.py b/contrib/pyzfs/libzfs_core/_libzfs_core.py index fcfa5be31b1f..fa74ad9a760c 100644 --- a/contrib/pyzfs/libzfs_core/_libzfs_core.py +++ b/contrib/pyzfs/libzfs_core/_libzfs_core.py @@ -1426,6 +1426,135 @@ def lzc_receive_with_cmdprops( return (int(c_read_bytes[0]), action_handle) +@_uncommitted() +def lzc_receive_with_heal( + snapname, fd, begin_record, force=False, corrective=True, resumable=False, + raw=False, origin=None, props=None, cmdprops=None, key=None, cleanup_fd=-1, + action_handle=0 +): + ''' + Like :func:`lzc_receive_cmdprops`, but allows the caller to pass an + additional 'corrective' argument. The 'corrective' boolean set to true + indicates that a corruption healing receive should be performed. + + :param bytes snapname: the name of the snapshot to create. + :param int fd: the file descriptor from which to read the stream. + :param begin_record: the stream's begin record. + :type begin_record: ``cffi`` `CData` representing the dmu_replay_record_t + structure. + :param bool force: whether to roll back or destroy the target filesystem + if that is required to receive the stream. + :param bool corrective: whether this stream should be used to heal data. + :param bool resumable: whether this stream should be treated as resumable. + If the receive fails due to premature stream termination, the + intermediate state will be preserved on disk and may subsequently be + resumed with :func:`lzc_send_resume`. + :param bool raw: whether this is a "raw" stream. + :param origin: the optional origin snapshot name if the stream is for a + clone. + :type origin: bytes or None + :param props: the properties to set on the snapshot as *received* + properties. + :type props: dict of bytes : Any + :param cmdprops: the properties to set on the snapshot as local overrides + to *received* properties. `bool` values are forcefully inherited while + every other value is set locally as if the command "zfs set" was + invoked immediately before the receive. + :type cmdprops: dict of bytes : Any + :param key: raw bytes representing user's wrapping key + :type key: bytes + :param int cleanup_fd: file descriptor used to set a cleanup-on-exit file + descriptor. + :param int action_handle: variable used to pass the handle for guid/ds + mapping: this should be set to zero on first call and will contain an + updated handle on success, it should be passed in subsequent calls. + + :return: a tuple with two elements where the first one is the number of + bytes read from the file descriptor and the second one is the + action_handle return value. + + :raises IOError: if an input / output error occurs while reading from the + ``fd``. + :raises DatasetExists: if the snapshot named ``snapname`` already exists. + :raises DatasetExists: if the stream is a full stream and the destination + filesystem already exists. + :raises DatasetExists: if ``force`` is `True` but the destination + filesystem could not be rolled back to a matching snapshot because a + newer snapshot exists and it is an origin of a cloned filesystem. + :raises StreamMismatch: if an incremental stream is received and the latest + snapshot of the destination filesystem does not match the source + snapshot of the stream. + :raises StreamMismatch: if a full stream is received and the destination + filesystem already exists and it has at least one snapshot, and + ``force`` is `False`. + :raises StreamMismatch: if an incremental clone stream is received but the + specified ``origin`` is not the actual received origin. + :raises DestinationModified: if an incremental stream is received and the + destination filesystem has been modified since the last snapshot and + ``force`` is `False`. + :raises DestinationModified: if a full stream is received and the + destination filesystem already exists and it does not have any + snapshots, and ``force`` is `False`. + :raises DatasetNotFound: if the destination filesystem and its parent do + not exist. + :raises DatasetNotFound: if the ``origin`` is not `None` and does not + exist. + :raises DatasetBusy: if ``force`` is `True` but the destination filesystem + could not be rolled back to a matching snapshot because a newer + snapshot is held and could not be destroyed. + :raises DatasetBusy: if another receive operation is being performed on the + destination filesystem. + :raises EncryptionKeyNotLoaded: if corrective is set to true indicates the + key must be loaded to do a non-raw corrective recv on an encrypted + dataset. + :raises BadStream: if corrective is set to true indicates that + corrective recv was not able to reconstruct a corrupted block. + :raises BadStream: if the stream is corrupt or it is not recognized or it + is a compound stream or it is a clone stream, but ``origin`` is `None`. + :raises BadStream: if a clone stream is received and the destination + filesystem already exists. + :raises StreamFeatureNotSupported: if corrective is set to true indicates + stream is not compatible with the data in the pool. + :raises StreamFeatureNotSupported: if the stream has a feature that is not + supported on this side. + :raises ReceivePropertyFailure: if one or more of the specified properties + is invalid or has an invalid type or value. + :raises NameInvalid: if the name of either snapshot is invalid. + :raises NameTooLong: if the name of either snapshot is too long. + ''' + + if origin is not None: + c_origin = origin + else: + c_origin = _ffi.NULL + if action_handle is not None: + c_action_handle = _ffi.new("uint64_t *") + else: + c_action_handle = _ffi.NULL + c_read_bytes = _ffi.new("uint64_t *") + c_errflags = _ffi.new("uint64_t *") + if props is None: + props = {} + if cmdprops is None: + cmdprops = {} + if key is None: + key = b"" + else: + key = bytes(key) + + nvlist = nvlist_in(props) + cmdnvlist = nvlist_in(cmdprops) + properrs = {} + with nvlist_out(properrs) as c_errors: + ret = _lib.lzc_receive_with_heal( + snapname, nvlist, cmdnvlist, key, len(key), c_origin, + force, corrective, resumable, raw, fd, begin_record, cleanup_fd, + c_read_bytes, c_errflags, c_action_handle, c_errors) + errors.lzc_receive_translate_errors( + ret, snapname, fd, force, raw, False, False, origin, properrs) + return (int(c_read_bytes[0]), action_handle) + + @_uncommitted() def lzc_reopen(poolname, restart=True): ''' diff --git a/contrib/pyzfs/libzfs_core/bindings/libzfs_core.py b/contrib/pyzfs/libzfs_core/bindings/libzfs_core.py index 1b46a0891944..bcb9ed379e21 100644 --- a/contrib/pyzfs/libzfs_core/bindings/libzfs_core.py +++ b/contrib/pyzfs/libzfs_core/bindings/libzfs_core.py @@ -112,6 +112,10 @@ uint8_t *, uint_t, const char *, boolean_t, boolean_t, boolean_t, int, const dmu_replay_record_t *, int, uint64_t *, uint64_t *, uint64_t *, nvlist_t **); + int lzc_receive_with_heal(const char *, nvlist_t *, nvlist_t *, + uint8_t *, uint_t, const char *, boolean_t, boolean_t, boolean_t, + boolean_t, int, const dmu_replay_record_t *, int, uint64_t *, + uint64_t *, uint64_t *, nvlist_t **); int lzc_receive_with_header(const char *, nvlist_t *, const char *, boolean_t, boolean_t, boolean_t, int, const dmu_replay_record_t *); int lzc_release(nvlist_t *, nvlist_t **); diff --git a/contrib/pyzfs/libzfs_core/test/test_libzfs_core.py b/contrib/pyzfs/libzfs_core/test/test_libzfs_core.py index d949d88d5a1e..c94ae6de6bbf 100644 --- a/contrib/pyzfs/libzfs_core/test/test_libzfs_core.py +++ b/contrib/pyzfs/libzfs_core/test/test_libzfs_core.py @@ -1903,6 +1903,8 @@ def test_send_to_ro_file(self): with self.assertRaises(lzc_exc.StreamIOError) as ctx: lzc.lzc_send(snap, None, fd) os.close(fd) + os.unlink(output.name) + self.assertEqual(ctx.exception.errno, errno.EBADF) def test_recv_full(self): @@ -2909,6 +2911,27 @@ def test_recv_with_cmdprops(self): self.assertEqual(fs.getProperty("compression"), b"on") self.assertEqual(fs.getProperty("ns:prop"), b"val") + def test_recv_with_heal(self): + snap = ZFSTest.pool.makeName(b"fs1@snap1") + fs = ZFSTest.pool.getFilesystem(b"fs1") + props = {} + cmdprops = { + b"compression": 0x01, + b"ns:prop": b"val" + } + + lzc.lzc_snapshot([snap]) + with tempfile.TemporaryFile(suffix='.zstream') as stream: + lzc.lzc_send(snap, None, stream.fileno()) + stream.seek(0) + (header, c_header) = lzc.receive_header(stream.fileno()) + lzc.lzc_receive_with_heal( + snap, stream.fileno(), c_header, props=props, + cmdprops=cmdprops) + self.assertExists(snap) + self.assertEqual(fs.getProperty("compression"), b"on") + self.assertEqual(fs.getProperty("ns:prop"), b"val") + def test_recv_with_cmdprops_and_recvprops(self): fromsnap = ZFSTest.pool.makeName(b"fs1@snap1") fs = ZFSTest.pool.getFilesystem(b"recv") @@ -4132,7 +4155,8 @@ def __init__(self, size=128 * 1024 * 1024, readonly=False, filesystems=[]): cachefile = 'none' self._zpool_create = [ 'zpool', 'create', '-o', 'cachefile=' + cachefile, - '-O', 'mountpoint=legacy', self._pool_name, self._pool_file_path] + '-O', 'mountpoint=legacy', '-O', 'compression=off', + self._pool_name, self._pool_file_path] try: os.ftruncate(fd, size) os.close(fd) diff --git a/contrib/pyzfs/setup.py.in b/contrib/pyzfs/setup.py.in index 934b3189ebe1..43a1accfaf02 100644 --- a/contrib/pyzfs/setup.py.in +++ b/contrib/pyzfs/setup.py.in @@ -13,9 +13,10 @@ # See the License for the specific language governing permissions and # limitations under the License. # -from __future__ import absolute_import, division, print_function - from setuptools import setup, find_packages +import os.path + +srcdir = "@abs_top_srcdir@/contrib/pyzfs" setup( name="pyzfs", @@ -44,7 +45,8 @@ setup( "libzfs_core", ], - packages=find_packages(), + packages=find_packages(where=srcdir), + package_dir={"": os.path.relpath(srcdir)}, include_package_data=True, install_requires=[ "cffi", diff --git a/contrib/zcp/Makefile.am b/contrib/zcp/Makefile.am index e6a777ad7ba7..fc3f01a233cb 100644 --- a/contrib/zcp/Makefile.am +++ b/contrib/zcp/Makefile.am @@ -1 +1 @@ -EXTRA_DIST = autosnap.lua +dist_noinst_DATA += %D%/autosnap.lua diff --git a/copy-builtin b/copy-builtin index cd6f259092ed..18cc741b58e7 100755 --- a/copy-builtin +++ b/copy-builtin @@ -43,32 +43,8 @@ config ZFS If unsure, say N. EOF -add_after() -{ - FILE="$1" - MARKER="$2" - NEW="$3" - - while IFS='' read -r LINE - do - printf "%s\n" "$LINE" - - if [ -n "$MARKER" ] && [ "$LINE" = "$MARKER" ] - then - printf "%s\n" "$NEW" - MARKER='' - if IFS='' read -r LINE - then - [ "$LINE" != "$NEW" ] && printf "%s\n" "$LINE" - fi - fi - done < "$FILE" > "$FILE.new" - - mv "$FILE.new" "$FILE" -} - -add_after "$KERNEL_DIR/fs/Kconfig" 'if BLOCK' 'source "fs/zfs/Kconfig"' -add_after "$KERNEL_DIR/fs/Makefile" 'endif' 'obj-$(CONFIG_ZFS) += zfs/' +sed -i '/source "fs\/ext2\/Kconfig\"/i\source "fs/zfs/Kconfig"' "$KERNEL_DIR/fs/Kconfig" +echo 'obj-$(CONFIG_ZFS) += zfs/' >> "$KERNEL_DIR/fs/Makefile" echo "$0: done. now you can build the kernel with ZFS support." >&2 echo "$0: make sure you enable ZFS support (CONFIG_ZFS) before building." >&2 diff --git a/etc/Makefile.am b/etc/Makefile.am index 179c2400978c..b4b3ae1f5798 100644 --- a/etc/Makefile.am +++ b/etc/Makefile.am @@ -1,9 +1,98 @@ -include $(top_srcdir)/config/Shellcheck.am +sudoersddir = $(sysconfdir)/sudoers.d +sudoersd_DATA = \ + %D%/sudoers.d/zfs + +dist_noinst_DATA += $(sudoersd_DATA) + + +sysconf_zfsdir = $(sysconfdir)/zfs + +dist_sysconf_zfs_DATA = \ + %D%/zfs/vdev_id.conf.alias.example \ + %D%/zfs/vdev_id.conf.multipath.example \ + %D%/zfs/vdev_id.conf.sas_direct.example \ + %D%/zfs/vdev_id.conf.sas_switch.example \ + %D%/zfs/vdev_id.conf.scsi.example + +sysconf_zfs_DATA = \ + %D%/zfs/zfs-functions + +SUBSTFILES += $(sysconf_zfs_DATA) +SHELLCHECKSCRIPTS += $(sysconf_zfs_DATA) +$(call SHELLCHECK_OPTS,$(sysconf_zfs_DATA)): SHELLCHECK_SHELL = sh -SUBDIRS = zfs sudoers.d -SHELLCHECKDIRS = default $(ZFS_INIT_SYSV) zfs if BUILD_LINUX -SUBDIRS += default $(ZFS_INIT_SYSTEMD) $(ZFS_INIT_SYSV) $(ZFS_MODULE_LOAD) +initconf_DATA = \ + %D%/default/zfs + +SUBSTFILES += $(initconf_DATA) +SHELLCHECKSCRIPTS += $(initconf_DATA) +$(call SHELLCHECK_OPTS,$(initconf_DATA)): SHELLCHECK_SHELL = sh + + +if INIT_SYSV +dist_noinst_DATA += %D%/init.d/README.md + +init_SCRIPTS = \ + %D%/init.d/zfs-import \ + %D%/init.d/zfs-load-key \ + %D%/init.d/zfs-mount \ + %D%/init.d/zfs-share \ + %D%/init.d/zfs-zed + +SUBSTFILES += $(init_SCRIPTS) +SHELLCHECKSCRIPTS += $(init_SCRIPTS) +$(call SHELLCHECK_OPTS,$(init_SCRIPTS)): SHELLCHECK_SHELL = sh +endif + + +if INIT_SYSTEMD +dist_systemdpreset_DATA = \ + %D%/systemd/system/50-zfs.preset + +systemdunit_DATA = \ + %D%/systemd/system/zfs-import-cache.service \ + %D%/systemd/system/zfs-import-scan.service \ + %D%/systemd/system/zfs-import.target \ + %D%/systemd/system/zfs-mount.service \ + %D%/systemd/system/zfs-scrub-monthly@.timer \ + %D%/systemd/system/zfs-scrub-weekly@.timer \ + %D%/systemd/system/zfs-scrub@.service \ + %D%/systemd/system/zfs-trim-monthly@.timer \ + %D%/systemd/system/zfs-trim-weekly@.timer \ + %D%/systemd/system/zfs-trim@.service \ + %D%/systemd/system/zfs-share.service \ + %D%/systemd/system/zfs-volume-wait.service \ + %D%/systemd/system/zfs-volumes.target \ + %D%/systemd/system/zfs-zed.service \ + %D%/systemd/system/zfs.target + +SUBSTFILES += $(systemdunit_DATA) + +INSTALL_DATA_HOOKS += systemd-install-data-hook +systemd-install-data-hook: + $(MKDIR_P) "$(DESTDIR)$(systemdunitdir)" + ln -sf /dev/null "$(DESTDIR)$(systemdunitdir)/zfs-import.service" + + +systemdgenerator_PROGRAMS = \ + %D%/systemd/system-generators/zfs-mount-generator + +%C%_systemd_system_generators_zfs_mount_generator_SOURCES = \ + %D%/systemd/system-generators/zfs-mount-generator.c + +%C%_systemd_system_generators_zfs_mount_generator_LDADD = \ + libzfs.la + +%C%_systemd_system_generators_zfs_mount_generator_LDFLAGS = -pthread + +CPPCHECKTARGETS += $(systemdgenerator_PROGRAMS) +endif + + +if WANT_MODULES_LOAD_D +dist_modulesload_DATA = \ + %D%/modules-load.d/zfs.conf +endif endif -DIST_SUBDIRS = default init.d zfs systemd modules-load.d sudoers.d diff --git a/etc/default/Makefile.am b/etc/default/Makefile.am deleted file mode 100644 index 0f7c96698d45..000000000000 --- a/etc/default/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -include $(top_srcdir)/config/Substfiles.am -include $(top_srcdir)/config/Shellcheck.am - -initconf_SCRIPTS = zfs - -SUBSTFILES += $(initconf_SCRIPTS) - -SHELLCHECK_SHELL = sh diff --git a/etc/init.d/Makefile.am b/etc/init.d/Makefile.am deleted file mode 100644 index 625d0b91fd11..000000000000 --- a/etc/init.d/Makefile.am +++ /dev/null @@ -1,10 +0,0 @@ -include $(top_srcdir)/config/Substfiles.am -include $(top_srcdir)/config/Shellcheck.am - -EXTRA_DIST += README.md - -init_SCRIPTS = zfs-import zfs-load-key zfs-mount zfs-share zfs-zed - -SUBSTFILES += $(init_SCRIPTS) - -SHELLCHECK_SHELL = sh diff --git a/etc/init.d/zfs-mount.in b/etc/init.d/zfs-mount.in index d196d1555a45..df28c6c951c5 100755 --- a/etc/init.d/zfs-mount.in +++ b/etc/init.d/zfs-mount.in @@ -63,7 +63,7 @@ do_depend() # Mount all datasets/filesystems do_mount() { - local verbose overlay i mntpt + local verbose overlay check_boolean "$VERBOSE_MOUNT" && verbose=v check_boolean "$DO_OVERLAY_MOUNTS" && overlay=O @@ -71,102 +71,18 @@ do_mount() zfs_action "Mounting ZFS filesystem(s)" \ "$ZFS" mount "-a$verbose$overlay" "$MOUNT_EXTRA_OPTIONS" - # Require each volume/filesystem to have 'noauto' and no fsck - # option. This shouldn't really be necessary, as long as one - # can get zfs-import to run sufficiently early on in the boot - # process - before local mounts. This is just here in case/if - # this isn't possible. - check_boolean "$VERBOSE_MOUNT" && \ - zfs_log_begin_msg "Mounting volumes and filesystems registered in fstab" - - read_mtab "^/dev/(zd|zvol)" - read_fstab "^/dev/(zd|zvol)" - i=0; var="FSTAB_0" - while [ -n "$(eval echo "\$$var")" ] - do - mntpt=$(eval echo "\$$var") - dev=$(eval echo "\$FSTAB_dev_$i") - if ! in_mtab "$mntpt" && ! is_mounted "$mntpt" && [ -e "$dev" ] - then - check_boolean "$VERBOSE_MOUNT" && \ - zfs_log_progress_msg "$mntpt " - fsck "$dev" && mount "$mntpt" - fi - - i=$((i + 1)) - var=$(eval echo "FSTAB_$i") - done - - read_mtab "[[:space:]]zfs[[:space:]]" - read_fstab "[[:space:]]zfs[[:space:]]" - i=0; var=$(eval echo "FSTAB_$i") - while [ -n "$(eval echo "\$$var")" ] - do - mntpt=$(eval echo "\$$var") - if ! in_mtab "$mntpt" && ! is_mounted "$mntpt" - then - check_boolean "$VERBOSE_MOUNT" && \ - zfs_log_progress_msg "$mntpt " - mount "$mntpt" - fi - - i=$((i + 1)) - var=$(eval echo "FSTAB_$i") - done - check_boolean "$VERBOSE_MOUNT" && zfs_log_end_msg 0 - return 0 } # Unmount all filesystems do_unmount() { - local i var mntpt - # This shouldn't really be necessary, as long as one can get # zfs-import to run sufficiently late in the shutdown/reboot process # - after unmounting local filesystems. This is just here in case/if # this isn't possible. zfs_action "Unmounting ZFS filesystems" "$ZFS" unmount -a - check_boolean "$VERBOSE_MOUNT" && \ - zfs_log_begin_msg "Unmounting volumes and filesystems registered in fstab" - - read_mtab "^/dev/(zd|zvol)" - read_fstab "^/dev/(zd|zvol)" - i=0; var="FSTAB_0" - while [ -n "$(eval echo "\$$var")" ] - do - mntpt=$(eval echo "\$$var") - dev=$(eval echo "\$FSTAB_dev_$i") - if in_mtab "$mntpt" - then - check_boolean "$VERBOSE_MOUNT" && \ - zfs_log_progress_msg "$mntpt " - umount "$mntpt" - fi - - i=$((i + 1)) - var=$(eval echo "FSTAB_$i") - done - - read_mtab "[[:space:]]zfs[[:space:]]" - read_fstab "[[:space:]]zfs[[:space:]]" - i=0; var="FSTAB_0" - while [ -n "$(eval echo "\$$var")" ] - do - mntpt=$(eval echo "\$$var") - if in_mtab "$mntpt"; then - check_boolean "$VERBOSE_MOUNT" && \ - zfs_log_progress_msg "$mntpt " - umount "$mntpt" - fi - - i=$((i + 1)) - var=$(eval echo "FSTAB_$i") - done - check_boolean "$VERBOSE_MOUNT" && zfs_log_end_msg 0 - return 0 } diff --git a/etc/init.d/zfs-zed.in b/etc/init.d/zfs-zed.in index 47f742259b27..e9cf8867403c 100755 --- a/etc/init.d/zfs-zed.in +++ b/etc/init.d/zfs-zed.in @@ -69,8 +69,7 @@ do_stop() then # No pools imported, it is/should be safe/possible to # unload modules. - zfs_action "Unloading modules" rmmod zfs zunicode \ - zavl zcommon znvpair zlua spl + zfs_action "Unloading modules" rmmod zfs spl return "$?" fi } diff --git a/etc/modules-load.d/.gitignore b/etc/modules-load.d/.gitignore deleted file mode 100644 index fee921708337..000000000000 --- a/etc/modules-load.d/.gitignore +++ /dev/null @@ -1 +0,0 @@ -*.conf diff --git a/etc/modules-load.d/Makefile.am b/etc/modules-load.d/Makefile.am deleted file mode 100644 index 8a2955767b1e..000000000000 --- a/etc/modules-load.d/Makefile.am +++ /dev/null @@ -1,2 +0,0 @@ -dist_modulesload_DATA = \ - zfs.conf diff --git a/etc/sudoers.d/Makefile.am b/etc/sudoers.d/Makefile.am deleted file mode 100644 index 6f7ac8dbfd61..000000000000 --- a/etc/sudoers.d/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -sudoersddir = $(sysconfdir)/sudoers.d -sudoersd_DATA = zfs - -EXTRA_DIST = \ - zfs diff --git a/etc/systemd/Makefile.am b/etc/systemd/Makefile.am deleted file mode 100644 index 66232a5ff197..000000000000 --- a/etc/systemd/Makefile.am +++ /dev/null @@ -1,4 +0,0 @@ -include $(top_srcdir)/config/Shellcheck.am - -SUBDIRS = system system-generators -SHELLCHECKDIRS = system-generators diff --git a/etc/systemd/system-generators/Makefile.am b/etc/systemd/system-generators/Makefile.am deleted file mode 100644 index e5920bf39203..000000000000 --- a/etc/systemd/system-generators/Makefile.am +++ /dev/null @@ -1,14 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -systemdgenerator_PROGRAMS = \ - zfs-mount-generator - -zfs_mount_generator_SOURCES = \ - zfs-mount-generator.c - -zfs_mount_generator_LDADD = \ - $(abs_top_builddir)/lib/libzfs/libzfs.la - -zfs_mount_generator_LDFLAGS = -pthread - -include $(top_srcdir)/config/CppCheck.am diff --git a/etc/systemd/system-generators/zfs-mount-generator.c b/etc/systemd/system-generators/zfs-mount-generator.c index f4c6c26a0b34..b07574e72afe 100644 --- a/etc/systemd/system-generators/zfs-mount-generator.c +++ b/etc/systemd/system-generators/zfs-mount-generator.c @@ -681,25 +681,28 @@ line_worker(char *line, const char *cachefile) } *(tofree++) = linktgt; - char *dependencies[][2] = { + struct dep { + const char *type; + char *list; + } deps[] = { {"wants", wantedby}, {"requires", requiredby}, {} }; - for (__typeof__(&*dependencies) dep = &*dependencies; **dep; ++dep) { - if (!(*dep)[1]) + for (struct dep *dep = deps; dep->type; ++dep) { + if (!dep->list) continue; - for (char *reqby = strtok_r((*dep)[1], " ", &toktmp); + for (char *reqby = strtok_r(dep->list, " ", &toktmp); reqby; reqby = strtok_r(NULL, " ", &toktmp)) { char *depdir; if (asprintf( - &depdir, "%s.%s", reqby, (*dep)[0]) == -1) { + &depdir, "%s.%s", reqby, dep->type) == -1) { fprintf(stderr, PROGNAME "[%d]: %s: " "out of memory for dependent dir name " "\"%s.%s\"!\n", - getpid(), dataset, reqby, (*dep)[0]); + getpid(), dataset, reqby, dep->type); continue; } diff --git a/etc/systemd/system/.gitignore b/etc/systemd/system/.gitignore index 4813c65a25a8..95d33fdd436d 100644 --- a/etc/systemd/system/.gitignore +++ b/etc/systemd/system/.gitignore @@ -1,4 +1,3 @@ *.service *.target -*.preset *.timer diff --git a/etc/systemd/system/50-zfs.preset.in b/etc/systemd/system/50-zfs.preset similarity index 100% rename from etc/systemd/system/50-zfs.preset.in rename to etc/systemd/system/50-zfs.preset diff --git a/etc/systemd/system/Makefile.am b/etc/systemd/system/Makefile.am deleted file mode 100644 index 5e65e1db420c..000000000000 --- a/etc/systemd/system/Makefile.am +++ /dev/null @@ -1,24 +0,0 @@ -include $(top_srcdir)/config/Substfiles.am - -systemdpreset_DATA = \ - 50-zfs.preset - -systemdunit_DATA = \ - zfs-zed.service \ - zfs-import-cache.service \ - zfs-import-scan.service \ - zfs-mount.service \ - zfs-share.service \ - zfs-volume-wait.service \ - zfs-import.target \ - zfs-volumes.target \ - zfs.target \ - zfs-scrub-monthly@.timer \ - zfs-scrub-weekly@.timer \ - zfs-scrub@.service - -SUBSTFILES += $(systemdpreset_DATA) $(systemdunit_DATA) - -install-data-hook: - $(MKDIR_P) "$(DESTDIR)$(systemdunitdir)" - ln -sf /dev/null "$(DESTDIR)$(systemdunitdir)/zfs-import.service" diff --git a/etc/systemd/system/zfs-import-cache.service.in b/etc/systemd/system/zfs-import-cache.service.in index 5e5c6281c9d7..08c11698bcce 100644 --- a/etc/systemd/system/zfs-import-cache.service.in +++ b/etc/systemd/system/zfs-import-cache.service.in @@ -14,6 +14,7 @@ ConditionPathIsDirectory=/sys/module/zfs [Service] Type=oneshot RemainAfterExit=yes +EnvironmentFile=-@initconfdir@/zfs ExecStart=@sbindir@/zpool import -c @sysconfdir@/zfs/zpool.cache -aN $ZPOOL_IMPORT_OPTS [Install] diff --git a/etc/systemd/system/zfs-import-scan.service.in b/etc/systemd/system/zfs-import-scan.service.in index d3c083f7e9cf..c37f70f13b71 100644 --- a/etc/systemd/system/zfs-import-scan.service.in +++ b/etc/systemd/system/zfs-import-scan.service.in @@ -13,6 +13,7 @@ ConditionPathIsDirectory=/sys/module/zfs [Service] Type=oneshot RemainAfterExit=yes +EnvironmentFile=-@initconfdir@/zfs ExecStart=@sbindir@/zpool import -aN -o cachefile=none $ZPOOL_IMPORT_OPTS [Install] diff --git a/etc/systemd/system/zfs-mount.service.in b/etc/systemd/system/zfs-mount.service.in index 3ab82fb033c1..66d894923f4a 100644 --- a/etc/systemd/system/zfs-mount.service.in +++ b/etc/systemd/system/zfs-mount.service.in @@ -11,6 +11,7 @@ ConditionPathIsDirectory=/sys/module/zfs [Service] Type=oneshot RemainAfterExit=yes +EnvironmentFile=-@initconfdir@/zfs ExecStart=@sbindir@/zfs mount -a [Install] diff --git a/etc/systemd/system/zfs-scrub@.service.in b/etc/systemd/system/zfs-scrub@.service.in index bebe91d746ae..8ffffeb0cf6c 100644 --- a/etc/systemd/system/zfs-scrub@.service.in +++ b/etc/systemd/system/zfs-scrub@.service.in @@ -7,8 +7,9 @@ ConditionACPower=true ConditionPathIsDirectory=/sys/module/zfs [Service] +EnvironmentFile=-@initconfdir@/zfs ExecStart=/bin/sh -c '\ -if @sbindir@/zpool status %i | grep "scrub in progress"; then\ +if @sbindir@/zpool status %i | grep -q "scrub in progress"; then\ exec @sbindir@/zpool wait -t scrub %i;\ else exec @sbindir@/zpool scrub -w %i; fi' ExecStop=-/bin/sh -c '@sbindir@/zpool scrub -p %i 2>/dev/null || true' diff --git a/etc/systemd/system/zfs-share.service.in b/etc/systemd/system/zfs-share.service.in index 745077513c30..263055e5281f 100644 --- a/etc/systemd/system/zfs-share.service.in +++ b/etc/systemd/system/zfs-share.service.in @@ -13,6 +13,7 @@ ConditionPathIsDirectory=/sys/module/zfs [Service] Type=oneshot RemainAfterExit=yes +EnvironmentFile=-@initconfdir@/zfs ExecStart=@sbindir@/zfs share -a [Install] diff --git a/etc/systemd/system/zfs-trim-monthly@.timer.in b/etc/systemd/system/zfs-trim-monthly@.timer.in new file mode 100644 index 000000000000..8c13ffb304b3 --- /dev/null +++ b/etc/systemd/system/zfs-trim-monthly@.timer.in @@ -0,0 +1,12 @@ +[Unit] +Description=Monthly zpool trim timer for %i +Documentation=man:zpool-trim(8) + +[Timer] +OnCalendar=monthly +Persistent=true +RandomizedDelaySec=1h +Unit=zfs-trim@%i.service + +[Install] +WantedBy=timers.target diff --git a/etc/systemd/system/zfs-trim-weekly@.timer.in b/etc/systemd/system/zfs-trim-weekly@.timer.in new file mode 100644 index 000000000000..dced3d88b5c9 --- /dev/null +++ b/etc/systemd/system/zfs-trim-weekly@.timer.in @@ -0,0 +1,12 @@ +[Unit] +Description=Weekly zpool trim timer for %i +Documentation=man:zpool-trim(8) + +[Timer] +OnCalendar=weekly +Persistent=true +RandomizedDelaySec=1h +Unit=zfs-trim@%i.service + +[Install] +WantedBy=timers.target diff --git a/etc/systemd/system/zfs-trim@.service.in b/etc/systemd/system/zfs-trim@.service.in new file mode 100644 index 000000000000..423fb448c16f --- /dev/null +++ b/etc/systemd/system/zfs-trim@.service.in @@ -0,0 +1,15 @@ +[Unit] +Description=zpool trim on %i +Documentation=man:zpool-trim(8) +Requires=zfs.target +After=zfs.target +ConditionACPower=true +ConditionPathIsDirectory=/sys/module/zfs + +[Service] +EnvironmentFile=-@initconfdir@/zfs +ExecStart=/bin/sh -c '\ +if @sbindir@/zpool status %i | grep -q "(trimming)"; then\ +exec @sbindir@/zpool wait -t trim %i;\ +else exec @sbindir@/zpool trim -w %i; fi' +ExecStop=-/bin/sh -c '@sbindir@/zpool trim -s %i 2>/dev/null || true' diff --git a/etc/systemd/system/zfs-volume-wait.service.in b/etc/systemd/system/zfs-volume-wait.service.in index 4c77724d8bbb..110c0f5f52ee 100644 --- a/etc/systemd/system/zfs-volume-wait.service.in +++ b/etc/systemd/system/zfs-volume-wait.service.in @@ -8,6 +8,7 @@ ConditionPathIsDirectory=/sys/module/zfs [Service] Type=oneshot RemainAfterExit=yes +EnvironmentFile=-@initconfdir@/zfs ExecStart=@bindir@/zvol_wait [Install] diff --git a/etc/systemd/system/zfs-zed.service.in b/etc/systemd/system/zfs-zed.service.in index 008075138f02..73a83e59e510 100644 --- a/etc/systemd/system/zfs-zed.service.in +++ b/etc/systemd/system/zfs-zed.service.in @@ -4,6 +4,7 @@ Documentation=man:zed(8) ConditionPathIsDirectory=/sys/module/zfs [Service] +EnvironmentFile=-@initconfdir@/zfs ExecStart=@sbindir@/zed -F Restart=on-abort diff --git a/etc/zfs/Makefile.am b/etc/zfs/Makefile.am deleted file mode 100644 index 8a67d548bf7e..000000000000 --- a/etc/zfs/Makefile.am +++ /dev/null @@ -1,18 +0,0 @@ -include $(top_srcdir)/config/Substfiles.am -include $(top_srcdir)/config/Shellcheck.am - -pkgsysconfdir = $(sysconfdir)/zfs - -dist_pkgsysconf_DATA = \ - vdev_id.conf.alias.example \ - vdev_id.conf.sas_direct.example \ - vdev_id.conf.sas_switch.example \ - vdev_id.conf.multipath.example \ - vdev_id.conf.scsi.example - -pkgsysconf_SCRIPTS = \ - zfs-functions - -SUBSTFILES += $(pkgsysconf_SCRIPTS) - -SHELLCHECK_SHELL = sh diff --git a/include/Makefile.am b/include/Makefile.am index 4da43afd850d..1a7f67e9c440 100644 --- a/include/Makefile.am +++ b/include/Makefile.am @@ -1,4 +1,10 @@ -SUBDIRS = sys os +if BUILD_LINUX +include $(srcdir)/%D%/os/linux/Makefile.am +endif +if BUILD_FREEBSD +include $(srcdir)/%D%/os/freebsd/Makefile.am +endif + COMMON_H = \ cityhash.h \ @@ -7,27 +13,189 @@ COMMON_H = \ zfs_deleg.h \ zfs_fletcher.h \ zfs_namecheck.h \ - zfs_prop.h + zfs_prop.h \ + \ + sys/abd.h \ + sys/abd_impl.h \ + sys/aggsum.h \ + sys/arc.h \ + sys/arc_impl.h \ + sys/avl.h \ + sys/avl_impl.h \ + sys/bitops.h \ + sys/blake3.h \ + sys/blkptr.h \ + sys/bplist.h \ + sys/bpobj.h \ + sys/bptree.h \ + sys/bqueue.h \ + sys/btree.h \ + sys/dataset_kstats.h \ + sys/dbuf.h \ + sys/ddt.h \ + sys/dmu.h \ + sys/dmu_impl.h \ + sys/dmu_objset.h \ + sys/dmu_recv.h \ + sys/dmu_redact.h \ + sys/dmu_send.h \ + sys/dmu_traverse.h \ + sys/dmu_tx.h \ + sys/dmu_zfetch.h \ + sys/dnode.h \ + sys/dsl_bookmark.h \ + sys/dsl_crypt.h \ + sys/dsl_dataset.h \ + sys/dsl_deadlist.h \ + sys/dsl_deleg.h \ + sys/dsl_destroy.h \ + sys/dsl_dir.h \ + sys/dsl_pool.h \ + sys/dsl_prop.h \ + sys/dsl_scan.h \ + sys/dsl_synctask.h \ + sys/dsl_userhold.h \ + sys/edonr.h \ + sys/efi_partition.h \ + sys/frame.h \ + sys/hkdf.h \ + sys/metaslab.h \ + sys/metaslab_impl.h \ + sys/mmp.h \ + sys/mntent.h \ + sys/mod.h \ + sys/multilist.h \ + sys/nvpair.h \ + sys/nvpair_impl.h \ + sys/objlist.h \ + sys/pathname.h \ + sys/qat.h \ + sys/range_tree.h \ + sys/rrwlock.h \ + sys/sa.h \ + sys/sa_impl.h \ + sys/skein.h \ + sys/spa.h \ + sys/spa_boot.h \ + sys/spa_checkpoint.h \ + sys/spa_checksum.h \ + sys/spa_impl.h \ + sys/spa_log_spacemap.h \ + sys/space_map.h \ + sys/space_reftree.h \ + sys/sysevent.h \ + sys/txg.h \ + sys/txg_impl.h \ + sys/u8_textprep.h \ + sys/u8_textprep_data.h \ + sys/uberblock.h \ + sys/uberblock_impl.h \ + sys/uio_impl.h \ + sys/unique.h \ + sys/uuid.h \ + sys/vdev.h \ + sys/vdev_disk.h \ + sys/vdev_draid.h \ + sys/vdev_file.h \ + sys/vdev_impl.h \ + sys/vdev_indirect_births.h \ + sys/vdev_indirect_mapping.h \ + sys/vdev_initialize.h \ + sys/vdev_raidz.h \ + sys/vdev_raidz_impl.h \ + sys/vdev_rebuild.h \ + sys/vdev_removal.h \ + sys/vdev_trim.h \ + sys/xvattr.h \ + sys/zap.h \ + sys/zap_impl.h \ + sys/zap_leaf.h \ + sys/zcp.h \ + sys/zcp_global.h \ + sys/zcp_iter.h \ + sys/zcp_prop.h \ + sys/zcp_set.h \ + sys/zfeature.h \ + sys/zfs_acl.h \ + sys/zfs_bootenv.h \ + sys/zfs_chksum.h \ + sys/zfs_context.h \ + sys/zfs_debug.h \ + sys/zfs_delay.h \ + sys/zfs_file.h \ + sys/zfs_fuid.h \ + sys/zfs_project.h \ + sys/zfs_quota.h \ + sys/zfs_racct.h \ + sys/zfs_ratelimit.h \ + sys/zfs_refcount.h \ + sys/zfs_rlock.h \ + sys/zfs_sa.h \ + sys/zfs_stat.h \ + sys/zfs_sysfs.h \ + sys/zfs_vfsops.h \ + sys/zfs_vnops.h \ + sys/zfs_znode.h \ + sys/zil.h \ + sys/zil_impl.h \ + sys/zio.h \ + sys/zio_checksum.h \ + sys/zio_compress.h \ + sys/zio_crypt.h \ + sys/zio_impl.h \ + sys/zio_priority.h \ + sys/zrlock.h \ + sys/zthr.h \ + \ + sys/crypto/api.h \ + sys/crypto/common.h \ + sys/crypto/icp.h \ + \ + sys/fm/protocol.h \ + sys/fm/util.h \ + sys/fm/fs/zfs.h \ + \ + sys/fs/zfs.h \ + \ + sys/lua/lauxlib.h \ + sys/lua/lua.h \ + sys/lua/luaconf.h \ + sys/lua/lualib.h \ + \ + sys/sysevent/dev.h \ + sys/sysevent/eventdefs.h \ + \ + sys/zstd/zstd.h + + +KERNEL_H = \ + sys/zfs_ioctl.h \ + sys/zfs_ioctl_impl.h \ + sys/zfs_onexit.h \ + sys/zvol.h \ + sys/zvol_impl.h + USER_H = \ libnvpair.h \ - libuutil_common.h \ libuutil.h \ + libuutil_common.h \ libuutil_impl.h \ libzfs.h \ - libzfsbootenv.h \ libzfs_core.h \ + libzfsbootenv.h \ libzutil.h \ thread_pool.h + if CONFIG_USER libzfsdir = $(includedir)/libzfs -libzfs_HEADERS = $(COMMON_H) $(USER_H) +nobase_libzfs_HEADERS = $(COMMON_H) $(USER_H) endif +kerneldir = $(prefix)/src/zfs-$(VERSION)/include if CONFIG_KERNEL if BUILD_LINUX -kerneldir = @prefix@/src/zfs-$(VERSION)/include -kernel_HEADERS = $(COMMON_H) +nobase_kernel_HEADERS = $(COMMON_H) $(KERNEL_H) endif endif diff --git a/include/libnvpair.h b/include/libnvpair.h index bc50c3b7e1f8..738afaec7ec1 100644 --- a/include/libnvpair.h +++ b/include/libnvpair.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/libuutil.h b/include/libuutil.h index cadc20d2d8f3..cb3d366c4764 100644 --- a/include/libuutil.h +++ b/include/libuutil.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -85,14 +85,14 @@ extern void uu_warn(const char *, ...) __attribute__((format(printf, 1, 2))); extern void uu_vwarn(const char *, va_list) __attribute__((format(printf, 1, 0))); -extern void uu_die(const char *, ...) - __attribute__((format(printf, 1, 2))) __NORETURN; -extern void uu_vdie(const char *, va_list) - __attribute__((format(printf, 1, 0))) __NORETURN; -extern void uu_xdie(int, const char *, ...) - __attribute__((format(printf, 2, 3))) __NORETURN; -extern void uu_vxdie(int, const char *, va_list) - __attribute__((format(printf, 2, 0))) __NORETURN; +extern __attribute__((noreturn)) void uu_die(const char *, ...) + __attribute__((format(printf, 1, 2))); +extern __attribute__((noreturn)) void uu_vdie(const char *, va_list) + __attribute__((format(printf, 1, 0))); +extern __attribute__((noreturn)) void uu_xdie(int, const char *, ...) + __attribute__((format(printf, 2, 3))); +extern __attribute__((noreturn)) void uu_vxdie(int, const char *, va_list) + __attribute__((format(printf, 2, 0))); /* * Exit status functions (not to be used directly) diff --git a/include/libuutil_common.h b/include/libuutil_common.h index 52ac4887f75c..cdfa0c2f556f 100644 --- a/include/libuutil_common.h +++ b/include/libuutil_common.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/libuutil_impl.h b/include/libuutil_impl.h index 753bbff2461d..b9907447afd9 100644 --- a/include/libuutil_impl.h +++ b/include/libuutil_impl.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/libzfs.h b/include/libzfs.h index c0e53b88a636..4948cd0d34c8 100644 --- a/include/libzfs.h +++ b/include/libzfs.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -35,14 +35,13 @@ #define _LIBZFS_H extern __attribute__((visibility("default"))) #include +#include #include #include #include #include -#include #include #include -#include #include #ifdef __cplusplus @@ -151,6 +150,7 @@ typedef enum zfs_error { EZFS_EXPORT_IN_PROGRESS, /* currently exporting the pool */ EZFS_REBUILDING, /* resilvering (sequential reconstrution) */ EZFS_VDEV_NOTSUP, /* ops not supported for this type of vdev */ + EZFS_NOT_USER_NAMESPACE, /* a file is not a user namespace */ EZFS_UNKNOWN } zfs_error_t; @@ -423,9 +423,9 @@ typedef enum { ZPOOL_STATUS_OK } zpool_status_t; -_LIBZFS_H zpool_status_t zpool_get_status(zpool_handle_t *, char **, +_LIBZFS_H zpool_status_t zpool_get_status(zpool_handle_t *, const char **, zpool_errata_t *); -_LIBZFS_H zpool_status_t zpool_import_status(nvlist_t *, char **, +_LIBZFS_H zpool_status_t zpool_import_status(nvlist_t *, const char **, zpool_errata_t *); /* @@ -829,6 +829,9 @@ typedef struct recvflags { /* force unmount while recv snapshot (private) */ boolean_t forceunmount; + + /* use this recv to check (and heal if needed) an existing snapshot */ + boolean_t heal; } recvflags_t; _LIBZFS_H int zfs_receive(libzfs_handle_t *, const char *, nvlist_t *, @@ -870,45 +873,34 @@ _LIBZFS_H int zfs_unmountall(zfs_handle_t *, int); _LIBZFS_H int zfs_mount_delegation_check(void); #if defined(__linux__) || defined(__APPLE__) -_LIBZFS_H int zfs_parse_mount_options(char *mntopts, unsigned long *mntflags, - unsigned long *zfsflags, int sloppy, char *badopt, char *mtabopt); +_LIBZFS_H int zfs_parse_mount_options(const char *mntopts, + unsigned long *mntflags, unsigned long *zfsflags, int sloppy, char *badopt, + char *mtabopt); _LIBZFS_H void zfs_adjust_mount_options(zfs_handle_t *zhp, const char *mntpoint, char *mntopts, char *mtabopt); #endif /* * Share support functions. + * + * enum sa_protocol * lists are terminated with SA_NO_PROTOCOL, + * NULL means "all/any known to this libzfs". */ -_LIBZFS_H boolean_t zfs_is_shared(zfs_handle_t *); -_LIBZFS_H int zfs_share(zfs_handle_t *); -_LIBZFS_H int zfs_unshare(zfs_handle_t *); - -/* - * Protocol-specific share support functions. - */ -_LIBZFS_H boolean_t zfs_is_shared_nfs(zfs_handle_t *, char **); -_LIBZFS_H boolean_t zfs_is_shared_smb(zfs_handle_t *, char **); -_LIBZFS_H int zfs_share_nfs(zfs_handle_t *); -_LIBZFS_H int zfs_share_smb(zfs_handle_t *); -_LIBZFS_H int zfs_shareall(zfs_handle_t *); -_LIBZFS_H int zfs_unshare_nfs(zfs_handle_t *, const char *); -_LIBZFS_H int zfs_unshare_smb(zfs_handle_t *, const char *); -_LIBZFS_H int zfs_unshareall_nfs(zfs_handle_t *); -_LIBZFS_H int zfs_unshareall_smb(zfs_handle_t *); -_LIBZFS_H int zfs_unshareall_bypath(zfs_handle_t *, const char *); -_LIBZFS_H int zfs_unshareall_bytype(zfs_handle_t *, const char *, const char *); -_LIBZFS_H int zfs_unshareall(zfs_handle_t *); -_LIBZFS_H int zfs_deleg_share_nfs(libzfs_handle_t *, char *, char *, char *, - void *, void *, int, zfs_share_op_t); -_LIBZFS_H void zfs_commit_nfs_shares(void); -_LIBZFS_H void zfs_commit_smb_shares(void); -_LIBZFS_H void zfs_commit_all_shares(void); -_LIBZFS_H void zfs_commit_shares(const char *); +#define SA_NO_PROTOCOL -1 + +_LIBZFS_H boolean_t zfs_is_shared(zfs_handle_t *zhp, char **where, + const enum sa_protocol *proto); +_LIBZFS_H int zfs_share(zfs_handle_t *zhp, const enum sa_protocol *proto); +_LIBZFS_H int zfs_unshare(zfs_handle_t *zhp, const char *mountpoint, + const enum sa_protocol *proto); +_LIBZFS_H int zfs_unshareall(zfs_handle_t *zhp, + const enum sa_protocol *proto); +_LIBZFS_H void zfs_commit_shares(const enum sa_protocol *proto); _LIBZFS_H int zfs_nicestrtonum(libzfs_handle_t *, const char *, uint64_t *); /* - * Utility functions to run an _LIBZFS_Hal process. + * Utility functions to run an external process. */ #define STDOUT_VERBOSE 0x01 #define STDERR_VERBOSE 0x02 @@ -922,13 +914,13 @@ _LIBZFS_H int libzfs_run_process_get_stdout_nopath(const char *, char *[], _LIBZFS_H void libzfs_free_str_array(char **, int); -_LIBZFS_H int libzfs_envvar_is_set(char *); +_LIBZFS_H boolean_t libzfs_envvar_is_set(const char *); /* * Utility functions for zfs version */ -_LIBZFS_H void zfs_version_userland(char *, int); -_LIBZFS_H int zfs_version_kernel(char *, int); +_LIBZFS_H const char *zfs_version_userland(void); +_LIBZFS_H char *zfs_version_kernel(void); _LIBZFS_H int zfs_version_print(void); /* @@ -992,6 +984,15 @@ _LIBZFS_H int zpool_nextboot(libzfs_handle_t *, uint64_t, uint64_t, #endif /* __FreeBSD__ */ +#ifdef __linux__ + +/* + * Add or delete the given filesystem to/from the given user namespace. + */ +_LIBZFS_H int zfs_userns(zfs_handle_t *zhp, const char *nspath, int attach); + +#endif + #ifdef __cplusplus } #endif diff --git a/include/libzfs_core.h b/include/libzfs_core.h index 7acc03fc71bb..14a4857c35da 100644 --- a/include/libzfs_core.h +++ b/include/libzfs_core.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -21,9 +21,9 @@ /* * Copyright (c) 2012, 2020 by Delphix. All rights reserved. - * Copyright (c) 2017 Datto Inc. * Copyright 2017 RackTop Systems. * Copyright (c) 2017 Open-E, Inc. All Rights Reserved. + * Copyright (c) 2019 Datto Inc. */ #ifndef _LIBZFS_CORE_H @@ -86,6 +86,7 @@ enum lzc_send_flags { LZC_SEND_FLAG_SAVED = 1 << 4, }; +_LIBZFS_CORE_H int lzc_send_wrapper(int (*)(int, void *), int, void *); _LIBZFS_CORE_H int lzc_send(const char *, const char *, int, enum lzc_send_flags); _LIBZFS_CORE_H int lzc_send_resume(const char *, const char *, int, @@ -113,6 +114,10 @@ _LIBZFS_CORE_H int lzc_receive_with_cmdprops(const char *, nvlist_t *, nvlist_t *, uint8_t *, uint_t, const char *, boolean_t, boolean_t, boolean_t, int, const struct dmu_replay_record *, int, uint64_t *, uint64_t *, uint64_t *, nvlist_t **); +_LIBZFS_CORE_H int lzc_receive_with_heal(const char *, nvlist_t *, nvlist_t *, + uint8_t *, uint_t, const char *, boolean_t, boolean_t, boolean_t, boolean_t, + int, const struct dmu_replay_record *, int, uint64_t *, uint64_t *, + uint64_t *, nvlist_t **); _LIBZFS_CORE_H int lzc_send_space(const char *, const char *, enum lzc_send_flags, uint64_t *); _LIBZFS_CORE_H int lzc_send_space_resume_redacted(const char *, const char *, diff --git a/include/libzutil.h b/include/libzutil.h index c0a660ea7067..9b86c351bd2d 100644 --- a/include/libzutil.h +++ b/include/libzutil.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -97,8 +97,8 @@ _LIBZUTIL_H int zfs_append_partition(char *path, size_t max_len); _LIBZUTIL_H int zfs_resolve_shortname(const char *name, char *path, size_t pathlen); -_LIBZUTIL_H char *zfs_strip_partition(char *); -_LIBZUTIL_H char *zfs_strip_path(char *); +_LIBZUTIL_H char *zfs_strip_partition(const char *); +_LIBZUTIL_H const char *zfs_strip_path(const char *); _LIBZUTIL_H int zfs_strcmp_pathname(const char *, const char *, int); @@ -155,9 +155,9 @@ struct zfs_cmd; #define ANSI_RESET "\033[0m" #define ANSI_BOLD "\033[1m" -_LIBZUTIL_H void color_start(char *color); +_LIBZUTIL_H void color_start(const char *color); _LIBZUTIL_H void color_end(void); -_LIBZUTIL_H int printf_color(char *color, char *format, ...); +_LIBZUTIL_H int printf_color(const char *color, const char *format, ...); _LIBZUTIL_H const char *zfs_basename(const char *path); _LIBZUTIL_H ssize_t zfs_dirnamelen(const char *path); diff --git a/include/os/Makefile.am b/include/os/Makefile.am deleted file mode 100644 index 7eab1abde984..000000000000 --- a/include/os/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -if BUILD_LINUX -SUBDIRS = linux -endif -if BUILD_FREEBSD -SUBDIRS = freebsd -endif diff --git a/include/os/freebsd/Makefile.am b/include/os/freebsd/Makefile.am index 3c87d4a0e791..5ddb7cd710b8 100644 --- a/include/os/freebsd/Makefile.am +++ b/include/os/freebsd/Makefile.am @@ -1 +1,90 @@ -SUBDIRS = linux spl zfs +noinst_HEADERS = \ + %D%/linux/compiler.h \ + %D%/linux/types.h \ + \ + %D%/spl/acl/acl_common.h \ + \ + %D%/spl/rpc/xdr.h \ + \ + %D%/spl/sys/acl.h \ + %D%/spl/sys/acl_impl.h \ + %D%/spl/sys/atomic.h \ + %D%/spl/sys/byteorder.h \ + %D%/spl/sys/callb.h \ + %D%/spl/sys/ccompat.h \ + %D%/spl/sys/ccompile.h \ + %D%/spl/sys/cmn_err.h \ + %D%/spl/sys/condvar.h \ + %D%/spl/sys/cred.h \ + %D%/spl/sys/ctype.h \ + %D%/spl/sys/debug.h \ + %D%/spl/sys/dirent.h \ + %D%/spl/sys/disp.h \ + %D%/spl/sys/dkio.h \ + %D%/spl/sys/extdirent.h \ + %D%/spl/sys/fcntl.h \ + %D%/spl/sys/file.h \ + %D%/spl/sys/freebsd_rwlock.h \ + %D%/spl/sys/idmap.h \ + %D%/spl/sys/inttypes.h \ + %D%/spl/sys/isa_defs.h \ + %D%/spl/sys/kmem.h \ + %D%/spl/sys/kmem_cache.h \ + %D%/spl/sys/kstat.h \ + %D%/spl/sys/list.h \ + %D%/spl/sys/list_impl.h \ + %D%/spl/sys/lock.h \ + %D%/spl/sys/misc.h \ + %D%/spl/sys/mod_os.h \ + %D%/spl/sys/mode.h \ + %D%/spl/sys/mount.h \ + %D%/spl/sys/mutex.h \ + %D%/spl/sys/param.h \ + %D%/spl/sys/policy.h \ + %D%/spl/sys/proc.h \ + %D%/spl/sys/processor.h \ + %D%/spl/sys/procfs_list.h \ + %D%/spl/sys/random.h \ + %D%/spl/sys/rwlock.h \ + %D%/spl/sys/sdt.h \ + %D%/spl/sys/sid.h \ + %D%/spl/sys/sig.h \ + %D%/spl/sys/simd.h \ + %D%/spl/sys/simd_x86.h \ + %D%/spl/sys/spl_condvar.h \ + %D%/spl/sys/string.h \ + %D%/spl/sys/sunddi.h \ + %D%/spl/sys/sysmacros.h \ + %D%/spl/sys/systeminfo.h \ + %D%/spl/sys/systm.h \ + %D%/spl/sys/taskq.h \ + %D%/spl/sys/thread.h \ + %D%/spl/sys/time.h \ + %D%/spl/sys/timer.h \ + %D%/spl/sys/trace.h \ + %D%/spl/sys/trace_zfs.h \ + %D%/spl/sys/types.h \ + %D%/spl/sys/types32.h \ + %D%/spl/sys/uio.h \ + %D%/spl/sys/uuid.h \ + %D%/spl/sys/vfs.h \ + %D%/spl/sys/vm.h \ + %D%/spl/sys/vmsystm.h \ + %D%/spl/sys/vnode.h \ + %D%/spl/sys/vnode_impl.h \ + %D%/spl/sys/wmsum.h \ + %D%/spl/sys/zmod.h \ + %D%/spl/sys/zone.h \ + \ + %D%/zfs/sys/freebsd_crypto.h \ + %D%/zfs/sys/sha2.h \ + %D%/zfs/sys/vdev_os.h \ + %D%/zfs/sys/zfs_bootenv_os.h \ + %D%/zfs/sys/zfs_context_os.h \ + %D%/zfs/sys/zfs_ctldir.h \ + %D%/zfs/sys/zfs_dir.h \ + %D%/zfs/sys/zfs_ioctl_compat.h \ + %D%/zfs/sys/zfs_vfsops_os.h \ + %D%/zfs/sys/zfs_vnops_os.h \ + %D%/zfs/sys/zfs_znode_impl.h \ + %D%/zfs/sys/zpl.h diff --git a/include/os/freebsd/linux/Makefile.am b/include/os/freebsd/linux/Makefile.am deleted file mode 100644 index 00cff7f5dc65..000000000000 --- a/include/os/freebsd/linux/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -KERNEL_H = \ - compiler.h \ - types.h - -noinst_HEADERS = $(KERNEL_H) diff --git a/include/os/freebsd/spl/Makefile.am b/include/os/freebsd/spl/Makefile.am deleted file mode 100644 index b321825cb77e..000000000000 --- a/include/os/freebsd/spl/Makefile.am +++ /dev/null @@ -1 +0,0 @@ -SUBDIRS = acl rpc sys diff --git a/include/os/freebsd/spl/acl/Makefile.am b/include/os/freebsd/spl/acl/Makefile.am deleted file mode 100644 index 5c0698d02ea6..000000000000 --- a/include/os/freebsd/spl/acl/Makefile.am +++ /dev/null @@ -1,4 +0,0 @@ -KERNEL_H = \ - acl_common.h - -noinst_HEADERS = $(KERNEL_H) diff --git a/include/os/freebsd/spl/acl/acl_common.h b/include/os/freebsd/spl/acl/acl_common.h index 44f5bed592f6..2b77bdb6ea3d 100644 --- a/include/os/freebsd/spl/acl/acl_common.h +++ b/include/os/freebsd/spl/acl/acl_common.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/rpc/Makefile.am b/include/os/freebsd/spl/rpc/Makefile.am deleted file mode 100644 index f6faf4b188be..000000000000 --- a/include/os/freebsd/spl/rpc/Makefile.am +++ /dev/null @@ -1,4 +0,0 @@ -KERNEL_H = \ - xdr.h - -noinst_HEADERS = $(KERNEL_H) diff --git a/include/os/freebsd/spl/sys/Makefile.am b/include/os/freebsd/spl/sys/Makefile.am deleted file mode 100644 index 232aaf569fa2..000000000000 --- a/include/os/freebsd/spl/sys/Makefile.am +++ /dev/null @@ -1,75 +0,0 @@ -KERNEL_H = \ - acl_impl.h \ - acl.h \ - atomic.h \ - byteorder.h \ - callb.h \ - ccompat.h \ - ccompile.h \ - cmn_err.h \ - condvar.h \ - cred.h \ - ctype.h \ - debug.h \ - dirent.h \ - disp.h \ - dkio.h \ - extdirent.h \ - fcntl.h \ - file.h \ - freebsd_rwlock.h \ - idmap.h \ - inttypes.h \ - isa_defs.h \ - kmem_cache.h \ - kidmap.h \ - kmem.h \ - kstat.h \ - list_impl.h \ - list.h \ - lock.h \ - Makefile.am \ - misc.h \ - mod_os.h \ - mode.h \ - mount.h \ - mutex.h \ - param.h \ - policy.h \ - proc.h \ - processor.h \ - procfs_list.h \ - random.h \ - rwlock.h \ - sdt.h \ - sid.h \ - sig.h \ - simd_x86.h \ - simd.h \ - spl_condvar.h \ - string.h \ - strings.h \ - sunddi.h \ - sysmacros.h \ - systeminfo.h \ - systm.h \ - taskq.h \ - thread.h \ - time.h \ - timer.h \ - trace_zfs.h \ - trace.h \ - types.h \ - types32.h \ - uio.h \ - uuid.h \ - vfs.h \ - vm.h \ - vmsystm.h \ - vnode_impl.h \ - vnode.h \ - wmsum.h \ - zmod.h \ - zone.h - -noinst_HEADERS = $(KERNEL_H) diff --git a/include/os/freebsd/spl/sys/acl.h b/include/os/freebsd/spl/sys/acl.h index ee50b0a18368..e757a601a842 100644 --- a/include/os/freebsd/spl/sys/acl.h +++ b/include/os/freebsd/spl/sys/acl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/acl_impl.h b/include/os/freebsd/spl/sys/acl_impl.h index 1efbd6d73bd0..de5961ef7740 100644 --- a/include/os/freebsd/spl/sys/acl_impl.h +++ b/include/os/freebsd/spl/sys/acl_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/byteorder.h b/include/os/freebsd/spl/sys/byteorder.h index 0b3d01eb3759..4ea56094cea5 100644 --- a/include/os/freebsd/spl/sys/byteorder.h +++ b/include/os/freebsd/spl/sys/byteorder.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/callb.h b/include/os/freebsd/spl/sys/callb.h index cc67b0263c51..006a90cae8c0 100644 --- a/include/os/freebsd/spl/sys/callb.h +++ b/include/os/freebsd/spl/sys/callb.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -132,7 +132,7 @@ typedef struct callb_cpr { #define CALLB_CPR_INIT(cp, lockp, func, name) { \ strlcpy(curthread->td_name, (name), \ sizeof (curthread->td_name)); \ - bzero((caddr_t)(cp), sizeof (callb_cpr_t)); \ + memset(cp, 0, sizeof (callb_cpr_t)); \ (cp)->cc_lockp = lockp; \ (cp)->cc_id = callb_add(func, (void *)(cp), \ CB_CL_CPR_DAEMON, name); \ diff --git a/include/os/freebsd/spl/sys/ccompile.h b/include/os/freebsd/spl/sys/ccompile.h index 23e637983475..26cf4db87aea 100644 --- a/include/os/freebsd/spl/sys/ccompile.h +++ b/include/os/freebsd/spl/sys/ccompile.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -42,9 +42,6 @@ extern "C" { #endif #define EXPORT_SYMBOL(x) -#define MODULE_AUTHOR(s) -#define MODULE_DESCRIPTION(s) -#define MODULE_LICENSE(s) #define module_param(a, b, c) #define module_param_call(a, b, c, d, e) #define module_param_named(a, b, c, d) @@ -77,10 +74,12 @@ extern "C" { #ifndef LOCORE #ifndef HAVE_RPC_TYPES +#ifndef _KERNEL typedef int bool_t; typedef int enum_t; #endif #endif +#endif #ifndef __cplusplus #define __init diff --git a/include/os/freebsd/spl/sys/cmn_err.h b/include/os/freebsd/spl/sys/cmn_err.h index ddc2f0049e59..88132337f038 100644 --- a/include/os/freebsd/spl/sys/cmn_err.h +++ b/include/os/freebsd/spl/sys/cmn_err.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/cred.h b/include/os/freebsd/spl/sys/cred.h index 86f79011d6da..06d537126285 100644 --- a/include/os/freebsd/spl/sys/cred.h +++ b/include/os/freebsd/spl/sys/cred.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -48,138 +48,20 @@ extern "C" { typedef struct ucred cred_t; #define CRED() curthread->td_ucred -#define kcred (thread0.td_ucred) - -#define KUID_TO_SUID(x) (x) -#define KGID_TO_SGID(x) (x) -#define crgetuid(cred) ((cred)->cr_uid) -#define crgetruid(cred) ((cred)->cr_ruid) -#define crgetgid(cred) ((cred)->cr_gid) -#define crgetgroups(cred) ((cred)->cr_groups) -#define crgetngroups(cred) ((cred)->cr_ngroups) -#define crgetsid(cred, i) (NULL) -struct proc; /* cred.h is included in proc.h */ -struct prcred; -struct ksid; -struct ksidlist; -struct credklpd; -struct credgrp; - -struct auditinfo_addr; /* cred.h is included in audit.h */ - -extern int ngroups_max; /* * kcred is used when you need all privileges. */ +#define kcred (thread0.td_ucred) -extern void cred_init(void); -extern void crfree(cred_t *); -extern cred_t *cralloc(void); /* all but ref uninitialized */ -extern cred_t *cralloc_ksid(void); /* cralloc() + ksid alloc'ed */ -extern cred_t *crget(void); /* initialized */ -extern void crcopy_to(cred_t *, cred_t *); -extern cred_t *crdup(cred_t *); -extern void crdup_to(cred_t *, cred_t *); -extern cred_t *crgetcred(void); -extern void crset(struct proc *, cred_t *); -extern void crset_zone_privall(cred_t *); -extern int supgroupmember(gid_t, const cred_t *); -extern int hasprocperm(const cred_t *, const cred_t *); -extern int prochasprocperm(struct proc *, struct proc *, const cred_t *); -extern int crcmp(const cred_t *, const cred_t *); -extern cred_t *zone_kcred(void); - -extern gid_t crgetrgid(const cred_t *); -extern gid_t crgetsgid(const cred_t *); - -#define crgetzoneid(cr) ((cr)->cr_prison->pr_id) -extern projid_t crgetprojid(const cred_t *); - -extern cred_t *crgetmapped(const cred_t *); - - -extern const struct auditinfo_addr *crgetauinfo(const cred_t *); -extern struct auditinfo_addr *crgetauinfo_modifiable(cred_t *); - -extern uint_t crgetref(const cred_t *); - -extern const gid_t *crgetggroups(const struct credgrp *); - - -/* - * Sets real, effective and/or saved uid/gid; - * -1 argument accepted as "no change". - */ -extern int crsetresuid(cred_t *, uid_t, uid_t, uid_t); -extern int crsetresgid(cred_t *, gid_t, gid_t, gid_t); - -/* - * Sets real, effective and saved uids/gids all to the same - * values. Both values must be non-negative and <= MAXUID - */ -extern int crsetugid(cred_t *, uid_t, gid_t); - -/* - * Functions to handle the supplemental group list. - */ -extern struct credgrp *crgrpcopyin(int, gid_t *); -extern void crgrprele(struct credgrp *); -extern void crsetcredgrp(cred_t *, struct credgrp *); - -/* - * Private interface for setting zone association of credential. - */ -struct zone; -extern void crsetzone(cred_t *, struct zone *); -extern struct zone *crgetzone(const cred_t *); - -/* - * Private interface for setting project id in credential. - */ -extern void crsetprojid(cred_t *, projid_t); - -/* - * Private interface for nfs. - */ -extern cred_t *crnetadjust(cred_t *); - -/* - * Private interface for procfs. - */ -extern void cred2prcred(const cred_t *, struct prcred *); - -/* - * Private interfaces for Rampart Trusted Solaris. - */ -struct ts_label_s; -extern struct ts_label_s *crgetlabel(const cred_t *); -extern boolean_t crisremote(const cred_t *); - -/* - * Private interfaces for ephemeral uids. - */ -#define VALID_UID(id, zn) \ - ((id) <= MAXUID || valid_ephemeral_uid((zn), (id))) - -#define VALID_GID(id, zn) \ - ((id) <= MAXUID || valid_ephemeral_gid((zn), (id))) - -extern boolean_t valid_ephemeral_uid(struct zone *, uid_t); -extern boolean_t valid_ephemeral_gid(struct zone *, gid_t); - -extern int eph_uid_alloc(struct zone *, int, uid_t *, int); -extern int eph_gid_alloc(struct zone *, int, gid_t *, int); - -extern void crsetsid(cred_t *, struct ksid *, int); -extern void crsetsidlist(cred_t *, struct ksidlist *); - -extern struct ksidlist *crgetsidlist(const cred_t *); - -extern int crsetpriv(cred_t *, ...); - -extern struct credklpd *crgetcrklpd(const cred_t *); -extern void crsetcrklpd(cred_t *, struct credklpd *); +#define KUID_TO_SUID(x) (x) +#define KGID_TO_SGID(x) (x) +#define crgetuid(cr) ((cr)->cr_uid) +#define crgetruid(cr) ((cr)->cr_ruid) +#define crgetgid(cr) ((cr)->cr_gid) +#define crgetgroups(cr) ((cr)->cr_groups) +#define crgetngroups(cr) ((cr)->cr_ngroups) +#define crgetzoneid(cr) ((cr)->cr_prison->pr_id) #ifdef __cplusplus } diff --git a/include/os/freebsd/spl/sys/dkio.h b/include/os/freebsd/spl/sys/dkio.h index aed54ba50893..009886cb0763 100644 --- a/include/os/freebsd/spl/sys/dkio.h +++ b/include/os/freebsd/spl/sys/dkio.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/extdirent.h b/include/os/freebsd/spl/sys/extdirent.h index b22e8e8563a2..d6927ae40bbb 100644 --- a/include/os/freebsd/spl/sys/extdirent.h +++ b/include/os/freebsd/spl/sys/extdirent.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/idmap.h b/include/os/freebsd/spl/sys/idmap.h index 39eeb905c72b..bd2383794f18 100644 --- a/include/os/freebsd/spl/sys/idmap.h +++ b/include/os/freebsd/spl/sys/idmap.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/isa_defs.h b/include/os/freebsd/spl/sys/isa_defs.h index 399d510b5f9b..817521cc2cb9 100644 --- a/include/os/freebsd/spl/sys/isa_defs.h +++ b/include/os/freebsd/spl/sys/isa_defs.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/kidmap.h b/include/os/freebsd/spl/sys/kidmap.h deleted file mode 100644 index dc0cf5988a42..000000000000 --- a/include/os/freebsd/spl/sys/kidmap.h +++ /dev/null @@ -1,41 +0,0 @@ -/* - * Copyright (c) 2007 Pawel Jakub Dawidek - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS - * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT - * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY - * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF - * SUCH DAMAGE. - * - * $FreeBSD$ - */ - -#ifndef _OPENSOLARIS_SYS_KIDMAP_H_ -#define _OPENSOLARIS_SYS_KIDMAP_H_ - -#include - -typedef int32_t idmap_stat; -typedef void idmap_get_handle_t; - -#define kidmap_get_create() (NULL) -#define kidmap_get_destroy(hdl) do { } while (0) -#define kidmap_get_mappings(hdl) (NULL) - -#endif /* _OPENSOLARIS_SYS_KIDMAP_H_ */ diff --git a/include/os/freebsd/spl/sys/kmem.h b/include/os/freebsd/spl/sys/kmem.h index dc3b4f5d7877..a81cb1fb521d 100644 --- a/include/os/freebsd/spl/sys/kmem.h +++ b/include/os/freebsd/spl/sys/kmem.h @@ -73,7 +73,7 @@ extern uint64_t spl_kmem_cache_entry_size(kmem_cache_t *cache); void *zfs_kmem_alloc(size_t size, int kmflags); void zfs_kmem_free(void *buf, size_t size); uint64_t kmem_size(void); -kmem_cache_t *kmem_cache_create(char *name, size_t bufsize, size_t align, +kmem_cache_t *kmem_cache_create(const char *name, size_t bufsize, size_t align, int (*constructor)(void *, void *, int), void (*destructor)(void *, void *), void (*reclaim)(void *) __unused, void *private, vmem_t *vmp, int cflags); void kmem_cache_destroy(kmem_cache_t *cache); diff --git a/include/os/freebsd/spl/sys/list.h b/include/os/freebsd/spl/sys/list.h index 6db92ed42955..6c32402a7bd1 100644 --- a/include/os/freebsd/spl/sys/list.h +++ b/include/os/freebsd/spl/sys/list.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/list_impl.h b/include/os/freebsd/spl/sys/list_impl.h index a6614f9a38c2..09b70232e8ee 100644 --- a/include/os/freebsd/spl/sys/list_impl.h +++ b/include/os/freebsd/spl/sys/list_impl.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/misc.h b/include/os/freebsd/spl/sys/misc.h index 3481507d2c33..2e4efc60544a 100644 --- a/include/os/freebsd/spl/sys/misc.h +++ b/include/os/freebsd/spl/sys/misc.h @@ -52,8 +52,6 @@ struct opensolaris_utsname { char *machine; }; -extern char hw_serial[11]; - #define task_io_account_read(n) #define task_io_account_write(n) diff --git a/include/os/freebsd/spl/sys/mod_os.h b/include/os/freebsd/spl/sys/mod_os.h index 293bd7d2b983..3a9ebbfc3bc4 100644 --- a/include/os/freebsd/spl/sys/mod_os.h +++ b/include/os/freebsd/spl/sys/mod_os.h @@ -31,11 +31,6 @@ #include -#define ZFS_MODULE_DESCRIPTION(s) -#define ZFS_MODULE_AUTHOR(s) -#define ZFS_MODULE_LICENSE(s) -#define ZFS_MODULE_VERSION(s) - #define EXPORT_SYMBOL(x) #define module_param(a, b, c) #define MODULE_PARM_DESC(a, b) diff --git a/include/os/freebsd/spl/sys/processor.h b/include/os/freebsd/spl/sys/processor.h index 53149840f21f..40464e53d5c4 100644 --- a/include/os/freebsd/spl/sys/processor.h +++ b/include/os/freebsd/spl/sys/processor.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/procfs_list.h b/include/os/freebsd/spl/sys/procfs_list.h index 4bc603756ea4..4e98675053d2 100644 --- a/include/os/freebsd/spl/sys/procfs_list.h +++ b/include/os/freebsd/spl/sys/procfs_list.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/sid.h b/include/os/freebsd/spl/sys/sid.h index d3fab8b24744..f249d05d55a0 100644 --- a/include/os/freebsd/spl/sys/sid.h +++ b/include/os/freebsd/spl/sys/sid.h @@ -29,7 +29,6 @@ #ifndef _OPENSOLARIS_SYS_SID_H_ #define _OPENSOLARIS_SYS_SID_H_ #include -#include typedef struct ksiddomain { char *kd_name; /* Domain part of SID */ @@ -59,28 +58,4 @@ ksiddomain_rele(ksiddomain_t *kd) kmem_free(kd, sizeof (*kd)); } -static __inline uint_t -ksid_getid(ksid_t *ks) -{ - - panic("%s has been unexpectedly called", __func__); -} - -static __inline const char * -ksid_getdomain(ksid_t *ks) -{ - - panic("%s has been unexpectedly called", __func__); -} - -static __inline uint_t -ksid_getrid(ksid_t *ks) -{ - - panic("%s has been unexpectedly called", __func__); -} - -#define kidmap_getsidbyuid(zone, uid, sid_prefix, rid) (1) -#define kidmap_getsidbygid(zone, gid, sid_prefix, rid) (1) - #endif /* _OPENSOLARIS_SYS_SID_H_ */ diff --git a/include/os/freebsd/spl/sys/strings.h b/include/os/freebsd/spl/sys/strings.h deleted file mode 100644 index 651685d30473..000000000000 --- a/include/os/freebsd/spl/sys/strings.h +++ /dev/null @@ -1 +0,0 @@ -/* do not delete */ diff --git a/include/os/freebsd/spl/sys/sunddi.h b/include/os/freebsd/spl/sys/sunddi.h index bfbc3e10a1d2..28d40121d92e 100644 --- a/include/os/freebsd/spl/sys/sunddi.h +++ b/include/os/freebsd/spl/sys/sunddi.h @@ -48,7 +48,6 @@ typedef int ddi_devid_t; #define ddi_prop_free(x) (void)0 #define ddi_root_node() (void)0 -extern int ddi_strtoul(const char *, char **, int, unsigned long *); extern int ddi_strtol(const char *, char **, int, long *); extern int ddi_strtoull(const char *, char **, int, unsigned long long *); extern int ddi_strtoll(const char *, char **, int, long long *); diff --git a/include/os/freebsd/spl/sys/sysmacros.h b/include/os/freebsd/spl/sys/sysmacros.h index 7e3ab8915542..1c8475a7092c 100644 --- a/include/os/freebsd/spl/sys/sysmacros.h +++ b/include/os/freebsd/spl/sys/sysmacros.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/taskq.h b/include/os/freebsd/spl/sys/taskq.h index 3040549e043d..30579b391711 100644 --- a/include/os/freebsd/spl/sys/taskq.h +++ b/include/os/freebsd/spl/sys/taskq.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/types.h b/include/os/freebsd/spl/sys/types.h index b2897978c2d8..b1308df29503 100644 --- a/include/os/freebsd/spl/sys/types.h +++ b/include/os/freebsd/spl/sys/types.h @@ -78,9 +78,6 @@ typedef id_t ctid_t; typedef mode_t o_mode_t; typedef uint64_t pgcnt_t; -#define B_FALSE 0 -#define B_TRUE 1 - typedef short index_t; typedef off_t offset_t; #ifndef _PTRDIFF_T_DECLARED @@ -90,13 +87,17 @@ typedef __ptrdiff_t ptrdiff_t; /* pointer difference */ typedef int64_t rlim64_t; typedef int major_t; -#else #ifdef NEED_SOLARIS_BOOLEAN #if defined(__XOPEN_OR_POSIX) typedef enum { _B_FALSE, _B_TRUE } boolean_t; #else typedef enum { B_FALSE, B_TRUE } boolean_t; #endif /* defined(__XOPEN_OR_POSIX) */ +#else + +#define B_FALSE 0 +#define B_TRUE 1 + #endif typedef u_longlong_t u_offset_t; diff --git a/include/os/freebsd/spl/sys/uuid.h b/include/os/freebsd/spl/sys/uuid.h index 26d46e8d6214..9e76d07b8014 100644 --- a/include/os/freebsd/spl/sys/uuid.h +++ b/include/os/freebsd/spl/sys/uuid.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/vnode.h b/include/os/freebsd/spl/sys/vnode.h index d2c900854acb..e0f79a1116b2 100644 --- a/include/os/freebsd/spl/sys/vnode.h +++ b/include/os/freebsd/spl/sys/vnode.h @@ -136,17 +136,7 @@ vn_flush_cached_data(vnode_t *vp, boolean_t sync) #define va_blksize va_blocksize #define MAXOFFSET_T OFF_MAX -#define EXCL 0 -#define FCREAT O_CREAT -#define FTRUNC O_TRUNC -#define FEXCL O_EXCL -#ifndef FDSYNC -#define FDSYNC FFSYNC -#endif -#define FRSYNC FFSYNC -#define FSYNC FFSYNC -#define FOFFMAX 0x00 #define FIGNORECASE 0x00 /* diff --git a/include/os/freebsd/spl/sys/vnode_impl.h b/include/os/freebsd/spl/sys/vnode_impl.h index c82b1fc9ad18..3e698d7ac92a 100644 --- a/include/os/freebsd/spl/sys/vnode_impl.h +++ b/include/os/freebsd/spl/sys/vnode_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/spl/sys/zmod.h b/include/os/freebsd/spl/sys/zmod.h index c606b1db5f61..6e26a568bd44 100644 --- a/include/os/freebsd/spl/sys/zmod.h +++ b/include/os/freebsd/spl/sys/zmod.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/zfs/Makefile.am b/include/os/freebsd/zfs/Makefile.am deleted file mode 100644 index 081839c48c8f..000000000000 --- a/include/os/freebsd/zfs/Makefile.am +++ /dev/null @@ -1 +0,0 @@ -SUBDIRS = sys diff --git a/include/os/freebsd/zfs/sys/Makefile.am b/include/os/freebsd/zfs/sys/Makefile.am deleted file mode 100644 index 392bb4ae3477..000000000000 --- a/include/os/freebsd/zfs/sys/Makefile.am +++ /dev/null @@ -1,15 +0,0 @@ -KERNEL_H = \ - freebsd_crypto.h \ - sha2.h \ - vdev_os.h \ - zfs_bootenv_os.h \ - zfs_context_os.h \ - zfs_ctldir.h \ - zfs_dir.h \ - zfs_ioctl_compat.h \ - zfs_vfsops_os.h \ - zfs_vnops_os.h \ - zfs_znode_impl.h \ - zpl.h - -noinst_HEADERS = $(KERNEL_H) diff --git a/include/os/freebsd/zfs/sys/freebsd_crypto.h b/include/os/freebsd/zfs/sys/freebsd_crypto.h index a3ed4182656c..a61a6cd88c13 100644 --- a/include/os/freebsd/zfs/sys/freebsd_crypto.h +++ b/include/os/freebsd/zfs/sys/freebsd_crypto.h @@ -42,8 +42,6 @@ #define SUN_CKM_AES_GCM "CKM_AES_GCM" #define SUN_CKM_SHA512_HMAC "CKM_SHA512_HMAC" -#define CRYPTO_KEY_RAW 1 - #define CRYPTO_BITS2BYTES(n) ((n) == 0 ? 0 : (((n) - 1) >> 3) + 1) #define CRYPTO_BYTES2BITS(n) ((n) << 3) @@ -61,12 +59,11 @@ typedef struct freebsd_crypt_session { typedef void *crypto_mechanism_t; typedef void *crypto_ctx_template_t; /* - * Unlike the ICP crypto_key type, this only + * Like the ICP crypto_key type, this only * supports (the equivalent of - * CRYPTO_KEY_RAW). + * the former CRYPTO_KEY_RAW). */ typedef struct crypto_key { - int ck_format; /* Unused, but minimizes code diff */ void *ck_data; size_t ck_length; } crypto_key_t; diff --git a/include/os/freebsd/zfs/sys/sha2.h b/include/os/freebsd/zfs/sys/sha2.h index e3923e4ca37c..1f520eba0038 100644 --- a/include/os/freebsd/zfs/sys/sha2.h +++ b/include/os/freebsd/zfs/sys/sha2.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -33,9 +33,6 @@ extern "C" { #endif -#define SHA2_HMAC_MIN_KEY_LEN 1 /* SHA2-HMAC min key length in bytes */ -#define SHA2_HMAC_MAX_KEY_LEN INT_MAX /* SHA2-HMAC max key length in bytes */ - #define SHA256_DIGEST_LENGTH 32 /* SHA256 digest length in bytes */ #define SHA384_DIGEST_LENGTH 48 /* SHA384 digest length in bytes */ #define SHA512_DIGEST_LENGTH 64 /* SHA512 digest length in bytes */ diff --git a/include/os/freebsd/zfs/sys/vdev_os.h b/include/os/freebsd/zfs/sys/vdev_os.h index 59da954b90e6..ef1b01743b61 100644 --- a/include/os/freebsd/zfs/sys/vdev_os.h +++ b/include/os/freebsd/zfs/sys/vdev_os.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/zfs/sys/zfs_ctldir.h b/include/os/freebsd/zfs/sys/zfs_ctldir.h index da02863a78e6..14d75df33df0 100644 --- a/include/os/freebsd/zfs/sys/zfs_ctldir.h +++ b/include/os/freebsd/zfs/sys/zfs_ctldir.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/zfs/sys/zfs_dir.h b/include/os/freebsd/zfs/sys/zfs_dir.h index 4197e1188c9b..1980b646e38b 100644 --- a/include/os/freebsd/zfs/sys/zfs_dir.h +++ b/include/os/freebsd/zfs/sys/zfs_dir.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/zfs/sys/zfs_ioctl_compat.h b/include/os/freebsd/zfs/sys/zfs_ioctl_compat.h index d36a6d2ce7e2..79b30b5088c4 100644 --- a/include/os/freebsd/zfs/sys/zfs_ioctl_compat.h +++ b/include/os/freebsd/zfs/sys/zfs_ioctl_compat.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/zfs/sys/zfs_vfsops_os.h b/include/os/freebsd/zfs/sys/zfs_vfsops_os.h index ccbbf4f73224..c44f7c6f06b1 100644 --- a/include/os/freebsd/zfs/sys/zfs_vfsops_os.h +++ b/include/os/freebsd/zfs/sys/zfs_vfsops_os.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/freebsd/zfs/sys/zfs_znode_impl.h b/include/os/freebsd/zfs/sys/zfs_znode_impl.h index 096c9e16d347..f76a841472f2 100644 --- a/include/os/freebsd/zfs/sys/zfs_znode_impl.h +++ b/include/os/freebsd/zfs/sys/zfs_znode_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -121,25 +121,29 @@ typedef struct zfs_soft_state { #define zn_rlimit_fsize(zp, uio) \ vn_rlimit_fsize(ZTOV(zp), GET_UIO_STRUCT(uio), zfs_uio_td(uio)) +#define ZFS_ENTER_ERROR(zfsvfs, error) do { \ + ZFS_TEARDOWN_ENTER_READ((zfsvfs), FTAG); \ + if (__predict_false((zfsvfs)->z_unmounted)) { \ + ZFS_TEARDOWN_EXIT_READ(zfsvfs, FTAG); \ + return (error); \ + } \ +} while (0) + /* Called on entry to each ZFS vnode and vfs operation */ -#define ZFS_ENTER(zfsvfs) \ - { \ - ZFS_TEARDOWN_ENTER_READ((zfsvfs), FTAG); \ - if (__predict_false((zfsvfs)->z_unmounted)) { \ - ZFS_TEARDOWN_EXIT_READ(zfsvfs, FTAG); \ - return (EIO); \ - } \ - } +#define ZFS_ENTER(zfsvfs) ZFS_ENTER_ERROR(zfsvfs, EIO) /* Must be called before exiting the vop */ -#define ZFS_EXIT(zfsvfs) ZFS_TEARDOWN_EXIT_READ(zfsvfs, FTAG) +#define ZFS_EXIT(zfsvfs) ZFS_TEARDOWN_EXIT_READ(zfsvfs, FTAG) + +#define ZFS_VERIFY_ZP_ERROR(zp, error) do { \ + if (__predict_false((zp)->z_sa_hdl == NULL)) { \ + ZFS_EXIT((zp)->z_zfsvfs); \ + return (error); \ + } \ +} while (0) /* Verifies the znode is valid */ -#define ZFS_VERIFY_ZP(zp) \ - if (__predict_false((zp)->z_sa_hdl == NULL)) { \ - ZFS_EXIT((zp)->z_zfsvfs); \ - return (EIO); \ - } \ +#define ZFS_VERIFY_ZP(zp) ZFS_VERIFY_ZP_ERROR(zp, EIO) /* * Macros for dealing with dmu_buf_hold diff --git a/include/os/linux/Makefile.am b/include/os/linux/Makefile.am index 605a1fcb7506..4d6901c694c8 100644 --- a/include/os/linux/Makefile.am +++ b/include/os/linux/Makefile.am @@ -1 +1,111 @@ -SUBDIRS = kernel spl zfs +if CONFIG_KERNEL +kernel_linuxdir = $(kerneldir)/linux +kernel_linux_HEADERS = \ + %D%/kernel/linux/blkdev_compat.h \ + %D%/kernel/linux/compiler_compat.h \ + %D%/kernel/linux/dcache_compat.h \ + %D%/kernel/linux/kmap_compat.h \ + %D%/kernel/linux/mod_compat.h \ + %D%/kernel/linux/page_compat.h \ + %D%/kernel/linux/percpu_compat.h \ + %D%/kernel/linux/simd.h \ + %D%/kernel/linux/simd_aarch64.h \ + %D%/kernel/linux/simd_powerpc.h \ + %D%/kernel/linux/simd_x86.h \ + %D%/kernel/linux/utsname_compat.h \ + %D%/kernel/linux/vfs_compat.h \ + %D%/kernel/linux/xattr_compat.h + +kernel_sysdir = $(kerneldir)/sys +kernel_sys_HEADERS = \ + %D%/zfs/sys/policy.h \ + %D%/zfs/sys/sha2.h \ + %D%/zfs/sys/trace_acl.h \ + %D%/zfs/sys/trace_arc.h \ + %D%/zfs/sys/trace_common.h \ + %D%/zfs/sys/trace_dbgmsg.h \ + %D%/zfs/sys/trace_dbuf.h \ + %D%/zfs/sys/trace_dmu.h \ + %D%/zfs/sys/trace_dnode.h \ + %D%/zfs/sys/trace_multilist.h \ + %D%/zfs/sys/trace_rrwlock.h \ + %D%/zfs/sys/trace_txg.h \ + %D%/zfs/sys/trace_vdev.h \ + %D%/zfs/sys/trace_zfs.h \ + %D%/zfs/sys/trace_zil.h \ + %D%/zfs/sys/trace_zio.h \ + %D%/zfs/sys/trace_zrlock.h \ + %D%/zfs/sys/zfs_bootenv_os.h \ + %D%/zfs/sys/zfs_context_os.h \ + %D%/zfs/sys/zfs_ctldir.h \ + %D%/zfs/sys/zfs_dir.h \ + %D%/zfs/sys/zfs_vfsops_os.h \ + %D%/zfs/sys/zfs_vnops_os.h \ + %D%/zfs/sys/zfs_znode_impl.h \ + %D%/zfs/sys/zpl.h + +kernel_spl_rpcdir = $(kerneldir)/spl/rpc +kernel_spl_rpc_HEADERS = \ + %D%/spl/rpc/xdr.h + +kernel_spl_sysdir = $(kerneldir)/spl/sys +kernel_spl_sys_HEADERS = \ + %D%/spl/sys/acl.h \ + %D%/spl/sys/atomic.h \ + %D%/spl/sys/byteorder.h \ + %D%/spl/sys/callb.h \ + %D%/spl/sys/callo.h \ + %D%/spl/sys/cmn_err.h \ + %D%/spl/sys/condvar.h \ + %D%/spl/sys/cred.h \ + %D%/spl/sys/ctype.h \ + %D%/spl/sys/debug.h \ + %D%/spl/sys/disp.h \ + %D%/spl/sys/dkio.h \ + %D%/spl/sys/errno.h \ + %D%/spl/sys/fcntl.h \ + %D%/spl/sys/file.h \ + %D%/spl/sys/inttypes.h \ + %D%/spl/sys/isa_defs.h \ + %D%/spl/sys/kmem.h \ + %D%/spl/sys/kmem_cache.h \ + %D%/spl/sys/kstat.h \ + %D%/spl/sys/list.h \ + %D%/spl/sys/mod_os.h \ + %D%/spl/sys/mutex.h \ + %D%/spl/sys/param.h \ + %D%/spl/sys/proc.h \ + %D%/spl/sys/processor.h \ + %D%/spl/sys/procfs_list.h \ + %D%/spl/sys/random.h \ + %D%/spl/sys/rwlock.h \ + %D%/spl/sys/shrinker.h \ + %D%/spl/sys/sid.h \ + %D%/spl/sys/signal.h \ + %D%/spl/sys/simd.h \ + %D%/spl/sys/stat.h \ + %D%/spl/sys/string.h \ + %D%/spl/sys/sunddi.h \ + %D%/spl/sys/sysmacros.h \ + %D%/spl/sys/systeminfo.h \ + %D%/spl/sys/taskq.h \ + %D%/spl/sys/thread.h \ + %D%/spl/sys/time.h \ + %D%/spl/sys/timer.h \ + %D%/spl/sys/trace.h \ + %D%/spl/sys/trace_spl.h \ + %D%/spl/sys/trace_taskq.h \ + %D%/spl/sys/tsd.h \ + %D%/spl/sys/types.h \ + %D%/spl/sys/types32.h \ + %D%/spl/sys/uio.h \ + %D%/spl/sys/user.h \ + %D%/spl/sys/vfs.h \ + %D%/spl/sys/vmem.h \ + %D%/spl/sys/vmsystm.h \ + %D%/spl/sys/vnode.h \ + %D%/spl/sys/wait.h \ + %D%/spl/sys/wmsum.h \ + %D%/spl/sys/zmod.h \ + %D%/spl/sys/zone.h +endif diff --git a/include/os/linux/kernel/Makefile.am b/include/os/linux/kernel/Makefile.am deleted file mode 100644 index 08b2f5fc5c99..000000000000 --- a/include/os/linux/kernel/Makefile.am +++ /dev/null @@ -1 +0,0 @@ -SUBDIRS = linux diff --git a/include/os/linux/kernel/linux/Makefile.am b/include/os/linux/kernel/linux/Makefile.am deleted file mode 100644 index 6ff0df506d9c..000000000000 --- a/include/os/linux/kernel/linux/Makefile.am +++ /dev/null @@ -1,22 +0,0 @@ -KERNEL_H = \ - dcache_compat.h \ - xattr_compat.h \ - vfs_compat.h \ - blkdev_compat.h \ - utsname_compat.h \ - kmap_compat.h \ - percpu_compat.h \ - simd.h \ - simd_x86.h \ - simd_aarch64.h \ - simd_powerpc.h \ - mod_compat.h \ - page_compat.h \ - compiler_compat.h - -if CONFIG_KERNEL -if BUILD_LINUX -kerneldir = @prefix@/src/zfs-$(VERSION)/include/linux -kernel_HEADERS = $(KERNEL_H) -endif -endif diff --git a/include/os/linux/kernel/linux/blkdev_compat.h b/include/os/linux/kernel/linux/blkdev_compat.h index 9fa8884bb7a1..cdcea166903d 100644 --- a/include/os/linux/kernel/linux/blkdev_compat.h +++ b/include/os/linux/kernel/linux/blkdev_compat.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -34,6 +34,11 @@ #include #include #include /* for SECTOR_* */ +#include + +#ifdef HAVE_BLK_MQ +#include +#endif #ifndef HAVE_BLK_QUEUE_FLAG_SET static inline void @@ -495,21 +500,45 @@ blk_queue_discard_granularity(struct request_queue *q, unsigned int dg) } /* + * 5.19 API, + * bdev_max_discard_sectors() + * + * 2.6.32 API, + * blk_queue_discard() + */ +static inline boolean_t +bdev_discard_supported(struct block_device *bdev) +{ +#if defined(HAVE_BDEV_MAX_DISCARD_SECTORS) + return (!!bdev_max_discard_sectors(bdev)); +#elif defined(HAVE_BLK_QUEUE_DISCARD) + return (!!blk_queue_discard(bdev_get_queue(bdev))); +#else +#error "Unsupported kernel" +#endif +} + +/* + * 5.19 API, + * bdev_max_secure_erase_sectors() + * * 4.8 API, * blk_queue_secure_erase() * * 2.6.36 - 4.7 API, * blk_queue_secdiscard() */ -static inline int -blk_queue_discard_secure(struct request_queue *q) +static inline boolean_t +bdev_secure_discard_supported(struct block_device *bdev) { -#if defined(HAVE_BLK_QUEUE_SECURE_ERASE) - return (blk_queue_secure_erase(q)); +#if defined(HAVE_BDEV_MAX_SECURE_ERASE_SECTORS) + return (!!bdev_max_secure_erase_sectors(bdev)); +#elif defined(HAVE_BLK_QUEUE_SECURE_ERASE) + return (!!blk_queue_secure_erase(bdev_get_queue(bdev))); #elif defined(HAVE_BLK_QUEUE_SECDISCARD) - return (blk_queue_secdiscard(q)); + return (!!blk_queue_secdiscard(bdev_get_queue(bdev))); #else - return (0); +#error "Unsupported kernel" #endif } @@ -527,7 +556,10 @@ blk_generic_start_io_acct(struct request_queue *q __attribute__((unused)), struct gendisk *disk __attribute__((unused)), int rw __attribute__((unused)), struct bio *bio) { -#if defined(HAVE_DISK_IO_ACCT) +#if defined(HAVE_BDEV_IO_ACCT) + return (bdev_start_io_acct(bio->bi_bdev, bio_sectors(bio), + bio_op(bio), jiffies)); +#elif defined(HAVE_DISK_IO_ACCT) return (disk_start_io_acct(disk, bio_sectors(bio), bio_op(bio))); #elif defined(HAVE_BIO_IO_ACCT) return (bio_start_io_acct(bio)); @@ -550,7 +582,9 @@ blk_generic_end_io_acct(struct request_queue *q __attribute__((unused)), struct gendisk *disk __attribute__((unused)), int rw __attribute__((unused)), struct bio *bio, unsigned long start_time) { -#if defined(HAVE_DISK_IO_ACCT) +#if defined(HAVE_BDEV_IO_ACCT) + bdev_end_io_acct(bio->bi_bdev, bio_op(bio), start_time); +#elif defined(HAVE_DISK_IO_ACCT) disk_end_io_acct(disk, bio_op(bio), start_time); #elif defined(HAVE_BIO_IO_ACCT) bio_end_io_acct(bio, start_time); @@ -579,4 +613,110 @@ blk_generic_alloc_queue(make_request_fn make_request, int node_id) } #endif /* !HAVE_SUBMIT_BIO_IN_BLOCK_DEVICE_OPERATIONS */ +/* + * All the io_*() helper functions below can operate on a bio, or a rq, but + * not both. The older submit_bio() codepath will pass a bio, and the + * newer blk-mq codepath will pass a rq. + */ +static inline int +io_data_dir(struct bio *bio, struct request *rq) +{ +#ifdef HAVE_BLK_MQ + if (rq != NULL) { + if (op_is_write(req_op(rq))) { + return (WRITE); + } else { + return (READ); + } + } +#else + ASSERT3P(rq, ==, NULL); +#endif + return (bio_data_dir(bio)); +} + +static inline int +io_is_flush(struct bio *bio, struct request *rq) +{ +#ifdef HAVE_BLK_MQ + if (rq != NULL) + return (req_op(rq) == REQ_OP_FLUSH); +#else + ASSERT3P(rq, ==, NULL); +#endif + return (bio_is_flush(bio)); +} + +static inline int +io_is_discard(struct bio *bio, struct request *rq) +{ +#ifdef HAVE_BLK_MQ + if (rq != NULL) + return (req_op(rq) == REQ_OP_DISCARD); +#else + ASSERT3P(rq, ==, NULL); +#endif + return (bio_is_discard(bio)); +} + +static inline int +io_is_secure_erase(struct bio *bio, struct request *rq) +{ +#ifdef HAVE_BLK_MQ + if (rq != NULL) + return (req_op(rq) == REQ_OP_SECURE_ERASE); +#else + ASSERT3P(rq, ==, NULL); +#endif + return (bio_is_secure_erase(bio)); +} + +static inline int +io_is_fua(struct bio *bio, struct request *rq) +{ +#ifdef HAVE_BLK_MQ + if (rq != NULL) + return (rq->cmd_flags & REQ_FUA); +#else + ASSERT3P(rq, ==, NULL); +#endif + return (bio_is_fua(bio)); +} + + +static inline uint64_t +io_offset(struct bio *bio, struct request *rq) +{ +#ifdef HAVE_BLK_MQ + if (rq != NULL) + return (blk_rq_pos(rq) << 9); +#else + ASSERT3P(rq, ==, NULL); +#endif + return (BIO_BI_SECTOR(bio) << 9); +} + +static inline uint64_t +io_size(struct bio *bio, struct request *rq) +{ +#ifdef HAVE_BLK_MQ + if (rq != NULL) + return (blk_rq_bytes(rq)); +#else + ASSERT3P(rq, ==, NULL); +#endif + return (BIO_BI_SIZE(bio)); +} + +static inline int +io_has_data(struct bio *bio, struct request *rq) +{ +#ifdef HAVE_BLK_MQ + if (rq != NULL) + return (bio_has_data(rq->bio)); +#else + ASSERT3P(rq, ==, NULL); +#endif + return (bio_has_data(bio)); +} #endif /* _ZFS_BLKDEV_H */ diff --git a/include/os/linux/kernel/linux/compiler_compat.h b/include/os/linux/kernel/linux/compiler_compat.h index a65689642361..98f7d7608043 100644 --- a/include/os/linux/kernel/linux/compiler_compat.h +++ b/include/os/linux/kernel/linux/compiler_compat.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/kernel/linux/dcache_compat.h b/include/os/linux/kernel/linux/dcache_compat.h index d0588a82e9ad..0fbd92458679 100644 --- a/include/os/linux/kernel/linux/dcache_compat.h +++ b/include/os/linux/kernel/linux/dcache_compat.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/kernel/linux/kmap_compat.h b/include/os/linux/kernel/linux/kmap_compat.h index 42f463ab9ae9..7f9c00af802b 100644 --- a/include/os/linux/kernel/linux/kmap_compat.h +++ b/include/os/linux/kernel/linux/kmap_compat.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/kernel/linux/mod_compat.h b/include/os/linux/kernel/linux/mod_compat.h index a90bdf7cf2d2..a091bbfe179d 100644 --- a/include/os/linux/kernel/linux/mod_compat.h +++ b/include/os/linux/kernel/linux/mod_compat.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -160,11 +160,4 @@ enum scope_prefix_types { #define ZFS_MODULE_PARAM_ARGS const char *buf, zfs_kernel_param_t *kp -#define ZFS_MODULE_DESCRIPTION(s) MODULE_DESCRIPTION(s) -#define ZFS_MODULE_AUTHOR(s) MODULE_AUTHOR(s) -#define ZFS_MODULE_LICENSE(s) MODULE_LICENSE(s) -#define ZFS_MODULE_VERSION(s) MODULE_VERSION(s) - -#define module_init_early(fn) module_init(fn) - #endif /* _MOD_COMPAT_H */ diff --git a/include/os/linux/kernel/linux/percpu_compat.h b/include/os/linux/kernel/linux/percpu_compat.h index e7a4242c466c..bf3a5a01c27e 100644 --- a/include/os/linux/kernel/linux/percpu_compat.h +++ b/include/os/linux/kernel/linux/percpu_compat.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/kernel/linux/simd.h b/include/os/linux/kernel/linux/simd.h index 4cde248e200c..b83c536883be 100644 --- a/include/os/linux/kernel/linux/simd.h +++ b/include/os/linux/kernel/linux/simd.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/kernel/linux/simd_aarch64.h b/include/os/linux/kernel/linux/simd_aarch64.h index 50937e97ced1..d56a093d4ec2 100644 --- a/include/os/linux/kernel/linux/simd_aarch64.h +++ b/include/os/linux/kernel/linux/simd_aarch64.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/kernel/linux/simd_powerpc.h b/include/os/linux/kernel/linux/simd_powerpc.h index 108cef22f56f..764c5dc51f95 100644 --- a/include/os/linux/kernel/linux/simd_powerpc.h +++ b/include/os/linux/kernel/linux/simd_powerpc.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -57,25 +57,45 @@ #include #include -#define kfpu_allowed() 1 -#define kfpu_begin() \ - { \ - preempt_disable(); \ - enable_kernel_altivec(); \ - } +#define kfpu_allowed() 1 + #if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 5, 0) #define kfpu_end() \ { \ + disable_kernel_vsx(); \ disable_kernel_altivec(); \ preempt_enable(); \ } +#define kfpu_begin() \ + { \ + preempt_disable(); \ + enable_kernel_altivec(); \ + enable_kernel_vsx(); \ + } #else -/* seems that before 4.5 no-one bothered disabling ... */ +/* seems that before 4.5 no-one bothered */ +#define kfpu_begin() #define kfpu_end() preempt_enable() #endif #define kfpu_init() 0 #define kfpu_fini() ((void) 0) +static inline boolean_t +zfs_vsx_available(void) +{ + boolean_t res; +#if defined(__powerpc64__) + u64 msr; +#else + u32 msr; +#endif + kfpu_begin(); + __asm volatile("mfmsr %0" : "=r"(msr)); + res = (msr & 0x800000) != 0; + kfpu_end(); + return (res); +} + /* * Check if AltiVec instruction set is available */ diff --git a/include/os/linux/kernel/linux/simd_x86.h b/include/os/linux/kernel/linux/simd_x86.h index 6d4c7a09fe82..2f6c3165ac7a 100644 --- a/include/os/linux/kernel/linux/simd_x86.h +++ b/include/os/linux/kernel/linux/simd_x86.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -85,23 +85,21 @@ #undef CONFIG_X86_DEBUG_FPU #endif +/* + * The following cases are for kernels which export either the + * kernel_fpu_* or __kernel_fpu_* functions. + */ +#if defined(KERNEL_EXPORTS_X86_FPU) + #if defined(HAVE_KERNEL_FPU_API_HEADER) #include +#if defined(HAVE_KERNEL_FPU_INTERNAL_HEADER) #include -#if defined(HAVE_KERNEL_FPU_XCR_HEADER) -#include #endif #else #include -#include #endif -/* - * The following cases are for kernels which export either the - * kernel_fpu_* or __kernel_fpu_* functions. - */ -#if defined(KERNEL_EXPORTS_X86_FPU) - #define kfpu_allowed() 1 #define kfpu_init() 0 #define kfpu_fini() ((void) 0) @@ -136,29 +134,74 @@ * When the kernel_fpu_* symbols are unavailable then provide our own * versions which allow the FPU to be safely used. */ -#if defined(HAVE_KERNEL_FPU_INTERNAL) || defined(HAVE_KERNEL_FPU_XSAVE_INTERNAL) - -#if defined(HAVE_KERNEL_FPU_XSAVE_INTERNAL) -/* - * Some sanity checks. - * HAVE_KERNEL_FPU_INTERNAL and HAVE_KERNEL_FPU_XSAVE_INTERNAL are exclusive. - */ #if defined(HAVE_KERNEL_FPU_INTERNAL) -#error "HAVE_KERNEL_FPU_INTERNAL and HAVE_KERNEL_FPU_XSAVE_INTERNAL defined" -#endif + /* - * For kernels >= 5.16 we have to use inline assembly with the XSAVE{,OPT,S} - * instructions, so we need the toolchain to support at least XSAVE. + * For kernels not exporting *kfpu_{begin,end} we have to use inline assembly + * with the XSAVE{,OPT,S} instructions, so we need the toolchain to support at + * least XSAVE. */ #if !defined(HAVE_XSAVE) #error "Toolchain needs to support the XSAVE assembler instruction" #endif -#endif #include #include -extern union fpregs_state **zfs_kfpu_fpregs; +extern uint8_t **zfs_kfpu_fpregs; + +/* + * Return the size in bytes required by the XSAVE instruction for an + * XSAVE area containing all the user state components supported by this CPU. + * See: Intel 64 and IA-32 Architectures Software Developer’s Manual. + * Dec. 2021. Vol. 2A p. 3-222. + */ +static inline uint32_t +get_xsave_area_size(void) +{ + if (!boot_cpu_has(X86_FEATURE_OSXSAVE)) { + return (0); + } + /* + * Call CPUID with leaf 13 and subleaf 0. The size is in ecx. + * We don't need to check for cpuid_max here, since if this CPU has + * OSXSAVE set, it has leaf 13 (0x0D) as well. + */ + uint32_t eax, ebx, ecx, edx; + + eax = 13U; + ecx = 0U; + __asm__ __volatile__("cpuid" + : "=a" (eax), "=b" (ebx), "=c" (ecx), "=d" (edx) + : "a" (eax), "c" (ecx)); + + return (ecx); +} + +/* + * Return the allocation order of the maximum buffer size required to save the + * FPU state on this architecture. The value returned is the same as Linux' + * get_order() function would return (i.e. 2^order = nr. of pages required). + * Currently this will always return 0 since the save area is below 4k even for + * a full fledged AVX-512 implementation. + */ +static inline int +get_fpuregs_save_area_order(void) +{ + size_t area_size = (size_t)get_xsave_area_size(); + + /* + * If we are dealing with a CPU not supporting XSAVE, + * get_xsave_area_size() will return 0. Thus the maximum memory + * required is the FXSAVE area size which is 512 bytes. See: Intel 64 + * and IA-32 Architectures Software Developer’s Manual. Dec. 2021. + * Vol. 2A p. 3-451. + */ + if (area_size == 0) { + area_size = 512; + } + return (get_order(area_size)); +} /* * Initialize per-cpu variables to store FPU state. @@ -167,11 +210,11 @@ static inline void kfpu_fini(void) { int cpu; + int order = get_fpuregs_save_area_order(); for_each_possible_cpu(cpu) { if (zfs_kfpu_fpregs[cpu] != NULL) { - free_pages((unsigned long)zfs_kfpu_fpregs[cpu], - get_order(sizeof (union fpregs_state))); + free_pages((unsigned long)zfs_kfpu_fpregs[cpu], order); } } @@ -181,8 +224,9 @@ kfpu_fini(void) static inline int kfpu_init(void) { - zfs_kfpu_fpregs = kzalloc(num_possible_cpus() * - sizeof (union fpregs_state *), GFP_KERNEL); + zfs_kfpu_fpregs = kzalloc(num_possible_cpus() * sizeof (uint8_t *), + GFP_KERNEL); + if (zfs_kfpu_fpregs == NULL) return (-ENOMEM); @@ -191,8 +235,8 @@ kfpu_init(void) * the target memory. Since kmalloc() provides no alignment * guarantee instead use alloc_pages_node(). */ - unsigned int order = get_order(sizeof (union fpregs_state)); int cpu; + int order = get_fpuregs_save_area_order(); for_each_possible_cpu(cpu) { struct page *page = alloc_pages_node(cpu_to_node(cpu), @@ -209,9 +253,6 @@ kfpu_init(void) } #define kfpu_allowed() 1 -#if defined(HAVE_KERNEL_FPU_INTERNAL) -#define ex_handler_fprestore ex_handler_default -#endif /* * FPU save and restore instructions. @@ -226,21 +267,6 @@ kfpu_init(void) #define kfpu_fxsr_clean(rval) __asm("fnclex; emms; fildl %P[addr]" \ : : [addr] "m" (rval)); -#if defined(HAVE_KERNEL_FPU_INTERNAL) -static inline void -kfpu_save_xsave(struct xregs_state *addr, uint64_t mask) -{ - uint32_t low, hi; - int err; - - low = mask; - hi = mask >> 32; - XSTATE_XSAVE(addr, low, hi, err); - WARN_ON_ONCE(err); -} -#endif /* defined(HAVE_KERNEL_FPU_INTERNAL) */ - -#if defined(HAVE_KERNEL_FPU_XSAVE_INTERNAL) #define kfpu_do_xsave(instruction, addr, mask) \ { \ uint32_t low, hi; \ @@ -252,10 +278,9 @@ kfpu_save_xsave(struct xregs_state *addr, uint64_t mask) : [dst] "m" (*(addr)), "a" (low), "d" (hi) \ : "memory"); \ } -#endif /* defined(HAVE_KERNEL_FPU_XSAVE_INTERNAL) */ static inline void -kfpu_save_fxsr(struct fxregs_state *addr) +kfpu_save_fxsr(uint8_t *addr) { if (IS_ENABLED(CONFIG_X86_32)) kfpu_fxsave(addr); @@ -264,40 +289,11 @@ kfpu_save_fxsr(struct fxregs_state *addr) } static inline void -kfpu_save_fsave(struct fregs_state *addr) +kfpu_save_fsave(uint8_t *addr) { kfpu_fnsave(addr); } -#if defined(HAVE_KERNEL_FPU_INTERNAL) -static inline void -kfpu_begin(void) -{ - /* - * Preemption and interrupts must be disabled for the critical - * region where the FPU state is being modified. - */ - preempt_disable(); - local_irq_disable(); - - /* - * The current FPU registers need to be preserved by kfpu_begin() - * and restored by kfpu_end(). They are stored in a dedicated - * per-cpu variable, not in the task struct, this allows any user - * FPU state to be correctly preserved and restored. - */ - union fpregs_state *state = zfs_kfpu_fpregs[smp_processor_id()]; - if (static_cpu_has(X86_FEATURE_XSAVE)) { - kfpu_save_xsave(&state->xsave, ~0); - } else if (static_cpu_has(X86_FEATURE_FXSR)) { - kfpu_save_fxsr(&state->fxsave); - } else { - kfpu_save_fsave(&state->fsave); - } -} -#endif /* defined(HAVE_KERNEL_FPU_INTERNAL) */ - -#if defined(HAVE_KERNEL_FPU_XSAVE_INTERNAL) static inline void kfpu_begin(void) { @@ -314,42 +310,28 @@ kfpu_begin(void) * per-cpu variable, not in the task struct, this allows any user * FPU state to be correctly preserved and restored. */ - union fpregs_state *state = zfs_kfpu_fpregs[smp_processor_id()]; + uint8_t *state = zfs_kfpu_fpregs[smp_processor_id()]; #if defined(HAVE_XSAVES) if (static_cpu_has(X86_FEATURE_XSAVES)) { - kfpu_do_xsave("xsaves", &state->xsave, ~0); + kfpu_do_xsave("xsaves", state, ~0); return; } #endif #if defined(HAVE_XSAVEOPT) if (static_cpu_has(X86_FEATURE_XSAVEOPT)) { - kfpu_do_xsave("xsaveopt", &state->xsave, ~0); + kfpu_do_xsave("xsaveopt", state, ~0); return; } #endif if (static_cpu_has(X86_FEATURE_XSAVE)) { - kfpu_do_xsave("xsave", &state->xsave, ~0); + kfpu_do_xsave("xsave", state, ~0); } else if (static_cpu_has(X86_FEATURE_FXSR)) { - kfpu_save_fxsr(&state->fxsave); + kfpu_save_fxsr(state); } else { - kfpu_save_fsave(&state->fsave); + kfpu_save_fsave(state); } } -#endif /* defined(HAVE_KERNEL_FPU_XSAVE_INTERNAL) */ -#if defined(HAVE_KERNEL_FPU_INTERNAL) -static inline void -kfpu_restore_xsave(struct xregs_state *addr, uint64_t mask) -{ - uint32_t low, hi; - - low = mask; - hi = mask >> 32; - XSTATE_XRESTORE(addr, low, hi); -} -#endif /* defined(HAVE_KERNEL_FPU_INTERNAL) */ - -#if defined(HAVE_KERNEL_FPU_XSAVE_INTERNAL) #define kfpu_do_xrstor(instruction, addr, mask) \ { \ uint32_t low, hi; \ @@ -361,10 +343,9 @@ kfpu_restore_xsave(struct xregs_state *addr, uint64_t mask) : [src] "m" (*(addr)), "a" (low), "d" (hi) \ : "memory"); \ } -#endif /* defined(HAVE_KERNEL_FPU_XSAVE_INTERNAL) */ static inline void -kfpu_restore_fxsr(struct fxregs_state *addr) +kfpu_restore_fxsr(uint8_t *addr) { /* * On AuthenticAMD K7 and K8 processors the fxrstor instruction only @@ -382,67 +363,40 @@ kfpu_restore_fxsr(struct fxregs_state *addr) } static inline void -kfpu_restore_fsave(struct fregs_state *addr) +kfpu_restore_fsave(uint8_t *addr) { kfpu_frstor(addr); } -#if defined(HAVE_KERNEL_FPU_INTERNAL) static inline void kfpu_end(void) { - union fpregs_state *state = zfs_kfpu_fpregs[smp_processor_id()]; - - if (static_cpu_has(X86_FEATURE_XSAVE)) { - kfpu_restore_xsave(&state->xsave, ~0); - } else if (static_cpu_has(X86_FEATURE_FXSR)) { - kfpu_restore_fxsr(&state->fxsave); - } else { - kfpu_restore_fsave(&state->fsave); - } - - local_irq_enable(); - preempt_enable(); -} -#endif /* defined(HAVE_KERNEL_FPU_INTERNAL) */ - -#if defined(HAVE_KERNEL_FPU_XSAVE_INTERNAL) -static inline void -kfpu_end(void) -{ - union fpregs_state *state = zfs_kfpu_fpregs[smp_processor_id()]; + uint8_t *state = zfs_kfpu_fpregs[smp_processor_id()]; #if defined(HAVE_XSAVES) if (static_cpu_has(X86_FEATURE_XSAVES)) { - kfpu_do_xrstor("xrstors", &state->xsave, ~0); + kfpu_do_xrstor("xrstors", state, ~0); goto out; } #endif if (static_cpu_has(X86_FEATURE_XSAVE)) { - kfpu_do_xrstor("xrstor", &state->xsave, ~0); + kfpu_do_xrstor("xrstor", state, ~0); } else if (static_cpu_has(X86_FEATURE_FXSR)) { - kfpu_save_fxsr(&state->fxsave); + kfpu_restore_fxsr(state); } else { - kfpu_save_fsave(&state->fsave); + kfpu_restore_fsave(state); } out: local_irq_enable(); preempt_enable(); } -#endif /* defined(HAVE_KERNEL_FPU_XSAVE_INTERNAL) */ #else -/* - * FPU support is unavailable. - */ -#define kfpu_allowed() 0 -#define kfpu_begin() do {} while (0) -#define kfpu_end() do {} while (0) -#define kfpu_init() 0 -#define kfpu_fini() ((void) 0) +#error "Exactly one of KERNEL_EXPORTS_X86_FPU or HAVE_KERNEL_FPU_INTERNAL" \ + " must be defined" -#endif /* defined(HAVE_KERNEL_FPU_INTERNAL || HAVE_KERNEL_FPU_XSAVE_INTERNAL) */ +#endif /* defined(HAVE_KERNEL_FPU_INTERNAL */ #endif /* defined(KERNEL_EXPORTS_X86_FPU) */ /* @@ -452,6 +406,25 @@ kfpu_end(void) /* * Detect register set support */ + +/* + * Check if OS supports AVX and AVX2 by checking XCR0 + * Only call this function if CPUID indicates that AVX feature is + * supported by the CPU, otherwise it might be an illegal instruction. + */ +static inline uint64_t +zfs_xgetbv(uint32_t index) +{ + uint32_t eax, edx; + /* xgetbv - instruction byte code */ + __asm__ __volatile__(".byte 0x0f; .byte 0x01; .byte 0xd0" + : "=a" (eax), "=d" (edx) + : "c" (index)); + + return ((((uint64_t)edx)<<32) | (uint64_t)eax); +} + + static inline boolean_t __simd_state_enabled(const uint64_t state) { @@ -466,7 +439,7 @@ __simd_state_enabled(const uint64_t state) if (!has_osxsave) return (B_FALSE); - xcr0 = xgetbv(0); + xcr0 = zfs_xgetbv(0); return ((xcr0 & state) == state); } diff --git a/include/os/linux/kernel/linux/utsname_compat.h b/include/os/linux/kernel/linux/utsname_compat.h index 88da45cf504e..4a027bde8788 100644 --- a/include/os/linux/kernel/linux/utsname_compat.h +++ b/include/os/linux/kernel/linux/utsname_compat.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/kernel/linux/vfs_compat.h b/include/os/linux/kernel/linux/vfs_compat.h index 91e908598fbb..eeed0a388ce4 100644 --- a/include/os/linux/kernel/linux/vfs_compat.h +++ b/include/os/linux/kernel/linux/vfs_compat.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -296,11 +296,7 @@ static inline struct dentry *file_dentry(const struct file *f) static inline uid_t zfs_uid_read_impl(struct inode *ip) { -#ifdef HAVE_SUPER_USER_NS - return (from_kuid(ip->i_sb->s_user_ns, ip->i_uid)); -#else return (from_kuid(kcred->user_ns, ip->i_uid)); -#endif } static inline uid_t zfs_uid_read(struct inode *ip) @@ -310,11 +306,7 @@ static inline uid_t zfs_uid_read(struct inode *ip) static inline gid_t zfs_gid_read_impl(struct inode *ip) { -#ifdef HAVE_SUPER_USER_NS - return (from_kgid(ip->i_sb->s_user_ns, ip->i_gid)); -#else return (from_kgid(kcred->user_ns, ip->i_gid)); -#endif } static inline gid_t zfs_gid_read(struct inode *ip) @@ -324,20 +316,12 @@ static inline gid_t zfs_gid_read(struct inode *ip) static inline void zfs_uid_write(struct inode *ip, uid_t uid) { -#ifdef HAVE_SUPER_USER_NS - ip->i_uid = make_kuid(ip->i_sb->s_user_ns, uid); -#else ip->i_uid = make_kuid(kcred->user_ns, uid); -#endif } static inline void zfs_gid_write(struct inode *ip, gid_t gid) { -#ifdef HAVE_SUPER_USER_NS - ip->i_gid = make_kgid(ip->i_sb->s_user_ns, gid); -#else ip->i_gid = make_kgid(kcred->user_ns, gid); -#endif } /* diff --git a/include/os/linux/kernel/linux/xattr_compat.h b/include/os/linux/kernel/linux/xattr_compat.h index 54690727eab9..21c88dd07719 100644 --- a/include/os/linux/kernel/linux/xattr_compat.h +++ b/include/os/linux/kernel/linux/xattr_compat.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/spl/Makefile.am b/include/os/linux/spl/Makefile.am deleted file mode 100644 index bd781c08f143..000000000000 --- a/include/os/linux/spl/Makefile.am +++ /dev/null @@ -1 +0,0 @@ -SUBDIRS = rpc sys diff --git a/include/os/linux/spl/rpc/Makefile.am b/include/os/linux/spl/rpc/Makefile.am deleted file mode 100644 index 13d804fce9b6..000000000000 --- a/include/os/linux/spl/rpc/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -KERNEL_H = \ - xdr.h - -if CONFIG_KERNEL -kerneldir = @prefix@/src/zfs-$(VERSION)/include/spl/rpc -kernel_HEADERS = $(KERNEL_H) -endif diff --git a/include/os/linux/spl/sys/Makefile.am b/include/os/linux/spl/sys/Makefile.am deleted file mode 100644 index 48c27f970fc9..000000000000 --- a/include/os/linux/spl/sys/Makefile.am +++ /dev/null @@ -1,64 +0,0 @@ -KERNEL_H = \ - acl.h \ - atomic.h \ - byteorder.h \ - callb.h \ - callo.h \ - cmn_err.h \ - condvar.h \ - cred.h \ - ctype.h \ - debug.h \ - disp.h \ - dkio.h \ - errno.h \ - fcntl.h \ - file.h \ - inttypes.h \ - isa_defs.h \ - kmem_cache.h \ - kmem.h \ - kstat.h \ - list.h \ - mod_os.h \ - mutex.h \ - param.h \ - processor.h \ - proc.h \ - procfs_list.h \ - random.h \ - rwlock.h \ - shrinker.h \ - sid.h \ - signal.h \ - simd.h \ - stat.h \ - strings.h \ - sunddi.h \ - sysmacros.h \ - systeminfo.h \ - taskq.h \ - thread.h \ - time.h \ - timer.h \ - trace.h \ - trace_spl.h \ - trace_taskq.h \ - tsd.h \ - types32.h \ - types.h \ - uio.h \ - user.h \ - vfs.h \ - vmem.h \ - vmsystm.h \ - vnode.h \ - wait.h \ - wmsum.h \ - zmod.h \ - zone.h - -if CONFIG_KERNEL -kerneldir = @prefix@/src/zfs-$(VERSION)/include/spl/sys -kernel_HEADERS = $(KERNEL_H) -endif diff --git a/include/os/linux/spl/sys/cred.h b/include/os/linux/spl/sys/cred.h index 9cc85deb5c8a..b7d3f38d70bb 100644 --- a/include/os/linux/spl/sys/cred.h +++ b/include/os/linux/spl/sys/cred.h @@ -49,12 +49,7 @@ extern void crhold(cred_t *cr); extern void crfree(cred_t *cr); extern uid_t crgetuid(const cred_t *cr); extern uid_t crgetruid(const cred_t *cr); -extern uid_t crgetsuid(const cred_t *cr); -extern uid_t crgetfsuid(const cred_t *cr); extern gid_t crgetgid(const cred_t *cr); -extern gid_t crgetrgid(const cred_t *cr); -extern gid_t crgetsgid(const cred_t *cr); -extern gid_t crgetfsgid(const cred_t *cr); extern int crgetngroups(const cred_t *cr); extern gid_t *crgetgroups(const cred_t *cr); extern int groupmember(gid_t gid, const cred_t *cr); diff --git a/include/os/linux/spl/sys/errno.h b/include/os/linux/spl/sys/errno.h index f6d9212a613f..347b879a50c1 100644 --- a/include/os/linux/spl/sys/errno.h +++ b/include/os/linux/spl/sys/errno.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/spl/sys/isa_defs.h b/include/os/linux/spl/sys/isa_defs.h index 2207ee20256c..a032aae91658 100644 --- a/include/os/linux/spl/sys/isa_defs.h +++ b/include/os/linux/spl/sys/isa_defs.h @@ -95,7 +95,7 @@ #define _ALIGNMENT_REQUIRED 1 /* arm arch specific defines */ -#elif defined(__arm) || defined(__arm__) || defined(__aarch64__) +#elif defined(__arm) || defined(__arm__) #if !defined(__arm) #define __arm @@ -105,17 +105,11 @@ #define __arm__ #endif -#if defined(__aarch64__) -#if !defined(_LP64) -#define _LP64 -#endif -#else #if !defined(_ILP32) #define _ILP32 #endif -#endif -#if defined(__ARMEL__) || defined(__AARCH64EL__) +#if defined(__ARMEL__) #define _ZFS_LITTLE_ENDIAN #else #define _ZFS_BIG_ENDIAN @@ -127,6 +121,19 @@ */ #define _ALIGNMENT_REQUIRED 1 +/* aarch64 arch specific defines */ +#elif defined(__aarch64__) + +#if !defined(_LP64) +#define _LP64 +#endif + +#if defined(__AARCH64EL__) +#define _ZFS_LITTLE_ENDIAN +#else +#define _ZFS_BIG_ENDIAN +#endif + /* sparc arch specific defines */ #elif defined(__sparc) || defined(__sparc__) diff --git a/include/os/linux/spl/sys/kmem_cache.h b/include/os/linux/spl/sys/kmem_cache.h index 48006ec5d27e..bd0ad5052d3d 100644 --- a/include/os/linux/spl/sys/kmem_cache.h +++ b/include/os/linux/spl/sys/kmem_cache.h @@ -183,7 +183,7 @@ typedef struct spl_kmem_cache { } spl_kmem_cache_t; #define kmem_cache_t spl_kmem_cache_t -extern spl_kmem_cache_t *spl_kmem_cache_create(char *name, size_t size, +extern spl_kmem_cache_t *spl_kmem_cache_create(const char *name, size_t size, size_t align, spl_kmem_ctor_t ctor, spl_kmem_dtor_t dtor, void *reclaim, void *priv, void *vmp, int flags); extern void spl_kmem_cache_set_move(spl_kmem_cache_t *, diff --git a/include/os/linux/spl/sys/procfs_list.h b/include/os/linux/spl/sys/procfs_list.h index 9bb437f55cf7..959d9444e63d 100644 --- a/include/os/linux/spl/sys/procfs_list.h +++ b/include/os/linux/spl/sys/procfs_list.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/spl/sys/string.h b/include/os/linux/spl/sys/string.h new file mode 100644 index 000000000000..38134dcf4c76 --- /dev/null +++ b/include/os/linux/spl/sys/string.h @@ -0,0 +1 @@ +#include diff --git a/include/os/linux/spl/sys/strings.h b/include/os/linux/spl/sys/strings.h deleted file mode 100644 index 48e417d14605..000000000000 --- a/include/os/linux/spl/sys/strings.h +++ /dev/null @@ -1,30 +0,0 @@ -/* - * Copyright (C) 2018 Lawrence Livermore National Security, LLC. - * Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER). - * UCRL-CODE-235197 - * - * This file is part of the SPL, Solaris Porting Layer. - * - * The SPL is free software; you can redistribute it and/or modify it - * under the terms of the GNU General Public License as published by the - * Free Software Foundation; either version 2 of the License, or (at your - * option) any later version. - * - * The SPL is distributed in the hope that it will be useful, but WITHOUT - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or - * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License - * for more details. - * - * You should have received a copy of the GNU General Public License along - * with the SPL. If not, see . - */ -#ifndef _SPL_SYS_STRINGS_H -#define _SPL_SYS_STRINGS_H - -#include - -#define bzero(ptr, size) memset(ptr, 0, size) -#define bcopy(src, dest, size) memmove(dest, src, size) -#define bcmp(src, dest, size) memcmp((src), (dest), (size_t)(size)) - -#endif /* _SPL_SYS_STRINGS_H */ diff --git a/include/os/linux/spl/sys/sunddi.h b/include/os/linux/spl/sys/sunddi.h index 8524ec9c300e..df6a204d5cf9 100644 --- a/include/os/linux/spl/sys/sunddi.h +++ b/include/os/linux/spl/sys/sunddi.h @@ -46,7 +46,6 @@ typedef int ddi_devid_t; #define ddi_prop_free(x) (void)0 #define ddi_root_node() (void)0 -extern int ddi_strtoul(const char *, char **, int, unsigned long *); extern int ddi_strtol(const char *, char **, int, long *); extern int ddi_strtoull(const char *, char **, int, unsigned long long *); extern int ddi_strtoll(const char *, char **, int, long long *); diff --git a/include/os/linux/spl/sys/taskq.h b/include/os/linux/spl/sys/taskq.h index 5f8eb19dceb7..2a6cd8283d16 100644 --- a/include/os/linux/spl/sys/taskq.h +++ b/include/os/linux/spl/sys/taskq.h @@ -163,8 +163,6 @@ extern taskq_t *taskq_of_curthread(void); ((void) sizeof (dc), \ taskq_create(name, nthreads, maxclsyspri, min, max, flags)) -extern boolean_t taskq_empty(taskq_t *); - int spl_taskq_init(void); void spl_taskq_fini(void); diff --git a/include/os/linux/spl/sys/thread.h b/include/os/linux/spl/sys/thread.h index 6248866a9225..bc88ff4efb67 100644 --- a/include/os/linux/spl/sys/thread.h +++ b/include/os/linux/spl/sys/thread.h @@ -53,7 +53,7 @@ typedef void (*thread_func_t)(void *); __thread_create(stk, stksize, (thread_func_t)func, #func, \ arg, len, pp, state, pri) -#define thread_exit() __thread_exit() +#define thread_exit() spl_thread_exit() #define thread_join(t) VERIFY(0) #define curthread current #define getcomm() current->comm @@ -62,10 +62,16 @@ typedef void (*thread_func_t)(void *); extern kthread_t *__thread_create(caddr_t stk, size_t stksize, thread_func_t func, const char *name, void *args, size_t len, proc_t *pp, int state, pri_t pri); -extern void __thread_exit(void); extern struct task_struct *spl_kthread_create(int (*func)(void *), void *data, const char namefmt[], ...); +static inline __attribute__((noreturn)) void +spl_thread_exit(void) +{ + tsd_exit(); + SPL_KTHREAD_COMPLETE_AND_EXIT(NULL, 0); +} + extern proc_t p0; #ifdef HAVE_SIGINFO diff --git a/include/os/linux/spl/sys/trace.h b/include/os/linux/spl/sys/trace.h index b148ace6abd3..21ac0e4f84de 100644 --- a/include/os/linux/spl/sys/trace.h +++ b/include/os/linux/spl/sys/trace.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/spl/sys/trace_spl.h b/include/os/linux/spl/sys/trace_spl.h index bffd91d9129b..6f0f7b271d56 100644 --- a/include/os/linux/spl/sys/trace_spl.h +++ b/include/os/linux/spl/sys/trace_spl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/spl/sys/trace_taskq.h b/include/os/linux/spl/sys/trace_taskq.h index 002e1af1706d..0e25e642f279 100644 --- a/include/os/linux/spl/sys/trace_taskq.h +++ b/include/os/linux/spl/sys/trace_taskq.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/spl/sys/uio.h b/include/os/linux/spl/sys/uio.h index 439eec986236..fe2b5c07a018 100644 --- a/include/os/linux/spl/sys/uio.h +++ b/include/os/linux/spl/sys/uio.h @@ -69,9 +69,20 @@ typedef struct zfs_uio { uint16_t uio_fmode; uint16_t uio_extflg; ssize_t uio_resid; + size_t uio_skip; + + struct request *rq; + + /* + * Used for saving rq_for_each_segment() state between calls + * to zfs_uiomove_bvec_rq(). + */ + struct req_iterator iter; + struct bio_vec bv; } zfs_uio_t; + #define zfs_uio_segflg(u) (u)->uio_segflg #define zfs_uio_offset(u) (u)->uio_loffset #define zfs_uio_resid(u) (u)->uio_resid @@ -116,17 +127,33 @@ zfs_uio_iovec_init(zfs_uio_t *uio, const struct iovec *iov, } static inline void -zfs_uio_bvec_init(zfs_uio_t *uio, struct bio *bio) +zfs_uio_bvec_init(zfs_uio_t *uio, struct bio *bio, struct request *rq) { - uio->uio_bvec = &bio->bi_io_vec[BIO_BI_IDX(bio)]; - uio->uio_iovcnt = bio->bi_vcnt - BIO_BI_IDX(bio); - uio->uio_loffset = BIO_BI_SECTOR(bio) << 9; + /* Either bio or rq will be set, but not both */ + ASSERT3P(uio, !=, bio); + + if (bio) { + uio->uio_iovcnt = bio->bi_vcnt - BIO_BI_IDX(bio); + uio->uio_bvec = &bio->bi_io_vec[BIO_BI_IDX(bio)]; + } else { + uio->uio_bvec = NULL; + uio->uio_iovcnt = 0; + memset(&uio->iter, 0, sizeof (uio->iter)); + } + + uio->uio_loffset = io_offset(bio, rq); uio->uio_segflg = UIO_BVEC; uio->uio_fault_disable = B_FALSE; uio->uio_fmode = 0; uio->uio_extflg = 0; - uio->uio_resid = BIO_BI_SIZE(bio); - uio->uio_skip = BIO_BI_SKIP(bio); + uio->uio_resid = io_size(bio, rq); + if (bio) { + uio->uio_skip = BIO_BI_SKIP(bio); + } else { + uio->uio_skip = 0; + } + + uio->rq = rq; } #if defined(HAVE_VFS_IOV_ITER) diff --git a/include/os/linux/spl/sys/zone.h b/include/os/linux/spl/sys/zone.h index 00e30f690c38..5978a6285fa1 100644 --- a/include/os/linux/spl/sys/zone.h +++ b/include/os/linux/spl/sys/zone.h @@ -25,11 +25,34 @@ #define _SPL_ZONE_H #include +#include -#define GLOBAL_ZONEID 0 +#include +#include -#define zone_dataset_visible(x, y) (1) -#define crgetzoneid(x) (GLOBAL_ZONEID) -#define INGLOBALZONE(z) (1) +/* + * Attach the given dataset to the given user namespace. + */ +extern int zone_dataset_attach(cred_t *, const char *, int); + +/* + * Detach the given dataset from the given user namespace. + */ +extern int zone_dataset_detach(cred_t *, const char *, int); + +/* + * Returns true if the named pool/dataset is visible in the current zone. + */ +extern int zone_dataset_visible(const char *dataset, int *write); + +int spl_zone_init(void); +void spl_zone_fini(void); + +extern unsigned int crgetzoneid(const cred_t *); +extern unsigned int global_zoneid(void); +extern boolean_t inglobalzone(proc_t *); + +#define INGLOBALZONE(x) inglobalzone(x) +#define GLOBAL_ZONEID global_zoneid() #endif /* SPL_ZONE_H */ diff --git a/include/os/linux/zfs/Makefile.am b/include/os/linux/zfs/Makefile.am deleted file mode 100644 index 081839c48c8f..000000000000 --- a/include/os/linux/zfs/Makefile.am +++ /dev/null @@ -1 +0,0 @@ -SUBDIRS = sys diff --git a/include/os/linux/zfs/sys/Makefile.am b/include/os/linux/zfs/sys/Makefile.am deleted file mode 100644 index a075db476e40..000000000000 --- a/include/os/linux/zfs/sys/Makefile.am +++ /dev/null @@ -1,31 +0,0 @@ -KERNEL_H = \ - policy.h \ - sha2.h \ - trace_acl.h \ - trace_arc.h \ - trace_common.h \ - trace_zfs.h \ - trace_dbgmsg.h \ - trace_dbuf.h \ - trace_dmu.h \ - trace_dnode.h \ - trace_multilist.h \ - trace_rrwlock.h \ - trace_txg.h \ - trace_vdev.h \ - trace_zil.h \ - trace_zio.h \ - trace_zrlock.h \ - zfs_bootenv_os.h \ - zfs_context_os.h \ - zfs_ctldir.h \ - zfs_dir.h \ - zfs_vfsops_os.h \ - zfs_vnops_os.h \ - zfs_znode_impl.h \ - zpl.h - -if CONFIG_KERNEL -kerneldir = @prefix@/src/zfs-$(VERSION)/include/sys -kernel_HEADERS = $(KERNEL_H) -endif diff --git a/include/os/linux/zfs/sys/policy.h b/include/os/linux/zfs/sys/policy.h index 61afc3765504..3bd7ce36b85d 100644 --- a/include/os/linux/zfs/sys/policy.h +++ b/include/os/linux/zfs/sys/policy.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/sha2.h b/include/os/linux/zfs/sys/sha2.h index 4dd966b6cab3..ef37139dd4da 100644 --- a/include/os/linux/zfs/sys/sha2.h +++ b/include/os/linux/zfs/sys/sha2.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -33,9 +33,6 @@ extern "C" { #endif -#define SHA2_HMAC_MIN_KEY_LEN 1 /* SHA2-HMAC min key length in bytes */ -#define SHA2_HMAC_MAX_KEY_LEN INT_MAX /* SHA2-HMAC max key length in bytes */ - #define SHA256_DIGEST_LENGTH 32 /* SHA256 digest length in bytes */ #define SHA384_DIGEST_LENGTH 48 /* SHA384 digest length in bytes */ #define SHA512_DIGEST_LENGTH 64 /* SHA512 digest length in bytes */ diff --git a/include/os/linux/zfs/sys/trace_acl.h b/include/os/linux/zfs/sys/trace_acl.h index f8e0aa8acaf8..6a73545fecda 100644 --- a/include/os/linux/zfs/sys/trace_acl.h +++ b/include/os/linux/zfs/sys/trace_acl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -58,6 +58,8 @@ DECLARE_EVENT_CLASS(zfs_ace_class, __field(uint64_t, z_size) __field(uint64_t, z_pflags) __field(uint32_t, z_sync_cnt) + __field(uint32_t, z_sync_writes_cnt) + __field(uint32_t, z_async_writes_cnt) __field(mode_t, z_mode) __field(boolean_t, z_is_sa) __field(boolean_t, z_is_mapped) @@ -91,6 +93,8 @@ DECLARE_EVENT_CLASS(zfs_ace_class, __entry->z_size = zn->z_size; __entry->z_pflags = zn->z_pflags; __entry->z_sync_cnt = zn->z_sync_cnt; + __entry->z_sync_writes_cnt = zn->z_sync_writes_cnt; + __entry->z_async_writes_cnt = zn->z_async_writes_cnt; __entry->z_mode = zn->z_mode; __entry->z_is_sa = zn->z_is_sa; __entry->z_is_mapped = zn->z_is_mapped; @@ -116,16 +120,18 @@ DECLARE_EVENT_CLASS(zfs_ace_class, TP_printk("zn { id %llu unlinked %u atime_dirty %u " "zn_prefetch %u blksz %u seq %u " "mapcnt %llu size %llu pflags %llu " - "sync_cnt %u mode 0x%x is_sa %d " - "is_mapped %d is_ctldir %d is_stale %d inode { " + "sync_cnt %u sync_writes_cnt %u async_writes_cnt %u " + "mode 0x%x is_sa %d is_mapped %d " + "is_ctldir %d is_stale %d inode { " "uid %u gid %u ino %lu nlink %u size %lli " "blkbits %u bytes %u mode 0x%x generation %x } } " "ace { type %u flags %u access_mask %u } mask_matched %u", __entry->z_id, __entry->z_unlinked, __entry->z_atime_dirty, __entry->z_zn_prefetch, __entry->z_blksz, __entry->z_seq, __entry->z_mapcnt, __entry->z_size, - __entry->z_pflags, __entry->z_sync_cnt, __entry->z_mode, - __entry->z_is_sa, __entry->z_is_mapped, + __entry->z_pflags, __entry->z_sync_cnt, + __entry->z_sync_writes_cnt, __entry->z_async_writes_cnt, + __entry->z_mode, __entry->z_is_sa, __entry->z_is_mapped, __entry->z_is_ctldir, __entry->z_is_stale, __entry->i_uid, __entry->i_gid, __entry->i_ino, __entry->i_nlink, __entry->i_size, __entry->i_blkbits, diff --git a/include/os/linux/zfs/sys/trace_arc.h b/include/os/linux/zfs/sys/trace_arc.h index a318a62b05a4..d8e73337622c 100644 --- a/include/os/linux/zfs/sys/trace_arc.h +++ b/include/os/linux/zfs/sys/trace_arc.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/trace_common.h b/include/os/linux/zfs/sys/trace_common.h index 6922d1a1810a..74d77c1c4b21 100644 --- a/include/os/linux/zfs/sys/trace_common.h +++ b/include/os/linux/zfs/sys/trace_common.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/trace_dbgmsg.h b/include/os/linux/zfs/sys/trace_dbgmsg.h index 89722bab4c94..19f533baaaaf 100644 --- a/include/os/linux/zfs/sys/trace_dbgmsg.h +++ b/include/os/linux/zfs/sys/trace_dbgmsg.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/trace_dbuf.h b/include/os/linux/zfs/sys/trace_dbuf.h index ba7c6b1a0144..11d25be35bc4 100644 --- a/include/os/linux/zfs/sys/trace_dbuf.h +++ b/include/os/linux/zfs/sys/trace_dbuf.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/trace_dmu.h b/include/os/linux/zfs/sys/trace_dmu.h index 40a4c02eed19..b031d4a2c901 100644 --- a/include/os/linux/zfs/sys/trace_dmu.h +++ b/include/os/linux/zfs/sys/trace_dmu.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/trace_dnode.h b/include/os/linux/zfs/sys/trace_dnode.h index b4b391492bc3..2697e871b85f 100644 --- a/include/os/linux/zfs/sys/trace_dnode.h +++ b/include/os/linux/zfs/sys/trace_dnode.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/trace_multilist.h b/include/os/linux/zfs/sys/trace_multilist.h index 638578407a0c..33b9aeac9213 100644 --- a/include/os/linux/zfs/sys/trace_multilist.h +++ b/include/os/linux/zfs/sys/trace_multilist.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/trace_rrwlock.h b/include/os/linux/zfs/sys/trace_rrwlock.h index 4c74d6257309..dbc94fee0136 100644 --- a/include/os/linux/zfs/sys/trace_rrwlock.h +++ b/include/os/linux/zfs/sys/trace_rrwlock.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/trace_txg.h b/include/os/linux/zfs/sys/trace_txg.h index f77e35a78340..c95220b1b5d6 100644 --- a/include/os/linux/zfs/sys/trace_txg.h +++ b/include/os/linux/zfs/sys/trace_txg.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/trace_vdev.h b/include/os/linux/zfs/sys/trace_vdev.h index 079668d35f12..6a3f6c202527 100644 --- a/include/os/linux/zfs/sys/trace_vdev.h +++ b/include/os/linux/zfs/sys/trace_vdev.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/trace_zfs.h b/include/os/linux/zfs/sys/trace_zfs.h index 0e19f8d186d0..5f82e1c093a6 100644 --- a/include/os/linux/zfs/sys/trace_zfs.h +++ b/include/os/linux/zfs/sys/trace_zfs.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/trace_zil.h b/include/os/linux/zfs/sys/trace_zil.h index 8ffda452766c..6dd18c5974b9 100644 --- a/include/os/linux/zfs/sys/trace_zil.h +++ b/include/os/linux/zfs/sys/trace_zil.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/trace_zio.h b/include/os/linux/zfs/sys/trace_zio.h index 8655e245c043..223eb559ea7e 100644 --- a/include/os/linux/zfs/sys/trace_zio.h +++ b/include/os/linux/zfs/sys/trace_zio.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/trace_zrlock.h b/include/os/linux/zfs/sys/trace_zrlock.h index b504e66f9f7b..6c797c6e231c 100644 --- a/include/os/linux/zfs/sys/trace_zrlock.h +++ b/include/os/linux/zfs/sys/trace_zrlock.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/zfs_context_os.h b/include/os/linux/zfs/sys/zfs_context_os.h index 9e4260558285..04a5f0c0d239 100644 --- a/include/os/linux/zfs/sys/zfs_context_os.h +++ b/include/os/linux/zfs/sys/zfs_context_os.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -32,4 +32,9 @@ #define HAVE_LARGE_STACKS 1 #endif +#if defined(CONFIG_UML) +#undef setjmp +#undef longjmp +#endif + #endif diff --git a/include/os/linux/zfs/sys/zfs_ctldir.h b/include/os/linux/zfs/sys/zfs_ctldir.h index beee34979b64..ad16ab5e4444 100644 --- a/include/os/linux/zfs/sys/zfs_ctldir.h +++ b/include/os/linux/zfs/sys/zfs_ctldir.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/zfs_dir.h b/include/os/linux/zfs/sys/zfs_dir.h index 0f15e43452b2..91d873ea4b7f 100644 --- a/include/os/linux/zfs/sys/zfs_dir.h +++ b/include/os/linux/zfs/sys/zfs_dir.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/zfs_vfsops_os.h b/include/os/linux/zfs/sys/zfs_vfsops_os.h index 8e03ae99a7fd..697ae2018ec4 100644 --- a/include/os/linux/zfs/sys/zfs_vfsops_os.h +++ b/include/os/linux/zfs/sys/zfs_vfsops_os.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/zfs_vnops_os.h b/include/os/linux/zfs/sys/zfs_vnops_os.h index 47f91e4a6cf4..22ca625023b0 100644 --- a/include/os/linux/zfs/sys/zfs_vnops_os.h +++ b/include/os/linux/zfs/sys/zfs_vnops_os.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -70,7 +70,7 @@ extern int zfs_space(znode_t *zp, int cmd, flock64_t *bfp, int flag, extern int zfs_fid(struct inode *ip, fid_t *fidp); extern int zfs_getpage(struct inode *ip, struct page *pl[], int nr_pages); extern int zfs_putpage(struct inode *ip, struct page *pp, - struct writeback_control *wbc); + struct writeback_control *wbc, boolean_t for_sync); extern int zfs_dirty_inode(struct inode *ip, int flags); extern int zfs_map(struct inode *ip, offset_t off, caddr_t *addrp, size_t len, unsigned long vm_flags); diff --git a/include/os/linux/zfs/sys/zfs_znode_impl.h b/include/os/linux/zfs/sys/zfs_znode_impl.h index 2e1bd857614d..a6fa06a3f1a8 100644 --- a/include/os/linux/zfs/sys/zfs_znode_impl.h +++ b/include/os/linux/zfs/sys/zfs_znode_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/os/linux/zfs/sys/zpl.h b/include/os/linux/zfs/sys/zpl.h index afb16e5c7907..95f08f5416d0 100644 --- a/include/os/linux/zfs/sys/zpl.h +++ b/include/os/linux/zfs/sys/zpl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/Makefile.am b/include/sys/Makefile.am deleted file mode 100644 index 54573fbe1b1c..000000000000 --- a/include/sys/Makefile.am +++ /dev/null @@ -1,151 +0,0 @@ -SUBDIRS = fm fs crypto lua sysevent zstd - -COMMON_H = \ - abd.h \ - abd_impl.h \ - aggsum.h \ - arc.h \ - arc_impl.h \ - avl.h \ - avl_impl.h \ - bitops.h \ - blkptr.h \ - bplist.h \ - bpobj.h \ - bptree.h \ - btree.h \ - bqueue.h \ - dataset_kstats.h \ - dbuf.h \ - ddt.h \ - dmu.h \ - dmu_impl.h \ - dmu_objset.h \ - dmu_recv.h \ - dmu_redact.h \ - dmu_send.h \ - dmu_traverse.h \ - dmu_tx.h \ - dmu_zfetch.h \ - dnode.h \ - dsl_bookmark.h \ - dsl_dataset.h \ - dsl_deadlist.h \ - dsl_deleg.h \ - dsl_destroy.h \ - dsl_dir.h \ - dsl_crypt.h \ - dsl_pool.h \ - dsl_prop.h \ - dsl_scan.h \ - dsl_synctask.h \ - dsl_userhold.h \ - edonr.h \ - efi_partition.h \ - frame.h \ - hkdf.h \ - metaslab.h \ - metaslab_impl.h \ - mmp.h \ - mntent.h \ - mod.h \ - multilist.h \ - nvpair.h \ - nvpair_impl.h \ - objlist.h \ - pathname.h \ - qat.h \ - range_tree.h \ - rrwlock.h \ - sa.h \ - sa_impl.h \ - skein.h \ - spa_boot.h \ - spa_checkpoint.h \ - spa_log_spacemap.h \ - space_map.h \ - space_reftree.h \ - spa.h \ - spa_impl.h \ - spa_checksum.h \ - sysevent.h \ - txg.h \ - txg_impl.h \ - u8_textprep_data.h \ - u8_textprep.h \ - uberblock.h \ - uberblock_impl.h \ - uio_impl.h \ - unique.h \ - uuid.h \ - vdev_disk.h \ - vdev_file.h \ - vdev.h \ - vdev_draid.h \ - vdev_impl.h \ - vdev_indirect_births.h \ - vdev_indirect_mapping.h \ - vdev_initialize.h \ - vdev_raidz.h \ - vdev_raidz_impl.h \ - vdev_rebuild.h \ - vdev_removal.h \ - vdev_trim.h \ - xvattr.h \ - zap.h \ - zap_impl.h \ - zap_leaf.h \ - zcp.h \ - zcp_global.h \ - zcp_iter.h \ - zcp_prop.h \ - zcp_set.h \ - zfeature.h \ - zfs_acl.h \ - zfs_bootenv.h \ - zfs_context.h \ - zfs_debug.h \ - zfs_delay.h \ - zfs_file.h \ - zfs_fuid.h \ - zfs_project.h \ - zfs_quota.h \ - zfs_racct.h \ - zfs_ratelimit.h \ - zfs_refcount.h \ - zfs_rlock.h \ - zfs_sa.h \ - zfs_stat.h \ - zfs_sysfs.h \ - zfs_vfsops.h \ - zfs_vnops.h \ - zfs_znode.h \ - zil.h \ - zil_impl.h \ - zio_checksum.h \ - zio_compress.h \ - zio_crypt.h \ - zio.h \ - zio_impl.h \ - zio_priority.h \ - zrlock.h \ - zthr.h - -KERNEL_H = \ - zfs_ioctl.h \ - zfs_ioctl_impl.h \ - zfs_onexit.h \ - zvol.h \ - zvol_impl.h - -if CONFIG_USER -libzfsdir = $(includedir)/libzfs/sys -libzfs_HEADERS = $(COMMON_H) -endif - -if CONFIG_KERNEL -if BUILD_LINUX -kerneldir = @prefix@/src/zfs-$(VERSION)/include/sys -kernel_HEADERS = $(COMMON_H) $(KERNEL_H) -endif -endif diff --git a/include/sys/abd.h b/include/sys/abd.h index 5c6bd0c271d4..82c51cb05cbc 100644 --- a/include/sys/abd.h +++ b/include/sys/abd.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/abd_impl.h b/include/sys/abd_impl.h index e96f1edfc8ce..40546d4af137 100644 --- a/include/sys/abd_impl.h +++ b/include/sys/abd_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/arc.h b/include/sys/arc.h index 4f9bfe48d7e3..fc8605122434 100644 --- a/include/sys/arc.h +++ b/include/sys/arc.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -264,12 +264,12 @@ int arc_untransform(arc_buf_t *buf, spa_t *spa, const zbookmark_phys_t *zb, void arc_convert_to_raw(arc_buf_t *buf, uint64_t dsobj, boolean_t byteorder, dmu_object_type_t ot, const uint8_t *salt, const uint8_t *iv, const uint8_t *mac); -arc_buf_t *arc_alloc_buf(spa_t *spa, void *tag, arc_buf_contents_t type, +arc_buf_t *arc_alloc_buf(spa_t *spa, const void *tag, arc_buf_contents_t type, int32_t size); -arc_buf_t *arc_alloc_compressed_buf(spa_t *spa, void *tag, +arc_buf_t *arc_alloc_compressed_buf(spa_t *spa, const void *tag, uint64_t psize, uint64_t lsize, enum zio_compress compression_type, uint8_t complevel); -arc_buf_t *arc_alloc_raw_buf(spa_t *spa, void *tag, uint64_t dsobj, +arc_buf_t *arc_alloc_raw_buf(spa_t *spa, const void *tag, uint64_t dsobj, boolean_t byteorder, const uint8_t *salt, const uint8_t *iv, const uint8_t *mac, dmu_object_type_t ot, uint64_t psize, uint64_t lsize, enum zio_compress compression_type, uint8_t complevel); @@ -281,14 +281,14 @@ arc_buf_t *arc_loan_raw_buf(spa_t *spa, uint64_t dsobj, boolean_t byteorder, const uint8_t *salt, const uint8_t *iv, const uint8_t *mac, dmu_object_type_t ot, uint64_t psize, uint64_t lsize, enum zio_compress compression_type, uint8_t complevel); -void arc_return_buf(arc_buf_t *buf, void *tag); -void arc_loan_inuse_buf(arc_buf_t *buf, void *tag); -void arc_buf_destroy(arc_buf_t *buf, void *tag); +void arc_return_buf(arc_buf_t *buf, const void *tag); +void arc_loan_inuse_buf(arc_buf_t *buf, const void *tag); +void arc_buf_destroy(arc_buf_t *buf, const void *tag); void arc_buf_info(arc_buf_t *buf, arc_buf_info_t *abi, int state_index); uint64_t arc_buf_size(arc_buf_t *buf); uint64_t arc_buf_lsize(arc_buf_t *buf); void arc_buf_access(arc_buf_t *buf); -void arc_release(arc_buf_t *buf, void *tag); +void arc_release(arc_buf_t *buf, const void *tag); int arc_released(arc_buf_t *buf); void arc_buf_sigsegv(int sig, siginfo_t *si, void *unused); void arc_buf_freeze(arc_buf_t *buf); diff --git a/include/sys/arc_impl.h b/include/sys/arc_impl.h index 755e87fe6e0e..4e2cff7e2ed2 100644 --- a/include/sys/arc_impl.h +++ b/include/sys/arc_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/avl.h b/include/sys/avl.h index 20e88f2a6b06..8818e3edb292 100644 --- a/include/sys/avl.h +++ b/include/sys/avl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/avl_impl.h b/include/sys/avl_impl.h index c464a62a1ca6..85277b42b471 100644 --- a/include/sys/avl_impl.h +++ b/include/sys/avl_impl.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/bitops.h b/include/sys/bitops.h index 69d07d76552a..5c477b38b205 100644 --- a/include/sys/bitops.h +++ b/include/sys/bitops.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/blake3.h b/include/sys/blake3.h new file mode 100644 index 000000000000..19500585f382 --- /dev/null +++ b/include/sys/blake3.h @@ -0,0 +1,125 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Based on BLAKE3 v1.3.1, https://github.com/BLAKE3-team/BLAKE3 + * Copyright (c) 2019-2020 Samuel Neves and Jack O'Connor + * Copyright (c) 2021 Tino Reichardt + */ + +#ifndef BLAKE3_H +#define BLAKE3_H + +#ifdef _KERNEL +#include +#else +#include +#include +#endif + +#ifdef __cplusplus +extern "C" { +#endif + +#define BLAKE3_KEY_LEN 32 +#define BLAKE3_OUT_LEN 32 +#define BLAKE3_MAX_DEPTH 54 +#define BLAKE3_BLOCK_LEN 64 +#define BLAKE3_CHUNK_LEN 1024 + +/* + * This struct is a private implementation detail. + * It has to be here because it's part of BLAKE3_CTX below. + */ +typedef struct { + uint32_t cv[8]; + uint64_t chunk_counter; + uint8_t buf[BLAKE3_BLOCK_LEN]; + uint8_t buf_len; + uint8_t blocks_compressed; + uint8_t flags; +} blake3_chunk_state_t; + +typedef struct { + uint32_t key[8]; + blake3_chunk_state_t chunk; + uint8_t cv_stack_len; + + /* + * The stack size is MAX_DEPTH + 1 because we do lazy merging. For + * example, with 7 chunks, we have 3 entries in the stack. Adding an + * 8th chunk requires a 4th entry, rather than merging everything down + * to 1, because we don't know whether more input is coming. This is + * different from how the reference implementation does things. + */ + uint8_t cv_stack[(BLAKE3_MAX_DEPTH + 1) * BLAKE3_OUT_LEN]; + + /* const blake3_impl_ops_t *ops */ + const void *ops; +} BLAKE3_CTX; + +/* init the context for hash operation */ +void Blake3_Init(BLAKE3_CTX *ctx); + +/* init the context for a MAC and/or tree hash operation */ +void Blake3_InitKeyed(BLAKE3_CTX *ctx, const uint8_t key[BLAKE3_KEY_LEN]); + +/* process the input bytes */ +void Blake3_Update(BLAKE3_CTX *ctx, const void *input, size_t input_len); + +/* finalize the hash computation and output the result */ +void Blake3_Final(const BLAKE3_CTX *ctx, uint8_t *out); + +/* finalize the hash computation and output the result */ +void Blake3_FinalSeek(const BLAKE3_CTX *ctx, uint64_t seek, uint8_t *out, + size_t out_len); + +/* these are pre-allocated contexts */ +extern void **blake3_per_cpu_ctx; +extern void blake3_per_cpu_ctx_init(void); +extern void blake3_per_cpu_ctx_fini(void); + +/* return number of supported implementations */ +extern int blake3_get_impl_count(void); + +/* return id of selected implementation */ +extern int blake3_get_impl_id(void); + +/* return name of selected implementation */ +extern const char *blake3_get_impl_name(void); + +/* setup id as fastest implementation */ +extern void blake3_set_impl_fastest(uint32_t id); + +/* set implementation by id */ +extern void blake3_set_impl_id(uint32_t id); + +/* set implementation by name */ +extern int blake3_set_impl_name(const char *name); + +/* set startup implementation */ +extern void blake3_setup_impl(void); + +#ifdef __cplusplus +} +#endif + +#endif /* BLAKE3_H */ diff --git a/include/sys/bplist.h b/include/sys/bplist.h index f8deaf8437e6..53dd346767fe 100644 --- a/include/sys/bplist.h +++ b/include/sys/bplist.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/bpobj.h b/include/sys/bpobj.h index 16e403526cff..84f0ee76c44e 100644 --- a/include/sys/bpobj.h +++ b/include/sys/bpobj.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/bptree.h b/include/sys/bptree.h index 327c128bf493..9d189446ab69 100644 --- a/include/sys/bptree.h +++ b/include/sys/bptree.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/btree.h b/include/sys/btree.h index 3b53476c7c68..a901d654ef1c 100644 --- a/include/sys/btree.h +++ b/include/sys/btree.h @@ -72,7 +72,11 @@ extern kmem_cache_t *zfs_btree_leaf_cache; typedef struct zfs_btree_hdr { struct zfs_btree_core *bth_parent; - boolean_t bth_core; + /* + * Set to -1 to indicate core nodes. Other values represent first + * valid element offset for leaf nodes. + */ + uint32_t bth_first; /* * For both leaf and core nodes, represents the number of elements in * the node. For core nodes, they will have bth_count + 1 children. @@ -91,9 +95,12 @@ typedef struct zfs_btree_leaf { uint8_t btl_elems[]; } zfs_btree_leaf_t; +#define BTREE_LEAF_ESIZE (BTREE_LEAF_SIZE - \ + offsetof(zfs_btree_leaf_t, btl_elems)) + typedef struct zfs_btree_index { zfs_btree_hdr_t *bti_node; - uint64_t bti_offset; + uint32_t bti_offset; /* * True if the location is before the list offset, false if it's at * the listed offset. @@ -105,6 +112,7 @@ typedef struct btree { zfs_btree_hdr_t *bt_root; int64_t bt_height; size_t bt_elem_size; + uint32_t bt_leaf_cap; uint64_t bt_num_elems; uint64_t bt_num_nodes; zfs_btree_leaf_t *bt_bulk; // non-null if bulk loading diff --git a/include/sys/crypto/Makefile.am b/include/sys/crypto/Makefile.am deleted file mode 100644 index eb31f6a45743..000000000000 --- a/include/sys/crypto/Makefile.am +++ /dev/null @@ -1,16 +0,0 @@ -COMMON_H = \ - api.h \ - common.h \ - icp.h - -if CONFIG_USER -libzfsdir = $(includedir)/libzfs/sys/crypto -libzfs_HEADERS = $(COMMON_H) -endif - -if CONFIG_KERNEL -if BUILD_LINUX -kerneldir = @prefix@/src/zfs-$(VERSION)/include/sys/crypto -kernel_HEADERS = $(COMMON_H) -endif -endif diff --git a/include/sys/crypto/api.h b/include/sys/crypto/api.h index 17c9a645922e..88e0ac4d9699 100644 --- a/include/sys/crypto/api.h +++ b/include/sys/crypto/api.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -33,30 +33,12 @@ extern "C" { #include #include -typedef long crypto_req_id_t; -typedef void *crypto_bc_t; typedef void *crypto_context_t; typedef void *crypto_ctx_template_t; -typedef uint32_t crypto_call_flag_t; - -/* crypto_call_flag's values */ -#define CRYPTO_ALWAYS_QUEUE 0x00000001 /* ALWAYS queue the req. */ -#define CRYPTO_NOTIFY_OPDONE 0x00000002 /* Notify intermediate steps */ -#define CRYPTO_SKIP_REQID 0x00000004 /* Skip request ID generation */ -#define CRYPTO_RESTRICTED 0x00000008 /* cannot use restricted prov */ - -typedef struct { - crypto_call_flag_t cr_flag; - void (*cr_callback_func)(void *, int); - void *cr_callback_arg; - crypto_req_id_t cr_reqid; -} crypto_call_req_t; - /* * Returns the mechanism type corresponding to a mechanism name. */ - #define CRYPTO_MECH_INVALID ((uint64_t)-1) extern crypto_mech_type_t crypto_mech2id(const char *name); @@ -64,359 +46,26 @@ extern crypto_mech_type_t crypto_mech2id(const char *name); * Create and destroy context templates. */ extern int crypto_create_ctx_template(crypto_mechanism_t *mech, - crypto_key_t *key, crypto_ctx_template_t *tmpl, int kmflag); + crypto_key_t *key, crypto_ctx_template_t *tmpl); extern void crypto_destroy_ctx_template(crypto_ctx_template_t tmpl); -/* - * Single and multi-part digest operations. - */ -extern int crypto_digest(crypto_mechanism_t *mech, crypto_data_t *data, - crypto_data_t *digest, crypto_call_req_t *cr); -extern int crypto_digest_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_data_t *, crypto_data_t *, - crypto_call_req_t *); -extern int crypto_digest_init(crypto_mechanism_t *mech, crypto_context_t *ctxp, - crypto_call_req_t *cr); -extern int crypto_digest_init_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_context_t *, crypto_call_req_t *); -extern int crypto_digest_update(crypto_context_t ctx, crypto_data_t *data, - crypto_call_req_t *cr); -extern int crypto_digest_final(crypto_context_t ctx, crypto_data_t *digest, - crypto_call_req_t *cr); - /* * Single and multi-part MAC operations. */ extern int crypto_mac(crypto_mechanism_t *mech, crypto_data_t *data, - crypto_key_t *key, crypto_ctx_template_t tmpl, crypto_data_t *mac, - crypto_call_req_t *cr); -extern int crypto_mac_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_data_t *, crypto_key_t *, - crypto_ctx_template_t, crypto_data_t *, crypto_call_req_t *); -extern int crypto_mac_verify(crypto_mechanism_t *mech, crypto_data_t *data, - crypto_key_t *key, crypto_ctx_template_t tmpl, crypto_data_t *mac, - crypto_call_req_t *cr); -extern int crypto_mac_verify_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_data_t *, crypto_key_t *, - crypto_ctx_template_t, crypto_data_t *, crypto_call_req_t *); + crypto_key_t *key, crypto_ctx_template_t tmpl, crypto_data_t *mac); extern int crypto_mac_init(crypto_mechanism_t *mech, crypto_key_t *key, - crypto_ctx_template_t tmpl, crypto_context_t *ctxp, crypto_call_req_t *cr); -extern int crypto_mac_init_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_ctx_template_t, - crypto_context_t *, crypto_call_req_t *); -extern int crypto_mac_update(crypto_context_t ctx, crypto_data_t *data, - crypto_call_req_t *cr); -extern int crypto_mac_final(crypto_context_t ctx, crypto_data_t *data, - crypto_call_req_t *cr); + crypto_ctx_template_t tmpl, crypto_context_t *ctxp); +extern int crypto_mac_update(crypto_context_t ctx, crypto_data_t *data); +extern int crypto_mac_final(crypto_context_t ctx, crypto_data_t *data); /* - * Single and multi-part sign with private key operations. - */ -extern int crypto_sign(crypto_mechanism_t *mech, crypto_key_t *key, - crypto_data_t *data, crypto_ctx_template_t tmpl, - crypto_data_t *signature, crypto_call_req_t *cr); -extern int crypto_sign_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, - crypto_ctx_template_t, crypto_data_t *, crypto_call_req_t *); -extern int crypto_sign_init(crypto_mechanism_t *mech, crypto_key_t *key, - crypto_ctx_template_t tmpl, crypto_context_t *ctxp, crypto_call_req_t *cr); -extern int crypto_sign_init_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_ctx_template_t, - crypto_context_t *, crypto_call_req_t *); -extern int crypto_sign_update(crypto_context_t ctx, crypto_data_t *data, - crypto_call_req_t *cr); -extern int crypto_sign_final(crypto_context_t ctx, crypto_data_t *signature, - crypto_call_req_t *cr); -extern int crypto_sign_recover_init_prov(crypto_provider_t, - crypto_session_id_t, crypto_mechanism_t *, crypto_key_t *, - crypto_ctx_template_t tmpl, crypto_context_t *, crypto_call_req_t *); -extern int crypto_sign_recover(crypto_mechanism_t *mech, crypto_key_t *key, - crypto_data_t *data, crypto_ctx_template_t tmpl, crypto_data_t *signature, - crypto_call_req_t *cr); -extern int crypto_sign_recover_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, - crypto_ctx_template_t, crypto_data_t *, crypto_call_req_t *); - -/* - * Single and multi-part verify with public key operations. - */ -extern int crypto_verify(crypto_mechanism_t *mech, crypto_key_t *key, - crypto_data_t *data, crypto_ctx_template_t tmpl, crypto_data_t *signature, - crypto_call_req_t *cr); -extern int crypto_verify_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, - crypto_ctx_template_t, crypto_data_t *, crypto_call_req_t *); -extern int crypto_verify_init(crypto_mechanism_t *mech, crypto_key_t *key, - crypto_ctx_template_t tmpl, crypto_context_t *ctxp, crypto_call_req_t *cr); -extern int crypto_verify_init_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_ctx_template_t, - crypto_context_t *, crypto_call_req_t *); -extern int crypto_verify_update(crypto_context_t ctx, crypto_data_t *data, - crypto_call_req_t *cr); -extern int crypto_verify_final(crypto_context_t ctx, crypto_data_t *signature, - crypto_call_req_t *cr); -extern int crypto_verify_recover_init_prov(crypto_provider_t, - crypto_session_id_t, crypto_mechanism_t *, crypto_key_t *, - crypto_ctx_template_t tmpl, crypto_context_t *, crypto_call_req_t *); -extern int crypto_verify_recover(crypto_mechanism_t *mech, crypto_key_t *key, - crypto_data_t *signature, crypto_ctx_template_t tmpl, crypto_data_t *data, - crypto_call_req_t *cr); -extern int crypto_verify_recover_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, - crypto_ctx_template_t, crypto_data_t *, crypto_call_req_t *); - -/* - * Single and multi-part encryption operations. + * Single-part encryption/decryption operations. */ extern int crypto_encrypt(crypto_mechanism_t *mech, crypto_data_t *plaintext, - crypto_key_t *key, crypto_ctx_template_t tmpl, crypto_data_t *ciphertext, - crypto_call_req_t *cr); -extern int crypto_encrypt_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_data_t *, crypto_key_t *, - crypto_ctx_template_t, crypto_data_t *, crypto_call_req_t *); -extern int crypto_encrypt_init(crypto_mechanism_t *mech, crypto_key_t *key, - crypto_ctx_template_t tmpl, crypto_context_t *ctxp, crypto_call_req_t *cr); -extern int crypto_encrypt_init_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_ctx_template_t, - crypto_context_t *, crypto_call_req_t *); -extern int crypto_encrypt_update(crypto_context_t ctx, - crypto_data_t *plaintext, crypto_data_t *ciphertext, - crypto_call_req_t *cr); -extern int crypto_encrypt_final(crypto_context_t ctx, - crypto_data_t *ciphertext, crypto_call_req_t *cr); - -/* - * Single and multi-part decryption operations. - */ + crypto_key_t *key, crypto_ctx_template_t tmpl, crypto_data_t *ciphertext); extern int crypto_decrypt(crypto_mechanism_t *mech, crypto_data_t *ciphertext, - crypto_key_t *key, crypto_ctx_template_t tmpl, crypto_data_t *plaintext, - crypto_call_req_t *cr); -extern int crypto_decrypt_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_data_t *, crypto_key_t *, - crypto_ctx_template_t, crypto_data_t *, crypto_call_req_t *); -extern int crypto_decrypt_init(crypto_mechanism_t *mech, crypto_key_t *key, - crypto_ctx_template_t tmpl, crypto_context_t *ctxp, - crypto_call_req_t *cr); -extern int crypto_decrypt_init_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_ctx_template_t, - crypto_context_t *, crypto_call_req_t *); -extern int crypto_decrypt_update(crypto_context_t ctx, - crypto_data_t *ciphertext, crypto_data_t *plaintext, - crypto_call_req_t *cr); -extern int crypto_decrypt_final(crypto_context_t ctx, crypto_data_t *plaintext, - crypto_call_req_t *cr); - -/* - * Single and multi-part encrypt/MAC dual operations. - */ -extern int crypto_encrypt_mac(crypto_mechanism_t *encr_mech, - crypto_mechanism_t *mac_mech, crypto_data_t *pt, - crypto_key_t *encr_key, crypto_key_t *mac_key, - crypto_ctx_template_t encr_tmpl, crypto_ctx_template_t mac_tmpl, - crypto_dual_data_t *ct, crypto_data_t *mac, crypto_call_req_t *cr); -extern int crypto_encrypt_mac_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_mechanism_t *, crypto_data_t *, - crypto_key_t *, crypto_key_t *, crypto_ctx_template_t, - crypto_ctx_template_t, crypto_dual_data_t *, crypto_data_t *, - crypto_call_req_t *); -extern int crypto_encrypt_mac_init(crypto_mechanism_t *encr_mech, - crypto_mechanism_t *mac_mech, crypto_key_t *encr_key, - crypto_key_t *mac_key, crypto_ctx_template_t encr_tmpl, - crypto_ctx_template_t mac_tmpl, crypto_context_t *ctxp, - crypto_call_req_t *cr); -extern int crypto_encrypt_mac_init_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_mechanism_t *, crypto_key_t *, crypto_key_t *, - crypto_ctx_template_t, crypto_ctx_template_t, crypto_context_t *, - crypto_call_req_t *); -extern int crypto_encrypt_mac_update(crypto_context_t ctx, - crypto_data_t *pt, crypto_dual_data_t *ct, crypto_call_req_t *cr); -extern int crypto_encrypt_mac_final(crypto_context_t ctx, - crypto_dual_data_t *ct, crypto_data_t *mac, crypto_call_req_t *cr); - -/* - * Single and multi-part MAC/decrypt dual operations. - */ -extern int crypto_mac_decrypt(crypto_mechanism_t *mac_mech, - crypto_mechanism_t *decr_mech, crypto_dual_data_t *ct, - crypto_key_t *mac_key, crypto_key_t *decr_key, - crypto_ctx_template_t mac_tmpl, crypto_ctx_template_t decr_tmpl, - crypto_data_t *mac, crypto_data_t *pt, crypto_call_req_t *cr); -extern int crypto_mac_decrypt_prov(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *mac_mech, crypto_mechanism_t *decr_mech, - crypto_dual_data_t *ct, crypto_key_t *mac_key, crypto_key_t *decr_key, - crypto_ctx_template_t mac_tmpl, crypto_ctx_template_t decr_tmpl, - crypto_data_t *mac, crypto_data_t *pt, crypto_call_req_t *cr); -extern int crypto_mac_verify_decrypt(crypto_mechanism_t *mac_mech, - crypto_mechanism_t *decr_mech, crypto_dual_data_t *ct, - crypto_key_t *mac_key, crypto_key_t *decr_key, - crypto_ctx_template_t mac_tmpl, crypto_ctx_template_t decr_tmpl, - crypto_data_t *mac, crypto_data_t *pt, crypto_call_req_t *cr); -extern int crypto_mac_verify_decrypt_prov(crypto_provider_t, - crypto_session_id_t, crypto_mechanism_t *mac_mech, - crypto_mechanism_t *decr_mech, crypto_dual_data_t *ct, - crypto_key_t *mac_key, crypto_key_t *decr_key, - crypto_ctx_template_t mac_tmpl, crypto_ctx_template_t decr_tmpl, - crypto_data_t *mac, crypto_data_t *pt, crypto_call_req_t *cr); -extern int crypto_mac_decrypt_init(crypto_mechanism_t *mac_mech, - crypto_mechanism_t *decr_mech, crypto_key_t *mac_key, - crypto_key_t *decr_key, crypto_ctx_template_t mac_tmpl, - crypto_ctx_template_t decr_tmpl, crypto_context_t *ctxp, - crypto_call_req_t *cr); -extern int crypto_mac_decrypt_init_prov(crypto_provider_t, - crypto_session_id_t, crypto_mechanism_t *mac_mech, - crypto_mechanism_t *decr_mech, crypto_key_t *mac_key, - crypto_key_t *decr_key, crypto_ctx_template_t mac_tmpl, - crypto_ctx_template_t decr_tmpl, crypto_context_t *ctxp, - crypto_call_req_t *cr); -extern int crypto_mac_decrypt_update(crypto_context_t ctx, - crypto_dual_data_t *ct, crypto_data_t *pt, crypto_call_req_t *cr); -extern int crypto_mac_decrypt_final(crypto_context_t ctx, crypto_data_t *mac, - crypto_data_t *pt, crypto_call_req_t *cr); - -/* Session Management */ -extern int crypto_session_open(crypto_provider_t, crypto_session_id_t *, - crypto_call_req_t *); -extern int crypto_session_close(crypto_provider_t, crypto_session_id_t, - crypto_call_req_t *); -extern int crypto_session_login(crypto_provider_t, crypto_session_id_t, - crypto_user_type_t, char *, size_t, crypto_call_req_t *); -extern int crypto_session_logout(crypto_provider_t, crypto_session_id_t, - crypto_call_req_t *); - -/* Object Management */ -extern int crypto_object_copy(crypto_provider_t, crypto_session_id_t, - crypto_object_id_t, crypto_object_attribute_t *, uint_t, - crypto_object_id_t *, crypto_call_req_t *); -extern int crypto_object_create(crypto_provider_t, crypto_session_id_t, - crypto_object_attribute_t *, uint_t, crypto_object_id_t *, - crypto_call_req_t *); -extern int crypto_object_destroy(crypto_provider_t, crypto_session_id_t, - crypto_object_id_t, crypto_call_req_t *); -extern int crypto_object_get_attribute_value(crypto_provider_t, - crypto_session_id_t, crypto_object_id_t, crypto_object_attribute_t *, - uint_t, crypto_call_req_t *); -extern int crypto_object_get_size(crypto_provider_t, crypto_session_id_t, - crypto_object_id_t, size_t *, crypto_call_req_t *); -extern int crypto_object_find_final(crypto_provider_t, void *, - crypto_call_req_t *); -extern int crypto_object_find_init(crypto_provider_t, crypto_session_id_t, - crypto_object_attribute_t *, uint_t, void **, crypto_call_req_t *); -extern int crypto_object_find(crypto_provider_t, void *, crypto_object_id_t *, - uint_t *, uint_t, crypto_call_req_t *); -extern int crypto_object_set_attribute_value(crypto_provider_t, - crypto_session_id_t, crypto_object_id_t, crypto_object_attribute_t *, - uint_t, crypto_call_req_t *); - -/* Key Management */ -extern int crypto_key_derive(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_object_attribute_t *, - uint_t, crypto_object_id_t *, crypto_call_req_t *); -extern int crypto_key_generate(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_object_attribute_t *, uint_t, - crypto_object_id_t *, crypto_call_req_t *); -extern int crypto_key_generate_pair(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_object_attribute_t *, uint_t, - crypto_object_attribute_t *, uint_t, crypto_object_id_t *, - crypto_object_id_t *, crypto_call_req_t *); -extern int crypto_key_unwrap(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, uchar_t *, size_t *, - crypto_object_attribute_t *, uint_t, crypto_object_id_t *, - crypto_call_req_t *); -extern int crypto_key_wrap(crypto_provider_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_object_id_t *, uchar_t *, - size_t *, crypto_call_req_t *); -extern int crypto_key_check_prov(crypto_provider_t, crypto_mechanism_t *mech, - crypto_key_t *key); -extern int crypto_key_check(crypto_mechanism_t *mech, crypto_key_t *key); - - -/* - * Routines to cancel a single asynchronous request or all asynchronous - * requests associated with a particular context. - */ -extern void crypto_cancel_req(crypto_req_id_t req); -extern void crypto_cancel_ctx(crypto_context_t ctx); - -/* - * crypto_get_mech_list(9F) allocates and returns the list of currently - * supported cryptographic mechanisms. - */ -extern crypto_mech_name_t *crypto_get_mech_list(uint_t *count, int kmflag); -extern void crypto_free_mech_list(crypto_mech_name_t *mech_names, - uint_t count); - -extern crypto_provider_t crypto_get_provider(char *, char *, char *); -extern int crypto_get_provinfo(crypto_provider_t, crypto_provider_ext_info_t *); -extern void crypto_release_provider(crypto_provider_t); - -/* - * A kernel consumer can request to be notified when some particular event - * occurs. The valid events, callback function type, and functions to - * be called to register or unregister for notification are defined below. - */ - -#define CRYPTO_EVENT_MECHS_CHANGED 0x00000001 -#define CRYPTO_EVENT_PROVIDER_REGISTERED 0x00000002 -#define CRYPTO_EVENT_PROVIDER_UNREGISTERED 0x00000004 - -typedef enum { - CRYPTO_MECH_ADDED = 1, - CRYPTO_MECH_REMOVED -} crypto_event_change_t; - -/* The event_arg argument structure for CRYPTO_EVENT_PROVIDERS_CHANGE event */ -typedef struct crypto_notify_event_change { - crypto_mech_name_t ec_mech_name; - crypto_provider_type_t ec_provider_type; - crypto_event_change_t ec_change; -} crypto_notify_event_change_t; - -typedef void *crypto_notify_handle_t; -typedef void (*crypto_notify_callback_t)(uint32_t event_mask, void *event_arg); - -extern crypto_notify_handle_t crypto_notify_events( - crypto_notify_callback_t nf, uint32_t event_mask); -extern void crypto_unnotify_events(crypto_notify_handle_t); - -/* - * crypto_bufcall(9F) group of routines. - */ -extern crypto_bc_t crypto_bufcall_alloc(void); -extern int crypto_bufcall_free(crypto_bc_t bc); -extern int crypto_bufcall(crypto_bc_t bc, void (*func)(void *arg), void *arg); -extern int crypto_unbufcall(crypto_bc_t bc); - -/* - * To obtain the list of key size ranges supported by a mechanism. - */ - -#define CRYPTO_MECH_USAGE_ENCRYPT 0x00000001 -#define CRYPTO_MECH_USAGE_DECRYPT 0x00000002 -#define CRYPTO_MECH_USAGE_MAC 0x00000004 - -typedef uint32_t crypto_mech_usage_t; - -typedef struct crypto_mechanism_info { - size_t mi_min_key_size; - size_t mi_max_key_size; - crypto_keysize_unit_t mi_keysize_unit; /* for mi_xxx_key_size */ - crypto_mech_usage_t mi_usage; -} crypto_mechanism_info_t; - -#ifdef _SYSCALL32 - -typedef struct crypto_mechanism_info32 { - size32_t mi_min_key_size; - size32_t mi_max_key_size; - crypto_keysize_unit_t mi_keysize_unit; /* for mi_xxx_key_size */ - crypto_mech_usage_t mi_usage; -} crypto_mechanism_info32_t; - -#endif /* _SYSCALL32 */ - -extern int crypto_get_all_mech_info(crypto_mech_type_t, - crypto_mechanism_info_t **, uint_t *, int); -extern void crypto_free_all_mech_info(crypto_mechanism_info_t *, uint_t); + crypto_key_t *key, crypto_ctx_template_t tmpl, crypto_data_t *plaintext); #ifdef __cplusplus } diff --git a/include/sys/crypto/common.h b/include/sys/crypto/common.h index 9a239225cd10..261e88eceeea 100644 --- a/include/sys/crypto/common.h +++ b/include/sys/crypto/common.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -51,16 +51,6 @@ typedef struct crypto_mechanism { size_t cm_param_len; /* mech. parameter len */ } crypto_mechanism_t; -#ifdef _SYSCALL32 - -typedef struct crypto_mechanism32 { - crypto_mech_type_t cm_type; /* mechanism type */ - caddr32_t cm_param; /* mech. parameter */ - size32_t cm_param_len; /* mech. parameter len */ -} crypto_mechanism32_t; - -#endif /* _SYSCALL32 */ - /* CK_AES_CTR_PARAMS provides parameters to the CKM_AES_CTR mechanism */ typedef struct CK_AES_CTR_PARAMS { ulong_t ulCounterBits; @@ -94,89 +84,14 @@ typedef struct CK_AES_GMAC_PARAMS { ulong_t ulAADLen; } CK_AES_GMAC_PARAMS; -/* - * CK_ECDH1_DERIVE_PARAMS provides the parameters to the - * CKM_ECDH1_KEY_DERIVE mechanism - */ -typedef struct CK_ECDH1_DERIVE_PARAMS { - ulong_t kdf; - ulong_t ulSharedDataLen; - uchar_t *pSharedData; - ulong_t ulPublicDataLen; - uchar_t *pPublicData; -} CK_ECDH1_DERIVE_PARAMS; - -#ifdef _SYSCALL32 - -/* needed for 32-bit applications running on 64-bit kernels */ -typedef struct CK_AES_CTR_PARAMS32 { - uint32_t ulCounterBits; - uint8_t cb[16]; -} CK_AES_CTR_PARAMS32; - -/* needed for 32-bit applications running on 64-bit kernels */ -typedef struct CK_AES_CCM_PARAMS32 { - uint32_t ulMACSize; - uint32_t ulNonceSize; - uint32_t ulAuthDataSize; - uint32_t ulDataSize; - caddr32_t nonce; - caddr32_t authData; -} CK_AES_CCM_PARAMS32; - -/* needed for 32-bit applications running on 64-bit kernels */ -typedef struct CK_AES_GCM_PARAMS32 { - caddr32_t pIv; - uint32_t ulIvLen; - uint32_t ulIvBits; - caddr32_t pAAD; - uint32_t ulAADLen; - uint32_t ulTagBits; -} CK_AES_GCM_PARAMS32; - -/* needed for 32-bit applications running on 64-bit kernels */ -typedef struct CK_AES_GMAC_PARAMS32 { - caddr32_t pIv; - caddr32_t pAAD; - uint32_t ulAADLen; -} CK_AES_GMAC_PARAMS32; - -typedef struct CK_ECDH1_DERIVE_PARAMS32 { - uint32_t kdf; - uint32_t ulSharedDataLen; - caddr32_t pSharedData; - uint32_t ulPublicDataLen; - caddr32_t pPublicData; -} CK_ECDH1_DERIVE_PARAMS32; - -#endif /* _SYSCALL32 */ - /* * The measurement unit bit flag for a mechanism's minimum or maximum key size. * The unit are mechanism dependent. It can be in bits or in bytes. */ typedef uint32_t crypto_keysize_unit_t; -/* - * The following bit flags are valid in cm_mech_flags field in - * the crypto_mech_info_t structure of the SPI. - * - * Only the first two bit flags are valid in mi_keysize_unit - * field in the crypto_mechanism_info_t structure of the API. - */ -#define CRYPTO_KEYSIZE_UNIT_IN_BITS 0x00000001 -#define CRYPTO_KEYSIZE_UNIT_IN_BYTES 0x00000002 -#define CRYPTO_CAN_SHARE_OPSTATE 0x00000004 /* supports sharing */ - /* Mechanisms supported out-of-the-box */ -#define SUN_CKM_MD4 "CKM_MD4" -#define SUN_CKM_MD5 "CKM_MD5" -#define SUN_CKM_MD5_HMAC "CKM_MD5_HMAC" -#define SUN_CKM_MD5_HMAC_GENERAL "CKM_MD5_HMAC_GENERAL" -#define SUN_CKM_SHA1 "CKM_SHA_1" -#define SUN_CKM_SHA1_HMAC "CKM_SHA_1_HMAC" -#define SUN_CKM_SHA1_HMAC_GENERAL "CKM_SHA_1_HMAC_GENERAL" #define SUN_CKM_SHA256 "CKM_SHA256" #define SUN_CKM_SHA256_HMAC "CKM_SHA256_HMAC" #define SUN_CKM_SHA256_HMAC_GENERAL "CKM_SHA256_HMAC_GENERAL" @@ -188,44 +103,12 @@ typedef uint32_t crypto_keysize_unit_t; #define SUN_CKM_SHA512_HMAC_GENERAL "CKM_SHA512_HMAC_GENERAL" #define SUN_CKM_SHA512_224 "CKM_SHA512_224" #define SUN_CKM_SHA512_256 "CKM_SHA512_256" -#define SUN_CKM_DES_CBC "CKM_DES_CBC" -#define SUN_CKM_DES3_CBC "CKM_DES3_CBC" -#define SUN_CKM_DES_ECB "CKM_DES_ECB" -#define SUN_CKM_DES3_ECB "CKM_DES3_ECB" -#define SUN_CKM_BLOWFISH_CBC "CKM_BLOWFISH_CBC" -#define SUN_CKM_BLOWFISH_ECB "CKM_BLOWFISH_ECB" #define SUN_CKM_AES_CBC "CKM_AES_CBC" #define SUN_CKM_AES_ECB "CKM_AES_ECB" #define SUN_CKM_AES_CTR "CKM_AES_CTR" #define SUN_CKM_AES_CCM "CKM_AES_CCM" #define SUN_CKM_AES_GCM "CKM_AES_GCM" #define SUN_CKM_AES_GMAC "CKM_AES_GMAC" -#define SUN_CKM_AES_CFB128 "CKM_AES_CFB128" -#define SUN_CKM_RC4 "CKM_RC4" -#define SUN_CKM_RSA_PKCS "CKM_RSA_PKCS" -#define SUN_CKM_RSA_X_509 "CKM_RSA_X_509" -#define SUN_CKM_MD5_RSA_PKCS "CKM_MD5_RSA_PKCS" -#define SUN_CKM_SHA1_RSA_PKCS "CKM_SHA1_RSA_PKCS" -#define SUN_CKM_SHA256_RSA_PKCS "CKM_SHA256_RSA_PKCS" -#define SUN_CKM_SHA384_RSA_PKCS "CKM_SHA384_RSA_PKCS" -#define SUN_CKM_SHA512_RSA_PKCS "CKM_SHA512_RSA_PKCS" -#define SUN_CKM_EC_KEY_PAIR_GEN "CKM_EC_KEY_PAIR_GEN" -#define SUN_CKM_ECDH1_DERIVE "CKM_ECDH1_DERIVE" -#define SUN_CKM_ECDSA_SHA1 "CKM_ECDSA_SHA1" -#define SUN_CKM_ECDSA "CKM_ECDSA" - -/* Shared operation context format for CKM_RC4 */ -typedef struct { -#if defined(__amd64) - uint32_t i, j; - uint32_t arr[256]; - uint32_t flag; -#else - uchar_t arr[256]; - uchar_t i, j; -#endif /* __amd64 */ - uint64_t pad; /* For 64-bit alignment */ -} arcfour_state_t; /* Data arguments of cryptographic operations */ @@ -238,140 +121,22 @@ typedef struct crypto_data { crypto_data_format_t cd_format; /* Format identifier */ off_t cd_offset; /* Offset from the beginning */ size_t cd_length; /* # of bytes in use */ - caddr_t cd_miscdata; /* ancillary data */ union { /* Raw format */ - iovec_t cdu_raw; /* Pointer and length */ + iovec_t cd_raw; /* Pointer and length */ /* uio scatter-gather format */ - zfs_uio_t *cdu_uio; - - } cdu; /* Crypto Data Union */ + zfs_uio_t *cd_uio; + }; /* Crypto Data Union */ } crypto_data_t; -#define cd_raw cdu.cdu_raw -#define cd_uio cdu.cdu_uio -#define cd_mp cdu.cdu_mp - -typedef struct crypto_dual_data { - crypto_data_t dd_data; /* The data */ - off_t dd_offset2; /* Used by dual operation */ - size_t dd_len2; /* # of bytes to take */ -} crypto_dual_data_t; - -#define dd_format dd_data.cd_format -#define dd_offset1 dd_data.cd_offset -#define dd_len1 dd_data.cd_length -#define dd_miscdata dd_data.cd_miscdata -#define dd_raw dd_data.cd_raw -#define dd_uio dd_data.cd_uio -#define dd_mp dd_data.cd_mp - /* The keys, and their contents */ -typedef enum { - CRYPTO_KEY_RAW = 1, /* ck_data is a cleartext key */ - CRYPTO_KEY_REFERENCE, /* ck_obj_id is an opaque reference */ - CRYPTO_KEY_ATTR_LIST /* ck_attrs is a list of object attributes */ -} crypto_key_format_t; - -typedef uint64_t crypto_attr_type_t; - -/* Attribute types to use for passing a RSA public key or a private key. */ -#define SUN_CKA_MODULUS 0x00000120 -#define SUN_CKA_MODULUS_BITS 0x00000121 -#define SUN_CKA_PUBLIC_EXPONENT 0x00000122 -#define SUN_CKA_PRIVATE_EXPONENT 0x00000123 -#define SUN_CKA_PRIME_1 0x00000124 -#define SUN_CKA_PRIME_2 0x00000125 -#define SUN_CKA_EXPONENT_1 0x00000126 -#define SUN_CKA_EXPONENT_2 0x00000127 -#define SUN_CKA_COEFFICIENT 0x00000128 -#define SUN_CKA_PRIME 0x00000130 -#define SUN_CKA_SUBPRIME 0x00000131 -#define SUN_CKA_BASE 0x00000132 - -#define CKK_EC 0x00000003 -#define CKK_GENERIC_SECRET 0x00000010 -#define CKK_RC4 0x00000012 -#define CKK_AES 0x0000001F -#define CKK_DES 0x00000013 -#define CKK_DES2 0x00000014 -#define CKK_DES3 0x00000015 - -#define CKO_PUBLIC_KEY 0x00000002 -#define CKO_PRIVATE_KEY 0x00000003 -#define CKA_CLASS 0x00000000 -#define CKA_VALUE 0x00000011 -#define CKA_KEY_TYPE 0x00000100 -#define CKA_VALUE_LEN 0x00000161 -#define CKA_EC_PARAMS 0x00000180 -#define CKA_EC_POINT 0x00000181 - -typedef uint32_t crypto_object_id_t; - -typedef struct crypto_object_attribute { - crypto_attr_type_t oa_type; /* attribute type */ - caddr_t oa_value; /* attribute value */ - ssize_t oa_value_len; /* length of attribute value */ -} crypto_object_attribute_t; - -typedef struct crypto_key { - crypto_key_format_t ck_format; /* format identifier */ - union { - /* for CRYPTO_KEY_RAW ck_format */ - struct { - uint_t cku_v_length; /* # of bits in ck_data */ - void *cku_v_data; /* ptr to key value */ - } cku_key_value; - - /* for CRYPTO_KEY_REFERENCE ck_format */ - crypto_object_id_t cku_key_id; /* reference to object key */ - - /* for CRYPTO_KEY_ATTR_LIST ck_format */ - struct { - uint_t cku_a_count; /* number of attributes */ - crypto_object_attribute_t *cku_a_oattr; - } cku_key_attrs; - } cku_data; /* Crypto Key union */ +typedef struct { + uint_t ck_length; /* # of bits in ck_data */ + void *ck_data; /* ptr to key value */ } crypto_key_t; -#ifdef _SYSCALL32 - -typedef struct crypto_object_attribute32 { - uint64_t oa_type; /* attribute type */ - caddr32_t oa_value; /* attribute value */ - ssize32_t oa_value_len; /* length of attribute value */ -} crypto_object_attribute32_t; - -typedef struct crypto_key32 { - crypto_key_format_t ck_format; /* format identifier */ - union { - /* for CRYPTO_KEY_RAW ck_format */ - struct { - uint32_t cku_v_length; /* # of bytes in ck_data */ - caddr32_t cku_v_data; /* ptr to key value */ - } cku_key_value; - - /* for CRYPTO_KEY_REFERENCE ck_format */ - crypto_object_id_t cku_key_id; /* reference to object key */ - - /* for CRYPTO_KEY_ATTR_LIST ck_format */ - struct { - uint32_t cku_a_count; /* number of attributes */ - caddr32_t cku_a_oattr; - } cku_key_attrs; - } cku_data; /* Crypto Key union */ -} crypto_key32_t; - -#endif /* _SYSCALL32 */ - -#define ck_data cku_data.cku_key_value.cku_v_data -#define ck_length cku_data.cku_key_value.cku_v_length -#define ck_obj_id cku_data.cku_key_id -#define ck_count cku_data.cku_key_attrs.cku_a_count -#define ck_attrs cku_data.cku_key_attrs.cku_a_oattr - /* * Raw key lengths are expressed in number of bits. * The following macro returns the minimum number of @@ -383,198 +148,37 @@ typedef struct crypto_key32 { /* Providers */ -typedef enum { - CRYPTO_HW_PROVIDER = 0, - CRYPTO_SW_PROVIDER, - CRYPTO_LOGICAL_PROVIDER -} crypto_provider_type_t; - typedef uint32_t crypto_provider_id_t; #define KCF_PROVID_INVALID ((uint32_t)-1) -typedef struct crypto_provider_entry { - crypto_provider_id_t pe_provider_id; - uint_t pe_mechanism_count; -} crypto_provider_entry_t; - -typedef struct crypto_dev_list_entry { - char le_dev_name[MAXNAMELEN]; - uint_t le_dev_instance; - uint_t le_mechanism_count; -} crypto_dev_list_entry_t; - -/* User type for authentication ioctls and SPI entry points */ - -typedef enum crypto_user_type { - CRYPTO_SO = 0, - CRYPTO_USER -} crypto_user_type_t; - -/* Version for provider management ioctls and SPI entry points */ - -typedef struct crypto_version { - uchar_t cv_major; - uchar_t cv_minor; -} crypto_version_t; - /* session data structure opaque to the consumer */ typedef void *crypto_session_t; -/* provider data structure opaque to the consumer */ -typedef void *crypto_provider_t; - -/* Limits used by both consumers and providers */ -#define CRYPTO_EXT_SIZE_LABEL 32 -#define CRYPTO_EXT_SIZE_MANUF 32 -#define CRYPTO_EXT_SIZE_MODEL 16 -#define CRYPTO_EXT_SIZE_SERIAL 16 -#define CRYPTO_EXT_SIZE_TIME 16 - -typedef struct crypto_provider_ext_info { - uchar_t ei_label[CRYPTO_EXT_SIZE_LABEL]; - uchar_t ei_manufacturerID[CRYPTO_EXT_SIZE_MANUF]; - uchar_t ei_model[CRYPTO_EXT_SIZE_MODEL]; - uchar_t ei_serial_number[CRYPTO_EXT_SIZE_SERIAL]; - ulong_t ei_flags; - ulong_t ei_max_session_count; - ulong_t ei_max_pin_len; - ulong_t ei_min_pin_len; - ulong_t ei_total_public_memory; - ulong_t ei_free_public_memory; - ulong_t ei_total_private_memory; - ulong_t ei_free_private_memory; - crypto_version_t ei_hardware_version; - crypto_version_t ei_firmware_version; - uchar_t ei_time[CRYPTO_EXT_SIZE_TIME]; - int ei_hash_max_input_len; - int ei_hmac_max_input_len; -} crypto_provider_ext_info_t; - -typedef uint_t crypto_session_id_t; - -typedef enum cmd_type { - COPY_FROM_DATA, - COPY_TO_DATA, - COMPARE_TO_DATA, - MD5_DIGEST_DATA, - SHA1_DIGEST_DATA, - SHA2_DIGEST_DATA, - GHASH_DATA -} cmd_type_t; - -#define CRYPTO_DO_UPDATE 0x01 -#define CRYPTO_DO_FINAL 0x02 -#define CRYPTO_DO_MD5 0x04 -#define CRYPTO_DO_SHA1 0x08 -#define CRYPTO_DO_SIGN 0x10 -#define CRYPTO_DO_VERIFY 0x20 -#define CRYPTO_DO_SHA2 0x40 - #define PROVIDER_OWNS_KEY_SCHEDULE 0x00000001 /* * Common cryptographic status and error codes. */ #define CRYPTO_SUCCESS 0x00000000 -#define CRYPTO_CANCEL 0x00000001 #define CRYPTO_HOST_MEMORY 0x00000002 -#define CRYPTO_GENERAL_ERROR 0x00000003 #define CRYPTO_FAILED 0x00000004 #define CRYPTO_ARGUMENTS_BAD 0x00000005 -#define CRYPTO_ATTRIBUTE_READ_ONLY 0x00000006 -#define CRYPTO_ATTRIBUTE_SENSITIVE 0x00000007 -#define CRYPTO_ATTRIBUTE_TYPE_INVALID 0x00000008 -#define CRYPTO_ATTRIBUTE_VALUE_INVALID 0x00000009 -#define CRYPTO_CANCELED 0x0000000A -#define CRYPTO_DATA_INVALID 0x0000000B #define CRYPTO_DATA_LEN_RANGE 0x0000000C -#define CRYPTO_DEVICE_ERROR 0x0000000D -#define CRYPTO_DEVICE_MEMORY 0x0000000E -#define CRYPTO_DEVICE_REMOVED 0x0000000F -#define CRYPTO_ENCRYPTED_DATA_INVALID 0x00000010 #define CRYPTO_ENCRYPTED_DATA_LEN_RANGE 0x00000011 -#define CRYPTO_KEY_HANDLE_INVALID 0x00000012 #define CRYPTO_KEY_SIZE_RANGE 0x00000013 #define CRYPTO_KEY_TYPE_INCONSISTENT 0x00000014 -#define CRYPTO_KEY_NOT_NEEDED 0x00000015 -#define CRYPTO_KEY_CHANGED 0x00000016 -#define CRYPTO_KEY_NEEDED 0x00000017 -#define CRYPTO_KEY_INDIGESTIBLE 0x00000018 -#define CRYPTO_KEY_FUNCTION_NOT_PERMITTED 0x00000019 -#define CRYPTO_KEY_NOT_WRAPPABLE 0x0000001A -#define CRYPTO_KEY_UNEXTRACTABLE 0x0000001B #define CRYPTO_MECHANISM_INVALID 0x0000001C #define CRYPTO_MECHANISM_PARAM_INVALID 0x0000001D -#define CRYPTO_OBJECT_HANDLE_INVALID 0x0000001E -#define CRYPTO_OPERATION_IS_ACTIVE 0x0000001F -#define CRYPTO_OPERATION_NOT_INITIALIZED 0x00000020 -#define CRYPTO_PIN_INCORRECT 0x00000021 -#define CRYPTO_PIN_INVALID 0x00000022 -#define CRYPTO_PIN_LEN_RANGE 0x00000023 -#define CRYPTO_PIN_EXPIRED 0x00000024 -#define CRYPTO_PIN_LOCKED 0x00000025 -#define CRYPTO_SESSION_CLOSED 0x00000026 -#define CRYPTO_SESSION_COUNT 0x00000027 -#define CRYPTO_SESSION_HANDLE_INVALID 0x00000028 -#define CRYPTO_SESSION_READ_ONLY 0x00000029 -#define CRYPTO_SESSION_EXISTS 0x0000002A -#define CRYPTO_SESSION_READ_ONLY_EXISTS 0x0000002B -#define CRYPTO_SESSION_READ_WRITE_SO_EXISTS 0x0000002C #define CRYPTO_SIGNATURE_INVALID 0x0000002D -#define CRYPTO_SIGNATURE_LEN_RANGE 0x0000002E -#define CRYPTO_TEMPLATE_INCOMPLETE 0x0000002F -#define CRYPTO_TEMPLATE_INCONSISTENT 0x00000030 -#define CRYPTO_UNWRAPPING_KEY_HANDLE_INVALID 0x00000031 -#define CRYPTO_UNWRAPPING_KEY_SIZE_RANGE 0x00000032 -#define CRYPTO_UNWRAPPING_KEY_TYPE_INCONSISTENT 0x00000033 -#define CRYPTO_USER_ALREADY_LOGGED_IN 0x00000034 -#define CRYPTO_USER_NOT_LOGGED_IN 0x00000035 -#define CRYPTO_USER_PIN_NOT_INITIALIZED 0x00000036 -#define CRYPTO_USER_TYPE_INVALID 0x00000037 -#define CRYPTO_USER_ANOTHER_ALREADY_LOGGED_IN 0x00000038 -#define CRYPTO_USER_TOO_MANY_TYPES 0x00000039 -#define CRYPTO_WRAPPED_KEY_INVALID 0x0000003A -#define CRYPTO_WRAPPED_KEY_LEN_RANGE 0x0000003B -#define CRYPTO_WRAPPING_KEY_HANDLE_INVALID 0x0000003C -#define CRYPTO_WRAPPING_KEY_SIZE_RANGE 0x0000003D -#define CRYPTO_WRAPPING_KEY_TYPE_INCONSISTENT 0x0000003E -#define CRYPTO_RANDOM_SEED_NOT_SUPPORTED 0x0000003F -#define CRYPTO_RANDOM_NO_RNG 0x00000040 -#define CRYPTO_DOMAIN_PARAMS_INVALID 0x00000041 #define CRYPTO_BUFFER_TOO_SMALL 0x00000042 -#define CRYPTO_INFORMATION_SENSITIVE 0x00000043 #define CRYPTO_NOT_SUPPORTED 0x00000044 -#define CRYPTO_QUEUED 0x00000045 -#define CRYPTO_BUFFER_TOO_BIG 0x00000046 #define CRYPTO_INVALID_CONTEXT 0x00000047 #define CRYPTO_INVALID_MAC 0x00000048 #define CRYPTO_MECH_NOT_SUPPORTED 0x00000049 -#define CRYPTO_INCONSISTENT_ATTRIBUTE 0x0000004A -#define CRYPTO_NO_PERMISSION 0x0000004B #define CRYPTO_INVALID_PROVIDER_ID 0x0000004C -#define CRYPTO_VERSION_MISMATCH 0x0000004D #define CRYPTO_BUSY 0x0000004E #define CRYPTO_UNKNOWN_PROVIDER 0x0000004F -#define CRYPTO_MODVERIFICATION_FAILED 0x00000050 -#define CRYPTO_OLD_CTX_TEMPLATE 0x00000051 -#define CRYPTO_WEAK_KEY 0x00000052 -#define CRYPTO_FIPS140_ERROR 0x00000053 -/* - * Don't forget to update CRYPTO_LAST_ERROR and the error_number_table[] - * in kernelUtil.c when new error code is added. - */ -#define CRYPTO_LAST_ERROR 0x00000053 - -/* - * Special values that can be used to indicate that information is unavailable - * or that there is not practical limit. These values can be used - * by fields of the SPI crypto_provider_ext_info(9S) structure. - * The value of CRYPTO_UNAVAILABLE_INFO should be the same as - * CK_UNAVAILABLE_INFO in the PKCS#11 spec. - */ -#define CRYPTO_UNAVAILABLE_INFO ((ulong_t)(-1)) -#define CRYPTO_EFFECTIVELY_INFINITE 0x0 #ifdef __cplusplus } diff --git a/include/sys/crypto/icp.h b/include/sys/crypto/icp.h index ae7f7eae529e..8c3f19886fd8 100644 --- a/include/sys/crypto/icp.h +++ b/include/sys/crypto/icp.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/dataset_kstats.h b/include/sys/dataset_kstats.h index b165b98576dd..40cf5258a2e7 100644 --- a/include/sys/dataset_kstats.h +++ b/include/sys/dataset_kstats.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -30,6 +30,7 @@ #include #include #include +#include typedef struct dataset_sum_stats_t { wmsum_t dss_writes; @@ -56,14 +57,19 @@ typedef struct dataset_kstat_values { * entry is removed from the unlinked set */ kstat_named_t dkv_nunlinked; + /* + * Per dataset zil kstats + */ + zil_kstat_values_t dkv_zil_stats; } dataset_kstat_values_t; typedef struct dataset_kstats { dataset_sum_stats_t dk_sums; + zil_sums_t dk_zil_sums; kstat_t *dk_kstats; } dataset_kstats_t; -void dataset_kstats_create(dataset_kstats_t *, objset_t *); +int dataset_kstats_create(dataset_kstats_t *, objset_t *); void dataset_kstats_destroy(dataset_kstats_t *); void dataset_kstats_update_write_kstats(dataset_kstats_t *, int64_t); diff --git a/include/sys/dbuf.h b/include/sys/dbuf.h index 9017cf345724..a3aaac5e87ce 100644 --- a/include/sys/dbuf.h +++ b/include/sys/dbuf.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -321,16 +321,15 @@ typedef struct dmu_buf_impl { uint8_t db_dirtycnt; } dmu_buf_impl_t; -/* Note: the dbuf hash table is exposed only for the mdb module */ -#define DBUF_MUTEXES 2048 -#define DBUF_HASH_MUTEX(h, idx) (&(h)->hash_mutexes[(idx) & (DBUF_MUTEXES-1)]) +#define DBUF_RWLOCKS 8192 +#define DBUF_HASH_RWLOCK(h, idx) (&(h)->hash_rwlocks[(idx) & (DBUF_RWLOCKS-1)]) typedef struct dbuf_hash_table { uint64_t hash_table_mask; dmu_buf_impl_t **hash_table; - kmutex_t hash_mutexes[DBUF_MUTEXES] ____cacheline_aligned; + krwlock_t hash_rwlocks[DBUF_RWLOCKS] ____cacheline_aligned; } dbuf_hash_table_t; -typedef void (*dbuf_prefetch_fn)(void *, boolean_t); +typedef void (*dbuf_prefetch_fn)(void *, uint64_t, uint64_t, boolean_t); uint64_t dbuf_whichblock(const struct dnode *di, const int64_t level, const uint64_t offset); @@ -340,12 +339,12 @@ int dbuf_spill_set_blksz(dmu_buf_t *db, uint64_t blksz, dmu_tx_t *tx); void dbuf_rm_spill(struct dnode *dn, dmu_tx_t *tx); -dmu_buf_impl_t *dbuf_hold(struct dnode *dn, uint64_t blkid, void *tag); +dmu_buf_impl_t *dbuf_hold(struct dnode *dn, uint64_t blkid, const void *tag); dmu_buf_impl_t *dbuf_hold_level(struct dnode *dn, int level, uint64_t blkid, - void *tag); + const void *tag); int dbuf_hold_impl(struct dnode *dn, uint8_t level, uint64_t blkid, boolean_t fail_sparse, boolean_t fail_uncached, - void *tag, dmu_buf_impl_t **dbp); + const void *tag, dmu_buf_impl_t **dbp); int dbuf_prefetch_impl(struct dnode *dn, int64_t level, uint64_t blkid, zio_priority_t prio, arc_flags_t aflags, dbuf_prefetch_fn cb, @@ -353,13 +352,14 @@ int dbuf_prefetch_impl(struct dnode *dn, int64_t level, uint64_t blkid, int dbuf_prefetch(struct dnode *dn, int64_t level, uint64_t blkid, zio_priority_t prio, arc_flags_t aflags); -void dbuf_add_ref(dmu_buf_impl_t *db, void *tag); +void dbuf_add_ref(dmu_buf_impl_t *db, const void *tag); boolean_t dbuf_try_add_ref(dmu_buf_t *db, objset_t *os, uint64_t obj, - uint64_t blkid, void *tag); + uint64_t blkid, const void *tag); uint64_t dbuf_refcount(dmu_buf_impl_t *db); -void dbuf_rele(dmu_buf_impl_t *db, void *tag); -void dbuf_rele_and_unlock(dmu_buf_impl_t *db, void *tag, boolean_t evicting); +void dbuf_rele(dmu_buf_impl_t *db, const void *tag); +void dbuf_rele_and_unlock(dmu_buf_impl_t *db, const void *tag, + boolean_t evicting); dmu_buf_impl_t *dbuf_find(struct objset *os, uint64_t object, uint8_t level, uint64_t blkid); @@ -386,8 +386,10 @@ void dbuf_destroy(dmu_buf_impl_t *db); void dbuf_unoverride(dbuf_dirty_record_t *dr); void dbuf_sync_list(list_t *list, int level, dmu_tx_t *tx); void dbuf_release_bp(dmu_buf_impl_t *db); -db_lock_type_t dmu_buf_lock_parent(dmu_buf_impl_t *db, krw_t rw, void *tag); -void dmu_buf_unlock_parent(dmu_buf_impl_t *db, db_lock_type_t type, void *tag); +db_lock_type_t dmu_buf_lock_parent(dmu_buf_impl_t *db, krw_t rw, + const void *tag); +void dmu_buf_unlock_parent(dmu_buf_impl_t *db, db_lock_type_t type, + const void *tag); void dbuf_free_range(struct dnode *dn, uint64_t start, uint64_t end, struct dmu_tx *); diff --git a/include/sys/ddt.h b/include/sys/ddt.h index 25be6f56dddc..d72401dcf7a4 100644 --- a/include/sys/ddt.h +++ b/include/sys/ddt.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/dmu.h b/include/sys/dmu.h index 1ddff0d4e4e7..5a3d7d6a5055 100644 --- a/include/sys/dmu.h +++ b/include/sys/dmu.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -319,11 +319,12 @@ void zfs_znode_byteswap(void *buf, size_t size); typedef void dmu_objset_create_sync_func_t(objset_t *os, void *arg, cred_t *cr, dmu_tx_t *tx); -int dmu_objset_hold(const char *name, void *tag, objset_t **osp); +int dmu_objset_hold(const char *name, const void *tag, objset_t **osp); int dmu_objset_own(const char *name, dmu_objset_type_t type, - boolean_t readonly, boolean_t key_required, void *tag, objset_t **osp); -void dmu_objset_rele(objset_t *os, void *tag); -void dmu_objset_disown(objset_t *os, boolean_t key_required, void *tag); + boolean_t readonly, boolean_t key_required, const void *tag, + objset_t **osp); +void dmu_objset_rele(objset_t *os, const void *tag); +void dmu_objset_disown(objset_t *os, boolean_t key_required, const void *tag); int dmu_objset_open_ds(struct dsl_dataset *ds, objset_t **osp); void dmu_objset_evict_dbufs(objset_t *os); @@ -407,7 +408,7 @@ uint64_t dmu_object_alloc_dnsize(objset_t *os, dmu_object_type_t ot, int dnodesize, dmu_tx_t *tx); uint64_t dmu_object_alloc_hold(objset_t *os, dmu_object_type_t ot, int blocksize, int indirect_blockshift, dmu_object_type_t bonustype, - int bonuslen, int dnodesize, dnode_t **allocated_dnode, void *tag, + int bonuslen, int dnodesize, dnode_t **allocated_dnode, const void *tag, dmu_tx_t *tx); int dmu_object_claim(objset_t *os, uint64_t object, dmu_object_type_t ot, int blocksize, dmu_object_type_t bonus_type, int bonus_len, dmu_tx_t *tx); @@ -524,8 +525,9 @@ void dmu_write_policy(objset_t *os, dnode_t *dn, int level, int wp, * * Returns ENOENT, EIO, or 0. */ -int dmu_bonus_hold(objset_t *os, uint64_t object, void *tag, dmu_buf_t **dbp); -int dmu_bonus_hold_by_dnode(dnode_t *dn, void *tag, dmu_buf_t **dbp, +int dmu_bonus_hold(objset_t *os, uint64_t object, const void *tag, + dmu_buf_t **dbp); +int dmu_bonus_hold_by_dnode(dnode_t *dn, const void *tag, dmu_buf_t **dbp, uint32_t flags); int dmu_bonus_max(void); int dmu_set_bonus(dmu_buf_t *, int, dmu_tx_t *); @@ -537,11 +539,11 @@ int dmu_rm_spill(objset_t *, uint64_t, dmu_tx_t *); * Special spill buffer support used by "SA" framework */ -int dmu_spill_hold_by_bonus(dmu_buf_t *bonus, uint32_t flags, void *tag, +int dmu_spill_hold_by_bonus(dmu_buf_t *bonus, uint32_t flags, const void *tag, dmu_buf_t **dbp); int dmu_spill_hold_by_dnode(dnode_t *dn, uint32_t flags, - void *tag, dmu_buf_t **dbp); -int dmu_spill_hold_existing(dmu_buf_t *bonus, void *tag, dmu_buf_t **dbp); + const void *tag, dmu_buf_t **dbp); +int dmu_spill_hold_existing(dmu_buf_t *bonus, const void *tag, dmu_buf_t **dbp); /* * Obtain the DMU buffer from the specified object which contains the @@ -558,19 +560,20 @@ int dmu_spill_hold_existing(dmu_buf_t *bonus, void *tag, dmu_buf_t **dbp); * The object number must be a valid, allocated object number. */ int dmu_buf_hold(objset_t *os, uint64_t object, uint64_t offset, - void *tag, dmu_buf_t **, int flags); + const void *tag, dmu_buf_t **, int flags); int dmu_buf_hold_array(objset_t *os, uint64_t object, uint64_t offset, - uint64_t length, int read, void *tag, int *numbufsp, dmu_buf_t ***dbpp); + uint64_t length, int read, const void *tag, int *numbufsp, + dmu_buf_t ***dbpp); int dmu_buf_hold_by_dnode(dnode_t *dn, uint64_t offset, - void *tag, dmu_buf_t **dbp, int flags); + const void *tag, dmu_buf_t **dbp, int flags); int dmu_buf_hold_array_by_dnode(dnode_t *dn, uint64_t offset, - uint64_t length, boolean_t read, void *tag, int *numbufsp, + uint64_t length, boolean_t read, const void *tag, int *numbufsp, dmu_buf_t ***dbpp, uint32_t flags); /* * Add a reference to a dmu buffer that has already been held via * dmu_buf_hold() in the current context. */ -void dmu_buf_add_ref(dmu_buf_t *db, void* tag); +void dmu_buf_add_ref(dmu_buf_t *db, const void *tag); /* * Attempt to add a reference to a dmu buffer that is in an unknown state, @@ -580,9 +583,9 @@ void dmu_buf_add_ref(dmu_buf_t *db, void* tag); * one hold by a user other than the syncer. */ boolean_t dmu_buf_try_add_ref(dmu_buf_t *, objset_t *os, uint64_t object, - uint64_t blkid, void *tag); + uint64_t blkid, const void *tag); -void dmu_buf_rele(dmu_buf_t *db, void *tag); +void dmu_buf_rele(dmu_buf_t *db, const void *tag); uint64_t dmu_buf_refcount(dmu_buf_t *db); uint64_t dmu_buf_user_refcount(dmu_buf_t *db); @@ -597,9 +600,9 @@ uint64_t dmu_buf_user_refcount(dmu_buf_t *db); * individually with dmu_buf_rele. */ int dmu_buf_hold_array_by_bonus(dmu_buf_t *db, uint64_t offset, - uint64_t length, boolean_t read, void *tag, + uint64_t length, boolean_t read, const void *tag, int *numbufsp, dmu_buf_t ***dbpp); -void dmu_buf_rele_array(dmu_buf_t **, int numbufs, void *tag); +void dmu_buf_rele_array(dmu_buf_t **, int numbufs, const void *tag); typedef void dmu_buf_evict_func_t(void *user_ptr); @@ -895,12 +898,12 @@ typedef struct dmu_object_type_info { boolean_t ot_metadata; boolean_t ot_dbuf_metadata_cache; boolean_t ot_encrypt; - char *ot_name; + const char *ot_name; } dmu_object_type_info_t; typedef const struct dmu_object_byteswap_info { arc_byteswap_func_t ob_func; - char *ob_name; + const char *ob_name; } dmu_object_byteswap_info_t; extern const dmu_object_type_info_t dmu_ot[DMU_OT_NUMTYPES]; @@ -1067,6 +1070,8 @@ int dmu_diff(const char *tosnap_name, const char *fromsnap_name, #define ZFS_CRC64_POLY 0xC96C5795D7870F42ULL /* ECMA-182, reflected form */ extern uint64_t zfs_crc64_table[256]; +extern int dmu_prefetch_max; + #ifdef __cplusplus } #endif diff --git a/include/sys/dmu_impl.h b/include/sys/dmu_impl.h index def4aadba1d0..ce6ae3c665ac 100644 --- a/include/sys/dmu_impl.h +++ b/include/sys/dmu_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -248,7 +248,7 @@ typedef struct dmu_sendstatus { void dmu_object_zapify(objset_t *, uint64_t, dmu_object_type_t, dmu_tx_t *); void dmu_object_free_zapified(objset_t *, uint64_t, dmu_tx_t *); int dmu_buf_hold_noread(objset_t *, uint64_t, uint64_t, - void *, dmu_buf_t **); + const void *, dmu_buf_t **); #ifdef __cplusplus } diff --git a/include/sys/dmu_objset.h b/include/sys/dmu_objset.h index 7ade2dc91247..d22c682875d8 100644 --- a/include/sys/dmu_objset.h +++ b/include/sys/dmu_objset.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -201,19 +201,19 @@ struct objset { #define DMU_PROJECTUSED_DNODE(os) ((os)->os_projectused_dnode.dnh_dnode) /* called from zpl */ -int dmu_objset_hold(const char *name, void *tag, objset_t **osp); -int dmu_objset_hold_flags(const char *name, boolean_t decrypt, void *tag, +int dmu_objset_hold(const char *name, const void *tag, objset_t **osp); +int dmu_objset_hold_flags(const char *name, boolean_t decrypt, const void *tag, objset_t **osp); int dmu_objset_own(const char *name, dmu_objset_type_t type, - boolean_t readonly, boolean_t decrypt, void *tag, objset_t **osp); + boolean_t readonly, boolean_t decrypt, const void *tag, objset_t **osp); int dmu_objset_own_obj(struct dsl_pool *dp, uint64_t obj, dmu_objset_type_t type, boolean_t readonly, boolean_t decrypt, - void *tag, objset_t **osp); + const void *tag, objset_t **osp); void dmu_objset_refresh_ownership(struct dsl_dataset *ds, - struct dsl_dataset **newds, boolean_t decrypt, void *tag); -void dmu_objset_rele(objset_t *os, void *tag); -void dmu_objset_rele_flags(objset_t *os, boolean_t decrypt, void *tag); -void dmu_objset_disown(objset_t *os, boolean_t decrypt, void *tag); + struct dsl_dataset **newds, boolean_t decrypt, const void *tag); +void dmu_objset_rele(objset_t *os, const void *tag); +void dmu_objset_rele_flags(objset_t *os, boolean_t decrypt, const void *tag); +void dmu_objset_disown(objset_t *os, boolean_t decrypt, const void *tag); int dmu_objset_from_ds(struct dsl_dataset *ds, objset_t **osp); void dmu_objset_stats(objset_t *os, nvlist_t *nv); diff --git a/include/sys/dmu_recv.h b/include/sys/dmu_recv.h index 1fdb986e2ed6..538c73610a59 100644 --- a/include/sys/dmu_recv.h +++ b/include/sys/dmu_recv.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -24,6 +24,7 @@ * Copyright (c) 2012, 2020 by Delphix. All rights reserved. * Copyright 2011 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2013, Joyent, Inc. All rights reserved. + * Copyright (c) 2019 Datto Inc. */ #ifndef _DMU_RECV_H @@ -47,6 +48,7 @@ typedef struct dmu_recv_cookie { boolean_t drc_byteswap; uint64_t drc_featureflags; boolean_t drc_force; + boolean_t drc_heal; boolean_t drc_resumable; boolean_t drc_should_save; boolean_t drc_raw; @@ -78,7 +80,7 @@ typedef struct dmu_recv_cookie { } dmu_recv_cookie_t; int dmu_recv_begin(char *, char *, dmu_replay_record_t *, - boolean_t, boolean_t, nvlist_t *, nvlist_t *, char *, + boolean_t, boolean_t, boolean_t, nvlist_t *, nvlist_t *, char *, dmu_recv_cookie_t *, zfs_file_t *, offset_t *); int dmu_recv_stream(dmu_recv_cookie_t *, offset_t *); int dmu_recv_end(dmu_recv_cookie_t *, void *); diff --git a/include/sys/dmu_redact.h b/include/sys/dmu_redact.h index 85f4b0522891..c18e2be103b7 100644 --- a/include/sys/dmu_redact.h +++ b/include/sys/dmu_redact.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/dmu_send.h b/include/sys/dmu_send.h index d150f816c945..061b81532fb1 100644 --- a/include/sys/dmu_send.h +++ b/include/sys/dmu_send.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/dmu_traverse.h b/include/sys/dmu_traverse.h index d76bfe3c9af3..7a0b38da7302 100644 --- a/include/sys/dmu_traverse.h +++ b/include/sys/dmu_traverse.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/dmu_tx.h b/include/sys/dmu_tx.h index 71a9ac7ca7bf..81e1ef6c1477 100644 --- a/include/sys/dmu_tx.h +++ b/include/sys/dmu_tx.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -124,8 +124,8 @@ typedef struct dmu_tx_stats { kstat_named_t dmu_tx_dirty_throttle; kstat_named_t dmu_tx_dirty_delay; kstat_named_t dmu_tx_dirty_over_max; - kstat_named_t dmu_tx_wrlog_over_max; kstat_named_t dmu_tx_dirty_frees_delay; + kstat_named_t dmu_tx_wrlog_delay; kstat_named_t dmu_tx_quota; } dmu_tx_stats_t; diff --git a/include/sys/dmu_zfetch.h b/include/sys/dmu_zfetch.h index 4c220b0c79e5..ad3bc040756c 100644 --- a/include/sys/dmu_zfetch.h +++ b/include/sys/dmu_zfetch.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -49,20 +49,18 @@ typedef struct zfetch { typedef struct zstream { uint64_t zs_blkid; /* expect next access at this blkid */ - uint64_t zs_pf_blkid1; /* first block to prefetch */ - uint64_t zs_pf_blkid; /* block to prefetch up to */ - - /* - * We will next prefetch the L1 indirect block of this level-0 - * block id. - */ - uint64_t zs_ipf_blkid1; /* first block to prefetch */ - uint64_t zs_ipf_blkid; /* block to prefetch up to */ + unsigned int zs_pf_dist; /* data prefetch distance in bytes */ + unsigned int zs_ipf_dist; /* L1 prefetch distance in bytes */ + uint64_t zs_pf_start; /* first data block to prefetch */ + uint64_t zs_pf_end; /* data block to prefetch up to */ + uint64_t zs_ipf_start; /* first data block to prefetch L1 */ + uint64_t zs_ipf_end; /* data block to prefetch L1 up to */ list_node_t zs_node; /* link for zf_stream */ hrtime_t zs_atime; /* time last prefetch issued */ zfetch_t *zs_fetch; /* parent fetch */ boolean_t zs_missed; /* stream saw cache misses */ + boolean_t zs_more; /* need more distant prefetch */ zfs_refcount_t zs_callers; /* number of pending callers */ /* * Number of stream references: dnode, callers and pending blocks. diff --git a/include/sys/dnode.h b/include/sys/dnode.h index 3f5fcc958c36..28d8847a1725 100644 --- a/include/sys/dnode.h +++ b/include/sys/dnode.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -334,7 +334,7 @@ struct dnode { kcondvar_t dn_notxholds; kcondvar_t dn_nodnholds; enum dnode_dirtycontext dn_dirtyctx; - void *dn_dirtyctx_firstset; /* dbg: contents meaningless */ + const void *dn_dirtyctx_firstset; /* dbg: contents meaningless */ /* protected by own devices */ zfs_refcount_t dn_tx_holds; @@ -418,16 +418,16 @@ void dnode_setbonus_type(dnode_t *dn, dmu_object_type_t, dmu_tx_t *tx); void dnode_rm_spill(dnode_t *dn, dmu_tx_t *tx); int dnode_hold(struct objset *dd, uint64_t object, - void *ref, dnode_t **dnp); + const void *ref, dnode_t **dnp); int dnode_hold_impl(struct objset *dd, uint64_t object, int flag, int dn_slots, - void *ref, dnode_t **dnp); -boolean_t dnode_add_ref(dnode_t *dn, void *ref); -void dnode_rele(dnode_t *dn, void *ref); -void dnode_rele_and_unlock(dnode_t *dn, void *tag, boolean_t evicting); + const void *ref, dnode_t **dnp); +boolean_t dnode_add_ref(dnode_t *dn, const void *ref); +void dnode_rele(dnode_t *dn, const void *ref); +void dnode_rele_and_unlock(dnode_t *dn, const void *tag, boolean_t evicting); int dnode_try_claim(objset_t *os, uint64_t object, int slots); boolean_t dnode_is_dirty(dnode_t *dn); void dnode_setdirty(dnode_t *dn, dmu_tx_t *tx); -void dnode_set_dirtyctx(dnode_t *dn, dmu_tx_t *tx, void *tag); +void dnode_set_dirtyctx(dnode_t *dn, dmu_tx_t *tx, const void *tag); void dnode_sync(dnode_t *dn, dmu_tx_t *tx); void dnode_allocate(dnode_t *dn, dmu_object_type_t ot, int blocksize, int ibs, dmu_object_type_t bonustype, int bonuslen, int dn_slots, dmu_tx_t *tx); @@ -616,7 +616,7 @@ extern dnode_stats_t dnode_stats; #else #define dprintf_dnode(db, fmt, ...) -#define DNODE_VERIFY(dn) +#define DNODE_VERIFY(dn) ((void) sizeof ((uintptr_t)(dn))) #define FREE_VERIFY(db, start, end, tx) #endif diff --git a/include/sys/dsl_bookmark.h b/include/sys/dsl_bookmark.h index 70f4813449aa..353c5c2d260f 100644 --- a/include/sys/dsl_bookmark.h +++ b/include/sys/dsl_bookmark.h @@ -114,7 +114,7 @@ typedef struct dsl_bookmark_create_redacted_arg { redaction_list_t **dbcra_rl; uint64_t dbcra_numsnaps; uint64_t *dbcra_snaps; - void *dbcra_tag; + const void *dbcra_tag; } dsl_bookmark_create_redacted_arg_t; int dsl_bookmark_create(nvlist_t *, nvlist_t *); @@ -122,7 +122,7 @@ int dsl_bookmark_create_nvl_validate(nvlist_t *); int dsl_bookmark_create_check(void *arg, dmu_tx_t *tx); void dsl_bookmark_create_sync(void *arg, dmu_tx_t *tx); int dsl_bookmark_create_redacted(const char *, const char *, uint64_t, - uint64_t *, void *, redaction_list_t **); + uint64_t *, const void *, redaction_list_t **); int dsl_get_bookmarks(const char *, nvlist_t *, nvlist_t *); int dsl_get_bookmarks_impl(dsl_dataset_t *, nvlist_t *, nvlist_t *); int dsl_get_bookmark_props(const char *, const char *, nvlist_t *); @@ -131,12 +131,12 @@ int dsl_bookmark_lookup(struct dsl_pool *, const char *, struct dsl_dataset *, zfs_bookmark_phys_t *); int dsl_bookmark_lookup_impl(dsl_dataset_t *, const char *, zfs_bookmark_phys_t *); -int dsl_redaction_list_hold_obj(struct dsl_pool *, uint64_t, void *, +int dsl_redaction_list_hold_obj(struct dsl_pool *, uint64_t, const void *, redaction_list_t **); -void dsl_redaction_list_rele(redaction_list_t *, void *); +void dsl_redaction_list_rele(redaction_list_t *, const void *); void dsl_redaction_list_long_hold(struct dsl_pool *, redaction_list_t *, - void *); -void dsl_redaction_list_long_rele(redaction_list_t *, void *); + const void *); +void dsl_redaction_list_long_rele(redaction_list_t *, const void *); boolean_t dsl_redaction_list_long_held(redaction_list_t *); int dsl_bookmark_init_ds(dsl_dataset_t *); void dsl_bookmark_fini_ds(dsl_dataset_t *); diff --git a/include/sys/dsl_crypt.h b/include/sys/dsl_crypt.h index 835720c87872..db594eece1c3 100644 --- a/include/sys/dsl_crypt.h +++ b/include/sys/dsl_crypt.h @@ -174,19 +174,20 @@ boolean_t dsl_dir_incompatible_encryption_version(dsl_dir_t *dd); void spa_keystore_init(spa_keystore_t *sk); void spa_keystore_fini(spa_keystore_t *sk); -void spa_keystore_dsl_key_rele(spa_t *spa, dsl_crypto_key_t *dck, void *tag); +void spa_keystore_dsl_key_rele(spa_t *spa, dsl_crypto_key_t *dck, + const void *tag); int spa_keystore_load_wkey_impl(spa_t *spa, dsl_wrapping_key_t *wkey); int spa_keystore_load_wkey(const char *dsname, dsl_crypto_params_t *dcp, boolean_t noop); int spa_keystore_unload_wkey_impl(spa_t *spa, uint64_t ddobj); int spa_keystore_unload_wkey(const char *dsname); -int spa_keystore_create_mapping(spa_t *spa, struct dsl_dataset *ds, void *tag, - dsl_key_mapping_t **km_out); -int spa_keystore_remove_mapping(spa_t *spa, uint64_t dsobj, void *tag); -void key_mapping_add_ref(dsl_key_mapping_t *km, void *tag); -void key_mapping_rele(spa_t *spa, dsl_key_mapping_t *km, void *tag); -int spa_keystore_lookup_key(spa_t *spa, uint64_t dsobj, void *tag, +int spa_keystore_create_mapping(spa_t *spa, struct dsl_dataset *ds, + const void *tag, dsl_key_mapping_t **km_out); +int spa_keystore_remove_mapping(spa_t *spa, uint64_t dsobj, const void *tag); +void key_mapping_add_ref(dsl_key_mapping_t *km, const void *tag); +void key_mapping_rele(spa_t *spa, dsl_key_mapping_t *km, const void *tag); +int spa_keystore_lookup_key(spa_t *spa, uint64_t dsobj, const void *tag, dsl_crypto_key_t **dck_out); int dsl_crypto_populate_key_nvlist(struct objset *os, diff --git a/include/sys/dsl_dataset.h b/include/sys/dsl_dataset.h index 02147171ae14..36307c63151f 100644 --- a/include/sys/dsl_dataset.h +++ b/include/sys/dsl_dataset.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -220,7 +220,7 @@ typedef struct dsl_dataset { kmutex_t ds_lock; objset_t *ds_objset; uint64_t ds_userrefs; - void *ds_owner; + const void *ds_owner; /* * Long holds prevent the ds from being destroyed; they allow the @@ -319,32 +319,34 @@ typedef enum ds_hold_flags { DS_HOLD_FLAG_DECRYPT = 1 << 0 /* needs access to encrypted data */ } ds_hold_flags_t; -int dsl_dataset_hold(struct dsl_pool *dp, const char *name, void *tag, +int dsl_dataset_hold(struct dsl_pool *dp, const char *name, const void *tag, dsl_dataset_t **dsp); int dsl_dataset_hold_flags(struct dsl_pool *dp, const char *name, - ds_hold_flags_t flags, void *tag, dsl_dataset_t **dsp); + ds_hold_flags_t flags, const void *tag, dsl_dataset_t **dsp); boolean_t dsl_dataset_try_add_ref(struct dsl_pool *dp, dsl_dataset_t *ds, - void *tag); + const void *tag); int dsl_dataset_create_key_mapping(dsl_dataset_t *ds); int dsl_dataset_hold_obj_flags(struct dsl_pool *dp, uint64_t dsobj, - ds_hold_flags_t flags, void *tag, dsl_dataset_t **); + ds_hold_flags_t flags, const void *tag, dsl_dataset_t **); void dsl_dataset_remove_key_mapping(dsl_dataset_t *ds); int dsl_dataset_hold_obj(struct dsl_pool *dp, uint64_t dsobj, - void *tag, dsl_dataset_t **); + const void *tag, dsl_dataset_t **); void dsl_dataset_rele_flags(dsl_dataset_t *ds, ds_hold_flags_t flags, - void *tag); -void dsl_dataset_rele(dsl_dataset_t *ds, void *tag); + const void *tag); +void dsl_dataset_rele(dsl_dataset_t *ds, const void *tag); int dsl_dataset_own(struct dsl_pool *dp, const char *name, - ds_hold_flags_t flags, void *tag, dsl_dataset_t **dsp); + ds_hold_flags_t flags, const void *tag, dsl_dataset_t **dsp); int dsl_dataset_own_force(struct dsl_pool *dp, const char *name, - ds_hold_flags_t flags, void *tag, dsl_dataset_t **dsp); + ds_hold_flags_t flags, const void *tag, dsl_dataset_t **dsp); int dsl_dataset_own_obj(struct dsl_pool *dp, uint64_t dsobj, - ds_hold_flags_t flags, void *tag, dsl_dataset_t **dsp); + ds_hold_flags_t flags, const void *tag, dsl_dataset_t **dsp); int dsl_dataset_own_obj_force(struct dsl_pool *dp, uint64_t dsobj, - ds_hold_flags_t flags, void *tag, dsl_dataset_t **dsp); -void dsl_dataset_disown(dsl_dataset_t *ds, ds_hold_flags_t flags, void *tag); + ds_hold_flags_t flags, const void *tag, dsl_dataset_t **dsp); +void dsl_dataset_disown(dsl_dataset_t *ds, ds_hold_flags_t flags, + const void *tag); void dsl_dataset_name(dsl_dataset_t *ds, char *name); -boolean_t dsl_dataset_tryown(dsl_dataset_t *ds, void *tag, boolean_t override); +boolean_t dsl_dataset_tryown(dsl_dataset_t *ds, const void *tag, + boolean_t override); int dsl_dataset_namelen(dsl_dataset_t *ds); boolean_t dsl_dataset_has_owner(dsl_dataset_t *ds); uint64_t dsl_dataset_create_sync(dsl_dir_t *pds, const char *lastname, @@ -373,6 +375,7 @@ boolean_t dsl_dataset_modified_since_snap(dsl_dataset_t *ds, void dsl_dataset_sync(dsl_dataset_t *ds, zio_t *zio, dmu_tx_t *tx); void dsl_dataset_sync_done(dsl_dataset_t *ds, dmu_tx_t *tx); +void dsl_dataset_feature_set_activation(const blkptr_t *bp, dsl_dataset_t *ds); void dsl_dataset_block_born(dsl_dataset_t *ds, const blkptr_t *bp, dmu_tx_t *tx); int dsl_dataset_block_kill(dsl_dataset_t *ds, const blkptr_t *bp, @@ -487,6 +490,9 @@ boolean_t dsl_dataset_get_uint64_array_feature(dsl_dataset_t *ds, void dsl_dataset_activate_redaction(dsl_dataset_t *ds, uint64_t *redact_snaps, uint64_t num_redact_snaps, dmu_tx_t *tx); +int dsl_dataset_oldest_snapshot(spa_t *spa, uint64_t head_ds, uint64_t min_txg, + uint64_t *oldest_dsobj); + #ifdef ZFS_DEBUG #define dprintf_ds(ds, fmt, ...) do { \ if (zfs_flags & ZFS_DEBUG_DPRINTF) { \ diff --git a/include/sys/dsl_deadlist.h b/include/sys/dsl_deadlist.h index 64358bb5fc0b..a94bba56ff7a 100644 --- a/include/sys/dsl_deadlist.h +++ b/include/sys/dsl_deadlist.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/dsl_deleg.h b/include/sys/dsl_deleg.h index 7f46233a889b..d6abac90bbcc 100644 --- a/include/sys/dsl_deleg.h +++ b/include/sys/dsl_deleg.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/dsl_destroy.h b/include/sys/dsl_destroy.h index 208d75bacffa..1a9b672a260b 100644 --- a/include/sys/dsl_destroy.h +++ b/include/sys/dsl_destroy.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/dsl_dir.h b/include/sys/dsl_dir.h index 993e44354475..f35b5500142c 100644 --- a/include/sys/dsl_dir.h +++ b/include/sys/dsl_dir.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -136,12 +136,12 @@ dsl_dir_phys(dsl_dir_t *dd) return (dd->dd_dbuf->db_data); } -void dsl_dir_rele(dsl_dir_t *dd, void *tag); -void dsl_dir_async_rele(dsl_dir_t *dd, void *tag); -int dsl_dir_hold(dsl_pool_t *dp, const char *name, void *tag, +void dsl_dir_rele(dsl_dir_t *dd, const void *tag); +void dsl_dir_async_rele(dsl_dir_t *dd, const void *tag); +int dsl_dir_hold(dsl_pool_t *dp, const char *name, const void *tag, dsl_dir_t **, const char **tail); int dsl_dir_hold_obj(dsl_pool_t *dp, uint64_t ddobj, - const char *tail, void *tag, dsl_dir_t **); + const char *tail, const void *tag, dsl_dir_t **); void dsl_dir_name(dsl_dir_t *dd, char *buf); int dsl_dir_namelen(dsl_dir_t *dd); uint64_t dsl_dir_create_sync(dsl_pool_t *dp, dsl_dir_t *pds, diff --git a/include/sys/dsl_pool.h b/include/sys/dsl_pool.h index 0283a8c589c1..226bb7f2c23a 100644 --- a/include/sys/dsl_pool.h +++ b/include/sys/dsl_pool.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -83,7 +83,6 @@ typedef struct zfs_blkstat { typedef struct zfs_all_blkstats { zfs_blkstat_t zab_type[DN_MAX_LEVELS + 1][DMU_OT_TOTAL + 1]; - kmutex_t zab_lock; } zfs_all_blkstats_t; @@ -162,8 +161,9 @@ int dsl_pool_sync_context(dsl_pool_t *dp); uint64_t dsl_pool_adjustedsize(dsl_pool_t *dp, zfs_space_check_t slop_policy); uint64_t dsl_pool_unreserved_space(dsl_pool_t *dp, zfs_space_check_t slop_policy); +uint64_t dsl_pool_deferred_space(dsl_pool_t *dp); void dsl_pool_wrlog_count(dsl_pool_t *dp, int64_t size, uint64_t txg); -boolean_t dsl_pool_wrlog_over_max(dsl_pool_t *dp); +boolean_t dsl_pool_need_wrlog_delay(dsl_pool_t *dp); void dsl_pool_dirty_space(dsl_pool_t *dp, int64_t space, dmu_tx_t *tx); void dsl_pool_undirty_space(dsl_pool_t *dp, int64_t space, uint64_t txg); void dsl_free(dsl_pool_t *dp, uint64_t txg, const blkptr_t *bpp); @@ -177,9 +177,9 @@ void dsl_pool_mos_diduse_space(dsl_pool_t *dp, void dsl_pool_ckpoint_diduse_space(dsl_pool_t *dp, int64_t used, int64_t comp, int64_t uncomp); boolean_t dsl_pool_need_dirty_delay(dsl_pool_t *dp); -void dsl_pool_config_enter(dsl_pool_t *dp, void *tag); -void dsl_pool_config_enter_prio(dsl_pool_t *dp, void *tag); -void dsl_pool_config_exit(dsl_pool_t *dp, void *tag); +void dsl_pool_config_enter(dsl_pool_t *dp, const void *tag); +void dsl_pool_config_enter_prio(dsl_pool_t *dp, const void *tag); +void dsl_pool_config_exit(dsl_pool_t *dp, const void *tag); boolean_t dsl_pool_config_held(dsl_pool_t *dp); boolean_t dsl_pool_config_held_writer(dsl_pool_t *dp); @@ -192,8 +192,8 @@ int dsl_pool_user_release(dsl_pool_t *dp, uint64_t dsobj, const char *tag, dmu_tx_t *tx); void dsl_pool_clean_tmp_userrefs(dsl_pool_t *dp); int dsl_pool_open_special_dir(dsl_pool_t *dp, const char *name, dsl_dir_t **); -int dsl_pool_hold(const char *name, void *tag, dsl_pool_t **dp); -void dsl_pool_rele(dsl_pool_t *dp, void *tag); +int dsl_pool_hold(const char *name, const void *tag, dsl_pool_t **dp); +void dsl_pool_rele(dsl_pool_t *dp, const void *tag); void dsl_pool_create_obsolete_bpobj(dsl_pool_t *dp, dmu_tx_t *tx); void dsl_pool_destroy_obsolete_bpobj(dsl_pool_t *dp, dmu_tx_t *tx); diff --git a/include/sys/dsl_prop.h b/include/sys/dsl_prop.h index fba8f908dc9e..7a84f2b6922a 100644 --- a/include/sys/dsl_prop.h +++ b/include/sys/dsl_prop.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/dsl_scan.h b/include/sys/dsl_scan.h index fb1f1d65bad4..8925b5815a37 100644 --- a/include/sys/dsl_scan.h +++ b/include/sys/dsl_scan.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -155,7 +155,7 @@ typedef struct dsl_scan { dsl_scan_phys_t scn_phys; /* on disk representation of scan */ dsl_scan_phys_t scn_phys_cached; avl_tree_t scn_queue; /* queue of datasets to scan */ - uint64_t scn_bytes_pending; /* outstanding data to issue */ + uint64_t scn_queues_pending; /* outstanding data to issue */ } dsl_scan_t; typedef struct dsl_scan_io_queue dsl_scan_io_queue_t; diff --git a/include/sys/dsl_synctask.h b/include/sys/dsl_synctask.h index 5a5b306419f1..cbdb20ec1d3a 100644 --- a/include/sys/dsl_synctask.h +++ b/include/sys/dsl_synctask.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/dsl_userhold.h b/include/sys/dsl_userhold.h index 071aeb86d1f1..c165edab3c53 100644 --- a/include/sys/dsl_userhold.h +++ b/include/sys/dsl_userhold.h @@ -7,7 +7,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/efi_partition.h b/include/sys/efi_partition.h index af3e98a07954..c4d7fd5088b5 100644 --- a/include/sys/efi_partition.h +++ b/include/sys/efi_partition.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -73,18 +73,15 @@ typedef struct efi_gpe_Attrs { RequiredPartition :1; } efi_gpe_Attrs_t; +/* MBR partition identification tags */ +#define V_UNASSIGNED 0x00 /* unassigned partition */ +#define V_USR 0x04 /* Usr filesystem */ +#define V_RESERVED 0x0b /* SMI reserved data */ + /* * 6a96237f-1dd2-11b2-99a6-080020736631 V_UNASSIGNED (not used as such) - * 6a82cb45-1dd2-11b2-99a6-080020736631 V_BOOT - * 6a85cf4d-1dd2-11b2-99a6-080020736631 V_ROOT - * 6a87c46f-1dd2-11b2-99a6-080020736631 V_SWAP * 6a898cc3-1dd2-11b2-99a6-080020736631 V_USR - * 6a8b642b-1dd2-11b2-99a6-080020736631 V_BACKUP - * 6a8d2ac7-1dd2-11b2-99a6-080020736631 V_STAND (not used) - * 6a8ef2e9-1dd2-11b2-99a6-080020736631 V_VAR - * 6a90ba39-1dd2-11b2-99a6-080020736631 V_HOME - * 6a9283a5-1dd2-11b2-99a6-080020736631 V_ALTSCTR - * 6a945a3b-1dd2-11b2-99a6-080020736631 V_CACHE + * 6a945a3b-1dd2-11b2-99a6-080020736631 V_RESERVED */ #define EFI_UNUSED { 0x00000000, 0x0000, 0x0000, 0x00, 0x00, \ @@ -363,6 +360,11 @@ struct partition64 { #endif #ifndef _KERNEL +#define VT_ERROR (-2) /* errno supplies specific error */ +#define VT_EIO (-3) /* I/O error accessing vtoc */ +#define VT_EINVAL (-4) /* illegal value in vtoc or request */ +#define VT_ENOSPC (-6) /* requested space not found */ + _SYS_EFI_PARTITION_H int efi_debug; _SYS_EFI_PARTITION_H int efi_alloc_and_init(int, uint32_t, struct dk_gpt **); _SYS_EFI_PARTITION_H int efi_alloc_and_read(int, struct dk_gpt **); diff --git a/include/sys/fm/Makefile.am b/include/sys/fm/Makefile.am deleted file mode 100644 index 7c6c3d49b6e9..000000000000 --- a/include/sys/fm/Makefile.am +++ /dev/null @@ -1,17 +0,0 @@ -SUBDIRS = fs - -COMMON_H = \ - protocol.h \ - util.h - -if CONFIG_USER -libzfsdir = $(includedir)/libzfs/sys/fm -libzfs_HEADERS = $(COMMON_H) -endif - -if CONFIG_KERNEL -if BUILD_LINUX -kerneldir = @prefix@/src/zfs-$(VERSION)/include/sys/fm -kernel_HEADERS = $(COMMON_H) -endif -endif diff --git a/include/sys/fm/fs/Makefile.am b/include/sys/fm/fs/Makefile.am deleted file mode 100644 index a662753a9e97..000000000000 --- a/include/sys/fm/fs/Makefile.am +++ /dev/null @@ -1,14 +0,0 @@ -COMMON_H = \ - zfs.h - -if CONFIG_USER -libzfsdir = $(includedir)/libzfs/sys/fm/fs -libzfs_HEADERS = $(COMMON_H) -endif - -if CONFIG_KERNEL -if BUILD_LINUX -kerneldir = @prefix@/src/zfs-$(VERSION)/include/sys/fm/fs -kernel_HEADERS = $(COMMON_H) -endif -endif diff --git a/include/sys/fm/fs/zfs.h b/include/sys/fm/fs/zfs.h index cd080c8ee667..97cb14aee36a 100644 --- a/include/sys/fm/fs/zfs.h +++ b/include/sys/fm/fs/zfs.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/fm/protocol.h b/include/sys/fm/protocol.h index 78031f7c15ec..d4a9751c8aeb 100644 --- a/include/sys/fm/protocol.h +++ b/include/sys/fm/protocol.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/fm/util.h b/include/sys/fm/util.h index 5fb6d1d6072b..7f91ca1da573 100644 --- a/include/sys/fm/util.h +++ b/include/sys/fm/util.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/fs/Makefile.am b/include/sys/fs/Makefile.am deleted file mode 100644 index 6a93053c8e2b..000000000000 --- a/include/sys/fs/Makefile.am +++ /dev/null @@ -1,14 +0,0 @@ -COMMON_H = \ - zfs.h - -if CONFIG_USER -libzfsdir = $(includedir)/libzfs/sys/fs -libzfs_HEADERS = $(COMMON_H) -endif - -if CONFIG_KERNEL -if BUILD_LINUX -kerneldir = @prefix@/src/zfs-$(VERSION)/include/sys/fs -kernel_HEADERS = $(COMMON_H) -endif -endif diff --git a/include/sys/fs/zfs.h b/include/sys/fs/zfs.h index e7fc2f459e18..237b503626d3 100644 --- a/include/sys/fs/zfs.h +++ b/include/sys/fs/zfs.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -50,6 +50,7 @@ extern "C" { * combined into masks that can be passed to various functions. */ typedef enum { + ZFS_TYPE_INVALID = 0, ZFS_TYPE_FILESYSTEM = (1 << 0), ZFS_TYPE_SNAPSHOT = (1 << 1), ZFS_TYPE_VOLUME = (1 << 2), @@ -92,6 +93,7 @@ typedef enum dmu_objset_type { typedef enum { ZPROP_CONT = -2, ZPROP_INVAL = -1, + ZPROP_USERPROP = ZPROP_INVAL, ZFS_PROP_TYPE = 0, ZFS_PROP_CREATION, ZFS_PROP_USED, @@ -309,7 +311,7 @@ typedef int (*zprop_func)(int, void *); */ typedef enum { VDEV_PROP_INVAL = -1, -#define VDEV_PROP_USER VDEV_PROP_INVAL + VDEV_PROP_USERPROP = VDEV_PROP_INVAL, VDEV_PROP_NAME, VDEV_PROP_CAPACITY, VDEV_PROP_STATE, @@ -1184,6 +1186,7 @@ typedef struct vdev_stat { uint64_t vs_logical_ashift; /* vdev_logical_ashift */ uint64_t vs_physical_ashift; /* vdev_physical_ashift */ uint64_t vs_noalloc; /* allocations halted? */ + uint64_t vs_pspace; /* physical capacity */ } vdev_stat_t; #define VDEV_STAT_VALID(field, uint64_t_field_count) \ @@ -1290,6 +1293,7 @@ typedef struct ddt_histogram { #define ZVOL_DRIVER "zvol" #define ZFS_DRIVER "zfs" #define ZFS_DEV "/dev/zfs" +#define ZFS_DEVDIR "/dev" #define ZFS_SUPER_MAGIC 0x2fc12fc1 @@ -1447,7 +1451,9 @@ typedef enum zfs_ioc { ZFS_IOC_EVENTS_SEEK, /* 0x83 (Linux) */ ZFS_IOC_NEXTBOOT, /* 0x84 (FreeBSD) */ ZFS_IOC_JAIL, /* 0x85 (FreeBSD) */ + ZFS_IOC_USERNS_ATTACH = ZFS_IOC_JAIL, /* 0x85 (Linux) */ ZFS_IOC_UNJAIL, /* 0x86 (FreeBSD) */ + ZFS_IOC_USERNS_DETACH = ZFS_IOC_UNJAIL, /* 0x86 (Linux) */ ZFS_IOC_SET_BOOTENV, /* 0x87 */ ZFS_IOC_GET_BOOTENV, /* 0x88 */ ZFS_IOC_LAST @@ -1458,6 +1464,41 @@ typedef enum zfs_ioc { */ #define BLKZNAME _IOR(0x12, 125, char[ZFS_MAX_DATASET_NAME_LEN]) +#ifdef __linux__ + +/* + * IOCTLs to update and retrieve additional file level attributes on + * Linux. + */ +#define ZFS_IOC_GETDOSFLAGS _IOR(0x83, 1, uint64_t) +#define ZFS_IOC_SETDOSFLAGS _IOW(0x83, 2, uint64_t) + +/* + * Additional file level attributes, that are stored + * in the upper half of z_pflags + */ +#define ZFS_READONLY 0x0000000100000000ull +#define ZFS_HIDDEN 0x0000000200000000ull +#define ZFS_SYSTEM 0x0000000400000000ull +#define ZFS_ARCHIVE 0x0000000800000000ull +#define ZFS_IMMUTABLE 0x0000001000000000ull +#define ZFS_NOUNLINK 0x0000002000000000ull +#define ZFS_APPENDONLY 0x0000004000000000ull +#define ZFS_NODUMP 0x0000008000000000ull +#define ZFS_OPAQUE 0x0000010000000000ull +#define ZFS_AV_QUARANTINED 0x0000020000000000ull +#define ZFS_AV_MODIFIED 0x0000040000000000ull +#define ZFS_REPARSE 0x0000080000000000ull +#define ZFS_OFFLINE 0x0000100000000000ull +#define ZFS_SPARSE 0x0000200000000000ull + +#define ZFS_DOS_FL_USER_VISIBLE (ZFS_IMMUTABLE | ZFS_APPENDONLY | \ + ZFS_NOUNLINK | ZFS_ARCHIVE | ZFS_NODUMP | ZFS_SYSTEM | \ + ZFS_HIDDEN | ZFS_READONLY | ZFS_REPARSE | ZFS_OFFLINE | \ + ZFS_SPARSE) + +#endif + /* * ZFS-specific error codes used for returning descriptive errors * to the userland through zfs ioctls. @@ -1493,6 +1534,7 @@ typedef enum { ZFS_ERR_REBUILD_IN_PROGRESS, ZFS_ERR_BADPROP, ZFS_ERR_VDEV_NOTSUP, + ZFS_ERR_NOT_USER_NAMESPACE, } zfs_errno_t; /* @@ -1709,7 +1751,6 @@ typedef enum { #define ZFS_EV_HIST_DSID "history_dsid" #define ZFS_EV_RESILVER_TYPE "resilver_type" - /* * We currently support block sizes from 512 bytes to 16MB. * The benefits of larger blocks, and thus larger IO, need to be weighed @@ -1731,7 +1772,6 @@ typedef enum { #define SPA_OLD_MAXBLOCKSIZE (1ULL << SPA_OLD_MAXBLOCKSHIFT) #define SPA_MAXBLOCKSIZE (1ULL << SPA_MAXBLOCKSHIFT) - /* supported encryption algorithms */ enum zio_encrypt { ZIO_CRYPT_INHERIT = 0, @@ -1749,6 +1789,34 @@ enum zio_encrypt { #define ZIO_CRYPT_ON_VALUE ZIO_CRYPT_AES_256_GCM #define ZIO_CRYPT_DEFAULT ZIO_CRYPT_OFF +/* + * xattr namespace prefixes. These are forbidden in xattr names. + * + * For cross-platform compatibility, xattrs in the user namespace should not be + * prefixed with the namespace name, but for backwards compatibility with older + * ZFS on Linux versions we do prefix the namespace. + */ +#define ZFS_XA_NS_FREEBSD_PREFIX "freebsd:" +#define ZFS_XA_NS_FREEBSD_PREFIX_LEN strlen("freebsd:") +#define ZFS_XA_NS_LINUX_SECURITY_PREFIX "security." +#define ZFS_XA_NS_LINUX_SECURITY_PREFIX_LEN strlen("security.") +#define ZFS_XA_NS_LINUX_SYSTEM_PREFIX "system." +#define ZFS_XA_NS_LINUX_SYSTEM_PREFIX_LEN strlen("system.") +#define ZFS_XA_NS_LINUX_TRUSTED_PREFIX "trusted." +#define ZFS_XA_NS_LINUX_TRUSTED_PREFIX_LEN strlen("trusted.") +#define ZFS_XA_NS_LINUX_USER_PREFIX "user." +#define ZFS_XA_NS_LINUX_USER_PREFIX_LEN strlen("user.") + +#define ZFS_XA_NS_PREFIX_MATCH(ns, name) \ + (strncmp(name, ZFS_XA_NS_##ns##_PREFIX, \ + ZFS_XA_NS_##ns##_PREFIX_LEN) == 0) + +#define ZFS_XA_NS_PREFIX_FORBIDDEN(name) \ + (ZFS_XA_NS_PREFIX_MATCH(FREEBSD, name) || \ + ZFS_XA_NS_PREFIX_MATCH(LINUX_SECURITY, name) || \ + ZFS_XA_NS_PREFIX_MATCH(LINUX_SYSTEM, name) || \ + ZFS_XA_NS_PREFIX_MATCH(LINUX_TRUSTED, name) || \ + ZFS_XA_NS_PREFIX_MATCH(LINUX_USER, name)) #ifdef __cplusplus } diff --git a/include/sys/lua/Makefile.am b/include/sys/lua/Makefile.am deleted file mode 100644 index 8b4dafaa8cf7..000000000000 --- a/include/sys/lua/Makefile.am +++ /dev/null @@ -1,17 +0,0 @@ -COMMON_H = \ - lua.h \ - luaconf.h \ - lualib.h \ - lauxlib.h - -if CONFIG_USER -libzfsdir = $(includedir)/libzfs/sys/lua -libzfs_HEADERS = $(COMMON_H) -endif - -if CONFIG_KERNEL -if BUILD_LINUX -kerneldir = @prefix@/src/zfs-$(VERSION)/include/sys/lua -kernel_HEADERS = $(COMMON_H) -endif -endif diff --git a/include/sys/metaslab.h b/include/sys/metaslab.h index 129a68be41c5..fec080139a2b 100644 --- a/include/sys/metaslab.h +++ b/include/sys/metaslab.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -49,11 +49,14 @@ int metaslab_init(metaslab_group_t *, uint64_t, uint64_t, uint64_t, metaslab_t **); void metaslab_fini(metaslab_t *); +void metaslab_set_unflushed_dirty(metaslab_t *, boolean_t); void metaslab_set_unflushed_txg(metaslab_t *, uint64_t, dmu_tx_t *); void metaslab_set_estimated_condensed_size(metaslab_t *, uint64_t, dmu_tx_t *); +boolean_t metaslab_unflushed_dirty(metaslab_t *); uint64_t metaslab_unflushed_txg(metaslab_t *); uint64_t metaslab_estimated_condensed_size(metaslab_t *); int metaslab_sort_by_flushed(const void *, const void *); +void metaslab_unflushed_bump(metaslab_t *, dmu_tx_t *, boolean_t); uint64_t metaslab_unflushed_changes_memused(metaslab_t *); int metaslab_load(metaslab_t *); @@ -128,9 +131,9 @@ uint64_t metaslab_group_get_space(metaslab_group_t *); void metaslab_group_histogram_verify(metaslab_group_t *); uint64_t metaslab_group_fragmentation(metaslab_group_t *); void metaslab_group_histogram_remove(metaslab_group_t *, metaslab_t *); -void metaslab_group_alloc_decrement(spa_t *, uint64_t, void *, int, int, +void metaslab_group_alloc_decrement(spa_t *, uint64_t, const void *, int, int, boolean_t); -void metaslab_group_alloc_verify(spa_t *, const blkptr_t *, void *, int); +void metaslab_group_alloc_verify(spa_t *, const blkptr_t *, const void *, int); void metaslab_recalculate_weight_and_sort(metaslab_t *); void metaslab_disable(metaslab_t *); void metaslab_enable(metaslab_t *, boolean_t, boolean_t); diff --git a/include/sys/metaslab_impl.h b/include/sys/metaslab_impl.h index 3dbee4c17fef..5beb1b737775 100644 --- a/include/sys/metaslab_impl.h +++ b/include/sys/metaslab_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -553,6 +553,7 @@ struct metaslab { * log space maps. */ uint64_t ms_unflushed_txg; + boolean_t ms_unflushed_dirty; /* updated every time we are done syncing the metaslab's space map */ uint64_t ms_synced_length; diff --git a/include/sys/mntent.h b/include/sys/mntent.h index 8d578f67b8a7..5bb7e080cda8 100644 --- a/include/sys/mntent.h +++ b/include/sys/mntent.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -108,5 +108,8 @@ #define MNTOPT_NOACL "noacl" /* likewise */ #define MNTOPT_POSIXACL "posixacl" /* likewise */ #define MNTOPT_MNTPOINT "mntpoint" /* mount point hint */ +#define MNTOPT_CASESENSITIVE "casesensitive" /* case sensitivity */ +#define MNTOPT_CASEINSENSITIVE "caseinsensitive" /* case insensitivity */ +#define MNTOPT_CASEMIXED "casemixed" /* case mixed */ #endif /* _SYS_MNTENT_H */ diff --git a/include/sys/mod.h b/include/sys/mod.h index a5a73ed0ee00..aba211423773 100644 --- a/include/sys/mod.h +++ b/include/sys/mod.h @@ -30,11 +30,6 @@ * Exported symbols */ #define EXPORT_SYMBOL(x) - -#define ZFS_MODULE_DESCRIPTION(s) -#define ZFS_MODULE_AUTHOR(s) -#define ZFS_MODULE_LICENSE(s) -#define ZFS_MODULE_VERSION(s) #endif #endif /* SYS_MOD_H */ diff --git a/include/sys/nvpair.h b/include/sys/nvpair.h index 81494b62d7ec..09b3a81f7c19 100644 --- a/include/sys/nvpair.h +++ b/include/sys/nvpair.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/nvpair_impl.h b/include/sys/nvpair_impl.h index 809e5c454712..6cae256285e5 100644 --- a/include/sys/nvpair_impl.h +++ b/include/sys/nvpair_impl.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/pathname.h b/include/sys/pathname.h index 52f21316c23d..054223170db1 100644 --- a/include/sys/pathname.h +++ b/include/sys/pathname.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/qat.h b/include/sys/qat.h index fe0f2c672f97..76360ba99042 100644 --- a/include/sys/qat.h +++ b/include/sys/qat.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/range_tree.h b/include/sys/range_tree.h index 895d802572d8..d6f60e795288 100644 --- a/include/sys/range_tree.h +++ b/include/sys/range_tree.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -64,11 +64,7 @@ typedef struct range_tree { uint8_t rt_shift; uint64_t rt_start; const range_tree_ops_t *rt_ops; - - /* rt_btree_compare should only be set if rt_arg is a b-tree */ void *rt_arg; - int (*rt_btree_compare)(const void *, const void *); - uint64_t rt_gap; /* allowable inter-segment gap */ /* @@ -278,9 +274,9 @@ rs_set_fill(range_seg_t *rs, range_tree_t *rt, uint64_t fill) typedef void range_tree_func_t(void *arg, uint64_t start, uint64_t size); -range_tree_t *range_tree_create_impl(const range_tree_ops_t *ops, +range_tree_t *range_tree_create_gap(const range_tree_ops_t *ops, range_seg_type_t type, void *arg, uint64_t start, uint64_t shift, - int (*zfs_btree_compare) (const void *, const void *), uint64_t gap); + uint64_t gap); range_tree_t *range_tree_create(const range_tree_ops_t *ops, range_seg_type_t type, void *arg, uint64_t start, uint64_t shift); void range_tree_destroy(range_tree_t *rt); @@ -316,13 +312,6 @@ void range_tree_remove_xor_add_segment(uint64_t start, uint64_t end, void range_tree_remove_xor_add(range_tree_t *rt, range_tree_t *removefrom, range_tree_t *addto); -void rt_btree_create(range_tree_t *rt, void *arg); -void rt_btree_destroy(range_tree_t *rt, void *arg); -void rt_btree_add(range_tree_t *rt, range_seg_t *rs, void *arg); -void rt_btree_remove(range_tree_t *rt, range_seg_t *rs, void *arg); -void rt_btree_vacate(range_tree_t *rt, void *arg); -extern const range_tree_ops_t rt_btree_ops; - #ifdef __cplusplus } #endif diff --git a/include/sys/rrwlock.h b/include/sys/rrwlock.h index 8d296ef28ffa..367732a8391c 100644 --- a/include/sys/rrwlock.h +++ b/include/sys/rrwlock.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -72,11 +72,11 @@ typedef struct rrwlock { */ void rrw_init(rrwlock_t *rrl, boolean_t track_all); void rrw_destroy(rrwlock_t *rrl); -void rrw_enter(rrwlock_t *rrl, krw_t rw, void *tag); -void rrw_enter_read(rrwlock_t *rrl, void *tag); -void rrw_enter_read_prio(rrwlock_t *rrl, void *tag); +void rrw_enter(rrwlock_t *rrl, krw_t rw, const void *tag); +void rrw_enter_read(rrwlock_t *rrl, const void *tag); +void rrw_enter_read_prio(rrwlock_t *rrl, const void *tag); void rrw_enter_write(rrwlock_t *rrl); -void rrw_exit(rrwlock_t *rrl, void *tag); +void rrw_exit(rrwlock_t *rrl, const void *tag); boolean_t rrw_held(rrwlock_t *rrl, krw_t rw); void rrw_tsd_destroy(void *arg); @@ -99,10 +99,10 @@ typedef struct rrmlock { void rrm_init(rrmlock_t *rrl, boolean_t track_all); void rrm_destroy(rrmlock_t *rrl); -void rrm_enter(rrmlock_t *rrl, krw_t rw, void *tag); -void rrm_enter_read(rrmlock_t *rrl, void *tag); +void rrm_enter(rrmlock_t *rrl, krw_t rw, const void *tag); +void rrm_enter_read(rrmlock_t *rrl, const void *tag); void rrm_enter_write(rrmlock_t *rrl); -void rrm_exit(rrmlock_t *rrl, void *tag); +void rrm_exit(rrmlock_t *rrl, const void *tag); boolean_t rrm_held(rrmlock_t *rrl, krw_t rw); #define RRM_READ_HELD(x) rrm_held(x, RW_READER) diff --git a/include/sys/sa.h b/include/sys/sa.h index 32f6bd0ccd95..c551acecab30 100644 --- a/include/sys/sa.h +++ b/include/sys/sa.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -49,7 +49,7 @@ typedef uint16_t sa_attr_type_t; * Attribute to register support for. */ typedef struct sa_attr_reg { - char *sa_name; /* attribute name */ + const char *sa_name; /* attribute name */ uint16_t sa_length; sa_bswap_type_t sa_byteswap; /* bswap function enum */ sa_attr_type_t sa_attr; /* filled in during registration */ @@ -124,8 +124,8 @@ int sa_handle_get(objset_t *, uint64_t, void *userp, int sa_handle_get_from_db(objset_t *, dmu_buf_t *, void *userp, sa_handle_type_t, sa_handle_t **); void sa_handle_destroy(sa_handle_t *); -int sa_buf_hold(objset_t *, uint64_t, void *, dmu_buf_t **); -void sa_buf_rele(dmu_buf_t *, void *); +int sa_buf_hold(objset_t *, uint64_t, const void *, dmu_buf_t **); +void sa_buf_rele(dmu_buf_t *, const void *); int sa_lookup(sa_handle_t *, sa_attr_type_t, void *buf, uint32_t buflen); int sa_update(sa_handle_t *, sa_attr_type_t, void *buf, uint32_t buflen, dmu_tx_t *); diff --git a/include/sys/sa_impl.h b/include/sys/sa_impl.h index fa10aff8a306..744c8dcb7dfb 100644 --- a/include/sys/sa_impl.h +++ b/include/sys/sa_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/spa.h b/include/sys/spa.h index 896b0f9563ae..e185ce6b1d8e 100644 --- a/include/sys/spa.h +++ b/include/sys/spa.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -26,10 +26,10 @@ * Copyright 2013 Saso Kiselkov. All rights reserved. * Copyright (c) 2014 Integros [integros.com] * Copyright 2017 Joyent, Inc. - * Copyright (c) 2017, 2019, Datto Inc. All rights reserved. * Copyright (c) 2017, Intel Corporation. * Copyright (c) 2019, Allan Jude * Copyright (c) 2019, Klara Inc. + * Copyright (c) 2019, Datto Inc. */ #ifndef _SYS_SPA_H @@ -606,7 +606,7 @@ typedef struct blkptr { #define SNPRINTF_BLKPTR(func, ws, buf, size, bp, type, checksum, compress) \ { \ - static const char *copyname[] = \ + static const char *const copyname[] = \ { "zero", "single", "double", "triple" }; \ int len = 0; \ int copies = 0; \ @@ -743,8 +743,8 @@ typedef enum trim_type { } trim_type_t; /* state manipulation functions */ -extern int spa_open(const char *pool, spa_t **, void *tag); -extern int spa_open_rewind(const char *pool, spa_t **, void *tag, +extern int spa_open(const char *pool, spa_t **, const void *tag); +extern int spa_open_rewind(const char *pool, spa_t **, const void *tag, nvlist_t *policy, nvlist_t **config); extern int spa_get_stats(const char *pool, nvlist_t **config, char *altroot, size_t buflen); @@ -801,8 +801,8 @@ extern int spa_vdev_trim(spa_t *spa, nvlist_t *nv, uint64_t cmd_type, uint64_t rate, boolean_t partial, boolean_t secure, nvlist_t *vdev_errlist); extern int spa_vdev_setpath(spa_t *spa, uint64_t guid, const char *newpath); extern int spa_vdev_setfru(spa_t *spa, uint64_t guid, const char *newfru); -extern int spa_vdev_split_mirror(spa_t *spa, char *newname, nvlist_t *config, - nvlist_t *props, boolean_t exp); +extern int spa_vdev_split_mirror(spa_t *spa, const char *newname, + nvlist_t *config, nvlist_t *props, boolean_t exp); /* spare state (which is global across all pools) */ extern void spa_spare_add(vdev_t *vd); @@ -860,9 +860,9 @@ extern void spa_remove(spa_t *spa); extern spa_t *spa_next(spa_t *prev); /* Refcount functions */ -extern void spa_open_ref(spa_t *spa, void *tag); -extern void spa_close(spa_t *spa, void *tag); -extern void spa_async_close(spa_t *spa, void *tag); +extern void spa_open_ref(spa_t *spa, const void *tag); +extern void spa_close(spa_t *spa, const void *tag); +extern void spa_async_close(spa_t *spa, const void *tag); extern boolean_t spa_refcount_zero(spa_t *spa); #define SCL_NONE 0x00 @@ -898,6 +898,7 @@ typedef struct spa_stats { spa_history_kstat_t tx_assign_histogram; spa_history_list_t mmp_history; spa_history_kstat_t state; /* pool state */ + spa_history_kstat_t guid; /* pool guid */ spa_history_kstat_t iostats; } spa_stats_t; @@ -970,7 +971,8 @@ extern int spa_import_progress_set_state(uint64_t pool_guid, spa_load_state_t spa_load_state); /* Pool configuration locks */ -extern int spa_config_tryenter(spa_t *spa, int locks, void *tag, krw_t rw); +extern int spa_config_tryenter(spa_t *spa, int locks, const void *tag, + krw_t rw); extern void spa_config_enter(spa_t *spa, int locks, const void *tag, krw_t rw); extern void spa_config_exit(spa_t *spa, int locks, const void *tag); extern int spa_config_held(spa_t *spa, int locks, krw_t rw); @@ -980,7 +982,7 @@ extern uint64_t spa_vdev_enter(spa_t *spa); extern uint64_t spa_vdev_detach_enter(spa_t *spa, uint64_t guid); extern uint64_t spa_vdev_config_enter(spa_t *spa); extern void spa_vdev_config_exit(spa_t *spa, vdev_t *vd, uint64_t txg, - int error, char *tag); + int error, const char *tag); extern int spa_vdev_exit(spa_t *spa, vdev_t *vd, uint64_t txg, int error); /* Pool vdev state change lock */ @@ -1132,6 +1134,7 @@ extern const char *spa_state_to_name(spa_t *spa); /* error handling */ struct zbookmark_phys; extern void spa_log_error(spa_t *spa, const zbookmark_phys_t *zb); +extern void spa_remove_error(spa_t *spa, zbookmark_phys_t *zb); extern int zfs_ereport_post(const char *clazz, spa_t *spa, vdev_t *vd, const zbookmark_phys_t *zb, zio_t *zio, uint64_t state); extern boolean_t zfs_ereport_is_valid(const char *clazz, spa_t *spa, vdev_t *vd, @@ -1144,11 +1147,17 @@ extern void zfs_post_remove(spa_t *spa, vdev_t *vd); extern void zfs_post_state_change(spa_t *spa, vdev_t *vd, uint64_t laststate); extern void zfs_post_autoreplace(spa_t *spa, vdev_t *vd); extern uint64_t spa_get_errlog_size(spa_t *spa); -extern int spa_get_errlog(spa_t *spa, void *uaddr, size_t *count); +extern int spa_get_errlog(spa_t *spa, void *uaddr, uint64_t *count); extern void spa_errlog_rotate(spa_t *spa); extern void spa_errlog_drain(spa_t *spa); extern void spa_errlog_sync(spa_t *spa, uint64_t txg); extern void spa_get_errlists(spa_t *spa, avl_tree_t *last, avl_tree_t *scrub); +extern void spa_delete_dataset_errlog(spa_t *spa, uint64_t ds, dmu_tx_t *tx); +extern void spa_swap_errlog(spa_t *spa, uint64_t new_head_ds, + uint64_t old_head_ds, dmu_tx_t *tx); +extern void sync_error_list(spa_t *spa, avl_tree_t *t, uint64_t *obj, + dmu_tx_t *tx); +extern void spa_upgrade_errlog(spa_t *spa, dmu_tx_t *tx); /* vdev cache */ extern void vdev_cache_stat_init(void); @@ -1183,6 +1192,11 @@ extern int spa_wait_tag(const char *name, zpool_wait_activity_t activity, extern void spa_notify_waiters(spa_t *spa); extern void spa_wake_waiters(spa_t *spa); +extern void spa_import_os(spa_t *spa); +extern void spa_export_os(spa_t *spa); +extern void spa_activate_os(spa_t *spa); +extern void spa_deactivate_os(spa_t *spa); + /* module param call functions */ int param_set_deadman_ziotime(ZFS_MODULE_PARAM_ARGS); int param_set_deadman_synctime(ZFS_MODULE_PARAM_ARGS); diff --git a/include/sys/spa_boot.h b/include/sys/spa_boot.h index 1d3622f5a108..4a69efdda945 100644 --- a/include/sys/spa_boot.h +++ b/include/sys/spa_boot.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/spa_checkpoint.h b/include/sys/spa_checkpoint.h index 9be2b6eeab3c..e4475ff35f44 100644 --- a/include/sys/spa_checkpoint.h +++ b/include/sys/spa_checkpoint.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/spa_checksum.h b/include/sys/spa_checksum.h index b87990105a71..2202afdeb8da 100644 --- a/include/sys/spa_checksum.h +++ b/include/sys/spa_checksum.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/spa_impl.h b/include/sys/spa_impl.h index 9946c4e3c316..469b1266e453 100644 --- a/include/sys/spa_impl.h +++ b/include/sys/spa_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -25,8 +25,8 @@ * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. * Copyright 2013 Saso Kiselkov. All rights reserved. * Copyright (c) 2016 Actifio, Inc. All rights reserved. - * Copyright (c) 2017 Datto Inc. * Copyright (c) 2017, Intel Corporation. + * Copyright (c) 2019 Datto Inc. */ #ifndef _SYS_SPA_IMPL_H @@ -349,6 +349,7 @@ struct spa { kmutex_t spa_errlist_lock; /* error list/ereport lock */ avl_tree_t spa_errlist_last; /* last error list */ avl_tree_t spa_errlist_scrub; /* scrub error list */ + avl_tree_t spa_errlist_healed; /* list of healed blocks */ uint64_t spa_deflate; /* should we deflate? */ uint64_t spa_history; /* history object */ kmutex_t spa_history_lock; /* history lock */ diff --git a/include/sys/spa_log_spacemap.h b/include/sys/spa_log_spacemap.h index b2ed77fac3e4..f59e69917833 100644 --- a/include/sys/spa_log_spacemap.h +++ b/include/sys/spa_log_spacemap.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -30,7 +30,10 @@ typedef struct log_summary_entry { uint64_t lse_start; /* start TXG */ + uint64_t lse_end; /* last TXG */ + uint64_t lse_txgcount; /* # of TXGs */ uint64_t lse_mscount; /* # of metaslabs needed to be flushed */ + uint64_t lse_msdcount; /* # of dirty metaslabs needed to be flushed */ uint64_t lse_blkcount; /* blocks held by this entry */ list_node_t lse_node; } log_summary_entry_t; @@ -50,6 +53,7 @@ typedef struct spa_log_sm { uint64_t sls_nblocks; /* number of blocks in this log */ uint64_t sls_mscount; /* # of metaslabs flushed in the log's txg */ avl_node_t sls_node; /* node in spa_sm_logs_by_txg */ + space_map_t *sls_sm; /* space map pointer, if open */ } spa_log_sm_t; int spa_ld_log_spacemaps(spa_t *); @@ -68,8 +72,9 @@ uint64_t spa_log_sm_memused(spa_t *); void spa_log_sm_decrement_mscount(spa_t *, uint64_t); void spa_log_sm_increment_current_mscount(spa_t *); -void spa_log_summary_add_flushed_metaslab(spa_t *); -void spa_log_summary_decrement_mscount(spa_t *, uint64_t); +void spa_log_summary_add_flushed_metaslab(spa_t *, boolean_t); +void spa_log_summary_dirty_flushed_metaslab(spa_t *, uint64_t); +void spa_log_summary_decrement_mscount(spa_t *, uint64_t, boolean_t); void spa_log_summary_decrement_blkcount(spa_t *, uint64_t); boolean_t spa_flush_all_logs_requested(spa_t *); diff --git a/include/sys/space_map.h b/include/sys/space_map.h index cb81e710bd1e..14c5beccee55 100644 --- a/include/sys/space_map.h +++ b/include/sys/space_map.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/space_reftree.h b/include/sys/space_reftree.h index ca9d41dc1388..b7a846aec624 100644 --- a/include/sys/space_reftree.h +++ b/include/sys/space_reftree.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/sysevent.h b/include/sys/sysevent.h index 6510297d601f..f8ae17497366 100644 --- a/include/sys/sysevent.h +++ b/include/sys/sysevent.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/sysevent/Makefile.am b/include/sys/sysevent/Makefile.am deleted file mode 100644 index 64e53763951a..000000000000 --- a/include/sys/sysevent/Makefile.am +++ /dev/null @@ -1,15 +0,0 @@ -COMMON_H = \ - eventdefs.h \ - dev.h - -if CONFIG_USER -libzfsdir = $(includedir)/libzfs/sys/sysevent -libzfs_HEADERS = $(COMMON_H) -endif - -if CONFIG_KERNEL -if BUILD_LINUX -kerneldir = @prefix@/src/zfs-$(VERSION)/include/sys/sysevent -kernel_HEADERS = $(COMMON_H) -endif -endif diff --git a/include/sys/sysevent/dev.h b/include/sys/sysevent/dev.h index 1117538d822d..da6539b4a0d5 100644 --- a/include/sys/sysevent/dev.h +++ b/include/sys/sysevent/dev.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/sysevent/eventdefs.h b/include/sys/sysevent/eventdefs.h index 2067b355afb4..eb1dfd16c0fd 100644 --- a/include/sys/sysevent/eventdefs.h +++ b/include/sys/sysevent/eventdefs.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/txg.h b/include/sys/txg.h index f38f0006c040..c0ee348a61bc 100644 --- a/include/sys/txg.h +++ b/include/sys/txg.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/txg_impl.h b/include/sys/txg_impl.h index 047d51b94c66..45fde2e1f351 100644 --- a/include/sys/txg_impl.h +++ b/include/sys/txg_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/u8_textprep.h b/include/sys/u8_textprep.h index 09ab13af268c..e82037de4fe4 100644 --- a/include/sys/u8_textprep.h +++ b/include/sys/u8_textprep.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/u8_textprep_data.h b/include/sys/u8_textprep_data.h index 03f71f26c9e1..2a97966ee56e 100644 --- a/include/sys/u8_textprep_data.h +++ b/include/sys/u8_textprep_data.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/uberblock.h b/include/sys/uberblock.h index 044e438387c0..ff3a8c81232a 100644 --- a/include/sys/uberblock.h +++ b/include/sys/uberblock.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/uberblock_impl.h b/include/sys/uberblock_impl.h index 91699e65131a..03bcfa8f4dd1 100644 --- a/include/sys/uberblock_impl.h +++ b/include/sys/uberblock_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/uio_impl.h b/include/sys/uio_impl.h index cde3ef40485b..aa34edda5f6a 100644 --- a/include/sys/uio_impl.h +++ b/include/sys/uio_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/unique.h b/include/sys/unique.h index d4ba32e5c642..bc7944657521 100644 --- a/include/sys/unique.h +++ b/include/sys/unique.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/uuid.h b/include/sys/uuid.h index eab4622a6d9a..19f1baa4432e 100644 --- a/include/sys/uuid.h +++ b/include/sys/uuid.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/vdev.h b/include/sys/vdev.h index 4e507d081957..5fec1d51a5f2 100644 --- a/include/sys/vdev.h +++ b/include/sys/vdev.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -104,7 +104,7 @@ extern void vdev_metaslab_fini(vdev_t *vd); extern void vdev_metaslab_set_size(vdev_t *); extern void vdev_expand(vdev_t *vd, uint64_t txg); extern void vdev_split(vdev_t *vd); -extern void vdev_deadman(vdev_t *vd, char *tag); +extern void vdev_deadman(vdev_t *vd, const char *tag); typedef void vdev_xlate_func_t(void *arg, range_seg64_t *physical_rs); diff --git a/include/sys/vdev_disk.h b/include/sys/vdev_disk.h index a7e19fbf0c4b..02c583777ebc 100644 --- a/include/sys/vdev_disk.h +++ b/include/sys/vdev_disk.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/vdev_draid.h b/include/sys/vdev_draid.h index dd334acbacf1..a204f1e3c74a 100644 --- a/include/sys/vdev_draid.h +++ b/include/sys/vdev_draid.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/vdev_file.h b/include/sys/vdev_file.h index 1514a44fcabb..fddecbfe1ab5 100644 --- a/include/sys/vdev_file.h +++ b/include/sys/vdev_file.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/vdev_impl.h b/include/sys/vdev_impl.h index db8fbdeb06df..d22abfbc2598 100644 --- a/include/sys/vdev_impl.h +++ b/include/sys/vdev_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/vdev_initialize.h b/include/sys/vdev_initialize.h index 81d39ebebcb2..4e63f063cb66 100644 --- a/include/sys/vdev_initialize.h +++ b/include/sys/vdev_initialize.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/vdev_raidz.h b/include/sys/vdev_raidz.h index c7cf0af6d945..e34b6e4b158e 100644 --- a/include/sys/vdev_raidz.h +++ b/include/sys/vdev_raidz.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/vdev_raidz_impl.h b/include/sys/vdev_raidz_impl.h index 890e725e18d8..12f5eff22c62 100644 --- a/include/sys/vdev_raidz_impl.h +++ b/include/sys/vdev_raidz_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/vdev_rebuild.h b/include/sys/vdev_rebuild.h index b59fbe153903..c4cfe0c56762 100644 --- a/include/sys/vdev_rebuild.h +++ b/include/sys/vdev_rebuild.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/vdev_trim.h b/include/sys/vdev_trim.h index 16f4be2a41f8..2a217f0d43ce 100644 --- a/include/sys/vdev_trim.h +++ b/include/sys/vdev_trim.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/xvattr.h b/include/sys/xvattr.h index 0463bdfbc9d8..a7994db894b9 100644 --- a/include/sys/xvattr.h +++ b/include/sys/xvattr.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -40,7 +40,7 @@ #define _SYS_XVATTR_H #include -#include +#include #define AV_SCANSTAMP_SZ 32 /* length of anti-virus scanstamp */ @@ -282,7 +282,7 @@ typedef struct xvattr { static inline void xva_init(xvattr_t *xvap) { - bzero(xvap, sizeof (xvattr_t)); + memset(xvap, 0, sizeof (xvattr_t)); xvap->xva_mapsize = XVA_MAPSIZE; xvap->xva_magic = XVA_MAGIC; xvap->xva_vattr.va_mask = ATTR_XVATTR; diff --git a/include/sys/zap.h b/include/sys/zap.h index b19b4643879c..308a7c7284d7 100644 --- a/include/sys/zap.h +++ b/include/sys/zap.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -110,7 +110,12 @@ typedef enum zap_flags { * already randomly distributed. */ ZAP_FLAG_PRE_HASHED_KEY = 1 << 2, +#if defined(__linux__) && defined(_KERNEL) +} zfs_zap_flags_t; +#define zap_flags_t zfs_zap_flags_t +#else } zap_flags_t; +#endif /* * Create a new zapobj with no attributes and return its object number. @@ -134,7 +139,7 @@ uint64_t zap_create_flags_dnsize(objset_t *os, int normflags, uint64_t zap_create_hold(objset_t *os, int normflags, zap_flags_t flags, dmu_object_type_t ot, int leaf_blockshift, int indirect_blockshift, dmu_object_type_t bonustype, int bonuslen, int dnodesize, - dnode_t **allocated_dnode, void *tag, dmu_tx_t *tx); + dnode_t **allocated_dnode, const void *tag, dmu_tx_t *tx); uint64_t zap_create_link(objset_t *os, dmu_object_type_t ot, uint64_t parent_obj, const char *name, dmu_tx_t *tx); diff --git a/include/sys/zap_impl.h b/include/sys/zap_impl.h index 250dde3ce235..fab677964e7c 100644 --- a/include/sys/zap_impl.h +++ b/include/sys/zap_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -199,8 +199,9 @@ typedef struct zap_name { boolean_t zap_match(zap_name_t *zn, const char *matchname); int zap_lockdir(objset_t *os, uint64_t obj, dmu_tx_t *tx, - krw_t lti, boolean_t fatreader, boolean_t adding, void *tag, zap_t **zapp); -void zap_unlockdir(zap_t *zap, void *tag); + krw_t lti, boolean_t fatreader, boolean_t adding, const void *tag, + zap_t **zapp); +void zap_unlockdir(zap_t *zap, const void *tag); void zap_evict_sync(void *dbu); zap_name_t *zap_name_alloc(zap_t *zap, const char *key, matchtype_t mt); void zap_name_free(zap_name_t *zn); @@ -217,10 +218,10 @@ int fzap_lookup(zap_name_t *zn, char *realname, int rn_len, boolean_t *normalization_conflictp); void fzap_prefetch(zap_name_t *zn); int fzap_add(zap_name_t *zn, uint64_t integer_size, uint64_t num_integers, - const void *val, void *tag, dmu_tx_t *tx); + const void *val, const void *tag, dmu_tx_t *tx); int fzap_update(zap_name_t *zn, int integer_size, uint64_t num_integers, const void *val, - void *tag, dmu_tx_t *tx); + const void *tag, dmu_tx_t *tx); int fzap_length(zap_name_t *zn, uint64_t *integer_size, uint64_t *num_integers); int fzap_remove(zap_name_t *zn, dmu_tx_t *tx); @@ -230,7 +231,7 @@ void zap_put_leaf(struct zap_leaf *l); int fzap_add_cd(zap_name_t *zn, uint64_t integer_size, uint64_t num_integers, - const void *val, uint32_t cd, void *tag, dmu_tx_t *tx); + const void *val, uint32_t cd, const void *tag, dmu_tx_t *tx); void fzap_upgrade(zap_t *zap, dmu_tx_t *tx, zap_flags_t flags); #ifdef __cplusplus diff --git a/include/sys/zap_leaf.h b/include/sys/zap_leaf.h index a3da1036a5ee..ebc67c2bf465 100644 --- a/include/sys/zap_leaf.h +++ b/include/sys/zap_leaf.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zcp.h b/include/sys/zcp.h index d7b1dfaa2ea5..f0a78f9cb5c4 100644 --- a/include/sys/zcp.h +++ b/include/sys/zcp.h @@ -177,7 +177,7 @@ void zcp_parse_args(lua_State *, const char *, const zcp_arg_t *, int zcp_nvlist_to_lua(lua_State *, nvlist_t *, char *, int); int zcp_dataset_hold_error(lua_State *, dsl_pool_t *, const char *, int); struct dsl_dataset *zcp_dataset_hold(lua_State *, dsl_pool_t *, - const char *, void *); + const char *, const void *); typedef int (zcp_lib_func_t)(lua_State *); typedef struct zcp_lib_info { diff --git a/include/sys/zcp_iter.h b/include/sys/zcp_iter.h index 1d92d0c6d10c..fa6eeef25edd 100644 --- a/include/sys/zcp_iter.h +++ b/include/sys/zcp_iter.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfeature.h b/include/sys/zfeature.h index 5abde149a615..bf9361374d33 100644 --- a/include/sys/zfeature.h +++ b/include/sys/zfeature.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfs_acl.h b/include/sys/zfs_acl.h index 98387a49adbe..c4d2dddd7b1f 100644 --- a/include/sys/zfs_acl.h +++ b/include/sys/zfs_acl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/stropts.h b/include/sys/zfs_chksum.h similarity index 63% rename from lib/libspl/include/sys/stropts.h rename to include/sys/zfs_chksum.h index 08c2e79bc53c..a0e1b35189bb 100644 --- a/lib/libspl/include/sys/stropts.h +++ b/include/sys/zfs_chksum.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -18,12 +18,31 @@ * * CDDL HEADER END */ + /* - * Copyright 2010 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. + * Copyright (c) 2021 Tino Reichardt */ -#ifndef _LIBSPL_SYS_STROPTS_H -#define _LIBSPL_SYS_STROPTS_H +#ifndef _ZFS_CHKSUM_H +#define _ZFS_CHKSUM_H + +#ifdef _KERNEL +#include +#else +#include +#include +#endif + +#ifdef __cplusplus +extern "C" { +#endif + +/* Benchmark the chksums of ZFS when the module is loading */ +void chksum_init(void); +void chksum_fini(void); + +#ifdef __cplusplus +} +#endif -#endif /* _LIBSPL_SYS_STROPTS_H */ +#endif /* _ZFS_CHKSUM_H */ diff --git a/include/sys/zfs_context.h b/include/sys/zfs_context.h index d53b2c3a0baa..aa4f78789631 100644 --- a/include/sys/zfs_context.h +++ b/include/sys/zfs_context.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -55,7 +55,7 @@ extern "C" { #include #include #include -#include +#include #include #include #include @@ -91,7 +91,6 @@ extern "C" { #include #include #include -#include #include #include #include @@ -153,8 +152,8 @@ extern void dprintf_setup(int *argc, char **argv); extern void cmn_err(int, const char *, ...); extern void vcmn_err(int, const char *, va_list); -extern void panic(const char *, ...) __NORETURN; -extern void vpanic(const char *, va_list) __NORETURN; +extern __attribute__((noreturn)) void panic(const char *, ...); +extern __attribute__((noreturn)) void vpanic(const char *, va_list); #define fm_panic panic @@ -509,7 +508,6 @@ extern taskq_t *taskq_of_curthread(void); extern int taskq_cancel_id(taskq_t *, taskqid_t); extern void system_taskq_init(void); extern void system_taskq_fini(void); -extern boolean_t taskq_empty(taskq_t *); #define XVA_MAPSIZE 3 #define XVA_MAGIC 0x78766174 @@ -694,10 +692,6 @@ extern char *kmem_asprintf(const char *fmt, ...); /* * Hostname information */ -extern char hw_serial[]; /* for userland-emulated hostid access */ -extern int ddi_strtoul(const char *str, char **nptr, int base, - unsigned long *result); - extern int ddi_strtoull(const char *str, char **nptr, int base, u_longlong_t *result); diff --git a/include/sys/zfs_debug.h b/include/sys/zfs_debug.h index 7b103510dd07..481209b241aa 100644 --- a/include/sys/zfs_debug.h +++ b/include/sys/zfs_debug.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfs_delay.h b/include/sys/zfs_delay.h index 40e617dba961..56ac1f3c439b 100644 --- a/include/sys/zfs_delay.h +++ b/include/sys/zfs_delay.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfs_file.h b/include/sys/zfs_file.h index 02cd1a6f041a..e944165adc40 100644 --- a/include/sys/zfs_file.h +++ b/include/sys/zfs_file.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfs_fuid.h b/include/sys/zfs_fuid.h index b5b37db29468..d6b2942d1bec 100644 --- a/include/sys/zfs_fuid.h +++ b/include/sys/zfs_fuid.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -112,15 +112,13 @@ extern zfs_fuid_info_t *zfs_fuid_info_alloc(void); extern void zfs_fuid_info_free(zfs_fuid_info_t *); extern boolean_t zfs_groupmember(zfsvfs_t *, uint64_t, cred_t *); void zfs_fuid_sync(zfsvfs_t *, dmu_tx_t *); -extern int zfs_fuid_find_by_domain(zfsvfs_t *, const char *domain, - char **retdomain, boolean_t addok); extern const char *zfs_fuid_find_by_idx(zfsvfs_t *zfsvfs, uint32_t idx); extern void zfs_fuid_txhold(zfsvfs_t *zfsvfs, dmu_tx_t *tx); extern int zfs_id_to_fuidstr(zfsvfs_t *zfsvfs, const char *domain, uid_t rid, char *buf, size_t len, boolean_t addok); #endif -char *zfs_fuid_idx_domain(avl_tree_t *, uint32_t); +const char *zfs_fuid_idx_domain(avl_tree_t *, uint32_t); void zfs_fuid_avl_tree_create(avl_tree_t *, avl_tree_t *); uint64_t zfs_fuid_table_load(objset_t *, uint64_t, avl_tree_t *, avl_tree_t *); void zfs_fuid_table_destroy(avl_tree_t *, avl_tree_t *); diff --git a/include/sys/zfs_ioctl.h b/include/sys/zfs_ioctl.h index 4fb15636ecb8..a44c96364418 100644 --- a/include/sys/zfs_ioctl.h +++ b/include/sys/zfs_ioctl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -124,6 +124,7 @@ typedef enum drr_headertype { * default use of "zfs send" won't encounter the bug mentioned above. */ #define DMU_BACKUP_FEATURE_SWITCH_TO_LARGE_BLOCKS (1 << 27) +#define DMU_BACKUP_FEATURE_BLAKE3 (1 << 28) /* * Mask of all supported backup features @@ -134,7 +135,7 @@ typedef enum drr_headertype { DMU_BACKUP_FEATURE_COMPRESSED | DMU_BACKUP_FEATURE_LARGE_DNODE | \ DMU_BACKUP_FEATURE_RAW | DMU_BACKUP_FEATURE_HOLDS | \ DMU_BACKUP_FEATURE_REDACTED | DMU_BACKUP_FEATURE_SWITCH_TO_LARGE_BLOCKS | \ - DMU_BACKUP_FEATURE_ZSTD) + DMU_BACKUP_FEATURE_ZSTD | DMU_BACKUP_FEATURE_BLAKE3) /* Are all features in the given flag word currently supported? */ #define DMU_STREAM_SUPPORTED(x) (!((x) & ~DMU_BACKUP_FEATURE_MASK)) diff --git a/include/sys/zfs_ioctl_impl.h b/include/sys/zfs_ioctl_impl.h index f9e4f6e6c4b2..0bf9fa6ff193 100644 --- a/include/sys/zfs_ioctl_impl.h +++ b/include/sys/zfs_ioctl_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfs_onexit.h b/include/sys/zfs_onexit.h index fd3030e3ac2d..18930fe01c7a 100644 --- a/include/sys/zfs_onexit.h +++ b/include/sys/zfs_onexit.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfs_project.h b/include/sys/zfs_project.h index 81a238905225..8a46e5e068db 100644 --- a/include/sys/zfs_project.h +++ b/include/sys/zfs_project.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfs_quota.h b/include/sys/zfs_quota.h index b215b8dd0013..4567cc651afb 100644 --- a/include/sys/zfs_quota.h +++ b/include/sys/zfs_quota.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfs_racct.h b/include/sys/zfs_racct.h index cfcdd336ea42..0e8bd04c1a13 100644 --- a/include/sys/zfs_racct.h +++ b/include/sys/zfs_racct.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfs_refcount.h b/include/sys/zfs_refcount.h index 2f59ebb32b07..42f846b8920a 100644 --- a/include/sys/zfs_refcount.h +++ b/include/sys/zfs_refcount.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfs_rlock.h b/include/sys/zfs_rlock.h index 2302abb37337..5e5d6d68d6c5 100644 --- a/include/sys/zfs_rlock.h +++ b/include/sys/zfs_rlock.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfs_sa.h b/include/sys/zfs_sa.h index a0c383807aa4..1b4b8abf0244 100644 --- a/include/sys/zfs_sa.h +++ b/include/sys/zfs_sa.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -138,7 +138,7 @@ void zfs_sa_symlink(struct znode *, char *link, int len, dmu_tx_t *); void zfs_sa_get_scanstamp(struct znode *, xvattr_t *); void zfs_sa_set_scanstamp(struct znode *, xvattr_t *, dmu_tx_t *); int zfs_sa_get_xattr(struct znode *); -int zfs_sa_set_xattr(struct znode *); +int zfs_sa_set_xattr(struct znode *, const char *, const void *, size_t); void zfs_sa_upgrade(struct sa_handle *, dmu_tx_t *); void zfs_sa_upgrade_txholds(dmu_tx_t *, struct znode *); void zfs_sa_init(void); diff --git a/include/sys/zfs_stat.h b/include/sys/zfs_stat.h index 465aefaa2063..1589f945cbd7 100644 --- a/include/sys/zfs_stat.h +++ b/include/sys/zfs_stat.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfs_sysfs.h b/include/sys/zfs_sysfs.h index d1cb2ef4321c..6fe9b7a9cd2c 100644 --- a/include/sys/zfs_sysfs.h +++ b/include/sys/zfs_sysfs.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfs_vfsops.h b/include/sys/zfs_vfsops.h index a438c86f0a0c..19ae7b77b459 100644 --- a/include/sys/zfs_vfsops.h +++ b/include/sys/zfs_vfsops.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfs_vnops.h b/include/sys/zfs_vnops.h index 18259f0dc9b5..edff8f681dd4 100644 --- a/include/sys/zfs_vnops.h +++ b/include/sys/zfs_vnops.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zfs_znode.h b/include/sys/zfs_znode.h index 1bf25a77d3a0..b223c4b3b30b 100644 --- a/include/sys/zfs_znode.h +++ b/include/sys/zfs_znode.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -37,7 +37,7 @@ extern "C" { /* * Additional file level attributes, that are stored - * in the upper half of zp_flags + * in the upper half of z_pflags */ #define ZFS_READONLY 0x0000000100000000ull #define ZFS_HIDDEN 0x0000000200000000ull @@ -199,6 +199,8 @@ typedef struct znode { uint64_t z_size; /* file size (cached) */ uint64_t z_pflags; /* pflags (cached) */ uint32_t z_sync_cnt; /* synchronous open count */ + uint32_t z_sync_writes_cnt; /* synchronous write count */ + uint32_t z_async_writes_cnt; /* asynchronous write count */ mode_t z_mode; /* mode (cached) */ kmutex_t z_acl_lock; /* acl data lock */ zfs_acl_t *z_acl_cached; /* cached acl */ @@ -286,6 +288,8 @@ extern void zfs_log_acl(zilog_t *zilog, dmu_tx_t *tx, znode_t *zp, vsecattr_t *vsecp, zfs_fuid_info_t *fuidp); extern void zfs_xvattr_set(znode_t *zp, xvattr_t *xvap, dmu_tx_t *tx); extern void zfs_upgrade(zfsvfs_t *zfsvfs, dmu_tx_t *tx); +extern void zfs_log_setsaxattr(zilog_t *zilog, dmu_tx_t *tx, int txtype, + znode_t *zp, const char *name, const void *value, size_t size); extern void zfs_znode_update_vfs(struct znode *); diff --git a/include/sys/zil.h b/include/sys/zil.h index 8e5a49da2929..cec04f120ce3 100644 --- a/include/sys/zil.h +++ b/include/sys/zil.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -33,6 +33,7 @@ #include #include #include +#include #ifdef __cplusplus extern "C" { @@ -162,7 +163,8 @@ typedef enum zil_create { #define TX_MKDIR_ATTR 18 /* mkdir with attr */ #define TX_MKDIR_ACL_ATTR 19 /* mkdir with ACL + attrs */ #define TX_WRITE2 20 /* dmu_sync EALREADY write */ -#define TX_MAX_TYPE 21 /* Max transaction type */ +#define TX_SETSAXATTR 21 /* Set sa xattrs on file */ +#define TX_MAX_TYPE 22 /* Max transaction type */ /* * The transactions for mkdir, symlink, remove, rmdir, link, and rename @@ -182,7 +184,8 @@ typedef enum zil_create { (txtype) == TX_SETATTR || \ (txtype) == TX_ACL_V0 || \ (txtype) == TX_ACL || \ - (txtype) == TX_WRITE2) + (txtype) == TX_WRITE2 || \ + (txtype) == TX_SETSAXATTR) /* * The number of dnode slots consumed by the object is stored in the 8 @@ -221,6 +224,15 @@ typedef struct { uint64_t lr_foid; /* object id */ } lr_ooo_t; +/* + * Additional lr_attr_t fields. + */ +typedef struct { + uint64_t lr_attr_attrs; /* all of the attributes */ + uint64_t lr_attr_crtime[2]; /* create time */ + uint8_t lr_attr_scanstamp[32]; +} lr_attr_end_t; + /* * Handle option extended vattr attributes. * @@ -231,7 +243,7 @@ typedef struct { typedef struct { uint32_t lr_attr_masksize; /* number of elements in array */ uint32_t lr_attr_bitmap; /* First entry of array */ - /* remainder of array and any additional fields */ + /* remainder of array and additional lr_attr_end_t fields */ } lr_attr_t; /* @@ -335,6 +347,13 @@ typedef struct { /* optional attribute lr_attr_t may be here */ } lr_setattr_t; +typedef struct { + lr_t lr_common; /* common portion of log record */ + uint64_t lr_foid; /* file object to change attributes */ + uint64_t lr_size; + /* xattr name and value follows */ +} lr_setsaxattr_t; + typedef struct { lr_t lr_common; /* common portion of log record */ uint64_t lr_foid; /* obj id of file */ @@ -454,12 +473,34 @@ typedef struct zil_stats { */ kstat_named_t zil_itx_metaslab_slog_count; kstat_named_t zil_itx_metaslab_slog_bytes; -} zil_stats_t; - -#define ZIL_STAT_INCR(stat, val) \ - atomic_add_64(&zil_stats.stat.value.ui64, (val)); -#define ZIL_STAT_BUMP(stat) \ - ZIL_STAT_INCR(stat, 1); +} zil_kstat_values_t; + +typedef struct zil_sums { + wmsum_t zil_commit_count; + wmsum_t zil_commit_writer_count; + wmsum_t zil_itx_count; + wmsum_t zil_itx_indirect_count; + wmsum_t zil_itx_indirect_bytes; + wmsum_t zil_itx_copied_count; + wmsum_t zil_itx_copied_bytes; + wmsum_t zil_itx_needcopy_count; + wmsum_t zil_itx_needcopy_bytes; + wmsum_t zil_itx_metaslab_normal_count; + wmsum_t zil_itx_metaslab_normal_bytes; + wmsum_t zil_itx_metaslab_slog_count; + wmsum_t zil_itx_metaslab_slog_bytes; +} zil_sums_t; + +#define ZIL_STAT_INCR(zil, stat, val) \ + do { \ + int64_t tmpval = (val); \ + wmsum_add(&(zil_sums_global.stat), tmpval); \ + if ((zil)->zl_sums) \ + wmsum_add(&((zil)->zl_sums->stat), tmpval); \ + } while (0) + +#define ZIL_STAT_BUMP(zil, stat) \ + ZIL_STAT_INCR(zil, stat, 1); typedef int zil_parse_blk_func_t(zilog_t *zilog, const blkptr_t *bp, void *arg, uint64_t txg); @@ -479,7 +520,8 @@ extern void zil_fini(void); extern zilog_t *zil_alloc(objset_t *os, zil_header_t *zh_phys); extern void zil_free(zilog_t *zilog); -extern zilog_t *zil_open(objset_t *os, zil_get_data_t *get_data); +extern zilog_t *zil_open(objset_t *os, zil_get_data_t *get_data, + zil_sums_t *zil_sums); extern void zil_close(zilog_t *zilog); extern void zil_replay(objset_t *os, void *arg, @@ -519,6 +561,11 @@ extern void zil_set_logbias(zilog_t *zilog, uint64_t slogval); extern uint64_t zil_max_copied_data(zilog_t *zilog); extern uint64_t zil_max_log_data(zilog_t *zilog); +extern void zil_sums_init(zil_sums_t *zs); +extern void zil_sums_fini(zil_sums_t *zs); +extern void zil_kstat_values_update(zil_kstat_values_t *zs, + zil_sums_t *zil_sums); + extern int zil_replay_disable; #ifdef __cplusplus diff --git a/include/sys/zil_impl.h b/include/sys/zil_impl.h index d2f4018653a6..bb85bf6d1eb1 100644 --- a/include/sys/zil_impl.h +++ b/include/sys/zil_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -99,7 +99,7 @@ typedef struct lwb { char *lwb_buf; /* log write buffer */ zio_t *lwb_write_zio; /* zio for the lwb buffer */ zio_t *lwb_root_zio; /* root zio for lwb write and flushes */ - dmu_tx_t *lwb_tx; /* tx for log block allocation */ + uint64_t lwb_issued_txg; /* the txg when the write is issued */ uint64_t lwb_max_txg; /* highest txg in this lwb */ list_node_t lwb_node; /* zilog->zl_lwb_list linkage */ list_t lwb_itxs; /* list of itx's */ @@ -209,6 +209,12 @@ struct zilog { uint_t zl_prev_rotor; /* rotor for zl_prev[] */ txg_node_t zl_dirty_link; /* protected by dp_dirty_zilogs list */ uint64_t zl_dirty_max_txg; /* highest txg used to dirty zilog */ + + kmutex_t zl_lwb_io_lock; /* protect following members */ + uint64_t zl_lwb_inflight[TXG_SIZE]; /* io issued, but not done */ + kcondvar_t zl_lwb_io_cv; /* signal when the flush is done */ + uint64_t zl_lwb_max_issued_txg; /* max txg when lwb io issued */ + /* * Max block size for this ZIL. Note that this can not be changed * while the ZIL is in use because consumers (ZPL/zvol) need to take @@ -216,6 +222,9 @@ struct zilog { * (see zil_max_copied_data()). */ uint64_t zl_max_block_size; + + /* Pointer for per dataset zil sums */ + zil_sums_t *zl_sums; }; typedef struct zil_bp_node { diff --git a/include/sys/zio.h b/include/sys/zio.h index 07135d1e2a07..23fdda457bc3 100644 --- a/include/sys/zio.h +++ b/include/sys/zio.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -89,6 +89,7 @@ enum zio_checksum { ZIO_CHECKSUM_SHA512, ZIO_CHECKSUM_SKEIN, ZIO_CHECKSUM_EDONR, + ZIO_CHECKSUM_BLAKE3, ZIO_CHECKSUM_FUNCTIONS }; @@ -125,7 +126,7 @@ enum zio_checksum { #define ZIO_COMPRESS_LEGACY_ON_VALUE ZIO_COMPRESS_LZJB #define ZIO_COMPRESS_LZ4_ON_VALUE ZIO_COMPRESS_LZ4 -#define ZIO_COMPRESS_DEFAULT ZIO_COMPRESS_OFF +#define ZIO_COMPRESS_DEFAULT ZIO_COMPRESS_ON #define BOOTFS_COMPRESS_VALID(compress) \ ((compress) == ZIO_COMPRESS_LZJB || \ @@ -283,6 +284,13 @@ extern const char *const zio_type_name[ZIO_TYPES]; * Note: this structure is passed between userland and the kernel, and is * stored on disk (by virtue of being incorporated into other on-disk * structures, e.g. dsl_scan_phys_t). + * + * If the head_errlog feature is enabled a different on-disk format for error + * logs is used. This introduces the use of an error bookmark, a four-tuple + * that uniquely identifies any error block + * in the pool. The birth transaction group is used to track whether the block + * has been overwritten by newer data or added to a snapshot since its marking + * as an error. */ struct zbookmark_phys { uint64_t zb_objset; @@ -291,6 +299,13 @@ struct zbookmark_phys { uint64_t zb_blkid; }; +typedef struct zbookmark_err_phys { + uint64_t zb_object; + int64_t zb_level; + uint64_t zb_blkid; + uint64_t zb_birth; +} zbookmark_err_phys_t; + #define SET_BOOKMARK(zb, objset, object, level, blkid) \ { \ (zb)->zb_objset = objset; \ @@ -519,6 +534,8 @@ extern zio_t *zio_null(zio_t *pio, spa_t *spa, vdev_t *vd, extern zio_t *zio_root(spa_t *spa, zio_done_func_t *done, void *priv, enum zio_flag flags); +extern void zio_destroy(zio_t *zio); + extern zio_t *zio_read(zio_t *pio, spa_t *spa, const blkptr_t *bp, struct abd *data, uint64_t lsize, zio_done_func_t *done, void *priv, zio_priority_t priority, enum zio_flag flags, const zbookmark_phys_t *zb); @@ -574,7 +591,7 @@ extern void zio_execute(void *zio); extern void zio_interrupt(void *zio); extern void zio_delay_init(zio_t *zio); extern void zio_delay_interrupt(zio_t *zio); -extern void zio_deadman(zio_t *zio, char *tag); +extern void zio_deadman(zio_t *zio, const char *tag); extern zio_t *zio_walk_parents(zio_t *cio, zio_link_t **); extern zio_t *zio_walk_children(zio_t *pio, zio_link_t **); @@ -642,7 +659,8 @@ extern int zio_inject_fault(char *name, int flags, int *id, extern int zio_inject_list_next(int *id, char *name, size_t buflen, struct zinject_record *record); extern int zio_clear_fault(int id); -extern void zio_handle_panic_injection(spa_t *spa, char *tag, uint64_t type); +extern void zio_handle_panic_injection(spa_t *spa, const char *tag, + uint64_t type); extern int zio_handle_decrypt_injection(spa_t *spa, const zbookmark_phys_t *zb, uint64_t type, int error); extern int zio_handle_fault_injection(zio_t *zio, int error); @@ -680,6 +698,8 @@ extern void spa_handle_ignored_writes(spa_t *spa); /* zbookmark_phys functions */ boolean_t zbookmark_subtree_completed(const struct dnode_phys *dnp, const zbookmark_phys_t *subtree_root, const zbookmark_phys_t *last_block); +boolean_t zbookmark_subtree_tbd(const struct dnode_phys *dnp, + const zbookmark_phys_t *subtree_root, const zbookmark_phys_t *last_block); int zbookmark_compare(uint16_t dbss1, uint8_t ibs1, uint16_t dbss2, uint8_t ibs2, const zbookmark_phys_t *zb1, const zbookmark_phys_t *zb2); diff --git a/include/sys/zio_checksum.h b/include/sys/zio_checksum.h index 9a73a626229b..84e371d6c3ef 100644 --- a/include/sys/zio_checksum.h +++ b/include/sys/zio_checksum.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -21,7 +21,8 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2014, 2016 by Delphix. All rights reserved. - * Copyright Saso Kiselkov 2013, All rights reserved. + * Copyright (c) 2013 Saso Kiselkov, All rights reserved. + * Copyright (c) 2021 Tino Reichardt */ #ifndef _SYS_ZIO_CHECKSUM_H @@ -89,7 +90,7 @@ typedef const struct zio_checksum_info { zio_checksum_tmpl_init_t *ci_tmpl_init; zio_checksum_tmpl_free_t *ci_tmpl_free; zio_checksum_flags_t ci_flags; - char *ci_name; /* descriptive name */ + const char *ci_name; /* descriptive name */ } zio_checksum_info_t; typedef struct zio_bad_cksum { @@ -101,12 +102,14 @@ typedef struct zio_bad_cksum { uint8_t zbc_has_cksum; /* expected/actual valid */ } zio_bad_cksum_t; -_SYS_ZIO_CHECKSUM_H zio_checksum_info_t +_SYS_ZIO_CHECKSUM_H const zio_checksum_info_t zio_checksum_table[ZIO_CHECKSUM_FUNCTIONS]; /* * Checksum routines. */ + +/* SHA2 */ extern zio_checksum_t abd_checksum_SHA256; extern zio_checksum_t abd_checksum_SHA512_native; extern zio_checksum_t abd_checksum_SHA512_byteswap; @@ -123,6 +126,13 @@ extern zio_checksum_t abd_checksum_edonr_byteswap; extern zio_checksum_tmpl_init_t abd_checksum_edonr_tmpl_init; extern zio_checksum_tmpl_free_t abd_checksum_edonr_tmpl_free; +/* BLAKE3 */ +extern zio_checksum_t abd_checksum_blake3_native; +extern zio_checksum_t abd_checksum_blake3_byteswap; +extern zio_checksum_tmpl_init_t abd_checksum_blake3_tmpl_init; +extern zio_checksum_tmpl_free_t abd_checksum_blake3_tmpl_free; + +/* Fletcher 4 */ _SYS_ZIO_CHECKSUM_H zio_abd_checksum_func_t fletcher_4_abd_ops; extern zio_checksum_t abd_fletcher_4_native; extern zio_checksum_t abd_fletcher_4_byteswap; diff --git a/include/sys/zio_compress.h b/include/sys/zio_compress.h index 4a22ad2a2742..a736d8091986 100644 --- a/include/sys/zio_compress.h +++ b/include/sys/zio_compress.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -145,14 +145,14 @@ typedef int zio_decompress_abd_func_t(abd_t *src, void *dst, * Information about each compression function. */ typedef const struct zio_compress_info { - char *ci_name; + const char *ci_name; int ci_level; zio_compress_func_t *ci_compress; zio_decompress_func_t *ci_decompress; zio_decompresslevel_func_t *ci_decompress_level; } zio_compress_info_t; -extern zio_compress_info_t zio_compress_table[ZIO_COMPRESS_FUNCTIONS]; +extern const zio_compress_info_t zio_compress_table[ZIO_COMPRESS_FUNCTIONS]; /* * lz4 compression init & free diff --git a/include/sys/zio_crypt.h b/include/sys/zio_crypt.h index f1edd76f0d8e..6a3efabb0405 100644 --- a/include/sys/zio_crypt.h +++ b/include/sys/zio_crypt.h @@ -67,7 +67,7 @@ typedef struct zio_crypt_info { size_t ci_keylen; /* human-readable name of the encryption algorithm */ - char *ci_name; + const char *ci_name; } zio_crypt_info_t; extern const zio_crypt_info_t zio_crypt_table[ZIO_CRYPT_FUNCTIONS]; diff --git a/include/sys/zio_impl.h b/include/sys/zio_impl.h index 4c998571653a..199cca291edf 100644 --- a/include/sys/zio_impl.h +++ b/include/sys/zio_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zrlock.h b/include/sys/zrlock.h index b6eba1a18ff4..cddaae52dc38 100644 --- a/include/sys/zrlock.h +++ b/include/sys/zrlock.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zstd/Makefile.am b/include/sys/zstd/Makefile.am deleted file mode 100644 index 16666fe63355..000000000000 --- a/include/sys/zstd/Makefile.am +++ /dev/null @@ -1,18 +0,0 @@ -COMMON_H = \ - $(top_srcdir)/include/sys/zstd/zstd.h - -KERNEL_H = - -USER_H = - -EXTRA_DIST = $(COMMON_H) $(KERNEL_H) $(USER_H) - -if CONFIG_USER -libzfsdir = $(includedir)/libzfs/sys/zstd -libzfs_HEADERS = $(COMMON_H) $(USER_H) -endif - -if CONFIG_KERNEL -kerneldir = @prefix@/src/zfs-$(VERSION)/include/sys/zstd -kernel_HEADERS = $(COMMON_H) $(KERNEL_H) -endif diff --git a/include/sys/zstd/zstd.h b/include/sys/zstd/zstd.h index ca32a7464556..d8c3fa86dce3 100644 --- a/include/sys/zstd/zstd.h +++ b/include/sys/zstd/zstd.h @@ -78,6 +78,8 @@ typedef struct zfs_zstd_meta { * kstat helper macros */ #define ZSTDSTAT(stat) (zstd_stats.stat.value.ui64) +#define ZSTDSTAT_ZERO(stat) \ + atomic_store_64(&zstd_stats.stat.value.ui64, 0) #define ZSTDSTAT_ADD(stat, val) \ atomic_add_64(&zstd_stats.stat.value.ui64, (val)) #define ZSTDSTAT_SUB(stat, val) \ @@ -90,6 +92,8 @@ void zstd_fini(void); size_t zfs_zstd_compress(void *s_start, void *d_start, size_t s_len, size_t d_len, int level); +size_t zfs_zstd_compress_wrap(void *s_start, void *d_start, size_t s_len, + size_t d_len, int level); int zfs_zstd_get_level(void *s_start, size_t s_len, uint8_t *level); int zfs_zstd_decompress_level(void *s_start, void *d_start, size_t s_len, size_t d_len, uint8_t *level); diff --git a/include/sys/zvol.h b/include/sys/zvol.h index a0f18001304e..15a3034aefb5 100644 --- a/include/sys/zvol.h +++ b/include/sys/zvol.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/sys/zvol_impl.h b/include/sys/zvol_impl.h index 94203347066b..3243917bcd3f 100644 --- a/include/sys/zvol_impl.h +++ b/include/sys/zvol_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/thread_pool.h b/include/thread_pool.h index 43090c3c6644..2e0aed5f7f57 100644 --- a/include/thread_pool.h +++ b/include/thread_pool.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -28,7 +28,6 @@ #define _THREAD_POOL_H_ extern __attribute__((visibility("default"))) #include -#include #include #ifdef __cplusplus diff --git a/include/zfeature_common.h b/include/zfeature_common.h index 874cbd9ff714..0930bc900f82 100644 --- a/include/zfeature_common.h +++ b/include/zfeature_common.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -75,6 +75,9 @@ typedef enum spa_feature { SPA_FEATURE_DEVICE_REBUILD, SPA_FEATURE_ZSTD_COMPRESS, SPA_FEATURE_DRAID, + SPA_FEATURE_ZILSAXATTR, + SPA_FEATURE_HEAD_ERRLOG, + SPA_FEATURE_BLAKE3, SPA_FEATURES } spa_feature_t; diff --git a/include/zfs_comutil.h b/include/zfs_comutil.h index ea2da4a15f1d..25f0959e00b1 100644 --- a/include/zfs_comutil.h +++ b/include/zfs_comutil.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -34,7 +34,7 @@ extern "C" { #endif _ZFS_COMUTIL_H boolean_t zfs_allocatable_devs(nvlist_t *); -_ZFS_COMUTIL_H boolean_t zfs_special_devs(nvlist_t *, char *); +_ZFS_COMUTIL_H boolean_t zfs_special_devs(nvlist_t *, const char *); _ZFS_COMUTIL_H void zpool_get_load_policy(nvlist_t *, zpool_load_policy_t *); _ZFS_COMUTIL_H int zfs_zpl_version_map(int spa_version); diff --git a/include/zfs_deleg.h b/include/zfs_deleg.h index 77f64786b089..92331b4eeba5 100644 --- a/include/zfs_deleg.h +++ b/include/zfs_deleg.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -81,7 +81,7 @@ typedef enum { } zfs_deleg_note_t; typedef struct zfs_deleg_perm_tab { - char *z_perm; + const char *z_perm; zfs_deleg_note_t z_note; } zfs_deleg_perm_tab_t; diff --git a/include/zfs_fletcher.h b/include/zfs_fletcher.h index 28cb8ba234e5..db5b539648a6 100644 --- a/include/zfs_fletcher.h +++ b/include/zfs_fletcher.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/zfs_namecheck.h b/include/zfs_namecheck.h index 4739b065c51c..d3a923909c69 100644 --- a/include/zfs_namecheck.h +++ b/include/zfs_namecheck.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/include/zfs_prop.h b/include/zfs_prop.h index c7363f701530..3513c4b40932 100644 --- a/include/zfs_prop.h +++ b/include/zfs_prop.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -76,10 +76,11 @@ typedef struct { /* fs | vol | snap; or pool */ const char *pd_values; /* string telling acceptable values */ const char *pd_colname; /* column header for "zfs list" */ - boolean_t pd_rightalign; /* column alignment for "zfs list" */ - boolean_t pd_visible; /* do we list this property with the */ + boolean_t pd_rightalign: 1; /* column alignment for "zfs list" */ + boolean_t pd_visible: 1; /* do we list this property with the */ /* "zfs get" help message */ - boolean_t pd_zfs_mod_supported; /* supported by running zfs module */ + boolean_t pd_zfs_mod_supported: 1; /* supported by running zfs module */ + boolean_t pd_always_flex: 1; /* never fixed-width */ const zprop_index_t *pd_table; /* for index properties, a table */ /* defining the possible values */ size_t pd_table_size; /* number of entries in pd_table[] */ @@ -112,19 +113,20 @@ _ZFS_PROP_H zprop_desc_t *vdev_prop_get_table(void); */ _ZFS_PROP_H void zprop_register_impl(int, const char *, zprop_type_t, uint64_t, const char *, zprop_attr_t, int, const char *, const char *, - boolean_t, boolean_t, const zprop_index_t *, + boolean_t, boolean_t, boolean_t, const zprop_index_t *, const struct zfs_mod_supported_features *); _ZFS_PROP_H void zprop_register_string(int, const char *, const char *, zprop_attr_t attr, int, const char *, const char *, const struct zfs_mod_supported_features *); _ZFS_PROP_H void zprop_register_number(int, const char *, uint64_t, - zprop_attr_t, int, const char *, const char *, + zprop_attr_t, int, const char *, const char *, boolean_t, const struct zfs_mod_supported_features *); _ZFS_PROP_H void zprop_register_index(int, const char *, uint64_t, zprop_attr_t, int, const char *, const char *, const zprop_index_t *, const struct zfs_mod_supported_features *); _ZFS_PROP_H void zprop_register_hidden(int, const char *, zprop_type_t, - zprop_attr_t, int, const char *, const struct zfs_mod_supported_features *); + zprop_attr_t, int, const char *, boolean_t, + const struct zfs_mod_supported_features *); /* * Common routines for zfs and zpool property management diff --git a/lib/Makefile.am b/lib/Makefile.am index f07975cc03fc..499ebdaeba9b 100644 --- a/lib/Makefile.am +++ b/lib/Makefile.am @@ -33,50 +33,91 @@ # | | | # \-------- libspl ----+------/ # -# * - A stable ABI is provided for these libraries -# # # NB: GNU Automake Manual, Chapter 8.3.5: Libtool Convenience Libraries # These nine libraries are intermediary build components. # -SUBDIRS = libavl libicp libshare libspl libtpool libzstd -CPPCHECKDIRS = libavl libicp libnvpair libshare libspl libtpool libunicode -CPPCHECKDIRS += libuutil libzfs libzfs_core libzfsbootenv libzpool libzutil +# * - A stable ABI is provided for these libraries; +# when performing an ABI check the following options are applied: +# +# --no-unreferenced-symbols: Exclude symbols which are not referenced by +# any debug information. Without this _init() and _fini() are incorrectly +# reported on CentOS7 for libuutil.so. +# +# --headers-dir1: Limit ABI checks to public OpenZFS headers, otherwise +# changes in public system headers are also reported. +# +# --suppressions: Honor a suppressions file for each library to provide +# a mechanism for suppressing harmless warnings. +# +noinst_LTLIBRARIES = +lib_LTLIBRARIES = +pkgconfig_DATA = +include $(srcdir)/%D%/libavl/Makefile.am +include $(srcdir)/%D%/libicp/Makefile.am +include $(srcdir)/%D%/libnvpair/Makefile.am +include $(srcdir)/%D%/libshare/Makefile.am +include $(srcdir)/%D%/libspl/Makefile.am +include $(srcdir)/%D%/libtpool/Makefile.am +include $(srcdir)/%D%/libunicode/Makefile.am +include $(srcdir)/%D%/libuutil/Makefile.am +include $(srcdir)/%D%/libzfs_core/Makefile.am +include $(srcdir)/%D%/libzfs/Makefile.am +include $(srcdir)/%D%/libzfsbootenv/Makefile.am +include $(srcdir)/%D%/libzpool/Makefile.am +include $(srcdir)/%D%/libzstd/Makefile.am +include $(srcdir)/%D%/libzutil/Makefile.am if BUILD_LINUX -SUBDIRS += libefi -CPPCHECKDIRS += libefi +include $(srcdir)/%D%/libefi/Makefile.am endif -# libnvpair is installed as part of the final build product -# libzutil depends on it, so it must be compiled before libzutil -SUBDIRS += libnvpair -# libzutil depends on libefi if present -SUBDIRS += libzutil libunicode +PHONY += lib +lib: $(noinst_LTLIBRARIES) $(lib_LTLIBRARIES) -# These five libraries, which are installed as the final build product, -# incorporate the eight convenience libraries given above. -DISTLIBS = libuutil libzfs_core libzfs libzpool libzfsbootenv -SUBDIRS += $(DISTLIBS) -DISTLIBS += libnvpair -# An ABI is stored for each of these libraries. Note that libzpool.so -# is only linked against by ztest and zdb and no stable ABI is provided. -ABILIBS = libnvpair libuutil libzfs_core libzfs libzfsbootenv +PHONY += checkabi storeabi check_libabi_version allow_libabi_only_for_x86_64 -PHONY = checkabi storeabi cppcheck -checkabi: $(ABILIBS) - set -e ; for dir in $(ABILIBS) ; do \ - $(MAKE) -C $$dir checkabi ; \ - done +check_libabi_version: + if [ $$(abidw -v | $(SED) 's/[^0-9]//g') -lt 200 ]; then \ + printf '%s\n' "" \ + "*** Please use libabigail 2.0.0 version or newer;" \ + "*** otherwise results are not consistent!" \ + "(or see https://github.com/openzfs/libabigail-docker)"; \ + exit 1; \ + fi -storeabi: $(ABILIBS) - set -e ; for dir in $(ABILIBS) ; do \ - $(MAKE) -C $$dir storeabi ; \ - done +allow_libabi_only_for_x86_64: + echo '*** ABI definitions provided apply only to x86_64:' + echo '*** not checking or storing ABI and assuming success.' -cppcheck: $(CPPCHECKDIRS) - set -e ; for dir in $(CPPCHECKDIRS) ; do \ - $(MAKE) -C $$dir cppcheck ; \ +if TARGET_CPU_X86_64 +# These should depend on $(lib_LTLIBRARIES), but this breaks on CI when bound into Docker +checkabi: check_libabi_version + err=0; \ + for lib in $(lib_LTLIBRARIES); do \ + lib=$${lib%.la}; \ + [ -f $(srcdir)/lib/$$lib/$$lib.suppr ] || continue; \ + echo $$lib:; \ + abidiff --no-unreferenced-symbols \ + --headers-dir1 include \ + --suppressions $(srcdir)/lib/$$lib/$$lib.suppr \ + $(srcdir)/lib/$$lib/$$lib.abi .libs/$$lib.so || err=$$((err + 1)); \ + done; \ + exit $$err + +storeabi: check_libabi_version + for lib in $(lib_LTLIBRARIES); do \ + lib=$${lib%.la}; \ + [ -f $(srcdir)/lib/$$lib/$$lib.suppr ] || continue; \ + abidw --no-show-locs \ + --no-corpus-path \ + --no-comp-dir-path \ + --type-id-style hash \ + .libs/$$lib.so > $(srcdir)/lib/$$lib/$$lib.abi; \ done +else +checkabi: allow_libabi_only_for_x86_64 +storeabi: allow_libabi_only_for_x86_64 +endif diff --git a/lib/libavl/Makefile.am b/lib/libavl/Makefile.am index 3166febd02c5..3b302ee9deae 100644 --- a/lib/libavl/Makefile.am +++ b/lib/libavl/Makefile.am @@ -1,17 +1,7 @@ -include $(top_srcdir)/config/Rules.am +libavl_la_CFLAGS = $(AM_CFLAGS) $(KERNEL_CFLAGS) $(LIBRARY_CFLAGS) +libavl_la_CFLAGS += -fvisibility=hidden -VPATH = $(top_srcdir)/module/avl/ - -# Includes kernel code, generate warnings for large stack frames -AM_CFLAGS += $(FRAME_LARGER_THAN) -AM_CFLAGS += -fvisibility=hidden - -noinst_LTLIBRARIES = libavl.la - -KERNEL_C = \ - avl.c +noinst_LTLIBRARIES += libavl.la nodist_libavl_la_SOURCES = \ - $(KERNEL_C) - -include $(top_srcdir)/config/CppCheck.am + module/avl/avl.c diff --git a/lib/libefi/Makefile.am b/lib/libefi/Makefile.am index 580319a31495..5c3e57346c86 100644 --- a/lib/libefi/Makefile.am +++ b/lib/libefi/Makefile.am @@ -1,15 +1,11 @@ -include $(top_srcdir)/config/Rules.am +libefi_la_CFLAGS = $(AM_CFLAGS) $(LIBRARY_CFLAGS) +libefi_la_CFLAGS += $(LIBUUID_CFLAGS) $(ZLIB_CFLAGS) +libefi_la_CFLAGS += -fvisibility=hidden -AM_CFLAGS += $(LIBUUID_CFLAGS) $(ZLIB_CFLAGS) -AM_CFLAGS += -fvisibility=hidden +noinst_LTLIBRARIES += libefi.la +CPPCHECKTARGETS += libefi.la -noinst_LTLIBRARIES = libefi.la - -USER_C = \ - rdwr_efi.c - -libefi_la_SOURCES = $(USER_C) +libefi_la_SOURCES = \ + %D%/rdwr_efi.c libefi_la_LIBADD = $(LIBUUID_LIBS) $(ZLIB_LIBS) - -include $(top_srcdir)/config/CppCheck.am diff --git a/lib/libefi/rdwr_efi.c b/lib/libefi/rdwr_efi.c index b4fec0c8600a..f159a022496c 100644 --- a/lib/libefi/rdwr_efi.c +++ b/lib/libefi/rdwr_efi.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -29,14 +29,12 @@ #include #include #include -#include #include #include #include #include #include #include -#include #include #include #include @@ -857,7 +855,6 @@ efi_read(int fd, struct dk_gpt *vtoc) } for (i = 0; i < vtoc->efi_nparts; i++) { - UUID_LE_CONVERT(vtoc->efi_parts[i].p_guid, efi_parts[i].efi_gpe_PartitionTypeGUID); @@ -865,7 +862,7 @@ efi_read(int fd, struct dk_gpt *vtoc) j < sizeof (conversion_array) / sizeof (struct uuid_to_ptag); j++) { - if (bcmp(&vtoc->efi_parts[i].p_guid, + if (memcmp(&vtoc->efi_parts[i].p_guid, &conversion_array[j].uuid, sizeof (struct uuid)) == 0) { vtoc->efi_parts[i].p_tag = j; @@ -920,18 +917,17 @@ write_pmbr(int fd, struct dk_gpt *vtoc) /* LINTED -- always longlong aligned */ dk_ioc.dki_data = (efi_gpt_t *)buf; if (efi_ioctl(fd, DKIOCGETEFI, &dk_ioc) == -1) { - (void) memcpy(&mb, buf, sizeof (mb)); - bzero(&mb, sizeof (mb)); + memset(&mb, 0, sizeof (mb)); mb.signature = LE_16(MBB_MAGIC); } else { (void) memcpy(&mb, buf, sizeof (mb)); if (mb.signature != LE_16(MBB_MAGIC)) { - bzero(&mb, sizeof (mb)); + memset(&mb, 0, sizeof (mb)); mb.signature = LE_16(MBB_MAGIC); } } - bzero(&mb.parts, sizeof (mb.parts)); + memset(&mb.parts, 0, sizeof (mb.parts)); cp = (uchar_t *)&mb.parts[0]; /* bootable or not */ *cp++ = 0; @@ -1455,8 +1451,8 @@ efi_write(int fd, struct dk_gpt *vtoc) (void) uuid_generate((uchar_t *) &vtoc->efi_parts[i].p_uguid); } - bcopy(&vtoc->efi_parts[i].p_uguid, - &efi_parts[i].efi_gpe_UniquePartitionGUID, + memcpy(&efi_parts[i].efi_gpe_UniquePartitionGUID, + &vtoc->efi_parts[i].p_uguid, sizeof (uuid_t)); } efi->efi_gpt_PartitionEntryArrayCRC32 = diff --git a/lib/libicp/Makefile.am b/lib/libicp/Makefile.am index 831d2fedff4d..b7f1d0e1b1e4 100644 --- a/lib/libicp/Makefile.am +++ b/lib/libicp/Makefile.am @@ -1,70 +1,69 @@ -include $(top_srcdir)/config/Rules.am +libicp_la_CCASFLAGS = $(AM_CCASFLAGS) +libicp_la_CFLAGS = $(AM_CFLAGS) $(KERNEL_CFLAGS) $(LIBRARY_CFLAGS) -VPATH = \ - $(top_srcdir)/module/icp \ - $(top_srcdir)/lib/libicp +noinst_LTLIBRARIES += libicp.la -# Includes kernel code, generate warnings for large stack frames -AM_CFLAGS += $(FRAME_LARGER_THAN) - -noinst_LTLIBRARIES = libicp.la +nodist_libicp_la_SOURCES = \ + module/icp/spi/kcf_spi.c \ + module/icp/api/kcf_ctxops.c \ + module/icp/api/kcf_cipher.c \ + module/icp/api/kcf_mac.c \ + module/icp/algs/aes/aes_impl_aesni.c \ + module/icp/algs/aes/aes_impl_generic.c \ + module/icp/algs/aes/aes_impl_x86-64.c \ + module/icp/algs/aes/aes_impl.c \ + module/icp/algs/aes/aes_modes.c \ + module/icp/algs/blake3/blake3.c \ + module/icp/algs/blake3/blake3_generic.c \ + module/icp/algs/blake3/blake3_impl.c \ + module/icp/algs/blake3/blake3_x86-64.c \ + module/icp/algs/edonr/edonr.c \ + module/icp/algs/modes/modes.c \ + module/icp/algs/modes/cbc.c \ + module/icp/algs/modes/gcm_generic.c \ + module/icp/algs/modes/gcm_pclmulqdq.c \ + module/icp/algs/modes/gcm.c \ + module/icp/algs/modes/ctr.c \ + module/icp/algs/modes/ccm.c \ + module/icp/algs/modes/ecb.c \ + module/icp/algs/sha2/sha2.c \ + module/icp/algs/skein/skein.c \ + module/icp/algs/skein/skein_block.c \ + module/icp/algs/skein/skein_iv.c \ + module/icp/illumos-crypto.c \ + module/icp/io/aes.c \ + module/icp/io/sha2_mod.c \ + module/icp/io/skein_mod.c \ + module/icp/core/kcf_sched.c \ + module/icp/core/kcf_prov_lib.c \ + module/icp/core/kcf_callprov.c \ + module/icp/core/kcf_mech_tabs.c \ + module/icp/core/kcf_prov_tabs.c -if TARGET_CPU_X86_64 -ASM_SOURCES_C = asm-x86_64/aes/aeskey.c -ASM_SOURCES_AS = \ - asm-x86_64/aes/aes_amd64.S \ - asm-x86_64/aes/aes_aesni.S \ - asm-x86_64/modes/gcm_pclmulqdq.S \ - asm-x86_64/modes/aesni-gcm-x86_64.S \ - asm-x86_64/modes/ghash-x86_64.S \ - asm-x86_64/sha2/sha256_impl.S \ - asm-x86_64/sha2/sha512_impl.S -else -ASM_SOURCES_C = -ASM_SOURCES_AS = +if TARGET_CPU_AARCH64 +nodist_libicp_la_SOURCES += \ + module/icp/asm-aarch64/blake3/b3_aarch64_sse2.S \ + module/icp/asm-aarch64/blake3/b3_aarch64_sse41.S endif -KERNEL_C = \ - spi/kcf_spi.c \ - api/kcf_ctxops.c \ - api/kcf_digest.c \ - api/kcf_cipher.c \ - api/kcf_miscapi.c \ - api/kcf_mac.c \ - algs/aes/aes_impl_aesni.c \ - algs/aes/aes_impl_generic.c \ - algs/aes/aes_impl_x86-64.c \ - algs/aes/aes_impl.c \ - algs/aes/aes_modes.c \ - algs/edonr/edonr.c \ - algs/modes/modes.c \ - algs/modes/cbc.c \ - algs/modes/gcm_generic.c \ - algs/modes/gcm_pclmulqdq.c \ - algs/modes/gcm.c \ - algs/modes/ctr.c \ - algs/modes/ccm.c \ - algs/modes/ecb.c \ - algs/sha2/sha2.c \ - algs/skein/skein.c \ - algs/skein/skein_block.c \ - algs/skein/skein_iv.c \ - illumos-crypto.c \ - io/aes.c \ - io/sha2_mod.c \ - io/skein_mod.c \ - os/modhash.c \ - core/kcf_sched.c \ - core/kcf_prov_lib.c \ - core/kcf_callprov.c \ - core/kcf_mech_tabs.c \ - core/kcf_prov_tabs.c \ - $(ASM_SOURCES_C) - -KERNEL_ASM = $(ASM_SOURCES_AS) - -nodist_libicp_la_SOURCES = \ - $(KERNEL_C) \ - $(KERNEL_ASM) +if TARGET_CPU_POWERPC +nodist_libicp_la_SOURCES += \ + module/icp/asm-ppc64/blake3/b3_ppc64le_sse2.S \ + module/icp/asm-ppc64/blake3/b3_ppc64le_sse41.S +endif -include $(top_srcdir)/config/CppCheck.am +if TARGET_CPU_X86_64 +nodist_libicp_la_SOURCES += \ + module/icp/asm-x86_64/aes/aeskey.c \ + module/icp/asm-x86_64/aes/aes_amd64.S \ + module/icp/asm-x86_64/aes/aes_aesni.S \ + module/icp/asm-x86_64/modes/gcm_pclmulqdq.S \ + module/icp/asm-x86_64/modes/aesni-gcm-x86_64.S \ + module/icp/asm-x86_64/modes/ghash-x86_64.S \ + module/icp/asm-x86_64/sha2/sha256_impl.S \ + module/icp/asm-x86_64/sha2/sha512_impl.S \ + module/icp/asm-x86_64/blake3/blake3_avx2.S \ + module/icp/asm-x86_64/blake3/blake3_avx512.S \ + module/icp/asm-x86_64/blake3/blake3_sse2.S \ + module/icp/asm-x86_64/blake3/blake3_sse41.S +endif diff --git a/lib/libnvpair/Makefile.am b/lib/libnvpair/Makefile.am index 7b9ebebe7906..87b8d32aa175 100644 --- a/lib/libnvpair/Makefile.am +++ b/lib/libnvpair/Makefile.am @@ -1,36 +1,26 @@ -include $(top_srcdir)/config/Rules.am +libnvpair_la_CFLAGS = $(AM_CFLAGS) $(KERNEL_CFLAGS) $(LIBRARY_CFLAGS) +libnvpair_la_CFLAGS += $(LIBTIRPC_CFLAGS) +libnvpair_la_CFLAGS += -fvisibility=hidden -VPATH = \ - $(top_srcdir)/module/nvpair \ - $(top_srcdir)/lib/libnvpair +# wchar_t is undefined-signedness, but we compare to >=0; this warns with unsigned wchar_t +%D%/libnvpair_la-libnvpair_json.$(OBJEXT) : CFLAGS += -Wno-type-limits +%D%/libnvpair_la-libnvpair_json.l$(OBJEXT): CFLAGS += -Wno-type-limits -# Includes kernel code, generate warnings for large stack frames -# and required CFLAGS for libtirpc -AM_CFLAGS += $(FRAME_LARGER_THAN) $(LIBTIRPC_CFLAGS) -AM_CFLAGS += -fvisibility=hidden - -lib_LTLIBRARIES = libnvpair.la - -include $(top_srcdir)/config/Abigail.am - -USER_C = \ - libnvpair.c \ - libnvpair_json.c \ - nvpair_alloc_system.c - -KERNEL_C = \ - nvpair_alloc_fixed.c \ - nvpair.c \ - fnvpair.c +lib_LTLIBRARIES += libnvpair.la +CPPCHECKTARGETS += libnvpair.la dist_libnvpair_la_SOURCES = \ - $(USER_C) + %D%/libnvpair.c \ + %D%/libnvpair_json.c \ + %D%/nvpair_alloc_system.c nodist_libnvpair_la_SOURCES = \ - $(KERNEL_C) + module/nvpair/nvpair_alloc_fixed.c \ + module/nvpair/nvpair.c \ + module/nvpair/fnvpair.c libnvpair_la_LIBADD = \ - $(abs_top_builddir)/lib/libspl/libspl_assert.la + libspl_assert.la libnvpair_la_LIBADD += $(LIBTIRPC_LIBS) $(LTLIBINTL) @@ -42,7 +32,4 @@ endif libnvpair_la_LDFLAGS += -version-info 3:0:0 -include $(top_srcdir)/config/CppCheck.am - -# Library ABI -EXTRA_DIST = libnvpair.abi libnvpair.suppr +dist_noinst_DATA += %D%/libnvpair.abi %D%/libnvpair.suppr diff --git a/lib/libnvpair/libnvpair.abi b/lib/libnvpair/libnvpair.abi index f9874da81f82..de15237da583 100644 --- a/lib/libnvpair/libnvpair.abi +++ b/lib/libnvpair/libnvpair.abi @@ -2794,7 +2794,7 @@ - + diff --git a/lib/libnvpair/libnvpair.c b/lib/libnvpair/libnvpair.c index 0123bb772048..48423226743a 100644 --- a/lib/libnvpair/libnvpair.c +++ b/lib/libnvpair/libnvpair.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libnvpair/libnvpair_json.c b/lib/libnvpair/libnvpair_json.c index 15b6f4afaf73..19acea8f5094 100644 --- a/lib/libnvpair/libnvpair_json.c +++ b/lib/libnvpair/libnvpair_json.c @@ -15,7 +15,7 @@ #include #include -#include +#include #include #include @@ -46,12 +46,10 @@ static int nvlist_print_json_string(FILE *fp, const char *input) { - mbstate_t mbr; + mbstate_t mbr = {0}; wchar_t c; size_t sz; - bzero(&mbr, sizeof (mbr)); - FPRINTF(fp, "\""); while ((sz = mbrtowc(&c, input, MB_CUR_MAX, &mbr)) > 0) { if (sz == (size_t)-1 || sz == (size_t)-2) { diff --git a/lib/libnvpair/nvpair_alloc_system.c b/lib/libnvpair/nvpair_alloc_system.c index 59806ea4dc93..ade10f58f605 100644 --- a/lib/libnvpair/nvpair_alloc_system.c +++ b/lib/libnvpair/nvpair_alloc_system.c @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libshare/Makefile.am b/lib/libshare/Makefile.am index dff3e5382d6e..48d8cb832428 100644 --- a/lib/libshare/Makefile.am +++ b/lib/libshare/Makefile.am @@ -1,30 +1,27 @@ -include $(top_srcdir)/config/Rules.am +libshare_la_CFLAGS = $(AM_CFLAGS) $(LIBRARY_CFLAGS) +libshare_la_CFLAGS += -fvisibility=hidden -DEFAULT_INCLUDES += -I$(srcdir) +libshare_la_CPPFLAGS = $(AM_CPPFLAGS) +libshare_la_CPPFLAGS += -I$(srcdir)/%D% -AM_CFLAGS += -fvisibility=hidden +noinst_LTLIBRARIES += libshare.la +CPPCHECKTARGETS += libshare.la -noinst_LTLIBRARIES = libshare.la - -USER_C = \ - libshare_impl.h \ - libshare.c \ - nfs.c \ - nfs.h \ - smb.h +libshare_la_SOURCES = \ + %D%/libshare_impl.h \ + %D%/libshare.c \ + %D%/nfs.c \ + %D%/nfs.h \ + %D%/smb.h if BUILD_LINUX -USER_C += \ - os/linux/nfs.c \ - os/linux/smb.c +libshare_la_SOURCES += \ + %D%/os/linux/nfs.c \ + %D%/os/linux/smb.c endif if BUILD_FREEBSD -USER_C += \ - os/freebsd/nfs.c \ - os/freebsd/smb.c +libshare_la_SOURCES += \ + %D%/os/freebsd/nfs.c \ + %D%/os/freebsd/smb.c endif - -libshare_la_SOURCES = $(USER_C) - -include $(top_srcdir)/config/CppCheck.am diff --git a/lib/libshare/libshare.c b/lib/libshare/libshare.c index a228645fbf8f..d6257aa1ef30 100644 --- a/lib/libshare/libshare.c +++ b/lib/libshare/libshare.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -27,8 +27,9 @@ #include #include +#include +#include #include -#include #include #include #include @@ -37,139 +38,70 @@ #include #include #include "libshare_impl.h" -#include "nfs.h" -#include "smb.h" -static sa_share_impl_t alloc_share(const char *zfsname, const char *path); -static void free_share(sa_share_impl_t share); - -static int fstypes_count; -static sa_fstype_t *fstypes; - -sa_fstype_t * -register_fstype(const char *name, const sa_share_ops_t *ops) -{ - sa_fstype_t *fstype; - - fstype = calloc(1, sizeof (sa_fstype_t)); - - if (fstype == NULL) - return (NULL); - - fstype->name = name; - fstype->ops = ops; - fstype->fsinfo_index = fstypes_count; +#define init_share(zfsname, path, shareopts) \ + { \ + .sa_zfsname = zfsname, \ + .sa_mountpoint = path, \ + .sa_shareopts = shareopts, \ + } - fstypes_count++; +#define VALIDATE_PROTOCOL(proto, ...) \ + if ((proto) < 0 || (proto) >= SA_PROTOCOL_COUNT) \ + return __VA_ARGS__ - fstype->next = fstypes; - fstypes = fstype; +const char *const sa_protocol_names[SA_PROTOCOL_COUNT] = { + [SA_PROTOCOL_NFS] = "nfs", + [SA_PROTOCOL_SMB] = "smb", +}; - return (fstype); -} - -__attribute__((constructor)) static void -libshare_init(void) -{ - libshare_nfs_init(); - libshare_smb_init(); -} +static const sa_fstype_t *fstypes[SA_PROTOCOL_COUNT] = + {&libshare_nfs_type, &libshare_smb_type}; int sa_enable_share(const char *zfsname, const char *mountpoint, - const char *shareopts, char *protocol) + const char *shareopts, enum sa_protocol protocol) { - int rc, ret = SA_OK; - boolean_t found_protocol = B_FALSE; - sa_fstype_t *fstype; - - sa_share_impl_t impl_share = alloc_share(zfsname, mountpoint); - if (impl_share == NULL) - return (SA_NO_MEMORY); - - fstype = fstypes; - while (fstype != NULL) { - if (strcmp(fstype->name, protocol) == 0) { - - rc = fstype->ops->update_shareopts(impl_share, - shareopts); - if (rc != SA_OK) - break; - - rc = fstype->ops->enable_share(impl_share); - if (rc != SA_OK) - ret = rc; + VALIDATE_PROTOCOL(protocol, SA_INVALID_PROTOCOL); - found_protocol = B_TRUE; - } - - fstype = fstype->next; - } - free_share(impl_share); - - return (found_protocol ? ret : SA_INVALID_PROTOCOL); + const struct sa_share_impl args = + init_share(zfsname, mountpoint, shareopts); + return (fstypes[protocol]->enable_share(&args)); } int -sa_disable_share(const char *mountpoint, char *protocol) +sa_disable_share(const char *mountpoint, enum sa_protocol protocol) { - int rc, ret = SA_OK; - boolean_t found_protocol = B_FALSE; - sa_fstype_t *fstype; - - sa_share_impl_t impl_share = alloc_share(NULL, mountpoint); - if (impl_share == NULL) - return (SA_NO_MEMORY); + VALIDATE_PROTOCOL(protocol, SA_INVALID_PROTOCOL); - fstype = fstypes; - while (fstype != NULL) { - if (strcmp(fstype->name, protocol) == 0) { - - rc = fstype->ops->disable_share(impl_share); - if (rc != SA_OK) - ret = rc; - - found_protocol = B_TRUE; - } - - fstype = fstype->next; - } - free_share(impl_share); - - return (found_protocol ? ret : SA_INVALID_PROTOCOL); + const struct sa_share_impl args = init_share(NULL, mountpoint, NULL); + return (fstypes[protocol]->disable_share(&args)); } boolean_t -sa_is_shared(const char *mountpoint, char *protocol) +sa_is_shared(const char *mountpoint, enum sa_protocol protocol) { - sa_fstype_t *fstype; - boolean_t ret = B_FALSE; - - /* guid value is not used */ - sa_share_impl_t impl_share = alloc_share(NULL, mountpoint); - if (impl_share == NULL) - return (B_FALSE); + VALIDATE_PROTOCOL(protocol, B_FALSE); - fstype = fstypes; - while (fstype != NULL) { - if (strcmp(fstype->name, protocol) == 0) { - ret = fstype->ops->is_shared(impl_share); - } - fstype = fstype->next; - } - free_share(impl_share); - return (ret); + const struct sa_share_impl args = init_share(NULL, mountpoint, NULL); + return (fstypes[protocol]->is_shared(&args)); } void -sa_commit_shares(const char *protocol) +sa_commit_shares(enum sa_protocol protocol) { - sa_fstype_t *fstype = fstypes; - while (fstype != NULL) { - if (strcmp(fstype->name, protocol) == 0) - fstype->ops->commit_shares(); - fstype = fstype->next; - } + /* CSTYLED */ + VALIDATE_PROTOCOL(protocol, ); + + fstypes[protocol]->commit_shares(); +} + +int +sa_validate_shareopts(const char *options, enum sa_protocol protocol) +{ + VALIDATE_PROTOCOL(protocol, SA_INVALID_PROTOCOL); + + return (fstypes[protocol]->validate_shareopts(options)); } /* @@ -177,189 +109,85 @@ sa_commit_shares(const char *protocol) * * convert an error value to an error string */ -char * +const char * sa_errorstr(int err) { static char errstr[32]; - char *ret = NULL; switch (err) { case SA_OK: - ret = dgettext(TEXT_DOMAIN, "ok"); - break; + return (dgettext(TEXT_DOMAIN, "ok")); case SA_NO_SUCH_PATH: - ret = dgettext(TEXT_DOMAIN, "path doesn't exist"); - break; + return (dgettext(TEXT_DOMAIN, "path doesn't exist")); case SA_NO_MEMORY: - ret = dgettext(TEXT_DOMAIN, "no memory"); - break; + return (dgettext(TEXT_DOMAIN, "no memory")); case SA_DUPLICATE_NAME: - ret = dgettext(TEXT_DOMAIN, "name in use"); - break; + return (dgettext(TEXT_DOMAIN, "name in use")); case SA_BAD_PATH: - ret = dgettext(TEXT_DOMAIN, "bad path"); - break; + return (dgettext(TEXT_DOMAIN, "bad path")); case SA_NO_SUCH_GROUP: - ret = dgettext(TEXT_DOMAIN, "no such group"); - break; + return (dgettext(TEXT_DOMAIN, "no such group")); case SA_CONFIG_ERR: - ret = dgettext(TEXT_DOMAIN, "configuration error"); - break; + return (dgettext(TEXT_DOMAIN, "configuration error")); case SA_SYSTEM_ERR: - ret = dgettext(TEXT_DOMAIN, "system error"); - break; + return (dgettext(TEXT_DOMAIN, "system error")); case SA_SYNTAX_ERR: - ret = dgettext(TEXT_DOMAIN, "syntax error"); - break; + return (dgettext(TEXT_DOMAIN, "syntax error")); case SA_NO_PERMISSION: - ret = dgettext(TEXT_DOMAIN, "no permission"); - break; + return (dgettext(TEXT_DOMAIN, "no permission")); case SA_BUSY: - ret = dgettext(TEXT_DOMAIN, "busy"); - break; + return (dgettext(TEXT_DOMAIN, "busy")); case SA_NO_SUCH_PROP: - ret = dgettext(TEXT_DOMAIN, "no such property"); - break; + return (dgettext(TEXT_DOMAIN, "no such property")); case SA_INVALID_NAME: - ret = dgettext(TEXT_DOMAIN, "invalid name"); - break; + return (dgettext(TEXT_DOMAIN, "invalid name")); case SA_INVALID_PROTOCOL: - ret = dgettext(TEXT_DOMAIN, "invalid protocol"); - break; + return (dgettext(TEXT_DOMAIN, "invalid protocol")); case SA_NOT_ALLOWED: - ret = dgettext(TEXT_DOMAIN, "operation not allowed"); - break; + return (dgettext(TEXT_DOMAIN, "operation not allowed")); case SA_BAD_VALUE: - ret = dgettext(TEXT_DOMAIN, "bad property value"); - break; + return (dgettext(TEXT_DOMAIN, "bad property value")); case SA_INVALID_SECURITY: - ret = dgettext(TEXT_DOMAIN, "invalid security type"); - break; + return (dgettext(TEXT_DOMAIN, "invalid security type")); case SA_NO_SUCH_SECURITY: - ret = dgettext(TEXT_DOMAIN, "security type not found"); - break; + return (dgettext(TEXT_DOMAIN, "security type not found")); case SA_VALUE_CONFLICT: - ret = dgettext(TEXT_DOMAIN, "property value conflict"); - break; + return (dgettext(TEXT_DOMAIN, "property value conflict")); case SA_NOT_IMPLEMENTED: - ret = dgettext(TEXT_DOMAIN, "not implemented"); - break; + return (dgettext(TEXT_DOMAIN, "not implemented")); case SA_INVALID_PATH: - ret = dgettext(TEXT_DOMAIN, "invalid path"); - break; + return (dgettext(TEXT_DOMAIN, "invalid path")); case SA_NOT_SUPPORTED: - ret = dgettext(TEXT_DOMAIN, "operation not supported"); - break; + return (dgettext(TEXT_DOMAIN, "operation not supported")); case SA_PROP_SHARE_ONLY: - ret = dgettext(TEXT_DOMAIN, "property not valid for group"); - break; + return (dgettext(TEXT_DOMAIN, "property not valid for group")); case SA_NOT_SHARED: - ret = dgettext(TEXT_DOMAIN, "not shared"); - break; + return (dgettext(TEXT_DOMAIN, "not shared")); case SA_NO_SUCH_RESOURCE: - ret = dgettext(TEXT_DOMAIN, "no such resource"); - break; + return (dgettext(TEXT_DOMAIN, "no such resource")); case SA_RESOURCE_REQUIRED: - ret = dgettext(TEXT_DOMAIN, "resource name required"); - break; + return (dgettext(TEXT_DOMAIN, "resource name required")); case SA_MULTIPLE_ERROR: - ret = dgettext(TEXT_DOMAIN, "errors from multiple protocols"); - break; + return (dgettext(TEXT_DOMAIN, + "errors from multiple protocols")); case SA_PATH_IS_SUBDIR: - ret = dgettext(TEXT_DOMAIN, "path is a subpath of share"); - break; + return (dgettext(TEXT_DOMAIN, "path is a subpath of share")); case SA_PATH_IS_PARENTDIR: - ret = dgettext(TEXT_DOMAIN, "path is parent of a share"); - break; + return (dgettext(TEXT_DOMAIN, "path is parent of a share")); case SA_NO_SECTION: - ret = dgettext(TEXT_DOMAIN, "protocol requires a section"); - break; + return (dgettext(TEXT_DOMAIN, "protocol requires a section")); case SA_NO_PROPERTIES: - ret = dgettext(TEXT_DOMAIN, "properties not found"); - break; + return (dgettext(TEXT_DOMAIN, "properties not found")); case SA_NO_SUCH_SECTION: - ret = dgettext(TEXT_DOMAIN, "section not found"); - break; + return (dgettext(TEXT_DOMAIN, "section not found")); case SA_PASSWORD_ENC: - ret = dgettext(TEXT_DOMAIN, "passwords must be encrypted"); - break; + return (dgettext(TEXT_DOMAIN, "passwords must be encrypted")); case SA_SHARE_EXISTS: - ret = dgettext(TEXT_DOMAIN, "path or file is already shared"); - break; + return (dgettext(TEXT_DOMAIN, + "path or file is already shared")); default: (void) snprintf(errstr, sizeof (errstr), dgettext(TEXT_DOMAIN, "unknown %d"), err); - ret = errstr; + return (errstr); } - return (ret); -} - -int -sa_validate_shareopts(char *options, char *proto) -{ - sa_fstype_t *fstype; - - fstype = fstypes; - while (fstype != NULL) { - if (strcmp(fstype->name, proto) != 0) { - fstype = fstype->next; - continue; - } - - return (fstype->ops->validate_shareopts(options)); - } - - return (SA_INVALID_PROTOCOL); -} - -static sa_share_impl_t -alloc_share(const char *zfsname, const char *mountpoint) -{ - sa_share_impl_t impl_share; - - impl_share = calloc(1, sizeof (struct sa_share_impl)); - - if (impl_share == NULL) - return (NULL); - - if (mountpoint != NULL && - ((impl_share->sa_mountpoint = strdup(mountpoint)) == NULL)) { - free(impl_share); - return (NULL); - } - - if (zfsname != NULL && - ((impl_share->sa_zfsname = strdup(zfsname)) == NULL)) { - free(impl_share->sa_mountpoint); - free(impl_share); - return (NULL); - } - - impl_share->sa_fsinfo = calloc(fstypes_count, - sizeof (sa_share_fsinfo_t)); - if (impl_share->sa_fsinfo == NULL) { - free(impl_share->sa_mountpoint); - free(impl_share->sa_zfsname); - free(impl_share); - return (NULL); - } - - return (impl_share); -} - -static void -free_share(sa_share_impl_t impl_share) -{ - sa_fstype_t *fstype; - - fstype = fstypes; - while (fstype != NULL) { - fstype->ops->clear_shareopts(impl_share); - fstype = fstype->next; - } - - free(impl_share->sa_mountpoint); - free(impl_share->sa_zfsname); - free(impl_share->sa_fsinfo); - free(impl_share); } diff --git a/lib/libshare/libshare_impl.h b/lib/libshare/libshare_impl.h index 63a6907539e0..b845eb2d8acd 100644 --- a/lib/libshare/libshare_impl.h +++ b/lib/libshare/libshare_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -27,39 +27,20 @@ #ifndef _LIBSPL_LIBSHARE_IMPL_H #define _LIBSPL_LIBSHARE_IMPL_H -typedef struct sa_share_fsinfo { - char *shareopts; -} sa_share_fsinfo_t; - -typedef struct sa_share_impl { - char *sa_mountpoint; - char *sa_zfsname; - - sa_share_fsinfo_t *sa_fsinfo; /* per-fstype information */ +typedef const struct sa_share_impl { + const char *sa_zfsname; + const char *sa_mountpoint; + const char *sa_shareopts; } *sa_share_impl_t; -#define FSINFO(impl_share, fstype) \ - (&(impl_share->sa_fsinfo[fstype->fsinfo_index])) - -typedef struct sa_share_ops { - int (*enable_share)(sa_share_impl_t share); - int (*disable_share)(sa_share_impl_t share); - boolean_t (*is_shared)(sa_share_impl_t share); - int (*validate_shareopts)(const char *shareopts); - int (*update_shareopts)(sa_share_impl_t impl_share, - const char *shareopts); - void (*clear_shareopts)(sa_share_impl_t impl_share); - int (*commit_shares)(void); -} sa_share_ops_t; - -typedef struct sa_fstype { - struct sa_fstype *next; - - const char *name; - const sa_share_ops_t *ops; - int fsinfo_index; +typedef struct { + int (*const enable_share)(sa_share_impl_t share); + int (*const disable_share)(sa_share_impl_t share); + boolean_t (*const is_shared)(sa_share_impl_t share); + int (*const validate_shareopts)(const char *shareopts); + int (*const commit_shares)(void); } sa_fstype_t; -sa_fstype_t *register_fstype(const char *name, const sa_share_ops_t *ops); +extern const sa_fstype_t libshare_nfs_type, libshare_smb_type; #endif /* _LIBSPL_LIBSHARE_IMPL_H */ diff --git a/lib/libshare/nfs.c b/lib/libshare/nfs.c index 2a901bcec503..161bbfb0ceb9 100644 --- a/lib/libshare/nfs.c +++ b/lib/libshare/nfs.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -24,39 +24,37 @@ #include #include #include +#include #include #include #include #include "nfs.h" -static int nfs_lock_fd = -1; - - /* * nfs_exports_[lock|unlock] are used to guard against conconcurrent * updates to the exports file. Each protocol is responsible for * providing the necessary locking to ensure consistency. */ static int -nfs_exports_lock(const char *name) +nfs_exports_lock(const char *name, int *nfs_lock_fd) { int err; - nfs_lock_fd = open(name, O_RDWR | O_CREAT | O_CLOEXEC, 0600); - if (nfs_lock_fd == -1) { + *nfs_lock_fd = open(name, O_RDWR | O_CREAT | O_CLOEXEC, 0600); + if (*nfs_lock_fd == -1) { err = errno; fprintf(stderr, "failed to lock %s: %s\n", name, strerror(err)); return (err); } - while ((err = flock(nfs_lock_fd, LOCK_EX)) != 0 && errno == EINTR) + while ((err = flock(*nfs_lock_fd, LOCK_EX)) != 0 && errno == EINTR) ; if (err != 0) { err = errno; fprintf(stderr, "failed to lock %s: %s\n", name, strerror(err)); - (void) close(nfs_lock_fd); - nfs_lock_fd = -1; + (void) close(*nfs_lock_fd); + *nfs_lock_fd = -1; return (err); } @@ -64,17 +62,16 @@ nfs_exports_lock(const char *name) } static void -nfs_exports_unlock(const char *name) +nfs_exports_unlock(const char *name, int *nfs_lock_fd) { - verify(nfs_lock_fd > 0); + verify(*nfs_lock_fd > 0); - if (flock(nfs_lock_fd, LOCK_UN) != 0) { + if (flock(*nfs_lock_fd, LOCK_UN) != 0) fprintf(stderr, "failed to unlock %s: %s\n", name, strerror(errno)); - } - (void) close(nfs_lock_fd); - nfs_lock_fd = -1; + (void) close(*nfs_lock_fd); + *nfs_lock_fd = -1; } struct tmpfile { @@ -147,6 +144,34 @@ nfs_fini_tmpfile(const char *exports, struct tmpfile *tmpf) return (SA_OK); } +int +nfs_escape_mountpoint(const char *mp, char **out, boolean_t *need_free) +{ + if (strpbrk(mp, "\t\n\v\f\r \\") == NULL) { + *out = (char *)mp; + *need_free = B_FALSE; + return (SA_OK); + } else { + size_t len = strlen(mp); + *out = malloc(len * 4 + 1); + if (!*out) + return (SA_NO_MEMORY); + *need_free = B_TRUE; + + char *oc = *out; + for (const char *c = mp; c < mp + len; ++c) + if (memchr("\t\n\v\f\r \\", *c, + strlen("\t\n\v\f\r \\"))) { + sprintf(oc, "\\%03hho", *c); + oc += 4; + } else + *oc++ = *c; + *oc = '\0'; + } + + return (SA_OK); +} + static int nfs_process_exports(const char *exports, const char *mountpoint, boolean_t (*cbk)(void *userdata, char *line, boolean_t found_mountpoint), @@ -157,8 +182,16 @@ nfs_process_exports(const char *exports, const char *mountpoint, FILE *oldfp = fopen(exports, "re"); if (oldfp != NULL) { + boolean_t need_mp_free; + char *mp; + if ((error = nfs_escape_mountpoint(mountpoint, + &mp, &need_mp_free)) != SA_OK) { + (void) fclose(oldfp); + return (error); + } + char *buf = NULL, *sep; - size_t buflen = 0, mplen = strlen(mountpoint); + size_t buflen = 0, mplen = strlen(mp); while (cont && getline(&buf, &buflen, oldfp) != -1) { if (buf[0] == '\n' || buf[0] == '#') @@ -167,9 +200,11 @@ nfs_process_exports(const char *exports, const char *mountpoint, cont = cbk(userdata, buf, (sep = strpbrk(buf, "\t \n")) != NULL && sep - buf == mplen && - strncmp(buf, mountpoint, mplen) == 0); + strncmp(buf, mp, mplen) == 0); } free(buf); + if (need_mp_free) + free(mp); if (ferror(oldfp) != 0) error = ferror(oldfp); @@ -216,13 +251,13 @@ nfs_toggle_share(const char *lockfile, const char *exports, const char *expdir, sa_share_impl_t impl_share, int(*cbk)(sa_share_impl_t impl_share, FILE *tmpfile)) { - int error; + int error, nfs_lock_fd = -1; struct tmpfile tmpf; if (!nfs_init_tmpfile(exports, expdir, &tmpf)) return (SA_SYSTEM_ERR); - error = nfs_exports_lock(lockfile); + error = nfs_exports_lock(lockfile, &nfs_lock_fd); if (error != 0) { nfs_abort_tmpfile(&tmpf); return (error); @@ -237,12 +272,12 @@ nfs_toggle_share(const char *lockfile, const char *exports, goto fullerr; error = nfs_fini_tmpfile(exports, &tmpf); - nfs_exports_unlock(lockfile); + nfs_exports_unlock(lockfile, &nfs_lock_fd); return (error); fullerr: nfs_abort_tmpfile(&tmpf); - nfs_exports_unlock(lockfile); + nfs_exports_unlock(lockfile, &nfs_lock_fd); return (error); } diff --git a/lib/libshare/nfs.h b/lib/libshare/nfs.h index cfac274c3d26..58523c8f02e2 100644 --- a/lib/libshare/nfs.h +++ b/lib/libshare/nfs.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -28,8 +28,7 @@ #define FILE_HEADER "# !!! DO NOT EDIT THIS FILE MANUALLY !!!\n\n" -void libshare_nfs_init(void); - +int nfs_escape_mountpoint(const char *mp, char **out, boolean_t *need_free); boolean_t nfs_is_shared_impl(const char *exports, sa_share_impl_t impl_share); int nfs_toggle_share(const char *lockfile, const char *exports, const char *expdir, sa_share_impl_t impl_share, diff --git a/lib/libshare/os/freebsd/nfs.c b/lib/libshare/os/freebsd/nfs.c index 494442cb520f..78977a25f4f5 100644 --- a/lib/libshare/os/freebsd/nfs.c +++ b/lib/libshare/os/freebsd/nfs.c @@ -52,10 +52,8 @@ __FBSDID("$FreeBSD$"); #define ZFS_EXPORTS_FILE "/etc/zfs/exports" #define ZFS_EXPORTS_LOCK ZFS_EXPORTS_FILE".lock" -static sa_fstype_t *nfs_fstype; - /* - * This function translate options to a format acceptable by exports(5), eg. + * This function translates options to a format acceptable by exports(5), eg. * * -ro -network=192.168.0.0 -mask=255.255.255.0 -maproot=0 \ * zfs.freebsd.org 69.147.83.54 @@ -72,17 +70,14 @@ static sa_fstype_t *nfs_fstype; * * ro, maproot, mapall, mask, network, sec, alldirs, public, webnfs, * index, quiet - * - * NOTE: This function returns a static buffer and thus is not thread-safe. */ -static char * -translate_opts(const char *shareopts) +static int +translate_opts(const char *shareopts, FILE *out) { - static const char *known_opts[] = { "ro", "maproot", "mapall", "mask", - "network", "sec", "alldirs", "public", "webnfs", "index", "quiet", - NULL }; - static char newopts[OPTSSIZE]; - char oldopts[OPTSSIZE]; + static const char *const known_opts[] = { "ro", "maproot", "mapall", + "mask", "network", "sec", "alldirs", "public", "webnfs", "index", + "quiet" }; + char oldopts[OPTSSIZE], newopts[OPTSSIZE]; char *o, *s = NULL; unsigned int i; size_t len; @@ -93,7 +88,7 @@ translate_opts(const char *shareopts) while ((o = strsep(&s, "-, ")) != NULL) { if (o[0] == '\0') continue; - for (i = 0; known_opts[i] != NULL; i++) { + for (i = 0; i < ARRAY_SIZE(known_opts); ++i) { len = strlen(known_opts[i]); if (strncmp(known_opts[i], o, len) == 0 && (o[len] == '\0' || o[len] == '=')) { @@ -104,23 +99,34 @@ translate_opts(const char *shareopts) strlcat(newopts, o, sizeof (newopts)); strlcat(newopts, " ", sizeof (newopts)); } - return (newopts); + return (fputs(newopts, out)); } static int nfs_enable_share_impl(sa_share_impl_t impl_share, FILE *tmpfile) { - char *shareopts = FSINFO(impl_share, nfs_fstype)->shareopts; + const char *shareopts = impl_share->sa_shareopts; if (strcmp(shareopts, "on") == 0) shareopts = ""; - if (fprintf(tmpfile, "%s\t%s\n", impl_share->sa_mountpoint, - translate_opts(shareopts)) < 0) { + boolean_t need_free; + char *mp; + int rc = nfs_escape_mountpoint(impl_share->sa_mountpoint, &mp, + &need_free); + if (rc != SA_OK) + return (rc); + + if (fputs(mp, tmpfile) == EOF || + fputc('\t', tmpfile) == EOF || + translate_opts(shareopts, tmpfile) == EOF || + fputc('\n', tmpfile) == EOF) { fprintf(stderr, "failed to write to temporary file\n"); - return (SA_SYSTEM_ERR); + rc = SA_SYSTEM_ERR; } - return (SA_OK); + if (need_free) + free(mp); + return (rc); } static int @@ -155,22 +161,10 @@ nfs_is_shared(sa_share_impl_t impl_share) static int nfs_validate_shareopts(const char *shareopts) { + (void) shareopts; return (SA_OK); } -static int -nfs_update_shareopts(sa_share_impl_t impl_share, const char *shareopts) -{ - FSINFO(impl_share, nfs_fstype)->shareopts = (char *)shareopts; - return (SA_OK); -} - -static void -nfs_clear_shareopts(sa_share_impl_t impl_share) -{ - FSINFO(impl_share, nfs_fstype)->shareopts = NULL; -} - /* * Commit the shares by restarting mountd. */ @@ -201,22 +195,11 @@ nfs_commit_shares(void) return (SA_OK); } -static const sa_share_ops_t nfs_shareops = { +const sa_fstype_t libshare_nfs_type = { .enable_share = nfs_enable_share, .disable_share = nfs_disable_share, .is_shared = nfs_is_shared, .validate_shareopts = nfs_validate_shareopts, - .update_shareopts = nfs_update_shareopts, - .clear_shareopts = nfs_clear_shareopts, .commit_shares = nfs_commit_shares, }; - -/* - * Initializes the NFS functionality of libshare. - */ -void -libshare_nfs_init(void) -{ - nfs_fstype = register_fstype("nfs", &nfs_shareops); -} diff --git a/lib/libshare/os/freebsd/smb.c b/lib/libshare/os/freebsd/smb.c index 5b606ab96955..37a3802e4b89 100644 --- a/lib/libshare/os/freebsd/smb.c +++ b/lib/libshare/os/freebsd/smb.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -23,23 +23,9 @@ * Copyright (c) 2020 by Delphix. All rights reserved. */ -#include -#include #include -#include -#include -#include -#include -#include -#include -#include -#include -#include #include #include "libshare_impl.h" -#include "smb.h" - -static sa_fstype_t *smb_fstype; /* * Enables SMB sharing for the specified share. @@ -47,7 +33,8 @@ static sa_fstype_t *smb_fstype; static int smb_enable_share(sa_share_impl_t impl_share) { - fprintf(stderr, "No SMB support in FreeBSD yet.\n"); + (void) impl_share; + fputs("No SMB support in FreeBSD yet.\n", stderr); return (SA_NOT_SUPPORTED); } /* @@ -56,7 +43,8 @@ smb_enable_share(sa_share_impl_t impl_share) static int smb_disable_share(sa_share_impl_t impl_share) { - fprintf(stderr, "No SMB support in FreeBSD yet.\n"); + (void) impl_share; + fputs("No SMB support in FreeBSD yet.\n", stderr); return (SA_NOT_SUPPORTED); } @@ -66,7 +54,8 @@ smb_disable_share(sa_share_impl_t impl_share) static int smb_validate_shareopts(const char *shareopts) { - fprintf(stderr, "No SMB support in FreeBSD yet.\n"); + (void) shareopts; + fputs("No SMB support in FreeBSD yet.\n", stderr); return (SA_NOT_SUPPORTED); } @@ -76,53 +65,22 @@ smb_validate_shareopts(const char *shareopts) static boolean_t smb_is_share_active(sa_share_impl_t impl_share) { + (void) impl_share; return (B_FALSE); } -/* - * Called to update a share's options. A share's options might be out of - * date if the share was loaded from disk and the "sharesmb" dataset - * property has changed in the meantime. This function also takes care - * of re-enabling the share if necessary. - */ -static int -smb_update_shareopts(sa_share_impl_t impl_share, const char *shareopts) -{ - return (SA_OK); -} - static int smb_update_shares(void) { /* Not implemented */ return (0); } -/* - * Clears a share's SMB options. Used by libshare to - * clean up shares that are about to be free()'d. - */ -static void -smb_clear_shareopts(sa_share_impl_t impl_share) -{ - FSINFO(impl_share, smb_fstype)->shareopts = NULL; -} -static const sa_share_ops_t smb_shareops = { +const sa_fstype_t libshare_smb_type = { .enable_share = smb_enable_share, .disable_share = smb_disable_share, .is_shared = smb_is_share_active, .validate_shareopts = smb_validate_shareopts, - .update_shareopts = smb_update_shareopts, - .clear_shareopts = smb_clear_shareopts, .commit_shares = smb_update_shares, }; - -/* - * Initializes the SMB functionality of libshare. - */ -void -libshare_smb_init(void) -{ - smb_fstype = register_fstype("smb", &smb_shareops); -} diff --git a/lib/libshare/os/linux/nfs.c b/lib/libshare/os/linux/nfs.c index 002c5e24ca66..0870f37e5818 100644 --- a/lib/libshare/os/linux/nfs.c +++ b/lib/libshare/os/linux/nfs.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -29,7 +29,6 @@ #include #include #include -#include #include #include #include @@ -46,7 +45,8 @@ #define ZFS_EXPORTS_FILE ZFS_EXPORTS_DIR"/zfs.exports" #define ZFS_EXPORTS_LOCK ZFS_EXPORTS_FILE".lock" -static sa_fstype_t *nfs_fstype; + +static boolean_t nfs_available(void); typedef int (*nfs_shareopt_callback_t)(const char *opt, const char *value, void *cookie); @@ -230,7 +230,6 @@ foreach_nfs_host(sa_share_impl_t impl_share, FILE *tmpfile, nfs_host_callback_t callback, void *cookie) { nfs_host_cookie_t udata; - char *shareopts; udata.callback = callback; udata.sharepath = impl_share->sa_mountpoint; @@ -238,10 +237,8 @@ foreach_nfs_host(sa_share_impl_t impl_share, FILE *tmpfile, udata.tmpfile = tmpfile; udata.security = "sys"; - shareopts = FSINFO(impl_share, nfs_fstype)->shareopts; - - return (foreach_nfs_shareopt(shareopts, foreach_nfs_host_cb, - &udata)); + return (foreach_nfs_shareopt(impl_share->sa_shareopts, + foreach_nfs_host_cb, &udata)); } /* @@ -300,6 +297,11 @@ add_linux_shareopt(char **plinux_opts, const char *key, const char *value) return (SA_OK); } +static int string_cmp(const void *lhs, const void *rhs) { + const char *const *l = lhs, *const *r = rhs; + return (strcmp(*l, *r)); +} + /* * Validates and converts a single Solaris share option to its Linux * equivalent. @@ -307,6 +309,15 @@ add_linux_shareopt(char **plinux_opts, const char *key, const char *value) static int get_linux_shareopts_cb(const char *key, const char *value, void *cookie) { + /* This list must remain sorted, since we bsearch() it */ + static const char *const valid_keys[] = { "all_squash", "anongid", + "anonuid", "async", "auth_nlm", "crossmnt", "fsid", "fsuid", "hide", + "insecure", "insecure_locks", "mountpoint", "mp", "no_acl", + "no_all_squash", "no_auth_nlm", "no_root_squash", + "no_subtree_check", "no_wdelay", "nohide", "refer", "replicas", + "root_squash", "secure", "secure_locks", "subtree_check", "sync", + "wdelay" }; + char **plinux_opts = (char **)cookie; /* host-specific options, these are taken care of elsewhere */ @@ -325,26 +336,9 @@ get_linux_shareopts_cb(const char *key, const char *value, void *cookie) if (strcmp(key, "nosub") == 0) key = "subtree_check"; - if (strcmp(key, "insecure") != 0 && strcmp(key, "secure") != 0 && - strcmp(key, "async") != 0 && strcmp(key, "sync") != 0 && - strcmp(key, "no_wdelay") != 0 && strcmp(key, "wdelay") != 0 && - strcmp(key, "nohide") != 0 && strcmp(key, "hide") != 0 && - strcmp(key, "crossmnt") != 0 && - strcmp(key, "no_subtree_check") != 0 && - strcmp(key, "subtree_check") != 0 && - strcmp(key, "insecure_locks") != 0 && - strcmp(key, "secure_locks") != 0 && - strcmp(key, "no_auth_nlm") != 0 && strcmp(key, "auth_nlm") != 0 && - strcmp(key, "no_acl") != 0 && strcmp(key, "mountpoint") != 0 && - strcmp(key, "mp") != 0 && strcmp(key, "fsuid") != 0 && - strcmp(key, "refer") != 0 && strcmp(key, "replicas") != 0 && - strcmp(key, "root_squash") != 0 && - strcmp(key, "no_root_squash") != 0 && - strcmp(key, "all_squash") != 0 && - strcmp(key, "no_all_squash") != 0 && strcmp(key, "fsid") != 0 && - strcmp(key, "anonuid") != 0 && strcmp(key, "anongid") != 0) { + if (bsearch(&key, valid_keys, ARRAY_SIZE(valid_keys), + sizeof (*valid_keys), string_cmp) == NULL) return (SA_SYNTAX_ERR); - } (void) add_linux_shareopt(plinux_opts, key, value); @@ -396,14 +390,21 @@ nfs_add_entry(FILE *tmpfile, const char *sharepath, if (linux_opts == NULL) linux_opts = ""; - if (fprintf(tmpfile, "%s %s(sec=%s,%s,%s)\n", sharepath, + boolean_t need_free; + char *mp; + int rc = nfs_escape_mountpoint(sharepath, &mp, &need_free); + if (rc != SA_OK) + return (rc); + if (fprintf(tmpfile, "%s %s(sec=%s,%s,%s)\n", mp, get_linux_hostspec(host), security, access_opts, linux_opts) < 0) { fprintf(stderr, "failed to write to temporary file\n"); - return (SA_SYSTEM_ERR); + rc = SA_SYSTEM_ERR; } - return (SA_OK); + if (need_free) + free(mp); + return (rc); } /* @@ -412,11 +413,8 @@ nfs_add_entry(FILE *tmpfile, const char *sharepath, static int nfs_enable_share_impl(sa_share_impl_t impl_share, FILE *tmpfile) { - char *shareopts, *linux_opts; - int error; - - shareopts = FSINFO(impl_share, nfs_fstype)->shareopts; - error = get_linux_shareopts(shareopts, &linux_opts); + char *linux_opts = NULL; + int error = get_linux_shareopts(impl_share->sa_shareopts, &linux_opts); if (error != SA_OK) return (error); @@ -429,6 +427,9 @@ nfs_enable_share_impl(sa_share_impl_t impl_share, FILE *tmpfile) static int nfs_enable_share(sa_share_impl_t impl_share) { + if (!nfs_available()) + return (SA_SYSTEM_ERR); + return (nfs_toggle_share( ZFS_EXPORTS_LOCK, ZFS_EXPORTS_FILE, ZFS_EXPORTS_DIR, impl_share, nfs_enable_share_impl)); @@ -447,6 +448,9 @@ nfs_disable_share_impl(sa_share_impl_t impl_share, FILE *tmpfile) static int nfs_disable_share(sa_share_impl_t impl_share) { + if (!nfs_available()) + return (SA_SYSTEM_ERR); + return (nfs_toggle_share( ZFS_EXPORTS_LOCK, ZFS_EXPORTS_FILE, ZFS_EXPORTS_DIR, impl_share, nfs_disable_share_impl)); @@ -455,6 +459,9 @@ nfs_disable_share(sa_share_impl_t impl_share) static boolean_t nfs_is_shared(sa_share_impl_t impl_share) { + if (!nfs_available()) + return (SA_SYSTEM_ERR); + return (nfs_is_shared_impl(ZFS_EXPORTS_FILE, impl_share)); } @@ -464,11 +471,8 @@ nfs_is_shared(sa_share_impl_t impl_share) static int nfs_validate_shareopts(const char *shareopts) { - char *linux_opts; - int error; - - error = get_linux_shareopts(shareopts, &linux_opts); - + char *linux_opts = NULL; + int error = get_linux_shareopts(shareopts, &linux_opts); if (error != SA_OK) return (error); @@ -476,51 +480,41 @@ nfs_validate_shareopts(const char *shareopts) return (SA_OK); } -static int -nfs_update_shareopts(sa_share_impl_t impl_share, const char *shareopts) -{ - FSINFO(impl_share, nfs_fstype)->shareopts = (char *)shareopts; - return (SA_OK); -} - -/* - * Clears a share's NFS options. Used by libshare to - * clean up shares that are about to be free()'d. - */ -static void -nfs_clear_shareopts(sa_share_impl_t impl_share) -{ - FSINFO(impl_share, nfs_fstype)->shareopts = NULL; -} - static int nfs_commit_shares(void) { + if (!nfs_available()) + return (SA_SYSTEM_ERR); + char *argv[] = { - "/usr/sbin/exportfs", - "-ra", + (char *)"/usr/sbin/exportfs", + (char *)"-ra", NULL }; return (libzfs_run_process(argv[0], argv, 0)); } -static const sa_share_ops_t nfs_shareops = { +const sa_fstype_t libshare_nfs_type = { .enable_share = nfs_enable_share, .disable_share = nfs_disable_share, .is_shared = nfs_is_shared, .validate_shareopts = nfs_validate_shareopts, - .update_shareopts = nfs_update_shareopts, - .clear_shareopts = nfs_clear_shareopts, .commit_shares = nfs_commit_shares, }; -/* - * Initializes the NFS functionality of libshare. - */ -void -libshare_nfs_init(void) +static boolean_t +nfs_available(void) { - nfs_fstype = register_fstype("nfs", &nfs_shareops); + static int avail; + + if (!avail) { + if (access("/usr/sbin/exportfs", F_OK) != 0) + avail = -1; + else + avail = 1; + } + + return (avail == 1); } diff --git a/lib/libshare/os/linux/smb.c b/lib/libshare/os/linux/smb.c index 312ffb97d120..157b19aa85f4 100644 --- a/lib/libshare/os/linux/smb.c +++ b/lib/libshare/os/linux/smb.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -50,7 +50,6 @@ #include #include #include -#include #include #include #include @@ -64,9 +63,7 @@ static boolean_t smb_available(void); -static sa_fstype_t *smb_fstype; - -smb_share_t *smb_shares; +static smb_share_t *smb_shares; static int smb_disable_share(sa_share_impl_t impl_share); static boolean_t smb_is_share_active(sa_share_impl_t impl_share); @@ -219,46 +216,39 @@ smb_retrieve_shares(void) static int smb_enable_share_one(const char *sharename, const char *sharepath) { - char *argv[10], *pos; char name[SMB_NAME_MAX], comment[SMB_COMMENT_MAX]; - int rc; /* Support ZFS share name regexp '[[:alnum:]_-.: ]' */ strlcpy(name, sharename, sizeof (name)); - name [sizeof (name)-1] = '\0'; - - pos = name; - while (*pos != '\0') { - switch (*pos) { + for (char *itr = name; *itr != '\0'; ++itr) + switch (*itr) { case '/': case '-': case ':': case ' ': - *pos = '_'; + *itr = '_'; } - ++pos; - } - /* * CMD: net -S NET_CMD_ARG_HOST usershare add Test1 /share/Test1 \ * "Comment" "Everyone:F" */ snprintf(comment, sizeof (comment), "Comment: %s", sharepath); - argv[0] = NET_CMD_PATH; - argv[1] = (char *)"-S"; - argv[2] = NET_CMD_ARG_HOST; - argv[3] = (char *)"usershare"; - argv[4] = (char *)"add"; - argv[5] = (char *)name; - argv[6] = (char *)sharepath; - argv[7] = (char *)comment; - argv[8] = (char *)"Everyone:F"; - argv[9] = NULL; - - rc = libzfs_run_process(argv[0], argv, 0); - if (rc < 0) + char *argv[] = { + (char *)NET_CMD_PATH, + (char *)"-S", + (char *)NET_CMD_ARG_HOST, + (char *)"usershare", + (char *)"add", + name, + (char *)sharepath, + comment, + (char *)"Everyone:F", + NULL, + }; + + if (libzfs_run_process(argv[0], argv, 0) != 0) return (SA_SYSTEM_ERR); /* Reload the share file */ @@ -273,19 +263,16 @@ smb_enable_share_one(const char *sharename, const char *sharepath) static int smb_enable_share(sa_share_impl_t impl_share) { - char *shareopts; - if (!smb_available()) return (SA_SYSTEM_ERR); if (smb_is_share_active(impl_share)) smb_disable_share(impl_share); - shareopts = FSINFO(impl_share, smb_fstype)->shareopts; - if (shareopts == NULL) /* on/off */ + if (impl_share->sa_shareopts == NULL) /* on/off */ return (SA_SYSTEM_ERR); - if (strcmp(shareopts, "off") == 0) + if (strcmp(impl_share->sa_shareopts, "off") == 0) return (SA_OK); /* Magic: Enable (i.e., 'create new') share */ @@ -299,20 +286,18 @@ smb_enable_share(sa_share_impl_t impl_share) static int smb_disable_share_one(const char *sharename) { - int rc; - char *argv[7]; - /* CMD: net -S NET_CMD_ARG_HOST usershare delete Test1 */ - argv[0] = NET_CMD_PATH; - argv[1] = (char *)"-S"; - argv[2] = NET_CMD_ARG_HOST; - argv[3] = (char *)"usershare"; - argv[4] = (char *)"delete"; - argv[5] = (char *)sharename; - argv[6] = NULL; - - rc = libzfs_run_process(argv[0], argv, 0); - if (rc < 0) + char *argv[] = { + (char *)NET_CMD_PATH, + (char *)"-S", + (char *)NET_CMD_ARG_HOST, + (char *)"usershare", + (char *)"delete", + (char *)sharename, + NULL, + }; + + if (libzfs_run_process(argv[0], argv, 0) != 0) return (SA_SYSTEM_ERR); else return (SA_OK); @@ -324,8 +309,6 @@ smb_disable_share_one(const char *sharename) static int smb_disable_share(sa_share_impl_t impl_share) { - smb_share_t *shares = smb_shares; - if (!smb_available()) { /* * The share can't possibly be active, so nothing @@ -334,12 +317,9 @@ smb_disable_share(sa_share_impl_t impl_share) return (SA_OK); } - while (shares != NULL) { - if (strcmp(impl_share->sa_mountpoint, shares->path) == 0) - return (smb_disable_share_one(shares->name)); - - shares = shares->next; - } + for (const smb_share_t *i = smb_shares; i != NULL; i = i->next) + if (strcmp(impl_share->sa_mountpoint, i->path) == 0) + return (smb_disable_share_one(i->name)); return (SA_OK); } @@ -363,40 +343,19 @@ smb_validate_shareopts(const char *shareopts) static boolean_t smb_is_share_active(sa_share_impl_t impl_share) { - smb_share_t *iter = smb_shares; - if (!smb_available()) return (B_FALSE); /* Retrieve the list of (possible) active shares */ smb_retrieve_shares(); - while (iter != NULL) { - if (strcmp(impl_share->sa_mountpoint, iter->path) == 0) + for (const smb_share_t *i = smb_shares; i != NULL; i = i->next) + if (strcmp(impl_share->sa_mountpoint, i->path) == 0) return (B_TRUE); - iter = iter->next; - } - return (B_FALSE); } -/* - * Called to update a share's options. A share's options might be out of - * date if the share was loaded from disk and the "sharesmb" dataset - * property has changed in the meantime. This function also takes care - * of re-enabling the share if necessary. - */ -static int -smb_update_shareopts(sa_share_impl_t impl_share, const char *shareopts) -{ - if (!impl_share) - return (SA_SYSTEM_ERR); - - FSINFO(impl_share, smb_fstype)->shareopts = (char *)shareopts; - return (SA_OK); -} - static int smb_update_shares(void) { @@ -404,24 +363,12 @@ smb_update_shares(void) return (0); } -/* - * Clears a share's SMB options. Used by libshare to - * clean up shares that are about to be free()'d. - */ -static void -smb_clear_shareopts(sa_share_impl_t impl_share) -{ - FSINFO(impl_share, smb_fstype)->shareopts = NULL; -} - -static const sa_share_ops_t smb_shareops = { +const sa_fstype_t libshare_smb_type = { .enable_share = smb_enable_share, .disable_share = smb_disable_share, .is_shared = smb_is_share_active, .validate_shareopts = smb_validate_shareopts, - .update_shareopts = smb_update_shareopts, - .clear_shareopts = smb_clear_shareopts, .commit_shares = smb_update_shares, }; @@ -431,23 +378,18 @@ static const sa_share_ops_t smb_shareops = { static boolean_t smb_available(void) { - struct stat statbuf; + static int avail; - if (lstat(SHARE_DIR, &statbuf) != 0 || - !S_ISDIR(statbuf.st_mode)) - return (B_FALSE); - - if (access(NET_CMD_PATH, F_OK) != 0) - return (B_FALSE); + if (!avail) { + struct stat statbuf; - return (B_TRUE); -} + if (access(NET_CMD_PATH, F_OK) != 0 || + lstat(SHARE_DIR, &statbuf) != 0 || + !S_ISDIR(statbuf.st_mode)) + avail = -1; + else + avail = 1; + } -/* - * Initializes the SMB functionality of libshare. - */ -void -libshare_smb_init(void) -{ - smb_fstype = register_fstype("smb", &smb_shareops); + return (avail == 1); } diff --git a/lib/libshare/smb.h b/lib/libshare/smb.h index 8ea44677f9ae..23d27c7b9134 100644 --- a/lib/libshare/smb.h +++ b/lib/libshare/smb.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -43,7 +43,3 @@ typedef struct smb_share_s { struct smb_share_s *next; } smb_share_t; - -extern smb_share_t *smb_shares; - -void libshare_smb_init(void); diff --git a/lib/libspl/Makefile.am b/lib/libspl/Makefile.am index 8457df6dcdf4..822bef7e7a8d 100644 --- a/lib/libspl/Makefile.am +++ b/lib/libspl/Makefile.am @@ -1,47 +1,45 @@ -include $(top_srcdir)/config/Rules.am +include $(srcdir)/%D%/include/Makefile.am -SUBDIRS = include +libspl_assert_la_CFLAGS = $(AM_CFLAGS) $(LIBRARY_CFLAGS) +libspl_la_CFLAGS = $(libspl_assert_la_CFLAGS) -noinst_LTLIBRARIES = libspl_assert.la libspl.la +noinst_LTLIBRARIES += libspl_assert.la libspl.la +CPPCHECKTARGETS += libspl_assert.la libspl.la libspl_assert_la_SOURCES = \ - assert.c - -USER_C = \ - libspl_impl.h \ - atomic.c \ - getexecname.c \ - list.c \ - mkdirp.c \ - page.c \ - strlcat.c \ - strlcpy.c \ - timestamp.c \ - include/sys/list.h \ - include/sys/list_impl.h + %D%/assert.c + +libspl_la_SOURCES = \ + %D%/libspl_impl.h \ + %D%/atomic.c \ + %D%/getexecname.c \ + %D%/list.c \ + %D%/mkdirp.c \ + %D%/page.c \ + %D%/strlcat.c \ + %D%/strlcpy.c \ + %D%/timestamp.c \ + %D%/include/sys/list.h \ + %D%/include/sys/list_impl.h if BUILD_LINUX -USER_C += \ - os/linux/getexecname.c \ - os/linux/gethostid.c \ - os/linux/getmntany.c \ - os/linux/zone.c +libspl_la_SOURCES += \ + %D%/os/linux/getexecname.c \ + %D%/os/linux/gethostid.c \ + %D%/os/linux/getmntany.c \ + %D%/os/linux/zone.c endif if BUILD_FREEBSD -USER_C += \ - os/freebsd/getexecname.c \ - os/freebsd/gethostid.c \ - os/freebsd/getmntany.c \ - os/freebsd/mnttab.c \ - os/freebsd/zone.c +libspl_la_SOURCES += \ + %D%/os/freebsd/getexecname.c \ + %D%/os/freebsd/gethostid.c \ + %D%/os/freebsd/getmntany.c \ + %D%/os/freebsd/mnttab.c \ + %D%/os/freebsd/zone.c endif -libspl_la_SOURCES = $(USER_C) - libspl_la_LIBADD = \ libspl_assert.la libspl_la_LIBADD += $(LIBATOMIC_LIBS) $(LIBCLOCK_GETTIME) - -include $(top_srcdir)/config/CppCheck.am diff --git a/lib/libspl/assert.c b/lib/libspl/assert.c index ad8fdcd8cf0a..207da3a1d860 100644 --- a/lib/libspl/assert.c +++ b/lib/libspl/assert.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/atomic.c b/lib/libspl/atomic.c index 9307cea194e6..ba14b113f585 100644 --- a/lib/libspl/atomic.c +++ b/lib/libspl/atomic.c @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/getexecname.c b/lib/libspl/getexecname.c index dca7162034f7..95583b31ee6f 100644 --- a/lib/libspl/getexecname.c +++ b/lib/libspl/getexecname.c @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/Makefile.am b/lib/libspl/include/Makefile.am index 20996366030c..6f0e1818d22e 100644 --- a/lib/libspl/include/Makefile.am +++ b/lib/libspl/include/Makefile.am @@ -1,22 +1,94 @@ -SUBDIRS = rpc sys util os - libspldir = $(includedir)/libspl libspl_HEADERS = \ - assert.h \ - atomic.h \ - libdevinfo.h \ - libgen.h \ - libshare.h \ - limits.h \ - locale.h \ - statcommon.h \ - stdio.h \ - stdlib.h \ - string.h \ - stropts.h \ - thread.h \ - tzfile.h \ - ucred.h \ - umem.h \ - unistd.h \ - zone.h + %D%/assert.h \ + %D%/atomic.h \ + %D%/libgen.h \ + %D%/libshare.h \ + %D%/statcommon.h \ + %D%/stdlib.h \ + %D%/string.h \ + %D%/umem.h \ + %D%/unistd.h \ + %D%/zone.h + +if BUILD_FREEBSD +libspl_HEADERS += \ + %D%/os/freebsd/fcntl.h +endif + + +libspl_rpcdir = $(libspldir)/rpc +libspl_rpc_HEADERS = \ + %D%/rpc/xdr.h + + +libspl_sysdir = $(libspldir)/sys +libspl_sys_HEADERS = \ + %D%/sys/acl.h \ + %D%/sys/acl_impl.h \ + %D%/sys/callb.h \ + %D%/sys/cmn_err.h \ + %D%/sys/cred.h \ + %D%/sys/debug.h \ + %D%/sys/dkio.h \ + %D%/sys/dklabel.h \ + %D%/sys/feature_tests.h \ + %D%/sys/inttypes.h \ + %D%/sys/isa_defs.h \ + %D%/sys/kmem.h \ + %D%/sys/kstat.h \ + %D%/sys/list.h \ + %D%/sys/list_impl.h \ + %D%/sys/mhd.h \ + %D%/sys/mkdev.h \ + %D%/sys/policy.h \ + %D%/sys/poll.h \ + %D%/sys/priv.h \ + %D%/sys/processor.h \ + %D%/sys/sha2.h \ + %D%/sys/simd.h \ + %D%/sys/stack.h \ + %D%/sys/stdtypes.h \ + %D%/sys/string.h \ + %D%/sys/sunddi.h \ + %D%/sys/systeminfo.h \ + %D%/sys/time.h \ + %D%/sys/trace_spl.h \ + %D%/sys/trace_zfs.h \ + %D%/sys/types.h \ + %D%/sys/types32.h \ + %D%/sys/uio.h \ + %D%/sys/vnode.h \ + %D%/sys/wmsum.h \ + %D%/sys/zone.h + +if BUILD_LINUX +libspl_sys_HEADERS += \ + %D%/os/linux/sys/byteorder.h \ + %D%/os/linux/sys/errno.h \ + %D%/os/linux/sys/mnttab.h \ + %D%/os/linux/sys/mount.h \ + %D%/os/linux/sys/param.h \ + %D%/os/linux/sys/stat.h \ + %D%/os/linux/sys/sysmacros.h \ + %D%/os/linux/sys/zfs_context_os.h +endif + +if BUILD_FREEBSD +libspl_sys_HEADERS += \ + %D%/os/freebsd/sys/byteorder.h \ + %D%/os/freebsd/sys/fcntl.h \ + %D%/os/freebsd/sys/file.h \ + %D%/os/freebsd/sys/mnttab.h \ + %D%/os/freebsd/sys/mount.h \ + %D%/os/freebsd/sys/param.h \ + %D%/os/freebsd/sys/stat.h \ + %D%/os/freebsd/sys/sysmacros.h \ + %D%/os/freebsd/sys/vfs.h \ + %D%/os/freebsd/sys/zfs_context_os.h +endif + + +libspl_sys_dktpdir = $(libspl_sysdir)/dktp +libspl_sys_dktp_HEADERS = \ + %D%/sys/dktp/fdisk.h diff --git a/lib/libspl/include/assert.h b/lib/libspl/include/assert.h index e968a2310774..aaaa0af096ef 100644 --- a/lib/libspl/include/assert.h +++ b/lib/libspl/include/assert.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/atomic.h b/lib/libspl/include/atomic.h index 8dd1d654a486..1249d42b604a 100644 --- a/lib/libspl/include/atomic.h +++ b/lib/libspl/include/atomic.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/libdevinfo.h b/lib/libspl/include/libdevinfo.h deleted file mode 100644 index be1d291f4051..000000000000 --- a/lib/libspl/include/libdevinfo.h +++ /dev/null @@ -1,30 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License, Version 1.0 only - * (the "License"). You may not use this file except in compliance - * with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2008 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#ifndef _LIBSPL_LIBDEVINFO_H -#define _LIBSPL_LIBDEVINFO_H - -#endif /* _LIBSPL_LIBDEVINFO_H */ diff --git a/lib/libspl/include/libgen.h b/lib/libspl/include/libgen.h index c46d7454e49f..d8c80509e99e 100644 --- a/lib/libspl/include/libgen.h +++ b/lib/libspl/include/libgen.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/libshare.h b/lib/libspl/include/libshare.h index 5d06b163a3ba..d976f096ac39 100644 --- a/lib/libspl/include/libshare.h +++ b/lib/libspl/include/libshare.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -27,28 +27,28 @@ #ifndef _LIBSPL_LIBSHARE_H #define _LIBSPL_LIBSHARE_H extern __attribute__((visibility("default"))) -/* API Initialization */ -#define SA_INIT_SHARE_API 0x0001 /* init share specific interface */ -#define SA_INIT_CONTROL_API 0x0002 /* init control specific interface */ +#include /* * defined error values */ - #define SA_OK 0 -#define SA_NO_SUCH_PATH 1 /* provided path doesn't exist */ +#define SA_SYSTEM_ERR 7 /* system error, use errno */ +#define SA_SYNTAX_ERR 8 /* syntax error on command line */ #define SA_NO_MEMORY 2 /* no memory for data structures */ +#define SA_INVALID_PROTOCOL 13 /* specified protocol not valid */ +#define SA_NOT_SUPPORTED 21 /* operation not supported for proto */ + +/* The following errors are never returned by libshare */ +#define SA_NO_SUCH_PATH 1 /* provided path doesn't exist */ #define SA_DUPLICATE_NAME 3 /* object name is already in use */ #define SA_BAD_PATH 4 /* not a full path */ #define SA_NO_SUCH_GROUP 5 /* group is not defined */ #define SA_CONFIG_ERR 6 /* system configuration error */ -#define SA_SYSTEM_ERR 7 /* system error, use errno */ -#define SA_SYNTAX_ERR 8 /* syntax error on command line */ #define SA_NO_PERMISSION 9 /* no permission for operation */ #define SA_BUSY 10 /* resource is busy */ #define SA_NO_SUCH_PROP 11 /* property doesn't exist */ #define SA_INVALID_NAME 12 /* name of object is invalid */ -#define SA_INVALID_PROTOCOL 13 /* specified protocol not valid */ #define SA_NOT_ALLOWED 14 /* operation not allowed */ #define SA_BAD_VALUE 15 /* bad value for property */ #define SA_INVALID_SECURITY 16 /* invalid security type */ @@ -56,7 +56,6 @@ #define SA_VALUE_CONFLICT 18 /* property value conflict */ #define SA_NOT_IMPLEMENTED 19 /* plugin interface not implemented */ #define SA_INVALID_PATH 20 /* path is sub-dir of existing share */ -#define SA_NOT_SUPPORTED 21 /* operation not supported for proto */ #define SA_PROP_SHARE_ONLY 22 /* property valid on share only */ #define SA_NOT_SHARED 23 /* path is not shared */ #define SA_NO_SUCH_RESOURCE 24 /* resource not found */ @@ -71,16 +70,26 @@ #define SA_SHARE_EXISTS 33 /* path or file is already shared */ /* initialization */ -_LIBSPL_LIBSHARE_H char *sa_errorstr(int); +_LIBSPL_LIBSHARE_H const char *sa_errorstr(int); + +/* available protocols */ +enum sa_protocol { + SA_PROTOCOL_NFS, + SA_PROTOCOL_SMB, /* ABI: add before _COUNT */ + SA_PROTOCOL_COUNT, +}; + +/* lower-case */ +_LIBSPL_LIBSHARE_H const char *const sa_protocol_names[SA_PROTOCOL_COUNT]; /* share control */ _LIBSPL_LIBSHARE_H int sa_enable_share(const char *, const char *, const char *, - char *); -_LIBSPL_LIBSHARE_H int sa_disable_share(const char *, char *); -_LIBSPL_LIBSHARE_H boolean_t sa_is_shared(const char *, char *); -_LIBSPL_LIBSHARE_H void sa_commit_shares(const char *); + enum sa_protocol); +_LIBSPL_LIBSHARE_H int sa_disable_share(const char *, enum sa_protocol); +_LIBSPL_LIBSHARE_H boolean_t sa_is_shared(const char *, enum sa_protocol); +_LIBSPL_LIBSHARE_H void sa_commit_shares(enum sa_protocol); /* protocol specific interfaces */ -_LIBSPL_LIBSHARE_H int sa_validate_shareopts(char *, char *); +_LIBSPL_LIBSHARE_H int sa_validate_shareopts(const char *, enum sa_protocol); #endif /* _LIBSPL_LIBSHARE_H */ diff --git a/lib/libspl/include/limits.h b/lib/libspl/include/limits.h deleted file mode 100644 index 5d996eb846d1..000000000000 --- a/lib/libspl/include/limits.h +++ /dev/null @@ -1,45 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License, Version 1.0 only - * (the "License"). You may not use this file except in compliance - * with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2008 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#include_next -#include - -#ifndef _LIBSPL_LIMITS_H -#define _LIBSPL_LIMITS_H - -#ifndef DBL_DIG -#define DBL_DIG 15 -#define DBL_MAX 1.7976931348623157081452E+308 -#define DBL_MIN 2.2250738585072013830903E-308 -#endif - -#ifndef FLT_DIG -#define FLT_DIG 6 -#define FLT_MAX 3.4028234663852885981170E+38F -#define FLT_MIN 1.1754943508222875079688E-38F -#endif - -#endif /* _LIBSPL_LIMITS_H */ diff --git a/lib/libspl/include/locale.h b/lib/libspl/include/locale.h deleted file mode 100644 index 6c74df72072e..000000000000 --- a/lib/libspl/include/locale.h +++ /dev/null @@ -1,35 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License, Version 1.0 only - * (the "License"). You may not use this file except in compliance - * with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2008 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#include_next - -#ifndef _LIBSPL_LOCALE_H -#define _LIBSPL_LOCALE_H - -#include -#include - -#endif diff --git a/lib/libspl/include/os/Makefile.am b/lib/libspl/include/os/Makefile.am deleted file mode 100644 index 7b362e02ad59..000000000000 --- a/lib/libspl/include/os/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -if BUILD_FREEBSD -SUBDIRS = freebsd -endif - -if BUILD_LINUX -SUBDIRS = linux -endif diff --git a/lib/libspl/include/os/freebsd/Makefile.am b/lib/libspl/include/os/freebsd/Makefile.am deleted file mode 100644 index f06325ee3e4e..000000000000 --- a/lib/libspl/include/os/freebsd/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -SUBDIRS = sys - -libspldir = $(includedir)/libspl -libspl_HEADERS = \ - fcntl.h diff --git a/lib/libspl/include/os/freebsd/sys/Makefile.am b/lib/libspl/include/os/freebsd/sys/Makefile.am deleted file mode 100644 index 7a854608079c..000000000000 --- a/lib/libspl/include/os/freebsd/sys/Makefile.am +++ /dev/null @@ -1,12 +0,0 @@ -libspldir = $(includedir)/libspl/sys -libspl_HEADERS = \ - byteorder.h \ - fcntl.h \ - file.h \ - mnttab.h \ - mount.h \ - param.h \ - stat.h \ - sysmacros.h \ - vfs.h \ - zfs_context_os.h diff --git a/lib/libspl/include/os/freebsd/sys/byteorder.h b/lib/libspl/include/os/freebsd/sys/byteorder.h index cd692d3616e0..8de1104dc8a2 100644 --- a/lib/libspl/include/os/freebsd/sys/byteorder.h +++ b/lib/libspl/include/os/freebsd/sys/byteorder.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -43,7 +43,7 @@ #include #include #include -#include +#include #if defined(__GNUC__) && defined(_ASM_INLINES) && \ (defined(__i386) || defined(__amd64)) diff --git a/lib/libspl/include/os/freebsd/sys/file.h b/lib/libspl/include/os/freebsd/sys/file.h index 27fd2888f326..f46b883f524a 100644 --- a/lib/libspl/include/os/freebsd/sys/file.h +++ b/lib/libspl/include/os/freebsd/sys/file.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -29,14 +29,6 @@ #include_next -#define FCREAT O_CREAT -#define FTRUNC O_TRUNC -#define FSYNC O_SYNC -#define FDSYNC O_DSYNC -#define FEXCL O_EXCL - -#define FNODSYNC 0x10000 /* fsync pseudo flag */ -#define FNOFOLLOW 0x20000 /* don't follow symlinks */ #define FIGNORECASE 0x80000 /* request case-insensitive lookups */ #endif diff --git a/lib/libspl/include/os/freebsd/sys/mnttab.h b/lib/libspl/include/os/freebsd/sys/mnttab.h index c08349bdf9bd..514e1b67547a 100644 --- a/lib/libspl/include/os/freebsd/sys/mnttab.h +++ b/lib/libspl/include/os/freebsd/sys/mnttab.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -79,7 +79,7 @@ extern int _sol_getmntent(FILE *fp, struct mnttab *mp); extern int getextmntent(const char *path, struct extmnttab *entry, struct stat64 *statbuf); extern void statfs2mnttab(struct statfs *sfs, struct mnttab *mp); -char *hasmntopt(struct mnttab *mnt, char *opt); -int getmntent(FILE *fp, struct mnttab *mp); +extern char *hasmntopt(struct mnttab *mnt, const char *opt); +extern int getmntent(FILE *fp, struct mnttab *mp); #endif diff --git a/lib/libspl/include/os/freebsd/sys/mount.h b/lib/libspl/include/os/freebsd/sys/mount.h index e99518571270..61c421fe4edb 100644 --- a/lib/libspl/include/os/freebsd/sys/mount.h +++ b/lib/libspl/include/os/freebsd/sys/mount.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/os/freebsd/sys/param.h b/lib/libspl/include/os/freebsd/sys/param.h index cb5260ea3d7e..15d3ff0dcb56 100644 --- a/lib/libspl/include/os/freebsd/sys/param.h +++ b/lib/libspl/include/os/freebsd/sys/param.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/os/freebsd/sys/stat.h b/lib/libspl/include/os/freebsd/sys/stat.h index 38c684d62a1b..88773cceb951 100644 --- a/lib/libspl/include/os/freebsd/sys/stat.h +++ b/lib/libspl/include/os/freebsd/sys/stat.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/os/freebsd/sys/zfs_context_os.h b/lib/libspl/include/os/freebsd/sys/zfs_context_os.h index b9bf487c2aef..9b9d69bddcf7 100644 --- a/lib/libspl/include/os/freebsd/sys/zfs_context_os.h +++ b/lib/libspl/include/os/freebsd/sys/zfs_context_os.h @@ -30,6 +30,5 @@ #define ZFS_CONTEXT_OS_H_ #define HAVE_LARGE_STACKS 1 -#define ZFS_EXPORTS_PATH "/etc/zfs/exports" #endif diff --git a/lib/libspl/include/os/linux/Makefile.am b/lib/libspl/include/os/linux/Makefile.am deleted file mode 100644 index 081839c48c8f..000000000000 --- a/lib/libspl/include/os/linux/Makefile.am +++ /dev/null @@ -1 +0,0 @@ -SUBDIRS = sys diff --git a/lib/libspl/include/os/linux/sys/Makefile.am b/lib/libspl/include/os/linux/sys/Makefile.am deleted file mode 100644 index 1ec07a76d354..000000000000 --- a/lib/libspl/include/os/linux/sys/Makefile.am +++ /dev/null @@ -1,10 +0,0 @@ -libspldir = $(includedir)/libspl/sys -libspl_HEADERS = \ - byteorder.h \ - errno.h \ - mnttab.h \ - mount.h \ - param.h \ - stat.h \ - sysmacros.h \ - zfs_context_os.h diff --git a/lib/libspl/include/os/linux/sys/byteorder.h b/lib/libspl/include/os/linux/sys/byteorder.h index d5ee3e26f5a5..c8413ea76352 100644 --- a/lib/libspl/include/os/linux/sys/byteorder.h +++ b/lib/libspl/include/os/linux/sys/byteorder.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -46,7 +46,7 @@ #endif #include -#include +#include #ifdef __cplusplus extern "C" { diff --git a/lib/libspl/include/os/linux/sys/errno.h b/lib/libspl/include/os/linux/sys/errno.h index 30d20ab895c5..f0ad60b0bcff 100644 --- a/lib/libspl/include/os/linux/sys/errno.h +++ b/lib/libspl/include/os/linux/sys/errno.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/os/linux/sys/mnttab.h b/lib/libspl/include/os/linux/sys/mnttab.h index 1957293d5c69..7b540a383d27 100644 --- a/lib/libspl/include/os/linux/sys/mnttab.h +++ b/lib/libspl/include/os/linux/sys/mnttab.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -74,7 +74,7 @@ extern int getmntany(FILE *fp, struct mnttab *mp, struct mnttab *mpref); extern int _sol_getmntent(FILE *fp, struct mnttab *mp); extern int getextmntent(const char *path, struct extmnttab *mp, struct stat64 *statbuf); -static inline char *_sol_hasmntopt(struct mnttab *mnt, char *opt) +static inline char *_sol_hasmntopt(struct mnttab *mnt, const char *opt) { struct mntent mnt_new; diff --git a/lib/libspl/include/os/linux/sys/mount.h b/lib/libspl/include/os/linux/sys/mount.h index d7c6f750e23d..a98fd6974b90 100644 --- a/lib/libspl/include/os/linux/sys/mount.h +++ b/lib/libspl/include/os/linux/sys/mount.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/os/linux/sys/param.h b/lib/libspl/include/os/linux/sys/param.h index 26335187fdca..c35678529ba8 100644 --- a/lib/libspl/include/os/linux/sys/param.h +++ b/lib/libspl/include/os/linux/sys/param.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/os/linux/sys/stat.h b/lib/libspl/include/os/linux/sys/stat.h index 3e8d27e4c19a..e7f592620512 100644 --- a/lib/libspl/include/os/linux/sys/stat.h +++ b/lib/libspl/include/os/linux/sys/stat.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/os/linux/sys/sysmacros.h b/lib/libspl/include/os/linux/sys/sysmacros.h index 31f347c6fd5a..5765ee25c6cb 100644 --- a/lib/libspl/include/os/linux/sys/sysmacros.h +++ b/lib/libspl/include/os/linux/sys/sysmacros.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/os/linux/sys/zfs_context_os.h b/lib/libspl/include/os/linux/sys/zfs_context_os.h index 81ced5207749..04abfa09d8f1 100644 --- a/lib/libspl/include/os/linux/sys/zfs_context_os.h +++ b/lib/libspl/include/os/linux/sys/zfs_context_os.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/rpc/Makefile.am b/lib/libspl/include/rpc/Makefile.am deleted file mode 100644 index 7fe1d7fea4d7..000000000000 --- a/lib/libspl/include/rpc/Makefile.am +++ /dev/null @@ -1,3 +0,0 @@ -libspldir = $(includedir)/libspl/rpc -libspl_HEADERS = \ - xdr.h diff --git a/lib/libspl/include/rpc/xdr.h b/lib/libspl/include/rpc/xdr.h index 51d71f693bbf..882b72456c21 100644 --- a/lib/libspl/include/rpc/xdr.h +++ b/lib/libspl/include/rpc/xdr.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/statcommon.h b/lib/libspl/include/statcommon.h index 1f376f5c7c24..971997a447a5 100644 --- a/lib/libspl/include/statcommon.h +++ b/lib/libspl/include/statcommon.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/stdio.h b/lib/libspl/include/stdio.h deleted file mode 100644 index 6152b09f1a97..000000000000 --- a/lib/libspl/include/stdio.h +++ /dev/null @@ -1,34 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License, Version 1.0 only - * (the "License"). You may not use this file except in compliance - * with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2008 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#include_next - -#ifndef _LIBSPL_STDIO_H -#define _LIBSPL_STDIO_H - -#define enable_extended_FILE_stdio(fd, sig) ((void) 0) - -#endif diff --git a/lib/libspl/include/stdlib.h b/lib/libspl/include/stdlib.h index a4ce4f781fc5..2dd371463d40 100644 --- a/lib/libspl/include/stdlib.h +++ b/lib/libspl/include/stdlib.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/string.h b/lib/libspl/include/string.h index a7d40fa61943..f8605790a6f3 100644 --- a/lib/libspl/include/string.h +++ b/lib/libspl/include/string.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/stropts.h b/lib/libspl/include/stropts.h deleted file mode 100644 index 37acd4052b0b..000000000000 --- a/lib/libspl/include/stropts.h +++ /dev/null @@ -1,25 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License (the "License"). - * You may not use this file except in compliance with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ - -#ifndef _LIBSPL_STROPTS_H -#define _LIBSPL_STROPTS_H - -#endif /* _LIBSPL_STROPTS_H */ diff --git a/lib/libspl/include/sys/Makefile.am b/lib/libspl/include/sys/Makefile.am deleted file mode 100644 index 6816a012533f..000000000000 --- a/lib/libspl/include/sys/Makefile.am +++ /dev/null @@ -1,48 +0,0 @@ -SUBDIRS = dktp - -libspldir = $(includedir)/libspl/sys -libspl_HEADERS = \ - acl.h \ - acl_impl.h \ - callb.h \ - cmn_err.h \ - cred.h \ - debug.h \ - dkio.h \ - dklabel.h \ - feature_tests.h \ - int_limits.h \ - int_types.h \ - inttypes.h \ - isa_defs.h \ - kmem.h \ - kstat.h \ - list.h \ - list_impl.h \ - mhd.h \ - mkdev.h \ - policy.h \ - poll.h \ - priv.h \ - processor.h \ - sha2.h \ - simd.h \ - stack.h \ - stdtypes.h \ - strings.h \ - stropts.h \ - sunddi.h \ - systeminfo.h \ - time.h \ - trace_spl.h \ - trace_zfs.h \ - types32.h \ - types.h \ - tzfile.h \ - uio.h \ - va_list.h \ - varargs.h \ - vnode.h \ - vtoc.h \ - wmsum.h \ - zone.h diff --git a/lib/libspl/include/sys/acl.h b/lib/libspl/include/sys/acl.h index 31168421b088..43646f3e28fc 100644 --- a/lib/libspl/include/sys/acl.h +++ b/lib/libspl/include/sys/acl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/acl_impl.h b/lib/libspl/include/sys/acl_impl.h index 717334906415..e7f7bcd907cd 100644 --- a/lib/libspl/include/sys/acl_impl.h +++ b/lib/libspl/include/sys/acl_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/callb.h b/lib/libspl/include/sys/callb.h index 8ffd18788865..783208ed16a4 100644 --- a/lib/libspl/include/sys/callb.h +++ b/lib/libspl/include/sys/callb.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/cmn_err.h b/lib/libspl/include/sys/cmn_err.h index 63ff4eb29bc8..f4db082c22a4 100644 --- a/lib/libspl/include/sys/cmn_err.h +++ b/lib/libspl/include/sys/cmn_err.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/cred.h b/lib/libspl/include/sys/cred.h index 463b3abfc977..45eb65089df1 100644 --- a/lib/libspl/include/sys/cred.h +++ b/lib/libspl/include/sys/cred.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/debug.h b/lib/libspl/include/sys/debug.h index af18da94804c..ef3f0afb68c3 100644 --- a/lib/libspl/include/sys/debug.h +++ b/lib/libspl/include/sys/debug.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/dkio.h b/lib/libspl/include/sys/dkio.h index f3c641f669b7..9517b580bdf5 100644 --- a/lib/libspl/include/sys/dkio.h +++ b/lib/libspl/include/sys/dkio.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/dklabel.h b/lib/libspl/include/sys/dklabel.h index 8c2ca06c0cbc..a1fddd34799a 100644 --- a/lib/libspl/include/sys/dklabel.h +++ b/lib/libspl/include/sys/dklabel.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/dktp/Makefile.am b/lib/libspl/include/sys/dktp/Makefile.am deleted file mode 100644 index 4ad3695d8abc..000000000000 --- a/lib/libspl/include/sys/dktp/Makefile.am +++ /dev/null @@ -1,4 +0,0 @@ -libspldir = $(includedir)/libspl/sys/dktp -libspl_HEADERS = \ - fdisk.h - diff --git a/lib/libspl/include/sys/dktp/fdisk.h b/lib/libspl/include/sys/dktp/fdisk.h index e90135f362e6..53db7d815438 100644 --- a/lib/libspl/include/sys/dktp/fdisk.h +++ b/lib/libspl/include/sys/dktp/fdisk.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/feature_tests.h b/lib/libspl/include/sys/feature_tests.h index be0721b50529..396052f125af 100644 --- a/lib/libspl/include/sys/feature_tests.h +++ b/lib/libspl/include/sys/feature_tests.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -28,7 +28,6 @@ #define _SYS_FEATURE_TESTS_H #define ____cacheline_aligned -#define __NORETURN __attribute__((__noreturn__)) #if !defined(zfs_fallthrough) && !defined(_LIBCPP_VERSION) #if defined(HAVE_IMPLICIT_FALLTHROUGH) diff --git a/lib/libspl/include/sys/int_limits.h b/lib/libspl/include/sys/int_limits.h deleted file mode 100644 index 7af68cdb2998..000000000000 --- a/lib/libspl/include/sys/int_limits.h +++ /dev/null @@ -1,30 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License, Version 1.0 only - * (the "License"). You may not use this file except in compliance - * with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2006 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#ifndef _LIBSPL_SYS_INT_LIMITS_H -#define _LIBSPL_SYS_INT_LIMITS_H - -#endif diff --git a/lib/libspl/include/sys/int_types.h b/lib/libspl/include/sys/int_types.h deleted file mode 100644 index 51e9e0285490..000000000000 --- a/lib/libspl/include/sys/int_types.h +++ /dev/null @@ -1,32 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License, Version 1.0 only - * (the "License"). You may not use this file except in compliance - * with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2006 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#ifndef _SOL_SYS_INT_TYPES_H -#define _SOL_SYS_INT_TYPES_H - -#include - -#endif diff --git a/lib/libspl/include/sys/inttypes.h b/lib/libspl/include/sys/inttypes.h index d7d063985316..85ddef9ee74c 100644 --- a/lib/libspl/include/sys/inttypes.h +++ b/lib/libspl/include/sys/inttypes.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/isa_defs.h b/lib/libspl/include/sys/isa_defs.h index 8c0932f57654..756adff15ac8 100644 --- a/lib/libspl/include/sys/isa_defs.h +++ b/lib/libspl/include/sys/isa_defs.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -126,7 +126,7 @@ extern "C" { #endif /* arm arch specific defines */ -#elif defined(__arm) || defined(__arm__) || defined(__aarch64__) +#elif defined(__arm) || defined(__arm__) #if !defined(__arm) #define __arm @@ -136,17 +136,11 @@ extern "C" { #define __arm__ #endif -#if defined(__aarch64__) -#if !defined(_LP64) -#define _LP64 -#endif -#else #if !defined(_ILP32) #define _ILP32 #endif -#endif -#if defined(__ARMEL__) || defined(__AARCH64EL__) +#if defined(__ARMEL__) #define _ZFS_LITTLE_ENDIAN #else #define _ZFS_BIG_ENDIAN @@ -158,6 +152,21 @@ extern "C" { #define HAVE_EFFICIENT_UNALIGNED_ACCESS #endif +/* aarch64 arch specific defines */ +#elif defined(__aarch64__) + +#if !defined(_LP64) +#define _LP64 +#endif + +#if defined(__AARCH64EL__) +#define _ZFS_LITTLE_ENDIAN +#else +#define _ZFS_BIG_ENDIAN +#endif + +#define _SUNOS_VTOC_16 + /* sparc arch specific defines */ #elif defined(__sparc) || defined(__sparc__) diff --git a/lib/libspl/include/sys/kmem.h b/lib/libspl/include/sys/kmem.h index 06c614e51d1c..9a4f63d0221c 100644 --- a/lib/libspl/include/sys/kmem.h +++ b/lib/libspl/include/sys/kmem.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/kstat.h b/lib/libspl/include/sys/kstat.h index f73fb92eb797..e6057bf6b98e 100644 --- a/lib/libspl/include/sys/kstat.h +++ b/lib/libspl/include/sys/kstat.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -92,39 +92,6 @@ typedef struct kstat { void *ks_lock; /* protects this kstat's data */ } kstat_t; -#ifdef _SYSCALL32 - -typedef int32_t kid32_t; - -typedef struct kstat32 { - /* - * Fields relevant to both kernel and user - */ - hrtime_t ks_crtime; - caddr32_t ks_next; /* struct kstat pointer */ - kid32_t ks_kid; - char ks_module[KSTAT_STRLEN]; - uint8_t ks_resv; - int32_t ks_instance; - char ks_name[KSTAT_STRLEN]; - uint8_t ks_type; - char ks_class[KSTAT_STRLEN]; - uint8_t ks_flags; - caddr32_t ks_data; /* type-specific data */ - uint32_t ks_ndata; - size32_t ks_data_size; - hrtime_t ks_snaptime; - /* - * Fields relevant to kernel only (only needed here for padding) - */ - int32_t _ks_update; - caddr32_t _ks_private; - int32_t _ks_snapshot; - caddr32_t _ks_lock; -} kstat32_t; - -#endif /* _SYSCALL32 */ - /* * kstat structure and locking strategy * @@ -383,9 +350,9 @@ typedef struct kstat32 { * * ksp->ks_snaptime = gethrtime(); * if (rw == KSTAT_WRITE) - * bcopy(buf, ksp->ks_data, ksp->ks_data_size); + * memcpy(ksp->ks_data, buf, ksp->ks_data_size); * else - * bcopy(ksp->ks_data, buf, ksp->ks_data_size); + * memcpy(buf, ksp->ks_data, ksp->ks_data_size); * return (0); * * A more illuminating example is taking a snapshot of a linked list: @@ -394,7 +361,7 @@ typedef struct kstat32 { * if (rw == KSTAT_WRITE) * return (EACCES); ... See below ... * for (foo = first_foo; foo; foo = foo->next) { - * bcopy((char *) foo, (char *) buf, sizeof (struct foo)); + * memcpy(buf, foo, sizeof (struct foo)); * buf = ((struct foo *) buf) + 1; * } * return (0); @@ -423,12 +390,12 @@ typedef struct kstat32 { * uint_t i; * * ... Do the regular copy ... - * bcopy(ksp->ks_data, buf, sizeof (kstat_named_t) * ksp->ks_ndata); + * memcpy(buf, ksp->ks_data, sizeof (kstat_named_t) * ksp->ks_ndata); * * for (i = 0; i < ksp->ks_ndata; i++, knp++) { * if (knp[i].data_type == KSTAT_DATA_STRING && * KSTAT_NAMED_STR_PTR(knp) != NULL) { - * bcopy(KSTAT_NAMED_STR_PTR(knp), end, + * memcpy(end, KSTAT_NAMED_STR_PTR(knp), * KSTAT_NAMED_STR_BUFLEN(knp)); * KSTAT_NAMED_STR_PTR(knp) = end; * end += KSTAT_NAMED_STR_BUFLEN(knp); @@ -467,7 +434,7 @@ typedef struct kstat_named { * 64-bit compilation environments or 32-bit non-maximally conformant * C89 or C90 ANSI C compilation environments (cc -Xt and cc -Xa). In the * C99 ANSI C compilation environment, the long long type is supported. - * The _INT64_TYPE is defined by the implementation (see sys/int_types.h). + * The _INT64_TYPE is defined by the implementation (see sys/inttypes.h). */ #if defined(_INT64_TYPE) int64_t i64; diff --git a/lib/libspl/include/sys/list.h b/lib/libspl/include/sys/list.h index 6db92ed42955..6c32402a7bd1 100644 --- a/lib/libspl/include/sys/list.h +++ b/lib/libspl/include/sys/list.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/list_impl.h b/lib/libspl/include/sys/list_impl.h index b5655b972c11..24c1ceb2a9fa 100644 --- a/lib/libspl/include/sys/list_impl.h +++ b/lib/libspl/include/sys/list_impl.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/mhd.h b/lib/libspl/include/sys/mhd.h index fcc062d51c84..956009b499ee 100644 --- a/lib/libspl/include/sys/mhd.h +++ b/lib/libspl/include/sys/mhd.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/mkdev.h b/lib/libspl/include/sys/mkdev.h index 5978de65de50..4878920f3f13 100644 --- a/lib/libspl/include/sys/mkdev.h +++ b/lib/libspl/include/sys/mkdev.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/policy.h b/lib/libspl/include/sys/policy.h index 2f695b35cd0b..b0a810f09841 100644 --- a/lib/libspl/include/sys/policy.h +++ b/lib/libspl/include/sys/policy.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/poll.h b/lib/libspl/include/sys/poll.h index 6ab0bddc0be5..8f6d591537b2 100644 --- a/lib/libspl/include/sys/poll.h +++ b/lib/libspl/include/sys/poll.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/priv.h b/lib/libspl/include/sys/priv.h index 76c76d1830c2..c3f8b7bc41c7 100644 --- a/lib/libspl/include/sys/priv.h +++ b/lib/libspl/include/sys/priv.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/processor.h b/lib/libspl/include/sys/processor.h index 78e95d01f1b5..d778438c1b3f 100644 --- a/lib/libspl/include/sys/processor.h +++ b/lib/libspl/include/sys/processor.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/sha2.h b/lib/libspl/include/sys/sha2.h index e2f66d225e25..40db1a678cea 100644 --- a/lib/libspl/include/sys/sha2.h +++ b/lib/libspl/include/sys/sha2.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -33,9 +33,6 @@ extern "C" { #endif -#define SHA2_HMAC_MIN_KEY_LEN 1 /* SHA2-HMAC min key length in bytes */ -#define SHA2_HMAC_MAX_KEY_LEN INT_MAX /* SHA2-HMAC max key length in bytes */ - #define SHA256_DIGEST_LENGTH 32 /* SHA256 digest length in bytes */ #define SHA384_DIGEST_LENGTH 48 /* SHA384 digest length in bytes */ #define SHA512_DIGEST_LENGTH 64 /* SHA512 digest length in bytes */ diff --git a/lib/libspl/include/sys/simd.h b/lib/libspl/include/sys/simd.h index dceedb698fe0..c9d86a0808f1 100644 --- a/lib/libspl/include/sys/simd.h +++ b/lib/libspl/include/sys/simd.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -468,6 +468,7 @@ zfs_avx512vbmi_available(void) static jmp_buf env; static void sigillhandler(int x) { + (void) x; longjmp(env, 1); } #endif @@ -490,6 +491,24 @@ zfs_altivec_available(void) #endif return (has_altivec); } +static inline boolean_t +zfs_vsx_available(void) +{ + boolean_t has_vsx = B_FALSE; +#if defined(__ALTIVEC__) && !defined(__FreeBSD__) + sighandler_t savesig; + savesig = signal(SIGILL, sigillhandler); + if (setjmp(env)) { + signal(SIGILL, savesig); + has_vsx = B_FALSE; + } else { + __asm__ __volatile__("xssubsp 0,0,0\n"); + signal(SIGILL, savesig); + has_vsx = B_TRUE; + } +#endif + return (has_vsx); +} #else #define kfpu_allowed() 0 diff --git a/lib/libspl/include/sys/stack.h b/lib/libspl/include/sys/stack.h index 59807e97b6a0..ed63d3cb9da2 100644 --- a/lib/libspl/include/sys/stack.h +++ b/lib/libspl/include/sys/stack.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/stdtypes.h b/lib/libspl/include/sys/stdtypes.h index c26e2dc96c4e..9ad51a03f27c 100644 --- a/lib/libspl/include/sys/stdtypes.h +++ b/lib/libspl/include/sys/stdtypes.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/string.h b/lib/libspl/include/sys/string.h new file mode 100644 index 000000000000..3b2f5900276f --- /dev/null +++ b/lib/libspl/include/sys/string.h @@ -0,0 +1 @@ +#include diff --git a/lib/libspl/include/sys/strings.h b/lib/libspl/include/sys/strings.h deleted file mode 100644 index c142047dcdb8..000000000000 --- a/lib/libspl/include/sys/strings.h +++ /dev/null @@ -1,33 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License, Version 1.0 only - * (the "License"). You may not use this file except in compliance - * with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2006 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#ifndef _LIBSPL_SYS_STRINGS_H -#define _LIBSPL_SYS_STRINGS_H - -#include -#include - -#endif diff --git a/lib/libspl/include/sys/sunddi.h b/lib/libspl/include/sys/sunddi.h index ccd2b29b9b09..4e2c370f7b80 100644 --- a/lib/libspl/include/sys/sunddi.h +++ b/lib/libspl/include/sys/sunddi.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/systeminfo.h b/lib/libspl/include/sys/systeminfo.h index cc6c1793c00e..e0418db89fc6 100644 --- a/lib/libspl/include/sys/systeminfo.h +++ b/lib/libspl/include/sys/systeminfo.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/time.h b/lib/libspl/include/sys/time.h index c9f6165047d2..e692415a041c 100644 --- a/lib/libspl/include/sys/time.h +++ b/lib/libspl/include/sys/time.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/types.h b/lib/libspl/include/sys/types.h index ea02ffac93ac..5a84123d201f 100644 --- a/lib/libspl/include/sys/types.h +++ b/lib/libspl/include/sys/types.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -37,14 +37,14 @@ #include #include_next #include -#include +#include #include #ifndef HAVE_INTTYPES #include #endif /* HAVE_INTTYPES */ -typedef int zoneid_t; +typedef uint_t zoneid_t; typedef int projid_t; /* diff --git a/lib/libspl/include/sys/types32.h b/lib/libspl/include/sys/types32.h index bb41aa34bee0..eadc67c7122a 100644 --- a/lib/libspl/include/sys/types32.h +++ b/lib/libspl/include/sys/types32.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/tzfile.h b/lib/libspl/include/sys/tzfile.h deleted file mode 100644 index e30e7566366e..000000000000 --- a/lib/libspl/include/sys/tzfile.h +++ /dev/null @@ -1,164 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License (the "License"). - * You may not use this file except in compliance with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2007 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -/* - * from Arthur Olson's 6.1 - */ - -#ifndef _LIBSPL_SYS_TZFILE_H -#define _LIBSPL_SYS_TZFILE_H - -#ifdef __cplusplus -extern "C" { -#endif - -/* - * Information about time zone files. - */ - -#define TZDIR "/usr/share/lib/zoneinfo" /* Time zone object file directory */ - -#define TZDEFAULT (getenv("TZ")) - -#define TZDEFRULES "posixrules" - -/* - * Each file begins with. . . - */ - -struct tzhead { - char tzh_reserved[24]; /* reserved for future use */ - char tzh_ttisstdcnt[4]; /* coded number of trans. time flags */ - char tzh_leapcnt[4]; /* coded number of leap seconds */ - char tzh_timecnt[4]; /* coded number of transition times */ - char tzh_typecnt[4]; /* coded number of local time types */ - char tzh_charcnt[4]; /* coded number of abbr. chars */ -}; - -/* - * . . .followed by. . . - * - * tzh_timecnt (char [4])s coded transition times a la time(2) - * tzh_timecnt (unsigned char)s types of local time starting at above - * tzh_typecnt repetitions of - * one (char [4]) coded GMT offset in seconds - * one (unsigned char) used to set tm_isdst - * one (unsigned char) that's an abbreviation list index - * tzh_charcnt (char)s '\0'-terminated zone abbreviations - * tzh_leapcnt repetitions of - * one (char [4]) coded leap second transition times - * one (char [4]) total correction after above - * tzh_ttisstdcnt (char)s indexed by type; if TRUE, transition - * time is standard time, if FALSE, - * transition time is wall clock time - * if absent, transition times are - * assumed to be wall clock time - */ - -/* - * In the current implementation, "tzset()" refuses to deal with files that - * exceed any of the limits below. - */ - -/* - * The TZ_MAX_TIMES value below is enough to handle a bit more than a - * year's worth of solar time (corrected daily to the nearest second) or - * 138 years of Pacific Presidential Election time - * (where there are three time zone transitions every fourth year). - */ -#define TZ_MAX_TIMES 370 - -#define TZ_MAX_TYPES 256 /* Limited by what (unsigned char)'s can hold */ - -#define TZ_MAX_CHARS 50 /* Maximum number of abbreviation characters */ - -#define TZ_MAX_LEAPS 50 /* Maximum number of leap second corrections */ - -#define SECSPERMIN 60 -#define MINSPERHOUR 60 -#define HOURSPERDAY 24 -#define DAYSPERWEEK 7 -#define DAYSPERNYEAR 365 -#define DAYSPERLYEAR 366 -#define SECSPERHOUR (SECSPERMIN * MINSPERHOUR) -#define SECSPERDAY ((long)SECSPERHOUR * HOURSPERDAY) -#define MONSPERYEAR 12 - -#define TM_SUNDAY 0 -#define TM_MONDAY 1 -#define TM_TUESDAY 2 -#define TM_WEDNESDAY 3 -#define TM_THURSDAY 4 -#define TM_FRIDAY 5 -#define TM_SATURDAY 6 - -#define TM_JANUARY 0 -#define TM_FEBRUARY 1 -#define TM_MARCH 2 -#define TM_APRIL 3 -#define TM_MAY 4 -#define TM_JUNE 5 -#define TM_JULY 6 -#define TM_AUGUST 7 -#define TM_SEPTEMBER 8 -#define TM_OCTOBER 9 -#define TM_NOVEMBER 10 -#define TM_DECEMBER 11 - -#define TM_YEAR_BASE 1900 - -#define EPOCH_YEAR 1970 -#define EPOCH_WDAY TM_THURSDAY - -/* - * Accurate only for the past couple of centuries; - * that will probably do. - */ - -#define isleap(y) (((y) % 4) == 0 && ((y) % 100) != 0 || ((y) % 400) == 0) - -/* - * Use of the underscored variants may cause problems if you move your code to - * certain System-V-based systems; for maximum portability, use the - * underscore-free variants. The underscored variants are provided for - * backward compatibility only; they may disappear from future versions of - * this file. - */ - -#define SECS_PER_MIN SECSPERMIN -#define MINS_PER_HOUR MINSPERHOUR -#define HOURS_PER_DAY HOURSPERDAY -#define DAYS_PER_WEEK DAYSPERWEEK -#define DAYS_PER_NYEAR DAYSPERNYEAR -#define DAYS_PER_LYEAR DAYSPERLYEAR -#define SECS_PER_HOUR SECSPERHOUR -#define SECS_PER_DAY SECSPERDAY -#define MONS_PER_YEAR MONSPERYEAR - -#ifdef __cplusplus -} -#endif - -#endif /* _LIBSPL_SYS_TZFILE_H */ diff --git a/lib/libspl/include/sys/uio.h b/lib/libspl/include/sys/uio.h index 81ade54b5409..e9e21819d4f8 100644 --- a/lib/libspl/include/sys/uio.h +++ b/lib/libspl/include/sys/uio.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/va_list.h b/lib/libspl/include/sys/va_list.h deleted file mode 100644 index a36f5c77daa9..000000000000 --- a/lib/libspl/include/sys/va_list.h +++ /dev/null @@ -1,32 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License, Version 1.0 only - * (the "License"). You may not use this file except in compliance - * with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2006 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#ifndef _SYS_VA_LIST_H -#define _SYS_VA_LIST_H - -#include - -#endif diff --git a/lib/libspl/include/sys/varargs.h b/lib/libspl/include/sys/varargs.h deleted file mode 100644 index 3d00a3361d87..000000000000 --- a/lib/libspl/include/sys/varargs.h +++ /dev/null @@ -1,30 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License, Version 1.0 only - * (the "License"). You may not use this file except in compliance - * with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2006 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#ifndef _LIBSPL_SYS_VARARGS_H -#define _LIBSPL_SYS_VARARGS_H - -#endif diff --git a/lib/libspl/include/sys/vnode.h b/lib/libspl/include/sys/vnode.h index efcdd2c5a5ae..111cc8fd8357 100644 --- a/lib/libspl/include/sys/vnode.h +++ b/lib/libspl/include/sys/vnode.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/sys/vtoc.h b/lib/libspl/include/sys/vtoc.h deleted file mode 100644 index 5d8448b628dc..000000000000 --- a/lib/libspl/include/sys/vtoc.h +++ /dev/null @@ -1,350 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License (the "License"). - * You may not use this file except in compliance with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ - -/* - * Copyright 2009 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - - -/* Copyright (c) 1984, 1986, 1987, 1988, 1989 AT&T */ -/* All Rights Reserved */ - - -#ifndef _SYS_VTOC_H -#define _SYS_VTOC_H - -#include - -#ifdef __cplusplus -extern "C" { -#endif - -/* - * Note: the VTOC is not implemented fully, nor in the manner - * that AT&T implements it. AT&T puts the vtoc structure - * into a sector, usually the second sector (pdsector is first). - * - * Sun incorporates the tag, flag, version, and volume vtoc fields into - * its Disk Label, which already has some vtoc-equivalent fields. - * Upon reading the vtoc with read_vtoc(), the following exceptions - * occur: - * v_bootinfo [all] returned as zero - * v_sanity returned as VTOC_SANE - * if Disk Label was sane - * v_sectorsz returned as 512 - * v_reserved [all] returned as zero - * timestamp [all] returned as zero - * - * See dklabel.h, read_vtoc(), and write_vtoc(). - */ - -#define V_NUMPAR NDKMAP /* The number of partitions */ - /* (from dkio.h) */ - -#define VTOC_SANE 0x600DDEEE /* Indicates a sane VTOC */ -#define V_VERSION 0x01 /* layout version number */ -#define V_EXTVERSION V_VERSION /* extvtoc layout version number */ - -/* - * Partition identification tags - */ -#define V_UNASSIGNED 0x00 /* unassigned partition */ -#define V_BOOT 0x01 /* Boot partition */ -#define V_ROOT 0x02 /* Root filesystem */ -#define V_SWAP 0x03 /* Swap filesystem */ -#define V_USR 0x04 /* Usr filesystem */ -#define V_BACKUP 0x05 /* full disk */ -#define V_STAND 0x06 /* Stand partition */ -#define V_VAR 0x07 /* Var partition */ -#define V_HOME 0x08 /* Home partition */ -#define V_ALTSCTR 0x09 /* Alternate sector partition */ -#define V_CACHE 0x0a /* Cache (cachefs) partition */ -#define V_RESERVED 0x0b /* SMI reserved data */ - -/* - * Partition permission flags - */ -#define V_UNMNT 0x01 /* Unmountable partition */ -#define V_RONLY 0x10 /* Read only */ - -/* - * error codes for reading & writing vtoc - */ -#define VT_ERROR (-2) /* errno supplies specific error */ -#define VT_EIO (-3) /* I/O error accessing vtoc */ -#define VT_EINVAL (-4) /* illegal value in vtoc or request */ -#define VT_ENOTSUP (-5) /* VTOC op. not supported */ -#define VT_ENOSPC (-6) /* requested space not found */ -#define VT_EOVERFLOW (-7) /* VTOC op. data struct limited */ - -struct partition { - ushort_t p_tag; /* ID tag of partition */ - ushort_t p_flag; /* permission flags */ - uint64_t p_start; /* start sector no of partition */ - long p_size; /* # of blocks in partition */ -}; - -struct vtoc { - unsigned long v_bootinfo[3]; /* info needed by mboot (unsupported) */ - unsigned long v_sanity; /* to verify vtoc sanity */ - unsigned long v_version; /* layout version */ - char v_volume[LEN_DKL_VVOL]; /* volume name */ - ushort_t v_sectorsz; /* sector size in bytes */ - ushort_t v_nparts; /* number of partitions */ - unsigned long v_reserved[10]; /* free space */ - struct partition v_part[V_NUMPAR]; /* partition headers */ - time_t timestamp[V_NUMPAR]; /* partition timestamp (unsupported) */ - char v_asciilabel[LEN_DKL_ASCII]; /* for compatibility */ -}; - -struct extpartition { - ushort_t p_tag; /* ID tag of partition */ - ushort_t p_flag; /* permission flags */ - ushort_t p_pad[2]; - diskaddr_t p_start; /* start sector no of partition */ - diskaddr_t p_size; /* # of blocks in partition */ -}; - - -struct extvtoc { - uint64_t v_bootinfo[3]; /* info needed by mboot (unsupported) */ - uint64_t v_sanity; /* to verify vtoc sanity */ - uint64_t v_version; /* layout version */ - char v_volume[LEN_DKL_VVOL]; /* volume name */ - ushort_t v_sectorsz; /* sector size in bytes */ - ushort_t v_nparts; /* number of partitions */ - ushort_t pad[2]; - uint64_t v_reserved[10]; - struct extpartition v_part[V_NUMPAR]; /* partition headers */ - uint64_t timestamp[V_NUMPAR]; /* partition timestamp (unsupported) */ - char v_asciilabel[LEN_DKL_ASCII]; /* for compatibility */ -}; - -#ifdef _KERNEL -#define extvtoctovtoc(extv, v) \ - { \ - int i; \ - v.v_bootinfo[0] = (unsigned long)extv.v_bootinfo[0]; \ - v.v_bootinfo[1] = (unsigned long)extv.v_bootinfo[1]; \ - v.v_bootinfo[2] = (unsigned long)extv.v_bootinfo[2]; \ - v.v_sanity = (unsigned long)extv.v_sanity; \ - v.v_version = (unsigned long)extv.v_version; \ - bcopy(extv.v_volume, v.v_volume, LEN_DKL_VVOL); \ - v.v_sectorsz = extv.v_sectorsz; \ - v.v_nparts = extv.v_nparts; \ - for (i = 0; i < 10; i++) \ - v.v_reserved[i] = (unsigned long)extv.v_reserved[i]; \ - for (i = 0; i < V_NUMPAR; i++) { \ - v.v_part[i].p_tag = extv.v_part[i].p_tag; \ - v.v_part[i].p_flag = extv.v_part[i].p_flag; \ - v.v_part[i].p_start = (uint64_t)extv.v_part[i].p_start; \ - v.v_part[i].p_size = (long)extv.v_part[i].p_size; \ - v.timestamp[i] = (time_t)extv.timestamp[i]; \ - } \ - bcopy(extv.v_asciilabel, v.v_asciilabel, LEN_DKL_ASCII); \ - } - -#define vtoctoextvtoc(v, extv) \ - { \ - int i; \ - extv.v_bootinfo[0] = (uint64_t)v.v_bootinfo[0]; \ - extv.v_bootinfo[1] = (uint64_t)v.v_bootinfo[1]; \ - extv.v_bootinfo[2] = (uint64_t)v.v_bootinfo[2]; \ - extv.v_sanity = (uint64_t)v.v_sanity; \ - extv.v_version = (uint64_t)v.v_version; \ - bcopy(v.v_volume, extv.v_volume, LEN_DKL_VVOL); \ - extv.v_sectorsz = v.v_sectorsz; \ - extv.v_nparts = v.v_nparts; \ - for (i = 0; i < 10; i++) \ - extv.v_reserved[i] = (uint64_t)v.v_reserved[i]; \ - for (i = 0; i < V_NUMPAR; i++) { \ - extv.v_part[i].p_tag = v.v_part[i].p_tag; \ - extv.v_part[i].p_flag = v.v_part[i].p_flag; \ - extv.v_part[i].p_start = \ - (diskaddr_t)(unsigned long)v.v_part[i].p_start; \ - extv.v_part[i].p_size = \ - (diskaddr_t)(unsigned long)v.v_part[i].p_size; \ - extv.timestamp[i] = (uint64_t)v.timestamp[i]; \ - } \ - bcopy(v.v_asciilabel, extv.v_asciilabel, LEN_DKL_ASCII); \ - } -#endif /* _KERNEL */ - -#if defined(_SYSCALL32) -struct partition32 { - uint16_t p_tag; /* ID tag of partition */ - uint16_t p_flag; /* permission flags */ - daddr32_t p_start; /* start sector no of partition */ - int32_t p_size; /* # of blocks in partition */ -}; - -struct vtoc32 { - uint32_t v_bootinfo[3]; /* info needed by mboot (unsupported) */ - uint32_t v_sanity; /* to verify vtoc sanity */ - uint32_t v_version; /* layout version */ - char v_volume[LEN_DKL_VVOL]; /* volume name */ - uint16_t v_sectorsz; /* sector size in bytes */ - uint16_t v_nparts; /* number of partitions */ - uint32_t v_reserved[10]; /* free space */ - struct partition32 v_part[V_NUMPAR]; /* partition headers */ - time32_t timestamp[V_NUMPAR]; /* partition timestamp (unsupported) */ - char v_asciilabel[LEN_DKL_ASCII]; /* for compatibility */ -}; - -#define vtoc32tovtoc(v32, v) \ - { \ - int i; \ - v.v_bootinfo[0] = v32.v_bootinfo[0]; \ - v.v_bootinfo[1] = v32.v_bootinfo[1]; \ - v.v_bootinfo[2] = v32.v_bootinfo[2]; \ - v.v_sanity = v32.v_sanity; \ - v.v_version = v32.v_version; \ - bcopy(v32.v_volume, v.v_volume, LEN_DKL_VVOL); \ - v.v_sectorsz = v32.v_sectorsz; \ - v.v_nparts = v32.v_nparts; \ - v.v_version = v32.v_version; \ - for (i = 0; i < 10; i++) \ - v.v_reserved[i] = v32.v_reserved[i]; \ - for (i = 0; i < V_NUMPAR; i++) { \ - v.v_part[i].p_tag = (ushort_t)v32.v_part[i].p_tag; \ - v.v_part[i].p_flag = (ushort_t)v32.v_part[i].p_flag; \ - v.v_part[i].p_start = (unsigned)v32.v_part[i].p_start; \ - v.v_part[i].p_size = (unsigned)v32.v_part[i].p_size; \ - } \ - for (i = 0; i < V_NUMPAR; i++) \ - v.timestamp[i] = (time_t)v32.timestamp[i]; \ - bcopy(v32.v_asciilabel, v.v_asciilabel, LEN_DKL_ASCII); \ - } - -#define vtoc32toextvtoc(v32, extv) \ - { \ - int i; \ - extv.v_bootinfo[0] = v32.v_bootinfo[0]; \ - extv.v_bootinfo[1] = v32.v_bootinfo[1]; \ - extv.v_bootinfo[2] = v32.v_bootinfo[2]; \ - extv.v_sanity = v32.v_sanity; \ - extv.v_version = v32.v_version; \ - bcopy(v32.v_volume, extv.v_volume, LEN_DKL_VVOL); \ - extv.v_sectorsz = v32.v_sectorsz; \ - extv.v_nparts = v32.v_nparts; \ - extv.v_version = v32.v_version; \ - for (i = 0; i < 10; i++) \ - extv.v_reserved[i] = v32.v_reserved[i]; \ - for (i = 0; i < V_NUMPAR; i++) { \ - extv.v_part[i].p_tag = (ushort_t)v32.v_part[i].p_tag; \ - extv.v_part[i].p_flag = (ushort_t)v32.v_part[i].p_flag; \ - extv.v_part[i].p_start = (diskaddr_t)v32.v_part[i].p_start; \ - extv.v_part[i].p_size = (diskaddr_t)v32.v_part[i].p_size; \ - extv.timestamp[i] = (time_t)v32.timestamp[i]; \ - } \ - bcopy(v32.v_asciilabel, extv.v_asciilabel, LEN_DKL_ASCII); \ - } - - -#define vtoctovtoc32(v, v32) \ - { \ - int i; \ - v32.v_bootinfo[0] = v.v_bootinfo[0]; \ - v32.v_bootinfo[1] = v.v_bootinfo[1]; \ - v32.v_bootinfo[2] = v.v_bootinfo[2]; \ - v32.v_sanity = v.v_sanity; \ - v32.v_version = v.v_version; \ - bcopy(v.v_volume, v32.v_volume, LEN_DKL_VVOL); \ - v32.v_sectorsz = v.v_sectorsz; \ - v32.v_nparts = v.v_nparts; \ - v32.v_version = v.v_version; \ - for (i = 0; i < 10; i++) \ - v32.v_reserved[i] = v.v_reserved[i]; \ - for (i = 0; i < V_NUMPAR; i++) { \ - v32.v_part[i].p_tag = (ushort_t)v.v_part[i].p_tag; \ - v32.v_part[i].p_flag = (ushort_t)v.v_part[i].p_flag; \ - v32.v_part[i].p_start = (unsigned)v.v_part[i].p_start; \ - v32.v_part[i].p_size = (unsigned)v.v_part[i].p_size; \ - } \ - for (i = 0; i < V_NUMPAR; i++) { \ - if (v.timestamp[i] > TIME32_MAX) \ - v32.timestamp[i] = TIME32_MAX; \ - else \ - v32.timestamp[i] = (time32_t)v.timestamp[i]; \ - } \ - bcopy(v.v_asciilabel, v32.v_asciilabel, LEN_DKL_ASCII); \ - } - -#define extvtoctovtoc32(extv, v32) \ - { \ - int i; \ - v32.v_bootinfo[0] = extv.v_bootinfo[0]; \ - v32.v_bootinfo[1] = extv.v_bootinfo[1]; \ - v32.v_bootinfo[2] = extv.v_bootinfo[2]; \ - v32.v_sanity = extv.v_sanity; \ - v32.v_version = extv.v_version; \ - bcopy(extv.v_volume, v32.v_volume, LEN_DKL_VVOL); \ - v32.v_sectorsz = extv.v_sectorsz; \ - v32.v_nparts = extv.v_nparts; \ - v32.v_version = extv.v_version; \ - for (i = 0; i < 10; i++) \ - v32.v_reserved[i] = extv.v_reserved[i]; \ - for (i = 0; i < V_NUMPAR; i++) { \ - v32.v_part[i].p_tag = (ushort_t)extv.v_part[i].p_tag; \ - v32.v_part[i].p_flag = (ushort_t)extv.v_part[i].p_flag; \ - v32.v_part[i].p_start = (unsigned)extv.v_part[i].p_start; \ - v32.v_part[i].p_size = (unsigned)extv.v_part[i].p_size; \ - } \ - for (i = 0; i < V_NUMPAR; i++) { \ - if (extv.timestamp[i] > TIME32_MAX) \ - v32.timestamp[i] = TIME32_MAX; \ - else \ - v32.timestamp[i] = (time32_t)extv.timestamp[i]; \ - } \ - bcopy(extv.v_asciilabel, v32.v_asciilabel, LEN_DKL_ASCII); \ - } - - -#endif /* _SYSCALL32 */ - -/* - * These defines are the mode parameter for the checksum routines. - */ -#define CK_CHECKSUM 0 /* check checksum */ -#define CK_MAKESUM 1 /* generate checksum */ - -#if defined(__STDC__) - -extern int read_vtoc(int, struct vtoc *); -extern int write_vtoc(int, struct vtoc *); -extern int read_extvtoc(int, struct extvtoc *); -extern int write_extvtoc(int, struct extvtoc *); - -#else - -extern int read_vtoc(); -extern int write_vtoc(); -extern int read_extvtoc(); -extern int write_extvtoc(); - -#endif /* __STDC__ */ - -#ifdef __cplusplus -} -#endif - -#endif /* _SYS_VTOC_H */ diff --git a/lib/libspl/include/sys/zone.h b/lib/libspl/include/sys/zone.h index bbb964dcef22..00d740fb6fff 100644 --- a/lib/libspl/include/sys/zone.h +++ b/lib/libspl/include/sys/zone.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/thread.h b/lib/libspl/include/thread.h deleted file mode 100644 index 74694e23eed5..000000000000 --- a/lib/libspl/include/thread.h +++ /dev/null @@ -1,30 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License, Version 1.0 only - * (the "License"). You may not use this file except in compliance - * with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2009 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#ifndef _LIBSPL_THREAD_H -#define _LIBSPL_THREAD_H - -#endif /* _LIBSPL_THREAD_H */ diff --git a/lib/libspl/include/tzfile.h b/lib/libspl/include/tzfile.h deleted file mode 100644 index 7bd4087cd5d1..000000000000 --- a/lib/libspl/include/tzfile.h +++ /dev/null @@ -1,32 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License, Version 1.0 only - * (the "License"). You may not use this file except in compliance - * with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2007 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#ifndef _LIBSPL_TZFILE_H -#define _LIBSPL_TZFILE_H - -#include - -#endif /* _LIBSPL_TZFILE_H */ diff --git a/lib/libspl/include/ucred.h b/lib/libspl/include/ucred.h deleted file mode 100644 index 8178fdec4c74..000000000000 --- a/lib/libspl/include/ucred.h +++ /dev/null @@ -1,32 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License, Version 1.0 only - * (the "License"). You may not use this file except in compliance - * with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2007 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#ifndef _LIBSPL_UCRED_H -#define _LIBSPL_UCRED_H - -typedef int ucred_t; - -#endif diff --git a/lib/libspl/include/umem.h b/lib/libspl/include/umem.h index eee0dc97578a..2a366e29472d 100644 --- a/lib/libspl/include/umem.h +++ b/lib/libspl/include/umem.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -140,7 +140,7 @@ umem_nofail_callback(umem_nofail_callback_t *cb __maybe_unused) static inline umem_cache_t * umem_cache_create( - char *name, size_t bufsize, size_t align, + const char *name, size_t bufsize, size_t align, umem_constructor_t *constructor, umem_destructor_t *destructor, umem_reclaim_t *reclaim, diff --git a/lib/libspl/include/unistd.h b/lib/libspl/include/unistd.h index 0246991b4b61..edb610e87848 100644 --- a/lib/libspl/include/unistd.h +++ b/lib/libspl/include/unistd.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/include/util/Makefile.am b/lib/libspl/include/util/Makefile.am deleted file mode 100644 index ab553bc80313..000000000000 --- a/lib/libspl/include/util/Makefile.am +++ /dev/null @@ -1,3 +0,0 @@ -libspldir = $(includedir)/libspl -libspl_HEADERS = \ - sscanf.h diff --git a/lib/libspl/include/util/sscanf.h b/lib/libspl/include/util/sscanf.h deleted file mode 100644 index ead36acaba3e..000000000000 --- a/lib/libspl/include/util/sscanf.h +++ /dev/null @@ -1,30 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License, Version 1.0 only - * (the "License"). You may not use this file except in compliance - * with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2010 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#ifndef _LIBSPL_UTIL_SSCANF_H -#define _LIBSPL_UTIL_SSCANF_H - -#endif diff --git a/lib/libspl/include/zone.h b/lib/libspl/include/zone.h index b0ac2d9bc610..341d574a7460 100644 --- a/lib/libspl/include/zone.h +++ b/lib/libspl/include/zone.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -33,7 +33,17 @@ extern "C" { #endif -#define GLOBAL_ZONEID 0 +#ifdef __FreeBSD__ +#define GLOBAL_ZONEID 0 +#else +/* + * Hardcoded in the kernel's root user namespace. A "better" way to get + * this would be by using ioctl_ns(2), but this would need to be performed + * recursively on NS_GET_PARENT and then NS_GET_USERNS. Also, that's only + * supported since Linux 4.9. + */ +#define GLOBAL_ZONEID 4026531837U +#endif extern zoneid_t getzoneid(void); diff --git a/lib/libspl/libspl_impl.h b/lib/libspl/libspl_impl.h index cda56e64c962..3a19df64440c 100644 --- a/lib/libspl/libspl_impl.h +++ b/lib/libspl/libspl_impl.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/list.c b/lib/libspl/list.c index 0f2f3731b235..24403698627c 100644 --- a/lib/libspl/list.c +++ b/lib/libspl/list.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/mkdirp.c b/lib/libspl/mkdirp.c index fce2c1c82eb7..dff95a192af7 100644 --- a/lib/libspl/mkdirp.c +++ b/lib/libspl/mkdirp.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/os/freebsd/getexecname.c b/lib/libspl/os/freebsd/getexecname.c index 256b28c1b70e..5f1692adf1dd 100644 --- a/lib/libspl/os/freebsd/getexecname.c +++ b/lib/libspl/os/freebsd/getexecname.c @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/os/freebsd/gethostid.c b/lib/libspl/os/freebsd/gethostid.c index 7bd567fe61b5..4e095d670710 100644 --- a/lib/libspl/os/freebsd/gethostid.c +++ b/lib/libspl/os/freebsd/gethostid.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/os/freebsd/getmntany.c b/lib/libspl/os/freebsd/getmntany.c index 0ef24059e84f..fc875fcb133f 100644 --- a/lib/libspl/os/freebsd/getmntany.c +++ b/lib/libspl/os/freebsd/getmntany.c @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/os/freebsd/mnttab.c b/lib/libspl/os/freebsd/mnttab.c index d830257fbd16..f0cc04d89ded 100644 --- a/lib/libspl/os/freebsd/mnttab.c +++ b/lib/libspl/os/freebsd/mnttab.c @@ -39,6 +39,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -66,7 +67,7 @@ mntopt(char **p) } char * -hasmntopt(struct mnttab *mnt, char *opt) +hasmntopt(struct mnttab *mnt, const char *opt) { char tmpopts[MNT_LINE_MAX]; char *f, *opts = tmpopts; @@ -136,6 +137,7 @@ statfs2mnttab(struct statfs *sfs, struct mnttab *mp) mp->mnt_mntopts = gmntopts; } +static pthread_rwlock_t gsfs_lock = PTHREAD_RWLOCK_INITIALIZER; static struct statfs *gsfs = NULL; static int allfs = 0; @@ -145,6 +147,8 @@ statfs_init(void) struct statfs *sfs; int error; + (void) pthread_rwlock_wrlock(&gsfs_lock); + if (gsfs != NULL) { free(gsfs); gsfs = NULL; @@ -162,6 +166,7 @@ statfs_init(void) sfs = realloc(gsfs, allfs * sizeof (gsfs[0])); if (sfs != NULL) gsfs = sfs; + (void) pthread_rwlock_unlock(&gsfs_lock); return (0); fail: error = errno; @@ -169,6 +174,7 @@ statfs_init(void) free(gsfs); gsfs = NULL; allfs = 0; + (void) pthread_rwlock_unlock(&gsfs_lock); return (error); } @@ -181,6 +187,8 @@ getmntany(FILE *fd __unused, struct mnttab *mgetp, struct mnttab *mrefp) if (error != 0) return (error); + (void) pthread_rwlock_rdlock(&gsfs_lock); + for (i = 0; i < allfs; i++) { if (mrefp->mnt_special != NULL && strcmp(mrefp->mnt_special, gsfs[i].f_mntfromname) != 0) { @@ -195,8 +203,10 @@ getmntany(FILE *fd __unused, struct mnttab *mgetp, struct mnttab *mrefp) continue; } statfs2mnttab(&gsfs[i], mgetp); + (void) pthread_rwlock_unlock(&gsfs_lock); return (0); } + (void) pthread_rwlock_unlock(&gsfs_lock); return (-1); } @@ -214,9 +224,13 @@ getmntent(FILE *fp, struct mnttab *mp) if (error != 0) return (error); } - if (nfs >= allfs) + (void) pthread_rwlock_rdlock(&gsfs_lock); + if (nfs >= allfs) { + (void) pthread_rwlock_unlock(&gsfs_lock); return (-1); + } statfs2mnttab(&gsfs[nfs], mp); + (void) pthread_rwlock_unlock(&gsfs_lock); if (lseek(fileno(fp), 1, SEEK_CUR) == -1) return (errno); return (0); diff --git a/lib/libspl/os/linux/getexecname.c b/lib/libspl/os/linux/getexecname.c index a640556bcbec..4b54c193973a 100644 --- a/lib/libspl/os/linux/getexecname.c +++ b/lib/libspl/os/linux/getexecname.c @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/os/linux/gethostid.c b/lib/libspl/os/linux/gethostid.c index c04b7fd3eef3..4f0f73c89b5d 100644 --- a/lib/libspl/os/linux/gethostid.c +++ b/lib/libspl/os/linux/gethostid.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/os/linux/getmntany.c b/lib/libspl/os/linux/getmntany.c index 3713ff38e17f..7e3722cd373e 100644 --- a/lib/libspl/os/linux/getmntany.c +++ b/lib/libspl/os/linux/getmntany.c @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/os/linux/zone.c b/lib/libspl/os/linux/zone.c index 393a16ad5cdd..e37b34975df6 100644 --- a/lib/libspl/os/linux/zone.c +++ b/lib/libspl/os/linux/zone.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -23,10 +23,40 @@ * Use is subject to license terms. */ +#include +#include +#include +#include +#include +#include + #include zoneid_t getzoneid(void) { - return (GLOBAL_ZONEID); + char path[PATH_MAX]; + char buf[128] = { '\0' }; + char *cp; + + int c = snprintf(path, sizeof (path), "/proc/self/ns/user"); + /* This API doesn't have any error checking... */ + if (c < 0) + return (0); + + ssize_t r = readlink(path, buf, sizeof (buf) - 1); + if (r < 0) + return (0); + + cp = strchr(buf, '['); + if (cp == NULL) + return (0); + cp++; + + unsigned long n = strtoul(cp, NULL, 10); + if (n == ULONG_MAX && errno == ERANGE) + return (0); + zoneid_t z = (zoneid_t)n; + + return (z); } diff --git a/lib/libspl/page.c b/lib/libspl/page.c index 5b0d3f2e5786..fb8827867781 100644 --- a/lib/libspl/page.c +++ b/lib/libspl/page.c @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/strlcat.c b/lib/libspl/strlcat.c index 4528d875ed5e..88e244534ca1 100644 --- a/lib/libspl/strlcat.c +++ b/lib/libspl/strlcat.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/strlcpy.c b/lib/libspl/strlcpy.c index d483b91f6121..c3f59bd84b0d 100644 --- a/lib/libspl/strlcpy.c +++ b/lib/libspl/strlcpy.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libspl/timestamp.c b/lib/libspl/timestamp.c index 22ecb3940739..9b435221f5fb 100644 --- a/lib/libspl/timestamp.c +++ b/lib/libspl/timestamp.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -44,7 +44,7 @@ void print_timestamp(uint_t timestamp_fmt) { time_t t = time(NULL); - static char *fmt = NULL; + static const char *fmt = NULL; /* We only need to retrieve this once per invocation */ if (fmt == NULL) @@ -54,9 +54,10 @@ print_timestamp(uint_t timestamp_fmt) (void) printf("%lld\n", (longlong_t)t); } else if (timestamp_fmt == DDATE) { char dstr[64]; + struct tm tm; int len; - len = strftime(dstr, sizeof (dstr), fmt, localtime(&t)); + len = strftime(dstr, sizeof (dstr), fmt, localtime_r(&t, &tm)); if (len > 0) (void) printf("%s\n", dstr); } diff --git a/lib/libtpool/Makefile.am b/lib/libtpool/Makefile.am index 40fd137b4335..5a2b8a5701da 100644 --- a/lib/libtpool/Makefile.am +++ b/lib/libtpool/Makefile.am @@ -1,13 +1,11 @@ -include $(top_srcdir)/config/Rules.am +libtpool_la_CFLAGS = $(AM_CFLAGS) $(LIBRARY_CFLAGS) +libtpool_la_CFLAGS += -fvisibility=hidden +# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61118 +libtpool_la_CFLAGS += $(NO_CLOBBERED) -AM_CFLAGS += -fvisibility=hidden +noinst_LTLIBRARIES += libtpool.la +CPPCHECKTARGETS += libtpool.la -noinst_LTLIBRARIES = libtpool.la - -USER_C = \ - thread_pool.c \ - thread_pool_impl.h - -libtpool_la_SOURCES = $(USER_C) - -include $(top_srcdir)/config/CppCheck.am +libtpool_la_SOURCES = \ + %D%/thread_pool.c \ + %D%/thread_pool_impl.h diff --git a/lib/libtpool/thread_pool.c b/lib/libtpool/thread_pool.c index 58b706ddebf5..3eef3d1e4819 100644 --- a/lib/libtpool/thread_pool.c +++ b/lib/libtpool/thread_pool.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libtpool/thread_pool_impl.h b/lib/libtpool/thread_pool_impl.h index 5349c2ade8c3..6cde8d0d235c 100644 --- a/lib/libtpool/thread_pool_impl.h +++ b/lib/libtpool/thread_pool_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libunicode/Makefile.am b/lib/libunicode/Makefile.am index b82975f68efd..906759471163 100644 --- a/lib/libunicode/Makefile.am +++ b/lib/libunicode/Makefile.am @@ -1,17 +1,7 @@ -include $(top_srcdir)/config/Rules.am +libunicode_la_CFLAGS = $(AM_CFLAGS) $(KERNEL_CFLAGS) $(LIBRARY_CFLAGS) -VPATH = $(top_srcdir)/module/unicode - -# Includes kernel code, generate warnings for large stack frames -AM_CFLAGS += $(FRAME_LARGER_THAN) - -noinst_LTLIBRARIES = libunicode.la - -KERNEL_C = \ - u8_textprep.c \ - uconv.c +noinst_LTLIBRARIES += libunicode.la nodist_libunicode_la_SOURCES = \ - $(KERNEL_C) - -include $(top_srcdir)/config/CppCheck.am + module/unicode/u8_textprep.c \ + module/unicode/uconv.c diff --git a/lib/libuutil/Makefile.am b/lib/libuutil/Makefile.am index 5a911f85f7de..339f9a064745 100644 --- a/lib/libuutil/Makefile.am +++ b/lib/libuutil/Makefile.am @@ -1,23 +1,20 @@ -include $(top_srcdir)/config/Rules.am +libuutil_la_CFLAGS = $(AM_CFLAGS) $(LIBRARY_CFLAGS) $(LIBRARY_CFLAGS) -lib_LTLIBRARIES = libuutil.la +lib_LTLIBRARIES += libuutil.la +CPPCHECKTARGETS += libuutil.la -include $(top_srcdir)/config/Abigail.am - -USER_C = \ - uu_alloc.c \ - uu_avl.c \ - uu_ident.c \ - uu_list.c \ - uu_misc.c \ - uu_pname.c \ - uu_string.c - -libuutil_la_SOURCES = $(USER_C) +libuutil_la_SOURCES = \ + %D%/uu_alloc.c \ + %D%/uu_avl.c \ + %D%/uu_ident.c \ + %D%/uu_list.c \ + %D%/uu_misc.c \ + %D%/uu_pname.c \ + %D%/uu_string.c libuutil_la_LIBADD = \ - $(abs_top_builddir)/lib/libavl/libavl.la \ - $(abs_top_builddir)/lib/libspl/libspl.la + libavl.la \ + libspl.la libuutil_la_LIBADD += $(LTLIBINTL) @@ -29,7 +26,4 @@ endif libuutil_la_LDFLAGS += -version-info 3:0:0 -include $(top_srcdir)/config/CppCheck.am - -# Library ABI -EXTRA_DIST = libuutil.abi libuutil.suppr +dist_noinst_DATA += %D%/libuutil.abi %D%/libuutil.suppr diff --git a/lib/libuutil/libuutil.abi b/lib/libuutil/libuutil.abi index bf13d62e2f04..766d8843000d 100644 --- a/lib/libuutil/libuutil.abi +++ b/lib/libuutil/libuutil.abi @@ -5,8 +5,6 @@ - - @@ -366,7 +364,7 @@ - + @@ -430,6 +428,11 @@ + + + + + @@ -450,7 +453,7 @@ - + @@ -475,11 +478,6 @@ - - - - - @@ -552,6 +550,11 @@ + + + + + @@ -572,7 +575,7 @@ - + @@ -597,11 +600,6 @@ - - - - - @@ -642,6 +640,12 @@ + + + + + + @@ -666,12 +670,6 @@ - - - - - - @@ -858,6 +856,9 @@ + + + @@ -892,18 +893,66 @@ - - + - + - + - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -972,16 +1021,16 @@ - + - + - + - + @@ -993,65 +1042,6 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - @@ -1062,11 +1052,16 @@ + + + + + @@ -1086,7 +1081,7 @@ - + @@ -1122,6 +1117,7 @@ + @@ -1154,7 +1150,6 @@ - @@ -1307,15 +1302,6 @@ - - - - - - - - - @@ -1342,6 +1328,15 @@ + + + + + + + + + diff --git a/lib/libuutil/uu_alloc.c b/lib/libuutil/uu_alloc.c index 2bef759d525e..2850bf6d1b35 100644 --- a/lib/libuutil/uu_alloc.c +++ b/lib/libuutil/uu_alloc.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libuutil/uu_avl.c b/lib/libuutil/uu_avl.c index 53def0e073fd..12f7bd4bcc72 100644 --- a/lib/libuutil/uu_avl.c +++ b/lib/libuutil/uu_avl.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libuutil/uu_ident.c b/lib/libuutil/uu_ident.c index 382139316ebc..d0cab0a64d09 100644 --- a/lib/libuutil/uu_ident.c +++ b/lib/libuutil/uu_ident.c @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libuutil/uu_list.c b/lib/libuutil/uu_list.c index c3a447d01de3..5ece4b14701e 100644 --- a/lib/libuutil/uu_list.c +++ b/lib/libuutil/uu_list.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libuutil/uu_misc.c b/lib/libuutil/uu_misc.c index a8478ace9a73..762d00c3cd32 100644 --- a/lib/libuutil/uu_misc.c +++ b/lib/libuutil/uu_misc.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libuutil/uu_pname.c b/lib/libuutil/uu_pname.c index 43c9e77564ee..37c093731ef9 100644 --- a/lib/libuutil/uu_pname.c +++ b/lib/libuutil/uu_pname.c @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -40,8 +40,8 @@ static const char *pname; -static void -uu_die_internal(int status, const char *format, va_list alist) __NORETURN; +static __attribute__((noreturn)) void +uu_die_internal(int status, const char *format, va_list alist); int uu_exit_ok_value = EXIT_SUCCESS; int uu_exit_fatal_value = EXIT_FAILURE; @@ -110,7 +110,7 @@ uu_warn(const char *format, ...) va_end(alist); } -static __attribute__((format(printf, 2, 0))) __NORETURN void +static __attribute__((format(printf, 2, 0))) __attribute__((noreturn)) void uu_die_internal(int status, const char *format, va_list alist) { uu_warn_internal(errno, format, alist); diff --git a/lib/libuutil/uu_string.c b/lib/libuutil/uu_string.c index 67024c3b50b4..eff0727b7564 100644 --- a/lib/libuutil/uu_string.c +++ b/lib/libuutil/uu_string.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libzfs/Makefile.am b/lib/libzfs/Makefile.am index e23f7c162afb..f5eb84679204 100644 --- a/lib/libzfs/Makefile.am +++ b/lib/libzfs/Makefile.am @@ -1,81 +1,64 @@ -include $(top_srcdir)/config/Rules.am +libzfs_la_CFLAGS = $(AM_CFLAGS) $(LIBRARY_CFLAGS) +libzfs_la_CFLAGS += $(LIBCRYPTO_CFLAGS) $(ZLIB_CFLAGS) +libzfs_la_CFLAGS += -fvisibility=hidden -VPATH = \ - $(top_srcdir)/module/icp \ - $(top_srcdir)/module/zcommon \ - $(top_srcdir)/lib/libzfs - -# Suppress unused but set variable warnings often due to ASSERTs -AM_CFLAGS += $(NO_UNUSED_BUT_SET_VARIABLE) -AM_CFLAGS += $(LIBCRYPTO_CFLAGS) $(ZLIB_CFLAGS) -AM_CFLAGS += -fvisibility=hidden - -pkgconfig_DATA = libzfs.pc - -lib_LTLIBRARIES = libzfs.la - -include $(top_srcdir)/config/Abigail.am - -USER_C = \ - libzfs_impl.h \ - libzfs_changelist.c \ - libzfs_config.c \ - libzfs_crypto.c \ - libzfs_dataset.c \ - libzfs_diff.c \ - libzfs_import.c \ - libzfs_iter.c \ - libzfs_mount.c \ - libzfs_pool.c \ - libzfs_sendrecv.c \ - libzfs_status.c \ - libzfs_util.c +lib_LTLIBRARIES += libzfs.la +CPPCHECKTARGETS += libzfs.la +dist_libzfs_la_SOURCES = \ + %D%/libzfs_impl.h \ + %D%/libzfs_changelist.c \ + %D%/libzfs_config.c \ + %D%/libzfs_crypto.c \ + %D%/libzfs_dataset.c \ + %D%/libzfs_diff.c \ + %D%/libzfs_import.c \ + %D%/libzfs_iter.c \ + %D%/libzfs_mount.c \ + %D%/libzfs_pool.c \ + %D%/libzfs_sendrecv.c \ + %D%/libzfs_status.c \ + %D%/libzfs_util.c if BUILD_FREEBSD -USER_C += \ - os/freebsd/libzfs_compat.c \ - os/freebsd/libzfs_zmount.c +dist_libzfs_la_SOURCES += \ + %D%/os/freebsd/libzfs_compat.c \ + %D%/os/freebsd/libzfs_zmount.c endif if BUILD_LINUX -USER_C += \ - os/linux/libzfs_mount_os.c \ - os/linux/libzfs_pool_os.c \ - os/linux/libzfs_sendrecv_os.c \ - os/linux/libzfs_util_os.c +dist_libzfs_la_SOURCES += \ + %D%/os/linux/libzfs_mount_os.c \ + %D%/os/linux/libzfs_pool_os.c \ + %D%/os/linux/libzfs_util_os.c endif -KERNEL_C = \ - algs/sha2/sha2.c \ - cityhash.c \ - zfeature_common.c \ - zfs_comutil.c \ - zfs_deleg.c \ - zfs_fletcher.c \ - zfs_fletcher_aarch64_neon.c \ - zfs_fletcher_avx512.c \ - zfs_fletcher_intel.c \ - zfs_fletcher_sse.c \ - zfs_fletcher_superscalar.c \ - zfs_fletcher_superscalar4.c \ - zfs_namecheck.c \ - zfs_prop.c \ - zpool_prop.c \ - zprop_common.c - -dist_libzfs_la_SOURCES = \ - $(USER_C) - nodist_libzfs_la_SOURCES = \ - $(KERNEL_C) + module/icp/algs/sha2/sha2.c \ + \ + module/zcommon/cityhash.c \ + module/zcommon/zfeature_common.c \ + module/zcommon/zfs_comutil.c \ + module/zcommon/zfs_deleg.c \ + module/zcommon/zfs_fletcher.c \ + module/zcommon/zfs_fletcher_aarch64_neon.c \ + module/zcommon/zfs_fletcher_avx512.c \ + module/zcommon/zfs_fletcher_intel.c \ + module/zcommon/zfs_fletcher_sse.c \ + module/zcommon/zfs_fletcher_superscalar.c \ + module/zcommon/zfs_fletcher_superscalar4.c \ + module/zcommon/zfs_namecheck.c \ + module/zcommon/zfs_prop.c \ + module/zcommon/zpool_prop.c \ + module/zcommon/zprop_common.c + libzfs_la_LIBADD = \ - $(abs_top_builddir)/lib/libshare/libshare.la \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la \ - $(abs_top_builddir)/lib/libzutil/libzutil.la \ - $(abs_top_builddir)/lib/libuutil/libuutil.la + libshare.la \ + libzfs_core.la \ + libnvpair.la \ + libzutil.la \ + libuutil.la libzfs_la_LIBADD += -lm $(LIBCRYPTO_LIBS) $(ZLIB_LIBS) $(LIBFETCH_LIBS) $(LTLIBINTL) @@ -91,10 +74,7 @@ endif libzfs_la_LDFLAGS += -version-info 5:0:1 -include $(top_srcdir)/config/CppCheck.am - -# Library ABI -EXTRA_DIST = libzfs.abi libzfs.suppr +pkgconfig_DATA += %D%/libzfs.pc -# Licensing data -EXTRA_DIST += THIRDPARTYLICENSE.openssl THIRDPARTYLICENSE.openssl.descrip +dist_noinst_DATA += %D%/libzfs.abi %D%/libzfs.suppr +dist_noinst_DATA += %D%/THIRDPARTYLICENSE.openssl %D%/THIRDPARTYLICENSE.openssl.descrip diff --git a/lib/libzfs/libzfs.abi b/lib/libzfs/libzfs.abi index 8f586804c9dd..fb5e01b82c40 100644 --- a/lib/libzfs/libzfs.abi +++ b/lib/libzfs/libzfs.abi @@ -10,6 +10,7 @@ + @@ -287,10 +288,7 @@ - - - @@ -338,8 +336,6 @@ - - @@ -418,9 +414,6 @@ - - - @@ -439,13 +432,8 @@ - - - - - - + @@ -595,117 +583,15 @@ - + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + @@ -847,67 +733,113 @@ - + + + + + - + - + - + - + - - - - - - + + + + + + + + + + + + - - - - - + + + + + + + + + + + + + + - + - - + + - - + + - - + + - - + + - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + + + + - + @@ -919,7 +851,7 @@ - + @@ -970,6 +902,11 @@ + + + + + @@ -990,7 +927,7 @@ - + @@ -1015,11 +952,6 @@ - - - - - @@ -1092,6 +1024,11 @@ + + + + + @@ -1112,7 +1049,7 @@ - + @@ -1137,11 +1074,6 @@ - - - - - @@ -1182,6 +1114,12 @@ + + + + + + @@ -1206,12 +1144,6 @@ - - - - - - @@ -1257,12 +1189,12 @@ - + - + @@ -1373,7 +1305,7 @@ - + @@ -1381,12 +1313,12 @@ - + - + @@ -1605,18 +1537,18 @@ - - + + - + - + @@ -1624,7 +1556,7 @@ - + @@ -1632,13 +1564,13 @@ - + - + @@ -1650,6 +1582,75 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -1724,75 +1725,6 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - @@ -1844,1926 +1776,876 @@ - - - - - - - - + + - - - + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + + + + + + + + + - + - + - + - - - - - - - - - - - - - - - - - - - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + - + - + - + - + + + + + + + + + + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + - - - - - - - - - - - + + + + + + + - - + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + - - + + - + + + + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + - - + + - - + + - - + + + + + + + - + - + - + - + - - - - - - - - + + - - + + - - + + - - - - - + + - - - - - + + - - - - - + + - - - - - + + - - - - - + + - - + + + + + + + + + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + - + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + - + - + - - + + + + + + + - + - - + + - - + + - + - + - - - - + - + - - + + - - + + - - + + - - + + - - + + - - + + + + + + + + + + + + + + + + + + + + - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - + + + - - - + + + + - - + + - - - + + + + + - - - - + + + + - - - - + + + + + + + + + + + + + + + + + + + + + + + + - - - - + + + + + + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + - - - - + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + - - - - + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + - + - + - + - + - - - - - - - - - - - - - - - - - - - - - + - - + + - - + + - - - - - - - - - - - - - - - - - - - - - - - + + - - - - - - - + - - + + - - + + - - + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + - - - + + + - - + + + + + + + + + + + + + + - + - - - + + + - + + + + + - - - + - - - - - - - - + + + + + + + + + + + + + - - - - - - - + + + + + + + + + + + + + + - - + - + - - - - - - + + + + + + + + + + + + + - - - - + + - - - + + + + - + - - + + + + + + + + + + - + + + + + + + + + + + + - + - - + + + + + + + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + - - - + + + + + + - - - + + + + + - + - + + + + + - - - + + + - - - + + + - - - - - + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + @@ -3958,7 +2840,7 @@ - + @@ -3968,7 +2850,7 @@ - + @@ -4022,7 +2904,7 @@ - + @@ -4079,10 +2961,7 @@ - - - - + @@ -4095,24 +2974,16 @@ - - - - - - - - - - - - - - - + + + + + + + + - @@ -4148,80 +3019,30 @@ - - - - + - - - - - - - - - - + + - - - - - - - - - - + - - - - - - - - - - - - - - + - - - - - - - - + - - - - - - - - - - - + @@ -4250,7 +3071,7 @@ - + @@ -4709,1141 +3530,2306 @@ - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - - + + + + + - - + - - + - - - - + + + - - - + + + - - - - + + + - - - - + + + - - - - - + + - - - - + + + - - - + + + + + + + + + - - - - - - + + + - - - + + + + - - - - - + + + + - - - - - - + + + - - - - + + + - - - - - + + + + + + + - - - - - - - + + + - - - - - + + + - - - - - - - - - - + + + + - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - + + + + + + + + + + + + - - - - - - + + + + + + + + + + + + + - - - - + + - - - - + + + - - - - - + + - - - - + + + + + + + + - - - - - + + + + + - - - - - - + + + + + - - + + + + + - - - - + - - - - + - - - - + - - - - + - - - - + - - - - + - - - - + - - - - + - - + + + + + - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - - + - - - - + - - - - + - - - - + - + - + - - - - + - + - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + - - - - + + + - - - - - + + + - - - - - + + + - - - - - - - - - - + + + + - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - - - - - + + - - - + + - + - - + + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + - - + + - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - - - - - + + + + + + + + + + + + + + + - - - - - + + + + + + + + + + - - - + + + - - - + + - + + + + + + + + + + + + + + + + + + + - - - - - - + + - - + + - + - + - + + + + - - + + + + + - + - - + + - - + + - - + + + + + - - + + + + + - - + + + + + - - + + + + + - - + + + + + - - + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + - - - - - - - - - - - + + + + - - + + + + + - - - - - - + + + + - - - + + + + + - + + - - + + + + + - + + + + - - - - + + + + + + + + + + + - - + + + + + + + + + + + + + + + + + + + - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - - - - + + + + + - + + + + - - - - + + + - - - + + + + + - - - - - - + + + + - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + - + + + + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - - - - - + - + + - + - + + + + - + - + - + - - + + - - + + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + - - + + - - - - - + + - - + + - - + + - - + + - - - - - - - + + + + + + + + - - - + + - - - - + + + - - - + + + + + + + + + + + + + + + + + + + + - - - - - + + + + - - - + + + + - - - + + + + + - - - + + + - - + + - - + + - - - + + + - - + + + + + + + + + + + + + + - - - - - - - - - - + + + - - - + + + + - - - - + + + + + + - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - + + + + - - - + + + + - - - - - - + + - - - + + + - + + + + + + + + + + + + + + + - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - - - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + - - - - - + + + - - - - - - - + + + - - - - - - - - - - - - + + - - - - - + + + + + + + + + + + + + + + + + - - - - + + + + + + + + + - - - - + + + + + + + + + + - - - - + + + + + + + + + + - - - - + + + + + + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/lib/libzfs/libzfs_changelist.c b/lib/libzfs/libzfs_changelist.c index f2282ee01177..e5e735d38e00 100644 --- a/lib/libzfs/libzfs_changelist.c +++ b/lib/libzfs/libzfs_changelist.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -98,6 +98,7 @@ changelist_prefix(prop_changelist_t *clp) prop_changenode_t *cn; uu_avl_walk_t *walk; int ret = 0; + const enum sa_protocol smb[] = {SA_PROTOCOL_SMB, SA_NO_PROTOCOL}; boolean_t commit_smb_shares = B_FALSE; if (clp->cl_prop != ZFS_PROP_MOUNTPOINT && @@ -137,7 +138,8 @@ changelist_prefix(prop_changelist_t *clp) } break; case ZFS_PROP_SHARESMB: - (void) zfs_unshare_smb(cn->cn_handle, NULL); + (void) zfs_unshare(cn->cn_handle, NULL, + smb); commit_smb_shares = B_TRUE; break; @@ -148,7 +150,7 @@ changelist_prefix(prop_changelist_t *clp) } if (commit_smb_shares) - zfs_commit_smb_shares(); + zfs_commit_shares(smb); uu_avl_walk_end(walk); if (ret == -1) @@ -257,25 +259,33 @@ changelist_postfix(prop_changelist_t *clp) * if the filesystem is currently shared, so that we can * adopt any new options. */ + const enum sa_protocol nfs[] = + {SA_PROTOCOL_NFS, SA_NO_PROTOCOL}; if (sharenfs && mounted) { - errors += zfs_share_nfs(cn->cn_handle); + errors += zfs_share(cn->cn_handle, nfs); commit_nfs_shares = B_TRUE; } else if (cn->cn_shared || clp->cl_waslegacy) { - errors += zfs_unshare_nfs(cn->cn_handle, NULL); + errors += zfs_unshare(cn->cn_handle, NULL, nfs); commit_nfs_shares = B_TRUE; } + const enum sa_protocol smb[] = + {SA_PROTOCOL_SMB, SA_NO_PROTOCOL}; if (sharesmb && mounted) { - errors += zfs_share_smb(cn->cn_handle); + errors += zfs_share(cn->cn_handle, smb); commit_smb_shares = B_TRUE; } else if (cn->cn_shared || clp->cl_waslegacy) { - errors += zfs_unshare_smb(cn->cn_handle, NULL); + errors += zfs_unshare(cn->cn_handle, NULL, smb); commit_smb_shares = B_TRUE; } } + + enum sa_protocol proto[SA_PROTOCOL_COUNT + 1], *p = proto; if (commit_nfs_shares) - zfs_commit_nfs_shares(); + *p++ = SA_PROTOCOL_NFS; if (commit_smb_shares) - zfs_commit_smb_shares(); + *p++ = SA_PROTOCOL_SMB; + *p++ = SA_NO_PROTOCOL; + zfs_commit_shares(proto); uu_avl_walk_end(walk); return (errors ? -1 : 0); @@ -345,7 +355,7 @@ changelist_rename(prop_changelist_t *clp, const char *src, const char *dst) * unshare all the datasets in the list. */ int -changelist_unshare(prop_changelist_t *clp, const zfs_share_proto_t *proto) +changelist_unshare(prop_changelist_t *clp, const enum sa_protocol *proto) { prop_changenode_t *cn; uu_avl_walk_t *walk; @@ -359,11 +369,12 @@ changelist_unshare(prop_changelist_t *clp, const zfs_share_proto_t *proto) return (-1); while ((cn = uu_avl_walk_next(walk)) != NULL) { - if (zfs_unshare_proto(cn->cn_handle, NULL, proto) != 0) + if (zfs_unshare(cn->cn_handle, NULL, proto) != 0) ret = -1; } - zfs_commit_proto(proto); + for (const enum sa_protocol *p = proto; *p != SA_NO_PROTOCOL; ++p) + sa_commit_shares(*p); uu_avl_walk_end(walk); return (ret); @@ -447,16 +458,11 @@ changelist_add_mounted(zfs_handle_t *zhp, void *data) ASSERT3U(clp->cl_prop, ==, ZFS_PROP_MOUNTPOINT); - if ((cn = zfs_alloc(zfs_get_handle(zhp), - sizeof (prop_changenode_t))) == NULL) { - zfs_close(zhp); - return (ENOMEM); - } - + cn = zfs_alloc(zfs_get_handle(zhp), sizeof (prop_changenode_t)); cn->cn_handle = zhp; cn->cn_mounted = zfs_is_mounted(zhp, NULL); ASSERT3U(cn->cn_mounted, ==, B_TRUE); - cn->cn_shared = zfs_is_shared(zhp); + cn->cn_shared = zfs_is_shared(zhp, NULL, NULL); cn->cn_zoned = zfs_prop_get_int(zhp, ZFS_PROP_ZONED); cn->cn_needpost = B_TRUE; @@ -522,16 +528,11 @@ change_one(zfs_handle_t *zhp, void *data) (clp->cl_shareprop != ZPROP_INVAL && (share_sourcetype == ZPROP_SRC_DEFAULT || share_sourcetype == ZPROP_SRC_INHERITED))) { - if ((cn = zfs_alloc(zfs_get_handle(zhp), - sizeof (prop_changenode_t))) == NULL) { - ret = -1; - goto out; - } - + cn = zfs_alloc(zfs_get_handle(zhp), sizeof (prop_changenode_t)); cn->cn_handle = zhp; cn->cn_mounted = (clp->cl_gflags & CL_GATHER_MOUNT_ALWAYS) || zfs_is_mounted(zhp, NULL); - cn->cn_shared = zfs_is_shared(zhp); + cn->cn_shared = zfs_is_shared(zhp, NULL, NULL); cn->cn_zoned = zfs_prop_get_int(zhp, ZFS_PROP_ZONED); cn->cn_needpost = B_TRUE; @@ -630,8 +631,7 @@ changelist_gather(zfs_handle_t *zhp, zfs_prop_t prop, int gather_flags, char property[ZFS_MAXPROPLEN]; boolean_t legacy = B_FALSE; - if ((clp = zfs_alloc(zhp->zfs_hdl, sizeof (prop_changelist_t))) == NULL) - return (NULL); + clp = zfs_alloc(zhp->zfs_hdl, sizeof (prop_changelist_t)); /* * For mountpoint-related tasks, we want to sort everything by @@ -744,17 +744,11 @@ changelist_gather(zfs_handle_t *zhp, zfs_prop_t prop, int gather_flags, * Always add ourself to the list. We add ourselves to the end so that * we're the last to be unmounted. */ - if ((cn = zfs_alloc(zhp->zfs_hdl, - sizeof (prop_changenode_t))) == NULL) { - zfs_close(temp); - changelist_free(clp); - return (NULL); - } - + cn = zfs_alloc(zhp->zfs_hdl, sizeof (prop_changenode_t)); cn->cn_handle = temp; cn->cn_mounted = (clp->cl_gflags & CL_GATHER_MOUNT_ALWAYS) || zfs_is_mounted(temp, NULL); - cn->cn_shared = zfs_is_shared(temp); + cn->cn_shared = zfs_is_shared(temp, NULL, NULL); cn->cn_zoned = zfs_prop_get_int(zhp, ZFS_PROP_ZONED); cn->cn_needpost = B_TRUE; diff --git a/lib/libzfs/libzfs_config.c b/lib/libzfs/libzfs_config.c index dd590159660a..437afb3b7467 100644 --- a/lib/libzfs/libzfs_config.c +++ b/lib/libzfs/libzfs_config.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -126,8 +126,7 @@ namespace_reload(libzfs_handle_t *hdl) return (no_memory(hdl)); } - if (zcmd_alloc_dst_nvlist(hdl, &zc, 0) != 0) - return (-1); + zcmd_alloc_dst_nvlist(hdl, &zc, 0); for (;;) { zc.zc_cookie = hdl->libzfs_ns_gen; @@ -141,10 +140,7 @@ namespace_reload(libzfs_handle_t *hdl) return (0); case ENOMEM: - if (zcmd_expand_dst_nvlist(hdl, &zc) != 0) { - zcmd_free_nvlists(&zc); - return (-1); - } + zcmd_expand_dst_nvlist(hdl, &zc); break; default: @@ -181,19 +177,9 @@ namespace_reload(libzfs_handle_t *hdl) nvlist_t *child; uu_avl_index_t where; - if ((cn = zfs_alloc(hdl, sizeof (config_node_t))) == NULL) { - nvlist_free(config); - return (-1); - } - - if ((cn->cn_name = zfs_strdup(hdl, - nvpair_name(elem))) == NULL) { - free(cn); - nvlist_free(config); - return (-1); - } - - verify(nvpair_value_nvlist(elem, &child) == 0); + cn = zfs_alloc(hdl, sizeof (config_node_t)); + cn->cn_name = zfs_strdup(hdl, nvpair_name(elem)); + child = fnvpair_value_nvlist(elem); if (nvlist_dup(child, &cn->cn_config, 0) != 0) { free(cn->cn_name); free(cn); @@ -273,8 +259,7 @@ zpool_refresh_stats(zpool_handle_t *zhp, boolean_t *missing) if (zhp->zpool_config_size == 0) zhp->zpool_config_size = 1 << 16; - if (zcmd_alloc_dst_nvlist(hdl, &zc, zhp->zpool_config_size) != 0) - return (-1); + zcmd_alloc_dst_nvlist(hdl, &zc, zhp->zpool_config_size); for (;;) { if (zfs_ioctl(zhp->zpool_hdl, ZFS_IOC_POOL_STATS, @@ -286,12 +271,9 @@ zpool_refresh_stats(zpool_handle_t *zhp, boolean_t *missing) break; } - if (errno == ENOMEM) { - if (zcmd_expand_dst_nvlist(hdl, &zc) != 0) { - zcmd_free_nvlists(&zc); - return (-1); - } - } else { + if (errno == ENOMEM) + zcmd_expand_dst_nvlist(hdl, &zc); + else { zcmd_free_nvlists(&zc); if (errno == ENOENT || errno == EINVAL) *missing = B_TRUE; diff --git a/lib/libzfs/libzfs_crypto.c b/lib/libzfs/libzfs_crypto.c index e7d3b09c67aa..c241aeaa4da0 100644 --- a/lib/libzfs/libzfs_crypto.c +++ b/lib/libzfs/libzfs_crypto.c @@ -614,7 +614,6 @@ get_key_material_https(libzfs_handle_t *hdl, const char *uri, kfdok: if ((key = fdopen(kfd, "r+")) == NULL) { ret = errno; - free(path); (void) close(kfd); zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "Couldn't reopen temporary file: %s"), strerror(ret)); @@ -784,12 +783,10 @@ derive_key(libzfs_handle_t *hdl, zfs_keyformat_t format, uint64_t iters, *key_out = NULL; key = zfs_alloc(hdl, WRAPPING_KEY_LEN); - if (!key) - return (ENOMEM); switch (format) { case ZFS_KEYFORMAT_RAW: - bcopy(key_material, key, WRAPPING_KEY_LEN); + memcpy(key, key_material, WRAPPING_KEY_LEN); break; case ZFS_KEYFORMAT_HEX: ret = hex_key_to_raw((char *)key_material, @@ -852,7 +849,8 @@ encryption_feature_is_enabled(zpool_handle_t *zph) static int populate_create_encryption_params_nvlists(libzfs_handle_t *hdl, zfs_handle_t *zhp, boolean_t newkey, zfs_keyformat_t keyformat, - char *keylocation, nvlist_t *props, uint8_t **wkeydata, uint_t *wkeylen) + const char *keylocation, nvlist_t *props, uint8_t **wkeydata, + uint_t *wkeylen) { int ret; uint64_t iters = 0, salt = 0; @@ -1006,7 +1004,7 @@ zfs_crypto_create(libzfs_handle_t *hdl, char *parent_name, nvlist_t *props, uint_t *wkeylen_out) { int ret; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; uint64_t crypt = ZIO_CRYPT_INHERIT, pcrypt = ZIO_CRYPT_INHERIT; uint64_t keyformat = ZFS_KEYFORMAT_NONE; char *keylocation = NULL; @@ -1124,7 +1122,7 @@ zfs_crypto_create(libzfs_handle_t *hdl, char *parent_name, nvlist_t *props, /* default to prompt if no keylocation is specified */ if (keyformat != ZFS_KEYFORMAT_NONE && keylocation == NULL) { - keylocation = "prompt"; + keylocation = (char *)"prompt"; ret = nvlist_add_string(props, zfs_prop_to_name(ZFS_PROP_KEYLOCATION), keylocation); if (ret != 0) @@ -1177,7 +1175,7 @@ zfs_crypto_clone_check(libzfs_handle_t *hdl, zfs_handle_t *origin_zhp, char *parent_name, nvlist_t *props) { (void) origin_zhp, (void) parent_name; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "Encryption clone error")); @@ -1279,7 +1277,7 @@ zfs_crypto_load_key(zfs_handle_t *zhp, boolean_t noop, const char *alt_keylocation) { int ret, attempts = 0; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; uint64_t keystatus, iters = 0, salt = 0; uint64_t keyformat = ZFS_KEYFORMAT_NONE; char prop_keylocation[MAXNAMELEN]; @@ -1447,7 +1445,7 @@ int zfs_crypto_unload_key(zfs_handle_t *zhp) { int ret; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; char prop_encroot[MAXNAMELEN]; uint64_t keystatus, keyformat; boolean_t is_encroot; @@ -1583,7 +1581,7 @@ int zfs_crypto_rewrap(zfs_handle_t *zhp, nvlist_t *raw_props, boolean_t inheritkey) { int ret; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; boolean_t is_encroot; nvlist_t *props = NULL; uint8_t *wkeydata = NULL; @@ -1702,7 +1700,7 @@ zfs_crypto_rewrap(zfs_handle_t *zhp, nvlist_t *raw_props, boolean_t inheritkey) /* default to prompt if no keylocation is specified */ if (keylocation == NULL) { - keylocation = "prompt"; + keylocation = (char *)"prompt"; ret = nvlist_add_string(props, zfs_prop_to_name(ZFS_PROP_KEYLOCATION), keylocation); diff --git a/lib/libzfs/libzfs_dataset.c b/lib/libzfs/libzfs_dataset.c index 6b9f43285c89..a8b94f758229 100644 --- a/lib/libzfs/libzfs_dataset.c +++ b/lib/libzfs/libzfs_dataset.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -49,7 +49,6 @@ #include #include #include -#include #ifdef HAVE_IDMAP #include #include @@ -234,7 +233,6 @@ process_user_props(zfs_handle_t *zhp, nvlist_t *props) { libzfs_handle_t *hdl = zhp->zfs_hdl; nvpair_t *elem; - nvlist_t *propval; nvlist_t *nvl; if (nvlist_alloc(&nvl, NV_UNIQUE_NAME, 0) != 0) { @@ -247,7 +245,7 @@ process_user_props(zfs_handle_t *zhp, nvlist_t *props) if (!zfs_prop_user(nvpair_name(elem))) continue; - verify(nvpair_value_nvlist(elem, &propval) == 0); + nvlist_t *propval = fnvpair_value_nvlist(elem); if (nvlist_add_nvlist(nvl, nvpair_name(elem), propval) != 0) { nvlist_free(nvl); (void) no_memory(hdl); @@ -332,13 +330,10 @@ get_stats_ioctl(zfs_handle_t *zhp, zfs_cmd_t *zc) (void) strlcpy(zc->zc_name, zhp->zfs_name, sizeof (zc->zc_name)); while (zfs_ioctl(hdl, ZFS_IOC_OBJSET_STATS, zc) != 0) { - if (errno == ENOMEM) { - if (zcmd_expand_dst_nvlist(hdl, zc) != 0) { - return (-1); - } - } else { + if (errno == ENOMEM) + zcmd_expand_dst_nvlist(hdl, zc); + else return (-1); - } } return (0); } @@ -354,17 +349,14 @@ get_recvd_props_ioctl(zfs_handle_t *zhp) zfs_cmd_t zc = {"\0"}; int err; - if (zcmd_alloc_dst_nvlist(hdl, &zc, 0) != 0) - return (-1); + zcmd_alloc_dst_nvlist(hdl, &zc, 0); (void) strlcpy(zc.zc_name, zhp->zfs_name, sizeof (zc.zc_name)); while (zfs_ioctl(hdl, ZFS_IOC_OBJSET_RECVD_PROPS, &zc) != 0) { - if (errno == ENOMEM) { - if (zcmd_expand_dst_nvlist(hdl, &zc) != 0) { - return (-1); - } - } else { + if (errno == ENOMEM) + zcmd_expand_dst_nvlist(hdl, &zc); + else { zcmd_free_nvlists(&zc); return (-1); } @@ -416,8 +408,8 @@ get_stats(zfs_handle_t *zhp) int rc = 0; zfs_cmd_t zc = {"\0"}; - if (zcmd_alloc_dst_nvlist(zhp->zfs_hdl, &zc, 0) != 0) - return (-1); + zcmd_alloc_dst_nvlist(zhp->zfs_hdl, &zc, 0); + if (get_stats_ioctl(zhp, &zc) != 0) rc = -1; else if (put_stats_zhdl(zhp, &zc) != 0) @@ -449,14 +441,19 @@ make_dataset_handle_common(zfs_handle_t *zhp, zfs_cmd_t *zc) * We've managed to open the dataset and gather statistics. Determine * the high-level type. */ - if (zhp->zfs_dmustats.dds_type == DMU_OST_ZVOL) + if (zhp->zfs_dmustats.dds_type == DMU_OST_ZVOL) { zhp->zfs_head_type = ZFS_TYPE_VOLUME; - else if (zhp->zfs_dmustats.dds_type == DMU_OST_ZFS) + } else if (zhp->zfs_dmustats.dds_type == DMU_OST_ZFS) { zhp->zfs_head_type = ZFS_TYPE_FILESYSTEM; - else if (zhp->zfs_dmustats.dds_type == DMU_OST_OTHER) + } else if (zhp->zfs_dmustats.dds_type == DMU_OST_OTHER) { + errno = EINVAL; return (-1); - else + } else if (zhp->zfs_dmustats.dds_inconsistent) { + errno = EBUSY; + return (-1); + } else { abort(); + } if (zhp->zfs_dmustats.dds_is_snapshot) zhp->zfs_type = ZFS_TYPE_SNAPSHOT; @@ -485,10 +482,8 @@ make_dataset_handle(libzfs_handle_t *hdl, const char *path) zhp->zfs_hdl = hdl; (void) strlcpy(zhp->zfs_name, path, sizeof (zhp->zfs_name)); - if (zcmd_alloc_dst_nvlist(hdl, &zc, 0) != 0) { - free(zhp); - return (NULL); - } + zcmd_alloc_dst_nvlist(hdl, &zc, 0); + if (get_stats_ioctl(zhp, &zc) == -1) { zcmd_free_nvlists(&zc); free(zhp); @@ -532,6 +527,7 @@ make_dataset_simple_handle_zc(zfs_handle_t *pzhp, zfs_cmd_t *zc) zhp->zfs_head_type = pzhp->zfs_type; zhp->zfs_type = ZFS_TYPE_SNAPSHOT; zhp->zpool_hdl = zpool_handle(zhp); + zhp->zfs_dmustats = zc->zc_objset_stats; return (zhp); } @@ -683,7 +679,7 @@ zfs_handle_t * zfs_open(libzfs_handle_t *hdl, const char *path, int types) { zfs_handle_t *zhp; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; char *bookp; (void) snprintf(errbuf, sizeof (errbuf), @@ -887,7 +883,7 @@ libzfs_mnttab_find(libzfs_handle_t *hdl, const char *fsname, return (ENOENT); srch.mnt_special = (char *)fsname; - srch.mnt_fstype = MNTTYPE_ZFS; + srch.mnt_fstype = (char *)MNTTYPE_ZFS; ret = getmntany(mnttab, entry, &srch) ? ENOENT : 0; (void) fclose(mnttab); return (ret); @@ -1027,7 +1023,7 @@ zfs_valid_proplist(libzfs_handle_t *hdl, zfs_type_t type, nvlist_t *nvl, const char *propname = nvpair_name(elem); prop = zfs_name_to_prop(propname); - if (prop == ZPROP_INVAL && zfs_prop_user(propname)) { + if (prop == ZPROP_USERPROP && zfs_prop_user(propname)) { /* * This is a user property: make sure it's a * string, and that it's less than ZAP_MAXNAMELEN. @@ -1066,7 +1062,7 @@ zfs_valid_proplist(libzfs_handle_t *hdl, zfs_type_t type, nvlist_t *nvl, goto error; } - if (prop == ZPROP_INVAL && zfs_prop_userquota(propname)) { + if (prop == ZPROP_USERPROP && zfs_prop_userquota(propname)) { zfs_userquota_prop_t uqtype; char *newpropname = NULL; char domain[128]; @@ -1148,7 +1144,8 @@ zfs_valid_proplist(libzfs_handle_t *hdl, zfs_type_t type, nvlist_t *nvl, } free(newpropname); continue; - } else if (prop == ZPROP_INVAL && zfs_prop_written(propname)) { + } else if (prop == ZPROP_USERPROP && + zfs_prop_written(propname)) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "'%s' is readonly"), propname); @@ -1310,8 +1307,7 @@ zfs_valid_proplist(libzfs_handle_t *hdl, zfs_type_t type, nvlist_t *nvl, /* Replace the label string with the internal form. */ (void) nvlist_remove(ret, zfs_prop_to_name(prop), DATA_TYPE_STRING); - verify(nvlist_add_string(ret, zfs_prop_to_name(prop), - hex) == 0); + fnvlist_add_string(ret, zfs_prop_to_name(prop), hex); free(hex); break; @@ -1423,14 +1419,15 @@ zfs_valid_proplist(libzfs_handle_t *hdl, zfs_type_t type, nvlist_t *nvl, prop == ZFS_PROP_SHARESMB) && strcmp(strval, "on") != 0 && strcmp(strval, "off") != 0) { - zfs_share_proto_t proto; + enum sa_protocol proto; if (prop == ZFS_PROP_SHARESMB) - proto = PROTO_SMB; + proto = SA_PROTOCOL_SMB; else - proto = PROTO_NFS; + proto = SA_PROTOCOL_NFS; - if (zfs_parse_options(strval, proto) != SA_OK) { + if (sa_validate_shareopts(strval, proto) != + SA_OK) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "'%s' cannot be set to invalid " "options"), propname); @@ -1721,7 +1718,7 @@ int zfs_prop_set(zfs_handle_t *zhp, const char *propname, const char *propval) { int ret = -1; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; libzfs_handle_t *hdl = zhp->zfs_hdl; nvlist_t *nvl = NULL; @@ -1755,7 +1752,7 @@ zfs_prop_set_list(zfs_handle_t *zhp, nvlist_t *props) int ret = -1; prop_changelist_t **cls = NULL; int cl_idx; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; libzfs_handle_t *hdl = zhp->zfs_hdl; nvlist_t *nvl; int nvl_len = 0; @@ -1844,9 +1841,8 @@ zfs_prop_set_list(zfs_handle_t *zhp, nvlist_t *props) */ (void) strlcpy(zc.zc_name, zhp->zfs_name, sizeof (zc.zc_name)); - if ((ret = zcmd_write_src_nvlist(hdl, &zc, nvl)) != 0 || - (ret = zcmd_alloc_dst_nvlist(hdl, &zc, 0)) != 0) - goto error; + zcmd_write_src_nvlist(hdl, &zc, nvl); + zcmd_alloc_dst_nvlist(hdl, &zc, 0); ret = zfs_ioctl(hdl, ZFS_IOC_SET_PROP, &zc); @@ -1882,8 +1878,7 @@ zfs_prop_set_list(zfs_handle_t *zhp, nvlist_t *props) zfs_prop_to_name(ZFS_PROP_VOLSIZE), old_volsize) != 0) goto error; - if (zcmd_write_src_nvlist(hdl, &zc, nvl) != 0) - goto error; + zcmd_write_src_nvlist(hdl, &zc, nvl); (void) zfs_ioctl(hdl, ZFS_IOC_SET_PROP, &zc); } } else { @@ -1937,14 +1932,14 @@ zfs_prop_inherit(zfs_handle_t *zhp, const char *propname, boolean_t received) int ret; prop_changelist_t *cl; libzfs_handle_t *hdl = zhp->zfs_hdl; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; zfs_prop_t prop; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot inherit %s for '%s'"), propname, zhp->zfs_name); zc.zc_cookie = received; - if ((prop = zfs_name_to_prop(propname)) == ZPROP_INVAL) { + if ((prop = zfs_name_to_prop(propname)) == ZPROP_USERPROP) { /* * For user properties, the amount of work we have to do is very * small, so just do it here. @@ -2051,13 +2046,13 @@ getprop_uint64(zfs_handle_t *zhp, zfs_prop_t prop, char **source) *source = NULL; if (nvlist_lookup_nvlist(zhp->zfs_props, zfs_prop_to_name(prop), &nv) == 0) { - verify(nvlist_lookup_uint64(nv, ZPROP_VALUE, &value) == 0); + value = fnvlist_lookup_uint64(nv, ZPROP_VALUE); (void) nvlist_lookup_string(nv, ZPROP_SOURCE, source); } else { verify(!zhp->zfs_props_table || zhp->zfs_props_table[prop] == B_TRUE); value = zfs_prop_default_numeric(prop); - *source = ""; + *source = (char *)""; } return (value); @@ -2078,7 +2073,7 @@ getprop_string(zfs_handle_t *zhp, zfs_prop_t prop, char **source) verify(!zhp->zfs_props_table || zhp->zfs_props_table[prop] == B_TRUE); value = zfs_prop_default_string(prop); - *source = ""; + *source = (char *)""; } return (value); @@ -2120,8 +2115,8 @@ get_numeric_property(zfs_handle_t *zhp, zfs_prop_t prop, zprop_source_t *src, zfs_cmd_t zc = {"\0"}; nvlist_t *zplprops = NULL; struct mnttab mnt; - char *mntopt_on = NULL; - char *mntopt_off = NULL; + const char *mntopt_on = NULL; + const char *mntopt_off = NULL; boolean_t received = zfs_is_recvd_props_mode(zhp); *source = NULL; @@ -2192,18 +2187,15 @@ get_numeric_property(zfs_handle_t *zhp, zfs_prop_t prop, zprop_source_t *src, libzfs_handle_t *hdl = zhp->zfs_hdl; struct mnttab entry; - if (libzfs_mnttab_find(hdl, zhp->zfs_name, &entry) == 0) { + if (libzfs_mnttab_find(hdl, zhp->zfs_name, &entry) == 0) zhp->zfs_mntopts = zfs_strdup(hdl, entry.mnt_mntopts); - if (zhp->zfs_mntopts == NULL) - return (-1); - } zhp->zfs_mntcheck = B_TRUE; } if (zhp->zfs_mntopts == NULL) - mnt.mnt_mntopts = ""; + mnt.mnt_mntopts = (char *)""; else mnt.mnt_mntopts = zhp->zfs_mntopts; @@ -2264,8 +2256,8 @@ get_numeric_property(zfs_handle_t *zhp, zfs_prop_t prop, zprop_source_t *src, case ZFS_PROP_NORMALIZE: case ZFS_PROP_UTF8ONLY: case ZFS_PROP_CASE: - if (zcmd_alloc_dst_nvlist(zhp->zfs_hdl, &zc, 0) != 0) - return (-1); + zcmd_alloc_dst_nvlist(zhp->zfs_hdl, &zc, 0); + (void) strlcpy(zc.zc_name, zhp->zfs_name, sizeof (zc.zc_name)); if (zfs_ioctl(zhp->zfs_hdl, ZFS_IOC_OBJSET_ZPLPROPS, &zc)) { zcmd_free_nvlists(&zc); @@ -2292,6 +2284,19 @@ get_numeric_property(zfs_handle_t *zhp, zfs_prop_t prop, zprop_source_t *src, *val = zhp->zfs_dmustats.dds_redacted; break; + case ZFS_PROP_CREATETXG: + /* + * We can directly read createtxg property from zfs + * handle for Filesystem, Snapshot and ZVOL types. + */ + if ((zhp->zfs_type == ZFS_TYPE_FILESYSTEM) || + (zhp->zfs_type == ZFS_TYPE_SNAPSHOT) || + (zhp->zfs_type == ZFS_TYPE_VOLUME)) { + *val = zhp->zfs_dmustats.dds_creation_txg; + break; + } + zfs_fallthrough; + default: switch (zfs_prop_get_type(prop)) { case PROP_TYPE_NUMBER: @@ -2366,7 +2371,7 @@ zfs_prop_get_recvd(zfs_handle_t *zhp, const char *propname, char *propbuf, prop = zfs_name_to_prop(propname); - if (prop != ZPROP_INVAL) { + if (prop != ZPROP_USERPROP) { uint64_t cookie; if (!nvlist_exists(zhp->zfs_recvd_props, propname)) return (-1); @@ -2380,8 +2385,7 @@ zfs_prop_get_recvd(zfs_handle_t *zhp, const char *propname, char *propbuf, if (nvlist_lookup_nvlist(zhp->zfs_recvd_props, propname, &propval) != 0) return (-1); - verify(nvlist_lookup_string(propval, ZPROP_VALUE, - &recvdval) == 0); + recvdval = fnvlist_lookup_string(propval, ZPROP_VALUE); (void) strlcpy(propbuf, recvdval, proplen); } @@ -2495,13 +2499,11 @@ zfs_get_clones_nvl(zfs_handle_t *zhp) } nvlist_free(nv); nvlist_free(value); - verify(0 == nvlist_lookup_nvlist(zhp->zfs_props, - zfs_prop_to_name(ZFS_PROP_CLONES), &nv)); + nv = fnvlist_lookup_nvlist(zhp->zfs_props, + zfs_prop_to_name(ZFS_PROP_CLONES)); } - verify(nvlist_lookup_nvlist(nv, ZPROP_VALUE, &value) == 0); - - return (value); + return (fnvlist_lookup_nvlist(nv, ZPROP_VALUE)); } static int @@ -2898,6 +2900,8 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop_t prop, char *propbuf, size_t proplen, break; case ZFS_PROP_GUID: + case ZFS_PROP_KEY_GUID: + case ZFS_PROP_IVSET_GUID: case ZFS_PROP_CREATETXG: case ZFS_PROP_OBJSETID: case ZFS_PROP_PBKDF2_ITERS: @@ -3413,7 +3417,7 @@ check_parents(libzfs_handle_t *hdl, const char *path, uint64_t *zoned, char parent[ZFS_MAX_DATASET_NAME_LEN]; char *slash; zfs_handle_t *zhp; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; uint64_t is_zoned; (void) snprintf(errbuf, sizeof (errbuf), @@ -3566,14 +3570,14 @@ create_parents(libzfs_handle_t *hdl, char *target, int prefixlen) goto ancestorerr; } - if (zfs_share(h) != 0) { + if (zfs_share(h, NULL) != 0) { opname = dgettext(TEXT_DOMAIN, "share"); goto ancestorerr; } zfs_close(h); } - zfs_commit_all_shares(); + zfs_commit_shares(NULL); return (0); @@ -3591,7 +3595,7 @@ zfs_create_ancestors(libzfs_handle_t *hdl, const char *path) { int prefix; char *path_copy; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; int rc = 0; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, @@ -3635,7 +3639,7 @@ zfs_create(libzfs_handle_t *hdl, const char *path, zfs_type_t type, zpool_handle_t *zpool_handle; uint8_t *wkeydata = NULL; uint_t wkeylen = 0; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; char parent[ZFS_MAX_DATASET_NAME_LEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, @@ -3856,7 +3860,7 @@ zfs_check_snap_cb(zfs_handle_t *zhp, void *arg) return (EINVAL); if (lzc_exists(name)) - verify(nvlist_add_boolean(dd->nvl, name) == 0); + fnvlist_add_boolean(dd->nvl, name); rv = zfs_iter_filesystems(zhp, zfs_check_snap_cb, dd); zfs_close(zhp); @@ -3873,7 +3877,7 @@ zfs_destroy_snaps(zfs_handle_t *zhp, char *snapname, boolean_t defer) struct destroydata dd = { 0 }; dd.snapname = snapname; - verify(nvlist_alloc(&dd.nvl, NV_UNIQUE_NAME, 0) == 0); + dd.nvl = fnvlist_alloc(); (void) zfs_check_snap_cb(zfs_handle_dup(zhp), &dd); if (nvlist_empty(dd.nvl)) { @@ -3883,7 +3887,7 @@ zfs_destroy_snaps(zfs_handle_t *zhp, char *snapname, boolean_t defer) } else { ret = zfs_destroy_snaps_nvl(zhp->zfs_hdl, dd.nvl, defer); } - nvlist_free(dd.nvl); + fnvlist_free(dd.nvl); return (ret); } @@ -3908,7 +3912,7 @@ zfs_destroy_snaps_nvl(libzfs_handle_t *hdl, nvlist_t *snaps, boolean_t defer) } if (nvlist_empty(errlist)) { - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot destroy snapshots")); @@ -3916,7 +3920,7 @@ zfs_destroy_snaps_nvl(libzfs_handle_t *hdl, nvlist_t *snaps, boolean_t defer) } for (pair = nvlist_next_nvpair(errlist, NULL); pair != NULL; pair = nvlist_next_nvpair(errlist, pair)) { - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot destroy snapshot %s"), nvpair_name(pair)); @@ -3945,7 +3949,7 @@ zfs_clone(zfs_handle_t *zhp, const char *target, nvlist_t *props) { char parent[ZFS_MAX_DATASET_NAME_LEN]; int ret; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; libzfs_handle_t *hdl = zhp->zfs_hdl; uint64_t zoned; @@ -3967,13 +3971,10 @@ zfs_clone(zfs_handle_t *zhp, const char *target, nvlist_t *props) /* do the clone */ if (props) { - zfs_type_t type; + zfs_type_t type = ZFS_TYPE_FILESYSTEM; - if (ZFS_IS_VOLUME(zhp)) { + if (ZFS_IS_VOLUME(zhp)) type = ZFS_TYPE_VOLUME; - } else { - type = ZFS_TYPE_FILESYSTEM; - } if ((props = zfs_valid_proplist(hdl, type, props, zoned, zhp, zhp->zpool_hdl, B_TRUE, errbuf)) == NULL) return (-1); @@ -4032,7 +4033,7 @@ zfs_promote(zfs_handle_t *zhp) libzfs_handle_t *hdl = zhp->zfs_hdl; char snapname[ZFS_MAX_DATASET_NAME_LEN]; int ret; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot promote '%s'"), zhp->zfs_name); @@ -4114,7 +4115,7 @@ int zfs_snapshot_nvl(libzfs_handle_t *hdl, nvlist_t *snaps, nvlist_t *props) { int ret; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; nvpair_t *elem; nvlist_t *errors; zpool_handle_t *zpool_hdl; @@ -4199,7 +4200,7 @@ zfs_snapshot(libzfs_handle_t *hdl, const char *path, boolean_t recursive, char fsname[ZFS_MAX_DATASET_NAME_LEN]; char *cp; zfs_handle_t *zhp; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot snapshot %s"), path); @@ -4217,7 +4218,7 @@ zfs_snapshot(libzfs_handle_t *hdl, const char *path, boolean_t recursive, return (-1); } - verify(nvlist_alloc(&sd.sd_nvl, NV_UNIQUE_NAME, 0) == 0); + sd.sd_nvl = fnvlist_alloc(); if (recursive) { (void) zfs_snapshot_cb(zfs_handle_dup(zhp), &sd); } else { @@ -4225,7 +4226,7 @@ zfs_snapshot(libzfs_handle_t *hdl, const char *path, boolean_t recursive, } ret = zfs_snapshot_nvl(hdl, sd.sd_nvl, props); - nvlist_free(sd.sd_nvl); + fnvlist_free(sd.sd_nvl); zfs_close(zhp); return (ret); } @@ -4342,7 +4343,7 @@ zfs_rollback(zfs_handle_t *zhp, zfs_handle_t *snap, boolean_t force) */ err = lzc_rollback_to(zhp->zfs_name, snap->zfs_name); if (err != 0) { - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot rollback '%s'"), @@ -4401,7 +4402,7 @@ zfs_rename(zfs_handle_t *zhp, const char *target, renameflags_t flags) char parent[ZFS_MAX_DATASET_NAME_LEN]; char property[ZFS_MAXPROPLEN]; libzfs_handle_t *hdl = zhp->zfs_hdl; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; /* if we have the same exact name, just return success */ if (strcmp(zhp->zfs_name, target) == 0) @@ -4506,10 +4507,6 @@ zfs_rename(zfs_handle_t *zhp, const char *target, renameflags_t flags) } if (flags.recursive) { char *parentname = zfs_strdup(zhp->zfs_hdl, zhp->zfs_name); - if (parentname == NULL) { - ret = -1; - goto error; - } delim = strchr(parentname, '@'); *delim = '\0'; zfs_handle_t *zhrp = zfs_open(zhp->zfs_hdl, parentname, @@ -4653,7 +4650,7 @@ zfs_expand_proplist(zfs_handle_t *zhp, zprop_list_t **plp, boolean_t received, */ start = plp; while (*start != NULL) { - if ((*start)->pl_prop == ZPROP_INVAL) + if ((*start)->pl_prop == ZPROP_USERPROP) break; start = &(*start)->pl_next; } @@ -4671,15 +4668,10 @@ zfs_expand_proplist(zfs_handle_t *zhp, zprop_list_t **plp, boolean_t received, } if (*last == NULL) { - if ((entry = zfs_alloc(hdl, - sizeof (zprop_list_t))) == NULL || - ((entry->pl_user_prop = zfs_strdup(hdl, - nvpair_name(elem)))) == NULL) { - free(entry); - return (-1); - } - - entry->pl_prop = ZPROP_INVAL; + entry = zfs_alloc(hdl, sizeof (zprop_list_t)); + entry->pl_user_prop = + zfs_strdup(hdl, nvpair_name(elem)); + entry->pl_prop = ZPROP_USERPROP; entry->pl_width = strlen(nvpair_name(elem)); entry->pl_all = B_TRUE; *last = entry; @@ -4694,7 +4686,7 @@ zfs_expand_proplist(zfs_handle_t *zhp, zprop_list_t **plp, boolean_t received, if (entry->pl_fixed && !literal) continue; - if (entry->pl_prop != ZPROP_INVAL) { + if (entry->pl_prop != ZPROP_USERPROP) { if (zfs_prop_get(zhp, entry->pl_prop, buf, sizeof (buf), NULL, NULL, 0, literal) == 0) { if (strlen(buf) > entry->pl_width) @@ -4708,8 +4700,8 @@ zfs_expand_proplist(zfs_handle_t *zhp, zprop_list_t **plp, boolean_t received, } else { if (nvlist_lookup_nvlist(userprops, entry->pl_user_prop, &propval) == 0) { - verify(nvlist_lookup_string(propval, - ZPROP_VALUE, &strval) == 0); + strval = fnvlist_lookup_string(propval, + ZPROP_VALUE); if (strlen(strval) > entry->pl_width) entry->pl_width = strlen(strval); } @@ -4743,13 +4735,14 @@ zfs_prune_proplist(zfs_handle_t *zhp, uint8_t *props) next = nvlist_next_nvpair(zhp->zfs_props, curr); /* - * User properties will result in ZPROP_INVAL, and since we + * User properties will result in ZPROP_USERPROP (an alias + * for ZPROP_INVAL), and since we * only know how to prune standard ZFS properties, we always * leave these in the list. This can also happen if we * encounter an unknown DSL property (when running older * software, for example). */ - if (zfs_prop != ZPROP_INVAL && props[zfs_prop] == B_FALSE) + if (zfs_prop != ZPROP_USERPROP && props[zfs_prop] == B_FALSE) (void) nvlist_remove(zhp->zfs_props, nvpair_name(curr), nvpair_type(curr)); curr = next; @@ -4791,10 +4784,7 @@ zfs_smb_acl_mgmt(libzfs_handle_t *hdl, char *dataset, char *path, (void) no_memory(hdl); return (-1); } - if (zcmd_write_src_nvlist(hdl, &zc, nvlist) != 0) { - nvlist_free(nvlist); - return (-1); - } + zcmd_write_src_nvlist(hdl, &zc, nvlist); break; case ZFS_SMB_ACL_PURGE: break; @@ -4928,7 +4918,7 @@ zfs_hold(zfs_handle_t *zhp, const char *snapname, const char *tag, (void) zfs_hold_one(zfs_handle_dup(zhp), &ha); if (nvlist_empty(ha.nvl)) { - char errbuf[1024]; + char errbuf[ERRBUFLEN]; fnvlist_free(ha.nvl); ret = ENOENT; @@ -4952,7 +4942,7 @@ zfs_hold_nvl(zfs_handle_t *zhp, int cleanup_fd, nvlist_t *holds) int ret; nvlist_t *errors; libzfs_handle_t *hdl = zhp->zfs_hdl; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; nvpair_t *elem; errors = NULL; @@ -5054,7 +5044,7 @@ zfs_release(zfs_handle_t *zhp, const char *snapname, const char *tag, nvlist_t *errors = NULL; nvpair_t *elem; libzfs_handle_t *hdl = zhp->zfs_hdl; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; ha.nvl = fnvlist_alloc(); ha.snapname = snapname; @@ -5134,7 +5124,7 @@ zfs_get_fsacl(zfs_handle_t *zhp, nvlist_t **nvl) int nvsz = 2048; void *nvbuf; int err = 0; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; assert(zhp->zfs_type == ZFS_TYPE_VOLUME || zhp->zfs_type == ZFS_TYPE_FILESYSTEM); @@ -5198,7 +5188,7 @@ zfs_set_fsacl(zfs_handle_t *zhp, boolean_t un, nvlist_t *nvl) zfs_cmd_t zc = {"\0"}; libzfs_handle_t *hdl = zhp->zfs_hdl; char *nvbuf; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; size_t nvsz; int err; @@ -5250,7 +5240,7 @@ int zfs_get_holds(zfs_handle_t *zhp, nvlist_t **nvl) { int err; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; err = lzc_get_holds(zhp->zfs_name, nvl); diff --git a/lib/libzfs/libzfs_diff.c b/lib/libzfs/libzfs_diff.c index 8884ac536a03..93f6e19e9127 100644 --- a/lib/libzfs/libzfs_diff.c +++ b/lib/libzfs/libzfs_diff.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -41,7 +41,6 @@ #include #include #include -#include #include #include #include @@ -602,11 +601,10 @@ get_snapshot_names(differ_info_t *di, const char *fromsnap, di->isclone = B_TRUE; di->fromsnap = zfs_strdup(hdl, fromsnap); - if (tsnlen) { + if (tsnlen) di->tosnap = zfs_strdup(hdl, tosnap); - } else { + else return (make_temp_snapshot(di)); - } } else { int dslen = fdslen ? fdslen : tdslen; @@ -711,7 +709,7 @@ zfs_show_diffs(zfs_handle_t *zhp, int outfd, const char *fromsnap, const char *tosnap, int flags) { zfs_cmd_t zc = {"\0"}; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; differ_info_t di = { 0 }; pthread_t tid; int pipefd[2]; diff --git a/lib/libzfs/libzfs_impl.h b/lib/libzfs/libzfs_impl.h index cf8d0d4e6742..64de6d7b22b4 100644 --- a/lib/libzfs/libzfs_impl.h +++ b/lib/libzfs/libzfs_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -44,6 +44,8 @@ extern "C" { #endif +#define ERRBUFLEN 1024 + struct libzfs_handle { int libzfs_error; int libzfs_fd; @@ -105,22 +107,6 @@ struct zpool_handle { diskaddr_t zpool_start_block; }; -typedef enum { - PROTO_NFS = 0, - PROTO_SMB = 1, - PROTO_END = 2 -} zfs_share_proto_t; - -/* - * The following can be used as a bitmask and any new values - * added must preserve that capability. - */ -typedef enum { - SHARED_NOT_SHARED = 0x0, - SHARED_NFS = 0x2, - SHARED_SMB = 0x4 -} zfs_share_type_t; - typedef int (*zfs_uri_handler_fn_t)(struct libzfs_handle *, const char *, const char *, zfs_keyformat_t, boolean_t, uint8_t **, size_t *); @@ -175,10 +161,10 @@ extern int zprop_expand_list(libzfs_handle_t *hdl, zprop_list_t **plp, typedef struct prop_changelist prop_changelist_t; -extern int zcmd_alloc_dst_nvlist(libzfs_handle_t *, zfs_cmd_t *, size_t); -extern int zcmd_write_src_nvlist(libzfs_handle_t *, zfs_cmd_t *, nvlist_t *); -extern int zcmd_write_conf_nvlist(libzfs_handle_t *, zfs_cmd_t *, nvlist_t *); -extern int zcmd_expand_dst_nvlist(libzfs_handle_t *, zfs_cmd_t *); +extern void zcmd_alloc_dst_nvlist(libzfs_handle_t *, zfs_cmd_t *, size_t); +extern void zcmd_write_src_nvlist(libzfs_handle_t *, zfs_cmd_t *, nvlist_t *); +extern void zcmd_write_conf_nvlist(libzfs_handle_t *, zfs_cmd_t *, nvlist_t *); +extern void zcmd_expand_dst_nvlist(libzfs_handle_t *, zfs_cmd_t *); extern int zcmd_read_dst_nvlist(libzfs_handle_t *, zfs_cmd_t *, nvlist_t **); extern void zcmd_free_nvlists(zfs_cmd_t *); @@ -189,7 +175,7 @@ extern void changelist_remove(prop_changelist_t *, const char *); extern void changelist_free(prop_changelist_t *); extern prop_changelist_t *changelist_gather(zfs_handle_t *, zfs_prop_t, int, int); -extern int changelist_unshare(prop_changelist_t *, const zfs_share_proto_t *); +extern int changelist_unshare(prop_changelist_t *, const enum sa_protocol *); extern int changelist_haszonedchild(prop_changelist_t *); extern void remove_mountpoint(zfs_handle_t *); @@ -209,11 +195,8 @@ extern int zfs_validate_name(libzfs_handle_t *hdl, const char *path, int type, extern void namespace_clear(libzfs_handle_t *); -extern int zfs_parse_options(char *, zfs_share_proto_t); - typedef struct { zfs_prop_t p_prop; - char *p_name; int p_share_err; int p_unshare_err; } proto_table_t; @@ -227,7 +210,7 @@ typedef struct differ_info { char *ds; char *dsmnt; char *tmpsnap; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; boolean_t isclone; boolean_t scripted; boolean_t classify; @@ -240,25 +223,13 @@ typedef struct differ_info { int datafd; } differ_info_t; -extern int do_mount(zfs_handle_t *zhp, const char *mntpt, char *opts, +extern int do_mount(zfs_handle_t *zhp, const char *mntpt, const char *opts, int flags); extern int do_unmount(zfs_handle_t *zhp, const char *mntpt, int flags); -extern int zfs_mount_delegation_check(void); -extern int zfs_share_proto(zfs_handle_t *zhp, const zfs_share_proto_t *proto); -extern int zfs_unshare_proto(zfs_handle_t *, const char *, - const zfs_share_proto_t *); -extern int unshare_one(libzfs_handle_t *hdl, const char *name, - const char *mountpoint, zfs_share_proto_t proto); -extern boolean_t zfs_is_mountable(zfs_handle_t *zhp, char *buf, size_t buflen, - zprop_source_t *source, int flags); -extern zfs_share_type_t is_shared(const char *mountpoint, - zfs_share_proto_t proto); extern int libzfs_load_module(void); extern int zpool_relabel_disk(libzfs_handle_t *hdl, const char *path, const char *msg); extern int find_shares_object(differ_info_t *di); -extern void libzfs_set_pipe_max(int infd); -extern void zfs_commit_proto(const zfs_share_proto_t *); #ifdef __cplusplus } diff --git a/lib/libzfs/libzfs_import.c b/lib/libzfs/libzfs_import.c index 0d375b355166..5023f06f7cad 100644 --- a/lib/libzfs/libzfs_import.c +++ b/lib/libzfs/libzfs_import.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -48,7 +48,6 @@ pool_active(libzfs_handle_t *hdl, const char *name, uint64_t guid, boolean_t *isactive) { zpool_handle_t *zhp; - uint64_t theguid; if (zpool_open_silent(hdl, name, &zhp) != 0) return (-1); @@ -58,8 +57,8 @@ pool_active(libzfs_handle_t *hdl, const char *name, uint64_t guid, return (0); } - verify(nvlist_lookup_uint64(zhp->zpool_config, ZPOOL_CONFIG_POOL_GUID, - &theguid) == 0); + uint64_t theguid = fnvlist_lookup_uint64(zhp->zpool_config, + ZPOOL_CONFIG_POOL_GUID); zpool_close(zhp); @@ -74,23 +73,15 @@ refresh_config(libzfs_handle_t *hdl, nvlist_t *config) zfs_cmd_t zc = {"\0"}; int err, dstbuf_size; - if (zcmd_write_conf_nvlist(hdl, &zc, config) != 0) - return (NULL); + zcmd_write_conf_nvlist(hdl, &zc, config); dstbuf_size = MAX(CONFIG_BUF_MINSIZE, zc.zc_nvlist_conf_size * 32); - if (zcmd_alloc_dst_nvlist(hdl, &zc, dstbuf_size) != 0) { - zcmd_free_nvlists(&zc); - return (NULL); - } + zcmd_alloc_dst_nvlist(hdl, &zc, dstbuf_size); while ((err = zfs_ioctl(hdl, ZFS_IOC_POOL_TRYIMPORT, - &zc)) != 0 && errno == ENOMEM) { - if (zcmd_expand_dst_nvlist(hdl, &zc) != 0) { - zcmd_free_nvlists(&zc); - return (NULL); - } - } + &zc)) != 0 && errno == ENOMEM) + zcmd_expand_dst_nvlist(hdl, &zc); if (err) { zcmd_free_nvlists(&zc); @@ -146,10 +137,9 @@ zpool_clear_label(int fd) struct stat64 statbuf; int l; vdev_label_t *label; - l2arc_dev_hdr_phys_t *l2dhdr; uint64_t size; - int labels_cleared = 0, header_cleared = 0; - boolean_t clear_l2arc_header = B_FALSE; + boolean_t labels_cleared = B_FALSE, clear_l2arc_header = B_FALSE, + header_cleared = B_FALSE; if (fstat64_blk(fd, &statbuf) == -1) return (0); @@ -159,11 +149,6 @@ zpool_clear_label(int fd) if ((label = calloc(1, sizeof (vdev_label_t))) == NULL) return (-1); - if ((l2dhdr = calloc(1, sizeof (l2arc_dev_hdr_phys_t))) == NULL) { - free(label); - return (-1); - } - for (l = 0; l < VDEV_LABELS; l++) { uint64_t state, guid, l2cache; nvlist_t *config; @@ -213,24 +198,22 @@ zpool_clear_label(int fd) size_t label_size = sizeof (vdev_label_t) - (2 * VDEV_PAD_SIZE); if (pwrite64(fd, label, label_size, label_offset(size, l) + - (2 * VDEV_PAD_SIZE)) == label_size) { - labels_cleared++; - } + (2 * VDEV_PAD_SIZE)) == label_size) + labels_cleared = B_TRUE; } - /* Clear the L2ARC header. */ if (clear_l2arc_header) { - memset(l2dhdr, 0, sizeof (l2arc_dev_hdr_phys_t)); - if (pwrite64(fd, l2dhdr, sizeof (l2arc_dev_hdr_phys_t), - VDEV_LABEL_START_SIZE) == sizeof (l2arc_dev_hdr_phys_t)) { - header_cleared++; - } + _Static_assert(sizeof (*label) >= sizeof (l2arc_dev_hdr_phys_t), + "label < l2arc_dev_hdr_phys_t"); + memset(label, 0, sizeof (l2arc_dev_hdr_phys_t)); + if (pwrite64(fd, label, sizeof (l2arc_dev_hdr_phys_t), + VDEV_LABEL_START_SIZE) == sizeof (l2arc_dev_hdr_phys_t)) + header_cleared = B_TRUE; } free(label); - free(l2dhdr); - if (labels_cleared == 0) + if (!labels_cleared || (clear_l2arc_header && !header_cleared)) return (-1); return (0); @@ -239,12 +222,10 @@ zpool_clear_label(int fd) static boolean_t find_guid(nvlist_t *nv, uint64_t guid) { - uint64_t tmp; nvlist_t **child; uint_t c, children; - verify(nvlist_lookup_uint64(nv, ZPOOL_CONFIG_GUID, &tmp) == 0); - if (tmp == guid) + if (fnvlist_lookup_uint64(nv, ZPOOL_CONFIG_GUID) == guid) return (B_TRUE); if (nvlist_lookup_nvlist_array(nv, ZPOOL_CONFIG_CHILDREN, @@ -268,18 +249,16 @@ find_aux(zpool_handle_t *zhp, void *data) { aux_cbdata_t *cbp = data; nvlist_t **list; - uint_t i, count; - uint64_t guid; - nvlist_t *nvroot; + uint_t count; - verify(nvlist_lookup_nvlist(zhp->zpool_config, ZPOOL_CONFIG_VDEV_TREE, - &nvroot) == 0); + nvlist_t *nvroot = fnvlist_lookup_nvlist(zhp->zpool_config, + ZPOOL_CONFIG_VDEV_TREE); if (nvlist_lookup_nvlist_array(nvroot, cbp->cb_type, &list, &count) == 0) { - for (i = 0; i < count; i++) { - verify(nvlist_lookup_uint64(list[i], - ZPOOL_CONFIG_GUID, &guid) == 0); + for (uint_t i = 0; i < count; i++) { + uint64_t guid = fnvlist_lookup_uint64(list[i], + ZPOOL_CONFIG_GUID); if (guid == cbp->cb_guid) { cbp->cb_zhp = zhp; return (1); @@ -301,9 +280,9 @@ zpool_in_use(libzfs_handle_t *hdl, int fd, pool_state_t *state, char **namestr, boolean_t *inuse) { nvlist_t *config; - char *name; + char *name = NULL; boolean_t ret; - uint64_t guid, vdev_guid; + uint64_t guid = 0, vdev_guid; zpool_handle_t *zhp; nvlist_t *pool_config; uint64_t stateval, isspare; @@ -320,16 +299,12 @@ zpool_in_use(libzfs_handle_t *hdl, int fd, pool_state_t *state, char **namestr, if (config == NULL) return (0); - verify(nvlist_lookup_uint64(config, ZPOOL_CONFIG_POOL_STATE, - &stateval) == 0); - verify(nvlist_lookup_uint64(config, ZPOOL_CONFIG_GUID, - &vdev_guid) == 0); + stateval = fnvlist_lookup_uint64(config, ZPOOL_CONFIG_POOL_STATE); + vdev_guid = fnvlist_lookup_uint64(config, ZPOOL_CONFIG_GUID); if (stateval != POOL_STATE_SPARE && stateval != POOL_STATE_L2CACHE) { - verify(nvlist_lookup_string(config, ZPOOL_CONFIG_POOL_NAME, - &name) == 0); - verify(nvlist_lookup_uint64(config, ZPOOL_CONFIG_POOL_GUID, - &guid) == 0); + name = fnvlist_lookup_string(config, ZPOOL_CONFIG_POOL_NAME); + guid = fnvlist_lookup_uint64(config, ZPOOL_CONFIG_POOL_GUID); } switch (stateval) { @@ -378,10 +353,8 @@ zpool_in_use(libzfs_handle_t *hdl, int fd, pool_state_t *state, char **namestr, if ((zhp = zpool_open_canfail(hdl, name)) != NULL && (pool_config = zpool_get_config(zhp, NULL)) != NULL) { - nvlist_t *nvroot; - - verify(nvlist_lookup_nvlist(pool_config, - ZPOOL_CONFIG_VDEV_TREE, &nvroot) == 0); + nvlist_t *nvroot = fnvlist_lookup_nvlist( + pool_config, ZPOOL_CONFIG_VDEV_TREE); ret = find_guid(nvroot, vdev_guid); } else { ret = B_FALSE; @@ -453,12 +426,7 @@ zpool_in_use(libzfs_handle_t *hdl, int fd, pool_state_t *state, char **namestr, if (ret) { - if ((*namestr = zfs_strdup(hdl, name)) == NULL) { - if (cb.cb_zhp) - zpool_close(cb.cb_zhp); - nvlist_free(config); - return (-1); - } + *namestr = zfs_strdup(hdl, name); *state = (pool_state_t)stateval; } diff --git a/lib/libzfs/libzfs_iter.c b/lib/libzfs/libzfs_iter.c index 3c537be79487..a716521ab17d 100644 --- a/lib/libzfs/libzfs_iter.c +++ b/lib/libzfs/libzfs_iter.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -28,7 +28,7 @@ #include #include -#include +#include #include #include #include @@ -75,10 +75,7 @@ zfs_do_list_ioctl(zfs_handle_t *zhp, int arg, zfs_cmd_t *zc) switch (errno) { case ENOMEM: /* expand nvlist memory and try again */ - if (zcmd_expand_dst_nvlist(zhp->zfs_hdl, zc) != 0) { - zcmd_free_nvlists(zc); - return (-1); - } + zcmd_expand_dst_nvlist(zhp->zfs_hdl, zc); zc->zc_cookie = orig_cookie; goto top; /* @@ -113,8 +110,7 @@ zfs_iter_filesystems(zfs_handle_t *zhp, zfs_iter_f func, void *data) if (zhp->zfs_type != ZFS_TYPE_FILESYSTEM) return (0); - if (zcmd_alloc_dst_nvlist(zhp->zfs_hdl, &zc, 0) != 0) - return (-1); + zcmd_alloc_dst_nvlist(zhp->zfs_hdl, &zc, 0); while ((ret = zfs_do_list_ioctl(zhp, ZFS_IOC_DATASET_LIST_NEXT, &zc)) == 0) { @@ -154,8 +150,7 @@ zfs_iter_snapshots(zfs_handle_t *zhp, boolean_t simple, zfs_iter_f func, zc.zc_simple = simple; - if (zcmd_alloc_dst_nvlist(zhp->zfs_hdl, &zc, 0) != 0) - return (-1); + zcmd_alloc_dst_nvlist(zhp->zfs_hdl, &zc, 0); if (min_txg != 0) { range_nvl = fnvlist_alloc(); @@ -167,12 +162,8 @@ zfs_iter_snapshots(zfs_handle_t *zhp, boolean_t simple, zfs_iter_f func, fnvlist_add_uint64(range_nvl, SNAP_ITER_MAX_TXG, max_txg); } - if (range_nvl != NULL && - zcmd_write_src_nvlist(zhp->zfs_hdl, &zc, range_nvl) != 0) { - zcmd_free_nvlists(&zc); - fnvlist_free(range_nvl); - return (-1); - } + if (range_nvl != NULL) + zcmd_write_src_nvlist(zhp->zfs_hdl, &zc, range_nvl); while ((ret = zfs_do_list_ioctl(zhp, ZFS_IOC_SNAPSHOT_LIST_NEXT, &zc)) == 0) { diff --git a/lib/libzfs/libzfs_mount.c b/lib/libzfs/libzfs_mount.c index 62c46be9532b..fdfdd8d2808a 100644 --- a/lib/libzfs/libzfs_mount.c +++ b/lib/libzfs/libzfs_mount.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -41,23 +41,13 @@ * zfs_unmount() * zfs_unmountall() * - * This file also contains the functions used to manage sharing filesystems via - * NFS and iSCSI: + * This file also contains the functions used to manage sharing filesystems: * * zfs_is_shared() * zfs_share() * zfs_unshare() - * - * zfs_is_shared_nfs() - * zfs_is_shared_smb() - * zfs_share_proto() - * zfs_shareall(); - * zfs_unshare_nfs() - * zfs_unshare_smb() - * zfs_unshareall_nfs() - * zfs_unshareall_smb() * zfs_unshareall() - * zfs_unshareall_bypath() + * zfs_commit_shares() * * The following functions are available for pool consumers, and will * mount/unmount and share/unshare all datasets within pool: @@ -74,7 +64,7 @@ #include #include #include -#include +#include #include #include #include @@ -95,31 +85,18 @@ static int mount_tp_nthr = 512; /* tpool threads for multi-threaded mounting */ static void zfs_mount_task(void *); -static zfs_share_type_t zfs_is_shared_proto(zfs_handle_t *, char **, - zfs_share_proto_t); -/* - * The share protocols table must be in the same order as the zfs_share_proto_t - * enum in libzfs_impl.h - */ -static const proto_table_t proto_table[PROTO_END] = { - {ZFS_PROP_SHARENFS, "nfs", EZFS_SHARENFSFAILED, EZFS_UNSHARENFSFAILED}, - {ZFS_PROP_SHARESMB, "smb", EZFS_SHARESMBFAILED, EZFS_UNSHARESMBFAILED}, +static const proto_table_t proto_table[SA_PROTOCOL_COUNT] = { + [SA_PROTOCOL_NFS] = + {ZFS_PROP_SHARENFS, EZFS_SHARENFSFAILED, EZFS_UNSHARENFSFAILED}, + [SA_PROTOCOL_SMB] = + {ZFS_PROP_SHARESMB, EZFS_SHARESMBFAILED, EZFS_UNSHARESMBFAILED}, }; -static const zfs_share_proto_t nfs_only[] = { - PROTO_NFS, - PROTO_END -}; - -static const zfs_share_proto_t smb_only[] = { - PROTO_SMB, - PROTO_END -}; -static const zfs_share_proto_t share_all_proto[] = { - PROTO_NFS, - PROTO_SMB, - PROTO_END +static const enum sa_protocol share_all_proto[SA_PROTOCOL_COUNT + 1] = { + SA_PROTOCOL_NFS, + SA_PROTOCOL_SMB, + SA_NO_PROTOCOL }; @@ -260,7 +237,7 @@ zfs_is_mountable_internal(zfs_handle_t *zhp) * Returns true if the given dataset is mountable, false otherwise. Returns the * mountpoint in 'buf'. */ -boolean_t +static boolean_t zfs_is_mountable(zfs_handle_t *zhp, char *buf, size_t buflen, zprop_source_t *source, int flags) { @@ -312,7 +289,7 @@ zfs_is_mountable(zfs_handle_t *zhp, char *buf, size_t buflen, static int zfs_add_option(zfs_handle_t *zhp, char *options, int len, - zfs_prop_t prop, char *on, char *off) + zfs_prop_t prop, const char *on, const char *off) { char *source; uint64_t value; @@ -634,16 +611,16 @@ zfs_unmount(zfs_handle_t *zhp, const char *mountpoint, int flags) /* * Unshare and unmount the filesystem */ - if (zfs_unshare_proto(zhp, mntpt, share_all_proto) != 0) { + if (zfs_unshare(zhp, mntpt, share_all_proto) != 0) { free(mntpt); return (-1); } - zfs_commit_all_shares(); + zfs_commit_shares(NULL); if (unmount_one(zhp, mntpt, flags) != 0) { free(mntpt); - (void) zfs_shareall(zhp); - zfs_commit_all_shares(); + (void) zfs_share(zhp, NULL); + zfs_commit_shares(NULL); return (-1); } @@ -702,58 +679,20 @@ zfs_unmountall(zfs_handle_t *zhp, int flags) return (ret); } -boolean_t -zfs_is_shared(zfs_handle_t *zhp) -{ - zfs_share_type_t rc = 0; - const zfs_share_proto_t *curr_proto; - - if (ZFS_IS_VOLUME(zhp)) - return (B_FALSE); - - for (curr_proto = share_all_proto; *curr_proto != PROTO_END; - curr_proto++) - rc |= zfs_is_shared_proto(zhp, NULL, *curr_proto); - - return (rc ? B_TRUE : B_FALSE); -} - /* * Unshare a filesystem by mountpoint. */ -int +static int unshare_one(libzfs_handle_t *hdl, const char *name, const char *mountpoint, - zfs_share_proto_t proto) + enum sa_protocol proto) { - int err; - - err = sa_disable_share(mountpoint, proto_table[proto].p_name); - if (err != SA_OK) { + int err = sa_disable_share(mountpoint, proto); + if (err != SA_OK) return (zfs_error_fmt(hdl, proto_table[proto].p_unshare_err, dgettext(TEXT_DOMAIN, "cannot unshare '%s': %s"), name, sa_errorstr(err))); - } - return (0); -} -/* - * Query libshare for the given mountpoint and protocol, returning - * a zfs_share_type_t value. - */ -zfs_share_type_t -is_shared(const char *mountpoint, zfs_share_proto_t proto) -{ - if (sa_is_shared(mountpoint, proto_table[proto].p_name)) { - switch (proto) { - case PROTO_NFS: - return (SHARED_NFS); - case PROTO_SMB: - return (SHARED_SMB); - default: - return (SHARED_NOT_SHARED); - } - } - return (SHARED_NOT_SHARED); + return (0); } /* @@ -762,19 +701,22 @@ is_shared(const char *mountpoint, zfs_share_proto_t proto) * on "libshare" to do the dirty work for us. */ int -zfs_share_proto(zfs_handle_t *zhp, const zfs_share_proto_t *proto) +zfs_share(zfs_handle_t *zhp, const enum sa_protocol *proto) { char mountpoint[ZFS_MAXPROPLEN]; char shareopts[ZFS_MAXPROPLEN]; char sourcestr[ZFS_MAXPROPLEN]; - const zfs_share_proto_t *curr_proto; + const enum sa_protocol *curr_proto; zprop_source_t sourcetype; int err = 0; + if (proto == NULL) + proto = share_all_proto; + if (!zfs_is_mountable(zhp, mountpoint, sizeof (mountpoint), NULL, 0)) return (0); - for (curr_proto = proto; *curr_proto != PROTO_END; curr_proto++) { + for (curr_proto = proto; *curr_proto != SA_NO_PROTOCOL; curr_proto++) { /* * Return success if there are no share options. */ @@ -794,7 +736,7 @@ zfs_share_proto(zfs_handle_t *zhp, const zfs_share_proto_t *proto) continue; err = sa_enable_share(zfs_get_name(zhp), mountpoint, shareopts, - proto_table[*curr_proto].p_name); + *curr_proto); if (err != SA_OK) { return (zfs_error_fmt(zhp->zfs_hdl, proto_table[*curr_proto].p_share_err, @@ -806,188 +748,88 @@ zfs_share_proto(zfs_handle_t *zhp, const zfs_share_proto_t *proto) return (0); } -int -zfs_share(zfs_handle_t *zhp) -{ - assert(!ZFS_IS_VOLUME(zhp)); - return (zfs_share_proto(zhp, share_all_proto)); -} - -int -zfs_unshare(zfs_handle_t *zhp) -{ - assert(!ZFS_IS_VOLUME(zhp)); - return (zfs_unshareall(zhp)); -} - /* * Check to see if the filesystem is currently shared. */ -static zfs_share_type_t -zfs_is_shared_proto(zfs_handle_t *zhp, char **where, zfs_share_proto_t proto) -{ - char *mountpoint; - zfs_share_type_t rc; - - if (!zfs_is_mounted(zhp, &mountpoint)) - return (SHARED_NOT_SHARED); - - if ((rc = is_shared(mountpoint, proto)) - != SHARED_NOT_SHARED) { - if (where != NULL) - *where = mountpoint; - else - free(mountpoint); - return (rc); - } else { - free(mountpoint); - return (SHARED_NOT_SHARED); - } -} - -boolean_t -zfs_is_shared_nfs(zfs_handle_t *zhp, char **where) -{ - return (zfs_is_shared_proto(zhp, where, - PROTO_NFS) != SHARED_NOT_SHARED); -} - boolean_t -zfs_is_shared_smb(zfs_handle_t *zhp, char **where) -{ - return (zfs_is_shared_proto(zhp, where, - PROTO_SMB) != SHARED_NOT_SHARED); -} - -/* - * zfs_parse_options(options, proto) - * - * Call the legacy parse interface to get the protocol specific - * options using the NULL arg to indicate that this is a "parse" only. - */ -int -zfs_parse_options(char *options, zfs_share_proto_t proto) +zfs_is_shared(zfs_handle_t *zhp, char **where, + const enum sa_protocol *proto) { - return (sa_validate_shareopts(options, proto_table[proto].p_name)); -} + char *mountpoint; + if (proto == NULL) + proto = share_all_proto; -void -zfs_commit_proto(const zfs_share_proto_t *proto) -{ - const zfs_share_proto_t *curr_proto; - for (curr_proto = proto; *curr_proto != PROTO_END; curr_proto++) - sa_commit_shares(proto_table[*curr_proto].p_name); -} + if (ZFS_IS_VOLUME(zhp)) + return (B_FALSE); -void -zfs_commit_nfs_shares(void) -{ - zfs_commit_proto(nfs_only); -} + if (!zfs_is_mounted(zhp, &mountpoint)) + return (B_FALSE); -void -zfs_commit_smb_shares(void) -{ - zfs_commit_proto(smb_only); -} + for (const enum sa_protocol *p = proto; *p != SA_NO_PROTOCOL; ++p) + if (sa_is_shared(mountpoint, *p)) { + if (where != NULL) + *where = mountpoint; + else + free(mountpoint); + return (B_TRUE); + } -void -zfs_commit_all_shares(void) -{ - zfs_commit_proto(share_all_proto); + free(mountpoint); + return (B_FALSE); } void -zfs_commit_shares(const char *proto) +zfs_commit_shares(const enum sa_protocol *proto) { if (proto == NULL) - zfs_commit_proto(share_all_proto); - else if (strcmp(proto, "nfs") == 0) - zfs_commit_proto(nfs_only); - else if (strcmp(proto, "smb") == 0) - zfs_commit_proto(smb_only); -} + proto = share_all_proto; -int -zfs_share_nfs(zfs_handle_t *zhp) -{ - return (zfs_share_proto(zhp, nfs_only)); -} - -int -zfs_share_smb(zfs_handle_t *zhp) -{ - return (zfs_share_proto(zhp, smb_only)); -} - -int -zfs_shareall(zfs_handle_t *zhp) -{ - return (zfs_share_proto(zhp, share_all_proto)); + for (const enum sa_protocol *p = proto; *p != SA_NO_PROTOCOL; ++p) + sa_commit_shares(*p); } /* * Unshare the given filesystem. */ int -zfs_unshare_proto(zfs_handle_t *zhp, const char *mountpoint, - const zfs_share_proto_t *proto) +zfs_unshare(zfs_handle_t *zhp, const char *mountpoint, + const enum sa_protocol *proto) { libzfs_handle_t *hdl = zhp->zfs_hdl; struct mnttab entry; - char *mntpt = NULL; - /* check to see if need to unmount the filesystem */ - if (mountpoint != NULL) - mntpt = zfs_strdup(hdl, mountpoint); + if (proto == NULL) + proto = share_all_proto; if (mountpoint != NULL || ((zfs_get_type(zhp) == ZFS_TYPE_FILESYSTEM) && libzfs_mnttab_find(hdl, zfs_get_name(zhp), &entry) == 0)) { - const zfs_share_proto_t *curr_proto; - - if (mountpoint == NULL) - mntpt = zfs_strdup(zhp->zfs_hdl, entry.mnt_mountp); - for (curr_proto = proto; *curr_proto != PROTO_END; - curr_proto++) { + /* check to see if need to unmount the filesystem */ + const char *mntpt = mountpoint ?: entry.mnt_mountp; - if (is_shared(mntpt, *curr_proto)) { - if (unshare_one(hdl, zhp->zfs_name, - mntpt, *curr_proto) != 0) { - if (mntpt != NULL) - free(mntpt); + for (const enum sa_protocol *curr_proto = proto; + *curr_proto != SA_NO_PROTOCOL; curr_proto++) + if (sa_is_shared(mntpt, *curr_proto) && + unshare_one(hdl, zhp->zfs_name, + mntpt, *curr_proto) != 0) return (-1); - } - } - } } - if (mntpt != NULL) - free(mntpt); return (0); } -int -zfs_unshare_nfs(zfs_handle_t *zhp, const char *mountpoint) -{ - return (zfs_unshare_proto(zhp, mountpoint, nfs_only)); -} - -int -zfs_unshare_smb(zfs_handle_t *zhp, const char *mountpoint) -{ - return (zfs_unshare_proto(zhp, mountpoint, smb_only)); -} - /* * Same as zfs_unmountall(), but for NFS and SMB unshares. */ -static int -zfs_unshareall_proto(zfs_handle_t *zhp, const zfs_share_proto_t *proto) +int +zfs_unshareall(zfs_handle_t *zhp, const enum sa_protocol *proto) { prop_changelist_t *clp; int ret; + if (proto == NULL) + proto = share_all_proto; + clp = changelist_gather(zhp, ZFS_PROP_SHARENFS, 0, 0); if (clp == NULL) return (-1); @@ -998,44 +840,6 @@ zfs_unshareall_proto(zfs_handle_t *zhp, const zfs_share_proto_t *proto) return (ret); } -int -zfs_unshareall_nfs(zfs_handle_t *zhp) -{ - return (zfs_unshareall_proto(zhp, nfs_only)); -} - -int -zfs_unshareall_smb(zfs_handle_t *zhp) -{ - return (zfs_unshareall_proto(zhp, smb_only)); -} - -int -zfs_unshareall(zfs_handle_t *zhp) -{ - return (zfs_unshareall_proto(zhp, share_all_proto)); -} - -int -zfs_unshareall_bypath(zfs_handle_t *zhp, const char *mountpoint) -{ - return (zfs_unshare_proto(zhp, mountpoint, share_all_proto)); -} - -int -zfs_unshareall_bytype(zfs_handle_t *zhp, const char *mountpoint, - const char *proto) -{ - if (proto == NULL) - return (zfs_unshare_proto(zhp, mountpoint, share_all_proto)); - if (strcmp(proto, "nfs") == 0) - return (zfs_unshare_proto(zhp, mountpoint, nfs_only)); - else if (strcmp(proto, "smb") == 0) - return (zfs_unshare_proto(zhp, mountpoint, smb_only)); - else - return (1); -} - /* * Remove the mountpoint associated with the current dataset, if necessary. * We only remove the underlying directory if: @@ -1326,7 +1130,7 @@ zfs_share_one(zfs_handle_t *zhp, void *arg) mount_state_t *ms = arg; int ret = 0; - if (zfs_share(zhp) != 0) + if (zfs_share(zhp, NULL) != 0) ret = ms->ms_mntstatus = -1; return (ret); } @@ -1499,7 +1303,7 @@ zpool_enable_datasets(zpool_handle_t *zhp, const char *mntopts, int flags) if (ms.ms_mntstatus != 0) ret = ms.ms_mntstatus; else - zfs_commit_all_shares(); + zfs_commit_shares(NULL); out: for (int i = 0; i < cb.cb_used; i++) @@ -1572,29 +1376,19 @@ zpool_disable_datasets(zpool_handle_t *zhp, boolean_t force) */ if (used == alloc) { if (alloc == 0) { - - if ((sets = zfs_alloc(hdl, - 8 * sizeof (struct sets_s))) == NULL) - goto out; - + sets = zfs_alloc(hdl, + 8 * sizeof (struct sets_s)); alloc = 8; } else { - void *ptr; - - if ((ptr = zfs_realloc(hdl, sets, + sets = zfs_realloc(hdl, sets, alloc * sizeof (struct sets_s), - alloc * 2 * sizeof (struct sets_s))) - == NULL) - goto out; - sets = ptr; + alloc * 2 * sizeof (struct sets_s)); alloc *= 2; } } - if ((sets[used].mountpoint = zfs_strdup(hdl, - entry.mnt_mountp)) == NULL) - goto out; + sets[used].mountpoint = zfs_strdup(hdl, entry.mnt_mountp); /* * This is allowed to fail, in case there is some I/O error. It @@ -1618,16 +1412,14 @@ zpool_disable_datasets(zpool_handle_t *zhp, boolean_t force) * Walk through and first unshare everything. */ for (i = 0; i < used; i++) { - const zfs_share_proto_t *curr_proto; - for (curr_proto = share_all_proto; *curr_proto != PROTO_END; - curr_proto++) { - if (is_shared(sets[i].mountpoint, *curr_proto) && + for (enum sa_protocol i = 0; i < SA_PROTOCOL_COUNT; ++i) { + if (sa_is_shared(sets[i].mountpoint, i) && unshare_one(hdl, sets[i].mountpoint, - sets[i].mountpoint, *curr_proto) != 0) + sets[i].mountpoint, i) != 0) goto out; } } - zfs_commit_all_shares(); + zfs_commit_shares(NULL); /* * Now unmount everything, removing the underlying directories as diff --git a/lib/libzfs/libzfs_pool.c b/lib/libzfs/libzfs_pool.c index 7fafeae9e5a8..877eb2be0ada 100644 --- a/lib/libzfs/libzfs_pool.c +++ b/lib/libzfs/libzfs_pool.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -79,16 +79,12 @@ zpool_get_all_props(zpool_handle_t *zhp) (void) strlcpy(zc.zc_name, zhp->zpool_name, sizeof (zc.zc_name)); - if (zcmd_alloc_dst_nvlist(hdl, &zc, 0) != 0) - return (-1); + zcmd_alloc_dst_nvlist(hdl, &zc, 0); while (zfs_ioctl(hdl, ZFS_IOC_POOL_GET_PROPS, &zc) != 0) { - if (errno == ENOMEM) { - if (zcmd_expand_dst_nvlist(hdl, &zc) != 0) { - zcmd_free_nvlists(&zc); - return (-1); - } - } else { + if (errno == ENOMEM) + zcmd_expand_dst_nvlist(hdl, &zc); + else { zcmd_free_nvlists(&zc); return (-1); } @@ -123,18 +119,16 @@ zpool_get_prop_string(zpool_handle_t *zhp, zpool_prop_t prop, zprop_source_t *src) { nvlist_t *nv, *nvl; - uint64_t ival; - char *value; + const char *value; zprop_source_t source; nvl = zhp->zpool_props; if (nvlist_lookup_nvlist(nvl, zpool_prop_to_name(prop), &nv) == 0) { - verify(nvlist_lookup_uint64(nv, ZPROP_SOURCE, &ival) == 0); - source = ival; - verify(nvlist_lookup_string(nv, ZPROP_VALUE, &value) == 0); + source = fnvlist_lookup_uint64(nv, ZPROP_SOURCE); + value = fnvlist_lookup_string(nv, ZPROP_VALUE); } else { source = ZPROP_SRC_DEFAULT; - if ((value = (char *)zpool_prop_default_string(prop)) == NULL) + if ((value = zpool_prop_default_string(prop)) == NULL) value = "-"; } @@ -169,9 +163,8 @@ zpool_get_prop_int(zpool_handle_t *zhp, zpool_prop_t prop, zprop_source_t *src) nvl = zhp->zpool_props; if (nvlist_lookup_nvlist(nvl, zpool_prop_to_name(prop), &nv) == 0) { - verify(nvlist_lookup_uint64(nv, ZPROP_SOURCE, &value) == 0); - source = value; - verify(nvlist_lookup_uint64(nv, ZPROP_VALUE, &value) == 0); + source = fnvlist_lookup_uint64(nv, ZPROP_SOURCE); + value = fnvlist_lookup_uint64(nv, ZPROP_VALUE); } else { source = ZPROP_SRC_DEFAULT; value = zpool_prop_default_numeric(prop); @@ -255,9 +248,6 @@ zpool_get_state_str(zpool_handle_t *zhp) { zpool_errata_t errata; zpool_status_t status; - nvlist_t *nvroot; - vdev_stat_t *vs; - uint_t vsc; const char *str; status = zpool_get_status(zhp, NULL, &errata); @@ -268,11 +258,11 @@ zpool_get_state_str(zpool_handle_t *zhp) status == ZPOOL_STATUS_IO_FAILURE_MMP) { str = gettext("SUSPENDED"); } else { - verify(nvlist_lookup_nvlist(zpool_get_config(zhp, NULL), - ZPOOL_CONFIG_VDEV_TREE, &nvroot) == 0); - verify(nvlist_lookup_uint64_array(nvroot, - ZPOOL_CONFIG_VDEV_STATS, (uint64_t **)&vs, &vsc) - == 0); + nvlist_t *nvroot = fnvlist_lookup_nvlist( + zpool_get_config(zhp, NULL), ZPOOL_CONFIG_VDEV_TREE); + uint_t vsc; + vdev_stat_t *vs = (vdev_stat_t *)fnvlist_lookup_uint64_array( + nvroot, ZPOOL_CONFIG_VDEV_STATS, &vsc); str = zpool_state_to_name(vs->vs_state, vs->vs_aux); } return (str); @@ -347,6 +337,8 @@ zpool_get_prop(zpool_handle_t *zhp, zpool_prop_t prop, char *buf, case ZPOOL_PROP_FREEING: case ZPOOL_PROP_LEAKED: case ZPOOL_PROP_ASHIFT: + case ZPOOL_PROP_MAXBLOCKSIZE: + case ZPOOL_PROP_MAXDNODESIZE: if (literal) (void) snprintf(buf, len, "%llu", (u_longlong_t)intval); @@ -784,7 +776,7 @@ zpool_set_prop(zpool_handle_t *zhp, const char *propname, const char *propval) { zfs_cmd_t zc = {"\0"}; int ret = -1; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; nvlist_t *nvl = NULL; nvlist_t *realprops; uint64_t version; @@ -817,10 +809,7 @@ zpool_set_prop(zpool_handle_t *zhp, const char *propname, const char *propval) */ (void) strlcpy(zc.zc_name, zhp->zpool_name, sizeof (zc.zc_name)); - if (zcmd_write_src_nvlist(zhp->zpool_hdl, &zc, nvl) != 0) { - nvlist_free(nvl); - return (-1); - } + zcmd_write_src_nvlist(zhp->zpool_hdl, &zc, nvl); ret = zfs_ioctl(zhp->zpool_hdl, ZFS_IOC_POOL_SET_PROPS, &zc); @@ -865,7 +854,7 @@ zpool_expand_proplist(zpool_handle_t *zhp, zprop_list_t **plp, for (i = 0; i < SPA_FEATURES; i++) { zprop_list_t *entry = zfs_alloc(hdl, sizeof (zprop_list_t)); - entry->pl_prop = ZPROP_INVAL; + entry->pl_prop = ZPROP_USERPROP; entry->pl_user_prop = zfs_asprintf(hdl, "feature@%s", spa_feature_table[i].fi_uname); entry->pl_width = strlen(entry->pl_user_prop); @@ -909,7 +898,7 @@ zpool_expand_proplist(zpool_handle_t *zhp, zprop_list_t **plp, } entry = zfs_alloc(hdl, sizeof (zprop_list_t)); - entry->pl_prop = ZPROP_INVAL; + entry->pl_prop = ZPROP_USERPROP; entry->pl_user_prop = propname; entry->pl_width = strlen(entry->pl_user_prop); entry->pl_all = B_TRUE; @@ -922,7 +911,7 @@ zpool_expand_proplist(zpool_handle_t *zhp, zprop_list_t **plp, if (entry->pl_fixed && !literal) continue; - if (entry->pl_prop != ZPROP_INVAL && + if (entry->pl_prop != ZPROP_USERPROP && zpool_get_prop(zhp, entry->pl_prop, buf, sizeof (buf), NULL, literal) == 0) { if (strlen(buf) > entry->pl_width) @@ -978,19 +967,16 @@ vdev_expand_proplist(zpool_handle_t *zhp, const char *vdevname, /* Skip properties that are not user defined */ if ((prop = vdev_name_to_prop(propname)) != - VDEV_PROP_USER) + VDEV_PROP_USERPROP) continue; if (nvpair_value_nvlist(elem, &propval) != 0) continue; - verify(nvlist_lookup_string(propval, ZPROP_VALUE, - &strval) == 0); - - if ((entry = zfs_alloc(zhp->zpool_hdl, - sizeof (zprop_list_t))) == NULL) - return (ENOMEM); + strval = fnvlist_lookup_string(propval, ZPROP_VALUE); + entry = zfs_alloc(zhp->zpool_hdl, + sizeof (zprop_list_t)); entry->pl_prop = prop; entry->pl_user_prop = zfs_strdup(zhp->zpool_hdl, propname); @@ -1184,8 +1170,7 @@ zpool_open_canfail(libzfs_handle_t *hdl, const char *pool) return (NULL); } - if ((zhp = zfs_alloc(hdl, sizeof (zpool_handle_t))) == NULL) - return (NULL); + zhp = zfs_alloc(hdl, sizeof (zpool_handle_t)); zhp->zpool_hdl = hdl; (void) strlcpy(zhp->zpool_name, pool, sizeof (zhp->zpool_name)); @@ -1216,8 +1201,7 @@ zpool_open_silent(libzfs_handle_t *hdl, const char *pool, zpool_handle_t **ret) zpool_handle_t *zhp; boolean_t missing; - if ((zhp = zfs_alloc(hdl, sizeof (zpool_handle_t))) == NULL) - return (-1); + zhp = zfs_alloc(hdl, sizeof (zpool_handle_t)); zhp->zpool_hdl = hdl; (void) strlcpy(zhp->zpool_name, pool, sizeof (zhp->zpool_name)); @@ -1384,23 +1368,22 @@ zpool_create(libzfs_handle_t *hdl, const char *pool, nvlist_t *nvroot, nvlist_t *hidden_args = NULL; uint8_t *wkeydata = NULL; uint_t wkeylen = 0; - char msg[1024]; + char errbuf[ERRBUFLEN]; int ret = -1; - (void) snprintf(msg, sizeof (msg), dgettext(TEXT_DOMAIN, + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot create '%s'"), pool); if (!zpool_name_valid(hdl, B_FALSE, pool)) - return (zfs_error(hdl, EZFS_INVALIDNAME, msg)); + return (zfs_error(hdl, EZFS_INVALIDNAME, errbuf)); - if (zcmd_write_conf_nvlist(hdl, &zc, nvroot) != 0) - return (-1); + zcmd_write_conf_nvlist(hdl, &zc, nvroot); if (props) { prop_flags_t flags = { .create = B_TRUE, .import = B_FALSE }; if ((zc_props = zpool_valid_proplist(hdl, pool, props, - SPA_VERSION_1, flags, msg)) == NULL) { + SPA_VERSION_1, flags, errbuf)) == NULL) { goto create_failed; } } @@ -1414,7 +1397,7 @@ zpool_create(libzfs_handle_t *hdl, const char *pool, nvlist_t *nvroot, strcmp(zonestr, "on") == 0); if ((zc_fsprops = zfs_valid_proplist(hdl, ZFS_TYPE_FILESYSTEM, - fsprops, zoned, NULL, NULL, B_TRUE, msg)) == NULL) { + fsprops, zoned, NULL, NULL, B_TRUE, errbuf)) == NULL) { goto create_failed; } @@ -1424,7 +1407,7 @@ zpool_create(libzfs_handle_t *hdl, const char *pool, nvlist_t *nvroot, zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "%s property requires a special vdev"), zfs_prop_to_name(ZFS_PROP_SPECIAL_SMALL_BLOCKS)); - (void) zfs_error(hdl, EZFS_BADPROP, msg); + (void) zfs_error(hdl, EZFS_BADPROP, errbuf); goto create_failed; } @@ -1434,7 +1417,7 @@ zpool_create(libzfs_handle_t *hdl, const char *pool, nvlist_t *nvroot, } if (zfs_crypto_create(hdl, NULL, zc_fsprops, props, B_TRUE, &wkeydata, &wkeylen) != 0) { - zfs_error(hdl, EZFS_CRYPTOFAILED, msg); + zfs_error(hdl, EZFS_CRYPTOFAILED, errbuf); goto create_failed; } if (nvlist_add_nvlist(zc_props, @@ -1455,8 +1438,8 @@ zpool_create(libzfs_handle_t *hdl, const char *pool, nvlist_t *nvroot, } } - if (zc_props && zcmd_write_src_nvlist(hdl, &zc, zc_props) != 0) - goto create_failed; + if (zc_props) + zcmd_write_src_nvlist(hdl, &zc, zc_props); (void) strlcpy(zc.zc_name, pool, sizeof (zc.zc_name)); @@ -1482,7 +1465,7 @@ zpool_create(libzfs_handle_t *hdl, const char *pool, nvlist_t *nvroot, "one or more vdevs refer to the same device, or " "one of\nthe devices is part of an active md or " "lvm device")); - return (zfs_error(hdl, EZFS_BADDEV, msg)); + return (zfs_error(hdl, EZFS_BADDEV, errbuf)); case ERANGE: /* @@ -1497,7 +1480,7 @@ zpool_create(libzfs_handle_t *hdl, const char *pool, nvlist_t *nvroot, */ zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "record size invalid")); - return (zfs_error(hdl, EZFS_BADPROP, msg)); + return (zfs_error(hdl, EZFS_BADPROP, errbuf)); case EOVERFLOW: /* @@ -1516,12 +1499,12 @@ zpool_create(libzfs_handle_t *hdl, const char *pool, nvlist_t *nvroot, "one or more devices is less than the " "minimum size (%s)"), buf); } - return (zfs_error(hdl, EZFS_BADDEV, msg)); + return (zfs_error(hdl, EZFS_BADDEV, errbuf)); case ENOSPC: zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "one or more devices is out of space")); - return (zfs_error(hdl, EZFS_BADDEV, msg)); + return (zfs_error(hdl, EZFS_BADDEV, errbuf)); case EINVAL: if (zpool_has_draid_vdev(nvroot) && @@ -1529,13 +1512,14 @@ zpool_create(libzfs_handle_t *hdl, const char *pool, nvlist_t *nvroot, zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "dRAID vdevs are unsupported by the " "kernel")); - return (zfs_error(hdl, EZFS_BADDEV, msg)); + return (zfs_error(hdl, EZFS_BADDEV, errbuf)); } else { - return (zpool_standard_error(hdl, errno, msg)); + return (zpool_standard_error(hdl, errno, + errbuf)); } default: - return (zpool_standard_error(hdl, errno, msg)); + return (zpool_standard_error(hdl, errno, errbuf)); } } @@ -1559,7 +1543,7 @@ zpool_destroy(zpool_handle_t *zhp, const char *log_str) zfs_cmd_t zc = {"\0"}; zfs_handle_t *zfp = NULL; libzfs_handle_t *hdl = zhp->zpool_hdl; - char msg[1024]; + char errbuf[ERRBUFLEN]; if (zhp->zpool_state == POOL_STATE_ACTIVE && (zfp = zfs_open(hdl, zhp->zpool_name, ZFS_TYPE_FILESYSTEM)) == NULL) @@ -1569,15 +1553,15 @@ zpool_destroy(zpool_handle_t *zhp, const char *log_str) zc.zc_history = (uint64_t)(uintptr_t)log_str; if (zfs_ioctl(hdl, ZFS_IOC_POOL_DESTROY, &zc) != 0) { - (void) snprintf(msg, sizeof (msg), dgettext(TEXT_DOMAIN, + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot destroy '%s'"), zhp->zpool_name); if (errno == EROFS) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "one or more devices is read only")); - (void) zfs_error(hdl, EZFS_BADDEV, msg); + (void) zfs_error(hdl, EZFS_BADDEV, errbuf); } else { - (void) zpool_standard_error(hdl, errno, msg); + (void) zpool_standard_error(hdl, errno, errbuf); } if (zfp) @@ -1600,14 +1584,14 @@ int zpool_checkpoint(zpool_handle_t *zhp) { libzfs_handle_t *hdl = zhp->zpool_hdl; - char msg[1024]; + char errbuf[ERRBUFLEN]; int error; error = lzc_pool_checkpoint(zhp->zpool_name); if (error != 0) { - (void) snprintf(msg, sizeof (msg), dgettext(TEXT_DOMAIN, + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot checkpoint '%s'"), zhp->zpool_name); - (void) zpool_standard_error(hdl, error, msg); + (void) zpool_standard_error(hdl, error, errbuf); return (-1); } @@ -1621,14 +1605,14 @@ int zpool_discard_checkpoint(zpool_handle_t *zhp) { libzfs_handle_t *hdl = zhp->zpool_hdl; - char msg[1024]; + char errbuf[ERRBUFLEN]; int error; error = lzc_pool_checkpoint_discard(zhp->zpool_name); if (error != 0) { - (void) snprintf(msg, sizeof (msg), dgettext(TEXT_DOMAIN, + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot discard checkpoint in '%s'"), zhp->zpool_name); - (void) zpool_standard_error(hdl, error, msg); + (void) zpool_standard_error(hdl, error, errbuf); return (-1); } @@ -1645,11 +1629,11 @@ zpool_add(zpool_handle_t *zhp, nvlist_t *nvroot) zfs_cmd_t zc = {"\0"}; int ret; libzfs_handle_t *hdl = zhp->zpool_hdl; - char msg[1024]; + char errbuf[ERRBUFLEN]; nvlist_t **spares, **l2cache; uint_t nspares, nl2cache; - (void) snprintf(msg, sizeof (msg), dgettext(TEXT_DOMAIN, + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot add to '%s'"), zhp->zpool_name); if (zpool_get_prop_int(zhp, ZPOOL_PROP_VERSION, NULL) < @@ -1658,7 +1642,7 @@ zpool_add(zpool_handle_t *zhp, nvlist_t *nvroot) &spares, &nspares) == 0) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "pool must be " "upgraded to add hot spares")); - return (zfs_error(hdl, EZFS_BADVERSION, msg)); + return (zfs_error(hdl, EZFS_BADVERSION, errbuf)); } if (zpool_get_prop_int(zhp, ZPOOL_PROP_VERSION, NULL) < @@ -1667,11 +1651,10 @@ zpool_add(zpool_handle_t *zhp, nvlist_t *nvroot) &l2cache, &nl2cache) == 0) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "pool must be " "upgraded to add cache devices")); - return (zfs_error(hdl, EZFS_BADVERSION, msg)); + return (zfs_error(hdl, EZFS_BADVERSION, errbuf)); } - if (zcmd_write_conf_nvlist(hdl, &zc, nvroot) != 0) - return (-1); + zcmd_write_conf_nvlist(hdl, &zc, nvroot); (void) strlcpy(zc.zc_name, zhp->zpool_name, sizeof (zc.zc_name)); if (zfs_ioctl(hdl, ZFS_IOC_VDEV_ADD, &zc) != 0) { @@ -1685,7 +1668,7 @@ zpool_add(zpool_handle_t *zhp, nvlist_t *nvroot) */ zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "one or more vdevs refer to the same device")); - (void) zfs_error(hdl, EZFS_BADDEV, msg); + (void) zfs_error(hdl, EZFS_BADDEV, errbuf); break; case EINVAL: @@ -1702,7 +1685,7 @@ zpool_add(zpool_handle_t *zhp, nvlist_t *nvroot) "raidz or dRAID vdevs")); } - (void) zfs_error(hdl, EZFS_BADDEV, msg); + (void) zfs_error(hdl, EZFS_BADDEV, errbuf); break; case EOVERFLOW: @@ -1722,17 +1705,17 @@ zpool_add(zpool_handle_t *zhp, nvlist_t *nvroot) "device is less than the minimum " "size (%s)"), buf); } - (void) zfs_error(hdl, EZFS_BADDEV, msg); + (void) zfs_error(hdl, EZFS_BADDEV, errbuf); break; case ENOTSUP: zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "pool must be upgraded to add these vdevs")); - (void) zfs_error(hdl, EZFS_BADVERSION, msg); + (void) zfs_error(hdl, EZFS_BADVERSION, errbuf); break; default: - (void) zpool_standard_error(hdl, errno, msg); + (void) zpool_standard_error(hdl, errno, errbuf); } ret = -1; @@ -1994,20 +1977,13 @@ void zpool_print_unsup_feat(nvlist_t *config) { nvlist_t *nvinfo, *unsup_feat; - nvpair_t *nvp; - verify(nvlist_lookup_nvlist(config, ZPOOL_CONFIG_LOAD_INFO, &nvinfo) == - 0); - verify(nvlist_lookup_nvlist(nvinfo, ZPOOL_CONFIG_UNSUP_FEAT, - &unsup_feat) == 0); - - for (nvp = nvlist_next_nvpair(unsup_feat, NULL); nvp != NULL; - nvp = nvlist_next_nvpair(unsup_feat, nvp)) { - char *desc; - - verify(nvpair_type(nvp) == DATA_TYPE_STRING); - verify(nvpair_value_string(nvp, &desc) == 0); + nvinfo = fnvlist_lookup_nvlist(config, ZPOOL_CONFIG_LOAD_INFO); + unsup_feat = fnvlist_lookup_nvlist(nvinfo, ZPOOL_CONFIG_UNSUP_FEAT); + for (nvpair_t *nvp = nvlist_next_nvpair(unsup_feat, NULL); + nvp != NULL; nvp = nvlist_next_nvpair(unsup_feat, nvp)) { + char *desc = fnvpair_value_string(nvp); if (strlen(desc) > 0) (void) printf("\t%s (%s)\n", nvpair_name(nvp), desc); else @@ -2034,10 +2010,9 @@ zpool_import_props(libzfs_handle_t *hdl, nvlist_t *config, const char *newname, char *origname; int ret; int error = 0; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; - verify(nvlist_lookup_string(config, ZPOOL_CONFIG_POOL_NAME, - &origname) == 0); + origname = fnvlist_lookup_string(config, ZPOOL_CONFIG_POOL_NAME); (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot import pool '%s'"), origname); @@ -2056,41 +2031,26 @@ zpool_import_props(libzfs_handle_t *hdl, nvlist_t *config, const char *newname, uint64_t version; prop_flags_t flags = { .create = B_FALSE, .import = B_TRUE }; - verify(nvlist_lookup_uint64(config, ZPOOL_CONFIG_VERSION, - &version) == 0); + version = fnvlist_lookup_uint64(config, ZPOOL_CONFIG_VERSION); if ((props = zpool_valid_proplist(hdl, origname, props, version, flags, errbuf)) == NULL) return (-1); - if (zcmd_write_src_nvlist(hdl, &zc, props) != 0) { - nvlist_free(props); - return (-1); - } + zcmd_write_src_nvlist(hdl, &zc, props); nvlist_free(props); } (void) strlcpy(zc.zc_name, thename, sizeof (zc.zc_name)); - verify(nvlist_lookup_uint64(config, ZPOOL_CONFIG_POOL_GUID, - &zc.zc_guid) == 0); + zc.zc_guid = fnvlist_lookup_uint64(config, ZPOOL_CONFIG_POOL_GUID); - if (zcmd_write_conf_nvlist(hdl, &zc, config) != 0) { - zcmd_free_nvlists(&zc); - return (-1); - } - if (zcmd_alloc_dst_nvlist(hdl, &zc, zc.zc_nvlist_conf_size * 2) != 0) { - zcmd_free_nvlists(&zc); - return (-1); - } + zcmd_write_conf_nvlist(hdl, &zc, config); + zcmd_alloc_dst_nvlist(hdl, &zc, zc.zc_nvlist_conf_size * 2); zc.zc_cookie = flags; while ((ret = zfs_ioctl(hdl, ZFS_IOC_POOL_IMPORT, &zc)) != 0 && - errno == ENOMEM) { - if (zcmd_expand_dst_nvlist(hdl, &zc) != 0) { - zcmd_free_nvlists(&zc); - return (-1); - } - } + errno == ENOMEM) + zcmd_expand_dst_nvlist(hdl, &zc); if (ret != 0) error = errno; @@ -2151,7 +2111,7 @@ zpool_import_props(libzfs_handle_t *hdl, nvlist_t *config, const char *newname, case EREMOTEIO: if (nv != NULL && nvlist_lookup_nvlist(nv, ZPOOL_CONFIG_LOAD_INFO, &nvinfo) == 0) { - char *hostname = ""; + const char *hostname = ""; uint64_t hostid = 0; mmp_state_t mmp_state; @@ -2557,11 +2517,11 @@ zpool_trim(zpool_handle_t *zhp, pool_trim_func_t cmd_type, nvlist_t *vds, goto out; } } else { - char msg[1024]; + char errbuf[ERRBUFLEN]; - (void) snprintf(msg, sizeof (msg), + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "operation failed")); - zpool_standard_error(zhp->zpool_hdl, err, msg); + zpool_standard_error(zhp->zpool_hdl, err, errbuf); retval = -1; goto out; } @@ -2586,7 +2546,7 @@ int zpool_scan(zpool_handle_t *zhp, pool_scan_func_t func, pool_scrub_cmd_t cmd) { zfs_cmd_t zc = {"\0"}; - char msg[1024]; + char errbuf[ERRBUFLEN]; int err; libzfs_handle_t *hdl = zhp->zpool_hdl; @@ -2609,21 +2569,22 @@ zpool_scan(zpool_handle_t *zhp, pool_scan_func_t func, pool_scrub_cmd_t cmd) if (func == POOL_SCAN_SCRUB) { if (cmd == POOL_SCRUB_PAUSE) { - (void) snprintf(msg, sizeof (msg), dgettext(TEXT_DOMAIN, - "cannot pause scrubbing %s"), zc.zc_name); + (void) snprintf(errbuf, sizeof (errbuf), + dgettext(TEXT_DOMAIN, "cannot pause scrubbing %s"), + zc.zc_name); } else { assert(cmd == POOL_SCRUB_NORMAL); - (void) snprintf(msg, sizeof (msg), dgettext(TEXT_DOMAIN, - "cannot scrub %s"), zc.zc_name); + (void) snprintf(errbuf, sizeof (errbuf), + dgettext(TEXT_DOMAIN, "cannot scrub %s"), + zc.zc_name); } } else if (func == POOL_SCAN_RESILVER) { assert(cmd == POOL_SCRUB_NORMAL); - (void) snprintf(msg, sizeof (msg), dgettext(TEXT_DOMAIN, + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot restart resilver on %s"), zc.zc_name); } else if (func == POOL_SCAN_NONE) { - (void) snprintf(msg, sizeof (msg), - dgettext(TEXT_DOMAIN, "cannot cancel scrubbing %s"), - zc.zc_name); + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, + "cannot cancel scrubbing %s"), zc.zc_name); } else { assert(!"unexpected result"); } @@ -2633,25 +2594,26 @@ zpool_scan(zpool_handle_t *zhp, pool_scan_func_t func, pool_scrub_cmd_t cmd) pool_scan_stat_t *ps = NULL; uint_t psc; - verify(nvlist_lookup_nvlist(zhp->zpool_config, - ZPOOL_CONFIG_VDEV_TREE, &nvroot) == 0); + nvroot = fnvlist_lookup_nvlist(zhp->zpool_config, + ZPOOL_CONFIG_VDEV_TREE); (void) nvlist_lookup_uint64_array(nvroot, ZPOOL_CONFIG_SCAN_STATS, (uint64_t **)&ps, &psc); if (ps && ps->pss_func == POOL_SCAN_SCRUB && ps->pss_state == DSS_SCANNING) { if (cmd == POOL_SCRUB_PAUSE) - return (zfs_error(hdl, EZFS_SCRUB_PAUSED, msg)); + return (zfs_error(hdl, EZFS_SCRUB_PAUSED, + errbuf)); else - return (zfs_error(hdl, EZFS_SCRUBBING, msg)); + return (zfs_error(hdl, EZFS_SCRUBBING, errbuf)); } else { - return (zfs_error(hdl, EZFS_RESILVERING, msg)); + return (zfs_error(hdl, EZFS_RESILVERING, errbuf)); } } else if (err == ENOENT) { - return (zfs_error(hdl, EZFS_NO_SCRUB, msg)); + return (zfs_error(hdl, EZFS_NO_SCRUB, errbuf)); } else if (err == ENOTSUP && func == POOL_SCAN_RESILVER) { - return (zfs_error(hdl, EZFS_NO_RESILVER_DEFER, msg)); + return (zfs_error(hdl, EZFS_NO_RESILVER_DEFER, errbuf)); } else { - return (zpool_standard_error(hdl, err, msg)); + return (zpool_standard_error(hdl, err, errbuf)); } } @@ -2682,11 +2644,9 @@ vdev_to_nvlist_iter(nvlist_t *nv, nvlist_t *search, boolean_t *avail_spare, switch (nvpair_type(pair)) { case DATA_TYPE_UINT64: if (strcmp(srchkey, ZPOOL_CONFIG_GUID) == 0) { - uint64_t srchval, theguid; - - verify(nvpair_value_uint64(pair, &srchval) == 0); - verify(nvlist_lookup_uint64(nv, ZPOOL_CONFIG_GUID, - &theguid) == 0); + uint64_t srchval = fnvpair_value_uint64(pair); + uint64_t theguid = fnvlist_lookup_uint64(nv, + ZPOOL_CONFIG_GUID); if (theguid == srchval) return (nv); } @@ -2695,7 +2655,7 @@ vdev_to_nvlist_iter(nvlist_t *nv, nvlist_t *search, boolean_t *avail_spare, case DATA_TYPE_STRING: { char *srchval, *val; - verify(nvpair_value_string(pair, &srchval) == 0); + srchval = fnvpair_value_string(pair); if (nvlist_lookup_string(nv, srchkey, &val) != 0) break; @@ -2747,9 +2707,8 @@ vdev_to_nvlist_iter(nvlist_t *nv, nvlist_t *search, boolean_t *avail_spare, } verify(zpool_vdev_is_interior(type)); - verify(nvlist_lookup_uint64(nv, ZPOOL_CONFIG_ID, - &id) == 0); + id = fnvlist_lookup_uint64(nv, ZPOOL_CONFIG_ID); errno = 0; vdev_id = strtoull(idx, &end, 10); @@ -2775,8 +2734,8 @@ vdev_to_nvlist_iter(nvlist_t *nv, nvlist_t *search, boolean_t *avail_spare, free(type); return (NULL); } - verify(nvlist_lookup_uint64(nv, - ZPOOL_CONFIG_NPARITY, &vdev_parity) == 0); + vdev_parity = fnvlist_lookup_uint64(nv, + ZPOOL_CONFIG_NPARITY); if ((int)vdev_parity != parity) { free(type); break; @@ -2865,25 +2824,24 @@ zpool_find_vdev_by_physpath(zpool_handle_t *zhp, const char *ppath, uint64_t guid; char *end; - verify(nvlist_alloc(&search, NV_UNIQUE_NAME, KM_SLEEP) == 0); + search = fnvlist_alloc(); guid = strtoull(ppath, &end, 0); if (guid != 0 && *end == '\0') { - verify(nvlist_add_uint64(search, ZPOOL_CONFIG_GUID, guid) == 0); + fnvlist_add_uint64(search, ZPOOL_CONFIG_GUID, guid); } else { - verify(nvlist_add_string(search, ZPOOL_CONFIG_PHYS_PATH, - ppath) == 0); + fnvlist_add_string(search, ZPOOL_CONFIG_PHYS_PATH, ppath); } - verify(nvlist_lookup_nvlist(zhp->zpool_config, ZPOOL_CONFIG_VDEV_TREE, - &nvroot) == 0); + nvroot = fnvlist_lookup_nvlist(zhp->zpool_config, + ZPOOL_CONFIG_VDEV_TREE); *avail_spare = B_FALSE; *l2cache = B_FALSE; if (log != NULL) *log = B_FALSE; ret = vdev_to_nvlist_iter(nvroot, search, avail_spare, l2cache, log); - nvlist_free(search); + fnvlist_free(search); return (ret); } @@ -2916,26 +2874,26 @@ zpool_find_vdev(zpool_handle_t *zhp, const char *path, boolean_t *avail_spare, nvlist_t *nvroot, *search, *ret; uint64_t guid; - verify(nvlist_alloc(&search, NV_UNIQUE_NAME, KM_SLEEP) == 0); + search = fnvlist_alloc(); guid = strtoull(path, &end, 0); if (guid != 0 && *end == '\0') { - verify(nvlist_add_uint64(search, ZPOOL_CONFIG_GUID, guid) == 0); + fnvlist_add_uint64(search, ZPOOL_CONFIG_GUID, guid); } else if (zpool_vdev_is_interior(path)) { - verify(nvlist_add_string(search, ZPOOL_CONFIG_TYPE, path) == 0); + fnvlist_add_string(search, ZPOOL_CONFIG_TYPE, path); } else { - verify(nvlist_add_string(search, ZPOOL_CONFIG_PATH, path) == 0); + fnvlist_add_string(search, ZPOOL_CONFIG_PATH, path); } - verify(nvlist_lookup_nvlist(zhp->zpool_config, ZPOOL_CONFIG_VDEV_TREE, - &nvroot) == 0); + nvroot = fnvlist_lookup_nvlist(zhp->zpool_config, + ZPOOL_CONFIG_VDEV_TREE); *avail_spare = B_FALSE; *l2cache = B_FALSE; if (log != NULL) *log = B_FALSE; ret = vdev_to_nvlist_iter(nvroot, search, avail_spare, l2cache, log); - nvlist_free(search); + fnvlist_free(search); return (ret); } @@ -3099,7 +3057,6 @@ static uint64_t zpool_vdev_path_to_guid_impl(zpool_handle_t *zhp, const char *path, boolean_t *is_spare, boolean_t *is_l2cache, boolean_t *is_log) { - uint64_t guid; boolean_t spare = B_FALSE, l2cache = B_FALSE, log = B_FALSE; nvlist_t *tgt; @@ -3107,7 +3064,6 @@ zpool_vdev_path_to_guid_impl(zpool_handle_t *zhp, const char *path, &log)) == NULL) return (0); - verify(nvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID, &guid) == 0); if (is_spare != NULL) *is_spare = spare; if (is_l2cache != NULL) @@ -3115,7 +3071,7 @@ zpool_vdev_path_to_guid_impl(zpool_handle_t *zhp, const char *path, if (is_log != NULL) *is_log = log; - return (guid); + return (fnvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID)); } /* Convert a vdev path to a GUID. Returns GUID or 0 on error. */ @@ -3134,31 +3090,31 @@ zpool_vdev_online(zpool_handle_t *zhp, const char *path, int flags, vdev_state_t *newstate) { zfs_cmd_t zc = {"\0"}; - char msg[1024]; - char *pathname; + char errbuf[ERRBUFLEN]; nvlist_t *tgt; boolean_t avail_spare, l2cache, islog; libzfs_handle_t *hdl = zhp->zpool_hdl; - int error; if (flags & ZFS_ONLINE_EXPAND) { - (void) snprintf(msg, sizeof (msg), + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot expand %s"), path); } else { - (void) snprintf(msg, sizeof (msg), + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot online %s"), path); } (void) strlcpy(zc.zc_name, zhp->zpool_name, sizeof (zc.zc_name)); if ((tgt = zpool_find_vdev(zhp, path, &avail_spare, &l2cache, &islog)) == NULL) - return (zfs_error(hdl, EZFS_NODEVICE, msg)); + return (zfs_error(hdl, EZFS_NODEVICE, errbuf)); - verify(nvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID, &zc.zc_guid) == 0); + zc.zc_guid = fnvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID); if (avail_spare) - return (zfs_error(hdl, EZFS_ISSPARE, msg)); + return (zfs_error(hdl, EZFS_ISSPARE, errbuf)); +#ifndef __FreeBSD__ + char *pathname; if ((flags & ZFS_ONLINE_EXPAND || zpool_get_prop_int(zhp, ZPOOL_PROP_AUTOEXPAND, NULL)) && nvlist_lookup_string(tgt, ZPOOL_CONFIG_PATH, &pathname) == 0) { @@ -3173,28 +3129,30 @@ zpool_vdev_online(zpool_handle_t *zhp, const char *path, int flags, if (l2cache) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "cannot expand cache devices")); - return (zfs_error(hdl, EZFS_VDEVNOTSUP, msg)); + return (zfs_error(hdl, EZFS_VDEVNOTSUP, errbuf)); } if (wholedisk) { const char *fullpath = path; char buf[MAXPATHLEN]; + int error; if (path[0] != '/') { error = zfs_resolve_shortname(path, buf, sizeof (buf)); if (error != 0) return (zfs_error(hdl, EZFS_NODEVICE, - msg)); + errbuf)); fullpath = buf; } - error = zpool_relabel_disk(hdl, fullpath, msg); + error = zpool_relabel_disk(hdl, fullpath, errbuf); if (error != 0) return (error); } } +#endif zc.zc_cookie = VDEV_STATE_ONLINE; zc.zc_obj = flags; @@ -3204,9 +3162,9 @@ zpool_vdev_online(zpool_handle_t *zhp, const char *path, int flags, zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "was split " "from this pool into a new one. Use '%s' " "instead"), "zpool detach"); - return (zfs_error(hdl, EZFS_POSTSPLIT_ONLINE, msg)); + return (zfs_error(hdl, EZFS_POSTSPLIT_ONLINE, errbuf)); } - return (zpool_standard_error(hdl, errno, msg)); + return (zpool_standard_error(hdl, errno, errbuf)); } *newstate = zc.zc_cookie; @@ -3220,23 +3178,23 @@ int zpool_vdev_offline(zpool_handle_t *zhp, const char *path, boolean_t istmp) { zfs_cmd_t zc = {"\0"}; - char msg[1024]; + char errbuf[ERRBUFLEN]; nvlist_t *tgt; boolean_t avail_spare, l2cache; libzfs_handle_t *hdl = zhp->zpool_hdl; - (void) snprintf(msg, sizeof (msg), + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot offline %s"), path); (void) strlcpy(zc.zc_name, zhp->zpool_name, sizeof (zc.zc_name)); if ((tgt = zpool_find_vdev(zhp, path, &avail_spare, &l2cache, NULL)) == NULL) - return (zfs_error(hdl, EZFS_NODEVICE, msg)); + return (zfs_error(hdl, EZFS_NODEVICE, errbuf)); - verify(nvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID, &zc.zc_guid) == 0); + zc.zc_guid = fnvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID); if (avail_spare) - return (zfs_error(hdl, EZFS_ISSPARE, msg)); + return (zfs_error(hdl, EZFS_ISSPARE, errbuf)); zc.zc_cookie = VDEV_STATE_OFFLINE; zc.zc_obj = istmp ? ZFS_OFFLINE_TEMPORARY : 0; @@ -3250,16 +3208,16 @@ zpool_vdev_offline(zpool_handle_t *zhp, const char *path, boolean_t istmp) /* * There are no other replicas of this device. */ - return (zfs_error(hdl, EZFS_NOREPLICAS, msg)); + return (zfs_error(hdl, EZFS_NOREPLICAS, errbuf)); case EEXIST: /* * The log device has unplayed logs */ - return (zfs_error(hdl, EZFS_UNPLAYED_LOGS, msg)); + return (zfs_error(hdl, EZFS_UNPLAYED_LOGS, errbuf)); default: - return (zpool_standard_error(hdl, errno, msg)); + return (zpool_standard_error(hdl, errno, errbuf)); } } @@ -3270,10 +3228,10 @@ int zpool_vdev_fault(zpool_handle_t *zhp, uint64_t guid, vdev_aux_t aux) { zfs_cmd_t zc = {"\0"}; - char msg[1024]; + char errbuf[ERRBUFLEN]; libzfs_handle_t *hdl = zhp->zpool_hdl; - (void) snprintf(msg, sizeof (msg), + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot fault %llu"), (u_longlong_t)guid); (void) strlcpy(zc.zc_name, zhp->zpool_name, sizeof (zc.zc_name)); @@ -3290,10 +3248,10 @@ zpool_vdev_fault(zpool_handle_t *zhp, uint64_t guid, vdev_aux_t aux) /* * There are no other replicas of this device. */ - return (zfs_error(hdl, EZFS_NOREPLICAS, msg)); + return (zfs_error(hdl, EZFS_NOREPLICAS, errbuf)); default: - return (zpool_standard_error(hdl, errno, msg)); + return (zpool_standard_error(hdl, errno, errbuf)); } } @@ -3305,10 +3263,10 @@ int zpool_vdev_degrade(zpool_handle_t *zhp, uint64_t guid, vdev_aux_t aux) { zfs_cmd_t zc = {"\0"}; - char msg[1024]; + char errbuf[ERRBUFLEN]; libzfs_handle_t *hdl = zhp->zpool_hdl; - (void) snprintf(msg, sizeof (msg), + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot degrade %llu"), (u_longlong_t)guid); (void) strlcpy(zc.zc_name, zhp->zpool_name, sizeof (zc.zc_name)); @@ -3319,7 +3277,7 @@ zpool_vdev_degrade(zpool_handle_t *zhp, uint64_t guid, vdev_aux_t aux) if (zfs_ioctl(hdl, ZFS_IOC_VDEV_SET_STATE, &zc) == 0) return (0); - return (zpool_standard_error(hdl, errno, msg)); + return (zpool_standard_error(hdl, errno, errbuf)); } /* @@ -3331,13 +3289,10 @@ is_replacing_spare(nvlist_t *search, nvlist_t *tgt, int which) { nvlist_t **child; uint_t c, children; - char *type; if (nvlist_lookup_nvlist_array(search, ZPOOL_CONFIG_CHILDREN, &child, &children) == 0) { - verify(nvlist_lookup_string(search, ZPOOL_CONFIG_TYPE, - &type) == 0); - + char *type = fnvlist_lookup_string(search, ZPOOL_CONFIG_TYPE); if ((strcmp(type, VDEV_TYPE_SPARE) == 0 || strcmp(type, VDEV_TYPE_DRAID_SPARE) == 0) && children == 2 && child[which] == tgt) @@ -3360,7 +3315,7 @@ zpool_vdev_attach(zpool_handle_t *zhp, const char *old_disk, const char *new_disk, nvlist_t *nvroot, int replacing, boolean_t rebuild) { zfs_cmd_t zc = {"\0"}; - char msg[1024]; + char errbuf[ERRBUFLEN]; int ret; nvlist_t *tgt; boolean_t avail_spare, l2cache, islog; @@ -3372,24 +3327,24 @@ zpool_vdev_attach(zpool_handle_t *zhp, const char *old_disk, libzfs_handle_t *hdl = zhp->zpool_hdl; if (replacing) - (void) snprintf(msg, sizeof (msg), dgettext(TEXT_DOMAIN, + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot replace %s with %s"), old_disk, new_disk); else - (void) snprintf(msg, sizeof (msg), dgettext(TEXT_DOMAIN, + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot attach %s to %s"), new_disk, old_disk); (void) strlcpy(zc.zc_name, zhp->zpool_name, sizeof (zc.zc_name)); if ((tgt = zpool_find_vdev(zhp, old_disk, &avail_spare, &l2cache, &islog)) == NULL) - return (zfs_error(hdl, EZFS_NODEVICE, msg)); + return (zfs_error(hdl, EZFS_NODEVICE, errbuf)); if (avail_spare) - return (zfs_error(hdl, EZFS_ISSPARE, msg)); + return (zfs_error(hdl, EZFS_ISSPARE, errbuf)); if (l2cache) - return (zfs_error(hdl, EZFS_ISL2CACHE, msg)); + return (zfs_error(hdl, EZFS_ISL2CACHE, errbuf)); - verify(nvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID, &zc.zc_guid) == 0); + zc.zc_guid = fnvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID); zc.zc_cookie = replacing; zc.zc_simple = rebuild; @@ -3397,18 +3352,18 @@ zpool_vdev_attach(zpool_handle_t *zhp, const char *old_disk, zfeature_lookup_guid("org.openzfs:device_rebuild", NULL) != 0) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "the loaded zfs module doesn't support device rebuilds")); - return (zfs_error(hdl, EZFS_POOL_NOTSUP, msg)); + return (zfs_error(hdl, EZFS_POOL_NOTSUP, errbuf)); } if (nvlist_lookup_nvlist_array(nvroot, ZPOOL_CONFIG_CHILDREN, &child, &children) != 0 || children != 1) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "new device must be a single disk")); - return (zfs_error(hdl, EZFS_INVALCONFIG, msg)); + return (zfs_error(hdl, EZFS_INVALCONFIG, errbuf)); } - verify(nvlist_lookup_nvlist(zpool_get_config(zhp, NULL), - ZPOOL_CONFIG_VDEV_TREE, &config_root) == 0); + config_root = fnvlist_lookup_nvlist(zpool_get_config(zhp, NULL), + ZPOOL_CONFIG_VDEV_TREE); if ((newname = zpool_vdev_name(NULL, NULL, child[0], 0)) == NULL) return (-1); @@ -3425,13 +3380,12 @@ zpool_vdev_attach(zpool_handle_t *zhp, const char *old_disk, zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "can only be replaced by another hot spare")); free(newname); - return (zfs_error(hdl, EZFS_BADTARGET, msg)); + return (zfs_error(hdl, EZFS_BADTARGET, errbuf)); } free(newname); - if (zcmd_write_conf_nvlist(hdl, &zc, nvroot) != 0) - return (-1); + zcmd_write_conf_nvlist(hdl, &zc, nvroot); ret = zfs_ioctl(hdl, ZFS_IOC_VDEV_ATTACH, &zc); @@ -3484,7 +3438,7 @@ zpool_vdev_attach(zpool_handle_t *zhp, const char *old_disk, "disks")); } } - (void) zfs_error(hdl, EZFS_BADTARGET, msg); + (void) zfs_error(hdl, EZFS_BADTARGET, errbuf); break; case EINVAL: @@ -3493,14 +3447,14 @@ zpool_vdev_attach(zpool_handle_t *zhp, const char *old_disk, */ zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "new device must be a single disk")); - (void) zfs_error(hdl, EZFS_INVALCONFIG, msg); + (void) zfs_error(hdl, EZFS_INVALCONFIG, errbuf); break; case EBUSY: zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "%s is busy, " "or device removal is in progress"), new_disk); - (void) zfs_error(hdl, EZFS_BADDEV, msg); + (void) zfs_error(hdl, EZFS_BADDEV, errbuf); break; case EOVERFLOW: @@ -3509,7 +3463,7 @@ zpool_vdev_attach(zpool_handle_t *zhp, const char *old_disk, */ zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "device is too small")); - (void) zfs_error(hdl, EZFS_BADDEV, msg); + (void) zfs_error(hdl, EZFS_BADDEV, errbuf); break; case EDOM: @@ -3519,18 +3473,18 @@ zpool_vdev_attach(zpool_handle_t *zhp, const char *old_disk, zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "new device has a different optimal sector size; use the " "option '-o ashift=N' to override the optimal size")); - (void) zfs_error(hdl, EZFS_BADDEV, msg); + (void) zfs_error(hdl, EZFS_BADDEV, errbuf); break; case ENAMETOOLONG: /* * The resulting top-level vdev spec won't fit in the label. */ - (void) zfs_error(hdl, EZFS_DEVOVERFLOW, msg); + (void) zfs_error(hdl, EZFS_DEVOVERFLOW, errbuf); break; default: - (void) zpool_standard_error(hdl, errno, msg); + (void) zpool_standard_error(hdl, errno, errbuf); } return (-1); @@ -3543,26 +3497,26 @@ int zpool_vdev_detach(zpool_handle_t *zhp, const char *path) { zfs_cmd_t zc = {"\0"}; - char msg[1024]; + char errbuf[ERRBUFLEN]; nvlist_t *tgt; boolean_t avail_spare, l2cache; libzfs_handle_t *hdl = zhp->zpool_hdl; - (void) snprintf(msg, sizeof (msg), + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot detach %s"), path); (void) strlcpy(zc.zc_name, zhp->zpool_name, sizeof (zc.zc_name)); if ((tgt = zpool_find_vdev(zhp, path, &avail_spare, &l2cache, NULL)) == NULL) - return (zfs_error(hdl, EZFS_NODEVICE, msg)); + return (zfs_error(hdl, EZFS_NODEVICE, errbuf)); if (avail_spare) - return (zfs_error(hdl, EZFS_ISSPARE, msg)); + return (zfs_error(hdl, EZFS_ISSPARE, errbuf)); if (l2cache) - return (zfs_error(hdl, EZFS_ISL2CACHE, msg)); + return (zfs_error(hdl, EZFS_ISL2CACHE, errbuf)); - verify(nvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID, &zc.zc_guid) == 0); + zc.zc_guid = fnvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID); if (zfs_ioctl(hdl, ZFS_IOC_VDEV_DETACH, &zc) == 0) return (0); @@ -3575,18 +3529,18 @@ zpool_vdev_detach(zpool_handle_t *zhp, const char *path) */ zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "only " "applicable to mirror and replacing vdevs")); - (void) zfs_error(hdl, EZFS_BADTARGET, msg); + (void) zfs_error(hdl, EZFS_BADTARGET, errbuf); break; case EBUSY: /* * There are no other replicas of this device. */ - (void) zfs_error(hdl, EZFS_NOREPLICAS, msg); + (void) zfs_error(hdl, EZFS_NOREPLICAS, errbuf); break; default: - (void) zpool_standard_error(hdl, errno, msg); + (void) zpool_standard_error(hdl, errno, errbuf); } return (-1); @@ -3641,7 +3595,7 @@ zpool_vdev_split(zpool_handle_t *zhp, char *newname, nvlist_t **newroot, nvlist_t *props, splitflags_t flags) { zfs_cmd_t zc = {"\0"}; - char msg[1024], *bias; + char errbuf[ERRBUFLEN], *bias; nvlist_t *tree, *config, **child, **newchild, *newconfig = NULL; nvlist_t **varray = NULL, *zc_props = NULL; uint_t c, children, newchildren, lastlog = 0, vcount, found = 0; @@ -3650,11 +3604,11 @@ zpool_vdev_split(zpool_handle_t *zhp, char *newname, nvlist_t **newroot, boolean_t freelist = B_FALSE, memory_err = B_TRUE; int retval = 0; - (void) snprintf(msg, sizeof (msg), + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "Unable to split %s"), zhp->zpool_name); if (!zpool_name_valid(hdl, B_FALSE, newname)) - return (zfs_error(hdl, EZFS_INVALIDNAME, msg)); + return (zfs_error(hdl, EZFS_INVALIDNAME, errbuf)); if ((config = zpool_get_config(zhp, NULL)) == NULL) { (void) fprintf(stderr, gettext("Internal error: unable to " @@ -3662,14 +3616,13 @@ zpool_vdev_split(zpool_handle_t *zhp, char *newname, nvlist_t **newroot, return (-1); } - verify(nvlist_lookup_nvlist(config, ZPOOL_CONFIG_VDEV_TREE, &tree) - == 0); - verify(nvlist_lookup_uint64(config, ZPOOL_CONFIG_VERSION, &vers) == 0); + tree = fnvlist_lookup_nvlist(config, ZPOOL_CONFIG_VDEV_TREE); + vers = fnvlist_lookup_uint64(config, ZPOOL_CONFIG_VERSION); if (props) { prop_flags_t flags = { .create = B_FALSE, .import = B_TRUE }; if ((zc_props = zpool_valid_proplist(hdl, zhp->zpool_name, - props, vers, flags, msg)) == NULL) + props, vers, flags, errbuf)) == NULL) return (-1); (void) nvlist_lookup_uint64(zc_props, zpool_prop_to_name(ZPOOL_PROP_READONLY), &readonly); @@ -3731,8 +3684,7 @@ zpool_vdev_split(zpool_handle_t *zhp, char *newname, nvlist_t **newroot, continue; } lastlog = 0; - verify(nvlist_lookup_string(child[c], ZPOOL_CONFIG_TYPE, &type) - == 0); + type = fnvlist_lookup_string(child[c], ZPOOL_CONFIG_TYPE); if (strcmp(type, VDEV_TYPE_INDIRECT) == 0) { vdev = child[c]; @@ -3742,7 +3694,7 @@ zpool_vdev_split(zpool_handle_t *zhp, char *newname, nvlist_t **newroot, } else if (strcmp(type, VDEV_TYPE_MIRROR) != 0) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "Source pool must be composed only of mirrors\n")); - retval = zfs_error(hdl, EZFS_INVALCONFIG, msg); + retval = zfs_error(hdl, EZFS_INVALCONFIG, errbuf); goto out; } @@ -3790,7 +3742,7 @@ zpool_vdev_split(zpool_handle_t *zhp, char *newname, nvlist_t **newroot, if (found != newchildren) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "Device list must " "include at most one disk from each mirror")); - retval = zfs_error(hdl, EZFS_INVALCONFIG, msg); + retval = zfs_error(hdl, EZFS_INVALCONFIG, errbuf); goto out; } @@ -3839,13 +3791,12 @@ zpool_vdev_split(zpool_handle_t *zhp, char *newname, nvlist_t **newroot, zc.zc_cookie = ZPOOL_EXPORT_AFTER_SPLIT; (void) strlcpy(zc.zc_name, zhp->zpool_name, sizeof (zc.zc_name)); (void) strlcpy(zc.zc_string, newname, sizeof (zc.zc_string)); - if (zcmd_write_conf_nvlist(hdl, &zc, newconfig) != 0) - goto out; - if (zc_props != NULL && zcmd_write_src_nvlist(hdl, &zc, zc_props) != 0) - goto out; + zcmd_write_conf_nvlist(hdl, &zc, newconfig); + if (zc_props != NULL) + zcmd_write_src_nvlist(hdl, &zc, zc_props); if (zfs_ioctl(hdl, ZFS_IOC_VDEV_SPLIT, &zc) != 0) { - retval = zpool_standard_error(hdl, errno, msg); + retval = zpool_standard_error(hdl, errno, errbuf); goto out; } @@ -3884,31 +3835,31 @@ int zpool_vdev_remove(zpool_handle_t *zhp, const char *path) { zfs_cmd_t zc = {"\0"}; - char msg[1024]; + char errbuf[ERRBUFLEN]; nvlist_t *tgt; boolean_t avail_spare, l2cache, islog; libzfs_handle_t *hdl = zhp->zpool_hdl; uint64_t version; - (void) snprintf(msg, sizeof (msg), + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot remove %s"), path); if (zpool_is_draid_spare(path)) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "dRAID spares cannot be removed")); - return (zfs_error(hdl, EZFS_NODEVICE, msg)); + return (zfs_error(hdl, EZFS_NODEVICE, errbuf)); } (void) strlcpy(zc.zc_name, zhp->zpool_name, sizeof (zc.zc_name)); if ((tgt = zpool_find_vdev(zhp, path, &avail_spare, &l2cache, &islog)) == NULL) - return (zfs_error(hdl, EZFS_NODEVICE, msg)); + return (zfs_error(hdl, EZFS_NODEVICE, errbuf)); version = zpool_get_prop_int(zhp, ZPOOL_PROP_VERSION, NULL); if (islog && version < SPA_VERSION_HOLES) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "pool must be upgraded to support log removal")); - return (zfs_error(hdl, EZFS_BADVERSION, msg)); + return (zfs_error(hdl, EZFS_BADVERSION, errbuf)); } zc.zc_guid = fnvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID); @@ -3922,7 +3873,7 @@ zpool_vdev_remove(zpool_handle_t *zhp, const char *path) zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "invalid config; all top-level vdevs must " "have the same sector size and not be raidz.")); - (void) zfs_error(hdl, EZFS_INVALCONFIG, msg); + (void) zfs_error(hdl, EZFS_INVALCONFIG, errbuf); break; case EBUSY: @@ -3933,21 +3884,21 @@ zpool_vdev_remove(zpool_handle_t *zhp, const char *path) zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "Pool busy; removal may already be in progress")); } - (void) zfs_error(hdl, EZFS_BUSY, msg); + (void) zfs_error(hdl, EZFS_BUSY, errbuf); break; case EACCES: if (islog) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "Mount encrypted datasets to replay logs.")); - (void) zfs_error(hdl, EZFS_BUSY, msg); + (void) zfs_error(hdl, EZFS_BUSY, errbuf); } else { - (void) zpool_standard_error(hdl, errno, msg); + (void) zpool_standard_error(hdl, errno, errbuf); } break; default: - (void) zpool_standard_error(hdl, errno, msg); + (void) zpool_standard_error(hdl, errno, errbuf); } return (-1); } @@ -3955,39 +3906,38 @@ zpool_vdev_remove(zpool_handle_t *zhp, const char *path) int zpool_vdev_remove_cancel(zpool_handle_t *zhp) { - zfs_cmd_t zc; - char msg[1024]; + zfs_cmd_t zc = {{0}}; + char errbuf[ERRBUFLEN]; libzfs_handle_t *hdl = zhp->zpool_hdl; - (void) snprintf(msg, sizeof (msg), + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot cancel removal")); - bzero(&zc, sizeof (zc)); (void) strlcpy(zc.zc_name, zhp->zpool_name, sizeof (zc.zc_name)); zc.zc_cookie = 1; if (zfs_ioctl(hdl, ZFS_IOC_VDEV_REMOVE, &zc) == 0) return (0); - return (zpool_standard_error(hdl, errno, msg)); + return (zpool_standard_error(hdl, errno, errbuf)); } int zpool_vdev_indirect_size(zpool_handle_t *zhp, const char *path, uint64_t *sizep) { - char msg[1024]; + char errbuf[ERRBUFLEN]; nvlist_t *tgt; boolean_t avail_spare, l2cache, islog; libzfs_handle_t *hdl = zhp->zpool_hdl; - (void) snprintf(msg, sizeof (msg), + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot determine indirect size of %s"), path); if ((tgt = zpool_find_vdev(zhp, path, &avail_spare, &l2cache, &islog)) == NULL) - return (zfs_error(hdl, EZFS_NODEVICE, msg)); + return (zfs_error(hdl, EZFS_NODEVICE, errbuf)); if (avail_spare || l2cache || islog) { *sizep = 0; @@ -3997,7 +3947,7 @@ zpool_vdev_indirect_size(zpool_handle_t *zhp, const char *path, if (nvlist_lookup_uint64(tgt, ZPOOL_CONFIG_INDIRECT_SIZE, sizep) != 0) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "indirect size not available")); - return (zfs_error(hdl, EINVAL, msg)); + return (zfs_error(hdl, EINVAL, errbuf)); } return (0); } @@ -4009,7 +3959,7 @@ int zpool_clear(zpool_handle_t *zhp, const char *path, nvlist_t *rewindnvl) { zfs_cmd_t zc = {"\0"}; - char msg[1024]; + char errbuf[ERRBUFLEN]; nvlist_t *tgt; zpool_load_policy_t policy; boolean_t avail_spare, l2cache; @@ -4018,11 +3968,11 @@ zpool_clear(zpool_handle_t *zhp, const char *path, nvlist_t *rewindnvl) int error; if (path) - (void) snprintf(msg, sizeof (msg), + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot clear errors for %s"), path); else - (void) snprintf(msg, sizeof (msg), + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot clear errors for %s"), zhp->zpool_name); @@ -4030,35 +3980,27 @@ zpool_clear(zpool_handle_t *zhp, const char *path, nvlist_t *rewindnvl) if (path) { if ((tgt = zpool_find_vdev(zhp, path, &avail_spare, &l2cache, NULL)) == NULL) - return (zfs_error(hdl, EZFS_NODEVICE, msg)); + return (zfs_error(hdl, EZFS_NODEVICE, errbuf)); /* * Don't allow error clearing for hot spares. Do allow * error clearing for l2cache devices. */ if (avail_spare) - return (zfs_error(hdl, EZFS_ISSPARE, msg)); + return (zfs_error(hdl, EZFS_ISSPARE, errbuf)); - verify(nvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID, - &zc.zc_guid) == 0); + zc.zc_guid = fnvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID); } zpool_get_load_policy(rewindnvl, &policy); zc.zc_cookie = policy.zlp_rewind; - if (zcmd_alloc_dst_nvlist(hdl, &zc, zhp->zpool_config_size * 2) != 0) - return (-1); - - if (zcmd_write_src_nvlist(hdl, &zc, rewindnvl) != 0) - return (-1); + zcmd_alloc_dst_nvlist(hdl, &zc, zhp->zpool_config_size * 2); + zcmd_write_src_nvlist(hdl, &zc, rewindnvl); while ((error = zfs_ioctl(hdl, ZFS_IOC_CLEAR, &zc)) != 0 && - errno == ENOMEM) { - if (zcmd_expand_dst_nvlist(hdl, &zc) != 0) { - zcmd_free_nvlists(&zc); - return (-1); - } - } + errno == ENOMEM) + zcmd_expand_dst_nvlist(hdl, &zc); if (!error || ((policy.zlp_rewind & ZPOOL_TRY_REWIND) && errno != EPERM && errno != EACCES)) { @@ -4075,7 +4017,7 @@ zpool_clear(zpool_handle_t *zhp, const char *path, nvlist_t *rewindnvl) } zcmd_free_nvlists(&zc); - return (zpool_standard_error(hdl, errno, msg)); + return (zpool_standard_error(hdl, errno, errbuf)); } /* @@ -4085,10 +4027,10 @@ int zpool_vdev_clear(zpool_handle_t *zhp, uint64_t guid) { zfs_cmd_t zc = {"\0"}; - char msg[1024]; + char errbuf[ERRBUFLEN]; libzfs_handle_t *hdl = zhp->zpool_hdl; - (void) snprintf(msg, sizeof (msg), + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot clear errors for %llx"), (u_longlong_t)guid); @@ -4099,7 +4041,7 @@ zpool_vdev_clear(zpool_handle_t *zhp, uint64_t guid) if (zfs_ioctl(hdl, ZFS_IOC_CLEAR, &zc) == 0) return (0); - return (zpool_standard_error(hdl, errno, msg)); + return (zpool_standard_error(hdl, errno, errbuf)); } /* @@ -4108,18 +4050,18 @@ zpool_vdev_clear(zpool_handle_t *zhp, uint64_t guid) int zpool_reguid(zpool_handle_t *zhp) { - char msg[1024]; + char errbuf[ERRBUFLEN]; libzfs_handle_t *hdl = zhp->zpool_hdl; zfs_cmd_t zc = {"\0"}; - (void) snprintf(msg, sizeof (msg), + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot reguid '%s'"), zhp->zpool_name); (void) strlcpy(zc.zc_name, zhp->zpool_name, sizeof (zc.zc_name)); if (zfs_ioctl(hdl, ZFS_IOC_POOL_REGUID, &zc) == 0) return (0); - return (zpool_standard_error(hdl, errno, msg)); + return (zpool_standard_error(hdl, errno, errbuf)); } /* @@ -4184,32 +4126,25 @@ char * zpool_vdev_name(libzfs_handle_t *hdl, zpool_handle_t *zhp, nvlist_t *nv, int name_flags) { - char *path, *type, *env; + char *type, *tpath; + const char *path; uint64_t value; char buf[PATH_BUF_LEN]; - char tmpbuf[PATH_BUF_LEN]; + char tmpbuf[PATH_BUF_LEN * 2]; /* * vdev_name will be "root"/"root-0" for the root vdev, but it is the * zpool name that will be displayed to the user. */ - verify(nvlist_lookup_string(nv, ZPOOL_CONFIG_TYPE, &type) == 0); + type = fnvlist_lookup_string(nv, ZPOOL_CONFIG_TYPE); if (zhp != NULL && strcmp(type, "root") == 0) return (zfs_strdup(hdl, zpool_get_name(zhp))); - env = getenv("ZPOOL_VDEV_NAME_PATH"); - if (env && (strtoul(env, NULL, 0) > 0 || - !strncasecmp(env, "YES", 3) || !strncasecmp(env, "ON", 2))) + if (libzfs_envvar_is_set("ZPOOL_VDEV_NAME_PATH")) name_flags |= VDEV_NAME_PATH; - - env = getenv("ZPOOL_VDEV_NAME_GUID"); - if (env && (strtoul(env, NULL, 0) > 0 || - !strncasecmp(env, "YES", 3) || !strncasecmp(env, "ON", 2))) + if (libzfs_envvar_is_set("ZPOOL_VDEV_NAME_GUID")) name_flags |= VDEV_NAME_GUID; - - env = getenv("ZPOOL_VDEV_NAME_FOLLOW_LINKS"); - if (env && (strtoul(env, NULL, 0) > 0 || - !strncasecmp(env, "YES", 3) || !strncasecmp(env, "ON", 2))) + if (libzfs_envvar_is_set("ZPOOL_VDEV_NAME_FOLLOW_LINKS")) name_flags |= VDEV_NAME_FOLLOW_LINKS; if (nvlist_lookup_uint64(nv, ZPOOL_CONFIG_NOT_PRESENT, &value) == 0 || @@ -4217,7 +4152,9 @@ zpool_vdev_name(libzfs_handle_t *hdl, zpool_handle_t *zhp, nvlist_t *nv, (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_GUID, &value); (void) snprintf(buf, sizeof (buf), "%llu", (u_longlong_t)value); path = buf; - } else if (nvlist_lookup_string(nv, ZPOOL_CONFIG_PATH, &path) == 0) { + } else if (nvlist_lookup_string(nv, ZPOOL_CONFIG_PATH, &tpath) == 0) { + path = tpath; + if (name_flags & VDEV_NAME_FOLLOW_LINKS) { char *rp = realpath(path, NULL); if (rp) { @@ -4250,8 +4187,7 @@ zpool_vdev_name(libzfs_handle_t *hdl, zpool_handle_t *zhp, nvlist_t *nv, * If it's a raidz device, we need to stick in the parity level. */ if (strcmp(path, VDEV_TYPE_RAIDZ) == 0) { - verify(nvlist_lookup_uint64(nv, ZPOOL_CONFIG_NPARITY, - &value) == 0); + value = fnvlist_lookup_uint64(nv, ZPOOL_CONFIG_NPARITY); (void) snprintf(buf, sizeof (buf), "%s%llu", path, (u_longlong_t)value); path = buf; @@ -4267,12 +4203,12 @@ zpool_vdev_name(libzfs_handle_t *hdl, zpool_handle_t *zhp, nvlist_t *nv, verify(nvlist_lookup_nvlist_array(nv, ZPOOL_CONFIG_CHILDREN, &child, &children) == 0); - verify(nvlist_lookup_uint64(nv, - ZPOOL_CONFIG_NPARITY, &nparity) == 0); - verify(nvlist_lookup_uint64(nv, - ZPOOL_CONFIG_DRAID_NDATA, &ndata) == 0); - verify(nvlist_lookup_uint64(nv, - ZPOOL_CONFIG_DRAID_NSPARES, &nspares) == 0); + nparity = fnvlist_lookup_uint64(nv, + ZPOOL_CONFIG_NPARITY); + ndata = fnvlist_lookup_uint64(nv, + ZPOOL_CONFIG_DRAID_NDATA); + nspares = fnvlist_lookup_uint64(nv, + ZPOOL_CONFIG_DRAID_NSPARES); path = zpool_draid_name(buf, sizeof (buf), ndata, nparity, nspares, children); @@ -4283,9 +4219,8 @@ zpool_vdev_name(libzfs_handle_t *hdl, zpool_handle_t *zhp, nvlist_t *nv, * naming convention. */ if (name_flags & VDEV_NAME_TYPE_ID) { - uint64_t id; - verify(nvlist_lookup_uint64(nv, ZPOOL_CONFIG_ID, - &id) == 0); + uint64_t id = fnvlist_lookup_uint64(nv, + ZPOOL_CONFIG_ID); (void) snprintf(tmpbuf, sizeof (tmpbuf), "%s-%llu", path, (u_longlong_t)id); path = tmpbuf; @@ -4319,8 +4254,7 @@ zpool_get_errlog(zpool_handle_t *zhp, nvlist_t **nverrlistp) * has increased, allocate more space and continue until we get the * entire list. */ - verify(nvlist_lookup_uint64(zhp->zpool_config, ZPOOL_CONFIG_ERRCOUNT, - &count) == 0); + count = fnvlist_lookup_uint64(zhp->zpool_config, ZPOOL_CONFIG_ERRCOUNT); if (count == 0) return (0); zc.zc_nvlist_dst = (uintptr_t)zfs_alloc(zhp->zpool_hdl, @@ -4437,13 +4371,11 @@ zpool_log_history(libzfs_handle_t *hdl, const char *message) { zfs_cmd_t zc = {"\0"}; nvlist_t *args; - int err; args = fnvlist_alloc(); fnvlist_add_string(args, "message", message); - err = zcmd_write_src_nvlist(hdl, &zc, args); - if (err == 0) - err = zfs_ioctl(hdl, ZFS_IOC_LOG_HISTORY, &zc); + zcmd_write_src_nvlist(hdl, &zc, args); + int err = zfs_ioctl(hdl, ZFS_IOC_LOG_HISTORY, &zc); nvlist_free(args); zcmd_free_nvlists(&zc); return (err); @@ -4505,17 +4437,17 @@ int zpool_get_history(zpool_handle_t *zhp, nvlist_t **nvhisp, uint64_t *off, boolean_t *eof) { + libzfs_handle_t *hdl = zhp->zpool_hdl; char *buf; int buflen = 128 * 1024; nvlist_t **records = NULL; uint_t numrecords = 0; - int err, i; + int err = 0, i; uint64_t start = *off; - buf = malloc(buflen); - if (buf == NULL) - return (ENOMEM); - /* process about 1MB a time */ + buf = zfs_alloc(hdl, buflen); + + /* process about 1MiB a time */ while (*off - start < 1024 * 1024) { uint64_t bytes_read = buflen; uint64_t leftover; @@ -4530,8 +4462,12 @@ zpool_get_history(zpool_handle_t *zhp, nvlist_t **nvhisp, uint64_t *off, } if ((err = zpool_history_unpack(buf, bytes_read, - &leftover, &records, &numrecords)) != 0) + &leftover, &records, &numrecords)) != 0) { + zpool_standard_error_fmt(hdl, err, + dgettext(TEXT_DOMAIN, + "cannot get history for '%s'"), zhp->zpool_name); break; + } *off -= leftover; if (leftover == bytes_read) { /* @@ -4540,18 +4476,16 @@ zpool_get_history(zpool_handle_t *zhp, nvlist_t **nvhisp, uint64_t *off, */ buflen *= 2; free(buf); - buf = malloc(buflen); - if (buf == NULL) - return (ENOMEM); + buf = zfs_alloc(hdl, buflen); } } free(buf); if (!err) { - verify(nvlist_alloc(nvhisp, NV_UNIQUE_NAME, 0) == 0); - verify(nvlist_add_nvlist_array(*nvhisp, ZPOOL_HIST_RECORD, - (const nvlist_t **)records, numrecords) == 0); + *nvhisp = fnvlist_alloc(); + fnvlist_add_nvlist_array(*nvhisp, ZPOOL_HIST_RECORD, + (const nvlist_t **)records, numrecords); } for (i = 0; i < numrecords; i++) nvlist_free(records[i]); @@ -4585,8 +4519,7 @@ zpool_events_next(libzfs_handle_t *hdl, nvlist_t **nvp, if (flags & ZEVENT_NONBLOCK) zc.zc_guid = ZEVENT_NONBLOCK; - if (zcmd_alloc_dst_nvlist(hdl, &zc, ZEVENT_SIZE) != 0) - return (-1); + zcmd_alloc_dst_nvlist(hdl, &zc, ZEVENT_SIZE); retry: if (zfs_ioctl(hdl, ZFS_IOC_EVENTS_NEXT, &zc) != 0) { @@ -4603,13 +4536,8 @@ zpool_events_next(libzfs_handle_t *hdl, nvlist_t **nvp, goto out; case ENOMEM: - if (zcmd_expand_dst_nvlist(hdl, &zc) != 0) { - error = zfs_error_fmt(hdl, EZFS_NOMEM, - dgettext(TEXT_DOMAIN, "cannot get event")); - goto out; - } else { - goto retry; - } + zcmd_expand_dst_nvlist(hdl, &zc); + goto retry; default: error = zpool_standard_error_fmt(hdl, errno, dgettext(TEXT_DOMAIN, "cannot get event")); @@ -5073,7 +5001,7 @@ zpool_vdev_guid(zpool_handle_t *zhp, const char *vdevname, uint64_t *vdev_guid) verify(zhp != NULL); if (zpool_get_state(zhp) == POOL_STATE_UNAVAIL) { - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "pool is in an unavailable state")); return (zfs_error(zhp->zpool_hdl, EZFS_POOLUNAVAIL, errbuf)); @@ -5081,14 +5009,14 @@ zpool_vdev_guid(zpool_handle_t *zhp, const char *vdevname, uint64_t *vdev_guid) if ((tgt = zpool_find_vdev(zhp, vdevname, &avail_spare, &l2cache, NULL)) == NULL) { - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "can not find %s in %s"), vdevname, zhp->zpool_name); return (zfs_error(zhp->zpool_hdl, EZFS_NODEVICE, errbuf)); } - verify(nvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID, vdev_guid) == 0); + *vdev_guid = fnvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID); return (0); } @@ -5101,19 +5029,16 @@ zpool_get_vdev_prop_value(nvlist_t *nvprop, vdev_prop_t prop, char *prop_name, char *buf, size_t len, zprop_source_t *srctype, boolean_t literal) { nvlist_t *nv; + const char *strval; uint64_t intval; - char *strval; zprop_source_t src = ZPROP_SRC_NONE; - if (prop == VDEV_PROP_USER) { + if (prop == VDEV_PROP_USERPROP) { /* user property, prop_name must contain the property name */ assert(prop_name != NULL); if (nvlist_lookup_nvlist(nvprop, prop_name, &nv) == 0) { - verify(nvlist_lookup_uint64(nv, ZPROP_SOURCE, - &intval) == 0); - src = intval; - verify(nvlist_lookup_string(nv, ZPROP_VALUE, - &strval) == 0); + src = fnvlist_lookup_uint64(nv, ZPROP_SOURCE); + strval = fnvlist_lookup_string(nv, ZPROP_VALUE); } else { /* user prop not found */ return (-1); @@ -5130,15 +5055,11 @@ zpool_get_vdev_prop_value(nvlist_t *nvprop, vdev_prop_t prop, char *prop_name, switch (vdev_prop_get_type(prop)) { case PROP_TYPE_STRING: if (nvlist_lookup_nvlist(nvprop, prop_name, &nv) == 0) { - verify(nvlist_lookup_uint64(nv, ZPROP_SOURCE, - &intval) == 0); - src = intval; - verify(nvlist_lookup_string(nv, ZPROP_VALUE, - &strval) == 0); + src = fnvlist_lookup_uint64(nv, ZPROP_SOURCE); + strval = fnvlist_lookup_string(nv, ZPROP_VALUE); } else { src = ZPROP_SRC_DEFAULT; - if ((strval = (char *)vdev_prop_default_string(prop)) - == NULL) + if ((strval = vdev_prop_default_string(prop)) == NULL) strval = "-"; } (void) strlcpy(buf, strval, len); @@ -5146,11 +5067,8 @@ zpool_get_vdev_prop_value(nvlist_t *nvprop, vdev_prop_t prop, char *prop_name, case PROP_TYPE_NUMBER: if (nvlist_lookup_nvlist(nvprop, prop_name, &nv) == 0) { - verify(nvlist_lookup_uint64(nv, ZPROP_SOURCE, - &intval) == 0); - src = intval; - verify(nvlist_lookup_uint64(nv, ZPROP_VALUE, - &intval) == 0); + src = fnvlist_lookup_uint64(nv, ZPROP_SOURCE); + intval = fnvlist_lookup_uint64(nv, ZPROP_VALUE); } else { src = ZPROP_SRC_DEFAULT; intval = vdev_prop_default_numeric(prop); @@ -5160,6 +5078,7 @@ zpool_get_vdev_prop_value(nvlist_t *nvprop, vdev_prop_t prop, char *prop_name, case VDEV_PROP_ASIZE: case VDEV_PROP_PSIZE: case VDEV_PROP_SIZE: + case VDEV_PROP_BOOTSIZE: case VDEV_PROP_ALLOCATED: case VDEV_PROP_FREE: case VDEV_PROP_READ_ERRORS: @@ -5229,11 +5148,8 @@ zpool_get_vdev_prop_value(nvlist_t *nvprop, vdev_prop_t prop, char *prop_name, case PROP_TYPE_INDEX: if (nvlist_lookup_nvlist(nvprop, prop_name, &nv) == 0) { - verify(nvlist_lookup_uint64(nv, ZPROP_SOURCE, - &intval) == 0); - src = intval; - verify(nvlist_lookup_uint64(nv, ZPROP_VALUE, - &intval) == 0); + src = fnvlist_lookup_uint64(nv, ZPROP_SOURCE); + intval = fnvlist_lookup_uint64(nv, ZPROP_VALUE); } else { src = ZPROP_SRC_DEFAULT; intval = vdev_prop_default_numeric(prop); @@ -5265,7 +5181,7 @@ zpool_get_vdev_prop(zpool_handle_t *zhp, const char *vdevname, vdev_prop_t prop, { nvlist_t *reqnvl, *reqprops; nvlist_t *retprops = NULL; - uint64_t vdev_guid; + uint64_t vdev_guid = 0; int ret; if ((ret = zpool_vdev_guid(zhp, vdevname, &vdev_guid)) != 0) @@ -5278,7 +5194,7 @@ zpool_get_vdev_prop(zpool_handle_t *zhp, const char *vdevname, vdev_prop_t prop, fnvlist_add_uint64(reqnvl, ZPOOL_VDEV_PROPS_GET_VDEV, vdev_guid); - if (prop != VDEV_PROP_USER) { + if (prop != VDEV_PROP_USERPROP) { /* prop_name overrides prop value */ if (prop_name != NULL) prop = vdev_name_to_prop(prop_name); @@ -5302,7 +5218,7 @@ zpool_get_vdev_prop(zpool_handle_t *zhp, const char *vdevname, vdev_prop_t prop, ret = zpool_get_vdev_prop_value(retprops, prop, prop_name, buf, len, srctype, literal); } else { - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot get vdev property %s from" " %s in %s"), prop_name, vdevname, zhp->zpool_name); @@ -5324,7 +5240,7 @@ zpool_get_all_vdev_props(zpool_handle_t *zhp, const char *vdevname, nvlist_t **outnvl) { nvlist_t *nvl = NULL; - uint64_t vdev_guid; + uint64_t vdev_guid = 0; int ret; if ((ret = zpool_vdev_guid(zhp, vdevname, &vdev_guid)) != 0) @@ -5340,7 +5256,7 @@ zpool_get_all_vdev_props(zpool_handle_t *zhp, const char *vdevname, nvlist_free(nvl); if (ret) { - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot get vdev properties for" " %s in %s"), vdevname, zhp->zpool_name); @@ -5381,7 +5297,7 @@ zpool_set_vdev_prop(zpool_handle_t *zhp, const char *vdevname, return (no_memory(zhp->zpool_hdl)); } - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot set property %s for %s on %s"), propname, vdevname, zhp->zpool_name); diff --git a/lib/libzfs/libzfs_sendrecv.c b/lib/libzfs/libzfs_sendrecv.c index 2ee4beb3c93a..640051e3b029 100644 --- a/lib/libzfs/libzfs_sendrecv.c +++ b/lib/libzfs/libzfs_sendrecv.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -38,7 +38,7 @@ #include #include #include -#include +#include #include #include #include @@ -294,11 +294,13 @@ send_iterate_snap(zfs_handle_t *zhp, void *arg) uint64_t guid = zhp->zfs_dmustats.dds_guid; uint64_t txg = zhp->zfs_dmustats.dds_creation_txg; boolean_t isfromsnap, istosnap, istosnapwithnofrom; - char *snapname = strrchr(zhp->zfs_name, '@') + 1; + char *snapname; const char *from = sd->fromsnap; const char *to = sd->tosnap; - assert(snapname != (NULL + 1)); + snapname = strrchr(zhp->zfs_name, '@'); + assert(snapname != NULL); + ++snapname; isfromsnap = (from != NULL && strcmp(from, snapname) == 0); istosnap = (to != NULL && strcmp(to, snapname) == 0); @@ -434,6 +436,29 @@ send_iterate_prop(zfs_handle_t *zhp, boolean_t received_only, nvlist_t *nv) } } +/* + * returns snapshot guid + * and returns 0 if the snapshot does not exist + */ +static uint64_t +get_snap_guid(libzfs_handle_t *hdl, const char *fs, const char *snap) +{ + char name[MAXPATHLEN + 1]; + uint64_t guid = 0; + + if (fs == NULL || fs[0] == '\0' || snap == NULL || snap[0] == '\0') + return (guid); + + (void) snprintf(name, sizeof (name), "%s@%s", fs, snap); + zfs_handle_t *zhp = zfs_open(hdl, name, ZFS_TYPE_SNAPSHOT); + if (zhp != NULL) { + guid = zfs_prop_get_int(zhp, ZFS_PROP_GUID); + zfs_close(zhp); + } + + return (guid); +} + /* * returns snapshot creation txg * and returns 0 if the snapshot does not exist @@ -732,7 +757,7 @@ zfs_send_space(zfs_handle_t *zhp, const char *snapname, const char *from, if (error == 0) return (0); - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "warning: cannot estimate space for '%s'"), snapname); @@ -802,11 +827,12 @@ dump_ioctl(zfs_handle_t *zhp, const char *fromsnap, uint64_t fromsnap_obj, } if (zfs_ioctl(zhp->zfs_hdl, ZFS_IOC_SEND, &zc) != 0) { - char errbuf[1024]; + char errbuf[ERRBUFLEN]; int error = errno; - (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, - "warning: cannot send '%s'"), zhp->zfs_name); + (void) snprintf(errbuf, sizeof (errbuf), "%s '%s'", + dgettext(TEXT_DOMAIN, "warning: cannot send"), + zhp->zfs_name); if (debugnv != NULL) { fnvlist_add_uint64(thisdbg, "error", error); @@ -848,6 +874,11 @@ dump_ioctl(zfs_handle_t *zhp, const char *fromsnap, uint64_t fromsnap_obj, case EINVAL: zfs_error_aux(hdl, "%s", strerror(errno)); return (zfs_error(hdl, EZFS_BADBACKUP, errbuf)); + case ENOTSUP: + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "large blocks detected but large_blocks feature " + "is inactive; raw send unsupported")); + return (zfs_error(hdl, EZFS_NOTSUP, errbuf)); default: return (zfs_standard_error(hdl, errno, errbuf)); @@ -907,7 +938,7 @@ send_progress_thread(void *arg) uint64_t blocks; char buf[16]; time_t t; - struct tm *tm; + struct tm tm; int err; if (!pa->pa_parsable) { @@ -931,33 +962,47 @@ send_progress_thread(void *arg) } (void) time(&t); - tm = localtime(&t); + localtime_r(&t, &tm); if (pa->pa_verbosity >= 2 && pa->pa_parsable) { (void) fprintf(stderr, "%02d:%02d:%02d\t%llu\t%llu\t%s\n", - tm->tm_hour, tm->tm_min, tm->tm_sec, + tm.tm_hour, tm.tm_min, tm.tm_sec, (u_longlong_t)bytes, (u_longlong_t)blocks, zhp->zfs_name); } else if (pa->pa_verbosity >= 2) { zfs_nicenum(bytes, buf, sizeof (buf)); (void) fprintf(stderr, "%02d:%02d:%02d %5s %8llu %s\n", - tm->tm_hour, tm->tm_min, tm->tm_sec, + tm.tm_hour, tm.tm_min, tm.tm_sec, buf, (u_longlong_t)blocks, zhp->zfs_name); } else if (pa->pa_parsable) { (void) fprintf(stderr, "%02d:%02d:%02d\t%llu\t%s\n", - tm->tm_hour, tm->tm_min, tm->tm_sec, + tm.tm_hour, tm.tm_min, tm.tm_sec, (u_longlong_t)bytes, zhp->zfs_name); } else { zfs_nicebytes(bytes, buf, sizeof (buf)); (void) fprintf(stderr, "%02d:%02d:%02d %5s %s\n", - tm->tm_hour, tm->tm_min, tm->tm_sec, + tm.tm_hour, tm.tm_min, tm.tm_sec, buf, zhp->zfs_name); } } } +static boolean_t +send_progress_thread_exit(libzfs_handle_t *hdl, pthread_t ptid) +{ + void *status = NULL; + (void) pthread_cancel(ptid); + (void) pthread_join(ptid, &status); + int error = (int)(uintptr_t)status; + if (error != 0 && status != PTHREAD_CANCELED) + return (zfs_standard_error(hdl, error, + dgettext(TEXT_DOMAIN, "progress thread exited nonzero"))); + else + return (B_FALSE); +} + static void send_print_verbose(FILE *fout, const char *tosnap, const char *fromsnap, uint64_t size, boolean_t parsable) @@ -1130,17 +1175,9 @@ dump_snapshot(zfs_handle_t *zhp, void *arg) err = dump_ioctl(zhp, sdd->prevsnap, sdd->prevsnap_obj, fromorigin, sdd->outfd, flags, sdd->debugnv); - if (sdd->progress) { - void *status = NULL; - (void) pthread_cancel(tid); - (void) pthread_join(tid, &status); - int error = (int)(uintptr_t)status; - if (error != 0 && status != PTHREAD_CANCELED) { - return (zfs_standard_error(zhp->zfs_hdl, error, - dgettext(TEXT_DOMAIN, - "progress thread exited nonzero"))); - } - } + if (sdd->progress && + send_progress_thread_exit(zhp->zfs_hdl, tid)) + return (-1); } (void) strcpy(sdd->prevsnap, thissnap); @@ -1502,20 +1539,8 @@ estimate_size(zfs_handle_t *zhp, const char *from, int fd, sendflags_t *flags, lzc_flags_from_sendflags(flags), resumeobj, resumeoff, bytes, redactbook, fd, &size); - if (flags->progress) { - void *status = NULL; - (void) pthread_cancel(ptid); - (void) pthread_join(ptid, &status); - int error = (int)(uintptr_t)status; - if (error != 0 && status != PTHREAD_CANCELED) { - char errbuf[1024]; - (void) snprintf(errbuf, sizeof (errbuf), - dgettext(TEXT_DOMAIN, "progress thread exited " - "nonzero")); - return (zfs_standard_error(zhp->zfs_hdl, error, - errbuf)); - } - } + if (flags->progress && send_progress_thread_exit(zhp->zfs_hdl, ptid)) + return (-1); if (err != 0) { zfs_error_aux(zhp->zfs_hdl, "%s", strerror(err)); @@ -1618,7 +1643,7 @@ find_redact_book(libzfs_handle_t *hdl, const char *path, const uint64_t *redact_snap_guids, int num_redact_snaps, char **bookname) { - char errbuf[1024]; + char errbuf[ERRBUFLEN]; nvlist_t *bmarks; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, @@ -1679,10 +1704,10 @@ lzc_flags_from_resume_nvl(nvlist_t *resume_nvl) } static int -zfs_send_resume_impl(libzfs_handle_t *hdl, sendflags_t *flags, int outfd, - nvlist_t *resume_nvl) +zfs_send_resume_impl_cb_impl(libzfs_handle_t *hdl, sendflags_t *flags, + int outfd, nvlist_t *resume_nvl) { - char errbuf[1024]; + char errbuf[ERRBUFLEN]; char *toname; char *fromname = NULL; uint64_t resumeobj, resumeoff, toguid, fromguid, bytes; @@ -1827,21 +1852,10 @@ zfs_send_resume_impl(libzfs_handle_t *hdl, sendflags_t *flags, int outfd, if (redact_book != NULL) free(redact_book); - if (flags->progress) { - void *status = NULL; - (void) pthread_cancel(tid); - (void) pthread_join(tid, &status); - int error = (int)(uintptr_t)status; - if (error != 0 && status != PTHREAD_CANCELED) { - char errbuf[1024]; - (void) snprintf(errbuf, sizeof (errbuf), - dgettext(TEXT_DOMAIN, - "progress thread exited nonzero")); - return (zfs_standard_error(hdl, error, errbuf)); - } - } + if (flags->progress && send_progress_thread_exit(hdl, tid)) + return (-1); - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "warning: cannot send '%s'"), zhp->zfs_name); @@ -1890,12 +1904,38 @@ zfs_send_resume_impl(libzfs_handle_t *hdl, sendflags_t *flags, int outfd, return (error); } +struct zfs_send_resume_impl { + libzfs_handle_t *hdl; + sendflags_t *flags; + nvlist_t *resume_nvl; +}; + +static int +zfs_send_resume_impl_cb(int outfd, void *arg) +{ + struct zfs_send_resume_impl *zsri = arg; + return (zfs_send_resume_impl_cb_impl(zsri->hdl, zsri->flags, outfd, + zsri->resume_nvl)); +} + +static int +zfs_send_resume_impl(libzfs_handle_t *hdl, sendflags_t *flags, int outfd, + nvlist_t *resume_nvl) +{ + struct zfs_send_resume_impl zsri = { + .hdl = hdl, + .flags = flags, + .resume_nvl = resume_nvl, + }; + return (lzc_send_wrapper(zfs_send_resume_impl_cb, outfd, &zsri)); +} + int zfs_send_resume(libzfs_handle_t *hdl, sendflags_t *flags, int outfd, const char *resume_token) { int ret; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; nvlist_t *resume_nvl; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, @@ -1926,7 +1966,7 @@ zfs_send_saved(zfs_handle_t *zhp, sendflags_t *flags, int outfd, uint64_t saved_guid = 0, resume_guid = 0; uint64_t obj = 0, off = 0, bytes = 0; char token_buf[ZFS_MAXPROPLEN]; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "saved send failed")); @@ -2048,9 +2088,9 @@ send_prelim_records(zfs_handle_t *zhp, const char *from, int fd, /* name of filesystem/volume that contains snapshot we are sending */ char tofs[ZFS_MAX_DATASET_NAME_LEN]; /* short name of snap we are sending */ - char *tosnap = ""; + const char *tosnap = ""; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "warning: cannot send '%s'"), zhp->zfs_name); if (zhp->zfs_type == ZFS_TYPE_FILESYSTEM && zfs_prop_get_int(zhp, @@ -2167,13 +2207,15 @@ send_prelim_records(zfs_handle_t *zhp, const char *from, int fd, * if "replicate" is set. If "doall" is set, dump all the intermediate * snapshots. The DMU_COMPOUNDSTREAM header is used in the "doall" * case too. If "props" is set, send properties. + * + * Pre-wrapped (cf. lzc_send_wrapper()). */ -int -zfs_send(zfs_handle_t *zhp, const char *fromsnap, const char *tosnap, +static int +zfs_send_cb_impl(zfs_handle_t *zhp, const char *fromsnap, const char *tosnap, sendflags_t *flags, int outfd, snapfilter_cb_t filter_func, void *cb_arg, nvlist_t **debugnvp) { - char errbuf[1024]; + char errbuf[ERRBUFLEN]; send_dump_data_t sdd = { 0 }; int err = 0; nvlist_t *fss = NULL; @@ -2352,9 +2394,9 @@ zfs_send(zfs_handle_t *zhp, const char *fromsnap, const char *tosnap, * there was some error, because it might not be totally * failed. */ - err = send_conclusion_record(outfd, NULL); - if (err != 0) - return (zfs_standard_error(zhp->zfs_hdl, err, errbuf)); + int err2 = send_conclusion_record(outfd, NULL); + if (err2 != 0) + return (zfs_standard_error(zhp->zfs_hdl, err2, errbuf)); } return (err || sdd.err); @@ -2371,6 +2413,42 @@ zfs_send(zfs_handle_t *zhp, const char *fromsnap, const char *tosnap, return (err); } +struct zfs_send { + zfs_handle_t *zhp; + const char *fromsnap; + const char *tosnap; + sendflags_t *flags; + snapfilter_cb_t *filter_func; + void *cb_arg; + nvlist_t **debugnvp; +}; + +static int +zfs_send_cb(int outfd, void *arg) +{ + struct zfs_send *zs = arg; + return (zfs_send_cb_impl(zs->zhp, zs->fromsnap, zs->tosnap, zs->flags, + outfd, zs->filter_func, zs->cb_arg, zs->debugnvp)); +} + +int +zfs_send(zfs_handle_t *zhp, const char *fromsnap, const char *tosnap, + sendflags_t *flags, int outfd, snapfilter_cb_t filter_func, + void *cb_arg, nvlist_t **debugnvp) +{ + struct zfs_send arg = { + .zhp = zhp, + .fromsnap = fromsnap, + .tosnap = tosnap, + .flags = flags, + .filter_func = filter_func, + .cb_arg = cb_arg, + .debugnvp = debugnvp, + }; + return (lzc_send_wrapper(zfs_send_cb, outfd, &arg)); +} + + static zfs_handle_t * name_to_dir_handle(libzfs_handle_t *hdl, const char *snapname) { @@ -2447,10 +2525,12 @@ snapshot_is_before(zfs_handle_t *earlier, zfs_handle_t *later) * The "zhp" argument is the handle of the dataset to send (typically a * snapshot). The "from" argument is the full name of the snapshot or * bookmark that is the incremental source. + * + * Pre-wrapped (cf. lzc_send_wrapper()). */ -int -zfs_send_one(zfs_handle_t *zhp, const char *from, int fd, sendflags_t *flags, - const char *redactbook) +static int +zfs_send_one_cb_impl(zfs_handle_t *zhp, const char *from, int fd, + sendflags_t *flags, const char *redactbook) { int err; libzfs_handle_t *hdl = zhp->zfs_hdl; @@ -2458,7 +2538,7 @@ zfs_send_one(zfs_handle_t *zhp, const char *from, int fd, sendflags_t *flags, pthread_t ptid; progress_arg_t pa = { 0 }; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "warning: cannot send '%s'"), name); @@ -2572,17 +2652,8 @@ zfs_send_one(zfs_handle_t *zhp, const char *from, int fd, sendflags_t *flags, err = lzc_send_redacted(name, from, fd, lzc_flags_from_sendflags(flags), redactbook); - if (flags->progress) { - void *status = NULL; - if (err != 0) - (void) pthread_cancel(ptid); - (void) pthread_join(ptid, &status); - int error = (int)(uintptr_t)status; - if (error != 0 && status != PTHREAD_CANCELED) - return (zfs_standard_error_fmt(hdl, error, - dgettext(TEXT_DOMAIN, - "progress thread exited nonzero"))); - } + if (flags->progress && send_progress_thread_exit(hdl, ptid)) + return (-1); if (err == 0 && (flags->props || flags->holds || flags->backup)) { /* Write the final end record. */ @@ -2631,6 +2702,11 @@ zfs_send_one(zfs_handle_t *zhp, const char *from, int fd, sendflags_t *flags, case EROFS: zfs_error_aux(hdl, "%s", strerror(errno)); return (zfs_error(hdl, EZFS_BADBACKUP, errbuf)); + case ENOTSUP: + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "large blocks detected but large_blocks feature " + "is inactive; raw send unsupported")); + return (zfs_error(hdl, EZFS_NOTSUP, errbuf)); default: return (zfs_standard_error(hdl, errno, errbuf)); @@ -2639,6 +2715,34 @@ zfs_send_one(zfs_handle_t *zhp, const char *from, int fd, sendflags_t *flags, return (err != 0); } +struct zfs_send_one { + zfs_handle_t *zhp; + const char *from; + sendflags_t *flags; + const char *redactbook; +}; + +static int +zfs_send_one_cb(int fd, void *arg) +{ + struct zfs_send_one *zso = arg; + return (zfs_send_one_cb_impl(zso->zhp, zso->from, fd, zso->flags, + zso->redactbook)); +} + +int +zfs_send_one(zfs_handle_t *zhp, const char *from, int fd, sendflags_t *flags, + const char *redactbook) +{ + struct zfs_send_one zso = { + .zhp = zhp, + .from = from, + .flags = flags, + .redactbook = redactbook, + }; + return (lzc_send_wrapper(zfs_send_one_cb, fd, &zso)); +} + /* * Routines specific to "zfs recv" */ @@ -2681,8 +2785,6 @@ recv_read_nvlist(libzfs_handle_t *hdl, int fd, int len, nvlist_t **nvp, int err; buf = zfs_alloc(hdl, len); - if (buf == NULL) - return (ENOMEM); if (len > hdl->libzfs_max_nvlist) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "nvlist too large")); @@ -3424,12 +3526,10 @@ recv_incremental_replication(libzfs_handle_t *hdl, const char *tofs, zc.zc_cookie = B_TRUE; /* received */ (void) snprintf(zc.zc_name, sizeof (zc.zc_name), "%s@%s", fsname, nvpair_name(snapelem)); - if (zcmd_write_src_nvlist(hdl, &zc, - props) == 0) { - (void) zfs_ioctl(hdl, - ZFS_IOC_SET_PROP, &zc); - zcmd_free_nvlists(&zc); - } + zcmd_write_src_nvlist(hdl, &zc, props); + (void) zfs_ioctl(hdl, + ZFS_IOC_SET_PROP, &zc); + zcmd_free_nvlists(&zc); } /* check for different snapname */ @@ -3587,7 +3687,7 @@ zfs_receive_package(libzfs_handle_t *hdl, int fd, const char *destname, char *cp; char tofs[ZFS_MAX_DATASET_NAME_LEN]; char sendfs[ZFS_MAX_DATASET_NAME_LEN]; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; dmu_replay_record_t drre; int error; boolean_t anyerr = B_FALSE; @@ -3804,7 +3904,7 @@ recv_skip(libzfs_handle_t *hdl, int fd, boolean_t byteswap) dmu_replay_record_t *drr; void *buf = zfs_alloc(hdl, SPA_MAXBLOCKSIZE); uint64_t payload_size; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot receive")); @@ -4168,11 +4268,11 @@ zfs_receive_one(libzfs_handle_t *hdl, int infd, const char *tosnap, avl_tree_t *stream_avl, char **top_zfs, const char *finalsnap, nvlist_t *cmdprops) { - time_t begin_time; + struct timespec begin_time; int ioctl_err, ioctl_errno, err; char *cp; struct drr_begin *drrb = &drr->drr_u.drr_begin; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; const char *chopprefix; boolean_t newfs = B_FALSE; boolean_t stream_wantsnewfs, stream_resumingnewfs; @@ -4188,13 +4288,13 @@ zfs_receive_one(libzfs_handle_t *hdl, int infd, const char *tosnap, boolean_t recursive; char *snapname = NULL; char destsnap[MAXPATHLEN * 2]; - char origin[MAXNAMELEN]; + char origin[MAXNAMELEN] = {0}; char name[MAXPATHLEN]; - char tmp_keylocation[MAXNAMELEN]; + char tmp_keylocation[MAXNAMELEN] = {0}; nvlist_t *rcvprops = NULL; /* props received from the send stream */ nvlist_t *oxprops = NULL; /* override (-o) and exclude (-x) props */ nvlist_t *origprops = NULL; /* original props (if destination exists) */ - zfs_type_t type; + zfs_type_t type = ZFS_TYPE_INVALID; boolean_t toplevel = B_FALSE; boolean_t zoned = B_FALSE; boolean_t hastoken = B_FALSE; @@ -4202,9 +4302,10 @@ zfs_receive_one(libzfs_handle_t *hdl, int infd, const char *tosnap, uint8_t *wkeydata = NULL; uint_t wkeylen = 0; - begin_time = time(NULL); - bzero(origin, MAXNAMELEN); - bzero(tmp_keylocation, MAXNAMELEN); +#ifndef CLOCK_MONOTONIC_RAW +#define CLOCK_MONOTONIC_RAW CLOCK_MONOTONIC +#endif + clock_gettime(CLOCK_MONOTONIC_RAW, &begin_time); (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot receive")); @@ -4463,9 +4564,34 @@ zfs_receive_one(libzfs_handle_t *hdl, int infd, const char *tosnap, redacted = DMU_GET_FEATUREFLAGS(drrb->drr_versioninfo) & DMU_BACKUP_FEATURE_REDACTED; - if (zfs_dataset_exists(hdl, name, ZFS_TYPE_DATASET)) { + if (flags->heal) { + if (flags->isprefix || flags->istail || flags->force || + flags->canmountoff || flags->resumable || flags->nomount || + flags->skipholds) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "corrective recv can not be used when combined with" + " this flag")); + err = zfs_error(hdl, EZFS_INVALIDNAME, errbuf); + goto out; + } + uint64_t guid = + get_snap_guid(hdl, name, strchr(destsnap, '@') + 1); + if (guid == 0) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "corrective recv must specify an existing snapshot" + " to heal")); + err = zfs_error(hdl, EZFS_INVALIDNAME, errbuf); + goto out; + } else if (guid != drrb->drr_toguid) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "local snapshot doesn't match the snapshot" + " in the provided stream")); + err = zfs_error(hdl, EZFS_WRONG_PARENT, errbuf); + goto out; + } + } else if (zfs_dataset_exists(hdl, name, ZFS_TYPE_DATASET)) { zfs_cmd_t zc = {"\0"}; - zfs_handle_t *zhp; + zfs_handle_t *zhp = NULL; boolean_t encrypted; (void) strcpy(zc.zc_name, name); @@ -4659,8 +4785,9 @@ zfs_receive_one(libzfs_handle_t *hdl, int infd, const char *tosnap, } if (flags->verbose) { - (void) printf("%s %s stream of %s into %s\n", + (void) printf("%s %s%s stream of %s into %s\n", flags->dryrun ? "would receive" : "receiving", + flags->heal ? " corrective" : "", drrb->drr_fromguid ? "incremental" : "full", drrb->drr_toname, destsnap); (void) fflush(stdout); @@ -4730,10 +4857,17 @@ zfs_receive_one(libzfs_handle_t *hdl, int infd, const char *tosnap, goto out; } - err = ioctl_err = lzc_receive_with_cmdprops(destsnap, rcvprops, - oxprops, wkeydata, wkeylen, origin, flags->force, flags->resumable, - raw, infd, drr_noswap, -1, &read_bytes, &errflags, - NULL, &prop_errors); + if (flags->heal) { + err = ioctl_err = lzc_receive_with_heal(destsnap, rcvprops, + oxprops, wkeydata, wkeylen, origin, flags->force, + flags->heal, flags->resumable, raw, infd, drr_noswap, -1, + &read_bytes, &errflags, NULL, &prop_errors); + } else { + err = ioctl_err = lzc_receive_with_cmdprops(destsnap, rcvprops, + oxprops, wkeydata, wkeylen, origin, flags->force, + flags->resumable, raw, infd, drr_noswap, -1, &read_bytes, + &errflags, NULL, &prop_errors); + } ioctl_errno = ioctl_err; prop_errflags = errflags; @@ -4781,10 +4915,9 @@ zfs_receive_one(libzfs_handle_t *hdl, int infd, const char *tosnap, (void) strcpy(zc.zc_name, destsnap); zc.zc_cookie = B_TRUE; /* received */ - if (zcmd_write_src_nvlist(hdl, &zc, snapprops_nvlist) == 0) { - (void) zfs_ioctl(hdl, ZFS_IOC_SET_PROP, &zc); - zcmd_free_nvlists(&zc); - } + zcmd_write_src_nvlist(hdl, &zc, snapprops_nvlist); + (void) zfs_ioctl(hdl, ZFS_IOC_SET_PROP, &zc); + zcmd_free_nvlists(&zc); } if (err == 0 && snapholds_nvlist) { nvpair_t *pair; @@ -4856,7 +4989,12 @@ zfs_receive_one(libzfs_handle_t *hdl, int infd, const char *tosnap, (void) zfs_error(hdl, EZFS_BADRESTORE, errbuf); break; case EACCES: - if (raw && stream_wantsnewfs) { + if (flags->heal) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "key must be loaded to do a non-raw " + "corrective recv on an encrypted " + "dataset.")); + } else if (raw && stream_wantsnewfs) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "failed to create encryption key")); } else if (raw && !stream_wantsnewfs) { @@ -4896,8 +5034,14 @@ zfs_receive_one(libzfs_handle_t *hdl, int infd, const char *tosnap, break; case ECKSUM: case ZFS_ERR_STREAM_TRUNCATED: - recv_ecksum_set_aux(hdl, destsnap, flags->resumable, - ioctl_err == ECKSUM); + if (flags->heal) + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "corrective receive was not able to " + "reconstruct the data needed for " + "healing.")); + else + recv_ecksum_set_aux(hdl, destsnap, + flags->resumable, ioctl_err == ECKSUM); (void) zfs_error(hdl, EZFS_BADSTREAM, errbuf); break; case ZFS_ERR_STREAM_LARGE_BLOCK_MISMATCH: @@ -4907,8 +5051,14 @@ zfs_receive_one(libzfs_handle_t *hdl, int infd, const char *tosnap, (void) zfs_error(hdl, EZFS_BADSTREAM, errbuf); break; case ENOTSUP: - zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, - "pool must be upgraded to receive this stream.")); + if (flags->heal) + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "stream is not compatible with the " + "data in the pool.")); + else + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "pool must be upgraded to receive this " + "stream.")); (void) zfs_error(hdl, EZFS_BADVERSION, errbuf); break; case EDQUOT: @@ -4987,14 +5137,23 @@ zfs_receive_one(libzfs_handle_t *hdl, int infd, const char *tosnap, char buf1[64]; char buf2[64]; uint64_t bytes = read_bytes; - time_t delta = time(NULL) - begin_time; - if (delta == 0) - delta = 1; + struct timespec delta; + clock_gettime(CLOCK_MONOTONIC_RAW, &delta); + if (begin_time.tv_nsec > delta.tv_nsec) { + delta.tv_nsec = + 1000000000 + delta.tv_nsec - begin_time.tv_nsec; + delta.tv_sec -= 1; + } else + delta.tv_nsec -= begin_time.tv_nsec; + delta.tv_sec -= begin_time.tv_sec; + if (delta.tv_sec == 0 && delta.tv_nsec == 0) + delta.tv_nsec = 1; + double delta_f = delta.tv_sec + (delta.tv_nsec / 1e9); zfs_nicebytes(bytes, buf1, sizeof (buf1)); - zfs_nicebytes(bytes/delta, buf2, sizeof (buf1)); + zfs_nicebytes(bytes / delta_f, buf2, sizeof (buf2)); - (void) printf("received %s stream in %lld seconds (%s/sec)\n", - buf1, (longlong_t)delta, buf2); + (void) printf("received %s stream in %.2f seconds (%s/sec)\n", + buf1, delta_f, buf2); } err = 0; @@ -5031,7 +5190,7 @@ zfs_receive_checkprops(libzfs_handle_t *hdl, nvlist_t *props, name = nvpair_name(nvp); prop = zfs_name_to_prop(name); - if (prop == ZPROP_INVAL) { + if (prop == ZPROP_USERPROP) { if (!zfs_prop_user(name)) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "%s: invalid property '%s'"), errbuf, name); @@ -5075,7 +5234,7 @@ zfs_receive_impl(libzfs_handle_t *hdl, const char *tosnap, int err; dmu_replay_record_t drr, drr_noswap; struct drr_begin *drrb = &drr.drr_u.drr_begin; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; zio_cksum_t zcksum = { { 0 } }; uint64_t featureflags; int hdrtype; @@ -5119,7 +5278,7 @@ zfs_receive_impl(libzfs_handle_t *hdl, const char *tosnap, * We computed the checksum in the wrong byteorder in * recv_read() above; do it again correctly. */ - bzero(&zcksum, sizeof (zio_cksum_t)); + memset(&zcksum, 0, sizeof (zio_cksum_t)); fletcher_4_incremental_byteswap(&drr, sizeof (drr), &zcksum); flags->byteswap = B_TRUE; @@ -5228,13 +5387,6 @@ zfs_receive(libzfs_handle_t *hdl, const char *tosnap, nvlist_t *props, return (-2); } - /* - * It is not uncommon for gigabytes to be processed in zfs receive. - * Speculatively increase the buffer size if supported by the platform. - */ - if (S_ISFIFO(sb.st_mode)) - libzfs_set_pipe_max(infd); - if (props) { err = nvlist_lookup_string(props, "origin", &originsnap); if (err && err != ENOENT) diff --git a/lib/libzfs/libzfs_status.c b/lib/libzfs/libzfs_status.c index b7713b2bfe09..6999d9afc5cd 100644 --- a/lib/libzfs/libzfs_status.c +++ b/lib/libzfs/libzfs_status.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -56,7 +56,7 @@ * in include/libzfs.h. Note that there are some status results which go past * the end of this table, and hence have no associated message ID. */ -static char *zfs_msgid_table[] = { +static const char *const zfs_msgid_table[] = { "ZFS-8000-14", /* ZPOOL_STATUS_CORRUPT_CACHE */ "ZFS-8000-2Q", /* ZPOOL_STATUS_MISSING_DEV_R */ "ZFS-8000-3C", /* ZPOOL_STATUS_MISSING_DEV_NR */ @@ -159,8 +159,7 @@ find_vdev_problem(nvlist_t *vdev, int (*func)(vdev_stat_t *, uint_t), boolean_t ignore_replacing) { nvlist_t **child; - vdev_stat_t *vs; - uint_t c, vsc, children; + uint_t c, children; /* * Ignore problems within a 'replacing' vdev, since we're presumably in @@ -169,10 +168,7 @@ find_vdev_problem(nvlist_t *vdev, int (*func)(vdev_stat_t *, uint_t), * later. */ if (ignore_replacing == B_TRUE) { - char *type; - - verify(nvlist_lookup_string(vdev, ZPOOL_CONFIG_TYPE, - &type) == 0); + char *type = fnvlist_lookup_string(vdev, ZPOOL_CONFIG_TYPE); if (strcmp(type, VDEV_TYPE_REPLACING) == 0) return (B_FALSE); } @@ -183,9 +179,9 @@ find_vdev_problem(nvlist_t *vdev, int (*func)(vdev_stat_t *, uint_t), if (find_vdev_problem(child[c], func, ignore_replacing)) return (B_TRUE); } else { - verify(nvlist_lookup_uint64_array(vdev, ZPOOL_CONFIG_VDEV_STATS, - (uint64_t **)&vs, &vsc) == 0); - + uint_t vsc; + vdev_stat_t *vs = (vdev_stat_t *)fnvlist_lookup_uint64_array( + vdev, ZPOOL_CONFIG_VDEV_STATS, &vsc); if (func(vs, vsc) != 0) return (B_TRUE); } @@ -224,26 +220,21 @@ static zpool_status_t check_status(nvlist_t *config, boolean_t isimport, zpool_errata_t *erratap, const char *compat) { - nvlist_t *nvroot; - vdev_stat_t *vs; pool_scan_stat_t *ps = NULL; uint_t vsc, psc; uint64_t nerr; - uint64_t version; - uint64_t stateval; uint64_t suspended; uint64_t hostid = 0; uint64_t errata = 0; unsigned long system_hostid = get_system_hostid(); - verify(nvlist_lookup_uint64(config, ZPOOL_CONFIG_VERSION, - &version) == 0); - verify(nvlist_lookup_nvlist(config, ZPOOL_CONFIG_VDEV_TREE, - &nvroot) == 0); - verify(nvlist_lookup_uint64_array(nvroot, ZPOOL_CONFIG_VDEV_STATS, - (uint64_t **)&vs, &vsc) == 0); - verify(nvlist_lookup_uint64(config, ZPOOL_CONFIG_POOL_STATE, - &stateval) == 0); + uint64_t version = fnvlist_lookup_uint64(config, ZPOOL_CONFIG_VERSION); + nvlist_t *nvroot = fnvlist_lookup_nvlist(config, + ZPOOL_CONFIG_VDEV_TREE); + vdev_stat_t *vs = (vdev_stat_t *)fnvlist_lookup_uint64_array(nvroot, + ZPOOL_CONFIG_VDEV_STATS, &vsc); + uint64_t stateval = fnvlist_lookup_uint64(config, + ZPOOL_CONFIG_POOL_STATE); /* * Currently resilvering a vdev @@ -337,10 +328,8 @@ check_status(nvlist_t *config, boolean_t isimport, */ if (vs->vs_state == VDEV_STATE_CANT_OPEN && vs->vs_aux == VDEV_AUX_UNSUP_FEAT) { - nvlist_t *nvinfo; - - verify(nvlist_lookup_nvlist(config, ZPOOL_CONFIG_LOAD_INFO, - &nvinfo) == 0); + nvlist_t *nvinfo = fnvlist_lookup_nvlist(config, + ZPOOL_CONFIG_LOAD_INFO); if (nvlist_exists(nvinfo, ZPOOL_CONFIG_CAN_RDONLY)) return (ZPOOL_STATUS_UNSUP_FEAT_WRITE); return (ZPOOL_STATUS_UNSUP_FEAT_READ); @@ -506,7 +495,8 @@ check_status(nvlist_t *config, boolean_t isimport, } zpool_status_t -zpool_get_status(zpool_handle_t *zhp, char **msgid, zpool_errata_t *errata) +zpool_get_status(zpool_handle_t *zhp, const char **msgid, + zpool_errata_t *errata) { /* * pass in the desired feature set, as @@ -530,7 +520,8 @@ zpool_get_status(zpool_handle_t *zhp, char **msgid, zpool_errata_t *errata) } zpool_status_t -zpool_import_status(nvlist_t *config, char **msgid, zpool_errata_t *errata) +zpool_import_status(nvlist_t *config, const char **msgid, + zpool_errata_t *errata) { zpool_status_t ret = check_status(config, B_TRUE, errata, NULL); diff --git a/lib/libzfs/libzfs_util.c b/lib/libzfs/libzfs_util.c index 3e33ef503b35..cca86d2d7828 100644 --- a/lib/libzfs/libzfs_util.c +++ b/lib/libzfs/libzfs_util.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -299,6 +299,9 @@ libzfs_error_description(libzfs_handle_t *hdl) case EZFS_VDEV_NOTSUP: return (dgettext(TEXT_DOMAIN, "operation not supported " "on this type of vdev")); + case EZFS_NOT_USER_NAMESPACE: + return (dgettext(TEXT_DOMAIN, "the provided file " + "was not a user namespace file")); case EZFS_UNKNOWN: return (dgettext(TEXT_DOMAIN, "unknown error")); default: @@ -485,6 +488,9 @@ zfs_standard_error_fmt(libzfs_handle_t *hdl, int error, const char *fmt, ...) case ZFS_ERR_BADPROP: zfs_verror(hdl, EZFS_BADPROP, fmt, ap); break; + case ZFS_ERR_NOT_USER_NAMESPACE: + zfs_verror(hdl, EZFS_NOT_USER_NAMESPACE, fmt, ap); + break; default: zfs_error_aux(hdl, "%s", strerror(error)); zfs_verror(hdl, EZFS_UNKNOWN, fmt, ap); @@ -807,7 +813,7 @@ zfs_realloc(libzfs_handle_t *hdl, void *ptr, size_t oldsize, size_t newsize) return (NULL); } - bzero((char *)ret + oldsize, (newsize - oldsize)); + memset((char *)ret + oldsize, 0, newsize - oldsize); return (ret); } @@ -993,16 +999,13 @@ libzfs_free_str_array(char **strs, int count) * * Returns 0 otherwise. */ -int -libzfs_envvar_is_set(char *envvar) +boolean_t +libzfs_envvar_is_set(const char *envvar) { char *env = getenv(envvar); - if (env && (strtoul(env, NULL, 0) > 0 || + return (env && (strtoul(env, NULL, 0) > 0 || (!strncasecmp(env, "YES", 3) && strnlen(env, 4) == 3) || - (!strncasecmp(env, "ON", 2) && strnlen(env, 3) == 2))) - return (1); - - return (0); + (!strncasecmp(env, "ON", 2) && strnlen(env, 3) == 2))); } libzfs_handle_t * @@ -1149,7 +1152,7 @@ zfs_path_to_zhandle(libzfs_handle_t *hdl, const char *path, zfs_type_t argtype) * Initialize the zc_nvlist_dst member to prepare for receiving an nvlist from * an ioctl(). */ -int +void zcmd_alloc_dst_nvlist(libzfs_handle_t *hdl, zfs_cmd_t *zc, size_t len) { if (len == 0) @@ -1157,10 +1160,6 @@ zcmd_alloc_dst_nvlist(libzfs_handle_t *hdl, zfs_cmd_t *zc, size_t len) zc->zc_nvlist_dst_size = len; zc->zc_nvlist_dst = (uint64_t)(uintptr_t)zfs_alloc(hdl, zc->zc_nvlist_dst_size); - if (zc->zc_nvlist_dst == 0) - return (-1); - - return (0); } /* @@ -1168,16 +1167,12 @@ zcmd_alloc_dst_nvlist(libzfs_handle_t *hdl, zfs_cmd_t *zc, size_t len) * expand the nvlist to the size specified in 'zc_nvlist_dst_size', which was * filled in by the kernel to indicate the actual required size. */ -int +void zcmd_expand_dst_nvlist(libzfs_handle_t *hdl, zfs_cmd_t *zc) { free((void *)(uintptr_t)zc->zc_nvlist_dst); zc->zc_nvlist_dst = (uint64_t)(uintptr_t)zfs_alloc(hdl, zc->zc_nvlist_dst_size); - if (zc->zc_nvlist_dst == 0) - return (-1); - - return (0); } /* @@ -1194,38 +1189,33 @@ zcmd_free_nvlists(zfs_cmd_t *zc) zc->zc_nvlist_dst = 0; } -static int +static void zcmd_write_nvlist_com(libzfs_handle_t *hdl, uint64_t *outnv, uint64_t *outlen, nvlist_t *nvl) { char *packed; - size_t len; - verify(nvlist_size(nvl, &len, NV_ENCODE_NATIVE) == 0); - - if ((packed = zfs_alloc(hdl, len)) == NULL) - return (-1); + size_t len = fnvlist_size(nvl); + packed = zfs_alloc(hdl, len); verify(nvlist_pack(nvl, &packed, &len, NV_ENCODE_NATIVE, 0) == 0); *outnv = (uint64_t)(uintptr_t)packed; *outlen = len; - - return (0); } -int +void zcmd_write_conf_nvlist(libzfs_handle_t *hdl, zfs_cmd_t *zc, nvlist_t *nvl) { - return (zcmd_write_nvlist_com(hdl, &zc->zc_nvlist_conf, - &zc->zc_nvlist_conf_size, nvl)); + zcmd_write_nvlist_com(hdl, &zc->zc_nvlist_conf, + &zc->zc_nvlist_conf_size, nvl); } -int +void zcmd_write_src_nvlist(libzfs_handle_t *hdl, zfs_cmd_t *zc, nvlist_t *nvl) { - return (zcmd_write_nvlist_com(hdl, &zc->zc_nvlist_src, - &zc->zc_nvlist_src_size, nvl)); + zcmd_write_nvlist_com(hdl, &zc->zc_nvlist_src, + &zc->zc_nvlist_src_size, nvl); } /* @@ -1289,7 +1279,7 @@ zprop_print_headers(zprop_get_cbdata_t *cbp, zfs_type_t type) /* * 'PROPERTY' column */ - if (pl->pl_prop != ZPROP_INVAL) { + if (pl->pl_prop != ZPROP_USERPROP) { const char *propname = (type == ZFS_TYPE_POOL) ? zpool_prop_to_name(pl->pl_prop) : ((type == ZFS_TYPE_VDEV) ? @@ -1751,14 +1741,10 @@ zprop_parse_value(libzfs_handle_t *hdl, nvpair_t *elem, int prop, } static int -addlist(libzfs_handle_t *hdl, char *propname, zprop_list_t **listp, +addlist(libzfs_handle_t *hdl, const char *propname, zprop_list_t **listp, zfs_type_t type) { - int prop; - zprop_list_t *entry; - - prop = zprop_name_to_prop(propname, type); - + int prop = zprop_name_to_prop(propname, type); if (prop != ZPROP_INVAL && !zprop_valid_for_type(prop, type, B_FALSE)) prop = ZPROP_INVAL; @@ -1766,7 +1752,7 @@ addlist(libzfs_handle_t *hdl, char *propname, zprop_list_t **listp, * Return failure if no property table entry was found and this isn't * a user-defined property. */ - if (prop == ZPROP_INVAL && ((type == ZFS_TYPE_POOL && + if (prop == ZPROP_USERPROP && ((type == ZFS_TYPE_POOL && !zpool_prop_feature(propname) && !zpool_prop_unsupported(propname)) || ((type == ZFS_TYPE_DATASET) && !zfs_prop_user(propname) && @@ -1778,16 +1764,11 @@ addlist(libzfs_handle_t *hdl, char *propname, zprop_list_t **listp, dgettext(TEXT_DOMAIN, "bad property list"))); } - if ((entry = zfs_alloc(hdl, sizeof (zprop_list_t))) == NULL) - return (-1); + zprop_list_t *entry = zfs_alloc(hdl, sizeof (*entry)); entry->pl_prop = prop; - if (prop == ZPROP_INVAL) { - if ((entry->pl_user_prop = zfs_strdup(hdl, propname)) == - NULL) { - free(entry); - return (-1); - } + if (prop == ZPROP_USERPROP) { + entry->pl_user_prop = zfs_strdup(hdl, propname); entry->pl_width = strlen(propname); } else { entry->pl_width = zprop_width(prop, &entry->pl_fixed, @@ -1827,62 +1808,25 @@ zprop_get_list(libzfs_handle_t *hdl, char *props, zprop_list_t **listp, "bad property list"))); } - /* - * It would be nice to use getsubopt() here, but the inclusion of column - * aliases makes this more effort than it's worth. - */ - while (*props != '\0') { - size_t len; - char *p; - char c; - - if ((p = strchr(props, ',')) == NULL) { - len = strlen(props); - p = props + len; - } else { - len = p - props; - } - - /* - * Check for empty options. - */ - if (len == 0) { - zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, - "empty property name")); - return (zfs_error(hdl, EZFS_BADPROP, - dgettext(TEXT_DOMAIN, "bad property list"))); - } - - /* - * Check all regular property names. - */ - c = props[len]; - props[len] = '\0'; - - if (strcmp(props, "space") == 0) { - static char *spaceprops[] = { + for (char *p; (p = strsep(&props, ",")); ) + if (strcmp(p, "space") == 0) { + static const char *const spaceprops[] = { "name", "avail", "used", "usedbysnapshots", "usedbydataset", "usedbyrefreservation", - "usedbychildren", NULL + "usedbychildren" }; - int i; - for (i = 0; spaceprops[i]; i++) { + for (int i = 0; i < ARRAY_SIZE(spaceprops); i++) { if (addlist(hdl, spaceprops[i], listp, type)) return (-1); listp = &(*listp)->pl_next; } } else { - if (addlist(hdl, props, listp, type)) + if (addlist(hdl, p, listp, type)) return (-1); listp = &(*listp)->pl_next; } - props = p; - if (c == ',') - props++; - } - return (0); } @@ -1911,8 +1855,7 @@ zprop_expand_list_cb(int prop, void *cb) zprop_list_t *entry; expand_data_t *edp = cb; - if ((entry = zfs_alloc(edp->hdl, sizeof (zprop_list_t))) == NULL) - return (ZPROP_INVAL); + entry = zfs_alloc(edp->hdl, sizeof (zprop_list_t)); entry->pl_prop = prop; entry->pl_width = zprop_width(prop, &entry->pl_fixed, edp->type); @@ -1951,9 +1894,7 @@ zprop_expand_list(libzfs_handle_t *hdl, zprop_list_t **plp, zfs_type_t type) * Add 'name' to the beginning of the list, which is handled * specially. */ - if ((entry = zfs_alloc(hdl, sizeof (zprop_list_t))) == NULL) - return (-1); - + entry = zfs_alloc(hdl, sizeof (zprop_list_t)); entry->pl_prop = ((type == ZFS_TYPE_POOL) ? ZPOOL_PROP_NAME : ((type == ZFS_TYPE_VDEV) ? VDEV_PROP_NAME : ZFS_PROP_NAME)); entry->pl_width = zprop_width(entry->pl_prop, @@ -1972,37 +1913,30 @@ zprop_iter(zprop_func func, void *cb, boolean_t show_all, boolean_t ordered, return (zprop_iter_common(func, cb, show_all, ordered, type)); } -/* - * Fill given version buffer with zfs userland version - */ -void -zfs_version_userland(char *version, int len) +const char * +zfs_version_userland(void) { - (void) strlcpy(version, ZFS_META_ALIAS, len); + return (ZFS_META_ALIAS); } /* * Prints both zfs userland and kernel versions - * Returns 0 on success, and -1 on error (with errno set) + * Returns 0 on success, and -1 on error */ int zfs_version_print(void) { - char zver_userland[128]; - char zver_kernel[128]; + (void) puts(ZFS_META_ALIAS); - zfs_version_userland(zver_userland, sizeof (zver_userland)); - - (void) printf("%s\n", zver_userland); - - if (zfs_version_kernel(zver_kernel, sizeof (zver_kernel)) == -1) { + char *kver = zfs_version_kernel(); + if (kver == NULL) { fprintf(stderr, "zfs_version_kernel() failed: %s\n", strerror(errno)); return (-1); } - (void) printf("zfs-kmod-%s\n", zver_kernel); - + (void) printf("zfs-kmod-%s\n", kver); + free(kver); return (0); } @@ -2065,22 +1999,22 @@ use_color(void) * color_end(); */ void -color_start(char *color) +color_start(const char *color) { if (use_color()) - printf("%s", color); + fputs(color, stdout); } void color_end(void) { if (use_color()) - printf(ANSI_RESET); + fputs(ANSI_RESET, stdout); } /* printf() with a color. If color is NULL, then do a normal printf. */ int -printf_color(char *color, char *format, ...) +printf_color(const char *color, const char *format, ...) { va_list aptr; int rc; diff --git a/lib/libzfs/os/freebsd/libzfs_compat.c b/lib/libzfs/os/freebsd/libzfs_compat.c index e3f17662a11e..f4900943c4af 100644 --- a/lib/libzfs/os/freebsd/libzfs_compat.c +++ b/lib/libzfs/os/freebsd/libzfs_compat.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -38,11 +38,6 @@ #define ZFS_KMOD "openzfs" #endif -void -libzfs_set_pipe_max(int infd) -{ - /* FreeBSD automatically resizes */ -} static int execvPe(const char *name, const char *path, char * const *argv, @@ -108,9 +103,9 @@ execvPe(const char *name, const char *path, char * const *argv, 16); continue; } - bcopy(p, buf, lp); + memcpy(buf, p, lp); buf[lp] = '/'; - bcopy(name, buf + lp + 1, ln); + memcpy(buf + lp + 1, name, ln); buf[lp + ln + 1] = '\0'; retry: (void) execve(bp, argv, envp); @@ -140,7 +135,7 @@ retry: (void) execve(bp, argv, envp); if (cnt > 0) { memp[0] = argv[0]; memp[1] = bp; - bcopy(argv + 1, memp + 2, + memcpy(memp + 2, argv + 1, cnt * sizeof (char *)); } else { memp[0] = "sh"; @@ -198,18 +193,16 @@ execvpe(const char *name, char * const argv[], char * const envp[]) return (execvPe(name, path, argv, envp)); } -#define ERRBUFLEN 256 - static __thread char errbuf[ERRBUFLEN]; const char * libzfs_error_init(int error) { char *msg = errbuf; - size_t len, msglen = ERRBUFLEN; + size_t msglen = sizeof (errbuf); if (modfind("zfs") < 0) { - len = snprintf(msg, msglen, dgettext(TEXT_DOMAIN, + size_t len = snprintf(msg, msglen, dgettext(TEXT_DOMAIN, "Failed to load %s module: "), ZFS_KMOD); msg += len; msglen -= len; @@ -251,24 +244,28 @@ libzfs_load_module(void) int zpool_relabel_disk(libzfs_handle_t *hdl, const char *path, const char *msg) { + (void) hdl, (void) path, (void) msg; return (0); } int zpool_label_disk(libzfs_handle_t *hdl, zpool_handle_t *zhp, const char *name) { + (void) hdl, (void) zhp, (void) name; return (0); } int find_shares_object(differ_info_t *di) { + (void) di; return (0); } int zfs_destroy_snaps_nvl_os(libzfs_handle_t *hdl, nvlist_t *snaps) { + (void) hdl, (void) snaps; return (0); } @@ -280,7 +277,6 @@ zfs_jail(zfs_handle_t *zhp, int jailid, int attach) { libzfs_handle_t *hdl = zhp->zfs_hdl; zfs_cmd_t zc = {"\0"}; - char errbuf[1024]; unsigned long cmd; int ret; @@ -309,6 +305,10 @@ zfs_jail(zfs_handle_t *zhp, int jailid, int attach) zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "vdevs can not be jailed")); return (zfs_error(hdl, EZFS_BADTYPE, errbuf)); + case ZFS_TYPE_INVALID: + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "invalid zfs_type_t: ZFS_TYPE_INVALID")); + return (zfs_error(hdl, EZFS_BADTYPE, errbuf)); case ZFS_TYPE_POOL: case ZFS_TYPE_FILESYSTEM: /* OK */ @@ -336,29 +336,35 @@ zpool_nextboot(libzfs_handle_t *hdl, uint64_t pool_guid, uint64_t dev_guid, { zfs_cmd_t zc = {"\0"}; nvlist_t *args; - int error; args = fnvlist_alloc(); fnvlist_add_uint64(args, ZPOOL_CONFIG_POOL_GUID, pool_guid); fnvlist_add_uint64(args, ZPOOL_CONFIG_GUID, dev_guid); fnvlist_add_string(args, "command", command); - error = zcmd_write_src_nvlist(hdl, &zc, args); - if (error == 0) - error = zfs_ioctl(hdl, ZFS_IOC_NEXTBOOT, &zc); + zcmd_write_src_nvlist(hdl, &zc, args); + int error = zfs_ioctl(hdl, ZFS_IOC_NEXTBOOT, &zc); zcmd_free_nvlists(&zc); nvlist_free(args); return (error); } /* - * Fill given version buffer with zfs kernel version. - * Returns 0 on success, and -1 on error (with errno set) + * Return allocated loaded module version, or NULL on error (with errno set) */ -int -zfs_version_kernel(char *version, int len) +char * +zfs_version_kernel(void) { - size_t l = len; - - return (sysctlbyname("vfs.zfs.version.module", - version, &l, NULL, 0)); + size_t l; + if (sysctlbyname("vfs.zfs.version.module", + NULL, &l, NULL, 0) == -1) + return (NULL); + char *version = malloc(l); + if (version == NULL) + return (NULL); + if (sysctlbyname("vfs.zfs.version.module", + version, &l, NULL, 0) == -1) { + free(version); + return (NULL); + } + return (version); } diff --git a/lib/libzfs/os/freebsd/libzfs_zmount.c b/lib/libzfs/os/freebsd/libzfs_zmount.c index 79a8fe111f29..6a26cae1e090 100644 --- a/lib/libzfs/os/freebsd/libzfs_zmount.c +++ b/lib/libzfs/os/freebsd/libzfs_zmount.c @@ -74,8 +74,8 @@ build_iovec(struct iovec **iov, int *iovlen, const char *name, void *val, } static int -do_mount_(const char *spec, const char *dir, int mflag, char *fstype, - char *dataptr, int datalen, char *optptr, int optlen) +do_mount_(const char *spec, const char *dir, int mflag, + char *dataptr, int datalen, const char *optptr, int optlen) { struct iovec *iov; char *optstr, *p, *tofree; @@ -83,12 +83,10 @@ do_mount_(const char *spec, const char *dir, int mflag, char *fstype, assert(spec != NULL); assert(dir != NULL); - assert(fstype != NULL); - assert(strcmp(fstype, MNTTYPE_ZFS) == 0); - assert(dataptr == NULL); - assert(datalen == 0); + assert(dataptr == NULL), (void) dataptr; + assert(datalen == 0), (void) datalen; assert(optptr != NULL); - assert(optlen > 0); + assert(optlen > 0), (void) optlen; tofree = optstr = strdup(optptr); assert(optstr != NULL); @@ -99,7 +97,8 @@ do_mount_(const char *spec, const char *dir, int mflag, char *fstype, build_iovec(&iov, &iovlen, "update", NULL, 0); if (mflag & MS_RDONLY) build_iovec(&iov, &iovlen, "ro", NULL, 0); - build_iovec(&iov, &iovlen, "fstype", fstype, (size_t)-1); + build_iovec(&iov, &iovlen, "fstype", __DECONST(char *, MNTTYPE_ZFS), + (size_t)-1); build_iovec(&iov, &iovlen, "fspath", __DECONST(char *, dir), (size_t)-1); build_iovec(&iov, &iovlen, "from", __DECONST(char *, spec), (size_t)-1); @@ -113,10 +112,10 @@ do_mount_(const char *spec, const char *dir, int mflag, char *fstype, } int -do_mount(zfs_handle_t *zhp, const char *mntpt, char *opts, int flags) +do_mount(zfs_handle_t *zhp, const char *mntpt, const char *opts, int flags) { - return (do_mount_(zfs_get_name(zhp), mntpt, flags, MNTTYPE_ZFS, NULL, 0, + return (do_mount_(zfs_get_name(zhp), mntpt, flags, NULL, 0, opts, sizeof (mntpt))); } diff --git a/lib/libzfs/os/linux/libzfs_mount_os.c b/lib/libzfs/os/linux/libzfs_mount_os.c index 1b01caeb7079..f0bf3dcc6c6b 100644 --- a/lib/libzfs/os/linux/libzfs_mount_os.c +++ b/lib/libzfs/os/linux/libzfs_mount_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -37,7 +37,7 @@ #include #include #include -#include +#include #include #include #include @@ -84,6 +84,13 @@ static const option_map_t option_map[] = { { MNTOPT_ACL, MS_POSIXACL, ZS_COMMENT }, { MNTOPT_NOACL, MS_COMMENT, ZS_COMMENT }, { MNTOPT_POSIXACL, MS_POSIXACL, ZS_COMMENT }, + /* + * Case sensitive options are just listed here to silently + * ignore the error if passed with zfs mount command. + */ + { MNTOPT_CASESENSITIVE, MS_COMMENT, ZS_COMMENT }, + { MNTOPT_CASEINSENSITIVE, MS_COMMENT, ZS_COMMENT }, + { MNTOPT_CASEMIXED, MS_COMMENT, ZS_COMMENT }, #ifdef MS_NOATIME { MNTOPT_NOATIME, MS_NOATIME, ZS_COMMENT }, { MNTOPT_ATIME, MS_COMMENT, ZS_COMMENT }, @@ -180,7 +187,7 @@ parse_option(char *mntopt, unsigned long *mntflags, * otherwise they are considered fatal are copied in to badopt. */ int -zfs_parse_mount_options(char *mntopts, unsigned long *mntflags, +zfs_parse_mount_options(const char *mntopts, unsigned long *mntflags, unsigned long *zfsflags, int sloppy, char *badopt, char *mtabopt) { int error = 0, quote = 0, flag = 0, count = 0; @@ -320,7 +327,7 @@ zfs_adjust_mount_options(zfs_handle_t *zhp, const char *mntpoint, * make due with return value from the mount process. */ int -do_mount(zfs_handle_t *zhp, const char *mntpt, char *opts, int flags) +do_mount(zfs_handle_t *zhp, const char *mntpt, const char *opts, int flags) { const char *src = zfs_get_name(zhp); int error = 0; @@ -341,10 +348,10 @@ do_mount(zfs_handle_t *zhp, const char *mntpt, char *opts, int flags) } } else { char *argv[9] = { - "/bin/mount", - "--no-canonicalize", - "-t", MNTTYPE_ZFS, - "-o", opts, + (char *)"/bin/mount", + (char *)"--no-canonicalize", + (char *)"-t", (char *)MNTTYPE_ZFS, + (char *)"-o", (char *)opts, (char *)src, (char *)mntpt, (char *)NULL }; @@ -384,23 +391,17 @@ do_unmount(zfs_handle_t *zhp, const char *mntpt, int flags) return (rv < 0 ? errno : 0); } - char force_opt[] = "-f"; - char lazy_opt[] = "-l"; char *argv[7] = { - "/bin/umount", - "-t", MNTTYPE_ZFS, + (char *)"/bin/umount", + (char *)"-t", (char *)MNTTYPE_ZFS, NULL, NULL, NULL, NULL }; int rc, count = 3; - if (flags & MS_FORCE) { - argv[count] = force_opt; - count++; - } + if (flags & MS_FORCE) + argv[count++] = (char *)"-f"; - if (flags & MS_DETACH) { - argv[count] = lazy_opt; - count++; - } + if (flags & MS_DETACH) + argv[count++] = (char *)"-l"; argv[count] = (char *)mntpt; rc = libzfs_run_process(argv[0], argv, STDOUT_VERBOSE|STDERR_VERBOSE); diff --git a/lib/libzfs/os/linux/libzfs_pool_os.c b/lib/libzfs/os/linux/libzfs_pool_os.c index 90eb8db50790..cab12d39e6e4 100644 --- a/lib/libzfs/os/linux/libzfs_pool_os.c +++ b/lib/libzfs/os/linux/libzfs_pool_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -34,14 +34,13 @@ #include #include #include -#include +#include #include #include #include #include #include #include -#include #include #include #include @@ -217,17 +216,15 @@ zpool_label_disk(libzfs_handle_t *hdl, zpool_handle_t *zhp, const char *name) size_t resv = EFI_MIN_RESV_SIZE; uint64_t slice_size; diskaddr_t start_block; - char errbuf[1024]; + char errbuf[ERRBUFLEN]; /* prepare an error message just in case */ (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot label '%s'"), name); if (zhp) { - nvlist_t *nvroot; - - verify(nvlist_lookup_nvlist(zhp->zpool_config, - ZPOOL_CONFIG_VDEV_TREE, &nvroot) == 0); + nvlist_t *nvroot = fnvlist_lookup_nvlist(zhp->zpool_config, + ZPOOL_CONFIG_VDEV_TREE); if (zhp->zpool_start_block == 0) start_block = find_start_block(nvroot); @@ -280,8 +277,9 @@ zpool_label_disk(libzfs_handle_t *hdl, zpool_handle_t *zhp, const char *name) * Why we use V_USR: V_BACKUP confuses users, and is considered * disposable by some EFI utilities (since EFI doesn't have a backup * slice). V_UNASSIGNED is supposed to be used only for zero size - * partitions, and efi_write() will fail if we use it. V_ROOT, V_BOOT, - * etc. were all pretty specific. V_USR is as close to reality as we + * partitions, and efi_write() will fail if we use it. + * Other available types were all pretty specific. + * V_USR is as close to reality as we * can get, in the absence of V_OTHER. */ vtoc->efi_parts[0].p_tag = V_USR; diff --git a/lib/libzfs/os/linux/libzfs_sendrecv_os.c b/lib/libzfs/os/linux/libzfs_sendrecv_os.c deleted file mode 100644 index 593c38ec62df..000000000000 --- a/lib/libzfs/os/linux/libzfs_sendrecv_os.c +++ /dev/null @@ -1,52 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License (the "License"). - * You may not use this file except in compliance with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ - - -#include - -#include "../../libzfs_impl.h" - -#ifndef F_SETPIPE_SZ -#define F_SETPIPE_SZ (F_SETLEASE + 7) -#endif /* F_SETPIPE_SZ */ - -#ifndef F_GETPIPE_SZ -#define F_GETPIPE_SZ (F_GETLEASE + 7) -#endif /* F_GETPIPE_SZ */ - -void -libzfs_set_pipe_max(int infd) -{ - FILE *procf = fopen("/proc/sys/fs/pipe-max-size", "re"); - - if (procf != NULL) { - unsigned long max_psize; - long cur_psize; - if (fscanf(procf, "%lu", &max_psize) > 0) { - cur_psize = fcntl(infd, F_GETPIPE_SZ); - if (cur_psize > 0 && - max_psize > (unsigned long) cur_psize) - fcntl(infd, F_SETPIPE_SZ, - max_psize); - } - fclose(procf); - } -} diff --git a/lib/libzfs/os/linux/libzfs_util_os.c b/lib/libzfs/os/linux/libzfs_util_os.c index a32aa1513b39..4a44aa1f1d0d 100644 --- a/lib/libzfs/os/linux/libzfs_util_os.c +++ b/lib/libzfs/os/linux/libzfs_util_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -19,21 +19,28 @@ * CDDL HEADER END */ +/* + * Copyright (c) 2021 Klara, Inc. + */ +#include #include #include #include +#include +#include #include #include #include #include -#include -#include -#include -#include +#include #include +#include +#include +#include #include #include +#include #include #include @@ -57,7 +64,7 @@ libzfs_error_init(int error) switch (error) { case ENXIO: return (dgettext(TEXT_DOMAIN, "The ZFS modules are not " - "loaded.\nTry running '/sbin/modprobe zfs' as root " + "loaded.\nTry running 'modprobe zfs' as root " "to load them.")); case ENOENT: return (dgettext(TEXT_DOMAIN, "/dev/zfs and /proc/self/mounts " @@ -65,7 +72,7 @@ libzfs_error_init(int error) "-t proc proc /proc' as root.")); case ENOEXEC: return (dgettext(TEXT_DOMAIN, "The ZFS modules cannot be " - "auto-loaded.\nTry running '/sbin/modprobe zfs' as " + "auto-loaded.\nTry running 'modprobe zfs' as " "root to manually load them.")); case EACCES: return (dgettext(TEXT_DOMAIN, "Permission denied the " @@ -76,93 +83,80 @@ libzfs_error_init(int error) } } -static int -libzfs_module_loaded(const char *module) -{ - const char path_prefix[] = "/sys/module/"; - char path[256]; - - memcpy(path, path_prefix, sizeof (path_prefix) - 1); - strcpy(path + sizeof (path_prefix) - 1, module); - - return (access(path, F_OK) == 0); -} - /* - * Verify the required ZFS_DEV device is available and optionally attempt - * to load the ZFS modules. Under normal circumstances the modules - * should already have been loaded by some external mechanism. + * zfs(4) is loaded by udev if there's a fstype=zfs device present, + * but if there isn't, load them automatically; + * always wait for ZFS_DEV to appear via udev. * * Environment variables: - * - ZFS_MODULE_LOADING="YES|yes|ON|on" - Attempt to load modules. - * - ZFS_MODULE_TIMEOUT="" - Seconds to wait for ZFS_DEV + * - ZFS_MODULE_TIMEOUT="" - Seconds to wait for ZFS_DEV, + * defaults to 10, max. 10 min. */ -static int -libzfs_load_module_impl(const char *module) +int +libzfs_load_module(void) { - char *argv[4] = {"/sbin/modprobe", "-q", (char *)module, (char *)0}; - char *load_str, *timeout_str; - long timeout = 10; /* seconds */ - long busy_timeout = 10; /* milliseconds */ - int load = 0, fd; - hrtime_t start; - - /* Optionally request module loading */ - if (!libzfs_module_loaded(module)) { - load_str = getenv("ZFS_MODULE_LOADING"); - if (load_str) { - if (!strncasecmp(load_str, "YES", strlen("YES")) || - !strncasecmp(load_str, "ON", strlen("ON"))) - load = 1; - else - load = 0; - } + if (access(ZFS_DEV, F_OK) == 0) + return (0); - if (load) { - if (libzfs_run_process("/sbin/modprobe", argv, 0)) - return (ENOEXEC); - } + if (access(ZFS_SYSFS_DIR, F_OK) != 0) { + char *argv[] = {(char *)"modprobe", (char *)"zfs", NULL}; + if (libzfs_run_process("modprobe", argv, 0)) + return (ENOEXEC); - if (!libzfs_module_loaded(module)) + if (access(ZFS_SYSFS_DIR, F_OK) != 0) return (ENXIO); } - /* - * Device creation by udev is asynchronous and waiting may be - * required. Busy wait for 10ms and then fall back to polling every - * 10ms for the allowed timeout (default 10s, max 10m). This is - * done to optimize for the common case where the device is - * immediately available and to avoid penalizing the possible - * case where udev is slow or unable to create the device. - */ - timeout_str = getenv("ZFS_MODULE_TIMEOUT"); - if (timeout_str) { - timeout = strtol(timeout_str, NULL, 0); - timeout = MAX(MIN(timeout, (10 * 60)), 0); /* 0 <= N <= 600 */ + const char *timeout_str = getenv("ZFS_MODULE_TIMEOUT"); + int seconds = 10; + if (timeout_str) + seconds = MIN(strtol(timeout_str, NULL, 0), 600); + struct itimerspec timeout = {.it_value.tv_sec = MAX(seconds, 0)}; + + int ino = inotify_init1(IN_CLOEXEC); + if (ino == -1) + return (ENOENT); + inotify_add_watch(ino, ZFS_DEVDIR, IN_CREATE); + + if (access(ZFS_DEV, F_OK) == 0) { + close(ino); + return (0); + } else if (seconds == 0) { + close(ino); + return (ENOENT); } - start = gethrtime(); - do { - fd = open(ZFS_DEV, O_RDWR | O_CLOEXEC); - if (fd >= 0) { - (void) close(fd); - return (0); - } else if (errno != ENOENT) { - return (errno); - } else if (NSEC2MSEC(gethrtime() - start) < busy_timeout) { - sched_yield(); - } else { - usleep(10 * MILLISEC); - } - } while (NSEC2MSEC(gethrtime() - start) < (timeout * MILLISEC)); + size_t evsz = sizeof (struct inotify_event) + NAME_MAX + 1; + struct inotify_event *ev = alloca(evsz); - return (ENOENT); -} + int tout = timerfd_create(CLOCK_MONOTONIC, TFD_CLOEXEC); + if (tout == -1) { + close(ino); + return (ENOENT); + } + timerfd_settime(tout, 0, &timeout, NULL); -int -libzfs_load_module(void) -{ - return (libzfs_load_module_impl(ZFS_DRIVER)); + int ret = ENOENT; + struct pollfd pfds[] = { + {.fd = ino, .events = POLLIN}, + {.fd = tout, .events = POLLIN}, + }; + while (poll(pfds, ARRAY_SIZE(pfds), -1) != -1) { + if (pfds[0].revents & POLLIN) { + verify(read(ino, ev, evsz) > + sizeof (struct inotify_event)); + if (strcmp(ev->name, &ZFS_DEV[sizeof (ZFS_DEVDIR)]) + == 0) { + ret = 0; + break; + } + } + if (pfds[1].revents & POLLIN) + break; + } + close(tout); + close(ino); + return (ret); } int @@ -192,31 +186,95 @@ zfs_destroy_snaps_nvl_os(libzfs_handle_t *hdl, nvlist_t *snaps) } /* - * Fill given version buffer with zfs kernel version read from ZFS_SYSFS_DIR - * Returns 0 on success, and -1 on error (with errno set) + * Return allocated loaded module version, or NULL on error (with errno set) + */ +char * +zfs_version_kernel(void) +{ + FILE *f = fopen(ZFS_SYSFS_DIR "/version", "re"); + if (f == NULL) + return (NULL); + + char *ret = NULL; + size_t l; + ssize_t read; + if ((read = getline(&ret, &l, f)) == -1) { + int err = errno; + fclose(f); + errno = err; + return (NULL); + } + + fclose(f); + if (ret[read - 1] == '\n') + ret[read - 1] = '\0'; + return (ret); +} + +/* + * Add or delete the given filesystem to/from the given user namespace. */ int -zfs_version_kernel(char *version, int len) +zfs_userns(zfs_handle_t *zhp, const char *nspath, int attach) { - int _errno; - int fd; - int rlen; - - if ((fd = open(ZFS_SYSFS_DIR "/version", O_RDONLY | O_CLOEXEC)) == -1) - return (-1); - - if ((rlen = read(fd, version, len)) == -1) { - version[0] = '\0'; - _errno = errno; - (void) close(fd); - errno = _errno; - return (-1); + libzfs_handle_t *hdl = zhp->zfs_hdl; + zfs_cmd_t zc = {"\0"}; + char errbuf[1024]; + unsigned long cmd; + int ret; + + if (attach) { + (void) snprintf(errbuf, sizeof (errbuf), + dgettext(TEXT_DOMAIN, "cannot add '%s' to namespace"), + zhp->zfs_name); + } else { + (void) snprintf(errbuf, sizeof (errbuf), + dgettext(TEXT_DOMAIN, "cannot remove '%s' from namespace"), + zhp->zfs_name); } - version[rlen-1] = '\0'; /* discard '\n' */ + switch (zhp->zfs_type) { + case ZFS_TYPE_VOLUME: + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "volumes can not be namespaced")); + return (zfs_error(hdl, EZFS_BADTYPE, errbuf)); + case ZFS_TYPE_SNAPSHOT: + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "snapshots can not be namespaced")); + return (zfs_error(hdl, EZFS_BADTYPE, errbuf)); + case ZFS_TYPE_BOOKMARK: + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "bookmarks can not be namespaced")); + return (zfs_error(hdl, EZFS_BADTYPE, errbuf)); + case ZFS_TYPE_VDEV: + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "vdevs can not be namespaced")); + return (zfs_error(hdl, EZFS_BADTYPE, errbuf)); + case ZFS_TYPE_INVALID: + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "invalid zfs_type_t: ZFS_TYPE_INVALID")); + return (zfs_error(hdl, EZFS_BADTYPE, errbuf)); + case ZFS_TYPE_POOL: + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "pools can not be namespaced")); + return (zfs_error(hdl, EZFS_BADTYPE, errbuf)); + case ZFS_TYPE_FILESYSTEM: + break; + } + assert(zhp->zfs_type == ZFS_TYPE_FILESYSTEM); - if (close(fd) == -1) - return (-1); + (void) strlcpy(zc.zc_name, zhp->zfs_name, sizeof (zc.zc_name)); + zc.zc_objset_type = DMU_OST_ZFS; + zc.zc_cleanup_fd = open(nspath, O_RDONLY); + if (zc.zc_cleanup_fd < 0) { + return (zfs_error(hdl, EZFS_NOT_USER_NAMESPACE, errbuf)); + } - return (0); + cmd = attach ? ZFS_IOC_USERNS_ATTACH : ZFS_IOC_USERNS_DETACH; + if ((ret = zfs_ioctl(hdl, cmd, &zc)) != 0) + zfs_standard_error(hdl, errno, errbuf); + + (void) close(zc.zc_cleanup_fd); + + return (ret); } diff --git a/lib/libzfs_core/Makefile.am b/lib/libzfs_core/Makefile.am index 64cb76f1995b..d1c6fb86d186 100644 --- a/lib/libzfs_core/Makefile.am +++ b/lib/libzfs_core/Makefile.am @@ -1,37 +1,31 @@ -include $(top_srcdir)/config/Rules.am +libzfs_core_la_CFLAGS = $(AM_CFLAGS) $(LIBRARY_CFLAGS) +libzfs_core_la_CFLAGS += -fvisibility=hidden -pkgconfig_DATA = libzfs_core.pc +lib_LTLIBRARIES += libzfs_core.la +CPPCHECKTARGETS += libzfs_core.la -AM_CFLAGS += -fvisibility=hidden - -lib_LTLIBRARIES = libzfs_core.la - -include $(top_srcdir)/config/Abigail.am - -USER_C = \ - libzfs_core.c +libzfs_core_la_SOURCES = \ + %D%/libzfs_core.c if BUILD_LINUX -USER_C += \ - os/linux/libzfs_core_ioctl.c +libzfs_core_la_SOURCES += \ + %D%/os/linux/libzfs_core_ioctl.c endif +libzfs_core_la_CPPFLAGS = $(AM_CPPFLAGS) if BUILD_FREEBSD -DEFAULT_INCLUDES += -I$(top_srcdir)/include/os/freebsd/zfs +libzfs_core_la_CPPFLAGS += -Iinclude/os/freebsd/zfs -USER_C += \ - os/freebsd/libzfs_core_ioctl.c +libzfs_core_la_SOURCES += \ + %D%/os/freebsd/libzfs_core_ioctl.c -VPATH += $(top_srcdir)/module/os/freebsd/zfs - -nodist_libzfs_core_la_SOURCES = zfs_ioctl_compat.c +nodist_libzfs_core_la_SOURCES = \ + module/os/freebsd/zfs/zfs_ioctl_compat.c endif -libzfs_core_la_SOURCES = $(USER_C) - libzfs_core_la_LIBADD = \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la \ - $(abs_top_builddir)/lib/libspl/libspl.la + libnvpair.la \ + libspl.la libzfs_core_la_LIBADD += $(LTLIBINTL) @@ -47,7 +41,6 @@ endif libzfs_core_la_LDFLAGS += -version-info 3:0:0 -include $(top_srcdir)/config/CppCheck.am +pkgconfig_DATA += %D%/libzfs_core.pc -# Library ABI -EXTRA_DIST = libzfs_core.abi libzfs_core.suppr +dist_noinst_DATA += %D%/libzfs_core.abi %D%/libzfs_core.suppr diff --git a/lib/libzfs_core/libzfs_core.abi b/lib/libzfs_core/libzfs_core.abi index 111287d3ef56..7e340e1d4cc5 100644 --- a/lib/libzfs_core/libzfs_core.abi +++ b/lib/libzfs_core/libzfs_core.abi @@ -6,8 +6,6 @@ - - @@ -183,6 +181,7 @@ + @@ -195,6 +194,7 @@ + @@ -215,14 +215,12 @@ - - + - + - @@ -246,7 +244,6 @@ - @@ -289,6 +286,11 @@ + + + + + @@ -309,7 +311,7 @@ - + @@ -334,11 +336,6 @@ - - - - - @@ -411,6 +408,11 @@ + + + + + @@ -431,7 +433,7 @@ - + @@ -456,11 +458,6 @@ - - - - - @@ -501,6 +498,12 @@ + + + + + + @@ -525,12 +528,6 @@ - - - - - - @@ -718,6 +715,9 @@ + + + @@ -752,18 +752,66 @@ - - + - + - + - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -832,16 +880,16 @@ - + - + - + - + @@ -853,65 +901,6 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - @@ -922,11 +911,16 @@ + + + + + @@ -946,7 +940,7 @@ - + @@ -1012,6 +1006,7 @@ + @@ -1574,10 +1569,12 @@ + + @@ -1657,6 +1654,12 @@ + + + + + + @@ -1739,6 +1742,26 @@ + + + + + + + + + + + + + + + + + + + + @@ -1919,7 +1942,11 @@ - + + + + + diff --git a/lib/libzfs_core/libzfs_core.c b/lib/libzfs_core/libzfs_core.c index 99652bc4a03a..16bd9af1bbc8 100644 --- a/lib/libzfs_core/libzfs_core.c +++ b/lib/libzfs_core/libzfs_core.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -22,10 +22,10 @@ /* * Copyright (c) 2012, 2020 by Delphix. All rights reserved. * Copyright (c) 2013 Steven Hartland. All rights reserved. - * Copyright (c) 2017 Datto Inc. * Copyright 2017 RackTop Systems. * Copyright (c) 2017 Open-E, Inc. All Rights Reserved. * Copyright (c) 2019, 2020 by Christian Schwarz. All rights reserved. + * Copyright (c) 2019 Datto Inc. */ /* @@ -91,6 +91,9 @@ #include #include #include +#if __FreeBSD__ +#define BIG_PIPE_SIZE (64 * 1024) /* From sys/pipe.h */ +#endif static int g_fd = -1; static pthread_mutex_t g_lock = PTHREAD_MUTEX_INITIALIZER; @@ -586,6 +589,129 @@ lzc_get_holds(const char *snapname, nvlist_t **holdsp) return (lzc_ioctl(ZFS_IOC_GET_HOLDS, snapname, NULL, holdsp)); } +static unsigned int +max_pipe_buffer(int infd) +{ +#if __linux__ + static unsigned int max; + if (max == 0) { + max = 1048576; /* fs/pipe.c default */ + + FILE *procf = fopen("/proc/sys/fs/pipe-max-size", "re"); + if (procf != NULL) { + if (fscanf(procf, "%u", &max) <= 0) { + /* ignore error: max untouched if parse fails */ + } + fclose(procf); + } + } + + unsigned int cur = fcntl(infd, F_GETPIPE_SZ); + /* + * Sadly, Linux has an unfixed deadlock if you do SETPIPE_SZ on a pipe + * with data in it. + * cf. #13232, https://bugzilla.kernel.org/show_bug.cgi?id=212295 + * + * And since the problem is in waking up the writer, there's nothing + * we can do about it from here. + * + * So if people want to, they can set this, but they + * may regret it... + */ + if (getenv("ZFS_SET_PIPE_MAX") == NULL) + return (cur); + if (cur < max && fcntl(infd, F_SETPIPE_SZ, max) != -1) + cur = max; + return (cur); +#else + /* FreeBSD automatically resizes */ + (void) infd; + return (BIG_PIPE_SIZE); +#endif +} + +#if __linux__ +struct send_worker_ctx { + int from; /* read end of pipe, with send data; closed on exit */ + int to; /* original arbitrary output fd; mustn't be a pipe */ +}; + +static void * +send_worker(void *arg) +{ + struct send_worker_ctx *ctx = arg; + unsigned int bufsiz = max_pipe_buffer(ctx->from); + ssize_t rd; + + while ((rd = splice(ctx->from, NULL, ctx->to, NULL, bufsiz, + SPLICE_F_MOVE | SPLICE_F_MORE)) > 0) + ; + + int err = (rd == -1) ? errno : 0; + close(ctx->from); + return ((void *)(uintptr_t)err); +} +#endif + +/* + * Since Linux 5.10, 4d03e3cc59828c82ee89ea6e27a2f3cdf95aaadf + * ("fs: don't allow kernel reads and writes without iter ops"), + * ZFS_IOC_SEND* will EINVAL when writing to /dev/null, /dev/zero, &c. + * + * This wrapper transparently executes func() with a pipe + * by spawning a thread to copy from that pipe to the original output + * in the background. + * + * Returns the error from func(), if nonzero, + * otherwise the error from the thread. + * + * No-op if orig_fd is -1, already a pipe (but the buffer size is bumped), + * and on not-Linux; as such, it is safe to wrap/call wrapped functions + * in a wrapped context. + */ +int +lzc_send_wrapper(int (*func)(int, void *), int orig_fd, void *data) +{ +#if __linux__ + struct stat sb; + if (orig_fd != -1 && fstat(orig_fd, &sb) == -1) + return (errno); + if (orig_fd == -1 || S_ISFIFO(sb.st_mode)) { + if (orig_fd != -1) + (void) max_pipe_buffer(orig_fd); + return (func(orig_fd, data)); + } + if ((fcntl(orig_fd, F_GETFL) & O_ACCMODE) == O_RDONLY) + return (errno = EBADF); + + int rw[2]; + if (pipe2(rw, O_CLOEXEC) == -1) + return (errno); + + int err; + pthread_t send_thread; + struct send_worker_ctx ctx = {.from = rw[0], .to = orig_fd}; + if ((err = pthread_create(&send_thread, NULL, send_worker, &ctx)) + != 0) { + close(rw[0]); + close(rw[1]); + return (errno = err); + } + + err = func(rw[1], data); + + void *send_err; + close(rw[1]); + pthread_join(send_thread, &send_err); + if (err == 0 && send_err != 0) + errno = err = (uintptr_t)send_err; + + return (err); +#else + return (func(orig_fd, data)); +#endif +} + /* * Generate a zfs send stream for the specified snapshot and write it to * the specified file descriptor. @@ -656,9 +782,11 @@ lzc_send_resume(const char *snapname, const char *from, int fd, * redactnv: nvlist of string -> boolean(ignored) containing the names of all * the snapshots that we should redact with respect to. * redactbook: Name of the redaction bookmark to create. + * + * Pre-wrapped. */ -int -lzc_send_resume_redacted(const char *snapname, const char *from, int fd, +static int +lzc_send_resume_redacted_cb_impl(const char *snapname, const char *from, int fd, enum lzc_send_flags flags, uint64_t resumeobj, uint64_t resumeoff, const char *redactbook) { @@ -691,6 +819,40 @@ lzc_send_resume_redacted(const char *snapname, const char *from, int fd, return (err); } +struct lzc_send_resume_redacted { + const char *snapname; + const char *from; + enum lzc_send_flags flags; + uint64_t resumeobj; + uint64_t resumeoff; + const char *redactbook; +}; + +static int +lzc_send_resume_redacted_cb(int fd, void *arg) +{ + struct lzc_send_resume_redacted *zsrr = arg; + return (lzc_send_resume_redacted_cb_impl(zsrr->snapname, zsrr->from, + fd, zsrr->flags, zsrr->resumeobj, zsrr->resumeoff, + zsrr->redactbook)); +} + +int +lzc_send_resume_redacted(const char *snapname, const char *from, int fd, + enum lzc_send_flags flags, uint64_t resumeobj, uint64_t resumeoff, + const char *redactbook) +{ + struct lzc_send_resume_redacted zsrr = { + .snapname = snapname, + .from = from, + .flags = flags, + .resumeobj = resumeobj, + .resumeoff = resumeoff, + .redactbook = redactbook, + }; + return (lzc_send_wrapper(lzc_send_resume_redacted_cb, fd, &zsrr)); +} + /* * "from" can be NULL, a snapshot, or a bookmark. * @@ -706,9 +868,11 @@ lzc_send_resume_redacted(const char *snapname, const char *from, int fd, * significantly more I/O and be less efficient than a send space estimation on * an equivalent snapshot. This process is also used if redact_snaps is * non-null. + * + * Pre-wrapped. */ -int -lzc_send_space_resume_redacted(const char *snapname, const char *from, +static int +lzc_send_space_resume_redacted_cb_impl(const char *snapname, const char *from, enum lzc_send_flags flags, uint64_t resumeobj, uint64_t resumeoff, uint64_t resume_bytes, const char *redactbook, int fd, uint64_t *spacep) { @@ -745,6 +909,45 @@ lzc_send_space_resume_redacted(const char *snapname, const char *from, return (err); } +struct lzc_send_space_resume_redacted { + const char *snapname; + const char *from; + enum lzc_send_flags flags; + uint64_t resumeobj; + uint64_t resumeoff; + uint64_t resume_bytes; + const char *redactbook; + uint64_t *spacep; +}; + +static int +lzc_send_space_resume_redacted_cb(int fd, void *arg) +{ + struct lzc_send_space_resume_redacted *zssrr = arg; + return (lzc_send_space_resume_redacted_cb_impl(zssrr->snapname, + zssrr->from, zssrr->flags, zssrr->resumeobj, zssrr->resumeoff, + zssrr->resume_bytes, zssrr->redactbook, fd, zssrr->spacep)); +} + +int +lzc_send_space_resume_redacted(const char *snapname, const char *from, + enum lzc_send_flags flags, uint64_t resumeobj, uint64_t resumeoff, + uint64_t resume_bytes, const char *redactbook, int fd, uint64_t *spacep) +{ + struct lzc_send_space_resume_redacted zssrr = { + .snapname = snapname, + .from = from, + .flags = flags, + .resumeobj = resumeobj, + .resumeoff = resumeoff, + .resume_bytes = resume_bytes, + .redactbook = redactbook, + .spacep = spacep, + }; + return (lzc_send_wrapper(lzc_send_space_resume_redacted_cb, + fd, &zssrr)); +} + int lzc_send_space(const char *snapname, const char *from, enum lzc_send_flags flags, uint64_t *spacep) @@ -783,7 +986,7 @@ recv_read(int fd, void *buf, int ilen) static int recv_impl(const char *snapname, nvlist_t *recvdprops, nvlist_t *localprops, uint8_t *wkeydata, uint_t wkeylen, const char *origin, boolean_t force, - boolean_t resumable, boolean_t raw, int input_fd, + boolean_t heal, boolean_t resumable, boolean_t raw, int input_fd, const dmu_replay_record_t *begin_record, uint64_t *read_bytes, uint64_t *errflags, nvlist_t **errors) { @@ -811,6 +1014,16 @@ recv_impl(const char *snapname, nvlist_t *recvdprops, nvlist_t *localprops, *slashp = '\0'; } + /* + * It is not uncommon for gigabytes to be processed by zfs receive. + * Speculatively increase the buffer size if supported by the platform. + */ + struct stat sb; + if (fstat(input_fd, &sb) == -1) + return (errno); + if (S_ISFIFO(sb.st_mode)) + (void) max_pipe_buffer(input_fd); + /* * The begin_record is normally a non-byteswapped BEGIN record. * For resumable streams it may be set to any non-byteswapped @@ -828,7 +1041,7 @@ recv_impl(const char *snapname, nvlist_t *recvdprops, nvlist_t *localprops, /* * All receives with a payload should use the new interface. */ - if (resumable || raw || wkeydata != NULL || payload) { + if (resumable || heal || raw || wkeydata != NULL || payload) { nvlist_t *outnvl = NULL; nvlist_t *innvl = fnvlist_alloc(); @@ -868,6 +1081,8 @@ recv_impl(const char *snapname, nvlist_t *recvdprops, nvlist_t *localprops, if (resumable) fnvlist_add_boolean(innvl, "resumable"); + if (heal) + fnvlist_add_boolean(innvl, "heal"); error = lzc_ioctl(ZFS_IOC_RECV_NEW, fsname, innvl, &outnvl); @@ -967,7 +1182,7 @@ lzc_receive(const char *snapname, nvlist_t *props, const char *origin, boolean_t force, boolean_t raw, int fd) { return (recv_impl(snapname, props, NULL, NULL, 0, origin, force, - B_FALSE, raw, fd, NULL, NULL, NULL, NULL)); + B_FALSE, B_FALSE, raw, fd, NULL, NULL, NULL, NULL)); } /* @@ -981,7 +1196,7 @@ lzc_receive_resumable(const char *snapname, nvlist_t *props, const char *origin, boolean_t force, boolean_t raw, int fd) { return (recv_impl(snapname, props, NULL, NULL, 0, origin, force, - B_TRUE, raw, fd, NULL, NULL, NULL, NULL)); + B_FALSE, B_TRUE, raw, fd, NULL, NULL, NULL, NULL)); } /* @@ -1004,7 +1219,7 @@ lzc_receive_with_header(const char *snapname, nvlist_t *props, return (EINVAL); return (recv_impl(snapname, props, NULL, NULL, 0, origin, force, - resumable, raw, fd, begin_record, NULL, NULL, NULL)); + B_FALSE, resumable, raw, fd, begin_record, NULL, NULL, NULL)); } /* @@ -1034,7 +1249,7 @@ lzc_receive_one(const char *snapname, nvlist_t *props, { (void) action_handle, (void) cleanup_fd; return (recv_impl(snapname, props, NULL, NULL, 0, origin, force, - resumable, raw, input_fd, begin_record, + B_FALSE, resumable, raw, input_fd, begin_record, read_bytes, errflags, errors)); } @@ -1056,7 +1271,27 @@ lzc_receive_with_cmdprops(const char *snapname, nvlist_t *props, { (void) action_handle, (void) cleanup_fd; return (recv_impl(snapname, props, cmdprops, wkeydata, wkeylen, origin, - force, resumable, raw, input_fd, begin_record, + force, B_FALSE, resumable, raw, input_fd, begin_record, + read_bytes, errflags, errors)); +} + +/* + * Like lzc_receive_with_cmdprops, but allows the caller to pass an additional + * 'heal' argument. + * + * The heal arguments tells us to heal the provided snapshot using the provided + * send stream + */ +int lzc_receive_with_heal(const char *snapname, nvlist_t *props, + nvlist_t *cmdprops, uint8_t *wkeydata, uint_t wkeylen, const char *origin, + boolean_t force, boolean_t heal, boolean_t resumable, boolean_t raw, + int input_fd, const dmu_replay_record_t *begin_record, int cleanup_fd, + uint64_t *read_bytes, uint64_t *errflags, uint64_t *action_handle, + nvlist_t **errors) +{ + (void) action_handle, (void) cleanup_fd; + return (recv_impl(snapname, props, cmdprops, wkeydata, wkeylen, origin, + force, heal, resumable, raw, input_fd, begin_record, read_bytes, errflags, errors)); } diff --git a/lib/libzfs_core/os/freebsd/libzfs_core_ioctl.c b/lib/libzfs_core/os/freebsd/libzfs_core_ioctl.c index b8394886d034..36fbbd88c2ac 100644 --- a/lib/libzfs_core/os/freebsd/libzfs_core_ioctl.c +++ b/lib/libzfs_core/os/freebsd/libzfs_core_ioctl.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libzfs_core/os/linux/libzfs_core_ioctl.c b/lib/libzfs_core/os/linux/libzfs_core_ioctl.c index 9b44a4e3be0d..116cddfc45d3 100644 --- a/lib/libzfs_core/os/linux/libzfs_core_ioctl.c +++ b/lib/libzfs_core/os/linux/libzfs_core_ioctl.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libzfsbootenv/Makefile.am b/lib/libzfsbootenv/Makefile.am index 0c454a5e031b..118f154821fc 100644 --- a/lib/libzfsbootenv/Makefile.am +++ b/lib/libzfsbootenv/Makefile.am @@ -1,31 +1,20 @@ -include $(top_srcdir)/config/Rules.am +libzfsbootenv_la_CFLAGS = $(AM_CFLAGS) $(LIBRARY_CFLAGS) +libzfsbootenv_la_CFLAGS += -fvisibility=hidden -pkgconfig_DATA = libzfsbootenv.pc +lib_LTLIBRARIES += libzfsbootenv.la +CPPCHECKTARGETS += libzfsbootenv.la -AM_CFLAGS += -fvisibility=hidden - -lib_LTLIBRARIES = libzfsbootenv.la - -include $(top_srcdir)/config/Abigail.am - -if BUILD_FREEBSD -DEFAULT_INCLUDES += -I$(top_srcdir)/include/os/freebsd/zfs -endif -if BUILD_LINUX -DEFAULT_INCLUDES += -I$(top_srcdir)/include/os/linux/zfs -endif - -USER_C = \ - lzbe_device.c \ - lzbe_pair.c \ - lzbe_util.c +libzfsbootenv_la_CPPFLAGS = $(AM_CPPFLAGS) +libzfsbootenv_la_CPPFLAGS += -I$(srcdir)/include/os/@ac_system_l@/zfs dist_libzfsbootenv_la_SOURCES = \ - $(USER_C) + %D%/lzbe_device.c \ + %D%/lzbe_pair.c \ + %D%/lzbe_util.c libzfsbootenv_la_LIBADD = \ - $(abs_top_builddir)/lib/libzfs/libzfs.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la + libzfs.la \ + libnvpair.la libzfsbootenv_la_LDFLAGS = @@ -35,7 +24,6 @@ endif libzfsbootenv_la_LDFLAGS += -version-info 1:0:0 -include $(top_srcdir)/config/CppCheck.am +pkgconfig_DATA += %D%/libzfsbootenv.pc -# Library ABI -EXTRA_DIST = libzfsbootenv.abi libzfsbootenv.suppr +dist_noinst_DATA += %D%/libzfsbootenv.abi %D%/libzfsbootenv.suppr diff --git a/lib/libzfsbootenv/libzfsbootenv.abi b/lib/libzfsbootenv/libzfsbootenv.abi index f1401a14f7a0..86ec25cf8470 100644 --- a/lib/libzfsbootenv/libzfsbootenv.abi +++ b/lib/libzfsbootenv/libzfsbootenv.abi @@ -5,8 +5,6 @@ - - @@ -20,6 +18,7 @@ + @@ -84,21 +83,16 @@ + + + + + + - - - - - - - - - - - @@ -167,16 +161,16 @@ - + - + - + - + @@ -188,19 +182,20 @@ - - - + + + + + - diff --git a/lib/libzpool/Makefile.am b/lib/libzpool/Makefile.am index e49577ec4a63..eaa920e56106 100644 --- a/lib/libzpool/Makefile.am +++ b/lib/libzpool/Makefile.am @@ -1,223 +1,198 @@ -include $(top_srcdir)/config/Rules.am +libzpool_la_CFLAGS = $(AM_CFLAGS) $(KERNEL_CFLAGS) $(LIBRARY_CFLAGS) +libzpool_la_CFLAGS += $(ZLIB_CFLAGS) -VPATH = \ - $(top_srcdir)/module/zfs \ - $(top_srcdir)/module/zcommon \ - $(top_srcdir)/module/lua \ - $(top_srcdir)/module/os/linux/zfs \ - $(top_srcdir)/lib/libzpool +libzpool_la_CPPFLAGS = $(AM_CPPFLAGS) $(FORCEDEBUG_CPPFLAGS) +libzpool_la_CPPFLAGS += -I$(srcdir)/include/os/@ac_system_l@/zfs +libzpool_la_CPPFLAGS += -DLIB_ZPOOL_BUILD -if BUILD_FREEBSD -DEFAULT_INCLUDES += -I$(top_srcdir)/include/os/freebsd/zfs -endif -if BUILD_LINUX -DEFAULT_INCLUDES += -I$(top_srcdir)/include/os/linux/zfs -endif - -# Unconditionally enable debugging for libzpool -AM_CPPFLAGS += -DDEBUG -UNDEBUG -DZFS_DEBUG - -# Suppress unused but set variable warnings often due to ASSERTs -AM_CFLAGS += $(NO_UNUSED_BUT_SET_VARIABLE) - -# Includes kernel code generate warnings for large stack frames -AM_CFLAGS += $(FRAME_LARGER_THAN) - -AM_CFLAGS += $(ZLIB_CFLAGS) - -AM_CFLAGS += -DLIB_ZPOOL_BUILD - -lib_LTLIBRARIES = libzpool.la - -USER_C = \ - kernel.c \ - taskq.c \ - util.c - -KERNEL_C = \ - zfeature_common.c \ - zfs_comutil.c \ - zfs_deleg.c \ - zfs_fletcher.c \ - zfs_fletcher_aarch64_neon.c \ - zfs_fletcher_avx512.c \ - zfs_fletcher_intel.c \ - zfs_fletcher_sse.c \ - zfs_fletcher_superscalar.c \ - zfs_fletcher_superscalar4.c \ - zfs_namecheck.c \ - zfs_prop.c \ - zpool_prop.c \ - zprop_common.c \ - abd.c \ - abd_os.c \ - aggsum.c \ - arc.c \ - arc_os.c \ - blkptr.c \ - bplist.c \ - bpobj.c \ - bptree.c \ - bqueue.c \ - btree.c \ - cityhash.c \ - dbuf.c \ - dbuf_stats.c \ - ddt.c \ - ddt_zap.c \ - dmu.c \ - dmu_diff.c \ - dmu_object.c \ - dmu_objset.c \ - dmu_recv.c \ - dmu_redact.c \ - dmu_send.c \ - dmu_traverse.c \ - dmu_tx.c \ - dmu_zfetch.c \ - dnode.c \ - dnode_sync.c \ - dsl_bookmark.c \ - dsl_crypt.c \ - dsl_dataset.c \ - dsl_deadlist.c \ - dsl_deleg.c \ - dsl_destroy.c \ - dsl_dir.c \ - dsl_pool.c \ - dsl_prop.c \ - dsl_scan.c \ - dsl_synctask.c \ - dsl_userhold.c \ - edonr_zfs.c \ - fm.c \ - gzip.c \ - hkdf.c \ - lz4.c \ - lz4_zfs.c \ - lzjb.c \ - metaslab.c \ - mmp.c \ - multilist.c \ - objlist.c \ - pathname.c \ - range_tree.c \ - refcount.c \ - rrwlock.c \ - sa.c \ - sha256.c \ - skein_zfs.c \ - spa.c \ - spa_boot.c \ - spa_checkpoint.c \ - spa_config.c \ - spa_errlog.c \ - spa_history.c \ - spa_log_spacemap.c \ - spa_misc.c \ - spa_stats.c \ - space_map.c \ - space_reftree.c \ - trace.c \ - txg.c \ - uberblock.c \ - unique.c \ - vdev.c \ - vdev_cache.c \ - vdev_draid.c \ - vdev_draid_rand.c \ - vdev_file.c \ - vdev_indirect.c \ - vdev_indirect_births.c \ - vdev_indirect_mapping.c \ - vdev_initialize.c \ - vdev_label.c \ - vdev_mirror.c \ - vdev_missing.c \ - vdev_queue.c \ - vdev_raidz.c \ - vdev_raidz_math.c \ - vdev_raidz_math_aarch64_neon.c \ - vdev_raidz_math_aarch64_neonx2.c \ - vdev_raidz_math_avx2.c \ - vdev_raidz_math_avx512bw.c \ - vdev_raidz_math_avx512f.c \ - vdev_raidz_math_powerpc_altivec.c \ - vdev_raidz_math_scalar.c \ - vdev_raidz_math_sse2.c \ - vdev_raidz_math_ssse3.c \ - vdev_rebuild.c \ - vdev_removal.c \ - vdev_root.c \ - vdev_trim.c \ - zap.c \ - zap_leaf.c \ - zap_micro.c \ - zcp.c \ - zcp_get.c \ - zcp_global.c \ - zcp_iter.c \ - zcp_set.c \ - zcp_synctask.c \ - zfeature.c \ - zfs_byteswap.c \ - zfs_debug.c \ - zfs_fm.c \ - zfs_fuid.c \ - zfs_racct.c \ - zfs_sa.c \ - zfs_ratelimit.c \ - zfs_rlock.c \ - zfs_znode.c \ - zil.c \ - zio.c \ - zio_checksum.c \ - zio_compress.c \ - zio_crypt.c \ - zio_inject.c \ - zle.c \ - zrlock.c \ - zthr.c - -LUA_C = \ - lapi.c \ - lauxlib.c \ - lbaselib.c \ - lcode.c \ - lcompat.c \ - lcorolib.c \ - lctype.c \ - ldebug.c \ - ldo.c \ - lfunc.c \ - lgc.c \ - llex.c \ - lmem.c \ - lobject.c \ - lopcodes.c \ - lparser.c \ - lstate.c \ - lstring.c \ - lstrlib.c \ - ltable.c \ - ltablib.c \ - ltm.c \ - lvm.c \ - lzio.c +lib_LTLIBRARIES += libzpool.la +CPPCHECKTARGETS += libzpool.la dist_libzpool_la_SOURCES = \ - $(USER_C) + %D%/kernel.c \ + %D%/taskq.c \ + %D%/util.c nodist_libzpool_la_SOURCES = \ - $(KERNEL_C) \ - $(LUA_C) + module/lua/lapi.c \ + module/lua/lauxlib.c \ + module/lua/lbaselib.c \ + module/lua/lcode.c \ + module/lua/lcompat.c \ + module/lua/lcorolib.c \ + module/lua/lctype.c \ + module/lua/ldebug.c \ + module/lua/ldo.c \ + module/lua/lfunc.c \ + module/lua/lgc.c \ + module/lua/llex.c \ + module/lua/lmem.c \ + module/lua/lobject.c \ + module/lua/lopcodes.c \ + module/lua/lparser.c \ + module/lua/lstate.c \ + module/lua/lstring.c \ + module/lua/lstrlib.c \ + module/lua/ltable.c \ + module/lua/ltablib.c \ + module/lua/ltm.c \ + module/lua/lvm.c \ + module/lua/lzio.c \ + \ + module/os/linux/zfs/abd_os.c \ + module/os/linux/zfs/arc_os.c \ + module/os/linux/zfs/trace.c \ + module/os/linux/zfs/vdev_file.c \ + module/os/linux/zfs/zfs_debug.c \ + module/os/linux/zfs/zfs_racct.c \ + module/os/linux/zfs/zfs_znode.c \ + module/os/linux/zfs/zio_crypt.c \ + \ + module/zcommon/cityhash.c \ + module/zcommon/zfeature_common.c \ + module/zcommon/zfs_comutil.c \ + module/zcommon/zfs_deleg.c \ + module/zcommon/zfs_fletcher.c \ + module/zcommon/zfs_fletcher_aarch64_neon.c \ + module/zcommon/zfs_fletcher_avx512.c \ + module/zcommon/zfs_fletcher_intel.c \ + module/zcommon/zfs_fletcher_sse.c \ + module/zcommon/zfs_fletcher_superscalar.c \ + module/zcommon/zfs_fletcher_superscalar4.c \ + module/zcommon/zfs_namecheck.c \ + module/zcommon/zfs_prop.c \ + module/zcommon/zpool_prop.c \ + module/zcommon/zprop_common.c \ + \ + module/zfs/abd.c \ + module/zfs/aggsum.c \ + module/zfs/arc.c \ + module/zfs/blake3_zfs.c \ + module/zfs/blkptr.c \ + module/zfs/bplist.c \ + module/zfs/bpobj.c \ + module/zfs/bptree.c \ + module/zfs/bqueue.c \ + module/zfs/btree.c \ + module/zfs/dbuf.c \ + module/zfs/dbuf_stats.c \ + module/zfs/ddt.c \ + module/zfs/ddt_zap.c \ + module/zfs/dmu.c \ + module/zfs/dmu_diff.c \ + module/zfs/dmu_object.c \ + module/zfs/dmu_objset.c \ + module/zfs/dmu_recv.c \ + module/zfs/dmu_redact.c \ + module/zfs/dmu_send.c \ + module/zfs/dmu_traverse.c \ + module/zfs/dmu_tx.c \ + module/zfs/dmu_zfetch.c \ + module/zfs/dnode.c \ + module/zfs/dnode_sync.c \ + module/zfs/dsl_bookmark.c \ + module/zfs/dsl_crypt.c \ + module/zfs/dsl_dataset.c \ + module/zfs/dsl_deadlist.c \ + module/zfs/dsl_deleg.c \ + module/zfs/dsl_destroy.c \ + module/zfs/dsl_dir.c \ + module/zfs/dsl_pool.c \ + module/zfs/dsl_prop.c \ + module/zfs/dsl_scan.c \ + module/zfs/dsl_synctask.c \ + module/zfs/dsl_userhold.c \ + module/zfs/edonr_zfs.c \ + module/zfs/fm.c \ + module/zfs/gzip.c \ + module/zfs/hkdf.c \ + module/zfs/lz4.c \ + module/zfs/lz4_zfs.c \ + module/zfs/lzjb.c \ + module/zfs/metaslab.c \ + module/zfs/mmp.c \ + module/zfs/multilist.c \ + module/zfs/objlist.c \ + module/zfs/pathname.c \ + module/zfs/range_tree.c \ + module/zfs/refcount.c \ + module/zfs/rrwlock.c \ + module/zfs/sa.c \ + module/zfs/sha256.c \ + module/zfs/skein_zfs.c \ + module/zfs/spa.c \ + module/zfs/spa_boot.c \ + module/zfs/spa_checkpoint.c \ + module/zfs/spa_config.c \ + module/zfs/spa_errlog.c \ + module/zfs/spa_history.c \ + module/zfs/spa_log_spacemap.c \ + module/zfs/spa_misc.c \ + module/zfs/spa_stats.c \ + module/zfs/space_map.c \ + module/zfs/space_reftree.c \ + module/zfs/txg.c \ + module/zfs/uberblock.c \ + module/zfs/unique.c \ + module/zfs/vdev.c \ + module/zfs/vdev_cache.c \ + module/zfs/vdev_draid.c \ + module/zfs/vdev_draid_rand.c \ + module/zfs/vdev_indirect.c \ + module/zfs/vdev_indirect_births.c \ + module/zfs/vdev_indirect_mapping.c \ + module/zfs/vdev_initialize.c \ + module/zfs/vdev_label.c \ + module/zfs/vdev_mirror.c \ + module/zfs/vdev_missing.c \ + module/zfs/vdev_queue.c \ + module/zfs/vdev_raidz.c \ + module/zfs/vdev_raidz_math.c \ + module/zfs/vdev_raidz_math_aarch64_neon.c \ + module/zfs/vdev_raidz_math_aarch64_neonx2.c \ + module/zfs/vdev_raidz_math_avx2.c \ + module/zfs/vdev_raidz_math_avx512bw.c \ + module/zfs/vdev_raidz_math_avx512f.c \ + module/zfs/vdev_raidz_math_powerpc_altivec.c \ + module/zfs/vdev_raidz_math_scalar.c \ + module/zfs/vdev_raidz_math_sse2.c \ + module/zfs/vdev_raidz_math_ssse3.c \ + module/zfs/vdev_rebuild.c \ + module/zfs/vdev_removal.c \ + module/zfs/vdev_root.c \ + module/zfs/vdev_trim.c \ + module/zfs/zap.c \ + module/zfs/zap_leaf.c \ + module/zfs/zap_micro.c \ + module/zfs/zcp.c \ + module/zfs/zcp_get.c \ + module/zfs/zcp_global.c \ + module/zfs/zcp_iter.c \ + module/zfs/zcp_set.c \ + module/zfs/zcp_synctask.c \ + module/zfs/zfeature.c \ + module/zfs/zfs_byteswap.c \ + module/zfs/zfs_chksum.c \ + module/zfs/zfs_fm.c \ + module/zfs/zfs_fuid.c \ + module/zfs/zfs_ratelimit.c \ + module/zfs/zfs_rlock.c \ + module/zfs/zfs_sa.c \ + module/zfs/zil.c \ + module/zfs/zio.c \ + module/zfs/zio_checksum.c \ + module/zfs/zio_compress.c \ + module/zfs/zio_inject.c \ + module/zfs/zle.c \ + module/zfs/zrlock.c \ + module/zfs/zthr.c libzpool_la_LIBADD = \ - $(abs_top_builddir)/lib/libicp/libicp.la \ - $(abs_top_builddir)/lib/libunicode/libunicode.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la \ - $(abs_top_builddir)/lib/libzstd/libzstd.la \ - $(abs_top_builddir)/lib/libzutil/libzutil.la + libicp.la \ + libunicode.la \ + libnvpair.la \ + libzstd.la \ + libzutil.la libzpool_la_LIBADD += $(LIBCLOCK_GETTIME) $(ZLIB_LIBS) -ldl -lm @@ -234,8 +209,6 @@ endif libzpool_la_LDFLAGS += -version-info 5:0:0 if TARGET_CPU_POWERPC -vdev_raidz_math_powerpc_altivec.$(OBJEXT): CFLAGS += -maltivec -vdev_raidz_math_powerpc_altivec.l$(OBJEXT): CFLAGS += -maltivec +module/zfs/libzpool_la-vdev_raidz_math_powerpc_altivec.$(OBJEXT) : CFLAGS += -maltivec +module/zfs/libzpool_la-vdev_raidz_math_powerpc_altivec.l$(OBJEXT): CFLAGS += -maltivec endif - -include $(top_srcdir)/config/CppCheck.am diff --git a/lib/libzpool/kernel.c b/lib/libzpool/kernel.c index 41e0e7815c43..1f58acb0404f 100644 --- a/lib/libzpool/kernel.c +++ b/lib/libzpool/kernel.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -53,7 +53,7 @@ */ uint64_t physmem; -char hw_serial[HW_HOSTID_LEN]; +uint32_t hostid; struct utsname hw_utsname; /* If set, all blocks read will be copied to the specified directory. */ @@ -75,12 +75,28 @@ struct proc p0; #define TS_STACK_MIN MAX(PTHREAD_STACK_MIN, 32768) #define TS_STACK_MAX (256 * 1024) +struct zk_thread_wrapper { + void (*func)(void *); + void *arg; +}; + +static void * +zk_thread_wrapper(void *arg) +{ + struct zk_thread_wrapper ztw; + memcpy(&ztw, arg, sizeof (ztw)); + free(arg); + ztw.func(ztw.arg); + return (NULL); +} + kthread_t * zk_thread_create(void (*func)(void *), void *arg, size_t stksize, int state) { pthread_attr_t attr; pthread_t tid; char *stkstr; + struct zk_thread_wrapper *ztw; int detachstate = PTHREAD_CREATE_DETACHED; VERIFY0(pthread_attr_init(&attr)); @@ -117,7 +133,10 @@ zk_thread_create(void (*func)(void *), void *arg, size_t stksize, int state) VERIFY0(pthread_attr_setstacksize(&attr, stksize)); VERIFY0(pthread_attr_setguardsize(&attr, PAGESIZE)); - VERIFY0(pthread_create(&tid, &attr, (void *(*)(void *))func, arg)); + VERIFY(ztw = malloc(sizeof (*ztw))); + ztw->func = func; + ztw->arg = arg; + VERIFY0(pthread_create(&tid, &attr, zk_thread_wrapper, ztw)); VERIFY0(pthread_attr_destroy(&attr)); return ((void *)(uintptr_t)tid); @@ -280,7 +299,7 @@ zone_get_hostid(void *zonep) * We're emulating the system's hostid in userland. */ (void) zonep; - return (strtoul(hw_serial, NULL, 10)); + return (hostid); } int @@ -614,7 +633,7 @@ __dprintf(boolean_t dprint, const char *file, const char *func, static char ce_prefix[CE_IGNORE][10] = { "", "NOTICE: ", "WARNING: ", "" }; static char ce_suffix[CE_IGNORE][2] = { "", "\n", "\n", "" }; -void +__attribute__((noreturn)) void vpanic(const char *fmt, va_list adx) { (void) fprintf(stderr, "error: "); @@ -624,7 +643,7 @@ vpanic(const char *fmt, va_list adx) abort(); /* think of it as a "user-level crash dump" */ } -void +__attribute__((noreturn)) void panic(const char *fmt, ...) { va_list adx; @@ -747,18 +766,6 @@ random_get_pseudo_bytes(uint8_t *ptr, size_t len) return (random_get_bytes_common(ptr, len, urandom_fd)); } -int -ddi_strtoul(const char *hw_serial, char **nptr, int base, unsigned long *result) -{ - (void) nptr; - char *end; - - *result = strtoul(hw_serial, &end, base); - if (*result == 0) - return (errno); - return (0); -} - int ddi_strtoull(const char *str, char **nptr, int base, u_longlong_t *result) { @@ -804,8 +811,7 @@ kernel_init(int mode) dprintf("physmem = %llu pages (%.2f GB)\n", (u_longlong_t)physmem, (double)physmem * sysconf(_SC_PAGE_SIZE) / (1ULL << 30)); - (void) snprintf(hw_serial, sizeof (hw_serial), "%ld", - (mode & SPA_MODE_WRITE) ? get_system_hostid() : 0); + hostid = (mode & SPA_MODE_WRITE) ? get_system_hostid() : 0; random_init(); @@ -996,8 +1002,6 @@ kmem_cache_reap_active(void) return (0); } -void *zvol_tag = "zvol_tag"; - void zvol_create_minor(const char *name) { @@ -1405,3 +1409,27 @@ zfsvfs_update_fromname(const char *oldname, const char *newname) { (void) oldname, (void) newname; } + +void +spa_import_os(spa_t *spa) +{ + (void) spa; +} + +void +spa_export_os(spa_t *spa) +{ + (void) spa; +} + +void +spa_activate_os(spa_t *spa) +{ + (void) spa; +} + +void +spa_deactivate_os(spa_t *spa) +{ + (void) spa; +} diff --git a/lib/libzpool/taskq.c b/lib/libzpool/taskq.c index b72ca3d3996c..8d6f1c93d8c9 100644 --- a/lib/libzpool/taskq.c +++ b/lib/libzpool/taskq.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -211,7 +211,7 @@ taskq_wait_outstanding(taskq_t *tq, taskqid_t id) taskq_wait(tq); } -static void +static __attribute__((noreturn)) void taskq_thread(void *arg) { taskq_t *tq = arg; @@ -363,12 +363,6 @@ taskq_cancel_id(taskq_t *tq, taskqid_t id) return (ENOENT); } -boolean_t -taskq_empty(taskq_t *tq) -{ - return (tq->tq_task.tqent_next == &tq->tq_task || tq->tq_active == 0); -} - void system_taskq_init(void) { diff --git a/lib/libzpool/util.c b/lib/libzpool/util.c index b9ee52a61fc4..833e51398b1c 100644 --- a/lib/libzpool/util.c +++ b/lib/libzpool/util.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -64,7 +64,8 @@ show_vdev_stats(const char *desc, const char *ctype, nvlist_t *nv, int indent) } if (desc != NULL) { - char *suffix = "", *bias = NULL; + const char *suffix = ""; + char *bias = NULL; char bias_suffix[32]; (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_IS_LOG, &is_log); @@ -117,7 +118,7 @@ show_vdev_stats(const char *desc, const char *ctype, nvlist_t *nv, int indent) int len; if (nvlist_lookup_string(cnv, ZPOOL_CONFIG_PATH, &cname) && nvlist_lookup_string(cnv, ZPOOL_CONFIG_TYPE, &cname)) - cname = ""; + cname = (char *)""; len = strlen(cname) + 2; tname = umem_zalloc(len, UMEM_NOFAIL); (void) strlcpy(tname, cname, len); diff --git a/lib/libzstd/Makefile.am b/lib/libzstd/Makefile.am index c9ed7e2aafbc..49bfb328a6f7 100644 --- a/lib/libzstd/Makefile.am +++ b/lib/libzstd/Makefile.am @@ -1,23 +1,32 @@ -include $(top_srcdir)/config/Rules.am - -VPATH = $(top_srcdir)/module/zstd - +libzstd_la_CFLAGS = $(AM_CFLAGS) $(LIBRARY_CFLAGS) # -fno-tree-vectorize is set for gcc in zstd/common/compiler.h # Set it for other compilers, too. -AM_CFLAGS += -fno-tree-vectorize - -noinst_LTLIBRARIES = libzstd.la - -KERNEL_C = \ - lib/zstd.c \ - zfs_zstd.c - -nodist_libzstd_la_SOURCES = $(KERNEL_C) - -lib/zstd.$(OBJEXT): CFLAGS += -fno-tree-vectorize -include $(top_srcdir)/module/zstd/include/zstd_compat_wrapper.h -Wp,-w -lib/zstd.l$(OBJEXT): CFLAGS += -fno-tree-vectorize -include $(top_srcdir)/module/zstd/include/zstd_compat_wrapper.h -Wp,-w - -zfs_zstd.$(OBJEXT): CFLAGS += -include $(top_srcdir)/module/zstd/include/zstd_compat_wrapper.h -zfs_zstd.l$(OBJEXT): CFLAGS += -include $(top_srcdir)/module/zstd/include/zstd_compat_wrapper.h - -include $(top_srcdir)/config/CppCheck.am +libzstd_la_CFLAGS += -fno-tree-vectorize + +noinst_LTLIBRARIES += libzstd.la + +nodist_libzstd_la_SOURCES = \ + module/zstd/lib/common/entropy_common.c \ + module/zstd/lib/common/error_private.c \ + module/zstd/lib/common/fse_decompress.c \ + module/zstd/lib/common/pool.c \ + module/zstd/lib/common/zstd_common.c \ + module/zstd/lib/compress/fse_compress.c \ + module/zstd/lib/compress/hist.c \ + module/zstd/lib/compress/huf_compress.c \ + module/zstd/lib/compress/zstd_compress_literals.c \ + module/zstd/lib/compress/zstd_compress_sequences.c \ + module/zstd/lib/compress/zstd_compress_superblock.c \ + module/zstd/lib/compress/zstd_compress.c \ + module/zstd/lib/compress/zstd_double_fast.c \ + module/zstd/lib/compress/zstd_fast.c \ + module/zstd/lib/compress/zstd_lazy.c \ + module/zstd/lib/compress/zstd_ldm.c \ + module/zstd/lib/compress/zstd_opt.c \ + module/zstd/lib/decompress/huf_decompress.c \ + module/zstd/lib/decompress/zstd_ddict.c \ + module/zstd/lib/decompress/zstd_decompress.c \ + module/zstd/lib/decompress/zstd_decompress_block.c \ + module/zstd/zfs_zstd.c + +libzstd_la_CFLAGS += -include $(top_srcdir)/module/zstd/include/zstd_compat_wrapper.h -fno-tree-vectorize -Wp,-w $(AM_CFLAGS) diff --git a/lib/libzutil/Makefile.am b/lib/libzutil/Makefile.am index b163250619ba..ecdf940508b2 100644 --- a/lib/libzutil/Makefile.am +++ b/lib/libzutil/Makefile.am @@ -1,47 +1,42 @@ -include $(top_srcdir)/config/Rules.am +libzutil_la_CFLAGS = $(AM_CFLAGS) $(LIBRARY_CFLAGS) +libzutil_la_CFLAGS += $(LIBBLKID_CFLAGS) $(LIBUDEV_CFLAGS) +libzutil_la_CFLAGS += -fvisibility=hidden -# Suppress unused but set variable warnings often due to ASSERTs -AM_CFLAGS += $(NO_UNUSED_BUT_SET_VARIABLE) -AM_CFLAGS += $(LIBBLKID_CFLAGS) $(LIBUDEV_CFLAGS) -AM_CFLAGS += -fvisibility=hidden +libzutil_la_CPPFLAGS = $(AM_CPPFLAGS) +libzutil_la_CPPFLAGS += -I$(srcdir)/%D% -DEFAULT_INCLUDES += -I$(srcdir) +noinst_LTLIBRARIES += libzutil.la +CPPCHECKTARGETS += libzutil.la -noinst_LTLIBRARIES = libzutil.la - -USER_C = \ - zutil_device_path.c \ - zutil_import.c \ - zutil_import.h \ - zutil_nicenum.c \ - zutil_pool.c +libzutil_la_SOURCES = \ + %D%/zutil_device_path.c \ + %D%/zutil_import.c \ + %D%/zutil_import.h \ + %D%/zutil_nicenum.c \ + %D%/zutil_pool.c if BUILD_LINUX -USER_C += \ - os/linux/zutil_device_path_os.c \ - os/linux/zutil_import_os.c +libzutil_la_SOURCES += \ + %D%/os/linux/zutil_device_path_os.c \ + %D%/os/linux/zutil_import_os.c endif if BUILD_FREEBSD -USER_C += \ - os/freebsd/zutil_device_path_os.c \ - os/freebsd/zutil_import_os.c +libzutil_la_SOURCES += \ + %D%/os/freebsd/zutil_device_path_os.c \ + %D%/os/freebsd/zutil_import_os.c endif -libzutil_la_SOURCES = $(USER_C) - libzutil_la_LIBADD = \ - $(abs_top_builddir)/lib/libavl/libavl.la \ - $(abs_top_builddir)/lib/libtpool/libtpool.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la \ - $(abs_top_builddir)/lib/libspl/libspl.la + libavl.la \ + libtpool.la \ + libnvpair.la \ + libspl.la if BUILD_LINUX libzutil_la_LIBADD += \ - $(abs_top_builddir)/lib/libefi/libefi.la \ + libefi.la \ -lrt endif libzutil_la_LIBADD += -lm $(LIBBLKID_LIBS) $(LIBUDEV_LIBS) - -include $(top_srcdir)/config/CppCheck.am diff --git a/lib/libzutil/os/freebsd/zutil_device_path_os.c b/lib/libzutil/os/freebsd/zutil_device_path_os.c index ac4748ec7ac8..9c67d3af085a 100644 --- a/lib/libzutil/os/freebsd/zutil_device_path_os.c +++ b/lib/libzutil/os/freebsd/zutil_device_path_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -40,7 +40,7 @@ * Note: The caller must free the returned string. */ char * -zfs_strip_partition(char *dev) +zfs_strip_partition(const char *dev) { return (strdup(dev)); } @@ -56,8 +56,8 @@ zfs_append_partition(char *path, size_t max_len) * On FreeBSD we only want to remove "/dev/" from the beginning of * paths if present. */ -char * -zfs_strip_path(char *path) +const char * +zfs_strip_path(const char *path) { if (strncmp(path, _PATH_DEV, sizeof (_PATH_DEV) - 1) == 0) return (path + sizeof (_PATH_DEV) - 1); diff --git a/lib/libzutil/os/freebsd/zutil_import_os.c b/lib/libzutil/os/freebsd/zutil_import_os.c index da9a7ded936a..19ba58e79a03 100644 --- a/lib/libzutil/os/freebsd/zutil_import_os.c +++ b/lib/libzutil/os/freebsd/zutil_import_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libzutil/os/linux/zutil_device_path_os.c b/lib/libzutil/os/linux/zutil_device_path_os.c index 17247233bcf2..f081ef53da79 100644 --- a/lib/libzutil/os/linux/zutil_device_path_os.c +++ b/lib/libzutil/os/linux/zutil_device_path_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -81,7 +81,7 @@ zfs_append_partition(char *path, size_t max_len) * caller must free the returned string */ char * -zfs_strip_partition(char *path) +zfs_strip_partition(const char *path) { char *tmp = strdup(path); char *part = NULL, *d = NULL; @@ -117,7 +117,7 @@ zfs_strip_partition(char *path) * Returned string must be freed. */ static char * -zfs_strip_partition_path(char *path) +zfs_strip_partition_path(const char *path) { char *newpath = strdup(path); char *sd_offset; @@ -148,10 +148,18 @@ zfs_strip_partition_path(char *path) /* * Strip the unwanted portion of a device path. */ -char * -zfs_strip_path(char *path) +const char * +zfs_strip_path(const char *path) { - return (strrchr(path, '/') + 1); + size_t spath_count; + const char *const *spaths = zpool_default_search_paths(&spath_count); + + for (size_t i = 0; i < spath_count; ++i) + if (strncmp(path, spaths[i], strlen(spaths[i])) == 0 && + path[strlen(spaths[i])] == '/') + return (path + strlen(spaths[i]) + 1); + + return (path); } /* @@ -527,7 +535,7 @@ zfs_dev_is_dm(const char *dev_name) boolean_t zfs_dev_is_whole_disk(const char *dev_name) { - struct dk_gpt *label; + struct dk_gpt *label = NULL; int fd; if ((fd = open(dev_name, O_RDONLY | O_DIRECT | O_CLOEXEC)) < 0) @@ -613,22 +621,24 @@ zfs_get_underlying_path(const char *dev_name) /* * A disk is considered a multipath whole disk when: * DEVNAME key value has "dm-" - * DM_NAME key value has "mpath" prefix - * DM_UUID key exists + * DM_UUID key exists and starts with 'mpath-' * ID_PART_TABLE_TYPE key does not exist or is not gpt + * ID_FS_LABEL key does not exist (disk isn't labeled) */ static boolean_t -udev_mpath_whole_disk(struct udev_device *dev) +is_mpath_udev_sane(struct udev_device *dev) { - const char *devname, *type, *uuid; + const char *devname, *type, *uuid, *label; devname = udev_device_get_property_value(dev, "DEVNAME"); type = udev_device_get_property_value(dev, "ID_PART_TABLE_TYPE"); uuid = udev_device_get_property_value(dev, "DM_UUID"); + label = udev_device_get_property_value(dev, "ID_FS_LABEL"); if ((devname != NULL && strncmp(devname, "/dev/dm-", 8) == 0) && ((type == NULL) || (strcmp(type, "gpt") != 0)) && - (uuid != NULL)) { + ((uuid != NULL) && (strncmp(uuid, "mpath-", 6) == 0)) && + (label == NULL)) { return (B_TRUE); } @@ -636,7 +646,11 @@ udev_mpath_whole_disk(struct udev_device *dev) } /* - * Check if a disk is effectively a multipath whole disk + * Check if a disk is a multipath "blank" disk: + * + * 1. The disk has udev values that suggest it's a multipath disk + * 2. The disk is not currently labeled with a filesystem of any type + * 3. There are no partitions on the disk */ boolean_t is_mpath_whole_disk(const char *path) @@ -645,7 +659,6 @@ is_mpath_whole_disk(const char *path) struct udev_device *dev = NULL; char nodepath[MAXPATHLEN]; char *sysname; - boolean_t wholedisk = B_FALSE; if (realpath(path, nodepath) == NULL) return (B_FALSE); @@ -660,10 +673,11 @@ is_mpath_whole_disk(const char *path) return (B_FALSE); } - wholedisk = udev_mpath_whole_disk(dev); - + /* Sanity check some udev values */ + boolean_t is_sane = is_mpath_udev_sane(dev); udev_device_unref(dev); - return (wholedisk); + + return (is_sane); } #else /* HAVE_LIBUDEV */ diff --git a/lib/libzutil/os/linux/zutil_import_os.c b/lib/libzutil/os/linux/zutil_import_os.c index 60d3369c9c45..eb2417b47987 100644 --- a/lib/libzutil/os/linux/zutil_import_os.c +++ b/lib/libzutil/os/linux/zutil_import_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -264,36 +264,36 @@ zpool_default_search_paths(size_t *count) * index in the passed 'order' variable, otherwise return an error. */ static int -zfs_path_order(char *name, int *order) +zfs_path_order(const char *name, int *order) { - int i, error = ENOENT; - char *dir, *env, *envdup, *tmp = NULL; + const char *env = getenv("ZPOOL_IMPORT_PATH"); - env = getenv("ZPOOL_IMPORT_PATH"); if (env) { - envdup = strdup(env); - for (dir = strtok_r(envdup, ":", &tmp), i = 0; - dir != NULL; - dir = strtok_r(NULL, ":", &tmp), i++) { - if (strncmp(name, dir, strlen(dir)) == 0) { - *order = i; - error = 0; + for (int i = 0; ; ++i) { + env += strspn(env, ":"); + size_t dirlen = strcspn(env, ":"); + if (dirlen) { + if (strncmp(name, env, dirlen) == 0) { + *order = i; + return (0); + } + + env += dirlen; + } else break; - } } - free(envdup); } else { - for (i = 0; i < ARRAY_SIZE(zpool_default_import_path); i++) { + for (int i = 0; i < ARRAY_SIZE(zpool_default_import_path); + ++i) { if (strncmp(name, zpool_default_import_path[i], strlen(zpool_default_import_path[i])) == 0) { *order = i; - error = 0; - break; + return (0); } } } - return (error); + return (ENOENT); } /* @@ -328,7 +328,9 @@ zpool_find_import_blkid(libpc_handle_t *hdl, pthread_mutex_t *lock, return (EINVAL); } - error = blkid_dev_set_search(iter, "TYPE", "zfs_member"); + /* Only const char *s since 2.32 */ + error = blkid_dev_set_search(iter, + (char *)"TYPE", (char *)"zfs_member"); if (error != 0) { blkid_dev_iterate_end(iter); blkid_put_cache(cache); @@ -754,6 +756,9 @@ encode_device_strings(const char *path, vdev_dev_strs_t *ds, return (ret); #else + (void) path; + (void) ds; + (void) wholedisk; return (ENOENT); #endif } diff --git a/lib/libzutil/zutil_device_path.c b/lib/libzutil/zutil_device_path.c index 435c444b2460..0425018e1022 100644 --- a/lib/libzutil/zutil_device_path.c +++ b/lib/libzutil/zutil_device_path.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -56,35 +56,36 @@ zfs_dirnamelen(const char *path) int zfs_resolve_shortname(const char *name, char *path, size_t len) { - int i, error = -1; - char *dir, *env, *envdup, *tmp = NULL; - - env = getenv("ZPOOL_IMPORT_PATH"); - errno = ENOENT; + const char *env = getenv("ZPOOL_IMPORT_PATH"); if (env) { - envdup = strdup(env); - for (dir = strtok_r(envdup, ":", &tmp); - dir != NULL && error != 0; - dir = strtok_r(NULL, ":", &tmp)) { - (void) snprintf(path, len, "%s/%s", dir, name); - error = access(path, F_OK); + for (;;) { + env += strspn(env, ":"); + size_t dirlen = strcspn(env, ":"); + if (dirlen) { + (void) snprintf(path, len, "%.*s/%s", + (int)dirlen, env, name); + if (access(path, F_OK) == 0) + return (0); + + env += dirlen; + } else + break; } - free(envdup); } else { - const char * const *zpool_default_import_path; size_t count; + const char *const *zpool_default_import_path = + zpool_default_search_paths(&count); - zpool_default_import_path = zpool_default_search_paths(&count); - - for (i = 0; i < count && error < 0; i++) { + for (size_t i = 0; i < count; ++i) { (void) snprintf(path, len, "%s/%s", zpool_default_import_path[i], name); - error = access(path, F_OK); + if (access(path, F_OK) == 0) + return (0); } } - return (error ? ENOENT : 0); + return (errno = ENOENT); } /* @@ -100,7 +101,7 @@ zfs_strcmp_shortname(const char *name, const char *cmp_name, int wholedisk) int path_len, cmp_len, i = 0, error = ENOENT; char *dir, *env, *envdup = NULL, *tmp = NULL; char path_name[MAXPATHLEN]; - const char * const *zpool_default_import_path = NULL; + const char *const *zpool_default_import_path = NULL; size_t count; cmp_len = strlen(cmp_name); diff --git a/lib/libzutil/zutil_import.c b/lib/libzutil/zutil_import.c index d7547c4249ea..0bbf232f24a4 100644 --- a/lib/libzutil/zutil_import.c +++ b/lib/libzutil/zutil_import.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -47,7 +47,9 @@ * using our derived config, and record the results. */ +#ifdef HAVE_AIO_H #include +#endif #include #include #include @@ -982,6 +984,9 @@ zpool_read_label_slow(int fd, nvlist_t **config, int *num_labels) int zpool_read_label(int fd, nvlist_t **config, int *num_labels) { +#ifndef HAVE_AIO_H + return (zpool_read_label_slow(fd, config, num_labels)); +#else struct stat64 statbuf; struct aiocb aiocbs[VDEV_LABELS]; struct aiocb *aiocbps[VDEV_LABELS]; @@ -1104,6 +1109,7 @@ zpool_read_label(int fd, nvlist_t **config, int *num_labels) *config = expected_config; return (0); +#endif } /* @@ -1537,7 +1543,7 @@ discover_cached_paths(libpc_handle_t *hdl, nvlist_t *nv, */ if (nvlist_lookup_string(nv, ZPOOL_CONFIG_PATH, &path) == 0) { if ((dl = zfs_dirnamelen(path)) == -1) - path = "."; + path = (char *)"."; else path[dl] = '\0'; return (zpool_find_import_scan_dir(hdl, lock, cache, diff --git a/lib/libzutil/zutil_import.h b/lib/libzutil/zutil_import.h index 0108eb45c5cf..b68a7fcfcb78 100644 --- a/lib/libzutil/zutil_import.h +++ b/lib/libzutil/zutil_import.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libzutil/zutil_nicenum.c b/lib/libzutil/zutil_nicenum.c index 4dcac1f855ff..e05bcb40a8ce 100644 --- a/lib/libzutil/zutil_nicenum.c +++ b/lib/libzutil/zutil_nicenum.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/lib/libzutil/zutil_pool.c b/lib/libzutil/zutil_pool.c index 734650f3cffc..288a0033cd13 100644 --- a/lib/libzutil/zutil_pool.c +++ b/lib/libzutil/zutil_pool.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -120,8 +120,9 @@ zpool_history_unpack(char *buf, uint64_t bytes_read, uint64_t *leftover, break; /* unpack record */ - if (nvlist_unpack(buf + sizeof (reclen), reclen, &nv, 0) != 0) - return (ENOMEM); + int err = nvlist_unpack(buf + sizeof (reclen), reclen, &nv, 0); + if (err != 0) + return (err); bytes_read -= sizeof (reclen) + reclen; buf += sizeof (reclen) + reclen; diff --git a/man/Makefile.am b/man/Makefile.am index 8ab1b757242c..12f818372f37 100644 --- a/man/Makefile.am +++ b/man/Makefile.am @@ -1,117 +1,126 @@ -include $(top_srcdir)/config/Substfiles.am - -EXTRA_DIST += \ - man1/cstyle.1 +dist_noinst_man_MANS = \ + %D%/man1/cstyle.1 dist_man_MANS = \ - man1/zhack.1 \ - man1/ztest.1 \ - man1/raidz_test.1 \ - man1/zvol_wait.1 \ - man1/arcstat.1 \ + %D%/man1/arcstat.1 \ + %D%/man1/raidz_test.1 \ + %D%/man1/test-runner.1 \ + %D%/man1/zhack.1 \ + %D%/man1/ztest.1 \ + %D%/man1/zvol_wait.1 \ \ - man5/vdev_id.conf.5 \ + %D%/man5/vdev_id.conf.5 \ \ - man4/spl.4 \ - man4/zfs.4 \ + %D%/man4/spl.4 \ + %D%/man4/zfs.4 \ \ - man7/zpool-features.7 \ - man7/zfsconcepts.7 \ - man7/zfsprops.7 \ - man7/zpoolconcepts.7 \ - man7/zpoolprops.7 \ + %D%/man7/dracut.zfs.7 \ + %D%/man7/zfsconcepts.7 \ + %D%/man7/zfsprops.7 \ + %D%/man7/zpool-features.7 \ + %D%/man7/zpoolconcepts.7 \ + %D%/man7/zpoolprops.7 \ \ - man8/fsck.zfs.8 \ - man8/mount.zfs.8 \ - man8/vdev_id.8 \ - man8/zdb.8 \ - man8/zfs.8 \ - man8/zfs-allow.8 \ - man8/zfs-bookmark.8 \ - man8/zfs-change-key.8 \ - man8/zfs-clone.8 \ - man8/zfs-create.8 \ - man8/zfs-destroy.8 \ - man8/zfs-diff.8 \ - man8/zfs-get.8 \ - man8/zfs-groupspace.8 \ - man8/zfs-hold.8 \ - man8/zfs-inherit.8 \ - man8/zfs-jail.8 \ - man8/zfs-list.8 \ - man8/zfs-load-key.8 \ - man8/zfs-mount.8 \ - man8/zfs-program.8 \ - man8/zfs-project.8 \ - man8/zfs-projectspace.8 \ - man8/zfs-promote.8 \ - man8/zfs-receive.8 \ - man8/zfs-recv.8 \ - man8/zfs-redact.8 \ - man8/zfs-release.8 \ - man8/zfs-rename.8 \ - man8/zfs-rollback.8 \ - man8/zfs-send.8 \ - man8/zfs-set.8 \ - man8/zfs-share.8 \ - man8/zfs-snapshot.8 \ - man8/zfs-unallow.8 \ - man8/zfs-unjail.8 \ - man8/zfs-unload-key.8 \ - man8/zfs-unmount.8 \ - man8/zfs-upgrade.8 \ - man8/zfs-userspace.8 \ - man8/zfs-wait.8 \ - man8/zfs_ids_to_path.8 \ - man8/zgenhostid.8 \ - man8/zinject.8 \ - man8/zpool.8 \ - man8/zpool-add.8 \ - man8/zpool-attach.8 \ - man8/zpool-checkpoint.8 \ - man8/zpool-clear.8 \ - man8/zpool-create.8 \ - man8/zpool-destroy.8 \ - man8/zpool-detach.8 \ - man8/zpool-events.8 \ - man8/zpool-export.8 \ - man8/zpool-get.8 \ - man8/zpool-history.8 \ - man8/zpool-import.8 \ - man8/zpool-initialize.8 \ - man8/zpool-iostat.8 \ - man8/zpool-labelclear.8 \ - man8/zpool-list.8 \ - man8/zpool-offline.8 \ - man8/zpool-online.8 \ - man8/zpool-reguid.8 \ - man8/zpool-remove.8 \ - man8/zpool-reopen.8 \ - man8/zpool-replace.8 \ - man8/zpool-resilver.8 \ - man8/zpool-scrub.8 \ - man8/zpool-set.8 \ - man8/zpool-split.8 \ - man8/zpool-status.8 \ - man8/zpool-sync.8 \ - man8/zpool-trim.8 \ - man8/zpool-upgrade.8 \ - man8/zpool-wait.8 \ - man8/zstream.8 \ - man8/zstreamdump.8 \ - man8/zpool_influxdb.8 + %D%/man8/fsck.zfs.8 \ + %D%/man8/mount.zfs.8 \ + %D%/man8/vdev_id.8 \ + %D%/man8/zdb.8 \ + %D%/man8/zfs.8 \ + %D%/man8/zfs-allow.8 \ + %D%/man8/zfs-bookmark.8 \ + %D%/man8/zfs-change-key.8 \ + %D%/man8/zfs-clone.8 \ + %D%/man8/zfs-create.8 \ + %D%/man8/zfs-destroy.8 \ + %D%/man8/zfs-diff.8 \ + %D%/man8/zfs-get.8 \ + %D%/man8/zfs-groupspace.8 \ + %D%/man8/zfs-hold.8 \ + %D%/man8/zfs-inherit.8 \ + %D%/man8/zfs-jail.8 \ + %D%/man8/zfs-list.8 \ + %D%/man8/zfs-load-key.8 \ + %D%/man8/zfs-mount.8 \ + %D%/man8/zfs-program.8 \ + %D%/man8/zfs-project.8 \ + %D%/man8/zfs-projectspace.8 \ + %D%/man8/zfs-promote.8 \ + %D%/man8/zfs-receive.8 \ + %D%/man8/zfs-recv.8 \ + %D%/man8/zfs-redact.8 \ + %D%/man8/zfs-release.8 \ + %D%/man8/zfs-rename.8 \ + %D%/man8/zfs-rollback.8 \ + %D%/man8/zfs-send.8 \ + %D%/man8/zfs-set.8 \ + %D%/man8/zfs-share.8 \ + %D%/man8/zfs-snapshot.8 \ + %D%/man8/zfs-unallow.8 \ + %D%/man8/zfs-unjail.8 \ + %D%/man8/zfs-unload-key.8 \ + %D%/man8/zfs-unmount.8 \ + %D%/man8/zfs-unzone.8 \ + %D%/man8/zfs-upgrade.8 \ + %D%/man8/zfs-userspace.8 \ + %D%/man8/zfs-wait.8 \ + %D%/man8/zfs-zone.8 \ + %D%/man8/zfs_ids_to_path.8 \ + %D%/man8/zgenhostid.8 \ + %D%/man8/zinject.8 \ + %D%/man8/zpool.8 \ + %D%/man8/zpool-add.8 \ + %D%/man8/zpool-attach.8 \ + %D%/man8/zpool-checkpoint.8 \ + %D%/man8/zpool-clear.8 \ + %D%/man8/zpool-create.8 \ + %D%/man8/zpool-destroy.8 \ + %D%/man8/zpool-detach.8 \ + %D%/man8/zpool-events.8 \ + %D%/man8/zpool-export.8 \ + %D%/man8/zpool-get.8 \ + %D%/man8/zpool-history.8 \ + %D%/man8/zpool-import.8 \ + %D%/man8/zpool-initialize.8 \ + %D%/man8/zpool-iostat.8 \ + %D%/man8/zpool-labelclear.8 \ + %D%/man8/zpool-list.8 \ + %D%/man8/zpool-offline.8 \ + %D%/man8/zpool-online.8 \ + %D%/man8/zpool-reguid.8 \ + %D%/man8/zpool-remove.8 \ + %D%/man8/zpool-reopen.8 \ + %D%/man8/zpool-replace.8 \ + %D%/man8/zpool-resilver.8 \ + %D%/man8/zpool-scrub.8 \ + %D%/man8/zpool-set.8 \ + %D%/man8/zpool-split.8 \ + %D%/man8/zpool-status.8 \ + %D%/man8/zpool-sync.8 \ + %D%/man8/zpool-trim.8 \ + %D%/man8/zpool-upgrade.8 \ + %D%/man8/zpool-wait.8 \ + %D%/man8/zstream.8 \ + %D%/man8/zstreamdump.8 \ + %D%/man8/zpool_influxdb.8 nodist_man_MANS = \ - man8/zed.8 \ - man8/zfs-mount-generator.8 + %D%/man8/zed.8 \ + %D%/man8/zfs-mount-generator.8 + +dist_noinst_DATA += $(dist_noinst_man_MANS) $(dist_man_MANS) SUBSTFILES += $(nodist_man_MANS) +CHECKS += mancheck +mancheck: + $(top_srcdir)/scripts/mancheck.sh $(srcdir)/%D% + if BUILD_LINUX # The manual pager in most Linux distros defaults to "BSD" when .Os is blank, # but leaving it blank makes things a lot easier on # FreeBSD when OpenZFS is vendored in the base system. -install-data-hook: - cd $(DESTDIR)$(mandir) && $(SED) ${ac_inplace} -e 's/^\.Os$$/.Os OpenZFS/' $(dist_man_MANS) $(nodist_man_MANS) +INSTALL_DATA_HOOKS += man-install-data-hook +man-install-data-hook: + cd $(DESTDIR)$(mandir) && $(SED) $(ac_inplace) 's/^\.Os$$/.Os OpenZFS/' $(subst %D%/,,$(dist_man_MANS) $(nodist_man_MANS)) endif diff --git a/man/man1/cstyle.1 b/man/man1/cstyle.1 index f5f9ec78f827..726db298437e 100644 --- a/man/man1/cstyle.1 +++ b/man/man1/cstyle.1 @@ -8,7 +8,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -30,7 +30,6 @@ .Sh SYNOPSIS .Nm .Op Fl chpvCP -.Op Fl o Ar construct Ns Op , Ns Ar construct Ns … .Oo Ar file Oc Ns … .Sh DESCRIPTION .Nm @@ -45,7 +44,6 @@ does not mean that you've followed Sun's C style. .Em Caveat emptor . . .Sh OPTIONS -The following options are supported: .Bl -tag -width "-c" .It Fl c Check continuation line indentation inside of functions. @@ -59,9 +57,6 @@ continuation line problems within functions only. The checks have some limitations; see .Sy CONTINUATION CHECKING , below. -.It Fl h -Performs heuristic checks that are sometimes wrong. -Not generally used. .It Fl p Performs some of the more picky checks. Includes ANSI @@ -74,10 +69,6 @@ Used as part of the putback checks. Verbose output; includes the text of the line of error, and, for .Fl c , the first statement in the current continuation block. -.It Fl C -Ignore errors in header comments (i.e. block comments starting in the -first column). -Not generally used. .It Fl P Check for use of non-POSIX types. Historically, types like @@ -91,16 +82,17 @@ types etc. This detects any use of the deprecated types. Used as part of the putback checks. -.It Fl o Ar construct Ns Op , Ns Ar construct Ns … -Available constructs include: -.Bl -tag -compact -width "doxygen" -.It Sy doxygen -Allow doxygen-style block comments -.Pq Sy /** No and Sy /*!\& . -.It Sy splint -Allow splint-style lint comments -.Pq Sy /*@ Ns ... Ns Sy @*/ . +.It Fl g +Also print GitHub-Actions-style +.Li ::error +output. .El +. +.Sh ENVIRONMENT +.Bl -tag -compact -width ".Ev CI" +.It Ev CI +If set and nonempty, equivalent to +.Fl g . .El . .Sh CONTINUATION CHECKING diff --git a/man/man1/raidz_test.1 b/man/man1/raidz_test.1 index 4283a4b527f3..d027230d0dc9 100644 --- a/man/man1/raidz_test.1 +++ b/man/man1/raidz_test.1 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -40,7 +40,7 @@ The purpose of this tool is to run all supported raidz implementation and verify the results of all methods. It also contains a parameter sweep option where all -parameters affecting a RAIDZ block are verified (like ashift size, data offset, +parameters affecting a RAID-Z block are verified (like ashift size, data offset, data size, etc.). The tool also supports a benchmarking mode using the .Fl B diff --git a/tests/test-runner/man/test-runner.1 b/man/man1/test-runner.1 similarity index 99% rename from tests/test-runner/man/test-runner.1 rename to man/man1/test-runner.1 index f7cbcbc5b9e9..b823aaa3e1a0 100644 --- a/tests/test-runner/man/test-runner.1 +++ b/man/man1/test-runner.1 @@ -210,6 +210,8 @@ to be consumed by the run command. .It Fl d Dry run mode. Execute no tests, but print a description of each test that would have been run. +.It Fl m +Enable kmemleak reporting (Linux only) .It Fl g Create test groups from any directories found while searching for tests. .It Fl o Ar outputdir diff --git a/man/man1/zhack.1 b/man/man1/zhack.1 index b03b87a1bdab..42d4040d8be9 100644 --- a/man/man1/zhack.1 +++ b/man/man1/zhack.1 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man1/ztest.1 b/man/man1/ztest.1 index fd1374a2f106..6b9ef9616ea4 100644 --- a/man/man1/ztest.1 +++ b/man/man1/ztest.1 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -157,7 +157,17 @@ Time per pass. Max loops in .Fn spa_freeze . .It Fl B , -alt-ztest Ns = -Alternate ztest path. +Path to alternate ("older") +.Nm ztest +to drive, which will be used to initialise the pool, and, a stochastic half the time, to run the tests. +The parallel +.Pa lib +directory is prepended to +.Ev LD_LIBRARY_PATH ; +i.e. given +.Fl B Pa ./chroots/lenny/usr/bin/ Ns Nm , +.Pa ./chroots/lenny/usr/lib +will be loaded. .It Fl C , -vdev-class-state Ns = Ns Sy on Ns | Ns Sy off Ns | Ns Sy random No (default: Sy random ) The vdev allocation class state. .It Fl o , -option Ns = Ns Ar variable Ns = Ns Ar value diff --git a/man/man4/spl.4 b/man/man4/spl.4 index 11cde14ae5ca..28a2583a0d5b 100644 --- a/man/man4/spl.4 +++ b/man/man4/spl.4 @@ -2,7 +2,7 @@ .\" The contents of this file are subject to the terms of the Common Development .\" and Distribution License (the "License"). You may not use this file except .\" in compliance with the License. You can obtain a copy of the license at -.\" usr/src/OPENSOLARIS.LICENSE or http://www.opensolaris.org/os/licensing. +.\" usr/src/OPENSOLARIS.LICENSE or https://opensource.org/licenses/CDDL-1.0. .\" .\" See the License for the specific language governing permissions and .\" limitations under the License. When distributing Covered Code, include this diff --git a/man/man4/zfs.4 b/man/man4/zfs.4 index b2e850f840a1..cc55ee32ba24 100644 --- a/man/man4/zfs.4 +++ b/man/man4/zfs.4 @@ -5,7 +5,7 @@ .\" The contents of this file are subject to the terms of the Common Development .\" and Distribution License (the "License"). You may not use this file except .\" in compliance with the License. You can obtain a copy of the license at -.\" usr/src/OPENSOLARIS.LICENSE or http://www.opensolaris.org/os/licensing. +.\" usr/src/OPENSOLARIS.LICENSE or https://opensource.org/licenses/CDDL-1.0. .\" .\" See the License for the specific language governing permissions and .\" limitations under the License. When distributing Covered Code, include this @@ -70,7 +70,7 @@ to a log2 fraction of the target ARC size. dnode slots allocated in a single operation as a power of 2. The default value minimizes lock contention for the bulk operation performed. . -.It Sy dmu_prefetch_max Ns = Ns Sy 134217728 Ns B Po 128MB Pc Pq int +.It Sy dmu_prefetch_max Ns = Ns Sy 134217728 Ns B Po 128 MiB Pc Pq int Limit the amount we can prefetch with one call to this amount in bytes. This helps to limit the amount of memory that can be used by prefetching. . @@ -164,7 +164,7 @@ If set to .Sy 100 we TRIM twice the space required to accommodate upcoming writes. A minimum of -.Sy 64MB +.Sy 64 MiB will be trimmed. It also enables TRIM of the whole L2ARC device upon creation or addition to an existing pool or if the header of the device is @@ -194,12 +194,12 @@ to enable caching/reading prefetches to/from L2ARC. .It Sy l2arc_norw Ns = Ns Sy 0 Ns | Ns 1 Pq int No reads during writes. . -.It Sy l2arc_write_boost Ns = Ns Sy 8388608 Ns B Po 8MB Pc Pq ulong +.It Sy l2arc_write_boost Ns = Ns Sy 8388608 Ns B Po 8 MiB Pc Pq ulong Cold L2ARC devices will have .Sy l2arc_write_max increased by this amount while they remain cold. . -.It Sy l2arc_write_max Ns = Ns Sy 8388608 Ns B Po 8MB Pc Pq ulong +.It Sy l2arc_write_max Ns = Ns Sy 8388608 Ns B Po 8 MiB Pc Pq ulong Max write bytes per interval. . .It Sy l2arc_rebuild_enabled Ns = Ns Sy 1 Ns | Ns 0 Pq int @@ -209,35 +209,35 @@ or attaching an L2ARC device (e.g. the L2ARC device is slow in reading stored log metadata, or the metadata has become somehow fragmented/unusable). . -.It Sy l2arc_rebuild_blocks_min_l2size Ns = Ns Sy 1073741824 Ns B Po 1GB Pc Pq ulong +.It Sy l2arc_rebuild_blocks_min_l2size Ns = Ns Sy 1073741824 Ns B Po 1 GiB Pc Pq ulong Mininum size of an L2ARC device required in order to write log blocks in it. The log blocks are used upon importing the pool to rebuild the persistent L2ARC. .Pp -For L2ARC devices less than 1GB, the amount of data +For L2ARC devices less than 1 GiB, the amount of data .Fn l2arc_evict evicts is significant compared to the amount of restored L2ARC data. In this case, do not write log blocks in L2ARC in order not to waste space. . -.It Sy metaslab_aliquot Ns = Ns Sy 524288 Ns B Po 512kB Pc Pq ulong +.It Sy metaslab_aliquot Ns = Ns Sy 1048576 Ns B Po 1 MiB Pc Pq ulong Metaslab granularity, in bytes. This is roughly similar to what would be referred to as the "stripe size" in traditional RAID arrays. -In normal operation, ZFS will try to write this amount of data -to a top-level vdev before moving on to the next one. +In normal operation, ZFS will try to write this amount of data to each disk +before moving on to the next top-level vdev. . .It Sy metaslab_bias_enabled Ns = Ns Sy 1 Ns | Ns 0 Pq int Enable metaslab group biasing based on their vdevs' over- or under-utilization relative to the pool. . -.It Sy metaslab_force_ganging Ns = Ns Sy 16777217 Ns B Ns B Po 16MB + 1B Pc Pq ulong +.It Sy metaslab_force_ganging Ns = Ns Sy 16777217 Ns B Po 16 MiB + 1 B Pc Pq ulong Make some blocks above a certain size be gang blocks. This option is used by the test suite to facilitate testing. . -.It Sy zfs_history_output_max Ns = Ns Sy 1048576 Ns B Ns B Po 1MB Pc Pq int +.It Sy zfs_history_output_max Ns = Ns Sy 1048576 Ns B Po 1 MiB Pc Pq int When attempting to log an output nvlist of an ioctl in the on-disk history, the output will not be stored if it is larger than this size (in bytes). This must be less than -.Sy DMU_MAX_ACCESS Pq 64MB . +.Sy DMU_MAX_ACCESS Pq 64 MiB . This applies primarily to .Fn zfs_ioc_channel_program Pq cf. Xr zfs-program 8 . . @@ -261,7 +261,7 @@ Prevent metaslabs from being unloaded. .It Sy metaslab_fragmentation_factor_enabled Ns = Ns Sy 1 Ns | Ns 0 Pq int Enable use of the fragmentation metric in computing metaslab weights. . -.It Sy metaslab_df_max_search Ns = Ns Sy 16777216 Ns B Po 16MB Pc Pq int +.It Sy metaslab_df_max_search Ns = Ns Sy 16777216 Ns B Po 16 MiB Pc Pq int Maximum distance to search forward from the last offset. Without this limit, fragmented pools can see .Em >100`000 @@ -270,7 +270,7 @@ iterations and becomes the performance limiting factor on high-performance storage. .Pp With the default setting of -.Sy 16MB , +.Sy 16 MiB , we typically see less than .Em 500 iterations, even with very fragmented @@ -279,7 +279,7 @@ pools. The maximum number of iterations possible is .Sy metaslab_df_max_search / 2^(ashift+1) . With the default setting of -.Sy 16MB +.Sy 16 MiB this is .Em 16*1024 Pq with Sy ashift Ns = Ns Sy 9 or @@ -293,7 +293,7 @@ this tunable controls which segment is used. If set, we will use the largest free segment. If unset, we will use a segment of at least the requested size. . -.It Sy zfs_metaslab_max_size_cache_sec Ns = Ns Sy 3600 Ns s Po 1h Pc Pq ulong +.It Sy zfs_metaslab_max_size_cache_sec Ns = Ns Sy 3600 Ns s Po 1 hour Pc Pq ulong When we unload a metaslab, we cache the size of the largest free chunk. We use that cached size to determine whether or not to load a metaslab for a given allocation. @@ -344,7 +344,7 @@ and the allocation can't actually be satisfied .It Sy zfs_vdev_default_ms_count Ns = Ns Sy 200 Pq int When a vdev is added, target this number of metaslabs per top-level vdev. . -.It Sy zfs_vdev_default_ms_shift Ns = Ns Sy 29 Po 512MB Pc Pq int +.It Sy zfs_vdev_default_ms_shift Ns = Ns Sy 29 Po 512 MiB Pc Pq int Default limit for metaslab size. . .It Sy zfs_vdev_max_auto_ashift Ns = Ns Sy ASHIFT_MAX Po 16 Pc Pq ulong @@ -380,7 +380,7 @@ Note that both this many TXGs and .Sy metaslab_unload_delay_ms milliseconds must pass before unloading will occur. . -.It Sy metaslab_unload_delay_ms Ns = Ns Sy 600000 Ns ms Po 10min Pc Pq int +.It Sy metaslab_unload_delay_ms Ns = Ns Sy 600000 Ns ms Po 10 min Pc Pq int After a metaslab is used, we keep it loaded for this many milliseconds, to attempt to reduce unnecessary reloading. Note, that both this many milliseconds and @@ -454,7 +454,14 @@ If we have less than this amount of free space, most ZPL operations (e.g. write, create) will return .Sy ENOSPC . . -.It Sy vdev_removal_max_span Ns = Ns Sy 32768 Ns B Po 32kB Pc Pq int +.It Sy spa_upgrade_errlog_limit Ns = Ns Sy 0 Pq uint +Limits the number of on-disk error log entries that will be converted to the +new format when enabling the +.Sy head_errlog +feature. +The default is to convert all log entries. +. +.It Sy vdev_removal_max_span Ns = Ns Sy 32768 Ns B Po 32 KiB Pc Pq int During top-level vdev removal, chunks of data are copied from the vdev which may include free space in order to trade bandwidth for IOPS. This parameter determines the maximum span of free space, in bytes, @@ -465,10 +472,10 @@ The default value here was chosen to align with which is a similar concept when doing regular reads (but there's no reason it has to be the same). . -.It Sy vdev_file_logical_ashift Ns = Ns Sy 9 Po 512B Pc Pq ulong +.It Sy vdev_file_logical_ashift Ns = Ns Sy 9 Po 512 B Pc Pq ulong Logical ashift for file-based devices. . -.It Sy vdev_file_physical_ashift Ns = Ns Sy 9 Po 512B Pc Pq ulong +.It Sy vdev_file_physical_ashift Ns = Ns Sy 9 Po 512 B Pc Pq ulong Physical ashift for file-based devices. . .It Sy zap_iterate_prefetch Ns = Ns Sy 1 Ns | Ns 0 Pq int @@ -477,20 +484,31 @@ prefetch the entire object (all leaf blocks). However, this is limited by .Sy dmu_prefetch_max . . -.It Sy zfetch_array_rd_sz Ns = Ns Sy 1048576 Ns B Po 1MB Pc Pq ulong +.It Sy zfetch_array_rd_sz Ns = Ns Sy 1048576 Ns B Po 1 MiB Pc Pq ulong If prefetching is enabled, disable prefetching for reads larger than this size. . -.It Sy zfetch_max_distance Ns = Ns Sy 8388608 Ns B Po 8MB Pc Pq uint +.It Sy zfetch_min_distance Ns = Ns Sy 4194304 Ns B Po 4 MiB Pc Pq uint +Min bytes to prefetch per stream. +Prefetch distance starts from the demand access size and quickly grows to +this value, doubling on each hit. +After that it may grow further by 1/8 per hit, but only if some prefetch +since last time haven't completed in time to satisfy demand request, i.e. +prefetch depth didn't cover the read latency or the pool got saturated. +. +.It Sy zfetch_max_distance Ns = Ns Sy 67108864 Ns B Po 64 MiB Pc Pq uint Max bytes to prefetch per stream. . -.It Sy zfetch_max_idistance Ns = Ns Sy 67108864 Ns B Po 64MB Pc Pq uint +.It Sy zfetch_max_idistance Ns = Ns Sy 67108864 Ns B Po 64 MiB Pc Pq uint Max bytes to prefetch indirects for per stream. . .It Sy zfetch_max_streams Ns = Ns Sy 8 Pq uint Max number of streams per zfetch (prefetch streams per file). . -.It Sy zfetch_min_sec_reap Ns = Ns Sy 2 Pq uint -Min time before an active prefetch stream can be reclaimed +.It Sy zfetch_min_sec_reap Ns = Ns Sy 1 Pq uint +Min time before inactive prefetch stream can be reclaimed +. +.It Sy zfetch_max_sec_reap Ns = Ns Sy 2 Pq uint +Max time before inactive prefetch stream can be deleted . .It Sy zfs_abd_scatter_enabled Ns = Ns Sy 1 Ns | Ns 0 Pq int Enables ARC from using scatter/gather lists and forces all allocations to be @@ -506,7 +524,7 @@ The value of .Sy MAX_ORDER depends on kernel configuration. . -.It Sy zfs_abd_scatter_min_size Ns = Ns Sy 1536 Ns B Po 1.5kB Pc Pq uint +.It Sy zfs_abd_scatter_min_size Ns = Ns Sy 1536 Ns B Po 1.5 KiB Pc Pq uint This is the minimum allocation size that will use scatter (page-based) ABDs. Smaller allocations will use linear ABDs. . @@ -538,10 +556,10 @@ Percentage of ARC dnodes to try to scan in response to demand for non-metadata when the number of bytes consumed by dnodes exceeds .Sy zfs_arc_dnode_limit . . -.It Sy zfs_arc_average_blocksize Ns = Ns Sy 8192 Ns B Po 8kB Pc Pq int +.It Sy zfs_arc_average_blocksize Ns = Ns Sy 8192 Ns B Po 8 KiB Pc Pq int The ARC's buffer hash table is sized based on the assumption of an average block size of this value. -This works out to roughly 1MB of hash table per 1GB of physical memory +This works out to roughly 1 MiB of hash table per 1 GiB of physical memory with 8-byte pointers. For configurations with a known larger average block size, this value can be increased to reduce the memory footprint. @@ -552,9 +570,9 @@ When .Fn arc_get_data_impl waits for this percent of the requested amount of data to be evicted. For example, by default, for every -.Em 2kB +.Em 2 KiB that's evicted, -.Em 1kB +.Em 1 KiB of it may be "reused" by a new allocation. Since this is above .Sy 100 Ns % , @@ -595,12 +613,12 @@ Under Linux, half of system memory will be used as the limit. Under .Fx , the larger of -.Sy all_system_memory No \- Sy 1GB +.Sy all_system_memory No \- Sy 1 GiB and .Sy 5/8 No \(mu Sy all_system_memory will be used as the limit. This value must be at least -.Sy 67108864 Ns B Pq 64MB . +.Sy 67108864 Ns B Pq 64 MiB . .Pp This value can be changed dynamically, with some caveats. It cannot be set back to @@ -668,7 +686,7 @@ to evict the required number of metadata buffers. Min size of ARC in bytes. .No If set to Sy 0 , arc_c_min will default to consuming the larger of -.Sy 32MB +.Sy 32 MiB and .Sy all_system_memory No / Sy 32 . . @@ -709,7 +727,7 @@ If equivalent to a quarter of the user-wired memory limit under .Fx and to -.Sy 134217728 Ns B Pq 128MB +.Sy 134217728 Ns B Pq 128 MiB under Linux. . .It Sy zfs_multilist_num_sublists Ns = Ns Sy 0 Pq int @@ -787,10 +805,10 @@ Note that in practice, the kernel's shrinker can ask us to evict up to about four times this for one allocation attempt. .Pp The default limit of -.Sy 10000 Pq in practice, Em 160MB No per allocation attempt with 4kB pages +.Sy 10000 Pq in practice, Em 160 MiB No per allocation attempt with 4 KiB pages limits the amount of time spent attempting to reclaim ARC memory to -less than 100ms per allocation attempt, -even with a small average compressed block size of ~8kB. +less than 100 ms per allocation attempt, +even with a small average compressed block size of ~8 KiB. .Pp The parameter can be set to 0 (zero) to disable the limit, and only applies on Linux. @@ -798,7 +816,7 @@ and only applies on Linux. .It Sy zfs_arc_sys_free Ns = Ns Sy 0 Ns B Pq ulong The target number of bytes the ARC should leave as free memory on the system. If zero, equivalent to the bigger of -.Sy 512kB No and Sy all_system_memory/64 . +.Sy 512 KiB No and Sy all_system_memory/64 . . .It Sy zfs_autoimport_disable Ns = Ns Sy 1 Ns | Ns 0 Pq int Disable pool import at module load by ignoring the cache file @@ -839,12 +857,12 @@ bytes of memory and if the obsolete space map object uses more than bytes on-disk. The condensing process is an attempt to save memory by removing obsolete mappings. . -.It Sy zfs_condense_max_obsolete_bytes Ns = Ns Sy 1073741824 Ns B Po 1GB Pc Pq ulong +.It Sy zfs_condense_max_obsolete_bytes Ns = Ns Sy 1073741824 Ns B Po 1 GiB Pc Pq ulong Only attempt to condense indirect vdev mappings if the on-disk size of the obsolete space map object is greater than this number of bytes .Pq see Sy zfs_condense_indirect_vdevs_enable . . -.It Sy zfs_condense_min_mapping_bytes Ns = Ns Sy 131072 Ns B Po 128kB Pc Pq ulong +.It Sy zfs_condense_min_mapping_bytes Ns = Ns Sy 131072 Ns B Po 128 KiB Pc Pq ulong Minimum size vdev mapping to attempt to condense .Pq see Sy zfs_condense_indirect_vdevs_enable . . @@ -860,7 +878,7 @@ to the file clears the log. This setting does not influence debug prints due to .Sy zfs_flags . . -.It Sy zfs_dbgmsg_maxsize Ns = Ns Sy 4194304 Ns B Po 4MB Pc Pq int +.It Sy zfs_dbgmsg_maxsize Ns = Ns Sy 4194304 Ns B Po 4 MiB Pc Pq int Maximum size of the internal ZFS debug log. . .It Sy zfs_dbuf_state_index Ns = Ns Sy 0 Pq int @@ -880,7 +898,7 @@ is set, then the deadman behavior is invoked as described by .Sy zfs_deadman_failmode . By default, the deadman is enabled and set to .Sy wait -which results in "hung" I/Os only being logged. +which results in "hung" I/O operations only being logged. The deadman is automatically disabled when a pool gets suspended. . .It Sy zfs_deadman_failmode Ns = Ns Sy wait Pq charp @@ -900,21 +918,21 @@ This can be used to facilitate automatic fail-over to a properly configured fail-over partner. .El . -.It Sy zfs_deadman_checktime_ms Ns = Ns Sy 60000 Ns ms Po 1min Pc Pq int +.It Sy zfs_deadman_checktime_ms Ns = Ns Sy 60000 Ns ms Po 1 min Pc Pq int Check time in milliseconds. This defines the frequency at which we check for hung I/O requests and potentially invoke the .Sy zfs_deadman_failmode behavior. . -.It Sy zfs_deadman_synctime_ms Ns = Ns Sy 600000 Ns ms Po 10min Pc Pq ulong +.It Sy zfs_deadman_synctime_ms Ns = Ns Sy 600000 Ns ms Po 10 min Pc Pq ulong Interval in milliseconds after which the deadman is triggered and also the interval after which a pool sync operation is considered to be "hung". Once this limit is exceeded the deadman will be invoked every .Sy zfs_deadman_checktime_ms milliseconds until the pool sync completes. . -.It Sy zfs_deadman_ziotime_ms Ns = Ns Sy 300000 Ns ms Po 5min Pc Pq ulong +.It Sy zfs_deadman_ziotime_ms Ns = Ns Sy 300000 Ns ms Po 5 min Pc Pq ulong Interval in milliseconds after which the deadman is triggered and an individual I/O operation is considered to be "hung". As long as the operation remains "hung", @@ -964,10 +982,10 @@ will result in objects waiting when there is not actually contention on the same object. . .It Sy zfs_slow_io_events_per_second Ns = Ns Sy 20 Ns /s Pq int -Rate limit delay and deadman zevents (which report slow I/Os) to this many per +Rate limit delay and deadman zevents (which report slow I/O operations) to this many per second. . -.It Sy zfs_unflushed_max_mem_amt Ns = Ns Sy 1073741824 Ns B Po 1GB Pc Pq ulong +.It Sy zfs_unflushed_max_mem_amt Ns = Ns Sy 1073741824 Ns B Po 1 GiB Pc Pq ulong Upper-bound limit for unflushed metadata changes to be held by the log spacemap in memory, in bytes. . @@ -975,22 +993,22 @@ log spacemap in memory, in bytes. Part of overall system memory that ZFS allows to be used for unflushed metadata changes by the log spacemap, in millionths. . -.It Sy zfs_unflushed_log_block_max Ns = Ns Sy 262144 Po 256k Pc Pq ulong +.It Sy zfs_unflushed_log_block_max Ns = Ns Sy 131072 Po 128k Pc Pq ulong Describes the maximum number of log spacemap blocks allowed for each pool. The default value means that the space in all the log spacemaps can add up to no more than -.Sy 262144 +.Sy 131072 blocks (which means -.Em 32GB +.Em 16 GiB of logical space before compression and ditto blocks, assuming that blocksize is -.Em 128kB ) . +.Em 128 KiB ) . .Pp This tunable is important because it involves a trade-off between import time after an unclean export and the frequency of flushing metaslabs. The higher this number is, the more log blocks we allow when the pool is active which means that we flush metaslabs less often and thus decrease -the number of I/Os for spacemap updates per TXG. +the number of I/O operations for spacemap updates per TXG. At the same time though, that means that in the event of an unclean export, there will be more log spacemap blocks for us to read, inducing overhead in the import time of the pool. @@ -1011,7 +1029,12 @@ Thus we always allow at least this many log blocks. .It Sy zfs_unflushed_log_block_pct Ns = Ns Sy 400 Ns % Pq ulong Tunable used to determine the number of blocks that can be used for the spacemap log, expressed as a percentage of the total number of -metaslabs in the pool. +unflushed metaslabs in the pool. +. +.It Sy zfs_unflushed_log_txg_max Ns = Ns Sy 1000 Pq ulong +Tunable limiting maximum time in TXGs any metaslab may remain unflushed. +It effectively limits maximum number of unflushed per-TXG spacemap logs +that need to be read after unclean pool export. . .It Sy zfs_unlink_suspend_progress Ns = Ns Sy 0 Ns | Ns 1 Pq uint When enabled, files will not be asynchronously removed from the list of pending @@ -1086,9 +1109,9 @@ This should be less than . .It Sy zfs_wrlog_data_max Ns = Pq int The upper limit of write-transaction zil log data size in bytes. -Once it is reached, write operation is blocked, until log data is cleared out -after transaction group sync. Because of some overhead, it should be set -at least 2 times the size of +Write operations are throttled when approaching the limit until log data is +cleared out after transaction group sync. +Because of some overhead, it should be set at least 2 times the size of .Sy zfs_dirty_data_max .No to prevent harming normal write throughput. It also should be smaller than the size of the slog device if slog is present. @@ -1382,7 +1405,7 @@ Similar to .Sy zfs_free_min_time_ms , but for cleanup of old indirection records for removed vdevs. . -.It Sy zfs_immediate_write_sz Ns = Ns Sy 32768 Ns B Po 32kB Pc Pq long +.It Sy zfs_immediate_write_sz Ns = Ns Sy 32768 Ns B Po 32 KiB Pc Pq long Largest data block to write to the ZIL. Larger blocks will be treated as if the dataset being written to had the .Sy logbias Ns = Ns Sy throughput @@ -1392,7 +1415,7 @@ property set. Pattern written to vdev free space by .Xr zpool-initialize 8 . . -.It Sy zfs_initialize_chunk_size Ns = Ns Sy 1048576 Ns B Po 1MB Pc Pq ulong +.It Sy zfs_initialize_chunk_size Ns = Ns Sy 1048576 Ns B Po 1 MiB Pc Pq ulong Size of writes used by .Xr zpool-initialize 8 . This option is used by the test suite. @@ -1440,7 +1463,7 @@ This option is used by the test suite to trigger race conditions. The maximum execution time limit that can be set for a ZFS channel program, specified as a number of Lua instructions. . -.It Sy zfs_lua_max_memlimit Ns = Ns Sy 104857600 Po 100MB Pc Pq ulong +.It Sy zfs_lua_max_memlimit Ns = Ns Sy 104857600 Po 100 MiB Pc Pq ulong The maximum memory limit that can be set for a ZFS channel program, specified in bytes. . @@ -1456,15 +1479,15 @@ feature uses to estimate incoming log blocks. .It Sy zfs_max_logsm_summary_length Ns = Ns Sy 10 Pq ulong Maximum number of rows allowed in the summary of the spacemap log. . -.It Sy zfs_max_recordsize Ns = Ns Sy 1048576 Po 1MB Pc Pq int +.It Sy zfs_max_recordsize Ns = Ns Sy 16777216 Po 16 MiB Pc Pq int We currently support block sizes from -.Em 512B No to Em 16MB . +.Em 512 Po 512 B Pc No to Em 16777216 Po 16 MiB Pc . The benefits of larger blocks, and thus larger I/O, need to be weighed against the cost of COWing a giant block to modify one byte. Additionally, very large blocks can have an impact on I/O latency, and also potentially on the memory allocator. -Therefore, we do not allow the recordsize to be set larger than this tunable. -Larger blocks can be created by changing it, +Therefore, we formerly forbade creating blocks larger than 1M. +Larger blocks could be created by changing it, and pools with larger blocks can always be imported and used, regardless of this setting. . @@ -1522,7 +1545,7 @@ into the special allocation class. Historical statistics for this many latest multihost updates will be available in .Pa /proc/spl/kstat/zfs/ Ns Ao Ar pool Ac Ns Pa /multihost . . -.It Sy zfs_multihost_interval Ns = Ns Sy 1000 Ns ms Po 1s Pc Pq ulong +.It Sy zfs_multihost_interval Ns = Ns Sy 1000 Ns ms Po 1 s Pc Pq ulong Used to control the frequency of multihost writes which are performed when the .Sy multihost pool property is on. @@ -1555,7 +1578,7 @@ delay found in the best uberblock indicates actual multihost updates happened at longer intervals than .Sy zfs_multihost_interval . A minimum of -.Em 100ms +.Em 100 ms is enforced. .Pp .Sy 0 No is equivalent to Sy 1 . @@ -1604,7 +1627,7 @@ When enabled forces ZFS to sync data when flags are used allowing holes in a file to be accurately reported. When disabled holes will not be reported in recently dirtied files. . -.It Sy zfs_pd_bytes_max Ns = Ns Sy 52428800 Ns B Po 50MB Pc Pq int +.It Sy zfs_pd_bytes_max Ns = Ns Sy 52428800 Ns B Po 50 MiB Pc Pq int The number of bytes which should be prefetched during a pool traversal, like .Nm zfs Cm send or other data crawling operations. @@ -1622,8 +1645,8 @@ After this threshold is crossed, additional frees will wait until the next TXG. . .It Sy zfs_prefetch_disable Ns = Ns Sy 0 Ns | Ns 1 Pq int Disable predictive prefetch. -Note that it leaves "prescient" prefetch (for. e.g.\& -.Nm zfs Cm send ) +Note that it leaves "prescient" prefetch +.Pq for, e.g., Nm zfs Cm send intact. Unlike predictive prefetch, prescient prefetch never issues I/O that ends up not being needed, so it can't hurt performance. @@ -1643,7 +1666,7 @@ Disable QAT hardware acceleration for AES-GCM encryption. May be unset after the ZFS modules have been loaded to initialize the QAT hardware as long as support is compiled in and the QAT driver is present. . -.It Sy zfs_vnops_read_chunk_size Ns = Ns Sy 1048576 Ns B Po 1MB Pc Pq long +.It Sy zfs_vnops_read_chunk_size Ns = Ns Sy 1048576 Ns B Po 1 MiB Pc Pq long Bytes to read per chunk. . .It Sy zfs_read_history Ns = Ns Sy 0 Pq int @@ -1653,7 +1676,7 @@ Historical statistics for this many latest reads will be available in .It Sy zfs_read_history_hits Ns = Ns Sy 0 Ns | Ns 1 Pq int Include cache hits in read history . -.It Sy zfs_rebuild_max_segment Ns = Ns Sy 1048576 Ns B Po 1MB Pc Pq ulong +.It Sy zfs_rebuild_max_segment Ns = Ns Sy 1048576 Ns B Po 1 MiB Pc Pq ulong Maximum read segment size to issue when sequentially resilvering a top-level vdev. . @@ -1663,7 +1686,7 @@ completes in order to verify the checksums of all blocks which have been resilvered. This is enabled by default and strongly recommended. . -.It Sy zfs_rebuild_vdev_limit Ns = Ns Sy 33554432 Ns B Po 32MB Pc Pq ulong +.It Sy zfs_rebuild_vdev_limit Ns = Ns Sy 33554432 Ns B Po 32 MiB Pc Pq ulong Maximum amount of I/O that can be concurrently issued for a sequential resilver per leaf device, given in bytes. . @@ -1683,8 +1706,8 @@ This should only be used as a last resort, as it typically results in leaked space, or worse. . .It Sy zfs_removal_ignore_errors Ns = Ns Sy 0 Ns | Ns 1 Pq int -Ignore hard IO errors during device removal. -When set, if a device encounters a hard IO error during the removal process +Ignore hard I/O errors during device removal. +When set, if a device encounters a hard I/O error during the removal process the removal will not be cancelled. This can result in a normally recoverable block becoming permanently damaged and is hence not recommended. @@ -1695,7 +1718,7 @@ pool cannot be returned to a healthy state prior to removing the device. This is used by the test suite so that it can ensure that certain actions happen while in the middle of a removal. . -.It Sy zfs_remove_max_segment Ns = Ns Sy 16777216 Ns B Po 16MB Pc Pq int +.It Sy zfs_remove_max_segment Ns = Ns Sy 16777216 Ns B Po 16 MiB Pc Pq int The largest contiguous segment that we will attempt to allocate when removing a device. If there is a performance problem with attempting to allocate large blocks, @@ -1708,7 +1731,7 @@ Ignore the feature, causing an operation that would start a resilver to immediately restart the one in progress. . -.It Sy zfs_resilver_min_time_ms Ns = Ns Sy 3000 Ns ms Po 3s Pc Pq int +.It Sy zfs_resilver_min_time_ms Ns = Ns Sy 3000 Ns ms Po 3 s Pc Pq int Resilvers are processed by the sync thread. While resilvering, it will spend at least this much time working on a resilver between TXG flushes. @@ -1719,12 +1742,12 @@ even if there were unrepairable errors. Intended to be used during pool repair or recovery to stop resilvering when the pool is next imported. . -.It Sy zfs_scrub_min_time_ms Ns = Ns Sy 1000 Ns ms Po 1s Pc Pq int +.It Sy zfs_scrub_min_time_ms Ns = Ns Sy 1000 Ns ms Po 1 s Pc Pq int Scrubs are processed by the sync thread. While scrubbing, it will spend at least this much time working on a scrub between TXG flushes. . -.It Sy zfs_scan_checkpoint_intval Ns = Ns Sy 7200 Ns s Po 2h Pc Pq int +.It Sy zfs_scan_checkpoint_intval Ns = Ns Sy 7200 Ns s Po 2 hour Pc Pq int To preserve progress across reboots, the sequential scan algorithm periodically needs to stop metadata scanning and issue all the verification I/O to disk. The frequency of this flushing is determined by this tunable. @@ -1761,7 +1784,7 @@ Otherwise indicates that the legacy algorithm will be used, where I/O is initiated as soon as it is discovered. Unsetting will not affect scrubs or resilvers that are already in progress. . -.It Sy zfs_scan_max_ext_gap Ns = Ns Sy 2097152 Ns B Po 2MB Pc Pq int +.It Sy zfs_scan_max_ext_gap Ns = Ns Sy 2097152 Ns B Po 2 MiB Pc Pq int Sets the largest gap in bytes between scrub/resilver I/O operations that will still be considered sequential for sorting purposes. Changing this value will not @@ -1790,7 +1813,7 @@ When disabled, the memory limit may be exceeded by fast disks. Freezes a scrub/resilver in progress without actually pausing it. Intended for testing/debugging. . -.It Sy zfs_scan_vdev_limit Ns = Ns Sy 4194304 Ns B Po 4MB Pc Pq int +.It Sy zfs_scan_vdev_limit Ns = Ns Sy 4194304 Ns B Po 4 MiB Pc Pq int Maximum amount of data that can be concurrently issued at once for scrubs and resilvers per leaf device, given in bytes. . @@ -1810,7 +1833,7 @@ The fill fraction of the internal queues. The fill fraction controls the timing with which internal threads are woken up. . -.It Sy zfs_send_no_prefetch_queue_length Ns = Ns Sy 1048576 Ns B Po 1MB Pc Pq int +.It Sy zfs_send_no_prefetch_queue_length Ns = Ns Sy 1048576 Ns B Po 1 MiB Pc Pq int The maximum number of bytes allowed in .Nm zfs Cm send Ns 's internal queues. @@ -1821,7 +1844,7 @@ The fill fraction of the prefetch queue. The fill fraction controls the timing with which internal threads are woken up. . -.It Sy zfs_send_queue_length Ns = Ns Sy 16777216 Ns B Po 16MB Pc Pq int +.It Sy zfs_send_queue_length Ns = Ns Sy 16777216 Ns B Po 16 MiB Pc Pq int The maximum number of bytes allowed that will be prefetched by .Nm zfs Cm send . This value must be at least twice the maximum block size in use. @@ -1832,20 +1855,31 @@ The fill fraction of the queue. The fill fraction controls the timing with which internal threads are woken up. . -.It Sy zfs_recv_queue_length Ns = Ns Sy 16777216 Ns B Po 16MB Pc Pq int +.It Sy zfs_recv_queue_length Ns = Ns Sy 16777216 Ns B Po 16 MiB Pc Pq int The maximum number of bytes allowed in the .Nm zfs Cm receive queue. This value must be at least twice the maximum block size in use. . -.It Sy zfs_recv_write_batch_size Ns = Ns Sy 1048576 Ns B Po 1MB Pc Pq int +.It Sy zfs_recv_write_batch_size Ns = Ns Sy 1048576 Ns B Po 1 MiB Pc Pq int The maximum amount of data, in bytes, that .Nm zfs Cm receive will write in one DMU transaction. This is the uncompressed size, even when receiving a compressed send stream. This setting will not reduce the write size below a single block. Capped at a maximum of -.Sy 32MB . +.Sy 32 MiB . +. +.It Sy zfs_recv_best_effort_corrective Ns = Ns Sy 0 Pq int +When this variable is set to non-zero a corrective receive: +.Bl -enum -compact -offset 4n -width "1." +.It +Does not enforce the restriction of source & destination snapshot GUIDs +matching. +.It +If there is an error during healing, the healing receive is not +terminated instead it moves on to the next record. +.El . .It Sy zfs_override_estimate_recordsize Ns = Ns Sy 0 Ns | Ns 1 Pq ulong Setting this variable overrides the default logic for estimating block @@ -1860,7 +1894,7 @@ and you require accurate zfs send size estimates. Flushing of data to disk is done in passes. Defer frees starting in this pass. . -.It Sy zfs_spa_discard_memory_limit Ns = Ns Sy 16777216 Ns B Po 16MB Pc Pq int +.It Sy zfs_spa_discard_memory_limit Ns = Ns Sy 16777216 Ns B Po 16 MiB Pc Pq int Maximum memory used for prefetching a checkpoint's space map on each vdev while discarding the checkpoint. . @@ -1882,11 +1916,11 @@ the average number of sync passes; because when we turn compression off, many blocks' size will change, and thus we have to re-allocate (not overwrite) them. It also increases the number of -.Em 128kB +.Em 128 KiB allocations (e.g. for indirect blocks and spacemaps) because these will not be compressed. The -.Em 128kB +.Em 128 KiB allocations are especially detrimental to performance on highly fragmented systems, which may have very few free segments of this size, and may need to load new metaslabs to satisfy these allocations. @@ -1901,11 +1935,11 @@ The default value of .Sy 75% will create a maximum of one thread per CPU. . -.It Sy zfs_trim_extent_bytes_max Ns = Ns Sy 134217728 Ns B Po 128MB Pc Pq uint +.It Sy zfs_trim_extent_bytes_max Ns = Ns Sy 134217728 Ns B Po 128 MiB Pc Pq uint Maximum size of TRIM command. Larger ranges will be split into chunks no larger than this value before issuing. . -.It Sy zfs_trim_extent_bytes_min Ns = Ns Sy 32768 Ns B Po 32kB Pc Pq uint +.It Sy zfs_trim_extent_bytes_min Ns = Ns Sy 32768 Ns B Po 32 KiB Pc Pq uint Minimum size of TRIM commands. TRIM ranges smaller than this will be skipped, unless they're part of a larger range which was chunked. @@ -1948,25 +1982,25 @@ Historical statistics for this many latest TXGs will be available in Flush dirty data to disk at least every this many seconds (maximum TXG duration). . .It Sy zfs_vdev_aggregate_trim Ns = Ns Sy 0 Ns | Ns 1 Pq int -Allow TRIM I/Os to be aggregated. +Allow TRIM I/O operations to be aggregated. This is normally not helpful because the extents to be trimmed will have been already been aggregated by the metaslab. This option is provided for debugging and performance analysis. . -.It Sy zfs_vdev_aggregation_limit Ns = Ns Sy 1048576 Ns B Po 1MB Pc Pq int +.It Sy zfs_vdev_aggregation_limit Ns = Ns Sy 1048576 Ns B Po 1 MiB Pc Pq int Max vdev I/O aggregation size. . -.It Sy zfs_vdev_aggregation_limit_non_rotating Ns = Ns Sy 131072 Ns B Po 128kB Pc Pq int +.It Sy zfs_vdev_aggregation_limit_non_rotating Ns = Ns Sy 131072 Ns B Po 128 KiB Pc Pq int Max vdev I/O aggregation size for non-rotating media. . -.It Sy zfs_vdev_cache_bshift Ns = Ns Sy 16 Po 64kB Pc Pq int +.It Sy zfs_vdev_cache_bshift Ns = Ns Sy 16 Po 64 KiB Pc Pq int Shift size to inflate reads to. . -.It Sy zfs_vdev_cache_max Ns = Ns Sy 16384 Ns B Po 16kB Pc Pq int +.It Sy zfs_vdev_cache_max Ns = Ns Sy 16384 Ns B Po 16 KiB Pc Pq int Inflate reads smaller than this value to meet the .Sy zfs_vdev_cache_bshift size -.Pq default Sy 64kB . +.Pq default Sy 64 KiB . . .It Sy zfs_vdev_cache_size Ns = Ns Sy 0 Pq int Total size of the per-disk cache in bytes. @@ -1988,7 +2022,7 @@ lacks locality as defined by Operations within this that are not immediately following the previous operation are incremented by half. . -.It Sy zfs_vdev_mirror_rotating_seek_offset Ns = Ns Sy 1048576 Ns B Po 1MB Pc Pq int +.It Sy zfs_vdev_mirror_rotating_seek_offset Ns = Ns Sy 1048576 Ns B Po 1 MiB Pc Pq int The maximum distance for the last queued I/O operation in which the balancing algorithm considers an operation to have locality. .No See Sx ZFS I/O SCHEDULER . @@ -2006,11 +2040,11 @@ locality as defined by the Operations within this that are not immediately following the previous operation are incremented by half. . -.It Sy zfs_vdev_read_gap_limit Ns = Ns Sy 32768 Ns B Po 32kB Pc Pq int +.It Sy zfs_vdev_read_gap_limit Ns = Ns Sy 32768 Ns B Po 32 KiB Pc Pq int Aggregate read I/O operations if the on-disk gap between them is within this threshold. . -.It Sy zfs_vdev_write_gap_limit Ns = Ns Sy 4096 Ns B Po 4kB Pc Pq int +.It Sy zfs_vdev_write_gap_limit Ns = Ns Sy 4096 Ns B Po 4 KiB Pc Pq int Aggregate write I/O operations if the on-disk gap between them is within this threshold. . @@ -2058,7 +2092,7 @@ Setting this to .Sy 0 disables duplicate detection. . -.It Sy zfs_zevent_retain_expire_secs Ns = Ns Sy 900 Ns s Po 15min Pc Pq int +.It Sy zfs_zevent_retain_expire_secs Ns = Ns Sy 900 Ns s Po 15 min Pc Pq int Lifespan for a recent ereport that was retained for duplicate checking. . .It Sy zfs_zil_clean_taskq_maxalloc Ns = Ns Sy 1048576 Pq int @@ -2077,10 +2111,10 @@ The default value of .Sy 100% will create a maximum of one thread per cpu. . -.It Sy zil_maxblocksize Ns = Ns Sy 131072 Ns B Po 128kB Pc Pq int +.It Sy zil_maxblocksize Ns = Ns Sy 131072 Ns B Po 128 KiB Pc Pq int This sets the maximum block size used by the ZIL. On very fragmented pools, lowering this -.Pq typically to Sy 36kB +.Pq typically to Sy 36 KiB can improve performance. . .It Sy zil_nocacheflush Ns = Ns Sy 0 Ns | Ns 1 Pq int @@ -2093,11 +2127,21 @@ if a volatile out-of-order write cache is enabled. Disable intent logging replay. Can be disabled for recovery from corrupted ZIL. . -.It Sy zil_slog_bulk Ns = Ns Sy 786432 Ns B Po 768kB Pc Pq ulong +.It Sy zil_slog_bulk Ns = Ns Sy 786432 Ns B Po 768 KiB Pc Pq ulong Limit SLOG write size per commit executed with synchronous priority. Any writes above that will be executed with lower (asynchronous) priority to limit potential SLOG device abuse by single active ZIL writer. . +.It Sy zfs_zil_saxattr Ns = Ns Sy 1 Ns | Ns 0 Pq int +Setting this tunable to zero disables ZIL logging of new +.Sy xattr Ns = Ns Sy sa +records if the +.Sy org.openzfs:zilsaxattr +feature is enabled on the pool. +This would only be necessary to work around bugs in the ZIL logging or replay +code for this record type. +The tunable has no effect if the feature is disabled. +. .It Sy zfs_embedded_slog_min_ms Ns = Ns Sy 64 Pq int Usually, one metaslab from each normal-class vdev is dedicated for use by the ZIL to log synchronous writes. @@ -2106,6 +2150,14 @@ However, if there are fewer than metaslabs in the vdev, this functionality is disabled. This ensures that we don't set aside an unreasonable amount of space for the ZIL. . +.It Sy zstd_earlyabort_pass Ns = Ns Sy 1 Pq int +Whether heuristic for detection of incompressible data with zstd levels >= 3 +using LZ4 and zstd-1 passes is enabled. +. +.It Sy zstd_abort_size Ns = Ns Sy 131072 Pq int +Minimal uncompressed size (inclusive) of a record before the early abort +heuristic will be attempted. +. .It Sy zio_deadman_log_all Ns = Ns Sy 0 Ns | Ns 1 Pq int If non-zero, the zio deadman will produce debugging messages .Pq see Sy zfs_dbgmsg_enable @@ -2115,7 +2167,7 @@ diagnostic information for hang conditions which don't involve a mutex or other locking primitive: typically conditions in which a thread in the zio pipeline is looping indefinitely. . -.It Sy zio_slow_io_ms Ns = Ns Sy 30000 Ns ms Po 30s Pc Pq int +.It Sy zio_slow_io_ms Ns = Ns Sy 30000 Ns ms Po 30 s Pc Pq int When an I/O operation takes more than this much time to complete, it's marked as slow. Each slow operation causes a delay zevent. @@ -2129,6 +2181,30 @@ When enabled, the maximum number of pending allocations per top-level vdev is limited by .Sy zfs_vdev_queue_depth_pct . . +.It Sy zfs_xattr_compat Ns = Ns 0 Ns | Ns 1 Pq int +Control the naming scheme used when setting new xattrs in the user namespace. +If +.Sy 0 +.Pq the default on Linux , +user namespace xattr names are prefixed with the namespace, to be backwards +compatible with previous versions of ZFS on Linux. +If +.Sy 1 +.Pq the default on Fx , +user namespace xattr names are not prefixed, to be backwards compatible with +previous versions of ZFS on illumos and +.Fx . +.Pp +Either naming scheme can be read on this and future versions of ZFS, regardless +of this tunable, but legacy ZFS on illumos or +.Fx +are unable to read user namespace xattrs written in the Linux format, and +legacy versions of ZFS on Linux are unable to read user namespace xattrs written +in the legacy ZFS format. +.Pp +An existing xattr with the alternate naming scheme is removed when overwriting +the xattr so as to not accumulate duplicates. +. .It Sy zio_requeue_io_start_cut_in_line Ns = Ns Sy 0 Ns | Ns 1 Pq int Prioritize requeued I/O. . @@ -2167,7 +2243,7 @@ many blocks, where block size is determined by the .Sy volblocksize property of a zvol. . -.It Sy zvol_prefetch_bytes Ns = Ns Sy 131072 Ns B Po 128kB Pc Pq uint +.It Sy zvol_prefetch_bytes Ns = Ns Sy 131072 Ns B Po 128 KiB Pc Pq uint When adding a zvol to the system, prefetch this many bytes from the start and end of the volume. Prefetching these regions of the volume is desirable, @@ -2183,9 +2259,74 @@ for each I/O submitter. When unset, requests are handled asynchronously by a thread pool. The number of requests which can be handled concurrently is controlled by .Sy zvol_threads . +.Sy zvol_request_sync +is ignored when running on a kernel that supports block multiqueue +.Pq Li blk-mq . +. +.It Sy zvol_threads Ns = Ns Sy 0 Pq uint +The number of system wide threads to use for processing zvol block IOs. +If +.Sy 0 +(the default) then internally set +.Sy zvol_threads +to the number of CPUs present or 32 (whichever is greater). +. +.It Sy zvol_blk_mq_threads Ns = Ns Sy 0 Pq uint +The number of threads per zvol to use for queuing IO requests. +This parameter will only appear if your kernel supports +.Li blk-mq +and is only read and assigned to a zvol at zvol load time. +If +.Sy 0 +(the default) then internally set +.Sy zvol_blk_mq_threads +to the number of CPUs present. . -.It Sy zvol_threads Ns = Ns Sy 32 Pq uint -Max number of threads which can handle zvol I/O requests concurrently. +.It Sy zvol_use_blk_mq Ns = Ns Sy 0 Ns | Ns 1 Pq uint +Set to +.Sy 1 +to use the +.Li blk-mq +API for zvols. +Set to +.Sy 0 +(the default) to use the legacy zvol APIs. +This setting can give better or worse zvol performance depending on +the workload. +This parameter will only appear if your kernel supports +.Li blk-mq +and is only read and assigned to a zvol at zvol load time. +. +.It Sy zvol_blk_mq_blocks_per_thread Ns = Ns Sy 8 Pq uint +If +.Sy zvol_use_blk_mq +is enabled, then process this number of +.Sy volblocksize Ns -sized blocks per zvol thread. +This tunable can be use to favor better performance for zvol reads (lower +values) or writes (higher values). +If set to +.Sy 0 , +then the zvol layer will process the maximum number of blocks +per thread that it can. +This parameter will only appear if your kernel supports +.Li blk-mq +and is only applied at each zvol's load time. +. +.It Sy zvol_blk_mq_queue_depth Ns = Ns Sy 0 Pq uint +The queue_depth value for the zvol +.Li blk-mq +interface. +This parameter will only appear if your kernel supports +.Li blk-mq +and is only applied at each zvol's load time. +If +.Sy 0 +(the default) then use the kernel's default queue depth. +Values are clamped to the kernel's +.Dv BLKDEV_MIN_RQ +and +.Dv BLKDEV_MAX_RQ Ns / Ns Dv BLKDEV_DEFAULT_RQ +limits. . .It Sy zvol_volmode Ns = Ns Sy 1 Pq uint Defines zvol block devices behaviour when @@ -2359,7 +2500,7 @@ delay Note, that since the delay is added to the outstanding time remaining on the most recent transaction it's effectively the inverse of IOPS. Here, the midpoint of -.Em 500us +.Em 500 us translates to .Em 2000 IOPS . The shape of the curve diff --git a/man/man7/dracut.zfs.7 b/man/man7/dracut.zfs.7 new file mode 100644 index 000000000000..0f446fe2fe3f --- /dev/null +++ b/man/man7/dracut.zfs.7 @@ -0,0 +1,278 @@ +.\" SPDX-License-Identifier: 0BSD +.\" +.Dd April 4, 2022 +.Dt DRACUT.ZFS 7 +.Os +. +.Sh NAME +.Nm dracut.zfs +.Nd overview of ZFS dracut hooks +. +.Sh SYNOPSIS +.Bd -literal -compact + parse-zfs.sh \(-> dracut-cmdline.service + | \(da + | … + | \(da + \e\(em\(em\(em\(em\(em\(em\(em\(em\(-> dracut-initqueue.service + | zfs-import-opts.sh + zfs-load-module.service \(da | | + | | sysinit.target \(da | + \(da | | zfs-import-scan.service \(da +zfs-import-scan.service \(da \(da | zfs-import-cache.service + | zfs-import-cache.service basic.target | | + \e__________________| | \(da \(da + \(da | zfs-load-key.sh + zfs-env-bootfs.service | | + \(da \(da \(da + zfs-import.target \(-> dracut-pre-mount.service + | \(ua | + | dracut-zfs-generator | + | ____________________/| + |/ \(da + | sysroot.mount \(<-\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em dracut-zfs-generator + | | \(da | + | \(da sysroot-{usr,etc,lib,&c.}.mount | + | initrd-root-fs.target \(<-\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em or \(da + | | zfs-nonroot-necessities.service + | \(da | + \(da dracut-mount.service | + zfs-snapshot-bootfs.service | | + | \(da | + \(da … | + zfs-rollback-bootfs.service | | + | \(da | + | sysroot-usr.mount \(<-\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em\(em/ + | | + | \(da + | initrd-fs.target + \e______________________ | + \e| + \(da + export-zfs.sh initrd.target + | | + \(da \(da + dracut-shutdown.service … + | + \(da + zfs-needshutdown.sh \(-> initrd-cleanup.service +.Ed +.Pp +Compare +.Xr dracut.bootup 7 +for the full flowchart. +. +.Sh DESCRIPTION +Under dracut, booting with +.No ZFS-on- Ns Pa / +is facilitated by a number of hooks in the +.Nm 90zfs +module. +.Pp +Booting into a ZFS dataset requires +.Sy mountpoint Ns = Ns Pa / +to be set on the dataset containing the root filesystem (henceforth "the boot dataset") and at the very least either the +.Sy bootfs +property to be set to that dataset, or the +.Sy root= +kernel cmdline (or dracut drop-in) argument to specify it. +.Pp +All children of the boot dataset with +.Sy canmount Ns = Ns Sy on +with +.Sy mountpoint Ns s +matching +.Pa /etc , /bin , /lib , /lib?? , /libx32 , No and Pa /usr +globs are deemed essential and will be mounted as well. +.Pp +.Xr zfs-mount-generator 8 +is recommended for proper functioning of the system afterward (correct mount properties, remounting, &c.). +. +.Sh CMDLINE +.Ss Standard +.Bl -tag -compact -width ".Sy root=zfs:AUTO , root=zfs: , root=zfs , Op Sy root=" +.It Sy root=zfs:\& Ns Ar dataset , Sy root=ZFS= Ns Ar dataset +Use +.Ar dataset +as the boot dataset. +All pluses +.Pq Sq + +are replaced with spaces +.Pq Sq \ . +. +.It Sy root=zfs:AUTO , root=zfs:\& , root=zfs , Op Sy root= +After import, search for the first pool with the +.Sy bootfs +property set, use its value as-if specified as the +.Ar dataset +above. +. +.It Sy rootfstype=zfs root= Ns Ar dataset +Equivalent to +.Sy root=zfs:\& Ns Ar dataset . +. +.It Sy rootfstype=zfs Op Sy root= +Equivalent to +.Sy root=zfs:AUTO . +. +.It Sy rootflags= Ns Ar flags +Mount the boot dataset with +.Fl o Ar flags ; +cf.\& +.Sx Temporary Mount Point Properties +in +.Xr zfsprops 7 . +These properties will not last, since all filesystems will be re-mounted from the real root. +. +.It Sy debug +If specified, +.Nm dracut-zfs-generator +logs to the journal. +.El +.Pp +Be careful about setting neither +.Sy rootfstype=zfs +nor +.Sy root=zfs:\& Ns Ar dataset +\(em other automatic boot selection methods, like +.Nm systemd-gpt-auto-generator +and +.Nm systemd-fstab-generator +might take precedent. +. +.Ss ZFS-specific +.Bl -tag -compact -width ".Sy bootfs.snapshot Ns Op Sy = Ns Ar snapshot-name" +.It Sy bootfs.snapshot Ns Op Sy = Ns Ar snapshot-name +Execute +.Nm zfs Cm snapshot Ar boot-dataset Ns Sy @ Ns Ar snapshot-name +before pivoting to the real root. +.Ar snapshot-name +defaults to the current kernel release. +. +.It Sy bootfs.rollback Ns Op Sy = Ns Ar snapshot-name +Execute +.Nm zfs Cm snapshot Fl Rf Ar boot-dataset Ns Sy @ Ns Ar snapshot-name +before pivoting to the real root. +.Ar snapshot-name +defaults to the current kernel release. +. +.It Sy spl_hostid= Ns Ar host-id +Use +.Xr zgenhostid 8 +to set the host ID to +.Ar host-id ; +otherwise, +.Pa /etc/hostid +inherited from the real root is used. +. +.It Sy zfs_force , zfs.force , zfsforce +Appends +.Fl f +to all +.Nm zpool Cm import +invocations; primarily useful in conjunction with +.Sy spl_hostid= , +or if no host ID was inherited. +.El +. +.Sh FILES +.Bl -tag -width 0 +.It Pa parse-zfs.sh Pq Sy cmdline +Processes +.Sy spl_hostid= . +If +.Sy root= +matches a known pattern, above, provides +.Pa /dev/root +and delays the initqueue until +.Xr zfs 4 +is loaded, +. +.It Pa zfs-import-opts.sh Pq Nm systemd No environment generator +Turns +.Sy zfs_force , zfs.force , No or Sy zfsforce +into +.Ev ZPOOL_IMPORT_OPTS Ns = Ns Fl f +for +.Pa zfs-import-scan.service +or +.Pa zfs-import-cache.service . +. +.It Pa zfs-load-key.sh Pq Sy pre-mount +Loads encryption keys for the boot dataset and its essential descendants. +.Bl -tag -compact -offset 4n -width ".Sy keylocation Ns = Ns Sy https:// Ns Ar URL , Sy keylocation Ns = Ns Sy http:// Ns Ar URL" +.It Sy keylocation Ns = Ns Sy prompt +Is prompted for via +.Nm systemd-ask-password +thrice. +. +.It Sy keylocation Ns = Ns Sy https:// Ns Ar URL , Sy keylocation Ns = Ns Sy http:// Ns Ar URL +.Pa network-online.target +is started before loading. +. +.It Sy keylocation Ns = Ns Sy file:// Ns Ar path +If +.Ar path +doesn't exist, +.Nm udevadm No is Cm settle Ns d . +If it still doesn't, it's waited for for up to +.Sy 10 Ns s . +.El +. +.It Pa zfs-env-bootfs.service Pq Nm systemd No service +After pool import, sets +.Ev BOOTFS Ns = +in the systemd environment to the first non-null +.Sy bootfs +value in iteration order. +. +.It Pa dracut-zfs-generator Pq Nm systemd No generator +Generates +.Pa sysroot.mount Pq using Sy rootflags= , No if any . +If an explicit boot dataset was specified, also generates essential mountpoints +.Pq Pa sysroot-etc.mount , sysroot-bin.mount , No &c.\& , +otherwise generates +.Pa zfs-nonroot-necessities.service +which mounts them explicitly after +.Pa /sysroot +using +.Ev BOOTFS Ns = . +. +.It Pa zfs-snapshot-bootfs.service , zfs-rollback-bootfs.service Pq Nm systemd No services +Consume +.Sy bootfs.snapshot +and +.Sy bootfs.rollback +as described in +.Sx CMDLINE . +Use +.Ev BOOTFS Ns = +if no explicit boot dataset was specified. +. +.It Pa zfs-needshutdown.sh Pq Sy cleanup +If any pools were imported, signals that shutdown hooks are required. +. +.It Pa export-zfs.sh Pq Sy shutdown +Forcibly exports all pools. +. +.It Pa /etc/hostid , /etc/zfs/zpool.cache , /etc/zfs/vdev_id.conf Pq regular files +Included verbatim, hostonly. +. +.It Pa mount-zfs.sh Pq Sy mount +Does nothing on +.Nm systemd +systems +.Pq if Pa dracut-zfs-generator No succeeded . +Otherwise, loads encryption key for the boot dataset from the console or via plymouth. +It may not work at all! +.El +. +.Sh SEE ALSO +.Xr dracut.bootup 7 , +.Xr zfsprops 7 , +.Xr zpoolprops 7 , +.Xr dracut-shutdown.service 8 , +.Xr systemd-fstab-generator 8 , +.Xr systemd-gpt-auto-generator 8 , +.Xr zfs-mount-generator 8 , +.Xr zgenhostid 8 diff --git a/man/man7/vdevprops.7 b/man/man7/vdevprops.7 index ab6c77064194..b98bda064c70 100644 --- a/man/man7/vdevprops.7 +++ b/man/man7/vdevprops.7 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man7/zfsconcepts.7 b/man/man7/zfsconcepts.7 index f958035f72df..31c8b53e7fab 100644 --- a/man/man7/zfsconcepts.7 +++ b/man/man7/zfsconcepts.7 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -193,10 +193,10 @@ Calculating the exact requirement depends heavily on the type of data stored in the pool. .Pp Enabling deduplication on an improperly-designed system can result in -performance issues (slow IO and administrative operations). +performance issues (slow I/O and administrative operations). It can potentially lead to problems importing a pool due to memory exhaustion. Deduplication can consume significant processing power (CPU) and memory as well -as generate additional disk IO. +as generate additional disk I/O. .Pp Before creating a pool with deduplication enabled, ensure that you have planned your hardware requirements appropriately and implemented appropriate recovery diff --git a/man/man7/zfsprops.7 b/man/man7/zfsprops.7 index c6aa8daa964f..5a6b9594d9a7 100644 --- a/man/man7/zfsprops.7 +++ b/man/man7/zfsprops.7 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -388,7 +388,7 @@ privilege with can access everyone's usage. .Pp The -.Sy userused Ns @ Ns Ar ... +.Sy userused Ns @ Ns Ar … properties are not displayed by .Nm zfs Cm get Sy all . The user's name must be appended after the @@ -743,7 +743,7 @@ This property is not inherited. .It Xo .Sy checksum Ns = Ns Sy on Ns | Ns Sy off Ns | Ns Sy fletcher2 Ns | Ns .Sy fletcher4 Ns | Ns Sy sha256 Ns | Ns Sy noparity Ns | Ns -.Sy sha512 Ns | Ns Sy skein Ns | Ns Sy edonr +.Sy sha512 Ns | Ns Sy skein Ns | Ns Sy edonr Ns | Ns Sy blake3 .Xc Controls the checksum used to verify data integrity. The default value is @@ -768,8 +768,9 @@ a recommended practice. The .Sy sha512 , .Sy skein , +.Sy edonr , and -.Sy edonr +.Sy blake3 checksum algorithms require enabling the appropriate features on the pool. .Pp Please see @@ -784,9 +785,9 @@ Changing this property affects only newly-written data. .Xc Controls the compression algorithm used for this dataset. .Pp -Setting compression to +When set to .Sy on -indicates that the current default compression algorithm should be used. +(the default), indicates that the current default compression algorithm should be used. The default balances compression and decompression speed, with compression ratio and is expected to work well on a wide variety of workloads. Unlike all other settings for this property, @@ -872,14 +873,17 @@ This is done using .Sy zstd-fast- Ns Ar N , where .Ar N -is an integer in [1-9,10,20,30,...,100,500,1000] which maps to a negative +is an integer in +.Bq Sy 1 Ns - Ns Sy 10 , 20 , 30 , No … , Sy 100 , 500 , 1000 +which maps to a negative .Sy zstd level. -The lower the level the faster the compression - -.Ar 1000 No provides the fastest compression and lowest compression ratio. +The lower the level the faster the compression \(em +.Sy 1000 +provides the fastest compression and lowest compression ratio. .Sy zstd-fast is equivalent to -.Sy zstd-fast-1 . +.Sy zstd-fast- Ns Ar 1 . .Pp The .Sy zle @@ -901,7 +905,7 @@ after compression, otherwise the compression will not be considered worthwhile and the block saved uncompressed. Note that when the logical block is less than 8 times the disk sector size this effectively reduces the necessary compression -ratio; for example, 8kB blocks on disks with 4kB disk sectors must compress to 1/2 +ratio; for example, 8 KiB blocks on disks with 4 KiB disk sectors must compress to 1/2 or less of their original size. .It Xo .Sy context Ns = Ns Sy none Ns | Ns @@ -981,7 +985,7 @@ mount options. .It Xo .Sy dedup Ns = Ns Sy off Ns | Ns Sy on Ns | Ns Sy verify Ns | Ns .Sy sha256 Ns Oo , Ns Sy verify Oc Ns | Ns Sy sha512 Ns Oo , Ns Sy verify Oc Ns | Ns Sy skein Ns Oo , Ns Sy verify Oc Ns | Ns -.Sy edonr , Ns Sy verify +.Sy edonr , Ns Sy verify Ns | Ns Sy blake3 Ns Oo , Ns Sy verify Oc Ns .Xc Configures deduplication for a dataset. The default value is @@ -1196,7 +1200,7 @@ blocks into the special allocation class. Blocks smaller than or equal to this value will be assigned to the special allocation class while greater blocks will be assigned to the regular class. -Valid values are zero or a power of two from 512B up to 1M. +Valid values are zero or a power of two from 512 up to 1048576 (1 MiB). The default size is 0 which means no small file blocks will be allocated in the special class. .Pp @@ -1315,7 +1319,7 @@ can get and set everyone's quota. This property is not available on volumes, on file systems before version 4, or on pools before version 15. The -.Sy userquota@ Ns Ar ... +.Sy userquota@ Ns Ar … properties are not displayed by .Nm zfs Cm get Sy all . The user's name must be appended after the @@ -1423,13 +1427,13 @@ Use of this property for general purpose file systems is strongly discouraged, and may adversely affect performance. .Pp The size specified must be a power of two greater than or equal to -.Ar 512B +.Ar 512 B and less than or equal to -.Ar 128kB . +.Ar 128 KiB . If the .Sy large_blocks feature is enabled on the pool, the size may be up to -.Ar 1MB . +.Ar 1 MiB . See .Xr zpool-features 7 for details on ZFS feature flags. @@ -1610,8 +1614,11 @@ the file systems are unshared. .Pp The share is created with the ACL (Access Control List) "Everyone:F" ("F" stands for "full permissions", i.e. read and write permissions) and no guest -access (which means Samba must be able to authenticate a real user, system -passwd/shadow, LDAP or smbpasswd based) by default. +access (which means Samba must be able to authenticate a real user \(em +.Xr passwd 5 Ns / Ns Xr shadow 5 Ns - , +LDAP- or +.Xr smbpasswd 5 Ns -based ) +by default. This means that any additional access control (disallow specific user specific access etc) must be done on the underlying file system. .It Sy sharenfs Ns = Ns Sy on Ns | Ns Sy off Ns | Ns Ar opts @@ -1816,31 +1823,34 @@ The default value is This property is not used by OpenZFS. .It Sy xattr Ns = Ns Sy on Ns | Ns Sy off Ns | Ns Sy sa Controls whether extended attributes are enabled for this file system. -Two styles of extended attributes are supported: either directory based -or system attribute based. +Two styles of extended attributes are supported: either directory-based +or system-attribute-based. .Pp The default value of .Sy on -enables directory based extended attributes. +enables directory-based extended attributes. This style of extended attribute imposes no practical limit on either the size or number of attributes which can be set on a file. Although under Linux the .Xr getxattr 2 and .Xr setxattr 2 -system calls limit the maximum size to 64K. +system calls limit the maximum size to +.Sy 64K . This is the most compatible style of extended attribute and is supported by all ZFS implementations. .Pp -System attribute based xattrs can be enabled by setting the value to +System-attribute-based xattrs can be enabled by setting the value to .Sy sa . The key advantage of this type of xattr is improved performance. Storing extended attributes as system attributes -significantly decreases the amount of disk IO required. -Up to 64K of data may be stored per-file in the space reserved for system attributes. +significantly decreases the amount of disk I/O required. +Up to +.Sy 64K +of data may be stored per-file in the space reserved for system attributes. If there is not enough space available for an extended attribute -then it will be automatically written as a directory based xattr. -System attribute based extended attributes are not accessible +then it will be automatically written as a directory-based xattr. +System-attribute-based extended attributes are not accessible on platforms which do not support the .Sy xattr Ns = Ns Sy sa feature. @@ -1850,7 +1860,7 @@ on both .Fx and Linux. .Pp -The use of system attribute based xattrs is strongly encouraged for users of +The use of system-attribute-based xattrs is strongly encouraged for users of SELinux or POSIX ACLs. Both of these features heavily rely on extended attributes and benefit significantly from the reduced access time. @@ -1875,8 +1885,7 @@ feature and are not relevant on other platforms. The default value is .Sy off . .It Sy zoned Ns = Ns Sy on Ns | Ns Sy off -Controls whether the dataset is managed from a non-global zone. -Zones are a Solaris feature and are not relevant on other platforms. +Controls whether the dataset is managed from a non-global zone or namespace. The default value is .Sy off . .El diff --git a/man/man7/zpool-features.7 b/man/man7/zpool-features.7 index d10ca7f441f6..09f1e50decda 100644 --- a/man/man7/zpool-features.7 +++ b/man/man7/zpool-features.7 @@ -5,7 +5,7 @@ .\" The contents of this file are subject to the terms of the Common Development .\" and Distribution License (the "License"). You may not use this file except .\" in compliance with the License. You can obtain a copy of the license at -.\" usr/src/OPENSOLARIS.LICENSE or http://www.opensolaris.org/os/licensing. +.\" usr/src/OPENSOLARIS.LICENSE or https://opensource.org/licenses/CDDL-1.0. .\" .\" See the License for the specific language governing permissions and .\" limitations under the License. When distributing Covered Code, include this @@ -18,7 +18,7 @@ .\" Copyright (c) 2019, Allan Jude .\" Copyright (c) 2021, Colm Buckley .\" -.Dd May 31, 2021 +.Dd June 23, 2022 .Dt ZPOOL-FEATURES 7 .Os . @@ -27,8 +27,10 @@ .Nd description of ZFS pool features . .Sh DESCRIPTION -ZFS pool on-disk format versions are specified via "features" which replace -the old on-disk format numbers (the last supported on-disk format number is 28). +ZFS pool on-disk format versions are specified via +.Dq features +which replace the old on-disk format numbers +.Pq the last supported on-disk format number is 28 . To enable a feature on a pool use the .Nm zpool Cm upgrade , or set the @@ -62,10 +64,12 @@ implementation that created the pool for information about those features. Each supported feature also has a short name. By convention a feature's short name is the portion of its GUID which follows the .Sq \&: -(i.e. +.Po +i.e. .Ar com.example : Ns Ar feature-name would have the short name -.Ar feature-name ) , +.Ar feature-name +.Pc , however a feature's short name may differ across ZFS implementations if following the convention would result in name conflicts. . @@ -110,9 +114,11 @@ These features are referred to as If all unsupported features on a pool are read-only compatible, the pool can be imported in read-only mode by setting the .Sy readonly -property during import (see +property during import +.Po see .Xr zpool-import 8 -for details on importing pools). +for details on importing pools +.Pc . . .Ss Unsupported features For each unsupported feature enabled on an imported pool, a pool property @@ -143,21 +149,25 @@ The feature facilitates this by allowing feature sets to be read from text files. When set to .Sy off -(the default), compatibility feature sets are disabled -(i.e. all features are enabled); when set to +.Pq the default , +compatibility feature sets are disabled +.Pq i.e. all features are enabled ; +when set to .Sy legacy , no features are enabled. When set to a comma-separated list of filenames -(each filename may either be an absolute path, or relative to +.Po +each filename may either be an absolute path, or relative to .Pa /etc/zfs/compatibility.d or -.Pa /usr/share/zfs/compatibility.d ) , +.Pa /usr/share/zfs/compatibility.d +.Pc , the lists of requested features are read from those files, separated by whitespace and/or commas. Only features present in all files are enabled. .Pp Simple sanity checks are applied to the files: -they must be between 1B and 16kB in size, and must end with a newline character. +they must be between 1 B and 16 KiB in size, and must end with a newline character. .Pp The requested features are applied when a pool is created using .Nm zpool Cm create Fl o Sy compatibility Ns = Ns Ar … @@ -291,7 +301,9 @@ This feature enables support for separate allocation classes. .Pp This feature becomes .Sy active -when a dedicated allocation class vdev (dedup or special) is created with the +when a dedicated allocation class vdev +.Pq dedup or special +is created with the .Nm zpool Cm create No or Nm zpool Cm add No commands . With device removal, it can be returned to the .Sy enabled @@ -326,6 +338,12 @@ while .Sy freeing is non-zero. . +.feature org.openzfs blake3 no extensible_dataset +This feature enables the use of the BLAKE3 hash algorithm for checksum and dedup. +BLAKE3 is a secure hash algorithm focused on high performance. +.Pp +.checksum-spiel blake3 +. .feature com.delphix bookmarks yes extensible_dataset This feature enables use of the .Nm zfs Cm bookmark @@ -350,8 +368,9 @@ state when all v2 bookmarks are destroyed. .feature com.delphix bookmark_written no bookmark extensible_dataset bookmark_v2 This feature enables additional bookmark accounting fields, enabling the .Sy written Ns # Ns Ar bookmark -property (space written since a bookmark) and estimates of -send stream sizes for incrementals from bookmarks. +property +.Pq space written since a bookmark +and estimates of send stream sizes for incrementals from bookmarks. .Pp This feature becomes .Sy active @@ -366,7 +385,8 @@ This feature enables the ability for the and .Nm zpool Cm replace commands to perform sequential reconstruction -(instead of healing reconstruction) when resilvering. +.Pq instead of healing reconstruction +when resilvering. .Pp Sequential reconstruction resilvers a device in LBA order without immediately verifying the checksums. @@ -403,8 +423,8 @@ on a top-level vdev, and will never return to being This feature enables use of the .Sy draid vdev type. -dRAID is a variant of raidz which provides integrated distributed -hot spares that allow faster resilvering while retaining the benefits of raidz. +dRAID is a variant of RAID-Z which provides integrated distributed +hot spares that allow faster resilvering while retaining the benefits of RAID-Z. Data, parity, and spare space are organized in redundancy groups and distributed evenly over all of the devices. .Pp @@ -418,8 +438,10 @@ vdev to an existing pool. . .feature org.illumos edonr no extensible_dataset This feature enables the use of the Edon-R hash algorithm for checksum, -including for nopwrite (if compression is also enabled, an overwrite of -a block whose checksum matches the data being written will be ignored). +including for nopwrite +.Po if compression is also enabled, an overwrite of +a block whose checksum matches the data being written will be ignored +.Pc . In an abundance of caution, Edon-R requires verification when used with dedup: .Nm zfs Cm set Sy dedup Ns = Ns Sy edonr , Ns Sy verify @@ -427,15 +449,19 @@ dedup: .Pp Edon-R is a very high-performance hash algorithm that was part of the NIST SHA-3 competition. -It provides extremely high hash performance (over 350% faster than SHA-256), +It provides extremely high hash performance +.Pq over 350% faster than SHA-256 , but was not selected because of its unsuitability as a general purpose secure hash algorithm. This implementation utilizes the new salted checksumming functionality in ZFS, which means that the checksum is pre-seeded with a secret -256-bit random key (stored on the pool) before being fed the data block -to be checksummed. +256-bit random key +.Pq stored on the pool +before being fed the data block to be checksummed. Thus the produced checksums are unique to a given pool, preventing hash collision attacks on systems with dedup. +.Pp +.checksum-spiel edonr . .feature com.delphix embedded_data no This feature improves the performance and compression ratio of @@ -444,10 +470,15 @@ Blocks whose contents can compress to 112 bytes or smaller can take advantage of this feature. .Pp When this feature is enabled, the contents of highly-compressible blocks are -stored in the block "pointer" itself (a misnomer in this case, as it contains -the compressed data, rather than a pointer to its location on disk). -Thus the space of the block (one sector, typically 512B or 4kB) is saved, -and no additional I/O is needed to read and write the data block. +stored in the block +.Dq pointer +itself +.Po a misnomer in this case, as it contains +the compressed data, rather than a pointer to its location on disk +.Pc . +Thus the space of the block +.Pq one sector, typically 512 B or 4 KiB +is saved, and no additional I/O is needed to read and write the data block. . \*[instant-never] . @@ -457,7 +488,9 @@ number of snapshots of a single filesystem or volume, and also reduces the disk space required. .Pp When there are many snapshots, each snapshot uses many Block Pointer -Objects (bpobjs) to track blocks associated with that snapshot. +Objects +.Pq bpobjs +to track blocks associated with that snapshot. However, in common use cases, most of these bpobjs are empty. This feature allows us to create each bpobj on-demand, thus eliminating the empty bpobjs. @@ -474,8 +507,7 @@ This has no user-visible impact, but other features may depend on this feature. .Pp This feature becomes .Sy active - as soon as it is enabled and will -never return to being +as soon as it is enabled and will never return to being .Sy enabled . . .feature com.datto encryption no bookmark_v2 extensible_dataset @@ -504,8 +536,20 @@ can be created at the point in the tree on which the limits are set. .Pp This feature is .Sy active -once either of the limit properties has been set on a dataset. -Once activated the feature is never deactivated. +once either of the limit properties has been set on a dataset +and will never return to being +.Sy enabled . +. +.feature com.delphix head_errlog no +This feature enables the upgraded version of errlog, which required an on-disk +error log format change. +Now the error log of each head dataset is stored separately in the zap object +and keyed by the head id. +With this feature enabled, every dataset affected by an error block is listed +in the output of +.Nm zpool Cm status . +.Pp +\*[instant-never] . .feature com.delphix hole_birth no enabled_txg This feature has/had bugs, the result of which is that, if you do a @@ -517,7 +561,12 @@ will not match the source. Its use by .Nm zfs Cm send Fl i has been disabled by default -.Pq see Sy send_holes_without_birth_time No in Xr zfs 4 . +.Po +see +.Sy send_holes_without_birth_time +in +.Xr zfs 4 +.Pc . .Pp This feature improves performance of incremental sends .Pq Nm zfs Cm send Fl i @@ -530,8 +579,10 @@ contains information about every block that changed between .Sy A No and Sy B . Blocks which did not change between those snapshots can be identified and omitted from the stream using a piece of metadata called -the "block birth time", but birth times are not recorded for holes -(blocks filled only with zeroes). +the +.Dq block birth time , +but birth times are not recorded for holes +.Pq blocks filled only with zeroes . Since holes created after .Sy A No cannot be distinguished from holes created before Sy A , information about every hole in the entire filesystem or zvol @@ -539,9 +590,9 @@ is included in the send stream. .Pp For workloads where holes are rare this is not a problem. However, when incrementally replicating filesystems or zvols with many holes -(for example a zvol formatted with another filesystem) a lot of time will -be spent sending and receiving unnecessary information about holes that -already exist on the receiving side. +.Pq for example a zvol formatted with another filesystem +a lot of time will be spent sending and receiving unnecessary information +about holes that already exist on the receiving side. .Pp Once the .Sy hole_birth @@ -554,29 +605,29 @@ already exist on the receiving side. \*[instant-never] . .feature org.open-zfs large_blocks no extensible_dataset -This feature allows the record size on a dataset to be set larger than 128kB. +This feature allows the record size on a dataset to be set larger than 128 KiB. .Pp This feature becomes .Sy active -once a dataset contains a file with a block size larger than 128kB, +once a dataset contains a file with a block size larger than 128 KiB, and will return to being .Sy enabled -once all filesystems that have ever had their recordsize larger than 128kB +once all filesystems that have ever had their recordsize larger than 128 KiB are destroyed. . .feature org.zfsonlinux large_dnode no extensible_dataset -This feature allows the size of dnodes in a dataset to be set larger than 512B. +This feature allows the size of dnodes in a dataset to be set larger than 512 B. . This feature becomes .Sy active -once a dataset contains an object with a dnode larger than 512B, +once a dataset contains an object with a dnode larger than 512 B, which occurs as a result of setting the .Sy dnodesize dataset property to a value other than .Sy legacy . The feature will return to being .Sy enabled -once all filesystems that have ever contained a dnode larger than 512B +once all filesystems that have ever contained a dnode larger than 512 B are destroyed. Large dnodes allow more data to be stored in the bonus buffer, thus potentially improving performance by avoiding the use of spill blocks. @@ -638,7 +689,7 @@ When the feature is set to .Sy enabled , the administrator can use -.Xr dumpadm 1M +.Xr dumpadm 8 to configure a dump device on a pool comprised of multiple vdevs. .Pp Under @@ -658,7 +709,9 @@ This feature is an enhancement of .Sy device_removal , which will over time reduce the memory used to track removed devices. When indirect blocks are freed or remapped, -we note that their part of the indirect mapping is "obsolete" – no longer needed. +we note that their part of the indirect mapping is +.Dq obsolete +– no longer needed. .Pp This feature becomes .Sy active @@ -669,7 +722,8 @@ command is used on a top-level vdev, and will never return to being . .feature org.zfsonlinux project_quota yes extensible_dataset This feature allows administrators to account the spaces and objects usage -information against the project identifier (ID). +information against the project identifier +.Pq ID . .Pp The project ID is an object-based attribute. When upgrading an existing filesystem, @@ -679,7 +733,8 @@ their parent directories' project ID if the parent's inherit flag is set .Pq via Nm chattr Sy [+-]P No or Nm zfs Cm project Fl s Ns | Ns Fl C . Otherwise, the new object's project ID will be zero. An object's project ID can be changed at any time by the owner -(or privileged user) via +.Pq or privileged user +via .Nm chattr Fl p Ar prjid or .Nm zfs Cm project Fl p Ar prjid . @@ -700,8 +755,8 @@ For more information about redacted sends, see . .feature com.delphix redacted_datasets no extensible_dataset This feature enables the receiving of redacted -.Nm zfs Cm send Ns -streams. which create redacted datasets when received. +.Nm zfs Cm send +streams, which create redacted datasets when received. These datasets are missing some of their blocks, and so cannot be safely mounted, and their contents cannot be safely read. For more information about redacted receives, see @@ -721,7 +776,8 @@ when the deferred resilver begins. . .feature org.illumos sha512 no extensible_dataset This feature enables the use of the SHA-512/256 truncated hash algorithm -(FIPS 180-4) for checksum and dedup. +.Pq FIPS 180-4 +for checksum and dedup. The native 64-bit arithmetic of SHA-512 provides an approximate 50% performance boost over SHA-256 on 64-bit hardware and is thus a good minimum-change replacement candidate @@ -737,11 +793,12 @@ This feature enables the use of the Skein hash algorithm for checksum and dedup. Skein is a high-performance secure hash algorithm that was a finalist in the NIST SHA-3 competition. It provides a very high security margin and high performance on 64-bit hardware -(80% faster than SHA-256). +.Pq 80% faster than SHA-256 . This implementation also utilizes the new salted checksumming functionality in ZFS, which means that the checksum is pre-seeded with a -secret 256-bit random key (stored on the pool) before being fed the data -block to be checksummed. +secret 256-bit random key +.Pq stored on the pool +before being fed the data block to be checksummed. Thus the produced checksums are unique to a given pool, preventing hash collision attacks on systems with dedup. .Pp @@ -759,7 +816,9 @@ and never returns back to being . .feature com.delphix spacemap_v2 yes This feature enables the use of the new space map encoding which -consists of two words (instead of one) whenever it is advantageous. +consists of two words +.Pq instead of one +whenever it is advantageous. The new encoding allows space maps to represent large regions of space more efficiently on-disk while also increasing their maximum addressable offset. @@ -778,6 +837,24 @@ by user and group. \*[instant-never] \*[remount-upgrade] . +.feature org.openzfs zilsaxattr yes extensible_dataset +This feature enables +.Sy xattr Ns = Ns Sy sa +extended attribute logging in the ZIL. +If enabled, extended attribute changes +.Pq both Sy xattrdir Ns = Ns Sy dir No and Sy xattr Ns = Ns Sy sa +are guaranteed to be durable if either the dataset had +.Sy sync Ns = Ns Sy always +set at the time the changes were made, or +.Xr sync 2 +is called on the dataset after the changes were made. +.Pp +This feature becomes +.Sy active +when a ZIL is created for at least one dataset and will be returned to the +.Sy enabled +state when it is destroyed for all datasets that use this feature. +. .feature com.delphix zpool_checkpoint yes This feature enables the .Nm zpool Cm checkpoint @@ -835,4 +912,5 @@ are destroyed. .El . .Sh SEE ALSO +.Xr zfs 8 , .Xr zpool 8 diff --git a/man/man7/zpoolconcepts.7 b/man/man7/zpoolconcepts.7 index 58132baf5025..c181ea3b8890 100644 --- a/man/man7/zpoolconcepts.7 +++ b/man/man7/zpoolconcepts.7 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -107,7 +107,7 @@ Unlike raidz, dRAID uses a fixed stripe width (padding as necessary with zeros) to allow fully sequential resilvering. This fixed stripe width significantly effects both usable capacity and IOPS. For example, with the default -.Em D=8 No and Em 4kB No disk sectors the minimum allocation size is Em 32kB . +.Em D=8 No and Em 4 KiB No disk sectors the minimum allocation size is Em 32 KiB . If using compression, this relatively large allocation size can reduce the effective compression ratio. When using ZFS volumes and dRAID, the default of the @@ -123,7 +123,7 @@ In regards to I/O, performance is similar to raidz since for any read all Delivered random IOPS can be reasonably approximated as .Sy floor((N-S)/(D+P))*single_drive_IOPS . .Pp -Like raidzm a dRAID can have single-, double-, or triple-parity. +Like raidz, a dRAID can have single-, double-, or triple-parity. The .Sy draid1 , .Sy draid2 , @@ -422,13 +422,13 @@ asynchronously when importing the pool in L2ARC (persistent L2ARC). This can be disabled by setting .Sy l2arc_rebuild_enabled Ns = Ns Sy 0 . For cache devices smaller than -.Em 1GB , +.Em 1 GiB , we do not write the metadata structures required for rebuilding the L2ARC in order not to waste space. This can be changed with .Sy l2arc_rebuild_blocks_min_l2size . The cache device header -.Pq Em 512B +.Pq Em 512 B is updated even if no metadata structures are written. Setting .Sy l2arc_headroom Ns = Ns Sy 0 diff --git a/man/man7/zpoolprops.7 b/man/man7/zpoolprops.7 index 5bd1c2bf3128..a150f6d43706 100644 --- a/man/man7/zpoolprops.7 +++ b/man/man7/zpoolprops.7 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -188,7 +188,7 @@ layer and a ZFS internal exception list. I/O operations will be aligned to the specified size boundaries. Additionally, the minimum (disk) write size will be set to the specified size, so this represents a -space vs. performance trade-off. +space/performance trade-off. For optimal performance, the pool sector size should be greater than or equal to the sector size of the underlying disks. The typical case for setting this property is when diff --git a/man/man8/fsck.zfs.8 b/man/man8/fsck.zfs.8 index 0ce7576ebe63..a3506798b38a 100644 --- a/man/man8/fsck.zfs.8 +++ b/man/man8/fsck.zfs.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/mount.zfs.8 b/man/man8/mount.zfs.8 index 2101f70cd595..0840ac531c63 100644 --- a/man/man8/mount.zfs.8 +++ b/man/man8/mount.zfs.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zdb.8 b/man/man8/zdb.8 index 55c575f0db1b..31ee601538ce 100644 --- a/man/man8/zdb.8 +++ b/man/man8/zdb.8 @@ -25,18 +25,18 @@ .Nm .Op Fl AbcdDFGhikLMNPsvXYy .Op Fl e Oo Fl V Oc Oo Fl p Ar path Oc Ns … -.Op Fl I Ar inflight I/Os +.Op Fl I Ar inflight-I/O-ops .Oo Fl o Ar var Ns = Ns Ar value Oc Ns … .Op Fl t Ar txg .Op Fl U Ar cache .Op Fl x Ar dumpdir -.Op Ar poolname Ns Op / Ns Ar dataset | objset ID +.Op Ar poolname Ns Op / Ns Ar dataset Ns | Ns Ar objset-ID .Op Ar object Ns | Ns Ar range Ns … .Nm .Op Fl AdiPv .Op Fl e Oo Fl V Oc Oo Fl p Ar path Oc Ns … .Op Fl U Ar cache -.Ar poolname Ns Op Ar / Ns Ar dataset | objset ID +.Ar poolname Ns Op Ar / Ns Ar dataset Ns | Ns Ar objset-ID .Op Ar object Ns | Ns Ar range Ns … .Nm .Fl C @@ -84,7 +84,7 @@ some amount of consistency checking. It is a not a general purpose tool and options .Pq and facilities may change. -This is not a +It is not a .Xr fsck 8 utility. .Pp @@ -140,9 +140,9 @@ size, and object count. See .Fl N for determining if -.Op Ar poolname Ns Op / Ns Ar dataset | objset ID +.Ar poolname Ns Op / Ns Ar dataset Ns | Ns Ar objset-ID is to use the specified -.Op Ar dataset | objset ID +.Ar dataset Ns | Ns Ar objset-ID as a string (dataset name) or a number (objset ID) when datasets have numeric names. .Pp @@ -198,7 +198,7 @@ compression ratio inflation due to the zfs copies property .Pq Sy copies , and an overall effective ratio -.Pq Sy dedup No * Sy compress No / Sy copies . +.Pq Sy dedup No \(mu Sy compress No / Sy copies . .It Fl DD Display a histogram of deduplication statistics, showing the allocated .Pq physically present on disk @@ -287,9 +287,9 @@ Display every spacemap record. Same as .Fl d but force zdb to interpret the -.Op Ar dataset | objset ID +.Op Ar dataset Ns | Ns Ar objset-ID in -.Op Ar poolname Ns Op / Ns Ar dataset | objset ID +.Op Ar poolname Ns Op / Ns Ar dataset Ns | Ns Ar objset-ID as a numeric objset ID. .It Fl O Ar dataset path Look up the specified @@ -404,13 +404,13 @@ transactions. Dump the contents of the zfs_dbgmsg buffer before exiting .Nm . zfs_dbgmsg is a buffer used by ZFS to dump advanced debug information. -.It Fl I , -inflight Ns = Ns Ar inflight I/Os -Limit the number of outstanding checksum I/Os to the specified value. +.It Fl I , -inflight Ns = Ns Ar inflight-I/O-ops +Limit the number of outstanding checksum I/O operations to the specified value. The default value is 200. This option affects the performance of the .Fl c option. -.It Fl o , -option Ns = Ns Ar var Ns = Ns Ar value … +.It Fl o , -option Ns = Ns Ar var Ns = Ns Ar value Ns … Set the given global libzpool variable to the provided value. The value must be an unsigned 32-bit integer. Currently only little-endian systems are supported to avoid accidentally setting @@ -464,12 +464,7 @@ If no options are specified, all information about the named pool will be displayed at default verbosity. . .Sh EXAMPLES -.Bl -tag -width Ds -.It Xo -.Sy Example 1 : -Display the configuration of imported pool -.Ar rpool -.Xc +.Ss Example 1 : No Display the configuration of imported pool Ar rpool .Bd -literal .No # Nm zdb Fl C Ar rpool MOS Configuration: @@ -477,22 +472,16 @@ MOS Configuration: name: 'rpool' … .Ed -.It Xo -.Sy Example 2 : -Display basic dataset information about -.Ar rpool -.Xc +. +.Ss Example 2 : No Display basic dataset information about Ar rpool .Bd -literal .No # Nm zdb Fl d Ar rpool Dataset mos [META], ID 0, cr_txg 4, 26.9M, 1051 objects Dataset rpool/swap [ZVOL], ID 59, cr_txg 356, 486M, 2 objects … .Ed -.It Xo -.Sy Example 3 : -Display basic information about object 0 in -.Ar rpool/export/home -.Xc +. +.Ss Example 3 : No Display basic information about object 0 in Ar rpool/export/home .Bd -literal .No # Nm zdb Fl d Ar rpool/export/home 0 Dataset rpool/export/home [ZPL], ID 137, cr_txg 1546, 32K, 8 objects @@ -500,11 +489,8 @@ Dataset rpool/export/home [ZPL], ID 137, cr_txg 1546, 32K, 8 objects Object lvl iblk dblk dsize lsize %full type 0 7 16K 16K 15.0K 16K 25.00 DMU dnode .Ed -.It Xo -.Sy Example 4 : -Display the predicted effect of enabling deduplication on -.Ar rpool -.Xc +. +.Ss Example 4 : No Display the predicted effect of enabling deduplication on Ar rpool .Bd -literal .No # Nm zdb Fl S Ar rpool Simulated DDT histogram: @@ -518,7 +504,6 @@ refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE … dedup = 1.11, compress = 1.80, copies = 1.00, dedup * compress / copies = 2.00 .Ed -.El . .Sh SEE ALSO .Xr zfs 8 , diff --git a/man/man8/zed.8.in b/man/man8/zed.8.in index d3297605206e..6c51f10695cc 100644 --- a/man/man8/zed.8.in +++ b/man/man8/zed.8.in @@ -75,7 +75,7 @@ Custom .Ev $PATH for zedlets to use. Normally zedlets run in a locked-down environment, with hardcoded paths to the ZFS commands -.Pq Ev $ZFS , $ZPOOL , $ZED , ... , +.Pq Ev $ZFS , $ZPOOL , $ZED , … , and a hard-coded .Ev $PATH . This is done for security reasons. diff --git a/man/man8/zfs-allow.8 b/man/man8/zfs-allow.8 index bbd62edc2896..c88cf9f00924 100644 --- a/man/man8/zfs-allow.8 +++ b/man/man8/zfs-allow.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -29,7 +29,7 @@ .\" Copyright 2018 Nexenta Systems, Inc. .\" Copyright 2019 Joyent, Inc. .\" -.Dd May 27, 2021 +.Dd March 16, 2022 .Dt ZFS-ALLOW 8 .Os . @@ -215,19 +215,19 @@ send subcommand share subcommand Allows sharing file systems over NFS or SMB protocols snapshot subcommand Must also have the \fBmount\fR ability -groupquota other Allows accessing any \fBgroupquota@\fI...\fR property -groupobjquota other Allows accessing any \fBgroupobjquota@\fI...\fR property -groupused other Allows reading any \fBgroupused@\fI...\fR property -groupobjused other Allows reading any \fBgroupobjused@\fI...\fR property +groupquota other Allows accessing any \fBgroupquota@\fI…\fR property +groupobjquota other Allows accessing any \fBgroupobjquota@\fI…\fR property +groupused other Allows reading any \fBgroupused@\fI…\fR property +groupobjused other Allows reading any \fBgroupobjused@\fI…\fR property userprop other Allows changing any user property -userquota other Allows accessing any \fBuserquota@\fI...\fR property -userobjquota other Allows accessing any \fBuserobjquota@\fI...\fR property -userused other Allows reading any \fBuserused@\fI...\fR property -userobjused other Allows reading any \fBuserobjused@\fI...\fR property -projectobjquota other Allows accessing any \fBprojectobjquota@\fI...\fR property -projectquota other Allows accessing any \fBprojectquota@\fI...\fR property -projectobjused other Allows reading any \fBprojectobjused@\fI...\fR property -projectused other Allows reading any \fBprojectused@\fI...\fR property +userquota other Allows accessing any \fBuserquota@\fI…\fR property +userobjquota other Allows accessing any \fBuserobjquota@\fI…\fR property +userused other Allows reading any \fBuserused@\fI…\fR property +userobjused other Allows reading any \fBuserobjused@\fI…\fR property +projectobjquota other Allows accessing any \fBprojectobjquota@\fI…\fR property +projectquota other Allows accessing any \fBprojectquota@\fI…\fR property +projectobjused other Allows reading any \fBprojectobjused@\fI…\fR property +projectused other Allows reading any \fBprojectused@\fI…\fR property aclinherit property aclmode property @@ -384,3 +384,109 @@ Removes permissions from a permission set. If no permissions are specified, then all permissions are removed, thus removing the set entirely. .El +. +.Sh EXAMPLES +.\" These are, respectively, examples 17, 18, 19, 20 from zfs.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Delegating ZFS Administration Permissions on a ZFS Dataset +The following example shows how to set permissions so that user +.Ar cindys +can create, destroy, mount, and take snapshots on +.Ar tank/cindys . +The permissions on +.Ar tank/cindys +are also displayed. +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm allow Sy cindys create , Ns Sy destroy , Ns Sy mount , Ns Sy snapshot Ar tank/cindys +.No # Nm zfs Cm allow Ar tank/cindys +---- Permissions on tank/cindys -------------------------------------- +Local+Descendent permissions: + user cindys create,destroy,mount,snapshot +.Ed +.Pp +Because the +.Ar tank/cindys +mount point permission is set to 755 by default, user +.Ar cindys +will be unable to mount file systems under +.Ar tank/cindys . +Add an ACE similar to the following syntax to provide mount point access: +.Dl # Cm chmod No A+user: Ns Ar cindys Ns :add_subdirectory:allow Ar /tank/cindys +. +.Ss Example 2 : No Delegating Create Time Permissions on a ZFS Dataset +The following example shows how to grant anyone in the group +.Ar staff +to create file systems in +.Ar tank/users . +This syntax also allows staff members to destroy their own file systems, but not +destroy anyone else's file system. +The permissions on +.Ar tank/users +are also displayed. +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm allow Ar staff Sy create , Ns Sy mount Ar tank/users +.No # Nm zfs Cm allow Fl c Sy destroy Ar tank/users +.No # Nm zfs Cm allow Ar tank/users +---- Permissions on tank/users --------------------------------------- +Permission sets: + destroy +Local+Descendent permissions: + group staff create,mount +.Ed +. +.Ss Example 3 : No Defining and Granting a Permission Set on a ZFS Dataset +The following example shows how to define and grant a permission set on the +.Ar tank/users +file system. +The permissions on +.Ar tank/users +are also displayed. +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm allow Fl s No @ Ns Ar pset Sy create , Ns Sy destroy , Ns Sy snapshot , Ns Sy mount Ar tank/users +.No # Nm zfs Cm allow staff No @ Ns Ar pset tank/users +.No # Nm zfs Cm allow Ar tank/users +---- Permissions on tank/users --------------------------------------- +Permission sets: + @pset create,destroy,mount,snapshot +Local+Descendent permissions: + group staff @pset +.Ed +. +.Ss Example 4 : No Delegating Property Permissions on a ZFS Dataset +The following example shows to grant the ability to set quotas and reservations +on the +.Ar users/home +file system. +The permissions on +.Ar users/home +are also displayed. +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm allow Ar cindys Sy quota , Ns Sy reservation Ar users/home +.No # Nm zfs Cm allow Ar users/home +---- Permissions on users/home --------------------------------------- +Local+Descendent permissions: + user cindys quota,reservation +cindys% zfs set quota=10G users/home/marks +cindys% zfs get quota users/home/marks +NAME PROPERTY VALUE SOURCE +users/home/marks quota 10G local +.Ed +. +.Ss Example 5 : No Removing ZFS Delegated Permissions on a ZFS Dataset +The following example shows how to remove the snapshot permission from the +.Ar staff +group on the +.Sy tank/users +file system. +The permissions on +.Sy tank/users +are also displayed. +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm unallow Ar staff Sy snapshot Ar tank/users +.No # Nm zfs Cm allow Ar tank/users +---- Permissions on tank/users --------------------------------------- +Permission sets: + @pset create,destroy,mount,snapshot +Local+Descendent permissions: + group staff @pset +.Ed diff --git a/man/man8/zfs-bookmark.8 b/man/man8/zfs-bookmark.8 index 094a7b30902f..01364b0f5052 100644 --- a/man/man8/zfs-bookmark.8 +++ b/man/man8/zfs-bookmark.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -30,7 +30,7 @@ .\" Copyright 2019 Joyent, Inc. .\" Copyright (c) 2019, 2020 by Christian Schwarz. All Rights Reserved. .\" -.Dd May 27, 2021 +.Dd May 12, 2022 .Dt ZFS-BOOKMARK 8 .Os . @@ -61,6 +61,14 @@ for details on ZFS feature flags and the .Sy bookmarks feature. . +.Sh EXAMPLES +.\" These are, respectively, examples 23 from zfs.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Creating a bookmark +The following example creates a bookmark to a snapshot. +This bookmark can then be used instead of a snapshot in send streams. +.Dl # Nm zfs Cm bookmark Ar rpool Ns @ Ns Ar snapshot rpool Ns # Ns Ar bookmark +. .Sh SEE ALSO .Xr zfs-destroy 8 , .Xr zfs-send 8 , diff --git a/man/man8/zfs-clone.8 b/man/man8/zfs-clone.8 index 0640244f2009..fee629951c36 100644 --- a/man/man8/zfs-clone.8 +++ b/man/man8/zfs-clone.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -29,7 +29,7 @@ .\" Copyright 2018 Nexenta Systems, Inc. .\" Copyright 2019 Joyent, Inc. .\" -.Dd May 27, 2021 +.Dd March 16, 2022 .Dt ZFS-CLONE 8 .Os . @@ -65,6 +65,32 @@ If the target filesystem or volume already exists, the operation completes successfully. .El . +.Sh EXAMPLES +.\" These are, respectively, examples 9, 10 from zfs.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Creating a ZFS Clone +The following command creates a writable file system whose initial contents are +the same as +.Ar pool/home/bob@yesterday . +.Dl # Nm zfs Cm clone Ar pool/home/bob@yesterday pool/clone +. +.Ss Example 2 : No Promoting a ZFS Clone +The following commands illustrate how to test out changes to a file system, and +then replace the original file system with the changed one, using clones, clone +promotion, and renaming: +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm create Ar pool/project/production + populate /pool/project/production with data +.No # Nm zfs Cm snapshot Ar pool/project/production Ns @ Ns Ar today +.No # Nm zfs Cm clone Ar pool/project/production@today pool/project/beta + make changes to /pool/project/beta and test them +.No # Nm zfs Cm promote Ar pool/project/beta +.No # Nm zfs Cm rename Ar pool/project/production pool/project/legacy +.No # Nm zfs Cm rename Ar pool/project/beta pool/project/production + once the legacy version is no longer needed, it can be destroyed +.No # Nm zfs Cm destroy Ar pool/project/legacy +.Ed +. .Sh SEE ALSO .Xr zfs-promote 8 , .Xr zfs-snapshot 8 diff --git a/man/man8/zfs-create.8 b/man/man8/zfs-create.8 index 55397fa661d5..a7b6097c37f1 100644 --- a/man/man8/zfs-create.8 +++ b/man/man8/zfs-create.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -29,7 +29,7 @@ .\" Copyright 2018 Nexenta Systems, Inc. .\" Copyright 2019 Joyent, Inc. .\" -.Dd December 1, 2020 +.Dd March 16, 2022 .Dt ZFS-CREATE 8 .Os . @@ -243,6 +243,39 @@ enable the swap area using the command. Swapping to files on ZFS filesystems is not supported. . +.Sh EXAMPLES +.\" These are, respectively, examples 1, 10 from zfs.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Creating a ZFS File System Hierarchy +The following commands create a file system named +.Ar pool/home +and a file system named +.Ar pool/home/bob . +The mount point +.Pa /export/home +is set for the parent file system, and is automatically inherited by the child +file system. +.Dl # Nm zfs Cm create Ar pool/home +.Dl # Nm zfs Cm set Sy mountpoint Ns = Ns Ar /export/home pool/home +.Dl # Nm zfs Cm create Ar pool/home/bob +. +.Ss Example 2 : No Promoting a ZFS Clone +The following commands illustrate how to test out changes to a file system, and +then replace the original file system with the changed one, using clones, clone +promotion, and renaming: +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm create Ar pool/project/production + populate /pool/project/production with data +.No # Nm zfs Cm snapshot Ar pool/project/production Ns @ Ns Ar today +.No # Nm zfs Cm clone Ar pool/project/production@today pool/project/beta + make changes to /pool/project/beta and test them +.No # Nm zfs Cm promote Ar pool/project/beta +.No # Nm zfs Cm rename Ar pool/project/production pool/project/legacy +.No # Nm zfs Cm rename Ar pool/project/beta pool/project/production + once the legacy version is no longer needed, it can be destroyed +.No # Nm zfs Cm destroy Ar pool/project/legacy +.Ed +. .Sh SEE ALSO .Xr zfs-destroy 8 , .Xr zfs-list 8 , diff --git a/man/man8/zfs-destroy.8 b/man/man8/zfs-destroy.8 index 51d9b7ab8e7d..fe4f19d18e9f 100644 --- a/man/man8/zfs-destroy.8 +++ b/man/man8/zfs-destroy.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -29,7 +29,7 @@ .\" Copyright 2018 Nexenta Systems, Inc. .\" Copyright 2019 Joyent, Inc. .\" -.Dd June 30, 2019 +.Dd March 16, 2022 .Dt ZFS-DESTROY 8 .Os . @@ -173,6 +173,54 @@ behavior for mounted file systems in use. The given bookmark is destroyed. .El . +.Sh EXAMPLES +.\" These are, respectively, examples 3, 10, 15 from zfs.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Creating and Destroying Multiple Snapshots +The following command creates snapshots named +.Ar yesterday No of Ar pool/home +and all of its descendent file systems. +Each snapshot is mounted on demand in the +.Pa .zfs/snapshot +directory at the root of its file system. +The second command destroys the newly created snapshots. +.Dl # Nm zfs Cm snapshot Fl r Ar pool/home Ns @ Ns Ar yesterday +.Dl # Nm zfs Cm destroy Fl r Ar pool/home Ns @ Ns Ar yesterday +. +.Ss Example 2 : No Promoting a ZFS Clone +The following commands illustrate how to test out changes to a file system, and +then replace the original file system with the changed one, using clones, clone +promotion, and renaming: +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm create Ar pool/project/production + populate /pool/project/production with data +.No # Nm zfs Cm snapshot Ar pool/project/production Ns @ Ns Ar today +.No # Nm zfs Cm clone Ar pool/project/production@today pool/project/beta + make changes to /pool/project/beta and test them +.No # Nm zfs Cm promote Ar pool/project/beta +.No # Nm zfs Cm rename Ar pool/project/production pool/project/legacy +.No # Nm zfs Cm rename Ar pool/project/beta pool/project/production + once the legacy version is no longer needed, it can be destroyed +.No # Nm zfs Cm destroy Ar pool/project/legacy +.Ed +. +.Ss Example 3 : No Performing a Rolling Snapshot +The following example shows how to maintain a history of snapshots with a +consistent naming scheme. +To keep a week's worth of snapshots, the user destroys the oldest snapshot, +renames the remaining snapshots, and then creates a new snapshot, as follows: +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm destroy Fl r Ar pool/users@7daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@6daysago No @ Ns Ar 7daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@5daysago No @ Ns Ar 6daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@4daysago No @ Ns Ar 5daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@3daysago No @ Ns Ar 4daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@2daysago No @ Ns Ar 3daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@yesterday No @ Ns Ar 2daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@today No @ Ns Ar yesterday +.No # Nm zfs Cm snapshot Fl r Ar pool/users Ns @ Ns Ar today +.Ed +. .Sh SEE ALSO .Xr zfs-create 8 , .Xr zfs-hold 8 diff --git a/man/man8/zfs-diff.8 b/man/man8/zfs-diff.8 index a347f325203e..16aa20969f91 100644 --- a/man/man8/zfs-diff.8 +++ b/man/man8/zfs-diff.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -29,7 +29,7 @@ .\" Copyright 2018 Nexenta Systems, Inc. .\" Copyright 2019 Joyent, Inc. .\" -.Dd May 29, 2021 +.Dd March 16, 2022 .Dt ZFS-DIFF 8 .Os . @@ -98,5 +98,24 @@ Do not non-ASCII paths. .El . +.Sh EXAMPLES +.\" These are, respectively, examples 22 from zfs.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Showing the differences between a snapshot and a ZFS Dataset +The following example shows how to see what has changed between a prior +snapshot of a ZFS dataset and its current state. +The +.Fl F +option is used to indicate type information for the files affected. +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm diff Fl F Ar tank/test@before tank/test +M / /tank/test/ +M F /tank/test/linked (+1) +R F /tank/test/oldname -> /tank/test/newname +- F /tank/test/deleted ++ F /tank/test/created +M F /tank/test/modified +.Ed +. .Sh SEE ALSO .Xr zfs-snapshot 8 diff --git a/man/man8/zfs-hold.8 b/man/man8/zfs-hold.8 index 5e4652092e80..c2c7b7dc26a3 100644 --- a/man/man8/zfs-hold.8 +++ b/man/man8/zfs-hold.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zfs-jail.8 b/man/man8/zfs-jail.8 index 4f9faaea9bf5..72e83f7fe3eb 100644 --- a/man/man8/zfs-jail.8 +++ b/man/man8/zfs-jail.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zfs-list.8 b/man/man8/zfs-list.8 index 5200483868ff..9f6a73ab956d 100644 --- a/man/man8/zfs-list.8 +++ b/man/man8/zfs-list.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -29,7 +29,7 @@ .\" Copyright 2018 Nexenta Systems, Inc. .\" Copyright 2019 Joyent, Inc. .\" -.Dd May 27, 2021 +.Dd March 16, 2022 .Dt ZFS-LIST 8 .Os . @@ -70,10 +70,6 @@ The following fields are displayed: Used for scripting mode. Do not print headers and separate fields by a single tab instead of arbitrary white space. -.It Fl S Ar property -Same as the -.Fl s -option, but sorts by property in descending order. .It Fl d Ar depth Recursively display any children of the dataset, limiting the recursion to .Ar depth . @@ -142,6 +138,10 @@ the specified ordering. If no sorting options are specified the existing behavior of .Nm zfs Cm list is preserved. +.It Fl S Ar property +Same as +.Fl s , +but sorts by property in descending order. .It Fl t Ar type A comma-separated list of types to display, where .Ar type @@ -157,6 +157,27 @@ For example, specifying displays only snapshots. .El . +.Sh EXAMPLES +.\" These are, respectively, examples 5 from zfs.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Listing ZFS Datasets +The following command lists all active file systems and volumes in the system. +Snapshots are displayed if +.Sy listsnaps Ns = Ns Sy on . +The default is +.Sy off . +See +.Xr zpoolprops 7 +for more information on pool properties. +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm list +NAME USED AVAIL REFER MOUNTPOINT +pool 450K 457G 18K /pool +pool/home 315K 457G 21K /export/home +pool/home/anne 18K 457G 18K /export/home/anne +pool/home/bob 276K 457G 276K /export/home/bob +.Ed +. .Sh SEE ALSO .Xr zfsprops 7 , .Xr zfs-get 8 diff --git a/man/man8/zfs-load-key.8 b/man/man8/zfs-load-key.8 index b12a79e0150a..98682c95a0a1 100644 --- a/man/man8/zfs-load-key.8 +++ b/man/man8/zfs-load-key.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zfs-mount.8 b/man/man8/zfs-mount.8 index 42ce6b5ca155..873190eb8622 100644 --- a/man/man8/zfs-mount.8 +++ b/man/man8/zfs-mount.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zfs-program.8 b/man/man8/zfs-program.8 index 4a9718cdcfcb..06415b2190eb 100644 --- a/man/man8/zfs-program.8 +++ b/man/man8/zfs-program.8 @@ -73,7 +73,7 @@ The default limit is 10 million instructions, and it can be set to a maximum of Memory limit, in bytes. If a channel program attempts to allocate more memory than the given limit, it will be stopped and an error returned. -The default memory limit is 10 MB, and can be set to a maximum of 100 MB. +The default memory limit is 10 MiB, and can be set to a maximum of 100 MiB. .El .Pp All remaining argument strings will be passed directly to the Lua script as @@ -96,16 +96,17 @@ argv = args["argv"] -- argv == {1="arg1", 2="arg2", ...} .Ed .Pp -If invoked from the libZFS interface, an arbitrary argument list can be +If invoked from the libzfs interface, an arbitrary argument list can be passed to the channel program, which is accessible via the same -"..." syntax in Lua: +.Qq Li ... +syntax in Lua: .Bd -literal -compact -offset indent args = ... -- args == {"foo"="bar", "baz"={...}, ...} .Ed .Pp Note that because Lua arrays are 1-indexed, arrays passed to Lua from the -libZFS interface will have their indices incremented by 1. +libzfs interface will have their indices incremented by 1. That is, the element in .Va arr[0] @@ -166,7 +167,7 @@ See the section below for function-specific details on error return codes. . .Ss Lua to C Value Conversion -When invoking a channel program via the libZFS interface, it is necessary to +When invoking a channel program via the libzfs interface, it is necessary to translate arguments and return values from Lua values to their C equivalents, and vice-versa. .Pp diff --git a/man/man8/zfs-project.8 b/man/man8/zfs-project.8 index 0edaad0820d7..1c3abc76c4a1 100644 --- a/man/man8/zfs-project.8 +++ b/man/man8/zfs-project.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zfs-promote.8 b/man/man8/zfs-promote.8 index ba8cd5f6da80..1c14c3a0bc9d 100644 --- a/man/man8/zfs-promote.8 +++ b/man/man8/zfs-promote.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -29,7 +29,7 @@ .\" Copyright 2018 Nexenta Systems, Inc. .\" Copyright 2019 Joyent, Inc. .\" -.Dd June 30, 2019 +.Dd March 16, 2022 .Dt ZFS-PROMOTE 8 .Os . @@ -59,6 +59,26 @@ The .Nm zfs Cm rename subcommand can be used to rename any conflicting snapshots. . +.Sh EXAMPLES +.\" These are, respectively, examples 10 from zfs.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Promoting a ZFS Clone +The following commands illustrate how to test out changes to a file system, and +then replace the original file system with the changed one, using clones, clone +promotion, and renaming: +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm create Ar pool/project/production + populate /pool/project/production with data +.No # Nm zfs Cm snapshot Ar pool/project/production Ns @ Ns Ar today +.No # Nm zfs Cm clone Ar pool/project/production@today pool/project/beta + make changes to /pool/project/beta and test them +.No # Nm zfs Cm promote Ar pool/project/beta +.No # Nm zfs Cm rename Ar pool/project/production pool/project/legacy +.No # Nm zfs Cm rename Ar pool/project/beta pool/project/production + once the legacy version is no longer needed, it can be destroyed +.No # Nm zfs Cm destroy Ar pool/project/legacy +.Ed +. .Sh SEE ALSO .Xr zfs-clone 8 , .Xr zfs-rename 8 diff --git a/man/man8/zfs-receive.8 b/man/man8/zfs-receive.8 index fc1e943bd61b..22cb567c1c96 100644 --- a/man/man8/zfs-receive.8 +++ b/man/man8/zfs-receive.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -29,7 +29,7 @@ .\" Copyright 2018 Nexenta Systems, Inc. .\" Copyright 2019 Joyent, Inc. .\" -.Dd February 16, 2020 +.Dd April 26, 2022 .Dt ZFS-RECEIVE 8 .Os . @@ -57,6 +57,12 @@ .Fl A .Ar filesystem Ns | Ns Ar volume . +.Nm +.Cm receive +.Fl c +.Op Fl vn +.Ar filesystem Ns | Ns Ar snapshot +. .Sh DESCRIPTION .Bl -tag -width "" .It Xo @@ -393,8 +399,68 @@ restrictions (e.g. set-once) apply equally to Abort an interrupted .Nm zfs Cm receive Fl s , deleting its saved partially received state. +.It Xo +.Nm zfs +.Cm receive +.Fl c +.Op Fl vn +.Ar filesystem Ns | Ns Ar snapshot +.Xc +Attempt to correct data corruption in the specified dataset, +by using the provided stream as the source of healthy data. +This method of healing can only heal data blocks present in the stream. +Metadata can not be healed by corrective receive. +Running a scrub is recommended post-healing to ensure all corruption was +healed. +.Pp +It's important to consider why corruption has happened in the first place +since if you have slowly failing hardware periodically healing the data +is not going to save you from data loss later on when the hardware fails +completely. .El . +.Sh EXAMPLES +.\" These are, respectively, examples 12, 13 from zfs.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Remotely Replicating ZFS Data +The following commands send a full stream and then an incremental stream to a +remote machine, restoring them into +.Em poolB/received/fs@a +and +.Em poolB/received/fs@b , +respectively. +.Em poolB +must contain the file system +.Em poolB/received , +and must not initially contain +.Em poolB/received/fs . +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm send Ar pool/fs@a | +.No " " Nm ssh Ar host Nm zfs Cm receive Ar poolB/received/fs Ns @ Ns Ar a +.No # Nm zfs Cm send Fl i Ar a pool/fs@b | +.No " " Nm ssh Ar host Nm zfs Cm receive Ar poolB/received/fs +.Ed +. +.Ss Example 2 : No Using the Nm zfs Cm receive Fl d No Option +The following command sends a full stream of +.Ar poolA/fsA/fsB@snap +to a remote machine, receiving it into +.Ar poolB/received/fsA/fsB@snap . +The +.Ar fsA/fsB@snap +portion of the received snapshot's name is determined from the name of the sent +snapshot. +.Ar poolB +must contain the file system +.Ar poolB/received . +If +.Ar poolB/received/fsA +does not exist, it is created as an empty file system. +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm send Ar poolA/fsA/fsB@snap | +.No " " Nm ssh Ar host Nm zfs Cm receive Fl d Ar poolB/received +.Ed +. .Sh SEE ALSO .Xr zfs-send 8 , .Xr zstream 8 diff --git a/man/man8/zfs-rename.8 b/man/man8/zfs-rename.8 index 6caee50657ee..cb1dfc995d5b 100644 --- a/man/man8/zfs-rename.8 +++ b/man/man8/zfs-rename.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -29,7 +29,7 @@ .\" Copyright 2018 Nexenta Systems, Inc. .\" Copyright 2019 Joyent, Inc. .\" -.Dd September 1, 2020 +.Dd March 16, 2022 .Dt ZFS-RENAME 8 .Os . @@ -121,3 +121,40 @@ the file system is not unmounted even if this option is not given. Recursively rename the snapshots of all descendent datasets. Snapshots are the only dataset that can be renamed recursively. .El +. +.Sh EXAMPLES +.\" These are, respectively, examples 10, 15 from zfs.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Promoting a ZFS Clone +The following commands illustrate how to test out changes to a file system, and +then replace the original file system with the changed one, using clones, clone +promotion, and renaming: +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm create Ar pool/project/production + populate /pool/project/production with data +.No # Nm zfs Cm snapshot Ar pool/project/production Ns @ Ns Ar today +.No # Nm zfs Cm clone Ar pool/project/production@today pool/project/beta + make changes to /pool/project/beta and test them +.No # Nm zfs Cm promote Ar pool/project/beta +.No # Nm zfs Cm rename Ar pool/project/production pool/project/legacy +.No # Nm zfs Cm rename Ar pool/project/beta pool/project/production + once the legacy version is no longer needed, it can be destroyed +.No # Nm zfs Cm destroy Ar pool/project/legacy +.Ed +. +.Ss Example 2 : No Performing a Rolling Snapshot +The following example shows how to maintain a history of snapshots with a +consistent naming scheme. +To keep a week's worth of snapshots, the user destroys the oldest snapshot, +renames the remaining snapshots, and then creates a new snapshot, as follows: +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm destroy Fl r Ar pool/users@7daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@6daysago No @ Ns Ar 7daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@5daysago No @ Ns Ar 6daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@4daysago No @ Ns Ar 5daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@3daysago No @ Ns Ar 4daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@2daysago No @ Ns Ar 3daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@yesterday No @ Ns Ar 2daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@today No @ Ns Ar yesterday +.No # Nm zfs Cm snapshot Fl r Ar pool/users Ns @ Ns Ar today +.Ed diff --git a/man/man8/zfs-rollback.8 b/man/man8/zfs-rollback.8 index 08e914b47659..c12c564ef671 100644 --- a/man/man8/zfs-rollback.8 +++ b/man/man8/zfs-rollback.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -29,7 +29,7 @@ .\" Copyright 2018 Nexenta Systems, Inc. .\" Copyright 2019 Joyent, Inc. .\" -.Dd May 27, 2021 +.Dd March 16, 2022 .Dt ZFS-ROLLBACK 8 .Os . @@ -71,5 +71,16 @@ option to force an unmount of any clone file systems that are to be destroyed. Destroy any snapshots and bookmarks more recent than the one specified. .El . +.Sh EXAMPLES +.\" These are, respectively, examples 8 from zfs.8 +.\" Make sure to update them bidirectionally +.Ss Example 8 : No Rolling Back a ZFS File System +The following command reverts the contents of +.Ar pool/home/anne +to the snapshot named +.Ar yesterday , +deleting all intermediate snapshots: +.Dl # Nm zfs Cm rollback Fl r Ar pool/home/anne Ns @ Ns Ar yesterday +. .Sh SEE ALSO .Xr zfs-snapshot 8 diff --git a/man/man8/zfs-send.8 b/man/man8/zfs-send.8 index e83a92e4b341..b033e15a1d81 100644 --- a/man/man8/zfs-send.8 +++ b/man/man8/zfs-send.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -29,7 +29,7 @@ .\" Copyright 2018 Nexenta Systems, Inc. .\" Copyright 2019 Joyent, Inc. .\" -.Dd April 15, 2021 +.Dd March 16, 2022 .Dt ZFS-SEND 8 .Os . @@ -39,7 +39,8 @@ .Sh SYNOPSIS .Nm zfs .Cm send -.Op Fl DLPRbcehnpsvw +.Op Fl DLPbcehnpsvw +.Op Fl R Op Fl X Ar dataset Ns Oo , Ns Ar dataset Oc Ns … .Op Oo Fl I Ns | Ns Fl i Oc Ar snapshot .Ar snapshot .Nm zfs @@ -72,7 +73,8 @@ .It Xo .Nm zfs .Cm send -.Op Fl DLPRbcehnpvw +.Op Fl DLPbcehnpsvw +.Op Fl R Op Fl X Ar dataset Ns Oo , Ns Ar dataset Oc Ns … .Op Oo Fl I Ns | Ns Fl i Oc Ar snapshot .Ar snapshot .Xc @@ -100,12 +102,12 @@ The incremental source may be specified as with the .Fl i option. .It Fl L , -large-block -Generate a stream which may contain blocks larger than 128KB. +Generate a stream which may contain blocks larger than 128 KiB. This flag has no effect if the .Sy large_blocks pool feature is disabled, or if the .Sy recordsize -property of this filesystem has never been set above 128KB. +property of this filesystem has never been set above 128 KiB. The receiving system must have the .Sy large_blocks pool feature enabled as well. @@ -140,6 +142,16 @@ If the flag is used to send encrypted datasets, then .Fl w must also be specified. +.It Fl X , -exclude Ar dataset Ns Oo , Ns Ar dataset Oc Ns … +With +.Fl R , +.Fl X +specifies a set of datasets (and, hence, their descendants), +to be excluded from the send stream. +The root dataset may not be excluded. +.Fl X Ar a Fl X Ar b +is equivalent to +.Fl X Ar a , Ns Ar b . .It Fl e , -embed Generate a more compact stream by using .Sy WRITE_EMBEDDED @@ -305,12 +317,12 @@ Deduplicated send is no longer supported. This flag is accepted for backwards compatibility, but a regular, non-deduplicated stream will be generated. .It Fl L , -large-block -Generate a stream which may contain blocks larger than 128KB. +Generate a stream which may contain blocks larger than 128 KiB. This flag has no effect if the .Sy large_blocks pool feature is disabled, or if the .Sy recordsize -property of this filesystem has never been set above 128KB. +property of this filesystem has never been set above 128 KiB. The receiving system must have the .Sy large_blocks pool feature enabled as well. @@ -648,6 +660,48 @@ ones on the source, and are ready to be used, while the parent snapshot on the target contains none of the username and password data present on the source, because it was removed by the redacted send operation. . +.Sh EXAMPLES +.\" These are, respectively, examples 12, 13 from zfs.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Remotely Replicating ZFS Data +The following commands send a full stream and then an incremental stream to a +remote machine, restoring them into +.Em poolB/received/fs@a +and +.Em poolB/received/fs@b , +respectively. +.Em poolB +must contain the file system +.Em poolB/received , +and must not initially contain +.Em poolB/received/fs . +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm send Ar pool/fs@a | +.No " " Nm ssh Ar host Nm zfs Cm receive Ar poolB/received/fs Ns @ Ns Ar a +.No # Nm zfs Cm send Fl i Ar a pool/fs@b | +.No " " Nm ssh Ar host Nm zfs Cm receive Ar poolB/received/fs +.Ed +. +.Ss Example 2 : No Using the Nm zfs Cm receive Fl d No Option +The following command sends a full stream of +.Ar poolA/fsA/fsB@snap +to a remote machine, receiving it into +.Ar poolB/received/fsA/fsB@snap . +The +.Ar fsA/fsB@snap +portion of the received snapshot's name is determined from the name of the sent +snapshot. +.Ar poolB +must contain the file system +.Ar poolB/received . +If +.Ar poolB/received/fsA +does not exist, it is created as an empty file system. +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm send Ar poolA/fsA/fsB@snap | +.No " " Nm ssh Ar host Nm zfs Cm receive Fl d Ar poolB/received +.Ed +. .Sh SEE ALSO .Xr zfs-bookmark 8 , .Xr zfs-receive 8 , diff --git a/man/man8/zfs-set.8 b/man/man8/zfs-set.8 index a3588cc26638..4cabdcd8bd83 100644 --- a/man/man8/zfs-set.8 +++ b/man/man8/zfs-set.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -29,7 +29,7 @@ .\" Copyright 2018 Nexenta Systems, Inc. .\" Copyright 2019 Joyent, Inc. .\" -.Dd June 2, 2021 +.Dd March 16, 2022 .Dt ZFS-SET 8 .Os . @@ -170,13 +170,140 @@ inherited. .It Fl r Recursively inherit the given property for all children. .It Fl S -Revert the property to the received value if one exists; otherwise operate as -if the +Revert the property to the received value, if one exists; +otherwise, for non-inheritable properties, to the default; +otherwise, operate as if the .Fl S option was not specified. .El .El . +.Sh EXAMPLES +.\" These are, respectively, examples 1, 4, 6, 7, 11, 14, 16 from zfs.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Creating a ZFS File System Hierarchy +The following commands create a file system named +.Ar pool/home +and a file system named +.Ar pool/home/bob . +The mount point +.Pa /export/home +is set for the parent file system, and is automatically inherited by the child +file system. +.Dl # Nm zfs Cm create Ar pool/home +.Dl # Nm zfs Cm set Sy mountpoint Ns = Ns Ar /export/home pool/home +.Dl # Nm zfs Cm create Ar pool/home/bob +. +.Ss Example 2 : No Disabling and Enabling File System Compression +The following command disables the +.Sy compression +property for all file systems under +.Ar pool/home . +The next command explicitly enables +.Sy compression +for +.Ar pool/home/anne . +.Dl # Nm zfs Cm set Sy compression Ns = Ns Sy off Ar pool/home +.Dl # Nm zfs Cm set Sy compression Ns = Ns Sy on Ar pool/home/anne +. +.Ss Example 3 : No Setting a Quota on a ZFS File System +The following command sets a quota of 50 Gbytes for +.Ar pool/home/bob : +.Dl # Nm zfs Cm set Sy quota Ns = Ns Ar 50G pool/home/bob +. +.Ss Example 4 : No Listing ZFS Properties +The following command lists all properties for +.Ar pool/home/bob : +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm get Sy all Ar pool/home/bob +NAME PROPERTY VALUE SOURCE +pool/home/bob type filesystem - +pool/home/bob creation Tue Jul 21 15:53 2009 - +pool/home/bob used 21K - +pool/home/bob available 20.0G - +pool/home/bob referenced 21K - +pool/home/bob compressratio 1.00x - +pool/home/bob mounted yes - +pool/home/bob quota 20G local +pool/home/bob reservation none default +pool/home/bob recordsize 128K default +pool/home/bob mountpoint /pool/home/bob default +pool/home/bob sharenfs off default +pool/home/bob checksum on default +pool/home/bob compression on local +pool/home/bob atime on default +pool/home/bob devices on default +pool/home/bob exec on default +pool/home/bob setuid on default +pool/home/bob readonly off default +pool/home/bob zoned off default +pool/home/bob snapdir hidden default +pool/home/bob acltype off default +pool/home/bob aclmode discard default +pool/home/bob aclinherit restricted default +pool/home/bob canmount on default +pool/home/bob xattr on default +pool/home/bob copies 1 default +pool/home/bob version 4 - +pool/home/bob utf8only off - +pool/home/bob normalization none - +pool/home/bob casesensitivity sensitive - +pool/home/bob vscan off default +pool/home/bob nbmand off default +pool/home/bob sharesmb off default +pool/home/bob refquota none default +pool/home/bob refreservation none default +pool/home/bob primarycache all default +pool/home/bob secondarycache all default +pool/home/bob usedbysnapshots 0 - +pool/home/bob usedbydataset 21K - +pool/home/bob usedbychildren 0 - +pool/home/bob usedbyrefreservation 0 - +.Ed +.Pp +The following command gets a single property value: +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm get Fl H o Sy value compression Ar pool/home/bob +on +.Ed +.Pp +The following command lists all properties with local settings for +.Ar pool/home/bob : +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm get Fl r s Sy local Fl o Sy name , Ns Sy property , Ns Sy value all Ar pool/home/bob +NAME PROPERTY VALUE +pool/home/bob quota 20G +pool/home/bob compression on +.Ed +. +.Ss Example 5 : No Inheriting ZFS Properties +The following command causes +.Ar pool/home/bob No and Ar pool/home/anne +to inherit the +.Sy checksum +property from their parent. +.Dl # Nm zfs Cm inherit Sy checksum Ar pool/home/bob pool/home/anne +. +.Ss Example 6 : No Setting User Properties +The following example sets the user-defined +.Ar com.example : Ns Ar department +property for a dataset: +.Dl # Nm zfs Cm set Ar com.example : Ns Ar department Ns = Ns Ar 12345 tank/accounting +. +.Ss Example 7 : No Setting sharenfs Property Options on a ZFS File System +The following commands show how to set +.Sy sharenfs +property options to enable read-write +access for a set of IP addresses and to enable root access for system +.Qq neo +on the +.Ar tank/home +file system: +.Dl # Nm zfs Cm set Sy sharenfs Ns = Ns ' Ns Ar rw Ns =@123.123.0.0/16:[::1],root= Ns Ar neo Ns ' tank/home +.Pp +If you are using DNS for host name resolution, +specify the fully-qualified hostname. +. .Sh SEE ALSO .Xr zfsprops 7 , .Xr zfs-list 8 diff --git a/man/man8/zfs-share.8 b/man/man8/zfs-share.8 index 89121ead0958..52087f0b9220 100644 --- a/man/man8/zfs-share.8 +++ b/man/man8/zfs-share.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zfs-snapshot.8 b/man/man8/zfs-snapshot.8 index 225123f44b2b..864c65c7002b 100644 --- a/man/man8/zfs-snapshot.8 +++ b/man/man8/zfs-snapshot.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -29,7 +29,7 @@ .\" Copyright 2018 Nexenta Systems, Inc. .\" Copyright 2019 Joyent, Inc. .\" -.Dd May 27, 2021 +.Dd March 16, 2022 .Dt ZFS-SNAPSHOT 8 .Os . @@ -65,6 +65,64 @@ for details. Recursively create snapshots of all descendent datasets .El . +.Sh EXAMPLES +.\" These are, respectively, examples 2, 3, 10, 15 from zfs.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Creating a ZFS Snapshot +The following command creates a snapshot named +.Ar yesterday . +This snapshot is mounted on demand in the +.Pa .zfs/snapshot +directory at the root of the +.Ar pool/home/bob +file system. +.Dl # Nm zfs Cm snapshot Ar pool/home/bob Ns @ Ns Ar yesterday +. +.Ss Example 2 : No Creating and Destroying Multiple Snapshots +The following command creates snapshots named +.Ar yesterday No of Ar pool/home +and all of its descendent file systems. +Each snapshot is mounted on demand in the +.Pa .zfs/snapshot +directory at the root of its file system. +The second command destroys the newly created snapshots. +.Dl # Nm zfs Cm snapshot Fl r Ar pool/home Ns @ Ns Ar yesterday +.Dl # Nm zfs Cm destroy Fl r Ar pool/home Ns @ Ns Ar yesterday +. +.Ss Example 3 : No Promoting a ZFS Clone +The following commands illustrate how to test out changes to a file system, and +then replace the original file system with the changed one, using clones, clone +promotion, and renaming: +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm create Ar pool/project/production + populate /pool/project/production with data +.No # Nm zfs Cm snapshot Ar pool/project/production Ns @ Ns Ar today +.No # Nm zfs Cm clone Ar pool/project/production@today pool/project/beta + make changes to /pool/project/beta and test them +.No # Nm zfs Cm promote Ar pool/project/beta +.No # Nm zfs Cm rename Ar pool/project/production pool/project/legacy +.No # Nm zfs Cm rename Ar pool/project/beta pool/project/production + once the legacy version is no longer needed, it can be destroyed +.No # Nm zfs Cm destroy Ar pool/project/legacy +.Ed +. +.Ss Example 4 : No Performing a Rolling Snapshot +The following example shows how to maintain a history of snapshots with a +consistent naming scheme. +To keep a week's worth of snapshots, the user destroys the oldest snapshot, +renames the remaining snapshots, and then creates a new snapshot, as follows: +.Bd -literal -compact -offset Ds +.No # Nm zfs Cm destroy Fl r Ar pool/users@7daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@6daysago No @ Ns Ar 7daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@5daysago No @ Ns Ar 6daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@4daysago No @ Ns Ar 5daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@3daysago No @ Ns Ar 4daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@2daysago No @ Ns Ar 3daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@yesterday No @ Ns Ar 2daysago +.No # Nm zfs Cm rename Fl r Ar pool/users@today No @ Ns Ar yesterday +.No # Nm zfs Cm snapshot Fl r Ar pool/users Ns @ Ns Ar today +.Ed +. .Sh SEE ALSO .Xr zfs-bookmark 8 , .Xr zfs-clone 8 , diff --git a/man/man8/zfs-unzone.8 b/man/man8/zfs-unzone.8 new file mode 120000 index 000000000000..9052b28aa880 --- /dev/null +++ b/man/man8/zfs-unzone.8 @@ -0,0 +1 @@ +zfs-zone.8 \ No newline at end of file diff --git a/man/man8/zfs-upgrade.8 b/man/man8/zfs-upgrade.8 index f3620faa6135..2744b07424d5 100644 --- a/man/man8/zfs-upgrade.8 +++ b/man/man8/zfs-upgrade.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zfs-userspace.8 b/man/man8/zfs-userspace.8 index b7bd61b5709a..bc6b607e2911 100644 --- a/man/man8/zfs-userspace.8 +++ b/man/man8/zfs-userspace.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zfs-wait.8 b/man/man8/zfs-wait.8 index 81bc15636541..9d1f24f195b7 100644 --- a/man/man8/zfs-wait.8 +++ b/man/man8/zfs-wait.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zfs-zone.8 b/man/man8/zfs-zone.8 new file mode 100644 index 000000000000..0ee4409a2f17 --- /dev/null +++ b/man/man8/zfs-zone.8 @@ -0,0 +1,116 @@ +.\" +.\" CDDL HEADER START +.\" +.\" The contents of this file are subject to the terms of the +.\" Common Development and Distribution License (the "License"). +.\" You may not use this file except in compliance with the License. +.\" +.\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +.\" or https://opensource.org/licenses/CDDL-1.0. +.\" See the License for the specific language governing permissions +.\" and limitations under the License. +.\" +.\" When distributing Covered Code, include this CDDL HEADER in each +.\" file and include the License file at usr/src/OPENSOLARIS.LICENSE. +.\" If applicable, add the following below this CDDL HEADER, with the +.\" fields enclosed by brackets "[]" replaced with your own identifying +.\" information: Portions Copyright [yyyy] [name of copyright owner] +.\" +.\" CDDL HEADER END +.\" +.\" Copyright (c) 2009 Sun Microsystems, Inc. All Rights Reserved. +.\" Copyright 2011 Joshua M. Clulow +.\" Copyright (c) 2011, 2019 by Delphix. All rights reserved. +.\" Copyright (c) 2011, Pawel Jakub Dawidek +.\" Copyright (c) 2012, Glen Barber +.\" Copyright (c) 2012, Bryan Drewery +.\" Copyright (c) 2013, Steven Hartland +.\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved. +.\" Copyright (c) 2014, Joyent, Inc. All rights reserved. +.\" Copyright (c) 2014 by Adam Stevko. All rights reserved. +.\" Copyright (c) 2014 Integros [integros.com] +.\" Copyright (c) 2014, Xin LI +.\" Copyright (c) 2014-2015, The FreeBSD Foundation, All Rights Reserved. +.\" Copyright (c) 2016 Nexenta Systems, Inc. All Rights Reserved. +.\" Copyright 2019 Richard Laager. All rights reserved. +.\" Copyright 2018 Nexenta Systems, Inc. +.\" Copyright 2019 Joyent, Inc. +.\" Copyright 2021 Klara, Inc. +.\" +.Dd June 3, 2022 +.Dt ZFS-ZONE 8 +.Os +. +.Sh NAME +.Nm zfs-zone , +.Nm zfs-unzone +.Nd attach and detach ZFS filesystems to user namespaces +.Sh SYNOPSIS +.Nm zfs Cm zone +.Ar nsfile +.Ar filesystem +.Nm zfs Cm unzone +.Ar nsfile +.Ar filesystem +. +.Sh DESCRIPTION +.Bl -tag -width "" +.It Xo +.Nm zfs +.Cm zone +.Ar nsfile +.Ar filesystem +.Xc +Attach the specified +.Ar filesystem +to the user namespace identified by +.Ar nsfile . +From now on this file system tree can be managed from within a user namespace +if the +.Sy zoned +property has been set. +.Pp +You cannot attach a zoned dataset's children to another user namespace. +You can also not attach the root file system +of the user namespace or any dataset +which needs to be mounted before the zfs service +is run inside the user namespace, +as it would be attached unmounted until it is +mounted from the service inside the user namespace. +.Pp +To allow management of the dataset from within a user namespace, the +.Sy zoned +property has to be set and the user namespaces needs access to the +.Pa /dev/zfs +device. +The +.Sy quota +property cannot be changed from within a user namespace. +.Pp +After a dataset is attached to a user namespace and the +.Sy zoned +property is set, +a zoned file system cannot be mounted outside the user namespace, +since the user namespace administrator might have set the mount point +to an unacceptable value. +.It Xo +.Nm zfs +.Cm unzone +.Ar nsfile +.Ar filesystem +.Xc +Detach the specified +.Ar filesystem +from the user namespace identified by +.Ar nsfile . +.El +.Sh EXAMPLES +.Ss Example 1 : No Delegating a Dataset to a User Namespace +The following example delegates the +.Ar tank/users +dataset to a user namespace identified by user namespace file +.Pa /proc/1234/ns/user . +.Dl # Nm zfs Cm zone Ar /proc/1234/ns/user Ar tank/users +. +.Sh SEE ALSO +.Xr zfsprops 7 diff --git a/man/man8/zfs.8 b/man/man8/zfs.8 index 48453ef46c0d..52c07925764c 100644 --- a/man/man8/zfs.8 +++ b/man/man8/zfs.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -36,7 +36,7 @@ .\" Copyright 2018 Nexenta Systems, Inc. .\" Copyright 2019 Joyent, Inc. .\" -.Dd June 30, 2019 +.Dd May 12, 2022 .Dt ZFS 8 .Os . @@ -57,13 +57,32 @@ The .Nm command configures ZFS datasets within a ZFS storage pool, as described in .Xr zpool 8 . -A dataset is identified by a unique path within the ZFS namespace. -For example: -.Dl pool/{filesystem,volume,snapshot} +A dataset is identified by a unique path within the ZFS namespace: .Pp -where the maximum length of a dataset name is -.Sy MAXNAMELEN Pq 256B -and the maximum amount of nesting allowed in a path is 50 levels deep. +.D1 Ar pool Ns Oo Sy / Ns Ar component Oc Ns Sy / Ns Ar component +.Pp +for example: +.Pp +.Dl rpool/var/log +.Pp +The maximum length of a dataset name is +.Sy ZFS_MAX_DATASET_NAME_LEN No - 1 +ASCII characters (currently 255) satisfying +.Sy [A-Za-z_.:/ -] . +Additionally snapshots are allowed to contain a single +.Sy @ +character, while bookmarks are allowed to contain a single +.Sy # +character. +.Sy / +is used as separator between components. +The maximum amount of nesting allowed in a path is +.Sy zfs_max_dataset_nesting +levels deep. +ZFS tunables +.Pq Sy zfs_* +are explained in +.Xr zfs 4 . .Pp A dataset can be one of the following: .Bl -tag -offset Ds -width "file system" @@ -226,7 +245,7 @@ option reverted to the received value if one exists. Displays space consumed by, and quotas on, each user, group, or project in the specified filesystem or snapshot. .It Xr zfs-project 8 -List, set, or clear project ID and/or inherit flag on the file(s) or directories. +List, set, or clear project ID and/or inherit flag on the files or directories. .El . .Ss Mountpoints @@ -263,7 +282,8 @@ Add or change an encryption key on the specified dataset. .It Xr zfs-load-key 8 Load the key for the specified encrypted dataset, enabling access. .It Xr zfs-unload-key 8 -Unload a key for the specified dataset, removing the ability to access the dataset. +Unload a key for the specified dataset, +removing the ability to access the dataset. .El . .Ss Channel Programs @@ -299,9 +319,22 @@ if an error occurs, and if invalid command line options were specified. . .Sh EXAMPLES -.Bl -tag -width "" -. -.It Sy Example 1 : No Creating a ZFS File System Hierarchy +.\" Examples 1, 4, 6, 7, 11, 14, 16 are shared with zfs-set.8. +.\" Examples 1, 10 are shared with zfs-create.8. +.\" Examples 2, 3, 10, 15 are also shared with zfs-snapshot.8. +.\" Examples 3, 10, 15 are shared with zfs-destroy.8. +.\" Examples 5 are shared with zfs-list.8. +.\" Examples 8 are shared with zfs-rollback.8. +.\" Examples 9, 10 are shared with zfs-clone.8. +.\" Examples 10 are also shared with zfs-promote.8. +.\" Examples 10, 15 also are shared with zfs-rename.8. +.\" Examples 12, 13 are shared with zfs-send.8. +.\" Examples 12, 13 are also shared with zfs-receive.8. +.\" Examples 17, 18, 19, 20, 21 are shared with zfs-allow.8. +.\" Examples 22 are shared with zfs-diff.8. +.\" Examples 23 are shared with zfs-bookmark.8. +.\" Make sure to update them omnidirectionally +.Ss Example 1 : No Creating a ZFS File System Hierarchy The following commands create a file system named .Ar pool/home and a file system named @@ -314,7 +347,7 @@ file system. .Dl # Nm zfs Cm set Sy mountpoint Ns = Ns Ar /export/home pool/home .Dl # Nm zfs Cm create Ar pool/home/bob . -.It Sy Example 2 : No Creating a ZFS Snapshot +.Ss Example 2 : No Creating a ZFS Snapshot The following command creates a snapshot named .Ar yesterday . This snapshot is mounted on demand in the @@ -324,7 +357,7 @@ directory at the root of the file system. .Dl # Nm zfs Cm snapshot Ar pool/home/bob Ns @ Ns Ar yesterday . -.It Sy Example 3 : No Creating and Destroying Multiple Snapshots +.Ss Example 3 : No Creating and Destroying Multiple Snapshots The following command creates snapshots named .Ar yesterday No of Ar pool/home and all of its descendent file systems. @@ -335,7 +368,7 @@ The second command destroys the newly created snapshots. .Dl # Nm zfs Cm snapshot Fl r Ar pool/home Ns @ Ns Ar yesterday .Dl # Nm zfs Cm destroy Fl r Ar pool/home Ns @ Ns Ar yesterday . -.It Sy Example 4 : No Disabling and Enabling File System Compression +.Ss Example 4 : No Disabling and Enabling File System Compression The following command disables the .Sy compression property for all file systems under @@ -347,7 +380,7 @@ for .Dl # Nm zfs Cm set Sy compression Ns = Ns Sy off Ar pool/home .Dl # Nm zfs Cm set Sy compression Ns = Ns Sy on Ar pool/home/anne . -.It Sy Example 5 : No Listing ZFS Datasets +.Ss Example 5 : No Listing ZFS Datasets The following command lists all active file systems and volumes in the system. Snapshots are displayed if .Sy listsnaps Ns = Ns Sy on . @@ -365,12 +398,12 @@ pool/home/anne 18K 457G 18K /export/home/anne pool/home/bob 276K 457G 276K /export/home/bob .Ed . -.It Sy Example 6 : No Setting a Quota on a ZFS File System +.Ss Example 6 : No Setting a Quota on a ZFS File System The following command sets a quota of 50 Gbytes for .Ar pool/home/bob : .Dl # Nm zfs Cm set Sy quota Ns = Ns Ar 50G pool/home/bob . -.It Sy Example 7 : No Listing ZFS Properties +.Ss Example 7 : No Listing ZFS Properties The following command lists all properties for .Ar pool/home/bob : .Bd -literal -compact -offset Ds @@ -435,7 +468,7 @@ pool/home/bob quota 20G pool/home/bob compression on .Ed . -.It Sy Example 8 : No Rolling Back a ZFS File System +.Ss Example 8 : No Rolling Back a ZFS File System The following command reverts the contents of .Ar pool/home/anne to the snapshot named @@ -443,13 +476,13 @@ to the snapshot named deleting all intermediate snapshots: .Dl # Nm zfs Cm rollback Fl r Ar pool/home/anne Ns @ Ns Ar yesterday . -.It Sy Example 9 : No Creating a ZFS Clone +.Ss Example 9 : No Creating a ZFS Clone The following command creates a writable file system whose initial contents are the same as .Ar pool/home/bob@yesterday . .Dl # Nm zfs Cm clone Ar pool/home/bob@yesterday pool/clone . -.It Sy Example 10 : No Promoting a ZFS Clone +.Ss Example 10 : No Promoting a ZFS Clone The following commands illustrate how to test out changes to a file system, and then replace the original file system with the changed one, using clones, clone promotion, and renaming: @@ -466,7 +499,7 @@ promotion, and renaming: .No # Nm zfs Cm destroy Ar pool/project/legacy .Ed . -.It Sy Example 11 : No Inheriting ZFS Properties +.Ss Example 11 : No Inheriting ZFS Properties The following command causes .Ar pool/home/bob No and Ar pool/home/anne to inherit the @@ -474,7 +507,7 @@ to inherit the property from their parent. .Dl # Nm zfs Cm inherit Sy checksum Ar pool/home/bob pool/home/anne . -.It Sy Example 12 : No Remotely Replicating ZFS Data +.Ss Example 12 : No Remotely Replicating ZFS Data The following commands send a full stream and then an incremental stream to a remote machine, restoring them into .Em poolB/received/fs@a @@ -493,7 +526,7 @@ and must not initially contain .No " " Nm ssh Ar host Nm zfs Cm receive Ar poolB/received/fs .Ed . -.It Sy Example 13 : No Using the Nm zfs Cm receive Fl d No Option +.Ss Example 13 : No Using the Nm zfs Cm receive Fl d No Option The following command sends a full stream of .Ar poolA/fsA/fsB@snap to a remote machine, receiving it into @@ -513,13 +546,13 @@ does not exist, it is created as an empty file system. .No " " Nm ssh Ar host Nm zfs Cm receive Fl d Ar poolB/received .Ed . -.It Sy Example 14 : No Setting User Properties +.Ss Example 14 : No Setting User Properties The following example sets the user-defined .Ar com.example : Ns Ar department property for a dataset: .Dl # Nm zfs Cm set Ar com.example : Ns Ar department Ns = Ns Ar 12345 tank/accounting . -.It Sy Example 15 : No Performing a Rolling Snapshot +.Ss Example 15 : No Performing a Rolling Snapshot The following example shows how to maintain a history of snapshots with a consistent naming scheme. To keep a week's worth of snapshots, the user destroys the oldest snapshot, @@ -536,7 +569,7 @@ renames the remaining snapshots, and then creates a new snapshot, as follows: .No # Nm zfs Cm snapshot Fl r Ar pool/users Ns @ Ns Ar today .Ed . -.It Sy Example 16 : No Setting sharenfs Property Options on a ZFS File System +.Ss Example 16 : No Setting sharenfs Property Options on a ZFS File System The following commands show how to set .Sy sharenfs property options to enable read-write @@ -550,7 +583,7 @@ file system: If you are using DNS for host name resolution, specify the fully-qualified hostname. . -.It Sy Example 17 : No Delegating ZFS Administration Permissions on a ZFS Dataset +.Ss Example 17 : No Delegating ZFS Administration Permissions on a ZFS Dataset The following example shows how to set permissions so that user .Ar cindys can create, destroy, mount, and take snapshots on @@ -573,9 +606,9 @@ mount point permission is set to 755 by default, user will be unable to mount file systems under .Ar tank/cindys . Add an ACE similar to the following syntax to provide mount point access: -.Dl # Cm chmod No A+user: Ns Ar cindys Ns :add_subdirectory:allow Ar /tank/cindys +.Dl # Cm chmod No A+user : Ns Ar cindys Ns :add_subdirectory:allow Ar /tank/cindys . -.It Sy Example 18 : No Delegating Create Time Permissions on a ZFS Dataset +.Ss Example 18 : No Delegating Create Time Permissions on a ZFS Dataset The following example shows how to grant anyone in the group .Ar staff to create file systems in @@ -596,7 +629,7 @@ Local+Descendent permissions: group staff create,mount .Ed . -.It Sy Example 19 : No Defining and Granting a Permission Set on a ZFS Dataset +.Ss Example 19 : No Defining and Granting a Permission Set on a ZFS Dataset The following example shows how to define and grant a permission set on the .Ar tank/users file system. @@ -614,7 +647,7 @@ Local+Descendent permissions: group staff @pset .Ed . -.It Sy Example 20 : No Delegating Property Permissions on a ZFS Dataset +.Ss Example 20 : No Delegating Property Permissions on a ZFS Dataset The following example shows to grant the ability to set quotas and reservations on the .Ar users/home @@ -634,7 +667,7 @@ NAME PROPERTY VALUE SOURCE users/home/marks quota 10G local .Ed . -.It Sy Example 21 : No Removing ZFS Delegated Permissions on a ZFS Dataset +.Ss Example 21 : No Removing ZFS Delegated Permissions on a ZFS Dataset The following example shows how to remove the snapshot permission from the .Ar staff group on the @@ -653,7 +686,7 @@ Local+Descendent permissions: group staff @pset .Ed . -.It Sy Example 22 : No Showing the differences between a snapshot and a ZFS Dataset +.Ss Example 22 : No Showing the differences between a snapshot and a ZFS Dataset The following example shows how to see what has changed between a prior snapshot of a ZFS dataset and its current state. The @@ -669,12 +702,12 @@ R F /tank/test/oldname -> /tank/test/newname M F /tank/test/modified .Ed . -.It Sy Example 23 : No Creating a bookmark -The following example create a bookmark to a snapshot. -This bookmark can then be used instead of snapshot in send streams. +.Ss Example 23 : No Creating a bookmark +The following example creates a bookmark to a snapshot. +This bookmark can then be used instead of a snapshot in send streams. .Dl # Nm zfs Cm bookmark Ar rpool Ns @ Ns Ar snapshot rpool Ns # Ns Ar bookmark . -.It Sy Example 24 : No Setting Sy sharesmb No Property Options on a ZFS File System +.Ss Example 24 : No Setting Sy sharesmb No Property Options on a ZFS File System The following example show how to share SMB filesystem through ZFS. Note that a user and their password must be given. .Dl # Nm smbmount Ar //127.0.0.1/share_tmp /mnt/tmp Fl o No user=workgroup/turbo,password=obrut,uid=1000 @@ -701,10 +734,9 @@ in case you need to modify any options of the share afterwards. Do note that any changes done with the .Xr net 8 command will be undone if the share is ever unshared (like via a reboot). -.El . .Sh ENVIRONMENT VARIABLES -.Bl -tag -width "ZFS_MOUNT_HELPER" +.Bl -tag -width "ZFS_MODULE_TIMEOUT" .It Sy ZFS_MOUNT_HELPER Cause .Nm zfs Cm mount @@ -712,6 +744,28 @@ to use .Xr mount 8 to mount ZFS datasets. This option is provided for backwards compatibility with older ZFS versions. +. +.It Sy ZFS_SET_PIPE_MAX +Tells +.Nm zfs +to set the maximum pipe size for sends/recieves. +Disabled by default on Linux +due to an unfixed deadlock in Linux's pipe size handling code. +. +.\" Shared with zpool.8 +.It Sy ZFS_MODULE_TIMEOUT +Time, in seconds, to wait for +.Pa /dev/zfs +to appear. +Defaults to +.Sy 10 , +max +.Sy 600 Pq 10 minutes . +If +.Pf < Sy 0 , +wait forever; if +.Sy 0 , +don't wait. .El . .Sh INTERFACE STABILITY diff --git a/man/man8/zfs_ids_to_path.8 b/man/man8/zfs_ids_to_path.8 index d5b74678b29c..ba093d1cf913 100644 --- a/man/man8/zfs_ids_to_path.8 +++ b/man/man8/zfs_ids_to_path.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zgenhostid.8 b/man/man8/zgenhostid.8 index e157578cf0bb..f7f2bdb4ac53 100644 --- a/man/man8/zgenhostid.8 +++ b/man/man8/zgenhostid.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zinject.8 b/man/man8/zinject.8 index a29346929988..f69dc6890ff7 100644 --- a/man/man8/zinject.8 +++ b/man/man8/zinject.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -71,18 +71,18 @@ Force a vdev into the DEGRADED or FAULTED state. .Fl D Ar latency : Ns Ar lanes .Ar pool .Xc -Add an artificial delay to IO requests on a particular +Add an artificial delay to I/O requests on a particular device, such that the requests take a minimum of .Ar latency milliseconds to complete. Each delay has an associated number of .Ar lanes which defines the number of concurrent -IO requests that can be processed. +I/O requests that can be processed. .Pp For example, with a single lane delay of 10 ms .No (\& Ns Fl D Ar 10 : Ns Ar 1 ) , -the device will only be able to service a single IO request +the device will only be able to service a single I/O request at a time with each request taking 10 ms to complete. So, if only a single request is submitted every 10 ms, the average latency will be 10 ms; but if more than one request @@ -188,7 +188,7 @@ Each number is in hexadecimal, and only one block can be specified. .It Fl C Ar dvas Inject the given error only into specific DVAs. The mask should be specified as a list of 0-indexed DVAs separated by commas -.No (ex. Ar 0,2 Ns No ). +.No (e.g. Ar 0,2 Ns No ). This option is not applicable to logical data errors such as .Sy decompress and diff --git a/man/man8/zpool-add.8 b/man/man8/zpool-add.8 index 26cf33c5538c..8ccdcccc7b06 100644 --- a/man/man8/zpool-add.8 +++ b/man/man8/zpool-add.8 @@ -5,7 +5,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -25,7 +25,7 @@ .\" Copyright 2017 Nexenta Systems, Inc. .\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved. .\" -.Dd May 27, 2021 +.Dd March 16, 2022 .Dt ZPOOL-ADD 8 .Os . @@ -93,6 +93,29 @@ The only property supported at the moment is .Sy ashift . .El . +.Sh EXAMPLES +.\" These are, respectively, examples 5, 13 from zpool.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Adding a Mirror to a ZFS Storage Pool +The following command adds two mirrored disks to the pool +.Ar tank , +assuming the pool is already made up of two-way mirrors. +The additional space is immediately available to any datasets within the pool. +.Dl # Nm zpool Cm add Ar tank Sy mirror Pa sda sdb +. +.Ss Example 2 : No Adding Cache Devices to a ZFS Pool +The following command adds two disks for use as cache devices to a ZFS storage +pool: +.Dl # Nm zpool Cm add Ar pool Sy cache Pa sdc sdd +.Pp +Once added, the cache devices gradually fill with content from main memory. +Depending on the size of your cache devices, it could take over an hour for +them to fill. +Capacity and reads can be monitored using the +.Cm iostat +subcommand as follows: +.Dl # Nm zpool Cm iostat Fl v Ar pool 5 +. .Sh SEE ALSO .Xr zpool-attach 8 , .Xr zpool-import 8 , diff --git a/man/man8/zpool-attach.8 b/man/man8/zpool-attach.8 index 9dfa35a107d3..73535cbdf108 100644 --- a/man/man8/zpool-attach.8 +++ b/man/man8/zpool-attach.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool-checkpoint.8 b/man/man8/zpool-checkpoint.8 index d5add14aed2f..7900b323822f 100644 --- a/man/man8/zpool-checkpoint.8 +++ b/man/man8/zpool-checkpoint.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool-clear.8 b/man/man8/zpool-clear.8 index 0b256b28bd21..7b9d40c74ebd 100644 --- a/man/man8/zpool-clear.8 +++ b/man/man8/zpool-clear.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool-create.8 b/man/man8/zpool-create.8 index e902c770076d..76e5476ea047 100644 --- a/man/man8/zpool-create.8 +++ b/man/man8/zpool-create.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -27,7 +27,7 @@ .\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved. .\" Copyright (c) 2021, Colm Buckley .\" -.Dd June 2, 2021 +.Dd March 16, 2022 .Dt ZPOOL-CREATE 8 .Os . @@ -205,6 +205,38 @@ such as virtual machines or physical machines whose pools live on network block devices. .El . +.Sh EXAMPLES +.\" These are, respectively, examples 1, 2, 3, 4, 11, 12 from zpool.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Creating a RAID-Z Storage Pool +The following command creates a pool with a single raidz root vdev that +consists of six disks: +.Dl # Nm zpool Cm create Ar tank Sy raidz Pa sda sdb sdc sdd sde sdf +. +.Ss Example 2 : No Creating a Mirrored Storage Pool +The following command creates a pool with two mirrors, where each mirror +contains two disks: +.Dl # Nm zpool Cm create Ar tank Sy mirror Pa sda sdb Sy mirror Pa sdc sdd +. +.Ss Example 3 : No Creating a ZFS Storage Pool by Using Partitions +The following command creates a non-redundant pool using two disk partitions: +.Dl # Nm zpool Cm create Ar tank Pa sda1 sdb2 +. +.Ss Example 4 : No Creating a ZFS Storage Pool by Using Files +The following command creates a non-redundant pool using files. +While not recommended, a pool based on files can be useful for experimental +purposes. +.Dl # Nm zpool Cm create Ar tank Pa /path/to/file/a /path/to/file/b +. +.Ss Example 5 : No Managing Hot Spares +The following command creates a new pool with an available hot spare: +.Dl # Nm zpool Cm create Ar tank Sy mirror Pa sda sdb Sy spare Pa sdc +. +.Ss Example 6 : No Creating a ZFS Pool with Mirrored Separate Intent Logs +The following command creates a ZFS storage pool consisting of two, two-way +mirrors and mirrored log devices: +.Dl # Nm zpool Cm create Ar pool Sy mirror Pa sda sdb Sy mirror Pa sdc sdd Sy log mirror Pa sde sdf +. .Sh SEE ALSO .Xr zpool-destroy 8 , .Xr zpool-export 8 , diff --git a/man/man8/zpool-destroy.8 b/man/man8/zpool-destroy.8 index a2f6729c8a76..b1ac18c8b4fd 100644 --- a/man/man8/zpool-destroy.8 +++ b/man/man8/zpool-destroy.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -26,7 +26,7 @@ .\" Copyright 2017 Nexenta Systems, Inc. .\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved. .\" -.Dd May 31, 2021 +.Dd March 16, 2022 .Dt ZPOOL-DESTROY 8 .Os . @@ -46,3 +46,12 @@ This command tries to unmount any active datasets before destroying the pool. .It Fl f Forcefully unmount all active datasets. .El +. +.Sh EXAMPLES +.\" These are, respectively, examples 7 from zpool.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Destroying a ZFS Storage Pool +The following command destroys the pool +.Ar tank +and any datasets contained within: +.Dl # Nm zpool Cm destroy Fl f Ar tank diff --git a/man/man8/zpool-detach.8 b/man/man8/zpool-detach.8 index 952dd7882a90..3d5ca9f146ad 100644 --- a/man/man8/zpool-detach.8 +++ b/man/man8/zpool-detach.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool-events.8 b/man/man8/zpool-events.8 index 55f7babae2bd..80fb21878f9c 100644 --- a/man/man8/zpool-events.8 +++ b/man/man8/zpool-events.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool-export.8 b/man/man8/zpool-export.8 index a15291a1f598..3b0edcef5528 100644 --- a/man/man8/zpool-export.8 +++ b/man/man8/zpool-export.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -26,7 +26,7 @@ .\" Copyright 2017 Nexenta Systems, Inc. .\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved. .\" -.Dd February 16, 2020 +.Dd March 16, 2022 .Dt ZPOOL-EXPORT 8 .Os . @@ -68,5 +68,14 @@ is currently being used. This may lead to potential data corruption. .El . +.Sh EXAMPLES +.\" These are, respectively, examples 8 from zpool.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Exporting a ZFS Storage Pool +The following command exports the devices in pool +.Ar tank +so that they can be relocated or later imported: +.Dl # Nm zpool Cm export Ar tank +. .Sh SEE ALSO .Xr zpool-import 8 diff --git a/man/man8/zpool-get.8 b/man/man8/zpool-get.8 index 4ef9a1b5ec44..78a39b07d749 100644 --- a/man/man8/zpool-get.8 +++ b/man/man8/zpool-get.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool-history.8 b/man/man8/zpool-history.8 index 2a2d500b8b8c..c6aa8884e598 100644 --- a/man/man8/zpool-history.8 +++ b/man/man8/zpool-history.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool-import.8 b/man/man8/zpool-import.8 index 39b0e17ef586..dab6e1f55771 100644 --- a/man/man8/zpool-import.8 +++ b/man/man8/zpool-import.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -26,7 +26,7 @@ .\" Copyright 2017 Nexenta Systems, Inc. .\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved. .\" -.Dd August 9, 2019 +.Dd March 16, 2022 .Dt ZPOOL-IMPORT 8 .Os . @@ -71,7 +71,8 @@ .Xc Lists pools available to import. If the -.Fl d or +.Fl d +or .Fl c options are not specified, this command searches for devices using libblkid on Linux and geom on @@ -404,6 +405,30 @@ when not explicitly specified. .El .El . +.Sh EXAMPLES +.\" These are, respectively, examples 9 from zpool.8 +.\" Make sure to update them bidirectionally +.Ss Example 9 : No Importing a ZFS Storage Pool +The following command displays available pools, and then imports the pool +.Ar tank +for use on the system. +The results from this command are similar to the following: +.Bd -literal -compact -offset Ds +.No # Nm zpool Cm import + pool: tank + id: 15451357997522795478 + state: ONLINE +action: The pool can be imported using its name or numeric identifier. +config: + + tank ONLINE + mirror ONLINE + sda ONLINE + sdb ONLINE + +.No # Nm zpool Cm import Ar tank +.Ed +. .Sh SEE ALSO .Xr zpool-export 8 , .Xr zpool-list 8 , diff --git a/man/man8/zpool-initialize.8 b/man/man8/zpool-initialize.8 index 0a108180dbbe..eae711bff429 100644 --- a/man/man8/zpool-initialize.8 +++ b/man/man8/zpool-initialize.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool-iostat.8 b/man/man8/zpool-iostat.8 index 969c74cf398e..76fdee7d09c6 100644 --- a/man/man8/zpool-iostat.8 +++ b/man/man8/zpool-iostat.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -26,7 +26,7 @@ .\" Copyright 2017 Nexenta Systems, Inc. .\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved. .\" -.Dd May 27, 2021 +.Dd March 16, 2022 .Dt ZPOOL-IOSTAT 8 .Os . @@ -258,6 +258,46 @@ If you specify an interval, the measurements will be sampled from the end of the interval. .El . +.Sh EXAMPLES +.\" These are, respectively, examples 13, 16 from zpool.8 +.\" Make sure to update them bidirectionally +.Ss Example 13 : No Adding Cache Devices to a ZFS Pool +The following command adds two disks for use as cache devices to a ZFS storage +pool: +.Dl # Nm zpool Cm add Ar pool Sy cache Pa sdc sdd +.Pp +Once added, the cache devices gradually fill with content from main memory. +Depending on the size of your cache devices, it could take over an hour for +them to fill. +Capacity and reads can be monitored using the +.Cm iostat +subcommand as follows: +.Dl # Nm zpool Cm iostat Fl v Ar pool 5 +. +.Ss Example 16 : No Adding output columns +Additional columns can be added to the +.Nm zpool Cm status No and Nm zpool Cm iostat No output with Fl c . +.Bd -literal -compact -offset Ds +.No # Nm zpool Cm status Fl c Pa vendor , Ns Pa model , Ns Pa size + NAME STATE READ WRITE CKSUM vendor model size + tank ONLINE 0 0 0 + mirror-0 ONLINE 0 0 0 + U1 ONLINE 0 0 0 SEAGATE ST8000NM0075 7.3T + U10 ONLINE 0 0 0 SEAGATE ST8000NM0075 7.3T + U11 ONLINE 0 0 0 SEAGATE ST8000NM0075 7.3T + U12 ONLINE 0 0 0 SEAGATE ST8000NM0075 7.3T + U13 ONLINE 0 0 0 SEAGATE ST8000NM0075 7.3T + U14 ONLINE 0 0 0 SEAGATE ST8000NM0075 7.3T + +.No # Nm zpool Cm iostat Fl vc Pa size + capacity operations bandwidth +pool alloc free read write read write size +---------- ----- ----- ----- ----- ----- ----- ---- +rpool 14.6G 54.9G 4 55 250K 2.69M + sda1 14.6G 54.9G 4 55 250K 2.69M 70G +---------- ----- ----- ----- ----- ----- ----- ---- +.Ed +. .Sh SEE ALSO .Xr iostat 1 , .Xr smartctl 8 , diff --git a/man/man8/zpool-labelclear.8 b/man/man8/zpool-labelclear.8 index c7edc911604d..3d7b60b5587d 100644 --- a/man/man8/zpool-labelclear.8 +++ b/man/man8/zpool-labelclear.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -45,7 +45,8 @@ Removes ZFS label information from the specified If the .Ar device is a cache device, it also removes the L2ARC header -(persistent L2ARC). The +(persistent L2ARC). +The .Ar device must not be part of an active pool configuration. .Bl -tag -width Ds diff --git a/man/man8/zpool-list.8 b/man/man8/zpool-list.8 index dd4e13c16042..9e905d52dddc 100644 --- a/man/man8/zpool-list.8 +++ b/man/man8/zpool-list.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -26,7 +26,7 @@ .\" Copyright 2017 Nexenta Systems, Inc. .\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved. .\" -.Dd August 9, 2019 +.Dd March 16, 2022 .Dt ZPOOL-LIST 8 .Os . @@ -107,6 +107,40 @@ Reports usage statistics for individual vdevs within the pool, in addition to the pool-wide statistics. .El . +.Sh EXAMPLES +.\" These are, respectively, examples 6, 15 from zpool.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Listing Available ZFS Storage Pools +The following command lists all available pools on the system. +In this case, the pool +.Ar zion +is faulted due to a missing device. +The results from this command are similar to the following: +.Bd -literal -compact -offset Ds +.No # Nm zpool Cm list +NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT +rpool 19.9G 8.43G 11.4G - 33% 42% 1.00x ONLINE - +tank 61.5G 20.0G 41.5G - 48% 32% 1.00x ONLINE - +zion - - - - - - - FAULTED - +.Ed +. +.Ss Example 2 : No Displaying expanded space on a device +The following command displays the detailed information for the pool +.Ar data . +This pool is comprised of a single raidz vdev where one of its devices +increased its capacity by 10 GiB. +In this example, the pool will not be able to utilize this extra capacity until +all the devices under the raidz vdev have been expanded. +.Bd -literal -compact -offset Ds +.No # Nm zpool Cm list Fl v Ar data +NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT +data 23.9G 14.6G 9.30G - 48% 61% 1.00x ONLINE - + raidz1 23.9G 14.6G 9.30G - 48% + sda - - - - - + sdb - - - 10G - + sdc - - - - - +.Ed +. .Sh SEE ALSO .Xr zpool-import 8 , .Xr zpool-status 8 diff --git a/man/man8/zpool-offline.8 b/man/man8/zpool-offline.8 index 9b2cf59cf414..edcf1d06ab67 100644 --- a/man/man8/zpool-offline.8 +++ b/man/man8/zpool-offline.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool-reguid.8 b/man/man8/zpool-reguid.8 index 7bb7c1c726b2..1fd4ddd9a77d 100644 --- a/man/man8/zpool-reguid.8 +++ b/man/man8/zpool-reguid.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool-remove.8 b/man/man8/zpool-remove.8 index a14218ee17fd..1b5ae72f8da7 100644 --- a/man/man8/zpool-remove.8 +++ b/man/man8/zpool-remove.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -26,12 +26,14 @@ .\" Copyright 2017 Nexenta Systems, Inc. .\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved. .\" -.Dd August 9, 2019 +.Dd March 16, 2022 .Dt ZPOOL-REMOVE 8 .Os +. .Sh NAME .Nm zpool-remove .Nd remove devices from ZFS storage pool +. .Sh SYNOPSIS .Nm zpool .Cm remove @@ -41,6 +43,7 @@ .Cm remove .Fl s .Ar pool +. .Sh DESCRIPTION .Bl -tag -width Ds .It Xo @@ -66,7 +69,7 @@ command initiates the removal and returns, while the evacuation continues in the background. The removal progress can be monitored with .Nm zpool Cm status . -If an IO error is encountered during the removal process it will be cancelled. +If an I/O error is encountered during the removal process it will be cancelled. The .Sy device_removal feature flag must be enabled to remove a top-level vdev, see @@ -102,6 +105,45 @@ Waits until the removal has completed before returning. .Xc Stops and cancels an in-progress removal of a top-level vdev. .El +. +.Sh EXAMPLES +.\" These are, respectively, examples 14 from zpool.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Removing a Mirrored top-level (Log or Data) Device +The following commands remove the mirrored log device +.Sy mirror-2 +and mirrored top-level data device +.Sy mirror-1 . +.Pp +Given this configuration: +.Bd -literal -compact -offset Ds + pool: tank + state: ONLINE + scrub: none requested +config: + + NAME STATE READ WRITE CKSUM + tank ONLINE 0 0 0 + mirror-0 ONLINE 0 0 0 + sda ONLINE 0 0 0 + sdb ONLINE 0 0 0 + mirror-1 ONLINE 0 0 0 + sdc ONLINE 0 0 0 + sdd ONLINE 0 0 0 + logs + mirror-2 ONLINE 0 0 0 + sde ONLINE 0 0 0 + sdf ONLINE 0 0 0 +.Ed +.Pp +The command to remove the mirrored log +.Ar mirror-2 No is: +.Dl # Nm zpool Cm remove Ar tank mirror-2 +.Pp +The command to remove the mirrored data +.Ar mirror-1 No is: +.Dl # Nm zpool Cm remove Ar tank mirror-1 +. .Sh SEE ALSO .Xr zpool-add 8 , .Xr zpool-detach 8 , diff --git a/man/man8/zpool-reopen.8 b/man/man8/zpool-reopen.8 index f1f8606f12c7..5cf7c8635211 100644 --- a/man/man8/zpool-reopen.8 +++ b/man/man8/zpool-reopen.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool-replace.8 b/man/man8/zpool-replace.8 index cc61fa3ea37e..ba897ea704b9 100644 --- a/man/man8/zpool-replace.8 +++ b/man/man8/zpool-replace.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool-resilver.8 b/man/man8/zpool-resilver.8 index 1ef316ac1825..92fdaccafce2 100644 --- a/man/man8/zpool-resilver.8 +++ b/man/man8/zpool-resilver.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool-scrub.8 b/man/man8/zpool-scrub.8 index 69ae825b6158..f5b9ccd2e51a 100644 --- a/man/man8/zpool-scrub.8 +++ b/man/man8/zpool-scrub.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -97,10 +97,8 @@ again. Wait until scrub has completed before returning. .El .Sh EXAMPLES -.Bl -tag -width "Exam" -.It Sy Example 1 : Status of pool with ongoing scrub: -Output: -.Bd -literal -compact -offset Ds +.Ss Example 1 : No Status of pool with ongoing scrub: +.Bd -literal -compact .No # Nm zpool Cm status ... scan: scrub in progress since Sun Jul 25 16:07:49 2021 @@ -108,14 +106,10 @@ Output: 0B repaired, 16.91% done, 00:00:04 to go ... .Ed -Where: -.Bl -dash -offset indent -.It -Metadata which references 403M of file data has been +.Pp +Where metadata which references 403M of file data has been scanned at 100M/s, and 68.4M of that file data has been scrubbed sequentially at 10.0M/s. -.El -.El .Sh PERIODIC SCRUB On machines using systemd, scrub timers can be enabled on per-pool basis. .Nm weekly diff --git a/man/man8/zpool-split.8 b/man/man8/zpool-split.8 index c3b05c2366bf..eb98f48741f5 100644 --- a/man/man8/zpool-split.8 +++ b/man/man8/zpool-split.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool-status.8 b/man/man8/zpool-status.8 index 7c825f69d8e2..2c882a9a9286 100644 --- a/man/man8/zpool-status.8 +++ b/man/man8/zpool-status.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -26,7 +26,7 @@ .\" Copyright 2017 Nexenta Systems, Inc. .\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved. .\" -.Dd June 2, 2021 +.Dd March 16, 2022 .Dt ZPOOL-STATUS 8 .Os . @@ -93,12 +93,12 @@ and referenced .Pq logically referenced in the pool block counts and sizes by reference count. .It Fl s -Display the number of leaf VDEV slow IOs. -This is the number of IOs that -didn't complete in +Display the number of leaf vdev slow I/O operations. +This is the number of I/O operations that didn't complete in .Sy zio_slow_io_ms -milliseconds (default 30 seconds). -This does not necessarily mean the IOs failed to complete, just took an +milliseconds +.Pq Sy 30000 No by default . +This does not necessarily mean the I/O operations failed to complete, just took an unreasonably long amount of time. This may indicate a problem with the underlying storage. .It Fl t @@ -124,6 +124,33 @@ unavailable. Warnings about pools not using the latest on-disk format will not be included. .El . +.Sh EXAMPLES +.\" These are, respectively, examples 16 from zpool.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Adding output columns +Additional columns can be added to the +.Nm zpool Cm status No and Nm zpool Cm iostat No output with Fl c . +.Bd -literal -compact -offset Ds +.No # Nm zpool Cm status Fl c Pa vendor , Ns Pa model , Ns Pa size + NAME STATE READ WRITE CKSUM vendor model size + tank ONLINE 0 0 0 + mirror-0 ONLINE 0 0 0 + U1 ONLINE 0 0 0 SEAGATE ST8000NM0075 7.3T + U10 ONLINE 0 0 0 SEAGATE ST8000NM0075 7.3T + U11 ONLINE 0 0 0 SEAGATE ST8000NM0075 7.3T + U12 ONLINE 0 0 0 SEAGATE ST8000NM0075 7.3T + U13 ONLINE 0 0 0 SEAGATE ST8000NM0075 7.3T + U14 ONLINE 0 0 0 SEAGATE ST8000NM0075 7.3T + +.No # Nm zpool Cm iostat Fl vc Pa size + capacity operations bandwidth +pool alloc free read write read write size +---------- ----- ----- ----- ----- ----- ----- ---- +rpool 14.6G 54.9G 4 55 250K 2.69M + sda1 14.6G 54.9G 4 55 250K 2.69M 70G +---------- ----- ----- ----- ----- ----- ----- ---- +.Ed +. .Sh SEE ALSO .Xr zpool-events 8 , .Xr zpool-history 8 , diff --git a/man/man8/zpool-sync.8 b/man/man8/zpool-sync.8 index aa68a5729e9f..33800c5793dc 100644 --- a/man/man8/zpool-sync.8 +++ b/man/man8/zpool-sync.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool-trim.8 b/man/man8/zpool-trim.8 index d9a7b4400301..7eeaf2de3dc4 100644 --- a/man/man8/zpool-trim.8 +++ b/man/man8/zpool-trim.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -84,8 +84,29 @@ with no flags on the relevant target devices. .It Fl w , -wait Wait until the devices are done being trimmed before returning. .El +.Sh PERIODIC TRIM +On machines using systemd, trim timers can be enabled on a per-pool basis. +.Nm weekly +and +.Nm monthly +timer units are provided. +.Bl -tag -width Ds +.It Xo +.Xc +.Nm systemctl +.Cm enable +.Cm zfs-trim-\fIweekly\fB@\fIrpool\fB.timer +.Cm --now +.It Xo +.Xc +.Nm systemctl +.Cm enable +.Cm zfs-trim-\fImonthly\fB@\fIotherpool\fB.timer +.Cm --now +.El . .Sh SEE ALSO +.Xr systemd.timer 5 , .Xr zpoolprops 7 , .Xr zpool-initialize 8 , .Xr zpool-wait 8 diff --git a/man/man8/zpool-upgrade.8 b/man/man8/zpool-upgrade.8 index 1b13bad898bf..a2421f5846ba 100644 --- a/man/man8/zpool-upgrade.8 +++ b/man/man8/zpool-upgrade.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -27,7 +27,7 @@ .\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved. .\" Copyright (c) 2021, Colm Buckley .\" -.Dd August 9, 2019 +.Dd March 16, 2022 .Dt ZPOOL-UPGRADE 8 .Os . @@ -102,6 +102,17 @@ supported legacy version number. .El .El . +.Sh EXAMPLES +.\" These are, respectively, examples 10 from zpool.8 +.\" Make sure to update them bidirectionally +.Ss Example 1 : No Upgrading All ZFS Storage Pools to the Current Version +The following command upgrades all ZFS Storage pools to the current version of +the software: +.Bd -literal -compact -offset Ds +.No # Nm zpool Cm upgrade Fl a +This system is currently running ZFS version 2. +.Ed +. .Sh SEE ALSO .Xr zpool-features 7 , .Xr zpoolconcepts 7 , diff --git a/man/man8/zpool-wait.8 b/man/man8/zpool-wait.8 index 38f4812ace10..683b0141425c 100644 --- a/man/man8/zpool-wait.8 +++ b/man/man8/zpool-wait.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" diff --git a/man/man8/zpool.8 b/man/man8/zpool.8 index e2de528a301f..a837db38f955 100644 --- a/man/man8/zpool.8 +++ b/man/man8/zpool.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -26,7 +26,7 @@ .\" Copyright 2017 Nexenta Systems, Inc. .\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved. .\" -.Dd June 2, 2021 +.Dd March 16, 2022 .Dt ZPOOL 8 .Os . @@ -148,8 +148,8 @@ for the specified storage pool(s). .It Xr zpool-status 8 Displays the detailed health status for the given pools. .It Xr zpool-iostat 8 -Displays logical I/O statistics for the given pools/vdevs. Physical I/Os may -be observed via +Displays logical I/O statistics for the given pools/vdevs. +Physical I/O operations may be observed via .Xr iostat 1 . .It Xr zpool-events 8 Lists all recent events generated by the ZFS kernel modules. @@ -230,35 +230,45 @@ Invalid command line options were specified. .El . .Sh EXAMPLES -.Bl -tag -width "Exam" -.It Sy Example 1 : No Creating a RAID-Z Storage Pool +.\" Examples 1, 2, 3, 4, 11, 12 are shared with zpool-create.8. +.\" Examples 5, 13 are shared with zpool-add.8. +.\" Examples 6, 15 are shared with zpool-list.8. +.\" Examples 7 are shared with zpool-destroy.8. +.\" Examples 8 are shared with zpool-export.8. +.\" Examples 9 are shared with zpool-import.8. +.\" Examples 10 are shared with zpool-upgrade.8. +.\" Examples 14 are shared with zpool-remove.8. +.\" Examples 16 are shared with zpool-status.8. +.\" Examples 13, 16 are also shared with zpool-iostat.8. +.\" Make sure to update them omnidirectionally +.Ss Example 1 : No Creating a RAID-Z Storage Pool The following command creates a pool with a single raidz root vdev that consists of six disks: -.Dl # Nm zpool Cm create Ar tank Sy raidz Ar sda sdb sdc sdd sde sdf +.Dl # Nm zpool Cm create Ar tank Sy raidz Pa sda sdb sdc sdd sde sdf . -.It Sy Example 2 : No Creating a Mirrored Storage Pool +.Ss Example 2 : No Creating a Mirrored Storage Pool The following command creates a pool with two mirrors, where each mirror contains two disks: -.Dl # Nm zpool Cm create Ar tank Sy mirror Ar sda sdb Sy mirror Ar sdc sdd +.Dl # Nm zpool Cm create Ar tank Sy mirror Pa sda sdb Sy mirror Pa sdc sdd . -.It Sy Example 3 : No Creating a ZFS Storage Pool by Using Partitions -The following command creates an unmirrored pool using two disk partitions: -.Dl # Nm zpool Cm create Ar tank sda1 sdb2 +.Ss Example 3 : No Creating a ZFS Storage Pool by Using Partitions +The following command creates a non-redundant pool using two disk partitions: +.Dl # Nm zpool Cm create Ar tank Pa sda1 sdb2 . -.It Sy Example 4 : No Creating a ZFS Storage Pool by Using Files -The following command creates an unmirrored pool using files. +.Ss Example 4 : No Creating a ZFS Storage Pool by Using Files +The following command creates a non-redundant pool using files. While not recommended, a pool based on files can be useful for experimental purposes. -.Dl # Nm zpool Cm create Ar tank /path/to/file/a /path/to/file/b +.Dl # Nm zpool Cm create Ar tank Pa /path/to/file/a /path/to/file/b . -.It Sy Example 5 : No Adding a Mirror to a ZFS Storage Pool +.Ss Example 5 : No Adding a Mirror to a ZFS Storage Pool The following command adds two mirrored disks to the pool .Ar tank , assuming the pool is already made up of two-way mirrors. The additional space is immediately available to any datasets within the pool. -.Dl # Nm zpool Cm add Ar tank Sy mirror Ar sda sdb +.Dl # Nm zpool Cm add Ar tank Sy mirror Pa sda sdb . -.It Sy Example 6 : No Listing Available ZFS Storage Pools +.Ss Example 6 : No Listing Available ZFS Storage Pools The following command lists all available pools on the system. In this case, the pool .Ar zion @@ -272,19 +282,19 @@ tank 61.5G 20.0G 41.5G - 48% 32% 1.00x ONLINE - zion - - - - - - - FAULTED - .Ed . -.It Sy Example 7 : No Destroying a ZFS Storage Pool +.Ss Example 7 : No Destroying a ZFS Storage Pool The following command destroys the pool .Ar tank and any datasets contained within: .Dl # Nm zpool Cm destroy Fl f Ar tank . -.It Sy Example 8 : No Exporting a ZFS Storage Pool +.Ss Example 8 : No Exporting a ZFS Storage Pool The following command exports the devices in pool .Ar tank so that they can be relocated or later imported: .Dl # Nm zpool Cm export Ar tank . -.It Sy Example 9 : No Importing a ZFS Storage Pool +.Ss Example 9 : No Importing a ZFS Storage Pool The following command displays available pools, and then imports the pool .Ar tank for use on the system. @@ -305,7 +315,7 @@ config: .No # Nm zpool Cm import Ar tank .Ed . -.It Sy Example 10 : No Upgrading All ZFS Storage Pools to the Current Version +.Ss Example 10 : No Upgrading All ZFS Storage Pools to the Current Version The following command upgrades all ZFS Storage pools to the current version of the software: .Bd -literal -compact -offset Ds @@ -313,30 +323,30 @@ the software: This system is currently running ZFS version 2. .Ed . -.It Sy Example 11 : No Managing Hot Spares +.Ss Example 11 : No Managing Hot Spares The following command creates a new pool with an available hot spare: -.Dl # Nm zpool Cm create Ar tank Sy mirror Ar sda sdb Sy spare Ar sdc +.Dl # Nm zpool Cm create Ar tank Sy mirror Pa sda sdb Sy spare Pa sdc .Pp If one of the disks were to fail, the pool would be reduced to the degraded state. The failed device can be replaced using the following command: -.Dl # Nm zpool Cm replace Ar tank sda sdd +.Dl # Nm zpool Cm replace Ar tank Pa sda sdd .Pp Once the data has been resilvered, the spare is automatically removed and is made available for use should another device fail. The hot spare can be permanently removed from the pool using the following command: -.Dl # Nm zpool Cm remove Ar tank sdc +.Dl # Nm zpool Cm remove Ar tank Pa sdc . -.It Sy Example 12 : No Creating a ZFS Pool with Mirrored Separate Intent Logs +.Ss Example 12 : No Creating a ZFS Pool with Mirrored Separate Intent Logs The following command creates a ZFS storage pool consisting of two, two-way mirrors and mirrored log devices: -.Dl # Nm zpool Cm create Ar pool Sy mirror Ar sda sdb Sy mirror Ar sdc sdd Sy log mirror Ar sde sdf +.Dl # Nm zpool Cm create Ar pool Sy mirror Pa sda sdb Sy mirror Pa sdc sdd Sy log mirror Pa sde sdf . -.It Sy Example 13 : No Adding Cache Devices to a ZFS Pool +.Ss Example 13 : No Adding Cache Devices to a ZFS Pool The following command adds two disks for use as cache devices to a ZFS storage pool: -.Dl # Nm zpool Cm add Ar pool Sy cache Ar sdc sdd +.Dl # Nm zpool Cm add Ar pool Sy cache Pa sdc sdd .Pp Once added, the cache devices gradually fill with content from main memory. Depending on the size of your cache devices, it could take over an hour for @@ -346,7 +356,7 @@ Capacity and reads can be monitored using the subcommand as follows: .Dl # Nm zpool Cm iostat Fl v Ar pool 5 . -.It Sy Example 14 : No Removing a Mirrored top-level (Log or Data) Device +.Ss Example 14 : No Removing a Mirrored top-level (Log or Data) Device The following commands remove the mirrored log device .Sy mirror-2 and mirrored top-level data device @@ -381,11 +391,11 @@ The command to remove the mirrored data .Ar mirror-1 No is: .Dl # Nm zpool Cm remove Ar tank mirror-1 . -.It Sy Example 15 : No Displaying expanded space on a device +.Ss Example 15 : No Displaying expanded space on a device The following command displays the detailed information for the pool .Ar data . This pool is comprised of a single raidz vdev where one of its devices -increased its capacity by 10GB. +increased its capacity by 10 GiB. In this example, the pool will not be able to utilize this extra capacity until all the devices under the raidz vdev have been expanded. .Bd -literal -compact -offset Ds @@ -398,11 +408,11 @@ data 23.9G 14.6G 9.30G - 48% 61% 1.00x ONLINE - sdc - - - - - .Ed . -.It Sy Example 16 : No Adding output columns +.Ss Example 16 : No Adding output columns Additional columns can be added to the .Nm zpool Cm status No and Nm zpool Cm iostat No output with Fl c . .Bd -literal -compact -offset Ds -.No # Nm zpool Cm status Fl c Ar vendor , Ns Ar model , Ns Ar size +.No # Nm zpool Cm status Fl c Pa vendor , Ns Pa model , Ns Pa size NAME STATE READ WRITE CKSUM vendor model size tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 @@ -413,7 +423,7 @@ Additional columns can be added to the U13 ONLINE 0 0 0 SEAGATE ST8000NM0075 7.3T U14 ONLINE 0 0 0 SEAGATE ST8000NM0075 7.3T -.No # Nm zpool Cm iostat Fl vc Ar size +.No # Nm zpool Cm iostat Fl vc Pa size capacity operations bandwidth pool alloc free read write read write size ---------- ----- ----- ----- ----- ----- ----- ---- @@ -421,7 +431,6 @@ rpool 14.6G 54.9G 4 55 250K 2.69M sda1 14.6G 54.9G 4 55 250K 2.69M 70G ---------- ----- ----- ----- ----- ----- ----- ---- .Ed -.El . .Sh ENVIRONMENT VARIABLES .Bl -tag -compact -width "ZPOOL_IMPORT_UDEV_TIMEOUT_MS" @@ -432,7 +441,7 @@ to dump core on exit for the purposes of running .Sy ::findleaks . .It Sy ZFS_COLOR Use ANSI color in -.Nm zpool status +.Nm zpool Cm status output. .It Sy ZPOOL_IMPORT_PATH The search path for devices or files to use with the pool. @@ -449,7 +458,7 @@ The maximum time in milliseconds that will wait for an expected device to be available. .It Sy ZPOOL_STATUS_NON_NATIVE_ASHIFT_IGNORE If set, suppress warning about non-native vdev ashift in -.Nm zpool status . +.Nm zpool Cm status . The value is not used, only the presence or absence of the variable matters. .It Sy ZPOOL_VDEV_NAME_GUID Cause @@ -474,14 +483,14 @@ This behavior is identical to the command line option. .It Sy ZFS_VDEV_DEVID_OPT_OUT Older OpenZFS implementations had issues when attempting to display pool -config VDEV names if a +config vdev names if a .Sy devid NVP value is present in the pool's config. .Pp For example, a pool that originated on illumos platform would have a .Sy devid value in the config and -.Nm zpool status +.Nm zpool Cm status would fail when listing the config. This would also be true for future Linux-based pools. .Pp @@ -497,12 +506,12 @@ by setting .Pp .It Sy ZPOOL_SCRIPTS_AS_ROOT Allow a privileged user to run -.Nm zpool status/iostat Fl c . +.Nm zpool Cm status Ns / Ns Cm iostat Fl c . Normally, only unprivileged users are allowed to run .Fl c . .It Sy ZPOOL_SCRIPTS_PATH The search path for scripts when running -.Nm zpool status/iostat Fl c . +.Nm zpool Cm status Ns / Ns Cm iostat Fl c . This is a colon-separated list of directories and overrides the default .Pa ~/.zpool.d and @@ -510,11 +519,25 @@ and search paths. .It Sy ZPOOL_SCRIPTS_ENABLED Allow a user to run -.Nm zpool status/iostat Fl c . +.Nm zpool Cm status Ns / Ns Cm iostat Fl c . If .Sy ZPOOL_SCRIPTS_ENABLED is not set, it is assumed that the user is allowed to run .Nm zpool Cm status Ns / Ns Cm iostat Fl c . +.\" Shared with zfs.8 +.It Sy ZFS_MODULE_TIMEOUT +Time, in seconds, to wait for +.Pa /dev/zfs +to appear. +Defaults to +.Sy 10 , +max +.Sy 600 Pq 10 minutes . +If +.Pf < Sy 0 , +wait forever; if +.Sy 0 , +don't wait. .El . .Sh INTERFACE STABILITY diff --git a/man/man8/zstream.8 b/man/man8/zstream.8 index c0322ee3ace0..dd5342000216 100644 --- a/man/man8/zstream.8 +++ b/man/man8/zstream.8 @@ -6,7 +6,7 @@ .\" You may not use this file except in compliance with the License. .\" .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -.\" or http://www.opensolaris.org/os/licensing. +.\" or https://opensource.org/licenses/CDDL-1.0. .\" See the License for the specific language governing permissions .\" and limitations under the License. .\" @@ -20,7 +20,7 @@ .\" .\" Copyright (c) 2020 by Delphix. All rights reserved. .\" -.Dd May 8, 2021 +.Dd March 25, 2022 .Dt ZSTREAM 8 .Os . @@ -33,6 +33,10 @@ .Op Fl Cvd .Op Ar file .Nm +.Cm decompress +.Op Fl v +.Op Ar object Ns Sy \&, Ns Ar offset Ns Op Sy \&, Ns Ar type Ns ... +.Nm .Cm redup .Op Fl v .Ar file @@ -82,6 +86,36 @@ alias is provided for compatibility and is equivalent to running Dumps zfs resume token information .It Xo .Nm +.Cm decompress +.Op Fl v +.Op Ar object Ns Sy \&, Ns Ar offset Ns Op Sy \&, Ns Ar type Ns ... +.Xc +Decompress selected records in a ZFS send stream provided on standard input, +when the compression type recorded in ZFS metadata may be incorrect. +Specify the object number and byte offset of each record that you wish to +decompress. +Optionally specify the compression type. +Valid compression types include +.Sy gzip , +.Sy lz4 , +.Sy lzjb , +.Sy zstd , +and +.Sy zle . +The default is +.Sy lz4 . +Every record for that object beginning at that offset will be decompressed, if +possible. +It may not be possible, because the record may be corrupted in some but not +all of the stream's snapshots. +The repaired stream will be written to standard output. +.Bl -tag -width "-v" +.It Fl v +Verbose. +Print summary of decompressed records. +.El +.It Xo +.Nm .Cm redup .Op Fl v .Ar file @@ -111,7 +145,24 @@ Print summary of converted records. .El .El . +.Sh EXAMPLES +Heal a dataset that was corrupted due to OpenZFS bug #12762. +First, determine which records are corrupt. +That cannot be done automatically; it requires information beyond ZFS's +metadata. +If object +.Sy 128 +is corrupted at offset +.Sy 0 +and is compressed using +.Sy lz4 , +then run this command: +.Bd -literal +.No # Nm zfs Ar send Fl c Ar … | Nm zstream decompress Ar 128,0,lz4 | \ +Nm zfs recv Ar … +.Ed .Sh SEE ALSO .Xr zfs 8 , .Xr zfs-receive 8 , -.Xr zfs-send 8 +.Xr zfs-send 8 , +.Lk https://github.com/openzfs/zfs/issues/12762 diff --git a/module/Kbuild.in b/module/Kbuild.in index 1507965c5750..4803952cbfed 100644 --- a/module/Kbuild.in +++ b/module/Kbuild.in @@ -1,20 +1,6 @@ # When integrated in to a monolithic kernel the spl module must appear # first. This ensures its module initialization function is run before # any of the other module initialization functions which depend on it. -ZFS_MODULES += spl/ -ZFS_MODULES += avl/ -ZFS_MODULES += icp/ -ZFS_MODULES += lua/ -ZFS_MODULES += nvpair/ -ZFS_MODULES += unicode/ -ZFS_MODULES += zcommon/ -ZFS_MODULES += zfs/ -ZFS_MODULES += zstd/ - -# The rest is only relevant when run by kbuild -ifneq ($(KERNELRELEASE),) - -obj-$(CONFIG_ZFS) := $(ZFS_MODULES) ZFS_MODULE_CFLAGS += -std=gnu99 -Wno-declaration-after-statement ZFS_MODULE_CFLAGS += -Wmissing-prototypes @@ -22,10 +8,16 @@ ZFS_MODULE_CFLAGS += @KERNEL_DEBUG_CFLAGS@ @NO_FORMAT_ZERO_LENGTH@ ifneq ($(KBUILD_EXTMOD),) zfs_include = @abs_top_srcdir@/include +icp_include = @abs_srcdir@/icp/include +zstd_include = @abs_srcdir@/zstd/include ZFS_MODULE_CFLAGS += -include @abs_top_builddir@/zfs_config.h ZFS_MODULE_CFLAGS += -I@abs_top_builddir@/include +src = @abs_srcdir@ +obj = @abs_builddir@ else zfs_include = $(srctree)/include/zfs +icp_include = $(srctree)/$(src)/icp/include +zstd_include = $(srctree)/$(src)/zstd/include ZFS_MODULE_CFLAGS += -include $(zfs_include)/zfs_config.h endif @@ -36,12 +28,447 @@ ZFS_MODULE_CFLAGS += -I$(zfs_include) ZFS_MODULE_CPPFLAGS += -D_KERNEL ZFS_MODULE_CPPFLAGS += @KERNEL_DEBUG_CPPFLAGS@ +# KASAN enables -Werror=frame-larger-than=1024, which +# breaks oh so many parts of our build. +ifeq ($(CONFIG_KASAN),y) +ZFS_MODULE_CFLAGS += -Wno-error=frame-larger-than= +endif + ifneq ($(KBUILD_EXTMOD),) @CONFIG_QAT_TRUE@ZFS_MODULE_CFLAGS += -I@QAT_SRC@/include @CONFIG_QAT_TRUE@KBUILD_EXTRA_SYMBOLS += @QAT_SYMBOLS@ endif -subdir-asflags-y := $(ZFS_MODULE_CFLAGS) $(ZFS_MODULE_CPPFLAGS) -subdir-ccflags-y := $(ZFS_MODULE_CFLAGS) $(ZFS_MODULE_CPPFLAGS) +asflags-y := $(ZFS_MODULE_CFLAGS) $(ZFS_MODULE_CPPFLAGS) +ccflags-y := $(ZFS_MODULE_CFLAGS) $(ZFS_MODULE_CPPFLAGS) + +# Suppress unused-value warnings in sparc64 architecture headers +ccflags-$(CONFIG_SPARC64) += -Wno-unused-value + + +obj-$(CONFIG_ZFS) := spl.o zfs.o + +SPL_OBJS := \ + spl-atomic.o \ + spl-condvar.o \ + spl-cred.o \ + spl-err.o \ + spl-generic.o \ + spl-kmem-cache.o \ + spl-kmem.o \ + spl-kstat.o \ + spl-proc.o \ + spl-procfs-list.o \ + spl-taskq.o \ + spl-thread.o \ + spl-trace.o \ + spl-tsd.o \ + spl-vmem.o \ + spl-xdr.o \ + spl-zlib.o \ + spl-zone.o + +spl-objs += $(addprefix os/linux/spl/,$(SPL_OBJS)) + +zfs-objs += avl/avl.o + +ICP_OBJS := \ + algs/aes/aes_impl.o \ + algs/aes/aes_impl_generic.o \ + algs/aes/aes_modes.o \ + algs/blake3/blake3.o \ + algs/blake3/blake3_generic.o \ + algs/blake3/blake3_impl.o \ + algs/blake3/blake3_x86-64.o \ + algs/edonr/edonr.o \ + algs/modes/cbc.o \ + algs/modes/ccm.o \ + algs/modes/ctr.o \ + algs/modes/ecb.o \ + algs/modes/gcm.o \ + algs/modes/gcm_generic.o \ + algs/modes/modes.o \ + algs/sha2/sha2.o \ + algs/skein/skein.o \ + algs/skein/skein_block.o \ + algs/skein/skein_iv.o \ + api/kcf_cipher.o \ + api/kcf_ctxops.o \ + api/kcf_mac.o \ + core/kcf_callprov.o \ + core/kcf_mech_tabs.o \ + core/kcf_prov_lib.o \ + core/kcf_prov_tabs.o \ + core/kcf_sched.o \ + illumos-crypto.o \ + io/aes.o \ + io/sha2_mod.o \ + io/skein_mod.o \ + spi/kcf_spi.o + +ICP_OBJS_X86_64 := \ + asm-x86_64/aes/aes_aesni.o \ + asm-x86_64/aes/aes_amd64.o \ + asm-x86_64/aes/aeskey.o \ + asm-x86_64/blake3/blake3_avx2.o \ + asm-x86_64/blake3/blake3_avx512.o \ + asm-x86_64/blake3/blake3_sse2.o \ + asm-x86_64/blake3/blake3_sse41.o \ + asm-x86_64/modes/aesni-gcm-x86_64.o \ + asm-x86_64/modes/gcm_pclmulqdq.o \ + asm-x86_64/modes/ghash-x86_64.o \ + asm-x86_64/sha2/sha256_impl.o \ + asm-x86_64/sha2/sha512_impl.o + + +ICP_OBJS_X86 := \ + algs/aes/aes_impl_aesni.o \ + algs/aes/aes_impl_x86-64.o \ + algs/modes/gcm_pclmulqdq.o + + +ICP_OBJS_ARM64 := \ + asm-aarch64/blake3/b3_aarch64_sse2.o \ + asm-aarch64/blake3/b3_aarch64_sse41.o + + +ICP_OBJS_PPC_PPC64 := \ + asm-ppc64/blake3/b3_ppc64le_sse2.o \ + asm-ppc64/blake3/b3_ppc64le_sse41.o + +zfs-objs += $(addprefix icp/,$(ICP_OBJS)) +zfs-$(CONFIG_X86) += $(addprefix icp/,$(ICP_OBJS_X86)) +zfs-$(CONFIG_UML_X86)+= $(addprefix icp/,$(ICP_OBJS_X86)) +zfs-$(CONFIG_X86_64) += $(addprefix icp/,$(ICP_OBJS_X86_64)) +zfs-$(CONFIG_ARM64) += $(addprefix icp/,$(ICP_OBJS_ARM64)) +zfs-$(CONFIG_PPC) += $(addprefix icp/,$(ICP_OBJS_PPC_PPC64)) +zfs-$(CONFIG_PPC64) += $(addprefix icp/,$(ICP_OBJS_PPC_PPC64)) + +$(addprefix $(obj)/icp/,$(ICP_OBJS) $(ICP_OBJS_X86) $(ICP_OBJS_X86_64) \ + $(ICP_OBJS_ARM64) $(ICP_OBJS_PPC_PPC64)) : asflags-y += -I$(icp_include) + +$(addprefix $(obj)/icp/,$(ICP_OBJS) $(ICP_OBJS_X86) $(ICP_OBJS_X86_64) \ + $(ICP_OBJS_ARM64) $(ICP_OBJS_PPC_PPC64)) : ccflags-y += -I$(icp_include) + +# Suppress objtool "can't find jump dest instruction at" warnings. They +# are caused by the constants which are defined in the text section of the +# assembly file using .byte instructions (e.g. bswap_mask). The objtool +# utility tries to interpret them as opcodes and obviously fails doing so. +OBJECT_FILES_NON_STANDARD_aesni-gcm-x86_64.o := y +OBJECT_FILES_NON_STANDARD_ghash-x86_64.o := y + +# Suppress objtool "unsupported stack pointer realignment" warnings. We are +# not using a DRAP register while aligning the stack to a 64 byte boundary. +# See #6950 for the reasoning. +OBJECT_FILES_NON_STANDARD_sha256_impl.o := y +OBJECT_FILES_NON_STANDARD_sha512_impl.o := y + + +LUA_OBJS := \ + lapi.o \ + lauxlib.o \ + lbaselib.o \ + lcode.o \ + lcompat.o \ + lcorolib.o \ + lctype.o \ + ldebug.o \ + ldo.o \ + lfunc.o \ + lgc.o \ + llex.o \ + lmem.o \ + lobject.o \ + lopcodes.o \ + lparser.o \ + lstate.o \ + lstring.o \ + lstrlib.o \ + ltable.o \ + ltablib.o \ + ltm.o \ + lvm.o \ + lzio.o \ + setjmp/setjmp.o + +zfs-objs += $(addprefix lua/,$(LUA_OBJS)) + + +NVPAIR_OBJS := \ + fnvpair.o \ + nvpair.o \ + nvpair_alloc_fixed.o \ + nvpair_alloc_spl.o + +zfs-objs += $(addprefix nvpair/,$(NVPAIR_OBJS)) + + +UNICODE_OBJS := \ + u8_textprep.o \ + uconv.o + +zfs-objs += $(addprefix unicode/,$(UNICODE_OBJS)) + + +ZCOMMON_OBJS := \ + cityhash.o \ + zfeature_common.o \ + zfs_comutil.o \ + zfs_deleg.o \ + zfs_fletcher.o \ + zfs_fletcher_superscalar.o \ + zfs_fletcher_superscalar4.o \ + zfs_namecheck.o \ + zfs_prop.o \ + zpool_prop.o \ + zprop_common.o + +ZCOMMON_OBJS_X86 := \ + zfs_fletcher_avx512.o \ + zfs_fletcher_intel.o \ + zfs_fletcher_sse.o + +ZCOMMON_OBJS_ARM64 := \ + zfs_fletcher_aarch64_neon.o + +zfs-objs += $(addprefix zcommon/,$(ZCOMMON_OBJS)) +zfs-$(CONFIG_X86) += $(addprefix zcommon/,$(ZCOMMON_OBJS_X86)) +zfs-$(CONFIG_UML_X86)+= $(addprefix zcommon/,$(ZCOMMON_OBJS_X86)) +zfs-$(CONFIG_ARM64) += $(addprefix zcommon/,$(ZCOMMON_OBJS_ARM64)) + + +# Zstd uses -O3 by default, so we should follow +ZFS_ZSTD_FLAGS := -O3 + +# -fno-tree-vectorize gets set for gcc in zstd/common/compiler.h +# Set it for other compilers, too. +ZFS_ZSTD_FLAGS += -fno-tree-vectorize + +# SSE register return with SSE disabled if -march=znverX is passed +ZFS_ZSTD_FLAGS += -U__BMI__ + +# Quiet warnings about frame size due to unused code in unmodified zstd lib +ZFS_ZSTD_FLAGS += -Wframe-larger-than=20480 + +ZSTD_OBJS := \ + zfs_zstd.o \ + zstd_sparc.o + +ZSTD_UPSTREAM_OBJS := \ + lib/common/entropy_common.o \ + lib/common/error_private.o \ + lib/common/fse_decompress.o \ + lib/common/pool.o \ + lib/common/zstd_common.o \ + lib/compress/fse_compress.o \ + lib/compress/hist.o \ + lib/compress/huf_compress.o \ + lib/compress/zstd_compress.o \ + lib/compress/zstd_compress_literals.o \ + lib/compress/zstd_compress_sequences.o \ + lib/compress/zstd_compress_superblock.o \ + lib/compress/zstd_double_fast.o \ + lib/compress/zstd_fast.o \ + lib/compress/zstd_lazy.o \ + lib/compress/zstd_ldm.o \ + lib/compress/zstd_opt.o \ + lib/decompress/huf_decompress.o \ + lib/decompress/zstd_ddict.o \ + lib/decompress/zstd_decompress.o \ + lib/decompress/zstd_decompress_block.o + +zfs-objs += $(addprefix zstd/,$(ZSTD_OBJS) $(ZSTD_UPSTREAM_OBJS)) + +# Disable aarch64 neon SIMD instructions for kernel mode +$(addprefix $(obj)/zstd/,$(ZSTD_OBJS) $(ZSTD_UPSTREAM_OBJS)) : ccflags-y += -I$(zstd_include) $(ZFS_ZSTD_FLAGS) +$(addprefix $(obj)/zstd/,$(ZSTD_OBJS) $(ZSTD_UPSTREAM_OBJS)) : asflags-y += -I$(zstd_include) +$(addprefix $(obj)/zstd/,$(ZSTD_UPSTREAM_OBJS)) : ccflags-y += -include $(zstd_include)/aarch64_compat.h -include $(zstd_include)/zstd_compat_wrapper.h -Wp,-w +$(obj)/zstd/zfs_zstd.o : ccflags-y += -include $(zstd_include)/zstd_compat_wrapper.h + + +ZFS_OBJS := \ + abd.o \ + aggsum.o \ + arc.o \ + blake3_zfs.o \ + blkptr.o \ + bplist.o \ + bpobj.o \ + bptree.o \ + bqueue.o \ + btree.o \ + dataset_kstats.o \ + dbuf.o \ + dbuf_stats.o \ + ddt.o \ + ddt_zap.o \ + dmu.o \ + dmu_diff.o \ + dmu_object.o \ + dmu_objset.o \ + dmu_recv.o \ + dmu_redact.o \ + dmu_send.o \ + dmu_traverse.o \ + dmu_tx.o \ + dmu_zfetch.o \ + dnode.o \ + dnode_sync.o \ + dsl_bookmark.o \ + dsl_crypt.o \ + dsl_dataset.o \ + dsl_deadlist.o \ + dsl_deleg.o \ + dsl_destroy.o \ + dsl_dir.o \ + dsl_pool.o \ + dsl_prop.o \ + dsl_scan.o \ + dsl_synctask.o \ + dsl_userhold.o \ + edonr_zfs.o \ + fm.o \ + gzip.o \ + hkdf.o \ + lz4.o \ + lz4_zfs.o \ + lzjb.o \ + metaslab.o \ + mmp.o \ + multilist.o \ + objlist.o \ + pathname.o \ + range_tree.o \ + refcount.o \ + rrwlock.o \ + sa.o \ + sha256.o \ + skein_zfs.o \ + spa.o \ + spa_boot.o \ + spa_checkpoint.o \ + spa_config.o \ + spa_errlog.o \ + spa_history.o \ + spa_log_spacemap.o \ + spa_misc.o \ + spa_stats.o \ + space_map.o \ + space_reftree.o \ + txg.o \ + uberblock.o \ + unique.o \ + vdev.o \ + vdev_cache.o \ + vdev_draid.o \ + vdev_draid_rand.o \ + vdev_indirect.o \ + vdev_indirect_births.o \ + vdev_indirect_mapping.o \ + vdev_initialize.o \ + vdev_label.o \ + vdev_mirror.o \ + vdev_missing.o \ + vdev_queue.o \ + vdev_raidz.o \ + vdev_raidz_math.o \ + vdev_raidz_math_scalar.o \ + vdev_rebuild.o \ + vdev_removal.o \ + vdev_root.o \ + vdev_trim.o \ + zap.o \ + zap_leaf.o \ + zap_micro.o \ + zcp.o \ + zcp_get.o \ + zcp_global.o \ + zcp_iter.o \ + zcp_set.o \ + zcp_synctask.o \ + zfeature.o \ + zfs_byteswap.o \ + zfs_chksum.o \ + zfs_fm.o \ + zfs_fuid.o \ + zfs_ioctl.o \ + zfs_log.o \ + zfs_onexit.o \ + zfs_quota.o \ + zfs_ratelimit.o \ + zfs_replay.o \ + zfs_rlock.o \ + zfs_sa.o \ + zfs_vnops.o \ + zil.o \ + zio.o \ + zio_checksum.o \ + zio_compress.o \ + zio_inject.o \ + zle.o \ + zrlock.o \ + zthr.o \ + zvol.o + +ZFS_OBJS_OS := \ + abd_os.o \ + arc_os.o \ + mmp_os.o \ + policy.o \ + qat.o \ + qat_compress.o \ + qat_crypt.o \ + spa_misc_os.o \ + trace.o \ + vdev_disk.o \ + vdev_file.o \ + zfs_acl.o \ + zfs_ctldir.o \ + zfs_debug.o \ + zfs_dir.o \ + zfs_file_os.o \ + zfs_ioctl_os.o \ + zfs_racct.o \ + zfs_sysfs.o \ + zfs_uio.o \ + zfs_vfsops.o \ + zfs_vnops_os.o \ + zfs_znode.o \ + zio_crypt.o \ + zpl_ctldir.o \ + zpl_export.o \ + zpl_file.o \ + zpl_inode.o \ + zpl_super.o \ + zpl_xattr.o \ + zvol_os.o + +ZFS_OBJS_X86 := \ + vdev_raidz_math_avx2.o \ + vdev_raidz_math_avx512bw.o \ + vdev_raidz_math_avx512f.o \ + vdev_raidz_math_sse2.o \ + vdev_raidz_math_ssse3.o + +ZFS_OBJS_ARM64 := \ + vdev_raidz_math_aarch64_neon.o \ + vdev_raidz_math_aarch64_neonx2.o + +ZFS_OBJS_PPC_PPC64 := \ + vdev_raidz_math_powerpc_altivec.o + +zfs-objs += $(addprefix zfs/,$(ZFS_OBJS)) $(addprefix os/linux/zfs/,$(ZFS_OBJS_OS)) +zfs-$(CONFIG_X86) += $(addprefix zfs/,$(ZFS_OBJS_X86)) +zfs-$(CONFIG_UML_X86)+= $(addprefix zfs/,$(ZFS_OBJS_X86)) +zfs-$(CONFIG_ARM64) += $(addprefix zfs/,$(ZFS_OBJS_ARM64)) +zfs-$(CONFIG_PPC) += $(addprefix zfs/,$(ZFS_OBJS_PPC_PPC64)) +zfs-$(CONFIG_PPC64) += $(addprefix zfs/,$(ZFS_OBJS_PPC_PPC64)) + +# Suppress incorrect warnings from versions of objtool which are not +# aware of x86 EVEX prefix instructions used for AVX512. +OBJECT_FILES_NON_STANDARD_vdev_raidz_math_avx512bw.o := y +OBJECT_FILES_NON_STANDARD_vdev_raidz_math_avx512f.o := y +ifeq ($(CONFIG_ALTIVEC),y) +$(obj)/zfs/vdev_raidz_math_powerpc_altivec.o : c_flags += -maltivec endif diff --git a/module/Makefile.bsd b/module/Makefile.bsd index 5172394e6ac8..589ca60b29be 100644 --- a/module/Makefile.bsd +++ b/module/Makefile.bsd @@ -10,6 +10,10 @@ INCDIR=${.CURDIR:H}/include KMOD= openzfs .PATH: ${SRCDIR}/avl \ + ${SRCDIR}/icp/algs/blake3 \ + ${SRCDIR}/icp/asm-aarch64/blake3 \ + ${SRCDIR}/icp/asm-ppc64/blake3 \ + ${SRCDIR}/icp/asm-x86_64/blake3 \ ${SRCDIR}/lua \ ${SRCDIR}/nvpair \ ${SRCDIR}/icp/algs/edonr \ @@ -19,7 +23,9 @@ KMOD= openzfs ${SRCDIR}/zcommon \ ${SRCDIR}/zfs \ ${SRCDIR}/zstd \ - ${SRCDIR}/zstd/lib + ${SRCDIR}/zstd/lib/common \ + ${SRCDIR}/zstd/lib/compress \ + ${SRCDIR}/zstd/lib/decompress @@ -29,14 +35,16 @@ CFLAGS+= -I${INCDIR}/os/freebsd CFLAGS+= -I${INCDIR}/os/freebsd/spl CFLAGS+= -I${INCDIR}/os/freebsd/zfs CFLAGS+= -I${SRCDIR}/zstd/include +CFLAGS+= -I${SRCDIR}/icp/include CFLAGS+= -include ${INCDIR}/os/freebsd/spl/sys/ccompile.h CFLAGS+= -D__KERNEL__ -DFREEBSD_NAMECACHE -DBUILDING_ZFS -D__BSD_VISIBLE=1 \ -DHAVE_UIO_ZEROCOPY -DWITHOUT_NETDUMP -D__KERNEL -D_SYS_CONDVAR_H_ \ - -D_SYS_VMEM_H_ -DKDTRACE_HOOKS -DSMP -DHAVE_KSID -DCOMPAT_FREEBSD11 + -D_SYS_VMEM_H_ -DKDTRACE_HOOKS -DSMP -DCOMPAT_FREEBSD11 .if ${MACHINE_ARCH} == "amd64" -CFLAGS+= -DHAVE_AVX2 -DHAVE_AVX -D__x86_64 -DHAVE_SSE2 -DHAVE_AVX512F -DHAVE_SSSE3 +CFLAGS+= -D__x86_64 -DHAVE_SSE2 -DHAVE_SSSE3 -DHAVE_SSE4_1 -DHAVE_SSE4_2 \ + -DHAVE_AVX -DHAVE_AVX2 -DHAVE_AVX512F -DHAVE_AVX512VL .endif .if defined(WITH_DEBUG) && ${WITH_DEBUG} == "true" @@ -71,12 +79,32 @@ CFLAGS+= -DBITS_PER_LONG=64 SRCS= vnode_if.h device_if.h bus_if.h -# avl +#avl SRCS+= avl.c # icp SRCS+= edonr.c +#icp/algs/blake3 +SRCS+= blake3.c \ + blake3_generic.c \ + blake3_impl.c \ + blake3_x86-64.c + +#icp/asm-aarch64/blake3 +SRCS+= b3_aarch64_sse2.S \ + b3_aarch64_sse41.S + +#icp/asm-ppc64/blake3 +SRCS+= b3_ppc64le_sse2.S \ + b3_ppc64le_sse41.S + +#icp/asm-x86_64/blake3 +SRCS+= blake3_avx2.S \ + blake3_avx512.S \ + blake3_sse2.S \ + blake3_sse41.S + #lua SRCS+= lapi.c \ lauxlib.c \ @@ -187,6 +215,7 @@ SRCS+= zfeature_common.c \ SRCS+= abd.c \ aggsum.c \ arc.c \ + blake3_zfs.c \ blkptr.c \ bplist.c \ bpobj.c \ @@ -289,6 +318,7 @@ SRCS+= abd.c \ zcp_synctask.c \ zfeature.c \ zfs_byteswap.c \ + zfs_chksum.c \ zfs_file_os.c \ zfs_fm.c \ zfs_fuid.c \ @@ -313,7 +343,27 @@ SRCS+= abd.c \ #zstd SRCS+= zfs_zstd.c \ - zstd.c + entropy_common.c \ + error_private.c \ + fse_decompress.c \ + pool.c \ + zstd_common.c \ + fse_compress.c \ + hist.c \ + huf_compress.c \ + zstd_compress.c \ + zstd_compress_literals.c \ + zstd_compress_sequences.c \ + zstd_compress_superblock.c \ + zstd_double_fast.c \ + zstd_fast.c \ + zstd_lazy.c \ + zstd_ldm.c \ + zstd_opt.c \ + huf_decompress.c \ + zstd_ddict.c \ + zstd_decompress.c \ + zstd_decompress_block.c beforeinstall: .if ${MK_DEBUG_FILES} != "no" @@ -373,4 +423,25 @@ CFLAGS.zil.c= -Wno-cast-qual CFLAGS.zio.c= -Wno-cast-qual CFLAGS.zrlock.c= -Wno-cast-qual CFLAGS.zfs_zstd.c= -Wno-cast-qual -Wno-pointer-arith -CFLAGS.zstd.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.entropy_common.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.error_private.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.fse_decompress.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.pool.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.xxhash.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.zstd_common.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.fse_compress.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.hist.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.huf_compress.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.zstd_compress.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.zstd_compress_literals.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.zstd_compress_sequences.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.zstd_compress_superblock.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.zstd_double_fast.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.zstd_fast.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.zstd_lazy.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.zstd_ldm.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.zstd_opt.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.huf_decompress.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.zstd_ddict.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.zstd_decompress.c= -fno-tree-vectorize -U__BMI__ +CFLAGS.zstd_decompress_block.c= -fno-tree-vectorize -U__BMI__ diff --git a/module/Makefile.in b/module/Makefile.in index b6338430e812..5b71e1abf79e 100644 --- a/module/Makefile.in +++ b/module/Makefile.in @@ -3,19 +3,19 @@ include Kbuild INSTALL_MOD_DIR ?= extra INSTALL_MOD_PATH ?= $(DESTDIR) -SUBDIR_TARGETS = icp lua zstd - all: modules distclean maintainer-clean: clean -install: modules_install -uninstall: modules_uninstall +install: modules_install data_install +uninstall: modules_uninstall data_uninstall check: .PHONY: all distclean maintainer-clean install uninstall check distdir \ modules modules-Linux modules-FreeBSD modules-unknown \ clean clean-Linux clean-FreeBSD \ modules_install modules_install-Linux modules_install-FreeBSD \ + data_install data_install-Linux data_install-FreeBSD \ modules_uninstall modules_uninstall-Linux modules_uninstall-FreeBSD \ + data_uninstall data_uninstall-Linux data_uninstall-FreeBSD \ cppcheck cppcheck-Linux cppcheck-FreeBSD # For FreeBSD, use debug options from ./configure if not overridden. @@ -51,7 +51,8 @@ endif FMAKE = env -u MAKEFLAGS make $(FMAKEFLAGS) modules-Linux: - list='$(SUBDIR_TARGETS)'; for td in $$list; do $(MAKE) -C $$td; done + mkdir -p $(sort $(dir $(spl-objs) $(spl-))) + mkdir -p $(sort $(dir $(zfs-objs) $(zfs-))) $(MAKE) -C @LINUX_OBJ@ $(if @KERNEL_CC@,CC=@KERNEL_CC@) \ $(if @KERNEL_LD@,LD=@KERNEL_LD@) $(if @KERNEL_LLVM@,LLVM=@KERNEL_LLVM@) \ M="$$PWD" @KERNEL_MAKE@ CONFIG_ZFS=m modules @@ -77,16 +78,20 @@ clean-FreeBSD: clean: clean-@ac_system@ -modules_install-Linux: +.PHONY: modules_uninstall-Linux-legacy +modules_uninstall-Linux-legacy: + $(RM) -r $(addprefix $(KMODDIR)/$(INSTALL_MOD_DIR)/,spl/ avl/ icp/ lua/ nvpair/ unicode/ zcommon/ zfs/ zstd/) + +KMODDIR := $(INSTALL_MOD_PATH)/lib/modules/@LINUX_VERSION@ +modules_install-Linux: modules_uninstall-Linux-legacy @# Install the kernel modules $(MAKE) -C @LINUX_OBJ@ M="$$PWD" modules_install \ INSTALL_MOD_PATH=$(INSTALL_MOD_PATH) \ INSTALL_MOD_DIR=$(INSTALL_MOD_DIR) \ KERNELRELEASE=@LINUX_VERSION@ @# Remove extraneous build products when packaging - kmoddir=$(INSTALL_MOD_PATH)/lib/modules/@LINUX_VERSION@; \ if [ -n "$(DESTDIR)" ]; then \ - find $$kmoddir -name 'modules.*' -delete; \ + find $(KMODDIR) -name 'modules.*' -delete; \ fi @# Debian ships tiny fake System.map files that are @# syntactically valid but just say @@ -107,18 +112,32 @@ modules_install-FreeBSD: modules_install: modules_install-@ac_system@ -modules_uninstall-Linux: +data_install-Linux: + @mkdir -p $(DESTDIR)/@prefix@/src/zfs-@VERSION@/@LINUX_VERSION@ + cp ../zfs.release ../zfs_config.h @LINUX_SYMBOLS@ $(DESTDIR)/@prefix@/src/zfs-@VERSION@/@LINUX_VERSION@ + +data_install-FreeBSD: + @ + +data_install: data_install-@ac_system@ + +modules_uninstall-Linux: modules_uninstall-Linux-legacy @# Uninstall the kernel modules - kmoddir=$(INSTALL_MOD_PATH)/lib/modules/@LINUX_VERSION@; \ - for objdir in $(ZFS_MODULES); do \ - $(RM) -R $$kmoddir/$(INSTALL_MOD_DIR)/$$objdir; \ - done + $(RM) $(addprefix $(KMODDIR)/$(INSTALL_MOD_DIR)/,zfs.ko spl.ko) modules_uninstall-FreeBSD: @false modules_uninstall: modules_uninstall-@ac_system@ +data_uninstall-Linux: + $(RM) $(addprefix $(DESTDIR)/@prefix@/src/zfs-@VERSION@/@LINUX_VERSION@/,zfs.release zfs_config.h @LINUX_SYMBOLS@) + +data_uninstall-FreeBSD: + @ + +data_uninstall: data_uninstall-@ac_system@ + cppcheck-Linux: @CPPCHECK@ -j@CPU_COUNT@ --std=c99 --quiet --force --error-exitcode=2 \ --inline-suppr \ @@ -126,14 +145,15 @@ cppcheck-Linux: --suppress=noValidConfiguration \ --enable=warning,information -D_KERNEL \ --include=@LINUX_OBJ@/include/generated/autoconf.h \ - --include=@top_srcdir@/zfs_config.h \ + --include=@top_builddir@/zfs_config.h \ --config-exclude=@LINUX_OBJ@/include \ + -i zstd/lib \ -I @LINUX_OBJ@/include \ -I @top_srcdir@/include/os/linux/kernel \ -I @top_srcdir@/include/os/linux/spl \ -I @top_srcdir@/include/os/linux/zfs \ -I @top_srcdir@/include \ - avl icp lua nvpair spl unicode zcommon zfs zstd os/linux + avl icp lua nvpair unicode zcommon zfs zstd os/linux cppcheck-FreeBSD: @true @@ -141,9 +161,11 @@ cppcheck-FreeBSD: cppcheck: cppcheck-@ac_system@ distdir: - (cd @srcdir@ && find $(ZFS_MODULES) os -name '*.[chS]') | \ - while read path; do \ - mkdir -p $$distdir/$${path%/*}; \ - cp @srcdir@/$$path $$distdir/$$path; \ - done; \ + cd @srcdir@ && find . -name '*.[chS]' -exec sh -c 'for f; do mkdir -p $$distdir/$${f%/*}; cp @srcdir@/$$f $$distdir/$$f; done' _ {} + cp @srcdir@/Makefile.bsd $$distdir/Makefile.bsd + +gen-zstd-symbols: + for obj in $(addprefix zstd/,$(ZSTD_UPSTREAM_OBJS)); do echo; echo "/* $${obj#zstd/}: */"; @OBJDUMP@ -t $$obj | awk '$$2 == "g" && !/ zfs_/ {print "#define\t" $$6 " zfs_" $$6}' | sort; done >> zstd/include/zstd_compat_wrapper.h + +check-zstd-symbols: + @OBJDUMP@ -t $(addprefix zstd/,$(ZSTD_UPSTREAM_OBJS)) | awk '/file format/ {print} $$2 == "g" && !/ zfs_/ {++ret; print} END {exit ret}' diff --git a/module/avl/Makefile.in b/module/avl/Makefile.in deleted file mode 100644 index 991d5f95b8c0..000000000000 --- a/module/avl/Makefile.in +++ /dev/null @@ -1,10 +0,0 @@ -ifneq ($(KBUILD_EXTMOD),) -src = @abs_srcdir@ -obj = @abs_builddir@ -endif - -MODULE := zavl - -obj-$(CONFIG_ZFS) := $(MODULE).o - -$(MODULE)-objs += avl.o diff --git a/module/avl/avl.c b/module/avl/avl.c index 3891a2d62880..b788ed28a4d5 100644 --- a/module/avl/avl.c +++ b/module/avl/avl.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -108,21 +108,6 @@ #include #include -/* - * Small arrays to translate between balance (or diff) values and child indices. - * - * Code that deals with binary tree data structures will randomly use - * left and right children when examining a tree. C "if()" statements - * which evaluate randomly suffer from very poor hardware branch prediction. - * In this code we avoid some of the branch mispredictions by using the - * following translation arrays. They replace random branches with an - * additional memory reference. Since the translation arrays are both very - * small the data should remain efficiently in cache. - */ -static const int avl_child2balance[] = {-1, 1}; -static const int avl_balance2child[] = {0, 0, 1}; - - /* * Walk from one node to the previous valued node (ie. an infix walk * towards the left). At any given node we do one of 2 things: @@ -278,8 +263,7 @@ avl_find(avl_tree_t *tree, const void *value, avl_index_t *where) #endif return (AVL_NODE2DATA(node, off)); } - child = avl_balance2child[1 + diff]; - + child = (diff > 0); } if (where != NULL) @@ -527,7 +511,7 @@ avl_insert(avl_tree_t *tree, void *new_data, avl_index_t where) * Compute the new balance */ old_balance = AVL_XBALANCE(node); - new_balance = old_balance + avl_child2balance[which_child]; + new_balance = old_balance + (which_child ? 1 : -1); /* * If we introduced equal balance, then we are done immediately @@ -693,7 +677,7 @@ avl_remove(avl_tree_t *tree, void *data) * choose node to swap from whichever side is taller */ old_balance = AVL_XBALANCE(delete); - left = avl_balance2child[old_balance + 1]; + left = (old_balance > 0); right = 1 - left; /* @@ -777,7 +761,7 @@ avl_remove(avl_tree_t *tree, void *data) */ node = parent; old_balance = AVL_XBALANCE(node); - new_balance = old_balance - avl_child2balance[which_child]; + new_balance = old_balance - (which_child ? 1 : -1); parent = AVL_XPARENT(node); which_child = AVL_XCHILD(node); @@ -1044,28 +1028,6 @@ avl_destroy_nodes(avl_tree_t *tree, void **cookie) return (AVL_NODE2DATA(node, off)); } -#if defined(_KERNEL) - -static int __init -avl_init(void) -{ - return (0); -} - -static void __exit -avl_fini(void) -{ -} - -module_init(avl_init); -module_exit(avl_fini); -#endif - -ZFS_MODULE_DESCRIPTION("Generic AVL tree implementation"); -ZFS_MODULE_AUTHOR(ZFS_META_AUTHOR); -ZFS_MODULE_LICENSE(ZFS_META_LICENSE); -ZFS_MODULE_VERSION(ZFS_META_VERSION "-" ZFS_META_RELEASE); - EXPORT_SYMBOL(avl_create); EXPORT_SYMBOL(avl_find); EXPORT_SYMBOL(avl_insert); diff --git a/module/icp/Makefile.in b/module/icp/Makefile.in deleted file mode 100644 index f51fcac6d9e1..000000000000 --- a/module/icp/Makefile.in +++ /dev/null @@ -1,93 +0,0 @@ -ifneq ($(KBUILD_EXTMOD),) -src = @abs_srcdir@ -obj = @abs_builddir@ -icp_include = $(src)/include -else -icp_include = $(srctree)/$(src)/include -endif - -MODULE := icp - -obj-$(CONFIG_ZFS) := $(MODULE).o - -asflags-y := -I$(icp_include) -ccflags-y := -I$(icp_include) - -$(MODULE)-objs += illumos-crypto.o -$(MODULE)-objs += api/kcf_cipher.o -$(MODULE)-objs += api/kcf_digest.o -$(MODULE)-objs += api/kcf_mac.o -$(MODULE)-objs += api/kcf_miscapi.o -$(MODULE)-objs += api/kcf_ctxops.o -$(MODULE)-objs += core/kcf_callprov.o -$(MODULE)-objs += core/kcf_prov_tabs.o -$(MODULE)-objs += core/kcf_sched.o -$(MODULE)-objs += core/kcf_mech_tabs.o -$(MODULE)-objs += core/kcf_prov_lib.o -$(MODULE)-objs += spi/kcf_spi.o -$(MODULE)-objs += io/aes.o -$(MODULE)-objs += io/sha2_mod.o -$(MODULE)-objs += io/skein_mod.o -$(MODULE)-objs += os/modhash.o -$(MODULE)-objs += algs/modes/cbc.o -$(MODULE)-objs += algs/modes/ccm.o -$(MODULE)-objs += algs/modes/ctr.o -$(MODULE)-objs += algs/modes/ecb.o -$(MODULE)-objs += algs/modes/gcm_generic.o -$(MODULE)-objs += algs/modes/gcm.o -$(MODULE)-objs += algs/modes/modes.o -$(MODULE)-objs += algs/aes/aes_impl_generic.o -$(MODULE)-objs += algs/aes/aes_impl.o -$(MODULE)-objs += algs/aes/aes_modes.o -$(MODULE)-objs += algs/edonr/edonr.o -$(MODULE)-objs += algs/sha2/sha2.o -$(MODULE)-objs += algs/skein/skein.o -$(MODULE)-objs += algs/skein/skein_block.o -$(MODULE)-objs += algs/skein/skein_iv.o - -$(MODULE)-$(CONFIG_X86_64) += asm-x86_64/aes/aeskey.o -$(MODULE)-$(CONFIG_X86_64) += asm-x86_64/aes/aes_amd64.o -$(MODULE)-$(CONFIG_X86_64) += asm-x86_64/aes/aes_aesni.o -$(MODULE)-$(CONFIG_X86_64) += asm-x86_64/modes/gcm_pclmulqdq.o -$(MODULE)-$(CONFIG_X86_64) += asm-x86_64/modes/aesni-gcm-x86_64.o -$(MODULE)-$(CONFIG_X86_64) += asm-x86_64/modes/ghash-x86_64.o -$(MODULE)-$(CONFIG_X86_64) += asm-x86_64/sha2/sha256_impl.o -$(MODULE)-$(CONFIG_X86_64) += asm-x86_64/sha2/sha512_impl.o - -$(MODULE)-$(CONFIG_X86) += algs/modes/gcm_pclmulqdq.o -$(MODULE)-$(CONFIG_X86) += algs/aes/aes_impl_aesni.o -$(MODULE)-$(CONFIG_X86) += algs/aes/aes_impl_x86-64.o - -# Suppress objtool "can't find jump dest instruction at" warnings. They -# are caused by the constants which are defined in the text section of the -# assembly file using .byte instructions (e.g. bswap_mask). The objtool -# utility tries to interpret them as opcodes and obviously fails doing so. -OBJECT_FILES_NON_STANDARD_aesni-gcm-x86_64.o := y -OBJECT_FILES_NON_STANDARD_ghash-x86_64.o := y -# Suppress objtool "unsupported stack pointer realignment" warnings. We are -# not using a DRAP register while aligning the stack to a 64 byte boundary. -# See #6950 for the reasoning. -OBJECT_FILES_NON_STANDARD_sha256_impl.o := y -OBJECT_FILES_NON_STANDARD_sha512_impl.o := y - -ICP_DIRS = \ - api \ - core \ - spi \ - io \ - os \ - algs \ - algs/aes \ - algs/edonr \ - algs/modes \ - algs/sha2 \ - algs/skein \ - asm-x86_64 \ - asm-x86_64/aes \ - asm-x86_64/modes \ - asm-x86_64/sha2 \ - asm-i386 \ - asm-generic - -all: - mkdir -p $(ICP_DIRS) diff --git a/module/icp/algs/aes/aes_impl.c b/module/icp/algs/aes/aes_impl.c index c238bee2170b..7c92a7c8d136 100644 --- a/module/icp/algs/aes/aes_impl.c +++ b/module/icp/algs/aes/aes_impl.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -47,7 +47,7 @@ aes_init_keysched(const uint8_t *cipherKey, uint_t keyBits, void *keysched) union { uint64_t ka64[4]; uint32_t ka32[8]; - } keyarr; + } keyarr; switch (keyBits) { case 128: @@ -81,7 +81,7 @@ aes_init_keysched(const uint8_t *cipherKey, uint_t keyBits, void *keysched) keyarr.ka64[i] = *((uint64_t *)&cipherKey[j]); } } else { - bcopy(cipherKey, keyarr.ka32, keysize); + memcpy(keyarr.ka32, cipherKey, keysize); } } else { /* byte swap */ @@ -132,7 +132,7 @@ aes_encrypt_block(const void *ks, const uint8_t *pt, uint8_t *ct) buffer[2] = htonl(*(uint32_t *)(void *)&pt[8]); buffer[3] = htonl(*(uint32_t *)(void *)&pt[12]); } else - bcopy(pt, &buffer, AES_BLOCK_LEN); + memcpy(&buffer, pt, AES_BLOCK_LEN); ops->encrypt(&ksch->encr_ks.ks32[0], ksch->nr, buffer, buffer); @@ -143,7 +143,7 @@ aes_encrypt_block(const void *ks, const uint8_t *pt, uint8_t *ct) *(uint32_t *)(void *)&ct[8] = htonl(buffer[2]); *(uint32_t *)(void *)&ct[12] = htonl(buffer[3]); } else - bcopy(&buffer, ct, AES_BLOCK_LEN); + memcpy(ct, &buffer, AES_BLOCK_LEN); } return (CRYPTO_SUCCESS); } @@ -179,7 +179,7 @@ aes_decrypt_block(const void *ks, const uint8_t *ct, uint8_t *pt) buffer[2] = htonl(*(uint32_t *)(void *)&ct[8]); buffer[3] = htonl(*(uint32_t *)(void *)&ct[12]); } else - bcopy(ct, &buffer, AES_BLOCK_LEN); + memcpy(&buffer, ct, AES_BLOCK_LEN); ops->decrypt(&ksch->decr_ks.ks32[0], ksch->nr, buffer, buffer); @@ -190,7 +190,7 @@ aes_decrypt_block(const void *ks, const uint8_t *ct, uint8_t *pt) *(uint32_t *)(void *)&pt[8] = htonl(buffer[2]); *(uint32_t *)(void *)&pt[12] = htonl(buffer[3]); } else - bcopy(&buffer, pt, AES_BLOCK_LEN); + memcpy(pt, &buffer, AES_BLOCK_LEN); } return (CRYPTO_SUCCESS); } @@ -337,7 +337,7 @@ aes_impl_init(void) } static const struct { - char *name; + const char *name; uint32_t sel; } aes_impl_opts[] = { { "cycle", IMPL_CYCLE }, diff --git a/module/icp/algs/aes/aes_impl_aesni.c b/module/icp/algs/aes/aes_impl_aesni.c index 4b5eefd71b17..5bc1bf92dad4 100644 --- a/module/icp/algs/aes/aes_impl_aesni.c +++ b/module/icp/algs/aes/aes_impl_aesni.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/icp/algs/aes/aes_impl_generic.c b/module/icp/algs/aes/aes_impl_generic.c index 427c096c6ab3..ae13c0b85578 100644 --- a/module/icp/algs/aes/aes_impl_generic.c +++ b/module/icp/algs/aes/aes_impl_generic.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/icp/algs/aes/aes_impl_x86-64.c b/module/icp/algs/aes/aes_impl_x86-64.c index 19f8fd5012cf..f4f206a00935 100644 --- a/module/icp/algs/aes/aes_impl_x86-64.c +++ b/module/icp/algs/aes/aes_impl_x86-64.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/icp/algs/aes/aes_modes.c b/module/icp/algs/aes/aes_modes.c index 9e4b498fffcb..6a25496d050e 100644 --- a/module/icp/algs/aes/aes_modes.c +++ b/module/icp/algs/aes/aes_modes.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/icp/algs/blake3/blake3.c b/module/icp/algs/blake3/blake3.c new file mode 100644 index 000000000000..b9600207b673 --- /dev/null +++ b/module/icp/algs/blake3/blake3.c @@ -0,0 +1,732 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Based on BLAKE3 v1.3.1, https://github.com/BLAKE3-team/BLAKE3 + * Copyright (c) 2019-2020 Samuel Neves and Jack O'Connor + * Copyright (c) 2021-2022 Tino Reichardt + */ + +#include +#include + +#include "blake3_impl.h" + +/* + * We need 1056 byte stack for blake3_compress_subtree_wide() + * - we define this pragma to make gcc happy + */ +#if defined(__GNUC__) +#pragma GCC diagnostic ignored "-Wframe-larger-than=" +#endif + +/* internal used */ +typedef struct { + uint32_t input_cv[8]; + uint64_t counter; + uint8_t block[BLAKE3_BLOCK_LEN]; + uint8_t block_len; + uint8_t flags; +} output_t; + +/* internal flags */ +enum blake3_flags { + CHUNK_START = 1 << 0, + CHUNK_END = 1 << 1, + PARENT = 1 << 2, + ROOT = 1 << 3, + KEYED_HASH = 1 << 4, + DERIVE_KEY_CONTEXT = 1 << 5, + DERIVE_KEY_MATERIAL = 1 << 6, +}; + +/* internal start */ +static void chunk_state_init(blake3_chunk_state_t *ctx, + const uint32_t key[8], uint8_t flags) +{ + memcpy(ctx->cv, key, BLAKE3_KEY_LEN); + ctx->chunk_counter = 0; + memset(ctx->buf, 0, BLAKE3_BLOCK_LEN); + ctx->buf_len = 0; + ctx->blocks_compressed = 0; + ctx->flags = flags; +} + +static void chunk_state_reset(blake3_chunk_state_t *ctx, + const uint32_t key[8], uint64_t chunk_counter) +{ + memcpy(ctx->cv, key, BLAKE3_KEY_LEN); + ctx->chunk_counter = chunk_counter; + ctx->blocks_compressed = 0; + memset(ctx->buf, 0, BLAKE3_BLOCK_LEN); + ctx->buf_len = 0; +} + +static size_t chunk_state_len(const blake3_chunk_state_t *ctx) +{ + return (BLAKE3_BLOCK_LEN * (size_t)ctx->blocks_compressed) + + ((size_t)ctx->buf_len); +} + +static size_t chunk_state_fill_buf(blake3_chunk_state_t *ctx, + const uint8_t *input, size_t input_len) +{ + size_t take = BLAKE3_BLOCK_LEN - ((size_t)ctx->buf_len); + if (take > input_len) { + take = input_len; + } + uint8_t *dest = ctx->buf + ((size_t)ctx->buf_len); + memcpy(dest, input, take); + ctx->buf_len += (uint8_t)take; + return (take); +} + +static uint8_t chunk_state_maybe_start_flag(const blake3_chunk_state_t *ctx) +{ + if (ctx->blocks_compressed == 0) { + return (CHUNK_START); + } else { + return (0); + } +} + +static output_t make_output(const uint32_t input_cv[8], + const uint8_t *block, uint8_t block_len, + uint64_t counter, uint8_t flags) +{ + output_t ret; + memcpy(ret.input_cv, input_cv, 32); + memcpy(ret.block, block, BLAKE3_BLOCK_LEN); + ret.block_len = block_len; + ret.counter = counter; + ret.flags = flags; + return (ret); +} + +/* + * Chaining values within a given chunk (specifically the compress_in_place + * interface) are represented as words. This avoids unnecessary bytes<->words + * conversion overhead in the portable implementation. However, the hash_many + * interface handles both user input and parent node blocks, so it accepts + * bytes. For that reason, chaining values in the CV stack are represented as + * bytes. + */ +static void output_chaining_value(const blake3_impl_ops_t *ops, + const output_t *ctx, uint8_t cv[32]) +{ + uint32_t cv_words[8]; + memcpy(cv_words, ctx->input_cv, 32); + ops->compress_in_place(cv_words, ctx->block, ctx->block_len, + ctx->counter, ctx->flags); + store_cv_words(cv, cv_words); +} + +static void output_root_bytes(const blake3_impl_ops_t *ops, const output_t *ctx, + uint64_t seek, uint8_t *out, size_t out_len) +{ + uint64_t output_block_counter = seek / 64; + size_t offset_within_block = seek % 64; + uint8_t wide_buf[64]; + while (out_len > 0) { + ops->compress_xof(ctx->input_cv, ctx->block, ctx->block_len, + output_block_counter, ctx->flags | ROOT, wide_buf); + size_t available_bytes = 64 - offset_within_block; + size_t memcpy_len; + if (out_len > available_bytes) { + memcpy_len = available_bytes; + } else { + memcpy_len = out_len; + } + memcpy(out, wide_buf + offset_within_block, memcpy_len); + out += memcpy_len; + out_len -= memcpy_len; + output_block_counter += 1; + offset_within_block = 0; + } +} + +static void chunk_state_update(const blake3_impl_ops_t *ops, + blake3_chunk_state_t *ctx, const uint8_t *input, size_t input_len) +{ + if (ctx->buf_len > 0) { + size_t take = chunk_state_fill_buf(ctx, input, input_len); + input += take; + input_len -= take; + if (input_len > 0) { + ops->compress_in_place(ctx->cv, ctx->buf, + BLAKE3_BLOCK_LEN, ctx->chunk_counter, + ctx->flags|chunk_state_maybe_start_flag(ctx)); + ctx->blocks_compressed += 1; + ctx->buf_len = 0; + memset(ctx->buf, 0, BLAKE3_BLOCK_LEN); + } + } + + while (input_len > BLAKE3_BLOCK_LEN) { + ops->compress_in_place(ctx->cv, input, BLAKE3_BLOCK_LEN, + ctx->chunk_counter, + ctx->flags|chunk_state_maybe_start_flag(ctx)); + ctx->blocks_compressed += 1; + input += BLAKE3_BLOCK_LEN; + input_len -= BLAKE3_BLOCK_LEN; + } + + size_t take = chunk_state_fill_buf(ctx, input, input_len); + input += take; + input_len -= take; +} + +static output_t chunk_state_output(const blake3_chunk_state_t *ctx) +{ + uint8_t block_flags = + ctx->flags | chunk_state_maybe_start_flag(ctx) | CHUNK_END; + return (make_output(ctx->cv, ctx->buf, ctx->buf_len, ctx->chunk_counter, + block_flags)); +} + +static output_t parent_output(const uint8_t block[BLAKE3_BLOCK_LEN], + const uint32_t key[8], uint8_t flags) +{ + return (make_output(key, block, BLAKE3_BLOCK_LEN, 0, flags | PARENT)); +} + +/* + * Given some input larger than one chunk, return the number of bytes that + * should go in the left subtree. This is the largest power-of-2 number of + * chunks that leaves at least 1 byte for the right subtree. + */ +static size_t left_len(size_t content_len) +{ + /* + * Subtract 1 to reserve at least one byte for the right side. + * content_len + * should always be greater than BLAKE3_CHUNK_LEN. + */ + size_t full_chunks = (content_len - 1) / BLAKE3_CHUNK_LEN; + return (round_down_to_power_of_2(full_chunks) * BLAKE3_CHUNK_LEN); +} + +/* + * Use SIMD parallelism to hash up to MAX_SIMD_DEGREE chunks at the same time + * on a single thread. Write out the chunk chaining values and return the + * number of chunks hashed. These chunks are never the root and never empty; + * those cases use a different codepath. + */ +static size_t compress_chunks_parallel(const blake3_impl_ops_t *ops, + const uint8_t *input, size_t input_len, const uint32_t key[8], + uint64_t chunk_counter, uint8_t flags, uint8_t *out) +{ + const uint8_t *chunks_array[MAX_SIMD_DEGREE]; + size_t input_position = 0; + size_t chunks_array_len = 0; + while (input_len - input_position >= BLAKE3_CHUNK_LEN) { + chunks_array[chunks_array_len] = &input[input_position]; + input_position += BLAKE3_CHUNK_LEN; + chunks_array_len += 1; + } + + ops->hash_many(chunks_array, chunks_array_len, BLAKE3_CHUNK_LEN / + BLAKE3_BLOCK_LEN, key, chunk_counter, B_TRUE, flags, CHUNK_START, + CHUNK_END, out); + + /* + * Hash the remaining partial chunk, if there is one. Note that the + * empty chunk (meaning the empty message) is a different codepath. + */ + if (input_len > input_position) { + uint64_t counter = chunk_counter + (uint64_t)chunks_array_len; + blake3_chunk_state_t chunk_state; + chunk_state_init(&chunk_state, key, flags); + chunk_state.chunk_counter = counter; + chunk_state_update(ops, &chunk_state, &input[input_position], + input_len - input_position); + output_t output = chunk_state_output(&chunk_state); + output_chaining_value(ops, &output, &out[chunks_array_len * + BLAKE3_OUT_LEN]); + return (chunks_array_len + 1); + } else { + return (chunks_array_len); + } +} + +/* + * Use SIMD parallelism to hash up to MAX_SIMD_DEGREE parents at the same time + * on a single thread. Write out the parent chaining values and return the + * number of parents hashed. (If there's an odd input chaining value left over, + * return it as an additional output.) These parents are never the root and + * never empty; those cases use a different codepath. + */ +static size_t compress_parents_parallel(const blake3_impl_ops_t *ops, + const uint8_t *child_chaining_values, size_t num_chaining_values, + const uint32_t key[8], uint8_t flags, uint8_t *out) +{ + const uint8_t *parents_array[MAX_SIMD_DEGREE_OR_2]; + size_t parents_array_len = 0; + + while (num_chaining_values - (2 * parents_array_len) >= 2) { + parents_array[parents_array_len] = &child_chaining_values[2 * + parents_array_len * BLAKE3_OUT_LEN]; + parents_array_len += 1; + } + + ops->hash_many(parents_array, parents_array_len, 1, key, 0, B_FALSE, + flags | PARENT, 0, 0, out); + + /* If there's an odd child left over, it becomes an output. */ + if (num_chaining_values > 2 * parents_array_len) { + memcpy(&out[parents_array_len * BLAKE3_OUT_LEN], + &child_chaining_values[2 * parents_array_len * + BLAKE3_OUT_LEN], BLAKE3_OUT_LEN); + return (parents_array_len + 1); + } else { + return (parents_array_len); + } +} + +/* + * The wide helper function returns (writes out) an array of chaining values + * and returns the length of that array. The number of chaining values returned + * is the dyanmically detected SIMD degree, at most MAX_SIMD_DEGREE. Or fewer, + * if the input is shorter than that many chunks. The reason for maintaining a + * wide array of chaining values going back up the tree, is to allow the + * implementation to hash as many parents in parallel as possible. + * + * As a special case when the SIMD degree is 1, this function will still return + * at least 2 outputs. This guarantees that this function doesn't perform the + * root compression. (If it did, it would use the wrong flags, and also we + * wouldn't be able to implement exendable ouput.) Note that this function is + * not used when the whole input is only 1 chunk long; that's a different + * codepath. + * + * Why not just have the caller split the input on the first update(), instead + * of implementing this special rule? Because we don't want to limit SIMD or + * multi-threading parallelism for that update(). + */ +static size_t blake3_compress_subtree_wide(const blake3_impl_ops_t *ops, + const uint8_t *input, size_t input_len, const uint32_t key[8], + uint64_t chunk_counter, uint8_t flags, uint8_t *out) +{ + /* + * Note that the single chunk case does *not* bump the SIMD degree up + * to 2 when it is 1. If this implementation adds multi-threading in + * the future, this gives us the option of multi-threading even the + * 2-chunk case, which can help performance on smaller platforms. + */ + if (input_len <= (size_t)(ops->degree * BLAKE3_CHUNK_LEN)) { + return (compress_chunks_parallel(ops, input, input_len, key, + chunk_counter, flags, out)); + } + + + /* + * With more than simd_degree chunks, we need to recurse. Start by + * dividing the input into left and right subtrees. (Note that this is + * only optimal as long as the SIMD degree is a power of 2. If we ever + * get a SIMD degree of 3 or something, we'll need a more complicated + * strategy.) + */ + size_t left_input_len = left_len(input_len); + size_t right_input_len = input_len - left_input_len; + const uint8_t *right_input = &input[left_input_len]; + uint64_t right_chunk_counter = chunk_counter + + (uint64_t)(left_input_len / BLAKE3_CHUNK_LEN); + + /* + * Make space for the child outputs. Here we use MAX_SIMD_DEGREE_OR_2 + * to account for the special case of returning 2 outputs when the + * SIMD degree is 1. + */ + uint8_t cv_array[2 * MAX_SIMD_DEGREE_OR_2 * BLAKE3_OUT_LEN]; + size_t degree = ops->degree; + if (left_input_len > BLAKE3_CHUNK_LEN && degree == 1) { + + /* + * The special case: We always use a degree of at least two, + * to make sure there are two outputs. Except, as noted above, + * at the chunk level, where we allow degree=1. (Note that the + * 1-chunk-input case is a different codepath.) + */ + degree = 2; + } + uint8_t *right_cvs = &cv_array[degree * BLAKE3_OUT_LEN]; + + /* + * Recurse! If this implementation adds multi-threading support in the + * future, this is where it will go. + */ + size_t left_n = blake3_compress_subtree_wide(ops, input, left_input_len, + key, chunk_counter, flags, cv_array); + size_t right_n = blake3_compress_subtree_wide(ops, right_input, + right_input_len, key, right_chunk_counter, flags, right_cvs); + + /* + * The special case again. If simd_degree=1, then we'll have left_n=1 + * and right_n=1. Rather than compressing them into a single output, + * return them directly, to make sure we always have at least two + * outputs. + */ + if (left_n == 1) { + memcpy(out, cv_array, 2 * BLAKE3_OUT_LEN); + return (2); + } + + /* Otherwise, do one layer of parent node compression. */ + size_t num_chaining_values = left_n + right_n; + return compress_parents_parallel(ops, cv_array, + num_chaining_values, key, flags, out); +} + +/* + * Hash a subtree with compress_subtree_wide(), and then condense the resulting + * list of chaining values down to a single parent node. Don't compress that + * last parent node, however. Instead, return its message bytes (the + * concatenated chaining values of its children). This is necessary when the + * first call to update() supplies a complete subtree, because the topmost + * parent node of that subtree could end up being the root. It's also necessary + * for extended output in the general case. + * + * As with compress_subtree_wide(), this function is not used on inputs of 1 + * chunk or less. That's a different codepath. + */ +static void compress_subtree_to_parent_node(const blake3_impl_ops_t *ops, + const uint8_t *input, size_t input_len, const uint32_t key[8], + uint64_t chunk_counter, uint8_t flags, uint8_t out[2 * BLAKE3_OUT_LEN]) +{ + uint8_t cv_array[MAX_SIMD_DEGREE_OR_2 * BLAKE3_OUT_LEN]; + size_t num_cvs = blake3_compress_subtree_wide(ops, input, input_len, + key, chunk_counter, flags, cv_array); + + /* + * If MAX_SIMD_DEGREE is greater than 2 and there's enough input, + * compress_subtree_wide() returns more than 2 chaining values. Condense + * them into 2 by forming parent nodes repeatedly. + */ + uint8_t out_array[MAX_SIMD_DEGREE_OR_2 * BLAKE3_OUT_LEN / 2]; + while (num_cvs > 2) { + num_cvs = compress_parents_parallel(ops, cv_array, num_cvs, key, + flags, out_array); + memcpy(cv_array, out_array, num_cvs * BLAKE3_OUT_LEN); + } + memcpy(out, cv_array, 2 * BLAKE3_OUT_LEN); +} + +static void hasher_init_base(BLAKE3_CTX *ctx, const uint32_t key[8], + uint8_t flags) +{ + memcpy(ctx->key, key, BLAKE3_KEY_LEN); + chunk_state_init(&ctx->chunk, key, flags); + ctx->cv_stack_len = 0; + ctx->ops = blake3_impl_get_ops(); +} + +/* + * As described in hasher_push_cv() below, we do "lazy merging", delaying + * merges until right before the next CV is about to be added. This is + * different from the reference implementation. Another difference is that we + * aren't always merging 1 chunk at a time. Instead, each CV might represent + * any power-of-two number of chunks, as long as the smaller-above-larger + * stack order is maintained. Instead of the "count the trailing 0-bits" + * algorithm described in the spec, we use a "count the total number of + * 1-bits" variant that doesn't require us to retain the subtree size of the + * CV on top of the stack. The principle is the same: each CV that should + * remain in the stack is represented by a 1-bit in the total number of chunks + * (or bytes) so far. + */ +static void hasher_merge_cv_stack(BLAKE3_CTX *ctx, uint64_t total_len) +{ + size_t post_merge_stack_len = (size_t)popcnt(total_len); + while (ctx->cv_stack_len > post_merge_stack_len) { + uint8_t *parent_node = + &ctx->cv_stack[(ctx->cv_stack_len - 2) * BLAKE3_OUT_LEN]; + output_t output = + parent_output(parent_node, ctx->key, ctx->chunk.flags); + output_chaining_value(ctx->ops, &output, parent_node); + ctx->cv_stack_len -= 1; + } +} + +/* + * In reference_impl.rs, we merge the new CV with existing CVs from the stack + * before pushing it. We can do that because we know more input is coming, so + * we know none of the merges are root. + * + * This setting is different. We want to feed as much input as possible to + * compress_subtree_wide(), without setting aside anything for the chunk_state. + * If the user gives us 64 KiB, we want to parallelize over all 64 KiB at once + * as a single subtree, if at all possible. + * + * This leads to two problems: + * 1) This 64 KiB input might be the only call that ever gets made to update. + * In this case, the root node of the 64 KiB subtree would be the root node + * of the whole tree, and it would need to be ROOT finalized. We can't + * compress it until we know. + * 2) This 64 KiB input might complete a larger tree, whose root node is + * similarly going to be the the root of the whole tree. For example, maybe + * we have 196 KiB (that is, 128 + 64) hashed so far. We can't compress the + * node at the root of the 256 KiB subtree until we know how to finalize it. + * + * The second problem is solved with "lazy merging". That is, when we're about + * to add a CV to the stack, we don't merge it with anything first, as the + * reference impl does. Instead we do merges using the *previous* CV that was + * added, which is sitting on top of the stack, and we put the new CV + * (unmerged) on top of the stack afterwards. This guarantees that we never + * merge the root node until finalize(). + * + * Solving the first problem requires an additional tool, + * compress_subtree_to_parent_node(). That function always returns the top + * *two* chaining values of the subtree it's compressing. We then do lazy + * merging with each of them separately, so that the second CV will always + * remain unmerged. (That also helps us support extendable output when we're + * hashing an input all-at-once.) + */ +static void hasher_push_cv(BLAKE3_CTX *ctx, uint8_t new_cv[BLAKE3_OUT_LEN], + uint64_t chunk_counter) +{ + hasher_merge_cv_stack(ctx, chunk_counter); + memcpy(&ctx->cv_stack[ctx->cv_stack_len * BLAKE3_OUT_LEN], new_cv, + BLAKE3_OUT_LEN); + ctx->cv_stack_len += 1; +} + +void +Blake3_Init(BLAKE3_CTX *ctx) +{ + hasher_init_base(ctx, BLAKE3_IV, 0); +} + +void +Blake3_InitKeyed(BLAKE3_CTX *ctx, const uint8_t key[BLAKE3_KEY_LEN]) +{ + uint32_t key_words[8]; + load_key_words(key, key_words); + hasher_init_base(ctx, key_words, KEYED_HASH); +} + +static void +Blake3_Update2(BLAKE3_CTX *ctx, const void *input, size_t input_len) +{ + /* + * Explicitly checking for zero avoids causing UB by passing a null + * pointer to memcpy. This comes up in practice with things like: + * std::vector v; + * blake3_hasher_update(&hasher, v.data(), v.size()); + */ + if (input_len == 0) { + return; + } + + const uint8_t *input_bytes = (const uint8_t *)input; + + /* + * If we have some partial chunk bytes in the internal chunk_state, we + * need to finish that chunk first. + */ + if (chunk_state_len(&ctx->chunk) > 0) { + size_t take = BLAKE3_CHUNK_LEN - chunk_state_len(&ctx->chunk); + if (take > input_len) { + take = input_len; + } + chunk_state_update(ctx->ops, &ctx->chunk, input_bytes, take); + input_bytes += take; + input_len -= take; + /* + * If we've filled the current chunk and there's more coming, + * finalize this chunk and proceed. In this case we know it's + * not the root. + */ + if (input_len > 0) { + output_t output = chunk_state_output(&ctx->chunk); + uint8_t chunk_cv[32]; + output_chaining_value(ctx->ops, &output, chunk_cv); + hasher_push_cv(ctx, chunk_cv, ctx->chunk.chunk_counter); + chunk_state_reset(&ctx->chunk, ctx->key, + ctx->chunk.chunk_counter + 1); + } else { + return; + } + } + + /* + * Now the chunk_state is clear, and we have more input. If there's + * more than a single chunk (so, definitely not the root chunk), hash + * the largest whole subtree we can, with the full benefits of SIMD + * (and maybe in the future, multi-threading) parallelism. Two + * restrictions: + * - The subtree has to be a power-of-2 number of chunks. Only + * subtrees along the right edge can be incomplete, and we don't know + * where the right edge is going to be until we get to finalize(). + * - The subtree must evenly divide the total number of chunks up + * until this point (if total is not 0). If the current incomplete + * subtree is only waiting for 1 more chunk, we can't hash a subtree + * of 4 chunks. We have to complete the current subtree first. + * Because we might need to break up the input to form powers of 2, or + * to evenly divide what we already have, this part runs in a loop. + */ + while (input_len > BLAKE3_CHUNK_LEN) { + size_t subtree_len = round_down_to_power_of_2(input_len); + uint64_t count_so_far = + ctx->chunk.chunk_counter * BLAKE3_CHUNK_LEN; + /* + * Shrink the subtree_len until it evenly divides the count so + * far. We know that subtree_len itself is a power of 2, so we + * can use a bitmasking trick instead of an actual remainder + * operation. (Note that if the caller consistently passes + * power-of-2 inputs of the same size, as is hopefully + * typical, this loop condition will always fail, and + * subtree_len will always be the full length of the input.) + * + * An aside: We don't have to shrink subtree_len quite this + * much. For example, if count_so_far is 1, we could pass 2 + * chunks to compress_subtree_to_parent_node. Since we'll get + * 2 CVs back, we'll still get the right answer in the end, + * and we might get to use 2-way SIMD parallelism. The problem + * with this optimization, is that it gets us stuck always + * hashing 2 chunks. The total number of chunks will remain + * odd, and we'll never graduate to higher degrees of + * parallelism. See + * https://github.com/BLAKE3-team/BLAKE3/issues/69. + */ + while ((((uint64_t)(subtree_len - 1)) & count_so_far) != 0) { + subtree_len /= 2; + } + /* + * The shrunken subtree_len might now be 1 chunk long. If so, + * hash that one chunk by itself. Otherwise, compress the + * subtree into a pair of CVs. + */ + uint64_t subtree_chunks = subtree_len / BLAKE3_CHUNK_LEN; + if (subtree_len <= BLAKE3_CHUNK_LEN) { + blake3_chunk_state_t chunk_state; + chunk_state_init(&chunk_state, ctx->key, + ctx->chunk.flags); + chunk_state.chunk_counter = ctx->chunk.chunk_counter; + chunk_state_update(ctx->ops, &chunk_state, input_bytes, + subtree_len); + output_t output = chunk_state_output(&chunk_state); + uint8_t cv[BLAKE3_OUT_LEN]; + output_chaining_value(ctx->ops, &output, cv); + hasher_push_cv(ctx, cv, chunk_state.chunk_counter); + } else { + /* + * This is the high-performance happy path, though + * getting here depends on the caller giving us a long + * enough input. + */ + uint8_t cv_pair[2 * BLAKE3_OUT_LEN]; + compress_subtree_to_parent_node(ctx->ops, input_bytes, + subtree_len, ctx->key, ctx-> chunk.chunk_counter, + ctx->chunk.flags, cv_pair); + hasher_push_cv(ctx, cv_pair, ctx->chunk.chunk_counter); + hasher_push_cv(ctx, &cv_pair[BLAKE3_OUT_LEN], + ctx->chunk.chunk_counter + (subtree_chunks / 2)); + } + ctx->chunk.chunk_counter += subtree_chunks; + input_bytes += subtree_len; + input_len -= subtree_len; + } + + /* + * If there's any remaining input less than a full chunk, add it to + * the chunk state. In that case, also do a final merge loop to make + * sure the subtree stack doesn't contain any unmerged pairs. The + * remaining input means we know these merges are non-root. This merge + * loop isn't strictly necessary here, because hasher_push_chunk_cv + * already does its own merge loop, but it simplifies + * blake3_hasher_finalize below. + */ + if (input_len > 0) { + chunk_state_update(ctx->ops, &ctx->chunk, input_bytes, + input_len); + hasher_merge_cv_stack(ctx, ctx->chunk.chunk_counter); + } +} + +void +Blake3_Update(BLAKE3_CTX *ctx, const void *input, size_t todo) +{ + size_t done = 0; + const uint8_t *data = input; + const size_t block_max = 1024 * 64; + + /* max feed buffer to leave the stack size small */ + while (todo != 0) { + size_t block = (todo >= block_max) ? block_max : todo; + Blake3_Update2(ctx, data + done, block); + done += block; + todo -= block; + } +} + +void +Blake3_Final(const BLAKE3_CTX *ctx, uint8_t *out) +{ + Blake3_FinalSeek(ctx, 0, out, BLAKE3_OUT_LEN); +} + +void +Blake3_FinalSeek(const BLAKE3_CTX *ctx, uint64_t seek, uint8_t *out, + size_t out_len) +{ + /* + * Explicitly checking for zero avoids causing UB by passing a null + * pointer to memcpy. This comes up in practice with things like: + * std::vector v; + * blake3_hasher_finalize(&hasher, v.data(), v.size()); + */ + if (out_len == 0) { + return; + } + /* If the subtree stack is empty, then the current chunk is the root. */ + if (ctx->cv_stack_len == 0) { + output_t output = chunk_state_output(&ctx->chunk); + output_root_bytes(ctx->ops, &output, seek, out, out_len); + return; + } + /* + * If there are any bytes in the chunk state, finalize that chunk and + * do a roll-up merge between that chunk hash and every subtree in the + * stack. In this case, the extra merge loop at the end of + * blake3_hasher_update guarantees that none of the subtrees in the + * stack need to be merged with each other first. Otherwise, if there + * are no bytes in the chunk state, then the top of the stack is a + * chunk hash, and we start the merge from that. + */ + output_t output; + size_t cvs_remaining; + if (chunk_state_len(&ctx->chunk) > 0) { + cvs_remaining = ctx->cv_stack_len; + output = chunk_state_output(&ctx->chunk); + } else { + /* There are always at least 2 CVs in the stack in this case. */ + cvs_remaining = ctx->cv_stack_len - 2; + output = parent_output(&ctx->cv_stack[cvs_remaining * 32], + ctx->key, ctx->chunk.flags); + } + while (cvs_remaining > 0) { + cvs_remaining -= 1; + uint8_t parent_block[BLAKE3_BLOCK_LEN]; + memcpy(parent_block, &ctx->cv_stack[cvs_remaining * 32], 32); + output_chaining_value(ctx->ops, &output, &parent_block[32]); + output = parent_output(parent_block, ctx->key, + ctx->chunk.flags); + } + output_root_bytes(ctx->ops, &output, seek, out, out_len); +} diff --git a/module/icp/algs/blake3/blake3_generic.c b/module/icp/algs/blake3/blake3_generic.c new file mode 100644 index 000000000000..6c1eb33e89c0 --- /dev/null +++ b/module/icp/algs/blake3/blake3_generic.c @@ -0,0 +1,202 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Based on BLAKE3 v1.3.1, https://github.com/BLAKE3-team/BLAKE3 + * Copyright (c) 2019-2020 Samuel Neves and Jack O'Connor + * Copyright (c) 2021-2022 Tino Reichardt + */ + +#include +#include "blake3_impl.h" + +#define rotr32(x, n) (((x) >> (n)) | ((x) << (32 - (n)))) +static inline void g(uint32_t *state, size_t a, size_t b, size_t c, size_t d, + uint32_t x, uint32_t y) +{ + state[a] = state[a] + state[b] + x; + state[d] = rotr32(state[d] ^ state[a], 16); + state[c] = state[c] + state[d]; + state[b] = rotr32(state[b] ^ state[c], 12); + state[a] = state[a] + state[b] + y; + state[d] = rotr32(state[d] ^ state[a], 8); + state[c] = state[c] + state[d]; + state[b] = rotr32(state[b] ^ state[c], 7); +} + +static inline void round_fn(uint32_t state[16], const uint32_t *msg, + size_t round) +{ + /* Select the message schedule based on the round. */ + const uint8_t *schedule = BLAKE3_MSG_SCHEDULE[round]; + + /* Mix the columns. */ + g(state, 0, 4, 8, 12, msg[schedule[0]], msg[schedule[1]]); + g(state, 1, 5, 9, 13, msg[schedule[2]], msg[schedule[3]]); + g(state, 2, 6, 10, 14, msg[schedule[4]], msg[schedule[5]]); + g(state, 3, 7, 11, 15, msg[schedule[6]], msg[schedule[7]]); + + /* Mix the rows. */ + g(state, 0, 5, 10, 15, msg[schedule[8]], msg[schedule[9]]); + g(state, 1, 6, 11, 12, msg[schedule[10]], msg[schedule[11]]); + g(state, 2, 7, 8, 13, msg[schedule[12]], msg[schedule[13]]); + g(state, 3, 4, 9, 14, msg[schedule[14]], msg[schedule[15]]); +} + +static inline void compress_pre(uint32_t state[16], const uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags) +{ + uint32_t block_words[16]; + block_words[0] = load32(block + 4 * 0); + block_words[1] = load32(block + 4 * 1); + block_words[2] = load32(block + 4 * 2); + block_words[3] = load32(block + 4 * 3); + block_words[4] = load32(block + 4 * 4); + block_words[5] = load32(block + 4 * 5); + block_words[6] = load32(block + 4 * 6); + block_words[7] = load32(block + 4 * 7); + block_words[8] = load32(block + 4 * 8); + block_words[9] = load32(block + 4 * 9); + block_words[10] = load32(block + 4 * 10); + block_words[11] = load32(block + 4 * 11); + block_words[12] = load32(block + 4 * 12); + block_words[13] = load32(block + 4 * 13); + block_words[14] = load32(block + 4 * 14); + block_words[15] = load32(block + 4 * 15); + + state[0] = cv[0]; + state[1] = cv[1]; + state[2] = cv[2]; + state[3] = cv[3]; + state[4] = cv[4]; + state[5] = cv[5]; + state[6] = cv[6]; + state[7] = cv[7]; + state[8] = BLAKE3_IV[0]; + state[9] = BLAKE3_IV[1]; + state[10] = BLAKE3_IV[2]; + state[11] = BLAKE3_IV[3]; + state[12] = counter_low(counter); + state[13] = counter_high(counter); + state[14] = (uint32_t)block_len; + state[15] = (uint32_t)flags; + + round_fn(state, &block_words[0], 0); + round_fn(state, &block_words[0], 1); + round_fn(state, &block_words[0], 2); + round_fn(state, &block_words[0], 3); + round_fn(state, &block_words[0], 4); + round_fn(state, &block_words[0], 5); + round_fn(state, &block_words[0], 6); +} + +static inline void blake3_compress_in_place_generic(uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags) +{ + uint32_t state[16]; + compress_pre(state, cv, block, block_len, counter, flags); + cv[0] = state[0] ^ state[8]; + cv[1] = state[1] ^ state[9]; + cv[2] = state[2] ^ state[10]; + cv[3] = state[3] ^ state[11]; + cv[4] = state[4] ^ state[12]; + cv[5] = state[5] ^ state[13]; + cv[6] = state[6] ^ state[14]; + cv[7] = state[7] ^ state[15]; +} + +static inline void hash_one_generic(const uint8_t *input, size_t blocks, + const uint32_t key[8], uint64_t counter, uint8_t flags, + uint8_t flags_start, uint8_t flags_end, uint8_t out[BLAKE3_OUT_LEN]) +{ + uint32_t cv[8]; + memcpy(cv, key, BLAKE3_KEY_LEN); + uint8_t block_flags = flags | flags_start; + while (blocks > 0) { + if (blocks == 1) { + block_flags |= flags_end; + } + blake3_compress_in_place_generic(cv, input, BLAKE3_BLOCK_LEN, + counter, block_flags); + input = &input[BLAKE3_BLOCK_LEN]; + blocks -= 1; + block_flags = flags; + } + store_cv_words(out, cv); +} + +static inline void blake3_compress_xof_generic(const uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags, uint8_t out[64]) +{ + uint32_t state[16]; + compress_pre(state, cv, block, block_len, counter, flags); + + store32(&out[0 * 4], state[0] ^ state[8]); + store32(&out[1 * 4], state[1] ^ state[9]); + store32(&out[2 * 4], state[2] ^ state[10]); + store32(&out[3 * 4], state[3] ^ state[11]); + store32(&out[4 * 4], state[4] ^ state[12]); + store32(&out[5 * 4], state[5] ^ state[13]); + store32(&out[6 * 4], state[6] ^ state[14]); + store32(&out[7 * 4], state[7] ^ state[15]); + store32(&out[8 * 4], state[8] ^ cv[0]); + store32(&out[9 * 4], state[9] ^ cv[1]); + store32(&out[10 * 4], state[10] ^ cv[2]); + store32(&out[11 * 4], state[11] ^ cv[3]); + store32(&out[12 * 4], state[12] ^ cv[4]); + store32(&out[13 * 4], state[13] ^ cv[5]); + store32(&out[14 * 4], state[14] ^ cv[6]); + store32(&out[15 * 4], state[15] ^ cv[7]); +} + +static inline void blake3_hash_many_generic(const uint8_t * const *inputs, + size_t num_inputs, size_t blocks, const uint32_t key[8], uint64_t counter, + boolean_t increment_counter, uint8_t flags, uint8_t flags_start, + uint8_t flags_end, uint8_t *out) +{ + while (num_inputs > 0) { + hash_one_generic(inputs[0], blocks, key, counter, flags, + flags_start, flags_end, out); + if (increment_counter) { + counter += 1; + } + inputs += 1; + num_inputs -= 1; + out = &out[BLAKE3_OUT_LEN]; + } +} + +static inline boolean_t blake3_is_generic_supported(void) +{ + return (B_TRUE); +} + +const blake3_impl_ops_t blake3_generic_impl = { + .compress_in_place = blake3_compress_in_place_generic, + .compress_xof = blake3_compress_xof_generic, + .hash_many = blake3_hash_many_generic, + .is_supported = blake3_is_generic_supported, + .degree = 4, + .name = "generic" +}; diff --git a/module/icp/algs/blake3/blake3_impl.c b/module/icp/algs/blake3/blake3_impl.c new file mode 100644 index 000000000000..10741c82de7e --- /dev/null +++ b/module/icp/algs/blake3/blake3_impl.c @@ -0,0 +1,284 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2021-2022 Tino Reichardt + */ + +#include +#include + +#include "blake3_impl.h" + +static const blake3_impl_ops_t *const blake3_impls[] = { + &blake3_generic_impl, +#if defined(__aarch64__) || \ + (defined(__x86_64) && defined(HAVE_SSE2)) || \ + (defined(__PPC64__) && defined(__LITTLE_ENDIAN__)) + &blake3_sse2_impl, +#endif +#if defined(__aarch64__) || \ + (defined(__x86_64) && defined(HAVE_SSE4_1)) || \ + (defined(__PPC64__) && defined(__LITTLE_ENDIAN__)) + &blake3_sse41_impl, +#endif +#if defined(__x86_64) && defined(HAVE_SSE4_1) && defined(HAVE_AVX2) + &blake3_avx2_impl, +#endif +#if defined(__x86_64) && defined(HAVE_AVX512F) && defined(HAVE_AVX512VL) + &blake3_avx512_impl, +#endif +}; + +/* this pointer holds current ops for implementation */ +static const blake3_impl_ops_t *blake3_selected_impl = &blake3_generic_impl; + +/* special implementation selections */ +#define IMPL_FASTEST (UINT32_MAX) +#define IMPL_CYCLE (UINT32_MAX-1) +#define IMPL_USER (UINT32_MAX-2) +#define IMPL_PARAM (UINT32_MAX-3) + +#define IMPL_READ(i) (*(volatile uint32_t *) &(i)) +static uint32_t icp_blake3_impl = IMPL_FASTEST; + +#define BLAKE3_IMPL_NAME_MAX 16 + +/* id of fastest implementation */ +static uint32_t blake3_fastest_id = 0; + +/* currently used id */ +static uint32_t blake3_current_id = 0; + +/* id of module parameter (-1 == unused) */ +static int blake3_param_id = -1; + +/* return number of supported implementations */ +int +blake3_get_impl_count(void) +{ + static int impls = 0; + int i; + + if (impls) + return (impls); + + for (i = 0; i < ARRAY_SIZE(blake3_impls); i++) { + if (!blake3_impls[i]->is_supported()) continue; + impls++; + } + + return (impls); +} + +/* return id of selected implementation */ +int +blake3_get_impl_id(void) +{ + return (blake3_current_id); +} + +/* return name of selected implementation */ +const char * +blake3_get_impl_name(void) +{ + return (blake3_selected_impl->name); +} + +/* setup id as fastest implementation */ +void +blake3_set_impl_fastest(uint32_t id) +{ + blake3_fastest_id = id; +} + +/* set implementation by id */ +void +blake3_set_impl_id(uint32_t id) +{ + int i, cid; + + /* select fastest */ + if (id == IMPL_FASTEST) + id = blake3_fastest_id; + + /* select next or first */ + if (id == IMPL_CYCLE) + id = (++blake3_current_id) % blake3_get_impl_count(); + + /* 0..N for the real impl */ + for (i = 0, cid = 0; i < ARRAY_SIZE(blake3_impls); i++) { + if (!blake3_impls[i]->is_supported()) continue; + if (cid == id) { + blake3_current_id = cid; + blake3_selected_impl = blake3_impls[i]; + return; + } + cid++; + } +} + +/* set implementation by name */ +int +blake3_set_impl_name(const char *name) +{ + int i, cid; + + if (strcmp(name, "fastest") == 0) { + atomic_swap_32(&icp_blake3_impl, IMPL_FASTEST); + blake3_set_impl_id(IMPL_FASTEST); + return (0); + } else if (strcmp(name, "cycle") == 0) { + atomic_swap_32(&icp_blake3_impl, IMPL_CYCLE); + blake3_set_impl_id(IMPL_CYCLE); + return (0); + } + + for (i = 0, cid = 0; i < ARRAY_SIZE(blake3_impls); i++) { + if (!blake3_impls[i]->is_supported()) continue; + if (strcmp(name, blake3_impls[i]->name) == 0) { + if (icp_blake3_impl == IMPL_PARAM) { + blake3_param_id = cid; + return (0); + } + blake3_selected_impl = blake3_impls[i]; + blake3_current_id = cid; + return (0); + } + cid++; + } + + return (-EINVAL); +} + +/* setup implementation */ +void +blake3_setup_impl(void) +{ + switch (IMPL_READ(icp_blake3_impl)) { + case IMPL_PARAM: + blake3_set_impl_id(blake3_param_id); + atomic_swap_32(&icp_blake3_impl, IMPL_USER); + break; + case IMPL_FASTEST: + blake3_set_impl_id(IMPL_FASTEST); + break; + case IMPL_CYCLE: + blake3_set_impl_id(IMPL_CYCLE); + break; + default: + blake3_set_impl_id(blake3_current_id); + break; + } +} + +/* return selected implementation */ +const blake3_impl_ops_t * +blake3_impl_get_ops(void) +{ + /* each call to ops will cycle */ + if (icp_blake3_impl == IMPL_CYCLE) + blake3_set_impl_id(IMPL_CYCLE); + + return (blake3_selected_impl); +} + +#if defined(_KERNEL) +void **blake3_per_cpu_ctx; + +void +blake3_per_cpu_ctx_init(void) +{ + /* + * Create "The Godfather" ptr to hold all blake3 ctx + */ + blake3_per_cpu_ctx = kmem_alloc(max_ncpus * sizeof (void *), KM_SLEEP); + for (int i = 0; i < max_ncpus; i++) { + blake3_per_cpu_ctx[i] = kmem_alloc(sizeof (BLAKE3_CTX), + KM_SLEEP); + } +} + +void +blake3_per_cpu_ctx_fini(void) +{ + for (int i = 0; i < max_ncpus; i++) { + memset(blake3_per_cpu_ctx[i], 0, sizeof (BLAKE3_CTX)); + kmem_free(blake3_per_cpu_ctx[i], sizeof (BLAKE3_CTX)); + } + memset(blake3_per_cpu_ctx, 0, max_ncpus * sizeof (void *)); + kmem_free(blake3_per_cpu_ctx, max_ncpus * sizeof (void *)); +} +#endif + +#if defined(_KERNEL) && defined(__linux__) +static int +icp_blake3_impl_set(const char *name, zfs_kernel_param_t *kp) +{ + char req_name[BLAKE3_IMPL_NAME_MAX]; + size_t i; + + /* sanitize input */ + i = strnlen(name, BLAKE3_IMPL_NAME_MAX); + if (i == 0 || i >= BLAKE3_IMPL_NAME_MAX) + return (-EINVAL); + + strlcpy(req_name, name, BLAKE3_IMPL_NAME_MAX); + while (i > 0 && isspace(req_name[i-1])) + i--; + req_name[i] = '\0'; + + atomic_swap_32(&icp_blake3_impl, IMPL_PARAM); + return (blake3_set_impl_name(req_name)); +} + +static int +icp_blake3_impl_get(char *buffer, zfs_kernel_param_t *kp) +{ + int i, cid, cnt = 0; + char *fmt; + + /* cycling */ + fmt = (icp_blake3_impl == IMPL_CYCLE) ? "[cycle] " : "cycle "; + cnt += sprintf(buffer + cnt, fmt); + + /* fastest one */ + fmt = (icp_blake3_impl == IMPL_FASTEST) ? "[fastest] " : "fastest "; + cnt += sprintf(buffer + cnt, fmt); + + /* user selected */ + for (i = 0, cid = 0; i < ARRAY_SIZE(blake3_impls); i++) { + if (!blake3_impls[i]->is_supported()) continue; + fmt = (icp_blake3_impl == IMPL_USER && + cid == blake3_current_id) ? "[%s] " : "%s "; + cnt += sprintf(buffer + cnt, fmt, blake3_impls[i]->name); + cid++; + } + + buffer[cnt] = 0; + + return (cnt); +} + +module_param_call(icp_blake3_impl, icp_blake3_impl_set, icp_blake3_impl_get, + NULL, 0644); +MODULE_PARM_DESC(icp_blake3_impl, "Select BLAKE3 implementation."); +#endif diff --git a/module/icp/algs/blake3/blake3_impl.h b/module/icp/algs/blake3/blake3_impl.h new file mode 100644 index 000000000000..5254061c7378 --- /dev/null +++ b/module/icp/algs/blake3/blake3_impl.h @@ -0,0 +1,213 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Based on BLAKE3 v1.3.1, https://github.com/BLAKE3-team/BLAKE3 + * Copyright (c) 2019-2020 Samuel Neves and Jack O'Connor + * Copyright (c) 2021-2022 Tino Reichardt + */ + +#ifndef BLAKE3_IMPL_H +#define BLAKE3_IMPL_H + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include + +/* + * Methods used to define BLAKE3 assembler implementations + */ +typedef void (*blake3_compress_in_place_f)(uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], + uint8_t block_len, uint64_t counter, + uint8_t flags); + +typedef void (*blake3_compress_xof_f)(const uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags, uint8_t out[64]); + +typedef void (*blake3_hash_many_f)(const uint8_t * const *inputs, + size_t num_inputs, size_t blocks, const uint32_t key[8], + uint64_t counter, boolean_t increment_counter, uint8_t flags, + uint8_t flags_start, uint8_t flags_end, uint8_t *out); + +typedef boolean_t (*blake3_is_supported_f)(void); + +typedef struct blake3_impl_ops { + blake3_compress_in_place_f compress_in_place; + blake3_compress_xof_f compress_xof; + blake3_hash_many_f hash_many; + blake3_is_supported_f is_supported; + int degree; + const char *name; +} blake3_impl_ops_t; + +/* Return selected BLAKE3 implementation ops */ +extern const blake3_impl_ops_t *blake3_impl_get_ops(void); + +extern const blake3_impl_ops_t blake3_generic_impl; + +#if defined(__aarch64__) || \ + (defined(__x86_64) && defined(HAVE_SSE2)) || \ + (defined(__PPC64__) && defined(__LITTLE_ENDIAN__)) +extern const blake3_impl_ops_t blake3_sse2_impl; +#endif + +#if defined(__aarch64__) || \ + (defined(__x86_64) && defined(HAVE_SSE4_1)) || \ + (defined(__PPC64__) && defined(__LITTLE_ENDIAN__)) +extern const blake3_impl_ops_t blake3_sse41_impl; +#endif + +#if defined(__x86_64) && defined(HAVE_SSE4_1) && defined(HAVE_AVX2) +extern const blake3_impl_ops_t blake3_avx2_impl; +#endif + +#if defined(__x86_64) && defined(HAVE_AVX512F) && defined(HAVE_AVX512VL) +extern const blake3_impl_ops_t blake3_avx512_impl; +#endif + +#if defined(__x86_64) +#define MAX_SIMD_DEGREE 16 +#else +#define MAX_SIMD_DEGREE 4 +#endif + +#define MAX_SIMD_DEGREE_OR_2 (MAX_SIMD_DEGREE > 2 ? MAX_SIMD_DEGREE : 2) + +static const uint32_t BLAKE3_IV[8] = { + 0x6A09E667UL, 0xBB67AE85UL, 0x3C6EF372UL, 0xA54FF53AUL, + 0x510E527FUL, 0x9B05688CUL, 0x1F83D9ABUL, 0x5BE0CD19UL}; + +static const uint8_t BLAKE3_MSG_SCHEDULE[7][16] = { + {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}, + {2, 6, 3, 10, 7, 0, 4, 13, 1, 11, 12, 5, 9, 14, 15, 8}, + {3, 4, 10, 12, 13, 2, 7, 14, 6, 5, 9, 0, 11, 15, 8, 1}, + {10, 7, 12, 9, 14, 3, 13, 15, 4, 0, 11, 2, 5, 8, 1, 6}, + {12, 13, 9, 11, 15, 10, 14, 8, 7, 2, 5, 3, 0, 1, 6, 4}, + {9, 14, 11, 5, 8, 12, 15, 1, 13, 3, 0, 10, 2, 6, 4, 7}, + {11, 15, 5, 0, 1, 9, 8, 6, 14, 10, 2, 12, 3, 4, 7, 13}, +}; + +/* Find index of the highest set bit */ +static inline unsigned int highest_one(uint64_t x) { +#if defined(__GNUC__) || defined(__clang__) + return (63 ^ __builtin_clzll(x)); +#elif defined(_MSC_VER) && defined(IS_X86_64) + unsigned long index; + _BitScanReverse64(&index, x); + return (index); +#elif defined(_MSC_VER) && defined(IS_X86_32) + if (x >> 32) { + unsigned long index; + _BitScanReverse(&index, x >> 32); + return (32 + index); + } else { + unsigned long index; + _BitScanReverse(&index, x); + return (index); + } +#else + unsigned int c = 0; + if (x & 0xffffffff00000000ULL) { x >>= 32; c += 32; } + if (x & 0x00000000ffff0000ULL) { x >>= 16; c += 16; } + if (x & 0x000000000000ff00ULL) { x >>= 8; c += 8; } + if (x & 0x00000000000000f0ULL) { x >>= 4; c += 4; } + if (x & 0x000000000000000cULL) { x >>= 2; c += 2; } + if (x & 0x0000000000000002ULL) { c += 1; } + return (c); +#endif +} + +/* Count the number of 1 bits. */ +static inline unsigned int popcnt(uint64_t x) { + unsigned int count = 0; + + while (x != 0) { + count += 1; + x &= x - 1; + } + + return (count); +} + +/* + * Largest power of two less than or equal to x. + * As a special case, returns 1 when x is 0. + */ +static inline uint64_t round_down_to_power_of_2(uint64_t x) { + return (1ULL << highest_one(x | 1)); +} + +static inline uint32_t counter_low(uint64_t counter) { + return ((uint32_t)counter); +} + +static inline uint32_t counter_high(uint64_t counter) { + return ((uint32_t)(counter >> 32)); +} + +static inline uint32_t load32(const void *src) { + const uint8_t *p = (const uint8_t *)src; + return ((uint32_t)(p[0]) << 0) | ((uint32_t)(p[1]) << 8) | + ((uint32_t)(p[2]) << 16) | ((uint32_t)(p[3]) << 24); +} + +static inline void load_key_words(const uint8_t key[BLAKE3_KEY_LEN], + uint32_t key_words[8]) { + key_words[0] = load32(&key[0 * 4]); + key_words[1] = load32(&key[1 * 4]); + key_words[2] = load32(&key[2 * 4]); + key_words[3] = load32(&key[3 * 4]); + key_words[4] = load32(&key[4 * 4]); + key_words[5] = load32(&key[5 * 4]); + key_words[6] = load32(&key[6 * 4]); + key_words[7] = load32(&key[7 * 4]); +} + +static inline void store32(void *dst, uint32_t w) { + uint8_t *p = (uint8_t *)dst; + p[0] = (uint8_t)(w >> 0); + p[1] = (uint8_t)(w >> 8); + p[2] = (uint8_t)(w >> 16); + p[3] = (uint8_t)(w >> 24); +} + +static inline void store_cv_words(uint8_t bytes_out[32], uint32_t cv_words[8]) { + store32(&bytes_out[0 * 4], cv_words[0]); + store32(&bytes_out[1 * 4], cv_words[1]); + store32(&bytes_out[2 * 4], cv_words[2]); + store32(&bytes_out[3 * 4], cv_words[3]); + store32(&bytes_out[4 * 4], cv_words[4]); + store32(&bytes_out[5 * 4], cv_words[5]); + store32(&bytes_out[6 * 4], cv_words[6]); + store32(&bytes_out[7 * 4], cv_words[7]); +} + +#ifdef __cplusplus +} +#endif + +#endif /* BLAKE3_IMPL_H */ diff --git a/module/icp/algs/blake3/blake3_x86-64.c b/module/icp/algs/blake3/blake3_x86-64.c new file mode 100644 index 000000000000..a7552bdde4a8 --- /dev/null +++ b/module/icp/algs/blake3/blake3_x86-64.c @@ -0,0 +1,248 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2021-2022 Tino Reichardt + */ + +#include "blake3_impl.h" + +#if defined(__aarch64__) || \ + (defined(__x86_64) && defined(HAVE_SSE2)) || \ + (defined(__PPC64__) && defined(__LITTLE_ENDIAN__)) + +extern void zfs_blake3_compress_in_place_sse2(uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags); + +extern void zfs_blake3_compress_xof_sse2(const uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags, uint8_t out[64]); + +extern void zfs_blake3_hash_many_sse2(const uint8_t * const *inputs, + size_t num_inputs, size_t blocks, const uint32_t key[8], + uint64_t counter, boolean_t increment_counter, uint8_t flags, + uint8_t flags_start, uint8_t flags_end, uint8_t *out); + +static void blake3_compress_in_place_sse2(uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags) { + kfpu_begin(); + zfs_blake3_compress_in_place_sse2(cv, block, block_len, counter, + flags); + kfpu_end(); +} + +static void blake3_compress_xof_sse2(const uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags, uint8_t out[64]) { + kfpu_begin(); + zfs_blake3_compress_xof_sse2(cv, block, block_len, counter, flags, + out); + kfpu_end(); +} + +static void blake3_hash_many_sse2(const uint8_t * const *inputs, + size_t num_inputs, size_t blocks, const uint32_t key[8], + uint64_t counter, boolean_t increment_counter, uint8_t flags, + uint8_t flags_start, uint8_t flags_end, uint8_t *out) { + kfpu_begin(); + zfs_blake3_hash_many_sse2(inputs, num_inputs, blocks, key, counter, + increment_counter, flags, flags_start, flags_end, out); + kfpu_end(); +} + +static boolean_t blake3_is_sse2_supported(void) +{ +#if defined(__x86_64) + return (kfpu_allowed() && zfs_sse2_available()); +#elif defined(__PPC64__) + return (kfpu_allowed() && zfs_vsx_available()); +#else + return (kfpu_allowed()); +#endif +} + +const blake3_impl_ops_t blake3_sse2_impl = { + .compress_in_place = blake3_compress_in_place_sse2, + .compress_xof = blake3_compress_xof_sse2, + .hash_many = blake3_hash_many_sse2, + .is_supported = blake3_is_sse2_supported, + .degree = 4, + .name = "sse2" +}; +#endif + +#if defined(__aarch64__) || \ + (defined(__x86_64) && defined(HAVE_SSE2)) || \ + (defined(__PPC64__) && defined(__LITTLE_ENDIAN__)) + +extern void zfs_blake3_compress_in_place_sse41(uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags); + +extern void zfs_blake3_compress_xof_sse41(const uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags, uint8_t out[64]); + +extern void zfs_blake3_hash_many_sse41(const uint8_t * const *inputs, + size_t num_inputs, size_t blocks, const uint32_t key[8], + uint64_t counter, boolean_t increment_counter, uint8_t flags, + uint8_t flags_start, uint8_t flags_end, uint8_t *out); + +static void blake3_compress_in_place_sse41(uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags) { + kfpu_begin(); + zfs_blake3_compress_in_place_sse41(cv, block, block_len, counter, + flags); + kfpu_end(); +} + +static void blake3_compress_xof_sse41(const uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags, uint8_t out[64]) { + kfpu_begin(); + zfs_blake3_compress_xof_sse41(cv, block, block_len, counter, flags, + out); + kfpu_end(); +} + +static void blake3_hash_many_sse41(const uint8_t * const *inputs, + size_t num_inputs, size_t blocks, const uint32_t key[8], + uint64_t counter, boolean_t increment_counter, uint8_t flags, + uint8_t flags_start, uint8_t flags_end, uint8_t *out) { + kfpu_begin(); + zfs_blake3_hash_many_sse41(inputs, num_inputs, blocks, key, counter, + increment_counter, flags, flags_start, flags_end, out); + kfpu_end(); +} + +static boolean_t blake3_is_sse41_supported(void) +{ +#if defined(__x86_64) + return (kfpu_allowed() && zfs_sse4_1_available()); +#elif defined(__PPC64__) + return (kfpu_allowed() && zfs_vsx_available()); +#else + return (kfpu_allowed()); +#endif +} + +const blake3_impl_ops_t blake3_sse41_impl = { + .compress_in_place = blake3_compress_in_place_sse41, + .compress_xof = blake3_compress_xof_sse41, + .hash_many = blake3_hash_many_sse41, + .is_supported = blake3_is_sse41_supported, + .degree = 4, + .name = "sse41" +}; +#endif + +#if defined(__x86_64) && defined(HAVE_SSE4_1) && defined(HAVE_AVX2) +extern void zfs_blake3_hash_many_avx2(const uint8_t * const *inputs, + size_t num_inputs, size_t blocks, const uint32_t key[8], + uint64_t counter, boolean_t increment_counter, uint8_t flags, + uint8_t flags_start, uint8_t flags_end, uint8_t *out); + +static void blake3_hash_many_avx2(const uint8_t * const *inputs, + size_t num_inputs, size_t blocks, const uint32_t key[8], + uint64_t counter, boolean_t increment_counter, uint8_t flags, + uint8_t flags_start, uint8_t flags_end, uint8_t *out) { + kfpu_begin(); + zfs_blake3_hash_many_avx2(inputs, num_inputs, blocks, key, counter, + increment_counter, flags, flags_start, flags_end, out); + kfpu_end(); +} + +static boolean_t blake3_is_avx2_supported(void) +{ + return (kfpu_allowed() && zfs_sse4_1_available() && + zfs_avx2_available()); +} + +const blake3_impl_ops_t blake3_avx2_impl = { + .compress_in_place = blake3_compress_in_place_sse41, + .compress_xof = blake3_compress_xof_sse41, + .hash_many = blake3_hash_many_avx2, + .is_supported = blake3_is_avx2_supported, + .degree = 8, + .name = "avx2" +}; +#endif + +#if defined(__x86_64) && defined(HAVE_AVX512F) && defined(HAVE_AVX512VL) +extern void zfs_blake3_compress_in_place_avx512(uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags); + +extern void zfs_blake3_compress_xof_avx512(const uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags, uint8_t out[64]); + +extern void zfs_blake3_hash_many_avx512(const uint8_t * const *inputs, + size_t num_inputs, size_t blocks, const uint32_t key[8], + uint64_t counter, boolean_t increment_counter, uint8_t flags, + uint8_t flags_start, uint8_t flags_end, uint8_t *out); + +static void blake3_compress_in_place_avx512(uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags) { + kfpu_begin(); + zfs_blake3_compress_in_place_avx512(cv, block, block_len, counter, + flags); + kfpu_end(); +} + +static void blake3_compress_xof_avx512(const uint32_t cv[8], + const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, + uint64_t counter, uint8_t flags, uint8_t out[64]) { + kfpu_begin(); + zfs_blake3_compress_xof_avx512(cv, block, block_len, counter, flags, + out); + kfpu_end(); +} + +static void blake3_hash_many_avx512(const uint8_t * const *inputs, + size_t num_inputs, size_t blocks, const uint32_t key[8], + uint64_t counter, boolean_t increment_counter, uint8_t flags, + uint8_t flags_start, uint8_t flags_end, uint8_t *out) { + kfpu_begin(); + zfs_blake3_hash_many_avx512(inputs, num_inputs, blocks, key, counter, + increment_counter, flags, flags_start, flags_end, out); + kfpu_end(); +} + +static boolean_t blake3_is_avx512_supported(void) +{ + return (kfpu_allowed() && zfs_avx512f_available() && + zfs_avx512vl_available()); +} + +const blake3_impl_ops_t blake3_avx512_impl = { + .compress_in_place = blake3_compress_in_place_avx512, + .compress_xof = blake3_compress_xof_avx512, + .hash_many = blake3_hash_many_avx512, + .is_supported = blake3_is_avx512_supported, + .degree = 16, + .name = "avx512" +}; +#endif diff --git a/module/icp/algs/edonr/edonr.c b/module/icp/algs/edonr/edonr.c index 20418eaa73cf..345133d7433a 100644 --- a/module/icp/algs/edonr/edonr.c +++ b/module/icp/algs/edonr/edonr.c @@ -35,7 +35,7 @@ * cryptographic use. Users of Edon-R must interface directly to this module. */ -#include +#include #include #include @@ -47,10 +47,7 @@ #define hashState384(x) ((x)->pipe->p512) #define hashState512(x) ((x)->pipe->p512) -/* shift and rotate shortcuts */ -#define shl(x, n) ((x) << n) -#define shr(x, n) ((x) >> n) - +/* rotate shortcuts */ #define rotl32(x, n) (((x) << (n)) | ((x) >> (32 - (n)))) #define rotr32(x, n) (((x) >> (n)) | ((x) << (32 - (n)))) @@ -470,32 +467,32 @@ EdonRInit(EdonRState *state, size_t hashbitlen) state->hashbitlen = 224; state->bits_processed = 0; state->unprocessed_bits = 0; - bcopy(i224p2, hashState224(state)->DoublePipe, - 16 * sizeof (uint32_t)); + memcpy(hashState224(state)->DoublePipe, i224p2, + sizeof (i224p2)); break; case 256: state->hashbitlen = 256; state->bits_processed = 0; state->unprocessed_bits = 0; - bcopy(i256p2, hashState256(state)->DoublePipe, - 16 * sizeof (uint32_t)); + memcpy(hashState256(state)->DoublePipe, i256p2, + sizeof (i256p2)); break; case 384: state->hashbitlen = 384; state->bits_processed = 0; state->unprocessed_bits = 0; - bcopy(i384p2, hashState384(state)->DoublePipe, - 16 * sizeof (uint64_t)); + memcpy(hashState384(state)->DoublePipe, i384p2, + sizeof (i384p2)); break; case 512: state->hashbitlen = 512; state->bits_processed = 0; state->unprocessed_bits = 0; - bcopy(i512p2, hashState224(state)->DoublePipe, - 16 * sizeof (uint64_t)); + memcpy(hashState512(state)->DoublePipe, i512p2, + sizeof (i512p2)); break; } } @@ -520,8 +517,9 @@ EdonRUpdate(EdonRState *state, const uint8_t *data, size_t databitlen) ASSERT(state->unprocessed_bits + databitlen <= EdonR256_BLOCK_SIZE * 8); - bcopy(data, hashState256(state)->LastPart - + (state->unprocessed_bits >> 3), LastBytes); + memcpy(hashState256(state)->LastPart + + (state->unprocessed_bits >> 3), + data, LastBytes); state->unprocessed_bits += (int)databitlen; databitlen = state->unprocessed_bits; /* LINTED E_BAD_PTR_CAST_ALIGN */ @@ -542,7 +540,8 @@ EdonRUpdate(EdonRState *state, const uint8_t *data, size_t databitlen) 1) & 0x01ff; data32 += bits_processed >> 5; /* byte size update */ - bcopy(data32, hashState256(state)->LastPart, LastBytes); + memmove(hashState256(state)->LastPart, + data32, LastBytes); } break; @@ -555,8 +554,9 @@ EdonRUpdate(EdonRState *state, const uint8_t *data, size_t databitlen) ASSERT(state->unprocessed_bits + databitlen <= EdonR512_BLOCK_SIZE * 8); - bcopy(data, hashState512(state)->LastPart - + (state->unprocessed_bits >> 3), LastBytes); + memcpy(hashState512(state)->LastPart + + (state->unprocessed_bits >> 3), + data, LastBytes); state->unprocessed_bits += (int)databitlen; databitlen = state->unprocessed_bits; /* LINTED E_BAD_PTR_CAST_ALIGN */ @@ -577,7 +577,8 @@ EdonRUpdate(EdonRState *state, const uint8_t *data, size_t databitlen) 1) & 0x03ff; data64 += bits_processed >> 6; /* byte size update */ - bcopy(data64, hashState512(state)->LastPart, LastBytes); + memmove(hashState512(state)->LastPart, + data64, LastBytes); } break; } @@ -682,7 +683,7 @@ EdonRFinal(EdonRState *state, uint8_t *hashval) for (j = 0; j < EdonR224_DIGEST_SIZE >> 2; j++) st_swap32(s32[j], d32 + j); #else - bcopy(hashState256(state)->DoublePipe + 9, hashval, + memcpy(hashval, hashState256(state)->DoublePipe + 9, EdonR224_DIGEST_SIZE); #endif break; @@ -696,7 +697,7 @@ EdonRFinal(EdonRState *state, uint8_t *hashval) for (j = 0; j < EdonR256_DIGEST_SIZE >> 2; j++) st_swap32(s32[j], d32 + j); #else - bcopy(hashState256(state)->DoublePipe + 8, hashval, + memcpy(hashval, hashState256(state)->DoublePipe + 8, EdonR256_DIGEST_SIZE); #endif break; @@ -710,7 +711,7 @@ EdonRFinal(EdonRState *state, uint8_t *hashval) for (j = 0; j < EdonR384_DIGEST_SIZE >> 3; j++) st_swap64(s64[j], d64 + j); #else - bcopy(hashState384(state)->DoublePipe + 10, hashval, + memcpy(hashval, hashState384(state)->DoublePipe + 10, EdonR384_DIGEST_SIZE); #endif break; @@ -724,7 +725,7 @@ EdonRFinal(EdonRState *state, uint8_t *hashval) for (j = 0; j < EdonR512_DIGEST_SIZE >> 3; j++) st_swap64(s64[j], d64 + j); #else - bcopy(hashState512(state)->DoublePipe + 8, hashval, + memcpy(hashval, hashState512(state)->DoublePipe + 8, EdonR512_DIGEST_SIZE); #endif break; diff --git a/module/icp/algs/modes/cbc.c b/module/icp/algs/modes/cbc.c index bddb5b64ddd3..d0219fb24c49 100644 --- a/module/icp/algs/modes/cbc.c +++ b/module/icp/algs/modes/cbc.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -51,8 +51,8 @@ cbc_encrypt_contiguous_blocks(cbc_ctx_t *ctx, char *data, size_t length, if (length + ctx->cbc_remainder_len < block_size) { /* accumulate bytes here and return */ - bcopy(datap, - (uint8_t *)ctx->cbc_remainder + ctx->cbc_remainder_len, + memcpy((uint8_t *)ctx->cbc_remainder + ctx->cbc_remainder_len, + datap, length); ctx->cbc_remainder_len += length; ctx->cbc_copy_to = datap; @@ -70,8 +70,8 @@ cbc_encrypt_contiguous_blocks(cbc_ctx_t *ctx, char *data, size_t length, if (need > remainder) return (CRYPTO_DATA_LEN_RANGE); - bcopy(datap, &((uint8_t *)ctx->cbc_remainder) - [ctx->cbc_remainder_len], need); + memcpy(&((uint8_t *)ctx->cbc_remainder) + [ctx->cbc_remainder_len], datap, need); blockp = (uint8_t *)ctx->cbc_remainder; } else { @@ -91,10 +91,10 @@ cbc_encrypt_contiguous_blocks(cbc_ctx_t *ctx, char *data, size_t length, if (out_data_1_len == block_size) { copy_block(lastp, out_data_1); } else { - bcopy(lastp, out_data_1, out_data_1_len); + memcpy(out_data_1, lastp, out_data_1_len); if (out_data_2 != NULL) { - bcopy(lastp + out_data_1_len, - out_data_2, + memcpy(out_data_2, + lastp + out_data_1_len, block_size - out_data_1_len); } } @@ -113,7 +113,7 @@ cbc_encrypt_contiguous_blocks(cbc_ctx_t *ctx, char *data, size_t length, /* Incomplete last block. */ if (remainder > 0 && remainder < block_size) { - bcopy(datap, ctx->cbc_remainder, remainder); + memcpy(ctx->cbc_remainder, datap, remainder); ctx->cbc_remainder_len = remainder; ctx->cbc_copy_to = datap; goto out; @@ -157,8 +157,8 @@ cbc_decrypt_contiguous_blocks(cbc_ctx_t *ctx, char *data, size_t length, if (length + ctx->cbc_remainder_len < block_size) { /* accumulate bytes here and return */ - bcopy(datap, - (uint8_t *)ctx->cbc_remainder + ctx->cbc_remainder_len, + memcpy((uint8_t *)ctx->cbc_remainder + ctx->cbc_remainder_len, + datap, length); ctx->cbc_remainder_len += length; ctx->cbc_copy_to = datap; @@ -176,8 +176,8 @@ cbc_decrypt_contiguous_blocks(cbc_ctx_t *ctx, char *data, size_t length, if (need > remainder) return (CRYPTO_ENCRYPTED_DATA_LEN_RANGE); - bcopy(datap, &((uint8_t *)ctx->cbc_remainder) - [ctx->cbc_remainder_len], need); + memcpy(&((uint8_t *)ctx->cbc_remainder) + [ctx->cbc_remainder_len], datap, need); blockp = (uint8_t *)ctx->cbc_remainder; } else { @@ -203,9 +203,9 @@ cbc_decrypt_contiguous_blocks(cbc_ctx_t *ctx, char *data, size_t length, crypto_get_ptrs(out, &iov_or_mp, &offset, &out_data_1, &out_data_1_len, &out_data_2, block_size); - bcopy(blockp, out_data_1, out_data_1_len); + memcpy(out_data_1, blockp, out_data_1_len); if (out_data_2 != NULL) { - bcopy(blockp + out_data_1_len, out_data_2, + memcpy(out_data_2, blockp + out_data_1_len, block_size - out_data_1_len); } @@ -224,7 +224,7 @@ cbc_decrypt_contiguous_blocks(cbc_ctx_t *ctx, char *data, size_t length, /* Incomplete last block. */ if (remainder > 0 && remainder < block_size) { - bcopy(datap, ctx->cbc_remainder, remainder); + memcpy(ctx->cbc_remainder, datap, remainder); ctx->cbc_remainder_len = remainder; ctx->cbc_lastp = lastp; ctx->cbc_copy_to = datap; @@ -242,19 +242,12 @@ int cbc_init_ctx(cbc_ctx_t *cbc_ctx, char *param, size_t param_len, size_t block_size, void (*copy_block)(uint8_t *, uint64_t *)) { - /* - * Copy IV into context. - * - * If cm_param == NULL then the IV comes from the - * cd_miscdata field in the crypto_data structure. - */ - if (param != NULL) { - ASSERT(param_len == block_size); - copy_block((uchar_t *)param, cbc_ctx->cbc_iv); - } + /* Copy IV into context. */ + ASSERT3P(param, !=, NULL); + ASSERT3U(param_len, ==, block_size); + + copy_block((uchar_t *)param, cbc_ctx->cbc_iv); - cbc_ctx->cbc_lastp = (uint8_t *)&cbc_ctx->cbc_iv[0]; - cbc_ctx->cbc_flags |= CBC_MODE; return (CRYPTO_SUCCESS); } diff --git a/module/icp/algs/modes/ccm.c b/module/icp/algs/modes/ccm.c index a41cbc395fd6..ed5498dafaa1 100644 --- a/module/icp/algs/modes/ccm.c +++ b/module/icp/algs/modes/ccm.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -59,8 +59,8 @@ ccm_mode_encrypt_contiguous_blocks(ccm_ctx_t *ctx, char *data, size_t length, if (length + ctx->ccm_remainder_len < block_size) { /* accumulate bytes here and return */ - bcopy(datap, - (uint8_t *)ctx->ccm_remainder + ctx->ccm_remainder_len, + memcpy((uint8_t *)ctx->ccm_remainder + ctx->ccm_remainder_len, + datap, length); ctx->ccm_remainder_len += length; ctx->ccm_copy_to = datap; @@ -80,8 +80,8 @@ ccm_mode_encrypt_contiguous_blocks(ccm_ctx_t *ctx, char *data, size_t length, if (need > remainder) return (CRYPTO_DATA_LEN_RANGE); - bcopy(datap, &((uint8_t *)ctx->ccm_remainder) - [ctx->ccm_remainder_len], need); + memcpy(&((uint8_t *)ctx->ccm_remainder) + [ctx->ccm_remainder_len], datap, need); blockp = (uint8_t *)ctx->ccm_remainder; } else { @@ -132,10 +132,10 @@ ccm_mode_encrypt_contiguous_blocks(ccm_ctx_t *ctx, char *data, size_t length, if (out_data_1_len == block_size) { copy_block(lastp, out_data_1); } else { - bcopy(lastp, out_data_1, out_data_1_len); + memcpy(out_data_1, lastp, out_data_1_len); if (out_data_2 != NULL) { - bcopy(lastp + out_data_1_len, - out_data_2, + memcpy(out_data_2, + lastp + out_data_1_len, block_size - out_data_1_len); } } @@ -154,7 +154,7 @@ ccm_mode_encrypt_contiguous_blocks(ccm_ctx_t *ctx, char *data, size_t length, /* Incomplete last block. */ if (remainder > 0 && remainder < block_size) { - bcopy(datap, ctx->ccm_remainder, remainder); + memcpy(ctx->ccm_remainder, datap, remainder); ctx->ccm_remainder_len = remainder; ctx->ccm_copy_to = datap; goto out; @@ -224,10 +224,10 @@ ccm_encrypt_final(ccm_ctx_t *ctx, crypto_data_t *out, size_t block_size, /* ccm_mac_input_buf is not used for encryption */ macp = (uint8_t *)ctx->ccm_mac_input_buf; - bzero(macp, block_size); + memset(macp, 0, block_size); /* copy remainder to temporary buffer */ - bcopy(ctx->ccm_remainder, macp, ctx->ccm_remainder_len); + memcpy(macp, ctx->ccm_remainder, ctx->ccm_remainder_len); /* calculate the CBC MAC */ xor_block(macp, mac_buf); @@ -254,33 +254,32 @@ ccm_encrypt_final(ccm_ctx_t *ctx, crypto_data_t *out, size_t block_size, ctx->ccm_remainder_len + ctx->ccm_mac_len); if (ctx->ccm_remainder_len > 0) { - /* copy temporary block to where it belongs */ if (out_data_2 == NULL) { /* everything will fit in out_data_1 */ - bcopy(macp, out_data_1, ctx->ccm_remainder_len); - bcopy(ccm_mac_p, out_data_1 + ctx->ccm_remainder_len, + memcpy(out_data_1, macp, ctx->ccm_remainder_len); + memcpy(out_data_1 + ctx->ccm_remainder_len, ccm_mac_p, ctx->ccm_mac_len); } else { - if (out_data_1_len < ctx->ccm_remainder_len) { - size_t data_2_len_used; - bcopy(macp, out_data_1, out_data_1_len); + memcpy(out_data_1, macp, out_data_1_len); data_2_len_used = ctx->ccm_remainder_len - out_data_1_len; - bcopy((uint8_t *)macp + out_data_1_len, - out_data_2, data_2_len_used); - bcopy(ccm_mac_p, out_data_2 + data_2_len_used, + memcpy(out_data_2, + (uint8_t *)macp + out_data_1_len, + data_2_len_used); + memcpy(out_data_2 + data_2_len_used, + ccm_mac_p, ctx->ccm_mac_len); } else { - bcopy(macp, out_data_1, out_data_1_len); + memcpy(out_data_1, macp, out_data_1_len); if (out_data_1_len == ctx->ccm_remainder_len) { /* mac will be in out_data_2 */ - bcopy(ccm_mac_p, out_data_2, + memcpy(out_data_2, ccm_mac_p, ctx->ccm_mac_len); } else { size_t len_not_used = out_data_1_len - @@ -290,11 +289,11 @@ ccm_encrypt_final(ccm_ctx_t *ctx, crypto_data_t *out, size_t block_size, * out_data_1, part of the mac will be * in out_data_2 */ - bcopy(ccm_mac_p, - out_data_1 + ctx->ccm_remainder_len, - len_not_used); - bcopy(ccm_mac_p + len_not_used, - out_data_2, + memcpy(out_data_1 + + ctx->ccm_remainder_len, + ccm_mac_p, len_not_used); + memcpy(out_data_2, + ccm_mac_p + len_not_used, ctx->ccm_mac_len - len_not_used); } @@ -302,9 +301,9 @@ ccm_encrypt_final(ccm_ctx_t *ctx, crypto_data_t *out, size_t block_size, } } else { /* copy block to where it belongs */ - bcopy(ccm_mac_p, out_data_1, out_data_1_len); + memcpy(out_data_1, ccm_mac_p, out_data_1_len); if (out_data_2 != NULL) { - bcopy(ccm_mac_p + out_data_1_len, out_data_2, + memcpy(out_data_2, ccm_mac_p + out_data_1_len, block_size - out_data_1_len); } } @@ -372,7 +371,7 @@ ccm_mode_decrypt_contiguous_blocks(ccm_ctx_t *ctx, char *data, size_t length, } tmp = (uint8_t *)ctx->ccm_mac_input_buf; - bcopy(datap, tmp + pm_len, length); + memcpy(tmp + pm_len, datap, length); ctx->ccm_processed_mac_len += length; return (CRYPTO_SUCCESS); @@ -405,15 +404,15 @@ ccm_mode_decrypt_contiguous_blocks(ccm_ctx_t *ctx, char *data, size_t length, mac_len = length - pt_part; ctx->ccm_processed_mac_len = mac_len; - bcopy(data + pt_part, ctx->ccm_mac_input_buf, mac_len); + memcpy(ctx->ccm_mac_input_buf, data + pt_part, mac_len); if (pt_part + ctx->ccm_remainder_len < block_size) { /* * since this is last of the ciphertext, will * just decrypt with it here */ - bcopy(datap, &((uint8_t *)ctx->ccm_remainder) - [ctx->ccm_remainder_len], pt_part); + memcpy(&((uint8_t *)ctx->ccm_remainder) + [ctx->ccm_remainder_len], datap, pt_part); ctx->ccm_remainder_len += pt_part; ccm_decrypt_incomplete_block(ctx, encrypt_block); ctx->ccm_processed_data_len += ctx->ccm_remainder_len; @@ -424,9 +423,9 @@ ccm_mode_decrypt_contiguous_blocks(ccm_ctx_t *ctx, char *data, size_t length, length = pt_part; } } else if (length + ctx->ccm_remainder_len < block_size) { - /* accumulate bytes here and return */ - bcopy(datap, - (uint8_t *)ctx->ccm_remainder + ctx->ccm_remainder_len, + /* accumulate bytes here and return */ + memcpy((uint8_t *)ctx->ccm_remainder + ctx->ccm_remainder_len, + datap, length); ctx->ccm_remainder_len += length; ctx->ccm_copy_to = datap; @@ -441,8 +440,8 @@ ccm_mode_decrypt_contiguous_blocks(ccm_ctx_t *ctx, char *data, size_t length, if (need > remainder) return (CRYPTO_ENCRYPTED_DATA_LEN_RANGE); - bcopy(datap, &((uint8_t *)ctx->ccm_remainder) - [ctx->ccm_remainder_len], need); + memcpy(&((uint8_t *)ctx->ccm_remainder) + [ctx->ccm_remainder_len], datap, need); blockp = (uint8_t *)ctx->ccm_remainder; } else { @@ -492,7 +491,7 @@ ccm_mode_decrypt_contiguous_blocks(ccm_ctx_t *ctx, char *data, size_t length, /* Incomplete last block */ if (remainder > 0 && remainder < block_size) { - bcopy(datap, ctx->ccm_remainder, remainder); + memcpy(ctx->ccm_remainder, datap, remainder); ctx->ccm_remainder_len = remainder; ctx->ccm_copy_to = datap; if (ctx->ccm_processed_mac_len > 0) { @@ -539,10 +538,9 @@ ccm_decrypt_final(ccm_ctx_t *ctx, crypto_data_t *out, size_t block_size, macp = (uint8_t *)ctx->ccm_tmp; while (mac_remain > 0) { - if (mac_remain < block_size) { - bzero(macp, block_size); - bcopy(pt, macp, mac_remain); + memset(macp, 0, block_size); + memcpy(macp, pt, mac_remain); mac_remain = 0; } else { copy_block(pt, macp); @@ -560,7 +558,7 @@ ccm_decrypt_final(ccm_ctx_t *ctx, crypto_data_t *out, size_t block_size, calculate_ccm_mac((ccm_ctx_t *)ctx, ccm_mac_p, encrypt_block); /* compare the input CCM MAC value with what we calculated */ - if (bcmp(ctx->ccm_mac_input_buf, ccm_mac_p, ctx->ccm_mac_len)) { + if (memcmp(ctx->ccm_mac_input_buf, ccm_mac_p, ctx->ccm_mac_len)) { /* They don't match */ return (CRYPTO_INVALID_MAC); } else { @@ -654,10 +652,10 @@ ccm_format_initial_blocks(uchar_t *nonce, ulong_t nonceSize, b0[0] = (have_adata << 6) | (((t - 2) / 2) << 3) | (q - 1); /* copy the nonce value into b0 */ - bcopy(nonce, &(b0[1]), nonceSize); + memcpy(&(b0[1]), nonce, nonceSize); /* store the length of the payload into b0 */ - bzero(&(b0[1+nonceSize]), q); + memset(&(b0[1+nonceSize]), 0, q); payloadSize = aes_ctx->ccm_data_len; limit = 8 < q ? 8 : q; @@ -673,9 +671,9 @@ ccm_format_initial_blocks(uchar_t *nonce, ulong_t nonceSize, cb[0] = 0x07 & (q-1); /* first byte */ /* copy the nonce value into the counter block */ - bcopy(nonce, &(cb[1]), nonceSize); + memcpy(&(cb[1]), nonce, nonceSize); - bzero(&(cb[1+nonceSize]), q); + memset(&(cb[1+nonceSize]), 0, q); /* Create the mask for the counter field based on the size of nonce */ q <<= 3; @@ -782,7 +780,7 @@ ccm_init(ccm_ctx_t *ctx, unsigned char *nonce, size_t nonce_len, /* The IV for CBC MAC for AES CCM mode is always zero */ ivp = (uint8_t *)ctx->ccm_tmp; - bzero(ivp, block_size); + memset(ivp, 0, block_size); xor_block(ivp, mac_buf); @@ -800,14 +798,14 @@ ccm_init(ccm_ctx_t *ctx, unsigned char *nonce, size_t nonce_len, /* 1st block: it contains encoded associated data, and some data */ authp = (uint8_t *)ctx->ccm_tmp; - bzero(authp, block_size); - bcopy(encoded_a, authp, encoded_a_len); + memset(authp, 0, block_size); + memcpy(authp, encoded_a, encoded_a_len); processed = block_size - encoded_a_len; if (processed > auth_data_len) { /* in case auth_data is very small */ processed = auth_data_len; } - bcopy(auth_data, authp+encoded_a_len, processed); + memcpy(authp+encoded_a_len, auth_data, processed); /* xor with previous buffer */ xor_block(authp, mac_buf); encrypt_block(ctx->ccm_keysched, mac_buf, mac_buf); @@ -823,8 +821,8 @@ ccm_init(ccm_ctx_t *ctx, unsigned char *nonce, size_t nonce_len, * There's not a block full of data, pad rest of * buffer with zero */ - bzero(authp, block_size); - bcopy(&(auth_data[processed]), authp, remainder); + memset(authp, 0, block_size); + memcpy(authp, &(auth_data[processed]), remainder); datap = (uint8_t *)authp; remainder = 0; } else { diff --git a/module/icp/algs/modes/ctr.c b/module/icp/algs/modes/ctr.c index 82295cda877e..c116ba3662ba 100644 --- a/module/icp/algs/modes/ctr.c +++ b/module/icp/algs/modes/ctr.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -52,8 +52,8 @@ ctr_mode_contiguous_blocks(ctr_ctx_t *ctx, char *data, size_t length, if (length + ctx->ctr_remainder_len < block_size) { /* accumulate bytes here and return */ - bcopy(datap, - (uint8_t *)ctx->ctr_remainder + ctx->ctr_remainder_len, + memcpy((uint8_t *)ctx->ctr_remainder + ctx->ctr_remainder_len, + datap, length); ctx->ctr_remainder_len += length; ctx->ctr_copy_to = datap; @@ -71,8 +71,8 @@ ctr_mode_contiguous_blocks(ctr_ctx_t *ctx, char *data, size_t length, if (need > remainder) return (CRYPTO_DATA_LEN_RANGE); - bcopy(datap, &((uint8_t *)ctx->ctr_remainder) - [ctx->ctr_remainder_len], need); + memcpy(&((uint8_t *)ctx->ctr_remainder) + [ctx->ctr_remainder_len], datap, need); blockp = (uint8_t *)ctx->ctr_remainder; } else { @@ -114,9 +114,9 @@ ctr_mode_contiguous_blocks(ctr_ctx_t *ctx, char *data, size_t length, &out_data_1_len, &out_data_2, block_size); /* copy block to where it belongs */ - bcopy(lastp, out_data_1, out_data_1_len); + memcpy(out_data_1, lastp, out_data_1_len); if (out_data_2 != NULL) { - bcopy(lastp + out_data_1_len, out_data_2, + memcpy(out_data_2, lastp + out_data_1_len, block_size - out_data_1_len); } /* update offset */ @@ -134,7 +134,7 @@ ctr_mode_contiguous_blocks(ctr_ctx_t *ctx, char *data, size_t length, /* Incomplete last block. */ if (remainder > 0 && remainder < block_size) { - bcopy(datap, ctx->ctr_remainder, remainder); + memcpy(ctx->ctr_remainder, datap, remainder); ctx->ctr_remainder_len = remainder; ctx->ctr_copy_to = datap; goto out; @@ -176,10 +176,11 @@ ctr_mode_final(ctr_ctx_t *ctx, crypto_data_t *out, crypto_get_ptrs(out, &iov_or_mp, &offset, &out_data_1, &out_data_1_len, &out_data_2, ctx->ctr_remainder_len); - bcopy(p, out_data_1, out_data_1_len); + memcpy(out_data_1, p, out_data_1_len); if (out_data_2 != NULL) { - bcopy((uint8_t *)p + out_data_1_len, - out_data_2, ctx->ctr_remainder_len - out_data_1_len); + memcpy(out_data_2, + (uint8_t *)p + out_data_1_len, + ctx->ctr_remainder_len - out_data_1_len); } out->cd_offset += ctx->ctr_remainder_len; ctx->ctr_remainder_len = 0; diff --git a/module/icp/algs/modes/ecb.c b/module/icp/algs/modes/ecb.c index ffbdb9d57d0a..e2d8e71c161c 100644 --- a/module/icp/algs/modes/ecb.c +++ b/module/icp/algs/modes/ecb.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -49,8 +49,8 @@ ecb_cipher_contiguous_blocks(ecb_ctx_t *ctx, char *data, size_t length, if (length + ctx->ecb_remainder_len < block_size) { /* accumulate bytes here and return */ - bcopy(datap, - (uint8_t *)ctx->ecb_remainder + ctx->ecb_remainder_len, + memcpy((uint8_t *)ctx->ecb_remainder + ctx->ecb_remainder_len, + datap, length); ctx->ecb_remainder_len += length; ctx->ecb_copy_to = datap; @@ -68,8 +68,8 @@ ecb_cipher_contiguous_blocks(ecb_ctx_t *ctx, char *data, size_t length, if (need > remainder) return (CRYPTO_DATA_LEN_RANGE); - bcopy(datap, &((uint8_t *)ctx->ecb_remainder) - [ctx->ecb_remainder_len], need); + memcpy(&((uint8_t *)ctx->ecb_remainder) + [ctx->ecb_remainder_len], datap, need); blockp = (uint8_t *)ctx->ecb_remainder; } else { @@ -81,9 +81,9 @@ ecb_cipher_contiguous_blocks(ecb_ctx_t *ctx, char *data, size_t length, &out_data_1_len, &out_data_2, block_size); /* copy block to where it belongs */ - bcopy(lastp, out_data_1, out_data_1_len); + memcpy(out_data_1, lastp, out_data_1_len); if (out_data_2 != NULL) { - bcopy(lastp + out_data_1_len, out_data_2, + memcpy(out_data_2, lastp + out_data_1_len, block_size - out_data_1_len); } /* update offset */ @@ -101,7 +101,7 @@ ecb_cipher_contiguous_blocks(ecb_ctx_t *ctx, char *data, size_t length, /* Incomplete last block. */ if (remainder > 0 && remainder < block_size) { - bcopy(datap, ctx->ecb_remainder, remainder); + memcpy(ctx->ecb_remainder, datap, remainder); ctx->ecb_remainder_len = remainder; ctx->ecb_copy_to = datap; goto out; diff --git a/module/icp/algs/modes/gcm.c b/module/icp/algs/modes/gcm.c index 8b3793daa5cf..ca328d54a7e6 100644 --- a/module/icp/algs/modes/gcm.c +++ b/module/icp/algs/modes/gcm.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -108,8 +108,8 @@ gcm_mode_encrypt_contiguous_blocks(gcm_ctx_t *ctx, char *data, size_t length, if (length + ctx->gcm_remainder_len < block_size) { /* accumulate bytes here and return */ - bcopy(datap, - (uint8_t *)ctx->gcm_remainder + ctx->gcm_remainder_len, + memcpy((uint8_t *)ctx->gcm_remainder + ctx->gcm_remainder_len, + datap, length); ctx->gcm_remainder_len += length; if (ctx->gcm_copy_to == NULL) { @@ -130,8 +130,8 @@ gcm_mode_encrypt_contiguous_blocks(gcm_ctx_t *ctx, char *data, size_t length, if (need > remainder) return (CRYPTO_DATA_LEN_RANGE); - bcopy(datap, &((uint8_t *)ctx->gcm_remainder) - [ctx->gcm_remainder_len], need); + memcpy(&((uint8_t *)ctx->gcm_remainder) + [ctx->gcm_remainder_len], datap, need); blockp = (uint8_t *)ctx->gcm_remainder; } else { @@ -162,10 +162,10 @@ gcm_mode_encrypt_contiguous_blocks(gcm_ctx_t *ctx, char *data, size_t length, if (out_data_1_len == block_size) { copy_block(lastp, out_data_1); } else { - bcopy(lastp, out_data_1, out_data_1_len); + memcpy(out_data_1, lastp, out_data_1_len); if (out_data_2 != NULL) { - bcopy(lastp + out_data_1_len, - out_data_2, + memcpy(out_data_2, + lastp + out_data_1_len, block_size - out_data_1_len); } } @@ -187,7 +187,7 @@ gcm_mode_encrypt_contiguous_blocks(gcm_ctx_t *ctx, char *data, size_t length, /* Incomplete last block. */ if (remainder > 0 && remainder < block_size) { - bcopy(datap, ctx->gcm_remainder, remainder); + memcpy(ctx->gcm_remainder, datap, remainder); ctx->gcm_remainder_len = remainder; ctx->gcm_copy_to = datap; goto out; @@ -245,7 +245,7 @@ gcm_encrypt_final(gcm_ctx_t *ctx, crypto_data_t *out, size_t block_size, (uint8_t *)ctx->gcm_tmp); macp = (uint8_t *)ctx->gcm_remainder; - bzero(macp + ctx->gcm_remainder_len, + memset(macp + ctx->gcm_remainder_len, 0, block_size - ctx->gcm_remainder_len); /* XOR with counter block */ @@ -309,8 +309,8 @@ gcm_decrypt_incomplete_block(gcm_ctx_t *ctx, size_t block_size, size_t index, counterp = (uint8_t *)ctx->gcm_tmp; /* authentication tag */ - bzero((uint8_t *)ctx->gcm_tmp, block_size); - bcopy(datap, (uint8_t *)ctx->gcm_tmp, ctx->gcm_remainder_len); + memset((uint8_t *)ctx->gcm_tmp, 0, block_size); + memcpy((uint8_t *)ctx->gcm_tmp, datap, ctx->gcm_remainder_len); /* add ciphertext to the hash */ GHASH(ctx, ctx->gcm_tmp, ctx->gcm_ghash, gcm_impl_get_ops()); @@ -342,7 +342,7 @@ gcm_mode_decrypt_contiguous_blocks(gcm_ctx_t *ctx, char *data, size_t length, */ if (length > 0) { new_len = ctx->gcm_pt_buf_len + length; - new = vmem_alloc(new_len, ctx->gcm_kmflag); + new = vmem_alloc(new_len, KM_SLEEP); if (new == NULL) { vmem_free(ctx->gcm_pt_buf, ctx->gcm_pt_buf_len); ctx->gcm_pt_buf = NULL; @@ -350,7 +350,7 @@ gcm_mode_decrypt_contiguous_blocks(gcm_ctx_t *ctx, char *data, size_t length, } if (ctx->gcm_pt_buf != NULL) { - bcopy(ctx->gcm_pt_buf, new, ctx->gcm_pt_buf_len); + memcpy(new, ctx->gcm_pt_buf, ctx->gcm_pt_buf_len); vmem_free(ctx->gcm_pt_buf, ctx->gcm_pt_buf_len); } else { ASSERT0(ctx->gcm_pt_buf_len); @@ -358,7 +358,7 @@ gcm_mode_decrypt_contiguous_blocks(gcm_ctx_t *ctx, char *data, size_t length, ctx->gcm_pt_buf = new; ctx->gcm_pt_buf_len = new_len; - bcopy(data, &ctx->gcm_pt_buf[ctx->gcm_processed_data_len], + memcpy(&ctx->gcm_pt_buf[ctx->gcm_processed_data_len], data, length); ctx->gcm_processed_data_len += length; } @@ -397,7 +397,7 @@ gcm_decrypt_final(gcm_ctx_t *ctx, crypto_data_t *out, size_t block_size, while (remainder > 0) { /* Incomplete last block */ if (remainder < block_size) { - bcopy(blockp, ctx->gcm_remainder, remainder); + memcpy(ctx->gcm_remainder, blockp, remainder); ctx->gcm_remainder_len = remainder; /* * not expecting anymore ciphertext, just @@ -438,7 +438,7 @@ gcm_decrypt_final(gcm_ctx_t *ctx, crypto_data_t *out, size_t block_size, xor_block((uint8_t *)ctx->gcm_J0, ghash); /* compare the input authentication tag with what we calculated */ - if (bcmp(&ctx->gcm_pt_buf[pt_len], ghash, ctx->gcm_tag_len)) { + if (memcmp(&ctx->gcm_pt_buf[pt_len], ghash, ctx->gcm_tag_len)) { /* They don't match */ return (CRYPTO_INVALID_MAC); } else { @@ -495,7 +495,7 @@ gcm_format_initial_blocks(uchar_t *iv, ulong_t iv_len, ghash = (uint8_t *)ctx->gcm_ghash; cb = (uint8_t *)ctx->gcm_cb; if (iv_len == 12) { - bcopy(iv, cb, 12); + memcpy(cb, iv, 12); cb[12] = 0; cb[13] = 0; cb[14] = 0; @@ -506,8 +506,8 @@ gcm_format_initial_blocks(uchar_t *iv, ulong_t iv_len, /* GHASH the IV */ do { if (remainder < block_size) { - bzero(cb, block_size); - bcopy(&(iv[processed]), cb, remainder); + memset(cb, 0, block_size); + memcpy(cb, &(iv[processed]), remainder); datap = (uint8_t *)cb; remainder = 0; } else { @@ -539,7 +539,7 @@ gcm_init(gcm_ctx_t *ctx, unsigned char *iv, size_t iv_len, size_t remainder, processed; /* encrypt zero block to get subkey H */ - bzero(ctx->gcm_H, sizeof (ctx->gcm_H)); + memset(ctx->gcm_H, 0, sizeof (ctx->gcm_H)); encrypt_block(ctx->gcm_keysched, (uint8_t *)ctx->gcm_H, (uint8_t *)ctx->gcm_H); @@ -549,8 +549,8 @@ gcm_init(gcm_ctx_t *ctx, unsigned char *iv, size_t iv_len, gops = gcm_impl_get_ops(); authp = (uint8_t *)ctx->gcm_tmp; ghash = (uint8_t *)ctx->gcm_ghash; - bzero(authp, block_size); - bzero(ghash, block_size); + memset(authp, 0, block_size); + memset(ghash, 0, block_size); processed = 0; remainder = auth_data_len; @@ -562,9 +562,9 @@ gcm_init(gcm_ctx_t *ctx, unsigned char *iv, size_t iv_len, */ if (auth_data != NULL) { - bzero(authp, block_size); - bcopy(&(auth_data[processed]), - authp, remainder); + memset(authp, 0, block_size); + memcpy(authp, &(auth_data[processed]), + remainder); } else { ASSERT0(remainder); } @@ -654,7 +654,7 @@ gcm_init_ctx(gcm_ctx_t *gcm_ctx, char *param, size_t block_size, } gcm_ctx->gcm_htab_len = htab_len; gcm_ctx->gcm_Htable = - (uint64_t *)kmem_alloc(htab_len, gcm_ctx->gcm_kmflag); + (uint64_t *)kmem_alloc(htab_len, KM_SLEEP); if (gcm_ctx->gcm_Htable == NULL) { return (CRYPTO_HOST_MEMORY); @@ -729,7 +729,7 @@ gmac_init_ctx(gcm_ctx_t *gcm_ctx, char *param, size_t block_size, } gcm_ctx->gcm_htab_len = htab_len; gcm_ctx->gcm_Htable = - (uint64_t *)kmem_alloc(htab_len, gcm_ctx->gcm_kmflag); + (uint64_t *)kmem_alloc(htab_len, KM_SLEEP); if (gcm_ctx->gcm_Htable == NULL) { return (CRYPTO_HOST_MEMORY); @@ -780,12 +780,6 @@ gmac_alloc_ctx(int kmflag) return (gcm_ctx); } -void -gcm_set_kmflag(gcm_ctx_t *ctx, int kmflag) -{ - ctx->gcm_kmflag = kmflag; -} - /* GCM implementation that contains the fastest methods */ static gcm_impl_ops_t gcm_fastest_impl = { .name = "fastest" @@ -812,7 +806,7 @@ static gcm_impl_ops_t *gcm_supp_impl[ARRAY_SIZE(gcm_all_impl)]; * fallback to the fastest generic implementation. */ const gcm_impl_ops_t * -gcm_impl_get_ops() +gcm_impl_get_ops(void) { if (!kfpu_allowed()) return (&gcm_generic_impl); @@ -913,7 +907,7 @@ gcm_impl_init(void) } static const struct { - char *name; + const char *name; uint32_t sel; } gcm_impl_opts[] = { { "cycle", IMPL_CYCLE }, @@ -1145,10 +1139,10 @@ gcm_simd_get_htab_size(boolean_t simd_mode) static inline void gcm_clear_ctx(gcm_ctx_t *ctx) { - bzero(ctx->gcm_remainder, sizeof (ctx->gcm_remainder)); - bzero(ctx->gcm_H, sizeof (ctx->gcm_H)); - bzero(ctx->gcm_J0, sizeof (ctx->gcm_J0)); - bzero(ctx->gcm_tmp, sizeof (ctx->gcm_tmp)); + memset(ctx->gcm_remainder, 0, sizeof (ctx->gcm_remainder)); + memset(ctx->gcm_H, 0, sizeof (ctx->gcm_H)); + memset(ctx->gcm_J0, 0, sizeof (ctx->gcm_J0)); + memset(ctx->gcm_tmp, 0, sizeof (ctx->gcm_tmp)); } /* Increment the GCM counter block by n. */ @@ -1193,8 +1187,8 @@ gcm_mode_encrypt_contiguous_blocks_avx(gcm_ctx_t *ctx, char *data, need = block_size - ctx->gcm_remainder_len; if (length < need) { /* Accumulate bytes here and return. */ - bcopy(datap, (uint8_t *)ctx->gcm_remainder + - ctx->gcm_remainder_len, length); + memcpy((uint8_t *)ctx->gcm_remainder + + ctx->gcm_remainder_len, datap, length); ctx->gcm_remainder_len += length; if (ctx->gcm_copy_to == NULL) { @@ -1203,8 +1197,8 @@ gcm_mode_encrypt_contiguous_blocks_avx(gcm_ctx_t *ctx, char *data, return (CRYPTO_SUCCESS); } else { /* Complete incomplete block. */ - bcopy(datap, (uint8_t *)ctx->gcm_remainder + - ctx->gcm_remainder_len, need); + memcpy((uint8_t *)ctx->gcm_remainder + + ctx->gcm_remainder_len, datap, need); ctx->gcm_copy_to = NULL; } @@ -1212,7 +1206,7 @@ gcm_mode_encrypt_contiguous_blocks_avx(gcm_ctx_t *ctx, char *data, /* Allocate a buffer to encrypt to if there is enough input. */ if (bleft >= GCM_AVX_MIN_ENCRYPT_BYTES) { - ct_buf = vmem_alloc(chunk_size, ctx->gcm_kmflag); + ct_buf = vmem_alloc(chunk_size, KM_SLEEP); if (ct_buf == NULL) { return (CRYPTO_HOST_MEMORY); } @@ -1282,7 +1276,7 @@ gcm_mode_encrypt_contiguous_blocks_avx(gcm_ctx_t *ctx, char *data, /* Less than GCM_AVX_MIN_ENCRYPT_BYTES remain, operate on blocks. */ while (bleft > 0) { if (bleft < block_size) { - bcopy(datap, ctx->gcm_remainder, bleft); + memcpy(ctx->gcm_remainder, datap, bleft); ctx->gcm_remainder_len = bleft; ctx->gcm_copy_to = datap; goto out; @@ -1341,7 +1335,7 @@ gcm_encrypt_final_avx(gcm_ctx_t *ctx, crypto_data_t *out, size_t block_size) const uint32_t *cb = (uint32_t *)ctx->gcm_cb; aes_encrypt_intel(keysched, aes_rounds, cb, (uint32_t *)tmp); - bzero(remainder + rem_len, block_size - rem_len); + memset(remainder + rem_len, 0, block_size - rem_len); for (int i = 0; i < rem_len; i++) { remainder[i] ^= tmp[i]; } @@ -1437,8 +1431,8 @@ gcm_decrypt_final_avx(gcm_ctx_t *ctx, crypto_data_t *out, size_t block_size) if (bleft < block_size) { uint8_t *lastb = (uint8_t *)ctx->gcm_remainder; - bzero(lastb, block_size); - bcopy(datap, lastb, bleft); + memset(lastb, 0, block_size); + memcpy(lastb, datap, bleft); /* The GCM processing. */ GHASH_AVX(ctx, lastb, block_size); aes_encrypt_intel(key->encr_ks.ks32, key->nr, cb, tmp); @@ -1474,7 +1468,7 @@ gcm_decrypt_final_avx(gcm_ctx_t *ctx, crypto_data_t *out, size_t block_size) kfpu_end(); /* Compare the input authentication tag with what we calculated. */ - if (bcmp(&ctx->gcm_pt_buf[pt_len], ghash, ctx->gcm_tag_len)) { + if (memcmp(&ctx->gcm_pt_buf[pt_len], ghash, ctx->gcm_tag_len)) { /* They don't match. */ return (CRYPTO_INVALID_MAC); } @@ -1506,8 +1500,8 @@ gcm_init_avx(gcm_ctx_t *ctx, unsigned char *iv, size_t iv_len, ASSERT(block_size == GCM_BLOCK_LEN); /* Init H (encrypt zero block) and create the initial counter block. */ - bzero(ctx->gcm_ghash, sizeof (ctx->gcm_ghash)); - bzero(H, sizeof (ctx->gcm_H)); + memset(ctx->gcm_ghash, 0, sizeof (ctx->gcm_ghash)); + memset(H, 0, sizeof (ctx->gcm_H)); kfpu_begin(); aes_encrypt_intel(keysched, aes_rounds, (const uint32_t *)H, (uint32_t *)H); @@ -1515,13 +1509,13 @@ gcm_init_avx(gcm_ctx_t *ctx, unsigned char *iv, size_t iv_len, gcm_init_htab_avx(ctx->gcm_Htable, H); if (iv_len == 12) { - bcopy(iv, cb, 12); + memcpy(cb, iv, 12); cb[12] = 0; cb[13] = 0; cb[14] = 0; cb[15] = 1; /* We need the ICB later. */ - bcopy(cb, ctx->gcm_J0, sizeof (ctx->gcm_J0)); + memcpy(ctx->gcm_J0, cb, sizeof (ctx->gcm_J0)); } else { /* * Most consumers use 12 byte IVs, so it's OK to use the @@ -1559,8 +1553,8 @@ gcm_init_avx(gcm_ctx_t *ctx, unsigned char *iv, size_t iv_len, /* Zero pad and hash incomplete last block. */ uint8_t *authp = (uint8_t *)ctx->gcm_tmp; - bzero(authp, block_size); - bcopy(datap, authp, incomp); + memset(authp, 0, block_size); + memcpy(authp, datap, incomp); GHASH_AVX(ctx, authp, block_size); } } diff --git a/module/icp/algs/modes/gcm_generic.c b/module/icp/algs/modes/gcm_generic.c index 16b57998a92f..84e26d09cdcf 100644 --- a/module/icp/algs/modes/gcm_generic.c +++ b/module/icp/algs/modes/gcm_generic.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/icp/algs/modes/gcm_pclmulqdq.c b/module/icp/algs/modes/gcm_pclmulqdq.c index 05920115ce86..c2c8bc221203 100644 --- a/module/icp/algs/modes/gcm_pclmulqdq.c +++ b/module/icp/algs/modes/gcm_pclmulqdq.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/icp/algs/modes/modes.c b/module/icp/algs/modes/modes.c index 59743c7d6829..b98db0ac14ec 100644 --- a/module/icp/algs/modes/modes.c +++ b/module/icp/algs/modes/modes.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -155,7 +155,7 @@ crypto_free_mode_ctx(void *ctx) #ifdef CAN_USE_GCM_ASM if (((gcm_ctx_t *)ctx)->gcm_Htable != NULL) { gcm_ctx_t *gcm_ctx = (gcm_ctx_t *)ctx; - bzero(gcm_ctx->gcm_Htable, gcm_ctx->gcm_htab_len); + memset(gcm_ctx->gcm_Htable, 0, gcm_ctx->gcm_htab_len); kmem_free(gcm_ctx->gcm_Htable, gcm_ctx->gcm_htab_len); } #endif diff --git a/module/icp/algs/sha2/sha2.c b/module/icp/algs/sha2/sha2.c index 6f1e9b7193d4..151432f1a5df 100644 --- a/module/icp/algs/sha2/sha2.c +++ b/module/icp/algs/sha2/sha2.c @@ -190,7 +190,7 @@ SHA256Transform(SHA2_CTX *ctx, const uint8_t *blk) #endif /* __sparc */ if ((uintptr_t)blk & 0x3) { /* not 4-byte aligned? */ - bcopy(blk, ctx->buf_un.buf32, sizeof (ctx->buf_un.buf32)); + memcpy(ctx->buf_un.buf32, blk, sizeof (ctx->buf_un.buf32)); blk = (uint8_t *)ctx->buf_un.buf32; } @@ -406,7 +406,7 @@ SHA512Transform(SHA2_CTX *ctx, const uint8_t *blk) if ((uintptr_t)blk & 0x7) { /* not 8-byte aligned? */ - bcopy(blk, ctx->buf_un.buf64, sizeof (ctx->buf_un.buf64)); + memcpy(ctx->buf_un.buf64, blk, sizeof (ctx->buf_un.buf64)); blk = (uint8_t *)ctx->buf_un.buf64; } @@ -823,14 +823,14 @@ SHA2Update(SHA2_CTX *ctx, const void *inptr, size_t input_len) /* * general optimization: * - * only do initial bcopy() and SHA2Transform() if + * only do initial memcpy() and SHA2Transform() if * buf_index != 0. if buf_index == 0, we're just - * wasting our time doing the bcopy() since there + * wasting our time doing the memcpy() since there * wasn't any data left over from a previous call to * SHA2Update(). */ if (buf_index) { - bcopy(input, &ctx->buf_un.buf8[buf_index], buf_len); + memcpy(&ctx->buf_un.buf8[buf_index], input, buf_len); if (algotype <= SHA256_HMAC_GEN_MECH_INFO_TYPE) SHA256Transform(ctx, ctx->buf_un.buf8); else @@ -873,7 +873,7 @@ SHA2Update(SHA2_CTX *ctx, const void *inptr, size_t input_len) * general optimization: * * if i and input_len are the same, return now instead - * of calling bcopy(), since the bcopy() in this case + * of calling memcpy(), since the memcpy() in this case * will be an expensive noop. */ @@ -884,7 +884,7 @@ SHA2Update(SHA2_CTX *ctx, const void *inptr, size_t input_len) } /* buffer remaining input */ - bcopy(&input[i], &ctx->buf_un.buf8[buf_index], input_len - i); + memcpy(&ctx->buf_un.buf8[buf_index], &input[i], input_len - i); } @@ -936,7 +936,7 @@ SHA2Final(void *digest, SHA2_CTX *ctx) */ Encode64(digest, ctx->state.s64, sizeof (uint64_t) * 3); Encode64(last, &ctx->state.s64[3], sizeof (uint64_t)); - bcopy(last, (uint8_t *)digest + 24, 4); + memcpy((uint8_t *)digest + 24, last, 4); } else if (algotype == SHA512_256_MECH_INFO_TYPE) { Encode64(digest, ctx->state.s64, sizeof (uint64_t) * 4); } else { @@ -946,7 +946,7 @@ SHA2Final(void *digest, SHA2_CTX *ctx) } /* zeroize sensitive information */ - bzero(ctx, sizeof (*ctx)); + memset(ctx, 0, sizeof (*ctx)); } #ifdef _KERNEL diff --git a/module/icp/algs/skein/skein.c b/module/icp/algs/skein/skein.c index 83fe84260307..41ed2dd44e9e 100644 --- a/module/icp/algs/skein/skein.c +++ b/module/icp/algs/skein/skein.c @@ -26,16 +26,16 @@ Skein_256_Init(Skein_256_Ctxt_t *ctx, size_t hashBitLen) switch (hashBitLen) { /* use pre-computed values, where available */ #ifndef SKEIN_NO_PRECOMP case 256: - bcopy(SKEIN_256_IV_256, ctx->X, sizeof (ctx->X)); + memcpy(ctx->X, SKEIN_256_IV_256, sizeof (ctx->X)); break; case 224: - bcopy(SKEIN_256_IV_224, ctx->X, sizeof (ctx->X)); + memcpy(ctx->X, SKEIN_256_IV_224, sizeof (ctx->X)); break; case 160: - bcopy(SKEIN_256_IV_160, ctx->X, sizeof (ctx->X)); + memcpy(ctx->X, SKEIN_256_IV_160, sizeof (ctx->X)); break; case 128: - bcopy(SKEIN_256_IV_128, ctx->X, sizeof (ctx->X)); + memcpy(ctx->X, SKEIN_256_IV_128, sizeof (ctx->X)); break; #endif default: @@ -53,11 +53,11 @@ Skein_256_Init(Skein_256_Ctxt_t *ctx, size_t hashBitLen) cfg.w[1] = Skein_Swap64(hashBitLen); cfg.w[2] = Skein_Swap64(SKEIN_CFG_TREE_INFO_SEQUENTIAL); /* zero pad config block */ - bzero(&cfg.w[3], sizeof (cfg) - 3 * sizeof (cfg.w[0])); + memset(&cfg.w[3], 0, sizeof (cfg) - 3 * sizeof (cfg.w[0])); /* compute the initial chaining values from config block */ /* zero the chaining variables */ - bzero(ctx->X, sizeof (ctx->X)); + memset(ctx->X, 0, sizeof (ctx->X)); Skein_256_Process_Block(ctx, cfg.b, 1, SKEIN_CFG_STR_LEN); break; } @@ -91,7 +91,7 @@ Skein_256_InitExt(Skein_256_Ctxt_t *ctx, size_t hashBitLen, uint64_t treeInfo, /* compute the initial chaining values ctx->X[], based on key */ if (keyBytes == 0) { /* is there a key? */ /* no key: use all zeroes as key for config block */ - bzero(ctx->X, sizeof (ctx->X)); + memset(ctx->X, 0, sizeof (ctx->X)); } else { /* here to pre-process a key */ Skein_assert(sizeof (cfg.b) >= sizeof (ctx->X)); @@ -101,13 +101,13 @@ Skein_256_InitExt(Skein_256_Ctxt_t *ctx, size_t hashBitLen, uint64_t treeInfo, /* set tweaks: T0 = 0; T1 = KEY type */ Skein_Start_New_Type(ctx, KEY); /* zero the initial chaining variables */ - bzero(ctx->X, sizeof (ctx->X)); + memset(ctx->X, 0, sizeof (ctx->X)); /* hash the key */ (void) Skein_256_Update(ctx, key, keyBytes); /* put result into cfg.b[] */ (void) Skein_256_Final_Pad(ctx, cfg.b); /* copy over into ctx->X[] */ - bcopy(cfg.b, ctx->X, sizeof (cfg.b)); + memcpy(ctx->X, cfg.b, sizeof (cfg.b)); #if SKEIN_NEED_SWAP { uint_t i; @@ -124,7 +124,7 @@ Skein_256_InitExt(Skein_256_Ctxt_t *ctx, size_t hashBitLen, uint64_t treeInfo, ctx->h.hashBitLen = hashBitLen; /* output hash bit count */ Skein_Start_New_Type(ctx, CFG_FINAL); - bzero(&cfg.w, sizeof (cfg.w)); /* pre-pad cfg.w[] with zeroes */ + memset(&cfg.w, 0, sizeof (cfg.w)); /* pre-pad cfg.w[] with zeroes */ cfg.w[0] = Skein_Swap64(SKEIN_SCHEMA_VER); cfg.w[1] = Skein_Swap64(hashBitLen); /* hash result length in bits */ /* tree hash config info (or SKEIN_CFG_TREE_INFO_SEQUENTIAL) */ @@ -161,7 +161,7 @@ Skein_256_Update(Skein_256_Ctxt_t *ctx, const uint8_t *msg, size_t msgByteCnt) if (n) { /* check on our logic here */ Skein_assert(n < msgByteCnt); - bcopy(msg, &ctx->b[ctx->h.bCnt], n); + memcpy(&ctx->b[ctx->h.bCnt], msg, n); msgByteCnt -= n; msg += n; ctx->h.bCnt += n; @@ -189,7 +189,7 @@ Skein_256_Update(Skein_256_Ctxt_t *ctx, const uint8_t *msg, size_t msgByteCnt) /* copy any remaining source message data bytes into b[] */ if (msgByteCnt) { Skein_assert(msgByteCnt + ctx->h.bCnt <= SKEIN_256_BLOCK_BYTES); - bcopy(msg, &ctx->b[ctx->h.bCnt], msgByteCnt); + memcpy(&ctx->b[ctx->h.bCnt], msg, msgByteCnt); ctx->h.bCnt += msgByteCnt; } @@ -209,7 +209,7 @@ Skein_256_Final(Skein_256_Ctxt_t *ctx, uint8_t *hashVal) ctx->h.T[1] |= SKEIN_T1_FLAG_FINAL; /* tag as the final block */ /* zero pad b[] if necessary */ if (ctx->h.bCnt < SKEIN_256_BLOCK_BYTES) - bzero(&ctx->b[ctx->h.bCnt], + memset(&ctx->b[ctx->h.bCnt], 0, SKEIN_256_BLOCK_BYTES - ctx->h.bCnt); /* process the final block */ @@ -221,13 +221,12 @@ Skein_256_Final(Skein_256_Ctxt_t *ctx, uint8_t *hashVal) /* run Threefish in "counter mode" to generate output */ /* zero out b[], so it can hold the counter */ - bzero(ctx->b, sizeof (ctx->b)); + memset(ctx->b, 0, sizeof (ctx->b)); /* keep a local copy of counter mode "key" */ - bcopy(ctx->X, X, sizeof (X)); + memcpy(X, ctx->X, sizeof (X)); for (i = 0; i * SKEIN_256_BLOCK_BYTES < byteCnt; i++) { /* build the counter block */ - uint64_t tmp = Skein_Swap64((uint64_t)i); - bcopy(&tmp, ctx->b, sizeof (tmp)); + *(uint64_t *)ctx->b = Skein_Swap64((uint64_t)i); Skein_Start_New_Type(ctx, OUT_FINAL); /* run "counter mode" */ Skein_256_Process_Block(ctx, ctx->b, 1, sizeof (uint64_t)); @@ -240,7 +239,7 @@ Skein_256_Final(Skein_256_Ctxt_t *ctx, uint8_t *hashVal) Skein_Show_Final(256, &ctx->h, n, hashVal + i * SKEIN_256_BLOCK_BYTES); /* restore the counter mode key for next time */ - bcopy(X, ctx->X, sizeof (X)); + memcpy(ctx->X, X, sizeof (X)); } return (SKEIN_SUCCESS); } @@ -262,16 +261,16 @@ Skein_512_Init(Skein_512_Ctxt_t *ctx, size_t hashBitLen) switch (hashBitLen) { /* use pre-computed values, where available */ #ifndef SKEIN_NO_PRECOMP case 512: - bcopy(SKEIN_512_IV_512, ctx->X, sizeof (ctx->X)); + memcpy(ctx->X, SKEIN_512_IV_512, sizeof (ctx->X)); break; case 384: - bcopy(SKEIN_512_IV_384, ctx->X, sizeof (ctx->X)); + memcpy(ctx->X, SKEIN_512_IV_384, sizeof (ctx->X)); break; case 256: - bcopy(SKEIN_512_IV_256, ctx->X, sizeof (ctx->X)); + memcpy(ctx->X, SKEIN_512_IV_256, sizeof (ctx->X)); break; case 224: - bcopy(SKEIN_512_IV_224, ctx->X, sizeof (ctx->X)); + memcpy(ctx->X, SKEIN_512_IV_224, sizeof (ctx->X)); break; #endif default: @@ -289,11 +288,11 @@ Skein_512_Init(Skein_512_Ctxt_t *ctx, size_t hashBitLen) cfg.w[1] = Skein_Swap64(hashBitLen); cfg.w[2] = Skein_Swap64(SKEIN_CFG_TREE_INFO_SEQUENTIAL); /* zero pad config block */ - bzero(&cfg.w[3], sizeof (cfg) - 3 * sizeof (cfg.w[0])); + memset(&cfg.w[3], 0, sizeof (cfg) - 3 * sizeof (cfg.w[0])); /* compute the initial chaining values from config block */ /* zero the chaining variables */ - bzero(ctx->X, sizeof (ctx->X)); + memset(ctx->X, 0, sizeof (ctx->X)); Skein_512_Process_Block(ctx, cfg.b, 1, SKEIN_CFG_STR_LEN); break; } @@ -328,7 +327,7 @@ Skein_512_InitExt(Skein_512_Ctxt_t *ctx, size_t hashBitLen, uint64_t treeInfo, /* compute the initial chaining values ctx->X[], based on key */ if (keyBytes == 0) { /* is there a key? */ /* no key: use all zeroes as key for config block */ - bzero(ctx->X, sizeof (ctx->X)); + memset(ctx->X, 0, sizeof (ctx->X)); } else { /* here to pre-process a key */ Skein_assert(sizeof (cfg.b) >= sizeof (ctx->X)); @@ -338,12 +337,12 @@ Skein_512_InitExt(Skein_512_Ctxt_t *ctx, size_t hashBitLen, uint64_t treeInfo, /* set tweaks: T0 = 0; T1 = KEY type */ Skein_Start_New_Type(ctx, KEY); /* zero the initial chaining variables */ - bzero(ctx->X, sizeof (ctx->X)); + memset(ctx->X, 0, sizeof (ctx->X)); (void) Skein_512_Update(ctx, key, keyBytes); /* hash the key */ /* put result into cfg.b[] */ (void) Skein_512_Final_Pad(ctx, cfg.b); /* copy over into ctx->X[] */ - bcopy(cfg.b, ctx->X, sizeof (cfg.b)); + memcpy(ctx->X, cfg.b, sizeof (cfg.b)); #if SKEIN_NEED_SWAP { uint_t i; @@ -360,7 +359,7 @@ Skein_512_InitExt(Skein_512_Ctxt_t *ctx, size_t hashBitLen, uint64_t treeInfo, ctx->h.hashBitLen = hashBitLen; /* output hash bit count */ Skein_Start_New_Type(ctx, CFG_FINAL); - bzero(&cfg.w, sizeof (cfg.w)); /* pre-pad cfg.w[] with zeroes */ + memset(&cfg.w, 0, sizeof (cfg.w)); /* pre-pad cfg.w[] with zeroes */ cfg.w[0] = Skein_Swap64(SKEIN_SCHEMA_VER); cfg.w[1] = Skein_Swap64(hashBitLen); /* hash result length in bits */ /* tree hash config info (or SKEIN_CFG_TREE_INFO_SEQUENTIAL) */ @@ -397,7 +396,7 @@ Skein_512_Update(Skein_512_Ctxt_t *ctx, const uint8_t *msg, size_t msgByteCnt) if (n) { /* check on our logic here */ Skein_assert(n < msgByteCnt); - bcopy(msg, &ctx->b[ctx->h.bCnt], n); + memcpy(&ctx->b[ctx->h.bCnt], msg, n); msgByteCnt -= n; msg += n; ctx->h.bCnt += n; @@ -425,7 +424,7 @@ Skein_512_Update(Skein_512_Ctxt_t *ctx, const uint8_t *msg, size_t msgByteCnt) /* copy any remaining source message data bytes into b[] */ if (msgByteCnt) { Skein_assert(msgByteCnt + ctx->h.bCnt <= SKEIN_512_BLOCK_BYTES); - bcopy(msg, &ctx->b[ctx->h.bCnt], msgByteCnt); + memcpy(&ctx->b[ctx->h.bCnt], msg, msgByteCnt); ctx->h.bCnt += msgByteCnt; } @@ -445,7 +444,7 @@ Skein_512_Final(Skein_512_Ctxt_t *ctx, uint8_t *hashVal) ctx->h.T[1] |= SKEIN_T1_FLAG_FINAL; /* tag as the final block */ /* zero pad b[] if necessary */ if (ctx->h.bCnt < SKEIN_512_BLOCK_BYTES) - bzero(&ctx->b[ctx->h.bCnt], + memset(&ctx->b[ctx->h.bCnt], 0, SKEIN_512_BLOCK_BYTES - ctx->h.bCnt); /* process the final block */ @@ -457,13 +456,12 @@ Skein_512_Final(Skein_512_Ctxt_t *ctx, uint8_t *hashVal) /* run Threefish in "counter mode" to generate output */ /* zero out b[], so it can hold the counter */ - bzero(ctx->b, sizeof (ctx->b)); + memset(ctx->b, 0, sizeof (ctx->b)); /* keep a local copy of counter mode "key" */ - bcopy(ctx->X, X, sizeof (X)); + memcpy(X, ctx->X, sizeof (X)); for (i = 0; i * SKEIN_512_BLOCK_BYTES < byteCnt; i++) { /* build the counter block */ - uint64_t tmp = Skein_Swap64((uint64_t)i); - bcopy(&tmp, ctx->b, sizeof (tmp)); + *(uint64_t *)ctx->b = Skein_Swap64((uint64_t)i); Skein_Start_New_Type(ctx, OUT_FINAL); /* run "counter mode" */ Skein_512_Process_Block(ctx, ctx->b, 1, sizeof (uint64_t)); @@ -476,7 +474,7 @@ Skein_512_Final(Skein_512_Ctxt_t *ctx, uint8_t *hashVal) Skein_Show_Final(512, &ctx->h, n, hashVal + i * SKEIN_512_BLOCK_BYTES); /* restore the counter mode key for next time */ - bcopy(X, ctx->X, sizeof (X)); + memcpy(ctx->X, X, sizeof (X)); } return (SKEIN_SUCCESS); } @@ -498,13 +496,13 @@ Skein1024_Init(Skein1024_Ctxt_t *ctx, size_t hashBitLen) switch (hashBitLen) { /* use pre-computed values, where available */ #ifndef SKEIN_NO_PRECOMP case 512: - bcopy(SKEIN1024_IV_512, ctx->X, sizeof (ctx->X)); + memcpy(ctx->X, SKEIN1024_IV_512, sizeof (ctx->X)); break; case 384: - bcopy(SKEIN1024_IV_384, ctx->X, sizeof (ctx->X)); + memcpy(ctx->X, SKEIN1024_IV_384, sizeof (ctx->X)); break; case 1024: - bcopy(SKEIN1024_IV_1024, ctx->X, sizeof (ctx->X)); + memcpy(ctx->X, SKEIN1024_IV_1024, sizeof (ctx->X)); break; #endif default: @@ -522,11 +520,11 @@ Skein1024_Init(Skein1024_Ctxt_t *ctx, size_t hashBitLen) cfg.w[1] = Skein_Swap64(hashBitLen); cfg.w[2] = Skein_Swap64(SKEIN_CFG_TREE_INFO_SEQUENTIAL); /* zero pad config block */ - bzero(&cfg.w[3], sizeof (cfg) - 3 * sizeof (cfg.w[0])); + memset(&cfg.w[3], 0, sizeof (cfg) - 3 * sizeof (cfg.w[0])); /* compute the initial chaining values from config block */ /* zero the chaining variables */ - bzero(ctx->X, sizeof (ctx->X)); + memset(ctx->X, 0, sizeof (ctx->X)); Skein1024_Process_Block(ctx, cfg.b, 1, SKEIN_CFG_STR_LEN); break; } @@ -561,7 +559,7 @@ Skein1024_InitExt(Skein1024_Ctxt_t *ctx, size_t hashBitLen, uint64_t treeInfo, /* compute the initial chaining values ctx->X[], based on key */ if (keyBytes == 0) { /* is there a key? */ /* no key: use all zeroes as key for config block */ - bzero(ctx->X, sizeof (ctx->X)); + memset(ctx->X, 0, sizeof (ctx->X)); } else { /* here to pre-process a key */ Skein_assert(sizeof (cfg.b) >= sizeof (ctx->X)); /* do a mini-Init right here */ @@ -570,12 +568,12 @@ Skein1024_InitExt(Skein1024_Ctxt_t *ctx, size_t hashBitLen, uint64_t treeInfo, /* set tweaks: T0 = 0; T1 = KEY type */ Skein_Start_New_Type(ctx, KEY); /* zero the initial chaining variables */ - bzero(ctx->X, sizeof (ctx->X)); + memset(ctx->X, 0, sizeof (ctx->X)); (void) Skein1024_Update(ctx, key, keyBytes); /* hash the key */ /* put result into cfg.b[] */ (void) Skein1024_Final_Pad(ctx, cfg.b); /* copy over into ctx->X[] */ - bcopy(cfg.b, ctx->X, sizeof (cfg.b)); + memcpy(ctx->X, cfg.b, sizeof (cfg.b)); #if SKEIN_NEED_SWAP { uint_t i; @@ -592,7 +590,7 @@ Skein1024_InitExt(Skein1024_Ctxt_t *ctx, size_t hashBitLen, uint64_t treeInfo, ctx->h.hashBitLen = hashBitLen; /* output hash bit count */ Skein_Start_New_Type(ctx, CFG_FINAL); - bzero(&cfg.w, sizeof (cfg.w)); /* pre-pad cfg.w[] with zeroes */ + memset(&cfg.w, 0, sizeof (cfg.w)); /* pre-pad cfg.w[] with zeroes */ cfg.w[0] = Skein_Swap64(SKEIN_SCHEMA_VER); /* hash result length in bits */ cfg.w[1] = Skein_Swap64(hashBitLen); @@ -630,7 +628,7 @@ Skein1024_Update(Skein1024_Ctxt_t *ctx, const uint8_t *msg, size_t msgByteCnt) if (n) { /* check on our logic here */ Skein_assert(n < msgByteCnt); - bcopy(msg, &ctx->b[ctx->h.bCnt], n); + memcpy(&ctx->b[ctx->h.bCnt], msg, n); msgByteCnt -= n; msg += n; ctx->h.bCnt += n; @@ -658,7 +656,7 @@ Skein1024_Update(Skein1024_Ctxt_t *ctx, const uint8_t *msg, size_t msgByteCnt) /* copy any remaining source message data bytes into b[] */ if (msgByteCnt) { Skein_assert(msgByteCnt + ctx->h.bCnt <= SKEIN1024_BLOCK_BYTES); - bcopy(msg, &ctx->b[ctx->h.bCnt], msgByteCnt); + memcpy(&ctx->b[ctx->h.bCnt], msg, msgByteCnt); ctx->h.bCnt += msgByteCnt; } @@ -678,7 +676,7 @@ Skein1024_Final(Skein1024_Ctxt_t *ctx, uint8_t *hashVal) ctx->h.T[1] |= SKEIN_T1_FLAG_FINAL; /* tag as the final block */ /* zero pad b[] if necessary */ if (ctx->h.bCnt < SKEIN1024_BLOCK_BYTES) - bzero(&ctx->b[ctx->h.bCnt], + memset(&ctx->b[ctx->h.bCnt], 0, SKEIN1024_BLOCK_BYTES - ctx->h.bCnt); /* process the final block */ @@ -690,13 +688,12 @@ Skein1024_Final(Skein1024_Ctxt_t *ctx, uint8_t *hashVal) /* run Threefish in "counter mode" to generate output */ /* zero out b[], so it can hold the counter */ - bzero(ctx->b, sizeof (ctx->b)); + memset(ctx->b, 0, sizeof (ctx->b)); /* keep a local copy of counter mode "key" */ - bcopy(ctx->X, X, sizeof (X)); + memcpy(X, ctx->X, sizeof (X)); for (i = 0; i * SKEIN1024_BLOCK_BYTES < byteCnt; i++) { /* build the counter block */ - uint64_t tmp = Skein_Swap64((uint64_t)i); - bcopy(&tmp, ctx->b, sizeof (tmp)); + *(uint64_t *)ctx->b = Skein_Swap64((uint64_t)i); Skein_Start_New_Type(ctx, OUT_FINAL); /* run "counter mode" */ Skein1024_Process_Block(ctx, ctx->b, 1, sizeof (uint64_t)); @@ -709,7 +706,7 @@ Skein1024_Final(Skein1024_Ctxt_t *ctx, uint8_t *hashVal) Skein_Show_Final(1024, &ctx->h, n, hashVal + i * SKEIN1024_BLOCK_BYTES); /* restore the counter mode key for next time */ - bcopy(X, ctx->X, sizeof (X)); + memcpy(ctx->X, X, sizeof (X)); } return (SKEIN_SUCCESS); } @@ -727,7 +724,7 @@ Skein_256_Final_Pad(Skein_256_Ctxt_t *ctx, uint8_t *hashVal) ctx->h.T[1] |= SKEIN_T1_FLAG_FINAL; /* tag as the final block */ /* zero pad b[] if necessary */ if (ctx->h.bCnt < SKEIN_256_BLOCK_BYTES) - bzero(&ctx->b[ctx->h.bCnt], + memset(&ctx->b[ctx->h.bCnt], 0, SKEIN_256_BLOCK_BYTES - ctx->h.bCnt); /* process the final block */ Skein_256_Process_Block(ctx, ctx->b, 1, ctx->h.bCnt); @@ -748,7 +745,7 @@ Skein_512_Final_Pad(Skein_512_Ctxt_t *ctx, uint8_t *hashVal) ctx->h.T[1] |= SKEIN_T1_FLAG_FINAL; /* tag as the final block */ /* zero pad b[] if necessary */ if (ctx->h.bCnt < SKEIN_512_BLOCK_BYTES) - bzero(&ctx->b[ctx->h.bCnt], + memset(&ctx->b[ctx->h.bCnt], 0, SKEIN_512_BLOCK_BYTES - ctx->h.bCnt); /* process the final block */ Skein_512_Process_Block(ctx, ctx->b, 1, ctx->h.bCnt); @@ -770,7 +767,7 @@ Skein1024_Final_Pad(Skein1024_Ctxt_t *ctx, uint8_t *hashVal) ctx->h.T[1] |= SKEIN_T1_FLAG_FINAL; /* zero pad b[] if necessary */ if (ctx->h.bCnt < SKEIN1024_BLOCK_BYTES) - bzero(&ctx->b[ctx->h.bCnt], + memset(&ctx->b[ctx->h.bCnt], 0, SKEIN1024_BLOCK_BYTES - ctx->h.bCnt); /* process the final block */ Skein1024_Process_Block(ctx, ctx->b, 1, ctx->h.bCnt); @@ -798,13 +795,12 @@ Skein_256_Output(Skein_256_Ctxt_t *ctx, uint8_t *hashVal) /* run Threefish in "counter mode" to generate output */ /* zero out b[], so it can hold the counter */ - bzero(ctx->b, sizeof (ctx->b)); + memset(ctx->b, 0, sizeof (ctx->b)); /* keep a local copy of counter mode "key" */ - bcopy(ctx->X, X, sizeof (X)); + memcpy(X, ctx->X, sizeof (X)); for (i = 0; i * SKEIN_256_BLOCK_BYTES < byteCnt; i++) { /* build the counter block */ - uint64_t tmp = Skein_Swap64((uint64_t)i); - bcopy(&tmp, ctx->b, sizeof (tmp)); + *(uint64_t *)ctx->b = Skein_Swap64((uint64_t)i); Skein_Start_New_Type(ctx, OUT_FINAL); /* run "counter mode" */ Skein_256_Process_Block(ctx, ctx->b, 1, sizeof (uint64_t)); @@ -817,7 +813,7 @@ Skein_256_Output(Skein_256_Ctxt_t *ctx, uint8_t *hashVal) Skein_Show_Final(256, &ctx->h, n, hashVal + i * SKEIN_256_BLOCK_BYTES); /* restore the counter mode key for next time */ - bcopy(X, ctx->X, sizeof (X)); + memcpy(ctx->X, X, sizeof (X)); } return (SKEIN_SUCCESS); } @@ -838,13 +834,12 @@ Skein_512_Output(Skein_512_Ctxt_t *ctx, uint8_t *hashVal) /* run Threefish in "counter mode" to generate output */ /* zero out b[], so it can hold the counter */ - bzero(ctx->b, sizeof (ctx->b)); + memset(ctx->b, 0, sizeof (ctx->b)); /* keep a local copy of counter mode "key" */ - bcopy(ctx->X, X, sizeof (X)); + memcpy(X, ctx->X, sizeof (X)); for (i = 0; i * SKEIN_512_BLOCK_BYTES < byteCnt; i++) { /* build the counter block */ - uint64_t tmp = Skein_Swap64((uint64_t)i); - bcopy(&tmp, ctx->b, sizeof (tmp)); + *(uint64_t *)ctx->b = Skein_Swap64((uint64_t)i); Skein_Start_New_Type(ctx, OUT_FINAL); /* run "counter mode" */ Skein_512_Process_Block(ctx, ctx->b, 1, sizeof (uint64_t)); @@ -857,7 +852,7 @@ Skein_512_Output(Skein_512_Ctxt_t *ctx, uint8_t *hashVal) Skein_Show_Final(256, &ctx->h, n, hashVal + i * SKEIN_512_BLOCK_BYTES); /* restore the counter mode key for next time */ - bcopy(X, ctx->X, sizeof (X)); + memcpy(ctx->X, X, sizeof (X)); } return (SKEIN_SUCCESS); } @@ -878,13 +873,12 @@ Skein1024_Output(Skein1024_Ctxt_t *ctx, uint8_t *hashVal) /* run Threefish in "counter mode" to generate output */ /* zero out b[], so it can hold the counter */ - bzero(ctx->b, sizeof (ctx->b)); + memset(ctx->b, 0, sizeof (ctx->b)); /* keep a local copy of counter mode "key" */ - bcopy(ctx->X, X, sizeof (X)); + memcpy(X, ctx->X, sizeof (X)); for (i = 0; i * SKEIN1024_BLOCK_BYTES < byteCnt; i++) { /* build the counter block */ - uint64_t tmp = Skein_Swap64((uint64_t)i); - bcopy(&tmp, ctx->b, sizeof (tmp)); + *(uint64_t *)ctx->b = Skein_Swap64((uint64_t)i); Skein_Start_New_Type(ctx, OUT_FINAL); /* run "counter mode" */ Skein1024_Process_Block(ctx, ctx->b, 1, sizeof (uint64_t)); @@ -897,7 +891,7 @@ Skein1024_Output(Skein1024_Ctxt_t *ctx, uint8_t *hashVal) Skein_Show_Final(256, &ctx->h, n, hashVal + i * SKEIN1024_BLOCK_BYTES); /* restore the counter mode key for next time */ - bcopy(X, ctx->X, sizeof (X)); + memcpy(ctx->X, X, sizeof (X)); } return (SKEIN_SUCCESS); } diff --git a/module/icp/algs/skein/skein_impl.h b/module/icp/algs/skein/skein_impl.h index 1fa249e95e4b..eff19ce83f81 100644 --- a/module/icp/algs/skein/skein_impl.h +++ b/module/icp/algs/skein/skein_impl.h @@ -25,7 +25,7 @@ #define _SKEIN_IMPL_H_ #include -#include +#include #include "skein_impl.h" #include "skein_port.h" diff --git a/module/icp/algs/skein/skein_port.h b/module/icp/algs/skein/skein_port.h index ce4353082552..96d1266d019e 100644 --- a/module/icp/algs/skein/skein_port.h +++ b/module/icp/algs/skein/skein_port.h @@ -50,9 +50,9 @@ #else /* here for x86 and x86-64 CPUs (and other detected little-endian CPUs) */ #define SKEIN_NEED_SWAP (0) -#define Skein_Put64_LSB_First(dst08, src64, bCnt) bcopy(src64, dst08, bCnt) +#define Skein_Put64_LSB_First(dst08, src64, bCnt) memcpy(dst08, src64, bCnt) #define Skein_Get64_LSB_First(dst64, src08, wCnt) \ - bcopy(src08, dst64, 8 * (wCnt)) + memcpy(dst64, src08, 8 * (wCnt)) #endif #endif /* ifndef SKEIN_NEED_SWAP */ diff --git a/module/icp/api/kcf_cipher.c b/module/icp/api/kcf_cipher.c index d6aa48147edb..4bea46807197 100644 --- a/module/icp/api/kcf_cipher.c +++ b/module/icp/api/kcf_cipher.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -34,253 +34,11 @@ * Encryption and decryption routines. */ -/* - * The following are the possible returned values common to all the routines - * below. The applicability of some of these return values depends on the - * presence of the arguments. - * - * CRYPTO_SUCCESS: The operation completed successfully. - * CRYPTO_QUEUED: A request was submitted successfully. The callback - * routine will be called when the operation is done. - * CRYPTO_INVALID_MECH_NUMBER, CRYPTO_INVALID_MECH_PARAM, or - * CRYPTO_INVALID_MECH for problems with the 'mech'. - * CRYPTO_INVALID_DATA for bogus 'data' - * CRYPTO_HOST_MEMORY for failure to allocate memory to handle this work. - * CRYPTO_INVALID_CONTEXT: Not a valid context. - * CRYPTO_BUSY: Cannot process the request now. Schedule a - * crypto_bufcall(), or try later. - * CRYPTO_NOT_SUPPORTED and CRYPTO_MECH_NOT_SUPPORTED: No provider is - * capable of a function or a mechanism. - * CRYPTO_INVALID_KEY: bogus 'key' argument. - * CRYPTO_INVALID_PLAINTEXT: bogus 'plaintext' argument. - * CRYPTO_INVALID_CIPHERTEXT: bogus 'ciphertext' argument. - */ /* - * crypto_cipher_init_prov() + * crypto_encrypt() * * Arguments: - * - * pd: provider descriptor - * sid: session id - * mech: crypto_mechanism_t pointer. - * mech_type is a valid value previously returned by - * crypto_mech2id(); - * When the mech's parameter is not NULL, its definition depends - * on the standard definition of the mechanism. - * key: pointer to a crypto_key_t structure. - * tmpl: a crypto_ctx_template_t, opaque template of a context of an - * encryption or decryption with the 'mech' using 'key'. - * 'tmpl' is created by a previous call to - * crypto_create_ctx_template(). - * ctxp: Pointer to a crypto_context_t. - * func: CRYPTO_FG_ENCRYPT or CRYPTO_FG_DECRYPT. - * cr: crypto_call_req_t calling conditions and call back info. - * - * Description: - * This is a common function invoked internally by both - * crypto_encrypt_init() and crypto_decrypt_init(). - * Asynchronously submits a request for, or synchronously performs the - * initialization of an encryption or a decryption operation. - * When possible and applicable, will internally use the pre-expanded key - * schedule from the context template, tmpl. - * When complete and successful, 'ctxp' will contain a crypto_context_t - * valid for later calls to encrypt_update() and encrypt_final(), or - * decrypt_update() and decrypt_final(). - * The caller should hold a reference on the specified provider - * descriptor before calling this function. - * - * Context: - * Process or interrupt, according to the semantics dictated by the 'cr'. - * - * Returns: - * See comment in the beginning of the file. - */ -static int -crypto_cipher_init_prov(crypto_provider_t provider, crypto_session_id_t sid, - crypto_mechanism_t *mech, crypto_key_t *key, - crypto_spi_ctx_template_t tmpl, crypto_context_t *ctxp, - crypto_call_req_t *crq, crypto_func_group_t func) -{ - int error; - crypto_ctx_t *ctx; - kcf_req_params_t params; - kcf_provider_desc_t *pd = provider; - kcf_provider_desc_t *real_provider = pd; - - ASSERT(KCF_PROV_REFHELD(pd)); - - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) { - if (func == CRYPTO_FG_ENCRYPT) { - error = kcf_get_hardware_provider(mech->cm_type, - CRYPTO_MECH_INVALID, CHECK_RESTRICT(crq), pd, - &real_provider, CRYPTO_FG_ENCRYPT); - } else { - error = kcf_get_hardware_provider(mech->cm_type, - CRYPTO_MECH_INVALID, CHECK_RESTRICT(crq), pd, - &real_provider, CRYPTO_FG_DECRYPT); - } - - if (error != CRYPTO_SUCCESS) - return (error); - } - - /* Allocate and initialize the canonical context */ - if ((ctx = kcf_new_ctx(crq, real_provider, sid)) == NULL) { - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) - KCF_PROV_REFRELE(real_provider); - return (CRYPTO_HOST_MEMORY); - } - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(crq, pd)) { - crypto_mechanism_t lmech; - - lmech = *mech; - KCF_SET_PROVIDER_MECHNUM(mech->cm_type, real_provider, &lmech); - - if (func == CRYPTO_FG_ENCRYPT) - error = KCF_PROV_ENCRYPT_INIT(real_provider, ctx, - &lmech, key, tmpl, KCF_SWFP_RHNDL(crq)); - else { - ASSERT(func == CRYPTO_FG_DECRYPT); - - error = KCF_PROV_DECRYPT_INIT(real_provider, ctx, - &lmech, key, tmpl, KCF_SWFP_RHNDL(crq)); - } - KCF_PROV_INCRSTATS(pd, error); - - goto done; - } - - /* Check if context sharing is possible */ - if (pd->pd_prov_type == CRYPTO_HW_PROVIDER && - key->ck_format == CRYPTO_KEY_RAW && - KCF_CAN_SHARE_OPSTATE(pd, mech->cm_type)) { - kcf_context_t *tctxp = (kcf_context_t *)ctx; - kcf_provider_desc_t *tpd = NULL; - crypto_mech_info_t *sinfo; - - if ((kcf_get_sw_prov(mech->cm_type, &tpd, &tctxp->kc_mech, - B_FALSE) == CRYPTO_SUCCESS)) { - int tlen; - - sinfo = &(KCF_TO_PROV_MECHINFO(tpd, mech->cm_type)); - /* - * key->ck_length from the consumer is always in bits. - * We convert it to be in the same unit registered by - * the provider in order to do a comparison. - */ - if (sinfo->cm_mech_flags & CRYPTO_KEYSIZE_UNIT_IN_BYTES) - tlen = key->ck_length >> 3; - else - tlen = key->ck_length; - /* - * Check if the software provider can support context - * sharing and support this key length. - */ - if ((sinfo->cm_mech_flags & CRYPTO_CAN_SHARE_OPSTATE) && - (tlen >= sinfo->cm_min_key_length) && - (tlen <= sinfo->cm_max_key_length)) { - ctx->cc_flags = CRYPTO_INIT_OPSTATE; - tctxp->kc_sw_prov_desc = tpd; - } else - KCF_PROV_REFRELE(tpd); - } - } - - if (func == CRYPTO_FG_ENCRYPT) { - KCF_WRAP_ENCRYPT_OPS_PARAMS(¶ms, KCF_OP_INIT, sid, - mech, key, NULL, NULL, tmpl); - } else { - ASSERT(func == CRYPTO_FG_DECRYPT); - KCF_WRAP_DECRYPT_OPS_PARAMS(¶ms, KCF_OP_INIT, sid, - mech, key, NULL, NULL, tmpl); - } - - error = kcf_submit_request(real_provider, ctx, crq, ¶ms, - B_FALSE); - - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) - KCF_PROV_REFRELE(real_provider); - -done: - if ((error == CRYPTO_SUCCESS) || (error == CRYPTO_QUEUED)) - *ctxp = (crypto_context_t)ctx; - else { - /* Release the hold done in kcf_new_ctx(). */ - KCF_CONTEXT_REFRELE((kcf_context_t *)ctx->cc_framework_private); - } - - return (error); -} - -/* - * Same as crypto_cipher_init_prov(), but relies on the scheduler to pick - * an appropriate provider. See crypto_cipher_init_prov() comments for more - * details. - */ -static int -crypto_cipher_init(crypto_mechanism_t *mech, crypto_key_t *key, - crypto_ctx_template_t tmpl, crypto_context_t *ctxp, - crypto_call_req_t *crq, crypto_func_group_t func) -{ - int error; - kcf_mech_entry_t *me; - kcf_provider_desc_t *pd; - kcf_ctx_template_t *ctx_tmpl; - crypto_spi_ctx_template_t spi_ctx_tmpl = NULL; - kcf_prov_tried_t *list = NULL; - -retry: - /* pd is returned held */ - if ((pd = kcf_get_mech_provider(mech->cm_type, &me, &error, - list, func, CHECK_RESTRICT(crq), 0)) == NULL) { - if (list != NULL) - kcf_free_triedlist(list); - return (error); - } - - /* - * For SW providers, check the validity of the context template - * It is very rare that the generation number mis-matches, so - * is acceptable to fail here, and let the consumer recover by - * freeing this tmpl and create a new one for the key and new SW - * provider - */ - if ((pd->pd_prov_type == CRYPTO_SW_PROVIDER) && - ((ctx_tmpl = (kcf_ctx_template_t *)tmpl) != NULL)) { - if (ctx_tmpl->ct_generation != me->me_gen_swprov) { - if (list != NULL) - kcf_free_triedlist(list); - KCF_PROV_REFRELE(pd); - return (CRYPTO_OLD_CTX_TEMPLATE); - } else { - spi_ctx_tmpl = ctx_tmpl->ct_prov_tmpl; - } - } - - error = crypto_cipher_init_prov(pd, pd->pd_sid, mech, key, - spi_ctx_tmpl, ctxp, crq, func); - if (error != CRYPTO_SUCCESS && error != CRYPTO_QUEUED && - IS_RECOVERABLE(error)) { - /* Add pd to the linked list of providers tried. */ - if (kcf_insert_triedlist(&list, pd, KCF_KMFLAG(crq)) != NULL) - goto retry; - } - - if (list != NULL) - kcf_free_triedlist(list); - - KCF_PROV_REFRELE(pd); - return (error); -} - -/* - * crypto_encrypt_prov() - * - * Arguments: - * pd: provider descriptor * sid: session id * mech: crypto_mechanism_t pointer. * mech_type is a valid value previously returned by @@ -294,7 +52,6 @@ crypto_cipher_init(crypto_mechanism_t *mech, crypto_key_t *key, * tmpl: a crypto_ctx_template_t, opaque template of a context of an * encryption with the 'mech' using 'key'. 'tmpl' is created by * a previous call to crypto_create_ctx_template(). - * cr: crypto_call_req_t calling conditions and call back info. * * Description: * Asynchronously submits a request for, or synchronously performs a @@ -302,57 +59,17 @@ crypto_cipher_init(crypto_mechanism_t *mech, crypto_key_t *key, * the key 'key'. * When complete and successful, 'ciphertext' will contain the encrypted * message. - * - * Context: - * Process or interrupt, according to the semantics dictated by the 'cr'. + * Relies on the KCF scheduler to pick a provider. * * Returns: * See comment in the beginning of the file. */ int -crypto_encrypt_prov(crypto_provider_t provider, crypto_session_id_t sid, - crypto_mechanism_t *mech, crypto_data_t *plaintext, crypto_key_t *key, - crypto_ctx_template_t tmpl, crypto_data_t *ciphertext, - crypto_call_req_t *crq) -{ - kcf_req_params_t params; - kcf_provider_desc_t *pd = provider; - kcf_provider_desc_t *real_provider = pd; - int error; - - ASSERT(KCF_PROV_REFHELD(pd)); - - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) { - error = kcf_get_hardware_provider(mech->cm_type, - CRYPTO_MECH_INVALID, CHECK_RESTRICT(crq), pd, - &real_provider, CRYPTO_FG_ENCRYPT_ATOMIC); - - if (error != CRYPTO_SUCCESS) - return (error); - } - - KCF_WRAP_ENCRYPT_OPS_PARAMS(¶ms, KCF_OP_ATOMIC, sid, mech, key, - plaintext, ciphertext, tmpl); - - error = kcf_submit_request(real_provider, NULL, crq, ¶ms, B_FALSE); - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) - KCF_PROV_REFRELE(real_provider); - - return (error); -} - -/* - * Same as crypto_encrypt_prov(), but relies on the scheduler to pick - * a provider. See crypto_encrypt_prov() for more details. - */ -int crypto_encrypt(crypto_mechanism_t *mech, crypto_data_t *plaintext, - crypto_key_t *key, crypto_ctx_template_t tmpl, crypto_data_t *ciphertext, - crypto_call_req_t *crq) + crypto_key_t *key, crypto_ctx_template_t tmpl, crypto_data_t *ciphertext) { int error; kcf_mech_entry_t *me; - kcf_req_params_t params; kcf_provider_desc_t *pd; kcf_ctx_template_t *ctx_tmpl; crypto_spi_ctx_template_t spi_ctx_tmpl = NULL; @@ -361,52 +78,23 @@ crypto_encrypt(crypto_mechanism_t *mech, crypto_data_t *plaintext, retry: /* pd is returned held */ if ((pd = kcf_get_mech_provider(mech->cm_type, &me, &error, - list, CRYPTO_FG_ENCRYPT_ATOMIC, CHECK_RESTRICT(crq), - plaintext->cd_length)) == NULL) { + list, CRYPTO_FG_ENCRYPT_ATOMIC)) == NULL) { if (list != NULL) kcf_free_triedlist(list); return (error); } - /* - * For SW providers, check the validity of the context template - * It is very rare that the generation number mis-matches, so - * is acceptable to fail here, and let the consumer recover by - * freeing this tmpl and create a new one for the key and new SW - * provider - */ - if ((pd->pd_prov_type == CRYPTO_SW_PROVIDER) && - ((ctx_tmpl = (kcf_ctx_template_t *)tmpl) != NULL)) { - if (ctx_tmpl->ct_generation != me->me_gen_swprov) { - if (list != NULL) - kcf_free_triedlist(list); - KCF_PROV_REFRELE(pd); - return (CRYPTO_OLD_CTX_TEMPLATE); - } else { - spi_ctx_tmpl = ctx_tmpl->ct_prov_tmpl; - } - } - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(crq, pd)) { - crypto_mechanism_t lmech; + if (((ctx_tmpl = (kcf_ctx_template_t *)tmpl) != NULL)) + spi_ctx_tmpl = ctx_tmpl->ct_prov_tmpl; - lmech = *mech; - KCF_SET_PROVIDER_MECHNUM(mech->cm_type, pd, &lmech); + crypto_mechanism_t lmech = *mech; + KCF_SET_PROVIDER_MECHNUM(mech->cm_type, pd, &lmech); + error = KCF_PROV_ENCRYPT_ATOMIC(pd, &lmech, key, + plaintext, ciphertext, spi_ctx_tmpl); - error = KCF_PROV_ENCRYPT_ATOMIC(pd, pd->pd_sid, &lmech, key, - plaintext, ciphertext, spi_ctx_tmpl, KCF_SWFP_RHNDL(crq)); - KCF_PROV_INCRSTATS(pd, error); - } else { - KCF_WRAP_ENCRYPT_OPS_PARAMS(¶ms, KCF_OP_ATOMIC, pd->pd_sid, - mech, key, plaintext, ciphertext, spi_ctx_tmpl); - error = kcf_submit_request(pd, NULL, crq, ¶ms, B_FALSE); - } - - if (error != CRYPTO_SUCCESS && error != CRYPTO_QUEUED && - IS_RECOVERABLE(error)) { + if (error != CRYPTO_SUCCESS && IS_RECOVERABLE(error)) { /* Add pd to the linked list of providers tried. */ - if (kcf_insert_triedlist(&list, pd, KCF_KMFLAG(crq)) != NULL) + if (kcf_insert_triedlist(&list, pd, KM_SLEEP) != NULL) goto retry; } @@ -417,147 +105,6 @@ crypto_encrypt(crypto_mechanism_t *mech, crypto_data_t *plaintext, return (error); } -/* - * crypto_encrypt_init_prov() - * - * Calls crypto_cipher_init_prov() to initialize an encryption operation. - */ -int -crypto_encrypt_init_prov(crypto_provider_t pd, crypto_session_id_t sid, - crypto_mechanism_t *mech, crypto_key_t *key, - crypto_ctx_template_t tmpl, crypto_context_t *ctxp, - crypto_call_req_t *crq) -{ - return (crypto_cipher_init_prov(pd, sid, mech, key, tmpl, ctxp, crq, - CRYPTO_FG_ENCRYPT)); -} - -/* - * crypto_encrypt_init() - * - * Calls crypto_cipher_init() to initialize an encryption operation - */ -int -crypto_encrypt_init(crypto_mechanism_t *mech, crypto_key_t *key, - crypto_ctx_template_t tmpl, crypto_context_t *ctxp, - crypto_call_req_t *crq) -{ - return (crypto_cipher_init(mech, key, tmpl, ctxp, crq, - CRYPTO_FG_ENCRYPT)); -} - -/* - * crypto_encrypt_update() - * - * Arguments: - * context: A crypto_context_t initialized by encrypt_init(). - * plaintext: The message part to be encrypted - * ciphertext: Storage for the encrypted message part. - * cr: crypto_call_req_t calling conditions and call back info. - * - * Description: - * Asynchronously submits a request for, or synchronously performs a - * part of an encryption operation. - * - * Context: - * Process or interrupt, according to the semantics dictated by the 'cr'. - * - * Returns: - * See comment in the beginning of the file. - */ -int -crypto_encrypt_update(crypto_context_t context, crypto_data_t *plaintext, - crypto_data_t *ciphertext, crypto_call_req_t *cr) -{ - crypto_ctx_t *ctx = (crypto_ctx_t *)context; - kcf_context_t *kcf_ctx; - kcf_provider_desc_t *pd; - int error; - kcf_req_params_t params; - - if ((ctx == NULL) || - ((kcf_ctx = (kcf_context_t *)ctx->cc_framework_private) == NULL) || - ((pd = kcf_ctx->kc_prov_desc) == NULL)) { - return (CRYPTO_INVALID_CONTEXT); - } - - ASSERT(pd->pd_prov_type != CRYPTO_LOGICAL_PROVIDER); - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(cr, pd)) { - error = KCF_PROV_ENCRYPT_UPDATE(pd, ctx, plaintext, - ciphertext, NULL); - KCF_PROV_INCRSTATS(pd, error); - return (error); - } - - /* Check if we should use a software provider for small jobs */ - if ((ctx->cc_flags & CRYPTO_USE_OPSTATE) && cr == NULL) { - if (plaintext->cd_length < kcf_ctx->kc_mech->me_threshold && - kcf_ctx->kc_sw_prov_desc != NULL && - KCF_IS_PROV_USABLE(kcf_ctx->kc_sw_prov_desc)) { - pd = kcf_ctx->kc_sw_prov_desc; - } - } - - KCF_WRAP_ENCRYPT_OPS_PARAMS(¶ms, KCF_OP_UPDATE, - ctx->cc_session, NULL, NULL, plaintext, ciphertext, NULL); - error = kcf_submit_request(pd, ctx, cr, ¶ms, B_FALSE); - - return (error); -} - -/* - * crypto_encrypt_final() - * - * Arguments: - * context: A crypto_context_t initialized by encrypt_init(). - * ciphertext: Storage for the last part of encrypted message - * cr: crypto_call_req_t calling conditions and call back info. - * - * Description: - * Asynchronously submits a request for, or synchronously performs the - * final part of an encryption operation. - * - * Context: - * Process or interrupt, according to the semantics dictated by the 'cr'. - * - * Returns: - * See comment in the beginning of the file. - */ -int -crypto_encrypt_final(crypto_context_t context, crypto_data_t *ciphertext, - crypto_call_req_t *cr) -{ - crypto_ctx_t *ctx = (crypto_ctx_t *)context; - kcf_context_t *kcf_ctx; - kcf_provider_desc_t *pd; - int error; - kcf_req_params_t params; - - if ((ctx == NULL) || - ((kcf_ctx = (kcf_context_t *)ctx->cc_framework_private) == NULL) || - ((pd = kcf_ctx->kc_prov_desc) == NULL)) { - return (CRYPTO_INVALID_CONTEXT); - } - - ASSERT(pd->pd_prov_type != CRYPTO_LOGICAL_PROVIDER); - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(cr, pd)) { - error = KCF_PROV_ENCRYPT_FINAL(pd, ctx, ciphertext, NULL); - KCF_PROV_INCRSTATS(pd, error); - } else { - KCF_WRAP_ENCRYPT_OPS_PARAMS(¶ms, KCF_OP_FINAL, - ctx->cc_session, NULL, NULL, NULL, ciphertext, NULL); - error = kcf_submit_request(pd, ctx, cr, ¶ms, B_FALSE); - } - - /* Release the hold done in kcf_new_ctx() during init step. */ - KCF_CONTEXT_COND_RELEASE(error, kcf_ctx); - return (error); -} - /* * crypto_decrypt_prov() * @@ -576,7 +123,6 @@ crypto_encrypt_final(crypto_context_t context, crypto_data_t *ciphertext, * tmpl: a crypto_ctx_template_t, opaque template of a context of an * encryption with the 'mech' using 'key'. 'tmpl' is created by * a previous call to crypto_create_ctx_template(). - * cr: crypto_call_req_t calling conditions and call back info. * * Description: * Asynchronously submits a request for, or synchronously performs a @@ -584,58 +130,17 @@ crypto_encrypt_final(crypto_context_t context, crypto_data_t *ciphertext, * the key 'key'. * When complete and successful, 'plaintext' will contain the decrypted * message. - * - * Context: - * Process or interrupt, according to the semantics dictated by the 'cr'. + * Relies on the KCF scheduler to choose a provider. * * Returns: * See comment in the beginning of the file. */ int -crypto_decrypt_prov(crypto_provider_t provider, crypto_session_id_t sid, - crypto_mechanism_t *mech, crypto_data_t *ciphertext, crypto_key_t *key, - crypto_ctx_template_t tmpl, crypto_data_t *plaintext, - crypto_call_req_t *crq) -{ - kcf_req_params_t params; - kcf_provider_desc_t *pd = provider; - kcf_provider_desc_t *real_provider = pd; - int rv; - - ASSERT(KCF_PROV_REFHELD(pd)); - - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) { - rv = kcf_get_hardware_provider(mech->cm_type, - CRYPTO_MECH_INVALID, CHECK_RESTRICT(crq), pd, - &real_provider, CRYPTO_FG_DECRYPT_ATOMIC); - - if (rv != CRYPTO_SUCCESS) - return (rv); - } - - KCF_WRAP_DECRYPT_OPS_PARAMS(¶ms, KCF_OP_ATOMIC, sid, mech, key, - ciphertext, plaintext, tmpl); - - rv = kcf_submit_request(real_provider, NULL, crq, ¶ms, B_FALSE); - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) - KCF_PROV_REFRELE(real_provider); - - return (rv); -} - -/* - * Same as crypto_decrypt_prov(), but relies on the KCF scheduler to - * choose a provider. See crypto_decrypt_prov() comments for more - * information. - */ -int crypto_decrypt(crypto_mechanism_t *mech, crypto_data_t *ciphertext, - crypto_key_t *key, crypto_ctx_template_t tmpl, crypto_data_t *plaintext, - crypto_call_req_t *crq) + crypto_key_t *key, crypto_ctx_template_t tmpl, crypto_data_t *plaintext) { int error; kcf_mech_entry_t *me; - kcf_req_params_t params; kcf_provider_desc_t *pd; kcf_ctx_template_t *ctx_tmpl; crypto_spi_ctx_template_t spi_ctx_tmpl = NULL; @@ -644,52 +149,24 @@ crypto_decrypt(crypto_mechanism_t *mech, crypto_data_t *ciphertext, retry: /* pd is returned held */ if ((pd = kcf_get_mech_provider(mech->cm_type, &me, &error, - list, CRYPTO_FG_DECRYPT_ATOMIC, CHECK_RESTRICT(crq), - ciphertext->cd_length)) == NULL) { + list, CRYPTO_FG_DECRYPT_ATOMIC)) == NULL) { if (list != NULL) kcf_free_triedlist(list); return (error); } - /* - * For SW providers, check the validity of the context template - * It is very rare that the generation number mis-matches, so - * is acceptable to fail here, and let the consumer recover by - * freeing this tmpl and create a new one for the key and new SW - * provider - */ - if ((pd->pd_prov_type == CRYPTO_SW_PROVIDER) && - ((ctx_tmpl = (kcf_ctx_template_t *)tmpl) != NULL)) { - if (ctx_tmpl->ct_generation != me->me_gen_swprov) { - if (list != NULL) - kcf_free_triedlist(list); - KCF_PROV_REFRELE(pd); - return (CRYPTO_OLD_CTX_TEMPLATE); - } else { - spi_ctx_tmpl = ctx_tmpl->ct_prov_tmpl; - } - } - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(crq, pd)) { - crypto_mechanism_t lmech; + if (((ctx_tmpl = (kcf_ctx_template_t *)tmpl) != NULL)) + spi_ctx_tmpl = ctx_tmpl->ct_prov_tmpl; - lmech = *mech; - KCF_SET_PROVIDER_MECHNUM(mech->cm_type, pd, &lmech); + crypto_mechanism_t lmech = *mech; + KCF_SET_PROVIDER_MECHNUM(mech->cm_type, pd, &lmech); - error = KCF_PROV_DECRYPT_ATOMIC(pd, pd->pd_sid, &lmech, key, - ciphertext, plaintext, spi_ctx_tmpl, KCF_SWFP_RHNDL(crq)); - KCF_PROV_INCRSTATS(pd, error); - } else { - KCF_WRAP_DECRYPT_OPS_PARAMS(¶ms, KCF_OP_ATOMIC, pd->pd_sid, - mech, key, ciphertext, plaintext, spi_ctx_tmpl); - error = kcf_submit_request(pd, NULL, crq, ¶ms, B_FALSE); - } + error = KCF_PROV_DECRYPT_ATOMIC(pd, &lmech, key, + ciphertext, plaintext, spi_ctx_tmpl); - if (error != CRYPTO_SUCCESS && error != CRYPTO_QUEUED && - IS_RECOVERABLE(error)) { + if (error != CRYPTO_SUCCESS && IS_RECOVERABLE(error)) { /* Add pd to the linked list of providers tried. */ - if (kcf_insert_triedlist(&list, pd, KCF_KMFLAG(crq)) != NULL) + if (kcf_insert_triedlist(&list, pd, KM_SLEEP) != NULL) goto retry; } @@ -700,231 +177,7 @@ crypto_decrypt(crypto_mechanism_t *mech, crypto_data_t *ciphertext, return (error); } -/* - * crypto_decrypt_init_prov() - * - * Calls crypto_cipher_init_prov() to initialize a decryption operation - */ -int -crypto_decrypt_init_prov(crypto_provider_t pd, crypto_session_id_t sid, - crypto_mechanism_t *mech, crypto_key_t *key, - crypto_ctx_template_t tmpl, crypto_context_t *ctxp, - crypto_call_req_t *crq) -{ - return (crypto_cipher_init_prov(pd, sid, mech, key, tmpl, ctxp, crq, - CRYPTO_FG_DECRYPT)); -} - -/* - * crypto_decrypt_init() - * - * Calls crypto_cipher_init() to initialize a decryption operation - */ -int -crypto_decrypt_init(crypto_mechanism_t *mech, crypto_key_t *key, - crypto_ctx_template_t tmpl, crypto_context_t *ctxp, - crypto_call_req_t *crq) -{ - return (crypto_cipher_init(mech, key, tmpl, ctxp, crq, - CRYPTO_FG_DECRYPT)); -} - -/* - * crypto_decrypt_update() - * - * Arguments: - * context: A crypto_context_t initialized by decrypt_init(). - * ciphertext: The message part to be decrypted - * plaintext: Storage for the decrypted message part. - * cr: crypto_call_req_t calling conditions and call back info. - * - * Description: - * Asynchronously submits a request for, or synchronously performs a - * part of an decryption operation. - * - * Context: - * Process or interrupt, according to the semantics dictated by the 'cr'. - * - * Returns: - * See comment in the beginning of the file. - */ -int -crypto_decrypt_update(crypto_context_t context, crypto_data_t *ciphertext, - crypto_data_t *plaintext, crypto_call_req_t *cr) -{ - crypto_ctx_t *ctx = (crypto_ctx_t *)context; - kcf_context_t *kcf_ctx; - kcf_provider_desc_t *pd; - int error; - kcf_req_params_t params; - - if ((ctx == NULL) || - ((kcf_ctx = (kcf_context_t *)ctx->cc_framework_private) == NULL) || - ((pd = kcf_ctx->kc_prov_desc) == NULL)) { - return (CRYPTO_INVALID_CONTEXT); - } - - ASSERT(pd->pd_prov_type != CRYPTO_LOGICAL_PROVIDER); - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(cr, pd)) { - error = KCF_PROV_DECRYPT_UPDATE(pd, ctx, ciphertext, - plaintext, NULL); - KCF_PROV_INCRSTATS(pd, error); - return (error); - } - - /* Check if we should use a software provider for small jobs */ - if ((ctx->cc_flags & CRYPTO_USE_OPSTATE) && cr == NULL) { - if (ciphertext->cd_length < kcf_ctx->kc_mech->me_threshold && - kcf_ctx->kc_sw_prov_desc != NULL && - KCF_IS_PROV_USABLE(kcf_ctx->kc_sw_prov_desc)) { - pd = kcf_ctx->kc_sw_prov_desc; - } - } - - KCF_WRAP_DECRYPT_OPS_PARAMS(¶ms, KCF_OP_UPDATE, - ctx->cc_session, NULL, NULL, ciphertext, plaintext, NULL); - error = kcf_submit_request(pd, ctx, cr, ¶ms, B_FALSE); - - return (error); -} - -/* - * crypto_decrypt_final() - * - * Arguments: - * context: A crypto_context_t initialized by decrypt_init(). - * plaintext: Storage for the last part of the decrypted message - * cr: crypto_call_req_t calling conditions and call back info. - * - * Description: - * Asynchronously submits a request for, or synchronously performs the - * final part of a decryption operation. - * - * Context: - * Process or interrupt, according to the semantics dictated by the 'cr'. - * - * Returns: - * See comment in the beginning of the file. - */ -int -crypto_decrypt_final(crypto_context_t context, crypto_data_t *plaintext, - crypto_call_req_t *cr) -{ - crypto_ctx_t *ctx = (crypto_ctx_t *)context; - kcf_context_t *kcf_ctx; - kcf_provider_desc_t *pd; - int error; - kcf_req_params_t params; - - if ((ctx == NULL) || - ((kcf_ctx = (kcf_context_t *)ctx->cc_framework_private) == NULL) || - ((pd = kcf_ctx->kc_prov_desc) == NULL)) { - return (CRYPTO_INVALID_CONTEXT); - } - - ASSERT(pd->pd_prov_type != CRYPTO_LOGICAL_PROVIDER); - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(cr, pd)) { - error = KCF_PROV_DECRYPT_FINAL(pd, ctx, plaintext, - NULL); - KCF_PROV_INCRSTATS(pd, error); - } else { - KCF_WRAP_DECRYPT_OPS_PARAMS(¶ms, KCF_OP_FINAL, - ctx->cc_session, NULL, NULL, NULL, plaintext, NULL); - error = kcf_submit_request(pd, ctx, cr, ¶ms, B_FALSE); - } - - /* Release the hold done in kcf_new_ctx() during init step. */ - KCF_CONTEXT_COND_RELEASE(error, kcf_ctx); - return (error); -} - -/* - * See comments for crypto_encrypt_update(). - */ -int -crypto_encrypt_single(crypto_context_t context, crypto_data_t *plaintext, - crypto_data_t *ciphertext, crypto_call_req_t *cr) -{ - crypto_ctx_t *ctx = (crypto_ctx_t *)context; - kcf_context_t *kcf_ctx; - kcf_provider_desc_t *pd; - int error; - kcf_req_params_t params; - - if ((ctx == NULL) || - ((kcf_ctx = (kcf_context_t *)ctx->cc_framework_private) == NULL) || - ((pd = kcf_ctx->kc_prov_desc) == NULL)) { - return (CRYPTO_INVALID_CONTEXT); - } - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(cr, pd)) { - error = KCF_PROV_ENCRYPT(pd, ctx, plaintext, - ciphertext, NULL); - KCF_PROV_INCRSTATS(pd, error); - } else { - KCF_WRAP_ENCRYPT_OPS_PARAMS(¶ms, KCF_OP_SINGLE, pd->pd_sid, - NULL, NULL, plaintext, ciphertext, NULL); - error = kcf_submit_request(pd, ctx, cr, ¶ms, B_FALSE); - } - - /* Release the hold done in kcf_new_ctx() during init step. */ - KCF_CONTEXT_COND_RELEASE(error, kcf_ctx); - return (error); -} - -/* - * See comments for crypto_decrypt_update(). - */ -int -crypto_decrypt_single(crypto_context_t context, crypto_data_t *ciphertext, - crypto_data_t *plaintext, crypto_call_req_t *cr) -{ - crypto_ctx_t *ctx = (crypto_ctx_t *)context; - kcf_context_t *kcf_ctx; - kcf_provider_desc_t *pd; - int error; - kcf_req_params_t params; - - if ((ctx == NULL) || - ((kcf_ctx = (kcf_context_t *)ctx->cc_framework_private) == NULL) || - ((pd = kcf_ctx->kc_prov_desc) == NULL)) { - return (CRYPTO_INVALID_CONTEXT); - } - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(cr, pd)) { - error = KCF_PROV_DECRYPT(pd, ctx, ciphertext, - plaintext, NULL); - KCF_PROV_INCRSTATS(pd, error); - } else { - KCF_WRAP_DECRYPT_OPS_PARAMS(¶ms, KCF_OP_SINGLE, pd->pd_sid, - NULL, NULL, ciphertext, plaintext, NULL); - error = kcf_submit_request(pd, ctx, cr, ¶ms, B_FALSE); - } - - /* Release the hold done in kcf_new_ctx() during init step. */ - KCF_CONTEXT_COND_RELEASE(error, kcf_ctx); - return (error); -} - #if defined(_KERNEL) -EXPORT_SYMBOL(crypto_encrypt_prov); EXPORT_SYMBOL(crypto_encrypt); -EXPORT_SYMBOL(crypto_encrypt_init_prov); -EXPORT_SYMBOL(crypto_encrypt_init); -EXPORT_SYMBOL(crypto_encrypt_update); -EXPORT_SYMBOL(crypto_encrypt_final); -EXPORT_SYMBOL(crypto_decrypt_prov); EXPORT_SYMBOL(crypto_decrypt); -EXPORT_SYMBOL(crypto_decrypt_init_prov); -EXPORT_SYMBOL(crypto_decrypt_init); -EXPORT_SYMBOL(crypto_decrypt_update); -EXPORT_SYMBOL(crypto_decrypt_final); -EXPORT_SYMBOL(crypto_encrypt_single); -EXPORT_SYMBOL(crypto_decrypt_single); #endif diff --git a/module/icp/api/kcf_ctxops.c b/module/icp/api/kcf_ctxops.c index 21b0977d3634..4fa281676b81 100644 --- a/module/icp/api/kcf_ctxops.c +++ b/module/icp/api/kcf_ctxops.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -48,7 +48,6 @@ * ptmpl: a storage for the opaque crypto_ctx_template_t, allocated and * initialized by the software provider this routine is * dispatched to. - * kmflag: KM_SLEEP/KM_NOSLEEP mem. alloc. flag. * * Description: * Redirects the call to the software provider of the specified @@ -69,7 +68,7 @@ */ int crypto_create_ctx_template(crypto_mechanism_t *mech, crypto_key_t *key, - crypto_ctx_template_t *ptmpl, int kmflag) + crypto_ctx_template_t *ptmpl) { int error; kcf_mech_entry_t *me; @@ -90,7 +89,7 @@ crypto_create_ctx_template(crypto_mechanism_t *mech, crypto_key_t *key, return (error); if ((ctx_tmpl = (kcf_ctx_template_t *)kmem_alloc( - sizeof (kcf_ctx_template_t), kmflag)) == NULL) { + sizeof (kcf_ctx_template_t), KM_SLEEP)) == NULL) { KCF_PROV_REFRELE(pd); return (CRYPTO_HOST_MEMORY); } @@ -101,10 +100,9 @@ crypto_create_ctx_template(crypto_mechanism_t *mech, crypto_key_t *key, prov_mech.cm_param_len = mech->cm_param_len; error = KCF_PROV_CREATE_CTX_TEMPLATE(pd, &prov_mech, key, - &(ctx_tmpl->ct_prov_tmpl), &(ctx_tmpl->ct_size), KCF_RHNDL(kmflag)); + &(ctx_tmpl->ct_prov_tmpl), &(ctx_tmpl->ct_size)); if (error == CRYPTO_SUCCESS) { - ctx_tmpl->ct_generation = me->me_gen_swprov; *ptmpl = ctx_tmpl; } else { kmem_free(ctx_tmpl, sizeof (kcf_ctx_template_t)); @@ -140,7 +138,7 @@ crypto_destroy_ctx_template(crypto_ctx_template_t tmpl) ASSERT(ctx_tmpl->ct_prov_tmpl != NULL); - bzero(ctx_tmpl->ct_prov_tmpl, ctx_tmpl->ct_size); + memset(ctx_tmpl->ct_prov_tmpl, 0, ctx_tmpl->ct_size); kmem_free(ctx_tmpl->ct_prov_tmpl, ctx_tmpl->ct_size); kmem_free(ctx_tmpl, sizeof (kcf_ctx_template_t)); } diff --git a/module/icp/api/kcf_digest.c b/module/icp/api/kcf_digest.c deleted file mode 100644 index aa68d69bc162..000000000000 --- a/module/icp/api/kcf_digest.c +++ /dev/null @@ -1,491 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License (the "License"). - * You may not use this file except in compliance with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2007 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#include -#include -#include -#include -#include -#include - -/* - * Message digest routines - */ - -/* - * The following are the possible returned values common to all the routines - * below. The applicability of some of these return values depends on the - * presence of the arguments. - * - * CRYPTO_SUCCESS: The operation completed successfully. - * CRYPTO_QUEUED: A request was submitted successfully. The callback - * routine will be called when the operation is done. - * CRYPTO_MECHANISM_INVALID or CRYPTO_INVALID_MECH_PARAM - * for problems with the 'mech'. - * CRYPTO_INVALID_DATA for bogus 'data' - * CRYPTO_HOST_MEMORY for failure to allocate memory to handle this work. - * CRYPTO_INVALID_CONTEXT: Not a valid context. - * CRYPTO_BUSY: Cannot process the request now. Schedule a - * crypto_bufcall(), or try later. - * CRYPTO_NOT_SUPPORTED and CRYPTO_MECH_NOT_SUPPORTED: - * No provider is capable of a function or a mechanism. - */ - - -/* - * crypto_digest_prov() - * - * Arguments: - * pd: pointer to the descriptor of the provider to use for this - * operation. - * sid: provider session id. - * mech: crypto_mechanism_t pointer. - * mech_type is a valid value previously returned by - * crypto_mech2id(); - * When the mech's parameter is not NULL, its definition depends - * on the standard definition of the mechanism. - * data: The message to be digested. - * digest: Storage for the digest. The length needed depends on the - * mechanism. - * cr: crypto_call_req_t calling conditions and call back info. - * - * Description: - * Asynchronously submits a request for, or synchronously performs the - * digesting operation of 'data' on the specified - * provider with the specified session. - * When complete and successful, 'digest' will contain the digest value. - * The caller should hold a reference on the specified provider - * descriptor before calling this function. - * - * Context: - * Process or interrupt, according to the semantics dictated by the 'cr'. - * - * Returns: - * See comment in the beginning of the file. - */ -int -crypto_digest_prov(crypto_provider_t provider, crypto_session_id_t sid, - crypto_mechanism_t *mech, crypto_data_t *data, crypto_data_t *digest, - crypto_call_req_t *crq) -{ - kcf_req_params_t params; - kcf_provider_desc_t *pd = provider; - kcf_provider_desc_t *real_provider = pd; - int rv; - - ASSERT(KCF_PROV_REFHELD(pd)); - - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) { - rv = kcf_get_hardware_provider(mech->cm_type, - CRYPTO_MECH_INVALID, CHECK_RESTRICT(crq), - pd, &real_provider, CRYPTO_FG_DIGEST_ATOMIC); - - if (rv != CRYPTO_SUCCESS) - return (rv); - } - KCF_WRAP_DIGEST_OPS_PARAMS(¶ms, KCF_OP_ATOMIC, sid, mech, NULL, - data, digest); - - /* no crypto context to carry between multiple parts. */ - rv = kcf_submit_request(real_provider, NULL, crq, ¶ms, B_FALSE); - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) - KCF_PROV_REFRELE(real_provider); - - return (rv); -} - - -/* - * Same as crypto_digest_prov(), but relies on the KCF scheduler to - * choose a provider. See crypto_digest_prov() comments for more information. - */ -int -crypto_digest(crypto_mechanism_t *mech, crypto_data_t *data, - crypto_data_t *digest, crypto_call_req_t *crq) -{ - int error; - kcf_provider_desc_t *pd; - kcf_req_params_t params; - kcf_prov_tried_t *list = NULL; - -retry: - /* The pd is returned held */ - if ((pd = kcf_get_mech_provider(mech->cm_type, NULL, &error, list, - CRYPTO_FG_DIGEST_ATOMIC, CHECK_RESTRICT(crq), - data->cd_length)) == NULL) { - if (list != NULL) - kcf_free_triedlist(list); - return (error); - } - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(crq, pd)) { - crypto_mechanism_t lmech; - - lmech = *mech; - KCF_SET_PROVIDER_MECHNUM(mech->cm_type, pd, &lmech); - error = KCF_PROV_DIGEST_ATOMIC(pd, pd->pd_sid, &lmech, data, - digest, KCF_SWFP_RHNDL(crq)); - KCF_PROV_INCRSTATS(pd, error); - } else { - if (pd->pd_prov_type == CRYPTO_HW_PROVIDER && - (pd->pd_flags & CRYPTO_HASH_NO_UPDATE) && - (data->cd_length > pd->pd_hash_limit)) { - error = CRYPTO_BUFFER_TOO_BIG; - } else { - KCF_WRAP_DIGEST_OPS_PARAMS(¶ms, KCF_OP_ATOMIC, - pd->pd_sid, mech, NULL, data, digest); - - /* no crypto context to carry between multiple parts. */ - error = kcf_submit_request(pd, NULL, crq, ¶ms, - B_FALSE); - } - } - - if (error != CRYPTO_SUCCESS && error != CRYPTO_QUEUED && - IS_RECOVERABLE(error)) { - /* Add pd to the linked list of providers tried. */ - if (kcf_insert_triedlist(&list, pd, KCF_KMFLAG(crq)) != NULL) - goto retry; - } - - if (list != NULL) - kcf_free_triedlist(list); - - KCF_PROV_REFRELE(pd); - return (error); -} - -/* - * crypto_digest_init_prov() - * - * pd: pointer to the descriptor of the provider to use for this - * operation. - * sid: provider session id. - * mech: crypto_mechanism_t pointer. - * mech_type is a valid value previously returned by - * crypto_mech2id(); - * When the mech's parameter is not NULL, its definition depends - * on the standard definition of the mechanism. - * ctxp: Pointer to a crypto_context_t. - * cr: crypto_call_req_t calling conditions and call back info. - * - * Description: - * Asynchronously submits a request for, or synchronously performs the - * initialization of a message digest operation on the specified - * provider with the specified session. - * When complete and successful, 'ctxp' will contain a crypto_context_t - * valid for later calls to digest_update() and digest_final(). - * The caller should hold a reference on the specified provider - * descriptor before calling this function. - */ -int -crypto_digest_init_prov(crypto_provider_t provider, crypto_session_id_t sid, - crypto_mechanism_t *mech, crypto_context_t *ctxp, crypto_call_req_t *crq) -{ - int error; - crypto_ctx_t *ctx; - kcf_req_params_t params; - kcf_provider_desc_t *pd = provider; - kcf_provider_desc_t *real_provider = pd; - - ASSERT(KCF_PROV_REFHELD(pd)); - - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) { - error = kcf_get_hardware_provider(mech->cm_type, - CRYPTO_MECH_INVALID, CHECK_RESTRICT(crq), pd, - &real_provider, CRYPTO_FG_DIGEST); - - if (error != CRYPTO_SUCCESS) - return (error); - } - - /* Allocate and initialize the canonical context */ - if ((ctx = kcf_new_ctx(crq, real_provider, sid)) == NULL) { - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) - KCF_PROV_REFRELE(real_provider); - return (CRYPTO_HOST_MEMORY); - } - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(crq, pd)) { - crypto_mechanism_t lmech; - - lmech = *mech; - KCF_SET_PROVIDER_MECHNUM(mech->cm_type, real_provider, &lmech); - error = KCF_PROV_DIGEST_INIT(real_provider, ctx, &lmech, - KCF_SWFP_RHNDL(crq)); - KCF_PROV_INCRSTATS(pd, error); - } else { - KCF_WRAP_DIGEST_OPS_PARAMS(¶ms, KCF_OP_INIT, sid, - mech, NULL, NULL, NULL); - error = kcf_submit_request(real_provider, ctx, crq, ¶ms, - B_FALSE); - } - - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) - KCF_PROV_REFRELE(real_provider); - - if ((error == CRYPTO_SUCCESS) || (error == CRYPTO_QUEUED)) - *ctxp = (crypto_context_t)ctx; - else { - /* Release the hold done in kcf_new_ctx(). */ - KCF_CONTEXT_REFRELE((kcf_context_t *)ctx->cc_framework_private); - } - - return (error); -} - -/* - * Same as crypto_digest_init_prov(), but relies on the KCF scheduler - * to choose a provider. See crypto_digest_init_prov() comments for - * more information. - */ -int -crypto_digest_init(crypto_mechanism_t *mech, crypto_context_t *ctxp, - crypto_call_req_t *crq) -{ - int error; - kcf_provider_desc_t *pd; - kcf_prov_tried_t *list = NULL; - -retry: - /* The pd is returned held */ - if ((pd = kcf_get_mech_provider(mech->cm_type, NULL, &error, - list, CRYPTO_FG_DIGEST, CHECK_RESTRICT(crq), 0)) == NULL) { - if (list != NULL) - kcf_free_triedlist(list); - return (error); - } - - if (pd->pd_prov_type == CRYPTO_HW_PROVIDER && - (pd->pd_flags & CRYPTO_HASH_NO_UPDATE)) { - /* - * The hardware provider has limited digest support. - * So, we fallback early here to using a software provider. - * - * XXX - need to enhance to do the fallback later in - * crypto_digest_update() if the size of accumulated input data - * exceeds the maximum size digestable by hardware provider. - */ - error = CRYPTO_BUFFER_TOO_BIG; - } else { - error = crypto_digest_init_prov(pd, pd->pd_sid, - mech, ctxp, crq); - } - - if (error != CRYPTO_SUCCESS && error != CRYPTO_QUEUED && - IS_RECOVERABLE(error)) { - /* Add pd to the linked list of providers tried. */ - if (kcf_insert_triedlist(&list, pd, KCF_KMFLAG(crq)) != NULL) - goto retry; - } - - if (list != NULL) - kcf_free_triedlist(list); - KCF_PROV_REFRELE(pd); - return (error); -} - -/* - * crypto_digest_update() - * - * Arguments: - * context: A crypto_context_t initialized by digest_init(). - * data: The part of message to be digested. - * cr: crypto_call_req_t calling conditions and call back info. - * - * Description: - * Asynchronously submits a request for, or synchronously performs a - * part of a message digest operation. - * - * Context: - * Process or interrupt, according to the semantics dictated by the 'cr'. - * - * Returns: - * See comment in the beginning of the file. - */ -int -crypto_digest_update(crypto_context_t context, crypto_data_t *data, - crypto_call_req_t *cr) -{ - crypto_ctx_t *ctx = (crypto_ctx_t *)context; - kcf_context_t *kcf_ctx; - kcf_provider_desc_t *pd; - int error; - kcf_req_params_t params; - - if ((ctx == NULL) || - ((kcf_ctx = (kcf_context_t *)ctx->cc_framework_private) == NULL) || - ((pd = kcf_ctx->kc_prov_desc) == NULL)) { - return (CRYPTO_INVALID_CONTEXT); - } - - ASSERT(pd->pd_prov_type != CRYPTO_LOGICAL_PROVIDER); - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(cr, pd)) { - error = KCF_PROV_DIGEST_UPDATE(pd, ctx, data, NULL); - KCF_PROV_INCRSTATS(pd, error); - } else { - KCF_WRAP_DIGEST_OPS_PARAMS(¶ms, KCF_OP_UPDATE, - ctx->cc_session, NULL, NULL, data, NULL); - error = kcf_submit_request(pd, ctx, cr, ¶ms, B_FALSE); - } - - return (error); -} - -/* - * crypto_digest_final() - * - * Arguments: - * context: A crypto_context_t initialized by digest_init(). - * digest: The storage for the digest. - * cr: crypto_call_req_t calling conditions and call back info. - * - * Description: - * Asynchronously submits a request for, or synchronously performs the - * final part of a message digest operation. - * - * Context: - * Process or interrupt, according to the semantics dictated by the 'cr'. - * - * Returns: - * See comment in the beginning of the file. - */ -int -crypto_digest_final(crypto_context_t context, crypto_data_t *digest, - crypto_call_req_t *cr) -{ - crypto_ctx_t *ctx = (crypto_ctx_t *)context; - kcf_context_t *kcf_ctx; - kcf_provider_desc_t *pd; - int error; - kcf_req_params_t params; - - if ((ctx == NULL) || - ((kcf_ctx = (kcf_context_t *)ctx->cc_framework_private) == NULL) || - ((pd = kcf_ctx->kc_prov_desc) == NULL)) { - return (CRYPTO_INVALID_CONTEXT); - } - - ASSERT(pd->pd_prov_type != CRYPTO_LOGICAL_PROVIDER); - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(cr, pd)) { - error = KCF_PROV_DIGEST_FINAL(pd, ctx, digest, NULL); - KCF_PROV_INCRSTATS(pd, error); - } else { - KCF_WRAP_DIGEST_OPS_PARAMS(¶ms, KCF_OP_FINAL, - ctx->cc_session, NULL, NULL, NULL, digest); - error = kcf_submit_request(pd, ctx, cr, ¶ms, B_FALSE); - } - - /* Release the hold done in kcf_new_ctx() during init step. */ - KCF_CONTEXT_COND_RELEASE(error, kcf_ctx); - return (error); -} - -/* - * Performs a digest update on the specified key. Note that there is - * no k-API crypto_digest_key() equivalent of this function. - */ -int -crypto_digest_key_prov(crypto_context_t context, crypto_key_t *key, - crypto_call_req_t *cr) -{ - crypto_ctx_t *ctx = (crypto_ctx_t *)context; - kcf_context_t *kcf_ctx; - kcf_provider_desc_t *pd; - int error; - kcf_req_params_t params; - - if ((ctx == NULL) || - ((kcf_ctx = (kcf_context_t *)ctx->cc_framework_private) == NULL) || - ((pd = kcf_ctx->kc_prov_desc) == NULL)) { - return (CRYPTO_INVALID_CONTEXT); - } - - ASSERT(pd->pd_prov_type != CRYPTO_LOGICAL_PROVIDER); - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(cr, pd)) { - error = KCF_PROV_DIGEST_KEY(pd, ctx, key, NULL); - KCF_PROV_INCRSTATS(pd, error); - } else { - KCF_WRAP_DIGEST_OPS_PARAMS(¶ms, KCF_OP_DIGEST_KEY, - ctx->cc_session, NULL, key, NULL, NULL); - error = kcf_submit_request(pd, ctx, cr, ¶ms, B_FALSE); - } - - return (error); -} - -/* - * See comments for crypto_digest_update() and crypto_digest_final(). - */ -int -crypto_digest_single(crypto_context_t context, crypto_data_t *data, - crypto_data_t *digest, crypto_call_req_t *cr) -{ - crypto_ctx_t *ctx = (crypto_ctx_t *)context; - kcf_context_t *kcf_ctx; - kcf_provider_desc_t *pd; - int error; - kcf_req_params_t params; - - if ((ctx == NULL) || - ((kcf_ctx = (kcf_context_t *)ctx->cc_framework_private) == NULL) || - ((pd = kcf_ctx->kc_prov_desc) == NULL)) { - return (CRYPTO_INVALID_CONTEXT); - } - - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(cr, pd)) { - error = KCF_PROV_DIGEST(pd, ctx, data, digest, NULL); - KCF_PROV_INCRSTATS(pd, error); - } else { - KCF_WRAP_DIGEST_OPS_PARAMS(¶ms, KCF_OP_SINGLE, pd->pd_sid, - NULL, NULL, data, digest); - error = kcf_submit_request(pd, ctx, cr, ¶ms, B_FALSE); - } - - /* Release the hold done in kcf_new_ctx() during init step. */ - KCF_CONTEXT_COND_RELEASE(error, kcf_ctx); - return (error); -} - -#if defined(_KERNEL) -EXPORT_SYMBOL(crypto_digest_prov); -EXPORT_SYMBOL(crypto_digest); -EXPORT_SYMBOL(crypto_digest_init_prov); -EXPORT_SYMBOL(crypto_digest_init); -EXPORT_SYMBOL(crypto_digest_update); -EXPORT_SYMBOL(crypto_digest_final); -EXPORT_SYMBOL(crypto_digest_key_prov); -EXPORT_SYMBOL(crypto_digest_single); -#endif diff --git a/module/icp/api/kcf_mac.c b/module/icp/api/kcf_mac.c index a7722d8f914c..287467e68350 100644 --- a/module/icp/api/kcf_mac.c +++ b/module/icp/api/kcf_mac.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -40,15 +40,12 @@ * presence of the arguments. * * CRYPTO_SUCCESS: The operation completed successfully. - * CRYPTO_QUEUED: A request was submitted successfully. The callback - * routine will be called when the operation is done. * CRYPTO_INVALID_MECH_NUMBER, CRYPTO_INVALID_MECH_PARAM, or * CRYPTO_INVALID_MECH for problems with the 'mech'. * CRYPTO_INVALID_DATA for bogus 'data' * CRYPTO_HOST_MEMORY for failure to allocate memory to handle this work. * CRYPTO_INVALID_CONTEXT: Not a valid context. - * CRYPTO_BUSY: Cannot process the request now. Schedule a - * crypto_bufcall(), or try later. + * CRYPTO_BUSY: Cannot process the request now. Try later. * CRYPTO_NOT_SUPPORTED and CRYPTO_MECH_NOT_SUPPORTED: No provider is * capable of a function or a mechanism. * CRYPTO_INVALID_KEY: bogus 'key' argument. @@ -70,7 +67,6 @@ * tmpl: a crypto_ctx_template_t, opaque template of a context of a * MAC with the 'mech' using 'key'. 'tmpl' is created by * a previous call to crypto_create_ctx_template(). - * cr: crypto_call_req_t calling conditions and call back info. * * Description: * Asynchronously submits a request for, or synchronously performs a @@ -79,55 +75,17 @@ * the specified session id. * When complete and successful, 'mac' will contain the message * authentication code. - * - * Context: - * Process or interrupt, according to the semantics dictated by the 'crq'. + * Relies on the KCF scheduler to choose a provider. * * Returns: * See comment in the beginning of the file. */ int -crypto_mac_prov(crypto_provider_t provider, crypto_session_id_t sid, - crypto_mechanism_t *mech, crypto_data_t *data, crypto_key_t *key, - crypto_ctx_template_t tmpl, crypto_data_t *mac, crypto_call_req_t *crq) -{ - kcf_req_params_t params; - kcf_provider_desc_t *pd = provider; - kcf_provider_desc_t *real_provider = pd; - int rv; - - ASSERT(KCF_PROV_REFHELD(pd)); - - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) { - rv = kcf_get_hardware_provider(mech->cm_type, - CRYPTO_MECH_INVALID, CHECK_RESTRICT(crq), pd, - &real_provider, CRYPTO_FG_MAC_ATOMIC); - - if (rv != CRYPTO_SUCCESS) - return (rv); - } - - KCF_WRAP_MAC_OPS_PARAMS(¶ms, KCF_OP_ATOMIC, sid, mech, key, - data, mac, tmpl); - rv = kcf_submit_request(real_provider, NULL, crq, ¶ms, B_FALSE); - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) - KCF_PROV_REFRELE(real_provider); - - return (rv); -} - -/* - * Same as crypto_mac_prov(), but relies on the KCF scheduler to choose - * a provider. See crypto_mac() comments for more information. - */ -int crypto_mac(crypto_mechanism_t *mech, crypto_data_t *data, - crypto_key_t *key, crypto_ctx_template_t tmpl, crypto_data_t *mac, - crypto_call_req_t *crq) + crypto_key_t *key, crypto_ctx_template_t tmpl, crypto_data_t *mac) { int error; kcf_mech_entry_t *me; - kcf_req_params_t params; kcf_provider_desc_t *pd; kcf_ctx_template_t *ctx_tmpl; crypto_spi_ctx_template_t spi_ctx_tmpl = NULL; @@ -136,187 +94,23 @@ crypto_mac(crypto_mechanism_t *mech, crypto_data_t *data, retry: /* The pd is returned held */ if ((pd = kcf_get_mech_provider(mech->cm_type, &me, &error, - list, CRYPTO_FG_MAC_ATOMIC, CHECK_RESTRICT(crq), - data->cd_length)) == NULL) { + list, CRYPTO_FG_MAC_ATOMIC)) == NULL) { if (list != NULL) kcf_free_triedlist(list); return (error); } - /* - * For SW providers, check the validity of the context template - * It is very rare that the generation number mis-matches, so - * is acceptable to fail here, and let the consumer recover by - * freeing this tmpl and create a new one for the key and new SW - * provider - */ - if ((pd->pd_prov_type == CRYPTO_SW_PROVIDER) && - ((ctx_tmpl = (kcf_ctx_template_t *)tmpl) != NULL)) { - if (ctx_tmpl->ct_generation != me->me_gen_swprov) { - if (list != NULL) - kcf_free_triedlist(list); - KCF_PROV_REFRELE(pd); - return (CRYPTO_OLD_CTX_TEMPLATE); - } else { - spi_ctx_tmpl = ctx_tmpl->ct_prov_tmpl; - } - } - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(crq, pd)) { - crypto_mechanism_t lmech; - - lmech = *mech; - KCF_SET_PROVIDER_MECHNUM(mech->cm_type, pd, &lmech); - - error = KCF_PROV_MAC_ATOMIC(pd, pd->pd_sid, &lmech, key, data, - mac, spi_ctx_tmpl, KCF_SWFP_RHNDL(crq)); - KCF_PROV_INCRSTATS(pd, error); - } else { - if (pd->pd_prov_type == CRYPTO_HW_PROVIDER && - (pd->pd_flags & CRYPTO_HASH_NO_UPDATE) && - (data->cd_length > pd->pd_hash_limit)) { - /* - * XXX - We need a check to see if this is indeed - * a HMAC. So far, all kernel clients use - * this interface only for HMAC. So, this is fine - * for now. - */ - error = CRYPTO_BUFFER_TOO_BIG; - } else { - KCF_WRAP_MAC_OPS_PARAMS(¶ms, KCF_OP_ATOMIC, - pd->pd_sid, mech, key, data, mac, spi_ctx_tmpl); + if (((ctx_tmpl = (kcf_ctx_template_t *)tmpl) != NULL)) + spi_ctx_tmpl = ctx_tmpl->ct_prov_tmpl; - error = kcf_submit_request(pd, NULL, crq, ¶ms, - KCF_ISDUALREQ(crq)); - } - } + crypto_mechanism_t lmech = *mech; + KCF_SET_PROVIDER_MECHNUM(mech->cm_type, pd, &lmech); + error = KCF_PROV_MAC_ATOMIC(pd, &lmech, key, data, + mac, spi_ctx_tmpl); - if (error != CRYPTO_SUCCESS && error != CRYPTO_QUEUED && - IS_RECOVERABLE(error)) { + if (error != CRYPTO_SUCCESS && IS_RECOVERABLE(error)) { /* Add pd to the linked list of providers tried. */ - if (kcf_insert_triedlist(&list, pd, KCF_KMFLAG(crq)) != NULL) - goto retry; - } - - if (list != NULL) - kcf_free_triedlist(list); - - KCF_PROV_REFRELE(pd); - return (error); -} - -/* - * Single part operation to compute the MAC corresponding to the specified - * 'data' and to verify that it matches the MAC specified by 'mac'. - * The other arguments are the same as the function crypto_mac_prov(). - */ -int -crypto_mac_verify_prov(crypto_provider_t provider, crypto_session_id_t sid, - crypto_mechanism_t *mech, crypto_data_t *data, crypto_key_t *key, - crypto_ctx_template_t tmpl, crypto_data_t *mac, crypto_call_req_t *crq) -{ - kcf_req_params_t params; - kcf_provider_desc_t *pd = provider; - kcf_provider_desc_t *real_provider = pd; - int rv; - - ASSERT(KCF_PROV_REFHELD(pd)); - - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) { - rv = kcf_get_hardware_provider(mech->cm_type, - CRYPTO_MECH_INVALID, CHECK_RESTRICT(crq), pd, - &real_provider, CRYPTO_FG_MAC_ATOMIC); - - if (rv != CRYPTO_SUCCESS) - return (rv); - } - - KCF_WRAP_MAC_OPS_PARAMS(¶ms, KCF_OP_MAC_VERIFY_ATOMIC, sid, mech, - key, data, mac, tmpl); - rv = kcf_submit_request(real_provider, NULL, crq, ¶ms, B_FALSE); - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) - KCF_PROV_REFRELE(real_provider); - - return (rv); -} - -/* - * Same as crypto_mac_verify_prov(), but relies on the KCF scheduler to choose - * a provider. See crypto_mac_verify_prov() comments for more information. - */ -int -crypto_mac_verify(crypto_mechanism_t *mech, crypto_data_t *data, - crypto_key_t *key, crypto_ctx_template_t tmpl, crypto_data_t *mac, - crypto_call_req_t *crq) -{ - int error; - kcf_mech_entry_t *me; - kcf_req_params_t params; - kcf_provider_desc_t *pd; - kcf_ctx_template_t *ctx_tmpl; - crypto_spi_ctx_template_t spi_ctx_tmpl = NULL; - kcf_prov_tried_t *list = NULL; - -retry: - /* The pd is returned held */ - if ((pd = kcf_get_mech_provider(mech->cm_type, &me, &error, - list, CRYPTO_FG_MAC_ATOMIC, CHECK_RESTRICT(crq), - data->cd_length)) == NULL) { - if (list != NULL) - kcf_free_triedlist(list); - return (error); - } - - /* - * For SW providers, check the validity of the context template - * It is very rare that the generation number mis-matches, so - * is acceptable to fail here, and let the consumer recover by - * freeing this tmpl and create a new one for the key and new SW - * provider - */ - if ((pd->pd_prov_type == CRYPTO_SW_PROVIDER) && - ((ctx_tmpl = (kcf_ctx_template_t *)tmpl) != NULL)) { - if (ctx_tmpl->ct_generation != me->me_gen_swprov) { - if (list != NULL) - kcf_free_triedlist(list); - KCF_PROV_REFRELE(pd); - return (CRYPTO_OLD_CTX_TEMPLATE); - } else { - spi_ctx_tmpl = ctx_tmpl->ct_prov_tmpl; - } - } - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(crq, pd)) { - crypto_mechanism_t lmech; - - lmech = *mech; - KCF_SET_PROVIDER_MECHNUM(mech->cm_type, pd, &lmech); - - error = KCF_PROV_MAC_VERIFY_ATOMIC(pd, pd->pd_sid, &lmech, key, - data, mac, spi_ctx_tmpl, KCF_SWFP_RHNDL(crq)); - KCF_PROV_INCRSTATS(pd, error); - } else { - if (pd->pd_prov_type == CRYPTO_HW_PROVIDER && - (pd->pd_flags & CRYPTO_HASH_NO_UPDATE) && - (data->cd_length > pd->pd_hash_limit)) { - /* see comments in crypto_mac() */ - error = CRYPTO_BUFFER_TOO_BIG; - } else { - KCF_WRAP_MAC_OPS_PARAMS(¶ms, - KCF_OP_MAC_VERIFY_ATOMIC, pd->pd_sid, mech, - key, data, mac, spi_ctx_tmpl); - - error = kcf_submit_request(pd, NULL, crq, ¶ms, - KCF_ISDUALREQ(crq)); - } - } - - if (error != CRYPTO_SUCCESS && error != CRYPTO_QUEUED && - IS_RECOVERABLE(error)) { - /* Add pd to the linked list of providers tried. */ - if (kcf_insert_triedlist(&list, pd, KCF_KMFLAG(crq)) != NULL) + if (kcf_insert_triedlist(&list, pd, KM_SLEEP) != NULL) goto retry; } @@ -333,7 +127,6 @@ crypto_mac_verify(crypto_mechanism_t *mech, crypto_data_t *data, * Arguments: * pd: pointer to the descriptor of the provider to use for this * operation. - * sid: provider session id. * mech: crypto_mechanism_t pointer. * mech_type is a valid value previously returned by * crypto_mech2id(); @@ -344,7 +137,6 @@ crypto_mac_verify(crypto_mechanism_t *mech, crypto_data_t *data, * MAC with the 'mech' using 'key'. 'tmpl' is created by * a previous call to crypto_create_ctx_template(). * ctxp: Pointer to a crypto_context_t. - * cr: crypto_call_req_t calling conditions and call back info. * * Description: * Asynchronously submits a request for, or synchronously performs the @@ -357,61 +149,29 @@ crypto_mac_verify(crypto_mechanism_t *mech, crypto_data_t *data, * The caller should hold a reference on the specified provider * descriptor before calling this function. * - * Context: - * Process or interrupt, according to the semantics dictated by the 'cr'. - * * Returns: * See comment in the beginning of the file. */ -int -crypto_mac_init_prov(crypto_provider_t provider, crypto_session_id_t sid, +static int +crypto_mac_init_prov(kcf_provider_desc_t *pd, crypto_mechanism_t *mech, crypto_key_t *key, crypto_spi_ctx_template_t tmpl, - crypto_context_t *ctxp, crypto_call_req_t *crq) + crypto_context_t *ctxp) { int rv; crypto_ctx_t *ctx; - kcf_req_params_t params; - kcf_provider_desc_t *pd = provider; kcf_provider_desc_t *real_provider = pd; ASSERT(KCF_PROV_REFHELD(pd)); - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) { - rv = kcf_get_hardware_provider(mech->cm_type, - CRYPTO_MECH_INVALID, CHECK_RESTRICT(crq), pd, - &real_provider, CRYPTO_FG_MAC); - - if (rv != CRYPTO_SUCCESS) - return (rv); - } - /* Allocate and initialize the canonical context */ - if ((ctx = kcf_new_ctx(crq, real_provider, sid)) == NULL) { - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) - KCF_PROV_REFRELE(real_provider); + if ((ctx = kcf_new_ctx(real_provider)) == NULL) return (CRYPTO_HOST_MEMORY); - } - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(crq, pd)) { - crypto_mechanism_t lmech; + crypto_mechanism_t lmech = *mech; + KCF_SET_PROVIDER_MECHNUM(mech->cm_type, real_provider, &lmech); + rv = KCF_PROV_MAC_INIT(real_provider, ctx, &lmech, key, tmpl); - lmech = *mech; - KCF_SET_PROVIDER_MECHNUM(mech->cm_type, real_provider, &lmech); - rv = KCF_PROV_MAC_INIT(real_provider, ctx, &lmech, key, tmpl, - KCF_SWFP_RHNDL(crq)); - KCF_PROV_INCRSTATS(pd, rv); - } else { - KCF_WRAP_MAC_OPS_PARAMS(¶ms, KCF_OP_INIT, sid, mech, key, - NULL, NULL, tmpl); - rv = kcf_submit_request(real_provider, ctx, crq, ¶ms, - B_FALSE); - } - - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) - KCF_PROV_REFRELE(real_provider); - - if ((rv == CRYPTO_SUCCESS) || (rv == CRYPTO_QUEUED)) + if (rv == CRYPTO_SUCCESS) *ctxp = (crypto_context_t)ctx; else { /* Release the hold done in kcf_new_ctx(). */ @@ -428,8 +188,7 @@ crypto_mac_init_prov(crypto_provider_t provider, crypto_session_id_t sid, */ int crypto_mac_init(crypto_mechanism_t *mech, crypto_key_t *key, - crypto_ctx_template_t tmpl, crypto_context_t *ctxp, - crypto_call_req_t *crq) + crypto_ctx_template_t tmpl, crypto_context_t *ctxp) { int error; kcf_mech_entry_t *me; @@ -441,51 +200,27 @@ crypto_mac_init(crypto_mechanism_t *mech, crypto_key_t *key, retry: /* The pd is returned held */ if ((pd = kcf_get_mech_provider(mech->cm_type, &me, &error, - list, CRYPTO_FG_MAC, CHECK_RESTRICT(crq), 0)) == NULL) { + list, CRYPTO_FG_MAC)) == NULL) { if (list != NULL) kcf_free_triedlist(list); return (error); } /* - * For SW providers, check the validity of the context template + * Check the validity of the context template * It is very rare that the generation number mis-matches, so * is acceptable to fail here, and let the consumer recover by - * freeing this tmpl and create a new one for the key and new SW - * provider + * freeing this tmpl and create a new one for the key and new provider */ - if ((pd->pd_prov_type == CRYPTO_SW_PROVIDER) && - ((ctx_tmpl = (kcf_ctx_template_t *)tmpl) != NULL)) { - if (ctx_tmpl->ct_generation != me->me_gen_swprov) { - if (list != NULL) - kcf_free_triedlist(list); - KCF_PROV_REFRELE(pd); - return (CRYPTO_OLD_CTX_TEMPLATE); - } else { - spi_ctx_tmpl = ctx_tmpl->ct_prov_tmpl; - } - } + if (((ctx_tmpl = (kcf_ctx_template_t *)tmpl) != NULL)) + spi_ctx_tmpl = ctx_tmpl->ct_prov_tmpl; - if (pd->pd_prov_type == CRYPTO_HW_PROVIDER && - (pd->pd_flags & CRYPTO_HASH_NO_UPDATE)) { - /* - * The hardware provider has limited HMAC support. - * So, we fallback early here to using a software provider. - * - * XXX - need to enhance to do the fallback later in - * crypto_mac_update() if the size of accumulated input data - * exceeds the maximum size digestable by hardware provider. - */ - error = CRYPTO_BUFFER_TOO_BIG; - } else { - error = crypto_mac_init_prov(pd, pd->pd_sid, mech, key, - spi_ctx_tmpl, ctxp, crq); - } - if (error != CRYPTO_SUCCESS && error != CRYPTO_QUEUED && - IS_RECOVERABLE(error)) { + error = crypto_mac_init_prov(pd, mech, key, + spi_ctx_tmpl, ctxp); + if (error != CRYPTO_SUCCESS && IS_RECOVERABLE(error)) { /* Add pd to the linked list of providers tried. */ - if (kcf_insert_triedlist(&list, pd, KCF_KMFLAG(crq)) != NULL) + if (kcf_insert_triedlist(&list, pd, KM_SLEEP) != NULL) goto retry; } @@ -502,27 +237,19 @@ crypto_mac_init(crypto_mechanism_t *mech, crypto_key_t *key, * Arguments: * context: A crypto_context_t initialized by mac_init(). * data: The message part to be MAC'ed - * cr: crypto_call_req_t calling conditions and call back info. * * Description: - * Asynchronously submits a request for, or synchronously performs a - * part of a MAC operation. - * - * Context: - * Process or interrupt, according to the semantics dictated by the 'cr'. + * Synchronously performs a part of a MAC operation. * * Returns: * See comment in the beginning of the file. */ int -crypto_mac_update(crypto_context_t context, crypto_data_t *data, - crypto_call_req_t *cr) +crypto_mac_update(crypto_context_t context, crypto_data_t *data) { crypto_ctx_t *ctx = (crypto_ctx_t *)context; kcf_context_t *kcf_ctx; kcf_provider_desc_t *pd; - kcf_req_params_t params; - int rv; if ((ctx == NULL) || ((kcf_ctx = (kcf_context_t *)ctx->cc_framework_private) == NULL) || @@ -530,19 +257,7 @@ crypto_mac_update(crypto_context_t context, crypto_data_t *data, return (CRYPTO_INVALID_CONTEXT); } - ASSERT(pd->pd_prov_type != CRYPTO_LOGICAL_PROVIDER); - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(cr, pd)) { - rv = KCF_PROV_MAC_UPDATE(pd, ctx, data, NULL); - KCF_PROV_INCRSTATS(pd, rv); - } else { - KCF_WRAP_MAC_OPS_PARAMS(¶ms, KCF_OP_UPDATE, - ctx->cc_session, NULL, NULL, data, NULL, NULL); - rv = kcf_submit_request(pd, ctx, cr, ¶ms, B_FALSE); - } - - return (rv); + return (KCF_PROV_MAC_UPDATE(pd, ctx, data)); } /* @@ -551,27 +266,19 @@ crypto_mac_update(crypto_context_t context, crypto_data_t *data, * Arguments: * context: A crypto_context_t initialized by mac_init(). * mac: Storage for the message authentication code. - * cr: crypto_call_req_t calling conditions and call back info. * * Description: - * Asynchronously submits a request for, or synchronously performs a - * part of a message authentication operation. - * - * Context: - * Process or interrupt, according to the semantics dictated by the 'cr'. + * Synchronously performs a part of a message authentication operation. * * Returns: * See comment in the beginning of the file. */ int -crypto_mac_final(crypto_context_t context, crypto_data_t *mac, - crypto_call_req_t *cr) +crypto_mac_final(crypto_context_t context, crypto_data_t *mac) { crypto_ctx_t *ctx = (crypto_ctx_t *)context; kcf_context_t *kcf_ctx; kcf_provider_desc_t *pd; - kcf_req_params_t params; - int rv; if ((ctx == NULL) || ((kcf_ctx = (kcf_context_t *)ctx->cc_framework_private) == NULL) || @@ -579,67 +286,16 @@ crypto_mac_final(crypto_context_t context, crypto_data_t *mac, return (CRYPTO_INVALID_CONTEXT); } - ASSERT(pd->pd_prov_type != CRYPTO_LOGICAL_PROVIDER); - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(cr, pd)) { - rv = KCF_PROV_MAC_FINAL(pd, ctx, mac, NULL); - KCF_PROV_INCRSTATS(pd, rv); - } else { - KCF_WRAP_MAC_OPS_PARAMS(¶ms, KCF_OP_FINAL, - ctx->cc_session, NULL, NULL, NULL, mac, NULL); - rv = kcf_submit_request(pd, ctx, cr, ¶ms, B_FALSE); - } + int rv = KCF_PROV_MAC_FINAL(pd, ctx, mac); /* Release the hold done in kcf_new_ctx() during init step. */ KCF_CONTEXT_COND_RELEASE(rv, kcf_ctx); return (rv); } -/* - * See comments for crypto_mac_update() and crypto_mac_final(). - */ -int -crypto_mac_single(crypto_context_t context, crypto_data_t *data, - crypto_data_t *mac, crypto_call_req_t *cr) -{ - crypto_ctx_t *ctx = (crypto_ctx_t *)context; - kcf_context_t *kcf_ctx; - kcf_provider_desc_t *pd; - int error; - kcf_req_params_t params; - - - if ((ctx == NULL) || - ((kcf_ctx = (kcf_context_t *)ctx->cc_framework_private) == NULL) || - ((pd = kcf_ctx->kc_prov_desc) == NULL)) { - return (CRYPTO_INVALID_CONTEXT); - } - - - /* The fast path for SW providers. */ - if (CHECK_FASTPATH(cr, pd)) { - error = KCF_PROV_MAC(pd, ctx, data, mac, NULL); - KCF_PROV_INCRSTATS(pd, error); - } else { - KCF_WRAP_MAC_OPS_PARAMS(¶ms, KCF_OP_SINGLE, pd->pd_sid, - NULL, NULL, data, mac, NULL); - error = kcf_submit_request(pd, ctx, cr, ¶ms, B_FALSE); - } - - /* Release the hold done in kcf_new_ctx() during init step. */ - KCF_CONTEXT_COND_RELEASE(error, kcf_ctx); - return (error); -} - #if defined(_KERNEL) -EXPORT_SYMBOL(crypto_mac_prov); EXPORT_SYMBOL(crypto_mac); -EXPORT_SYMBOL(crypto_mac_verify_prov); -EXPORT_SYMBOL(crypto_mac_verify); -EXPORT_SYMBOL(crypto_mac_init_prov); EXPORT_SYMBOL(crypto_mac_init); EXPORT_SYMBOL(crypto_mac_update); EXPORT_SYMBOL(crypto_mac_final); -EXPORT_SYMBOL(crypto_mac_single); #endif diff --git a/module/icp/api/kcf_miscapi.c b/module/icp/api/kcf_miscapi.c deleted file mode 100644 index 5c0d60391f44..000000000000 --- a/module/icp/api/kcf_miscapi.c +++ /dev/null @@ -1,127 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License (the "License"). - * You may not use this file except in compliance with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2008 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#include -#include -#include -#include -#include - -/* - * All event subscribers are put on a list. kcf_notify_list_lock - * protects changes to this list. - * - * The following locking order is maintained in the code - The - * global kcf_notify_list_lock followed by the individual lock - * in a kcf_ntfy_elem structure (kn_lock). - */ -kmutex_t ntfy_list_lock; -kcondvar_t ntfy_list_cv; /* cv the service thread waits on */ -static kcf_ntfy_elem_t *ntfy_list_head; - -/* - * crypto_mech2id() - * - * Arguments: - * . mechname: A null-terminated string identifying the mechanism name. - * - * Description: - * Walks the mechanisms tables, looking for an entry that matches the - * mechname. Once it find it, it builds the 64-bit mech_type and returns - * it. If there are no hardware or software providers for the mechanism, - * but there is an unloaded software provider, this routine will attempt - * to load it. - * - * Context: - * Process and interruption. - * - * Returns: - * The unique mechanism identified by 'mechname', if found. - * CRYPTO_MECH_INVALID otherwise. - */ -crypto_mech_type_t -crypto_mech2id(const char *mechname) -{ - return (crypto_mech2id_common(mechname, B_TRUE)); -} - -/* - * We walk the notification list and do the callbacks. - */ -void -kcf_walk_ntfylist(uint32_t event, void *event_arg) -{ - kcf_ntfy_elem_t *nep; - int nelem = 0; - - mutex_enter(&ntfy_list_lock); - - /* - * Count how many clients are on the notification list. We need - * this count to ensure that clients which joined the list after we - * have started this walk, are not wrongly notified. - */ - for (nep = ntfy_list_head; nep != NULL; nep = nep->kn_next) - nelem++; - - for (nep = ntfy_list_head; (nep != NULL && nelem); nep = nep->kn_next) { - nelem--; - - /* - * Check if this client is interested in the - * event. - */ - if (!(nep->kn_event_mask & event)) - continue; - - mutex_enter(&nep->kn_lock); - nep->kn_state = NTFY_RUNNING; - mutex_exit(&nep->kn_lock); - mutex_exit(&ntfy_list_lock); - - /* - * We invoke the callback routine with no locks held. Another - * client could have joined the list meanwhile. This is fine - * as we maintain nelem as stated above. The NULL check in the - * for loop guards against shrinkage. Also, any callers of - * crypto_unnotify_events() at this point cv_wait till kn_state - * changes to NTFY_WAITING. Hence, nep is assured to be valid. - */ - (*nep->kn_func)(event, event_arg); - - mutex_enter(&nep->kn_lock); - nep->kn_state = NTFY_WAITING; - cv_broadcast(&nep->kn_cv); - mutex_exit(&nep->kn_lock); - - mutex_enter(&ntfy_list_lock); - } - - mutex_exit(&ntfy_list_lock); -} - -#if defined(_KERNEL) -EXPORT_SYMBOL(crypto_mech2id); -#endif diff --git a/module/icp/asm-aarch64/blake3/b3_aarch64_sse2.S b/module/icp/asm-aarch64/blake3/b3_aarch64_sse2.S new file mode 100644 index 000000000000..8237f0eb5a4e --- /dev/null +++ b/module/icp/asm-aarch64/blake3/b3_aarch64_sse2.S @@ -0,0 +1,2450 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Based on BLAKE3 v1.3.1, https://github.com/BLAKE3-team/BLAKE3 + * Copyright (c) 2019-2022 Samuel Neves and Matthew Krupcale + * Copyright (c) 2022 Tino Reichardt + * + * This is converted assembly: SSE2 -> ARMv8-A + * Used tools: SIMDe https://github.com/simd-everywhere/simde + */ + +#if defined(__aarch64__) + .text + .section .rodata.cst16,"aM",@progbits,16 + .p2align 4 +.LCPI0_0: + .word 1779033703 + .word 3144134277 + .word 1013904242 + .word 2773480762 +.LCPI0_1: + .xword 0 + .xword -4294967296 +.LCPI0_2: + .xword -1 + .xword 4294967295 + .text + .globl zfs_blake3_compress_in_place_sse2 + .p2align 2 + .type zfs_blake3_compress_in_place_sse2,@function +zfs_blake3_compress_in_place_sse2: + .cfi_startproc + ldp q3, q2, [x0] + ldp q5, q6, [x1] + add x10, x1, #32 + lsr x11, x3, #32 + fmov s4, w3 + ld2 { v17.4s, v18.4s }, [x10] + adrp x10, .LCPI0_2 + and w8, w2, #0xff + mov v4.s[1], w11 + ldr q1, [x10, :lo12:.LCPI0_2] + and w9, w4, #0xff + adrp x12, .LCPI0_0 + mov v4.s[2], w8 + uzp1 v19.4s, v5.4s, v6.4s + add v3.4s, v2.4s, v3.4s + ldr q7, [x12, :lo12:.LCPI0_0] + mov v4.s[3], w9 + add v3.4s, v3.4s, v19.4s + uzp2 v5.4s, v5.4s, v6.4s + ext v21.16b, v18.16b, v18.16b, #12 + uzp1 v6.4s, v19.4s, v19.4s + ext v22.16b, v19.16b, v19.16b, #12 + eor v4.16b, v3.16b, v4.16b + ext v20.16b, v17.16b, v17.16b, #12 + ext v6.16b, v6.16b, v19.16b, #8 + ext v19.16b, v19.16b, v22.16b, #12 + zip1 v22.2d, v21.2d, v5.2d + rev32 v24.8h, v4.8h + mov v4.16b, v1.16b + zip2 v23.4s, v5.4s, v21.4s + uzp2 v6.4s, v6.4s, v5.4s + bsl v4.16b, v22.16b, v20.16b + add v3.4s, v3.4s, v5.4s + zip1 v5.4s, v23.4s, v20.4s + zip1 v22.4s, v20.4s, v23.4s + add v23.4s, v24.4s, v7.4s + ext v7.16b, v6.16b, v6.16b, #4 + ext v25.16b, v4.16b, v4.16b, #12 + ext v5.16b, v22.16b, v5.16b, #8 + eor v2.16b, v23.16b, v2.16b + uzp1 v4.4s, v4.4s, v25.4s + uzp1 v22.4s, v7.4s, v7.4s + ext v25.16b, v7.16b, v7.16b, #12 + ext v22.16b, v22.16b, v7.16b, #8 + ext v7.16b, v7.16b, v25.16b, #12 + ushr v25.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + orr v2.16b, v2.16b, v25.16b + add v3.4s, v3.4s, v2.4s + eor v24.16b, v3.16b, v24.16b + add v3.4s, v3.4s, v17.4s + ushr v17.4s, v24.4s, #8 + shl v18.4s, v24.4s, #24 + orr v17.16b, v18.16b, v17.16b + add v18.4s, v17.4s, v23.4s + eor v2.16b, v18.16b, v2.16b + ushr v23.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + ext v3.16b, v3.16b, v3.16b, #12 + orr v2.16b, v2.16b, v23.16b + ext v17.16b, v17.16b, v17.16b, #8 + add v3.4s, v2.4s, v3.4s + adrp x11, .LCPI0_1 + eor v17.16b, v3.16b, v17.16b + ldr q16, [x11, :lo12:.LCPI0_1] + ext v18.16b, v18.16b, v18.16b, #4 + rev32 v24.8h, v17.8h + movi v0.2d, #0xffffffff00000000 + add v23.4s, v3.4s, v21.4s + mov v21.s[1], v20.s[2] + add v20.4s, v18.4s, v24.4s + bit v19.16b, v21.16b, v0.16b + eor v3.16b, v20.16b, v2.16b + uzp2 v2.4s, v22.4s, v19.4s + zip1 v17.2d, v5.2d, v19.2d + zip2 v18.4s, v19.4s, v5.4s + ushr v21.4s, v3.4s, #12 + shl v3.4s, v3.4s, #20 + ext v22.16b, v2.16b, v2.16b, #4 + bsl v16.16b, v4.16b, v17.16b + zip1 v17.4s, v18.4s, v4.4s + zip1 v18.4s, v4.4s, v18.4s + orr v21.16b, v3.16b, v21.16b + ext v25.16b, v16.16b, v16.16b, #12 + ext v3.16b, v18.16b, v17.16b, #8 + uzp1 v18.4s, v22.4s, v22.4s + ext v26.16b, v22.16b, v22.16b, #12 + add v23.4s, v23.4s, v21.4s + uzp1 v17.4s, v16.4s, v25.4s + ext v16.16b, v18.16b, v22.16b, #8 + ext v18.16b, v22.16b, v26.16b, #12 + eor v22.16b, v23.16b, v24.16b + add v6.4s, v23.4s, v6.4s + ushr v23.4s, v22.4s, #8 + shl v22.4s, v22.4s, #24 + orr v22.16b, v22.16b, v23.16b + add v20.4s, v22.4s, v20.4s + eor v21.16b, v20.16b, v21.16b + ushr v23.4s, v21.4s, #7 + shl v21.4s, v21.4s, #25 + ext v6.16b, v6.16b, v6.16b, #4 + orr v21.16b, v21.16b, v23.16b + ext v22.16b, v22.16b, v22.16b, #8 + add v6.4s, v21.4s, v6.4s + eor v22.16b, v6.16b, v22.16b + ext v20.16b, v20.16b, v20.16b, #12 + add v6.4s, v6.4s, v19.4s + rev32 v19.8h, v22.8h + add v20.4s, v20.4s, v19.4s + eor v21.16b, v20.16b, v21.16b + ushr v22.4s, v21.4s, #12 + shl v21.4s, v21.4s, #20 + orr v21.16b, v21.16b, v22.16b + add v6.4s, v6.4s, v21.4s + eor v19.16b, v6.16b, v19.16b + ushr v22.4s, v19.4s, #8 + shl v19.4s, v19.4s, #24 + orr v19.16b, v19.16b, v22.16b + add v20.4s, v19.4s, v20.4s + eor v21.16b, v20.16b, v21.16b + ext v6.16b, v6.16b, v6.16b, #12 + ushr v22.4s, v21.4s, #7 + shl v21.4s, v21.4s, #25 + add v6.4s, v6.4s, v4.4s + orr v21.16b, v21.16b, v22.16b + ext v19.16b, v19.16b, v19.16b, #8 + add v6.4s, v6.4s, v21.4s + eor v19.16b, v6.16b, v19.16b + ext v20.16b, v20.16b, v20.16b, #4 + rev32 v19.8h, v19.8h + add v20.4s, v20.4s, v19.4s + add v6.4s, v6.4s, v5.4s + mov v5.s[1], v4.s[2] + eor v4.16b, v20.16b, v21.16b + ushr v21.4s, v4.4s, #12 + shl v4.4s, v4.4s, #20 + orr v21.16b, v4.16b, v21.16b + add v6.4s, v6.4s, v21.4s + eor v19.16b, v6.16b, v19.16b + add v2.4s, v6.4s, v2.4s + ushr v6.4s, v19.4s, #8 + shl v19.4s, v19.4s, #24 + orr v6.16b, v19.16b, v6.16b + add v19.4s, v6.4s, v20.4s + eor v20.16b, v19.16b, v21.16b + ushr v21.4s, v20.4s, #7 + shl v20.4s, v20.4s, #25 + ext v2.16b, v2.16b, v2.16b, #4 + orr v20.16b, v20.16b, v21.16b + ext v6.16b, v6.16b, v6.16b, #8 + add v2.4s, v20.4s, v2.4s + eor v6.16b, v2.16b, v6.16b + ext v19.16b, v19.16b, v19.16b, #12 + rev32 v6.8h, v6.8h + add v19.4s, v19.4s, v6.4s + mov v22.16b, v0.16b + eor v20.16b, v19.16b, v20.16b + bsl v22.16b, v5.16b, v7.16b + ushr v21.4s, v20.4s, #12 + shl v20.4s, v20.4s, #20 + add v2.4s, v2.4s, v22.4s + orr v20.16b, v20.16b, v21.16b + add v2.4s, v2.4s, v20.4s + eor v6.16b, v2.16b, v6.16b + ushr v21.4s, v6.4s, #8 + shl v6.4s, v6.4s, #24 + orr v6.16b, v6.16b, v21.16b + add v19.4s, v6.4s, v19.4s + eor v20.16b, v19.16b, v20.16b + ext v2.16b, v2.16b, v2.16b, #12 + ushr v21.4s, v20.4s, #7 + shl v20.4s, v20.4s, #25 + add v2.4s, v2.4s, v17.4s + orr v20.16b, v20.16b, v21.16b + ext v6.16b, v6.16b, v6.16b, #8 + add v2.4s, v2.4s, v20.4s + eor v6.16b, v2.16b, v6.16b + uzp2 v5.4s, v16.4s, v22.4s + zip1 v7.2d, v3.2d, v22.2d + zip2 v16.4s, v22.4s, v3.4s + ext v19.16b, v19.16b, v19.16b, #4 + rev32 v22.8h, v6.8h + ext v23.16b, v5.16b, v5.16b, #4 + bif v7.16b, v17.16b, v1.16b + zip1 v24.4s, v16.4s, v17.4s + zip1 v16.4s, v17.4s, v16.4s + add v21.4s, v2.4s, v3.4s + mov v3.s[1], v17.s[2] + add v17.4s, v19.4s, v22.4s + mov v19.16b, v0.16b + ext v25.16b, v7.16b, v7.16b, #12 + ext v4.16b, v16.16b, v24.16b, #8 + uzp1 v16.4s, v23.4s, v23.4s + bsl v19.16b, v3.16b, v18.16b + eor v2.16b, v17.16b, v20.16b + uzp1 v7.4s, v7.4s, v25.4s + ext v25.16b, v16.16b, v23.16b, #8 + zip1 v3.2d, v4.2d, v19.2d + ushr v20.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + ext v24.16b, v23.16b, v23.16b, #12 + uzp2 v6.4s, v25.4s, v19.4s + zip2 v18.4s, v19.4s, v4.4s + bif v3.16b, v7.16b, v1.16b + orr v20.16b, v2.16b, v20.16b + ext v16.16b, v23.16b, v24.16b, #12 + ext v23.16b, v6.16b, v6.16b, #4 + zip1 v24.4s, v18.4s, v7.4s + zip1 v18.4s, v7.4s, v18.4s + ext v25.16b, v3.16b, v3.16b, #12 + add v21.4s, v21.4s, v20.4s + ext v2.16b, v18.16b, v24.16b, #8 + uzp1 v18.4s, v23.4s, v23.4s + ext v24.16b, v23.16b, v23.16b, #12 + uzp1 v3.4s, v3.4s, v25.4s + eor v22.16b, v21.16b, v22.16b + ext v25.16b, v18.16b, v23.16b, #8 + dup v18.4s, v2.s[3] + ext v23.16b, v23.16b, v24.16b, #12 + add v5.4s, v21.4s, v5.4s + trn1 v21.4s, v3.4s, v3.4s + ushr v24.4s, v22.4s, #8 + shl v22.4s, v22.4s, #24 + ext v18.16b, v21.16b, v18.16b, #8 + orr v21.16b, v22.16b, v24.16b + add v17.4s, v21.4s, v17.4s + eor v20.16b, v17.16b, v20.16b + ushr v22.4s, v20.4s, #7 + shl v20.4s, v20.4s, #25 + ext v5.16b, v5.16b, v5.16b, #4 + orr v20.16b, v20.16b, v22.16b + ext v21.16b, v21.16b, v21.16b, #8 + add v5.4s, v20.4s, v5.4s + eor v21.16b, v5.16b, v21.16b + ext v17.16b, v17.16b, v17.16b, #12 + add v5.4s, v5.4s, v19.4s + rev32 v19.8h, v21.8h + add v17.4s, v17.4s, v19.4s + eor v20.16b, v17.16b, v20.16b + ushr v21.4s, v20.4s, #12 + shl v20.4s, v20.4s, #20 + orr v20.16b, v20.16b, v21.16b + add v5.4s, v5.4s, v20.4s + eor v19.16b, v5.16b, v19.16b + ushr v21.4s, v19.4s, #8 + shl v19.4s, v19.4s, #24 + orr v19.16b, v19.16b, v21.16b + add v17.4s, v19.4s, v17.4s + eor v20.16b, v17.16b, v20.16b + ext v5.16b, v5.16b, v5.16b, #12 + ushr v21.4s, v20.4s, #7 + shl v20.4s, v20.4s, #25 + add v5.4s, v5.4s, v7.4s + orr v20.16b, v20.16b, v21.16b + ext v19.16b, v19.16b, v19.16b, #8 + add v5.4s, v5.4s, v20.4s + eor v19.16b, v5.16b, v19.16b + ext v17.16b, v17.16b, v17.16b, #4 + rev32 v22.8h, v19.8h + add v21.4s, v5.4s, v4.4s + mov v4.s[1], v7.s[2] + add v19.4s, v17.4s, v22.4s + bit v16.16b, v4.16b, v0.16b + eor v5.16b, v19.16b, v20.16b + uzp2 v4.4s, v25.4s, v16.4s + zip1 v7.2d, v2.2d, v16.2d + zip2 v17.4s, v16.4s, v2.4s + ushr v20.4s, v5.4s, #12 + shl v5.4s, v5.4s, #20 + ext v24.16b, v4.16b, v4.16b, #4 + bif v7.16b, v3.16b, v1.16b + zip1 v25.4s, v17.4s, v3.4s + zip1 v17.4s, v3.4s, v17.4s + orr v20.16b, v5.16b, v20.16b + ext v26.16b, v7.16b, v7.16b, #12 + ext v5.16b, v17.16b, v25.16b, #8 + uzp1 v17.4s, v24.4s, v24.4s + ext v25.16b, v24.16b, v24.16b, #12 + bit v23.16b, v18.16b, v0.16b + add v21.4s, v21.4s, v20.4s + uzp1 v7.4s, v7.4s, v26.4s + ext v26.16b, v17.16b, v24.16b, #8 + ext v17.16b, v24.16b, v25.16b, #12 + eor v22.16b, v21.16b, v22.16b + add v6.4s, v21.4s, v6.4s + zip1 v21.2d, v5.2d, v23.2d + zip2 v24.4s, v23.4s, v5.4s + bif v21.16b, v7.16b, v1.16b + zip1 v1.4s, v24.4s, v7.4s + zip1 v24.4s, v7.4s, v24.4s + ext v1.16b, v24.16b, v1.16b, #8 + ushr v24.4s, v22.4s, #8 + shl v22.4s, v22.4s, #24 + orr v22.16b, v22.16b, v24.16b + add v19.4s, v22.4s, v19.4s + ext v24.16b, v21.16b, v21.16b, #12 + eor v20.16b, v19.16b, v20.16b + uzp1 v21.4s, v21.4s, v24.4s + ushr v24.4s, v20.4s, #7 + shl v20.4s, v20.4s, #25 + orr v20.16b, v20.16b, v24.16b + ext v6.16b, v6.16b, v6.16b, #4 + ext v22.16b, v22.16b, v22.16b, #8 + add v6.4s, v20.4s, v6.4s + eor v22.16b, v6.16b, v22.16b + ext v19.16b, v19.16b, v19.16b, #12 + add v6.4s, v6.4s, v16.4s + rev32 v16.8h, v22.8h + add v19.4s, v19.4s, v16.4s + eor v20.16b, v19.16b, v20.16b + ushr v22.4s, v20.4s, #12 + shl v20.4s, v20.4s, #20 + orr v20.16b, v20.16b, v22.16b + add v6.4s, v6.4s, v20.4s + eor v16.16b, v6.16b, v16.16b + ext v6.16b, v6.16b, v6.16b, #12 + add v3.4s, v6.4s, v3.4s + ushr v6.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + orr v6.16b, v16.16b, v6.16b + add v16.4s, v6.4s, v19.4s + eor v19.16b, v16.16b, v20.16b + ushr v20.4s, v19.4s, #7 + shl v19.4s, v19.4s, #25 + orr v19.16b, v19.16b, v20.16b + ext v6.16b, v6.16b, v6.16b, #8 + add v3.4s, v3.4s, v19.4s + eor v6.16b, v3.16b, v6.16b + ext v16.16b, v16.16b, v16.16b, #4 + add v2.4s, v3.4s, v2.4s + rev32 v3.8h, v6.8h + add v6.4s, v16.4s, v3.4s + eor v16.16b, v6.16b, v19.16b + ushr v19.4s, v16.4s, #12 + shl v16.4s, v16.4s, #20 + orr v16.16b, v16.16b, v19.16b + add v2.4s, v2.4s, v16.4s + eor v3.16b, v2.16b, v3.16b + add v2.4s, v2.4s, v4.4s + ushr v4.4s, v3.4s, #8 + shl v3.4s, v3.4s, #24 + orr v3.16b, v3.16b, v4.16b + add v4.4s, v3.4s, v6.4s + eor v6.16b, v4.16b, v16.16b + ushr v16.4s, v6.4s, #7 + shl v6.4s, v6.4s, #25 + ext v2.16b, v2.16b, v2.16b, #4 + orr v6.16b, v6.16b, v16.16b + ext v3.16b, v3.16b, v3.16b, #8 + add v2.4s, v6.4s, v2.4s + eor v3.16b, v2.16b, v3.16b + ext v4.16b, v4.16b, v4.16b, #12 + rev32 v3.8h, v3.8h + add v4.4s, v4.4s, v3.4s + eor v6.16b, v4.16b, v6.16b + ushr v16.4s, v6.4s, #12 + shl v6.4s, v6.4s, #20 + add v2.4s, v2.4s, v23.4s + orr v6.16b, v6.16b, v16.16b + add v2.4s, v2.4s, v6.4s + eor v3.16b, v2.16b, v3.16b + ushr v16.4s, v3.4s, #8 + shl v3.4s, v3.4s, #24 + orr v3.16b, v3.16b, v16.16b + add v4.4s, v3.4s, v4.4s + eor v6.16b, v4.16b, v6.16b + ext v2.16b, v2.16b, v2.16b, #12 + ushr v16.4s, v6.4s, #7 + shl v6.4s, v6.4s, #25 + add v2.4s, v2.4s, v7.4s + orr v6.16b, v6.16b, v16.16b + ext v3.16b, v3.16b, v3.16b, #8 + add v2.4s, v2.4s, v6.4s + eor v3.16b, v2.16b, v3.16b + ext v4.16b, v4.16b, v4.16b, #4 + rev32 v3.8h, v3.8h + add v2.4s, v2.4s, v5.4s + mov v5.s[1], v7.s[2] + add v4.4s, v4.4s, v3.4s + bsl v0.16b, v5.16b, v17.16b + eor v5.16b, v4.16b, v6.16b + ushr v6.4s, v5.4s, #12 + shl v5.4s, v5.4s, #20 + orr v5.16b, v5.16b, v6.16b + add v2.4s, v2.4s, v5.4s + eor v3.16b, v2.16b, v3.16b + ushr v6.4s, v3.4s, #8 + shl v3.4s, v3.4s, #24 + orr v3.16b, v3.16b, v6.16b + add v4.4s, v3.4s, v4.4s + uzp2 v18.4s, v26.4s, v18.4s + eor v5.16b, v4.16b, v5.16b + add v2.4s, v2.4s, v18.4s + ushr v6.4s, v5.4s, #7 + shl v5.4s, v5.4s, #25 + ext v2.16b, v2.16b, v2.16b, #4 + orr v5.16b, v5.16b, v6.16b + ext v3.16b, v3.16b, v3.16b, #8 + add v2.4s, v5.4s, v2.4s + eor v3.16b, v2.16b, v3.16b + ext v4.16b, v4.16b, v4.16b, #12 + add v0.4s, v2.4s, v0.4s + rev32 v2.8h, v3.8h + add v3.4s, v4.4s, v2.4s + eor v4.16b, v3.16b, v5.16b + ushr v5.4s, v4.4s, #12 + shl v4.4s, v4.4s, #20 + orr v4.16b, v4.16b, v5.16b + add v0.4s, v0.4s, v4.4s + eor v2.16b, v0.16b, v2.16b + ushr v5.4s, v2.4s, #8 + shl v2.4s, v2.4s, #24 + orr v2.16b, v2.16b, v5.16b + add v3.4s, v2.4s, v3.4s + eor v4.16b, v3.16b, v4.16b + ext v0.16b, v0.16b, v0.16b, #12 + ushr v5.4s, v4.4s, #7 + shl v4.4s, v4.4s, #25 + add v0.4s, v0.4s, v21.4s + orr v4.16b, v4.16b, v5.16b + ext v2.16b, v2.16b, v2.16b, #8 + add v0.4s, v0.4s, v4.4s + eor v2.16b, v0.16b, v2.16b + ext v3.16b, v3.16b, v3.16b, #4 + add v0.4s, v0.4s, v1.4s + rev32 v1.8h, v2.8h + add v2.4s, v3.4s, v1.4s + eor v3.16b, v2.16b, v4.16b + ushr v4.4s, v3.4s, #12 + shl v3.4s, v3.4s, #20 + orr v3.16b, v3.16b, v4.16b + add v0.4s, v0.4s, v3.4s + eor v1.16b, v0.16b, v1.16b + ushr v4.4s, v1.4s, #8 + shl v1.4s, v1.4s, #24 + orr v1.16b, v1.16b, v4.16b + add v2.4s, v1.4s, v2.4s + eor v3.16b, v2.16b, v3.16b + ext v0.16b, v0.16b, v0.16b, #4 + ext v2.16b, v2.16b, v2.16b, #12 + ushr v4.4s, v3.4s, #7 + shl v3.4s, v3.4s, #25 + ext v1.16b, v1.16b, v1.16b, #8 + eor v0.16b, v2.16b, v0.16b + orr v2.16b, v3.16b, v4.16b + eor v1.16b, v2.16b, v1.16b + stp q0, q1, [x0] + ret +.Lfunc_end0: + .size zfs_blake3_compress_in_place_sse2, .Lfunc_end0-zfs_blake3_compress_in_place_sse2 + .cfi_endproc + + .section .rodata.cst16,"aM",@progbits,16 + .p2align 4 +.LCPI1_0: + .word 1779033703 + .word 3144134277 + .word 1013904242 + .word 2773480762 +.LCPI1_1: + .xword 0 + .xword -4294967296 +.LCPI1_2: + .xword -1 + .xword 4294967295 + .text + .globl zfs_blake3_compress_xof_sse2 + .p2align 2 + .type zfs_blake3_compress_xof_sse2,@function +zfs_blake3_compress_xof_sse2: + .cfi_startproc + ldp q3, q2, [x0] + ldp q5, q6, [x1] + add x10, x1, #32 + lsr x11, x3, #32 + fmov s4, w3 + ld2 { v17.4s, v18.4s }, [x10] + adrp x10, .LCPI1_2 + and w8, w2, #0xff + mov v4.s[1], w11 + ldr q1, [x10, :lo12:.LCPI1_2] + and w9, w4, #0xff + adrp x12, .LCPI1_0 + mov v4.s[2], w8 + uzp1 v19.4s, v5.4s, v6.4s + add v3.4s, v2.4s, v3.4s + ldr q7, [x12, :lo12:.LCPI1_0] + mov v4.s[3], w9 + add v3.4s, v3.4s, v19.4s + uzp2 v5.4s, v5.4s, v6.4s + ext v21.16b, v18.16b, v18.16b, #12 + uzp1 v6.4s, v19.4s, v19.4s + ext v22.16b, v19.16b, v19.16b, #12 + eor v4.16b, v3.16b, v4.16b + ext v20.16b, v17.16b, v17.16b, #12 + ext v6.16b, v6.16b, v19.16b, #8 + ext v19.16b, v19.16b, v22.16b, #12 + zip1 v22.2d, v21.2d, v5.2d + rev32 v24.8h, v4.8h + mov v4.16b, v1.16b + zip2 v23.4s, v5.4s, v21.4s + uzp2 v6.4s, v6.4s, v5.4s + bsl v4.16b, v22.16b, v20.16b + add v3.4s, v3.4s, v5.4s + zip1 v5.4s, v23.4s, v20.4s + zip1 v22.4s, v20.4s, v23.4s + add v23.4s, v24.4s, v7.4s + ext v7.16b, v6.16b, v6.16b, #4 + ext v25.16b, v4.16b, v4.16b, #12 + ext v5.16b, v22.16b, v5.16b, #8 + eor v2.16b, v23.16b, v2.16b + uzp1 v4.4s, v4.4s, v25.4s + uzp1 v22.4s, v7.4s, v7.4s + ext v25.16b, v7.16b, v7.16b, #12 + ext v22.16b, v22.16b, v7.16b, #8 + ext v7.16b, v7.16b, v25.16b, #12 + ushr v25.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + orr v2.16b, v2.16b, v25.16b + add v3.4s, v3.4s, v2.4s + eor v24.16b, v3.16b, v24.16b + add v3.4s, v3.4s, v17.4s + ushr v17.4s, v24.4s, #8 + shl v18.4s, v24.4s, #24 + orr v17.16b, v18.16b, v17.16b + add v18.4s, v17.4s, v23.4s + eor v2.16b, v18.16b, v2.16b + ushr v23.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + ext v3.16b, v3.16b, v3.16b, #12 + orr v2.16b, v2.16b, v23.16b + ext v17.16b, v17.16b, v17.16b, #8 + add v3.4s, v2.4s, v3.4s + adrp x11, .LCPI1_1 + eor v17.16b, v3.16b, v17.16b + ldr q16, [x11, :lo12:.LCPI1_1] + ext v18.16b, v18.16b, v18.16b, #4 + rev32 v24.8h, v17.8h + movi v0.2d, #0xffffffff00000000 + add v23.4s, v3.4s, v21.4s + mov v21.s[1], v20.s[2] + add v20.4s, v18.4s, v24.4s + bit v19.16b, v21.16b, v0.16b + eor v3.16b, v20.16b, v2.16b + uzp2 v2.4s, v22.4s, v19.4s + zip1 v17.2d, v5.2d, v19.2d + zip2 v18.4s, v19.4s, v5.4s + ushr v21.4s, v3.4s, #12 + shl v3.4s, v3.4s, #20 + ext v22.16b, v2.16b, v2.16b, #4 + bsl v16.16b, v4.16b, v17.16b + zip1 v17.4s, v18.4s, v4.4s + zip1 v18.4s, v4.4s, v18.4s + orr v21.16b, v3.16b, v21.16b + ext v25.16b, v16.16b, v16.16b, #12 + ext v3.16b, v18.16b, v17.16b, #8 + uzp1 v18.4s, v22.4s, v22.4s + ext v26.16b, v22.16b, v22.16b, #12 + add v23.4s, v23.4s, v21.4s + uzp1 v17.4s, v16.4s, v25.4s + ext v16.16b, v18.16b, v22.16b, #8 + ext v18.16b, v22.16b, v26.16b, #12 + eor v22.16b, v23.16b, v24.16b + add v6.4s, v23.4s, v6.4s + ushr v23.4s, v22.4s, #8 + shl v22.4s, v22.4s, #24 + orr v22.16b, v22.16b, v23.16b + add v20.4s, v22.4s, v20.4s + eor v21.16b, v20.16b, v21.16b + ushr v23.4s, v21.4s, #7 + shl v21.4s, v21.4s, #25 + ext v6.16b, v6.16b, v6.16b, #4 + orr v21.16b, v21.16b, v23.16b + ext v22.16b, v22.16b, v22.16b, #8 + add v6.4s, v21.4s, v6.4s + eor v22.16b, v6.16b, v22.16b + ext v20.16b, v20.16b, v20.16b, #12 + add v6.4s, v6.4s, v19.4s + rev32 v19.8h, v22.8h + add v20.4s, v20.4s, v19.4s + eor v21.16b, v20.16b, v21.16b + ushr v22.4s, v21.4s, #12 + shl v21.4s, v21.4s, #20 + orr v21.16b, v21.16b, v22.16b + add v6.4s, v6.4s, v21.4s + eor v19.16b, v6.16b, v19.16b + ushr v22.4s, v19.4s, #8 + shl v19.4s, v19.4s, #24 + orr v19.16b, v19.16b, v22.16b + add v20.4s, v19.4s, v20.4s + eor v21.16b, v20.16b, v21.16b + ext v6.16b, v6.16b, v6.16b, #12 + ushr v22.4s, v21.4s, #7 + shl v21.4s, v21.4s, #25 + add v6.4s, v6.4s, v4.4s + orr v21.16b, v21.16b, v22.16b + ext v19.16b, v19.16b, v19.16b, #8 + add v6.4s, v6.4s, v21.4s + eor v19.16b, v6.16b, v19.16b + ext v20.16b, v20.16b, v20.16b, #4 + rev32 v19.8h, v19.8h + add v20.4s, v20.4s, v19.4s + add v6.4s, v6.4s, v5.4s + mov v5.s[1], v4.s[2] + eor v4.16b, v20.16b, v21.16b + ushr v21.4s, v4.4s, #12 + shl v4.4s, v4.4s, #20 + orr v21.16b, v4.16b, v21.16b + add v6.4s, v6.4s, v21.4s + eor v19.16b, v6.16b, v19.16b + add v2.4s, v6.4s, v2.4s + ushr v6.4s, v19.4s, #8 + shl v19.4s, v19.4s, #24 + orr v6.16b, v19.16b, v6.16b + add v19.4s, v6.4s, v20.4s + eor v20.16b, v19.16b, v21.16b + ushr v21.4s, v20.4s, #7 + shl v20.4s, v20.4s, #25 + ext v2.16b, v2.16b, v2.16b, #4 + orr v20.16b, v20.16b, v21.16b + ext v6.16b, v6.16b, v6.16b, #8 + add v2.4s, v20.4s, v2.4s + eor v6.16b, v2.16b, v6.16b + ext v19.16b, v19.16b, v19.16b, #12 + rev32 v6.8h, v6.8h + add v19.4s, v19.4s, v6.4s + mov v22.16b, v0.16b + eor v20.16b, v19.16b, v20.16b + bsl v22.16b, v5.16b, v7.16b + ushr v21.4s, v20.4s, #12 + shl v20.4s, v20.4s, #20 + add v2.4s, v2.4s, v22.4s + orr v20.16b, v20.16b, v21.16b + add v2.4s, v2.4s, v20.4s + eor v6.16b, v2.16b, v6.16b + ushr v21.4s, v6.4s, #8 + shl v6.4s, v6.4s, #24 + orr v6.16b, v6.16b, v21.16b + add v19.4s, v6.4s, v19.4s + eor v20.16b, v19.16b, v20.16b + ext v2.16b, v2.16b, v2.16b, #12 + ushr v21.4s, v20.4s, #7 + shl v20.4s, v20.4s, #25 + add v2.4s, v2.4s, v17.4s + orr v20.16b, v20.16b, v21.16b + ext v6.16b, v6.16b, v6.16b, #8 + add v2.4s, v2.4s, v20.4s + eor v6.16b, v2.16b, v6.16b + uzp2 v5.4s, v16.4s, v22.4s + zip1 v7.2d, v3.2d, v22.2d + zip2 v16.4s, v22.4s, v3.4s + ext v19.16b, v19.16b, v19.16b, #4 + rev32 v22.8h, v6.8h + ext v23.16b, v5.16b, v5.16b, #4 + bif v7.16b, v17.16b, v1.16b + zip1 v24.4s, v16.4s, v17.4s + zip1 v16.4s, v17.4s, v16.4s + add v21.4s, v2.4s, v3.4s + mov v3.s[1], v17.s[2] + add v17.4s, v19.4s, v22.4s + mov v19.16b, v0.16b + ext v25.16b, v7.16b, v7.16b, #12 + ext v4.16b, v16.16b, v24.16b, #8 + uzp1 v16.4s, v23.4s, v23.4s + bsl v19.16b, v3.16b, v18.16b + eor v2.16b, v17.16b, v20.16b + uzp1 v7.4s, v7.4s, v25.4s + ext v25.16b, v16.16b, v23.16b, #8 + zip1 v3.2d, v4.2d, v19.2d + ushr v20.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + ext v24.16b, v23.16b, v23.16b, #12 + uzp2 v6.4s, v25.4s, v19.4s + zip2 v18.4s, v19.4s, v4.4s + bif v3.16b, v7.16b, v1.16b + orr v20.16b, v2.16b, v20.16b + ext v16.16b, v23.16b, v24.16b, #12 + ext v23.16b, v6.16b, v6.16b, #4 + zip1 v24.4s, v18.4s, v7.4s + zip1 v18.4s, v7.4s, v18.4s + ext v25.16b, v3.16b, v3.16b, #12 + add v21.4s, v21.4s, v20.4s + ext v2.16b, v18.16b, v24.16b, #8 + uzp1 v18.4s, v23.4s, v23.4s + ext v24.16b, v23.16b, v23.16b, #12 + uzp1 v3.4s, v3.4s, v25.4s + eor v22.16b, v21.16b, v22.16b + ext v25.16b, v18.16b, v23.16b, #8 + dup v18.4s, v2.s[3] + ext v23.16b, v23.16b, v24.16b, #12 + add v5.4s, v21.4s, v5.4s + trn1 v21.4s, v3.4s, v3.4s + ushr v24.4s, v22.4s, #8 + shl v22.4s, v22.4s, #24 + ext v18.16b, v21.16b, v18.16b, #8 + orr v21.16b, v22.16b, v24.16b + add v17.4s, v21.4s, v17.4s + eor v20.16b, v17.16b, v20.16b + ushr v22.4s, v20.4s, #7 + shl v20.4s, v20.4s, #25 + ext v5.16b, v5.16b, v5.16b, #4 + orr v20.16b, v20.16b, v22.16b + ext v21.16b, v21.16b, v21.16b, #8 + add v5.4s, v20.4s, v5.4s + eor v21.16b, v5.16b, v21.16b + ext v17.16b, v17.16b, v17.16b, #12 + add v5.4s, v5.4s, v19.4s + rev32 v19.8h, v21.8h + add v17.4s, v17.4s, v19.4s + eor v20.16b, v17.16b, v20.16b + ushr v21.4s, v20.4s, #12 + shl v20.4s, v20.4s, #20 + orr v20.16b, v20.16b, v21.16b + add v5.4s, v5.4s, v20.4s + eor v19.16b, v5.16b, v19.16b + ushr v21.4s, v19.4s, #8 + shl v19.4s, v19.4s, #24 + orr v19.16b, v19.16b, v21.16b + add v17.4s, v19.4s, v17.4s + eor v20.16b, v17.16b, v20.16b + ext v5.16b, v5.16b, v5.16b, #12 + ushr v21.4s, v20.4s, #7 + shl v20.4s, v20.4s, #25 + add v5.4s, v5.4s, v7.4s + orr v20.16b, v20.16b, v21.16b + ext v19.16b, v19.16b, v19.16b, #8 + add v5.4s, v5.4s, v20.4s + eor v19.16b, v5.16b, v19.16b + ext v17.16b, v17.16b, v17.16b, #4 + rev32 v22.8h, v19.8h + add v21.4s, v5.4s, v4.4s + mov v4.s[1], v7.s[2] + add v19.4s, v17.4s, v22.4s + bit v16.16b, v4.16b, v0.16b + eor v5.16b, v19.16b, v20.16b + uzp2 v4.4s, v25.4s, v16.4s + zip1 v7.2d, v2.2d, v16.2d + zip2 v17.4s, v16.4s, v2.4s + ushr v20.4s, v5.4s, #12 + shl v5.4s, v5.4s, #20 + ext v24.16b, v4.16b, v4.16b, #4 + bif v7.16b, v3.16b, v1.16b + zip1 v25.4s, v17.4s, v3.4s + zip1 v17.4s, v3.4s, v17.4s + orr v20.16b, v5.16b, v20.16b + ext v26.16b, v7.16b, v7.16b, #12 + ext v5.16b, v17.16b, v25.16b, #8 + uzp1 v17.4s, v24.4s, v24.4s + ext v25.16b, v24.16b, v24.16b, #12 + bit v23.16b, v18.16b, v0.16b + add v21.4s, v21.4s, v20.4s + uzp1 v7.4s, v7.4s, v26.4s + ext v26.16b, v17.16b, v24.16b, #8 + ext v17.16b, v24.16b, v25.16b, #12 + eor v22.16b, v21.16b, v22.16b + add v6.4s, v21.4s, v6.4s + zip1 v21.2d, v5.2d, v23.2d + zip2 v24.4s, v23.4s, v5.4s + bif v21.16b, v7.16b, v1.16b + zip1 v1.4s, v24.4s, v7.4s + zip1 v24.4s, v7.4s, v24.4s + ext v1.16b, v24.16b, v1.16b, #8 + ushr v24.4s, v22.4s, #8 + shl v22.4s, v22.4s, #24 + orr v22.16b, v22.16b, v24.16b + add v19.4s, v22.4s, v19.4s + ext v24.16b, v21.16b, v21.16b, #12 + eor v20.16b, v19.16b, v20.16b + uzp1 v21.4s, v21.4s, v24.4s + ushr v24.4s, v20.4s, #7 + shl v20.4s, v20.4s, #25 + orr v20.16b, v20.16b, v24.16b + ext v6.16b, v6.16b, v6.16b, #4 + ext v22.16b, v22.16b, v22.16b, #8 + add v6.4s, v20.4s, v6.4s + eor v22.16b, v6.16b, v22.16b + ext v19.16b, v19.16b, v19.16b, #12 + add v6.4s, v6.4s, v16.4s + rev32 v16.8h, v22.8h + add v19.4s, v19.4s, v16.4s + eor v20.16b, v19.16b, v20.16b + ushr v22.4s, v20.4s, #12 + shl v20.4s, v20.4s, #20 + orr v20.16b, v20.16b, v22.16b + add v6.4s, v6.4s, v20.4s + eor v16.16b, v6.16b, v16.16b + ext v6.16b, v6.16b, v6.16b, #12 + add v3.4s, v6.4s, v3.4s + ushr v6.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + orr v6.16b, v16.16b, v6.16b + add v16.4s, v6.4s, v19.4s + eor v19.16b, v16.16b, v20.16b + ushr v20.4s, v19.4s, #7 + shl v19.4s, v19.4s, #25 + orr v19.16b, v19.16b, v20.16b + ext v6.16b, v6.16b, v6.16b, #8 + add v3.4s, v3.4s, v19.4s + eor v6.16b, v3.16b, v6.16b + ext v16.16b, v16.16b, v16.16b, #4 + add v2.4s, v3.4s, v2.4s + rev32 v3.8h, v6.8h + add v6.4s, v16.4s, v3.4s + eor v16.16b, v6.16b, v19.16b + ushr v19.4s, v16.4s, #12 + shl v16.4s, v16.4s, #20 + orr v16.16b, v16.16b, v19.16b + add v2.4s, v2.4s, v16.4s + eor v3.16b, v2.16b, v3.16b + add v2.4s, v2.4s, v4.4s + ushr v4.4s, v3.4s, #8 + shl v3.4s, v3.4s, #24 + orr v3.16b, v3.16b, v4.16b + add v4.4s, v3.4s, v6.4s + eor v6.16b, v4.16b, v16.16b + ushr v16.4s, v6.4s, #7 + shl v6.4s, v6.4s, #25 + ext v2.16b, v2.16b, v2.16b, #4 + orr v6.16b, v6.16b, v16.16b + ext v3.16b, v3.16b, v3.16b, #8 + add v2.4s, v6.4s, v2.4s + eor v3.16b, v2.16b, v3.16b + ext v4.16b, v4.16b, v4.16b, #12 + rev32 v3.8h, v3.8h + add v4.4s, v4.4s, v3.4s + eor v6.16b, v4.16b, v6.16b + ushr v16.4s, v6.4s, #12 + shl v6.4s, v6.4s, #20 + add v2.4s, v2.4s, v23.4s + orr v6.16b, v6.16b, v16.16b + add v2.4s, v2.4s, v6.4s + eor v3.16b, v2.16b, v3.16b + ushr v16.4s, v3.4s, #8 + shl v3.4s, v3.4s, #24 + orr v3.16b, v3.16b, v16.16b + add v4.4s, v3.4s, v4.4s + eor v6.16b, v4.16b, v6.16b + ext v2.16b, v2.16b, v2.16b, #12 + ushr v16.4s, v6.4s, #7 + shl v6.4s, v6.4s, #25 + add v2.4s, v2.4s, v7.4s + orr v6.16b, v6.16b, v16.16b + ext v3.16b, v3.16b, v3.16b, #8 + add v2.4s, v2.4s, v6.4s + eor v3.16b, v2.16b, v3.16b + ext v4.16b, v4.16b, v4.16b, #4 + rev32 v3.8h, v3.8h + add v2.4s, v2.4s, v5.4s + mov v5.s[1], v7.s[2] + add v4.4s, v4.4s, v3.4s + bsl v0.16b, v5.16b, v17.16b + eor v5.16b, v4.16b, v6.16b + ushr v6.4s, v5.4s, #12 + shl v5.4s, v5.4s, #20 + orr v5.16b, v5.16b, v6.16b + add v2.4s, v2.4s, v5.4s + eor v3.16b, v2.16b, v3.16b + ushr v6.4s, v3.4s, #8 + shl v3.4s, v3.4s, #24 + orr v3.16b, v3.16b, v6.16b + add v4.4s, v3.4s, v4.4s + uzp2 v18.4s, v26.4s, v18.4s + eor v5.16b, v4.16b, v5.16b + add v2.4s, v2.4s, v18.4s + ushr v6.4s, v5.4s, #7 + shl v5.4s, v5.4s, #25 + ext v2.16b, v2.16b, v2.16b, #4 + orr v5.16b, v5.16b, v6.16b + ext v3.16b, v3.16b, v3.16b, #8 + add v2.4s, v5.4s, v2.4s + eor v3.16b, v2.16b, v3.16b + ext v4.16b, v4.16b, v4.16b, #12 + add v0.4s, v2.4s, v0.4s + rev32 v2.8h, v3.8h + add v3.4s, v4.4s, v2.4s + eor v4.16b, v3.16b, v5.16b + ushr v5.4s, v4.4s, #12 + shl v4.4s, v4.4s, #20 + orr v4.16b, v4.16b, v5.16b + add v0.4s, v0.4s, v4.4s + eor v2.16b, v0.16b, v2.16b + ushr v5.4s, v2.4s, #8 + shl v2.4s, v2.4s, #24 + orr v2.16b, v2.16b, v5.16b + add v3.4s, v2.4s, v3.4s + eor v4.16b, v3.16b, v4.16b + ext v0.16b, v0.16b, v0.16b, #12 + ushr v5.4s, v4.4s, #7 + shl v4.4s, v4.4s, #25 + add v0.4s, v0.4s, v21.4s + orr v4.16b, v4.16b, v5.16b + ext v2.16b, v2.16b, v2.16b, #8 + add v0.4s, v0.4s, v4.4s + eor v2.16b, v0.16b, v2.16b + ext v3.16b, v3.16b, v3.16b, #4 + add v0.4s, v0.4s, v1.4s + rev32 v1.8h, v2.8h + add v2.4s, v3.4s, v1.4s + eor v3.16b, v2.16b, v4.16b + ushr v4.4s, v3.4s, #12 + shl v3.4s, v3.4s, #20 + orr v3.16b, v3.16b, v4.16b + add v0.4s, v0.4s, v3.4s + eor v1.16b, v0.16b, v1.16b + ushr v4.4s, v1.4s, #8 + shl v1.4s, v1.4s, #24 + orr v1.16b, v1.16b, v4.16b + add v2.4s, v1.4s, v2.4s + eor v3.16b, v2.16b, v3.16b + ushr v4.4s, v3.4s, #7 + shl v3.4s, v3.4s, #25 + ext v0.16b, v0.16b, v0.16b, #4 + ext v1.16b, v1.16b, v1.16b, #8 + ext v2.16b, v2.16b, v2.16b, #12 + orr v3.16b, v3.16b, v4.16b + eor v0.16b, v2.16b, v0.16b + eor v3.16b, v3.16b, v1.16b + stp q0, q3, [x5] + ldr q0, [x0] + eor v0.16b, v0.16b, v2.16b + str q0, [x5, #32] + ldr q0, [x0, #16] + eor v0.16b, v0.16b, v1.16b + str q0, [x5, #48] + ret +.Lfunc_end1: + .size zfs_blake3_compress_xof_sse2, .Lfunc_end1-zfs_blake3_compress_xof_sse2 + .cfi_endproc + + .section .rodata.cst16,"aM",@progbits,16 + .p2align 4 +.LCPI2_0: + .word 0 + .word 1 + .word 2 + .word 3 + .text + .globl zfs_blake3_hash_many_sse2 + .p2align 2 + .type zfs_blake3_hash_many_sse2,@function +zfs_blake3_hash_many_sse2: + .cfi_startproc + stp d15, d14, [sp, #-160]! + stp d13, d12, [sp, #16] + stp d11, d10, [sp, #32] + stp d9, d8, [sp, #48] + stp x29, x30, [sp, #64] + stp x28, x27, [sp, #80] + stp x26, x25, [sp, #96] + stp x24, x23, [sp, #112] + stp x22, x21, [sp, #128] + stp x20, x19, [sp, #144] + mov x29, sp + sub sp, sp, #384 + .cfi_def_cfa w29, 160 + .cfi_offset w19, -8 + .cfi_offset w20, -16 + .cfi_offset w21, -24 + .cfi_offset w22, -32 + .cfi_offset w23, -40 + .cfi_offset w24, -48 + .cfi_offset w25, -56 + .cfi_offset w26, -64 + .cfi_offset w27, -72 + .cfi_offset w28, -80 + .cfi_offset w30, -88 + .cfi_offset w29, -96 + .cfi_offset b8, -104 + .cfi_offset b9, -112 + .cfi_offset b10, -120 + .cfi_offset b11, -128 + .cfi_offset b12, -136 + .cfi_offset b13, -144 + .cfi_offset b14, -152 + .cfi_offset b15, -160 + ldr x26, [x29, #168] + ldrb w27, [x29, #160] + mov w19, w6 + mov x20, x4 + mov x22, x2 + mov x28, x1 + cmp x1, #4 + mov x24, x0 + str x3, [sp, #40] + b.lo .LBB2_8 + adrp x9, .LCPI2_0 + ldr q0, [x9, :lo12:.LCPI2_0] + sbfx w11, w5, #0, #1 + dup v1.4s, w11 + mov w9, #58983 + mov w10, #44677 + and v0.16b, v1.16b, v0.16b + mov w11, #62322 + mov w12, #62778 + orr w8, w7, w19 + movk w9, #27145, lsl #16 + movk w10, #47975, lsl #16 + movk w11, #15470, lsl #16 + str q0, [sp, #16] + orr v0.4s, #128, lsl #24 + movk w12, #42319, lsl #16 + str q0, [sp] +.LBB2_2: + ldr x0, [sp, #40] + mov x13, x0 + ld1r { v20.4s }, [x13], #4 + add x14, x0, #8 + add x15, x0, #12 + add x16, x0, #16 + add x17, x0, #20 + add x18, x0, #24 + add x0, x0, #28 + ld1r { v17.4s }, [x14] + ld1r { v6.4s }, [x15] + ld1r { v8.4s }, [x16] + ld1r { v9.4s }, [x17] + ld1r { v31.4s }, [x18] + ld1r { v26.4s }, [x13] + ld1r { v15.4s }, [x0] + cbz x22, .LBB2_7 + ldr q1, [sp, #16] + dup v0.4s, w20 + ldp x13, x14, [x24] + ldp x15, x16, [x24, #16] + add v1.4s, v0.4s, v1.4s + movi v0.4s, #128, lsl #24 + str q1, [sp, #64] + eor v0.16b, v1.16b, v0.16b + ldr q1, [sp] + lsr x18, x20, #32 + mov x17, xzr + cmgt v0.4s, v1.4s, v0.4s + dup v1.4s, w18 + sub v0.4s, v1.4s, v0.4s + mov w18, w8 + str q0, [sp, #48] +.LBB2_4: + mov w2, #16 + bfi x2, x17, #6, #58 + ldr q1, [x13, x2] + ldr q3, [x14, x2] + ldr q2, [x15, x2] + ldr q4, [x16, x2] + mov w2, #32 + bfi x2, x17, #6, #58 + ldr q5, [x13, x2] + ldr q18, [x14, x2] + ldr q19, [x15, x2] + ldr q23, [x16, x2] + mov w2, #48 + lsl x3, x17, #6 + bfi x2, x17, #6, #58 + add x17, x17, #1 + ldr q0, [x13, x3] + ldr q21, [x14, x3] + ldr q7, [x15, x3] + ldr q16, [x16, x3] + cmp x17, x22 + ldr q13, [x13, x2] + ldr q14, [x14, x2] + ldr q29, [x15, x2] + ldr q10, [x16, x2] + csel w2, w27, wzr, eq + orr w18, w2, w18 + mov x0, xzr + and w18, w18, #0xff + add x3, x3, #256 +.LBB2_5: + ldr x2, [x24, x0] + add x0, x0, #8 + cmp x0, #32 + add x2, x2, x3 + prfm pldl1keep, [x2] + b.ne .LBB2_5 + dup v22.4s, w18 + str q22, [sp, #192] + zip1 v27.4s, v0.4s, v21.4s + zip2 v21.4s, v0.4s, v21.4s + zip1 v0.4s, v7.4s, v16.4s + zip2 v22.4s, v7.4s, v16.4s + zip1 v7.4s, v1.4s, v3.4s + zip1 v25.4s, v2.4s, v4.4s + zip2 v16.4s, v2.4s, v4.4s + zip1 v11.4s, v19.4s, v23.4s + zip2 v12.4s, v19.4s, v23.4s + zip1 v19.4s, v13.4s, v14.4s + zip2 v23.4s, v13.4s, v14.4s + zip1 v13.4s, v29.4s, v10.4s + zip2 v14.4s, v29.4s, v10.4s + add v10.4s, v20.4s, v8.4s + add v2.4s, v26.4s, v9.4s + ext v20.16b, v22.16b, v21.16b, #8 + ext v26.16b, v25.16b, v7.16b, #8 + zip2 v24.4s, v1.4s, v3.4s + add v1.4s, v6.4s, v15.4s + ext v6.16b, v0.16b, v27.16b, #8 + ext v20.16b, v21.16b, v20.16b, #8 + mov v21.d[1], v22.d[0] + ext v22.16b, v7.16b, v26.16b, #8 + mov v7.d[1], v25.d[0] + add v3.4s, v17.4s, v31.4s + str q1, [sp, #144] + ext v1.16b, v27.16b, v6.16b, #8 + mov v6.16b, v7.16b + zip1 v28.4s, v5.4s, v18.4s + stur q1, [x29, #-80] + mov v1.16b, v27.16b + mov v27.16b, v24.16b + add v3.4s, v3.4s, v6.4s + ldr q6, [sp, #64] + ext v29.16b, v16.16b, v24.16b, #8 + mov v1.d[1], v0.d[0] + ext v0.16b, v11.16b, v28.16b, #8 + mov v27.d[1], v16.d[0] + ext v16.16b, v14.16b, v23.16b, #8 + stur q7, [x29, #-144] + ext v7.16b, v24.16b, v29.16b, #8 + ext v29.16b, v28.16b, v0.16b, #8 + ext v0.16b, v23.16b, v16.16b, #8 + mov v23.d[1], v14.d[0] + stp q0, q23, [sp, #80] + add v0.4s, v10.4s, v1.4s + eor v16.16b, v0.16b, v6.16b + ldr q6, [sp, #48] + add v2.4s, v2.4s, v21.4s + mov v28.d[1], v11.d[0] + zip2 v18.4s, v5.4s, v18.4s + eor v10.16b, v2.16b, v6.16b + movi v6.4s, #64 + eor v11.16b, v3.16b, v6.16b + ldr q6, [sp, #144] + dup v17.4s, w9 + ext v30.16b, v12.16b, v18.16b, #8 + rev32 v16.8h, v16.8h + dup v5.4s, w10 + ext v25.16b, v18.16b, v30.16b, #8 + mov v30.16b, v23.16b + mov v23.16b, v1.16b + str q1, [sp, #160] + rev32 v10.8h, v10.8h + add v1.4s, v16.4s, v17.4s + add v17.4s, v6.4s, v27.4s + ldr q6, [sp, #192] + dup v4.4s, w11 + rev32 v11.8h, v11.8h + add v5.4s, v10.4s, v5.4s + eor v8.16b, v1.16b, v8.16b + stur q21, [x29, #-128] + mov v18.d[1], v12.d[0] + add v4.4s, v11.4s, v4.4s + eor v9.16b, v5.16b, v9.16b + ushr v12.4s, v8.4s, #12 + shl v8.4s, v8.4s, #20 + ldur q21, [x29, #-80] + ext v26.16b, v13.16b, v19.16b, #8 + eor v31.16b, v4.16b, v31.16b + orr v8.16b, v8.16b, v12.16b + ushr v12.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + ext v26.16b, v19.16b, v26.16b, #8 + mov v19.d[1], v13.d[0] + orr v9.16b, v9.16b, v12.16b + ushr v12.4s, v31.4s, #12 + shl v31.4s, v31.4s, #20 + eor v13.16b, v17.16b, v6.16b + orr v31.16b, v31.16b, v12.16b + dup v12.4s, w12 + rev32 v13.8h, v13.8h + add v12.4s, v13.4s, v12.4s + add v0.4s, v0.4s, v21.4s + eor v14.16b, v12.16b, v15.16b + add v0.4s, v0.4s, v8.4s + add v2.4s, v2.4s, v20.4s + ushr v15.4s, v14.4s, #12 + shl v14.4s, v14.4s, #20 + eor v16.16b, v0.16b, v16.16b + add v2.4s, v2.4s, v9.4s + add v3.4s, v3.4s, v22.4s + orr v14.16b, v14.16b, v15.16b + ushr v15.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + eor v10.16b, v2.16b, v10.16b + add v3.4s, v3.4s, v31.4s + add v17.4s, v17.4s, v7.4s + orr v16.16b, v16.16b, v15.16b + ushr v15.4s, v10.4s, #8 + shl v10.4s, v10.4s, #24 + eor v11.16b, v3.16b, v11.16b + add v17.4s, v17.4s, v14.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v11.4s, #8 + shl v11.4s, v11.4s, #24 + eor v13.16b, v17.16b, v13.16b + add v1.4s, v16.4s, v1.4s + orr v11.16b, v11.16b, v15.16b + ushr v15.4s, v13.4s, #8 + shl v13.4s, v13.4s, #24 + eor v8.16b, v1.16b, v8.16b + add v5.4s, v10.4s, v5.4s + orr v13.16b, v13.16b, v15.16b + ushr v15.4s, v8.4s, #7 + shl v8.4s, v8.4s, #25 + eor v9.16b, v5.16b, v9.16b + add v4.4s, v11.4s, v4.4s + orr v8.16b, v8.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v31.16b, v4.16b, v31.16b + add v12.4s, v13.4s, v12.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #7 + shl v31.4s, v31.4s, #25 + eor v14.16b, v12.16b, v14.16b + add v0.4s, v0.4s, v28.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v14.4s, #7 + shl v14.4s, v14.4s, #25 + add v0.4s, v0.4s, v9.4s + add v2.4s, v2.4s, v18.4s + orr v14.16b, v14.16b, v15.16b + eor v13.16b, v0.16b, v13.16b + add v2.4s, v2.4s, v31.4s + add v3.4s, v3.4s, v19.4s + rev32 v13.8h, v13.8h + eor v16.16b, v2.16b, v16.16b + add v3.4s, v3.4s, v14.4s + add v17.4s, v17.4s, v30.4s + add v4.4s, v4.4s, v13.4s + rev32 v16.8h, v16.8h + eor v10.16b, v3.16b, v10.16b + add v17.4s, v17.4s, v8.4s + eor v9.16b, v4.16b, v9.16b + add v12.4s, v12.4s, v16.4s + rev32 v10.8h, v10.8h + eor v11.16b, v17.16b, v11.16b + mov v24.16b, v7.16b + stur q7, [x29, #-112] + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v31.16b, v12.16b, v31.16b + add v1.4s, v1.4s, v10.4s + rev32 v11.8h, v11.8h + mov v7.16b, v26.16b + add v3.4s, v3.4s, v26.4s + ldr q26, [sp, #80] + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #12 + shl v31.4s, v31.4s, #20 + eor v14.16b, v1.16b, v14.16b + add v5.4s, v5.4s, v11.4s + add v0.4s, v0.4s, v29.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v14.4s, #12 + shl v14.4s, v14.4s, #20 + eor v8.16b, v5.16b, v8.16b + add v0.4s, v0.4s, v9.4s + add v2.4s, v2.4s, v25.4s + orr v14.16b, v14.16b, v15.16b + ushr v15.4s, v8.4s, #12 + shl v8.4s, v8.4s, #20 + eor v13.16b, v0.16b, v13.16b + add v2.4s, v2.4s, v31.4s + orr v8.16b, v8.16b, v15.16b + ushr v15.4s, v13.4s, #8 + shl v13.4s, v13.4s, #24 + eor v16.16b, v2.16b, v16.16b + add v3.4s, v3.4s, v14.4s + add v17.4s, v17.4s, v26.4s + orr v13.16b, v13.16b, v15.16b + ushr v15.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + eor v10.16b, v3.16b, v10.16b + add v17.4s, v17.4s, v8.4s + orr v16.16b, v16.16b, v15.16b + ushr v15.4s, v10.4s, #8 + shl v10.4s, v10.4s, #24 + eor v11.16b, v17.16b, v11.16b + add v4.4s, v13.4s, v4.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v11.4s, #8 + shl v11.4s, v11.4s, #24 + eor v9.16b, v4.16b, v9.16b + add v12.4s, v16.4s, v12.4s + str q22, [sp, #128] + orr v11.16b, v11.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v31.16b, v12.16b, v31.16b + add v1.4s, v10.4s, v1.4s + ldur q22, [x29, #-128] + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #7 + shl v31.4s, v31.4s, #25 + eor v14.16b, v1.16b, v14.16b + add v5.4s, v11.4s, v5.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v14.4s, #7 + shl v14.4s, v14.4s, #25 + eor v8.16b, v5.16b, v8.16b + mov v6.16b, v18.16b + orr v14.16b, v14.16b, v15.16b + ushr v15.4s, v8.4s, #7 + shl v8.4s, v8.4s, #25 + ldur q18, [x29, #-144] + orr v8.16b, v8.16b, v15.16b + add v0.4s, v0.4s, v22.4s + add v0.4s, v0.4s, v8.4s + add v2.4s, v2.4s, v20.4s + eor v16.16b, v0.16b, v16.16b + add v2.4s, v2.4s, v9.4s + add v3.4s, v3.4s, v24.4s + rev32 v16.8h, v16.8h + eor v10.16b, v2.16b, v10.16b + add v3.4s, v3.4s, v31.4s + add v17.4s, v17.4s, v18.4s + add v1.4s, v1.4s, v16.4s + rev32 v10.8h, v10.8h + eor v11.16b, v3.16b, v11.16b + add v17.4s, v17.4s, v14.4s + eor v8.16b, v1.16b, v8.16b + add v5.4s, v5.4s, v10.4s + rev32 v11.8h, v11.8h + eor v13.16b, v17.16b, v13.16b + ushr v15.4s, v8.4s, #12 + shl v8.4s, v8.4s, #20 + eor v9.16b, v5.16b, v9.16b + add v4.4s, v4.4s, v11.4s + rev32 v13.8h, v13.8h + orr v8.16b, v8.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v31.16b, v4.16b, v31.16b + add v12.4s, v12.4s, v13.4s + add v0.4s, v0.4s, v27.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #12 + shl v31.4s, v31.4s, #20 + eor v14.16b, v12.16b, v14.16b + add v0.4s, v0.4s, v8.4s + add v2.4s, v2.4s, v6.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v14.4s, #12 + shl v14.4s, v14.4s, #20 + eor v16.16b, v0.16b, v16.16b + add v2.4s, v2.4s, v9.4s + add v3.4s, v3.4s, v23.4s + orr v14.16b, v14.16b, v15.16b + ushr v15.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + eor v10.16b, v2.16b, v10.16b + add v3.4s, v3.4s, v31.4s + add v17.4s, v17.4s, v7.4s + orr v16.16b, v16.16b, v15.16b + ushr v15.4s, v10.4s, #8 + shl v10.4s, v10.4s, #24 + eor v11.16b, v3.16b, v11.16b + add v17.4s, v17.4s, v14.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v11.4s, #8 + shl v11.4s, v11.4s, #24 + eor v13.16b, v17.16b, v13.16b + add v1.4s, v16.4s, v1.4s + orr v11.16b, v11.16b, v15.16b + ushr v15.4s, v13.4s, #8 + shl v13.4s, v13.4s, #24 + eor v8.16b, v1.16b, v8.16b + add v5.4s, v10.4s, v5.4s + orr v13.16b, v13.16b, v15.16b + ushr v15.4s, v8.4s, #7 + shl v8.4s, v8.4s, #25 + eor v9.16b, v5.16b, v9.16b + add v4.4s, v11.4s, v4.4s + orr v8.16b, v8.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v31.16b, v4.16b, v31.16b + add v12.4s, v13.4s, v12.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #7 + shl v31.4s, v31.4s, #25 + eor v14.16b, v12.16b, v14.16b + add v0.4s, v0.4s, v21.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v14.4s, #7 + shl v14.4s, v14.4s, #25 + add v0.4s, v0.4s, v9.4s + add v2.4s, v2.4s, v19.4s + orr v14.16b, v14.16b, v15.16b + eor v13.16b, v0.16b, v13.16b + add v2.4s, v2.4s, v31.4s + add v3.4s, v3.4s, v29.4s + str q28, [sp, #112] + rev32 v13.8h, v13.8h + eor v16.16b, v2.16b, v16.16b + add v3.4s, v3.4s, v14.4s + add v17.4s, v17.4s, v26.4s + add v4.4s, v4.4s, v13.4s + rev32 v16.8h, v16.8h + eor v10.16b, v3.16b, v10.16b + add v17.4s, v17.4s, v8.4s + ldp q28, q23, [sp, #112] + eor v9.16b, v4.16b, v9.16b + add v12.4s, v12.4s, v16.4s + rev32 v10.8h, v10.8h + eor v11.16b, v17.16b, v11.16b + ldr q21, [sp, #96] + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v31.16b, v12.16b, v31.16b + add v1.4s, v1.4s, v10.4s + rev32 v11.8h, v11.8h + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #12 + shl v31.4s, v31.4s, #20 + eor v14.16b, v1.16b, v14.16b + add v5.4s, v5.4s, v11.4s + add v0.4s, v0.4s, v25.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v14.4s, #12 + shl v14.4s, v14.4s, #20 + eor v8.16b, v5.16b, v8.16b + add v0.4s, v0.4s, v9.4s + add v2.4s, v2.4s, v23.4s + orr v14.16b, v14.16b, v15.16b + ushr v15.4s, v8.4s, #12 + shl v8.4s, v8.4s, #20 + eor v13.16b, v0.16b, v13.16b + add v2.4s, v2.4s, v31.4s + add v3.4s, v3.4s, v21.4s + orr v8.16b, v8.16b, v15.16b + ushr v15.4s, v13.4s, #8 + shl v13.4s, v13.4s, #24 + eor v16.16b, v2.16b, v16.16b + add v3.4s, v3.4s, v14.4s + add v17.4s, v17.4s, v28.4s + orr v13.16b, v13.16b, v15.16b + ushr v15.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + eor v10.16b, v3.16b, v10.16b + add v17.4s, v17.4s, v8.4s + orr v16.16b, v16.16b, v15.16b + ushr v15.4s, v10.4s, #8 + shl v10.4s, v10.4s, #24 + eor v11.16b, v17.16b, v11.16b + add v4.4s, v13.4s, v4.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v11.4s, #8 + shl v11.4s, v11.4s, #24 + eor v9.16b, v4.16b, v9.16b + add v12.4s, v16.4s, v12.4s + orr v11.16b, v11.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v31.16b, v12.16b, v31.16b + add v1.4s, v10.4s, v1.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #7 + shl v31.4s, v31.4s, #25 + eor v14.16b, v1.16b, v14.16b + add v5.4s, v11.4s, v5.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v14.4s, #7 + shl v14.4s, v14.4s, #25 + eor v8.16b, v5.16b, v8.16b + mov v30.16b, v29.16b + mov v29.16b, v25.16b + orr v14.16b, v14.16b, v15.16b + ushr v15.4s, v8.4s, #7 + shl v8.4s, v8.4s, #25 + ldur q25, [x29, #-112] + orr v8.16b, v8.16b, v15.16b + add v0.4s, v0.4s, v20.4s + add v0.4s, v0.4s, v8.4s + add v2.4s, v2.4s, v6.4s + eor v16.16b, v0.16b, v16.16b + add v2.4s, v2.4s, v9.4s + add v3.4s, v3.4s, v7.4s + rev32 v16.8h, v16.8h + eor v10.16b, v2.16b, v10.16b + add v3.4s, v3.4s, v31.4s + add v17.4s, v17.4s, v25.4s + add v1.4s, v1.4s, v16.4s + rev32 v10.8h, v10.8h + eor v11.16b, v3.16b, v11.16b + add v17.4s, v17.4s, v14.4s + eor v8.16b, v1.16b, v8.16b + add v5.4s, v5.4s, v10.4s + rev32 v11.8h, v11.8h + eor v13.16b, v17.16b, v13.16b + ushr v15.4s, v8.4s, #12 + shl v8.4s, v8.4s, #20 + eor v9.16b, v5.16b, v9.16b + add v4.4s, v4.4s, v11.4s + rev32 v13.8h, v13.8h + orr v8.16b, v8.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v31.16b, v4.16b, v31.16b + add v12.4s, v12.4s, v13.4s + add v0.4s, v0.4s, v18.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #12 + shl v31.4s, v31.4s, #20 + eor v14.16b, v12.16b, v14.16b + add v0.4s, v0.4s, v8.4s + add v2.4s, v2.4s, v19.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v14.4s, #12 + shl v14.4s, v14.4s, #20 + eor v16.16b, v0.16b, v16.16b + add v2.4s, v2.4s, v9.4s + add v3.4s, v3.4s, v22.4s + orr v14.16b, v14.16b, v15.16b + ushr v15.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + eor v10.16b, v2.16b, v10.16b + add v3.4s, v3.4s, v31.4s + add v17.4s, v17.4s, v21.4s + orr v16.16b, v16.16b, v15.16b + ushr v15.4s, v10.4s, #8 + shl v10.4s, v10.4s, #24 + eor v11.16b, v3.16b, v11.16b + add v17.4s, v17.4s, v14.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v11.4s, #8 + shl v11.4s, v11.4s, #24 + eor v13.16b, v17.16b, v13.16b + add v1.4s, v16.4s, v1.4s + orr v11.16b, v11.16b, v15.16b + ushr v15.4s, v13.4s, #8 + shl v13.4s, v13.4s, #24 + eor v8.16b, v1.16b, v8.16b + add v5.4s, v10.4s, v5.4s + orr v13.16b, v13.16b, v15.16b + ushr v15.4s, v8.4s, #7 + shl v8.4s, v8.4s, #25 + eor v9.16b, v5.16b, v9.16b + add v4.4s, v11.4s, v4.4s + orr v8.16b, v8.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v31.16b, v4.16b, v31.16b + add v12.4s, v13.4s, v12.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #7 + shl v31.4s, v31.4s, #25 + eor v14.16b, v12.16b, v14.16b + add v0.4s, v0.4s, v27.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v14.4s, #7 + shl v14.4s, v14.4s, #25 + add v0.4s, v0.4s, v9.4s + add v2.4s, v2.4s, v30.4s + orr v14.16b, v14.16b, v15.16b + eor v13.16b, v0.16b, v13.16b + add v2.4s, v2.4s, v31.4s + add v3.4s, v3.4s, v29.4s + rev32 v13.8h, v13.8h + eor v16.16b, v2.16b, v16.16b + add v3.4s, v3.4s, v14.4s + add v17.4s, v17.4s, v28.4s + add v4.4s, v4.4s, v13.4s + rev32 v16.8h, v16.8h + eor v10.16b, v3.16b, v10.16b + add v17.4s, v17.4s, v8.4s + eor v9.16b, v4.16b, v9.16b + add v12.4s, v12.4s, v16.4s + rev32 v10.8h, v10.8h + eor v11.16b, v17.16b, v11.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v31.16b, v12.16b, v31.16b + add v1.4s, v1.4s, v10.4s + rev32 v11.8h, v11.8h + ldr q24, [sp, #160] + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #12 + shl v31.4s, v31.4s, #20 + eor v14.16b, v1.16b, v14.16b + add v5.4s, v5.4s, v11.4s + stur q7, [x29, #-64] + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v14.4s, #12 + shl v14.4s, v14.4s, #20 + eor v8.16b, v5.16b, v8.16b + mov v7.16b, v26.16b + add v3.4s, v3.4s, v26.4s + ldur q26, [x29, #-80] + orr v14.16b, v14.16b, v15.16b + ushr v15.4s, v8.4s, #12 + shl v8.4s, v8.4s, #20 + add v0.4s, v0.4s, v23.4s + orr v8.16b, v8.16b, v15.16b + add v15.4s, v0.4s, v9.4s + add v2.4s, v2.4s, v24.4s + eor v0.16b, v15.16b, v13.16b + add v2.4s, v2.4s, v31.4s + ushr v13.4s, v0.4s, #8 + shl v0.4s, v0.4s, #24 + eor v16.16b, v2.16b, v16.16b + add v3.4s, v3.4s, v14.4s + add v17.4s, v17.4s, v26.4s + orr v0.16b, v0.16b, v13.16b + ushr v13.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + eor v10.16b, v3.16b, v10.16b + add v17.4s, v17.4s, v8.4s + orr v16.16b, v16.16b, v13.16b + ushr v13.4s, v10.4s, #8 + shl v10.4s, v10.4s, #24 + eor v11.16b, v17.16b, v11.16b + add v4.4s, v0.4s, v4.4s + orr v10.16b, v10.16b, v13.16b + ushr v13.4s, v11.4s, #8 + shl v11.4s, v11.4s, #24 + eor v9.16b, v4.16b, v9.16b + add v12.4s, v16.4s, v12.4s + orr v11.16b, v11.16b, v13.16b + ushr v13.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v31.16b, v12.16b, v31.16b + orr v9.16b, v9.16b, v13.16b + ushr v13.4s, v31.4s, #7 + shl v31.4s, v31.4s, #25 + add v1.4s, v10.4s, v1.4s + orr v31.16b, v31.16b, v13.16b + eor v13.16b, v1.16b, v14.16b + add v5.4s, v11.4s, v5.4s + ushr v14.4s, v13.4s, #7 + shl v13.4s, v13.4s, #25 + eor v8.16b, v5.16b, v8.16b + orr v13.16b, v13.16b, v14.16b + ushr v14.4s, v8.4s, #7 + shl v8.4s, v8.4s, #25 + stur q6, [x29, #-96] + orr v8.16b, v8.16b, v14.16b + add v14.4s, v15.4s, v6.4s + ldur q6, [x29, #-64] + mov v18.16b, v19.16b + add v14.4s, v14.4s, v8.4s + add v2.4s, v2.4s, v18.4s + eor v16.16b, v14.16b, v16.16b + add v2.4s, v2.4s, v9.4s + add v3.4s, v3.4s, v21.4s + rev32 v16.8h, v16.8h + eor v10.16b, v2.16b, v10.16b + add v3.4s, v3.4s, v31.4s + add v17.4s, v17.4s, v6.4s + add v1.4s, v1.4s, v16.4s + rev32 v10.8h, v10.8h + eor v11.16b, v3.16b, v11.16b + add v17.4s, v17.4s, v13.4s + eor v8.16b, v1.16b, v8.16b + add v5.4s, v5.4s, v10.4s + rev32 v11.8h, v11.8h + eor v0.16b, v17.16b, v0.16b + ushr v15.4s, v8.4s, #12 + shl v8.4s, v8.4s, #20 + eor v9.16b, v5.16b, v9.16b + add v4.4s, v4.4s, v11.4s + rev32 v0.8h, v0.8h + str q27, [sp, #176] + mov v27.16b, v30.16b + orr v8.16b, v8.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v31.16b, v4.16b, v31.16b + add v12.4s, v12.4s, v0.4s + add v14.4s, v14.4s, v25.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #12 + shl v31.4s, v31.4s, #20 + eor v13.16b, v12.16b, v13.16b + add v14.4s, v14.4s, v8.4s + add v2.4s, v2.4s, v27.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v13.4s, #12 + shl v13.4s, v13.4s, #20 + eor v16.16b, v14.16b, v16.16b + add v2.4s, v2.4s, v9.4s + add v3.4s, v3.4s, v20.4s + orr v13.16b, v13.16b, v15.16b + ushr v15.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + eor v10.16b, v2.16b, v10.16b + add v3.4s, v3.4s, v31.4s + add v17.4s, v17.4s, v7.4s + orr v16.16b, v16.16b, v15.16b + ushr v15.4s, v10.4s, #8 + shl v10.4s, v10.4s, #24 + eor v11.16b, v3.16b, v11.16b + add v17.4s, v17.4s, v13.4s + mov v30.16b, v23.16b + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v11.4s, #8 + shl v11.4s, v11.4s, #24 + eor v0.16b, v17.16b, v0.16b + add v1.4s, v16.4s, v1.4s + ldur q23, [x29, #-144] + orr v11.16b, v11.16b, v15.16b + ushr v15.4s, v0.4s, #8 + shl v0.4s, v0.4s, #24 + eor v8.16b, v1.16b, v8.16b + add v5.4s, v10.4s, v5.4s + orr v0.16b, v0.16b, v15.16b + ushr v15.4s, v8.4s, #7 + shl v8.4s, v8.4s, #25 + eor v9.16b, v5.16b, v9.16b + add v4.4s, v11.4s, v4.4s + orr v8.16b, v8.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v31.16b, v4.16b, v31.16b + add v12.4s, v0.4s, v12.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #7 + shl v31.4s, v31.4s, #25 + eor v13.16b, v12.16b, v13.16b + add v14.4s, v14.4s, v23.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v13.4s, #7 + shl v13.4s, v13.4s, #25 + add v14.4s, v14.4s, v9.4s + add v2.4s, v2.4s, v29.4s + orr v13.16b, v13.16b, v15.16b + eor v0.16b, v14.16b, v0.16b + add v2.4s, v2.4s, v31.4s + add v3.4s, v3.4s, v30.4s + rev32 v0.8h, v0.8h + eor v16.16b, v2.16b, v16.16b + add v3.4s, v3.4s, v13.4s + add v17.4s, v17.4s, v26.4s + add v4.4s, v4.4s, v0.4s + rev32 v16.8h, v16.8h + eor v10.16b, v3.16b, v10.16b + add v17.4s, v17.4s, v8.4s + ldur q22, [x29, #-128] + eor v9.16b, v4.16b, v9.16b + add v12.4s, v12.4s, v16.4s + rev32 v10.8h, v10.8h + eor v11.16b, v17.16b, v11.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v31.16b, v12.16b, v31.16b + add v1.4s, v1.4s, v10.4s + rev32 v11.8h, v11.8h + ldr q26, [sp, #176] + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #12 + shl v31.4s, v31.4s, #20 + eor v13.16b, v1.16b, v13.16b + add v5.4s, v5.4s, v11.4s + add v14.4s, v14.4s, v24.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v13.4s, #12 + shl v13.4s, v13.4s, #20 + eor v8.16b, v5.16b, v8.16b + add v14.4s, v14.4s, v9.4s + add v2.4s, v2.4s, v22.4s + orr v13.16b, v13.16b, v15.16b + ushr v15.4s, v8.4s, #12 + shl v8.4s, v8.4s, #20 + eor v0.16b, v14.16b, v0.16b + add v2.4s, v2.4s, v31.4s + add v3.4s, v3.4s, v28.4s + orr v8.16b, v8.16b, v15.16b + ushr v15.4s, v0.4s, #8 + shl v0.4s, v0.4s, #24 + eor v16.16b, v2.16b, v16.16b + add v3.4s, v3.4s, v13.4s + add v17.4s, v17.4s, v26.4s + orr v0.16b, v0.16b, v15.16b + ushr v15.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + eor v10.16b, v3.16b, v10.16b + add v17.4s, v17.4s, v8.4s + orr v16.16b, v16.16b, v15.16b + ushr v15.4s, v10.4s, #8 + shl v10.4s, v10.4s, #24 + eor v11.16b, v17.16b, v11.16b + add v4.4s, v0.4s, v4.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v11.4s, #8 + shl v11.4s, v11.4s, #24 + eor v9.16b, v4.16b, v9.16b + add v12.4s, v16.4s, v12.4s + orr v11.16b, v11.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v31.16b, v12.16b, v31.16b + add v1.4s, v10.4s, v1.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #7 + shl v31.4s, v31.4s, #25 + eor v13.16b, v1.16b, v13.16b + add v5.4s, v11.4s, v5.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v13.4s, #7 + shl v13.4s, v13.4s, #25 + eor v8.16b, v5.16b, v8.16b + orr v13.16b, v13.16b, v15.16b + ushr v15.4s, v8.4s, #7 + shl v8.4s, v8.4s, #25 + orr v8.16b, v8.16b, v15.16b + add v14.4s, v14.4s, v18.4s + add v14.4s, v14.4s, v8.4s + add v2.4s, v2.4s, v27.4s + eor v16.16b, v14.16b, v16.16b + add v2.4s, v2.4s, v9.4s + add v3.4s, v3.4s, v7.4s + rev32 v16.8h, v16.8h + eor v10.16b, v2.16b, v10.16b + add v3.4s, v3.4s, v31.4s + add v17.4s, v17.4s, v21.4s + add v1.4s, v1.4s, v16.4s + rev32 v10.8h, v10.8h + eor v11.16b, v3.16b, v11.16b + add v17.4s, v17.4s, v13.4s + eor v8.16b, v1.16b, v8.16b + add v5.4s, v5.4s, v10.4s + rev32 v11.8h, v11.8h + eor v0.16b, v17.16b, v0.16b + add v14.4s, v14.4s, v6.4s + ldur q6, [x29, #-96] + ushr v15.4s, v8.4s, #12 + shl v8.4s, v8.4s, #20 + eor v9.16b, v5.16b, v9.16b + add v4.4s, v4.4s, v11.4s + rev32 v0.8h, v0.8h + stur q20, [x29, #-160] + mov v20.16b, v29.16b + orr v8.16b, v8.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v31.16b, v4.16b, v31.16b + add v12.4s, v12.4s, v0.4s + mov v19.16b, v29.16b + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #12 + shl v31.4s, v31.4s, #20 + eor v13.16b, v12.16b, v13.16b + add v14.4s, v14.4s, v8.4s + add v2.4s, v2.4s, v20.4s + mov v19.16b, v28.16b + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v13.4s, #12 + shl v13.4s, v13.4s, #20 + eor v16.16b, v14.16b, v16.16b + add v2.4s, v2.4s, v9.4s + add v3.4s, v3.4s, v6.4s + orr v13.16b, v13.16b, v15.16b + ushr v15.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + eor v10.16b, v2.16b, v10.16b + add v3.4s, v3.4s, v31.4s + add v17.4s, v17.4s, v19.4s + orr v16.16b, v16.16b, v15.16b + ushr v15.4s, v10.4s, #8 + shl v10.4s, v10.4s, #24 + eor v11.16b, v3.16b, v11.16b + add v17.4s, v17.4s, v13.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v11.4s, #8 + shl v11.4s, v11.4s, #24 + eor v0.16b, v17.16b, v0.16b + add v1.4s, v16.4s, v1.4s + orr v11.16b, v11.16b, v15.16b + ushr v15.4s, v0.4s, #8 + shl v0.4s, v0.4s, #24 + eor v8.16b, v1.16b, v8.16b + add v5.4s, v10.4s, v5.4s + orr v0.16b, v0.16b, v15.16b + ushr v15.4s, v8.4s, #7 + shl v8.4s, v8.4s, #25 + eor v9.16b, v5.16b, v9.16b + add v4.4s, v11.4s, v4.4s + orr v8.16b, v8.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v31.16b, v4.16b, v31.16b + add v12.4s, v0.4s, v12.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #7 + shl v31.4s, v31.4s, #25 + eor v13.16b, v12.16b, v13.16b + add v14.4s, v14.4s, v25.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v13.4s, #7 + shl v13.4s, v13.4s, #25 + add v14.4s, v14.4s, v9.4s + add v2.4s, v2.4s, v30.4s + orr v13.16b, v13.16b, v15.16b + eor v0.16b, v14.16b, v0.16b + add v2.4s, v2.4s, v31.4s + add v3.4s, v3.4s, v24.4s + rev32 v0.8h, v0.8h + eor v16.16b, v2.16b, v16.16b + add v3.4s, v3.4s, v13.4s + add v17.4s, v17.4s, v26.4s + mov v29.16b, v27.16b + add v4.4s, v4.4s, v0.4s + rev32 v16.8h, v16.8h + eor v10.16b, v3.16b, v10.16b + add v17.4s, v17.4s, v8.4s + ldur q27, [x29, #-160] + eor v9.16b, v4.16b, v9.16b + add v12.4s, v12.4s, v16.4s + rev32 v10.8h, v10.8h + eor v11.16b, v17.16b, v11.16b + ldur q6, [x29, #-80] + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v31.16b, v12.16b, v31.16b + add v1.4s, v1.4s, v10.4s + rev32 v11.8h, v11.8h + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #12 + shl v31.4s, v31.4s, #20 + eor v13.16b, v1.16b, v13.16b + add v5.4s, v5.4s, v11.4s + add v14.4s, v14.4s, v22.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v13.4s, #12 + shl v13.4s, v13.4s, #20 + eor v8.16b, v5.16b, v8.16b + add v14.4s, v14.4s, v9.4s + add v2.4s, v2.4s, v27.4s + orr v13.16b, v13.16b, v15.16b + ushr v15.4s, v8.4s, #12 + shl v8.4s, v8.4s, #20 + eor v0.16b, v14.16b, v0.16b + add v2.4s, v2.4s, v31.4s + add v3.4s, v3.4s, v6.4s + orr v8.16b, v8.16b, v15.16b + ushr v15.4s, v0.4s, #8 + shl v0.4s, v0.4s, #24 + eor v16.16b, v2.16b, v16.16b + add v3.4s, v3.4s, v13.4s + add v17.4s, v17.4s, v23.4s + orr v0.16b, v0.16b, v15.16b + ushr v15.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + eor v10.16b, v3.16b, v10.16b + add v17.4s, v17.4s, v8.4s + orr v16.16b, v16.16b, v15.16b + ushr v15.4s, v10.4s, #8 + shl v10.4s, v10.4s, #24 + eor v11.16b, v17.16b, v11.16b + add v4.4s, v0.4s, v4.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v11.4s, #8 + shl v11.4s, v11.4s, #24 + eor v9.16b, v4.16b, v9.16b + add v12.4s, v16.4s, v12.4s + orr v11.16b, v11.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v31.16b, v12.16b, v31.16b + add v1.4s, v10.4s, v1.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #7 + shl v31.4s, v31.4s, #25 + eor v13.16b, v1.16b, v13.16b + add v5.4s, v11.4s, v5.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v13.4s, #7 + shl v13.4s, v13.4s, #25 + eor v8.16b, v5.16b, v8.16b + orr v13.16b, v13.16b, v15.16b + ushr v15.4s, v8.4s, #7 + shl v8.4s, v8.4s, #25 + orr v8.16b, v8.16b, v15.16b + add v14.4s, v14.4s, v29.4s + add v14.4s, v14.4s, v8.4s + add v2.4s, v2.4s, v20.4s + mov v28.16b, v7.16b + eor v16.16b, v14.16b, v16.16b + add v2.4s, v2.4s, v9.4s + add v3.4s, v3.4s, v19.4s + rev32 v16.8h, v16.8h + eor v10.16b, v2.16b, v10.16b + add v3.4s, v3.4s, v31.4s + add v17.4s, v17.4s, v28.4s + add v1.4s, v1.4s, v16.4s + rev32 v10.8h, v10.8h + eor v11.16b, v3.16b, v11.16b + add v17.4s, v17.4s, v13.4s + eor v8.16b, v1.16b, v8.16b + add v5.4s, v5.4s, v10.4s + rev32 v11.8h, v11.8h + eor v0.16b, v17.16b, v0.16b + ushr v15.4s, v8.4s, #12 + shl v8.4s, v8.4s, #20 + eor v9.16b, v5.16b, v9.16b + add v4.4s, v4.4s, v11.4s + rev32 v0.8h, v0.8h + orr v8.16b, v8.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v31.16b, v4.16b, v31.16b + add v12.4s, v12.4s, v0.4s + add v14.4s, v14.4s, v21.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v31.4s, #12 + shl v31.4s, v31.4s, #20 + eor v13.16b, v12.16b, v13.16b + add v14.4s, v14.4s, v8.4s + add v2.4s, v2.4s, v30.4s + orr v31.16b, v31.16b, v15.16b + ushr v15.4s, v13.4s, #12 + shl v13.4s, v13.4s, #20 + eor v16.16b, v14.16b, v16.16b + add v2.4s, v2.4s, v9.4s + orr v13.16b, v13.16b, v15.16b + ushr v15.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + eor v10.16b, v2.16b, v10.16b + orr v16.16b, v16.16b, v15.16b + ushr v15.4s, v10.4s, #8 + shl v10.4s, v10.4s, #24 + add v3.4s, v3.4s, v18.4s + orr v10.16b, v10.16b, v15.16b + add v15.4s, v3.4s, v31.4s + eor v3.16b, v15.16b, v11.16b + ushr v11.4s, v3.4s, #8 + shl v3.4s, v3.4s, #24 + orr v11.16b, v3.16b, v11.16b + add v3.4s, v17.4s, v6.4s + add v17.4s, v3.4s, v13.4s + eor v0.16b, v17.16b, v0.16b + ushr v3.4s, v0.4s, #8 + shl v0.4s, v0.4s, #24 + add v1.4s, v16.4s, v1.4s + orr v0.16b, v0.16b, v3.16b + eor v3.16b, v1.16b, v8.16b + ushr v8.4s, v3.4s, #7 + shl v3.4s, v3.4s, #25 + add v5.4s, v10.4s, v5.4s + orr v8.16b, v3.16b, v8.16b + eor v3.16b, v5.16b, v9.16b + add v4.4s, v11.4s, v4.4s + ushr v9.4s, v3.4s, #7 + shl v3.4s, v3.4s, #25 + eor v31.16b, v4.16b, v31.16b + mov v7.16b, v23.16b + mov v23.16b, v28.16b + mov v28.16b, v6.16b + orr v3.16b, v3.16b, v9.16b + ushr v9.4s, v31.4s, #7 + shl v31.4s, v31.4s, #25 + ldur q6, [x29, #-64] + orr v31.16b, v31.16b, v9.16b + add v9.4s, v0.4s, v12.4s + eor v12.16b, v9.16b, v13.16b + ushr v13.4s, v12.4s, #7 + shl v12.4s, v12.4s, #25 + orr v12.16b, v12.16b, v13.16b + add v13.4s, v14.4s, v6.4s + add v13.4s, v13.4s, v3.4s + eor v0.16b, v13.16b, v0.16b + add v2.4s, v2.4s, v24.4s + rev32 v14.8h, v0.8h + add v0.4s, v2.4s, v31.4s + add v6.4s, v4.4s, v14.4s + eor v2.16b, v0.16b, v16.16b + eor v3.16b, v6.16b, v3.16b + rev32 v16.8h, v2.8h + ushr v4.4s, v3.4s, #12 + shl v3.4s, v3.4s, #20 + add v2.4s, v9.4s, v16.4s + orr v4.16b, v3.16b, v4.16b + eor v3.16b, v2.16b, v31.16b + ushr v31.4s, v3.4s, #12 + shl v3.4s, v3.4s, #20 + orr v3.16b, v3.16b, v31.16b + add v31.4s, v15.4s, v22.4s + add v31.4s, v31.4s, v12.4s + add v17.4s, v17.4s, v7.4s + eor v9.16b, v31.16b, v10.16b + add v17.4s, v17.4s, v8.4s + rev32 v9.8h, v9.8h + eor v11.16b, v17.16b, v11.16b + add v1.4s, v1.4s, v9.4s + rev32 v11.8h, v11.8h + eor v10.16b, v1.16b, v12.16b + add v5.4s, v5.4s, v11.4s + ushr v12.4s, v10.4s, #12 + shl v10.4s, v10.4s, #20 + eor v8.16b, v5.16b, v8.16b + orr v10.16b, v10.16b, v12.16b + ushr v12.4s, v8.4s, #12 + shl v8.4s, v8.4s, #20 + orr v8.16b, v8.16b, v12.16b + add v12.4s, v13.4s, v27.4s + add v12.4s, v12.4s, v4.4s + eor v13.16b, v12.16b, v14.16b + ldur q14, [x29, #-96] + mov v25.16b, v29.16b + add v29.4s, v12.4s, v20.4s + add v20.4s, v31.4s, v26.4s + add v0.4s, v0.4s, v14.4s + add v0.4s, v0.4s, v3.4s + eor v16.16b, v0.16b, v16.16b + add v0.4s, v0.4s, v30.4s + ldur q30, [x29, #-112] + add v20.4s, v20.4s, v10.4s + eor v31.16b, v20.16b, v9.16b + add v20.4s, v20.4s, v28.4s + add v17.4s, v17.4s, v30.4s + add v17.4s, v17.4s, v8.4s + eor v9.16b, v17.16b, v11.16b + ushr v28.4s, v13.4s, #8 + shl v11.4s, v13.4s, #24 + orr v28.16b, v11.16b, v28.16b + ushr v11.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + orr v16.16b, v16.16b, v11.16b + ushr v11.4s, v31.4s, #8 + shl v31.4s, v31.4s, #24 + add v6.4s, v28.4s, v6.4s + orr v31.16b, v31.16b, v11.16b + ushr v11.4s, v9.4s, #8 + shl v9.4s, v9.4s, #24 + add v2.4s, v16.4s, v2.4s + eor v4.16b, v6.16b, v4.16b + orr v9.16b, v9.16b, v11.16b + add v1.4s, v31.4s, v1.4s + eor v3.16b, v2.16b, v3.16b + ushr v11.4s, v4.4s, #7 + shl v4.4s, v4.4s, #25 + add v5.4s, v9.4s, v5.4s + eor v10.16b, v1.16b, v10.16b + orr v4.16b, v4.16b, v11.16b + ushr v11.4s, v3.4s, #7 + shl v3.4s, v3.4s, #25 + eor v8.16b, v5.16b, v8.16b + orr v3.16b, v3.16b, v11.16b + ushr v11.4s, v10.4s, #7 + shl v10.4s, v10.4s, #25 + orr v10.16b, v10.16b, v11.16b + ushr v11.4s, v8.4s, #7 + shl v8.4s, v8.4s, #25 + orr v8.16b, v8.16b, v11.16b + add v29.4s, v29.4s, v8.4s + eor v16.16b, v29.16b, v16.16b + add v0.4s, v0.4s, v4.4s + mov v12.16b, v26.16b + add v17.4s, v17.4s, v19.4s + add v26.4s, v29.4s, v23.4s + eor v29.16b, v0.16b, v31.16b + add v20.4s, v20.4s, v3.4s + rev32 v16.8h, v16.8h + stur q18, [x29, #-176] + mov v18.16b, v27.16b + add v0.4s, v0.4s, v24.4s + eor v27.16b, v20.16b, v9.16b + add v17.4s, v17.4s, v10.4s + rev32 v24.8h, v29.8h + add v1.4s, v1.4s, v16.4s + add v20.4s, v20.4s, v25.4s + eor v25.16b, v17.16b, v28.16b + rev32 v27.8h, v27.8h + add v5.4s, v5.4s, v24.4s + eor v28.16b, v1.16b, v8.16b + rev32 v25.8h, v25.8h + add v6.4s, v6.4s, v27.4s + eor v4.16b, v5.16b, v4.16b + ushr v31.4s, v28.4s, #12 + shl v28.4s, v28.4s, #20 + add v2.4s, v2.4s, v25.4s + eor v3.16b, v6.16b, v3.16b + orr v28.16b, v28.16b, v31.16b + ushr v31.4s, v4.4s, #12 + shl v4.4s, v4.4s, #20 + eor v29.16b, v2.16b, v10.16b + orr v4.16b, v4.16b, v31.16b + ushr v31.4s, v3.4s, #12 + shl v3.4s, v3.4s, #20 + add v26.4s, v26.4s, v28.4s + orr v3.16b, v3.16b, v31.16b + ushr v31.4s, v29.4s, #12 + shl v29.4s, v29.4s, #20 + eor v16.16b, v26.16b, v16.16b + add v0.4s, v0.4s, v4.4s + add v17.4s, v17.4s, v12.4s + orr v29.16b, v29.16b, v31.16b + eor v24.16b, v0.16b, v24.16b + add v0.4s, v0.4s, v22.4s + add v20.4s, v20.4s, v3.4s + ushr v22.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + add v23.4s, v26.4s, v21.4s + eor v21.16b, v20.16b, v27.16b + add v17.4s, v17.4s, v29.4s + orr v16.16b, v16.16b, v22.16b + ushr v22.4s, v24.4s, #8 + shl v24.4s, v24.4s, #24 + eor v25.16b, v17.16b, v25.16b + orr v22.16b, v24.16b, v22.16b + ushr v24.4s, v21.4s, #8 + shl v21.4s, v21.4s, #24 + orr v21.16b, v21.16b, v24.16b + ushr v24.4s, v25.4s, #8 + shl v25.4s, v25.4s, #24 + add v1.4s, v16.4s, v1.4s + orr v24.16b, v25.16b, v24.16b + add v5.4s, v22.4s, v5.4s + eor v25.16b, v1.16b, v28.16b + add v6.4s, v21.4s, v6.4s + eor v4.16b, v5.16b, v4.16b + ushr v27.4s, v25.4s, #7 + shl v25.4s, v25.4s, #25 + add v2.4s, v24.4s, v2.4s + eor v3.16b, v6.16b, v3.16b + orr v25.16b, v25.16b, v27.16b + ushr v27.4s, v4.4s, #7 + shl v4.4s, v4.4s, #25 + ldur q19, [x29, #-176] + eor v26.16b, v2.16b, v29.16b + orr v4.16b, v4.16b, v27.16b + ushr v27.4s, v3.4s, #7 + shl v3.4s, v3.4s, #25 + orr v3.16b, v3.16b, v27.16b + ushr v27.4s, v26.4s, #7 + shl v26.4s, v26.4s, #25 + add v20.4s, v20.4s, v18.4s + add v17.4s, v17.4s, v30.4s + orr v26.16b, v26.16b, v27.16b + add v0.4s, v0.4s, v3.4s + eor v16.16b, v0.16b, v16.16b + add v0.4s, v0.4s, v19.4s + add v19.4s, v20.4s, v26.4s + add v17.4s, v17.4s, v25.4s + eor v20.16b, v19.16b, v22.16b + add v7.4s, v19.4s, v7.4s + eor v19.16b, v17.16b, v21.16b + ldur q21, [x29, #-64] + add v23.4s, v23.4s, v4.4s + eor v24.16b, v23.16b, v24.16b + rev32 v16.8h, v16.8h + add v17.4s, v17.4s, v21.4s + rev32 v21.8h, v24.8h + add v6.4s, v6.4s, v21.4s + rev32 v20.8h, v20.8h + add v2.4s, v2.4s, v16.4s + eor v4.16b, v6.16b, v4.16b + rev32 v19.8h, v19.8h + add v1.4s, v1.4s, v20.4s + eor v3.16b, v2.16b, v3.16b + ushr v24.4s, v4.4s, #12 + shl v4.4s, v4.4s, #20 + add v5.4s, v5.4s, v19.4s + eor v22.16b, v1.16b, v26.16b + orr v4.16b, v4.16b, v24.16b + ushr v24.4s, v3.4s, #12 + shl v3.4s, v3.4s, #20 + add v18.4s, v23.4s, v14.4s + eor v23.16b, v5.16b, v25.16b + orr v3.16b, v3.16b, v24.16b + ushr v24.4s, v22.4s, #12 + shl v22.4s, v22.4s, #20 + orr v22.16b, v22.16b, v24.16b + ushr v24.4s, v23.4s, #12 + shl v23.4s, v23.4s, #20 + orr v23.16b, v23.16b, v24.16b + add v18.4s, v18.4s, v4.4s + add v0.4s, v0.4s, v3.4s + add v24.4s, v17.4s, v23.4s + eor v17.16b, v18.16b, v21.16b + add v7.4s, v7.4s, v22.4s + eor v16.16b, v0.16b, v16.16b + ushr v21.4s, v17.4s, #8 + shl v17.4s, v17.4s, #24 + eor v20.16b, v7.16b, v20.16b + orr v21.16b, v17.16b, v21.16b + ushr v17.4s, v16.4s, #8 + shl v16.4s, v16.4s, #24 + eor v19.16b, v24.16b, v19.16b + orr v16.16b, v16.16b, v17.16b + ushr v17.4s, v20.4s, #8 + shl v20.4s, v20.4s, #24 + orr v25.16b, v20.16b, v17.16b + ushr v17.4s, v19.4s, #8 + shl v19.4s, v19.4s, #24 + orr v19.16b, v19.16b, v17.16b + add v1.4s, v25.4s, v1.4s + eor v22.16b, v1.16b, v22.16b + eor v20.16b, v1.16b, v18.16b + add v1.4s, v19.4s, v5.4s + eor v26.16b, v1.16b, v0.16b + add v0.4s, v21.4s, v6.4s + eor v5.16b, v1.16b, v23.16b + eor v1.16b, v0.16b, v4.16b + eor v17.16b, v0.16b, v7.16b + add v0.4s, v16.4s, v2.4s + eor v2.16b, v0.16b, v3.16b + eor v6.16b, v0.16b, v24.16b + ushr v0.4s, v1.4s, #7 + shl v1.4s, v1.4s, #25 + orr v0.16b, v1.16b, v0.16b + ushr v1.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + orr v1.16b, v2.16b, v1.16b + ushr v2.4s, v22.4s, #7 + shl v3.4s, v22.4s, #25 + orr v2.16b, v3.16b, v2.16b + ushr v3.4s, v5.4s, #7 + shl v4.4s, v5.4s, #25 + orr v3.16b, v4.16b, v3.16b + eor v8.16b, v16.16b, v3.16b + eor v9.16b, v25.16b, v0.16b + eor v31.16b, v1.16b, v19.16b + cmp x17, x22 + eor v15.16b, v2.16b, v21.16b + mov w18, w19 + b.ne .LBB2_4 +.LBB2_7: + zip1 v0.4s, v20.4s, v26.4s + zip2 v1.4s, v20.4s, v26.4s + zip1 v2.4s, v17.4s, v6.4s + zip2 v3.4s, v17.4s, v6.4s + zip1 v4.4s, v8.4s, v9.4s + zip2 v5.4s, v8.4s, v9.4s + zip1 v6.4s, v31.4s, v15.4s + zip2 v7.4s, v31.4s, v15.4s + add x13, x20, #4 + tst w5, #0x1 + sub x28, x28, #4 + zip1 v16.2d, v0.2d, v2.2d + zip2 v0.2d, v0.2d, v2.2d + zip1 v2.2d, v1.2d, v3.2d + zip2 v1.2d, v1.2d, v3.2d + zip1 v3.2d, v4.2d, v6.2d + zip2 v4.2d, v4.2d, v6.2d + zip1 v6.2d, v5.2d, v7.2d + zip2 v5.2d, v5.2d, v7.2d + add x24, x24, #32 + csel x20, x13, x20, ne + cmp x28, #3 + stp q16, q3, [x26] + stp q0, q4, [x26, #32] + stp q2, q6, [x26, #64] + stp q1, q5, [x26, #96] + add x26, x26, #128 + b.hi .LBB2_2 +.LBB2_8: + cbz x28, .LBB2_16 + orr w8, w7, w19 + and x21, x5, #0x1 + stur w8, [x29, #-64] +.LBB2_10: + ldr x8, [sp, #40] + ldr x25, [x24] + ldur w4, [x29, #-64] + ldp q1, q0, [x8] + mov x8, x22 + stp q1, q0, [x29, #-48] +.LBB2_11: + subs x23, x8, #1 + b.eq .LBB2_13 + cbnz x8, .LBB2_14 + b .LBB2_15 +.LBB2_13: + orr w4, w4, w27 +.LBB2_14: + sub x0, x29, #48 + mov w2, #64 + mov x1, x25 + mov x3, x20 + bl zfs_blake3_compress_in_place_sse2 + add x25, x25, #64 + mov x8, x23 + mov w4, w19 + b .LBB2_11 +.LBB2_15: + ldp q0, q1, [x29, #-48] + add x20, x20, x21 + add x24, x24, #8 + subs x28, x28, #1 + stp q0, q1, [x26], #32 + b.ne .LBB2_10 +.LBB2_16: + add sp, sp, #384 + ldp x20, x19, [sp, #144] + ldp x22, x21, [sp, #128] + ldp x24, x23, [sp, #112] + ldp x26, x25, [sp, #96] + ldp x28, x27, [sp, #80] + ldp x29, x30, [sp, #64] + ldp d9, d8, [sp, #48] + ldp d11, d10, [sp, #32] + ldp d13, d12, [sp, #16] + ldp d15, d14, [sp], #160 + ret +.Lfunc_end2: + .size zfs_blake3_hash_many_sse2, .Lfunc_end2-zfs_blake3_hash_many_sse2 + .cfi_endproc + .section ".note.GNU-stack","",@progbits +#endif diff --git a/module/icp/asm-aarch64/blake3/b3_aarch64_sse41.S b/module/icp/asm-aarch64/blake3/b3_aarch64_sse41.S new file mode 100644 index 000000000000..a05baec96ce5 --- /dev/null +++ b/module/icp/asm-aarch64/blake3/b3_aarch64_sse41.S @@ -0,0 +1,2463 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Based on BLAKE3 v1.3.1, https://github.com/BLAKE3-team/BLAKE3 + * Copyright (c) 2019-2022 Samuel Neves + * Copyright (c) 2022 Tino Reichardt + * + * This is converted assembly: SSE4.1 -> ARMv8-A + * Used tools: SIMDe https://github.com/simd-everywhere/simde + */ + +#if defined(__aarch64__) + .text + .section .rodata.cst16,"aM",@progbits,16 + .p2align 4 +.LCPI0_0: + .byte 2 + .byte 3 + .byte 0 + .byte 1 + .byte 6 + .byte 7 + .byte 4 + .byte 5 + .byte 10 + .byte 11 + .byte 8 + .byte 9 + .byte 14 + .byte 15 + .byte 12 + .byte 13 +.LCPI0_1: + .word 1779033703 + .word 3144134277 + .word 1013904242 + .word 2773480762 +.LCPI0_2: + .byte 1 + .byte 2 + .byte 3 + .byte 0 + .byte 5 + .byte 6 + .byte 7 + .byte 4 + .byte 9 + .byte 10 + .byte 11 + .byte 8 + .byte 13 + .byte 14 + .byte 15 + .byte 12 +.LCPI0_3: + .byte 0 + .byte 1 + .byte 2 + .byte 3 + .byte 20 + .byte 21 + .byte 22 + .byte 23 + .byte 8 + .byte 9 + .byte 10 + .byte 11 + .byte 28 + .byte 29 + .byte 30 + .byte 31 +.LCPI0_4: + .byte 0 + .byte 1 + .byte 2 + .byte 3 + .byte 4 + .byte 5 + .byte 6 + .byte 7 + .byte 8 + .byte 9 + .byte 10 + .byte 11 + .byte 28 + .byte 29 + .byte 30 + .byte 31 + .text + .globl zfs_blake3_compress_in_place_sse41 + .p2align 2 + .type zfs_blake3_compress_in_place_sse41,@function +zfs_blake3_compress_in_place_sse41: + .cfi_startproc + ldp q7, q6, [x0] + ldp q17, q18, [x1] + add x12, x1, #32 + ld2 { v4.4s, v5.4s }, [x12] + lsr x10, x3, #32 + fmov s16, w3 + adrp x13, .LCPI0_0 + adrp x11, .LCPI0_1 + and w8, w2, #0xff + mov v16.s[1], w10 + ldr q0, [x13, :lo12:.LCPI0_0] + ldr q20, [x11, :lo12:.LCPI0_1] + adrp x11, .LCPI0_4 + and w9, w4, #0xff + ldr q2, [x11, :lo12:.LCPI0_4] + mov v16.s[2], w8 + uzp1 v21.4s, v17.4s, v18.4s + add v7.4s, v6.4s, v7.4s + adrp x12, .LCPI0_3 + mov v16.s[3], w9 + uzp2 v18.4s, v17.4s, v18.4s + add v7.4s, v7.4s, v21.4s + ext v17.16b, v5.16b, v5.16b, #12 + ldr q3, [x12, :lo12:.LCPI0_3] + ext v24.16b, v4.16b, v4.16b, #12 + eor v16.16b, v7.16b, v16.16b + mov v27.16b, v17.16b + uzp1 v19.4s, v21.4s, v21.4s + ext v25.16b, v21.16b, v21.16b, #12 + zip2 v28.4s, v18.4s, v17.4s + tbl v29.16b, { v16.16b }, v0.16b + mov v27.s[1], v24.s[2] + zip1 v23.2d, v17.2d, v18.2d + ext v19.16b, v19.16b, v21.16b, #8 + add v22.4s, v29.4s, v20.4s + ext v26.16b, v21.16b, v25.16b, #12 + tbl v20.16b, { v23.16b, v24.16b }, v2.16b + zip1 v21.4s, v28.4s, v24.4s + zip1 v23.4s, v24.4s, v28.4s + uzp2 v19.4s, v19.4s, v18.4s + eor v24.16b, v22.16b, v6.16b + ext v25.16b, v20.16b, v20.16b, #12 + ext v6.16b, v23.16b, v21.16b, #8 + add v7.4s, v7.4s, v18.4s + ext v18.16b, v19.16b, v19.16b, #4 + tbl v16.16b, { v26.16b, v27.16b }, v3.16b + uzp1 v21.4s, v20.4s, v25.4s + mov v26.16b, v6.16b + ext v23.16b, v18.16b, v18.16b, #12 + mov v26.s[1], v21.s[2] + adrp x10, .LCPI0_2 + ext v25.16b, v18.16b, v23.16b, #12 + uzp1 v23.4s, v18.4s, v18.4s + ldr q1, [x10, :lo12:.LCPI0_2] + ext v18.16b, v23.16b, v18.16b, #8 + ushr v23.4s, v24.4s, #12 + shl v24.4s, v24.4s, #20 + orr v23.16b, v24.16b, v23.16b + add v7.4s, v7.4s, v23.4s + eor v27.16b, v29.16b, v7.16b + add v4.4s, v7.4s, v4.4s + tbl v7.16b, { v25.16b, v26.16b }, v3.16b + tbl v26.16b, { v27.16b }, v1.16b + add v22.4s, v22.4s, v26.4s + uzp2 v18.4s, v18.4s, v16.4s + eor v23.16b, v23.16b, v22.16b + ext v5.16b, v18.16b, v18.16b, #4 + ushr v27.4s, v23.4s, #7 + shl v23.4s, v23.4s, #25 + uzp1 v25.4s, v5.4s, v5.4s + orr v23.16b, v23.16b, v27.16b + ext v28.16b, v4.16b, v4.16b, #12 + ext v4.16b, v25.16b, v5.16b, #8 + ext v25.16b, v26.16b, v26.16b, #8 + add v26.4s, v28.4s, v23.4s + eor v25.16b, v26.16b, v25.16b + ext v22.16b, v22.16b, v22.16b, #4 + tbl v25.16b, { v25.16b }, v0.16b + add v22.4s, v22.4s, v25.4s + eor v23.16b, v23.16b, v22.16b + add v17.4s, v26.4s, v17.4s + ushr v26.4s, v23.4s, #12 + shl v23.4s, v23.4s, #20 + orr v23.16b, v23.16b, v26.16b + add v17.4s, v17.4s, v23.4s + eor v25.16b, v25.16b, v17.16b + add v17.4s, v17.4s, v19.4s + tbl v19.16b, { v25.16b }, v1.16b + add v22.4s, v22.4s, v19.4s + eor v23.16b, v23.16b, v22.16b + ushr v25.4s, v23.4s, #7 + shl v23.4s, v23.4s, #25 + ext v17.16b, v17.16b, v17.16b, #4 + orr v23.16b, v23.16b, v25.16b + ext v19.16b, v19.16b, v19.16b, #8 + add v17.4s, v17.4s, v23.4s + eor v19.16b, v17.16b, v19.16b + ext v22.16b, v22.16b, v22.16b, #12 + tbl v19.16b, { v19.16b }, v0.16b + add v22.4s, v22.4s, v19.4s + eor v23.16b, v23.16b, v22.16b + ushr v25.4s, v23.4s, #12 + shl v23.4s, v23.4s, #20 + add v17.4s, v17.4s, v16.4s + orr v23.16b, v23.16b, v25.16b + add v17.4s, v17.4s, v23.4s + ext v25.16b, v17.16b, v17.16b, #12 + eor v17.16b, v19.16b, v17.16b + tbl v17.16b, { v17.16b }, v1.16b + add v19.4s, v22.4s, v17.4s + eor v22.16b, v23.16b, v19.16b + add v25.4s, v25.4s, v21.4s + zip1 v20.2d, v6.2d, v16.2d + ushr v23.4s, v22.4s, #7 + shl v22.4s, v22.4s, #25 + zip2 v24.4s, v16.4s, v6.4s + tbl v26.16b, { v20.16b, v21.16b }, v2.16b + orr v22.16b, v22.16b, v23.16b + zip1 v16.4s, v24.4s, v21.4s + zip1 v20.4s, v21.4s, v24.4s + ext v21.16b, v26.16b, v26.16b, #12 + ext v17.16b, v17.16b, v17.16b, #8 + add v25.4s, v25.4s, v22.4s + ext v16.16b, v20.16b, v16.16b, #8 + uzp1 v21.4s, v26.4s, v21.4s + eor v26.16b, v25.16b, v17.16b + ext v19.16b, v19.16b, v19.16b, #4 + tbl v26.16b, { v26.16b }, v0.16b + mov v29.16b, v16.16b + add v19.4s, v19.4s, v26.4s + ext v27.16b, v5.16b, v5.16b, #12 + mov v29.s[1], v21.s[2] + eor v22.16b, v22.16b, v19.16b + ext v28.16b, v5.16b, v27.16b, #12 + ushr v27.4s, v22.4s, #12 + shl v22.4s, v22.4s, #20 + add v6.4s, v25.4s, v6.4s + orr v22.16b, v22.16b, v27.16b + add v6.4s, v6.4s, v22.4s + eor v26.16b, v26.16b, v6.16b + add v6.4s, v6.4s, v18.4s + tbl v18.16b, { v26.16b }, v1.16b + add v19.4s, v19.4s, v18.4s + eor v22.16b, v22.16b, v19.16b + ushr v26.4s, v22.4s, #7 + shl v22.4s, v22.4s, #25 + ext v6.16b, v6.16b, v6.16b, #4 + orr v22.16b, v22.16b, v26.16b + ext v18.16b, v18.16b, v18.16b, #8 + add v6.4s, v6.4s, v22.4s + eor v18.16b, v6.16b, v18.16b + ext v19.16b, v19.16b, v19.16b, #12 + tbl v18.16b, { v18.16b }, v0.16b + add v19.4s, v19.4s, v18.4s + eor v22.16b, v22.16b, v19.16b + ushr v26.4s, v22.4s, #12 + shl v22.4s, v22.4s, #20 + add v6.4s, v6.4s, v7.4s + orr v22.16b, v22.16b, v26.16b + add v6.4s, v6.4s, v22.4s + ext v26.16b, v6.16b, v6.16b, #12 + eor v6.16b, v18.16b, v6.16b + uzp2 v4.4s, v4.4s, v7.4s + zip2 v25.4s, v7.4s, v16.4s + add v26.4s, v26.4s, v21.4s + zip1 v20.2d, v16.2d, v7.2d + tbl v6.16b, { v6.16b }, v1.16b + ext v24.16b, v4.16b, v4.16b, #4 + tbl v27.16b, { v20.16b, v21.16b }, v2.16b + zip1 v7.4s, v25.4s, v21.4s + zip1 v20.4s, v21.4s, v25.4s + add v18.4s, v19.4s, v6.4s + uzp1 v5.4s, v24.4s, v24.4s + ext v21.16b, v27.16b, v27.16b, #12 + ext v7.16b, v20.16b, v7.16b, #8 + eor v19.16b, v22.16b, v18.16b + ext v5.16b, v5.16b, v24.16b, #8 + tbl v17.16b, { v28.16b, v29.16b }, v3.16b + uzp1 v21.4s, v27.4s, v21.4s + mov v28.16b, v7.16b + ushr v22.4s, v19.4s, #7 + shl v19.4s, v19.4s, #25 + ext v23.16b, v24.16b, v24.16b, #12 + uzp2 v5.4s, v5.4s, v17.4s + mov v28.s[1], v21.s[2] + orr v19.16b, v19.16b, v22.16b + ext v27.16b, v24.16b, v23.16b, #12 + ext v23.16b, v5.16b, v5.16b, #4 + ext v6.16b, v6.16b, v6.16b, #8 + ext v25.16b, v18.16b, v18.16b, #4 + add v18.4s, v26.4s, v19.4s + uzp1 v24.4s, v23.4s, v23.4s + eor v6.16b, v18.16b, v6.16b + ext v24.16b, v24.16b, v23.16b, #8 + add v16.4s, v18.4s, v16.4s + tbl v18.16b, { v27.16b, v28.16b }, v3.16b + tbl v27.16b, { v6.16b }, v0.16b + uzp2 v6.4s, v24.4s, v18.4s + add v24.4s, v25.4s, v27.4s + eor v19.16b, v19.16b, v24.16b + ushr v25.4s, v19.4s, #12 + shl v19.4s, v19.4s, #20 + orr v19.16b, v19.16b, v25.16b + add v16.4s, v16.4s, v19.4s + eor v25.16b, v27.16b, v16.16b + add v4.4s, v16.4s, v4.4s + tbl v16.16b, { v25.16b }, v1.16b + add v24.4s, v24.4s, v16.4s + eor v19.16b, v19.16b, v24.16b + ushr v25.4s, v19.4s, #7 + shl v19.4s, v19.4s, #25 + ext v4.16b, v4.16b, v4.16b, #4 + orr v19.16b, v19.16b, v25.16b + ext v16.16b, v16.16b, v16.16b, #8 + add v4.4s, v4.4s, v19.4s + eor v16.16b, v4.16b, v16.16b + ext v24.16b, v24.16b, v24.16b, #12 + tbl v25.16b, { v16.16b }, v0.16b + add v24.4s, v24.4s, v25.4s + eor v16.16b, v19.16b, v24.16b + ushr v19.4s, v16.4s, #12 + shl v16.4s, v16.4s, #20 + add v4.4s, v4.4s, v17.4s + orr v19.16b, v16.16b, v19.16b + add v27.4s, v4.4s, v19.4s + eor v25.16b, v25.16b, v27.16b + tbl v25.16b, { v25.16b }, v1.16b + add v24.4s, v24.4s, v25.4s + zip2 v26.4s, v17.4s, v7.4s + ext v4.16b, v27.16b, v27.16b, #12 + eor v19.16b, v19.16b, v24.16b + add v28.4s, v4.4s, v21.4s + zip1 v20.2d, v7.2d, v17.2d + zip1 v4.4s, v26.4s, v21.4s + zip1 v17.4s, v21.4s, v26.4s + ushr v26.4s, v19.4s, #7 + shl v19.4s, v19.4s, #25 + orr v19.16b, v19.16b, v26.16b + ext v25.16b, v25.16b, v25.16b, #8 + add v27.4s, v28.4s, v19.4s + eor v25.16b, v27.16b, v25.16b + ext v24.16b, v24.16b, v24.16b, #4 + tbl v25.16b, { v25.16b }, v0.16b + add v24.4s, v24.4s, v25.4s + eor v19.16b, v19.16b, v24.16b + add v7.4s, v27.4s, v7.4s + ushr v27.4s, v19.4s, #12 + shl v19.4s, v19.4s, #20 + orr v19.16b, v19.16b, v27.16b + add v7.4s, v7.4s, v19.4s + eor v25.16b, v25.16b, v7.16b + add v5.4s, v7.4s, v5.4s + tbl v7.16b, { v25.16b }, v1.16b + add v24.4s, v24.4s, v7.4s + eor v19.16b, v19.16b, v24.16b + ushr v25.4s, v19.4s, #7 + shl v19.4s, v19.4s, #25 + ext v5.16b, v5.16b, v5.16b, #4 + orr v19.16b, v19.16b, v25.16b + ext v7.16b, v7.16b, v7.16b, #8 + add v5.4s, v5.4s, v19.4s + eor v7.16b, v5.16b, v7.16b + ext v24.16b, v24.16b, v24.16b, #12 + tbl v7.16b, { v7.16b }, v0.16b + add v24.4s, v24.4s, v7.4s + eor v19.16b, v19.16b, v24.16b + ushr v25.4s, v19.4s, #12 + shl v19.4s, v19.4s, #20 + tbl v16.16b, { v20.16b, v21.16b }, v2.16b + add v5.4s, v5.4s, v18.4s + orr v19.16b, v19.16b, v25.16b + ext v20.16b, v16.16b, v16.16b, #12 + ext v4.16b, v17.16b, v4.16b, #8 + add v5.4s, v5.4s, v19.4s + uzp1 v21.4s, v16.4s, v20.4s + mov v17.16b, v4.16b + ext v25.16b, v5.16b, v5.16b, #12 + mov v17.s[1], v21.s[2] + add v25.4s, v25.4s, v21.4s + zip1 v20.2d, v4.2d, v18.2d + ext v22.16b, v23.16b, v23.16b, #12 + zip2 v26.4s, v18.4s, v4.4s + tbl v18.16b, { v20.16b, v21.16b }, v2.16b + eor v5.16b, v7.16b, v5.16b + ext v16.16b, v23.16b, v22.16b, #12 + ext v22.16b, v6.16b, v6.16b, #4 + zip1 v27.4s, v26.4s, v21.4s + zip1 v20.4s, v21.4s, v26.4s + ext v21.16b, v18.16b, v18.16b, #12 + tbl v5.16b, { v5.16b }, v1.16b + ext v20.16b, v20.16b, v27.16b, #8 + uzp1 v27.4s, v18.4s, v21.4s + uzp1 v18.4s, v22.4s, v22.4s + add v21.4s, v24.4s, v5.4s + ext v18.16b, v18.16b, v22.16b, #8 + eor v19.16b, v19.16b, v21.16b + tbl v7.16b, { v16.16b, v17.16b }, v3.16b + uzp2 v18.4s, v18.4s, v17.4s + zip2 v16.4s, v16.4s, v20.4s + ushr v17.4s, v19.4s, #7 + shl v19.4s, v19.4s, #25 + orr v17.16b, v19.16b, v17.16b + ext v5.16b, v5.16b, v5.16b, #8 + add v19.4s, v25.4s, v17.4s + eor v5.16b, v19.16b, v5.16b + ext v21.16b, v21.16b, v21.16b, #4 + tbl v5.16b, { v5.16b }, v0.16b + add v4.4s, v19.4s, v4.4s + add v19.4s, v21.4s, v5.4s + eor v17.16b, v17.16b, v19.16b + ushr v21.4s, v17.4s, #12 + shl v17.4s, v17.4s, #20 + orr v17.16b, v17.16b, v21.16b + add v4.4s, v4.4s, v17.4s + eor v5.16b, v5.16b, v4.16b + tbl v5.16b, { v5.16b }, v1.16b + add v4.4s, v4.4s, v6.4s + add v6.4s, v19.4s, v5.4s + eor v17.16b, v17.16b, v6.16b + ushr v19.4s, v17.4s, #7 + shl v17.4s, v17.4s, #25 + ext v4.16b, v4.16b, v4.16b, #4 + orr v17.16b, v17.16b, v19.16b + ext v5.16b, v5.16b, v5.16b, #8 + add v4.4s, v4.4s, v17.4s + eor v5.16b, v4.16b, v5.16b + ext v6.16b, v6.16b, v6.16b, #12 + tbl v5.16b, { v5.16b }, v0.16b + add v6.4s, v6.4s, v5.4s + eor v17.16b, v17.16b, v6.16b + ushr v19.4s, v17.4s, #12 + shl v17.4s, v17.4s, #20 + add v4.4s, v4.4s, v7.4s + orr v17.16b, v17.16b, v19.16b + add v4.4s, v4.4s, v17.4s + eor v5.16b, v5.16b, v4.16b + tbl v5.16b, { v5.16b }, v1.16b + mov v29.16b, v20.16b + ext v4.16b, v4.16b, v4.16b, #12 + add v6.4s, v6.4s, v5.4s + mov v29.s[1], v27.s[2] + add v4.4s, v4.4s, v27.4s + zip1 v26.2d, v20.2d, v7.2d + zip1 v7.4s, v16.4s, v27.4s + zip1 v16.4s, v27.4s, v16.4s + eor v17.16b, v17.16b, v6.16b + ext v7.16b, v16.16b, v7.16b, #8 + ushr v16.4s, v17.4s, #7 + shl v17.4s, v17.4s, #25 + orr v16.16b, v17.16b, v16.16b + ext v5.16b, v5.16b, v5.16b, #8 + add v4.4s, v4.4s, v16.4s + eor v5.16b, v4.16b, v5.16b + ext v6.16b, v6.16b, v6.16b, #4 + tbl v5.16b, { v5.16b }, v0.16b + add v6.4s, v6.4s, v5.4s + eor v16.16b, v16.16b, v6.16b + ushr v17.4s, v16.4s, #12 + shl v16.4s, v16.4s, #20 + add v4.4s, v4.4s, v20.4s + orr v16.16b, v16.16b, v17.16b + add v4.4s, v4.4s, v16.4s + eor v5.16b, v5.16b, v4.16b + tbl v5.16b, { v5.16b }, v1.16b + add v6.4s, v6.4s, v5.4s + eor v16.16b, v16.16b, v6.16b + add v4.4s, v4.4s, v18.4s + ushr v17.4s, v16.4s, #7 + shl v16.4s, v16.4s, #25 + ext v23.16b, v22.16b, v22.16b, #12 + ext v4.16b, v4.16b, v4.16b, #4 + orr v16.16b, v16.16b, v17.16b + ext v28.16b, v22.16b, v23.16b, #12 + ext v5.16b, v5.16b, v5.16b, #8 + add v4.4s, v16.4s, v4.4s + tbl v3.16b, { v28.16b, v29.16b }, v3.16b + eor v5.16b, v4.16b, v5.16b + ext v6.16b, v6.16b, v6.16b, #12 + add v3.4s, v4.4s, v3.4s + tbl v4.16b, { v5.16b }, v0.16b + add v5.4s, v6.4s, v4.4s + eor v6.16b, v16.16b, v5.16b + ushr v16.4s, v6.4s, #12 + shl v6.4s, v6.4s, #20 + orr v6.16b, v6.16b, v16.16b + tbl v2.16b, { v26.16b, v27.16b }, v2.16b + add v3.4s, v3.4s, v6.4s + ext v19.16b, v2.16b, v2.16b, #12 + eor v4.16b, v4.16b, v3.16b + uzp1 v2.4s, v2.4s, v19.4s + ext v3.16b, v3.16b, v3.16b, #12 + tbl v4.16b, { v4.16b }, v1.16b + add v2.4s, v3.4s, v2.4s + add v3.4s, v5.4s, v4.4s + eor v5.16b, v6.16b, v3.16b + ushr v6.4s, v5.4s, #7 + shl v5.4s, v5.4s, #25 + orr v5.16b, v5.16b, v6.16b + ext v4.16b, v4.16b, v4.16b, #8 + add v2.4s, v2.4s, v5.4s + eor v4.16b, v2.16b, v4.16b + ext v3.16b, v3.16b, v3.16b, #4 + tbl v0.16b, { v4.16b }, v0.16b + add v3.4s, v3.4s, v0.4s + eor v4.16b, v5.16b, v3.16b + ushr v5.4s, v4.4s, #12 + shl v4.4s, v4.4s, #20 + add v2.4s, v2.4s, v7.4s + orr v4.16b, v4.16b, v5.16b + add v2.4s, v2.4s, v4.4s + eor v0.16b, v0.16b, v2.16b + tbl v0.16b, { v0.16b }, v1.16b + add v1.4s, v3.4s, v0.4s + eor v3.16b, v4.16b, v1.16b + ext v2.16b, v2.16b, v2.16b, #4 + ext v1.16b, v1.16b, v1.16b, #12 + ushr v4.4s, v3.4s, #7 + shl v3.4s, v3.4s, #25 + ext v0.16b, v0.16b, v0.16b, #8 + eor v1.16b, v2.16b, v1.16b + orr v2.16b, v3.16b, v4.16b + eor v0.16b, v2.16b, v0.16b + stp q1, q0, [x0] + ret +.Lfunc_end0: + .size zfs_blake3_compress_in_place_sse41, .Lfunc_end0-zfs_blake3_compress_in_place_sse41 + .cfi_endproc + + .section .rodata.cst16,"aM",@progbits,16 + .p2align 4 +.LCPI1_0: + .byte 2 + .byte 3 + .byte 0 + .byte 1 + .byte 6 + .byte 7 + .byte 4 + .byte 5 + .byte 10 + .byte 11 + .byte 8 + .byte 9 + .byte 14 + .byte 15 + .byte 12 + .byte 13 +.LCPI1_1: + .word 1779033703 + .word 3144134277 + .word 1013904242 + .word 2773480762 +.LCPI1_2: + .byte 1 + .byte 2 + .byte 3 + .byte 0 + .byte 5 + .byte 6 + .byte 7 + .byte 4 + .byte 9 + .byte 10 + .byte 11 + .byte 8 + .byte 13 + .byte 14 + .byte 15 + .byte 12 +.LCPI1_3: + .byte 0 + .byte 1 + .byte 2 + .byte 3 + .byte 20 + .byte 21 + .byte 22 + .byte 23 + .byte 8 + .byte 9 + .byte 10 + .byte 11 + .byte 28 + .byte 29 + .byte 30 + .byte 31 +.LCPI1_4: + .byte 0 + .byte 1 + .byte 2 + .byte 3 + .byte 4 + .byte 5 + .byte 6 + .byte 7 + .byte 8 + .byte 9 + .byte 10 + .byte 11 + .byte 28 + .byte 29 + .byte 30 + .byte 31 + .text + .globl zfs_blake3_compress_xof_sse41 + .p2align 2 + .type zfs_blake3_compress_xof_sse41,@function +zfs_blake3_compress_xof_sse41: + .cfi_startproc + ldp q7, q6, [x0] + ldp q17, q18, [x1] + add x12, x1, #32 + ld2 { v4.4s, v5.4s }, [x12] + lsr x10, x3, #32 + fmov s16, w3 + adrp x13, .LCPI1_0 + adrp x11, .LCPI1_1 + and w8, w2, #0xff + mov v16.s[1], w10 + ldr q0, [x13, :lo12:.LCPI1_0] + ldr q20, [x11, :lo12:.LCPI1_1] + adrp x11, .LCPI1_4 + and w9, w4, #0xff + ldr q2, [x11, :lo12:.LCPI1_4] + mov v16.s[2], w8 + uzp1 v21.4s, v17.4s, v18.4s + add v7.4s, v6.4s, v7.4s + adrp x12, .LCPI1_3 + mov v16.s[3], w9 + uzp2 v18.4s, v17.4s, v18.4s + add v7.4s, v7.4s, v21.4s + ext v17.16b, v5.16b, v5.16b, #12 + ldr q3, [x12, :lo12:.LCPI1_3] + ext v24.16b, v4.16b, v4.16b, #12 + eor v16.16b, v7.16b, v16.16b + mov v27.16b, v17.16b + uzp1 v19.4s, v21.4s, v21.4s + ext v25.16b, v21.16b, v21.16b, #12 + zip2 v28.4s, v18.4s, v17.4s + tbl v29.16b, { v16.16b }, v0.16b + mov v27.s[1], v24.s[2] + zip1 v23.2d, v17.2d, v18.2d + ext v19.16b, v19.16b, v21.16b, #8 + add v22.4s, v29.4s, v20.4s + ext v26.16b, v21.16b, v25.16b, #12 + tbl v20.16b, { v23.16b, v24.16b }, v2.16b + zip1 v21.4s, v28.4s, v24.4s + zip1 v23.4s, v24.4s, v28.4s + uzp2 v19.4s, v19.4s, v18.4s + eor v24.16b, v22.16b, v6.16b + ext v25.16b, v20.16b, v20.16b, #12 + ext v6.16b, v23.16b, v21.16b, #8 + add v7.4s, v7.4s, v18.4s + ext v18.16b, v19.16b, v19.16b, #4 + tbl v16.16b, { v26.16b, v27.16b }, v3.16b + uzp1 v21.4s, v20.4s, v25.4s + mov v26.16b, v6.16b + ext v23.16b, v18.16b, v18.16b, #12 + mov v26.s[1], v21.s[2] + adrp x10, .LCPI1_2 + ext v25.16b, v18.16b, v23.16b, #12 + uzp1 v23.4s, v18.4s, v18.4s + ldr q1, [x10, :lo12:.LCPI1_2] + ext v18.16b, v23.16b, v18.16b, #8 + ushr v23.4s, v24.4s, #12 + shl v24.4s, v24.4s, #20 + orr v23.16b, v24.16b, v23.16b + add v7.4s, v7.4s, v23.4s + eor v27.16b, v29.16b, v7.16b + add v4.4s, v7.4s, v4.4s + tbl v7.16b, { v25.16b, v26.16b }, v3.16b + tbl v26.16b, { v27.16b }, v1.16b + add v22.4s, v22.4s, v26.4s + uzp2 v18.4s, v18.4s, v16.4s + eor v23.16b, v23.16b, v22.16b + ext v5.16b, v18.16b, v18.16b, #4 + ushr v27.4s, v23.4s, #7 + shl v23.4s, v23.4s, #25 + uzp1 v25.4s, v5.4s, v5.4s + orr v23.16b, v23.16b, v27.16b + ext v28.16b, v4.16b, v4.16b, #12 + ext v4.16b, v25.16b, v5.16b, #8 + ext v25.16b, v26.16b, v26.16b, #8 + add v26.4s, v28.4s, v23.4s + eor v25.16b, v26.16b, v25.16b + ext v22.16b, v22.16b, v22.16b, #4 + tbl v25.16b, { v25.16b }, v0.16b + add v22.4s, v22.4s, v25.4s + eor v23.16b, v23.16b, v22.16b + add v17.4s, v26.4s, v17.4s + ushr v26.4s, v23.4s, #12 + shl v23.4s, v23.4s, #20 + orr v23.16b, v23.16b, v26.16b + add v17.4s, v17.4s, v23.4s + eor v25.16b, v25.16b, v17.16b + add v17.4s, v17.4s, v19.4s + tbl v19.16b, { v25.16b }, v1.16b + add v22.4s, v22.4s, v19.4s + eor v23.16b, v23.16b, v22.16b + ushr v25.4s, v23.4s, #7 + shl v23.4s, v23.4s, #25 + ext v17.16b, v17.16b, v17.16b, #4 + orr v23.16b, v23.16b, v25.16b + ext v19.16b, v19.16b, v19.16b, #8 + add v17.4s, v17.4s, v23.4s + eor v19.16b, v17.16b, v19.16b + ext v22.16b, v22.16b, v22.16b, #12 + tbl v19.16b, { v19.16b }, v0.16b + add v22.4s, v22.4s, v19.4s + eor v23.16b, v23.16b, v22.16b + ushr v25.4s, v23.4s, #12 + shl v23.4s, v23.4s, #20 + add v17.4s, v17.4s, v16.4s + orr v23.16b, v23.16b, v25.16b + add v17.4s, v17.4s, v23.4s + ext v25.16b, v17.16b, v17.16b, #12 + eor v17.16b, v19.16b, v17.16b + tbl v17.16b, { v17.16b }, v1.16b + add v19.4s, v22.4s, v17.4s + eor v22.16b, v23.16b, v19.16b + add v25.4s, v25.4s, v21.4s + zip1 v20.2d, v6.2d, v16.2d + ushr v23.4s, v22.4s, #7 + shl v22.4s, v22.4s, #25 + zip2 v24.4s, v16.4s, v6.4s + tbl v26.16b, { v20.16b, v21.16b }, v2.16b + orr v22.16b, v22.16b, v23.16b + zip1 v16.4s, v24.4s, v21.4s + zip1 v20.4s, v21.4s, v24.4s + ext v21.16b, v26.16b, v26.16b, #12 + ext v17.16b, v17.16b, v17.16b, #8 + add v25.4s, v25.4s, v22.4s + ext v16.16b, v20.16b, v16.16b, #8 + uzp1 v21.4s, v26.4s, v21.4s + eor v26.16b, v25.16b, v17.16b + ext v19.16b, v19.16b, v19.16b, #4 + tbl v26.16b, { v26.16b }, v0.16b + mov v29.16b, v16.16b + add v19.4s, v19.4s, v26.4s + ext v27.16b, v5.16b, v5.16b, #12 + mov v29.s[1], v21.s[2] + eor v22.16b, v22.16b, v19.16b + ext v28.16b, v5.16b, v27.16b, #12 + ushr v27.4s, v22.4s, #12 + shl v22.4s, v22.4s, #20 + add v6.4s, v25.4s, v6.4s + orr v22.16b, v22.16b, v27.16b + add v6.4s, v6.4s, v22.4s + eor v26.16b, v26.16b, v6.16b + add v6.4s, v6.4s, v18.4s + tbl v18.16b, { v26.16b }, v1.16b + add v19.4s, v19.4s, v18.4s + eor v22.16b, v22.16b, v19.16b + ushr v26.4s, v22.4s, #7 + shl v22.4s, v22.4s, #25 + ext v6.16b, v6.16b, v6.16b, #4 + orr v22.16b, v22.16b, v26.16b + ext v18.16b, v18.16b, v18.16b, #8 + add v6.4s, v6.4s, v22.4s + eor v18.16b, v6.16b, v18.16b + ext v19.16b, v19.16b, v19.16b, #12 + tbl v18.16b, { v18.16b }, v0.16b + add v19.4s, v19.4s, v18.4s + eor v22.16b, v22.16b, v19.16b + ushr v26.4s, v22.4s, #12 + shl v22.4s, v22.4s, #20 + add v6.4s, v6.4s, v7.4s + orr v22.16b, v22.16b, v26.16b + add v6.4s, v6.4s, v22.4s + ext v26.16b, v6.16b, v6.16b, #12 + eor v6.16b, v18.16b, v6.16b + uzp2 v4.4s, v4.4s, v7.4s + zip2 v25.4s, v7.4s, v16.4s + add v26.4s, v26.4s, v21.4s + zip1 v20.2d, v16.2d, v7.2d + tbl v6.16b, { v6.16b }, v1.16b + ext v24.16b, v4.16b, v4.16b, #4 + tbl v27.16b, { v20.16b, v21.16b }, v2.16b + zip1 v7.4s, v25.4s, v21.4s + zip1 v20.4s, v21.4s, v25.4s + add v18.4s, v19.4s, v6.4s + uzp1 v5.4s, v24.4s, v24.4s + ext v21.16b, v27.16b, v27.16b, #12 + ext v7.16b, v20.16b, v7.16b, #8 + eor v19.16b, v22.16b, v18.16b + ext v5.16b, v5.16b, v24.16b, #8 + tbl v17.16b, { v28.16b, v29.16b }, v3.16b + uzp1 v21.4s, v27.4s, v21.4s + mov v28.16b, v7.16b + ushr v22.4s, v19.4s, #7 + shl v19.4s, v19.4s, #25 + ext v23.16b, v24.16b, v24.16b, #12 + uzp2 v5.4s, v5.4s, v17.4s + mov v28.s[1], v21.s[2] + orr v19.16b, v19.16b, v22.16b + ext v27.16b, v24.16b, v23.16b, #12 + ext v23.16b, v5.16b, v5.16b, #4 + ext v6.16b, v6.16b, v6.16b, #8 + ext v25.16b, v18.16b, v18.16b, #4 + add v18.4s, v26.4s, v19.4s + uzp1 v24.4s, v23.4s, v23.4s + eor v6.16b, v18.16b, v6.16b + ext v24.16b, v24.16b, v23.16b, #8 + add v16.4s, v18.4s, v16.4s + tbl v18.16b, { v27.16b, v28.16b }, v3.16b + tbl v27.16b, { v6.16b }, v0.16b + uzp2 v6.4s, v24.4s, v18.4s + add v24.4s, v25.4s, v27.4s + eor v19.16b, v19.16b, v24.16b + ushr v25.4s, v19.4s, #12 + shl v19.4s, v19.4s, #20 + orr v19.16b, v19.16b, v25.16b + add v16.4s, v16.4s, v19.4s + eor v25.16b, v27.16b, v16.16b + add v4.4s, v16.4s, v4.4s + tbl v16.16b, { v25.16b }, v1.16b + add v24.4s, v24.4s, v16.4s + eor v19.16b, v19.16b, v24.16b + ushr v25.4s, v19.4s, #7 + shl v19.4s, v19.4s, #25 + ext v4.16b, v4.16b, v4.16b, #4 + orr v19.16b, v19.16b, v25.16b + ext v16.16b, v16.16b, v16.16b, #8 + add v4.4s, v4.4s, v19.4s + eor v16.16b, v4.16b, v16.16b + ext v24.16b, v24.16b, v24.16b, #12 + tbl v25.16b, { v16.16b }, v0.16b + add v24.4s, v24.4s, v25.4s + eor v16.16b, v19.16b, v24.16b + ushr v19.4s, v16.4s, #12 + shl v16.4s, v16.4s, #20 + add v4.4s, v4.4s, v17.4s + orr v19.16b, v16.16b, v19.16b + add v27.4s, v4.4s, v19.4s + eor v25.16b, v25.16b, v27.16b + tbl v25.16b, { v25.16b }, v1.16b + add v24.4s, v24.4s, v25.4s + zip2 v26.4s, v17.4s, v7.4s + ext v4.16b, v27.16b, v27.16b, #12 + eor v19.16b, v19.16b, v24.16b + add v28.4s, v4.4s, v21.4s + zip1 v20.2d, v7.2d, v17.2d + zip1 v4.4s, v26.4s, v21.4s + zip1 v17.4s, v21.4s, v26.4s + ushr v26.4s, v19.4s, #7 + shl v19.4s, v19.4s, #25 + orr v19.16b, v19.16b, v26.16b + ext v25.16b, v25.16b, v25.16b, #8 + add v27.4s, v28.4s, v19.4s + eor v25.16b, v27.16b, v25.16b + ext v24.16b, v24.16b, v24.16b, #4 + tbl v25.16b, { v25.16b }, v0.16b + add v24.4s, v24.4s, v25.4s + eor v19.16b, v19.16b, v24.16b + add v7.4s, v27.4s, v7.4s + ushr v27.4s, v19.4s, #12 + shl v19.4s, v19.4s, #20 + orr v19.16b, v19.16b, v27.16b + add v7.4s, v7.4s, v19.4s + eor v25.16b, v25.16b, v7.16b + add v5.4s, v7.4s, v5.4s + tbl v7.16b, { v25.16b }, v1.16b + add v24.4s, v24.4s, v7.4s + eor v19.16b, v19.16b, v24.16b + ushr v25.4s, v19.4s, #7 + shl v19.4s, v19.4s, #25 + ext v5.16b, v5.16b, v5.16b, #4 + orr v19.16b, v19.16b, v25.16b + ext v7.16b, v7.16b, v7.16b, #8 + add v5.4s, v5.4s, v19.4s + eor v7.16b, v5.16b, v7.16b + ext v24.16b, v24.16b, v24.16b, #12 + tbl v7.16b, { v7.16b }, v0.16b + add v24.4s, v24.4s, v7.4s + eor v19.16b, v19.16b, v24.16b + ushr v25.4s, v19.4s, #12 + shl v19.4s, v19.4s, #20 + tbl v16.16b, { v20.16b, v21.16b }, v2.16b + add v5.4s, v5.4s, v18.4s + orr v19.16b, v19.16b, v25.16b + ext v20.16b, v16.16b, v16.16b, #12 + ext v4.16b, v17.16b, v4.16b, #8 + add v5.4s, v5.4s, v19.4s + uzp1 v21.4s, v16.4s, v20.4s + mov v17.16b, v4.16b + ext v25.16b, v5.16b, v5.16b, #12 + mov v17.s[1], v21.s[2] + add v25.4s, v25.4s, v21.4s + zip1 v20.2d, v4.2d, v18.2d + ext v22.16b, v23.16b, v23.16b, #12 + zip2 v26.4s, v18.4s, v4.4s + tbl v18.16b, { v20.16b, v21.16b }, v2.16b + eor v5.16b, v7.16b, v5.16b + ext v16.16b, v23.16b, v22.16b, #12 + ext v22.16b, v6.16b, v6.16b, #4 + zip1 v27.4s, v26.4s, v21.4s + zip1 v20.4s, v21.4s, v26.4s + ext v21.16b, v18.16b, v18.16b, #12 + tbl v5.16b, { v5.16b }, v1.16b + ext v20.16b, v20.16b, v27.16b, #8 + uzp1 v27.4s, v18.4s, v21.4s + uzp1 v18.4s, v22.4s, v22.4s + add v21.4s, v24.4s, v5.4s + ext v18.16b, v18.16b, v22.16b, #8 + eor v19.16b, v19.16b, v21.16b + tbl v7.16b, { v16.16b, v17.16b }, v3.16b + uzp2 v18.4s, v18.4s, v17.4s + zip2 v16.4s, v16.4s, v20.4s + ushr v17.4s, v19.4s, #7 + shl v19.4s, v19.4s, #25 + orr v17.16b, v19.16b, v17.16b + ext v5.16b, v5.16b, v5.16b, #8 + add v19.4s, v25.4s, v17.4s + eor v5.16b, v19.16b, v5.16b + ext v21.16b, v21.16b, v21.16b, #4 + tbl v5.16b, { v5.16b }, v0.16b + add v4.4s, v19.4s, v4.4s + add v19.4s, v21.4s, v5.4s + eor v17.16b, v17.16b, v19.16b + ushr v21.4s, v17.4s, #12 + shl v17.4s, v17.4s, #20 + orr v17.16b, v17.16b, v21.16b + add v4.4s, v4.4s, v17.4s + eor v5.16b, v5.16b, v4.16b + tbl v5.16b, { v5.16b }, v1.16b + add v4.4s, v4.4s, v6.4s + add v6.4s, v19.4s, v5.4s + eor v17.16b, v17.16b, v6.16b + ushr v19.4s, v17.4s, #7 + shl v17.4s, v17.4s, #25 + ext v4.16b, v4.16b, v4.16b, #4 + orr v17.16b, v17.16b, v19.16b + ext v5.16b, v5.16b, v5.16b, #8 + add v4.4s, v4.4s, v17.4s + eor v5.16b, v4.16b, v5.16b + ext v6.16b, v6.16b, v6.16b, #12 + tbl v5.16b, { v5.16b }, v0.16b + add v6.4s, v6.4s, v5.4s + eor v17.16b, v17.16b, v6.16b + ushr v19.4s, v17.4s, #12 + shl v17.4s, v17.4s, #20 + add v4.4s, v4.4s, v7.4s + orr v17.16b, v17.16b, v19.16b + add v4.4s, v4.4s, v17.4s + eor v5.16b, v5.16b, v4.16b + tbl v5.16b, { v5.16b }, v1.16b + mov v29.16b, v20.16b + ext v4.16b, v4.16b, v4.16b, #12 + add v6.4s, v6.4s, v5.4s + mov v29.s[1], v27.s[2] + add v4.4s, v4.4s, v27.4s + zip1 v26.2d, v20.2d, v7.2d + zip1 v7.4s, v16.4s, v27.4s + zip1 v16.4s, v27.4s, v16.4s + eor v17.16b, v17.16b, v6.16b + ext v7.16b, v16.16b, v7.16b, #8 + ushr v16.4s, v17.4s, #7 + shl v17.4s, v17.4s, #25 + orr v16.16b, v17.16b, v16.16b + ext v5.16b, v5.16b, v5.16b, #8 + add v4.4s, v4.4s, v16.4s + eor v5.16b, v4.16b, v5.16b + ext v6.16b, v6.16b, v6.16b, #4 + tbl v5.16b, { v5.16b }, v0.16b + add v6.4s, v6.4s, v5.4s + eor v16.16b, v16.16b, v6.16b + ushr v17.4s, v16.4s, #12 + shl v16.4s, v16.4s, #20 + add v4.4s, v4.4s, v20.4s + orr v16.16b, v16.16b, v17.16b + add v4.4s, v4.4s, v16.4s + eor v5.16b, v5.16b, v4.16b + tbl v5.16b, { v5.16b }, v1.16b + add v6.4s, v6.4s, v5.4s + eor v16.16b, v16.16b, v6.16b + add v4.4s, v4.4s, v18.4s + ushr v17.4s, v16.4s, #7 + shl v16.4s, v16.4s, #25 + ext v23.16b, v22.16b, v22.16b, #12 + ext v4.16b, v4.16b, v4.16b, #4 + orr v16.16b, v16.16b, v17.16b + ext v28.16b, v22.16b, v23.16b, #12 + ext v5.16b, v5.16b, v5.16b, #8 + add v4.4s, v16.4s, v4.4s + tbl v3.16b, { v28.16b, v29.16b }, v3.16b + eor v5.16b, v4.16b, v5.16b + ext v6.16b, v6.16b, v6.16b, #12 + add v3.4s, v4.4s, v3.4s + tbl v4.16b, { v5.16b }, v0.16b + add v5.4s, v6.4s, v4.4s + eor v6.16b, v16.16b, v5.16b + ushr v16.4s, v6.4s, #12 + shl v6.4s, v6.4s, #20 + orr v6.16b, v6.16b, v16.16b + tbl v2.16b, { v26.16b, v27.16b }, v2.16b + add v3.4s, v3.4s, v6.4s + ext v19.16b, v2.16b, v2.16b, #12 + eor v4.16b, v4.16b, v3.16b + uzp1 v2.4s, v2.4s, v19.4s + ext v3.16b, v3.16b, v3.16b, #12 + tbl v4.16b, { v4.16b }, v1.16b + add v2.4s, v3.4s, v2.4s + add v3.4s, v5.4s, v4.4s + eor v5.16b, v6.16b, v3.16b + ushr v6.4s, v5.4s, #7 + shl v5.4s, v5.4s, #25 + orr v5.16b, v5.16b, v6.16b + ext v4.16b, v4.16b, v4.16b, #8 + add v2.4s, v2.4s, v5.4s + eor v4.16b, v2.16b, v4.16b + ext v3.16b, v3.16b, v3.16b, #4 + tbl v0.16b, { v4.16b }, v0.16b + add v3.4s, v3.4s, v0.4s + eor v4.16b, v5.16b, v3.16b + ushr v5.4s, v4.4s, #12 + shl v4.4s, v4.4s, #20 + add v2.4s, v2.4s, v7.4s + orr v4.16b, v4.16b, v5.16b + add v2.4s, v2.4s, v4.4s + eor v0.16b, v0.16b, v2.16b + tbl v0.16b, { v0.16b }, v1.16b + add v1.4s, v3.4s, v0.4s + eor v3.16b, v4.16b, v1.16b + ushr v4.4s, v3.4s, #7 + shl v3.4s, v3.4s, #25 + ext v2.16b, v2.16b, v2.16b, #4 + ext v0.16b, v0.16b, v0.16b, #8 + ext v1.16b, v1.16b, v1.16b, #12 + orr v3.16b, v3.16b, v4.16b + eor v2.16b, v2.16b, v1.16b + eor v3.16b, v3.16b, v0.16b + stp q2, q3, [x5] + ldr q2, [x0] + eor v1.16b, v2.16b, v1.16b + str q1, [x5, #32] + ldr q1, [x0, #16] + eor v0.16b, v1.16b, v0.16b + str q0, [x5, #48] + ret +.Lfunc_end1: + .size zfs_blake3_compress_xof_sse41, .Lfunc_end1-zfs_blake3_compress_xof_sse41 + .cfi_endproc + + .section .rodata.cst16,"aM",@progbits,16 + .p2align 4 +.LCPI2_0: + .word 0 + .word 1 + .word 2 + .word 3 +.LCPI2_1: + .byte 2 + .byte 3 + .byte 0 + .byte 1 + .byte 6 + .byte 7 + .byte 4 + .byte 5 + .byte 10 + .byte 11 + .byte 8 + .byte 9 + .byte 14 + .byte 15 + .byte 12 + .byte 13 +.LCPI2_2: + .byte 1 + .byte 2 + .byte 3 + .byte 0 + .byte 5 + .byte 6 + .byte 7 + .byte 4 + .byte 9 + .byte 10 + .byte 11 + .byte 8 + .byte 13 + .byte 14 + .byte 15 + .byte 12 + .text + .globl zfs_blake3_hash_many_sse41 + .p2align 2 + .type zfs_blake3_hash_many_sse41,@function +zfs_blake3_hash_many_sse41: + .cfi_startproc + stp d15, d14, [sp, #-160]! + stp d13, d12, [sp, #16] + stp d11, d10, [sp, #32] + stp d9, d8, [sp, #48] + stp x29, x30, [sp, #64] + stp x28, x27, [sp, #80] + stp x26, x25, [sp, #96] + stp x24, x23, [sp, #112] + stp x22, x21, [sp, #128] + stp x20, x19, [sp, #144] + mov x29, sp + sub sp, sp, #448 + .cfi_def_cfa w29, 160 + .cfi_offset w19, -8 + .cfi_offset w20, -16 + .cfi_offset w21, -24 + .cfi_offset w22, -32 + .cfi_offset w23, -40 + .cfi_offset w24, -48 + .cfi_offset w25, -56 + .cfi_offset w26, -64 + .cfi_offset w27, -72 + .cfi_offset w28, -80 + .cfi_offset w30, -88 + .cfi_offset w29, -96 + .cfi_offset b8, -104 + .cfi_offset b9, -112 + .cfi_offset b10, -120 + .cfi_offset b11, -128 + .cfi_offset b12, -136 + .cfi_offset b13, -144 + .cfi_offset b14, -152 + .cfi_offset b15, -160 + ldr x26, [x29, #168] + ldrb w27, [x29, #160] + mov w19, w6 + mov x20, x4 + mov x22, x2 + mov x28, x1 + cmp x1, #4 + mov x24, x0 + str x3, [sp, #40] + b.lo .LBB2_8 + adrp x11, .LCPI2_0 + ldr q0, [x11, :lo12:.LCPI2_0] + sbfx w13, w5, #0, #1 + dup v1.4s, w13 + mov w10, #58983 + mov w11, #44677 + mov w12, #62322 + and v0.16b, v1.16b, v0.16b + mov w13, #62778 + orr w8, w7, w19 + adrp x9, .LCPI2_1 + movk w10, #27145, lsl #16 + movk w11, #47975, lsl #16 + movk w12, #15470, lsl #16 + movk w13, #42319, lsl #16 + str q0, [sp, #16] + orr v0.4s, #128, lsl #24 + adrp x14, .LCPI2_2 + str q0, [sp] +.LBB2_2: + ldr x2, [sp, #40] + mov x15, x2 + ld1r { v7.4s }, [x15], #4 + add x16, x2, #8 + add x17, x2, #12 + add x18, x2, #16 + add x0, x2, #20 + add x3, x2, #24 + add x2, x2, #28 + ld1r { v6.4s }, [x16] + ld1r { v17.4s }, [x17] + ld1r { v10.4s }, [x18] + ld1r { v11.4s }, [x0] + ld1r { v19.4s }, [x3] + ld1r { v18.4s }, [x15] + ld1r { v16.4s }, [x2] + cbz x22, .LBB2_7 + ldr q1, [sp, #16] + dup v0.4s, w20 + ldp x15, x16, [x24] + ldp x17, x18, [x24, #16] + add v1.4s, v0.4s, v1.4s + movi v0.4s, #128, lsl #24 + str q1, [sp, #64] + eor v0.16b, v1.16b, v0.16b + ldr q1, [sp] + lsr x2, x20, #32 + mov x0, xzr + mov w6, w8 + cmgt v0.4s, v1.4s, v0.4s + dup v1.4s, w2 + sub v0.4s, v1.4s, v0.4s + str q0, [sp, #48] +.LBB2_4: + mov w4, #16 + stp q16, q17, [sp, #192] + bfi x4, x0, #6, #58 + ldr q1, [x15, x4] + ldr q3, [x16, x4] + ldr q2, [x17, x4] + ldr q4, [x18, x4] + mov w4, #32 + bfi x4, x0, #6, #58 + ldr q5, [x15, x4] + ldr q20, [x16, x4] + ldr q21, [x17, x4] + ldr q22, [x18, x4] + mov w4, #48 + lsl x3, x0, #6 + bfi x4, x0, #6, #58 + add x0, x0, #1 + ldr q0, [x15, x3] + ldr q23, [x16, x3] + ldr q16, [x17, x3] + ldr q17, [x18, x3] + cmp x0, x22 + ldr q25, [x15, x4] + ldr q14, [x16, x4] + ldr q28, [x17, x4] + ldr q31, [x18, x4] + csel w4, w27, wzr, eq + orr w4, w4, w6 + mov x2, xzr + and w6, w4, #0xff + add x3, x3, #256 +.LBB2_5: + ldr x4, [x24, x2] + add x2, x2, #8 + cmp x2, #32 + add x4, x4, x3 + prfm pldl1keep, [x4] + b.ne .LBB2_5 + zip1 v29.4s, v0.4s, v23.4s + zip2 v23.4s, v0.4s, v23.4s + zip1 v0.4s, v16.4s, v17.4s + zip2 v24.4s, v16.4s, v17.4s + zip1 v9.4s, v1.4s, v3.4s + zip2 v26.4s, v1.4s, v3.4s + zip1 v27.4s, v2.4s, v4.4s + zip2 v17.4s, v2.4s, v4.4s + zip1 v12.4s, v21.4s, v22.4s + zip2 v13.4s, v21.4s, v22.4s + add v2.4s, v7.4s, v10.4s + add v1.4s, v18.4s, v11.4s + ext v7.16b, v0.16b, v29.16b, #8 + ext v22.16b, v24.16b, v23.16b, #8 + zip1 v30.4s, v5.4s, v20.4s + zip2 v20.4s, v5.4s, v20.4s + stp q1, q2, [sp, #112] + ext v2.16b, v29.16b, v7.16b, #8 + mov v29.d[1], v0.d[0] + ext v18.16b, v23.16b, v22.16b, #8 + mov v23.d[1], v24.d[0] + zip1 v21.4s, v25.4s, v14.4s + zip2 v4.4s, v25.4s, v14.4s + zip1 v14.4s, v28.4s, v31.4s + zip2 v15.4s, v28.4s, v31.4s + add v8.4s, v6.4s, v19.4s + ext v28.16b, v27.16b, v9.16b, #8 + ext v31.16b, v17.16b, v26.16b, #8 + stur q2, [x29, #-208] + mov v7.16b, v29.16b + ext v0.16b, v12.16b, v30.16b, #8 + stp q23, q29, [x29, #-80] + mov v2.16b, v19.16b + ext v19.16b, v13.16b, v20.16b, #8 + mov v29.16b, v9.16b + ext v25.16b, v9.16b, v28.16b, #8 + mov v29.d[1], v27.d[0] + ext v24.16b, v26.16b, v31.16b, #8 + mov v26.d[1], v17.d[0] + ext v17.16b, v15.16b, v4.16b, #8 + ext v27.16b, v30.16b, v0.16b, #8 + ext v0.16b, v20.16b, v19.16b, #8 + stp q0, q25, [sp, #80] + ext v0.16b, v4.16b, v17.16b, #8 + str q0, [sp, #224] + ldr q0, [sp, #128] + mov v6.16b, v23.16b + mov v22.16b, v4.16b + ldr q16, [x9, :lo12:.LCPI2_1] + add v17.4s, v0.4s, v7.4s + ldr q0, [sp, #112] + mov v30.d[1], v12.d[0] + add v7.4s, v8.4s, v29.4s + mov v20.d[1], v13.d[0] + add v4.4s, v0.4s, v6.4s + ldr q0, [sp, #64] + dup v3.4s, w12 + ext v28.16b, v14.16b, v21.16b, #8 + dup v1.4s, w10 + eor v19.16b, v17.16b, v0.16b + ldr q0, [sp, #48] + ext v23.16b, v21.16b, v28.16b, #8 + mov v21.d[1], v14.d[0] + tbl v14.16b, { v19.16b }, v16.16b + eor v12.16b, v4.16b, v0.16b + movi v0.4s, #64 + eor v13.16b, v7.16b, v0.16b + tbl v13.16b, { v13.16b }, v16.16b + add v6.4s, v13.4s, v3.4s + dup v5.4s, w11 + tbl v12.16b, { v12.16b }, v16.16b + add v1.4s, v14.4s, v1.4s + eor v9.16b, v6.16b, v2.16b + ldp q2, q0, [sp, #192] + add v5.4s, v12.4s, v5.4s + eor v19.16b, v1.16b, v10.16b + eor v10.16b, v5.16b, v11.16b + ushr v11.4s, v19.4s, #12 + shl v19.4s, v19.4s, #20 + orr v11.16b, v19.16b, v11.16b + ushr v19.4s, v10.4s, #12 + shl v10.4s, v10.4s, #20 + mov v22.d[1], v15.d[0] + orr v10.16b, v10.16b, v19.16b + ushr v19.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + add v15.4s, v0.4s, v2.4s + orr v9.16b, v9.16b, v19.16b + dup v19.4s, w6 + add v15.4s, v15.4s, v26.4s + eor v19.16b, v15.16b, v19.16b + tbl v3.16b, { v19.16b }, v16.16b + dup v19.4s, w13 + add v8.4s, v3.4s, v19.4s + ldur q31, [x29, #-208] + eor v19.16b, v8.16b, v2.16b + ushr v0.4s, v19.4s, #12 + shl v19.4s, v19.4s, #20 + orr v2.16b, v19.16b, v0.16b + ldr q19, [x14, :lo12:.LCPI2_2] + add v17.4s, v17.4s, v31.4s + add v17.4s, v17.4s, v11.4s + eor v14.16b, v14.16b, v17.16b + tbl v14.16b, { v14.16b }, v19.16b + add v1.4s, v1.4s, v14.4s + eor v11.16b, v1.16b, v11.16b + add v4.4s, v4.4s, v18.4s + ushr v0.4s, v11.4s, #7 + shl v11.4s, v11.4s, #25 + add v4.4s, v4.4s, v10.4s + orr v0.16b, v11.16b, v0.16b + eor v11.16b, v12.16b, v4.16b + tbl v11.16b, { v11.16b }, v19.16b + add v5.4s, v5.4s, v11.4s + eor v10.16b, v5.16b, v10.16b + add v7.4s, v7.4s, v25.4s + ushr v12.4s, v10.4s, #7 + shl v10.4s, v10.4s, #25 + add v7.4s, v7.4s, v9.4s + orr v10.16b, v10.16b, v12.16b + eor v12.16b, v13.16b, v7.16b + tbl v12.16b, { v12.16b }, v19.16b + add v6.4s, v6.4s, v12.4s + eor v9.16b, v6.16b, v9.16b + ushr v13.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + orr v9.16b, v9.16b, v13.16b + add v13.4s, v15.4s, v24.4s + add v13.4s, v13.4s, v2.4s + eor v3.16b, v3.16b, v13.16b + tbl v3.16b, { v3.16b }, v19.16b + add v8.4s, v8.4s, v3.4s + eor v2.16b, v8.16b, v2.16b + add v17.4s, v17.4s, v30.4s + ushr v15.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + add v17.4s, v17.4s, v10.4s + add v4.4s, v4.4s, v20.4s + orr v2.16b, v2.16b, v15.16b + eor v3.16b, v3.16b, v17.16b + add v4.4s, v4.4s, v9.4s + add v7.4s, v7.4s, v21.4s + tbl v3.16b, { v3.16b }, v16.16b + eor v14.16b, v14.16b, v4.16b + add v7.4s, v7.4s, v2.4s + add v13.4s, v13.4s, v22.4s + mov v28.16b, v26.16b + stur q26, [x29, #-112] + mov v26.16b, v18.16b + mov v18.16b, v24.16b + stur q24, [x29, #-160] + add v6.4s, v6.4s, v3.4s + mov v24.16b, v20.16b + tbl v14.16b, { v14.16b }, v16.16b + eor v11.16b, v11.16b, v7.16b + add v13.4s, v13.4s, v0.4s + ldr q20, [sp, #80] + eor v10.16b, v6.16b, v10.16b + add v8.4s, v8.4s, v14.4s + tbl v11.16b, { v11.16b }, v16.16b + eor v12.16b, v12.16b, v13.16b + stp q30, q22, [x29, #-192] + ushr v15.4s, v10.4s, #12 + shl v10.4s, v10.4s, #20 + eor v9.16b, v8.16b, v9.16b + add v1.4s, v1.4s, v11.4s + tbl v12.16b, { v12.16b }, v16.16b + mov v30.16b, v27.16b + add v17.4s, v17.4s, v27.4s + ldr q27, [sp, #224] + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v2.16b, v1.16b, v2.16b + add v5.4s, v5.4s, v12.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + eor v0.16b, v5.16b, v0.16b + add v17.4s, v17.4s, v10.4s + add v4.4s, v4.4s, v20.4s + orr v2.16b, v2.16b, v15.16b + ushr v15.4s, v0.4s, #12 + shl v0.4s, v0.4s, #20 + eor v3.16b, v3.16b, v17.16b + add v4.4s, v4.4s, v9.4s + add v7.4s, v7.4s, v23.4s + orr v0.16b, v0.16b, v15.16b + tbl v3.16b, { v3.16b }, v19.16b + eor v14.16b, v14.16b, v4.16b + add v7.4s, v7.4s, v2.4s + add v13.4s, v13.4s, v27.4s + add v6.4s, v6.4s, v3.4s + tbl v14.16b, { v14.16b }, v19.16b + eor v11.16b, v11.16b, v7.16b + add v13.4s, v13.4s, v0.4s + eor v10.16b, v6.16b, v10.16b + add v8.4s, v8.4s, v14.4s + tbl v11.16b, { v11.16b }, v19.16b + eor v12.16b, v12.16b, v13.16b + stur q21, [x29, #-144] + ushr v15.4s, v10.4s, #7 + shl v10.4s, v10.4s, #25 + eor v9.16b, v8.16b, v9.16b + add v1.4s, v1.4s, v11.4s + tbl v12.16b, { v12.16b }, v19.16b + ldur q21, [x29, #-80] + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v2.16b, v1.16b, v2.16b + add v5.4s, v5.4s, v12.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + eor v0.16b, v5.16b, v0.16b + orr v2.16b, v2.16b, v15.16b + ushr v15.4s, v0.4s, #7 + shl v0.4s, v0.4s, #25 + orr v0.16b, v0.16b, v15.16b + add v17.4s, v17.4s, v21.4s + add v17.4s, v17.4s, v0.4s + add v4.4s, v4.4s, v26.4s + eor v14.16b, v14.16b, v17.16b + add v4.4s, v4.4s, v10.4s + add v7.4s, v7.4s, v18.4s + tbl v14.16b, { v14.16b }, v16.16b + eor v11.16b, v11.16b, v4.16b + add v7.4s, v7.4s, v9.4s + add v13.4s, v13.4s, v29.4s + add v1.4s, v1.4s, v14.4s + tbl v11.16b, { v11.16b }, v16.16b + eor v12.16b, v12.16b, v7.16b + add v13.4s, v13.4s, v2.4s + eor v0.16b, v0.16b, v1.16b + add v5.4s, v5.4s, v11.4s + tbl v12.16b, { v12.16b }, v16.16b + eor v3.16b, v3.16b, v13.16b + ldur q22, [x29, #-64] + ushr v15.4s, v0.4s, #12 + shl v0.4s, v0.4s, #20 + eor v10.16b, v5.16b, v10.16b + add v6.4s, v6.4s, v12.4s + tbl v3.16b, { v3.16b }, v16.16b + orr v0.16b, v0.16b, v15.16b + ushr v15.4s, v10.4s, #12 + shl v10.4s, v10.4s, #20 + eor v9.16b, v6.16b, v9.16b + add v8.4s, v8.4s, v3.4s + add v17.4s, v17.4s, v28.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v2.16b, v8.16b, v2.16b + add v17.4s, v17.4s, v0.4s + add v4.4s, v4.4s, v24.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + eor v14.16b, v14.16b, v17.16b + add v4.4s, v4.4s, v10.4s + add v7.4s, v7.4s, v22.4s + orr v2.16b, v2.16b, v15.16b + tbl v14.16b, { v14.16b }, v19.16b + eor v11.16b, v11.16b, v4.16b + add v7.4s, v7.4s, v9.4s + add v13.4s, v13.4s, v23.4s + add v1.4s, v1.4s, v14.4s + tbl v11.16b, { v11.16b }, v19.16b + eor v12.16b, v12.16b, v7.16b + add v13.4s, v13.4s, v2.4s + eor v0.16b, v0.16b, v1.16b + add v5.4s, v5.4s, v11.4s + tbl v12.16b, { v12.16b }, v19.16b + eor v3.16b, v3.16b, v13.16b + ldur q22, [x29, #-144] + ushr v15.4s, v0.4s, #7 + shl v0.4s, v0.4s, #25 + eor v10.16b, v5.16b, v10.16b + add v6.4s, v6.4s, v12.4s + tbl v3.16b, { v3.16b }, v19.16b + orr v0.16b, v0.16b, v15.16b + ushr v15.4s, v10.4s, #7 + shl v10.4s, v10.4s, #25 + eor v9.16b, v6.16b, v9.16b + add v8.4s, v8.4s, v3.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v2.16b, v8.16b, v2.16b + add v17.4s, v17.4s, v31.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + add v17.4s, v17.4s, v10.4s + add v4.4s, v4.4s, v22.4s + orr v2.16b, v2.16b, v15.16b + eor v3.16b, v3.16b, v17.16b + add v4.4s, v4.4s, v9.4s + add v7.4s, v7.4s, v30.4s + tbl v3.16b, { v3.16b }, v16.16b + eor v14.16b, v14.16b, v4.16b + add v7.4s, v7.4s, v2.4s + add v13.4s, v13.4s, v27.4s + add v6.4s, v6.4s, v3.4s + tbl v14.16b, { v14.16b }, v16.16b + eor v11.16b, v11.16b, v7.16b + add v13.4s, v13.4s, v0.4s + ldr q27, [sp, #96] + mov v21.16b, v26.16b + stur q26, [x29, #-96] + mov v28.16b, v31.16b + eor v10.16b, v6.16b, v10.16b + add v8.4s, v8.4s, v14.4s + tbl v11.16b, { v11.16b }, v16.16b + eor v12.16b, v12.16b, v13.16b + ldp q31, q26, [x29, #-192] + ushr v15.4s, v10.4s, #12 + shl v10.4s, v10.4s, #20 + eor v9.16b, v8.16b, v9.16b + add v1.4s, v1.4s, v11.4s + tbl v12.16b, { v12.16b }, v16.16b + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v2.16b, v1.16b, v2.16b + add v5.4s, v5.4s, v12.4s + add v17.4s, v17.4s, v20.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + eor v0.16b, v5.16b, v0.16b + add v17.4s, v17.4s, v10.4s + add v4.4s, v4.4s, v27.4s + orr v2.16b, v2.16b, v15.16b + ushr v15.4s, v0.4s, #12 + shl v0.4s, v0.4s, #20 + eor v3.16b, v3.16b, v17.16b + add v4.4s, v4.4s, v9.4s + add v7.4s, v7.4s, v26.4s + orr v0.16b, v0.16b, v15.16b + tbl v3.16b, { v3.16b }, v19.16b + eor v14.16b, v14.16b, v4.16b + add v7.4s, v7.4s, v2.4s + add v13.4s, v13.4s, v31.4s + add v6.4s, v6.4s, v3.4s + tbl v14.16b, { v14.16b }, v19.16b + eor v11.16b, v11.16b, v7.16b + add v13.4s, v13.4s, v0.4s + eor v10.16b, v6.16b, v10.16b + add v8.4s, v8.4s, v14.4s + tbl v11.16b, { v11.16b }, v19.16b + eor v12.16b, v12.16b, v13.16b + ushr v15.4s, v10.4s, #7 + shl v10.4s, v10.4s, #25 + eor v9.16b, v8.16b, v9.16b + add v1.4s, v1.4s, v11.4s + tbl v12.16b, { v12.16b }, v19.16b + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v2.16b, v1.16b, v2.16b + add v5.4s, v5.4s, v12.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + eor v0.16b, v5.16b, v0.16b + mov v18.16b, v24.16b + mov v24.16b, v20.16b + orr v2.16b, v2.16b, v15.16b + ushr v15.4s, v0.4s, #7 + shl v0.4s, v0.4s, #25 + ldur q20, [x29, #-160] + orr v0.16b, v0.16b, v15.16b + add v17.4s, v17.4s, v21.4s + add v17.4s, v17.4s, v0.4s + add v4.4s, v4.4s, v18.4s + eor v14.16b, v14.16b, v17.16b + add v4.4s, v4.4s, v10.4s + add v7.4s, v7.4s, v23.4s + tbl v14.16b, { v14.16b }, v16.16b + eor v11.16b, v11.16b, v4.16b + add v7.4s, v7.4s, v9.4s + add v13.4s, v13.4s, v20.4s + add v1.4s, v1.4s, v14.4s + tbl v11.16b, { v11.16b }, v16.16b + eor v12.16b, v12.16b, v7.16b + add v13.4s, v13.4s, v2.4s + eor v0.16b, v0.16b, v1.16b + add v5.4s, v5.4s, v11.4s + tbl v12.16b, { v12.16b }, v16.16b + eor v3.16b, v3.16b, v13.16b + ldur q25, [x29, #-80] + ushr v15.4s, v0.4s, #12 + shl v0.4s, v0.4s, #20 + eor v10.16b, v5.16b, v10.16b + add v6.4s, v6.4s, v12.4s + tbl v3.16b, { v3.16b }, v16.16b + orr v0.16b, v0.16b, v15.16b + ushr v15.4s, v10.4s, #12 + shl v10.4s, v10.4s, #20 + eor v9.16b, v6.16b, v9.16b + add v8.4s, v8.4s, v3.4s + add v17.4s, v17.4s, v29.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v2.16b, v8.16b, v2.16b + add v17.4s, v17.4s, v0.4s + add v4.4s, v4.4s, v22.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + eor v14.16b, v14.16b, v17.16b + add v4.4s, v4.4s, v10.4s + add v7.4s, v7.4s, v25.4s + orr v2.16b, v2.16b, v15.16b + tbl v14.16b, { v14.16b }, v19.16b + eor v11.16b, v11.16b, v4.16b + add v7.4s, v7.4s, v9.4s + add v13.4s, v13.4s, v26.4s + add v1.4s, v1.4s, v14.4s + tbl v11.16b, { v11.16b }, v19.16b + eor v12.16b, v12.16b, v7.16b + add v13.4s, v13.4s, v2.4s + ldur q25, [x29, #-112] + eor v0.16b, v0.16b, v1.16b + add v5.4s, v5.4s, v11.4s + tbl v12.16b, { v12.16b }, v19.16b + eor v3.16b, v3.16b, v13.16b + ushr v15.4s, v0.4s, #7 + shl v0.4s, v0.4s, #25 + eor v10.16b, v5.16b, v10.16b + add v6.4s, v6.4s, v12.4s + tbl v3.16b, { v3.16b }, v19.16b + orr v0.16b, v0.16b, v15.16b + ushr v15.4s, v10.4s, #7 + shl v10.4s, v10.4s, #25 + eor v9.16b, v6.16b, v9.16b + add v8.4s, v8.4s, v3.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v2.16b, v8.16b, v2.16b + add v17.4s, v17.4s, v25.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + add v17.4s, v17.4s, v10.4s + add v4.4s, v4.4s, v30.4s + orr v2.16b, v2.16b, v15.16b + eor v3.16b, v3.16b, v17.16b + add v4.4s, v4.4s, v9.4s + add v7.4s, v7.4s, v24.4s + tbl v3.16b, { v3.16b }, v16.16b + eor v14.16b, v14.16b, v4.16b + add v7.4s, v7.4s, v2.4s + add v13.4s, v13.4s, v31.4s + add v6.4s, v6.4s, v3.4s + tbl v14.16b, { v14.16b }, v16.16b + eor v11.16b, v11.16b, v7.16b + add v13.4s, v13.4s, v0.4s + ldur q25, [x29, #-64] + eor v10.16b, v6.16b, v10.16b + add v8.4s, v8.4s, v14.4s + tbl v11.16b, { v11.16b }, v16.16b + eor v12.16b, v12.16b, v13.16b + ldr q31, [sp, #224] + ushr v15.4s, v10.4s, #12 + shl v10.4s, v10.4s, #20 + eor v9.16b, v8.16b, v9.16b + add v1.4s, v1.4s, v11.4s + tbl v12.16b, { v12.16b }, v16.16b + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v2.16b, v1.16b, v2.16b + add v5.4s, v5.4s, v12.4s + add v17.4s, v17.4s, v27.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + eor v0.16b, v5.16b, v0.16b + add v17.4s, v17.4s, v10.4s + add v4.4s, v4.4s, v25.4s + orr v2.16b, v2.16b, v15.16b + ushr v15.4s, v0.4s, #12 + shl v0.4s, v0.4s, #20 + eor v3.16b, v3.16b, v17.16b + add v4.4s, v4.4s, v9.4s + add v7.4s, v7.4s, v31.4s + orr v0.16b, v0.16b, v15.16b + tbl v3.16b, { v3.16b }, v19.16b + eor v14.16b, v14.16b, v4.16b + add v7.4s, v7.4s, v2.4s + add v13.4s, v13.4s, v28.4s + add v6.4s, v6.4s, v3.4s + tbl v14.16b, { v14.16b }, v19.16b + eor v11.16b, v11.16b, v7.16b + add v13.4s, v13.4s, v0.4s + eor v10.16b, v6.16b, v10.16b + add v8.4s, v8.4s, v14.4s + tbl v11.16b, { v11.16b }, v19.16b + eor v12.16b, v12.16b, v13.16b + ushr v15.4s, v10.4s, #7 + shl v10.4s, v10.4s, #25 + eor v9.16b, v8.16b, v9.16b + add v1.4s, v1.4s, v11.4s + tbl v12.16b, { v12.16b }, v19.16b + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v2.16b, v1.16b, v2.16b + add v5.4s, v5.4s, v12.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + eor v0.16b, v5.16b, v0.16b + orr v2.16b, v2.16b, v15.16b + ushr v15.4s, v0.4s, #7 + shl v0.4s, v0.4s, #25 + orr v0.16b, v0.16b, v15.16b + add v17.4s, v17.4s, v18.4s + add v17.4s, v17.4s, v0.4s + add v4.4s, v4.4s, v22.4s + eor v14.16b, v14.16b, v17.16b + add v4.4s, v4.4s, v10.4s + add v7.4s, v7.4s, v26.4s + tbl v14.16b, { v14.16b }, v16.16b + eor v11.16b, v11.16b, v4.16b + add v7.4s, v7.4s, v9.4s + add v13.4s, v13.4s, v23.4s + add v1.4s, v1.4s, v14.4s + tbl v11.16b, { v11.16b }, v16.16b + eor v12.16b, v12.16b, v7.16b + add v13.4s, v13.4s, v2.4s + mov v21.16b, v29.16b + stur q29, [x29, #-128] + mov v29.16b, v30.16b + mov v30.16b, v27.16b + mov v27.16b, v18.16b + str q18, [sp, #176] + eor v0.16b, v0.16b, v1.16b + mov v18.16b, v22.16b + add v5.4s, v5.4s, v11.4s + tbl v12.16b, { v12.16b }, v16.16b + eor v3.16b, v3.16b, v13.16b + ldur q22, [x29, #-96] + ushr v15.4s, v0.4s, #12 + shl v0.4s, v0.4s, #20 + eor v10.16b, v5.16b, v10.16b + add v6.4s, v6.4s, v12.4s + tbl v3.16b, { v3.16b }, v16.16b + orr v0.16b, v0.16b, v15.16b + ushr v15.4s, v10.4s, #12 + shl v10.4s, v10.4s, #20 + eor v9.16b, v6.16b, v9.16b + add v8.4s, v8.4s, v3.4s + add v17.4s, v17.4s, v20.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v2.16b, v8.16b, v2.16b + add v17.4s, v17.4s, v0.4s + add v4.4s, v4.4s, v29.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + eor v14.16b, v14.16b, v17.16b + add v4.4s, v4.4s, v10.4s + add v7.4s, v7.4s, v22.4s + orr v2.16b, v2.16b, v15.16b + tbl v14.16b, { v14.16b }, v19.16b + eor v11.16b, v11.16b, v4.16b + add v7.4s, v7.4s, v9.4s + add v13.4s, v13.4s, v31.4s + add v1.4s, v1.4s, v14.4s + tbl v11.16b, { v11.16b }, v19.16b + eor v12.16b, v12.16b, v7.16b + add v13.4s, v13.4s, v2.4s + eor v0.16b, v0.16b, v1.16b + add v5.4s, v5.4s, v11.4s + tbl v12.16b, { v12.16b }, v19.16b + eor v3.16b, v3.16b, v13.16b + ushr v15.4s, v0.4s, #7 + shl v0.4s, v0.4s, #25 + eor v10.16b, v5.16b, v10.16b + add v6.4s, v6.4s, v12.4s + tbl v3.16b, { v3.16b }, v19.16b + orr v0.16b, v0.16b, v15.16b + ushr v15.4s, v10.4s, #7 + shl v10.4s, v10.4s, #25 + eor v9.16b, v6.16b, v9.16b + add v8.4s, v8.4s, v3.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v2.16b, v8.16b, v2.16b + add v17.4s, v17.4s, v21.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + add v17.4s, v17.4s, v10.4s + add v4.4s, v4.4s, v24.4s + orr v2.16b, v2.16b, v15.16b + eor v3.16b, v3.16b, v17.16b + add v4.4s, v4.4s, v9.4s + add v7.4s, v7.4s, v30.4s + tbl v3.16b, { v3.16b }, v16.16b + eor v14.16b, v14.16b, v4.16b + add v7.4s, v7.4s, v2.4s + add v13.4s, v13.4s, v28.4s + add v6.4s, v6.4s, v3.4s + mov v22.16b, v24.16b + tbl v14.16b, { v14.16b }, v16.16b + eor v11.16b, v11.16b, v7.16b + add v13.4s, v13.4s, v0.4s + ldur q24, [x29, #-80] + eor v10.16b, v6.16b, v10.16b + add v8.4s, v8.4s, v14.4s + mov v21.16b, v30.16b + tbl v11.16b, { v11.16b }, v16.16b + eor v12.16b, v12.16b, v13.16b + ldur q30, [x29, #-192] + mov v20.16b, v29.16b + ushr v15.4s, v10.4s, #12 + shl v10.4s, v10.4s, #20 + eor v9.16b, v8.16b, v9.16b + add v1.4s, v1.4s, v11.4s + tbl v12.16b, { v12.16b }, v16.16b + ldur q29, [x29, #-112] + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v2.16b, v1.16b, v2.16b + add v5.4s, v5.4s, v12.4s + add v17.4s, v17.4s, v25.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + eor v0.16b, v5.16b, v0.16b + add v17.4s, v17.4s, v10.4s + add v4.4s, v4.4s, v24.4s + orr v2.16b, v2.16b, v15.16b + ushr v15.4s, v0.4s, #12 + shl v0.4s, v0.4s, #20 + eor v3.16b, v3.16b, v17.16b + add v4.4s, v4.4s, v9.4s + add v7.4s, v7.4s, v30.4s + orr v0.16b, v0.16b, v15.16b + tbl v3.16b, { v3.16b }, v19.16b + eor v14.16b, v14.16b, v4.16b + add v7.4s, v7.4s, v2.4s + add v13.4s, v13.4s, v29.4s + add v6.4s, v6.4s, v3.4s + tbl v14.16b, { v14.16b }, v19.16b + eor v11.16b, v11.16b, v7.16b + add v13.4s, v13.4s, v0.4s + eor v10.16b, v6.16b, v10.16b + add v8.4s, v8.4s, v14.4s + tbl v11.16b, { v11.16b }, v19.16b + eor v12.16b, v12.16b, v13.16b + ushr v15.4s, v10.4s, #7 + shl v10.4s, v10.4s, #25 + eor v9.16b, v8.16b, v9.16b + add v1.4s, v1.4s, v11.4s + tbl v12.16b, { v12.16b }, v19.16b + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v2.16b, v1.16b, v2.16b + add v5.4s, v5.4s, v12.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + eor v0.16b, v5.16b, v0.16b + orr v2.16b, v2.16b, v15.16b + ushr v15.4s, v0.4s, #7 + shl v0.4s, v0.4s, #25 + orr v0.16b, v0.16b, v15.16b + add v17.4s, v17.4s, v18.4s + add v17.4s, v17.4s, v0.4s + add v4.4s, v4.4s, v20.4s + eor v14.16b, v14.16b, v17.16b + add v4.4s, v4.4s, v10.4s + add v7.4s, v7.4s, v31.4s + tbl v14.16b, { v14.16b }, v16.16b + eor v11.16b, v11.16b, v4.16b + add v7.4s, v7.4s, v9.4s + add v13.4s, v13.4s, v26.4s + add v1.4s, v1.4s, v14.4s + tbl v11.16b, { v11.16b }, v16.16b + eor v12.16b, v12.16b, v7.16b + add v13.4s, v13.4s, v2.4s + eor v0.16b, v0.16b, v1.16b + add v5.4s, v5.4s, v11.4s + tbl v12.16b, { v12.16b }, v16.16b + eor v3.16b, v3.16b, v13.16b + ushr v15.4s, v0.4s, #12 + shl v0.4s, v0.4s, #20 + eor v10.16b, v5.16b, v10.16b + add v6.4s, v6.4s, v12.4s + tbl v3.16b, { v3.16b }, v16.16b + orr v0.16b, v0.16b, v15.16b + ushr v15.4s, v10.4s, #12 + shl v10.4s, v10.4s, #20 + eor v9.16b, v6.16b, v9.16b + add v8.4s, v8.4s, v3.4s + add v17.4s, v17.4s, v23.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v2.16b, v8.16b, v2.16b + add v17.4s, v17.4s, v0.4s + add v4.4s, v4.4s, v22.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + eor v14.16b, v14.16b, v17.16b + add v4.4s, v4.4s, v10.4s + add v7.4s, v7.4s, v27.4s + orr v2.16b, v2.16b, v15.16b + tbl v14.16b, { v14.16b }, v19.16b + eor v11.16b, v11.16b, v4.16b + add v7.4s, v7.4s, v9.4s + add v13.4s, v13.4s, v30.4s + add v1.4s, v1.4s, v14.4s + tbl v11.16b, { v11.16b }, v19.16b + eor v12.16b, v12.16b, v7.16b + add v13.4s, v13.4s, v2.4s + ldur q27, [x29, #-160] + eor v0.16b, v0.16b, v1.16b + add v5.4s, v5.4s, v11.4s + tbl v12.16b, { v12.16b }, v19.16b + eor v3.16b, v3.16b, v13.16b + ushr v15.4s, v0.4s, #7 + shl v0.4s, v0.4s, #25 + eor v10.16b, v5.16b, v10.16b + add v6.4s, v6.4s, v12.4s + tbl v3.16b, { v3.16b }, v19.16b + orr v0.16b, v0.16b, v15.16b + ushr v15.4s, v10.4s, #7 + shl v10.4s, v10.4s, #25 + eor v9.16b, v6.16b, v9.16b + add v8.4s, v8.4s, v3.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v2.16b, v8.16b, v2.16b + add v17.4s, v17.4s, v27.4s + mov v28.16b, v25.16b + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + add v17.4s, v17.4s, v10.4s + add v4.4s, v4.4s, v21.4s + orr v2.16b, v2.16b, v15.16b + eor v3.16b, v3.16b, v17.16b + add v4.4s, v4.4s, v9.4s + add v7.4s, v7.4s, v28.4s + tbl v3.16b, { v3.16b }, v16.16b + eor v14.16b, v14.16b, v4.16b + add v7.4s, v7.4s, v2.4s + add v13.4s, v13.4s, v29.4s + mov v25.16b, v31.16b + add v6.4s, v6.4s, v3.4s + tbl v14.16b, { v14.16b }, v16.16b + eor v11.16b, v11.16b, v7.16b + add v13.4s, v13.4s, v0.4s + ldur q31, [x29, #-96] + eor v10.16b, v6.16b, v10.16b + add v8.4s, v8.4s, v14.4s + tbl v11.16b, { v11.16b }, v16.16b + eor v12.16b, v12.16b, v13.16b + ldur q28, [x29, #-208] + mov v18.16b, v20.16b + str q20, [sp, #144] + ushr v15.4s, v10.4s, #12 + shl v10.4s, v10.4s, #20 + eor v9.16b, v8.16b, v9.16b + add v1.4s, v1.4s, v11.4s + tbl v12.16b, { v12.16b }, v16.16b + ldur q20, [x29, #-128] + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v2.16b, v1.16b, v2.16b + add v5.4s, v5.4s, v12.4s + add v17.4s, v17.4s, v24.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + eor v0.16b, v5.16b, v0.16b + add v17.4s, v17.4s, v10.4s + add v4.4s, v4.4s, v31.4s + orr v2.16b, v2.16b, v15.16b + ushr v15.4s, v0.4s, #12 + shl v0.4s, v0.4s, #20 + eor v3.16b, v3.16b, v17.16b + add v4.4s, v4.4s, v9.4s + add v7.4s, v7.4s, v28.4s + orr v0.16b, v0.16b, v15.16b + tbl v3.16b, { v3.16b }, v19.16b + eor v14.16b, v14.16b, v4.16b + add v7.4s, v7.4s, v2.4s + add v13.4s, v13.4s, v20.4s + add v6.4s, v6.4s, v3.4s + tbl v14.16b, { v14.16b }, v19.16b + eor v11.16b, v11.16b, v7.16b + add v13.4s, v13.4s, v0.4s + eor v10.16b, v6.16b, v10.16b + add v8.4s, v8.4s, v14.4s + tbl v11.16b, { v11.16b }, v19.16b + eor v12.16b, v12.16b, v13.16b + ushr v15.4s, v10.4s, #7 + shl v10.4s, v10.4s, #25 + eor v9.16b, v8.16b, v9.16b + add v1.4s, v1.4s, v11.4s + tbl v12.16b, { v12.16b }, v19.16b + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v2.16b, v1.16b, v2.16b + add v5.4s, v5.4s, v12.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + eor v0.16b, v5.16b, v0.16b + orr v2.16b, v2.16b, v15.16b + ushr v15.4s, v0.4s, #7 + shl v0.4s, v0.4s, #25 + orr v0.16b, v0.16b, v15.16b + add v17.4s, v17.4s, v18.4s + add v17.4s, v17.4s, v0.4s + add v4.4s, v4.4s, v22.4s + eor v14.16b, v14.16b, v17.16b + add v4.4s, v4.4s, v10.4s + add v7.4s, v7.4s, v30.4s + tbl v14.16b, { v14.16b }, v16.16b + eor v11.16b, v11.16b, v4.16b + add v7.4s, v7.4s, v9.4s + add v13.4s, v13.4s, v25.4s + add v1.4s, v1.4s, v14.4s + tbl v11.16b, { v11.16b }, v16.16b + eor v12.16b, v12.16b, v7.16b + add v13.4s, v13.4s, v2.4s + eor v0.16b, v0.16b, v1.16b + add v5.4s, v5.4s, v11.4s + tbl v12.16b, { v12.16b }, v16.16b + eor v3.16b, v3.16b, v13.16b + add v17.4s, v17.4s, v26.4s + mov v26.16b, v21.16b + add v4.4s, v4.4s, v21.4s + ldur q21, [x29, #-144] + ushr v15.4s, v0.4s, #12 + shl v0.4s, v0.4s, #20 + eor v10.16b, v5.16b, v10.16b + add v6.4s, v6.4s, v12.4s + tbl v3.16b, { v3.16b }, v16.16b + orr v0.16b, v0.16b, v15.16b + ushr v15.4s, v10.4s, #12 + shl v10.4s, v10.4s, #20 + eor v9.16b, v6.16b, v9.16b + add v8.4s, v8.4s, v3.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v2.16b, v8.16b, v2.16b + add v17.4s, v17.4s, v0.4s + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + eor v14.16b, v14.16b, v17.16b + add v4.4s, v4.4s, v10.4s + add v7.4s, v7.4s, v21.4s + orr v2.16b, v2.16b, v15.16b + tbl v14.16b, { v14.16b }, v19.16b + eor v11.16b, v11.16b, v4.16b + add v7.4s, v7.4s, v9.4s + add v13.4s, v13.4s, v28.4s + add v1.4s, v1.4s, v14.4s + tbl v11.16b, { v11.16b }, v19.16b + eor v12.16b, v12.16b, v7.16b + add v13.4s, v13.4s, v2.4s + str q23, [sp, #160] + eor v0.16b, v0.16b, v1.16b + add v5.4s, v5.4s, v11.4s + tbl v12.16b, { v12.16b }, v19.16b + eor v3.16b, v3.16b, v13.16b + add v17.4s, v17.4s, v23.4s + ldur q23, [x29, #-64] + ushr v15.4s, v0.4s, #7 + shl v0.4s, v0.4s, #25 + eor v10.16b, v5.16b, v10.16b + add v6.4s, v6.4s, v12.4s + tbl v3.16b, { v3.16b }, v19.16b + orr v0.16b, v0.16b, v15.16b + ushr v15.4s, v10.4s, #7 + shl v10.4s, v10.4s, #25 + eor v9.16b, v6.16b, v9.16b + add v8.4s, v8.4s, v3.4s + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v2.16b, v8.16b, v2.16b + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + add v17.4s, v17.4s, v10.4s + add v4.4s, v4.4s, v23.4s + orr v2.16b, v2.16b, v15.16b + eor v3.16b, v3.16b, v17.16b + add v4.4s, v4.4s, v9.4s + add v7.4s, v7.4s, v24.4s + tbl v3.16b, { v3.16b }, v16.16b + eor v14.16b, v14.16b, v4.16b + add v7.4s, v7.4s, v2.4s + add v6.4s, v6.4s, v3.4s + tbl v14.16b, { v14.16b }, v16.16b + eor v11.16b, v11.16b, v7.16b + add v13.4s, v13.4s, v20.4s + eor v10.16b, v6.16b, v10.16b + add v8.4s, v8.4s, v14.4s + tbl v11.16b, { v11.16b }, v16.16b + add v13.4s, v13.4s, v0.4s + ldr q20, [sp, #176] + ushr v15.4s, v10.4s, #12 + shl v10.4s, v10.4s, #20 + eor v9.16b, v8.16b, v9.16b + add v1.4s, v1.4s, v11.4s + eor v12.16b, v12.16b, v13.16b + orr v10.16b, v10.16b, v15.16b + ushr v15.4s, v9.4s, #12 + shl v9.4s, v9.4s, #20 + eor v2.16b, v1.16b, v2.16b + tbl v12.16b, { v12.16b }, v16.16b + orr v9.16b, v9.16b, v15.16b + ushr v15.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + add v5.4s, v5.4s, v12.4s + add v17.4s, v17.4s, v31.4s + orr v2.16b, v2.16b, v15.16b + eor v0.16b, v5.16b, v0.16b + add v17.4s, v17.4s, v10.4s + add v4.4s, v4.4s, v20.4s + add v7.4s, v7.4s, v29.4s + ushr v15.4s, v0.4s, #12 + shl v0.4s, v0.4s, #20 + eor v3.16b, v3.16b, v17.16b + add v4.4s, v4.4s, v9.4s + add v7.4s, v7.4s, v2.4s + orr v0.16b, v0.16b, v15.16b + mov v15.16b, v31.16b + add v17.4s, v17.4s, v22.4s + eor v31.16b, v14.16b, v4.16b + eor v22.16b, v11.16b, v7.16b + add v11.4s, v13.4s, v27.4s + tbl v3.16b, { v3.16b }, v19.16b + add v11.4s, v11.4s, v0.4s + tbl v31.16b, { v31.16b }, v19.16b + add v6.4s, v6.4s, v3.4s + eor v12.16b, v12.16b, v11.16b + tbl v22.16b, { v22.16b }, v19.16b + add v8.4s, v8.4s, v31.4s + eor v10.16b, v6.16b, v10.16b + add v30.4s, v11.4s, v30.4s + tbl v11.16b, { v12.16b }, v19.16b + add v1.4s, v1.4s, v22.4s + eor v9.16b, v8.16b, v9.16b + ushr v12.4s, v10.4s, #7 + shl v10.4s, v10.4s, #25 + add v5.4s, v5.4s, v11.4s + eor v2.16b, v1.16b, v2.16b + orr v10.16b, v10.16b, v12.16b + ushr v12.4s, v9.4s, #7 + shl v9.4s, v9.4s, #25 + eor v0.16b, v5.16b, v0.16b + orr v9.16b, v9.16b, v12.16b + ushr v12.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + orr v2.16b, v2.16b, v12.16b + ushr v12.4s, v0.4s, #7 + shl v0.4s, v0.4s, #25 + orr v0.16b, v0.16b, v12.16b + add v4.4s, v4.4s, v26.4s + add v17.4s, v17.4s, v0.4s + add v7.4s, v7.4s, v28.4s + mov v18.16b, v27.16b + eor v31.16b, v31.16b, v17.16b + add v4.4s, v4.4s, v10.4s + add v27.4s, v30.4s, v2.4s + eor v22.16b, v22.16b, v4.16b + add v7.4s, v7.4s, v9.4s + eor v3.16b, v3.16b, v27.16b + add v26.4s, v27.4s, v29.4s + tbl v27.16b, { v31.16b }, v16.16b + eor v28.16b, v11.16b, v7.16b + tbl v22.16b, { v22.16b }, v16.16b + add v1.4s, v1.4s, v27.4s + add v4.4s, v4.4s, v23.4s + ldr q23, [sp, #144] + tbl v28.16b, { v28.16b }, v16.16b + tbl v3.16b, { v3.16b }, v16.16b + add v5.4s, v5.4s, v22.4s + eor v0.16b, v0.16b, v1.16b + add v6.4s, v6.4s, v28.4s + add v29.4s, v8.4s, v3.4s + eor v30.16b, v5.16b, v10.16b + ushr v8.4s, v0.4s, #12 + shl v0.4s, v0.4s, #20 + eor v31.16b, v6.16b, v9.16b + orr v0.16b, v0.16b, v8.16b + ushr v8.4s, v30.4s, #12 + shl v30.4s, v30.4s, #20 + eor v2.16b, v29.16b, v2.16b + orr v30.16b, v30.16b, v8.16b + ushr v8.4s, v31.4s, #12 + shl v31.4s, v31.4s, #20 + add v17.4s, v17.4s, v25.4s + add v7.4s, v7.4s, v23.4s + orr v31.16b, v31.16b, v8.16b + ushr v8.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + ldur q23, [x29, #-176] + orr v2.16b, v2.16b, v8.16b + add v17.4s, v17.4s, v0.4s + eor v27.16b, v27.16b, v17.16b + add v4.4s, v4.4s, v30.4s + add v25.4s, v26.4s, v2.4s + eor v22.16b, v22.16b, v4.16b + add v4.4s, v4.4s, v24.4s + add v7.4s, v7.4s, v31.4s + eor v3.16b, v3.16b, v25.16b + add v24.4s, v25.4s, v18.4s + tbl v25.16b, { v27.16b }, v19.16b + add v17.4s, v17.4s, v23.4s + eor v23.16b, v28.16b, v7.16b + tbl v22.16b, { v22.16b }, v19.16b + add v1.4s, v1.4s, v25.4s + tbl v23.16b, { v23.16b }, v19.16b + tbl v3.16b, { v3.16b }, v19.16b + add v5.4s, v5.4s, v22.4s + eor v0.16b, v0.16b, v1.16b + add v6.4s, v6.4s, v23.4s + add v26.4s, v29.4s, v3.4s + eor v27.16b, v5.16b, v30.16b + ushr v29.4s, v0.4s, #7 + shl v0.4s, v0.4s, #25 + eor v28.16b, v6.16b, v31.16b + orr v0.16b, v0.16b, v29.16b + ushr v29.4s, v27.4s, #7 + shl v27.4s, v27.4s, #25 + eor v2.16b, v26.16b, v2.16b + orr v27.16b, v27.16b, v29.16b + ushr v29.4s, v28.4s, #7 + shl v28.4s, v28.4s, #25 + ldur q18, [x29, #-128] + orr v28.16b, v28.16b, v29.16b + ushr v29.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + add v7.4s, v7.4s, v15.4s + orr v2.16b, v2.16b, v29.16b + add v17.4s, v17.4s, v27.4s + add v4.4s, v4.4s, v28.4s + add v7.4s, v7.4s, v2.4s + eor v3.16b, v3.16b, v17.16b + add v17.4s, v17.4s, v20.4s + eor v20.16b, v25.16b, v4.16b + add v4.4s, v4.4s, v21.4s + eor v21.16b, v22.16b, v7.16b + add v7.4s, v7.4s, v18.4s + add v18.4s, v24.4s, v0.4s + eor v22.16b, v23.16b, v18.16b + ldr q23, [sp, #160] + tbl v3.16b, { v3.16b }, v16.16b + tbl v20.16b, { v20.16b }, v16.16b + add v6.4s, v6.4s, v3.4s + add v18.4s, v18.4s, v23.4s + tbl v21.16b, { v21.16b }, v16.16b + tbl v16.16b, { v22.16b }, v16.16b + add v22.4s, v26.4s, v20.4s + eor v23.16b, v6.16b, v27.16b + add v1.4s, v1.4s, v21.4s + eor v24.16b, v22.16b, v28.16b + ushr v25.4s, v23.4s, #12 + shl v23.4s, v23.4s, #20 + add v5.4s, v5.4s, v16.4s + eor v2.16b, v1.16b, v2.16b + orr v23.16b, v23.16b, v25.16b + ushr v25.4s, v24.4s, #12 + shl v24.4s, v24.4s, #20 + eor v0.16b, v5.16b, v0.16b + orr v24.16b, v24.16b, v25.16b + ushr v25.4s, v2.4s, #12 + shl v2.4s, v2.4s, #20 + orr v2.16b, v2.16b, v25.16b + ushr v25.4s, v0.4s, #12 + shl v0.4s, v0.4s, #20 + orr v0.16b, v0.16b, v25.16b + add v25.4s, v7.4s, v2.4s + add v26.4s, v18.4s, v0.4s + eor v18.16b, v21.16b, v25.16b + add v17.4s, v17.4s, v23.4s + add v4.4s, v4.4s, v24.4s + eor v16.16b, v16.16b, v26.16b + tbl v21.16b, { v18.16b }, v19.16b + eor v3.16b, v3.16b, v17.16b + eor v7.16b, v20.16b, v4.16b + tbl v16.16b, { v16.16b }, v19.16b + add v1.4s, v1.4s, v21.4s + tbl v3.16b, { v3.16b }, v19.16b + tbl v20.16b, { v7.16b }, v19.16b + eor v2.16b, v1.16b, v2.16b + eor v7.16b, v1.16b, v17.16b + add v1.4s, v5.4s, v16.4s + eor v0.16b, v1.16b, v0.16b + eor v18.16b, v1.16b, v4.16b + add v1.4s, v6.4s, v3.4s + eor v4.16b, v1.16b, v23.16b + eor v6.16b, v25.16b, v1.16b + add v1.4s, v22.4s, v20.4s + eor v5.16b, v1.16b, v24.16b + eor v17.16b, v26.16b, v1.16b + ushr v1.4s, v4.4s, #7 + shl v4.4s, v4.4s, #25 + orr v1.16b, v4.16b, v1.16b + ushr v4.4s, v5.4s, #7 + shl v5.4s, v5.4s, #25 + orr v4.16b, v5.16b, v4.16b + ushr v5.4s, v2.4s, #7 + shl v2.4s, v2.4s, #25 + orr v2.16b, v2.16b, v5.16b + ushr v5.4s, v0.4s, #7 + shl v0.4s, v0.4s, #25 + orr v0.16b, v0.16b, v5.16b + eor v10.16b, v0.16b, v20.16b + eor v11.16b, v1.16b, v21.16b + eor v19.16b, v4.16b, v16.16b + cmp x0, x22 + eor v16.16b, v2.16b, v3.16b + mov w6, w19 + b.ne .LBB2_4 +.LBB2_7: + zip1 v0.4s, v7.4s, v18.4s + zip2 v1.4s, v7.4s, v18.4s + zip1 v2.4s, v6.4s, v17.4s + zip2 v3.4s, v6.4s, v17.4s + zip1 v4.4s, v10.4s, v11.4s + zip2 v5.4s, v10.4s, v11.4s + zip1 v6.4s, v19.4s, v16.4s + zip2 v7.4s, v19.4s, v16.4s + add x15, x20, #4 + tst w5, #0x1 + sub x28, x28, #4 + zip1 v16.2d, v0.2d, v2.2d + zip2 v0.2d, v0.2d, v2.2d + zip1 v2.2d, v1.2d, v3.2d + zip2 v1.2d, v1.2d, v3.2d + zip1 v3.2d, v4.2d, v6.2d + zip2 v4.2d, v4.2d, v6.2d + zip1 v6.2d, v5.2d, v7.2d + zip2 v5.2d, v5.2d, v7.2d + add x24, x24, #32 + csel x20, x15, x20, ne + cmp x28, #3 + stp q16, q3, [x26] + stp q0, q4, [x26, #32] + stp q2, q6, [x26, #64] + stp q1, q5, [x26, #96] + add x26, x26, #128 + b.hi .LBB2_2 +.LBB2_8: + cbz x28, .LBB2_16 + orr w8, w7, w19 + and x21, x5, #0x1 + stur w8, [x29, #-64] +.LBB2_10: + ldr x8, [sp, #40] + ldr x25, [x24] + ldur w4, [x29, #-64] + ldp q1, q0, [x8] + mov x8, x22 + stp q1, q0, [x29, #-48] +.LBB2_11: + subs x23, x8, #1 + b.eq .LBB2_13 + cbnz x8, .LBB2_14 + b .LBB2_15 +.LBB2_13: + orr w4, w4, w27 +.LBB2_14: + sub x0, x29, #48 + mov w2, #64 + mov x1, x25 + mov x3, x20 + bl zfs_blake3_compress_in_place_sse41 + add x25, x25, #64 + mov x8, x23 + mov w4, w19 + b .LBB2_11 +.LBB2_15: + ldp q0, q1, [x29, #-48] + add x20, x20, x21 + add x24, x24, #8 + subs x28, x28, #1 + stp q0, q1, [x26], #32 + b.ne .LBB2_10 +.LBB2_16: + add sp, sp, #448 + ldp x20, x19, [sp, #144] + ldp x22, x21, [sp, #128] + ldp x24, x23, [sp, #112] + ldp x26, x25, [sp, #96] + ldp x28, x27, [sp, #80] + ldp x29, x30, [sp, #64] + ldp d9, d8, [sp, #48] + ldp d11, d10, [sp, #32] + ldp d13, d12, [sp, #16] + ldp d15, d14, [sp], #160 + ret +.Lfunc_end2: + .size zfs_blake3_hash_many_sse41, .Lfunc_end2-zfs_blake3_hash_many_sse41 + .cfi_endproc + .section ".note.GNU-stack","",@progbits +#endif diff --git a/module/icp/asm-ppc64/blake3/b3_ppc64le_sse2.S b/module/icp/asm-ppc64/blake3/b3_ppc64le_sse2.S new file mode 100644 index 000000000000..ae8d0fad7c83 --- /dev/null +++ b/module/icp/asm-ppc64/blake3/b3_ppc64le_sse2.S @@ -0,0 +1,2823 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Based on BLAKE3 v1.3.1, https://github.com/BLAKE3-team/BLAKE3 + * Copyright (c) 2019-2022 Samuel Neves and Matthew Krupcale + * Copyright (c) 2022 Tino Reichardt + * + * This is converted assembly: SSE2 -> POWER8 PPC64 Little Endian + * Used tools: SIMDe https://github.com/simd-everywhere/simde + */ + +#if (defined(__PPC64__) && defined(__LITTLE_ENDIAN__)) + .text + .abiversion 2 + .section .rodata.cst16,"aM",@progbits,16 + .p2align 4 +.LCPI0_0: + .byte 29 + .byte 28 + .byte 31 + .byte 30 + .byte 25 + .byte 24 + .byte 27 + .byte 26 + .byte 21 + .byte 20 + .byte 23 + .byte 22 + .byte 17 + .byte 16 + .byte 19 + .byte 18 +.LCPI0_1: + .long 1779033703 + .long 3144134277 + .long 1013904242 + .long 2773480762 +.LCPI0_2: + .byte 27 + .byte 26 + .byte 25 + .byte 24 + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 3 + .byte 2 + .byte 1 + .byte 0 +.LCPI0_3: + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .byte 27 + .byte 26 + .byte 25 + .byte 24 +.LCPI0_4: + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 27 + .byte 26 + .byte 25 + .byte 24 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 3 + .byte 2 + .byte 1 + .byte 0 +.LCPI0_5: + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .byte 3 + .byte 2 + .byte 1 + .byte 0 +.LCPI0_6: + .short 1 + .short 2 + .short 4 + .short 8 + .short 16 + .short 32 + .short 64 + .short 128 +.LCPI0_7: + .short 0 + .short 0 + .short 4 + .short 8 + .short 0 + .short 0 + .short 64 + .short 128 +.LCPI0_8: + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 31 + .byte 30 + .byte 29 + .byte 28 + .byte 31 + .byte 30 + .byte 29 + .byte 28 +.LCPI0_9: + .short 0 + .short 0 + .short 0 + .short 0 + .short 0 + .short 0 + .short 64 + .short 128 +.LCPI0_10: + .byte 31 + .byte 30 + .byte 29 + .byte 28 + .byte 7 + .byte 6 + .byte 5 + .byte 4 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .byte 27 + .byte 26 + .byte 25 + .byte 24 +.LCPI0_11: + .byte 31 + .byte 30 + .byte 29 + .byte 28 + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 27 + .byte 26 + .byte 25 + .byte 24 +.LCPI0_12: + .byte 27 + .byte 26 + .byte 25 + .byte 24 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 15 + .byte 14 + .byte 13 + .byte 12 + .byte 31 + .byte 30 + .byte 29 + .byte 28 +.LCPI0_13: + .byte 31 + .byte 30 + .byte 29 + .byte 28 + .byte 15 + .byte 14 + .byte 13 + .byte 12 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 27 + .byte 26 + .byte 25 + .byte 24 +.LCPI0_14: + .byte 27 + .byte 26 + .byte 25 + .byte 24 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .text + .globl zfs_blake3_compress_in_place_sse2 + .p2align 2 + .type zfs_blake3_compress_in_place_sse2,@function +zfs_blake3_compress_in_place_sse2: +.Lfunc_begin0: + .cfi_startproc +.Lfunc_gep0: + addis 2, 12, .TOC.-.Lfunc_gep0@ha + addi 2, 2, .TOC.-.Lfunc_gep0@l +.Lfunc_lep0: + .localentry zfs_blake3_compress_in_place_sse2, .Lfunc_lep0-.Lfunc_gep0 + li 8, -64 + mtvsrd 35, 5 + li 5, 16 + lfdx 0, 0, 4 + vspltisw 12, 9 + stxvd2x 60, 1, 8 + li 8, -48 + mtvsrd 36, 7 + lfd 2, 16(4) + stxvd2x 61, 1, 8 + li 8, -32 + lfd 1, 8(4) + mtvsrwz 37, 6 + rldicl 6, 6, 32, 32 + addis 7, 2, .LCPI0_2@toc@ha + stxvd2x 62, 1, 8 + li 8, -16 + addi 7, 7, .LCPI0_2@toc@l + stxvd2x 63, 1, 8 + li 8, 0 + lvx 9, 0, 7 + li 7, 48 + mtvsrd 34, 8 + xxmrghd 32, 1, 0 + lxvd2x 0, 0, 3 + lxvd2x 1, 3, 5 + lfd 3, 24(4) + addis 8, 2, .LCPI0_5@toc@ha + vmrghb 3, 2, 3 + addi 8, 8, .LCPI0_5@toc@l + vmrghb 4, 2, 4 + vspltb 2, 2, 7 + xxmrghd 33, 3, 2 + vpkudum 7, 1, 0 + vmrglh 3, 2, 3 + vmrglh 2, 2, 4 + mtvsrwz 36, 6 + addis 6, 2, .LCPI0_0@toc@ha + addi 6, 6, .LCPI0_0@toc@l + vperm 10, 1, 0, 9 + vmrghw 4, 4, 5 + xxswapd 37, 1 + lxvd2x 1, 4, 7 + addis 7, 2, .LCPI0_8@toc@ha + addi 7, 7, .LCPI0_8@toc@l + vmrglw 2, 2, 3 + xxswapd 35, 0 + xxswapd 41, 1 + xxspltd 62, 42, 1 + vadduwm 3, 7, 3 + vadduwm 6, 3, 5 + xxmrgld 36, 34, 36 + lvx 2, 0, 6 + addis 6, 2, .LCPI0_1@toc@ha + addi 6, 6, .LCPI0_1@toc@l + xxlxor 35, 38, 36 + lvx 4, 0, 6 + li 6, 32 + lxvd2x 0, 4, 6 + addis 4, 2, .LCPI0_3@toc@ha + addis 6, 2, .LCPI0_7@toc@ha + vperm 8, 3, 3, 2 + vspltisw 3, 10 + addi 4, 4, .LCPI0_3@toc@l + addi 6, 6, .LCPI0_7@toc@l + vadduwm 3, 3, 3 + vadduwm 11, 8, 4 + xxlxor 36, 43, 37 + vadduwm 5, 6, 10 + vrlw 0, 4, 3 + vspltisw 4, 12 + vadduwm 4, 4, 4 + vadduwm 1, 0, 5 + xxlxor 37, 33, 40 + xxswapd 40, 0 + vrlw 6, 5, 4 + vspltisw 5, -16 + vpkudum 13, 9, 8 + vsubuwm 5, 12, 5 + lvx 12, 0, 4 + addis 4, 2, .LCPI0_4@toc@ha + addi 4, 4, .LCPI0_4@toc@l + vadduwm 11, 6, 11 + xxswapd 0, 38 + vadduwm 1, 1, 13 + xxsldwi 50, 45, 45, 1 + xxlxor 32, 43, 32 + xxsldwi 43, 43, 43, 3 + xxsldwi 33, 33, 33, 1 + vperm 12, 8, 9, 12 + vrlw 0, 0, 5 + vadduwm 1, 0, 1 + xxlxor 38, 33, 0 + vadduwm 1, 1, 12 + vperm 6, 6, 6, 2 + vadduwm 15, 6, 11 + lvx 11, 0, 4 + addis 4, 2, .LCPI0_6@toc@ha + addi 4, 4, .LCPI0_6@toc@l + xxlxor 32, 47, 32 + lvx 17, 0, 4 + addis 4, 2, .LCPI0_9@toc@ha + vperm 14, 10, 7, 11 + addi 4, 4, .LCPI0_9@toc@l + vrlw 0, 0, 3 + vadduwm 1, 0, 1 + xxlxor 38, 33, 38 + vrlw 6, 6, 4 + vadduwm 8, 6, 15 + xxswapd 0, 38 + lvx 6, 0, 8 + xxlxor 32, 40, 32 + xxsldwi 40, 40, 40, 1 + vperm 13, 12, 18, 6 + vrlw 9, 0, 5 + vadduwm 0, 1, 14 + lvx 1, 0, 7 + xxsldwi 46, 46, 46, 3 + xxsldwi 32, 32, 32, 3 + vperm 7, 7, 7, 1 + vadduwm 15, 9, 0 + xxlxor 32, 47, 0 + vperm 16, 0, 0, 2 + lvx 0, 0, 6 + addis 6, 2, .LCPI0_10@toc@ha + vcmpequh 0, 0, 17 + vadduwm 19, 16, 8 + xxlxor 40, 51, 41 + xxsel 45, 39, 45, 32 + vrlw 31, 8, 3 + lvx 8, 0, 4 + addis 4, 2, .LCPI0_11@toc@ha + addi 4, 4, .LCPI0_11@toc@l + vcmpequh 7, 8, 17 + vadduwm 8, 15, 13 + vadduwm 15, 31, 8 + lvx 8, 0, 4 + addi 4, 6, .LCPI0_10@toc@l + lvx 17, 0, 4 + addis 4, 2, .LCPI0_12@toc@ha + xxlxor 41, 47, 48 + xxsldwi 47, 47, 47, 1 + addi 4, 4, .LCPI0_12@toc@l + xxlnor 48, 39, 39 + vrlw 29, 9, 4 + vperm 9, 16, 16, 8 + xxland 48, 50, 39 + vperm 17, 30, 12, 17 + vperm 16, 16, 16, 8 + vmrghw 12, 12, 10 + lvx 10, 0, 4 + addis 4, 2, .LCPI0_13@toc@ha + vadduwm 19, 29, 19 + addi 4, 4, .LCPI0_13@toc@l + xxlxor 63, 51, 63 + xxsldwi 51, 51, 51, 3 + xxland 0, 49, 41 + vrlw 17, 31, 5 + xxlor 48, 0, 48 + xxswapd 0, 61 + vperm 18, 12, 18, 10 + vadduwm 15, 15, 16 + xxland 60, 48, 39 + vadduwm 15, 17, 15 + vperm 28, 28, 28, 8 + xxlxor 63, 47, 0 + vadduwm 15, 15, 18 + vperm 31, 31, 31, 2 + vperm 30, 18, 16, 6 + vadduwm 19, 31, 19 + xxlxor 44, 51, 49 + vrlw 12, 12, 3 + vadduwm 15, 12, 15 + xxlxor 49, 47, 63 + vperm 31, 13, 14, 11 + vrlw 17, 17, 4 + vperm 14, 14, 14, 1 + vadduwm 15, 15, 31 + vadduwm 19, 17, 19 + xxswapd 0, 49 + xxsldwi 47, 47, 47, 3 + xxsel 46, 46, 62, 32 + xxlxor 44, 51, 44 + xxsldwi 51, 51, 51, 1 + vrlw 12, 12, 5 + vadduwm 15, 12, 15 + xxlxor 49, 47, 0 + vperm 17, 17, 17, 2 + vadduwm 19, 17, 19 + xxlxor 44, 51, 44 + vrlw 29, 12, 3 + vadduwm 12, 15, 14 + vadduwm 15, 29, 12 + lvx 12, 0, 4 + addis 4, 2, .LCPI0_14@toc@ha + addi 4, 4, .LCPI0_14@toc@l + xxlxor 49, 47, 49 + xxsldwi 47, 47, 47, 1 + vperm 30, 13, 18, 12 + vrlw 17, 17, 4 + vmrghw 13, 18, 13 + xxland 0, 62, 41 + vadduwm 19, 17, 19 + vperm 16, 13, 16, 10 + xxlxor 61, 51, 61 + xxsldwi 50, 51, 51, 3 + xxsldwi 51, 63, 63, 3 + vrlw 30, 29, 5 + xxlor 61, 60, 0 + xxswapd 0, 49 + vperm 31, 14, 19, 11 + vadduwm 15, 15, 29 + vperm 19, 19, 19, 1 + vadduwm 15, 30, 15 + xxlxor 49, 47, 0 + vadduwm 15, 15, 16 + vperm 17, 17, 17, 2 + vadduwm 18, 17, 18 + xxlxor 45, 50, 62 + vperm 30, 16, 29, 6 + vrlw 13, 13, 3 + vadduwm 15, 13, 15 + xxlxor 49, 47, 49 + vadduwm 15, 15, 31 + xxsldwi 63, 63, 63, 3 + vrlw 17, 17, 4 + xxsldwi 47, 47, 47, 3 + vadduwm 18, 17, 18 + xxswapd 0, 49 + xxlxor 45, 50, 45 + xxsldwi 50, 50, 50, 1 + vrlw 13, 13, 5 + vadduwm 15, 13, 15 + xxlxor 49, 47, 0 + vperm 17, 17, 17, 2 + vadduwm 18, 17, 18 + xxlxor 45, 50, 45 + vrlw 28, 13, 3 + xxsel 45, 51, 62, 32 + xxland 51, 61, 39 + vperm 30, 14, 16, 12 + vadduwm 15, 15, 13 + vperm 19, 19, 19, 8 + vmrghw 14, 16, 14 + vadduwm 15, 28, 15 + xxlxor 49, 47, 49 + xxsldwi 47, 47, 47, 1 + xxland 0, 62, 41 + vrlw 17, 17, 4 + xxlor 51, 51, 0 + vadduwm 15, 15, 19 + vadduwm 18, 17, 18 + xxswapd 0, 49 + xxlxor 60, 50, 60 + xxsldwi 48, 50, 50, 3 + vperm 18, 14, 29, 10 + vrlw 30, 28, 5 + vperm 29, 18, 19, 6 + vadduwm 15, 30, 15 + xxlxor 49, 47, 0 + vadduwm 15, 15, 18 + vperm 17, 17, 17, 2 + vadduwm 16, 17, 16 + xxlxor 46, 48, 62 + vperm 30, 13, 31, 11 + vrlw 14, 14, 3 + vperm 31, 31, 31, 1 + vadduwm 15, 14, 15 + xxlxor 49, 47, 49 + vadduwm 15, 15, 30 + vrlw 17, 17, 4 + xxsldwi 47, 47, 47, 3 + vadduwm 16, 17, 16 + xxswapd 0, 49 + xxlxor 46, 48, 46 + xxsldwi 48, 48, 48, 1 + vrlw 14, 14, 5 + vadduwm 15, 14, 15 + xxlxor 49, 47, 0 + vperm 17, 17, 17, 2 + vadduwm 16, 17, 16 + xxlxor 46, 48, 46 + vrlw 28, 14, 3 + xxsel 46, 63, 61, 32 + xxland 63, 51, 39 + vperm 29, 13, 18, 12 + vadduwm 15, 15, 14 + vperm 31, 31, 31, 8 + vmrghw 13, 18, 13 + vadduwm 15, 28, 15 + xxlxor 49, 47, 49 + xxsldwi 47, 47, 47, 1 + xxland 0, 61, 41 + vrlw 17, 17, 4 + xxlor 63, 63, 0 + vperm 13, 13, 19, 10 + xxsldwi 51, 62, 62, 3 + vadduwm 15, 15, 31 + vperm 30, 14, 19, 11 + vadduwm 16, 17, 16 + xxswapd 0, 49 + xxlxor 60, 48, 60 + xxsldwi 48, 48, 48, 3 + vrlw 29, 28, 5 + vadduwm 15, 29, 15 + xxlxor 49, 47, 0 + vadduwm 15, 15, 13 + vperm 17, 17, 17, 2 + vadduwm 16, 17, 16 + xxlxor 50, 48, 61 + vrlw 18, 18, 3 + vadduwm 15, 18, 15 + xxlxor 49, 47, 49 + vadduwm 15, 15, 30 + vrlw 17, 17, 4 + xxsldwi 47, 47, 47, 3 + vadduwm 11, 17, 16 + xxswapd 0, 49 + xxlxor 48, 43, 50 + xxsldwi 43, 43, 43, 1 + vperm 18, 19, 19, 1 + vrlw 16, 16, 5 + vperm 19, 13, 31, 6 + vadduwm 15, 16, 15 + xxlxor 49, 47, 0 + vperm 17, 17, 17, 2 + vadduwm 29, 17, 11 + xxlxor 43, 61, 48 + vrlw 16, 11, 3 + xxsel 43, 50, 51, 32 + xxland 50, 63, 39 + vperm 19, 14, 13, 12 + vadduwm 15, 15, 11 + vperm 18, 18, 18, 8 + vmrghw 13, 13, 14 + vadduwm 15, 16, 15 + xxlxor 49, 47, 49 + xxsldwi 47, 47, 47, 1 + xxland 0, 51, 41 + lvx 19, 0, 4 + vrlw 17, 17, 4 + xxlor 50, 50, 0 + vperm 13, 13, 31, 10 + xxsldwi 63, 62, 62, 3 + vadduwm 15, 15, 18 + vperm 19, 11, 31, 19 + vadduwm 29, 17, 29 + xxswapd 0, 49 + vperm 1, 31, 31, 1 + xxlxor 48, 61, 48 + xxsldwi 46, 61, 61, 3 + vperm 6, 13, 18, 6 + vrlw 16, 16, 5 + xxsel 32, 33, 38, 32 + xxland 38, 50, 39 + vadduwm 15, 16, 15 + vperm 7, 11, 13, 12 + xxlxor 49, 47, 0 + vadduwm 15, 15, 13 + vperm 17, 17, 17, 2 + vperm 6, 6, 6, 8 + vadduwm 14, 17, 14 + xxlxor 48, 46, 48 + vrlw 16, 16, 3 + vadduwm 15, 16, 15 + xxlxor 49, 47, 49 + xxsldwi 47, 47, 47, 3 + vrlw 17, 17, 4 + vadduwm 15, 15, 19 + vadduwm 14, 17, 14 + xxswapd 0, 49 + xxlxor 48, 46, 48 + xxsldwi 46, 46, 46, 1 + vrlw 16, 16, 5 + vadduwm 15, 16, 15 + xxlxor 49, 47, 0 + vadduwm 0, 15, 0 + vperm 17, 17, 17, 2 + xxland 0, 39, 41 + xxlor 38, 38, 0 + vadduwm 14, 17, 14 + xxlxor 48, 46, 48 + vrlw 16, 16, 3 + vadduwm 0, 16, 0 + xxlxor 33, 32, 49 + xxsldwi 32, 32, 32, 1 + vrlw 1, 1, 4 + vadduwm 0, 0, 6 + vadduwm 8, 1, 14 + xxswapd 0, 33 + xxlxor 44, 40, 48 + xxsldwi 38, 40, 40, 3 + vrlw 7, 12, 5 + vadduwm 0, 7, 0 + xxlxor 33, 32, 0 + vperm 2, 1, 1, 2 + vmrghw 1, 13, 11 + vadduwm 6, 2, 6 + vperm 1, 1, 18, 10 + xxlxor 39, 38, 39 + vrlw 3, 7, 3 + vadduwm 0, 0, 1 + vadduwm 0, 3, 0 + xxlxor 34, 32, 34 + xxsldwi 0, 32, 32, 3 + vrlw 2, 2, 4 + vadduwm 4, 2, 6 + xxswapd 2, 34 + xxlxor 35, 36, 35 + xxsldwi 1, 36, 36, 1 + vrlw 3, 3, 5 + xxlxor 0, 1, 0 + xxswapd 0, 0 + xxlxor 1, 35, 2 + stxvd2x 0, 0, 3 + xxswapd 1, 1 + stxvd2x 1, 3, 5 + li 3, -16 + lxvd2x 63, 1, 3 + li 3, -32 + lxvd2x 62, 1, 3 + li 3, -48 + lxvd2x 61, 1, 3 + li 3, -64 + lxvd2x 60, 1, 3 + blr + .long 0 + .quad 0 +.Lfunc_end0: + .size zfs_blake3_compress_in_place_sse2, .Lfunc_end0-.Lfunc_begin0 + .cfi_endproc + + .section .rodata.cst16,"aM",@progbits,16 + .p2align 4 +.LCPI1_0: + .byte 29 + .byte 28 + .byte 31 + .byte 30 + .byte 25 + .byte 24 + .byte 27 + .byte 26 + .byte 21 + .byte 20 + .byte 23 + .byte 22 + .byte 17 + .byte 16 + .byte 19 + .byte 18 +.LCPI1_1: + .long 1779033703 + .long 3144134277 + .long 1013904242 + .long 2773480762 +.LCPI1_2: + .byte 27 + .byte 26 + .byte 25 + .byte 24 + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 3 + .byte 2 + .byte 1 + .byte 0 +.LCPI1_3: + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .byte 27 + .byte 26 + .byte 25 + .byte 24 +.LCPI1_4: + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 27 + .byte 26 + .byte 25 + .byte 24 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 3 + .byte 2 + .byte 1 + .byte 0 +.LCPI1_5: + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .byte 3 + .byte 2 + .byte 1 + .byte 0 +.LCPI1_6: + .short 1 + .short 2 + .short 4 + .short 8 + .short 16 + .short 32 + .short 64 + .short 128 +.LCPI1_7: + .short 0 + .short 0 + .short 4 + .short 8 + .short 0 + .short 0 + .short 64 + .short 128 +.LCPI1_8: + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 31 + .byte 30 + .byte 29 + .byte 28 + .byte 31 + .byte 30 + .byte 29 + .byte 28 +.LCPI1_9: + .short 0 + .short 0 + .short 0 + .short 0 + .short 0 + .short 0 + .short 64 + .short 128 +.LCPI1_10: + .byte 31 + .byte 30 + .byte 29 + .byte 28 + .byte 7 + .byte 6 + .byte 5 + .byte 4 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .byte 27 + .byte 26 + .byte 25 + .byte 24 +.LCPI1_11: + .byte 31 + .byte 30 + .byte 29 + .byte 28 + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 27 + .byte 26 + .byte 25 + .byte 24 +.LCPI1_12: + .byte 27 + .byte 26 + .byte 25 + .byte 24 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 15 + .byte 14 + .byte 13 + .byte 12 + .byte 31 + .byte 30 + .byte 29 + .byte 28 +.LCPI1_13: + .byte 31 + .byte 30 + .byte 29 + .byte 28 + .byte 15 + .byte 14 + .byte 13 + .byte 12 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 27 + .byte 26 + .byte 25 + .byte 24 +.LCPI1_14: + .byte 27 + .byte 26 + .byte 25 + .byte 24 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .text + .globl zfs_blake3_compress_xof_sse2 + .p2align 2 + .type zfs_blake3_compress_xof_sse2,@function +zfs_blake3_compress_xof_sse2: +.Lfunc_begin1: + .cfi_startproc +.Lfunc_gep1: + addis 2, 12, .TOC.-.Lfunc_gep1@ha + addi 2, 2, .TOC.-.Lfunc_gep1@l +.Lfunc_lep1: + .localentry zfs_blake3_compress_xof_sse2, .Lfunc_lep1-.Lfunc_gep1 + li 9, -80 + mtvsrd 35, 5 + li 5, 16 + lfdx 0, 0, 4 + addis 10, 2, .LCPI1_2@toc@ha + vspltisw 12, 9 + std 30, -16(1) + addis 12, 2, .LCPI1_8@toc@ha + addis 30, 2, .LCPI1_5@toc@ha + addis 11, 2, .LCPI1_7@toc@ha + stxvd2x 60, 1, 9 + li 9, -64 + mtvsrd 36, 7 + lfd 2, 16(4) + addi 10, 10, .LCPI1_2@toc@l + addi 12, 12, .LCPI1_8@toc@l + addi 11, 11, .LCPI1_7@toc@l + stxvd2x 61, 1, 9 + li 9, -48 + lfd 3, 24(4) + mtvsrwz 37, 6 + rldicl 6, 6, 32, 32 + lvx 9, 0, 10 + stxvd2x 62, 1, 9 + li 9, -32 + li 10, 32 + stxvd2x 63, 1, 9 + li 9, 0 + mtvsrd 34, 9 + xxmrghd 33, 3, 2 + lfd 1, 8(4) + vmrghb 3, 2, 3 + vmrghb 4, 2, 4 + vspltb 2, 2, 7 + xxmrghd 32, 1, 0 + lxvd2x 0, 0, 3 + lxvd2x 1, 3, 5 + vpkudum 7, 1, 0 + vmrglh 3, 2, 3 + vmrglh 2, 2, 4 + mtvsrwz 36, 6 + addis 6, 2, .LCPI1_0@toc@ha + addi 6, 6, .LCPI1_0@toc@l + vperm 10, 1, 0, 9 + vmrghw 4, 4, 5 + xxswapd 37, 1 + vmrglw 2, 2, 3 + xxswapd 35, 0 + lxvd2x 0, 4, 10 + xxspltd 62, 42, 1 + vadduwm 3, 7, 3 + vadduwm 6, 3, 5 + xxmrgld 36, 34, 36 + lvx 2, 0, 6 + addis 6, 2, .LCPI1_1@toc@ha + addi 6, 6, .LCPI1_1@toc@l + xxlxor 35, 38, 36 + lvx 4, 0, 6 + li 6, 48 + lxvd2x 1, 4, 6 + addis 4, 2, .LCPI1_3@toc@ha + vperm 8, 3, 3, 2 + vspltisw 3, 10 + addi 4, 4, .LCPI1_3@toc@l + xxswapd 41, 1 + vadduwm 3, 3, 3 + vadduwm 11, 8, 4 + xxlxor 36, 43, 37 + vadduwm 5, 6, 10 + vrlw 0, 4, 3 + vspltisw 4, 12 + vadduwm 4, 4, 4 + vadduwm 1, 0, 5 + xxlxor 37, 33, 40 + xxswapd 40, 0 + vrlw 6, 5, 4 + vspltisw 5, -16 + vpkudum 13, 9, 8 + vsubuwm 5, 12, 5 + lvx 12, 0, 4 + addis 4, 2, .LCPI1_4@toc@ha + addi 4, 4, .LCPI1_4@toc@l + vadduwm 11, 6, 11 + xxswapd 0, 38 + vadduwm 1, 1, 13 + xxsldwi 50, 45, 45, 1 + xxlxor 32, 43, 32 + xxsldwi 43, 43, 43, 3 + xxsldwi 33, 33, 33, 1 + vperm 12, 8, 9, 12 + vrlw 0, 0, 5 + vadduwm 1, 0, 1 + xxlxor 38, 33, 0 + vadduwm 1, 1, 12 + vperm 6, 6, 6, 2 + vadduwm 15, 6, 11 + lvx 11, 0, 4 + addis 4, 2, .LCPI1_6@toc@ha + addi 4, 4, .LCPI1_6@toc@l + xxlxor 32, 47, 32 + lvx 17, 0, 4 + addi 4, 30, .LCPI1_5@toc@l + vperm 14, 10, 7, 11 + vrlw 0, 0, 3 + vadduwm 1, 0, 1 + xxlxor 38, 33, 38 + vrlw 6, 6, 4 + vadduwm 8, 6, 15 + xxswapd 0, 38 + lvx 6, 0, 4 + addis 4, 2, .LCPI1_9@toc@ha + addi 4, 4, .LCPI1_9@toc@l + xxlxor 32, 40, 32 + xxsldwi 40, 40, 40, 1 + vperm 13, 12, 18, 6 + vrlw 9, 0, 5 + vadduwm 0, 1, 14 + lvx 1, 0, 12 + xxsldwi 46, 46, 46, 3 + xxsldwi 32, 32, 32, 3 + vperm 7, 7, 7, 1 + vadduwm 15, 9, 0 + xxlxor 32, 47, 0 + vperm 16, 0, 0, 2 + lvx 0, 0, 11 + addis 11, 2, .LCPI1_10@toc@ha + vcmpequh 0, 0, 17 + vadduwm 19, 16, 8 + xxlxor 40, 51, 41 + xxsel 45, 39, 45, 32 + vrlw 31, 8, 3 + lvx 8, 0, 4 + addis 4, 2, .LCPI1_11@toc@ha + addi 4, 4, .LCPI1_11@toc@l + vcmpequh 7, 8, 17 + vadduwm 8, 15, 13 + vadduwm 15, 31, 8 + lvx 8, 0, 4 + addi 4, 11, .LCPI1_10@toc@l + lvx 17, 0, 4 + addis 4, 2, .LCPI1_12@toc@ha + xxlxor 41, 47, 48 + xxsldwi 47, 47, 47, 1 + addi 4, 4, .LCPI1_12@toc@l + xxlnor 48, 39, 39 + vrlw 29, 9, 4 + vperm 9, 16, 16, 8 + xxland 48, 50, 39 + vperm 17, 30, 12, 17 + vperm 16, 16, 16, 8 + vmrghw 12, 12, 10 + lvx 10, 0, 4 + addis 4, 2, .LCPI1_13@toc@ha + vadduwm 19, 29, 19 + addi 4, 4, .LCPI1_13@toc@l + xxlxor 63, 51, 63 + xxsldwi 51, 51, 51, 3 + xxland 0, 49, 41 + vrlw 17, 31, 5 + xxlor 48, 0, 48 + xxswapd 0, 61 + vperm 18, 12, 18, 10 + vadduwm 15, 15, 16 + xxland 60, 48, 39 + vadduwm 15, 17, 15 + vperm 28, 28, 28, 8 + xxlxor 63, 47, 0 + vadduwm 15, 15, 18 + vperm 31, 31, 31, 2 + vperm 30, 18, 16, 6 + vadduwm 19, 31, 19 + xxlxor 44, 51, 49 + vrlw 12, 12, 3 + vadduwm 15, 12, 15 + xxlxor 49, 47, 63 + vperm 31, 13, 14, 11 + vrlw 17, 17, 4 + vperm 14, 14, 14, 1 + vadduwm 15, 15, 31 + vadduwm 19, 17, 19 + xxswapd 0, 49 + xxsldwi 47, 47, 47, 3 + xxsel 46, 46, 62, 32 + xxlxor 44, 51, 44 + xxsldwi 51, 51, 51, 1 + vrlw 12, 12, 5 + vadduwm 15, 12, 15 + xxlxor 49, 47, 0 + vperm 17, 17, 17, 2 + vadduwm 19, 17, 19 + xxlxor 44, 51, 44 + vrlw 29, 12, 3 + vadduwm 12, 15, 14 + vadduwm 15, 29, 12 + lvx 12, 0, 4 + addis 4, 2, .LCPI1_14@toc@ha + addi 4, 4, .LCPI1_14@toc@l + xxlxor 49, 47, 49 + xxsldwi 47, 47, 47, 1 + vperm 30, 13, 18, 12 + vrlw 17, 17, 4 + vmrghw 13, 18, 13 + xxland 0, 62, 41 + vadduwm 19, 17, 19 + vperm 16, 13, 16, 10 + xxlxor 61, 51, 61 + xxsldwi 50, 51, 51, 3 + xxsldwi 51, 63, 63, 3 + vrlw 30, 29, 5 + xxlor 61, 60, 0 + xxswapd 0, 49 + vperm 31, 14, 19, 11 + vadduwm 15, 15, 29 + vperm 19, 19, 19, 1 + vadduwm 15, 30, 15 + xxlxor 49, 47, 0 + vadduwm 15, 15, 16 + vperm 17, 17, 17, 2 + vadduwm 18, 17, 18 + xxlxor 45, 50, 62 + vperm 30, 16, 29, 6 + vrlw 13, 13, 3 + vadduwm 15, 13, 15 + xxlxor 49, 47, 49 + vadduwm 15, 15, 31 + xxsldwi 63, 63, 63, 3 + vrlw 17, 17, 4 + xxsldwi 47, 47, 47, 3 + vadduwm 18, 17, 18 + xxswapd 0, 49 + xxlxor 45, 50, 45 + xxsldwi 50, 50, 50, 1 + vrlw 13, 13, 5 + vadduwm 15, 13, 15 + xxlxor 49, 47, 0 + vperm 17, 17, 17, 2 + vadduwm 18, 17, 18 + xxlxor 45, 50, 45 + vrlw 28, 13, 3 + xxsel 45, 51, 62, 32 + xxland 51, 61, 39 + vperm 30, 14, 16, 12 + vadduwm 15, 15, 13 + vperm 19, 19, 19, 8 + vmrghw 14, 16, 14 + vadduwm 15, 28, 15 + xxlxor 49, 47, 49 + xxsldwi 47, 47, 47, 1 + xxland 0, 62, 41 + vrlw 17, 17, 4 + xxlor 51, 51, 0 + vadduwm 15, 15, 19 + vadduwm 18, 17, 18 + xxswapd 0, 49 + xxlxor 60, 50, 60 + xxsldwi 48, 50, 50, 3 + vperm 18, 14, 29, 10 + vrlw 30, 28, 5 + vperm 29, 18, 19, 6 + vadduwm 15, 30, 15 + xxlxor 49, 47, 0 + vadduwm 15, 15, 18 + vperm 17, 17, 17, 2 + vadduwm 16, 17, 16 + xxlxor 46, 48, 62 + vperm 30, 13, 31, 11 + vrlw 14, 14, 3 + vperm 31, 31, 31, 1 + vadduwm 15, 14, 15 + xxlxor 49, 47, 49 + vadduwm 15, 15, 30 + vrlw 17, 17, 4 + xxsldwi 47, 47, 47, 3 + vadduwm 16, 17, 16 + xxswapd 0, 49 + xxlxor 46, 48, 46 + xxsldwi 48, 48, 48, 1 + vrlw 14, 14, 5 + vadduwm 15, 14, 15 + xxlxor 49, 47, 0 + vperm 17, 17, 17, 2 + vadduwm 16, 17, 16 + xxlxor 46, 48, 46 + vrlw 28, 14, 3 + xxsel 46, 63, 61, 32 + xxland 63, 51, 39 + vperm 29, 13, 18, 12 + vadduwm 15, 15, 14 + vperm 31, 31, 31, 8 + vmrghw 13, 18, 13 + vadduwm 15, 28, 15 + xxlxor 49, 47, 49 + xxsldwi 47, 47, 47, 1 + xxland 0, 61, 41 + vrlw 17, 17, 4 + xxlor 63, 63, 0 + vperm 13, 13, 19, 10 + xxsldwi 51, 62, 62, 3 + vadduwm 15, 15, 31 + vperm 30, 14, 19, 11 + vadduwm 16, 17, 16 + xxswapd 0, 49 + xxlxor 60, 48, 60 + xxsldwi 48, 48, 48, 3 + vrlw 29, 28, 5 + vadduwm 15, 29, 15 + xxlxor 49, 47, 0 + vadduwm 15, 15, 13 + vperm 17, 17, 17, 2 + vadduwm 16, 17, 16 + xxlxor 50, 48, 61 + vrlw 18, 18, 3 + vadduwm 15, 18, 15 + xxlxor 49, 47, 49 + vadduwm 15, 15, 30 + vrlw 17, 17, 4 + xxsldwi 47, 47, 47, 3 + vadduwm 11, 17, 16 + xxswapd 0, 49 + xxlxor 48, 43, 50 + xxsldwi 43, 43, 43, 1 + vperm 18, 19, 19, 1 + vrlw 16, 16, 5 + vperm 19, 13, 31, 6 + vadduwm 15, 16, 15 + xxlxor 49, 47, 0 + vperm 17, 17, 17, 2 + vadduwm 29, 17, 11 + xxlxor 43, 61, 48 + vrlw 16, 11, 3 + xxsel 43, 50, 51, 32 + xxland 50, 63, 39 + vperm 19, 14, 13, 12 + vadduwm 15, 15, 11 + vperm 18, 18, 18, 8 + vmrghw 13, 13, 14 + vadduwm 15, 16, 15 + xxlxor 49, 47, 49 + xxsldwi 47, 47, 47, 1 + xxland 0, 51, 41 + lvx 19, 0, 4 + vrlw 17, 17, 4 + xxlor 50, 50, 0 + vperm 13, 13, 31, 10 + xxsldwi 63, 62, 62, 3 + vadduwm 15, 15, 18 + vperm 19, 11, 31, 19 + vadduwm 29, 17, 29 + xxswapd 0, 49 + vperm 1, 31, 31, 1 + xxlxor 48, 61, 48 + xxsldwi 46, 61, 61, 3 + vperm 6, 13, 18, 6 + vrlw 16, 16, 5 + xxsel 32, 33, 38, 32 + xxland 38, 50, 39 + vadduwm 15, 16, 15 + vperm 7, 11, 13, 12 + xxlxor 49, 47, 0 + vadduwm 15, 15, 13 + vperm 17, 17, 17, 2 + vperm 6, 6, 6, 8 + vadduwm 14, 17, 14 + xxlxor 48, 46, 48 + vrlw 16, 16, 3 + vadduwm 15, 16, 15 + xxlxor 49, 47, 49 + xxsldwi 47, 47, 47, 3 + vrlw 17, 17, 4 + vadduwm 15, 15, 19 + vadduwm 14, 17, 14 + xxswapd 0, 49 + xxlxor 48, 46, 48 + xxsldwi 46, 46, 46, 1 + vrlw 16, 16, 5 + vadduwm 15, 16, 15 + xxlxor 49, 47, 0 + vadduwm 0, 15, 0 + vperm 17, 17, 17, 2 + xxland 0, 39, 41 + xxlor 38, 38, 0 + vadduwm 14, 17, 14 + xxlxor 48, 46, 48 + vrlw 16, 16, 3 + vadduwm 0, 16, 0 + xxlxor 33, 32, 49 + xxsldwi 32, 32, 32, 1 + vrlw 1, 1, 4 + vadduwm 0, 0, 6 + vadduwm 8, 1, 14 + xxswapd 0, 33 + xxlxor 44, 40, 48 + xxsldwi 38, 40, 40, 3 + vrlw 7, 12, 5 + vadduwm 0, 7, 0 + xxlxor 33, 32, 0 + vperm 2, 1, 1, 2 + vmrghw 1, 13, 11 + vadduwm 6, 2, 6 + vperm 1, 1, 18, 10 + xxlxor 39, 38, 39 + vrlw 3, 7, 3 + vadduwm 0, 0, 1 + vadduwm 0, 3, 0 + xxlxor 34, 32, 34 + xxsldwi 0, 32, 32, 3 + vrlw 2, 2, 4 + vadduwm 4, 2, 6 + xxswapd 2, 34 + xxlxor 35, 36, 35 + xxsldwi 1, 36, 36, 1 + vrlw 3, 3, 5 + xxlxor 0, 1, 0 + xxswapd 0, 0 + xxlxor 3, 35, 2 + stxvd2x 0, 0, 8 + xxswapd 3, 3 + stxvd2x 3, 8, 5 + lfdx 0, 0, 3 + lfd 3, 8(3) + xxmrghd 34, 3, 0 + xxlxor 0, 1, 34 + xxswapd 0, 0 + stxvd2x 0, 8, 10 + lfd 0, 16(3) + lfd 1, 24(3) + li 3, -32 + xxmrghd 34, 1, 0 + xxlxor 0, 2, 34 + xxswapd 0, 0 + stxvd2x 0, 8, 6 + lxvd2x 63, 1, 3 + li 3, -48 + ld 30, -16(1) + lxvd2x 62, 1, 3 + li 3, -64 + lxvd2x 61, 1, 3 + li 3, -80 + lxvd2x 60, 1, 3 + blr + .long 0 + .quad 0 +.Lfunc_end1: + .size zfs_blake3_compress_xof_sse2, .Lfunc_end1-.Lfunc_begin1 + .cfi_endproc + + .globl zfs_blake3_hash_many_sse2 + .p2align 2 + .type zfs_blake3_hash_many_sse2,@function +zfs_blake3_hash_many_sse2: +.Lfunc_begin2: + .cfi_startproc +.Lfunc_gep2: + addis 2, 12, .TOC.-.Lfunc_gep2@ha + addi 2, 2, .TOC.-.Lfunc_gep2@l +.Lfunc_lep2: + .localentry zfs_blake3_hash_many_sse2, .Lfunc_lep2-.Lfunc_gep2 + mfocrf 12, 32 + mflr 0 + std 0, 16(1) + stw 12, 8(1) + stdu 1, -256(1) + .cfi_def_cfa_offset 256 + .cfi_offset lr, 16 + .cfi_offset r17, -120 + .cfi_offset r18, -112 + .cfi_offset r19, -104 + .cfi_offset r20, -96 + .cfi_offset r21, -88 + .cfi_offset r22, -80 + .cfi_offset r23, -72 + .cfi_offset r24, -64 + .cfi_offset r25, -56 + .cfi_offset r26, -48 + .cfi_offset r27, -40 + .cfi_offset r28, -32 + .cfi_offset r29, -24 + .cfi_offset r30, -16 + .cfi_offset cr2, 8 + std 26, 208(1) + mr 26, 4 + cmpldi 1, 4, 4 + andi. 4, 8, 1 + std 18, 144(1) + std 19, 152(1) + crmove 8, 1 + ld 19, 360(1) + lwz 18, 352(1) + std 24, 192(1) + std 25, 200(1) + std 27, 216(1) + std 28, 224(1) + mr 24, 10 + mr 28, 6 + mr 27, 5 + mr 25, 3 + std 29, 232(1) + std 30, 240(1) + mr 30, 9 + mr 29, 7 + std 17, 136(1) + std 20, 160(1) + std 21, 168(1) + std 22, 176(1) + std 23, 184(1) + blt 1, .LBB2_3 + li 3, 0 + li 4, 1 + clrldi 23, 30, 32 + isel 22, 4, 3, 8 + clrldi 21, 24, 32 + clrldi 20, 18, 32 +.LBB2_2: + mr 3, 25 + mr 4, 27 + mr 5, 28 + mr 6, 29 + mr 7, 22 + mr 8, 23 + mr 9, 21 + mr 10, 20 + std 19, 32(1) + bl blake3_hash4_sse2 + addi 26, 26, -4 + addi 3, 29, 4 + addi 25, 25, 32 + addi 19, 19, 128 + cmpldi 26, 3 + isel 29, 3, 29, 8 + bgt 0, .LBB2_2 +.LBB2_3: + cmpldi 26, 0 + beq 0, .LBB2_11 + li 3, 0 + li 4, 1 + or 21, 24, 30 + li 20, 16 + addi 24, 1, 96 + isel 22, 4, 3, 8 +.LBB2_5: + lxvd2x 0, 28, 20 + ld 23, 0(25) + mr 17, 27 + mr 3, 21 + stxvd2x 0, 24, 20 + lxvd2x 0, 0, 28 + stxvd2x 0, 0, 24 +.LBB2_6: + cmpldi 17, 1 + beq 0, .LBB2_8 + cmpldi 17, 0 + bne 0, .LBB2_9 + b .LBB2_10 +.LBB2_8: + or 3, 3, 18 +.LBB2_9: + clrldi 7, 3, 56 + mr 3, 24 + mr 4, 23 + li 5, 64 + mr 6, 29 + bl zfs_blake3_compress_in_place_sse2 + addi 23, 23, 64 + addi 17, 17, -1 + mr 3, 30 + b .LBB2_6 +.LBB2_10: + lxvd2x 0, 24, 20 + addi 26, 26, -1 + add 29, 29, 22 + addi 25, 25, 8 + cmpldi 26, 0 + stxvd2x 0, 19, 20 + lxvd2x 0, 0, 24 + stxvd2x 0, 0, 19 + addi 19, 19, 32 + bne 0, .LBB2_5 +.LBB2_11: + ld 30, 240(1) + ld 29, 232(1) + ld 28, 224(1) + ld 27, 216(1) + ld 26, 208(1) + ld 25, 200(1) + ld 24, 192(1) + ld 23, 184(1) + ld 22, 176(1) + ld 21, 168(1) + ld 20, 160(1) + ld 19, 152(1) + ld 18, 144(1) + ld 17, 136(1) + addi 1, 1, 256 + ld 0, 16(1) + lwz 12, 8(1) + mtocrf 32, 12 + mtlr 0 + blr + .long 0 + .quad 0 +.Lfunc_end2: + .size zfs_blake3_hash_many_sse2, .Lfunc_end2-.Lfunc_begin2 + .cfi_endproc + + .section .rodata.cst16,"aM",@progbits,16 + .p2align 4 +.LCPI3_0: + .quad 4294967296 + .quad 12884901890 +.LCPI3_1: + .byte 29 + .byte 28 + .byte 31 + .byte 30 + .byte 25 + .byte 24 + .byte 27 + .byte 26 + .byte 21 + .byte 20 + .byte 23 + .byte 22 + .byte 17 + .byte 16 + .byte 19 + .byte 18 +.LCPI3_2: + .long 1779033703 + .long 1779033703 + .long 1779033703 + .long 1779033703 +.LCPI3_3: + .long 3144134277 + .long 3144134277 + .long 3144134277 + .long 3144134277 +.LCPI3_4: + .long 1013904242 + .long 1013904242 + .long 1013904242 + .long 1013904242 +.LCPI3_5: + .long 2773480762 + .long 2773480762 + .long 2773480762 + .long 2773480762 + .text + .p2align 2 + .type blake3_hash4_sse2,@function +blake3_hash4_sse2: +.Lfunc_begin3: + .cfi_startproc +.Lfunc_gep3: + addis 2, 12, .TOC.-.Lfunc_gep3@ha + addi 2, 2, .TOC.-.Lfunc_gep3@l +.Lfunc_lep3: + .localentry blake3_hash4_sse2, .Lfunc_lep3-.Lfunc_gep3 + stdu 1, -400(1) + .cfi_def_cfa_offset 400 + .cfi_offset r22, -152 + .cfi_offset r23, -144 + .cfi_offset r24, -136 + .cfi_offset r25, -128 + .cfi_offset r26, -120 + .cfi_offset r27, -112 + .cfi_offset r28, -104 + .cfi_offset r29, -96 + .cfi_offset r30, -88 + .cfi_offset f23, -72 + .cfi_offset f24, -64 + .cfi_offset f25, -56 + .cfi_offset f26, -48 + .cfi_offset f27, -40 + .cfi_offset f28, -32 + .cfi_offset f29, -24 + .cfi_offset f30, -16 + .cfi_offset f31, -8 + .cfi_offset v20, -352 + .cfi_offset v21, -336 + .cfi_offset v22, -320 + .cfi_offset v23, -304 + .cfi_offset v24, -288 + .cfi_offset v25, -272 + .cfi_offset v26, -256 + .cfi_offset v27, -240 + .cfi_offset v28, -224 + .cfi_offset v29, -208 + .cfi_offset v30, -192 + .cfi_offset v31, -176 + li 11, 48 + li 0, 8 + std 30, 312(1) + li 30, 12 + li 12, 4 + lfiwzx 0, 0, 5 + stxvd2x 52, 1, 11 + li 11, 64 + lfiwzx 2, 5, 0 + li 0, 20 + lfiwzx 3, 5, 30 + stxvd2x 53, 1, 11 + li 11, 80 + li 30, 24 + lfiwzx 4, 5, 0 + li 0, 28 + stxvd2x 54, 1, 11 + li 11, 96 + lfiwzx 1, 5, 12 + lfiwzx 6, 5, 30 + xxspltw 45, 0, 1 + cmpldi 4, 0 + std 22, 248(1) + stxvd2x 55, 1, 11 + li 11, 112 + lfiwzx 7, 5, 0 + xxspltw 40, 2, 1 + std 23, 256(1) + xxspltw 38, 3, 1 + xxspltw 50, 4, 1 + std 24, 264(1) + std 25, 272(1) + std 26, 280(1) + xxspltw 54, 7, 1 + std 27, 288(1) + std 28, 296(1) + std 29, 304(1) + stxvd2x 56, 1, 11 + li 11, 128 + stfd 23, 328(1) + stxvd2x 57, 1, 11 + li 11, 144 + stfd 24, 336(1) + stxvd2x 58, 1, 11 + li 11, 160 + stfd 25, 344(1) + stxvd2x 59, 1, 11 + li 11, 176 + xxspltw 59, 1, 1 + stxvd2x 60, 1, 11 + li 11, 192 + stfd 26, 352(1) + stxvd2x 61, 1, 11 + li 11, 208 + stfd 27, 360(1) + stxvd2x 62, 1, 11 + li 11, 224 + xxspltw 62, 6, 1 + stxvd2x 63, 1, 11 + li 11, 16 + stfd 28, 368(1) + lfiwzx 5, 5, 11 + ld 5, 432(1) + stfd 29, 376(1) + stfd 30, 384(1) + stfd 31, 392(1) + xxspltw 61, 5, 1 + beq 0, .LBB3_5 + addis 30, 2, .LCPI3_0@toc@ha + neg 7, 7 + xxleqv 34, 34, 34 + addis 28, 2, .LCPI3_2@toc@ha + addis 27, 2, .LCPI3_3@toc@ha + addis 26, 2, .LCPI3_4@toc@ha + addis 25, 2, .LCPI3_5@toc@ha + ld 29, 24(3) + addi 0, 30, .LCPI3_0@toc@l + mtfprwz 1, 7 + addis 7, 2, .LCPI3_1@toc@ha + ld 30, 16(3) + lxvd2x 0, 0, 0 + mtfprwz 2, 6 + rldicl 6, 6, 32, 32 + addi 0, 7, .LCPI3_1@toc@l + ld 7, 8(3) + vslw 2, 2, 2 + lvx 5, 0, 0 + addi 0, 28, .LCPI3_2@toc@l + addi 28, 27, .LCPI3_3@toc@l + addi 27, 26, .LCPI3_4@toc@l + addi 26, 25, .LCPI3_5@toc@l + or 25, 9, 8 + li 9, 0 + xxspltw 36, 2, 1 + xxswapd 35, 0 + xxspltw 0, 1, 1 + xxland 35, 0, 35 + mtfprwz 0, 6 + ld 6, 0(3) + addi 3, 3, -8 + vadduwm 4, 3, 4 + xxlor 35, 35, 34 + xxlxor 34, 36, 34 + xxlor 9, 36, 36 + vspltisw 4, 4 + vcmpgtsw 2, 3, 2 + xxspltw 35, 0, 1 + xxlor 10, 36, 36 + vsubuwm 2, 3, 2 + xxlor 11, 34, 34 + lvx 2, 0, 0 + li 0, 32 + xxlor 12, 34, 34 + lvx 2, 0, 28 + li 28, 48 + xxlor 13, 34, 34 + lvx 2, 0, 27 + li 27, 0 + xxlor 31, 34, 34 + lvx 2, 0, 26 + xxlor 30, 34, 34 +.LBB3_2: + mr 26, 27 + addi 27, 27, 1 + xxlor 28, 40, 40 + cmpld 27, 4 + sldi 26, 26, 6 + xxlor 24, 45, 45 + iseleq 24, 10, 9 + add 23, 6, 26 + add 22, 30, 26 + lxvd2x 0, 6, 26 + lxvd2x 1, 7, 26 + or 25, 24, 25 + add 24, 7, 26 + lxvd2x 2, 30, 26 + lxvd2x 3, 29, 26 + xxlor 29, 38, 38 + lxvd2x 4, 23, 11 + lxvd2x 6, 24, 11 + clrlwi 25, 25, 24 + lxvd2x 7, 22, 11 + lxvd2x 8, 23, 0 + mtfprd 5, 25 + add 25, 29, 26 + xxswapd 34, 0 + lxvd2x 0, 25, 11 + xxswapd 36, 1 + xxswapd 33, 2 + lxvd2x 1, 24, 0 + lxvd2x 2, 22, 0 + xxswapd 39, 3 + xxswapd 32, 4 + lxvd2x 3, 25, 0 + lxvd2x 4, 23, 28 + xxswapd 49, 6 + xxswapd 51, 7 + lxvd2x 6, 24, 28 + xxswapd 58, 8 + lxvd2x 7, 22, 28 + lxvd2x 8, 25, 28 + xxswapd 60, 0 + mr 25, 3 + xxswapd 57, 1 + xxswapd 53, 2 + xxswapd 52, 3 + xxswapd 56, 4 + xxswapd 55, 6 + xxswapd 0, 5 + xxswapd 40, 7 + xxswapd 41, 8 + mtctr 12 +.LBB3_3: + ldu 24, 8(25) + add 24, 24, 26 + addi 24, 24, 256 + dcbt 0, 24 + bdnz .LBB3_3 + vmrgew 3, 4, 2 + vspltisw 31, 9 + mr 25, 8 + vmrglw 10, 4, 2 + vspltisw 14, 10 + vmrghw 6, 4, 2 + xxspltw 0, 0, 3 + vmrgew 4, 17, 0 + vmrglw 11, 17, 0 + vmrghw 16, 17, 0 + vmrgew 0, 25, 26 + vmrgew 13, 7, 1 + vmrglw 2, 7, 1 + vmrghw 7, 7, 1 + xxlor 25, 36, 36 + vmrgew 4, 28, 19 + xxlor 26, 32, 32 + vmrglw 0, 25, 26 + vmrglw 1, 28, 19 + xxmrgld 47, 34, 42 + xxlor 44, 28, 28 + vmrghw 25, 25, 26 + xxlor 23, 36, 36 + vmrghw 4, 28, 19 + vspltisw 19, -16 + xxlor 5, 32, 32 + vmrgew 0, 20, 21 + xxmrgld 34, 33, 43 + vmrglw 28, 20, 21 + vmrghw 21, 20, 21 + vmrglw 20, 23, 24 + vmrghw 26, 23, 24 + vmrglw 17, 9, 8 + xxlor 8, 32, 32 + vmrgew 0, 23, 24 + xxmrgld 56, 39, 38 + vmrgew 23, 9, 8 + xxlor 33, 24, 24 + xxlor 2, 34, 34 + vadduwm 11, 15, 1 + xxmrgld 33, 36, 48 + xxlor 6, 47, 47 + xxlor 27, 32, 32 + vmrghw 0, 9, 8 + vspltisw 9, 12 + vsubuwm 8, 31, 19 + xxmrgld 51, 23, 25 + vadduwm 31, 2, 12 + xxlor 34, 10, 10 + vadduwm 10, 14, 14 + vslw 15, 2, 2 + xxlor 34, 29, 29 + vadduwm 14, 24, 27 + xxlor 24, 48, 48 + vadduwm 16, 1, 2 + xxmrgld 34, 45, 35 + vadduwm 31, 31, 30 + xxmrghd 36, 36, 24 + vadduwm 11, 11, 29 + vadduwm 14, 14, 18 + vadduwm 13, 16, 22 + xxlxor 47, 63, 47 + xxlor 1, 9, 9 + xxlor 1, 11, 11 + xxlxor 48, 43, 9 + vadduwm 11, 11, 2 + xxlor 7, 34, 34 + xxmrghd 34, 39, 38 + xxlxor 39, 46, 11 + xxlor 1, 50, 50 + xxlxor 50, 45, 0 + vperm 15, 15, 15, 5 + vperm 16, 16, 16, 5 + vperm 7, 7, 7, 5 + vperm 18, 18, 18, 5 + xxlor 4, 33, 33 + xxlor 33, 31, 31 + vadduwm 14, 14, 2 + xxlor 3, 34, 34 + xxlor 34, 12, 12 + xxlor 35, 13, 13 + vadduwm 6, 15, 1 + xxlor 33, 30, 30 + vadduwm 2, 16, 2 + vadduwm 3, 7, 3 + vadduwm 12, 18, 1 + xxlxor 59, 34, 61 + xxlxor 61, 35, 1 + xxlxor 33, 38, 62 + xxlxor 62, 44, 54 + vrlw 22, 27, 10 + vrlw 29, 29, 10 + vrlw 1, 1, 10 + vrlw 30, 30, 10 + vadduwm 31, 31, 19 + vadduwm 13, 13, 4 + vadduwm 11, 22, 11 + vadduwm 14, 29, 14 + vadduwm 31, 1, 31 + vadduwm 13, 30, 13 + vadduwm 9, 9, 9 + xxlor 1, 36, 36 + xxlxor 48, 43, 48 + xxlxor 36, 46, 39 + xxmrgld 39, 60, 5 + xxlxor 47, 63, 47 + xxlxor 50, 45, 50 + vrlw 16, 16, 9 + vrlw 28, 4, 9 + xxmrgld 36, 53, 57 + vrlw 15, 15, 9 + xxmrghd 57, 53, 57 + vrlw 18, 18, 9 + vadduwm 14, 14, 4 + xxlor 0, 36, 36 + xxmrgld 36, 49, 52 + vadduwm 2, 16, 2 + xxmrgld 49, 8, 26 + vadduwm 3, 28, 3 + vadduwm 6, 15, 6 + vadduwm 12, 18, 12 + xxlxor 54, 34, 54 + xxlxor 61, 35, 61 + xxlxor 33, 38, 33 + xxlxor 62, 44, 62 + vrlw 29, 29, 8 + vrlw 20, 1, 8 + xxmrgld 33, 55, 27 + vrlw 30, 30, 8 + vrlw 22, 22, 8 + vadduwm 11, 11, 7 + xxlor 5, 39, 39 + xxmrgld 39, 32, 58 + vadduwm 31, 31, 4 + vadduwm 11, 29, 11 + vadduwm 13, 13, 7 + vadduwm 14, 20, 14 + vadduwm 31, 30, 31 + vadduwm 13, 22, 13 + xxlor 28, 36, 36 + xxlxor 50, 43, 50 + xxlxor 48, 46, 48 + xxlxor 36, 63, 60 + xxlxor 47, 45, 47 + vperm 18, 18, 18, 5 + vperm 16, 16, 16, 5 + vperm 4, 4, 4, 5 + vperm 15, 15, 15, 5 + vadduwm 11, 11, 17 + vmr 28, 17 + xxmrghd 49, 32, 58 + vadduwm 6, 18, 6 + vadduwm 12, 16, 12 + vadduwm 21, 4, 2 + vadduwm 3, 15, 3 + xxlxor 34, 38, 61 + xxlxor 61, 44, 52 + xxlxor 62, 53, 62 + xxlxor 54, 35, 54 + vrlw 20, 2, 10 + vrlw 29, 29, 10 + vrlw 0, 30, 10 + vrlw 30, 22, 10 + vadduwm 14, 14, 25 + vadduwm 31, 31, 1 + vadduwm 13, 13, 17 + vadduwm 11, 20, 11 + vadduwm 14, 29, 14 + vadduwm 31, 0, 31 + vadduwm 13, 30, 13 + xxlxor 50, 43, 50 + xxlxor 48, 46, 48 + xxlxor 36, 63, 36 + xxlxor 47, 45, 47 + vrlw 18, 18, 9 + vrlw 16, 16, 9 + vrlw 4, 4, 9 + vrlw 15, 15, 9 + vadduwm 11, 11, 24 + xxlor 8, 56, 56 + vadduwm 6, 18, 6 + vadduwm 12, 16, 12 + vadduwm 24, 4, 21 + vadduwm 3, 15, 3 + xxlxor 55, 38, 52 + xxlxor 61, 44, 61 + xxlxor 62, 35, 62 + xxlxor 32, 56, 32 + vrlw 30, 30, 8 + vrlw 23, 23, 8 + vrlw 29, 29, 8 + vrlw 0, 0, 8 + xxlor 25, 51, 51 + vmr 26, 17 + xxlor 49, 3, 3 + xxlor 52, 1, 1 + xxlor 51, 2, 2 + vadduwm 14, 14, 17 + vadduwm 31, 31, 20 + vadduwm 13, 13, 19 + vadduwm 11, 30, 11 + vadduwm 14, 23, 14 + vadduwm 31, 29, 31 + vadduwm 13, 0, 13 + xxlxor 48, 43, 48 + xxlxor 36, 46, 36 + xxlxor 47, 63, 47 + xxlxor 50, 45, 50 + vperm 16, 16, 16, 5 + vperm 4, 4, 4, 5 + vperm 15, 15, 15, 5 + vperm 18, 18, 18, 5 + xxlor 29, 39, 39 + xxlor 59, 4, 4 + vadduwm 24, 16, 24 + vadduwm 3, 4, 3 + vadduwm 6, 15, 6 + vadduwm 12, 18, 12 + xxlxor 62, 56, 62 + xxlxor 55, 35, 55 + xxlxor 61, 38, 61 + xxlxor 32, 44, 32 + vrlw 30, 30, 10 + vrlw 23, 23, 10 + vrlw 29, 29, 10 + vrlw 0, 0, 10 + xxlor 53, 0, 0 + xxlor 39, 6, 6 + vadduwm 11, 11, 27 + vadduwm 14, 14, 21 + vadduwm 31, 31, 7 + vadduwm 13, 13, 1 + vadduwm 11, 30, 11 + vadduwm 14, 23, 14 + vadduwm 31, 29, 31 + vadduwm 13, 0, 13 + xxlxor 48, 43, 48 + xxlxor 36, 46, 36 + xxlxor 47, 63, 47 + xxlxor 50, 45, 50 + vrlw 16, 16, 9 + vrlw 4, 4, 9 + vrlw 15, 15, 9 + vrlw 18, 18, 9 + xxlor 34, 7, 7 + vadduwm 31, 31, 28 + vadduwm 24, 16, 24 + vadduwm 3, 4, 3 + vadduwm 6, 15, 6 + vadduwm 12, 18, 12 + xxlxor 62, 56, 62 + xxlxor 55, 35, 55 + xxlxor 61, 38, 61 + xxlxor 32, 44, 32 + vrlw 23, 23, 8 + vrlw 29, 29, 8 + vrlw 0, 0, 8 + vrlw 30, 30, 8 + vadduwm 11, 11, 2 + xxlor 34, 28, 28 + vadduwm 13, 13, 26 + vadduwm 14, 14, 2 + vadduwm 11, 23, 11 + vadduwm 14, 29, 14 + vadduwm 31, 0, 31 + vadduwm 13, 30, 13 + xxlxor 50, 43, 50 + xxlxor 48, 46, 48 + xxlxor 36, 63, 36 + xxlxor 47, 45, 47 + vperm 18, 18, 18, 5 + vperm 16, 16, 16, 5 + vperm 4, 4, 4, 5 + vperm 15, 15, 15, 5 + xxlor 2, 58, 58 + xxlor 39, 25, 25 + vadduwm 6, 18, 6 + vadduwm 12, 16, 12 + vadduwm 24, 4, 24 + vadduwm 3, 15, 3 + xxlxor 55, 38, 55 + xxlxor 61, 44, 61 + xxlxor 32, 56, 32 + xxlxor 62, 35, 62 + vrlw 23, 23, 10 + vrlw 29, 29, 10 + vrlw 0, 0, 10 + vrlw 30, 30, 10 + xxlor 54, 29, 29 + xxlor 58, 5, 5 + vadduwm 11, 11, 25 + vadduwm 14, 14, 7 + vadduwm 31, 31, 22 + vadduwm 13, 13, 26 + vadduwm 11, 23, 11 + vadduwm 14, 29, 14 + vadduwm 31, 0, 31 + vadduwm 13, 30, 13 + xxlxor 50, 43, 50 + xxlxor 48, 46, 48 + xxlxor 36, 63, 36 + xxlxor 47, 45, 47 + vrlw 18, 18, 9 + vrlw 16, 16, 9 + vrlw 4, 4, 9 + vrlw 15, 15, 9 + vadduwm 11, 11, 17 + vadduwm 14, 14, 21 + vadduwm 6, 18, 6 + vadduwm 12, 16, 12 + vadduwm 24, 4, 24 + vadduwm 3, 15, 3 + xxlxor 55, 38, 55 + xxlxor 61, 44, 61 + xxlxor 62, 35, 62 + xxlxor 32, 56, 32 + vrlw 30, 30, 8 + vrlw 23, 23, 8 + vrlw 29, 29, 8 + vrlw 0, 0, 8 + vadduwm 31, 31, 1 + vadduwm 13, 13, 20 + vadduwm 11, 30, 11 + vadduwm 14, 23, 14 + vadduwm 31, 29, 31 + vadduwm 13, 0, 13 + xxlxor 48, 43, 48 + xxlxor 36, 46, 36 + xxlxor 47, 63, 47 + xxlxor 50, 45, 50 + vperm 16, 16, 16, 5 + vperm 4, 4, 4, 5 + vperm 15, 15, 15, 5 + vperm 18, 18, 18, 5 + xxlor 0, 33, 33 + xxlor 33, 8, 8 + vadduwm 24, 16, 24 + vadduwm 3, 4, 3 + vadduwm 6, 15, 6 + vadduwm 12, 18, 12 + xxlxor 62, 56, 62 + xxlxor 55, 35, 55 + xxlxor 61, 38, 61 + xxlxor 32, 44, 32 + vrlw 30, 30, 10 + vrlw 23, 23, 10 + vrlw 29, 29, 10 + vrlw 0, 0, 10 + vadduwm 11, 11, 19 + vadduwm 14, 14, 2 + vadduwm 31, 31, 1 + vadduwm 13, 13, 22 + vadduwm 11, 30, 11 + vadduwm 14, 23, 14 + vadduwm 31, 29, 31 + vadduwm 13, 0, 13 + xxlxor 48, 43, 48 + xxlxor 36, 46, 36 + xxlxor 47, 63, 47 + xxlxor 50, 45, 50 + vrlw 16, 16, 9 + vrlw 4, 4, 9 + vrlw 15, 15, 9 + vrlw 18, 18, 9 + vadduwm 11, 11, 27 + vadduwm 14, 14, 28 + vadduwm 24, 16, 24 + vadduwm 3, 4, 3 + vadduwm 6, 15, 6 + vadduwm 12, 18, 12 + xxlxor 62, 56, 62 + xxlxor 55, 35, 55 + xxlxor 61, 38, 61 + xxlxor 32, 44, 32 + vrlw 23, 23, 8 + vrlw 29, 29, 8 + vrlw 0, 0, 8 + vrlw 30, 30, 8 + vadduwm 31, 31, 25 + vadduwm 13, 13, 26 + vadduwm 11, 23, 11 + vadduwm 14, 29, 14 + vadduwm 31, 0, 31 + vadduwm 13, 30, 13 + xxlxor 50, 43, 50 + xxlxor 48, 46, 48 + xxlxor 36, 63, 36 + xxlxor 47, 45, 47 + vperm 18, 18, 18, 5 + vperm 16, 16, 16, 5 + vperm 4, 4, 4, 5 + vperm 15, 15, 15, 5 + xxlor 3, 7, 7 + vadduwm 11, 11, 7 + vadduwm 6, 18, 6 + vadduwm 12, 16, 12 + vadduwm 24, 4, 24 + vadduwm 3, 15, 3 + xxlxor 55, 38, 55 + xxlxor 61, 44, 61 + xxlxor 32, 56, 32 + xxlxor 62, 35, 62 + vrlw 23, 23, 10 + vrlw 29, 29, 10 + vrlw 0, 0, 10 + vrlw 30, 30, 10 + xxlor 33, 6, 6 + xxlor 58, 2, 2 + xxlor 39, 3, 3 + vadduwm 14, 14, 1 + vadduwm 31, 31, 26 + vadduwm 13, 13, 7 + vadduwm 11, 23, 11 + vadduwm 14, 29, 14 + vadduwm 31, 0, 31 + vadduwm 13, 30, 13 + xxlxor 50, 43, 50 + xxlxor 48, 46, 48 + xxlxor 36, 63, 36 + xxlxor 47, 45, 47 + vrlw 18, 18, 9 + vrlw 16, 16, 9 + vrlw 4, 4, 9 + vrlw 15, 15, 9 + xxlor 52, 0, 0 + vadduwm 11, 11, 21 + vadduwm 6, 18, 6 + vadduwm 12, 16, 12 + vadduwm 24, 4, 24 + vadduwm 3, 15, 3 + xxlxor 55, 38, 55 + xxlxor 61, 44, 61 + xxlxor 62, 35, 62 + xxlxor 32, 56, 32 + vrlw 30, 30, 8 + vrlw 23, 23, 8 + vrlw 29, 29, 8 + vrlw 0, 0, 8 + vadduwm 14, 14, 2 + vadduwm 31, 31, 22 + vadduwm 13, 13, 20 + vadduwm 11, 30, 11 + vadduwm 14, 23, 14 + vadduwm 31, 29, 31 + vadduwm 13, 0, 13 + xxlxor 48, 43, 48 + xxlxor 36, 46, 36 + xxlxor 47, 63, 47 + xxlxor 50, 45, 50 + vperm 16, 16, 16, 5 + vperm 4, 4, 4, 5 + vperm 15, 15, 15, 5 + vperm 18, 18, 18, 5 + xxlor 7, 49, 49 + vmr 17, 2 + vadduwm 24, 16, 24 + vadduwm 3, 4, 3 + vadduwm 6, 15, 6 + vadduwm 12, 18, 12 + xxlxor 62, 56, 62 + xxlxor 55, 35, 55 + xxlxor 61, 38, 61 + xxlxor 32, 44, 32 + vrlw 30, 30, 10 + vrlw 23, 23, 10 + vrlw 29, 29, 10 + vrlw 0, 0, 10 + xxlor 54, 1, 1 + xxlor 34, 7, 7 + vadduwm 11, 11, 22 + vadduwm 14, 14, 28 + vadduwm 31, 31, 2 + vadduwm 13, 13, 26 + vadduwm 11, 30, 11 + vadduwm 14, 23, 14 + vadduwm 31, 29, 31 + vadduwm 13, 0, 13 + xxlxor 48, 43, 48 + xxlxor 36, 46, 36 + xxlxor 47, 63, 47 + xxlxor 50, 45, 50 + vrlw 16, 16, 9 + vrlw 4, 4, 9 + vrlw 15, 15, 9 + vrlw 18, 18, 9 + xxlor 59, 25, 25 + vadduwm 11, 11, 19 + vadduwm 24, 16, 24 + vadduwm 3, 4, 3 + vadduwm 6, 15, 6 + vadduwm 12, 18, 12 + xxlxor 62, 56, 62 + xxlxor 55, 35, 55 + xxlxor 61, 38, 61 + xxlxor 32, 44, 32 + vrlw 23, 23, 8 + vrlw 29, 29, 8 + vrlw 0, 0, 8 + vrlw 30, 30, 8 + vadduwm 14, 14, 25 + vadduwm 31, 31, 27 + vadduwm 13, 13, 7 + vadduwm 11, 23, 11 + vadduwm 14, 29, 14 + vadduwm 31, 0, 31 + vadduwm 13, 30, 13 + xxlxor 50, 43, 50 + xxlxor 48, 46, 48 + xxlxor 36, 63, 36 + xxlxor 47, 45, 47 + vperm 18, 18, 18, 5 + vperm 16, 16, 16, 5 + vperm 4, 4, 4, 5 + vperm 15, 15, 15, 5 + vmr 2, 19 + xxlor 0, 7, 7 + vadduwm 6, 18, 6 + vadduwm 12, 16, 12 + vadduwm 24, 4, 24 + vadduwm 3, 15, 3 + xxlxor 55, 38, 55 + xxlxor 61, 44, 61 + xxlxor 32, 56, 32 + xxlxor 62, 35, 62 + vrlw 23, 23, 10 + vrlw 29, 29, 10 + vrlw 0, 0, 10 + vrlw 30, 30, 10 + xxlor 1, 51, 51 + xxlor 7, 39, 39 + xxlor 51, 8, 8 + xxlor 39, 5, 5 + xxlor 34, 4, 4 + vadduwm 11, 11, 1 + vadduwm 14, 14, 19 + vadduwm 31, 31, 7 + vadduwm 13, 13, 2 + vadduwm 11, 23, 11 + vadduwm 14, 29, 14 + vadduwm 31, 0, 31 + vadduwm 13, 30, 13 + xxlxor 50, 43, 50 + xxlxor 48, 46, 48 + xxlxor 36, 63, 36 + xxlxor 47, 45, 47 + vrlw 18, 18, 9 + vrlw 16, 16, 9 + vrlw 4, 4, 9 + vrlw 15, 15, 9 + xxlor 2, 53, 53 + vmr 21, 28 + vadduwm 6, 18, 6 + vadduwm 12, 16, 12 + vadduwm 24, 4, 24 + vadduwm 3, 15, 3 + xxlxor 55, 38, 55 + xxlxor 61, 44, 61 + xxlxor 62, 35, 62 + xxlxor 32, 56, 32 + vrlw 30, 30, 8 + vrlw 23, 23, 8 + vrlw 29, 29, 8 + vrlw 0, 0, 8 + xxlor 53, 29, 29 + vadduwm 11, 11, 17 + vadduwm 14, 14, 28 + vadduwm 31, 31, 26 + vadduwm 13, 13, 21 + vadduwm 11, 30, 11 + vadduwm 14, 23, 14 + vadduwm 31, 29, 31 + vadduwm 13, 0, 13 + xxlxor 48, 43, 48 + xxlxor 36, 46, 36 + xxlxor 47, 63, 47 + xxlxor 50, 45, 50 + vperm 16, 16, 16, 5 + vperm 4, 4, 4, 5 + vperm 15, 15, 15, 5 + vperm 18, 18, 18, 5 + vadduwm 11, 11, 20 + xxlor 5, 52, 52 + vadduwm 24, 16, 24 + vadduwm 3, 4, 3 + vadduwm 6, 15, 6 + vadduwm 12, 18, 12 + xxlxor 62, 56, 62 + xxlxor 55, 35, 55 + xxlxor 61, 38, 61 + xxlxor 32, 44, 32 + vrlw 30, 30, 10 + vrlw 23, 23, 10 + vrlw 29, 29, 10 + vrlw 0, 0, 10 + xxlor 52, 2, 2 + vadduwm 14, 14, 25 + vadduwm 31, 31, 20 + vadduwm 13, 13, 7 + vadduwm 11, 30, 11 + vadduwm 14, 23, 14 + vadduwm 31, 29, 31 + vadduwm 13, 0, 13 + xxlxor 48, 43, 48 + xxlxor 36, 46, 36 + xxlxor 47, 63, 47 + xxlxor 50, 45, 50 + vrlw 16, 16, 9 + vrlw 4, 4, 9 + vrlw 15, 15, 9 + vrlw 18, 18, 9 + vadduwm 11, 11, 22 + vadduwm 14, 14, 27 + vadduwm 24, 16, 24 + vadduwm 3, 4, 3 + vadduwm 6, 15, 6 + vadduwm 12, 18, 12 + xxlxor 62, 56, 62 + xxlxor 55, 35, 55 + xxlxor 61, 38, 61 + xxlxor 32, 44, 32 + vrlw 23, 23, 8 + vrlw 29, 29, 8 + vrlw 0, 0, 8 + vrlw 30, 30, 8 + vadduwm 31, 31, 1 + vadduwm 13, 13, 2 + vadduwm 11, 23, 11 + vadduwm 14, 29, 14 + vadduwm 31, 0, 31 + vadduwm 13, 30, 13 + xxlxor 50, 43, 50 + xxlxor 48, 46, 48 + xxlxor 36, 63, 36 + xxlxor 47, 45, 47 + vperm 18, 18, 18, 5 + vperm 16, 16, 16, 5 + vperm 4, 4, 4, 5 + vperm 15, 15, 15, 5 + xxlor 3, 29, 29 + xxlor 4, 49, 49 + vadduwm 6, 18, 6 + vadduwm 12, 16, 12 + vadduwm 24, 4, 24 + vadduwm 3, 15, 3 + xxlxor 55, 38, 55 + xxlxor 61, 44, 61 + xxlxor 32, 56, 32 + xxlxor 62, 35, 62 + vrlw 23, 23, 10 + vrlw 29, 29, 10 + vrlw 0, 0, 10 + vrlw 30, 30, 10 + vmr 17, 28 + xxlor 2, 54, 54 + xxlor 3, 34, 34 + xxlor 34, 8, 8 + xxlor 51, 0, 0 + xxlor 60, 7, 7 + xxlor 54, 1, 1 + vadduwm 11, 11, 2 + vadduwm 14, 14, 19 + vadduwm 31, 31, 28 + vadduwm 13, 13, 22 + vadduwm 11, 23, 11 + vadduwm 14, 29, 14 + vadduwm 31, 0, 31 + vadduwm 13, 30, 13 + xxlxor 50, 43, 50 + xxlxor 48, 46, 48 + xxlxor 36, 63, 36 + xxlxor 47, 45, 47 + vrlw 18, 18, 9 + vrlw 16, 16, 9 + vrlw 4, 4, 9 + vrlw 15, 15, 9 + vadduwm 11, 11, 17 + vadduwm 14, 14, 25 + vadduwm 6, 18, 6 + vadduwm 12, 16, 12 + vadduwm 24, 4, 24 + vadduwm 3, 15, 3 + xxlxor 55, 38, 55 + xxlxor 61, 44, 61 + xxlxor 62, 35, 62 + xxlxor 32, 56, 32 + vrlw 30, 30, 8 + vrlw 23, 23, 8 + vrlw 29, 29, 8 + vrlw 0, 0, 8 + vadduwm 31, 31, 7 + vadduwm 13, 13, 26 + vadduwm 11, 30, 11 + vadduwm 14, 23, 14 + vadduwm 31, 29, 31 + vadduwm 13, 0, 13 + xxlxor 48, 43, 48 + xxlxor 36, 46, 36 + xxlxor 47, 63, 47 + xxlxor 50, 45, 50 + vperm 16, 16, 16, 5 + vperm 4, 4, 4, 5 + vperm 15, 15, 15, 5 + vperm 18, 18, 18, 5 + xxlor 6, 39, 39 + xxlor 39, 4, 4 + vadduwm 24, 16, 24 + vadduwm 3, 4, 3 + vadduwm 6, 15, 6 + vadduwm 12, 18, 12 + xxlxor 62, 56, 62 + xxlxor 55, 35, 55 + xxlxor 61, 38, 61 + xxlxor 32, 44, 32 + vrlw 30, 30, 10 + vrlw 23, 23, 10 + vrlw 29, 29, 10 + vrlw 0, 0, 10 + vadduwm 11, 11, 21 + vadduwm 14, 14, 27 + vadduwm 31, 31, 7 + vadduwm 13, 13, 28 + vadduwm 11, 30, 11 + vadduwm 14, 23, 14 + vadduwm 31, 29, 31 + vadduwm 13, 0, 13 + xxlxor 48, 43, 48 + xxlxor 36, 46, 36 + xxlxor 47, 63, 47 + xxlxor 50, 45, 50 + vrlw 16, 16, 9 + vrlw 4, 4, 9 + vrlw 15, 15, 9 + vrlw 18, 18, 9 + xxlor 0, 49, 49 + xxlor 49, 5, 5 + vadduwm 24, 16, 24 + vadduwm 3, 4, 3 + vadduwm 6, 15, 6 + vadduwm 12, 18, 12 + xxlxor 62, 56, 62 + xxlxor 55, 35, 55 + xxlxor 61, 38, 61 + xxlxor 32, 44, 32 + vrlw 23, 23, 8 + vrlw 29, 29, 8 + vrlw 0, 0, 8 + vrlw 30, 30, 8 + vadduwm 11, 11, 17 + vadduwm 14, 14, 1 + vadduwm 31, 31, 2 + vadduwm 13, 13, 22 + vadduwm 11, 23, 11 + vadduwm 14, 29, 14 + vadduwm 31, 0, 31 + vadduwm 13, 30, 13 + xxlxor 50, 43, 50 + xxlxor 48, 46, 48 + xxlxor 36, 63, 36 + xxlxor 47, 45, 47 + vperm 18, 18, 18, 5 + vperm 16, 16, 16, 5 + vperm 4, 4, 4, 5 + vperm 15, 15, 15, 5 + xxlor 34, 3, 3 + xxlor 49, 2, 2 + vadduwm 6, 18, 6 + vadduwm 12, 16, 12 + vadduwm 24, 4, 24 + vadduwm 3, 15, 3 + xxlxor 55, 38, 55 + xxlxor 61, 44, 61 + xxlxor 32, 56, 32 + xxlxor 62, 35, 62 + vrlw 23, 23, 10 + vrlw 29, 29, 10 + vrlw 0, 0, 10 + vrlw 30, 30, 10 + vadduwm 11, 11, 19 + vadduwm 14, 14, 20 + vadduwm 31, 31, 2 + vadduwm 13, 13, 17 + vadduwm 11, 23, 11 + vadduwm 14, 29, 14 + vadduwm 31, 0, 31 + vadduwm 13, 30, 13 + xxlxor 50, 43, 50 + xxlxor 48, 46, 48 + xxlxor 36, 63, 36 + xxlxor 47, 45, 47 + vrlw 18, 18, 9 + vrlw 16, 16, 9 + vrlw 4, 4, 9 + vrlw 15, 15, 9 + vadduwm 14, 14, 27 + vadduwm 11, 11, 25 + vadduwm 6, 18, 6 + vadduwm 12, 16, 12 + vadduwm 27, 4, 24 + vadduwm 3, 15, 3 + xxlxor 57, 38, 55 + xxlxor 61, 44, 61 + xxlxor 62, 35, 62 + xxlxor 32, 59, 32 + xxlor 39, 7, 7 + vrlw 30, 30, 8 + vrlw 25, 25, 8 + vrlw 29, 29, 8 + vrlw 0, 0, 8 + xxlor 1, 58, 58 + vmr 26, 19 + vadduwm 19, 31, 7 + xxlor 39, 6, 6 + vadduwm 11, 30, 11 + vadduwm 7, 13, 7 + vadduwm 13, 25, 14 + vadduwm 14, 29, 19 + vadduwm 7, 0, 7 + xxlxor 48, 43, 48 + xxlxor 36, 45, 36 + xxlxor 47, 46, 47 + xxlxor 50, 39, 50 + vperm 16, 16, 16, 5 + vperm 4, 4, 4, 5 + vperm 15, 15, 15, 5 + vperm 18, 18, 18, 5 + xxlor 51, 1, 1 + vadduwm 13, 13, 1 + vadduwm 11, 11, 19 + vadduwm 19, 16, 27 + vadduwm 3, 4, 3 + vadduwm 6, 15, 6 + vadduwm 12, 18, 12 + xxlxor 63, 51, 62 + xxlxor 62, 35, 57 + xxlxor 61, 38, 61 + xxlxor 32, 44, 32 + vrlw 31, 31, 10 + vrlw 30, 30, 10 + vrlw 29, 29, 10 + vrlw 0, 0, 10 + xxlor 33, 0, 0 + vadduwm 7, 7, 2 + vadduwm 14, 14, 1 + vadduwm 11, 31, 11 + vadduwm 13, 30, 13 + vadduwm 14, 29, 14 + vadduwm 7, 0, 7 + xxlxor 48, 43, 48 + xxlxor 36, 45, 36 + xxlxor 47, 46, 47 + xxlxor 50, 39, 50 + vrlw 16, 16, 9 + vrlw 4, 4, 9 + vrlw 15, 15, 9 + vrlw 18, 18, 9 + xxlor 60, 8, 8 + vadduwm 1, 11, 21 + vadduwm 11, 13, 28 + vadduwm 13, 16, 19 + vadduwm 3, 4, 3 + vadduwm 6, 15, 6 + vadduwm 12, 18, 12 + xxlxor 51, 45, 63 + xxlxor 63, 35, 62 + xxlxor 62, 38, 61 + xxlxor 32, 44, 32 + vrlw 31, 31, 8 + vrlw 30, 30, 8 + vrlw 0, 0, 8 + vrlw 19, 19, 8 + vadduwm 14, 14, 26 + vadduwm 7, 7, 17 + vadduwm 1, 31, 1 + vadduwm 11, 30, 11 + vadduwm 14, 0, 14 + vadduwm 7, 19, 7 + xxlxor 50, 33, 50 + xxlxor 48, 43, 48 + xxlxor 36, 46, 36 + xxlxor 47, 39, 47 + vperm 18, 18, 18, 5 + vperm 16, 16, 16, 5 + vperm 4, 4, 4, 5 + vperm 15, 15, 15, 5 + xxlor 34, 4, 4 + vadduwm 14, 14, 22 + vadduwm 6, 18, 6 + vadduwm 12, 16, 12 + vadduwm 13, 4, 13 + vadduwm 3, 15, 3 + xxlxor 49, 38, 63 + xxlxor 63, 44, 62 + xxlxor 32, 45, 32 + xxlxor 51, 35, 51 + vrlw 17, 17, 10 + vrlw 31, 31, 10 + vrlw 0, 0, 10 + vrlw 10, 19, 10 + vadduwm 11, 11, 2 + xxlor 34, 5, 5 + vadduwm 1, 1, 20 + vadduwm 2, 7, 2 + vadduwm 7, 31, 11 + vadduwm 11, 0, 14 + vadduwm 2, 10, 2 + vadduwm 1, 17, 1 + xxlxor 36, 43, 36 + xxlxor 46, 34, 47 + vrlw 4, 4, 9 + vrlw 14, 14, 9 + xxlxor 47, 33, 50 + xxlxor 48, 39, 48 + vrlw 15, 15, 9 + vrlw 9, 16, 9 + vadduwm 13, 4, 13 + vadduwm 3, 14, 3 + xxlxor 32, 45, 32 + xxlxor 45, 45, 33 + xxlxor 33, 35, 42 + xxlxor 59, 35, 39 + vadduwm 3, 15, 6 + vadduwm 6, 9, 12 + xxlxor 39, 35, 49 + xxlxor 42, 38, 63 + vrlw 1, 1, 8 + vrlw 7, 7, 8 + vrlw 10, 10, 8 + vrlw 0, 0, 8 + xxlxor 40, 35, 43 + xxlxor 38, 38, 34 + xxlxor 61, 33, 41 + xxlxor 50, 39, 36 + xxlxor 62, 42, 46 + xxlxor 54, 32, 47 + bne 0, .LBB3_2 +.LBB3_5: + vmrglw 2, 27, 13 + li 3, 32 + li 4, 48 + vmrglw 4, 6, 8 + vmrglw 0, 18, 29 + vmrglw 1, 22, 30 + vmrghw 3, 27, 13 + vmrghw 5, 6, 8 + vmrghw 6, 18, 29 + vmrghw 7, 22, 30 + xxmrgld 40, 36, 34 + xxmrghd 34, 36, 34 + xxmrgld 41, 33, 32 + xxswapd 0, 40 + xxmrgld 36, 37, 35 + xxmrghd 35, 37, 35 + xxmrghd 37, 33, 32 + xxswapd 1, 41 + xxmrgld 32, 39, 38 + xxmrghd 33, 39, 38 + xxswapd 2, 34 + xxswapd 4, 36 + xxswapd 3, 37 + stxvd2x 0, 0, 5 + xxswapd 5, 32 + stxvd2x 1, 5, 11 + xxswapd 0, 35 + xxswapd 1, 33 + stxvd2x 2, 5, 3 + li 3, 64 + stxvd2x 3, 5, 4 + li 4, 80 + stxvd2x 4, 5, 3 + li 3, 96 + stxvd2x 5, 5, 4 + li 4, 112 + stxvd2x 0, 5, 3 + stxvd2x 1, 5, 4 + li 3, 224 + lxvd2x 63, 1, 3 + li 3, 208 + lfd 31, 392(1) + ld 30, 312(1) + ld 29, 304(1) + lxvd2x 62, 1, 3 + li 3, 192 + lfd 30, 384(1) + ld 28, 296(1) + ld 27, 288(1) + lxvd2x 61, 1, 3 + li 3, 176 + lfd 29, 376(1) + ld 26, 280(1) + ld 25, 272(1) + lxvd2x 60, 1, 3 + li 3, 160 + lfd 28, 368(1) + ld 24, 264(1) + ld 23, 256(1) + lxvd2x 59, 1, 3 + li 3, 144 + lfd 27, 360(1) + ld 22, 248(1) + lxvd2x 58, 1, 3 + li 3, 128 + lfd 26, 352(1) + lxvd2x 57, 1, 3 + li 3, 112 + lfd 25, 344(1) + lxvd2x 56, 1, 3 + li 3, 96 + lfd 24, 336(1) + lxvd2x 55, 1, 3 + li 3, 80 + lfd 23, 328(1) + lxvd2x 54, 1, 3 + li 3, 64 + lxvd2x 53, 1, 3 + li 3, 48 + lxvd2x 52, 1, 3 + addi 1, 1, 400 + blr + .long 0 + .quad 0 +.Lfunc_end3: + .size blake3_hash4_sse2, .Lfunc_end3-.Lfunc_begin3 + .cfi_endproc + .section ".note.GNU-stack","",@progbits +#endif diff --git a/module/icp/asm-ppc64/blake3/b3_ppc64le_sse41.S b/module/icp/asm-ppc64/blake3/b3_ppc64le_sse41.S new file mode 100644 index 000000000000..315561d4497a --- /dev/null +++ b/module/icp/asm-ppc64/blake3/b3_ppc64le_sse41.S @@ -0,0 +1,3064 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Based on BLAKE3 v1.3.1, https://github.com/BLAKE3-team/BLAKE3 + * Copyright (c) 2019-2022 Samuel Neves + * Copyright (c) 2022 Tino Reichardt + * + * This is converted assembly: SSE4.1 -> POWER8 PPC64 Little Endian + * Used tools: SIMDe https://github.com/simd-everywhere/simde + */ + +#if (defined(__PPC64__) && defined(__LITTLE_ENDIAN__)) + .text + .abiversion 2 + .section .rodata.cst16,"aM",@progbits,16 + .p2align 4 +.LCPI0_0: + .byte 31 + .byte 14 + .byte 13 + .byte 12 + .byte 30 + .byte 10 + .byte 9 + .byte 8 + .byte 29 + .byte 6 + .byte 5 + .byte 4 + .byte 28 + .byte 2 + .byte 1 + .byte 0 +.LCPI0_1: + .byte 2 + .byte 3 + .byte 0 + .byte 1 + .byte 6 + .byte 7 + .byte 4 + .byte 5 + .byte 10 + .byte 11 + .byte 8 + .byte 9 + .byte 14 + .byte 15 + .byte 12 + .byte 13 +.LCPI0_2: + .byte 29 + .byte 28 + .byte 31 + .byte 30 + .byte 25 + .byte 24 + .byte 27 + .byte 26 + .byte 21 + .byte 20 + .byte 23 + .byte 22 + .byte 17 + .byte 16 + .byte 19 + .byte 18 +.LCPI0_3: + .long 1779033703 + .long 3144134277 + .long 1013904242 + .long 2773480762 +.LCPI0_4: + .byte 27 + .byte 26 + .byte 25 + .byte 24 + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 3 + .byte 2 + .byte 1 + .byte 0 +.LCPI0_5: + .byte 1 + .byte 2 + .byte 3 + .byte 0 + .byte 5 + .byte 6 + .byte 7 + .byte 4 + .byte 9 + .byte 10 + .byte 11 + .byte 8 + .byte 13 + .byte 14 + .byte 15 + .byte 12 +.LCPI0_6: + .byte 30 + .byte 29 + .byte 28 + .byte 31 + .byte 26 + .byte 25 + .byte 24 + .byte 27 + .byte 22 + .byte 21 + .byte 20 + .byte 23 + .byte 18 + .byte 17 + .byte 16 + .byte 19 +.LCPI0_7: + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .byte 27 + .byte 26 + .byte 25 + .byte 24 +.LCPI0_8: + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 27 + .byte 26 + .byte 25 + .byte 24 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 3 + .byte 2 + .byte 1 + .byte 0 +.LCPI0_9: + .byte 31 + .byte 31 + .byte 31 + .byte 31 + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 31 + .byte 31 + .byte 31 + .byte 31 + .byte 3 + .byte 2 + .byte 1 + .byte 0 +.LCPI0_10: + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 31 + .byte 31 + .byte 31 + .byte 31 + .byte 31 + .byte 30 + .byte 29 + .byte 28 + .byte 31 + .byte 31 + .byte 31 + .byte 31 +.LCPI0_11: + .byte 31 + .byte 30 + .byte 29 + .byte 28 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 3 + .byte 2 + .byte 1 + .byte 0 +.LCPI0_12: + .byte 31 + .byte 30 + .byte 29 + .byte 28 + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .byte 27 + .byte 26 + .byte 25 + .byte 24 +.LCPI0_13: + .byte 27 + .byte 26 + .byte 25 + .byte 24 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 15 + .byte 14 + .byte 13 + .byte 12 + .byte 31 + .byte 30 + .byte 29 + .byte 28 +.LCPI0_14: + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .text + .globl zfs_blake3_compress_in_place_sse41 + .p2align 2 + .type zfs_blake3_compress_in_place_sse41,@function +zfs_blake3_compress_in_place_sse41: +.Lfunc_begin0: + .cfi_startproc +.Lfunc_gep0: + addis 2, 12, .TOC.-.Lfunc_gep0@ha + addi 2, 2, .TOC.-.Lfunc_gep0@l +.Lfunc_lep0: + .localentry zfs_blake3_compress_in_place_sse41, .Lfunc_lep0-.Lfunc_gep0 + li 8, -64 + mtvsrd 34, 5 + li 5, 16 + lfdx 0, 0, 4 + vspltisw 13, -16 + stxvd2x 60, 1, 8 + li 8, -48 + mtvsrd 35, 7 + lfd 2, 16(4) + lfd 3, 24(4) + addis 7, 2, .LCPI0_0@toc@ha + stxvd2x 61, 1, 8 + li 8, -32 + mtvsrwz 36, 6 + rldicl 6, 6, 32, 32 + stxvd2x 62, 1, 8 + li 8, -16 + vmrghb 2, 3, 2 + stxvd2x 63, 1, 8 + mtvsrwz 35, 6 + addi 6, 7, .LCPI0_0@toc@l + addis 7, 2, .LCPI0_2@toc@ha + lfd 1, 8(4) + xxmrghd 32, 3, 2 + lvx 6, 0, 6 + xxlxor 33, 33, 33 + addis 6, 2, .LCPI0_1@toc@ha + addi 7, 7, .LCPI0_2@toc@l + vmrghw 3, 3, 4 + addi 6, 6, .LCPI0_1@toc@l + vspltisw 14, 9 + xxmrghd 37, 1, 0 + lxvd2x 0, 0, 3 + lxvd2x 1, 3, 5 + vperm 2, 1, 2, 6 + vpkudum 9, 0, 5 + xxswapd 36, 0 + xxswapd 38, 1 + xxmrgld 34, 34, 35 + lvx 3, 0, 7 + addis 7, 2, .LCPI0_4@toc@ha + addi 7, 7, .LCPI0_4@toc@l + vadduwm 4, 9, 4 + lvx 11, 0, 7 + addis 7, 2, .LCPI0_6@toc@ha + addi 7, 7, .LCPI0_6@toc@l + vadduwm 7, 4, 6 + lvx 4, 0, 6 + addis 6, 2, .LCPI0_3@toc@ha + addi 6, 6, .LCPI0_3@toc@l + vperm 11, 0, 5, 11 + lvx 0, 0, 7 + li 7, 48 + xxlxor 40, 39, 34 + lvx 10, 0, 6 + addis 6, 2, .LCPI0_5@toc@ha + lxvd2x 1, 4, 7 + vcmpgtsb 2, 1, 4 + addi 6, 6, .LCPI0_5@toc@l + vperm 4, 8, 8, 3 + vspltisw 8, 10 + xxlandc 44, 36, 34 + vadduwm 4, 8, 8 + vadduwm 8, 12, 10 + xxlxor 37, 40, 38 + vrlw 6, 5, 4 + vadduwm 5, 7, 11 + vadduwm 7, 6, 5 + lvx 5, 0, 6 + li 6, 32 + lxvd2x 0, 4, 6 + addis 4, 2, .LCPI0_7@toc@ha + addis 6, 2, .LCPI0_9@toc@ha + xxlxor 42, 39, 44 + xxswapd 44, 1 + addi 4, 4, .LCPI0_7@toc@l + addi 6, 6, .LCPI0_9@toc@l + vcmpgtsb 5, 1, 5 + vperm 1, 10, 10, 0 + xxswapd 42, 0 + vpkudum 16, 12, 10 + xxlandc 47, 33, 37 + vsubuwm 1, 14, 13 + lvx 14, 0, 4 + addis 4, 2, .LCPI0_8@toc@ha + vadduwm 8, 15, 8 + xxswapd 45, 47 + addi 4, 4, .LCPI0_8@toc@l + vadduwm 7, 7, 16 + xxsldwi 48, 48, 48, 1 + xxlxor 38, 40, 38 + xxsldwi 40, 40, 40, 3 + xxsldwi 39, 39, 39, 1 + vperm 14, 10, 12, 14 + vrlw 6, 6, 1 + vadduwm 7, 6, 7 + xxlxor 45, 39, 45 + vperm 13, 13, 13, 3 + xxlandc 45, 45, 34 + vadduwm 8, 13, 8 + xxlxor 38, 40, 38 + vrlw 10, 6, 4 + vadduwm 6, 7, 14 + vadduwm 7, 10, 6 + xxlxor 38, 39, 45 + vperm 12, 6, 6, 0 + lvx 6, 0, 4 + addis 4, 2, .LCPI0_10@toc@ha + addi 4, 4, .LCPI0_10@toc@l + vperm 13, 11, 9, 6 + xxlandc 44, 44, 37 + vadduwm 15, 12, 8 + vadduwm 7, 7, 13 + xxsldwi 45, 45, 45, 3 + xxlxor 40, 47, 42 + xxsldwi 47, 47, 47, 1 + xxsldwi 39, 39, 39, 3 + vrlw 10, 8, 1 + xxswapd 40, 44 + vadduwm 17, 10, 7 + lvx 7, 0, 4 + addis 4, 2, .LCPI0_11@toc@ha + addi 4, 4, .LCPI0_11@toc@l + xxlxor 44, 49, 40 + lvx 8, 0, 6 + vperm 18, 9, 9, 7 + lvx 9, 0, 4 + addis 4, 2, .LCPI0_12@toc@ha + vperm 12, 12, 12, 3 + addi 4, 4, .LCPI0_12@toc@l + vperm 19, 14, 16, 8 + xxlandc 63, 44, 34 + vperm 12, 19, 18, 9 + vadduwm 15, 31, 15 + xxlxor 42, 47, 42 + vrlw 18, 10, 4 + vadduwm 10, 17, 12 + vadduwm 17, 18, 10 + xxlxor 42, 49, 63 + xxmrgld 63, 43, 46 + xxsldwi 49, 49, 49, 1 + vmrghw 14, 14, 11 + vperm 19, 10, 10, 0 + lvx 10, 0, 4 + addis 4, 2, .LCPI0_13@toc@ha + addi 4, 4, .LCPI0_13@toc@l + lvx 11, 0, 4 + addis 4, 2, .LCPI0_14@toc@ha + vperm 31, 16, 31, 10 + addi 4, 4, .LCPI0_14@toc@l + vperm 14, 14, 16, 11 + xxlandc 51, 51, 37 + vadduwm 15, 19, 15 + xxswapd 51, 51 + vadduwm 17, 17, 31 + xxlxor 50, 47, 50 + xxsldwi 47, 47, 47, 3 + vperm 30, 14, 31, 8 + vrlw 18, 18, 1 + vadduwm 17, 18, 17 + xxlxor 51, 49, 51 + vadduwm 17, 17, 14 + vperm 19, 19, 19, 3 + xxlandc 51, 51, 34 + vadduwm 15, 19, 15 + xxlxor 48, 47, 50 + vrlw 16, 16, 4 + vadduwm 17, 16, 17 + xxlxor 50, 49, 51 + vperm 19, 12, 13, 6 + vperm 18, 18, 18, 0 + vperm 13, 13, 13, 7 + vadduwm 17, 17, 19 + xxlandc 50, 50, 37 + xxsldwi 49, 49, 49, 3 + vperm 13, 30, 13, 9 + vadduwm 15, 18, 15 + xxswapd 50, 50 + xxmrgld 62, 44, 46 + vmrghw 12, 14, 12 + xxlxor 48, 47, 48 + xxsldwi 47, 47, 47, 1 + vrlw 16, 16, 1 + vperm 30, 31, 30, 10 + vperm 12, 12, 31, 11 + vadduwm 17, 16, 17 + xxlxor 50, 49, 50 + vadduwm 17, 17, 13 + vperm 18, 18, 18, 3 + vperm 31, 12, 30, 8 + xxlandc 50, 50, 34 + vadduwm 15, 18, 15 + xxlxor 48, 47, 48 + vrlw 16, 16, 4 + vadduwm 17, 16, 17 + xxlxor 50, 49, 50 + xxsldwi 49, 49, 49, 1 + vperm 18, 18, 18, 0 + vadduwm 17, 17, 30 + xxlandc 50, 50, 37 + vadduwm 15, 18, 15 + xxswapd 50, 50 + xxlxor 48, 47, 48 + xxsldwi 46, 47, 47, 3 + vrlw 16, 16, 1 + vadduwm 17, 16, 17 + xxlxor 50, 49, 50 + vadduwm 17, 17, 12 + vperm 18, 18, 18, 3 + xxlandc 47, 50, 34 + xxsldwi 50, 51, 51, 3 + vadduwm 14, 15, 14 + vperm 19, 13, 18, 6 + xxlxor 48, 46, 48 + vperm 18, 18, 18, 7 + vrlw 16, 16, 4 + vadduwm 17, 16, 17 + xxlxor 47, 49, 47 + vadduwm 17, 17, 19 + vperm 15, 15, 15, 0 + xxsldwi 49, 49, 49, 3 + xxlandc 47, 47, 37 + vadduwm 14, 15, 14 + xxswapd 47, 47 + xxlxor 48, 46, 48 + xxsldwi 46, 46, 46, 1 + vrlw 16, 16, 1 + vadduwm 17, 16, 17 + xxlxor 47, 49, 47 + vperm 15, 15, 15, 3 + xxlandc 47, 47, 34 + vadduwm 29, 15, 14 + vperm 14, 31, 18, 9 + xxmrgld 50, 45, 44 + xxlxor 48, 61, 48 + vmrghw 12, 12, 13 + vrlw 16, 16, 4 + vperm 18, 30, 18, 10 + vadduwm 17, 17, 14 + vadduwm 17, 16, 17 + xxlxor 47, 49, 47 + xxsldwi 49, 49, 49, 1 + vperm 15, 15, 15, 0 + vadduwm 17, 17, 18 + xxlandc 47, 47, 37 + vadduwm 31, 15, 29 + xxswapd 47, 47 + xxlxor 48, 63, 48 + xxsldwi 45, 63, 63, 3 + vperm 31, 12, 30, 11 + vrlw 16, 16, 1 + vadduwm 17, 16, 17 + xxlxor 47, 49, 47 + vperm 15, 15, 15, 3 + xxlandc 47, 47, 34 + vadduwm 13, 15, 13 + xxlxor 44, 45, 48 + vadduwm 16, 17, 31 + xxsldwi 49, 51, 51, 3 + vrlw 12, 12, 4 + vperm 19, 14, 17, 6 + vadduwm 16, 12, 16 + xxlxor 47, 48, 47 + vperm 15, 15, 15, 0 + xxlandc 47, 47, 37 + vadduwm 13, 15, 13 + xxswapd 47, 47 + xxlxor 44, 45, 44 + xxsldwi 45, 45, 45, 1 + vrlw 30, 12, 1 + vadduwm 12, 16, 19 + xxsldwi 44, 44, 44, 3 + vadduwm 16, 30, 12 + xxlxor 44, 48, 47 + vperm 15, 17, 17, 7 + vperm 12, 12, 12, 3 + vperm 17, 31, 18, 8 + xxlandc 61, 44, 34 + vperm 12, 17, 15, 9 + vadduwm 13, 29, 13 + xxlxor 47, 45, 62 + xxmrgld 62, 46, 63 + vmrghw 14, 31, 14 + vrlw 15, 15, 4 + vadduwm 16, 16, 12 + vperm 30, 18, 30, 10 + vperm 14, 14, 18, 11 + xxsldwi 50, 51, 51, 3 + vadduwm 16, 15, 16 + xxlxor 49, 48, 61 + xxsldwi 48, 48, 48, 1 + vperm 19, 12, 18, 6 + vperm 17, 17, 17, 0 + vadduwm 16, 16, 30 + xxmrgld 60, 44, 46 + vmrghw 12, 14, 12 + vperm 28, 30, 28, 10 + xxlandc 49, 49, 37 + vadduwm 13, 17, 13 + xxswapd 49, 49 + vperm 12, 12, 30, 11 + xxlxor 47, 45, 47 + xxsldwi 45, 45, 45, 3 + vrlw 15, 15, 1 + vperm 8, 12, 28, 8 + vadduwm 16, 15, 16 + xxlxor 49, 48, 49 + vadduwm 16, 16, 14 + vperm 17, 17, 17, 3 + xxlandc 49, 49, 34 + vadduwm 13, 17, 13 + xxlxor 47, 45, 47 + vrlw 15, 15, 4 + vadduwm 16, 15, 16 + xxlxor 49, 48, 49 + vperm 17, 17, 17, 0 + xxlandc 49, 49, 37 + vadduwm 31, 17, 13 + xxlxor 45, 63, 47 + vrlw 15, 13, 1 + vadduwm 13, 16, 19 + xxswapd 48, 49 + xxsldwi 51, 51, 51, 3 + xxsldwi 45, 45, 45, 3 + vadduwm 17, 15, 13 + xxlxor 45, 49, 48 + lvx 16, 0, 4 + vperm 29, 13, 13, 3 + vperm 13, 18, 18, 7 + xxsldwi 50, 63, 63, 1 + vperm 16, 14, 30, 16 + vperm 7, 19, 19, 7 + xxlandc 63, 61, 34 + vadduwm 18, 31, 18 + vperm 29, 16, 13, 9 + xxlxor 47, 50, 47 + vperm 6, 16, 19, 6 + vrlw 15, 15, 4 + vperm 7, 8, 7, 9 + vadduwm 17, 17, 29 + xxmrgld 41, 61, 44 + vadduwm 17, 15, 17 + vperm 9, 28, 9, 10 + xxlxor 63, 49, 63 + xxsldwi 49, 49, 49, 1 + vperm 31, 31, 31, 0 + vadduwm 17, 17, 28 + xxlandc 63, 63, 37 + vadduwm 18, 31, 18 + xxswapd 63, 63 + xxlxor 47, 50, 47 + xxsldwi 46, 50, 50, 3 + vrlw 15, 15, 1 + vadduwm 17, 15, 17 + xxlxor 63, 49, 63 + vadduwm 17, 17, 12 + vperm 31, 31, 31, 3 + xxlandc 50, 63, 34 + vadduwm 14, 18, 14 + xxlxor 47, 46, 47 + vrlw 15, 15, 4 + vadduwm 17, 15, 17 + xxlxor 50, 49, 50 + vadduwm 6, 17, 6 + vperm 18, 18, 18, 0 + xxsldwi 38, 38, 38, 3 + xxlandc 50, 50, 37 + vadduwm 14, 18, 14 + xxswapd 48, 50 + xxlxor 47, 46, 47 + xxsldwi 46, 46, 46, 1 + vrlw 15, 15, 1 + vadduwm 6, 15, 6 + xxlxor 48, 38, 48 + vadduwm 6, 6, 7 + vperm 16, 16, 16, 3 + xxlandc 48, 48, 34 + vadduwm 14, 16, 14 + xxlxor 40, 46, 47 + vrlw 8, 8, 4 + vadduwm 6, 8, 6 + xxlxor 39, 38, 48 + xxsldwi 38, 38, 38, 1 + vperm 7, 7, 7, 0 + vadduwm 6, 6, 9 + xxlandc 39, 39, 37 + vadduwm 14, 7, 14 + xxswapd 39, 39 + xxlxor 40, 46, 40 + xxsldwi 41, 46, 46, 3 + vrlw 8, 8, 1 + vadduwm 6, 8, 6 + xxlxor 39, 38, 39 + vperm 3, 7, 7, 3 + vmrghw 7, 12, 13 + xxlandc 34, 35, 34 + vperm 7, 7, 28, 11 + vadduwm 3, 2, 9 + xxlxor 40, 35, 40 + vrlw 4, 8, 4 + vadduwm 6, 6, 7 + vadduwm 6, 4, 6 + xxlxor 34, 38, 34 + xxsldwi 0, 38, 38, 3 + vperm 2, 2, 2, 0 + xxlandc 34, 34, 37 + vadduwm 3, 2, 3 + xxswapd 34, 34 + xxlxor 36, 35, 36 + xxsldwi 1, 35, 35, 1 + vrlw 4, 4, 1 + xxlxor 0, 1, 0 + xxswapd 0, 0 + xxlxor 1, 36, 34 + stxvd2x 0, 0, 3 + xxswapd 1, 1 + stxvd2x 1, 3, 5 + li 3, -16 + lxvd2x 63, 1, 3 + li 3, -32 + lxvd2x 62, 1, 3 + li 3, -48 + lxvd2x 61, 1, 3 + li 3, -64 + lxvd2x 60, 1, 3 + blr + .long 0 + .quad 0 +.Lfunc_end0: + .size zfs_blake3_compress_in_place_sse41, .Lfunc_end0-.Lfunc_begin0 + .cfi_endproc + + .section .rodata.cst16,"aM",@progbits,16 + .p2align 4 +.LCPI1_0: + .byte 31 + .byte 14 + .byte 13 + .byte 12 + .byte 30 + .byte 10 + .byte 9 + .byte 8 + .byte 29 + .byte 6 + .byte 5 + .byte 4 + .byte 28 + .byte 2 + .byte 1 + .byte 0 +.LCPI1_1: + .byte 2 + .byte 3 + .byte 0 + .byte 1 + .byte 6 + .byte 7 + .byte 4 + .byte 5 + .byte 10 + .byte 11 + .byte 8 + .byte 9 + .byte 14 + .byte 15 + .byte 12 + .byte 13 +.LCPI1_2: + .byte 29 + .byte 28 + .byte 31 + .byte 30 + .byte 25 + .byte 24 + .byte 27 + .byte 26 + .byte 21 + .byte 20 + .byte 23 + .byte 22 + .byte 17 + .byte 16 + .byte 19 + .byte 18 +.LCPI1_3: + .long 1779033703 + .long 3144134277 + .long 1013904242 + .long 2773480762 +.LCPI1_4: + .byte 27 + .byte 26 + .byte 25 + .byte 24 + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 3 + .byte 2 + .byte 1 + .byte 0 +.LCPI1_5: + .byte 1 + .byte 2 + .byte 3 + .byte 0 + .byte 5 + .byte 6 + .byte 7 + .byte 4 + .byte 9 + .byte 10 + .byte 11 + .byte 8 + .byte 13 + .byte 14 + .byte 15 + .byte 12 +.LCPI1_6: + .byte 30 + .byte 29 + .byte 28 + .byte 31 + .byte 26 + .byte 25 + .byte 24 + .byte 27 + .byte 22 + .byte 21 + .byte 20 + .byte 23 + .byte 18 + .byte 17 + .byte 16 + .byte 19 +.LCPI1_7: + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .byte 27 + .byte 26 + .byte 25 + .byte 24 +.LCPI1_8: + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 27 + .byte 26 + .byte 25 + .byte 24 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 3 + .byte 2 + .byte 1 + .byte 0 +.LCPI1_9: + .byte 31 + .byte 31 + .byte 31 + .byte 31 + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 31 + .byte 31 + .byte 31 + .byte 31 + .byte 3 + .byte 2 + .byte 1 + .byte 0 +.LCPI1_10: + .byte 19 + .byte 18 + .byte 17 + .byte 16 + .byte 31 + .byte 31 + .byte 31 + .byte 31 + .byte 31 + .byte 30 + .byte 29 + .byte 28 + .byte 31 + .byte 31 + .byte 31 + .byte 31 +.LCPI1_11: + .byte 31 + .byte 30 + .byte 29 + .byte 28 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 3 + .byte 2 + .byte 1 + .byte 0 +.LCPI1_12: + .byte 31 + .byte 30 + .byte 29 + .byte 28 + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .byte 27 + .byte 26 + .byte 25 + .byte 24 +.LCPI1_13: + .byte 27 + .byte 26 + .byte 25 + .byte 24 + .byte 11 + .byte 10 + .byte 9 + .byte 8 + .byte 15 + .byte 14 + .byte 13 + .byte 12 + .byte 31 + .byte 30 + .byte 29 + .byte 28 +.LCPI1_14: + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 23 + .byte 22 + .byte 21 + .byte 20 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .byte 3 + .byte 2 + .byte 1 + .byte 0 + .text + .globl zfs_blake3_compress_xof_sse41 + .p2align 2 + .type zfs_blake3_compress_xof_sse41,@function +zfs_blake3_compress_xof_sse41: +.Lfunc_begin1: + .cfi_startproc +.Lfunc_gep1: + addis 2, 12, .TOC.-.Lfunc_gep1@ha + addi 2, 2, .TOC.-.Lfunc_gep1@l +.Lfunc_lep1: + .localentry zfs_blake3_compress_xof_sse41, .Lfunc_lep1-.Lfunc_gep1 + li 9, -64 + mtvsrd 34, 5 + li 5, 16 + lfdx 0, 0, 4 + vspltisw 13, -16 + addis 11, 2, .LCPI1_9@toc@ha + stxvd2x 60, 1, 9 + li 9, -48 + mtvsrd 35, 7 + lfd 1, 8(4) + lfd 2, 16(4) + addis 7, 2, .LCPI1_0@toc@ha + stxvd2x 61, 1, 9 + li 9, -32 + mtvsrwz 36, 6 + rldicl 6, 6, 32, 32 + stxvd2x 62, 1, 9 + li 9, -16 + vmrghb 2, 3, 2 + stxvd2x 63, 1, 9 + mtvsrwz 35, 6 + addi 6, 7, .LCPI1_0@toc@l + addis 7, 2, .LCPI1_2@toc@ha + lfd 3, 24(4) + xxmrghd 37, 1, 0 + lvx 6, 0, 6 + xxlxor 33, 33, 33 + lxvd2x 0, 0, 3 + addis 6, 2, .LCPI1_1@toc@ha + addi 7, 7, .LCPI1_2@toc@l + vmrghw 3, 3, 4 + lxvd2x 1, 3, 5 + addi 6, 6, .LCPI1_1@toc@l + vspltisw 14, 9 + xxmrghd 32, 3, 2 + xxswapd 36, 0 + vperm 2, 1, 2, 6 + xxswapd 38, 1 + vpkudum 9, 0, 5 + xxmrgld 34, 34, 35 + lvx 3, 0, 7 + addis 7, 2, .LCPI1_4@toc@ha + addi 7, 7, .LCPI1_4@toc@l + vadduwm 4, 9, 4 + lvx 11, 0, 7 + addis 7, 2, .LCPI1_6@toc@ha + addi 7, 7, .LCPI1_6@toc@l + vadduwm 7, 4, 6 + lvx 4, 0, 6 + addis 6, 2, .LCPI1_3@toc@ha + addi 6, 6, .LCPI1_3@toc@l + vperm 11, 0, 5, 11 + lvx 0, 0, 7 + li 7, 32 + xxlxor 40, 39, 34 + lvx 10, 0, 6 + addis 6, 2, .LCPI1_5@toc@ha + lxvd2x 0, 4, 7 + vcmpgtsb 2, 1, 4 + addi 6, 6, .LCPI1_5@toc@l + vperm 4, 8, 8, 3 + vspltisw 8, 10 + xxlandc 44, 36, 34 + vadduwm 4, 8, 8 + vadduwm 8, 12, 10 + xxlxor 37, 40, 38 + vrlw 6, 5, 4 + vadduwm 5, 7, 11 + vadduwm 7, 6, 5 + lvx 5, 0, 6 + li 6, 48 + lxvd2x 1, 4, 6 + addis 4, 2, .LCPI1_7@toc@ha + xxlxor 42, 39, 44 + addi 4, 4, .LCPI1_7@toc@l + vcmpgtsb 5, 1, 5 + vperm 1, 10, 10, 0 + xxswapd 42, 0 + xxswapd 44, 1 + vpkudum 16, 12, 10 + xxlandc 47, 33, 37 + vsubuwm 1, 14, 13 + lvx 14, 0, 4 + addis 4, 2, .LCPI1_8@toc@ha + vadduwm 8, 15, 8 + xxswapd 45, 47 + addi 4, 4, .LCPI1_8@toc@l + xxlxor 38, 40, 38 + xxsldwi 40, 40, 40, 3 + vadduwm 7, 7, 16 + xxsldwi 48, 48, 48, 1 + vrlw 6, 6, 1 + xxsldwi 39, 39, 39, 1 + vperm 14, 10, 12, 14 + vadduwm 7, 6, 7 + xxlxor 45, 39, 45 + vperm 13, 13, 13, 3 + xxlandc 45, 45, 34 + vadduwm 8, 13, 8 + xxlxor 38, 40, 38 + vrlw 10, 6, 4 + vadduwm 6, 7, 14 + vadduwm 7, 10, 6 + xxlxor 38, 39, 45 + vperm 12, 6, 6, 0 + lvx 6, 0, 4 + addis 4, 2, .LCPI1_10@toc@ha + addi 4, 4, .LCPI1_10@toc@l + vperm 13, 11, 9, 6 + xxlandc 44, 44, 37 + vadduwm 15, 12, 8 + vadduwm 7, 7, 13 + xxsldwi 45, 45, 45, 3 + xxlxor 40, 47, 42 + xxsldwi 47, 47, 47, 1 + xxsldwi 39, 39, 39, 3 + vrlw 10, 8, 1 + xxswapd 40, 44 + vadduwm 17, 10, 7 + lvx 7, 0, 4 + addi 4, 11, .LCPI1_9@toc@l + xxlxor 44, 49, 40 + lvx 8, 0, 4 + addis 4, 2, .LCPI1_11@toc@ha + vperm 18, 9, 9, 7 + addi 4, 4, .LCPI1_11@toc@l + vperm 12, 12, 12, 3 + lvx 9, 0, 4 + addis 4, 2, .LCPI1_12@toc@ha + vperm 19, 14, 16, 8 + addi 4, 4, .LCPI1_12@toc@l + xxlandc 63, 44, 34 + vperm 12, 19, 18, 9 + vadduwm 15, 31, 15 + xxlxor 42, 47, 42 + vrlw 18, 10, 4 + vadduwm 10, 17, 12 + vadduwm 17, 18, 10 + xxlxor 42, 49, 63 + xxmrgld 63, 43, 46 + xxsldwi 49, 49, 49, 1 + vmrghw 14, 14, 11 + vperm 19, 10, 10, 0 + lvx 10, 0, 4 + addis 4, 2, .LCPI1_13@toc@ha + addi 4, 4, .LCPI1_13@toc@l + lvx 11, 0, 4 + addis 4, 2, .LCPI1_14@toc@ha + vperm 31, 16, 31, 10 + addi 4, 4, .LCPI1_14@toc@l + vperm 14, 14, 16, 11 + xxlandc 51, 51, 37 + vadduwm 15, 19, 15 + xxswapd 51, 51 + vadduwm 17, 17, 31 + xxlxor 50, 47, 50 + xxsldwi 47, 47, 47, 3 + vperm 30, 14, 31, 8 + vrlw 18, 18, 1 + vadduwm 17, 18, 17 + xxlxor 51, 49, 51 + vadduwm 17, 17, 14 + vperm 19, 19, 19, 3 + xxlandc 51, 51, 34 + vadduwm 15, 19, 15 + xxlxor 48, 47, 50 + vrlw 16, 16, 4 + vadduwm 17, 16, 17 + xxlxor 50, 49, 51 + vperm 19, 12, 13, 6 + vperm 18, 18, 18, 0 + vperm 13, 13, 13, 7 + vadduwm 17, 17, 19 + xxlandc 50, 50, 37 + xxsldwi 49, 49, 49, 3 + vperm 13, 30, 13, 9 + vadduwm 15, 18, 15 + xxswapd 50, 50 + xxmrgld 62, 44, 46 + vmrghw 12, 14, 12 + xxlxor 48, 47, 48 + xxsldwi 47, 47, 47, 1 + vrlw 16, 16, 1 + vperm 30, 31, 30, 10 + vperm 12, 12, 31, 11 + vadduwm 17, 16, 17 + xxlxor 50, 49, 50 + vadduwm 17, 17, 13 + vperm 18, 18, 18, 3 + vperm 31, 12, 30, 8 + xxlandc 50, 50, 34 + vadduwm 15, 18, 15 + xxlxor 48, 47, 48 + vrlw 16, 16, 4 + vadduwm 17, 16, 17 + xxlxor 50, 49, 50 + xxsldwi 49, 49, 49, 1 + vperm 18, 18, 18, 0 + vadduwm 17, 17, 30 + xxlandc 50, 50, 37 + vadduwm 15, 18, 15 + xxswapd 50, 50 + xxlxor 48, 47, 48 + xxsldwi 46, 47, 47, 3 + vrlw 16, 16, 1 + vadduwm 17, 16, 17 + xxlxor 50, 49, 50 + vadduwm 17, 17, 12 + vperm 18, 18, 18, 3 + xxlandc 47, 50, 34 + xxsldwi 50, 51, 51, 3 + vadduwm 14, 15, 14 + vperm 19, 13, 18, 6 + xxlxor 48, 46, 48 + vperm 18, 18, 18, 7 + vrlw 16, 16, 4 + vadduwm 17, 16, 17 + xxlxor 47, 49, 47 + vadduwm 17, 17, 19 + vperm 15, 15, 15, 0 + xxsldwi 49, 49, 49, 3 + xxlandc 47, 47, 37 + vadduwm 14, 15, 14 + xxswapd 47, 47 + xxlxor 48, 46, 48 + xxsldwi 46, 46, 46, 1 + vrlw 16, 16, 1 + vadduwm 17, 16, 17 + xxlxor 47, 49, 47 + vperm 15, 15, 15, 3 + xxlandc 47, 47, 34 + vadduwm 29, 15, 14 + vperm 14, 31, 18, 9 + xxmrgld 50, 45, 44 + xxlxor 48, 61, 48 + vmrghw 12, 12, 13 + vrlw 16, 16, 4 + vperm 18, 30, 18, 10 + vadduwm 17, 17, 14 + vadduwm 17, 16, 17 + xxlxor 47, 49, 47 + xxsldwi 49, 49, 49, 1 + vperm 15, 15, 15, 0 + vadduwm 17, 17, 18 + xxlandc 47, 47, 37 + vadduwm 31, 15, 29 + xxswapd 47, 47 + xxlxor 48, 63, 48 + xxsldwi 45, 63, 63, 3 + vperm 31, 12, 30, 11 + vrlw 16, 16, 1 + vadduwm 17, 16, 17 + xxlxor 47, 49, 47 + vperm 15, 15, 15, 3 + xxlandc 47, 47, 34 + vadduwm 13, 15, 13 + xxlxor 44, 45, 48 + vadduwm 16, 17, 31 + xxsldwi 49, 51, 51, 3 + vrlw 12, 12, 4 + vperm 19, 14, 17, 6 + vadduwm 16, 12, 16 + xxlxor 47, 48, 47 + vperm 15, 15, 15, 0 + xxlandc 47, 47, 37 + vadduwm 13, 15, 13 + xxswapd 47, 47 + xxlxor 44, 45, 44 + xxsldwi 45, 45, 45, 1 + vrlw 30, 12, 1 + vadduwm 12, 16, 19 + xxsldwi 44, 44, 44, 3 + vadduwm 16, 30, 12 + xxlxor 44, 48, 47 + vperm 15, 17, 17, 7 + vperm 12, 12, 12, 3 + vperm 17, 31, 18, 8 + xxlandc 61, 44, 34 + vperm 12, 17, 15, 9 + vadduwm 13, 29, 13 + xxlxor 47, 45, 62 + xxmrgld 62, 46, 63 + vmrghw 14, 31, 14 + vrlw 15, 15, 4 + vadduwm 16, 16, 12 + vperm 30, 18, 30, 10 + vperm 14, 14, 18, 11 + xxsldwi 50, 51, 51, 3 + vadduwm 16, 15, 16 + xxlxor 49, 48, 61 + xxsldwi 48, 48, 48, 1 + vperm 19, 12, 18, 6 + vperm 17, 17, 17, 0 + vadduwm 16, 16, 30 + xxmrgld 60, 44, 46 + vmrghw 12, 14, 12 + vperm 28, 30, 28, 10 + xxlandc 49, 49, 37 + vadduwm 13, 17, 13 + xxswapd 49, 49 + vperm 12, 12, 30, 11 + xxlxor 47, 45, 47 + xxsldwi 45, 45, 45, 3 + vrlw 15, 15, 1 + vperm 8, 12, 28, 8 + vadduwm 16, 15, 16 + xxlxor 49, 48, 49 + vadduwm 16, 16, 14 + vperm 17, 17, 17, 3 + xxlandc 49, 49, 34 + vadduwm 13, 17, 13 + xxlxor 47, 45, 47 + vrlw 15, 15, 4 + vadduwm 16, 15, 16 + xxlxor 49, 48, 49 + vperm 17, 17, 17, 0 + xxlandc 49, 49, 37 + vadduwm 31, 17, 13 + xxlxor 45, 63, 47 + vrlw 15, 13, 1 + vadduwm 13, 16, 19 + xxswapd 48, 49 + xxsldwi 51, 51, 51, 3 + xxsldwi 45, 45, 45, 3 + vadduwm 17, 15, 13 + xxlxor 45, 49, 48 + lvx 16, 0, 4 + vperm 29, 13, 13, 3 + vperm 13, 18, 18, 7 + xxsldwi 50, 63, 63, 1 + vperm 16, 14, 30, 16 + vperm 7, 19, 19, 7 + xxlandc 63, 61, 34 + vadduwm 18, 31, 18 + vperm 29, 16, 13, 9 + xxlxor 47, 50, 47 + vperm 6, 16, 19, 6 + vrlw 15, 15, 4 + vperm 7, 8, 7, 9 + vadduwm 17, 17, 29 + xxmrgld 41, 61, 44 + vadduwm 17, 15, 17 + vperm 9, 28, 9, 10 + xxlxor 63, 49, 63 + xxsldwi 49, 49, 49, 1 + vperm 31, 31, 31, 0 + vadduwm 17, 17, 28 + xxlandc 63, 63, 37 + vadduwm 18, 31, 18 + xxswapd 63, 63 + xxlxor 47, 50, 47 + xxsldwi 46, 50, 50, 3 + vrlw 15, 15, 1 + vadduwm 17, 15, 17 + xxlxor 63, 49, 63 + vadduwm 17, 17, 12 + vperm 31, 31, 31, 3 + xxlandc 50, 63, 34 + vadduwm 14, 18, 14 + xxlxor 47, 46, 47 + vrlw 15, 15, 4 + vadduwm 17, 15, 17 + xxlxor 50, 49, 50 + vadduwm 6, 17, 6 + vperm 18, 18, 18, 0 + xxsldwi 38, 38, 38, 3 + xxlandc 50, 50, 37 + vadduwm 14, 18, 14 + xxswapd 48, 50 + xxlxor 47, 46, 47 + xxsldwi 46, 46, 46, 1 + vrlw 15, 15, 1 + vadduwm 6, 15, 6 + xxlxor 48, 38, 48 + vadduwm 6, 6, 7 + vperm 16, 16, 16, 3 + xxlandc 48, 48, 34 + vadduwm 14, 16, 14 + xxlxor 40, 46, 47 + vrlw 8, 8, 4 + vadduwm 6, 8, 6 + xxlxor 39, 38, 48 + xxsldwi 38, 38, 38, 1 + vperm 7, 7, 7, 0 + vadduwm 6, 6, 9 + xxlandc 39, 39, 37 + vadduwm 14, 7, 14 + xxswapd 39, 39 + xxlxor 40, 46, 40 + xxsldwi 41, 46, 46, 3 + vrlw 8, 8, 1 + vadduwm 6, 8, 6 + xxlxor 39, 38, 39 + vperm 3, 7, 7, 3 + vmrghw 7, 12, 13 + xxlandc 34, 35, 34 + vperm 7, 7, 28, 11 + vadduwm 3, 2, 9 + xxlxor 40, 35, 40 + vrlw 4, 8, 4 + vadduwm 6, 6, 7 + vadduwm 6, 4, 6 + xxlxor 34, 38, 34 + xxsldwi 0, 38, 38, 3 + vperm 2, 2, 2, 0 + xxlandc 34, 34, 37 + vadduwm 3, 2, 3 + xxswapd 34, 34 + xxlxor 36, 35, 36 + xxsldwi 1, 35, 35, 1 + vrlw 4, 4, 1 + xxlxor 0, 1, 0 + xxswapd 0, 0 + xxlxor 2, 36, 34 + stxvd2x 0, 0, 8 + xxswapd 2, 2 + stxvd2x 2, 8, 5 + lfdx 0, 0, 3 + lfd 2, 8(3) + xxmrghd 35, 2, 0 + xxlxor 0, 1, 35 + xxswapd 0, 0 + stxvd2x 0, 8, 7 + lfd 0, 16(3) + lfd 1, 24(3) + li 3, -16 + xxmrghd 35, 1, 0 + xxlxor 0, 34, 35 + xxswapd 0, 0 + stxvd2x 0, 8, 6 + lxvd2x 63, 1, 3 + li 3, -32 + lxvd2x 62, 1, 3 + li 3, -48 + lxvd2x 61, 1, 3 + li 3, -64 + lxvd2x 60, 1, 3 + blr + .long 0 + .quad 0 +.Lfunc_end1: + .size zfs_blake3_compress_xof_sse41, .Lfunc_end1-.Lfunc_begin1 + .cfi_endproc + + .globl zfs_blake3_hash_many_sse41 + .p2align 2 + .type zfs_blake3_hash_many_sse41,@function +zfs_blake3_hash_many_sse41: +.Lfunc_begin2: + .cfi_startproc +.Lfunc_gep2: + addis 2, 12, .TOC.-.Lfunc_gep2@ha + addi 2, 2, .TOC.-.Lfunc_gep2@l +.Lfunc_lep2: + .localentry zfs_blake3_hash_many_sse41, .Lfunc_lep2-.Lfunc_gep2 + mfocrf 12, 32 + mflr 0 + std 0, 16(1) + stw 12, 8(1) + stdu 1, -256(1) + .cfi_def_cfa_offset 256 + .cfi_offset lr, 16 + .cfi_offset r17, -120 + .cfi_offset r18, -112 + .cfi_offset r19, -104 + .cfi_offset r20, -96 + .cfi_offset r21, -88 + .cfi_offset r22, -80 + .cfi_offset r23, -72 + .cfi_offset r24, -64 + .cfi_offset r25, -56 + .cfi_offset r26, -48 + .cfi_offset r27, -40 + .cfi_offset r28, -32 + .cfi_offset r29, -24 + .cfi_offset r30, -16 + .cfi_offset cr2, 8 + std 26, 208(1) + mr 26, 4 + cmpldi 1, 4, 4 + andi. 4, 8, 1 + std 18, 144(1) + std 19, 152(1) + crmove 8, 1 + ld 19, 360(1) + lwz 18, 352(1) + std 24, 192(1) + std 25, 200(1) + std 27, 216(1) + std 28, 224(1) + mr 24, 10 + mr 28, 6 + mr 27, 5 + mr 25, 3 + std 29, 232(1) + std 30, 240(1) + mr 30, 9 + mr 29, 7 + std 17, 136(1) + std 20, 160(1) + std 21, 168(1) + std 22, 176(1) + std 23, 184(1) + blt 1, .LBB2_3 + li 3, 0 + li 4, 1 + clrldi 23, 30, 32 + isel 22, 4, 3, 8 + clrldi 21, 24, 32 + clrldi 20, 18, 32 +.LBB2_2: + mr 3, 25 + mr 4, 27 + mr 5, 28 + mr 6, 29 + mr 7, 22 + mr 8, 23 + mr 9, 21 + mr 10, 20 + std 19, 32(1) + bl blake3_hash4_sse41 + addi 26, 26, -4 + addi 3, 29, 4 + addi 25, 25, 32 + addi 19, 19, 128 + cmpldi 26, 3 + isel 29, 3, 29, 8 + bgt 0, .LBB2_2 +.LBB2_3: + cmpldi 26, 0 + beq 0, .LBB2_11 + li 3, 0 + li 4, 1 + or 21, 24, 30 + li 20, 16 + addi 24, 1, 96 + isel 22, 4, 3, 8 +.LBB2_5: + lxvd2x 0, 28, 20 + ld 23, 0(25) + mr 17, 27 + mr 3, 21 + stxvd2x 0, 24, 20 + lxvd2x 0, 0, 28 + stxvd2x 0, 0, 24 +.LBB2_6: + cmpldi 17, 1 + beq 0, .LBB2_8 + cmpldi 17, 0 + bne 0, .LBB2_9 + b .LBB2_10 +.LBB2_8: + or 3, 3, 18 +.LBB2_9: + clrldi 7, 3, 56 + mr 3, 24 + mr 4, 23 + li 5, 64 + mr 6, 29 + bl zfs_blake3_compress_in_place_sse41 + addi 23, 23, 64 + addi 17, 17, -1 + mr 3, 30 + b .LBB2_6 +.LBB2_10: + lxvd2x 0, 24, 20 + addi 26, 26, -1 + add 29, 29, 22 + addi 25, 25, 8 + cmpldi 26, 0 + stxvd2x 0, 19, 20 + lxvd2x 0, 0, 24 + stxvd2x 0, 0, 19 + addi 19, 19, 32 + bne 0, .LBB2_5 +.LBB2_11: + ld 30, 240(1) + ld 29, 232(1) + ld 28, 224(1) + ld 27, 216(1) + ld 26, 208(1) + ld 25, 200(1) + ld 24, 192(1) + ld 23, 184(1) + ld 22, 176(1) + ld 21, 168(1) + ld 20, 160(1) + ld 19, 152(1) + ld 18, 144(1) + ld 17, 136(1) + addi 1, 1, 256 + ld 0, 16(1) + lwz 12, 8(1) + mtocrf 32, 12 + mtlr 0 + blr + .long 0 + .quad 0 +.Lfunc_end2: + .size zfs_blake3_hash_many_sse41, .Lfunc_end2-.Lfunc_begin2 + .cfi_endproc + + .section .rodata.cst16,"aM",@progbits,16 + .p2align 4 +.LCPI3_0: + .quad 4294967296 + .quad 12884901890 +.LCPI3_1: + .byte 2 + .byte 3 + .byte 0 + .byte 1 + .byte 6 + .byte 7 + .byte 4 + .byte 5 + .byte 10 + .byte 11 + .byte 8 + .byte 9 + .byte 14 + .byte 15 + .byte 12 + .byte 13 +.LCPI3_2: + .byte 1 + .byte 2 + .byte 3 + .byte 0 + .byte 5 + .byte 6 + .byte 7 + .byte 4 + .byte 9 + .byte 10 + .byte 11 + .byte 8 + .byte 13 + .byte 14 + .byte 15 + .byte 12 +.LCPI3_3: + .byte 29 + .byte 28 + .byte 31 + .byte 30 + .byte 25 + .byte 24 + .byte 27 + .byte 26 + .byte 21 + .byte 20 + .byte 23 + .byte 22 + .byte 17 + .byte 16 + .byte 19 + .byte 18 +.LCPI3_4: + .long 1779033703 + .long 1779033703 + .long 1779033703 + .long 1779033703 +.LCPI3_5: + .long 3144134277 + .long 3144134277 + .long 3144134277 + .long 3144134277 +.LCPI3_6: + .long 1013904242 + .long 1013904242 + .long 1013904242 + .long 1013904242 +.LCPI3_7: + .long 2773480762 + .long 2773480762 + .long 2773480762 + .long 2773480762 +.LCPI3_8: + .byte 30 + .byte 29 + .byte 28 + .byte 31 + .byte 26 + .byte 25 + .byte 24 + .byte 27 + .byte 22 + .byte 21 + .byte 20 + .byte 23 + .byte 18 + .byte 17 + .byte 16 + .byte 19 + .text + .p2align 2 + .type blake3_hash4_sse41,@function +blake3_hash4_sse41: +.Lfunc_begin3: + .cfi_startproc +.Lfunc_gep3: + addis 2, 12, .TOC.-.Lfunc_gep3@ha + addi 2, 2, .TOC.-.Lfunc_gep3@l +.Lfunc_lep3: + .localentry blake3_hash4_sse41, .Lfunc_lep3-.Lfunc_gep3 + stdu 1, -416(1) + .cfi_def_cfa_offset 416 + .cfi_offset r22, -176 + .cfi_offset r23, -168 + .cfi_offset r24, -160 + .cfi_offset r25, -152 + .cfi_offset r26, -144 + .cfi_offset r27, -136 + .cfi_offset r28, -128 + .cfi_offset r29, -120 + .cfi_offset r30, -112 + .cfi_offset f20, -96 + .cfi_offset f21, -88 + .cfi_offset f22, -80 + .cfi_offset f23, -72 + .cfi_offset f24, -64 + .cfi_offset f25, -56 + .cfi_offset f26, -48 + .cfi_offset f27, -40 + .cfi_offset f28, -32 + .cfi_offset f29, -24 + .cfi_offset f30, -16 + .cfi_offset f31, -8 + .cfi_offset v20, -368 + .cfi_offset v21, -352 + .cfi_offset v22, -336 + .cfi_offset v23, -320 + .cfi_offset v24, -304 + .cfi_offset v25, -288 + .cfi_offset v26, -272 + .cfi_offset v27, -256 + .cfi_offset v28, -240 + .cfi_offset v29, -224 + .cfi_offset v30, -208 + .cfi_offset v31, -192 + li 11, 48 + li 0, 8 + std 30, 304(1) + li 30, 12 + li 12, 4 + lfiwzx 0, 0, 5 + stxvd2x 52, 1, 11 + li 11, 64 + lfiwzx 2, 5, 0 + li 0, 20 + lfiwzx 3, 5, 30 + stxvd2x 53, 1, 11 + li 11, 80 + li 30, 24 + lfiwzx 4, 5, 0 + li 0, 28 + stxvd2x 54, 1, 11 + li 11, 96 + lfiwzx 1, 5, 12 + lfiwzx 6, 5, 30 + xxspltw 47, 0, 1 + cmpldi 4, 0 + std 22, 240(1) + stxvd2x 55, 1, 11 + li 11, 112 + lfiwzx 7, 5, 0 + xxspltw 40, 2, 1 + std 23, 248(1) + xxspltw 39, 3, 1 + std 24, 256(1) + std 25, 264(1) + xxspltw 51, 1, 1 + xxspltw 43, 6, 1 + std 26, 272(1) + xxspltw 41, 7, 1 + std 27, 280(1) + std 28, 288(1) + std 29, 296(1) + stxvd2x 56, 1, 11 + li 11, 128 + stfd 20, 320(1) + stxvd2x 57, 1, 11 + li 11, 144 + stfd 21, 328(1) + stxvd2x 58, 1, 11 + li 11, 160 + stfd 22, 336(1) + stxvd2x 59, 1, 11 + li 11, 176 + stfd 23, 344(1) + stxvd2x 60, 1, 11 + li 11, 192 + stfd 24, 352(1) + stxvd2x 61, 1, 11 + li 11, 208 + stfd 25, 360(1) + stxvd2x 62, 1, 11 + li 11, 224 + stfd 26, 368(1) + stxvd2x 63, 1, 11 + li 11, 16 + xxspltw 63, 4, 1 + lfiwzx 5, 5, 11 + ld 5, 448(1) + stfd 27, 376(1) + stfd 28, 384(1) + stfd 29, 392(1) + stfd 30, 400(1) + stfd 31, 408(1) + xxspltw 50, 5, 1 + beq 0, .LBB3_5 + addis 30, 2, .LCPI3_0@toc@ha + neg 7, 7 + xxleqv 34, 34, 34 + addis 28, 2, .LCPI3_5@toc@ha + addis 27, 2, .LCPI3_6@toc@ha + addis 26, 2, .LCPI3_7@toc@ha + addis 29, 2, .LCPI3_4@toc@ha + addis 25, 2, .LCPI3_8@toc@ha + addi 0, 30, .LCPI3_0@toc@l + mtfprwz 2, 7 + addis 7, 2, .LCPI3_1@toc@ha + addis 30, 2, .LCPI3_3@toc@ha + addi 24, 29, .LCPI3_4@toc@l + ld 29, 24(3) + lxvd2x 1, 0, 0 + mtfprwz 0, 6 + rldicl 6, 6, 32, 32 + addi 0, 30, .LCPI3_3@toc@l + ld 30, 16(3) + xxspltw 2, 2, 1 + vslw 2, 2, 2 + xxspltw 37, 0, 1 + mtfprwz 0, 6 + addi 6, 7, .LCPI3_1@toc@l + addis 7, 2, .LCPI3_2@toc@ha + xxswapd 35, 1 + xxlxor 36, 36, 36 + xxspltw 33, 0, 1 + xxland 35, 2, 35 + vadduwm 0, 3, 5 + lvx 5, 0, 6 + addi 6, 7, .LCPI3_2@toc@l + ld 7, 8(3) + xxlor 35, 35, 34 + xxlxor 34, 32, 34 + xxlor 9, 32, 32 + lvx 0, 0, 6 + ld 6, 0(3) + addi 3, 3, -8 + vcmpgtsw 2, 3, 2 + lvx 3, 0, 0 + addi 0, 28, .LCPI3_5@toc@l + addi 28, 27, .LCPI3_6@toc@l + addi 27, 26, .LCPI3_7@toc@l + addi 26, 25, .LCPI3_8@toc@l + or 25, 9, 8 + li 9, 0 + vcmpgtsb 5, 4, 5 + vcmpgtsb 0, 4, 0 + xxlor 11, 35, 35 + lvx 3, 0, 24 + xxlor 12, 35, 35 + vsubuwm 2, 1, 2 + xxlnor 10, 37, 37 + xxlor 13, 34, 34 + lvx 2, 0, 0 + li 0, 32 + xxlnor 31, 32, 32 + xxlor 30, 34, 34 + lvx 2, 0, 28 + li 28, 48 + xxlor 29, 34, 34 + lvx 2, 0, 27 + li 27, 0 + xxlor 28, 34, 34 + lvx 2, 0, 26 + xxlor 27, 34, 34 +.LBB3_2: + mr 26, 27 + addi 27, 27, 1 + xxlor 23, 39, 39 + cmpld 27, 4 + sldi 26, 26, 6 + xxlor 24, 40, 40 + iseleq 24, 10, 9 + add 23, 6, 26 + add 22, 30, 26 + lxvd2x 0, 6, 26 + lxvd2x 1, 7, 26 + or 25, 24, 25 + add 24, 7, 26 + lxvd2x 2, 30, 26 + lxvd2x 3, 29, 26 + xxlor 26, 47, 47 + lxvd2x 4, 23, 11 + lxvd2x 6, 24, 11 + clrlwi 25, 25, 24 + xxlor 25, 51, 51 + lxvd2x 7, 22, 11 + lxvd2x 8, 23, 0 + mtfprd 5, 25 + add 25, 29, 26 + xxswapd 34, 0 + lxvd2x 0, 25, 11 + xxswapd 38, 1 + xxswapd 32, 2 + lxvd2x 1, 24, 0 + lxvd2x 2, 22, 0 + xxswapd 40, 3 + xxswapd 39, 4 + lxvd2x 3, 25, 0 + lxvd2x 4, 23, 28 + xxswapd 60, 6 + xxswapd 47, 7 + lxvd2x 6, 24, 28 + xxswapd 57, 8 + lxvd2x 7, 22, 28 + lxvd2x 8, 25, 28 + xxswapd 58, 0 + mr 25, 3 + xxswapd 53, 1 + xxswapd 56, 2 + xxswapd 52, 3 + xxswapd 55, 4 + xxswapd 54, 6 + xxswapd 0, 5 + xxswapd 42, 7 + xxswapd 48, 8 + mtctr 12 +.LBB3_3: + ldu 24, 8(25) + add 24, 24, 26 + addi 24, 24, 256 + dcbt 0, 24 + bdnz .LBB3_3 + vmrgew 4, 28, 7 + vspltisw 14, 9 + mr 25, 8 + vmrgew 27, 6, 2 + vspltisw 17, 4 + vmrglw 12, 6, 2 + vspltisw 19, 10 + vmrghw 30, 6, 2 + xxspltw 0, 0, 3 + vmrglw 2, 8, 0 + vmrghw 13, 8, 0 + xxlor 7, 36, 36 + vmrgew 4, 21, 25 + vmrglw 29, 28, 7 + vmrghw 1, 28, 7 + vmrglw 28, 26, 15 + xxmrgld 37, 34, 44 + vmrgew 7, 26, 15 + vmrghw 15, 26, 15 + xxlor 21, 36, 36 + vmrglw 4, 21, 25 + vmrghw 21, 21, 25 + vmrglw 25, 20, 24 + xxmrgld 34, 60, 61 + vmrghw 26, 20, 24 + xxlor 38, 26, 26 + vmrgew 3, 8, 0 + xxlor 5, 36, 36 + vmrgew 4, 20, 24 + vspltisw 24, -16 + vmrglw 20, 22, 23 + xxmrgld 57, 57, 5 + vmrglw 8, 16, 10 + vmrghw 0, 16, 10 + vadduwm 12, 19, 19 + xxlor 8, 37, 37 + xxlor 20, 36, 36 + vmrgew 4, 22, 23 + vmrghw 23, 22, 23 + xxmrgld 40, 40, 52 + vmrgew 22, 16, 10 + vsubuwm 10, 14, 24 + vslw 14, 17, 17 + vadduwm 17, 5, 6 + xxmrgld 37, 47, 33 + xxlor 22, 36, 36 + xxmrgld 36, 45, 62 + xxlor 38, 25, 25 + xxlor 2, 34, 34 + vadduwm 19, 4, 6 + xxmrgld 38, 39, 7 + xxlor 3, 36, 36 + xxmrghd 39, 47, 33 + xxlor 36, 24, 24 + xxmrgld 33, 58, 53 + vadduwm 17, 17, 18 + vadduwm 29, 2, 4 + xxmrgld 36, 35, 59 + xxlor 34, 23, 23 + xxmrghd 35, 45, 62 + xxlor 1, 9, 9 + vadduwm 28, 5, 2 + xxlor 1, 13, 13 + vadduwm 19, 19, 31 + vadduwm 24, 29, 11 + vadduwm 28, 28, 9 + xxlxor 61, 49, 9 + xxlor 1, 41, 41 + xxlor 41, 11, 11 + xxlxor 34, 51, 13 + vperm 29, 29, 29, 9 + xxlxor 46, 56, 46 + vperm 2, 2, 2, 9 + xxlxor 59, 60, 0 + vperm 14, 14, 14, 9 + vperm 30, 27, 27, 9 + vadduwm 19, 19, 3 + xxlor 4, 35, 35 + xxland 61, 61, 10 + xxlor 35, 12, 12 + xxland 34, 34, 10 + vadduwm 27, 29, 3 + xxlor 35, 30, 30 + vadduwm 17, 17, 4 + xxlor 26, 36, 36 + xxland 46, 46, 10 + vadduwm 3, 2, 3 + xxlor 36, 29, 29 + xxland 62, 62, 10 + xxlxor 45, 59, 50 + xxlxor 50, 35, 63 + vadduwm 31, 14, 4 + xxlor 36, 28, 28 + xxlor 6, 37, 37 + vadduwm 16, 30, 4 + xxlxor 43, 63, 43 + xxlxor 37, 48, 1 + vrlw 4, 13, 12 + vrlw 18, 18, 12 + vrlw 11, 11, 12 + vrlw 5, 5, 12 + vadduwm 15, 24, 6 + vadduwm 28, 28, 7 + vadduwm 17, 4, 17 + vadduwm 19, 18, 19 + vadduwm 15, 11, 15 + vadduwm 28, 5, 28 + xxlor 25, 38, 38 + xxlxor 61, 49, 61 + xxlxor 34, 51, 34 + xxlxor 46, 47, 46 + xxlxor 62, 60, 62 + xxlor 38, 27, 27 + vadduwm 19, 19, 1 + vperm 29, 29, 29, 6 + vperm 2, 2, 2, 6 + vperm 24, 14, 14, 6 + vperm 30, 30, 30, 6 + xxlor 5, 33, 33 + vadduwm 17, 17, 25 + xxland 61, 61, 31 + xxland 34, 34, 31 + xxland 56, 56, 31 + xxland 62, 62, 31 + vadduwm 27, 29, 27 + vadduwm 3, 2, 3 + vadduwm 31, 24, 31 + vadduwm 16, 30, 16 + xxlxor 36, 59, 36 + xxlxor 50, 35, 50 + xxlxor 43, 63, 43 + xxlxor 37, 48, 37 + vrlw 1, 18, 10 + xxmrgld 50, 32, 55 + vrlw 11, 11, 10 + xxmrghd 55, 32, 55 + vrlw 5, 5, 10 + vrlw 4, 4, 10 + vadduwm 15, 15, 8 + vadduwm 28, 28, 18 + vadduwm 17, 1, 17 + vadduwm 19, 11, 19 + vadduwm 15, 5, 15 + vadduwm 28, 4, 28 + xxlor 7, 57, 57 + xxlxor 62, 49, 62 + xxlxor 61, 51, 61 + xxlxor 57, 47, 34 + xxlxor 34, 60, 56 + vperm 24, 30, 30, 9 + xxmrgld 62, 20, 21 + vperm 29, 29, 29, 9 + vperm 25, 25, 25, 9 + vperm 2, 2, 2, 9 + vmr 14, 8 + xxmrghd 40, 58, 53 + xxmrgld 58, 54, 22 + vadduwm 17, 17, 30 + xxland 56, 56, 10 + vadduwm 21, 19, 8 + xxland 61, 61, 10 + xxland 51, 57, 10 + xxland 34, 34, 10 + vadduwm 31, 24, 31 + vadduwm 16, 29, 16 + vadduwm 27, 19, 27 + vadduwm 3, 2, 3 + xxlxor 33, 63, 33 + xxlxor 43, 48, 43 + xxlxor 37, 59, 37 + xxlxor 36, 35, 36 + vrlw 1, 1, 12 + vrlw 11, 11, 12 + vrlw 5, 5, 12 + vrlw 4, 4, 12 + vadduwm 0, 15, 26 + vadduwm 15, 28, 23 + vadduwm 17, 1, 17 + vadduwm 28, 11, 21 + vadduwm 0, 5, 0 + vadduwm 15, 4, 15 + xxlxor 56, 49, 56 + xxlxor 61, 60, 61 + xxlxor 51, 32, 51 + xxlxor 34, 47, 34 + vperm 24, 24, 24, 6 + vperm 29, 29, 29, 6 + vperm 19, 19, 19, 6 + vperm 2, 2, 2, 6 + vmr 13, 8 + xxlor 53, 3, 3 + xxland 56, 56, 31 + xxland 61, 61, 31 + xxland 51, 51, 31 + xxland 34, 34, 31 + vadduwm 31, 24, 31 + vadduwm 16, 29, 16 + vadduwm 27, 19, 27 + vadduwm 3, 2, 3 + xxlxor 33, 63, 33 + xxlxor 43, 48, 43 + xxlxor 36, 35, 36 + xxlxor 37, 59, 37 + vrlw 4, 4, 10 + vrlw 1, 1, 10 + vrlw 11, 11, 10 + vrlw 5, 5, 10 + xxlor 52, 4, 4 + xxlor 40, 2, 2 + vadduwm 17, 17, 21 + vadduwm 28, 28, 20 + vadduwm 0, 0, 7 + vadduwm 15, 15, 8 + vadduwm 17, 4, 17 + vadduwm 28, 1, 28 + vadduwm 0, 11, 0 + vadduwm 15, 5, 15 + xxlxor 61, 49, 61 + xxlxor 51, 60, 51 + xxlxor 34, 32, 34 + xxlxor 56, 47, 56 + vperm 29, 29, 29, 9 + vperm 19, 19, 19, 9 + vperm 2, 2, 2, 9 + vperm 24, 24, 24, 9 + vmr 25, 26 + xxlor 3, 39, 39 + xxland 61, 61, 10 + xxland 51, 51, 10 + xxland 34, 34, 10 + xxland 56, 56, 10 + vadduwm 27, 29, 27 + vadduwm 3, 19, 3 + vadduwm 31, 2, 31 + vadduwm 16, 24, 16 + xxlxor 36, 59, 36 + xxlxor 33, 35, 33 + xxlxor 43, 63, 43 + xxlxor 37, 48, 37 + vrlw 4, 4, 12 + vrlw 1, 1, 12 + vrlw 11, 11, 12 + vrlw 5, 5, 12 + xxlor 54, 6, 6 + xxlor 58, 5, 5 + xxlor 39, 8, 8 + vadduwm 17, 17, 22 + vadduwm 28, 28, 26 + vadduwm 0, 0, 7 + vadduwm 15, 15, 25 + vadduwm 17, 4, 17 + vadduwm 28, 1, 28 + vadduwm 0, 11, 0 + vadduwm 15, 5, 15 + xxlxor 61, 49, 61 + xxlxor 51, 60, 51 + xxlxor 34, 32, 34 + xxlxor 56, 47, 56 + vperm 29, 29, 29, 6 + vperm 19, 19, 19, 6 + vperm 2, 2, 2, 6 + vperm 24, 24, 24, 6 + xxlor 39, 26, 26 + vadduwm 28, 28, 14 + xxland 61, 61, 31 + xxland 51, 51, 31 + xxland 34, 34, 31 + xxland 56, 56, 31 + vadduwm 27, 29, 27 + vadduwm 3, 19, 3 + vadduwm 31, 2, 31 + vadduwm 16, 24, 16 + xxlxor 36, 59, 36 + xxlxor 33, 35, 33 + xxlxor 43, 63, 43 + xxlxor 37, 48, 37 + vrlw 1, 1, 10 + vrlw 11, 11, 10 + vrlw 5, 5, 10 + vrlw 4, 4, 10 + vadduwm 17, 17, 7 + vadduwm 0, 0, 30 + vadduwm 15, 15, 23 + vadduwm 17, 1, 17 + vadduwm 28, 11, 28 + vadduwm 0, 5, 0 + vadduwm 15, 4, 15 + xxlxor 56, 49, 56 + xxlxor 61, 60, 61 + xxlxor 51, 32, 51 + xxlxor 34, 47, 34 + vperm 24, 24, 24, 9 + vperm 29, 29, 29, 9 + vperm 19, 19, 19, 9 + vperm 2, 2, 2, 9 + xxlor 24, 55, 55 + vadduwm 17, 17, 13 + xxland 56, 56, 10 + xxland 61, 61, 10 + xxland 51, 51, 10 + xxland 34, 34, 10 + vadduwm 31, 24, 31 + vadduwm 16, 29, 16 + vadduwm 27, 19, 27 + vadduwm 3, 2, 3 + xxlxor 33, 63, 33 + xxlxor 43, 48, 43 + xxlxor 37, 59, 37 + xxlxor 36, 35, 36 + vrlw 1, 1, 12 + vrlw 11, 11, 12 + vrlw 5, 5, 12 + vrlw 4, 4, 12 + vmr 23, 13 + xxlor 45, 25, 25 + xxlor 39, 7, 7 + vadduwm 28, 28, 13 + vadduwm 0, 0, 18 + vadduwm 15, 15, 7 + vadduwm 17, 1, 17 + vadduwm 28, 11, 28 + vadduwm 0, 5, 0 + vadduwm 15, 4, 15 + xxlxor 56, 49, 56 + xxlxor 61, 60, 61 + xxlxor 51, 32, 51 + xxlxor 34, 47, 34 + vperm 24, 24, 24, 6 + vperm 29, 29, 29, 6 + vperm 19, 19, 19, 6 + vperm 2, 2, 2, 6 + xxlor 2, 46, 46 + xxlor 46, 3, 3 + xxland 56, 56, 31 + xxland 61, 61, 31 + xxland 51, 51, 31 + xxland 34, 34, 31 + vadduwm 31, 24, 31 + vadduwm 16, 29, 16 + vadduwm 27, 19, 27 + vadduwm 3, 2, 3 + xxlxor 33, 63, 33 + xxlxor 43, 48, 43 + xxlxor 36, 35, 36 + xxlxor 37, 59, 37 + vrlw 4, 4, 10 + vrlw 1, 1, 10 + vrlw 11, 11, 10 + vrlw 5, 5, 10 + vadduwm 17, 17, 20 + vadduwm 28, 28, 26 + vadduwm 0, 0, 25 + vadduwm 15, 15, 14 + vadduwm 17, 4, 17 + vadduwm 28, 1, 28 + vadduwm 0, 11, 0 + vadduwm 15, 5, 15 + xxlxor 61, 49, 61 + xxlxor 51, 60, 51 + xxlxor 34, 32, 34 + xxlxor 56, 47, 56 + vperm 29, 29, 29, 9 + vperm 19, 19, 19, 9 + vperm 2, 2, 2, 9 + vperm 24, 24, 24, 9 + xxlor 52, 2, 2 + vadduwm 17, 17, 8 + xxland 61, 61, 10 + xxland 51, 51, 10 + xxland 34, 34, 10 + xxland 56, 56, 10 + vadduwm 27, 29, 27 + vadduwm 3, 19, 3 + vadduwm 31, 2, 31 + vadduwm 16, 24, 16 + xxlxor 36, 59, 36 + xxlxor 33, 35, 33 + xxlxor 43, 63, 43 + xxlxor 37, 48, 37 + vrlw 4, 4, 12 + vrlw 1, 1, 12 + vrlw 11, 11, 12 + vrlw 5, 5, 12 + vadduwm 28, 28, 20 + vadduwm 0, 0, 21 + vadduwm 15, 15, 18 + vadduwm 17, 4, 17 + vadduwm 28, 1, 28 + vadduwm 0, 11, 0 + vadduwm 15, 5, 15 + xxlxor 61, 49, 61 + xxlxor 51, 60, 51 + xxlxor 34, 32, 34 + xxlxor 56, 47, 56 + vperm 29, 29, 29, 6 + vperm 19, 19, 19, 6 + vperm 2, 2, 2, 6 + vperm 24, 24, 24, 6 + vadduwm 17, 17, 22 + vadduwm 28, 28, 30 + xxland 61, 61, 31 + xxland 51, 51, 31 + xxland 34, 34, 31 + xxland 56, 56, 31 + vadduwm 27, 29, 27 + vadduwm 3, 19, 3 + vadduwm 31, 2, 31 + vadduwm 16, 24, 16 + xxlxor 36, 59, 36 + xxlxor 33, 35, 33 + xxlxor 43, 63, 43 + xxlxor 37, 48, 37 + vrlw 1, 1, 10 + vrlw 11, 11, 10 + vrlw 5, 5, 10 + vrlw 4, 4, 10 + vadduwm 0, 0, 23 + vadduwm 15, 15, 7 + vadduwm 17, 1, 17 + vadduwm 28, 11, 28 + vadduwm 0, 5, 0 + vadduwm 15, 4, 15 + xxlxor 56, 49, 56 + xxlxor 61, 60, 61 + xxlxor 51, 32, 51 + xxlxor 34, 47, 34 + vperm 24, 24, 24, 9 + vperm 29, 29, 29, 9 + vperm 19, 19, 19, 9 + vperm 2, 2, 2, 9 + xxlor 5, 4, 4 + xxlor 4, 58, 58 + xxland 56, 56, 10 + xxland 61, 61, 10 + xxland 51, 51, 10 + xxland 34, 34, 10 + vadduwm 31, 24, 31 + vadduwm 16, 29, 16 + vadduwm 27, 19, 27 + vadduwm 3, 2, 3 + xxlxor 33, 63, 33 + xxlxor 43, 48, 43 + xxlxor 37, 59, 37 + xxlxor 36, 35, 36 + vrlw 1, 1, 12 + vrlw 11, 11, 12 + vrlw 5, 5, 12 + vrlw 4, 4, 12 + xxlor 39, 8, 8 + xxlor 54, 24, 24 + xxlor 58, 26, 26 + vadduwm 17, 17, 13 + vadduwm 28, 28, 7 + vadduwm 0, 0, 22 + vadduwm 15, 15, 26 + vadduwm 17, 1, 17 + vadduwm 28, 11, 28 + vadduwm 0, 5, 0 + vadduwm 15, 4, 15 + xxlxor 56, 49, 56 + xxlxor 61, 60, 61 + xxlxor 51, 32, 51 + xxlxor 34, 47, 34 + vperm 24, 24, 24, 6 + vperm 29, 29, 29, 6 + vperm 19, 19, 19, 6 + vperm 2, 2, 2, 6 + xxlor 3, 53, 53 + xxlor 53, 4, 4 + xxland 56, 56, 31 + xxland 61, 61, 31 + xxland 51, 51, 31 + xxland 34, 34, 31 + vadduwm 31, 24, 31 + vadduwm 16, 29, 16 + vadduwm 27, 19, 27 + vadduwm 3, 2, 3 + xxlxor 33, 63, 33 + xxlxor 43, 48, 43 + xxlxor 36, 35, 36 + xxlxor 37, 59, 37 + vrlw 4, 4, 10 + vrlw 1, 1, 10 + vrlw 11, 11, 10 + vrlw 5, 5, 10 + vadduwm 17, 17, 21 + vadduwm 28, 28, 20 + vadduwm 0, 0, 18 + vadduwm 15, 15, 25 + vadduwm 17, 4, 17 + vadduwm 28, 1, 28 + vadduwm 0, 11, 0 + vadduwm 15, 5, 15 + xxlxor 61, 49, 61 + xxlxor 51, 60, 51 + xxlxor 34, 32, 34 + xxlxor 56, 47, 56 + vperm 29, 29, 29, 9 + vperm 19, 19, 19, 9 + vperm 2, 2, 2, 9 + vperm 24, 24, 24, 9 + xxlor 2, 55, 55 + vmr 23, 18 + xxland 61, 61, 10 + xxland 51, 51, 10 + xxland 34, 34, 10 + xxland 56, 56, 10 + vadduwm 27, 29, 27 + vadduwm 3, 19, 3 + vadduwm 31, 2, 31 + vadduwm 16, 24, 16 + xxlxor 36, 59, 36 + xxlxor 33, 35, 33 + xxlxor 43, 63, 43 + xxlxor 37, 48, 37 + vrlw 4, 4, 12 + vrlw 1, 1, 12 + vrlw 11, 11, 12 + vrlw 5, 5, 12 + xxlor 50, 5, 5 + vadduwm 17, 17, 14 + vadduwm 28, 28, 30 + vadduwm 0, 0, 18 + vadduwm 15, 15, 22 + vadduwm 17, 4, 17 + vadduwm 28, 1, 28 + vadduwm 0, 11, 0 + vadduwm 15, 5, 15 + xxlxor 61, 49, 61 + xxlxor 51, 60, 51 + xxlxor 34, 32, 34 + xxlxor 56, 47, 56 + vperm 29, 29, 29, 6 + vperm 19, 19, 19, 6 + vperm 2, 2, 2, 6 + vperm 24, 24, 24, 6 + xxlor 25, 40, 40 + vmr 8, 13 + xxland 61, 61, 31 + xxland 51, 51, 31 + xxland 34, 34, 31 + xxland 56, 56, 31 + vadduwm 27, 29, 27 + vadduwm 3, 19, 3 + vadduwm 31, 2, 31 + vadduwm 16, 24, 16 + xxlxor 36, 59, 36 + xxlxor 33, 35, 33 + xxlxor 43, 63, 43 + xxlxor 37, 48, 37 + xxlor 45, 25, 25 + vrlw 1, 1, 10 + vrlw 11, 11, 10 + vrlw 5, 5, 10 + vrlw 4, 4, 10 + vadduwm 17, 17, 13 + xxlor 45, 2, 2 + vadduwm 0, 0, 8 + vadduwm 28, 28, 13 + vadduwm 15, 15, 26 + vadduwm 17, 1, 17 + vadduwm 28, 11, 28 + vadduwm 0, 5, 0 + vadduwm 15, 4, 15 + xxlxor 56, 49, 56 + xxlxor 61, 60, 61 + xxlxor 51, 32, 51 + xxlxor 34, 47, 34 + vperm 24, 24, 24, 9 + vperm 29, 29, 29, 9 + vperm 19, 19, 19, 9 + vperm 2, 2, 2, 9 + xxlor 4, 57, 57 + xxlor 26, 46, 46 + xxland 56, 56, 10 + xxland 61, 61, 10 + xxland 51, 51, 10 + xxland 34, 34, 10 + vadduwm 31, 24, 31 + vadduwm 16, 29, 16 + vadduwm 27, 19, 27 + vadduwm 3, 2, 3 + xxlxor 33, 63, 33 + xxlxor 43, 48, 43 + xxlxor 37, 59, 37 + xxlxor 36, 35, 36 + vrlw 1, 1, 12 + vrlw 11, 11, 12 + vrlw 5, 5, 12 + vrlw 4, 4, 12 + xxlor 8, 62, 62 + xxlor 57, 3, 3 + xxlor 46, 7, 7 + xxlor 62, 6, 6 + vadduwm 17, 17, 7 + vadduwm 28, 28, 25 + vadduwm 0, 0, 14 + vadduwm 15, 15, 30 + vadduwm 17, 1, 17 + vadduwm 28, 11, 28 + vadduwm 0, 5, 0 + vadduwm 15, 4, 15 + xxlxor 56, 49, 56 + xxlxor 61, 60, 61 + xxlxor 51, 32, 51 + xxlxor 34, 47, 34 + vperm 24, 24, 24, 6 + vperm 29, 29, 29, 6 + vperm 19, 19, 19, 6 + vperm 2, 2, 2, 6 + vadduwm 17, 17, 20 + xxlor 3, 52, 52 + xxland 56, 56, 31 + xxland 61, 61, 31 + xxland 51, 51, 31 + xxland 34, 34, 31 + vadduwm 31, 24, 31 + vadduwm 16, 29, 16 + vadduwm 27, 19, 27 + vadduwm 3, 2, 3 + xxlxor 33, 63, 33 + xxlxor 43, 48, 43 + xxlxor 36, 35, 36 + xxlxor 37, 59, 37 + vrlw 4, 4, 10 + vrlw 1, 1, 10 + vrlw 11, 11, 10 + vrlw 5, 5, 10 + xxlor 52, 8, 8 + vadduwm 0, 0, 22 + vadduwm 28, 28, 20 + vadduwm 15, 15, 23 + vadduwm 17, 4, 17 + vadduwm 28, 1, 28 + vadduwm 0, 11, 0 + vadduwm 15, 5, 15 + xxlxor 61, 49, 61 + xxlxor 51, 60, 51 + xxlxor 34, 32, 34 + xxlxor 56, 47, 56 + vperm 29, 29, 29, 9 + vperm 19, 19, 19, 9 + vperm 2, 2, 2, 9 + vperm 24, 24, 24, 9 + xxlor 6, 55, 55 + xxlor 55, 4, 4 + xxland 61, 61, 10 + xxland 51, 51, 10 + xxland 34, 34, 10 + xxland 56, 56, 10 + vadduwm 27, 29, 27 + vadduwm 3, 19, 3 + vadduwm 31, 2, 31 + vadduwm 16, 24, 16 + xxlxor 36, 59, 36 + xxlxor 33, 35, 33 + xxlxor 43, 63, 43 + xxlxor 37, 48, 37 + vrlw 4, 4, 12 + vrlw 1, 1, 12 + vrlw 11, 11, 12 + vrlw 5, 5, 12 + vadduwm 17, 17, 23 + vadduwm 28, 28, 13 + vadduwm 0, 0, 21 + vadduwm 15, 15, 14 + vadduwm 17, 4, 17 + vadduwm 28, 1, 28 + vadduwm 0, 11, 0 + vadduwm 15, 5, 15 + xxlxor 61, 49, 61 + xxlxor 51, 60, 51 + xxlxor 34, 32, 34 + xxlxor 56, 47, 56 + vperm 29, 29, 29, 6 + vperm 19, 19, 19, 6 + vperm 2, 2, 2, 6 + vperm 24, 24, 24, 6 + xxlor 4, 53, 53 + xxlor 53, 26, 26 + xxland 61, 61, 31 + xxland 51, 51, 31 + xxland 34, 34, 31 + xxland 56, 56, 31 + vadduwm 27, 29, 27 + vadduwm 3, 19, 3 + vadduwm 31, 2, 31 + vadduwm 16, 24, 16 + xxlxor 36, 59, 36 + xxlxor 33, 35, 33 + xxlxor 43, 63, 43 + xxlxor 37, 48, 37 + vrlw 1, 1, 10 + vrlw 11, 11, 10 + vrlw 5, 5, 10 + vrlw 4, 4, 10 + vadduwm 17, 17, 21 + vadduwm 28, 28, 8 + vadduwm 0, 0, 7 + vadduwm 15, 15, 30 + vadduwm 17, 1, 17 + vadduwm 28, 11, 28 + vadduwm 0, 5, 0 + vadduwm 15, 4, 15 + xxlxor 56, 49, 56 + xxlxor 61, 60, 61 + xxlxor 51, 32, 51 + xxlxor 34, 47, 34 + vperm 24, 24, 24, 9 + vperm 29, 29, 29, 9 + vperm 19, 19, 19, 9 + vperm 2, 2, 2, 9 + xxlor 5, 25, 25 + xxlor 2, 58, 58 + xxland 56, 56, 10 + xxland 61, 61, 10 + xxland 51, 51, 10 + xxland 34, 34, 10 + vadduwm 31, 24, 31 + vadduwm 16, 29, 16 + vadduwm 27, 19, 27 + vadduwm 3, 2, 3 + xxlxor 33, 63, 33 + xxlxor 43, 48, 43 + xxlxor 37, 59, 37 + xxlxor 36, 35, 36 + vrlw 1, 1, 12 + vrlw 11, 11, 12 + vrlw 5, 5, 12 + vrlw 4, 4, 12 + vmr 22, 26 + vadduwm 0, 0, 26 + xxlor 58, 5, 5 + vadduwm 17, 17, 25 + vadduwm 28, 28, 18 + vadduwm 15, 15, 26 + vadduwm 17, 1, 17 + vadduwm 28, 11, 28 + vadduwm 0, 5, 0 + vadduwm 15, 4, 15 + xxlxor 56, 49, 56 + xxlxor 61, 60, 61 + xxlxor 51, 32, 51 + xxlxor 34, 47, 34 + vperm 24, 24, 24, 6 + vperm 29, 29, 29, 6 + vperm 19, 19, 19, 6 + vperm 2, 2, 2, 6 + xxlor 7, 24, 24 + xxlor 8, 57, 57 + xxland 56, 56, 31 + xxland 61, 61, 31 + xxland 51, 51, 31 + xxland 34, 34, 31 + vadduwm 31, 24, 31 + vadduwm 16, 29, 16 + vadduwm 27, 19, 27 + vadduwm 3, 2, 3 + xxlxor 33, 63, 33 + xxlxor 43, 48, 43 + xxlxor 36, 35, 36 + xxlxor 37, 59, 37 + vrlw 4, 4, 10 + vrlw 1, 1, 10 + vrlw 11, 11, 10 + vrlw 5, 5, 10 + xxlor 57, 7, 7 + vadduwm 17, 17, 20 + vadduwm 28, 28, 13 + vadduwm 0, 0, 14 + vadduwm 15, 15, 25 + vadduwm 17, 4, 17 + vadduwm 28, 1, 28 + vadduwm 0, 11, 0 + vadduwm 15, 5, 15 + xxlxor 61, 49, 61 + xxlxor 51, 60, 51 + xxlxor 34, 32, 34 + xxlxor 56, 47, 56 + vperm 29, 29, 29, 9 + vperm 19, 19, 19, 9 + vperm 2, 2, 2, 9 + vperm 24, 24, 24, 9 + xxlor 5, 52, 52 + xxlor 23, 45, 45 + xxland 61, 61, 10 + xxland 51, 51, 10 + xxland 34, 34, 10 + xxland 56, 56, 10 + vadduwm 27, 29, 27 + vadduwm 3, 19, 3 + vadduwm 31, 2, 31 + vadduwm 16, 24, 16 + xxlxor 36, 59, 36 + xxlxor 33, 35, 33 + xxlxor 43, 63, 43 + xxlxor 37, 48, 37 + vrlw 4, 4, 12 + vrlw 1, 1, 12 + vrlw 11, 11, 12 + vrlw 5, 5, 12 + xxlor 52, 6, 6 + vadduwm 28, 28, 8 + vmr 13, 8 + xxlor 40, 3, 3 + vadduwm 17, 17, 20 + vadduwm 0, 0, 8 + vadduwm 15, 15, 22 + vadduwm 17, 4, 17 + vadduwm 28, 1, 28 + vadduwm 0, 11, 0 + vadduwm 15, 5, 15 + xxlxor 61, 49, 61 + xxlxor 51, 60, 51 + xxlxor 34, 32, 34 + xxlxor 56, 47, 56 + vperm 29, 29, 29, 6 + vperm 19, 19, 19, 6 + vperm 2, 2, 2, 6 + vperm 24, 24, 24, 6 + xxlor 25, 39, 39 + vmr 7, 30 + xxland 61, 61, 31 + xxland 51, 51, 31 + xxland 34, 34, 31 + xxland 56, 56, 31 + vadduwm 27, 29, 27 + vadduwm 3, 19, 3 + vadduwm 31, 2, 31 + vadduwm 16, 24, 16 + xxlxor 36, 59, 36 + xxlxor 33, 35, 33 + xxlxor 43, 63, 43 + xxlxor 37, 48, 37 + vrlw 1, 1, 10 + vrlw 11, 11, 10 + vrlw 5, 5, 10 + vrlw 4, 4, 10 + vmr 30, 18 + xxlor 24, 46, 46 + xxlor 46, 25, 25 + xxlor 50, 8, 8 + vadduwm 17, 17, 23 + vadduwm 28, 28, 14 + vadduwm 0, 0, 18 + vadduwm 15, 15, 26 + vadduwm 17, 1, 17 + vadduwm 28, 11, 28 + vadduwm 0, 5, 0 + vadduwm 15, 4, 15 + xxlxor 56, 49, 56 + xxlxor 61, 60, 61 + xxlxor 51, 32, 51 + xxlxor 34, 47, 34 + vperm 24, 24, 24, 9 + vperm 29, 29, 29, 9 + vperm 19, 19, 19, 9 + vperm 2, 2, 2, 9 + xxlor 6, 58, 58 + xxlor 58, 4, 4 + xxland 56, 56, 10 + xxland 61, 61, 10 + xxland 51, 51, 10 + xxland 34, 34, 10 + vadduwm 31, 24, 31 + vadduwm 16, 29, 16 + vadduwm 27, 19, 27 + vadduwm 3, 2, 3 + xxlxor 33, 63, 33 + xxlxor 43, 48, 43 + xxlxor 37, 59, 37 + xxlxor 36, 35, 36 + vrlw 1, 1, 12 + vrlw 11, 11, 12 + vrlw 5, 5, 12 + vrlw 4, 4, 12 + vadduwm 17, 17, 30 + vadduwm 28, 28, 26 + vadduwm 0, 0, 7 + vadduwm 15, 15, 21 + vadduwm 17, 1, 17 + vadduwm 28, 11, 28 + vadduwm 0, 5, 0 + vadduwm 15, 4, 15 + xxlxor 56, 49, 56 + xxlxor 61, 60, 61 + xxlxor 51, 32, 51 + xxlxor 34, 47, 34 + vperm 24, 24, 24, 6 + vperm 29, 29, 29, 6 + vperm 19, 19, 19, 6 + vperm 2, 2, 2, 6 + xxlor 40, 23, 23 + vadduwm 13, 28, 13 + vadduwm 8, 17, 8 + xxland 49, 56, 31 + xxland 61, 61, 31 + xxland 51, 51, 31 + xxland 34, 34, 31 + vadduwm 31, 17, 31 + vadduwm 16, 29, 16 + vadduwm 28, 19, 27 + vadduwm 3, 2, 3 + xxlxor 33, 63, 33 + xxlxor 43, 48, 43 + xxlxor 36, 35, 36 + xxlxor 37, 60, 37 + vrlw 4, 4, 10 + vrlw 1, 1, 10 + vrlw 11, 11, 10 + vrlw 5, 5, 10 + xxlor 2, 55, 55 + vmr 23, 30 + xxlor 62, 24, 24 + vadduwm 0, 0, 22 + vadduwm 15, 15, 30 + vadduwm 8, 4, 8 + vadduwm 13, 1, 13 + vadduwm 0, 11, 0 + vadduwm 15, 5, 15 + xxlxor 61, 40, 61 + xxlxor 51, 45, 51 + xxlxor 34, 32, 34 + xxlxor 49, 47, 49 + vperm 29, 29, 29, 9 + vperm 19, 19, 19, 9 + vperm 2, 2, 2, 9 + vperm 17, 17, 17, 9 + vadduwm 13, 13, 14 + xxlor 46, 5, 5 + xxland 61, 61, 10 + xxland 51, 51, 10 + xxland 34, 34, 10 + xxland 49, 49, 10 + vadduwm 28, 29, 28 + vadduwm 3, 19, 3 + vadduwm 31, 2, 31 + vadduwm 16, 17, 16 + xxlxor 36, 60, 36 + xxlxor 33, 35, 33 + xxlxor 43, 63, 43 + xxlxor 37, 48, 37 + vrlw 4, 4, 12 + vrlw 1, 1, 12 + vrlw 11, 11, 12 + vrlw 5, 5, 12 + vadduwm 8, 8, 25 + vadduwm 0, 0, 14 + vadduwm 15, 15, 7 + vadduwm 8, 4, 8 + vadduwm 13, 1, 13 + vadduwm 0, 11, 0 + vadduwm 15, 5, 15 + xxlxor 62, 40, 61 + xxlxor 51, 45, 51 + xxlxor 34, 32, 34 + xxlxor 49, 47, 49 + vperm 30, 30, 30, 6 + vperm 19, 19, 19, 6 + vperm 2, 2, 2, 6 + vperm 17, 17, 17, 6 + vadduwm 29, 8, 20 + vadduwm 8, 13, 18 + xxland 45, 62, 31 + xxland 51, 51, 31 + xxland 34, 34, 31 + xxland 49, 49, 31 + vadduwm 30, 13, 28 + vadduwm 3, 19, 3 + vadduwm 31, 2, 31 + vadduwm 16, 17, 16 + xxlxor 36, 62, 36 + xxlxor 33, 35, 33 + xxlxor 43, 63, 43 + xxlxor 37, 48, 37 + vrlw 1, 1, 10 + vrlw 11, 11, 10 + vrlw 5, 5, 10 + vrlw 4, 4, 10 + vadduwm 0, 0, 23 + vadduwm 7, 15, 21 + vadduwm 29, 1, 29 + vadduwm 8, 11, 8 + vadduwm 0, 5, 0 + vadduwm 7, 4, 7 + xxlxor 47, 61, 49 + xxlxor 45, 40, 45 + xxlxor 49, 32, 51 + xxlxor 34, 39, 34 + vperm 15, 15, 15, 9 + vperm 13, 13, 13, 9 + vperm 17, 17, 17, 9 + vperm 2, 2, 2, 9 + xxlor 46, 3, 3 + vadduwm 9, 29, 26 + vadduwm 8, 8, 14 + xxland 46, 47, 10 + xxland 45, 45, 10 + xxland 47, 49, 10 + xxland 34, 34, 10 + vadduwm 17, 14, 31 + vadduwm 16, 13, 16 + vadduwm 18, 15, 30 + vadduwm 3, 2, 3 + xxlxor 33, 49, 33 + xxlxor 43, 48, 43 + xxlxor 37, 50, 37 + xxlxor 36, 35, 36 + vrlw 1, 1, 12 + vrlw 11, 11, 12 + vrlw 5, 5, 12 + vrlw 4, 4, 12 + xxlor 44, 6, 6 + xxlor 0, 10, 10 + vadduwm 0, 0, 12 + xxlor 44, 2, 2 + vadduwm 9, 1, 9 + vadduwm 7, 7, 12 + vadduwm 8, 11, 8 + vadduwm 7, 4, 7 + vadduwm 0, 5, 0 + xxlxor 34, 39, 34 + xxlxor 44, 32, 47 + vperm 2, 2, 2, 6 + xxlxor 46, 41, 46 + xxlxor 45, 40, 45 + vperm 12, 12, 12, 6 + vperm 14, 14, 14, 6 + vperm 13, 13, 13, 6 + xxland 34, 34, 31 + xxlor 1, 31, 31 + vadduwm 3, 2, 3 + xxland 44, 44, 31 + xxlxor 36, 35, 36 + xxlxor 51, 35, 40 + xxland 35, 46, 31 + xxland 38, 45, 31 + vadduwm 15, 12, 18 + vadduwm 8, 3, 17 + vadduwm 13, 6, 16 + xxlxor 37, 47, 37 + xxlxor 33, 40, 33 + xxlxor 43, 45, 43 + vrlw 4, 4, 10 + vrlw 1, 1, 10 + vrlw 11, 11, 10 + vrlw 5, 5, 10 + xxlxor 47, 47, 41 + xxlxor 40, 40, 32 + xxlxor 39, 45, 39 + xxlxor 50, 36, 38 + xxlxor 63, 33, 44 + xxlxor 43, 43, 34 + xxlxor 41, 37, 35 + bne 0, .LBB3_2 +.LBB3_5: + vmrglw 2, 19, 15 + li 3, 32 + li 4, 48 + vmrglw 4, 7, 8 + vmrglw 0, 31, 18 + vmrglw 1, 9, 11 + vmrghw 3, 19, 15 + vmrghw 5, 7, 8 + vmrghw 6, 31, 18 + vmrghw 7, 9, 11 + xxmrgld 40, 36, 34 + xxmrghd 34, 36, 34 + xxmrgld 41, 33, 32 + xxswapd 0, 40 + xxmrgld 36, 37, 35 + xxmrghd 35, 37, 35 + xxmrghd 37, 33, 32 + xxswapd 1, 41 + xxmrgld 32, 39, 38 + xxmrghd 33, 39, 38 + xxswapd 2, 34 + xxswapd 4, 36 + xxswapd 3, 37 + stxvd2x 0, 0, 5 + xxswapd 5, 32 + stxvd2x 1, 5, 11 + xxswapd 0, 35 + xxswapd 1, 33 + stxvd2x 2, 5, 3 + li 3, 64 + stxvd2x 3, 5, 4 + li 4, 80 + stxvd2x 4, 5, 3 + li 3, 96 + stxvd2x 5, 5, 4 + li 4, 112 + stxvd2x 0, 5, 3 + stxvd2x 1, 5, 4 + li 3, 224 + lxvd2x 63, 1, 3 + li 3, 208 + lfd 31, 408(1) + ld 30, 304(1) + ld 29, 296(1) + lxvd2x 62, 1, 3 + li 3, 192 + lfd 30, 400(1) + ld 28, 288(1) + ld 27, 280(1) + lxvd2x 61, 1, 3 + li 3, 176 + lfd 29, 392(1) + ld 26, 272(1) + ld 25, 264(1) + lxvd2x 60, 1, 3 + li 3, 160 + lfd 28, 384(1) + ld 24, 256(1) + ld 23, 248(1) + lxvd2x 59, 1, 3 + li 3, 144 + lfd 27, 376(1) + ld 22, 240(1) + lxvd2x 58, 1, 3 + li 3, 128 + lfd 26, 368(1) + lxvd2x 57, 1, 3 + li 3, 112 + lfd 25, 360(1) + lxvd2x 56, 1, 3 + li 3, 96 + lfd 24, 352(1) + lxvd2x 55, 1, 3 + li 3, 80 + lfd 23, 344(1) + lxvd2x 54, 1, 3 + li 3, 64 + lfd 22, 336(1) + lxvd2x 53, 1, 3 + li 3, 48 + lfd 21, 328(1) + lxvd2x 52, 1, 3 + lfd 20, 320(1) + addi 1, 1, 416 + blr + .long 0 + .quad 0 +.Lfunc_end3: + .size blake3_hash4_sse41, .Lfunc_end3-.Lfunc_begin3 + .cfi_endproc + .section ".note.GNU-stack","",@progbits +#endif diff --git a/module/icp/asm-x86_64/aes/aes_aesni.S b/module/icp/asm-x86_64/aes/aes_aesni.S index 1a8669ccd1d6..f622235bd15b 100644 --- a/module/icp/asm-x86_64/aes/aes_aesni.S +++ b/module/icp/asm-x86_64/aes/aes_aesni.S @@ -208,7 +208,7 @@ _key_expansion_256a_local: pxor %xmm1, %xmm0 movups %xmm0, (%rcx) add $0x10, %rcx - ret + RET nop SET_SIZE(_key_expansion_128) SET_SIZE(_key_expansion_256a) @@ -236,7 +236,7 @@ _key_expansion_192a_local: shufps $0b01001110, %xmm2, %xmm1 movups %xmm1, 0x10(%rcx) add $0x20, %rcx - ret + RET SET_SIZE(_key_expansion_192a) @@ -257,7 +257,7 @@ _key_expansion_192b_local: movups %xmm0, (%rcx) add $0x10, %rcx - ret + RET SET_SIZE(_key_expansion_192b) @@ -271,7 +271,7 @@ _key_expansion_256b_local: pxor %xmm1, %xmm2 movups %xmm2, (%rcx) add $0x10, %rcx - ret + RET SET_SIZE(_key_expansion_256b) @@ -376,7 +376,7 @@ rijndael_key_setup_enc_intel_local: mov $14, %rax // return # rounds = 14 #endif FRAME_END - ret + RET .align 4 .Lenc_key192: @@ -413,7 +413,7 @@ rijndael_key_setup_enc_intel_local: mov $12, %rax // return # rounds = 12 #endif FRAME_END - ret + RET .align 4 .Lenc_key128: @@ -453,13 +453,13 @@ rijndael_key_setup_enc_intel_local: mov $10, %rax // return # rounds = 10 #endif FRAME_END - ret + RET .Lenc_key_invalid_param: #ifdef OPENSSL_INTERFACE mov $-1, %rax // user key or AES key pointer is NULL FRAME_END - ret + RET #else /* FALLTHROUGH */ #endif /* OPENSSL_INTERFACE */ @@ -471,7 +471,7 @@ rijndael_key_setup_enc_intel_local: xor %rax, %rax // a key pointer is NULL or invalid keysize #endif /* OPENSSL_INTERFACE */ FRAME_END - ret + RET SET_SIZE(rijndael_key_setup_enc_intel) @@ -548,7 +548,7 @@ FRAME_BEGIN // OpenSolaris: rax = # rounds (10, 12, or 14) or 0 for error // OpenSSL: rax = 0 for OK, or non-zero for error FRAME_END - ret + RET SET_SIZE(rijndael_key_setup_dec_intel) @@ -655,7 +655,7 @@ ENTRY_NP(aes_encrypt_intel) aesenclast %KEY, %STATE // last round movups %STATE, (%OUTP) // output - ret + RET SET_SIZE(aes_encrypt_intel) @@ -738,7 +738,7 @@ ENTRY_NP(aes_decrypt_intel) aesdeclast %KEY, %STATE // last round movups %STATE, (%OUTP) // output - ret + RET SET_SIZE(aes_decrypt_intel) #endif /* lint || __lint */ diff --git a/module/icp/asm-x86_64/aes/aes_amd64.S b/module/icp/asm-x86_64/aes/aes_amd64.S index d16cc9996e25..f546e8933be1 100644 --- a/module/icp/asm-x86_64/aes/aes_amd64.S +++ b/module/icp/asm-x86_64/aes/aes_amd64.S @@ -785,7 +785,7 @@ ENTRY_NP(aes_encrypt_amd64) mov 2*8(%rsp), %rbp mov 3*8(%rsp), %r12 add $[4*8], %rsp - ret + RET SET_SIZE(aes_encrypt_amd64) @@ -896,7 +896,7 @@ ENTRY_NP(aes_decrypt_amd64) mov 2*8(%rsp), %rbp mov 3*8(%rsp), %r12 add $[4*8], %rsp - ret + RET SET_SIZE(aes_decrypt_amd64) #endif /* lint || __lint */ diff --git a/module/icp/asm-x86_64/aes/aestab2.h b/module/icp/asm-x86_64/aes/aestab2.h index eb13f72b10d8..003534e0fa50 100644 --- a/module/icp/asm-x86_64/aes/aestab2.h +++ b/module/icp/asm-x86_64/aes/aestab2.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/icp/asm-x86_64/blake3/blake3_avx2.S b/module/icp/asm-x86_64/blake3/blake3_avx2.S new file mode 100644 index 000000000000..f4d9cb766d46 --- /dev/null +++ b/module/icp/asm-x86_64/blake3/blake3_avx2.S @@ -0,0 +1,1845 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Based on BLAKE3 v1.3.1, https://github.com/BLAKE3-team/BLAKE3 + * Copyright (c) 2019-2020 Samuel Neves + * Copyright (c) 2022 Tino Reichardt + */ + +#if defined(HAVE_AVX2) + +#define _ASM +#include + +#if defined(__ELF__) && defined(__CET__) && defined(__has_include) +#if __has_include() +#include +#endif +#endif + +#if !defined(_CET_ENDBR) +#define _CET_ENDBR +#endif + +.intel_syntax noprefix +.global zfs_blake3_hash_many_avx2 +.text + +.type zfs_blake3_hash_many_avx2,@function +.p2align 6 +zfs_blake3_hash_many_avx2: + _CET_ENDBR + push r15 + push r14 + push r13 + push r12 + push rbx + push rbp + mov rbp, rsp + sub rsp, 680 + and rsp, 0xFFFFFFFFFFFFFFC0 + neg r9d + vmovd xmm0, r9d + vpbroadcastd ymm0, xmm0 + vmovdqa ymmword ptr [rsp+0x280], ymm0 + vpand ymm1, ymm0, ymmword ptr [ADD0+rip] + vpand ymm2, ymm0, ymmword ptr [ADD1+rip] + vmovdqa ymmword ptr [rsp+0x220], ymm2 + vmovd xmm2, r8d + vpbroadcastd ymm2, xmm2 + vpaddd ymm2, ymm2, ymm1 + vmovdqa ymmword ptr [rsp+0x240], ymm2 + vpxor ymm1, ymm1, ymmword ptr [CMP_MSB_MASK+rip] + vpxor ymm2, ymm2, ymmword ptr [CMP_MSB_MASK+rip] + vpcmpgtd ymm2, ymm1, ymm2 + shr r8, 32 + vmovd xmm3, r8d + vpbroadcastd ymm3, xmm3 + vpsubd ymm3, ymm3, ymm2 + vmovdqa ymmword ptr [rsp+0x260], ymm3 + shl rdx, 6 + mov qword ptr [rsp+0x2A0], rdx + cmp rsi, 8 + jc 3f +2: + vpbroadcastd ymm0, dword ptr [rcx] + vpbroadcastd ymm1, dword ptr [rcx+0x4] + vpbroadcastd ymm2, dword ptr [rcx+0x8] + vpbroadcastd ymm3, dword ptr [rcx+0xC] + vpbroadcastd ymm4, dword ptr [rcx+0x10] + vpbroadcastd ymm5, dword ptr [rcx+0x14] + vpbroadcastd ymm6, dword ptr [rcx+0x18] + vpbroadcastd ymm7, dword ptr [rcx+0x1C] + mov r8, qword ptr [rdi] + mov r9, qword ptr [rdi+0x8] + mov r10, qword ptr [rdi+0x10] + mov r11, qword ptr [rdi+0x18] + mov r12, qword ptr [rdi+0x20] + mov r13, qword ptr [rdi+0x28] + mov r14, qword ptr [rdi+0x30] + mov r15, qword ptr [rdi+0x38] + movzx eax, byte ptr [rbp+0x38] + movzx ebx, byte ptr [rbp+0x40] + or eax, ebx + xor edx, edx +.p2align 5 +9: + movzx ebx, byte ptr [rbp+0x48] + or ebx, eax + add rdx, 64 + cmp rdx, qword ptr [rsp+0x2A0] + cmove eax, ebx + mov dword ptr [rsp+0x200], eax + vmovups xmm8, xmmword ptr [r8+rdx-0x40] + vinsertf128 ymm8, ymm8, xmmword ptr [r12+rdx-0x40], 0x01 + vmovups xmm9, xmmword ptr [r9+rdx-0x40] + vinsertf128 ymm9, ymm9, xmmword ptr [r13+rdx-0x40], 0x01 + vunpcklpd ymm12, ymm8, ymm9 + vunpckhpd ymm13, ymm8, ymm9 + vmovups xmm10, xmmword ptr [r10+rdx-0x40] + vinsertf128 ymm10, ymm10, xmmword ptr [r14+rdx-0x40], 0x01 + vmovups xmm11, xmmword ptr [r11+rdx-0x40] + vinsertf128 ymm11, ymm11, xmmword ptr [r15+rdx-0x40], 0x01 + vunpcklpd ymm14, ymm10, ymm11 + vunpckhpd ymm15, ymm10, ymm11 + vshufps ymm8, ymm12, ymm14, 136 + vmovaps ymmword ptr [rsp], ymm8 + vshufps ymm9, ymm12, ymm14, 221 + vmovaps ymmword ptr [rsp+0x20], ymm9 + vshufps ymm10, ymm13, ymm15, 136 + vmovaps ymmword ptr [rsp+0x40], ymm10 + vshufps ymm11, ymm13, ymm15, 221 + vmovaps ymmword ptr [rsp+0x60], ymm11 + vmovups xmm8, xmmword ptr [r8+rdx-0x30] + vinsertf128 ymm8, ymm8, xmmword ptr [r12+rdx-0x30], 0x01 + vmovups xmm9, xmmword ptr [r9+rdx-0x30] + vinsertf128 ymm9, ymm9, xmmword ptr [r13+rdx-0x30], 0x01 + vunpcklpd ymm12, ymm8, ymm9 + vunpckhpd ymm13, ymm8, ymm9 + vmovups xmm10, xmmword ptr [r10+rdx-0x30] + vinsertf128 ymm10, ymm10, xmmword ptr [r14+rdx-0x30], 0x01 + vmovups xmm11, xmmword ptr [r11+rdx-0x30] + vinsertf128 ymm11, ymm11, xmmword ptr [r15+rdx-0x30], 0x01 + vunpcklpd ymm14, ymm10, ymm11 + vunpckhpd ymm15, ymm10, ymm11 + vshufps ymm8, ymm12, ymm14, 136 + vmovaps ymmword ptr [rsp+0x80], ymm8 + vshufps ymm9, ymm12, ymm14, 221 + vmovaps ymmword ptr [rsp+0xA0], ymm9 + vshufps ymm10, ymm13, ymm15, 136 + vmovaps ymmword ptr [rsp+0xC0], ymm10 + vshufps ymm11, ymm13, ymm15, 221 + vmovaps ymmword ptr [rsp+0xE0], ymm11 + vmovups xmm8, xmmword ptr [r8+rdx-0x20] + vinsertf128 ymm8, ymm8, xmmword ptr [r12+rdx-0x20], 0x01 + vmovups xmm9, xmmword ptr [r9+rdx-0x20] + vinsertf128 ymm9, ymm9, xmmword ptr [r13+rdx-0x20], 0x01 + vunpcklpd ymm12, ymm8, ymm9 + vunpckhpd ymm13, ymm8, ymm9 + vmovups xmm10, xmmword ptr [r10+rdx-0x20] + vinsertf128 ymm10, ymm10, xmmword ptr [r14+rdx-0x20], 0x01 + vmovups xmm11, xmmword ptr [r11+rdx-0x20] + vinsertf128 ymm11, ymm11, xmmword ptr [r15+rdx-0x20], 0x01 + vunpcklpd ymm14, ymm10, ymm11 + vunpckhpd ymm15, ymm10, ymm11 + vshufps ymm8, ymm12, ymm14, 136 + vmovaps ymmword ptr [rsp+0x100], ymm8 + vshufps ymm9, ymm12, ymm14, 221 + vmovaps ymmword ptr [rsp+0x120], ymm9 + vshufps ymm10, ymm13, ymm15, 136 + vmovaps ymmword ptr [rsp+0x140], ymm10 + vshufps ymm11, ymm13, ymm15, 221 + vmovaps ymmword ptr [rsp+0x160], ymm11 + vmovups xmm8, xmmword ptr [r8+rdx-0x10] + vinsertf128 ymm8, ymm8, xmmword ptr [r12+rdx-0x10], 0x01 + vmovups xmm9, xmmword ptr [r9+rdx-0x10] + vinsertf128 ymm9, ymm9, xmmword ptr [r13+rdx-0x10], 0x01 + vunpcklpd ymm12, ymm8, ymm9 + vunpckhpd ymm13, ymm8, ymm9 + vmovups xmm10, xmmword ptr [r10+rdx-0x10] + vinsertf128 ymm10, ymm10, xmmword ptr [r14+rdx-0x10], 0x01 + vmovups xmm11, xmmword ptr [r11+rdx-0x10] + vinsertf128 ymm11, ymm11, xmmword ptr [r15+rdx-0x10], 0x01 + vunpcklpd ymm14, ymm10, ymm11 + vunpckhpd ymm15, ymm10, ymm11 + vshufps ymm8, ymm12, ymm14, 136 + vmovaps ymmword ptr [rsp+0x180], ymm8 + vshufps ymm9, ymm12, ymm14, 221 + vmovaps ymmword ptr [rsp+0x1A0], ymm9 + vshufps ymm10, ymm13, ymm15, 136 + vmovaps ymmword ptr [rsp+0x1C0], ymm10 + vshufps ymm11, ymm13, ymm15, 221 + vmovaps ymmword ptr [rsp+0x1E0], ymm11 + vpbroadcastd ymm15, dword ptr [rsp+0x200] + prefetcht0 [r8+rdx+0x80] + prefetcht0 [r12+rdx+0x80] + prefetcht0 [r9+rdx+0x80] + prefetcht0 [r13+rdx+0x80] + prefetcht0 [r10+rdx+0x80] + prefetcht0 [r14+rdx+0x80] + prefetcht0 [r11+rdx+0x80] + prefetcht0 [r15+rdx+0x80] + vpaddd ymm0, ymm0, ymmword ptr [rsp] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x40] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x80] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0xC0] + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxor ymm12, ymm0, ymmword ptr [rsp+0x240] + vpxor ymm13, ymm1, ymmword ptr [rsp+0x260] + vpxor ymm14, ymm2, ymmword ptr [BLAKE3_BLOCK_LEN+rip] + vpxor ymm15, ymm3, ymm15 + vbroadcasti128 ymm8, xmmword ptr [ROT16+rip] + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpshufb ymm15, ymm15, ymm8 + vpaddd ymm8, ymm12, ymmword ptr [BLAKE3_IV_0+rip] + vpaddd ymm9, ymm13, ymmword ptr [BLAKE3_IV_1+rip] + vpaddd ymm10, ymm14, ymmword ptr [BLAKE3_IV_2+rip] + vpaddd ymm11, ymm15, ymmword ptr [BLAKE3_IV_3+rip] + vpxor ymm4, ymm4, ymm8 + vpxor ymm5, ymm5, ymm9 + vpxor ymm6, ymm6, ymm10 + vpxor ymm7, ymm7, ymm11 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm4, 12 + vpslld ymm4, ymm4, 20 + vpor ymm4, ymm4, ymm8 + vpsrld ymm8, ymm5, 12 + vpslld ymm5, ymm5, 20 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 12 + vpslld ymm6, ymm6, 20 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 12 + vpslld ymm7, ymm7, 20 + vpor ymm7, ymm7, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x20] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x60] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0xA0] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0xE0] + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxor ymm12, ymm12, ymm0 + vpxor ymm13, ymm13, ymm1 + vpxor ymm14, ymm14, ymm2 + vpxor ymm15, ymm15, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT8+rip] + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpshufb ymm15, ymm15, ymm8 + vpaddd ymm8, ymm12, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxor ymm4, ymm4, ymm8 + vpxor ymm5, ymm5, ymm9 + vpxor ymm6, ymm6, ymm10 + vpxor ymm7, ymm7, ymm11 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm4, 7 + vpslld ymm4, ymm4, 25 + vpor ymm4, ymm4, ymm8 + vpsrld ymm8, ymm5, 7 + vpslld ymm5, ymm5, 25 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 7 + vpslld ymm6, ymm6, 25 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 7 + vpslld ymm7, ymm7, 25 + vpor ymm7, ymm7, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x100] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x140] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x180] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x1C0] + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxor ymm15, ymm15, ymm0 + vpxor ymm12, ymm12, ymm1 + vpxor ymm13, ymm13, ymm2 + vpxor ymm14, ymm14, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT16+rip] + vpshufb ymm15, ymm15, ymm8 + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm13, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm14 + vpxor ymm5, ymm5, ymm10 + vpxor ymm6, ymm6, ymm11 + vpxor ymm7, ymm7, ymm8 + vpxor ymm4, ymm4, ymm9 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm5, 12 + vpslld ymm5, ymm5, 20 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 12 + vpslld ymm6, ymm6, 20 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 12 + vpslld ymm7, ymm7, 20 + vpor ymm7, ymm7, ymm8 + vpsrld ymm8, ymm4, 12 + vpslld ymm4, ymm4, 20 + vpor ymm4, ymm4, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x120] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x160] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x1A0] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x1E0] + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxor ymm15, ymm15, ymm0 + vpxor ymm12, ymm12, ymm1 + vpxor ymm13, ymm13, ymm2 + vpxor ymm14, ymm14, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT8+rip] + vpshufb ymm15, ymm15, ymm8 + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm13, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm14 + vpxor ymm5, ymm5, ymm10 + vpxor ymm6, ymm6, ymm11 + vpxor ymm7, ymm7, ymm8 + vpxor ymm4, ymm4, ymm9 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm5, 7 + vpslld ymm5, ymm5, 25 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 7 + vpslld ymm6, ymm6, 25 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 7 + vpslld ymm7, ymm7, 25 + vpor ymm7, ymm7, ymm8 + vpsrld ymm8, ymm4, 7 + vpslld ymm4, ymm4, 25 + vpor ymm4, ymm4, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x40] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x60] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0xE0] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x80] + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxor ymm12, ymm12, ymm0 + vpxor ymm13, ymm13, ymm1 + vpxor ymm14, ymm14, ymm2 + vpxor ymm15, ymm15, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT16+rip] + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpshufb ymm15, ymm15, ymm8 + vpaddd ymm8, ymm12, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxor ymm4, ymm4, ymm8 + vpxor ymm5, ymm5, ymm9 + vpxor ymm6, ymm6, ymm10 + vpxor ymm7, ymm7, ymm11 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm4, 12 + vpslld ymm4, ymm4, 20 + vpor ymm4, ymm4, ymm8 + vpsrld ymm8, ymm5, 12 + vpslld ymm5, ymm5, 20 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 12 + vpslld ymm6, ymm6, 20 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 12 + vpslld ymm7, ymm7, 20 + vpor ymm7, ymm7, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0xC0] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x140] + vpaddd ymm2, ymm2, ymmword ptr [rsp] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x1A0] + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxor ymm12, ymm12, ymm0 + vpxor ymm13, ymm13, ymm1 + vpxor ymm14, ymm14, ymm2 + vpxor ymm15, ymm15, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT8+rip] + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpshufb ymm15, ymm15, ymm8 + vpaddd ymm8, ymm12, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxor ymm4, ymm4, ymm8 + vpxor ymm5, ymm5, ymm9 + vpxor ymm6, ymm6, ymm10 + vpxor ymm7, ymm7, ymm11 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm4, 7 + vpslld ymm4, ymm4, 25 + vpor ymm4, ymm4, ymm8 + vpsrld ymm8, ymm5, 7 + vpslld ymm5, ymm5, 25 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 7 + vpslld ymm6, ymm6, 25 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 7 + vpslld ymm7, ymm7, 25 + vpor ymm7, ymm7, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x20] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x180] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x120] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x1E0] + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxor ymm15, ymm15, ymm0 + vpxor ymm12, ymm12, ymm1 + vpxor ymm13, ymm13, ymm2 + vpxor ymm14, ymm14, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT16+rip] + vpshufb ymm15, ymm15, ymm8 + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm13, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm14 + vpxor ymm5, ymm5, ymm10 + vpxor ymm6, ymm6, ymm11 + vpxor ymm7, ymm7, ymm8 + vpxor ymm4, ymm4, ymm9 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm5, 12 + vpslld ymm5, ymm5, 20 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 12 + vpslld ymm6, ymm6, 20 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 12 + vpslld ymm7, ymm7, 20 + vpor ymm7, ymm7, ymm8 + vpsrld ymm8, ymm4, 12 + vpslld ymm4, ymm4, 20 + vpor ymm4, ymm4, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x160] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0xA0] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x1C0] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x100] + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxor ymm15, ymm15, ymm0 + vpxor ymm12, ymm12, ymm1 + vpxor ymm13, ymm13, ymm2 + vpxor ymm14, ymm14, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT8+rip] + vpshufb ymm15, ymm15, ymm8 + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm13, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm14 + vpxor ymm5, ymm5, ymm10 + vpxor ymm6, ymm6, ymm11 + vpxor ymm7, ymm7, ymm8 + vpxor ymm4, ymm4, ymm9 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm5, 7 + vpslld ymm5, ymm5, 25 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 7 + vpslld ymm6, ymm6, 25 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 7 + vpslld ymm7, ymm7, 25 + vpor ymm7, ymm7, ymm8 + vpsrld ymm8, ymm4, 7 + vpslld ymm4, ymm4, 25 + vpor ymm4, ymm4, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x60] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x140] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x1A0] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0xE0] + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxor ymm12, ymm12, ymm0 + vpxor ymm13, ymm13, ymm1 + vpxor ymm14, ymm14, ymm2 + vpxor ymm15, ymm15, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT16+rip] + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpshufb ymm15, ymm15, ymm8 + vpaddd ymm8, ymm12, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxor ymm4, ymm4, ymm8 + vpxor ymm5, ymm5, ymm9 + vpxor ymm6, ymm6, ymm10 + vpxor ymm7, ymm7, ymm11 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm4, 12 + vpslld ymm4, ymm4, 20 + vpor ymm4, ymm4, ymm8 + vpsrld ymm8, ymm5, 12 + vpslld ymm5, ymm5, 20 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 12 + vpslld ymm6, ymm6, 20 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 12 + vpslld ymm7, ymm7, 20 + vpor ymm7, ymm7, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x80] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x180] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x40] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x1C0] + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxor ymm12, ymm12, ymm0 + vpxor ymm13, ymm13, ymm1 + vpxor ymm14, ymm14, ymm2 + vpxor ymm15, ymm15, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT8+rip] + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpshufb ymm15, ymm15, ymm8 + vpaddd ymm8, ymm12, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxor ymm4, ymm4, ymm8 + vpxor ymm5, ymm5, ymm9 + vpxor ymm6, ymm6, ymm10 + vpxor ymm7, ymm7, ymm11 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm4, 7 + vpslld ymm4, ymm4, 25 + vpor ymm4, ymm4, ymm8 + vpsrld ymm8, ymm5, 7 + vpslld ymm5, ymm5, 25 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 7 + vpslld ymm6, ymm6, 25 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 7 + vpslld ymm7, ymm7, 25 + vpor ymm7, ymm7, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0xC0] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x120] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x160] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x100] + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxor ymm15, ymm15, ymm0 + vpxor ymm12, ymm12, ymm1 + vpxor ymm13, ymm13, ymm2 + vpxor ymm14, ymm14, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT16+rip] + vpshufb ymm15, ymm15, ymm8 + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm13, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm14 + vpxor ymm5, ymm5, ymm10 + vpxor ymm6, ymm6, ymm11 + vpxor ymm7, ymm7, ymm8 + vpxor ymm4, ymm4, ymm9 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm5, 12 + vpslld ymm5, ymm5, 20 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 12 + vpslld ymm6, ymm6, 20 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 12 + vpslld ymm7, ymm7, 20 + vpor ymm7, ymm7, ymm8 + vpsrld ymm8, ymm4, 12 + vpslld ymm4, ymm4, 20 + vpor ymm4, ymm4, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0xA0] + vpaddd ymm1, ymm1, ymmword ptr [rsp] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x1E0] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x20] + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxor ymm15, ymm15, ymm0 + vpxor ymm12, ymm12, ymm1 + vpxor ymm13, ymm13, ymm2 + vpxor ymm14, ymm14, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT8+rip] + vpshufb ymm15, ymm15, ymm8 + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm13, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm14 + vpxor ymm5, ymm5, ymm10 + vpxor ymm6, ymm6, ymm11 + vpxor ymm7, ymm7, ymm8 + vpxor ymm4, ymm4, ymm9 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm5, 7 + vpslld ymm5, ymm5, 25 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 7 + vpslld ymm6, ymm6, 25 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 7 + vpslld ymm7, ymm7, 25 + vpor ymm7, ymm7, ymm8 + vpsrld ymm8, ymm4, 7 + vpslld ymm4, ymm4, 25 + vpor ymm4, ymm4, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x140] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x180] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x1C0] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x1A0] + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxor ymm12, ymm12, ymm0 + vpxor ymm13, ymm13, ymm1 + vpxor ymm14, ymm14, ymm2 + vpxor ymm15, ymm15, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT16+rip] + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpshufb ymm15, ymm15, ymm8 + vpaddd ymm8, ymm12, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxor ymm4, ymm4, ymm8 + vpxor ymm5, ymm5, ymm9 + vpxor ymm6, ymm6, ymm10 + vpxor ymm7, ymm7, ymm11 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm4, 12 + vpslld ymm4, ymm4, 20 + vpor ymm4, ymm4, ymm8 + vpsrld ymm8, ymm5, 12 + vpslld ymm5, ymm5, 20 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 12 + vpslld ymm6, ymm6, 20 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 12 + vpslld ymm7, ymm7, 20 + vpor ymm7, ymm7, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0xE0] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x120] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x60] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x1E0] + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxor ymm12, ymm12, ymm0 + vpxor ymm13, ymm13, ymm1 + vpxor ymm14, ymm14, ymm2 + vpxor ymm15, ymm15, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT8+rip] + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpshufb ymm15, ymm15, ymm8 + vpaddd ymm8, ymm12, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxor ymm4, ymm4, ymm8 + vpxor ymm5, ymm5, ymm9 + vpxor ymm6, ymm6, ymm10 + vpxor ymm7, ymm7, ymm11 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm4, 7 + vpslld ymm4, ymm4, 25 + vpor ymm4, ymm4, ymm8 + vpsrld ymm8, ymm5, 7 + vpslld ymm5, ymm5, 25 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 7 + vpslld ymm6, ymm6, 25 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 7 + vpslld ymm7, ymm7, 25 + vpor ymm7, ymm7, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x80] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x160] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0xA0] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x20] + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxor ymm15, ymm15, ymm0 + vpxor ymm12, ymm12, ymm1 + vpxor ymm13, ymm13, ymm2 + vpxor ymm14, ymm14, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT16+rip] + vpshufb ymm15, ymm15, ymm8 + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm13, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm14 + vpxor ymm5, ymm5, ymm10 + vpxor ymm6, ymm6, ymm11 + vpxor ymm7, ymm7, ymm8 + vpxor ymm4, ymm4, ymm9 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm5, 12 + vpslld ymm5, ymm5, 20 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 12 + vpslld ymm6, ymm6, 20 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 12 + vpslld ymm7, ymm7, 20 + vpor ymm7, ymm7, ymm8 + vpsrld ymm8, ymm4, 12 + vpslld ymm4, ymm4, 20 + vpor ymm4, ymm4, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x40] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x100] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0xC0] + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxor ymm15, ymm15, ymm0 + vpxor ymm12, ymm12, ymm1 + vpxor ymm13, ymm13, ymm2 + vpxor ymm14, ymm14, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT8+rip] + vpshufb ymm15, ymm15, ymm8 + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm13, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm14 + vpxor ymm5, ymm5, ymm10 + vpxor ymm6, ymm6, ymm11 + vpxor ymm7, ymm7, ymm8 + vpxor ymm4, ymm4, ymm9 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm5, 7 + vpslld ymm5, ymm5, 25 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 7 + vpslld ymm6, ymm6, 25 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 7 + vpslld ymm7, ymm7, 25 + vpor ymm7, ymm7, ymm8 + vpsrld ymm8, ymm4, 7 + vpslld ymm4, ymm4, 25 + vpor ymm4, ymm4, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x180] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x120] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x1E0] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x1C0] + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxor ymm12, ymm12, ymm0 + vpxor ymm13, ymm13, ymm1 + vpxor ymm14, ymm14, ymm2 + vpxor ymm15, ymm15, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT16+rip] + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpshufb ymm15, ymm15, ymm8 + vpaddd ymm8, ymm12, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxor ymm4, ymm4, ymm8 + vpxor ymm5, ymm5, ymm9 + vpxor ymm6, ymm6, ymm10 + vpxor ymm7, ymm7, ymm11 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm4, 12 + vpslld ymm4, ymm4, 20 + vpor ymm4, ymm4, ymm8 + vpsrld ymm8, ymm5, 12 + vpslld ymm5, ymm5, 20 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 12 + vpslld ymm6, ymm6, 20 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 12 + vpslld ymm7, ymm7, 20 + vpor ymm7, ymm7, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x1A0] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x160] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x140] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x100] + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxor ymm12, ymm12, ymm0 + vpxor ymm13, ymm13, ymm1 + vpxor ymm14, ymm14, ymm2 + vpxor ymm15, ymm15, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT8+rip] + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpshufb ymm15, ymm15, ymm8 + vpaddd ymm8, ymm12, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxor ymm4, ymm4, ymm8 + vpxor ymm5, ymm5, ymm9 + vpxor ymm6, ymm6, ymm10 + vpxor ymm7, ymm7, ymm11 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm4, 7 + vpslld ymm4, ymm4, 25 + vpor ymm4, ymm4, ymm8 + vpsrld ymm8, ymm5, 7 + vpslld ymm5, ymm5, 25 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 7 + vpslld ymm6, ymm6, 25 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 7 + vpslld ymm7, ymm7, 25 + vpor ymm7, ymm7, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0xE0] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0xA0] + vpaddd ymm2, ymm2, ymmword ptr [rsp] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0xC0] + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxor ymm15, ymm15, ymm0 + vpxor ymm12, ymm12, ymm1 + vpxor ymm13, ymm13, ymm2 + vpxor ymm14, ymm14, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT16+rip] + vpshufb ymm15, ymm15, ymm8 + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm13, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm14 + vpxor ymm5, ymm5, ymm10 + vpxor ymm6, ymm6, ymm11 + vpxor ymm7, ymm7, ymm8 + vpxor ymm4, ymm4, ymm9 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm5, 12 + vpslld ymm5, ymm5, 20 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 12 + vpslld ymm6, ymm6, 20 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 12 + vpslld ymm7, ymm7, 20 + vpor ymm7, ymm7, ymm8 + vpsrld ymm8, ymm4, 12 + vpslld ymm4, ymm4, 20 + vpor ymm4, ymm4, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x40] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x60] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x20] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x80] + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxor ymm15, ymm15, ymm0 + vpxor ymm12, ymm12, ymm1 + vpxor ymm13, ymm13, ymm2 + vpxor ymm14, ymm14, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT8+rip] + vpshufb ymm15, ymm15, ymm8 + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm13, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm14 + vpxor ymm5, ymm5, ymm10 + vpxor ymm6, ymm6, ymm11 + vpxor ymm7, ymm7, ymm8 + vpxor ymm4, ymm4, ymm9 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm5, 7 + vpslld ymm5, ymm5, 25 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 7 + vpslld ymm6, ymm6, 25 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 7 + vpslld ymm7, ymm7, 25 + vpor ymm7, ymm7, ymm8 + vpsrld ymm8, ymm4, 7 + vpslld ymm4, ymm4, 25 + vpor ymm4, ymm4, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x120] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x160] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x100] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x1E0] + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxor ymm12, ymm12, ymm0 + vpxor ymm13, ymm13, ymm1 + vpxor ymm14, ymm14, ymm2 + vpxor ymm15, ymm15, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT16+rip] + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpshufb ymm15, ymm15, ymm8 + vpaddd ymm8, ymm12, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxor ymm4, ymm4, ymm8 + vpxor ymm5, ymm5, ymm9 + vpxor ymm6, ymm6, ymm10 + vpxor ymm7, ymm7, ymm11 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm4, 12 + vpslld ymm4, ymm4, 20 + vpor ymm4, ymm4, ymm8 + vpsrld ymm8, ymm5, 12 + vpslld ymm5, ymm5, 20 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 12 + vpslld ymm6, ymm6, 20 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 12 + vpslld ymm7, ymm7, 20 + vpor ymm7, ymm7, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x1C0] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0xA0] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x180] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x20] + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxor ymm12, ymm12, ymm0 + vpxor ymm13, ymm13, ymm1 + vpxor ymm14, ymm14, ymm2 + vpxor ymm15, ymm15, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT8+rip] + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpshufb ymm15, ymm15, ymm8 + vpaddd ymm8, ymm12, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxor ymm4, ymm4, ymm8 + vpxor ymm5, ymm5, ymm9 + vpxor ymm6, ymm6, ymm10 + vpxor ymm7, ymm7, ymm11 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm4, 7 + vpslld ymm4, ymm4, 25 + vpor ymm4, ymm4, ymm8 + vpsrld ymm8, ymm5, 7 + vpslld ymm5, ymm5, 25 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 7 + vpslld ymm6, ymm6, 25 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 7 + vpslld ymm7, ymm7, 25 + vpor ymm7, ymm7, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x1A0] + vpaddd ymm1, ymm1, ymmword ptr [rsp] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x40] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x80] + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxor ymm15, ymm15, ymm0 + vpxor ymm12, ymm12, ymm1 + vpxor ymm13, ymm13, ymm2 + vpxor ymm14, ymm14, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT16+rip] + vpshufb ymm15, ymm15, ymm8 + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm13, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm14 + vpxor ymm5, ymm5, ymm10 + vpxor ymm6, ymm6, ymm11 + vpxor ymm7, ymm7, ymm8 + vpxor ymm4, ymm4, ymm9 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm5, 12 + vpslld ymm5, ymm5, 20 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 12 + vpslld ymm6, ymm6, 20 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 12 + vpslld ymm7, ymm7, 20 + vpor ymm7, ymm7, ymm8 + vpsrld ymm8, ymm4, 12 + vpslld ymm4, ymm4, 20 + vpor ymm4, ymm4, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x60] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x140] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0xC0] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0xE0] + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxor ymm15, ymm15, ymm0 + vpxor ymm12, ymm12, ymm1 + vpxor ymm13, ymm13, ymm2 + vpxor ymm14, ymm14, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT8+rip] + vpshufb ymm15, ymm15, ymm8 + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm13, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm14 + vpxor ymm5, ymm5, ymm10 + vpxor ymm6, ymm6, ymm11 + vpxor ymm7, ymm7, ymm8 + vpxor ymm4, ymm4, ymm9 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm5, 7 + vpslld ymm5, ymm5, 25 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 7 + vpslld ymm6, ymm6, 25 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 7 + vpslld ymm7, ymm7, 25 + vpor ymm7, ymm7, ymm8 + vpsrld ymm8, ymm4, 7 + vpslld ymm4, ymm4, 25 + vpor ymm4, ymm4, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x160] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0xA0] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x20] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x100] + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxor ymm12, ymm12, ymm0 + vpxor ymm13, ymm13, ymm1 + vpxor ymm14, ymm14, ymm2 + vpxor ymm15, ymm15, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT16+rip] + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpshufb ymm15, ymm15, ymm8 + vpaddd ymm8, ymm12, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxor ymm4, ymm4, ymm8 + vpxor ymm5, ymm5, ymm9 + vpxor ymm6, ymm6, ymm10 + vpxor ymm7, ymm7, ymm11 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm4, 12 + vpslld ymm4, ymm4, 20 + vpor ymm4, ymm4, ymm8 + vpsrld ymm8, ymm5, 12 + vpslld ymm5, ymm5, 20 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 12 + vpslld ymm6, ymm6, 20 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 12 + vpslld ymm7, ymm7, 20 + vpor ymm7, ymm7, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x1E0] + vpaddd ymm1, ymm1, ymmword ptr [rsp] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x120] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0xC0] + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxor ymm12, ymm12, ymm0 + vpxor ymm13, ymm13, ymm1 + vpxor ymm14, ymm14, ymm2 + vpxor ymm15, ymm15, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT8+rip] + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpshufb ymm15, ymm15, ymm8 + vpaddd ymm8, ymm12, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxor ymm4, ymm4, ymm8 + vpxor ymm5, ymm5, ymm9 + vpxor ymm6, ymm6, ymm10 + vpxor ymm7, ymm7, ymm11 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm4, 7 + vpslld ymm4, ymm4, 25 + vpor ymm4, ymm4, ymm8 + vpsrld ymm8, ymm5, 7 + vpslld ymm5, ymm5, 25 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 7 + vpslld ymm6, ymm6, 25 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 7 + vpslld ymm7, ymm7, 25 + vpor ymm7, ymm7, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x1C0] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x40] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x60] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0xE0] + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxor ymm15, ymm15, ymm0 + vpxor ymm12, ymm12, ymm1 + vpxor ymm13, ymm13, ymm2 + vpxor ymm14, ymm14, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT16+rip] + vpshufb ymm15, ymm15, ymm8 + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm13, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm14 + vpxor ymm5, ymm5, ymm10 + vpxor ymm6, ymm6, ymm11 + vpxor ymm7, ymm7, ymm8 + vpxor ymm4, ymm4, ymm9 + vmovdqa ymmword ptr [rsp+0x200], ymm8 + vpsrld ymm8, ymm5, 12 + vpslld ymm5, ymm5, 20 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 12 + vpslld ymm6, ymm6, 20 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 12 + vpslld ymm7, ymm7, 20 + vpor ymm7, ymm7, ymm8 + vpsrld ymm8, ymm4, 12 + vpslld ymm4, ymm4, 20 + vpor ymm4, ymm4, ymm8 + vpaddd ymm0, ymm0, ymmword ptr [rsp+0x140] + vpaddd ymm1, ymm1, ymmword ptr [rsp+0x180] + vpaddd ymm2, ymm2, ymmword ptr [rsp+0x80] + vpaddd ymm3, ymm3, ymmword ptr [rsp+0x1A0] + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxor ymm15, ymm15, ymm0 + vpxor ymm12, ymm12, ymm1 + vpxor ymm13, ymm13, ymm2 + vpxor ymm14, ymm14, ymm3 + vbroadcasti128 ymm8, xmmword ptr [ROT8+rip] + vpshufb ymm15, ymm15, ymm8 + vpshufb ymm12, ymm12, ymm8 + vpshufb ymm13, ymm13, ymm8 + vpshufb ymm14, ymm14, ymm8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm13, ymmword ptr [rsp+0x200] + vpaddd ymm9, ymm9, ymm14 + vpxor ymm5, ymm5, ymm10 + vpxor ymm6, ymm6, ymm11 + vpxor ymm7, ymm7, ymm8 + vpxor ymm4, ymm4, ymm9 + vpxor ymm0, ymm0, ymm8 + vpxor ymm1, ymm1, ymm9 + vpxor ymm2, ymm2, ymm10 + vpxor ymm3, ymm3, ymm11 + vpsrld ymm8, ymm5, 7 + vpslld ymm5, ymm5, 25 + vpor ymm5, ymm5, ymm8 + vpsrld ymm8, ymm6, 7 + vpslld ymm6, ymm6, 25 + vpor ymm6, ymm6, ymm8 + vpsrld ymm8, ymm7, 7 + vpslld ymm7, ymm7, 25 + vpor ymm7, ymm7, ymm8 + vpsrld ymm8, ymm4, 7 + vpslld ymm4, ymm4, 25 + vpor ymm4, ymm4, ymm8 + vpxor ymm4, ymm4, ymm12 + vpxor ymm5, ymm5, ymm13 + vpxor ymm6, ymm6, ymm14 + vpxor ymm7, ymm7, ymm15 + movzx eax, byte ptr [rbp+0x38] + jne 9b + mov rbx, qword ptr [rbp+0x50] + vunpcklps ymm8, ymm0, ymm1 + vunpcklps ymm9, ymm2, ymm3 + vunpckhps ymm10, ymm0, ymm1 + vunpcklps ymm11, ymm4, ymm5 + vunpcklps ymm0, ymm6, ymm7 + vshufps ymm12, ymm8, ymm9, 78 + vblendps ymm1, ymm8, ymm12, 0xCC + vshufps ymm8, ymm11, ymm0, 78 + vunpckhps ymm13, ymm2, ymm3 + vblendps ymm2, ymm11, ymm8, 0xCC + vblendps ymm3, ymm12, ymm9, 0xCC + vperm2f128 ymm12, ymm1, ymm2, 0x20 + vmovups ymmword ptr [rbx], ymm12 + vunpckhps ymm14, ymm4, ymm5 + vblendps ymm4, ymm8, ymm0, 0xCC + vunpckhps ymm15, ymm6, ymm7 + vperm2f128 ymm7, ymm3, ymm4, 0x20 + vmovups ymmword ptr [rbx+0x20], ymm7 + vshufps ymm5, ymm10, ymm13, 78 + vblendps ymm6, ymm5, ymm13, 0xCC + vshufps ymm13, ymm14, ymm15, 78 + vblendps ymm10, ymm10, ymm5, 0xCC + vblendps ymm14, ymm14, ymm13, 0xCC + vperm2f128 ymm8, ymm10, ymm14, 0x20 + vmovups ymmword ptr [rbx+0x40], ymm8 + vblendps ymm15, ymm13, ymm15, 0xCC + vperm2f128 ymm13, ymm6, ymm15, 0x20 + vmovups ymmword ptr [rbx+0x60], ymm13 + vperm2f128 ymm9, ymm1, ymm2, 0x31 + vperm2f128 ymm11, ymm3, ymm4, 0x31 + vmovups ymmword ptr [rbx+0x80], ymm9 + vperm2f128 ymm14, ymm10, ymm14, 0x31 + vperm2f128 ymm15, ymm6, ymm15, 0x31 + vmovups ymmword ptr [rbx+0xA0], ymm11 + vmovups ymmword ptr [rbx+0xC0], ymm14 + vmovups ymmword ptr [rbx+0xE0], ymm15 + vmovdqa ymm0, ymmword ptr [rsp+0x220] + vpaddd ymm1, ymm0, ymmword ptr [rsp+0x240] + vmovdqa ymmword ptr [rsp+0x240], ymm1 + vpxor ymm0, ymm0, ymmword ptr [CMP_MSB_MASK+rip] + vpxor ymm2, ymm1, ymmword ptr [CMP_MSB_MASK+rip] + vpcmpgtd ymm2, ymm0, ymm2 + vmovdqa ymm0, ymmword ptr [rsp+0x260] + vpsubd ymm2, ymm0, ymm2 + vmovdqa ymmword ptr [rsp+0x260], ymm2 + add rdi, 64 + add rbx, 256 + mov qword ptr [rbp+0x50], rbx + sub rsi, 8 + cmp rsi, 8 + jnc 2b + test rsi, rsi + jnz 3f +4: + vzeroupper + mov rsp, rbp + pop rbp + pop rbx + pop r12 + pop r13 + pop r14 + pop r15 + RET +.p2align 5 +3: + mov rbx, qword ptr [rbp+0x50] + mov r15, qword ptr [rsp+0x2A0] + movzx r13d, byte ptr [rbp+0x38] + movzx r12d, byte ptr [rbp+0x48] + test rsi, 0x4 + je 3f + vbroadcasti128 ymm0, xmmword ptr [rcx] + vbroadcasti128 ymm1, xmmword ptr [rcx+0x10] + vmovdqa ymm8, ymm0 + vmovdqa ymm9, ymm1 + vbroadcasti128 ymm12, xmmword ptr [rsp+0x240] + vbroadcasti128 ymm13, xmmword ptr [rsp+0x260] + vpunpckldq ymm14, ymm12, ymm13 + vpunpckhdq ymm15, ymm12, ymm13 + vpermq ymm14, ymm14, 0x50 + vpermq ymm15, ymm15, 0x50 + vbroadcasti128 ymm12, xmmword ptr [BLAKE3_BLOCK_LEN+rip] + vpblendd ymm14, ymm14, ymm12, 0x44 + vpblendd ymm15, ymm15, ymm12, 0x44 + vmovdqa ymmword ptr [rsp], ymm14 + vmovdqa ymmword ptr [rsp+0x20], ymm15 + mov r8, qword ptr [rdi] + mov r9, qword ptr [rdi+0x8] + mov r10, qword ptr [rdi+0x10] + mov r11, qword ptr [rdi+0x18] + movzx eax, byte ptr [rbp+0x40] + or eax, r13d + xor edx, edx +.p2align 5 +2: + mov r14d, eax + or eax, r12d + add rdx, 64 + cmp rdx, r15 + cmovne eax, r14d + mov dword ptr [rsp+0x200], eax + vmovups ymm2, ymmword ptr [r8+rdx-0x40] + vinsertf128 ymm2, ymm2, xmmword ptr [r9+rdx-0x40], 0x01 + vmovups ymm3, ymmword ptr [r8+rdx-0x30] + vinsertf128 ymm3, ymm3, xmmword ptr [r9+rdx-0x30], 0x01 + vshufps ymm4, ymm2, ymm3, 136 + vshufps ymm5, ymm2, ymm3, 221 + vmovups ymm2, ymmword ptr [r8+rdx-0x20] + vinsertf128 ymm2, ymm2, xmmword ptr [r9+rdx-0x20], 0x01 + vmovups ymm3, ymmword ptr [r8+rdx-0x10] + vinsertf128 ymm3, ymm3, xmmword ptr [r9+rdx-0x10], 0x01 + vshufps ymm6, ymm2, ymm3, 136 + vshufps ymm7, ymm2, ymm3, 221 + vpshufd ymm6, ymm6, 0x93 + vpshufd ymm7, ymm7, 0x93 + vmovups ymm10, ymmword ptr [r10+rdx-0x40] + vinsertf128 ymm10, ymm10, xmmword ptr [r11+rdx-0x40], 0x01 + vmovups ymm11, ymmword ptr [r10+rdx-0x30] + vinsertf128 ymm11, ymm11, xmmword ptr [r11+rdx-0x30], 0x01 + vshufps ymm12, ymm10, ymm11, 136 + vshufps ymm13, ymm10, ymm11, 221 + vmovups ymm10, ymmword ptr [r10+rdx-0x20] + vinsertf128 ymm10, ymm10, xmmword ptr [r11+rdx-0x20], 0x01 + vmovups ymm11, ymmword ptr [r10+rdx-0x10] + vinsertf128 ymm11, ymm11, xmmword ptr [r11+rdx-0x10], 0x01 + vshufps ymm14, ymm10, ymm11, 136 + vshufps ymm15, ymm10, ymm11, 221 + vpshufd ymm14, ymm14, 0x93 + vpshufd ymm15, ymm15, 0x93 + prefetcht0 [r8+rdx+0x80] + prefetcht0 [r9+rdx+0x80] + prefetcht0 [r10+rdx+0x80] + prefetcht0 [r11+rdx+0x80] + vpbroadcastd ymm2, dword ptr [rsp+0x200] + vmovdqa ymm3, ymmword ptr [rsp] + vmovdqa ymm11, ymmword ptr [rsp+0x20] + vpblendd ymm3, ymm3, ymm2, 0x88 + vpblendd ymm11, ymm11, ymm2, 0x88 + vbroadcasti128 ymm2, xmmword ptr [BLAKE3_IV+rip] + vmovdqa ymm10, ymm2 + mov al, 7 +9: + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm8, ymm8, ymm12 + vmovdqa ymmword ptr [rsp+0x40], ymm4 + nop + vmovdqa ymmword ptr [rsp+0x60], ymm12 + nop + vpaddd ymm0, ymm0, ymm1 + vpaddd ymm8, ymm8, ymm9 + vpxor ymm3, ymm3, ymm0 + vpxor ymm11, ymm11, ymm8 + vbroadcasti128 ymm4, xmmword ptr [ROT16+rip] + vpshufb ymm3, ymm3, ymm4 + vpshufb ymm11, ymm11, ymm4 + vpaddd ymm2, ymm2, ymm3 + vpaddd ymm10, ymm10, ymm11 + vpxor ymm1, ymm1, ymm2 + vpxor ymm9, ymm9, ymm10 + vpsrld ymm4, ymm1, 12 + vpslld ymm1, ymm1, 20 + vpor ymm1, ymm1, ymm4 + vpsrld ymm4, ymm9, 12 + vpslld ymm9, ymm9, 20 + vpor ymm9, ymm9, ymm4 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm8, ymm8, ymm13 + vpaddd ymm0, ymm0, ymm1 + vpaddd ymm8, ymm8, ymm9 + vmovdqa ymmword ptr [rsp+0x80], ymm5 + vmovdqa ymmword ptr [rsp+0xA0], ymm13 + vpxor ymm3, ymm3, ymm0 + vpxor ymm11, ymm11, ymm8 + vbroadcasti128 ymm4, xmmword ptr [ROT8+rip] + vpshufb ymm3, ymm3, ymm4 + vpshufb ymm11, ymm11, ymm4 + vpaddd ymm2, ymm2, ymm3 + vpaddd ymm10, ymm10, ymm11 + vpxor ymm1, ymm1, ymm2 + vpxor ymm9, ymm9, ymm10 + vpsrld ymm4, ymm1, 7 + vpslld ymm1, ymm1, 25 + vpor ymm1, ymm1, ymm4 + vpsrld ymm4, ymm9, 7 + vpslld ymm9, ymm9, 25 + vpor ymm9, ymm9, ymm4 + vpshufd ymm0, ymm0, 0x93 + vpshufd ymm8, ymm8, 0x93 + vpshufd ymm3, ymm3, 0x4E + vpshufd ymm11, ymm11, 0x4E + vpshufd ymm2, ymm2, 0x39 + vpshufd ymm10, ymm10, 0x39 + vpaddd ymm0, ymm0, ymm6 + vpaddd ymm8, ymm8, ymm14 + vpaddd ymm0, ymm0, ymm1 + vpaddd ymm8, ymm8, ymm9 + vpxor ymm3, ymm3, ymm0 + vpxor ymm11, ymm11, ymm8 + vbroadcasti128 ymm4, xmmword ptr [ROT16+rip] + vpshufb ymm3, ymm3, ymm4 + vpshufb ymm11, ymm11, ymm4 + vpaddd ymm2, ymm2, ymm3 + vpaddd ymm10, ymm10, ymm11 + vpxor ymm1, ymm1, ymm2 + vpxor ymm9, ymm9, ymm10 + vpsrld ymm4, ymm1, 12 + vpslld ymm1, ymm1, 20 + vpor ymm1, ymm1, ymm4 + vpsrld ymm4, ymm9, 12 + vpslld ymm9, ymm9, 20 + vpor ymm9, ymm9, ymm4 + vpaddd ymm0, ymm0, ymm7 + vpaddd ymm8, ymm8, ymm15 + vpaddd ymm0, ymm0, ymm1 + vpaddd ymm8, ymm8, ymm9 + vpxor ymm3, ymm3, ymm0 + vpxor ymm11, ymm11, ymm8 + vbroadcasti128 ymm4, xmmword ptr [ROT8+rip] + vpshufb ymm3, ymm3, ymm4 + vpshufb ymm11, ymm11, ymm4 + vpaddd ymm2, ymm2, ymm3 + vpaddd ymm10, ymm10, ymm11 + vpxor ymm1, ymm1, ymm2 + vpxor ymm9, ymm9, ymm10 + vpsrld ymm4, ymm1, 7 + vpslld ymm1, ymm1, 25 + vpor ymm1, ymm1, ymm4 + vpsrld ymm4, ymm9, 7 + vpslld ymm9, ymm9, 25 + vpor ymm9, ymm9, ymm4 + vpshufd ymm0, ymm0, 0x39 + vpshufd ymm8, ymm8, 0x39 + vpshufd ymm3, ymm3, 0x4E + vpshufd ymm11, ymm11, 0x4E + vpshufd ymm2, ymm2, 0x93 + vpshufd ymm10, ymm10, 0x93 + dec al + je 9f + vmovdqa ymm4, ymmword ptr [rsp+0x40] + vmovdqa ymm5, ymmword ptr [rsp+0x80] + vshufps ymm12, ymm4, ymm5, 214 + vpshufd ymm13, ymm4, 0x0F + vpshufd ymm4, ymm12, 0x39 + vshufps ymm12, ymm6, ymm7, 250 + vpblendd ymm13, ymm13, ymm12, 0xAA + vpunpcklqdq ymm12, ymm7, ymm5 + vpblendd ymm12, ymm12, ymm6, 0x88 + vpshufd ymm12, ymm12, 0x78 + vpunpckhdq ymm5, ymm5, ymm7 + vpunpckldq ymm6, ymm6, ymm5 + vpshufd ymm7, ymm6, 0x1E + vmovdqa ymmword ptr [rsp+0x40], ymm13 + vmovdqa ymmword ptr [rsp+0x80], ymm12 + vmovdqa ymm12, ymmword ptr [rsp+0x60] + vmovdqa ymm13, ymmword ptr [rsp+0xA0] + vshufps ymm5, ymm12, ymm13, 214 + vpshufd ymm6, ymm12, 0x0F + vpshufd ymm12, ymm5, 0x39 + vshufps ymm5, ymm14, ymm15, 250 + vpblendd ymm6, ymm6, ymm5, 0xAA + vpunpcklqdq ymm5, ymm15, ymm13 + vpblendd ymm5, ymm5, ymm14, 0x88 + vpshufd ymm5, ymm5, 0x78 + vpunpckhdq ymm13, ymm13, ymm15 + vpunpckldq ymm14, ymm14, ymm13 + vpshufd ymm15, ymm14, 0x1E + vmovdqa ymm13, ymm6 + vmovdqa ymm14, ymm5 + vmovdqa ymm5, ymmword ptr [rsp+0x40] + vmovdqa ymm6, ymmword ptr [rsp+0x80] + jmp 9b +9: + vpxor ymm0, ymm0, ymm2 + vpxor ymm1, ymm1, ymm3 + vpxor ymm8, ymm8, ymm10 + vpxor ymm9, ymm9, ymm11 + mov eax, r13d + cmp rdx, r15 + jne 2b + vmovdqu xmmword ptr [rbx], xmm0 + vmovdqu xmmword ptr [rbx+0x10], xmm1 + vextracti128 xmmword ptr [rbx+0x20], ymm0, 0x01 + vextracti128 xmmword ptr [rbx+0x30], ymm1, 0x01 + vmovdqu xmmword ptr [rbx+0x40], xmm8 + vmovdqu xmmword ptr [rbx+0x50], xmm9 + vextracti128 xmmword ptr [rbx+0x60], ymm8, 0x01 + vextracti128 xmmword ptr [rbx+0x70], ymm9, 0x01 + vmovaps xmm8, xmmword ptr [rsp+0x280] + vmovaps xmm0, xmmword ptr [rsp+0x240] + vmovaps xmm1, xmmword ptr [rsp+0x250] + vmovaps xmm2, xmmword ptr [rsp+0x260] + vmovaps xmm3, xmmword ptr [rsp+0x270] + vblendvps xmm0, xmm0, xmm1, xmm8 + vblendvps xmm2, xmm2, xmm3, xmm8 + vmovaps xmmword ptr [rsp+0x240], xmm0 + vmovaps xmmword ptr [rsp+0x260], xmm2 + add rbx, 128 + add rdi, 32 + sub rsi, 4 +3: + test rsi, 0x2 + je 3f + vbroadcasti128 ymm0, xmmword ptr [rcx] + vbroadcasti128 ymm1, xmmword ptr [rcx+0x10] + vmovd xmm13, dword ptr [rsp+0x240] + vpinsrd xmm13, xmm13, dword ptr [rsp+0x260], 1 + vpinsrd xmm13, xmm13, dword ptr [BLAKE3_BLOCK_LEN+rip], 2 + vmovd xmm14, dword ptr [rsp+0x244] + vpinsrd xmm14, xmm14, dword ptr [rsp+0x264], 1 + vpinsrd xmm14, xmm14, dword ptr [BLAKE3_BLOCK_LEN+rip], 2 + vinserti128 ymm13, ymm13, xmm14, 0x01 + vbroadcasti128 ymm14, xmmword ptr [ROT16+rip] + vbroadcasti128 ymm15, xmmword ptr [ROT8+rip] + mov r8, qword ptr [rdi] + mov r9, qword ptr [rdi+0x8] + movzx eax, byte ptr [rbp+0x40] + or eax, r13d + xor edx, edx +.p2align 5 +2: + mov r14d, eax + or eax, r12d + add rdx, 64 + cmp rdx, r15 + cmovne eax, r14d + mov dword ptr [rsp+0x200], eax + vbroadcasti128 ymm2, xmmword ptr [BLAKE3_IV+rip] + vpbroadcastd ymm8, dword ptr [rsp+0x200] + vpblendd ymm3, ymm13, ymm8, 0x88 + vmovups ymm8, ymmword ptr [r8+rdx-0x40] + vinsertf128 ymm8, ymm8, xmmword ptr [r9+rdx-0x40], 0x01 + vmovups ymm9, ymmword ptr [r8+rdx-0x30] + vinsertf128 ymm9, ymm9, xmmword ptr [r9+rdx-0x30], 0x01 + vshufps ymm4, ymm8, ymm9, 136 + vshufps ymm5, ymm8, ymm9, 221 + vmovups ymm8, ymmword ptr [r8+rdx-0x20] + vinsertf128 ymm8, ymm8, xmmword ptr [r9+rdx-0x20], 0x01 + vmovups ymm9, ymmword ptr [r8+rdx-0x10] + vinsertf128 ymm9, ymm9, xmmword ptr [r9+rdx-0x10], 0x01 + vshufps ymm6, ymm8, ymm9, 136 + vshufps ymm7, ymm8, ymm9, 221 + vpshufd ymm6, ymm6, 0x93 + vpshufd ymm7, ymm7, 0x93 + mov al, 7 +9: + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm0, ymm0, ymm1 + vpxor ymm3, ymm3, ymm0 + vpshufb ymm3, ymm3, ymm14 + vpaddd ymm2, ymm2, ymm3 + vpxor ymm1, ymm1, ymm2 + vpsrld ymm8, ymm1, 12 + vpslld ymm1, ymm1, 20 + vpor ymm1, ymm1, ymm8 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm0, ymm0, ymm1 + vpxor ymm3, ymm3, ymm0 + vpshufb ymm3, ymm3, ymm15 + vpaddd ymm2, ymm2, ymm3 + vpxor ymm1, ymm1, ymm2 + vpsrld ymm8, ymm1, 7 + vpslld ymm1, ymm1, 25 + vpor ymm1, ymm1, ymm8 + vpshufd ymm0, ymm0, 0x93 + vpshufd ymm3, ymm3, 0x4E + vpshufd ymm2, ymm2, 0x39 + vpaddd ymm0, ymm0, ymm6 + vpaddd ymm0, ymm0, ymm1 + vpxor ymm3, ymm3, ymm0 + vpshufb ymm3, ymm3, ymm14 + vpaddd ymm2, ymm2, ymm3 + vpxor ymm1, ymm1, ymm2 + vpsrld ymm8, ymm1, 12 + vpslld ymm1, ymm1, 20 + vpor ymm1, ymm1, ymm8 + vpaddd ymm0, ymm0, ymm7 + vpaddd ymm0, ymm0, ymm1 + vpxor ymm3, ymm3, ymm0 + vpshufb ymm3, ymm3, ymm15 + vpaddd ymm2, ymm2, ymm3 + vpxor ymm1, ymm1, ymm2 + vpsrld ymm8, ymm1, 7 + vpslld ymm1, ymm1, 25 + vpor ymm1, ymm1, ymm8 + vpshufd ymm0, ymm0, 0x39 + vpshufd ymm3, ymm3, 0x4E + vpshufd ymm2, ymm2, 0x93 + dec al + jz 9f + vshufps ymm8, ymm4, ymm5, 214 + vpshufd ymm9, ymm4, 0x0F + vpshufd ymm4, ymm8, 0x39 + vshufps ymm8, ymm6, ymm7, 250 + vpblendd ymm9, ymm9, ymm8, 0xAA + vpunpcklqdq ymm8, ymm7, ymm5 + vpblendd ymm8, ymm8, ymm6, 0x88 + vpshufd ymm8, ymm8, 0x78 + vpunpckhdq ymm5, ymm5, ymm7 + vpunpckldq ymm6, ymm6, ymm5 + vpshufd ymm7, ymm6, 0x1E + vmovdqa ymm5, ymm9 + vmovdqa ymm6, ymm8 + jmp 9b +9: + vpxor ymm0, ymm0, ymm2 + vpxor ymm1, ymm1, ymm3 + mov eax, r13d + cmp rdx, r15 + jne 2b + vmovdqu xmmword ptr [rbx], xmm0 + vmovdqu xmmword ptr [rbx+0x10], xmm1 + vextracti128 xmmword ptr [rbx+0x20], ymm0, 0x01 + vextracti128 xmmword ptr [rbx+0x30], ymm1, 0x01 + vmovaps ymm8, ymmword ptr [rsp+0x280] + vmovaps ymm0, ymmword ptr [rsp+0x240] + vmovups ymm1, ymmword ptr [rsp+0x248] + vmovaps ymm2, ymmword ptr [rsp+0x260] + vmovups ymm3, ymmword ptr [rsp+0x268] + vblendvps ymm0, ymm0, ymm1, ymm8 + vblendvps ymm2, ymm2, ymm3, ymm8 + vmovaps ymmword ptr [rsp+0x240], ymm0 + vmovaps ymmword ptr [rsp+0x260], ymm2 + add rbx, 64 + add rdi, 16 + sub rsi, 2 +3: + test rsi, 0x1 + je 4b + vmovdqu xmm0, xmmword ptr [rcx] + vmovdqu xmm1, xmmword ptr [rcx+0x10] + vmovd xmm3, dword ptr [rsp+0x240] + vpinsrd xmm3, xmm3, dword ptr [rsp+0x260], 1 + vpinsrd xmm13, xmm3, dword ptr [BLAKE3_BLOCK_LEN+rip], 2 + vmovdqa xmm14, xmmword ptr [ROT16+rip] + vmovdqa xmm15, xmmword ptr [ROT8+rip] + mov r8, qword ptr [rdi] + movzx eax, byte ptr [rbp+0x40] + or eax, r13d + xor edx, edx +.p2align 5 +2: + mov r14d, eax + or eax, r12d + add rdx, 64 + cmp rdx, r15 + cmovne eax, r14d + vmovdqa xmm2, xmmword ptr [BLAKE3_IV+rip] + vmovdqa xmm3, xmm13 + vpinsrd xmm3, xmm3, eax, 3 + vmovups xmm8, xmmword ptr [r8+rdx-0x40] + vmovups xmm9, xmmword ptr [r8+rdx-0x30] + vshufps xmm4, xmm8, xmm9, 136 + vshufps xmm5, xmm8, xmm9, 221 + vmovups xmm8, xmmword ptr [r8+rdx-0x20] + vmovups xmm9, xmmword ptr [r8+rdx-0x10] + vshufps xmm6, xmm8, xmm9, 136 + vshufps xmm7, xmm8, xmm9, 221 + vpshufd xmm6, xmm6, 0x93 + vpshufd xmm7, xmm7, 0x93 + mov al, 7 +9: + vpaddd xmm0, xmm0, xmm4 + vpaddd xmm0, xmm0, xmm1 + vpxor xmm3, xmm3, xmm0 + vpshufb xmm3, xmm3, xmm14 + vpaddd xmm2, xmm2, xmm3 + vpxor xmm1, xmm1, xmm2 + vpsrld xmm8, xmm1, 12 + vpslld xmm1, xmm1, 20 + vpor xmm1, xmm1, xmm8 + vpaddd xmm0, xmm0, xmm5 + vpaddd xmm0, xmm0, xmm1 + vpxor xmm3, xmm3, xmm0 + vpshufb xmm3, xmm3, xmm15 + vpaddd xmm2, xmm2, xmm3 + vpxor xmm1, xmm1, xmm2 + vpsrld xmm8, xmm1, 7 + vpslld xmm1, xmm1, 25 + vpor xmm1, xmm1, xmm8 + vpshufd xmm0, xmm0, 0x93 + vpshufd xmm3, xmm3, 0x4E + vpshufd xmm2, xmm2, 0x39 + vpaddd xmm0, xmm0, xmm6 + vpaddd xmm0, xmm0, xmm1 + vpxor xmm3, xmm3, xmm0 + vpshufb xmm3, xmm3, xmm14 + vpaddd xmm2, xmm2, xmm3 + vpxor xmm1, xmm1, xmm2 + vpsrld xmm8, xmm1, 12 + vpslld xmm1, xmm1, 20 + vpor xmm1, xmm1, xmm8 + vpaddd xmm0, xmm0, xmm7 + vpaddd xmm0, xmm0, xmm1 + vpxor xmm3, xmm3, xmm0 + vpshufb xmm3, xmm3, xmm15 + vpaddd xmm2, xmm2, xmm3 + vpxor xmm1, xmm1, xmm2 + vpsrld xmm8, xmm1, 7 + vpslld xmm1, xmm1, 25 + vpor xmm1, xmm1, xmm8 + vpshufd xmm0, xmm0, 0x39 + vpshufd xmm3, xmm3, 0x4E + vpshufd xmm2, xmm2, 0x93 + dec al + jz 9f + vshufps xmm8, xmm4, xmm5, 214 + vpshufd xmm9, xmm4, 0x0F + vpshufd xmm4, xmm8, 0x39 + vshufps xmm8, xmm6, xmm7, 250 + vpblendd xmm9, xmm9, xmm8, 0xAA + vpunpcklqdq xmm8, xmm7, xmm5 + vpblendd xmm8, xmm8, xmm6, 0x88 + vpshufd xmm8, xmm8, 0x78 + vpunpckhdq xmm5, xmm5, xmm7 + vpunpckldq xmm6, xmm6, xmm5 + vpshufd xmm7, xmm6, 0x1E + vmovdqa xmm5, xmm9 + vmovdqa xmm6, xmm8 + jmp 9b +9: + vpxor xmm0, xmm0, xmm2 + vpxor xmm1, xmm1, xmm3 + mov eax, r13d + cmp rdx, r15 + jne 2b + vmovdqu xmmword ptr [rbx], xmm0 + vmovdqu xmmword ptr [rbx+0x10], xmm1 + jmp 4b + +.size zfs_blake3_hash_many_avx2, . - zfs_blake3_hash_many_avx2 + +#ifdef __APPLE__ +.static_data +#else +.section .rodata +#endif + +.p2align 6 +ADD0: + .long 0, 1, 2, 3, 4, 5, 6, 7 +ADD1: + .long 8, 8, 8, 8, 8, 8, 8, 8 +BLAKE3_IV_0: + .long 0x6A09E667, 0x6A09E667, 0x6A09E667, 0x6A09E667 + .long 0x6A09E667, 0x6A09E667, 0x6A09E667, 0x6A09E667 +BLAKE3_IV_1: + .long 0xBB67AE85, 0xBB67AE85, 0xBB67AE85, 0xBB67AE85 + .long 0xBB67AE85, 0xBB67AE85, 0xBB67AE85, 0xBB67AE85 +BLAKE3_IV_2: + .long 0x3C6EF372, 0x3C6EF372, 0x3C6EF372, 0x3C6EF372 + .long 0x3C6EF372, 0x3C6EF372, 0x3C6EF372, 0x3C6EF372 +BLAKE3_IV_3: + .long 0xA54FF53A, 0xA54FF53A, 0xA54FF53A, 0xA54FF53A + .long 0xA54FF53A, 0xA54FF53A, 0xA54FF53A, 0xA54FF53A +BLAKE3_BLOCK_LEN: + .long 0x00000040, 0x00000040, 0x00000040, 0x00000040 + .long 0x00000040, 0x00000040, 0x00000040, 0x00000040 +ROT16: + .byte 2, 3, 0, 1, 6, 7, 4, 5, 10, 11, 8, 9, 14, 15, 12, 13 +ROT8: + .byte 1, 2, 3, 0, 5, 6, 7, 4, 9, 10, 11, 8, 13, 14, 15, 12 +CMP_MSB_MASK: + .long 0x80000000, 0x80000000, 0x80000000, 0x80000000 + .long 0x80000000, 0x80000000, 0x80000000, 0x80000000 +BLAKE3_IV: + .long 0x6A09E667, 0xBB67AE85, 0x3C6EF372, 0xA54FF53A +#endif /* HAVE_AVX2 */ + +#ifdef __ELF__ +.section .note.GNU-stack,"",%progbits +#endif diff --git a/module/icp/asm-x86_64/blake3/blake3_avx512.S b/module/icp/asm-x86_64/blake3/blake3_avx512.S new file mode 100644 index 000000000000..71b5715c88c1 --- /dev/null +++ b/module/icp/asm-x86_64/blake3/blake3_avx512.S @@ -0,0 +1,2618 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Based on BLAKE3 v1.3.1, https://github.com/BLAKE3-team/BLAKE3 + * Copyright (c) 2019-2020 Samuel Neves + * Copyright (c) 2022 Tino Reichardt + */ + +#if defined(HAVE_AVX512F) && defined(HAVE_AVX512VL) + +#define _ASM +#include + +#if defined(__ELF__) && defined(__CET__) && defined(__has_include) +#if __has_include() +#include +#endif +#endif + +#if !defined(_CET_ENDBR) +#define _CET_ENDBR +#endif + +.intel_syntax noprefix +.global zfs_blake3_hash_many_avx512 +.global zfs_blake3_compress_in_place_avx512 +.global zfs_blake3_compress_xof_avx512 +.text + +.type zfs_blake3_hash_many_avx512,@function +.type zfs_blake3_compress_xof_avx512,@function +.type zfs_blake3_compress_in_place_avx512,@function + +.p2align 6 +zfs_blake3_hash_many_avx512: + _CET_ENDBR + push r15 + push r14 + push r13 + push r12 + push rbx + push rbp + mov rbp, rsp + sub rsp, 144 + and rsp, 0xFFFFFFFFFFFFFFC0 + neg r9 + kmovw k1, r9d + vmovd xmm0, r8d + vpbroadcastd ymm0, xmm0 + shr r8, 32 + vmovd xmm1, r8d + vpbroadcastd ymm1, xmm1 + vmovdqa ymm4, ymm1 + vmovdqa ymm5, ymm1 + vpaddd ymm2, ymm0, ymmword ptr [ADD0+rip] + vpaddd ymm3, ymm0, ymmword ptr [ADD0+32+rip] + vpcmpltud k2, ymm2, ymm0 + vpcmpltud k3, ymm3, ymm0 + vpaddd ymm4 {k2}, ymm4, dword ptr [ADD1+rip] {1to8} + vpaddd ymm5 {k3}, ymm5, dword ptr [ADD1+rip] {1to8} + knotw k2, k1 + vmovdqa32 ymm2 {k2}, ymm0 + vmovdqa32 ymm3 {k2}, ymm0 + vmovdqa32 ymm4 {k2}, ymm1 + vmovdqa32 ymm5 {k2}, ymm1 + vmovdqa ymmword ptr [rsp], ymm2 + vmovdqa ymmword ptr [rsp+0x1*0x20], ymm3 + vmovdqa ymmword ptr [rsp+0x2*0x20], ymm4 + vmovdqa ymmword ptr [rsp+0x3*0x20], ymm5 + shl rdx, 6 + mov qword ptr [rsp+0x80], rdx + cmp rsi, 16 + jc 3f +2: + vpbroadcastd zmm0, dword ptr [rcx] + vpbroadcastd zmm1, dword ptr [rcx+0x1*0x4] + vpbroadcastd zmm2, dword ptr [rcx+0x2*0x4] + vpbroadcastd zmm3, dword ptr [rcx+0x3*0x4] + vpbroadcastd zmm4, dword ptr [rcx+0x4*0x4] + vpbroadcastd zmm5, dword ptr [rcx+0x5*0x4] + vpbroadcastd zmm6, dword ptr [rcx+0x6*0x4] + vpbroadcastd zmm7, dword ptr [rcx+0x7*0x4] + movzx eax, byte ptr [rbp+0x38] + movzx ebx, byte ptr [rbp+0x40] + or eax, ebx + xor edx, edx +.p2align 5 +9: + movzx ebx, byte ptr [rbp+0x48] + or ebx, eax + add rdx, 64 + cmp rdx, qword ptr [rsp+0x80] + cmove eax, ebx + mov dword ptr [rsp+0x88], eax + mov r8, qword ptr [rdi] + mov r9, qword ptr [rdi+0x8] + mov r10, qword ptr [rdi+0x10] + mov r11, qword ptr [rdi+0x18] + mov r12, qword ptr [rdi+0x40] + mov r13, qword ptr [rdi+0x48] + mov r14, qword ptr [rdi+0x50] + mov r15, qword ptr [rdi+0x58] + vmovdqu32 ymm16, ymmword ptr [rdx+r8-0x2*0x20] + vinserti32x8 zmm16, zmm16, ymmword ptr [rdx+r12-0x2*0x20], 0x01 + vmovdqu32 ymm17, ymmword ptr [rdx+r9-0x2*0x20] + vinserti32x8 zmm17, zmm17, ymmword ptr [rdx+r13-0x2*0x20], 0x01 + vpunpcklqdq zmm8, zmm16, zmm17 + vpunpckhqdq zmm9, zmm16, zmm17 + vmovdqu32 ymm18, ymmword ptr [rdx+r10-0x2*0x20] + vinserti32x8 zmm18, zmm18, ymmword ptr [rdx+r14-0x2*0x20], 0x01 + vmovdqu32 ymm19, ymmword ptr [rdx+r11-0x2*0x20] + vinserti32x8 zmm19, zmm19, ymmword ptr [rdx+r15-0x2*0x20], 0x01 + vpunpcklqdq zmm10, zmm18, zmm19 + vpunpckhqdq zmm11, zmm18, zmm19 + mov r8, qword ptr [rdi+0x20] + mov r9, qword ptr [rdi+0x28] + mov r10, qword ptr [rdi+0x30] + mov r11, qword ptr [rdi+0x38] + mov r12, qword ptr [rdi+0x60] + mov r13, qword ptr [rdi+0x68] + mov r14, qword ptr [rdi+0x70] + mov r15, qword ptr [rdi+0x78] + vmovdqu32 ymm16, ymmword ptr [rdx+r8-0x2*0x20] + vinserti32x8 zmm16, zmm16, ymmword ptr [rdx+r12-0x2*0x20], 0x01 + vmovdqu32 ymm17, ymmword ptr [rdx+r9-0x2*0x20] + vinserti32x8 zmm17, zmm17, ymmword ptr [rdx+r13-0x2*0x20], 0x01 + vpunpcklqdq zmm12, zmm16, zmm17 + vpunpckhqdq zmm13, zmm16, zmm17 + vmovdqu32 ymm18, ymmword ptr [rdx+r10-0x2*0x20] + vinserti32x8 zmm18, zmm18, ymmword ptr [rdx+r14-0x2*0x20], 0x01 + vmovdqu32 ymm19, ymmword ptr [rdx+r11-0x2*0x20] + vinserti32x8 zmm19, zmm19, ymmword ptr [rdx+r15-0x2*0x20], 0x01 + vpunpcklqdq zmm14, zmm18, zmm19 + vpunpckhqdq zmm15, zmm18, zmm19 + vmovdqa32 zmm27, zmmword ptr [INDEX0+rip] + vmovdqa32 zmm31, zmmword ptr [INDEX1+rip] + vshufps zmm16, zmm8, zmm10, 136 + vshufps zmm17, zmm12, zmm14, 136 + vmovdqa32 zmm20, zmm16 + vpermt2d zmm16, zmm27, zmm17 + vpermt2d zmm20, zmm31, zmm17 + vshufps zmm17, zmm8, zmm10, 221 + vshufps zmm30, zmm12, zmm14, 221 + vmovdqa32 zmm21, zmm17 + vpermt2d zmm17, zmm27, zmm30 + vpermt2d zmm21, zmm31, zmm30 + vshufps zmm18, zmm9, zmm11, 136 + vshufps zmm8, zmm13, zmm15, 136 + vmovdqa32 zmm22, zmm18 + vpermt2d zmm18, zmm27, zmm8 + vpermt2d zmm22, zmm31, zmm8 + vshufps zmm19, zmm9, zmm11, 221 + vshufps zmm8, zmm13, zmm15, 221 + vmovdqa32 zmm23, zmm19 + vpermt2d zmm19, zmm27, zmm8 + vpermt2d zmm23, zmm31, zmm8 + mov r8, qword ptr [rdi] + mov r9, qword ptr [rdi+0x8] + mov r10, qword ptr [rdi+0x10] + mov r11, qword ptr [rdi+0x18] + mov r12, qword ptr [rdi+0x40] + mov r13, qword ptr [rdi+0x48] + mov r14, qword ptr [rdi+0x50] + mov r15, qword ptr [rdi+0x58] + vmovdqu32 ymm24, ymmword ptr [r8+rdx-0x1*0x20] + vinserti32x8 zmm24, zmm24, ymmword ptr [r12+rdx-0x1*0x20], 0x01 + vmovdqu32 ymm25, ymmword ptr [r9+rdx-0x1*0x20] + vinserti32x8 zmm25, zmm25, ymmword ptr [r13+rdx-0x1*0x20], 0x01 + vpunpcklqdq zmm8, zmm24, zmm25 + vpunpckhqdq zmm9, zmm24, zmm25 + vmovdqu32 ymm24, ymmword ptr [r10+rdx-0x1*0x20] + vinserti32x8 zmm24, zmm24, ymmword ptr [r14+rdx-0x1*0x20], 0x01 + vmovdqu32 ymm25, ymmword ptr [r11+rdx-0x1*0x20] + vinserti32x8 zmm25, zmm25, ymmword ptr [r15+rdx-0x1*0x20], 0x01 + vpunpcklqdq zmm10, zmm24, zmm25 + vpunpckhqdq zmm11, zmm24, zmm25 + prefetcht0 [r8+rdx+0x80] + prefetcht0 [r12+rdx+0x80] + prefetcht0 [r9+rdx+0x80] + prefetcht0 [r13+rdx+0x80] + prefetcht0 [r10+rdx+0x80] + prefetcht0 [r14+rdx+0x80] + prefetcht0 [r11+rdx+0x80] + prefetcht0 [r15+rdx+0x80] + mov r8, qword ptr [rdi+0x20] + mov r9, qword ptr [rdi+0x28] + mov r10, qword ptr [rdi+0x30] + mov r11, qword ptr [rdi+0x38] + mov r12, qword ptr [rdi+0x60] + mov r13, qword ptr [rdi+0x68] + mov r14, qword ptr [rdi+0x70] + mov r15, qword ptr [rdi+0x78] + vmovdqu32 ymm24, ymmword ptr [r8+rdx-0x1*0x20] + vinserti32x8 zmm24, zmm24, ymmword ptr [r12+rdx-0x1*0x20], 0x01 + vmovdqu32 ymm25, ymmword ptr [r9+rdx-0x1*0x20] + vinserti32x8 zmm25, zmm25, ymmword ptr [r13+rdx-0x1*0x20], 0x01 + vpunpcklqdq zmm12, zmm24, zmm25 + vpunpckhqdq zmm13, zmm24, zmm25 + vmovdqu32 ymm24, ymmword ptr [r10+rdx-0x1*0x20] + vinserti32x8 zmm24, zmm24, ymmword ptr [r14+rdx-0x1*0x20], 0x01 + vmovdqu32 ymm25, ymmword ptr [r11+rdx-0x1*0x20] + vinserti32x8 zmm25, zmm25, ymmword ptr [r15+rdx-0x1*0x20], 0x01 + vpunpcklqdq zmm14, zmm24, zmm25 + vpunpckhqdq zmm15, zmm24, zmm25 + prefetcht0 [r8+rdx+0x80] + prefetcht0 [r12+rdx+0x80] + prefetcht0 [r9+rdx+0x80] + prefetcht0 [r13+rdx+0x80] + prefetcht0 [r10+rdx+0x80] + prefetcht0 [r14+rdx+0x80] + prefetcht0 [r11+rdx+0x80] + prefetcht0 [r15+rdx+0x80] + vshufps zmm24, zmm8, zmm10, 136 + vshufps zmm30, zmm12, zmm14, 136 + vmovdqa32 zmm28, zmm24 + vpermt2d zmm24, zmm27, zmm30 + vpermt2d zmm28, zmm31, zmm30 + vshufps zmm25, zmm8, zmm10, 221 + vshufps zmm30, zmm12, zmm14, 221 + vmovdqa32 zmm29, zmm25 + vpermt2d zmm25, zmm27, zmm30 + vpermt2d zmm29, zmm31, zmm30 + vshufps zmm26, zmm9, zmm11, 136 + vshufps zmm8, zmm13, zmm15, 136 + vmovdqa32 zmm30, zmm26 + vpermt2d zmm26, zmm27, zmm8 + vpermt2d zmm30, zmm31, zmm8 + vshufps zmm8, zmm9, zmm11, 221 + vshufps zmm10, zmm13, zmm15, 221 + vpermi2d zmm27, zmm8, zmm10 + vpermi2d zmm31, zmm8, zmm10 + vpbroadcastd zmm8, dword ptr [BLAKE3_IV_0+rip] + vpbroadcastd zmm9, dword ptr [BLAKE3_IV_1+rip] + vpbroadcastd zmm10, dword ptr [BLAKE3_IV_2+rip] + vpbroadcastd zmm11, dword ptr [BLAKE3_IV_3+rip] + vmovdqa32 zmm12, zmmword ptr [rsp] + vmovdqa32 zmm13, zmmword ptr [rsp+0x1*0x40] + vpbroadcastd zmm14, dword ptr [BLAKE3_BLOCK_LEN+rip] + vpbroadcastd zmm15, dword ptr [rsp+0x22*0x4] + vpaddd zmm0, zmm0, zmm16 + vpaddd zmm1, zmm1, zmm18 + vpaddd zmm2, zmm2, zmm20 + vpaddd zmm3, zmm3, zmm22 + vpaddd zmm0, zmm0, zmm4 + vpaddd zmm1, zmm1, zmm5 + vpaddd zmm2, zmm2, zmm6 + vpaddd zmm3, zmm3, zmm7 + vpxord zmm12, zmm12, zmm0 + vpxord zmm13, zmm13, zmm1 + vpxord zmm14, zmm14, zmm2 + vpxord zmm15, zmm15, zmm3 + vprord zmm12, zmm12, 16 + vprord zmm13, zmm13, 16 + vprord zmm14, zmm14, 16 + vprord zmm15, zmm15, 16 + vpaddd zmm8, zmm8, zmm12 + vpaddd zmm9, zmm9, zmm13 + vpaddd zmm10, zmm10, zmm14 + vpaddd zmm11, zmm11, zmm15 + vpxord zmm4, zmm4, zmm8 + vpxord zmm5, zmm5, zmm9 + vpxord zmm6, zmm6, zmm10 + vpxord zmm7, zmm7, zmm11 + vprord zmm4, zmm4, 12 + vprord zmm5, zmm5, 12 + vprord zmm6, zmm6, 12 + vprord zmm7, zmm7, 12 + vpaddd zmm0, zmm0, zmm17 + vpaddd zmm1, zmm1, zmm19 + vpaddd zmm2, zmm2, zmm21 + vpaddd zmm3, zmm3, zmm23 + vpaddd zmm0, zmm0, zmm4 + vpaddd zmm1, zmm1, zmm5 + vpaddd zmm2, zmm2, zmm6 + vpaddd zmm3, zmm3, zmm7 + vpxord zmm12, zmm12, zmm0 + vpxord zmm13, zmm13, zmm1 + vpxord zmm14, zmm14, zmm2 + vpxord zmm15, zmm15, zmm3 + vprord zmm12, zmm12, 8 + vprord zmm13, zmm13, 8 + vprord zmm14, zmm14, 8 + vprord zmm15, zmm15, 8 + vpaddd zmm8, zmm8, zmm12 + vpaddd zmm9, zmm9, zmm13 + vpaddd zmm10, zmm10, zmm14 + vpaddd zmm11, zmm11, zmm15 + vpxord zmm4, zmm4, zmm8 + vpxord zmm5, zmm5, zmm9 + vpxord zmm6, zmm6, zmm10 + vpxord zmm7, zmm7, zmm11 + vprord zmm4, zmm4, 7 + vprord zmm5, zmm5, 7 + vprord zmm6, zmm6, 7 + vprord zmm7, zmm7, 7 + vpaddd zmm0, zmm0, zmm24 + vpaddd zmm1, zmm1, zmm26 + vpaddd zmm2, zmm2, zmm28 + vpaddd zmm3, zmm3, zmm30 + vpaddd zmm0, zmm0, zmm5 + vpaddd zmm1, zmm1, zmm6 + vpaddd zmm2, zmm2, zmm7 + vpaddd zmm3, zmm3, zmm4 + vpxord zmm15, zmm15, zmm0 + vpxord zmm12, zmm12, zmm1 + vpxord zmm13, zmm13, zmm2 + vpxord zmm14, zmm14, zmm3 + vprord zmm15, zmm15, 16 + vprord zmm12, zmm12, 16 + vprord zmm13, zmm13, 16 + vprord zmm14, zmm14, 16 + vpaddd zmm10, zmm10, zmm15 + vpaddd zmm11, zmm11, zmm12 + vpaddd zmm8, zmm8, zmm13 + vpaddd zmm9, zmm9, zmm14 + vpxord zmm5, zmm5, zmm10 + vpxord zmm6, zmm6, zmm11 + vpxord zmm7, zmm7, zmm8 + vpxord zmm4, zmm4, zmm9 + vprord zmm5, zmm5, 12 + vprord zmm6, zmm6, 12 + vprord zmm7, zmm7, 12 + vprord zmm4, zmm4, 12 + vpaddd zmm0, zmm0, zmm25 + vpaddd zmm1, zmm1, zmm27 + vpaddd zmm2, zmm2, zmm29 + vpaddd zmm3, zmm3, zmm31 + vpaddd zmm0, zmm0, zmm5 + vpaddd zmm1, zmm1, zmm6 + vpaddd zmm2, zmm2, zmm7 + vpaddd zmm3, zmm3, zmm4 + vpxord zmm15, zmm15, zmm0 + vpxord zmm12, zmm12, zmm1 + vpxord zmm13, zmm13, zmm2 + vpxord zmm14, zmm14, zmm3 + vprord zmm15, zmm15, 8 + vprord zmm12, zmm12, 8 + vprord zmm13, zmm13, 8 + vprord zmm14, zmm14, 8 + vpaddd zmm10, zmm10, zmm15 + vpaddd zmm11, zmm11, zmm12 + vpaddd zmm8, zmm8, zmm13 + vpaddd zmm9, zmm9, zmm14 + vpxord zmm5, zmm5, zmm10 + vpxord zmm6, zmm6, zmm11 + vpxord zmm7, zmm7, zmm8 + vpxord zmm4, zmm4, zmm9 + vprord zmm5, zmm5, 7 + vprord zmm6, zmm6, 7 + vprord zmm7, zmm7, 7 + vprord zmm4, zmm4, 7 + vpaddd zmm0, zmm0, zmm18 + vpaddd zmm1, zmm1, zmm19 + vpaddd zmm2, zmm2, zmm23 + vpaddd zmm3, zmm3, zmm20 + vpaddd zmm0, zmm0, zmm4 + vpaddd zmm1, zmm1, zmm5 + vpaddd zmm2, zmm2, zmm6 + vpaddd zmm3, zmm3, zmm7 + vpxord zmm12, zmm12, zmm0 + vpxord zmm13, zmm13, zmm1 + vpxord zmm14, zmm14, zmm2 + vpxord zmm15, zmm15, zmm3 + vprord zmm12, zmm12, 16 + vprord zmm13, zmm13, 16 + vprord zmm14, zmm14, 16 + vprord zmm15, zmm15, 16 + vpaddd zmm8, zmm8, zmm12 + vpaddd zmm9, zmm9, zmm13 + vpaddd zmm10, zmm10, zmm14 + vpaddd zmm11, zmm11, zmm15 + vpxord zmm4, zmm4, zmm8 + vpxord zmm5, zmm5, zmm9 + vpxord zmm6, zmm6, zmm10 + vpxord zmm7, zmm7, zmm11 + vprord zmm4, zmm4, 12 + vprord zmm5, zmm5, 12 + vprord zmm6, zmm6, 12 + vprord zmm7, zmm7, 12 + vpaddd zmm0, zmm0, zmm22 + vpaddd zmm1, zmm1, zmm26 + vpaddd zmm2, zmm2, zmm16 + vpaddd zmm3, zmm3, zmm29 + vpaddd zmm0, zmm0, zmm4 + vpaddd zmm1, zmm1, zmm5 + vpaddd zmm2, zmm2, zmm6 + vpaddd zmm3, zmm3, zmm7 + vpxord zmm12, zmm12, zmm0 + vpxord zmm13, zmm13, zmm1 + vpxord zmm14, zmm14, zmm2 + vpxord zmm15, zmm15, zmm3 + vprord zmm12, zmm12, 8 + vprord zmm13, zmm13, 8 + vprord zmm14, zmm14, 8 + vprord zmm15, zmm15, 8 + vpaddd zmm8, zmm8, zmm12 + vpaddd zmm9, zmm9, zmm13 + vpaddd zmm10, zmm10, zmm14 + vpaddd zmm11, zmm11, zmm15 + vpxord zmm4, zmm4, zmm8 + vpxord zmm5, zmm5, zmm9 + vpxord zmm6, zmm6, zmm10 + vpxord zmm7, zmm7, zmm11 + vprord zmm4, zmm4, 7 + vprord zmm5, zmm5, 7 + vprord zmm6, zmm6, 7 + vprord zmm7, zmm7, 7 + vpaddd zmm0, zmm0, zmm17 + vpaddd zmm1, zmm1, zmm28 + vpaddd zmm2, zmm2, zmm25 + vpaddd zmm3, zmm3, zmm31 + vpaddd zmm0, zmm0, zmm5 + vpaddd zmm1, zmm1, zmm6 + vpaddd zmm2, zmm2, zmm7 + vpaddd zmm3, zmm3, zmm4 + vpxord zmm15, zmm15, zmm0 + vpxord zmm12, zmm12, zmm1 + vpxord zmm13, zmm13, zmm2 + vpxord zmm14, zmm14, zmm3 + vprord zmm15, zmm15, 16 + vprord zmm12, zmm12, 16 + vprord zmm13, zmm13, 16 + vprord zmm14, zmm14, 16 + vpaddd zmm10, zmm10, zmm15 + vpaddd zmm11, zmm11, zmm12 + vpaddd zmm8, zmm8, zmm13 + vpaddd zmm9, zmm9, zmm14 + vpxord zmm5, zmm5, zmm10 + vpxord zmm6, zmm6, zmm11 + vpxord zmm7, zmm7, zmm8 + vpxord zmm4, zmm4, zmm9 + vprord zmm5, zmm5, 12 + vprord zmm6, zmm6, 12 + vprord zmm7, zmm7, 12 + vprord zmm4, zmm4, 12 + vpaddd zmm0, zmm0, zmm27 + vpaddd zmm1, zmm1, zmm21 + vpaddd zmm2, zmm2, zmm30 + vpaddd zmm3, zmm3, zmm24 + vpaddd zmm0, zmm0, zmm5 + vpaddd zmm1, zmm1, zmm6 + vpaddd zmm2, zmm2, zmm7 + vpaddd zmm3, zmm3, zmm4 + vpxord zmm15, zmm15, zmm0 + vpxord zmm12, zmm12, zmm1 + vpxord zmm13, zmm13, zmm2 + vpxord zmm14, zmm14, zmm3 + vprord zmm15, zmm15, 8 + vprord zmm12, zmm12, 8 + vprord zmm13, zmm13, 8 + vprord zmm14, zmm14, 8 + vpaddd zmm10, zmm10, zmm15 + vpaddd zmm11, zmm11, zmm12 + vpaddd zmm8, zmm8, zmm13 + vpaddd zmm9, zmm9, zmm14 + vpxord zmm5, zmm5, zmm10 + vpxord zmm6, zmm6, zmm11 + vpxord zmm7, zmm7, zmm8 + vpxord zmm4, zmm4, zmm9 + vprord zmm5, zmm5, 7 + vprord zmm6, zmm6, 7 + vprord zmm7, zmm7, 7 + vprord zmm4, zmm4, 7 + vpaddd zmm0, zmm0, zmm19 + vpaddd zmm1, zmm1, zmm26 + vpaddd zmm2, zmm2, zmm29 + vpaddd zmm3, zmm3, zmm23 + vpaddd zmm0, zmm0, zmm4 + vpaddd zmm1, zmm1, zmm5 + vpaddd zmm2, zmm2, zmm6 + vpaddd zmm3, zmm3, zmm7 + vpxord zmm12, zmm12, zmm0 + vpxord zmm13, zmm13, zmm1 + vpxord zmm14, zmm14, zmm2 + vpxord zmm15, zmm15, zmm3 + vprord zmm12, zmm12, 16 + vprord zmm13, zmm13, 16 + vprord zmm14, zmm14, 16 + vprord zmm15, zmm15, 16 + vpaddd zmm8, zmm8, zmm12 + vpaddd zmm9, zmm9, zmm13 + vpaddd zmm10, zmm10, zmm14 + vpaddd zmm11, zmm11, zmm15 + vpxord zmm4, zmm4, zmm8 + vpxord zmm5, zmm5, zmm9 + vpxord zmm6, zmm6, zmm10 + vpxord zmm7, zmm7, zmm11 + vprord zmm4, zmm4, 12 + vprord zmm5, zmm5, 12 + vprord zmm6, zmm6, 12 + vprord zmm7, zmm7, 12 + vpaddd zmm0, zmm0, zmm20 + vpaddd zmm1, zmm1, zmm28 + vpaddd zmm2, zmm2, zmm18 + vpaddd zmm3, zmm3, zmm30 + vpaddd zmm0, zmm0, zmm4 + vpaddd zmm1, zmm1, zmm5 + vpaddd zmm2, zmm2, zmm6 + vpaddd zmm3, zmm3, zmm7 + vpxord zmm12, zmm12, zmm0 + vpxord zmm13, zmm13, zmm1 + vpxord zmm14, zmm14, zmm2 + vpxord zmm15, zmm15, zmm3 + vprord zmm12, zmm12, 8 + vprord zmm13, zmm13, 8 + vprord zmm14, zmm14, 8 + vprord zmm15, zmm15, 8 + vpaddd zmm8, zmm8, zmm12 + vpaddd zmm9, zmm9, zmm13 + vpaddd zmm10, zmm10, zmm14 + vpaddd zmm11, zmm11, zmm15 + vpxord zmm4, zmm4, zmm8 + vpxord zmm5, zmm5, zmm9 + vpxord zmm6, zmm6, zmm10 + vpxord zmm7, zmm7, zmm11 + vprord zmm4, zmm4, 7 + vprord zmm5, zmm5, 7 + vprord zmm6, zmm6, 7 + vprord zmm7, zmm7, 7 + vpaddd zmm0, zmm0, zmm22 + vpaddd zmm1, zmm1, zmm25 + vpaddd zmm2, zmm2, zmm27 + vpaddd zmm3, zmm3, zmm24 + vpaddd zmm0, zmm0, zmm5 + vpaddd zmm1, zmm1, zmm6 + vpaddd zmm2, zmm2, zmm7 + vpaddd zmm3, zmm3, zmm4 + vpxord zmm15, zmm15, zmm0 + vpxord zmm12, zmm12, zmm1 + vpxord zmm13, zmm13, zmm2 + vpxord zmm14, zmm14, zmm3 + vprord zmm15, zmm15, 16 + vprord zmm12, zmm12, 16 + vprord zmm13, zmm13, 16 + vprord zmm14, zmm14, 16 + vpaddd zmm10, zmm10, zmm15 + vpaddd zmm11, zmm11, zmm12 + vpaddd zmm8, zmm8, zmm13 + vpaddd zmm9, zmm9, zmm14 + vpxord zmm5, zmm5, zmm10 + vpxord zmm6, zmm6, zmm11 + vpxord zmm7, zmm7, zmm8 + vpxord zmm4, zmm4, zmm9 + vprord zmm5, zmm5, 12 + vprord zmm6, zmm6, 12 + vprord zmm7, zmm7, 12 + vprord zmm4, zmm4, 12 + vpaddd zmm0, zmm0, zmm21 + vpaddd zmm1, zmm1, zmm16 + vpaddd zmm2, zmm2, zmm31 + vpaddd zmm3, zmm3, zmm17 + vpaddd zmm0, zmm0, zmm5 + vpaddd zmm1, zmm1, zmm6 + vpaddd zmm2, zmm2, zmm7 + vpaddd zmm3, zmm3, zmm4 + vpxord zmm15, zmm15, zmm0 + vpxord zmm12, zmm12, zmm1 + vpxord zmm13, zmm13, zmm2 + vpxord zmm14, zmm14, zmm3 + vprord zmm15, zmm15, 8 + vprord zmm12, zmm12, 8 + vprord zmm13, zmm13, 8 + vprord zmm14, zmm14, 8 + vpaddd zmm10, zmm10, zmm15 + vpaddd zmm11, zmm11, zmm12 + vpaddd zmm8, zmm8, zmm13 + vpaddd zmm9, zmm9, zmm14 + vpxord zmm5, zmm5, zmm10 + vpxord zmm6, zmm6, zmm11 + vpxord zmm7, zmm7, zmm8 + vpxord zmm4, zmm4, zmm9 + vprord zmm5, zmm5, 7 + vprord zmm6, zmm6, 7 + vprord zmm7, zmm7, 7 + vprord zmm4, zmm4, 7 + vpaddd zmm0, zmm0, zmm26 + vpaddd zmm1, zmm1, zmm28 + vpaddd zmm2, zmm2, zmm30 + vpaddd zmm3, zmm3, zmm29 + vpaddd zmm0, zmm0, zmm4 + vpaddd zmm1, zmm1, zmm5 + vpaddd zmm2, zmm2, zmm6 + vpaddd zmm3, zmm3, zmm7 + vpxord zmm12, zmm12, zmm0 + vpxord zmm13, zmm13, zmm1 + vpxord zmm14, zmm14, zmm2 + vpxord zmm15, zmm15, zmm3 + vprord zmm12, zmm12, 16 + vprord zmm13, zmm13, 16 + vprord zmm14, zmm14, 16 + vprord zmm15, zmm15, 16 + vpaddd zmm8, zmm8, zmm12 + vpaddd zmm9, zmm9, zmm13 + vpaddd zmm10, zmm10, zmm14 + vpaddd zmm11, zmm11, zmm15 + vpxord zmm4, zmm4, zmm8 + vpxord zmm5, zmm5, zmm9 + vpxord zmm6, zmm6, zmm10 + vpxord zmm7, zmm7, zmm11 + vprord zmm4, zmm4, 12 + vprord zmm5, zmm5, 12 + vprord zmm6, zmm6, 12 + vprord zmm7, zmm7, 12 + vpaddd zmm0, zmm0, zmm23 + vpaddd zmm1, zmm1, zmm25 + vpaddd zmm2, zmm2, zmm19 + vpaddd zmm3, zmm3, zmm31 + vpaddd zmm0, zmm0, zmm4 + vpaddd zmm1, zmm1, zmm5 + vpaddd zmm2, zmm2, zmm6 + vpaddd zmm3, zmm3, zmm7 + vpxord zmm12, zmm12, zmm0 + vpxord zmm13, zmm13, zmm1 + vpxord zmm14, zmm14, zmm2 + vpxord zmm15, zmm15, zmm3 + vprord zmm12, zmm12, 8 + vprord zmm13, zmm13, 8 + vprord zmm14, zmm14, 8 + vprord zmm15, zmm15, 8 + vpaddd zmm8, zmm8, zmm12 + vpaddd zmm9, zmm9, zmm13 + vpaddd zmm10, zmm10, zmm14 + vpaddd zmm11, zmm11, zmm15 + vpxord zmm4, zmm4, zmm8 + vpxord zmm5, zmm5, zmm9 + vpxord zmm6, zmm6, zmm10 + vpxord zmm7, zmm7, zmm11 + vprord zmm4, zmm4, 7 + vprord zmm5, zmm5, 7 + vprord zmm6, zmm6, 7 + vprord zmm7, zmm7, 7 + vpaddd zmm0, zmm0, zmm20 + vpaddd zmm1, zmm1, zmm27 + vpaddd zmm2, zmm2, zmm21 + vpaddd zmm3, zmm3, zmm17 + vpaddd zmm0, zmm0, zmm5 + vpaddd zmm1, zmm1, zmm6 + vpaddd zmm2, zmm2, zmm7 + vpaddd zmm3, zmm3, zmm4 + vpxord zmm15, zmm15, zmm0 + vpxord zmm12, zmm12, zmm1 + vpxord zmm13, zmm13, zmm2 + vpxord zmm14, zmm14, zmm3 + vprord zmm15, zmm15, 16 + vprord zmm12, zmm12, 16 + vprord zmm13, zmm13, 16 + vprord zmm14, zmm14, 16 + vpaddd zmm10, zmm10, zmm15 + vpaddd zmm11, zmm11, zmm12 + vpaddd zmm8, zmm8, zmm13 + vpaddd zmm9, zmm9, zmm14 + vpxord zmm5, zmm5, zmm10 + vpxord zmm6, zmm6, zmm11 + vpxord zmm7, zmm7, zmm8 + vpxord zmm4, zmm4, zmm9 + vprord zmm5, zmm5, 12 + vprord zmm6, zmm6, 12 + vprord zmm7, zmm7, 12 + vprord zmm4, zmm4, 12 + vpaddd zmm0, zmm0, zmm16 + vpaddd zmm1, zmm1, zmm18 + vpaddd zmm2, zmm2, zmm24 + vpaddd zmm3, zmm3, zmm22 + vpaddd zmm0, zmm0, zmm5 + vpaddd zmm1, zmm1, zmm6 + vpaddd zmm2, zmm2, zmm7 + vpaddd zmm3, zmm3, zmm4 + vpxord zmm15, zmm15, zmm0 + vpxord zmm12, zmm12, zmm1 + vpxord zmm13, zmm13, zmm2 + vpxord zmm14, zmm14, zmm3 + vprord zmm15, zmm15, 8 + vprord zmm12, zmm12, 8 + vprord zmm13, zmm13, 8 + vprord zmm14, zmm14, 8 + vpaddd zmm10, zmm10, zmm15 + vpaddd zmm11, zmm11, zmm12 + vpaddd zmm8, zmm8, zmm13 + vpaddd zmm9, zmm9, zmm14 + vpxord zmm5, zmm5, zmm10 + vpxord zmm6, zmm6, zmm11 + vpxord zmm7, zmm7, zmm8 + vpxord zmm4, zmm4, zmm9 + vprord zmm5, zmm5, 7 + vprord zmm6, zmm6, 7 + vprord zmm7, zmm7, 7 + vprord zmm4, zmm4, 7 + vpaddd zmm0, zmm0, zmm28 + vpaddd zmm1, zmm1, zmm25 + vpaddd zmm2, zmm2, zmm31 + vpaddd zmm3, zmm3, zmm30 + vpaddd zmm0, zmm0, zmm4 + vpaddd zmm1, zmm1, zmm5 + vpaddd zmm2, zmm2, zmm6 + vpaddd zmm3, zmm3, zmm7 + vpxord zmm12, zmm12, zmm0 + vpxord zmm13, zmm13, zmm1 + vpxord zmm14, zmm14, zmm2 + vpxord zmm15, zmm15, zmm3 + vprord zmm12, zmm12, 16 + vprord zmm13, zmm13, 16 + vprord zmm14, zmm14, 16 + vprord zmm15, zmm15, 16 + vpaddd zmm8, zmm8, zmm12 + vpaddd zmm9, zmm9, zmm13 + vpaddd zmm10, zmm10, zmm14 + vpaddd zmm11, zmm11, zmm15 + vpxord zmm4, zmm4, zmm8 + vpxord zmm5, zmm5, zmm9 + vpxord zmm6, zmm6, zmm10 + vpxord zmm7, zmm7, zmm11 + vprord zmm4, zmm4, 12 + vprord zmm5, zmm5, 12 + vprord zmm6, zmm6, 12 + vprord zmm7, zmm7, 12 + vpaddd zmm0, zmm0, zmm29 + vpaddd zmm1, zmm1, zmm27 + vpaddd zmm2, zmm2, zmm26 + vpaddd zmm3, zmm3, zmm24 + vpaddd zmm0, zmm0, zmm4 + vpaddd zmm1, zmm1, zmm5 + vpaddd zmm2, zmm2, zmm6 + vpaddd zmm3, zmm3, zmm7 + vpxord zmm12, zmm12, zmm0 + vpxord zmm13, zmm13, zmm1 + vpxord zmm14, zmm14, zmm2 + vpxord zmm15, zmm15, zmm3 + vprord zmm12, zmm12, 8 + vprord zmm13, zmm13, 8 + vprord zmm14, zmm14, 8 + vprord zmm15, zmm15, 8 + vpaddd zmm8, zmm8, zmm12 + vpaddd zmm9, zmm9, zmm13 + vpaddd zmm10, zmm10, zmm14 + vpaddd zmm11, zmm11, zmm15 + vpxord zmm4, zmm4, zmm8 + vpxord zmm5, zmm5, zmm9 + vpxord zmm6, zmm6, zmm10 + vpxord zmm7, zmm7, zmm11 + vprord zmm4, zmm4, 7 + vprord zmm5, zmm5, 7 + vprord zmm6, zmm6, 7 + vprord zmm7, zmm7, 7 + vpaddd zmm0, zmm0, zmm23 + vpaddd zmm1, zmm1, zmm21 + vpaddd zmm2, zmm2, zmm16 + vpaddd zmm3, zmm3, zmm22 + vpaddd zmm0, zmm0, zmm5 + vpaddd zmm1, zmm1, zmm6 + vpaddd zmm2, zmm2, zmm7 + vpaddd zmm3, zmm3, zmm4 + vpxord zmm15, zmm15, zmm0 + vpxord zmm12, zmm12, zmm1 + vpxord zmm13, zmm13, zmm2 + vpxord zmm14, zmm14, zmm3 + vprord zmm15, zmm15, 16 + vprord zmm12, zmm12, 16 + vprord zmm13, zmm13, 16 + vprord zmm14, zmm14, 16 + vpaddd zmm10, zmm10, zmm15 + vpaddd zmm11, zmm11, zmm12 + vpaddd zmm8, zmm8, zmm13 + vpaddd zmm9, zmm9, zmm14 + vpxord zmm5, zmm5, zmm10 + vpxord zmm6, zmm6, zmm11 + vpxord zmm7, zmm7, zmm8 + vpxord zmm4, zmm4, zmm9 + vprord zmm5, zmm5, 12 + vprord zmm6, zmm6, 12 + vprord zmm7, zmm7, 12 + vprord zmm4, zmm4, 12 + vpaddd zmm0, zmm0, zmm18 + vpaddd zmm1, zmm1, zmm19 + vpaddd zmm2, zmm2, zmm17 + vpaddd zmm3, zmm3, zmm20 + vpaddd zmm0, zmm0, zmm5 + vpaddd zmm1, zmm1, zmm6 + vpaddd zmm2, zmm2, zmm7 + vpaddd zmm3, zmm3, zmm4 + vpxord zmm15, zmm15, zmm0 + vpxord zmm12, zmm12, zmm1 + vpxord zmm13, zmm13, zmm2 + vpxord zmm14, zmm14, zmm3 + vprord zmm15, zmm15, 8 + vprord zmm12, zmm12, 8 + vprord zmm13, zmm13, 8 + vprord zmm14, zmm14, 8 + vpaddd zmm10, zmm10, zmm15 + vpaddd zmm11, zmm11, zmm12 + vpaddd zmm8, zmm8, zmm13 + vpaddd zmm9, zmm9, zmm14 + vpxord zmm5, zmm5, zmm10 + vpxord zmm6, zmm6, zmm11 + vpxord zmm7, zmm7, zmm8 + vpxord zmm4, zmm4, zmm9 + vprord zmm5, zmm5, 7 + vprord zmm6, zmm6, 7 + vprord zmm7, zmm7, 7 + vprord zmm4, zmm4, 7 + vpaddd zmm0, zmm0, zmm25 + vpaddd zmm1, zmm1, zmm27 + vpaddd zmm2, zmm2, zmm24 + vpaddd zmm3, zmm3, zmm31 + vpaddd zmm0, zmm0, zmm4 + vpaddd zmm1, zmm1, zmm5 + vpaddd zmm2, zmm2, zmm6 + vpaddd zmm3, zmm3, zmm7 + vpxord zmm12, zmm12, zmm0 + vpxord zmm13, zmm13, zmm1 + vpxord zmm14, zmm14, zmm2 + vpxord zmm15, zmm15, zmm3 + vprord zmm12, zmm12, 16 + vprord zmm13, zmm13, 16 + vprord zmm14, zmm14, 16 + vprord zmm15, zmm15, 16 + vpaddd zmm8, zmm8, zmm12 + vpaddd zmm9, zmm9, zmm13 + vpaddd zmm10, zmm10, zmm14 + vpaddd zmm11, zmm11, zmm15 + vpxord zmm4, zmm4, zmm8 + vpxord zmm5, zmm5, zmm9 + vpxord zmm6, zmm6, zmm10 + vpxord zmm7, zmm7, zmm11 + vprord zmm4, zmm4, 12 + vprord zmm5, zmm5, 12 + vprord zmm6, zmm6, 12 + vprord zmm7, zmm7, 12 + vpaddd zmm0, zmm0, zmm30 + vpaddd zmm1, zmm1, zmm21 + vpaddd zmm2, zmm2, zmm28 + vpaddd zmm3, zmm3, zmm17 + vpaddd zmm0, zmm0, zmm4 + vpaddd zmm1, zmm1, zmm5 + vpaddd zmm2, zmm2, zmm6 + vpaddd zmm3, zmm3, zmm7 + vpxord zmm12, zmm12, zmm0 + vpxord zmm13, zmm13, zmm1 + vpxord zmm14, zmm14, zmm2 + vpxord zmm15, zmm15, zmm3 + vprord zmm12, zmm12, 8 + vprord zmm13, zmm13, 8 + vprord zmm14, zmm14, 8 + vprord zmm15, zmm15, 8 + vpaddd zmm8, zmm8, zmm12 + vpaddd zmm9, zmm9, zmm13 + vpaddd zmm10, zmm10, zmm14 + vpaddd zmm11, zmm11, zmm15 + vpxord zmm4, zmm4, zmm8 + vpxord zmm5, zmm5, zmm9 + vpxord zmm6, zmm6, zmm10 + vpxord zmm7, zmm7, zmm11 + vprord zmm4, zmm4, 7 + vprord zmm5, zmm5, 7 + vprord zmm6, zmm6, 7 + vprord zmm7, zmm7, 7 + vpaddd zmm0, zmm0, zmm29 + vpaddd zmm1, zmm1, zmm16 + vpaddd zmm2, zmm2, zmm18 + vpaddd zmm3, zmm3, zmm20 + vpaddd zmm0, zmm0, zmm5 + vpaddd zmm1, zmm1, zmm6 + vpaddd zmm2, zmm2, zmm7 + vpaddd zmm3, zmm3, zmm4 + vpxord zmm15, zmm15, zmm0 + vpxord zmm12, zmm12, zmm1 + vpxord zmm13, zmm13, zmm2 + vpxord zmm14, zmm14, zmm3 + vprord zmm15, zmm15, 16 + vprord zmm12, zmm12, 16 + vprord zmm13, zmm13, 16 + vprord zmm14, zmm14, 16 + vpaddd zmm10, zmm10, zmm15 + vpaddd zmm11, zmm11, zmm12 + vpaddd zmm8, zmm8, zmm13 + vpaddd zmm9, zmm9, zmm14 + vpxord zmm5, zmm5, zmm10 + vpxord zmm6, zmm6, zmm11 + vpxord zmm7, zmm7, zmm8 + vpxord zmm4, zmm4, zmm9 + vprord zmm5, zmm5, 12 + vprord zmm6, zmm6, 12 + vprord zmm7, zmm7, 12 + vprord zmm4, zmm4, 12 + vpaddd zmm0, zmm0, zmm19 + vpaddd zmm1, zmm1, zmm26 + vpaddd zmm2, zmm2, zmm22 + vpaddd zmm3, zmm3, zmm23 + vpaddd zmm0, zmm0, zmm5 + vpaddd zmm1, zmm1, zmm6 + vpaddd zmm2, zmm2, zmm7 + vpaddd zmm3, zmm3, zmm4 + vpxord zmm15, zmm15, zmm0 + vpxord zmm12, zmm12, zmm1 + vpxord zmm13, zmm13, zmm2 + vpxord zmm14, zmm14, zmm3 + vprord zmm15, zmm15, 8 + vprord zmm12, zmm12, 8 + vprord zmm13, zmm13, 8 + vprord zmm14, zmm14, 8 + vpaddd zmm10, zmm10, zmm15 + vpaddd zmm11, zmm11, zmm12 + vpaddd zmm8, zmm8, zmm13 + vpaddd zmm9, zmm9, zmm14 + vpxord zmm5, zmm5, zmm10 + vpxord zmm6, zmm6, zmm11 + vpxord zmm7, zmm7, zmm8 + vpxord zmm4, zmm4, zmm9 + vprord zmm5, zmm5, 7 + vprord zmm6, zmm6, 7 + vprord zmm7, zmm7, 7 + vprord zmm4, zmm4, 7 + vpaddd zmm0, zmm0, zmm27 + vpaddd zmm1, zmm1, zmm21 + vpaddd zmm2, zmm2, zmm17 + vpaddd zmm3, zmm3, zmm24 + vpaddd zmm0, zmm0, zmm4 + vpaddd zmm1, zmm1, zmm5 + vpaddd zmm2, zmm2, zmm6 + vpaddd zmm3, zmm3, zmm7 + vpxord zmm12, zmm12, zmm0 + vpxord zmm13, zmm13, zmm1 + vpxord zmm14, zmm14, zmm2 + vpxord zmm15, zmm15, zmm3 + vprord zmm12, zmm12, 16 + vprord zmm13, zmm13, 16 + vprord zmm14, zmm14, 16 + vprord zmm15, zmm15, 16 + vpaddd zmm8, zmm8, zmm12 + vpaddd zmm9, zmm9, zmm13 + vpaddd zmm10, zmm10, zmm14 + vpaddd zmm11, zmm11, zmm15 + vpxord zmm4, zmm4, zmm8 + vpxord zmm5, zmm5, zmm9 + vpxord zmm6, zmm6, zmm10 + vpxord zmm7, zmm7, zmm11 + vprord zmm4, zmm4, 12 + vprord zmm5, zmm5, 12 + vprord zmm6, zmm6, 12 + vprord zmm7, zmm7, 12 + vpaddd zmm0, zmm0, zmm31 + vpaddd zmm1, zmm1, zmm16 + vpaddd zmm2, zmm2, zmm25 + vpaddd zmm3, zmm3, zmm22 + vpaddd zmm0, zmm0, zmm4 + vpaddd zmm1, zmm1, zmm5 + vpaddd zmm2, zmm2, zmm6 + vpaddd zmm3, zmm3, zmm7 + vpxord zmm12, zmm12, zmm0 + vpxord zmm13, zmm13, zmm1 + vpxord zmm14, zmm14, zmm2 + vpxord zmm15, zmm15, zmm3 + vprord zmm12, zmm12, 8 + vprord zmm13, zmm13, 8 + vprord zmm14, zmm14, 8 + vprord zmm15, zmm15, 8 + vpaddd zmm8, zmm8, zmm12 + vpaddd zmm9, zmm9, zmm13 + vpaddd zmm10, zmm10, zmm14 + vpaddd zmm11, zmm11, zmm15 + vpxord zmm4, zmm4, zmm8 + vpxord zmm5, zmm5, zmm9 + vpxord zmm6, zmm6, zmm10 + vpxord zmm7, zmm7, zmm11 + vprord zmm4, zmm4, 7 + vprord zmm5, zmm5, 7 + vprord zmm6, zmm6, 7 + vprord zmm7, zmm7, 7 + vpaddd zmm0, zmm0, zmm30 + vpaddd zmm1, zmm1, zmm18 + vpaddd zmm2, zmm2, zmm19 + vpaddd zmm3, zmm3, zmm23 + vpaddd zmm0, zmm0, zmm5 + vpaddd zmm1, zmm1, zmm6 + vpaddd zmm2, zmm2, zmm7 + vpaddd zmm3, zmm3, zmm4 + vpxord zmm15, zmm15, zmm0 + vpxord zmm12, zmm12, zmm1 + vpxord zmm13, zmm13, zmm2 + vpxord zmm14, zmm14, zmm3 + vprord zmm15, zmm15, 16 + vprord zmm12, zmm12, 16 + vprord zmm13, zmm13, 16 + vprord zmm14, zmm14, 16 + vpaddd zmm10, zmm10, zmm15 + vpaddd zmm11, zmm11, zmm12 + vpaddd zmm8, zmm8, zmm13 + vpaddd zmm9, zmm9, zmm14 + vpxord zmm5, zmm5, zmm10 + vpxord zmm6, zmm6, zmm11 + vpxord zmm7, zmm7, zmm8 + vpxord zmm4, zmm4, zmm9 + vprord zmm5, zmm5, 12 + vprord zmm6, zmm6, 12 + vprord zmm7, zmm7, 12 + vprord zmm4, zmm4, 12 + vpaddd zmm0, zmm0, zmm26 + vpaddd zmm1, zmm1, zmm28 + vpaddd zmm2, zmm2, zmm20 + vpaddd zmm3, zmm3, zmm29 + vpaddd zmm0, zmm0, zmm5 + vpaddd zmm1, zmm1, zmm6 + vpaddd zmm2, zmm2, zmm7 + vpaddd zmm3, zmm3, zmm4 + vpxord zmm15, zmm15, zmm0 + vpxord zmm12, zmm12, zmm1 + vpxord zmm13, zmm13, zmm2 + vpxord zmm14, zmm14, zmm3 + vprord zmm15, zmm15, 8 + vprord zmm12, zmm12, 8 + vprord zmm13, zmm13, 8 + vprord zmm14, zmm14, 8 + vpaddd zmm10, zmm10, zmm15 + vpaddd zmm11, zmm11, zmm12 + vpaddd zmm8, zmm8, zmm13 + vpaddd zmm9, zmm9, zmm14 + vpxord zmm5, zmm5, zmm10 + vpxord zmm6, zmm6, zmm11 + vpxord zmm7, zmm7, zmm8 + vpxord zmm4, zmm4, zmm9 + vprord zmm5, zmm5, 7 + vprord zmm6, zmm6, 7 + vprord zmm7, zmm7, 7 + vprord zmm4, zmm4, 7 + vpxord zmm0, zmm0, zmm8 + vpxord zmm1, zmm1, zmm9 + vpxord zmm2, zmm2, zmm10 + vpxord zmm3, zmm3, zmm11 + vpxord zmm4, zmm4, zmm12 + vpxord zmm5, zmm5, zmm13 + vpxord zmm6, zmm6, zmm14 + vpxord zmm7, zmm7, zmm15 + movzx eax, byte ptr [rbp+0x38] + jne 9b + mov rbx, qword ptr [rbp+0x50] + vpunpckldq zmm16, zmm0, zmm1 + vpunpckhdq zmm17, zmm0, zmm1 + vpunpckldq zmm18, zmm2, zmm3 + vpunpckhdq zmm19, zmm2, zmm3 + vpunpckldq zmm20, zmm4, zmm5 + vpunpckhdq zmm21, zmm4, zmm5 + vpunpckldq zmm22, zmm6, zmm7 + vpunpckhdq zmm23, zmm6, zmm7 + vpunpcklqdq zmm0, zmm16, zmm18 + vpunpckhqdq zmm1, zmm16, zmm18 + vpunpcklqdq zmm2, zmm17, zmm19 + vpunpckhqdq zmm3, zmm17, zmm19 + vpunpcklqdq zmm4, zmm20, zmm22 + vpunpckhqdq zmm5, zmm20, zmm22 + vpunpcklqdq zmm6, zmm21, zmm23 + vpunpckhqdq zmm7, zmm21, zmm23 + vshufi32x4 zmm16, zmm0, zmm4, 0x88 + vshufi32x4 zmm17, zmm1, zmm5, 0x88 + vshufi32x4 zmm18, zmm2, zmm6, 0x88 + vshufi32x4 zmm19, zmm3, zmm7, 0x88 + vshufi32x4 zmm20, zmm0, zmm4, 0xDD + vshufi32x4 zmm21, zmm1, zmm5, 0xDD + vshufi32x4 zmm22, zmm2, zmm6, 0xDD + vshufi32x4 zmm23, zmm3, zmm7, 0xDD + vshufi32x4 zmm0, zmm16, zmm17, 0x88 + vshufi32x4 zmm1, zmm18, zmm19, 0x88 + vshufi32x4 zmm2, zmm20, zmm21, 0x88 + vshufi32x4 zmm3, zmm22, zmm23, 0x88 + vshufi32x4 zmm4, zmm16, zmm17, 0xDD + vshufi32x4 zmm5, zmm18, zmm19, 0xDD + vshufi32x4 zmm6, zmm20, zmm21, 0xDD + vshufi32x4 zmm7, zmm22, zmm23, 0xDD + vmovdqu32 zmmword ptr [rbx], zmm0 + vmovdqu32 zmmword ptr [rbx+0x1*0x40], zmm1 + vmovdqu32 zmmword ptr [rbx+0x2*0x40], zmm2 + vmovdqu32 zmmword ptr [rbx+0x3*0x40], zmm3 + vmovdqu32 zmmword ptr [rbx+0x4*0x40], zmm4 + vmovdqu32 zmmword ptr [rbx+0x5*0x40], zmm5 + vmovdqu32 zmmword ptr [rbx+0x6*0x40], zmm6 + vmovdqu32 zmmword ptr [rbx+0x7*0x40], zmm7 + vmovdqa32 zmm0, zmmword ptr [rsp] + vmovdqa32 zmm1, zmmword ptr [rsp+0x1*0x40] + vmovdqa32 zmm2, zmm0 + vpaddd zmm2{k1}, zmm0, dword ptr [ADD16+rip] {1to16} + vpcmpltud k2, zmm2, zmm0 + vpaddd zmm1 {k2}, zmm1, dword ptr [ADD1+rip] {1to16} + vmovdqa32 zmmword ptr [rsp], zmm2 + vmovdqa32 zmmword ptr [rsp+0x1*0x40], zmm1 + add rdi, 128 + add rbx, 512 + mov qword ptr [rbp+0x50], rbx + sub rsi, 16 + cmp rsi, 16 + jnc 2b + test rsi, rsi + jnz 3f +4: + vzeroupper + mov rsp, rbp + pop rbp + pop rbx + pop r12 + pop r13 + pop r14 + pop r15 + RET +.p2align 6 +3: + test esi, 0x8 + je 3f + vpbroadcastd ymm0, dword ptr [rcx] + vpbroadcastd ymm1, dword ptr [rcx+0x4] + vpbroadcastd ymm2, dword ptr [rcx+0x8] + vpbroadcastd ymm3, dword ptr [rcx+0xC] + vpbroadcastd ymm4, dword ptr [rcx+0x10] + vpbroadcastd ymm5, dword ptr [rcx+0x14] + vpbroadcastd ymm6, dword ptr [rcx+0x18] + vpbroadcastd ymm7, dword ptr [rcx+0x1C] + mov r8, qword ptr [rdi] + mov r9, qword ptr [rdi+0x8] + mov r10, qword ptr [rdi+0x10] + mov r11, qword ptr [rdi+0x18] + mov r12, qword ptr [rdi+0x20] + mov r13, qword ptr [rdi+0x28] + mov r14, qword ptr [rdi+0x30] + mov r15, qword ptr [rdi+0x38] + movzx eax, byte ptr [rbp+0x38] + movzx ebx, byte ptr [rbp+0x40] + or eax, ebx + xor edx, edx +2: + movzx ebx, byte ptr [rbp+0x48] + or ebx, eax + add rdx, 64 + cmp rdx, qword ptr [rsp+0x80] + cmove eax, ebx + mov dword ptr [rsp+0x88], eax + vmovups xmm8, xmmword ptr [r8+rdx-0x40] + vinsertf128 ymm8, ymm8, xmmword ptr [r12+rdx-0x40], 0x01 + vmovups xmm9, xmmword ptr [r9+rdx-0x40] + vinsertf128 ymm9, ymm9, xmmword ptr [r13+rdx-0x40], 0x01 + vunpcklpd ymm12, ymm8, ymm9 + vunpckhpd ymm13, ymm8, ymm9 + vmovups xmm10, xmmword ptr [r10+rdx-0x40] + vinsertf128 ymm10, ymm10, xmmword ptr [r14+rdx-0x40], 0x01 + vmovups xmm11, xmmword ptr [r11+rdx-0x40] + vinsertf128 ymm11, ymm11, xmmword ptr [r15+rdx-0x40], 0x01 + vunpcklpd ymm14, ymm10, ymm11 + vunpckhpd ymm15, ymm10, ymm11 + vshufps ymm16, ymm12, ymm14, 136 + vshufps ymm17, ymm12, ymm14, 221 + vshufps ymm18, ymm13, ymm15, 136 + vshufps ymm19, ymm13, ymm15, 221 + vmovups xmm8, xmmword ptr [r8+rdx-0x30] + vinsertf128 ymm8, ymm8, xmmword ptr [r12+rdx-0x30], 0x01 + vmovups xmm9, xmmword ptr [r9+rdx-0x30] + vinsertf128 ymm9, ymm9, xmmword ptr [r13+rdx-0x30], 0x01 + vunpcklpd ymm12, ymm8, ymm9 + vunpckhpd ymm13, ymm8, ymm9 + vmovups xmm10, xmmword ptr [r10+rdx-0x30] + vinsertf128 ymm10, ymm10, xmmword ptr [r14+rdx-0x30], 0x01 + vmovups xmm11, xmmword ptr [r11+rdx-0x30] + vinsertf128 ymm11, ymm11, xmmword ptr [r15+rdx-0x30], 0x01 + vunpcklpd ymm14, ymm10, ymm11 + vunpckhpd ymm15, ymm10, ymm11 + vshufps ymm20, ymm12, ymm14, 136 + vshufps ymm21, ymm12, ymm14, 221 + vshufps ymm22, ymm13, ymm15, 136 + vshufps ymm23, ymm13, ymm15, 221 + vmovups xmm8, xmmword ptr [r8+rdx-0x20] + vinsertf128 ymm8, ymm8, xmmword ptr [r12+rdx-0x20], 0x01 + vmovups xmm9, xmmword ptr [r9+rdx-0x20] + vinsertf128 ymm9, ymm9, xmmword ptr [r13+rdx-0x20], 0x01 + vunpcklpd ymm12, ymm8, ymm9 + vunpckhpd ymm13, ymm8, ymm9 + vmovups xmm10, xmmword ptr [r10+rdx-0x20] + vinsertf128 ymm10, ymm10, xmmword ptr [r14+rdx-0x20], 0x01 + vmovups xmm11, xmmword ptr [r11+rdx-0x20] + vinsertf128 ymm11, ymm11, xmmword ptr [r15+rdx-0x20], 0x01 + vunpcklpd ymm14, ymm10, ymm11 + vunpckhpd ymm15, ymm10, ymm11 + vshufps ymm24, ymm12, ymm14, 136 + vshufps ymm25, ymm12, ymm14, 221 + vshufps ymm26, ymm13, ymm15, 136 + vshufps ymm27, ymm13, ymm15, 221 + vmovups xmm8, xmmword ptr [r8+rdx-0x10] + vinsertf128 ymm8, ymm8, xmmword ptr [r12+rdx-0x10], 0x01 + vmovups xmm9, xmmword ptr [r9+rdx-0x10] + vinsertf128 ymm9, ymm9, xmmword ptr [r13+rdx-0x10], 0x01 + vunpcklpd ymm12, ymm8, ymm9 + vunpckhpd ymm13, ymm8, ymm9 + vmovups xmm10, xmmword ptr [r10+rdx-0x10] + vinsertf128 ymm10, ymm10, xmmword ptr [r14+rdx-0x10], 0x01 + vmovups xmm11, xmmword ptr [r11+rdx-0x10] + vinsertf128 ymm11, ymm11, xmmword ptr [r15+rdx-0x10], 0x01 + vunpcklpd ymm14, ymm10, ymm11 + vunpckhpd ymm15, ymm10, ymm11 + vshufps ymm28, ymm12, ymm14, 136 + vshufps ymm29, ymm12, ymm14, 221 + vshufps ymm30, ymm13, ymm15, 136 + vshufps ymm31, ymm13, ymm15, 221 + vpbroadcastd ymm8, dword ptr [BLAKE3_IV_0+rip] + vpbroadcastd ymm9, dword ptr [BLAKE3_IV_1+rip] + vpbroadcastd ymm10, dword ptr [BLAKE3_IV_2+rip] + vpbroadcastd ymm11, dword ptr [BLAKE3_IV_3+rip] + vmovdqa ymm12, ymmword ptr [rsp] + vmovdqa ymm13, ymmword ptr [rsp+0x40] + vpbroadcastd ymm14, dword ptr [BLAKE3_BLOCK_LEN+rip] + vpbroadcastd ymm15, dword ptr [rsp+0x88] + vpaddd ymm0, ymm0, ymm16 + vpaddd ymm1, ymm1, ymm18 + vpaddd ymm2, ymm2, ymm20 + vpaddd ymm3, ymm3, ymm22 + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxord ymm12, ymm12, ymm0 + vpxord ymm13, ymm13, ymm1 + vpxord ymm14, ymm14, ymm2 + vpxord ymm15, ymm15, ymm3 + vprord ymm12, ymm12, 16 + vprord ymm13, ymm13, 16 + vprord ymm14, ymm14, 16 + vprord ymm15, ymm15, 16 + vpaddd ymm8, ymm8, ymm12 + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxord ymm4, ymm4, ymm8 + vpxord ymm5, ymm5, ymm9 + vpxord ymm6, ymm6, ymm10 + vpxord ymm7, ymm7, ymm11 + vprord ymm4, ymm4, 12 + vprord ymm5, ymm5, 12 + vprord ymm6, ymm6, 12 + vprord ymm7, ymm7, 12 + vpaddd ymm0, ymm0, ymm17 + vpaddd ymm1, ymm1, ymm19 + vpaddd ymm2, ymm2, ymm21 + vpaddd ymm3, ymm3, ymm23 + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxord ymm12, ymm12, ymm0 + vpxord ymm13, ymm13, ymm1 + vpxord ymm14, ymm14, ymm2 + vpxord ymm15, ymm15, ymm3 + vprord ymm12, ymm12, 8 + vprord ymm13, ymm13, 8 + vprord ymm14, ymm14, 8 + vprord ymm15, ymm15, 8 + vpaddd ymm8, ymm8, ymm12 + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxord ymm4, ymm4, ymm8 + vpxord ymm5, ymm5, ymm9 + vpxord ymm6, ymm6, ymm10 + vpxord ymm7, ymm7, ymm11 + vprord ymm4, ymm4, 7 + vprord ymm5, ymm5, 7 + vprord ymm6, ymm6, 7 + vprord ymm7, ymm7, 7 + vpaddd ymm0, ymm0, ymm24 + vpaddd ymm1, ymm1, ymm26 + vpaddd ymm2, ymm2, ymm28 + vpaddd ymm3, ymm3, ymm30 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxord ymm15, ymm15, ymm0 + vpxord ymm12, ymm12, ymm1 + vpxord ymm13, ymm13, ymm2 + vpxord ymm14, ymm14, ymm3 + vprord ymm15, ymm15, 16 + vprord ymm12, ymm12, 16 + vprord ymm13, ymm13, 16 + vprord ymm14, ymm14, 16 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm8, ymm13 + vpaddd ymm9, ymm9, ymm14 + vpxord ymm5, ymm5, ymm10 + vpxord ymm6, ymm6, ymm11 + vpxord ymm7, ymm7, ymm8 + vpxord ymm4, ymm4, ymm9 + vprord ymm5, ymm5, 12 + vprord ymm6, ymm6, 12 + vprord ymm7, ymm7, 12 + vprord ymm4, ymm4, 12 + vpaddd ymm0, ymm0, ymm25 + vpaddd ymm1, ymm1, ymm27 + vpaddd ymm2, ymm2, ymm29 + vpaddd ymm3, ymm3, ymm31 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxord ymm15, ymm15, ymm0 + vpxord ymm12, ymm12, ymm1 + vpxord ymm13, ymm13, ymm2 + vpxord ymm14, ymm14, ymm3 + vprord ymm15, ymm15, 8 + vprord ymm12, ymm12, 8 + vprord ymm13, ymm13, 8 + vprord ymm14, ymm14, 8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm8, ymm13 + vpaddd ymm9, ymm9, ymm14 + vpxord ymm5, ymm5, ymm10 + vpxord ymm6, ymm6, ymm11 + vpxord ymm7, ymm7, ymm8 + vpxord ymm4, ymm4, ymm9 + vprord ymm5, ymm5, 7 + vprord ymm6, ymm6, 7 + vprord ymm7, ymm7, 7 + vprord ymm4, ymm4, 7 + vpaddd ymm0, ymm0, ymm18 + vpaddd ymm1, ymm1, ymm19 + vpaddd ymm2, ymm2, ymm23 + vpaddd ymm3, ymm3, ymm20 + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxord ymm12, ymm12, ymm0 + vpxord ymm13, ymm13, ymm1 + vpxord ymm14, ymm14, ymm2 + vpxord ymm15, ymm15, ymm3 + vprord ymm12, ymm12, 16 + vprord ymm13, ymm13, 16 + vprord ymm14, ymm14, 16 + vprord ymm15, ymm15, 16 + vpaddd ymm8, ymm8, ymm12 + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxord ymm4, ymm4, ymm8 + vpxord ymm5, ymm5, ymm9 + vpxord ymm6, ymm6, ymm10 + vpxord ymm7, ymm7, ymm11 + vprord ymm4, ymm4, 12 + vprord ymm5, ymm5, 12 + vprord ymm6, ymm6, 12 + vprord ymm7, ymm7, 12 + vpaddd ymm0, ymm0, ymm22 + vpaddd ymm1, ymm1, ymm26 + vpaddd ymm2, ymm2, ymm16 + vpaddd ymm3, ymm3, ymm29 + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxord ymm12, ymm12, ymm0 + vpxord ymm13, ymm13, ymm1 + vpxord ymm14, ymm14, ymm2 + vpxord ymm15, ymm15, ymm3 + vprord ymm12, ymm12, 8 + vprord ymm13, ymm13, 8 + vprord ymm14, ymm14, 8 + vprord ymm15, ymm15, 8 + vpaddd ymm8, ymm8, ymm12 + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxord ymm4, ymm4, ymm8 + vpxord ymm5, ymm5, ymm9 + vpxord ymm6, ymm6, ymm10 + vpxord ymm7, ymm7, ymm11 + vprord ymm4, ymm4, 7 + vprord ymm5, ymm5, 7 + vprord ymm6, ymm6, 7 + vprord ymm7, ymm7, 7 + vpaddd ymm0, ymm0, ymm17 + vpaddd ymm1, ymm1, ymm28 + vpaddd ymm2, ymm2, ymm25 + vpaddd ymm3, ymm3, ymm31 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxord ymm15, ymm15, ymm0 + vpxord ymm12, ymm12, ymm1 + vpxord ymm13, ymm13, ymm2 + vpxord ymm14, ymm14, ymm3 + vprord ymm15, ymm15, 16 + vprord ymm12, ymm12, 16 + vprord ymm13, ymm13, 16 + vprord ymm14, ymm14, 16 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm8, ymm13 + vpaddd ymm9, ymm9, ymm14 + vpxord ymm5, ymm5, ymm10 + vpxord ymm6, ymm6, ymm11 + vpxord ymm7, ymm7, ymm8 + vpxord ymm4, ymm4, ymm9 + vprord ymm5, ymm5, 12 + vprord ymm6, ymm6, 12 + vprord ymm7, ymm7, 12 + vprord ymm4, ymm4, 12 + vpaddd ymm0, ymm0, ymm27 + vpaddd ymm1, ymm1, ymm21 + vpaddd ymm2, ymm2, ymm30 + vpaddd ymm3, ymm3, ymm24 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxord ymm15, ymm15, ymm0 + vpxord ymm12, ymm12, ymm1 + vpxord ymm13, ymm13, ymm2 + vpxord ymm14, ymm14, ymm3 + vprord ymm15, ymm15, 8 + vprord ymm12, ymm12, 8 + vprord ymm13, ymm13, 8 + vprord ymm14, ymm14, 8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm8, ymm13 + vpaddd ymm9, ymm9, ymm14 + vpxord ymm5, ymm5, ymm10 + vpxord ymm6, ymm6, ymm11 + vpxord ymm7, ymm7, ymm8 + vpxord ymm4, ymm4, ymm9 + vprord ymm5, ymm5, 7 + vprord ymm6, ymm6, 7 + vprord ymm7, ymm7, 7 + vprord ymm4, ymm4, 7 + vpaddd ymm0, ymm0, ymm19 + vpaddd ymm1, ymm1, ymm26 + vpaddd ymm2, ymm2, ymm29 + vpaddd ymm3, ymm3, ymm23 + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxord ymm12, ymm12, ymm0 + vpxord ymm13, ymm13, ymm1 + vpxord ymm14, ymm14, ymm2 + vpxord ymm15, ymm15, ymm3 + vprord ymm12, ymm12, 16 + vprord ymm13, ymm13, 16 + vprord ymm14, ymm14, 16 + vprord ymm15, ymm15, 16 + vpaddd ymm8, ymm8, ymm12 + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxord ymm4, ymm4, ymm8 + vpxord ymm5, ymm5, ymm9 + vpxord ymm6, ymm6, ymm10 + vpxord ymm7, ymm7, ymm11 + vprord ymm4, ymm4, 12 + vprord ymm5, ymm5, 12 + vprord ymm6, ymm6, 12 + vprord ymm7, ymm7, 12 + vpaddd ymm0, ymm0, ymm20 + vpaddd ymm1, ymm1, ymm28 + vpaddd ymm2, ymm2, ymm18 + vpaddd ymm3, ymm3, ymm30 + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxord ymm12, ymm12, ymm0 + vpxord ymm13, ymm13, ymm1 + vpxord ymm14, ymm14, ymm2 + vpxord ymm15, ymm15, ymm3 + vprord ymm12, ymm12, 8 + vprord ymm13, ymm13, 8 + vprord ymm14, ymm14, 8 + vprord ymm15, ymm15, 8 + vpaddd ymm8, ymm8, ymm12 + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxord ymm4, ymm4, ymm8 + vpxord ymm5, ymm5, ymm9 + vpxord ymm6, ymm6, ymm10 + vpxord ymm7, ymm7, ymm11 + vprord ymm4, ymm4, 7 + vprord ymm5, ymm5, 7 + vprord ymm6, ymm6, 7 + vprord ymm7, ymm7, 7 + vpaddd ymm0, ymm0, ymm22 + vpaddd ymm1, ymm1, ymm25 + vpaddd ymm2, ymm2, ymm27 + vpaddd ymm3, ymm3, ymm24 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxord ymm15, ymm15, ymm0 + vpxord ymm12, ymm12, ymm1 + vpxord ymm13, ymm13, ymm2 + vpxord ymm14, ymm14, ymm3 + vprord ymm15, ymm15, 16 + vprord ymm12, ymm12, 16 + vprord ymm13, ymm13, 16 + vprord ymm14, ymm14, 16 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm8, ymm13 + vpaddd ymm9, ymm9, ymm14 + vpxord ymm5, ymm5, ymm10 + vpxord ymm6, ymm6, ymm11 + vpxord ymm7, ymm7, ymm8 + vpxord ymm4, ymm4, ymm9 + vprord ymm5, ymm5, 12 + vprord ymm6, ymm6, 12 + vprord ymm7, ymm7, 12 + vprord ymm4, ymm4, 12 + vpaddd ymm0, ymm0, ymm21 + vpaddd ymm1, ymm1, ymm16 + vpaddd ymm2, ymm2, ymm31 + vpaddd ymm3, ymm3, ymm17 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxord ymm15, ymm15, ymm0 + vpxord ymm12, ymm12, ymm1 + vpxord ymm13, ymm13, ymm2 + vpxord ymm14, ymm14, ymm3 + vprord ymm15, ymm15, 8 + vprord ymm12, ymm12, 8 + vprord ymm13, ymm13, 8 + vprord ymm14, ymm14, 8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm8, ymm13 + vpaddd ymm9, ymm9, ymm14 + vpxord ymm5, ymm5, ymm10 + vpxord ymm6, ymm6, ymm11 + vpxord ymm7, ymm7, ymm8 + vpxord ymm4, ymm4, ymm9 + vprord ymm5, ymm5, 7 + vprord ymm6, ymm6, 7 + vprord ymm7, ymm7, 7 + vprord ymm4, ymm4, 7 + vpaddd ymm0, ymm0, ymm26 + vpaddd ymm1, ymm1, ymm28 + vpaddd ymm2, ymm2, ymm30 + vpaddd ymm3, ymm3, ymm29 + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxord ymm12, ymm12, ymm0 + vpxord ymm13, ymm13, ymm1 + vpxord ymm14, ymm14, ymm2 + vpxord ymm15, ymm15, ymm3 + vprord ymm12, ymm12, 16 + vprord ymm13, ymm13, 16 + vprord ymm14, ymm14, 16 + vprord ymm15, ymm15, 16 + vpaddd ymm8, ymm8, ymm12 + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxord ymm4, ymm4, ymm8 + vpxord ymm5, ymm5, ymm9 + vpxord ymm6, ymm6, ymm10 + vpxord ymm7, ymm7, ymm11 + vprord ymm4, ymm4, 12 + vprord ymm5, ymm5, 12 + vprord ymm6, ymm6, 12 + vprord ymm7, ymm7, 12 + vpaddd ymm0, ymm0, ymm23 + vpaddd ymm1, ymm1, ymm25 + vpaddd ymm2, ymm2, ymm19 + vpaddd ymm3, ymm3, ymm31 + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxord ymm12, ymm12, ymm0 + vpxord ymm13, ymm13, ymm1 + vpxord ymm14, ymm14, ymm2 + vpxord ymm15, ymm15, ymm3 + vprord ymm12, ymm12, 8 + vprord ymm13, ymm13, 8 + vprord ymm14, ymm14, 8 + vprord ymm15, ymm15, 8 + vpaddd ymm8, ymm8, ymm12 + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxord ymm4, ymm4, ymm8 + vpxord ymm5, ymm5, ymm9 + vpxord ymm6, ymm6, ymm10 + vpxord ymm7, ymm7, ymm11 + vprord ymm4, ymm4, 7 + vprord ymm5, ymm5, 7 + vprord ymm6, ymm6, 7 + vprord ymm7, ymm7, 7 + vpaddd ymm0, ymm0, ymm20 + vpaddd ymm1, ymm1, ymm27 + vpaddd ymm2, ymm2, ymm21 + vpaddd ymm3, ymm3, ymm17 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxord ymm15, ymm15, ymm0 + vpxord ymm12, ymm12, ymm1 + vpxord ymm13, ymm13, ymm2 + vpxord ymm14, ymm14, ymm3 + vprord ymm15, ymm15, 16 + vprord ymm12, ymm12, 16 + vprord ymm13, ymm13, 16 + vprord ymm14, ymm14, 16 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm8, ymm13 + vpaddd ymm9, ymm9, ymm14 + vpxord ymm5, ymm5, ymm10 + vpxord ymm6, ymm6, ymm11 + vpxord ymm7, ymm7, ymm8 + vpxord ymm4, ymm4, ymm9 + vprord ymm5, ymm5, 12 + vprord ymm6, ymm6, 12 + vprord ymm7, ymm7, 12 + vprord ymm4, ymm4, 12 + vpaddd ymm0, ymm0, ymm16 + vpaddd ymm1, ymm1, ymm18 + vpaddd ymm2, ymm2, ymm24 + vpaddd ymm3, ymm3, ymm22 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxord ymm15, ymm15, ymm0 + vpxord ymm12, ymm12, ymm1 + vpxord ymm13, ymm13, ymm2 + vpxord ymm14, ymm14, ymm3 + vprord ymm15, ymm15, 8 + vprord ymm12, ymm12, 8 + vprord ymm13, ymm13, 8 + vprord ymm14, ymm14, 8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm8, ymm13 + vpaddd ymm9, ymm9, ymm14 + vpxord ymm5, ymm5, ymm10 + vpxord ymm6, ymm6, ymm11 + vpxord ymm7, ymm7, ymm8 + vpxord ymm4, ymm4, ymm9 + vprord ymm5, ymm5, 7 + vprord ymm6, ymm6, 7 + vprord ymm7, ymm7, 7 + vprord ymm4, ymm4, 7 + vpaddd ymm0, ymm0, ymm28 + vpaddd ymm1, ymm1, ymm25 + vpaddd ymm2, ymm2, ymm31 + vpaddd ymm3, ymm3, ymm30 + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxord ymm12, ymm12, ymm0 + vpxord ymm13, ymm13, ymm1 + vpxord ymm14, ymm14, ymm2 + vpxord ymm15, ymm15, ymm3 + vprord ymm12, ymm12, 16 + vprord ymm13, ymm13, 16 + vprord ymm14, ymm14, 16 + vprord ymm15, ymm15, 16 + vpaddd ymm8, ymm8, ymm12 + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxord ymm4, ymm4, ymm8 + vpxord ymm5, ymm5, ymm9 + vpxord ymm6, ymm6, ymm10 + vpxord ymm7, ymm7, ymm11 + vprord ymm4, ymm4, 12 + vprord ymm5, ymm5, 12 + vprord ymm6, ymm6, 12 + vprord ymm7, ymm7, 12 + vpaddd ymm0, ymm0, ymm29 + vpaddd ymm1, ymm1, ymm27 + vpaddd ymm2, ymm2, ymm26 + vpaddd ymm3, ymm3, ymm24 + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxord ymm12, ymm12, ymm0 + vpxord ymm13, ymm13, ymm1 + vpxord ymm14, ymm14, ymm2 + vpxord ymm15, ymm15, ymm3 + vprord ymm12, ymm12, 8 + vprord ymm13, ymm13, 8 + vprord ymm14, ymm14, 8 + vprord ymm15, ymm15, 8 + vpaddd ymm8, ymm8, ymm12 + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxord ymm4, ymm4, ymm8 + vpxord ymm5, ymm5, ymm9 + vpxord ymm6, ymm6, ymm10 + vpxord ymm7, ymm7, ymm11 + vprord ymm4, ymm4, 7 + vprord ymm5, ymm5, 7 + vprord ymm6, ymm6, 7 + vprord ymm7, ymm7, 7 + vpaddd ymm0, ymm0, ymm23 + vpaddd ymm1, ymm1, ymm21 + vpaddd ymm2, ymm2, ymm16 + vpaddd ymm3, ymm3, ymm22 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxord ymm15, ymm15, ymm0 + vpxord ymm12, ymm12, ymm1 + vpxord ymm13, ymm13, ymm2 + vpxord ymm14, ymm14, ymm3 + vprord ymm15, ymm15, 16 + vprord ymm12, ymm12, 16 + vprord ymm13, ymm13, 16 + vprord ymm14, ymm14, 16 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm8, ymm13 + vpaddd ymm9, ymm9, ymm14 + vpxord ymm5, ymm5, ymm10 + vpxord ymm6, ymm6, ymm11 + vpxord ymm7, ymm7, ymm8 + vpxord ymm4, ymm4, ymm9 + vprord ymm5, ymm5, 12 + vprord ymm6, ymm6, 12 + vprord ymm7, ymm7, 12 + vprord ymm4, ymm4, 12 + vpaddd ymm0, ymm0, ymm18 + vpaddd ymm1, ymm1, ymm19 + vpaddd ymm2, ymm2, ymm17 + vpaddd ymm3, ymm3, ymm20 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxord ymm15, ymm15, ymm0 + vpxord ymm12, ymm12, ymm1 + vpxord ymm13, ymm13, ymm2 + vpxord ymm14, ymm14, ymm3 + vprord ymm15, ymm15, 8 + vprord ymm12, ymm12, 8 + vprord ymm13, ymm13, 8 + vprord ymm14, ymm14, 8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm8, ymm13 + vpaddd ymm9, ymm9, ymm14 + vpxord ymm5, ymm5, ymm10 + vpxord ymm6, ymm6, ymm11 + vpxord ymm7, ymm7, ymm8 + vpxord ymm4, ymm4, ymm9 + vprord ymm5, ymm5, 7 + vprord ymm6, ymm6, 7 + vprord ymm7, ymm7, 7 + vprord ymm4, ymm4, 7 + vpaddd ymm0, ymm0, ymm25 + vpaddd ymm1, ymm1, ymm27 + vpaddd ymm2, ymm2, ymm24 + vpaddd ymm3, ymm3, ymm31 + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxord ymm12, ymm12, ymm0 + vpxord ymm13, ymm13, ymm1 + vpxord ymm14, ymm14, ymm2 + vpxord ymm15, ymm15, ymm3 + vprord ymm12, ymm12, 16 + vprord ymm13, ymm13, 16 + vprord ymm14, ymm14, 16 + vprord ymm15, ymm15, 16 + vpaddd ymm8, ymm8, ymm12 + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxord ymm4, ymm4, ymm8 + vpxord ymm5, ymm5, ymm9 + vpxord ymm6, ymm6, ymm10 + vpxord ymm7, ymm7, ymm11 + vprord ymm4, ymm4, 12 + vprord ymm5, ymm5, 12 + vprord ymm6, ymm6, 12 + vprord ymm7, ymm7, 12 + vpaddd ymm0, ymm0, ymm30 + vpaddd ymm1, ymm1, ymm21 + vpaddd ymm2, ymm2, ymm28 + vpaddd ymm3, ymm3, ymm17 + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxord ymm12, ymm12, ymm0 + vpxord ymm13, ymm13, ymm1 + vpxord ymm14, ymm14, ymm2 + vpxord ymm15, ymm15, ymm3 + vprord ymm12, ymm12, 8 + vprord ymm13, ymm13, 8 + vprord ymm14, ymm14, 8 + vprord ymm15, ymm15, 8 + vpaddd ymm8, ymm8, ymm12 + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxord ymm4, ymm4, ymm8 + vpxord ymm5, ymm5, ymm9 + vpxord ymm6, ymm6, ymm10 + vpxord ymm7, ymm7, ymm11 + vprord ymm4, ymm4, 7 + vprord ymm5, ymm5, 7 + vprord ymm6, ymm6, 7 + vprord ymm7, ymm7, 7 + vpaddd ymm0, ymm0, ymm29 + vpaddd ymm1, ymm1, ymm16 + vpaddd ymm2, ymm2, ymm18 + vpaddd ymm3, ymm3, ymm20 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxord ymm15, ymm15, ymm0 + vpxord ymm12, ymm12, ymm1 + vpxord ymm13, ymm13, ymm2 + vpxord ymm14, ymm14, ymm3 + vprord ymm15, ymm15, 16 + vprord ymm12, ymm12, 16 + vprord ymm13, ymm13, 16 + vprord ymm14, ymm14, 16 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm8, ymm13 + vpaddd ymm9, ymm9, ymm14 + vpxord ymm5, ymm5, ymm10 + vpxord ymm6, ymm6, ymm11 + vpxord ymm7, ymm7, ymm8 + vpxord ymm4, ymm4, ymm9 + vprord ymm5, ymm5, 12 + vprord ymm6, ymm6, 12 + vprord ymm7, ymm7, 12 + vprord ymm4, ymm4, 12 + vpaddd ymm0, ymm0, ymm19 + vpaddd ymm1, ymm1, ymm26 + vpaddd ymm2, ymm2, ymm22 + vpaddd ymm3, ymm3, ymm23 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxord ymm15, ymm15, ymm0 + vpxord ymm12, ymm12, ymm1 + vpxord ymm13, ymm13, ymm2 + vpxord ymm14, ymm14, ymm3 + vprord ymm15, ymm15, 8 + vprord ymm12, ymm12, 8 + vprord ymm13, ymm13, 8 + vprord ymm14, ymm14, 8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm8, ymm13 + vpaddd ymm9, ymm9, ymm14 + vpxord ymm5, ymm5, ymm10 + vpxord ymm6, ymm6, ymm11 + vpxord ymm7, ymm7, ymm8 + vpxord ymm4, ymm4, ymm9 + vprord ymm5, ymm5, 7 + vprord ymm6, ymm6, 7 + vprord ymm7, ymm7, 7 + vprord ymm4, ymm4, 7 + vpaddd ymm0, ymm0, ymm27 + vpaddd ymm1, ymm1, ymm21 + vpaddd ymm2, ymm2, ymm17 + vpaddd ymm3, ymm3, ymm24 + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxord ymm12, ymm12, ymm0 + vpxord ymm13, ymm13, ymm1 + vpxord ymm14, ymm14, ymm2 + vpxord ymm15, ymm15, ymm3 + vprord ymm12, ymm12, 16 + vprord ymm13, ymm13, 16 + vprord ymm14, ymm14, 16 + vprord ymm15, ymm15, 16 + vpaddd ymm8, ymm8, ymm12 + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxord ymm4, ymm4, ymm8 + vpxord ymm5, ymm5, ymm9 + vpxord ymm6, ymm6, ymm10 + vpxord ymm7, ymm7, ymm11 + vprord ymm4, ymm4, 12 + vprord ymm5, ymm5, 12 + vprord ymm6, ymm6, 12 + vprord ymm7, ymm7, 12 + vpaddd ymm0, ymm0, ymm31 + vpaddd ymm1, ymm1, ymm16 + vpaddd ymm2, ymm2, ymm25 + vpaddd ymm3, ymm3, ymm22 + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm1, ymm1, ymm5 + vpaddd ymm2, ymm2, ymm6 + vpaddd ymm3, ymm3, ymm7 + vpxord ymm12, ymm12, ymm0 + vpxord ymm13, ymm13, ymm1 + vpxord ymm14, ymm14, ymm2 + vpxord ymm15, ymm15, ymm3 + vprord ymm12, ymm12, 8 + vprord ymm13, ymm13, 8 + vprord ymm14, ymm14, 8 + vprord ymm15, ymm15, 8 + vpaddd ymm8, ymm8, ymm12 + vpaddd ymm9, ymm9, ymm13 + vpaddd ymm10, ymm10, ymm14 + vpaddd ymm11, ymm11, ymm15 + vpxord ymm4, ymm4, ymm8 + vpxord ymm5, ymm5, ymm9 + vpxord ymm6, ymm6, ymm10 + vpxord ymm7, ymm7, ymm11 + vprord ymm4, ymm4, 7 + vprord ymm5, ymm5, 7 + vprord ymm6, ymm6, 7 + vprord ymm7, ymm7, 7 + vpaddd ymm0, ymm0, ymm30 + vpaddd ymm1, ymm1, ymm18 + vpaddd ymm2, ymm2, ymm19 + vpaddd ymm3, ymm3, ymm23 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxord ymm15, ymm15, ymm0 + vpxord ymm12, ymm12, ymm1 + vpxord ymm13, ymm13, ymm2 + vpxord ymm14, ymm14, ymm3 + vprord ymm15, ymm15, 16 + vprord ymm12, ymm12, 16 + vprord ymm13, ymm13, 16 + vprord ymm14, ymm14, 16 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm8, ymm13 + vpaddd ymm9, ymm9, ymm14 + vpxord ymm5, ymm5, ymm10 + vpxord ymm6, ymm6, ymm11 + vpxord ymm7, ymm7, ymm8 + vpxord ymm4, ymm4, ymm9 + vprord ymm5, ymm5, 12 + vprord ymm6, ymm6, 12 + vprord ymm7, ymm7, 12 + vprord ymm4, ymm4, 12 + vpaddd ymm0, ymm0, ymm26 + vpaddd ymm1, ymm1, ymm28 + vpaddd ymm2, ymm2, ymm20 + vpaddd ymm3, ymm3, ymm29 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm1, ymm1, ymm6 + vpaddd ymm2, ymm2, ymm7 + vpaddd ymm3, ymm3, ymm4 + vpxord ymm15, ymm15, ymm0 + vpxord ymm12, ymm12, ymm1 + vpxord ymm13, ymm13, ymm2 + vpxord ymm14, ymm14, ymm3 + vprord ymm15, ymm15, 8 + vprord ymm12, ymm12, 8 + vprord ymm13, ymm13, 8 + vprord ymm14, ymm14, 8 + vpaddd ymm10, ymm10, ymm15 + vpaddd ymm11, ymm11, ymm12 + vpaddd ymm8, ymm8, ymm13 + vpaddd ymm9, ymm9, ymm14 + vpxord ymm5, ymm5, ymm10 + vpxord ymm6, ymm6, ymm11 + vpxord ymm7, ymm7, ymm8 + vpxord ymm4, ymm4, ymm9 + vprord ymm5, ymm5, 7 + vprord ymm6, ymm6, 7 + vprord ymm7, ymm7, 7 + vprord ymm4, ymm4, 7 + vpxor ymm0, ymm0, ymm8 + vpxor ymm1, ymm1, ymm9 + vpxor ymm2, ymm2, ymm10 + vpxor ymm3, ymm3, ymm11 + vpxor ymm4, ymm4, ymm12 + vpxor ymm5, ymm5, ymm13 + vpxor ymm6, ymm6, ymm14 + vpxor ymm7, ymm7, ymm15 + movzx eax, byte ptr [rbp+0x38] + jne 2b + mov rbx, qword ptr [rbp+0x50] + vunpcklps ymm8, ymm0, ymm1 + vunpcklps ymm9, ymm2, ymm3 + vunpckhps ymm10, ymm0, ymm1 + vunpcklps ymm11, ymm4, ymm5 + vunpcklps ymm0, ymm6, ymm7 + vshufps ymm12, ymm8, ymm9, 78 + vblendps ymm1, ymm8, ymm12, 0xCC + vshufps ymm8, ymm11, ymm0, 78 + vunpckhps ymm13, ymm2, ymm3 + vblendps ymm2, ymm11, ymm8, 0xCC + vblendps ymm3, ymm12, ymm9, 0xCC + vperm2f128 ymm12, ymm1, ymm2, 0x20 + vmovups ymmword ptr [rbx], ymm12 + vunpckhps ymm14, ymm4, ymm5 + vblendps ymm4, ymm8, ymm0, 0xCC + vunpckhps ymm15, ymm6, ymm7 + vperm2f128 ymm7, ymm3, ymm4, 0x20 + vmovups ymmword ptr [rbx+0x20], ymm7 + vshufps ymm5, ymm10, ymm13, 78 + vblendps ymm6, ymm5, ymm13, 0xCC + vshufps ymm13, ymm14, ymm15, 78 + vblendps ymm10, ymm10, ymm5, 0xCC + vblendps ymm14, ymm14, ymm13, 0xCC + vperm2f128 ymm8, ymm10, ymm14, 0x20 + vmovups ymmword ptr [rbx+0x40], ymm8 + vblendps ymm15, ymm13, ymm15, 0xCC + vperm2f128 ymm13, ymm6, ymm15, 0x20 + vmovups ymmword ptr [rbx+0x60], ymm13 + vperm2f128 ymm9, ymm1, ymm2, 0x31 + vperm2f128 ymm11, ymm3, ymm4, 0x31 + vmovups ymmword ptr [rbx+0x80], ymm9 + vperm2f128 ymm14, ymm10, ymm14, 0x31 + vperm2f128 ymm15, ymm6, ymm15, 0x31 + vmovups ymmword ptr [rbx+0xA0], ymm11 + vmovups ymmword ptr [rbx+0xC0], ymm14 + vmovups ymmword ptr [rbx+0xE0], ymm15 + vmovdqa ymm0, ymmword ptr [rsp] + vmovdqa ymm2, ymmword ptr [rsp+0x2*0x20] + vmovdqa32 ymm0 {k1}, ymmword ptr [rsp+0x1*0x20] + vmovdqa32 ymm2 {k1}, ymmword ptr [rsp+0x3*0x20] + vmovdqa ymmword ptr [rsp], ymm0 + vmovdqa ymmword ptr [rsp+0x2*0x20], ymm2 + add rbx, 256 + mov qword ptr [rbp+0x50], rbx + add rdi, 64 + sub rsi, 8 +3: + mov rbx, qword ptr [rbp+0x50] + mov r15, qword ptr [rsp+0x80] + movzx r13, byte ptr [rbp+0x38] + movzx r12, byte ptr [rbp+0x48] + test esi, 0x4 + je 3f + vbroadcasti32x4 zmm0, xmmword ptr [rcx] + vbroadcasti32x4 zmm1, xmmword ptr [rcx+0x1*0x10] + vmovdqa xmm12, xmmword ptr [rsp] + vmovdqa xmm13, xmmword ptr [rsp+0x4*0x10] + vpunpckldq xmm14, xmm12, xmm13 + vpunpckhdq xmm15, xmm12, xmm13 + vpermq ymm14, ymm14, 0xDC + vpermq ymm15, ymm15, 0xDC + vpbroadcastd zmm12, dword ptr [BLAKE3_BLOCK_LEN+rip] + vinserti32x8 zmm13, zmm14, ymm15, 0x01 + mov eax, 17476 + kmovw k2, eax + vpblendmd zmm13 {k2}, zmm13, zmm12 + vbroadcasti32x4 zmm15, xmmword ptr [BLAKE3_IV+rip] + mov r8, qword ptr [rdi] + mov r9, qword ptr [rdi+0x8] + mov r10, qword ptr [rdi+0x10] + mov r11, qword ptr [rdi+0x18] + mov eax, 43690 + kmovw k3, eax + mov eax, 34952 + kmovw k4, eax + movzx eax, byte ptr [rbp+0x40] + or eax, r13d + xor edx, edx +.p2align 5 +2: + mov r14d, eax + or eax, r12d + add rdx, 64 + cmp rdx, r15 + cmovne eax, r14d + mov dword ptr [rsp+0x88], eax + vmovdqa32 zmm2, zmm15 + vpbroadcastd zmm8, dword ptr [rsp+0x22*0x4] + vpblendmd zmm3 {k4}, zmm13, zmm8 + vmovups zmm8, zmmword ptr [r8+rdx-0x1*0x40] + vinserti32x4 zmm8, zmm8, xmmword ptr [r9+rdx-0x4*0x10], 0x01 + vinserti32x4 zmm8, zmm8, xmmword ptr [r10+rdx-0x4*0x10], 0x02 + vinserti32x4 zmm8, zmm8, xmmword ptr [r11+rdx-0x4*0x10], 0x03 + vmovups zmm9, zmmword ptr [r8+rdx-0x30] + vinserti32x4 zmm9, zmm9, xmmword ptr [r9+rdx-0x3*0x10], 0x01 + vinserti32x4 zmm9, zmm9, xmmword ptr [r10+rdx-0x3*0x10], 0x02 + vinserti32x4 zmm9, zmm9, xmmword ptr [r11+rdx-0x3*0x10], 0x03 + vshufps zmm4, zmm8, zmm9, 136 + vshufps zmm5, zmm8, zmm9, 221 + vmovups zmm8, zmmword ptr [r8+rdx-0x20] + vinserti32x4 zmm8, zmm8, xmmword ptr [r9+rdx-0x2*0x10], 0x01 + vinserti32x4 zmm8, zmm8, xmmword ptr [r10+rdx-0x2*0x10], 0x02 + vinserti32x4 zmm8, zmm8, xmmword ptr [r11+rdx-0x2*0x10], 0x03 + vmovups zmm9, zmmword ptr [r8+rdx-0x10] + vinserti32x4 zmm9, zmm9, xmmword ptr [r9+rdx-0x1*0x10], 0x01 + vinserti32x4 zmm9, zmm9, xmmword ptr [r10+rdx-0x1*0x10], 0x02 + vinserti32x4 zmm9, zmm9, xmmword ptr [r11+rdx-0x1*0x10], 0x03 + vshufps zmm6, zmm8, zmm9, 136 + vshufps zmm7, zmm8, zmm9, 221 + vpshufd zmm6, zmm6, 0x93 + vpshufd zmm7, zmm7, 0x93 + mov al, 7 +9: + vpaddd zmm0, zmm0, zmm4 + vpaddd zmm0, zmm0, zmm1 + vpxord zmm3, zmm3, zmm0 + vprord zmm3, zmm3, 16 + vpaddd zmm2, zmm2, zmm3 + vpxord zmm1, zmm1, zmm2 + vprord zmm1, zmm1, 12 + vpaddd zmm0, zmm0, zmm5 + vpaddd zmm0, zmm0, zmm1 + vpxord zmm3, zmm3, zmm0 + vprord zmm3, zmm3, 8 + vpaddd zmm2, zmm2, zmm3 + vpxord zmm1, zmm1, zmm2 + vprord zmm1, zmm1, 7 + vpshufd zmm0, zmm0, 0x93 + vpshufd zmm3, zmm3, 0x4E + vpshufd zmm2, zmm2, 0x39 + vpaddd zmm0, zmm0, zmm6 + vpaddd zmm0, zmm0, zmm1 + vpxord zmm3, zmm3, zmm0 + vprord zmm3, zmm3, 16 + vpaddd zmm2, zmm2, zmm3 + vpxord zmm1, zmm1, zmm2 + vprord zmm1, zmm1, 12 + vpaddd zmm0, zmm0, zmm7 + vpaddd zmm0, zmm0, zmm1 + vpxord zmm3, zmm3, zmm0 + vprord zmm3, zmm3, 8 + vpaddd zmm2, zmm2, zmm3 + vpxord zmm1, zmm1, zmm2 + vprord zmm1, zmm1, 7 + vpshufd zmm0, zmm0, 0x39 + vpshufd zmm3, zmm3, 0x4E + vpshufd zmm2, zmm2, 0x93 + dec al + jz 9f + vshufps zmm8, zmm4, zmm5, 214 + vpshufd zmm9, zmm4, 0x0F + vpshufd zmm4, zmm8, 0x39 + vshufps zmm8, zmm6, zmm7, 250 + vpblendmd zmm9 {k3}, zmm9, zmm8 + vpunpcklqdq zmm8, zmm7, zmm5 + vpblendmd zmm8 {k4}, zmm8, zmm6 + vpshufd zmm8, zmm8, 0x78 + vpunpckhdq zmm5, zmm5, zmm7 + vpunpckldq zmm6, zmm6, zmm5 + vpshufd zmm7, zmm6, 0x1E + vmovdqa32 zmm5, zmm9 + vmovdqa32 zmm6, zmm8 + jmp 9b +9: + vpxord zmm0, zmm0, zmm2 + vpxord zmm1, zmm1, zmm3 + mov eax, r13d + cmp rdx, r15 + jne 2b + vmovdqu xmmword ptr [rbx], xmm0 + vmovdqu xmmword ptr [rbx+0x10], xmm1 + vextracti128 xmmword ptr [rbx+0x20], ymm0, 0x01 + vextracti128 xmmword ptr [rbx+0x30], ymm1, 0x01 + vextracti32x4 xmmword ptr [rbx+0x4*0x10], zmm0, 0x02 + vextracti32x4 xmmword ptr [rbx+0x5*0x10], zmm1, 0x02 + vextracti32x4 xmmword ptr [rbx+0x6*0x10], zmm0, 0x03 + vextracti32x4 xmmword ptr [rbx+0x7*0x10], zmm1, 0x03 + vmovdqa xmm0, xmmword ptr [rsp] + vmovdqa xmm2, xmmword ptr [rsp+0x40] + vmovdqa32 xmm0 {k1}, xmmword ptr [rsp+0x1*0x10] + vmovdqa32 xmm2 {k1}, xmmword ptr [rsp+0x5*0x10] + vmovdqa xmmword ptr [rsp], xmm0 + vmovdqa xmmword ptr [rsp+0x40], xmm2 + add rbx, 128 + add rdi, 32 + sub rsi, 4 +3: + test esi, 0x2 + je 3f + vbroadcasti128 ymm0, xmmword ptr [rcx] + vbroadcasti128 ymm1, xmmword ptr [rcx+0x10] + vmovd xmm13, dword ptr [rsp] + vpinsrd xmm13, xmm13, dword ptr [rsp+0x40], 1 + vpinsrd xmm13, xmm13, dword ptr [BLAKE3_BLOCK_LEN+rip], 2 + vmovd xmm14, dword ptr [rsp+0x4] + vpinsrd xmm14, xmm14, dword ptr [rsp+0x44], 1 + vpinsrd xmm14, xmm14, dword ptr [BLAKE3_BLOCK_LEN+rip], 2 + vinserti128 ymm13, ymm13, xmm14, 0x01 + mov r8, qword ptr [rdi] + mov r9, qword ptr [rdi+0x8] + movzx eax, byte ptr [rbp+0x40] + or eax, r13d + xor edx, edx +.p2align 5 +2: + mov r14d, eax + or eax, r12d + add rdx, 64 + cmp rdx, r15 + cmovne eax, r14d + mov dword ptr [rsp+0x88], eax + vbroadcasti128 ymm2, xmmword ptr [BLAKE3_IV+rip] + vpbroadcastd ymm8, dword ptr [rsp+0x88] + vpblendd ymm3, ymm13, ymm8, 0x88 + vmovups ymm8, ymmword ptr [r8+rdx-0x40] + vinsertf128 ymm8, ymm8, xmmword ptr [r9+rdx-0x40], 0x01 + vmovups ymm9, ymmword ptr [r8+rdx-0x30] + vinsertf128 ymm9, ymm9, xmmword ptr [r9+rdx-0x30], 0x01 + vshufps ymm4, ymm8, ymm9, 136 + vshufps ymm5, ymm8, ymm9, 221 + vmovups ymm8, ymmword ptr [r8+rdx-0x20] + vinsertf128 ymm8, ymm8, xmmword ptr [r9+rdx-0x20], 0x01 + vmovups ymm9, ymmword ptr [r8+rdx-0x10] + vinsertf128 ymm9, ymm9, xmmword ptr [r9+rdx-0x10], 0x01 + vshufps ymm6, ymm8, ymm9, 136 + vshufps ymm7, ymm8, ymm9, 221 + vpshufd ymm6, ymm6, 0x93 + vpshufd ymm7, ymm7, 0x93 + mov al, 7 +9: + vpaddd ymm0, ymm0, ymm4 + vpaddd ymm0, ymm0, ymm1 + vpxord ymm3, ymm3, ymm0 + vprord ymm3, ymm3, 16 + vpaddd ymm2, ymm2, ymm3 + vpxord ymm1, ymm1, ymm2 + vprord ymm1, ymm1, 12 + vpaddd ymm0, ymm0, ymm5 + vpaddd ymm0, ymm0, ymm1 + vpxord ymm3, ymm3, ymm0 + vprord ymm3, ymm3, 8 + vpaddd ymm2, ymm2, ymm3 + vpxord ymm1, ymm1, ymm2 + vprord ymm1, ymm1, 7 + vpshufd ymm0, ymm0, 0x93 + vpshufd ymm3, ymm3, 0x4E + vpshufd ymm2, ymm2, 0x39 + vpaddd ymm0, ymm0, ymm6 + vpaddd ymm0, ymm0, ymm1 + vpxord ymm3, ymm3, ymm0 + vprord ymm3, ymm3, 16 + vpaddd ymm2, ymm2, ymm3 + vpxord ymm1, ymm1, ymm2 + vprord ymm1, ymm1, 12 + vpaddd ymm0, ymm0, ymm7 + vpaddd ymm0, ymm0, ymm1 + vpxord ymm3, ymm3, ymm0 + vprord ymm3, ymm3, 8 + vpaddd ymm2, ymm2, ymm3 + vpxord ymm1, ymm1, ymm2 + vprord ymm1, ymm1, 7 + vpshufd ymm0, ymm0, 0x39 + vpshufd ymm3, ymm3, 0x4E + vpshufd ymm2, ymm2, 0x93 + dec al + jz 9f + vshufps ymm8, ymm4, ymm5, 214 + vpshufd ymm9, ymm4, 0x0F + vpshufd ymm4, ymm8, 0x39 + vshufps ymm8, ymm6, ymm7, 250 + vpblendd ymm9, ymm9, ymm8, 0xAA + vpunpcklqdq ymm8, ymm7, ymm5 + vpblendd ymm8, ymm8, ymm6, 0x88 + vpshufd ymm8, ymm8, 0x78 + vpunpckhdq ymm5, ymm5, ymm7 + vpunpckldq ymm6, ymm6, ymm5 + vpshufd ymm7, ymm6, 0x1E + vmovdqa ymm5, ymm9 + vmovdqa ymm6, ymm8 + jmp 9b +9: + vpxor ymm0, ymm0, ymm2 + vpxor ymm1, ymm1, ymm3 + mov eax, r13d + cmp rdx, r15 + jne 2b + vmovdqu xmmword ptr [rbx], xmm0 + vmovdqu xmmword ptr [rbx+0x10], xmm1 + vextracti128 xmmword ptr [rbx+0x20], ymm0, 0x01 + vextracti128 xmmword ptr [rbx+0x30], ymm1, 0x01 + vmovdqa xmm0, xmmword ptr [rsp] + vmovdqa xmm2, xmmword ptr [rsp+0x4*0x10] + vmovdqu32 xmm0 {k1}, xmmword ptr [rsp+0x8] + vmovdqu32 xmm2 {k1}, xmmword ptr [rsp+0x48] + vmovdqa xmmword ptr [rsp], xmm0 + vmovdqa xmmword ptr [rsp+0x4*0x10], xmm2 + add rbx, 64 + add rdi, 16 + sub rsi, 2 +3: + test esi, 0x1 + je 4b + vmovdqu xmm0, xmmword ptr [rcx] + vmovdqu xmm1, xmmword ptr [rcx+0x10] + vmovd xmm14, dword ptr [rsp] + vpinsrd xmm14, xmm14, dword ptr [rsp+0x40], 1 + vpinsrd xmm14, xmm14, dword ptr [BLAKE3_BLOCK_LEN+rip], 2 + vmovdqa xmm15, xmmword ptr [BLAKE3_IV+rip] + mov r8, qword ptr [rdi] + movzx eax, byte ptr [rbp+0x40] + or eax, r13d + xor edx, edx +.p2align 5 +2: + mov r14d, eax + or eax, r12d + add rdx, 64 + cmp rdx, r15 + cmovne eax, r14d + vpinsrd xmm3, xmm14, eax, 3 + vmovdqa xmm2, xmm15 + vmovups xmm8, xmmword ptr [r8+rdx-0x40] + vmovups xmm9, xmmword ptr [r8+rdx-0x30] + vshufps xmm4, xmm8, xmm9, 136 + vshufps xmm5, xmm8, xmm9, 221 + vmovups xmm8, xmmword ptr [r8+rdx-0x20] + vmovups xmm9, xmmword ptr [r8+rdx-0x10] + vshufps xmm6, xmm8, xmm9, 136 + vshufps xmm7, xmm8, xmm9, 221 + vpshufd xmm6, xmm6, 0x93 + vpshufd xmm7, xmm7, 0x93 + mov al, 7 +9: + vpaddd xmm0, xmm0, xmm4 + vpaddd xmm0, xmm0, xmm1 + vpxord xmm3, xmm3, xmm0 + vprord xmm3, xmm3, 16 + vpaddd xmm2, xmm2, xmm3 + vpxord xmm1, xmm1, xmm2 + vprord xmm1, xmm1, 12 + vpaddd xmm0, xmm0, xmm5 + vpaddd xmm0, xmm0, xmm1 + vpxord xmm3, xmm3, xmm0 + vprord xmm3, xmm3, 8 + vpaddd xmm2, xmm2, xmm3 + vpxord xmm1, xmm1, xmm2 + vprord xmm1, xmm1, 7 + vpshufd xmm0, xmm0, 0x93 + vpshufd xmm3, xmm3, 0x4E + vpshufd xmm2, xmm2, 0x39 + vpaddd xmm0, xmm0, xmm6 + vpaddd xmm0, xmm0, xmm1 + vpxord xmm3, xmm3, xmm0 + vprord xmm3, xmm3, 16 + vpaddd xmm2, xmm2, xmm3 + vpxord xmm1, xmm1, xmm2 + vprord xmm1, xmm1, 12 + vpaddd xmm0, xmm0, xmm7 + vpaddd xmm0, xmm0, xmm1 + vpxord xmm3, xmm3, xmm0 + vprord xmm3, xmm3, 8 + vpaddd xmm2, xmm2, xmm3 + vpxord xmm1, xmm1, xmm2 + vprord xmm1, xmm1, 7 + vpshufd xmm0, xmm0, 0x39 + vpshufd xmm3, xmm3, 0x4E + vpshufd xmm2, xmm2, 0x93 + dec al + jz 9f + vshufps xmm8, xmm4, xmm5, 214 + vpshufd xmm9, xmm4, 0x0F + vpshufd xmm4, xmm8, 0x39 + vshufps xmm8, xmm6, xmm7, 250 + vpblendd xmm9, xmm9, xmm8, 0xAA + vpunpcklqdq xmm8, xmm7, xmm5 + vpblendd xmm8, xmm8, xmm6, 0x88 + vpshufd xmm8, xmm8, 0x78 + vpunpckhdq xmm5, xmm5, xmm7 + vpunpckldq xmm6, xmm6, xmm5 + vpshufd xmm7, xmm6, 0x1E + vmovdqa xmm5, xmm9 + vmovdqa xmm6, xmm8 + jmp 9b +9: + vpxor xmm0, xmm0, xmm2 + vpxor xmm1, xmm1, xmm3 + mov eax, r13d + cmp rdx, r15 + jne 2b + vmovdqu xmmword ptr [rbx], xmm0 + vmovdqu xmmword ptr [rbx+0x10], xmm1 + jmp 4b +.p2align 6 +zfs_blake3_compress_in_place_avx512: + _CET_ENDBR + vmovdqu xmm0, xmmword ptr [rdi] + vmovdqu xmm1, xmmword ptr [rdi+0x10] + movzx eax, r8b + movzx edx, dl + shl rax, 32 + add rdx, rax + vmovq xmm3, rcx + vmovq xmm4, rdx + vpunpcklqdq xmm3, xmm3, xmm4 + vmovaps xmm2, xmmword ptr [BLAKE3_IV+rip] + vmovups xmm8, xmmword ptr [rsi] + vmovups xmm9, xmmword ptr [rsi+0x10] + vshufps xmm4, xmm8, xmm9, 136 + vshufps xmm5, xmm8, xmm9, 221 + vmovups xmm8, xmmword ptr [rsi+0x20] + vmovups xmm9, xmmword ptr [rsi+0x30] + vshufps xmm6, xmm8, xmm9, 136 + vshufps xmm7, xmm8, xmm9, 221 + vpshufd xmm6, xmm6, 0x93 + vpshufd xmm7, xmm7, 0x93 + mov al, 7 +9: + vpaddd xmm0, xmm0, xmm4 + vpaddd xmm0, xmm0, xmm1 + vpxord xmm3, xmm3, xmm0 + vprord xmm3, xmm3, 16 + vpaddd xmm2, xmm2, xmm3 + vpxord xmm1, xmm1, xmm2 + vprord xmm1, xmm1, 12 + vpaddd xmm0, xmm0, xmm5 + vpaddd xmm0, xmm0, xmm1 + vpxord xmm3, xmm3, xmm0 + vprord xmm3, xmm3, 8 + vpaddd xmm2, xmm2, xmm3 + vpxord xmm1, xmm1, xmm2 + vprord xmm1, xmm1, 7 + vpshufd xmm0, xmm0, 0x93 + vpshufd xmm3, xmm3, 0x4E + vpshufd xmm2, xmm2, 0x39 + vpaddd xmm0, xmm0, xmm6 + vpaddd xmm0, xmm0, xmm1 + vpxord xmm3, xmm3, xmm0 + vprord xmm3, xmm3, 16 + vpaddd xmm2, xmm2, xmm3 + vpxord xmm1, xmm1, xmm2 + vprord xmm1, xmm1, 12 + vpaddd xmm0, xmm0, xmm7 + vpaddd xmm0, xmm0, xmm1 + vpxord xmm3, xmm3, xmm0 + vprord xmm3, xmm3, 8 + vpaddd xmm2, xmm2, xmm3 + vpxord xmm1, xmm1, xmm2 + vprord xmm1, xmm1, 7 + vpshufd xmm0, xmm0, 0x39 + vpshufd xmm3, xmm3, 0x4E + vpshufd xmm2, xmm2, 0x93 + dec al + jz 9f + vshufps xmm8, xmm4, xmm5, 214 + vpshufd xmm9, xmm4, 0x0F + vpshufd xmm4, xmm8, 0x39 + vshufps xmm8, xmm6, xmm7, 250 + vpblendd xmm9, xmm9, xmm8, 0xAA + vpunpcklqdq xmm8, xmm7, xmm5 + vpblendd xmm8, xmm8, xmm6, 0x88 + vpshufd xmm8, xmm8, 0x78 + vpunpckhdq xmm5, xmm5, xmm7 + vpunpckldq xmm6, xmm6, xmm5 + vpshufd xmm7, xmm6, 0x1E + vmovdqa xmm5, xmm9 + vmovdqa xmm6, xmm8 + jmp 9b +9: + vpxor xmm0, xmm0, xmm2 + vpxor xmm1, xmm1, xmm3 + vmovdqu xmmword ptr [rdi], xmm0 + vmovdqu xmmword ptr [rdi+0x10], xmm1 + RET + +.p2align 6 +zfs_blake3_compress_xof_avx512: + _CET_ENDBR + vmovdqu xmm0, xmmword ptr [rdi] + vmovdqu xmm1, xmmword ptr [rdi+0x10] + movzx eax, r8b + movzx edx, dl + shl rax, 32 + add rdx, rax + vmovq xmm3, rcx + vmovq xmm4, rdx + vpunpcklqdq xmm3, xmm3, xmm4 + vmovaps xmm2, xmmword ptr [BLAKE3_IV+rip] + vmovups xmm8, xmmword ptr [rsi] + vmovups xmm9, xmmword ptr [rsi+0x10] + vshufps xmm4, xmm8, xmm9, 136 + vshufps xmm5, xmm8, xmm9, 221 + vmovups xmm8, xmmword ptr [rsi+0x20] + vmovups xmm9, xmmword ptr [rsi+0x30] + vshufps xmm6, xmm8, xmm9, 136 + vshufps xmm7, xmm8, xmm9, 221 + vpshufd xmm6, xmm6, 0x93 + vpshufd xmm7, xmm7, 0x93 + mov al, 7 +9: + vpaddd xmm0, xmm0, xmm4 + vpaddd xmm0, xmm0, xmm1 + vpxord xmm3, xmm3, xmm0 + vprord xmm3, xmm3, 16 + vpaddd xmm2, xmm2, xmm3 + vpxord xmm1, xmm1, xmm2 + vprord xmm1, xmm1, 12 + vpaddd xmm0, xmm0, xmm5 + vpaddd xmm0, xmm0, xmm1 + vpxord xmm3, xmm3, xmm0 + vprord xmm3, xmm3, 8 + vpaddd xmm2, xmm2, xmm3 + vpxord xmm1, xmm1, xmm2 + vprord xmm1, xmm1, 7 + vpshufd xmm0, xmm0, 0x93 + vpshufd xmm3, xmm3, 0x4E + vpshufd xmm2, xmm2, 0x39 + vpaddd xmm0, xmm0, xmm6 + vpaddd xmm0, xmm0, xmm1 + vpxord xmm3, xmm3, xmm0 + vprord xmm3, xmm3, 16 + vpaddd xmm2, xmm2, xmm3 + vpxord xmm1, xmm1, xmm2 + vprord xmm1, xmm1, 12 + vpaddd xmm0, xmm0, xmm7 + vpaddd xmm0, xmm0, xmm1 + vpxord xmm3, xmm3, xmm0 + vprord xmm3, xmm3, 8 + vpaddd xmm2, xmm2, xmm3 + vpxord xmm1, xmm1, xmm2 + vprord xmm1, xmm1, 7 + vpshufd xmm0, xmm0, 0x39 + vpshufd xmm3, xmm3, 0x4E + vpshufd xmm2, xmm2, 0x93 + dec al + jz 9f + vshufps xmm8, xmm4, xmm5, 214 + vpshufd xmm9, xmm4, 0x0F + vpshufd xmm4, xmm8, 0x39 + vshufps xmm8, xmm6, xmm7, 250 + vpblendd xmm9, xmm9, xmm8, 0xAA + vpunpcklqdq xmm8, xmm7, xmm5 + vpblendd xmm8, xmm8, xmm6, 0x88 + vpshufd xmm8, xmm8, 0x78 + vpunpckhdq xmm5, xmm5, xmm7 + vpunpckldq xmm6, xmm6, xmm5 + vpshufd xmm7, xmm6, 0x1E + vmovdqa xmm5, xmm9 + vmovdqa xmm6, xmm8 + jmp 9b +9: + vpxor xmm0, xmm0, xmm2 + vpxor xmm1, xmm1, xmm3 + vpxor xmm2, xmm2, [rdi] + vpxor xmm3, xmm3, [rdi+0x10] + vmovdqu xmmword ptr [r9], xmm0 + vmovdqu xmmword ptr [r9+0x10], xmm1 + vmovdqu xmmword ptr [r9+0x20], xmm2 + vmovdqu xmmword ptr [r9+0x30], xmm3 + RET + +.size zfs_blake3_hash_many_avx512, . - zfs_blake3_hash_many_avx512 +.size zfs_blake3_compress_in_place_avx512, . - zfs_blake3_compress_in_place_avx512 +.size zfs_blake3_compress_xof_avx512, . - zfs_blake3_compress_xof_avx512 + +#ifdef __APPLE__ +.static_data +#else +.section .rodata +#endif + +.p2align 6 +INDEX0: + .long 0, 1, 2, 3, 16, 17, 18, 19 + .long 8, 9, 10, 11, 24, 25, 26, 27 +INDEX1: + .long 4, 5, 6, 7, 20, 21, 22, 23 + .long 12, 13, 14, 15, 28, 29, 30, 31 +ADD0: + .long 0, 1, 2, 3, 4, 5, 6, 7 + .long 8, 9, 10, 11, 12, 13, 14, 15 +ADD1: .long 1 + +ADD16: .long 16 +BLAKE3_BLOCK_LEN: + .long 64 +.p2align 6 +BLAKE3_IV: +BLAKE3_IV_0: + .long 0x6A09E667 +BLAKE3_IV_1: + .long 0xBB67AE85 +BLAKE3_IV_2: + .long 0x3C6EF372 +BLAKE3_IV_3: + .long 0xA54FF53A + +#endif /* HAVE_AVX512 */ + +#ifdef __ELF__ +.section .note.GNU-stack,"",%progbits +#endif diff --git a/module/icp/asm-x86_64/blake3/blake3_sse2.S b/module/icp/asm-x86_64/blake3/blake3_sse2.S new file mode 100644 index 000000000000..20689a7dcef5 --- /dev/null +++ b/module/icp/asm-x86_64/blake3/blake3_sse2.S @@ -0,0 +1,2323 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Based on BLAKE3 v1.3.1, https://github.com/BLAKE3-team/BLAKE3 + * Copyright (c) 2019-2020 Samuel Neves and Matthew Krupcale + * Copyright (c) 2022 Tino Reichardt + */ + +#if defined(HAVE_SSE2) + +#define _ASM +#include + +#if defined(__ELF__) && defined(__CET__) && defined(__has_include) +#if __has_include() +#include +#endif +#endif + +#if !defined(_CET_ENDBR) +#define _CET_ENDBR +#endif + +.intel_syntax noprefix +.global zfs_blake3_hash_many_sse2 +.global zfs_blake3_compress_in_place_sse2 +.global zfs_blake3_compress_xof_sse2 + +.text +.type zfs_blake3_hash_many_sse2,@function +.type zfs_blake3_compress_in_place_sse2,@function +.type zfs_blake3_compress_xof_sse2,@function + + .p2align 6 +zfs_blake3_hash_many_sse2: + _CET_ENDBR + push r15 + push r14 + push r13 + push r12 + push rbx + push rbp + mov rbp, rsp + sub rsp, 360 + and rsp, 0xFFFFFFFFFFFFFFC0 + neg r9d + movd xmm0, r9d + pshufd xmm0, xmm0, 0x00 + movdqa xmmword ptr [rsp+0x130], xmm0 + movdqa xmm1, xmm0 + pand xmm1, xmmword ptr [ADD0+rip] + pand xmm0, xmmword ptr [ADD1+rip] + movdqa xmmword ptr [rsp+0x150], xmm0 + movd xmm0, r8d + pshufd xmm0, xmm0, 0x00 + paddd xmm0, xmm1 + movdqa xmmword ptr [rsp+0x110], xmm0 + pxor xmm0, xmmword ptr [CMP_MSB_MASK+rip] + pxor xmm1, xmmword ptr [CMP_MSB_MASK+rip] + pcmpgtd xmm1, xmm0 + shr r8, 32 + movd xmm2, r8d + pshufd xmm2, xmm2, 0x00 + psubd xmm2, xmm1 + movdqa xmmword ptr [rsp+0x120], xmm2 + mov rbx, qword ptr [rbp+0x50] + mov r15, rdx + shl r15, 6 + movzx r13d, byte ptr [rbp+0x38] + movzx r12d, byte ptr [rbp+0x48] + cmp rsi, 4 + jc 3f +2: + movdqu xmm3, xmmword ptr [rcx] + pshufd xmm0, xmm3, 0x00 + pshufd xmm1, xmm3, 0x55 + pshufd xmm2, xmm3, 0xAA + pshufd xmm3, xmm3, 0xFF + movdqu xmm7, xmmword ptr [rcx+0x10] + pshufd xmm4, xmm7, 0x00 + pshufd xmm5, xmm7, 0x55 + pshufd xmm6, xmm7, 0xAA + pshufd xmm7, xmm7, 0xFF + mov r8, qword ptr [rdi] + mov r9, qword ptr [rdi+0x8] + mov r10, qword ptr [rdi+0x10] + mov r11, qword ptr [rdi+0x18] + movzx eax, byte ptr [rbp+0x40] + or eax, r13d + xor edx, edx +9: + mov r14d, eax + or eax, r12d + add rdx, 64 + cmp rdx, r15 + cmovne eax, r14d + movdqu xmm8, xmmword ptr [r8+rdx-0x40] + movdqu xmm9, xmmword ptr [r9+rdx-0x40] + movdqu xmm10, xmmword ptr [r10+rdx-0x40] + movdqu xmm11, xmmword ptr [r11+rdx-0x40] + movdqa xmm12, xmm8 + punpckldq xmm8, xmm9 + punpckhdq xmm12, xmm9 + movdqa xmm14, xmm10 + punpckldq xmm10, xmm11 + punpckhdq xmm14, xmm11 + movdqa xmm9, xmm8 + punpcklqdq xmm8, xmm10 + punpckhqdq xmm9, xmm10 + movdqa xmm13, xmm12 + punpcklqdq xmm12, xmm14 + punpckhqdq xmm13, xmm14 + movdqa xmmword ptr [rsp], xmm8 + movdqa xmmword ptr [rsp+0x10], xmm9 + movdqa xmmword ptr [rsp+0x20], xmm12 + movdqa xmmword ptr [rsp+0x30], xmm13 + movdqu xmm8, xmmword ptr [r8+rdx-0x30] + movdqu xmm9, xmmword ptr [r9+rdx-0x30] + movdqu xmm10, xmmword ptr [r10+rdx-0x30] + movdqu xmm11, xmmword ptr [r11+rdx-0x30] + movdqa xmm12, xmm8 + punpckldq xmm8, xmm9 + punpckhdq xmm12, xmm9 + movdqa xmm14, xmm10 + punpckldq xmm10, xmm11 + punpckhdq xmm14, xmm11 + movdqa xmm9, xmm8 + punpcklqdq xmm8, xmm10 + punpckhqdq xmm9, xmm10 + movdqa xmm13, xmm12 + punpcklqdq xmm12, xmm14 + punpckhqdq xmm13, xmm14 + movdqa xmmword ptr [rsp+0x40], xmm8 + movdqa xmmword ptr [rsp+0x50], xmm9 + movdqa xmmword ptr [rsp+0x60], xmm12 + movdqa xmmword ptr [rsp+0x70], xmm13 + movdqu xmm8, xmmword ptr [r8+rdx-0x20] + movdqu xmm9, xmmword ptr [r9+rdx-0x20] + movdqu xmm10, xmmword ptr [r10+rdx-0x20] + movdqu xmm11, xmmword ptr [r11+rdx-0x20] + movdqa xmm12, xmm8 + punpckldq xmm8, xmm9 + punpckhdq xmm12, xmm9 + movdqa xmm14, xmm10 + punpckldq xmm10, xmm11 + punpckhdq xmm14, xmm11 + movdqa xmm9, xmm8 + punpcklqdq xmm8, xmm10 + punpckhqdq xmm9, xmm10 + movdqa xmm13, xmm12 + punpcklqdq xmm12, xmm14 + punpckhqdq xmm13, xmm14 + movdqa xmmword ptr [rsp+0x80], xmm8 + movdqa xmmword ptr [rsp+0x90], xmm9 + movdqa xmmword ptr [rsp+0xA0], xmm12 + movdqa xmmword ptr [rsp+0xB0], xmm13 + movdqu xmm8, xmmword ptr [r8+rdx-0x10] + movdqu xmm9, xmmword ptr [r9+rdx-0x10] + movdqu xmm10, xmmword ptr [r10+rdx-0x10] + movdqu xmm11, xmmword ptr [r11+rdx-0x10] + movdqa xmm12, xmm8 + punpckldq xmm8, xmm9 + punpckhdq xmm12, xmm9 + movdqa xmm14, xmm10 + punpckldq xmm10, xmm11 + punpckhdq xmm14, xmm11 + movdqa xmm9, xmm8 + punpcklqdq xmm8, xmm10 + punpckhqdq xmm9, xmm10 + movdqa xmm13, xmm12 + punpcklqdq xmm12, xmm14 + punpckhqdq xmm13, xmm14 + movdqa xmmword ptr [rsp+0xC0], xmm8 + movdqa xmmword ptr [rsp+0xD0], xmm9 + movdqa xmmword ptr [rsp+0xE0], xmm12 + movdqa xmmword ptr [rsp+0xF0], xmm13 + movdqa xmm9, xmmword ptr [BLAKE3_IV_1+rip] + movdqa xmm10, xmmword ptr [BLAKE3_IV_2+rip] + movdqa xmm11, xmmword ptr [BLAKE3_IV_3+rip] + movdqa xmm12, xmmword ptr [rsp+0x110] + movdqa xmm13, xmmword ptr [rsp+0x120] + movdqa xmm14, xmmword ptr [BLAKE3_BLOCK_LEN+rip] + movd xmm15, eax + pshufd xmm15, xmm15, 0x00 + prefetcht0 [r8+rdx+0x80] + prefetcht0 [r9+rdx+0x80] + prefetcht0 [r10+rdx+0x80] + prefetcht0 [r11+rdx+0x80] + paddd xmm0, xmmword ptr [rsp] + paddd xmm1, xmmword ptr [rsp+0x20] + paddd xmm2, xmmword ptr [rsp+0x40] + paddd xmm3, xmmword ptr [rsp+0x60] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + pshuflw xmm12, xmm12, 0xB1 + pshufhw xmm12, xmm12, 0xB1 + pshuflw xmm13, xmm13, 0xB1 + pshufhw xmm13, xmm13, 0xB1 + pshuflw xmm14, xmm14, 0xB1 + pshufhw xmm14, xmm14, 0xB1 + pshuflw xmm15, xmm15, 0xB1 + pshufhw xmm15, xmm15, 0xB1 + movdqa xmm8, xmmword ptr [BLAKE3_IV_0+rip] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x10] + paddd xmm1, xmmword ptr [rsp+0x30] + paddd xmm2, xmmword ptr [rsp+0x50] + paddd xmm3, xmmword ptr [rsp+0x70] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmm12 + psrld xmm12, 8 + pslld xmm8, 24 + pxor xmm12, xmm8 + movdqa xmm8, xmm13 + psrld xmm13, 8 + pslld xmm8, 24 + pxor xmm13, xmm8 + movdqa xmm8, xmm14 + psrld xmm14, 8 + pslld xmm8, 24 + pxor xmm14, xmm8 + movdqa xmm8, xmm15 + psrld xmm15, 8 + pslld xmm8, 24 + pxor xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x80] + paddd xmm1, xmmword ptr [rsp+0xA0] + paddd xmm2, xmmword ptr [rsp+0xC0] + paddd xmm3, xmmword ptr [rsp+0xE0] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + pshuflw xmm15, xmm15, 0xB1 + pshufhw xmm15, xmm15, 0xB1 + pshuflw xmm12, xmm12, 0xB1 + pshufhw xmm12, xmm12, 0xB1 + pshuflw xmm13, xmm13, 0xB1 + pshufhw xmm13, xmm13, 0xB1 + pshuflw xmm14, xmm14, 0xB1 + pshufhw xmm14, xmm14, 0xB1 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0x90] + paddd xmm1, xmmword ptr [rsp+0xB0] + paddd xmm2, xmmword ptr [rsp+0xD0] + paddd xmm3, xmmword ptr [rsp+0xF0] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmm15 + psrld xmm15, 8 + pslld xmm8, 24 + pxor xmm15, xmm8 + movdqa xmm8, xmm12 + psrld xmm12, 8 + pslld xmm8, 24 + pxor xmm12, xmm8 + movdqa xmm8, xmm13 + psrld xmm13, 8 + pslld xmm8, 24 + pxor xmm13, xmm8 + movdqa xmm8, xmm14 + psrld xmm14, 8 + pslld xmm8, 24 + pxor xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0x20] + paddd xmm1, xmmword ptr [rsp+0x30] + paddd xmm2, xmmword ptr [rsp+0x70] + paddd xmm3, xmmword ptr [rsp+0x40] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + pshuflw xmm12, xmm12, 0xB1 + pshufhw xmm12, xmm12, 0xB1 + pshuflw xmm13, xmm13, 0xB1 + pshufhw xmm13, xmm13, 0xB1 + pshuflw xmm14, xmm14, 0xB1 + pshufhw xmm14, xmm14, 0xB1 + pshuflw xmm15, xmm15, 0xB1 + pshufhw xmm15, xmm15, 0xB1 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x60] + paddd xmm1, xmmword ptr [rsp+0xA0] + paddd xmm2, xmmword ptr [rsp] + paddd xmm3, xmmword ptr [rsp+0xD0] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmm12 + psrld xmm12, 8 + pslld xmm8, 24 + pxor xmm12, xmm8 + movdqa xmm8, xmm13 + psrld xmm13, 8 + pslld xmm8, 24 + pxor xmm13, xmm8 + movdqa xmm8, xmm14 + psrld xmm14, 8 + pslld xmm8, 24 + pxor xmm14, xmm8 + movdqa xmm8, xmm15 + psrld xmm15, 8 + pslld xmm8, 24 + pxor xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x10] + paddd xmm1, xmmword ptr [rsp+0xC0] + paddd xmm2, xmmword ptr [rsp+0x90] + paddd xmm3, xmmword ptr [rsp+0xF0] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + pshuflw xmm15, xmm15, 0xB1 + pshufhw xmm15, xmm15, 0xB1 + pshuflw xmm12, xmm12, 0xB1 + pshufhw xmm12, xmm12, 0xB1 + pshuflw xmm13, xmm13, 0xB1 + pshufhw xmm13, xmm13, 0xB1 + pshuflw xmm14, xmm14, 0xB1 + pshufhw xmm14, xmm14, 0xB1 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0xB0] + paddd xmm1, xmmword ptr [rsp+0x50] + paddd xmm2, xmmword ptr [rsp+0xE0] + paddd xmm3, xmmword ptr [rsp+0x80] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmm15 + psrld xmm15, 8 + pslld xmm8, 24 + pxor xmm15, xmm8 + movdqa xmm8, xmm12 + psrld xmm12, 8 + pslld xmm8, 24 + pxor xmm12, xmm8 + movdqa xmm8, xmm13 + psrld xmm13, 8 + pslld xmm8, 24 + pxor xmm13, xmm8 + movdqa xmm8, xmm14 + psrld xmm14, 8 + pslld xmm8, 24 + pxor xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0x30] + paddd xmm1, xmmword ptr [rsp+0xA0] + paddd xmm2, xmmword ptr [rsp+0xD0] + paddd xmm3, xmmword ptr [rsp+0x70] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + pshuflw xmm12, xmm12, 0xB1 + pshufhw xmm12, xmm12, 0xB1 + pshuflw xmm13, xmm13, 0xB1 + pshufhw xmm13, xmm13, 0xB1 + pshuflw xmm14, xmm14, 0xB1 + pshufhw xmm14, xmm14, 0xB1 + pshuflw xmm15, xmm15, 0xB1 + pshufhw xmm15, xmm15, 0xB1 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x40] + paddd xmm1, xmmword ptr [rsp+0xC0] + paddd xmm2, xmmword ptr [rsp+0x20] + paddd xmm3, xmmword ptr [rsp+0xE0] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmm12 + psrld xmm12, 8 + pslld xmm8, 24 + pxor xmm12, xmm8 + movdqa xmm8, xmm13 + psrld xmm13, 8 + pslld xmm8, 24 + pxor xmm13, xmm8 + movdqa xmm8, xmm14 + psrld xmm14, 8 + pslld xmm8, 24 + pxor xmm14, xmm8 + movdqa xmm8, xmm15 + psrld xmm15, 8 + pslld xmm8, 24 + pxor xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x60] + paddd xmm1, xmmword ptr [rsp+0x90] + paddd xmm2, xmmword ptr [rsp+0xB0] + paddd xmm3, xmmword ptr [rsp+0x80] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + pshuflw xmm15, xmm15, 0xB1 + pshufhw xmm15, xmm15, 0xB1 + pshuflw xmm12, xmm12, 0xB1 + pshufhw xmm12, xmm12, 0xB1 + pshuflw xmm13, xmm13, 0xB1 + pshufhw xmm13, xmm13, 0xB1 + pshuflw xmm14, xmm14, 0xB1 + pshufhw xmm14, xmm14, 0xB1 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0x50] + paddd xmm1, xmmword ptr [rsp] + paddd xmm2, xmmword ptr [rsp+0xF0] + paddd xmm3, xmmword ptr [rsp+0x10] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmm15 + psrld xmm15, 8 + pslld xmm8, 24 + pxor xmm15, xmm8 + movdqa xmm8, xmm12 + psrld xmm12, 8 + pslld xmm8, 24 + pxor xmm12, xmm8 + movdqa xmm8, xmm13 + psrld xmm13, 8 + pslld xmm8, 24 + pxor xmm13, xmm8 + movdqa xmm8, xmm14 + psrld xmm14, 8 + pslld xmm8, 24 + pxor xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0xA0] + paddd xmm1, xmmword ptr [rsp+0xC0] + paddd xmm2, xmmword ptr [rsp+0xE0] + paddd xmm3, xmmword ptr [rsp+0xD0] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + pshuflw xmm12, xmm12, 0xB1 + pshufhw xmm12, xmm12, 0xB1 + pshuflw xmm13, xmm13, 0xB1 + pshufhw xmm13, xmm13, 0xB1 + pshuflw xmm14, xmm14, 0xB1 + pshufhw xmm14, xmm14, 0xB1 + pshuflw xmm15, xmm15, 0xB1 + pshufhw xmm15, xmm15, 0xB1 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x70] + paddd xmm1, xmmword ptr [rsp+0x90] + paddd xmm2, xmmword ptr [rsp+0x30] + paddd xmm3, xmmword ptr [rsp+0xF0] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmm12 + psrld xmm12, 8 + pslld xmm8, 24 + pxor xmm12, xmm8 + movdqa xmm8, xmm13 + psrld xmm13, 8 + pslld xmm8, 24 + pxor xmm13, xmm8 + movdqa xmm8, xmm14 + psrld xmm14, 8 + pslld xmm8, 24 + pxor xmm14, xmm8 + movdqa xmm8, xmm15 + psrld xmm15, 8 + pslld xmm8, 24 + pxor xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x40] + paddd xmm1, xmmword ptr [rsp+0xB0] + paddd xmm2, xmmword ptr [rsp+0x50] + paddd xmm3, xmmword ptr [rsp+0x10] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + pshuflw xmm15, xmm15, 0xB1 + pshufhw xmm15, xmm15, 0xB1 + pshuflw xmm12, xmm12, 0xB1 + pshufhw xmm12, xmm12, 0xB1 + pshuflw xmm13, xmm13, 0xB1 + pshufhw xmm13, xmm13, 0xB1 + pshuflw xmm14, xmm14, 0xB1 + pshufhw xmm14, xmm14, 0xB1 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp] + paddd xmm1, xmmword ptr [rsp+0x20] + paddd xmm2, xmmword ptr [rsp+0x80] + paddd xmm3, xmmword ptr [rsp+0x60] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmm15 + psrld xmm15, 8 + pslld xmm8, 24 + pxor xmm15, xmm8 + movdqa xmm8, xmm12 + psrld xmm12, 8 + pslld xmm8, 24 + pxor xmm12, xmm8 + movdqa xmm8, xmm13 + psrld xmm13, 8 + pslld xmm8, 24 + pxor xmm13, xmm8 + movdqa xmm8, xmm14 + psrld xmm14, 8 + pslld xmm8, 24 + pxor xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0xC0] + paddd xmm1, xmmword ptr [rsp+0x90] + paddd xmm2, xmmword ptr [rsp+0xF0] + paddd xmm3, xmmword ptr [rsp+0xE0] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + pshuflw xmm12, xmm12, 0xB1 + pshufhw xmm12, xmm12, 0xB1 + pshuflw xmm13, xmm13, 0xB1 + pshufhw xmm13, xmm13, 0xB1 + pshuflw xmm14, xmm14, 0xB1 + pshufhw xmm14, xmm14, 0xB1 + pshuflw xmm15, xmm15, 0xB1 + pshufhw xmm15, xmm15, 0xB1 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0xD0] + paddd xmm1, xmmword ptr [rsp+0xB0] + paddd xmm2, xmmword ptr [rsp+0xA0] + paddd xmm3, xmmword ptr [rsp+0x80] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmm12 + psrld xmm12, 8 + pslld xmm8, 24 + pxor xmm12, xmm8 + movdqa xmm8, xmm13 + psrld xmm13, 8 + pslld xmm8, 24 + pxor xmm13, xmm8 + movdqa xmm8, xmm14 + psrld xmm14, 8 + pslld xmm8, 24 + pxor xmm14, xmm8 + movdqa xmm8, xmm15 + psrld xmm15, 8 + pslld xmm8, 24 + pxor xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x70] + paddd xmm1, xmmword ptr [rsp+0x50] + paddd xmm2, xmmword ptr [rsp] + paddd xmm3, xmmword ptr [rsp+0x60] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + pshuflw xmm15, xmm15, 0xB1 + pshufhw xmm15, xmm15, 0xB1 + pshuflw xmm12, xmm12, 0xB1 + pshufhw xmm12, xmm12, 0xB1 + pshuflw xmm13, xmm13, 0xB1 + pshufhw xmm13, xmm13, 0xB1 + pshuflw xmm14, xmm14, 0xB1 + pshufhw xmm14, xmm14, 0xB1 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0x20] + paddd xmm1, xmmword ptr [rsp+0x30] + paddd xmm2, xmmword ptr [rsp+0x10] + paddd xmm3, xmmword ptr [rsp+0x40] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmm15 + psrld xmm15, 8 + pslld xmm8, 24 + pxor xmm15, xmm8 + movdqa xmm8, xmm12 + psrld xmm12, 8 + pslld xmm8, 24 + pxor xmm12, xmm8 + movdqa xmm8, xmm13 + psrld xmm13, 8 + pslld xmm8, 24 + pxor xmm13, xmm8 + movdqa xmm8, xmm14 + psrld xmm14, 8 + pslld xmm8, 24 + pxor xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0x90] + paddd xmm1, xmmword ptr [rsp+0xB0] + paddd xmm2, xmmword ptr [rsp+0x80] + paddd xmm3, xmmword ptr [rsp+0xF0] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + pshuflw xmm12, xmm12, 0xB1 + pshufhw xmm12, xmm12, 0xB1 + pshuflw xmm13, xmm13, 0xB1 + pshufhw xmm13, xmm13, 0xB1 + pshuflw xmm14, xmm14, 0xB1 + pshufhw xmm14, xmm14, 0xB1 + pshuflw xmm15, xmm15, 0xB1 + pshufhw xmm15, xmm15, 0xB1 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0xE0] + paddd xmm1, xmmword ptr [rsp+0x50] + paddd xmm2, xmmword ptr [rsp+0xC0] + paddd xmm3, xmmword ptr [rsp+0x10] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmm12 + psrld xmm12, 8 + pslld xmm8, 24 + pxor xmm12, xmm8 + movdqa xmm8, xmm13 + psrld xmm13, 8 + pslld xmm8, 24 + pxor xmm13, xmm8 + movdqa xmm8, xmm14 + psrld xmm14, 8 + pslld xmm8, 24 + pxor xmm14, xmm8 + movdqa xmm8, xmm15 + psrld xmm15, 8 + pslld xmm8, 24 + pxor xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0xD0] + paddd xmm1, xmmword ptr [rsp] + paddd xmm2, xmmword ptr [rsp+0x20] + paddd xmm3, xmmword ptr [rsp+0x40] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + pshuflw xmm15, xmm15, 0xB1 + pshufhw xmm15, xmm15, 0xB1 + pshuflw xmm12, xmm12, 0xB1 + pshufhw xmm12, xmm12, 0xB1 + pshuflw xmm13, xmm13, 0xB1 + pshufhw xmm13, xmm13, 0xB1 + pshuflw xmm14, xmm14, 0xB1 + pshufhw xmm14, xmm14, 0xB1 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0x30] + paddd xmm1, xmmword ptr [rsp+0xA0] + paddd xmm2, xmmword ptr [rsp+0x60] + paddd xmm3, xmmword ptr [rsp+0x70] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmm15 + psrld xmm15, 8 + pslld xmm8, 24 + pxor xmm15, xmm8 + movdqa xmm8, xmm12 + psrld xmm12, 8 + pslld xmm8, 24 + pxor xmm12, xmm8 + movdqa xmm8, xmm13 + psrld xmm13, 8 + pslld xmm8, 24 + pxor xmm13, xmm8 + movdqa xmm8, xmm14 + psrld xmm14, 8 + pslld xmm8, 24 + pxor xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0xB0] + paddd xmm1, xmmword ptr [rsp+0x50] + paddd xmm2, xmmword ptr [rsp+0x10] + paddd xmm3, xmmword ptr [rsp+0x80] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + pshuflw xmm12, xmm12, 0xB1 + pshufhw xmm12, xmm12, 0xB1 + pshuflw xmm13, xmm13, 0xB1 + pshufhw xmm13, xmm13, 0xB1 + pshuflw xmm14, xmm14, 0xB1 + pshufhw xmm14, xmm14, 0xB1 + pshuflw xmm15, xmm15, 0xB1 + pshufhw xmm15, xmm15, 0xB1 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0xF0] + paddd xmm1, xmmword ptr [rsp] + paddd xmm2, xmmword ptr [rsp+0x90] + paddd xmm3, xmmword ptr [rsp+0x60] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmm12 + psrld xmm12, 8 + pslld xmm8, 24 + pxor xmm12, xmm8 + movdqa xmm8, xmm13 + psrld xmm13, 8 + pslld xmm8, 24 + pxor xmm13, xmm8 + movdqa xmm8, xmm14 + psrld xmm14, 8 + pslld xmm8, 24 + pxor xmm14, xmm8 + movdqa xmm8, xmm15 + psrld xmm15, 8 + pslld xmm8, 24 + pxor xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0xE0] + paddd xmm1, xmmword ptr [rsp+0x20] + paddd xmm2, xmmword ptr [rsp+0x30] + paddd xmm3, xmmword ptr [rsp+0x70] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + pshuflw xmm15, xmm15, 0xB1 + pshufhw xmm15, xmm15, 0xB1 + pshuflw xmm12, xmm12, 0xB1 + pshufhw xmm12, xmm12, 0xB1 + pshuflw xmm13, xmm13, 0xB1 + pshufhw xmm13, xmm13, 0xB1 + pshuflw xmm14, xmm14, 0xB1 + pshufhw xmm14, xmm14, 0xB1 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0xA0] + paddd xmm1, xmmword ptr [rsp+0xC0] + paddd xmm2, xmmword ptr [rsp+0x40] + paddd xmm3, xmmword ptr [rsp+0xD0] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmm15 + psrld xmm15, 8 + pslld xmm8, 24 + pxor xmm15, xmm8 + movdqa xmm8, xmm12 + psrld xmm12, 8 + pslld xmm8, 24 + pxor xmm12, xmm8 + movdqa xmm8, xmm13 + psrld xmm13, 8 + pslld xmm8, 24 + pxor xmm13, xmm8 + movdqa xmm8, xmm14 + psrld xmm14, 8 + pslld xmm8, 24 + pxor xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + pxor xmm0, xmm8 + pxor xmm1, xmm9 + pxor xmm2, xmm10 + pxor xmm3, xmm11 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + pxor xmm4, xmm12 + pxor xmm5, xmm13 + pxor xmm6, xmm14 + pxor xmm7, xmm15 + mov eax, r13d + jne 9b + movdqa xmm9, xmm0 + punpckldq xmm0, xmm1 + punpckhdq xmm9, xmm1 + movdqa xmm11, xmm2 + punpckldq xmm2, xmm3 + punpckhdq xmm11, xmm3 + movdqa xmm1, xmm0 + punpcklqdq xmm0, xmm2 + punpckhqdq xmm1, xmm2 + movdqa xmm3, xmm9 + punpcklqdq xmm9, xmm11 + punpckhqdq xmm3, xmm11 + movdqu xmmword ptr [rbx], xmm0 + movdqu xmmword ptr [rbx+0x20], xmm1 + movdqu xmmword ptr [rbx+0x40], xmm9 + movdqu xmmword ptr [rbx+0x60], xmm3 + movdqa xmm9, xmm4 + punpckldq xmm4, xmm5 + punpckhdq xmm9, xmm5 + movdqa xmm11, xmm6 + punpckldq xmm6, xmm7 + punpckhdq xmm11, xmm7 + movdqa xmm5, xmm4 + punpcklqdq xmm4, xmm6 + punpckhqdq xmm5, xmm6 + movdqa xmm7, xmm9 + punpcklqdq xmm9, xmm11 + punpckhqdq xmm7, xmm11 + movdqu xmmword ptr [rbx+0x10], xmm4 + movdqu xmmword ptr [rbx+0x30], xmm5 + movdqu xmmword ptr [rbx+0x50], xmm9 + movdqu xmmword ptr [rbx+0x70], xmm7 + movdqa xmm1, xmmword ptr [rsp+0x110] + movdqa xmm0, xmm1 + paddd xmm1, xmmword ptr [rsp+0x150] + movdqa xmmword ptr [rsp+0x110], xmm1 + pxor xmm0, xmmword ptr [CMP_MSB_MASK+rip] + pxor xmm1, xmmword ptr [CMP_MSB_MASK+rip] + pcmpgtd xmm0, xmm1 + movdqa xmm1, xmmword ptr [rsp+0x120] + psubd xmm1, xmm0 + movdqa xmmword ptr [rsp+0x120], xmm1 + add rbx, 128 + add rdi, 32 + sub rsi, 4 + cmp rsi, 4 + jnc 2b + test rsi, rsi + jnz 3f +4: + mov rsp, rbp + pop rbp + pop rbx + pop r12 + pop r13 + pop r14 + pop r15 + RET +.p2align 5 +3: + test esi, 0x2 + je 3f + movups xmm0, xmmword ptr [rcx] + movups xmm1, xmmword ptr [rcx+0x10] + movaps xmm8, xmm0 + movaps xmm9, xmm1 + movd xmm13, dword ptr [rsp+0x110] + movd xmm14, dword ptr [rsp+0x120] + punpckldq xmm13, xmm14 + movaps xmmword ptr [rsp], xmm13 + movd xmm14, dword ptr [rsp+0x114] + movd xmm13, dword ptr [rsp+0x124] + punpckldq xmm14, xmm13 + movaps xmmword ptr [rsp+0x10], xmm14 + mov r8, qword ptr [rdi] + mov r9, qword ptr [rdi+0x8] + movzx eax, byte ptr [rbp+0x40] + or eax, r13d + xor edx, edx +2: + mov r14d, eax + or eax, r12d + add rdx, 64 + cmp rdx, r15 + cmovne eax, r14d + movaps xmm2, xmmword ptr [BLAKE3_IV+rip] + movaps xmm10, xmm2 + movups xmm4, xmmword ptr [r8+rdx-0x40] + movups xmm5, xmmword ptr [r8+rdx-0x30] + movaps xmm3, xmm4 + shufps xmm4, xmm5, 136 + shufps xmm3, xmm5, 221 + movaps xmm5, xmm3 + movups xmm6, xmmword ptr [r8+rdx-0x20] + movups xmm7, xmmword ptr [r8+rdx-0x10] + movaps xmm3, xmm6 + shufps xmm6, xmm7, 136 + pshufd xmm6, xmm6, 0x93 + shufps xmm3, xmm7, 221 + pshufd xmm7, xmm3, 0x93 + movups xmm12, xmmword ptr [r9+rdx-0x40] + movups xmm13, xmmword ptr [r9+rdx-0x30] + movaps xmm11, xmm12 + shufps xmm12, xmm13, 136 + shufps xmm11, xmm13, 221 + movaps xmm13, xmm11 + movups xmm14, xmmword ptr [r9+rdx-0x20] + movups xmm15, xmmword ptr [r9+rdx-0x10] + movaps xmm11, xmm14 + shufps xmm14, xmm15, 136 + pshufd xmm14, xmm14, 0x93 + shufps xmm11, xmm15, 221 + pshufd xmm15, xmm11, 0x93 + shl rax, 0x20 + or rax, 0x40 + movq xmm3, rax + movdqa xmmword ptr [rsp+0x20], xmm3 + movaps xmm3, xmmword ptr [rsp] + movaps xmm11, xmmword ptr [rsp+0x10] + punpcklqdq xmm3, xmmword ptr [rsp+0x20] + punpcklqdq xmm11, xmmword ptr [rsp+0x20] + mov al, 7 +9: + paddd xmm0, xmm4 + paddd xmm8, xmm12 + movaps xmmword ptr [rsp+0x20], xmm4 + movaps xmmword ptr [rsp+0x30], xmm12 + paddd xmm0, xmm1 + paddd xmm8, xmm9 + pxor xmm3, xmm0 + pxor xmm11, xmm8 + pshuflw xmm3, xmm3, 0xB1 + pshufhw xmm3, xmm3, 0xB1 + pshuflw xmm11, xmm11, 0xB1 + pshufhw xmm11, xmm11, 0xB1 + paddd xmm2, xmm3 + paddd xmm10, xmm11 + pxor xmm1, xmm2 + pxor xmm9, xmm10 + movdqa xmm4, xmm1 + pslld xmm1, 20 + psrld xmm4, 12 + por xmm1, xmm4 + movdqa xmm4, xmm9 + pslld xmm9, 20 + psrld xmm4, 12 + por xmm9, xmm4 + paddd xmm0, xmm5 + paddd xmm8, xmm13 + movaps xmmword ptr [rsp+0x40], xmm5 + movaps xmmword ptr [rsp+0x50], xmm13 + paddd xmm0, xmm1 + paddd xmm8, xmm9 + pxor xmm3, xmm0 + pxor xmm11, xmm8 + movdqa xmm13, xmm3 + psrld xmm3, 8 + pslld xmm13, 24 + pxor xmm3, xmm13 + movdqa xmm13, xmm11 + psrld xmm11, 8 + pslld xmm13, 24 + pxor xmm11, xmm13 + paddd xmm2, xmm3 + paddd xmm10, xmm11 + pxor xmm1, xmm2 + pxor xmm9, xmm10 + movdqa xmm4, xmm1 + pslld xmm1, 25 + psrld xmm4, 7 + por xmm1, xmm4 + movdqa xmm4, xmm9 + pslld xmm9, 25 + psrld xmm4, 7 + por xmm9, xmm4 + pshufd xmm0, xmm0, 0x93 + pshufd xmm8, xmm8, 0x93 + pshufd xmm3, xmm3, 0x4E + pshufd xmm11, xmm11, 0x4E + pshufd xmm2, xmm2, 0x39 + pshufd xmm10, xmm10, 0x39 + paddd xmm0, xmm6 + paddd xmm8, xmm14 + paddd xmm0, xmm1 + paddd xmm8, xmm9 + pxor xmm3, xmm0 + pxor xmm11, xmm8 + pshuflw xmm3, xmm3, 0xB1 + pshufhw xmm3, xmm3, 0xB1 + pshuflw xmm11, xmm11, 0xB1 + pshufhw xmm11, xmm11, 0xB1 + paddd xmm2, xmm3 + paddd xmm10, xmm11 + pxor xmm1, xmm2 + pxor xmm9, xmm10 + movdqa xmm4, xmm1 + pslld xmm1, 20 + psrld xmm4, 12 + por xmm1, xmm4 + movdqa xmm4, xmm9 + pslld xmm9, 20 + psrld xmm4, 12 + por xmm9, xmm4 + paddd xmm0, xmm7 + paddd xmm8, xmm15 + paddd xmm0, xmm1 + paddd xmm8, xmm9 + pxor xmm3, xmm0 + pxor xmm11, xmm8 + movdqa xmm13, xmm3 + psrld xmm3, 8 + pslld xmm13, 24 + pxor xmm3, xmm13 + movdqa xmm13, xmm11 + psrld xmm11, 8 + pslld xmm13, 24 + pxor xmm11, xmm13 + paddd xmm2, xmm3 + paddd xmm10, xmm11 + pxor xmm1, xmm2 + pxor xmm9, xmm10 + movdqa xmm4, xmm1 + pslld xmm1, 25 + psrld xmm4, 7 + por xmm1, xmm4 + movdqa xmm4, xmm9 + pslld xmm9, 25 + psrld xmm4, 7 + por xmm9, xmm4 + pshufd xmm0, xmm0, 0x39 + pshufd xmm8, xmm8, 0x39 + pshufd xmm3, xmm3, 0x4E + pshufd xmm11, xmm11, 0x4E + pshufd xmm2, xmm2, 0x93 + pshufd xmm10, xmm10, 0x93 + dec al + je 9f + movdqa xmm12, xmmword ptr [rsp+0x20] + movdqa xmm5, xmmword ptr [rsp+0x40] + pshufd xmm13, xmm12, 0x0F + shufps xmm12, xmm5, 214 + pshufd xmm4, xmm12, 0x39 + movdqa xmm12, xmm6 + shufps xmm12, xmm7, 250 + pand xmm13, xmmword ptr [PBLENDW_0x33_MASK+rip] + pand xmm12, xmmword ptr [PBLENDW_0xCC_MASK+rip] + por xmm13, xmm12 + movdqa xmmword ptr [rsp+0x20], xmm13 + movdqa xmm12, xmm7 + punpcklqdq xmm12, xmm5 + movdqa xmm13, xmm6 + pand xmm12, xmmword ptr [PBLENDW_0x3F_MASK+rip] + pand xmm13, xmmword ptr [PBLENDW_0xC0_MASK+rip] + por xmm12, xmm13 + pshufd xmm12, xmm12, 0x78 + punpckhdq xmm5, xmm7 + punpckldq xmm6, xmm5 + pshufd xmm7, xmm6, 0x1E + movdqa xmmword ptr [rsp+0x40], xmm12 + movdqa xmm5, xmmword ptr [rsp+0x30] + movdqa xmm13, xmmword ptr [rsp+0x50] + pshufd xmm6, xmm5, 0x0F + shufps xmm5, xmm13, 214 + pshufd xmm12, xmm5, 0x39 + movdqa xmm5, xmm14 + shufps xmm5, xmm15, 250 + pand xmm6, xmmword ptr [PBLENDW_0x33_MASK+rip] + pand xmm5, xmmword ptr [PBLENDW_0xCC_MASK+rip] + por xmm6, xmm5 + movdqa xmm5, xmm15 + punpcklqdq xmm5, xmm13 + movdqa xmmword ptr [rsp+0x30], xmm2 + movdqa xmm2, xmm14 + pand xmm5, xmmword ptr [PBLENDW_0x3F_MASK+rip] + pand xmm2, xmmword ptr [PBLENDW_0xC0_MASK+rip] + por xmm5, xmm2 + movdqa xmm2, xmmword ptr [rsp+0x30] + pshufd xmm5, xmm5, 0x78 + punpckhdq xmm13, xmm15 + punpckldq xmm14, xmm13 + pshufd xmm15, xmm14, 0x1E + movdqa xmm13, xmm6 + movdqa xmm14, xmm5 + movdqa xmm5, xmmword ptr [rsp+0x20] + movdqa xmm6, xmmword ptr [rsp+0x40] + jmp 9b +9: + pxor xmm0, xmm2 + pxor xmm1, xmm3 + pxor xmm8, xmm10 + pxor xmm9, xmm11 + mov eax, r13d + cmp rdx, r15 + jne 2b + movups xmmword ptr [rbx], xmm0 + movups xmmword ptr [rbx+0x10], xmm1 + movups xmmword ptr [rbx+0x20], xmm8 + movups xmmword ptr [rbx+0x30], xmm9 + mov eax, dword ptr [rsp+0x130] + neg eax + mov r10d, dword ptr [rsp+0x110+8*rax] + mov r11d, dword ptr [rsp+0x120+8*rax] + mov dword ptr [rsp+0x110], r10d + mov dword ptr [rsp+0x120], r11d + add rdi, 16 + add rbx, 64 + sub rsi, 2 +3: + test esi, 0x1 + je 4b + movups xmm0, xmmword ptr [rcx] + movups xmm1, xmmword ptr [rcx+0x10] + movd xmm13, dword ptr [rsp+0x110] + movd xmm14, dword ptr [rsp+0x120] + punpckldq xmm13, xmm14 + mov r8, qword ptr [rdi] + movzx eax, byte ptr [rbp+0x40] + or eax, r13d + xor edx, edx +2: + mov r14d, eax + or eax, r12d + add rdx, 64 + cmp rdx, r15 + cmovne eax, r14d + movaps xmm2, xmmword ptr [BLAKE3_IV+rip] + shl rax, 32 + or rax, 64 + movq xmm12, rax + movdqa xmm3, xmm13 + punpcklqdq xmm3, xmm12 + movups xmm4, xmmword ptr [r8+rdx-0x40] + movups xmm5, xmmword ptr [r8+rdx-0x30] + movaps xmm8, xmm4 + shufps xmm4, xmm5, 136 + shufps xmm8, xmm5, 221 + movaps xmm5, xmm8 + movups xmm6, xmmword ptr [r8+rdx-0x20] + movups xmm7, xmmword ptr [r8+rdx-0x10] + movaps xmm8, xmm6 + shufps xmm6, xmm7, 136 + pshufd xmm6, xmm6, 0x93 + shufps xmm8, xmm7, 221 + pshufd xmm7, xmm8, 0x93 + mov al, 7 +9: + paddd xmm0, xmm4 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshuflw xmm3, xmm3, 0xB1 + pshufhw xmm3, xmm3, 0xB1 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 20 + psrld xmm11, 12 + por xmm1, xmm11 + paddd xmm0, xmm5 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + movdqa xmm14, xmm3 + psrld xmm3, 8 + pslld xmm14, 24 + pxor xmm3, xmm14 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 25 + psrld xmm11, 7 + por xmm1, xmm11 + pshufd xmm0, xmm0, 0x93 + pshufd xmm3, xmm3, 0x4E + pshufd xmm2, xmm2, 0x39 + paddd xmm0, xmm6 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshuflw xmm3, xmm3, 0xB1 + pshufhw xmm3, xmm3, 0xB1 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 20 + psrld xmm11, 12 + por xmm1, xmm11 + paddd xmm0, xmm7 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + movdqa xmm14, xmm3 + psrld xmm3, 8 + pslld xmm14, 24 + pxor xmm3, xmm14 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 25 + psrld xmm11, 7 + por xmm1, xmm11 + pshufd xmm0, xmm0, 0x39 + pshufd xmm3, xmm3, 0x4E + pshufd xmm2, xmm2, 0x93 + dec al + jz 9f + movdqa xmm8, xmm4 + shufps xmm8, xmm5, 214 + pshufd xmm9, xmm4, 0x0F + pshufd xmm4, xmm8, 0x39 + movdqa xmm8, xmm6 + shufps xmm8, xmm7, 250 + pand xmm9, xmmword ptr [PBLENDW_0x33_MASK+rip] + pand xmm8, xmmword ptr [PBLENDW_0xCC_MASK+rip] + por xmm9, xmm8 + movdqa xmm8, xmm7 + punpcklqdq xmm8, xmm5 + movdqa xmm10, xmm6 + pand xmm8, xmmword ptr [PBLENDW_0x3F_MASK+rip] + pand xmm10, xmmword ptr [PBLENDW_0xC0_MASK+rip] + por xmm8, xmm10 + pshufd xmm8, xmm8, 0x78 + punpckhdq xmm5, xmm7 + punpckldq xmm6, xmm5 + pshufd xmm7, xmm6, 0x1E + movdqa xmm5, xmm9 + movdqa xmm6, xmm8 + jmp 9b +9: + pxor xmm0, xmm2 + pxor xmm1, xmm3 + mov eax, r13d + cmp rdx, r15 + jne 2b + movups xmmword ptr [rbx], xmm0 + movups xmmword ptr [rbx+0x10], xmm1 + jmp 4b + +.p2align 6 +zfs_blake3_compress_in_place_sse2: + _CET_ENDBR + movups xmm0, xmmword ptr [rdi] + movups xmm1, xmmword ptr [rdi+0x10] + movaps xmm2, xmmword ptr [BLAKE3_IV+rip] + shl r8, 32 + add rdx, r8 + movq xmm3, rcx + movq xmm4, rdx + punpcklqdq xmm3, xmm4 + movups xmm4, xmmword ptr [rsi] + movups xmm5, xmmword ptr [rsi+0x10] + movaps xmm8, xmm4 + shufps xmm4, xmm5, 136 + shufps xmm8, xmm5, 221 + movaps xmm5, xmm8 + movups xmm6, xmmword ptr [rsi+0x20] + movups xmm7, xmmword ptr [rsi+0x30] + movaps xmm8, xmm6 + shufps xmm6, xmm7, 136 + pshufd xmm6, xmm6, 0x93 + shufps xmm8, xmm7, 221 + pshufd xmm7, xmm8, 0x93 + mov al, 7 +9: + paddd xmm0, xmm4 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshuflw xmm3, xmm3, 0xB1 + pshufhw xmm3, xmm3, 0xB1 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 20 + psrld xmm11, 12 + por xmm1, xmm11 + paddd xmm0, xmm5 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + movdqa xmm14, xmm3 + psrld xmm3, 8 + pslld xmm14, 24 + pxor xmm3, xmm14 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 25 + psrld xmm11, 7 + por xmm1, xmm11 + pshufd xmm0, xmm0, 0x93 + pshufd xmm3, xmm3, 0x4E + pshufd xmm2, xmm2, 0x39 + paddd xmm0, xmm6 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshuflw xmm3, xmm3, 0xB1 + pshufhw xmm3, xmm3, 0xB1 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 20 + psrld xmm11, 12 + por xmm1, xmm11 + paddd xmm0, xmm7 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + movdqa xmm14, xmm3 + psrld xmm3, 8 + pslld xmm14, 24 + pxor xmm3, xmm14 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 25 + psrld xmm11, 7 + por xmm1, xmm11 + pshufd xmm0, xmm0, 0x39 + pshufd xmm3, xmm3, 0x4E + pshufd xmm2, xmm2, 0x93 + dec al + jz 9f + movdqa xmm8, xmm4 + shufps xmm8, xmm5, 214 + pshufd xmm9, xmm4, 0x0F + pshufd xmm4, xmm8, 0x39 + movdqa xmm8, xmm6 + shufps xmm8, xmm7, 250 + pand xmm9, xmmword ptr [PBLENDW_0x33_MASK+rip] + pand xmm8, xmmword ptr [PBLENDW_0xCC_MASK+rip] + por xmm9, xmm8 + movdqa xmm8, xmm7 + punpcklqdq xmm8, xmm5 + movdqa xmm10, xmm6 + pand xmm8, xmmword ptr [PBLENDW_0x3F_MASK+rip] + pand xmm10, xmmword ptr [PBLENDW_0xC0_MASK+rip] + por xmm8, xmm10 + pshufd xmm8, xmm8, 0x78 + punpckhdq xmm5, xmm7 + punpckldq xmm6, xmm5 + pshufd xmm7, xmm6, 0x1E + movdqa xmm5, xmm9 + movdqa xmm6, xmm8 + jmp 9b +9: + pxor xmm0, xmm2 + pxor xmm1, xmm3 + movups xmmword ptr [rdi], xmm0 + movups xmmword ptr [rdi+0x10], xmm1 + RET + +.p2align 6 +zfs_blake3_compress_xof_sse2: + _CET_ENDBR + movups xmm0, xmmword ptr [rdi] + movups xmm1, xmmword ptr [rdi+0x10] + movaps xmm2, xmmword ptr [BLAKE3_IV+rip] + movzx eax, r8b + movzx edx, dl + shl rax, 32 + add rdx, rax + movq xmm3, rcx + movq xmm4, rdx + punpcklqdq xmm3, xmm4 + movups xmm4, xmmword ptr [rsi] + movups xmm5, xmmword ptr [rsi+0x10] + movaps xmm8, xmm4 + shufps xmm4, xmm5, 136 + shufps xmm8, xmm5, 221 + movaps xmm5, xmm8 + movups xmm6, xmmword ptr [rsi+0x20] + movups xmm7, xmmword ptr [rsi+0x30] + movaps xmm8, xmm6 + shufps xmm6, xmm7, 136 + pshufd xmm6, xmm6, 0x93 + shufps xmm8, xmm7, 221 + pshufd xmm7, xmm8, 0x93 + mov al, 7 +9: + paddd xmm0, xmm4 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshuflw xmm3, xmm3, 0xB1 + pshufhw xmm3, xmm3, 0xB1 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 20 + psrld xmm11, 12 + por xmm1, xmm11 + paddd xmm0, xmm5 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + movdqa xmm14, xmm3 + psrld xmm3, 8 + pslld xmm14, 24 + pxor xmm3, xmm14 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 25 + psrld xmm11, 7 + por xmm1, xmm11 + pshufd xmm0, xmm0, 0x93 + pshufd xmm3, xmm3, 0x4E + pshufd xmm2, xmm2, 0x39 + paddd xmm0, xmm6 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshuflw xmm3, xmm3, 0xB1 + pshufhw xmm3, xmm3, 0xB1 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 20 + psrld xmm11, 12 + por xmm1, xmm11 + paddd xmm0, xmm7 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + movdqa xmm14, xmm3 + psrld xmm3, 8 + pslld xmm14, 24 + pxor xmm3, xmm14 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 25 + psrld xmm11, 7 + por xmm1, xmm11 + pshufd xmm0, xmm0, 0x39 + pshufd xmm3, xmm3, 0x4E + pshufd xmm2, xmm2, 0x93 + dec al + jz 9f + movdqa xmm8, xmm4 + shufps xmm8, xmm5, 214 + pshufd xmm9, xmm4, 0x0F + pshufd xmm4, xmm8, 0x39 + movdqa xmm8, xmm6 + shufps xmm8, xmm7, 250 + pand xmm9, xmmword ptr [PBLENDW_0x33_MASK+rip] + pand xmm8, xmmword ptr [PBLENDW_0xCC_MASK+rip] + por xmm9, xmm8 + movdqa xmm8, xmm7 + punpcklqdq xmm8, xmm5 + movdqa xmm10, xmm6 + pand xmm8, xmmword ptr [PBLENDW_0x3F_MASK+rip] + pand xmm10, xmmword ptr [PBLENDW_0xC0_MASK+rip] + por xmm8, xmm10 + pshufd xmm8, xmm8, 0x78 + punpckhdq xmm5, xmm7 + punpckldq xmm6, xmm5 + pshufd xmm7, xmm6, 0x1E + movdqa xmm5, xmm9 + movdqa xmm6, xmm8 + jmp 9b +9: + movdqu xmm4, xmmword ptr [rdi] + movdqu xmm5, xmmword ptr [rdi+0x10] + pxor xmm0, xmm2 + pxor xmm1, xmm3 + pxor xmm2, xmm4 + pxor xmm3, xmm5 + movups xmmword ptr [r9], xmm0 + movups xmmword ptr [r9+0x10], xmm1 + movups xmmword ptr [r9+0x20], xmm2 + movups xmmword ptr [r9+0x30], xmm3 + RET + +.size zfs_blake3_hash_many_sse2, . - zfs_blake3_hash_many_sse2 +.size zfs_blake3_compress_in_place_sse2, . - zfs_blake3_compress_in_place_sse2 +.size zfs_blake3_compress_xof_sse2, . - zfs_blake3_compress_xof_sse2 + +#ifdef __APPLE__ +.static_data +#else +.section .rodata +#endif +.p2align 6 +BLAKE3_IV: + .long 0x6A09E667, 0xBB67AE85 + .long 0x3C6EF372, 0xA54FF53A +ADD0: + .long 0, 1, 2, 3 +ADD1: + .long 4, 4, 4, 4 +BLAKE3_IV_0: + .long 0x6A09E667, 0x6A09E667, 0x6A09E667, 0x6A09E667 +BLAKE3_IV_1: + .long 0xBB67AE85, 0xBB67AE85, 0xBB67AE85, 0xBB67AE85 +BLAKE3_IV_2: + .long 0x3C6EF372, 0x3C6EF372, 0x3C6EF372, 0x3C6EF372 +BLAKE3_IV_3: + .long 0xA54FF53A, 0xA54FF53A, 0xA54FF53A, 0xA54FF53A +BLAKE3_BLOCK_LEN: + .long 64, 64, 64, 64 +CMP_MSB_MASK: + .long 0x80000000, 0x80000000, 0x80000000, 0x80000000 +PBLENDW_0x33_MASK: + .long 0xFFFFFFFF, 0x00000000, 0xFFFFFFFF, 0x00000000 +PBLENDW_0xCC_MASK: + .long 0x00000000, 0xFFFFFFFF, 0x00000000, 0xFFFFFFFF +PBLENDW_0x3F_MASK: + .long 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0x00000000 +PBLENDW_0xC0_MASK: + .long 0x00000000, 0x00000000, 0x00000000, 0xFFFFFFFF + +#endif /* HAVE_SSE2 */ + +#ifdef __ELF__ +.section .note.GNU-stack,"",%progbits +#endif diff --git a/module/icp/asm-x86_64/blake3/blake3_sse41.S b/module/icp/asm-x86_64/blake3/blake3_sse41.S new file mode 100644 index 000000000000..c5975a4f0877 --- /dev/null +++ b/module/icp/asm-x86_64/blake3/blake3_sse41.S @@ -0,0 +1,2058 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Based on BLAKE3 v1.3.1, https://github.com/BLAKE3-team/BLAKE3 + * Copyright (c) 2019-2020 Samuel Neves + * Copyright (c) 2022 Tino Reichardt + */ + +#if defined(HAVE_SSE4_1) + +#define _ASM +#include + +#if defined(__ELF__) && defined(__CET__) && defined(__has_include) +#if __has_include() +#include +#endif +#endif + +#if !defined(_CET_ENDBR) +#define _CET_ENDBR +#endif + +.intel_syntax noprefix +.global zfs_blake3_compress_in_place_sse41 +.global zfs_blake3_compress_xof_sse41 +.global zfs_blake3_hash_many_sse41 + +.text +.type zfs_blake3_hash_many_sse41,@function +.type zfs_blake3_compress_in_place_sse41,@function +.type zfs_blake3_compress_xof_sse41,@function + +.p2align 6 +zfs_blake3_hash_many_sse41: + _CET_ENDBR + push r15 + push r14 + push r13 + push r12 + push rbx + push rbp + mov rbp, rsp + sub rsp, 360 + and rsp, 0xFFFFFFFFFFFFFFC0 + neg r9d + movd xmm0, r9d + pshufd xmm0, xmm0, 0x00 + movdqa xmmword ptr [rsp+0x130], xmm0 + movdqa xmm1, xmm0 + pand xmm1, xmmword ptr [ADD0+rip] + pand xmm0, xmmword ptr [ADD1+rip] + movdqa xmmword ptr [rsp+0x150], xmm0 + movd xmm0, r8d + pshufd xmm0, xmm0, 0x00 + paddd xmm0, xmm1 + movdqa xmmword ptr [rsp+0x110], xmm0 + pxor xmm0, xmmword ptr [CMP_MSB_MASK+rip] + pxor xmm1, xmmword ptr [CMP_MSB_MASK+rip] + pcmpgtd xmm1, xmm0 + shr r8, 32 + movd xmm2, r8d + pshufd xmm2, xmm2, 0x00 + psubd xmm2, xmm1 + movdqa xmmword ptr [rsp+0x120], xmm2 + mov rbx, qword ptr [rbp+0x50] + mov r15, rdx + shl r15, 6 + movzx r13d, byte ptr [rbp+0x38] + movzx r12d, byte ptr [rbp+0x48] + cmp rsi, 4 + jc 3f +2: + movdqu xmm3, xmmword ptr [rcx] + pshufd xmm0, xmm3, 0x00 + pshufd xmm1, xmm3, 0x55 + pshufd xmm2, xmm3, 0xAA + pshufd xmm3, xmm3, 0xFF + movdqu xmm7, xmmword ptr [rcx+0x10] + pshufd xmm4, xmm7, 0x00 + pshufd xmm5, xmm7, 0x55 + pshufd xmm6, xmm7, 0xAA + pshufd xmm7, xmm7, 0xFF + mov r8, qword ptr [rdi] + mov r9, qword ptr [rdi+0x8] + mov r10, qword ptr [rdi+0x10] + mov r11, qword ptr [rdi+0x18] + movzx eax, byte ptr [rbp+0x40] + or eax, r13d + xor edx, edx +9: + mov r14d, eax + or eax, r12d + add rdx, 64 + cmp rdx, r15 + cmovne eax, r14d + movdqu xmm8, xmmword ptr [r8+rdx-0x40] + movdqu xmm9, xmmword ptr [r9+rdx-0x40] + movdqu xmm10, xmmword ptr [r10+rdx-0x40] + movdqu xmm11, xmmword ptr [r11+rdx-0x40] + movdqa xmm12, xmm8 + punpckldq xmm8, xmm9 + punpckhdq xmm12, xmm9 + movdqa xmm14, xmm10 + punpckldq xmm10, xmm11 + punpckhdq xmm14, xmm11 + movdqa xmm9, xmm8 + punpcklqdq xmm8, xmm10 + punpckhqdq xmm9, xmm10 + movdqa xmm13, xmm12 + punpcklqdq xmm12, xmm14 + punpckhqdq xmm13, xmm14 + movdqa xmmword ptr [rsp], xmm8 + movdqa xmmword ptr [rsp+0x10], xmm9 + movdqa xmmword ptr [rsp+0x20], xmm12 + movdqa xmmword ptr [rsp+0x30], xmm13 + movdqu xmm8, xmmword ptr [r8+rdx-0x30] + movdqu xmm9, xmmword ptr [r9+rdx-0x30] + movdqu xmm10, xmmword ptr [r10+rdx-0x30] + movdqu xmm11, xmmword ptr [r11+rdx-0x30] + movdqa xmm12, xmm8 + punpckldq xmm8, xmm9 + punpckhdq xmm12, xmm9 + movdqa xmm14, xmm10 + punpckldq xmm10, xmm11 + punpckhdq xmm14, xmm11 + movdqa xmm9, xmm8 + punpcklqdq xmm8, xmm10 + punpckhqdq xmm9, xmm10 + movdqa xmm13, xmm12 + punpcklqdq xmm12, xmm14 + punpckhqdq xmm13, xmm14 + movdqa xmmword ptr [rsp+0x40], xmm8 + movdqa xmmword ptr [rsp+0x50], xmm9 + movdqa xmmword ptr [rsp+0x60], xmm12 + movdqa xmmword ptr [rsp+0x70], xmm13 + movdqu xmm8, xmmword ptr [r8+rdx-0x20] + movdqu xmm9, xmmword ptr [r9+rdx-0x20] + movdqu xmm10, xmmword ptr [r10+rdx-0x20] + movdqu xmm11, xmmword ptr [r11+rdx-0x20] + movdqa xmm12, xmm8 + punpckldq xmm8, xmm9 + punpckhdq xmm12, xmm9 + movdqa xmm14, xmm10 + punpckldq xmm10, xmm11 + punpckhdq xmm14, xmm11 + movdqa xmm9, xmm8 + punpcklqdq xmm8, xmm10 + punpckhqdq xmm9, xmm10 + movdqa xmm13, xmm12 + punpcklqdq xmm12, xmm14 + punpckhqdq xmm13, xmm14 + movdqa xmmword ptr [rsp+0x80], xmm8 + movdqa xmmword ptr [rsp+0x90], xmm9 + movdqa xmmword ptr [rsp+0xA0], xmm12 + movdqa xmmword ptr [rsp+0xB0], xmm13 + movdqu xmm8, xmmword ptr [r8+rdx-0x10] + movdqu xmm9, xmmword ptr [r9+rdx-0x10] + movdqu xmm10, xmmword ptr [r10+rdx-0x10] + movdqu xmm11, xmmword ptr [r11+rdx-0x10] + movdqa xmm12, xmm8 + punpckldq xmm8, xmm9 + punpckhdq xmm12, xmm9 + movdqa xmm14, xmm10 + punpckldq xmm10, xmm11 + punpckhdq xmm14, xmm11 + movdqa xmm9, xmm8 + punpcklqdq xmm8, xmm10 + punpckhqdq xmm9, xmm10 + movdqa xmm13, xmm12 + punpcklqdq xmm12, xmm14 + punpckhqdq xmm13, xmm14 + movdqa xmmword ptr [rsp+0xC0], xmm8 + movdqa xmmword ptr [rsp+0xD0], xmm9 + movdqa xmmword ptr [rsp+0xE0], xmm12 + movdqa xmmword ptr [rsp+0xF0], xmm13 + movdqa xmm9, xmmword ptr [BLAKE3_IV_1+rip] + movdqa xmm10, xmmword ptr [BLAKE3_IV_2+rip] + movdqa xmm11, xmmword ptr [BLAKE3_IV_3+rip] + movdqa xmm12, xmmword ptr [rsp+0x110] + movdqa xmm13, xmmword ptr [rsp+0x120] + movdqa xmm14, xmmword ptr [BLAKE3_BLOCK_LEN+rip] + movd xmm15, eax + pshufd xmm15, xmm15, 0x00 + prefetcht0 [r8+rdx+0x80] + prefetcht0 [r9+rdx+0x80] + prefetcht0 [r10+rdx+0x80] + prefetcht0 [r11+rdx+0x80] + paddd xmm0, xmmword ptr [rsp] + paddd xmm1, xmmword ptr [rsp+0x20] + paddd xmm2, xmmword ptr [rsp+0x40] + paddd xmm3, xmmword ptr [rsp+0x60] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmmword ptr [ROT16+rip] + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + pshufb xmm15, xmm8 + movdqa xmm8, xmmword ptr [BLAKE3_IV_0+rip] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x10] + paddd xmm1, xmmword ptr [rsp+0x30] + paddd xmm2, xmmword ptr [rsp+0x50] + paddd xmm3, xmmword ptr [rsp+0x70] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmmword ptr [ROT8+rip] + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + pshufb xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x80] + paddd xmm1, xmmword ptr [rsp+0xA0] + paddd xmm2, xmmword ptr [rsp+0xC0] + paddd xmm3, xmmword ptr [rsp+0xE0] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmmword ptr [ROT16+rip] + pshufb xmm15, xmm8 + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0x90] + paddd xmm1, xmmword ptr [rsp+0xB0] + paddd xmm2, xmmword ptr [rsp+0xD0] + paddd xmm3, xmmword ptr [rsp+0xF0] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmmword ptr [ROT8+rip] + pshufb xmm15, xmm8 + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0x20] + paddd xmm1, xmmword ptr [rsp+0x30] + paddd xmm2, xmmword ptr [rsp+0x70] + paddd xmm3, xmmword ptr [rsp+0x40] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmmword ptr [ROT16+rip] + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + pshufb xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x60] + paddd xmm1, xmmword ptr [rsp+0xA0] + paddd xmm2, xmmword ptr [rsp] + paddd xmm3, xmmword ptr [rsp+0xD0] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmmword ptr [ROT8+rip] + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + pshufb xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x10] + paddd xmm1, xmmword ptr [rsp+0xC0] + paddd xmm2, xmmword ptr [rsp+0x90] + paddd xmm3, xmmword ptr [rsp+0xF0] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmmword ptr [ROT16+rip] + pshufb xmm15, xmm8 + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0xB0] + paddd xmm1, xmmword ptr [rsp+0x50] + paddd xmm2, xmmword ptr [rsp+0xE0] + paddd xmm3, xmmword ptr [rsp+0x80] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmmword ptr [ROT8+rip] + pshufb xmm15, xmm8 + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0x30] + paddd xmm1, xmmword ptr [rsp+0xA0] + paddd xmm2, xmmword ptr [rsp+0xD0] + paddd xmm3, xmmword ptr [rsp+0x70] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmmword ptr [ROT16+rip] + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + pshufb xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x40] + paddd xmm1, xmmword ptr [rsp+0xC0] + paddd xmm2, xmmword ptr [rsp+0x20] + paddd xmm3, xmmword ptr [rsp+0xE0] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmmword ptr [ROT8+rip] + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + pshufb xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x60] + paddd xmm1, xmmword ptr [rsp+0x90] + paddd xmm2, xmmword ptr [rsp+0xB0] + paddd xmm3, xmmword ptr [rsp+0x80] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmmword ptr [ROT16+rip] + pshufb xmm15, xmm8 + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0x50] + paddd xmm1, xmmword ptr [rsp] + paddd xmm2, xmmword ptr [rsp+0xF0] + paddd xmm3, xmmword ptr [rsp+0x10] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmmword ptr [ROT8+rip] + pshufb xmm15, xmm8 + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0xA0] + paddd xmm1, xmmword ptr [rsp+0xC0] + paddd xmm2, xmmword ptr [rsp+0xE0] + paddd xmm3, xmmword ptr [rsp+0xD0] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmmword ptr [ROT16+rip] + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + pshufb xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x70] + paddd xmm1, xmmword ptr [rsp+0x90] + paddd xmm2, xmmword ptr [rsp+0x30] + paddd xmm3, xmmword ptr [rsp+0xF0] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmmword ptr [ROT8+rip] + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + pshufb xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x40] + paddd xmm1, xmmword ptr [rsp+0xB0] + paddd xmm2, xmmword ptr [rsp+0x50] + paddd xmm3, xmmword ptr [rsp+0x10] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmmword ptr [ROT16+rip] + pshufb xmm15, xmm8 + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp] + paddd xmm1, xmmword ptr [rsp+0x20] + paddd xmm2, xmmword ptr [rsp+0x80] + paddd xmm3, xmmword ptr [rsp+0x60] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmmword ptr [ROT8+rip] + pshufb xmm15, xmm8 + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0xC0] + paddd xmm1, xmmword ptr [rsp+0x90] + paddd xmm2, xmmword ptr [rsp+0xF0] + paddd xmm3, xmmword ptr [rsp+0xE0] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmmword ptr [ROT16+rip] + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + pshufb xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0xD0] + paddd xmm1, xmmword ptr [rsp+0xB0] + paddd xmm2, xmmword ptr [rsp+0xA0] + paddd xmm3, xmmword ptr [rsp+0x80] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmmword ptr [ROT8+rip] + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + pshufb xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0x70] + paddd xmm1, xmmword ptr [rsp+0x50] + paddd xmm2, xmmword ptr [rsp] + paddd xmm3, xmmword ptr [rsp+0x60] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmmword ptr [ROT16+rip] + pshufb xmm15, xmm8 + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0x20] + paddd xmm1, xmmword ptr [rsp+0x30] + paddd xmm2, xmmword ptr [rsp+0x10] + paddd xmm3, xmmword ptr [rsp+0x40] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmmword ptr [ROT8+rip] + pshufb xmm15, xmm8 + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0x90] + paddd xmm1, xmmword ptr [rsp+0xB0] + paddd xmm2, xmmword ptr [rsp+0x80] + paddd xmm3, xmmword ptr [rsp+0xF0] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmmword ptr [ROT16+rip] + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + pshufb xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0xE0] + paddd xmm1, xmmword ptr [rsp+0x50] + paddd xmm2, xmmword ptr [rsp+0xC0] + paddd xmm3, xmmword ptr [rsp+0x10] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmmword ptr [ROT8+rip] + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + pshufb xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0xD0] + paddd xmm1, xmmword ptr [rsp] + paddd xmm2, xmmword ptr [rsp+0x20] + paddd xmm3, xmmword ptr [rsp+0x40] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmmword ptr [ROT16+rip] + pshufb xmm15, xmm8 + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0x30] + paddd xmm1, xmmword ptr [rsp+0xA0] + paddd xmm2, xmmword ptr [rsp+0x60] + paddd xmm3, xmmword ptr [rsp+0x70] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmmword ptr [ROT8+rip] + pshufb xmm15, xmm8 + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0xB0] + paddd xmm1, xmmword ptr [rsp+0x50] + paddd xmm2, xmmword ptr [rsp+0x10] + paddd xmm3, xmmword ptr [rsp+0x80] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmmword ptr [ROT16+rip] + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + pshufb xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0xF0] + paddd xmm1, xmmword ptr [rsp] + paddd xmm2, xmmword ptr [rsp+0x90] + paddd xmm3, xmmword ptr [rsp+0x60] + paddd xmm0, xmm4 + paddd xmm1, xmm5 + paddd xmm2, xmm6 + paddd xmm3, xmm7 + pxor xmm12, xmm0 + pxor xmm13, xmm1 + pxor xmm14, xmm2 + pxor xmm15, xmm3 + movdqa xmm8, xmmword ptr [ROT8+rip] + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + pshufb xmm15, xmm8 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm12 + paddd xmm9, xmm13 + paddd xmm10, xmm14 + paddd xmm11, xmm15 + pxor xmm4, xmm8 + pxor xmm5, xmm9 + pxor xmm6, xmm10 + pxor xmm7, xmm11 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + paddd xmm0, xmmword ptr [rsp+0xE0] + paddd xmm1, xmmword ptr [rsp+0x20] + paddd xmm2, xmmword ptr [rsp+0x30] + paddd xmm3, xmmword ptr [rsp+0x70] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmmword ptr [ROT16+rip] + pshufb xmm15, xmm8 + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + movdqa xmmword ptr [rsp+0x100], xmm8 + movdqa xmm8, xmm5 + psrld xmm8, 12 + pslld xmm5, 20 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 12 + pslld xmm6, 20 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 12 + pslld xmm7, 20 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 12 + pslld xmm4, 20 + por xmm4, xmm8 + paddd xmm0, xmmword ptr [rsp+0xA0] + paddd xmm1, xmmword ptr [rsp+0xC0] + paddd xmm2, xmmword ptr [rsp+0x40] + paddd xmm3, xmmword ptr [rsp+0xD0] + paddd xmm0, xmm5 + paddd xmm1, xmm6 + paddd xmm2, xmm7 + paddd xmm3, xmm4 + pxor xmm15, xmm0 + pxor xmm12, xmm1 + pxor xmm13, xmm2 + pxor xmm14, xmm3 + movdqa xmm8, xmmword ptr [ROT8+rip] + pshufb xmm15, xmm8 + pshufb xmm12, xmm8 + pshufb xmm13, xmm8 + pshufb xmm14, xmm8 + paddd xmm10, xmm15 + paddd xmm11, xmm12 + movdqa xmm8, xmmword ptr [rsp+0x100] + paddd xmm8, xmm13 + paddd xmm9, xmm14 + pxor xmm5, xmm10 + pxor xmm6, xmm11 + pxor xmm7, xmm8 + pxor xmm4, xmm9 + pxor xmm0, xmm8 + pxor xmm1, xmm9 + pxor xmm2, xmm10 + pxor xmm3, xmm11 + movdqa xmm8, xmm5 + psrld xmm8, 7 + pslld xmm5, 25 + por xmm5, xmm8 + movdqa xmm8, xmm6 + psrld xmm8, 7 + pslld xmm6, 25 + por xmm6, xmm8 + movdqa xmm8, xmm7 + psrld xmm8, 7 + pslld xmm7, 25 + por xmm7, xmm8 + movdqa xmm8, xmm4 + psrld xmm8, 7 + pslld xmm4, 25 + por xmm4, xmm8 + pxor xmm4, xmm12 + pxor xmm5, xmm13 + pxor xmm6, xmm14 + pxor xmm7, xmm15 + mov eax, r13d + jne 9b + movdqa xmm9, xmm0 + punpckldq xmm0, xmm1 + punpckhdq xmm9, xmm1 + movdqa xmm11, xmm2 + punpckldq xmm2, xmm3 + punpckhdq xmm11, xmm3 + movdqa xmm1, xmm0 + punpcklqdq xmm0, xmm2 + punpckhqdq xmm1, xmm2 + movdqa xmm3, xmm9 + punpcklqdq xmm9, xmm11 + punpckhqdq xmm3, xmm11 + movdqu xmmword ptr [rbx], xmm0 + movdqu xmmword ptr [rbx+0x20], xmm1 + movdqu xmmword ptr [rbx+0x40], xmm9 + movdqu xmmword ptr [rbx+0x60], xmm3 + movdqa xmm9, xmm4 + punpckldq xmm4, xmm5 + punpckhdq xmm9, xmm5 + movdqa xmm11, xmm6 + punpckldq xmm6, xmm7 + punpckhdq xmm11, xmm7 + movdqa xmm5, xmm4 + punpcklqdq xmm4, xmm6 + punpckhqdq xmm5, xmm6 + movdqa xmm7, xmm9 + punpcklqdq xmm9, xmm11 + punpckhqdq xmm7, xmm11 + movdqu xmmword ptr [rbx+0x10], xmm4 + movdqu xmmword ptr [rbx+0x30], xmm5 + movdqu xmmword ptr [rbx+0x50], xmm9 + movdqu xmmword ptr [rbx+0x70], xmm7 + movdqa xmm1, xmmword ptr [rsp+0x110] + movdqa xmm0, xmm1 + paddd xmm1, xmmword ptr [rsp+0x150] + movdqa xmmword ptr [rsp+0x110], xmm1 + pxor xmm0, xmmword ptr [CMP_MSB_MASK+rip] + pxor xmm1, xmmword ptr [CMP_MSB_MASK+rip] + pcmpgtd xmm0, xmm1 + movdqa xmm1, xmmword ptr [rsp+0x120] + psubd xmm1, xmm0 + movdqa xmmword ptr [rsp+0x120], xmm1 + add rbx, 128 + add rdi, 32 + sub rsi, 4 + cmp rsi, 4 + jnc 2b + test rsi, rsi + jnz 3f +4: + mov rsp, rbp + pop rbp + pop rbx + pop r12 + pop r13 + pop r14 + pop r15 + RET +.p2align 5 +3: + test esi, 0x2 + je 3f + movups xmm0, xmmword ptr [rcx] + movups xmm1, xmmword ptr [rcx+0x10] + movaps xmm8, xmm0 + movaps xmm9, xmm1 + movd xmm13, dword ptr [rsp+0x110] + pinsrd xmm13, dword ptr [rsp+0x120], 1 + pinsrd xmm13, dword ptr [BLAKE3_BLOCK_LEN+rip], 2 + movaps xmmword ptr [rsp], xmm13 + movd xmm14, dword ptr [rsp+0x114] + pinsrd xmm14, dword ptr [rsp+0x124], 1 + pinsrd xmm14, dword ptr [BLAKE3_BLOCK_LEN+rip], 2 + movaps xmmword ptr [rsp+0x10], xmm14 + mov r8, qword ptr [rdi] + mov r9, qword ptr [rdi+0x8] + movzx eax, byte ptr [rbp+0x40] + or eax, r13d + xor edx, edx +2: + mov r14d, eax + or eax, r12d + add rdx, 64 + cmp rdx, r15 + cmovne eax, r14d + movaps xmm2, xmmword ptr [BLAKE3_IV+rip] + movaps xmm10, xmm2 + movups xmm4, xmmword ptr [r8+rdx-0x40] + movups xmm5, xmmword ptr [r8+rdx-0x30] + movaps xmm3, xmm4 + shufps xmm4, xmm5, 136 + shufps xmm3, xmm5, 221 + movaps xmm5, xmm3 + movups xmm6, xmmword ptr [r8+rdx-0x20] + movups xmm7, xmmword ptr [r8+rdx-0x10] + movaps xmm3, xmm6 + shufps xmm6, xmm7, 136 + pshufd xmm6, xmm6, 0x93 + shufps xmm3, xmm7, 221 + pshufd xmm7, xmm3, 0x93 + movups xmm12, xmmword ptr [r9+rdx-0x40] + movups xmm13, xmmword ptr [r9+rdx-0x30] + movaps xmm11, xmm12 + shufps xmm12, xmm13, 136 + shufps xmm11, xmm13, 221 + movaps xmm13, xmm11 + movups xmm14, xmmword ptr [r9+rdx-0x20] + movups xmm15, xmmword ptr [r9+rdx-0x10] + movaps xmm11, xmm14 + shufps xmm14, xmm15, 136 + pshufd xmm14, xmm14, 0x93 + shufps xmm11, xmm15, 221 + pshufd xmm15, xmm11, 0x93 + movaps xmm3, xmmword ptr [rsp] + movaps xmm11, xmmword ptr [rsp+0x10] + pinsrd xmm3, eax, 3 + pinsrd xmm11, eax, 3 + mov al, 7 +9: + paddd xmm0, xmm4 + paddd xmm8, xmm12 + movaps xmmword ptr [rsp+0x20], xmm4 + movaps xmmword ptr [rsp+0x30], xmm12 + paddd xmm0, xmm1 + paddd xmm8, xmm9 + pxor xmm3, xmm0 + pxor xmm11, xmm8 + movaps xmm12, xmmword ptr [ROT16+rip] + pshufb xmm3, xmm12 + pshufb xmm11, xmm12 + paddd xmm2, xmm3 + paddd xmm10, xmm11 + pxor xmm1, xmm2 + pxor xmm9, xmm10 + movdqa xmm4, xmm1 + pslld xmm1, 20 + psrld xmm4, 12 + por xmm1, xmm4 + movdqa xmm4, xmm9 + pslld xmm9, 20 + psrld xmm4, 12 + por xmm9, xmm4 + paddd xmm0, xmm5 + paddd xmm8, xmm13 + movaps xmmword ptr [rsp+0x40], xmm5 + movaps xmmword ptr [rsp+0x50], xmm13 + paddd xmm0, xmm1 + paddd xmm8, xmm9 + pxor xmm3, xmm0 + pxor xmm11, xmm8 + movaps xmm13, xmmword ptr [ROT8+rip] + pshufb xmm3, xmm13 + pshufb xmm11, xmm13 + paddd xmm2, xmm3 + paddd xmm10, xmm11 + pxor xmm1, xmm2 + pxor xmm9, xmm10 + movdqa xmm4, xmm1 + pslld xmm1, 25 + psrld xmm4, 7 + por xmm1, xmm4 + movdqa xmm4, xmm9 + pslld xmm9, 25 + psrld xmm4, 7 + por xmm9, xmm4 + pshufd xmm0, xmm0, 0x93 + pshufd xmm8, xmm8, 0x93 + pshufd xmm3, xmm3, 0x4E + pshufd xmm11, xmm11, 0x4E + pshufd xmm2, xmm2, 0x39 + pshufd xmm10, xmm10, 0x39 + paddd xmm0, xmm6 + paddd xmm8, xmm14 + paddd xmm0, xmm1 + paddd xmm8, xmm9 + pxor xmm3, xmm0 + pxor xmm11, xmm8 + pshufb xmm3, xmm12 + pshufb xmm11, xmm12 + paddd xmm2, xmm3 + paddd xmm10, xmm11 + pxor xmm1, xmm2 + pxor xmm9, xmm10 + movdqa xmm4, xmm1 + pslld xmm1, 20 + psrld xmm4, 12 + por xmm1, xmm4 + movdqa xmm4, xmm9 + pslld xmm9, 20 + psrld xmm4, 12 + por xmm9, xmm4 + paddd xmm0, xmm7 + paddd xmm8, xmm15 + paddd xmm0, xmm1 + paddd xmm8, xmm9 + pxor xmm3, xmm0 + pxor xmm11, xmm8 + pshufb xmm3, xmm13 + pshufb xmm11, xmm13 + paddd xmm2, xmm3 + paddd xmm10, xmm11 + pxor xmm1, xmm2 + pxor xmm9, xmm10 + movdqa xmm4, xmm1 + pslld xmm1, 25 + psrld xmm4, 7 + por xmm1, xmm4 + movdqa xmm4, xmm9 + pslld xmm9, 25 + psrld xmm4, 7 + por xmm9, xmm4 + pshufd xmm0, xmm0, 0x39 + pshufd xmm8, xmm8, 0x39 + pshufd xmm3, xmm3, 0x4E + pshufd xmm11, xmm11, 0x4E + pshufd xmm2, xmm2, 0x93 + pshufd xmm10, xmm10, 0x93 + dec al + je 9f + movdqa xmm12, xmmword ptr [rsp+0x20] + movdqa xmm5, xmmword ptr [rsp+0x40] + pshufd xmm13, xmm12, 0x0F + shufps xmm12, xmm5, 214 + pshufd xmm4, xmm12, 0x39 + movdqa xmm12, xmm6 + shufps xmm12, xmm7, 250 + pblendw xmm13, xmm12, 0xCC + movdqa xmm12, xmm7 + punpcklqdq xmm12, xmm5 + pblendw xmm12, xmm6, 0xC0 + pshufd xmm12, xmm12, 0x78 + punpckhdq xmm5, xmm7 + punpckldq xmm6, xmm5 + pshufd xmm7, xmm6, 0x1E + movdqa xmmword ptr [rsp+0x20], xmm13 + movdqa xmmword ptr [rsp+0x40], xmm12 + movdqa xmm5, xmmword ptr [rsp+0x30] + movdqa xmm13, xmmword ptr [rsp+0x50] + pshufd xmm6, xmm5, 0x0F + shufps xmm5, xmm13, 214 + pshufd xmm12, xmm5, 0x39 + movdqa xmm5, xmm14 + shufps xmm5, xmm15, 250 + pblendw xmm6, xmm5, 0xCC + movdqa xmm5, xmm15 + punpcklqdq xmm5, xmm13 + pblendw xmm5, xmm14, 0xC0 + pshufd xmm5, xmm5, 0x78 + punpckhdq xmm13, xmm15 + punpckldq xmm14, xmm13 + pshufd xmm15, xmm14, 0x1E + movdqa xmm13, xmm6 + movdqa xmm14, xmm5 + movdqa xmm5, xmmword ptr [rsp+0x20] + movdqa xmm6, xmmword ptr [rsp+0x40] + jmp 9b +9: + pxor xmm0, xmm2 + pxor xmm1, xmm3 + pxor xmm8, xmm10 + pxor xmm9, xmm11 + mov eax, r13d + cmp rdx, r15 + jne 2b + movups xmmword ptr [rbx], xmm0 + movups xmmword ptr [rbx+0x10], xmm1 + movups xmmword ptr [rbx+0x20], xmm8 + movups xmmword ptr [rbx+0x30], xmm9 + movdqa xmm0, xmmword ptr [rsp+0x130] + movdqa xmm1, xmmword ptr [rsp+0x110] + movdqa xmm2, xmmword ptr [rsp+0x120] + movdqu xmm3, xmmword ptr [rsp+0x118] + movdqu xmm4, xmmword ptr [rsp+0x128] + blendvps xmm1, xmm3, xmm0 + blendvps xmm2, xmm4, xmm0 + movdqa xmmword ptr [rsp+0x110], xmm1 + movdqa xmmword ptr [rsp+0x120], xmm2 + add rdi, 16 + add rbx, 64 + sub rsi, 2 +3: + test esi, 0x1 + je 4b + movups xmm0, xmmword ptr [rcx] + movups xmm1, xmmword ptr [rcx+0x10] + movd xmm13, dword ptr [rsp+0x110] + pinsrd xmm13, dword ptr [rsp+0x120], 1 + pinsrd xmm13, dword ptr [BLAKE3_BLOCK_LEN+rip], 2 + movaps xmm14, xmmword ptr [ROT8+rip] + movaps xmm15, xmmword ptr [ROT16+rip] + mov r8, qword ptr [rdi] + movzx eax, byte ptr [rbp+0x40] + or eax, r13d + xor edx, edx +2: + mov r14d, eax + or eax, r12d + add rdx, 64 + cmp rdx, r15 + cmovne eax, r14d + movaps xmm2, xmmword ptr [BLAKE3_IV+rip] + movaps xmm3, xmm13 + pinsrd xmm3, eax, 3 + movups xmm4, xmmword ptr [r8+rdx-0x40] + movups xmm5, xmmword ptr [r8+rdx-0x30] + movaps xmm8, xmm4 + shufps xmm4, xmm5, 136 + shufps xmm8, xmm5, 221 + movaps xmm5, xmm8 + movups xmm6, xmmword ptr [r8+rdx-0x20] + movups xmm7, xmmword ptr [r8+rdx-0x10] + movaps xmm8, xmm6 + shufps xmm6, xmm7, 136 + pshufd xmm6, xmm6, 0x93 + shufps xmm8, xmm7, 221 + pshufd xmm7, xmm8, 0x93 + mov al, 7 +9: + paddd xmm0, xmm4 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshufb xmm3, xmm15 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 20 + psrld xmm11, 12 + por xmm1, xmm11 + paddd xmm0, xmm5 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshufb xmm3, xmm14 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 25 + psrld xmm11, 7 + por xmm1, xmm11 + pshufd xmm0, xmm0, 0x93 + pshufd xmm3, xmm3, 0x4E + pshufd xmm2, xmm2, 0x39 + paddd xmm0, xmm6 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshufb xmm3, xmm15 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 20 + psrld xmm11, 12 + por xmm1, xmm11 + paddd xmm0, xmm7 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshufb xmm3, xmm14 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 25 + psrld xmm11, 7 + por xmm1, xmm11 + pshufd xmm0, xmm0, 0x39 + pshufd xmm3, xmm3, 0x4E + pshufd xmm2, xmm2, 0x93 + dec al + jz 9f + movdqa xmm8, xmm4 + shufps xmm8, xmm5, 214 + pshufd xmm9, xmm4, 0x0F + pshufd xmm4, xmm8, 0x39 + movdqa xmm8, xmm6 + shufps xmm8, xmm7, 250 + pblendw xmm9, xmm8, 0xCC + movdqa xmm8, xmm7 + punpcklqdq xmm8, xmm5 + pblendw xmm8, xmm6, 0xC0 + pshufd xmm8, xmm8, 0x78 + punpckhdq xmm5, xmm7 + punpckldq xmm6, xmm5 + pshufd xmm7, xmm6, 0x1E + movdqa xmm5, xmm9 + movdqa xmm6, xmm8 + jmp 9b +9: + pxor xmm0, xmm2 + pxor xmm1, xmm3 + mov eax, r13d + cmp rdx, r15 + jne 2b + movups xmmword ptr [rbx], xmm0 + movups xmmword ptr [rbx+0x10], xmm1 + jmp 4b +.p2align 6 +zfs_blake3_compress_in_place_sse41: + _CET_ENDBR + movups xmm0, xmmword ptr [rdi] + movups xmm1, xmmword ptr [rdi+0x10] + movaps xmm2, xmmword ptr [BLAKE3_IV+rip] + shl r8, 32 + add rdx, r8 + movq xmm3, rcx + movq xmm4, rdx + punpcklqdq xmm3, xmm4 + movups xmm4, xmmword ptr [rsi] + movups xmm5, xmmword ptr [rsi+0x10] + movaps xmm8, xmm4 + shufps xmm4, xmm5, 136 + shufps xmm8, xmm5, 221 + movaps xmm5, xmm8 + movups xmm6, xmmword ptr [rsi+0x20] + movups xmm7, xmmword ptr [rsi+0x30] + movaps xmm8, xmm6 + shufps xmm6, xmm7, 136 + pshufd xmm6, xmm6, 0x93 + shufps xmm8, xmm7, 221 + pshufd xmm7, xmm8, 0x93 + movaps xmm14, xmmword ptr [ROT8+rip] + movaps xmm15, xmmword ptr [ROT16+rip] + mov al, 7 +9: + paddd xmm0, xmm4 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshufb xmm3, xmm15 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 20 + psrld xmm11, 12 + por xmm1, xmm11 + paddd xmm0, xmm5 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshufb xmm3, xmm14 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 25 + psrld xmm11, 7 + por xmm1, xmm11 + pshufd xmm0, xmm0, 0x93 + pshufd xmm3, xmm3, 0x4E + pshufd xmm2, xmm2, 0x39 + paddd xmm0, xmm6 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshufb xmm3, xmm15 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 20 + psrld xmm11, 12 + por xmm1, xmm11 + paddd xmm0, xmm7 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshufb xmm3, xmm14 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 25 + psrld xmm11, 7 + por xmm1, xmm11 + pshufd xmm0, xmm0, 0x39 + pshufd xmm3, xmm3, 0x4E + pshufd xmm2, xmm2, 0x93 + dec al + jz 9f + movdqa xmm8, xmm4 + shufps xmm8, xmm5, 214 + pshufd xmm9, xmm4, 0x0F + pshufd xmm4, xmm8, 0x39 + movdqa xmm8, xmm6 + shufps xmm8, xmm7, 250 + pblendw xmm9, xmm8, 0xCC + movdqa xmm8, xmm7 + punpcklqdq xmm8, xmm5 + pblendw xmm8, xmm6, 0xC0 + pshufd xmm8, xmm8, 0x78 + punpckhdq xmm5, xmm7 + punpckldq xmm6, xmm5 + pshufd xmm7, xmm6, 0x1E + movdqa xmm5, xmm9 + movdqa xmm6, xmm8 + jmp 9b +9: + pxor xmm0, xmm2 + pxor xmm1, xmm3 + movups xmmword ptr [rdi], xmm0 + movups xmmword ptr [rdi+0x10], xmm1 + RET +.p2align 6 +zfs_blake3_compress_xof_sse41: + _CET_ENDBR + movups xmm0, xmmword ptr [rdi] + movups xmm1, xmmword ptr [rdi+0x10] + movaps xmm2, xmmword ptr [BLAKE3_IV+rip] + movzx eax, r8b + movzx edx, dl + shl rax, 32 + add rdx, rax + movq xmm3, rcx + movq xmm4, rdx + punpcklqdq xmm3, xmm4 + movups xmm4, xmmword ptr [rsi] + movups xmm5, xmmword ptr [rsi+0x10] + movaps xmm8, xmm4 + shufps xmm4, xmm5, 136 + shufps xmm8, xmm5, 221 + movaps xmm5, xmm8 + movups xmm6, xmmword ptr [rsi+0x20] + movups xmm7, xmmword ptr [rsi+0x30] + movaps xmm8, xmm6 + shufps xmm6, xmm7, 136 + pshufd xmm6, xmm6, 0x93 + shufps xmm8, xmm7, 221 + pshufd xmm7, xmm8, 0x93 + movaps xmm14, xmmword ptr [ROT8+rip] + movaps xmm15, xmmword ptr [ROT16+rip] + mov al, 7 +9: + paddd xmm0, xmm4 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshufb xmm3, xmm15 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 20 + psrld xmm11, 12 + por xmm1, xmm11 + paddd xmm0, xmm5 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshufb xmm3, xmm14 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 25 + psrld xmm11, 7 + por xmm1, xmm11 + pshufd xmm0, xmm0, 0x93 + pshufd xmm3, xmm3, 0x4E + pshufd xmm2, xmm2, 0x39 + paddd xmm0, xmm6 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshufb xmm3, xmm15 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 20 + psrld xmm11, 12 + por xmm1, xmm11 + paddd xmm0, xmm7 + paddd xmm0, xmm1 + pxor xmm3, xmm0 + pshufb xmm3, xmm14 + paddd xmm2, xmm3 + pxor xmm1, xmm2 + movdqa xmm11, xmm1 + pslld xmm1, 25 + psrld xmm11, 7 + por xmm1, xmm11 + pshufd xmm0, xmm0, 0x39 + pshufd xmm3, xmm3, 0x4E + pshufd xmm2, xmm2, 0x93 + dec al + jz 9f + movdqa xmm8, xmm4 + shufps xmm8, xmm5, 214 + pshufd xmm9, xmm4, 0x0F + pshufd xmm4, xmm8, 0x39 + movdqa xmm8, xmm6 + shufps xmm8, xmm7, 250 + pblendw xmm9, xmm8, 0xCC + movdqa xmm8, xmm7 + punpcklqdq xmm8, xmm5 + pblendw xmm8, xmm6, 0xC0 + pshufd xmm8, xmm8, 0x78 + punpckhdq xmm5, xmm7 + punpckldq xmm6, xmm5 + pshufd xmm7, xmm6, 0x1E + movdqa xmm5, xmm9 + movdqa xmm6, xmm8 + jmp 9b +9: + movdqu xmm4, xmmword ptr [rdi] + movdqu xmm5, xmmword ptr [rdi+0x10] + pxor xmm0, xmm2 + pxor xmm1, xmm3 + pxor xmm2, xmm4 + pxor xmm3, xmm5 + movups xmmword ptr [r9], xmm0 + movups xmmword ptr [r9+0x10], xmm1 + movups xmmword ptr [r9+0x20], xmm2 + movups xmmword ptr [r9+0x30], xmm3 + RET + +.size zfs_blake3_hash_many_sse41, . - zfs_blake3_hash_many_sse41 +.size zfs_blake3_compress_in_place_sse41, . - zfs_blake3_compress_in_place_sse41 +.size zfs_blake3_compress_xof_sse41, . - zfs_blake3_compress_xof_sse41 + +#ifdef __APPLE__ +.static_data +#else +.section .rodata +#endif +.p2align 6 +BLAKE3_IV: + .long 0x6A09E667, 0xBB67AE85 + .long 0x3C6EF372, 0xA54FF53A +ROT16: + .byte 2, 3, 0, 1, 6, 7, 4, 5, 10, 11, 8, 9, 14, 15, 12, 13 +ROT8: + .byte 1, 2, 3, 0, 5, 6, 7, 4, 9, 10, 11, 8, 13, 14, 15, 12 +ADD0: + .long 0, 1, 2, 3 +ADD1: + .long 4, 4, 4, 4 +BLAKE3_IV_0: + .long 0x6A09E667, 0x6A09E667, 0x6A09E667, 0x6A09E667 +BLAKE3_IV_1: + .long 0xBB67AE85, 0xBB67AE85, 0xBB67AE85, 0xBB67AE85 +BLAKE3_IV_2: + .long 0x3C6EF372, 0x3C6EF372, 0x3C6EF372, 0x3C6EF372 +BLAKE3_IV_3: + .long 0xA54FF53A, 0xA54FF53A, 0xA54FF53A, 0xA54FF53A +BLAKE3_BLOCK_LEN: + .long 64, 64, 64, 64 +CMP_MSB_MASK: + .long 0x80000000, 0x80000000, 0x80000000, 0x80000000 + +#endif /* HAVE_SSE4_1 */ + +#ifdef __ELF__ +.section .note.GNU-stack,"",%progbits +#endif diff --git a/module/icp/asm-x86_64/modes/aesni-gcm-x86_64.S b/module/icp/asm-x86_64/modes/aesni-gcm-x86_64.S index dc71ae2c1c89..70e419c2e4ab 100644 --- a/module/icp/asm-x86_64/modes/aesni-gcm-x86_64.S +++ b/module/icp/asm-x86_64/modes/aesni-gcm-x86_64.S @@ -1201,7 +1201,7 @@ aesni_gcm_encrypt: .align 32 clear_fpu_regs_avx: vzeroall - ret + RET .size clear_fpu_regs_avx,.-clear_fpu_regs_avx /* @@ -1219,7 +1219,7 @@ gcm_xor_avx: movdqu (%rsi), %xmm1 pxor %xmm1, %xmm0 movdqu %xmm0, (%rsi) - ret + RET .size gcm_xor_avx,.-gcm_xor_avx /* @@ -1236,7 +1236,7 @@ atomic_toggle_boolean_nv: jz 1f movl $1, %eax 1: - ret + RET .size atomic_toggle_boolean_nv,.-atomic_toggle_boolean_nv .align 64 diff --git a/module/icp/asm-x86_64/modes/gcm_pclmulqdq.S b/module/icp/asm-x86_64/modes/gcm_pclmulqdq.S index 74eacbbe6388..eb9514e10cda 100644 --- a/module/icp/asm-x86_64/modes/gcm_pclmulqdq.S +++ b/module/icp/asm-x86_64/modes/gcm_pclmulqdq.S @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -244,7 +244,7 @@ ENTRY_NP(gcm_mul_pclmulqdq) // // Return // - ret + RET SET_SIZE(gcm_mul_pclmulqdq) #endif /* lint || __lint */ diff --git a/module/icp/asm-x86_64/sha2/sha256_impl.S b/module/icp/asm-x86_64/sha2/sha256_impl.S index 951297c72ff8..1391bd59a017 100644 --- a/module/icp/asm-x86_64/sha2/sha256_impl.S +++ b/module/icp/asm-x86_64/sha2/sha256_impl.S @@ -2058,7 +2058,7 @@ ENTRY_NP(SHA256TransformBlocks) .cfi_adjust_cfa_offset -8 .cfi_restore %rbx - ret + RET .cfi_endproc SET_SIZE(SHA256TransformBlocks) diff --git a/module/icp/asm-x86_64/sha2/sha512_impl.S b/module/icp/asm-x86_64/sha2/sha512_impl.S index 921d3d8cddae..e61e96957bc6 100644 --- a/module/icp/asm-x86_64/sha2/sha512_impl.S +++ b/module/icp/asm-x86_64/sha2/sha512_impl.S @@ -2059,7 +2059,7 @@ ENTRY_NP(SHA512TransformBlocks) .cfi_adjust_cfa_offset -8 .cfi_restore %rbx - ret + RET .cfi_endproc SET_SIZE(SHA512TransformBlocks) diff --git a/module/icp/core/kcf_callprov.c b/module/icp/core/kcf_callprov.c index 345014d0a1e4..f06b3cd00bcf 100644 --- a/module/icp/core/kcf_callprov.c +++ b/module/icp/core/kcf_callprov.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -27,9 +27,6 @@ #include #include -static int kcf_emulate_dual(kcf_provider_desc_t *, crypto_ctx_t *, - kcf_req_params_t *); - void kcf_free_triedlist(kcf_prov_tried_t *list) { @@ -71,168 +68,6 @@ is_in_triedlist(kcf_provider_desc_t *pd, kcf_prov_tried_t *triedl) return (B_FALSE); } -/* - * Search a mech entry's hardware provider list for the specified - * provider. Return true if found. - */ -static boolean_t -is_valid_provider_for_mech(kcf_provider_desc_t *pd, kcf_mech_entry_t *me, - crypto_func_group_t fg) -{ - kcf_prov_mech_desc_t *prov_chain; - - prov_chain = me->me_hw_prov_chain; - if (prov_chain != NULL) { - ASSERT(me->me_num_hwprov > 0); - for (; prov_chain != NULL; prov_chain = prov_chain->pm_next) { - if (prov_chain->pm_prov_desc == pd && - IS_FG_SUPPORTED(prov_chain, fg)) { - return (B_TRUE); - } - } - } - return (B_FALSE); -} - -/* - * This routine, given a logical provider, returns the least loaded - * provider belonging to the logical provider. The provider must be - * able to do the specified mechanism, i.e. check that the mechanism - * hasn't been disabled. In addition, just in case providers are not - * entirely equivalent, the provider's entry point is checked for - * non-nullness. This is accomplished by having the caller pass, as - * arguments, the offset of the function group (offset_1), and the - * offset of the function within the function group (offset_2). - * Returns NULL if no provider can be found. - */ -int -kcf_get_hardware_provider(crypto_mech_type_t mech_type_1, - crypto_mech_type_t mech_type_2, boolean_t call_restrict, - kcf_provider_desc_t *old, kcf_provider_desc_t **new, crypto_func_group_t fg) -{ - kcf_provider_desc_t *provider, *real_pd = old; - kcf_provider_desc_t *gpd = NULL; /* good provider */ - kcf_provider_desc_t *bpd = NULL; /* busy provider */ - kcf_provider_list_t *p; - kcf_ops_class_t class; - kcf_mech_entry_t *me; - const kcf_mech_entry_tab_t *me_tab; - int index, len, gqlen = INT_MAX, rv = CRYPTO_SUCCESS; - - /* get the mech entry for the specified mechanism */ - class = KCF_MECH2CLASS(mech_type_1); - if ((class < KCF_FIRST_OPSCLASS) || (class > KCF_LAST_OPSCLASS)) { - return (CRYPTO_MECHANISM_INVALID); - } - - me_tab = &kcf_mech_tabs_tab[class]; - index = KCF_MECH2INDEX(mech_type_1); - if ((index < 0) || (index >= me_tab->met_size)) { - return (CRYPTO_MECHANISM_INVALID); - } - - me = &((me_tab->met_tab)[index]); - mutex_enter(&me->me_mutex); - - /* - * We assume the provider descriptor will not go away because - * it is being held somewhere, i.e. its reference count has been - * incremented. In the case of the crypto module, the provider - * descriptor is held by the session structure. - */ - if (old->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) { - if (old->pd_provider_list == NULL) { - real_pd = NULL; - rv = CRYPTO_DEVICE_ERROR; - goto out; - } - /* - * Find the least loaded real provider. KCF_PROV_LOAD gives - * the load (number of pending requests) of the provider. - */ - mutex_enter(&old->pd_lock); - p = old->pd_provider_list; - while (p != NULL) { - provider = p->pl_provider; - - ASSERT(provider->pd_prov_type != - CRYPTO_LOGICAL_PROVIDER); - - if (call_restrict && - (provider->pd_flags & KCF_PROV_RESTRICTED)) { - p = p->pl_next; - continue; - } - - if (!is_valid_provider_for_mech(provider, me, fg)) { - p = p->pl_next; - continue; - } - - /* provider does second mech */ - if (mech_type_2 != CRYPTO_MECH_INVALID) { - int i; - - i = KCF_TO_PROV_MECH_INDX(provider, - mech_type_2); - if (i == KCF_INVALID_INDX) { - p = p->pl_next; - continue; - } - } - - if (provider->pd_state != KCF_PROV_READY) { - /* choose BUSY if no READY providers */ - if (provider->pd_state == KCF_PROV_BUSY) - bpd = provider; - p = p->pl_next; - continue; - } - - len = KCF_PROV_LOAD(provider); - if (len < gqlen) { - gqlen = len; - gpd = provider; - } - - p = p->pl_next; - } - - if (gpd != NULL) { - real_pd = gpd; - KCF_PROV_REFHOLD(real_pd); - } else if (bpd != NULL) { - real_pd = bpd; - KCF_PROV_REFHOLD(real_pd); - } else { - /* can't find provider */ - real_pd = NULL; - rv = CRYPTO_MECHANISM_INVALID; - } - mutex_exit(&old->pd_lock); - - } else { - if (!KCF_IS_PROV_USABLE(old) || - (call_restrict && (old->pd_flags & KCF_PROV_RESTRICTED))) { - real_pd = NULL; - rv = CRYPTO_DEVICE_ERROR; - goto out; - } - - if (!is_valid_provider_for_mech(old, me, fg)) { - real_pd = NULL; - rv = CRYPTO_MECHANISM_INVALID; - goto out; - } - - KCF_PROV_REFHOLD(real_pd); - } -out: - mutex_exit(&me->me_mutex); - *new = real_pd; - return (rv); -} - /* * Return the best provider for the specified mechanism. The provider * is held and it is the caller's responsibility to release it when done. @@ -243,18 +78,13 @@ kcf_get_hardware_provider(crypto_mech_type_t mech_type_1, * search to find one. This is fine as we assume there are only a few * number of providers in this list. If this assumption ever changes, * we should revisit this. - * - * call_restrict represents if the caller should not be allowed to - * use restricted providers. */ kcf_provider_desc_t * kcf_get_mech_provider(crypto_mech_type_t mech_type, kcf_mech_entry_t **mepp, - int *error, kcf_prov_tried_t *triedl, crypto_func_group_t fg, - boolean_t call_restrict, size_t data_size) + int *error, kcf_prov_tried_t *triedl, crypto_func_group_t fg) { - kcf_provider_desc_t *pd = NULL, *gpd = NULL; - kcf_prov_mech_desc_t *prov_chain, *mdesc; - int len, gqlen = INT_MAX; + kcf_provider_desc_t *pd = NULL; + kcf_prov_mech_desc_t *mdesc; kcf_ops_class_t class; int index; kcf_mech_entry_t *me; @@ -277,58 +107,12 @@ kcf_get_mech_provider(crypto_mech_type_t mech_type, kcf_mech_entry_t **mepp, if (mepp != NULL) *mepp = me; - mutex_enter(&me->me_mutex); - - prov_chain = me->me_hw_prov_chain; - - /* - * We check for the threshold for using a hardware provider for - * this amount of data. If there is no software provider available - * for the mechanism, then the threshold is ignored. - */ - if ((prov_chain != NULL) && - ((data_size == 0) || (me->me_threshold == 0) || - (data_size >= me->me_threshold) || - ((mdesc = me->me_sw_prov) == NULL) || - (!IS_FG_SUPPORTED(mdesc, fg)) || - (!KCF_IS_PROV_USABLE(mdesc->pm_prov_desc)))) { - ASSERT(me->me_num_hwprov > 0); - /* there is at least one provider */ - - /* - * Find the least loaded real provider. KCF_PROV_LOAD gives - * the load (number of pending requests) of the provider. - */ - while (prov_chain != NULL) { - pd = prov_chain->pm_prov_desc; - - if (!IS_FG_SUPPORTED(prov_chain, fg) || - !KCF_IS_PROV_USABLE(pd) || - IS_PROVIDER_TRIED(pd, triedl) || - (call_restrict && - (pd->pd_flags & KCF_PROV_RESTRICTED))) { - prov_chain = prov_chain->pm_next; - continue; - } - - if ((len = KCF_PROV_LOAD(pd)) < gqlen) { - gqlen = len; - gpd = pd; - } - - prov_chain = prov_chain->pm_next; - } - - pd = gpd; - } - - /* No HW provider for this mech, is there a SW provider? */ + /* Is there a provider? */ if (pd == NULL && (mdesc = me->me_sw_prov) != NULL) { pd = mdesc->pm_prov_desc; if (!IS_FG_SUPPORTED(mdesc, fg) || !KCF_IS_PROV_USABLE(pd) || - IS_PROVIDER_TRIED(pd, triedl) || - (call_restrict && (pd->pd_flags & KCF_PROV_RESTRICTED))) + IS_PROVIDER_TRIED(pd, triedl)) pd = NULL; } @@ -344,1224 +128,5 @@ kcf_get_mech_provider(crypto_mech_type_t mech_type, kcf_mech_entry_t **mepp, } else KCF_PROV_REFHOLD(pd); - mutex_exit(&me->me_mutex); return (pd); } - -/* - * Very similar to kcf_get_mech_provider(). Finds the best provider capable of - * a dual operation with both me1 and me2. - * When no dual-ops capable providers are available, return the best provider - * for me1 only, and sets *prov_mt2 to CRYPTO_INVALID_MECHID; - * We assume/expect that a slower HW capable of the dual is still - * faster than the 2 fastest providers capable of the individual ops - * separately. - */ -kcf_provider_desc_t * -kcf_get_dual_provider(crypto_mechanism_t *mech1, crypto_mechanism_t *mech2, - kcf_mech_entry_t **mepp, crypto_mech_type_t *prov_mt1, - crypto_mech_type_t *prov_mt2, int *error, kcf_prov_tried_t *triedl, - crypto_func_group_t fg1, crypto_func_group_t fg2, boolean_t call_restrict, - size_t data_size) -{ - kcf_provider_desc_t *pd = NULL, *pdm1 = NULL, *pdm1m2 = NULL; - kcf_prov_mech_desc_t *prov_chain, *mdesc; - int len, gqlen = INT_MAX, dgqlen = INT_MAX; - crypto_mech_info_list_t *mil; - crypto_mech_type_t m2id = mech2->cm_type; - kcf_mech_entry_t *me; - - /* when mech is a valid mechanism, me will be its mech_entry */ - if (kcf_get_mech_entry(mech1->cm_type, &me) != KCF_SUCCESS) { - *error = CRYPTO_MECHANISM_INVALID; - return (NULL); - } - - *prov_mt2 = CRYPTO_MECH_INVALID; - - if (mepp != NULL) - *mepp = me; - mutex_enter(&me->me_mutex); - - prov_chain = me->me_hw_prov_chain; - /* - * We check the threshold for using a hardware provider for - * this amount of data. If there is no software provider available - * for the first mechanism, then the threshold is ignored. - */ - if ((prov_chain != NULL) && - ((data_size == 0) || (me->me_threshold == 0) || - (data_size >= me->me_threshold) || - ((mdesc = me->me_sw_prov) == NULL) || - (!IS_FG_SUPPORTED(mdesc, fg1)) || - (!KCF_IS_PROV_USABLE(mdesc->pm_prov_desc)))) { - /* there is at least one provider */ - ASSERT(me->me_num_hwprov > 0); - - /* - * Find the least loaded provider capable of the combo - * me1 + me2, and save a pointer to the least loaded - * provider capable of me1 only. - */ - while (prov_chain != NULL) { - pd = prov_chain->pm_prov_desc; - len = KCF_PROV_LOAD(pd); - - if (!IS_FG_SUPPORTED(prov_chain, fg1) || - !KCF_IS_PROV_USABLE(pd) || - IS_PROVIDER_TRIED(pd, triedl) || - (call_restrict && - (pd->pd_flags & KCF_PROV_RESTRICTED))) { - prov_chain = prov_chain->pm_next; - continue; - } - - /* Save the best provider capable of m1 */ - if (len < gqlen) { - *prov_mt1 = - prov_chain->pm_mech_info.cm_mech_number; - gqlen = len; - pdm1 = pd; - } - - /* See if pd can do me2 too */ - for (mil = prov_chain->pm_mi_list; - mil != NULL; mil = mil->ml_next) { - if ((mil->ml_mech_info.cm_func_group_mask & - fg2) == 0) - continue; - - if ((mil->ml_kcf_mechid == m2id) && - (len < dgqlen)) { - /* Bingo! */ - dgqlen = len; - pdm1m2 = pd; - *prov_mt2 = - mil->ml_mech_info.cm_mech_number; - *prov_mt1 = prov_chain-> - pm_mech_info.cm_mech_number; - break; - } - } - - prov_chain = prov_chain->pm_next; - } - - pd = (pdm1m2 != NULL) ? pdm1m2 : pdm1; - } - - /* no HW provider for this mech, is there a SW provider? */ - if (pd == NULL && (mdesc = me->me_sw_prov) != NULL) { - pd = mdesc->pm_prov_desc; - if (!IS_FG_SUPPORTED(mdesc, fg1) || - !KCF_IS_PROV_USABLE(pd) || - IS_PROVIDER_TRIED(pd, triedl) || - (call_restrict && (pd->pd_flags & KCF_PROV_RESTRICTED))) - pd = NULL; - else { - /* See if pd can do me2 too */ - for (mil = me->me_sw_prov->pm_mi_list; - mil != NULL; mil = mil->ml_next) { - if ((mil->ml_mech_info.cm_func_group_mask & - fg2) == 0) - continue; - - if (mil->ml_kcf_mechid == m2id) { - /* Bingo! */ - *prov_mt2 = - mil->ml_mech_info.cm_mech_number; - break; - } - } - *prov_mt1 = me->me_sw_prov->pm_mech_info.cm_mech_number; - } - } - - if (pd == NULL) - *error = CRYPTO_MECH_NOT_SUPPORTED; - else - KCF_PROV_REFHOLD(pd); - - mutex_exit(&me->me_mutex); - return (pd); -} - -/* - * Do the actual work of calling the provider routines. - * - * pd - Provider structure - * ctx - Context for this operation - * params - Parameters for this operation - * rhndl - Request handle to use for notification - * - * The return values are the same as that of the respective SPI. - */ -int -common_submit_request(kcf_provider_desc_t *pd, crypto_ctx_t *ctx, - kcf_req_params_t *params, crypto_req_handle_t rhndl) -{ - int err = CRYPTO_ARGUMENTS_BAD; - kcf_op_type_t optype; - - optype = params->rp_optype; - - switch (params->rp_opgrp) { - case KCF_OG_DIGEST: { - kcf_digest_ops_params_t *dops = ¶ms->rp_u.digest_params; - - switch (optype) { - case KCF_OP_INIT: - /* - * We should do this only here and not in KCF_WRAP_* - * macros. This is because we may want to try other - * providers, in case we recover from a failure. - */ - KCF_SET_PROVIDER_MECHNUM(dops->do_framework_mechtype, - pd, &dops->do_mech); - - err = KCF_PROV_DIGEST_INIT(pd, ctx, &dops->do_mech, - rhndl); - break; - - case KCF_OP_SINGLE: - err = KCF_PROV_DIGEST(pd, ctx, dops->do_data, - dops->do_digest, rhndl); - break; - - case KCF_OP_UPDATE: - err = KCF_PROV_DIGEST_UPDATE(pd, ctx, - dops->do_data, rhndl); - break; - - case KCF_OP_FINAL: - err = KCF_PROV_DIGEST_FINAL(pd, ctx, - dops->do_digest, rhndl); - break; - - case KCF_OP_ATOMIC: - ASSERT(ctx == NULL); - KCF_SET_PROVIDER_MECHNUM(dops->do_framework_mechtype, - pd, &dops->do_mech); - err = KCF_PROV_DIGEST_ATOMIC(pd, dops->do_sid, - &dops->do_mech, dops->do_data, dops->do_digest, - rhndl); - break; - - case KCF_OP_DIGEST_KEY: - err = KCF_PROV_DIGEST_KEY(pd, ctx, dops->do_digest_key, - rhndl); - break; - - default: - break; - } - break; - } - - case KCF_OG_MAC: { - kcf_mac_ops_params_t *mops = ¶ms->rp_u.mac_params; - - switch (optype) { - case KCF_OP_INIT: - KCF_SET_PROVIDER_MECHNUM(mops->mo_framework_mechtype, - pd, &mops->mo_mech); - - err = KCF_PROV_MAC_INIT(pd, ctx, &mops->mo_mech, - mops->mo_key, mops->mo_templ, rhndl); - break; - - case KCF_OP_SINGLE: - err = KCF_PROV_MAC(pd, ctx, mops->mo_data, - mops->mo_mac, rhndl); - break; - - case KCF_OP_UPDATE: - err = KCF_PROV_MAC_UPDATE(pd, ctx, mops->mo_data, - rhndl); - break; - - case KCF_OP_FINAL: - err = KCF_PROV_MAC_FINAL(pd, ctx, mops->mo_mac, rhndl); - break; - - case KCF_OP_ATOMIC: - ASSERT(ctx == NULL); - KCF_SET_PROVIDER_MECHNUM(mops->mo_framework_mechtype, - pd, &mops->mo_mech); - - err = KCF_PROV_MAC_ATOMIC(pd, mops->mo_sid, - &mops->mo_mech, mops->mo_key, mops->mo_data, - mops->mo_mac, mops->mo_templ, rhndl); - break; - - case KCF_OP_MAC_VERIFY_ATOMIC: - ASSERT(ctx == NULL); - KCF_SET_PROVIDER_MECHNUM(mops->mo_framework_mechtype, - pd, &mops->mo_mech); - - err = KCF_PROV_MAC_VERIFY_ATOMIC(pd, mops->mo_sid, - &mops->mo_mech, mops->mo_key, mops->mo_data, - mops->mo_mac, mops->mo_templ, rhndl); - break; - - default: - break; - } - break; - } - - case KCF_OG_ENCRYPT: { - kcf_encrypt_ops_params_t *eops = ¶ms->rp_u.encrypt_params; - - switch (optype) { - case KCF_OP_INIT: - KCF_SET_PROVIDER_MECHNUM(eops->eo_framework_mechtype, - pd, &eops->eo_mech); - - err = KCF_PROV_ENCRYPT_INIT(pd, ctx, &eops->eo_mech, - eops->eo_key, eops->eo_templ, rhndl); - break; - - case KCF_OP_SINGLE: - err = KCF_PROV_ENCRYPT(pd, ctx, eops->eo_plaintext, - eops->eo_ciphertext, rhndl); - break; - - case KCF_OP_UPDATE: - err = KCF_PROV_ENCRYPT_UPDATE(pd, ctx, - eops->eo_plaintext, eops->eo_ciphertext, rhndl); - break; - - case KCF_OP_FINAL: - err = KCF_PROV_ENCRYPT_FINAL(pd, ctx, - eops->eo_ciphertext, rhndl); - break; - - case KCF_OP_ATOMIC: - ASSERT(ctx == NULL); - KCF_SET_PROVIDER_MECHNUM(eops->eo_framework_mechtype, - pd, &eops->eo_mech); - - err = KCF_PROV_ENCRYPT_ATOMIC(pd, eops->eo_sid, - &eops->eo_mech, eops->eo_key, eops->eo_plaintext, - eops->eo_ciphertext, eops->eo_templ, rhndl); - break; - - default: - break; - } - break; - } - - case KCF_OG_DECRYPT: { - kcf_decrypt_ops_params_t *dcrops = ¶ms->rp_u.decrypt_params; - - switch (optype) { - case KCF_OP_INIT: - KCF_SET_PROVIDER_MECHNUM(dcrops->dop_framework_mechtype, - pd, &dcrops->dop_mech); - - err = KCF_PROV_DECRYPT_INIT(pd, ctx, &dcrops->dop_mech, - dcrops->dop_key, dcrops->dop_templ, rhndl); - break; - - case KCF_OP_SINGLE: - err = KCF_PROV_DECRYPT(pd, ctx, dcrops->dop_ciphertext, - dcrops->dop_plaintext, rhndl); - break; - - case KCF_OP_UPDATE: - err = KCF_PROV_DECRYPT_UPDATE(pd, ctx, - dcrops->dop_ciphertext, dcrops->dop_plaintext, - rhndl); - break; - - case KCF_OP_FINAL: - err = KCF_PROV_DECRYPT_FINAL(pd, ctx, - dcrops->dop_plaintext, rhndl); - break; - - case KCF_OP_ATOMIC: - ASSERT(ctx == NULL); - KCF_SET_PROVIDER_MECHNUM(dcrops->dop_framework_mechtype, - pd, &dcrops->dop_mech); - - err = KCF_PROV_DECRYPT_ATOMIC(pd, dcrops->dop_sid, - &dcrops->dop_mech, dcrops->dop_key, - dcrops->dop_ciphertext, dcrops->dop_plaintext, - dcrops->dop_templ, rhndl); - break; - - default: - break; - } - break; - } - - case KCF_OG_SIGN: { - kcf_sign_ops_params_t *sops = ¶ms->rp_u.sign_params; - - switch (optype) { - case KCF_OP_INIT: - KCF_SET_PROVIDER_MECHNUM(sops->so_framework_mechtype, - pd, &sops->so_mech); - - err = KCF_PROV_SIGN_INIT(pd, ctx, &sops->so_mech, - sops->so_key, sops->so_templ, rhndl); - break; - - case KCF_OP_SIGN_RECOVER_INIT: - KCF_SET_PROVIDER_MECHNUM(sops->so_framework_mechtype, - pd, &sops->so_mech); - - err = KCF_PROV_SIGN_RECOVER_INIT(pd, ctx, - &sops->so_mech, sops->so_key, sops->so_templ, - rhndl); - break; - - case KCF_OP_SINGLE: - err = KCF_PROV_SIGN(pd, ctx, sops->so_data, - sops->so_signature, rhndl); - break; - - case KCF_OP_SIGN_RECOVER: - err = KCF_PROV_SIGN_RECOVER(pd, ctx, - sops->so_data, sops->so_signature, rhndl); - break; - - case KCF_OP_UPDATE: - err = KCF_PROV_SIGN_UPDATE(pd, ctx, sops->so_data, - rhndl); - break; - - case KCF_OP_FINAL: - err = KCF_PROV_SIGN_FINAL(pd, ctx, sops->so_signature, - rhndl); - break; - - case KCF_OP_ATOMIC: - ASSERT(ctx == NULL); - KCF_SET_PROVIDER_MECHNUM(sops->so_framework_mechtype, - pd, &sops->so_mech); - - err = KCF_PROV_SIGN_ATOMIC(pd, sops->so_sid, - &sops->so_mech, sops->so_key, sops->so_data, - sops->so_templ, sops->so_signature, rhndl); - break; - - case KCF_OP_SIGN_RECOVER_ATOMIC: - ASSERT(ctx == NULL); - KCF_SET_PROVIDER_MECHNUM(sops->so_framework_mechtype, - pd, &sops->so_mech); - - err = KCF_PROV_SIGN_RECOVER_ATOMIC(pd, sops->so_sid, - &sops->so_mech, sops->so_key, sops->so_data, - sops->so_templ, sops->so_signature, rhndl); - break; - - default: - break; - } - break; - } - - case KCF_OG_VERIFY: { - kcf_verify_ops_params_t *vops = ¶ms->rp_u.verify_params; - - switch (optype) { - case KCF_OP_INIT: - KCF_SET_PROVIDER_MECHNUM(vops->vo_framework_mechtype, - pd, &vops->vo_mech); - - err = KCF_PROV_VERIFY_INIT(pd, ctx, &vops->vo_mech, - vops->vo_key, vops->vo_templ, rhndl); - break; - - case KCF_OP_VERIFY_RECOVER_INIT: - KCF_SET_PROVIDER_MECHNUM(vops->vo_framework_mechtype, - pd, &vops->vo_mech); - - err = KCF_PROV_VERIFY_RECOVER_INIT(pd, ctx, - &vops->vo_mech, vops->vo_key, vops->vo_templ, - rhndl); - break; - - case KCF_OP_SINGLE: - err = KCF_PROV_VERIFY(pd, ctx, vops->vo_data, - vops->vo_signature, rhndl); - break; - - case KCF_OP_VERIFY_RECOVER: - err = KCF_PROV_VERIFY_RECOVER(pd, ctx, - vops->vo_signature, vops->vo_data, rhndl); - break; - - case KCF_OP_UPDATE: - err = KCF_PROV_VERIFY_UPDATE(pd, ctx, vops->vo_data, - rhndl); - break; - - case KCF_OP_FINAL: - err = KCF_PROV_VERIFY_FINAL(pd, ctx, vops->vo_signature, - rhndl); - break; - - case KCF_OP_ATOMIC: - ASSERT(ctx == NULL); - KCF_SET_PROVIDER_MECHNUM(vops->vo_framework_mechtype, - pd, &vops->vo_mech); - - err = KCF_PROV_VERIFY_ATOMIC(pd, vops->vo_sid, - &vops->vo_mech, vops->vo_key, vops->vo_data, - vops->vo_templ, vops->vo_signature, rhndl); - break; - - case KCF_OP_VERIFY_RECOVER_ATOMIC: - ASSERT(ctx == NULL); - KCF_SET_PROVIDER_MECHNUM(vops->vo_framework_mechtype, - pd, &vops->vo_mech); - - err = KCF_PROV_VERIFY_RECOVER_ATOMIC(pd, vops->vo_sid, - &vops->vo_mech, vops->vo_key, vops->vo_signature, - vops->vo_templ, vops->vo_data, rhndl); - break; - - default: - break; - } - break; - } - - case KCF_OG_ENCRYPT_MAC: { - kcf_encrypt_mac_ops_params_t *eops = - ¶ms->rp_u.encrypt_mac_params; - kcf_context_t *kcf_secondctx; - - switch (optype) { - case KCF_OP_INIT: - kcf_secondctx = ((kcf_context_t *) - (ctx->cc_framework_private))->kc_secondctx; - - if (kcf_secondctx != NULL) { - err = kcf_emulate_dual(pd, ctx, params); - break; - } - KCF_SET_PROVIDER_MECHNUM( - eops->em_framework_encr_mechtype, - pd, &eops->em_encr_mech); - - KCF_SET_PROVIDER_MECHNUM( - eops->em_framework_mac_mechtype, - pd, &eops->em_mac_mech); - - err = KCF_PROV_ENCRYPT_MAC_INIT(pd, ctx, - &eops->em_encr_mech, eops->em_encr_key, - &eops->em_mac_mech, eops->em_mac_key, - eops->em_encr_templ, eops->em_mac_templ, - rhndl); - - break; - - case KCF_OP_SINGLE: - err = KCF_PROV_ENCRYPT_MAC(pd, ctx, - eops->em_plaintext, eops->em_ciphertext, - eops->em_mac, rhndl); - break; - - case KCF_OP_UPDATE: - kcf_secondctx = ((kcf_context_t *) - (ctx->cc_framework_private))->kc_secondctx; - if (kcf_secondctx != NULL) { - err = kcf_emulate_dual(pd, ctx, params); - break; - } - err = KCF_PROV_ENCRYPT_MAC_UPDATE(pd, ctx, - eops->em_plaintext, eops->em_ciphertext, rhndl); - break; - - case KCF_OP_FINAL: - kcf_secondctx = ((kcf_context_t *) - (ctx->cc_framework_private))->kc_secondctx; - if (kcf_secondctx != NULL) { - err = kcf_emulate_dual(pd, ctx, params); - break; - } - err = KCF_PROV_ENCRYPT_MAC_FINAL(pd, ctx, - eops->em_ciphertext, eops->em_mac, rhndl); - break; - - case KCF_OP_ATOMIC: - ASSERT(ctx == NULL); - - KCF_SET_PROVIDER_MECHNUM( - eops->em_framework_encr_mechtype, - pd, &eops->em_encr_mech); - - KCF_SET_PROVIDER_MECHNUM( - eops->em_framework_mac_mechtype, - pd, &eops->em_mac_mech); - - err = KCF_PROV_ENCRYPT_MAC_ATOMIC(pd, eops->em_sid, - &eops->em_encr_mech, eops->em_encr_key, - &eops->em_mac_mech, eops->em_mac_key, - eops->em_plaintext, eops->em_ciphertext, - eops->em_mac, - eops->em_encr_templ, eops->em_mac_templ, - rhndl); - - break; - - default: - break; - } - break; - } - - case KCF_OG_MAC_DECRYPT: { - kcf_mac_decrypt_ops_params_t *dops = - ¶ms->rp_u.mac_decrypt_params; - kcf_context_t *kcf_secondctx; - - switch (optype) { - case KCF_OP_INIT: - kcf_secondctx = ((kcf_context_t *) - (ctx->cc_framework_private))->kc_secondctx; - - if (kcf_secondctx != NULL) { - err = kcf_emulate_dual(pd, ctx, params); - break; - } - KCF_SET_PROVIDER_MECHNUM( - dops->md_framework_mac_mechtype, - pd, &dops->md_mac_mech); - - KCF_SET_PROVIDER_MECHNUM( - dops->md_framework_decr_mechtype, - pd, &dops->md_decr_mech); - - err = KCF_PROV_MAC_DECRYPT_INIT(pd, ctx, - &dops->md_mac_mech, dops->md_mac_key, - &dops->md_decr_mech, dops->md_decr_key, - dops->md_mac_templ, dops->md_decr_templ, - rhndl); - - break; - - case KCF_OP_SINGLE: - err = KCF_PROV_MAC_DECRYPT(pd, ctx, - dops->md_ciphertext, dops->md_mac, - dops->md_plaintext, rhndl); - break; - - case KCF_OP_UPDATE: - kcf_secondctx = ((kcf_context_t *) - (ctx->cc_framework_private))->kc_secondctx; - if (kcf_secondctx != NULL) { - err = kcf_emulate_dual(pd, ctx, params); - break; - } - err = KCF_PROV_MAC_DECRYPT_UPDATE(pd, ctx, - dops->md_ciphertext, dops->md_plaintext, rhndl); - break; - - case KCF_OP_FINAL: - kcf_secondctx = ((kcf_context_t *) - (ctx->cc_framework_private))->kc_secondctx; - if (kcf_secondctx != NULL) { - err = kcf_emulate_dual(pd, ctx, params); - break; - } - err = KCF_PROV_MAC_DECRYPT_FINAL(pd, ctx, - dops->md_mac, dops->md_plaintext, rhndl); - break; - - case KCF_OP_ATOMIC: - ASSERT(ctx == NULL); - - KCF_SET_PROVIDER_MECHNUM( - dops->md_framework_mac_mechtype, - pd, &dops->md_mac_mech); - - KCF_SET_PROVIDER_MECHNUM( - dops->md_framework_decr_mechtype, - pd, &dops->md_decr_mech); - - err = KCF_PROV_MAC_DECRYPT_ATOMIC(pd, dops->md_sid, - &dops->md_mac_mech, dops->md_mac_key, - &dops->md_decr_mech, dops->md_decr_key, - dops->md_ciphertext, dops->md_mac, - dops->md_plaintext, - dops->md_mac_templ, dops->md_decr_templ, - rhndl); - - break; - - case KCF_OP_MAC_VERIFY_DECRYPT_ATOMIC: - ASSERT(ctx == NULL); - - KCF_SET_PROVIDER_MECHNUM( - dops->md_framework_mac_mechtype, - pd, &dops->md_mac_mech); - - KCF_SET_PROVIDER_MECHNUM( - dops->md_framework_decr_mechtype, - pd, &dops->md_decr_mech); - - err = KCF_PROV_MAC_VERIFY_DECRYPT_ATOMIC(pd, - dops->md_sid, &dops->md_mac_mech, dops->md_mac_key, - &dops->md_decr_mech, dops->md_decr_key, - dops->md_ciphertext, dops->md_mac, - dops->md_plaintext, - dops->md_mac_templ, dops->md_decr_templ, - rhndl); - - break; - - default: - break; - } - break; - } - - case KCF_OG_KEY: { - kcf_key_ops_params_t *kops = ¶ms->rp_u.key_params; - - ASSERT(ctx == NULL); - KCF_SET_PROVIDER_MECHNUM(kops->ko_framework_mechtype, pd, - &kops->ko_mech); - - switch (optype) { - case KCF_OP_KEY_GENERATE: - err = KCF_PROV_KEY_GENERATE(pd, kops->ko_sid, - &kops->ko_mech, - kops->ko_key_template, kops->ko_key_attribute_count, - kops->ko_key_object_id_ptr, rhndl); - break; - - case KCF_OP_KEY_GENERATE_PAIR: - err = KCF_PROV_KEY_GENERATE_PAIR(pd, kops->ko_sid, - &kops->ko_mech, - kops->ko_key_template, kops->ko_key_attribute_count, - kops->ko_private_key_template, - kops->ko_private_key_attribute_count, - kops->ko_key_object_id_ptr, - kops->ko_private_key_object_id_ptr, rhndl); - break; - - case KCF_OP_KEY_WRAP: - err = KCF_PROV_KEY_WRAP(pd, kops->ko_sid, - &kops->ko_mech, - kops->ko_key, kops->ko_key_object_id_ptr, - kops->ko_wrapped_key, kops->ko_wrapped_key_len_ptr, - rhndl); - break; - - case KCF_OP_KEY_UNWRAP: - err = KCF_PROV_KEY_UNWRAP(pd, kops->ko_sid, - &kops->ko_mech, - kops->ko_key, kops->ko_wrapped_key, - kops->ko_wrapped_key_len_ptr, - kops->ko_key_template, kops->ko_key_attribute_count, - kops->ko_key_object_id_ptr, rhndl); - break; - - case KCF_OP_KEY_DERIVE: - err = KCF_PROV_KEY_DERIVE(pd, kops->ko_sid, - &kops->ko_mech, - kops->ko_key, kops->ko_key_template, - kops->ko_key_attribute_count, - kops->ko_key_object_id_ptr, rhndl); - break; - - default: - break; - } - break; - } - - case KCF_OG_RANDOM: { - kcf_random_number_ops_params_t *rops = - ¶ms->rp_u.random_number_params; - - ASSERT(ctx == NULL); - - switch (optype) { - case KCF_OP_RANDOM_SEED: - err = KCF_PROV_SEED_RANDOM(pd, rops->rn_sid, - rops->rn_buf, rops->rn_buflen, rops->rn_entropy_est, - rops->rn_flags, rhndl); - break; - - case KCF_OP_RANDOM_GENERATE: - err = KCF_PROV_GENERATE_RANDOM(pd, rops->rn_sid, - rops->rn_buf, rops->rn_buflen, rhndl); - break; - - default: - break; - } - break; - } - - case KCF_OG_SESSION: { - kcf_session_ops_params_t *sops = ¶ms->rp_u.session_params; - - ASSERT(ctx == NULL); - switch (optype) { - case KCF_OP_SESSION_OPEN: - /* - * so_pd may be a logical provider, in which case - * we need to check whether it has been removed. - */ - if (KCF_IS_PROV_REMOVED(sops->so_pd)) { - err = CRYPTO_DEVICE_ERROR; - break; - } - err = KCF_PROV_SESSION_OPEN(pd, sops->so_sid_ptr, - rhndl, sops->so_pd); - break; - - case KCF_OP_SESSION_CLOSE: - /* - * so_pd may be a logical provider, in which case - * we need to check whether it has been removed. - */ - if (KCF_IS_PROV_REMOVED(sops->so_pd)) { - err = CRYPTO_DEVICE_ERROR; - break; - } - err = KCF_PROV_SESSION_CLOSE(pd, sops->so_sid, - rhndl, sops->so_pd); - break; - - case KCF_OP_SESSION_LOGIN: - err = KCF_PROV_SESSION_LOGIN(pd, sops->so_sid, - sops->so_user_type, sops->so_pin, - sops->so_pin_len, rhndl); - break; - - case KCF_OP_SESSION_LOGOUT: - err = KCF_PROV_SESSION_LOGOUT(pd, sops->so_sid, rhndl); - break; - - default: - break; - } - break; - } - - case KCF_OG_OBJECT: { - kcf_object_ops_params_t *jops = ¶ms->rp_u.object_params; - - ASSERT(ctx == NULL); - switch (optype) { - case KCF_OP_OBJECT_CREATE: - err = KCF_PROV_OBJECT_CREATE(pd, jops->oo_sid, - jops->oo_template, jops->oo_attribute_count, - jops->oo_object_id_ptr, rhndl); - break; - - case KCF_OP_OBJECT_COPY: - err = KCF_PROV_OBJECT_COPY(pd, jops->oo_sid, - jops->oo_object_id, - jops->oo_template, jops->oo_attribute_count, - jops->oo_object_id_ptr, rhndl); - break; - - case KCF_OP_OBJECT_DESTROY: - err = KCF_PROV_OBJECT_DESTROY(pd, jops->oo_sid, - jops->oo_object_id, rhndl); - break; - - case KCF_OP_OBJECT_GET_SIZE: - err = KCF_PROV_OBJECT_GET_SIZE(pd, jops->oo_sid, - jops->oo_object_id, jops->oo_object_size, rhndl); - break; - - case KCF_OP_OBJECT_GET_ATTRIBUTE_VALUE: - err = KCF_PROV_OBJECT_GET_ATTRIBUTE_VALUE(pd, - jops->oo_sid, jops->oo_object_id, - jops->oo_template, jops->oo_attribute_count, rhndl); - break; - - case KCF_OP_OBJECT_SET_ATTRIBUTE_VALUE: - err = KCF_PROV_OBJECT_SET_ATTRIBUTE_VALUE(pd, - jops->oo_sid, jops->oo_object_id, - jops->oo_template, jops->oo_attribute_count, rhndl); - break; - - case KCF_OP_OBJECT_FIND_INIT: - err = KCF_PROV_OBJECT_FIND_INIT(pd, jops->oo_sid, - jops->oo_template, jops->oo_attribute_count, - jops->oo_find_init_pp_ptr, rhndl); - break; - - case KCF_OP_OBJECT_FIND: - err = KCF_PROV_OBJECT_FIND(pd, jops->oo_find_pp, - jops->oo_object_id_ptr, jops->oo_max_object_count, - jops->oo_object_count_ptr, rhndl); - break; - - case KCF_OP_OBJECT_FIND_FINAL: - err = KCF_PROV_OBJECT_FIND_FINAL(pd, jops->oo_find_pp, - rhndl); - break; - - default: - break; - } - break; - } - - case KCF_OG_PROVMGMT: { - kcf_provmgmt_ops_params_t *pops = ¶ms->rp_u.provmgmt_params; - - ASSERT(ctx == NULL); - switch (optype) { - case KCF_OP_MGMT_EXTINFO: - /* - * po_pd may be a logical provider, in which case - * we need to check whether it has been removed. - */ - if (KCF_IS_PROV_REMOVED(pops->po_pd)) { - err = CRYPTO_DEVICE_ERROR; - break; - } - err = KCF_PROV_EXT_INFO(pd, pops->po_ext_info, rhndl, - pops->po_pd); - break; - - case KCF_OP_MGMT_INITTOKEN: - err = KCF_PROV_INIT_TOKEN(pd, pops->po_pin, - pops->po_pin_len, pops->po_label, rhndl); - break; - - case KCF_OP_MGMT_INITPIN: - err = KCF_PROV_INIT_PIN(pd, pops->po_sid, pops->po_pin, - pops->po_pin_len, rhndl); - break; - - case KCF_OP_MGMT_SETPIN: - err = KCF_PROV_SET_PIN(pd, pops->po_sid, - pops->po_old_pin, pops->po_old_pin_len, - pops->po_pin, pops->po_pin_len, rhndl); - break; - - default: - break; - } - break; - } - - case KCF_OG_NOSTORE_KEY: { - kcf_key_ops_params_t *kops = ¶ms->rp_u.key_params; - - ASSERT(ctx == NULL); - KCF_SET_PROVIDER_MECHNUM(kops->ko_framework_mechtype, pd, - &kops->ko_mech); - - switch (optype) { - case KCF_OP_KEY_GENERATE: - err = KCF_PROV_NOSTORE_KEY_GENERATE(pd, kops->ko_sid, - &kops->ko_mech, kops->ko_key_template, - kops->ko_key_attribute_count, - kops->ko_out_template1, - kops->ko_out_attribute_count1, rhndl); - break; - - case KCF_OP_KEY_GENERATE_PAIR: - err = KCF_PROV_NOSTORE_KEY_GENERATE_PAIR(pd, - kops->ko_sid, &kops->ko_mech, - kops->ko_key_template, kops->ko_key_attribute_count, - kops->ko_private_key_template, - kops->ko_private_key_attribute_count, - kops->ko_out_template1, - kops->ko_out_attribute_count1, - kops->ko_out_template2, - kops->ko_out_attribute_count2, - rhndl); - break; - - case KCF_OP_KEY_DERIVE: - err = KCF_PROV_NOSTORE_KEY_DERIVE(pd, kops->ko_sid, - &kops->ko_mech, kops->ko_key, - kops->ko_key_template, - kops->ko_key_attribute_count, - kops->ko_out_template1, - kops->ko_out_attribute_count1, rhndl); - break; - - default: - break; - } - break; - } - default: - break; - } /* end of switch(params->rp_opgrp) */ - - KCF_PROV_INCRSTATS(pd, err); - return (err); -} - - -/* - * Emulate the call for a multipart dual ops with 2 single steps. - * This routine is always called in the context of a working thread - * running kcf_svc_do_run(). - * The single steps are submitted in a pure synchronous way (blocking). - * When this routine returns, kcf_svc_do_run() will call kcf_aop_done() - * so the originating consumer's callback gets invoked. kcf_aop_done() - * takes care of freeing the operation context. So, this routine does - * not free the operation context. - * - * The provider descriptor is assumed held by the callers. - */ -static int -kcf_emulate_dual(kcf_provider_desc_t *pd, crypto_ctx_t *ctx, - kcf_req_params_t *params) -{ - int err = CRYPTO_ARGUMENTS_BAD; - kcf_op_type_t optype; - size_t save_len; - off_t save_offset; - - optype = params->rp_optype; - - switch (params->rp_opgrp) { - case KCF_OG_ENCRYPT_MAC: { - kcf_encrypt_mac_ops_params_t *cmops = - ¶ms->rp_u.encrypt_mac_params; - kcf_context_t *encr_kcf_ctx; - crypto_ctx_t *mac_ctx; - kcf_req_params_t encr_params; - - encr_kcf_ctx = (kcf_context_t *)(ctx->cc_framework_private); - - switch (optype) { - case KCF_OP_INIT: { - encr_kcf_ctx->kc_secondctx = NULL; - - KCF_WRAP_ENCRYPT_OPS_PARAMS(&encr_params, KCF_OP_INIT, - pd->pd_sid, &cmops->em_encr_mech, - cmops->em_encr_key, NULL, NULL, - cmops->em_encr_templ); - - err = kcf_submit_request(pd, ctx, NULL, &encr_params, - B_FALSE); - - /* It can't be CRYPTO_QUEUED */ - if (err != CRYPTO_SUCCESS) { - break; - } - - err = crypto_mac_init(&cmops->em_mac_mech, - cmops->em_mac_key, cmops->em_mac_templ, - (crypto_context_t *)&mac_ctx, NULL); - - if (err == CRYPTO_SUCCESS) { - encr_kcf_ctx->kc_secondctx = (kcf_context_t *) - mac_ctx->cc_framework_private; - KCF_CONTEXT_REFHOLD((kcf_context_t *) - mac_ctx->cc_framework_private); - } - - break; - - } - case KCF_OP_UPDATE: { - crypto_dual_data_t *ct = cmops->em_ciphertext; - crypto_data_t *pt = cmops->em_plaintext; - kcf_context_t *mac_kcf_ctx = encr_kcf_ctx->kc_secondctx; - crypto_ctx_t *mac_ctx = &mac_kcf_ctx->kc_glbl_ctx; - - KCF_WRAP_ENCRYPT_OPS_PARAMS(&encr_params, KCF_OP_UPDATE, - pd->pd_sid, NULL, NULL, pt, (crypto_data_t *)ct, - NULL); - - err = kcf_submit_request(pd, ctx, NULL, &encr_params, - B_FALSE); - - /* It can't be CRYPTO_QUEUED */ - if (err != CRYPTO_SUCCESS) { - break; - } - - save_offset = ct->dd_offset1; - save_len = ct->dd_len1; - if (ct->dd_len2 == 0) { - /* - * The previous encrypt step was an - * accumulation only and didn't produce any - * partial output - */ - if (ct->dd_len1 == 0) - break; - - } else { - ct->dd_offset1 = ct->dd_offset2; - ct->dd_len1 = ct->dd_len2; - } - err = crypto_mac_update((crypto_context_t)mac_ctx, - (crypto_data_t *)ct, NULL); - - ct->dd_offset1 = save_offset; - ct->dd_len1 = save_len; - - break; - } - case KCF_OP_FINAL: { - crypto_dual_data_t *ct = cmops->em_ciphertext; - crypto_data_t *mac = cmops->em_mac; - kcf_context_t *mac_kcf_ctx = encr_kcf_ctx->kc_secondctx; - crypto_ctx_t *mac_ctx = &mac_kcf_ctx->kc_glbl_ctx; - crypto_context_t mac_context = mac_ctx; - - KCF_WRAP_ENCRYPT_OPS_PARAMS(&encr_params, KCF_OP_FINAL, - pd->pd_sid, NULL, NULL, NULL, (crypto_data_t *)ct, - NULL); - - err = kcf_submit_request(pd, ctx, NULL, &encr_params, - B_FALSE); - - /* It can't be CRYPTO_QUEUED */ - if (err != CRYPTO_SUCCESS) { - crypto_cancel_ctx(mac_context); - break; - } - - if (ct->dd_len2 > 0) { - save_offset = ct->dd_offset1; - save_len = ct->dd_len1; - ct->dd_offset1 = ct->dd_offset2; - ct->dd_len1 = ct->dd_len2; - - err = crypto_mac_update(mac_context, - (crypto_data_t *)ct, NULL); - - ct->dd_offset1 = save_offset; - ct->dd_len1 = save_len; - - if (err != CRYPTO_SUCCESS) { - crypto_cancel_ctx(mac_context); - return (err); - } - } - - /* and finally, collect the MAC */ - err = crypto_mac_final(mac_context, mac, NULL); - break; - } - - default: - break; - } - KCF_PROV_INCRSTATS(pd, err); - break; - } - case KCF_OG_MAC_DECRYPT: { - kcf_mac_decrypt_ops_params_t *mdops = - ¶ms->rp_u.mac_decrypt_params; - kcf_context_t *decr_kcf_ctx; - crypto_ctx_t *mac_ctx; - kcf_req_params_t decr_params; - - decr_kcf_ctx = (kcf_context_t *)(ctx->cc_framework_private); - - switch (optype) { - case KCF_OP_INIT: { - decr_kcf_ctx->kc_secondctx = NULL; - - err = crypto_mac_init(&mdops->md_mac_mech, - mdops->md_mac_key, mdops->md_mac_templ, - (crypto_context_t *)&mac_ctx, NULL); - - /* It can't be CRYPTO_QUEUED */ - if (err != CRYPTO_SUCCESS) { - break; - } - - KCF_WRAP_DECRYPT_OPS_PARAMS(&decr_params, KCF_OP_INIT, - pd->pd_sid, &mdops->md_decr_mech, - mdops->md_decr_key, NULL, NULL, - mdops->md_decr_templ); - - err = kcf_submit_request(pd, ctx, NULL, &decr_params, - B_FALSE); - - /* It can't be CRYPTO_QUEUED */ - if (err != CRYPTO_SUCCESS) { - crypto_cancel_ctx((crypto_context_t)mac_ctx); - break; - } - - decr_kcf_ctx->kc_secondctx = (kcf_context_t *) - mac_ctx->cc_framework_private; - KCF_CONTEXT_REFHOLD((kcf_context_t *) - mac_ctx->cc_framework_private); - - break; - default: - break; - - } - case KCF_OP_UPDATE: { - crypto_dual_data_t *ct = mdops->md_ciphertext; - crypto_data_t *pt = mdops->md_plaintext; - kcf_context_t *mac_kcf_ctx = decr_kcf_ctx->kc_secondctx; - crypto_ctx_t *mac_ctx = &mac_kcf_ctx->kc_glbl_ctx; - - err = crypto_mac_update((crypto_context_t)mac_ctx, - (crypto_data_t *)ct, NULL); - - if (err != CRYPTO_SUCCESS) - break; - - save_offset = ct->dd_offset1; - save_len = ct->dd_len1; - - /* zero ct->dd_len2 means decrypt everything */ - if (ct->dd_len2 > 0) { - ct->dd_offset1 = ct->dd_offset2; - ct->dd_len1 = ct->dd_len2; - } - - err = crypto_decrypt_update((crypto_context_t)ctx, - (crypto_data_t *)ct, pt, NULL); - - ct->dd_offset1 = save_offset; - ct->dd_len1 = save_len; - - break; - } - case KCF_OP_FINAL: { - crypto_data_t *pt = mdops->md_plaintext; - crypto_data_t *mac = mdops->md_mac; - kcf_context_t *mac_kcf_ctx = decr_kcf_ctx->kc_secondctx; - crypto_ctx_t *mac_ctx = &mac_kcf_ctx->kc_glbl_ctx; - - err = crypto_mac_final((crypto_context_t)mac_ctx, - mac, NULL); - - if (err != CRYPTO_SUCCESS) { - crypto_cancel_ctx(ctx); - break; - } - - /* Get the last chunk of plaintext */ - KCF_CONTEXT_REFHOLD(decr_kcf_ctx); - err = crypto_decrypt_final((crypto_context_t)ctx, pt, - NULL); - - break; - } - } - break; - } - default: - - break; - } /* end of switch(params->rp_opgrp) */ - - return (err); -} diff --git a/module/icp/core/kcf_mech_tabs.c b/module/icp/core/kcf_mech_tabs.c index 60055e78af68..3d5063b28f6e 100644 --- a/module/icp/core/kcf_mech_tabs.c +++ b/module/icp/core/kcf_mech_tabs.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -27,7 +27,6 @@ #include #include #include -#include /* Cryptographic mechanisms tables and their access functions */ @@ -55,9 +54,6 @@ /* * Locking conventions: * -------------------- - * A global mutex, kcf_mech_tabs_lock, serializes writes to the - * mechanism table via kcf_create_mech_entry(). - * * A mutex is associated with every entry of the tables. * The mutex is acquired whenever the entry is accessed for * 1) retrieving the mech_id (comparing the mech name) @@ -72,9 +68,6 @@ * long enough to justify the cost of using rwlocks, so the per-mechanism * entry mutex won't be very *hot*. * - * When both kcf_mech_tabs_lock and a mech_entry mutex need to be held, - * kcf_mech_tabs_lock must always be acquired first. - * */ /* Mechanisms tables */ @@ -85,76 +78,30 @@ static kcf_mech_entry_t kcf_digest_mechs_tab[KCF_MAXDIGEST]; static kcf_mech_entry_t kcf_cipher_mechs_tab[KCF_MAXCIPHER]; static kcf_mech_entry_t kcf_mac_mechs_tab[KCF_MAXMAC]; -static kcf_mech_entry_t kcf_sign_mechs_tab[KCF_MAXSIGN]; -static kcf_mech_entry_t kcf_keyops_mechs_tab[KCF_MAXKEYOPS]; -static kcf_mech_entry_t kcf_misc_mechs_tab[KCF_MAXMISC]; const kcf_mech_entry_tab_t kcf_mech_tabs_tab[KCF_LAST_OPSCLASS + 1] = { {0, NULL}, /* No class zero */ {KCF_MAXDIGEST, kcf_digest_mechs_tab}, {KCF_MAXCIPHER, kcf_cipher_mechs_tab}, {KCF_MAXMAC, kcf_mac_mechs_tab}, - {KCF_MAXSIGN, kcf_sign_mechs_tab}, - {KCF_MAXKEYOPS, kcf_keyops_mechs_tab}, - {KCF_MAXMISC, kcf_misc_mechs_tab} }; -/* - * Per-algorithm internal thresholds for the minimum input size of before - * offloading to hardware provider. - * Dispatching a crypto operation to a hardware provider entails paying the - * cost of an additional context switch. Measurements with Sun Accelerator 4000 - * shows that 512-byte jobs or smaller are better handled in software. - * There is room for refinement here. - * - */ -static const int kcf_md5_threshold = 512; -static const int kcf_sha1_threshold = 512; -static const int kcf_des_threshold = 512; -static const int kcf_des3_threshold = 512; -static const int kcf_aes_threshold = 512; -static const int kcf_bf_threshold = 512; -static const int kcf_rc4_threshold = 512; - -static kmutex_t kcf_mech_tabs_lock; -static uint32_t kcf_gen_swprov = 0; - -static const int kcf_mech_hash_size = 256; -static mod_hash_t *kcf_mech_hash; /* mech name to id hash */ - -static crypto_mech_type_t -kcf_mech_hash_find(const char *mechname) -{ - mod_hash_val_t hv; - crypto_mech_type_t mt; - - mt = CRYPTO_MECH_INVALID; - if (mod_hash_find(kcf_mech_hash, (mod_hash_key_t)mechname, &hv) == 0) { - mt = *(crypto_mech_type_t *)hv; - ASSERT(mt != CRYPTO_MECH_INVALID); - } +static avl_tree_t kcf_mech_hash; - return (mt); +static int +kcf_mech_hash_compar(const void *lhs, const void *rhs) +{ + const kcf_mech_entry_t *l = lhs, *r = rhs; + int cmp = strncmp(l->me_name, r->me_name, CRYPTO_MAX_MECH_NAME); + return ((0 < cmp) - (cmp < 0)); } void kcf_destroy_mech_tabs(void) { - int i, max; - kcf_ops_class_t class; - kcf_mech_entry_t *me_tab; - - if (kcf_mech_hash) - mod_hash_destroy_hash(kcf_mech_hash); - - mutex_destroy(&kcf_mech_tabs_lock); - - for (class = KCF_FIRST_OPSCLASS; class <= KCF_LAST_OPSCLASS; class++) { - max = kcf_mech_tabs_tab[class].met_size; - me_tab = kcf_mech_tabs_tab[class].met_tab; - for (i = 0; i < max; i++) - mutex_destroy(&(me_tab[i].me_mutex)); - } + for (void *cookie = NULL; avl_destroy_nodes(&kcf_mech_hash, &cookie); ) + ; + avl_destroy(&kcf_mech_hash); } /* @@ -166,101 +113,8 @@ kcf_destroy_mech_tabs(void) void kcf_init_mech_tabs(void) { - kcf_ops_class_t class; - kcf_mech_entry_t *me_tab; - - /* Initializes the mutex locks. */ - - mutex_init(&kcf_mech_tabs_lock, NULL, MUTEX_DEFAULT, NULL); - - /* Then the pre-defined mechanism entries */ - - /* Two digests */ - (void) strncpy(kcf_digest_mechs_tab[0].me_name, SUN_CKM_MD5, - CRYPTO_MAX_MECH_NAME); - kcf_digest_mechs_tab[0].me_threshold = kcf_md5_threshold; - - (void) strncpy(kcf_digest_mechs_tab[1].me_name, SUN_CKM_SHA1, - CRYPTO_MAX_MECH_NAME); - kcf_digest_mechs_tab[1].me_threshold = kcf_sha1_threshold; - - /* The symmetric ciphers in various modes */ - (void) strncpy(kcf_cipher_mechs_tab[0].me_name, SUN_CKM_DES_CBC, - CRYPTO_MAX_MECH_NAME); - kcf_cipher_mechs_tab[0].me_threshold = kcf_des_threshold; - - (void) strncpy(kcf_cipher_mechs_tab[1].me_name, SUN_CKM_DES3_CBC, - CRYPTO_MAX_MECH_NAME); - kcf_cipher_mechs_tab[1].me_threshold = kcf_des3_threshold; - - (void) strncpy(kcf_cipher_mechs_tab[2].me_name, SUN_CKM_DES_ECB, - CRYPTO_MAX_MECH_NAME); - kcf_cipher_mechs_tab[2].me_threshold = kcf_des_threshold; - - (void) strncpy(kcf_cipher_mechs_tab[3].me_name, SUN_CKM_DES3_ECB, - CRYPTO_MAX_MECH_NAME); - kcf_cipher_mechs_tab[3].me_threshold = kcf_des3_threshold; - - (void) strncpy(kcf_cipher_mechs_tab[4].me_name, SUN_CKM_BLOWFISH_CBC, - CRYPTO_MAX_MECH_NAME); - kcf_cipher_mechs_tab[4].me_threshold = kcf_bf_threshold; - - (void) strncpy(kcf_cipher_mechs_tab[5].me_name, SUN_CKM_BLOWFISH_ECB, - CRYPTO_MAX_MECH_NAME); - kcf_cipher_mechs_tab[5].me_threshold = kcf_bf_threshold; - - (void) strncpy(kcf_cipher_mechs_tab[6].me_name, SUN_CKM_AES_CBC, - CRYPTO_MAX_MECH_NAME); - kcf_cipher_mechs_tab[6].me_threshold = kcf_aes_threshold; - - (void) strncpy(kcf_cipher_mechs_tab[7].me_name, SUN_CKM_AES_ECB, - CRYPTO_MAX_MECH_NAME); - kcf_cipher_mechs_tab[7].me_threshold = kcf_aes_threshold; - - (void) strncpy(kcf_cipher_mechs_tab[8].me_name, SUN_CKM_RC4, - CRYPTO_MAX_MECH_NAME); - kcf_cipher_mechs_tab[8].me_threshold = kcf_rc4_threshold; - - - /* 4 HMACs */ - (void) strncpy(kcf_mac_mechs_tab[0].me_name, SUN_CKM_MD5_HMAC, - CRYPTO_MAX_MECH_NAME); - kcf_mac_mechs_tab[0].me_threshold = kcf_md5_threshold; - - (void) strncpy(kcf_mac_mechs_tab[1].me_name, SUN_CKM_MD5_HMAC_GENERAL, - CRYPTO_MAX_MECH_NAME); - kcf_mac_mechs_tab[1].me_threshold = kcf_md5_threshold; - - (void) strncpy(kcf_mac_mechs_tab[2].me_name, SUN_CKM_SHA1_HMAC, - CRYPTO_MAX_MECH_NAME); - kcf_mac_mechs_tab[2].me_threshold = kcf_sha1_threshold; - - (void) strncpy(kcf_mac_mechs_tab[3].me_name, SUN_CKM_SHA1_HMAC_GENERAL, - CRYPTO_MAX_MECH_NAME); - kcf_mac_mechs_tab[3].me_threshold = kcf_sha1_threshold; - - - /* 1 random number generation pseudo mechanism */ - (void) strncpy(kcf_misc_mechs_tab[0].me_name, SUN_RANDOM, - CRYPTO_MAX_MECH_NAME); - - kcf_mech_hash = mod_hash_create_strhash_nodtr("kcf mech2id hash", - kcf_mech_hash_size, mod_hash_null_valdtor); - - for (class = KCF_FIRST_OPSCLASS; class <= KCF_LAST_OPSCLASS; class++) { - int max = kcf_mech_tabs_tab[class].met_size; - me_tab = kcf_mech_tabs_tab[class].met_tab; - for (int i = 0; i < max; i++) { - mutex_init(&(me_tab[i].me_mutex), NULL, - MUTEX_DEFAULT, NULL); - if (me_tab[i].me_name[0] != 0) { - me_tab[i].me_mechid = KCF_MECHID(class, i); - (void) mod_hash_insert(kcf_mech_hash, - (mod_hash_key_t)me_tab[i].me_name, - (mod_hash_val_t)&(me_tab[i].me_mechid)); - } - } - } + avl_create(&kcf_mech_hash, kcf_mech_hash_compar, + sizeof (kcf_mech_entry_t), offsetof(kcf_mech_entry_t, me_node)); } /* @@ -290,12 +144,8 @@ kcf_init_mech_tabs(void) * KCF_SUCCESS otherwise. */ static int -kcf_create_mech_entry(kcf_ops_class_t class, char *mechname) +kcf_create_mech_entry(kcf_ops_class_t class, const char *mechname) { - crypto_mech_type_t mt; - kcf_mech_entry_t *me_tab; - int i = 0, size; - if ((class < KCF_FIRST_OPSCLASS) || (class > KCF_LAST_OPSCLASS)) return (KCF_INVALID_MECH_CLASS); @@ -305,49 +155,28 @@ kcf_create_mech_entry(kcf_ops_class_t class, char *mechname) * First check if the mechanism is already in one of the tables. * The mech_entry could be in another class. */ - mutex_enter(&kcf_mech_tabs_lock); - mt = kcf_mech_hash_find(mechname); - if (mt != CRYPTO_MECH_INVALID) { - /* Nothing to do, regardless the suggested class. */ - mutex_exit(&kcf_mech_tabs_lock); + avl_index_t where = 0; + kcf_mech_entry_t tmptab; + strlcpy(tmptab.me_name, mechname, CRYPTO_MAX_MECH_NAME); + if (avl_find(&kcf_mech_hash, &tmptab, &where) != NULL) return (KCF_SUCCESS); - } /* Now take the next unused mech entry in the class's tab */ - me_tab = kcf_mech_tabs_tab[class].met_tab; - size = kcf_mech_tabs_tab[class].met_size; + kcf_mech_entry_t *me_tab = kcf_mech_tabs_tab[class].met_tab; + int size = kcf_mech_tabs_tab[class].met_size; - while (i < size) { - mutex_enter(&(me_tab[i].me_mutex)); + for (int i = 0; i < size; ++i) if (me_tab[i].me_name[0] == 0) { /* Found an empty spot */ - (void) strlcpy(me_tab[i].me_name, mechname, + strlcpy(me_tab[i].me_name, mechname, CRYPTO_MAX_MECH_NAME); - me_tab[i].me_name[CRYPTO_MAX_MECH_NAME-1] = '\0'; me_tab[i].me_mechid = KCF_MECHID(class, i); - /* - * No a-priori information about the new mechanism, so - * the threshold is set to zero. - */ - me_tab[i].me_threshold = 0; - mutex_exit(&(me_tab[i].me_mutex)); /* Add the new mechanism to the hash table */ - (void) mod_hash_insert(kcf_mech_hash, - (mod_hash_key_t)me_tab[i].me_name, - (mod_hash_val_t)&(me_tab[i].me_mechid)); - break; + avl_insert(&kcf_mech_hash, &me_tab[i], where); + return (KCF_SUCCESS); } - mutex_exit(&(me_tab[i].me_mutex)); - i++; - } - mutex_exit(&kcf_mech_tabs_lock); - - if (i == size) { - return (KCF_MECH_TAB_FULL); - } - - return (KCF_SUCCESS); + return (KCF_MECH_TAB_FULL); } /* @@ -375,16 +204,9 @@ kcf_add_mech_provider(short mech_indx, { int error; kcf_mech_entry_t *mech_entry = NULL; - crypto_mech_info_t *mech_info; - crypto_mech_type_t kcf_mech_type, mt; - kcf_prov_mech_desc_t *prov_mech, *prov_mech2; - crypto_func_group_t simple_fg_mask, dual_fg_mask; - crypto_mech_info_t *dmi; - crypto_mech_info_list_t *mil, *mil2; - kcf_mech_entry_t *me; - int i; - - ASSERT(prov_desc->pd_prov_type != CRYPTO_LOGICAL_PROVIDER); + const crypto_mech_info_t *mech_info; + crypto_mech_type_t kcf_mech_type; + kcf_prov_mech_desc_t *prov_mech; mech_info = &prov_desc->pd_mechanisms[mech_indx]; @@ -393,7 +215,7 @@ kcf_add_mech_provider(short mech_indx, * Find the class corresponding to the function group flag of * the mechanism. */ - kcf_mech_type = kcf_mech_hash_find(mech_info->cm_mech_name); + kcf_mech_type = crypto_mech2id(mech_info->cm_mech_name); if (kcf_mech_type == CRYPTO_MECH_INVALID) { crypto_func_group_t fg = mech_info->cm_func_group_mask; kcf_ops_class_t class; @@ -406,19 +228,8 @@ kcf_add_mech_provider(short mech_indx, class = KCF_CIPHER_CLASS; else if (fg & CRYPTO_FG_MAC || fg & CRYPTO_FG_MAC_ATOMIC) class = KCF_MAC_CLASS; - else if (fg & CRYPTO_FG_SIGN || fg & CRYPTO_FG_VERIFY || - fg & CRYPTO_FG_SIGN_ATOMIC || - fg & CRYPTO_FG_VERIFY_ATOMIC || - fg & CRYPTO_FG_SIGN_RECOVER || - fg & CRYPTO_FG_VERIFY_RECOVER) - class = KCF_SIGN_CLASS; - else if (fg & CRYPTO_FG_GENERATE || - fg & CRYPTO_FG_GENERATE_KEY_PAIR || - fg & CRYPTO_FG_WRAP || fg & CRYPTO_FG_UNWRAP || - fg & CRYPTO_FG_DERIVE) - class = KCF_KEYOPS_CLASS; else - class = KCF_MISC_CLASS; + __builtin_unreachable(); /* * Attempt to create a new mech_entry for the specified @@ -430,7 +241,7 @@ kcf_add_mech_provider(short mech_indx, return (error); } /* get the KCF mech type that was assigned to the mechanism */ - kcf_mech_type = kcf_mech_hash_find(mech_info->cm_mech_name); + kcf_mech_type = crypto_mech2id(mech_info->cm_mech_name); ASSERT(kcf_mech_type != CRYPTO_MECH_INVALID); } @@ -439,7 +250,8 @@ kcf_add_mech_provider(short mech_indx, /* allocate and initialize new kcf_prov_mech_desc */ prov_mech = kmem_zalloc(sizeof (kcf_prov_mech_desc_t), KM_SLEEP); - bcopy(mech_info, &prov_mech->pm_mech_info, sizeof (crypto_mech_info_t)); + memcpy(&prov_mech->pm_mech_info, mech_info, + sizeof (crypto_mech_info_t)); prov_mech->pm_prov_desc = prov_desc; prov_desc->pd_mech_indx[KCF_MECH2CLASS(kcf_mech_type)] [KCF_MECH2INDEX(kcf_mech_type)] = mech_indx; @@ -447,142 +259,32 @@ kcf_add_mech_provider(short mech_indx, KCF_PROV_REFHOLD(prov_desc); KCF_PROV_IREFHOLD(prov_desc); - dual_fg_mask = mech_info->cm_func_group_mask & CRYPTO_FG_DUAL_MASK; - - if (dual_fg_mask == ((crypto_func_group_t)0)) - goto add_entry; - - simple_fg_mask = (mech_info->cm_func_group_mask & - CRYPTO_FG_SIMPLEOP_MASK) | CRYPTO_FG_RANDOM; - - for (i = 0; i < prov_desc->pd_mech_list_count; i++) { - dmi = &prov_desc->pd_mechanisms[i]; - - /* skip self */ - if (dmi->cm_mech_number == mech_info->cm_mech_number) - continue; - - /* skip if not a dual operation mechanism */ - if (!(dmi->cm_func_group_mask & dual_fg_mask) || - (dmi->cm_func_group_mask & simple_fg_mask)) - continue; - - mt = kcf_mech_hash_find(dmi->cm_mech_name); - if (mt == CRYPTO_MECH_INVALID) - continue; - - if (kcf_get_mech_entry(mt, &me) != KCF_SUCCESS) - continue; - - mil = kmem_zalloc(sizeof (*mil), KM_SLEEP); - mil2 = kmem_zalloc(sizeof (*mil2), KM_SLEEP); - - /* - * Ignore hard-coded entries in the mech table - * if the provider hasn't registered. - */ - mutex_enter(&me->me_mutex); - if (me->me_hw_prov_chain == NULL && me->me_sw_prov == NULL) { - mutex_exit(&me->me_mutex); - kmem_free(mil, sizeof (*mil)); - kmem_free(mil2, sizeof (*mil2)); - continue; - } - - /* - * Add other dual mechanisms that have registered - * with the framework to this mechanism's - * cross-reference list. - */ - mil->ml_mech_info = *dmi; /* struct assignment */ - mil->ml_kcf_mechid = mt; - - /* add to head of list */ - mil->ml_next = prov_mech->pm_mi_list; - prov_mech->pm_mi_list = mil; - - if (prov_desc->pd_prov_type == CRYPTO_HW_PROVIDER) - prov_mech2 = me->me_hw_prov_chain; - else - prov_mech2 = me->me_sw_prov; - - if (prov_mech2 == NULL) { - kmem_free(mil2, sizeof (*mil2)); - mutex_exit(&me->me_mutex); - continue; - } - - /* - * Update all other cross-reference lists by - * adding this new mechanism. - */ - while (prov_mech2 != NULL) { - if (prov_mech2->pm_prov_desc == prov_desc) { - /* struct assignment */ - mil2->ml_mech_info = *mech_info; - mil2->ml_kcf_mechid = kcf_mech_type; - - /* add to head of list */ - mil2->ml_next = prov_mech2->pm_mi_list; - prov_mech2->pm_mi_list = mil2; - break; - } - prov_mech2 = prov_mech2->pm_next; - } - if (prov_mech2 == NULL) - kmem_free(mil2, sizeof (*mil2)); - - mutex_exit(&me->me_mutex); - } - -add_entry: /* * Add new kcf_prov_mech_desc at the front of HW providers * chain. */ - switch (prov_desc->pd_prov_type) { - - case CRYPTO_HW_PROVIDER: - mutex_enter(&mech_entry->me_mutex); - prov_mech->pm_me = mech_entry; - prov_mech->pm_next = mech_entry->me_hw_prov_chain; - mech_entry->me_hw_prov_chain = prov_mech; - mech_entry->me_num_hwprov++; - mutex_exit(&mech_entry->me_mutex); - break; - - case CRYPTO_SW_PROVIDER: - mutex_enter(&mech_entry->me_mutex); - if (mech_entry->me_sw_prov != NULL) { - /* - * There is already a SW provider for this mechanism. - * Since we allow only one SW provider per mechanism, - * report this condition. - */ - cmn_err(CE_WARN, "The cryptographic software provider " - "\"%s\" will not be used for %s. The provider " - "\"%s\" will be used for this mechanism " - "instead.", prov_desc->pd_description, - mech_info->cm_mech_name, - mech_entry->me_sw_prov->pm_prov_desc-> - pd_description); - KCF_PROV_REFRELE(prov_desc); - kmem_free(prov_mech, sizeof (kcf_prov_mech_desc_t)); - prov_mech = NULL; - } else { - /* - * Set the provider as the software provider for - * this mechanism. - */ - mech_entry->me_sw_prov = prov_mech; - - /* We'll wrap around after 4 billion registrations! */ - mech_entry->me_gen_swprov = kcf_gen_swprov++; - } - mutex_exit(&mech_entry->me_mutex); - break; - default: - break; + if (mech_entry->me_sw_prov != NULL) { + /* + * There is already a provider for this mechanism. + * Since we allow only one provider per mechanism, + * report this condition. + */ + cmn_err(CE_WARN, "The cryptographic provider " + "\"%s\" will not be used for %s. The provider " + "\"%s\" will be used for this mechanism " + "instead.", prov_desc->pd_description, + mech_info->cm_mech_name, + mech_entry->me_sw_prov->pm_prov_desc-> + pd_description); + KCF_PROV_REFRELE(prov_desc); + kmem_free(prov_mech, sizeof (kcf_prov_mech_desc_t)); + prov_mech = NULL; + } else { + /* + * Set the provider as the provider for + * this mechanism. + */ + mech_entry->me_sw_prov = prov_mech; } *pmdpp = prov_mech; @@ -606,18 +308,14 @@ kcf_add_mech_provider(short mech_indx, * User context only. */ void -kcf_remove_mech_provider(char *mech_name, kcf_provider_desc_t *prov_desc) +kcf_remove_mech_provider(const char *mech_name, kcf_provider_desc_t *prov_desc) { crypto_mech_type_t mech_type; - kcf_prov_mech_desc_t *prov_mech = NULL, *prov_chain; - kcf_prov_mech_desc_t **prev_entry_next; + kcf_prov_mech_desc_t *prov_mech = NULL; kcf_mech_entry_t *mech_entry; - crypto_mech_info_list_t *mil, *mil2, *next, **prev_next; - - ASSERT(prov_desc->pd_prov_type != CRYPTO_LOGICAL_PROVIDER); /* get the KCF mech type that was assigned to the mechanism */ - if ((mech_type = kcf_mech_hash_find(mech_name)) == + if ((mech_type = crypto_mech2id(mech_name)) == CRYPTO_MECH_INVALID) { /* * Provider was not allowed for this mech due to policy or @@ -635,88 +333,13 @@ kcf_remove_mech_provider(char *mech_name, kcf_provider_desc_t *prov_desc) return; } - mutex_enter(&mech_entry->me_mutex); - - switch (prov_desc->pd_prov_type) { - - case CRYPTO_HW_PROVIDER: - /* find the provider in the mech_entry chain */ - prev_entry_next = &mech_entry->me_hw_prov_chain; - prov_mech = mech_entry->me_hw_prov_chain; - while (prov_mech != NULL && - prov_mech->pm_prov_desc != prov_desc) { - prev_entry_next = &prov_mech->pm_next; - prov_mech = prov_mech->pm_next; - } - - if (prov_mech == NULL) { - /* entry not found, simply return */ - mutex_exit(&mech_entry->me_mutex); - return; - } - - /* remove provider entry from mech_entry chain */ - *prev_entry_next = prov_mech->pm_next; - ASSERT(mech_entry->me_num_hwprov > 0); - mech_entry->me_num_hwprov--; - break; - - case CRYPTO_SW_PROVIDER: - if (mech_entry->me_sw_prov == NULL || - mech_entry->me_sw_prov->pm_prov_desc != prov_desc) { - /* not the software provider for this mechanism */ - mutex_exit(&mech_entry->me_mutex); - return; - } - prov_mech = mech_entry->me_sw_prov; - mech_entry->me_sw_prov = NULL; - break; - default: - /* unexpected crypto_provider_type_t */ - mutex_exit(&mech_entry->me_mutex); + if (mech_entry->me_sw_prov == NULL || + mech_entry->me_sw_prov->pm_prov_desc != prov_desc) { + /* not the provider for this mechanism */ return; } - - mutex_exit(&mech_entry->me_mutex); - - /* Free the dual ops cross-reference lists */ - mil = prov_mech->pm_mi_list; - while (mil != NULL) { - next = mil->ml_next; - if (kcf_get_mech_entry(mil->ml_kcf_mechid, - &mech_entry) != KCF_SUCCESS) { - mil = next; - continue; - } - - mutex_enter(&mech_entry->me_mutex); - if (prov_desc->pd_prov_type == CRYPTO_HW_PROVIDER) - prov_chain = mech_entry->me_hw_prov_chain; - else - prov_chain = mech_entry->me_sw_prov; - - while (prov_chain != NULL) { - if (prov_chain->pm_prov_desc == prov_desc) { - prev_next = &prov_chain->pm_mi_list; - mil2 = prov_chain->pm_mi_list; - while (mil2 != NULL && - mil2->ml_kcf_mechid != mech_type) { - prev_next = &mil2->ml_next; - mil2 = mil2->ml_next; - } - if (mil2 != NULL) { - *prev_next = mil2->ml_next; - kmem_free(mil2, sizeof (*mil2)); - } - break; - } - prov_chain = prov_chain->pm_next; - } - - mutex_exit(&mech_entry->me_mutex); - kmem_free(mil, sizeof (crypto_mech_info_list_t)); - mil = next; - } + prov_mech = mech_entry->me_sw_prov; + mech_entry->me_sw_prov = NULL; /* free entry */ KCF_PROV_REFRELE(prov_mech->pm_prov_desc); @@ -769,16 +392,44 @@ kcf_get_mech_entry(crypto_mech_type_t mech_type, kcf_mech_entry_t **mep) return (KCF_SUCCESS); } -/* CURRENTLY UNSUPPORTED: attempting to load the module if it isn't found */ +/* + * crypto_mech2id() + * + * Arguments: + * . mechname: A null-terminated string identifying the mechanism name. + * + * Description: + * Walks the mechanisms tables, looking for an entry that matches the + * mechname. Once it find it, it builds the 64-bit mech_type and returns + * it. + * + * Context: + * Process and interruption. + * + * Returns: + * The unique mechanism identified by 'mechname', if found. + * CRYPTO_MECH_INVALID otherwise. + */ /* * Lookup the hash table for an entry that matches the mechname. - * If there are no hardware or software providers for the mechanism, - * but there is an unloaded software provider, this routine will attempt + * If there are no providers for the mechanism, + * but there is an unloaded provider, this routine will attempt * to load it. */ crypto_mech_type_t -crypto_mech2id_common(const char *mechname, boolean_t load_module) +crypto_mech2id(const char *mechname) { - (void) load_module; - return (kcf_mech_hash_find(mechname)); + kcf_mech_entry_t tmptab, *found; + strlcpy(tmptab.me_name, mechname, CRYPTO_MAX_MECH_NAME); + + if ((found = avl_find(&kcf_mech_hash, &tmptab, NULL))) { + ASSERT(found->me_mechid != CRYPTO_MECH_INVALID); + return (found->me_mechid); + } + + return (CRYPTO_MECH_INVALID); } + +#if defined(_KERNEL) +EXPORT_SYMBOL(crypto_mech2id); +#endif diff --git a/module/icp/core/kcf_prov_lib.c b/module/icp/core/kcf_prov_lib.c index 6e8853c56dc6..9dca3882e174 100644 --- a/module/icp/core/kcf_prov_lib.c +++ b/module/icp/core/kcf_prov_lib.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -33,14 +33,12 @@ */ /* - * Utility routine to apply the command, 'cmd', to the + * Utility routine to apply the command COPY_TO_DATA to the * data in the uio structure. */ -int -crypto_uio_data(crypto_data_t *data, uchar_t *buf, int len, cmd_type_t cmd, - void *digest_ctx, void (*update)(void)) +static int +crypto_uio_copy_to_data(crypto_data_t *data, uchar_t *buf, int len) { - (void) digest_ctx, (void) update; zfs_uio_t *uiop = data->cd_uio; off_t offset = data->cd_offset; size_t length = len; @@ -72,26 +70,8 @@ crypto_uio_data(crypto_data_t *data, uchar_t *buf, int len, cmd_type_t cmd, offset, length); datap = (uchar_t *)(zfs_uio_iovbase(uiop, vec_idx) + offset); - switch (cmd) { - case COPY_FROM_DATA: - bcopy(datap, buf, cur_len); - buf += cur_len; - break; - case COPY_TO_DATA: - bcopy(buf, datap, cur_len); - buf += cur_len; - break; - case COMPARE_TO_DATA: - if (bcmp(datap, buf, cur_len)) - return (CRYPTO_SIGNATURE_INVALID); - buf += cur_len; - break; - case MD5_DIGEST_DATA: - case SHA1_DIGEST_DATA: - case SHA2_DIGEST_DATA: - case GHASH_DATA: - return (CRYPTO_ARGUMENTS_BAD); - } + memcpy(datap, buf, cur_len); + buf += cur_len; length -= cur_len; vec_idx++; @@ -100,16 +80,11 @@ crypto_uio_data(crypto_data_t *data, uchar_t *buf, int len, cmd_type_t cmd, if (vec_idx == zfs_uio_iovcnt(uiop) && length > 0) { /* - * The end of the specified iovec's was reached but + * The end of the specified iovecs was reached but * the length requested could not be processed. */ - switch (cmd) { - case COPY_TO_DATA: - data->cd_length = len; - return (CRYPTO_BUFFER_TOO_SMALL); - default: - return (CRYPTO_DATA_LEN_RANGE); - } + data->cd_length = len; + return (CRYPTO_BUFFER_TOO_SMALL); } return (CRYPTO_SUCCESS); @@ -124,13 +99,12 @@ crypto_put_output_data(uchar_t *buf, crypto_data_t *output, int len) output->cd_length = len; return (CRYPTO_BUFFER_TOO_SMALL); } - bcopy(buf, (uchar_t *)(output->cd_raw.iov_base + - output->cd_offset), len); + memcpy((uchar_t *)(output->cd_raw.iov_base + + output->cd_offset), buf, len); break; case CRYPTO_DATA_UIO: - return (crypto_uio_data(output, buf, len, - COPY_TO_DATA, NULL, NULL)); + return (crypto_uio_copy_to_data(output, buf, len)); default: return (CRYPTO_ARGUMENTS_BAD); } @@ -140,33 +114,21 @@ crypto_put_output_data(uchar_t *buf, crypto_data_t *output, int len) int crypto_update_iov(void *ctx, crypto_data_t *input, crypto_data_t *output, - int (*cipher)(void *, caddr_t, size_t, crypto_data_t *), - void (*copy_block)(uint8_t *, uint64_t *)) + int (*cipher)(void *, caddr_t, size_t, crypto_data_t *)) { - common_ctx_t *common_ctx = ctx; - int rv; - ASSERT(input != output); - if (input->cd_miscdata != NULL) { - copy_block((uint8_t *)input->cd_miscdata, - &common_ctx->cc_iv[0]); - } if (input->cd_raw.iov_len < input->cd_length) return (CRYPTO_ARGUMENTS_BAD); - rv = (cipher)(ctx, input->cd_raw.iov_base + input->cd_offset, - input->cd_length, output); - - return (rv); + return ((cipher)(ctx, input->cd_raw.iov_base + input->cd_offset, + input->cd_length, output)); } int crypto_update_uio(void *ctx, crypto_data_t *input, crypto_data_t *output, - int (*cipher)(void *, caddr_t, size_t, crypto_data_t *), - void (*copy_block)(uint8_t *, uint64_t *)) + int (*cipher)(void *, caddr_t, size_t, crypto_data_t *)) { - common_ctx_t *common_ctx = ctx; zfs_uio_t *uiop = input->cd_uio; off_t offset = input->cd_offset; size_t length = input->cd_length; @@ -174,10 +136,6 @@ crypto_update_uio(void *ctx, crypto_data_t *input, crypto_data_t *output, size_t cur_len; ASSERT(input != output); - if (input->cd_miscdata != NULL) { - copy_block((uint8_t *)input->cd_miscdata, - &common_ctx->cc_iv[0]); - } if (zfs_uio_segflg(input->cd_uio) != UIO_SYSSPACE) { return (CRYPTO_ARGUMENTS_BAD); diff --git a/module/icp/core/kcf_prov_tabs.c b/module/icp/core/kcf_prov_tabs.c index 1f501f0ba864..865d4e19c6eb 100644 --- a/module/icp/core/kcf_prov_tabs.c +++ b/module/icp/core/kcf_prov_tabs.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -45,7 +45,7 @@ #include #include -#define KCF_MAX_PROVIDERS 512 /* max number of providers */ +#define KCF_MAX_PROVIDERS 8 /* max number of providers */ /* * Prov_tab is an array of providers which is updated when @@ -59,33 +59,25 @@ * * prov_tab entries are not updated from kcf.conf or by cryptoadm(1M). */ -static kcf_provider_desc_t **prov_tab = NULL; +static kcf_provider_desc_t *prov_tab[KCF_MAX_PROVIDERS]; static kmutex_t prov_tab_mutex; /* ensure exclusive access to the table */ static uint_t prov_tab_num = 0; /* number of providers in table */ -static uint_t prov_tab_max = KCF_MAX_PROVIDERS; void kcf_prov_tab_destroy(void) { mutex_destroy(&prov_tab_mutex); - - if (prov_tab) - kmem_free(prov_tab, prov_tab_max * - sizeof (kcf_provider_desc_t *)); } /* * Initialize a mutex and the KCF providers table, prov_tab. - * The providers table is dynamically allocated with prov_tab_max entries. + * The providers table is dynamically allocated with KCF_MAX_PROVIDERS entries. * Called from kcf module _init(). */ void kcf_prov_tab_init(void) { mutex_init(&prov_tab_mutex, NULL, MUTEX_DEFAULT, NULL); - - prov_tab = kmem_zalloc(prov_tab_max * sizeof (kcf_provider_desc_t *), - KM_SLEEP); } /* @@ -101,8 +93,6 @@ kcf_prov_tab_add_provider(kcf_provider_desc_t *prov_desc) { uint_t i; - ASSERT(prov_tab != NULL); - mutex_enter(&prov_tab_mutex); /* find free slot in providers table */ @@ -146,9 +136,6 @@ kcf_prov_tab_rem_provider(crypto_provider_id_t prov_id) { kcf_provider_desc_t *prov_desc; - ASSERT(prov_tab != NULL); - ASSERT(prov_tab_num >= 0); - /* * Validate provider id, since it can be specified by a 3rd-party * provider. @@ -204,92 +191,6 @@ kcf_prov_tab_lookup(crypto_provider_id_t prov_id) return (prov_desc); } -static void -allocate_ops_v1(const crypto_ops_t *src, crypto_ops_t *dst, - uint_t *mech_list_count) -{ - if (src->co_control_ops != NULL) - dst->co_control_ops = kmem_alloc(sizeof (crypto_control_ops_t), - KM_SLEEP); - - if (src->co_digest_ops != NULL) - dst->co_digest_ops = kmem_alloc(sizeof (crypto_digest_ops_t), - KM_SLEEP); - - if (src->co_cipher_ops != NULL) - dst->co_cipher_ops = kmem_alloc(sizeof (crypto_cipher_ops_t), - KM_SLEEP); - - if (src->co_mac_ops != NULL) - dst->co_mac_ops = kmem_alloc(sizeof (crypto_mac_ops_t), - KM_SLEEP); - - if (src->co_sign_ops != NULL) - dst->co_sign_ops = kmem_alloc(sizeof (crypto_sign_ops_t), - KM_SLEEP); - - if (src->co_verify_ops != NULL) - dst->co_verify_ops = kmem_alloc(sizeof (crypto_verify_ops_t), - KM_SLEEP); - - if (src->co_dual_ops != NULL) - dst->co_dual_ops = kmem_alloc(sizeof (crypto_dual_ops_t), - KM_SLEEP); - - if (src->co_dual_cipher_mac_ops != NULL) - dst->co_dual_cipher_mac_ops = kmem_alloc( - sizeof (crypto_dual_cipher_mac_ops_t), KM_SLEEP); - - if (src->co_random_ops != NULL) { - dst->co_random_ops = kmem_alloc( - sizeof (crypto_random_number_ops_t), KM_SLEEP); - - /* - * Allocate storage to store the array of supported mechanisms - * specified by provider. We allocate extra mechanism storage - * if the provider has random_ops since we keep an internal - * mechanism, SUN_RANDOM, in this case. - */ - (*mech_list_count)++; - } - - if (src->co_session_ops != NULL) - dst->co_session_ops = kmem_alloc(sizeof (crypto_session_ops_t), - KM_SLEEP); - - if (src->co_object_ops != NULL) - dst->co_object_ops = kmem_alloc(sizeof (crypto_object_ops_t), - KM_SLEEP); - - if (src->co_key_ops != NULL) - dst->co_key_ops = kmem_alloc(sizeof (crypto_key_ops_t), - KM_SLEEP); - - if (src->co_provider_ops != NULL) - dst->co_provider_ops = kmem_alloc( - sizeof (crypto_provider_management_ops_t), KM_SLEEP); - - if (src->co_ctx_ops != NULL) - dst->co_ctx_ops = kmem_alloc(sizeof (crypto_ctx_ops_t), - KM_SLEEP); -} - -static void -allocate_ops_v2(const crypto_ops_t *src, crypto_ops_t *dst) -{ - if (src->co_mech_ops != NULL) - dst->co_mech_ops = kmem_alloc(sizeof (crypto_mech_ops_t), - KM_SLEEP); -} - -static void -allocate_ops_v3(const crypto_ops_t *src, crypto_ops_t *dst) -{ - if (src->co_nostore_key_ops != NULL) - dst->co_nostore_key_ops = - kmem_alloc(sizeof (crypto_nostore_key_ops_t), KM_SLEEP); -} - /* * Allocate a provider descriptor. mech_list_count specifies the * number of mechanisms supported by the providers, and is used @@ -298,52 +199,11 @@ allocate_ops_v3(const crypto_ops_t *src, crypto_ops_t *dst) * since it is invoked from user context during provider registration. */ kcf_provider_desc_t * -kcf_alloc_provider_desc(const crypto_provider_info_t *info) +kcf_alloc_provider_desc(void) { - kcf_provider_desc_t *desc; - uint_t mech_list_count = info->pi_mech_list_count; - const crypto_ops_t *src_ops = info->pi_ops_vector; - - desc = kmem_zalloc(sizeof (kcf_provider_desc_t), KM_SLEEP); - - /* - * pd_description serves two purposes - * - Appears as a blank padded PKCS#11 style string, that will be - * returned to applications in CK_SLOT_INFO.slotDescription. - * This means that we should not have a null character in the - * first CRYPTO_PROVIDER_DESCR_MAX_LEN bytes. - * - Appears as a null-terminated string that can be used by - * other kcf routines. - * - * So, we allocate enough room for one extra null terminator - * which keeps every one happy. - */ - desc->pd_description = kmem_alloc(CRYPTO_PROVIDER_DESCR_MAX_LEN + 1, - KM_SLEEP); - (void) memset(desc->pd_description, ' ', - CRYPTO_PROVIDER_DESCR_MAX_LEN); - desc->pd_description[CRYPTO_PROVIDER_DESCR_MAX_LEN] = '\0'; - - /* - * Since the framework does not require the ops vector specified - * by the providers during registration to be persistent, - * KCF needs to allocate storage where copies of the ops - * vectors are copied. - */ - crypto_ops_t *opvec = kmem_zalloc(sizeof (crypto_ops_t), KM_SLEEP); - - if (info->pi_provider_type != CRYPTO_LOGICAL_PROVIDER) { - allocate_ops_v1(src_ops, opvec, &mech_list_count); - if (info->pi_interface_version >= CRYPTO_SPI_VERSION_2) - allocate_ops_v2(src_ops, opvec); - if (info->pi_interface_version == CRYPTO_SPI_VERSION_3) - allocate_ops_v3(src_ops, opvec); - } - desc->pd_ops_vector = opvec; + kcf_provider_desc_t *desc = + kmem_zalloc(sizeof (kcf_provider_desc_t), KM_SLEEP); - desc->pd_mech_list_count = mech_list_count; - desc->pd_mechanisms = kmem_zalloc(sizeof (crypto_mech_info_t) * - mech_list_count, KM_SLEEP); for (int i = 0; i < KCF_OPS_CLASSSIZE; i++) for (int j = 0; j < KCF_MAXMECHTAB; j++) desc->pd_mech_indx[i][j] = KCF_INVALID_INDX; @@ -352,7 +212,6 @@ kcf_alloc_provider_desc(const crypto_provider_info_t *info) desc->pd_state = KCF_PROV_ALLOCATED; mutex_init(&desc->pd_lock, NULL, MUTEX_DEFAULT, NULL); - cv_init(&desc->pd_resume_cv, NULL, CV_DEFAULT, NULL); cv_init(&desc->pd_remove_cv, NULL, CV_DEFAULT, NULL); return (desc); @@ -361,7 +220,7 @@ kcf_alloc_provider_desc(const crypto_provider_info_t *info) /* * Called by KCF_PROV_REFRELE when a provider's reference count drops * to zero. We free the descriptor when the last reference is released. - * However, for software providers, we do not free it when there is an + * However, for providers, we do not free it when there is an * unregister thread waiting. We signal that thread in this case and * that thread is responsible for freeing the descriptor. */ @@ -369,22 +228,16 @@ void kcf_provider_zero_refcnt(kcf_provider_desc_t *desc) { mutex_enter(&desc->pd_lock); - switch (desc->pd_prov_type) { - case CRYPTO_SW_PROVIDER: - if (desc->pd_state == KCF_PROV_REMOVED || - desc->pd_state == KCF_PROV_DISABLED) { - desc->pd_state = KCF_PROV_FREED; - cv_broadcast(&desc->pd_remove_cv); - mutex_exit(&desc->pd_lock); - break; - } - zfs_fallthrough; - - case CRYPTO_HW_PROVIDER: - case CRYPTO_LOGICAL_PROVIDER: + if (desc->pd_state == KCF_PROV_REMOVED || + desc->pd_state == KCF_PROV_DISABLED) { + desc->pd_state = KCF_PROV_FREED; + cv_broadcast(&desc->pd_remove_cv); mutex_exit(&desc->pd_lock); - kcf_free_provider_desc(desc); + return; } + + mutex_exit(&desc->pd_lock); + kcf_free_provider_desc(desc); } /* @@ -407,202 +260,15 @@ kcf_free_provider_desc(kcf_provider_desc_t *desc) /* free the kernel memory associated with the provider descriptor */ - if (desc->pd_description != NULL) - kmem_free(desc->pd_description, - CRYPTO_PROVIDER_DESCR_MAX_LEN + 1); - - if (desc->pd_ops_vector != NULL) { - - if (desc->pd_ops_vector->co_control_ops != NULL) - kmem_free(desc->pd_ops_vector->co_control_ops, - sizeof (crypto_control_ops_t)); - - if (desc->pd_ops_vector->co_digest_ops != NULL) - kmem_free(desc->pd_ops_vector->co_digest_ops, - sizeof (crypto_digest_ops_t)); - - if (desc->pd_ops_vector->co_cipher_ops != NULL) - kmem_free(desc->pd_ops_vector->co_cipher_ops, - sizeof (crypto_cipher_ops_t)); - - if (desc->pd_ops_vector->co_mac_ops != NULL) - kmem_free(desc->pd_ops_vector->co_mac_ops, - sizeof (crypto_mac_ops_t)); - - if (desc->pd_ops_vector->co_sign_ops != NULL) - kmem_free(desc->pd_ops_vector->co_sign_ops, - sizeof (crypto_sign_ops_t)); - - if (desc->pd_ops_vector->co_verify_ops != NULL) - kmem_free(desc->pd_ops_vector->co_verify_ops, - sizeof (crypto_verify_ops_t)); - - if (desc->pd_ops_vector->co_dual_ops != NULL) - kmem_free(desc->pd_ops_vector->co_dual_ops, - sizeof (crypto_dual_ops_t)); - - if (desc->pd_ops_vector->co_dual_cipher_mac_ops != NULL) - kmem_free(desc->pd_ops_vector->co_dual_cipher_mac_ops, - sizeof (crypto_dual_cipher_mac_ops_t)); - - if (desc->pd_ops_vector->co_random_ops != NULL) - kmem_free(desc->pd_ops_vector->co_random_ops, - sizeof (crypto_random_number_ops_t)); - - if (desc->pd_ops_vector->co_session_ops != NULL) - kmem_free(desc->pd_ops_vector->co_session_ops, - sizeof (crypto_session_ops_t)); - - if (desc->pd_ops_vector->co_object_ops != NULL) - kmem_free(desc->pd_ops_vector->co_object_ops, - sizeof (crypto_object_ops_t)); - - if (desc->pd_ops_vector->co_key_ops != NULL) - kmem_free(desc->pd_ops_vector->co_key_ops, - sizeof (crypto_key_ops_t)); - - if (desc->pd_ops_vector->co_provider_ops != NULL) - kmem_free(desc->pd_ops_vector->co_provider_ops, - sizeof (crypto_provider_management_ops_t)); - - if (desc->pd_ops_vector->co_ctx_ops != NULL) - kmem_free(desc->pd_ops_vector->co_ctx_ops, - sizeof (crypto_ctx_ops_t)); - - if (desc->pd_ops_vector->co_mech_ops != NULL) - kmem_free(desc->pd_ops_vector->co_mech_ops, - sizeof (crypto_mech_ops_t)); - - if (desc->pd_ops_vector->co_nostore_key_ops != NULL) - kmem_free(desc->pd_ops_vector->co_nostore_key_ops, - sizeof (crypto_nostore_key_ops_t)); - - kmem_free(desc->pd_ops_vector, sizeof (crypto_ops_t)); - } - - if (desc->pd_mechanisms != NULL) - /* free the memory associated with the mechanism info's */ - kmem_free(desc->pd_mechanisms, sizeof (crypto_mech_info_t) * - desc->pd_mech_list_count); - - if (desc->pd_sched_info.ks_taskq != NULL) - taskq_destroy(desc->pd_sched_info.ks_taskq); - mutex_destroy(&desc->pd_lock); - cv_destroy(&desc->pd_resume_cv); cv_destroy(&desc->pd_remove_cv); kmem_free(desc, sizeof (kcf_provider_desc_t)); } -/* - * Returns an array of hardware and logical provider descriptors, - * a.k.a the PKCS#11 slot list. A REFHOLD is done on each descriptor - * before the array is returned. The entire table can be freed by - * calling kcf_free_provider_tab(). - */ -int -kcf_get_slot_list(uint_t *count, kcf_provider_desc_t ***array, - boolean_t unverified) -{ - kcf_provider_desc_t *prov_desc; - kcf_provider_desc_t **p = NULL; - char *last; - uint_t cnt = 0; - uint_t i, j; - int rval = CRYPTO_SUCCESS; - size_t n, final_size; - - /* count the providers */ - mutex_enter(&prov_tab_mutex); - for (i = 0; i < KCF_MAX_PROVIDERS; i++) { - if ((prov_desc = prov_tab[i]) != NULL && - ((prov_desc->pd_prov_type == CRYPTO_HW_PROVIDER && - (prov_desc->pd_flags & CRYPTO_HIDE_PROVIDER) == 0) || - prov_desc->pd_prov_type == CRYPTO_LOGICAL_PROVIDER)) { - if (KCF_IS_PROV_USABLE(prov_desc) || - (unverified && KCF_IS_PROV_UNVERIFIED(prov_desc))) { - cnt++; - } - } - } - mutex_exit(&prov_tab_mutex); - - if (cnt == 0) - goto out; - - n = cnt * sizeof (kcf_provider_desc_t *); -again: - p = kmem_zalloc(n, KM_SLEEP); - - /* pointer to last entry in the array */ - last = (char *)&p[cnt-1]; - - mutex_enter(&prov_tab_mutex); - /* fill the slot list */ - for (i = 0, j = 0; i < KCF_MAX_PROVIDERS; i++) { - if ((prov_desc = prov_tab[i]) != NULL && - ((prov_desc->pd_prov_type == CRYPTO_HW_PROVIDER && - (prov_desc->pd_flags & CRYPTO_HIDE_PROVIDER) == 0) || - prov_desc->pd_prov_type == CRYPTO_LOGICAL_PROVIDER)) { - if (KCF_IS_PROV_USABLE(prov_desc) || - (unverified && KCF_IS_PROV_UNVERIFIED(prov_desc))) { - if ((char *)&p[j] > last) { - mutex_exit(&prov_tab_mutex); - kcf_free_provider_tab(cnt, p); - n = n << 1; - cnt = cnt << 1; - goto again; - } - p[j++] = prov_desc; - KCF_PROV_REFHOLD(prov_desc); - } - } - } - mutex_exit(&prov_tab_mutex); - - final_size = j * sizeof (kcf_provider_desc_t *); - cnt = j; - ASSERT(final_size <= n); - - /* check if buffer we allocated is too large */ - if (final_size < n) { - char *final_buffer = NULL; - - if (final_size > 0) { - final_buffer = kmem_alloc(final_size, KM_SLEEP); - bcopy(p, final_buffer, final_size); - } - kmem_free(p, n); - p = (kcf_provider_desc_t **)final_buffer; - } -out: - *count = cnt; - *array = p; - return (rval); -} - -/* - * Free an array of hardware provider descriptors. A REFRELE - * is done on each descriptor before the table is freed. - */ -void -kcf_free_provider_tab(uint_t count, kcf_provider_desc_t **array) -{ - kcf_provider_desc_t *prov_desc; - int i; - - for (i = 0; i < count; i++) { - if ((prov_desc = array[i]) != NULL) { - KCF_PROV_REFRELE(prov_desc); - } - } - kmem_free(array, count * sizeof (kcf_provider_desc_t *)); -} - /* * Returns in the location pointed to by pd a pointer to the descriptor - * for the software provider for the specified mechanism. + * for the provider for the specified mechanism. * The provider descriptor is returned held and it is the caller's * responsibility to release it when done. The mechanism entry * is returned if the optional argument mep is non NULL. @@ -620,24 +286,17 @@ kcf_get_sw_prov(crypto_mech_type_t mech_type, kcf_provider_desc_t **pd, if (kcf_get_mech_entry(mech_type, &me) != KCF_SUCCESS) return (CRYPTO_MECHANISM_INVALID); - /* - * Get the software provider for this mechanism. - * Lock the mech_entry until we grab the 'pd'. - */ - mutex_enter(&me->me_mutex); - + /* Get the provider for this mechanism. */ if (me->me_sw_prov == NULL || (*pd = me->me_sw_prov->pm_prov_desc) == NULL) { - /* no SW provider for this mechanism */ + /* no provider for this mechanism */ if (log_warn) - cmn_err(CE_WARN, "no SW provider for \"%s\"\n", + cmn_err(CE_WARN, "no provider for \"%s\"\n", me->me_name); - mutex_exit(&me->me_mutex); return (CRYPTO_MECH_NOT_SUPPORTED); } KCF_PROV_REFHOLD(*pd); - mutex_exit(&me->me_mutex); if (mep != NULL) *mep = me; diff --git a/module/icp/core/kcf_sched.c b/module/icp/core/kcf_sched.c index ee0fe0ac635d..360ecfb2be19 100644 --- a/module/icp/core/kcf_sched.c +++ b/module/icp/core/kcf_sched.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -35,761 +35,35 @@ #include #include -static kcf_global_swq_t *gswq; /* Global software queue */ - -/* Thread pool related variables */ -static kcf_pool_t *kcfpool; /* Thread pool of kcfd LWPs */ -static const int kcf_maxthreads = 2; -static const int kcf_minthreads = 1; - /* kmem caches used by the scheduler */ -static kmem_cache_t *kcf_sreq_cache; -static kmem_cache_t *kcf_areq_cache; static kmem_cache_t *kcf_context_cache; -/* Global request ID table */ -static kcf_reqid_table_t *kcf_reqid_table[REQID_TABLES]; - -/* KCF stats. Not protected. */ -static kcf_stats_t kcf_ksdata = { - { "total threads in pool", KSTAT_DATA_UINT32}, - { "idle threads in pool", KSTAT_DATA_UINT32}, - { "min threads in pool", KSTAT_DATA_UINT32}, - { "max threads in pool", KSTAT_DATA_UINT32}, - { "requests in gswq", KSTAT_DATA_UINT32}, - { "max requests in gswq", KSTAT_DATA_UINT32}, - { "threads for HW taskq", KSTAT_DATA_UINT32}, - { "minalloc for HW taskq", KSTAT_DATA_UINT32}, - { "maxalloc for HW taskq", KSTAT_DATA_UINT32} -}; - -static kstat_t *kcf_misc_kstat = NULL; -ulong_t kcf_swprov_hndl = 0; - -static kcf_areq_node_t *kcf_areqnode_alloc(kcf_provider_desc_t *, - kcf_context_t *, crypto_call_req_t *, kcf_req_params_t *, boolean_t); -static int kcf_disp_sw_request(kcf_areq_node_t *); -static void process_req_hwp(void *); -static int kcf_enqueue(kcf_areq_node_t *); -static void kcfpool_alloc(void); -static void kcf_reqid_delete(kcf_areq_node_t *areq); -static crypto_req_id_t kcf_reqid_insert(kcf_areq_node_t *areq); -static int kcf_misc_kstat_update(kstat_t *ksp, int rw); - /* * Create a new context. */ crypto_ctx_t * -kcf_new_ctx(crypto_call_req_t *crq, kcf_provider_desc_t *pd, - crypto_session_id_t sid) +kcf_new_ctx(kcf_provider_desc_t *pd) { crypto_ctx_t *ctx; kcf_context_t *kcf_ctx; - kcf_ctx = kmem_cache_alloc(kcf_context_cache, - (crq == NULL) ? KM_SLEEP : KM_NOSLEEP); + kcf_ctx = kmem_cache_alloc(kcf_context_cache, KM_SLEEP); if (kcf_ctx == NULL) return (NULL); /* initialize the context for the consumer */ kcf_ctx->kc_refcnt = 1; - kcf_ctx->kc_req_chain_first = NULL; - kcf_ctx->kc_req_chain_last = NULL; - kcf_ctx->kc_secondctx = NULL; KCF_PROV_REFHOLD(pd); kcf_ctx->kc_prov_desc = pd; kcf_ctx->kc_sw_prov_desc = NULL; - kcf_ctx->kc_mech = NULL; ctx = &kcf_ctx->kc_glbl_ctx; - ctx->cc_provider = pd->pd_prov_handle; - ctx->cc_session = sid; ctx->cc_provider_private = NULL; ctx->cc_framework_private = (void *)kcf_ctx; - ctx->cc_flags = 0; - ctx->cc_opstate = NULL; return (ctx); } -/* - * Allocate a new async request node. - * - * ictx - Framework private context pointer - * crq - Has callback function and argument. Should be non NULL. - * req - The parameters to pass to the SPI - */ -static kcf_areq_node_t * -kcf_areqnode_alloc(kcf_provider_desc_t *pd, kcf_context_t *ictx, - crypto_call_req_t *crq, kcf_req_params_t *req, boolean_t isdual) -{ - kcf_areq_node_t *arptr, *areq; - - ASSERT(crq != NULL); - arptr = kmem_cache_alloc(kcf_areq_cache, KM_NOSLEEP); - if (arptr == NULL) - return (NULL); - - arptr->an_state = REQ_ALLOCATED; - arptr->an_reqarg = *crq; - arptr->an_params = *req; - arptr->an_context = ictx; - arptr->an_isdual = isdual; - - arptr->an_next = arptr->an_prev = NULL; - KCF_PROV_REFHOLD(pd); - arptr->an_provider = pd; - arptr->an_tried_plist = NULL; - arptr->an_refcnt = 1; - arptr->an_idnext = arptr->an_idprev = NULL; - - /* - * Requests for context-less operations do not use the - * fields - an_is_my_turn, and an_ctxchain_next. - */ - if (ictx == NULL) - return (arptr); - - KCF_CONTEXT_REFHOLD(ictx); - /* - * Chain this request to the context. - */ - mutex_enter(&ictx->kc_in_use_lock); - arptr->an_ctxchain_next = NULL; - if ((areq = ictx->kc_req_chain_last) == NULL) { - arptr->an_is_my_turn = B_TRUE; - ictx->kc_req_chain_last = - ictx->kc_req_chain_first = arptr; - } else { - ASSERT(ictx->kc_req_chain_first != NULL); - arptr->an_is_my_turn = B_FALSE; - /* Insert the new request to the end of the chain. */ - areq->an_ctxchain_next = arptr; - ictx->kc_req_chain_last = arptr; - } - mutex_exit(&ictx->kc_in_use_lock); - - return (arptr); -} - -/* - * Queue the request node and do one of the following: - * - If there is an idle thread signal it to run. - * - If there is no idle thread and max running threads is not - * reached, signal the creator thread for more threads. - * - * If the two conditions above are not met, we don't need to do - * anything. The request will be picked up by one of the - * worker threads when it becomes available. - */ -static int -kcf_disp_sw_request(kcf_areq_node_t *areq) -{ - int err; - int cnt = 0; - - if ((err = kcf_enqueue(areq)) != 0) - return (err); - - if (kcfpool->kp_idlethreads > 0) { - /* Signal an idle thread to run */ - mutex_enter(&gswq->gs_lock); - cv_signal(&gswq->gs_cv); - mutex_exit(&gswq->gs_lock); - - return (CRYPTO_QUEUED); - } - - /* - * We keep the number of running threads to be at - * kcf_minthreads to reduce gs_lock contention. - */ - cnt = kcf_minthreads - - (kcfpool->kp_threads - kcfpool->kp_blockedthreads); - if (cnt > 0) { - /* - * The following ensures the number of threads in pool - * does not exceed kcf_maxthreads. - */ - cnt = MIN(cnt, kcf_maxthreads - (int)kcfpool->kp_threads); - if (cnt > 0) { - /* Signal the creator thread for more threads */ - mutex_enter(&kcfpool->kp_user_lock); - if (!kcfpool->kp_signal_create_thread) { - kcfpool->kp_signal_create_thread = B_TRUE; - kcfpool->kp_nthrs = cnt; - cv_signal(&kcfpool->kp_user_cv); - } - mutex_exit(&kcfpool->kp_user_lock); - } - } - - return (CRYPTO_QUEUED); -} - -/* - * This routine is called by the taskq associated with - * each hardware provider. We notify the kernel consumer - * via the callback routine in case of CRYPTO_SUCCESS or - * a failure. - * - * A request can be of type kcf_areq_node_t or of type - * kcf_sreq_node_t. - */ -static void -process_req_hwp(void *ireq) -{ - int error = 0; - crypto_ctx_t *ctx; - kcf_call_type_t ctype; - kcf_provider_desc_t *pd; - kcf_areq_node_t *areq = (kcf_areq_node_t *)ireq; - kcf_sreq_node_t *sreq = (kcf_sreq_node_t *)ireq; - - pd = ((ctype = GET_REQ_TYPE(ireq)) == CRYPTO_SYNCH) ? - sreq->sn_provider : areq->an_provider; - - /* - * Wait if flow control is in effect for the provider. A - * CRYPTO_PROVIDER_READY or CRYPTO_PROVIDER_FAILED - * notification will signal us. We also get signaled if - * the provider is unregistering. - */ - if (pd->pd_state == KCF_PROV_BUSY) { - mutex_enter(&pd->pd_lock); - while (pd->pd_state == KCF_PROV_BUSY) - cv_wait(&pd->pd_resume_cv, &pd->pd_lock); - mutex_exit(&pd->pd_lock); - } - - /* - * Bump the internal reference count while the request is being - * processed. This is how we know when it's safe to unregister - * a provider. This step must precede the pd_state check below. - */ - KCF_PROV_IREFHOLD(pd); - - /* - * Fail the request if the provider has failed. We return a - * recoverable error and the notified clients attempt any - * recovery. For async clients this is done in kcf_aop_done() - * and for sync clients it is done in the k-api routines. - */ - if (pd->pd_state >= KCF_PROV_FAILED) { - error = CRYPTO_DEVICE_ERROR; - goto bail; - } - - if (ctype == CRYPTO_SYNCH) { - mutex_enter(&sreq->sn_lock); - sreq->sn_state = REQ_INPROGRESS; - mutex_exit(&sreq->sn_lock); - - ctx = sreq->sn_context ? &sreq->sn_context->kc_glbl_ctx : NULL; - error = common_submit_request(sreq->sn_provider, ctx, - sreq->sn_params, sreq); - } else { - kcf_context_t *ictx; - ASSERT(ctype == CRYPTO_ASYNCH); - - /* - * We are in the per-hardware provider thread context and - * hence can sleep. Note that the caller would have done - * a taskq_dispatch(..., TQ_NOSLEEP) and would have returned. - */ - ctx = (ictx = areq->an_context) ? &ictx->kc_glbl_ctx : NULL; - - mutex_enter(&areq->an_lock); - /* - * We need to maintain ordering for multi-part requests. - * an_is_my_turn is set to B_TRUE initially for a request - * when it is enqueued and there are no other requests - * for that context. It is set later from kcf_aop_done() when - * the request before us in the chain of requests for the - * context completes. We get signaled at that point. - */ - if (ictx != NULL) { - ASSERT(ictx->kc_prov_desc == areq->an_provider); - - while (areq->an_is_my_turn == B_FALSE) { - cv_wait(&areq->an_turn_cv, &areq->an_lock); - } - } - areq->an_state = REQ_INPROGRESS; - mutex_exit(&areq->an_lock); - - error = common_submit_request(areq->an_provider, ctx, - &areq->an_params, areq); - } - -bail: - if (error == CRYPTO_QUEUED) { - /* - * The request is queued by the provider and we should - * get a crypto_op_notification() from the provider later. - * We notify the consumer at that time. - */ - return; - } else { /* CRYPTO_SUCCESS or other failure */ - KCF_PROV_IREFRELE(pd); - if (ctype == CRYPTO_SYNCH) - kcf_sop_done(sreq, error); - else - kcf_aop_done(areq, error); - } -} - -/* - * This routine checks if a request can be retried on another - * provider. If true, mech1 is initialized to point to the mechanism - * structure. mech2 is also initialized in case of a dual operation. fg - * is initialized to the correct crypto_func_group_t bit flag. They are - * initialized by this routine, so that the caller can pass them to a - * kcf_get_mech_provider() or kcf_get_dual_provider() with no further change. - * - * We check that the request is for a init or atomic routine and that - * it is for one of the operation groups used from k-api . - */ -static boolean_t -can_resubmit(kcf_areq_node_t *areq, crypto_mechanism_t **mech1, - crypto_mechanism_t **mech2, crypto_func_group_t *fg) -{ - kcf_req_params_t *params; - kcf_op_type_t optype; - - params = &areq->an_params; - optype = params->rp_optype; - - if (!(IS_INIT_OP(optype) || IS_ATOMIC_OP(optype))) - return (B_FALSE); - - switch (params->rp_opgrp) { - case KCF_OG_DIGEST: { - kcf_digest_ops_params_t *dops = ¶ms->rp_u.digest_params; - - dops->do_mech.cm_type = dops->do_framework_mechtype; - *mech1 = &dops->do_mech; - *fg = (optype == KCF_OP_INIT) ? CRYPTO_FG_DIGEST : - CRYPTO_FG_DIGEST_ATOMIC; - break; - } - - case KCF_OG_MAC: { - kcf_mac_ops_params_t *mops = ¶ms->rp_u.mac_params; - - mops->mo_mech.cm_type = mops->mo_framework_mechtype; - *mech1 = &mops->mo_mech; - *fg = (optype == KCF_OP_INIT) ? CRYPTO_FG_MAC : - CRYPTO_FG_MAC_ATOMIC; - break; - } - - case KCF_OG_SIGN: { - kcf_sign_ops_params_t *sops = ¶ms->rp_u.sign_params; - - sops->so_mech.cm_type = sops->so_framework_mechtype; - *mech1 = &sops->so_mech; - switch (optype) { - case KCF_OP_INIT: - *fg = CRYPTO_FG_SIGN; - break; - case KCF_OP_ATOMIC: - *fg = CRYPTO_FG_SIGN_ATOMIC; - break; - default: - ASSERT(optype == KCF_OP_SIGN_RECOVER_ATOMIC); - *fg = CRYPTO_FG_SIGN_RECOVER_ATOMIC; - } - break; - } - - case KCF_OG_VERIFY: { - kcf_verify_ops_params_t *vops = ¶ms->rp_u.verify_params; - - vops->vo_mech.cm_type = vops->vo_framework_mechtype; - *mech1 = &vops->vo_mech; - switch (optype) { - case KCF_OP_INIT: - *fg = CRYPTO_FG_VERIFY; - break; - case KCF_OP_ATOMIC: - *fg = CRYPTO_FG_VERIFY_ATOMIC; - break; - default: - ASSERT(optype == KCF_OP_VERIFY_RECOVER_ATOMIC); - *fg = CRYPTO_FG_VERIFY_RECOVER_ATOMIC; - } - break; - } - - case KCF_OG_ENCRYPT: { - kcf_encrypt_ops_params_t *eops = ¶ms->rp_u.encrypt_params; - - eops->eo_mech.cm_type = eops->eo_framework_mechtype; - *mech1 = &eops->eo_mech; - *fg = (optype == KCF_OP_INIT) ? CRYPTO_FG_ENCRYPT : - CRYPTO_FG_ENCRYPT_ATOMIC; - break; - } - - case KCF_OG_DECRYPT: { - kcf_decrypt_ops_params_t *dcrops = ¶ms->rp_u.decrypt_params; - - dcrops->dop_mech.cm_type = dcrops->dop_framework_mechtype; - *mech1 = &dcrops->dop_mech; - *fg = (optype == KCF_OP_INIT) ? CRYPTO_FG_DECRYPT : - CRYPTO_FG_DECRYPT_ATOMIC; - break; - } - - case KCF_OG_ENCRYPT_MAC: { - kcf_encrypt_mac_ops_params_t *eops = - ¶ms->rp_u.encrypt_mac_params; - - eops->em_encr_mech.cm_type = eops->em_framework_encr_mechtype; - *mech1 = &eops->em_encr_mech; - eops->em_mac_mech.cm_type = eops->em_framework_mac_mechtype; - *mech2 = &eops->em_mac_mech; - *fg = (optype == KCF_OP_INIT) ? CRYPTO_FG_ENCRYPT_MAC : - CRYPTO_FG_ENCRYPT_MAC_ATOMIC; - break; - } - - case KCF_OG_MAC_DECRYPT: { - kcf_mac_decrypt_ops_params_t *dops = - ¶ms->rp_u.mac_decrypt_params; - - dops->md_mac_mech.cm_type = dops->md_framework_mac_mechtype; - *mech1 = &dops->md_mac_mech; - dops->md_decr_mech.cm_type = dops->md_framework_decr_mechtype; - *mech2 = &dops->md_decr_mech; - *fg = (optype == KCF_OP_INIT) ? CRYPTO_FG_MAC_DECRYPT : - CRYPTO_FG_MAC_DECRYPT_ATOMIC; - break; - } - - default: - return (B_FALSE); - } - - return (B_TRUE); -} - -/* - * This routine is called when a request to a provider has failed - * with a recoverable error. This routine tries to find another provider - * and dispatches the request to the new provider, if one is available. - * We reuse the request structure. - * - * A return value of NULL from kcf_get_mech_provider() indicates - * we have tried the last provider. - */ -static int -kcf_resubmit_request(kcf_areq_node_t *areq) -{ - int error = CRYPTO_FAILED; - kcf_context_t *ictx; - kcf_provider_desc_t *old_pd; - kcf_provider_desc_t *new_pd; - crypto_mechanism_t *mech1 = NULL, *mech2 = NULL; - crypto_mech_type_t prov_mt1, prov_mt2; - crypto_func_group_t fg = 0; - - if (!can_resubmit(areq, &mech1, &mech2, &fg)) - return (error); - - old_pd = areq->an_provider; - /* - * Add old_pd to the list of providers already tried. We release - * the hold on old_pd (from the earlier kcf_get_mech_provider()) in - * kcf_free_triedlist(). - */ - if (kcf_insert_triedlist(&areq->an_tried_plist, old_pd, - KM_NOSLEEP) == NULL) - return (error); - - if (mech1 && !mech2) { - new_pd = kcf_get_mech_provider(mech1->cm_type, NULL, &error, - areq->an_tried_plist, fg, - (areq->an_reqarg.cr_flag & CRYPTO_RESTRICTED), 0); - } else { - ASSERT(mech1 != NULL && mech2 != NULL); - - new_pd = kcf_get_dual_provider(mech1, mech2, NULL, &prov_mt1, - &prov_mt2, &error, areq->an_tried_plist, fg, fg, - (areq->an_reqarg.cr_flag & CRYPTO_RESTRICTED), 0); - } - - if (new_pd == NULL) - return (error); - - /* - * We reuse the old context by resetting provider specific - * fields in it. - */ - if ((ictx = areq->an_context) != NULL) { - crypto_ctx_t *ctx; - - ASSERT(old_pd == ictx->kc_prov_desc); - KCF_PROV_REFRELE(ictx->kc_prov_desc); - KCF_PROV_REFHOLD(new_pd); - ictx->kc_prov_desc = new_pd; - - ctx = &ictx->kc_glbl_ctx; - ctx->cc_provider = new_pd->pd_prov_handle; - ctx->cc_session = new_pd->pd_sid; - ctx->cc_provider_private = NULL; - } - - /* We reuse areq. by resetting the provider and context fields. */ - KCF_PROV_REFRELE(old_pd); - KCF_PROV_REFHOLD(new_pd); - areq->an_provider = new_pd; - mutex_enter(&areq->an_lock); - areq->an_state = REQ_WAITING; - mutex_exit(&areq->an_lock); - - switch (new_pd->pd_prov_type) { - case CRYPTO_SW_PROVIDER: - error = kcf_disp_sw_request(areq); - break; - - case CRYPTO_HW_PROVIDER: { - taskq_t *taskq = new_pd->pd_sched_info.ks_taskq; - - if (taskq_dispatch(taskq, process_req_hwp, areq, TQ_NOSLEEP) == - TASKQID_INVALID) { - error = CRYPTO_HOST_MEMORY; - } else { - error = CRYPTO_QUEUED; - } - - break; - default: - break; - } - } - - return (error); -} - -/* - * Routine called by both ioctl and k-api. The consumer should - * bundle the parameters into a kcf_req_params_t structure. A bunch - * of macros are available in ops_impl.h for this bundling. They are: - * - * KCF_WRAP_DIGEST_OPS_PARAMS() - * KCF_WRAP_MAC_OPS_PARAMS() - * KCF_WRAP_ENCRYPT_OPS_PARAMS() - * KCF_WRAP_DECRYPT_OPS_PARAMS() ... etc. - * - * It is the caller's responsibility to free the ctx argument when - * appropriate. See the KCF_CONTEXT_COND_RELEASE macro for details. - */ -int -kcf_submit_request(kcf_provider_desc_t *pd, crypto_ctx_t *ctx, - crypto_call_req_t *crq, kcf_req_params_t *params, boolean_t cont) -{ - int error = CRYPTO_SUCCESS; - kcf_areq_node_t *areq; - kcf_sreq_node_t *sreq; - kcf_context_t *kcf_ctx; - taskq_t *taskq = pd->pd_sched_info.ks_taskq; - - kcf_ctx = ctx ? (kcf_context_t *)ctx->cc_framework_private : NULL; - - /* Synchronous cases */ - if (crq == NULL) { - switch (pd->pd_prov_type) { - case CRYPTO_SW_PROVIDER: - error = common_submit_request(pd, ctx, params, - KCF_RHNDL(KM_SLEEP)); - break; - - case CRYPTO_HW_PROVIDER: - /* - * Special case for CRYPTO_SYNCHRONOUS providers that - * never return a CRYPTO_QUEUED error. We skip any - * request allocation and call the SPI directly. - */ - if ((pd->pd_flags & CRYPTO_SYNCHRONOUS) && - taskq_empty(taskq)) { - KCF_PROV_IREFHOLD(pd); - if (pd->pd_state == KCF_PROV_READY) { - error = common_submit_request(pd, ctx, - params, KCF_RHNDL(KM_SLEEP)); - KCF_PROV_IREFRELE(pd); - ASSERT(error != CRYPTO_QUEUED); - break; - } - KCF_PROV_IREFRELE(pd); - } - - sreq = kmem_cache_alloc(kcf_sreq_cache, KM_SLEEP); - sreq->sn_state = REQ_ALLOCATED; - sreq->sn_rv = CRYPTO_FAILED; - sreq->sn_params = params; - - /* - * Note that we do not need to hold the context - * for synchronous case as the context will never - * become invalid underneath us. We do not need to hold - * the provider here either as the caller has a hold. - */ - sreq->sn_context = kcf_ctx; - ASSERT(KCF_PROV_REFHELD(pd)); - sreq->sn_provider = pd; - - ASSERT(taskq != NULL); - /* - * Call the SPI directly if the taskq is empty and the - * provider is not busy, else dispatch to the taskq. - * Calling directly is fine as this is the synchronous - * case. This is unlike the asynchronous case where we - * must always dispatch to the taskq. - */ - if (taskq_empty(taskq) && - pd->pd_state == KCF_PROV_READY) { - process_req_hwp(sreq); - } else { - /* - * We can not tell from taskq_dispatch() return - * value if we exceeded maxalloc. Hence the - * check here. Since we are allowed to wait in - * the synchronous case, we wait for the taskq - * to become empty. - */ - if (taskq->tq_nalloc >= crypto_taskq_maxalloc) { - taskq_wait(taskq); - } - - (void) taskq_dispatch(taskq, process_req_hwp, - sreq, TQ_SLEEP); - } - - /* - * Wait for the notification to arrive, - * if the operation is not done yet. - * Bug# 4722589 will make the wait a cv_wait_sig(). - */ - mutex_enter(&sreq->sn_lock); - while (sreq->sn_state < REQ_DONE) - cv_wait(&sreq->sn_cv, &sreq->sn_lock); - mutex_exit(&sreq->sn_lock); - - error = sreq->sn_rv; - kmem_cache_free(kcf_sreq_cache, sreq); - - break; - - default: - error = CRYPTO_FAILED; - break; - } - - } else { /* Asynchronous cases */ - switch (pd->pd_prov_type) { - case CRYPTO_SW_PROVIDER: - if (!(crq->cr_flag & CRYPTO_ALWAYS_QUEUE)) { - /* - * This case has less overhead since there is - * no switching of context. - */ - error = common_submit_request(pd, ctx, params, - KCF_RHNDL(KM_NOSLEEP)); - } else { - /* - * CRYPTO_ALWAYS_QUEUE is set. We need to - * queue the request and return. - */ - areq = kcf_areqnode_alloc(pd, kcf_ctx, crq, - params, cont); - if (areq == NULL) - error = CRYPTO_HOST_MEMORY; - else { - if (!(crq->cr_flag - & CRYPTO_SKIP_REQID)) { - /* - * Set the request handle. This handle - * is used for any crypto_cancel_req(9f) - * calls from the consumer. We have to - * do this before dispatching the - * request. - */ - crq->cr_reqid = kcf_reqid_insert(areq); - } - - error = kcf_disp_sw_request(areq); - /* - * There is an error processing this - * request. Remove the handle and - * release the request structure. - */ - if (error != CRYPTO_QUEUED) { - if (!(crq->cr_flag - & CRYPTO_SKIP_REQID)) - kcf_reqid_delete(areq); - KCF_AREQ_REFRELE(areq); - } - } - } - break; - - case CRYPTO_HW_PROVIDER: - /* - * We need to queue the request and return. - */ - areq = kcf_areqnode_alloc(pd, kcf_ctx, crq, params, - cont); - if (areq == NULL) { - error = CRYPTO_HOST_MEMORY; - goto done; - } - - ASSERT(taskq != NULL); - /* - * We can not tell from taskq_dispatch() return - * value if we exceeded maxalloc. Hence the check - * here. - */ - if (taskq->tq_nalloc >= crypto_taskq_maxalloc) { - error = CRYPTO_BUSY; - KCF_AREQ_REFRELE(areq); - goto done; - } - - if (!(crq->cr_flag & CRYPTO_SKIP_REQID)) { - /* - * Set the request handle. This handle is used - * for any crypto_cancel_req(9f) calls from the - * consumer. We have to do this before dispatching - * the request. - */ - crq->cr_reqid = kcf_reqid_insert(areq); - } - - if (taskq_dispatch(taskq, - process_req_hwp, areq, TQ_NOSLEEP) == - TASKQID_INVALID) { - error = CRYPTO_HOST_MEMORY; - if (!(crq->cr_flag & CRYPTO_SKIP_REQID)) - kcf_reqid_delete(areq); - KCF_AREQ_REFRELE(areq); - } else { - error = CRYPTO_QUEUED; - } - break; - - default: - error = CRYPTO_FAILED; - break; - } - } - -done: - return (error); -} - /* * We're done with this framework context, so free it. Note that freeing * framework context (kcf_context) frees the global context (crypto_ctx). @@ -805,12 +79,6 @@ kcf_free_context(kcf_context_t *kcf_ctx) { kcf_provider_desc_t *pd = kcf_ctx->kc_prov_desc; crypto_ctx_t *gctx = &kcf_ctx->kc_glbl_ctx; - kcf_context_t *kcf_secondctx = kcf_ctx->kc_secondctx; - - /* Release the second context, if any */ - - if (kcf_secondctx != NULL) - KCF_CONTEXT_REFRELE(kcf_secondctx); if (gctx->cc_provider_private != NULL) { mutex_enter(&pd->pd_lock); @@ -832,185 +100,9 @@ kcf_free_context(kcf_context_t *kcf_ctx) /* kcf_ctx->kc_prov_desc has a hold on pd */ KCF_PROV_REFRELE(kcf_ctx->kc_prov_desc); - /* check if this context is shared with a software provider */ - if ((gctx->cc_flags & CRYPTO_INIT_OPSTATE) && - kcf_ctx->kc_sw_prov_desc != NULL) { - KCF_PROV_REFRELE(kcf_ctx->kc_sw_prov_desc); - } - kmem_cache_free(kcf_context_cache, kcf_ctx); } -/* - * Free the request after releasing all the holds. - */ -void -kcf_free_req(kcf_areq_node_t *areq) -{ - KCF_PROV_REFRELE(areq->an_provider); - if (areq->an_context != NULL) - KCF_CONTEXT_REFRELE(areq->an_context); - - if (areq->an_tried_plist != NULL) - kcf_free_triedlist(areq->an_tried_plist); - kmem_cache_free(kcf_areq_cache, areq); -} - -/* - * Utility routine to remove a request from the chain of requests - * hanging off a context. - */ -static void -kcf_removereq_in_ctxchain(kcf_context_t *ictx, kcf_areq_node_t *areq) -{ - kcf_areq_node_t *cur, *prev; - - /* - * Get context lock, search for areq in the chain and remove it. - */ - ASSERT(ictx != NULL); - mutex_enter(&ictx->kc_in_use_lock); - prev = cur = ictx->kc_req_chain_first; - - while (cur != NULL) { - if (cur == areq) { - if (prev == cur) { - if ((ictx->kc_req_chain_first = - cur->an_ctxchain_next) == NULL) - ictx->kc_req_chain_last = NULL; - } else { - if (cur == ictx->kc_req_chain_last) - ictx->kc_req_chain_last = prev; - prev->an_ctxchain_next = cur->an_ctxchain_next; - } - - break; - } - prev = cur; - cur = cur->an_ctxchain_next; - } - mutex_exit(&ictx->kc_in_use_lock); -} - -/* - * Remove the specified node from the global software queue. - * - * The caller must hold the queue lock and request lock (an_lock). - */ -static void -kcf_remove_node(kcf_areq_node_t *node) -{ - kcf_areq_node_t *nextp = node->an_next; - kcf_areq_node_t *prevp = node->an_prev; - - if (nextp != NULL) - nextp->an_prev = prevp; - else - gswq->gs_last = prevp; - - if (prevp != NULL) - prevp->an_next = nextp; - else - gswq->gs_first = nextp; - - node->an_state = REQ_CANCELED; -} - -/* - * Add the request node to the end of the global software queue. - * - * The caller should not hold the queue lock. Returns 0 if the - * request is successfully queued. Returns CRYPTO_BUSY if the limit - * on the number of jobs is exceeded. - */ -static int -kcf_enqueue(kcf_areq_node_t *node) -{ - kcf_areq_node_t *tnode; - - mutex_enter(&gswq->gs_lock); - - if (gswq->gs_njobs >= gswq->gs_maxjobs) { - mutex_exit(&gswq->gs_lock); - return (CRYPTO_BUSY); - } - - if (gswq->gs_last == NULL) { - gswq->gs_first = gswq->gs_last = node; - } else { - ASSERT(gswq->gs_last->an_next == NULL); - tnode = gswq->gs_last; - tnode->an_next = node; - gswq->gs_last = node; - node->an_prev = tnode; - } - - gswq->gs_njobs++; - - /* an_lock not needed here as we hold gs_lock */ - node->an_state = REQ_WAITING; - - mutex_exit(&gswq->gs_lock); - - return (0); -} - -/* - * kmem_cache_alloc constructor for sync request structure. - */ -static int -kcf_sreq_cache_constructor(void *buf, void *cdrarg, int kmflags) -{ - (void) cdrarg, (void) kmflags; - kcf_sreq_node_t *sreq = (kcf_sreq_node_t *)buf; - - sreq->sn_type = CRYPTO_SYNCH; - cv_init(&sreq->sn_cv, NULL, CV_DEFAULT, NULL); - mutex_init(&sreq->sn_lock, NULL, MUTEX_DEFAULT, NULL); - - return (0); -} - -static void -kcf_sreq_cache_destructor(void *buf, void *cdrarg) -{ - (void) cdrarg; - kcf_sreq_node_t *sreq = (kcf_sreq_node_t *)buf; - - mutex_destroy(&sreq->sn_lock); - cv_destroy(&sreq->sn_cv); -} - -/* - * kmem_cache_alloc constructor for async request structure. - */ -static int -kcf_areq_cache_constructor(void *buf, void *cdrarg, int kmflags) -{ - (void) cdrarg, (void) kmflags; - kcf_areq_node_t *areq = (kcf_areq_node_t *)buf; - - areq->an_type = CRYPTO_ASYNCH; - areq->an_refcnt = 0; - mutex_init(&areq->an_lock, NULL, MUTEX_DEFAULT, NULL); - cv_init(&areq->an_done, NULL, CV_DEFAULT, NULL); - cv_init(&areq->an_turn_cv, NULL, CV_DEFAULT, NULL); - - return (0); -} - -static void -kcf_areq_cache_destructor(void *buf, void *cdrarg) -{ - (void) cdrarg; - kcf_areq_node_t *areq = (kcf_areq_node_t *)buf; - - ASSERT(areq->an_refcnt == 0); - mutex_destroy(&areq->an_lock); - cv_destroy(&areq->an_done); - cv_destroy(&areq->an_turn_cv); -} - /* * kmem_cache_alloc constructor for kcf_context structure. */ @@ -1021,7 +113,6 @@ kcf_context_cache_constructor(void *buf, void *cdrarg, int kmflags) kcf_context_t *kctx = (kcf_context_t *)buf; kctx->kc_refcnt = 0; - mutex_init(&kctx->kc_in_use_lock, NULL, MUTEX_DEFAULT, NULL); return (0); } @@ -1033,49 +124,13 @@ kcf_context_cache_destructor(void *buf, void *cdrarg) kcf_context_t *kctx = (kcf_context_t *)buf; ASSERT(kctx->kc_refcnt == 0); - mutex_destroy(&kctx->kc_in_use_lock); } void kcf_sched_destroy(void) { - int i; - - if (kcf_misc_kstat) - kstat_delete(kcf_misc_kstat); - - if (kcfpool) { - mutex_destroy(&kcfpool->kp_thread_lock); - cv_destroy(&kcfpool->kp_nothr_cv); - mutex_destroy(&kcfpool->kp_user_lock); - cv_destroy(&kcfpool->kp_user_cv); - - kmem_free(kcfpool, sizeof (kcf_pool_t)); - } - - for (i = 0; i < REQID_TABLES; i++) { - if (kcf_reqid_table[i]) { - mutex_destroy(&(kcf_reqid_table[i]->rt_lock)); - kmem_free(kcf_reqid_table[i], - sizeof (kcf_reqid_table_t)); - } - } - - if (gswq) { - mutex_destroy(&gswq->gs_lock); - cv_destroy(&gswq->gs_cv); - kmem_free(gswq, sizeof (kcf_global_swq_t)); - } - if (kcf_context_cache) kmem_cache_destroy(kcf_context_cache); - if (kcf_areq_cache) - kmem_cache_destroy(kcf_areq_cache); - if (kcf_sreq_cache) - kmem_cache_destroy(kcf_sreq_cache); - - mutex_destroy(&ntfy_list_lock); - cv_destroy(&ntfy_list_cv); } /* @@ -1084,9 +139,6 @@ kcf_sched_destroy(void) void kcf_sched_init(void) { - int i; - kcf_reqid_table_t *rt; - /* * Create all the kmem caches needed by the framework. We set the * align argument to 64, to get a slab aligned to 64-byte as well as @@ -1094,673 +146,7 @@ kcf_sched_init(void) * This helps to avoid false sharing as this is the size of the * CPU cache line. */ - kcf_sreq_cache = kmem_cache_create("kcf_sreq_cache", - sizeof (struct kcf_sreq_node), 64, kcf_sreq_cache_constructor, - kcf_sreq_cache_destructor, NULL, NULL, NULL, 0); - - kcf_areq_cache = kmem_cache_create("kcf_areq_cache", - sizeof (struct kcf_areq_node), 64, kcf_areq_cache_constructor, - kcf_areq_cache_destructor, NULL, NULL, NULL, 0); - kcf_context_cache = kmem_cache_create("kcf_context_cache", sizeof (struct kcf_context), 64, kcf_context_cache_constructor, kcf_context_cache_destructor, NULL, NULL, NULL, 0); - - gswq = kmem_alloc(sizeof (kcf_global_swq_t), KM_SLEEP); - - mutex_init(&gswq->gs_lock, NULL, MUTEX_DEFAULT, NULL); - cv_init(&gswq->gs_cv, NULL, CV_DEFAULT, NULL); - gswq->gs_njobs = 0; - gswq->gs_maxjobs = kcf_maxthreads * crypto_taskq_maxalloc; - gswq->gs_first = gswq->gs_last = NULL; - - /* Initialize the global reqid table */ - for (i = 0; i < REQID_TABLES; i++) { - rt = kmem_zalloc(sizeof (kcf_reqid_table_t), KM_SLEEP); - kcf_reqid_table[i] = rt; - mutex_init(&rt->rt_lock, NULL, MUTEX_DEFAULT, NULL); - rt->rt_curid = i; - } - - /* Allocate and initialize the thread pool */ - kcfpool_alloc(); - - /* Initialize the event notification list variables */ - mutex_init(&ntfy_list_lock, NULL, MUTEX_DEFAULT, NULL); - cv_init(&ntfy_list_cv, NULL, CV_DEFAULT, NULL); - - /* Create the kcf kstat */ - kcf_misc_kstat = kstat_create("kcf", 0, "framework_stats", "crypto", - KSTAT_TYPE_NAMED, sizeof (kcf_stats_t) / sizeof (kstat_named_t), - KSTAT_FLAG_VIRTUAL); - - if (kcf_misc_kstat != NULL) { - kcf_misc_kstat->ks_data = &kcf_ksdata; - kcf_misc_kstat->ks_update = kcf_misc_kstat_update; - kstat_install(kcf_misc_kstat); - } -} - -/* - * Signal the waiting sync client. - */ -void -kcf_sop_done(kcf_sreq_node_t *sreq, int error) -{ - mutex_enter(&sreq->sn_lock); - sreq->sn_state = REQ_DONE; - sreq->sn_rv = error; - cv_signal(&sreq->sn_cv); - mutex_exit(&sreq->sn_lock); -} - -/* - * Callback the async client with the operation status. - * We free the async request node and possibly the context. - * We also handle any chain of requests hanging off of - * the context. - */ -void -kcf_aop_done(kcf_areq_node_t *areq, int error) -{ - kcf_op_type_t optype; - boolean_t skip_notify = B_FALSE; - kcf_context_t *ictx; - kcf_areq_node_t *nextreq; - - /* - * Handle recoverable errors. This has to be done first - * before doing anything else in this routine so that - * we do not change the state of the request. - */ - if (error != CRYPTO_SUCCESS && IS_RECOVERABLE(error)) { - /* - * We try another provider, if one is available. Else - * we continue with the failure notification to the - * client. - */ - if (kcf_resubmit_request(areq) == CRYPTO_QUEUED) - return; - } - - mutex_enter(&areq->an_lock); - areq->an_state = REQ_DONE; - mutex_exit(&areq->an_lock); - - optype = (&areq->an_params)->rp_optype; - if ((ictx = areq->an_context) != NULL) { - /* - * A request after it is removed from the request - * queue, still stays on a chain of requests hanging - * of its context structure. It needs to be removed - * from this chain at this point. - */ - mutex_enter(&ictx->kc_in_use_lock); - nextreq = areq->an_ctxchain_next; - if (nextreq != NULL) { - mutex_enter(&nextreq->an_lock); - nextreq->an_is_my_turn = B_TRUE; - cv_signal(&nextreq->an_turn_cv); - mutex_exit(&nextreq->an_lock); - } - - ictx->kc_req_chain_first = nextreq; - if (nextreq == NULL) - ictx->kc_req_chain_last = NULL; - mutex_exit(&ictx->kc_in_use_lock); - - if (IS_SINGLE_OP(optype) || IS_FINAL_OP(optype)) { - ASSERT(nextreq == NULL); - KCF_CONTEXT_REFRELE(ictx); - } else if (error != CRYPTO_SUCCESS && IS_INIT_OP(optype)) { - /* - * NOTE - We do not release the context in case of update - * operations. We require the consumer to free it explicitly, - * in case it wants to abandon an update operation. This is done - * as there may be mechanisms in ECB mode that can continue - * even if an operation on a block fails. - */ - KCF_CONTEXT_REFRELE(ictx); - } - } - - /* Deal with the internal continuation to this request first */ - - if (areq->an_isdual) { - kcf_dual_req_t *next_arg; - next_arg = (kcf_dual_req_t *)areq->an_reqarg.cr_callback_arg; - next_arg->kr_areq = areq; - KCF_AREQ_REFHOLD(areq); - areq->an_isdual = B_FALSE; - - NOTIFY_CLIENT(areq, error); - return; - } - - /* - * If CRYPTO_NOTIFY_OPDONE flag is set, we should notify - * always. If this flag is clear, we skip the notification - * provided there are no errors. We check this flag for only - * init or update operations. It is ignored for single, final or - * atomic operations. - */ - skip_notify = (IS_UPDATE_OP(optype) || IS_INIT_OP(optype)) && - (!(areq->an_reqarg.cr_flag & CRYPTO_NOTIFY_OPDONE)) && - (error == CRYPTO_SUCCESS); - - if (!skip_notify) { - NOTIFY_CLIENT(areq, error); - } - - if (!(areq->an_reqarg.cr_flag & CRYPTO_SKIP_REQID)) - kcf_reqid_delete(areq); - - KCF_AREQ_REFRELE(areq); -} - -/* - * Allocate the thread pool and initialize all the fields. - */ -static void -kcfpool_alloc() -{ - kcfpool = kmem_alloc(sizeof (kcf_pool_t), KM_SLEEP); - - kcfpool->kp_threads = kcfpool->kp_idlethreads = 0; - kcfpool->kp_blockedthreads = 0; - kcfpool->kp_signal_create_thread = B_FALSE; - kcfpool->kp_nthrs = 0; - kcfpool->kp_user_waiting = B_FALSE; - - mutex_init(&kcfpool->kp_thread_lock, NULL, MUTEX_DEFAULT, NULL); - cv_init(&kcfpool->kp_nothr_cv, NULL, CV_DEFAULT, NULL); - - mutex_init(&kcfpool->kp_user_lock, NULL, MUTEX_DEFAULT, NULL); - cv_init(&kcfpool->kp_user_cv, NULL, CV_DEFAULT, NULL); -} - -/* - * Insert the async request in the hash table after assigning it - * an ID. Returns the ID. - * - * The ID is used by the caller to pass as an argument to a - * cancel_req() routine later. - */ -static crypto_req_id_t -kcf_reqid_insert(kcf_areq_node_t *areq) -{ - int indx; - crypto_req_id_t id; - kcf_areq_node_t *headp; - kcf_reqid_table_t *rt; - - rt = kcf_reqid_table[CPU_SEQID_UNSTABLE & REQID_TABLE_MASK]; - - mutex_enter(&rt->rt_lock); - - rt->rt_curid = id = - (rt->rt_curid - REQID_COUNTER_LOW) | REQID_COUNTER_HIGH; - SET_REQID(areq, id); - indx = REQID_HASH(id); - headp = areq->an_idnext = rt->rt_idhash[indx]; - areq->an_idprev = NULL; - if (headp != NULL) - headp->an_idprev = areq; - - rt->rt_idhash[indx] = areq; - mutex_exit(&rt->rt_lock); - - return (id); -} - -/* - * Delete the async request from the hash table. - */ -static void -kcf_reqid_delete(kcf_areq_node_t *areq) -{ - int indx; - kcf_areq_node_t *nextp, *prevp; - crypto_req_id_t id = GET_REQID(areq); - kcf_reqid_table_t *rt; - - rt = kcf_reqid_table[id & REQID_TABLE_MASK]; - indx = REQID_HASH(id); - - mutex_enter(&rt->rt_lock); - - nextp = areq->an_idnext; - prevp = areq->an_idprev; - if (nextp != NULL) - nextp->an_idprev = prevp; - if (prevp != NULL) - prevp->an_idnext = nextp; - else - rt->rt_idhash[indx] = nextp; - - SET_REQID(areq, 0); - cv_broadcast(&areq->an_done); - - mutex_exit(&rt->rt_lock); -} - -/* - * Cancel a single asynchronous request. - * - * We guarantee that no problems will result from calling - * crypto_cancel_req() for a request which is either running, or - * has already completed. We remove the request from any queues - * if it is possible. We wait for request completion if the - * request is dispatched to a provider. - * - * Calling context: - * Can be called from user context only. - * - * NOTE: We acquire the following locks in this routine (in order): - * - rt_lock (kcf_reqid_table_t) - * - gswq->gs_lock - * - areq->an_lock - * - ictx->kc_in_use_lock (from kcf_removereq_in_ctxchain()) - * - * This locking order MUST be maintained in code every where else. - */ -void -crypto_cancel_req(crypto_req_id_t id) -{ - int indx; - kcf_areq_node_t *areq; - kcf_provider_desc_t *pd; - kcf_context_t *ictx; - kcf_reqid_table_t *rt; - - rt = kcf_reqid_table[id & REQID_TABLE_MASK]; - indx = REQID_HASH(id); - - mutex_enter(&rt->rt_lock); - for (areq = rt->rt_idhash[indx]; areq; areq = areq->an_idnext) { - if (GET_REQID(areq) == id) { - /* - * We found the request. It is either still waiting - * in the framework queues or running at the provider. - */ - pd = areq->an_provider; - ASSERT(pd != NULL); - - switch (pd->pd_prov_type) { - case CRYPTO_SW_PROVIDER: - mutex_enter(&gswq->gs_lock); - mutex_enter(&areq->an_lock); - - /* This request can be safely canceled. */ - if (areq->an_state <= REQ_WAITING) { - /* Remove from gswq, global software queue. */ - kcf_remove_node(areq); - if ((ictx = areq->an_context) != NULL) - kcf_removereq_in_ctxchain(ictx, areq); - - mutex_exit(&areq->an_lock); - mutex_exit(&gswq->gs_lock); - mutex_exit(&rt->rt_lock); - - /* Remove areq from hash table and free it. */ - kcf_reqid_delete(areq); - KCF_AREQ_REFRELE(areq); - return; - } - - mutex_exit(&areq->an_lock); - mutex_exit(&gswq->gs_lock); - break; - - case CRYPTO_HW_PROVIDER: - /* - * There is no interface to remove an entry - * once it is on the taskq. So, we do not do - * anything for a hardware provider. - */ - break; - default: - break; - } - - /* - * The request is running. Wait for the request completion - * to notify us. - */ - KCF_AREQ_REFHOLD(areq); - while (GET_REQID(areq) == id) - cv_wait(&areq->an_done, &rt->rt_lock); - KCF_AREQ_REFRELE(areq); - break; - } - } - - mutex_exit(&rt->rt_lock); -} - -/* - * Cancel all asynchronous requests associated with the - * passed in crypto context and free it. - * - * A client SHOULD NOT call this routine after calling a crypto_*_final - * routine. This routine is called only during intermediate operations. - * The client should not use the crypto context after this function returns - * since we destroy it. - * - * Calling context: - * Can be called from user context only. - */ -void -crypto_cancel_ctx(crypto_context_t ctx) -{ - kcf_context_t *ictx; - kcf_areq_node_t *areq; - - if (ctx == NULL) - return; - - ictx = (kcf_context_t *)((crypto_ctx_t *)ctx)->cc_framework_private; - - mutex_enter(&ictx->kc_in_use_lock); - - /* Walk the chain and cancel each request */ - while ((areq = ictx->kc_req_chain_first) != NULL) { - /* - * We have to drop the lock here as we may have - * to wait for request completion. We hold the - * request before dropping the lock though, so that it - * won't be freed underneath us. - */ - KCF_AREQ_REFHOLD(areq); - mutex_exit(&ictx->kc_in_use_lock); - - crypto_cancel_req(GET_REQID(areq)); - KCF_AREQ_REFRELE(areq); - - mutex_enter(&ictx->kc_in_use_lock); - } - - mutex_exit(&ictx->kc_in_use_lock); - KCF_CONTEXT_REFRELE(ictx); -} - -/* - * Update kstats. - */ -static int -kcf_misc_kstat_update(kstat_t *ksp, int rw) -{ - uint_t tcnt; - kcf_stats_t *ks_data; - - if (rw == KSTAT_WRITE) - return (EACCES); - - ks_data = ksp->ks_data; - - ks_data->ks_thrs_in_pool.value.ui32 = kcfpool->kp_threads; - /* - * The failover thread is counted in kp_idlethreads in - * some corner cases. This is done to avoid doing more checks - * when submitting a request. We account for those cases below. - */ - if ((tcnt = kcfpool->kp_idlethreads) == (kcfpool->kp_threads + 1)) - tcnt--; - ks_data->ks_idle_thrs.value.ui32 = tcnt; - ks_data->ks_minthrs.value.ui32 = kcf_minthreads; - ks_data->ks_maxthrs.value.ui32 = kcf_maxthreads; - ks_data->ks_swq_njobs.value.ui32 = gswq->gs_njobs; - ks_data->ks_swq_maxjobs.value.ui32 = gswq->gs_maxjobs; - ks_data->ks_taskq_threads.value.ui32 = crypto_taskq_threads; - ks_data->ks_taskq_minalloc.value.ui32 = crypto_taskq_minalloc; - ks_data->ks_taskq_maxalloc.value.ui32 = crypto_taskq_maxalloc; - - return (0); -} - -/* - * Allocate and initialize a kcf_dual_req, used for saving the arguments of - * a dual operation or an atomic operation that has to be internally - * simulated with multiple single steps. - * crq determines the memory allocation flags. - */ - -kcf_dual_req_t * -kcf_alloc_req(crypto_call_req_t *crq) -{ - kcf_dual_req_t *kcr; - - kcr = kmem_alloc(sizeof (kcf_dual_req_t), KCF_KMFLAG(crq)); - - if (kcr == NULL) - return (NULL); - - /* Copy the whole crypto_call_req struct, as it isn't persistent */ - if (crq != NULL) - kcr->kr_callreq = *crq; - else - bzero(&(kcr->kr_callreq), sizeof (crypto_call_req_t)); - kcr->kr_areq = NULL; - kcr->kr_saveoffset = 0; - kcr->kr_savelen = 0; - - return (kcr); -} - -/* - * Callback routine for the next part of a simulated dual part. - * Schedules the next step. - * - * This routine can be called from interrupt context. - */ -void -kcf_next_req(void *next_req_arg, int status) -{ - kcf_dual_req_t *next_req = (kcf_dual_req_t *)next_req_arg; - kcf_req_params_t *params = &(next_req->kr_params); - kcf_areq_node_t *areq = next_req->kr_areq; - int error = status; - kcf_provider_desc_t *pd = NULL; - crypto_dual_data_t *ct = NULL; - - /* Stop the processing if an error occurred at this step */ - if (error != CRYPTO_SUCCESS) { -out: - areq->an_reqarg = next_req->kr_callreq; - KCF_AREQ_REFRELE(areq); - kmem_free(next_req, sizeof (kcf_dual_req_t)); - areq->an_isdual = B_FALSE; - kcf_aop_done(areq, error); - return; - } - - switch (params->rp_opgrp) { - case KCF_OG_MAC: { - - /* - * The next req is submitted with the same reqid as the - * first part. The consumer only got back that reqid, and - * should still be able to cancel the operation during its - * second step. - */ - kcf_mac_ops_params_t *mops = &(params->rp_u.mac_params); - crypto_ctx_template_t mac_tmpl; - kcf_mech_entry_t *me; - - ct = (crypto_dual_data_t *)mops->mo_data; - mac_tmpl = (crypto_ctx_template_t)mops->mo_templ; - - /* No expected recoverable failures, so no retry list */ - pd = kcf_get_mech_provider(mops->mo_framework_mechtype, - &me, &error, NULL, CRYPTO_FG_MAC_ATOMIC, - (areq->an_reqarg.cr_flag & CRYPTO_RESTRICTED), ct->dd_len2); - - if (pd == NULL) { - error = CRYPTO_MECH_NOT_SUPPORTED; - goto out; - } - /* Validate the MAC context template here */ - if ((pd->pd_prov_type == CRYPTO_SW_PROVIDER) && - (mac_tmpl != NULL)) { - kcf_ctx_template_t *ctx_mac_tmpl; - - ctx_mac_tmpl = (kcf_ctx_template_t *)mac_tmpl; - - if (ctx_mac_tmpl->ct_generation != me->me_gen_swprov) { - KCF_PROV_REFRELE(pd); - error = CRYPTO_OLD_CTX_TEMPLATE; - goto out; - } - mops->mo_templ = ctx_mac_tmpl->ct_prov_tmpl; - } - - break; - } - case KCF_OG_DECRYPT: { - kcf_decrypt_ops_params_t *dcrops = - &(params->rp_u.decrypt_params); - - ct = (crypto_dual_data_t *)dcrops->dop_ciphertext; - /* No expected recoverable failures, so no retry list */ - pd = kcf_get_mech_provider(dcrops->dop_framework_mechtype, - NULL, &error, NULL, CRYPTO_FG_DECRYPT_ATOMIC, - (areq->an_reqarg.cr_flag & CRYPTO_RESTRICTED), ct->dd_len1); - - if (pd == NULL) { - error = CRYPTO_MECH_NOT_SUPPORTED; - goto out; - } - break; - } - default: - break; - } - - /* The second step uses len2 and offset2 of the dual_data */ - next_req->kr_saveoffset = ct->dd_offset1; - next_req->kr_savelen = ct->dd_len1; - ct->dd_offset1 = ct->dd_offset2; - ct->dd_len1 = ct->dd_len2; - - /* preserve if the caller is restricted */ - if (areq->an_reqarg.cr_flag & CRYPTO_RESTRICTED) { - areq->an_reqarg.cr_flag = CRYPTO_RESTRICTED; - } else { - areq->an_reqarg.cr_flag = 0; - } - - areq->an_reqarg.cr_callback_func = kcf_last_req; - areq->an_reqarg.cr_callback_arg = next_req; - areq->an_isdual = B_TRUE; - - /* - * We would like to call kcf_submit_request() here. But, - * that is not possible as that routine allocates a new - * kcf_areq_node_t request structure, while we need to - * reuse the existing request structure. - */ - switch (pd->pd_prov_type) { - case CRYPTO_SW_PROVIDER: - error = common_submit_request(pd, NULL, params, - KCF_RHNDL(KM_NOSLEEP)); - break; - - case CRYPTO_HW_PROVIDER: { - kcf_provider_desc_t *old_pd; - taskq_t *taskq = pd->pd_sched_info.ks_taskq; - - /* - * Set the params for the second step in the - * dual-ops. - */ - areq->an_params = *params; - old_pd = areq->an_provider; - KCF_PROV_REFRELE(old_pd); - KCF_PROV_REFHOLD(pd); - areq->an_provider = pd; - - /* - * Note that we have to do a taskq_dispatch() - * here as we may be in interrupt context. - */ - if (taskq_dispatch(taskq, process_req_hwp, areq, - TQ_NOSLEEP) == (taskqid_t)0) { - error = CRYPTO_HOST_MEMORY; - } else { - error = CRYPTO_QUEUED; - } - break; - } - default: - break; - } - - /* - * We have to release the holds on the request and the provider - * in all cases. - */ - KCF_AREQ_REFRELE(areq); - KCF_PROV_REFRELE(pd); - - if (error != CRYPTO_QUEUED) { - /* restore, clean up, and invoke the client's callback */ - - ct->dd_offset1 = next_req->kr_saveoffset; - ct->dd_len1 = next_req->kr_savelen; - areq->an_reqarg = next_req->kr_callreq; - kmem_free(next_req, sizeof (kcf_dual_req_t)); - areq->an_isdual = B_FALSE; - kcf_aop_done(areq, error); - } -} - -/* - * Last part of an emulated dual operation. - * Clean up and restore ... - */ -void -kcf_last_req(void *last_req_arg, int status) -{ - kcf_dual_req_t *last_req = (kcf_dual_req_t *)last_req_arg; - - kcf_req_params_t *params = &(last_req->kr_params); - kcf_areq_node_t *areq = last_req->kr_areq; - crypto_dual_data_t *ct = NULL; - - switch (params->rp_opgrp) { - case KCF_OG_MAC: { - kcf_mac_ops_params_t *mops = &(params->rp_u.mac_params); - - ct = (crypto_dual_data_t *)mops->mo_data; - break; - } - case KCF_OG_DECRYPT: { - kcf_decrypt_ops_params_t *dcrops = - &(params->rp_u.decrypt_params); - - ct = (crypto_dual_data_t *)dcrops->dop_ciphertext; - break; - } - default: { - panic("invalid kcf_op_group_t %d", (int)params->rp_opgrp); - return; - } - } - ct->dd_offset1 = last_req->kr_saveoffset; - ct->dd_len1 = last_req->kr_savelen; - - /* The submitter used kcf_last_req as its callback */ - - if (areq == NULL) { - crypto_call_req_t *cr = &last_req->kr_callreq; - - (*(cr->cr_callback_func))(cr->cr_callback_arg, status); - kmem_free(last_req, sizeof (kcf_dual_req_t)); - return; - } - areq->an_reqarg = last_req->kr_callreq; - KCF_AREQ_REFRELE(areq); - kmem_free(last_req, sizeof (kcf_dual_req_t)); - areq->an_isdual = B_FALSE; - kcf_aop_done(areq, status); } diff --git a/module/icp/illumos-crypto.c b/module/icp/illumos-crypto.c index 5b2820220f2c..13f05c06ed5c 100644 --- a/module/icp/illumos-crypto.c +++ b/module/icp/illumos-crypto.c @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -36,7 +36,6 @@ #include #include #include -#include #include /* @@ -105,7 +104,7 @@ * ZFS Makefiles. */ -void __exit +void icp_fini(void) { skein_mod_fini(); @@ -114,16 +113,12 @@ icp_fini(void) kcf_sched_destroy(); kcf_prov_tab_destroy(); kcf_destroy_mech_tabs(); - mod_hash_fini(); } /* roughly equivalent to kcf.c: _init() */ int __init icp_init(void) { - /* initialize the mod hash module */ - mod_hash_init(); - /* initialize the mechanisms tables supported out-of-the-box */ kcf_init_mech_tabs(); @@ -144,10 +139,7 @@ icp_init(void) return (0); } -#if defined(_KERNEL) +#if defined(_KERNEL) && defined(__FreeBSD__) module_exit(icp_fini); module_init(icp_init); -MODULE_AUTHOR(ZFS_META_AUTHOR); -MODULE_LICENSE(ZFS_META_LICENSE); -MODULE_VERSION(ZFS_META_VERSION "-" ZFS_META_RELEASE); #endif diff --git a/module/icp/include/aes/aes_impl.h b/module/icp/include/aes/aes_impl.h index 41dccaa3848a..fe5c23974682 100644 --- a/module/icp/include/aes/aes_impl.h +++ b/module/icp/include/aes/aes_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -83,14 +83,7 @@ extern "C" { /* AES key size definitions */ #define AES_MINBITS 128 -#define AES_MINBYTES ((AES_MINBITS) >> 3) #define AES_MAXBITS 256 -#define AES_MAXBYTES ((AES_MAXBITS) >> 3) - -#define AES_MIN_KEY_BYTES ((AES_MINBITS) >> 3) -#define AES_MAX_KEY_BYTES ((AES_MAXBITS) >> 3) -#define AES_192_KEY_BYTES 24 -#define AES_IV_LEN 16 /* AES key schedule may be implemented with 32- or 64-bit elements: */ #define AES_32BIT_KS 32 diff --git a/module/icp/include/modes/gcm_impl.h b/module/icp/include/modes/gcm_impl.h index 28c8f63a7d46..3afc9e2c6317 100644 --- a/module/icp/include/modes/gcm_impl.h +++ b/module/icp/include/modes/gcm_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/icp/include/modes/modes.h b/module/icp/include/modes/modes.h index ab71197542eb..9217895d7ff2 100644 --- a/module/icp/include/modes/modes.h +++ b/module/icp/include/modes/modes.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -207,10 +207,6 @@ typedef struct ccm_ctx { * * gcm_len_a_len_c: 64-bit representations of the bit lengths of * AAD and ciphertext. - * - * gcm_kmflag: Current value of kmflag. Used for allocating - * the plaintext buffer during decryption and a - * gcm_avx_chunk_size'd buffer for avx enabled encryption. */ typedef struct gcm_ctx { struct common_ctx gcm_common; @@ -231,7 +227,6 @@ typedef struct gcm_ctx { uint64_t gcm_J0[2]; uint64_t gcm_len_a_len_c[2]; uint8_t *gcm_pt_buf; - int gcm_kmflag; #ifdef CAN_USE_GCM_ASM boolean_t gcm_use_avx; #endif @@ -402,7 +397,6 @@ extern void *ccm_alloc_ctx(int); extern void *gcm_alloc_ctx(int); extern void *gmac_alloc_ctx(int); extern void crypto_free_mode_ctx(void *); -extern void gcm_set_kmflag(gcm_ctx_t *, int); #ifdef __cplusplus } diff --git a/module/icp/include/sha2/sha2_consts.h b/module/icp/include/sha2/sha2_consts.h index 3a6645508fe9..b33ddf821843 100644 --- a/module/icp/include/sha2/sha2_consts.h +++ b/module/icp/include/sha2/sha2_consts.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/icp/include/sha2/sha2_impl.h b/module/icp/include/sha2/sha2_impl.h index b9768d344e95..0e89747eefd1 100644 --- a/module/icp/include/sha2/sha2_impl.h +++ b/module/icp/include/sha2/sha2_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/icp/include/sys/asm_linkage.h b/module/icp/include/sys/asm_linkage.h index 49a494b46e0b..84aa0854a9ff 100644 --- a/module/icp/include/sys/asm_linkage.h +++ b/module/icp/include/sys/asm_linkage.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/icp/include/sys/bitmap.h b/module/icp/include/sys/bitmap.h deleted file mode 100644 index 4e86ee70ed9e..000000000000 --- a/module/icp/include/sys/bitmap.h +++ /dev/null @@ -1,183 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License (the "License"). - * You may not use this file except in compliance with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ - -/* - * Copyright 2006 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -/* Copyright (c) 1984, 1986, 1987, 1988, 1989 AT&T */ -/* All Rights Reserved */ - - -#ifndef _SYS_BITMAP_H -#define _SYS_BITMAP_H - -#ifdef __cplusplus -extern "C" { -#endif - -#if defined(__GNUC__) && defined(_ASM_INLINES) && \ - (defined(__i386) || defined(__amd64)) -#include -#endif - -/* - * Operations on bitmaps of arbitrary size - * A bitmap is a vector of 1 or more ulong_t's. - * The user of the package is responsible for range checks and keeping - * track of sizes. - */ - -#ifdef _LP64 -#define BT_ULSHIFT 6 /* log base 2 of BT_NBIPUL, to extract word index */ -#define BT_ULSHIFT32 5 /* log base 2 of BT_NBIPUL, to extract word index */ -#else -#define BT_ULSHIFT 5 /* log base 2 of BT_NBIPUL, to extract word index */ -#endif - -#define BT_NBIPUL (1 << BT_ULSHIFT) /* n bits per ulong_t */ -#define BT_ULMASK (BT_NBIPUL - 1) /* to extract bit index */ - -#ifdef _LP64 -#define BT_NBIPUL32 (1 << BT_ULSHIFT32) /* n bits per ulong_t */ -#define BT_ULMASK32 (BT_NBIPUL32 - 1) /* to extract bit index */ -#define BT_ULMAXMASK 0xffffffffffffffff /* used by bt_getlowbit */ -#else -#define BT_ULMAXMASK 0xffffffff -#endif - -/* - * bitmap is a ulong_t *, bitindex an index_t - * - * The macros BT_WIM and BT_BIW internal; there is no need - * for users of this package to use them. - */ - -/* - * word in map - */ -#define BT_WIM(bitmap, bitindex) \ - ((bitmap)[(bitindex) >> BT_ULSHIFT]) -/* - * bit in word - */ -#define BT_BIW(bitindex) \ - (1UL << ((bitindex) & BT_ULMASK)) - -#ifdef _LP64 -#define BT_WIM32(bitmap, bitindex) \ - ((bitmap)[(bitindex) >> BT_ULSHIFT32]) - -#define BT_BIW32(bitindex) \ - (1UL << ((bitindex) & BT_ULMASK32)) -#endif - -/* - * These are public macros - * - * BT_BITOUL == n bits to n ulong_t's - */ -#define BT_BITOUL(nbits) \ - (((nbits) + BT_NBIPUL - 1l) / BT_NBIPUL) -#define BT_SIZEOFMAP(nbits) \ - (BT_BITOUL(nbits) * sizeof (ulong_t)) -#define BT_TEST(bitmap, bitindex) \ - ((BT_WIM((bitmap), (bitindex)) & BT_BIW(bitindex)) ? 1 : 0) -#define BT_SET(bitmap, bitindex) \ - { BT_WIM((bitmap), (bitindex)) |= BT_BIW(bitindex); } -#define BT_CLEAR(bitmap, bitindex) \ - { BT_WIM((bitmap), (bitindex)) &= ~BT_BIW(bitindex); } - -#ifdef _LP64 -#define BT_BITOUL32(nbits) \ - (((nbits) + BT_NBIPUL32 - 1l) / BT_NBIPUL32) -#define BT_SIZEOFMAP32(nbits) \ - (BT_BITOUL32(nbits) * sizeof (uint_t)) -#define BT_TEST32(bitmap, bitindex) \ - ((BT_WIM32((bitmap), (bitindex)) & BT_BIW32(bitindex)) ? 1 : 0) -#define BT_SET32(bitmap, bitindex) \ - { BT_WIM32((bitmap), (bitindex)) |= BT_BIW32(bitindex); } -#define BT_CLEAR32(bitmap, bitindex) \ - { BT_WIM32((bitmap), (bitindex)) &= ~BT_BIW32(bitindex); } -#endif /* _LP64 */ - - -/* - * BIT_ONLYONESET is a private macro not designed for bitmaps of - * arbitrary size. u must be an unsigned integer/long. It returns - * true if one and only one bit is set in u. - */ -#define BIT_ONLYONESET(u) \ - ((((u) == 0) ? 0 : ((u) & ((u) - 1)) == 0)) - -#ifndef _ASM - -/* - * return next available bit index from map with specified number of bits - */ -extern index_t bt_availbit(ulong_t *bitmap, size_t nbits); -/* - * find the highest order bit that is on, and is within or below - * the word specified by wx - */ -extern int bt_gethighbit(ulong_t *mapp, int wx); -extern int bt_range(ulong_t *bitmap, size_t *pos1, size_t *pos2, - size_t end_pos); -extern int bt_getlowbit(ulong_t *bitmap, size_t start, size_t stop); -extern void bt_copy(ulong_t *, ulong_t *, ulong_t); - -/* - * find the parity - */ -extern int odd_parity(ulong_t); - -/* - * Atomically set/clear bits - * Atomic exclusive operations will set "result" to "-1" - * if the bit is already set/cleared. "result" will be set - * to 0 otherwise. - */ -#define BT_ATOMIC_SET(bitmap, bitindex) \ - { atomic_or_ulong(&(BT_WIM(bitmap, bitindex)), BT_BIW(bitindex)); } -#define BT_ATOMIC_CLEAR(bitmap, bitindex) \ - { atomic_and_ulong(&(BT_WIM(bitmap, bitindex)), ~BT_BIW(bitindex)); } - -#define BT_ATOMIC_SET_EXCL(bitmap, bitindex, result) \ - { result = atomic_set_long_excl(&(BT_WIM(bitmap, bitindex)), \ - (bitindex) % BT_NBIPUL); } -#define BT_ATOMIC_CLEAR_EXCL(bitmap, bitindex, result) \ - { result = atomic_clear_long_excl(&(BT_WIM(bitmap, bitindex)), \ - (bitindex) % BT_NBIPUL); } - -/* - * Extracts bits between index h (high, inclusive) and l (low, exclusive) from - * u, which must be an unsigned integer. - */ -#define BITX(u, h, l) (((u) >> (l)) & ((1LU << ((h) - (l) + 1LU)) - 1LU)) - -#endif /* _ASM */ - -#ifdef __cplusplus -} -#endif - -#endif /* _SYS_BITMAP_H */ diff --git a/module/icp/include/sys/crypto/elfsign.h b/module/icp/include/sys/crypto/elfsign.h deleted file mode 100644 index 5432f0c8d607..000000000000 --- a/module/icp/include/sys/crypto/elfsign.h +++ /dev/null @@ -1,137 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License (the "License"). - * You may not use this file except in compliance with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2008 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#ifndef _SYS_CRYPTO_ELFSIGN_H -#define _SYS_CRYPTO_ELFSIGN_H - -#ifdef __cplusplus -extern "C" { -#endif - -/* - * Consolidation Private Interface for elfsign/libpkcs11/kcfd - */ - -#include - -/* - * Project Private structures and types used for communication between kcfd - * and KCF over the door. - */ - -typedef enum ELFsign_status_e { - ELFSIGN_UNKNOWN, - ELFSIGN_SUCCESS, - ELFSIGN_FAILED, - ELFSIGN_NOTSIGNED, - ELFSIGN_INVALID_CERTPATH, - ELFSIGN_INVALID_ELFOBJ, - ELFSIGN_RESTRICTED -} ELFsign_status_t; - -#define KCF_KCFD_VERSION1 1 -#define SIG_MAX_LENGTH 1024 - -#define ELF_SIGNATURE_SECTION ".SUNW_signature" - -typedef struct kcf_door_arg_s { - short da_version; - boolean_t da_iskernel; - - union { - char filename[MAXPATHLEN]; /* For request */ - - struct kcf_door_result_s { /* For response */ - ELFsign_status_t status; - uint32_t siglen; - uchar_t signature[1]; - } result; - } da_u; -} kcf_door_arg_t; - -typedef uint32_t filesig_vers_t; - -/* - * File Signature Structure - * Applicable to ELF and other file formats - */ -struct filesignatures { - uint32_t filesig_cnt; /* count of signatures */ - uint32_t filesig_pad; /* unused */ - union { - char filesig_data[1]; - struct filesig { /* one of these for each signature */ - uint32_t filesig_size; - filesig_vers_t filesig_version; - union { - struct filesig_version1 { - uint32_t filesig_v1_dnsize; - uint32_t filesig_v1_sigsize; - uint32_t filesig_v1_oidsize; - char filesig_v1_data[1]; - } filesig_v1; - struct filesig_version3 { - uint64_t filesig_v3_time; - uint32_t filesig_v3_dnsize; - uint32_t filesig_v3_sigsize; - uint32_t filesig_v3_oidsize; - char filesig_v3_data[1]; - } filesig_v3; - } _u2; - } filesig_sig; - uint64_t filesig_align; - } _u1; -}; -#define filesig_sig _u1.filesig_sig - -#define filesig_v1_dnsize _u2.filesig_v1.filesig_v1_dnsize -#define filesig_v1_sigsize _u2.filesig_v1.filesig_v1_sigsize -#define filesig_v1_oidsize _u2.filesig_v1.filesig_v1_oidsize -#define filesig_v1_data _u2.filesig_v1.filesig_v1_data - -#define filesig_v3_time _u2.filesig_v3.filesig_v3_time -#define filesig_v3_dnsize _u2.filesig_v3.filesig_v3_dnsize -#define filesig_v3_sigsize _u2.filesig_v3.filesig_v3_sigsize -#define filesig_v3_oidsize _u2.filesig_v3.filesig_v3_oidsize -#define filesig_v3_data _u2.filesig_v3.filesig_v3_data - -#define filesig_ALIGN(s) (((s) + sizeof (uint64_t) - 1) & \ - (-sizeof (uint64_t))) -#define filesig_next(ptr) (struct filesig *)((void *)((char *)(ptr) + \ - filesig_ALIGN((ptr)->filesig_size))) - -#define FILESIG_UNKNOWN 0 /* unrecognized version */ -#define FILESIG_VERSION1 1 /* version1, all but sig section */ -#define FILESIG_VERSION2 2 /* version1 format, SHF_ALLOC only */ -#define FILESIG_VERSION3 3 /* version3, all but sig section */ -#define FILESIG_VERSION4 4 /* version3 format, SHF_ALLOC only */ - -#define _PATH_KCFD_DOOR "/etc/svc/volatile/kcfd_door" - -#ifdef __cplusplus -} -#endif - -#endif /* _SYS_CRYPTO_ELFSIGN_H */ diff --git a/module/icp/include/sys/crypto/impl.h b/module/icp/include/sys/crypto/impl.h index 4906549b5687..32ac43475a3e 100644 --- a/module/icp/include/sys/crypto/impl.h +++ b/module/icp/include/sys/crypto/impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -34,79 +34,17 @@ #include #include #include +#include #ifdef __cplusplus extern "C" { #endif -#define KCF_MODULE "kcf" - /* * Prefixes convention: structures internal to the kernel cryptographic * framework start with 'kcf_'. Exposed structure start with 'crypto_'. */ -/* Provider stats. Not protected. */ -typedef struct kcf_prov_stats { - kstat_named_t ps_ops_total; - kstat_named_t ps_ops_passed; - kstat_named_t ps_ops_failed; - kstat_named_t ps_ops_busy_rval; -} kcf_prov_stats_t; - -/* Various kcf stats. Not protected. */ -typedef struct kcf_stats { - kstat_named_t ks_thrs_in_pool; - kstat_named_t ks_idle_thrs; - kstat_named_t ks_minthrs; - kstat_named_t ks_maxthrs; - kstat_named_t ks_swq_njobs; - kstat_named_t ks_swq_maxjobs; - kstat_named_t ks_taskq_threads; - kstat_named_t ks_taskq_minalloc; - kstat_named_t ks_taskq_maxalloc; -} kcf_stats_t; - -/* - * Keep all the information needed by the scheduler from - * this provider. - */ -typedef struct kcf_sched_info { - /* The number of operations dispatched. */ - uint64_t ks_ndispatches; - - /* The number of operations that failed. */ - uint64_t ks_nfails; - - /* The number of operations that returned CRYPTO_BUSY. */ - uint64_t ks_nbusy_rval; - - /* taskq used to dispatch crypto requests */ - taskq_t *ks_taskq; -} kcf_sched_info_t; - -/* - * pd_irefcnt approximates the number of inflight requests to the - * provider. Though we increment this counter during registration for - * other purposes, that base value is mostly same across all providers. - * So, it is a good measure of the load on a provider when it is not - * in a busy state. Once a provider notifies it is busy, requests - * backup in the taskq. So, we use tq_nalloc in that case which gives - * the number of task entries in the task queue. Note that we do not - * acquire any locks here as it is not critical to get the exact number - * and the lock contention may be too costly for this code path. - */ -#define KCF_PROV_LOAD(pd) ((pd)->pd_state != KCF_PROV_BUSY ? \ - (pd)->pd_irefcnt : (pd)->pd_sched_info.ks_taskq->tq_nalloc) - -#define KCF_PROV_INCRSTATS(pd, error) { \ - (pd)->pd_sched_info.ks_ndispatches++; \ - if (error == CRYPTO_BUSY) \ - (pd)->pd_sched_info.ks_nbusy_rval++; \ - else if (error != CRYPTO_SUCCESS && error != CRYPTO_QUEUED) \ - (pd)->pd_sched_info.ks_nfails++; \ -} - /* * The following two macros should be @@ -117,7 +55,7 @@ typedef struct kcf_sched_info { * When impl.h is broken up (bug# 4703218), this will be done. For now, * we hardcode these values. */ -#define KCF_OPS_CLASSSIZE 8 +#define KCF_OPS_CLASSSIZE 4 #define KCF_MAXMECHTAB 32 /* @@ -125,21 +63,17 @@ typedef struct kcf_sched_info { * the elements is important. * * Routines which get a provider or the list of providers - * should pick only those that are either in KCF_PROV_READY state - * or in KCF_PROV_BUSY state. + * should pick only those that are in KCF_PROV_READY state. */ typedef enum { KCF_PROV_ALLOCATED = 1, - KCF_PROV_UNVERIFIED, - KCF_PROV_VERIFICATION_FAILED, /* * state < KCF_PROV_READY means the provider can not * be used at all. */ KCF_PROV_READY, - KCF_PROV_BUSY, /* - * state > KCF_PROV_BUSY means the provider can not + * state > KCF_PROV_READY means the provider can not * be used for new requests. */ KCF_PROV_FAILED, @@ -152,90 +86,46 @@ typedef enum { KCF_PROV_FREED } kcf_prov_state_t; -#define KCF_IS_PROV_UNVERIFIED(pd) ((pd)->pd_state == KCF_PROV_UNVERIFIED) -#define KCF_IS_PROV_USABLE(pd) ((pd)->pd_state == KCF_PROV_READY || \ - (pd)->pd_state == KCF_PROV_BUSY) +#define KCF_IS_PROV_USABLE(pd) ((pd)->pd_state == KCF_PROV_READY) #define KCF_IS_PROV_REMOVED(pd) ((pd)->pd_state >= KCF_PROV_REMOVED) -/* Internal flags valid for pd_flags field */ -#define KCF_PROV_RESTRICTED 0x40000000 -#define KCF_LPROV_MEMBER 0x80000000 /* is member of a logical provider */ - /* * A provider descriptor structure. There is one such structure per * provider. It is allocated and initialized at registration time and * freed when the provider unregisters. * - * pd_prov_type: Provider type, hardware or software - * pd_sid: Session ID of the provider used by kernel clients. - * This is valid only for session-oriented providers. * pd_refcnt: Reference counter to this provider descriptor * pd_irefcnt: References held by the framework internal structs - * pd_lock: lock protects pd_state and pd_provider_list + * pd_lock: lock protects pd_state * pd_state: State value of the provider - * pd_provider_list: Used to cross-reference logical providers and their - * members. Not used for software providers. - * pd_resume_cv: cv to wait for state to change from KCF_PROV_BUSY - * pd_prov_handle: Provider handle specified by provider * pd_ops_vector: The ops vector specified by Provider * pd_mech_indx: Lookup table which maps a core framework mechanism * number to an index in pd_mechanisms array * pd_mechanisms: Array of mechanisms supported by the provider, specified * by the provider during registration - * pd_sched_info: Scheduling information associated with the provider * pd_mech_list_count: The number of entries in pi_mechanisms, specified * by the provider during registration - * pd_name: Device name or module name - * pd_instance: Device instance - * pd_module_id: Module ID returned by modload - * pd_mctlp: Pointer to modctl structure for this provider * pd_remove_cv: cv to wait on while the provider queue drains * pd_description: Provider description string - * pd_flags bitwise OR of pi_flags from crypto_provider_info_t - * and other internal flags defined above. - * pd_hash_limit Maximum data size that hash mechanisms of this provider - * can support. * pd_kcf_prov_handle: KCF-private handle assigned by KCF * pd_prov_id: Identification # assigned by KCF to provider - * pd_kstat: kstat associated with the provider - * pd_ks_data: kstat data */ typedef struct kcf_provider_desc { - crypto_provider_type_t pd_prov_type; - crypto_session_id_t pd_sid; uint_t pd_refcnt; uint_t pd_irefcnt; kmutex_t pd_lock; kcf_prov_state_t pd_state; - struct kcf_provider_list *pd_provider_list; - kcondvar_t pd_resume_cv; - crypto_provider_handle_t pd_prov_handle; const crypto_ops_t *pd_ops_vector; ushort_t pd_mech_indx[KCF_OPS_CLASSSIZE]\ [KCF_MAXMECHTAB]; - crypto_mech_info_t *pd_mechanisms; - kcf_sched_info_t pd_sched_info; + const crypto_mech_info_t *pd_mechanisms; uint_t pd_mech_list_count; - // char *pd_name; - // uint_t pd_instance; - // int pd_module_id; - // struct modctl *pd_mctlp; kcondvar_t pd_remove_cv; - char *pd_description; - uint_t pd_flags; - uint_t pd_hash_limit; + const char *pd_description; crypto_kcf_provider_handle_t pd_kcf_prov_handle; crypto_provider_id_t pd_prov_id; - kstat_t *pd_kstat; - kcf_prov_stats_t pd_ks_data; } kcf_provider_desc_t; -/* useful for making a list of providers */ -typedef struct kcf_provider_list { - struct kcf_provider_list *pl_next; - struct kcf_provider_desc *pl_provider; -} kcf_provider_list_t; - /* atomic operations in linux implicitly form a memory barrier */ #define membar_exit() @@ -273,14 +163,6 @@ typedef struct kcf_provider_list { } -/* list of crypto_mech_info_t valid as the second mech in a dual operation */ - -typedef struct crypto_mech_info_list { - struct crypto_mech_info_list *ml_next; - crypto_mech_type_t ml_kcf_mechid; /* KCF's id */ - crypto_mech_info_t ml_mech_info; -} crypto_mech_info_list_t; - /* * An element in a mechanism provider descriptors chain. * The kcf_prov_mech_desc_t is duplicated in every chain the provider belongs @@ -292,15 +174,9 @@ typedef struct kcf_prov_mech_desc { struct kcf_mech_entry *pm_me; /* Back to the head */ struct kcf_prov_mech_desc *pm_next; /* Next in the chain */ crypto_mech_info_t pm_mech_info; /* Provider mech info */ - crypto_mech_info_list_t *pm_mi_list; /* list for duals */ kcf_provider_desc_t *pm_prov_desc; /* Common desc. */ } kcf_prov_mech_desc_t; -/* and the notation shortcuts ... */ -#define pm_provider_type pm_prov_desc.pd_provider_type -#define pm_provider_handle pm_prov_desc.pd_provider_handle -#define pm_ops_vector pm_prov_desc.pd_ops_vector - /* * A mechanism entry in an xxx_mech_tab[]. me_pad was deemed * to be unnecessary and removed. @@ -308,47 +184,10 @@ typedef struct kcf_prov_mech_desc { typedef struct kcf_mech_entry { crypto_mech_name_t me_name; /* mechanism name */ crypto_mech_type_t me_mechid; /* Internal id for mechanism */ - kmutex_t me_mutex; /* access protection */ - kcf_prov_mech_desc_t *me_hw_prov_chain; /* list of HW providers */ - kcf_prov_mech_desc_t *me_sw_prov; /* SW provider */ - /* - * Number of HW providers in the chain. There is only one - * SW provider. So, we need only a count of HW providers. - */ - int me_num_hwprov; - /* - * When a SW provider is present, this is the generation number that - * ensures no objects from old SW providers are used in the new one - */ - uint32_t me_gen_swprov; - /* - * threshold for using hardware providers for this mech - */ - size_t me_threshold; + kcf_prov_mech_desc_t *me_sw_prov; /* provider */ + avl_node_t me_node; } kcf_mech_entry_t; -/* - * A policy descriptor structure. It is allocated and initialized - * when administrative ioctls load disabled mechanisms. - * - * pd_prov_type: Provider type, hardware or software - * pd_name: Device name or module name. - * pd_instance: Device instance. - * pd_refcnt: Reference counter for this policy descriptor - * pd_mutex: Protects array and count of disabled mechanisms. - * pd_disabled_count: Count of disabled mechanisms. - * pd_disabled_mechs: Array of disabled mechanisms. - */ -typedef struct kcf_policy_desc { - crypto_provider_type_t pd_prov_type; - char *pd_name; - uint_t pd_instance; - uint_t pd_refcnt; - kmutex_t pd_mutex; - uint_t pd_disabled_count; - crypto_mech_name_t *pd_disabled_mechs; -} kcf_policy_desc_t; - /* * If a component has a reference to a kcf_policy_desc_t, * it REFHOLD()s. A new policy descriptor which is referenced only @@ -370,44 +209,29 @@ typedef struct kcf_policy_desc { kcf_policy_free_desc(desc); \ } -/* - * This entry stores the name of a software module and its - * mechanisms. The mechanisms are 'hints' that are used to - * trigger loading of the module. - */ -typedef struct kcf_soft_conf_entry { - struct kcf_soft_conf_entry *ce_next; - char *ce_name; - crypto_mech_name_t *ce_mechs; - uint_t ce_count; -} kcf_soft_conf_entry_t; - -extern kmutex_t soft_config_mutex; -extern kcf_soft_conf_entry_t *soft_config_list; - /* * Global tables. The sizes are from the predefined PKCS#11 v2.20 mechanisms, * with a margin of few extra empty entry points */ #define KCF_MAXDIGEST 16 /* Digests */ -#define KCF_MAXCIPHER 64 /* Ciphers */ +#define KCF_MAXCIPHER 32 /* Ciphers */ #define KCF_MAXMAC 40 /* Message authentication codes */ -#define KCF_MAXSIGN 24 /* Sign/Verify */ -#define KCF_MAXKEYOPS 116 /* Key generation and derivation */ -#define KCF_MAXMISC 16 /* Others ... */ + +_Static_assert(KCF_MAXCIPHER == KCF_MAXMECHTAB, + "KCF_MAXCIPHER != KCF_MAXMECHTAB"); /* See KCF_MAXMECHTAB comment */ typedef enum { KCF_DIGEST_CLASS = 1, KCF_CIPHER_CLASS, KCF_MAC_CLASS, - KCF_SIGN_CLASS, - KCF_KEYOPS_CLASS, - KCF_MISC_CLASS } kcf_ops_class_t; #define KCF_FIRST_OPSCLASS KCF_DIGEST_CLASS -#define KCF_LAST_OPSCLASS KCF_MISC_CLASS +#define KCF_LAST_OPSCLASS KCF_MAC_CLASS +_Static_assert( + KCF_OPS_CLASSSIZE == (KCF_LAST_OPSCLASS - KCF_FIRST_OPSCLASS + 2), + "KCF_OPS_CLASSSIZE doesn't match kcf_ops_class_t!"); /* The table of all the kcf_xxx_mech_tab[]s, indexed by kcf_ops_class */ @@ -423,7 +247,7 @@ extern const kcf_mech_entry_tab_t kcf_mech_tabs_tab[]; #define KCF_MECH2CLASS(mech_type) ((kcf_ops_class_t)((mech_type) >> 32)) -#define KCF_MECH2INDEX(mech_type) ((int)(mech_type)) +#define KCF_MECH2INDEX(mech_type) ((int)((mech_type) & 0xFFFFFFFF)) #define KCF_TO_PROV_MECH_INDX(pd, mech_type) \ ((pd)->pd_mech_indx[KCF_MECH2CLASS(mech_type)] \ @@ -435,58 +259,6 @@ extern const kcf_mech_entry_tab_t kcf_mech_tabs_tab[]; #define KCF_TO_PROV_MECHNUM(pd, mech_type) \ (KCF_TO_PROV_MECHINFO(pd, mech_type).cm_mech_number) -#define KCF_CAN_SHARE_OPSTATE(pd, mech_type) \ - ((KCF_TO_PROV_MECHINFO(pd, mech_type).cm_mech_flags) & \ - CRYPTO_CAN_SHARE_OPSTATE) - -/* ps_refcnt is protected by cm_lock in the crypto_minor structure */ -typedef struct crypto_provider_session { - struct crypto_provider_session *ps_next; - crypto_session_id_t ps_session; - kcf_provider_desc_t *ps_provider; - kcf_provider_desc_t *ps_real_provider; - uint_t ps_refcnt; -} crypto_provider_session_t; - -typedef struct crypto_session_data { - kmutex_t sd_lock; - kcondvar_t sd_cv; - uint32_t sd_flags; - int sd_pre_approved_amount; - crypto_ctx_t *sd_digest_ctx; - crypto_ctx_t *sd_encr_ctx; - crypto_ctx_t *sd_decr_ctx; - crypto_ctx_t *sd_sign_ctx; - crypto_ctx_t *sd_verify_ctx; - crypto_ctx_t *sd_sign_recover_ctx; - crypto_ctx_t *sd_verify_recover_ctx; - kcf_provider_desc_t *sd_provider; - void *sd_find_init_cookie; - crypto_provider_session_t *sd_provider_session; -} crypto_session_data_t; - -#define CRYPTO_SESSION_IN_USE 0x00000001 -#define CRYPTO_SESSION_IS_BUSY 0x00000002 -#define CRYPTO_SESSION_IS_CLOSED 0x00000004 - -#define KCF_MAX_PIN_LEN 1024 - -/* - * Per-minor info. - * - * cm_lock protects everything in this structure except for cm_refcnt. - */ -typedef struct crypto_minor { - uint_t cm_refcnt; - kmutex_t cm_lock; - kcondvar_t cm_cv; - crypto_session_data_t **cm_session_table; - uint_t cm_session_table_count; - kcf_provider_desc_t **cm_provider_array; - uint_t cm_provider_count; - crypto_provider_session_t *cm_provider_session; -} crypto_minor_t; - /* * Return codes for internal functions */ @@ -497,848 +269,119 @@ typedef struct crypto_minor { #define KCF_MECH_TAB_FULL 0x4 /* Need more room in the mech tabs. */ #define KCF_INVALID_INDX ((ushort_t)-1) -/* - * kCF internal mechanism and function group for tracking RNG providers. - */ -#define SUN_RANDOM "random" -#define CRYPTO_FG_RANDOM 0x80000000 /* generate_random() */ - /* * Wrappers for ops vectors. In the wrapper definitions below, the pd * argument always corresponds to a pointer to a provider descriptor * of type kcf_prov_desc_t. */ -#define KCF_PROV_CONTROL_OPS(pd) ((pd)->pd_ops_vector->co_control_ops) -#define KCF_PROV_CTX_OPS(pd) ((pd)->pd_ops_vector->co_ctx_ops) #define KCF_PROV_DIGEST_OPS(pd) ((pd)->pd_ops_vector->co_digest_ops) #define KCF_PROV_CIPHER_OPS(pd) ((pd)->pd_ops_vector->co_cipher_ops) #define KCF_PROV_MAC_OPS(pd) ((pd)->pd_ops_vector->co_mac_ops) -#define KCF_PROV_SIGN_OPS(pd) ((pd)->pd_ops_vector->co_sign_ops) -#define KCF_PROV_VERIFY_OPS(pd) ((pd)->pd_ops_vector->co_verify_ops) -#define KCF_PROV_DUAL_OPS(pd) ((pd)->pd_ops_vector->co_dual_ops) -#define KCF_PROV_DUAL_CIPHER_MAC_OPS(pd) \ - ((pd)->pd_ops_vector->co_dual_cipher_mac_ops) -#define KCF_PROV_RANDOM_OPS(pd) ((pd)->pd_ops_vector->co_random_ops) -#define KCF_PROV_SESSION_OPS(pd) ((pd)->pd_ops_vector->co_session_ops) -#define KCF_PROV_OBJECT_OPS(pd) ((pd)->pd_ops_vector->co_object_ops) -#define KCF_PROV_KEY_OPS(pd) ((pd)->pd_ops_vector->co_key_ops) -#define KCF_PROV_PROVIDER_OPS(pd) ((pd)->pd_ops_vector->co_provider_ops) -#define KCF_PROV_MECH_OPS(pd) ((pd)->pd_ops_vector->co_mech_ops) -#define KCF_PROV_NOSTORE_KEY_OPS(pd) \ - ((pd)->pd_ops_vector->co_nostore_key_ops) - -/* - * Wrappers for crypto_control_ops(9S) entry points. - */ - -#define KCF_PROV_STATUS(pd, status) ( \ - (KCF_PROV_CONTROL_OPS(pd) && \ - KCF_PROV_CONTROL_OPS(pd)->provider_status) ? \ - KCF_PROV_CONTROL_OPS(pd)->provider_status( \ - (pd)->pd_prov_handle, status) : \ - CRYPTO_NOT_SUPPORTED) - -/* - * Wrappers for crypto_ctx_ops(9S) entry points. - */ - -#define KCF_PROV_CREATE_CTX_TEMPLATE(pd, mech, key, template, size, req) ( \ - (KCF_PROV_CTX_OPS(pd) && KCF_PROV_CTX_OPS(pd)->create_ctx_template) ? \ - KCF_PROV_CTX_OPS(pd)->create_ctx_template( \ - (pd)->pd_prov_handle, mech, key, template, size, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_FREE_CONTEXT(pd, ctx) ( \ - (KCF_PROV_CTX_OPS(pd) && KCF_PROV_CTX_OPS(pd)->free_context) ? \ - KCF_PROV_CTX_OPS(pd)->free_context(ctx) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_COPYIN_MECH(pd, umech, kmech, errorp, mode) ( \ - (KCF_PROV_MECH_OPS(pd) && KCF_PROV_MECH_OPS(pd)->copyin_mechanism) ? \ - KCF_PROV_MECH_OPS(pd)->copyin_mechanism( \ - (pd)->pd_prov_handle, umech, kmech, errorp, mode) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_COPYOUT_MECH(pd, kmech, umech, errorp, mode) ( \ - (KCF_PROV_MECH_OPS(pd) && KCF_PROV_MECH_OPS(pd)->copyout_mechanism) ? \ - KCF_PROV_MECH_OPS(pd)->copyout_mechanism( \ - (pd)->pd_prov_handle, kmech, umech, errorp, mode) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_FREE_MECH(pd, prov_mech) ( \ - (KCF_PROV_MECH_OPS(pd) && KCF_PROV_MECH_OPS(pd)->free_mechanism) ? \ - KCF_PROV_MECH_OPS(pd)->free_mechanism( \ - (pd)->pd_prov_handle, prov_mech) : CRYPTO_NOT_SUPPORTED) +#define KCF_PROV_CTX_OPS(pd) ((pd)->pd_ops_vector->co_ctx_ops) /* * Wrappers for crypto_digest_ops(9S) entry points. */ -#define KCF_PROV_DIGEST_INIT(pd, ctx, mech, req) ( \ +#define KCF_PROV_DIGEST_INIT(pd, ctx, mech) ( \ (KCF_PROV_DIGEST_OPS(pd) && KCF_PROV_DIGEST_OPS(pd)->digest_init) ? \ - KCF_PROV_DIGEST_OPS(pd)->digest_init(ctx, mech, req) : \ - CRYPTO_NOT_SUPPORTED) - -/* - * The _ (underscore) in _digest is needed to avoid replacing the - * function digest(). - */ -#define KCF_PROV_DIGEST(pd, ctx, data, _digest, req) ( \ - (KCF_PROV_DIGEST_OPS(pd) && KCF_PROV_DIGEST_OPS(pd)->digest) ? \ - KCF_PROV_DIGEST_OPS(pd)->digest(ctx, data, _digest, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_DIGEST_UPDATE(pd, ctx, data, req) ( \ - (KCF_PROV_DIGEST_OPS(pd) && KCF_PROV_DIGEST_OPS(pd)->digest_update) ? \ - KCF_PROV_DIGEST_OPS(pd)->digest_update(ctx, data, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_DIGEST_KEY(pd, ctx, key, req) ( \ - (KCF_PROV_DIGEST_OPS(pd) && KCF_PROV_DIGEST_OPS(pd)->digest_key) ? \ - KCF_PROV_DIGEST_OPS(pd)->digest_key(ctx, key, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_DIGEST_FINAL(pd, ctx, digest, req) ( \ - (KCF_PROV_DIGEST_OPS(pd) && KCF_PROV_DIGEST_OPS(pd)->digest_final) ? \ - KCF_PROV_DIGEST_OPS(pd)->digest_final(ctx, digest, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_DIGEST_ATOMIC(pd, session, mech, data, digest, req) ( \ - (KCF_PROV_DIGEST_OPS(pd) && KCF_PROV_DIGEST_OPS(pd)->digest_atomic) ? \ - KCF_PROV_DIGEST_OPS(pd)->digest_atomic( \ - (pd)->pd_prov_handle, session, mech, data, digest, req) : \ + KCF_PROV_DIGEST_OPS(pd)->digest_init(ctx, mech) : \ CRYPTO_NOT_SUPPORTED) /* * Wrappers for crypto_cipher_ops(9S) entry points. */ -#define KCF_PROV_ENCRYPT_INIT(pd, ctx, mech, key, template, req) ( \ +#define KCF_PROV_ENCRYPT_INIT(pd, ctx, mech, key, template) ( \ (KCF_PROV_CIPHER_OPS(pd) && KCF_PROV_CIPHER_OPS(pd)->encrypt_init) ? \ - KCF_PROV_CIPHER_OPS(pd)->encrypt_init(ctx, mech, key, template, \ - req) : \ + KCF_PROV_CIPHER_OPS(pd)->encrypt_init(ctx, mech, key, template) : \ CRYPTO_NOT_SUPPORTED) -#define KCF_PROV_ENCRYPT(pd, ctx, plaintext, ciphertext, req) ( \ - (KCF_PROV_CIPHER_OPS(pd) && KCF_PROV_CIPHER_OPS(pd)->encrypt) ? \ - KCF_PROV_CIPHER_OPS(pd)->encrypt(ctx, plaintext, ciphertext, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_ENCRYPT_UPDATE(pd, ctx, plaintext, ciphertext, req) ( \ - (KCF_PROV_CIPHER_OPS(pd) && KCF_PROV_CIPHER_OPS(pd)->encrypt_update) ? \ - KCF_PROV_CIPHER_OPS(pd)->encrypt_update(ctx, plaintext, \ - ciphertext, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_ENCRYPT_FINAL(pd, ctx, ciphertext, req) ( \ - (KCF_PROV_CIPHER_OPS(pd) && KCF_PROV_CIPHER_OPS(pd)->encrypt_final) ? \ - KCF_PROV_CIPHER_OPS(pd)->encrypt_final(ctx, ciphertext, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_ENCRYPT_ATOMIC(pd, session, mech, key, plaintext, ciphertext, \ - template, req) ( \ +#define KCF_PROV_ENCRYPT_ATOMIC(pd, mech, key, plaintext, ciphertext, \ + template) ( \ (KCF_PROV_CIPHER_OPS(pd) && KCF_PROV_CIPHER_OPS(pd)->encrypt_atomic) ? \ KCF_PROV_CIPHER_OPS(pd)->encrypt_atomic( \ - (pd)->pd_prov_handle, session, mech, key, plaintext, ciphertext, \ - template, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_DECRYPT_INIT(pd, ctx, mech, key, template, req) ( \ - (KCF_PROV_CIPHER_OPS(pd) && KCF_PROV_CIPHER_OPS(pd)->decrypt_init) ? \ - KCF_PROV_CIPHER_OPS(pd)->decrypt_init(ctx, mech, key, template, \ - req) : \ + mech, key, plaintext, ciphertext, template) : \ CRYPTO_NOT_SUPPORTED) -#define KCF_PROV_DECRYPT(pd, ctx, ciphertext, plaintext, req) ( \ - (KCF_PROV_CIPHER_OPS(pd) && KCF_PROV_CIPHER_OPS(pd)->decrypt) ? \ - KCF_PROV_CIPHER_OPS(pd)->decrypt(ctx, ciphertext, plaintext, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_DECRYPT_UPDATE(pd, ctx, ciphertext, plaintext, req) ( \ - (KCF_PROV_CIPHER_OPS(pd) && KCF_PROV_CIPHER_OPS(pd)->decrypt_update) ? \ - KCF_PROV_CIPHER_OPS(pd)->decrypt_update(ctx, ciphertext, \ - plaintext, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_DECRYPT_FINAL(pd, ctx, plaintext, req) ( \ - (KCF_PROV_CIPHER_OPS(pd) && KCF_PROV_CIPHER_OPS(pd)->decrypt_final) ? \ - KCF_PROV_CIPHER_OPS(pd)->decrypt_final(ctx, plaintext, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_DECRYPT_ATOMIC(pd, session, mech, key, ciphertext, plaintext, \ - template, req) ( \ +#define KCF_PROV_DECRYPT_ATOMIC(pd, mech, key, ciphertext, plaintext, \ + template) ( \ (KCF_PROV_CIPHER_OPS(pd) && KCF_PROV_CIPHER_OPS(pd)->decrypt_atomic) ? \ KCF_PROV_CIPHER_OPS(pd)->decrypt_atomic( \ - (pd)->pd_prov_handle, session, mech, key, ciphertext, plaintext, \ - template, req) : \ + mech, key, ciphertext, plaintext, template) : \ CRYPTO_NOT_SUPPORTED) /* * Wrappers for crypto_mac_ops(9S) entry points. */ -#define KCF_PROV_MAC_INIT(pd, ctx, mech, key, template, req) ( \ +#define KCF_PROV_MAC_INIT(pd, ctx, mech, key, template) ( \ (KCF_PROV_MAC_OPS(pd) && KCF_PROV_MAC_OPS(pd)->mac_init) ? \ - KCF_PROV_MAC_OPS(pd)->mac_init(ctx, mech, key, template, req) \ + KCF_PROV_MAC_OPS(pd)->mac_init(ctx, mech, key, template) \ : CRYPTO_NOT_SUPPORTED) /* * The _ (underscore) in _mac is needed to avoid replacing the * function mac(). */ -#define KCF_PROV_MAC(pd, ctx, data, _mac, req) ( \ - (KCF_PROV_MAC_OPS(pd) && KCF_PROV_MAC_OPS(pd)->mac) ? \ - KCF_PROV_MAC_OPS(pd)->mac(ctx, data, _mac, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_MAC_UPDATE(pd, ctx, data, req) ( \ +#define KCF_PROV_MAC_UPDATE(pd, ctx, data) ( \ (KCF_PROV_MAC_OPS(pd) && KCF_PROV_MAC_OPS(pd)->mac_update) ? \ - KCF_PROV_MAC_OPS(pd)->mac_update(ctx, data, req) : \ + KCF_PROV_MAC_OPS(pd)->mac_update(ctx, data) : \ CRYPTO_NOT_SUPPORTED) -#define KCF_PROV_MAC_FINAL(pd, ctx, mac, req) ( \ +#define KCF_PROV_MAC_FINAL(pd, ctx, mac) ( \ (KCF_PROV_MAC_OPS(pd) && KCF_PROV_MAC_OPS(pd)->mac_final) ? \ - KCF_PROV_MAC_OPS(pd)->mac_final(ctx, mac, req) : \ + KCF_PROV_MAC_OPS(pd)->mac_final(ctx, mac) : \ CRYPTO_NOT_SUPPORTED) -#define KCF_PROV_MAC_ATOMIC(pd, session, mech, key, data, mac, template, \ - req) ( \ +#define KCF_PROV_MAC_ATOMIC(pd, mech, key, data, mac, template) ( \ (KCF_PROV_MAC_OPS(pd) && KCF_PROV_MAC_OPS(pd)->mac_atomic) ? \ KCF_PROV_MAC_OPS(pd)->mac_atomic( \ - (pd)->pd_prov_handle, session, mech, key, data, mac, template, \ - req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_MAC_VERIFY_ATOMIC(pd, session, mech, key, data, mac, \ - template, req) ( \ - (KCF_PROV_MAC_OPS(pd) && KCF_PROV_MAC_OPS(pd)->mac_verify_atomic) ? \ - KCF_PROV_MAC_OPS(pd)->mac_verify_atomic( \ - (pd)->pd_prov_handle, session, mech, key, data, mac, template, \ - req) : \ - CRYPTO_NOT_SUPPORTED) - -/* - * Wrappers for crypto_sign_ops(9S) entry points. - */ - -#define KCF_PROV_SIGN_INIT(pd, ctx, mech, key, template, req) ( \ - (KCF_PROV_SIGN_OPS(pd) && KCF_PROV_SIGN_OPS(pd)->sign_init) ? \ - KCF_PROV_SIGN_OPS(pd)->sign_init( \ - ctx, mech, key, template, req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_SIGN(pd, ctx, data, sig, req) ( \ - (KCF_PROV_SIGN_OPS(pd) && KCF_PROV_SIGN_OPS(pd)->sign) ? \ - KCF_PROV_SIGN_OPS(pd)->sign(ctx, data, sig, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_SIGN_UPDATE(pd, ctx, data, req) ( \ - (KCF_PROV_SIGN_OPS(pd) && KCF_PROV_SIGN_OPS(pd)->sign_update) ? \ - KCF_PROV_SIGN_OPS(pd)->sign_update(ctx, data, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_SIGN_FINAL(pd, ctx, sig, req) ( \ - (KCF_PROV_SIGN_OPS(pd) && KCF_PROV_SIGN_OPS(pd)->sign_final) ? \ - KCF_PROV_SIGN_OPS(pd)->sign_final(ctx, sig, req) : \ + mech, key, data, mac, template) : \ CRYPTO_NOT_SUPPORTED) -#define KCF_PROV_SIGN_ATOMIC(pd, session, mech, key, data, template, \ - sig, req) ( \ - (KCF_PROV_SIGN_OPS(pd) && KCF_PROV_SIGN_OPS(pd)->sign_atomic) ? \ - KCF_PROV_SIGN_OPS(pd)->sign_atomic( \ - (pd)->pd_prov_handle, session, mech, key, data, sig, template, \ - req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_SIGN_RECOVER_INIT(pd, ctx, mech, key, template, \ - req) ( \ - (KCF_PROV_SIGN_OPS(pd) && KCF_PROV_SIGN_OPS(pd)->sign_recover_init) ? \ - KCF_PROV_SIGN_OPS(pd)->sign_recover_init(ctx, mech, key, template, \ - req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_SIGN_RECOVER(pd, ctx, data, sig, req) ( \ - (KCF_PROV_SIGN_OPS(pd) && KCF_PROV_SIGN_OPS(pd)->sign_recover) ? \ - KCF_PROV_SIGN_OPS(pd)->sign_recover(ctx, data, sig, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_SIGN_RECOVER_ATOMIC(pd, session, mech, key, data, template, \ - sig, req) ( \ - (KCF_PROV_SIGN_OPS(pd) && \ - KCF_PROV_SIGN_OPS(pd)->sign_recover_atomic) ? \ - KCF_PROV_SIGN_OPS(pd)->sign_recover_atomic( \ - (pd)->pd_prov_handle, session, mech, key, data, sig, template, \ - req) : CRYPTO_NOT_SUPPORTED) - /* - * Wrappers for crypto_verify_ops(9S) entry points. - */ - -#define KCF_PROV_VERIFY_INIT(pd, ctx, mech, key, template, req) ( \ - (KCF_PROV_VERIFY_OPS(pd) && KCF_PROV_VERIFY_OPS(pd)->verify_init) ? \ - KCF_PROV_VERIFY_OPS(pd)->verify_init(ctx, mech, key, template, \ - req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_VERIFY(pd, ctx, data, sig, req) ( \ - (KCF_PROV_VERIFY_OPS(pd) && KCF_PROV_VERIFY_OPS(pd)->do_verify) ? \ - KCF_PROV_VERIFY_OPS(pd)->do_verify(ctx, data, sig, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_VERIFY_UPDATE(pd, ctx, data, req) ( \ - (KCF_PROV_VERIFY_OPS(pd) && KCF_PROV_VERIFY_OPS(pd)->verify_update) ? \ - KCF_PROV_VERIFY_OPS(pd)->verify_update(ctx, data, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_VERIFY_FINAL(pd, ctx, sig, req) ( \ - (KCF_PROV_VERIFY_OPS(pd) && KCF_PROV_VERIFY_OPS(pd)->verify_final) ? \ - KCF_PROV_VERIFY_OPS(pd)->verify_final(ctx, sig, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_VERIFY_ATOMIC(pd, session, mech, key, data, template, sig, \ - req) ( \ - (KCF_PROV_VERIFY_OPS(pd) && KCF_PROV_VERIFY_OPS(pd)->verify_atomic) ? \ - KCF_PROV_VERIFY_OPS(pd)->verify_atomic( \ - (pd)->pd_prov_handle, session, mech, key, data, sig, template, \ - req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_VERIFY_RECOVER_INIT(pd, ctx, mech, key, template, \ - req) ( \ - (KCF_PROV_VERIFY_OPS(pd) && \ - KCF_PROV_VERIFY_OPS(pd)->verify_recover_init) ? \ - KCF_PROV_VERIFY_OPS(pd)->verify_recover_init(ctx, mech, key, \ - template, req) : CRYPTO_NOT_SUPPORTED) - -/* verify_recover() CSPI routine has different argument order than verify() */ -#define KCF_PROV_VERIFY_RECOVER(pd, ctx, sig, data, req) ( \ - (KCF_PROV_VERIFY_OPS(pd) && KCF_PROV_VERIFY_OPS(pd)->verify_recover) ? \ - KCF_PROV_VERIFY_OPS(pd)->verify_recover(ctx, sig, data, req) : \ - CRYPTO_NOT_SUPPORTED) - -/* - * verify_recover_atomic() CSPI routine has different argument order - * than verify_atomic(). - */ -#define KCF_PROV_VERIFY_RECOVER_ATOMIC(pd, session, mech, key, sig, \ - template, data, req) ( \ - (KCF_PROV_VERIFY_OPS(pd) && \ - KCF_PROV_VERIFY_OPS(pd)->verify_recover_atomic) ? \ - KCF_PROV_VERIFY_OPS(pd)->verify_recover_atomic( \ - (pd)->pd_prov_handle, session, mech, key, sig, data, template, \ - req) : CRYPTO_NOT_SUPPORTED) - -/* - * Wrappers for crypto_dual_ops(9S) entry points. - */ - -#define KCF_PROV_DIGEST_ENCRYPT_UPDATE(digest_ctx, encrypt_ctx, plaintext, \ - ciphertext, req) ( \ - (KCF_PROV_DUAL_OPS(pd) && \ - KCF_PROV_DUAL_OPS(pd)->digest_encrypt_update) ? \ - KCF_PROV_DUAL_OPS(pd)->digest_encrypt_update( \ - digest_ctx, encrypt_ctx, plaintext, ciphertext, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_DECRYPT_DIGEST_UPDATE(decrypt_ctx, digest_ctx, ciphertext, \ - plaintext, req) ( \ - (KCF_PROV_DUAL_OPS(pd) && \ - KCF_PROV_DUAL_OPS(pd)->decrypt_digest_update) ? \ - KCF_PROV_DUAL_OPS(pd)->decrypt_digest_update( \ - decrypt_ctx, digest_ctx, ciphertext, plaintext, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_SIGN_ENCRYPT_UPDATE(sign_ctx, encrypt_ctx, plaintext, \ - ciphertext, req) ( \ - (KCF_PROV_DUAL_OPS(pd) && \ - KCF_PROV_DUAL_OPS(pd)->sign_encrypt_update) ? \ - KCF_PROV_DUAL_OPS(pd)->sign_encrypt_update( \ - sign_ctx, encrypt_ctx, plaintext, ciphertext, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_DECRYPT_VERIFY_UPDATE(decrypt_ctx, verify_ctx, ciphertext, \ - plaintext, req) ( \ - (KCF_PROV_DUAL_OPS(pd) && \ - KCF_PROV_DUAL_OPS(pd)->decrypt_verify_update) ? \ - KCF_PROV_DUAL_OPS(pd)->decrypt_verify_update( \ - decrypt_ctx, verify_ctx, ciphertext, plaintext, req) : \ - CRYPTO_NOT_SUPPORTED) - -/* - * Wrappers for crypto_dual_cipher_mac_ops(9S) entry points. - */ - -#define KCF_PROV_ENCRYPT_MAC_INIT(pd, ctx, encr_mech, encr_key, mac_mech, \ - mac_key, encr_ctx_template, mac_ctx_template, req) ( \ - (KCF_PROV_DUAL_CIPHER_MAC_OPS(pd) && \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->encrypt_mac_init) ? \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->encrypt_mac_init( \ - ctx, encr_mech, encr_key, mac_mech, mac_key, encr_ctx_template, \ - mac_ctx_template, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_ENCRYPT_MAC(pd, ctx, plaintext, ciphertext, mac, req) ( \ - (KCF_PROV_DUAL_CIPHER_MAC_OPS(pd) && \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->encrypt_mac) ? \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->encrypt_mac( \ - ctx, plaintext, ciphertext, mac, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_ENCRYPT_MAC_UPDATE(pd, ctx, plaintext, ciphertext, req) ( \ - (KCF_PROV_DUAL_CIPHER_MAC_OPS(pd) && \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->encrypt_mac_update) ? \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->encrypt_mac_update( \ - ctx, plaintext, ciphertext, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_ENCRYPT_MAC_FINAL(pd, ctx, ciphertext, mac, req) ( \ - (KCF_PROV_DUAL_CIPHER_MAC_OPS(pd) && \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->encrypt_mac_final) ? \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->encrypt_mac_final( \ - ctx, ciphertext, mac, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_ENCRYPT_MAC_ATOMIC(pd, session, encr_mech, encr_key, \ - mac_mech, mac_key, plaintext, ciphertext, mac, \ - encr_ctx_template, mac_ctx_template, req) ( \ - (KCF_PROV_DUAL_CIPHER_MAC_OPS(pd) && \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->encrypt_mac_atomic) ? \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->encrypt_mac_atomic( \ - (pd)->pd_prov_handle, session, encr_mech, encr_key, \ - mac_mech, mac_key, plaintext, ciphertext, mac, \ - encr_ctx_template, mac_ctx_template, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_MAC_DECRYPT_INIT(pd, ctx, mac_mech, mac_key, decr_mech, \ - decr_key, mac_ctx_template, decr_ctx_template, req) ( \ - (KCF_PROV_DUAL_CIPHER_MAC_OPS(pd) && \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->mac_decrypt_init) ? \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->mac_decrypt_init( \ - ctx, mac_mech, mac_key, decr_mech, decr_key, mac_ctx_template, \ - decr_ctx_template, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_MAC_DECRYPT(pd, ctx, ciphertext, mac, plaintext, req) ( \ - (KCF_PROV_DUAL_CIPHER_MAC_OPS(pd) && \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->mac_decrypt) ? \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->mac_decrypt( \ - ctx, ciphertext, mac, plaintext, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_MAC_DECRYPT_UPDATE(pd, ctx, ciphertext, plaintext, req) ( \ - (KCF_PROV_DUAL_CIPHER_MAC_OPS(pd) && \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->mac_decrypt_update) ? \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->mac_decrypt_update( \ - ctx, ciphertext, plaintext, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_MAC_DECRYPT_FINAL(pd, ctx, mac, plaintext, req) ( \ - (KCF_PROV_DUAL_CIPHER_MAC_OPS(pd) && \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->mac_decrypt_final) ? \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->mac_decrypt_final( \ - ctx, mac, plaintext, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_MAC_DECRYPT_ATOMIC(pd, session, mac_mech, mac_key, \ - decr_mech, decr_key, ciphertext, mac, plaintext, \ - mac_ctx_template, decr_ctx_template, req) ( \ - (KCF_PROV_DUAL_CIPHER_MAC_OPS(pd) && \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->mac_decrypt_atomic) ? \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->mac_decrypt_atomic( \ - (pd)->pd_prov_handle, session, mac_mech, mac_key, \ - decr_mech, decr_key, ciphertext, mac, plaintext, \ - mac_ctx_template, decr_ctx_template, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_MAC_VERIFY_DECRYPT_ATOMIC(pd, session, mac_mech, mac_key, \ - decr_mech, decr_key, ciphertext, mac, plaintext, \ - mac_ctx_template, decr_ctx_template, req) ( \ - (KCF_PROV_DUAL_CIPHER_MAC_OPS(pd) && \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->mac_verify_decrypt_atomic \ - != NULL) ? \ - KCF_PROV_DUAL_CIPHER_MAC_OPS(pd)->mac_verify_decrypt_atomic( \ - (pd)->pd_prov_handle, session, mac_mech, mac_key, \ - decr_mech, decr_key, ciphertext, mac, plaintext, \ - mac_ctx_template, decr_ctx_template, req) : \ - CRYPTO_NOT_SUPPORTED) - -/* - * Wrappers for crypto_random_number_ops(9S) entry points. - */ - -#define KCF_PROV_SEED_RANDOM(pd, session, buf, len, est, flags, req) ( \ - (KCF_PROV_RANDOM_OPS(pd) && KCF_PROV_RANDOM_OPS(pd)->seed_random) ? \ - KCF_PROV_RANDOM_OPS(pd)->seed_random((pd)->pd_prov_handle, \ - session, buf, len, est, flags, req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_GENERATE_RANDOM(pd, session, buf, len, req) ( \ - (KCF_PROV_RANDOM_OPS(pd) && \ - KCF_PROV_RANDOM_OPS(pd)->generate_random) ? \ - KCF_PROV_RANDOM_OPS(pd)->generate_random((pd)->pd_prov_handle, \ - session, buf, len, req) : CRYPTO_NOT_SUPPORTED) - -/* - * Wrappers for crypto_session_ops(9S) entry points. - * - * ops_pd is the provider descriptor that supplies the ops_vector. - * pd is the descriptor that supplies the provider handle. - * Only session open/close needs two handles. - */ - -#define KCF_PROV_SESSION_OPEN(ops_pd, session, req, pd) ( \ - (KCF_PROV_SESSION_OPS(ops_pd) && \ - KCF_PROV_SESSION_OPS(ops_pd)->session_open) ? \ - KCF_PROV_SESSION_OPS(ops_pd)->session_open((pd)->pd_prov_handle, \ - session, req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_SESSION_CLOSE(ops_pd, session, req, pd) ( \ - (KCF_PROV_SESSION_OPS(ops_pd) && \ - KCF_PROV_SESSION_OPS(ops_pd)->session_close) ? \ - KCF_PROV_SESSION_OPS(ops_pd)->session_close((pd)->pd_prov_handle, \ - session, req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_SESSION_LOGIN(pd, session, user_type, pin, len, req) ( \ - (KCF_PROV_SESSION_OPS(pd) && \ - KCF_PROV_SESSION_OPS(pd)->session_login) ? \ - KCF_PROV_SESSION_OPS(pd)->session_login((pd)->pd_prov_handle, \ - session, user_type, pin, len, req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_SESSION_LOGOUT(pd, session, req) ( \ - (KCF_PROV_SESSION_OPS(pd) && \ - KCF_PROV_SESSION_OPS(pd)->session_logout) ? \ - KCF_PROV_SESSION_OPS(pd)->session_logout((pd)->pd_prov_handle, \ - session, req) : CRYPTO_NOT_SUPPORTED) - -/* - * Wrappers for crypto_object_ops(9S) entry points. - */ - -#define KCF_PROV_OBJECT_CREATE(pd, session, template, count, object, req) ( \ - (KCF_PROV_OBJECT_OPS(pd) && KCF_PROV_OBJECT_OPS(pd)->object_create) ? \ - KCF_PROV_OBJECT_OPS(pd)->object_create((pd)->pd_prov_handle, \ - session, template, count, object, req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_OBJECT_COPY(pd, session, object, template, count, \ - new_object, req) ( \ - (KCF_PROV_OBJECT_OPS(pd) && KCF_PROV_OBJECT_OPS(pd)->object_copy) ? \ - KCF_PROV_OBJECT_OPS(pd)->object_copy((pd)->pd_prov_handle, \ - session, object, template, count, new_object, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_OBJECT_DESTROY(pd, session, object, req) ( \ - (KCF_PROV_OBJECT_OPS(pd) && KCF_PROV_OBJECT_OPS(pd)->object_destroy) ? \ - KCF_PROV_OBJECT_OPS(pd)->object_destroy((pd)->pd_prov_handle, \ - session, object, req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_OBJECT_GET_SIZE(pd, session, object, size, req) ( \ - (KCF_PROV_OBJECT_OPS(pd) && \ - KCF_PROV_OBJECT_OPS(pd)->object_get_size) ? \ - KCF_PROV_OBJECT_OPS(pd)->object_get_size((pd)->pd_prov_handle, \ - session, object, size, req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_OBJECT_GET_ATTRIBUTE_VALUE(pd, session, object, template, \ - count, req) ( \ - (KCF_PROV_OBJECT_OPS(pd) && \ - KCF_PROV_OBJECT_OPS(pd)->object_get_attribute_value) ? \ - KCF_PROV_OBJECT_OPS(pd)->object_get_attribute_value( \ - (pd)->pd_prov_handle, session, object, template, count, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_OBJECT_SET_ATTRIBUTE_VALUE(pd, session, object, template, \ - count, req) ( \ - (KCF_PROV_OBJECT_OPS(pd) && \ - KCF_PROV_OBJECT_OPS(pd)->object_set_attribute_value) ? \ - KCF_PROV_OBJECT_OPS(pd)->object_set_attribute_value( \ - (pd)->pd_prov_handle, session, object, template, count, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_OBJECT_FIND_INIT(pd, session, template, count, ppriv, \ - req) ( \ - (KCF_PROV_OBJECT_OPS(pd) && \ - KCF_PROV_OBJECT_OPS(pd)->object_find_init) ? \ - KCF_PROV_OBJECT_OPS(pd)->object_find_init((pd)->pd_prov_handle, \ - session, template, count, ppriv, req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_OBJECT_FIND(pd, ppriv, objects, max_objects, object_count, \ - req) ( \ - (KCF_PROV_OBJECT_OPS(pd) && KCF_PROV_OBJECT_OPS(pd)->object_find) ? \ - KCF_PROV_OBJECT_OPS(pd)->object_find( \ - (pd)->pd_prov_handle, ppriv, objects, max_objects, object_count, \ - req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_OBJECT_FIND_FINAL(pd, ppriv, req) ( \ - (KCF_PROV_OBJECT_OPS(pd) && \ - KCF_PROV_OBJECT_OPS(pd)->object_find_final) ? \ - KCF_PROV_OBJECT_OPS(pd)->object_find_final( \ - (pd)->pd_prov_handle, ppriv, req) : CRYPTO_NOT_SUPPORTED) - -/* - * Wrappers for crypto_key_ops(9S) entry points. + * Wrappers for crypto_ctx_ops(9S) entry points. */ -#define KCF_PROV_KEY_GENERATE(pd, session, mech, template, count, object, \ - req) ( \ - (KCF_PROV_KEY_OPS(pd) && KCF_PROV_KEY_OPS(pd)->key_generate) ? \ - KCF_PROV_KEY_OPS(pd)->key_generate((pd)->pd_prov_handle, \ - session, mech, template, count, object, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_KEY_GENERATE_PAIR(pd, session, mech, pub_template, \ - pub_count, priv_template, priv_count, pub_key, priv_key, req) ( \ - (KCF_PROV_KEY_OPS(pd) && KCF_PROV_KEY_OPS(pd)->key_generate_pair) ? \ - KCF_PROV_KEY_OPS(pd)->key_generate_pair((pd)->pd_prov_handle, \ - session, mech, pub_template, pub_count, priv_template, \ - priv_count, pub_key, priv_key, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_KEY_WRAP(pd, session, mech, wrapping_key, key, wrapped_key, \ - wrapped_key_len, req) ( \ - (KCF_PROV_KEY_OPS(pd) && KCF_PROV_KEY_OPS(pd)->key_wrap) ? \ - KCF_PROV_KEY_OPS(pd)->key_wrap((pd)->pd_prov_handle, \ - session, mech, wrapping_key, key, wrapped_key, wrapped_key_len, \ - req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_KEY_UNWRAP(pd, session, mech, unwrapping_key, wrapped_key, \ - wrapped_key_len, template, count, key, req) ( \ - (KCF_PROV_KEY_OPS(pd) && KCF_PROV_KEY_OPS(pd)->key_unwrap) ? \ - KCF_PROV_KEY_OPS(pd)->key_unwrap((pd)->pd_prov_handle, \ - session, mech, unwrapping_key, wrapped_key, wrapped_key_len, \ - template, count, key, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_KEY_DERIVE(pd, session, mech, base_key, template, count, \ - key, req) ( \ - (KCF_PROV_KEY_OPS(pd) && KCF_PROV_KEY_OPS(pd)->key_derive) ? \ - KCF_PROV_KEY_OPS(pd)->key_derive((pd)->pd_prov_handle, \ - session, mech, base_key, template, count, key, req) : \ - CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_KEY_CHECK(pd, mech, key) ( \ - (KCF_PROV_KEY_OPS(pd) && KCF_PROV_KEY_OPS(pd)->key_check) ? \ - KCF_PROV_KEY_OPS(pd)->key_check((pd)->pd_prov_handle, mech, key) : \ +#define KCF_PROV_CREATE_CTX_TEMPLATE(pd, mech, key, template, size) ( \ + (KCF_PROV_CTX_OPS(pd) && KCF_PROV_CTX_OPS(pd)->create_ctx_template) ? \ + KCF_PROV_CTX_OPS(pd)->create_ctx_template( \ + mech, key, template, size) : \ CRYPTO_NOT_SUPPORTED) -/* - * Wrappers for crypto_provider_management_ops(9S) entry points. - * - * ops_pd is the provider descriptor that supplies the ops_vector. - * pd is the descriptor that supplies the provider handle. - * Only ext_info needs two handles. - */ - -#define KCF_PROV_EXT_INFO(ops_pd, provext_info, req, pd) ( \ - (KCF_PROV_PROVIDER_OPS(ops_pd) && \ - KCF_PROV_PROVIDER_OPS(ops_pd)->ext_info) ? \ - KCF_PROV_PROVIDER_OPS(ops_pd)->ext_info((pd)->pd_prov_handle, \ - provext_info, req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_INIT_TOKEN(pd, pin, pin_len, label, req) ( \ - (KCF_PROV_PROVIDER_OPS(pd) && KCF_PROV_PROVIDER_OPS(pd)->init_token) ? \ - KCF_PROV_PROVIDER_OPS(pd)->init_token((pd)->pd_prov_handle, \ - pin, pin_len, label, req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_INIT_PIN(pd, session, pin, pin_len, req) ( \ - (KCF_PROV_PROVIDER_OPS(pd) && KCF_PROV_PROVIDER_OPS(pd)->init_pin) ? \ - KCF_PROV_PROVIDER_OPS(pd)->init_pin((pd)->pd_prov_handle, \ - session, pin, pin_len, req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_SET_PIN(pd, session, old_pin, old_len, new_pin, new_len, \ - req) ( \ - (KCF_PROV_PROVIDER_OPS(pd) && KCF_PROV_PROVIDER_OPS(pd)->set_pin) ? \ - KCF_PROV_PROVIDER_OPS(pd)->set_pin((pd)->pd_prov_handle, \ - session, old_pin, old_len, new_pin, new_len, req) : \ - CRYPTO_NOT_SUPPORTED) - -/* - * Wrappers for crypto_nostore_key_ops(9S) entry points. - */ - -#define KCF_PROV_NOSTORE_KEY_GENERATE(pd, session, mech, template, count, \ - out_template, out_count, req) ( \ - (KCF_PROV_NOSTORE_KEY_OPS(pd) && \ - KCF_PROV_NOSTORE_KEY_OPS(pd)->nostore_key_generate) ? \ - KCF_PROV_NOSTORE_KEY_OPS(pd)->nostore_key_generate( \ - (pd)->pd_prov_handle, session, mech, template, count, \ - out_template, out_count, req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_NOSTORE_KEY_GENERATE_PAIR(pd, session, mech, pub_template, \ - pub_count, priv_template, priv_count, out_pub_template, \ - out_pub_count, out_priv_template, out_priv_count, req) ( \ - (KCF_PROV_NOSTORE_KEY_OPS(pd) && \ - KCF_PROV_NOSTORE_KEY_OPS(pd)->nostore_key_generate_pair) ? \ - KCF_PROV_NOSTORE_KEY_OPS(pd)->nostore_key_generate_pair( \ - (pd)->pd_prov_handle, session, mech, pub_template, pub_count, \ - priv_template, priv_count, out_pub_template, out_pub_count, \ - out_priv_template, out_priv_count, req) : CRYPTO_NOT_SUPPORTED) - -#define KCF_PROV_NOSTORE_KEY_DERIVE(pd, session, mech, base_key, template, \ - count, out_template, out_count, req) ( \ - (KCF_PROV_NOSTORE_KEY_OPS(pd) && \ - KCF_PROV_NOSTORE_KEY_OPS(pd)->nostore_key_derive) ? \ - KCF_PROV_NOSTORE_KEY_OPS(pd)->nostore_key_derive( \ - (pd)->pd_prov_handle, session, mech, base_key, template, count, \ - out_template, out_count, req) : CRYPTO_NOT_SUPPORTED) - -/* - * The following routines are exported by the kcf module (/kernel/misc/kcf) - * to the crypto and cryptoadmin modules. - */ - -/* Digest/mac/cipher entry points that take a provider descriptor and session */ -extern int crypto_digest_single(crypto_context_t, crypto_data_t *, - crypto_data_t *, crypto_call_req_t *); - -extern int crypto_mac_single(crypto_context_t, crypto_data_t *, - crypto_data_t *, crypto_call_req_t *); - -extern int crypto_encrypt_single(crypto_context_t, crypto_data_t *, - crypto_data_t *, crypto_call_req_t *); - -extern int crypto_decrypt_single(crypto_context_t, crypto_data_t *, - crypto_data_t *, crypto_call_req_t *); - - -/* Other private digest/mac/cipher entry points not exported through k-API */ -extern int crypto_digest_key_prov(crypto_context_t, crypto_key_t *, - crypto_call_req_t *); - -/* Private sign entry points exported by KCF */ -extern int crypto_sign_single(crypto_context_t, crypto_data_t *, - crypto_data_t *, crypto_call_req_t *); - -extern int crypto_sign_recover_single(crypto_context_t, crypto_data_t *, - crypto_data_t *, crypto_call_req_t *); - -/* Private verify entry points exported by KCF */ -extern int crypto_verify_single(crypto_context_t, crypto_data_t *, - crypto_data_t *, crypto_call_req_t *); - -extern int crypto_verify_recover_single(crypto_context_t, crypto_data_t *, - crypto_data_t *, crypto_call_req_t *); - -/* Private dual operations entry points exported by KCF */ -extern int crypto_digest_encrypt_update(crypto_context_t, crypto_context_t, - crypto_data_t *, crypto_data_t *, crypto_call_req_t *); -extern int crypto_decrypt_digest_update(crypto_context_t, crypto_context_t, - crypto_data_t *, crypto_data_t *, crypto_call_req_t *); -extern int crypto_sign_encrypt_update(crypto_context_t, crypto_context_t, - crypto_data_t *, crypto_data_t *, crypto_call_req_t *); -extern int crypto_decrypt_verify_update(crypto_context_t, crypto_context_t, - crypto_data_t *, crypto_data_t *, crypto_call_req_t *); - -/* Random Number Generation */ -int crypto_seed_random(crypto_provider_handle_t provider, uchar_t *buf, - size_t len, crypto_call_req_t *req); -int crypto_generate_random(crypto_provider_handle_t provider, uchar_t *buf, - size_t len, crypto_call_req_t *req); - -/* Provider Management */ -int crypto_get_provider_info(crypto_provider_id_t id, - crypto_provider_info_t **info, crypto_call_req_t *req); -int crypto_get_provider_mechanisms(crypto_minor_t *, crypto_provider_id_t id, - uint_t *count, crypto_mech_name_t **list); -int crypto_init_token(crypto_provider_handle_t provider, char *pin, - size_t pin_len, char *label, crypto_call_req_t *); -int crypto_init_pin(crypto_provider_handle_t provider, char *pin, - size_t pin_len, crypto_call_req_t *req); -int crypto_set_pin(crypto_provider_handle_t provider, char *old_pin, - size_t old_len, char *new_pin, size_t new_len, crypto_call_req_t *req); -void crypto_free_provider_list(crypto_provider_entry_t *list, uint_t count); -void crypto_free_provider_info(crypto_provider_info_t *info); +#define KCF_PROV_FREE_CONTEXT(pd, ctx) ( \ + (KCF_PROV_CTX_OPS(pd) && KCF_PROV_CTX_OPS(pd)->free_context) ? \ + KCF_PROV_CTX_OPS(pd)->free_context(ctx) : CRYPTO_NOT_SUPPORTED) -/* Administrative */ -int crypto_get_dev_list(uint_t *count, crypto_dev_list_entry_t **list); -int crypto_get_soft_list(uint_t *count, char **list, size_t *len); -int crypto_get_dev_info(char *name, uint_t instance, uint_t *count, - crypto_mech_name_t **list); -int crypto_get_soft_info(caddr_t name, uint_t *count, - crypto_mech_name_t **list); -int crypto_load_dev_disabled(char *name, uint_t instance, uint_t count, - crypto_mech_name_t *list); -int crypto_load_soft_disabled(caddr_t name, uint_t count, - crypto_mech_name_t *list); -int crypto_unload_soft_module(caddr_t path); -int crypto_load_soft_config(caddr_t name, uint_t count, - crypto_mech_name_t *list); -int crypto_load_door(uint_t did); -void crypto_free_mech_list(crypto_mech_name_t *list, uint_t count); -void crypto_free_dev_list(crypto_dev_list_entry_t *list, uint_t count); /* Miscellaneous */ -int crypto_get_mechanism_number(caddr_t name, crypto_mech_type_t *number); -int crypto_build_permitted_mech_names(kcf_provider_desc_t *, - crypto_mech_name_t **, uint_t *, int); extern void kcf_destroy_mech_tabs(void); extern void kcf_init_mech_tabs(void); extern int kcf_add_mech_provider(short, kcf_provider_desc_t *, kcf_prov_mech_desc_t **); -extern void kcf_remove_mech_provider(char *, kcf_provider_desc_t *); +extern void kcf_remove_mech_provider(const char *, kcf_provider_desc_t *); extern int kcf_get_mech_entry(crypto_mech_type_t, kcf_mech_entry_t **); -extern kcf_provider_desc_t *kcf_alloc_provider_desc( - const crypto_provider_info_t *); +extern kcf_provider_desc_t *kcf_alloc_provider_desc(void); extern void kcf_provider_zero_refcnt(kcf_provider_desc_t *); extern void kcf_free_provider_desc(kcf_provider_desc_t *); -extern void kcf_soft_config_init(void); -extern int get_sw_provider_for_mech(crypto_mech_name_t, char **); -extern crypto_mech_type_t crypto_mech2id_common(const char *, boolean_t); extern void undo_register_provider(kcf_provider_desc_t *, boolean_t); -extern void redo_register_provider(kcf_provider_desc_t *); -extern void kcf_rnd_init(void); -extern boolean_t kcf_rngprov_check(void); -extern int kcf_rnd_get_pseudo_bytes(uint8_t *, size_t); -extern int kcf_rnd_get_bytes(uint8_t *, size_t, boolean_t, boolean_t); -extern int random_add_pseudo_entropy(uint8_t *, size_t, uint_t); -extern void kcf_rnd_schedule_timeout(boolean_t); -extern int crypto_uio_data(crypto_data_t *, uchar_t *, int, cmd_type_t, - void *, void (*update)(void)); -extern int crypto_mblk_data(crypto_data_t *, uchar_t *, int, cmd_type_t, - void *, void (*update)(void)); extern int crypto_put_output_data(uchar_t *, crypto_data_t *, int); -extern int crypto_get_input_data(crypto_data_t *, uchar_t **, uchar_t *); -extern int crypto_copy_key_to_ctx(crypto_key_t *, crypto_key_t **, size_t *, - int kmflag); -extern int crypto_digest_data(crypto_data_t *, void *, uchar_t *, - void (*update)(void), void (*final)(void), uchar_t); extern int crypto_update_iov(void *, crypto_data_t *, crypto_data_t *, - int (*cipher)(void *, caddr_t, size_t, crypto_data_t *), - void (*copy_block)(uint8_t *, uint64_t *)); + int (*cipher)(void *, caddr_t, size_t, crypto_data_t *)); extern int crypto_update_uio(void *, crypto_data_t *, crypto_data_t *, - int (*cipher)(void *, caddr_t, size_t, crypto_data_t *), - void (*copy_block)(uint8_t *, uint64_t *)); -extern int crypto_update_mp(void *, crypto_data_t *, crypto_data_t *, - int (*cipher)(void *, caddr_t, size_t, crypto_data_t *), - void (*copy_block)(uint8_t *, uint64_t *)); -extern int crypto_get_key_attr(crypto_key_t *, crypto_attr_type_t, uchar_t **, - ssize_t *); + int (*cipher)(void *, caddr_t, size_t, crypto_data_t *)); /* Access to the provider's table */ extern void kcf_prov_tab_destroy(void); extern void kcf_prov_tab_init(void); extern int kcf_prov_tab_add_provider(kcf_provider_desc_t *); extern int kcf_prov_tab_rem_provider(crypto_provider_id_t); -extern kcf_provider_desc_t *kcf_prov_tab_lookup_by_name(char *); -extern kcf_provider_desc_t *kcf_prov_tab_lookup_by_dev(char *, uint_t); -extern int kcf_get_hw_prov_tab(uint_t *, kcf_provider_desc_t ***, int, - char *, uint_t, boolean_t); -extern int kcf_get_slot_list(uint_t *, kcf_provider_desc_t ***, boolean_t); -extern void kcf_free_provider_tab(uint_t, kcf_provider_desc_t **); extern kcf_provider_desc_t *kcf_prov_tab_lookup(crypto_provider_id_t); extern int kcf_get_sw_prov(crypto_mech_type_t, kcf_provider_desc_t **, kcf_mech_entry_t **, boolean_t); -/* Access to the policy table */ -extern boolean_t is_mech_disabled(kcf_provider_desc_t *, crypto_mech_name_t); -extern boolean_t is_mech_disabled_byname(crypto_provider_type_t, char *, - uint_t, crypto_mech_name_t); -extern void kcf_policy_tab_init(void); -extern void kcf_policy_free_desc(kcf_policy_desc_t *); -extern void kcf_policy_remove_by_name(char *, uint_t *, crypto_mech_name_t **); -extern void kcf_policy_remove_by_dev(char *, uint_t, uint_t *, - crypto_mech_name_t **); -extern kcf_policy_desc_t *kcf_policy_lookup_by_name(char *); -extern kcf_policy_desc_t *kcf_policy_lookup_by_dev(char *, uint_t); -extern int kcf_policy_load_soft_disabled(char *, uint_t, crypto_mech_name_t *, - uint_t *, crypto_mech_name_t **); -extern int kcf_policy_load_dev_disabled(char *, uint_t, uint_t, - crypto_mech_name_t *, uint_t *, crypto_mech_name_t **); -extern boolean_t in_soft_config_list(char *); - #ifdef __cplusplus } diff --git a/module/icp/include/sys/crypto/ops_impl.h b/module/icp/include/sys/crypto/ops_impl.h deleted file mode 100644 index 230d74b063fc..000000000000 --- a/module/icp/include/sys/crypto/ops_impl.h +++ /dev/null @@ -1,630 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License (the "License"). - * You may not use this file except in compliance with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2008 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#ifndef _SYS_CRYPTO_OPS_IMPL_H -#define _SYS_CRYPTO_OPS_IMPL_H - -/* - * Scheduler internal structures. - */ - -#ifdef __cplusplus -extern "C" { -#endif - -#include -#include -#include -#include -#include - -/* - * The parameters needed for each function group are batched - * in one structure. This is much simpler than having a - * separate structure for each function. - * - * In some cases, a field is generically named to keep the - * structure small. The comments indicate these cases. - */ -typedef struct kcf_digest_ops_params { - crypto_session_id_t do_sid; - crypto_mech_type_t do_framework_mechtype; - crypto_mechanism_t do_mech; - crypto_data_t *do_data; - crypto_data_t *do_digest; - crypto_key_t *do_digest_key; /* Argument for digest_key() */ -} kcf_digest_ops_params_t; - -typedef struct kcf_mac_ops_params { - crypto_session_id_t mo_sid; - crypto_mech_type_t mo_framework_mechtype; - crypto_mechanism_t mo_mech; - crypto_key_t *mo_key; - crypto_data_t *mo_data; - crypto_data_t *mo_mac; - crypto_spi_ctx_template_t mo_templ; -} kcf_mac_ops_params_t; - -typedef struct kcf_encrypt_ops_params { - crypto_session_id_t eo_sid; - crypto_mech_type_t eo_framework_mechtype; - crypto_mechanism_t eo_mech; - crypto_key_t *eo_key; - crypto_data_t *eo_plaintext; - crypto_data_t *eo_ciphertext; - crypto_spi_ctx_template_t eo_templ; -} kcf_encrypt_ops_params_t; - -typedef struct kcf_decrypt_ops_params { - crypto_session_id_t dop_sid; - crypto_mech_type_t dop_framework_mechtype; - crypto_mechanism_t dop_mech; - crypto_key_t *dop_key; - crypto_data_t *dop_ciphertext; - crypto_data_t *dop_plaintext; - crypto_spi_ctx_template_t dop_templ; -} kcf_decrypt_ops_params_t; - -typedef struct kcf_sign_ops_params { - crypto_session_id_t so_sid; - crypto_mech_type_t so_framework_mechtype; - crypto_mechanism_t so_mech; - crypto_key_t *so_key; - crypto_data_t *so_data; - crypto_data_t *so_signature; - crypto_spi_ctx_template_t so_templ; -} kcf_sign_ops_params_t; - -typedef struct kcf_verify_ops_params { - crypto_session_id_t vo_sid; - crypto_mech_type_t vo_framework_mechtype; - crypto_mechanism_t vo_mech; - crypto_key_t *vo_key; - crypto_data_t *vo_data; - crypto_data_t *vo_signature; - crypto_spi_ctx_template_t vo_templ; -} kcf_verify_ops_params_t; - -typedef struct kcf_encrypt_mac_ops_params { - crypto_session_id_t em_sid; - crypto_mech_type_t em_framework_encr_mechtype; - crypto_mechanism_t em_encr_mech; - crypto_key_t *em_encr_key; - crypto_mech_type_t em_framework_mac_mechtype; - crypto_mechanism_t em_mac_mech; - crypto_key_t *em_mac_key; - crypto_data_t *em_plaintext; - crypto_dual_data_t *em_ciphertext; - crypto_data_t *em_mac; - crypto_spi_ctx_template_t em_encr_templ; - crypto_spi_ctx_template_t em_mac_templ; -} kcf_encrypt_mac_ops_params_t; - -typedef struct kcf_mac_decrypt_ops_params { - crypto_session_id_t md_sid; - crypto_mech_type_t md_framework_mac_mechtype; - crypto_mechanism_t md_mac_mech; - crypto_key_t *md_mac_key; - crypto_mech_type_t md_framework_decr_mechtype; - crypto_mechanism_t md_decr_mech; - crypto_key_t *md_decr_key; - crypto_dual_data_t *md_ciphertext; - crypto_data_t *md_mac; - crypto_data_t *md_plaintext; - crypto_spi_ctx_template_t md_mac_templ; - crypto_spi_ctx_template_t md_decr_templ; -} kcf_mac_decrypt_ops_params_t; - -typedef struct kcf_random_number_ops_params { - crypto_session_id_t rn_sid; - uchar_t *rn_buf; - size_t rn_buflen; - uint_t rn_entropy_est; - uint32_t rn_flags; -} kcf_random_number_ops_params_t; - -/* - * so_pd is useful when the provider descriptor (pd) supplying the - * provider handle is different from the pd supplying the ops vector. - * This is the case for session open/close where so_pd can be the pd - * of a logical provider. The pd supplying the ops vector is passed - * as an argument to kcf_submit_request(). - */ -typedef struct kcf_session_ops_params { - crypto_session_id_t *so_sid_ptr; - crypto_session_id_t so_sid; - crypto_user_type_t so_user_type; - char *so_pin; - size_t so_pin_len; - kcf_provider_desc_t *so_pd; -} kcf_session_ops_params_t; - -typedef struct kcf_object_ops_params { - crypto_session_id_t oo_sid; - crypto_object_id_t oo_object_id; - crypto_object_attribute_t *oo_template; - uint_t oo_attribute_count; - crypto_object_id_t *oo_object_id_ptr; - size_t *oo_object_size; - void **oo_find_init_pp_ptr; - void *oo_find_pp; - uint_t oo_max_object_count; - uint_t *oo_object_count_ptr; -} kcf_object_ops_params_t; - -/* - * ko_key is used to encode wrapping key in key_wrap() and - * unwrapping key in key_unwrap(). ko_key_template and - * ko_key_attribute_count are used to encode public template - * and public template attr count in key_generate_pair(). - * kops->ko_key_object_id_ptr is used to encode public key - * in key_generate_pair(). - */ -typedef struct kcf_key_ops_params { - crypto_session_id_t ko_sid; - crypto_mech_type_t ko_framework_mechtype; - crypto_mechanism_t ko_mech; - crypto_object_attribute_t *ko_key_template; - uint_t ko_key_attribute_count; - crypto_object_id_t *ko_key_object_id_ptr; - crypto_object_attribute_t *ko_private_key_template; - uint_t ko_private_key_attribute_count; - crypto_object_id_t *ko_private_key_object_id_ptr; - crypto_key_t *ko_key; - uchar_t *ko_wrapped_key; - size_t *ko_wrapped_key_len_ptr; - crypto_object_attribute_t *ko_out_template1; - crypto_object_attribute_t *ko_out_template2; - uint_t ko_out_attribute_count1; - uint_t ko_out_attribute_count2; -} kcf_key_ops_params_t; - -/* - * po_pin and po_pin_len are used to encode new_pin and new_pin_len - * when wrapping set_pin() function parameters. - * - * po_pd is useful when the provider descriptor (pd) supplying the - * provider handle is different from the pd supplying the ops vector. - * This is true for the ext_info provider entry point where po_pd - * can be the pd of a logical provider. The pd supplying the ops vector - * is passed as an argument to kcf_submit_request(). - */ -typedef struct kcf_provmgmt_ops_params { - crypto_session_id_t po_sid; - char *po_pin; - size_t po_pin_len; - char *po_old_pin; - size_t po_old_pin_len; - char *po_label; - crypto_provider_ext_info_t *po_ext_info; - kcf_provider_desc_t *po_pd; -} kcf_provmgmt_ops_params_t; - -/* - * The operation type within a function group. - */ -typedef enum kcf_op_type { - /* common ops for all mechanisms */ - KCF_OP_INIT = 1, - KCF_OP_SINGLE, /* pkcs11 sense. So, INIT is already done */ - KCF_OP_UPDATE, - KCF_OP_FINAL, - KCF_OP_ATOMIC, - - /* digest_key op */ - KCF_OP_DIGEST_KEY, - - /* mac specific op */ - KCF_OP_MAC_VERIFY_ATOMIC, - - /* mac/cipher specific op */ - KCF_OP_MAC_VERIFY_DECRYPT_ATOMIC, - - /* sign_recover ops */ - KCF_OP_SIGN_RECOVER_INIT, - KCF_OP_SIGN_RECOVER, - KCF_OP_SIGN_RECOVER_ATOMIC, - - /* verify_recover ops */ - KCF_OP_VERIFY_RECOVER_INIT, - KCF_OP_VERIFY_RECOVER, - KCF_OP_VERIFY_RECOVER_ATOMIC, - - /* random number ops */ - KCF_OP_RANDOM_SEED, - KCF_OP_RANDOM_GENERATE, - - /* session management ops */ - KCF_OP_SESSION_OPEN, - KCF_OP_SESSION_CLOSE, - KCF_OP_SESSION_LOGIN, - KCF_OP_SESSION_LOGOUT, - - /* object management ops */ - KCF_OP_OBJECT_CREATE, - KCF_OP_OBJECT_COPY, - KCF_OP_OBJECT_DESTROY, - KCF_OP_OBJECT_GET_SIZE, - KCF_OP_OBJECT_GET_ATTRIBUTE_VALUE, - KCF_OP_OBJECT_SET_ATTRIBUTE_VALUE, - KCF_OP_OBJECT_FIND_INIT, - KCF_OP_OBJECT_FIND, - KCF_OP_OBJECT_FIND_FINAL, - - /* key management ops */ - KCF_OP_KEY_GENERATE, - KCF_OP_KEY_GENERATE_PAIR, - KCF_OP_KEY_WRAP, - KCF_OP_KEY_UNWRAP, - KCF_OP_KEY_DERIVE, - KCF_OP_KEY_CHECK, - - /* provider management ops */ - KCF_OP_MGMT_EXTINFO, - KCF_OP_MGMT_INITTOKEN, - KCF_OP_MGMT_INITPIN, - KCF_OP_MGMT_SETPIN -} kcf_op_type_t; - -/* - * The operation groups that need wrapping of parameters. This is somewhat - * similar to the function group type in spi.h except that this also includes - * all the functions that don't have a mechanism. - * - * The wrapper macros should never take these enum values as an argument. - * Rather, they are assigned in the macro itself since they are known - * from the macro name. - */ -typedef enum kcf_op_group { - KCF_OG_DIGEST = 1, - KCF_OG_MAC, - KCF_OG_ENCRYPT, - KCF_OG_DECRYPT, - KCF_OG_SIGN, - KCF_OG_VERIFY, - KCF_OG_ENCRYPT_MAC, - KCF_OG_MAC_DECRYPT, - KCF_OG_RANDOM, - KCF_OG_SESSION, - KCF_OG_OBJECT, - KCF_OG_KEY, - KCF_OG_PROVMGMT, - KCF_OG_NOSTORE_KEY -} kcf_op_group_t; - -/* - * The kcf_op_type_t enum values used here should be only for those - * operations for which there is a k-api routine in sys/crypto/api.h. - */ -#define IS_INIT_OP(ftype) ((ftype) == KCF_OP_INIT) -#define IS_SINGLE_OP(ftype) ((ftype) == KCF_OP_SINGLE) -#define IS_UPDATE_OP(ftype) ((ftype) == KCF_OP_UPDATE) -#define IS_FINAL_OP(ftype) ((ftype) == KCF_OP_FINAL) -#define IS_ATOMIC_OP(ftype) ( \ - (ftype) == KCF_OP_ATOMIC || (ftype) == KCF_OP_MAC_VERIFY_ATOMIC || \ - (ftype) == KCF_OP_MAC_VERIFY_DECRYPT_ATOMIC || \ - (ftype) == KCF_OP_SIGN_RECOVER_ATOMIC || \ - (ftype) == KCF_OP_VERIFY_RECOVER_ATOMIC) - -/* - * Keep the parameters associated with a request around. - * We need to pass them to the SPI. - */ -typedef struct kcf_req_params { - kcf_op_group_t rp_opgrp; - kcf_op_type_t rp_optype; - - union { - kcf_digest_ops_params_t digest_params; - kcf_mac_ops_params_t mac_params; - kcf_encrypt_ops_params_t encrypt_params; - kcf_decrypt_ops_params_t decrypt_params; - kcf_sign_ops_params_t sign_params; - kcf_verify_ops_params_t verify_params; - kcf_encrypt_mac_ops_params_t encrypt_mac_params; - kcf_mac_decrypt_ops_params_t mac_decrypt_params; - kcf_random_number_ops_params_t random_number_params; - kcf_session_ops_params_t session_params; - kcf_object_ops_params_t object_params; - kcf_key_ops_params_t key_params; - kcf_provmgmt_ops_params_t provmgmt_params; - } rp_u; -} kcf_req_params_t; - - -/* - * The ioctl/k-api code should bundle the parameters into a kcf_req_params_t - * structure before calling a scheduler routine. The following macros are - * available for that purpose. - * - * For the most part, the macro arguments closely correspond to the - * function parameters. In some cases, we use generic names. The comments - * for the structure should indicate these cases. - */ -#define KCF_WRAP_DIGEST_OPS_PARAMS(req, ftype, _sid, _mech, _key, \ - _data, _digest) { \ - kcf_digest_ops_params_t *dops = &(req)->rp_u.digest_params; \ - crypto_mechanism_t *mechp = _mech; \ - \ - (req)->rp_opgrp = KCF_OG_DIGEST; \ - (req)->rp_optype = ftype; \ - dops->do_sid = _sid; \ - if (mechp != NULL) { \ - dops->do_mech = *mechp; \ - dops->do_framework_mechtype = mechp->cm_type; \ - } \ - dops->do_digest_key = _key; \ - dops->do_data = _data; \ - dops->do_digest = _digest; \ -} - -#define KCF_WRAP_MAC_OPS_PARAMS(req, ftype, _sid, _mech, _key, \ - _data, _mac, _templ) { \ - kcf_mac_ops_params_t *mops = &(req)->rp_u.mac_params; \ - crypto_mechanism_t *mechp = _mech; \ - \ - (req)->rp_opgrp = KCF_OG_MAC; \ - (req)->rp_optype = ftype; \ - mops->mo_sid = _sid; \ - if (mechp != NULL) { \ - mops->mo_mech = *mechp; \ - mops->mo_framework_mechtype = mechp->cm_type; \ - } \ - mops->mo_key = _key; \ - mops->mo_data = _data; \ - mops->mo_mac = _mac; \ - mops->mo_templ = _templ; \ -} - -#define KCF_WRAP_ENCRYPT_OPS_PARAMS(req, ftype, _sid, _mech, _key, \ - _plaintext, _ciphertext, _templ) { \ - kcf_encrypt_ops_params_t *cops = &(req)->rp_u.encrypt_params; \ - crypto_mechanism_t *mechp = _mech; \ - \ - (req)->rp_opgrp = KCF_OG_ENCRYPT; \ - (req)->rp_optype = ftype; \ - cops->eo_sid = _sid; \ - if (mechp != NULL) { \ - cops->eo_mech = *mechp; \ - cops->eo_framework_mechtype = mechp->cm_type; \ - } \ - cops->eo_key = _key; \ - cops->eo_plaintext = _plaintext; \ - cops->eo_ciphertext = _ciphertext; \ - cops->eo_templ = _templ; \ -} - -#define KCF_WRAP_DECRYPT_OPS_PARAMS(req, ftype, _sid, _mech, _key, \ - _ciphertext, _plaintext, _templ) { \ - kcf_decrypt_ops_params_t *cops = &(req)->rp_u.decrypt_params; \ - crypto_mechanism_t *mechp = _mech; \ - \ - (req)->rp_opgrp = KCF_OG_DECRYPT; \ - (req)->rp_optype = ftype; \ - cops->dop_sid = _sid; \ - if (mechp != NULL) { \ - cops->dop_mech = *mechp; \ - cops->dop_framework_mechtype = mechp->cm_type; \ - } \ - cops->dop_key = _key; \ - cops->dop_ciphertext = _ciphertext; \ - cops->dop_plaintext = _plaintext; \ - cops->dop_templ = _templ; \ -} - -#define KCF_WRAP_SIGN_OPS_PARAMS(req, ftype, _sid, _mech, _key, \ - _data, _signature, _templ) { \ - kcf_sign_ops_params_t *sops = &(req)->rp_u.sign_params; \ - crypto_mechanism_t *mechp = _mech; \ - \ - (req)->rp_opgrp = KCF_OG_SIGN; \ - (req)->rp_optype = ftype; \ - sops->so_sid = _sid; \ - if (mechp != NULL) { \ - sops->so_mech = *mechp; \ - sops->so_framework_mechtype = mechp->cm_type; \ - } \ - sops->so_key = _key; \ - sops->so_data = _data; \ - sops->so_signature = _signature; \ - sops->so_templ = _templ; \ -} - -#define KCF_WRAP_VERIFY_OPS_PARAMS(req, ftype, _sid, _mech, _key, \ - _data, _signature, _templ) { \ - kcf_verify_ops_params_t *vops = &(req)->rp_u.verify_params; \ - crypto_mechanism_t *mechp = _mech; \ - \ - (req)->rp_opgrp = KCF_OG_VERIFY; \ - (req)->rp_optype = ftype; \ - vops->vo_sid = _sid; \ - if (mechp != NULL) { \ - vops->vo_mech = *mechp; \ - vops->vo_framework_mechtype = mechp->cm_type; \ - } \ - vops->vo_key = _key; \ - vops->vo_data = _data; \ - vops->vo_signature = _signature; \ - vops->vo_templ = _templ; \ -} - -#define KCF_WRAP_ENCRYPT_MAC_OPS_PARAMS(req, ftype, _sid, _encr_key, \ - _mac_key, _plaintext, _ciphertext, _mac, _encr_templ, _mac_templ) { \ - kcf_encrypt_mac_ops_params_t *cmops = &(req)->rp_u.encrypt_mac_params; \ - \ - (req)->rp_opgrp = KCF_OG_ENCRYPT_MAC; \ - (req)->rp_optype = ftype; \ - cmops->em_sid = _sid; \ - cmops->em_encr_key = _encr_key; \ - cmops->em_mac_key = _mac_key; \ - cmops->em_plaintext = _plaintext; \ - cmops->em_ciphertext = _ciphertext; \ - cmops->em_mac = _mac; \ - cmops->em_encr_templ = _encr_templ; \ - cmops->em_mac_templ = _mac_templ; \ -} - -#define KCF_WRAP_MAC_DECRYPT_OPS_PARAMS(req, ftype, _sid, _mac_key, \ - _decr_key, _ciphertext, _mac, _plaintext, _mac_templ, _decr_templ) { \ - kcf_mac_decrypt_ops_params_t *cmops = &(req)->rp_u.mac_decrypt_params; \ - \ - (req)->rp_opgrp = KCF_OG_MAC_DECRYPT; \ - (req)->rp_optype = ftype; \ - cmops->md_sid = _sid; \ - cmops->md_mac_key = _mac_key; \ - cmops->md_decr_key = _decr_key; \ - cmops->md_ciphertext = _ciphertext; \ - cmops->md_mac = _mac; \ - cmops->md_plaintext = _plaintext; \ - cmops->md_mac_templ = _mac_templ; \ - cmops->md_decr_templ = _decr_templ; \ -} - -#define KCF_WRAP_RANDOM_OPS_PARAMS(req, ftype, _sid, _buf, _buflen, \ - _est, _flags) { \ - kcf_random_number_ops_params_t *rops = \ - &(req)->rp_u.random_number_params; \ - \ - (req)->rp_opgrp = KCF_OG_RANDOM; \ - (req)->rp_optype = ftype; \ - rops->rn_sid = _sid; \ - rops->rn_buf = _buf; \ - rops->rn_buflen = _buflen; \ - rops->rn_entropy_est = _est; \ - rops->rn_flags = _flags; \ -} - -#define KCF_WRAP_SESSION_OPS_PARAMS(req, ftype, _sid_ptr, _sid, \ - _user_type, _pin, _pin_len, _pd) { \ - kcf_session_ops_params_t *sops = &(req)->rp_u.session_params; \ - \ - (req)->rp_opgrp = KCF_OG_SESSION; \ - (req)->rp_optype = ftype; \ - sops->so_sid_ptr = _sid_ptr; \ - sops->so_sid = _sid; \ - sops->so_user_type = _user_type; \ - sops->so_pin = _pin; \ - sops->so_pin_len = _pin_len; \ - sops->so_pd = _pd; \ -} - -#define KCF_WRAP_OBJECT_OPS_PARAMS(req, ftype, _sid, _object_id, \ - _template, _attribute_count, _object_id_ptr, _object_size, \ - _find_init_pp_ptr, _find_pp, _max_object_count, _object_count_ptr) { \ - kcf_object_ops_params_t *jops = &(req)->rp_u.object_params; \ - \ - (req)->rp_opgrp = KCF_OG_OBJECT; \ - (req)->rp_optype = ftype; \ - jops->oo_sid = _sid; \ - jops->oo_object_id = _object_id; \ - jops->oo_template = _template; \ - jops->oo_attribute_count = _attribute_count; \ - jops->oo_object_id_ptr = _object_id_ptr; \ - jops->oo_object_size = _object_size; \ - jops->oo_find_init_pp_ptr = _find_init_pp_ptr; \ - jops->oo_find_pp = _find_pp; \ - jops->oo_max_object_count = _max_object_count; \ - jops->oo_object_count_ptr = _object_count_ptr; \ -} - -#define KCF_WRAP_KEY_OPS_PARAMS(req, ftype, _sid, _mech, _key_template, \ - _key_attribute_count, _key_object_id_ptr, _private_key_template, \ - _private_key_attribute_count, _private_key_object_id_ptr, \ - _key, _wrapped_key, _wrapped_key_len_ptr) { \ - kcf_key_ops_params_t *kops = &(req)->rp_u.key_params; \ - crypto_mechanism_t *mechp = _mech; \ - \ - (req)->rp_opgrp = KCF_OG_KEY; \ - (req)->rp_optype = ftype; \ - kops->ko_sid = _sid; \ - if (mechp != NULL) { \ - kops->ko_mech = *mechp; \ - kops->ko_framework_mechtype = mechp->cm_type; \ - } \ - kops->ko_key_template = _key_template; \ - kops->ko_key_attribute_count = _key_attribute_count; \ - kops->ko_key_object_id_ptr = _key_object_id_ptr; \ - kops->ko_private_key_template = _private_key_template; \ - kops->ko_private_key_attribute_count = _private_key_attribute_count; \ - kops->ko_private_key_object_id_ptr = _private_key_object_id_ptr; \ - kops->ko_key = _key; \ - kops->ko_wrapped_key = _wrapped_key; \ - kops->ko_wrapped_key_len_ptr = _wrapped_key_len_ptr; \ -} - -#define KCF_WRAP_PROVMGMT_OPS_PARAMS(req, ftype, _sid, _old_pin, \ - _old_pin_len, _pin, _pin_len, _label, _ext_info, _pd) { \ - kcf_provmgmt_ops_params_t *pops = &(req)->rp_u.provmgmt_params; \ - \ - (req)->rp_opgrp = KCF_OG_PROVMGMT; \ - (req)->rp_optype = ftype; \ - pops->po_sid = _sid; \ - pops->po_pin = _pin; \ - pops->po_pin_len = _pin_len; \ - pops->po_old_pin = _old_pin; \ - pops->po_old_pin_len = _old_pin_len; \ - pops->po_label = _label; \ - pops->po_ext_info = _ext_info; \ - pops->po_pd = _pd; \ -} - -#define KCF_WRAP_NOSTORE_KEY_OPS_PARAMS(req, ftype, _sid, _mech, \ - _key_template, _key_attribute_count, _private_key_template, \ - _private_key_attribute_count, _key, _out_template1, \ - _out_attribute_count1, _out_template2, _out_attribute_count2) { \ - kcf_key_ops_params_t *kops = &(req)->rp_u.key_params; \ - crypto_mechanism_t *mechp = _mech; \ - \ - (req)->rp_opgrp = KCF_OG_NOSTORE_KEY; \ - (req)->rp_optype = ftype; \ - kops->ko_sid = _sid; \ - if (mechp != NULL) { \ - kops->ko_mech = *mechp; \ - kops->ko_framework_mechtype = mechp->cm_type; \ - } \ - kops->ko_key_template = _key_template; \ - kops->ko_key_attribute_count = _key_attribute_count; \ - kops->ko_key_object_id_ptr = NULL; \ - kops->ko_private_key_template = _private_key_template; \ - kops->ko_private_key_attribute_count = _private_key_attribute_count; \ - kops->ko_private_key_object_id_ptr = NULL; \ - kops->ko_key = _key; \ - kops->ko_wrapped_key = NULL; \ - kops->ko_wrapped_key_len_ptr = 0; \ - kops->ko_out_template1 = _out_template1; \ - kops->ko_out_template2 = _out_template2; \ - kops->ko_out_attribute_count1 = _out_attribute_count1; \ - kops->ko_out_attribute_count2 = _out_attribute_count2; \ -} - -#define KCF_SET_PROVIDER_MECHNUM(fmtype, pd, mechp) \ - (mechp)->cm_type = \ - KCF_TO_PROV_MECHNUM(pd, fmtype); - -#ifdef __cplusplus -} -#endif - -#endif /* _SYS_CRYPTO_OPS_IMPL_H */ diff --git a/module/icp/include/sys/crypto/sched_impl.h b/module/icp/include/sys/crypto/sched_impl.h index 29ef8021f0fc..1989d5244e2b 100644 --- a/module/icp/include/sys/crypto/sched_impl.h +++ b/module/icp/include/sys/crypto/sched_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -39,60 +39,6 @@ extern "C" { #include #include #include -#include - -typedef void (kcf_func_t)(void *, int); - -typedef enum kcf_req_status { - REQ_ALLOCATED = 1, - REQ_WAITING, /* At the framework level */ - REQ_INPROGRESS, /* At the provider level */ - REQ_DONE, - REQ_CANCELED -} kcf_req_status_t; - -typedef enum kcf_call_type { - CRYPTO_SYNCH = 1, - CRYPTO_ASYNCH -} kcf_call_type_t; - -#define CHECK_RESTRICT(crq) (crq != NULL && \ - ((crq)->cr_flag & CRYPTO_RESTRICTED)) - -#define CHECK_RESTRICT_FALSE B_FALSE - -#define CHECK_FASTPATH(crq, pd) ((crq) == NULL || \ - !((crq)->cr_flag & CRYPTO_ALWAYS_QUEUE)) && \ - (pd)->pd_prov_type == CRYPTO_SW_PROVIDER - -#define KCF_KMFLAG(crq) (((crq) == NULL) ? KM_SLEEP : KM_NOSLEEP) - -/* - * The framework keeps an internal handle to use in the adaptive - * asynchronous case. This is the case when a client has the - * CRYPTO_ALWAYS_QUEUE bit clear and a software provider is used for - * the request. The request is completed in the context of the calling - * thread and kernel memory must be allocated with KM_NOSLEEP. - * - * The framework passes a pointer to the handle in crypto_req_handle_t - * argument when it calls the SPI of the software provider. The macros - * KCF_RHNDL() and KCF_SWFP_RHNDL() are used to do this. - * - * When a provider asks the framework for kmflag value via - * crypto_kmflag(9S) we use REQHNDL2_KMFLAG() macro. - */ -extern ulong_t kcf_swprov_hndl; -#define KCF_RHNDL(kmflag) (((kmflag) == KM_SLEEP) ? NULL : &kcf_swprov_hndl) -#define KCF_SWFP_RHNDL(crq) (((crq) == NULL) ? NULL : &kcf_swprov_hndl) -#define REQHNDL2_KMFLAG(rhndl) \ - ((rhndl == &kcf_swprov_hndl) ? KM_NOSLEEP : KM_SLEEP) - -/* Internal call_req flags. They start after the public ones in api.h */ - -#define CRYPTO_SETDUAL 0x00001000 /* Set the 'cont' boolean before */ - /* submitting the request */ -#define KCF_ISDUALREQ(crq) \ - (((crq) == NULL) ? B_FALSE : (crq->cr_flag & CRYPTO_SETDUAL)) typedef struct kcf_prov_tried { kcf_provider_desc_t *pt_pd; @@ -106,178 +52,8 @@ typedef struct kcf_prov_tried { (tlist != NULL && is_in_triedlist(pd, tlist)) #define IS_RECOVERABLE(error) \ - (error == CRYPTO_BUFFER_TOO_BIG || \ - error == CRYPTO_BUSY || \ - error == CRYPTO_DEVICE_ERROR || \ - error == CRYPTO_DEVICE_MEMORY || \ - error == CRYPTO_KEY_SIZE_RANGE || \ - error == CRYPTO_NO_PERMISSION) - -#define KCF_ATOMIC_INCR(x) atomic_add_32(&(x), 1) -#define KCF_ATOMIC_DECR(x) atomic_add_32(&(x), -1) - -/* - * Node structure for synchronous requests. - */ -typedef struct kcf_sreq_node { - /* Should always be the first field in this structure */ - kcf_call_type_t sn_type; - /* - * sn_cv and sr_lock are used to wait for the - * operation to complete. sn_lock also protects - * the sn_state field. - */ - kcondvar_t sn_cv; - kmutex_t sn_lock; - kcf_req_status_t sn_state; - - /* - * Return value from the operation. This will be - * one of the CRYPTO_* errors defined in common.h. - */ - int sn_rv; - - /* - * parameters to call the SPI with. This can be - * a pointer as we know the caller context/stack stays. - */ - struct kcf_req_params *sn_params; - - /* Internal context for this request */ - struct kcf_context *sn_context; - - /* Provider handling this request */ - kcf_provider_desc_t *sn_provider; -} kcf_sreq_node_t; - -/* - * Node structure for asynchronous requests. A node can be on - * on a chain of requests hanging of the internal context - * structure and can be in the global software provider queue. - */ -typedef struct kcf_areq_node { - /* Should always be the first field in this structure */ - kcf_call_type_t an_type; - - /* an_lock protects the field an_state */ - kmutex_t an_lock; - kcf_req_status_t an_state; - crypto_call_req_t an_reqarg; - - /* - * parameters to call the SPI with. We need to - * save the params since the caller stack can go away. - */ - struct kcf_req_params an_params; - - /* - * The next two fields should be NULL for operations that - * don't need a context. - */ - /* Internal context for this request */ - struct kcf_context *an_context; - - /* next in chain of requests for context */ - struct kcf_areq_node *an_ctxchain_next; - - kcondvar_t an_turn_cv; - boolean_t an_is_my_turn; - boolean_t an_isdual; /* for internal reuse */ - - /* - * Next and previous nodes in the global software - * queue. These fields are NULL for a hardware - * provider since we use a taskq there. - */ - struct kcf_areq_node *an_next; - struct kcf_areq_node *an_prev; - - /* Provider handling this request */ - kcf_provider_desc_t *an_provider; - kcf_prov_tried_t *an_tried_plist; - - struct kcf_areq_node *an_idnext; /* Next in ID hash */ - struct kcf_areq_node *an_idprev; /* Prev in ID hash */ - kcondvar_t an_done; /* Signal request completion */ - uint_t an_refcnt; -} kcf_areq_node_t; - -#define KCF_AREQ_REFHOLD(areq) { \ - atomic_add_32(&(areq)->an_refcnt, 1); \ - ASSERT((areq)->an_refcnt != 0); \ -} - -#define KCF_AREQ_REFRELE(areq) { \ - ASSERT((areq)->an_refcnt != 0); \ - membar_exit(); \ - if (atomic_add_32_nv(&(areq)->an_refcnt, -1) == 0) \ - kcf_free_req(areq); \ -} - -#define GET_REQ_TYPE(arg) *((kcf_call_type_t *)(arg)) - -#define NOTIFY_CLIENT(areq, err) (*(areq)->an_reqarg.cr_callback_func)(\ - (areq)->an_reqarg.cr_callback_arg, err); - -/* For internally generated call requests for dual operations */ -typedef struct kcf_call_req { - crypto_call_req_t kr_callreq; /* external client call req */ - kcf_req_params_t kr_params; /* Params saved for next call */ - kcf_areq_node_t *kr_areq; /* Use this areq */ - off_t kr_saveoffset; - size_t kr_savelen; -} kcf_dual_req_t; - -/* - * The following are some what similar to macros in callo.h, which implement - * callout tables. - * - * The lower four bits of the ID are used to encode the table ID to - * index in to. The REQID_COUNTER_HIGH bit is used to avoid any check for - * wrap around when generating ID. We assume that there won't be a request - * which takes more time than 2^^(sizeof (long) - 5) other requests submitted - * after it. This ensures there won't be any ID collision. - */ -#define REQID_COUNTER_HIGH (1UL << (8 * sizeof (long) - 1)) -#define REQID_COUNTER_SHIFT 4 -#define REQID_COUNTER_LOW (1 << REQID_COUNTER_SHIFT) -#define REQID_TABLES 16 -#define REQID_TABLE_MASK (REQID_TABLES - 1) - -#define REQID_BUCKETS 512 -#define REQID_BUCKET_MASK (REQID_BUCKETS - 1) -#define REQID_HASH(id) (((id) >> REQID_COUNTER_SHIFT) & REQID_BUCKET_MASK) - -#define GET_REQID(areq) (areq)->an_reqarg.cr_reqid -#define SET_REQID(areq, val) GET_REQID(areq) = val - -/* - * Hash table for async requests. - */ -typedef struct kcf_reqid_table { - kmutex_t rt_lock; - crypto_req_id_t rt_curid; - kcf_areq_node_t *rt_idhash[REQID_BUCKETS]; -} kcf_reqid_table_t; - -/* - * Global software provider queue structure. Requests to be - * handled by a SW provider and have the ALWAYS_QUEUE flag set - * get queued here. - */ -typedef struct kcf_global_swq { - /* - * gs_cv and gs_lock are used to wait for new requests. - * gs_lock protects the changes to the queue. - */ - kcondvar_t gs_cv; - kmutex_t gs_lock; - uint_t gs_njobs; - uint_t gs_maxjobs; - kcf_areq_node_t *gs_first; - kcf_areq_node_t *gs_last; -} kcf_global_swq_t; - + (error == CRYPTO_BUSY || \ + error == CRYPTO_KEY_SIZE_RANGE) /* * Internal representation of a canonical context. We contain crypto_ctx_t @@ -287,30 +63,10 @@ typedef struct kcf_global_swq { typedef struct kcf_context { crypto_ctx_t kc_glbl_ctx; uint_t kc_refcnt; - kmutex_t kc_in_use_lock; - /* - * kc_req_chain_first and kc_req_chain_last are used to chain - * multiple async requests using the same context. They should be - * NULL for sync requests. - */ - kcf_areq_node_t *kc_req_chain_first; - kcf_areq_node_t *kc_req_chain_last; kcf_provider_desc_t *kc_prov_desc; /* Prov. descriptor */ kcf_provider_desc_t *kc_sw_prov_desc; /* Prov. descriptor */ - kcf_mech_entry_t *kc_mech; - struct kcf_context *kc_secondctx; /* for dual contexts */ } kcf_context_t; -/* - * Bump up the reference count on the framework private context. A - * global context or a request that references this structure should - * do a hold. - */ -#define KCF_CONTEXT_REFHOLD(ictx) { \ - atomic_add_32(&(ictx)->kc_refcnt, 1); \ - ASSERT((ictx)->kc_refcnt != 0); \ -} - /* * Decrement the reference count on the framework private context. * When the last reference is released, the framework private @@ -324,10 +80,9 @@ typedef struct kcf_context { } /* - * Check if we can release the context now. In case of CRYPTO_QUEUED - * we do not release it as we can do it only after the provider notified - * us. In case of CRYPTO_BUSY, the client can retry the request using - * the context, so we do not release the context. + * Check if we can release the context now. In case of CRYPTO_BUSY, + * the client can retry the request using the context, + * so we do not release the context. * * This macro should be called only from the final routine in * an init/update/final sequence. We do not release the context in case @@ -345,182 +100,33 @@ typedef struct kcf_context { * This macro determines whether we're done with a context. */ #define KCF_CONTEXT_DONE(rv) \ - ((rv) != CRYPTO_QUEUED && (rv) != CRYPTO_BUSY && \ - (rv) != CRYPTO_BUFFER_TOO_SMALL) + ((rv) != CRYPTO_BUSY && (rv) != CRYPTO_BUFFER_TOO_SMALL) + + +#define KCF_SET_PROVIDER_MECHNUM(fmtype, pd, mechp) \ + (mechp)->cm_type = \ + KCF_TO_PROV_MECHNUM(pd, fmtype); /* * A crypto_ctx_template_t is internally a pointer to this struct */ typedef struct kcf_ctx_template { - crypto_kcf_provider_handle_t ct_prov_handle; /* provider handle */ - uint_t ct_generation; /* generation # */ size_t ct_size; /* for freeing */ crypto_spi_ctx_template_t ct_prov_tmpl; /* context template */ - /* from the SW prov */ + /* from the provider */ } kcf_ctx_template_t; -/* - * Structure for pool of threads working on global software queue. - */ -typedef struct kcf_pool { - uint32_t kp_threads; /* Number of threads in pool */ - uint32_t kp_idlethreads; /* Idle threads in pool */ - uint32_t kp_blockedthreads; /* Blocked threads in pool */ - - /* - * cv & lock to monitor the condition when no threads - * are around. In this case the failover thread kicks in. - */ - kcondvar_t kp_nothr_cv; - kmutex_t kp_thread_lock; - - /* Userspace thread creator variables. */ - boolean_t kp_signal_create_thread; /* Create requested flag */ - int kp_nthrs; /* # of threads to create */ - boolean_t kp_user_waiting; /* Thread waiting for work */ - - /* - * cv & lock for the condition where more threads need to be - * created. kp_user_lock also protects the three fields above. - */ - kcondvar_t kp_user_cv; /* Creator cond. variable */ - kmutex_t kp_user_lock; /* Creator lock */ -} kcf_pool_t; - - -/* - * State of a crypto bufcall element. - */ -typedef enum cbuf_state { - CBUF_FREE = 1, - CBUF_WAITING, - CBUF_RUNNING -} cbuf_state_t; - -/* - * Structure of a crypto bufcall element. - */ -typedef struct kcf_cbuf_elem { - /* - * lock and cv to wait for CBUF_RUNNING to be done - * kc_lock also protects kc_state. - */ - kmutex_t kc_lock; - kcondvar_t kc_cv; - cbuf_state_t kc_state; - - struct kcf_cbuf_elem *kc_next; - struct kcf_cbuf_elem *kc_prev; - - void (*kc_func)(void *arg); - void *kc_arg; -} kcf_cbuf_elem_t; - -/* - * State of a notify element. - */ -typedef enum ntfy_elem_state { - NTFY_WAITING = 1, - NTFY_RUNNING -} ntfy_elem_state_t; - -/* - * Structure of a notify list element. - */ -typedef struct kcf_ntfy_elem { - /* - * lock and cv to wait for NTFY_RUNNING to be done. - * kn_lock also protects kn_state. - */ - kmutex_t kn_lock; - kcondvar_t kn_cv; - ntfy_elem_state_t kn_state; - - struct kcf_ntfy_elem *kn_next; - struct kcf_ntfy_elem *kn_prev; - - crypto_notify_callback_t kn_func; - uint32_t kn_event_mask; -} kcf_ntfy_elem_t; - - -/* - * The following values are based on the assumption that it would - * take around eight cpus to load a hardware provider (This is true for - * at least one product) and a kernel client may come from different - * low-priority interrupt levels. We will have CRYPTO_TASKQ_MIN number - * of cached taskq entries. The CRYPTO_TASKQ_MAX number is based on - * a throughput of 1GB/s using 512-byte buffers. These are just - * reasonable estimates and might need to change in future. - */ -#define CRYPTO_TASKQ_THREADS 8 -#define CRYPTO_TASKQ_MIN 64 -#define CRYPTO_TASKQ_MAX 2 * 1024 * 1024 - -extern const int crypto_taskq_threads; -extern const int crypto_taskq_minalloc; -extern const int crypto_taskq_maxalloc; - -/* - * All pending crypto bufcalls are put on a list. cbuf_list_lock - * protects changes to this list. - */ -extern kmutex_t cbuf_list_lock; -extern kcondvar_t cbuf_list_cv; -/* - * All event subscribers are put on a list. kcf_notify_list_lock - * protects changes to this list. - */ -extern kmutex_t ntfy_list_lock; -extern kcondvar_t ntfy_list_cv; - -boolean_t kcf_get_next_logical_provider_member(kcf_provider_desc_t *, - kcf_provider_desc_t *, kcf_provider_desc_t **); -extern int kcf_get_hardware_provider(crypto_mech_type_t, crypto_mech_type_t, - boolean_t, kcf_provider_desc_t *, kcf_provider_desc_t **, - crypto_func_group_t); -extern int kcf_get_hardware_provider_nomech(offset_t, offset_t, - boolean_t, kcf_provider_desc_t *, kcf_provider_desc_t **); extern void kcf_free_triedlist(kcf_prov_tried_t *); extern kcf_prov_tried_t *kcf_insert_triedlist(kcf_prov_tried_t **, kcf_provider_desc_t *, int); extern kcf_provider_desc_t *kcf_get_mech_provider(crypto_mech_type_t, - kcf_mech_entry_t **, int *, kcf_prov_tried_t *, crypto_func_group_t, - boolean_t, size_t); -extern kcf_provider_desc_t *kcf_get_dual_provider(crypto_mechanism_t *, - crypto_mechanism_t *, kcf_mech_entry_t **, crypto_mech_type_t *, - crypto_mech_type_t *, int *, kcf_prov_tried_t *, - crypto_func_group_t, crypto_func_group_t, boolean_t, size_t); -extern crypto_ctx_t *kcf_new_ctx(crypto_call_req_t *, kcf_provider_desc_t *, - crypto_session_id_t); -extern int kcf_submit_request(kcf_provider_desc_t *, crypto_ctx_t *, - crypto_call_req_t *, kcf_req_params_t *, boolean_t); + kcf_mech_entry_t **, int *, kcf_prov_tried_t *, crypto_func_group_t); +extern crypto_ctx_t *kcf_new_ctx(kcf_provider_desc_t *); extern void kcf_sched_destroy(void); extern void kcf_sched_init(void); -extern void kcf_sched_start(void); -extern void kcf_sop_done(kcf_sreq_node_t *, int); -extern void kcf_aop_done(kcf_areq_node_t *, int); -extern int common_submit_request(kcf_provider_desc_t *, - crypto_ctx_t *, kcf_req_params_t *, crypto_req_handle_t); extern void kcf_free_context(kcf_context_t *); -extern int kcf_svc_wait(int *); -extern int kcf_svc_do_run(void); -extern int kcf_need_signature_verification(kcf_provider_desc_t *); -extern void kcf_verify_signature(void *); -extern struct modctl *kcf_get_modctl(crypto_provider_info_t *); -extern void verify_unverified_providers(void); -extern void kcf_free_req(kcf_areq_node_t *areq); -extern void crypto_bufcall_service(void); - -extern void kcf_walk_ntfylist(uint32_t, void *); -extern void kcf_do_notify(kcf_provider_desc_t *, boolean_t); - -extern kcf_dual_req_t *kcf_alloc_req(crypto_call_req_t *); -extern void kcf_next_req(void *, int); -extern void kcf_last_req(void *, int); - #ifdef __cplusplus } #endif diff --git a/module/icp/include/sys/crypto/spi.h b/module/icp/include/sys/crypto/spi.h index 0f1b455c808e..63dfce7957a8 100644 --- a/module/icp/include/sys/crypto/spi.h +++ b/module/icp/include/sys/crypto/spi.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -43,38 +43,14 @@ extern "C" { #define __no_const #endif /* CONSTIFY_PLUGIN */ -#define CRYPTO_SPI_VERSION_1 1 -#define CRYPTO_SPI_VERSION_2 2 -#define CRYPTO_SPI_VERSION_3 3 - -/* - * Provider-private handle. This handle is specified by a provider - * when it registers by means of the pi_provider_handle field of - * the crypto_provider_info structure, and passed to the provider - * when its entry points are invoked. - */ -typedef void *crypto_provider_handle_t; - /* - * Context templates can be used to by software providers to pre-process + * Context templates can be used to by providers to pre-process * keying material, such as key schedules. They are allocated by - * a software provider create_ctx_template(9E) entry point, and passed + * a provider create_ctx_template(9E) entry point, and passed * as argument to initialization and atomic provider entry points. */ typedef void *crypto_spi_ctx_template_t; -/* - * Request handles are used by the kernel to identify an asynchronous - * request being processed by a provider. It is passed by the kernel - * to a hardware provider when submitting a request, and must be - * specified by a provider when calling crypto_op_notification(9F) - */ -typedef void *crypto_req_handle_t; - -/* Values for cc_flags field */ -#define CRYPTO_INIT_OPSTATE 0x00000001 /* allocate and init cc_opstate */ -#define CRYPTO_USE_OPSTATE 0x00000002 /* .. start using it as context */ - /* * The context structure is passed from the kernel to a provider. * It contains the information needed to process a multi-part or @@ -86,62 +62,10 @@ typedef void *crypto_req_handle_t; * as separate arguments to Provider routines. */ typedef struct crypto_ctx { - crypto_provider_handle_t cc_provider; - crypto_session_id_t cc_session; void *cc_provider_private; /* owned by provider */ void *cc_framework_private; /* owned by framework */ - uint32_t cc_flags; /* flags */ - void *cc_opstate; /* state */ } crypto_ctx_t; -/* - * Extended provider information. - */ - -/* - * valid values for ei_flags field of extended info structure - * They match the RSA Security, Inc PKCS#11 tokenInfo flags. - */ -#define CRYPTO_EXTF_RNG 0x00000001 -#define CRYPTO_EXTF_WRITE_PROTECTED 0x00000002 -#define CRYPTO_EXTF_LOGIN_REQUIRED 0x00000004 -#define CRYPTO_EXTF_USER_PIN_INITIALIZED 0x00000008 -#define CRYPTO_EXTF_CLOCK_ON_TOKEN 0x00000040 -#define CRYPTO_EXTF_PROTECTED_AUTHENTICATION_PATH 0x00000100 -#define CRYPTO_EXTF_DUAL_CRYPTO_OPERATIONS 0x00000200 -#define CRYPTO_EXTF_TOKEN_INITIALIZED 0x00000400 -#define CRYPTO_EXTF_USER_PIN_COUNT_LOW 0x00010000 -#define CRYPTO_EXTF_USER_PIN_FINAL_TRY 0x00020000 -#define CRYPTO_EXTF_USER_PIN_LOCKED 0x00040000 -#define CRYPTO_EXTF_USER_PIN_TO_BE_CHANGED 0x00080000 -#define CRYPTO_EXTF_SO_PIN_COUNT_LOW 0x00100000 -#define CRYPTO_EXTF_SO_PIN_FINAL_TRY 0x00200000 -#define CRYPTO_EXTF_SO_PIN_LOCKED 0x00400000 -#define CRYPTO_EXTF_SO_PIN_TO_BE_CHANGED 0x00800000 - -/* - * The crypto_control_ops structure contains pointers to control - * operations for cryptographic providers. It is passed through - * the crypto_ops(9S) structure when providers register with the - * kernel using crypto_register_provider(9F). - */ -typedef struct crypto_control_ops { - void (*provider_status)(crypto_provider_handle_t, uint_t *); -} __no_const crypto_control_ops_t; - -/* - * The crypto_ctx_ops structure contains points to context and context - * templates management operations for cryptographic providers. It is - * passed through the crypto_ops(9S) structure when providers register - * with the kernel using crypto_register_provider(9F). - */ -typedef struct crypto_ctx_ops { - int (*create_ctx_template)(crypto_provider_handle_t, - crypto_mechanism_t *, crypto_key_t *, - crypto_spi_ctx_template_t *, size_t *, crypto_req_handle_t); - int (*free_context)(crypto_ctx_t *); -} __no_const crypto_ctx_ops_t; - /* * The crypto_digest_ops structure contains pointers to digest * operations for cryptographic providers. It is passed through @@ -149,18 +73,13 @@ typedef struct crypto_ctx_ops { * kernel using crypto_register_provider(9F). */ typedef struct crypto_digest_ops { - int (*digest_init)(crypto_ctx_t *, crypto_mechanism_t *, - crypto_req_handle_t); - int (*digest)(crypto_ctx_t *, crypto_data_t *, crypto_data_t *, - crypto_req_handle_t); - int (*digest_update)(crypto_ctx_t *, crypto_data_t *, - crypto_req_handle_t); - int (*digest_key)(crypto_ctx_t *, crypto_key_t *, crypto_req_handle_t); - int (*digest_final)(crypto_ctx_t *, crypto_data_t *, - crypto_req_handle_t); - int (*digest_atomic)(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_data_t *, - crypto_data_t *, crypto_req_handle_t); + int (*digest_init)(crypto_ctx_t *, crypto_mechanism_t *); + int (*digest)(crypto_ctx_t *, crypto_data_t *, crypto_data_t *); + int (*digest_update)(crypto_ctx_t *, crypto_data_t *); + int (*digest_key)(crypto_ctx_t *, crypto_key_t *); + int (*digest_final)(crypto_ctx_t *, crypto_data_t *); + int (*digest_atomic)(crypto_mechanism_t *, crypto_data_t *, + crypto_data_t *); } __no_const crypto_digest_ops_t; /* @@ -172,29 +91,27 @@ typedef struct crypto_digest_ops { typedef struct crypto_cipher_ops { int (*encrypt_init)(crypto_ctx_t *, crypto_mechanism_t *, crypto_key_t *, - crypto_spi_ctx_template_t, crypto_req_handle_t); + crypto_spi_ctx_template_t); int (*encrypt)(crypto_ctx_t *, - crypto_data_t *, crypto_data_t *, crypto_req_handle_t); + crypto_data_t *, crypto_data_t *); int (*encrypt_update)(crypto_ctx_t *, - crypto_data_t *, crypto_data_t *, crypto_req_handle_t); + crypto_data_t *, crypto_data_t *); int (*encrypt_final)(crypto_ctx_t *, - crypto_data_t *, crypto_req_handle_t); - int (*encrypt_atomic)(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, - crypto_data_t *, crypto_spi_ctx_template_t, crypto_req_handle_t); + crypto_data_t *); + int (*encrypt_atomic)(crypto_mechanism_t *, crypto_key_t *, + crypto_data_t *, crypto_data_t *, crypto_spi_ctx_template_t); int (*decrypt_init)(crypto_ctx_t *, crypto_mechanism_t *, crypto_key_t *, - crypto_spi_ctx_template_t, crypto_req_handle_t); + crypto_spi_ctx_template_t); int (*decrypt)(crypto_ctx_t *, - crypto_data_t *, crypto_data_t *, crypto_req_handle_t); + crypto_data_t *, crypto_data_t *); int (*decrypt_update)(crypto_ctx_t *, - crypto_data_t *, crypto_data_t *, crypto_req_handle_t); + crypto_data_t *, crypto_data_t *); int (*decrypt_final)(crypto_ctx_t *, - crypto_data_t *, crypto_req_handle_t); - int (*decrypt_atomic)(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, - crypto_data_t *, crypto_spi_ctx_template_t, crypto_req_handle_t); + crypto_data_t *); + int (*decrypt_atomic)(crypto_mechanism_t *, crypto_key_t *, + crypto_data_t *, crypto_data_t *, crypto_spi_ctx_template_t); } __no_const crypto_cipher_ops_t; /* @@ -206,289 +123,30 @@ typedef struct crypto_cipher_ops { typedef struct crypto_mac_ops { int (*mac_init)(crypto_ctx_t *, crypto_mechanism_t *, crypto_key_t *, - crypto_spi_ctx_template_t, crypto_req_handle_t); + crypto_spi_ctx_template_t); int (*mac)(crypto_ctx_t *, - crypto_data_t *, crypto_data_t *, crypto_req_handle_t); + crypto_data_t *, crypto_data_t *); int (*mac_update)(crypto_ctx_t *, - crypto_data_t *, crypto_req_handle_t); + crypto_data_t *); int (*mac_final)(crypto_ctx_t *, - crypto_data_t *, crypto_req_handle_t); - int (*mac_atomic)(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, - crypto_data_t *, crypto_spi_ctx_template_t, - crypto_req_handle_t); - int (*mac_verify_atomic)(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, - crypto_data_t *, crypto_spi_ctx_template_t, - crypto_req_handle_t); + crypto_data_t *); + int (*mac_atomic)(crypto_mechanism_t *, crypto_key_t *, + crypto_data_t *, crypto_data_t *, crypto_spi_ctx_template_t); + int (*mac_verify_atomic)(crypto_mechanism_t *, crypto_key_t *, + crypto_data_t *, crypto_data_t *, crypto_spi_ctx_template_t); } __no_const crypto_mac_ops_t; /* - * The crypto_sign_ops structure contains pointers to signing - * operations for cryptographic providers. It is passed through - * the crypto_ops(9S) structure when providers register with the - * kernel using crypto_register_provider(9F). - */ -typedef struct crypto_sign_ops { - int (*sign_init)(crypto_ctx_t *, - crypto_mechanism_t *, crypto_key_t *, crypto_spi_ctx_template_t, - crypto_req_handle_t); - int (*sign)(crypto_ctx_t *, - crypto_data_t *, crypto_data_t *, crypto_req_handle_t); - int (*sign_update)(crypto_ctx_t *, - crypto_data_t *, crypto_req_handle_t); - int (*sign_final)(crypto_ctx_t *, - crypto_data_t *, crypto_req_handle_t); - int (*sign_atomic)(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, - crypto_data_t *, crypto_spi_ctx_template_t, - crypto_req_handle_t); - int (*sign_recover_init)(crypto_ctx_t *, crypto_mechanism_t *, - crypto_key_t *, crypto_spi_ctx_template_t, - crypto_req_handle_t); - int (*sign_recover)(crypto_ctx_t *, - crypto_data_t *, crypto_data_t *, crypto_req_handle_t); - int (*sign_recover_atomic)(crypto_provider_handle_t, - crypto_session_id_t, crypto_mechanism_t *, crypto_key_t *, - crypto_data_t *, crypto_data_t *, crypto_spi_ctx_template_t, - crypto_req_handle_t); -} __no_const crypto_sign_ops_t; - -/* - * The crypto_verify_ops structure contains pointers to verify - * operations for cryptographic providers. It is passed through - * the crypto_ops(9S) structure when providers register with the - * kernel using crypto_register_provider(9F). - */ -typedef struct crypto_verify_ops { - int (*verify_init)(crypto_ctx_t *, - crypto_mechanism_t *, crypto_key_t *, crypto_spi_ctx_template_t, - crypto_req_handle_t); - int (*do_verify)(crypto_ctx_t *, - crypto_data_t *, crypto_data_t *, crypto_req_handle_t); - int (*verify_update)(crypto_ctx_t *, - crypto_data_t *, crypto_req_handle_t); - int (*verify_final)(crypto_ctx_t *, - crypto_data_t *, crypto_req_handle_t); - int (*verify_atomic)(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, - crypto_data_t *, crypto_spi_ctx_template_t, - crypto_req_handle_t); - int (*verify_recover_init)(crypto_ctx_t *, crypto_mechanism_t *, - crypto_key_t *, crypto_spi_ctx_template_t, - crypto_req_handle_t); - int (*verify_recover)(crypto_ctx_t *, - crypto_data_t *, crypto_data_t *, crypto_req_handle_t); - int (*verify_recover_atomic)(crypto_provider_handle_t, - crypto_session_id_t, crypto_mechanism_t *, crypto_key_t *, - crypto_data_t *, crypto_data_t *, crypto_spi_ctx_template_t, - crypto_req_handle_t); -} __no_const crypto_verify_ops_t; - -/* - * The crypto_dual_ops structure contains pointers to dual - * cipher and sign/verify operations for cryptographic providers. - * It is passed through the crypto_ops(9S) structure when - * providers register with the kernel using - * crypto_register_provider(9F). - */ -typedef struct crypto_dual_ops { - int (*digest_encrypt_update)( - crypto_ctx_t *, crypto_ctx_t *, crypto_data_t *, - crypto_data_t *, crypto_req_handle_t); - int (*decrypt_digest_update)( - crypto_ctx_t *, crypto_ctx_t *, crypto_data_t *, - crypto_data_t *, crypto_req_handle_t); - int (*sign_encrypt_update)( - crypto_ctx_t *, crypto_ctx_t *, crypto_data_t *, - crypto_data_t *, crypto_req_handle_t); - int (*decrypt_verify_update)( - crypto_ctx_t *, crypto_ctx_t *, crypto_data_t *, - crypto_data_t *, crypto_req_handle_t); -} __no_const crypto_dual_ops_t; - -/* - * The crypto_dual_cipher_mac_ops structure contains pointers to dual - * cipher and MAC operations for cryptographic providers. - * It is passed through the crypto_ops(9S) structure when - * providers register with the kernel using - * crypto_register_provider(9F). - */ -typedef struct crypto_dual_cipher_mac_ops { - int (*encrypt_mac_init)(crypto_ctx_t *, - crypto_mechanism_t *, crypto_key_t *, crypto_mechanism_t *, - crypto_key_t *, crypto_spi_ctx_template_t, - crypto_spi_ctx_template_t, crypto_req_handle_t); - int (*encrypt_mac)(crypto_ctx_t *, - crypto_data_t *, crypto_dual_data_t *, crypto_data_t *, - crypto_req_handle_t); - int (*encrypt_mac_update)(crypto_ctx_t *, - crypto_data_t *, crypto_dual_data_t *, crypto_req_handle_t); - int (*encrypt_mac_final)(crypto_ctx_t *, - crypto_dual_data_t *, crypto_data_t *, crypto_req_handle_t); - int (*encrypt_mac_atomic)(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_mechanism_t *, - crypto_key_t *, crypto_data_t *, crypto_dual_data_t *, - crypto_data_t *, crypto_spi_ctx_template_t, - crypto_spi_ctx_template_t, crypto_req_handle_t); - - int (*mac_decrypt_init)(crypto_ctx_t *, - crypto_mechanism_t *, crypto_key_t *, crypto_mechanism_t *, - crypto_key_t *, crypto_spi_ctx_template_t, - crypto_spi_ctx_template_t, crypto_req_handle_t); - int (*mac_decrypt)(crypto_ctx_t *, - crypto_dual_data_t *, crypto_data_t *, crypto_data_t *, - crypto_req_handle_t); - int (*mac_decrypt_update)(crypto_ctx_t *, - crypto_dual_data_t *, crypto_data_t *, crypto_req_handle_t); - int (*mac_decrypt_final)(crypto_ctx_t *, - crypto_data_t *, crypto_data_t *, crypto_req_handle_t); - int (*mac_decrypt_atomic)(crypto_provider_handle_t, - crypto_session_id_t, crypto_mechanism_t *, crypto_key_t *, - crypto_mechanism_t *, crypto_key_t *, crypto_dual_data_t *, - crypto_data_t *, crypto_data_t *, crypto_spi_ctx_template_t, - crypto_spi_ctx_template_t, crypto_req_handle_t); - int (*mac_verify_decrypt_atomic)(crypto_provider_handle_t, - crypto_session_id_t, crypto_mechanism_t *, crypto_key_t *, - crypto_mechanism_t *, crypto_key_t *, crypto_dual_data_t *, - crypto_data_t *, crypto_data_t *, crypto_spi_ctx_template_t, - crypto_spi_ctx_template_t, crypto_req_handle_t); -} __no_const crypto_dual_cipher_mac_ops_t; - -/* - * The crypto_random_number_ops structure contains pointers to random - * number operations for cryptographic providers. It is passed through - * the crypto_ops(9S) structure when providers register with the - * kernel using crypto_register_provider(9F). - */ -typedef struct crypto_random_number_ops { - int (*seed_random)(crypto_provider_handle_t, crypto_session_id_t, - uchar_t *, size_t, uint_t, uint32_t, crypto_req_handle_t); - int (*generate_random)(crypto_provider_handle_t, crypto_session_id_t, - uchar_t *, size_t, crypto_req_handle_t); -} __no_const crypto_random_number_ops_t; - -/* - * Flag values for seed_random. - */ -#define CRYPTO_SEED_NOW 0x00000001 - -/* - * The crypto_session_ops structure contains pointers to session - * operations for cryptographic providers. It is passed through - * the crypto_ops(9S) structure when providers register with the - * kernel using crypto_register_provider(9F). - */ -typedef struct crypto_session_ops { - int (*session_open)(crypto_provider_handle_t, crypto_session_id_t *, - crypto_req_handle_t); - int (*session_close)(crypto_provider_handle_t, crypto_session_id_t, - crypto_req_handle_t); - int (*session_login)(crypto_provider_handle_t, crypto_session_id_t, - crypto_user_type_t, char *, size_t, crypto_req_handle_t); - int (*session_logout)(crypto_provider_handle_t, crypto_session_id_t, - crypto_req_handle_t); -} __no_const crypto_session_ops_t; - -/* - * The crypto_object_ops structure contains pointers to object - * operations for cryptographic providers. It is passed through - * the crypto_ops(9S) structure when providers register with the - * kernel using crypto_register_provider(9F). - */ -typedef struct crypto_object_ops { - int (*object_create)(crypto_provider_handle_t, crypto_session_id_t, - crypto_object_attribute_t *, uint_t, crypto_object_id_t *, - crypto_req_handle_t); - int (*object_copy)(crypto_provider_handle_t, crypto_session_id_t, - crypto_object_id_t, crypto_object_attribute_t *, uint_t, - crypto_object_id_t *, crypto_req_handle_t); - int (*object_destroy)(crypto_provider_handle_t, crypto_session_id_t, - crypto_object_id_t, crypto_req_handle_t); - int (*object_get_size)(crypto_provider_handle_t, crypto_session_id_t, - crypto_object_id_t, size_t *, crypto_req_handle_t); - int (*object_get_attribute_value)(crypto_provider_handle_t, - crypto_session_id_t, crypto_object_id_t, - crypto_object_attribute_t *, uint_t, crypto_req_handle_t); - int (*object_set_attribute_value)(crypto_provider_handle_t, - crypto_session_id_t, crypto_object_id_t, - crypto_object_attribute_t *, uint_t, crypto_req_handle_t); - int (*object_find_init)(crypto_provider_handle_t, crypto_session_id_t, - crypto_object_attribute_t *, uint_t, void **, - crypto_req_handle_t); - int (*object_find)(crypto_provider_handle_t, void *, - crypto_object_id_t *, uint_t, uint_t *, crypto_req_handle_t); - int (*object_find_final)(crypto_provider_handle_t, void *, - crypto_req_handle_t); -} __no_const crypto_object_ops_t; - -/* - * The crypto_key_ops structure contains pointers to key - * operations for cryptographic providers. It is passed through - * the crypto_ops(9S) structure when providers register with the - * kernel using crypto_register_provider(9F). - */ -typedef struct crypto_key_ops { - int (*key_generate)(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_object_attribute_t *, uint_t, - crypto_object_id_t *, crypto_req_handle_t); - int (*key_generate_pair)(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_object_attribute_t *, uint_t, - crypto_object_attribute_t *, uint_t, crypto_object_id_t *, - crypto_object_id_t *, crypto_req_handle_t); - int (*key_wrap)(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_object_id_t *, - uchar_t *, size_t *, crypto_req_handle_t); - int (*key_unwrap)(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, uchar_t *, size_t *, - crypto_object_attribute_t *, uint_t, - crypto_object_id_t *, crypto_req_handle_t); - int (*key_derive)(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_object_attribute_t *, - uint_t, crypto_object_id_t *, crypto_req_handle_t); - int (*key_check)(crypto_provider_handle_t, crypto_mechanism_t *, - crypto_key_t *); -} __no_const crypto_key_ops_t; - -/* - * The crypto_provider_management_ops structure contains pointers - * to management operations for cryptographic providers. It is passed - * through the crypto_ops(9S) structure when providers register with the - * kernel using crypto_register_provider(9F). + * The crypto_ctx_ops structure contains points to context and context + * templates management operations for cryptographic providers. It is + * passed through the crypto_ops(9S) structure when providers register + * with the kernel using crypto_register_provider(9F). */ -typedef struct crypto_provider_management_ops { - int (*ext_info)(crypto_provider_handle_t, - crypto_provider_ext_info_t *, crypto_req_handle_t); - int (*init_token)(crypto_provider_handle_t, char *, size_t, - char *, crypto_req_handle_t); - int (*init_pin)(crypto_provider_handle_t, crypto_session_id_t, - char *, size_t, crypto_req_handle_t); - int (*set_pin)(crypto_provider_handle_t, crypto_session_id_t, - char *, size_t, char *, size_t, crypto_req_handle_t); -} __no_const crypto_provider_management_ops_t; - -typedef struct crypto_mech_ops { - int (*copyin_mechanism)(crypto_provider_handle_t, - crypto_mechanism_t *, crypto_mechanism_t *, int *, int); - int (*copyout_mechanism)(crypto_provider_handle_t, - crypto_mechanism_t *, crypto_mechanism_t *, int *, int); - int (*free_mechanism)(crypto_provider_handle_t, crypto_mechanism_t *); -} __no_const crypto_mech_ops_t; - -typedef struct crypto_nostore_key_ops { - int (*nostore_key_generate)(crypto_provider_handle_t, - crypto_session_id_t, crypto_mechanism_t *, - crypto_object_attribute_t *, uint_t, crypto_object_attribute_t *, - uint_t, crypto_req_handle_t); - int (*nostore_key_generate_pair)(crypto_provider_handle_t, - crypto_session_id_t, crypto_mechanism_t *, - crypto_object_attribute_t *, uint_t, crypto_object_attribute_t *, - uint_t, crypto_object_attribute_t *, uint_t, - crypto_object_attribute_t *, uint_t, crypto_req_handle_t); - int (*nostore_key_derive)(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_object_attribute_t *, - uint_t, crypto_object_attribute_t *, uint_t, crypto_req_handle_t); -} __no_const crypto_nostore_key_ops_t; +typedef struct crypto_ctx_ops { + int (*create_ctx_template)(crypto_mechanism_t *, crypto_key_t *, + crypto_spi_ctx_template_t *, size_t *); + int (*free_context)(crypto_ctx_t *); +} __no_const crypto_ctx_ops_t; /* * The crypto_ops(9S) structure contains the structures containing @@ -497,58 +155,13 @@ typedef struct crypto_nostore_key_ops { * supplied by a provider when it registers with the kernel * by calling crypto_register_provider(9F). */ -typedef struct crypto_ops_v1 { - const crypto_control_ops_t *co_control_ops; +typedef struct crypto_ops { const crypto_digest_ops_t *co_digest_ops; const crypto_cipher_ops_t *co_cipher_ops; const crypto_mac_ops_t *co_mac_ops; - crypto_sign_ops_t *co_sign_ops; - crypto_verify_ops_t *co_verify_ops; - crypto_dual_ops_t *co_dual_ops; - crypto_dual_cipher_mac_ops_t *co_dual_cipher_mac_ops; - crypto_random_number_ops_t *co_random_ops; - crypto_session_ops_t *co_session_ops; - crypto_object_ops_t *co_object_ops; - crypto_key_ops_t *co_key_ops; - crypto_provider_management_ops_t *co_provider_ops; const crypto_ctx_ops_t *co_ctx_ops; -} crypto_ops_v1_t; - -typedef struct crypto_ops_v2 { - crypto_ops_v1_t v1_ops; - crypto_mech_ops_t *co_mech_ops; -} crypto_ops_v2_t; - -typedef struct crypto_ops_v3 { - crypto_ops_v2_t v2_ops; - crypto_nostore_key_ops_t *co_nostore_key_ops; -} crypto_ops_v3_t; - -typedef struct crypto_ops { - union { - crypto_ops_v3_t cou_v3; - crypto_ops_v2_t cou_v2; - crypto_ops_v1_t cou_v1; - } cou; } crypto_ops_t; -#define co_control_ops cou.cou_v1.co_control_ops -#define co_digest_ops cou.cou_v1.co_digest_ops -#define co_cipher_ops cou.cou_v1.co_cipher_ops -#define co_mac_ops cou.cou_v1.co_mac_ops -#define co_sign_ops cou.cou_v1.co_sign_ops -#define co_verify_ops cou.cou_v1.co_verify_ops -#define co_dual_ops cou.cou_v1.co_dual_ops -#define co_dual_cipher_mac_ops cou.cou_v1.co_dual_cipher_mac_ops -#define co_random_ops cou.cou_v1.co_random_ops -#define co_session_ops cou.cou_v1.co_session_ops -#define co_object_ops cou.cou_v1.co_object_ops -#define co_key_ops cou.cou_v1.co_key_ops -#define co_provider_ops cou.cou_v1.co_provider_ops -#define co_ctx_ops cou.cou_v1.co_ctx_ops -#define co_mech_ops cou.cou_v2.co_mech_ops -#define co_nostore_key_ops cou.cou_v3.co_nostore_key_ops - /* * The mechanism info structure crypto_mech_info_t contains a function group * bit mask cm_func_group_mask. This field, of type crypto_func_group_t, @@ -562,29 +175,11 @@ typedef uint32_t crypto_func_group_t; #define CRYPTO_FG_ENCRYPT 0x00000001 /* encrypt_init() */ #define CRYPTO_FG_DECRYPT 0x00000002 /* decrypt_init() */ #define CRYPTO_FG_DIGEST 0x00000004 /* digest_init() */ -#define CRYPTO_FG_SIGN 0x00000008 /* sign_init() */ -#define CRYPTO_FG_SIGN_RECOVER 0x00000010 /* sign_recover_init() */ -#define CRYPTO_FG_VERIFY 0x00000020 /* verify_init() */ -#define CRYPTO_FG_VERIFY_RECOVER 0x00000040 /* verify_recover_init() */ -#define CRYPTO_FG_GENERATE 0x00000080 /* key_generate() */ -#define CRYPTO_FG_GENERATE_KEY_PAIR 0x00000100 /* key_generate_pair() */ -#define CRYPTO_FG_WRAP 0x00000200 /* key_wrap() */ -#define CRYPTO_FG_UNWRAP 0x00000400 /* key_unwrap() */ -#define CRYPTO_FG_DERIVE 0x00000800 /* key_derive() */ #define CRYPTO_FG_MAC 0x00001000 /* mac_init() */ -#define CRYPTO_FG_ENCRYPT_MAC 0x00002000 /* encrypt_mac_init() */ -#define CRYPTO_FG_MAC_DECRYPT 0x00004000 /* decrypt_mac_init() */ #define CRYPTO_FG_ENCRYPT_ATOMIC 0x00008000 /* encrypt_atomic() */ #define CRYPTO_FG_DECRYPT_ATOMIC 0x00010000 /* decrypt_atomic() */ #define CRYPTO_FG_MAC_ATOMIC 0x00020000 /* mac_atomic() */ #define CRYPTO_FG_DIGEST_ATOMIC 0x00040000 /* digest_atomic() */ -#define CRYPTO_FG_SIGN_ATOMIC 0x00080000 /* sign_atomic() */ -#define CRYPTO_FG_SIGN_RECOVER_ATOMIC 0x00100000 /* sign_recover_atomic() */ -#define CRYPTO_FG_VERIFY_ATOMIC 0x00200000 /* verify_atomic() */ -#define CRYPTO_FG_VERIFY_RECOVER_ATOMIC 0x00400000 /* verify_recover_atomic() */ -#define CRYPTO_FG_ENCRYPT_MAC_ATOMIC 0x00800000 /* encrypt_mac_atomic() */ -#define CRYPTO_FG_MAC_DECRYPT_ATOMIC 0x01000000 /* mac_decrypt_atomic() */ -#define CRYPTO_FG_RESERVED 0x80000000 /* * Maximum length of the pi_provider_description field of the @@ -593,21 +188,6 @@ typedef uint32_t crypto_func_group_t; #define CRYPTO_PROVIDER_DESCR_MAX_LEN 64 -/* Bit mask for all the simple operations */ -#define CRYPTO_FG_SIMPLEOP_MASK (CRYPTO_FG_ENCRYPT | CRYPTO_FG_DECRYPT | \ - CRYPTO_FG_DIGEST | CRYPTO_FG_SIGN | CRYPTO_FG_VERIFY | CRYPTO_FG_MAC | \ - CRYPTO_FG_ENCRYPT_ATOMIC | CRYPTO_FG_DECRYPT_ATOMIC | \ - CRYPTO_FG_MAC_ATOMIC | CRYPTO_FG_DIGEST_ATOMIC | CRYPTO_FG_SIGN_ATOMIC | \ - CRYPTO_FG_VERIFY_ATOMIC) - -/* Bit mask for all the dual operations */ -#define CRYPTO_FG_MAC_CIPHER_MASK (CRYPTO_FG_ENCRYPT_MAC | \ - CRYPTO_FG_MAC_DECRYPT | CRYPTO_FG_ENCRYPT_MAC_ATOMIC | \ - CRYPTO_FG_MAC_DECRYPT_ATOMIC) - -/* Add other combos to CRYPTO_FG_DUAL_MASK */ -#define CRYPTO_FG_DUAL_MASK CRYPTO_FG_MAC_CIPHER_MASK - /* * The crypto_mech_info structure specifies one of the mechanisms * supported by a cryptographic provider. The pi_mechanisms field of @@ -618,21 +198,8 @@ typedef struct crypto_mech_info { crypto_mech_name_t cm_mech_name; crypto_mech_type_t cm_mech_number; crypto_func_group_t cm_func_group_mask; - ssize_t cm_min_key_length; - ssize_t cm_max_key_length; - uint32_t cm_mech_flags; } crypto_mech_info_t; -/* Alias the old name to the new name for compatibility. */ -#define cm_keysize_unit cm_mech_flags - -/* - * The following is used by a provider that sets - * CRYPTO_HASH_NO_UPDATE. It needs to specify the maximum - * input data size it can digest in this field. - */ -#define cm_max_input_length cm_max_key_length - /* * crypto_kcf_provider_handle_t is a handle allocated by the kernel. * It is returned after the provider registers with @@ -644,67 +211,15 @@ typedef uint_t crypto_kcf_provider_handle_t; /* * Provider information. Passed as argument to crypto_register_provider(9F). - * Describes the provider and its capabilities. Multiple providers can - * register for the same device instance. In this case, the same - * pi_provider_dev must be specified with a different pi_provider_handle. + * Describes the provider and its capabilities. */ -typedef struct crypto_provider_info_v1 { - uint_t pi_interface_version; - char *pi_provider_description; - crypto_provider_type_t pi_provider_type; - crypto_provider_handle_t pi_provider_handle; +typedef struct crypto_provider_info { + const char *pi_provider_description; const crypto_ops_t *pi_ops_vector; uint_t pi_mech_list_count; const crypto_mech_info_t *pi_mechanisms; - uint_t pi_logical_provider_count; - crypto_kcf_provider_handle_t *pi_logical_providers; -} crypto_provider_info_v1_t; - -typedef struct crypto_provider_info_v2 { - crypto_provider_info_v1_t v1_info; - uint_t pi_flags; -} crypto_provider_info_v2_t; - -typedef struct crypto_provider_info { - union { - crypto_provider_info_v2_t piu_v2; - crypto_provider_info_v1_t piu_v1; - } piu; } crypto_provider_info_t; -#define pi_interface_version piu.piu_v1.pi_interface_version -#define pi_provider_description piu.piu_v1.pi_provider_description -#define pi_provider_type piu.piu_v1.pi_provider_type -#define pi_provider_handle piu.piu_v1.pi_provider_handle -#define pi_ops_vector piu.piu_v1.pi_ops_vector -#define pi_mech_list_count piu.piu_v1.pi_mech_list_count -#define pi_mechanisms piu.piu_v1.pi_mechanisms -#define pi_logical_provider_count piu.piu_v1.pi_logical_provider_count -#define pi_logical_providers piu.piu_v1.pi_logical_providers -#define pi_flags piu.piu_v2.pi_flags - -/* hidden providers can only be accessed via a logical provider */ -#define CRYPTO_HIDE_PROVIDER 0x00000001 -/* - * provider can not do multi-part digest (updates) and has a limit - * on maximum input data that it can digest. - */ -#define CRYPTO_HASH_NO_UPDATE 0x00000002 - -/* provider can handle the request without returning a CRYPTO_QUEUED */ -#define CRYPTO_SYNCHRONOUS 0x00000004 - -#define CRYPTO_PIFLAGS_RESERVED2 0x40000000 -#define CRYPTO_PIFLAGS_RESERVED1 0x80000000 - -/* - * Provider status passed by a provider to crypto_provider_notification(9F) - * and returned by the provider_status(9E) entry point. - */ -#define CRYPTO_PROVIDER_READY 0 -#define CRYPTO_PROVIDER_BUSY 1 -#define CRYPTO_PROVIDER_FAILED 2 - /* * Functions exported by Solaris to cryptographic providers. Providers * call these functions to register and unregister, notify the kernel @@ -714,9 +229,6 @@ typedef struct crypto_provider_info { extern int crypto_register_provider(const crypto_provider_info_t *, crypto_kcf_provider_handle_t *); extern int crypto_unregister_provider(crypto_kcf_provider_handle_t); -extern void crypto_provider_notification(crypto_kcf_provider_handle_t, uint_t); -extern void crypto_op_notification(crypto_req_handle_t, int); -extern int crypto_kmflag(crypto_req_handle_t); #ifdef __cplusplus diff --git a/module/icp/include/sys/ia32/asm_linkage.h b/module/icp/include/sys/ia32/asm_linkage.h index 71588cb5ce5f..58964c5d4497 100644 --- a/module/icp/include/sys/ia32/asm_linkage.h +++ b/module/icp/include/sys/ia32/asm_linkage.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -30,6 +30,12 @@ #include #include +#if defined(__linux__) && defined(CONFIG_SLS) +#define RET ret; int3 +#else +#define RET ret +#endif + #ifdef __cplusplus extern "C" { #endif diff --git a/module/icp/include/sys/ia32/stack.h b/module/icp/include/sys/ia32/stack.h index 9e7c089e1182..9ed327b343a6 100644 --- a/module/icp/include/sys/ia32/stack.h +++ b/module/icp/include/sys/ia32/stack.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/icp/include/sys/ia32/trap.h b/module/icp/include/sys/ia32/trap.h index 55b94969b80b..3d74266326b2 100644 --- a/module/icp/include/sys/ia32/trap.h +++ b/module/icp/include/sys/ia32/trap.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/icp/include/sys/modhash.h b/module/icp/include/sys/modhash.h deleted file mode 100644 index 06b52ff02604..000000000000 --- a/module/icp/include/sys/modhash.h +++ /dev/null @@ -1,147 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License (the "License"). - * You may not use this file except in compliance with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2008 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#ifndef _SYS_MODHASH_H -#define _SYS_MODHASH_H - -/* - * Generic hash implementation for the kernel. - */ - -#ifdef __cplusplus -extern "C" { -#endif - -#include - -/* - * Opaque data types for storing keys and values - */ -typedef void *mod_hash_val_t; -typedef void *mod_hash_key_t; - -/* - * Opaque data type for reservation - */ -typedef void *mod_hash_hndl_t; - -/* - * Opaque type for hash itself. - */ -struct mod_hash; -typedef struct mod_hash mod_hash_t; - -/* - * String hash table - */ -mod_hash_t *mod_hash_create_strhash_nodtr(char *, size_t, - void (*)(mod_hash_val_t)); -mod_hash_t *mod_hash_create_strhash(char *, size_t, void (*)(mod_hash_val_t)); -void mod_hash_destroy_strhash(mod_hash_t *); -int mod_hash_strkey_cmp(mod_hash_key_t, mod_hash_key_t); -void mod_hash_strkey_dtor(mod_hash_key_t); -void mod_hash_strval_dtor(mod_hash_val_t); -uint_t mod_hash_bystr(void *, mod_hash_key_t); - -/* - * Pointer hash table - */ -mod_hash_t *mod_hash_create_ptrhash(char *, size_t, void (*)(mod_hash_val_t), - size_t); -void mod_hash_destroy_ptrhash(mod_hash_t *); -int mod_hash_ptrkey_cmp(mod_hash_key_t, mod_hash_key_t); -uint_t mod_hash_byptr(void *, mod_hash_key_t); - -/* - * ID hash table - */ -mod_hash_t *mod_hash_create_idhash(char *, size_t, void (*)(mod_hash_val_t)); -void mod_hash_destroy_idhash(mod_hash_t *); -int mod_hash_idkey_cmp(mod_hash_key_t, mod_hash_key_t); -uint_t mod_hash_byid(void *, mod_hash_key_t); -uint_t mod_hash_iddata_gen(size_t); - -/* - * Hash management functions - */ -mod_hash_t *mod_hash_create_extended(char *, size_t, void (*)(mod_hash_key_t), - void (*)(mod_hash_val_t), uint_t (*)(void *, mod_hash_key_t), void *, - int (*)(mod_hash_key_t, mod_hash_key_t), int); - -void mod_hash_destroy_hash(mod_hash_t *); -void mod_hash_clear(mod_hash_t *); - -/* - * Null key and value destructors - */ -void mod_hash_null_keydtor(mod_hash_key_t); -void mod_hash_null_valdtor(mod_hash_val_t); - -/* - * Basic hash operations - */ - -/* - * Error codes for insert, remove, find, destroy. - */ -#define MH_ERR_NOMEM -1 -#define MH_ERR_DUPLICATE -2 -#define MH_ERR_NOTFOUND -3 - -/* - * Return codes for hash walkers - */ -#define MH_WALK_CONTINUE 0 -#define MH_WALK_TERMINATE 1 - -/* - * Basic hash operations - */ -int mod_hash_insert(mod_hash_t *, mod_hash_key_t, mod_hash_val_t); -int mod_hash_replace(mod_hash_t *, mod_hash_key_t, mod_hash_val_t); -int mod_hash_remove(mod_hash_t *, mod_hash_key_t, mod_hash_val_t *); -int mod_hash_destroy(mod_hash_t *, mod_hash_key_t); -int mod_hash_find(mod_hash_t *, mod_hash_key_t, mod_hash_val_t *); -int mod_hash_find_cb(mod_hash_t *, mod_hash_key_t, mod_hash_val_t *, - void (*)(mod_hash_key_t, mod_hash_val_t)); -int mod_hash_find_cb_rval(mod_hash_t *, mod_hash_key_t, mod_hash_val_t *, - int (*)(mod_hash_key_t, mod_hash_val_t), int *); -void mod_hash_walk(mod_hash_t *, - uint_t (*)(mod_hash_key_t, mod_hash_val_t *, void *), void *); - -/* - * Reserving hash operations - */ -int mod_hash_reserve(mod_hash_t *, mod_hash_hndl_t *); -int mod_hash_reserve_nosleep(mod_hash_t *, mod_hash_hndl_t *); -void mod_hash_cancel(mod_hash_t *, mod_hash_hndl_t *); -int mod_hash_insert_reserve(mod_hash_t *, mod_hash_key_t, mod_hash_val_t, - mod_hash_hndl_t); - -#ifdef __cplusplus -} -#endif - -#endif /* _SYS_MODHASH_H */ diff --git a/module/icp/include/sys/modhash_impl.h b/module/icp/include/sys/modhash_impl.h deleted file mode 100644 index 3130773aa196..000000000000 --- a/module/icp/include/sys/modhash_impl.h +++ /dev/null @@ -1,108 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License (the "License"). - * You may not use this file except in compliance with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2006 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -#ifndef _SYS_MODHASH_IMPL_H -#define _SYS_MODHASH_IMPL_H - -/* - * Internal details for the kernel's generic hash implementation. - */ - -#ifdef __cplusplus -extern "C" { -#endif - -#include -#include - -struct mod_hash_entry { - mod_hash_key_t mhe_key; /* stored hash key */ - mod_hash_val_t mhe_val; /* stored hash value */ - struct mod_hash_entry *mhe_next; /* next item in chain */ -}; - -struct mod_hash_stat { - ulong_t mhs_hit; /* tried a 'find' and it succeeded */ - ulong_t mhs_miss; /* tried a 'find' but it failed */ - ulong_t mhs_coll; /* occur when insert fails because of dup's */ - ulong_t mhs_nelems; /* total number of stored key/value pairs */ - ulong_t mhs_nomem; /* number of times kmem_alloc failed */ -}; - -struct mod_hash { - krwlock_t mh_contents; /* lock protecting contents */ - char *mh_name; /* hash name */ - int mh_sleep; /* kmem_alloc flag */ - size_t mh_nchains; /* # of elements in mh_entries */ - - /* key and val destructor */ - void (*mh_kdtor)(mod_hash_key_t); - void (*mh_vdtor)(mod_hash_val_t); - - /* key comparator */ - int (*mh_keycmp)(mod_hash_key_t, mod_hash_key_t); - - /* hash algorithm, and algorithm-private data */ - uint_t (*mh_hashalg)(void *, mod_hash_key_t); - void *mh_hashalg_data; - - struct mod_hash *mh_next; /* next hash in list */ - - struct mod_hash_stat mh_stat; - - struct mod_hash_entry *mh_entries[1]; -}; - -/* - * MH_SIZE() - * Compute the size of a mod_hash_t, in bytes, given the number of - * elements it contains. - */ -#define MH_SIZE(n) \ - (sizeof (mod_hash_t) + ((n) - 1) * (sizeof (struct mod_hash_entry *))) - -/* - * Module initialization; called once. - */ -void mod_hash_fini(void); -void mod_hash_init(void); - -/* - * Internal routines. Use directly with care. - */ -uint_t i_mod_hash(mod_hash_t *, mod_hash_key_t); -int i_mod_hash_insert_nosync(mod_hash_t *, mod_hash_key_t, mod_hash_val_t, - mod_hash_hndl_t); -int i_mod_hash_remove_nosync(mod_hash_t *, mod_hash_key_t, mod_hash_val_t *); -int i_mod_hash_find_nosync(mod_hash_t *, mod_hash_key_t, mod_hash_val_t *); -void i_mod_hash_walk_nosync(mod_hash_t *, uint_t (*)(mod_hash_key_t, - mod_hash_val_t *, void *), void *); -void i_mod_hash_clear_nosync(mod_hash_t *hash); - -#ifdef __cplusplus -} -#endif - -#endif /* _SYS_MODHASH_IMPL_H */ diff --git a/module/icp/include/sys/stack.h b/module/icp/include/sys/stack.h index 64fecf409b5c..0bace018b5ab 100644 --- a/module/icp/include/sys/stack.h +++ b/module/icp/include/sys/stack.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/icp/include/sys/trap.h b/module/icp/include/sys/trap.h index 7f9fd375805f..2f47d43939c1 100644 --- a/module/icp/include/sys/trap.h +++ b/module/icp/include/sys/trap.h @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/icp/io/aes.c b/module/icp/io/aes.c index e1bd17335911..6ff51f9b7eda 100644 --- a/module/icp/io/aes.c +++ b/module/icp/io/aes.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -43,72 +43,52 @@ static const crypto_mech_info_t aes_mech_info_tab[] = { /* AES_ECB */ {SUN_CKM_AES_ECB, AES_ECB_MECH_INFO_TYPE, CRYPTO_FG_ENCRYPT | CRYPTO_FG_ENCRYPT_ATOMIC | - CRYPTO_FG_DECRYPT | CRYPTO_FG_DECRYPT_ATOMIC, - AES_MIN_KEY_BYTES, AES_MAX_KEY_BYTES, CRYPTO_KEYSIZE_UNIT_IN_BYTES}, + CRYPTO_FG_DECRYPT | CRYPTO_FG_DECRYPT_ATOMIC}, /* AES_CBC */ {SUN_CKM_AES_CBC, AES_CBC_MECH_INFO_TYPE, CRYPTO_FG_ENCRYPT | CRYPTO_FG_ENCRYPT_ATOMIC | - CRYPTO_FG_DECRYPT | CRYPTO_FG_DECRYPT_ATOMIC, - AES_MIN_KEY_BYTES, AES_MAX_KEY_BYTES, CRYPTO_KEYSIZE_UNIT_IN_BYTES}, + CRYPTO_FG_DECRYPT | CRYPTO_FG_DECRYPT_ATOMIC}, /* AES_CTR */ {SUN_CKM_AES_CTR, AES_CTR_MECH_INFO_TYPE, CRYPTO_FG_ENCRYPT | CRYPTO_FG_ENCRYPT_ATOMIC | - CRYPTO_FG_DECRYPT | CRYPTO_FG_DECRYPT_ATOMIC, - AES_MIN_KEY_BYTES, AES_MAX_KEY_BYTES, CRYPTO_KEYSIZE_UNIT_IN_BYTES}, + CRYPTO_FG_DECRYPT | CRYPTO_FG_DECRYPT_ATOMIC}, /* AES_CCM */ {SUN_CKM_AES_CCM, AES_CCM_MECH_INFO_TYPE, CRYPTO_FG_ENCRYPT | CRYPTO_FG_ENCRYPT_ATOMIC | - CRYPTO_FG_DECRYPT | CRYPTO_FG_DECRYPT_ATOMIC, - AES_MIN_KEY_BYTES, AES_MAX_KEY_BYTES, CRYPTO_KEYSIZE_UNIT_IN_BYTES}, + CRYPTO_FG_DECRYPT | CRYPTO_FG_DECRYPT_ATOMIC}, /* AES_GCM */ {SUN_CKM_AES_GCM, AES_GCM_MECH_INFO_TYPE, CRYPTO_FG_ENCRYPT | CRYPTO_FG_ENCRYPT_ATOMIC | - CRYPTO_FG_DECRYPT | CRYPTO_FG_DECRYPT_ATOMIC, - AES_MIN_KEY_BYTES, AES_MAX_KEY_BYTES, CRYPTO_KEYSIZE_UNIT_IN_BYTES}, + CRYPTO_FG_DECRYPT | CRYPTO_FG_DECRYPT_ATOMIC}, /* AES_GMAC */ {SUN_CKM_AES_GMAC, AES_GMAC_MECH_INFO_TYPE, CRYPTO_FG_ENCRYPT | CRYPTO_FG_ENCRYPT_ATOMIC | CRYPTO_FG_DECRYPT | CRYPTO_FG_DECRYPT_ATOMIC | - CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC | - CRYPTO_FG_SIGN | CRYPTO_FG_SIGN_ATOMIC | - CRYPTO_FG_VERIFY | CRYPTO_FG_VERIFY_ATOMIC, - AES_MIN_KEY_BYTES, AES_MAX_KEY_BYTES, CRYPTO_KEYSIZE_UNIT_IN_BYTES} -}; - -static void aes_provider_status(crypto_provider_handle_t, uint_t *); - -static const crypto_control_ops_t aes_control_ops = { - aes_provider_status + CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC}, }; static int aes_encrypt_init(crypto_ctx_t *, crypto_mechanism_t *, - crypto_key_t *, crypto_spi_ctx_template_t, crypto_req_handle_t); + crypto_key_t *, crypto_spi_ctx_template_t); static int aes_decrypt_init(crypto_ctx_t *, crypto_mechanism_t *, - crypto_key_t *, crypto_spi_ctx_template_t, crypto_req_handle_t); + crypto_key_t *, crypto_spi_ctx_template_t); static int aes_common_init(crypto_ctx_t *, crypto_mechanism_t *, - crypto_key_t *, crypto_spi_ctx_template_t, crypto_req_handle_t, boolean_t); + crypto_key_t *, crypto_spi_ctx_template_t, boolean_t); static int aes_common_init_ctx(aes_ctx_t *, crypto_spi_ctx_template_t *, crypto_mechanism_t *, crypto_key_t *, int, boolean_t); -static int aes_encrypt_final(crypto_ctx_t *, crypto_data_t *, - crypto_req_handle_t); -static int aes_decrypt_final(crypto_ctx_t *, crypto_data_t *, - crypto_req_handle_t); +static int aes_encrypt_final(crypto_ctx_t *, crypto_data_t *); +static int aes_decrypt_final(crypto_ctx_t *, crypto_data_t *); -static int aes_encrypt(crypto_ctx_t *, crypto_data_t *, crypto_data_t *, - crypto_req_handle_t); +static int aes_encrypt(crypto_ctx_t *, crypto_data_t *, crypto_data_t *); static int aes_encrypt_update(crypto_ctx_t *, crypto_data_t *, - crypto_data_t *, crypto_req_handle_t); -static int aes_encrypt_atomic(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, - crypto_data_t *, crypto_spi_ctx_template_t, crypto_req_handle_t); + crypto_data_t *); +static int aes_encrypt_atomic(crypto_mechanism_t *, crypto_key_t *, + crypto_data_t *, crypto_data_t *, crypto_spi_ctx_template_t); -static int aes_decrypt(crypto_ctx_t *, crypto_data_t *, crypto_data_t *, - crypto_req_handle_t); +static int aes_decrypt(crypto_ctx_t *, crypto_data_t *, crypto_data_t *); static int aes_decrypt_update(crypto_ctx_t *, crypto_data_t *, - crypto_data_t *, crypto_req_handle_t); -static int aes_decrypt_atomic(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, - crypto_data_t *, crypto_spi_ctx_template_t, crypto_req_handle_t); + crypto_data_t *); +static int aes_decrypt_atomic(crypto_mechanism_t *, crypto_key_t *, + crypto_data_t *, crypto_data_t *, crypto_spi_ctx_template_t); static const crypto_cipher_ops_t aes_cipher_ops = { .encrypt_init = aes_encrypt_init, @@ -123,12 +103,10 @@ static const crypto_cipher_ops_t aes_cipher_ops = { .decrypt_atomic = aes_decrypt_atomic }; -static int aes_mac_atomic(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, crypto_data_t *, - crypto_spi_ctx_template_t, crypto_req_handle_t); -static int aes_mac_verify_atomic(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, crypto_data_t *, - crypto_spi_ctx_template_t, crypto_req_handle_t); +static int aes_mac_atomic(crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, + crypto_data_t *, crypto_spi_ctx_template_t); +static int aes_mac_verify_atomic(crypto_mechanism_t *, crypto_key_t *, + crypto_data_t *, crypto_data_t *, crypto_spi_ctx_template_t); static const crypto_mac_ops_t aes_mac_ops = { .mac_init = NULL, @@ -139,9 +117,8 @@ static const crypto_mac_ops_t aes_mac_ops = { .mac_verify_atomic = aes_mac_verify_atomic }; -static int aes_create_ctx_template(crypto_provider_handle_t, - crypto_mechanism_t *, crypto_key_t *, crypto_spi_ctx_template_t *, - size_t *, crypto_req_handle_t); +static int aes_create_ctx_template(crypto_mechanism_t *, crypto_key_t *, + crypto_spi_ctx_template_t *, size_t *); static int aes_free_context(crypto_ctx_t *); static const crypto_ctx_ops_t aes_ctx_ops = { @@ -149,32 +126,19 @@ static const crypto_ctx_ops_t aes_ctx_ops = { .free_context = aes_free_context }; -static const crypto_ops_t aes_crypto_ops = {{{{{ - &aes_control_ops, +static const crypto_ops_t aes_crypto_ops = { NULL, &aes_cipher_ops, &aes_mac_ops, - NULL, - NULL, - NULL, - NULL, - NULL, - NULL, - NULL, - NULL, - NULL, - &aes_ctx_ops -}}}}}; + &aes_ctx_ops, +}; -static const crypto_provider_info_t aes_prov_info = {{{{ - CRYPTO_SPI_VERSION_1, +static const crypto_provider_info_t aes_prov_info = { "AES Software Provider", - CRYPTO_SW_PROVIDER, - NULL, &aes_crypto_ops, sizeof (aes_mech_info_tab) / sizeof (crypto_mech_info_t), aes_mech_info_tab -}}}}; +}; static crypto_kcf_provider_handle_t aes_prov_handle = 0; static crypto_data_t null_crypto_data = { CRYPTO_DATA_RAW }; @@ -208,7 +172,7 @@ aes_mod_fini(void) } static int -aes_check_mech_param(crypto_mechanism_t *mechanism, aes_ctx_t **ctx, int kmflag) +aes_check_mech_param(crypto_mechanism_t *mechanism, aes_ctx_t **ctx) { void *p = NULL; boolean_t param_required = B_TRUE; @@ -250,7 +214,7 @@ aes_check_mech_param(crypto_mechanism_t *mechanism, aes_ctx_t **ctx, int kmflag) rv = CRYPTO_MECHANISM_PARAM_INVALID; } if (ctx != NULL) { - p = (alloc_fun)(kmflag); + p = (alloc_fun)(KM_SLEEP); *ctx = p; } return (rv); @@ -262,52 +226,31 @@ aes_check_mech_param(crypto_mechanism_t *mechanism, aes_ctx_t **ctx, int kmflag) static int init_keysched(crypto_key_t *key, void *newbie) { - /* - * Only keys by value are supported by this module. - */ - switch (key->ck_format) { - case CRYPTO_KEY_RAW: - if (key->ck_length < AES_MINBITS || - key->ck_length > AES_MAXBITS) { - return (CRYPTO_KEY_SIZE_RANGE); - } - - /* key length must be either 128, 192, or 256 */ - if ((key->ck_length & 63) != 0) - return (CRYPTO_KEY_SIZE_RANGE); - break; - default: - return (CRYPTO_KEY_TYPE_INCONSISTENT); + if (key->ck_length < AES_MINBITS || + key->ck_length > AES_MAXBITS) { + return (CRYPTO_KEY_SIZE_RANGE); } + /* key length must be either 128, 192, or 256 */ + if ((key->ck_length & 63) != 0) + return (CRYPTO_KEY_SIZE_RANGE); + aes_init_keysched(key->ck_data, key->ck_length, newbie); return (CRYPTO_SUCCESS); } -/* - * KCF software provider control entry points. - */ -static void -aes_provider_status(crypto_provider_handle_t provider, uint_t *status) -{ - (void) provider; - *status = CRYPTO_PROVIDER_READY; -} - static int aes_encrypt_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, - crypto_key_t *key, crypto_spi_ctx_template_t template, - crypto_req_handle_t req) + crypto_key_t *key, crypto_spi_ctx_template_t template) { - return (aes_common_init(ctx, mechanism, key, template, req, B_TRUE)); + return (aes_common_init(ctx, mechanism, key, template, B_TRUE)); } static int aes_decrypt_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, - crypto_key_t *key, crypto_spi_ctx_template_t template, - crypto_req_handle_t req) + crypto_key_t *key, crypto_spi_ctx_template_t template) { - return (aes_common_init(ctx, mechanism, key, template, req, B_FALSE)); + return (aes_common_init(ctx, mechanism, key, template, B_FALSE)); } @@ -318,25 +261,16 @@ aes_decrypt_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, static int aes_common_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, crypto_key_t *key, crypto_spi_ctx_template_t template, - crypto_req_handle_t req, boolean_t is_encrypt_init) + boolean_t is_encrypt_init) { aes_ctx_t *aes_ctx; int rv; - int kmflag; - /* - * Only keys by value are supported by this module. - */ - if (key->ck_format != CRYPTO_KEY_RAW) { - return (CRYPTO_KEY_TYPE_INCONSISTENT); - } - - kmflag = crypto_kmflag(req); - if ((rv = aes_check_mech_param(mechanism, &aes_ctx, kmflag)) + if ((rv = aes_check_mech_param(mechanism, &aes_ctx)) != CRYPTO_SUCCESS) return (rv); - rv = aes_common_init_ctx(aes_ctx, template, mechanism, key, kmflag, + rv = aes_common_init_ctx(aes_ctx, template, mechanism, key, KM_SLEEP, is_encrypt_init); if (rv != CRYPTO_SUCCESS) { crypto_free_mode_ctx(aes_ctx); @@ -366,7 +300,7 @@ aes_copy_block64(uint8_t *in, uint64_t *out) static int aes_encrypt(crypto_ctx_t *ctx, crypto_data_t *plaintext, - crypto_data_t *ciphertext, crypto_req_handle_t req) + crypto_data_t *ciphertext) { int ret = CRYPTO_FAILED; @@ -418,7 +352,7 @@ aes_encrypt(crypto_ctx_t *ctx, crypto_data_t *plaintext, /* * Do an update on the specified input data. */ - ret = aes_encrypt_update(ctx, plaintext, ciphertext, req); + ret = aes_encrypt_update(ctx, plaintext, ciphertext); if (ret != CRYPTO_SUCCESS) { return (ret); } @@ -481,7 +415,7 @@ aes_encrypt(crypto_ctx_t *ctx, crypto_data_t *plaintext, static int aes_decrypt(crypto_ctx_t *ctx, crypto_data_t *ciphertext, - crypto_data_t *plaintext, crypto_req_handle_t req) + crypto_data_t *plaintext) { int ret = CRYPTO_FAILED; @@ -539,7 +473,7 @@ aes_decrypt(crypto_ctx_t *ctx, crypto_data_t *ciphertext, /* * Do an update on the specified input data. */ - ret = aes_decrypt_update(ctx, ciphertext, plaintext, req); + ret = aes_decrypt_update(ctx, ciphertext, plaintext); if (ret != CRYPTO_SUCCESS) { goto cleanup; } @@ -595,9 +529,8 @@ aes_decrypt(crypto_ctx_t *ctx, crypto_data_t *ciphertext, static int aes_encrypt_update(crypto_ctx_t *ctx, crypto_data_t *plaintext, - crypto_data_t *ciphertext, crypto_req_handle_t req) + crypto_data_t *ciphertext) { - (void) req; off_t saved_offset; size_t saved_length, out_len; int ret = CRYPTO_SUCCESS; @@ -628,13 +561,11 @@ aes_encrypt_update(crypto_ctx_t *ctx, crypto_data_t *plaintext, switch (plaintext->cd_format) { case CRYPTO_DATA_RAW: ret = crypto_update_iov(ctx->cc_provider_private, - plaintext, ciphertext, aes_encrypt_contiguous_blocks, - aes_copy_block64); + plaintext, ciphertext, aes_encrypt_contiguous_blocks); break; case CRYPTO_DATA_UIO: ret = crypto_update_uio(ctx->cc_provider_private, - plaintext, ciphertext, aes_encrypt_contiguous_blocks, - aes_copy_block64); + plaintext, ciphertext, aes_encrypt_contiguous_blocks); break; default: ret = CRYPTO_ARGUMENTS_BAD; @@ -666,7 +597,7 @@ aes_encrypt_update(crypto_ctx_t *ctx, crypto_data_t *plaintext, static int aes_decrypt_update(crypto_ctx_t *ctx, crypto_data_t *ciphertext, - crypto_data_t *plaintext, crypto_req_handle_t req) + crypto_data_t *plaintext) { off_t saved_offset; size_t saved_length, out_len; @@ -698,22 +629,17 @@ aes_decrypt_update(crypto_ctx_t *ctx, crypto_data_t *ciphertext, saved_offset = plaintext->cd_offset; saved_length = plaintext->cd_length; - if (aes_ctx->ac_flags & (GCM_MODE|GMAC_MODE)) - gcm_set_kmflag((gcm_ctx_t *)aes_ctx, crypto_kmflag(req)); - /* * Do the AES update on the specified input data. */ switch (ciphertext->cd_format) { case CRYPTO_DATA_RAW: ret = crypto_update_iov(ctx->cc_provider_private, - ciphertext, plaintext, aes_decrypt_contiguous_blocks, - aes_copy_block64); + ciphertext, plaintext, aes_decrypt_contiguous_blocks); break; case CRYPTO_DATA_UIO: ret = crypto_update_uio(ctx->cc_provider_private, - ciphertext, plaintext, aes_decrypt_contiguous_blocks, - aes_copy_block64); + ciphertext, plaintext, aes_decrypt_contiguous_blocks); break; default: ret = CRYPTO_ARGUMENTS_BAD; @@ -746,10 +672,8 @@ aes_decrypt_update(crypto_ctx_t *ctx, crypto_data_t *ciphertext, } static int -aes_encrypt_final(crypto_ctx_t *ctx, crypto_data_t *data, - crypto_req_handle_t req) +aes_encrypt_final(crypto_ctx_t *ctx, crypto_data_t *data) { - (void) req; aes_ctx_t *aes_ctx; int ret; @@ -803,10 +727,8 @@ aes_encrypt_final(crypto_ctx_t *ctx, crypto_data_t *data, } static int -aes_decrypt_final(crypto_ctx_t *ctx, crypto_data_t *data, - crypto_req_handle_t req) +aes_decrypt_final(crypto_ctx_t *ctx, crypto_data_t *data) { - (void) req; aes_ctx_t *aes_ctx; int ret; off_t saved_offset; @@ -906,13 +828,11 @@ aes_decrypt_final(crypto_ctx_t *ctx, crypto_data_t *data, } static int -aes_encrypt_atomic(crypto_provider_handle_t provider, - crypto_session_id_t session_id, crypto_mechanism_t *mechanism, +aes_encrypt_atomic(crypto_mechanism_t *mechanism, crypto_key_t *key, crypto_data_t *plaintext, crypto_data_t *ciphertext, - crypto_spi_ctx_template_t template, crypto_req_handle_t req) + crypto_spi_ctx_template_t template) { - (void) provider, (void) session_id; - aes_ctx_t aes_ctx; /* on the stack */ + aes_ctx_t aes_ctx = {{{{0}}}}; off_t saved_offset; size_t saved_length; size_t length_needed; @@ -935,13 +855,11 @@ aes_encrypt_atomic(crypto_provider_handle_t provider, return (CRYPTO_DATA_LEN_RANGE); } - if ((ret = aes_check_mech_param(mechanism, NULL, 0)) != CRYPTO_SUCCESS) + if ((ret = aes_check_mech_param(mechanism, NULL)) != CRYPTO_SUCCESS) return (ret); - bzero(&aes_ctx, sizeof (aes_ctx_t)); - ret = aes_common_init_ctx(&aes_ctx, template, mechanism, key, - crypto_kmflag(req), B_TRUE); + KM_SLEEP, B_TRUE); if (ret != CRYPTO_SUCCESS) return (ret); @@ -976,11 +894,11 @@ aes_encrypt_atomic(crypto_provider_handle_t provider, switch (plaintext->cd_format) { case CRYPTO_DATA_RAW: ret = crypto_update_iov(&aes_ctx, plaintext, ciphertext, - aes_encrypt_contiguous_blocks, aes_copy_block64); + aes_encrypt_contiguous_blocks); break; case CRYPTO_DATA_UIO: ret = crypto_update_uio(&aes_ctx, plaintext, ciphertext, - aes_encrypt_contiguous_blocks, aes_copy_block64); + aes_encrypt_contiguous_blocks); break; default: ret = CRYPTO_ARGUMENTS_BAD; @@ -1024,7 +942,7 @@ aes_encrypt_atomic(crypto_provider_handle_t provider, out: if (aes_ctx.ac_flags & PROVIDER_OWNS_KEY_SCHEDULE) { - bzero(aes_ctx.ac_keysched, aes_ctx.ac_keysched_len); + memset(aes_ctx.ac_keysched, 0, aes_ctx.ac_keysched_len); kmem_free(aes_ctx.ac_keysched, aes_ctx.ac_keysched_len); } #ifdef CAN_USE_GCM_ASM @@ -1033,7 +951,7 @@ aes_encrypt_atomic(crypto_provider_handle_t provider, gcm_ctx_t *ctx = (gcm_ctx_t *)&aes_ctx; - bzero(ctx->gcm_Htable, ctx->gcm_htab_len); + memset(ctx->gcm_Htable, 0, ctx->gcm_htab_len); kmem_free(ctx->gcm_Htable, ctx->gcm_htab_len); } #endif @@ -1042,13 +960,11 @@ aes_encrypt_atomic(crypto_provider_handle_t provider, } static int -aes_decrypt_atomic(crypto_provider_handle_t provider, - crypto_session_id_t session_id, crypto_mechanism_t *mechanism, +aes_decrypt_atomic(crypto_mechanism_t *mechanism, crypto_key_t *key, crypto_data_t *ciphertext, crypto_data_t *plaintext, - crypto_spi_ctx_template_t template, crypto_req_handle_t req) + crypto_spi_ctx_template_t template) { - (void) provider, (void) session_id; - aes_ctx_t aes_ctx; /* on the stack */ + aes_ctx_t aes_ctx = {{{{0}}}}; off_t saved_offset; size_t saved_length; size_t length_needed; @@ -1071,13 +987,11 @@ aes_decrypt_atomic(crypto_provider_handle_t provider, return (CRYPTO_ENCRYPTED_DATA_LEN_RANGE); } - if ((ret = aes_check_mech_param(mechanism, NULL, 0)) != CRYPTO_SUCCESS) + if ((ret = aes_check_mech_param(mechanism, NULL)) != CRYPTO_SUCCESS) return (ret); - bzero(&aes_ctx, sizeof (aes_ctx_t)); - ret = aes_common_init_ctx(&aes_ctx, template, mechanism, key, - crypto_kmflag(req), B_FALSE); + KM_SLEEP, B_FALSE); if (ret != CRYPTO_SUCCESS) return (ret); @@ -1107,21 +1021,17 @@ aes_decrypt_atomic(crypto_provider_handle_t provider, saved_offset = plaintext->cd_offset; saved_length = plaintext->cd_length; - if (mechanism->cm_type == AES_GCM_MECH_INFO_TYPE || - mechanism->cm_type == AES_GMAC_MECH_INFO_TYPE) - gcm_set_kmflag((gcm_ctx_t *)&aes_ctx, crypto_kmflag(req)); - /* * Do an update on the specified input data. */ switch (ciphertext->cd_format) { case CRYPTO_DATA_RAW: ret = crypto_update_iov(&aes_ctx, ciphertext, plaintext, - aes_decrypt_contiguous_blocks, aes_copy_block64); + aes_decrypt_contiguous_blocks); break; case CRYPTO_DATA_UIO: ret = crypto_update_uio(&aes_ctx, ciphertext, plaintext, - aes_decrypt_contiguous_blocks, aes_copy_block64); + aes_decrypt_contiguous_blocks); break; default: ret = CRYPTO_ARGUMENTS_BAD; @@ -1182,7 +1092,7 @@ aes_decrypt_atomic(crypto_provider_handle_t provider, out: if (aes_ctx.ac_flags & PROVIDER_OWNS_KEY_SCHEDULE) { - bzero(aes_ctx.ac_keysched, aes_ctx.ac_keysched_len); + memset(aes_ctx.ac_keysched, 0, aes_ctx.ac_keysched_len); kmem_free(aes_ctx.ac_keysched, aes_ctx.ac_keysched_len); } @@ -1199,7 +1109,7 @@ aes_decrypt_atomic(crypto_provider_handle_t provider, if (((gcm_ctx_t *)&aes_ctx)->gcm_Htable != NULL) { gcm_ctx_t *ctx = (gcm_ctx_t *)&aes_ctx; - bzero(ctx->gcm_Htable, ctx->gcm_htab_len); + memset(ctx->gcm_Htable, 0, ctx->gcm_htab_len); kmem_free(ctx->gcm_Htable, ctx->gcm_htab_len); } #endif @@ -1212,11 +1122,9 @@ aes_decrypt_atomic(crypto_provider_handle_t provider, * KCF software provider context template entry points. */ static int -aes_create_ctx_template(crypto_provider_handle_t provider, - crypto_mechanism_t *mechanism, crypto_key_t *key, - crypto_spi_ctx_template_t *tmpl, size_t *tmpl_size, crypto_req_handle_t req) +aes_create_ctx_template(crypto_mechanism_t *mechanism, crypto_key_t *key, + crypto_spi_ctx_template_t *tmpl, size_t *tmpl_size) { - (void) provider; void *keysched; size_t size; int rv; @@ -1229,8 +1137,7 @@ aes_create_ctx_template(crypto_provider_handle_t provider, mechanism->cm_type != AES_GMAC_MECH_INFO_TYPE) return (CRYPTO_MECHANISM_INVALID); - if ((keysched = aes_alloc_keysched(&size, - crypto_kmflag(req))) == NULL) { + if ((keysched = aes_alloc_keysched(&size, KM_SLEEP)) == NULL) { return (CRYPTO_HOST_MEMORY); } @@ -1239,7 +1146,7 @@ aes_create_ctx_template(crypto_provider_handle_t provider, * in the key. */ if ((rv = init_keysched(key, keysched)) != CRYPTO_SUCCESS) { - bzero(keysched, size); + memset(keysched, 0, size); kmem_free(keysched, size); return (rv); } @@ -1259,7 +1166,8 @@ aes_free_context(crypto_ctx_t *ctx) if (aes_ctx != NULL) { if (aes_ctx->ac_flags & PROVIDER_OWNS_KEY_SCHEDULE) { ASSERT(aes_ctx->ac_keysched_len != 0); - bzero(aes_ctx->ac_keysched, aes_ctx->ac_keysched_len); + memset(aes_ctx->ac_keysched, 0, + aes_ctx->ac_keysched_len); kmem_free(aes_ctx->ac_keysched, aes_ctx->ac_keysched_len); } @@ -1349,7 +1257,7 @@ aes_common_init_ctx(aes_ctx_t *aes_ctx, crypto_spi_ctx_template_t *template, if (rv != CRYPTO_SUCCESS) { if (aes_ctx->ac_flags & PROVIDER_OWNS_KEY_SCHEDULE) { - bzero(keysched, size); + memset(keysched, 0, size); kmem_free(keysched, size); } } @@ -1389,10 +1297,9 @@ process_gmac_mech(crypto_mechanism_t *mech, crypto_data_t *data, } static int -aes_mac_atomic(crypto_provider_handle_t provider, - crypto_session_id_t session_id, crypto_mechanism_t *mechanism, +aes_mac_atomic(crypto_mechanism_t *mechanism, crypto_key_t *key, crypto_data_t *data, crypto_data_t *mac, - crypto_spi_ctx_template_t template, crypto_req_handle_t req) + crypto_spi_ctx_template_t template) { CK_AES_GCM_PARAMS gcm_params; crypto_mechanism_t gcm_mech; @@ -1406,15 +1313,13 @@ aes_mac_atomic(crypto_provider_handle_t provider, gcm_mech.cm_param_len = sizeof (CK_AES_GCM_PARAMS); gcm_mech.cm_param = (char *)&gcm_params; - return (aes_encrypt_atomic(provider, session_id, &gcm_mech, - key, &null_crypto_data, mac, template, req)); + return (aes_encrypt_atomic(&gcm_mech, + key, &null_crypto_data, mac, template)); } static int -aes_mac_verify_atomic(crypto_provider_handle_t provider, - crypto_session_id_t session_id, crypto_mechanism_t *mechanism, - crypto_key_t *key, crypto_data_t *data, crypto_data_t *mac, - crypto_spi_ctx_template_t template, crypto_req_handle_t req) +aes_mac_verify_atomic(crypto_mechanism_t *mechanism, crypto_key_t *key, + crypto_data_t *data, crypto_data_t *mac, crypto_spi_ctx_template_t template) { CK_AES_GCM_PARAMS gcm_params; crypto_mechanism_t gcm_mech; @@ -1428,6 +1333,6 @@ aes_mac_verify_atomic(crypto_provider_handle_t provider, gcm_mech.cm_param_len = sizeof (CK_AES_GCM_PARAMS); gcm_mech.cm_param = (char *)&gcm_params; - return (aes_decrypt_atomic(provider, session_id, &gcm_mech, - key, mac, &null_crypto_data, template, req)); + return (aes_decrypt_atomic(&gcm_mech, + key, mac, &null_crypto_data, template)); } diff --git a/module/icp/io/sha2_mod.c b/module/icp/io/sha2_mod.c index 7ee16e1372ad..fadb58b81881 100644 --- a/module/icp/io/sha2_mod.c +++ b/module/icp/io/sha2_mod.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -46,7 +46,7 @@ (len) = (uint32_t)*((ulong_t *)(m)->cm_param); \ else { \ ulong_t tmp_ulong; \ - bcopy((m)->cm_param, &tmp_ulong, sizeof (ulong_t)); \ + memcpy(&tmp_ulong, (m)->cm_param, sizeof (ulong_t)); \ (len) = (uint32_t)tmp_ulong; \ } \ } @@ -63,86 +63,56 @@ static const crypto_mech_info_t sha2_mech_info_tab[] = { /* SHA256 */ {SUN_CKM_SHA256, SHA256_MECH_INFO_TYPE, - CRYPTO_FG_DIGEST | CRYPTO_FG_DIGEST_ATOMIC, - 0, 0, CRYPTO_KEYSIZE_UNIT_IN_BITS}, + CRYPTO_FG_DIGEST | CRYPTO_FG_DIGEST_ATOMIC}, /* SHA256-HMAC */ {SUN_CKM_SHA256_HMAC, SHA256_HMAC_MECH_INFO_TYPE, - CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC, - SHA2_HMAC_MIN_KEY_LEN, SHA2_HMAC_MAX_KEY_LEN, - CRYPTO_KEYSIZE_UNIT_IN_BYTES}, + CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC}, /* SHA256-HMAC GENERAL */ {SUN_CKM_SHA256_HMAC_GENERAL, SHA256_HMAC_GEN_MECH_INFO_TYPE, - CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC, - SHA2_HMAC_MIN_KEY_LEN, SHA2_HMAC_MAX_KEY_LEN, - CRYPTO_KEYSIZE_UNIT_IN_BYTES}, + CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC}, /* SHA384 */ {SUN_CKM_SHA384, SHA384_MECH_INFO_TYPE, - CRYPTO_FG_DIGEST | CRYPTO_FG_DIGEST_ATOMIC, - 0, 0, CRYPTO_KEYSIZE_UNIT_IN_BITS}, + CRYPTO_FG_DIGEST | CRYPTO_FG_DIGEST_ATOMIC}, /* SHA384-HMAC */ {SUN_CKM_SHA384_HMAC, SHA384_HMAC_MECH_INFO_TYPE, - CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC, - SHA2_HMAC_MIN_KEY_LEN, SHA2_HMAC_MAX_KEY_LEN, - CRYPTO_KEYSIZE_UNIT_IN_BYTES}, + CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC}, /* SHA384-HMAC GENERAL */ {SUN_CKM_SHA384_HMAC_GENERAL, SHA384_HMAC_GEN_MECH_INFO_TYPE, - CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC, - SHA2_HMAC_MIN_KEY_LEN, SHA2_HMAC_MAX_KEY_LEN, - CRYPTO_KEYSIZE_UNIT_IN_BYTES}, + CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC}, /* SHA512 */ {SUN_CKM_SHA512, SHA512_MECH_INFO_TYPE, - CRYPTO_FG_DIGEST | CRYPTO_FG_DIGEST_ATOMIC, - 0, 0, CRYPTO_KEYSIZE_UNIT_IN_BITS}, + CRYPTO_FG_DIGEST | CRYPTO_FG_DIGEST_ATOMIC}, /* SHA512-HMAC */ {SUN_CKM_SHA512_HMAC, SHA512_HMAC_MECH_INFO_TYPE, - CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC, - SHA2_HMAC_MIN_KEY_LEN, SHA2_HMAC_MAX_KEY_LEN, - CRYPTO_KEYSIZE_UNIT_IN_BYTES}, + CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC}, /* SHA512-HMAC GENERAL */ {SUN_CKM_SHA512_HMAC_GENERAL, SHA512_HMAC_GEN_MECH_INFO_TYPE, - CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC, - SHA2_HMAC_MIN_KEY_LEN, SHA2_HMAC_MAX_KEY_LEN, - CRYPTO_KEYSIZE_UNIT_IN_BYTES} + CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC}, }; -static void sha2_provider_status(crypto_provider_handle_t, uint_t *); - -static const crypto_control_ops_t sha2_control_ops = { - sha2_provider_status -}; - -static int sha2_digest_init(crypto_ctx_t *, crypto_mechanism_t *, - crypto_req_handle_t); -static int sha2_digest(crypto_ctx_t *, crypto_data_t *, crypto_data_t *, - crypto_req_handle_t); -static int sha2_digest_update(crypto_ctx_t *, crypto_data_t *, - crypto_req_handle_t); -static int sha2_digest_final(crypto_ctx_t *, crypto_data_t *, - crypto_req_handle_t); -static int sha2_digest_atomic(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_data_t *, crypto_data_t *, - crypto_req_handle_t); +static int sha2_digest_init(crypto_ctx_t *, crypto_mechanism_t *); +static int sha2_digest(crypto_ctx_t *, crypto_data_t *, crypto_data_t *); +static int sha2_digest_update(crypto_ctx_t *, crypto_data_t *); +static int sha2_digest_final(crypto_ctx_t *, crypto_data_t *); +static int sha2_digest_atomic(crypto_mechanism_t *, crypto_data_t *, + crypto_data_t *); static const crypto_digest_ops_t sha2_digest_ops = { .digest_init = sha2_digest_init, .digest = sha2_digest, .digest_update = sha2_digest_update, - .digest_key = NULL, .digest_final = sha2_digest_final, .digest_atomic = sha2_digest_atomic }; static int sha2_mac_init(crypto_ctx_t *, crypto_mechanism_t *, crypto_key_t *, - crypto_spi_ctx_template_t, crypto_req_handle_t); -static int sha2_mac_update(crypto_ctx_t *, crypto_data_t *, - crypto_req_handle_t); -static int sha2_mac_final(crypto_ctx_t *, crypto_data_t *, crypto_req_handle_t); -static int sha2_mac_atomic(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, crypto_data_t *, - crypto_spi_ctx_template_t, crypto_req_handle_t); -static int sha2_mac_verify_atomic(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, crypto_data_t *, - crypto_spi_ctx_template_t, crypto_req_handle_t); + crypto_spi_ctx_template_t); +static int sha2_mac_update(crypto_ctx_t *, crypto_data_t *); +static int sha2_mac_final(crypto_ctx_t *, crypto_data_t *); +static int sha2_mac_atomic(crypto_mechanism_t *, crypto_key_t *, + crypto_data_t *, crypto_data_t *, crypto_spi_ctx_template_t); +static int sha2_mac_verify_atomic(crypto_mechanism_t *, crypto_key_t *, + crypto_data_t *, crypto_data_t *, crypto_spi_ctx_template_t); static const crypto_mac_ops_t sha2_mac_ops = { .mac_init = sha2_mac_init, @@ -153,9 +123,8 @@ static const crypto_mac_ops_t sha2_mac_ops = { .mac_verify_atomic = sha2_mac_verify_atomic }; -static int sha2_create_ctx_template(crypto_provider_handle_t, - crypto_mechanism_t *, crypto_key_t *, crypto_spi_ctx_template_t *, - size_t *, crypto_req_handle_t); +static int sha2_create_ctx_template(crypto_mechanism_t *, crypto_key_t *, + crypto_spi_ctx_template_t *, size_t *); static int sha2_free_context(crypto_ctx_t *); static const crypto_ctx_ops_t sha2_ctx_ops = { @@ -163,32 +132,19 @@ static const crypto_ctx_ops_t sha2_ctx_ops = { .free_context = sha2_free_context }; -static const crypto_ops_t sha2_crypto_ops = {{{{{ - &sha2_control_ops, +static const crypto_ops_t sha2_crypto_ops = { &sha2_digest_ops, NULL, &sha2_mac_ops, - NULL, - NULL, - NULL, - NULL, - NULL, - NULL, - NULL, - NULL, - NULL, - &sha2_ctx_ops -}}}}}; + &sha2_ctx_ops, +}; -static const crypto_provider_info_t sha2_prov_info = {{{{ - CRYPTO_SPI_VERSION_1, +static const crypto_provider_info_t sha2_prov_info = { "SHA2 Software Provider", - CRYPTO_SW_PROVIDER, - NULL, &sha2_crypto_ops, sizeof (sha2_mech_info_tab) / sizeof (crypto_mech_info_t), sha2_mech_info_tab -}}}}; +}; static crypto_kcf_provider_handle_t sha2_prov_handle = 0; @@ -229,30 +185,18 @@ sha2_mod_fini(void) return (ret); } -/* - * KCF software provider control entry points. - */ -static void -sha2_provider_status(crypto_provider_handle_t provider, uint_t *status) -{ - (void) provider; - *status = CRYPTO_PROVIDER_READY; -} - /* * KCF software provider digest entry points. */ static int -sha2_digest_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, - crypto_req_handle_t req) +sha2_digest_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism) { /* * Allocate and initialize SHA2 context. */ - ctx->cc_provider_private = kmem_alloc(sizeof (sha2_ctx_t), - crypto_kmflag(req)); + ctx->cc_provider_private = kmem_alloc(sizeof (sha2_ctx_t), KM_SLEEP); if (ctx->cc_provider_private == NULL) return (CRYPTO_HOST_MEMORY); @@ -365,9 +309,9 @@ sha2_digest_final_uio(SHA2_CTX *sha2_ctx, crypto_data_t *digest, */ SHA2Final(digest_scratch, sha2_ctx); - bcopy(digest_scratch, (uchar_t *) + memcpy((uchar_t *) zfs_uio_iovbase(digest->cd_uio, vec_idx) + offset, - digest_len); + digest_scratch, digest_len); } else { SHA2Final((uchar_t *)zfs_uio_iovbase(digest-> cd_uio, vec_idx) + offset, @@ -392,8 +336,9 @@ sha2_digest_final_uio(SHA2_CTX *sha2_ctx, crypto_data_t *digest, cur_len = MIN(zfs_uio_iovlen(digest->cd_uio, vec_idx) - offset, length); - bcopy(digest_tmp + scratch_offset, + memcpy( zfs_uio_iovbase(digest->cd_uio, vec_idx) + offset, + digest_tmp + scratch_offset, cur_len); length -= cur_len; @@ -417,10 +362,8 @@ sha2_digest_final_uio(SHA2_CTX *sha2_ctx, crypto_data_t *digest, } static int -sha2_digest(crypto_ctx_t *ctx, crypto_data_t *data, crypto_data_t *digest, - crypto_req_handle_t req) +sha2_digest(crypto_ctx_t *ctx, crypto_data_t *data, crypto_data_t *digest) { - (void) req; int ret = CRYPTO_SUCCESS; uint_t sha_digest_len; @@ -505,10 +448,8 @@ sha2_digest(crypto_ctx_t *ctx, crypto_data_t *data, crypto_data_t *digest, } static int -sha2_digest_update(crypto_ctx_t *ctx, crypto_data_t *data, - crypto_req_handle_t req) +sha2_digest_update(crypto_ctx_t *ctx, crypto_data_t *data) { - (void) req; int ret = CRYPTO_SUCCESS; ASSERT(ctx->cc_provider_private != NULL); @@ -534,10 +475,8 @@ sha2_digest_update(crypto_ctx_t *ctx, crypto_data_t *data, } static int -sha2_digest_final(crypto_ctx_t *ctx, crypto_data_t *digest, - crypto_req_handle_t req) +sha2_digest_final(crypto_ctx_t *ctx, crypto_data_t *digest) { - (void) req; int ret = CRYPTO_SUCCESS; uint_t sha_digest_len; @@ -597,12 +536,9 @@ sha2_digest_final(crypto_ctx_t *ctx, crypto_data_t *digest, } static int -sha2_digest_atomic(crypto_provider_handle_t provider, - crypto_session_id_t session_id, crypto_mechanism_t *mechanism, - crypto_data_t *data, crypto_data_t *digest, - crypto_req_handle_t req) +sha2_digest_atomic(crypto_mechanism_t *mechanism, crypto_data_t *data, + crypto_data_t *digest) { - (void) provider, (void) session_id, (void) req; int ret = CRYPTO_SUCCESS; SHA2_CTX sha2_ctx; uint32_t sha_digest_len; @@ -695,8 +631,8 @@ sha2_digest_atomic(crypto_provider_handle_t provider, static void sha2_mac_init_ctx(sha2_hmac_ctx_t *ctx, void *keyval, uint_t length_in_bytes) { - uint64_t ipad[SHA512_HMAC_BLOCK_SIZE / sizeof (uint64_t)]; - uint64_t opad[SHA512_HMAC_BLOCK_SIZE / sizeof (uint64_t)]; + uint64_t ipad[SHA512_HMAC_BLOCK_SIZE / sizeof (uint64_t)] = {0}; + uint64_t opad[SHA512_HMAC_BLOCK_SIZE / sizeof (uint64_t)] = {0}; int i, block_size, blocks_per_int64; /* Determine the block size */ @@ -708,12 +644,12 @@ sha2_mac_init_ctx(sha2_hmac_ctx_t *ctx, void *keyval, uint_t length_in_bytes) blocks_per_int64 = SHA512_HMAC_BLOCK_SIZE / sizeof (uint64_t); } - (void) bzero(ipad, block_size); - (void) bzero(opad, block_size); + (void) memset(ipad, 0, block_size); + (void) memset(opad, 0, block_size); if (keyval != NULL) { - (void) bcopy(keyval, ipad, length_in_bytes); - (void) bcopy(keyval, opad, length_in_bytes); + (void) memcpy(ipad, keyval, length_in_bytes); + (void) memcpy(opad, keyval, length_in_bytes); } else { ASSERT0(length_in_bytes); } @@ -731,15 +667,13 @@ sha2_mac_init_ctx(sha2_hmac_ctx_t *ctx, void *keyval, uint_t length_in_bytes) /* perform SHA2 on opad */ SHA2Init(ctx->hc_mech_type, &ctx->hc_ocontext); SHA2Update(&ctx->hc_ocontext, (uint8_t *)opad, block_size); - } /* */ static int sha2_mac_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, - crypto_key_t *key, crypto_spi_ctx_template_t ctx_template, - crypto_req_handle_t req) + crypto_key_t *key, crypto_spi_ctx_template_t ctx_template) { int ret = CRYPTO_SUCCESS; uint_t keylen_in_bytes = CRYPTO_BITS2BYTES(key->ck_length); @@ -766,18 +700,15 @@ sha2_mac_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, return (CRYPTO_MECHANISM_INVALID); } - if (key->ck_format != CRYPTO_KEY_RAW) - return (CRYPTO_ARGUMENTS_BAD); - - ctx->cc_provider_private = kmem_alloc(sizeof (sha2_hmac_ctx_t), - crypto_kmflag(req)); + ctx->cc_provider_private = + kmem_alloc(sizeof (sha2_hmac_ctx_t), KM_SLEEP); if (ctx->cc_provider_private == NULL) return (CRYPTO_HOST_MEMORY); PROV_SHA2_HMAC_CTX(ctx)->hc_mech_type = mechanism->cm_type; if (ctx_template != NULL) { /* reuse context template */ - bcopy(ctx_template, PROV_SHA2_HMAC_CTX(ctx), + memcpy(PROV_SHA2_HMAC_CTX(ctx), ctx_template, sizeof (sha2_hmac_ctx_t)); } else { /* no context template, compute context */ @@ -815,7 +746,7 @@ sha2_mac_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, } if (ret != CRYPTO_SUCCESS) { - bzero(ctx->cc_provider_private, sizeof (sha2_hmac_ctx_t)); + memset(ctx->cc_provider_private, 0, sizeof (sha2_hmac_ctx_t)); kmem_free(ctx->cc_provider_private, sizeof (sha2_hmac_ctx_t)); ctx->cc_provider_private = NULL; } @@ -824,10 +755,8 @@ sha2_mac_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, } static int -sha2_mac_update(crypto_ctx_t *ctx, crypto_data_t *data, - crypto_req_handle_t req) +sha2_mac_update(crypto_ctx_t *ctx, crypto_data_t *data) { - (void) req; int ret = CRYPTO_SUCCESS; ASSERT(ctx->cc_provider_private != NULL); @@ -854,9 +783,8 @@ sha2_mac_update(crypto_ctx_t *ctx, crypto_data_t *data, } static int -sha2_mac_final(crypto_ctx_t *ctx, crypto_data_t *mac, crypto_req_handle_t req) +sha2_mac_final(crypto_ctx_t *ctx, crypto_data_t *mac) { - (void) req; int ret = CRYPTO_SUCCESS; uchar_t digest[SHA512_DIGEST_LENGTH]; uint32_t digest_len, sha_digest_len; @@ -922,8 +850,8 @@ sha2_mac_final(crypto_ctx_t *ctx, crypto_data_t *mac, crypto_req_handle_t req) */ SHA2Final(digest, &PROV_SHA2_HMAC_CTX(ctx)->hc_ocontext); - bcopy(digest, (unsigned char *)mac->cd_raw.iov_base + - mac->cd_offset, digest_len); + memcpy((unsigned char *)mac->cd_raw.iov_base + + mac->cd_offset, digest, digest_len); } else { SHA2Final((unsigned char *)mac->cd_raw.iov_base + mac->cd_offset, @@ -944,7 +872,7 @@ sha2_mac_final(crypto_ctx_t *ctx, crypto_data_t *mac, crypto_req_handle_t req) else mac->cd_length = 0; - bzero(ctx->cc_provider_private, sizeof (sha2_hmac_ctx_t)); + memset(ctx->cc_provider_private, 0, sizeof (sha2_hmac_ctx_t)); kmem_free(ctx->cc_provider_private, sizeof (sha2_hmac_ctx_t)); ctx->cc_provider_private = NULL; @@ -967,12 +895,10 @@ sha2_mac_final(crypto_ctx_t *ctx, crypto_data_t *mac, crypto_req_handle_t req) } static int -sha2_mac_atomic(crypto_provider_handle_t provider, - crypto_session_id_t session_id, crypto_mechanism_t *mechanism, +sha2_mac_atomic(crypto_mechanism_t *mechanism, crypto_key_t *key, crypto_data_t *data, crypto_data_t *mac, - crypto_spi_ctx_template_t ctx_template, crypto_req_handle_t req) + crypto_spi_ctx_template_t ctx_template) { - (void) provider, (void) session_id, (void) req; int ret = CRYPTO_SUCCESS; uchar_t digest[SHA512_DIGEST_LENGTH]; sha2_hmac_ctx_t sha2_hmac_ctx; @@ -1000,13 +926,9 @@ sha2_mac_atomic(crypto_provider_handle_t provider, return (CRYPTO_MECHANISM_INVALID); } - /* Add support for key by attributes (RFE 4706552) */ - if (key->ck_format != CRYPTO_KEY_RAW) - return (CRYPTO_ARGUMENTS_BAD); - if (ctx_template != NULL) { /* reuse context template */ - bcopy(ctx_template, &sha2_hmac_ctx, sizeof (sha2_hmac_ctx_t)); + memcpy(&sha2_hmac_ctx, ctx_template, sizeof (sha2_hmac_ctx_t)); } else { sha2_hmac_ctx.hc_mech_type = mechanism->cm_type; /* no context template, initialize context */ @@ -1079,8 +1001,8 @@ sha2_mac_atomic(crypto_provider_handle_t provider, * the user only what was requested. */ SHA2Final(digest, &sha2_hmac_ctx.hc_ocontext); - bcopy(digest, (unsigned char *)mac->cd_raw.iov_base + - mac->cd_offset, digest_len); + memcpy((unsigned char *)mac->cd_raw.iov_base + + mac->cd_offset, digest, digest_len); } else { SHA2Final((unsigned char *)mac->cd_raw.iov_base + mac->cd_offset, &sha2_hmac_ctx.hc_ocontext); @@ -1099,18 +1021,16 @@ sha2_mac_atomic(crypto_provider_handle_t provider, return (CRYPTO_SUCCESS); } bail: - bzero(&sha2_hmac_ctx, sizeof (sha2_hmac_ctx_t)); + memset(&sha2_hmac_ctx, 0, sizeof (sha2_hmac_ctx_t)); mac->cd_length = 0; return (ret); } static int -sha2_mac_verify_atomic(crypto_provider_handle_t provider, - crypto_session_id_t session_id, crypto_mechanism_t *mechanism, +sha2_mac_verify_atomic(crypto_mechanism_t *mechanism, crypto_key_t *key, crypto_data_t *data, crypto_data_t *mac, - crypto_spi_ctx_template_t ctx_template, crypto_req_handle_t req) + crypto_spi_ctx_template_t ctx_template) { - (void) provider, (void) session_id, (void) req; int ret = CRYPTO_SUCCESS; uchar_t digest[SHA512_DIGEST_LENGTH]; sha2_hmac_ctx_t sha2_hmac_ctx; @@ -1138,13 +1058,9 @@ sha2_mac_verify_atomic(crypto_provider_handle_t provider, return (CRYPTO_MECHANISM_INVALID); } - /* Add support for key by attributes (RFE 4706552) */ - if (key->ck_format != CRYPTO_KEY_RAW) - return (CRYPTO_ARGUMENTS_BAD); - if (ctx_template != NULL) { /* reuse context template */ - bcopy(ctx_template, &sha2_hmac_ctx, sizeof (sha2_hmac_ctx_t)); + memcpy(&sha2_hmac_ctx, ctx_template, sizeof (sha2_hmac_ctx_t)); } else { sha2_hmac_ctx.hc_mech_type = mechanism->cm_type; /* no context template, initialize context */ @@ -1221,7 +1137,7 @@ sha2_mac_verify_atomic(crypto_provider_handle_t provider, switch (mac->cd_format) { case CRYPTO_DATA_RAW: - if (bcmp(digest, (unsigned char *)mac->cd_raw.iov_base + + if (memcmp(digest, (unsigned char *)mac->cd_raw.iov_base + mac->cd_offset, digest_len) != 0) ret = CRYPTO_INVALID_MAC; break; @@ -1254,7 +1170,7 @@ sha2_mac_verify_atomic(crypto_provider_handle_t provider, cur_len = MIN(zfs_uio_iovlen(mac->cd_uio, vec_idx) - offset, length); - if (bcmp(digest + scratch_offset, + if (memcmp(digest + scratch_offset, zfs_uio_iovbase(mac->cd_uio, vec_idx) + offset, cur_len) != 0) { ret = CRYPTO_INVALID_MAC; @@ -1275,7 +1191,7 @@ sha2_mac_verify_atomic(crypto_provider_handle_t provider, return (ret); bail: - bzero(&sha2_hmac_ctx, sizeof (sha2_hmac_ctx_t)); + memset(&sha2_hmac_ctx, 0, sizeof (sha2_hmac_ctx_t)); mac->cd_length = 0; return (ret); } @@ -1285,12 +1201,9 @@ sha2_mac_verify_atomic(crypto_provider_handle_t provider, */ static int -sha2_create_ctx_template(crypto_provider_handle_t provider, - crypto_mechanism_t *mechanism, crypto_key_t *key, - crypto_spi_ctx_template_t *ctx_template, size_t *ctx_template_size, - crypto_req_handle_t req) +sha2_create_ctx_template(crypto_mechanism_t *mechanism, crypto_key_t *key, + crypto_spi_ctx_template_t *ctx_template, size_t *ctx_template_size) { - (void) provider; sha2_hmac_ctx_t *sha2_hmac_ctx_tmpl; uint_t keylen_in_bytes = CRYPTO_BITS2BYTES(key->ck_length); uint32_t sha_digest_len, sha_hmac_block_size; @@ -1316,15 +1229,10 @@ sha2_create_ctx_template(crypto_provider_handle_t provider, return (CRYPTO_MECHANISM_INVALID); } - /* Add support for key by attributes (RFE 4706552) */ - if (key->ck_format != CRYPTO_KEY_RAW) - return (CRYPTO_ARGUMENTS_BAD); - /* * Allocate and initialize SHA2 context. */ - sha2_hmac_ctx_tmpl = kmem_alloc(sizeof (sha2_hmac_ctx_t), - crypto_kmflag(req)); + sha2_hmac_ctx_tmpl = kmem_alloc(sizeof (sha2_hmac_ctx_t), KM_SLEEP); if (sha2_hmac_ctx_tmpl == NULL) return (CRYPTO_HOST_MEMORY); @@ -1374,7 +1282,7 @@ sha2_free_context(crypto_ctx_t *ctx) else ctx_len = sizeof (sha2_hmac_ctx_t); - bzero(ctx->cc_provider_private, ctx_len); + memset(ctx->cc_provider_private, 0, ctx_len); kmem_free(ctx->cc_provider_private, ctx_len); ctx->cc_provider_private = NULL; diff --git a/module/icp/io/skein_mod.c b/module/icp/io/skein_mod.c index d0917e71b12e..a2ed6cedd8c6 100644 --- a/module/icp/io/skein_mod.c +++ b/module/icp/io/skein_mod.c @@ -32,55 +32,38 @@ static const crypto_mech_info_t skein_mech_info_tab[] = { {CKM_SKEIN_256, SKEIN_256_MECH_INFO_TYPE, - CRYPTO_FG_DIGEST | CRYPTO_FG_DIGEST_ATOMIC, - 0, 0, CRYPTO_KEYSIZE_UNIT_IN_BITS}, + CRYPTO_FG_DIGEST | CRYPTO_FG_DIGEST_ATOMIC}, {CKM_SKEIN_256_MAC, SKEIN_256_MAC_MECH_INFO_TYPE, - CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC, 1, INT_MAX, - CRYPTO_KEYSIZE_UNIT_IN_BYTES}, + CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC}, {CKM_SKEIN_512, SKEIN_512_MECH_INFO_TYPE, - CRYPTO_FG_DIGEST | CRYPTO_FG_DIGEST_ATOMIC, - 0, 0, CRYPTO_KEYSIZE_UNIT_IN_BITS}, + CRYPTO_FG_DIGEST | CRYPTO_FG_DIGEST_ATOMIC}, {CKM_SKEIN_512_MAC, SKEIN_512_MAC_MECH_INFO_TYPE, - CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC, 1, INT_MAX, - CRYPTO_KEYSIZE_UNIT_IN_BYTES}, + CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC}, {CKM_SKEIN1024, SKEIN1024_MECH_INFO_TYPE, - CRYPTO_FG_DIGEST | CRYPTO_FG_DIGEST_ATOMIC, - 0, 0, CRYPTO_KEYSIZE_UNIT_IN_BITS}, + CRYPTO_FG_DIGEST | CRYPTO_FG_DIGEST_ATOMIC}, {CKM_SKEIN1024_MAC, SKEIN1024_MAC_MECH_INFO_TYPE, - CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC, 1, INT_MAX, - CRYPTO_KEYSIZE_UNIT_IN_BYTES} + CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC}, }; -static void skein_provider_status(crypto_provider_handle_t, uint_t *); - -static const crypto_control_ops_t skein_control_ops = { - skein_provider_status -}; - -static int skein_digest_init(crypto_ctx_t *, crypto_mechanism_t *, - crypto_req_handle_t); -static int skein_digest(crypto_ctx_t *, crypto_data_t *, crypto_data_t *, - crypto_req_handle_t); -static int skein_update(crypto_ctx_t *, crypto_data_t *, crypto_req_handle_t); -static int skein_final(crypto_ctx_t *, crypto_data_t *, crypto_req_handle_t); -static int skein_digest_atomic(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_data_t *, crypto_data_t *, - crypto_req_handle_t); +static int skein_digest_init(crypto_ctx_t *, crypto_mechanism_t *); +static int skein_digest(crypto_ctx_t *, crypto_data_t *, crypto_data_t *); +static int skein_update(crypto_ctx_t *, crypto_data_t *); +static int skein_final(crypto_ctx_t *, crypto_data_t *); +static int skein_digest_atomic(crypto_mechanism_t *, crypto_data_t *, + crypto_data_t *); static const crypto_digest_ops_t skein_digest_ops = { .digest_init = skein_digest_init, .digest = skein_digest, .digest_update = skein_update, - .digest_key = NULL, .digest_final = skein_final, .digest_atomic = skein_digest_atomic }; static int skein_mac_init(crypto_ctx_t *, crypto_mechanism_t *, crypto_key_t *, - crypto_spi_ctx_template_t, crypto_req_handle_t); -static int skein_mac_atomic(crypto_provider_handle_t, crypto_session_id_t, - crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, crypto_data_t *, - crypto_spi_ctx_template_t, crypto_req_handle_t); + crypto_spi_ctx_template_t); +static int skein_mac_atomic(crypto_mechanism_t *, crypto_key_t *, + crypto_data_t *, crypto_data_t *, crypto_spi_ctx_template_t); static const crypto_mac_ops_t skein_mac_ops = { .mac_init = skein_mac_init, @@ -91,9 +74,8 @@ static const crypto_mac_ops_t skein_mac_ops = { .mac_verify_atomic = NULL }; -static int skein_create_ctx_template(crypto_provider_handle_t, - crypto_mechanism_t *, crypto_key_t *, crypto_spi_ctx_template_t *, - size_t *, crypto_req_handle_t); +static int skein_create_ctx_template(crypto_mechanism_t *, crypto_key_t *, + crypto_spi_ctx_template_t *, size_t *); static int skein_free_context(crypto_ctx_t *); static const crypto_ctx_ops_t skein_ctx_ops = { @@ -101,32 +83,19 @@ static const crypto_ctx_ops_t skein_ctx_ops = { .free_context = skein_free_context }; -static const crypto_ops_t skein_crypto_ops = {{{{{ - &skein_control_ops, +static const crypto_ops_t skein_crypto_ops = { &skein_digest_ops, NULL, &skein_mac_ops, - NULL, - NULL, - NULL, - NULL, - NULL, - NULL, - NULL, - NULL, - NULL, &skein_ctx_ops, -}}}}}; +}; -static const crypto_provider_info_t skein_prov_info = {{{{ - CRYPTO_SPI_VERSION_1, +static const crypto_provider_info_t skein_prov_info = { "Skein Software Provider", - CRYPTO_SW_PROVIDER, - NULL, &skein_crypto_ops, sizeof (skein_mech_info_tab) / sizeof (crypto_mech_info_t), skein_mech_info_tab -}}}}; +}; static crypto_kcf_provider_handle_t skein_prov_handle = 0; @@ -222,16 +191,6 @@ skein_mod_fini(void) return (0); } -/* - * KCF software provider control entry points. - */ -static void -skein_provider_status(crypto_provider_handle_t provider, uint_t *status) -{ - (void) provider; - *status = CRYPTO_PROVIDER_READY; -} - /* * General Skein hashing helper functions. */ @@ -293,8 +252,7 @@ skein_digest_update_uio(skein_ctx_t *ctx, const crypto_data_t *data) * Performs a Final on a context and writes to a uio digest output. */ static int -skein_digest_final_uio(skein_ctx_t *ctx, crypto_data_t *digest, - crypto_req_handle_t req) +skein_digest_final_uio(skein_ctx_t *ctx, crypto_data_t *digest) { off_t offset = digest->cd_offset; uint_t vec_idx = 0; @@ -327,15 +285,15 @@ skein_digest_final_uio(skein_ctx_t *ctx, crypto_data_t *digest, size_t cur_len; digest_tmp = kmem_alloc(CRYPTO_BITS2BYTES( - ctx->sc_digest_bitlen), crypto_kmflag(req)); + ctx->sc_digest_bitlen), KM_SLEEP); if (digest_tmp == NULL) return (CRYPTO_HOST_MEMORY); SKEIN_OP(ctx, Final, digest_tmp); while (vec_idx < zfs_uio_iovcnt(uio) && length > 0) { cur_len = MIN(zfs_uio_iovlen(uio, vec_idx) - offset, length); - bcopy(digest_tmp + scratch_offset, - zfs_uio_iovbase(uio, vec_idx) + offset, cur_len); + memcpy(zfs_uio_iovbase(uio, vec_idx) + offset, + digest_tmp + scratch_offset, cur_len); length -= cur_len; vec_idx++; @@ -371,16 +329,14 @@ skein_digest_final_uio(skein_ctx_t *ctx, crypto_data_t *digest, * for Skein-1024). */ static int -skein_digest_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, - crypto_req_handle_t req) +skein_digest_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism) { int error = CRYPTO_SUCCESS; if (!VALID_SKEIN_DIGEST_MECH(mechanism->cm_type)) return (CRYPTO_MECHANISM_INVALID); - SKEIN_CTX_LVALUE(ctx) = kmem_alloc(sizeof (*SKEIN_CTX(ctx)), - crypto_kmflag(req)); + SKEIN_CTX_LVALUE(ctx) = kmem_alloc(sizeof (*SKEIN_CTX(ctx)), KM_SLEEP); if (SKEIN_CTX(ctx) == NULL) return (CRYPTO_HOST_MEMORY); @@ -393,7 +349,7 @@ skein_digest_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, return (CRYPTO_SUCCESS); errout: - bzero(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); + memset(SKEIN_CTX(ctx), 0, sizeof (*SKEIN_CTX(ctx))); kmem_free(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); SKEIN_CTX_LVALUE(ctx) = NULL; return (error); @@ -405,8 +361,7 @@ skein_digest_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, * see what to pass here. */ static int -skein_digest(crypto_ctx_t *ctx, crypto_data_t *data, crypto_data_t *digest, - crypto_req_handle_t req) +skein_digest(crypto_ctx_t *ctx, crypto_data_t *data, crypto_data_t *digest) { int error = CRYPTO_SUCCESS; @@ -419,15 +374,15 @@ skein_digest(crypto_ctx_t *ctx, crypto_data_t *data, crypto_data_t *digest, return (CRYPTO_BUFFER_TOO_SMALL); } - error = skein_update(ctx, data, req); + error = skein_update(ctx, data); if (error != CRYPTO_SUCCESS) { - bzero(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); + memset(SKEIN_CTX(ctx), 0, sizeof (*SKEIN_CTX(ctx))); kmem_free(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); SKEIN_CTX_LVALUE(ctx) = NULL; digest->cd_length = 0; return (error); } - error = skein_final(ctx, digest, req); + error = skein_final(ctx, digest); return (error); } @@ -438,9 +393,8 @@ skein_digest(crypto_ctx_t *ctx, crypto_data_t *data, crypto_data_t *digest, * Supported input data formats are raw, uio and mblk. */ static int -skein_update(crypto_ctx_t *ctx, crypto_data_t *data, crypto_req_handle_t req) +skein_update(crypto_ctx_t *ctx, crypto_data_t *data) { - (void) req; int error = CRYPTO_SUCCESS; ASSERT(SKEIN_CTX(ctx) != NULL); @@ -467,7 +421,7 @@ skein_update(crypto_ctx_t *ctx, crypto_data_t *data, crypto_req_handle_t req) * Supported output digest formats are raw, uio and mblk. */ static int -skein_final(crypto_ctx_t *ctx, crypto_data_t *digest, crypto_req_handle_t req) +skein_final(crypto_ctx_t *ctx, crypto_data_t *digest) { int error = CRYPTO_SUCCESS; @@ -486,7 +440,7 @@ skein_final(crypto_ctx_t *ctx, crypto_data_t *digest, crypto_req_handle_t req) (uint8_t *)digest->cd_raw.iov_base + digest->cd_offset); break; case CRYPTO_DATA_UIO: - error = skein_digest_final_uio(SKEIN_CTX(ctx), digest, req); + error = skein_digest_final_uio(SKEIN_CTX(ctx), digest); break; default: error = CRYPTO_ARGUMENTS_BAD; @@ -498,7 +452,7 @@ skein_final(crypto_ctx_t *ctx, crypto_data_t *digest, crypto_req_handle_t req) else digest->cd_length = 0; - bzero(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); + memset(SKEIN_CTX(ctx), 0, sizeof (*SKEIN_CTX(ctx))); kmem_free(SKEIN_CTX(ctx), sizeof (*(SKEIN_CTX(ctx)))); SKEIN_CTX_LVALUE(ctx) = NULL; @@ -512,11 +466,9 @@ skein_final(crypto_ctx_t *ctx, crypto_data_t *digest, crypto_req_handle_t req) * Supported input/output formats are raw, uio and mblk. */ static int -skein_digest_atomic(crypto_provider_handle_t provider, - crypto_session_id_t session_id, crypto_mechanism_t *mechanism, - crypto_data_t *data, crypto_data_t *digest, crypto_req_handle_t req) +skein_digest_atomic(crypto_mechanism_t *mechanism, crypto_data_t *data, + crypto_data_t *digest) { - (void) provider, (void) session_id, (void) req; int error; skein_ctx_t skein_ctx; crypto_ctx_t ctx; @@ -531,9 +483,9 @@ skein_digest_atomic(crypto_provider_handle_t provider, goto out; SKEIN_OP(&skein_ctx, Init, skein_ctx.sc_digest_bitlen); - if ((error = skein_update(&ctx, data, digest)) != CRYPTO_SUCCESS) + if ((error = skein_update(&ctx, data)) != CRYPTO_SUCCESS) goto out; - if ((error = skein_final(&ctx, data, digest)) != CRYPTO_SUCCESS) + if ((error = skein_final(&ctx, data)) != CRYPTO_SUCCESS) goto out; out: @@ -542,7 +494,7 @@ skein_digest_atomic(crypto_provider_handle_t provider, CRYPTO_BITS2BYTES(skein_ctx.sc_digest_bitlen); else digest->cd_length = 0; - bzero(&skein_ctx, sizeof (skein_ctx)); + memset(&skein_ctx, 0, sizeof (skein_ctx)); return (error); } @@ -559,8 +511,6 @@ skein_mac_ctx_build(skein_ctx_t *ctx, crypto_mechanism_t *mechanism, if (!VALID_SKEIN_MAC_MECH(mechanism->cm_type)) return (CRYPTO_MECHANISM_INVALID); - if (key->ck_format != CRYPTO_KEY_RAW) - return (CRYPTO_ARGUMENTS_BAD); ctx->sc_mech_type = mechanism->cm_type; error = skein_get_digest_bitlen(mechanism, &ctx->sc_digest_bitlen); if (error != CRYPTO_SUCCESS) @@ -584,18 +534,16 @@ skein_mac_ctx_build(skein_ctx_t *ctx, crypto_mechanism_t *mechanism, */ static int skein_mac_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, - crypto_key_t *key, crypto_spi_ctx_template_t ctx_template, - crypto_req_handle_t req) + crypto_key_t *key, crypto_spi_ctx_template_t ctx_template) { int error; - SKEIN_CTX_LVALUE(ctx) = kmem_alloc(sizeof (*SKEIN_CTX(ctx)), - crypto_kmflag(req)); + SKEIN_CTX_LVALUE(ctx) = kmem_alloc(sizeof (*SKEIN_CTX(ctx)), KM_SLEEP); if (SKEIN_CTX(ctx) == NULL) return (CRYPTO_HOST_MEMORY); if (ctx_template != NULL) { - bcopy(ctx_template, SKEIN_CTX(ctx), + memcpy(SKEIN_CTX(ctx), ctx_template, sizeof (*SKEIN_CTX(ctx))); } else { error = skein_mac_ctx_build(SKEIN_CTX(ctx), mechanism, key); @@ -605,7 +553,7 @@ skein_mac_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, return (CRYPTO_SUCCESS); errout: - bzero(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); + memset(SKEIN_CTX(ctx), 0, sizeof (*SKEIN_CTX(ctx))); kmem_free(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); return (error); } @@ -620,34 +568,32 @@ skein_mac_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, * function as to those of the partial operations above. */ static int -skein_mac_atomic(crypto_provider_handle_t provider, - crypto_session_id_t session_id, crypto_mechanism_t *mechanism, +skein_mac_atomic(crypto_mechanism_t *mechanism, crypto_key_t *key, crypto_data_t *data, crypto_data_t *mac, - crypto_spi_ctx_template_t ctx_template, crypto_req_handle_t req) + crypto_spi_ctx_template_t ctx_template) { /* faux crypto context just for skein_digest_{update,final} */ - (void) provider, (void) session_id; - int error; + int error; crypto_ctx_t ctx; skein_ctx_t skein_ctx; SKEIN_CTX_LVALUE(&ctx) = &skein_ctx; if (ctx_template != NULL) { - bcopy(ctx_template, &skein_ctx, sizeof (skein_ctx)); + memcpy(&skein_ctx, ctx_template, sizeof (skein_ctx)); } else { error = skein_mac_ctx_build(&skein_ctx, mechanism, key); if (error != CRYPTO_SUCCESS) goto errout; } - if ((error = skein_update(&ctx, data, req)) != CRYPTO_SUCCESS) + if ((error = skein_update(&ctx, data)) != CRYPTO_SUCCESS) goto errout; - if ((error = skein_final(&ctx, mac, req)) != CRYPTO_SUCCESS) + if ((error = skein_final(&ctx, mac)) != CRYPTO_SUCCESS) goto errout; return (CRYPTO_SUCCESS); errout: - bzero(&skein_ctx, sizeof (skein_ctx)); + memset(&skein_ctx, 0, sizeof (skein_ctx)); return (error); } @@ -661,16 +607,13 @@ skein_mac_atomic(crypto_provider_handle_t provider, * skein_mac_init. */ static int -skein_create_ctx_template(crypto_provider_handle_t provider, - crypto_mechanism_t *mechanism, crypto_key_t *key, - crypto_spi_ctx_template_t *ctx_template, size_t *ctx_template_size, - crypto_req_handle_t req) +skein_create_ctx_template(crypto_mechanism_t *mechanism, crypto_key_t *key, + crypto_spi_ctx_template_t *ctx_template, size_t *ctx_template_size) { - (void) provider; int error; skein_ctx_t *ctx_tmpl; - ctx_tmpl = kmem_alloc(sizeof (*ctx_tmpl), crypto_kmflag(req)); + ctx_tmpl = kmem_alloc(sizeof (*ctx_tmpl), KM_SLEEP); if (ctx_tmpl == NULL) return (CRYPTO_HOST_MEMORY); error = skein_mac_ctx_build(ctx_tmpl, mechanism, key); @@ -681,7 +624,7 @@ skein_create_ctx_template(crypto_provider_handle_t provider, return (CRYPTO_SUCCESS); errout: - bzero(ctx_tmpl, sizeof (*ctx_tmpl)); + memset(ctx_tmpl, 0, sizeof (*ctx_tmpl)); kmem_free(ctx_tmpl, sizeof (*ctx_tmpl)); return (error); } @@ -693,7 +636,7 @@ static int skein_free_context(crypto_ctx_t *ctx) { if (SKEIN_CTX(ctx) != NULL) { - bzero(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); + memset(SKEIN_CTX(ctx), 0, sizeof (*SKEIN_CTX(ctx))); kmem_free(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); SKEIN_CTX_LVALUE(ctx) = NULL; } diff --git a/module/icp/os/modhash.c b/module/icp/os/modhash.c deleted file mode 100644 index 8bd06973eff1..000000000000 --- a/module/icp/os/modhash.c +++ /dev/null @@ -1,927 +0,0 @@ -/* - * CDDL HEADER START - * - * The contents of this file are subject to the terms of the - * Common Development and Distribution License (the "License"). - * You may not use this file except in compliance with the License. - * - * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. - * See the License for the specific language governing permissions - * and limitations under the License. - * - * When distributing Covered Code, include this CDDL HEADER in each - * file and include the License file at usr/src/OPENSOLARIS.LICENSE. - * If applicable, add the following below this CDDL HEADER, with the - * fields enclosed by brackets "[]" replaced with your own identifying - * information: Portions Copyright [yyyy] [name of copyright owner] - * - * CDDL HEADER END - */ -/* - * Copyright 2008 Sun Microsystems, Inc. All rights reserved. - * Use is subject to license terms. - */ - -/* - * mod_hash: flexible hash table implementation. - * - * This is a reasonably fast, reasonably flexible hash table implementation - * which features pluggable hash algorithms to support storing arbitrary keys - * and values. It is designed to handle small (< 100,000 items) amounts of - * data. The hash uses chaining to resolve collisions, and does not feature a - * mechanism to grow the hash. Care must be taken to pick nchains to be large - * enough for the application at hand, or lots of time will be wasted searching - * hash chains. - * - * The client of the hash is required to supply a number of items to support - * the various hash functions: - * - * - Destructor functions for the key and value being hashed. - * A destructor is responsible for freeing an object when the hash - * table is no longer storing it. Since keys and values can be of - * arbitrary type, separate destructors for keys & values are used. - * These may be mod_hash_null_keydtor and mod_hash_null_valdtor if no - * destructor is needed for either a key or value. - * - * - A hashing algorithm which returns a uint_t representing a hash index - * The number returned need _not_ be between 0 and nchains. The mod_hash - * code will take care of doing that. The second argument (after the - * key) to the hashing function is a void * that represents - * hash_alg_data-- this is provided so that the hashing algorithm can - * maintain some state across calls, or keep algorithm-specific - * constants associated with the hash table. - * - * A pointer-hashing and a string-hashing algorithm are supplied in - * this file. - * - * - A key comparator (a la qsort). - * This is used when searching the hash chain. The key comparator - * determines if two keys match. It should follow the return value - * semantics of strcmp. - * - * string and pointer comparators are supplied in this file. - * - * mod_hash_create_strhash() and mod_hash_create_ptrhash() provide good - * examples of how to create a customized hash table. - * - * Basic hash operations: - * - * mod_hash_create_strhash(name, nchains, dtor), - * create a hash using strings as keys. - * NOTE: This create a hash which automatically cleans up the string - * values it is given for keys. - * - * mod_hash_create_ptrhash(name, nchains, dtor, key_elem_size): - * create a hash using pointers as keys. - * - * mod_hash_create_extended(name, nchains, kdtor, vdtor, - * hash_alg, hash_alg_data, - * keycmp, sleep) - * create a customized hash table. - * - * mod_hash_destroy_hash(hash): - * destroy the given hash table, calling the key and value destructors - * on each key-value pair stored in the hash. - * - * mod_hash_insert(hash, key, val): - * place a key, value pair into the given hash. - * duplicate keys are rejected. - * - * mod_hash_insert_reserve(hash, key, val, handle): - * place a key, value pair into the given hash, using handle to indicate - * the reserved storage for the pair. (no memory allocation is needed - * during a mod_hash_insert_reserve.) duplicate keys are rejected. - * - * mod_hash_reserve(hash, *handle): - * reserve storage for a key-value pair using the memory allocation - * policy of 'hash', returning the storage handle in 'handle'. - * - * mod_hash_reserve_nosleep(hash, *handle): reserve storage for a key-value - * pair ignoring the memory allocation policy of 'hash' and always without - * sleep, returning the storage handle in 'handle'. - * - * mod_hash_remove(hash, key, *val): - * remove a key-value pair with key 'key' from 'hash', destroying the - * stored key, and returning the value in val. - * - * mod_hash_replace(hash, key, val) - * atomically remove an existing key-value pair from a hash, and replace - * the key and value with the ones supplied. The removed key and value - * (if any) are destroyed. - * - * mod_hash_destroy(hash, key): - * remove a key-value pair with key 'key' from 'hash', destroying both - * stored key and stored value. - * - * mod_hash_find(hash, key, val): - * find a value in the hash table corresponding to the given key. - * - * mod_hash_find_cb(hash, key, val, found_callback) - * find a value in the hash table corresponding to the given key. - * If a value is found, call specified callback passing key and val to it. - * The callback is called with the hash lock held. - * It is intended to be used in situations where the act of locating the - * data must also modify it - such as in reference counting schemes. - * - * mod_hash_walk(hash, callback(key, elem, arg), arg) - * walks all the elements in the hashtable and invokes the callback - * function with the key/value pair for each element. the hashtable - * is locked for readers so the callback function should not attempt - * to do any updates to the hashable. the callback function should - * return MH_WALK_CONTINUE to continue walking the hashtable or - * MH_WALK_TERMINATE to abort the walk of the hashtable. - * - * mod_hash_clear(hash): - * clears the given hash table of entries, calling the key and value - * destructors for every element in the hash. - */ - -#include -#include -#include -#include - -/* - * MH_KEY_DESTROY() - * Invoke the key destructor. - */ -#define MH_KEY_DESTROY(hash, key) ((hash->mh_kdtor)(key)) - -/* - * MH_VAL_DESTROY() - * Invoke the value destructor. - */ -#define MH_VAL_DESTROY(hash, val) ((hash->mh_vdtor)(val)) - -/* - * MH_KEYCMP() - * Call the key comparator for the given hash keys. - */ -#define MH_KEYCMP(hash, key1, key2) ((hash->mh_keycmp)(key1, key2)) - -/* - * Cache for struct mod_hash_entry - */ -kmem_cache_t *mh_e_cache = NULL; -mod_hash_t *mh_head = NULL; -kmutex_t mh_head_lock; - -/* - * mod_hash_null_keydtor() - * mod_hash_null_valdtor() - * no-op key and value destructors. - */ -void -mod_hash_null_keydtor(mod_hash_key_t key) -{ - (void) key; -} - -void -mod_hash_null_valdtor(mod_hash_val_t val) -{ - (void) val; -} - -/* - * mod_hash_bystr() - * mod_hash_strkey_cmp() - * mod_hash_strkey_dtor() - * mod_hash_strval_dtor() - * Hash and key comparison routines for hashes with string keys. - * - * mod_hash_create_strhash() - * Create a hash using strings as keys - * - * The string hashing algorithm is from the "Dragon Book" -- - * "Compilers: Principles, Tools & Techniques", by Aho, Sethi, Ullman - */ - -uint_t -mod_hash_bystr(void *hash_data, mod_hash_key_t key) -{ - (void) hash_data; - uint_t hash = 0; - uint_t g; - char *p, *k = (char *)key; - - ASSERT(k); - for (p = k; *p != '\0'; p++) { - hash = (hash << 4) + *p; - if ((g = (hash & 0xf0000000)) != 0) { - hash ^= (g >> 24); - hash ^= g; - } - } - return (hash); -} - -int -mod_hash_strkey_cmp(mod_hash_key_t key1, mod_hash_key_t key2) -{ - return (strcmp((char *)key1, (char *)key2)); -} - -void -mod_hash_strkey_dtor(mod_hash_key_t key) -{ - char *c = (char *)key; - kmem_free(c, strlen(c) + 1); -} - -void -mod_hash_strval_dtor(mod_hash_val_t val) -{ - char *c = (char *)val; - kmem_free(c, strlen(c) + 1); -} - -mod_hash_t * -mod_hash_create_strhash_nodtr(char *name, size_t nchains, - void (*val_dtor)(mod_hash_val_t)) -{ - return mod_hash_create_extended(name, nchains, mod_hash_null_keydtor, - val_dtor, mod_hash_bystr, NULL, mod_hash_strkey_cmp, KM_SLEEP); -} - -mod_hash_t * -mod_hash_create_strhash(char *name, size_t nchains, - void (*val_dtor)(mod_hash_val_t)) -{ - return mod_hash_create_extended(name, nchains, mod_hash_strkey_dtor, - val_dtor, mod_hash_bystr, NULL, mod_hash_strkey_cmp, KM_SLEEP); -} - -void -mod_hash_destroy_strhash(mod_hash_t *strhash) -{ - ASSERT(strhash); - mod_hash_destroy_hash(strhash); -} - - -/* - * mod_hash_byptr() - * mod_hash_ptrkey_cmp() - * Hash and key comparison routines for hashes with pointer keys. - * - * mod_hash_create_ptrhash() - * mod_hash_destroy_ptrhash() - * Create a hash that uses pointers as keys. This hash algorithm - * picks an appropriate set of middle bits in the address to hash on - * based on the size of the hash table and a hint about the size of - * the items pointed at. - */ -uint_t -mod_hash_byptr(void *hash_data, mod_hash_key_t key) -{ - uintptr_t k = (uintptr_t)key; - k >>= (int)(uintptr_t)hash_data; - - return ((uint_t)k); -} - -int -mod_hash_ptrkey_cmp(mod_hash_key_t key1, mod_hash_key_t key2) -{ - uintptr_t k1 = (uintptr_t)key1; - uintptr_t k2 = (uintptr_t)key2; - if (k1 > k2) - return (-1); - else if (k1 < k2) - return (1); - else - return (0); -} - -mod_hash_t * -mod_hash_create_ptrhash(char *name, size_t nchains, - void (*val_dtor)(mod_hash_val_t), size_t key_elem_size) -{ - size_t rshift; - - /* - * We want to hash on the bits in the middle of the address word - * Bits far to the right in the word have little significance, and - * are likely to all look the same (for example, an array of - * 256-byte structures will have the bottom 8 bits of address - * words the same). So we want to right-shift each address to - * ignore the bottom bits. - * - * The high bits, which are also unused, will get taken out when - * mod_hash takes hashkey % nchains. - */ - rshift = highbit64(key_elem_size); - - return mod_hash_create_extended(name, nchains, mod_hash_null_keydtor, - val_dtor, mod_hash_byptr, (void *)rshift, mod_hash_ptrkey_cmp, - KM_SLEEP); -} - -void -mod_hash_destroy_ptrhash(mod_hash_t *hash) -{ - ASSERT(hash); - mod_hash_destroy_hash(hash); -} - -/* - * mod_hash_byid() - * mod_hash_idkey_cmp() - * Hash and key comparison routines for hashes with 32-bit unsigned keys. - * - * mod_hash_create_idhash() - * mod_hash_destroy_idhash() - * mod_hash_iddata_gen() - * Create a hash that uses numeric keys. - * - * The hash algorithm is documented in "Introduction to Algorithms" - * (Cormen, Leiserson, Rivest); when the hash table is created, it - * attempts to find the next largest prime above the number of hash - * slots. The hash index is then this number times the key modulo - * the hash size, or (key * prime) % nchains. - */ -uint_t -mod_hash_byid(void *hash_data, mod_hash_key_t key) -{ - uint_t kval = (uint_t)(uintptr_t)hash_data; - return ((uint_t)(uintptr_t)key * (uint_t)kval); -} - -int -mod_hash_idkey_cmp(mod_hash_key_t key1, mod_hash_key_t key2) -{ - return ((uint_t)(uintptr_t)key1 - (uint_t)(uintptr_t)key2); -} - -/* - * Generate the next largest prime number greater than nchains; this value - * is intended to be later passed in to mod_hash_create_extended() as the - * hash_data. - */ -uint_t -mod_hash_iddata_gen(size_t nchains) -{ - uint_t kval, i, prime; - - /* - * Pick the first (odd) prime greater than nchains. Make sure kval is - * odd (so start with nchains +1 or +2 as appropriate). - */ - kval = (nchains % 2 == 0) ? nchains + 1 : nchains + 2; - - for (;;) { - prime = 1; - for (i = 3; i * i <= kval; i += 2) { - if (kval % i == 0) - prime = 0; - } - if (prime == 1) - break; - kval += 2; - } - return (kval); -} - -mod_hash_t * -mod_hash_create_idhash(char *name, size_t nchains, - void (*val_dtor)(mod_hash_val_t)) -{ - uint_t kval = mod_hash_iddata_gen(nchains); - - return (mod_hash_create_extended(name, nchains, mod_hash_null_keydtor, - val_dtor, mod_hash_byid, (void *)(uintptr_t)kval, - mod_hash_idkey_cmp, KM_SLEEP)); -} - -void -mod_hash_destroy_idhash(mod_hash_t *hash) -{ - ASSERT(hash); - mod_hash_destroy_hash(hash); -} - -void -mod_hash_fini(void) -{ - mutex_destroy(&mh_head_lock); - - if (mh_e_cache) { - kmem_cache_destroy(mh_e_cache); - mh_e_cache = NULL; - } -} - -/* - * mod_hash_init() - * sets up globals, etc for mod_hash_* - */ -void -mod_hash_init(void) -{ - ASSERT(mh_e_cache == NULL); - mh_e_cache = kmem_cache_create("mod_hash_entries", - sizeof (struct mod_hash_entry), 0, NULL, NULL, NULL, NULL, - NULL, 0); - - mutex_init(&mh_head_lock, NULL, MUTEX_DEFAULT, NULL); -} - -/* - * mod_hash_create_extended() - * The full-blown hash creation function. - * - * notes: - * nchains - how many hash slots to create. More hash slots will - * result in shorter hash chains, but will consume - * slightly more memory up front. - * sleep - should be KM_SLEEP or KM_NOSLEEP, to indicate whether - * to sleep for memory, or fail in low-memory conditions. - * - * Fails only if KM_NOSLEEP was specified, and no memory was available. - */ -mod_hash_t * -mod_hash_create_extended( - char *hname, /* descriptive name for hash */ - size_t nchains, /* number of hash slots */ - void (*kdtor)(mod_hash_key_t), /* key destructor */ - void (*vdtor)(mod_hash_val_t), /* value destructor */ - uint_t (*hash_alg)(void *, mod_hash_key_t), /* hash algorithm */ - void *hash_alg_data, /* pass-thru arg for hash_alg */ - int (*keycmp)(mod_hash_key_t, mod_hash_key_t), /* key comparator */ - int sleep) /* whether to sleep for mem */ -{ - mod_hash_t *mod_hash; - size_t size; - ASSERT(hname && keycmp && hash_alg && vdtor && kdtor); - - if ((mod_hash = kmem_zalloc(MH_SIZE(nchains), sleep)) == NULL) - return (NULL); - - size = strlen(hname) + 1; - mod_hash->mh_name = kmem_alloc(size, sleep); - if (mod_hash->mh_name == NULL) { - kmem_free(mod_hash, MH_SIZE(nchains)); - return (NULL); - } - (void) strlcpy(mod_hash->mh_name, hname, size); - - rw_init(&mod_hash->mh_contents, NULL, RW_DEFAULT, NULL); - mod_hash->mh_sleep = sleep; - mod_hash->mh_nchains = nchains; - mod_hash->mh_kdtor = kdtor; - mod_hash->mh_vdtor = vdtor; - mod_hash->mh_hashalg = hash_alg; - mod_hash->mh_hashalg_data = hash_alg_data; - mod_hash->mh_keycmp = keycmp; - - /* - * Link the hash up on the list of hashes - */ - mutex_enter(&mh_head_lock); - mod_hash->mh_next = mh_head; - mh_head = mod_hash; - mutex_exit(&mh_head_lock); - - return (mod_hash); -} - -/* - * mod_hash_destroy_hash() - * destroy a hash table, destroying all of its stored keys and values - * as well. - */ -void -mod_hash_destroy_hash(mod_hash_t *hash) -{ - mod_hash_t *mhp, *mhpp; - - mutex_enter(&mh_head_lock); - /* - * Remove the hash from the hash list - */ - if (hash == mh_head) { /* removing 1st list elem */ - mh_head = mh_head->mh_next; - } else { - /* - * mhpp can start out NULL since we know the 1st elem isn't the - * droid we're looking for. - */ - mhpp = NULL; - for (mhp = mh_head; mhp != NULL; mhp = mhp->mh_next) { - if (mhp == hash) { - mhpp->mh_next = mhp->mh_next; - break; - } - mhpp = mhp; - } - } - mutex_exit(&mh_head_lock); - - /* - * Clean out keys and values. - */ - mod_hash_clear(hash); - - rw_destroy(&hash->mh_contents); - kmem_free(hash->mh_name, strlen(hash->mh_name) + 1); - kmem_free(hash, MH_SIZE(hash->mh_nchains)); -} - -/* - * i_mod_hash() - * Call the hashing algorithm for this hash table, with the given key. - */ -uint_t -i_mod_hash(mod_hash_t *hash, mod_hash_key_t key) -{ - uint_t h; - /* - * Prevent div by 0 problems; - * Also a nice shortcut when using a hash as a list - */ - if (hash->mh_nchains == 1) - return (0); - - h = (hash->mh_hashalg)(hash->mh_hashalg_data, key); - return (h % (hash->mh_nchains - 1)); -} - -/* - * i_mod_hash_insert_nosync() - * mod_hash_insert() - * mod_hash_insert_reserve() - * insert 'val' into the hash table, using 'key' as its key. If 'key' is - * already a key in the hash, an error will be returned, and the key-val - * pair will not be inserted. i_mod_hash_insert_nosync() supports a simple - * handle abstraction, allowing hash entry allocation to be separated from - * the hash insertion. this abstraction allows simple use of the mod_hash - * structure in situations where mod_hash_insert() with a KM_SLEEP - * allocation policy would otherwise be unsafe. - */ -int -i_mod_hash_insert_nosync(mod_hash_t *hash, mod_hash_key_t key, - mod_hash_val_t val, mod_hash_hndl_t handle) -{ - uint_t hashidx; - struct mod_hash_entry *entry; - - ASSERT(hash); - - /* - * If we've not been given reserved storage, allocate storage directly, - * using the hash's allocation policy. - */ - if (handle == (mod_hash_hndl_t)0) { - entry = kmem_cache_alloc(mh_e_cache, hash->mh_sleep); - if (entry == NULL) { - hash->mh_stat.mhs_nomem++; - return (MH_ERR_NOMEM); - } - } else { - entry = (struct mod_hash_entry *)handle; - } - - hashidx = i_mod_hash(hash, key); - entry->mhe_key = key; - entry->mhe_val = val; - entry->mhe_next = hash->mh_entries[hashidx]; - - hash->mh_entries[hashidx] = entry; - hash->mh_stat.mhs_nelems++; - - return (0); -} - -int -mod_hash_insert(mod_hash_t *hash, mod_hash_key_t key, mod_hash_val_t val) -{ - int res; - mod_hash_val_t v; - - rw_enter(&hash->mh_contents, RW_WRITER); - - /* - * Disallow duplicate keys in the hash - */ - if (i_mod_hash_find_nosync(hash, key, &v) == 0) { - rw_exit(&hash->mh_contents); - hash->mh_stat.mhs_coll++; - return (MH_ERR_DUPLICATE); - } - - res = i_mod_hash_insert_nosync(hash, key, val, (mod_hash_hndl_t)0); - rw_exit(&hash->mh_contents); - - return (res); -} - -int -mod_hash_insert_reserve(mod_hash_t *hash, mod_hash_key_t key, - mod_hash_val_t val, mod_hash_hndl_t handle) -{ - int res; - mod_hash_val_t v; - - rw_enter(&hash->mh_contents, RW_WRITER); - - /* - * Disallow duplicate keys in the hash - */ - if (i_mod_hash_find_nosync(hash, key, &v) == 0) { - rw_exit(&hash->mh_contents); - hash->mh_stat.mhs_coll++; - return (MH_ERR_DUPLICATE); - } - res = i_mod_hash_insert_nosync(hash, key, val, handle); - rw_exit(&hash->mh_contents); - - return (res); -} - -/* - * mod_hash_reserve() - * mod_hash_reserve_nosleep() - * mod_hash_cancel() - * Make or cancel a mod_hash_entry_t reservation. Reservations are used in - * mod_hash_insert_reserve() above. - */ -int -mod_hash_reserve(mod_hash_t *hash, mod_hash_hndl_t *handlep) -{ - *handlep = kmem_cache_alloc(mh_e_cache, hash->mh_sleep); - if (*handlep == NULL) { - hash->mh_stat.mhs_nomem++; - return (MH_ERR_NOMEM); - } - - return (0); -} - -int -mod_hash_reserve_nosleep(mod_hash_t *hash, mod_hash_hndl_t *handlep) -{ - *handlep = kmem_cache_alloc(mh_e_cache, KM_NOSLEEP); - if (*handlep == NULL) { - hash->mh_stat.mhs_nomem++; - return (MH_ERR_NOMEM); - } - - return (0); - -} - -void -mod_hash_cancel(mod_hash_t *hash, mod_hash_hndl_t *handlep) -{ - (void) hash; - kmem_cache_free(mh_e_cache, *handlep); - *handlep = (mod_hash_hndl_t)0; -} - -/* - * i_mod_hash_remove_nosync() - * mod_hash_remove() - * Remove an element from the hash table. - */ -int -i_mod_hash_remove_nosync(mod_hash_t *hash, mod_hash_key_t key, - mod_hash_val_t *val) -{ - int hashidx; - struct mod_hash_entry *e, *ep; - - hashidx = i_mod_hash(hash, key); - ep = NULL; /* e's parent */ - - for (e = hash->mh_entries[hashidx]; e != NULL; e = e->mhe_next) { - if (MH_KEYCMP(hash, e->mhe_key, key) == 0) - break; - ep = e; - } - - if (e == NULL) { /* not found */ - return (MH_ERR_NOTFOUND); - } - - if (ep == NULL) /* special case 1st element in bucket */ - hash->mh_entries[hashidx] = e->mhe_next; - else - ep->mhe_next = e->mhe_next; - - /* - * Clean up resources used by the node's key. - */ - MH_KEY_DESTROY(hash, e->mhe_key); - - *val = e->mhe_val; - kmem_cache_free(mh_e_cache, e); - hash->mh_stat.mhs_nelems--; - - return (0); -} - -int -mod_hash_remove(mod_hash_t *hash, mod_hash_key_t key, mod_hash_val_t *val) -{ - int res; - - rw_enter(&hash->mh_contents, RW_WRITER); - res = i_mod_hash_remove_nosync(hash, key, val); - rw_exit(&hash->mh_contents); - - return (res); -} - -/* - * mod_hash_replace() - * atomically remove an existing key-value pair from a hash, and replace - * the key and value with the ones supplied. The removed key and value - * (if any) are destroyed. - */ -int -mod_hash_replace(mod_hash_t *hash, mod_hash_key_t key, mod_hash_val_t val) -{ - int res; - mod_hash_val_t v; - - rw_enter(&hash->mh_contents, RW_WRITER); - - if (i_mod_hash_remove_nosync(hash, key, &v) == 0) { - /* - * mod_hash_remove() takes care of freeing up the key resources. - */ - MH_VAL_DESTROY(hash, v); - } - res = i_mod_hash_insert_nosync(hash, key, val, (mod_hash_hndl_t)0); - - rw_exit(&hash->mh_contents); - - return (res); -} - -/* - * mod_hash_destroy() - * Remove an element from the hash table matching 'key', and destroy it. - */ -int -mod_hash_destroy(mod_hash_t *hash, mod_hash_key_t key) -{ - mod_hash_val_t val; - int rv; - - rw_enter(&hash->mh_contents, RW_WRITER); - - if ((rv = i_mod_hash_remove_nosync(hash, key, &val)) == 0) { - /* - * mod_hash_remove() takes care of freeing up the key resources. - */ - MH_VAL_DESTROY(hash, val); - } - - rw_exit(&hash->mh_contents); - return (rv); -} - -/* - * i_mod_hash_find_nosync() - * mod_hash_find() - * Find a value in the hash table corresponding to the given key. - */ -int -i_mod_hash_find_nosync(mod_hash_t *hash, mod_hash_key_t key, - mod_hash_val_t *val) -{ - uint_t hashidx; - struct mod_hash_entry *e; - - hashidx = i_mod_hash(hash, key); - - for (e = hash->mh_entries[hashidx]; e != NULL; e = e->mhe_next) { - if (MH_KEYCMP(hash, e->mhe_key, key) == 0) { - *val = e->mhe_val; - hash->mh_stat.mhs_hit++; - return (0); - } - } - hash->mh_stat.mhs_miss++; - return (MH_ERR_NOTFOUND); -} - -int -mod_hash_find(mod_hash_t *hash, mod_hash_key_t key, mod_hash_val_t *val) -{ - int res; - - rw_enter(&hash->mh_contents, RW_READER); - res = i_mod_hash_find_nosync(hash, key, val); - rw_exit(&hash->mh_contents); - - return (res); -} - -int -mod_hash_find_cb(mod_hash_t *hash, mod_hash_key_t key, mod_hash_val_t *val, - void (*find_cb)(mod_hash_key_t, mod_hash_val_t)) -{ - int res; - - rw_enter(&hash->mh_contents, RW_READER); - res = i_mod_hash_find_nosync(hash, key, val); - if (res == 0) { - find_cb(key, *val); - } - rw_exit(&hash->mh_contents); - - return (res); -} - -int -mod_hash_find_cb_rval(mod_hash_t *hash, mod_hash_key_t key, mod_hash_val_t *val, - int (*find_cb)(mod_hash_key_t, mod_hash_val_t), int *cb_rval) -{ - int res; - - rw_enter(&hash->mh_contents, RW_READER); - res = i_mod_hash_find_nosync(hash, key, val); - if (res == 0) { - *cb_rval = find_cb(key, *val); - } - rw_exit(&hash->mh_contents); - - return (res); -} - -void -i_mod_hash_walk_nosync(mod_hash_t *hash, - uint_t (*callback)(mod_hash_key_t, mod_hash_val_t *, void *), void *arg) -{ - struct mod_hash_entry *e; - uint_t hashidx; - int res = MH_WALK_CONTINUE; - - for (hashidx = 0; - (hashidx < (hash->mh_nchains - 1)) && (res == MH_WALK_CONTINUE); - hashidx++) { - e = hash->mh_entries[hashidx]; - while ((e != NULL) && (res == MH_WALK_CONTINUE)) { - res = callback(e->mhe_key, e->mhe_val, arg); - e = e->mhe_next; - } - } -} - -/* - * mod_hash_walk() - * Walks all the elements in the hashtable and invokes the callback - * function with the key/value pair for each element. The hashtable - * is locked for readers so the callback function should not attempt - * to do any updates to the hashable. The callback function should - * return MH_WALK_CONTINUE to continue walking the hashtable or - * MH_WALK_TERMINATE to abort the walk of the hashtable. - */ -void -mod_hash_walk(mod_hash_t *hash, - uint_t (*callback)(mod_hash_key_t, mod_hash_val_t *, void *), void *arg) -{ - rw_enter(&hash->mh_contents, RW_READER); - i_mod_hash_walk_nosync(hash, callback, arg); - rw_exit(&hash->mh_contents); -} - - -/* - * i_mod_hash_clear_nosync() - * mod_hash_clear() - * Clears the given hash table by calling the destructor of every hash - * element and freeing up all mod_hash_entry's. - */ -void -i_mod_hash_clear_nosync(mod_hash_t *hash) -{ - int i; - struct mod_hash_entry *e, *old_e; - - for (i = 0; i < hash->mh_nchains; i++) { - e = hash->mh_entries[i]; - while (e != NULL) { - MH_KEY_DESTROY(hash, e->mhe_key); - MH_VAL_DESTROY(hash, e->mhe_val); - old_e = e; - e = e->mhe_next; - kmem_cache_free(mh_e_cache, old_e); - } - hash->mh_entries[i] = NULL; - } - hash->mh_stat.mhs_nelems = 0; -} - -void -mod_hash_clear(mod_hash_t *hash) -{ - ASSERT(hash); - rw_enter(&hash->mh_contents, RW_WRITER); - i_mod_hash_clear_nosync(hash); - rw_exit(&hash->mh_contents); -} diff --git a/module/icp/spi/kcf_spi.c b/module/icp/spi/kcf_spi.c index 25fe9b5b66be..b0af101990ed 100644 --- a/module/icp/spi/kcf_spi.c +++ b/module/icp/spi/kcf_spi.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -36,145 +36,35 @@ #include #include -/* - * minalloc and maxalloc values to be used for taskq_create(). - */ -const int crypto_taskq_threads = CRYPTO_TASKQ_THREADS; -const int crypto_taskq_minalloc = CRYPTO_TASKQ_MIN; -const int crypto_taskq_maxalloc = CRYPTO_TASKQ_MAX; - -static void remove_provider(kcf_provider_desc_t *); -static void process_logical_providers(const crypto_provider_info_t *, - kcf_provider_desc_t *); static int init_prov_mechs(const crypto_provider_info_t *, kcf_provider_desc_t *); -static int kcf_prov_kstat_update(kstat_t *, int); -static void delete_kstat(kcf_provider_desc_t *); - -static const kcf_prov_stats_t kcf_stats_ks_data_template = { - { "kcf_ops_total", KSTAT_DATA_UINT64 }, - { "kcf_ops_passed", KSTAT_DATA_UINT64 }, - { "kcf_ops_failed", KSTAT_DATA_UINT64 }, - { "kcf_ops_returned_busy", KSTAT_DATA_UINT64 } -}; - -#define KCF_SPI_COPY_OPS(src, dst, ops) if ((src)->ops != NULL) \ - memcpy((void *) (dst)->ops, (src)->ops, sizeof (*(src)->ops)); - -/* - * Copy an ops vector from src to dst. Used during provider registration - * to copy the ops vector from the provider info structure to the - * provider descriptor maintained by KCF. - * Copying the ops vector specified by the provider is needed since the - * framework does not require the provider info structure to be - * persistent. - */ -static void -copy_ops_vector_v1(const crypto_ops_t *src_ops, crypto_ops_t *dst_ops) -{ - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_control_ops); - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_digest_ops); - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_cipher_ops); - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_mac_ops); - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_sign_ops); - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_verify_ops); - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_dual_ops); - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_dual_cipher_mac_ops); - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_random_ops); - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_session_ops); - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_object_ops); - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_key_ops); - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_provider_ops); - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_ctx_ops); -} - -static void -copy_ops_vector_v2(const crypto_ops_t *src_ops, crypto_ops_t *dst_ops) -{ - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_mech_ops); -} - -static void -copy_ops_vector_v3(const crypto_ops_t *src_ops, crypto_ops_t *dst_ops) -{ - KCF_SPI_COPY_OPS(src_ops, dst_ops, co_nostore_key_ops); -} /* * This routine is used to add cryptographic providers to the KEF framework. * Providers pass a crypto_provider_info structure to crypto_register_provider() * and get back a handle. The crypto_provider_info structure contains a * list of mechanisms supported by the provider and an ops vector containing - * provider entry points. Hardware providers call this routine in their attach - * routines. Software providers call this routine in their _init() routine. + * provider entry points. Providers call this routine in their _init() routine. */ int crypto_register_provider(const crypto_provider_info_t *info, crypto_kcf_provider_handle_t *handle) { - char *ks_name; - kcf_provider_desc_t *prov_desc = NULL; int ret = CRYPTO_ARGUMENTS_BAD; - if (info->pi_interface_version > CRYPTO_SPI_VERSION_3) - return (CRYPTO_VERSION_MISMATCH); - - /* - * Check provider type, must be software, hardware, or logical. - */ - if (info->pi_provider_type != CRYPTO_HW_PROVIDER && - info->pi_provider_type != CRYPTO_SW_PROVIDER && - info->pi_provider_type != CRYPTO_LOGICAL_PROVIDER) - return (CRYPTO_ARGUMENTS_BAD); - /* * Allocate and initialize a new provider descriptor. We also * hold it and release it when done. */ - prov_desc = kcf_alloc_provider_desc(info); + prov_desc = kcf_alloc_provider_desc(); KCF_PROV_REFHOLD(prov_desc); - prov_desc->pd_prov_type = info->pi_provider_type; - - /* provider-private handle, opaque to KCF */ - prov_desc->pd_prov_handle = info->pi_provider_handle; - /* copy provider description string */ - if (info->pi_provider_description != NULL) { - /* - * pi_provider_descriptor is a string that can contain - * up to CRYPTO_PROVIDER_DESCR_MAX_LEN + 1 characters - * INCLUDING the terminating null character. A bcopy() - * is necessary here as pd_description should not have - * a null character. See comments in kcf_alloc_provider_desc() - * for details on pd_description field. - */ - bcopy(info->pi_provider_description, prov_desc->pd_description, - MIN(strlen(info->pi_provider_description), - (size_t)CRYPTO_PROVIDER_DESCR_MAX_LEN)); - } + prov_desc->pd_description = info->pi_provider_description; - if (info->pi_provider_type != CRYPTO_LOGICAL_PROVIDER) { - if (info->pi_ops_vector == NULL) { - goto bail; - } - crypto_ops_t *pvec = (crypto_ops_t *)prov_desc->pd_ops_vector; - copy_ops_vector_v1(info->pi_ops_vector, pvec); - if (info->pi_interface_version >= CRYPTO_SPI_VERSION_2) { - copy_ops_vector_v2(info->pi_ops_vector, pvec); - prov_desc->pd_flags = info->pi_flags; - } - if (info->pi_interface_version == CRYPTO_SPI_VERSION_3) { - copy_ops_vector_v3(info->pi_ops_vector, pvec); - } - } - - /* object_ops and nostore_key_ops are mutually exclusive */ - if (prov_desc->pd_ops_vector->co_object_ops && - prov_desc->pd_ops_vector->co_nostore_key_ops) { - goto bail; - } + /* Change from Illumos: the ops vector is persistent. */ + prov_desc->pd_ops_vector = info->pi_ops_vector; /* process the mechanisms supported by the provider */ if ((ret = init_prov_mechs(info, prov_desc)) != CRYPTO_SUCCESS) @@ -190,86 +80,15 @@ crypto_register_provider(const crypto_provider_info_t *info, } /* - * We create a taskq only for a hardware provider. The global - * software queue is used for software providers. We handle ordering + * The global queue is used for providers. We handle ordering * of multi-part requests in the taskq routine. So, it is safe to * have multiple threads for the taskq. We pass TASKQ_PREPOPULATE flag * to keep some entries cached to improve performance. */ - if (prov_desc->pd_prov_type == CRYPTO_HW_PROVIDER) - prov_desc->pd_sched_info.ks_taskq = taskq_create("kcf_taskq", - CRYPTO_TASKQ_THREADS, minclsyspri, - CRYPTO_TASKQ_MIN, CRYPTO_TASKQ_MAX, - TASKQ_PREPOPULATE); - else - prov_desc->pd_sched_info.ks_taskq = NULL; - - /* no kernel session to logical providers */ - if (prov_desc->pd_prov_type != CRYPTO_LOGICAL_PROVIDER) { - /* - * Open a session for session-oriented providers. This session - * is used for all kernel consumers. This is fine as a provider - * is required to support multiple thread access to a session. - * We can do this only after the taskq has been created as we - * do a kcf_submit_request() to open the session. - */ - if (KCF_PROV_SESSION_OPS(prov_desc) != NULL) { - kcf_req_params_t params; - - KCF_WRAP_SESSION_OPS_PARAMS(¶ms, - KCF_OP_SESSION_OPEN, &prov_desc->pd_sid, 0, - CRYPTO_USER, NULL, 0, prov_desc); - ret = kcf_submit_request(prov_desc, NULL, NULL, ¶ms, - B_FALSE); - - if (ret != CRYPTO_SUCCESS) { - undo_register_provider(prov_desc, B_TRUE); - ret = CRYPTO_FAILED; - goto bail; - } - } - } - - if (prov_desc->pd_prov_type != CRYPTO_LOGICAL_PROVIDER) { - /* - * Create the kstat for this provider. There is a kstat - * installed for each successfully registered provider. - * This kstat is deleted, when the provider unregisters. - */ - if (prov_desc->pd_prov_type == CRYPTO_SW_PROVIDER) { - ks_name = kmem_asprintf("%s_%s", - "NONAME", "provider_stats"); - } else { - ks_name = kmem_asprintf("%s_%d_%u_%s", - "NONAME", 0, prov_desc->pd_prov_id, - "provider_stats"); - } - - prov_desc->pd_kstat = kstat_create("kcf", 0, ks_name, "crypto", - KSTAT_TYPE_NAMED, sizeof (kcf_prov_stats_t) / - sizeof (kstat_named_t), KSTAT_FLAG_VIRTUAL); - - if (prov_desc->pd_kstat != NULL) { - bcopy(&kcf_stats_ks_data_template, - &prov_desc->pd_ks_data, - sizeof (kcf_stats_ks_data_template)); - prov_desc->pd_kstat->ks_data = &prov_desc->pd_ks_data; - KCF_PROV_REFHOLD(prov_desc); - KCF_PROV_IREFHOLD(prov_desc); - prov_desc->pd_kstat->ks_private = prov_desc; - prov_desc->pd_kstat->ks_update = kcf_prov_kstat_update; - kstat_install(prov_desc->pd_kstat); - } - kmem_strfree(ks_name); - } - - if (prov_desc->pd_prov_type == CRYPTO_HW_PROVIDER) - process_logical_providers(info, prov_desc); mutex_enter(&prov_desc->pd_lock); prov_desc->pd_state = KCF_PROV_READY; mutex_exit(&prov_desc->pd_lock); - kcf_do_notify(prov_desc, B_TRUE); *handle = prov_desc->pd_kcf_prov_handle; ret = CRYPTO_SUCCESS; @@ -281,8 +100,7 @@ crypto_register_provider(const crypto_provider_info_t *info, /* * This routine is used to notify the framework when a provider is being - * removed. Hardware providers call this routine in their detach routines. - * Software providers call this routine in their _fini() routine. + * removed. Providers call this routine in their _fini() routine. */ int crypto_unregister_provider(crypto_kcf_provider_handle_t handle) @@ -310,46 +128,30 @@ crypto_unregister_provider(crypto_kcf_provider_handle_t handle) saved_state = desc->pd_state; desc->pd_state = KCF_PROV_REMOVED; - if (saved_state == KCF_PROV_BUSY) { - /* - * The per-provider taskq threads may be waiting. We - * signal them so that they can start failing requests. - */ - cv_broadcast(&desc->pd_resume_cv); - } - - if (desc->pd_prov_type == CRYPTO_SW_PROVIDER) { + /* + * Check if this provider is currently being used. + * pd_irefcnt is the number of holds from the internal + * structures. We add one to account for the above lookup. + */ + if (desc->pd_refcnt > desc->pd_irefcnt + 1) { + desc->pd_state = saved_state; + mutex_exit(&desc->pd_lock); + /* Release reference held by kcf_prov_tab_lookup(). */ + KCF_PROV_REFRELE(desc); /* - * Check if this provider is currently being used. - * pd_irefcnt is the number of holds from the internal - * structures. We add one to account for the above lookup. + * The administrator will presumably stop the clients, + * thus removing the holds, when they get the busy + * return value. Any retry will succeed then. */ - if (desc->pd_refcnt > desc->pd_irefcnt + 1) { - desc->pd_state = saved_state; - mutex_exit(&desc->pd_lock); - /* Release reference held by kcf_prov_tab_lookup(). */ - KCF_PROV_REFRELE(desc); - /* - * The administrator presumably will stop the clients - * thus removing the holds, when they get the busy - * return value. Any retry will succeed then. - */ - return (CRYPTO_BUSY); - } + return (CRYPTO_BUSY); } mutex_exit(&desc->pd_lock); - if (desc->pd_prov_type != CRYPTO_SW_PROVIDER) { - remove_provider(desc); - } - - if (desc->pd_prov_type != CRYPTO_LOGICAL_PROVIDER) { - /* remove the provider from the mechanisms tables */ - for (mech_idx = 0; mech_idx < desc->pd_mech_list_count; - mech_idx++) { - kcf_remove_mech_provider( - desc->pd_mechanisms[mech_idx].cm_mech_name, desc); - } + /* remove the provider from the mechanisms tables */ + for (mech_idx = 0; mech_idx < desc->pd_mech_list_count; + mech_idx++) { + kcf_remove_mech_provider( + desc->pd_mechanisms[mech_idx].cm_mech_name, desc); } /* remove provider from providers table */ @@ -360,206 +162,34 @@ crypto_unregister_provider(crypto_kcf_provider_handle_t handle) return (CRYPTO_UNKNOWN_PROVIDER); } - delete_kstat(desc); - - if (desc->pd_prov_type == CRYPTO_SW_PROVIDER) { - /* Release reference held by kcf_prov_tab_lookup(). */ - KCF_PROV_REFRELE(desc); - - /* - * Wait till the existing requests complete. - */ - mutex_enter(&desc->pd_lock); - while (desc->pd_state != KCF_PROV_FREED) - cv_wait(&desc->pd_remove_cv, &desc->pd_lock); - mutex_exit(&desc->pd_lock); - } else { - /* - * Wait until requests that have been sent to the provider - * complete. - */ - mutex_enter(&desc->pd_lock); - while (desc->pd_irefcnt > 0) - cv_wait(&desc->pd_remove_cv, &desc->pd_lock); - mutex_exit(&desc->pd_lock); - } + /* Release reference held by kcf_prov_tab_lookup(). */ + KCF_PROV_REFRELE(desc); - kcf_do_notify(desc, B_FALSE); + /* + * Wait till the existing requests complete. + */ + mutex_enter(&desc->pd_lock); + while (desc->pd_state != KCF_PROV_FREED) + cv_wait(&desc->pd_remove_cv, &desc->pd_lock); + mutex_exit(&desc->pd_lock); - if (desc->pd_prov_type == CRYPTO_SW_PROVIDER) { - /* - * This is the only place where kcf_free_provider_desc() - * is called directly. KCF_PROV_REFRELE() should free the - * structure in all other places. - */ - ASSERT(desc->pd_state == KCF_PROV_FREED && - desc->pd_refcnt == 0); - kcf_free_provider_desc(desc); - } else { - KCF_PROV_REFRELE(desc); - } + /* + * This is the only place where kcf_free_provider_desc() + * is called directly. KCF_PROV_REFRELE() should free the + * structure in all other places. + */ + ASSERT(desc->pd_state == KCF_PROV_FREED && + desc->pd_refcnt == 0); + kcf_free_provider_desc(desc); return (CRYPTO_SUCCESS); } -/* - * This routine is used to notify the framework that the state of - * a cryptographic provider has changed. Valid state codes are: - * - * CRYPTO_PROVIDER_READY - * The provider indicates that it can process more requests. A provider - * will notify with this event if it previously has notified us with a - * CRYPTO_PROVIDER_BUSY. - * - * CRYPTO_PROVIDER_BUSY - * The provider can not take more requests. - * - * CRYPTO_PROVIDER_FAILED - * The provider encountered an internal error. The framework will not - * be sending any more requests to the provider. The provider may notify - * with a CRYPTO_PROVIDER_READY, if it is able to recover from the error. - * - * This routine can be called from user or interrupt context. - */ -void -crypto_provider_notification(crypto_kcf_provider_handle_t handle, uint_t state) -{ - kcf_provider_desc_t *pd; - - /* lookup the provider from the given handle */ - if ((pd = kcf_prov_tab_lookup((crypto_provider_id_t)handle)) == NULL) - return; - - mutex_enter(&pd->pd_lock); - - if (pd->pd_state <= KCF_PROV_VERIFICATION_FAILED) - goto out; - - if (pd->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) { - cmn_err(CE_WARN, "crypto_provider_notification: " - "logical provider (%x) ignored\n", handle); - goto out; - } - switch (state) { - case CRYPTO_PROVIDER_READY: - switch (pd->pd_state) { - case KCF_PROV_BUSY: - pd->pd_state = KCF_PROV_READY; - /* - * Signal the per-provider taskq threads that they - * can start submitting requests. - */ - cv_broadcast(&pd->pd_resume_cv); - break; - - case KCF_PROV_FAILED: - /* - * The provider recovered from the error. Let us - * use it now. - */ - pd->pd_state = KCF_PROV_READY; - break; - default: - break; - } - break; - - case CRYPTO_PROVIDER_BUSY: - switch (pd->pd_state) { - case KCF_PROV_READY: - pd->pd_state = KCF_PROV_BUSY; - break; - default: - break; - } - break; - - case CRYPTO_PROVIDER_FAILED: - /* - * We note the failure and return. The per-provider taskq - * threads check this flag and start failing the - * requests, if it is set. See process_req_hwp() for details. - */ - switch (pd->pd_state) { - case KCF_PROV_READY: - pd->pd_state = KCF_PROV_FAILED; - break; - - case KCF_PROV_BUSY: - pd->pd_state = KCF_PROV_FAILED; - /* - * The per-provider taskq threads may be waiting. We - * signal them so that they can start failing requests. - */ - cv_broadcast(&pd->pd_resume_cv); - break; - default: - break; - } - break; - default: - break; - } -out: - mutex_exit(&pd->pd_lock); - KCF_PROV_REFRELE(pd); -} - -/* - * This routine is used to notify the framework the result of - * an asynchronous request handled by a provider. Valid error - * codes are the same as the CRYPTO_* errors defined in common.h. - * - * This routine can be called from user or interrupt context. - */ -void -crypto_op_notification(crypto_req_handle_t handle, int error) -{ - kcf_call_type_t ctype; - - if (handle == NULL) - return; - - if ((ctype = GET_REQ_TYPE(handle)) == CRYPTO_SYNCH) { - kcf_sreq_node_t *sreq = (kcf_sreq_node_t *)handle; - - if (error != CRYPTO_SUCCESS) - sreq->sn_provider->pd_sched_info.ks_nfails++; - KCF_PROV_IREFRELE(sreq->sn_provider); - kcf_sop_done(sreq, error); - } else { - kcf_areq_node_t *areq = (kcf_areq_node_t *)handle; - - ASSERT(ctype == CRYPTO_ASYNCH); - if (error != CRYPTO_SUCCESS) - areq->an_provider->pd_sched_info.ks_nfails++; - KCF_PROV_IREFRELE(areq->an_provider); - kcf_aop_done(areq, error); - } -} - -/* - * This routine is used by software providers to determine - * whether to use KM_SLEEP or KM_NOSLEEP during memory allocation. - * Note that hardware providers can always use KM_SLEEP. So, - * they do not need to call this routine. - * - * This routine can be called from user or interrupt context. - */ -int -crypto_kmflag(crypto_req_handle_t handle) -{ - return (REQHNDL2_KMFLAG(handle)); -} - /* * Process the mechanism info structures specified by the provider * during registration. A NULL crypto_provider_info_t indicates * an already initialized provider descriptor. * - * Mechanisms are not added to the kernel's mechanism table if the - * provider is a logical provider. - * * Returns CRYPTO_SUCCESS on success, CRYPTO_ARGUMENTS if one * of the specified mechanisms was malformed, or CRYPTO_HOST_MEMORY * if the table of mechanisms is full. @@ -572,16 +202,6 @@ init_prov_mechs(const crypto_provider_info_t *info, kcf_provider_desc_t *desc) int err = CRYPTO_SUCCESS; kcf_prov_mech_desc_t *pmd; int desc_use_count = 0; - int mcount = desc->pd_mech_list_count; - - if (desc->pd_prov_type == CRYPTO_LOGICAL_PROVIDER) { - if (info != NULL) { - ASSERT(info->pi_mechanisms != NULL); - bcopy(info->pi_mechanisms, desc->pd_mechanisms, - sizeof (crypto_mech_info_t) * mcount); - } - return (CRYPTO_SUCCESS); - } /* * Copy the mechanism list from the provider info to the provider @@ -590,29 +210,9 @@ init_prov_mechs(const crypto_provider_info_t *info, kcf_provider_desc_t *desc) * mechanism, SUN_RANDOM, in this case. */ if (info != NULL) { - if (info->pi_ops_vector->co_random_ops != NULL) { - crypto_mech_info_t *rand_mi; - - /* - * Need the following check as it is possible to have - * a provider that implements just random_ops and has - * pi_mechanisms == NULL. - */ - if (info->pi_mechanisms != NULL) { - bcopy(info->pi_mechanisms, desc->pd_mechanisms, - sizeof (crypto_mech_info_t) * (mcount - 1)); - } - rand_mi = &desc->pd_mechanisms[mcount - 1]; - - bzero(rand_mi, sizeof (crypto_mech_info_t)); - (void) strncpy(rand_mi->cm_mech_name, SUN_RANDOM, - CRYPTO_MAX_MECH_NAME); - rand_mi->cm_func_group_mask = CRYPTO_FG_RANDOM; - } else { - ASSERT(info->pi_mechanisms != NULL); - bcopy(info->pi_mechanisms, desc->pd_mechanisms, - sizeof (crypto_mech_info_t) * mcount); - } + ASSERT(info->pi_mechanisms != NULL); + desc->pd_mech_list_count = info->pi_mech_list_count; + desc->pd_mechanisms = info->pi_mechanisms; } /* @@ -620,32 +220,6 @@ init_prov_mechs(const crypto_provider_info_t *info, kcf_provider_desc_t *desc) * to the corresponding KCF mechanism mech_entry chain. */ for (mech_idx = 0; mech_idx < desc->pd_mech_list_count; mech_idx++) { - crypto_mech_info_t *mi = &desc->pd_mechanisms[mech_idx]; - - if ((mi->cm_mech_flags & CRYPTO_KEYSIZE_UNIT_IN_BITS) && - (mi->cm_mech_flags & CRYPTO_KEYSIZE_UNIT_IN_BYTES)) { - err = CRYPTO_ARGUMENTS_BAD; - break; - } - - if (desc->pd_flags & CRYPTO_HASH_NO_UPDATE && - mi->cm_func_group_mask & CRYPTO_FG_DIGEST) { - /* - * We ask the provider to specify the limit - * per hash mechanism. But, in practice, a - * hardware limitation means all hash mechanisms - * will have the same maximum size allowed for - * input data. So, we make it a per provider - * limit to keep it simple. - */ - if (mi->cm_max_input_length == 0) { - err = CRYPTO_ARGUMENTS_BAD; - break; - } else { - desc->pd_hash_limit = mi->cm_max_input_length; - } - } - if ((err = kcf_add_mech_provider(mech_idx, desc, &pmd)) != KCF_SUCCESS) break; @@ -658,12 +232,12 @@ init_prov_mechs(const crypto_provider_info_t *info, kcf_provider_desc_t *desc) } /* - * Don't allow multiple software providers with disabled mechanisms + * Don't allow multiple providers with disabled mechanisms * to register. Subsequent enabling of mechanisms will result in - * an unsupported configuration, i.e. multiple software providers + * an unsupported configuration, i.e. multiple providers * per mechanism. */ - if (desc_use_count == 0 && desc->pd_prov_type == CRYPTO_SW_PROVIDER) + if (desc_use_count == 0) return (CRYPTO_ARGUMENTS_BAD); if (err == KCF_SUCCESS) @@ -684,35 +258,6 @@ init_prov_mechs(const crypto_provider_info_t *info, kcf_provider_desc_t *desc) return (CRYPTO_ARGUMENTS_BAD); } -/* - * Update routine for kstat. Only privileged users are allowed to - * access this information, since this information is sensitive. - * There are some cryptographic attacks (e.g. traffic analysis) - * which can use this information. - */ -static int -kcf_prov_kstat_update(kstat_t *ksp, int rw) -{ - kcf_prov_stats_t *ks_data; - kcf_provider_desc_t *pd = (kcf_provider_desc_t *)ksp->ks_private; - - if (rw == KSTAT_WRITE) - return (EACCES); - - ks_data = ksp->ks_data; - - ks_data->ps_ops_total.value.ui64 = pd->pd_sched_info.ks_ndispatches; - ks_data->ps_ops_failed.value.ui64 = pd->pd_sched_info.ks_nfails; - ks_data->ps_ops_busy_rval.value.ui64 = pd->pd_sched_info.ks_nbusy_rval; - ks_data->ps_ops_passed.value.ui64 = - pd->pd_sched_info.ks_ndispatches - - pd->pd_sched_info.ks_nfails - - pd->pd_sched_info.ks_nbusy_rval; - - return (0); -} - - /* * Utility routine called from failure paths in crypto_register_provider() * and from crypto_load_soft_disabled(). @@ -733,193 +278,3 @@ undo_register_provider(kcf_provider_desc_t *desc, boolean_t remove_prov) if (remove_prov) (void) kcf_prov_tab_rem_provider(desc->pd_prov_id); } - -/* - * Utility routine called from crypto_load_soft_disabled(). Callers - * should have done a prior undo_register_provider(). - */ -void -redo_register_provider(kcf_provider_desc_t *pd) -{ - /* process the mechanisms supported by the provider */ - (void) init_prov_mechs(NULL, pd); - - /* - * Hold provider in providers table. We should not call - * kcf_prov_tab_add_provider() here as the provider descriptor - * is still valid which means it has an entry in the provider - * table. - */ - KCF_PROV_REFHOLD(pd); - KCF_PROV_IREFHOLD(pd); -} - -/* - * Add provider (p1) to another provider's array of providers (p2). - * Hardware and logical providers use this array to cross-reference - * each other. - */ -static void -add_provider_to_array(kcf_provider_desc_t *p1, kcf_provider_desc_t *p2) -{ - kcf_provider_list_t *new; - - new = kmem_alloc(sizeof (kcf_provider_list_t), KM_SLEEP); - mutex_enter(&p2->pd_lock); - new->pl_next = p2->pd_provider_list; - p2->pd_provider_list = new; - KCF_PROV_IREFHOLD(p1); - new->pl_provider = p1; - mutex_exit(&p2->pd_lock); -} - -/* - * Remove provider (p1) from another provider's array of providers (p2). - * Hardware and logical providers use this array to cross-reference - * each other. - */ -static void -remove_provider_from_array(kcf_provider_desc_t *p1, kcf_provider_desc_t *p2) -{ - - kcf_provider_list_t *pl = NULL, **prev; - - mutex_enter(&p2->pd_lock); - for (pl = p2->pd_provider_list, prev = &p2->pd_provider_list; - pl != NULL; prev = &pl->pl_next, pl = pl->pl_next) { - if (pl->pl_provider == p1) { - break; - } - } - - if (p1 == NULL) { - mutex_exit(&p2->pd_lock); - return; - } - - /* detach and free kcf_provider_list structure */ - KCF_PROV_IREFRELE(p1); - *prev = pl->pl_next; - kmem_free(pl, sizeof (*pl)); - mutex_exit(&p2->pd_lock); -} - -/* - * Convert an array of logical provider handles (crypto_provider_id) - * stored in a crypto_provider_info structure into an array of provider - * descriptors (kcf_provider_desc_t) attached to a logical provider. - */ -static void -process_logical_providers(const crypto_provider_info_t *info, - kcf_provider_desc_t *hp) -{ - kcf_provider_desc_t *lp; - crypto_provider_id_t handle; - int count = info->pi_logical_provider_count; - int i; - - /* add hardware provider to each logical provider */ - for (i = 0; i < count; i++) { - handle = info->pi_logical_providers[i]; - lp = kcf_prov_tab_lookup((crypto_provider_id_t)handle); - if (lp == NULL) { - continue; - } - add_provider_to_array(hp, lp); - hp->pd_flags |= KCF_LPROV_MEMBER; - - /* - * A hardware provider has to have the provider descriptor of - * every logical provider it belongs to, so it can be removed - * from the logical provider if the hardware provider - * unregisters from the framework. - */ - add_provider_to_array(lp, hp); - KCF_PROV_REFRELE(lp); - } -} - -/* - * This routine removes a provider from all of the logical or - * hardware providers it belongs to, and frees the provider's - * array of pointers to providers. - */ -static void -remove_provider(kcf_provider_desc_t *pp) -{ - kcf_provider_desc_t *p; - kcf_provider_list_t *e, *next; - - mutex_enter(&pp->pd_lock); - for (e = pp->pd_provider_list; e != NULL; e = next) { - p = e->pl_provider; - remove_provider_from_array(pp, p); - if (p->pd_prov_type == CRYPTO_HW_PROVIDER && - p->pd_provider_list == NULL) - p->pd_flags &= ~KCF_LPROV_MEMBER; - KCF_PROV_IREFRELE(p); - next = e->pl_next; - kmem_free(e, sizeof (*e)); - } - pp->pd_provider_list = NULL; - mutex_exit(&pp->pd_lock); -} - -/* - * Dispatch events as needed for a provider. is_added flag tells - * whether the provider is registering or unregistering. - */ -void -kcf_do_notify(kcf_provider_desc_t *prov_desc, boolean_t is_added) -{ - int i; - crypto_notify_event_change_t ec; - - ASSERT(prov_desc->pd_state > KCF_PROV_VERIFICATION_FAILED); - - /* - * Inform interested clients of the mechanisms becoming - * available/unavailable. We skip this for logical providers - * as they do not affect mechanisms. - */ - if (prov_desc->pd_prov_type != CRYPTO_LOGICAL_PROVIDER) { - ec.ec_provider_type = prov_desc->pd_prov_type; - ec.ec_change = is_added ? CRYPTO_MECH_ADDED : - CRYPTO_MECH_REMOVED; - for (i = 0; i < prov_desc->pd_mech_list_count; i++) { - (void) strlcpy(ec.ec_mech_name, - prov_desc->pd_mechanisms[i].cm_mech_name, - CRYPTO_MAX_MECH_NAME); - kcf_walk_ntfylist(CRYPTO_EVENT_MECHS_CHANGED, &ec); - } - - } - - /* - * Inform interested clients about the new or departing provider. - * In case of a logical provider, we need to notify the event only - * for the logical provider and not for the underlying - * providers which are known by the KCF_LPROV_MEMBER bit. - */ - if (prov_desc->pd_prov_type == CRYPTO_LOGICAL_PROVIDER || - (prov_desc->pd_flags & KCF_LPROV_MEMBER) == 0) { - kcf_walk_ntfylist(is_added ? CRYPTO_EVENT_PROVIDER_REGISTERED : - CRYPTO_EVENT_PROVIDER_UNREGISTERED, prov_desc); - } -} - -static void -delete_kstat(kcf_provider_desc_t *desc) -{ - /* destroy the kstat created for this provider */ - if (desc->pd_kstat != NULL) { - kcf_provider_desc_t *kspd = desc->pd_kstat->ks_private; - - /* release reference held by desc->pd_kstat->ks_private */ - ASSERT(desc == kspd); - kstat_delete(kspd->pd_kstat); - desc->pd_kstat = NULL; - KCF_PROV_REFRELE(kspd); - KCF_PROV_IREFRELE(kspd); - } -} diff --git a/module/lua/Makefile.in b/module/lua/Makefile.in deleted file mode 100644 index 0a74c17e64e8..000000000000 --- a/module/lua/Makefile.in +++ /dev/null @@ -1,39 +0,0 @@ -ifneq ($(KBUILD_EXTMOD),) -src = @abs_srcdir@ -obj = @abs_builddir@ -endif - -MODULE := zlua - -obj-$(CONFIG_ZFS) := $(MODULE).o - -ccflags-y := -DLUA_USE_LONGLONG - -$(MODULE)-objs += lapi.o -$(MODULE)-objs += lauxlib.o -$(MODULE)-objs += lbaselib.o -$(MODULE)-objs += lcode.o -$(MODULE)-objs += lcompat.o -$(MODULE)-objs += lcorolib.o -$(MODULE)-objs += lctype.o -$(MODULE)-objs += ldebug.o -$(MODULE)-objs += ldo.o -$(MODULE)-objs += lfunc.o -$(MODULE)-objs += lgc.o -$(MODULE)-objs += llex.o -$(MODULE)-objs += lmem.o -$(MODULE)-objs += lobject.o -$(MODULE)-objs += lopcodes.o -$(MODULE)-objs += lparser.o -$(MODULE)-objs += lstate.o -$(MODULE)-objs += lstring.o -$(MODULE)-objs += lstrlib.o -$(MODULE)-objs += ltable.o -$(MODULE)-objs += ltablib.o -$(MODULE)-objs += ltm.o -$(MODULE)-objs += lvm.o -$(MODULE)-objs += lzio.o -$(MODULE)-objs += setjmp/setjmp.o - -all: - mkdir -p setjmp diff --git a/module/lua/lapi.c b/module/lua/lapi.c index 72b0037aa9a9..726e5c2ad4bb 100644 --- a/module/lua/lapi.c +++ b/module/lua/lapi.c @@ -1278,29 +1278,6 @@ LUA_API void lua_upvaluejoin (lua_State *L, int fidx1, int n1, luaC_objbarrier(L, f1, *up2); } -#if defined(_KERNEL) - -static int __init -lua_init(void) -{ - return (0); -} - -static void __exit -lua_fini(void) -{ -} - -module_init(lua_init); -module_exit(lua_fini); - -#endif - -ZFS_MODULE_DESCRIPTION("Lua Interpreter for ZFS"); -ZFS_MODULE_AUTHOR("Lua.org"); -ZFS_MODULE_LICENSE("Dual MIT/GPL"); -ZFS_MODULE_VERSION(ZFS_META_VERSION "-" ZFS_META_RELEASE); - EXPORT_SYMBOL(lua_absindex); EXPORT_SYMBOL(lua_atpanic); EXPORT_SYMBOL(lua_checkstack); diff --git a/module/lua/ldo.c b/module/lua/ldo.c index 01e5d6fd32d6..b9368d9ceab6 100644 --- a/module/lua/ldo.c +++ b/module/lua/ldo.c @@ -91,7 +91,7 @@ static intptr_t stack_remaining(void) { typedef struct _label_t { long long unsigned val[JMP_BUF_CNT]; } label_t; int setjmp(label_t *) __attribute__ ((__nothrow__)); -extern void longjmp(label_t *) __attribute__((__noreturn__)); +extern __attribute__((noreturn)) void longjmp(label_t *); #define LUAI_THROW(L,c) longjmp(&(c)->b) #define LUAI_TRY(L,c,a) if (setjmp(&(c)->b) == 0) { a } @@ -167,6 +167,13 @@ static void seterrorobj (lua_State *L, int errcode, StkId oldtop) { L->top = oldtop + 1; } +/* + * Silence infinite recursion warning which was added to -Wall in gcc 12.1 + */ +#if defined(HAVE_INFINITE_RECURSION) +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Winfinite-recursion" +#endif l_noret luaD_throw (lua_State *L, int errcode) { if (L->errorJmp) { /* thread has an error handler? */ @@ -189,6 +196,10 @@ l_noret luaD_throw (lua_State *L, int errcode) { } } +#if defined(HAVE_INFINITE_RECURSION) +#pragma GCC diagnostic pop +#endif + int luaD_rawrunprotected (lua_State *L, Pfunc f, void *ud) { unsigned short oldnCcalls = L->nCcalls; diff --git a/module/lua/setjmp/setjmp_i386.S b/module/lua/setjmp/setjmp_i386.S index 6d6a5f332688..0d0adfc351ca 100644 --- a/module/lua/setjmp/setjmp_i386.S +++ b/module/lua/setjmp/setjmp_i386.S @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/lua/setjmp/setjmp_x86_64.S b/module/lua/setjmp/setjmp_x86_64.S index a469cbad780e..fd661d72eedf 100644 --- a/module/lua/setjmp/setjmp_x86_64.S +++ b/module/lua/setjmp/setjmp_x86_64.S @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -35,6 +35,12 @@ x: .size x, [.-x] +#if defined(__linux__) && defined(CONFIG_SLS) +#define RET ret; int3 +#else +#define RET ret +#endif + /* * Setjmp and longjmp implement non-local gotos using state vectors * type label_t. @@ -52,7 +58,7 @@ x: movq 0(%rsp), %rdx /* return address */ movq %rdx, 56(%rdi) /* rip */ xorl %eax, %eax /* return 0 */ - ret + RET SET_SIZE(setjmp) ENTRY(longjmp) @@ -67,7 +73,7 @@ x: movq %rdx, 0(%rsp) xorl %eax, %eax incl %eax /* return 1 */ - ret + RET SET_SIZE(longjmp) #ifdef __ELF__ diff --git a/module/nvpair/Makefile.in b/module/nvpair/Makefile.in deleted file mode 100644 index d8145236674b..000000000000 --- a/module/nvpair/Makefile.in +++ /dev/null @@ -1,13 +0,0 @@ -ifneq ($(KBUILD_EXTMOD),) -src = @abs_srcdir@ -obj = @abs_builddir@ -endif - -MODULE := znvpair - -obj-$(CONFIG_ZFS) := $(MODULE).o - -$(MODULE)-objs += nvpair.o -$(MODULE)-objs += fnvpair.o -$(MODULE)-objs += nvpair_alloc_spl.o -$(MODULE)-objs += nvpair_alloc_fixed.o diff --git a/module/nvpair/fnvpair.c b/module/nvpair/fnvpair.c index 43c4b73590c4..86b0b4cdfc6e 100644 --- a/module/nvpair/fnvpair.c +++ b/module/nvpair/fnvpair.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/nvpair/nvpair.c b/module/nvpair/nvpair.c index b4463dd7308f..c8161f116b60 100644 --- a/module/nvpair/nvpair.c +++ b/module/nvpair/nvpair.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -40,7 +40,7 @@ #include #include #include -#include +#include #include #include @@ -53,7 +53,7 @@ #include #endif -#define skip_whitespace(p) while ((*(p) == ' ') || (*(p) == '\t')) p++ +#define skip_whitespace(p) while ((*(p) == ' ') || (*(p) == '\t')) (p)++ /* * nvpair.c - Provides kernel & userland interfaces for manipulating @@ -203,7 +203,7 @@ nv_mem_zalloc(nvpriv_t *nvp, size_t size) void *buf; if ((buf = nva->nva_ops->nv_ao_alloc(nva, size)) != NULL) - bzero(buf, size); + memset(buf, 0, size); return (buf); } @@ -219,7 +219,7 @@ nv_mem_free(nvpriv_t *nvp, void *buf, size_t size) static void nv_priv_init(nvpriv_t *priv, nv_alloc_t *nva, uint32_t stat) { - bzero(priv, sizeof (nvpriv_t)); + memset(priv, 0, sizeof (nvpriv_t)); priv->nvp_nva = nva; priv->nvp_stat = stat; @@ -1203,7 +1203,7 @@ nvlist_add_common(nvlist_t *nvl, const char *name, nvp->nvp_name_sz = name_sz; nvp->nvp_value_elem = nelem; nvp->nvp_type = type; - bcopy(name, NVP_NAME(nvp), name_sz); + memcpy(NVP_NAME(nvp), name, name_sz); switch (type) { case DATA_TYPE_BOOLEAN: @@ -1217,7 +1217,7 @@ nvlist_add_common(nvlist_t *nvl, const char *name, buf += nelem * sizeof (uint64_t); for (i = 0; i < nelem; i++) { int slen = strlen(strs[i]) + 1; - bcopy(strs[i], buf, slen); + memcpy(buf, strs[i], slen); cstrs[i] = buf; buf += slen; } @@ -1255,7 +1255,7 @@ nvlist_add_common(nvlist_t *nvl, const char *name, break; } default: - bcopy(data, NVP_VALUE(nvp), value_sz); + memcpy(NVP_VALUE(nvp), data, value_sz); } /* if unique name, remove before add */ @@ -1588,7 +1588,7 @@ nvpair_value_common(const nvpair_t *nvp, data_type_t type, uint_t *nelem, return (EINVAL); if ((value_sz = i_get_value_size(type, NULL, 1)) < 0) return (EINVAL); - bcopy(NVP_VALUE(nvp), data, (size_t)value_sz); + memcpy(data, NVP_VALUE(nvp), (size_t)value_sz); if (nelem != NULL) *nelem = 1; break; @@ -2540,7 +2540,7 @@ nvs_embedded_nvl_array(nvstream_t *nvs, nvpair_t *nvp, size_t *size) size_t len = nelem * sizeof (uint64_t); nvlist_t *embedded = (nvlist_t *)((uintptr_t)nvlp + len); - bzero(nvlp, len); /* don't trust packed data */ + memset(nvlp, 0, len); /* don't trust packed data */ for (i = 0; i < nelem; i++) { if (nvs_embedded(nvs, embedded) != 0) { nvpair_free(nvp); @@ -2820,15 +2820,15 @@ native_cp(nvstream_t *nvs, void *buf, size_t size) return (EFAULT); /* - * The bcopy() below eliminates alignment requirement + * The memcpy() below eliminates alignment requirement * on the buffer (stream) and is preferred over direct access. */ switch (nvs->nvs_op) { case NVS_OP_ENCODE: - bcopy(buf, native->n_curr, size); + memcpy(native->n_curr, buf, size); break; case NVS_OP_DECODE: - bcopy(native->n_curr, buf, size); + memcpy(buf, native->n_curr, size); break; default: return (EINVAL); @@ -2895,7 +2895,7 @@ nvs_native_nvl_fini(nvstream_t *nvs) if (native->n_curr + sizeof (int) > native->n_end) return (EFAULT); - bzero(native->n_curr, sizeof (int)); + memset(native->n_curr, 0, sizeof (int)); native->n_curr += sizeof (int); } @@ -2912,10 +2912,10 @@ nvpair_native_embedded(nvstream_t *nvs, nvpair_t *nvp) /* * Null out the pointer that is meaningless in the packed * structure. The address may not be aligned, so we have - * to use bzero. + * to use memset. */ - bzero((char *)packed + offsetof(nvlist_t, nvl_priv), - sizeof (uint64_t)); + memset((char *)packed + offsetof(nvlist_t, nvl_priv), + 0, sizeof (uint64_t)); } return (nvs_embedded(nvs, EMBEDDED_NVL(nvp))); @@ -2933,18 +2933,18 @@ nvpair_native_embedded_array(nvstream_t *nvs, nvpair_t *nvp) /* * Null out pointers that are meaningless in the packed * structure. The addresses may not be aligned, so we have - * to use bzero. + * to use memset. */ - bzero(value, len); + memset(value, 0, len); for (i = 0; i < NVP_NELEM(nvp); i++, packed++) /* * Null out the pointer that is meaningless in the * packed structure. The address may not be aligned, - * so we have to use bzero. + * so we have to use memset. */ - bzero((char *)packed + offsetof(nvlist_t, nvl_priv), - sizeof (uint64_t)); + memset((char *)packed + offsetof(nvlist_t, nvl_priv), + 0, sizeof (uint64_t)); } return (nvs_embedded_nvl_array(nvs, nvp, NULL)); @@ -2961,9 +2961,9 @@ nvpair_native_string_array(nvstream_t *nvs, nvpair_t *nvp) /* * Null out pointers that are meaningless in the packed * structure. The addresses may not be aligned, so we have - * to use bzero. + * to use memset. */ - bzero(strp, NVP_NELEM(nvp) * sizeof (uint64_t)); + memset(strp, 0, NVP_NELEM(nvp) * sizeof (uint64_t)); break; } case NVS_OP_DECODE: { @@ -2988,9 +2988,9 @@ nvs_native_nvp_op(nvstream_t *nvs, nvpair_t *nvp) int ret = 0; /* - * We do the initial bcopy of the data before we look at + * We do the initial memcpy of the data before we look at * the nvpair type, because when we're decoding, we won't - * have the correct values for the pair until we do the bcopy. + * have the correct values for the pair until we do the memcpy. */ switch (nvs->nvs_op) { case NVS_OP_ENCODE: @@ -3086,7 +3086,7 @@ nvs_native_nvpair(nvstream_t *nvs, nvpair_t *nvp, size_t *size) /* try to read the size value from the stream */ if (native->n_curr + sizeof (int32_t) > native->n_end) return (EFAULT); - bcopy(native->n_curr, &decode_len, sizeof (int32_t)); + memcpy(&decode_len, native->n_curr, sizeof (int32_t)); /* sanity check the size value */ if (decode_len < 0 || @@ -3451,7 +3451,7 @@ nvs_xdr_nvp_op(nvstream_t *nvs, nvpair_t *nvp) int i; if (nvs->nvs_op == NVS_OP_DECODE) - bzero(buf, len); /* don't trust packed data */ + memset(buf, 0, len); /* don't trust packed data */ for (i = 0; i < nelem; i++) { if (buflen <= len) @@ -3678,27 +3678,6 @@ nvs_xdr(nvstream_t *nvs, nvlist_t *nvl, char *buf, size_t *buflen) return (err); } -#if defined(_KERNEL) -static int __init -nvpair_init(void) -{ - return (0); -} - -static void __exit -nvpair_fini(void) -{ -} - -module_init(nvpair_init); -module_exit(nvpair_fini); -#endif - -ZFS_MODULE_DESCRIPTION("Generic name/value pair implementation"); -ZFS_MODULE_AUTHOR(ZFS_META_AUTHOR); -ZFS_MODULE_LICENSE(ZFS_META_LICENSE); -ZFS_MODULE_VERSION(ZFS_META_VERSION "-" ZFS_META_RELEASE); - EXPORT_SYMBOL(nv_alloc_init); EXPORT_SYMBOL(nv_alloc_reset); EXPORT_SYMBOL(nv_alloc_fini); diff --git a/module/nvpair/nvpair_alloc_fixed.c b/module/nvpair/nvpair_alloc_fixed.c index d7d3e7afd520..34adb86e0301 100644 --- a/module/nvpair/nvpair_alloc_fixed.c +++ b/module/nvpair/nvpair_alloc_fixed.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/nvpair/nvpair_alloc_spl.c b/module/nvpair/nvpair_alloc_spl.c index aa344b6423e8..4ecb029961f5 100644 --- a/module/nvpair/nvpair_alloc_spl.c +++ b/module/nvpair/nvpair_alloc_spl.c @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at * usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/freebsd/spl/acl_common.c b/module/os/freebsd/spl/acl_common.c index 7fd0e36e1ba7..b692ccdf232a 100644 --- a/module/os/freebsd/spl/acl_common.c +++ b/module/os/freebsd/spl/acl_common.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -37,7 +37,6 @@ #include #include #include -#include #include #include #include @@ -738,7 +737,7 @@ ace_mask_to_mode(uint32_t mask, o_mode_t *modep, boolean_t isdir) static void acevals_init(acevals_t *vals, uid_t key) { - bzero(vals, sizeof (*vals)); + memset(vals, 0, sizeof (*vals)); vals->allowed = ACE_MASK_UNDEFINED; vals->denied = ACE_MASK_UNDEFINED; vals->mask = ACE_MASK_UNDEFINED; diff --git a/module/os/freebsd/spl/callb.c b/module/os/freebsd/spl/callb.c index 0b7fefc89a26..ba13ea887938 100644 --- a/module/os/freebsd/spl/callb.c +++ b/module/os/freebsd/spl/callb.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -338,10 +338,10 @@ callb_generic_cpr(void *arg, int code) * The generic callback function associated with kernel threads which * are always considered safe. */ -/* ARGSUSED */ boolean_t callb_generic_cpr_safe(void *arg, int code) { + (void) arg, (void) code; return (B_TRUE); } /* diff --git a/module/os/freebsd/spl/list.c b/module/os/freebsd/spl/list.c index 62374a417704..ab6049cfbd43 100644 --- a/module/os/freebsd/spl/list.c +++ b/module/os/freebsd/spl/list.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/freebsd/spl/sha256c.c b/module/os/freebsd/spl/sha256c.c index 241cf8c9ae76..52cf0df6c99d 100644 --- a/module/os/freebsd/spl/sha256c.c +++ b/module/os/freebsd/spl/sha256c.c @@ -301,7 +301,7 @@ SHA256_Final(unsigned char digest[static SHA256_DIGEST_LENGTH], SHA256_CTX *ctx) be32enc_vect(digest, ctx->state, SHA256_DIGEST_LENGTH); /* Clear the context state */ - explicit_bzero(ctx, sizeof (*ctx)); + memset(ctx, 0, sizeof (*ctx)); } /* SHA-224: ******************************************************* */ @@ -351,7 +351,7 @@ SHA224_Final(unsigned char digest[static SHA224_DIGEST_LENGTH], SHA224_CTX *ctx) be32enc_vect(digest, ctx->state, SHA224_DIGEST_LENGTH); /* Clear the context state */ - explicit_bzero(ctx, sizeof (*ctx)); + memset(ctx, 0, sizeof (*ctx)); } #ifdef WEAK_REFS diff --git a/module/os/freebsd/spl/sha512c.c b/module/os/freebsd/spl/sha512c.c index 146f338f0ed4..254cc21565c1 100644 --- a/module/os/freebsd/spl/sha512c.c +++ b/module/os/freebsd/spl/sha512c.c @@ -333,7 +333,7 @@ SHA512_Final(unsigned char digest[static SHA512_DIGEST_LENGTH], SHA512_CTX *ctx) be64enc_vect(digest, ctx->state, SHA512_DIGEST_LENGTH); /* Clear the context state */ - explicit_bzero(ctx, sizeof (*ctx)); + memset(ctx, 0, sizeof (*ctx)); } /* SHA-512t: ******************************************************** */ @@ -377,7 +377,7 @@ SHA512_224_Final(unsigned char digest[static SHA512_224_DIGEST_LENGTH], be64enc_vect(digest, ctx->state, SHA512_224_DIGEST_LENGTH); /* Clear the context state */ - explicit_bzero(ctx, sizeof (*ctx)); + memset(ctx, 0, sizeof (*ctx)); } void @@ -417,7 +417,7 @@ SHA512_256_Final(unsigned char digest[static SHA512_256_DIGEST_LENGTH], be64enc_vect(digest, ctx->state, SHA512_256_DIGEST_LENGTH); /* Clear the context state */ - explicit_bzero(ctx, sizeof (*ctx)); + memset(ctx, 0, sizeof (*ctx)); } /* ** SHA-384: ******************************************************** */ @@ -467,7 +467,7 @@ SHA384_Final(unsigned char digest[static SHA384_DIGEST_LENGTH], SHA384_CTX *ctx) be64enc_vect(digest, ctx->state, SHA384_DIGEST_LENGTH); /* Clear the context state */ - explicit_bzero(ctx, sizeof (*ctx)); + memset(ctx, 0, sizeof (*ctx)); } #if 0 diff --git a/module/os/freebsd/spl/spl_acl.c b/module/os/freebsd/spl/spl_acl.c index 74c26d03f87f..4d67cbb183ec 100644 --- a/module/os/freebsd/spl/spl_acl.c +++ b/module/os/freebsd/spl/spl_acl.c @@ -40,7 +40,7 @@ struct zfs2bsd { int zb_bsd; }; -struct zfs2bsd perms[] = {{ACE_READ_DATA, ACL_READ_DATA}, +static const struct zfs2bsd perms[] = {{ACE_READ_DATA, ACL_READ_DATA}, {ACE_WRITE_DATA, ACL_WRITE_DATA}, {ACE_EXECUTE, ACL_EXECUTE}, {ACE_APPEND_DATA, ACL_APPEND_DATA}, @@ -56,7 +56,7 @@ struct zfs2bsd perms[] = {{ACE_READ_DATA, ACL_READ_DATA}, {ACE_SYNCHRONIZE, ACL_SYNCHRONIZE}, {0, 0}}; -struct zfs2bsd flags[] = {{ACE_FILE_INHERIT_ACE, +static const struct zfs2bsd flags[] = {{ACE_FILE_INHERIT_ACE, ACL_ENTRY_FILE_INHERIT}, {ACE_DIRECTORY_INHERIT_ACE, ACL_ENTRY_DIRECTORY_INHERIT}, @@ -122,7 +122,7 @@ acl_from_aces(struct acl *aclp, const ace_t *aces, int nentries) return (EINVAL); } - bzero(aclp, sizeof (*aclp)); + memset(aclp, 0, sizeof (*aclp)); aclp->acl_maxcnt = ACL_MAX_ENTRIES; aclp->acl_cnt = nentries; @@ -177,7 +177,7 @@ aces_from_acl(ace_t *aces, int *nentries, const struct acl *aclp) const struct acl_entry *entry; ace_t *ace; - bzero(aces, sizeof (*aces) * aclp->acl_cnt); + memset(aces, 0, sizeof (*aces) * aclp->acl_cnt); *nentries = aclp->acl_cnt; diff --git a/module/os/freebsd/spl/spl_cmn_err.c b/module/os/freebsd/spl/spl_cmn_err.c index 22c7338b7399..2157da8ebf20 100644 --- a/module/os/freebsd/spl/spl_cmn_err.c +++ b/module/os/freebsd/spl/spl_cmn_err.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/freebsd/spl/spl_kmem.c b/module/os/freebsd/spl/spl_kmem.c index ee8f1d851a48..ca9a677567d9 100644 --- a/module/os/freebsd/spl/spl_kmem.c +++ b/module/os/freebsd/spl/spl_kmem.c @@ -156,7 +156,7 @@ kmem_std_destructor(void *mem, int size __unused, void *private) } kmem_cache_t * -kmem_cache_create(char *name, size_t bufsize, size_t align, +kmem_cache_create(const char *name, size_t bufsize, size_t align, int (*constructor)(void *, void *, int), void (*destructor)(void *, void *), void (*reclaim)(void *) __unused, void *private, vmem_t *vmp, int cflags) { diff --git a/module/os/freebsd/spl/spl_misc.c b/module/os/freebsd/spl/spl_misc.c index 0354b986cd5f..e46271a039de 100644 --- a/module/os/freebsd/spl/spl_misc.c +++ b/module/os/freebsd/spl/spl_misc.c @@ -43,15 +43,11 @@ static struct opensolaris_utsname hw_utsname = { .machine = MACHINE }; -#ifndef KERNEL_STATIC -char hw_serial[11] = "0"; - utsname_t * utsname(void) { return (&hw_utsname); } -#endif static void opensolaris_utsname_init(void *arg) diff --git a/module/os/freebsd/spl/spl_string.c b/module/os/freebsd/spl/spl_string.c index 00b1df766ab9..523e10ff6936 100644 --- a/module/os/freebsd/spl/spl_string.c +++ b/module/os/freebsd/spl/spl_string.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/freebsd/spl/spl_sunddi.c b/module/os/freebsd/spl/spl_sunddi.c index ebec77bdb37f..2a3c027c9389 100644 --- a/module/os/freebsd/spl/spl_sunddi.c +++ b/module/os/freebsd/spl/spl_sunddi.c @@ -45,19 +45,6 @@ ddi_strtol(const char *str, char **nptr, int base, long *result) return (0); } -int -ddi_strtoul(const char *str, char **nptr, int base, unsigned long *result) -{ - - if (str == hw_serial) { - *result = prison0.pr_hostid; - return (0); - } - - *result = strtoul(str, nptr, base); - return (0); -} - int ddi_strtoull(const char *str, char **nptr, int base, unsigned long long *result) { diff --git a/module/os/freebsd/spl/spl_sysevent.c b/module/os/freebsd/spl/spl_sysevent.c index d5d50080fafd..16188c71b53d 100644 --- a/module/os/freebsd/spl/spl_sysevent.c +++ b/module/os/freebsd/spl/spl_sysevent.c @@ -250,7 +250,17 @@ sysevent_worker(void *arg __unused) nvlist_free(event); } } - zfs_zevent_destroy(ze); + + /* + * We avoid zfs_zevent_destroy() here because we're otherwise racing + * against fm_fini() destroying the zevent_lock. zfs_zevent_destroy() + * will currently only clear `ze->ze_zevent` from an event list then + * free `ze`, so just inline the free() here -- events have already + * been drained. + */ + VERIFY3P(ze->ze_zevent, ==, NULL); + kmem_free(ze, sizeof (zfs_zevent_t)); + kthread_exit(); } diff --git a/module/os/freebsd/spl/spl_uio.c b/module/os/freebsd/spl/spl_uio.c index 0bf251a1edac..ffccb6f2594e 100644 --- a/module/os/freebsd/spl/spl_uio.c +++ b/module/os/freebsd/spl/spl_uio.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/freebsd/spl/spl_vfs.c b/module/os/freebsd/spl/spl_vfs.c index 3f4feb140d5e..ff11f5d7acb8 100644 --- a/module/os/freebsd/spl/spl_vfs.c +++ b/module/os/freebsd/spl/spl_vfs.c @@ -85,7 +85,7 @@ vfs_setmntopt(vfs_t *vfsp, const char *name, const char *arg, } else { opt->len = strlen(arg) + 1; opt->value = malloc(opt->len, M_MOUNT, M_WAITOK); - bcopy(arg, opt->value, opt->len); + memcpy(opt->value, arg, opt->len); } MNT_ILOCK(vfsp); diff --git a/module/os/freebsd/spl/spl_zlib.c b/module/os/freebsd/spl/spl_zlib.c index 3644eba77ca1..8bd3bdedf268 100644 --- a/module/os/freebsd/spl/spl_zlib.c +++ b/module/os/freebsd/spl/spl_zlib.c @@ -40,19 +40,17 @@ __FBSDID("$FreeBSD$"); #include -/*ARGSUSED*/ static void * zcalloc(void *opaque, uint_t items, uint_t size) { - + (void) opaque; return (malloc((size_t)items*size, M_SOLARIS, M_NOWAIT)); } -/*ARGSUSED*/ static void zcfree(void *opaque, void *ptr) { - + (void) opaque; free(ptr, M_SOLARIS); } @@ -143,10 +141,9 @@ int z_compress_level(void *dest, size_t *destLen, const void *source, size_t sourceLen, int level) { - z_stream stream; + z_stream stream = {0}; int err; - bzero(&stream, sizeof (stream)); stream.next_in = (Byte *)source; stream.avail_in = (uInt)sourceLen; stream.next_out = dest; @@ -198,11 +195,9 @@ z_compress_level(void *dest, size_t *destLen, const void *source, int z_uncompress(void *dest, size_t *destLen, const void *source, size_t sourceLen) { - z_stream stream; + z_stream stream = {0}; int err; - bzero(&stream, sizeof (stream)); - stream.next_in = (Byte *)source; stream.avail_in = (uInt)sourceLen; stream.next_out = dest; diff --git a/module/os/freebsd/spl/spl_zone.c b/module/os/freebsd/spl/spl_zone.c index bd3f019b2fa6..658ef0bf056d 100644 --- a/module/os/freebsd/spl/spl_zone.c +++ b/module/os/freebsd/spl/spl_zone.c @@ -184,7 +184,7 @@ zone_dataset_visible(const char *dataset, int *write) LIST_FOREACH(zd, head, zd_next) { len = strlen(zd->zd_dataset); if (strlen(dataset) >= len && - bcmp(dataset, zd->zd_dataset, len) == 0 && + memcmp(dataset, zd->zd_dataset, len) == 0 && (dataset[len] == '\0' || dataset[len] == '/' || dataset[len] == '@')) { if (write) @@ -206,7 +206,7 @@ zone_dataset_visible(const char *dataset, int *write) if (dataset[len - 1] == '/') len--; /* Ignore trailing slash */ if (len < strlen(zd->zd_dataset) && - bcmp(dataset, zd->zd_dataset, len) == 0 && + memcmp(dataset, zd->zd_dataset, len) == 0 && zd->zd_dataset[len] == '/') { if (write) *write = 0; diff --git a/module/os/freebsd/zfs/abd_os.c b/module/os/freebsd/zfs/abd_os.c index 722a8898cde8..58a37df62b69 100644 --- a/module/os/freebsd/zfs/abd_os.c +++ b/module/os/freebsd/zfs/abd_os.c @@ -113,7 +113,6 @@ static kstat_t *abd_ksp; * memory by only using a single zero buffer for the scatter chunks. */ abd_t *abd_zero_scatter = NULL; -static char *abd_zero_buf = NULL; static uint_t abd_chunkcnt_for_bytes(size_t size) @@ -241,18 +240,16 @@ abd_free_struct_impl(abd_t *abd) /* * Allocate scatter ABD of size SPA_MAXBLOCKSIZE, where - * each chunk in the scatterlist will be set to abd_zero_buf. + * each chunk in the scatterlist will be set to the same area. */ +_Static_assert(ZERO_REGION_SIZE >= PAGE_SIZE, "zero_region too small"); static void abd_alloc_zero_scatter(void) { uint_t i, n; n = abd_chunkcnt_for_bytes(SPA_MAXBLOCKSIZE); - abd_zero_buf = kmem_cache_alloc(abd_chunk_cache, KM_PUSHPAGE); - bzero(abd_zero_buf, PAGE_SIZE); abd_zero_scatter = abd_alloc_struct(SPA_MAXBLOCKSIZE); - abd_zero_scatter->abd_flags |= ABD_FLAG_OWNER | ABD_FLAG_ZEROS; abd_zero_scatter->abd_size = SPA_MAXBLOCKSIZE; @@ -260,7 +257,7 @@ abd_alloc_zero_scatter(void) for (i = 0; i < n; i++) { ABD_SCATTER(abd_zero_scatter).abd_chunks[i] = - abd_zero_buf; + __DECONST(void *, zero_region); } ABDSTAT_BUMP(abdstat_scatter_cnt); @@ -275,7 +272,6 @@ abd_free_zero_scatter(void) abd_free_struct(abd_zero_scatter); abd_zero_scatter = NULL; - kmem_cache_free(abd_chunk_cache, abd_zero_buf); } static int diff --git a/module/os/freebsd/zfs/arc_os.c b/module/os/freebsd/zfs/arc_os.c index b6f19207f324..ca2bf884257f 100644 --- a/module/os/freebsd/zfs/arc_os.c +++ b/module/os/freebsd/zfs/arc_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/freebsd/zfs/crypto_os.c b/module/os/freebsd/zfs/crypto_os.c index f971b62bd124..1f139ea5b807 100644 --- a/module/os/freebsd/zfs/crypto_os.c +++ b/module/os/freebsd/zfs/crypto_os.c @@ -39,8 +39,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#else -#include #endif #include @@ -69,11 +67,11 @@ crypto_mac_init(struct hmac_ctx *ctx, const crypto_key_t *c_key) /* * This code is based on the similar code in geom/eli/g_eli_hmac.c */ - explicit_bzero(key, sizeof (key)); + memset(key, 0, sizeof (key)); if (c_key->ck_length == 0) /* do nothing */; else if (cl_bytes <= SHA512_HMAC_BLOCK_SIZE) - bcopy(c_key->ck_data, key, cl_bytes); + memcpy(key, c_key->ck_data, cl_bytes); else { /* * If key is longer than 128 bytes reset it to @@ -89,16 +87,16 @@ crypto_mac_init(struct hmac_ctx *ctx, const crypto_key_t *c_key) k_ipad[i] = key[i] ^ 0x36; k_opad[i] = key[i] ^ 0x5c; } - explicit_bzero(key, sizeof (key)); + memset(key, 0, sizeof (key)); /* Start inner SHA512. */ SHA512_Init(&ctx->innerctx); SHA512_Update(&ctx->innerctx, k_ipad, sizeof (k_ipad)); - explicit_bzero(k_ipad, sizeof (k_ipad)); + memset(k_ipad, 0, sizeof (k_ipad)); /* Start outer SHA512. */ SHA512_Init(&ctx->outerctx); SHA512_Update(&ctx->outerctx, k_opad, sizeof (k_opad)); - explicit_bzero(k_opad, sizeof (k_opad)); + memset(k_opad, 0, sizeof (k_opad)); } void @@ -119,12 +117,12 @@ crypto_mac_final(struct hmac_ctx *ctx, void *md, size_t mdsize) SHA512_Update(&ctx->outerctx, digest, sizeof (digest)); SHA512_Final(digest, &ctx->outerctx); - explicit_bzero(ctx, sizeof (*ctx)); + memset(ctx, 0, sizeof (*ctx)); /* mdsize == 0 means "Give me the whole hash!" */ if (mdsize == 0) mdsize = SHA512_DIGEST_LENGTH; - bcopy(digest, md, mdsize); - explicit_bzero(digest, sizeof (digest)); + memcpy(md, digest, mdsize); + memset(digest, 0, sizeof (digest)); } void @@ -151,35 +149,52 @@ freebsd_zfs_crypt_done(struct cryptop *crp) return (0); } +static int +freebsd_zfs_crypt_done_sync(struct cryptop *crp) +{ + + return (0); +} + void freebsd_crypt_freesession(freebsd_crypt_session_t *sess) { mtx_destroy(&sess->fs_lock); crypto_freesession(sess->fs_sid); - explicit_bzero(sess, sizeof (*sess)); + memset(sess, 0, sizeof (*sess)); } static int -zfs_crypto_dispatch(freebsd_crypt_session_t *session, struct cryptop *crp) +zfs_crypto_dispatch(freebsd_crypt_session_t *session, struct cryptop *crp) { int error; crp->crp_opaque = session; - crp->crp_callback = freebsd_zfs_crypt_done; for (;;) { +#if __FreeBSD_version < 1400004 + boolean_t async = ((crypto_ses2caps(crp->crp_session) & + CRYPTOCAP_F_SYNC) == 0); +#else + boolean_t async = !CRYPTO_SESS_SYNC(crp->crp_session); +#endif + crp->crp_callback = async ? freebsd_zfs_crypt_done : + freebsd_zfs_crypt_done_sync; error = crypto_dispatch(crp); - if (error) - break; - mtx_lock(&session->fs_lock); - while (session->fs_done == false) - msleep(crp, &session->fs_lock, 0, - "zfs_crypto", 0); - mtx_unlock(&session->fs_lock); + if (error == 0) { + if (async) { + mtx_lock(&session->fs_lock); + while (session->fs_done == false) { + msleep(crp, &session->fs_lock, 0, + "zfs_crypto", 0); + } + mtx_unlock(&session->fs_lock); + } + error = crp->crp_etype; + } - if (crp->crp_etype == ENOMEM) { + if (error == ENOMEM) { pause("zcrnomem", 1); - } else if (crp->crp_etype != EAGAIN) { - error = crp->crp_etype; + } else if (error != EAGAIN) { break; } crp->crp_etype = 0; @@ -210,12 +225,12 @@ freebsd_crypt_uio_debug_log(boolean_t encrypt, uint8_t *p = NULL; size_t total = 0; - printf("%s(%s, %p, { %s, %d, %d, %s }, %p, { %d, %p, %u }, " + printf("%s(%s, %p, { %s, %d, %d, %s }, %p, { %p, %u }, " "%p, %u, %u)\n", __FUNCTION__, encrypt ? "encrypt" : "decrypt", input_sessionp, c_info->ci_algname, c_info->ci_crypt_type, (unsigned int)c_info->ci_keylen, c_info->ci_name, - data_uio, key->ck_format, key->ck_data, + data_uio, key->ck_data, (unsigned int)key->ck_length, ivbuf, (unsigned int)datalen, (unsigned int)auth_len); printf("\tkey = { "); @@ -243,15 +258,15 @@ int freebsd_crypt_newsession(freebsd_crypt_session_t *sessp, const struct zio_crypt_info *c_info, crypto_key_t *key) { - struct crypto_session_params csp; + struct crypto_session_params csp = {0}; int error = 0; #ifdef FCRYPTO_DEBUG - printf("%s(%p, { %s, %d, %d, %s }, { %d, %p, %u })\n", + printf("%s(%p, { %s, %d, %d, %s }, { %p, %u })\n", __FUNCTION__, sessp, c_info->ci_algname, c_info->ci_crypt_type, (unsigned int)c_info->ci_keylen, c_info->ci_name, - key->ck_format, key->ck_data, (unsigned int)key->ck_length); + key->ck_data, (unsigned int)key->ck_length); printf("\tkey = { "); for (int i = 0; i < key->ck_length / 8; i++) { uint8_t *b = (uint8_t *)key->ck_data; @@ -259,7 +274,6 @@ freebsd_crypt_newsession(freebsd_crypt_session_t *sessp, } printf("}\n"); #endif - bzero(&csp, sizeof (csp)); csp.csp_mode = CSP_MODE_AEAD; csp.csp_cipher_key = key->ck_data; csp.csp_cipher_klen = key->ck_length / 8; @@ -364,7 +378,7 @@ freebsd_crypt_uio(boolean_t encrypt, crp->crp_payload_length = datalen; crp->crp_digest_start = auth_len + datalen; - bcopy(ivbuf, crp->crp_iv, ZIO_DATA_IV_LEN); + memcpy(crp->crp_iv, ivbuf, ZIO_DATA_IV_LEN); error = zfs_crypto_dispatch(session, crp); crypto_freereq(crp); out: @@ -384,18 +398,18 @@ int freebsd_crypt_newsession(freebsd_crypt_session_t *sessp, const struct zio_crypt_info *c_info, crypto_key_t *key) { - struct cryptoini cria, crie, *crip; + struct cryptoini cria = {0}, crie = {0}, *crip; struct enc_xform *xform; struct auth_hash *xauth; int error = 0; crypto_session_t sid; #ifdef FCRYPTO_DEBUG - printf("%s(%p, { %s, %d, %d, %s }, { %d, %p, %u })\n", + printf("%s(%p, { %s, %d, %d, %s }, { %p, %u })\n", __FUNCTION__, sessp, c_info->ci_algname, c_info->ci_crypt_type, (unsigned int)c_info->ci_keylen, c_info->ci_name, - key->ck_format, key->ck_data, (unsigned int)key->ck_length); + key->ck_data, (unsigned int)key->ck_length); printf("\tkey = { "); for (int i = 0; i < key->ck_length / 8; i++) { uint8_t *b = (uint8_t *)key->ck_data; @@ -452,9 +466,6 @@ freebsd_crypt_newsession(freebsd_crypt_session_t *sessp, xauth->name, xauth->keysize); #endif - bzero(&crie, sizeof (crie)); - bzero(&cria, sizeof (cria)); - crie.cri_alg = xform->type; crie.cri_key = key->ck_data; crie.cri_klen = key->ck_length; @@ -466,7 +477,7 @@ freebsd_crypt_newsession(freebsd_crypt_session_t *sessp, cria.cri_next = &crie; crie.cri_next = NULL; crip = &cria; - // Everything else is bzero'd + // Everything else is zero-initialised error = crypto_newsession(&sid, crip, CRYPTOCAP_F_HARDWARE | CRYPTOCAP_F_SOFTWARE); @@ -595,7 +606,7 @@ freebsd_crypt_uio(boolean_t encrypt, enc_desc->crd_inject = auth_len; enc_desc->crd_alg = xform->type; enc_desc->crd_flags = CRD_F_IV_EXPLICIT | CRD_F_IV_PRESENT; - bcopy(ivbuf, enc_desc->crd_iv, ZIO_DATA_IV_LEN); + memcpy(enc_desc->crd_iv, ivbuf, ZIO_DATA_IV_LEN); enc_desc->crd_next = NULL; #ifdef FCRYPTO_DEBUG diff --git a/module/os/freebsd/zfs/dmu_os.c b/module/os/freebsd/zfs/dmu_os.c index 38488dbda6f4..a5f486b95db4 100644 --- a/module/os/freebsd/zfs/dmu_os.c +++ b/module/os/freebsd/zfs/dmu_os.c @@ -119,7 +119,7 @@ dmu_write_pages(objset_t *os, uint64_t object, uint64_t offset, uint64_t size, db->db_offset + bufoff); thiscpy = MIN(PAGESIZE, tocpy - copied); va = zfs_map_page(*ma, &sf); - bcopy(va, (char *)db->db_data + bufoff, thiscpy); + memcpy((char *)db->db_data + bufoff, va, thiscpy); zfs_unmap_page(sf); ma += 1; bufoff += PAGESIZE; @@ -189,7 +189,7 @@ dmu_read_pages(objset_t *os, uint64_t object, vm_page_t *ma, int count, ASSERT3U(db->db_size, >, PAGE_SIZE); bufoff = IDX_TO_OFF(m->pindex) % db->db_size; va = zfs_map_page(m, &sf); - bcopy((char *)db->db_data + bufoff, va, PAGESIZE); + memcpy(va, (char *)db->db_data + bufoff, PAGESIZE); zfs_unmap_page(sf); vm_page_valid(m); dmu_page_lock(m); @@ -231,7 +231,7 @@ dmu_read_pages(objset_t *os, uint64_t object, vm_page_t *ma, int count, tocpy = MIN(db->db_size - bufoff, PAGESIZE - pgoff); ASSERT3S(tocpy, >=, 0); if (m != bogus_page) - bcopy((char *)db->db_data + bufoff, va + pgoff, tocpy); + memcpy(va + pgoff, (char *)db->db_data + bufoff, tocpy); pgoff += tocpy; ASSERT3S(pgoff, >=, 0); @@ -287,7 +287,7 @@ dmu_read_pages(objset_t *os, uint64_t object, vm_page_t *ma, int count, #endif if (pgoff != 0) { ASSERT3P(m, !=, bogus_page); - bzero(va + pgoff, PAGESIZE - pgoff); + memset(va + pgoff, 0, PAGESIZE - pgoff); zfs_unmap_page(sf); vm_page_valid(m); } @@ -309,11 +309,11 @@ dmu_read_pages(objset_t *os, uint64_t object, vm_page_t *ma, int count, bufoff = IDX_TO_OFF(m->pindex) % db->db_size; tocpy = MIN(db->db_size - bufoff, PAGESIZE); va = zfs_map_page(m, &sf); - bcopy((char *)db->db_data + bufoff, va, tocpy); + memcpy(va, (char *)db->db_data + bufoff, tocpy); if (tocpy < PAGESIZE) { ASSERT3S(i, ==, *rahead - 1); ASSERT3U((db->db_size & PAGE_MASK), !=, 0); - bzero(va + tocpy, PAGESIZE - tocpy); + memset(va + tocpy, 0, PAGESIZE - tocpy); } zfs_unmap_page(sf); vm_page_valid(m); diff --git a/module/os/freebsd/zfs/hkdf.c b/module/os/freebsd/zfs/hkdf.c index 8324ff2319b6..3a3c426fa48a 100644 --- a/module/os/freebsd/zfs/hkdf.c +++ b/module/os/freebsd/zfs/hkdf.c @@ -29,7 +29,6 @@ hkdf_sha512_extract(uint8_t *salt, uint_t salt_len, uint8_t *key_material, crypto_key_t key; /* initialize the salt as a crypto key */ - key.ck_format = CRYPTO_KEY_RAW; key.ck_length = CRYPTO_BYTES2BITS(salt_len); key.ck_data = salt; @@ -53,7 +52,6 @@ hkdf_sha512_expand(uint8_t *extract_key, uint8_t *info, uint_t info_len, return (SET_ERROR(EINVAL)); /* initialize the salt as a crypto key */ - key.ck_format = CRYPTO_KEY_RAW; key.ck_length = CRYPTO_BYTES2BITS(SHA512_DIGEST_LENGTH); key.ck_data = extract_key; @@ -65,7 +63,7 @@ hkdf_sha512_expand(uint8_t *extract_key, uint8_t *info, uint_t info_len, crypto_mac_update(&ctx, info, info_len); crypto_mac_update(&ctx, &c, 1); crypto_mac_final(&ctx, T, SHA512_DIGEST_LENGTH); - bcopy(T, out_buf + pos, + memcpy(out_buf + pos, T, (i != N) ? SHA512_DIGEST_LENGTH : (out_len - pos)); pos += SHA512_DIGEST_LENGTH; } diff --git a/module/os/freebsd/zfs/spa_os.c b/module/os/freebsd/zfs/spa_os.c index c8c833426131..251fafcc964e 100644 --- a/module/os/freebsd/zfs/spa_os.c +++ b/module/os/freebsd/zfs/spa_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -268,3 +268,27 @@ spa_history_zone(void) { return ("freebsd"); } + +void +spa_import_os(spa_t *spa) +{ + (void) spa; +} + +void +spa_export_os(spa_t *spa) +{ + (void) spa; +} + +void +spa_activate_os(spa_t *spa) +{ + (void) spa; +} + +void +spa_deactivate_os(spa_t *spa) +{ + (void) spa; +} diff --git a/module/os/freebsd/zfs/vdev_file.c b/module/os/freebsd/zfs/vdev_file.c index ef87d6610977..73cc6aa48c0b 100644 --- a/module/os/freebsd/zfs/vdev_file.c +++ b/module/os/freebsd/zfs/vdev_file.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/freebsd/zfs/vdev_geom.c b/module/os/freebsd/zfs/vdev_geom.c index 914e0e6ded66..f3b4846f4e68 100644 --- a/module/os/freebsd/zfs/vdev_geom.c +++ b/module/os/freebsd/zfs/vdev_geom.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -1131,8 +1131,12 @@ vdev_geom_fill_unmap_cb(void *buf, size_t len, void *priv) vm_offset_t addr = (vm_offset_t)buf; vm_offset_t end = addr + len; - if (bp->bio_ma_n == 0) + if (bp->bio_ma_n == 0) { bp->bio_ma_offset = addr & PAGE_MASK; + addr &= ~PAGE_MASK; + } else { + ASSERT0(P2PHASE(addr, PAGE_SIZE)); + } do { bp->bio_ma[bp->bio_ma_n++] = PHYS_TO_VM_PAGE(pmap_kextract(addr)); diff --git a/module/os/freebsd/zfs/vdev_label_os.c b/module/os/freebsd/zfs/vdev_label_os.c index 48f58807e800..bc856b930235 100644 --- a/module/os/freebsd/zfs/vdev_label_os.c +++ b/module/os/freebsd/zfs/vdev_label_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/freebsd/zfs/zfs_acl.c b/module/os/freebsd/zfs/zfs_acl.c index 838fad63662e..ad482ee9d49b 100644 --- a/module/os/freebsd/zfs/zfs_acl.c +++ b/module/os/freebsd/zfs/zfs_acl.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -144,10 +144,10 @@ zfs_ace_v0_set_who(void *acep, uint64_t who) ((zfs_oldace_t *)acep)->z_fuid = who; } -/*ARGSUSED*/ static size_t zfs_ace_v0_size(void *acep) { + (void) acep; return (sizeof (zfs_oldace_t)); } @@ -163,10 +163,10 @@ zfs_ace_v0_mask_off(void) return (offsetof(zfs_oldace_t, z_access_mask)); } -/*ARGSUSED*/ static int zfs_ace_v0_data(void *acep, void **datap) { + (void) acep; *datap = NULL; return (0); } @@ -631,11 +631,11 @@ zfs_acl_next_ace(zfs_acl_t *aclp, void *start, uint64_t *who, return (NULL); } -/*ARGSUSED*/ static uint64_t zfs_ace_walk(void *datap, uint64_t cookie, int aclcnt, uint16_t *flags, uint16_t *type, uint32_t *mask) { + (void) aclcnt; zfs_acl_t *aclp = datap; zfs_ace_hdr_t *acep = (zfs_ace_hdr_t *)(uintptr_t)cookie; uint64_t who; @@ -689,10 +689,10 @@ zfs_copy_ace_2_fuid(zfsvfs_t *zfsvfs, vtype_t obj_type, zfs_acl_t *aclp, zobjacep = (zfs_object_ace_t *)aceptr; aceobjp = (ace_object_t *)acep; - bcopy(aceobjp->a_obj_type, zobjacep->z_object_type, + memcpy(zobjacep->z_object_type, aceobjp->a_obj_type, sizeof (aceobjp->a_obj_type)); - bcopy(aceobjp->a_inherit_obj_type, - zobjacep->z_inherit_type, + memcpy(zobjacep->z_inherit_type, + aceobjp->a_inherit_obj_type, sizeof (aceobjp->a_inherit_obj_type)); acep = (ace_t *)((caddr_t)acep + sizeof (ace_object_t)); break; @@ -739,11 +739,11 @@ zfs_copy_fuid_2_ace(zfsvfs_t *zfsvfs, zfs_acl_t *aclp, cred_t *cr, } zobjacep = (zfs_object_ace_t *)zacep; objacep = (ace_object_t *)acep; - bcopy(zobjacep->z_object_type, - objacep->a_obj_type, + memcpy(objacep->a_obj_type, + zobjacep->z_object_type, sizeof (zobjacep->z_object_type)); - bcopy(zobjacep->z_inherit_type, - objacep->a_inherit_obj_type, + memcpy(objacep->a_inherit_obj_type, + zobjacep->z_inherit_type, sizeof (zobjacep->z_inherit_type)); ace_size = sizeof (ace_object_t); break; @@ -1094,7 +1094,7 @@ zfs_acl_node_read(znode_t *zp, boolean_t have_lock, zfs_acl_t **aclpp, znode_acl.z_acl_extern_obj, 0, aclnode->z_size, aclnode->z_acldata, DMU_READ_PREFETCH); } else { - bcopy(znode_acl.z_ace_data, aclnode->z_acldata, + memcpy(aclnode->z_acldata, znode_acl.z_ace_data, aclnode->z_size); } } else { @@ -1120,11 +1120,11 @@ zfs_acl_node_read(znode_t *zp, boolean_t have_lock, zfs_acl_t **aclpp, return (error); } -/*ARGSUSED*/ void zfs_acl_data_locator(void **dataptr, uint32_t *length, uint32_t buflen, boolean_t start, void *userdata) { + (void) buflen; zfs_acl_locator_cb_t *cb = (zfs_acl_locator_cb_t *)userdata; if (start) { @@ -1282,7 +1282,7 @@ zfs_aclset_common(znode_t *zp, zfs_acl_t *aclp, cred_t *cr, dmu_tx_t *tx) aclnode = list_next(&aclp->z_acl, aclnode)) { if (aclnode->z_ace_count == 0) continue; - bcopy(aclnode->z_acldata, start, + memcpy(start, aclnode->z_acldata, aclnode->z_size); start = (caddr_t)start + aclnode->z_size; } @@ -1564,7 +1564,7 @@ zfs_acl_inherit(zfsvfs_t *zfsvfs, vtype_t vtype, zfs_acl_t *paclp, if ((data1sz = paclp->z_ops->ace_data(pacep, &data1)) != 0) { data2sz = aclp->z_ops->ace_data(acep, &data2); VERIFY3U(data2sz, ==, data1sz); - bcopy(data1, data2, data2sz); + memcpy(data2, data1, data2sz); } aclp->z_acl_count++; @@ -1633,7 +1633,7 @@ zfs_acl_ids_create(znode_t *dzp, int flag, vattr_t *vap, cred_t *cr, ASSERT_VOP_ELOCKED(ZTOV(dzp), __func__); } else ASSERT3P(dzp->z_vnode, ==, NULL); - bzero(acl_ids, sizeof (zfs_acl_ids_t)); + memset(acl_ids, 0, sizeof (zfs_acl_ids_t)); acl_ids->z_mode = MAKEIMODE(vap->va_type, vap->va_mode); if (vsecp) @@ -1653,8 +1653,10 @@ zfs_acl_ids_create(znode_t *dzp, int flag, vattr_t *vap, cred_t *cr, ZFS_GROUP, &acl_ids->z_fuidp); gid = vap->va_gid; } else { - acl_ids->z_fuid = zfs_fuid_create_cred(zfsvfs, ZFS_OWNER, - cr, &acl_ids->z_fuidp); + uid_t id = crgetuid(cr); + if (IS_EPHEMERAL(id)) + id = UID_NOBODY; + acl_ids->z_fuid = (uint64_t)id; acl_ids->z_fgid = 0; if (vap->va_mask & AT_GID) { acl_ids->z_fgid = zfs_fuid_create(zfsvfs, @@ -1667,7 +1669,7 @@ zfs_acl_ids_create(znode_t *dzp, int flag, vattr_t *vap, cred_t *cr, acl_ids->z_fgid = 0; } if (acl_ids->z_fgid == 0) { - char *domain; + const char *domain; uint32_t rid; acl_ids->z_fgid = dzp->z_gid; @@ -1847,7 +1849,7 @@ zfs_getacl(znode_t *zp, vsecattr_t *vsecp, boolean_t skipaclchk, cred_t *cr) for (aclnode = list_head(&aclp->z_acl); aclnode; aclnode = list_next(&aclp->z_acl, aclnode)) { - bcopy(aclnode->z_acldata, start, + memcpy(start, aclnode->z_acldata, aclnode->z_size); start = (caddr_t)start + aclnode->z_size; } diff --git a/module/os/freebsd/zfs/zfs_ctldir.c b/module/os/freebsd/zfs/zfs_ctldir.c index 10e7bc64bb70..2c35b74cd3fd 100644 --- a/module/os/freebsd/zfs/zfs_ctldir.c +++ b/module/os/freebsd/zfs/zfs_ctldir.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -456,10 +456,10 @@ zfsctl_common_open(struct vop_open_args *ap) /* * Common close routine. Nothing to do here. */ -/* ARGSUSED */ static int zfsctl_common_close(struct vop_close_args *ap) { + (void) ap; return (0); } @@ -721,7 +721,7 @@ zfsctl_root_vptocnp(struct vop_vptocnp_args *ap) VOP_UNLOCK1(dvp); *ap->a_vpp = dvp; *ap->a_buflen -= sizeof (dotzfs_name); - bcopy(dotzfs_name, ap->a_buf + *ap->a_buflen, sizeof (dotzfs_name)); + memcpy(ap->a_buf + *ap->a_buflen, dotzfs_name, sizeof (dotzfs_name)); return (0); } @@ -1214,7 +1214,7 @@ zfsctl_snapshot_vptocnp(struct vop_vptocnp_args *ap) VOP_UNLOCK1(dvp); *ap->a_vpp = dvp; *ap->a_buflen -= len; - bcopy(node->sn_name, ap->a_buf + *ap->a_buflen, len); + memcpy(ap->a_buf + *ap->a_buflen, node->sn_name, len); } vfs_unbusy(mp); #if __FreeBSD_version >= 1300045 diff --git a/module/os/freebsd/zfs/zfs_debug.c b/module/os/freebsd/zfs/zfs_debug.c index 60596aa912f2..32fb5a872190 100644 --- a/module/os/freebsd/zfs/zfs_debug.c +++ b/module/os/freebsd/zfs/zfs_debug.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/freebsd/zfs/zfs_dir.c b/module/os/freebsd/zfs/zfs_dir.c index 7fff329a939c..6321f0b532ae 100644 --- a/module/os/freebsd/zfs/zfs_dir.c +++ b/module/os/freebsd/zfs/zfs_dir.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/freebsd/zfs/zfs_ioctl_compat.c b/module/os/freebsd/zfs/zfs_ioctl_compat.c index 81967bed73f9..d495cc0dc3f1 100644 --- a/module/os/freebsd/zfs/zfs_ioctl_compat.c +++ b/module/os/freebsd/zfs/zfs_ioctl_compat.c @@ -125,7 +125,7 @@ enum zfs_ioc_legacy { ZFS_IOC_LEGACY_LAST }; -unsigned static long zfs_ioctl_legacy_to_ozfs_[] = { +static unsigned long zfs_ioctl_legacy_to_ozfs_[] = { ZFS_IOC_POOL_CREATE, /* 0x00 */ ZFS_IOC_POOL_DESTROY, /* 0x01 */ ZFS_IOC_POOL_IMPORT, /* 0x02 */ @@ -208,7 +208,7 @@ unsigned static long zfs_ioctl_legacy_to_ozfs_[] = { ZFS_IOC_POOL_INITIALIZE, /* 0x4d:0x4f */ }; -unsigned static long zfs_ioctl_ozfs_to_legacy_common_[] = { +static unsigned long zfs_ioctl_ozfs_to_legacy_common_[] = { ZFS_IOC_POOL_CREATE, /* 0x00 */ ZFS_IOC_POOL_DESTROY, /* 0x01 */ ZFS_IOC_POOL_IMPORT, /* 0x02 */ @@ -297,7 +297,7 @@ unsigned static long zfs_ioctl_ozfs_to_legacy_common_[] = { ZFS_IOC_LEGACY_NONE, /* ZFS_IOC_WAIT_FS */ }; -unsigned static long zfs_ioctl_ozfs_to_legacy_platform_[] = { +static unsigned long zfs_ioctl_ozfs_to_legacy_platform_[] = { ZFS_IOC_LEGACY_NONE, /* ZFS_IOC_EVENTS_NEXT */ ZFS_IOC_LEGACY_NONE, /* ZFS_IOC_EVENTS_CLEAR */ ZFS_IOC_LEGACY_NONE, /* ZFS_IOC_EVENTS_SEEK */ diff --git a/module/os/freebsd/zfs/zfs_vfsops.c b/module/os/freebsd/zfs/zfs_vfsops.c index 0dff872b8f87..24e06b1a8809 100644 --- a/module/os/freebsd/zfs/zfs_vfsops.c +++ b/module/os/freebsd/zfs/zfs_vfsops.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -397,7 +397,6 @@ zfs_is_readonly(zfsvfs_t *zfsvfs) return (!!(zfsvfs->z_vfs->vfs_flag & VFS_RDONLY)); } -/*ARGSUSED*/ static int zfs_sync(vfs_t *vfsp, int waitfor) { @@ -1028,8 +1027,6 @@ zfsvfs_setup(zfsvfs_t *zfsvfs, boolean_t mounting) if (error) return (error); - zfsvfs->z_log = zil_open(zfsvfs->z_os, zfs_get_data); - /* * If we are not mounting (ie: online recv), then we don't * have to worry about replaying the log as we blocked all @@ -1039,7 +1036,11 @@ zfsvfs_setup(zfsvfs_t *zfsvfs, boolean_t mounting) boolean_t readonly; ASSERT3P(zfsvfs->z_kstat.dk_kstats, ==, NULL); - dataset_kstats_create(&zfsvfs->z_kstat, zfsvfs->z_os); + error = dataset_kstats_create(&zfsvfs->z_kstat, zfsvfs->z_os); + if (error) + return (error); + zfsvfs->z_log = zil_open(zfsvfs->z_os, zfs_get_data, + &zfsvfs->z_kstat.dk_zil_sums); /* * During replay we remove the read only flag to @@ -1110,6 +1111,10 @@ zfsvfs_setup(zfsvfs_t *zfsvfs, boolean_t mounting) /* restore readonly bit */ if (readonly != 0) zfsvfs->z_vfs->vfs_flag |= VFS_RDONLY; + } else { + ASSERT3P(zfsvfs->z_kstat.dk_kstats, !=, NULL); + zfsvfs->z_log = zil_open(zfsvfs->z_os, zfs_get_data, + &zfsvfs->z_kstat.dk_zil_sums); } /* @@ -1310,7 +1315,6 @@ fetch_osname_options(char *name, bool *checkpointrewind) } } -/*ARGSUSED*/ static int zfs_mount(vfs_t *vfsp) { @@ -1641,7 +1645,6 @@ zfsvfs_teardown(zfsvfs_t *zfsvfs, boolean_t unmounting) return (0); } -/*ARGSUSED*/ static int zfs_umount(vfs_t *vfsp, int fflag) { diff --git a/module/os/freebsd/zfs/zfs_vnops_os.c b/module/os/freebsd/zfs/zfs_vnops_os.c index 55e22cb91c9f..b46cc550c781 100644 --- a/module/os/freebsd/zfs/zfs_vnops_os.c +++ b/module/os/freebsd/zfs/zfs_vnops_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -97,6 +97,10 @@ VFS_SMR_DECLARE; +#if __FreeBSD_version < 1300103 +#define NDFREE_PNBUF(ndp) NDFREE((ndp), NDF_ONLY_PNBUF) +#endif + #if __FreeBSD_version >= 1300047 #define vm_page_wire_lock(pp) #define vm_page_wire_unlock(pp) @@ -220,11 +224,10 @@ typedef ulong_t cookie_t; * ZFS_EXIT(zfsvfs); // finished in zfs * return (error); // done, report error */ - -/* ARGSUSED */ static int zfs_open(vnode_t **vpp, int flag, cred_t *cr) { + (void) cr; znode_t *zp = VTOZ(*vpp); zfsvfs_t *zfsvfs = zp->z_zfsvfs; @@ -238,17 +241,17 @@ zfs_open(vnode_t **vpp, int flag, cred_t *cr) } /* Keep a count of the synchronous opens in the znode */ - if (flag & (FSYNC | FDSYNC)) + if (flag & O_SYNC) atomic_inc_32(&zp->z_sync_cnt); ZFS_EXIT(zfsvfs); return (0); } -/* ARGSUSED */ static int zfs_close(vnode_t *vp, int flag, int count, offset_t offset, cred_t *cr) { + (void) offset, (void) cr; znode_t *zp = VTOZ(vp); zfsvfs_t *zfsvfs = zp->z_zfsvfs; @@ -256,18 +259,18 @@ zfs_close(vnode_t *vp, int flag, int count, offset_t offset, cred_t *cr) ZFS_VERIFY_ZP(zp); /* Decrement the synchronous opens in the znode */ - if ((flag & (FSYNC | FDSYNC)) && (count == 1)) + if ((flag & O_SYNC) && (count == 1)) atomic_dec_32(&zp->z_sync_cnt); ZFS_EXIT(zfsvfs); return (0); } -/* ARGSUSED */ static int zfs_ioctl(vnode_t *vp, ulong_t com, intptr_t data, int flag, cred_t *cred, int *rvalp) { + (void) flag, (void) cred, (void) rvalp; loff_t off; int error; @@ -540,7 +543,7 @@ mappedread_sf(znode_t *zp, int nbytes, zfs_uio_t *uio) error = dmu_read(os, zp->z_id, start, bytes, va, DMU_READ_PREFETCH); if (bytes != PAGESIZE && error == 0) - bzero(va + bytes, PAGESIZE - bytes); + memset(va + bytes, 0, PAGESIZE - bytes); zfs_unmap_page(sf); zfs_vmobject_wlock_12(obj); #if __FreeBSD_version >= 1300081 @@ -764,7 +767,6 @@ zfs_lookup_lock(vnode_t *dvp, vnode_t *vp, const char *name, int lkflags) * Timestamps: * NA */ -/* ARGSUSED */ static int zfs_lookup(vnode_t *dvp, const char *nm, vnode_t **vpp, struct componentname *cnp, int nameiop, cred_t *cr, int flags, @@ -1034,20 +1036,18 @@ zfs_lookup(vnode_t *dvp, const char *nm, vnode_t **vpp, * dvp - ctime|mtime updated if new entry created * vp - ctime|mtime always, atime if new */ - -/* ARGSUSED */ int zfs_create(znode_t *dzp, const char *name, vattr_t *vap, int excl, int mode, znode_t **zpp, cred_t *cr, int flag, vsecattr_t *vsecp) { + (void) excl, (void) mode, (void) flag; znode_t *zp; zfsvfs_t *zfsvfs = dzp->z_zfsvfs; zilog_t *zilog; objset_t *os; dmu_tx_t *tx; int error; - ksid_t *ksid; - uid_t uid; + uid_t uid = crgetuid(cr); gid_t gid = crgetgid(cr); uint64_t projid = ZFS_DEFAULT_PROJID; zfs_acl_ids_t acl_ids; @@ -1061,13 +1061,6 @@ zfs_create(znode_t *dzp, const char *name, vattr_t *vap, int excl, int mode, * If we have an ephemeral id, ACL, or XVATTR then * make sure file system is at proper version */ - - ksid = crgetsid(cr, KSID_OWNER); - if (ksid) - uid = ksid_getid(ksid); - else - uid = crgetuid(cr); - if (zfsvfs->z_use_fuids == B_FALSE && (vsecp || (vap->va_mask & AT_XVATTR) || IS_EPHEMERAL(uid) || IS_EPHEMERAL(gid))) @@ -1201,8 +1194,6 @@ zfs_create(znode_t *dzp, const char *name, vattr_t *vap, int excl, int mode, * dvp - ctime|mtime * vp - ctime (if nlink > 0) */ - -/*ARGSUSED*/ static int zfs_remove_(vnode_t *dvp, vnode_t *vp, const char *name, cred_t *cr) { @@ -1390,19 +1381,18 @@ zfs_remove(znode_t *dzp, const char *name, cred_t *cr, int flags) * dvp - ctime|mtime updated * vp - ctime|mtime|atime updated */ -/*ARGSUSED*/ int zfs_mkdir(znode_t *dzp, const char *dirname, vattr_t *vap, znode_t **zpp, cred_t *cr, int flags, vsecattr_t *vsecp) { + (void) flags, (void) vsecp; znode_t *zp; zfsvfs_t *zfsvfs = dzp->z_zfsvfs; zilog_t *zilog; uint64_t txtype; dmu_tx_t *tx; int error; - ksid_t *ksid; - uid_t uid; + uid_t uid = crgetuid(cr); gid_t gid = crgetgid(cr); zfs_acl_ids_t acl_ids; boolean_t fuid_dirtied; @@ -1413,12 +1403,6 @@ zfs_mkdir(znode_t *dzp, const char *dirname, vattr_t *vap, znode_t **zpp, * If we have an ephemeral id, ACL, or XVATTR then * make sure file system is at proper version */ - - ksid = crgetsid(cr, KSID_OWNER); - if (ksid) - uid = ksid_getid(ksid); - else - uid = crgetuid(cr); if (zfsvfs->z_use_fuids == B_FALSE && ((vap->va_mask & AT_XVATTR) || IS_EPHEMERAL(uid) || IS_EPHEMERAL(gid))) @@ -1567,7 +1551,6 @@ cache_vop_rmdir(struct vnode *dvp, struct vnode *vp) * Timestamps: * dvp - ctime|mtime updated */ -/*ARGSUSED*/ static int zfs_rmdir_(vnode_t *dvp, vnode_t *vp, const char *name, cred_t *cr) { @@ -1668,7 +1651,6 @@ zfs_rmdir(znode_t *dzp, const char *name, znode_t *cwd, cred_t *cr, int flags) * We use 0 for '.', and 1 for '..'. If this is the root of the filesystem, * we use the offset 2 for the '.zfs' directory. */ -/* ARGSUSED */ static int zfs_readdir(vnode_t *vp, zfs_uio_t *uio, cred_t *cr, int *eofp, int *ncookies, cookie_t **cookies) @@ -1994,7 +1976,6 @@ zfs_readdir(vnode_t *vp, zfs_uio_t *uio, cred_t *cr, int *eofp, * * RETURN: 0 (always succeeds). */ -/* ARGSUSED */ static int zfs_getattr(vnode_t *vp, vattr_t *vap, int flags, cred_t *cr) { @@ -2208,7 +2189,6 @@ zfs_getattr(vnode_t *vp, vattr_t *vap, int flags, cred_t *cr) * Timestamps: * vp - ctime updated, mtime updated if size changed. */ -/* ARGSUSED */ int zfs_setattr(znode_t *zp, vattr_t *vap, int flags, cred_t *cr) { @@ -3191,7 +3171,6 @@ zfs_do_rename_impl(vnode_t *sdvp, vnode_t **svpp, struct componentname *scnp, * Timestamps: * sdvp,tdvp - ctime|mtime updated */ -/*ARGSUSED*/ static int zfs_do_rename(vnode_t *sdvp, vnode_t **svpp, struct componentname *scnp, vnode_t *tdvp, vnode_t **tvpp, struct componentname *tcnp, @@ -3514,11 +3493,11 @@ zfs_rename(znode_t *sdzp, const char *sname, znode_t *tdzp, const char *tname, * Timestamps: * dvp - ctime|mtime updated */ -/*ARGSUSED*/ int zfs_symlink(znode_t *dzp, const char *name, vattr_t *vap, const char *link, znode_t **zpp, cred_t *cr, int flags) { + (void) flags; znode_t *zp; dmu_tx_t *tx; zfsvfs_t *zfsvfs = dzp->z_zfsvfs; @@ -3653,10 +3632,10 @@ zfs_symlink(znode_t *dzp, const char *name, vattr_t *vap, * Timestamps: * vp - atime updated */ -/* ARGSUSED */ static int zfs_readlink(vnode_t *vp, zfs_uio_t *uio, cred_t *cr, caller_context_t *ct) { + (void) cr, (void) ct; znode_t *zp = VTOZ(vp); zfsvfs_t *zfsvfs = zp->z_zfsvfs; int error; @@ -3690,11 +3669,11 @@ zfs_readlink(vnode_t *vp, zfs_uio_t *uio, cred_t *cr, caller_context_t *ct) * tdvp - ctime|mtime updated * svp - ctime updated */ -/* ARGSUSED */ int zfs_link(znode_t *tdzp, znode_t *szp, const char *name, cred_t *cr, int flags) { + (void) flags; znode_t *tzp; zfsvfs_t *zfsvfs = tdzp->z_zfsvfs; zilog_t *zilog; @@ -3839,11 +3818,11 @@ zfs_link(znode_t *tdzp, znode_t *szp, const char *name, cred_t *cr, * Timestamps: * ip - ctime|mtime updated */ -/* ARGSUSED */ int zfs_space(znode_t *zp, int cmd, flock64_t *bfp, int flag, offset_t offset, cred_t *cr) { + (void) offset; zfsvfs_t *zfsvfs = ZTOZSB(zp); uint64_t off, len; int error; @@ -3890,10 +3869,10 @@ zfs_space(znode_t *zp, int cmd, flock64_t *bfp, int flag, return (error); } -/*ARGSUSED*/ static void zfs_inactive(vnode_t *vp, cred_t *cr, caller_context_t *ct) { + (void) cr, (void) ct; znode_t *zp = VTOZ(vp); zfsvfs_t *zfsvfs = zp->z_zfsvfs; int error; @@ -3942,10 +3921,10 @@ _Static_assert(sizeof (struct zfid_short) <= sizeof (struct fid), _Static_assert(sizeof (struct zfid_long) <= sizeof (struct fid), "struct zfid_long bigger than struct fid"); -/*ARGSUSED*/ static int zfs_fid(vnode_t *vp, fid_t *fidp, caller_context_t *ct) { + (void) ct; znode_t *zp = VTOZ(vp); zfsvfs_t *zfsvfs = zp->z_zfsvfs; uint32_t gen; @@ -4061,8 +4040,8 @@ zfs_getpages(struct vnode *vp, vm_page_t *ma, int count, int *rbehind, int pgsin_b, pgsin_a; int error; - ZFS_ENTER(zfsvfs); - ZFS_VERIFY_ZP(zp); + ZFS_ENTER_ERROR(zfsvfs, zfs_vm_pagerret_error); + ZFS_VERIFY_ZP_ERROR(zp, zfs_vm_pagerret_error); start = IDX_TO_OFF(ma[0]->pindex); end = IDX_TO_OFF(ma[count - 1]->pindex + 1); @@ -4186,19 +4165,18 @@ zfs_putpages(struct vnode *vp, vm_page_t *ma, size_t len, int flags, int err; int i; - ZFS_ENTER(zfsvfs); - ZFS_VERIFY_ZP(zp); - object = vp->v_object; - pcount = btoc(len); - ncount = pcount; - KASSERT(ma[0]->object == object, ("mismatching object")); KASSERT(len > 0 && (len & PAGE_MASK) == 0, ("unexpected length")); + pcount = btoc(len); + ncount = pcount; for (i = 0; i < pcount; i++) rtvals[i] = zfs_vm_pagerret_error; + ZFS_ENTER_ERROR(zfsvfs, zfs_vm_pagerret_error); + ZFS_VERIFY_ZP_ERROR(zp, zfs_vm_pagerret_error); + off = IDX_TO_OFF(ma[0]->pindex); blksz = zp->z_blksz; lo_off = rounddown(off, blksz); @@ -4424,11 +4402,11 @@ ioflags(int ioflags) int flags = 0; if (ioflags & IO_APPEND) - flags |= FAPPEND; + flags |= O_APPEND; if (ioflags & IO_NDELAY) - flags |= FNONBLOCK; + flags |= O_NONBLOCK; if (ioflags & IO_SYNC) - flags |= (FSYNC | FDSYNC | FRSYNC); + flags |= O_SYNC; return (flags); } @@ -4649,7 +4627,7 @@ zfs_freebsd_create(struct vop_create_args *ap) zfsvfs = ap->a_dvp->v_mount->mnt_data; *ap->a_vpp = NULL; - rc = zfs_create(VTOZ(ap->a_dvp), cnp->cn_nameptr, vap, !EXCL, mode, + rc = zfs_create(VTOZ(ap->a_dvp), cnp->cn_nameptr, vap, 0, mode, &zp, cnp->cn_cred, 0 /* flag */, NULL /* vsecattr */); if (rc == 0) *ap->a_vpp = ZTOV(zp); @@ -5253,43 +5231,56 @@ zfs_freebsd_pathconf(struct vop_pathconf_args *ap) } } +static int zfs_xattr_compat = 1; + +static int +zfs_check_attrname(const char *name) +{ + /* We don't allow '/' character in attribute name. */ + if (strchr(name, '/') != NULL) + return (SET_ERROR(EINVAL)); + /* We don't allow attribute names that start with a namespace prefix. */ + if (ZFS_XA_NS_PREFIX_FORBIDDEN(name)) + return (SET_ERROR(EINVAL)); + return (0); +} + /* * FreeBSD's extended attributes namespace defines file name prefix for ZFS' * extended attribute name: * - * NAMESPACE PREFIX - * system freebsd:system: - * user (none, can be used to access ZFS fsattr(5) attributes - * created on Solaris) + * NAMESPACE XATTR_COMPAT PREFIX + * system * freebsd:system: + * user 1 (none, can be used to access ZFS + * fsattr(5) attributes created on Solaris) + * user 0 user. */ static int zfs_create_attrname(int attrnamespace, const char *name, char *attrname, - size_t size) + size_t size, boolean_t compat) { const char *namespace, *prefix, *suffix; - /* We don't allow '/' character in attribute name. */ - if (strchr(name, '/') != NULL) - return (SET_ERROR(EINVAL)); - /* We don't allow attribute names that start with "freebsd:" string. */ - if (strncmp(name, "freebsd:", 8) == 0) - return (SET_ERROR(EINVAL)); - - bzero(attrname, size); + memset(attrname, 0, size); switch (attrnamespace) { case EXTATTR_NAMESPACE_USER: -#if 0 - prefix = "freebsd:"; - namespace = EXTATTR_NAMESPACE_USER_STRING; - suffix = ":"; -#else - /* - * This is the default namespace by which we can access all - * attributes created on Solaris. - */ - prefix = namespace = suffix = ""; -#endif + if (compat) { + /* + * This is the default namespace by which we can access + * all attributes created on Solaris. + */ + prefix = namespace = suffix = ""; + } else { + /* + * This is compatible with the user namespace encoding + * on Linux prior to xattr_compat, but nothing + * else. + */ + prefix = ""; + namespace = "user"; + suffix = "."; + } break; case EXTATTR_NAMESPACE_SYSTEM: prefix = "freebsd:"; @@ -5365,7 +5356,7 @@ zfs_getextattr_dir(struct vop_getextattr_args *ap, const char *attrname) #endif error = vn_open_cred(&nd, &flags, 0, VN_OPEN_INVFS, ap->a_cred, NULL); vp = nd.ni_vp; - NDFREE(&nd, NDF_ONLY_PNBUF); + NDFREE_PNBUF(&nd); if (error != 0) return (SET_ERROR(error)); @@ -5411,6 +5402,27 @@ zfs_getextattr_sa(struct vop_getextattr_args *ap, const char *attrname) return (0); } +static int +zfs_getextattr_impl(struct vop_getextattr_args *ap, boolean_t compat) +{ + znode_t *zp = VTOZ(ap->a_vp); + zfsvfs_t *zfsvfs = ZTOZSB(zp); + char attrname[EXTATTR_MAXNAMELEN+1]; + int error; + + error = zfs_create_attrname(ap->a_attrnamespace, ap->a_name, attrname, + sizeof (attrname), compat); + if (error != 0) + return (error); + + error = ENOENT; + if (zfsvfs->z_use_sa && zp->z_is_sa) + error = zfs_getextattr_sa(ap, attrname); + if (error == ENOENT) + error = zfs_getextattr_dir(ap, attrname); + return (error); +} + /* * Vnode operation to retrieve a named extended attribute. */ @@ -5419,7 +5431,6 @@ zfs_getextattr(struct vop_getextattr_args *ap) { znode_t *zp = VTOZ(ap->a_vp); zfsvfs_t *zfsvfs = ZTOZSB(zp); - char attrname[EXTATTR_MAXNAMELEN+1]; int error; /* @@ -5433,19 +5444,25 @@ zfs_getextattr(struct vop_getextattr_args *ap) if (error != 0) return (SET_ERROR(error)); - error = zfs_create_attrname(ap->a_attrnamespace, ap->a_name, attrname, - sizeof (attrname)); + error = zfs_check_attrname(ap->a_name); if (error != 0) return (error); error = ENOENT; ZFS_ENTER(zfsvfs); - ZFS_VERIFY_ZP(zp) + ZFS_VERIFY_ZP(zp); rw_enter(&zp->z_xattr_lock, RW_READER); - if (zfsvfs->z_use_sa && zp->z_is_sa) - error = zfs_getextattr_sa(ap, attrname); - if (error == ENOENT) - error = zfs_getextattr_dir(ap, attrname); + + error = zfs_getextattr_impl(ap, zfs_xattr_compat); + if ((error == ENOENT || error == ENOATTR) && + ap->a_attrnamespace == EXTATTR_NAMESPACE_USER) { + /* + * Fall back to the alternate namespace format if we failed to + * find a user xattr. + */ + error = zfs_getextattr_impl(ap, !zfs_xattr_compat); + } + rw_exit(&zp->z_xattr_lock); ZFS_EXIT(zfsvfs); if (error == ENOENT) @@ -5485,12 +5502,12 @@ zfs_deleteextattr_dir(struct vop_deleteextattr_args *ap, const char *attrname) error = namei(&nd); vp = nd.ni_vp; if (error != 0) { - NDFREE(&nd, NDF_ONLY_PNBUF); + NDFREE_PNBUF(&nd); return (SET_ERROR(error)); } error = VOP_REMOVE(nd.ni_dvp, vp, &nd.ni_cnd); - NDFREE(&nd, NDF_ONLY_PNBUF); + NDFREE_PNBUF(&nd); vput(nd.ni_dvp); if (vp == nd.ni_dvp) @@ -5520,7 +5537,7 @@ zfs_deleteextattr_sa(struct vop_deleteextattr_args *ap, const char *attrname) if (error != 0) error = SET_ERROR(error); else - error = zfs_sa_set_xattr(zp); + error = zfs_sa_set_xattr(zp, attrname, NULL, 0); if (error != 0) { zp->z_xattr_cached = NULL; nvlist_free(nvl); @@ -5528,6 +5545,27 @@ zfs_deleteextattr_sa(struct vop_deleteextattr_args *ap, const char *attrname) return (error); } +static int +zfs_deleteextattr_impl(struct vop_deleteextattr_args *ap, boolean_t compat) +{ + znode_t *zp = VTOZ(ap->a_vp); + zfsvfs_t *zfsvfs = ZTOZSB(zp); + char attrname[EXTATTR_MAXNAMELEN+1]; + int error; + + error = zfs_create_attrname(ap->a_attrnamespace, ap->a_name, attrname, + sizeof (attrname), compat); + if (error != 0) + return (error); + + error = ENOENT; + if (zfsvfs->z_use_sa && zp->z_is_sa) + error = zfs_deleteextattr_sa(ap, attrname); + if (error == ENOENT) + error = zfs_deleteextattr_dir(ap, attrname); + return (error); +} + /* * Vnode operation to remove a named attribute. */ @@ -5536,7 +5574,6 @@ zfs_deleteextattr(struct vop_deleteextattr_args *ap) { znode_t *zp = VTOZ(ap->a_vp); zfsvfs_t *zfsvfs = ZTOZSB(zp); - char attrname[EXTATTR_MAXNAMELEN+1]; int error; /* @@ -5550,32 +5587,24 @@ zfs_deleteextattr(struct vop_deleteextattr_args *ap) if (error != 0) return (SET_ERROR(error)); - error = zfs_create_attrname(ap->a_attrnamespace, ap->a_name, attrname, - sizeof (attrname)); + error = zfs_check_attrname(ap->a_name); if (error != 0) return (error); - size_t size = 0; - struct vop_getextattr_args vga = { - .a_vp = ap->a_vp, - .a_size = &size, - .a_cred = ap->a_cred, - .a_td = ap->a_td, - }; - error = ENOENT; ZFS_ENTER(zfsvfs); ZFS_VERIFY_ZP(zp); rw_enter(&zp->z_xattr_lock, RW_WRITER); - if (zfsvfs->z_use_sa && zp->z_is_sa) { - error = zfs_getextattr_sa(&vga, attrname); - if (error == 0) - error = zfs_deleteextattr_sa(ap, attrname); - } - if (error == ENOENT) { - error = zfs_getextattr_dir(&vga, attrname); - if (error == 0) - error = zfs_deleteextattr_dir(ap, attrname); + + error = zfs_deleteextattr_impl(ap, zfs_xattr_compat); + if ((error == ENOENT || error == ENOATTR) && + ap->a_attrnamespace == EXTATTR_NAMESPACE_USER) { + /* + * Fall back to the alternate namespace format if we failed to + * find a user xattr. + */ + error = zfs_deleteextattr_impl(ap, !zfs_xattr_compat); } + rw_exit(&zp->z_xattr_lock); ZFS_EXIT(zfsvfs); if (error == ENOENT) @@ -5617,7 +5646,7 @@ zfs_setextattr_dir(struct vop_setextattr_args *ap, const char *attrname) error = vn_open_cred(&nd, &flags, 0600, VN_OPEN_INVFS, ap->a_cred, NULL); vp = nd.ni_vp; - NDFREE(&nd, NDF_ONLY_PNBUF); + NDFREE_PNBUF(&nd); if (error != 0) return (SET_ERROR(error)); @@ -5665,9 +5694,9 @@ zfs_setextattr_sa(struct vop_setextattr_args *ap, const char *attrname) if (error != 0) error = SET_ERROR(error); } - kmem_free(buf, entry_size); if (error == 0) - error = zfs_sa_set_xattr(zp); + error = zfs_sa_set_xattr(zp, attrname, buf, entry_size); + kmem_free(buf, entry_size); if (error != 0) { zp->z_xattr_cached = NULL; nvlist_free(nvl); @@ -5675,60 +5704,87 @@ zfs_setextattr_sa(struct vop_setextattr_args *ap, const char *attrname) return (error); } -/* - * Vnode operation to set a named attribute. - */ static int -zfs_setextattr(struct vop_setextattr_args *ap) +zfs_setextattr_impl(struct vop_setextattr_args *ap, boolean_t compat) { znode_t *zp = VTOZ(ap->a_vp); zfsvfs_t *zfsvfs = ZTOZSB(zp); char attrname[EXTATTR_MAXNAMELEN+1]; int error; - /* - * If the xattr property is off, refuse the request. - */ - if (!(zfsvfs->z_flags & ZSB_XATTR)) - return (SET_ERROR(EOPNOTSUPP)); - - error = extattr_check_cred(ap->a_vp, ap->a_attrnamespace, - ap->a_cred, ap->a_td, VWRITE); - if (error != 0) - return (SET_ERROR(error)); - error = zfs_create_attrname(ap->a_attrnamespace, ap->a_name, attrname, - sizeof (attrname)); + sizeof (attrname), compat); if (error != 0) return (error); struct vop_deleteextattr_args vda = { .a_vp = ap->a_vp, + .a_attrnamespace = ap->a_attrnamespace, + .a_name = ap->a_name, .a_cred = ap->a_cred, .a_td = ap->a_td, }; error = ENOENT; - ZFS_ENTER(zfsvfs); - ZFS_VERIFY_ZP(zp); - rw_enter(&zp->z_xattr_lock, RW_WRITER); if (zfsvfs->z_use_sa && zp->z_is_sa && zfsvfs->z_xattr_sa) { error = zfs_setextattr_sa(ap, attrname); - if (error == 0) + if (error == 0) { /* * Successfully put into SA, we need to clear the one * in dir if present. */ zfs_deleteextattr_dir(&vda, attrname); + } } - if (error) { + if (error != 0) { error = zfs_setextattr_dir(ap, attrname); - if (error == 0 && zp->z_is_sa) + if (error == 0 && zp->z_is_sa) { /* * Successfully put into dir, we need to clear the one * in SA if present. */ zfs_deleteextattr_sa(&vda, attrname); + } + } + if (error == 0 && ap->a_attrnamespace == EXTATTR_NAMESPACE_USER) { + /* + * Also clear all versions of the alternate compat name. + */ + zfs_deleteextattr_impl(&vda, !compat); } + return (error); +} + +/* + * Vnode operation to set a named attribute. + */ +static int +zfs_setextattr(struct vop_setextattr_args *ap) +{ + znode_t *zp = VTOZ(ap->a_vp); + zfsvfs_t *zfsvfs = ZTOZSB(zp); + int error; + + /* + * If the xattr property is off, refuse the request. + */ + if (!(zfsvfs->z_flags & ZSB_XATTR)) + return (SET_ERROR(EOPNOTSUPP)); + + error = extattr_check_cred(ap->a_vp, ap->a_attrnamespace, + ap->a_cred, ap->a_td, VWRITE); + if (error != 0) + return (SET_ERROR(error)); + + error = zfs_check_attrname(ap->a_name); + if (error != 0) + return (error); + + ZFS_ENTER(zfsvfs); + ZFS_VERIFY_ZP(zp); + rw_enter(&zp->z_xattr_lock, RW_WRITER); + + error = zfs_setextattr_impl(ap, zfs_xattr_compat); + rw_exit(&zp->z_xattr_lock); ZFS_EXIT(zfsvfs); return (error); @@ -5777,7 +5833,7 @@ zfs_listextattr_dir(struct vop_listextattr_args *ap, const char *attrprefix) #endif error = namei(&nd); vp = nd.ni_vp; - NDFREE(&nd, NDF_ONLY_PNBUF); + NDFREE_PNBUF(&nd); if (error != 0) return (SET_ERROR(error)); @@ -5808,7 +5864,7 @@ zfs_listextattr_dir(struct vop_listextattr_args *ap, const char *attrprefix) if (dp->d_type != DT_REG && dp->d_type != DT_UNKNOWN) continue; else if (plen == 0 && - strncmp(dp->d_name, "freebsd:", 8) == 0) + ZFS_XA_NS_PREFIX_FORBIDDEN(dp->d_name)) continue; else if (strncmp(dp->d_name, attrprefix, plen) != 0) continue; @@ -5856,7 +5912,7 @@ zfs_listextattr_sa(struct vop_listextattr_args *ap, const char *attrprefix) ASSERT3U(nvpair_type(nvp), ==, DATA_TYPE_BYTE_ARRAY); const char *name = nvpair_name(nvp); - if (plen == 0 && strncmp(name, "freebsd:", 8) == 0) + if (plen == 0 && ZFS_XA_NS_PREFIX_FORBIDDEN(name)) continue; else if (strncmp(name, attrprefix, plen) != 0) continue; @@ -5883,6 +5939,26 @@ zfs_listextattr_sa(struct vop_listextattr_args *ap, const char *attrprefix) return (error); } +static int +zfs_listextattr_impl(struct vop_listextattr_args *ap, boolean_t compat) +{ + znode_t *zp = VTOZ(ap->a_vp); + zfsvfs_t *zfsvfs = ZTOZSB(zp); + char attrprefix[16]; + int error; + + error = zfs_create_attrname(ap->a_attrnamespace, "", attrprefix, + sizeof (attrprefix), compat); + if (error != 0) + return (error); + + if (zfsvfs->z_use_sa && zp->z_is_sa) + error = zfs_listextattr_sa(ap, attrprefix); + if (error == 0) + error = zfs_listextattr_dir(ap, attrprefix); + return (error); +} + /* * Vnode operation to retrieve extended attributes on a vnode. */ @@ -5891,7 +5967,6 @@ zfs_listextattr(struct vop_listextattr_args *ap) { znode_t *zp = VTOZ(ap->a_vp); zfsvfs_t *zfsvfs = ZTOZSB(zp); - char attrprefix[16]; int error; if (ap->a_size != NULL) @@ -5908,18 +5983,16 @@ zfs_listextattr(struct vop_listextattr_args *ap) if (error != 0) return (SET_ERROR(error)); - error = zfs_create_attrname(ap->a_attrnamespace, "", attrprefix, - sizeof (attrprefix)); - if (error != 0) - return (error); - ZFS_ENTER(zfsvfs); ZFS_VERIFY_ZP(zp); rw_enter(&zp->z_xattr_lock, RW_READER); - if (zfsvfs->z_use_sa && zp->z_is_sa) - error = zfs_listextattr_sa(ap, attrprefix); - if (error == 0) - error = zfs_listextattr_dir(ap, attrprefix); + + error = zfs_listextattr_impl(ap, zfs_xattr_compat); + if (error == 0 && ap->a_attrnamespace == EXTATTR_NAMESPACE_USER) { + /* Also list user xattrs with the alternate format. */ + error = zfs_listextattr_impl(ap, !zfs_xattr_compat); + } + rw_exit(&zp->z_xattr_lock); ZFS_EXIT(zfsvfs); return (error); @@ -6057,7 +6130,7 @@ zfs_vptocnp(struct vop_vptocnp_args *ap) } if (error == 0) { *ap->a_buflen -= len; - bcopy(name, ap->a_buf + *ap->a_buflen, len); + memcpy(ap->a_buf + *ap->a_buflen, name, len); *ap->a_vpp = ZTOV(dzp); } ZFS_EXIT(zfsvfs); @@ -6257,3 +6330,6 @@ struct vop_vector zfs_shareops = { #endif }; VFS_VOP_VECTOR_REGISTER(zfs_shareops); + +ZFS_MODULE_PARAM(zfs, zfs_, xattr_compat, INT, ZMOD_RW, + "Use legacy ZFS xattr naming for writing new user namespace xattrs"); diff --git a/module/os/freebsd/zfs/zfs_znode.c b/module/os/freebsd/zfs/zfs_znode.c index 2ffa403d028d..6345e9e69d30 100644 --- a/module/os/freebsd/zfs/zfs_znode.c +++ b/module/os/freebsd/zfs/zfs_znode.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -153,13 +153,16 @@ zfs_znode_cache_constructor(void *buf, void *arg, int kmflags) zp->z_xattr_cached = NULL; zp->z_xattr_parent = 0; zp->z_vnode = NULL; + zp->z_sync_writes_cnt = 0; + zp->z_async_writes_cnt = 0; + return (0); } -/*ARGSUSED*/ static void zfs_znode_cache_destructor(void *buf, void *arg) { + (void) arg; znode_t *zp = buf; ASSERT(!POINTER_IS_VALID(zp->z_zfsvfs)); @@ -172,6 +175,9 @@ zfs_znode_cache_destructor(void *buf, void *arg) ASSERT3P(zp->z_acl_cached, ==, NULL); ASSERT3P(zp->z_xattr_cached, ==, NULL); + + ASSERT0(atomic_load_32(&zp->z_sync_writes_cnt)); + ASSERT0(atomic_load_32(&zp->z_async_writes_cnt)); } @@ -453,6 +459,8 @@ zfs_znode_alloc(zfsvfs_t *zfsvfs, dmu_buf_t *db, int blksz, zp->z_blksz = blksz; zp->z_seq = 0x7A4653; zp->z_sync_cnt = 0; + zp->z_sync_writes_cnt = 0; + zp->z_async_writes_cnt = 0; #if __FreeBSD_version >= 1300139 atomic_store_ptr(&zp->z_cached_symlink, NULL); #endif @@ -833,7 +841,9 @@ zfs_xvattr_set(znode_t *zp, xvattr_t *xvap, dmu_tx_t *tx) xoap = xva_getxoptattr(xvap); ASSERT3P(xoap, !=, NULL); - ASSERT_VOP_IN_SEQC(ZTOV(zp)); + if (zp->z_zfsvfs->z_replay == B_FALSE) { + ASSERT_VOP_IN_SEQC(ZTOV(zp)); + } if (XVA_ISSET_REQ(xvap, XAT_CREATETIME)) { uint64_t times[2]; @@ -1804,7 +1814,7 @@ zfs_sa_setup(objset_t *osp, sa_attr_type_t **sa_table) static int zfs_grab_sa_handle(objset_t *osp, uint64_t obj, sa_handle_t **hdlp, - dmu_buf_t **db, void *tag) + dmu_buf_t **db, const void *tag) { dmu_object_info_t doi; int error; @@ -1831,7 +1841,7 @@ zfs_grab_sa_handle(objset_t *osp, uint64_t obj, sa_handle_t **hdlp, } static void -zfs_release_sa_handle(sa_handle_t *hdl, dmu_buf_t *db, void *tag) +zfs_release_sa_handle(sa_handle_t *hdl, dmu_buf_t *db, const void *tag) { sa_handle_destroy(hdl); sa_buf_rele(db, tag); @@ -1975,7 +1985,7 @@ zfs_obj_to_path_impl(objset_t *osp, uint64_t obj, sa_handle_t *hdl, complen = strlen(component); path -= complen; ASSERT3P(path, >=, buf); - bcopy(component, path, complen); + memcpy(path, component, complen); obj = pobj; if (sa_hdl != hdl) { diff --git a/module/os/freebsd/zfs/zio_crypt.c b/module/os/freebsd/zfs/zio_crypt.c index fbde8063a280..0410ddd65a5c 100644 --- a/module/os/freebsd/zfs/zio_crypt.c +++ b/module/os/freebsd/zfs/zio_crypt.c @@ -211,10 +211,10 @@ zio_crypt_key_destroy_early(zio_crypt_key_t *key) rw_destroy(&key->zk_salt_lock); /* free crypto templates */ - bzero(&key->zk_session, sizeof (key->zk_session)); + memset(&key->zk_session, 0, sizeof (key->zk_session)); /* zero out sensitive data */ - bzero(key, sizeof (zio_crypt_key_t)); + memset(key, 0, sizeof (zio_crypt_key_t)); } void @@ -242,7 +242,7 @@ zio_crypt_key_init(uint64_t crypt, zio_crypt_key_t *key) return (ENOTSUP); keydata_len = zio_crypt_table[crypt].ci_keylen; - bzero(key, sizeof (zio_crypt_key_t)); + memset(key, 0, sizeof (zio_crypt_key_t)); rw_init(&key->zk_salt_lock, NULL, RW_DEFAULT, NULL); /* fill keydata buffers and salt with random data */ @@ -270,11 +270,9 @@ zio_crypt_key_init(uint64_t crypt, zio_crypt_key_t *key) goto error; /* initialize keys for the ICP */ - key->zk_current_key.ck_format = CRYPTO_KEY_RAW; key->zk_current_key.ck_data = key->zk_current_keydata; key->zk_current_key.ck_length = CRYPTO_BYTES2BITS(keydata_len); - key->zk_hmac_key.ck_format = CRYPTO_KEY_RAW; key->zk_hmac_key.ck_data = &key->zk_hmac_key; key->zk_hmac_key.ck_length = CRYPTO_BYTES2BITS(SHA512_HMAC_KEYLEN); @@ -326,7 +324,7 @@ zio_crypt_key_change_salt(zio_crypt_key_t *key) goto out_unlock; /* assign the salt and reset the usage count */ - bcopy(salt, key->zk_salt, ZIO_DATA_SALT_LEN); + memcpy(key->zk_salt, salt, ZIO_DATA_SALT_LEN); key->zk_salt_count = 0; freebsd_crypt_freesession(&key->zk_session); @@ -354,7 +352,7 @@ zio_crypt_key_get_salt(zio_crypt_key_t *key, uint8_t *salt) rw_enter(&key->zk_salt_lock, RW_READER); - bcopy(key->zk_salt, salt, ZIO_DATA_SALT_LEN); + memcpy(salt, key->zk_salt, ZIO_DATA_SALT_LEN); salt_change = (atomic_inc_64_nv(&key->zk_salt_count) >= ZFS_CURRENT_MAX_SALT_USES); @@ -437,7 +435,6 @@ zio_crypt_key_wrap(crypto_key_t *cwkey, zio_crypt_key_t *key, uint8_t *iv, uint_t enc_len, keydata_len, aad_len; ASSERT3U(crypt, <, ZIO_CRYPT_FUNCTIONS); - ASSERT3U(cwkey->ck_format, ==, CRYPTO_KEY_RAW); zfs_uio_init(&cuio, &cuio_s); @@ -453,9 +450,8 @@ zio_crypt_key_wrap(crypto_key_t *cwkey, zio_crypt_key_t *key, uint8_t *iv, * the plain text (source) to the cipher buffer (dest). * We set iovecs[0] -- the authentication data -- below. */ - bcopy((void*)key->zk_master_keydata, keydata_out, keydata_len); - bcopy((void*)key->zk_hmac_keydata, hmac_keydata_out, - SHA512_HMAC_KEYLEN); + memcpy(keydata_out, key->zk_master_keydata, keydata_len); + memcpy(hmac_keydata_out, key->zk_hmac_keydata, SHA512_HMAC_KEYLEN); iovecs[1].iov_base = keydata_out; iovecs[1].iov_len = keydata_len; iovecs[2].iov_base = hmac_keydata_out; @@ -518,7 +514,6 @@ zio_crypt_key_unwrap(crypto_key_t *cwkey, uint64_t crypt, uint64_t version, uint_t enc_len, keydata_len, aad_len; ASSERT3U(crypt, <, ZIO_CRYPT_FUNCTIONS); - ASSERT3U(cwkey->ck_format, ==, CRYPTO_KEY_RAW); keydata_len = zio_crypt_table[crypt].ci_keylen; rw_init(&key->zk_salt_lock, NULL, RW_DEFAULT, NULL); @@ -533,12 +528,11 @@ zio_crypt_key_unwrap(crypto_key_t *cwkey, uint64_t crypt, uint64_t version, */ dst = key->zk_master_keydata; src = keydata; - - bcopy(src, dst, keydata_len); + memcpy(dst, src, keydata_len); dst = key->zk_hmac_keydata; src = hmac_keydata; - bcopy(src, dst, SHA512_HMAC_KEYLEN); + memcpy(dst, src, SHA512_HMAC_KEYLEN); iovecs[1].iov_base = key->zk_master_keydata; iovecs[1].iov_len = keydata_len; @@ -586,11 +580,9 @@ zio_crypt_key_unwrap(crypto_key_t *cwkey, uint64_t crypt, uint64_t version, goto error; /* initialize keys for ICP */ - key->zk_current_key.ck_format = CRYPTO_KEY_RAW; key->zk_current_key.ck_data = key->zk_current_keydata; key->zk_current_key.ck_length = CRYPTO_BYTES2BITS(keydata_len); - key->zk_hmac_key.ck_format = CRYPTO_KEY_RAW; key->zk_hmac_key.ck_data = key->zk_hmac_keydata; key->zk_hmac_key.ck_length = CRYPTO_BYTES2BITS(SHA512_HMAC_KEYLEN); @@ -624,7 +616,7 @@ zio_crypt_generate_iv(uint8_t *ivbuf) return (0); error: - bzero(ivbuf, ZIO_DATA_IV_LEN); + memset(ivbuf, 0, ZIO_DATA_IV_LEN); return (ret); } @@ -639,7 +631,7 @@ zio_crypt_do_hmac(zio_crypt_key_t *key, uint8_t *data, uint_t datalen, crypto_mac(&key->zk_hmac_key, data, datalen, raw_digestbuf, SHA512_DIGEST_LENGTH); - bcopy(raw_digestbuf, digestbuf, digestlen); + memcpy(digestbuf, raw_digestbuf, digestlen); return (0); } @@ -656,8 +648,8 @@ zio_crypt_generate_iv_salt_dedup(zio_crypt_key_t *key, uint8_t *data, if (ret != 0) return (ret); - bcopy(digestbuf, salt, ZIO_DATA_SALT_LEN); - bcopy(digestbuf + ZIO_DATA_SALT_LEN, ivbuf, ZIO_DATA_IV_LEN); + memcpy(salt, digestbuf, ZIO_DATA_SALT_LEN); + memcpy(ivbuf, digestbuf + ZIO_DATA_SALT_LEN, ZIO_DATA_IV_LEN); return (0); } @@ -680,18 +672,18 @@ zio_crypt_encode_params_bp(blkptr_t *bp, uint8_t *salt, uint8_t *iv) ASSERT(BP_IS_ENCRYPTED(bp)); if (!BP_SHOULD_BYTESWAP(bp)) { - bcopy(salt, &bp->blk_dva[2].dva_word[0], sizeof (uint64_t)); - bcopy(iv, &bp->blk_dva[2].dva_word[1], sizeof (uint64_t)); - bcopy(iv + sizeof (uint64_t), &val32, sizeof (uint32_t)); + memcpy(&bp->blk_dva[2].dva_word[0], salt, sizeof (uint64_t)); + memcpy(&bp->blk_dva[2].dva_word[1], iv, sizeof (uint64_t)); + memcpy(&val32, iv + sizeof (uint64_t), sizeof (uint32_t)); BP_SET_IV2(bp, val32); } else { - bcopy(salt, &val64, sizeof (uint64_t)); + memcpy(&val64, salt, sizeof (uint64_t)); bp->blk_dva[2].dva_word[0] = BSWAP_64(val64); - bcopy(iv, &val64, sizeof (uint64_t)); + memcpy(&val64, iv, sizeof (uint64_t)); bp->blk_dva[2].dva_word[1] = BSWAP_64(val64); - bcopy(iv + sizeof (uint64_t), &val32, sizeof (uint32_t)); + memcpy(&val32, iv + sizeof (uint64_t), sizeof (uint32_t)); BP_SET_IV2(bp, BSWAP_32(val32)); } } @@ -706,26 +698,26 @@ zio_crypt_decode_params_bp(const blkptr_t *bp, uint8_t *salt, uint8_t *iv) /* for convenience, so callers don't need to check */ if (BP_IS_AUTHENTICATED(bp)) { - bzero(salt, ZIO_DATA_SALT_LEN); - bzero(iv, ZIO_DATA_IV_LEN); + memset(salt, 0, ZIO_DATA_SALT_LEN); + memset(iv, 0, ZIO_DATA_IV_LEN); return; } if (!BP_SHOULD_BYTESWAP(bp)) { - bcopy(&bp->blk_dva[2].dva_word[0], salt, sizeof (uint64_t)); - bcopy(&bp->blk_dva[2].dva_word[1], iv, sizeof (uint64_t)); + memcpy(salt, &bp->blk_dva[2].dva_word[0], sizeof (uint64_t)); + memcpy(iv, &bp->blk_dva[2].dva_word[1], sizeof (uint64_t)); val32 = (uint32_t)BP_GET_IV2(bp); - bcopy(&val32, iv + sizeof (uint64_t), sizeof (uint32_t)); + memcpy(iv + sizeof (uint64_t), &val32, sizeof (uint32_t)); } else { val64 = BSWAP_64(bp->blk_dva[2].dva_word[0]); - bcopy(&val64, salt, sizeof (uint64_t)); + memcpy(salt, &val64, sizeof (uint64_t)); val64 = BSWAP_64(bp->blk_dva[2].dva_word[1]); - bcopy(&val64, iv, sizeof (uint64_t)); + memcpy(iv, &val64, sizeof (uint64_t)); val32 = BSWAP_32((uint32_t)BP_GET_IV2(bp)); - bcopy(&val32, iv + sizeof (uint64_t), sizeof (uint32_t)); + memcpy(iv + sizeof (uint64_t), &val32, sizeof (uint32_t)); } } @@ -738,14 +730,14 @@ zio_crypt_encode_mac_bp(blkptr_t *bp, uint8_t *mac) ASSERT3U(BP_GET_TYPE(bp), !=, DMU_OT_OBJSET); if (!BP_SHOULD_BYTESWAP(bp)) { - bcopy(mac, &bp->blk_cksum.zc_word[2], sizeof (uint64_t)); - bcopy(mac + sizeof (uint64_t), &bp->blk_cksum.zc_word[3], + memcpy(&bp->blk_cksum.zc_word[2], mac, sizeof (uint64_t)); + memcpy(&bp->blk_cksum.zc_word[3], mac + sizeof (uint64_t), sizeof (uint64_t)); } else { - bcopy(mac, &val64, sizeof (uint64_t)); + memcpy(&val64, mac, sizeof (uint64_t)); bp->blk_cksum.zc_word[2] = BSWAP_64(val64); - bcopy(mac + sizeof (uint64_t), &val64, sizeof (uint64_t)); + memcpy(&val64, mac + sizeof (uint64_t), sizeof (uint64_t)); bp->blk_cksum.zc_word[3] = BSWAP_64(val64); } } @@ -759,20 +751,20 @@ zio_crypt_decode_mac_bp(const blkptr_t *bp, uint8_t *mac) /* for convenience, so callers don't need to check */ if (BP_GET_TYPE(bp) == DMU_OT_OBJSET) { - bzero(mac, ZIO_DATA_MAC_LEN); + memset(mac, 0, ZIO_DATA_MAC_LEN); return; } if (!BP_SHOULD_BYTESWAP(bp)) { - bcopy(&bp->blk_cksum.zc_word[2], mac, sizeof (uint64_t)); - bcopy(&bp->blk_cksum.zc_word[3], mac + sizeof (uint64_t), + memcpy(mac, &bp->blk_cksum.zc_word[2], sizeof (uint64_t)); + memcpy(mac + sizeof (uint64_t), &bp->blk_cksum.zc_word[3], sizeof (uint64_t)); } else { val64 = BSWAP_64(bp->blk_cksum.zc_word[2]); - bcopy(&val64, mac, sizeof (uint64_t)); + memcpy(mac, &val64, sizeof (uint64_t)); val64 = BSWAP_64(bp->blk_cksum.zc_word[3]); - bcopy(&val64, mac + sizeof (uint64_t), sizeof (uint64_t)); + memcpy(mac + sizeof (uint64_t), &val64, sizeof (uint64_t)); } } @@ -781,8 +773,8 @@ zio_crypt_encode_mac_zil(void *data, uint8_t *mac) { zil_chain_t *zilc = data; - bcopy(mac, &zilc->zc_eck.zec_cksum.zc_word[2], sizeof (uint64_t)); - bcopy(mac + sizeof (uint64_t), &zilc->zc_eck.zec_cksum.zc_word[3], + memcpy(&zilc->zc_eck.zec_cksum.zc_word[2], mac, sizeof (uint64_t)); + memcpy(&zilc->zc_eck.zec_cksum.zc_word[3], mac + sizeof (uint64_t), sizeof (uint64_t)); } @@ -796,8 +788,8 @@ zio_crypt_decode_mac_zil(const void *data, uint8_t *mac) */ const zil_chain_t *zilc = data; - bcopy(&zilc->zc_eck.zec_cksum.zc_word[2], mac, sizeof (uint64_t)); - bcopy(&zilc->zc_eck.zec_cksum.zc_word[3], mac + sizeof (uint64_t), + memcpy(mac, &zilc->zc_eck.zec_cksum.zc_word[2], sizeof (uint64_t)); + memcpy(mac + sizeof (uint64_t), &zilc->zc_eck.zec_cksum.zc_word[3], sizeof (uint64_t)); } @@ -824,7 +816,7 @@ zio_crypt_copy_dnode_bonus(abd_t *src_abd, uint8_t *dst, uint_t datalen) if (dnp->dn_type != DMU_OT_NONE && DMU_OT_IS_ENCRYPTED(dnp->dn_bonustype) && dnp->dn_bonuslen != 0) { - bcopy(DN_BONUS(dnp), DN_BONUS(&ddnp[i]), + memcpy(DN_BONUS(&ddnp[i]), DN_BONUS(dnp), DN_MAX_BONUS_LEN(dnp)); } } @@ -952,7 +944,7 @@ zio_crypt_bp_do_aad_updates(uint8_t **aadp, uint_t *aad_len, uint64_t version, blkptr_auth_buf_t bab; zio_crypt_bp_auth_init(version, should_bswap, bp, &bab, &bab_len); - bcopy(&bab, *aadp, bab_len); + memcpy(*aadp, &bab, bab_len); *aadp += bab_len; *aad_len += bab_len; } @@ -967,7 +959,7 @@ zio_crypt_do_dnode_hmac_updates(crypto_context_t ctx, uint64_t version, uint8_t tmp_dncore[offsetof(dnode_phys_t, dn_blkptr)]; /* authenticate the core dnode (masking out non-portable bits) */ - bcopy(dnp, tmp_dncore, sizeof (tmp_dncore)); + memcpy(tmp_dncore, dnp, sizeof (tmp_dncore)); adnp = (dnode_phys_t *)tmp_dncore; if (le_bswap) { adnp->dn_datablkszsec = BSWAP_16(adnp->dn_datablkszsec); @@ -1063,7 +1055,7 @@ zio_crypt_do_objset_hmacs(zio_crypt_key_t *key, void *data, uint_t datalen, crypto_mac_final(ctx, raw_portable_mac, SHA512_DIGEST_LENGTH); - bcopy(raw_portable_mac, portable_mac, ZIO_OBJSET_MAC_LEN); + memcpy(portable_mac, raw_portable_mac, ZIO_OBJSET_MAC_LEN); /* * This is necessary here as we check next whether @@ -1092,7 +1084,7 @@ zio_crypt_do_objset_hmacs(zio_crypt_key_t *key, void *data, uint_t datalen, osp->os_userused_dnode.dn_type == DMU_OT_NONE && osp->os_groupused_dnode.dn_type == DMU_OT_NONE) || (datalen <= OBJSET_PHYS_SIZE_V1)) { - bzero(local_mac, ZIO_OBJSET_MAC_LEN); + memset(local_mac, 0, ZIO_OBJSET_MAC_LEN); return (0); } @@ -1135,13 +1127,13 @@ zio_crypt_do_objset_hmacs(zio_crypt_key_t *key, void *data, uint_t datalen, crypto_mac_final(ctx, raw_local_mac, SHA512_DIGEST_LENGTH); - bcopy(raw_local_mac, local_mac, ZIO_OBJSET_MAC_LEN); + memcpy(local_mac, raw_local_mac, ZIO_OBJSET_MAC_LEN); return (0); error: - bzero(portable_mac, ZIO_OBJSET_MAC_LEN); - bzero(local_mac, ZIO_OBJSET_MAC_LEN); + memset(portable_mac, 0, ZIO_OBJSET_MAC_LEN); + memset(local_mac, 0, ZIO_OBJSET_MAC_LEN); return (ret); } @@ -1178,11 +1170,11 @@ zio_crypt_do_indirect_mac_checksum_impl(boolean_t generate, void *buf, SHA2Final(digestbuf, &ctx); if (generate) { - bcopy(digestbuf, cksum, ZIO_DATA_MAC_LEN); + memcpy(cksum, digestbuf, ZIO_DATA_MAC_LEN); return (0); } - if (bcmp(digestbuf, cksum, ZIO_DATA_MAC_LEN) != 0) { + if (memcmp(digestbuf, cksum, ZIO_DATA_MAC_LEN) != 0) { #ifdef FCRYPTO_DEBUG printf("%s(%d): Setting ECKSUM\n", __FUNCTION__, __LINE__); #endif @@ -1270,7 +1262,7 @@ zio_crypt_init_uios_zil(boolean_t encrypt, uint8_t *plainbuf, src = cipherbuf; dst = plainbuf; } - bcopy(src, dst, datalen); + memcpy(dst, src, datalen); /* Find the start and end record of the log block. */ zilc = (zil_chain_t *)src; @@ -1309,7 +1301,7 @@ zio_crypt_init_uios_zil(boolean_t encrypt, uint8_t *plainbuf, * the embedded checksum will not have been calculated yet, so we don't * authenticate that. */ - bcopy(src, aadp, sizeof (zil_chain_t) - sizeof (zio_eck_t)); + memcpy(aadp, src, sizeof (zil_chain_t) - sizeof (zio_eck_t)); aadp += sizeof (zil_chain_t) - sizeof (zio_eck_t); aad_len += sizeof (zil_chain_t) - sizeof (zio_eck_t); @@ -1335,8 +1327,8 @@ zio_crypt_init_uios_zil(boolean_t encrypt, uint8_t *plainbuf, } /* copy the common lr_t */ - bcopy(slrp, dlrp, sizeof (lr_t)); - bcopy(slrp, aadp, sizeof (lr_t)); + memcpy(dlrp, slrp, sizeof (lr_t)); + memcpy(aadp, slrp, sizeof (lr_t)); aadp += sizeof (lr_t); aad_len += sizeof (lr_t); @@ -1353,11 +1345,12 @@ zio_crypt_init_uios_zil(boolean_t encrypt, uint8_t *plainbuf, dst_iovecs[vec].iov_len = crypt_len; /* copy the bp now since it will not be encrypted */ - bcopy(slrp + sizeof (lr_write_t) - sizeof (blkptr_t), - dlrp + sizeof (lr_write_t) - sizeof (blkptr_t), + memcpy(dlrp + sizeof (lr_write_t) - sizeof (blkptr_t), + slrp + sizeof (lr_write_t) - sizeof (blkptr_t), + sizeof (blkptr_t)); + memcpy(aadp, + slrp + sizeof (lr_write_t) - sizeof (blkptr_t), sizeof (blkptr_t)); - bcopy(slrp + sizeof (lr_write_t) - sizeof (blkptr_t), - aadp, sizeof (blkptr_t)); aadp += sizeof (blkptr_t); aad_len += sizeof (blkptr_t); vec++; @@ -1425,7 +1418,7 @@ zio_crypt_init_uios_dnode(boolean_t encrypt, uint64_t version, src = cipherbuf; dst = plainbuf; } - bcopy(src, dst, datalen); + memcpy(dst, src, datalen); sdnp = (dnode_phys_t *)src; ddnp = (dnode_phys_t *)dst; @@ -1468,10 +1461,11 @@ zio_crypt_init_uios_dnode(boolean_t encrypt, uint64_t version, dnp = &sdnp[i]; /* copy over the core fields and blkptrs (kept as plaintext) */ - bcopy(dnp, &ddnp[i], (uint8_t *)DN_BONUS(dnp) - (uint8_t *)dnp); + memcpy(&ddnp[i], dnp, + (uint8_t *)DN_BONUS(dnp) - (uint8_t *)dnp); if (dnp->dn_flags & DNODE_FLAG_SPILL_BLKPTR) { - bcopy(DN_SPILL_BLKPTR(dnp), DN_SPILL_BLKPTR(&ddnp[i]), + memcpy(DN_SPILL_BLKPTR(&ddnp[i]), DN_SPILL_BLKPTR(dnp), sizeof (blkptr_t)); } @@ -1486,7 +1480,7 @@ zio_crypt_init_uios_dnode(boolean_t encrypt, uint64_t version, * authenticated data. */ crypt_len = offsetof(dnode_phys_t, dn_blkptr); - bcopy(dnp, aadp, crypt_len); + memcpy(aadp, dnp, crypt_len); adnp = (dnode_phys_t *)aadp; adnp->dn_flags &= DNODE_CRYPT_PORTABLE_FLAGS_MASK; adnp->dn_used = 0; @@ -1523,8 +1517,8 @@ zio_crypt_init_uios_dnode(boolean_t encrypt, uint64_t version, vec++; total_len += crypt_len; } else { - bcopy(DN_BONUS(dnp), DN_BONUS(&ddnp[i]), crypt_len); - bcopy(DN_BONUS(dnp), aadp, crypt_len); + memcpy(DN_BONUS(&ddnp[i]), DN_BONUS(dnp), crypt_len); + memcpy(aadp, DN_BONUS(dnp), crypt_len); aadp += crypt_len; aad_len += crypt_len; } @@ -1567,7 +1561,7 @@ zio_crypt_init_uios_normal(boolean_t encrypt, uint8_t *plainbuf, ret = SET_ERROR(ENOMEM); goto error; } - bzero(cipher_iovecs, nr_cipher * sizeof (iovec_t)); + memset(cipher_iovecs, 0, nr_cipher * sizeof (iovec_t)); if (encrypt) { src = plainbuf; @@ -1576,7 +1570,7 @@ zio_crypt_init_uios_normal(boolean_t encrypt, uint8_t *plainbuf, src = cipherbuf; dst = plainbuf; } - bcopy(src, dst, datalen); + memcpy(dst, src, datalen); cipher_iovecs[0].iov_base = dst; cipher_iovecs[0].iov_len = datalen; @@ -1684,8 +1678,8 @@ zio_do_crypt_data(boolean_t encrypt, zio_crypt_key_t *key, zfs_uio_init(&puio, &puio_s); zfs_uio_init(&cuio, &cuio_s); - bzero(GET_UIO_STRUCT(&puio), sizeof (struct uio)); - bzero(GET_UIO_STRUCT(&cuio), sizeof (struct uio)); + memset(GET_UIO_STRUCT(&puio), 0, sizeof (struct uio)); + memset(GET_UIO_STRUCT(&cuio), 0, sizeof (struct uio)); #ifdef FCRYPTO_DEBUG printf("%s(%s, %p, %p, %d, %p, %p, %u, %s, %p, %p, %p)\n", @@ -1716,7 +1710,7 @@ zio_do_crypt_data(boolean_t encrypt, zio_crypt_key_t *key, rw_enter(&key->zk_salt_lock, RW_READER); locked = B_TRUE; - if (bcmp(salt, key->zk_salt, ZIO_DATA_SALT_LEN) == 0) { + if (memcmp(salt, key->zk_salt, ZIO_DATA_SALT_LEN) == 0) { ckey = &key->zk_current_key; tmpl = &key->zk_session; } else { @@ -1727,7 +1721,6 @@ zio_do_crypt_data(boolean_t encrypt, zio_crypt_key_t *key, salt, ZIO_DATA_SALT_LEN, enc_keydata, keydata_len); if (ret != 0) goto error; - tmp_ckey.ck_format = CRYPTO_KEY_RAW; tmp_ckey.ck_data = enc_keydata; tmp_ckey.ck_length = CRYPTO_BYTES2BITS(keydata_len); @@ -1748,7 +1741,7 @@ zio_do_crypt_data(boolean_t encrypt, zio_crypt_key_t *key, if (authbuf != NULL) zio_buf_free(authbuf, datalen); if (ckey == &tmp_ckey) - bzero(enc_keydata, keydata_len); + memset(enc_keydata, 0, keydata_len); zio_crypt_destroy_uio(&puio); zio_crypt_destroy_uio(&cuio); @@ -1760,14 +1753,14 @@ zio_do_crypt_data(boolean_t encrypt, zio_crypt_key_t *key, kmem_free(failed_decrypt_buf, failed_decrypt_size); failed_decrypt_buf = kmem_alloc(datalen, KM_SLEEP); failed_decrypt_size = datalen; - bcopy(cipherbuf, failed_decrypt_buf, datalen); + memcpy(failed_decrypt_buf, cipherbuf, datalen); } if (locked) rw_exit(&key->zk_salt_lock); if (authbuf != NULL) zio_buf_free(authbuf, datalen); if (ckey == &tmp_ckey) - bzero(enc_keydata, keydata_len); + memset(enc_keydata, 0, keydata_len); zio_crypt_destroy_uio(&puio); zio_crypt_destroy_uio(&cuio); return (SET_ERROR(ret)); diff --git a/module/os/freebsd/zfs/zvol_os.c b/module/os/freebsd/zfs/zvol_os.c index b72c6216a8f7..ac030f75323e 100644 --- a/module/os/freebsd/zfs/zvol_os.c +++ b/module/os/freebsd/zfs/zvol_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -203,7 +203,6 @@ static int zvol_geom_bio_getattr(struct bio *bp); * GEOM mode implementation */ -/*ARGSUSED*/ static int zvol_geom_open(struct g_provider *pp, int flag, int count) { @@ -312,15 +311,13 @@ zvol_geom_open(struct g_provider *pp, int flag, int count) err = SET_ERROR(EBUSY); goto out_opened; } -#ifdef FEXCL - if (flag & FEXCL) { + if (flag & O_EXCL) { if (zv->zv_open_count != 0) { err = SET_ERROR(EBUSY); goto out_opened; } zv->zv_flags |= ZVOL_EXCL; } -#endif zv->zv_open_count += count; out_opened: @@ -336,10 +333,10 @@ zvol_geom_open(struct g_provider *pp, int flag, int count) return (err); } -/*ARGSUSED*/ static int zvol_geom_close(struct g_provider *pp, int flag, int count) { + (void) flag; zvol_state_t *zv; boolean_t drop_suspend = B_TRUE; int new_open_count; @@ -953,18 +950,16 @@ zvol_cdev_open(struct cdev *dev, int flags, int fmt, struct thread *td) err = SET_ERROR(EBUSY); goto out_opened; } -#ifdef FEXCL - if (flags & FEXCL) { + if (flags & O_EXCL) { if (zv->zv_open_count != 0) { err = SET_ERROR(EBUSY); goto out_opened; } zv->zv_flags |= ZVOL_EXCL; } -#endif zv->zv_open_count++; - if (flags & (FSYNC | FDSYNC)) { + if (flags & O_SYNC) { zsd = &zv->zv_zso->zso_dev; zsd->zsd_sync_cnt++; if (zsd->zsd_sync_cnt == 1 && @@ -1038,7 +1033,7 @@ zvol_cdev_close(struct cdev *dev, int flags, int fmt, struct thread *td) * You may get multiple opens, but only one close. */ zv->zv_open_count--; - if (flags & (FSYNC | FDSYNC)) { + if (flags & O_SYNC) { zsd = &zv->zv_zso->zso_dev; zsd->zsd_sync_cnt--; } @@ -1194,7 +1189,7 @@ zvol_ensure_zilog(zvol_state_t *zv) } if (zv->zv_zilog == NULL) { zv->zv_zilog = zil_open(zv->zv_objset, - zvol_get_data); + zvol_get_data, &zv->zv_kstat.dk_zil_sums); zv->zv_flags |= ZVOL_WRITTEN_TO; /* replay / destroy done in zvol_os_create_minor() */ VERIFY0(zv->zv_zilog->zl_header->zh_flags & @@ -1427,8 +1422,12 @@ zvol_os_create_minor(const char *name) zv->zv_volsize = volsize; zv->zv_objset = os; + ASSERT3P(zv->zv_kstat.dk_kstats, ==, NULL); + error = dataset_kstats_create(&zv->zv_kstat, zv->zv_objset); + if (error) + goto out_dmu_objset_disown; ASSERT3P(zv->zv_zilog, ==, NULL); - zv->zv_zilog = zil_open(os, zvol_get_data); + zv->zv_zilog = zil_open(os, zvol_get_data, &zv->zv_kstat.dk_zil_sums); if (spa_writeable(dmu_objset_spa(os))) { if (zil_replay_disable) zil_destroy(zv->zv_zilog, B_FALSE); @@ -1437,8 +1436,6 @@ zvol_os_create_minor(const char *name) } zil_close(zv->zv_zilog); zv->zv_zilog = NULL; - ASSERT3P(zv->zv_kstat.dk_kstats, ==, NULL); - dataset_kstats_create(&zv->zv_kstat, zv->zv_objset); /* TODO: prefetch for geom tasting */ diff --git a/module/os/linux/spl/Makefile.in b/module/os/linux/spl/Makefile.in deleted file mode 100644 index b2325f91b4a7..000000000000 --- a/module/os/linux/spl/Makefile.in +++ /dev/null @@ -1,17 +0,0 @@ -$(MODULE)-objs += ../os/linux/spl/spl-atomic.o -$(MODULE)-objs += ../os/linux/spl/spl-condvar.o -$(MODULE)-objs += ../os/linux/spl/spl-cred.o -$(MODULE)-objs += ../os/linux/spl/spl-err.o -$(MODULE)-objs += ../os/linux/spl/spl-generic.o -$(MODULE)-objs += ../os/linux/spl/spl-kmem.o -$(MODULE)-objs += ../os/linux/spl/spl-kmem-cache.o -$(MODULE)-objs += ../os/linux/spl/spl-kstat.o -$(MODULE)-objs += ../os/linux/spl/spl-proc.o -$(MODULE)-objs += ../os/linux/spl/spl-procfs-list.o -$(MODULE)-objs += ../os/linux/spl/spl-taskq.o -$(MODULE)-objs += ../os/linux/spl/spl-thread.o -$(MODULE)-objs += ../os/linux/spl/spl-trace.o -$(MODULE)-objs += ../os/linux/spl/spl-tsd.o -$(MODULE)-objs += ../os/linux/spl/spl-vmem.o -$(MODULE)-objs += ../os/linux/spl/spl-xdr.o -$(MODULE)-objs += ../os/linux/spl/spl-zlib.o diff --git a/module/os/linux/spl/spl-cred.c b/module/os/linux/spl/spl-cred.c index 8fe1cc30ba99..f81b9540a639 100644 --- a/module/os/linux/spl/spl-cred.c +++ b/module/os/linux/spl/spl-cred.c @@ -128,7 +128,7 @@ groupmember(gid_t gid, const cred_t *cr) uid_t crgetuid(const cred_t *cr) { - return (KUID_TO_SUID(cr->euid)); + return (KUID_TO_SUID(cr->fsuid)); } /* Return the real user id */ @@ -138,44 +138,9 @@ crgetruid(const cred_t *cr) return (KUID_TO_SUID(cr->uid)); } -/* Return the saved user id */ -uid_t -crgetsuid(const cred_t *cr) -{ - return (KUID_TO_SUID(cr->suid)); -} - -/* Return the filesystem user id */ -uid_t -crgetfsuid(const cred_t *cr) -{ - return (KUID_TO_SUID(cr->fsuid)); -} - /* Return the effective group id */ gid_t crgetgid(const cred_t *cr) -{ - return (KGID_TO_SGID(cr->egid)); -} - -/* Return the real group id */ -gid_t -crgetrgid(const cred_t *cr) -{ - return (KGID_TO_SGID(cr->gid)); -} - -/* Return the saved group id */ -gid_t -crgetsgid(const cred_t *cr) -{ - return (KGID_TO_SGID(cr->sgid)); -} - -/* Return the filesystem group id */ -gid_t -crgetfsgid(const cred_t *cr) { return (KGID_TO_SGID(cr->fsgid)); } @@ -184,12 +149,7 @@ EXPORT_SYMBOL(crhold); EXPORT_SYMBOL(crfree); EXPORT_SYMBOL(crgetuid); EXPORT_SYMBOL(crgetruid); -EXPORT_SYMBOL(crgetsuid); -EXPORT_SYMBOL(crgetfsuid); EXPORT_SYMBOL(crgetgid); -EXPORT_SYMBOL(crgetrgid); -EXPORT_SYMBOL(crgetsgid); -EXPORT_SYMBOL(crgetfsgid); EXPORT_SYMBOL(crgetngroups); EXPORT_SYMBOL(crgetgroups); EXPORT_SYMBOL(groupmember); diff --git a/module/os/linux/spl/spl-err.c b/module/os/linux/spl/spl-err.c index 70512c63c9f5..c84c39b56bf7 100644 --- a/module/os/linux/spl/spl-err.c +++ b/module/os/linux/spl/spl-err.c @@ -100,6 +100,9 @@ vcmn_err(int ce, const char *fmt, va_list ap) break; case CE_PANIC: printk(KERN_EMERG "PANIC: %s\n", msg); + if (spl_panic_halt) + panic("%s", msg); + spl_dumpstack(); /* Halt the thread to facilitate further debugging */ diff --git a/module/os/linux/spl/spl-generic.c b/module/os/linux/spl/spl-generic.c index 5bf2f6912457..5179100d1665 100644 --- a/module/os/linux/spl/spl-generic.c +++ b/module/os/linux/spl/spl-generic.c @@ -42,7 +42,7 @@ #include #include #include -#include +#include #include #include #include @@ -425,22 +425,33 @@ EXPORT_SYMBOL(__aeabi_ldivmod); * functions against their Solaris counterparts. It is possible that I * may have misinterpreted the man page or the man page is incorrect. */ -int ddi_strtoul(const char *, char **, int, unsigned long *); int ddi_strtol(const char *, char **, int, long *); int ddi_strtoull(const char *, char **, int, unsigned long long *); int ddi_strtoll(const char *, char **, int, long long *); -#define define_ddi_strtoux(type, valtype) \ -int ddi_strtou##type(const char *str, char **endptr, \ +#define define_ddi_strtox(type, valtype) \ +int ddi_strto##type(const char *str, char **endptr, \ int base, valtype *result) \ { \ valtype last_value, value = 0; \ char *ptr = (char *)str; \ - int flag = 1, digit; \ + int digit, minus = 0; \ + \ + while (strchr(" \t\n\r\f", *ptr)) \ + ++ptr; \ \ if (strlen(ptr) == 0) \ return (EINVAL); \ \ + switch (*ptr) { \ + case '-': \ + minus = 1; \ + zfs_fallthrough; \ + case '+': \ + ++ptr; \ + break; \ + } \ + \ /* Auto-detect base based on prefix */ \ if (!base) { \ if (str[0] == '0') { \ @@ -474,46 +485,21 @@ int ddi_strtou##type(const char *str, char **endptr, \ if (last_value > value) /* Overflow */ \ return (ERANGE); \ \ - flag = 1; \ ptr++; \ } \ \ - if (flag) \ - *result = value; \ + *result = minus ? -value : value; \ \ if (endptr) \ - *endptr = (char *)(flag ? ptr : str); \ + *endptr = ptr; \ \ return (0); \ } \ -#define define_ddi_strtox(type, valtype) \ -int ddi_strto##type(const char *str, char **endptr, \ - int base, valtype *result) \ -{ \ - int rc; \ - \ - if (*str == '-') { \ - rc = ddi_strtou##type(str + 1, endptr, base, result); \ - if (!rc) { \ - if (*endptr == str + 1) \ - *endptr = (char *)str; \ - else \ - *result = -*result; \ - } \ - } else { \ - rc = ddi_strtou##type(str, endptr, base, result); \ - } \ - \ - return (rc); \ -} - -define_ddi_strtoux(l, unsigned long) define_ddi_strtox(l, long) -define_ddi_strtoux(ll, unsigned long long) +define_ddi_strtox(ull, unsigned long long) define_ddi_strtox(ll, long long) -EXPORT_SYMBOL(ddi_strtoul); EXPORT_SYMBOL(ddi_strtol); EXPORT_SYMBOL(ddi_strtoll); EXPORT_SYMBOL(ddi_strtoull); @@ -771,7 +757,6 @@ spl_init(void) { int rc = 0; - bzero(&p0, sizeof (proc_t)); spl_random_init(); if ((rc = spl_kvmem_init())) @@ -795,8 +780,13 @@ spl_init(void) if ((rc = spl_zlib_init())) goto out7; + if ((rc = spl_zone_init())) + goto out8; + return (rc); +out8: + spl_zlib_fini(); out7: spl_kstat_fini(); out6: @@ -816,6 +806,7 @@ spl_init(void) static void __exit spl_fini(void) { + spl_zone_fini(); spl_zlib_fini(); spl_kstat_fini(); spl_proc_fini(); @@ -829,7 +820,7 @@ spl_fini(void) module_init(spl_init); module_exit(spl_fini); -ZFS_MODULE_DESCRIPTION("Solaris Porting Layer"); -ZFS_MODULE_AUTHOR(ZFS_META_AUTHOR); -ZFS_MODULE_LICENSE("GPL"); -ZFS_MODULE_VERSION(ZFS_META_VERSION "-" ZFS_META_RELEASE); +MODULE_DESCRIPTION("Solaris Porting Layer"); +MODULE_AUTHOR(ZFS_META_AUTHOR); +MODULE_LICENSE("GPL"); +MODULE_VERSION(ZFS_META_VERSION "-" ZFS_META_RELEASE); diff --git a/module/os/linux/spl/spl-kmem-cache.c b/module/os/linux/spl/spl-kmem-cache.c index bb2b56880646..ba4ca49a2ac9 100644 --- a/module/os/linux/spl/spl-kmem-cache.c +++ b/module/os/linux/spl/spl-kmem-cache.c @@ -679,7 +679,7 @@ spl_magazine_destroy(spl_kmem_cache_t *skc) * KMC_NODEBUG Disable debugging (unsupported) */ spl_kmem_cache_t * -spl_kmem_cache_create(char *name, size_t size, size_t align, +spl_kmem_cache_create(const char *name, size_t size, size_t align, spl_kmem_ctor_t ctor, spl_kmem_dtor_t dtor, void *reclaim, void *priv, void *vmp, int flags) { @@ -1420,7 +1420,7 @@ EXPORT_SYMBOL(spl_kmem_cache_reap_now); * it should do no harm. */ int -spl_kmem_cache_reap_active() +spl_kmem_cache_reap_active(void) { return (0); } diff --git a/module/os/linux/spl/spl-procfs-list.c b/module/os/linux/spl/spl-procfs-list.c index 81501460f04f..a4a24dcae2bd 100644 --- a/module/os/linux/spl/spl-procfs-list.c +++ b/module/os/linux/spl/spl-procfs-list.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/linux/spl/spl-taskq.c b/module/os/linux/spl/spl-taskq.c index bd0052e00de9..0aab148975aa 100644 --- a/module/os/linux/spl/spl-taskq.c +++ b/module/os/linux/spl/spl-taskq.c @@ -1229,13 +1229,6 @@ taskq_destroy(taskq_t *tq) } EXPORT_SYMBOL(taskq_destroy); -boolean_t -taskq_empty(taskq_t *tq) -{ - return (tq->tq_lowest_id == tq->tq_next_id); -} -EXPORT_SYMBOL(taskq_empty); - static unsigned int spl_taskq_kick = 0; /* diff --git a/module/os/linux/spl/spl-thread.c b/module/os/linux/spl/spl-thread.c index 16d2ca1b133b..32a2d34b1d93 100644 --- a/module/os/linux/spl/spl-thread.c +++ b/module/os/linux/spl/spl-thread.c @@ -62,15 +62,6 @@ thread_generic_wrapper(void *arg) return (0); } -void -__thread_exit(void) -{ - tsd_exit(); - SPL_KTHREAD_COMPLETE_AND_EXIT(NULL, 0); - /* Unreachable */ -} -EXPORT_SYMBOL(__thread_exit); - /* * thread_create() may block forever if it cannot create a thread or * allocate memory. This is preferable to returning a NULL which Solaris diff --git a/module/os/linux/spl/spl-trace.c b/module/os/linux/spl/spl-trace.c index 7912a381294d..d3e53e541b8b 100644 --- a/module/os/linux/spl/spl-trace.c +++ b/module/os/linux/spl/spl-trace.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/linux/spl/spl-zone.c b/module/os/linux/spl/spl-zone.c new file mode 100644 index 000000000000..b8a8b7cd8cd8 --- /dev/null +++ b/module/os/linux/spl/spl-zone.c @@ -0,0 +1,424 @@ +/* + * Copyright (c) 2021 Klara Systems, Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +#include +#include +#include +#include +#include +#include + +#if defined(CONFIG_USER_NS) +#include +#include +#endif + +static kmutex_t zone_datasets_lock; +static struct list_head zone_datasets; + +typedef struct zone_datasets { + struct list_head zds_list; /* zone_datasets linkage */ + struct user_namespace *zds_userns; /* namespace reference */ + struct list_head zds_datasets; /* datasets for the namespace */ +} zone_datasets_t; + +typedef struct zone_dataset { + struct list_head zd_list; /* zone_dataset linkage */ + size_t zd_dsnamelen; /* length of name */ + char zd_dsname[0]; /* name of the member dataset */ +} zone_dataset_t; + +#if defined(CONFIG_USER_NS) && defined(HAVE_USER_NS_COMMON_INUM) +/* + * Returns: + * - 0 on success + * - EBADF if it cannot open the provided file descriptor + * - ENOTTY if the file itself is a not a user namespace file. We want to + * intercept this error in the ZFS layer. We cannot just return one of the + * ZFS_ERR_* errors here as we want to preserve the seperation of the ZFS + * and the SPL layers. + */ +static int +user_ns_get(int fd, struct user_namespace **userns) +{ + struct kstatfs st; + struct file *nsfile; + struct ns_common *ns; + int error; + + if ((nsfile = fget(fd)) == NULL) + return (EBADF); + if (vfs_statfs(&nsfile->f_path, &st) != 0) { + error = ENOTTY; + goto done; + } + if (st.f_type != NSFS_MAGIC) { + error = ENOTTY; + goto done; + } + ns = get_proc_ns(file_inode(nsfile)); + if (ns->ops->type != CLONE_NEWUSER) { + error = ENOTTY; + goto done; + } + *userns = container_of(ns, struct user_namespace, ns); + + error = 0; +done: + fput(nsfile); + + return (error); +} +#endif /* defined(CONFIG_USER_NS) && defined(HAVE_USER_NS_COMMON_INUM) */ + +static unsigned int +user_ns_zoneid(struct user_namespace *user_ns) +{ + unsigned int r; + +#if defined(HAVE_USER_NS_COMMON_INUM) + r = user_ns->ns.inum; +#else + r = user_ns->proc_inum; +#endif + + return (r); +} + +static struct zone_datasets * +zone_datasets_lookup(unsigned int nsinum) +{ + zone_datasets_t *zds; + + list_for_each_entry(zds, &zone_datasets, zds_list) { + if (user_ns_zoneid(zds->zds_userns) == nsinum) + return (zds); + } + return (NULL); +} + +#if defined(CONFIG_USER_NS) && defined(HAVE_USER_NS_COMMON_INUM) +static struct zone_dataset * +zone_dataset_lookup(zone_datasets_t *zds, const char *dataset, size_t dsnamelen) +{ + zone_dataset_t *zd; + + list_for_each_entry(zd, &zds->zds_datasets, zd_list) { + if (zd->zd_dsnamelen != dsnamelen) + continue; + if (strncmp(zd->zd_dsname, dataset, dsnamelen) == 0) + return (zd); + } + + return (NULL); +} + +static int +zone_dataset_cred_check(cred_t *cred) +{ + + if (!uid_eq(cred->uid, GLOBAL_ROOT_UID)) + return (EPERM); + + return (0); +} +#endif /* defined(CONFIG_USER_NS) && defined(HAVE_USER_NS_COMMON_INUM) */ + +static int +zone_dataset_name_check(const char *dataset, size_t *dsnamelen) +{ + + if (dataset[0] == '\0' || dataset[0] == '/') + return (ENOENT); + + *dsnamelen = strlen(dataset); + /* Ignore trailing slash, if supplied. */ + if (dataset[*dsnamelen - 1] == '/') + (*dsnamelen)--; + + return (0); +} + +int +zone_dataset_attach(cred_t *cred, const char *dataset, int userns_fd) +{ +#if defined(CONFIG_USER_NS) && defined(HAVE_USER_NS_COMMON_INUM) + struct user_namespace *userns; + zone_datasets_t *zds; + zone_dataset_t *zd; + int error; + size_t dsnamelen; + + if ((error = zone_dataset_cred_check(cred)) != 0) + return (error); + if ((error = zone_dataset_name_check(dataset, &dsnamelen)) != 0) + return (error); + if ((error = user_ns_get(userns_fd, &userns)) != 0) + return (error); + + mutex_enter(&zone_datasets_lock); + zds = zone_datasets_lookup(user_ns_zoneid(userns)); + if (zds == NULL) { + zds = kmem_alloc(sizeof (zone_datasets_t), KM_SLEEP); + INIT_LIST_HEAD(&zds->zds_list); + INIT_LIST_HEAD(&zds->zds_datasets); + zds->zds_userns = userns; + /* + * Lock the namespace by incresing its refcount to prevent + * the namespace ID from being reused. + */ + get_user_ns(userns); + list_add_tail(&zds->zds_list, &zone_datasets); + } else { + zd = zone_dataset_lookup(zds, dataset, dsnamelen); + if (zd != NULL) { + mutex_exit(&zone_datasets_lock); + return (EEXIST); + } + } + + zd = kmem_alloc(sizeof (zone_dataset_t) + dsnamelen + 1, KM_SLEEP); + zd->zd_dsnamelen = dsnamelen; + strncpy(zd->zd_dsname, dataset, dsnamelen); + zd->zd_dsname[dsnamelen] = '\0'; + INIT_LIST_HEAD(&zd->zd_list); + list_add_tail(&zd->zd_list, &zds->zds_datasets); + + mutex_exit(&zone_datasets_lock); + return (0); +#else + return (ENXIO); +#endif /* defined(CONFIG_USER_NS) && defined(HAVE_USER_NS_COMMON_INUM) */ +} +EXPORT_SYMBOL(zone_dataset_attach); + +int +zone_dataset_detach(cred_t *cred, const char *dataset, int userns_fd) +{ +#if defined(CONFIG_USER_NS) && defined(HAVE_USER_NS_COMMON_INUM) + struct user_namespace *userns; + zone_datasets_t *zds; + zone_dataset_t *zd; + int error; + size_t dsnamelen; + + if ((error = zone_dataset_cred_check(cred)) != 0) + return (error); + if ((error = zone_dataset_name_check(dataset, &dsnamelen)) != 0) + return (error); + if ((error = user_ns_get(userns_fd, &userns)) != 0) + return (error); + + mutex_enter(&zone_datasets_lock); + zds = zone_datasets_lookup(user_ns_zoneid(userns)); + if (zds != NULL) + zd = zone_dataset_lookup(zds, dataset, dsnamelen); + if (zds == NULL || zd == NULL) { + mutex_exit(&zone_datasets_lock); + return (ENOENT); + } + + list_del(&zd->zd_list); + kmem_free(zd, sizeof (*zd) + zd->zd_dsnamelen + 1); + + /* Prune the namespace entry if it has no more delegations. */ + if (list_empty(&zds->zds_datasets)) { + /* + * Decrease the refcount now that the namespace is no longer + * used. It is no longer necessary to prevent the namespace ID + * from being reused. + */ + put_user_ns(userns); + list_del(&zds->zds_list); + kmem_free(zds, sizeof (*zds)); + } + + mutex_exit(&zone_datasets_lock); + return (0); +#else + return (ENXIO); +#endif /* defined(CONFIG_USER_NS) && defined(HAVE_USER_NS_COMMON_INUM) */ +} +EXPORT_SYMBOL(zone_dataset_detach); + +/* + * A dataset is visible if: + * - It is a parent of a namespace entry. + * - It is one of the namespace entries. + * - It is a child of a namespace entry. + * + * A dataset is writable if: + * - It is one of the namespace entries. + * - It is a child of a namespace entry. + * + * The parent datasets of namespace entries are visible and + * read-only to provide a path back to the root of the pool. + */ +int +zone_dataset_visible(const char *dataset, int *write) +{ + zone_datasets_t *zds; + zone_dataset_t *zd; + size_t dsnamelen, zd_len; + int visible; + + /* Default to read-only, in case visible is returned. */ + if (write != NULL) + *write = 0; + if (zone_dataset_name_check(dataset, &dsnamelen) != 0) + return (0); + if (INGLOBALZONE(curproc)) { + if (write != NULL) + *write = 1; + return (1); + } + + mutex_enter(&zone_datasets_lock); + zds = zone_datasets_lookup(crgetzoneid(curproc->cred)); + if (zds == NULL) { + mutex_exit(&zone_datasets_lock); + return (0); + } + + visible = 0; + list_for_each_entry(zd, &zds->zds_datasets, zd_list) { + zd_len = strlen(zd->zd_dsname); + if (zd_len > dsnamelen) { + /* + * The name of the namespace entry is longer than that + * of the dataset, so it could be that the dataset is a + * parent of the namespace entry. + */ + visible = memcmp(zd->zd_dsname, dataset, + dsnamelen) == 0 && + zd->zd_dsname[dsnamelen] == '/'; + if (visible) + break; + } else if (zd_len == dsnamelen) { + /* + * The name of the namespace entry is as long as that + * of the dataset, so perhaps the dataset itself is the + * namespace entry. + */ + visible = memcmp(zd->zd_dsname, dataset, zd_len) == 0; + if (visible) { + if (write != NULL) + *write = 1; + break; + } + } else { + /* + * The name of the namespace entry is shorter than that + * of the dataset, so perhaps the dataset is a child of + * the namespace entry. + */ + visible = memcmp(zd->zd_dsname, dataset, + zd_len) == 0 && dataset[zd_len] == '/'; + if (visible) { + if (write != NULL) + *write = 1; + break; + } + } + } + + mutex_exit(&zone_datasets_lock); + return (visible); +} +EXPORT_SYMBOL(zone_dataset_visible); + +unsigned int +global_zoneid(void) +{ + unsigned int z = 0; + +#if defined(CONFIG_USER_NS) + z = user_ns_zoneid(&init_user_ns); +#endif + + return (z); +} +EXPORT_SYMBOL(global_zoneid); + +unsigned int +crgetzoneid(const cred_t *cr) +{ + unsigned int r = 0; + +#if defined(CONFIG_USER_NS) + r = user_ns_zoneid(cr->user_ns); +#endif + + return (r); +} +EXPORT_SYMBOL(crgetzoneid); + +boolean_t +inglobalzone(proc_t *proc) +{ +#if defined(CONFIG_USER_NS) + return (proc->cred->user_ns == &init_user_ns); +#else + return (B_TRUE); +#endif +} +EXPORT_SYMBOL(inglobalzone); + +int +spl_zone_init(void) +{ + mutex_init(&zone_datasets_lock, NULL, MUTEX_DEFAULT, NULL); + INIT_LIST_HEAD(&zone_datasets); + return (0); +} + +void +spl_zone_fini(void) +{ + zone_datasets_t *zds; + zone_dataset_t *zd; + + /* + * It would be better to assert an empty zone_datasets, but since + * there's no automatic mechanism for cleaning them up if the user + * namespace is destroyed, just do it here, since spl is about to go + * out of context. + */ + while (!list_empty(&zone_datasets)) { + zds = list_entry(zone_datasets.next, zone_datasets_t, zds_list); + while (!list_empty(&zds->zds_datasets)) { + zd = list_entry(zds->zds_datasets.next, + zone_dataset_t, zd_list); + list_del(&zd->zd_list); + kmem_free(zd, sizeof (*zd) + zd->zd_dsnamelen + 1); + put_user_ns(zds->zds_userns); + } + list_del(&zds->zds_list); + kmem_free(zds, sizeof (*zds)); + } + mutex_destroy(&zone_datasets_lock); +} diff --git a/module/os/linux/zfs/Makefile.in b/module/os/linux/zfs/Makefile.in deleted file mode 100644 index fa990776db83..000000000000 --- a/module/os/linux/zfs/Makefile.in +++ /dev/null @@ -1,38 +0,0 @@ -# -# Linux specific sources included from module/zfs/Makefile.in -# - -# Suppress unused-value warnings in sparc64 architecture headers -ccflags-$(CONFIG_SPARC64) += -Wno-unused-value - -$(MODULE)-objs += ../os/linux/zfs/abd_os.o -$(MODULE)-objs += ../os/linux/zfs/arc_os.o -$(MODULE)-objs += ../os/linux/zfs/mmp_os.o -$(MODULE)-objs += ../os/linux/zfs/policy.o -$(MODULE)-objs += ../os/linux/zfs/trace.o -$(MODULE)-objs += ../os/linux/zfs/qat.o -$(MODULE)-objs += ../os/linux/zfs/qat_compress.o -$(MODULE)-objs += ../os/linux/zfs/qat_crypt.o -$(MODULE)-objs += ../os/linux/zfs/spa_misc_os.o -$(MODULE)-objs += ../os/linux/zfs/vdev_disk.o -$(MODULE)-objs += ../os/linux/zfs/vdev_file.o -$(MODULE)-objs += ../os/linux/zfs/zfs_acl.o -$(MODULE)-objs += ../os/linux/zfs/zfs_ctldir.o -$(MODULE)-objs += ../os/linux/zfs/zfs_debug.o -$(MODULE)-objs += ../os/linux/zfs/zfs_dir.o -$(MODULE)-objs += ../os/linux/zfs/zfs_file_os.o -$(MODULE)-objs += ../os/linux/zfs/zfs_ioctl_os.o -$(MODULE)-objs += ../os/linux/zfs/zfs_racct.o -$(MODULE)-objs += ../os/linux/zfs/zfs_sysfs.o -$(MODULE)-objs += ../os/linux/zfs/zfs_uio.o -$(MODULE)-objs += ../os/linux/zfs/zfs_vfsops.o -$(MODULE)-objs += ../os/linux/zfs/zfs_vnops_os.o -$(MODULE)-objs += ../os/linux/zfs/zfs_znode.o -$(MODULE)-objs += ../os/linux/zfs/zio_crypt.o -$(MODULE)-objs += ../os/linux/zfs/zpl_ctldir.o -$(MODULE)-objs += ../os/linux/zfs/zpl_export.o -$(MODULE)-objs += ../os/linux/zfs/zpl_file.o -$(MODULE)-objs += ../os/linux/zfs/zpl_inode.o -$(MODULE)-objs += ../os/linux/zfs/zpl_super.o -$(MODULE)-objs += ../os/linux/zfs/zpl_xattr.o -$(MODULE)-objs += ../os/linux/zfs/zvol_os.o diff --git a/module/os/linux/zfs/abd_os.c b/module/os/linux/zfs/abd_os.c index 113aee58599f..2ab85f8cccd0 100644 --- a/module/os/linux/zfs/abd_os.c +++ b/module/os/linux/zfs/abd_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -182,8 +182,11 @@ abd_t *abd_zero_scatter = NULL; struct page; /* - * abd_zero_page we will be an allocated zero'd PAGESIZE buffer, which is - * assigned to set each of the pages of abd_zero_scatter. + * _KERNEL - Will point to ZERO_PAGE if it is available or it will be + * an allocated zero'd PAGESIZE buffer. + * Userspace - Will be an allocated zero'ed PAGESIZE buffer. + * + * abd_zero_page is assigned to each of the pages of abd_zero_scatter. */ static struct page *abd_zero_page = NULL; @@ -466,15 +469,19 @@ abd_alloc_zero_scatter(void) struct scatterlist *sg = NULL; struct sg_table table; gfp_t gfp = __GFP_NOWARN | GFP_NOIO; - gfp_t gfp_zero_page = gfp | __GFP_ZERO; int nr_pages = abd_chunkcnt_for_bytes(SPA_MAXBLOCKSIZE); int i = 0; +#if defined(HAVE_ZERO_PAGE_GPL_ONLY) + gfp_t gfp_zero_page = gfp | __GFP_ZERO; while ((abd_zero_page = __page_cache_alloc(gfp_zero_page)) == NULL) { ABDSTAT_BUMP(abdstat_scatter_page_alloc_retry); schedule_timeout_interruptible(1); } abd_mark_zfs_page(abd_zero_page); +#else + abd_zero_page = ZERO_PAGE(0); +#endif /* HAVE_ZERO_PAGE_GPL_ONLY */ while (sg_alloc_table(&table, nr_pages, gfp)) { ABDSTAT_BUMP(abdstat_scatter_sg_table_retry); @@ -613,7 +620,6 @@ abd_alloc_zero_scatter(void) ABD_SCATTER(abd_zero_scatter).abd_offset = 0; ABD_SCATTER(abd_zero_scatter).abd_nents = nr_pages; abd_zero_scatter->abd_size = SPA_MAXBLOCKSIZE; - zfs_refcount_create(&abd_zero_scatter->abd_children); ABD_SCATTER(abd_zero_scatter).abd_sgl = vmem_alloc(nr_pages * sizeof (struct scatterlist), KM_SLEEP); @@ -695,8 +701,10 @@ abd_free_zero_scatter(void) abd_zero_scatter = NULL; ASSERT3P(abd_zero_page, !=, NULL); #if defined(_KERNEL) +#if defined(HAVE_ZERO_PAGE_GPL_ONLY) abd_unmark_zfs_page(abd_zero_page); __free_page(abd_zero_page); +#endif /* HAVE_ZERO_PAGE_GPL_ONLY */ #else umem_free(abd_zero_page, PAGESIZE); #endif /* _KERNEL */ diff --git a/module/os/linux/zfs/arc_os.c b/module/os/linux/zfs/arc_os.c index a95e9c334af9..d2a176d77813 100644 --- a/module/os/linux/zfs/arc_os.c +++ b/module/os/linux/zfs/arc_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/linux/zfs/mmp_os.c b/module/os/linux/zfs/mmp_os.c index ff3ef1bf6ad9..d502127b5ba3 100644 --- a/module/os/linux/zfs/mmp_os.c +++ b/module/os/linux/zfs/mmp_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/linux/zfs/policy.c b/module/os/linux/zfs/policy.c index bbccb2e572d9..a69618978622 100644 --- a/module/os/linux/zfs/policy.c +++ b/module/os/linux/zfs/policy.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -61,7 +61,7 @@ priv_policy_ns(const cred_t *cr, int capability, int err, static int priv_policy(const cred_t *cr, int capability, int err) { - return (priv_policy_ns(cr, capability, err, NULL)); + return (priv_policy_ns(cr, capability, err, cr->user_ns)); } static int @@ -121,7 +121,7 @@ secpolicy_vnode_access2(const cred_t *cr, struct inode *ip, uid_t owner, int secpolicy_vnode_any_access(const cred_t *cr, struct inode *ip, uid_t owner) { - if (crgetfsuid(cr) == owner) + if (crgetuid(cr) == owner) return (0); if (zpl_inode_owner_or_capable(kcred->user_ns, ip)) @@ -147,7 +147,7 @@ secpolicy_vnode_any_access(const cred_t *cr, struct inode *ip, uid_t owner) int secpolicy_vnode_chown(const cred_t *cr, uid_t owner) { - if (crgetfsuid(cr) == owner) + if (crgetuid(cr) == owner) return (0); #if defined(CONFIG_USER_NS) @@ -184,7 +184,7 @@ secpolicy_vnode_remove(const cred_t *cr) int secpolicy_vnode_setdac(const cred_t *cr, uid_t owner) { - if (crgetfsuid(cr) == owner) + if (crgetuid(cr) == owner) return (0); #if defined(CONFIG_USER_NS) @@ -220,7 +220,7 @@ secpolicy_vnode_setids_setgids(const cred_t *cr, gid_t gid) if (!kgid_has_mapping(cr->user_ns, SGID_TO_KGID(gid))) return (EPERM); #endif - if (crgetfsgid(cr) != gid && !groupmember(gid, cr)) + if (crgetgid(cr) != gid && !groupmember(gid, cr)) return (priv_policy_user(cr, CAP_FSETID, EPERM)); return (0); @@ -286,7 +286,7 @@ secpolicy_setid_clear(vattr_t *vap, cred_t *cr) static int secpolicy_vnode_setid_modify(const cred_t *cr, uid_t owner) { - if (crgetfsuid(cr) == owner) + if (crgetuid(cr) == owner) return (0); #if defined(CONFIG_USER_NS) diff --git a/module/os/linux/zfs/qat.c b/module/os/linux/zfs/qat.c index 08613b3a2042..07e0cafabb0e 100644 --- a/module/os/linux/zfs/qat.c +++ b/module/os/linux/zfs/qat.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/linux/zfs/qat_compress.c b/module/os/linux/zfs/qat_compress.c index 1d099c95bc7c..7088f6bd1c6f 100644 --- a/module/os/linux/zfs/qat_compress.c +++ b/module/os/linux/zfs/qat_compress.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/linux/zfs/qat_crypt.c b/module/os/linux/zfs/qat_crypt.c index 4771b2f3bec5..0523a23c61e1 100644 --- a/module/os/linux/zfs/qat_crypt.c +++ b/module/os/linux/zfs/qat_crypt.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -367,7 +367,7 @@ qat_crypt(qat_encrypt_dir_t dir, uint8_t *src_buf, uint8_t *dst_buf, aad_len); if (status != CPA_STATUS_SUCCESS) goto fail; - bcopy(aad_buf, op_data.pAdditionalAuthData, aad_len); + memcpy(op_data.pAdditionalAuthData, aad_buf, aad_len); } bytes_left = enc_len; @@ -413,10 +413,10 @@ qat_crypt(qat_encrypt_dir_t dir, uint8_t *src_buf, uint8_t *dst_buf, op_data.messageLenToHashInBytes = 0; op_data.messageLenToCipherInBytes = enc_len; op_data.ivLenInBytes = ZIO_DATA_IV_LEN; - bcopy(iv_buf, op_data.pIv, ZIO_DATA_IV_LEN); + memcpy(op_data.pIv, iv_buf, ZIO_DATA_IV_LEN); /* if dir is QAT_DECRYPT, copy digest_buf to pDigestResult */ if (dir == QAT_DECRYPT) - bcopy(digest_buf, op_data.pDigestResult, ZIO_DATA_MAC_LEN); + memcpy(op_data.pDigestResult, digest_buf, ZIO_DATA_MAC_LEN); cb.verify_result = CPA_FALSE; init_completion(&cb.complete); @@ -435,7 +435,7 @@ qat_crypt(qat_encrypt_dir_t dir, uint8_t *src_buf, uint8_t *dst_buf, if (dir == QAT_ENCRYPT) { /* if dir is QAT_ENCRYPT, save pDigestResult to digest_buf */ - bcopy(op_data.pDigestResult, digest_buf, ZIO_DATA_MAC_LEN); + memcpy(digest_buf, op_data.pDigestResult, ZIO_DATA_MAC_LEN); QAT_STAT_INCR(encrypt_total_out_bytes, enc_len); } else { QAT_STAT_INCR(decrypt_total_out_bytes, enc_len); @@ -557,7 +557,7 @@ qat_checksum(uint64_t cksum, uint8_t *buf, uint64_t size, zio_cksum_t *zcp) goto fail; } - bcopy(digest_buffer, zcp, sizeof (zio_cksum_t)); + memcpy(zcp, digest_buffer, sizeof (zio_cksum_t)); fail: if (status != CPA_STATUS_SUCCESS) diff --git a/module/os/linux/zfs/spa_misc_os.c b/module/os/linux/zfs/spa_misc_os.c index 5672cd6d5c5e..f999df3b7db9 100644 --- a/module/os/linux/zfs/spa_misc_os.c +++ b/module/os/linux/zfs/spa_misc_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -108,3 +108,27 @@ spa_history_zone(void) { return ("linux"); } + +void +spa_import_os(spa_t *spa) +{ + (void) spa; +} + +void +spa_export_os(spa_t *spa) +{ + (void) spa; +} + +void +spa_activate_os(spa_t *spa) +{ + (void) spa; +} + +void +spa_deactivate_os(spa_t *spa) +{ + (void) spa; +} diff --git a/module/os/linux/zfs/trace.c b/module/os/linux/zfs/trace.c index a690822ae14c..32a188d169e3 100644 --- a/module/os/linux/zfs/trace.c +++ b/module/os/linux/zfs/trace.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/linux/zfs/vdev_disk.c b/module/os/linux/zfs/vdev_disk.c index 61518bad0b1c..e720d2e9045c 100644 --- a/module/os/linux/zfs/vdev_disk.c +++ b/module/os/linux/zfs/vdev_disk.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -304,8 +304,6 @@ vdev_disk_open(vdev_t *v, uint64_t *psize, uint64_t *max_psize, rw_exit(&vd->vd_lock); } - struct request_queue *q = bdev_get_queue(vd->vd_bdev); - /* Determine the physical block size */ int physical_block_size = bdev_physical_block_size(vd->vd_bdev); @@ -316,13 +314,13 @@ vdev_disk_open(vdev_t *v, uint64_t *psize, uint64_t *max_psize, v->vdev_nowritecache = B_FALSE; /* Set when device reports it supports TRIM. */ - v->vdev_has_trim = !!blk_queue_discard(q); + v->vdev_has_trim = bdev_discard_supported(vd->vd_bdev); /* Set when device reports it supports secure TRIM. */ - v->vdev_has_securetrim = !!blk_queue_discard_secure(q); + v->vdev_has_securetrim = bdev_secure_discard_supported(vd->vd_bdev); /* Inform the ZIO pipeline that we are non-rotational */ - v->vdev_nonrot = blk_queue_nonrot(q); + v->vdev_nonrot = blk_queue_nonrot(bdev_get_queue(vd->vd_bdev)); /* Physical volume size in bytes for the partition */ *psize = bdev_capacity(vd->vd_bdev); @@ -460,6 +458,13 @@ vdev_submit_bio_impl(struct bio *bio) #define preempt_schedule_notrace(x) preempt_schedule(x) #endif +/* + * As for the Linux 5.18 kernel bio_alloc() expects a block_device struct + * as an argument removing the need to set it with bio_set_dev(). This + * removes the need for all of the following compatibility code. + */ +#if !defined(HAVE_BIO_ALLOC_4ARG) + #ifdef HAVE_BIO_SET_DEV #if defined(CONFIG_BLK_CGROUP) && defined(HAVE_BIO_SET_DEV_GPL_ONLY) /* @@ -467,8 +472,11 @@ vdev_submit_bio_impl(struct bio *bio) * blkg_tryget() to use rcu_read_lock() instead of rcu_read_lock_sched(). * As a side effect the function was converted to GPL-only. Define our * own version when needed which uses rcu_read_lock_sched(). + * + * The Linux 5.17 kernel split linux/blk-cgroup.h into a private and a public + * part, moving blkg_tryget into the private one. Define our own version. */ -#if defined(HAVE_BLKG_TRYGET_GPL_ONLY) +#if defined(HAVE_BLKG_TRYGET_GPL_ONLY) || !defined(HAVE_BLKG_TRYGET) static inline bool vdev_blkg_tryget(struct blkcg_gq *blkg) { @@ -493,7 +501,7 @@ vdev_blkg_tryget(struct blkcg_gq *blkg) return (rc); } -#elif defined(HAVE_BLKG_TRYGET) +#else #define vdev_blkg_tryget(bg) blkg_tryget(bg) #endif #ifdef HAVE_BIO_SET_DEV_MACRO @@ -553,6 +561,7 @@ bio_set_dev(struct bio *bio, struct block_device *bdev) bio->bi_bdev = bdev; } #endif /* HAVE_BIO_SET_DEV */ +#endif /* !HAVE_BIO_ALLOC_4ARG */ static inline void vdev_submit_bio(struct bio *bio) @@ -563,6 +572,36 @@ vdev_submit_bio(struct bio *bio) current->bio_list = bio_list; } +static inline struct bio * +vdev_bio_alloc(struct block_device *bdev, gfp_t gfp_mask, + unsigned short nr_vecs) +{ + struct bio *bio; + +#ifdef HAVE_BIO_ALLOC_4ARG + bio = bio_alloc(bdev, nr_vecs, 0, gfp_mask); +#else + bio = bio_alloc(gfp_mask, nr_vecs); + if (likely(bio != NULL)) + bio_set_dev(bio, bdev); +#endif + + return (bio); +} + +static inline unsigned int +vdev_bio_max_segs(zio_t *zio, int bio_size, uint64_t abd_offset) +{ + unsigned long nr_segs = abd_nr_pages_off(zio->io_abd, + bio_size, abd_offset); + +#ifdef HAVE_BIO_MAX_SEGS + return (bio_max_segs(nr_segs)); +#else + return (MIN(nr_segs, BIO_MAX_PAGES)); +#endif +} + static int __vdev_disk_physio(struct block_device *bdev, zio_t *zio, size_t io_size, uint64_t io_offset, int rw, int flags) @@ -574,6 +613,7 @@ __vdev_disk_physio(struct block_device *bdev, zio_t *zio, int bio_count = 16; int error = 0; struct blk_plug plug; + unsigned short nr_vecs; /* * Accessing outside the block device is never allowed. @@ -625,15 +665,8 @@ __vdev_disk_physio(struct block_device *bdev, zio_t *zio, goto retry; } - /* bio_alloc() with __GFP_WAIT never returns NULL */ -#ifdef HAVE_BIO_MAX_SEGS - dr->dr_bio[i] = bio_alloc(GFP_NOIO, bio_max_segs( - abd_nr_pages_off(zio->io_abd, bio_size, abd_offset))); -#else - dr->dr_bio[i] = bio_alloc(GFP_NOIO, - MIN(abd_nr_pages_off(zio->io_abd, bio_size, abd_offset), - BIO_MAX_PAGES)); -#endif + nr_vecs = vdev_bio_max_segs(zio, bio_size, abd_offset); + dr->dr_bio[i] = vdev_bio_alloc(bdev, GFP_NOIO, nr_vecs); if (unlikely(dr->dr_bio[i] == NULL)) { vdev_disk_dio_free(dr); return (SET_ERROR(ENOMEM)); @@ -642,7 +675,6 @@ __vdev_disk_physio(struct block_device *bdev, zio_t *zio, /* Matching put called by vdev_disk_physio_completion */ vdev_disk_dio_get(dr); - bio_set_dev(dr->dr_bio[i], bdev); BIO_BI_SECTOR(dr->dr_bio[i]) = bio_offset >> 9; dr->dr_bio[i]->bi_end_io = vdev_disk_physio_completion; dr->dr_bio[i]->bi_private = dr; @@ -706,14 +738,12 @@ vdev_disk_io_flush(struct block_device *bdev, zio_t *zio) if (!q) return (SET_ERROR(ENXIO)); - bio = bio_alloc(GFP_NOIO, 0); - /* bio_alloc() with __GFP_WAIT never returns NULL */ + bio = vdev_bio_alloc(bdev, GFP_NOIO, 0); if (unlikely(bio == NULL)) return (SET_ERROR(ENOMEM)); bio->bi_end_io = vdev_disk_io_flush_completion; bio->bi_private = zio; - bio_set_dev(bio, bdev); bio_set_flush(bio); vdev_submit_bio(bio); invalidate_bdev(bdev); @@ -721,12 +751,38 @@ vdev_disk_io_flush(struct block_device *bdev, zio_t *zio) return (0); } +static int +vdev_disk_io_trim(zio_t *zio) +{ + vdev_t *v = zio->io_vd; + vdev_disk_t *vd = v->vdev_tsd; + +#if defined(HAVE_BLKDEV_ISSUE_SECURE_ERASE) + if (zio->io_trim_flags & ZIO_TRIM_SECURE) { + return (-blkdev_issue_secure_erase(vd->vd_bdev, + zio->io_offset >> 9, zio->io_size >> 9, GFP_NOFS)); + } else { + return (-blkdev_issue_discard(vd->vd_bdev, + zio->io_offset >> 9, zio->io_size >> 9, GFP_NOFS)); + } +#elif defined(HAVE_BLKDEV_ISSUE_DISCARD) + unsigned long trim_flags = 0; +#if defined(BLKDEV_DISCARD_SECURE) + if (zio->io_trim_flags & ZIO_TRIM_SECURE) + trim_flags |= BLKDEV_DISCARD_SECURE; +#endif + return (-blkdev_issue_discard(vd->vd_bdev, + zio->io_offset >> 9, zio->io_size >> 9, GFP_NOFS, trim_flags)); +#else +#error "Unsupported kernel" +#endif +} + static void vdev_disk_io_start(zio_t *zio) { vdev_t *v = zio->io_vd; vdev_disk_t *vd = v->vdev_tsd; - unsigned long trim_flags = 0; int rw, error; /* @@ -799,14 +855,7 @@ vdev_disk_io_start(zio_t *zio) break; case ZIO_TYPE_TRIM: -#if defined(BLKDEV_DISCARD_SECURE) - if (zio->io_trim_flags & ZIO_TRIM_SECURE) - trim_flags |= BLKDEV_DISCARD_SECURE; -#endif - zio->io_error = -blkdev_issue_discard(vd->vd_bdev, - zio->io_offset >> 9, zio->io_size >> 9, GFP_NOFS, - trim_flags); - + zio->io_error = vdev_disk_io_trim(zio); rw_exit(&vd->vd_lock); zio_interrupt(zio); return; diff --git a/module/os/linux/zfs/vdev_file.c b/module/os/linux/zfs/vdev_file.c index f073145326e3..46e412f6eeb4 100644 --- a/module/os/linux/zfs/vdev_file.c +++ b/module/os/linux/zfs/vdev_file.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/linux/zfs/zfs_acl.c b/module/os/linux/zfs/zfs_acl.c index 9f193e06ffd7..a139ee12c4d8 100644 --- a/module/os/linux/zfs/zfs_acl.c +++ b/module/os/linux/zfs/zfs_acl.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -144,10 +144,10 @@ zfs_ace_v0_set_who(void *acep, uint64_t who) ((zfs_oldace_t *)acep)->z_fuid = who; } -/*ARGSUSED*/ static size_t zfs_ace_v0_size(void *acep) { + (void) acep; return (sizeof (zfs_oldace_t)); } @@ -163,10 +163,10 @@ zfs_ace_v0_mask_off(void) return (offsetof(zfs_oldace_t, z_access_mask)); } -/*ARGSUSED*/ static int zfs_ace_v0_data(void *acep, void **datap) { + (void) acep; *datap = NULL; return (0); } @@ -629,11 +629,11 @@ zfs_acl_next_ace(zfs_acl_t *aclp, void *start, uint64_t *who, return (NULL); } -/*ARGSUSED*/ static uint64_t zfs_ace_walk(void *datap, uint64_t cookie, int aclcnt, uint16_t *flags, uint16_t *type, uint32_t *mask) { + (void) aclcnt; zfs_acl_t *aclp = datap; zfs_ace_hdr_t *acep = (zfs_ace_hdr_t *)(uintptr_t)cookie; uint64_t who; @@ -687,10 +687,10 @@ zfs_copy_ace_2_fuid(zfsvfs_t *zfsvfs, umode_t obj_mode, zfs_acl_t *aclp, zobjacep = (zfs_object_ace_t *)aceptr; aceobjp = (ace_object_t *)acep; - bcopy(aceobjp->a_obj_type, zobjacep->z_object_type, + memcpy(zobjacep->z_object_type, aceobjp->a_obj_type, sizeof (aceobjp->a_obj_type)); - bcopy(aceobjp->a_inherit_obj_type, - zobjacep->z_inherit_type, + memcpy(zobjacep->z_inherit_type, + aceobjp->a_inherit_obj_type, sizeof (aceobjp->a_inherit_obj_type)); acep = (ace_t *)((caddr_t)acep + sizeof (ace_object_t)); break; @@ -737,11 +737,11 @@ zfs_copy_fuid_2_ace(zfsvfs_t *zfsvfs, zfs_acl_t *aclp, cred_t *cr, } zobjacep = (zfs_object_ace_t *)zacep; objacep = (ace_object_t *)acep; - bcopy(zobjacep->z_object_type, - objacep->a_obj_type, + memcpy(objacep->a_obj_type, + zobjacep->z_object_type, sizeof (zobjacep->z_object_type)); - bcopy(zobjacep->z_inherit_type, - objacep->a_inherit_obj_type, + memcpy(objacep->a_inherit_obj_type, + zobjacep->z_inherit_type, sizeof (zobjacep->z_inherit_type)); ace_size = sizeof (ace_object_t); break; @@ -863,6 +863,26 @@ zfs_unix_to_v4(uint32_t access_mask) return (new_mask); } + +static int +zfs_v4_to_unix(uint32_t access_mask, int *unmapped) +{ + int new_mask = 0; + + *unmapped = access_mask & + (ACE_WRITE_OWNER | ACE_WRITE_ACL | ACE_DELETE); + + if (access_mask & WRITE_MASK) + new_mask |= S_IWOTH; + if (access_mask & ACE_READ_DATA) + new_mask |= S_IROTH; + if (access_mask & ACE_EXECUTE) + new_mask |= S_IXOTH; + + return (new_mask); +} + + static void zfs_set_ace(zfs_acl_t *aclp, void *acep, uint32_t access_mask, uint16_t access_type, uint64_t fuid, uint16_t entry_type) @@ -1102,7 +1122,7 @@ zfs_acl_node_read(struct znode *zp, boolean_t have_lock, zfs_acl_t **aclpp, znode_acl.z_acl_extern_obj, 0, aclnode->z_size, aclnode->z_acldata, DMU_READ_PREFETCH); } else { - bcopy(znode_acl.z_ace_data, aclnode->z_acldata, + memcpy(aclnode->z_acldata, znode_acl.z_ace_data, aclnode->z_size); } } else { @@ -1130,11 +1150,11 @@ zfs_acl_node_read(struct znode *zp, boolean_t have_lock, zfs_acl_t **aclpp, return (error); } -/*ARGSUSED*/ void zfs_acl_data_locator(void **dataptr, uint32_t *length, uint32_t buflen, boolean_t start, void *userdata) { + (void) buflen; zfs_acl_locator_cb_t *cb = (zfs_acl_locator_cb_t *)userdata; if (start) { @@ -1447,7 +1467,7 @@ zfs_aclset_common(znode_t *zp, zfs_acl_t *aclp, cred_t *cr, dmu_tx_t *tx) aclnode = list_next(&aclp->z_acl, aclnode)) { if (aclnode->z_ace_count == 0) continue; - bcopy(aclnode->z_acldata, start, + memcpy(start, aclnode->z_acldata, aclnode->z_size); start = (caddr_t)start + aclnode->z_size; } @@ -1727,7 +1747,7 @@ zfs_acl_inherit(zfsvfs_t *zfsvfs, umode_t va_mode, zfs_acl_t *paclp, if ((data1sz = paclp->z_ops->ace_data(pacep, &data1)) != 0) { VERIFY((data2sz = aclp->z_ops->ace_data(acep, &data2)) == data1sz); - bcopy(data1, data2, data2sz); + memcpy(data2, data1, data2sz); } aclp->z_acl_count++; @@ -1791,7 +1811,7 @@ zfs_acl_ids_create(znode_t *dzp, int flag, vattr_t *vap, cred_t *cr, boolean_t trim = B_FALSE; boolean_t inherited = B_FALSE; - bzero(acl_ids, sizeof (zfs_acl_ids_t)); + memset(acl_ids, 0, sizeof (zfs_acl_ids_t)); acl_ids->z_mode = vap->va_mode; if (vsecp) @@ -2016,7 +2036,7 @@ zfs_getacl(znode_t *zp, vsecattr_t *vsecp, boolean_t skipaclchk, cred_t *cr) for (aclnode = list_head(&aclp->z_acl); aclnode; aclnode = list_next(&aclp->z_acl, aclnode)) { - bcopy(aclnode->z_acldata, start, + memcpy(start, aclnode->z_acldata, aclnode->z_size); start = (caddr_t)start + aclnode->z_size; } @@ -2399,6 +2419,53 @@ zfs_has_access(znode_t *zp, cred_t *cr) return (B_TRUE); } +/* + * Simplified access check for case where ACL is known to not contain + * information beyond what is defined in the mode. In this case, we + * can pass along to the kernel / vfs generic_permission() check, which + * evaluates the mode and POSIX ACL. + * + * NFSv4 ACLs allow granting permissions that are usually relegated only + * to the file owner or superuser. Examples are ACE_WRITE_OWNER (chown), + * ACE_WRITE_ACL(chmod), and ACE_DELETE. ACE_DELETE requests must fail + * because with conventional posix permissions, right to delete file + * is determined by write bit on the parent dir. + * + * If unmappable perms are requested, then we must return EPERM + * and include those bits in the working_mode so that the caller of + * zfs_zaccess_common() can decide whether to perform additional + * policy / capability checks. EACCES is used in zfs_zaccess_aces_check() + * to indicate access check failed due to explicit DENY entry, and so + * we want to avoid that here. + */ +static int +zfs_zaccess_trivial(znode_t *zp, uint32_t *working_mode, cred_t *cr) +{ + int err, mask; + int unmapped = 0; + + ASSERT(zp->z_pflags & ZFS_ACL_TRIVIAL); + + mask = zfs_v4_to_unix(*working_mode, &unmapped); + if (mask == 0 || unmapped) { + *working_mode = unmapped; + return (unmapped ? SET_ERROR(EPERM) : 0); + } + +#if defined(HAVE_IOPS_PERMISSION_USERNS) + err = generic_permission(cr->user_ns, ZTOI(zp), mask); +#else + err = generic_permission(ZTOI(zp), mask); +#endif + if (err != 0) { + return (SET_ERROR(EPERM)); + } + + *working_mode = unmapped; + + return (0); +} + static int zfs_zaccess_common(znode_t *zp, uint32_t v4_mode, uint32_t *working_mode, boolean_t *check_privs, boolean_t skipaclchk, cred_t *cr) @@ -2450,6 +2517,9 @@ zfs_zaccess_common(znode_t *zp, uint32_t v4_mode, uint32_t *working_mode, return (SET_ERROR(EPERM)); } + if (zp->z_pflags & ZFS_ACL_TRIVIAL) + return (zfs_zaccess_trivial(zp, working_mode, cr)); + return (zfs_zaccess_aces_check(zp, working_mode, B_FALSE, cr)); } diff --git a/module/os/linux/zfs/zfs_ctldir.c b/module/os/linux/zfs/zfs_ctldir.c index f7e71461a3bd..32342d25ce6f 100644 --- a/module/os/linux/zfs/zfs_ctldir.c +++ b/module/os/linux/zfs/zfs_ctldir.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -162,7 +162,7 @@ zfsctl_snapshot_free(zfs_snapentry_t *se) zfs_refcount_destroy(&se->se_refcount); kmem_strfree(se->se_name); kmem_strfree(se->se_path); - rw_destroy(se->se_taskqid_lock); + rw_destroy(&se->se_taskqid_lock); kmem_free(se, sizeof (zfs_snapentry_t)); } @@ -496,6 +496,8 @@ zfsctl_inode_alloc(zfsvfs_t *zfsvfs, uint64_t id, zp->z_pflags = 0; zp->z_mode = 0; zp->z_sync_cnt = 0; + zp->z_sync_writes_cnt = 0; + zp->z_async_writes_cnt = 0; ip->i_generation = 0; ip->i_ino = id; ip->i_mode = (S_IFDIR | S_IRWXUGO); diff --git a/module/os/linux/zfs/zfs_debug.c b/module/os/linux/zfs/zfs_debug.c index be65f0a2e245..2a4e3f378948 100644 --- a/module/os/linux/zfs/zfs_debug.c +++ b/module/os/linux/zfs/zfs_debug.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/linux/zfs/zfs_dir.c b/module/os/linux/zfs/zfs_dir.c index 82b32d1cc3fa..6738d237b923 100644 --- a/module/os/linux/zfs/zfs_dir.c +++ b/module/os/linux/zfs/zfs_dir.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -297,7 +297,7 @@ zfs_dirent_lock(zfs_dirlock_t **dlpp, znode_t *dzp, char *name, */ dl->dl_namesize = strlen(dl->dl_name) + 1; name = kmem_alloc(dl->dl_namesize, KM_SLEEP); - bcopy(dl->dl_name, name, dl->dl_namesize); + memcpy(name, dl->dl_name, dl->dl_namesize); dl->dl_name = name; } @@ -625,7 +625,7 @@ zfs_purgedir(znode_t *dzp) skipped += 1; continue; } - bzero(&dl, sizeof (dl)); + memset(&dl, 0, sizeof (dl)); dl.dl_dzp = dzp; dl.dl_name = zap.za_name; diff --git a/module/os/linux/zfs/zfs_file_os.c b/module/os/linux/zfs/zfs_file_os.c index e12f7c3ced43..da80428402cd 100644 --- a/module/os/linux/zfs/zfs_file_os.c +++ b/module/os/linux/zfs/zfs_file_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/linux/zfs/zfs_ioctl_os.c b/module/os/linux/zfs/zfs_ioctl_os.c index fee3fe540b90..f8d1777b07a6 100644 --- a/module/os/linux/zfs/zfs_ioctl_os.c +++ b/module/os/linux/zfs/zfs_ioctl_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -37,6 +37,7 @@ * Copyright 2017 RackTop Systems. * Copyright (c) 2017 Open-E, Inc. All Rights Reserved. * Copyright (c) 2019 Datto Inc. + * Copyright (c) 2021 Klara, Inc. */ #include @@ -58,6 +59,8 @@ #include #include #include +#include +#include #include @@ -148,6 +151,48 @@ zfsdev_ioctl(struct file *filp, unsigned cmd, unsigned long arg) } +static int +zfs_ioc_userns_attach(zfs_cmd_t *zc) +{ + int error; + + if (zc == NULL) + return (SET_ERROR(EINVAL)); + + error = zone_dataset_attach(CRED(), zc->zc_name, zc->zc_cleanup_fd); + + /* + * Translate ENOTTY to ZFS_ERR_NOT_USER_NAMESPACE as we just arrived + * back from the SPL layer, which does not know about ZFS_ERR_* errors. + * See the comment at the user_ns_get() function in spl-zone.c for + * details. + */ + if (error == ENOTTY) + error = ZFS_ERR_NOT_USER_NAMESPACE; + + return (error); +} + +static int +zfs_ioc_userns_detach(zfs_cmd_t *zc) +{ + int error; + + if (zc == NULL) + return (SET_ERROR(EINVAL)); + + error = zone_dataset_detach(CRED(), zc->zc_name, zc->zc_cleanup_fd); + + /* + * See the comment in zfs_ioc_userns_attach() for details on what is + * going on here. + */ + if (error == ENOTTY) + error = ZFS_ERR_NOT_USER_NAMESPACE; + + return (error); +} + uint64_t zfs_max_nvlist_src_size_os(void) { @@ -166,6 +211,10 @@ zfs_ioctl_update_mount_cache(const char *dsname) void zfs_ioctl_init_os(void) { + zfs_ioctl_register_dataset_nolog(ZFS_IOC_USERNS_ATTACH, + zfs_ioc_userns_attach, zfs_secpolicy_config, POOL_CHECK_NONE); + zfs_ioctl_register_dataset_nolog(ZFS_IOC_USERNS_DETACH, + zfs_ioc_userns_detach, zfs_secpolicy_config, POOL_CHECK_NONE); } #ifdef CONFIG_COMPAT @@ -233,8 +282,8 @@ zfsdev_detach(void) #define ZFS_DEBUG_STR "" #endif -static int __init -openzfs_init(void) +static int +openzfs_init_os(void) { int error; @@ -259,8 +308,8 @@ openzfs_init(void) return (0); } -static void __exit -openzfs_fini(void) +static void +openzfs_fini_os(void) { zfs_sysfs_fini(); zfs_kmod_fini(); @@ -269,12 +318,58 @@ openzfs_fini(void) ZFS_META_VERSION, ZFS_META_RELEASE, ZFS_DEBUG_STR); } + +extern int __init zcommon_init(void); +extern void zcommon_fini(void); + +static int __init +openzfs_init(void) +{ + int err; + if ((err = zcommon_init()) != 0) + goto zcommon_failed; + if ((err = icp_init()) != 0) + goto icp_failed; + if ((err = zstd_init()) != 0) + goto zstd_failed; + if ((err = openzfs_init_os()) != 0) + goto openzfs_os_failed; + return (0); + +openzfs_os_failed: + zstd_fini(); +zstd_failed: + icp_fini(); +icp_failed: + zcommon_fini(); +zcommon_failed: + return (err); +} + +static void __exit +openzfs_fini(void) +{ + openzfs_fini_os(); + zstd_fini(); + icp_fini(); + zcommon_fini(); +} + #if defined(_KERNEL) module_init(openzfs_init); module_exit(openzfs_fini); #endif -ZFS_MODULE_DESCRIPTION("ZFS"); -ZFS_MODULE_AUTHOR(ZFS_META_AUTHOR); -ZFS_MODULE_LICENSE(ZFS_META_LICENSE); -ZFS_MODULE_VERSION(ZFS_META_VERSION "-" ZFS_META_RELEASE); +MODULE_ALIAS("zavl"); +MODULE_ALIAS("icp"); +MODULE_ALIAS("zlua"); +MODULE_ALIAS("znvpair"); +MODULE_ALIAS("zunicode"); +MODULE_ALIAS("zcommon"); +MODULE_ALIAS("zzstd"); +MODULE_DESCRIPTION("ZFS"); +MODULE_AUTHOR(ZFS_META_AUTHOR); +MODULE_LICENSE("Dual MIT/GPL"); /* lua */ +MODULE_LICENSE("Dual BSD/GPL"); /* zstd / misc */ +MODULE_LICENSE(ZFS_META_LICENSE); +MODULE_VERSION(ZFS_META_VERSION "-" ZFS_META_RELEASE); diff --git a/module/os/linux/zfs/zfs_sysfs.c b/module/os/linux/zfs/zfs_sysfs.c index 6f71382cf74e..4a2091f3c396 100644 --- a/module/os/linux/zfs/zfs_sysfs.c +++ b/module/os/linux/zfs/zfs_sysfs.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -65,16 +65,15 @@ /* * A zfs_mod_kobj_t represents a zfs kobject under '/sys/module/zfs' */ -struct zfs_mod_kobj; typedef struct zfs_mod_kobj zfs_mod_kobj_t; - struct zfs_mod_kobj { struct kobject zko_kobj; struct kobj_type zko_kobj_type; struct sysfs_ops zko_sysfs_ops; size_t zko_attr_count; struct attribute *zko_attr_list; /* allocated */ - struct attribute **zko_default_attrs; /* allocated */ + struct attribute_group zko_default_group; /* .attrs allocated */ + const struct attribute_group *zko_default_groups[2]; size_t zko_child_count; zfs_mod_kobj_t *zko_children; /* allocated */ }; @@ -127,10 +126,10 @@ zfs_kobj_release(struct kobject *kobj) zkobj->zko_attr_list = NULL; } - if (zkobj->zko_default_attrs != NULL) { - kmem_free(zkobj->zko_default_attrs, + if (zkobj->zko_default_group.attrs != NULL) { + kmem_free(zkobj->zko_default_group.attrs, DEFAULT_ATTR_SIZE(zkobj->zko_attr_count)); - zkobj->zko_default_attrs = NULL; + zkobj->zko_default_group.attrs = NULL; } if (zkobj->zko_child_count != 0) { @@ -154,11 +153,12 @@ zfs_kobj_add_attr(zfs_mod_kobj_t *zkobj, int attr_num, const char *attr_name) { VERIFY3U(attr_num, <, zkobj->zko_attr_count); ASSERT(zkobj->zko_attr_list); - ASSERT(zkobj->zko_default_attrs); + ASSERT(zkobj->zko_default_group.attrs); zkobj->zko_attr_list[attr_num].name = attr_name; zkobj->zko_attr_list[attr_num].mode = 0444; - zkobj->zko_default_attrs[attr_num] = &zkobj->zko_attr_list[attr_num]; + zkobj->zko_default_group.attrs[attr_num] = + &zkobj->zko_attr_list[attr_num]; sysfs_attr_init(&zkobj->zko_attr_list[attr_num]); } @@ -176,9 +176,9 @@ zfs_kobj_init(zfs_mod_kobj_t *zkobj, int attr_cnt, int child_cnt, return (ENOMEM); } /* this will always have at least one slot for NULL termination */ - zkobj->zko_default_attrs = kmem_zalloc(DEFAULT_ATTR_SIZE(attr_cnt), - KM_SLEEP); - if (zkobj->zko_default_attrs == NULL) { + zkobj->zko_default_group.attrs = + kmem_zalloc(DEFAULT_ATTR_SIZE(attr_cnt), KM_SLEEP); + if (zkobj->zko_default_group.attrs == NULL) { if (zkobj->zko_attr_list != NULL) { kmem_free(zkobj->zko_attr_list, ATTR_TABLE_SIZE(attr_cnt)); @@ -186,14 +186,19 @@ zfs_kobj_init(zfs_mod_kobj_t *zkobj, int attr_cnt, int child_cnt, return (ENOMEM); } zkobj->zko_attr_count = attr_cnt; - zkobj->zko_kobj_type.default_attrs = zkobj->zko_default_attrs; + zkobj->zko_default_groups[0] = &zkobj->zko_default_group; +#ifdef HAVE_SYSFS_DEFAULT_GROUPS + zkobj->zko_kobj_type.default_groups = zkobj->zko_default_groups; +#else + zkobj->zko_kobj_type.default_attrs = zkobj->zko_default_group.attrs; +#endif if (child_cnt > 0) { zkobj->zko_children = kmem_zalloc(CHILD_TABLE_SIZE(child_cnt), KM_SLEEP); if (zkobj->zko_children == NULL) { - if (zkobj->zko_default_attrs != NULL) { - kmem_free(zkobj->zko_default_attrs, + if (zkobj->zko_default_group.attrs != NULL) { + kmem_free(zkobj->zko_default_group.attrs, DEFAULT_ATTR_SIZE(attr_cnt)); } if (zkobj->zko_attr_list != NULL) { @@ -215,9 +220,9 @@ zfs_kobj_init(zfs_mod_kobj_t *zkobj, int attr_cnt, int child_cnt, static int zfs_kobj_add(zfs_mod_kobj_t *zkobj, struct kobject *parent, const char *name) { - /* zko_default_attrs must be NULL terminated */ - ASSERT(zkobj->zko_default_attrs != NULL); - ASSERT(zkobj->zko_default_attrs[zkobj->zko_attr_count] == NULL); + /* zko_default_group.attrs must be NULL terminated */ + ASSERT(zkobj->zko_default_group.attrs != NULL); + ASSERT(zkobj->zko_default_group.attrs[zkobj->zko_attr_count] == NULL); kobject_init(&zkobj->zko_kobj, &zkobj->zko_kobj_type); return (kobject_add(&zkobj->zko_kobj, parent, name)); @@ -226,7 +231,7 @@ zfs_kobj_add(zfs_mod_kobj_t *zkobj, struct kobject *parent, const char *name) /* * Each zfs property has these common attributes */ -static const char *zprop_attrs[] = { +static const char *const zprop_attrs[] = { "type", "readonly", "setonce", @@ -239,7 +244,7 @@ static const char *zprop_attrs[] = { #define ZFS_PROP_ATTR_COUNT ARRAY_SIZE(zprop_attrs) #define ZPOOL_PROP_ATTR_COUNT (ZFS_PROP_ATTR_COUNT - 1) -static const char *zprop_types[] = { +static const char *const zprop_types[] = { "number", "string", "index", @@ -250,7 +255,7 @@ typedef struct zfs_type_map { const char *ztm_name; } zfs_type_map_t; -static zfs_type_map_t type_map[] = { +static const zfs_type_map_t type_map[] = { {ZFS_TYPE_FILESYSTEM, "filesystem"}, {ZFS_TYPE_SNAPSHOT, "snapshot"}, {ZFS_TYPE_VOLUME, "volume"}, @@ -371,7 +376,7 @@ pool_property_show(struct kobject *kobj, struct attribute *attr, char *buf) * A user process can easily check if the running zfs kernel module * supports the new feature. */ -static const char *zfs_kernel_features[] = { +static const char *const zfs_kernel_features[] = { /* --> Add new kernel features here */ "com.delphix:vdev_initialize", "org.zfsonlinux:vdev_trim", @@ -439,7 +444,7 @@ zfs_kernel_features_init(zfs_mod_kobj_t *zfs_kobj, struct kobject *parent) /* * Each pool feature has these common attributes */ -static const char *pool_feature_attrs[] = { +static const char *const pool_feature_attrs[] = { "description", "guid", "uname", diff --git a/module/os/linux/zfs/zfs_uio.c b/module/os/linux/zfs/zfs_uio.c index a3d5d5f83b6f..3efd4ab159c6 100644 --- a/module/os/linux/zfs/zfs_uio.c +++ b/module/os/linux/zfs/zfs_uio.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -44,7 +44,7 @@ #include #include #include -#include +#include #include #include @@ -103,9 +103,9 @@ zfs_uiomove_iov(void *p, size_t n, zfs_uio_rw_t rw, zfs_uio_t *uio) break; case UIO_SYSSPACE: if (rw == UIO_READ) - bcopy(p, iov->iov_base + skip, cnt); + memcpy(iov->iov_base + skip, p, cnt); else - bcopy(iov->iov_base + skip, p, cnt); + memcpy(p, iov->iov_base + skip, cnt); break; default: ASSERT(0); @@ -126,7 +126,7 @@ zfs_uiomove_iov(void *p, size_t n, zfs_uio_rw_t rw, zfs_uio_t *uio) } static int -zfs_uiomove_bvec(void *p, size_t n, zfs_uio_rw_t rw, zfs_uio_t *uio) +zfs_uiomove_bvec_impl(void *p, size_t n, zfs_uio_rw_t rw, zfs_uio_t *uio) { const struct bio_vec *bv = uio->uio_bvec; size_t skip = uio->uio_skip; @@ -137,10 +137,13 @@ zfs_uiomove_bvec(void *p, size_t n, zfs_uio_rw_t rw, zfs_uio_t *uio) cnt = MIN(bv->bv_len - skip, n); paddr = zfs_kmap_atomic(bv->bv_page); - if (rw == UIO_READ) - bcopy(p, paddr + bv->bv_offset + skip, cnt); - else - bcopy(paddr + bv->bv_offset + skip, p, cnt); + if (rw == UIO_READ) { + /* Copy from buffer 'p' to the bvec data */ + memcpy(paddr + bv->bv_offset + skip, p, cnt); + } else { + /* Copy from bvec data to buffer 'p' */ + memcpy(p, paddr + bv->bv_offset + skip, cnt); + } zfs_kunmap_atomic(paddr); skip += cnt; @@ -158,6 +161,141 @@ zfs_uiomove_bvec(void *p, size_t n, zfs_uio_rw_t rw, zfs_uio_t *uio) return (0); } +#ifdef HAVE_BLK_MQ +static void +zfs_copy_bvec(void *p, size_t skip, size_t cnt, zfs_uio_rw_t rw, + struct bio_vec *bv) +{ + void *paddr; + + paddr = zfs_kmap_atomic(bv->bv_page); + if (rw == UIO_READ) { + /* Copy from buffer 'p' to the bvec data */ + memcpy(paddr + bv->bv_offset + skip, p, cnt); + } else { + /* Copy from bvec data to buffer 'p' */ + memcpy(p, paddr + bv->bv_offset + skip, cnt); + } + zfs_kunmap_atomic(paddr); +} + +/* + * Copy 'n' bytes of data between the buffer p[] and the data represented + * by the request in the uio. + */ +static int +zfs_uiomove_bvec_rq(void *p, size_t n, zfs_uio_rw_t rw, zfs_uio_t *uio) +{ + struct request *rq = uio->rq; + struct bio_vec bv; + struct req_iterator iter; + size_t this_seg_start; /* logical offset */ + size_t this_seg_end; /* logical offset */ + size_t skip_in_seg; + size_t copy_from_seg; + size_t orig_loffset; + int copied = 0; + + /* + * Get the original logical offset of this entire request (because + * uio->uio_loffset will be modified over time). + */ + orig_loffset = io_offset(NULL, rq); + this_seg_start = orig_loffset; + + rq_for_each_segment(bv, rq, iter) { + if (uio->iter.bio) { + /* + * If uio->iter.bio is present, then we know we've saved + * uio->iter from a previous call to this function, and + * we can skip ahead in this rq_for_each_segment() loop + * to where we last left off. That way, we don't need + * to iterate over tons of segments we've already + * processed - we can just restore the "saved state". + */ + iter = uio->iter; + bv = uio->bv; + this_seg_start = uio->uio_loffset; + memset(&uio->iter, 0, sizeof (uio->iter)); + continue; + } + + /* + * Lookup what the logical offset of the last byte of this + * segment is. + */ + this_seg_end = this_seg_start + bv.bv_len - 1; + + /* + * We only need to operate on segments that have data we're + * copying. + */ + if (uio->uio_loffset >= this_seg_start && + uio->uio_loffset <= this_seg_end) { + /* + * Some, or all, of the data in this segment needs to be + * copied. + */ + + /* + * We may be not be copying from the first byte in the + * segment. Figure out how many bytes to skip copying + * from the beginning of this segment. + */ + skip_in_seg = uio->uio_loffset - this_seg_start; + + /* + * Calculate the total number of bytes from this + * segment that we will be copying. + */ + copy_from_seg = MIN(bv.bv_len - skip_in_seg, n); + + /* Copy the bytes */ + zfs_copy_bvec(p, skip_in_seg, copy_from_seg, rw, &bv); + p = ((char *)p) + copy_from_seg; + + n -= copy_from_seg; + uio->uio_resid -= copy_from_seg; + uio->uio_loffset += copy_from_seg; + copied = 1; /* We copied some data */ + } + + if (n == 0) { + /* + * All done copying. Save our 'iter' value to the uio. + * This allows us to "save our state" and skip ahead in + * the rq_for_each_segment() loop the next time we call + * call zfs_uiomove_bvec_rq() on this uio (which we + * will be doing for any remaining data in the uio). + */ + uio->iter = iter; /* make a copy of the struct data */ + uio->bv = bv; + return (0); + } + + this_seg_start = this_seg_end + 1; + } + + if (!copied) { + /* Didn't copy anything */ + uio->uio_resid = 0; + } + return (0); +} +#endif + +static int +zfs_uiomove_bvec(void *p, size_t n, zfs_uio_rw_t rw, zfs_uio_t *uio) +{ +#ifdef HAVE_BLK_MQ + if (uio->rq != NULL) + return (zfs_uiomove_bvec_rq(p, n, rw, uio)); +#else + ASSERT3P(uio->rq, ==, NULL); +#endif + return (zfs_uiomove_bvec_impl(p, n, rw, uio)); +} + #if defined(HAVE_VFS_IOV_ITER) static int zfs_uiomove_iter(void *p, size_t n, zfs_uio_rw_t rw, zfs_uio_t *uio, @@ -248,7 +386,7 @@ zfs_uio_prefaultpages(ssize_t n, zfs_uio_t *uio) /* touch each page in this segment. */ p = iov->iov_base + skip; while (cnt) { - if (get_user(tmp, (uint8_t *)p)) + if (copy_from_user(&tmp, p, 1)) return (EFAULT); ulong_t incr = MIN(cnt, PAGESIZE); p += incr; @@ -256,7 +394,7 @@ zfs_uio_prefaultpages(ssize_t n, zfs_uio_t *uio) } /* touch the last byte in case it straddles a page. */ p--; - if (get_user(tmp, (uint8_t *)p)) + if (copy_from_user(&tmp, p, 1)) return (EFAULT); } } @@ -275,7 +413,7 @@ zfs_uiocopy(void *p, size_t n, zfs_uio_rw_t rw, zfs_uio_t *uio, size_t *cbytes) zfs_uio_t uio_copy; int ret; - bcopy(uio, &uio_copy, sizeof (zfs_uio_t)); + memcpy(&uio_copy, uio, sizeof (zfs_uio_t)); if (uio->uio_segflg == UIO_BVEC) ret = zfs_uiomove_bvec(p, n, rw, &uio_copy); @@ -300,8 +438,14 @@ zfs_uioskip(zfs_uio_t *uio, size_t n) { if (n > uio->uio_resid) return; - - if (uio->uio_segflg == UIO_BVEC) { + /* + * When using a uio with a struct request, we simply + * use uio_loffset as a pointer to the next logical byte to + * copy in the request. We don't have to do any fancy + * accounting with uio_bvec/uio_iovcnt since we don't use + * them. + */ + if (uio->uio_segflg == UIO_BVEC && uio->rq == NULL) { uio->uio_skip += n; while (uio->uio_iovcnt && uio->uio_skip >= uio->uio_bvec->bv_len) { diff --git a/module/os/linux/zfs/zfs_vfsops.c b/module/os/linux/zfs/zfs_vfsops.c index acf8c7b89522..eac3dcb6a55c 100644 --- a/module/os/linux/zfs/zfs_vfsops.c +++ b/module/os/linux/zfs/zfs_vfsops.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -256,10 +256,10 @@ zfs_is_readonly(zfsvfs_t *zfsvfs) return (!!(zfsvfs->z_sb->s_flags & SB_RDONLY)); } -/*ARGSUSED*/ int zfs_sync(struct super_block *sb, int wait, cred_t *cr) { + (void) cr; zfsvfs_t *zfsvfs = sb->s_fs_info; /* @@ -848,8 +848,6 @@ zfsvfs_setup(zfsvfs_t *zfsvfs, boolean_t mounting) if (error) return (error); - zfsvfs->z_log = zil_open(zfsvfs->z_os, zfs_get_data); - /* * If we are not mounting (ie: online recv), then we don't * have to worry about replaying the log as we blocked all @@ -857,7 +855,11 @@ zfsvfs_setup(zfsvfs_t *zfsvfs, boolean_t mounting) */ if (mounting) { ASSERT3P(zfsvfs->z_kstat.dk_kstats, ==, NULL); - dataset_kstats_create(&zfsvfs->z_kstat, zfsvfs->z_os); + error = dataset_kstats_create(&zfsvfs->z_kstat, zfsvfs->z_os); + if (error) + return (error); + zfsvfs->z_log = zil_open(zfsvfs->z_os, zfs_get_data, + &zfsvfs->z_kstat.dk_zil_sums); /* * During replay we remove the read only flag to @@ -921,6 +923,10 @@ zfsvfs_setup(zfsvfs_t *zfsvfs, boolean_t mounting) /* restore readonly bit */ if (readonly != 0) readonly_changed_cb(zfsvfs, B_TRUE); + } else { + ASSERT3P(zfsvfs->z_kstat.dk_kstats, !=, NULL); + zfsvfs->z_log = zil_open(zfsvfs->z_os, zfs_get_data, + &zfsvfs->z_kstat.dk_zil_sums); } /* @@ -1137,7 +1143,7 @@ zfs_statvfs(struct inode *ip, struct kstatfs *statp) * We have all of 40 characters to stuff a string here. * Is there anything useful we could/should provide? */ - bzero(statp->f_spare, sizeof (statp->f_spare)); + memset(statp->f_spare, 0, sizeof (statp->f_spare)); if (dmu_objset_projectquota_enabled(zfsvfs->z_os) && dmu_objset_projectquota_present(zfsvfs->z_os)) { @@ -1453,14 +1459,34 @@ zfs_domount(struct super_block *sb, zfs_mnt_t *zm, int silent) int error = 0; zfsvfs_t *zfsvfs = NULL; vfs_t *vfs = NULL; + int canwrite; + int dataset_visible_zone; ASSERT(zm); ASSERT(osname); + dataset_visible_zone = zone_dataset_visible(osname, &canwrite); + + /* + * Refuse to mount a filesystem if we are in a namespace and the + * dataset is not visible or writable in that namespace. + */ + if (!INGLOBALZONE(curproc) && + (!dataset_visible_zone || !canwrite)) { + return (SET_ERROR(EPERM)); + } + error = zfsvfs_parse_options(zm->mnt_data, &vfs); if (error) return (error); + /* + * If a non-writable filesystem is being mounted without the + * read-only flag, pretend it was set, as done for snapshots. + */ + if (!canwrite) + vfs->vfs_readonly = true; + error = zfsvfs_create(osname, vfs->vfs_readonly, &zfsvfs); if (error) { zfsvfs_vfs_free(vfs); @@ -1600,7 +1626,6 @@ zfs_preumount(struct super_block *sb) * Called once all other unmount released tear down has occurred. * It is our responsibility to release any remaining infrastructure. */ -/*ARGSUSED*/ int zfs_umount(struct super_block *sb) { @@ -2128,7 +2153,6 @@ zfs_get_vfs_flag_unmounted(objset_t *os) return (unmounted); } -/*ARGSUSED*/ void zfsvfs_update_fromname(const char *oldname, const char *newname) { @@ -2136,6 +2160,7 @@ zfsvfs_update_fromname(const char *oldname, const char *newname) * We don't need to do anything here, the devname is always current by * virtue of zfsvfs->z_sb->s_op->show_devname. */ + (void) oldname, (void) newname; } void diff --git a/module/os/linux/zfs/zfs_vnops_os.c b/module/os/linux/zfs/zfs_vnops_os.c index 3cc0e69cc4b5..0b3f7f2501ee 100644 --- a/module/os/linux/zfs/zfs_vnops_os.c +++ b/module/os/linux/zfs/zfs_vnops_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -174,11 +174,10 @@ * ZFS_EXIT(zfsvfs); // finished in zfs * return (error); // done, report error */ - -/* ARGSUSED */ int zfs_open(struct inode *ip, int mode, int flag, cred_t *cr) { + (void) cr; znode_t *zp = ITOZ(ip); zfsvfs_t *zfsvfs = ITOZSB(ip); @@ -200,10 +199,10 @@ zfs_open(struct inode *ip, int mode, int flag, cred_t *cr) return (0); } -/* ARGSUSED */ int zfs_close(struct inode *ip, int flag, cred_t *cr) { + (void) cr; znode_t *zp = ITOZ(ip); zfsvfs_t *zfsvfs = ITOZSB(ip); @@ -415,7 +414,6 @@ zfs_zrele_async(znode_t *zp) * Timestamps: * NA */ -/* ARGSUSED */ int zfs_lookup(znode_t *zdp, char *nm, znode_t **zpp, int flags, cred_t *cr, int *direntflags, pathname_t *realpnp) @@ -476,7 +474,7 @@ zfs_lookup(znode_t *zdp, char *nm, znode_t **zpp, int flags, cred_t *cr, */ if ((error = zfs_zaccess(*zpp, ACE_EXECUTE, 0, - B_FALSE, cr))) { + B_TRUE, cr))) { zrele(*zpp); *zpp = NULL; } @@ -535,8 +533,6 @@ zfs_lookup(znode_t *zdp, char *nm, znode_t **zpp, int flags, cred_t *cr, * dzp - ctime|mtime updated if new entry created * zp - ctime|mtime always, atime if new */ - -/* ARGSUSED */ int zfs_create(znode_t *dzp, char *name, vattr_t *vap, int excl, int mode, znode_t **zpp, cred_t *cr, int flag, vsecattr_t *vsecp) @@ -782,11 +778,11 @@ zfs_create(znode_t *dzp, char *name, vattr_t *vap, int excl, return (error); } -/* ARGSUSED */ int zfs_tmpfile(struct inode *dip, vattr_t *vap, int excl, int mode, struct inode **ipp, cred_t *cr, int flag, vsecattr_t *vsecp) { + (void) excl, (void) mode, (void) flag; znode_t *zp = NULL, *dzp = ITOZ(dip); zfsvfs_t *zfsvfs = ITOZSB(dip); objset_t *os; @@ -918,9 +914,8 @@ zfs_tmpfile(struct inode *dip, vattr_t *vap, int excl, * ip - ctime (if nlink > 0) */ -uint64_t null_xattr = 0; +static uint64_t null_xattr = 0; -/*ARGSUSED*/ int zfs_remove(znode_t *dzp, char *name, cred_t *cr, int flags) { @@ -1160,7 +1155,6 @@ zfs_remove(znode_t *dzp, char *name, cred_t *cr, int flags) * dzp - ctime|mtime updated * zpp - ctime|mtime|atime updated */ -/*ARGSUSED*/ int zfs_mkdir(znode_t *dzp, char *dirname, vattr_t *vap, znode_t **zpp, cred_t *cr, int flags, vsecattr_t *vsecp) @@ -1349,7 +1343,6 @@ zfs_mkdir(znode_t *dzp, char *dirname, vattr_t *vap, znode_t **zpp, * Timestamps: * dzp - ctime|mtime updated */ -/*ARGSUSED*/ int zfs_rmdir(znode_t *dzp, char *name, znode_t *cwd, cred_t *cr, int flags) @@ -1482,10 +1475,10 @@ zfs_rmdir(znode_t *dzp, char *name, znode_t *cwd, cred_t *cr, * We use 0 for '.', and 1 for '..'. If this is the root of the filesystem, * we use the offset 2 for the '.zfs' directory. */ -/* ARGSUSED */ int zfs_readdir(struct inode *ip, zpl_dir_context_t *ctx, cred_t *cr) { + (void) cr; znode_t *zp = ITOZ(ip); zfsvfs_t *zfsvfs = ITOZSB(ip); objset_t *os; @@ -1635,7 +1628,6 @@ zfs_readdir(struct inode *ip, zpl_dir_context_t *ctx, cred_t *cr) * * RETURN: 0 (always succeeds) */ -/* ARGSUSED */ int zfs_getattr_fast(struct user_namespace *user_ns, struct inode *ip, struct kstat *sp) @@ -1823,7 +1815,6 @@ zfs_setattr_dir(znode_t *dzp) * Timestamps: * ip - ctime updated, mtime updated if size changed. */ -/* ARGSUSED */ int zfs_setattr(znode_t *zp, vattr_t *vap, int flags, cred_t *cr) { @@ -2652,7 +2643,6 @@ zfs_rename_lock(znode_t *szp, znode_t *tdzp, znode_t *sdzp, zfs_zlock_t **zlpp) * Timestamps: * sdzp,tdzp - ctime|mtime updated */ -/*ARGSUSED*/ int zfs_rename(znode_t *sdzp, char *snm, znode_t *tdzp, char *tnm, cred_t *cr, int flags) @@ -3020,7 +3010,6 @@ zfs_rename(znode_t *sdzp, char *snm, znode_t *tdzp, char *tnm, * Timestamps: * dip - ctime|mtime updated */ -/*ARGSUSED*/ int zfs_symlink(znode_t *dzp, char *name, vattr_t *vap, char *link, znode_t **zpp, cred_t *cr, int flags) @@ -3188,10 +3177,10 @@ zfs_symlink(znode_t *dzp, char *name, vattr_t *vap, char *link, * Timestamps: * ip - atime updated */ -/* ARGSUSED */ int zfs_readlink(struct inode *ip, zfs_uio_t *uio, cred_t *cr) { + (void) cr; znode_t *zp = ITOZ(ip); zfsvfs_t *zfsvfs = ITOZSB(ip); int error; @@ -3227,7 +3216,6 @@ zfs_readlink(struct inode *ip, zfs_uio_t *uio, cred_t *cr) * tdzp - ctime|mtime updated * szp - ctime updated */ -/* ARGSUSED */ int zfs_link(znode_t *tdzp, znode_t *szp, char *name, cred_t *cr, int flags) @@ -3408,21 +3396,34 @@ zfs_link(znode_t *tdzp, znode_t *szp, char *name, cred_t *cr, } static void -zfs_putpage_commit_cb(void *arg) +zfs_putpage_sync_commit_cb(void *arg) +{ + struct page *pp = arg; + + ClearPageError(pp); + end_page_writeback(pp); +} + +static void +zfs_putpage_async_commit_cb(void *arg) { struct page *pp = arg; + znode_t *zp = ITOZ(pp->mapping->host); ClearPageError(pp); end_page_writeback(pp); + atomic_dec_32(&zp->z_async_writes_cnt); } /* * Push a page out to disk, once the page is on stable storage the * registered commit callback will be run as notification of completion. * - * IN: ip - page mapped for inode. - * pp - page to push (page is locked) - * wbc - writeback control data + * IN: ip - page mapped for inode. + * pp - page to push (page is locked) + * wbc - writeback control data + * for_sync - does the caller intend to wait synchronously for the + * page writeback to complete? * * RETURN: 0 if success * error code if failure @@ -3430,9 +3431,9 @@ zfs_putpage_commit_cb(void *arg) * Timestamps: * ip - ctime|mtime updated */ -/* ARGSUSED */ int -zfs_putpage(struct inode *ip, struct page *pp, struct writeback_control *wbc) +zfs_putpage(struct inode *ip, struct page *pp, struct writeback_control *wbc, + boolean_t for_sync) { znode_t *zp = ITOZ(ip); zfsvfs_t *zfsvfs = ITOZSB(ip); @@ -3530,6 +3531,16 @@ zfs_putpage(struct inode *ip, struct page *pp, struct writeback_control *wbc) zfs_rangelock_exit(lr); if (wbc->sync_mode != WB_SYNC_NONE) { + /* + * Speed up any non-sync page writebacks since + * they may take several seconds to complete. + * Refer to the comment in zpl_fsync() (when + * HAVE_FSYNC_RANGE is defined) for details. + */ + if (atomic_load_32(&zp->z_async_writes_cnt) > 0) { + zil_commit(zfsvfs->z_log, zp->z_id); + } + if (PageWriteback(pp)) #ifdef HAVE_PAGEMAP_FOLIO_WAIT_BIT folio_wait_bit(page_folio(pp), PG_writeback); @@ -3555,6 +3566,8 @@ zfs_putpage(struct inode *ip, struct page *pp, struct writeback_control *wbc) * was in fact not skipped and should not be counted as if it were. */ wbc->pages_skipped--; + if (!for_sync) + atomic_inc_32(&zp->z_async_writes_cnt); set_page_writeback(pp); unlock_page(pp); @@ -3569,9 +3582,15 @@ zfs_putpage(struct inode *ip, struct page *pp, struct writeback_control *wbc) dmu_tx_wait(tx); dmu_tx_abort(tx); +#ifdef HAVE_VFS_FILEMAP_DIRTY_FOLIO + filemap_dirty_folio(page_mapping(pp), page_folio(pp)); +#else __set_page_dirty_nobuffers(pp); +#endif ClearPageError(pp); end_page_writeback(pp); + if (!for_sync) + atomic_dec_32(&zp->z_async_writes_cnt); zfs_rangelock_exit(lr); ZFS_EXIT(zfsvfs); return (err); @@ -3596,7 +3615,9 @@ zfs_putpage(struct inode *ip, struct page *pp, struct writeback_control *wbc) err = sa_bulk_update(zp->z_sa_hdl, bulk, cnt, tx); zfs_log_write(zfsvfs->z_log, tx, TX_WRITE, zp, pgoff, pglen, 0, - zfs_putpage_commit_cb, pp); + for_sync ? zfs_putpage_sync_commit_cb : + zfs_putpage_async_commit_cb, pp); + dmu_tx_commit(tx); zfs_rangelock_exit(lr); @@ -3608,6 +3629,16 @@ zfs_putpage(struct inode *ip, struct page *pp, struct writeback_control *wbc) * performance reasons. */ zil_commit(zfsvfs->z_log, zp->z_id); + } else if (!for_sync && atomic_load_32(&zp->z_sync_writes_cnt) > 0) { + /* + * If the caller does not intend to wait synchronously + * for this page writeback to complete and there are active + * synchronous calls on this file, do a commit so that + * the latter don't accidentally end up waiting for + * our writeback to complete. Refer to the comment in + * zpl_fsync() (when HAVE_FSYNC_RANGE is defined) for details. + */ + zil_commit(zfsvfs->z_log, zp->z_id); } dataset_kstats_update_write_kstats(&zfsvfs->z_kstat, pglen); @@ -3687,7 +3718,6 @@ zfs_dirty_inode(struct inode *ip, int flags) return (error); } -/*ARGSUSED*/ void zfs_inactive(struct inode *ip) { @@ -3791,7 +3821,6 @@ zfs_fillpage(struct inode *ip, struct page *pl[], int nr_pages) * Timestamps: * vp - atime updated */ -/* ARGSUSED */ int zfs_getpage(struct inode *ip, struct page *pl[], int nr_pages) { @@ -3825,11 +3854,11 @@ zfs_getpage(struct inode *ip, struct page *pl[], int nr_pages) * RETURN: 0 if success * error code if failure */ -/*ARGSUSED*/ int zfs_map(struct inode *ip, offset_t off, caddr_t *addrp, size_t len, unsigned long vm_flags) { + (void) addrp; znode_t *zp = ITOZ(ip); zfsvfs_t *zfsvfs = ITOZSB(ip); @@ -3875,11 +3904,11 @@ zfs_map(struct inode *ip, offset_t off, caddr_t *addrp, size_t len, * Timestamps: * zp - ctime|mtime updated */ -/* ARGSUSED */ int zfs_space(znode_t *zp, int cmd, flock64_t *bfp, int flag, offset_t offset, cred_t *cr) { + (void) offset; zfsvfs_t *zfsvfs = ZTOZSB(zp); uint64_t off, len; int error; @@ -3926,7 +3955,6 @@ zfs_space(znode_t *zp, int cmd, flock64_t *bfp, int flag, return (error); } -/*ARGSUSED*/ int zfs_fid(struct inode *ip, fid_t *fidp) { diff --git a/module/os/linux/zfs/zfs_znode.c b/module/os/linux/zfs/zfs_znode.c index 65a3303e215f..73c21b6c00a8 100644 --- a/module/os/linux/zfs/zfs_znode.c +++ b/module/os/linux/zfs/zfs_znode.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -113,10 +113,10 @@ zfs_rangelock_cb(zfs_locked_range_t *new, void *arg) } } -/*ARGSUSED*/ static int zfs_znode_cache_constructor(void *buf, void *arg, int kmflags) { + (void) arg, (void) kmflags; znode_t *zp = buf; inode_init_once(ZTOI(zp)); @@ -134,13 +134,16 @@ zfs_znode_cache_constructor(void *buf, void *arg, int kmflags) zp->z_acl_cached = NULL; zp->z_xattr_cached = NULL; zp->z_xattr_parent = 0; + zp->z_sync_writes_cnt = 0; + zp->z_async_writes_cnt = 0; + return (0); } -/*ARGSUSED*/ static void zfs_znode_cache_destructor(void *buf, void *arg) { + (void) arg; znode_t *zp = buf; ASSERT(!list_link_active(&zp->z_link_node)); @@ -154,11 +157,15 @@ zfs_znode_cache_destructor(void *buf, void *arg) ASSERT3P(zp->z_dirlocks, ==, NULL); ASSERT3P(zp->z_acl_cached, ==, NULL); ASSERT3P(zp->z_xattr_cached, ==, NULL); + + ASSERT0(atomic_load_32(&zp->z_sync_writes_cnt)); + ASSERT0(atomic_load_32(&zp->z_async_writes_cnt)); } static int zfs_znode_hold_cache_constructor(void *buf, void *arg, int kmflags) { + (void) arg, (void) kmflags; znode_hold_t *zh = buf; mutex_init(&zh->zh_lock, NULL, MUTEX_DEFAULT, NULL); @@ -171,6 +178,7 @@ zfs_znode_hold_cache_constructor(void *buf, void *arg, int kmflags) static void zfs_znode_hold_cache_destructor(void *buf, void *arg) { + (void) arg; znode_hold_t *zh = buf; mutex_destroy(&zh->zh_lock); @@ -552,6 +560,8 @@ zfs_znode_alloc(zfsvfs_t *zfsvfs, dmu_buf_t *db, int blksz, zp->z_blksz = blksz; zp->z_seq = 0x7A4653; zp->z_sync_cnt = 0; + zp->z_sync_writes_cnt = 0; + zp->z_async_writes_cnt = 0; zfs_znode_sa_init(zfsvfs, zp, db, obj_type, hdl); @@ -1579,7 +1589,7 @@ zfs_zero_partial_page(znode_t *zp, uint64_t start, uint64_t len) flush_dcache_page(pp); pb = kmap(pp); - bzero(pb + off, len); + memset(pb + off, 0, len); kunmap(pp); if (mapping_writably_mapped(mp)) @@ -1991,7 +2001,7 @@ zfs_sa_setup(objset_t *osp, sa_attr_type_t **sa_table) static int zfs_grab_sa_handle(objset_t *osp, uint64_t obj, sa_handle_t **hdlp, - dmu_buf_t **db, void *tag) + dmu_buf_t **db, const void *tag) { dmu_object_info_t doi; int error; @@ -2018,7 +2028,7 @@ zfs_grab_sa_handle(objset_t *osp, uint64_t obj, sa_handle_t **hdlp, } static void -zfs_release_sa_handle(sa_handle_t *hdl, dmu_buf_t *db, void *tag) +zfs_release_sa_handle(sa_handle_t *hdl, dmu_buf_t *db, const void *tag) { sa_handle_destroy(hdl); sa_buf_rele(db, tag); @@ -2151,7 +2161,7 @@ zfs_obj_to_path_impl(objset_t *osp, uint64_t obj, sa_handle_t *hdl, component[0] = '/'; if (is_xattrdir) { - (void) sprintf(component + 1, ""); + strcpy(component + 1, ""); } else { error = zap_value_search(osp, pobj, obj, ZFS_DIRENT_OBJ(-1ULL), component + 1); @@ -2162,7 +2172,7 @@ zfs_obj_to_path_impl(objset_t *osp, uint64_t obj, sa_handle_t *hdl, complen = strlen(component); path -= complen; ASSERT(path >= buf); - bcopy(component, path, complen); + memcpy(path, component, complen); obj = pobj; if (sa_hdl != hdl) { diff --git a/module/os/linux/zfs/zio_crypt.c b/module/os/linux/zfs/zio_crypt.c index f60767855c67..dcab02b07894 100644 --- a/module/os/linux/zfs/zio_crypt.c +++ b/module/os/linux/zfs/zio_crypt.c @@ -216,7 +216,7 @@ zio_crypt_key_destroy(zio_crypt_key_t *key) crypto_destroy_ctx_template(key->zk_hmac_tmpl); /* zero out sensitive data */ - bzero(key, sizeof (zio_crypt_key_t)); + memset(key, 0, sizeof (zio_crypt_key_t)); } int @@ -230,7 +230,7 @@ zio_crypt_key_init(uint64_t crypt, zio_crypt_key_t *key) ASSERT3U(crypt, <, ZIO_CRYPT_FUNCTIONS); keydata_len = zio_crypt_table[crypt].ci_keylen; - bzero(key, sizeof (zio_crypt_key_t)); + memset(key, 0, sizeof (zio_crypt_key_t)); /* fill keydata buffers and salt with random data */ ret = random_get_bytes((uint8_t *)&key->zk_guid, sizeof (uint64_t)); @@ -257,11 +257,9 @@ zio_crypt_key_init(uint64_t crypt, zio_crypt_key_t *key) goto error; /* initialize keys for the ICP */ - key->zk_current_key.ck_format = CRYPTO_KEY_RAW; key->zk_current_key.ck_data = key->zk_current_keydata; key->zk_current_key.ck_length = CRYPTO_BYTES2BITS(keydata_len); - key->zk_hmac_key.ck_format = CRYPTO_KEY_RAW; key->zk_hmac_key.ck_data = &key->zk_hmac_key; key->zk_hmac_key.ck_length = CRYPTO_BYTES2BITS(SHA512_HMAC_KEYLEN); @@ -271,13 +269,13 @@ zio_crypt_key_init(uint64_t crypt, zio_crypt_key_t *key) */ mech.cm_type = crypto_mech2id(zio_crypt_table[crypt].ci_mechname); ret = crypto_create_ctx_template(&mech, &key->zk_current_key, - &key->zk_current_tmpl, KM_SLEEP); + &key->zk_current_tmpl); if (ret != CRYPTO_SUCCESS) key->zk_current_tmpl = NULL; mech.cm_type = crypto_mech2id(SUN_CKM_SHA512_HMAC); ret = crypto_create_ctx_template(&mech, &key->zk_hmac_key, - &key->zk_hmac_tmpl, KM_SLEEP); + &key->zk_hmac_tmpl); if (ret != CRYPTO_SUCCESS) key->zk_hmac_tmpl = NULL; @@ -319,13 +317,13 @@ zio_crypt_key_change_salt(zio_crypt_key_t *key) goto out_unlock; /* assign the salt and reset the usage count */ - bcopy(salt, key->zk_salt, ZIO_DATA_SALT_LEN); + memcpy(key->zk_salt, salt, ZIO_DATA_SALT_LEN); key->zk_salt_count = 0; /* destroy the old context template and create the new one */ crypto_destroy_ctx_template(key->zk_current_tmpl); ret = crypto_create_ctx_template(&mech, &key->zk_current_key, - &key->zk_current_tmpl, KM_SLEEP); + &key->zk_current_tmpl); if (ret != CRYPTO_SUCCESS) key->zk_current_tmpl = NULL; @@ -348,7 +346,7 @@ zio_crypt_key_get_salt(zio_crypt_key_t *key, uint8_t *salt) rw_enter(&key->zk_salt_lock, RW_READER); - bcopy(key->zk_salt, salt, ZIO_DATA_SALT_LEN); + memcpy(salt, key->zk_salt, ZIO_DATA_SALT_LEN); salt_change = (atomic_inc_64_nv(&key->zk_salt_count) >= ZFS_CURRENT_MAX_SALT_USES); @@ -387,7 +385,6 @@ zio_do_crypt_uio(boolean_t encrypt, uint64_t crypt, crypto_key_t *key, uint_t plain_full_len, maclen; ASSERT3U(crypt, <, ZIO_CRYPT_FUNCTIONS); - ASSERT3U(key->ck_format, ==, CRYPTO_KEY_RAW); /* lookup the encryption info */ crypt_info = zio_crypt_table[crypt]; @@ -441,26 +438,22 @@ zio_do_crypt_uio(boolean_t encrypt, uint64_t crypt, crypto_key_t *key, plaindata.cd_format = CRYPTO_DATA_UIO; plaindata.cd_offset = 0; plaindata.cd_uio = puio; - plaindata.cd_miscdata = NULL; plaindata.cd_length = plain_full_len; cipherdata.cd_format = CRYPTO_DATA_UIO; cipherdata.cd_offset = 0; cipherdata.cd_uio = cuio; - cipherdata.cd_miscdata = NULL; cipherdata.cd_length = datalen + maclen; /* perform the actual encryption */ if (encrypt) { - ret = crypto_encrypt(&mech, &plaindata, key, tmpl, &cipherdata, - NULL); + ret = crypto_encrypt(&mech, &plaindata, key, tmpl, &cipherdata); if (ret != CRYPTO_SUCCESS) { ret = SET_ERROR(EIO); goto error; } } else { - ret = crypto_decrypt(&mech, &cipherdata, key, tmpl, &plaindata, - NULL); + ret = crypto_decrypt(&mech, &cipherdata, key, tmpl, &plaindata); if (ret != CRYPTO_SUCCESS) { ASSERT3U(ret, ==, CRYPTO_INVALID_MAC); ret = SET_ERROR(ECKSUM); @@ -486,7 +479,6 @@ zio_crypt_key_wrap(crypto_key_t *cwkey, zio_crypt_key_t *key, uint8_t *iv, uint_t enc_len, keydata_len, aad_len; ASSERT3U(crypt, <, ZIO_CRYPT_FUNCTIONS); - ASSERT3U(cwkey->ck_format, ==, CRYPTO_KEY_RAW); keydata_len = zio_crypt_table[crypt].ci_keylen; @@ -557,7 +549,6 @@ zio_crypt_key_unwrap(crypto_key_t *cwkey, uint64_t crypt, uint64_t version, int ret; ASSERT3U(crypt, <, ZIO_CRYPT_FUNCTIONS); - ASSERT3U(cwkey->ck_format, ==, CRYPTO_KEY_RAW); rw_init(&key->zk_salt_lock, NULL, RW_DEFAULT, NULL); @@ -614,11 +605,9 @@ zio_crypt_key_unwrap(crypto_key_t *cwkey, uint64_t crypt, uint64_t version, goto error; /* initialize keys for ICP */ - key->zk_current_key.ck_format = CRYPTO_KEY_RAW; key->zk_current_key.ck_data = key->zk_current_keydata; key->zk_current_key.ck_length = CRYPTO_BYTES2BITS(keydata_len); - key->zk_hmac_key.ck_format = CRYPTO_KEY_RAW; key->zk_hmac_key.ck_data = key->zk_hmac_keydata; key->zk_hmac_key.ck_length = CRYPTO_BYTES2BITS(SHA512_HMAC_KEYLEN); @@ -628,13 +617,13 @@ zio_crypt_key_unwrap(crypto_key_t *cwkey, uint64_t crypt, uint64_t version, */ mech.cm_type = crypto_mech2id(zio_crypt_table[crypt].ci_mechname); ret = crypto_create_ctx_template(&mech, &key->zk_current_key, - &key->zk_current_tmpl, KM_SLEEP); + &key->zk_current_tmpl); if (ret != CRYPTO_SUCCESS) key->zk_current_tmpl = NULL; mech.cm_type = crypto_mech2id(SUN_CKM_SHA512_HMAC); ret = crypto_create_ctx_template(&mech, &key->zk_hmac_key, - &key->zk_hmac_tmpl, KM_SLEEP); + &key->zk_hmac_tmpl); if (ret != CRYPTO_SUCCESS) key->zk_hmac_tmpl = NULL; @@ -663,7 +652,7 @@ zio_crypt_generate_iv(uint8_t *ivbuf) return (0); error: - bzero(ivbuf, ZIO_DATA_IV_LEN); + memset(ivbuf, 0, ZIO_DATA_IV_LEN); return (ret); } @@ -698,18 +687,18 @@ zio_crypt_do_hmac(zio_crypt_key_t *key, uint8_t *data, uint_t datalen, /* generate the hmac */ ret = crypto_mac(&mech, &in_data, &key->zk_hmac_key, key->zk_hmac_tmpl, - &digest_data, NULL); + &digest_data); if (ret != CRYPTO_SUCCESS) { ret = SET_ERROR(EIO); goto error; } - bcopy(raw_digestbuf, digestbuf, digestlen); + memcpy(digestbuf, raw_digestbuf, digestlen); return (0); error: - bzero(digestbuf, digestlen); + memset(digestbuf, 0, digestlen); return (ret); } @@ -725,8 +714,8 @@ zio_crypt_generate_iv_salt_dedup(zio_crypt_key_t *key, uint8_t *data, if (ret != 0) return (ret); - bcopy(digestbuf, salt, ZIO_DATA_SALT_LEN); - bcopy(digestbuf + ZIO_DATA_SALT_LEN, ivbuf, ZIO_DATA_IV_LEN); + memcpy(salt, digestbuf, ZIO_DATA_SALT_LEN); + memcpy(ivbuf, digestbuf + ZIO_DATA_SALT_LEN, ZIO_DATA_IV_LEN); return (0); } @@ -749,18 +738,18 @@ zio_crypt_encode_params_bp(blkptr_t *bp, uint8_t *salt, uint8_t *iv) ASSERT(BP_IS_ENCRYPTED(bp)); if (!BP_SHOULD_BYTESWAP(bp)) { - bcopy(salt, &bp->blk_dva[2].dva_word[0], sizeof (uint64_t)); - bcopy(iv, &bp->blk_dva[2].dva_word[1], sizeof (uint64_t)); - bcopy(iv + sizeof (uint64_t), &val32, sizeof (uint32_t)); + memcpy(&bp->blk_dva[2].dva_word[0], salt, sizeof (uint64_t)); + memcpy(&bp->blk_dva[2].dva_word[1], iv, sizeof (uint64_t)); + memcpy(&val32, iv + sizeof (uint64_t), sizeof (uint32_t)); BP_SET_IV2(bp, val32); } else { - bcopy(salt, &val64, sizeof (uint64_t)); + memcpy(&val64, salt, sizeof (uint64_t)); bp->blk_dva[2].dva_word[0] = BSWAP_64(val64); - bcopy(iv, &val64, sizeof (uint64_t)); + memcpy(&val64, iv, sizeof (uint64_t)); bp->blk_dva[2].dva_word[1] = BSWAP_64(val64); - bcopy(iv + sizeof (uint64_t), &val32, sizeof (uint32_t)); + memcpy(&val32, iv + sizeof (uint64_t), sizeof (uint32_t)); BP_SET_IV2(bp, BSWAP_32(val32)); } } @@ -775,26 +764,26 @@ zio_crypt_decode_params_bp(const blkptr_t *bp, uint8_t *salt, uint8_t *iv) /* for convenience, so callers don't need to check */ if (BP_IS_AUTHENTICATED(bp)) { - bzero(salt, ZIO_DATA_SALT_LEN); - bzero(iv, ZIO_DATA_IV_LEN); + memset(salt, 0, ZIO_DATA_SALT_LEN); + memset(iv, 0, ZIO_DATA_IV_LEN); return; } if (!BP_SHOULD_BYTESWAP(bp)) { - bcopy(&bp->blk_dva[2].dva_word[0], salt, sizeof (uint64_t)); - bcopy(&bp->blk_dva[2].dva_word[1], iv, sizeof (uint64_t)); + memcpy(salt, &bp->blk_dva[2].dva_word[0], sizeof (uint64_t)); + memcpy(iv, &bp->blk_dva[2].dva_word[1], sizeof (uint64_t)); val32 = (uint32_t)BP_GET_IV2(bp); - bcopy(&val32, iv + sizeof (uint64_t), sizeof (uint32_t)); + memcpy(iv + sizeof (uint64_t), &val32, sizeof (uint32_t)); } else { val64 = BSWAP_64(bp->blk_dva[2].dva_word[0]); - bcopy(&val64, salt, sizeof (uint64_t)); + memcpy(salt, &val64, sizeof (uint64_t)); val64 = BSWAP_64(bp->blk_dva[2].dva_word[1]); - bcopy(&val64, iv, sizeof (uint64_t)); + memcpy(iv, &val64, sizeof (uint64_t)); val32 = BSWAP_32((uint32_t)BP_GET_IV2(bp)); - bcopy(&val32, iv + sizeof (uint64_t), sizeof (uint32_t)); + memcpy(iv + sizeof (uint64_t), &val32, sizeof (uint32_t)); } } @@ -807,14 +796,14 @@ zio_crypt_encode_mac_bp(blkptr_t *bp, uint8_t *mac) ASSERT3U(BP_GET_TYPE(bp), !=, DMU_OT_OBJSET); if (!BP_SHOULD_BYTESWAP(bp)) { - bcopy(mac, &bp->blk_cksum.zc_word[2], sizeof (uint64_t)); - bcopy(mac + sizeof (uint64_t), &bp->blk_cksum.zc_word[3], + memcpy(&bp->blk_cksum.zc_word[2], mac, sizeof (uint64_t)); + memcpy(&bp->blk_cksum.zc_word[3], mac + sizeof (uint64_t), sizeof (uint64_t)); } else { - bcopy(mac, &val64, sizeof (uint64_t)); + memcpy(&val64, mac, sizeof (uint64_t)); bp->blk_cksum.zc_word[2] = BSWAP_64(val64); - bcopy(mac + sizeof (uint64_t), &val64, sizeof (uint64_t)); + memcpy(&val64, mac + sizeof (uint64_t), sizeof (uint64_t)); bp->blk_cksum.zc_word[3] = BSWAP_64(val64); } } @@ -828,20 +817,20 @@ zio_crypt_decode_mac_bp(const blkptr_t *bp, uint8_t *mac) /* for convenience, so callers don't need to check */ if (BP_GET_TYPE(bp) == DMU_OT_OBJSET) { - bzero(mac, ZIO_DATA_MAC_LEN); + memset(mac, 0, ZIO_DATA_MAC_LEN); return; } if (!BP_SHOULD_BYTESWAP(bp)) { - bcopy(&bp->blk_cksum.zc_word[2], mac, sizeof (uint64_t)); - bcopy(&bp->blk_cksum.zc_word[3], mac + sizeof (uint64_t), + memcpy(mac, &bp->blk_cksum.zc_word[2], sizeof (uint64_t)); + memcpy(mac + sizeof (uint64_t), &bp->blk_cksum.zc_word[3], sizeof (uint64_t)); } else { val64 = BSWAP_64(bp->blk_cksum.zc_word[2]); - bcopy(&val64, mac, sizeof (uint64_t)); + memcpy(mac, &val64, sizeof (uint64_t)); val64 = BSWAP_64(bp->blk_cksum.zc_word[3]); - bcopy(&val64, mac + sizeof (uint64_t), sizeof (uint64_t)); + memcpy(mac + sizeof (uint64_t), &val64, sizeof (uint64_t)); } } @@ -850,8 +839,8 @@ zio_crypt_encode_mac_zil(void *data, uint8_t *mac) { zil_chain_t *zilc = data; - bcopy(mac, &zilc->zc_eck.zec_cksum.zc_word[2], sizeof (uint64_t)); - bcopy(mac + sizeof (uint64_t), &zilc->zc_eck.zec_cksum.zc_word[3], + memcpy(&zilc->zc_eck.zec_cksum.zc_word[2], mac, sizeof (uint64_t)); + memcpy(&zilc->zc_eck.zec_cksum.zc_word[3], mac + sizeof (uint64_t), sizeof (uint64_t)); } @@ -865,8 +854,8 @@ zio_crypt_decode_mac_zil(const void *data, uint8_t *mac) */ const zil_chain_t *zilc = data; - bcopy(&zilc->zc_eck.zec_cksum.zc_word[2], mac, sizeof (uint64_t)); - bcopy(&zilc->zc_eck.zec_cksum.zc_word[3], mac + sizeof (uint64_t), + memcpy(mac, &zilc->zc_eck.zec_cksum.zc_word[2], sizeof (uint64_t)); + memcpy(mac + sizeof (uint64_t), &zilc->zc_eck.zec_cksum.zc_word[3], sizeof (uint64_t)); } @@ -893,7 +882,7 @@ zio_crypt_copy_dnode_bonus(abd_t *src_abd, uint8_t *dst, uint_t datalen) if (dnp->dn_type != DMU_OT_NONE && DMU_OT_IS_ENCRYPTED(dnp->dn_bonustype) && dnp->dn_bonuslen != 0) { - bcopy(DN_BONUS(dnp), DN_BONUS(&ddnp[i]), + memcpy(DN_BONUS(&ddnp[i]), DN_BONUS(dnp), DN_MAX_BONUS_LEN(dnp)); } } @@ -1004,7 +993,7 @@ zio_crypt_bp_do_hmac_updates(crypto_context_t ctx, uint64_t version, cd.cd_raw.iov_base = (char *)&bab; cd.cd_raw.iov_len = cd.cd_length; - ret = crypto_mac_update(ctx, &cd, NULL); + ret = crypto_mac_update(ctx, &cd); if (ret != CRYPTO_SUCCESS) { ret = SET_ERROR(EIO); goto error; @@ -1035,7 +1024,7 @@ zio_crypt_bp_do_aad_updates(uint8_t **aadp, uint_t *aad_len, uint64_t version, blkptr_auth_buf_t bab; zio_crypt_bp_auth_init(version, should_bswap, bp, &bab, &bab_len); - bcopy(&bab, *aadp, bab_len); + memcpy(*aadp, &bab, bab_len); *aadp += bab_len; *aad_len += bab_len; } @@ -1059,7 +1048,7 @@ zio_crypt_do_dnode_hmac_updates(crypto_context_t ctx, uint64_t version, * of copying 512-64 unneeded bytes. The compiler seems to be fine * with that. */ - bcopy(dnp, &tmp_dncore, dn_core_size); + memcpy(&tmp_dncore, dnp, dn_core_size); adnp = &tmp_dncore; if (le_bswap) { @@ -1075,7 +1064,7 @@ zio_crypt_do_dnode_hmac_updates(crypto_context_t ctx, uint64_t version, cd.cd_raw.iov_base = (char *)adnp; cd.cd_raw.iov_len = cd.cd_length; - ret = crypto_mac_update(ctx, &cd, NULL); + ret = crypto_mac_update(ctx, &cd); if (ret != CRYPTO_SUCCESS) { ret = SET_ERROR(EIO); goto error; @@ -1148,7 +1137,7 @@ zio_crypt_do_objset_hmacs(zio_crypt_key_t *key, void *data, uint_t datalen, cd.cd_offset = 0; /* calculate the portable MAC from the portable fields and metadnode */ - ret = crypto_mac_init(&mech, &key->zk_hmac_key, NULL, &ctx, NULL); + ret = crypto_mac_init(&mech, &key->zk_hmac_key, NULL, &ctx); if (ret != CRYPTO_SUCCESS) { ret = SET_ERROR(EIO); goto error; @@ -1160,7 +1149,7 @@ zio_crypt_do_objset_hmacs(zio_crypt_key_t *key, void *data, uint_t datalen, cd.cd_raw.iov_base = (char *)&intval; cd.cd_raw.iov_len = cd.cd_length; - ret = crypto_mac_update(ctx, &cd, NULL); + ret = crypto_mac_update(ctx, &cd); if (ret != CRYPTO_SUCCESS) { ret = SET_ERROR(EIO); goto error; @@ -1178,7 +1167,7 @@ zio_crypt_do_objset_hmacs(zio_crypt_key_t *key, void *data, uint_t datalen, cd.cd_raw.iov_base = (char *)&intval; cd.cd_raw.iov_len = cd.cd_length; - ret = crypto_mac_update(ctx, &cd, NULL); + ret = crypto_mac_update(ctx, &cd); if (ret != CRYPTO_SUCCESS) { ret = SET_ERROR(EIO); goto error; @@ -1195,13 +1184,13 @@ zio_crypt_do_objset_hmacs(zio_crypt_key_t *key, void *data, uint_t datalen, cd.cd_raw.iov_base = (char *)raw_portable_mac; cd.cd_raw.iov_len = cd.cd_length; - ret = crypto_mac_final(ctx, &cd, NULL); + ret = crypto_mac_final(ctx, &cd); if (ret != CRYPTO_SUCCESS) { ret = SET_ERROR(EIO); goto error; } - bcopy(raw_portable_mac, portable_mac, ZIO_OBJSET_MAC_LEN); + memcpy(portable_mac, raw_portable_mac, ZIO_OBJSET_MAC_LEN); /* * This is necessary here as we check next whether @@ -1230,12 +1219,12 @@ zio_crypt_do_objset_hmacs(zio_crypt_key_t *key, void *data, uint_t datalen, osp->os_userused_dnode.dn_type == DMU_OT_NONE && osp->os_groupused_dnode.dn_type == DMU_OT_NONE) || (datalen <= OBJSET_PHYS_SIZE_V1)) { - bzero(local_mac, ZIO_OBJSET_MAC_LEN); + memset(local_mac, 0, ZIO_OBJSET_MAC_LEN); return (0); } /* calculate the local MAC from the userused and groupused dnodes */ - ret = crypto_mac_init(&mech, &key->zk_hmac_key, NULL, &ctx, NULL); + ret = crypto_mac_init(&mech, &key->zk_hmac_key, NULL, &ctx); if (ret != CRYPTO_SUCCESS) { ret = SET_ERROR(EIO); goto error; @@ -1253,7 +1242,7 @@ zio_crypt_do_objset_hmacs(zio_crypt_key_t *key, void *data, uint_t datalen, cd.cd_raw.iov_base = (char *)&intval; cd.cd_raw.iov_len = cd.cd_length; - ret = crypto_mac_update(ctx, &cd, NULL); + ret = crypto_mac_update(ctx, &cd); if (ret != CRYPTO_SUCCESS) { ret = SET_ERROR(EIO); goto error; @@ -1287,19 +1276,19 @@ zio_crypt_do_objset_hmacs(zio_crypt_key_t *key, void *data, uint_t datalen, cd.cd_raw.iov_base = (char *)raw_local_mac; cd.cd_raw.iov_len = cd.cd_length; - ret = crypto_mac_final(ctx, &cd, NULL); + ret = crypto_mac_final(ctx, &cd); if (ret != CRYPTO_SUCCESS) { ret = SET_ERROR(EIO); goto error; } - bcopy(raw_local_mac, local_mac, ZIO_OBJSET_MAC_LEN); + memcpy(local_mac, raw_local_mac, ZIO_OBJSET_MAC_LEN); return (0); error: - bzero(portable_mac, ZIO_OBJSET_MAC_LEN); - bzero(local_mac, ZIO_OBJSET_MAC_LEN); + memset(portable_mac, 0, ZIO_OBJSET_MAC_LEN); + memset(local_mac, 0, ZIO_OBJSET_MAC_LEN); return (ret); } @@ -1335,11 +1324,11 @@ zio_crypt_do_indirect_mac_checksum_impl(boolean_t generate, void *buf, SHA2Final(digestbuf, &ctx); if (generate) { - bcopy(digestbuf, cksum, ZIO_DATA_MAC_LEN); + memcpy(cksum, digestbuf, ZIO_DATA_MAC_LEN); return (0); } - if (bcmp(digestbuf, cksum, ZIO_DATA_MAC_LEN) != 0) + if (memcmp(digestbuf, cksum, ZIO_DATA_MAC_LEN) != 0) return (SET_ERROR(ECKSUM)); return (0); @@ -1420,7 +1409,7 @@ zio_crypt_init_uios_zil(boolean_t encrypt, uint8_t *plainbuf, nr_src = 1; nr_dst = 0; } - bzero(dst, datalen); + memset(dst, 0, datalen); /* find the start and end record of the log block */ zilc = (zil_chain_t *)src; @@ -1471,8 +1460,8 @@ zio_crypt_init_uios_zil(boolean_t encrypt, uint8_t *plainbuf, * the embedded checksum will not have been calculated yet, so we don't * authenticate that. */ - bcopy(src, dst, sizeof (zil_chain_t)); - bcopy(src, aadp, sizeof (zil_chain_t) - sizeof (zio_eck_t)); + memcpy(dst, src, sizeof (zil_chain_t)); + memcpy(aadp, src, sizeof (zil_chain_t) - sizeof (zio_eck_t)); aadp += sizeof (zil_chain_t) - sizeof (zio_eck_t); aad_len += sizeof (zil_chain_t) - sizeof (zio_eck_t); @@ -1493,8 +1482,8 @@ zio_crypt_init_uios_zil(boolean_t encrypt, uint8_t *plainbuf, } /* copy the common lr_t */ - bcopy(slrp, dlrp, sizeof (lr_t)); - bcopy(slrp, aadp, sizeof (lr_t)); + memcpy(dlrp, slrp, sizeof (lr_t)); + memcpy(aadp, slrp, sizeof (lr_t)); aadp += sizeof (lr_t); aad_len += sizeof (lr_t); @@ -1515,11 +1504,12 @@ zio_crypt_init_uios_zil(boolean_t encrypt, uint8_t *plainbuf, dst_iovecs[nr_iovecs].iov_len = crypt_len; /* copy the bp now since it will not be encrypted */ - bcopy(slrp + sizeof (lr_write_t) - sizeof (blkptr_t), - dlrp + sizeof (lr_write_t) - sizeof (blkptr_t), + memcpy(dlrp + sizeof (lr_write_t) - sizeof (blkptr_t), + slrp + sizeof (lr_write_t) - sizeof (blkptr_t), + sizeof (blkptr_t)); + memcpy(aadp, + slrp + sizeof (lr_write_t) - sizeof (blkptr_t), sizeof (blkptr_t)); - bcopy(slrp + sizeof (lr_write_t) - sizeof (blkptr_t), - aadp, sizeof (blkptr_t)); aadp += sizeof (blkptr_t); aad_len += sizeof (blkptr_t); nr_iovecs++; @@ -1666,10 +1656,11 @@ zio_crypt_init_uios_dnode(boolean_t encrypt, uint64_t version, dnp = &sdnp[i]; /* copy over the core fields and blkptrs (kept as plaintext) */ - bcopy(dnp, &ddnp[i], (uint8_t *)DN_BONUS(dnp) - (uint8_t *)dnp); + memcpy(&ddnp[i], dnp, + (uint8_t *)DN_BONUS(dnp) - (uint8_t *)dnp); if (dnp->dn_flags & DNODE_FLAG_SPILL_BLKPTR) { - bcopy(DN_SPILL_BLKPTR(dnp), DN_SPILL_BLKPTR(&ddnp[i]), + memcpy(DN_SPILL_BLKPTR(&ddnp[i]), DN_SPILL_BLKPTR(dnp), sizeof (blkptr_t)); } @@ -1684,7 +1675,7 @@ zio_crypt_init_uios_dnode(boolean_t encrypt, uint64_t version, * authenticated data. */ crypt_len = offsetof(dnode_phys_t, dn_blkptr); - bcopy(dnp, aadp, crypt_len); + memcpy(aadp, dnp, crypt_len); adnp = (dnode_phys_t *)aadp; adnp->dn_flags &= DNODE_CRYPT_PORTABLE_FLAGS_MASK; adnp->dn_used = 0; @@ -1727,8 +1718,8 @@ zio_crypt_init_uios_dnode(boolean_t encrypt, uint64_t version, nr_iovecs++; total_len += crypt_len; } else { - bcopy(DN_BONUS(dnp), DN_BONUS(&ddnp[i]), crypt_len); - bcopy(DN_BONUS(dnp), aadp, crypt_len); + memcpy(DN_BONUS(&ddnp[i]), DN_BONUS(dnp), crypt_len); + memcpy(aadp, DN_BONUS(dnp), crypt_len); aadp += crypt_len; aad_len += crypt_len; } @@ -1909,7 +1900,7 @@ zio_do_crypt_data(boolean_t encrypt, zio_crypt_key_t *key, rw_enter(&key->zk_salt_lock, RW_READER); locked = B_TRUE; - if (bcmp(salt, key->zk_salt, ZIO_DATA_SALT_LEN) == 0) { + if (memcmp(salt, key->zk_salt, ZIO_DATA_SALT_LEN) == 0) { ckey = &key->zk_current_key; tmpl = key->zk_current_tmpl; } else { @@ -1921,7 +1912,6 @@ zio_do_crypt_data(boolean_t encrypt, zio_crypt_key_t *key, if (ret != 0) goto error; - tmp_ckey.ck_format = CRYPTO_KEY_RAW; tmp_ckey.ck_data = enc_keydata; tmp_ckey.ck_length = CRYPTO_BYTES2BITS(keydata_len); @@ -1960,8 +1950,8 @@ zio_do_crypt_data(boolean_t encrypt, zio_crypt_key_t *key, /* If the hardware implementation fails fall back to software */ } - bzero(&puio, sizeof (zfs_uio_t)); - bzero(&cuio, sizeof (zfs_uio_t)); + memset(&puio, 0, sizeof (puio)); + memset(&cuio, 0, sizeof (cuio)); /* create uios for encryption */ ret = zio_crypt_init_uios(encrypt, key->zk_version, ot, plainbuf, @@ -1984,7 +1974,7 @@ zio_do_crypt_data(boolean_t encrypt, zio_crypt_key_t *key, if (authbuf != NULL) zio_buf_free(authbuf, datalen); if (ckey == &tmp_ckey) - bzero(enc_keydata, keydata_len); + memset(enc_keydata, 0, keydata_len); zio_crypt_destroy_uio(&puio); zio_crypt_destroy_uio(&cuio); @@ -1996,7 +1986,7 @@ zio_do_crypt_data(boolean_t encrypt, zio_crypt_key_t *key, if (authbuf != NULL) zio_buf_free(authbuf, datalen); if (ckey == &tmp_ckey) - bzero(enc_keydata, keydata_len); + memset(enc_keydata, 0, keydata_len); zio_crypt_destroy_uio(&puio); zio_crypt_destroy_uio(&cuio); diff --git a/module/os/linux/zfs/zpl_ctldir.c b/module/os/linux/zfs/zpl_ctldir.c index a640930a02e1..ec8f2938598f 100644 --- a/module/os/linux/zfs/zpl_ctldir.c +++ b/module/os/linux/zfs/zpl_ctldir.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -32,11 +32,13 @@ #include #include #include +#include +#include +#include /* * Common open routine. Disallow any write access. */ -/* ARGSUSED */ static int zpl_common_open(struct inode *ip, struct file *filp) { @@ -99,7 +101,6 @@ zpl_root_readdir(struct file *filp, void *dirent, filldir_t filldir) /* * Get root directory attributes. */ -/* ARGSUSED */ static int #ifdef HAVE_USERNS_IOPS_GETATTR zpl_root_getattr_impl(struct user_namespace *user_ns, @@ -110,10 +111,15 @@ zpl_root_getattr_impl(const struct path *path, struct kstat *stat, u32 request_mask, unsigned int query_flags) #endif { + (void) request_mask, (void) query_flags; struct inode *ip = path->dentry->d_inode; -#if defined(HAVE_GENERIC_FILLATTR_USERNS) && defined(HAVE_USERNS_IOPS_GETATTR) +#ifdef HAVE_USERNS_IOPS_GETATTR +#ifdef HAVE_GENERIC_FILLATTR_USERNS generic_fillattr(user_ns, ip, stat); +#else + (void) user_ns; +#endif #else generic_fillattr(ip, stat); #endif @@ -382,7 +388,6 @@ zpl_snapdir_mkdir(struct inode *dip, struct dentry *dentry, umode_t mode) /* * Get snapshot directory attributes. */ -/* ARGSUSED */ static int #ifdef HAVE_USERNS_IOPS_GETATTR zpl_snapdir_getattr_impl(struct user_namespace *user_ns, @@ -393,17 +398,36 @@ zpl_snapdir_getattr_impl(const struct path *path, struct kstat *stat, u32 request_mask, unsigned int query_flags) #endif { + (void) request_mask, (void) query_flags; struct inode *ip = path->dentry->d_inode; zfsvfs_t *zfsvfs = ITOZSB(ip); ZPL_ENTER(zfsvfs); -#if defined(HAVE_GENERIC_FILLATTR_USERNS) && defined(HAVE_USERNS_IOPS_GETATTR) +#ifdef HAVE_USERNS_IOPS_GETATTR +#ifdef HAVE_GENERIC_FILLATTR_USERNS generic_fillattr(user_ns, ip, stat); +#else + (void) user_ns; +#endif #else generic_fillattr(ip, stat); #endif stat->nlink = stat->size = 2; + + dsl_dataset_t *ds = dmu_objset_ds(zfsvfs->z_os); + if (dsl_dataset_phys(ds)->ds_snapnames_zapobj != 0) { + uint64_t snap_count; + int err = zap_count( + dmu_objset_pool(ds->ds_objset)->dp_meta_objset, + dsl_dataset_phys(ds)->ds_snapnames_zapobj, &snap_count); + if (err != 0) { + ZPL_EXIT(zfsvfs); + return (-err); + } + stat->nlink += snap_count; + } + stat->ctime = stat->mtime = dmu_objset_snap_cmtime(zfsvfs->z_os); stat->atime = current_time(ip); ZPL_EXIT(zfsvfs); @@ -524,7 +548,6 @@ zpl_shares_readdir(struct file *filp, void *dirent, filldir_t filldir) } #endif /* !HAVE_VFS_ITERATE && !HAVE_VFS_ITERATE_SHARED */ -/* ARGSUSED */ static int #ifdef HAVE_USERNS_IOPS_GETATTR zpl_shares_getattr_impl(struct user_namespace *user_ns, @@ -535,6 +558,7 @@ zpl_shares_getattr_impl(const struct path *path, struct kstat *stat, u32 request_mask, unsigned int query_flags) #endif { + (void) request_mask, (void) query_flags; struct inode *ip = path->dentry->d_inode; zfsvfs_t *zfsvfs = ITOZSB(ip); znode_t *dzp; @@ -543,8 +567,12 @@ zpl_shares_getattr_impl(const struct path *path, struct kstat *stat, ZPL_ENTER(zfsvfs); if (zfsvfs->z_shares_dir == 0) { -#if defined(HAVE_GENERIC_FILLATTR_USERNS) && defined(HAVE_USERNS_IOPS_GETATTR) +#ifdef HAVE_USERNS_IOPS_GETATTR +#ifdef HAVE_GENERIC_FILLATTR_USERNS generic_fillattr(user_ns, path->dentry->d_inode, stat); +#else + (void) user_ns; +#endif #else generic_fillattr(path->dentry->d_inode, stat); #endif @@ -556,8 +584,12 @@ zpl_shares_getattr_impl(const struct path *path, struct kstat *stat, error = -zfs_zget(zfsvfs, zfsvfs->z_shares_dir, &dzp); if (error == 0) { -#if defined(HAVE_GENERIC_FILLATTR_USERNS) && defined(HAVE_USERNS_IOPS_GETATTR) +#ifdef HAVE_USERNS_IOPS_GETATTR +#ifdef HAVE_GENERIC_FILLATTR_USERNS error = -zfs_getattr_fast(user_ns, ZTOI(dzp), stat); +#else + (void) user_ns; +#endif #else error = -zfs_getattr_fast(kcred->user_ns, ZTOI(dzp), stat); #endif diff --git a/module/os/linux/zfs/zpl_export.c b/module/os/linux/zfs/zpl_export.c index 5be63532d329..aa80b72e2d7a 100644 --- a/module/os/linux/zfs/zpl_export.c +++ b/module/os/linux/zfs/zpl_export.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/os/linux/zfs/zpl_file.c b/module/os/linux/zfs/zpl_file.c index 982b424197e9..43b7fb60a997 100644 --- a/module/os/linux/zfs/zpl_file.c +++ b/module/os/linux/zfs/zpl_file.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -33,9 +33,13 @@ #include #include #include -#ifdef HAVE_VFS_SET_PAGE_DIRTY_NOBUFFERS +#if defined(HAVE_VFS_SET_PAGE_DIRTY_NOBUFFERS) || \ + defined(HAVE_VFS_FILEMAP_DIRTY_FOLIO) #include #endif +#ifdef HAVE_VFS_FILEMAP_DIRTY_FOLIO +#include +#endif /* * When using fallocate(2) to preallocate space, inflate the requested @@ -161,17 +165,56 @@ static int zpl_fsync(struct file *filp, loff_t start, loff_t end, int datasync) { struct inode *inode = filp->f_mapping->host; + znode_t *zp = ITOZ(inode); + zfsvfs_t *zfsvfs = ITOZSB(inode); cred_t *cr = CRED(); int error; fstrans_cookie_t cookie; + /* + * The variables z_sync_writes_cnt and z_async_writes_cnt work in + * tandem so that sync writes can detect if there are any non-sync + * writes going on and vice-versa. The "vice-versa" part to this logic + * is located in zfs_putpage() where non-sync writes check if there are + * any ongoing sync writes. If any sync and non-sync writes overlap, + * we do a commit to complete the non-sync writes since the latter can + * potentially take several seconds to complete and thus block sync + * writes in the upcoming call to filemap_write_and_wait_range(). + */ + atomic_inc_32(&zp->z_sync_writes_cnt); + /* + * If the following check does not detect an overlapping non-sync write + * (say because it's just about to start), then it is guaranteed that + * the non-sync write will detect this sync write. This is because we + * always increment z_sync_writes_cnt / z_async_writes_cnt before doing + * the check on z_async_writes_cnt / z_sync_writes_cnt here and in + * zfs_putpage() respectively. + */ + if (atomic_load_32(&zp->z_async_writes_cnt) > 0) { + ZPL_ENTER(zfsvfs); + zil_commit(zfsvfs->z_log, zp->z_id); + ZPL_EXIT(zfsvfs); + } + error = filemap_write_and_wait_range(inode->i_mapping, start, end); + + /* + * The sync write is not complete yet but we decrement + * z_sync_writes_cnt since zfs_fsync() increments and decrements + * it internally. If a non-sync write starts just after the decrement + * operation but before we call zfs_fsync(), it may not detect this + * overlapping sync write but it does not matter since we have already + * gone past filemap_write_and_wait_range() and we won't block due to + * the non-sync write. + */ + atomic_dec_32(&zp->z_sync_writes_cnt); + if (error) return (error); crhold(cr); cookie = spl_fstrans_mark(); - error = -zfs_fsync(ITOZ(inode), datasync, cr); + error = -zfs_fsync(zp, datasync, cr); spl_fstrans_unmark(cookie); crfree(cr); ASSERT3S(error, <=, 0); @@ -413,6 +456,8 @@ zpl_aio_write(struct kiocb *kiocb, const struct iovec *iov, if (ret) return (ret); + kiocb->ki_pos = pos; + zfs_uio_t uio; zfs_uio_iovec_init(&uio, iov, nr_segs, kiocb->ki_pos, UIO_USERSPACE, count, 0); @@ -629,11 +674,19 @@ zpl_readpage_common(struct page *pp) return (error); } +#ifdef HAVE_VFS_READ_FOLIO +static int +zpl_read_folio(struct file *filp, struct folio *folio) +{ + return (zpl_readpage_common(&folio->page)); +} +#else static int zpl_readpage(struct file *filp, struct page *pp) { return (zpl_readpage_common(pp)); } +#endif static int zpl_readpage_filler(void *data, struct page *pp) @@ -647,24 +700,41 @@ zpl_readpage_filler(void *data, struct page *pp) * paging. For simplicity, the code relies on read_cache_pages() to * correctly lock each page for IO and call zpl_readpage(). */ +#ifdef HAVE_VFS_READPAGES static int zpl_readpages(struct file *filp, struct address_space *mapping, struct list_head *pages, unsigned nr_pages) { return (read_cache_pages(mapping, pages, zpl_readpage_filler, NULL)); } +#else +static void +zpl_readahead(struct readahead_control *ractl) +{ + struct page *page; + + while ((page = readahead_page(ractl)) != NULL) { + int ret; + + ret = zpl_readpage_filler(NULL, page); + put_page(page); + if (ret) + break; + } +} +#endif static int zpl_putpage(struct page *pp, struct writeback_control *wbc, void *data) { - struct address_space *mapping = data; + boolean_t *for_sync = data; fstrans_cookie_t cookie; ASSERT(PageLocked(pp)); ASSERT(!PageWriteback(pp)); cookie = spl_fstrans_mark(); - (void) zfs_putpage(mapping->host, pp, wbc); + (void) zfs_putpage(pp->mapping->host, pp, wbc, *for_sync); spl_fstrans_unmark(cookie); return (0); @@ -691,8 +761,9 @@ zpl_writepages(struct address_space *mapping, struct writeback_control *wbc) * we run it once in non-SYNC mode so that the ZIL gets all the data, * and then we commit it all in one go. */ + boolean_t for_sync = (sync_mode == WB_SYNC_ALL); wbc->sync_mode = WB_SYNC_NONE; - result = write_cache_pages(mapping, wbc, zpl_putpage, mapping); + result = write_cache_pages(mapping, wbc, zpl_putpage, &for_sync); if (sync_mode != wbc->sync_mode) { ZPL_ENTER(zfsvfs); ZPL_VERIFY_ZP(zp); @@ -708,7 +779,8 @@ zpl_writepages(struct address_space *mapping, struct writeback_control *wbc) * details). That being said, this is a no-op in most cases. */ wbc->sync_mode = sync_mode; - result = write_cache_pages(mapping, wbc, zpl_putpage, mapping); + result = write_cache_pages(mapping, wbc, zpl_putpage, + &for_sync); } return (result); } @@ -725,7 +797,9 @@ zpl_writepage(struct page *pp, struct writeback_control *wbc) if (ITOZSB(pp->mapping->host)->z_os->os_sync == ZFS_SYNC_ALWAYS) wbc->sync_mode = WB_SYNC_ALL; - return (zpl_putpage(pp, wbc, pp->mapping)); + boolean_t for_sync = (wbc->sync_mode == WB_SYNC_ALL); + + return (zpl_putpage(pp, wbc, &for_sync)); } /* @@ -764,11 +838,13 @@ zpl_fallocate_common(struct inode *ip, int mode, loff_t offset, loff_t len) if (mode & (test_mode)) { flock64_t bf; - if (offset > olen) - goto out_unmark; + if (mode & FALLOC_FL_KEEP_SIZE) { + if (offset > olen) + goto out_unmark; - if (offset + len > olen) - len = olen - offset; + if (offset + len > olen) + len = olen - offset; + } bf.l_type = F_WRLCK; bf.l_whence = SEEK_SET; bf.l_start = offset; @@ -905,21 +981,24 @@ __zpl_ioctl_setflags(struct inode *ip, uint32_t ioctl_flags, xvattr_t *xva) xva_init(xva); xoap = xva_getxoptattr(xva); - XVA_SET_REQ(xva, XAT_IMMUTABLE); - if (ioctl_flags & FS_IMMUTABLE_FL) - xoap->xoa_immutable = B_TRUE; - - XVA_SET_REQ(xva, XAT_APPENDONLY); - if (ioctl_flags & FS_APPEND_FL) - xoap->xoa_appendonly = B_TRUE; - - XVA_SET_REQ(xva, XAT_NODUMP); - if (ioctl_flags & FS_NODUMP_FL) - xoap->xoa_nodump = B_TRUE; - - XVA_SET_REQ(xva, XAT_PROJINHERIT); - if (ioctl_flags & ZFS_PROJINHERIT_FL) - xoap->xoa_projinherit = B_TRUE; +#define FLAG_CHANGE(iflag, zflag, xflag, xfield) do { \ + if (((ioctl_flags & (iflag)) && !(zfs_flags & (zflag))) || \ + ((zfs_flags & (zflag)) && !(ioctl_flags & (iflag)))) { \ + XVA_SET_REQ(xva, (xflag)); \ + (xfield) = ((ioctl_flags & (iflag)) != 0); \ + } \ +} while (0) + + FLAG_CHANGE(FS_IMMUTABLE_FL, ZFS_IMMUTABLE, XAT_IMMUTABLE, + xoap->xoa_immutable); + FLAG_CHANGE(FS_APPEND_FL, ZFS_APPENDONLY, XAT_APPENDONLY, + xoap->xoa_appendonly); + FLAG_CHANGE(FS_NODUMP_FL, ZFS_NODUMP, XAT_NODUMP, + xoap->xoa_nodump); + FLAG_CHANGE(ZFS_PROJINHERIT_FL, ZFS_PROJINHERIT, XAT_PROJINHERIT, + xoap->xoa_projinherit); + +#undef FLAG_CHANGE return (0); } @@ -998,6 +1077,94 @@ zpl_ioctl_setxattr(struct file *filp, void __user *arg) return (err); } +/* + * Expose Additional File Level Attributes of ZFS. + */ +static int +zpl_ioctl_getdosflags(struct file *filp, void __user *arg) +{ + struct inode *ip = file_inode(filp); + uint64_t dosflags = ITOZ(ip)->z_pflags; + dosflags &= ZFS_DOS_FL_USER_VISIBLE; + int err = copy_to_user(arg, &dosflags, sizeof (dosflags)); + + return (err); +} + +static int +__zpl_ioctl_setdosflags(struct inode *ip, uint64_t ioctl_flags, xvattr_t *xva) +{ + uint64_t zfs_flags = ITOZ(ip)->z_pflags; + xoptattr_t *xoap; + + if (ioctl_flags & (~ZFS_DOS_FL_USER_VISIBLE)) + return (-EOPNOTSUPP); + + if ((fchange(ioctl_flags, zfs_flags, ZFS_IMMUTABLE, ZFS_IMMUTABLE) || + fchange(ioctl_flags, zfs_flags, ZFS_APPENDONLY, ZFS_APPENDONLY)) && + !capable(CAP_LINUX_IMMUTABLE)) + return (-EPERM); + + if (!zpl_inode_owner_or_capable(kcred->user_ns, ip)) + return (-EACCES); + + xva_init(xva); + xoap = xva_getxoptattr(xva); + +#define FLAG_CHANGE(iflag, xflag, xfield) do { \ + if (((ioctl_flags & (iflag)) && !(zfs_flags & (iflag))) || \ + ((zfs_flags & (iflag)) && !(ioctl_flags & (iflag)))) { \ + XVA_SET_REQ(xva, (xflag)); \ + (xfield) = ((ioctl_flags & (iflag)) != 0); \ + } \ +} while (0) + + FLAG_CHANGE(ZFS_IMMUTABLE, XAT_IMMUTABLE, xoap->xoa_immutable); + FLAG_CHANGE(ZFS_APPENDONLY, XAT_APPENDONLY, xoap->xoa_appendonly); + FLAG_CHANGE(ZFS_NODUMP, XAT_NODUMP, xoap->xoa_nodump); + FLAG_CHANGE(ZFS_READONLY, XAT_READONLY, xoap->xoa_readonly); + FLAG_CHANGE(ZFS_HIDDEN, XAT_HIDDEN, xoap->xoa_hidden); + FLAG_CHANGE(ZFS_SYSTEM, XAT_SYSTEM, xoap->xoa_system); + FLAG_CHANGE(ZFS_ARCHIVE, XAT_ARCHIVE, xoap->xoa_archive); + FLAG_CHANGE(ZFS_NOUNLINK, XAT_NOUNLINK, xoap->xoa_nounlink); + FLAG_CHANGE(ZFS_REPARSE, XAT_REPARSE, xoap->xoa_reparse); + FLAG_CHANGE(ZFS_OFFLINE, XAT_OFFLINE, xoap->xoa_offline); + FLAG_CHANGE(ZFS_SPARSE, XAT_SPARSE, xoap->xoa_sparse); + +#undef FLAG_CHANGE + + return (0); +} + +/* + * Set Additional File Level Attributes of ZFS. + */ +static int +zpl_ioctl_setdosflags(struct file *filp, void __user *arg) +{ + struct inode *ip = file_inode(filp); + uint64_t dosflags; + cred_t *cr = CRED(); + xvattr_t xva; + int err; + fstrans_cookie_t cookie; + + if (copy_from_user(&dosflags, arg, sizeof (dosflags))) + return (-EFAULT); + + err = __zpl_ioctl_setdosflags(ip, dosflags, &xva); + if (err) + return (err); + + crhold(cr); + cookie = spl_fstrans_mark(); + err = -zfs_setattr(ITOZ(ip), (vattr_t *)&xva, 0, cr); + spl_fstrans_unmark(cookie); + crfree(cr); + + return (err); +} + static long zpl_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) { @@ -1012,6 +1179,10 @@ zpl_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) return (zpl_ioctl_getxattr(filp, (void *)arg)); case ZFS_IOC_FSSETXATTR: return (zpl_ioctl_setxattr(filp, (void *)arg)); + case ZFS_IOC_GETDOSFLAGS: + return (zpl_ioctl_getdosflags(filp, (void *)arg)); + case ZFS_IOC_SETDOSFLAGS: + return (zpl_ioctl_setdosflags(filp, (void *)arg)); default: return (-ENOTTY); } @@ -1040,14 +1211,25 @@ zpl_compat_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) const struct address_space_operations zpl_address_space_operations = { +#ifdef HAVE_VFS_READPAGES .readpages = zpl_readpages, +#else + .readahead = zpl_readahead, +#endif +#ifdef HAVE_VFS_READ_FOLIO + .read_folio = zpl_read_folio, +#else .readpage = zpl_readpage, +#endif .writepage = zpl_writepage, .writepages = zpl_writepages, .direct_IO = zpl_direct_IO, #ifdef HAVE_VFS_SET_PAGE_DIRTY_NOBUFFERS .set_page_dirty = __set_page_dirty_nobuffers, #endif +#ifdef HAVE_VFS_FILEMAP_DIRTY_FOLIO + .dirty_folio = filemap_dirty_folio, +#endif }; const struct file_operations zpl_file_operations = { diff --git a/module/os/linux/zfs/zpl_inode.c b/module/os/linux/zfs/zpl_inode.c index 24a8b036bf0f..7578753ed8ce 100644 --- a/module/os/linux/zfs/zpl_inode.c +++ b/module/os/linux/zfs/zpl_inode.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -116,14 +116,14 @@ zpl_vap_init(vattr_t *vap, struct inode *dir, umode_t mode, cred_t *cr) { vap->va_mask = ATTR_MODE; vap->va_mode = mode; - vap->va_uid = crgetfsuid(cr); + vap->va_uid = crgetuid(cr); if (dir && dir->i_mode & S_ISGID) { vap->va_gid = KGID_TO_SGID(dir->i_gid); if (S_ISDIR(mode)) vap->va_mode |= S_ISGID; } else { - vap->va_gid = crgetfsgid(cr); + vap->va_gid = crgetgid(cr); } } diff --git a/module/os/linux/zfs/zpl_super.c b/module/os/linux/zfs/zpl_super.c index c2fd3fee1401..cf879a2897b3 100644 --- a/module/os/linux/zfs/zpl_super.c +++ b/module/os/linux/zfs/zpl_super.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -233,6 +233,18 @@ __zpl_show_options(struct seq_file *seq, zfsvfs_t *zfsvfs) } #endif /* CONFIG_FS_POSIX_ACL */ + switch (zfsvfs->z_case) { + case ZFS_CASE_SENSITIVE: + seq_puts(seq, ",casesensitive"); + break; + case ZFS_CASE_INSENSITIVE: + seq_puts(seq, ",caseinsensitive"); + break; + default: + seq_puts(seq, ",casemixed"); + break; + } + return (0); } @@ -360,6 +372,7 @@ const struct super_operations zpl_super_operations = { struct file_system_type zpl_fs_type = { .owner = THIS_MODULE, .name = ZFS_DRIVER, + .fs_flags = FS_USERNS_MOUNT, .mount = zpl_mount, .kill_sb = zpl_kill_sb, }; diff --git a/module/os/linux/zfs/zpl_xattr.c b/module/os/linux/zfs/zpl_xattr.c index a1921ed08863..e7e299dcf1cd 100644 --- a/module/os/linux/zfs/zpl_xattr.c +++ b/module/os/linux/zfs/zpl_xattr.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -83,6 +83,13 @@ #include #include #include +#include + +enum xattr_permission { + XAPERM_DENY, + XAPERM_ALLOW, + XAPERM_COMPAT, +}; typedef struct xattr_filldir { size_t size; @@ -91,33 +98,10 @@ typedef struct xattr_filldir { struct dentry *dentry; } xattr_filldir_t; -static const struct xattr_handler *zpl_xattr_handler(const char *); - -static int -zpl_xattr_permission(xattr_filldir_t *xf, const char *name, int name_len) -{ - static const struct xattr_handler *handler; - struct dentry *d = xf->dentry; - - handler = zpl_xattr_handler(name); - if (!handler) - return (0); - - if (handler->list) { -#if defined(HAVE_XATTR_LIST_SIMPLE) - if (!handler->list(d)) - return (0); -#elif defined(HAVE_XATTR_LIST_DENTRY) - if (!handler->list(d, NULL, 0, name, name_len, 0)) - return (0); -#elif defined(HAVE_XATTR_LIST_HANDLER) - if (!handler->list(handler, d, NULL, 0, name, name_len)) - return (0); -#endif - } +static enum xattr_permission zpl_xattr_permission(xattr_filldir_t *, + const char *, int); - return (1); -} +static int zfs_xattr_compat = 0; /* * Determine is a given xattr name should be visible and if so copy it @@ -126,10 +110,27 @@ zpl_xattr_permission(xattr_filldir_t *xf, const char *name, int name_len) static int zpl_xattr_filldir(xattr_filldir_t *xf, const char *name, int name_len) { + enum xattr_permission perm; + /* Check permissions using the per-namespace list xattr handler. */ - if (!zpl_xattr_permission(xf, name, name_len)) + perm = zpl_xattr_permission(xf, name, name_len); + if (perm == XAPERM_DENY) return (0); + /* Prefix the name with "user." if it does not have a namespace. */ + if (perm == XAPERM_COMPAT) { + if (xf->buf) { + if (xf->offset + XATTR_USER_PREFIX_LEN + 1 > xf->size) + return (-ERANGE); + + memcpy(xf->buf + xf->offset, XATTR_USER_PREFIX, + XATTR_USER_PREFIX_LEN); + xf->buf[xf->offset + XATTR_USER_PREFIX_LEN] = '\0'; + } + + xf->offset += XATTR_USER_PREFIX_LEN; + } + /* When xf->buf is NULL only calculate the required size. */ if (xf->buf) { if (xf->offset + name_len + 1 > xf->size) @@ -492,8 +493,8 @@ zpl_xattr_set_dir(struct inode *ip, const char *name, const void *value, vap = kmem_zalloc(sizeof (vattr_t), KM_SLEEP); vap->va_mode = xattr_mode; vap->va_mask = ATTR_MODE; - vap->va_uid = crgetfsuid(cr); - vap->va_gid = crgetfsgid(cr); + vap->va_uid = crgetuid(cr); + vap->va_gid = crgetgid(cr); error = -zfs_create(dxzp, (char *)name, vap, 0, 0644, &xzp, cr, 0, NULL); @@ -578,7 +579,7 @@ zpl_xattr_set_sa(struct inode *ip, const char *name, const void *value, * will be reconstructed from the ARC when next accessed. */ if (error == 0) - error = -zfs_sa_set_xattr(zp); + error = -zfs_sa_set_xattr(zp, name, value, size); if (error) { nvlist_free(nvl); @@ -706,19 +707,28 @@ static int __zpl_xattr_user_get(struct inode *ip, const char *name, void *value, size_t size) { - char *xattr_name; int error; /* xattr_resolve_name will do this for us if this is defined */ #ifndef HAVE_XATTR_HANDLER_NAME if (strcmp(name, "") == 0) return (-EINVAL); #endif + if (ZFS_XA_NS_PREFIX_FORBIDDEN(name)) + return (-EINVAL); if (!(ITOZSB(ip)->z_flags & ZSB_XATTR)) return (-EOPNOTSUPP); - xattr_name = kmem_asprintf("%s%s", XATTR_USER_PREFIX, name); + /* + * Try to look up the name with the namespace prefix first for + * compatibility with xattrs from this platform. If that fails, + * try again without the namespace prefix for compatibility with + * other platforms. + */ + char *xattr_name = kmem_asprintf("%s%s", XATTR_USER_PREFIX, name); error = zpl_xattr_get(ip, xattr_name, value, size); kmem_strfree(xattr_name); + if (error == -ENODATA) + error = zpl_xattr_get(ip, name, value, size); return (error); } @@ -728,20 +738,59 @@ static int __zpl_xattr_user_set(struct inode *ip, const char *name, const void *value, size_t size, int flags) { - char *xattr_name; - int error; + int error = 0; /* xattr_resolve_name will do this for us if this is defined */ #ifndef HAVE_XATTR_HANDLER_NAME if (strcmp(name, "") == 0) return (-EINVAL); #endif + if (ZFS_XA_NS_PREFIX_FORBIDDEN(name)) + return (-EINVAL); if (!(ITOZSB(ip)->z_flags & ZSB_XATTR)) return (-EOPNOTSUPP); - xattr_name = kmem_asprintf("%s%s", XATTR_USER_PREFIX, name); - error = zpl_xattr_set(ip, xattr_name, value, size, flags); - kmem_strfree(xattr_name); - + /* + * Remove alternate compat version of the xattr so we only set the + * version specified by the zfs_xattr_compat tunable. + * + * The following flags must be handled correctly: + * + * XATTR_CREATE: fail if xattr already exists + * XATTR_REPLACE: fail if xattr does not exist + */ + char *prefixed_name = kmem_asprintf("%s%s", XATTR_USER_PREFIX, name); + const char *clear_name, *set_name; + if (zfs_xattr_compat) { + clear_name = prefixed_name; + set_name = name; + } else { + clear_name = name; + set_name = prefixed_name; + } + /* + * Clear the old value with the alternative name format, if it exists. + */ + error = zpl_xattr_set(ip, clear_name, NULL, 0, flags); + /* + * XATTR_CREATE was specified and we failed to clear the xattr + * because it already exists. Stop here. + */ + if (error == -EEXIST) + goto out; + /* + * If XATTR_REPLACE was specified and we succeeded to clear + * an xattr, we don't need to replace anything when setting + * the new value. If we failed with -ENODATA that's fine, + * there was nothing to be cleared and we can ignore the error. + */ + if (error == 0) + flags &= ~XATTR_REPLACE; + /* + * Set the new value with the configured name format. + */ + error = zpl_xattr_set(ip, set_name, value, size, flags); +out: + kmem_strfree(prefixed_name); return (error); } ZPL_XATTR_SET_WRAPPER(zpl_xattr_user_set); @@ -952,7 +1001,7 @@ zpl_set_acl_impl(struct inode *ip, struct posix_acl *acl, int type) * the inode to write the Posix mode bits. */ if (ip->i_mode != mode) { - ip->i_mode = mode; + ip->i_mode = ITOZ(ip)->z_mode = mode; ip->i_ctime = current_time(ip); zfs_mark_inode_dirty(ip); } @@ -1099,7 +1148,7 @@ zpl_init_acl(struct inode *ip, struct inode *dir) if (IS_ERR(acl)) return (PTR_ERR(acl)); if (!acl) { - ip->i_mode &= ~current_umask(); + ITOZ(ip)->z_mode = (ip->i_mode &= ~current_umask()); ip->i_ctime = current_time(ip); zfs_mark_inode_dirty(ip); return (0); @@ -1118,7 +1167,7 @@ zpl_init_acl(struct inode *ip, struct inode *dir) mode = ip->i_mode; error = __posix_acl_create(&acl, GFP_KERNEL, &mode); if (error >= 0) { - ip->i_mode = mode; + ip->i_mode = ITOZ(ip)->z_mode = mode; zfs_mark_inode_dirty(ip); if (error > 0) { error = zpl_set_acl_impl(ip, acl, @@ -1411,7 +1460,45 @@ zpl_xattr_handler(const char *name) return (NULL); } -#if !defined(HAVE_POSIX_ACL_RELEASE) || defined(HAVE_POSIX_ACL_RELEASE_GPL_ONLY) +static enum xattr_permission +zpl_xattr_permission(xattr_filldir_t *xf, const char *name, int name_len) +{ + const struct xattr_handler *handler; + struct dentry *d __maybe_unused = xf->dentry; + enum xattr_permission perm = XAPERM_ALLOW; + + handler = zpl_xattr_handler(name); + if (handler == NULL) { + /* Do not expose FreeBSD system namespace xattrs. */ + if (ZFS_XA_NS_PREFIX_MATCH(FREEBSD, name)) + return (XAPERM_DENY); + /* + * Anything that doesn't match a known namespace gets put in the + * user namespace for compatibility with other platforms. + */ + perm = XAPERM_COMPAT; + handler = &zpl_xattr_user_handler; + } + + if (handler->list) { +#if defined(HAVE_XATTR_LIST_SIMPLE) + if (!handler->list(d)) + return (XAPERM_DENY); +#elif defined(HAVE_XATTR_LIST_DENTRY) + if (!handler->list(d, NULL, 0, name, name_len, 0)) + return (XAPERM_DENY); +#elif defined(HAVE_XATTR_LIST_HANDLER) + if (!handler->list(handler, d, NULL, 0, name, name_len)) + return (XAPERM_DENY); +#endif + } + + return (perm); +} + +#if defined(CONFIG_FS_POSIX_ACL) && \ + (!defined(HAVE_POSIX_ACL_RELEASE) || \ + defined(HAVE_POSIX_ACL_RELEASE_GPL_ONLY)) struct acl_rel_struct { struct acl_rel_struct *next; struct posix_acl *acl; @@ -1510,3 +1597,6 @@ zpl_posix_acl_release_impl(struct posix_acl *acl) NULL, TQ_SLEEP, ddi_get_lbolt() + ACL_REL_SCHED); } #endif + +ZFS_MODULE_PARAM(zfs, zfs_, xattr_compat, INT, ZMOD_RW, + "Use legacy ZFS xattr naming for writing new user namespace xattrs"); diff --git a/module/os/linux/zfs/zvol_os.c b/module/os/linux/zfs/zvol_os.c index f772f416043e..18b9fbd0ea55 100644 --- a/module/os/linux/zfs/zvol_os.c +++ b/module/os/linux/zfs/zvol_os.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -41,17 +41,77 @@ #include #include +#ifdef HAVE_BLK_MQ +#include +#endif + +static void zvol_request_impl(zvol_state_t *zv, struct bio *bio, + struct request *rq, boolean_t force_sync); + static unsigned int zvol_major = ZVOL_MAJOR; static unsigned int zvol_request_sync = 0; static unsigned int zvol_prefetch_bytes = (128 * 1024); static unsigned long zvol_max_discard_blocks = 16384; -static unsigned int zvol_threads = 32; + +#ifndef HAVE_BLKDEV_GET_ERESTARTSYS static const unsigned int zvol_open_timeout_ms = 1000; +#endif + +static unsigned int zvol_threads = 0; +#ifdef HAVE_BLK_MQ +static unsigned int zvol_blk_mq_threads = 0; +static unsigned int zvol_blk_mq_actual_threads; +static boolean_t zvol_use_blk_mq = B_FALSE; + +/* + * The maximum number of volblocksize blocks to process per thread. Typically, + * write heavy workloads preform better with higher values here, and read + * heavy workloads preform better with lower values, but that's not a hard + * and fast rule. It's basically a knob to tune between "less overhead with + * less parallelism" and "more overhead, but more parallelism". + * + * '8' was chosen as a reasonable, balanced, default based off of sequential + * read and write tests to a zvol in an NVMe pool (with 16 CPUs). + */ +static unsigned int zvol_blk_mq_blocks_per_thread = 8; +#endif + +#ifndef BLKDEV_DEFAULT_RQ +/* BLKDEV_MAX_RQ was renamed to BLKDEV_DEFAULT_RQ in the 5.16 kernel */ +#define BLKDEV_DEFAULT_RQ BLKDEV_MAX_RQ +#endif + +/* + * Finalize our BIO or request. + */ +#ifdef HAVE_BLK_MQ +#define END_IO(zv, bio, rq, error) do { \ + if (bio) { \ + BIO_END_IO(bio, error); \ + } else { \ + blk_mq_end_request(rq, errno_to_bi_status(error)); \ + } \ +} while (0) +#else +#define END_IO(zv, bio, rq, error) BIO_END_IO(bio, error) +#endif + +#ifdef HAVE_BLK_MQ +static unsigned int zvol_blk_mq_queue_depth = BLKDEV_DEFAULT_RQ; +static unsigned int zvol_actual_blk_mq_queue_depth; +#endif struct zvol_state_os { struct gendisk *zvo_disk; /* generic disk */ struct request_queue *zvo_queue; /* request queue */ dev_t zvo_dev; /* device id */ + +#ifdef HAVE_BLK_MQ + struct blk_mq_tag_set tag_set; +#endif + + /* Set from the global 'zvol_use_blk_mq' at zvol load */ + boolean_t use_blk_mq; }; taskq_t *zvol_taskq; @@ -60,8 +120,14 @@ static struct ida zvol_ida; typedef struct zv_request_stack { zvol_state_t *zv; struct bio *bio; + struct request *rq; } zv_request_t; +typedef struct zv_work { + struct request *rq; + struct work_struct work; +} zv_work_t; + typedef struct zv_request_task { zv_request_t zvr; taskq_ent_t ent; @@ -83,6 +149,62 @@ zv_request_task_free(zv_request_task_t *task) kmem_free(task, sizeof (*task)); } +#ifdef HAVE_BLK_MQ + +/* + * This is called when a new block multiqueue request comes in. A request + * contains one or more BIOs. + */ +static blk_status_t zvol_mq_queue_rq(struct blk_mq_hw_ctx *hctx, + const struct blk_mq_queue_data *bd) +{ + struct request *rq = bd->rq; + zvol_state_t *zv = rq->q->queuedata; + + /* Tell the kernel that we are starting to process this request */ + blk_mq_start_request(rq); + + if (blk_rq_is_passthrough(rq)) { + /* Skip non filesystem request */ + blk_mq_end_request(rq, BLK_STS_IOERR); + return (BLK_STS_IOERR); + } + + zvol_request_impl(zv, NULL, rq, 0); + + /* Acknowledge to the kernel that we got this request */ + return (BLK_STS_OK); +} + +static struct blk_mq_ops zvol_blk_mq_queue_ops = { + .queue_rq = zvol_mq_queue_rq, +}; + +/* Initialize our blk-mq struct */ +static int zvol_blk_mq_alloc_tag_set(zvol_state_t *zv) +{ + struct zvol_state_os *zso = zv->zv_zso; + + memset(&zso->tag_set, 0, sizeof (zso->tag_set)); + + /* Initialize tag set. */ + zso->tag_set.ops = &zvol_blk_mq_queue_ops; + zso->tag_set.nr_hw_queues = zvol_blk_mq_actual_threads; + zso->tag_set.queue_depth = zvol_actual_blk_mq_queue_depth; + zso->tag_set.numa_node = NUMA_NO_NODE; + zso->tag_set.cmd_size = 0; + + /* + * We need BLK_MQ_F_BLOCKING here since we do blocking calls in + * zvol_request_impl() + */ + zso->tag_set.flags = BLK_MQ_F_SHOULD_MERGE | BLK_MQ_F_BLOCKING; + zso->tag_set.driver_data = zv; + + return (blk_mq_alloc_tag_set(&zso->tag_set)); +} +#endif /* HAVE_BLK_MQ */ + /* * Given a path, return TRUE if path is a ZVOL. */ @@ -104,38 +226,51 @@ static void zvol_write(zv_request_t *zvr) { struct bio *bio = zvr->bio; + struct request *rq = zvr->rq; int error = 0; zfs_uio_t uio; - - zfs_uio_bvec_init(&uio, bio); - zvol_state_t *zv = zvr->zv; + struct request_queue *q; + struct gendisk *disk; + unsigned long start_time = 0; + boolean_t acct = B_FALSE; + ASSERT3P(zv, !=, NULL); ASSERT3U(zv->zv_open_count, >, 0); ASSERT3P(zv->zv_zilog, !=, NULL); + q = zv->zv_zso->zvo_queue; + disk = zv->zv_zso->zvo_disk; + /* bio marked as FLUSH need to flush before write */ - if (bio_is_flush(bio)) + if (io_is_flush(bio, rq)) zil_commit(zv->zv_zilog, ZVOL_OBJ); /* Some requests are just for flush and nothing else. */ - if (uio.uio_resid == 0) { + if (io_size(bio, rq) == 0) { rw_exit(&zv->zv_suspend_lock); - BIO_END_IO(bio, 0); + END_IO(zv, bio, rq, 0); return; } - struct request_queue *q = zv->zv_zso->zvo_queue; - struct gendisk *disk = zv->zv_zso->zvo_disk; + zfs_uio_bvec_init(&uio, bio, rq); + ssize_t start_resid = uio.uio_resid; - unsigned long start_time; - boolean_t acct = blk_queue_io_stat(q); - if (acct) - start_time = blk_generic_start_io_acct(q, disk, WRITE, bio); + /* + * With use_blk_mq, accounting is done by blk_mq_start_request() + * and blk_mq_end_request(), so we can skip it here. + */ + if (bio) { + acct = blk_queue_io_stat(q); + if (acct) { + start_time = blk_generic_start_io_acct(q, disk, WRITE, + bio); + } + } boolean_t sync = - bio_is_fua(bio) || zv->zv_objset->os_sync == ZFS_SYNC_ALWAYS; + io_is_fua(bio, rq) || zv->zv_objset->os_sync == ZFS_SYNC_ALWAYS; zfs_locked_range_t *lr = zfs_rangelock_enter(&zv->zv_rangelock, uio.uio_loffset, uio.uio_resid, RL_WRITER); @@ -177,10 +312,11 @@ zvol_write(zv_request_t *zvr) rw_exit(&zv->zv_suspend_lock); - if (acct) + if (bio && acct) { blk_generic_end_io_acct(q, disk, WRITE, bio, start_time); + } - BIO_END_IO(bio, -error); + END_IO(zv, bio, rq, -error); } static void @@ -195,27 +331,33 @@ static void zvol_discard(zv_request_t *zvr) { struct bio *bio = zvr->bio; + struct request *rq = zvr->rq; zvol_state_t *zv = zvr->zv; - uint64_t start = BIO_BI_SECTOR(bio) << 9; - uint64_t size = BIO_BI_SIZE(bio); + uint64_t start = io_offset(bio, rq); + uint64_t size = io_size(bio, rq); uint64_t end = start + size; boolean_t sync; int error = 0; dmu_tx_t *tx; + struct request_queue *q = zv->zv_zso->zvo_queue; + struct gendisk *disk = zv->zv_zso->zvo_disk; + unsigned long start_time = 0; + + boolean_t acct = blk_queue_io_stat(q); ASSERT3P(zv, !=, NULL); ASSERT3U(zv->zv_open_count, >, 0); ASSERT3P(zv->zv_zilog, !=, NULL); - struct request_queue *q = zv->zv_zso->zvo_queue; - struct gendisk *disk = zv->zv_zso->zvo_disk; - unsigned long start_time; - - boolean_t acct = blk_queue_io_stat(q); - if (acct) - start_time = blk_generic_start_io_acct(q, disk, WRITE, bio); + if (bio) { + acct = blk_queue_io_stat(q); + if (acct) { + start_time = blk_generic_start_io_acct(q, disk, WRITE, + bio); + } + } - sync = bio_is_fua(bio) || zv->zv_objset->os_sync == ZFS_SYNC_ALWAYS; + sync = io_is_fua(bio, rq) || zv->zv_objset->os_sync == ZFS_SYNC_ALWAYS; if (end > zv->zv_volsize) { error = SET_ERROR(EIO); @@ -228,7 +370,7 @@ zvol_discard(zv_request_t *zvr) * the unaligned parts which is slow (read-modify-write) and useless * since we are not freeing any space by doing so. */ - if (!bio_is_secure_erase(bio)) { + if (!io_is_secure_erase(bio, rq)) { start = P2ROUNDUP(start, zv->zv_volblocksize); end = P2ALIGN(end, zv->zv_volblocksize); size = end - start; @@ -259,10 +401,12 @@ zvol_discard(zv_request_t *zvr) unlock: rw_exit(&zv->zv_suspend_lock); - if (acct) - blk_generic_end_io_acct(q, disk, WRITE, bio, start_time); + if (bio && acct) { + blk_generic_end_io_acct(q, disk, WRITE, bio, + start_time); + } - BIO_END_IO(bio, -error); + END_IO(zv, bio, rq, -error); } static void @@ -277,28 +421,41 @@ static void zvol_read(zv_request_t *zvr) { struct bio *bio = zvr->bio; + struct request *rq = zvr->rq; int error = 0; zfs_uio_t uio; - - zfs_uio_bvec_init(&uio, bio); - + boolean_t acct = B_FALSE; zvol_state_t *zv = zvr->zv; + struct request_queue *q; + struct gendisk *disk; + unsigned long start_time = 0; + ASSERT3P(zv, !=, NULL); ASSERT3U(zv->zv_open_count, >, 0); - struct request_queue *q = zv->zv_zso->zvo_queue; - struct gendisk *disk = zv->zv_zso->zvo_disk; + zfs_uio_bvec_init(&uio, bio, rq); + + q = zv->zv_zso->zvo_queue; + disk = zv->zv_zso->zvo_disk; + ssize_t start_resid = uio.uio_resid; - unsigned long start_time; - boolean_t acct = blk_queue_io_stat(q); - if (acct) - start_time = blk_generic_start_io_acct(q, disk, READ, bio); + /* + * When blk-mq is being used, accounting is done by + * blk_mq_start_request() and blk_mq_end_request(). + */ + if (bio) { + acct = blk_queue_io_stat(q); + if (acct) + start_time = blk_generic_start_io_acct(q, disk, READ, + bio); + } zfs_locked_range_t *lr = zfs_rangelock_enter(&zv->zv_rangelock, uio.uio_loffset, uio.uio_resid, RL_READER); uint64_t volsize = zv->zv_volsize; + while (uio.uio_resid > 0 && uio.uio_loffset < volsize) { uint64_t bytes = MIN(uio.uio_resid, DMU_MAX_ACCESS >> 1); @@ -322,10 +479,11 @@ zvol_read(zv_request_t *zvr) rw_exit(&zv->zv_suspend_lock); - if (acct) + if (bio && acct) { blk_generic_end_io_acct(q, disk, READ, bio, start_time); + } - BIO_END_IO(bio, -error); + END_IO(zv, bio, rq, -error); } static void @@ -336,52 +494,49 @@ zvol_read_task(void *arg) zv_request_task_free(task); } -#ifdef HAVE_SUBMIT_BIO_IN_BLOCK_DEVICE_OPERATIONS -#ifdef HAVE_BDEV_SUBMIT_BIO_RETURNS_VOID + +/* + * Process a BIO or request + * + * Either 'bio' or 'rq' should be set depending on if we are processing a + * bio or a request (both should not be set). + * + * force_sync: Set to 0 to defer processing to a background taskq + * Set to 1 to process data synchronously + */ static void -zvol_submit_bio(struct bio *bio) -#else -static blk_qc_t -zvol_submit_bio(struct bio *bio) -#endif -#else -static MAKE_REQUEST_FN_RET -zvol_request(struct request_queue *q, struct bio *bio) -#endif +zvol_request_impl(zvol_state_t *zv, struct bio *bio, struct request *rq, + boolean_t force_sync) { -#ifdef HAVE_SUBMIT_BIO_IN_BLOCK_DEVICE_OPERATIONS -#if defined(HAVE_BIO_BDEV_DISK) - struct request_queue *q = bio->bi_bdev->bd_disk->queue; -#else - struct request_queue *q = bio->bi_disk->queue; -#endif -#endif - zvol_state_t *zv = q->queuedata; fstrans_cookie_t cookie = spl_fstrans_mark(); - uint64_t offset = BIO_BI_SECTOR(bio) << 9; - uint64_t size = BIO_BI_SIZE(bio); - int rw = bio_data_dir(bio); + uint64_t offset = io_offset(bio, rq); + uint64_t size = io_size(bio, rq); + int rw = io_data_dir(bio, rq); + + if (zvol_request_sync) + force_sync = 1; + + zv_request_t zvr = { + .zv = zv, + .bio = bio, + .rq = rq, + }; - if (bio_has_data(bio) && offset + size > zv->zv_volsize) { - printk(KERN_INFO - "%s: bad access: offset=%llu, size=%lu\n", + if (io_has_data(bio, rq) && offset + size > zv->zv_volsize) { + printk(KERN_INFO "%s: bad access: offset=%llu, size=%lu\n", zv->zv_zso->zvo_disk->disk_name, (long long unsigned)offset, (long unsigned)size); - BIO_END_IO(bio, -SET_ERROR(EIO)); + END_IO(zv, bio, rq, -SET_ERROR(EIO)); goto out; } - zv_request_t zvr = { - .zv = zv, - .bio = bio, - }; zv_request_task_t *task; if (rw == WRITE) { if (unlikely(zv->zv_flags & ZVOL_RDONLY)) { - BIO_END_IO(bio, -SET_ERROR(EROFS)); + END_IO(zv, bio, rq, -SET_ERROR(EROFS)); goto out; } @@ -403,7 +558,7 @@ zvol_request(struct request_queue *q, struct bio *bio) rw_enter(&zv->zv_suspend_lock, RW_WRITER); if (zv->zv_zilog == NULL) { zv->zv_zilog = zil_open(zv->zv_objset, - zvol_get_data); + zvol_get_data, &zv->zv_kstat.dk_zil_sums); zv->zv_flags |= ZVOL_WRITTEN_TO; /* replay / destroy done in zvol_create_minor */ VERIFY0((zv->zv_zilog->zl_header->zh_flags & @@ -418,7 +573,7 @@ zvol_request(struct request_queue *q, struct bio *bio) * i/o may be a ZIL write (via zil_commit()), or a read of an * indirect block, or a read of a data block (if this is a * partial-block write). We will indicate that the i/o is - * complete by calling BIO_END_IO() from the taskq callback. + * complete by calling END_IO() from the taskq callback. * * This design allows the calling thread to continue and * initiate more concurrent operations by calling @@ -438,12 +593,12 @@ zvol_request(struct request_queue *q, struct bio *bio) * of one i/o at a time per zvol. However, an even better * design would be for zvol_request() to initiate the zio * directly, and then be notified by the zio_done callback, - * which would call BIO_END_IO(). Unfortunately, the DMU/ZIL + * which would call END_IO(). Unfortunately, the DMU/ZIL * interfaces lack this functionality (they block waiting for * the i/o to complete). */ - if (bio_is_discard(bio) || bio_is_secure_erase(bio)) { - if (zvol_request_sync) { + if (io_is_discard(bio, rq) || io_is_secure_erase(bio, rq)) { + if (force_sync) { zvol_discard(&zvr); } else { task = zv_request_task_create(zvr); @@ -451,7 +606,7 @@ zvol_request(struct request_queue *q, struct bio *bio) zvol_discard_task, task, 0, &task->ent); } } else { - if (zvol_request_sync) { + if (force_sync) { zvol_write(&zvr); } else { task = zv_request_task_create(zvr); @@ -466,14 +621,14 @@ zvol_request(struct request_queue *q, struct bio *bio) * data and require no additional handling. */ if (size == 0) { - BIO_END_IO(bio, 0); + END_IO(zv, bio, rq, 0); goto out; } rw_enter(&zv->zv_suspend_lock, RW_READER); /* See comment in WRITE case above. */ - if (zvol_request_sync) { + if (force_sync) { zvol_read(&zvr); } else { task = zv_request_task_create(zvr); @@ -484,8 +639,33 @@ zvol_request(struct request_queue *q, struct bio *bio) out: spl_fstrans_unmark(cookie); -#if (defined(HAVE_MAKE_REQUEST_FN_RET_QC) || \ - defined(HAVE_SUBMIT_BIO_IN_BLOCK_DEVICE_OPERATIONS)) && \ +} + +#ifdef HAVE_SUBMIT_BIO_IN_BLOCK_DEVICE_OPERATIONS +#ifdef HAVE_BDEV_SUBMIT_BIO_RETURNS_VOID +static void +zvol_submit_bio(struct bio *bio) +#else +static blk_qc_t +zvol_submit_bio(struct bio *bio) +#endif +#else +static MAKE_REQUEST_FN_RET +zvol_request(struct request_queue *q, struct bio *bio) +#endif +{ +#ifdef HAVE_SUBMIT_BIO_IN_BLOCK_DEVICE_OPERATIONS +#if defined(HAVE_BIO_BDEV_DISK) + struct request_queue *q = bio->bi_bdev->bd_disk->queue; +#else + struct request_queue *q = bio->bi_disk->queue; +#endif +#endif + zvol_state_t *zv = q->queuedata; + + zvol_request_impl(zv, bio, NULL, 0); +#if defined(HAVE_MAKE_REQUEST_FN_RET_QC) || \ + defined(HAVE_SUBMIT_BIO_IN_BLOCK_DEVICE_OPERATIONS) && \ !defined(HAVE_BDEV_SUBMIT_BIO_RETURNS_VOID) return (BLK_QC_T_NONE); #endif @@ -802,6 +982,27 @@ zvol_getgeo(struct block_device *bdev, struct hd_geometry *geo) return (0); } +/* + * Why have two separate block_device_operations structs? + * + * Normally we'd just have one, and assign 'submit_bio' as needed. However, + * it's possible the user's kernel is built with CONSTIFY_PLUGIN, meaning we + * can't just change submit_bio dynamically at runtime. So just create two + * separate structs to get around this. + */ +static const struct block_device_operations zvol_ops_blk_mq = { + .open = zvol_open, + .release = zvol_release, + .ioctl = zvol_ioctl, + .compat_ioctl = zvol_compat_ioctl, + .check_events = zvol_check_events, +#ifdef HAVE_BLOCK_DEVICE_OPERATIONS_REVALIDATE_DISK + .revalidate_disk = zvol_revalidate_disk, +#endif + .getgeo = zvol_getgeo, + .owner = THIS_MODULE, +}; + static const struct block_device_operations zvol_ops = { .open = zvol_open, .release = zvol_release, @@ -818,6 +1019,87 @@ static const struct block_device_operations zvol_ops = { #endif }; +static int +zvol_alloc_non_blk_mq(struct zvol_state_os *zso) +{ +#if defined(HAVE_SUBMIT_BIO_IN_BLOCK_DEVICE_OPERATIONS) +#if defined(HAVE_BLK_ALLOC_DISK) + zso->zvo_disk = blk_alloc_disk(NUMA_NO_NODE); + if (zso->zvo_disk == NULL) + return (1); + + zso->zvo_disk->minors = ZVOL_MINORS; + zso->zvo_queue = zso->zvo_disk->queue; +#else + zso->zvo_queue = blk_alloc_queue(NUMA_NO_NODE); + if (zso->zvo_queue == NULL) + return (1); + + zso->zvo_disk = alloc_disk(ZVOL_MINORS); + if (zso->zvo_disk == NULL) { + blk_cleanup_queue(zso->zvo_queue); + return (1); + } + + zso->zvo_disk->queue = zso->zvo_queue; +#endif /* HAVE_BLK_ALLOC_DISK */ +#else + zso->zvo_queue = blk_generic_alloc_queue(zvol_request, NUMA_NO_NODE); + if (zso->zvo_queue == NULL) + return (1); + + zso->zvo_disk = alloc_disk(ZVOL_MINORS); + if (zso->zvo_disk == NULL) { + blk_cleanup_queue(zso->zvo_queue); + return (1); + } + + zso->zvo_disk->queue = zso->zvo_queue; +#endif /* HAVE_SUBMIT_BIO_IN_BLOCK_DEVICE_OPERATIONS */ + return (0); + +} + +static int +zvol_alloc_blk_mq(zvol_state_t *zv) +{ +#ifdef HAVE_BLK_MQ + struct zvol_state_os *zso = zv->zv_zso; + + /* Allocate our blk-mq tag_set */ + if (zvol_blk_mq_alloc_tag_set(zv) != 0) + return (1); + +#if defined(HAVE_BLK_ALLOC_DISK) + zso->zvo_disk = blk_mq_alloc_disk(&zso->tag_set, zv); + if (zso->zvo_disk == NULL) { + blk_mq_free_tag_set(&zso->tag_set); + return (1); + } + zso->zvo_queue = zso->zvo_disk->queue; + zso->zvo_disk->minors = ZVOL_MINORS; +#else + zso->zvo_disk = alloc_disk(ZVOL_MINORS); + if (zso->zvo_disk == NULL) { + blk_cleanup_queue(zso->zvo_queue); + blk_mq_free_tag_set(&zso->tag_set); + return (1); + } + /* Allocate queue */ + zso->zvo_queue = blk_mq_init_queue(&zso->tag_set); + if (IS_ERR(zso->zvo_queue)) { + blk_mq_free_tag_set(&zso->tag_set); + return (1); + } + + /* Our queue is now created, assign it to our disk */ + zso->zvo_disk->queue = zso->zvo_queue; + +#endif +#endif + return (0); +} + /* * Allocate memory for a new zvol_state_t and setup the required * request queue and generic disk structures for the block device. @@ -828,6 +1110,7 @@ zvol_alloc(dev_t dev, const char *name) zvol_state_t *zv; struct zvol_state_os *zso; uint64_t volmode; + int ret; if (dsl_prop_get_integer(name, "volmode", &volmode, NULL) != 0) return (NULL); @@ -846,48 +1129,44 @@ zvol_alloc(dev_t dev, const char *name) list_link_init(&zv->zv_next); mutex_init(&zv->zv_state_lock, NULL, MUTEX_DEFAULT, NULL); -#ifdef HAVE_SUBMIT_BIO_IN_BLOCK_DEVICE_OPERATIONS -#ifdef HAVE_BLK_ALLOC_DISK - zso->zvo_disk = blk_alloc_disk(NUMA_NO_NODE); - if (zso->zvo_disk == NULL) - goto out_kmem; - - zso->zvo_disk->minors = ZVOL_MINORS; - zso->zvo_queue = zso->zvo_disk->queue; -#else - zso->zvo_queue = blk_alloc_queue(NUMA_NO_NODE); - if (zso->zvo_queue == NULL) - goto out_kmem; +#ifdef HAVE_BLK_MQ + zv->zv_zso->use_blk_mq = zvol_use_blk_mq; +#endif - zso->zvo_disk = alloc_disk(ZVOL_MINORS); - if (zso->zvo_disk == NULL) { - blk_cleanup_queue(zso->zvo_queue); - goto out_kmem; + /* + * The block layer has 3 interfaces for getting BIOs: + * + * 1. blk-mq request queues (new) + * 2. submit_bio() (oldest) + * 3. regular request queues (old). + * + * Each of those interfaces has two permutations: + * + * a) We have blk_alloc_disk()/blk_mq_alloc_disk(), which allocates + * both the disk and its queue (5.14 kernel or newer) + * + * b) We don't have blk_*alloc_disk(), and have to allocate the + * disk and the queue separately. (5.13 kernel or older) + */ + if (zv->zv_zso->use_blk_mq) { + ret = zvol_alloc_blk_mq(zv); + zso->zvo_disk->fops = &zvol_ops_blk_mq; + } else { + ret = zvol_alloc_non_blk_mq(zso); + zso->zvo_disk->fops = &zvol_ops; } - - zso->zvo_disk->queue = zso->zvo_queue; -#endif /* HAVE_BLK_ALLOC_DISK */ -#else - zso->zvo_queue = blk_generic_alloc_queue(zvol_request, NUMA_NO_NODE); - if (zso->zvo_queue == NULL) + if (ret != 0) goto out_kmem; - zso->zvo_disk = alloc_disk(ZVOL_MINORS); - if (zso->zvo_disk == NULL) { - blk_cleanup_queue(zso->zvo_queue); - goto out_kmem; - } - - zso->zvo_disk->queue = zso->zvo_queue; -#endif /* HAVE_SUBMIT_BIO_IN_BLOCK_DEVICE_OPERATIONS */ - blk_queue_set_write_cache(zso->zvo_queue, B_TRUE, B_TRUE); /* Limit read-ahead to a single page to prevent over-prefetching. */ blk_queue_set_read_ahead(zso->zvo_queue, 1); - /* Disable write merging in favor of the ZIO pipeline. */ - blk_queue_flag_set(QUEUE_FLAG_NOMERGES, zso->zvo_queue); + if (!zv->zv_zso->use_blk_mq) { + /* Disable write merging in favor of the ZIO pipeline. */ + blk_queue_flag_set(QUEUE_FLAG_NOMERGES, zso->zvo_queue); + } /* Enable /proc/diskstats */ blk_queue_flag_set(QUEUE_FLAG_IO_STAT, zso->zvo_queue); @@ -903,24 +1182,18 @@ zvol_alloc(dev_t dev, const char *name) zso->zvo_disk->major = zvol_major; zso->zvo_disk->events = DISK_EVENT_MEDIA_CHANGE; + /* + * Setting ZFS_VOLMODE_DEV disables partitioning on ZVOL devices. + * This is accomplished by limiting the number of minors for the + * device to one and explicitly disabling partition scanning. + */ if (volmode == ZFS_VOLMODE_DEV) { - /* - * ZFS_VOLMODE_DEV disable partitioning on ZVOL devices: set - * gendisk->minors = 1 as noted in include/linux/genhd.h. - * Also disable extended partition numbers (GENHD_FL_EXT_DEVT) - * and suppresses partition scanning (GENHD_FL_NO_PART_SCAN) - * setting gendisk->flags accordingly. - */ zso->zvo_disk->minors = 1; -#if defined(GENHD_FL_EXT_DEVT) - zso->zvo_disk->flags &= ~GENHD_FL_EXT_DEVT; -#endif -#if defined(GENHD_FL_NO_PART_SCAN) - zso->zvo_disk->flags |= GENHD_FL_NO_PART_SCAN; -#endif + zso->zvo_disk->flags &= ~ZFS_GENHD_FL_EXT_DEVT; + zso->zvo_disk->flags |= ZFS_GENHD_FL_NO_PART; } + zso->zvo_disk->first_minor = (dev & MINORMASK); - zso->zvo_disk->fops = &zvol_ops; zso->zvo_disk->private_data = zv; snprintf(zso->zvo_disk->disk_name, DISK_NAME_LEN, "%s%d", ZVOL_DEV_NAME, (dev & MINORMASK)); @@ -965,6 +1238,11 @@ zvol_os_free(zvol_state_t *zv) put_disk(zv->zv_zso->zvo_disk); #endif +#ifdef HAVE_BLK_MQ + if (zv->zv_zso->use_blk_mq) + blk_mq_free_tag_set(&zv->zv_zso->tag_set); +#endif + ida_simple_remove(&zvol_ida, MINOR(zv->zv_zso->zvo_dev) >> ZVOL_MINOR_BITS); @@ -1046,8 +1324,69 @@ zvol_os_create_minor(const char *name) blk_queue_max_hw_sectors(zv->zv_zso->zvo_queue, (DMU_MAX_ACCESS / 4) >> 9); - blk_queue_max_segments(zv->zv_zso->zvo_queue, UINT16_MAX); - blk_queue_max_segment_size(zv->zv_zso->zvo_queue, UINT_MAX); + + if (zv->zv_zso->use_blk_mq) { + /* + * IO requests can be really big (1MB). When an IO request + * comes in, it is passed off to zvol_read() or zvol_write() + * in a new thread, where it is chunked up into 'volblocksize' + * sized pieces and processed. So for example, if the request + * is a 1MB write and your volblocksize is 128k, one zvol_write + * thread will take that request and sequentially do ten 128k + * IOs. This is due to the fact that the thread needs to lock + * each volblocksize sized block. So you might be wondering: + * "instead of passing the whole 1MB request to one thread, + * why not pass ten individual 128k chunks to ten threads and + * process the whole write in parallel?" The short answer is + * that there's a sweet spot number of chunks that balances + * the greater parallelism with the added overhead of more + * threads. The sweet spot can be different depending on if you + * have a read or write heavy workload. Writes typically want + * high chunk counts while reads typically want lower ones. On + * a test pool with 6 NVMe drives in a 3x 2-disk mirror + * configuration, with volblocksize=8k, the sweet spot for good + * sequential reads and writes was at 8 chunks. + */ + + /* + * Below we tell the kernel how big we want our requests + * to be. You would think that blk_queue_io_opt() would be + * used to do this since it is used to "set optimal request + * size for the queue", but that doesn't seem to do + * anything - the kernel still gives you huge requests + * with tons of little PAGE_SIZE segments contained within it. + * + * Knowing that the kernel will just give you PAGE_SIZE segments + * no matter what, you can say "ok, I want PAGE_SIZE byte + * segments, and I want 'N' of them per request", where N is + * the correct number of segments for the volblocksize and + * number of chunks you want. + */ +#ifdef HAVE_BLK_MQ + if (zvol_blk_mq_blocks_per_thread != 0) { + unsigned int chunks; + chunks = MIN(zvol_blk_mq_blocks_per_thread, UINT16_MAX); + + blk_queue_max_segment_size(zv->zv_zso->zvo_queue, + PAGE_SIZE); + blk_queue_max_segments(zv->zv_zso->zvo_queue, + (zv->zv_volblocksize * chunks) / PAGE_SIZE); + } else { + /* + * Special case: zvol_blk_mq_blocks_per_thread = 0 + * Max everything out. + */ + blk_queue_max_segments(zv->zv_zso->zvo_queue, + UINT16_MAX); + blk_queue_max_segment_size(zv->zv_zso->zvo_queue, + UINT_MAX); + } +#endif + } else { + blk_queue_max_segments(zv->zv_zso->zvo_queue, UINT16_MAX); + blk_queue_max_segment_size(zv->zv_zso->zvo_queue, UINT_MAX); + } + blk_queue_physical_block_size(zv->zv_zso->zvo_queue, zv->zv_volblocksize); blk_queue_io_opt(zv->zv_zso->zvo_queue, zv->zv_volblocksize); @@ -1055,7 +1394,9 @@ zvol_os_create_minor(const char *name) (zvol_max_discard_blocks * zv->zv_volblocksize) >> 9); blk_queue_discard_granularity(zv->zv_zso->zvo_queue, zv->zv_volblocksize); +#ifdef QUEUE_FLAG_DISCARD blk_queue_flag_set(QUEUE_FLAG_DISCARD, zv->zv_zso->zvo_queue); +#endif #ifdef QUEUE_FLAG_NONROT blk_queue_flag_set(QUEUE_FLAG_NONROT, zv->zv_zso->zvo_queue); #endif @@ -1067,8 +1408,12 @@ zvol_os_create_minor(const char *name) blk_queue_flag_set(QUEUE_FLAG_SCSI_PASSTHROUGH, zv->zv_zso->zvo_queue); #endif + ASSERT3P(zv->zv_kstat.dk_kstats, ==, NULL); + error = dataset_kstats_create(&zv->zv_kstat, zv->zv_objset); + if (error) + goto out_dmu_objset_disown; ASSERT3P(zv->zv_zilog, ==, NULL); - zv->zv_zilog = zil_open(os, zvol_get_data); + zv->zv_zilog = zil_open(os, zvol_get_data, &zv->zv_kstat.dk_zil_sums); if (spa_writeable(dmu_objset_spa(os))) { if (zil_replay_disable) zil_destroy(zv->zv_zilog, B_FALSE); @@ -1077,8 +1422,6 @@ zvol_os_create_minor(const char *name) } zil_close(zv->zv_zilog); zv->zv_zilog = NULL; - ASSERT3P(zv->zv_kstat.dk_kstats, ==, NULL); - dataset_kstats_create(&zv->zv_kstat, zv->zv_objset); /* * When udev detects the addition of the device it will immediately @@ -1167,19 +1510,54 @@ int zvol_init(void) { int error; - int threads = MIN(MAX(zvol_threads, 1), 1024); + + /* + * zvol_threads is the module param the user passes in. + * + * zvol_actual_threads is what we use internally, since the user can + * pass zvol_thread = 0 to mean "use all the CPUs" (the default). + */ + static unsigned int zvol_actual_threads; + + if (zvol_threads == 0) { + /* + * See dde9380a1 for why 32 was chosen here. This should + * probably be refined to be some multiple of the number + * of CPUs. + */ + zvol_actual_threads = MAX(num_online_cpus(), 32); + } else { + zvol_actual_threads = MIN(MAX(zvol_threads, 1), 1024); + } error = register_blkdev(zvol_major, ZVOL_DRIVER); if (error) { printk(KERN_INFO "ZFS: register_blkdev() failed %d\n", error); return (error); } - zvol_taskq = taskq_create(ZVOL_DRIVER, threads, maxclsyspri, - threads * 2, INT_MAX, TASKQ_PREPOPULATE | TASKQ_DYNAMIC); + +#ifdef HAVE_BLK_MQ + if (zvol_blk_mq_queue_depth == 0) { + zvol_actual_blk_mq_queue_depth = BLKDEV_DEFAULT_RQ; + } else { + zvol_actual_blk_mq_queue_depth = + MAX(zvol_blk_mq_queue_depth, BLKDEV_MIN_RQ); + } + + if (zvol_blk_mq_threads == 0) { + zvol_blk_mq_actual_threads = num_online_cpus(); + } else { + zvol_blk_mq_actual_threads = MIN(MAX(zvol_blk_mq_threads, 1), + 1024); + } +#endif + zvol_taskq = taskq_create(ZVOL_DRIVER, zvol_actual_threads, maxclsyspri, + zvol_actual_threads, INT_MAX, TASKQ_PREPOPULATE | TASKQ_DYNAMIC); if (zvol_taskq == NULL) { unregister_blkdev(zvol_major, ZVOL_DRIVER); return (-ENOMEM); } + zvol_init_impl(); ida_init(&zvol_ida); return (0); @@ -1202,7 +1580,8 @@ module_param(zvol_major, uint, 0444); MODULE_PARM_DESC(zvol_major, "Major number for zvol device"); module_param(zvol_threads, uint, 0444); -MODULE_PARM_DESC(zvol_threads, "Max number of threads to handle I/O requests"); +MODULE_PARM_DESC(zvol_threads, "Number of threads to handle I/O requests. Set" + "to 0 to use all active CPUs"); module_param(zvol_request_sync, uint, 0644); MODULE_PARM_DESC(zvol_request_sync, "Synchronously handle bio requests"); @@ -1215,4 +1594,17 @@ MODULE_PARM_DESC(zvol_prefetch_bytes, "Prefetch N bytes at zvol start+end"); module_param(zvol_volmode, uint, 0644); MODULE_PARM_DESC(zvol_volmode, "Default volmode property value"); + +#ifdef HAVE_BLK_MQ +module_param(zvol_blk_mq_queue_depth, uint, 0644); +MODULE_PARM_DESC(zvol_blk_mq_queue_depth, "Default blk-mq queue depth"); + +module_param(zvol_use_blk_mq, uint, 0644); +MODULE_PARM_DESC(zvol_use_blk_mq, "Use the blk-mq API for zvols"); + +module_param(zvol_blk_mq_blocks_per_thread, uint, 0644); +MODULE_PARM_DESC(zvol_blk_mq_blocks_per_thread, + "Process volblocksize blocks per thread"); +#endif + /* END CSTYLED */ diff --git a/module/spl/Makefile.in b/module/spl/Makefile.in deleted file mode 100644 index cedbfe92b58a..000000000000 --- a/module/spl/Makefile.in +++ /dev/null @@ -1,13 +0,0 @@ -ifneq ($(KBUILD_EXTMOD),) -src = @abs_srcdir@ -obj = @abs_builddir@ -mfdir = $(obj) -else -mfdir = $(srctree)/$(src) -endif - -MODULE := spl - -obj-$(CONFIG_ZFS) := $(MODULE).o - -include $(mfdir)/../os/linux/spl/Makefile diff --git a/module/unicode/Makefile.in b/module/unicode/Makefile.in deleted file mode 100644 index 59c07c4555b7..000000000000 --- a/module/unicode/Makefile.in +++ /dev/null @@ -1,11 +0,0 @@ -ifneq ($(KBUILD_EXTMOD),) -src = @abs_srcdir@ -obj = @abs_builddir@ -endif - -MODULE := zunicode - -obj-$(CONFIG_ZFS) := $(MODULE).o - -$(MODULE)-objs += u8_textprep.o -$(MODULE)-objs += uconv.o diff --git a/module/unicode/u8_textprep.c b/module/unicode/u8_textprep.c index 863f69f7ba82..1940ee510d2c 100644 --- a/module/unicode/u8_textprep.c +++ b/module/unicode/u8_textprep.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -36,7 +36,7 @@ */ #include -#include +#include #include #include #include @@ -2129,27 +2129,6 @@ u8_textprep_str(char *inarray, size_t *inlen, char *outarray, size_t *outlen, return (ret_val); } -#if defined(_KERNEL) -static int __init -unicode_init(void) -{ - return (0); -} - -static void __exit -unicode_fini(void) -{ -} - -module_init(unicode_init); -module_exit(unicode_fini); -#endif - -ZFS_MODULE_DESCRIPTION("Unicode implementation"); -ZFS_MODULE_AUTHOR(ZFS_META_AUTHOR); -ZFS_MODULE_LICENSE(ZFS_META_LICENSE); -ZFS_MODULE_VERSION(ZFS_META_VERSION "-" ZFS_META_RELEASE); - EXPORT_SYMBOL(u8_validate); EXPORT_SYMBOL(u8_strcmp); EXPORT_SYMBOL(u8_textprep_str); diff --git a/module/unicode/uconv.c b/module/unicode/uconv.c index 6854aeab277f..4bd19ebdd242 100644 --- a/module/unicode/uconv.c +++ b/module/unicode/uconv.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zcommon/Makefile.in b/module/zcommon/Makefile.in deleted file mode 100644 index ebc538440445..000000000000 --- a/module/zcommon/Makefile.in +++ /dev/null @@ -1,28 +0,0 @@ -ifneq ($(KBUILD_EXTMOD),) -src = @abs_srcdir@ -obj = @abs_builddir@ -endif - -MODULE := zcommon - -obj-$(CONFIG_ZFS) := $(MODULE).o - -# Suppress unused-value warnings in sparc64 architecture headers -ccflags-$(CONFIG_SPARC64) += -Wno-unused-value - -$(MODULE)-objs += cityhash.o -$(MODULE)-objs += zfeature_common.o -$(MODULE)-objs += zfs_comutil.o -$(MODULE)-objs += zfs_deleg.o -$(MODULE)-objs += zfs_fletcher.o -$(MODULE)-objs += zfs_fletcher_superscalar.o -$(MODULE)-objs += zfs_fletcher_superscalar4.o -$(MODULE)-objs += zfs_namecheck.o -$(MODULE)-objs += zfs_prop.o -$(MODULE)-objs += zpool_prop.o -$(MODULE)-objs += zprop_common.o - -$(MODULE)-$(CONFIG_X86) += zfs_fletcher_intel.o -$(MODULE)-$(CONFIG_X86) += zfs_fletcher_sse.o -$(MODULE)-$(CONFIG_X86) += zfs_fletcher_avx512.o -$(MODULE)-$(CONFIG_ARM64) += zfs_fletcher_aarch64_neon.o diff --git a/module/zcommon/zfeature_common.c b/module/zcommon/zfeature_common.c index 529c52316f3e..ea45c9f8afa9 100644 --- a/module/zcommon/zfeature_common.c +++ b/module/zcommon/zfeature_common.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -695,6 +695,36 @@ zpool_feature_init(void) "org.openzfs:draid", "draid", "Support for distributed spare RAID", ZFEATURE_FLAG_MOS, ZFEATURE_TYPE_BOOLEAN, NULL, sfeatures); + { + static const spa_feature_t zilsaxattr_deps[] = { + SPA_FEATURE_EXTENSIBLE_DATASET, + SPA_FEATURE_NONE + }; + zfeature_register(SPA_FEATURE_ZILSAXATTR, + "org.openzfs:zilsaxattr", "zilsaxattr", + "Support for xattr=sa extended attribute logging in ZIL.", + ZFEATURE_FLAG_PER_DATASET | ZFEATURE_FLAG_READONLY_COMPAT, + ZFEATURE_TYPE_BOOLEAN, zilsaxattr_deps, sfeatures); + } + + zfeature_register(SPA_FEATURE_HEAD_ERRLOG, + "com.delphix:head_errlog", "head_errlog", + "Support for per-dataset on-disk error logs.", + ZFEATURE_FLAG_ACTIVATE_ON_ENABLE, ZFEATURE_TYPE_BOOLEAN, NULL, + sfeatures); + + { + static const spa_feature_t blake3_deps[] = { + SPA_FEATURE_EXTENSIBLE_DATASET, + SPA_FEATURE_NONE + }; + zfeature_register(SPA_FEATURE_BLAKE3, + "org.openzfs:blake3", "blake3", + "BLAKE3 hash algorithm.", + ZFEATURE_FLAG_PER_DATASET, ZFEATURE_TYPE_BOOLEAN, + blake3_deps, sfeatures); + } + zfs_mod_list_supported_free(sfeatures); } diff --git a/module/zcommon/zfs_comutil.c b/module/zcommon/zfs_comutil.c index 020e7e86c11f..36a9c8a35ae1 100644 --- a/module/zcommon/zfs_comutil.c +++ b/module/zcommon/zfs_comutil.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -68,7 +68,7 @@ zfs_allocatable_devs(nvlist_t *nv) * Are there special vdevs? */ boolean_t -zfs_special_devs(nvlist_t *nv, char *type) +zfs_special_devs(nvlist_t *nv, const char *type) { char *bias; uint_t c; @@ -84,11 +84,9 @@ zfs_special_devs(nvlist_t *nv, char *type) &bias) == 0) { if (strcmp(bias, VDEV_ALLOC_BIAS_SPECIAL) == 0 || strcmp(bias, VDEV_ALLOC_BIAS_DEDUP) == 0) { - if (type != NULL && strcmp(bias, type) == 0) { + if (type == NULL || + (type != NULL && strcmp(bias, type) == 0)) return (B_TRUE); - } else if (type == NULL) { - return (B_TRUE); - } } } } diff --git a/module/zcommon/zfs_deleg.c b/module/zcommon/zfs_deleg.c index 8a4a6ca86e13..40d207b503d8 100644 --- a/module/zcommon/zfs_deleg.c +++ b/module/zcommon/zfs_deleg.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -36,7 +36,7 @@ #include #include #endif -#include +#include #include #include "zfs_prop.h" #include "zfs_deleg.h" diff --git a/module/zcommon/zfs_fletcher.c b/module/zcommon/zfs_fletcher.c index 16773d4de077..9a201842ea6a 100644 --- a/module/zcommon/zfs_fletcher.c +++ b/module/zcommon/zfs_fletcher.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -126,8 +126,8 @@ * which has been filled either by: * * 1. a compression step, which will be mostly cached, or - * 2. a bcopy() or copyin(), which will be uncached (because the - * copy is cache-bypassing). + * 2. a memcpy() or copyin(), which will be uncached + * (because the copy is cache-bypassing). * * For both cached and uncached data, both fletcher checksums are much faster * than sha-256, and slower than 'off', which doesn't touch the data at all. diff --git a/module/zcommon/zfs_fletcher_aarch64_neon.c b/module/zcommon/zfs_fletcher_aarch64_neon.c index e84d69eb3415..3e14875d6c8c 100644 --- a/module/zcommon/zfs_fletcher_aarch64_neon.c +++ b/module/zcommon/zfs_fletcher_aarch64_neon.c @@ -45,14 +45,14 @@ #include #include -#include +#include #include ZFS_NO_SANITIZE_UNDEFINED static void fletcher_4_aarch64_neon_init(fletcher_4_ctx_t *ctx) { - bzero(ctx->aarch64_neon, 4 * sizeof (zfs_fletcher_aarch64_neon_t)); + memset(ctx->aarch64_neon, 0, 4 * sizeof (zfs_fletcher_aarch64_neon_t)); } ZFS_NO_SANITIZE_UNDEFINED diff --git a/module/zcommon/zfs_fletcher_avx512.c b/module/zcommon/zfs_fletcher_avx512.c index 8ee438ab9325..20c38d7e779b 100644 --- a/module/zcommon/zfs_fletcher_avx512.c +++ b/module/zcommon/zfs_fletcher_avx512.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -27,7 +27,7 @@ #include #include #include -#include +#include #include #include @@ -39,7 +39,7 @@ ZFS_NO_SANITIZE_UNDEFINED static void fletcher_4_avx512f_init(fletcher_4_ctx_t *ctx) { - bzero(ctx->avx512, 4 * sizeof (zfs_fletcher_avx512_t)); + memset(ctx->avx512, 0, 4 * sizeof (zfs_fletcher_avx512_t)); } ZFS_NO_SANITIZE_UNDEFINED diff --git a/module/zcommon/zfs_fletcher_intel.c b/module/zcommon/zfs_fletcher_intel.c index 16e61a96f8b1..42b6309d3609 100644 --- a/module/zcommon/zfs_fletcher_intel.c +++ b/module/zcommon/zfs_fletcher_intel.c @@ -43,15 +43,15 @@ #if defined(HAVE_AVX) && defined(HAVE_AVX2) #include +#include #include -#include #include ZFS_NO_SANITIZE_UNDEFINED static void fletcher_4_avx2_init(fletcher_4_ctx_t *ctx) { - bzero(ctx->avx, 4 * sizeof (zfs_fletcher_avx_t)); + memset(ctx->avx, 0, 4 * sizeof (zfs_fletcher_avx_t)); } ZFS_NO_SANITIZE_UNDEFINED diff --git a/module/zcommon/zfs_fletcher_sse.c b/module/zcommon/zfs_fletcher_sse.c index fc5938488e61..791bbd49f68f 100644 --- a/module/zcommon/zfs_fletcher_sse.c +++ b/module/zcommon/zfs_fletcher_sse.c @@ -45,15 +45,15 @@ #include #include +#include #include -#include #include ZFS_NO_SANITIZE_UNDEFINED static void fletcher_4_sse2_init(fletcher_4_ctx_t *ctx) { - bzero(ctx->sse, 4 * sizeof (zfs_fletcher_sse_t)); + memset(ctx->sse, 0, 4 * sizeof (zfs_fletcher_sse_t)); } ZFS_NO_SANITIZE_UNDEFINED diff --git a/module/zcommon/zfs_fletcher_superscalar.c b/module/zcommon/zfs_fletcher_superscalar.c index 73a74b9ae0ab..ba3fb54cbcff 100644 --- a/module/zcommon/zfs_fletcher_superscalar.c +++ b/module/zcommon/zfs_fletcher_superscalar.c @@ -44,14 +44,14 @@ #include #include #include -#include +#include #include ZFS_NO_SANITIZE_UNDEFINED static void fletcher_4_superscalar_init(fletcher_4_ctx_t *ctx) { - bzero(ctx->superscalar, 4 * sizeof (zfs_fletcher_superscalar_t)); + memset(ctx->superscalar, 0, 4 * sizeof (zfs_fletcher_superscalar_t)); } ZFS_NO_SANITIZE_UNDEFINED diff --git a/module/zcommon/zfs_fletcher_superscalar4.c b/module/zcommon/zfs_fletcher_superscalar4.c index 2dbf8bbb8146..e3eda029590a 100644 --- a/module/zcommon/zfs_fletcher_superscalar4.c +++ b/module/zcommon/zfs_fletcher_superscalar4.c @@ -44,14 +44,14 @@ #include #include #include -#include +#include #include ZFS_NO_SANITIZE_UNDEFINED static void fletcher_4_superscalar4_init(fletcher_4_ctx_t *ctx) { - bzero(ctx->superscalar, 4 * sizeof (zfs_fletcher_superscalar_t)); + memset(ctx->superscalar, 0, 4 * sizeof (zfs_fletcher_superscalar_t)); } ZFS_NO_SANITIZE_UNDEFINED diff --git a/module/zcommon/zfs_namecheck.c b/module/zcommon/zfs_namecheck.c index 7ecce451b42d..273b219a9794 100644 --- a/module/zcommon/zfs_namecheck.c +++ b/module/zcommon/zfs_namecheck.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zcommon/zfs_prop.c b/module/zcommon/zfs_prop.c index 895a86bb479b..0b0fc9015fa0 100644 --- a/module/zcommon/zfs_prop.c +++ b/module/zcommon/zfs_prop.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -84,6 +84,7 @@ zfs_prop_init(void) { "sha512", ZIO_CHECKSUM_SHA512 }, { "skein", ZIO_CHECKSUM_SKEIN }, { "edonr", ZIO_CHECKSUM_EDONR }, + { "blake3", ZIO_CHECKSUM_BLAKE3 }, { NULL } }; @@ -102,6 +103,9 @@ zfs_prop_init(void) ZIO_CHECKSUM_SKEIN | ZIO_CHECKSUM_VERIFY }, { "edonr,verify", ZIO_CHECKSUM_EDONR | ZIO_CHECKSUM_VERIFY }, + { "blake3", ZIO_CHECKSUM_BLAKE3 }, + { "blake3,verify", + ZIO_CHECKSUM_BLAKE3 | ZIO_CHECKSUM_VERIFY }, { NULL } }; @@ -394,12 +398,12 @@ zfs_prop_init(void) ZIO_CHECKSUM_DEFAULT, PROP_INHERIT, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, "on | off | fletcher2 | fletcher4 | sha256 | sha512 | skein" - " | edonr", + " | edonr | blake3", "CHECKSUM", checksum_table, sfeatures); zprop_register_index(ZFS_PROP_DEDUP, "dedup", ZIO_CHECKSUM_OFF, PROP_INHERIT, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, "on | off | verify | sha256[,verify] | sha512[,verify] | " - "skein[,verify] | edonr,verify", + "skein[,verify] | edonr,verify | blake3[,verify]", "DEDUP", dedup_table, sfeatures); zprop_register_index(ZFS_PROP_COMPRESSION, "compression", ZIO_COMPRESS_DEFAULT, PROP_INHERIT, @@ -589,125 +593,132 @@ zfs_prop_init(void) /* readonly number properties */ zprop_register_number(ZFS_PROP_USED, "used", 0, PROP_READONLY, - ZFS_TYPE_DATASET, "", "USED", sfeatures); + ZFS_TYPE_DATASET, "", "USED", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_AVAILABLE, "available", 0, PROP_READONLY, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, "", "AVAIL", - sfeatures); + B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_REFERENCED, "referenced", 0, PROP_READONLY, ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, "", - "REFER", sfeatures); + "REFER", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_COMPRESSRATIO, "compressratio", 0, PROP_READONLY, ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, - "<1.00x or higher if compressed>", "RATIO", sfeatures); + "<1.00x or higher if compressed>", "RATIO", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_REFRATIO, "refcompressratio", 0, PROP_READONLY, ZFS_TYPE_DATASET, - "<1.00x or higher if compressed>", "REFRATIO", sfeatures); + "<1.00x or higher if compressed>", "REFRATIO", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_VOLBLOCKSIZE, "volblocksize", ZVOL_DEFAULT_BLOCKSIZE, PROP_ONETIME, - ZFS_TYPE_VOLUME, "512 to 128k, power of 2", "VOLBLOCK", sfeatures); + ZFS_TYPE_VOLUME, "512 to 128k, power of 2", "VOLBLOCK", B_FALSE, + sfeatures); zprop_register_number(ZFS_PROP_USEDSNAP, "usedbysnapshots", 0, PROP_READONLY, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, "", - "USEDSNAP", sfeatures); + "USEDSNAP", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_USEDDS, "usedbydataset", 0, PROP_READONLY, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, "", - "USEDDS", sfeatures); + "USEDDS", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_USEDCHILD, "usedbychildren", 0, PROP_READONLY, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, "", - "USEDCHILD", sfeatures); + "USEDCHILD", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_USEDREFRESERV, "usedbyrefreservation", 0, PROP_READONLY, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, "", "USEDREFRESERV", - sfeatures); + B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_USERREFS, "userrefs", 0, PROP_READONLY, - ZFS_TYPE_SNAPSHOT, "", "USERREFS", sfeatures); + ZFS_TYPE_SNAPSHOT, "", "USERREFS", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_WRITTEN, "written", 0, PROP_READONLY, - ZFS_TYPE_DATASET, "", "WRITTEN", sfeatures); + ZFS_TYPE_DATASET, "", "WRITTEN", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_LOGICALUSED, "logicalused", 0, PROP_READONLY, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, "", - "LUSED", sfeatures); + "LUSED", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_LOGICALREFERENCED, "logicalreferenced", 0, PROP_READONLY, ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, "", - "LREFER", sfeatures); + "LREFER", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_FILESYSTEM_COUNT, "filesystem_count", UINT64_MAX, PROP_READONLY, ZFS_TYPE_FILESYSTEM, - "", "FSCOUNT", sfeatures); + "", "FSCOUNT", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_SNAPSHOT_COUNT, "snapshot_count", UINT64_MAX, PROP_READONLY, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, - "", "SSCOUNT", sfeatures); + "", "SSCOUNT", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_GUID, "guid", 0, PROP_READONLY, ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, "", "GUID", - sfeatures); + B_TRUE, sfeatures); zprop_register_number(ZFS_PROP_CREATETXG, "createtxg", 0, PROP_READONLY, ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, "", "CREATETXG", - sfeatures); + B_TRUE, sfeatures); zprop_register_number(ZFS_PROP_PBKDF2_ITERS, "pbkdf2iters", 0, PROP_ONETIME_DEFAULT, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, - "", "PBKDF2ITERS", sfeatures); + "", "PBKDF2ITERS", B_TRUE, sfeatures); zprop_register_number(ZFS_PROP_OBJSETID, "objsetid", 0, - PROP_READONLY, ZFS_TYPE_DATASET, "", "OBJSETID", sfeatures); + PROP_READONLY, ZFS_TYPE_DATASET, "", "OBJSETID", B_TRUE, + sfeatures); /* default number properties */ zprop_register_number(ZFS_PROP_QUOTA, "quota", 0, PROP_DEFAULT, - ZFS_TYPE_FILESYSTEM, " | none", "QUOTA", sfeatures); + ZFS_TYPE_FILESYSTEM, " | none", "QUOTA", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_RESERVATION, "reservation", 0, PROP_DEFAULT, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, - " | none", "RESERV", sfeatures); + " | none", "RESERV", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_VOLSIZE, "volsize", 0, PROP_DEFAULT, ZFS_TYPE_SNAPSHOT | ZFS_TYPE_VOLUME, "", "VOLSIZE", - sfeatures); + B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_REFQUOTA, "refquota", 0, PROP_DEFAULT, - ZFS_TYPE_FILESYSTEM, " | none", "REFQUOTA", sfeatures); + ZFS_TYPE_FILESYSTEM, " | none", "REFQUOTA", B_FALSE, + sfeatures); zprop_register_number(ZFS_PROP_REFRESERVATION, "refreservation", 0, PROP_DEFAULT, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, - " | none", "REFRESERV", sfeatures); + " | none", "REFRESERV", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_FILESYSTEM_LIMIT, "filesystem_limit", UINT64_MAX, PROP_DEFAULT, ZFS_TYPE_FILESYSTEM, - " | none", "FSLIMIT", sfeatures); + " | none", "FSLIMIT", B_FALSE, sfeatures); zprop_register_number(ZFS_PROP_SNAPSHOT_LIMIT, "snapshot_limit", UINT64_MAX, PROP_DEFAULT, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, - " | none", "SSLIMIT", sfeatures); + " | none", "SSLIMIT", B_FALSE, sfeatures); /* inherit number properties */ zprop_register_number(ZFS_PROP_RECORDSIZE, "recordsize", SPA_OLD_MAXBLOCKSIZE, PROP_INHERIT, - ZFS_TYPE_FILESYSTEM, "512 to 1M, power of 2", "RECSIZE", sfeatures); + ZFS_TYPE_FILESYSTEM, "512 to 1M, power of 2", "RECSIZE", B_FALSE, + sfeatures); zprop_register_number(ZFS_PROP_SPECIAL_SMALL_BLOCKS, "special_small_blocks", 0, PROP_INHERIT, ZFS_TYPE_FILESYSTEM, - "zero or 512 to 1M, power of 2", "SPECIAL_SMALL_BLOCKS", sfeatures); + "zero or 512 to 1M, power of 2", "SPECIAL_SMALL_BLOCKS", B_FALSE, + sfeatures); /* hidden properties */ zprop_register_hidden(ZFS_PROP_NUMCLONES, "numclones", PROP_TYPE_NUMBER, - PROP_READONLY, ZFS_TYPE_SNAPSHOT, "NUMCLONES", sfeatures); + PROP_READONLY, ZFS_TYPE_SNAPSHOT, "NUMCLONES", B_FALSE, sfeatures); zprop_register_hidden(ZFS_PROP_NAME, "name", PROP_TYPE_STRING, PROP_READONLY, ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, "NAME", - sfeatures); + B_TRUE, sfeatures); zprop_register_hidden(ZFS_PROP_ISCSIOPTIONS, "iscsioptions", PROP_TYPE_STRING, PROP_INHERIT, ZFS_TYPE_VOLUME, "ISCSIOPTIONS", - sfeatures); + B_TRUE, sfeatures); zprop_register_hidden(ZFS_PROP_STMF_SHAREINFO, "stmf_sbd_lu", PROP_TYPE_STRING, PROP_INHERIT, ZFS_TYPE_VOLUME, - "STMF_SBD_LU", sfeatures); + "STMF_SBD_LU", B_TRUE, sfeatures); zprop_register_hidden(ZFS_PROP_USERACCOUNTING, "useraccounting", PROP_TYPE_NUMBER, PROP_READONLY, ZFS_TYPE_DATASET, - "USERACCOUNTING", sfeatures); + "USERACCOUNTING", B_FALSE, sfeatures); zprop_register_hidden(ZFS_PROP_UNIQUE, "unique", PROP_TYPE_NUMBER, - PROP_READONLY, ZFS_TYPE_DATASET, "UNIQUE", sfeatures); + PROP_READONLY, ZFS_TYPE_DATASET, "UNIQUE", B_FALSE, sfeatures); zprop_register_hidden(ZFS_PROP_INCONSISTENT, "inconsistent", PROP_TYPE_NUMBER, PROP_READONLY, ZFS_TYPE_DATASET, "INCONSISTENT", - sfeatures); + B_FALSE, sfeatures); zprop_register_hidden(ZFS_PROP_IVSET_GUID, "ivsetguid", PROP_TYPE_NUMBER, PROP_READONLY, - ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, "IVSETGUID", sfeatures); + ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, "IVSETGUID", B_TRUE, + sfeatures); zprop_register_hidden(ZFS_PROP_PREV_SNAP, "prevsnap", PROP_TYPE_STRING, PROP_READONLY, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, "PREVSNAP", - sfeatures); + B_TRUE, sfeatures); zprop_register_hidden(ZFS_PROP_PBKDF2_SALT, "pbkdf2salt", PROP_TYPE_NUMBER, PROP_ONETIME_DEFAULT, - ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, "PBKDF2SALT", sfeatures); + ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, "PBKDF2SALT", B_FALSE, + sfeatures); zprop_register_hidden(ZFS_PROP_KEY_GUID, "keyguid", PROP_TYPE_NUMBER, - PROP_READONLY, ZFS_TYPE_DATASET, "KEYGUID", sfeatures); + PROP_READONLY, ZFS_TYPE_DATASET, "KEYGUID", B_TRUE, sfeatures); zprop_register_hidden(ZFS_PROP_REDACTED, "redacted", PROP_TYPE_NUMBER, - PROP_READONLY, ZFS_TYPE_DATASET, "REDACTED", sfeatures); + PROP_READONLY, ZFS_TYPE_DATASET, "REDACTED", B_FALSE, sfeatures); /* * Properties that are obsolete and not used. These are retained so @@ -715,12 +726,13 @@ zfs_prop_init(void) * have NULL pointers in the zfs_prop_table[]. */ zprop_register_hidden(ZFS_PROP_REMAPTXG, "remaptxg", PROP_TYPE_NUMBER, - PROP_READONLY, ZFS_TYPE_DATASET, "REMAPTXG", sfeatures); + PROP_READONLY, ZFS_TYPE_DATASET, "REMAPTXG", B_FALSE, sfeatures); /* oddball properties */ + /* 'creation' is a number but displayed as human-readable => flex */ zprop_register_impl(ZFS_PROP_CREATION, "creation", PROP_TYPE_NUMBER, 0, NULL, PROP_READONLY, ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, - "", "CREATION", B_FALSE, B_TRUE, NULL, sfeatures); + "", "CREATION", B_FALSE, B_TRUE, B_TRUE, NULL, sfeatures); zfs_mod_list_supported_free(sfeatures); } @@ -993,12 +1005,15 @@ zfs_prop_align_right(zfs_prop_t prop) #include -#if defined(HAVE_KERNEL_FPU_INTERNAL) || defined(HAVE_KERNEL_FPU_XSAVE_INTERNAL) -union fpregs_state **zfs_kfpu_fpregs; +#if defined(HAVE_KERNEL_FPU_INTERNAL) +uint8_t **zfs_kfpu_fpregs; EXPORT_SYMBOL(zfs_kfpu_fpregs); -#endif /* HAVE_KERNEL_FPU_INTERNAL || HAVE_KERNEL_FPU_XSAVE_INTERNAL */ +#endif /* defined(HAVE_KERNEL_FPU_INTERNAL) */ -static int __init +extern int __init zcommon_init(void); +extern void zcommon_fini(void); + +int __init zcommon_init(void) { int error = kfpu_init(); @@ -1010,22 +1025,19 @@ zcommon_init(void) return (0); } -static void __exit +void zcommon_fini(void) { fletcher_4_fini(); kfpu_fini(); } +#ifdef __FreeBSD__ module_init_early(zcommon_init); module_exit(zcommon_fini); - #endif -ZFS_MODULE_DESCRIPTION("Generic ZFS support"); -ZFS_MODULE_AUTHOR(ZFS_META_AUTHOR); -ZFS_MODULE_LICENSE(ZFS_META_LICENSE); -ZFS_MODULE_VERSION(ZFS_META_VERSION "-" ZFS_META_RELEASE); +#endif /* zfs dataset property functions */ EXPORT_SYMBOL(zfs_userquota_prop_prefixes); diff --git a/module/zcommon/zpool_prop.c b/module/zcommon/zpool_prop.c index 44bfe4add230..4737bd628ddf 100644 --- a/module/zcommon/zpool_prop.c +++ b/module/zcommon/zpool_prop.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -85,39 +85,45 @@ zpool_prop_init(void) /* readonly number properties */ zprop_register_number(ZPOOL_PROP_SIZE, "size", 0, PROP_READONLY, - ZFS_TYPE_POOL, "", "SIZE", sfeatures); + ZFS_TYPE_POOL, "", "SIZE", B_FALSE, sfeatures); zprop_register_number(ZPOOL_PROP_FREE, "free", 0, PROP_READONLY, - ZFS_TYPE_POOL, "", "FREE", sfeatures); + ZFS_TYPE_POOL, "", "FREE", B_FALSE, sfeatures); zprop_register_number(ZPOOL_PROP_FREEING, "freeing", 0, PROP_READONLY, - ZFS_TYPE_POOL, "", "FREEING", sfeatures); + ZFS_TYPE_POOL, "", "FREEING", B_FALSE, sfeatures); zprop_register_number(ZPOOL_PROP_CHECKPOINT, "checkpoint", 0, - PROP_READONLY, ZFS_TYPE_POOL, "", "CKPOINT", sfeatures); + PROP_READONLY, ZFS_TYPE_POOL, "", "CKPOINT", B_FALSE, + sfeatures); zprop_register_number(ZPOOL_PROP_LEAKED, "leaked", 0, PROP_READONLY, - ZFS_TYPE_POOL, "", "LEAKED", sfeatures); + ZFS_TYPE_POOL, "", "LEAKED", B_FALSE, sfeatures); zprop_register_number(ZPOOL_PROP_ALLOCATED, "allocated", 0, - PROP_READONLY, ZFS_TYPE_POOL, "", "ALLOC", sfeatures); + PROP_READONLY, ZFS_TYPE_POOL, "", "ALLOC", B_FALSE, + sfeatures); zprop_register_number(ZPOOL_PROP_EXPANDSZ, "expandsize", 0, - PROP_READONLY, ZFS_TYPE_POOL, "", "EXPANDSZ", sfeatures); + PROP_READONLY, ZFS_TYPE_POOL, "", "EXPANDSZ", B_FALSE, + sfeatures); zprop_register_number(ZPOOL_PROP_FRAGMENTATION, "fragmentation", 0, - PROP_READONLY, ZFS_TYPE_POOL, "", "FRAG", sfeatures); + PROP_READONLY, ZFS_TYPE_POOL, "", "FRAG", B_FALSE, + sfeatures); zprop_register_number(ZPOOL_PROP_CAPACITY, "capacity", 0, PROP_READONLY, - ZFS_TYPE_POOL, "", "CAP", sfeatures); + ZFS_TYPE_POOL, "", "CAP", B_FALSE, sfeatures); zprop_register_number(ZPOOL_PROP_GUID, "guid", 0, PROP_READONLY, - ZFS_TYPE_POOL, "", "GUID", sfeatures); + ZFS_TYPE_POOL, "", "GUID", B_TRUE, sfeatures); zprop_register_number(ZPOOL_PROP_LOAD_GUID, "load_guid", 0, PROP_READONLY, ZFS_TYPE_POOL, "", "LOAD_GUID", - sfeatures); + B_TRUE, sfeatures); zprop_register_number(ZPOOL_PROP_HEALTH, "health", 0, PROP_READONLY, - ZFS_TYPE_POOL, "", "HEALTH", sfeatures); + ZFS_TYPE_POOL, "", "HEALTH", B_FALSE, sfeatures); zprop_register_number(ZPOOL_PROP_DEDUPRATIO, "dedupratio", 0, PROP_READONLY, ZFS_TYPE_POOL, "<1.00x or higher if deduped>", - "DEDUP", sfeatures); + "DEDUP", B_FALSE, sfeatures); /* default number properties */ zprop_register_number(ZPOOL_PROP_VERSION, "version", SPA_VERSION, - PROP_DEFAULT, ZFS_TYPE_POOL, "", "VERSION", sfeatures); + PROP_DEFAULT, ZFS_TYPE_POOL, "", "VERSION", B_FALSE, + sfeatures); zprop_register_number(ZPOOL_PROP_ASHIFT, "ashift", 0, PROP_DEFAULT, - ZFS_TYPE_POOL, "", "ASHIFT", sfeatures); + ZFS_TYPE_POOL, "", "ASHIFT", B_FALSE, + sfeatures); /* default index (boolean) properties */ zprop_register_index(ZPOOL_PROP_DELEGATION, "delegation", 1, @@ -150,18 +156,18 @@ zpool_prop_init(void) /* hidden properties */ zprop_register_hidden(ZPOOL_PROP_NAME, "name", PROP_TYPE_STRING, - PROP_READONLY, ZFS_TYPE_POOL, "NAME", sfeatures); + PROP_READONLY, ZFS_TYPE_POOL, "NAME", B_TRUE, sfeatures); zprop_register_hidden(ZPOOL_PROP_MAXBLOCKSIZE, "maxblocksize", PROP_TYPE_NUMBER, PROP_READONLY, ZFS_TYPE_POOL, "MAXBLOCKSIZE", - sfeatures); + B_FALSE, sfeatures); zprop_register_hidden(ZPOOL_PROP_TNAME, "tname", PROP_TYPE_STRING, - PROP_ONETIME, ZFS_TYPE_POOL, "TNAME", sfeatures); + PROP_ONETIME, ZFS_TYPE_POOL, "TNAME", B_TRUE, sfeatures); zprop_register_hidden(ZPOOL_PROP_MAXDNODESIZE, "maxdnodesize", PROP_TYPE_NUMBER, PROP_READONLY, ZFS_TYPE_POOL, "MAXDNODESIZE", - sfeatures); + B_FALSE, sfeatures); zprop_register_hidden(ZPOOL_PROP_DEDUPDITTO, "dedupditto", PROP_TYPE_NUMBER, PROP_DEFAULT, ZFS_TYPE_POOL, "DEDUPDITTO", - sfeatures); + B_FALSE, sfeatures); zfs_mod_list_supported_free(sfeatures); } @@ -323,67 +329,85 @@ vdev_prop_init(void) /* readonly number properties */ zprop_register_number(VDEV_PROP_SIZE, "size", 0, PROP_READONLY, - ZFS_TYPE_VDEV, "", "SIZE", sfeatures); + ZFS_TYPE_VDEV, "", "SIZE", B_FALSE, sfeatures); zprop_register_number(VDEV_PROP_FREE, "free", 0, PROP_READONLY, - ZFS_TYPE_VDEV, "", "FREE", sfeatures); + ZFS_TYPE_VDEV, "", "FREE", B_FALSE, sfeatures); zprop_register_number(VDEV_PROP_ALLOCATED, "allocated", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "ALLOC", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "ALLOC", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_EXPANDSZ, "expandsize", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "EXPANDSZ", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "EXPANDSZ", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_FRAGMENTATION, "fragmentation", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "FRAG", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "FRAG", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_CAPACITY, "capacity", 0, PROP_READONLY, - ZFS_TYPE_VDEV, "", "CAP", sfeatures); + ZFS_TYPE_VDEV, "", "CAP", B_FALSE, sfeatures); zprop_register_number(VDEV_PROP_GUID, "guid", 0, PROP_READONLY, - ZFS_TYPE_VDEV, "", "GUID", sfeatures); + ZFS_TYPE_VDEV, "", "GUID", B_TRUE, sfeatures); zprop_register_number(VDEV_PROP_STATE, "state", 0, PROP_READONLY, - ZFS_TYPE_VDEV, "", "STATE", sfeatures); + ZFS_TYPE_VDEV, "", "STATE", B_FALSE, sfeatures); zprop_register_number(VDEV_PROP_BOOTSIZE, "bootsize", 0, PROP_READONLY, - ZFS_TYPE_VDEV, "", "BOOTSIZE", sfeatures); + ZFS_TYPE_VDEV, "", "BOOTSIZE", B_FALSE, sfeatures); zprop_register_number(VDEV_PROP_ASIZE, "asize", 0, PROP_READONLY, - ZFS_TYPE_VDEV, "", "ASIZE", sfeatures); + ZFS_TYPE_VDEV, "", "ASIZE", B_FALSE, sfeatures); zprop_register_number(VDEV_PROP_PSIZE, "psize", 0, PROP_READONLY, - ZFS_TYPE_VDEV, "", "PSIZE", sfeatures); + ZFS_TYPE_VDEV, "", "PSIZE", B_FALSE, sfeatures); zprop_register_number(VDEV_PROP_ASHIFT, "ashift", 0, PROP_READONLY, - ZFS_TYPE_VDEV, "", "ASHIFT", sfeatures); + ZFS_TYPE_VDEV, "", "ASHIFT", B_FALSE, sfeatures); zprop_register_number(VDEV_PROP_PARITY, "parity", 0, PROP_READONLY, - ZFS_TYPE_VDEV, "", "PARITY", sfeatures); + ZFS_TYPE_VDEV, "", "PARITY", B_FALSE, sfeatures); zprop_register_number(VDEV_PROP_NUMCHILDREN, "numchildren", 0, PROP_READONLY, ZFS_TYPE_VDEV, "", "NUMCHILD", - sfeatures); + B_FALSE, sfeatures); zprop_register_number(VDEV_PROP_READ_ERRORS, "read_errors", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "RDERR", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "RDERR", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_WRITE_ERRORS, "write_errors", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "WRERR", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "WRERR", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_CHECKSUM_ERRORS, "checksum_errors", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "CKERR", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "CKERR", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_INITIALIZE_ERRORS, "initialize_errors", 0, PROP_READONLY, ZFS_TYPE_VDEV, "", - "INITERR", sfeatures); + "INITERR", B_FALSE, sfeatures); zprop_register_number(VDEV_PROP_OPS_NULL, "null_ops", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "NULLOP", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "NULLOP", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_OPS_READ, "read_ops", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "READOP", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "READOP", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_OPS_WRITE, "write_ops", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "WRITEOP", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "WRITEOP", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_OPS_FREE, "free_ops", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "FREEOP", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "FREEOP", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_OPS_CLAIM, "claim_ops", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "CLAIMOP", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "CLAIMOP", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_OPS_TRIM, "trim_ops", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "TRIMOP", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "TRIMOP", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_BYTES_NULL, "null_bytes", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "NULLBYTE", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "NULLBYTE", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_BYTES_READ, "read_bytes", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "READBYTE", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "READBYTE", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_BYTES_WRITE, "write_bytes", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "WRITEBYTE", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "WRITEBYTE", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_BYTES_FREE, "free_bytes", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "FREEBYTE", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "FREEBYTE", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_BYTES_CLAIM, "claim_bytes", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "CLAIMBYTE", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "CLAIMBYTE", B_FALSE, + sfeatures); zprop_register_number(VDEV_PROP_BYTES_TRIM, "trim_bytes", 0, - PROP_READONLY, ZFS_TYPE_VDEV, "", "TRIMBYTE", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "", "TRIMBYTE", B_FALSE, + sfeatures); /* default numeric properties */ @@ -399,7 +423,7 @@ vdev_prop_init(void) /* hidden properties */ zprop_register_hidden(VDEV_PROP_NAME, "name", PROP_TYPE_STRING, - PROP_READONLY, ZFS_TYPE_VDEV, "NAME", sfeatures); + PROP_READONLY, ZFS_TYPE_VDEV, "NAME", B_TRUE, sfeatures); zfs_mod_list_supported_free(sfeatures); } diff --git a/module/zcommon/zprop_common.c b/module/zcommon/zprop_common.c index c32039c1b61d..eb635b38ceb5 100644 --- a/module/zcommon/zprop_common.c +++ b/module/zcommon/zprop_common.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -97,7 +97,8 @@ void zprop_register_impl(int prop, const char *name, zprop_type_t type, uint64_t numdefault, const char *strdefault, zprop_attr_t attr, int objset_types, const char *values, const char *colname, - boolean_t rightalign, boolean_t visible, const zprop_index_t *idx_tbl, + boolean_t rightalign, boolean_t visible, boolean_t flex, + const zprop_index_t *idx_tbl, const struct zfs_mod_supported_features *sfeatures) { zprop_desc_t *prop_tbl = zprop_get_proptable(objset_types); @@ -122,6 +123,7 @@ zprop_register_impl(int prop, const char *name, zprop_type_t type, pd->pd_visible = visible; pd->pd_zfs_mod_supported = zfs_mod_supported_prop(name, objset_types, sfeatures); + pd->pd_always_flex = flex; pd->pd_table = idx_tbl; pd->pd_table_size = 0; while (idx_tbl && (idx_tbl++)->pi_name != NULL) @@ -134,17 +136,20 @@ zprop_register_string(int prop, const char *name, const char *def, const char *colname, const struct zfs_mod_supported_features *sfeatures) { zprop_register_impl(prop, name, PROP_TYPE_STRING, 0, def, attr, - objset_types, values, colname, B_FALSE, B_TRUE, NULL, sfeatures); + objset_types, values, colname, B_FALSE, B_TRUE, B_TRUE, NULL, + sfeatures); } void zprop_register_number(int prop, const char *name, uint64_t def, zprop_attr_t attr, int objset_types, const char *values, - const char *colname, const struct zfs_mod_supported_features *sfeatures) + const char *colname, boolean_t flex, + const struct zfs_mod_supported_features *sfeatures) { zprop_register_impl(prop, name, PROP_TYPE_NUMBER, def, NULL, attr, - objset_types, values, colname, B_TRUE, B_TRUE, NULL, sfeatures); + objset_types, values, colname, B_TRUE, B_TRUE, flex, NULL, + sfeatures); } void @@ -154,17 +159,18 @@ zprop_register_index(int prop, const char *name, uint64_t def, const struct zfs_mod_supported_features *sfeatures) { zprop_register_impl(prop, name, PROP_TYPE_INDEX, def, NULL, attr, - objset_types, values, colname, B_FALSE, B_TRUE, idx_tbl, sfeatures); + objset_types, values, colname, B_FALSE, B_TRUE, B_TRUE, idx_tbl, + sfeatures); } void zprop_register_hidden(int prop, const char *name, zprop_type_t type, - zprop_attr_t attr, int objset_types, const char *colname, + zprop_attr_t attr, int objset_types, const char *colname, boolean_t flex, const struct zfs_mod_supported_features *sfeatures) { zprop_register_impl(prop, name, type, 0, NULL, attr, objset_types, NULL, colname, - type == PROP_TYPE_NUMBER, B_FALSE, NULL, sfeatures); + type == PROP_TYPE_NUMBER, B_FALSE, flex, NULL, sfeatures); } @@ -437,7 +443,10 @@ zprop_width(int prop, boolean_t *fixed, zfs_type_t type) prop_tbl = zprop_get_proptable(type); pd = &prop_tbl[prop]; - *fixed = B_TRUE; + if (type != ZFS_TYPE_POOL && type != ZFS_TYPE_VDEV) + type = ZFS_TYPE_FILESYSTEM; + + *fixed = !pd->pd_always_flex; /* * Start with the width of the column name. @@ -456,19 +465,14 @@ zprop_width(int prop, boolean_t *fixed, zfs_type_t type) */ if (ret < 5) ret = 5; - /* - * 'creation' is handled specially because it's a number - * internally, but displayed as a date string. - */ - if (prop == ZFS_PROP_CREATION) - *fixed = B_FALSE; /* * 'health' is handled specially because it's a number * internally, but displayed as a fixed 8 character string. */ - if (prop == ZPOOL_PROP_HEALTH) + if (type == ZFS_TYPE_POOL && prop == ZPOOL_PROP_HEALTH) ret = 8; break; + case PROP_TYPE_INDEX: idx = prop_tbl[prop].pd_table; for (i = 0; idx[i].pi_name != NULL; i++) { @@ -478,7 +482,6 @@ zprop_width(int prop, boolean_t *fixed, zfs_type_t type) break; case PROP_TYPE_STRING: - *fixed = B_FALSE; break; } diff --git a/module/zfs/Makefile.in b/module/zfs/Makefile.in deleted file mode 100644 index 30dc91a7eb59..000000000000 --- a/module/zfs/Makefile.in +++ /dev/null @@ -1,158 +0,0 @@ -ifneq ($(KBUILD_EXTMOD),) -src = @abs_srcdir@ -obj = @abs_builddir@ -mfdir = $(obj) -else -mfdir = $(srctree)/$(src) -endif - -MODULE := zfs - -obj-$(CONFIG_ZFS) := $(MODULE).o - -# Suppress unused-value warnings in sparc64 architecture headers -ccflags-$(CONFIG_SPARC64) += -Wno-unused-value - -$(MODULE)-objs += abd.o -$(MODULE)-objs += aggsum.o -$(MODULE)-objs += arc.o -$(MODULE)-objs += blkptr.o -$(MODULE)-objs += bplist.o -$(MODULE)-objs += bpobj.o -$(MODULE)-objs += bptree.o -$(MODULE)-objs += btree.o -$(MODULE)-objs += bqueue.o -$(MODULE)-objs += dataset_kstats.o -$(MODULE)-objs += dbuf.o -$(MODULE)-objs += dbuf_stats.o -$(MODULE)-objs += ddt.o -$(MODULE)-objs += ddt_zap.o -$(MODULE)-objs += dmu.o -$(MODULE)-objs += dmu_diff.o -$(MODULE)-objs += dmu_object.o -$(MODULE)-objs += dmu_objset.o -$(MODULE)-objs += dmu_recv.o -$(MODULE)-objs += dmu_redact.o -$(MODULE)-objs += dmu_send.o -$(MODULE)-objs += dmu_traverse.o -$(MODULE)-objs += dmu_tx.o -$(MODULE)-objs += dmu_zfetch.o -$(MODULE)-objs += dnode.o -$(MODULE)-objs += dnode_sync.o -$(MODULE)-objs += dsl_bookmark.o -$(MODULE)-objs += dsl_crypt.o -$(MODULE)-objs += dsl_dataset.o -$(MODULE)-objs += dsl_deadlist.o -$(MODULE)-objs += dsl_deleg.o -$(MODULE)-objs += dsl_destroy.o -$(MODULE)-objs += dsl_dir.o -$(MODULE)-objs += dsl_pool.o -$(MODULE)-objs += dsl_prop.o -$(MODULE)-objs += dsl_scan.o -$(MODULE)-objs += dsl_synctask.o -$(MODULE)-objs += dsl_userhold.o -$(MODULE)-objs += edonr_zfs.o -$(MODULE)-objs += fm.o -$(MODULE)-objs += gzip.o -$(MODULE)-objs += hkdf.o -$(MODULE)-objs += lz4.o -$(MODULE)-objs += lz4_zfs.o -$(MODULE)-objs += lzjb.o -$(MODULE)-objs += metaslab.o -$(MODULE)-objs += mmp.o -$(MODULE)-objs += multilist.o -$(MODULE)-objs += objlist.o -$(MODULE)-objs += pathname.o -$(MODULE)-objs += range_tree.o -$(MODULE)-objs += refcount.o -$(MODULE)-objs += rrwlock.o -$(MODULE)-objs += sa.o -$(MODULE)-objs += sha256.o -$(MODULE)-objs += skein_zfs.o -$(MODULE)-objs += spa.o -$(MODULE)-objs += spa_boot.o -$(MODULE)-objs += spa_checkpoint.o -$(MODULE)-objs += spa_config.o -$(MODULE)-objs += spa_errlog.o -$(MODULE)-objs += spa_history.o -$(MODULE)-objs += spa_log_spacemap.o -$(MODULE)-objs += spa_misc.o -$(MODULE)-objs += spa_stats.o -$(MODULE)-objs += space_map.o -$(MODULE)-objs += space_reftree.o -$(MODULE)-objs += txg.o -$(MODULE)-objs += uberblock.o -$(MODULE)-objs += unique.o -$(MODULE)-objs += vdev.o -$(MODULE)-objs += vdev_cache.o -$(MODULE)-objs += vdev_draid.o -$(MODULE)-objs += vdev_draid_rand.o -$(MODULE)-objs += vdev_indirect.o -$(MODULE)-objs += vdev_indirect_births.o -$(MODULE)-objs += vdev_indirect_mapping.o -$(MODULE)-objs += vdev_initialize.o -$(MODULE)-objs += vdev_label.o -$(MODULE)-objs += vdev_mirror.o -$(MODULE)-objs += vdev_missing.o -$(MODULE)-objs += vdev_queue.o -$(MODULE)-objs += vdev_raidz.o -$(MODULE)-objs += vdev_raidz_math.o -$(MODULE)-objs += vdev_raidz_math_scalar.o -$(MODULE)-objs += vdev_rebuild.o -$(MODULE)-objs += vdev_removal.o -$(MODULE)-objs += vdev_root.o -$(MODULE)-objs += vdev_trim.o -$(MODULE)-objs += zap.o -$(MODULE)-objs += zap_leaf.o -$(MODULE)-objs += zap_micro.o -$(MODULE)-objs += zcp.o -$(MODULE)-objs += zcp_get.o -$(MODULE)-objs += zcp_global.o -$(MODULE)-objs += zcp_iter.o -$(MODULE)-objs += zcp_set.o -$(MODULE)-objs += zcp_synctask.o -$(MODULE)-objs += zfeature.o -$(MODULE)-objs += zfs_byteswap.o -$(MODULE)-objs += zfs_fm.o -$(MODULE)-objs += zfs_fuid.o -$(MODULE)-objs += zfs_ioctl.o -$(MODULE)-objs += zfs_log.o -$(MODULE)-objs += zfs_onexit.o -$(MODULE)-objs += zfs_quota.o -$(MODULE)-objs += zfs_ratelimit.o -$(MODULE)-objs += zfs_replay.o -$(MODULE)-objs += zfs_rlock.o -$(MODULE)-objs += zfs_sa.o -$(MODULE)-objs += zfs_vnops.o -$(MODULE)-objs += zil.o -$(MODULE)-objs += zio.o -$(MODULE)-objs += zio_checksum.o -$(MODULE)-objs += zio_compress.o -$(MODULE)-objs += zio_inject.o -$(MODULE)-objs += zle.o -$(MODULE)-objs += zrlock.o -$(MODULE)-objs += zthr.o -$(MODULE)-objs += zvol.o - -# Suppress incorrect warnings from versions of objtool which are not -# aware of x86 EVEX prefix instructions used for AVX512. -OBJECT_FILES_NON_STANDARD_vdev_raidz_math_avx512bw.o := y -OBJECT_FILES_NON_STANDARD_vdev_raidz_math_avx512f.o := y - -$(MODULE)-$(CONFIG_X86) += vdev_raidz_math_sse2.o -$(MODULE)-$(CONFIG_X86) += vdev_raidz_math_ssse3.o -$(MODULE)-$(CONFIG_X86) += vdev_raidz_math_avx2.o -$(MODULE)-$(CONFIG_X86) += vdev_raidz_math_avx512f.o -$(MODULE)-$(CONFIG_X86) += vdev_raidz_math_avx512bw.o - -$(MODULE)-$(CONFIG_ARM64) += vdev_raidz_math_aarch64_neon.o -$(MODULE)-$(CONFIG_ARM64) += vdev_raidz_math_aarch64_neonx2.o - -$(MODULE)-$(CONFIG_PPC) += vdev_raidz_math_powerpc_altivec.o -$(MODULE)-$(CONFIG_PPC64) += vdev_raidz_math_powerpc_altivec.o - -ifeq ($(CONFIG_ALTIVEC),y) -$(obj)/vdev_raidz_math_powerpc_altivec.o: c_flags += -maltivec -endif - -include $(mfdir)/../os/linux/zfs/Makefile diff --git a/module/zfs/abd.c b/module/zfs/abd.c index b6d7ac6407e3..11a1e5112544 100644 --- a/module/zfs/abd.c +++ b/module/zfs/abd.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/aggsum.c b/module/zfs/aggsum.c index c4ea4f86fc5f..488c6ef3b6fc 100644 --- a/module/zfs/aggsum.c +++ b/module/zfs/aggsum.c @@ -87,7 +87,7 @@ static uint_t aggsum_borrow_shift = 4; void aggsum_init(aggsum_t *as, uint64_t value) { - bzero(as, sizeof (*as)); + memset(as, 0, sizeof (*as)); as->as_lower_bound = as->as_upper_bound = value; mutex_init(&as->as_lock, NULL, MUTEX_DEFAULT, NULL); /* diff --git a/module/zfs/arc.c b/module/zfs/arc.c index 7ca37c17642e..8f6ca1a28d68 100644 --- a/module/zfs/arc.c +++ b/module/zfs/arc.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -250,7 +250,7 @@ * since the physical block is about to be rewritten. The new data contents * will be contained in the arc_buf_t. As the I/O pipeline performs the write, * it may compress the data before writing it to disk. The ARC will be called - * with the transformed data and will bcopy the transformed on-disk block into + * with the transformed data and will memcpy the transformed on-disk block into * a newly allocated b_pabd. Writes are always done into buffers which have * either been loaned (and hence are new and don't have other readers) or * buffers which have been released (and hence have their own hdr, if there @@ -848,12 +848,13 @@ enum arc_hdr_alloc_flags { }; -static abd_t *arc_get_data_abd(arc_buf_hdr_t *, uint64_t, void *, int); -static void *arc_get_data_buf(arc_buf_hdr_t *, uint64_t, void *); -static void arc_get_data_impl(arc_buf_hdr_t *, uint64_t, void *, int); -static void arc_free_data_abd(arc_buf_hdr_t *, abd_t *, uint64_t, void *); -static void arc_free_data_buf(arc_buf_hdr_t *, void *, uint64_t, void *); -static void arc_free_data_impl(arc_buf_hdr_t *hdr, uint64_t size, void *tag); +static abd_t *arc_get_data_abd(arc_buf_hdr_t *, uint64_t, const void *, int); +static void *arc_get_data_buf(arc_buf_hdr_t *, uint64_t, const void *); +static void arc_get_data_impl(arc_buf_hdr_t *, uint64_t, const void *, int); +static void arc_free_data_abd(arc_buf_hdr_t *, abd_t *, uint64_t, const void *); +static void arc_free_data_buf(arc_buf_hdr_t *, void *, uint64_t, const void *); +static void arc_free_data_impl(arc_buf_hdr_t *hdr, uint64_t size, + const void *tag); static void arc_hdr_free_abd(arc_buf_hdr_t *, boolean_t); static void arc_hdr_alloc_abd(arc_buf_hdr_t *, int); static void arc_access(arc_buf_hdr_t *, kmutex_t *); @@ -928,7 +929,7 @@ static unsigned long l2arc_rebuild_blocks_min_l2size = 1024 * 1024 * 1024; /* L2ARC persistence rebuild control routines. */ void l2arc_rebuild_vdev(vdev_t *vd, boolean_t reopen); -static void l2arc_dev_rebuild_thread(void *arg); +static __attribute__((noreturn)) void l2arc_dev_rebuild_thread(void *arg); static int l2arc_rebuild(l2arc_dev_t *dev); /* L2ARC persistence read I/O routines. */ @@ -1132,7 +1133,7 @@ hdr_full_cons(void *vbuf, void *unused, int kmflag) (void) unused, (void) kmflag; arc_buf_hdr_t *hdr = vbuf; - bzero(hdr, HDR_FULL_SIZE); + memset(hdr, 0, HDR_FULL_SIZE); hdr->b_l1hdr.b_byteswap = DMU_BSWAP_NUMFUNCS; cv_init(&hdr->b_l1hdr.b_cv, NULL, CV_DEFAULT, NULL); zfs_refcount_create(&hdr->b_l1hdr.b_refcnt); @@ -1152,7 +1153,7 @@ hdr_full_crypt_cons(void *vbuf, void *unused, int kmflag) arc_buf_hdr_t *hdr = vbuf; hdr_full_cons(vbuf, unused, kmflag); - bzero(&hdr->b_crypt_hdr, sizeof (hdr->b_crypt_hdr)); + memset(&hdr->b_crypt_hdr, 0, sizeof (hdr->b_crypt_hdr)); arc_space_consume(sizeof (hdr->b_crypt_hdr), ARC_SPACE_HDRS); return (0); @@ -1164,7 +1165,7 @@ hdr_l2only_cons(void *vbuf, void *unused, int kmflag) (void) unused, (void) kmflag; arc_buf_hdr_t *hdr = vbuf; - bzero(hdr, HDR_L2ONLY_SIZE); + memset(hdr, 0, HDR_L2ONLY_SIZE); arc_space_consume(HDR_L2ONLY_SIZE, ARC_SPACE_L2HDRS); return (0); @@ -1176,7 +1177,7 @@ buf_cons(void *vbuf, void *unused, int kmflag) (void) unused, (void) kmflag; arc_buf_t *buf = vbuf; - bzero(buf, sizeof (arc_buf_t)); + memset(buf, 0, sizeof (arc_buf_t)); mutex_init(&buf->b_evict_lock, NULL, MUTEX_DEFAULT, NULL); arc_space_consume(sizeof (arc_buf_t), ARC_SPACE_HDRS); @@ -1204,11 +1205,11 @@ hdr_full_dest(void *vbuf, void *unused) static void hdr_full_crypt_dest(void *vbuf, void *unused) { - (void) unused; - arc_buf_hdr_t *hdr = vbuf; + (void) vbuf, (void) unused; hdr_full_dest(vbuf, unused); - arc_space_return(sizeof (hdr->b_crypt_hdr), ARC_SPACE_HDRS); + arc_space_return(sizeof (((arc_buf_hdr_t *)NULL)->b_crypt_hdr), + ARC_SPACE_HDRS); } static void @@ -1332,9 +1333,9 @@ arc_get_raw_params(arc_buf_t *buf, boolean_t *byteorder, uint8_t *salt, ASSERT(HDR_PROTECTED(hdr)); - bcopy(hdr->b_crypt_hdr.b_salt, salt, ZIO_DATA_SALT_LEN); - bcopy(hdr->b_crypt_hdr.b_iv, iv, ZIO_DATA_IV_LEN); - bcopy(hdr->b_crypt_hdr.b_mac, mac, ZIO_DATA_MAC_LEN); + memcpy(salt, hdr->b_crypt_hdr.b_salt, ZIO_DATA_SALT_LEN); + memcpy(iv, hdr->b_crypt_hdr.b_iv, ZIO_DATA_IV_LEN); + memcpy(mac, hdr->b_crypt_hdr.b_mac, ZIO_DATA_MAC_LEN); *byteorder = (hdr->b_l1hdr.b_byteswap == DMU_BSWAP_NUMFUNCS) ? ZFS_HOST_BYTEORDER : !ZFS_HOST_BYTEORDER; } @@ -1692,7 +1693,7 @@ arc_buf_try_copy_decompressed_data(arc_buf_t *buf) } if (!ARC_BUF_COMPRESSED(from)) { - bcopy(from->b_data, buf->b_data, arc_buf_size(buf)); + memcpy(buf->b_data, from->b_data, arc_buf_size(buf)); copied = B_TRUE; break; } @@ -2280,7 +2281,7 @@ arc_evictable_space_decrement(arc_buf_hdr_t *hdr, arc_state_t *state) * it is not evictable. */ static void -add_reference(arc_buf_hdr_t *hdr, void *tag) +add_reference(arc_buf_hdr_t *hdr, const void *tag) { arc_state_t *state; @@ -2316,7 +2317,7 @@ add_reference(arc_buf_hdr_t *hdr, void *tag) * list making it eligible for eviction. */ static int -remove_reference(arc_buf_hdr_t *hdr, kmutex_t *hash_lock, void *tag) +remove_reference(arc_buf_hdr_t *hdr, kmutex_t *hash_lock, const void *tag) { int cnt; arc_state_t *state = hdr->b_l1hdr.b_state; @@ -2740,8 +2741,8 @@ arc_can_share(arc_buf_hdr_t *hdr, arc_buf_t *buf) */ static int arc_buf_alloc_impl(arc_buf_hdr_t *hdr, spa_t *spa, const zbookmark_phys_t *zb, - void *tag, boolean_t encrypted, boolean_t compressed, boolean_t noauth, - boolean_t fill, arc_buf_t **ret) + const void *tag, boolean_t encrypted, boolean_t compressed, + boolean_t noauth, boolean_t fill, arc_buf_t **ret) { arc_buf_t *buf; arc_fill_flags_t flags = ARC_FILL_LOCKED; @@ -2841,7 +2842,7 @@ arc_buf_alloc_impl(arc_buf_hdr_t *hdr, spa_t *spa, const zbookmark_phys_t *zb, return (0); } -static char *arc_onloan_tag = "onloan"; +static const char *arc_onloan_tag = "onloan"; static inline void arc_loaned_bytes_update(int64_t delta) @@ -2900,7 +2901,7 @@ arc_loan_raw_buf(spa_t *spa, uint64_t dsobj, boolean_t byteorder, * Return a loaned arc buffer to the arc. */ void -arc_return_buf(arc_buf_t *buf, void *tag) +arc_return_buf(arc_buf_t *buf, const void *tag) { arc_buf_hdr_t *hdr = buf->b_hdr; @@ -2914,7 +2915,7 @@ arc_return_buf(arc_buf_t *buf, void *tag) /* Detach an arc_buf from a dbuf (tag) */ void -arc_loan_inuse_buf(arc_buf_t *buf, void *tag) +arc_loan_inuse_buf(arc_buf_t *buf, const void *tag) { arc_buf_hdr_t *hdr = buf->b_hdr; @@ -3349,7 +3350,7 @@ arc_hdr_realloc(arc_buf_hdr_t *hdr, kmem_cache_t *old, kmem_cache_t *new) ASSERT(MUTEX_HELD(HDR_LOCK(hdr))); buf_hash_remove(hdr); - bcopy(hdr, nhdr, HDR_L2ONLY_SIZE); + memcpy(nhdr, hdr, HDR_L2ONLY_SIZE); if (new == hdr_full_cache || new == hdr_full_crypt_cache) { arc_hdr_set_flags(nhdr, ARC_FLAG_HAS_L1HDR); @@ -3512,7 +3513,7 @@ arc_hdr_realloc_crypt(arc_buf_hdr_t *hdr, boolean_t need_crypt) } /* unset all members of the original hdr */ - bzero(&hdr->b_dva, sizeof (dva_t)); + memset(&hdr->b_dva, 0, sizeof (dva_t)); hdr->b_birth = 0; hdr->b_type = ARC_BUFC_INVALID; hdr->b_flags = 0; @@ -3537,9 +3538,9 @@ arc_hdr_realloc_crypt(arc_buf_hdr_t *hdr, boolean_t need_crypt) hdr->b_crypt_hdr.b_ot = DMU_OT_NONE; hdr->b_crypt_hdr.b_ebufcnt = 0; hdr->b_crypt_hdr.b_dsobj = 0; - bzero(hdr->b_crypt_hdr.b_salt, ZIO_DATA_SALT_LEN); - bzero(hdr->b_crypt_hdr.b_iv, ZIO_DATA_IV_LEN); - bzero(hdr->b_crypt_hdr.b_mac, ZIO_DATA_MAC_LEN); + memset(hdr->b_crypt_hdr.b_salt, 0, ZIO_DATA_SALT_LEN); + memset(hdr->b_crypt_hdr.b_iv, 0, ZIO_DATA_IV_LEN); + memset(hdr->b_crypt_hdr.b_mac, 0, ZIO_DATA_MAC_LEN); } buf_discard_identity(hdr); @@ -3577,11 +3578,11 @@ arc_convert_to_raw(arc_buf_t *buf, uint64_t dsobj, boolean_t byteorder, arc_cksum_free(hdr); if (salt != NULL) - bcopy(salt, hdr->b_crypt_hdr.b_salt, ZIO_DATA_SALT_LEN); + memcpy(hdr->b_crypt_hdr.b_salt, salt, ZIO_DATA_SALT_LEN); if (iv != NULL) - bcopy(iv, hdr->b_crypt_hdr.b_iv, ZIO_DATA_IV_LEN); + memcpy(hdr->b_crypt_hdr.b_iv, iv, ZIO_DATA_IV_LEN); if (mac != NULL) - bcopy(mac, hdr->b_crypt_hdr.b_mac, ZIO_DATA_MAC_LEN); + memcpy(hdr->b_crypt_hdr.b_mac, mac, ZIO_DATA_MAC_LEN); } /* @@ -3589,7 +3590,8 @@ arc_convert_to_raw(arc_buf_t *buf, uint64_t dsobj, boolean_t byteorder, * The buf is returned thawed since we expect the consumer to modify it. */ arc_buf_t * -arc_alloc_buf(spa_t *spa, void *tag, arc_buf_contents_t type, int32_t size) +arc_alloc_buf(spa_t *spa, const void *tag, arc_buf_contents_t type, + int32_t size) { arc_buf_hdr_t *hdr = arc_hdr_alloc(spa_load_guid(spa), size, size, B_FALSE, ZIO_COMPRESS_OFF, 0, type); @@ -3607,8 +3609,8 @@ arc_alloc_buf(spa_t *spa, void *tag, arc_buf_contents_t type, int32_t size) * for bufs containing metadata. */ arc_buf_t * -arc_alloc_compressed_buf(spa_t *spa, void *tag, uint64_t psize, uint64_t lsize, - enum zio_compress compression_type, uint8_t complevel) +arc_alloc_compressed_buf(spa_t *spa, const void *tag, uint64_t psize, + uint64_t lsize, enum zio_compress compression_type, uint8_t complevel) { ASSERT3U(lsize, >, 0); ASSERT3U(lsize, >=, psize); @@ -3635,9 +3637,9 @@ arc_alloc_compressed_buf(spa_t *spa, void *tag, uint64_t psize, uint64_t lsize, } arc_buf_t * -arc_alloc_raw_buf(spa_t *spa, void *tag, uint64_t dsobj, boolean_t byteorder, - const uint8_t *salt, const uint8_t *iv, const uint8_t *mac, - dmu_object_type_t ot, uint64_t psize, uint64_t lsize, +arc_alloc_raw_buf(spa_t *spa, const void *tag, uint64_t dsobj, + boolean_t byteorder, const uint8_t *salt, const uint8_t *iv, + const uint8_t *mac, dmu_object_type_t ot, uint64_t psize, uint64_t lsize, enum zio_compress compression_type, uint8_t complevel) { arc_buf_hdr_t *hdr; @@ -3657,9 +3659,9 @@ arc_alloc_raw_buf(spa_t *spa, void *tag, uint64_t dsobj, boolean_t byteorder, hdr->b_crypt_hdr.b_ot = ot; hdr->b_l1hdr.b_byteswap = (byteorder == ZFS_HOST_BYTEORDER) ? DMU_BSWAP_NUMFUNCS : DMU_OT_BYTESWAP(ot); - bcopy(salt, hdr->b_crypt_hdr.b_salt, ZIO_DATA_SALT_LEN); - bcopy(iv, hdr->b_crypt_hdr.b_iv, ZIO_DATA_IV_LEN); - bcopy(mac, hdr->b_crypt_hdr.b_mac, ZIO_DATA_MAC_LEN); + memcpy(hdr->b_crypt_hdr.b_salt, salt, ZIO_DATA_SALT_LEN); + memcpy(hdr->b_crypt_hdr.b_iv, iv, ZIO_DATA_IV_LEN); + memcpy(hdr->b_crypt_hdr.b_mac, mac, ZIO_DATA_MAC_LEN); /* * This buffer will be considered encrypted even if the ot is not an @@ -3844,7 +3846,7 @@ arc_hdr_destroy(arc_buf_hdr_t *hdr) } void -arc_buf_destroy(arc_buf_t *buf, void* tag) +arc_buf_destroy(arc_buf_t *buf, const void *tag) { arc_buf_hdr_t *hdr = buf->b_hdr; @@ -5207,7 +5209,7 @@ arc_is_overflowing(boolean_t use_reserve) } static abd_t * -arc_get_data_abd(arc_buf_hdr_t *hdr, uint64_t size, void *tag, +arc_get_data_abd(arc_buf_hdr_t *hdr, uint64_t size, const void *tag, int alloc_flags) { arc_buf_contents_t type = arc_buf_type(hdr); @@ -5222,7 +5224,7 @@ arc_get_data_abd(arc_buf_hdr_t *hdr, uint64_t size, void *tag, } static void * -arc_get_data_buf(arc_buf_hdr_t *hdr, uint64_t size, void *tag) +arc_get_data_buf(arc_buf_hdr_t *hdr, uint64_t size, const void *tag) { arc_buf_contents_t type = arc_buf_type(hdr); @@ -5321,7 +5323,7 @@ arc_wait_for_eviction(uint64_t amount, boolean_t use_reserve) * limit, we'll only signal the reclaim thread and continue on. */ static void -arc_get_data_impl(arc_buf_hdr_t *hdr, uint64_t size, void *tag, +arc_get_data_impl(arc_buf_hdr_t *hdr, uint64_t size, const void *tag, int alloc_flags) { arc_state_t *state = hdr->b_l1hdr.b_state; @@ -5389,14 +5391,15 @@ arc_get_data_impl(arc_buf_hdr_t *hdr, uint64_t size, void *tag, } static void -arc_free_data_abd(arc_buf_hdr_t *hdr, abd_t *abd, uint64_t size, void *tag) +arc_free_data_abd(arc_buf_hdr_t *hdr, abd_t *abd, uint64_t size, + const void *tag) { arc_free_data_impl(hdr, size, tag); abd_free(abd); } static void -arc_free_data_buf(arc_buf_hdr_t *hdr, void *buf, uint64_t size, void *tag) +arc_free_data_buf(arc_buf_hdr_t *hdr, void *buf, uint64_t size, const void *tag) { arc_buf_contents_t type = arc_buf_type(hdr); @@ -5413,7 +5416,7 @@ arc_free_data_buf(arc_buf_hdr_t *hdr, void *buf, uint64_t size, void *tag) * Free the arc data buffer. */ static void -arc_free_data_impl(arc_buf_hdr_t *hdr, uint64_t size, void *tag) +arc_free_data_impl(arc_buf_hdr_t *hdr, uint64_t size, const void *tag) { arc_state_t *state = hdr->b_l1hdr.b_state; arc_buf_contents_t type = arc_buf_type(hdr); @@ -5643,7 +5646,7 @@ arc_bcopy_func(zio_t *zio, const zbookmark_phys_t *zb, const blkptr_t *bp, if (buf == NULL) return; - bcopy(buf->b_data, arg, arc_buf_size(buf)); + memcpy(arg, buf->b_data, arc_buf_size(buf)); arc_buf_destroy(buf, arg); } @@ -6575,7 +6578,7 @@ arc_freed(spa_t *spa, const blkptr_t *bp) * a new hdr for the buffer. */ void -arc_release(arc_buf_t *buf, void *tag) +arc_release(arc_buf_t *buf, const void *tag) { arc_buf_hdr_t *hdr = buf->b_hdr; @@ -7106,11 +7109,11 @@ arc_write(zio_t *pio, spa_t *spa, uint64_t txg, localprop.zp_byteorder = (hdr->b_l1hdr.b_byteswap == DMU_BSWAP_NUMFUNCS) ? ZFS_HOST_BYTEORDER : !ZFS_HOST_BYTEORDER; - bcopy(hdr->b_crypt_hdr.b_salt, localprop.zp_salt, + memcpy(localprop.zp_salt, hdr->b_crypt_hdr.b_salt, ZIO_DATA_SALT_LEN); - bcopy(hdr->b_crypt_hdr.b_iv, localprop.zp_iv, + memcpy(localprop.zp_iv, hdr->b_crypt_hdr.b_iv, ZIO_DATA_IV_LEN); - bcopy(hdr->b_crypt_hdr.b_mac, localprop.zp_mac, + memcpy(localprop.zp_mac, hdr->b_crypt_hdr.b_mac, ZIO_DATA_MAC_LEN); if (DMU_OT_IS_ENCRYPTED(localprop.zp_type)) { localprop.zp_nopwrite = B_FALSE; @@ -8722,14 +8725,15 @@ l2arc_write_done(zio_t *zio) * block pointer in the header. */ if (i == 0) { - bzero(l2dhdr, dev->l2ad_dev_hdr_asize); + memset(l2dhdr, 0, + dev->l2ad_dev_hdr_asize); } else { - bzero(&l2dhdr->dh_start_lbps[i], + memset(&l2dhdr->dh_start_lbps[i], 0, sizeof (l2arc_log_blkptr_t)); } break; } - bcopy(lb_ptr_buf->lb_ptr, &l2dhdr->dh_start_lbps[i], + memcpy(&l2dhdr->dh_start_lbps[i], lb_ptr_buf->lb_ptr, sizeof (l2arc_log_blkptr_t)); lb_ptr_buf = list_next(&dev->l2ad_lbptr_list, lb_ptr_buf); @@ -9336,26 +9340,37 @@ l2arc_apply_transforms(spa_t *spa, arc_buf_hdr_t *hdr, uint64_t asize, } if (compress != ZIO_COMPRESS_OFF && !HDR_COMPRESSION_ENABLED(hdr)) { - cabd = abd_alloc_for_io(asize, ismd); - tmp = abd_borrow_buf(cabd, asize); + /* + * In some cases, we can wind up with size > asize, so + * we need to opt for the larger allocation option here. + * + * (We also need abd_return_buf_copy in all cases because + * it's an ASSERT() to modify the buffer before returning it + * with arc_return_buf(), and all the compressors + * write things before deciding to fail compression in nearly + * every case.) + */ + cabd = abd_alloc_for_io(size, ismd); + tmp = abd_borrow_buf(cabd, size); psize = zio_compress_data(compress, to_write, tmp, size, hdr->b_complevel); - if (psize >= size) { - abd_return_buf(cabd, tmp, asize); + if (psize >= asize) { + psize = HDR_GET_PSIZE(hdr); + abd_return_buf_copy(cabd, tmp, size); HDR_SET_COMPRESS(hdr, ZIO_COMPRESS_OFF); to_write = cabd; - abd_copy(to_write, hdr->b_l1hdr.b_pabd, size); - if (size != asize) - abd_zero_off(to_write, size, asize - size); + abd_copy(to_write, hdr->b_l1hdr.b_pabd, psize); + if (psize != asize) + abd_zero_off(to_write, psize, asize - psize); goto encrypt; } ASSERT3U(psize, <=, HDR_GET_PSIZE(hdr)); if (psize < asize) - bzero((char *)tmp + psize, asize - psize); + memset((char *)tmp + psize, 0, asize - psize); psize = HDR_GET_PSIZE(hdr); - abd_return_buf_copy(cabd, tmp, asize); + abd_return_buf_copy(cabd, tmp, size); to_write = cabd; } @@ -9388,7 +9403,7 @@ l2arc_apply_transforms(spa_t *spa, arc_buf_hdr_t *hdr, uint64_t asize, abd_zero_off(eabd, psize, asize - psize); /* assert that the MAC we got here matches the one we saved */ - ASSERT0(bcmp(mac, hdr->b_crypt_hdr.b_mac, ZIO_DATA_MAC_LEN)); + ASSERT0(memcmp(mac, hdr->b_crypt_hdr.b_mac, ZIO_DATA_MAC_LEN)); spa_keystore_dsl_key_rele(spa, dck, FTAG); if (to_write == cabd) @@ -9706,7 +9721,7 @@ l2arc_hdr_limit_reached(void) * This thread feeds the L2ARC at regular intervals. This is the beating * heart of the L2ARC. */ -static void +static __attribute__((noreturn)) void l2arc_feed_thread(void *unused) { (void) unused; @@ -9897,7 +9912,7 @@ l2arc_rebuild_dev(l2arc_dev_t *dev, boolean_t reopen) if (l2arc_trim_ahead > 0) { dev->l2ad_trim_all = B_TRUE; } else { - bzero(l2dhdr, l2dhdr_asize); + memset(l2dhdr, 0, l2dhdr_asize); l2arc_dev_hdr_update(dev); } } @@ -10145,7 +10160,7 @@ l2arc_spa_rebuild_start(spa_t *spa) /* * Main entry point for L2ARC rebuilding. */ -static void +static __attribute__((noreturn)) void l2arc_dev_rebuild_thread(void *arg) { l2arc_dev_t *dev = arg; @@ -10218,7 +10233,7 @@ l2arc_rebuild(l2arc_dev_t *dev) goto out; /* Prepare the rebuild process */ - bcopy(l2dhdr->dh_start_lbps, lbps, sizeof (lbps)); + memcpy(lbps, l2dhdr->dh_start_lbps, sizeof (lbps)); /* Start the rebuild process */ for (;;) { @@ -10264,7 +10279,7 @@ l2arc_rebuild(l2arc_dev_t *dev) lb_ptr_buf = kmem_zalloc(sizeof (l2arc_lb_ptr_buf_t), KM_SLEEP); lb_ptr_buf->lb_ptr = kmem_zalloc(sizeof (l2arc_log_blkptr_t), KM_SLEEP); - bcopy(&lbps[0], lb_ptr_buf->lb_ptr, + memcpy(lb_ptr_buf->lb_ptr, &lbps[0], sizeof (l2arc_log_blkptr_t)); mutex_enter(&dev->l2ad_mtx); list_insert_tail(&dev->l2ad_lbptr_list, lb_ptr_buf); @@ -10362,7 +10377,7 @@ l2arc_rebuild(l2arc_dev_t *dev) */ spa_history_log_internal(spa, "L2ARC rebuild", NULL, "no valid log blocks"); - bzero(l2dhdr, dev->l2ad_dev_hdr_asize); + memset(l2dhdr, 0, dev->l2ad_dev_hdr_asize); l2arc_dev_hdr_update(dev); } else if (err == ECANCELED) { /* @@ -10853,13 +10868,13 @@ l2arc_log_blk_commit(l2arc_dev_t *dev, zio_t *pio, l2arc_write_callback_t *cb) ZIO_CHECKSUM_FLETCHER_4); if (asize < sizeof (*lb)) { /* compression succeeded */ - bzero(tmpbuf + psize, asize - psize); + memset(tmpbuf + psize, 0, asize - psize); L2BLK_SET_COMPRESS( (&l2dhdr->dh_start_lbps[0])->lbp_prop, ZIO_COMPRESS_LZ4); } else { /* compression failed */ - bcopy(lb, tmpbuf, sizeof (*lb)); + memcpy(tmpbuf, lb, sizeof (*lb)); L2BLK_SET_COMPRESS( (&l2dhdr->dh_start_lbps[0])->lbp_prop, ZIO_COMPRESS_OFF); @@ -10885,7 +10900,7 @@ l2arc_log_blk_commit(l2arc_dev_t *dev, zio_t *pio, l2arc_write_callback_t *cb) * Include the committed log block's pointer in the list of pointers * to log blocks present in the L2ARC device. */ - bcopy(&l2dhdr->dh_start_lbps[0], lb_ptr_buf->lb_ptr, + memcpy(lb_ptr_buf->lb_ptr, &l2dhdr->dh_start_lbps[0], sizeof (l2arc_log_blkptr_t)); mutex_enter(&dev->l2ad_mtx); list_insert_head(&dev->l2ad_lbptr_list, lb_ptr_buf); @@ -10974,7 +10989,7 @@ l2arc_log_blk_insert(l2arc_dev_t *dev, const arc_buf_hdr_t *hdr) ASSERT(HDR_HAS_L2HDR(hdr)); le = &lb->lb_entries[index]; - bzero(le, sizeof (*le)); + memset(le, 0, sizeof (*le)); le->le_dva = hdr->b_dva; le->le_birth = hdr->b_birth; le->le_daddr = hdr->b_l2hdr.b_daddr; @@ -11044,20 +11059,20 @@ EXPORT_SYMBOL(arc_add_prune_callback); EXPORT_SYMBOL(arc_remove_prune_callback); ZFS_MODULE_PARAM_CALL(zfs_arc, zfs_arc_, min, param_set_arc_min, - param_get_long, ZMOD_RW, "Min arc size"); + param_get_long, ZMOD_RW, "Minimum ARC size in bytes"); ZFS_MODULE_PARAM_CALL(zfs_arc, zfs_arc_, max, param_set_arc_max, - param_get_long, ZMOD_RW, "Max arc size"); + param_get_long, ZMOD_RW, "Maximum ARC size in bytes"); ZFS_MODULE_PARAM_CALL(zfs_arc, zfs_arc_, meta_limit, param_set_arc_long, - param_get_long, ZMOD_RW, "Metadata limit for arc size"); + param_get_long, ZMOD_RW, "Metadata limit for ARC size in bytes"); ZFS_MODULE_PARAM_CALL(zfs_arc, zfs_arc_, meta_limit_percent, param_set_arc_long, param_get_long, ZMOD_RW, - "Percent of arc size for arc meta limit"); + "Percent of ARC size for ARC meta limit"); ZFS_MODULE_PARAM_CALL(zfs_arc, zfs_arc_, meta_min, param_set_arc_long, - param_get_long, ZMOD_RW, "Min arc metadata"); + param_get_long, ZMOD_RW, "Minimum ARC metadata size in bytes"); ZFS_MODULE_PARAM(zfs_arc, zfs_arc_, meta_prune, INT, ZMOD_RW, "Meta objects to scan for prune"); @@ -11069,16 +11084,16 @@ ZFS_MODULE_PARAM(zfs_arc, zfs_arc_, meta_strategy, INT, ZMOD_RW, "Meta reclaim strategy"); ZFS_MODULE_PARAM_CALL(zfs_arc, zfs_arc_, grow_retry, param_set_arc_int, - param_get_int, ZMOD_RW, "Seconds before growing arc size"); + param_get_int, ZMOD_RW, "Seconds before growing ARC size"); ZFS_MODULE_PARAM(zfs_arc, zfs_arc_, p_dampener_disable, INT, ZMOD_RW, "Disable arc_p adapt dampener"); ZFS_MODULE_PARAM_CALL(zfs_arc, zfs_arc_, shrink_shift, param_set_arc_int, - param_get_int, ZMOD_RW, "log2(fraction of arc to reclaim)"); + param_get_int, ZMOD_RW, "log2(fraction of ARC to reclaim)"); ZFS_MODULE_PARAM(zfs_arc, zfs_arc_, pc_percent, UINT, ZMOD_RW, - "Percent of pagecache to reclaim arc to"); + "Percent of pagecache to reclaim ARC to"); ZFS_MODULE_PARAM_CALL(zfs_arc, zfs_arc_, p_min_shift, param_set_arc_int, param_get_int, ZMOD_RW, "arc_c shift to calc min/max arc_p"); @@ -11087,7 +11102,7 @@ ZFS_MODULE_PARAM(zfs_arc, zfs_arc_, average_blocksize, INT, ZMOD_RD, "Target average block size"); ZFS_MODULE_PARAM(zfs, zfs_, compressed_arc_enabled, INT, ZMOD_RW, - "Disable compressed arc buffers"); + "Disable compressed ARC buffers"); ZFS_MODULE_PARAM_CALL(zfs_arc, zfs_arc_, min_prefetch_ms, param_set_arc_int, param_get_int, ZMOD_RW, "Min life of prefetch block in ms"); @@ -11148,7 +11163,7 @@ ZFS_MODULE_PARAM_CALL(zfs_arc, zfs_arc_, sys_free, param_set_arc_long, param_get_long, ZMOD_RW, "System free memory target size in bytes"); ZFS_MODULE_PARAM_CALL(zfs_arc, zfs_arc_, dnode_limit, param_set_arc_long, - param_get_long, ZMOD_RW, "Minimum bytes of dnodes in arc"); + param_get_long, ZMOD_RW, "Minimum bytes of dnodes in ARC"); ZFS_MODULE_PARAM_CALL(zfs_arc, zfs_arc_, dnode_limit_percent, param_set_arc_long, param_get_long, ZMOD_RW, diff --git a/module/zfs/blake3_zfs.c b/module/zfs/blake3_zfs.c new file mode 100644 index 000000000000..7560f30fd4e4 --- /dev/null +++ b/module/zfs/blake3_zfs.c @@ -0,0 +1,117 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2022 Tino Reichardt + */ + +#include +#include +#include +#include + +static int +blake3_incremental(void *buf, size_t size, void *arg) +{ + BLAKE3_CTX *ctx = arg; + + Blake3_Update(ctx, buf, size); + + return (0); +} + +/* + * Computes a native 256-bit BLAKE3 MAC checksum. Please note that this + * function requires the presence of a ctx_template that should be allocated + * using abd_checksum_blake3_tmpl_init. + */ +void +abd_checksum_blake3_native(abd_t *abd, uint64_t size, const void *ctx_template, + zio_cksum_t *zcp) +{ + ASSERT(ctx_template != 0); + +#if defined(_KERNEL) + BLAKE3_CTX *ctx = blake3_per_cpu_ctx[CPU_SEQID_UNSTABLE]; +#else + BLAKE3_CTX *ctx = kmem_alloc(sizeof (*ctx), KM_SLEEP); +#endif + + memcpy(ctx, ctx_template, sizeof (*ctx)); + (void) abd_iterate_func(abd, 0, size, blake3_incremental, ctx); + Blake3_Final(ctx, (uint8_t *)zcp); + +#if !defined(_KERNEL) + memset(ctx, 0, sizeof (*ctx)); + kmem_free(ctx, sizeof (*ctx)); +#endif +} + +/* + * Byteswapped version of abd_checksum_blake3_native. This just invokes + * the native checksum function and byteswaps the resulting checksum (since + * BLAKE3 is internally endian-insensitive). + */ +void +abd_checksum_blake3_byteswap(abd_t *abd, uint64_t size, + const void *ctx_template, zio_cksum_t *zcp) +{ + zio_cksum_t tmp; + + ASSERT(ctx_template != 0); + + abd_checksum_blake3_native(abd, size, ctx_template, &tmp); + zcp->zc_word[0] = BSWAP_64(tmp.zc_word[0]); + zcp->zc_word[1] = BSWAP_64(tmp.zc_word[1]); + zcp->zc_word[2] = BSWAP_64(tmp.zc_word[2]); + zcp->zc_word[3] = BSWAP_64(tmp.zc_word[3]); +} + +/* + * Allocates a BLAKE3 MAC template suitable for using in BLAKE3 MAC checksum + * computations and returns a pointer to it. + */ +void * +abd_checksum_blake3_tmpl_init(const zio_cksum_salt_t *salt) +{ + BLAKE3_CTX *ctx; + + ASSERT(sizeof (salt->zcs_bytes) == 32); + + /* init reference object */ + ctx = kmem_zalloc(sizeof (*ctx), KM_SLEEP); + Blake3_InitKeyed(ctx, salt->zcs_bytes); + + return (ctx); +} + +/* + * Frees a BLAKE3 context template previously allocated using + * zio_checksum_blake3_tmpl_init. + */ +void +abd_checksum_blake3_tmpl_free(void *ctx_template) +{ + BLAKE3_CTX *ctx = ctx_template; + + memset(ctx, 0, sizeof (*ctx)); + kmem_free(ctx, sizeof (*ctx)); +} diff --git a/module/zfs/blkptr.c b/module/zfs/blkptr.c index aa09ded8dba3..d85f0737f6f6 100644 --- a/module/zfs/blkptr.c +++ b/module/zfs/blkptr.c @@ -58,7 +58,7 @@ encode_embedded_bp_compressed(blkptr_t *bp, void *data, ASSERT3U(comp, >=, ZIO_COMPRESS_OFF); ASSERT3U(comp, <, ZIO_COMPRESS_FUNCTIONS); - bzero(bp, sizeof (*bp)); + memset(bp, 0, sizeof (*bp)); BP_SET_EMBEDDED(bp, B_TRUE); BP_SET_COMPRESS(bp, comp); BP_SET_BYTEORDER(bp, ZFS_HOST_BYTEORDER); diff --git a/module/zfs/bplist.c b/module/zfs/bplist.c index 47ea364ef26f..1c1f7892bb7d 100644 --- a/module/zfs/bplist.c +++ b/module/zfs/bplist.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/bpobj.c b/module/zfs/bpobj.c index 68f534c6b197..f7fded56518b 100644 --- a/module/zfs/bpobj.c +++ b/module/zfs/bpobj.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -156,7 +156,7 @@ bpobj_open(bpobj_t *bpo, objset_t *os, uint64_t object) if (err) return (err); - bzero(bpo, sizeof (*bpo)); + memset(bpo, 0, sizeof (*bpo)); mutex_init(&bpo->bpo_lock, NULL, MUTEX_DEFAULT, NULL); ASSERT(bpo->bpo_dbuf == NULL); @@ -805,12 +805,12 @@ bpobj_enqueue(bpobj_t *bpo, const blkptr_t *bp, boolean_t bp_freed, * set of BP's stored, and bpobj_iterate() wouldn't visit * all the space accounted for in the bpobj. */ - bzero(&stored_bp, sizeof (stored_bp)); + memset(&stored_bp, 0, sizeof (stored_bp)); stored_bp.blk_prop = bp->blk_prop; stored_bp.blk_birth = bp->blk_birth; } else if (!BP_GET_DEDUP(bp)) { /* The bpobj will compress better without the checksum */ - bzero(&stored_bp.blk_cksum, sizeof (stored_bp.blk_cksum)); + memset(&stored_bp.blk_cksum, 0, sizeof (stored_bp.blk_cksum)); } stored_bp.blk_fill = 0; diff --git a/module/zfs/bptree.c b/module/zfs/bptree.c index 4e9a4825e262..1f5d8e77bcc0 100644 --- a/module/zfs/bptree.c +++ b/module/zfs/bptree.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/btree.c b/module/zfs/btree.c index 5bcf621d5994..14cab4054cbc 100644 --- a/module/zfs/btree.c +++ b/module/zfs/btree.c @@ -56,15 +56,27 @@ kmem_cache_t *zfs_btree_leaf_cache; int zfs_btree_verify_intensity = 0; /* - * A convenience function to silence warnings from memmove's return value and - * change argument order to src, dest. + * Convenience functions to silence warnings from memcpy/memmove's + * return values and change argument order to src, dest. */ +static void +bcpy(const void *src, void *dest, size_t size) +{ + (void) memcpy(dest, src, size); +} + static void bmov(const void *src, void *dest, size_t size) { (void) memmove(dest, src, size); } +static boolean_t +zfs_btree_is_core(struct zfs_btree_hdr *hdr) +{ + return (hdr->bth_first == -1); +} + #ifdef _ILP32 #define BTREE_POISON 0xabadb10c #else @@ -76,59 +88,74 @@ zfs_btree_poison_node(zfs_btree_t *tree, zfs_btree_hdr_t *hdr) { #ifdef ZFS_DEBUG size_t size = tree->bt_elem_size; - if (!hdr->bth_core) { - zfs_btree_leaf_t *leaf = (zfs_btree_leaf_t *)hdr; - (void) memset(leaf->btl_elems + hdr->bth_count * size, 0x0f, - BTREE_LEAF_SIZE - sizeof (zfs_btree_hdr_t) - - hdr->bth_count * size); - } else { + if (zfs_btree_is_core(hdr)) { zfs_btree_core_t *node = (zfs_btree_core_t *)hdr; - for (int i = hdr->bth_count + 1; i <= BTREE_CORE_ELEMS; i++) { + for (uint32_t i = hdr->bth_count + 1; i <= BTREE_CORE_ELEMS; + i++) { node->btc_children[i] = (zfs_btree_hdr_t *)BTREE_POISON; } (void) memset(node->btc_elems + hdr->bth_count * size, 0x0f, (BTREE_CORE_ELEMS - hdr->bth_count) * size); + } else { + zfs_btree_leaf_t *leaf = (zfs_btree_leaf_t *)hdr; + (void) memset(leaf->btl_elems, 0x0f, hdr->bth_first * size); + (void) memset(leaf->btl_elems + + (hdr->bth_first + hdr->bth_count) * size, 0x0f, + BTREE_LEAF_ESIZE - + (hdr->bth_first + hdr->bth_count) * size); } #endif } static inline void zfs_btree_poison_node_at(zfs_btree_t *tree, zfs_btree_hdr_t *hdr, - uint64_t offset) + uint32_t idx, uint32_t count) { #ifdef ZFS_DEBUG size_t size = tree->bt_elem_size; - ASSERT3U(offset, >=, hdr->bth_count); - if (!hdr->bth_core) { - zfs_btree_leaf_t *leaf = (zfs_btree_leaf_t *)hdr; - (void) memset(leaf->btl_elems + offset * size, 0x0f, size); - } else { + if (zfs_btree_is_core(hdr)) { + ASSERT3U(idx, >=, hdr->bth_count); + ASSERT3U(idx, <=, BTREE_CORE_ELEMS); + ASSERT3U(idx + count, <=, BTREE_CORE_ELEMS); zfs_btree_core_t *node = (zfs_btree_core_t *)hdr; - node->btc_children[offset + 1] = - (zfs_btree_hdr_t *)BTREE_POISON; - (void) memset(node->btc_elems + offset * size, 0x0f, size); + for (uint32_t i = 1; i <= count; i++) { + node->btc_children[idx + i] = + (zfs_btree_hdr_t *)BTREE_POISON; + } + (void) memset(node->btc_elems + idx * size, 0x0f, count * size); + } else { + ASSERT3U(idx, <=, tree->bt_leaf_cap); + ASSERT3U(idx + count, <=, tree->bt_leaf_cap); + zfs_btree_leaf_t *leaf = (zfs_btree_leaf_t *)hdr; + (void) memset(leaf->btl_elems + + (hdr->bth_first + idx) * size, 0x0f, count * size); } #endif } static inline void zfs_btree_verify_poison_at(zfs_btree_t *tree, zfs_btree_hdr_t *hdr, - uint64_t offset) + uint32_t idx) { #ifdef ZFS_DEBUG size_t size = tree->bt_elem_size; - uint8_t eval = 0x0f; - if (hdr->bth_core) { + if (zfs_btree_is_core(hdr)) { + ASSERT3U(idx, <, BTREE_CORE_ELEMS); zfs_btree_core_t *node = (zfs_btree_core_t *)hdr; zfs_btree_hdr_t *cval = (zfs_btree_hdr_t *)BTREE_POISON; - VERIFY3P(node->btc_children[offset + 1], ==, cval); - for (int i = 0; i < size; i++) - VERIFY3U(node->btc_elems[offset * size + i], ==, eval); + VERIFY3P(node->btc_children[idx + 1], ==, cval); + for (size_t i = 0; i < size; i++) + VERIFY3U(node->btc_elems[idx * size + i], ==, 0x0f); } else { + ASSERT3U(idx, <, tree->bt_leaf_cap); zfs_btree_leaf_t *leaf = (zfs_btree_leaf_t *)hdr; - for (int i = 0; i < size; i++) - VERIFY3U(leaf->btl_elems[offset * size + i], ==, eval); + if (idx >= tree->bt_leaf_cap - hdr->bth_first) + return; + for (size_t i = 0; i < size; i++) { + VERIFY3U(leaf->btl_elems[(hdr->bth_first + idx) + * size + i], ==, 0x0f); + } } #endif } @@ -137,8 +164,7 @@ void zfs_btree_init(void) { zfs_btree_leaf_cache = kmem_cache_create("zfs_btree_leaf_cache", - BTREE_LEAF_SIZE, 0, NULL, NULL, NULL, NULL, - NULL, 0); + BTREE_LEAF_SIZE, 0, NULL, NULL, NULL, NULL, NULL, 0); } void @@ -151,17 +177,12 @@ void zfs_btree_create(zfs_btree_t *tree, int (*compar) (const void *, const void *), size_t size) { - /* - * We need a minimmum of 4 elements so that when we split a node we - * always have at least two elements in each node. This simplifies the - * logic in zfs_btree_bulk_finish, since it means the last leaf will - * always have a left sibling to share with (unless it's the root). - */ - ASSERT3U(size, <=, (BTREE_LEAF_SIZE - sizeof (zfs_btree_hdr_t)) / 4); + ASSERT3U(size, <=, BTREE_LEAF_ESIZE / 2); - bzero(tree, sizeof (*tree)); + memset(tree, 0, sizeof (*tree)); tree->bt_compar = compar; tree->bt_elem_size = size; + tree->bt_leaf_cap = P2ALIGN(BTREE_LEAF_ESIZE / size, 2); tree->bt_height = -1; tree->bt_bulk = NULL; } @@ -170,21 +191,20 @@ zfs_btree_create(zfs_btree_t *tree, int (*compar) (const void *, const void *), * Find value in the array of elements provided. Uses a simple binary search. */ static void * -zfs_btree_find_in_buf(zfs_btree_t *tree, uint8_t *buf, uint64_t nelems, +zfs_btree_find_in_buf(zfs_btree_t *tree, uint8_t *buf, uint32_t nelems, const void *value, zfs_btree_index_t *where) { - uint64_t max = nelems; - uint64_t min = 0; + uint32_t max = nelems; + uint32_t min = 0; while (max > min) { - uint64_t idx = (min + max) / 2; + uint32_t idx = (min + max) / 2; uint8_t *cur = buf + idx * tree->bt_elem_size; int comp = tree->bt_compar(cur, value); - if (comp == -1) { + if (comp < 0) { min = idx + 1; - } else if (comp == 1) { + } else if (comp > 0) { max = idx; } else { - ASSERT0(comp); where->bti_offset = idx; where->bti_before = B_FALSE; return (cur); @@ -219,12 +239,13 @@ zfs_btree_find(zfs_btree_t *tree, const void *value, zfs_btree_index_t *where) * bulk-insert mode are to insert new elements. */ zfs_btree_index_t idx; + size_t size = tree->bt_elem_size; if (tree->bt_bulk != NULL) { zfs_btree_leaf_t *last_leaf = tree->bt_bulk; - int compar = tree->bt_compar(last_leaf->btl_elems + - ((last_leaf->btl_hdr.bth_count - 1) * tree->bt_elem_size), - value); - if (compar < 0) { + int comp = tree->bt_compar(last_leaf->btl_elems + + (last_leaf->btl_hdr.bth_first + + last_leaf->btl_hdr.bth_count - 1) * size, value); + if (comp < 0) { /* * If what they're looking for is after the last * element, it's not in the tree. @@ -236,7 +257,7 @@ zfs_btree_find(zfs_btree_t *tree, const void *value, zfs_btree_index_t *where) where->bti_before = B_TRUE; } return (NULL); - } else if (compar == 0) { + } else if (comp == 0) { if (where != NULL) { where->bti_node = (zfs_btree_hdr_t *)last_leaf; where->bti_offset = @@ -244,18 +265,20 @@ zfs_btree_find(zfs_btree_t *tree, const void *value, zfs_btree_index_t *where) where->bti_before = B_FALSE; } return (last_leaf->btl_elems + - ((last_leaf->btl_hdr.bth_count - 1) * - tree->bt_elem_size)); + (last_leaf->btl_hdr.bth_first + + last_leaf->btl_hdr.bth_count - 1) * size); } - if (tree->bt_compar(last_leaf->btl_elems, value) <= 0) { + if (tree->bt_compar(last_leaf->btl_elems + + last_leaf->btl_hdr.bth_first * size, value) <= 0) { /* * If what they're looking for is after the first * element in the last leaf, it's in the last leaf or * it's not in the tree. */ void *d = zfs_btree_find_in_buf(tree, - last_leaf->btl_elems, last_leaf->btl_hdr.bth_count, - value, &idx); + last_leaf->btl_elems + + last_leaf->btl_hdr.bth_first * size, + last_leaf->btl_hdr.bth_count, value, &idx); if (where != NULL) { idx.bti_node = (zfs_btree_hdr_t *)last_leaf; @@ -266,7 +289,7 @@ zfs_btree_find(zfs_btree_t *tree, const void *value, zfs_btree_index_t *where) } zfs_btree_core_t *node = NULL; - uint64_t child = 0; + uint32_t child = 0; uint64_t depth = 0; /* @@ -296,7 +319,8 @@ zfs_btree_find(zfs_btree_t *tree, const void *value, zfs_btree_index_t *where) */ zfs_btree_leaf_t *leaf = (depth == 0 ? (zfs_btree_leaf_t *)tree->bt_root : (zfs_btree_leaf_t *)node); - void *d = zfs_btree_find_in_buf(tree, leaf->btl_elems, + void *d = zfs_btree_find_in_buf(tree, leaf->btl_elems + + leaf->btl_hdr.bth_first * size, leaf->btl_hdr.bth_count, value, &idx); if (where != NULL) { @@ -366,24 +390,23 @@ enum bt_shift_direction { * shift is determined by shape. The direction is determined by dir. */ static inline void -bt_shift_core(zfs_btree_t *tree, zfs_btree_core_t *node, uint64_t idx, - uint64_t count, uint64_t off, enum bt_shift_shape shape, +bt_shift_core(zfs_btree_t *tree, zfs_btree_core_t *node, uint32_t idx, + uint32_t count, uint32_t off, enum bt_shift_shape shape, enum bt_shift_direction dir) { size_t size = tree->bt_elem_size; - ASSERT(node->btc_hdr.bth_core); + ASSERT(zfs_btree_is_core(&node->btc_hdr)); uint8_t *e_start = node->btc_elems + idx * size; - int sign = (dir == BSD_LEFT ? -1 : +1); - uint8_t *e_out = e_start + sign * off * size; - uint64_t e_count = count; - bmov(e_start, e_out, e_count * size); + uint8_t *e_out = (dir == BSD_LEFT ? e_start - off * size : + e_start + off * size); + bmov(e_start, e_out, count * size); zfs_btree_hdr_t **c_start = node->btc_children + idx + (shape == BSS_TRAPEZOID ? 0 : 1); zfs_btree_hdr_t **c_out = (dir == BSD_LEFT ? c_start - off : c_start + off); - uint64_t c_count = count + (shape == BSS_TRAPEZOID ? 1 : 0); + uint32_t c_count = count + (shape == BSS_TRAPEZOID ? 1 : 0); bmov(c_start, c_out, c_count * sizeof (*c_start)); } @@ -394,8 +417,8 @@ bt_shift_core(zfs_btree_t *tree, zfs_btree_core_t *node, uint64_t idx, * false if it is a parallelogram. */ static inline void -bt_shift_core_left(zfs_btree_t *tree, zfs_btree_core_t *node, uint64_t idx, - uint64_t count, enum bt_shift_shape shape) +bt_shift_core_left(zfs_btree_t *tree, zfs_btree_core_t *node, uint32_t idx, + uint32_t count, enum bt_shift_shape shape) { bt_shift_core(tree, node, idx, count, 1, shape, BSD_LEFT); } @@ -405,8 +428,8 @@ bt_shift_core_left(zfs_btree_t *tree, zfs_btree_core_t *node, uint64_t idx, * Starts with elements[idx] and children[idx] and one more child than element. */ static inline void -bt_shift_core_right(zfs_btree_t *tree, zfs_btree_core_t *node, uint64_t idx, - uint64_t count, enum bt_shift_shape shape) +bt_shift_core_right(zfs_btree_t *tree, zfs_btree_core_t *node, uint32_t idx, + uint32_t count, enum bt_shift_shape shape) { bt_shift_core(tree, node, idx, count, 1, shape, BSD_RIGHT); } @@ -417,30 +440,78 @@ bt_shift_core_right(zfs_btree_t *tree, zfs_btree_core_t *node, uint64_t idx, * is determined by left. */ static inline void -bt_shift_leaf(zfs_btree_t *tree, zfs_btree_leaf_t *node, uint64_t idx, - uint64_t count, uint64_t off, enum bt_shift_direction dir) +bt_shift_leaf(zfs_btree_t *tree, zfs_btree_leaf_t *node, uint32_t idx, + uint32_t count, uint32_t off, enum bt_shift_direction dir) { size_t size = tree->bt_elem_size; - ASSERT(!node->btl_hdr.bth_core); + zfs_btree_hdr_t *hdr = &node->btl_hdr; + ASSERT(!zfs_btree_is_core(hdr)); - uint8_t *start = node->btl_elems + idx * size; - int sign = (dir == BSD_LEFT ? -1 : +1); - uint8_t *out = start + sign * off * size; + if (count == 0) + return; + uint8_t *start = node->btl_elems + (hdr->bth_first + idx) * size; + uint8_t *out = (dir == BSD_LEFT ? start - off * size : + start + off * size); bmov(start, out, count * size); } -static inline void -bt_shift_leaf_right(zfs_btree_t *tree, zfs_btree_leaf_t *leaf, uint64_t idx, - uint64_t count) +/* + * Grow leaf for n new elements before idx. + */ +static void +bt_grow_leaf(zfs_btree_t *tree, zfs_btree_leaf_t *leaf, uint32_t idx, + uint32_t n) { - bt_shift_leaf(tree, leaf, idx, count, 1, BSD_RIGHT); + zfs_btree_hdr_t *hdr = &leaf->btl_hdr; + ASSERT(!zfs_btree_is_core(hdr)); + ASSERT3U(idx, <=, hdr->bth_count); + uint32_t capacity = tree->bt_leaf_cap; + ASSERT3U(hdr->bth_count + n, <=, capacity); + boolean_t cl = (hdr->bth_first >= n); + boolean_t cr = (hdr->bth_first + hdr->bth_count + n <= capacity); + + if (cl && (!cr || idx <= hdr->bth_count / 2)) { + /* Grow left. */ + hdr->bth_first -= n; + bt_shift_leaf(tree, leaf, n, idx, n, BSD_LEFT); + } else if (cr) { + /* Grow right. */ + bt_shift_leaf(tree, leaf, idx, hdr->bth_count - idx, n, + BSD_RIGHT); + } else { + /* Grow both ways. */ + uint32_t fn = hdr->bth_first - + (capacity - (hdr->bth_count + n)) / 2; + hdr->bth_first -= fn; + bt_shift_leaf(tree, leaf, fn, idx, fn, BSD_LEFT); + bt_shift_leaf(tree, leaf, fn + idx, hdr->bth_count - idx, + n - fn, BSD_RIGHT); + } + hdr->bth_count += n; } -static inline void -bt_shift_leaf_left(zfs_btree_t *tree, zfs_btree_leaf_t *leaf, uint64_t idx, - uint64_t count) +/* + * Shrink leaf for count elements starting from idx. + */ +static void +bt_shrink_leaf(zfs_btree_t *tree, zfs_btree_leaf_t *leaf, uint32_t idx, + uint32_t n) { - bt_shift_leaf(tree, leaf, idx, count, 1, BSD_LEFT); + zfs_btree_hdr_t *hdr = &leaf->btl_hdr; + ASSERT(!zfs_btree_is_core(hdr)); + ASSERT3U(idx, <=, hdr->bth_count); + ASSERT3U(idx + n, <=, hdr->bth_count); + + if (idx <= (hdr->bth_count - n) / 2) { + bt_shift_leaf(tree, leaf, 0, idx, n, BSD_RIGHT); + zfs_btree_poison_node_at(tree, hdr, 0, n); + hdr->bth_first += n; + } else { + bt_shift_leaf(tree, leaf, idx + n, hdr->bth_count - idx - n, n, + BSD_LEFT); + zfs_btree_poison_node_at(tree, hdr, hdr->bth_count - n, n); + } + hdr->bth_count -= n; } /* @@ -448,32 +519,33 @@ bt_shift_leaf_left(zfs_btree_t *tree, zfs_btree_leaf_t *leaf, uint64_t idx, * parameter behaves the same as it does in the shift logic. */ static inline void -bt_transfer_core(zfs_btree_t *tree, zfs_btree_core_t *source, uint64_t sidx, - uint64_t count, zfs_btree_core_t *dest, uint64_t didx, +bt_transfer_core(zfs_btree_t *tree, zfs_btree_core_t *source, uint32_t sidx, + uint32_t count, zfs_btree_core_t *dest, uint32_t didx, enum bt_shift_shape shape) { size_t size = tree->bt_elem_size; - ASSERT(source->btc_hdr.bth_core); - ASSERT(dest->btc_hdr.bth_core); + ASSERT(zfs_btree_is_core(&source->btc_hdr)); + ASSERT(zfs_btree_is_core(&dest->btc_hdr)); - bmov(source->btc_elems + sidx * size, dest->btc_elems + didx * size, + bcpy(source->btc_elems + sidx * size, dest->btc_elems + didx * size, count * size); - uint64_t c_count = count + (shape == BSS_TRAPEZOID ? 1 : 0); - bmov(source->btc_children + sidx + (shape == BSS_TRAPEZOID ? 0 : 1), + uint32_t c_count = count + (shape == BSS_TRAPEZOID ? 1 : 0); + bcpy(source->btc_children + sidx + (shape == BSS_TRAPEZOID ? 0 : 1), dest->btc_children + didx + (shape == BSS_TRAPEZOID ? 0 : 1), c_count * sizeof (*source->btc_children)); } static inline void -bt_transfer_leaf(zfs_btree_t *tree, zfs_btree_leaf_t *source, uint64_t sidx, - uint64_t count, zfs_btree_leaf_t *dest, uint64_t didx) +bt_transfer_leaf(zfs_btree_t *tree, zfs_btree_leaf_t *source, uint32_t sidx, + uint32_t count, zfs_btree_leaf_t *dest, uint32_t didx) { size_t size = tree->bt_elem_size; - ASSERT(!source->btl_hdr.bth_core); - ASSERT(!dest->btl_hdr.bth_core); + ASSERT(!zfs_btree_is_core(&source->btl_hdr)); + ASSERT(!zfs_btree_is_core(&dest->btl_hdr)); - bmov(source->btl_elems + sidx * size, dest->btl_elems + didx * size, + bcpy(source->btl_elems + (source->btl_hdr.bth_first + sidx) * size, + dest->btl_elems + (dest->btl_hdr.bth_first + didx) * size, count * size); } @@ -482,30 +554,31 @@ bt_transfer_leaf(zfs_btree_t *tree, zfs_btree_leaf_t *source, uint64_t sidx, * put its location in where if non-null. */ static void * -zfs_btree_first_helper(zfs_btree_hdr_t *hdr, zfs_btree_index_t *where) +zfs_btree_first_helper(zfs_btree_t *tree, zfs_btree_hdr_t *hdr, + zfs_btree_index_t *where) { zfs_btree_hdr_t *node; - for (node = hdr; node->bth_core; node = - ((zfs_btree_core_t *)node)->btc_children[0]) + for (node = hdr; zfs_btree_is_core(node); + node = ((zfs_btree_core_t *)node)->btc_children[0]) ; - ASSERT(!node->bth_core); + ASSERT(!zfs_btree_is_core(node)); zfs_btree_leaf_t *leaf = (zfs_btree_leaf_t *)node; if (where != NULL) { where->bti_node = node; where->bti_offset = 0; where->bti_before = B_FALSE; } - return (&leaf->btl_elems[0]); + return (&leaf->btl_elems[node->bth_first * tree->bt_elem_size]); } /* Insert an element and a child into a core node at the given offset. */ static void zfs_btree_insert_core_impl(zfs_btree_t *tree, zfs_btree_core_t *parent, - uint64_t offset, zfs_btree_hdr_t *new_node, void *buf) + uint32_t offset, zfs_btree_hdr_t *new_node, void *buf) { - uint64_t size = tree->bt_elem_size; + size_t size = tree->bt_elem_size; zfs_btree_hdr_t *par_hdr = &parent->btc_hdr; ASSERT3P(par_hdr, ==, new_node->bth_parent); ASSERT3U(par_hdr->bth_count, <, BTREE_CORE_ELEMS); @@ -515,13 +588,13 @@ zfs_btree_insert_core_impl(zfs_btree_t *tree, zfs_btree_core_t *parent, par_hdr->bth_count); } /* Shift existing elements and children */ - uint64_t count = par_hdr->bth_count - offset; + uint32_t count = par_hdr->bth_count - offset; bt_shift_core_right(tree, parent, offset, count, BSS_PARALLELOGRAM); /* Insert new values */ parent->btc_children[offset + 1] = new_node; - bmov(buf, parent->btc_elems + offset * size, size); + bcpy(buf, parent->btc_elems + offset * size, size); par_hdr->bth_count++; } @@ -534,7 +607,7 @@ zfs_btree_insert_into_parent(zfs_btree_t *tree, zfs_btree_hdr_t *old_node, zfs_btree_hdr_t *new_node, void *buf) { ASSERT3P(old_node->bth_parent, ==, new_node->bth_parent); - uint64_t size = tree->bt_elem_size; + size_t size = tree->bt_elem_size; zfs_btree_core_t *parent = old_node->bth_parent; /* @@ -549,13 +622,13 @@ zfs_btree_insert_into_parent(zfs_btree_t *tree, zfs_btree_hdr_t *old_node, size, KM_SLEEP); zfs_btree_hdr_t *new_root_hdr = &new_root->btc_hdr; new_root_hdr->bth_parent = NULL; - new_root_hdr->bth_core = B_TRUE; + new_root_hdr->bth_first = -1; new_root_hdr->bth_count = 1; old_node->bth_parent = new_node->bth_parent = new_root; new_root->btc_children[0] = old_node; new_root->btc_children[1] = new_node; - bmov(buf, new_root->btc_elems, size); + bcpy(buf, new_root->btc_elems, size); tree->bt_height++; tree->bt_root = new_root_hdr; @@ -569,11 +642,11 @@ zfs_btree_insert_into_parent(zfs_btree_t *tree, zfs_btree_hdr_t *old_node, */ zfs_btree_hdr_t *par_hdr = &parent->btc_hdr; zfs_btree_index_t idx; - ASSERT(par_hdr->bth_core); + ASSERT(zfs_btree_is_core(par_hdr)); VERIFY3P(zfs_btree_find_in_buf(tree, parent->btc_elems, par_hdr->bth_count, buf, &idx), ==, NULL); ASSERT(idx.bti_before); - uint64_t offset = idx.bti_offset; + uint32_t offset = idx.bti_offset; ASSERT3U(offset, <=, par_hdr->bth_count); ASSERT3P(parent->btc_children[offset], ==, old_node); @@ -604,16 +677,16 @@ zfs_btree_insert_into_parent(zfs_btree_t *tree, zfs_btree_hdr_t *old_node, * We do this in two stages: first we split into two nodes, and then we * reuse our existing logic to insert the new element and child. */ - uint64_t move_count = MAX((BTREE_CORE_ELEMS / (tree->bt_bulk == NULL ? + uint32_t move_count = MAX((BTREE_CORE_ELEMS / (tree->bt_bulk == NULL ? 2 : 4)) - 1, 2); - uint64_t keep_count = BTREE_CORE_ELEMS - move_count - 1; + uint32_t keep_count = BTREE_CORE_ELEMS - move_count - 1; ASSERT3U(BTREE_CORE_ELEMS - move_count, >=, 2); tree->bt_num_nodes++; zfs_btree_core_t *new_parent = kmem_alloc(sizeof (zfs_btree_core_t) + BTREE_CORE_ELEMS * size, KM_SLEEP); zfs_btree_hdr_t *new_par_hdr = &new_parent->btc_hdr; new_par_hdr->bth_parent = par_hdr->bth_parent; - new_par_hdr->bth_core = B_TRUE; + new_par_hdr->bth_first = -1; new_par_hdr->bth_count = move_count; zfs_btree_poison_node(tree, new_par_hdr); @@ -624,7 +697,7 @@ zfs_btree_insert_into_parent(zfs_btree_t *tree, zfs_btree_hdr_t *old_node, /* Store the new separator in a buffer. */ uint8_t *tmp_buf = kmem_alloc(size, KM_SLEEP); - bmov(parent->btc_elems + keep_count * size, tmp_buf, + bcpy(parent->btc_elems + keep_count * size, tmp_buf, size); zfs_btree_poison_node(tree, par_hdr); @@ -636,7 +709,7 @@ zfs_btree_insert_into_parent(zfs_btree_t *tree, zfs_btree_hdr_t *old_node, /* * Move the new separator to the existing buffer. */ - bmov(tmp_buf, buf, size); + bcpy(tmp_buf, buf, size); } else if (offset > keep_count) { /* Insert the new node into the right half */ new_node->bth_parent = new_parent; @@ -646,7 +719,7 @@ zfs_btree_insert_into_parent(zfs_btree_t *tree, zfs_btree_hdr_t *old_node, /* * Move the new separator to the existing buffer. */ - bmov(tmp_buf, buf, size); + bcpy(tmp_buf, buf, size); } else { /* * Move the new separator into the right half, and replace it @@ -656,16 +729,16 @@ zfs_btree_insert_into_parent(zfs_btree_t *tree, zfs_btree_hdr_t *old_node, bt_shift_core_right(tree, new_parent, 0, move_count, BSS_TRAPEZOID); new_parent->btc_children[0] = new_node; - bmov(tmp_buf, new_parent->btc_elems, size); + bcpy(tmp_buf, new_parent->btc_elems, size); new_par_hdr->bth_count++; } kmem_free(tmp_buf, size); zfs_btree_poison_node(tree, par_hdr); - for (int i = 0; i <= new_parent->btc_hdr.bth_count; i++) + for (uint32_t i = 0; i <= new_parent->btc_hdr.bth_count; i++) new_parent->btc_children[i]->bth_parent = new_parent; - for (int i = 0; i <= parent->btc_hdr.bth_count; i++) + for (uint32_t i = 0; i <= parent->btc_hdr.bth_count; i++) ASSERT3P(parent->btc_children[i]->bth_parent, ==, parent); /* @@ -679,34 +752,32 @@ zfs_btree_insert_into_parent(zfs_btree_t *tree, zfs_btree_hdr_t *old_node, /* Insert an element into a leaf node at the given offset. */ static void zfs_btree_insert_leaf_impl(zfs_btree_t *tree, zfs_btree_leaf_t *leaf, - uint64_t idx, const void *value) + uint32_t idx, const void *value) { - uint64_t size = tree->bt_elem_size; - uint8_t *start = leaf->btl_elems + (idx * size); + size_t size = tree->bt_elem_size; zfs_btree_hdr_t *hdr = &leaf->btl_hdr; - uint64_t capacity __maybe_unused = P2ALIGN((BTREE_LEAF_SIZE - - sizeof (zfs_btree_hdr_t)) / size, 2); - uint64_t count = leaf->btl_hdr.bth_count - idx; - ASSERT3U(leaf->btl_hdr.bth_count, <, capacity); + ASSERT3U(leaf->btl_hdr.bth_count, <, tree->bt_leaf_cap); if (zfs_btree_verify_intensity >= 5) { zfs_btree_verify_poison_at(tree, &leaf->btl_hdr, leaf->btl_hdr.bth_count); } - bt_shift_leaf_right(tree, leaf, idx, count); - bmov(value, start, size); - hdr->bth_count++; + bt_grow_leaf(tree, leaf, idx, 1); + uint8_t *start = leaf->btl_elems + (hdr->bth_first + idx) * size; + bcpy(value, start, size); } +static void +zfs_btree_verify_order_helper(zfs_btree_t *tree, zfs_btree_hdr_t *hdr); + /* Helper function for inserting a new value into leaf at the given index. */ static void zfs_btree_insert_into_leaf(zfs_btree_t *tree, zfs_btree_leaf_t *leaf, - const void *value, uint64_t idx) + const void *value, uint32_t idx) { - uint64_t size = tree->bt_elem_size; - uint64_t capacity = P2ALIGN((BTREE_LEAF_SIZE - - sizeof (zfs_btree_hdr_t)) / size, 2); + size_t size = tree->bt_elem_size; + uint32_t capacity = tree->bt_leaf_cap; /* * If the leaf isn't full, shift the elements after idx and insert @@ -731,32 +802,36 @@ zfs_btree_insert_into_leaf(zfs_btree_t *tree, zfs_btree_leaf_t *leaf, * In either case, we're left with one extra element. The leftover * element will become the new dividing element between the two nodes. */ - uint64_t move_count = MAX(capacity / (tree->bt_bulk == NULL ? 2 : 4) - - 1, 2); - uint64_t keep_count = capacity - move_count - 1; - ASSERT3U(capacity - move_count, >=, 2); + uint32_t move_count = MAX(capacity / (tree->bt_bulk ? 4 : 2), 1) - 1; + uint32_t keep_count = capacity - move_count - 1; + ASSERT3U(keep_count, >=, 1); + /* If we insert on left. move one more to keep leaves balanced. */ + if (idx < keep_count) { + keep_count--; + move_count++; + } tree->bt_num_nodes++; zfs_btree_leaf_t *new_leaf = kmem_cache_alloc(zfs_btree_leaf_cache, KM_SLEEP); zfs_btree_hdr_t *new_hdr = &new_leaf->btl_hdr; new_hdr->bth_parent = leaf->btl_hdr.bth_parent; - new_hdr->bth_core = B_FALSE; + new_hdr->bth_first = (tree->bt_bulk ? 0 : capacity / 4) + + (idx >= keep_count && idx <= keep_count + move_count / 2); new_hdr->bth_count = move_count; zfs_btree_poison_node(tree, new_hdr); - leaf->btl_hdr.bth_count = keep_count; - if (tree->bt_bulk != NULL && leaf == tree->bt_bulk) tree->bt_bulk = new_leaf; /* Copy the back part to the new leaf. */ - bt_transfer_leaf(tree, leaf, keep_count + 1, move_count, new_leaf, - 0); + bt_transfer_leaf(tree, leaf, keep_count + 1, move_count, new_leaf, 0); /* We store the new separator in a buffer we control for simplicity. */ uint8_t *buf = kmem_alloc(size, KM_SLEEP); - bmov(leaf->btl_elems + (keep_count * size), buf, size); - zfs_btree_poison_node(tree, &leaf->btl_hdr); + bcpy(leaf->btl_elems + (leaf->btl_hdr.bth_first + keep_count) * size, + buf, size); + + bt_shrink_leaf(tree, leaf, keep_count, 1 + move_count); if (idx < keep_count) { /* Insert into the existing leaf. */ @@ -767,13 +842,11 @@ zfs_btree_insert_into_leaf(zfs_btree_t *tree, zfs_btree_leaf_t *leaf, 1, value); } else { /* - * Shift the elements in the new leaf to make room for the - * separator, and use the new value as the new separator. + * Insert planned separator into the new leaf, and use + * the new value as the new separator. */ - bt_shift_leaf_right(tree, new_leaf, 0, move_count); - bmov(buf, new_leaf->btl_elems, size); - bmov(value, buf, size); - new_hdr->bth_count++; + zfs_btree_insert_leaf_impl(tree, new_leaf, 0, buf); + bcpy(value, buf, size); } /* @@ -785,14 +858,15 @@ zfs_btree_insert_into_leaf(zfs_btree_t *tree, zfs_btree_leaf_t *leaf, kmem_free(buf, size); } -static uint64_t +static uint32_t zfs_btree_find_parent_idx(zfs_btree_t *tree, zfs_btree_hdr_t *hdr) { void *buf; - if (hdr->bth_core) { + if (zfs_btree_is_core(hdr)) { buf = ((zfs_btree_core_t *)hdr)->btc_elems; } else { - buf = ((zfs_btree_leaf_t *)hdr)->btl_elems; + buf = ((zfs_btree_leaf_t *)hdr)->btl_elems + + hdr->bth_first * tree->bt_elem_size; } zfs_btree_index_t idx; zfs_btree_core_t *parent = hdr->bth_parent; @@ -821,9 +895,8 @@ zfs_btree_bulk_finish(zfs_btree_t *tree) zfs_btree_leaf_t *leaf = tree->bt_bulk; zfs_btree_hdr_t *hdr = &leaf->btl_hdr; zfs_btree_core_t *parent = hdr->bth_parent; - uint64_t size = tree->bt_elem_size; - uint64_t capacity = P2ALIGN((BTREE_LEAF_SIZE - - sizeof (zfs_btree_hdr_t)) / size, 2); + size_t size = tree->bt_elem_size; + uint32_t capacity = tree->bt_leaf_cap; /* * The invariant doesn't apply to the root node, if that's the only @@ -848,56 +921,54 @@ zfs_btree_bulk_finish(zfs_btree_t *tree) .bti_offset = 0 }; VERIFY3P(zfs_btree_prev(tree, &idx, &idx), !=, NULL); - ASSERT(idx.bti_node->bth_core); + ASSERT(zfs_btree_is_core(idx.bti_node)); zfs_btree_core_t *common = (zfs_btree_core_t *)idx.bti_node; - uint64_t common_idx = idx.bti_offset; + uint32_t common_idx = idx.bti_offset; VERIFY3P(zfs_btree_prev(tree, &idx, &idx), !=, NULL); - ASSERT(!idx.bti_node->bth_core); + ASSERT(!zfs_btree_is_core(idx.bti_node)); zfs_btree_leaf_t *l_neighbor = (zfs_btree_leaf_t *)idx.bti_node; zfs_btree_hdr_t *l_hdr = idx.bti_node; - uint64_t move_count = (capacity / 2) - hdr->bth_count; + uint32_t move_count = (capacity / 2) - hdr->bth_count; ASSERT3U(l_neighbor->btl_hdr.bth_count - move_count, >=, capacity / 2); if (zfs_btree_verify_intensity >= 5) { - for (int i = 0; i < move_count; i++) { + for (uint32_t i = 0; i < move_count; i++) { zfs_btree_verify_poison_at(tree, hdr, leaf->btl_hdr.bth_count + i); } } /* First, shift elements in leaf back. */ - bt_shift_leaf(tree, leaf, 0, hdr->bth_count, move_count, - BSD_RIGHT); + bt_grow_leaf(tree, leaf, 0, move_count); /* Next, move the separator from the common ancestor to leaf. */ - uint8_t *separator = common->btc_elems + (common_idx * size); - uint8_t *out = leaf->btl_elems + ((move_count - 1) * size); - bmov(separator, out, size); - move_count--; + uint8_t *separator = common->btc_elems + common_idx * size; + uint8_t *out = leaf->btl_elems + + (hdr->bth_first + move_count - 1) * size; + bcpy(separator, out, size); /* * Now we move elements from the tail of the left neighbor to * fill the remaining spots in leaf. */ bt_transfer_leaf(tree, l_neighbor, l_hdr->bth_count - - move_count, move_count, leaf, 0); + (move_count - 1), move_count - 1, leaf, 0); /* * Finally, move the new last element in the left neighbor to * the separator. */ - bmov(l_neighbor->btl_elems + (l_hdr->bth_count - - move_count - 1) * size, separator, size); + bcpy(l_neighbor->btl_elems + (l_hdr->bth_first + + l_hdr->bth_count - move_count) * size, separator, size); /* Adjust the node's counts, and we're done. */ - l_hdr->bth_count -= move_count + 1; - hdr->bth_count += move_count + 1; + bt_shrink_leaf(tree, l_neighbor, l_hdr->bth_count - move_count, + move_count); ASSERT3U(l_hdr->bth_count, >=, capacity / 2); ASSERT3U(hdr->bth_count, >=, capacity / 2); - zfs_btree_poison_node(tree, l_hdr); } /* @@ -921,16 +992,16 @@ zfs_btree_bulk_finish(zfs_btree_t *tree) * splitting is 2, we never need to worry about not having a * left sibling (a sibling is a neighbor with the same parent). */ - uint64_t parent_idx = zfs_btree_find_parent_idx(tree, hdr); + uint32_t parent_idx = zfs_btree_find_parent_idx(tree, hdr); ASSERT3U(parent_idx, >, 0); zfs_btree_core_t *l_neighbor = (zfs_btree_core_t *)parent->btc_children[parent_idx - 1]; - uint64_t move_count = (capacity / 2) - hdr->bth_count; + uint32_t move_count = (capacity / 2) - hdr->bth_count; ASSERT3U(l_neighbor->btc_hdr.bth_count - move_count, >=, capacity / 2); if (zfs_btree_verify_intensity >= 5) { - for (int i = 0; i < move_count; i++) { + for (uint32_t i = 0; i < move_count; i++) { zfs_btree_verify_poison_at(tree, hdr, hdr->bth_count + i); } @@ -943,14 +1014,14 @@ zfs_btree_bulk_finish(zfs_btree_t *tree) uint8_t *separator = parent->btc_elems + ((parent_idx - 1) * size); uint8_t *e_out = cur->btc_elems + ((move_count - 1) * size); - bmov(separator, e_out, size); + bcpy(separator, e_out, size); /* * Now, move elements and children from the left node to the * right. We move one more child than elements. */ move_count--; - uint64_t move_idx = l_neighbor->btc_hdr.bth_count - move_count; + uint32_t move_idx = l_neighbor->btc_hdr.bth_count - move_count; bt_transfer_core(tree, l_neighbor, move_idx, move_count, cur, 0, BSS_TRAPEZOID); @@ -959,7 +1030,7 @@ zfs_btree_bulk_finish(zfs_btree_t *tree) * separator's position. */ move_idx--; - bmov(l_neighbor->btc_elems + move_idx * size, separator, size); + bcpy(l_neighbor->btc_elems + move_idx * size, separator, size); l_neighbor->btc_hdr.bth_count -= move_count + 1; hdr->bth_count += move_count + 1; @@ -969,11 +1040,12 @@ zfs_btree_bulk_finish(zfs_btree_t *tree) zfs_btree_poison_node(tree, &l_neighbor->btc_hdr); - for (int i = 0; i <= hdr->bth_count; i++) + for (uint32_t i = 0; i <= hdr->bth_count; i++) cur->btc_children[i]->bth_parent = cur; } tree->bt_bulk = NULL; + zfs_btree_verify(tree); } /* @@ -1013,13 +1085,13 @@ zfs_btree_add_idx(zfs_btree_t *tree, const void *value, zfs_btree_hdr_t *hdr = &leaf->btl_hdr; hdr->bth_parent = NULL; - hdr->bth_core = B_FALSE; + hdr->bth_first = 0; hdr->bth_count = 0; zfs_btree_poison_node(tree, hdr); zfs_btree_insert_into_leaf(tree, leaf, value, 0); tree->bt_bulk = leaf; - } else if (!where->bti_node->bth_core) { + } else if (!zfs_btree_is_core(where->bti_node)) { /* * If we're inserting into a leaf, go directly to the helper * function. @@ -1035,28 +1107,28 @@ zfs_btree_add_idx(zfs_btree_t *tree, const void *value, * value in the node at that spot and then insert the old * separator into the first slot in the subtree to the right. */ - ASSERT(where->bti_node->bth_core); zfs_btree_core_t *node = (zfs_btree_core_t *)where->bti_node; /* * We can ignore bti_before, because either way the value * should end up in bti_offset. */ - uint64_t off = where->bti_offset; + uint32_t off = where->bti_offset; zfs_btree_hdr_t *subtree = node->btc_children[off + 1]; size_t size = tree->bt_elem_size; uint8_t *buf = kmem_alloc(size, KM_SLEEP); - bmov(node->btc_elems + off * size, buf, size); - bmov(value, node->btc_elems + off * size, size); + bcpy(node->btc_elems + off * size, buf, size); + bcpy(value, node->btc_elems + off * size, size); /* * Find the first slot in the subtree to the right, insert * there. */ zfs_btree_index_t new_idx; - VERIFY3P(zfs_btree_first_helper(subtree, &new_idx), !=, NULL); + VERIFY3P(zfs_btree_first_helper(tree, subtree, &new_idx), !=, + NULL); ASSERT0(new_idx.bti_offset); - ASSERT(!new_idx.bti_node->bth_core); + ASSERT(!zfs_btree_is_core(new_idx.bti_node)); zfs_btree_insert_into_leaf(tree, (zfs_btree_leaf_t *)new_idx.bti_node, buf, 0); kmem_free(buf, size); @@ -1075,7 +1147,7 @@ zfs_btree_first(zfs_btree_t *tree, zfs_btree_index_t *where) ASSERT0(tree->bt_num_elems); return (NULL); } - return (zfs_btree_first_helper(tree->bt_root, where)); + return (zfs_btree_first_helper(tree, tree->bt_root, where)); } /* @@ -1088,7 +1160,7 @@ zfs_btree_last_helper(zfs_btree_t *btree, zfs_btree_hdr_t *hdr, { zfs_btree_hdr_t *node; - for (node = hdr; node->bth_core; node = + for (node = hdr; zfs_btree_is_core(node); node = ((zfs_btree_core_t *)node)->btc_children[node->bth_count]) ; @@ -1098,7 +1170,8 @@ zfs_btree_last_helper(zfs_btree_t *btree, zfs_btree_hdr_t *hdr, where->bti_offset = node->bth_count - 1; where->bti_before = B_FALSE; } - return (leaf->btl_elems + (node->bth_count - 1) * btree->bt_elem_size); + return (leaf->btl_elems + (node->bth_first + node->bth_count - 1) * + btree->bt_elem_size); } /* @@ -1131,8 +1204,8 @@ zfs_btree_next_helper(zfs_btree_t *tree, const zfs_btree_index_t *idx, return (NULL); } - uint64_t offset = idx->bti_offset; - if (!idx->bti_node->bth_core) { + uint32_t offset = idx->bti_offset; + if (!zfs_btree_is_core(idx->bti_node)) { /* * When finding the next element of an element in a leaf, * there are two cases. If the element isn't the last one in @@ -1143,20 +1216,21 @@ zfs_btree_next_helper(zfs_btree_t *tree, const zfs_btree_index_t *idx, * separator after our ancestor in its parent. */ zfs_btree_leaf_t *leaf = (zfs_btree_leaf_t *)idx->bti_node; - uint64_t new_off = offset + (idx->bti_before ? 0 : 1); + uint32_t new_off = offset + (idx->bti_before ? 0 : 1); if (leaf->btl_hdr.bth_count > new_off) { out_idx->bti_node = &leaf->btl_hdr; out_idx->bti_offset = new_off; out_idx->bti_before = B_FALSE; - return (leaf->btl_elems + new_off * tree->bt_elem_size); + return (leaf->btl_elems + (leaf->btl_hdr.bth_first + + new_off) * tree->bt_elem_size); } zfs_btree_hdr_t *prev = &leaf->btl_hdr; for (zfs_btree_core_t *node = leaf->btl_hdr.bth_parent; node != NULL; node = node->btc_hdr.bth_parent) { zfs_btree_hdr_t *hdr = &node->btc_hdr; - ASSERT(hdr->bth_core); - uint64_t i = zfs_btree_find_parent_idx(tree, prev); + ASSERT(zfs_btree_is_core(hdr)); + uint32_t i = zfs_btree_find_parent_idx(tree, prev); if (done_func != NULL) done_func(tree, prev); if (i == hdr->bth_count) { @@ -1178,7 +1252,7 @@ zfs_btree_next_helper(zfs_btree_t *tree, const zfs_btree_index_t *idx, } /* If we were before an element in a core node, return that element. */ - ASSERT(idx->bti_node->bth_core); + ASSERT(zfs_btree_is_core(idx->bti_node)); zfs_btree_core_t *node = (zfs_btree_core_t *)idx->bti_node; if (idx->bti_before) { out_idx->bti_before = B_FALSE; @@ -1190,7 +1264,7 @@ zfs_btree_next_helper(zfs_btree_t *tree, const zfs_btree_index_t *idx, * the subtree just to the right of the separator. */ zfs_btree_hdr_t *child = node->btc_children[offset + 1]; - return (zfs_btree_first_helper(child, out_idx)); + return (zfs_btree_first_helper(tree, child, out_idx)); } /* @@ -1217,8 +1291,8 @@ zfs_btree_prev(zfs_btree_t *tree, const zfs_btree_index_t *idx, return (NULL); } - uint64_t offset = idx->bti_offset; - if (!idx->bti_node->bth_core) { + uint32_t offset = idx->bti_offset; + if (!zfs_btree_is_core(idx->bti_node)) { /* * When finding the previous element of an element in a leaf, * there are two cases. If the element isn't the first one in @@ -1233,15 +1307,15 @@ zfs_btree_prev(zfs_btree_t *tree, const zfs_btree_index_t *idx, out_idx->bti_node = &leaf->btl_hdr; out_idx->bti_offset = offset - 1; out_idx->bti_before = B_FALSE; - return (leaf->btl_elems + (offset - 1) * - tree->bt_elem_size); + return (leaf->btl_elems + (leaf->btl_hdr.bth_first + + offset - 1) * tree->bt_elem_size); } zfs_btree_hdr_t *prev = &leaf->btl_hdr; for (zfs_btree_core_t *node = leaf->btl_hdr.bth_parent; node != NULL; node = node->btc_hdr.bth_parent) { zfs_btree_hdr_t *hdr = &node->btc_hdr; - ASSERT(hdr->bth_core); - uint64_t i = zfs_btree_find_parent_idx(tree, prev); + ASSERT(zfs_btree_is_core(hdr)); + uint32_t i = zfs_btree_find_parent_idx(tree, prev); if (i == 0) { prev = hdr; continue; @@ -1262,7 +1336,7 @@ zfs_btree_prev(zfs_btree_t *tree, const zfs_btree_index_t *idx, * The previous element from one in a core node is the last element in * the subtree just to the left of the separator. */ - ASSERT(idx->bti_node->bth_core); + ASSERT(zfs_btree_is_core(idx->bti_node)); zfs_btree_core_t *node = (zfs_btree_core_t *)idx->bti_node; zfs_btree_hdr_t *child = node->btc_children[offset]; return (zfs_btree_last_helper(tree, child, out_idx)); @@ -1279,13 +1353,14 @@ void * zfs_btree_get(zfs_btree_t *tree, zfs_btree_index_t *idx) { ASSERT(!idx->bti_before); - if (!idx->bti_node->bth_core) { + size_t size = tree->bt_elem_size; + if (!zfs_btree_is_core(idx->bti_node)) { zfs_btree_leaf_t *leaf = (zfs_btree_leaf_t *)idx->bti_node; - return (leaf->btl_elems + idx->bti_offset * tree->bt_elem_size); + return (leaf->btl_elems + (leaf->btl_hdr.bth_first + + idx->bti_offset) * size); } - ASSERT(idx->bti_node->bth_core); zfs_btree_core_t *node = (zfs_btree_core_t *)idx->bti_node; - return (node->btc_elems + idx->bti_offset * tree->bt_elem_size); + return (node->btc_elems + idx->bti_offset * size); } /* Add the given value to the tree. Must not already be in the tree. */ @@ -1302,7 +1377,7 @@ static void zfs_btree_node_destroy(zfs_btree_t *tree, zfs_btree_hdr_t *node) { tree->bt_num_nodes--; - if (!node->bth_core) { + if (!zfs_btree_is_core(node)) { kmem_cache_free(zfs_btree_leaf_cache, node); } else { kmem_free(node, sizeof (zfs_btree_core_t) + @@ -1320,7 +1395,7 @@ zfs_btree_remove_from_node(zfs_btree_t *tree, zfs_btree_core_t *node, zfs_btree_hdr_t *rm_hdr) { size_t size = tree->bt_elem_size; - uint64_t min_count = (BTREE_CORE_ELEMS / 2) - 1; + uint32_t min_count = (BTREE_CORE_ELEMS / 2) - 1; zfs_btree_hdr_t *hdr = &node->btc_hdr; /* * If the node is the root node and rm_hdr is one of two children, @@ -1337,7 +1412,7 @@ zfs_btree_remove_from_node(zfs_btree_t *tree, zfs_btree_core_t *node, return; } - uint64_t idx; + uint32_t idx; for (idx = 0; idx <= hdr->bth_count; idx++) { if (node->btc_children[idx] == rm_hdr) break; @@ -1357,7 +1432,7 @@ zfs_btree_remove_from_node(zfs_btree_t *tree, zfs_btree_core_t *node, bt_shift_core_left(tree, node, idx, hdr->bth_count - idx, BSS_PARALLELOGRAM); hdr->bth_count--; - zfs_btree_poison_node_at(tree, hdr, hdr->bth_count); + zfs_btree_poison_node_at(tree, hdr, hdr->bth_count, 1); return; } @@ -1378,13 +1453,13 @@ zfs_btree_remove_from_node(zfs_btree_t *tree, zfs_btree_core_t *node, * implementing in the future for completeness' sake. */ zfs_btree_core_t *parent = hdr->bth_parent; - uint64_t parent_idx = zfs_btree_find_parent_idx(tree, hdr); + uint32_t parent_idx = zfs_btree_find_parent_idx(tree, hdr); zfs_btree_hdr_t *l_hdr = (parent_idx == 0 ? NULL : parent->btc_children[parent_idx - 1]); if (l_hdr != NULL && l_hdr->bth_count > min_count) { /* We can take a node from the left neighbor. */ - ASSERT(l_hdr->bth_core); + ASSERT(zfs_btree_is_core(l_hdr)); zfs_btree_core_t *neighbor = (zfs_btree_core_t *)l_hdr; /* @@ -1399,20 +1474,19 @@ zfs_btree_remove_from_node(zfs_btree_t *tree, zfs_btree_core_t *node, */ uint8_t *separator = parent->btc_elems + (parent_idx - 1) * size; - bmov(separator, node->btc_elems, size); + bcpy(separator, node->btc_elems, size); /* Move the last child of neighbor to our first child slot. */ - zfs_btree_hdr_t **take_child = neighbor->btc_children + - l_hdr->bth_count; - bmov(take_child, node->btc_children, sizeof (*take_child)); + node->btc_children[0] = + neighbor->btc_children[l_hdr->bth_count]; node->btc_children[0]->bth_parent = node; /* Move the last element of neighbor to the separator spot. */ uint8_t *take_elem = neighbor->btc_elems + (l_hdr->bth_count - 1) * size; - bmov(take_elem, separator, size); + bcpy(take_elem, separator, size); l_hdr->bth_count--; - zfs_btree_poison_node_at(tree, l_hdr, l_hdr->bth_count); + zfs_btree_poison_node_at(tree, l_hdr, l_hdr->bth_count, 1); return; } @@ -1420,7 +1494,7 @@ zfs_btree_remove_from_node(zfs_btree_t *tree, zfs_btree_core_t *node, NULL : parent->btc_children[parent_idx + 1]); if (r_hdr != NULL && r_hdr->bth_count > min_count) { /* We can take a node from the right neighbor. */ - ASSERT(r_hdr->bth_core); + ASSERT(zfs_btree_is_core(r_hdr)); zfs_btree_core_t *neighbor = (zfs_btree_core_t *)r_hdr; /* @@ -1435,21 +1509,19 @@ zfs_btree_remove_from_node(zfs_btree_t *tree, zfs_btree_core_t *node, * element spot in node. */ uint8_t *separator = parent->btc_elems + parent_idx * size; - bmov(separator, node->btc_elems + (hdr->bth_count - 1) * size, + bcpy(separator, node->btc_elems + (hdr->bth_count - 1) * size, size); /* * Move the first child of neighbor to the last child spot in * node. */ - zfs_btree_hdr_t **take_child = neighbor->btc_children; - bmov(take_child, node->btc_children + hdr->bth_count, - sizeof (*take_child)); + node->btc_children[hdr->bth_count] = neighbor->btc_children[0]; node->btc_children[hdr->bth_count]->bth_parent = node; /* Move the first element of neighbor to the separator spot. */ uint8_t *take_elem = neighbor->btc_elems; - bmov(take_elem, separator, size); + bcpy(take_elem, separator, size); r_hdr->bth_count--; /* @@ -1458,7 +1530,7 @@ zfs_btree_remove_from_node(zfs_btree_t *tree, zfs_btree_core_t *node, */ bt_shift_core_left(tree, neighbor, 1, r_hdr->bth_count, BSS_TRAPEZOID); - zfs_btree_poison_node_at(tree, r_hdr, r_hdr->bth_count); + zfs_btree_poison_node_at(tree, r_hdr, r_hdr->bth_count, 1); return; } @@ -1473,7 +1545,7 @@ zfs_btree_remove_from_node(zfs_btree_t *tree, zfs_btree_core_t *node, * merging. */ zfs_btree_hdr_t *new_rm_hdr, *keep_hdr; - uint64_t new_idx = idx; + uint32_t new_idx = idx; if (l_hdr != NULL) { keep_hdr = l_hdr; new_rm_hdr = hdr; @@ -1485,14 +1557,14 @@ zfs_btree_remove_from_node(zfs_btree_t *tree, zfs_btree_core_t *node, parent_idx++; } - ASSERT(keep_hdr->bth_core); - ASSERT(new_rm_hdr->bth_core); + ASSERT(zfs_btree_is_core(keep_hdr)); + ASSERT(zfs_btree_is_core(new_rm_hdr)); zfs_btree_core_t *keep = (zfs_btree_core_t *)keep_hdr; zfs_btree_core_t *rm = (zfs_btree_core_t *)new_rm_hdr; if (zfs_btree_verify_intensity >= 5) { - for (int i = 0; i < new_rm_hdr->bth_count + 1; i++) { + for (uint32_t i = 0; i < new_rm_hdr->bth_count + 1; i++) { zfs_btree_verify_poison_at(tree, keep_hdr, keep_hdr->bth_count + i); } @@ -1502,14 +1574,14 @@ zfs_btree_remove_from_node(zfs_btree_t *tree, zfs_btree_core_t *node, uint8_t *e_out = keep->btc_elems + keep_hdr->bth_count * size; uint8_t *separator = parent->btc_elems + (parent_idx - 1) * size; - bmov(separator, e_out, size); + bcpy(separator, e_out, size); keep_hdr->bth_count++; /* Move all our elements and children into the left node. */ bt_transfer_core(tree, rm, 0, new_rm_hdr->bth_count, keep, keep_hdr->bth_count, BSS_TRAPEZOID); - uint64_t old_count = keep_hdr->bth_count; + uint32_t old_count = keep_hdr->bth_count; /* Update bookkeeping */ keep_hdr->bth_count += new_rm_hdr->bth_count; @@ -1527,13 +1599,13 @@ zfs_btree_remove_from_node(zfs_btree_t *tree, zfs_btree_core_t *node, /* Reparent all our children to point to the left node. */ zfs_btree_hdr_t **new_start = keep->btc_children + old_count - 1; - for (int i = 0; i < new_rm_hdr->bth_count + 1; i++) + for (uint32_t i = 0; i < new_rm_hdr->bth_count + 1; i++) new_start[i]->bth_parent = keep; - for (int i = 0; i <= keep_hdr->bth_count; i++) { + for (uint32_t i = 0; i <= keep_hdr->bth_count; i++) { ASSERT3P(keep->btc_children[i]->bth_parent, ==, keep); ASSERT3P(keep->btc_children[i], !=, rm_hdr); } - zfs_btree_poison_node_at(tree, keep_hdr, keep_hdr->bth_count); + zfs_btree_poison_node_at(tree, keep_hdr, keep_hdr->bth_count, 1); new_rm_hdr->bth_count = 0; zfs_btree_node_destroy(tree, new_rm_hdr); @@ -1546,9 +1618,7 @@ zfs_btree_remove_idx(zfs_btree_t *tree, zfs_btree_index_t *where) { size_t size = tree->bt_elem_size; zfs_btree_hdr_t *hdr = where->bti_node; - uint64_t idx = where->bti_offset; - uint64_t capacity = P2ALIGN((BTREE_LEAF_SIZE - - sizeof (zfs_btree_hdr_t)) / size, 2); + uint32_t idx = where->bti_offset; ASSERT(!where->bti_before); if (tree->bt_bulk != NULL) { @@ -1560,7 +1630,7 @@ zfs_btree_remove_idx(zfs_btree_t *tree, zfs_btree_index_t *where) */ uint8_t *value = zfs_btree_get(tree, where); uint8_t *tmp = kmem_alloc(size, KM_SLEEP); - bmov(value, tmp, size); + bcpy(value, tmp, size); zfs_btree_bulk_finish(tree); VERIFY3P(zfs_btree_find(tree, tmp, where), !=, NULL); kmem_free(tmp, size); @@ -1575,14 +1645,14 @@ zfs_btree_remove_idx(zfs_btree_t *tree, zfs_btree_index_t *where) * makes the rebalance logic not need to be recursive both upwards and * downwards. */ - if (hdr->bth_core) { + if (zfs_btree_is_core(hdr)) { zfs_btree_core_t *node = (zfs_btree_core_t *)hdr; zfs_btree_hdr_t *left_subtree = node->btc_children[idx]; void *new_value = zfs_btree_last_helper(tree, left_subtree, where); ASSERT3P(new_value, !=, NULL); - bmov(new_value, node->btc_elems + idx * size, size); + bcpy(new_value, node->btc_elems + idx * size, size); hdr = where->bti_node; idx = where->bti_offset; @@ -1594,19 +1664,18 @@ zfs_btree_remove_idx(zfs_btree_t *tree, zfs_btree_index_t *where) * elements after the idx to the left. After that, we rebalance if * needed. */ - ASSERT(!hdr->bth_core); + ASSERT(!zfs_btree_is_core(hdr)); zfs_btree_leaf_t *leaf = (zfs_btree_leaf_t *)hdr; ASSERT3U(hdr->bth_count, >, 0); - uint64_t min_count = (capacity / 2) - 1; + uint32_t min_count = (tree->bt_leaf_cap / 2) - 1; /* * If we're over the minimum size or this is the root, just overwrite * the value and return. */ if (hdr->bth_count > min_count || hdr->bth_parent == NULL) { - hdr->bth_count--; - bt_shift_leaf_left(tree, leaf, idx + 1, hdr->bth_count - idx); + bt_shrink_leaf(tree, leaf, idx, 1); if (hdr->bth_parent == NULL) { ASSERT0(tree->bt_height); if (hdr->bth_count == 0) { @@ -1615,8 +1684,6 @@ zfs_btree_remove_idx(zfs_btree_t *tree, zfs_btree_index_t *where) zfs_btree_node_destroy(tree, &leaf->btl_hdr); } } - if (tree->bt_root != NULL) - zfs_btree_poison_node_at(tree, hdr, hdr->bth_count); zfs_btree_verify(tree); return; } @@ -1636,33 +1703,33 @@ zfs_btree_remove_idx(zfs_btree_t *tree, zfs_btree_index_t *where) * worth implementing in the future for completeness' sake. */ zfs_btree_core_t *parent = hdr->bth_parent; - uint64_t parent_idx = zfs_btree_find_parent_idx(tree, hdr); + uint32_t parent_idx = zfs_btree_find_parent_idx(tree, hdr); zfs_btree_hdr_t *l_hdr = (parent_idx == 0 ? NULL : parent->btc_children[parent_idx - 1]); if (l_hdr != NULL && l_hdr->bth_count > min_count) { /* We can take a node from the left neighbor. */ - ASSERT(!l_hdr->bth_core); + ASSERT(!zfs_btree_is_core(l_hdr)); + zfs_btree_leaf_t *neighbor = (zfs_btree_leaf_t *)l_hdr; /* * Move our elements back by one spot to make room for the * stolen element and overwrite the element being removed. */ - bt_shift_leaf_right(tree, leaf, 0, idx); + bt_shift_leaf(tree, leaf, 0, idx, 1, BSD_RIGHT); + + /* Move the separator to our first spot. */ uint8_t *separator = parent->btc_elems + (parent_idx - 1) * size; - uint8_t *take_elem = ((zfs_btree_leaf_t *)l_hdr)->btl_elems + - (l_hdr->bth_count - 1) * size; - /* Move the separator to our first spot. */ - bmov(separator, leaf->btl_elems, size); + bcpy(separator, leaf->btl_elems + hdr->bth_first * size, size); /* Move our neighbor's last element to the separator. */ - bmov(take_elem, separator, size); - - /* Update the bookkeeping. */ - l_hdr->bth_count--; - zfs_btree_poison_node_at(tree, l_hdr, l_hdr->bth_count); + uint8_t *take_elem = neighbor->btl_elems + + (l_hdr->bth_first + l_hdr->bth_count - 1) * size; + bcpy(take_elem, separator, size); + /* Delete our neighbor's last element. */ + bt_shrink_leaf(tree, neighbor, l_hdr->bth_count - 1, 1); zfs_btree_verify(tree); return; } @@ -1671,7 +1738,7 @@ zfs_btree_remove_idx(zfs_btree_t *tree, zfs_btree_index_t *where) NULL : parent->btc_children[parent_idx + 1]); if (r_hdr != NULL && r_hdr->bth_count > min_count) { /* We can take a node from the right neighbor. */ - ASSERT(!r_hdr->bth_core); + ASSERT(!zfs_btree_is_core(r_hdr)); zfs_btree_leaf_t *neighbor = (zfs_btree_leaf_t *)r_hdr; /* @@ -1679,94 +1746,79 @@ zfs_btree_remove_idx(zfs_btree_t *tree, zfs_btree_index_t *where) * by one spot to make room for the stolen element and * overwrite the element being removed. */ - bt_shift_leaf_left(tree, leaf, idx + 1, hdr->bth_count - idx - - 1); + bt_shift_leaf(tree, leaf, idx + 1, hdr->bth_count - idx - 1, + 1, BSD_LEFT); - uint8_t *separator = parent->btc_elems + parent_idx * size; - uint8_t *take_elem = ((zfs_btree_leaf_t *)r_hdr)->btl_elems; /* Move the separator between us to our last spot. */ - bmov(separator, leaf->btl_elems + (hdr->bth_count - 1) * size, - size); + uint8_t *separator = parent->btc_elems + parent_idx * size; + bcpy(separator, leaf->btl_elems + (hdr->bth_first + + hdr->bth_count - 1) * size, size); /* Move our neighbor's first element to the separator. */ - bmov(take_elem, separator, size); - - /* Update the bookkeeping. */ - r_hdr->bth_count--; + uint8_t *take_elem = neighbor->btl_elems + + r_hdr->bth_first * size; + bcpy(take_elem, separator, size); - /* - * Move our neighbors elements forwards to overwrite the - * stolen element. - */ - bt_shift_leaf_left(tree, neighbor, 1, r_hdr->bth_count); - zfs_btree_poison_node_at(tree, r_hdr, r_hdr->bth_count); + /* Delete our neighbor's first element. */ + bt_shrink_leaf(tree, neighbor, 0, 1); zfs_btree_verify(tree); return; } /* * In this case, neither of our neighbors can spare an element, so we - * need to merge with one of them. We prefer the left one, - * arbitrarily. Move the separator into the leftmost merging node + * need to merge with one of them. We prefer the left one, arbitrarily. + * After remove we move the separator into the leftmost merging node * (which may be us or the left neighbor), and then move the right * merging node's elements. Once that's done, we go back and delete * the element we're removing. Finally, go into the parent and delete * the right merging node and the separator. This may cause further * merging. */ - zfs_btree_hdr_t *rm_hdr, *keep_hdr; - uint64_t new_idx = idx; + zfs_btree_hdr_t *rm_hdr, *k_hdr; if (l_hdr != NULL) { - keep_hdr = l_hdr; + k_hdr = l_hdr; rm_hdr = hdr; - new_idx += keep_hdr->bth_count + 1; // 449 } else { ASSERT3P(r_hdr, !=, NULL); - keep_hdr = hdr; + k_hdr = hdr; rm_hdr = r_hdr; parent_idx++; } - - ASSERT(!keep_hdr->bth_core); - ASSERT(!rm_hdr->bth_core); - ASSERT3U(keep_hdr->bth_count, ==, min_count); + ASSERT(!zfs_btree_is_core(k_hdr)); + ASSERT(!zfs_btree_is_core(rm_hdr)); + ASSERT3U(k_hdr->bth_count, ==, min_count); ASSERT3U(rm_hdr->bth_count, ==, min_count); - - zfs_btree_leaf_t *keep = (zfs_btree_leaf_t *)keep_hdr; + zfs_btree_leaf_t *keep = (zfs_btree_leaf_t *)k_hdr; zfs_btree_leaf_t *rm = (zfs_btree_leaf_t *)rm_hdr; if (zfs_btree_verify_intensity >= 5) { - for (int i = 0; i < rm_hdr->bth_count + 1; i++) { - zfs_btree_verify_poison_at(tree, keep_hdr, - keep_hdr->bth_count + i); + for (uint32_t i = 0; i < rm_hdr->bth_count + 1; i++) { + zfs_btree_verify_poison_at(tree, k_hdr, + k_hdr->bth_count + i); } } + /* - * Move the separator into the first open spot in the left - * neighbor. + * Remove the value from the node. It will go below the minimum, + * but we'll fix it in no time. */ - uint8_t *out = keep->btl_elems + keep_hdr->bth_count * size; - uint8_t *separator = parent->btc_elems + (parent_idx - 1) * - size; - bmov(separator, out, size); - keep_hdr->bth_count++; + bt_shrink_leaf(tree, leaf, idx, 1); - /* Move our elements to the left neighbor. */ - bt_transfer_leaf(tree, rm, 0, rm_hdr->bth_count, keep, - keep_hdr->bth_count); - - /* Update the bookkeeping. */ - keep_hdr->bth_count += rm_hdr->bth_count; - ASSERT3U(keep_hdr->bth_count, ==, min_count * 2 + 1); + /* Prepare space for elements to be moved from the right. */ + uint32_t k_count = k_hdr->bth_count; + bt_grow_leaf(tree, keep, k_count, 1 + rm_hdr->bth_count); + ASSERT3U(k_hdr->bth_count, ==, min_count * 2); - /* Remove the value from the node */ - keep_hdr->bth_count--; - bt_shift_leaf_left(tree, keep, new_idx + 1, keep_hdr->bth_count - - new_idx); - zfs_btree_poison_node_at(tree, keep_hdr, keep_hdr->bth_count); + /* Move the separator into the first open spot. */ + uint8_t *out = keep->btl_elems + (k_hdr->bth_first + k_count) * size; + uint8_t *separator = parent->btc_elems + (parent_idx - 1) * size; + bcpy(separator, out, size); - rm_hdr->bth_count = 0; + /* Move our elements to the left neighbor. */ + bt_transfer_leaf(tree, rm, 0, rm_hdr->bth_count, keep, k_count + 1); zfs_btree_node_destroy(tree, rm_hdr); + /* Remove the emptied node from the parent. */ zfs_btree_remove_from_node(tree, parent, rm_hdr); zfs_btree_verify(tree); @@ -1831,11 +1883,10 @@ zfs_btree_destroy_nodes(zfs_btree_t *tree, zfs_btree_index_t **cookie) static void zfs_btree_clear_helper(zfs_btree_t *tree, zfs_btree_hdr_t *hdr) { - if (hdr->bth_core) { + if (zfs_btree_is_core(hdr)) { zfs_btree_core_t *btc = (zfs_btree_core_t *)hdr; - for (int i = 0; i <= hdr->bth_count; i++) { + for (uint32_t i = 0; i <= hdr->bth_count; i++) zfs_btree_clear_helper(tree, btc->btc_children[i]); - } } zfs_btree_node_destroy(tree, hdr); @@ -1868,11 +1919,11 @@ zfs_btree_destroy(zfs_btree_t *tree) static void zfs_btree_verify_pointers_helper(zfs_btree_t *tree, zfs_btree_hdr_t *hdr) { - if (!hdr->bth_core) + if (!zfs_btree_is_core(hdr)) return; zfs_btree_core_t *node = (zfs_btree_core_t *)hdr; - for (int i = 0; i <= hdr->bth_count; i++) { + for (uint32_t i = 0; i <= hdr->bth_count; i++) { VERIFY3P(node->btc_children[i]->bth_parent, ==, hdr); zfs_btree_verify_pointers_helper(tree, node->btc_children[i]); } @@ -1897,12 +1948,10 @@ zfs_btree_verify_pointers(zfs_btree_t *tree) static uint64_t zfs_btree_verify_counts_helper(zfs_btree_t *tree, zfs_btree_hdr_t *hdr) { - if (!hdr->bth_core) { + if (!zfs_btree_is_core(hdr)) { if (tree->bt_root != hdr && tree->bt_bulk && hdr != &tree->bt_bulk->btl_hdr) { - uint64_t capacity = P2ALIGN((BTREE_LEAF_SIZE - - sizeof (zfs_btree_hdr_t)) / tree->bt_elem_size, 2); - VERIFY3U(hdr->bth_count, >=, (capacity / 2) - 1); + VERIFY3U(hdr->bth_count, >=, tree->bt_leaf_cap / 2 - 1); } return (hdr->bth_count); @@ -1912,7 +1961,7 @@ zfs_btree_verify_counts_helper(zfs_btree_t *tree, zfs_btree_hdr_t *hdr) uint64_t ret = hdr->bth_count; if (tree->bt_root != hdr && tree->bt_bulk == NULL) VERIFY3P(hdr->bth_count, >=, BTREE_CORE_ELEMS / 2 - 1); - for (int i = 0; i <= hdr->bth_count; i++) { + for (uint32_t i = 0; i <= hdr->bth_count; i++) { ret += zfs_btree_verify_counts_helper(tree, node->btc_children[i]); } @@ -1944,15 +1993,14 @@ static uint64_t zfs_btree_verify_height_helper(zfs_btree_t *tree, zfs_btree_hdr_t *hdr, int64_t height) { - if (!hdr->bth_core) { + if (!zfs_btree_is_core(hdr)) { VERIFY0(height); return (1); } - VERIFY(hdr->bth_core); zfs_btree_core_t *node = (zfs_btree_core_t *)hdr; uint64_t ret = 1; - for (int i = 0; i <= hdr->bth_count; i++) { + for (uint32_t i = 0; i <= hdr->bth_count; i++) { ret += zfs_btree_verify_height_helper(tree, node->btc_children[i], height - 1); } @@ -1984,24 +2032,26 @@ static void zfs_btree_verify_order_helper(zfs_btree_t *tree, zfs_btree_hdr_t *hdr) { size_t size = tree->bt_elem_size; - if (!hdr->bth_core) { + if (!zfs_btree_is_core(hdr)) { zfs_btree_leaf_t *leaf = (zfs_btree_leaf_t *)hdr; - for (int i = 1; i < hdr->bth_count; i++) { - VERIFY3S(tree->bt_compar(leaf->btl_elems + (i - 1) * - size, leaf->btl_elems + i * size), ==, -1); + for (uint32_t i = 1; i < hdr->bth_count; i++) { + VERIFY3S(tree->bt_compar(leaf->btl_elems + + (hdr->bth_first + i - 1) * size, + leaf->btl_elems + + (hdr->bth_first + i) * size), ==, -1); } return; } zfs_btree_core_t *node = (zfs_btree_core_t *)hdr; - for (int i = 1; i < hdr->bth_count; i++) { + for (uint32_t i = 1; i < hdr->bth_count; i++) { VERIFY3S(tree->bt_compar(node->btc_elems + (i - 1) * size, node->btc_elems + i * size), ==, -1); } - for (int i = 0; i < hdr->bth_count; i++) { + for (uint32_t i = 0; i < hdr->bth_count; i++) { uint8_t *left_child_last = NULL; zfs_btree_hdr_t *left_child_hdr = node->btc_children[i]; - if (left_child_hdr->bth_core) { + if (zfs_btree_is_core(left_child_hdr)) { zfs_btree_core_t *left_child = (zfs_btree_core_t *)left_child_hdr; left_child_last = left_child->btc_elems + @@ -2010,40 +2060,39 @@ zfs_btree_verify_order_helper(zfs_btree_t *tree, zfs_btree_hdr_t *hdr) zfs_btree_leaf_t *left_child = (zfs_btree_leaf_t *)left_child_hdr; left_child_last = left_child->btl_elems + - (left_child_hdr->bth_count - 1) * size; + (left_child_hdr->bth_first + + left_child_hdr->bth_count - 1) * size; } - if (tree->bt_compar(node->btc_elems + i * size, - left_child_last) != 1) { + int comp = tree->bt_compar(node->btc_elems + i * size, + left_child_last); + if (comp <= 0) { panic("btree: compar returned %d (expected 1) at " - "%px %d: compar(%px, %px)", tree->bt_compar( - node->btc_elems + i * size, left_child_last), - (void *)node, i, (void *)(node->btc_elems + i * - size), (void *)left_child_last); + "%px %d: compar(%px, %px)", comp, node, i, + node->btc_elems + i * size, left_child_last); } uint8_t *right_child_first = NULL; zfs_btree_hdr_t *right_child_hdr = node->btc_children[i + 1]; - if (right_child_hdr->bth_core) { + if (zfs_btree_is_core(right_child_hdr)) { zfs_btree_core_t *right_child = (zfs_btree_core_t *)right_child_hdr; right_child_first = right_child->btc_elems; } else { zfs_btree_leaf_t *right_child = (zfs_btree_leaf_t *)right_child_hdr; - right_child_first = right_child->btl_elems; + right_child_first = right_child->btl_elems + + right_child_hdr->bth_first * size; } - if (tree->bt_compar(node->btc_elems + i * size, - right_child_first) != -1) { + comp = tree->bt_compar(node->btc_elems + i * size, + right_child_first); + if (comp >= 0) { panic("btree: compar returned %d (expected -1) at " - "%px %d: compar(%px, %px)", tree->bt_compar( - node->btc_elems + i * size, right_child_first), - (void *)node, i, (void *)(node->btc_elems + i * - size), (void *)right_child_first); + "%px %d: compar(%px, %px)", comp, node, i, + node->btc_elems + i * size, right_child_first); } } - for (int i = 0; i <= hdr->bth_count; i++) { + for (uint32_t i = 0; i <= hdr->bth_count; i++) zfs_btree_verify_order_helper(tree, node->btc_children[i]); - } } /* Check that all elements in the tree are in sorted order. */ @@ -2064,27 +2113,26 @@ static void zfs_btree_verify_poison_helper(zfs_btree_t *tree, zfs_btree_hdr_t *hdr) { size_t size = tree->bt_elem_size; - if (!hdr->bth_core) { + if (!zfs_btree_is_core(hdr)) { zfs_btree_leaf_t *leaf = (zfs_btree_leaf_t *)hdr; - uint8_t val = 0x0f; - for (int i = hdr->bth_count * size; i < BTREE_LEAF_SIZE - - sizeof (zfs_btree_hdr_t); i++) { - VERIFY3U(leaf->btl_elems[i], ==, val); - } + for (size_t i = 0; i < hdr->bth_first * size; i++) + VERIFY3U(leaf->btl_elems[i], ==, 0x0f); + for (size_t i = (hdr->bth_first + hdr->bth_count) * size; + i < BTREE_LEAF_ESIZE; i++) + VERIFY3U(leaf->btl_elems[i], ==, 0x0f); } else { zfs_btree_core_t *node = (zfs_btree_core_t *)hdr; - uint8_t val = 0x0f; - for (int i = hdr->bth_count * size; i < BTREE_CORE_ELEMS * size; - i++) { - VERIFY3U(node->btc_elems[i], ==, val); - } + for (size_t i = hdr->bth_count * size; + i < BTREE_CORE_ELEMS * size; i++) + VERIFY3U(node->btc_elems[i], ==, 0x0f); - for (int i = hdr->bth_count + 1; i <= BTREE_CORE_ELEMS; i++) { + for (uint32_t i = hdr->bth_count + 1; i <= BTREE_CORE_ELEMS; + i++) { VERIFY3P(node->btc_children[i], ==, (zfs_btree_hdr_t *)BTREE_POISON); } - for (int i = 0; i <= hdr->bth_count; i++) { + for (uint32_t i = 0; i <= hdr->bth_count; i++) { zfs_btree_verify_poison_helper(tree, node->btc_children[i]); } diff --git a/module/zfs/dataset_kstats.c b/module/zfs/dataset_kstats.c index 3fbb24ddef5e..b63f42a21e44 100644 --- a/module/zfs/dataset_kstats.c +++ b/module/zfs/dataset_kstats.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -37,18 +37,33 @@ static dataset_kstat_values_t empty_dataset_kstats = { { "nread", KSTAT_DATA_UINT64 }, { "nunlinks", KSTAT_DATA_UINT64 }, { "nunlinked", KSTAT_DATA_UINT64 }, + { + { "zil_commit_count", KSTAT_DATA_UINT64 }, + { "zil_commit_writer_count", KSTAT_DATA_UINT64 }, + { "zil_itx_count", KSTAT_DATA_UINT64 }, + { "zil_itx_indirect_count", KSTAT_DATA_UINT64 }, + { "zil_itx_indirect_bytes", KSTAT_DATA_UINT64 }, + { "zil_itx_copied_count", KSTAT_DATA_UINT64 }, + { "zil_itx_copied_bytes", KSTAT_DATA_UINT64 }, + { "zil_itx_needcopy_count", KSTAT_DATA_UINT64 }, + { "zil_itx_needcopy_bytes", KSTAT_DATA_UINT64 }, + { "zil_itx_metaslab_normal_count", KSTAT_DATA_UINT64 }, + { "zil_itx_metaslab_normal_bytes", KSTAT_DATA_UINT64 }, + { "zil_itx_metaslab_slog_count", KSTAT_DATA_UINT64 }, + { "zil_itx_metaslab_slog_bytes", KSTAT_DATA_UINT64 } + } }; static int dataset_kstats_update(kstat_t *ksp, int rw) { dataset_kstats_t *dk = ksp->ks_private; - ASSERT3P(dk->dk_kstats->ks_data, ==, ksp->ks_data); + dataset_kstat_values_t *dkv = ksp->ks_data; + ASSERT3P(dk->dk_kstats->ks_data, ==, dkv); if (rw == KSTAT_WRITE) return (EACCES); - dataset_kstat_values_t *dkv = dk->dk_kstats->ks_data; dkv->dkv_writes.value.ui64 = wmsum_value(&dk->dk_sums.dss_writes); dkv->dkv_nwritten.value.ui64 = @@ -62,10 +77,12 @@ dataset_kstats_update(kstat_t *ksp, int rw) dkv->dkv_nunlinked.value.ui64 = wmsum_value(&dk->dk_sums.dss_nunlinked); + zil_kstat_values_update(&dkv->dkv_zil_stats, &dk->dk_zil_sums); + return (0); } -void +int dataset_kstats_create(dataset_kstats_t *dk, objset_t *objset) { /* @@ -75,7 +92,7 @@ dataset_kstats_create(dataset_kstats_t *dk, objset_t *objset) * a filesystem with many snapshots, we skip them for now. */ if (dmu_objset_is_snapshot(objset)) - return; + return (0); /* * At the time of this writing, KSTAT_STRLEN is 255 in Linux, @@ -94,13 +111,13 @@ dataset_kstats_create(dataset_kstats_t *dk, objset_t *objset) zfs_dbgmsg("failed to create dataset kstat for objset %lld: " " snprintf() for kstat module name returned %d", (unsigned long long)dmu_objset_id(objset), n); - return; + return (SET_ERROR(EINVAL)); } else if (n >= KSTAT_STRLEN) { zfs_dbgmsg("failed to create dataset kstat for objset %lld: " "kstat module name length (%d) exceeds limit (%d)", (unsigned long long)dmu_objset_id(objset), n, KSTAT_STRLEN); - return; + return (SET_ERROR(ENAMETOOLONG)); } char kstat_name[KSTAT_STRLEN]; @@ -110,7 +127,7 @@ dataset_kstats_create(dataset_kstats_t *dk, objset_t *objset) zfs_dbgmsg("failed to create dataset kstat for objset %lld: " " snprintf() for kstat name returned %d", (unsigned long long)dmu_objset_id(objset), n); - return; + return (SET_ERROR(EINVAL)); } ASSERT3U(n, <, KSTAT_STRLEN); @@ -119,11 +136,11 @@ dataset_kstats_create(dataset_kstats_t *dk, objset_t *objset) sizeof (empty_dataset_kstats) / sizeof (kstat_named_t), KSTAT_FLAG_VIRTUAL); if (kstat == NULL) - return; + return (SET_ERROR(ENOMEM)); dataset_kstat_values_t *dk_kstats = kmem_alloc(sizeof (empty_dataset_kstats), KM_SLEEP); - bcopy(&empty_dataset_kstats, dk_kstats, + memcpy(dk_kstats, &empty_dataset_kstats, sizeof (empty_dataset_kstats)); char *ds_name = kmem_zalloc(ZFS_MAX_DATASET_NAME_LEN, KM_SLEEP); @@ -137,15 +154,17 @@ dataset_kstats_create(dataset_kstats_t *dk, objset_t *objset) kstat->ks_private = dk; kstat->ks_data_size += ZFS_MAX_DATASET_NAME_LEN; - kstat_install(kstat); - dk->dk_kstats = kstat; - wmsum_init(&dk->dk_sums.dss_writes, 0); wmsum_init(&dk->dk_sums.dss_nwritten, 0); wmsum_init(&dk->dk_sums.dss_reads, 0); wmsum_init(&dk->dk_sums.dss_nread, 0); wmsum_init(&dk->dk_sums.dss_nunlinks, 0); wmsum_init(&dk->dk_sums.dss_nunlinked, 0); + zil_sums_init(&dk->dk_zil_sums); + + dk->dk_kstats = kstat; + kstat_install(kstat); + return (0); } void @@ -155,19 +174,19 @@ dataset_kstats_destroy(dataset_kstats_t *dk) return; dataset_kstat_values_t *dkv = dk->dk_kstats->ks_data; + kstat_delete(dk->dk_kstats); + dk->dk_kstats = NULL; kmem_free(KSTAT_NAMED_STR_PTR(&dkv->dkv_ds_name), KSTAT_NAMED_STR_BUFLEN(&dkv->dkv_ds_name)); kmem_free(dkv, sizeof (empty_dataset_kstats)); - kstat_delete(dk->dk_kstats); - dk->dk_kstats = NULL; - wmsum_fini(&dk->dk_sums.dss_writes); wmsum_fini(&dk->dk_sums.dss_nwritten); wmsum_fini(&dk->dk_sums.dss_reads); wmsum_fini(&dk->dk_sums.dss_nread); wmsum_fini(&dk->dk_sums.dss_nunlinks); wmsum_fini(&dk->dk_sums.dss_nunlinked); + zil_sums_fini(&dk->dk_zil_sums); } void diff --git a/module/zfs/dbuf.c b/module/zfs/dbuf.c index 26f0d72b2fec..b2d1b9568786 100644 --- a/module/zfs/dbuf.c +++ b/module/zfs/dbuf.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -280,7 +280,7 @@ dbuf_cons(void *vdb, void *unused, int kmflag) { (void) unused, (void) kmflag; dmu_buf_impl_t *db = vdb; - bzero(db, sizeof (dmu_buf_impl_t)); + memset(db, 0, sizeof (dmu_buf_impl_t)); mutex_init(&db->db_mtx, NULL, MUTEX_DEFAULT, NULL); rw_init(&db->db_rwlock, NULL, RW_DEFAULT, NULL); @@ -339,18 +339,18 @@ dbuf_find(objset_t *os, uint64_t obj, uint8_t level, uint64_t blkid) hv = dbuf_hash(os, obj, level, blkid); idx = hv & h->hash_table_mask; - mutex_enter(DBUF_HASH_MUTEX(h, idx)); + rw_enter(DBUF_HASH_RWLOCK(h, idx), RW_READER); for (db = h->hash_table[idx]; db != NULL; db = db->db_hash_next) { if (DBUF_EQUAL(db, os, obj, level, blkid)) { mutex_enter(&db->db_mtx); if (db->db_state != DB_EVICTING) { - mutex_exit(DBUF_HASH_MUTEX(h, idx)); + rw_exit(DBUF_HASH_RWLOCK(h, idx)); return (db); } mutex_exit(&db->db_mtx); } } - mutex_exit(DBUF_HASH_MUTEX(h, idx)); + rw_exit(DBUF_HASH_RWLOCK(h, idx)); return (NULL); } @@ -393,13 +393,13 @@ dbuf_hash_insert(dmu_buf_impl_t *db) hv = dbuf_hash(os, obj, level, blkid); idx = hv & h->hash_table_mask; - mutex_enter(DBUF_HASH_MUTEX(h, idx)); + rw_enter(DBUF_HASH_RWLOCK(h, idx), RW_WRITER); for (dbf = h->hash_table[idx], i = 0; dbf != NULL; dbf = dbf->db_hash_next, i++) { if (DBUF_EQUAL(dbf, os, obj, level, blkid)) { mutex_enter(&dbf->db_mtx); if (dbf->db_state != DB_EVICTING) { - mutex_exit(DBUF_HASH_MUTEX(h, idx)); + rw_exit(DBUF_HASH_RWLOCK(h, idx)); return (dbf); } mutex_exit(&dbf->db_mtx); @@ -417,7 +417,7 @@ dbuf_hash_insert(dmu_buf_impl_t *db) mutex_enter(&db->db_mtx); db->db_hash_next = h->hash_table[idx]; h->hash_table[idx] = db; - mutex_exit(DBUF_HASH_MUTEX(h, idx)); + rw_exit(DBUF_HASH_RWLOCK(h, idx)); uint64_t he = atomic_inc_64_nv(&dbuf_stats.hash_elements.value.ui64); DBUF_STAT_MAX(hash_elements_max, he); @@ -474,13 +474,13 @@ dbuf_hash_remove(dmu_buf_impl_t *db) /* * We mustn't hold db_mtx to maintain lock ordering: - * DBUF_HASH_MUTEX > db_mtx. + * DBUF_HASH_RWLOCK > db_mtx. */ ASSERT(zfs_refcount_is_zero(&db->db_holds)); ASSERT(db->db_state == DB_EVICTING); ASSERT(!MUTEX_HELD(&db->db_mtx)); - mutex_enter(DBUF_HASH_MUTEX(h, idx)); + rw_enter(DBUF_HASH_RWLOCK(h, idx), RW_WRITER); dbp = &h->hash_table[idx]; while ((dbf = *dbp) != db) { dbp = &dbf->db_hash_next; @@ -491,7 +491,7 @@ dbuf_hash_remove(dmu_buf_impl_t *db) if (h->hash_table[idx] && h->hash_table[idx]->db_hash_next == NULL) DBUF_STAT_BUMPDOWN(hash_chains); - mutex_exit(DBUF_HASH_MUTEX(h, idx)); + rw_exit(DBUF_HASH_RWLOCK(h, idx)); atomic_dec_64(&dbuf_stats.hash_elements.value.ui64); } @@ -778,7 +778,7 @@ dbuf_evict_one(void) * of the dbuf cache is at or below the maximum size. Once the dbuf is aged * out of the cache it is destroyed and becomes eligible for arc eviction. */ -static void +static __attribute__((noreturn)) void dbuf_evict_thread(void *unused) { (void) unused; @@ -914,8 +914,8 @@ dbuf_init(void) sizeof (dmu_buf_impl_t), 0, dbuf_cons, dbuf_dest, NULL, NULL, NULL, 0); - for (i = 0; i < DBUF_MUTEXES; i++) - mutex_init(&h->hash_mutexes[i], NULL, MUTEX_DEFAULT, NULL); + for (i = 0; i < DBUF_RWLOCKS; i++) + rw_init(&h->hash_rwlocks[i], NULL, RW_DEFAULT, NULL); dbuf_stats_init(h); @@ -981,8 +981,8 @@ dbuf_fini(void) dbuf_stats_destroy(); - for (i = 0; i < DBUF_MUTEXES; i++) - mutex_destroy(&h->hash_mutexes[i]); + for (i = 0; i < DBUF_RWLOCKS; i++) + rw_destroy(&h->hash_rwlocks[i]); #if defined(_KERNEL) /* * Large allocations which do not require contiguous pages @@ -1235,7 +1235,7 @@ dbuf_loan_arcbuf(dmu_buf_impl_t *db) mutex_exit(&db->db_mtx); abuf = arc_loan_buf(spa, B_FALSE, blksz); - bcopy(db->db.db_data, abuf->b_data, blksz); + memcpy(abuf->b_data, db->db.db_data, blksz); } else { abuf = db->db_buf; arc_loan_inuse_buf(abuf, db); @@ -1297,7 +1297,7 @@ dbuf_whichblock(const dnode_t *dn, const int64_t level, const uint64_t offset) * used when modifying or reading db_blkptr. */ db_lock_type_t -dmu_buf_lock_parent(dmu_buf_impl_t *db, krw_t rw, void *tag) +dmu_buf_lock_parent(dmu_buf_impl_t *db, krw_t rw, const void *tag) { enum db_lock_type ret = DLT_NONE; if (db->db_parent != NULL) { @@ -1322,7 +1322,7 @@ dmu_buf_lock_parent(dmu_buf_impl_t *db, krw_t rw, void *tag) * panic if we didn't pass the lock type in. */ void -dmu_buf_unlock_parent(dmu_buf_impl_t *db, db_lock_type_t type, void *tag) +dmu_buf_unlock_parent(dmu_buf_impl_t *db, db_lock_type_t type, const void *tag) { if (type == DLT_PARENT) rw_exit(&db->db_parent->db_rwlock); @@ -1356,7 +1356,7 @@ dbuf_read_done(zio_t *zio, const zbookmark_phys_t *zb, const blkptr_t *bp, /* freed in flight */ ASSERT(zio == NULL || zio->io_error == 0); arc_release(buf, db); - bzero(buf->b_data, db->db.db_size); + memset(buf->b_data, 0, db->db.db_size); arc_buf_freeze(buf); db->db_freed_in_flight = FALSE; dbuf_set_data(db, buf); @@ -1395,9 +1395,9 @@ dbuf_read_bonus(dmu_buf_impl_t *db, dnode_t *dn, uint32_t flags) db->db.db_data = kmem_alloc(max_bonuslen, KM_SLEEP); arc_space_consume(max_bonuslen, ARC_SPACE_BONUS); if (bonuslen < max_bonuslen) - bzero(db->db.db_data, max_bonuslen); + memset(db->db.db_data, 0, max_bonuslen); if (bonuslen) - bcopy(DN_BONUS(dn->dn_phys), db->db.db_data, bonuslen); + memcpy(db->db.db_data, DN_BONUS(dn->dn_phys), bonuslen); db->db_state = DB_CACHED; DTRACE_SET_STATE(db, "bonus buffer filled"); return (0); @@ -1446,7 +1446,7 @@ dbuf_read_hole(dmu_buf_impl_t *db, dnode_t *dn) if (is_hole) { dbuf_set_data(db, dbuf_alloc_arcbuf(db)); - bzero(db->db.db_data, db->db.db_size); + memset(db->db.db_data, 0, db->db.db_size); if (db->db_blkptr != NULL && db->db_level > 0 && BP_IS_HOLE(db->db_blkptr) && @@ -1522,7 +1522,7 @@ dbuf_read_verify_dnode_crypt(dmu_buf_impl_t *db, uint32_t flags) */ static int dbuf_read_impl(dmu_buf_impl_t *db, zio_t *zio, uint32_t flags, - db_lock_type_t dblt, void *tag) + db_lock_type_t dblt, const void *tag) { dnode_t *dn; zbookmark_phys_t zb; @@ -1657,7 +1657,7 @@ dbuf_fix_old_data(dmu_buf_impl_t *db, uint64_t txg) int bonuslen = DN_SLOTS_TO_BONUSLEN(dn->dn_num_slots); dr->dt.dl.dr_data = kmem_alloc(bonuslen, KM_SLEEP); arc_space_consume(bonuslen, ARC_SPACE_BONUS); - bcopy(db->db.db_data, dr->dt.dl.dr_data, bonuslen); + memcpy(dr->dt.dl.dr_data, db->db.db_data, bonuslen); } else if (zfs_refcount_count(&db->db_holds) > db->db_dirtycnt) { dnode_t *dn = DB_DNODE(db); int size = arc_buf_size(db->db_buf); @@ -1687,7 +1687,7 @@ dbuf_fix_old_data(dmu_buf_impl_t *db, uint64_t txg) } else { dr->dt.dl.dr_data = arc_alloc_buf(spa, db, type, size); } - bcopy(db->db.db_data, dr->dt.dl.dr_data->b_data, size); + memcpy(dr->dt.dl.dr_data->b_data, db->db.db_data, size); } else { db->db_buf = NULL; dbuf_clear_data(db); @@ -1985,7 +1985,7 @@ dbuf_free_range(dnode_t *dn, uint64_t start_blkid, uint64_t end_blkid, ASSERT(db->db.db_data != NULL); arc_release(db->db_buf, db); rw_enter(&db->db_rwlock, RW_WRITER); - bzero(db->db.db_data, db->db.db_size); + memset(db->db.db_data, 0, db->db.db_size); rw_exit(&db->db_rwlock); arc_buf_freeze(db->db_buf); } @@ -2022,10 +2022,10 @@ dbuf_new_size(dmu_buf_impl_t *db, int size, dmu_tx_t *tx) /* copy old block data to the new block */ old_buf = db->db_buf; - bcopy(old_buf->b_data, buf->b_data, MIN(osize, size)); + memcpy(buf->b_data, old_buf->b_data, MIN(osize, size)); /* zero the remainder */ if (size > osize) - bzero((uint8_t *)buf->b_data + osize, size - osize); + memset((uint8_t *)buf->b_data + osize, 0, size - osize); mutex_enter(&db->db_mtx); dbuf_set_data(db, buf); @@ -2655,9 +2655,9 @@ dmu_buf_set_crypt_params(dmu_buf_t *db_fake, boolean_t byteorder, dr->dt.dl.dr_has_raw_params = B_TRUE; dr->dt.dl.dr_byteorder = byteorder; - bcopy(salt, dr->dt.dl.dr_salt, ZIO_DATA_SALT_LEN); - bcopy(iv, dr->dt.dl.dr_iv, ZIO_DATA_IV_LEN); - bcopy(mac, dr->dt.dl.dr_mac, ZIO_DATA_MAC_LEN); + memcpy(dr->dt.dl.dr_salt, salt, ZIO_DATA_SALT_LEN); + memcpy(dr->dt.dl.dr_iv, iv, ZIO_DATA_IV_LEN); + memcpy(dr->dt.dl.dr_mac, mac, ZIO_DATA_MAC_LEN); } static void @@ -2690,7 +2690,7 @@ dmu_buf_fill_done(dmu_buf_t *dbuf, dmu_tx_t *tx) ASSERT(db->db_blkid != DMU_BONUS_BLKID); /* we were freed while filling */ /* XXX dbuf_undirty? */ - bzero(db->db.db_data, db->db.db_size); + memset(db->db.db_data, 0, db->db.db_size); db->db_freed_in_flight = FALSE; DTRACE_SET_STATE(db, "fill done handling freed in flight"); @@ -2802,7 +2802,7 @@ dbuf_assign_arcbuf(dmu_buf_impl_t *db, arc_buf_t *buf, dmu_tx_t *tx) ASSERT(!arc_is_encrypted(buf)); mutex_exit(&db->db_mtx); (void) dbuf_dirty(db, tx); - bcopy(buf->b_data, db->db.db_data, db->db.db_size); + memcpy(db->db.db_data, buf->b_data, db->db.db_size); arc_buf_destroy(buf, db); return; } @@ -2941,9 +2941,6 @@ dbuf_destroy(dmu_buf_impl_t *db) ASSERT3U(db->db_caching_status, ==, DB_NO_CACHE); ASSERT(!multilist_link_active(&db->db_cache_link)); - kmem_cache_free(dbuf_kmem_cache, db); - arc_space_return(sizeof (dmu_buf_impl_t), ARC_SPACE_DBUF); - /* * If this dbuf is referenced from an indirect dbuf, * decrement the ref count on the indirect dbuf. @@ -2952,6 +2949,9 @@ dbuf_destroy(dmu_buf_impl_t *db) mutex_enter(&parent->db_mtx); dbuf_rele_and_unlock(parent, db, B_TRUE); } + + kmem_cache_free(dbuf_kmem_cache, db); + arc_space_return(sizeof (dmu_buf_impl_t), ARC_SPACE_DBUF); } /* @@ -3185,8 +3185,10 @@ typedef struct dbuf_prefetch_arg { static void dbuf_prefetch_fini(dbuf_prefetch_arg_t *dpa, boolean_t io_done) { - if (dpa->dpa_cb != NULL) - dpa->dpa_cb(dpa->dpa_arg, io_done); + if (dpa->dpa_cb != NULL) { + dpa->dpa_cb(dpa->dpa_arg, dpa->dpa_zb.zb_level, + dpa->dpa_zb.zb_blkid, io_done); + } kmem_free(dpa, sizeof (*dpa)); } @@ -3197,9 +3199,10 @@ dbuf_issue_final_prefetch_done(zio_t *zio, const zbookmark_phys_t *zb, (void) zio, (void) zb, (void) iobp; dbuf_prefetch_arg_t *dpa = private; - dbuf_prefetch_fini(dpa, B_TRUE); if (abuf != NULL) arc_buf_destroy(abuf, private); + + dbuf_prefetch_fini(dpa, B_TRUE); } /* @@ -3320,7 +3323,8 @@ dbuf_prefetch_indirect_done(zio_t *zio, const zbookmark_phys_t *zb, dpa->dpa_zb.zb_object, dpa->dpa_curlevel, nextblkid); (void) arc_read(dpa->dpa_zio, dpa->dpa_spa, - bp, dbuf_prefetch_indirect_done, dpa, dpa->dpa_prio, + bp, dbuf_prefetch_indirect_done, dpa, + ZIO_PRIORITY_SYNC_READ, ZIO_FLAG_CANFAIL | ZIO_FLAG_SPECULATIVE, &iter_aflags, &zb); } @@ -3455,7 +3459,8 @@ dbuf_prefetch_impl(dnode_t *dn, int64_t level, uint64_t blkid, SET_BOOKMARK(&zb, ds != NULL ? ds->ds_object : DMU_META_OBJSET, dn->dn_object, curlevel, curblkid); (void) arc_read(dpa->dpa_zio, dpa->dpa_spa, - &bp, dbuf_prefetch_indirect_done, dpa, prio, + &bp, dbuf_prefetch_indirect_done, dpa, + ZIO_PRIORITY_SYNC_READ, ZIO_FLAG_CANFAIL | ZIO_FLAG_SPECULATIVE, &iter_aflags, &zb); } @@ -3467,7 +3472,7 @@ dbuf_prefetch_impl(dnode_t *dn, int64_t level, uint64_t blkid, return (1); no_issue: if (cb != NULL) - cb(arg, B_FALSE); + cb(arg, level, blkid, B_FALSE); return (0); } @@ -3516,7 +3521,7 @@ dbuf_hold_copy(dnode_t *dn, dmu_buf_impl_t *db) } rw_enter(&db->db_rwlock, RW_WRITER); - bcopy(data->b_data, db->db.db_data, arc_buf_size(data)); + memcpy(db->db.db_data, data->b_data, arc_buf_size(data)); rw_exit(&db->db_rwlock); } @@ -3527,7 +3532,7 @@ dbuf_hold_copy(dnode_t *dn, dmu_buf_impl_t *db) int dbuf_hold_impl(dnode_t *dn, uint8_t level, uint64_t blkid, boolean_t fail_sparse, boolean_t fail_uncached, - void *tag, dmu_buf_impl_t **dbp) + const void *tag, dmu_buf_impl_t **dbp) { dmu_buf_impl_t *db, *parent = NULL; @@ -3632,13 +3637,13 @@ dbuf_hold_impl(dnode_t *dn, uint8_t level, uint64_t blkid, } dmu_buf_impl_t * -dbuf_hold(dnode_t *dn, uint64_t blkid, void *tag) +dbuf_hold(dnode_t *dn, uint64_t blkid, const void *tag) { return (dbuf_hold_level(dn, 0, blkid, tag)); } dmu_buf_impl_t * -dbuf_hold_level(dnode_t *dn, int level, uint64_t blkid, void *tag) +dbuf_hold_level(dnode_t *dn, int level, uint64_t blkid, const void *tag) { dmu_buf_impl_t *db; int err = dbuf_hold_impl(dn, level, blkid, FALSE, FALSE, tag, &db); @@ -3679,7 +3684,7 @@ dbuf_rm_spill(dnode_t *dn, dmu_tx_t *tx) #pragma weak dmu_buf_add_ref = dbuf_add_ref void -dbuf_add_ref(dmu_buf_impl_t *db, void *tag) +dbuf_add_ref(dmu_buf_impl_t *db, const void *tag) { int64_t holds = zfs_refcount_add(&db->db_holds, tag); VERIFY3S(holds, >, 1); @@ -3688,7 +3693,7 @@ dbuf_add_ref(dmu_buf_impl_t *db, void *tag) #pragma weak dmu_buf_try_add_ref = dbuf_try_add_ref boolean_t dbuf_try_add_ref(dmu_buf_t *db_fake, objset_t *os, uint64_t obj, uint64_t blkid, - void *tag) + const void *tag) { dmu_buf_impl_t *db = (dmu_buf_impl_t *)db_fake; dmu_buf_impl_t *found_db; @@ -3717,14 +3722,14 @@ dbuf_try_add_ref(dmu_buf_t *db_fake, objset_t *os, uint64_t obj, uint64_t blkid, * dnode's parent dbuf evicting its dnode handles. */ void -dbuf_rele(dmu_buf_impl_t *db, void *tag) +dbuf_rele(dmu_buf_impl_t *db, const void *tag) { mutex_enter(&db->db_mtx); dbuf_rele_and_unlock(db, tag, B_FALSE); } void -dmu_buf_rele(dmu_buf_t *db, void *tag) +dmu_buf_rele(dmu_buf_t *db, const void *tag) { dbuf_rele((dmu_buf_impl_t *)db, tag); } @@ -3743,7 +3748,7 @@ dmu_buf_rele(dmu_buf_t *db, void *tag) * */ void -dbuf_rele_and_unlock(dmu_buf_impl_t *db, void *tag, boolean_t evicting) +dbuf_rele_and_unlock(dmu_buf_impl_t *db, const void *tag, boolean_t evicting) { int64_t holds; uint64_t size; @@ -3947,7 +3952,7 @@ dmu_buf_get_user(dmu_buf_t *db_fake) } void -dmu_buf_user_evict_wait() +dmu_buf_user_evict_wait(void) { taskq_wait(dbu_evict_taskq); } @@ -4040,7 +4045,7 @@ dbuf_sync_bonus(dbuf_dirty_record_t *dr, dmu_tx_t *tx) dnode_t *dn = dr->dr_dnode; ASSERT3U(DN_MAX_BONUS_LEN(dn->dn_phys), <=, DN_SLOTS_TO_BONUSLEN(dn->dn_phys->dn_extra_slots + 1)); - bcopy(data, DN_BONUS(dn->dn_phys), DN_MAX_BONUS_LEN(dn->dn_phys)); + memcpy(DN_BONUS(dn->dn_phys), data, DN_MAX_BONUS_LEN(dn->dn_phys)); dbuf_sync_leaf_verify_bonus_dnode(dr); @@ -4460,7 +4465,7 @@ dbuf_sync_leaf(dbuf_dirty_record_t *dr, dmu_tx_t *tx) } else { *datap = arc_alloc_buf(os->os_spa, db, type, psize); } - bcopy(db->db.db_data, (*datap)->b_data, psize); + memcpy((*datap)->b_data, db->db.db_data, psize); } db->db_data_pending = dr; @@ -4640,7 +4645,7 @@ dbuf_write_children_ready(zio_t *zio, arc_buf_t *buf, void *vdb) * zero out. */ rw_enter(&db->db_rwlock, RW_WRITER); - bzero(db->db.db_data, db->db.db_size); + memset(db->db.db_data, 0, db->db.db_size); rw_exit(&db->db_rwlock); } DB_DNODE_EXIT(db); diff --git a/module/zfs/dbuf_stats.c b/module/zfs/dbuf_stats.c index fa9a5f08060a..747fc337db48 100644 --- a/module/zfs/dbuf_stats.c +++ b/module/zfs/dbuf_stats.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -137,7 +137,7 @@ dbuf_stats_hash_table_data(char *buf, size_t size, void *data) if (size) buf[0] = 0; - mutex_enter(DBUF_HASH_MUTEX(h, dsh->idx)); + rw_enter(DBUF_HASH_RWLOCK(h, dsh->idx), RW_READER); for (db = h->hash_table[dsh->idx]; db != NULL; db = db->db_hash_next) { /* * Returning ENOMEM will cause the data and header functions @@ -158,7 +158,7 @@ dbuf_stats_hash_table_data(char *buf, size_t size, void *data) mutex_exit(&db->db_mtx); } - mutex_exit(DBUF_HASH_MUTEX(h, dsh->idx)); + rw_exit(DBUF_HASH_RWLOCK(h, dsh->idx)); return (error); } diff --git a/module/zfs/ddt.c b/module/zfs/ddt.c index f1415353f4b2..7880a899aeb1 100644 --- a/module/zfs/ddt.c +++ b/module/zfs/ddt.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -46,11 +46,11 @@ static kmem_cache_t *ddt_entry_cache; */ int zfs_dedup_prefetch = 0; -static const ddt_ops_t *ddt_ops[DDT_TYPES] = { +static const ddt_ops_t *const ddt_ops[DDT_TYPES] = { &ddt_zap_ops, }; -static const char *ddt_class_name[DDT_CLASSES] = { +static const char *const ddt_class_name[DDT_CLASSES] = { "ditto", "duplicate", "unique", @@ -99,7 +99,7 @@ ddt_object_destroy(ddt_t *ddt, enum ddt_type type, enum ddt_class class, VERIFY(zap_remove(os, DMU_POOL_DIRECTORY_OBJECT, name, tx) == 0); VERIFY(zap_remove(os, spa->spa_ddt_stat_object, name, tx) == 0); VERIFY(ddt_ops[type]->ddt_op_destroy(os, *objectp, tx) == 0); - bzero(&ddt->ddt_object_stats[type][class], sizeof (ddt_object_t)); + memset(&ddt->ddt_object_stats[type][class], 0, sizeof (ddt_object_t)); *objectp = 0; } @@ -322,7 +322,7 @@ ddt_phys_fill(ddt_phys_t *ddp, const blkptr_t *bp) void ddt_phys_clear(ddt_phys_t *ddp) { - bzero(ddp, sizeof (*ddp)); + memset(ddp, 0, sizeof (*ddp)); } void @@ -390,7 +390,7 @@ ddt_stat_generate(ddt_t *ddt, ddt_entry_t *dde, ddt_stat_t *dds) uint64_t lsize = DDK_GET_LSIZE(ddk); uint64_t psize = DDK_GET_PSIZE(ddk); - bzero(dds, sizeof (*dds)); + memset(dds, 0, sizeof (*dds)); for (int p = 0; p < DDT_PHYS_TYPES; p++, ddp++) { uint64_t dsize = 0; @@ -454,7 +454,7 @@ ddt_histogram_add(ddt_histogram_t *dst, const ddt_histogram_t *src) void ddt_histogram_stat(ddt_stat_t *dds, const ddt_histogram_t *ddh) { - bzero(dds, sizeof (*dds)); + memset(dds, 0, sizeof (*dds)); for (int h = 0; h < 64; h++) ddt_stat_add(dds, &ddh->ddh_stat[h], 0); @@ -532,7 +532,7 @@ ddt_get_dedup_dspace(spa_t *spa) if (spa->spa_dedup_dspace != ~0ULL) return (spa->spa_dedup_dspace); - bzero(&dds_total, sizeof (ddt_stat_t)); + memset(&dds_total, 0, sizeof (ddt_stat_t)); /* Calculate and cache the stats */ ddt_get_dedup_stats(spa, &dds_total); @@ -566,7 +566,7 @@ ddt_compress(void *src, uchar_t *dst, size_t s_len, size_t d_len) if (c_len == s_len) { cpfunc = ZIO_COMPRESS_OFF; - bcopy(src, dst, s_len); + memcpy(dst, src, s_len); } *version = cpfunc; @@ -586,7 +586,7 @@ ddt_decompress(uchar_t *src, void *dst, size_t s_len, size_t d_len) if (ci->ci_decompress != NULL) (void) ci->ci_decompress(src, dst, s_len, d_len, ci->ci_level); else - bcopy(src, dst, d_len); + memcpy(dst, src, d_len); if (((version & DDT_COMPRESS_BYTEORDER_MASK) != 0) != (ZFS_HOST_BYTEORDER != 0)) @@ -633,7 +633,7 @@ ddt_alloc(const ddt_key_t *ddk) ddt_entry_t *dde; dde = kmem_cache_alloc(ddt_entry_cache, KM_SLEEP); - bzero(dde, sizeof (ddt_entry_t)); + memset(dde, 0, sizeof (ddt_entry_t)); cv_init(&dde->dde_cv, NULL, CV_DEFAULT, NULL); dde->dde_key = *ddk; @@ -785,7 +785,7 @@ ddt_table_alloc(spa_t *spa, enum zio_checksum c) ddt_t *ddt; ddt = kmem_cache_alloc(ddt_cache, KM_SLEEP); - bzero(ddt, sizeof (ddt_t)); + memset(ddt, 0, sizeof (ddt_t)); mutex_init(&ddt->ddt_lock, NULL, MUTEX_DEFAULT, NULL); avl_create(&ddt->ddt_tree, ddt_entry_compare, @@ -847,7 +847,7 @@ ddt_load(spa_t *spa) /* * Seed the cached histograms. */ - bcopy(ddt->ddt_histogram, &ddt->ddt_histogram_cache, + memcpy(&ddt->ddt_histogram_cache, ddt->ddt_histogram, sizeof (ddt->ddt_histogram)); spa->spa_dedup_dspace = ~0ULL; } @@ -919,7 +919,7 @@ ddt_repair_start(ddt_t *ddt, const blkptr_t *bp) } } - bzero(dde->dde_phys, sizeof (dde->dde_phys)); + memset(dde->dde_phys, 0, sizeof (dde->dde_phys)); return (dde); } @@ -964,7 +964,7 @@ ddt_repair_entry(ddt_t *ddt, ddt_entry_t *dde, ddt_entry_t *rdde, zio_t *rio) for (int p = 0; p < DDT_PHYS_TYPES; p++, ddp++, rddp++) { if (ddp->ddp_phys_birth == 0 || ddp->ddp_phys_birth != rddp->ddp_phys_birth || - bcmp(ddp->ddp_dva, rddp->ddp_dva, sizeof (ddp->ddp_dva))) + memcmp(ddp->ddp_dva, rddp->ddp_dva, sizeof (ddp->ddp_dva))) continue; ddt_bp_create(ddt->ddt_checksum, ddk, ddp, &blk); zio_nowait(zio_rewrite(zio, zio->io_spa, 0, &blk, @@ -1108,7 +1108,7 @@ ddt_sync_table(ddt_t *ddt, dmu_tx_t *tx, uint64_t txg) } } - bcopy(ddt->ddt_histogram, &ddt->ddt_histogram_cache, + memcpy(&ddt->ddt_histogram_cache, ddt->ddt_histogram, sizeof (ddt->ddt_histogram)); spa->spa_dedup_dspace = ~0ULL; } diff --git a/module/zfs/ddt_zap.c b/module/zfs/ddt_zap.c index d0127f22e5ff..27dbbc55f121 100644 --- a/module/zfs/ddt_zap.c +++ b/module/zfs/ddt_zap.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/dmu.c b/module/zfs/dmu.c index 874ddc800870..58c88c7d7854 100644 --- a/module/zfs/dmu.c +++ b/module/zfs/dmu.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -86,7 +86,7 @@ static int zfs_dmu_offset_next_sync = 1; * helps to limit the amount of memory that can be used by prefetching. * Larger objects should be prefetched a bit at a time. */ -static int dmu_prefetch_max = 8 * SPA_MAXBLOCKSIZE; +int dmu_prefetch_max = 8 * SPA_MAXBLOCKSIZE; const dmu_object_type_info_t dmu_ot[DMU_OT_NUMTYPES] = { {DMU_BSWAP_UINT8, TRUE, FALSE, FALSE, "unallocated" }, @@ -160,7 +160,7 @@ const dmu_object_byteswap_info_t dmu_ot_byteswap[DMU_BSWAP_NUMFUNCS] = { static int dmu_buf_hold_noread_by_dnode(dnode_t *dn, uint64_t offset, - void *tag, dmu_buf_t **dbp) + const void *tag, dmu_buf_t **dbp) { uint64_t blkid; dmu_buf_impl_t *db; @@ -180,7 +180,7 @@ dmu_buf_hold_noread_by_dnode(dnode_t *dn, uint64_t offset, } int dmu_buf_hold_noread(objset_t *os, uint64_t object, uint64_t offset, - void *tag, dmu_buf_t **dbp) + const void *tag, dmu_buf_t **dbp) { dnode_t *dn; uint64_t blkid; @@ -207,7 +207,7 @@ dmu_buf_hold_noread(objset_t *os, uint64_t object, uint64_t offset, int dmu_buf_hold_by_dnode(dnode_t *dn, uint64_t offset, - void *tag, dmu_buf_t **dbp, int flags) + const void *tag, dmu_buf_t **dbp, int flags) { int err; int db_flags = DB_RF_CANFAIL; @@ -232,7 +232,7 @@ dmu_buf_hold_by_dnode(dnode_t *dn, uint64_t offset, int dmu_buf_hold(objset_t *os, uint64_t object, uint64_t offset, - void *tag, dmu_buf_t **dbp, int flags) + const void *tag, dmu_buf_t **dbp, int flags) { int err; int db_flags = DB_RF_CANFAIL; @@ -342,7 +342,7 @@ dmu_rm_spill(objset_t *os, uint64_t object, dmu_tx_t *tx) * has not yet been allocated a new bonus dbuf a will be allocated. * Returns ENOENT, EIO, or 0. */ -int dmu_bonus_hold_by_dnode(dnode_t *dn, void *tag, dmu_buf_t **dbp, +int dmu_bonus_hold_by_dnode(dnode_t *dn, const void *tag, dmu_buf_t **dbp, uint32_t flags) { dmu_buf_impl_t *db; @@ -389,7 +389,7 @@ int dmu_bonus_hold_by_dnode(dnode_t *dn, void *tag, dmu_buf_t **dbp, } int -dmu_bonus_hold(objset_t *os, uint64_t object, void *tag, dmu_buf_t **dbp) +dmu_bonus_hold(objset_t *os, uint64_t object, const void *tag, dmu_buf_t **dbp) { dnode_t *dn; int error; @@ -414,7 +414,8 @@ dmu_bonus_hold(objset_t *os, uint64_t object, void *tag, dmu_buf_t **dbp) * dmu_spill_hold_existing() should be used. */ int -dmu_spill_hold_by_dnode(dnode_t *dn, uint32_t flags, void *tag, dmu_buf_t **dbp) +dmu_spill_hold_by_dnode(dnode_t *dn, uint32_t flags, const void *tag, + dmu_buf_t **dbp) { dmu_buf_impl_t *db = NULL; int err; @@ -442,7 +443,7 @@ dmu_spill_hold_by_dnode(dnode_t *dn, uint32_t flags, void *tag, dmu_buf_t **dbp) } int -dmu_spill_hold_existing(dmu_buf_t *bonus, void *tag, dmu_buf_t **dbp) +dmu_spill_hold_existing(dmu_buf_t *bonus, const void *tag, dmu_buf_t **dbp) { dmu_buf_impl_t *db = (dmu_buf_impl_t *)bonus; dnode_t *dn; @@ -471,7 +472,7 @@ dmu_spill_hold_existing(dmu_buf_t *bonus, void *tag, dmu_buf_t **dbp) } int -dmu_spill_hold_by_bonus(dmu_buf_t *bonus, uint32_t flags, void *tag, +dmu_spill_hold_by_bonus(dmu_buf_t *bonus, uint32_t flags, const void *tag, dmu_buf_t **dbp) { dmu_buf_impl_t *db = (dmu_buf_impl_t *)bonus; @@ -498,7 +499,8 @@ dmu_spill_hold_by_bonus(dmu_buf_t *bonus, uint32_t flags, void *tag, */ int dmu_buf_hold_array_by_dnode(dnode_t *dn, uint64_t offset, uint64_t length, - boolean_t read, void *tag, int *numbufsp, dmu_buf_t ***dbpp, uint32_t flags) + boolean_t read, const void *tag, int *numbufsp, dmu_buf_t ***dbpp, + uint32_t flags) { dmu_buf_t **dbp; zstream_t *zs = NULL; @@ -518,6 +520,9 @@ dmu_buf_hold_array_by_dnode(dnode_t *dn, uint64_t offset, uint64_t length, dbuf_flags = DB_RF_CANFAIL | DB_RF_NEVERWAIT | DB_RF_HAVESTRUCT | DB_RF_NOPREFETCH; + if ((flags & DMU_READ_NO_DECRYPT) != 0) + dbuf_flags |= DB_RF_NO_DECRYPT; + rw_enter(&dn->dn_struct_rwlock, RW_READER); if (dn->dn_datablkshift) { int blkshift = dn->dn_datablkshift; @@ -619,7 +624,8 @@ dmu_buf_hold_array_by_dnode(dnode_t *dn, uint64_t offset, uint64_t length, int dmu_buf_hold_array(objset_t *os, uint64_t object, uint64_t offset, - uint64_t length, int read, void *tag, int *numbufsp, dmu_buf_t ***dbpp) + uint64_t length, int read, const void *tag, int *numbufsp, + dmu_buf_t ***dbpp) { dnode_t *dn; int err; @@ -638,7 +644,7 @@ dmu_buf_hold_array(objset_t *os, uint64_t object, uint64_t offset, int dmu_buf_hold_array_by_bonus(dmu_buf_t *db_fake, uint64_t offset, - uint64_t length, boolean_t read, void *tag, int *numbufsp, + uint64_t length, boolean_t read, const void *tag, int *numbufsp, dmu_buf_t ***dbpp) { dmu_buf_impl_t *db = (dmu_buf_impl_t *)db_fake; @@ -655,7 +661,7 @@ dmu_buf_hold_array_by_bonus(dmu_buf_t *db_fake, uint64_t offset, } void -dmu_buf_rele_array(dmu_buf_t **dbp_fake, int numbufs, void *tag) +dmu_buf_rele_array(dmu_buf_t **dbp_fake, int numbufs, const void *tag) { int i; dmu_buf_impl_t **dbp = (dmu_buf_impl_t **)dbp_fake; @@ -1012,7 +1018,7 @@ dmu_read_impl(dnode_t *dn, uint64_t offset, uint64_t size, if (dn->dn_maxblkid == 0) { uint64_t newsz = offset > dn->dn_datablksz ? 0 : MIN(size, dn->dn_datablksz - offset); - bzero((char *)buf + newsz, size - newsz); + memset((char *)buf + newsz, 0, size - newsz); size = newsz; } @@ -2077,9 +2083,9 @@ dmu_write_policy(objset_t *os, dnode_t *dn, int level, int wp, zio_prop_t *zp) zp->zp_nopwrite = nopwrite; zp->zp_encrypt = encrypt; zp->zp_byteorder = ZFS_HOST_BYTEORDER; - bzero(zp->zp_salt, ZIO_DATA_SALT_LEN); - bzero(zp->zp_iv, ZIO_DATA_IV_LEN); - bzero(zp->zp_mac, ZIO_DATA_MAC_LEN); + memset(zp->zp_salt, 0, ZIO_DATA_SALT_LEN); + memset(zp->zp_iv, 0, ZIO_DATA_IV_LEN); + memset(zp->zp_mac, 0, ZIO_DATA_MAC_LEN); zp->zp_zpl_smallblk = DMU_OT_IS_FILE(zp->zp_type) ? os->os_zpl_special_smallblock : 0; diff --git a/module/zfs/dmu_diff.c b/module/zfs/dmu_diff.c index 1382da267d8a..a2b1a27c88e9 100644 --- a/module/zfs/dmu_diff.c +++ b/module/zfs/dmu_diff.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/dmu_object.c b/module/zfs/dmu_object.c index 12cdbd68b104..640af59be550 100644 --- a/module/zfs/dmu_object.c +++ b/module/zfs/dmu_object.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -46,7 +46,7 @@ int dmu_object_alloc_chunk_shift = 7; static uint64_t dmu_object_alloc_impl(objset_t *os, dmu_object_type_t ot, int blocksize, int indirect_blockshift, dmu_object_type_t bonustype, int bonuslen, - int dnodesize, dnode_t **allocated_dnode, void *tag, dmu_tx_t *tx) + int dnodesize, dnode_t **allocated_dnode, const void *tag, dmu_tx_t *tx) { uint64_t object; uint64_t L1_dnode_count = DNODES_PER_BLOCK << @@ -255,7 +255,7 @@ dmu_object_alloc_dnsize(objset_t *os, dmu_object_type_t ot, int blocksize, uint64_t dmu_object_alloc_hold(objset_t *os, dmu_object_type_t ot, int blocksize, int indirect_blockshift, dmu_object_type_t bonustype, int bonuslen, - int dnodesize, dnode_t **allocated_dnode, void *tag, dmu_tx_t *tx) + int dnodesize, dnode_t **allocated_dnode, const void *tag, dmu_tx_t *tx) { return (dmu_object_alloc_impl(os, ot, blocksize, indirect_blockshift, bonustype, bonuslen, dnodesize, allocated_dnode, tag, tx)); diff --git a/module/zfs/dmu_objset.c b/module/zfs/dmu_objset.c index 9a74fa9ce360..b9e16f79efc5 100644 --- a/module/zfs/dmu_objset.c +++ b/module/zfs/dmu_objset.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -516,8 +516,8 @@ dmu_objset_open_impl(spa_t *spa, dsl_dataset_t *ds, blkptr_t *bp, if (arc_buf_size(os->os_phys_buf) < size) { arc_buf_t *buf = arc_alloc_buf(spa, &os->os_phys_buf, ARC_BUFC_METADATA, size); - bzero(buf->b_data, size); - bcopy(os->os_phys_buf->b_data, buf->b_data, + memset(buf->b_data, 0, size); + memcpy(buf->b_data, os->os_phys_buf->b_data, arc_buf_size(os->os_phys_buf)); arc_buf_destroy(os->os_phys_buf, &os->os_phys_buf); os->os_phys_buf = buf; @@ -531,7 +531,7 @@ dmu_objset_open_impl(spa_t *spa, dsl_dataset_t *ds, blkptr_t *bp, os->os_phys_buf = arc_alloc_buf(spa, &os->os_phys_buf, ARC_BUFC_METADATA, size); os->os_phys = os->os_phys_buf->b_data; - bzero(os->os_phys, size); + memset(os->os_phys, 0, size); } /* * These properties will be filled in by the logic in zfs_get_zplprop() @@ -714,7 +714,7 @@ dmu_objset_from_ds(dsl_dataset_t *ds, objset_t **osp) * can be held at a time. */ int -dmu_objset_hold_flags(const char *name, boolean_t decrypt, void *tag, +dmu_objset_hold_flags(const char *name, boolean_t decrypt, const void *tag, objset_t **osp) { dsl_pool_t *dp; @@ -742,14 +742,14 @@ dmu_objset_hold_flags(const char *name, boolean_t decrypt, void *tag, } int -dmu_objset_hold(const char *name, void *tag, objset_t **osp) +dmu_objset_hold(const char *name, const void *tag, objset_t **osp) { return (dmu_objset_hold_flags(name, B_FALSE, tag, osp)); } static int dmu_objset_own_impl(dsl_dataset_t *ds, dmu_objset_type_t type, - boolean_t readonly, boolean_t decrypt, void *tag, objset_t **osp) + boolean_t readonly, boolean_t decrypt, const void *tag, objset_t **osp) { (void) tag; @@ -789,7 +789,7 @@ dmu_objset_own_impl(dsl_dataset_t *ds, dmu_objset_type_t type, */ int dmu_objset_own(const char *name, dmu_objset_type_t type, - boolean_t readonly, boolean_t decrypt, void *tag, objset_t **osp) + boolean_t readonly, boolean_t decrypt, const void *tag, objset_t **osp) { dsl_pool_t *dp; dsl_dataset_t *ds; @@ -834,7 +834,7 @@ dmu_objset_own(const char *name, dmu_objset_type_t type, int dmu_objset_own_obj(dsl_pool_t *dp, uint64_t obj, dmu_objset_type_t type, - boolean_t readonly, boolean_t decrypt, void *tag, objset_t **osp) + boolean_t readonly, boolean_t decrypt, const void *tag, objset_t **osp) { dsl_dataset_t *ds; int err; @@ -855,7 +855,7 @@ dmu_objset_own_obj(dsl_pool_t *dp, uint64_t obj, dmu_objset_type_t type, } void -dmu_objset_rele_flags(objset_t *os, boolean_t decrypt, void *tag) +dmu_objset_rele_flags(objset_t *os, boolean_t decrypt, const void *tag) { ds_hold_flags_t flags; dsl_pool_t *dp = dmu_objset_pool(os); @@ -866,7 +866,7 @@ dmu_objset_rele_flags(objset_t *os, boolean_t decrypt, void *tag) } void -dmu_objset_rele(objset_t *os, void *tag) +dmu_objset_rele(objset_t *os, const void *tag) { dmu_objset_rele_flags(os, B_FALSE, tag); } @@ -884,7 +884,7 @@ dmu_objset_rele(objset_t *os, void *tag) */ void dmu_objset_refresh_ownership(dsl_dataset_t *ds, dsl_dataset_t **newds, - boolean_t decrypt, void *tag) + boolean_t decrypt, const void *tag) { dsl_pool_t *dp; char name[ZFS_MAX_DATASET_NAME_LEN]; @@ -904,7 +904,7 @@ dmu_objset_refresh_ownership(dsl_dataset_t *ds, dsl_dataset_t **newds, } void -dmu_objset_disown(objset_t *os, boolean_t decrypt, void *tag) +dmu_objset_disown(objset_t *os, boolean_t decrypt, const void *tag) { ds_hold_flags_t flags; @@ -1695,6 +1695,16 @@ dmu_objset_sync(objset_t *os, zio_t *pio, dmu_tx_t *tx) &zp, dmu_objset_write_ready, NULL, NULL, dmu_objset_write_done, os, ZIO_PRIORITY_ASYNC_WRITE, ZIO_FLAG_MUSTSUCCEED, &zb); + /* + * In the codepath dsl_dataset_sync()->dmu_objset_sync() we cannot + * rely on the zio above completing and calling back + * dmu_objset_write_done()->dsl_dataset_block_born() before + * dsl_dataset_sync() actually activates feature flags near its end. + * Decide here if any features need to be activated, before + * dsl_dataset_sync() completes its run. + */ + dsl_dataset_feature_set_activation(blkptr_copy, os->os_dsl_dataset); + /* * Sync special dnodes - the parent IO for the sync is the root block */ diff --git a/module/zfs/dmu_recv.c b/module/zfs/dmu_recv.c index 0d765921382f..55d03677feaa 100644 --- a/module/zfs/dmu_recv.c +++ b/module/zfs/dmu_recv.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -27,8 +27,11 @@ * Copyright (c) 2018, loli10K . All rights reserved. * Copyright (c) 2019, Klara Inc. * Copyright (c) 2019, Allan Jude + * Copyright (c) 2019 Datto Inc. + * Copyright (c) 2022 Axcient. */ +#include #include #include #include @@ -67,8 +70,9 @@ static int zfs_recv_queue_length = SPA_MAXBLOCKSIZE; static int zfs_recv_queue_ff = 20; static int zfs_recv_write_batch_size = 1024 * 1024; +static int zfs_recv_best_effort_corrective = 0; -static void *const dmu_recv_tag = "dmu_recv_tag"; +static const void *const dmu_recv_tag = "dmu_recv_tag"; const char *const recv_clone_name = "%recv"; static int receive_read_payload_and_next_header(dmu_recv_cookie_t *ra, int len, @@ -102,6 +106,8 @@ struct receive_writer_arg { boolean_t done; int err; + const char *tofs; + boolean_t heal; boolean_t resumable; boolean_t raw; /* DMU_BACKUP_FEATURE_RAW set */ boolean_t spill; /* DRR_FLAG_SPILL_BLOCK set */ @@ -121,6 +127,7 @@ struct receive_writer_arg { uint8_t or_iv[ZIO_DATA_IV_LEN]; uint8_t or_mac[ZIO_DATA_MAC_LEN]; boolean_t or_byteorder; + zio_t *heal_pio; }; typedef struct dmu_recv_begin_arg { @@ -343,9 +350,10 @@ static int recv_begin_check_existing_impl(dmu_recv_begin_arg_t *drba, dsl_dataset_t *ds, uint64_t fromguid, uint64_t featureflags) { - uint64_t val; + uint64_t obj; uint64_t children; int error; + dsl_dataset_t *snap; dsl_pool_t *dp = ds->ds_dir->dd_pool; boolean_t encrypted = ds->ds_dir->dd_crypto_obj != 0; boolean_t raw = (featureflags & DMU_BACKUP_FEATURE_RAW) != 0; @@ -354,7 +362,7 @@ recv_begin_check_existing_impl(dmu_recv_begin_arg_t *drba, dsl_dataset_t *ds, /* Temporary clone name must not exist. */ error = zap_lookup(dp->dp_meta_objset, dsl_dir_phys(ds->ds_dir)->dd_child_dir_zapobj, recv_clone_name, - 8, 1, &val); + 8, 1, &obj); if (error != ENOENT) return (error == 0 ? SET_ERROR(EBUSY) : error); @@ -362,12 +370,16 @@ recv_begin_check_existing_impl(dmu_recv_begin_arg_t *drba, dsl_dataset_t *ds, if (dsl_dataset_has_resume_receive_state(ds)) return (SET_ERROR(EBUSY)); - /* New snapshot name must not exist. */ + /* New snapshot name must not exist if we're not healing it. */ error = zap_lookup(dp->dp_meta_objset, dsl_dataset_phys(ds)->ds_snapnames_zapobj, - drba->drba_cookie->drc_tosnap, 8, 1, &val); - if (error != ENOENT) + drba->drba_cookie->drc_tosnap, 8, 1, &obj); + if (drba->drba_cookie->drc_heal) { + if (error != 0) + return (error); + } else if (error != ENOENT) { return (error == 0 ? SET_ERROR(EEXIST) : error); + } /* Must not have children if receiving a ZVOL. */ error = zap_count(dp->dp_meta_objset, @@ -392,8 +404,40 @@ recv_begin_check_existing_impl(dmu_recv_begin_arg_t *drba, dsl_dataset_t *ds, if (error != 0) return (error); - if (fromguid != 0) { - dsl_dataset_t *snap; + if (drba->drba_cookie->drc_heal) { + /* Encryption is incompatible with embedded data. */ + if (encrypted && embed) + return (SET_ERROR(EINVAL)); + + /* Healing is not supported when in 'force' mode. */ + if (drba->drba_cookie->drc_force) + return (SET_ERROR(EINVAL)); + + /* Must have keys loaded if doing encrypted non-raw recv. */ + if (encrypted && !raw) { + if (spa_keystore_lookup_key(dp->dp_spa, ds->ds_object, + NULL, NULL) != 0) + return (SET_ERROR(EACCES)); + } + + error = dsl_dataset_hold_obj(dp, obj, FTAG, &snap); + if (error != 0) + return (error); + + /* + * When not doing best effort corrective recv healing can only + * be done if the send stream is for the same snapshot as the + * one we are trying to heal. + */ + if (zfs_recv_best_effort_corrective == 0 && + drba->drba_cookie->drc_drrb->drr_toguid != + dsl_dataset_phys(snap)->ds_guid) { + dsl_dataset_rele(snap, FTAG); + return (SET_ERROR(ENOTSUP)); + } + dsl_dataset_rele(snap, FTAG); + } else if (fromguid != 0) { + /* Sanity check the incremental recv */ uint64_t obj = dsl_dataset_phys(ds)->ds_prev_snap_obj; /* Can't perform a raw receive on top of a non-raw receive */ @@ -459,7 +503,7 @@ recv_begin_check_existing_impl(dmu_recv_begin_arg_t *drba, dsl_dataset_t *ds, dsl_dataset_rele(snap, FTAG); } else { - /* if full, then must be forced */ + /* If full and not healing then must be forced. */ if (!drba->drba_cookie->drc_force) return (SET_ERROR(EEXIST)); @@ -626,6 +670,10 @@ dmu_recv_begin_check(void *arg, dmu_tx_t *tx) char buf[ZFS_MAX_DATASET_NAME_LEN]; objset_t *os; + /* healing recv must be done "into" an existing snapshot */ + if (drba->drba_cookie->drc_heal == B_TRUE) + return (SET_ERROR(ENOTSUP)); + /* * If it's a non-clone incremental, we are missing the * target fs, so fail the recv. @@ -807,7 +855,7 @@ dmu_recv_begin_sync(void *arg, dmu_tx_t *tx) error = dsl_dataset_hold_flags(dp, tofs, dsflags, FTAG, &ds); if (error == 0) { - /* create temporary clone */ + /* Create temporary clone unless we're doing corrective recv */ dsl_dataset_t *snap = NULL; if (drba->drba_cookie->drc_fromsnapobj != 0) { @@ -815,8 +863,15 @@ dmu_recv_begin_sync(void *arg, dmu_tx_t *tx) drba->drba_cookie->drc_fromsnapobj, FTAG, &snap)); ASSERT3P(dcp, ==, NULL); } - dsobj = dsl_dataset_create_sync(ds->ds_dir, recv_clone_name, - snap, crflags, drba->drba_cred, dcp, tx); + if (drc->drc_heal) { + /* When healing we want to use the provided snapshot */ + VERIFY0(dsl_dataset_snap_lookup(ds, drc->drc_tosnap, + &dsobj)); + } else { + dsobj = dsl_dataset_create_sync(ds->ds_dir, + recv_clone_name, snap, crflags, drba->drba_cred, + dcp, tx); + } if (drba->drba_cookie->drc_fromsnapobj != 0) dsl_dataset_rele(snap, FTAG); dsl_dataset_rele_flags(ds, dsflags, FTAG); @@ -933,7 +988,8 @@ dmu_recv_begin_sync(void *arg, dmu_tx_t *tx) */ rrw_enter(&newds->ds_bp_rwlock, RW_READER, FTAG); if (BP_IS_HOLE(dsl_dataset_get_blkptr(newds)) && - (featureflags & DMU_BACKUP_FEATURE_RAW) == 0) { + (featureflags & DMU_BACKUP_FEATURE_RAW) == 0 && + !drc->drc_heal) { (void) dmu_objset_create_impl(dp->dp_spa, newds, dsl_dataset_get_blkptr(newds), drrb->drr_type, tx); } @@ -1141,19 +1197,20 @@ dmu_recv_resume_begin_sync(void *arg, dmu_tx_t *tx) */ int dmu_recv_begin(char *tofs, char *tosnap, dmu_replay_record_t *drr_begin, - boolean_t force, boolean_t resumable, nvlist_t *localprops, + boolean_t force, boolean_t heal, boolean_t resumable, nvlist_t *localprops, nvlist_t *hidden_args, char *origin, dmu_recv_cookie_t *drc, zfs_file_t *fp, offset_t *voffp) { dmu_recv_begin_arg_t drba = { 0 }; int err; - bzero(drc, sizeof (dmu_recv_cookie_t)); + memset(drc, 0, sizeof (dmu_recv_cookie_t)); drc->drc_drr_begin = drr_begin; drc->drc_drrb = &drr_begin->drr_u.drr_begin; drc->drc_tosnap = tosnap; drc->drc_tofs = tofs; drc->drc_force = force; + drc->drc_heal = heal; drc->drc_resumable = resumable; drc->drc_cred = CRED(); drc->drc_proc = curproc; @@ -1211,7 +1268,6 @@ dmu_recv_begin(char *tofs, char *tosnap, dmu_replay_record_t *drr_begin, dmu_recv_resume_begin_check, dmu_recv_resume_begin_sync, &drba, 5, ZFS_SPACE_CHECK_NORMAL); } else { - /* * For non-raw, non-incremental, non-resuming receives the * user can specify encryption parameters on the command line @@ -1244,6 +1300,182 @@ dmu_recv_begin(char *tofs, char *tosnap, dmu_replay_record_t *drr_begin, return (err); } +/* + * Holds data need for corrective recv callback + */ +typedef struct cr_cb_data { + uint64_t size; + zbookmark_phys_t zb; + spa_t *spa; +} cr_cb_data_t; + +static void +corrective_read_done(zio_t *zio) +{ + cr_cb_data_t *data = zio->io_private; + /* Corruption corrected; update error log if needed */ + if (zio->io_error == 0) + spa_remove_error(data->spa, &data->zb); + kmem_free(data, sizeof (cr_cb_data_t)); + abd_free(zio->io_abd); +} + +/* + * zio_rewrite the data pointed to by bp with the data from the rrd's abd. + */ +static int +do_corrective_recv(struct receive_writer_arg *rwa, struct drr_write *drrw, + struct receive_record_arg *rrd, blkptr_t *bp) +{ + int err; + zio_t *io; + zbookmark_phys_t zb; + dnode_t *dn; + abd_t *abd = rrd->abd; + zio_cksum_t bp_cksum = bp->blk_cksum; + enum zio_flag flags = ZIO_FLAG_SPECULATIVE | + ZIO_FLAG_DONT_CACHE | ZIO_FLAG_DONT_RETRY | ZIO_FLAG_CANFAIL; + + if (rwa->raw) + flags |= ZIO_FLAG_RAW; + + err = dnode_hold(rwa->os, drrw->drr_object, FTAG, &dn); + if (err != 0) + return (err); + SET_BOOKMARK(&zb, dmu_objset_id(rwa->os), drrw->drr_object, 0, + dbuf_whichblock(dn, 0, drrw->drr_offset)); + dnode_rele(dn, FTAG); + + if (!rwa->raw && DRR_WRITE_COMPRESSED(drrw)) { + /* Decompress the stream data */ + abd_t *dabd = abd_alloc_linear( + drrw->drr_logical_size, B_FALSE); + err = zio_decompress_data(drrw->drr_compressiontype, + abd, abd_to_buf(dabd), abd_get_size(abd), + abd_get_size(dabd), NULL); + + if (err != 0) { + abd_free(dabd); + return (err); + } + /* Swap in the newly decompressed data into the abd */ + abd_free(abd); + abd = dabd; + } + + if (!rwa->raw && BP_GET_COMPRESS(bp) != ZIO_COMPRESS_OFF) { + /* Recompress the data */ + abd_t *cabd = abd_alloc_linear(BP_GET_PSIZE(bp), + B_FALSE); + uint64_t csize = zio_compress_data(BP_GET_COMPRESS(bp), + abd, abd_to_buf(cabd), abd_get_size(abd), + rwa->os->os_complevel); + abd_zero_off(cabd, csize, BP_GET_PSIZE(bp) - csize); + /* Swap in newly compressed data into the abd */ + abd_free(abd); + abd = cabd; + flags |= ZIO_FLAG_RAW_COMPRESS; + } + + /* + * The stream is not encrypted but the data on-disk is. + * We need to re-encrypt the buf using the same + * encryption type, salt, iv, and mac that was used to encrypt + * the block previosly. + */ + if (!rwa->raw && BP_USES_CRYPT(bp)) { + dsl_dataset_t *ds; + dsl_crypto_key_t *dck = NULL; + uint8_t salt[ZIO_DATA_SALT_LEN]; + uint8_t iv[ZIO_DATA_IV_LEN]; + uint8_t mac[ZIO_DATA_MAC_LEN]; + boolean_t no_crypt = B_FALSE; + dsl_pool_t *dp = dmu_objset_pool(rwa->os); + abd_t *eabd = abd_alloc_linear(BP_GET_PSIZE(bp), B_FALSE); + + zio_crypt_decode_params_bp(bp, salt, iv); + zio_crypt_decode_mac_bp(bp, mac); + + dsl_pool_config_enter(dp, FTAG); + err = dsl_dataset_hold_flags(dp, rwa->tofs, + DS_HOLD_FLAG_DECRYPT, FTAG, &ds); + if (err != 0) { + dsl_pool_config_exit(dp, FTAG); + abd_free(eabd); + return (SET_ERROR(EACCES)); + } + + /* Look up the key from the spa's keystore */ + err = spa_keystore_lookup_key(rwa->os->os_spa, + zb.zb_objset, FTAG, &dck); + if (err != 0) { + dsl_dataset_rele_flags(ds, DS_HOLD_FLAG_DECRYPT, + FTAG); + dsl_pool_config_exit(dp, FTAG); + abd_free(eabd); + return (SET_ERROR(EACCES)); + } + + err = zio_do_crypt_abd(B_TRUE, &dck->dck_key, + BP_GET_TYPE(bp), BP_SHOULD_BYTESWAP(bp), salt, iv, + mac, abd_get_size(abd), abd, eabd, &no_crypt); + + spa_keystore_dsl_key_rele(rwa->os->os_spa, dck, FTAG); + dsl_dataset_rele_flags(ds, DS_HOLD_FLAG_DECRYPT, FTAG); + dsl_pool_config_exit(dp, FTAG); + + ASSERT0(no_crypt); + if (err != 0) { + abd_free(eabd); + return (err); + } + /* Swap in the newly encrypted data into the abd */ + abd_free(abd); + abd = eabd; + + /* + * We want to prevent zio_rewrite() from trying to + * encrypt the data again + */ + flags |= ZIO_FLAG_RAW_ENCRYPT; + } + rrd->abd = abd; + + io = zio_rewrite(NULL, rwa->os->os_spa, bp->blk_birth, bp, abd, + BP_GET_PSIZE(bp), NULL, NULL, ZIO_PRIORITY_SYNC_WRITE, flags, &zb); + + ASSERT(abd_get_size(abd) == BP_GET_LSIZE(bp) || + abd_get_size(abd) == BP_GET_PSIZE(bp)); + + /* compute new bp checksum value and make sure it matches the old one */ + zio_checksum_compute(io, BP_GET_CHECKSUM(bp), abd, abd_get_size(abd)); + if (!ZIO_CHECKSUM_EQUAL(bp_cksum, io->io_bp->blk_cksum)) { + zio_destroy(io); + if (zfs_recv_best_effort_corrective != 0) + return (0); + return (SET_ERROR(ECKSUM)); + } + + /* Correct the corruption in place */ + err = zio_wait(io); + if (err == 0) { + cr_cb_data_t *cb_data = + kmem_alloc(sizeof (cr_cb_data_t), KM_SLEEP); + cb_data->spa = rwa->os->os_spa; + cb_data->size = drrw->drr_logical_size; + cb_data->zb = zb; + /* Test if healing worked by re-reading the bp */ + err = zio_wait(zio_read(rwa->heal_pio, rwa->os->os_spa, bp, + abd_alloc_for_io(drrw->drr_logical_size, B_FALSE), + drrw->drr_logical_size, corrective_read_done, + cb_data, ZIO_PRIORITY_ASYNC_READ, flags, NULL)); + } + if (err != 0 && zfs_recv_best_effort_corrective != 0) + err = 0; + + return (err); +} + static int receive_read(dmu_recv_cookie_t *drc, int len, void *buf) { @@ -1808,7 +2040,7 @@ receive_object(struct receive_writer_arg *rwa, struct drr_object *drro, dmu_buf_will_dirty(db, tx); ASSERT3U(db->db_size, >=, drro->drr_bonuslen); - bcopy(data, db->db_data, DRR_OBJECT_PAYLOAD_SIZE(drro)); + memcpy(db->db_data, data, DRR_OBJECT_PAYLOAD_SIZE(drro)); /* * Raw bonus buffers have their byteorder determined by the @@ -1828,7 +2060,6 @@ receive_object(struct receive_writer_arg *rwa, struct drr_object *drro, return (0); } -/* ARGSUSED */ noinline static int receive_freeobjects(struct receive_writer_arg *rwa, struct drr_freeobjects *drrfo) @@ -1950,11 +2181,11 @@ flush_write_batch_impl(struct receive_writer_arg *rwa) zp.zp_byteorder = ZFS_HOST_BYTEORDER ^ !!DRR_IS_RAW_BYTESWAPPED(drrw->drr_flags) ^ rwa->byteswap; - bcopy(drrw->drr_salt, zp.zp_salt, + memcpy(zp.zp_salt, drrw->drr_salt, ZIO_DATA_SALT_LEN); - bcopy(drrw->drr_iv, zp.zp_iv, + memcpy(zp.zp_iv, drrw->drr_iv, ZIO_DATA_IV_LEN); - bcopy(drrw->drr_mac, zp.zp_mac, + memcpy(zp.zp_mac, drrw->drr_mac, ZIO_DATA_MAC_LEN); if (DMU_OT_IS_ENCRYPTED(zp.zp_type)) { zp.zp_nopwrite = B_FALSE; @@ -2051,6 +2282,58 @@ receive_process_write_record(struct receive_writer_arg *rwa, !DMU_OT_IS_VALID(drrw->drr_type)) return (SET_ERROR(EINVAL)); + if (rwa->heal) { + blkptr_t *bp; + dmu_buf_t *dbp; + dnode_t *dn; + int flags = DB_RF_CANFAIL; + + if (rwa->raw) + flags |= DB_RF_NO_DECRYPT; + + if (rwa->byteswap) { + dmu_object_byteswap_t byteswap = + DMU_OT_BYTESWAP(drrw->drr_type); + dmu_ot_byteswap[byteswap].ob_func(abd_to_buf(rrd->abd), + DRR_WRITE_PAYLOAD_SIZE(drrw)); + } + + err = dmu_buf_hold_noread(rwa->os, drrw->drr_object, + drrw->drr_offset, FTAG, &dbp); + if (err != 0) + return (err); + + /* Try to read the object to see if it needs healing */ + err = dbuf_read((dmu_buf_impl_t *)dbp, NULL, flags); + /* + * We only try to heal when dbuf_read() returns a ECKSUMs. + * Other errors (even EIO) get returned to caller. + * EIO indicates that the device is not present/accessible, + * so writing to it will likely fail. + * If the block is healthy, we don't want to overwrite it + * unnecessarily. + */ + if (err != ECKSUM) { + dmu_buf_rele(dbp, FTAG); + return (err); + } + dn = dmu_buf_dnode_enter(dbp); + /* Make sure the on-disk block and recv record sizes match */ + if (drrw->drr_logical_size != + dn->dn_datablkszsec << SPA_MINBLOCKSHIFT) { + err = ENOTSUP; + dmu_buf_dnode_exit(dbp); + dmu_buf_rele(dbp, FTAG); + return (err); + } + /* Get the block pointer for the corrupted block */ + bp = dmu_buf_get_blkptr(dbp); + err = do_corrective_recv(rwa, drrw, rrd, bp); + dmu_buf_dnode_exit(dbp); + dmu_buf_rele(dbp, FTAG); + return (err); + } + /* * For resuming to work, records must be in increasing order * by (object, offset). @@ -2219,7 +2502,7 @@ receive_spill(struct receive_writer_arg *rwa, struct drr_spill *drrs, } } - bcopy(abd_to_buf(abd), abuf->b_data, DRR_SPILL_PAYLOAD_SIZE(drrs)); + memcpy(abuf->b_data, abd_to_buf(abd), DRR_SPILL_PAYLOAD_SIZE(drrs)); abd_free(abd); dbuf_assign_arcbuf((dmu_buf_impl_t *)db_spill, abuf, tx); @@ -2230,7 +2513,6 @@ receive_spill(struct receive_writer_arg *rwa, struct drr_spill *drrs, return (0); } -/* ARGSUSED */ noinline static int receive_free(struct receive_writer_arg *rwa, struct drr_free *drrf) { @@ -2293,9 +2575,9 @@ receive_object_range(struct receive_writer_arg *rwa, rwa->or_crypt_params_present = B_TRUE; rwa->or_firstobj = drror->drr_firstobj; rwa->or_numslots = drror->drr_numslots; - bcopy(drror->drr_salt, rwa->or_salt, ZIO_DATA_SALT_LEN); - bcopy(drror->drr_iv, rwa->or_iv, ZIO_DATA_IV_LEN); - bcopy(drror->drr_mac, rwa->or_mac, ZIO_DATA_MAC_LEN); + memcpy(rwa->or_salt, drror->drr_salt, ZIO_DATA_SALT_LEN); + memcpy(rwa->or_iv, drror->drr_iv, ZIO_DATA_IV_LEN); + memcpy(rwa->or_mac, drror->drr_mac, ZIO_DATA_MAC_LEN); rwa->or_byteorder = byteorder; return (0); @@ -2305,7 +2587,6 @@ receive_object_range(struct receive_writer_arg *rwa, * Until we have the ability to redact large ranges of data efficiently, we * process these records as frees. */ -/* ARGSUSED */ noinline static int receive_redact(struct receive_writer_arg *rwa, struct drr_redact *drrr) { @@ -2345,7 +2626,8 @@ dmu_recv_cleanup_ds(dmu_recv_cookie_t *drc) rrw_exit(&ds->ds_bp_rwlock, FTAG); dsl_dataset_name(ds, name); dsl_dataset_disown(ds, dsflags, dmu_recv_tag); - (void) dsl_destroy_head(name); + if (!drc->drc_heal) + (void) dsl_destroy_head(name); } } @@ -2454,7 +2736,6 @@ receive_read_payload_and_next_header(dmu_recv_cookie_t *drc, int len, void *buf) * numbers in the ignore list. In practice, we receive up to 32 object records * before receiving write records, so the list can have up to 32 nodes in it. */ -/* ARGSUSED */ static void receive_read_prefetch(dmu_recv_cookie_t *drc, uint64_t object, uint64_t offset, uint64_t length) @@ -2707,7 +2988,19 @@ receive_process_record(struct receive_writer_arg *rwa, ASSERT3U(rrd->bytes_read, >=, rwa->bytes_read); rwa->bytes_read = rrd->bytes_read; - if (rrd->header.drr_type != DRR_WRITE) { + /* We can only heal write records; other ones get ignored */ + if (rwa->heal && rrd->header.drr_type != DRR_WRITE) { + if (rrd->abd != NULL) { + abd_free(rrd->abd); + rrd->abd = NULL; + } else if (rrd->payload != NULL) { + kmem_free(rrd->payload, rrd->payload_size); + rrd->payload = NULL; + } + return (0); + } + + if (!rwa->heal && rrd->header.drr_type != DRR_WRITE) { err = flush_write_batch(rwa); if (err != 0) { if (rrd->abd != NULL) { @@ -2742,9 +3035,16 @@ receive_process_record(struct receive_writer_arg *rwa, case DRR_WRITE: { err = receive_process_write_record(rwa, rrd); - if (err != EAGAIN) { + if (rwa->heal) { + /* + * If healing - always free the abd after processing + */ + abd_free(rrd->abd); + rrd->abd = NULL; + } else if (err != EAGAIN) { /* - * On success, receive_process_write_record() returns + * On success, a non-healing + * receive_process_write_record() returns * EAGAIN to indicate that we do not want to free * the rrd or arc_buf. */ @@ -2806,7 +3106,7 @@ receive_process_record(struct receive_writer_arg *rwa, * dmu_recv_stream's worker thread; pull records off the queue, and then call * receive_process_record When we're done, signal the main thread and exit. */ -static void +static __attribute__((noreturn)) void receive_writer_thread(void *arg) { struct receive_writer_arg *rwa = arg; @@ -2835,8 +3135,9 @@ receive_writer_thread(void *arg) * EAGAIN indicates that this record has been saved (on * raw->write_batch), and will be used again, so we don't * free it. + * When healing data we always need to free the record. */ - if (err != EAGAIN) { + if (err != EAGAIN || rwa->heal) { if (rwa->err == 0) rwa->err = err; kmem_free(rrd, sizeof (*rrd)); @@ -2844,10 +3145,13 @@ receive_writer_thread(void *arg) } kmem_free(rrd, sizeof (*rrd)); - int err = flush_write_batch(rwa); - if (rwa->err == 0) - rwa->err = err; - + if (rwa->heal) { + zio_wait(rwa->heal_pio); + } else { + int err = flush_write_batch(rwa); + if (rwa->err == 0) + rwa->err = err; + } mutex_enter(&rwa->mutex); rwa->done = B_TRUE; cv_signal(&rwa->cv); @@ -2931,17 +3235,19 @@ dmu_recv_stream(dmu_recv_cookie_t *drc, offset_t *voffp) if (err != 0) goto out; - /* - * If this is a new dataset we set the key immediately. - * Otherwise we don't want to change the key until we - * are sure the rest of the receive succeeded so we stash - * the keynvl away until then. - */ - err = dsl_crypto_recv_raw(spa_name(drc->drc_os->os_spa), - drc->drc_ds->ds_object, drc->drc_fromsnapobj, - drc->drc_drrb->drr_type, keynvl, drc->drc_newfs); - if (err != 0) - goto out; + if (!drc->drc_heal) { + /* + * If this is a new dataset we set the key immediately. + * Otherwise we don't want to change the key until we + * are sure the rest of the receive succeeded so we + * stash the keynvl away until then. + */ + err = dsl_crypto_recv_raw(spa_name(drc->drc_os->os_spa), + drc->drc_ds->ds_object, drc->drc_fromsnapobj, + drc->drc_drrb->drr_type, keynvl, drc->drc_newfs); + if (err != 0) + goto out; + } /* see comment in dmu_recv_end_sync() */ drc->drc_ivset_guid = 0; @@ -2972,11 +3278,17 @@ dmu_recv_stream(dmu_recv_cookie_t *drc, offset_t *voffp) mutex_init(&rwa->mutex, NULL, MUTEX_DEFAULT, NULL); rwa->os = drc->drc_os; rwa->byteswap = drc->drc_byteswap; + rwa->heal = drc->drc_heal; + rwa->tofs = drc->drc_tofs; rwa->resumable = drc->drc_resumable; rwa->raw = drc->drc_raw; rwa->spill = drc->drc_spill; rwa->full = (drc->drc_drr_begin->drr_u.drr_begin.drr_fromguid == 0); rwa->os->os_raw_receive = drc->drc_raw; + if (drc->drc_heal) { + rwa->heal_pio = zio_root(drc->drc_os->os_spa, NULL, NULL, + ZIO_FLAG_GODFATHER); + } list_create(&rwa->write_batch, sizeof (struct receive_record_arg), offsetof(struct receive_record_arg, node.bqn_node)); @@ -3112,7 +3424,9 @@ dmu_recv_end_check(void *arg, dmu_tx_t *tx) ASSERT3P(drc->drc_ds->ds_owner, ==, dmu_recv_tag); - if (!drc->drc_newfs) { + if (drc->drc_heal) { + error = 0; + } else if (!drc->drc_newfs) { dsl_dataset_t *origin_head; error = dsl_dataset_hold(dp, drc->drc_tofs, FTAG, &origin_head); @@ -3188,13 +3502,18 @@ dmu_recv_end_sync(void *arg, dmu_tx_t *tx) dmu_recv_cookie_t *drc = arg; dsl_pool_t *dp = dmu_tx_pool(tx); boolean_t encrypted = drc->drc_ds->ds_dir->dd_crypto_obj != 0; - uint64_t newsnapobj; + uint64_t newsnapobj = 0; spa_history_log_internal_ds(drc->drc_ds, "finish receiving", tx, "snap=%s", drc->drc_tosnap); drc->drc_ds->ds_objset->os_raw_receive = B_FALSE; - if (!drc->drc_newfs) { + if (drc->drc_heal) { + if (drc->drc_keynvl != NULL) { + nvlist_free(drc->drc_keynvl); + drc->drc_keynvl = NULL; + } + } else if (!drc->drc_newfs) { dsl_dataset_t *origin_head; VERIFY0(dsl_dataset_hold(dp, drc->drc_tofs, FTAG, @@ -3308,7 +3627,7 @@ dmu_recv_end_sync(void *arg, dmu_tx_t *tx) * tunable is set, in which case we will leave the newly-generated * value. */ - if (drc->drc_raw && drc->drc_ivset_guid != 0) { + if (!drc->drc_heal && drc->drc_raw && drc->drc_ivset_guid != 0) { dmu_object_zapify(dp->dp_meta_objset, newsnapobj, DMU_OT_DSL_DATASET, tx); VERIFY0(zap_update(dp->dp_meta_objset, newsnapobj, @@ -3375,7 +3694,7 @@ dmu_recv_end(dmu_recv_cookie_t *drc, void *owner) if (error != 0) { dmu_recv_cleanup_ds(drc); nvlist_free(drc->drc_keynvl); - } else { + } else if (!drc->drc_heal) { if (drc->drc_newfs) { zvol_create_minor(drc->drc_tofs); } @@ -3405,3 +3724,7 @@ ZFS_MODULE_PARAM(zfs_recv, zfs_recv_, queue_ff, INT, ZMOD_RW, ZFS_MODULE_PARAM(zfs_recv, zfs_recv_, write_batch_size, INT, ZMOD_RW, "Maximum amount of writes to batch into one transaction"); + +ZFS_MODULE_PARAM(zfs_recv, zfs_recv_, best_effort_corrective, INT, ZMOD_RW, + "Ignore errors during corrective receive"); +/* END CSTYLED */ diff --git a/module/zfs/dmu_redact.c b/module/zfs/dmu_redact.c index 46f4982894b5..09ca7d509ce7 100644 --- a/module/zfs/dmu_redact.c +++ b/module/zfs/dmu_redact.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -351,7 +351,7 @@ redact_cb(spa_t *spa, zilog_t *zilog, const blkptr_t *bp, return (0); } -static void +static __attribute__((noreturn)) void redact_traverse_thread(void *arg) { struct redact_thread_arg *rt_arg = arg; @@ -837,7 +837,7 @@ struct redact_merge_thread_arg { int error_code; }; -static void +static __attribute__((noreturn)) void redact_merge_thread(void *arg) { struct redact_merge_thread_arg *rmta = arg; @@ -855,7 +855,7 @@ redact_merge_thread(void *arg) * object number. */ static int -hold_next_object(objset_t *os, struct redact_record *rec, void *tag, +hold_next_object(objset_t *os, struct redact_record *rec, const void *tag, uint64_t *object, dnode_t **dn) { int err = 0; diff --git a/module/zfs/dmu_send.c b/module/zfs/dmu_send.c index b388a3c1101b..5e6ced2bb30a 100644 --- a/module/zfs/dmu_send.c +++ b/module/zfs/dmu_send.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -379,7 +379,7 @@ dump_free(dmu_send_cookie_t *dscp, uint64_t object, uint64_t offset, } } /* create a FREE record and make it pending */ - bzero(dscp->dsc_drr, sizeof (dmu_replay_record_t)); + memset(dscp->dsc_drr, 0, sizeof (dmu_replay_record_t)); dscp->dsc_drr->drr_type = DRR_FREE; drrf->drr_object = object; drrf->drr_offset = offset; @@ -438,7 +438,7 @@ dump_redact(dmu_send_cookie_t *dscp, uint64_t object, uint64_t offset, } } /* create a REDACT record and make it pending */ - bzero(dscp->dsc_drr, sizeof (dmu_replay_record_t)); + memset(dscp->dsc_drr, 0, sizeof (dmu_replay_record_t)); dscp->dsc_drr->drr_type = DRR_REDACT; drrr->drr_object = object; drrr->drr_offset = offset; @@ -480,7 +480,7 @@ dmu_dump_write(dmu_send_cookie_t *dscp, dmu_object_type_t type, uint64_t object, dscp->dsc_pending_op = PENDING_NONE; } /* write a WRITE record */ - bzero(dscp->dsc_drr, sizeof (dmu_replay_record_t)); + memset(dscp->dsc_drr, 0, sizeof (dmu_replay_record_t)); dscp->dsc_drr->drr_type = DRR_WRITE; drrw->drr_object = object; drrw->drr_type = type; @@ -493,6 +493,7 @@ dmu_dump_write(dmu_send_cookie_t *dscp, dmu_object_type_t type, uint64_t object, (bp != NULL ? BP_GET_COMPRESS(bp) != ZIO_COMPRESS_OFF && io_compressed : lsize != psize); if (raw || compressed) { + ASSERT(bp != NULL); ASSERT(raw || dscp->dsc_featureflags & DMU_BACKUP_FEATURE_COMPRESSED); ASSERT(!BP_IS_EMBEDDED(bp)); @@ -571,7 +572,7 @@ dump_write_embedded(dmu_send_cookie_t *dscp, uint64_t object, uint64_t offset, ASSERT(BP_IS_EMBEDDED(bp)); - bzero(dscp->dsc_drr, sizeof (dmu_replay_record_t)); + memset(dscp->dsc_drr, 0, sizeof (dmu_replay_record_t)); dscp->dsc_drr->drr_type = DRR_WRITE_EMBEDDED; drrw->drr_object = object; drrw->drr_offset = offset; @@ -604,7 +605,7 @@ dump_spill(dmu_send_cookie_t *dscp, const blkptr_t *bp, uint64_t object, } /* write a SPILL record */ - bzero(dscp->dsc_drr, sizeof (dmu_replay_record_t)); + memset(dscp->dsc_drr, 0, sizeof (dmu_replay_record_t)); dscp->dsc_drr->drr_type = DRR_SPILL; drrs->drr_object = object; drrs->drr_length = blksz; @@ -686,7 +687,7 @@ dump_freeobjects(dmu_send_cookie_t *dscp, uint64_t firstobj, uint64_t numobjs) } /* write a FREEOBJECTS record */ - bzero(dscp->dsc_drr, sizeof (dmu_replay_record_t)); + memset(dscp->dsc_drr, 0, sizeof (dmu_replay_record_t)); dscp->dsc_drr->drr_type = DRR_FREEOBJECTS; drrfo->drr_firstobj = firstobj; drrfo->drr_numobjs = numobjs; @@ -727,7 +728,7 @@ dump_dnode(dmu_send_cookie_t *dscp, const blkptr_t *bp, uint64_t object, } /* write an OBJECT record */ - bzero(dscp->dsc_drr, sizeof (dmu_replay_record_t)); + memset(dscp->dsc_drr, 0, sizeof (dmu_replay_record_t)); dscp->dsc_drr->drr_type = DRR_OBJECT; drro->drr_object = object; drro->drr_type = dnp->dn_type; @@ -801,7 +802,7 @@ dump_dnode(dmu_send_cookie_t *dscp, const blkptr_t *bp, uint64_t object, struct send_range record; blkptr_t *bp = DN_SPILL_BLKPTR(dnp); - bzero(&record, sizeof (struct send_range)); + memset(&record, 0, sizeof (struct send_range)); record.type = DATA; record.object = object; record.eos_marker = B_FALSE; @@ -841,7 +842,7 @@ dump_object_range(dmu_send_cookie_t *dscp, const blkptr_t *bp, dscp->dsc_pending_op = PENDING_NONE; } - bzero(dscp->dsc_drr, sizeof (dmu_replay_record_t)); + memset(dscp->dsc_drr, 0, sizeof (dmu_replay_record_t)); dscp->dsc_drr->drr_type = DRR_OBJECT_RANGE; drror->drr_firstobj = firstobj; drror->drr_numslots = numslots; @@ -1017,6 +1018,9 @@ do_dump(dmu_send_cookie_t *dscp, struct send_range *range) if (srdp->datablksz > SPA_OLD_MAXBLOCKSIZE && !(dscp->dsc_featureflags & DMU_BACKUP_FEATURE_LARGE_BLOCKS)) { + if (dscp->dsc_featureflags & DMU_BACKUP_FEATURE_RAW) + return (SET_ERROR(ENOTSUP)); + while (srdp->datablksz > 0 && err == 0) { int n = MIN(srdp->datablksz, SPA_OLD_MAXBLOCKSIZE); @@ -1136,7 +1140,7 @@ send_cb(spa_t *spa, zilog_t *zilog, const blkptr_t *bp, record->sru.object.bp = *bp; size_t size = sizeof (*dnp) * (dnp->dn_extra_slots + 1); record->sru.object.dnp = kmem_alloc(size, KM_SLEEP); - bcopy(dnp, record->sru.object.dnp, size); + memcpy(record->sru.object.dnp, dnp, size); bqueue_enqueue(&sta->q, record, sizeof (*record)); return (0); } @@ -1234,7 +1238,7 @@ redact_list_cb(redact_block_phys_t *rb, void *arg) * error code of the thread in case something goes wrong, and pushes the End of * Stream record when the traverse_dataset call has finished. */ -static void +static __attribute__((noreturn)) void send_traverse_thread(void *arg) { struct send_thread_arg *st_arg = arg; @@ -1324,7 +1328,7 @@ get_next_range(bqueue_t *bq, struct send_range *prev) return (next); } -static void +static __attribute__((noreturn)) void redact_list_thread(void *arg) { struct redact_list_thread_arg *rlt_arg = arg; @@ -1519,7 +1523,7 @@ find_next_range(struct send_range **ranges, bqueue_t **qs, uint64_t *out_mask) * data from the redact_list_thread and use that to determine which blocks * should be redacted. */ -static void +static __attribute__((noreturn)) void send_merge_thread(void *arg) { struct send_merge_thread_arg *smt_arg = arg; @@ -1744,7 +1748,7 @@ enqueue_range(struct send_reader_thread_arg *srta, bqueue_t *q, dnode_t *dn, * some indirect blocks can be discarded because they're not holes. Second, * it issues prefetches for the data we need to send. */ -static void +static __attribute__((noreturn)) void send_reader_thread(void *arg) { struct send_reader_thread_arg *srta = arg; @@ -1913,7 +1917,7 @@ send_reader_thread(void *arg) struct dmu_send_params { /* Pool args */ - void *tag; // Tag that dp was held with, will be used to release dp. + const void *tag; // Tag dp was held with, will be used to release dp. dsl_pool_t *dp; /* To snapshot args */ const char *tosnap; @@ -2362,7 +2366,7 @@ dmu_send_impl(struct dmu_send_params *dspp) dsl_dataset_t *to_ds = dspp->to_ds; zfs_bookmark_phys_t *ancestor_zb = &dspp->ancestor_zb; dsl_pool_t *dp = dspp->dp; - void *tag = dspp->tag; + const void *tag = dspp->tag; err = dmu_objset_from_ds(to_ds, &os); if (err != 0) { @@ -2597,7 +2601,7 @@ dmu_send_impl(struct dmu_send_params *dspp) * the receive side that the stream is incomplete. */ if (!dspp->savedok) { - bzero(drr, sizeof (dmu_replay_record_t)); + memset(drr, 0, sizeof (dmu_replay_record_t)); drr->drr_type = DRR_END; drr->drr_u.drr_end.drr_checksum = dsc.dsc_zc; drr->drr_u.drr_end.drr_toguid = dsc.dsc_toguid; @@ -2698,7 +2702,7 @@ dmu_send_obj(const char *pool, uint64_t tosnap, uint64_t fromsnap, uint64_t size = dspp.numfromredactsnaps * sizeof (uint64_t); dspp.fromredactsnaps = kmem_zalloc(size, KM_SLEEP); - bcopy(fromredact, dspp.fromredactsnaps, size); + memcpy(dspp.fromredactsnaps, fromredact, size); } boolean_t is_before = @@ -2883,7 +2887,7 @@ dmu_send(const char *tosnap, const char *fromsnap, boolean_t embedok, sizeof (uint64_t); dspp.fromredactsnaps = kmem_zalloc(size, KM_SLEEP); - bcopy(fromredact, dspp.fromredactsnaps, + memcpy(dspp.fromredactsnaps, fromredact, size); } if (!dsl_dataset_is_before(dspp.to_ds, fromds, diff --git a/module/zfs/dmu_traverse.c b/module/zfs/dmu_traverse.c index 8afcd776a9eb..44b3c9fc5c9d 100644 --- a/module/zfs/dmu_traverse.c +++ b/module/zfs/dmu_traverse.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -168,8 +168,8 @@ resume_skip_check(traverse_data_t *td, const dnode_phys_t *dnp, * If we found the block we're trying to resume from, zero * the bookmark out to indicate that we have resumed. */ - if (bcmp(zb, td->td_resume, sizeof (*zb)) == 0) { - bzero(td->td_resume, sizeof (*zb)); + if (memcmp(zb, td->td_resume, sizeof (*zb)) == 0) { + memset(td->td_resume, 0, sizeof (*zb)); if (td->td_flags & TRAVERSE_POST) return (RESUME_SKIP_CHILDREN); } diff --git a/module/zfs/dmu_tx.c b/module/zfs/dmu_tx.c index b4735bb7ff54..28f64369d8dd 100644 --- a/module/zfs/dmu_tx.c +++ b/module/zfs/dmu_tx.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -53,8 +53,8 @@ dmu_tx_stats_t dmu_tx_stats = { { "dmu_tx_dirty_throttle", KSTAT_DATA_UINT64 }, { "dmu_tx_dirty_delay", KSTAT_DATA_UINT64 }, { "dmu_tx_dirty_over_max", KSTAT_DATA_UINT64 }, - { "dmu_tx_wrlog_over_max", KSTAT_DATA_UINT64 }, { "dmu_tx_dirty_frees_delay", KSTAT_DATA_UINT64 }, + { "dmu_tx_wrlog_delay", KSTAT_DATA_UINT64 }, { "dmu_tx_quota", KSTAT_DATA_UINT64 }, }; @@ -219,7 +219,6 @@ dmu_tx_check_ioerr(zio_t *zio, dnode_t *dn, int level, uint64_t blkid) return (err); } -/* ARGSUSED */ static void dmu_tx_count_write(dmu_tx_hold_t *txh, uint64_t off, uint64_t len) { @@ -780,34 +779,49 @@ static void dmu_tx_delay(dmu_tx_t *tx, uint64_t dirty) { dsl_pool_t *dp = tx->tx_pool; - uint64_t delay_min_bytes = + uint64_t delay_min_bytes, wrlog; + hrtime_t wakeup, tx_time = 0, now; + + /* Calculate minimum transaction time for the dirty data amount. */ + delay_min_bytes = zfs_dirty_data_max * zfs_delay_min_dirty_percent / 100; - hrtime_t wakeup, min_tx_time, now; + if (dirty > delay_min_bytes) { + /* + * The caller has already waited until we are under the max. + * We make them pass us the amount of dirty data so we don't + * have to handle the case of it being >= the max, which + * could cause a divide-by-zero if it's == the max. + */ + ASSERT3U(dirty, <, zfs_dirty_data_max); - if (dirty <= delay_min_bytes) - return; + tx_time = zfs_delay_scale * (dirty - delay_min_bytes) / + (zfs_dirty_data_max - dirty); + } - /* - * The caller has already waited until we are under the max. - * We make them pass us the amount of dirty data so we don't - * have to handle the case of it being >= the max, which could - * cause a divide-by-zero if it's == the max. - */ - ASSERT3U(dirty, <, zfs_dirty_data_max); + /* Calculate minimum transaction time for the TX_WRITE log size. */ + wrlog = aggsum_upper_bound(&dp->dp_wrlog_total); + delay_min_bytes = + zfs_wrlog_data_max * zfs_delay_min_dirty_percent / 100; + if (wrlog >= zfs_wrlog_data_max) { + tx_time = zfs_delay_max_ns; + } else if (wrlog > delay_min_bytes) { + tx_time = MAX(zfs_delay_scale * (wrlog - delay_min_bytes) / + (zfs_wrlog_data_max - wrlog), tx_time); + } + if (tx_time == 0) + return; + + tx_time = MIN(tx_time, zfs_delay_max_ns); now = gethrtime(); - min_tx_time = zfs_delay_scale * - (dirty - delay_min_bytes) / (zfs_dirty_data_max - dirty); - min_tx_time = MIN(min_tx_time, zfs_delay_max_ns); - if (now > tx->tx_start + min_tx_time) + if (now > tx->tx_start + tx_time) return; DTRACE_PROBE3(delay__mintime, dmu_tx_t *, tx, uint64_t, dirty, - uint64_t, min_tx_time); + uint64_t, tx_time); mutex_enter(&dp->dp_lock); - wakeup = MAX(tx->tx_start + min_tx_time, - dp->dp_last_wakeup + min_tx_time); + wakeup = MAX(tx->tx_start + tx_time, dp->dp_last_wakeup + tx_time); dp->dp_last_wakeup = wakeup; mutex_exit(&dp->dp_lock); @@ -885,8 +899,9 @@ dmu_tx_try_assign(dmu_tx_t *tx, uint64_t txg_how) } if (!tx->tx_dirty_delayed && - dsl_pool_wrlog_over_max(tx->tx_pool)) { - DMU_TX_STAT_BUMP(dmu_tx_wrlog_over_max); + dsl_pool_need_wrlog_delay(tx->tx_pool)) { + tx->tx_wait_dirty = B_TRUE; + DMU_TX_STAT_BUMP(dmu_tx_wrlog_delay); return (SET_ERROR(ERESTART)); } diff --git a/module/zfs/dmu_zfetch.c b/module/zfs/dmu_zfetch.c index a6facdc65bb4..101d2ee7b7a2 100644 --- a/module/zfs/dmu_zfetch.c +++ b/module/zfs/dmu_zfetch.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -48,9 +48,13 @@ static int zfs_prefetch_disable = B_FALSE; /* max # of streams per zfetch */ static unsigned int zfetch_max_streams = 8; /* min time before stream reclaim */ -static unsigned int zfetch_min_sec_reap = 2; -/* max bytes to prefetch per stream (default 8MB) */ -unsigned int zfetch_max_distance = 8 * 1024 * 1024; +static unsigned int zfetch_min_sec_reap = 1; +/* max time before stream delete */ +static unsigned int zfetch_max_sec_reap = 2; +/* min bytes to prefetch per stream (default 4MB) */ +static unsigned int zfetch_min_distance = 4 * 1024 * 1024; +/* max bytes to prefetch per stream (default 64MB) */ +unsigned int zfetch_max_distance = 64 * 1024 * 1024; /* max bytes to prefetch indirects for per stream (default 64MB) */ unsigned int zfetch_max_idistance = 64 * 1024 * 1024; /* max number of bytes in an array_read in which we allow prefetching (1MB) */ @@ -195,74 +199,99 @@ dmu_zfetch_fini(zfetch_t *zf) } /* - * If there aren't too many streams already, create a new stream. + * If there aren't too many active streams already, create one more. + * In process delete/reuse all streams without hits for zfetch_max_sec_reap. + * If needed, reuse oldest stream without hits for zfetch_min_sec_reap or ever. * The "blkid" argument is the next block that we expect this stream to access. - * While we're here, clean up old streams (which haven't been - * accessed for at least zfetch_min_sec_reap seconds). */ static void dmu_zfetch_stream_create(zfetch_t *zf, uint64_t blkid) { - zstream_t *zs_next; - hrtime_t now = gethrtime(); + zstream_t *zs, *zs_next, *zs_old = NULL; + hrtime_t now = gethrtime(), t; ASSERT(MUTEX_HELD(&zf->zf_lock)); /* - * Clean up old streams. + * Delete too old streams, reusing the first found one. */ - for (zstream_t *zs = list_head(&zf->zf_stream); - zs != NULL; zs = zs_next) { + t = now - SEC2NSEC(zfetch_max_sec_reap); + for (zs = list_head(&zf->zf_stream); zs != NULL; zs = zs_next) { zs_next = list_next(&zf->zf_stream, zs); /* * Skip if still active. 1 -- zf_stream reference. */ if (zfs_refcount_count(&zs->zs_refs) != 1) continue; - if (((now - zs->zs_atime) / NANOSEC) > - zfetch_min_sec_reap) + if (zs->zs_atime > t) + continue; + if (zs_old) dmu_zfetch_stream_remove(zf, zs); + else + zs_old = zs; + } + if (zs_old) { + zs = zs_old; + goto reuse; } /* * The maximum number of streams is normally zfetch_max_streams, * but for small files we lower it such that it's at least possible * for all the streams to be non-overlapping. - * - * If we are already at the maximum number of streams for this file, - * even after removing old streams, then don't create this stream. */ uint32_t max_streams = MAX(1, MIN(zfetch_max_streams, zf->zf_dnode->dn_maxblkid * zf->zf_dnode->dn_datablksz / zfetch_max_distance)); if (zf->zf_numstreams >= max_streams) { + t = now - SEC2NSEC(zfetch_min_sec_reap); + for (zs = list_head(&zf->zf_stream); zs != NULL; + zs = list_next(&zf->zf_stream, zs)) { + if (zfs_refcount_count(&zs->zs_refs) != 1) + continue; + if (zs->zs_atime > t) + continue; + if (zs_old == NULL || zs->zs_atime < zs_old->zs_atime) + zs_old = zs; + } + if (zs_old) { + zs = zs_old; + goto reuse; + } ZFETCHSTAT_BUMP(zfetchstat_max_streams); return; } - zstream_t *zs = kmem_zalloc(sizeof (*zs), KM_SLEEP); - zs->zs_blkid = blkid; - zs->zs_pf_blkid1 = blkid; - zs->zs_pf_blkid = blkid; - zs->zs_ipf_blkid1 = blkid; - zs->zs_ipf_blkid = blkid; - zs->zs_atime = now; + zs = kmem_zalloc(sizeof (*zs), KM_SLEEP); zs->zs_fetch = zf; - zs->zs_missed = B_FALSE; zfs_refcount_create(&zs->zs_callers); zfs_refcount_create(&zs->zs_refs); /* One reference for zf_stream. */ zfs_refcount_add(&zs->zs_refs, NULL); zf->zf_numstreams++; list_insert_head(&zf->zf_stream, zs); + +reuse: + zs->zs_blkid = blkid; + zs->zs_pf_dist = 0; + zs->zs_pf_start = blkid; + zs->zs_pf_end = blkid; + zs->zs_ipf_dist = 0; + zs->zs_ipf_start = blkid; + zs->zs_ipf_end = blkid; + /* Allow immediate stream reuse until first hit. */ + zs->zs_atime = now - SEC2NSEC(zfetch_min_sec_reap); + zs->zs_missed = B_FALSE; + zs->zs_more = B_FALSE; } static void -dmu_zfetch_stream_done(void *arg, boolean_t io_issued) +dmu_zfetch_done(void *arg, uint64_t level, uint64_t blkid, boolean_t io_issued) { - (void) io_issued; zstream_t *zs = arg; + if (io_issued && level == 0 && blkid < zs->zs_blkid) + zs->zs_more = B_TRUE; if (zfs_refcount_remove(&zs->zs_refs, NULL) == 0) dmu_zfetch_stream_fini(zs); } @@ -284,11 +313,6 @@ dmu_zfetch_prepare(zfetch_t *zf, uint64_t blkid, uint64_t nblks, boolean_t fetch_data, boolean_t have_lock) { zstream_t *zs; - int64_t pf_start, ipf_start; - int64_t pf_ahead_blks, max_blks; - int max_dist_blks, pf_nblks, ipf_nblks; - uint64_t end_of_access_blkid, maxblkid; - end_of_access_blkid = blkid + nblks; spa_t *spa = zf->zf_dnode->dn_objset->os_spa; if (zfs_prefetch_disable) @@ -317,7 +341,7 @@ dmu_zfetch_prepare(zfetch_t *zf, uint64_t blkid, uint64_t nblks, * A fast path for small files for which no prefetch will * happen. */ - maxblkid = zf->zf_dnode->dn_maxblkid; + uint64_t maxblkid = zf->zf_dnode->dn_maxblkid; if (maxblkid < 2) { if (!have_lock) rw_exit(&zf->zf_dnode->dn_struct_rwlock); @@ -345,6 +369,7 @@ dmu_zfetch_prepare(zfetch_t *zf, uint64_t blkid, uint64_t nblks, * If the file is ending, remove the matching stream if found. * If not found then it is too late to create a new one now. */ + uint64_t end_of_access_blkid = blkid + nblks; if (end_of_access_blkid >= maxblkid) { if (zs != NULL) dmu_zfetch_stream_remove(zf, zs); @@ -377,60 +402,48 @@ dmu_zfetch_prepare(zfetch_t *zf, uint64_t blkid, uint64_t nblks, /* * This access was to a block that we issued a prefetch for on - * behalf of this stream. Issue further prefetches for this stream. + * behalf of this stream. Calculate further prefetch distances. * - * Normally, we start prefetching where we stopped - * prefetching last (zs_pf_blkid). But when we get our first - * hit on this stream, zs_pf_blkid == zs_blkid, we don't - * want to prefetch the block we just accessed. In this case, - * start just after the block we just accessed. - */ - pf_start = MAX(zs->zs_pf_blkid, end_of_access_blkid); - if (zs->zs_pf_blkid1 < end_of_access_blkid) - zs->zs_pf_blkid1 = end_of_access_blkid; - if (zs->zs_ipf_blkid1 < end_of_access_blkid) - zs->zs_ipf_blkid1 = end_of_access_blkid; - - /* - * Double our amount of prefetched data, but don't let the - * prefetch get further ahead than zfetch_max_distance. + * Start prefetch from the demand access size (nblks). Double the + * distance every access up to zfetch_min_distance. After that only + * if needed increase the distance by 1/8 up to zfetch_max_distance. */ + unsigned int nbytes = nblks << zf->zf_dnode->dn_datablkshift; + unsigned int pf_nblks; if (fetch_data) { - max_dist_blks = - zfetch_max_distance >> zf->zf_dnode->dn_datablkshift; - /* - * Previously, we were (zs_pf_blkid - blkid) ahead. We - * want to now be double that, so read that amount again, - * plus the amount we are catching up by (i.e. the amount - * read just now). - */ - pf_ahead_blks = zs->zs_pf_blkid - blkid + nblks; - max_blks = max_dist_blks - (pf_start - end_of_access_blkid); - pf_nblks = MIN(pf_ahead_blks, max_blks); + if (unlikely(zs->zs_pf_dist < nbytes)) + zs->zs_pf_dist = nbytes; + else if (zs->zs_pf_dist < zfetch_min_distance) + zs->zs_pf_dist *= 2; + else if (zs->zs_more) + zs->zs_pf_dist += zs->zs_pf_dist / 8; + zs->zs_more = B_FALSE; + if (zs->zs_pf_dist > zfetch_max_distance) + zs->zs_pf_dist = zfetch_max_distance; + pf_nblks = zs->zs_pf_dist >> zf->zf_dnode->dn_datablkshift; } else { pf_nblks = 0; } + if (zs->zs_pf_start < end_of_access_blkid) + zs->zs_pf_start = end_of_access_blkid; + if (zs->zs_pf_end < end_of_access_blkid + pf_nblks) + zs->zs_pf_end = end_of_access_blkid + pf_nblks; - zs->zs_pf_blkid = pf_start + pf_nblks; - - /* - * Do the same for indirects, starting from where we stopped last, - * or where we will stop reading data blocks (and the indirects - * that point to them). - */ - ipf_start = MAX(zs->zs_ipf_blkid, zs->zs_pf_blkid); - max_dist_blks = zfetch_max_idistance >> zf->zf_dnode->dn_datablkshift; /* - * We want to double our distance ahead of the data prefetch - * (or reader, if we are not prefetching data). Previously, we - * were (zs_ipf_blkid - blkid) ahead. To double that, we read - * that amount again, plus the amount we are catching up by - * (i.e. the amount read now + the amount of data prefetched now). + * Do the same for indirects, starting where we will stop reading + * data blocks (and the indirects that point to them). */ - pf_ahead_blks = zs->zs_ipf_blkid - blkid + nblks + pf_nblks; - max_blks = max_dist_blks - (ipf_start - zs->zs_pf_blkid); - ipf_nblks = MIN(pf_ahead_blks, max_blks); - zs->zs_ipf_blkid = ipf_start + ipf_nblks; + if (unlikely(zs->zs_ipf_dist < nbytes)) + zs->zs_ipf_dist = nbytes; + else + zs->zs_ipf_dist *= 2; + if (zs->zs_ipf_dist > zfetch_max_idistance) + zs->zs_ipf_dist = zfetch_max_idistance; + pf_nblks = zs->zs_ipf_dist >> zf->zf_dnode->dn_datablkshift; + if (zs->zs_ipf_start < zs->zs_pf_end) + zs->zs_ipf_start = zs->zs_pf_end; + if (zs->zs_ipf_end < zs->zs_pf_end + pf_nblks) + zs->zs_ipf_end = zs->zs_pf_end + pf_nblks; zs->zs_blkid = end_of_access_blkid; /* Protect the stream from reclamation. */ @@ -471,13 +484,13 @@ dmu_zfetch_run(zstream_t *zs, boolean_t missed, boolean_t have_lock) mutex_enter(&zf->zf_lock); if (zs->zs_missed) { - pf_start = zs->zs_pf_blkid1; - pf_end = zs->zs_pf_blkid1 = zs->zs_pf_blkid; + pf_start = zs->zs_pf_start; + pf_end = zs->zs_pf_start = zs->zs_pf_end; } else { pf_start = pf_end = 0; } - ipf_start = MAX(zs->zs_pf_blkid1, zs->zs_ipf_blkid1); - ipf_end = zs->zs_ipf_blkid1 = zs->zs_ipf_blkid; + ipf_start = zs->zs_ipf_start; + ipf_end = zs->zs_ipf_start = zs->zs_ipf_end; mutex_exit(&zf->zf_lock); ASSERT3S(pf_start, <=, pf_end); ASSERT3S(ipf_start, <=, ipf_end); @@ -505,12 +518,12 @@ dmu_zfetch_run(zstream_t *zs, boolean_t missed, boolean_t have_lock) for (int64_t blk = pf_start; blk < pf_end; blk++) { issued += dbuf_prefetch_impl(zf->zf_dnode, 0, blk, ZIO_PRIORITY_ASYNC_READ, ARC_FLAG_PREDICTIVE_PREFETCH, - dmu_zfetch_stream_done, zs); + dmu_zfetch_done, zs); } for (int64_t iblk = ipf_start; iblk < ipf_end; iblk++) { issued += dbuf_prefetch_impl(zf->zf_dnode, 1, iblk, ZIO_PRIORITY_ASYNC_READ, ARC_FLAG_PREDICTIVE_PREFETCH, - dmu_zfetch_stream_done, zs); + dmu_zfetch_done, zs); } if (!have_lock) @@ -540,6 +553,12 @@ ZFS_MODULE_PARAM(zfs_prefetch, zfetch_, max_streams, UINT, ZMOD_RW, ZFS_MODULE_PARAM(zfs_prefetch, zfetch_, min_sec_reap, UINT, ZMOD_RW, "Min time before stream reclaim"); +ZFS_MODULE_PARAM(zfs_prefetch, zfetch_, max_sec_reap, UINT, ZMOD_RW, + "Max time before stream delete"); + +ZFS_MODULE_PARAM(zfs_prefetch, zfetch_, min_distance, UINT, ZMOD_RW, + "Min bytes to prefetch per stream"); + ZFS_MODULE_PARAM(zfs_prefetch, zfetch_, max_distance, UINT, ZMOD_RW, "Max bytes to prefetch per stream"); diff --git a/module/zfs/dnode.c b/module/zfs/dnode.c index 7b53b7cd0534..67fe1e2c9a0f 100644 --- a/module/zfs/dnode.c +++ b/module/zfs/dnode.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -128,15 +128,15 @@ dnode_cons(void *arg, void *unused, int kmflag) zfs_refcount_create(&dn->dn_tx_holds); list_link_init(&dn->dn_link); - bzero(&dn->dn_next_type[0], sizeof (dn->dn_next_type)); - bzero(&dn->dn_next_nblkptr[0], sizeof (dn->dn_next_nblkptr)); - bzero(&dn->dn_next_nlevels[0], sizeof (dn->dn_next_nlevels)); - bzero(&dn->dn_next_indblkshift[0], sizeof (dn->dn_next_indblkshift)); - bzero(&dn->dn_next_bonustype[0], sizeof (dn->dn_next_bonustype)); - bzero(&dn->dn_rm_spillblk[0], sizeof (dn->dn_rm_spillblk)); - bzero(&dn->dn_next_bonuslen[0], sizeof (dn->dn_next_bonuslen)); - bzero(&dn->dn_next_blksz[0], sizeof (dn->dn_next_blksz)); - bzero(&dn->dn_next_maxblkid[0], sizeof (dn->dn_next_maxblkid)); + memset(dn->dn_next_type, 0, sizeof (dn->dn_next_type)); + memset(dn->dn_next_nblkptr, 0, sizeof (dn->dn_next_nblkptr)); + memset(dn->dn_next_nlevels, 0, sizeof (dn->dn_next_nlevels)); + memset(dn->dn_next_indblkshift, 0, sizeof (dn->dn_next_indblkshift)); + memset(dn->dn_next_bonustype, 0, sizeof (dn->dn_next_bonustype)); + memset(dn->dn_rm_spillblk, 0, sizeof (dn->dn_rm_spillblk)); + memset(dn->dn_next_bonuslen, 0, sizeof (dn->dn_next_bonuslen)); + memset(dn->dn_next_blksz, 0, sizeof (dn->dn_next_blksz)); + memset(dn->dn_next_maxblkid, 0, sizeof (dn->dn_next_maxblkid)); for (int i = 0; i < TXG_SIZE; i++) { multilist_link_init(&dn->dn_dirty_link[i]); @@ -317,7 +317,7 @@ dnode_byteswap(dnode_phys_t *dnp) int i; if (dnp->dn_type == DMU_OT_NONE) { - bzero(dnp, sizeof (dnode_phys_t)); + memset(dnp, 0, sizeof (dnode_phys_t)); return; } @@ -342,20 +342,11 @@ dnode_byteswap(dnode_phys_t *dnp) * dnode dnode is smaller than a regular dnode. */ if (dnp->dn_bonuslen != 0) { - /* - * Note that the bonus length calculated here may be - * longer than the actual bonus buffer. This is because - * we always put the bonus buffer after the last block - * pointer (instead of packing it against the end of the - * dnode buffer). - */ - int off = (dnp->dn_nblkptr-1) * sizeof (blkptr_t); - int slots = dnp->dn_extra_slots + 1; - size_t len = DN_SLOTS_TO_BONUSLEN(slots) - off; dmu_object_byteswap_t byteswap; ASSERT(DMU_OT_IS_VALID(dnp->dn_bonustype)); byteswap = DMU_OT_BYTESWAP(dnp->dn_bonustype); - dmu_ot_byteswap[byteswap].ob_func(dnp->dn_bonus + off, len); + dmu_ot_byteswap[byteswap].ob_func(DN_BONUS(dnp), + DN_MAX_BONUS_LEN(dnp)); } /* Swap SPILL block if we have one */ @@ -395,7 +386,7 @@ dnode_setbonuslen(dnode_t *dn, int newsize, dmu_tx_t *tx) /* clear any data after the end of the new size */ size_t diff = dn->dn_bonuslen - newsize; char *data_end = ((char *)dn->dn_bonus->db.db_data) + newsize; - bzero(data_end, diff); + memset(data_end, 0, diff); } dn->dn_bonuslen = newsize; @@ -596,7 +587,7 @@ dnode_allocate(dnode_t *dn, dmu_object_type_t ot, int blocksize, int ibs, DNODE_STAT_BUMP(dnode_allocate); ASSERT(dn->dn_type == DMU_OT_NONE); - ASSERT(bcmp(dn->dn_phys, &dnode_phys_zero, sizeof (dnode_phys_t)) == 0); + ASSERT0(memcmp(dn->dn_phys, &dnode_phys_zero, sizeof (dnode_phys_t))); ASSERT(dn->dn_phys->dn_type == DMU_OT_NONE); ASSERT(ot != DMU_OT_NONE); ASSERT(DMU_OT_IS_VALID(ot)); @@ -749,8 +740,6 @@ dnode_reallocate(dnode_t *dn, dmu_object_type_t ot, int blocksize, static void dnode_move_impl(dnode_t *odn, dnode_t *ndn) { - int i; - ASSERT(!RW_LOCK_HELD(&odn->dn_struct_rwlock)); ASSERT(MUTEX_NOT_HELD(&odn->dn_mtx)); ASSERT(MUTEX_NOT_HELD(&odn->dn_dbufs_mtx)); @@ -774,29 +763,29 @@ dnode_move_impl(dnode_t *odn, dnode_t *ndn) ndn->dn_datablksz = odn->dn_datablksz; ndn->dn_maxblkid = odn->dn_maxblkid; ndn->dn_num_slots = odn->dn_num_slots; - bcopy(&odn->dn_next_type[0], &ndn->dn_next_type[0], + memcpy(ndn->dn_next_type, odn->dn_next_type, sizeof (odn->dn_next_type)); - bcopy(&odn->dn_next_nblkptr[0], &ndn->dn_next_nblkptr[0], + memcpy(ndn->dn_next_nblkptr, odn->dn_next_nblkptr, sizeof (odn->dn_next_nblkptr)); - bcopy(&odn->dn_next_nlevels[0], &ndn->dn_next_nlevels[0], + memcpy(ndn->dn_next_nlevels, odn->dn_next_nlevels, sizeof (odn->dn_next_nlevels)); - bcopy(&odn->dn_next_indblkshift[0], &ndn->dn_next_indblkshift[0], + memcpy(ndn->dn_next_indblkshift, odn->dn_next_indblkshift, sizeof (odn->dn_next_indblkshift)); - bcopy(&odn->dn_next_bonustype[0], &ndn->dn_next_bonustype[0], + memcpy(ndn->dn_next_bonustype, odn->dn_next_bonustype, sizeof (odn->dn_next_bonustype)); - bcopy(&odn->dn_rm_spillblk[0], &ndn->dn_rm_spillblk[0], + memcpy(ndn->dn_rm_spillblk, odn->dn_rm_spillblk, sizeof (odn->dn_rm_spillblk)); - bcopy(&odn->dn_next_bonuslen[0], &ndn->dn_next_bonuslen[0], + memcpy(ndn->dn_next_bonuslen, odn->dn_next_bonuslen, sizeof (odn->dn_next_bonuslen)); - bcopy(&odn->dn_next_blksz[0], &ndn->dn_next_blksz[0], + memcpy(ndn->dn_next_blksz, odn->dn_next_blksz, sizeof (odn->dn_next_blksz)); - bcopy(&odn->dn_next_maxblkid[0], &ndn->dn_next_maxblkid[0], + memcpy(ndn->dn_next_maxblkid, odn->dn_next_maxblkid, sizeof (odn->dn_next_maxblkid)); - for (i = 0; i < TXG_SIZE; i++) { + for (int i = 0; i < TXG_SIZE; i++) { list_move_tail(&ndn->dn_dirty_records[i], &odn->dn_dirty_records[i]); } - bcopy(&odn->dn_free_ranges[0], &ndn->dn_free_ranges[0], + memcpy(ndn->dn_free_ranges, odn->dn_free_ranges, sizeof (odn->dn_free_ranges)); ndn->dn_allocated_txg = odn->dn_allocated_txg; ndn->dn_free_txg = odn->dn_free_txg; @@ -850,7 +839,7 @@ dnode_move_impl(dnode_t *odn, dnode_t *ndn) /* * Satisfy the destructor. */ - for (i = 0; i < TXG_SIZE; i++) { + for (int i = 0; i < TXG_SIZE; i++) { list_create(&odn->dn_dirty_records[i], sizeof (dbuf_dirty_record_t), offsetof(dbuf_dirty_record_t, dr_dirty_node)); @@ -1270,7 +1259,7 @@ dnode_buf_evict_async(void *dbu) */ int dnode_hold_impl(objset_t *os, uint64_t object, int flag, int slots, - void *tag, dnode_t **dnp) + const void *tag, dnode_t **dnp) { int epb, idx, err; int drop_struct_lock = FALSE; @@ -1564,7 +1553,7 @@ dnode_hold_impl(objset_t *os, uint64_t object, int flag, int slots, * Return held dnode if the object is allocated, NULL if not. */ int -dnode_hold(objset_t *os, uint64_t object, void *tag, dnode_t **dnp) +dnode_hold(objset_t *os, uint64_t object, const void *tag, dnode_t **dnp) { return (dnode_hold_impl(os, object, DNODE_MUST_BE_ALLOCATED, 0, tag, dnp)); @@ -1576,7 +1565,7 @@ dnode_hold(objset_t *os, uint64_t object, void *tag, dnode_t **dnp) * new reference. */ boolean_t -dnode_add_ref(dnode_t *dn, void *tag) +dnode_add_ref(dnode_t *dn, const void *tag) { mutex_enter(&dn->dn_mtx); if (zfs_refcount_is_zero(&dn->dn_holds)) { @@ -1589,14 +1578,14 @@ dnode_add_ref(dnode_t *dn, void *tag) } void -dnode_rele(dnode_t *dn, void *tag) +dnode_rele(dnode_t *dn, const void *tag) { mutex_enter(&dn->dn_mtx); dnode_rele_and_unlock(dn, tag, B_FALSE); } void -dnode_rele_and_unlock(dnode_t *dn, void *tag, boolean_t evicting) +dnode_rele_and_unlock(dnode_t *dn, const void *tag, boolean_t evicting) { uint64_t refs; /* Get while the hold prevents the dnode from moving. */ @@ -2031,7 +2020,7 @@ dnode_dirty_l1range(dnode_t *dn, uint64_t start_blkid, uint64_t end_blkid, } void -dnode_set_dirtyctx(dnode_t *dn, dmu_tx_t *tx, void *tag) +dnode_set_dirtyctx(dnode_t *dn, dmu_tx_t *tx, const void *tag) { /* * Don't set dirtyctx to SYNC if we're just modifying this as we @@ -2081,7 +2070,7 @@ dnode_partial_zero(dnode_t *dn, uint64_t off, uint64_t blkoff, uint64_t len, dmu_buf_will_dirty(&db->db, tx); data = db->db.db_data; - bzero(data + blkoff, len); + memset(data + blkoff, 0, len); } dbuf_rele(db, FTAG); } diff --git a/module/zfs/dnode_sync.c b/module/zfs/dnode_sync.c index 12ab4bea145f..5eabfb833ef0 100644 --- a/module/zfs/dnode_sync.c +++ b/module/zfs/dnode_sync.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -82,7 +82,7 @@ dnode_increase_indirection(dnode_t *dn, dmu_tx_t *tx) ASSERT(db->db.db_data); ASSERT(arc_released(db->db_buf)); ASSERT3U(sizeof (blkptr_t) * nblkptr, <=, db->db.db_size); - bcopy(dn->dn_phys->dn_blkptr, db->db.db_data, + memcpy(db->db.db_data, dn->dn_phys->dn_blkptr, sizeof (blkptr_t) * nblkptr); arc_buf_freeze(db->db_buf); @@ -119,7 +119,7 @@ dnode_increase_indirection(dnode_t *dn, dmu_tx_t *tx) mutex_exit(&child->db_mtx); } - bzero(dn->dn_phys->dn_blkptr, sizeof (blkptr_t) * nblkptr); + memset(dn->dn_phys->dn_blkptr, 0, sizeof (blkptr_t) * nblkptr); rw_exit(&db->db_rwlock); if (dn->dn_dbuf != NULL) @@ -158,7 +158,7 @@ free_blocks(dnode_t *dn, blkptr_t *bp, int num, dmu_tx_t *tx) dmu_object_type_t type = BP_GET_TYPE(bp); uint64_t lvl = BP_GET_LEVEL(bp); - bzero(bp, sizeof (blkptr_t)); + memset(bp, 0, sizeof (blkptr_t)); if (spa_feature_is_active(dn->dn_objset->os_spa, SPA_FEATURE_HOLE_BIRTH)) { @@ -347,7 +347,7 @@ free_children(dmu_buf_impl_t *db, uint64_t blkid, uint64_t nblks, rw_enter(&db->db_rwlock, RW_WRITER); for (i = 0, bp = db->db.db_data; i < 1 << epbs; i++, bp++) ASSERT(BP_IS_HOLE(bp)); - bzero(db->db.db_data, db->db.db_size); + memset(db->db.db_data, 0, db->db.db_size); free_blocks(dn, db->db_blkptr, 1, tx); rw_exit(&db->db_rwlock); } @@ -597,7 +597,7 @@ dnode_sync_free(dnode_t *dn, dmu_tx_t *tx) ASSERT(dn->dn_free_txg > 0); if (dn->dn_allocated_txg != dn->dn_free_txg) dmu_buf_will_dirty(&dn->dn_dbuf->db, tx); - bzero(dn->dn_phys, sizeof (dnode_phys_t) * dn->dn_num_slots); + memset(dn->dn_phys, 0, sizeof (dnode_phys_t) * dn->dn_num_slots); dnode_free_interior_slots(dn); mutex_enter(&dn->dn_mtx); @@ -634,7 +634,7 @@ dnode_sync(dnode_t *dn, dmu_tx_t *tx) ASSERT(dmu_tx_is_syncing(tx)); ASSERT(dnp->dn_type != DMU_OT_NONE || dn->dn_allocated_txg); ASSERT(dnp->dn_type != DMU_OT_NONE || - bcmp(dnp, &zerodn, DNODE_MIN_SIZE) == 0); + memcmp(dnp, &zerodn, DNODE_MIN_SIZE) == 0); DNODE_VERIFY(dn); ASSERT(dn->dn_dbuf == NULL || arc_released(dn->dn_dbuf->db_buf)); @@ -827,7 +827,7 @@ dnode_sync(dnode_t *dn, dmu_tx_t *tx) ASSERT(dn->dn_allocated_txg == tx->tx_txg); if (dn->dn_next_nblkptr[txgoff] > dnp->dn_nblkptr) { /* zero the new blkptrs we are gaining */ - bzero(dnp->dn_blkptr + dnp->dn_nblkptr, + memset(dnp->dn_blkptr + dnp->dn_nblkptr, 0, sizeof (blkptr_t) * (dn->dn_next_nblkptr[txgoff] - dnp->dn_nblkptr)); #ifdef ZFS_DEBUG diff --git a/module/zfs/dsl_bookmark.c b/module/zfs/dsl_bookmark.c index b8e3523ffc2d..8ca7ba8957aa 100644 --- a/module/zfs/dsl_bookmark.c +++ b/module/zfs/dsl_bookmark.c @@ -37,7 +37,7 @@ static int dsl_bookmark_hold_ds(dsl_pool_t *dp, const char *fullname, - dsl_dataset_t **dsp, void *tag, char **shortnamep) + dsl_dataset_t **dsp, const void *tag, char **shortnamep) { char buf[ZFS_MAX_DATASET_NAME_LEN]; char *hashp; @@ -82,7 +82,7 @@ dsl_bookmark_lookup_impl(dsl_dataset_t *ds, const char *shortname, * Zero out the bookmark in case the one stored on disk * is in an older, shorter format. */ - bzero(bmark_phys, sizeof (*bmark_phys)); + memset(bmark_phys, 0, sizeof (*bmark_phys)); err = zap_lookup_norm(mos, bmark_zapobj, shortname, sizeof (uint64_t), sizeof (*bmark_phys) / sizeof (uint64_t), bmark_phys, mt, NULL, 0, @@ -160,10 +160,9 @@ dsl_bookmark_create_nvl_validate_pair(const char *bmark, const char *source) int dsl_bookmark_create_nvl_validate(nvlist_t *bmarks) { - char *first; - size_t first_len; + char *first = NULL; + size_t first_len = 0; - first = NULL; for (nvpair_t *pair = nvlist_next_nvpair(bmarks, NULL); pair != NULL; pair = nvlist_next_nvpair(bmarks, pair)) { @@ -347,6 +346,8 @@ dsl_bookmark_set_phys(zfs_bookmark_phys_t *zbm, dsl_dataset_t *snap) spa_t *spa = dsl_dataset_get_spa(snap); objset_t *mos = spa_get_dsl(spa)->dp_meta_objset; dsl_dataset_phys_t *dsp = dsl_dataset_phys(snap); + + memset(zbm, 0, sizeof (zfs_bookmark_phys_t)); zbm->zbm_guid = dsp->ds_guid; zbm->zbm_creation_txg = dsp->ds_creation_txg; zbm->zbm_creation_time = dsp->ds_creation_time; @@ -380,10 +381,6 @@ dsl_bookmark_set_phys(zfs_bookmark_phys_t *zbm, dsl_dataset_t *snap) &zbm->zbm_compressed_freed_before_next_snap, &zbm->zbm_uncompressed_freed_before_next_snap); dsl_dataset_rele(nextds, FTAG); - } else { - bzero(&zbm->zbm_flags, - sizeof (zfs_bookmark_phys_t) - - offsetof(zfs_bookmark_phys_t, zbm_flags)); } } @@ -426,8 +423,8 @@ dsl_bookmark_node_add(dsl_dataset_t *hds, dsl_bookmark_node_t *dbn, spa_feature_incr(dp->dp_spa, SPA_FEATURE_BOOKMARK_V2, tx); } - __attribute__((unused)) zfs_bookmark_phys_t zero_phys = { 0 }; - ASSERT0(bcmp(((char *)&dbn->dbn_phys) + bookmark_phys_size, + zfs_bookmark_phys_t zero_phys = { 0 }; + ASSERT0(memcmp(((char *)&dbn->dbn_phys) + bookmark_phys_size, &zero_phys, sizeof (zfs_bookmark_phys_t) - bookmark_phys_size)); VERIFY0(zap_add(mos, hds->ds_bookmarks_obj, dbn->dbn_name, @@ -441,8 +438,8 @@ dsl_bookmark_node_add(dsl_dataset_t *hds, dsl_bookmark_node_t *dbn, */ static void dsl_bookmark_create_sync_impl_snap(const char *bookmark, const char *snapshot, - dmu_tx_t *tx, uint64_t num_redact_snaps, uint64_t *redact_snaps, void *tag, - redaction_list_t **redaction_list) + dmu_tx_t *tx, uint64_t num_redact_snaps, uint64_t *redact_snaps, + const void *tag, redaction_list_t **redaction_list) { dsl_pool_t *dp = dmu_tx_pool(tx); objset_t *mos = dp->dp_meta_objset; @@ -482,7 +479,7 @@ dsl_bookmark_create_sync_impl_snap(const char *bookmark, const char *snapshot, sizeof (redaction_list_phys_t) + num_redact_snaps * sizeof (uint64_t)); dmu_buf_will_dirty(local_rl->rl_dbuf, tx); - bcopy(redact_snaps, local_rl->rl_phys->rlp_snaps, + memcpy(local_rl->rl_phys->rlp_snaps, redact_snaps, sizeof (uint64_t) * num_redact_snaps); local_rl->rl_phys->rlp_num_snaps = num_redact_snaps; if (bookmark_redacted) { @@ -667,7 +664,8 @@ dsl_bookmark_create_redacted_sync(void *arg, dmu_tx_t *tx) int dsl_bookmark_create_redacted(const char *bookmark, const char *snapshot, - uint64_t numsnaps, uint64_t *snapguids, void *tag, redaction_list_t **rl) + uint64_t numsnaps, uint64_t *snapguids, const void *tag, + redaction_list_t **rl) { dsl_bookmark_create_redacted_arg_t dbcra; @@ -1191,14 +1189,15 @@ dsl_redaction_list_long_held(redaction_list_t *rl) } void -dsl_redaction_list_long_hold(dsl_pool_t *dp, redaction_list_t *rl, void *tag) +dsl_redaction_list_long_hold(dsl_pool_t *dp, redaction_list_t *rl, + const void *tag) { ASSERT(dsl_pool_config_held(dp)); (void) zfs_refcount_add(&rl->rl_longholds, tag); } void -dsl_redaction_list_long_rele(redaction_list_t *rl, void *tag) +dsl_redaction_list_long_rele(redaction_list_t *rl, const void *tag) { (void) zfs_refcount_remove(&rl->rl_longholds, tag); } @@ -1213,13 +1212,13 @@ redaction_list_evict_sync(void *rlu) } void -dsl_redaction_list_rele(redaction_list_t *rl, void *tag) +dsl_redaction_list_rele(redaction_list_t *rl, const void *tag) { dmu_buf_rele(rl->rl_dbuf, tag); } int -dsl_redaction_list_hold_obj(dsl_pool_t *dp, uint64_t rlobj, void *tag, +dsl_redaction_list_hold_obj(dsl_pool_t *dp, uint64_t rlobj, const void *tag, redaction_list_t **rlp) { objset_t *mos = dp->dp_meta_objset; @@ -1294,7 +1293,7 @@ dsl_bookmark_ds_destroyed(dsl_dataset_t *ds, dmu_tx_t *tx) * The empty-string name can't be in the AVL, and it compares * before any entries with this TXG. */ - search.dbn_name = ""; + search.dbn_name = (char *)""; VERIFY3P(avl_find(&head->ds_bookmarks, &search, &idx), ==, NULL); dsl_bookmark_node_t *dbn = avl_nearest(&head->ds_bookmarks, idx, AVL_AFTER); @@ -1421,7 +1420,7 @@ dsl_bookmark_next_changed(dsl_dataset_t *head, dsl_dataset_t *origin, * The empty-string name can't be in the AVL, and it compares * before any entries with this TXG. */ - search.dbn_name = ""; + search.dbn_name = (char *)""; VERIFY3P(avl_find(&head->ds_bookmarks, &search, &idx), ==, NULL); dsl_bookmark_node_t *dbn = avl_nearest(&head->ds_bookmarks, idx, AVL_AFTER); diff --git a/module/zfs/dsl_crypt.c b/module/zfs/dsl_crypt.c index 1ea184de338c..25cb4f6ab1ad 100644 --- a/module/zfs/dsl_crypt.c +++ b/module/zfs/dsl_crypt.c @@ -80,13 +80,13 @@ int zfs_disable_ivset_guid_check = 0; static void -dsl_wrapping_key_hold(dsl_wrapping_key_t *wkey, void *tag) +dsl_wrapping_key_hold(dsl_wrapping_key_t *wkey, const void *tag) { (void) zfs_refcount_add(&wkey->wk_refcnt, tag); } static void -dsl_wrapping_key_rele(dsl_wrapping_key_t *wkey, void *tag) +dsl_wrapping_key_rele(dsl_wrapping_key_t *wkey, const void *tag) { (void) zfs_refcount_remove(&wkey->wk_refcnt, tag); } @@ -97,7 +97,7 @@ dsl_wrapping_key_free(dsl_wrapping_key_t *wkey) ASSERT0(zfs_refcount_count(&wkey->wk_refcnt)); if (wkey->wk_key.ck_data) { - bzero(wkey->wk_key.ck_data, + memset(wkey->wk_key.ck_data, 0, CRYPTO_BITS2BYTES(wkey->wk_key.ck_length)); kmem_free(wkey->wk_key.ck_data, CRYPTO_BITS2BYTES(wkey->wk_key.ck_length)); @@ -119,9 +119,8 @@ dsl_wrapping_key_create(uint8_t *wkeydata, zfs_keyformat_t keyformat, /* allocate and initialize the underlying crypto key */ wkey->wk_key.ck_data = kmem_alloc(WRAPPING_KEY_LEN, KM_SLEEP); - wkey->wk_key.ck_format = CRYPTO_KEY_RAW; wkey->wk_key.ck_length = CRYPTO_BYTES2BITS(WRAPPING_KEY_LEN); - bcopy(wkeydata, wkey->wk_key.ck_data, WRAPPING_KEY_LEN); + memcpy(wkey->wk_key.ck_data, wkeydata, WRAPPING_KEY_LEN); /* initialize the rest of the struct */ zfs_refcount_create(&wkey->wk_refcnt); @@ -369,7 +368,7 @@ dsl_dir_incompatible_encryption_version(dsl_dir_t *dd) static int spa_keystore_wkey_hold_ddobj_impl(spa_t *spa, uint64_t ddobj, - void *tag, dsl_wrapping_key_t **wkey_out) + const void *tag, dsl_wrapping_key_t **wkey_out) { int ret; dsl_wrapping_key_t search_wkey; @@ -399,7 +398,7 @@ spa_keystore_wkey_hold_ddobj_impl(spa_t *spa, uint64_t ddobj, } static int -spa_keystore_wkey_hold_dd(spa_t *spa, dsl_dir_t *dd, void *tag, +spa_keystore_wkey_hold_dd(spa_t *spa, dsl_dir_t *dd, const void *tag, dsl_wrapping_key_t **wkey_out) { int ret; @@ -515,7 +514,7 @@ dsl_crypto_key_free(dsl_crypto_key_t *dck) } static void -dsl_crypto_key_rele(dsl_crypto_key_t *dck, void *tag) +dsl_crypto_key_rele(dsl_crypto_key_t *dck, const void *tag) { if (zfs_refcount_remove(&dck->dck_holds, tag) == 0) dsl_crypto_key_free(dck); @@ -523,7 +522,7 @@ dsl_crypto_key_rele(dsl_crypto_key_t *dck, void *tag) static int dsl_crypto_key_open(objset_t *mos, dsl_wrapping_key_t *wkey, - uint64_t dckobj, void *tag, dsl_crypto_key_t **dck_out) + uint64_t dckobj, const void *tag, dsl_crypto_key_t **dck_out) { int ret; uint64_t crypt = 0, guid = 0, version = 0; @@ -592,7 +591,7 @@ dsl_crypto_key_open(objset_t *mos, dsl_wrapping_key_t *wkey, error: if (dck != NULL) { - bzero(dck, sizeof (dsl_crypto_key_t)); + memset(dck, 0, sizeof (dsl_crypto_key_t)); kmem_free(dck, sizeof (dsl_crypto_key_t)); } @@ -601,7 +600,7 @@ dsl_crypto_key_open(objset_t *mos, dsl_wrapping_key_t *wkey, } static int -spa_keystore_dsl_key_hold_impl(spa_t *spa, uint64_t dckobj, void *tag, +spa_keystore_dsl_key_hold_impl(spa_t *spa, uint64_t dckobj, const void *tag, dsl_crypto_key_t **dck_out) { int ret; @@ -632,7 +631,7 @@ spa_keystore_dsl_key_hold_impl(spa_t *spa, uint64_t dckobj, void *tag, } static int -spa_keystore_dsl_key_hold_dd(spa_t *spa, dsl_dir_t *dd, void *tag, +spa_keystore_dsl_key_hold_dd(spa_t *spa, dsl_dir_t *dd, const void *tag, dsl_crypto_key_t **dck_out) { int ret; @@ -690,7 +689,7 @@ spa_keystore_dsl_key_hold_dd(spa_t *spa, dsl_dir_t *dd, void *tag, } void -spa_keystore_dsl_key_rele(spa_t *spa, dsl_crypto_key_t *dck, void *tag) +spa_keystore_dsl_key_rele(spa_t *spa, dsl_crypto_key_t *dck, const void *tag) { rw_enter(&spa->spa_keystore.sk_dk_lock, RW_WRITER); @@ -937,7 +936,7 @@ spa_keystore_unload_wkey(const char *dsname) } void -key_mapping_add_ref(dsl_key_mapping_t *km, void *tag) +key_mapping_add_ref(dsl_key_mapping_t *km, const void *tag) { ASSERT3U(zfs_refcount_count(&km->km_refcnt), >=, 1); zfs_refcount_add(&km->km_refcnt, tag); @@ -954,7 +953,7 @@ key_mapping_add_ref(dsl_key_mapping_t *km, void *tag) * mapping after unmounting a dataset. */ void -key_mapping_rele(spa_t *spa, dsl_key_mapping_t *km, void *tag) +key_mapping_rele(spa_t *spa, dsl_key_mapping_t *km, const void *tag) { ASSERT3U(zfs_refcount_count(&km->km_refcnt), >=, 1); @@ -985,7 +984,7 @@ key_mapping_rele(spa_t *spa, dsl_key_mapping_t *km, void *tag) } int -spa_keystore_create_mapping(spa_t *spa, dsl_dataset_t *ds, void *tag, +spa_keystore_create_mapping(spa_t *spa, dsl_dataset_t *ds, const void *tag, dsl_key_mapping_t **km_out) { int ret; @@ -1044,7 +1043,7 @@ spa_keystore_create_mapping(spa_t *spa, dsl_dataset_t *ds, void *tag, } int -spa_keystore_remove_mapping(spa_t *spa, uint64_t dsobj, void *tag) +spa_keystore_remove_mapping(spa_t *spa, uint64_t dsobj, const void *tag) { int ret; dsl_key_mapping_t search_km; @@ -1082,7 +1081,7 @@ spa_keystore_remove_mapping(spa_t *spa, uint64_t dsobj, void *tag) * without getting a reference to it. */ int -spa_keystore_lookup_key(spa_t *spa, uint64_t dsobj, void *tag, +spa_keystore_lookup_key(spa_t *spa, uint64_t dsobj, const void *tag, dsl_crypto_key_t **dck_out) { int ret; @@ -1507,7 +1506,7 @@ spa_keystore_change_key_sync(void *arg, dmu_tx_t *tx) dsl_crypto_params_t *dcp = skcka->skcka_cp; dsl_wrapping_key_t *wkey = NULL, *found_wkey; dsl_wrapping_key_t wkey_search; - char *keylocation = dcp->cp_keylocation; + const char *keylocation = dcp->cp_keylocation; uint64_t rddobj, new_rddobj; /* create and initialize the wrapping key */ @@ -2096,8 +2095,8 @@ dsl_crypto_recv_raw_objset_sync(dsl_dataset_t *ds, dmu_objset_type_t ostype, * written out raw next time. */ arc_release(os->os_phys_buf, &os->os_phys_buf); - bcopy(portable_mac, os->os_phys->os_portable_mac, ZIO_OBJSET_MAC_LEN); - bzero(os->os_phys->os_local_mac, ZIO_OBJSET_MAC_LEN); + memcpy(os->os_phys->os_portable_mac, portable_mac, ZIO_OBJSET_MAC_LEN); + memset(os->os_phys->os_local_mac, 0, ZIO_OBJSET_MAC_LEN); os->os_flags &= ~OBJSET_FLAG_USERACCOUNTING_COMPLETE; os->os_next_write_raw[tx->tx_txg & TXG_MASK] = B_TRUE; @@ -2230,7 +2229,7 @@ dsl_crypto_recv_raw_key_sync(dsl_dataset_t *ds, nvlist_t *nvl, dmu_tx_t *tx) uint8_t *keydata, *hmac_keydata, *iv, *mac; uint64_t crypt, key_guid, keyformat, iters, salt; uint64_t version = ZIO_CRYPT_KEY_CURRENT_VERSION; - char *keylocation = "prompt"; + const char *keylocation = "prompt"; /* lookup the values we need to create the DSL Crypto Key */ crypt = fnvlist_lookup_uint64(nvl, DSL_CRYPTO_KEY_CRYPTO_SUITE); @@ -2548,7 +2547,7 @@ dsl_crypto_key_create_sync(uint64_t crypt, dsl_wrapping_key_t *wkey, DSL_CRYPTO_KEY_VERSION, sizeof (uint64_t), 1, &version, tx)); zio_crypt_key_destroy(&dck.dck_key); - bzero(&dck.dck_key, sizeof (zio_crypt_key_t)); + memset(&dck.dck_key, 0, sizeof (zio_crypt_key_t)); return (dck.dck_obj); } @@ -2688,14 +2687,15 @@ spa_do_crypt_objset_mac_abd(boolean_t generate, spa_t *spa, uint64_t dsobj, /* if we are generating encode the HMACs in the objset_phys_t */ if (generate) { - bcopy(portable_mac, osp->os_portable_mac, ZIO_OBJSET_MAC_LEN); - bcopy(local_mac, osp->os_local_mac, ZIO_OBJSET_MAC_LEN); + memcpy(osp->os_portable_mac, portable_mac, ZIO_OBJSET_MAC_LEN); + memcpy(osp->os_local_mac, local_mac, ZIO_OBJSET_MAC_LEN); abd_return_buf_copy(abd, buf, datalen); return (0); } - if (bcmp(portable_mac, osp->os_portable_mac, ZIO_OBJSET_MAC_LEN) != 0 || - bcmp(local_mac, osp->os_local_mac, ZIO_OBJSET_MAC_LEN) != 0) { + if (memcmp(portable_mac, osp->os_portable_mac, + ZIO_OBJSET_MAC_LEN) != 0 || + memcmp(local_mac, osp->os_local_mac, ZIO_OBJSET_MAC_LEN) != 0) { abd_return_buf(abd, buf, datalen); return (SET_ERROR(ECKSUM)); } @@ -2739,11 +2739,11 @@ spa_do_crypt_mac_abd(boolean_t generate, spa_t *spa, uint64_t dsobj, abd_t *abd, * Otherwise verify that the MAC matched what we expected. */ if (generate) { - bcopy(digestbuf, mac, ZIO_DATA_MAC_LEN); + memcpy(mac, digestbuf, ZIO_DATA_MAC_LEN); return (0); } - if (bcmp(digestbuf, mac, ZIO_DATA_MAC_LEN) != 0) + if (memcmp(digestbuf, mac, ZIO_DATA_MAC_LEN) != 0) return (SET_ERROR(ECKSUM)); return (0); @@ -2842,9 +2842,9 @@ spa_do_crypt_abd(boolean_t encrypt, spa_t *spa, const zbookmark_phys_t *zb, error: if (encrypt) { /* zero out any state we might have changed while encrypting */ - bzero(salt, ZIO_DATA_SALT_LEN); - bzero(iv, ZIO_DATA_IV_LEN); - bzero(mac, ZIO_DATA_MAC_LEN); + memset(salt, 0, ZIO_DATA_SALT_LEN); + memset(iv, 0, ZIO_DATA_IV_LEN); + memset(mac, 0, ZIO_DATA_MAC_LEN); abd_return_buf(pabd, plainbuf, datalen); abd_return_buf_copy(cabd, cipherbuf, datalen); } else { diff --git a/module/zfs/dsl_dataset.c b/module/zfs/dsl_dataset.c index 85b48fd12b63..5f1235583414 100644 --- a/module/zfs/dsl_dataset.c +++ b/module/zfs/dsl_dataset.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -73,12 +73,19 @@ * The SPA supports block sizes up to 16MB. However, very large blocks * can have an impact on i/o latency (e.g. tying up a spinning disk for * ~300ms), and also potentially on the memory allocator. Therefore, - * we do not allow the recordsize to be set larger than zfs_max_recordsize - * (default 1MB). Larger blocks can be created by changing this tunable, - * and pools with larger blocks can always be imported and used, regardless - * of this setting. + * we did not allow the recordsize to be set larger than zfs_max_recordsize + * (former default: 1MB). Larger blocks could be created by changing this + * tunable, and pools with larger blocks could always be imported and used, + * regardless of this setting. + * + * We do, however, still limit it by default to 1M on x86_32, because Linux's + * 3/1 memory split doesn't leave much room for 16M chunks. */ -int zfs_max_recordsize = 1 * 1024 * 1024; +#ifdef _ILP32 +int zfs_max_recordsize = 1 * 1024 * 1024; +#else +int zfs_max_recordsize = 16 * 1024 * 1024; +#endif static int zfs_allow_redacted_dataset_mount = 0; #define SWITCH64(x, y) \ @@ -123,6 +130,30 @@ parent_delta(dsl_dataset_t *ds, int64_t delta) return (new_bytes - old_bytes); } +void +dsl_dataset_feature_set_activation(const blkptr_t *bp, dsl_dataset_t *ds) +{ + spa_feature_t f; + if (BP_GET_LSIZE(bp) > SPA_OLD_MAXBLOCKSIZE) { + ds->ds_feature_activation[SPA_FEATURE_LARGE_BLOCKS] = + (void *)B_TRUE; + } + + f = zio_checksum_to_feature(BP_GET_CHECKSUM(bp)); + if (f != SPA_FEATURE_NONE) { + ASSERT3S(spa_feature_table[f].fi_type, ==, + ZFEATURE_TYPE_BOOLEAN); + ds->ds_feature_activation[f] = (void *)B_TRUE; + } + + f = zio_compress_to_feature(BP_GET_COMPRESS(bp)); + if (f != SPA_FEATURE_NONE) { + ASSERT3S(spa_feature_table[f].fi_type, ==, + ZFEATURE_TYPE_BOOLEAN); + ds->ds_feature_activation[f] = (void *)B_TRUE; + } +} + void dsl_dataset_block_born(dsl_dataset_t *ds, const blkptr_t *bp, dmu_tx_t *tx) { @@ -131,7 +162,6 @@ dsl_dataset_block_born(dsl_dataset_t *ds, const blkptr_t *bp, dmu_tx_t *tx) int compressed = BP_GET_PSIZE(bp); int uncompressed = BP_GET_UCSIZE(bp); int64_t delta; - spa_feature_t f; dprintf_bp(bp, "ds=%p", ds); @@ -156,25 +186,7 @@ dsl_dataset_block_born(dsl_dataset_t *ds, const blkptr_t *bp, dmu_tx_t *tx) dsl_dataset_phys(ds)->ds_uncompressed_bytes += uncompressed; dsl_dataset_phys(ds)->ds_unique_bytes += used; - if (BP_GET_LSIZE(bp) > SPA_OLD_MAXBLOCKSIZE) { - ds->ds_feature_activation[SPA_FEATURE_LARGE_BLOCKS] = - (void *)B_TRUE; - } - - - f = zio_checksum_to_feature(BP_GET_CHECKSUM(bp)); - if (f != SPA_FEATURE_NONE) { - ASSERT3S(spa_feature_table[f].fi_type, ==, - ZFEATURE_TYPE_BOOLEAN); - ds->ds_feature_activation[f] = (void *)B_TRUE; - } - - f = zio_compress_to_feature(BP_GET_COMPRESS(bp)); - if (f != SPA_FEATURE_NONE) { - ASSERT3S(spa_feature_table[f].fi_type, ==, - ZFEATURE_TYPE_BOOLEAN); - ds->ds_feature_activation[f] = (void *)B_TRUE; - } + dsl_dataset_feature_set_activation(bp, ds); /* * Track block for livelist, but ignore embedded blocks because @@ -539,7 +551,7 @@ dsl_dataset_snap_remove(dsl_dataset_t *ds, const char *name, dmu_tx_t *tx, } boolean_t -dsl_dataset_try_add_ref(dsl_pool_t *dp, dsl_dataset_t *ds, void *tag) +dsl_dataset_try_add_ref(dsl_pool_t *dp, dsl_dataset_t *ds, const void *tag) { dmu_buf_t *dbuf = ds->ds_dbuf; boolean_t result = B_FALSE; @@ -557,7 +569,7 @@ dsl_dataset_try_add_ref(dsl_pool_t *dp, dsl_dataset_t *ds, void *tag) } int -dsl_dataset_hold_obj(dsl_pool_t *dp, uint64_t dsobj, void *tag, +dsl_dataset_hold_obj(dsl_pool_t *dp, uint64_t dsobj, const void *tag, dsl_dataset_t **dsp) { objset_t *mos = dp->dp_meta_objset; @@ -746,7 +758,7 @@ dsl_dataset_create_key_mapping(dsl_dataset_t *ds) int dsl_dataset_hold_obj_flags(dsl_pool_t *dp, uint64_t dsobj, - ds_hold_flags_t flags, void *tag, dsl_dataset_t **dsp) + ds_hold_flags_t flags, const void *tag, dsl_dataset_t **dsp) { int err; @@ -767,7 +779,7 @@ dsl_dataset_hold_obj_flags(dsl_pool_t *dp, uint64_t dsobj, int dsl_dataset_hold_flags(dsl_pool_t *dp, const char *name, ds_hold_flags_t flags, - void *tag, dsl_dataset_t **dsp) + const void *tag, dsl_dataset_t **dsp) { dsl_dir_t *dd; const char *snapname; @@ -820,7 +832,7 @@ dsl_dataset_hold_flags(dsl_pool_t *dp, const char *name, ds_hold_flags_t flags, } int -dsl_dataset_hold(dsl_pool_t *dp, const char *name, void *tag, +dsl_dataset_hold(dsl_pool_t *dp, const char *name, const void *tag, dsl_dataset_t **dsp) { return (dsl_dataset_hold_flags(dp, name, 0, tag, dsp)); @@ -828,7 +840,7 @@ dsl_dataset_hold(dsl_pool_t *dp, const char *name, void *tag, static int dsl_dataset_own_obj_impl(dsl_pool_t *dp, uint64_t dsobj, ds_hold_flags_t flags, - void *tag, boolean_t override, dsl_dataset_t **dsp) + const void *tag, boolean_t override, dsl_dataset_t **dsp) { int err = dsl_dataset_hold_obj_flags(dp, dsobj, flags, tag, dsp); if (err != 0) @@ -844,21 +856,21 @@ dsl_dataset_own_obj_impl(dsl_pool_t *dp, uint64_t dsobj, ds_hold_flags_t flags, int dsl_dataset_own_obj(dsl_pool_t *dp, uint64_t dsobj, ds_hold_flags_t flags, - void *tag, dsl_dataset_t **dsp) + const void *tag, dsl_dataset_t **dsp) { return (dsl_dataset_own_obj_impl(dp, dsobj, flags, tag, B_FALSE, dsp)); } int dsl_dataset_own_obj_force(dsl_pool_t *dp, uint64_t dsobj, - ds_hold_flags_t flags, void *tag, dsl_dataset_t **dsp) + ds_hold_flags_t flags, const void *tag, dsl_dataset_t **dsp) { return (dsl_dataset_own_obj_impl(dp, dsobj, flags, tag, B_TRUE, dsp)); } static int dsl_dataset_own_impl(dsl_pool_t *dp, const char *name, ds_hold_flags_t flags, - void *tag, boolean_t override, dsl_dataset_t **dsp) + const void *tag, boolean_t override, dsl_dataset_t **dsp) { int err = dsl_dataset_hold_flags(dp, name, flags, tag, dsp); if (err != 0) @@ -872,14 +884,14 @@ dsl_dataset_own_impl(dsl_pool_t *dp, const char *name, ds_hold_flags_t flags, int dsl_dataset_own_force(dsl_pool_t *dp, const char *name, ds_hold_flags_t flags, - void *tag, dsl_dataset_t **dsp) + const void *tag, dsl_dataset_t **dsp) { return (dsl_dataset_own_impl(dp, name, flags, tag, B_TRUE, dsp)); } int dsl_dataset_own(dsl_pool_t *dp, const char *name, ds_hold_flags_t flags, - void *tag, dsl_dataset_t **dsp) + const void *tag, dsl_dataset_t **dsp) { return (dsl_dataset_own_impl(dp, name, flags, tag, B_FALSE, dsp)); } @@ -958,7 +970,7 @@ dsl_dataset_namelen(dsl_dataset_t *ds) } void -dsl_dataset_rele(dsl_dataset_t *ds, void *tag) +dsl_dataset_rele(dsl_dataset_t *ds, const void *tag) { dmu_buf_rele(ds->ds_dbuf, tag); } @@ -976,7 +988,8 @@ dsl_dataset_remove_key_mapping(dsl_dataset_t *ds) } void -dsl_dataset_rele_flags(dsl_dataset_t *ds, ds_hold_flags_t flags, void *tag) +dsl_dataset_rele_flags(dsl_dataset_t *ds, ds_hold_flags_t flags, + const void *tag) { if (flags & DS_HOLD_FLAG_DECRYPT) dsl_dataset_remove_key_mapping(ds); @@ -985,7 +998,7 @@ dsl_dataset_rele_flags(dsl_dataset_t *ds, ds_hold_flags_t flags, void *tag) } void -dsl_dataset_disown(dsl_dataset_t *ds, ds_hold_flags_t flags, void *tag) +dsl_dataset_disown(dsl_dataset_t *ds, ds_hold_flags_t flags, const void *tag) { ASSERT3P(ds->ds_owner, ==, tag); ASSERT(ds->ds_dbuf != NULL); @@ -998,7 +1011,7 @@ dsl_dataset_disown(dsl_dataset_t *ds, ds_hold_flags_t flags, void *tag) } boolean_t -dsl_dataset_tryown(dsl_dataset_t *ds, void *tag, boolean_t override) +dsl_dataset_tryown(dsl_dataset_t *ds, const void *tag, boolean_t override) { boolean_t gotit = FALSE; @@ -1148,7 +1161,7 @@ dsl_dataset_create_sync_dd(dsl_dir_t *dd, dsl_dataset_t *origin, VERIFY0(dmu_bonus_hold(mos, dsobj, FTAG, &dbuf)); dmu_buf_will_dirty(dbuf, tx); dsphys = dbuf->db_data; - bzero(dsphys, sizeof (dsl_dataset_phys_t)); + memset(dsphys, 0, sizeof (dsl_dataset_phys_t)); dsphys->ds_dir_obj = dd->dd_object; dsphys->ds_flags = flags; dsphys->ds_fsid_guid = unique_create(); @@ -1248,11 +1261,11 @@ dsl_dataset_zero_zil(dsl_dataset_t *ds, dmu_tx_t *tx) objset_t *os; VERIFY0(dmu_objset_from_ds(ds, &os)); - if (bcmp(&os->os_zil_header, &zero_zil, sizeof (zero_zil)) != 0) { + if (memcmp(&os->os_zil_header, &zero_zil, sizeof (zero_zil)) != 0) { dsl_pool_t *dp = ds->ds_dir->dd_pool; zio_t *zio; - bzero(&os->os_zil_header, sizeof (os->os_zil_header)); + memset(&os->os_zil_header, 0, sizeof (os->os_zil_header)); if (os->os_encrypted) os->os_next_write_raw[tx->tx_txg & TXG_MASK] = B_TRUE; @@ -1696,7 +1709,7 @@ dsl_dataset_snapshot_sync_impl(dsl_dataset_t *ds, const char *snapname, */ ASSERT(spa_version(dmu_tx_pool(tx)->dp_spa) >= SPA_VERSION_FAST_SNAP || dmu_objset_from_ds(ds, &os) != 0 || - bcmp(&os->os_phys->os_zil_header, &zero_zil, + memcmp(&os->os_phys->os_zil_header, &zero_zil, sizeof (zero_zil)) == 0); /* Should not snapshot a dirty dataset. */ @@ -1718,7 +1731,7 @@ dsl_dataset_snapshot_sync_impl(dsl_dataset_t *ds, const char *snapname, VERIFY0(dmu_bonus_hold(mos, dsobj, FTAG, &dbuf)); dmu_buf_will_dirty(dbuf, tx); dsphys = dbuf->db_data; - bzero(dsphys, sizeof (dsl_dataset_phys_t)); + memset(dsphys, 0, sizeof (dsl_dataset_phys_t)); dsphys->ds_dir_obj = ds->ds_dir->dd_object; dsphys->ds_fsid_guid = unique_create(); (void) random_get_pseudo_bytes((void*)&dsphys->ds_guid, @@ -2895,7 +2908,7 @@ dsl_dataset_modified_since_snap(dsl_dataset_t *ds, dsl_dataset_t *snap) return (B_TRUE); if (dmu_objset_from_ds(snap, &os_snap) != 0) return (B_TRUE); - return (bcmp(&os->os_phys->os_meta_dnode, + return (memcmp(&os->os_phys->os_meta_dnode, &os_snap->os_phys->os_meta_dnode, sizeof (os->os_phys->os_meta_dnode)) != 0); } @@ -3266,8 +3279,8 @@ struct promotenode { static int snaplist_space(list_t *l, uint64_t mintxg, uint64_t *spacep); static int promote_hold(dsl_dataset_promote_arg_t *ddpa, dsl_pool_t *dp, - void *tag); -static void promote_rele(dsl_dataset_promote_arg_t *ddpa, void *tag); + const void *tag); +static void promote_rele(dsl_dataset_promote_arg_t *ddpa, const void *tag); int dsl_dataset_promote_check(void *arg, dmu_tx_t *tx) @@ -3708,6 +3721,15 @@ dsl_dataset_promote_sync(void *arg, dmu_tx_t *tx) dsl_dir_rele(odd, FTAG); promote_rele(ddpa, FTAG); + + /* + * Transfer common error blocks from old head to new head. + */ + if (spa_feature_is_enabled(dp->dp_spa, SPA_FEATURE_HEAD_ERRLOG)) { + uint64_t old_head = origin_head->ds_object; + uint64_t new_head = hds->ds_object; + spa_swap_errlog(dp->dp_spa, new_head, old_head, tx); + } } /* @@ -3718,7 +3740,7 @@ dsl_dataset_promote_sync(void *arg, dmu_tx_t *tx) */ static int snaplist_make(dsl_pool_t *dp, - uint64_t first_obj, uint64_t last_obj, list_t *l, void *tag) + uint64_t first_obj, uint64_t last_obj, list_t *l, const void *tag) { uint64_t obj = last_obj; @@ -3763,7 +3785,7 @@ snaplist_space(list_t *l, uint64_t mintxg, uint64_t *spacep) } static void -snaplist_destroy(list_t *l, void *tag) +snaplist_destroy(list_t *l, const void *tag) { struct promotenode *snap; @@ -3779,7 +3801,7 @@ snaplist_destroy(list_t *l, void *tag) } static int -promote_hold(dsl_dataset_promote_arg_t *ddpa, dsl_pool_t *dp, void *tag) +promote_hold(dsl_dataset_promote_arg_t *ddpa, dsl_pool_t *dp, const void *tag) { int error; dsl_dir_t *dd; @@ -3829,7 +3851,7 @@ promote_hold(dsl_dataset_promote_arg_t *ddpa, dsl_pool_t *dp, void *tag) } static void -promote_rele(dsl_dataset_promote_arg_t *ddpa, void *tag) +promote_rele(dsl_dataset_promote_arg_t *ddpa, const void *tag) { snaplist_destroy(&ddpa->shared_snaps, tag); snaplist_destroy(&ddpa->clone_snaps, tag); @@ -4916,7 +4938,7 @@ dsl_dataset_activate_redaction(dsl_dataset_t *ds, uint64_t *redact_snaps, if (num_redact_snaps > 0) { ftuaa->array = kmem_alloc(num_redact_snaps * sizeof (uint64_t), KM_SLEEP); - bcopy(redact_snaps, ftuaa->array, num_redact_snaps * + memcpy(ftuaa->array, redact_snaps, num_redact_snaps * sizeof (uint64_t)); } dsl_dataset_activate_feature(dsobj, SPA_FEATURE_REDACTED_DATASETS, @@ -4924,13 +4946,38 @@ dsl_dataset_activate_redaction(dsl_dataset_t *ds, uint64_t *redact_snaps, ds->ds_feature[SPA_FEATURE_REDACTED_DATASETS] = ftuaa; } -#if defined(_LP64) -#define RECORDSIZE_PERM ZMOD_RW -#else -/* Limited to 1M on 32-bit platforms due to lack of virtual address space */ -#define RECORDSIZE_PERM ZMOD_RD -#endif -ZFS_MODULE_PARAM(zfs, zfs_, max_recordsize, INT, RECORDSIZE_PERM, +/* + * Find and return (in *oldest_dsobj) the oldest snapshot of the dsobj + * dataset whose birth time is >= min_txg. + */ +int +dsl_dataset_oldest_snapshot(spa_t *spa, uint64_t head_ds, uint64_t min_txg, + uint64_t *oldest_dsobj) +{ + dsl_dataset_t *ds; + dsl_pool_t *dp = spa->spa_dsl_pool; + + int error = dsl_dataset_hold_obj(dp, head_ds, FTAG, &ds); + if (error != 0) + return (error); + + uint64_t prev_obj = dsl_dataset_phys(ds)->ds_prev_snap_obj; + uint64_t prev_obj_txg = dsl_dataset_phys(ds)->ds_prev_snap_txg; + + while (prev_obj != 0 && min_txg < prev_obj_txg) { + dsl_dataset_rele(ds, FTAG); + if ((error = dsl_dataset_hold_obj(dp, prev_obj, + FTAG, &ds)) != 0) + return (error); + prev_obj_txg = dsl_dataset_phys(ds)->ds_prev_snap_txg; + prev_obj = dsl_dataset_phys(ds)->ds_prev_snap_obj; + } + *oldest_dsobj = ds->ds_object; + dsl_dataset_rele(ds, FTAG); + return (0); +} + +ZFS_MODULE_PARAM(zfs, zfs_, max_recordsize, INT, ZMOD_RW, "Max allowed record size"); ZFS_MODULE_PARAM(zfs, zfs_, allow_redacted_dataset_mount, INT, ZMOD_RW, @@ -4972,3 +5019,4 @@ EXPORT_SYMBOL(dsl_dsobj_to_dsname); EXPORT_SYMBOL(dsl_dataset_check_quota); EXPORT_SYMBOL(dsl_dataset_clone_swap_check_impl); EXPORT_SYMBOL(dsl_dataset_clone_swap_sync_impl); +EXPORT_SYMBOL(dsl_dataset_feature_set_activation); diff --git a/module/zfs/dsl_deadlist.c b/module/zfs/dsl_deadlist.c index e620510be6b1..7270924ec11f 100644 --- a/module/zfs/dsl_deadlist.c +++ b/module/zfs/dsl_deadlist.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -855,7 +855,7 @@ dsl_deadlist_merge(dsl_deadlist_t *dl, uint64_t obj, dmu_tx_t *tx) VERIFY0(dmu_bonus_hold(dl->dl_os, obj, FTAG, &bonus)); dlp = bonus->db_data; dmu_buf_will_dirty(bonus, tx); - bzero(dlp, sizeof (*dlp)); + memset(dlp, 0, sizeof (*dlp)); dmu_buf_rele(bonus, FTAG); mutex_exit(&dl->dl_lock); } diff --git a/module/zfs/dsl_deleg.c b/module/zfs/dsl_deleg.c index cf8a3c9bbdfb..645ad8e5b8dc 100644 --- a/module/zfs/dsl_deleg.c +++ b/module/zfs/dsl_deleg.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/dsl_destroy.c b/module/zfs/dsl_destroy.c index b32929b3320c..778b6b99b85e 100644 --- a/module/zfs/dsl_destroy.c +++ b/module/zfs/dsl_destroy.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -200,7 +200,7 @@ rck_alloc(dsl_dataset_t *clone) static void dsl_dir_remove_clones_key_impl(dsl_dir_t *dd, uint64_t mintxg, dmu_tx_t *tx, - list_t *stack, void *tag) + list_t *stack, const void *tag) { objset_t *mos = dd->dd_pool->dp_meta_objset; @@ -1153,6 +1153,9 @@ dsl_destroy_head_sync_impl(dsl_dataset_t *ds, dmu_tx_t *tx) dsl_destroy_snapshot_sync_impl(prev, B_FALSE, tx); dsl_dataset_rele(prev, FTAG); } + /* Delete errlog. */ + if (spa_feature_is_enabled(dp->dp_spa, SPA_FEATURE_HEAD_ERRLOG)) + spa_delete_dataset_errlog(dp->dp_spa, ds->ds_object, tx); } void diff --git a/module/zfs/dsl_dir.c b/module/zfs/dsl_dir.c index 43ef1897811c..cc870d1bc820 100644 --- a/module/zfs/dsl_dir.c +++ b/module/zfs/dsl_dir.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -160,7 +160,7 @@ dsl_dir_evict_async(void *dbu) int dsl_dir_hold_obj(dsl_pool_t *dp, uint64_t ddobj, - const char *tail, void *tag, dsl_dir_t **ddp) + const char *tail, const void *tag, dsl_dir_t **ddp) { dmu_buf_t *dbuf; dsl_dir_t *dd; @@ -320,7 +320,7 @@ dsl_dir_hold_obj(dsl_pool_t *dp, uint64_t ddobj, } void -dsl_dir_rele(dsl_dir_t *dd, void *tag) +dsl_dir_rele(dsl_dir_t *dd, const void *tag) { dprintf_dd(dd, "%s\n", ""); spa_close(dd->dd_pool->dp_spa, tag); @@ -335,7 +335,7 @@ dsl_dir_rele(dsl_dir_t *dd, void *tag) * the spa. */ void -dsl_dir_async_rele(dsl_dir_t *dd, void *tag) +dsl_dir_async_rele(dsl_dir_t *dd, const void *tag) { dprintf_dd(dd, "%s\n", ""); spa_async_close(dd->dd_pool->dp_spa, tag); @@ -449,7 +449,7 @@ getcomponent(const char *path, char *component, const char **nextp) * (*tail)[0] == '@' means that the last component is a snapshot. */ int -dsl_dir_hold(dsl_pool_t *dp, const char *name, void *tag, +dsl_dir_hold(dsl_pool_t *dp, const char *name, const void *tag, dsl_dir_t **ddp, const char **tailp) { char *buf; @@ -801,7 +801,7 @@ dsl_fs_ss_limit_check(dsl_dir_t *dd, uint64_t delta, zfs_prop_t prop, { objset_t *os = dd->dd_pool->dp_meta_objset; uint64_t limit, count; - char *count_prop; + const char *count_prop; enforce_res_t enforce; int err = 0; @@ -1334,13 +1334,21 @@ dsl_dir_tempreserve_impl(dsl_dir_t *dd, uint64_t asize, boolean_t netfree, /* * If they are requesting more space, and our current estimate * is over quota, they get to try again unless the actual - * on-disk is over quota and there are no pending changes (which - * may free up space for us). + * on-disk is over quota and there are no pending changes + * or deferred frees (which may free up space for us). */ if (used_on_disk + est_inflight >= quota) { - if (est_inflight > 0 || used_on_disk < quota || - (retval == ENOSPC && used_on_disk < quota)) - retval = ERESTART; + if (est_inflight > 0 || used_on_disk < quota) { + retval = SET_ERROR(ERESTART); + } else { + ASSERT3U(used_on_disk, >=, quota); + + if (retval == ENOSPC && (used_on_disk - quota) < + dsl_pool_deferred_space(dd->dd_pool)) { + retval = SET_ERROR(ERESTART); + } + } + dprintf_dd(dd, "failing: used=%lluK inflight = %lluK " "quota=%lluK tr=%lluK err=%d\n", (u_longlong_t)used_on_disk>>10, @@ -1348,7 +1356,7 @@ dsl_dir_tempreserve_impl(dsl_dir_t *dd, uint64_t asize, boolean_t netfree, (u_longlong_t)quota>>10, (u_longlong_t)asize>>10, retval); mutex_exit(&dd->dd_lock); DMU_TX_STAT_BUMP(dmu_tx_quota); - return (SET_ERROR(retval)); + return (retval); } /* We need to up our estimated delta before dropping dd_lock */ diff --git a/module/zfs/dsl_pool.c b/module/zfs/dsl_pool.c index 9a15f628479b..7a589cadbc32 100644 --- a/module/zfs/dsl_pool.c +++ b/module/zfs/dsl_pool.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -105,9 +105,8 @@ int zfs_dirty_data_max_percent = 10; int zfs_dirty_data_max_max_percent = 25; /* - * zfs_wrlog_data_max, the upper limit of TX_WRITE log data. - * Once it is reached, write operation is blocked, - * until log data is cleared out after txg sync. + * The upper limit of TX_WRITE log data. Write operations are throttled + * when approaching the limit until log data is cleared out after txg sync. * It only counts TX_WRITE log with WR_COPIED or WR_NEED_COPY. */ unsigned long zfs_wrlog_data_max = 0; @@ -439,10 +438,8 @@ dsl_pool_close(dsl_pool_t *dp) taskq_destroy(dp->dp_unlinked_drain_taskq); taskq_destroy(dp->dp_zrele_taskq); - if (dp->dp_blkstats != NULL) { - mutex_destroy(&dp->dp_blkstats->zab_lock); + if (dp->dp_blkstats != NULL) vmem_free(dp->dp_blkstats, sizeof (zfs_all_blkstats_t)); - } kmem_free(dp, sizeof (dsl_pool_t)); } @@ -623,15 +620,18 @@ dsl_pool_wrlog_count(dsl_pool_t *dp, int64_t size, uint64_t txg) /* Choose a value slightly bigger than min dirty sync bytes */ uint64_t sync_min = - zfs_dirty_data_max * (zfs_dirty_data_sync_percent + 10) / 100; + zfs_wrlog_data_max * (zfs_dirty_data_sync_percent + 10) / 200; if (aggsum_compare(&dp->dp_wrlog_pertxg[txg & TXG_MASK], sync_min) > 0) txg_kick(dp, txg); } boolean_t -dsl_pool_wrlog_over_max(dsl_pool_t *dp) +dsl_pool_need_wrlog_delay(dsl_pool_t *dp) { - return (aggsum_compare(&dp->dp_wrlog_total, zfs_wrlog_data_max) > 0); + uint64_t delay_min_bytes = + zfs_wrlog_data_max * zfs_delay_min_dirty_percent / 100; + + return (aggsum_compare(&dp->dp_wrlog_total, delay_min_bytes) > 0); } static void @@ -641,6 +641,9 @@ dsl_pool_wrlog_clear(dsl_pool_t *dp, uint64_t txg) delta = -(int64_t)aggsum_value(&dp->dp_wrlog_pertxg[txg & TXG_MASK]); aggsum_add(&dp->dp_wrlog_pertxg[txg & TXG_MASK], delta); aggsum_add(&dp->dp_wrlog_total, delta); + /* Compact per-CPU sums after the big change. */ + (void) aggsum_value(&dp->dp_wrlog_pertxg[txg & TXG_MASK]); + (void) aggsum_value(&dp->dp_wrlog_total); } #ifdef ZFS_DEBUG @@ -950,6 +953,12 @@ dsl_pool_unreserved_space(dsl_pool_t *dp, zfs_space_check_t slop_policy) return (quota); } +uint64_t +dsl_pool_deferred_space(dsl_pool_t *dp) +{ + return (metaslab_class_get_deferred(spa_normal_class(dp->dp_spa))); +} + boolean_t dsl_pool_need_dirty_delay(dsl_pool_t *dp) { @@ -1010,7 +1019,6 @@ dsl_pool_undirty_space(dsl_pool_t *dp, int64_t space, uint64_t txg) mutex_exit(&dp->dp_lock); } -/* ARGSUSED */ static int upgrade_clones_cb(dsl_pool_t *dp, dsl_dataset_t *hds, void *arg) { @@ -1101,7 +1109,6 @@ dsl_pool_upgrade_clones(dsl_pool_t *dp, dmu_tx_t *tx) tx, DS_FIND_CHILDREN | DS_FIND_SERIALIZE)); } -/* ARGSUSED */ static int upgrade_dir_clones_cb(dsl_pool_t *dp, dsl_dataset_t *ds, void *arg) { @@ -1380,7 +1387,7 @@ dsl_pool_user_release(dsl_pool_t *dp, uint64_t dsobj, const char *tag, */ int -dsl_pool_hold(const char *name, void *tag, dsl_pool_t **dp) +dsl_pool_hold(const char *name, const void *tag, dsl_pool_t **dp) { spa_t *spa; int error; @@ -1394,14 +1401,14 @@ dsl_pool_hold(const char *name, void *tag, dsl_pool_t **dp) } void -dsl_pool_rele(dsl_pool_t *dp, void *tag) +dsl_pool_rele(dsl_pool_t *dp, const void *tag) { dsl_pool_config_exit(dp, tag); spa_close(dp->dp_spa, tag); } void -dsl_pool_config_enter(dsl_pool_t *dp, void *tag) +dsl_pool_config_enter(dsl_pool_t *dp, const void *tag) { /* * We use a "reentrant" reader-writer lock, but not reentrantly. @@ -1420,14 +1427,14 @@ dsl_pool_config_enter(dsl_pool_t *dp, void *tag) } void -dsl_pool_config_enter_prio(dsl_pool_t *dp, void *tag) +dsl_pool_config_enter_prio(dsl_pool_t *dp, const void *tag) { ASSERT(!rrw_held(&dp->dp_config_rwlock, RW_READER)); rrw_enter_read_prio(&dp->dp_config_rwlock, tag); } void -dsl_pool_config_exit(dsl_pool_t *dp, void *tag) +dsl_pool_config_exit(dsl_pool_t *dp, const void *tag) { rrw_exit(&dp->dp_config_rwlock, tag); } diff --git a/module/zfs/dsl_prop.c b/module/zfs/dsl_prop.c index cfdafd2f4436..1d3d26124949 100644 --- a/module/zfs/dsl_prop.c +++ b/module/zfs/dsl_prop.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -88,7 +88,7 @@ dsl_prop_get_dd(dsl_dir_t *dd, const char *propname, setpoint[0] = '\0'; prop = zfs_name_to_prop(propname); - inheritable = (prop == ZPROP_INVAL || zfs_prop_inheritable(prop)); + inheritable = (prop == ZPROP_USERPROP || zfs_prop_inheritable(prop)); inheritstr = kmem_asprintf("%s%s", propname, ZPROP_INHERIT_SUFFIX); recvdstr = kmem_asprintf("%s%s", propname, ZPROP_RECVD_SUFFIX); @@ -168,7 +168,7 @@ dsl_prop_get_ds(dsl_dataset_t *ds, const char *propname, uint64_t zapobj; ASSERT(dsl_pool_config_held(ds->ds_dir->dd_pool)); - inheritable = (prop == ZPROP_INVAL || zfs_prop_inheritable(prop)); + inheritable = (prop == ZPROP_USERPROP || zfs_prop_inheritable(prop)); zapobj = dsl_dataset_phys(ds)->ds_props_obj; if (zapobj != 0) { @@ -1055,12 +1055,12 @@ dsl_prop_get_all_impl(objset_t *mos, uint64_t propobj, prop = zfs_name_to_prop(propname); /* Skip non-inheritable properties. */ - if ((flags & DSL_PROP_GET_INHERITING) && prop != ZPROP_INVAL && - !zfs_prop_inheritable(prop)) + if ((flags & DSL_PROP_GET_INHERITING) && + prop != ZPROP_USERPROP && !zfs_prop_inheritable(prop)) continue; /* Skip properties not valid for this type. */ - if ((flags & DSL_PROP_GET_SNAPSHOT) && prop != ZPROP_INVAL && + if ((flags & DSL_PROP_GET_SNAPSHOT) && prop != ZPROP_USERPROP && !zfs_prop_valid_for_type(prop, ZFS_TYPE_SNAPSHOT, B_FALSE)) continue; diff --git a/module/zfs/dsl_scan.c b/module/zfs/dsl_scan.c index c6a5807c92f5..2b76bed1b69b 100644 --- a/module/zfs/dsl_scan.c +++ b/module/zfs/dsl_scan.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -129,6 +129,7 @@ static void scan_ds_queue_sync(dsl_scan_t *scn, dmu_tx_t *tx); static uint64_t dsl_scan_count_data_disks(vdev_t *vd); extern int zfs_vdev_async_write_active_min_dirty_percent; +static int zfs_scan_blkstats = 0; /* * By default zfs will check to ensure it is not over the hard memory @@ -220,9 +221,9 @@ typedef struct { /* * This controls what conditions are placed on dsl_scan_sync_state(): - * SYNC_OPTIONAL) write out scn_phys iff scn_bytes_pending == 0 - * SYNC_MANDATORY) write out scn_phys always. scn_bytes_pending must be 0. - * SYNC_CACHED) if scn_bytes_pending == 0, write out scn_phys. Otherwise + * SYNC_OPTIONAL) write out scn_phys iff scn_queues_pending == 0 + * SYNC_MANDATORY) write out scn_phys always. scn_queues_pending must be 0. + * SYNC_CACHED) if scn_queues_pending == 0, write out scn_phys. Otherwise * write out the scn_phys_cached version. * See dsl_scan_sync_state for details. */ @@ -280,12 +281,14 @@ typedef struct scan_io { struct dsl_scan_io_queue { dsl_scan_t *q_scn; /* associated dsl_scan_t */ vdev_t *q_vd; /* top-level vdev that this queue represents */ + zio_t *q_zio; /* scn_zio_root child for waiting on IO */ /* trees used for sorting I/Os and extents of I/Os */ range_tree_t *q_exts_by_addr; - zfs_btree_t q_exts_by_size; + zfs_btree_t q_exts_by_size; avl_tree_t q_sios_by_addr; uint64_t q_sio_memused; + uint64_t q_last_ext_addr; /* members for zio rate limiting */ uint64_t q_maxinflight_bytes; @@ -393,7 +396,7 @@ dsl_scan_resilvering(dsl_pool_t *dp) static inline void sio2bp(const scan_io_t *sio, blkptr_t *bp) { - bzero(bp, sizeof (*bp)); + memset(bp, 0, sizeof (*bp)); bp->blk_prop = sio->sio_blk_prop; bp->blk_phys_birth = sio->sio_phys_birth; bp->blk_birth = sio->sio_birth; @@ -403,7 +406,7 @@ sio2bp(const scan_io_t *sio, blkptr_t *bp) ASSERT3U(sio->sio_nr_dvas, >, 0); ASSERT3U(sio->sio_nr_dvas, <=, SPA_DVAS_PER_BP); - bcopy(sio->sio_dva, bp->blk_dva, sio->sio_nr_dvas * sizeof (dva_t)); + memcpy(bp->blk_dva, sio->sio_dva, sio->sio_nr_dvas * sizeof (dva_t)); } static inline void @@ -508,7 +511,7 @@ dsl_scan_init(dsl_pool_t *dp, uint64_t txg) return (EOVERFLOW); } - bcopy(zaptmp, &scn->scn_phys, + memcpy(&scn->scn_phys, zaptmp, SCAN_PHYS_NUMINTS * sizeof (uint64_t)); scn->scn_phys.scn_flags = overflow; @@ -567,7 +570,7 @@ dsl_scan_init(dsl_pool_t *dp, uint64_t txg) } } - bcopy(&scn->scn_phys, &scn->scn_phys_cached, sizeof (scn->scn_phys)); + memcpy(&scn->scn_phys_cached, &scn->scn_phys, sizeof (scn->scn_phys)); /* reload the queue into the in-core state */ if (scn->scn_phys.scn_queue_obj != 0) { @@ -643,7 +646,7 @@ dsl_scan_is_paused_scrub(const dsl_scan_t *scn) * Because we can be running in the block sorting algorithm, we do not always * want to write out the record, only when it is "safe" to do so. This safety * condition is achieved by making sure that the sorting queues are empty - * (scn_bytes_pending == 0). When this condition is not true, the sync'd state + * (scn_queues_pending == 0). When this condition is not true, the sync'd state * is inconsistent with how much actual scanning progress has been made. The * kind of sync to be performed is specified by the sync_type argument. If the * sync is optional, we only sync if the queues are empty. If the sync is @@ -666,8 +669,8 @@ dsl_scan_sync_state(dsl_scan_t *scn, dmu_tx_t *tx, state_sync_type_t sync_type) int i; spa_t *spa = scn->scn_dp->dp_spa; - ASSERT(sync_type != SYNC_MANDATORY || scn->scn_bytes_pending == 0); - if (scn->scn_bytes_pending == 0) { + ASSERT(sync_type != SYNC_MANDATORY || scn->scn_queues_pending == 0); + if (scn->scn_queues_pending == 0) { for (i = 0; i < spa->spa_root_vdev->vdev_children; i++) { vdev_t *vd = spa->spa_root_vdev->vdev_child[i]; dsl_scan_io_queue_t *q = vd->vdev_scan_io_queue; @@ -689,7 +692,7 @@ dsl_scan_sync_state(dsl_scan_t *scn, dmu_tx_t *tx, state_sync_type_t sync_type) DMU_POOL_DIRECTORY_OBJECT, DMU_POOL_SCAN, sizeof (uint64_t), SCAN_PHYS_NUMINTS, &scn->scn_phys, tx)); - bcopy(&scn->scn_phys, &scn->scn_phys_cached, + memcpy(&scn->scn_phys_cached, &scn->scn_phys, sizeof (scn->scn_phys)); if (scn->scn_checkpointing) @@ -730,7 +733,7 @@ dsl_scan_setup_sync(void *arg, dmu_tx_t *tx) ASSERT(!dsl_scan_is_running(scn)); ASSERT(*funcp > POOL_SCAN_NONE && *funcp < POOL_SCAN_FUNCS); - bzero(&scn->scn_phys, sizeof (scn->scn_phys)); + memset(&scn->scn_phys, 0, sizeof (scn->scn_phys)); scn->scn_phys.scn_func = *funcp; scn->scn_phys.scn_state = DSS_SCANNING; scn->scn_phys.scn_min_txg = 0; @@ -792,13 +795,19 @@ dsl_scan_setup_sync(void *arg, dmu_tx_t *tx) /* back to the generic stuff */ - if (dp->dp_blkstats == NULL) { - dp->dp_blkstats = - vmem_alloc(sizeof (zfs_all_blkstats_t), KM_SLEEP); - mutex_init(&dp->dp_blkstats->zab_lock, NULL, - MUTEX_DEFAULT, NULL); + if (zfs_scan_blkstats) { + if (dp->dp_blkstats == NULL) { + dp->dp_blkstats = + vmem_alloc(sizeof (zfs_all_blkstats_t), KM_SLEEP); + } + memset(&dp->dp_blkstats->zab_type, 0, + sizeof (dp->dp_blkstats->zab_type)); + } else { + if (dp->dp_blkstats) { + vmem_free(dp->dp_blkstats, sizeof (zfs_all_blkstats_t)); + dp->dp_blkstats = NULL; + } } - bzero(&dp->dp_blkstats->zab_type, sizeof (dp->dp_blkstats->zab_type)); if (spa_version(spa) < SPA_VERSION_DSL_SCRUB) ot = DMU_OT_ZAP_OTHER; @@ -806,7 +815,7 @@ dsl_scan_setup_sync(void *arg, dmu_tx_t *tx) scn->scn_phys.scn_queue_obj = zap_create(dp->dp_meta_objset, ot ? ot : DMU_OT_SCAN_QUEUE, DMU_OT_NONE, 0, tx); - bcopy(&scn->scn_phys, &scn->scn_phys_cached, sizeof (scn->scn_phys)); + memcpy(&scn->scn_phys_cached, &scn->scn_phys, sizeof (scn->scn_phys)); dsl_scan_sync_state(scn, tx, SYNC_MANDATORY); @@ -1204,7 +1213,7 @@ scan_ds_queue_sync(dsl_scan_t *scn, dmu_tx_t *tx) dmu_object_type_t ot = (spa_version(spa) >= SPA_VERSION_DSL_SCRUB) ? DMU_OT_SCAN_QUEUE : DMU_OT_ZAP_OTHER; - ASSERT0(scn->scn_bytes_pending); + ASSERT0(scn->scn_queues_pending); ASSERT(scn->scn_phys.scn_queue_obj != 0); VERIFY0(dmu_object_free(dp->dp_meta_objset, @@ -1275,9 +1284,13 @@ dsl_scan_should_clear(dsl_scan_t *scn) mutex_enter(&tvd->vdev_scan_io_queue_lock); queue = tvd->vdev_scan_io_queue; if (queue != NULL) { - /* # extents in exts_by_size = # in exts_by_addr */ + /* + * # of extents in exts_by_addr = # in exts_by_size. + * B-tree efficiency is ~75%, but can be as low as 50%. + */ mused += zfs_btree_numnodes(&queue->q_exts_by_size) * - sizeof (range_seg_gap_t) + queue->q_sio_memused; + ((sizeof (range_seg_gap_t) + sizeof (uint64_t)) * + 3 / 2) + queue->q_sio_memused; } mutex_exit(&tvd->vdev_scan_io_queue_lock); } @@ -1285,7 +1298,7 @@ dsl_scan_should_clear(dsl_scan_t *scn) dprintf("current scan memory usage: %llu bytes\n", (longlong_t)mused); if (mused == 0) - ASSERT0(scn->scn_bytes_pending); + ASSERT0(scn->scn_queues_pending); /* * If we are above our hard limit, we need to clear out memory. @@ -1335,12 +1348,13 @@ dsl_scan_check_suspend(dsl_scan_t *scn, const zbookmark_phys_t *zb) uint64_t scan_time_ns = curr_time_ns - scn->scn_sync_start_time; uint64_t sync_time_ns = curr_time_ns - scn->scn_dp->dp_spa->spa_sync_starttime; - int dirty_pct = scn->scn_dp->dp_dirty_total * 100 / zfs_dirty_data_max; + uint64_t dirty_min_bytes = zfs_dirty_data_max * + zfs_vdev_async_write_active_min_dirty_percent / 100; int mintime = (scn->scn_phys.scn_func == POOL_SCAN_RESILVER) ? zfs_resilver_min_time_ms : zfs_scrub_min_time_ms; if ((NSEC2MSEC(scan_time_ns) > mintime && - (dirty_pct >= zfs_vdev_async_write_active_min_dirty_percent || + (scn->scn_dp->dp_dirty_total >= dirty_min_bytes || txg_sync_waiting(scn->scn_dp) || NSEC2SEC(sync_time_ns) >= zfs_txg_timeout)) || spa_shutting_down(scn->scn_dp->dp_spa) || @@ -1491,7 +1505,7 @@ scan_prefetch_queue_compare(const void *a, const void *b) } static void -scan_prefetch_ctx_rele(scan_prefetch_ctx_t *spc, void *tag) +scan_prefetch_ctx_rele(scan_prefetch_ctx_t *spc, const void *tag) { if (zfs_refcount_remove(&spc->spc_refcnt, tag) == 0) { zfs_refcount_destroy(&spc->spc_refcnt); @@ -1500,7 +1514,7 @@ scan_prefetch_ctx_rele(scan_prefetch_ctx_t *spc, void *tag) } static scan_prefetch_ctx_t * -scan_prefetch_ctx_create(dsl_scan_t *scn, dnode_phys_t *dnp, void *tag) +scan_prefetch_ctx_create(dsl_scan_t *scn, dnode_phys_t *dnp, const void *tag) { scan_prefetch_ctx_t *spc; @@ -1522,7 +1536,7 @@ scan_prefetch_ctx_create(dsl_scan_t *scn, dnode_phys_t *dnp, void *tag) } static void -scan_prefetch_ctx_add_ref(scan_prefetch_ctx_t *spc, void *tag) +scan_prefetch_ctx_add_ref(scan_prefetch_ctx_t *spc, const void *tag) { zfs_refcount_add(&spc->spc_refcnt, tag); } @@ -1788,18 +1802,17 @@ dsl_scan_check_resume(dsl_scan_t *scn, const dnode_phys_t *dnp, /* * If we found the block we're trying to resume from, or - * we went past it to a different object, zero it out to - * indicate that it's OK to start checking for suspending - * again. + * we went past it, zero it out to indicate that it's OK + * to start checking for suspending again. */ - if (bcmp(zb, &scn->scn_phys.scn_bookmark, sizeof (*zb)) == 0 || - zb->zb_object > scn->scn_phys.scn_bookmark.zb_object) { + if (zbookmark_subtree_tbd(dnp, zb, + &scn->scn_phys.scn_bookmark)) { dprintf("resuming at %llx/%llx/%llx/%llx\n", (longlong_t)zb->zb_objset, (longlong_t)zb->zb_object, (longlong_t)zb->zb_level, (longlong_t)zb->zb_blkid); - bzero(&scn->scn_phys.scn_bookmark, sizeof (*zb)); + memset(&scn->scn_phys.scn_bookmark, 0, sizeof (*zb)); } } return (B_FALSE); @@ -1822,6 +1835,7 @@ dsl_scan_recurse(dsl_scan_t *scn, dsl_dataset_t *ds, dmu_objset_type_t ostype, const zbookmark_phys_t *zb, dmu_tx_t *tx) { dsl_pool_t *dp = scn->scn_dp; + spa_t *spa = dp->dp_spa; int zio_flags = ZIO_FLAG_CANFAIL | ZIO_FLAG_SCAN_THREAD; int err; @@ -1836,7 +1850,7 @@ dsl_scan_recurse(dsl_scan_t *scn, dsl_dataset_t *ds, dmu_objset_type_t ostype, if (dnp != NULL && dnp->dn_bonuslen > DN_MAX_BONUS_LEN(dnp)) { scn->scn_phys.scn_errors++; - spa_log_error(dp->dp_spa, zb); + spa_log_error(spa, zb); return (SET_ERROR(EINVAL)); } @@ -1847,7 +1861,7 @@ dsl_scan_recurse(dsl_scan_t *scn, dsl_dataset_t *ds, dmu_objset_type_t ostype, int epb = BP_GET_LSIZE(bp) >> SPA_BLKPTRSHIFT; arc_buf_t *buf; - err = arc_read(NULL, dp->dp_spa, bp, arc_getbuf_func, &buf, + err = arc_read(NULL, spa, bp, arc_getbuf_func, &buf, ZIO_PRIORITY_SCRUB, zio_flags, &flags, zb); if (err) { scn->scn_phys.scn_errors++; @@ -1875,7 +1889,7 @@ dsl_scan_recurse(dsl_scan_t *scn, dsl_dataset_t *ds, dmu_objset_type_t ostype, zio_flags |= ZIO_FLAG_RAW; } - err = arc_read(NULL, dp->dp_spa, bp, arc_getbuf_func, &buf, + err = arc_read(NULL, spa, bp, arc_getbuf_func, &buf, ZIO_PRIORITY_SCRUB, zio_flags, &flags, zb); if (err) { scn->scn_phys.scn_errors++; @@ -1894,7 +1908,7 @@ dsl_scan_recurse(dsl_scan_t *scn, dsl_dataset_t *ds, dmu_objset_type_t ostype, objset_phys_t *osp; arc_buf_t *buf; - err = arc_read(NULL, dp->dp_spa, bp, arc_getbuf_func, &buf, + err = arc_read(NULL, spa, bp, arc_getbuf_func, &buf, ZIO_PRIORITY_SCRUB, zio_flags, &flags, zb); if (err) { scn->scn_phys.scn_errors++; @@ -1925,6 +1939,14 @@ dsl_scan_recurse(dsl_scan_t *scn, dsl_dataset_t *ds, dmu_objset_type_t ostype, DMU_USERUSED_OBJECT, tx); } arc_buf_destroy(buf, &buf); + } else if (!zfs_blkptr_verify(spa, bp, B_FALSE, BLK_VERIFY_LOG)) { + /* + * Sanity check the block pointer contents, this is handled + * by arc_read() for the cases above. + */ + scn->scn_phys.scn_errors++; + spa_log_error(spa, zb); + return (SET_ERROR(EINVAL)); } return (0); @@ -1975,19 +1997,6 @@ dsl_scan_visitbp(blkptr_t *bp, const zbookmark_phys_t *zb, scn->scn_visited_this_txg++; - /* - * This debugging is commented out to conserve stack space. This - * function is called recursively and the debugging adds several - * bytes to the stack for each call. It can be commented back in - * if required to debug an issue in dsl_scan_visitbp(). - * - * dprintf_bp(bp, - * "visiting ds=%p/%llu zb=%llx/%llx/%llx/%llx bp=%p", - * ds, ds ? ds->ds_object : 0, - * zb->zb_objset, zb->zb_object, zb->zb_level, zb->zb_blkid, - * bp); - */ - if (BP_IS_HOLE(bp)) { scn->scn_holes_this_txg++; return; @@ -1999,6 +2008,21 @@ dsl_scan_visitbp(blkptr_t *bp, const zbookmark_phys_t *zb, return; } + /* + * Check if this block contradicts any filesystem flags. + */ + spa_feature_t f = SPA_FEATURE_LARGE_BLOCKS; + if (BP_GET_LSIZE(bp) > SPA_OLD_MAXBLOCKSIZE) + ASSERT3B(dsl_dataset_feature_is_active(ds, f), ==, B_TRUE); + + f = zio_checksum_to_feature(BP_GET_CHECKSUM(bp)); + if (f != SPA_FEATURE_NONE) + ASSERT3B(dsl_dataset_feature_is_active(ds, f), ==, B_TRUE); + + f = zio_compress_to_feature(BP_GET_COMPRESS(bp)); + if (f != SPA_FEATURE_NONE) + ASSERT3B(dsl_dataset_feature_is_active(ds, f), ==, B_TRUE); + if (bp->blk_birth <= scn->scn_phys.scn_cur_min_txg) { scn->scn_lt_min_this_txg++; return; @@ -2651,12 +2675,10 @@ static void dsl_scan_ddt(dsl_scan_t *scn, dmu_tx_t *tx) { ddt_bookmark_t *ddb = &scn->scn_phys.scn_ddt_bookmark; - ddt_entry_t dde; + ddt_entry_t dde = {{{{0}}}}; int error; uint64_t n = 0; - bzero(&dde, sizeof (ddt_entry_t)); - while ((error = ddt_walk(scn->scn_dp->dp_spa, ddb, &dde)) == 0) { ddt_t *ddt; @@ -2749,7 +2771,7 @@ dsl_scan_visit(dsl_scan_t *scn, dmu_tx_t *tx) * In case we suspended right at the end of the ds, zero the * bookmark so we don't think that we're still trying to resume. */ - bzero(&scn->scn_phys.scn_bookmark, sizeof (zbookmark_phys_t)); + memset(&scn->scn_phys.scn_bookmark, 0, sizeof (zbookmark_phys_t)); /* * Keep pulling things out of the dataset avl queue. Updates to the @@ -2831,12 +2853,13 @@ scan_io_queue_check_suspend(dsl_scan_t *scn) uint64_t scan_time_ns = curr_time_ns - scn->scn_sync_start_time; uint64_t sync_time_ns = curr_time_ns - scn->scn_dp->dp_spa->spa_sync_starttime; - int dirty_pct = scn->scn_dp->dp_dirty_total * 100 / zfs_dirty_data_max; + uint64_t dirty_min_bytes = zfs_dirty_data_max * + zfs_vdev_async_write_active_min_dirty_percent / 100; int mintime = (scn->scn_phys.scn_func == POOL_SCAN_RESILVER) ? zfs_resilver_min_time_ms : zfs_scrub_min_time_ms; return ((NSEC2MSEC(scan_time_ns) > mintime && - (dirty_pct >= zfs_vdev_async_write_active_min_dirty_percent || + (scn->scn_dp->dp_dirty_total >= dirty_min_bytes || txg_sync_waiting(scn->scn_dp) || NSEC2SEC(sync_time_ns) >= zfs_txg_timeout)) || spa_shutting_down(scn->scn_dp->dp_spa)); @@ -2855,7 +2878,6 @@ scan_io_queue_issue(dsl_scan_io_queue_t *queue, list_t *io_list) { dsl_scan_t *scn = queue->q_scn; scan_io_t *sio; - int64_t bytes_issued = 0; boolean_t suspended = B_FALSE; while ((sio = list_head(io_list)) != NULL) { @@ -2867,16 +2889,12 @@ scan_io_queue_issue(dsl_scan_io_queue_t *queue, list_t *io_list) } sio2bp(sio, &bp); - bytes_issued += SIO_GET_ASIZE(sio); scan_exec_io(scn->scn_dp, &bp, sio->sio_flags, &sio->sio_zb, queue); (void) list_remove_head(io_list); scan_io_queues_update_zio_stats(queue, &bp); sio_free(sio); } - - atomic_add_64(&scn->scn_bytes_pending, -bytes_issued); - return (suspended); } @@ -2921,6 +2939,8 @@ scan_io_queue_gather(dsl_scan_io_queue_t *queue, range_seg_t *rs, list_t *list) next_sio = AVL_NEXT(&queue->q_sios_by_addr, sio); avl_remove(&queue->q_sios_by_addr, sio); + if (avl_is_empty(&queue->q_sios_by_addr)) + atomic_add_64(&queue->q_scn->scn_queues_pending, -1); queue->q_sio_memused -= SIO_GET_MUSED(sio); bytes_issued += SIO_GET_ASIZE(sio); @@ -2942,12 +2962,13 @@ scan_io_queue_gather(dsl_scan_io_queue_t *queue, range_seg_t *rs, list_t *list) range_tree_resize_segment(queue->q_exts_by_addr, rs, SIO_GET_OFFSET(sio), rs_get_end(rs, queue->q_exts_by_addr) - SIO_GET_OFFSET(sio)); - + queue->q_last_ext_addr = SIO_GET_OFFSET(sio); return (B_TRUE); } else { uint64_t rstart = rs_get_start(rs, queue->q_exts_by_addr); uint64_t rend = rs_get_end(rs, queue->q_exts_by_addr); range_tree_remove(queue->q_exts_by_addr, rstart, rend - rstart); + queue->q_last_ext_addr = -1; return (B_FALSE); } } @@ -2972,31 +2993,8 @@ scan_io_queue_fetch_ext(dsl_scan_io_queue_t *queue) ASSERT(MUTEX_HELD(&queue->q_vd->vdev_scan_io_queue_lock)); ASSERT(scn->scn_is_sorted); - /* handle tunable overrides */ - if (scn->scn_checkpointing || scn->scn_clearing) { - if (zfs_scan_issue_strategy == 1) { - return (range_tree_first(rt)); - } else if (zfs_scan_issue_strategy == 2) { - /* - * We need to get the original entry in the by_addr - * tree so we can modify it. - */ - range_seg_t *size_rs = - zfs_btree_first(&queue->q_exts_by_size, NULL); - if (size_rs == NULL) - return (NULL); - uint64_t start = rs_get_start(size_rs, rt); - uint64_t size = rs_get_end(size_rs, rt) - start; - range_seg_t *addr_rs = range_tree_find(rt, start, - size); - ASSERT3P(addr_rs, !=, NULL); - ASSERT3U(rs_get_start(size_rs, rt), ==, - rs_get_start(addr_rs, rt)); - ASSERT3U(rs_get_end(size_rs, rt), ==, - rs_get_end(addr_rs, rt)); - return (addr_rs); - } - } + if (!scn->scn_checkpointing && !scn->scn_clearing) + return (NULL); /* * During normal clearing, we want to issue our largest segments @@ -3007,28 +3005,42 @@ scan_io_queue_fetch_ext(dsl_scan_io_queue_t *queue) * so the way we are sorted now is as good as it will ever get. * In this case, we instead switch to issuing extents in LBA order. */ - if (scn->scn_checkpointing) { + if ((zfs_scan_issue_strategy < 1 && scn->scn_checkpointing) || + zfs_scan_issue_strategy == 1) return (range_tree_first(rt)); - } else if (scn->scn_clearing) { - /* - * We need to get the original entry in the by_addr - * tree so we can modify it. - */ - range_seg_t *size_rs = zfs_btree_first(&queue->q_exts_by_size, - NULL); - if (size_rs == NULL) - return (NULL); - uint64_t start = rs_get_start(size_rs, rt); - uint64_t size = rs_get_end(size_rs, rt) - start; - range_seg_t *addr_rs = range_tree_find(rt, start, size); - ASSERT3P(addr_rs, !=, NULL); - ASSERT3U(rs_get_start(size_rs, rt), ==, rs_get_start(addr_rs, - rt)); - ASSERT3U(rs_get_end(size_rs, rt), ==, rs_get_end(addr_rs, rt)); - return (addr_rs); - } else { - return (NULL); + + /* + * Try to continue previous extent if it is not completed yet. After + * shrink in scan_io_queue_gather() it may no longer be the best, but + * otherwise we leave shorter remnant every txg. + */ + uint64_t start; + uint64_t size = 1 << rt->rt_shift; + range_seg_t *addr_rs; + if (queue->q_last_ext_addr != -1) { + start = queue->q_last_ext_addr; + addr_rs = range_tree_find(rt, start, size); + if (addr_rs != NULL) + return (addr_rs); } + + /* + * Nothing to continue, so find new best extent. + */ + uint64_t *v = zfs_btree_first(&queue->q_exts_by_size, NULL); + if (v == NULL) + return (NULL); + queue->q_last_ext_addr = start = *v << rt->rt_shift; + + /* + * We need to get the original entry in the by_addr tree so we can + * modify it. + */ + addr_rs = range_tree_find(rt, start, size); + ASSERT3P(addr_rs, !=, NULL); + ASSERT3U(rs_get_start(addr_rs, rt), ==, start); + ASSERT3U(rs_get_end(addr_rs, rt), >, start); + return (addr_rs); } static void @@ -3037,15 +3049,19 @@ scan_io_queues_run_one(void *arg) dsl_scan_io_queue_t *queue = arg; kmutex_t *q_lock = &queue->q_vd->vdev_scan_io_queue_lock; boolean_t suspended = B_FALSE; - range_seg_t *rs = NULL; - scan_io_t *sio = NULL; + range_seg_t *rs; + scan_io_t *sio; + zio_t *zio; list_t sio_list; ASSERT(queue->q_scn->scn_is_sorted); list_create(&sio_list, sizeof (scan_io_t), offsetof(scan_io_t, sio_nodes.sio_list_node)); + zio = zio_null(queue->q_scn->scn_zio_root, queue->q_scn->scn_dp->dp_spa, + NULL, NULL, NULL, ZIO_FLAG_CANFAIL); mutex_enter(q_lock); + queue->q_zio = zio; /* Calculate maximum in-flight bytes for this vdev. */ queue->q_maxinflight_bytes = MAX(1, zfs_scan_vdev_limit * @@ -3060,12 +3076,12 @@ scan_io_queues_run_one(void *arg) /* loop until we run out of time or sios */ while ((rs = scan_io_queue_fetch_ext(queue)) != NULL) { uint64_t seg_start = 0, seg_end = 0; - boolean_t more_left = B_TRUE; + boolean_t more_left; ASSERT(list_is_empty(&sio_list)); /* loop while we still have sios left to process in this rs */ - while (more_left) { + do { scan_io_t *first_sio, *last_sio; /* @@ -3094,7 +3110,7 @@ scan_io_queues_run_one(void *arg) if (suspended) break; - } + } while (more_left); /* update statistics for debugging purposes */ scan_io_queues_update_seg_stats(queue, seg_start, seg_end); @@ -3112,7 +3128,9 @@ scan_io_queues_run_one(void *arg) scan_io_queue_insert_impl(queue, sio); } + queue->q_zio = NULL; mutex_exit(q_lock); + zio_nowait(zio); list_destroy(&sio_list); } @@ -3133,7 +3151,7 @@ scan_io_queues_run(dsl_scan_t *scn) ASSERT(scn->scn_is_sorted); ASSERT(spa_config_held(spa, SCL_CONFIG, RW_READER)); - if (scn->scn_bytes_pending == 0) + if (scn->scn_queues_pending == 0) return; if (scn->scn_taskq == NULL) { @@ -3765,7 +3783,7 @@ dsl_scan_sync(dsl_pool_t *dp, dmu_tx_t *tx) spa->spa_name, (longlong_t)tx->tx_txg); } - } else if (scn->scn_is_sorted && scn->scn_bytes_pending != 0) { + } else if (scn->scn_is_sorted && scn->scn_queues_pending != 0) { ASSERT(scn->scn_clearing); /* need to issue scrubbing IOs from per-vdev queues */ @@ -3795,7 +3813,7 @@ dsl_scan_sync(dsl_pool_t *dp, dmu_tx_t *tx) spa->spa_name); ASSERT3U(scn->scn_done_txg, !=, 0); ASSERT0(spa->spa_scrub_inflight); - ASSERT0(scn->scn_bytes_pending); + ASSERT0(scn->scn_queues_pending); dsl_scan_done(scn, B_TRUE, tx); sync_type = SYNC_MANDATORY; } @@ -3804,10 +3822,8 @@ dsl_scan_sync(dsl_pool_t *dp, dmu_tx_t *tx) } static void -count_block(dsl_scan_t *scn, zfs_all_blkstats_t *zab, const blkptr_t *bp) +count_block_issued(spa_t *spa, const blkptr_t *bp, boolean_t all) { - int i; - /* * Don't count embedded bp's, since we already did the work of * scanning these when we scanned the containing block. @@ -3822,18 +3838,13 @@ count_block(dsl_scan_t *scn, zfs_all_blkstats_t *zab, const blkptr_t *bp) * zio code will only try the first one unless there is an issue. * Therefore, we should only count the first DVA for these IOs. */ - if (scn->scn_is_sorted) { - atomic_add_64(&scn->scn_dp->dp_spa->spa_scan_pass_issued, - DVA_GET_ASIZE(&bp->blk_dva[0])); - } else { - spa_t *spa = scn->scn_dp->dp_spa; - - for (i = 0; i < BP_GET_NDVAS(bp); i++) { - atomic_add_64(&spa->spa_scan_pass_issued, - DVA_GET_ASIZE(&bp->blk_dva[i])); - } - } + atomic_add_64(&spa->spa_scan_pass_issued, + all ? BP_GET_ASIZE(bp) : DVA_GET_ASIZE(&bp->blk_dva[0])); +} +static void +count_block(zfs_all_blkstats_t *zab, const blkptr_t *bp) +{ /* * If we resume after a reboot, zab will be NULL; don't record * incomplete stats in that case. @@ -3841,9 +3852,7 @@ count_block(dsl_scan_t *scn, zfs_all_blkstats_t *zab, const blkptr_t *bp) if (zab == NULL) return; - mutex_enter(&zab->zab_lock); - - for (i = 0; i < 4; i++) { + for (int i = 0; i < 4; i++) { int l = (i < 2) ? BP_GET_LEVEL(bp) : DN_MAX_LEVELS; int t = (i & 1) ? BP_GET_TYPE(bp) : DMU_OT_TOTAL; @@ -3878,28 +3887,27 @@ count_block(dsl_scan_t *scn, zfs_all_blkstats_t *zab, const blkptr_t *bp) break; } } - - mutex_exit(&zab->zab_lock); } static void scan_io_queue_insert_impl(dsl_scan_io_queue_t *queue, scan_io_t *sio) { avl_index_t idx; - int64_t asize = SIO_GET_ASIZE(sio); dsl_scan_t *scn = queue->q_scn; ASSERT(MUTEX_HELD(&queue->q_vd->vdev_scan_io_queue_lock)); + if (unlikely(avl_is_empty(&queue->q_sios_by_addr))) + atomic_add_64(&scn->scn_queues_pending, 1); if (avl_find(&queue->q_sios_by_addr, sio, &idx) != NULL) { /* block is already scheduled for reading */ - atomic_add_64(&scn->scn_bytes_pending, -asize); sio_free(sio); return; } avl_insert(&queue->q_sios_by_addr, sio, idx); queue->q_sio_memused += SIO_GET_MUSED(sio); - range_tree_add(queue->q_exts_by_addr, SIO_GET_OFFSET(sio), asize); + range_tree_add(queue->q_exts_by_addr, SIO_GET_OFFSET(sio), + SIO_GET_ASIZE(sio)); } /* @@ -3912,7 +3920,6 @@ static void scan_io_queue_insert(dsl_scan_io_queue_t *queue, const blkptr_t *bp, int dva_i, int zio_flags, const zbookmark_phys_t *zb) { - dsl_scan_t *scn = queue->q_scn; scan_io_t *sio = sio_alloc(BP_GET_NDVAS(bp)); ASSERT0(BP_IS_GANG(bp)); @@ -3922,13 +3929,7 @@ scan_io_queue_insert(dsl_scan_io_queue_t *queue, const blkptr_t *bp, int dva_i, sio->sio_flags = zio_flags; sio->sio_zb = *zb; - /* - * Increment the bytes pending counter now so that we can't - * get an integer underflow in case the worker processes the - * zio before we get to incrementing this counter. - */ - atomic_add_64(&scn->scn_bytes_pending, SIO_GET_ASIZE(sio)); - + queue->q_last_ext_addr = -1; scan_io_queue_insert_impl(queue, sio); } @@ -3983,10 +3984,10 @@ dsl_scan_scrub_cb(dsl_pool_t *dp, boolean_t needs_io = B_FALSE; int zio_flags = ZIO_FLAG_SCAN_THREAD | ZIO_FLAG_RAW | ZIO_FLAG_CANFAIL; - + count_block(dp->dp_blkstats, bp); if (phys_birth <= scn->scn_phys.scn_min_txg || phys_birth >= scn->scn_phys.scn_max_txg) { - count_block(scn, dp->dp_blkstats, bp); + count_block_issued(spa, bp, B_TRUE); return (0); } @@ -4014,8 +4015,9 @@ dsl_scan_scrub_cb(dsl_pool_t *dp, * Keep track of how much data we've examined so that * zpool(8) status can make useful progress reports. */ - scn->scn_phys.scn_examined += DVA_GET_ASIZE(dva); - spa->spa_scan_pass_exam += DVA_GET_ASIZE(dva); + uint64_t asize = DVA_GET_ASIZE(dva); + scn->scn_phys.scn_examined += asize; + spa->spa_scan_pass_exam += asize; /* if it's a resilver, this may not be in the target range */ if (!needs_io) @@ -4026,7 +4028,7 @@ dsl_scan_scrub_cb(dsl_pool_t *dp, if (needs_io && !zfs_no_scrub_io) { dsl_scan_enqueue(dp, bp, zio_flags, zb); } else { - count_block(scn, dp->dp_blkstats, bp); + count_block_issued(spa, bp, B_TRUE); } /* do not relocate this block */ @@ -4077,6 +4079,7 @@ scan_exec_io(dsl_pool_t *dp, const blkptr_t *bp, int zio_flags, dsl_scan_t *scn = dp->dp_scan; size_t size = BP_GET_PSIZE(bp); abd_t *data = abd_alloc_for_io(size, B_FALSE); + zio_t *pio; if (queue == NULL) { ASSERT3U(scn->scn_maxinflight_bytes, >, 0); @@ -4085,6 +4088,7 @@ scan_exec_io(dsl_pool_t *dp, const blkptr_t *bp, int zio_flags, cv_wait(&spa->spa_scrub_io_cv, &spa->spa_scrub_lock); spa->spa_scrub_inflight += BP_GET_PSIZE(bp); mutex_exit(&spa->spa_scrub_lock); + pio = scn->scn_zio_root; } else { kmutex_t *q_lock = &queue->q_vd->vdev_scan_io_queue_lock; @@ -4093,12 +4097,14 @@ scan_exec_io(dsl_pool_t *dp, const blkptr_t *bp, int zio_flags, while (queue->q_inflight_bytes >= queue->q_maxinflight_bytes) cv_wait(&queue->q_zio_cv, q_lock); queue->q_inflight_bytes += BP_GET_PSIZE(bp); + pio = queue->q_zio; mutex_exit(q_lock); } - count_block(scn, dp->dp_blkstats, bp); - zio_nowait(zio_read(scn->scn_zio_root, spa, bp, data, size, - dsl_scan_scrub_done, queue, ZIO_PRIORITY_SCRUB, zio_flags, zb)); + ASSERT(pio != NULL); + count_block_issued(spa, bp, queue == NULL); + zio_nowait(zio_read(pio, spa, bp, data, size, dsl_scan_scrub_done, + queue, ZIO_PRIORITY_SCRUB, zio_flags, zb)); } /* @@ -4132,33 +4138,88 @@ scan_exec_io(dsl_pool_t *dp, const blkptr_t *bp, int zio_flags, * extents that are more completely filled (in a 3:2 ratio) vs just larger. * Note that as an optimization, we replace multiplication and division by * 100 with bitshifting by 7 (which effectively multiplies and divides by 128). + * + * Since we do not care if one extent is only few percent better than another, + * compress the score into 6 bits via binary logarithm AKA highbit64() and + * put into otherwise unused due to ashift high bits of offset. This allows + * to reduce q_exts_by_size B-tree elements to only 64 bits and compare them + * with single operation. Plus it makes scrubs more sequential and reduces + * chances that minor extent change move it within the B-tree. */ static int ext_size_compare(const void *x, const void *y) { - const range_seg_gap_t *rsa = x, *rsb = y; + const uint64_t *a = x, *b = y; - uint64_t sa = rsa->rs_end - rsa->rs_start; - uint64_t sb = rsb->rs_end - rsb->rs_start; - uint64_t score_a, score_b; + return (TREE_CMP(*a, *b)); +} - score_a = rsa->rs_fill + ((((rsa->rs_fill << 7) / sa) * - fill_weight * rsa->rs_fill) >> 7); - score_b = rsb->rs_fill + ((((rsb->rs_fill << 7) / sb) * - fill_weight * rsb->rs_fill) >> 7); +static void +ext_size_create(range_tree_t *rt, void *arg) +{ + (void) rt; + zfs_btree_t *size_tree = arg; - if (score_a > score_b) - return (-1); - if (score_a == score_b) { - if (rsa->rs_start < rsb->rs_start) - return (-1); - if (rsa->rs_start == rsb->rs_start) - return (0); - return (1); - } - return (1); + zfs_btree_create(size_tree, ext_size_compare, sizeof (uint64_t)); +} + +static void +ext_size_destroy(range_tree_t *rt, void *arg) +{ + (void) rt; + zfs_btree_t *size_tree = arg; + ASSERT0(zfs_btree_numnodes(size_tree)); + + zfs_btree_destroy(size_tree); } +static uint64_t +ext_size_value(range_tree_t *rt, range_seg_gap_t *rsg) +{ + (void) rt; + uint64_t size = rsg->rs_end - rsg->rs_start; + uint64_t score = rsg->rs_fill + ((((rsg->rs_fill << 7) / size) * + fill_weight * rsg->rs_fill) >> 7); + ASSERT3U(rt->rt_shift, >=, 8); + return (((uint64_t)(64 - highbit64(score)) << 56) | rsg->rs_start); +} + +static void +ext_size_add(range_tree_t *rt, range_seg_t *rs, void *arg) +{ + zfs_btree_t *size_tree = arg; + ASSERT3U(rt->rt_type, ==, RANGE_SEG_GAP); + uint64_t v = ext_size_value(rt, (range_seg_gap_t *)rs); + zfs_btree_add(size_tree, &v); +} + +static void +ext_size_remove(range_tree_t *rt, range_seg_t *rs, void *arg) +{ + zfs_btree_t *size_tree = arg; + ASSERT3U(rt->rt_type, ==, RANGE_SEG_GAP); + uint64_t v = ext_size_value(rt, (range_seg_gap_t *)rs); + zfs_btree_remove(size_tree, &v); +} + +static void +ext_size_vacate(range_tree_t *rt, void *arg) +{ + zfs_btree_t *size_tree = arg; + zfs_btree_clear(size_tree); + zfs_btree_destroy(size_tree); + + ext_size_create(rt, arg); +} + +static const range_tree_ops_t ext_size_ops = { + .rtop_create = ext_size_create, + .rtop_destroy = ext_size_destroy, + .rtop_add = ext_size_add, + .rtop_remove = ext_size_remove, + .rtop_vacate = ext_size_vacate +}; + /* * Comparator for the q_sios_by_addr tree. Sorting is simply performed * based on LBA-order (from lowest to highest). @@ -4181,9 +4242,10 @@ scan_io_queue_create(vdev_t *vd) q->q_scn = scn; q->q_vd = vd; q->q_sio_memused = 0; + q->q_last_ext_addr = -1; cv_init(&q->q_zio_cv, NULL, CV_DEFAULT, NULL); - q->q_exts_by_addr = range_tree_create_impl(&rt_btree_ops, RANGE_SEG_GAP, - &q->q_exts_by_size, 0, 0, ext_size_compare, zfs_scan_max_ext_gap); + q->q_exts_by_addr = range_tree_create_gap(&ext_size_ops, RANGE_SEG_GAP, + &q->q_exts_by_size, 0, vd->vdev_ashift, zfs_scan_max_ext_gap); avl_create(&q->q_sios_by_addr, sio_addr_compare, sizeof (scan_io_t), offsetof(scan_io_t, sio_nodes.sio_addr_node)); @@ -4201,21 +4263,20 @@ dsl_scan_io_queue_destroy(dsl_scan_io_queue_t *queue) dsl_scan_t *scn = queue->q_scn; scan_io_t *sio; void *cookie = NULL; - int64_t bytes_dequeued = 0; ASSERT(MUTEX_HELD(&queue->q_vd->vdev_scan_io_queue_lock)); + if (!avl_is_empty(&queue->q_sios_by_addr)) + atomic_add_64(&scn->scn_queues_pending, -1); while ((sio = avl_destroy_nodes(&queue->q_sios_by_addr, &cookie)) != NULL) { ASSERT(range_tree_contains(queue->q_exts_by_addr, SIO_GET_OFFSET(sio), SIO_GET_ASIZE(sio))); - bytes_dequeued += SIO_GET_ASIZE(sio); queue->q_sio_memused -= SIO_GET_MUSED(sio); sio_free(sio); } ASSERT0(queue->q_sio_memused); - atomic_add_64(&scn->scn_bytes_pending, -bytes_dequeued); range_tree_vacate(queue->q_exts_by_addr, NULL, queue); range_tree_destroy(queue->q_exts_by_addr); avl_destroy(&queue->q_sios_by_addr); @@ -4311,28 +4372,22 @@ dsl_scan_freed_dva(spa_t *spa, const blkptr_t *bp, int dva_i) sio_free(srch_sio); if (sio != NULL) { - int64_t asize = SIO_GET_ASIZE(sio); blkptr_t tmpbp; /* Got it while it was cold in the queue */ ASSERT3U(start, ==, SIO_GET_OFFSET(sio)); - ASSERT3U(size, ==, asize); + ASSERT3U(size, ==, SIO_GET_ASIZE(sio)); avl_remove(&queue->q_sios_by_addr, sio); + if (avl_is_empty(&queue->q_sios_by_addr)) + atomic_add_64(&scn->scn_queues_pending, -1); queue->q_sio_memused -= SIO_GET_MUSED(sio); ASSERT(range_tree_contains(queue->q_exts_by_addr, start, size)); range_tree_remove_fill(queue->q_exts_by_addr, start, size); - /* - * We only update scn_bytes_pending in the cold path, - * otherwise it will already have been accounted for as - * part of the zio's execution. - */ - atomic_add_64(&scn->scn_bytes_pending, -asize); - /* count the block as though we issued it */ sio2bp(sio, &tmpbp); - count_block(scn, dp->dp_blkstats, &tmpbp); + count_block_issued(spa, &tmpbp, B_FALSE); sio_free(sio); } @@ -4423,6 +4478,9 @@ ZFS_MODULE_PARAM(zfs, zfs_, max_async_dedup_frees, ULONG, ZMOD_RW, ZFS_MODULE_PARAM(zfs, zfs_, free_bpobj_enabled, INT, ZMOD_RW, "Enable processing of the free_bpobj"); +ZFS_MODULE_PARAM(zfs, zfs_, scan_blkstats, INT, ZMOD_RW, + "Enable block statistics calculation during scrub"); + ZFS_MODULE_PARAM(zfs, zfs_, scan_mem_lim_fact, INT, ZMOD_RW, "Fraction of RAM for scan hard limit"); diff --git a/module/zfs/dsl_synctask.c b/module/zfs/dsl_synctask.c index 9fc9d4011d11..409e12884d91 100644 --- a/module/zfs/dsl_synctask.c +++ b/module/zfs/dsl_synctask.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/dsl_userhold.c b/module/zfs/dsl_userhold.c index 75d153194a00..befb06cbdbcb 100644 --- a/module/zfs/dsl_userhold.c +++ b/module/zfs/dsl_userhold.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -346,7 +346,7 @@ dsl_dataset_user_hold(nvlist_t *holds, minor_t cleanup_minor, nvlist_t *errlist) return (ret); } -typedef int (dsl_holdfunc_t)(dsl_pool_t *dp, const char *name, void *tag, +typedef int (dsl_holdfunc_t)(dsl_pool_t *dp, const char *name, const void *tag, dsl_dataset_t **dsp); typedef struct dsl_dataset_user_release_arg { @@ -359,7 +359,7 @@ typedef struct dsl_dataset_user_release_arg { /* Place a dataset hold on the snapshot identified by passed dsobj string */ static int -dsl_dataset_hold_obj_string(dsl_pool_t *dp, const char *dsobj, void *tag, +dsl_dataset_hold_obj_string(dsl_pool_t *dp, const char *dsobj, const void *tag, dsl_dataset_t **dsp) { return (dsl_dataset_hold_obj(dp, zfs_strtonum(dsobj, NULL), tag, dsp)); diff --git a/module/zfs/edonr_zfs.c b/module/zfs/edonr_zfs.c index 938d684ec6d2..e56b4054c67e 100644 --- a/module/zfs/edonr_zfs.c +++ b/module/zfs/edonr_zfs.c @@ -45,7 +45,6 @@ edonr_incremental(void *buf, size_t size, void *arg) /* * Native zio_checksum interface for the Edon-R hash function. */ -/*ARGSUSED*/ void abd_checksum_edonr_native(abd_t *abd, uint64_t size, const void *ctx_template, zio_cksum_t *zcp) @@ -54,10 +53,10 @@ abd_checksum_edonr_native(abd_t *abd, uint64_t size, EdonRState ctx; ASSERT(ctx_template != NULL); - bcopy(ctx_template, &ctx, sizeof (ctx)); + memcpy(&ctx, ctx_template, sizeof (ctx)); (void) abd_iterate_func(abd, 0, size, edonr_incremental, &ctx); EdonRFinal(&ctx, digest); - bcopy(digest, zcp->zc_word, sizeof (zcp->zc_word)); + memcpy(zcp->zc_word, digest, sizeof (zcp->zc_word)); } /* @@ -109,8 +108,8 @@ abd_checksum_edonr_tmpl_init(const zio_cksum_salt_t *salt) void abd_checksum_edonr_tmpl_free(void *ctx_template) { - EdonRState *ctx = ctx_template; + EdonRState *ctx = ctx_template; - bzero(ctx, sizeof (*ctx)); + memset(ctx, 0, sizeof (*ctx)); kmem_free(ctx, sizeof (*ctx)); } diff --git a/module/zfs/fm.c b/module/zfs/fm.c index 5a52a881c687..e06fe4af6ddb 100644 --- a/module/zfs/fm.c +++ b/module/zfs/fm.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/gzip.c b/module/zfs/gzip.c index 48191241bd7d..f3b19446352a 100644 --- a/module/zfs/gzip.c +++ b/module/zfs/gzip.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -28,7 +28,6 @@ #include #include -#include #include #include @@ -66,7 +65,7 @@ gzip_compress(void *s_start, void *d_start, size_t s_len, size_t d_len, int n) if (d_len != s_len) return (s_len); - bcopy(s_start, d_start, s_len); + memcpy(d_start, s_start, s_len); return (s_len); } /* if hardware compression fails, do it again with software */ @@ -76,7 +75,7 @@ gzip_compress(void *s_start, void *d_start, size_t s_len, size_t d_len, int n) if (d_len != s_len) return (s_len); - bcopy(s_start, d_start, s_len); + memcpy(d_start, s_start, s_len); return (s_len); } diff --git a/module/zfs/hkdf.c b/module/zfs/hkdf.c index 14265472df7d..580544c8ac1a 100644 --- a/module/zfs/hkdf.c +++ b/module/zfs/hkdf.c @@ -36,7 +36,6 @@ hkdf_sha512_extract(uint8_t *salt, uint_t salt_len, uint8_t *key_material, mech.cm_param_len = 0; /* initialize the salt as a crypto key */ - key.ck_format = CRYPTO_KEY_RAW; key.ck_length = CRYPTO_BYTES2BITS(salt_len); key.ck_data = salt; @@ -53,7 +52,7 @@ hkdf_sha512_extract(uint8_t *salt, uint_t salt_len, uint8_t *key_material, output_cd.cd_raw.iov_base = (char *)out_buf; output_cd.cd_raw.iov_len = output_cd.cd_length; - ret = crypto_mac(&mech, &input_cd, &key, NULL, &output_cd, NULL); + ret = crypto_mac(&mech, &input_cd, &key, NULL, &output_cd); if (ret != CRYPTO_SUCCESS) return (SET_ERROR(EIO)); @@ -83,7 +82,6 @@ hkdf_sha512_expand(uint8_t *extract_key, uint8_t *info, uint_t info_len, mech.cm_param_len = 0; /* initialize the salt as a crypto key */ - key.ck_format = CRYPTO_KEY_RAW; key.ck_length = CRYPTO_BYTES2BITS(SHA512_DIGEST_LENGTH); key.ck_data = extract_key; @@ -110,19 +108,19 @@ hkdf_sha512_expand(uint8_t *extract_key, uint8_t *info, uint_t info_len, T_cd.cd_length = T_len; T_cd.cd_raw.iov_len = T_cd.cd_length; - ret = crypto_mac_init(&mech, &key, NULL, &ctx, NULL); + ret = crypto_mac_init(&mech, &key, NULL, &ctx); if (ret != CRYPTO_SUCCESS) return (SET_ERROR(EIO)); - ret = crypto_mac_update(ctx, &T_cd, NULL); + ret = crypto_mac_update(ctx, &T_cd); if (ret != CRYPTO_SUCCESS) return (SET_ERROR(EIO)); - ret = crypto_mac_update(ctx, &info_cd, NULL); + ret = crypto_mac_update(ctx, &info_cd); if (ret != CRYPTO_SUCCESS) return (SET_ERROR(EIO)); - ret = crypto_mac_update(ctx, &c_cd, NULL); + ret = crypto_mac_update(ctx, &c_cd); if (ret != CRYPTO_SUCCESS) return (SET_ERROR(EIO)); @@ -130,11 +128,11 @@ hkdf_sha512_expand(uint8_t *extract_key, uint8_t *info, uint_t info_len, T_cd.cd_length = T_len; T_cd.cd_raw.iov_len = T_cd.cd_length; - ret = crypto_mac_final(ctx, &T_cd, NULL); + ret = crypto_mac_final(ctx, &T_cd); if (ret != CRYPTO_SUCCESS) return (SET_ERROR(EIO)); - bcopy(T, out_buf + pos, + memcpy(out_buf + pos, T, (i != N) ? SHA512_DIGEST_LENGTH : (out_len - pos)); pos += SHA512_DIGEST_LENGTH; } diff --git a/module/zfs/lzjb.c b/module/zfs/lzjb.c index 1c536b110318..a24f17e0fe74 100644 --- a/module/zfs/lzjb.c +++ b/module/zfs/lzjb.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/metaslab.c b/module/zfs/metaslab.c index bd17c1fe862a..02cf121d83d7 100644 --- a/module/zfs/metaslab.c +++ b/module/zfs/metaslab.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -48,10 +48,10 @@ /* * Metaslab granularity, in bytes. This is roughly similar to what would be * referred to as the "stripe size" in traditional RAID arrays. In normal - * operation, we will try to write this amount of data to a top-level vdev - * before moving on to the next one. + * operation, we will try to write this amount of data to each disk before + * moving on to the next top-level vdev. */ -static unsigned long metaslab_aliquot = 512 << 10; +static unsigned long metaslab_aliquot = 1024 * 1024; /* * For testing, make some blocks above a certain size be gang blocks. @@ -899,7 +899,8 @@ metaslab_group_activate(metaslab_group_t *mg) if (++mg->mg_activation_count <= 0) return; - mg->mg_aliquot = metaslab_aliquot * MAX(1, mg->mg_vd->vdev_children); + mg->mg_aliquot = metaslab_aliquot * MAX(1, + vdev_get_ndisks(mg->mg_vd) - vdev_get_nparity(mg->mg_vd)); metaslab_group_alloc_update(mg); if ((mgprev = mc->mc_allocator[0].mca_rotor) == NULL) { @@ -1952,9 +1953,9 @@ metaslab_aux_histograms_clear(metaslab_t *msp) */ ASSERT(msp->ms_loaded); - bzero(msp->ms_synchist, sizeof (msp->ms_synchist)); + memset(msp->ms_synchist, 0, sizeof (msp->ms_synchist)); for (int t = 0; t < TXG_DEFER_SIZE; t++) - bzero(msp->ms_deferhist[t], sizeof (msp->ms_deferhist[t])); + memset(msp->ms_deferhist[t], 0, sizeof (msp->ms_deferhist[t])); } static void @@ -2044,13 +2045,13 @@ metaslab_aux_histograms_update_done(metaslab_t *msp, boolean_t defer_allowed) */ uint64_t hist_index = spa_syncing_txg(spa) % TXG_DEFER_SIZE; if (defer_allowed) { - bcopy(msp->ms_synchist, msp->ms_deferhist[hist_index], + memcpy(msp->ms_deferhist[hist_index], msp->ms_synchist, sizeof (msp->ms_synchist)); } else { - bzero(msp->ms_deferhist[hist_index], + memset(msp->ms_deferhist[hist_index], 0, sizeof (msp->ms_deferhist[hist_index])); } - bzero(msp->ms_synchist, sizeof (msp->ms_synchist)); + memset(msp->ms_synchist, 0, sizeof (msp->ms_synchist)); } /* @@ -2750,7 +2751,8 @@ metaslab_fini_flush_data(metaslab_t *msp) mutex_exit(&spa->spa_flushed_ms_lock); spa_log_sm_decrement_mscount(spa, metaslab_unflushed_txg(msp)); - spa_log_summary_decrement_mscount(spa, metaslab_unflushed_txg(msp)); + spa_log_summary_decrement_mscount(spa, metaslab_unflushed_txg(msp), + metaslab_unflushed_dirty(msp)); } uint64_t @@ -3728,50 +3730,45 @@ metaslab_condense(metaslab_t *msp, dmu_tx_t *tx) metaslab_flush_update(msp, tx); } -/* - * Called when the metaslab has been flushed (its own spacemap now reflects - * all the contents of the pool-wide spacemap log). Updates the metaslab's - * metadata and any pool-wide related log space map data (e.g. summary, - * obsolete logs, etc..) to reflect that. - */ static void -metaslab_flush_update(metaslab_t *msp, dmu_tx_t *tx) +metaslab_unflushed_add(metaslab_t *msp, dmu_tx_t *tx) { - metaslab_group_t *mg = msp->ms_group; - spa_t *spa = mg->mg_vd->vdev_spa; - - ASSERT(MUTEX_HELD(&msp->ms_lock)); - - ASSERT3U(spa_sync_pass(spa), ==, 1); + spa_t *spa = msp->ms_group->mg_vd->vdev_spa; + ASSERT(spa_syncing_log_sm(spa) != NULL); + ASSERT(msp->ms_sm != NULL); ASSERT(range_tree_is_empty(msp->ms_unflushed_allocs)); ASSERT(range_tree_is_empty(msp->ms_unflushed_frees)); - /* - * Just because a metaslab got flushed, that doesn't mean that - * it will pass through metaslab_sync_done(). Thus, make sure to - * update ms_synced_length here in case it doesn't. - */ - msp->ms_synced_length = space_map_length(msp->ms_sm); + mutex_enter(&spa->spa_flushed_ms_lock); + metaslab_set_unflushed_txg(msp, spa_syncing_txg(spa), tx); + metaslab_set_unflushed_dirty(msp, B_TRUE); + avl_add(&spa->spa_metaslabs_by_flushed, msp); + mutex_exit(&spa->spa_flushed_ms_lock); - /* - * We may end up here from metaslab_condense() without the - * feature being active. In that case this is a no-op. - */ - if (!spa_feature_is_active(spa, SPA_FEATURE_LOG_SPACEMAP)) - return; + spa_log_sm_increment_current_mscount(spa); + spa_log_summary_add_flushed_metaslab(spa, B_TRUE); +} +void +metaslab_unflushed_bump(metaslab_t *msp, dmu_tx_t *tx, boolean_t dirty) +{ + spa_t *spa = msp->ms_group->mg_vd->vdev_spa; ASSERT(spa_syncing_log_sm(spa) != NULL); ASSERT(msp->ms_sm != NULL); ASSERT(metaslab_unflushed_txg(msp) != 0); ASSERT3P(avl_find(&spa->spa_metaslabs_by_flushed, msp, NULL), ==, msp); + ASSERT(range_tree_is_empty(msp->ms_unflushed_allocs)); + ASSERT(range_tree_is_empty(msp->ms_unflushed_frees)); VERIFY3U(tx->tx_txg, <=, spa_final_dirty_txg(spa)); /* update metaslab's position in our flushing tree */ uint64_t ms_prev_flushed_txg = metaslab_unflushed_txg(msp); + boolean_t ms_prev_flushed_dirty = metaslab_unflushed_dirty(msp); mutex_enter(&spa->spa_flushed_ms_lock); avl_remove(&spa->spa_metaslabs_by_flushed, msp); metaslab_set_unflushed_txg(msp, spa_syncing_txg(spa), tx); + metaslab_set_unflushed_dirty(msp, dirty); avl_add(&spa->spa_metaslabs_by_flushed, msp); mutex_exit(&spa->spa_flushed_ms_lock); @@ -3779,17 +3776,47 @@ metaslab_flush_update(metaslab_t *msp, dmu_tx_t *tx) spa_log_sm_decrement_mscount(spa, ms_prev_flushed_txg); spa_log_sm_increment_current_mscount(spa); + /* update log space map summary */ + spa_log_summary_decrement_mscount(spa, ms_prev_flushed_txg, + ms_prev_flushed_dirty); + spa_log_summary_add_flushed_metaslab(spa, dirty); + /* cleanup obsolete logs if any */ - uint64_t log_blocks_before = spa_log_sm_nblocks(spa); spa_cleanup_old_sm_logs(spa, tx); - uint64_t log_blocks_after = spa_log_sm_nblocks(spa); - VERIFY3U(log_blocks_after, <=, log_blocks_before); +} - /* update log space map summary */ - uint64_t blocks_gone = log_blocks_before - log_blocks_after; - spa_log_summary_add_flushed_metaslab(spa); - spa_log_summary_decrement_mscount(spa, ms_prev_flushed_txg); - spa_log_summary_decrement_blkcount(spa, blocks_gone); +/* + * Called when the metaslab has been flushed (its own spacemap now reflects + * all the contents of the pool-wide spacemap log). Updates the metaslab's + * metadata and any pool-wide related log space map data (e.g. summary, + * obsolete logs, etc..) to reflect that. + */ +static void +metaslab_flush_update(metaslab_t *msp, dmu_tx_t *tx) +{ + metaslab_group_t *mg = msp->ms_group; + spa_t *spa = mg->mg_vd->vdev_spa; + + ASSERT(MUTEX_HELD(&msp->ms_lock)); + + ASSERT3U(spa_sync_pass(spa), ==, 1); + + /* + * Just because a metaslab got flushed, that doesn't mean that + * it will pass through metaslab_sync_done(). Thus, make sure to + * update ms_synced_length here in case it doesn't. + */ + msp->ms_synced_length = space_map_length(msp->ms_sm); + + /* + * We may end up here from metaslab_condense() without the + * feature being active. In that case this is a no-op. + */ + if (!spa_feature_is_active(spa, SPA_FEATURE_LOG_SPACEMAP) || + metaslab_unflushed_txg(msp) == 0) + return; + + metaslab_unflushed_bump(msp, tx, B_FALSE); } boolean_t @@ -4005,23 +4032,6 @@ metaslab_sync(metaslab_t *msp, uint64_t txg) ASSERT0(metaslab_allocated_space(msp)); } - if (metaslab_unflushed_txg(msp) == 0 && - spa_feature_is_active(spa, SPA_FEATURE_LOG_SPACEMAP)) { - ASSERT(spa_syncing_log_sm(spa) != NULL); - - metaslab_set_unflushed_txg(msp, spa_syncing_txg(spa), tx); - spa_log_sm_increment_current_mscount(spa); - spa_log_summary_add_flushed_metaslab(spa); - - ASSERT(msp->ms_sm != NULL); - mutex_enter(&spa->spa_flushed_ms_lock); - avl_add(&spa->spa_metaslabs_by_flushed, msp); - mutex_exit(&spa->spa_flushed_ms_lock); - - ASSERT(range_tree_is_empty(msp->ms_unflushed_allocs)); - ASSERT(range_tree_is_empty(msp->ms_unflushed_frees)); - } - if (!range_tree_is_empty(msp->ms_checkpointing) && vd->vdev_checkpoint_sm == NULL) { ASSERT(spa_has_checkpoint(spa)); @@ -4069,6 +4079,10 @@ metaslab_sync(metaslab_t *msp, uint64_t txg) space_map_t *log_sm = spa_syncing_log_sm(spa); if (log_sm != NULL) { ASSERT(spa_feature_is_enabled(spa, SPA_FEATURE_LOG_SPACEMAP)); + if (metaslab_unflushed_txg(msp) == 0) + metaslab_unflushed_add(msp, tx); + else if (!metaslab_unflushed_dirty(msp)) + metaslab_unflushed_bump(msp, tx, B_TRUE); space_map_write(log_sm, alloctree, SM_ALLOC, vd->vdev_id, tx); @@ -4494,8 +4508,8 @@ metaslab_trace_fini(zio_alloc_list_t *zal) */ static void -metaslab_group_alloc_increment(spa_t *spa, uint64_t vdev, void *tag, int flags, - int allocator) +metaslab_group_alloc_increment(spa_t *spa, uint64_t vdev, const void *tag, + int flags, int allocator) { if (!(flags & METASLAB_ASYNC_ALLOC) || (flags & METASLAB_DONT_THROTTLE)) @@ -4528,8 +4542,8 @@ metaslab_group_increment_qdepth(metaslab_group_t *mg, int allocator) } void -metaslab_group_alloc_decrement(spa_t *spa, uint64_t vdev, void *tag, int flags, - int allocator, boolean_t io_complete) +metaslab_group_alloc_decrement(spa_t *spa, uint64_t vdev, const void *tag, + int flags, int allocator, boolean_t io_complete) { if (!(flags & METASLAB_ASYNC_ALLOC) || (flags & METASLAB_DONT_THROTTLE)) @@ -4546,7 +4560,7 @@ metaslab_group_alloc_decrement(spa_t *spa, uint64_t vdev, void *tag, int flags, } void -metaslab_group_alloc_verify(spa_t *spa, const blkptr_t *bp, void *tag, +metaslab_group_alloc_verify(spa_t *spa, const blkptr_t *bp, const void *tag, int allocator) { #ifdef ZFS_DEBUG @@ -5296,7 +5310,7 @@ metaslab_alloc_dva(spa_t *spa, metaslab_class_t *mc, uint64_t psize, goto top; } - bzero(&dva[d], sizeof (dva_t)); + memset(&dva[d], 0, sizeof (dva_t)); metaslab_trace_add(zal, rotor, NULL, psize, d, TRACE_ENOSPC, allocator); return (SET_ERROR(ENOSPC)); @@ -5809,7 +5823,7 @@ metaslab_alloc(spa_t *spa, metaslab_class_t *mc, uint64_t psize, blkptr_t *bp, metaslab_group_alloc_decrement(spa, DVA_GET_VDEV(&dva[d]), zio, flags, allocator, B_FALSE); - bzero(&dva[d], sizeof (dva_t)); + memset(&dva[d], 0, sizeof (dva_t)); } spa_config_exit(spa, SCL_ALLOC, FTAG); return (error); @@ -6131,6 +6145,12 @@ metaslab_enable(metaslab_t *msp, boolean_t sync, boolean_t unload) mutex_exit(&mg->mg_ms_disabled_lock); } +void +metaslab_set_unflushed_dirty(metaslab_t *ms, boolean_t dirty) +{ + ms->ms_unflushed_dirty = dirty; +} + static void metaslab_update_ondisk_flush_data(metaslab_t *ms, dmu_tx_t *tx) { @@ -6167,15 +6187,16 @@ metaslab_update_ondisk_flush_data(metaslab_t *ms, dmu_tx_t *tx) void metaslab_set_unflushed_txg(metaslab_t *ms, uint64_t txg, dmu_tx_t *tx) { - spa_t *spa = ms->ms_group->mg_vd->vdev_spa; - - if (!spa_feature_is_active(spa, SPA_FEATURE_LOG_SPACEMAP)) - return; - ms->ms_unflushed_txg = txg; metaslab_update_ondisk_flush_data(ms, tx); } +boolean_t +metaslab_unflushed_dirty(metaslab_t *ms) +{ + return (ms->ms_unflushed_dirty); +} + uint64_t metaslab_unflushed_txg(metaslab_t *ms) { diff --git a/module/zfs/mmp.c b/module/zfs/mmp.c index abdce3a32e6a..92fd6c422330 100644 --- a/module/zfs/mmp.c +++ b/module/zfs/mmp.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -186,8 +186,8 @@ uint_t zfs_multihost_import_intervals = MMP_DEFAULT_IMPORT_INTERVALS; */ uint_t zfs_multihost_fail_intervals = MMP_DEFAULT_FAIL_INTERVALS; -static void *const mmp_tag = "mmp_write_uberblock"; -static void mmp_thread(void *arg); +static const void *const mmp_tag = "mmp_write_uberblock"; +static __attribute__((noreturn)) void mmp_thread(void *arg); void mmp_init(spa_t *spa) @@ -224,7 +224,6 @@ mmp_thread_exit(mmp_thread_t *mmp, kthread_t **mpp, callb_cpr_t *cpr) *mpp = NULL; cv_broadcast(&mmp->mmp_thread_cv); CALLB_CPR_EXIT(cpr); /* drops &mmp->mmp_thread_lock */ - thread_exit(); } void @@ -537,7 +536,7 @@ mmp_write_uberblock(spa_t *spa) zio_nowait(zio); } -static void +static __attribute__((noreturn)) void mmp_thread(void *arg) { spa_t *spa = (spa_t *)arg; @@ -698,6 +697,8 @@ mmp_thread(void *arg) mmp->mmp_zio_root = NULL; mmp_thread_exit(mmp, &mmp->mmp_thread, &cpr); + + thread_exit(); } /* diff --git a/module/zfs/pathname.c b/module/zfs/pathname.c index 84ab7b7e1111..51460d119106 100644 --- a/module/zfs/pathname.c +++ b/module/zfs/pathname.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/range_tree.c b/module/zfs/range_tree.c index f9fb97476b64..a2923d1664c7 100644 --- a/module/zfs/range_tree.c +++ b/module/zfs/range_tree.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -78,7 +78,7 @@ static inline void rs_copy(range_seg_t *src, range_seg_t *dest, range_tree_t *rt) { - ASSERT3U(rt->rt_type, <=, RANGE_SEG_NUM_TYPES); + ASSERT3U(rt->rt_type, <, RANGE_SEG_NUM_TYPES); size_t size = 0; switch (rt->rt_type) { case RANGE_SEG32: @@ -91,9 +91,9 @@ rs_copy(range_seg_t *src, range_seg_t *dest, range_tree_t *rt) size = sizeof (range_seg_gap_t); break; default: - VERIFY(0); + __builtin_unreachable(); } - bcopy(src, dest, size); + memcpy(dest, src, size); } void @@ -188,10 +188,8 @@ range_tree_seg_gap_compare(const void *x1, const void *x2) } range_tree_t * -range_tree_create_impl(const range_tree_ops_t *ops, range_seg_type_t type, - void *arg, uint64_t start, uint64_t shift, - int (*zfs_btree_compare) (const void *, const void *), - uint64_t gap) +range_tree_create_gap(const range_tree_ops_t *ops, range_seg_type_t type, + void *arg, uint64_t start, uint64_t shift, uint64_t gap) { range_tree_t *rt = kmem_zalloc(sizeof (range_tree_t), KM_SLEEP); @@ -223,7 +221,6 @@ range_tree_create_impl(const range_tree_ops_t *ops, range_seg_type_t type, rt->rt_type = type; rt->rt_start = start; rt->rt_shift = shift; - rt->rt_btree_compare = zfs_btree_compare; if (rt->rt_ops != NULL && rt->rt_ops->rtop_create != NULL) rt->rt_ops->rtop_create(rt, rt->rt_arg); @@ -235,7 +232,7 @@ range_tree_t * range_tree_create(const range_tree_ops_t *ops, range_seg_type_t type, void *arg, uint64_t start, uint64_t shift) { - return (range_tree_create_impl(ops, type, arg, start, shift, NULL, 0)); + return (range_tree_create_gap(ops, type, arg, start, shift, 0)); } void @@ -701,7 +698,7 @@ range_tree_vacate(range_tree_t *rt, range_tree_func_t *func, void *arg) zfs_btree_clear(&rt->rt_root); } - bzero(rt->rt_histogram, sizeof (rt->rt_histogram)); + memset(rt->rt_histogram, 0, sizeof (rt->rt_histogram)); rt->rt_space = 0; } @@ -741,74 +738,6 @@ range_tree_is_empty(range_tree_t *rt) return (range_tree_space(rt) == 0); } -void -rt_btree_create(range_tree_t *rt, void *arg) -{ - zfs_btree_t *size_tree = arg; - - size_t size; - switch (rt->rt_type) { - case RANGE_SEG32: - size = sizeof (range_seg32_t); - break; - case RANGE_SEG64: - size = sizeof (range_seg64_t); - break; - case RANGE_SEG_GAP: - size = sizeof (range_seg_gap_t); - break; - default: - panic("Invalid range seg type %d", rt->rt_type); - } - zfs_btree_create(size_tree, rt->rt_btree_compare, size); -} - -void -rt_btree_destroy(range_tree_t *rt, void *arg) -{ - (void) rt; - zfs_btree_t *size_tree = arg; - ASSERT0(zfs_btree_numnodes(size_tree)); - - zfs_btree_destroy(size_tree); -} - -void -rt_btree_add(range_tree_t *rt, range_seg_t *rs, void *arg) -{ - (void) rt; - zfs_btree_t *size_tree = arg; - - zfs_btree_add(size_tree, rs); -} - -void -rt_btree_remove(range_tree_t *rt, range_seg_t *rs, void *arg) -{ - (void) rt; - zfs_btree_t *size_tree = arg; - - zfs_btree_remove(size_tree, rs); -} - -void -rt_btree_vacate(range_tree_t *rt, void *arg) -{ - zfs_btree_t *size_tree = arg; - zfs_btree_clear(size_tree); - zfs_btree_destroy(size_tree); - - rt_btree_create(rt, arg); -} - -const range_tree_ops_t rt_btree_ops = { - .rtop_create = rt_btree_create, - .rtop_destroy = rt_btree_destroy, - .rtop_add = rt_btree_add, - .rtop_remove = rt_btree_remove, - .rtop_vacate = rt_btree_vacate -}; - /* * Remove any overlapping ranges between the given segment [start, end) * from removefrom. Add non-overlapping leftovers to addto. diff --git a/module/zfs/refcount.c b/module/zfs/refcount.c index 390b2fdedc70..b215df98d7a8 100644 --- a/module/zfs/refcount.c +++ b/module/zfs/refcount.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/rrwlock.c b/module/zfs/rrwlock.c index d23fc3ad1067..a8c438bb6ebd 100644 --- a/module/zfs/rrwlock.c +++ b/module/zfs/rrwlock.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -77,7 +77,7 @@ uint_t rrw_tsd_key; typedef struct rrw_node { struct rrw_node *rn_next; rrwlock_t *rn_rrl; - void *rn_tag; + const void *rn_tag; } rrw_node_t; static rrw_node_t * @@ -99,7 +99,7 @@ rrn_find(rrwlock_t *rrl) * Add a node to the head of the singly linked list. */ static void -rrn_add(rrwlock_t *rrl, void *tag) +rrn_add(rrwlock_t *rrl, const void *tag) { rrw_node_t *rn; @@ -115,7 +115,7 @@ rrn_add(rrwlock_t *rrl, void *tag) * thread's list and return TRUE; otherwise return FALSE. */ static boolean_t -rrn_find_and_remove(rrwlock_t *rrl, void *tag) +rrn_find_and_remove(rrwlock_t *rrl, const void *tag) { rrw_node_t *rn; rrw_node_t *prev = NULL; @@ -160,7 +160,7 @@ rrw_destroy(rrwlock_t *rrl) } static void -rrw_enter_read_impl(rrwlock_t *rrl, boolean_t prio, void *tag) +rrw_enter_read_impl(rrwlock_t *rrl, boolean_t prio, const void *tag) { mutex_enter(&rrl->rr_lock); #if !defined(ZFS_DEBUG) && defined(_KERNEL) @@ -192,7 +192,7 @@ rrw_enter_read_impl(rrwlock_t *rrl, boolean_t prio, void *tag) } void -rrw_enter_read(rrwlock_t *rrl, void *tag) +rrw_enter_read(rrwlock_t *rrl, const void *tag) { rrw_enter_read_impl(rrl, B_FALSE, tag); } @@ -204,7 +204,7 @@ rrw_enter_read(rrwlock_t *rrl, void *tag) * the pending writer does not work, so we have to give an explicit hint here. */ void -rrw_enter_read_prio(rrwlock_t *rrl, void *tag) +rrw_enter_read_prio(rrwlock_t *rrl, const void *tag) { rrw_enter_read_impl(rrl, B_TRUE, tag); } @@ -228,7 +228,7 @@ rrw_enter_write(rrwlock_t *rrl) } void -rrw_enter(rrwlock_t *rrl, krw_t rw, void *tag) +rrw_enter(rrwlock_t *rrl, krw_t rw, const void *tag) { if (rw == RW_READER) rrw_enter_read(rrl, tag); @@ -237,7 +237,7 @@ rrw_enter(rrwlock_t *rrl, krw_t rw, void *tag) } void -rrw_exit(rrwlock_t *rrl, void *tag) +rrw_exit(rrwlock_t *rrl, const void *tag) { mutex_enter(&rrl->rr_lock); #if !defined(ZFS_DEBUG) && defined(_KERNEL) @@ -339,7 +339,7 @@ rrm_destroy(rrmlock_t *rrl) } void -rrm_enter(rrmlock_t *rrl, krw_t rw, void *tag) +rrm_enter(rrmlock_t *rrl, krw_t rw, const void *tag) { if (rw == RW_READER) rrm_enter_read(rrl, tag); @@ -358,7 +358,7 @@ rrm_enter(rrmlock_t *rrl, krw_t rw, void *tag) #define RRM_TD_LOCK() (((uint32_t)(uintptr_t)(curthread)) % RRM_NUM_LOCKS) void -rrm_enter_read(rrmlock_t *rrl, void *tag) +rrm_enter_read(rrmlock_t *rrl, const void *tag) { rrw_enter_read(&rrl->locks[RRM_TD_LOCK()], tag); } @@ -373,7 +373,7 @@ rrm_enter_write(rrmlock_t *rrl) } void -rrm_exit(rrmlock_t *rrl, void *tag) +rrm_exit(rrmlock_t *rrl, const void *tag) { int i; diff --git a/module/zfs/sa.c b/module/zfs/sa.c index a078af159c1f..c094a8f0730a 100644 --- a/module/zfs/sa.c +++ b/module/zfs/sa.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -160,7 +160,7 @@ do { \ *(uint64_t *)((uintptr_t)t + 8) = \ *(uint64_t *)((uintptr_t)s + 8); \ } else { \ - bcopy(s, t, l); \ + memcpy(t, s, l); \ } \ } else { \ sa_copy_data(f, s, t, l); \ @@ -414,7 +414,7 @@ sa_add_layout_entry(objset_t *os, const sa_attr_type_t *attrs, int attr_count, tb->lot_attr_count = attr_count; tb->lot_attrs = kmem_alloc(sizeof (sa_attr_type_t) * attr_count, KM_SLEEP); - bcopy(attrs, tb->lot_attrs, sizeof (sa_attr_type_t) * attr_count); + memcpy(tb->lot_attrs, attrs, sizeof (sa_attr_type_t) * attr_count); tb->lot_num = lot_num; tb->lot_hash = hash; tb->lot_instance = 0; @@ -511,7 +511,7 @@ static void sa_copy_data(sa_data_locator_t *func, void *datastart, void *target, int buflen) { if (func == NULL) { - bcopy(datastart, target, buflen); + memcpy(target, datastart, buflen); } else { boolean_t start; int bytes; @@ -523,7 +523,7 @@ sa_copy_data(sa_data_locator_t *func, void *datastart, void *target, int buflen) bytes = 0; while (bytes < buflen) { func(&dataptr, &length, buflen, start, datastart); - bcopy(dataptr, saptr, length); + memcpy(saptr, dataptr, length); saptr = (void *)((caddr_t)saptr + length); bytes += length; start = B_FALSE; @@ -1068,8 +1068,8 @@ sa_setup(objset_t *os, uint64_t sa_obj, const sa_attr_reg_t *reg_attrs, za.za_num_integers); break; } - VERIFY(ddi_strtoull(za.za_name, NULL, 10, - (unsigned long long *)&lot_num) == 0); + VERIFY0(ddi_strtoull(za.za_name, NULL, 10, + (unsigned long long *)&lot_num)); (void) sa_add_layout_entry(os, lot_attrs, za.za_num_integers, lot_num, @@ -1449,13 +1449,13 @@ sa_handle_get(objset_t *objset, uint64_t objid, void *userp, } int -sa_buf_hold(objset_t *objset, uint64_t obj_num, void *tag, dmu_buf_t **db) +sa_buf_hold(objset_t *objset, uint64_t obj_num, const void *tag, dmu_buf_t **db) { return (dmu_bonus_hold(objset, obj_num, tag, db)); } void -sa_buf_rele(dmu_buf_t *db, void *tag) +sa_buf_rele(dmu_buf_t *db, const void *tag) { dmu_buf_rele(db, tag); } @@ -1664,8 +1664,9 @@ sa_add_projid(sa_handle_t *hdl, dmu_tx_t *tx, uint64_t projid) &xattr, 8); if (zp->z_pflags & ZFS_BONUS_SCANSTAMP) { - bcopy((caddr_t)db->db_data + ZFS_OLD_ZNODE_PHYS_SIZE, - scanstamp, AV_SCANSTAMP_SZ); + memcpy(scanstamp, + (caddr_t)db->db_data + ZFS_OLD_ZNODE_PHYS_SIZE, + AV_SCANSTAMP_SZ); SA_ADD_BULK_ATTR(attrs, count, SA_ZPL_SCANSTAMP(zfsvfs), NULL, scanstamp, AV_SCANSTAMP_SZ); zp->z_pflags &= ~ZFS_BONUS_SCANSTAMP; @@ -1873,7 +1874,7 @@ sa_modify_attrs(sa_handle_t *hdl, sa_attr_type_t newattr, if (dn->dn_bonuslen != 0) { bonus_data_size = hdl->sa_bonus->db_size; old_data[0] = kmem_alloc(bonus_data_size, KM_SLEEP); - bcopy(hdl->sa_bonus->db_data, old_data[0], + memcpy(old_data[0], hdl->sa_bonus->db_data, hdl->sa_bonus->db_size); bonus_attr_count = hdl->sa_bonus_tab->sa_layout->lot_attr_count; } else { @@ -1886,7 +1887,7 @@ sa_modify_attrs(sa_handle_t *hdl, sa_attr_type_t newattr, if ((error = sa_get_spill(hdl)) == 0) { spill_data_size = hdl->sa_spill->db_size; old_data[1] = vmem_alloc(spill_data_size, KM_SLEEP); - bcopy(hdl->sa_spill->db_data, old_data[1], + memcpy(old_data[1], hdl->sa_spill->db_data, hdl->sa_spill->db_size); spill_attr_count = hdl->sa_spill_tab->sa_layout->lot_attr_count; diff --git a/module/zfs/sha256.c b/module/zfs/sha256.c index c5b033cf019d..445d82ed020c 100644 --- a/module/zfs/sha256.c +++ b/module/zfs/sha256.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/skein_zfs.c b/module/zfs/skein_zfs.c index 11b9940e027e..4b2aca888eee 100644 --- a/module/zfs/skein_zfs.c +++ b/module/zfs/skein_zfs.c @@ -41,18 +41,17 @@ skein_incremental(void *buf, size_t size, void *arg) * function requires the presence of a ctx_template that should be allocated * using abd_checksum_skein_tmpl_init. */ -/*ARGSUSED*/ void abd_checksum_skein_native(abd_t *abd, uint64_t size, const void *ctx_template, zio_cksum_t *zcp) { - Skein_512_Ctxt_t ctx; + Skein_512_Ctxt_t ctx; ASSERT(ctx_template != NULL); - bcopy(ctx_template, &ctx, sizeof (ctx)); + memcpy(&ctx, ctx_template, sizeof (ctx)); (void) abd_iterate_func(abd, 0, size, skein_incremental, &ctx); (void) Skein_512_Final(&ctx, (uint8_t *)zcp); - bzero(&ctx, sizeof (ctx)); + memset(&ctx, 0, sizeof (ctx)); } /* @@ -80,9 +79,8 @@ abd_checksum_skein_byteswap(abd_t *abd, uint64_t size, void * abd_checksum_skein_tmpl_init(const zio_cksum_salt_t *salt) { - Skein_512_Ctxt_t *ctx; + Skein_512_Ctxt_t *ctx = kmem_zalloc(sizeof (*ctx), KM_SLEEP); - ctx = kmem_zalloc(sizeof (*ctx), KM_SLEEP); (void) Skein_512_InitExt(ctx, sizeof (zio_cksum_t) * 8, 0, salt->zcs_bytes, sizeof (salt->zcs_bytes)); return (ctx); @@ -95,8 +93,8 @@ abd_checksum_skein_tmpl_init(const zio_cksum_salt_t *salt) void abd_checksum_skein_tmpl_free(void *ctx_template) { - Skein_512_Ctxt_t *ctx = ctx_template; + Skein_512_Ctxt_t *ctx = ctx_template; - bzero(ctx, sizeof (*ctx)); + memset(ctx, 0, sizeof (*ctx)); kmem_free(ctx, sizeof (*ctx)); } diff --git a/module/zfs/spa.c b/module/zfs/spa.c index 647ee17e4c71..b2b59af42947 100644 --- a/module/zfs/spa.c +++ b/module/zfs/spa.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -164,7 +164,8 @@ static const zio_taskq_info_t zio_taskqs[ZIO_TYPES][ZIO_TASKQ_TYPES] = { static void spa_sync_version(void *arg, dmu_tx_t *tx); static void spa_sync_props(void *arg, dmu_tx_t *tx); static boolean_t spa_has_active_shared_spare(spa_t *spa); -static int spa_load_impl(spa_t *spa, spa_import_type_t type, char **ereport); +static int spa_load_impl(spa_t *spa, spa_import_type_t type, + const char **ereport); static void spa_vdev_resilver_done(spa_t *spa); static uint_t zio_taskq_batch_pct = 80; /* 1 thread per cpu in pset */ @@ -277,7 +278,7 @@ static int zfs_livelist_condense_new_alloc = 0; * Add a (source=src, propname=propval) list to an nvlist. */ static void -spa_prop_add_list(nvlist_t *nvl, zpool_prop_t prop, char *strval, +spa_prop_add_list(nvlist_t *nvl, zpool_prop_t prop, const char *strval, uint64_t intval, zprop_source_t src) { const char *propname = zpool_prop_to_name(prop); @@ -947,8 +948,8 @@ spa_get_errlists(spa_t *spa, avl_tree_t *last, avl_tree_t *scrub) { ASSERT(MUTEX_HELD(&spa->spa_errlist_lock)); - bcopy(&spa->spa_errlist_last, last, sizeof (avl_tree_t)); - bcopy(&spa->spa_errlist_scrub, scrub, sizeof (avl_tree_t)); + memcpy(last, &spa->spa_errlist_last, sizeof (avl_tree_t)); + memcpy(scrub, &spa->spa_errlist_scrub, sizeof (avl_tree_t)); avl_create(&spa->spa_errlist_scrub, spa_error_entry_compare, sizeof (spa_error_entry_t), @@ -1314,6 +1315,11 @@ spa_activate(spa_t *spa, spa_mode_t mode) avl_create(&spa->spa_errlist_last, spa_error_entry_compare, sizeof (spa_error_entry_t), offsetof(spa_error_entry_t, se_avl)); + avl_create(&spa->spa_errlist_healed, + spa_error_entry_compare, sizeof (spa_error_entry_t), + offsetof(spa_error_entry_t, se_avl)); + + spa_activate_os(spa); spa_keystore_init(&spa->spa_keystore); @@ -1422,6 +1428,7 @@ spa_deactivate(spa_t *spa) spa_errlog_drain(spa); avl_destroy(&spa->spa_errlist_scrub); avl_destroy(&spa->spa_errlist_last); + avl_destroy(&spa->spa_errlist_healed); spa_keystore_fini(&spa->spa_keystore); @@ -1451,6 +1458,9 @@ spa_deactivate(spa_t *spa) thread_join(spa->spa_did); spa->spa_did = 0; } + + spa_deactivate_os(spa); + } /* @@ -2305,9 +2315,6 @@ spa_load_verify_cb(spa_t *spa, zilog_t *zilog, const blkptr_t *bp, (void) zilog, (void) dnp; - if (zb->zb_level == ZB_DNODE_LEVEL || BP_IS_HOLE(bp) || - BP_IS_EMBEDDED(bp) || BP_IS_REDACTED(bp)) - return (0); /* * Note: normally this routine will not be called if * spa_load_verify_metadata is not set. However, it may be useful @@ -2315,6 +2322,22 @@ spa_load_verify_cb(spa_t *spa, zilog_t *zilog, const blkptr_t *bp, */ if (!spa_load_verify_metadata) return (0); + + /* + * Sanity check the block pointer in order to detect obvious damage + * before using the contents in subsequent checks or in zio_read(). + * When damaged consider it to be a metadata error since we cannot + * trust the BP_GET_TYPE and BP_GET_LEVEL values. + */ + if (!zfs_blkptr_verify(spa, bp, B_FALSE, BLK_VERIFY_LOG)) { + atomic_inc_64(&sle->sle_meta_count); + return (0); + } + + if (zb->zb_level == ZB_DNODE_LEVEL || BP_IS_HOLE(bp) || + BP_IS_EMBEDDED(bp) || BP_IS_REDACTED(bp)) + return (0); + if (!BP_IS_METADATA(bp) && (!spa_load_verify_data || !sle->sle_verify_data)) return (0); @@ -2956,7 +2979,7 @@ spa_try_repair(spa_t *spa, nvlist_t *config) static int spa_load(spa_t *spa, spa_load_state_t state, spa_import_type_t type) { - char *ereport = FM_EREPORT_ZFS_POOL; + const char *ereport = FM_EREPORT_ZFS_POOL; int error; spa->spa_load_state = state; @@ -3273,7 +3296,7 @@ spa_activity_check(spa_t *spa, uberblock_t *ub, nvlist_t *config) * ZPOOL_CONFIG_MMP_HOSTID - hostid from the active pool */ if (error == EREMOTEIO) { - char *hostname = ""; + const char *hostname = ""; uint64_t hostid = 0; if (mmp_label) { @@ -4190,6 +4213,7 @@ spa_ld_get_props(spa_t *spa) spa->spa_avz_action = AVZ_ACTION_INITIALIZE; ASSERT0(vdev_count_verify_zaps(spa->spa_root_vdev)); } else if (error != 0) { + nvlist_free(mos_config); return (spa_vdev_err(rvd, VDEV_AUX_CORRUPT_DATA, EIO)); } else if (!nvlist_exists(mos_config, ZPOOL_CONFIG_HAS_PER_VDEV_ZAPS)) { /* @@ -4350,7 +4374,7 @@ spa_ld_load_vdev_metadata(spa_t *spa) error = spa_ld_log_spacemaps(spa); if (error != 0) { - spa_load_failed(spa, "spa_ld_log_sm_data failed [error=%d]", + spa_load_failed(spa, "spa_ld_log_spacemaps failed [error=%d]", error); return (spa_vdev_err(rvd, VDEV_AUX_CORRUPT_DATA, error)); } @@ -4381,7 +4405,7 @@ spa_ld_load_dedup_tables(spa_t *spa) } static int -spa_ld_verify_logs(spa_t *spa, spa_import_type_t type, char **ereport) +spa_ld_verify_logs(spa_t *spa, spa_import_type_t type, const char **ereport) { vdev_t *rvd = spa->spa_root_vdev; @@ -4748,7 +4772,7 @@ spa_ld_mos_with_trusted_config(spa_t *spa, spa_import_type_t type, * config stored in the MOS. */ static int -spa_load_impl(spa_t *spa, spa_import_type_t type, char **ereport) +spa_load_impl(spa_t *spa, spa_import_type_t type, const char **ereport) { int error = 0; boolean_t missing_feat_write = B_FALSE; @@ -5139,8 +5163,8 @@ spa_load_best(spa_t *spa, spa_load_state_t state, uint64_t max_request, * ambiguous state. */ static int -spa_open_common(const char *pool, spa_t **spapp, void *tag, nvlist_t *nvpolicy, - nvlist_t **config) +spa_open_common(const char *pool, spa_t **spapp, const void *tag, + nvlist_t *nvpolicy, nvlist_t **config) { spa_t *spa; spa_load_state_t state = SPA_LOAD_OPEN; @@ -5256,14 +5280,14 @@ spa_open_common(const char *pool, spa_t **spapp, void *tag, nvlist_t *nvpolicy, } int -spa_open_rewind(const char *name, spa_t **spapp, void *tag, nvlist_t *policy, - nvlist_t **config) +spa_open_rewind(const char *name, spa_t **spapp, const void *tag, + nvlist_t *policy, nvlist_t **config) { return (spa_open_common(name, spapp, tag, policy, config)); } int -spa_open(const char *name, spa_t **spapp, void *tag) +spa_open(const char *name, spa_t **spapp, const void *tag) { return (spa_open_common(name, spapp, tag, NULL, NULL)); } @@ -6031,6 +6055,8 @@ spa_create(const char *pool, nvlist_t *nvroot, nvlist_t *props, spa->spa_minref = zfs_refcount_count(&spa->spa_refcount); spa->spa_load_state = SPA_LOAD_NONE; + spa_import_os(spa); + mutex_exit(&spa_namespace_lock); return (0); @@ -6214,6 +6240,8 @@ spa_import(char *pool, nvlist_t *config, nvlist_t *props, uint64_t flags) zvol_create_minors_recursive(pool); + spa_import_os(spa); + return (0); } @@ -6461,6 +6489,8 @@ spa_export_common(const char *pool, int new_state, nvlist_t **oldconfig, } export_spa: + spa_export_os(spa); + if (new_state == POOL_STATE_DESTROYED) spa_event_notify(spa, NULL, NULL, ESC_ZFS_POOL_DESTROY); else if (new_state == POOL_STATE_EXPORTED) @@ -7483,7 +7513,7 @@ spa_vdev_trim(spa_t *spa, nvlist_t *nv, uint64_t cmd_type, uint64_t rate, * Split a set of devices from their mirrors, and create a new pool from them. */ int -spa_vdev_split_mirror(spa_t *spa, char *newname, nvlist_t *config, +spa_vdev_split_mirror(spa_t *spa, const char *newname, nvlist_t *config, nvlist_t *props, boolean_t exp) { int error = 0; @@ -8128,7 +8158,7 @@ spa_async_autoexpand(spa_t *spa, vdev_t *vd) spa_event_notify(vd->vdev_spa, vd, NULL, ESC_ZFS_VDEV_AUTOEXPAND); } -static void +static __attribute__((noreturn)) void spa_async_thread(void *arg) { spa_t *spa = (spa_t *)arg; @@ -8484,7 +8514,7 @@ spa_sync_nvlist(spa_t *spa, uint64_t obj, nvlist_t *nv, dmu_tx_t *tx) VERIFY(nvlist_pack(nv, &packed, &nvsize, NV_ENCODE_XDR, KM_SLEEP) == 0); - bzero(packed + nvsize, bufsize - nvsize); + memset(packed + nvsize, 0, bufsize - nvsize); dmu_write(spa->spa_meta_objset, obj, 0, bufsize, packed, tx); diff --git a/module/zfs/spa_boot.c b/module/zfs/spa_boot.c index 674394650f82..fddb5c3c9683 100644 --- a/module/zfs/spa_boot.c +++ b/module/zfs/spa_boot.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/spa_checkpoint.c b/module/zfs/spa_checkpoint.c index 68c3ae2e0c31..b5b1dfa8a083 100644 --- a/module/zfs/spa_checkpoint.c +++ b/module/zfs/spa_checkpoint.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -166,7 +166,7 @@ spa_checkpoint_get_stats(spa_t *spa, pool_checkpoint_stat_t *pcs) if (!spa_feature_is_active(spa, SPA_FEATURE_POOL_CHECKPOINT)) return (SET_ERROR(ZFS_ERR_NO_CHECKPOINT)); - bzero(pcs, sizeof (pool_checkpoint_stat_t)); + memset(pcs, 0, sizeof (pool_checkpoint_stat_t)); int error = zap_contains(spa_meta_objset(spa), DMU_POOL_DIRECTORY_OBJECT, DMU_POOL_ZPOOL_CHECKPOINT); diff --git a/module/zfs/spa_config.c b/module/zfs/spa_config.c index 254031f31aff..91ac5c05e8af 100644 --- a/module/zfs/spa_config.c +++ b/module/zfs/spa_config.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -67,7 +67,7 @@ static uint64_t spa_config_generation = 1; * This can be overridden in userland to preserve an alternate namespace for * userland pools when doing testing. */ -char *spa_config_path = ZPOOL_CACHE; +char *spa_config_path = (char *)ZPOOL_CACHE; #ifdef _KERNEL static int zfs_autoimport_disable = B_TRUE; #endif diff --git a/module/zfs/spa_errlog.c b/module/zfs/spa_errlog.c index c6b28ea7d1b8..4572a6e56f0b 100644 --- a/module/zfs/spa_errlog.c +++ b/module/zfs/spa_errlog.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -20,7 +20,9 @@ */ /* * Copyright (c) 2006, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2013, 2014 by Delphix. All rights reserved. + * Copyright (c) 2013, 2014, Delphix. All rights reserved. + * Copyright (c) 2021, George Amanakis. All rights reserved. + * Copyright (c) 2019 Datto Inc. */ /* @@ -43,6 +45,16 @@ * calculation when the data is requested, storing the result so future queries * will be faster. * + * If the head_errlog feature is enabled, a different on-disk format is used. + * The error log of each head dataset is stored separately in the zap object + * and keyed by the head id. This enables listing every dataset affected in + * userland. In order to be able to track whether an error block has been + * modified or added to snapshots since it was marked as an error, a new tuple + * is introduced: zbookmark_err_phys_t. It allows the storage of the birth + * transaction group of an error block on-disk. The birth transaction group is + * used by check_filesystem() to assess whether this block was freed, + * re-written or added to a snapshot since its marking as an error. + * * This log is then shipped into an nvlist where the key is the dataset name and * the value is the object name. Userland is then responsible for uniquifying * this list and displaying it to the user. @@ -53,7 +65,19 @@ #include #include #include +#include +#include +#include +#define NAME_MAX_LEN 64 + +/* + * spa_upgrade_errlog_limit : A zfs module parameter that controls the number + * of on-disk error log entries that will be converted to the new + * format when enabling head_errlog. Defaults to 0 which converts + * all log entries. + */ +static uint32_t spa_upgrade_errlog_limit = 0; /* * Convert a bookmark to a string. @@ -67,9 +91,35 @@ bookmark_to_name(zbookmark_phys_t *zb, char *buf, size_t len) } /* - * Convert a string to a bookmark + * Convert an err_phys to a string. + */ +static void +errphys_to_name(zbookmark_err_phys_t *zep, char *buf, size_t len) +{ + (void) snprintf(buf, len, "%llx:%llx:%llx:%llx", + (u_longlong_t)zep->zb_object, (u_longlong_t)zep->zb_level, + (u_longlong_t)zep->zb_blkid, (u_longlong_t)zep->zb_birth); +} + +/* + * Convert a string to a err_phys. + */ +static void +name_to_errphys(char *buf, zbookmark_err_phys_t *zep) +{ + zep->zb_object = zfs_strtonum(buf, &buf); + ASSERT(*buf == ':'); + zep->zb_level = (int)zfs_strtonum(buf + 1, &buf); + ASSERT(*buf == ':'); + zep->zb_blkid = zfs_strtonum(buf + 1, &buf); + ASSERT(*buf == ':'); + zep->zb_birth = zfs_strtonum(buf + 1, &buf); + ASSERT(*buf == '\0'); +} + +/* + * Convert a string to a bookmark. */ -#ifdef _KERNEL static void name_to_bookmark(char *buf, zbookmark_phys_t *zb) { @@ -82,8 +132,74 @@ name_to_bookmark(char *buf, zbookmark_phys_t *zb) zb->zb_blkid = zfs_strtonum(buf + 1, &buf); ASSERT(*buf == '\0'); } + +#ifdef _KERNEL +static void +zep_to_zb(uint64_t dataset, zbookmark_err_phys_t *zep, zbookmark_phys_t *zb) +{ + zb->zb_objset = dataset; + zb->zb_object = zep->zb_object; + zb->zb_level = zep->zb_level; + zb->zb_blkid = zep->zb_blkid; +} #endif +static void +name_to_object(char *buf, uint64_t *obj) +{ + *obj = zfs_strtonum(buf, &buf); + ASSERT(*buf == '\0'); +} + +static int +get_head_and_birth_txg(spa_t *spa, zbookmark_err_phys_t *zep, uint64_t ds_obj, + uint64_t *head_dataset_id) +{ + dsl_pool_t *dp = spa->spa_dsl_pool; + dsl_dataset_t *ds; + objset_t *os; + + dsl_pool_config_enter(dp, FTAG); + int error = dsl_dataset_hold_obj(dp, ds_obj, FTAG, &ds); + if (error != 0) { + dsl_pool_config_exit(dp, FTAG); + return (error); + } + ASSERT(head_dataset_id); + *head_dataset_id = dsl_dir_phys(ds->ds_dir)->dd_head_dataset_obj; + + error = dmu_objset_from_ds(ds, &os); + if (error != 0) { + dsl_dataset_rele(ds, FTAG); + dsl_pool_config_exit(dp, FTAG); + return (error); + } + + dnode_t *dn; + blkptr_t bp; + + error = dnode_hold(os, zep->zb_object, FTAG, &dn); + if (error != 0) { + dsl_dataset_rele(ds, FTAG); + dsl_pool_config_exit(dp, FTAG); + return (error); + } + + rw_enter(&dn->dn_struct_rwlock, RW_READER); + error = dbuf_dnode_findbp(dn, zep->zb_level, zep->zb_blkid, &bp, NULL, + NULL); + + if (error == 0 && BP_IS_HOLE(&bp)) + error = SET_ERROR(ENOENT); + + zep->zb_birth = bp.blk_birth; + rw_exit(&dn->dn_struct_rwlock); + dnode_rele(dn, FTAG); + dsl_dataset_rele(ds, FTAG); + dsl_pool_config_exit(dp, FTAG); + return (error); +} + /* * Log an uncorrectable error to the persistent error log. We add it to the * spa's list of pending errors. The changes are actually synced out to disk @@ -128,91 +244,659 @@ spa_log_error(spa_t *spa, const zbookmark_phys_t *zb) mutex_exit(&spa->spa_errlist_lock); } +#ifdef _KERNEL +static int +find_birth_txg(dsl_dataset_t *ds, zbookmark_err_phys_t *zep, + uint64_t *birth_txg) +{ + objset_t *os; + int error = dmu_objset_from_ds(ds, &os); + if (error != 0) + return (error); + + dnode_t *dn; + blkptr_t bp; + + error = dnode_hold(os, zep->zb_object, FTAG, &dn); + if (error != 0) + return (error); + + rw_enter(&dn->dn_struct_rwlock, RW_READER); + error = dbuf_dnode_findbp(dn, zep->zb_level, zep->zb_blkid, &bp, NULL, + NULL); + + if (error == 0 && BP_IS_HOLE(&bp)) + error = SET_ERROR(ENOENT); + + *birth_txg = bp.blk_birth; + rw_exit(&dn->dn_struct_rwlock); + dnode_rele(dn, FTAG); + return (error); +} + /* - * Return the number of errors currently in the error log. This is actually the - * sum of both the last log and the current log, since we don't know the union - * of these logs until we reach userland. + * This function serves a double role. If only_count is true, it returns + * (in *count) how many times an error block belonging to this filesystem is + * referenced by snapshots or clones. If only_count is false, each time the + * error block is referenced by a snapshot or clone, it fills the userspace + * array at uaddr with the bookmarks of the error blocks. The array is filled + * from the back and *count is modified to be the number of unused entries at + * the beginning of the array. */ -uint64_t -spa_get_errlog_size(spa_t *spa) +static int +check_filesystem(spa_t *spa, uint64_t head_ds, zbookmark_err_phys_t *zep, + uint64_t *count, void *uaddr, boolean_t only_count) { - uint64_t total = 0, count; + dsl_dataset_t *ds; + dsl_pool_t *dp = spa->spa_dsl_pool; + + int error = dsl_dataset_hold_obj(dp, head_ds, FTAG, &ds); + if (error != 0) + return (error); + + uint64_t latest_txg; + uint64_t txg_to_consider = spa->spa_syncing_txg; + boolean_t check_snapshot = B_TRUE; + error = find_birth_txg(ds, zep, &latest_txg); + if (error == 0) { + if (zep->zb_birth == latest_txg) { + /* Block neither free nor rewritten. */ + if (!only_count) { + zbookmark_phys_t zb; + zep_to_zb(head_ds, zep, &zb); + if (copyout(&zb, (char *)uaddr + (*count - 1) + * sizeof (zbookmark_phys_t), + sizeof (zbookmark_phys_t)) != 0) { + dsl_dataset_rele(ds, FTAG); + return (SET_ERROR(EFAULT)); + } + (*count)--; + } else { + (*count)++; + } + check_snapshot = B_FALSE; + } else { + ASSERT3U(zep->zb_birth, <, latest_txg); + txg_to_consider = latest_txg; + } + } + + /* How many snapshots reference this block. */ + uint64_t snap_count; + error = zap_count(spa->spa_meta_objset, + dsl_dataset_phys(ds)->ds_snapnames_zapobj, &snap_count); + if (error != 0) { + dsl_dataset_rele(ds, FTAG); + return (error); + } + + if (snap_count == 0) { + /* File system has no snapshot. */ + dsl_dataset_rele(ds, FTAG); + return (0); + } + + uint64_t *snap_obj_array = kmem_alloc(snap_count * sizeof (uint64_t), + KM_SLEEP); + + int aff_snap_count = 0; + uint64_t snap_obj = dsl_dataset_phys(ds)->ds_prev_snap_obj; + uint64_t snap_obj_txg = dsl_dataset_phys(ds)->ds_prev_snap_txg; + + /* Check only snapshots created from this file system. */ + while (snap_obj != 0 && zep->zb_birth < snap_obj_txg && + snap_obj_txg <= txg_to_consider) { + + dsl_dataset_rele(ds, FTAG); + error = dsl_dataset_hold_obj(dp, snap_obj, FTAG, &ds); + if (error != 0) + goto out; + + if (dsl_dir_phys(ds->ds_dir)->dd_head_dataset_obj != head_ds) + break; + + boolean_t affected = B_TRUE; + if (check_snapshot) { + uint64_t blk_txg; + error = find_birth_txg(ds, zep, &blk_txg); + affected = (error == 0 && zep->zb_birth == blk_txg); + } + + if (affected) { + snap_obj_array[aff_snap_count] = snap_obj; + aff_snap_count++; + + if (!only_count) { + zbookmark_phys_t zb; + zep_to_zb(snap_obj, zep, &zb); + if (copyout(&zb, (char *)uaddr + (*count - 1) * + sizeof (zbookmark_phys_t), + sizeof (zbookmark_phys_t)) != 0) { + dsl_dataset_rele(ds, FTAG); + error = SET_ERROR(EFAULT); + goto out; + } + (*count)--; + } else { + (*count)++; + } + + /* + * Only clones whose origins were affected could also + * have affected snapshots. + */ + zap_cursor_t zc; + zap_attribute_t za; + for (zap_cursor_init(&zc, spa->spa_meta_objset, + dsl_dataset_phys(ds)->ds_next_clones_obj); + zap_cursor_retrieve(&zc, &za) == 0; + zap_cursor_advance(&zc)) { + error = check_filesystem(spa, + za.za_first_integer, zep, + count, uaddr, only_count); + + if (error != 0) { + zap_cursor_fini(&zc); + goto out; + } + } + zap_cursor_fini(&zc); + } + snap_obj_txg = dsl_dataset_phys(ds)->ds_prev_snap_txg; + snap_obj = dsl_dataset_phys(ds)->ds_prev_snap_obj; + } + dsl_dataset_rele(ds, FTAG); +out: + kmem_free(snap_obj_array, sizeof (*snap_obj_array)); + return (error); +} + +static int +find_top_affected_fs(spa_t *spa, uint64_t head_ds, zbookmark_err_phys_t *zep, + uint64_t *top_affected_fs) +{ + uint64_t oldest_dsobj; + int error = dsl_dataset_oldest_snapshot(spa, head_ds, zep->zb_birth, + &oldest_dsobj); + if (error != 0) + return (error); + + dsl_dataset_t *ds; + error = dsl_dataset_hold_obj(spa->spa_dsl_pool, oldest_dsobj, + FTAG, &ds); + if (error != 0) + return (error); + + *top_affected_fs = + dsl_dir_phys(ds->ds_dir)->dd_head_dataset_obj; + dsl_dataset_rele(ds, FTAG); + return (0); +} + +static int +process_error_block(spa_t *spa, uint64_t head_ds, zbookmark_err_phys_t *zep, + uint64_t *count, void *uaddr, boolean_t only_count) +{ + dsl_pool_t *dp = spa->spa_dsl_pool; + dsl_pool_config_enter(dp, FTAG); + uint64_t top_affected_fs; + + int error = find_top_affected_fs(spa, head_ds, zep, &top_affected_fs); + if (error == 0) + error = check_filesystem(spa, top_affected_fs, zep, count, + uaddr, only_count); + + dsl_pool_config_exit(dp, FTAG); + return (error); +} + +static uint64_t +get_errlog_size(spa_t *spa, uint64_t spa_err_obj) +{ + if (spa_err_obj == 0) + return (0); + uint64_t total = 0; + + zap_cursor_t zc; + zap_attribute_t za; + for (zap_cursor_init(&zc, spa->spa_meta_objset, spa_err_obj); + zap_cursor_retrieve(&zc, &za) == 0; zap_cursor_advance(&zc)) { + + zap_cursor_t head_ds_cursor; + zap_attribute_t head_ds_attr; + zbookmark_err_phys_t head_ds_block; + + uint64_t head_ds; + name_to_object(za.za_name, &head_ds); + + for (zap_cursor_init(&head_ds_cursor, spa->spa_meta_objset, + za.za_first_integer); zap_cursor_retrieve(&head_ds_cursor, + &head_ds_attr) == 0; zap_cursor_advance(&head_ds_cursor)) { + + name_to_errphys(head_ds_attr.za_name, &head_ds_block); + (void) process_error_block(spa, head_ds, &head_ds_block, + &total, NULL, B_TRUE); + } + zap_cursor_fini(&head_ds_cursor); + } + zap_cursor_fini(&zc); + return (total); +} + +static uint64_t +get_errlist_size(spa_t *spa, avl_tree_t *tree) +{ + if (avl_numnodes(tree) == 0) + return (0); + uint64_t total = 0; + + spa_error_entry_t *se; + for (se = avl_first(tree); se != NULL; se = AVL_NEXT(tree, se)) { + zbookmark_err_phys_t zep; + zep.zb_object = se->se_bookmark.zb_object; + zep.zb_level = se->se_bookmark.zb_level; + zep.zb_blkid = se->se_bookmark.zb_blkid; + + /* + * If we cannot find out the head dataset and birth txg of + * the present error block, we opt not to error out. In the + * next pool sync this information will be retrieved by + * sync_error_list() and written to the on-disk error log. + */ + uint64_t head_ds_obj; + if (get_head_and_birth_txg(spa, &zep, + se->se_bookmark.zb_objset, &head_ds_obj) == 0) + (void) process_error_block(spa, head_ds_obj, &zep, + &total, NULL, B_TRUE); + } + return (total); +} +#endif + +/* + * If a healed bookmark matches an entry in the error log we stash it in a tree + * so that we can later remove the related log entries in sync context. + */ +static void +spa_add_healed_error(spa_t *spa, uint64_t obj, zbookmark_phys_t *healed_zb) +{ + char name[NAME_MAX_LEN]; + + if (obj == 0) + return; + + bookmark_to_name(healed_zb, name, sizeof (name)); mutex_enter(&spa->spa_errlog_lock); - if (spa->spa_errlog_scrub != 0 && - zap_count(spa->spa_meta_objset, spa->spa_errlog_scrub, - &count) == 0) - total += count; - - if (spa->spa_errlog_last != 0 && !spa->spa_scrub_finished && - zap_count(spa->spa_meta_objset, spa->spa_errlog_last, - &count) == 0) - total += count; + if (zap_contains(spa->spa_meta_objset, obj, name) == 0) { + /* + * Found an error matching healed zb, add zb to our + * tree of healed errors + */ + avl_tree_t *tree = &spa->spa_errlist_healed; + spa_error_entry_t search; + spa_error_entry_t *new; + avl_index_t where; + search.se_bookmark = *healed_zb; + mutex_enter(&spa->spa_errlist_lock); + if (avl_find(tree, &search, &where) != NULL) { + mutex_exit(&spa->spa_errlist_lock); + mutex_exit(&spa->spa_errlog_lock); + return; + } + new = kmem_zalloc(sizeof (spa_error_entry_t), KM_SLEEP); + new->se_bookmark = *healed_zb; + avl_insert(tree, new, where); + mutex_exit(&spa->spa_errlist_lock); + } mutex_exit(&spa->spa_errlog_lock); +} + +/* + * If this error exists in the given tree remove it. + */ +static void +remove_error_from_list(spa_t *spa, avl_tree_t *t, const zbookmark_phys_t *zb) +{ + spa_error_entry_t search, *found; + avl_index_t where; mutex_enter(&spa->spa_errlist_lock); - total += avl_numnodes(&spa->spa_errlist_last); - total += avl_numnodes(&spa->spa_errlist_scrub); + search.se_bookmark = *zb; + if ((found = avl_find(t, &search, &where)) != NULL) { + avl_remove(t, found); + kmem_free(found, sizeof (spa_error_entry_t)); + } mutex_exit(&spa->spa_errlist_lock); +} - return (total); + +/* + * Removes all of the recv healed errors from both on-disk error logs + */ +static void +spa_remove_healed_errors(spa_t *spa, avl_tree_t *s, avl_tree_t *l, dmu_tx_t *tx) +{ + char name[NAME_MAX_LEN]; + spa_error_entry_t *se; + void *cookie = NULL; + + ASSERT(MUTEX_HELD(&spa->spa_errlog_lock)); + + while ((se = avl_destroy_nodes(&spa->spa_errlist_healed, + &cookie)) != NULL) { + remove_error_from_list(spa, s, &se->se_bookmark); + remove_error_from_list(spa, l, &se->se_bookmark); + bookmark_to_name(&se->se_bookmark, name, sizeof (name)); + kmem_free(se, sizeof (spa_error_entry_t)); + (void) zap_remove(spa->spa_meta_objset, + spa->spa_errlog_last, name, tx); + (void) zap_remove(spa->spa_meta_objset, + spa->spa_errlog_scrub, name, tx); + } +} + +/* + * Stash away healed bookmarks to remove them from the on-disk error logs + * later in spa_remove_healed_errors(). + */ +void +spa_remove_error(spa_t *spa, zbookmark_phys_t *zb) +{ + char name[NAME_MAX_LEN]; + + bookmark_to_name(zb, name, sizeof (name)); + + spa_add_healed_error(spa, spa->spa_errlog_last, zb); + spa_add_healed_error(spa, spa->spa_errlog_scrub, zb); } +/* + * Return the number of errors currently in the error log. This is actually the + * sum of both the last log and the current log, since we don't know the union + * of these logs until we reach userland. + */ +uint64_t +spa_get_errlog_size(spa_t *spa) +{ + uint64_t total = 0; + + if (!spa_feature_is_enabled(spa, SPA_FEATURE_HEAD_ERRLOG)) { + mutex_enter(&spa->spa_errlog_lock); + uint64_t count; + if (spa->spa_errlog_scrub != 0 && + zap_count(spa->spa_meta_objset, spa->spa_errlog_scrub, + &count) == 0) + total += count; + + if (spa->spa_errlog_last != 0 && !spa->spa_scrub_finished && + zap_count(spa->spa_meta_objset, spa->spa_errlog_last, + &count) == 0) + total += count; + mutex_exit(&spa->spa_errlog_lock); + + mutex_enter(&spa->spa_errlist_lock); + total += avl_numnodes(&spa->spa_errlist_last); + total += avl_numnodes(&spa->spa_errlist_scrub); + mutex_exit(&spa->spa_errlist_lock); + } else { #ifdef _KERNEL -static int -process_error_log(spa_t *spa, uint64_t obj, void *addr, size_t *count) + mutex_enter(&spa->spa_errlog_lock); + total += get_errlog_size(spa, spa->spa_errlog_last); + total += get_errlog_size(spa, spa->spa_errlog_scrub); + mutex_exit(&spa->spa_errlog_lock); + + mutex_enter(&spa->spa_errlist_lock); + total += get_errlist_size(spa, &spa->spa_errlist_last); + total += get_errlist_size(spa, &spa->spa_errlist_scrub); + mutex_exit(&spa->spa_errlist_lock); +#endif + } + return (total); +} + +/* + * This function sweeps through an on-disk error log and stores all bookmarks + * as error bookmarks in a new ZAP object. At the end we discard the old one, + * and spa_update_errlog() will set the spa's on-disk error log to new ZAP + * object. + */ +static void +sync_upgrade_errlog(spa_t *spa, uint64_t spa_err_obj, uint64_t *newobj, + dmu_tx_t *tx) { zap_cursor_t zc; zap_attribute_t za; zbookmark_phys_t zb; + uint64_t count; - if (obj == 0) - return (0); + *newobj = zap_create(spa->spa_meta_objset, DMU_OT_ERROR_LOG, + DMU_OT_NONE, 0, tx); - for (zap_cursor_init(&zc, spa->spa_meta_objset, obj); + /* + * If we cannnot perform the upgrade we should clear the old on-disk + * error logs. + */ + if (zap_count(spa->spa_meta_objset, spa_err_obj, &count) != 0) { + VERIFY0(dmu_object_free(spa->spa_meta_objset, spa_err_obj, tx)); + return; + } + + for (zap_cursor_init(&zc, spa->spa_meta_objset, spa_err_obj); zap_cursor_retrieve(&zc, &za) == 0; zap_cursor_advance(&zc)) { + if (spa_upgrade_errlog_limit != 0 && + zc.zc_cd == spa_upgrade_errlog_limit) + break; - if (*count == 0) { - zap_cursor_fini(&zc); - return (SET_ERROR(ENOMEM)); + name_to_bookmark(za.za_name, &zb); + + zbookmark_err_phys_t zep; + zep.zb_object = zb.zb_object; + zep.zb_level = zb.zb_level; + zep.zb_blkid = zb.zb_blkid; + + /* + * We cannot use get_head_and_birth_txg() because it will + * acquire the pool config lock, which we already have. In case + * of an error we simply continue. + */ + uint64_t head_dataset_obj; + dsl_pool_t *dp = spa->spa_dsl_pool; + dsl_dataset_t *ds; + objset_t *os; + + int error = dsl_dataset_hold_obj(dp, zb.zb_objset, FTAG, &ds); + if (error != 0) + continue; + + head_dataset_obj = + dsl_dir_phys(ds->ds_dir)->dd_head_dataset_obj; + + /* + * The objset and the dnode are required for getting the block + * pointer, which is used to determine if BP_IS_HOLE(). If + * getting the objset or the dnode fails, do not create a + * zap entry (presuming we know the dataset) as this may create + * spurious errors that we cannot ever resolve. If an error is + * truly persistent, it should re-appear after a scan. + */ + if (dmu_objset_from_ds(ds, &os) != 0) { + dsl_dataset_rele(ds, FTAG); + continue; } - name_to_bookmark(za.za_name, &zb); + dnode_t *dn; + blkptr_t bp; - if (copyout(&zb, (char *)addr + - (*count - 1) * sizeof (zbookmark_phys_t), - sizeof (zbookmark_phys_t)) != 0) { - zap_cursor_fini(&zc); - return (SET_ERROR(EFAULT)); + if (dnode_hold(os, zep.zb_object, FTAG, &dn) != 0) { + dsl_dataset_rele(ds, FTAG); + continue; } - *count -= 1; - } + rw_enter(&dn->dn_struct_rwlock, RW_READER); + error = dbuf_dnode_findbp(dn, zep.zb_level, zep.zb_blkid, &bp, + NULL, NULL); + + zep.zb_birth = bp.blk_birth; + rw_exit(&dn->dn_struct_rwlock); + dnode_rele(dn, FTAG); + dsl_dataset_rele(ds, FTAG); + + if (error != 0 || BP_IS_HOLE(&bp)) + continue; + + uint64_t err_obj; + error = zap_lookup_int_key(spa->spa_meta_objset, *newobj, + head_dataset_obj, &err_obj); + + if (error == ENOENT) { + err_obj = zap_create(spa->spa_meta_objset, + DMU_OT_ERROR_LOG, DMU_OT_NONE, 0, tx); + + (void) zap_update_int_key(spa->spa_meta_objset, + *newobj, head_dataset_obj, err_obj, tx); + } + char buf[64]; + errphys_to_name(&zep, buf, sizeof (buf)); + + const char *name = ""; + (void) zap_update(spa->spa_meta_objset, err_obj, + buf, 1, strlen(name) + 1, name, tx); + } zap_cursor_fini(&zc); + VERIFY0(dmu_object_free(spa->spa_meta_objset, spa_err_obj, tx)); +} + +void +spa_upgrade_errlog(spa_t *spa, dmu_tx_t *tx) +{ + uint64_t newobj = 0; + + mutex_enter(&spa->spa_errlog_lock); + if (spa->spa_errlog_last != 0) { + sync_upgrade_errlog(spa, spa->spa_errlog_last, &newobj, tx); + spa->spa_errlog_last = newobj; + } + + if (spa->spa_errlog_scrub != 0) { + sync_upgrade_errlog(spa, spa->spa_errlog_scrub, &newobj, tx); + spa->spa_errlog_scrub = newobj; + } + mutex_exit(&spa->spa_errlog_lock); +} + +#ifdef _KERNEL +/* + * If an error block is shared by two datasets it will be counted twice. For + * detailed message see spa_get_errlog_size() above. + */ +static int +process_error_log(spa_t *spa, uint64_t obj, void *uaddr, uint64_t *count) +{ + zap_cursor_t zc; + zap_attribute_t za; + + if (obj == 0) + return (0); + + if (!spa_feature_is_enabled(spa, SPA_FEATURE_HEAD_ERRLOG)) { + for (zap_cursor_init(&zc, spa->spa_meta_objset, obj); + zap_cursor_retrieve(&zc, &za) == 0; + zap_cursor_advance(&zc)) { + if (*count == 0) { + zap_cursor_fini(&zc); + return (SET_ERROR(ENOMEM)); + } + + zbookmark_phys_t zb; + name_to_bookmark(za.za_name, &zb); + + if (copyout(&zb, (char *)uaddr + + (*count - 1) * sizeof (zbookmark_phys_t), + sizeof (zbookmark_phys_t)) != 0) { + zap_cursor_fini(&zc); + return (SET_ERROR(EFAULT)); + } + *count -= 1; + + } + zap_cursor_fini(&zc); + return (0); + } + + for (zap_cursor_init(&zc, spa->spa_meta_objset, obj); + zap_cursor_retrieve(&zc, &za) == 0; + zap_cursor_advance(&zc)) { + + zap_cursor_t head_ds_cursor; + zap_attribute_t head_ds_attr; + + uint64_t head_ds_err_obj = za.za_first_integer; + uint64_t head_ds; + name_to_object(za.za_name, &head_ds); + for (zap_cursor_init(&head_ds_cursor, spa->spa_meta_objset, + head_ds_err_obj); zap_cursor_retrieve(&head_ds_cursor, + &head_ds_attr) == 0; zap_cursor_advance(&head_ds_cursor)) { + + zbookmark_err_phys_t head_ds_block; + name_to_errphys(head_ds_attr.za_name, &head_ds_block); + int error = process_error_block(spa, head_ds, + &head_ds_block, count, uaddr, B_FALSE); + + if (error != 0) { + zap_cursor_fini(&head_ds_cursor); + zap_cursor_fini(&zc); + return (error); + } + } + zap_cursor_fini(&head_ds_cursor); + } + zap_cursor_fini(&zc); return (0); } static int -process_error_list(avl_tree_t *list, void *addr, size_t *count) +process_error_list(spa_t *spa, avl_tree_t *list, void *uaddr, uint64_t *count) { spa_error_entry_t *se; - for (se = avl_first(list); se != NULL; se = AVL_NEXT(list, se)) { + if (!spa_feature_is_enabled(spa, SPA_FEATURE_HEAD_ERRLOG)) { + for (se = avl_first(list); se != NULL; + se = AVL_NEXT(list, se)) { - if (*count == 0) - return (SET_ERROR(ENOMEM)); + if (*count == 0) + return (SET_ERROR(ENOMEM)); - if (copyout(&se->se_bookmark, (char *)addr + - (*count - 1) * sizeof (zbookmark_phys_t), - sizeof (zbookmark_phys_t)) != 0) - return (SET_ERROR(EFAULT)); + if (copyout(&se->se_bookmark, (char *)uaddr + + (*count - 1) * sizeof (zbookmark_phys_t), + sizeof (zbookmark_phys_t)) != 0) + return (SET_ERROR(EFAULT)); - *count -= 1; + *count -= 1; + } + return (0); } + for (se = avl_first(list); se != NULL; se = AVL_NEXT(list, se)) { + zbookmark_err_phys_t zep; + zep.zb_object = se->se_bookmark.zb_object; + zep.zb_level = se->se_bookmark.zb_level; + zep.zb_blkid = se->se_bookmark.zb_blkid; + + uint64_t head_ds_obj; + int error = get_head_and_birth_txg(spa, &zep, + se->se_bookmark.zb_objset, &head_ds_obj); + if (error != 0) + return (error); + + error = process_error_block(spa, head_ds_obj, &zep, count, + uaddr, B_FALSE); + if (error != 0) + return (error); + } return (0); } #endif @@ -229,7 +913,7 @@ process_error_list(avl_tree_t *list, void *addr, size_t *count) * the error list lock when we are finished. */ int -spa_get_errlog(spa_t *spa, void *uaddr, size_t *count) +spa_get_errlog(spa_t *spa, void *uaddr, uint64_t *count) { int ret = 0; @@ -244,10 +928,10 @@ spa_get_errlog(spa_t *spa, void *uaddr, size_t *count) mutex_enter(&spa->spa_errlist_lock); if (!ret) - ret = process_error_list(&spa->spa_errlist_scrub, uaddr, + ret = process_error_list(spa, &spa->spa_errlist_scrub, uaddr, count); if (!ret) - ret = process_error_list(&spa->spa_errlist_last, uaddr, + ret = process_error_list(spa, &spa->spa_errlist_last, uaddr, count); mutex_exit(&spa->spa_errlist_lock); @@ -299,35 +983,89 @@ spa_errlog_drain(spa_t *spa) /* * Process a list of errors into the current on-disk log. */ -static void +void sync_error_list(spa_t *spa, avl_tree_t *t, uint64_t *obj, dmu_tx_t *tx) { spa_error_entry_t *se; - char buf[64]; + char buf[NAME_MAX_LEN]; void *cookie; - if (avl_numnodes(t) != 0) { - /* create log if necessary */ - if (*obj == 0) - *obj = zap_create(spa->spa_meta_objset, - DMU_OT_ERROR_LOG, DMU_OT_NONE, - 0, tx); + if (avl_numnodes(t) == 0) + return; - /* add errors to the current log */ - for (se = avl_first(t); se != NULL; se = AVL_NEXT(t, se)) { - char *name = se->se_name ? se->se_name : ""; + /* create log if necessary */ + if (*obj == 0) + *obj = zap_create(spa->spa_meta_objset, DMU_OT_ERROR_LOG, + DMU_OT_NONE, 0, tx); + /* add errors to the current log */ + if (!spa_feature_is_enabled(spa, SPA_FEATURE_HEAD_ERRLOG)) { + for (se = avl_first(t); se != NULL; se = AVL_NEXT(t, se)) { bookmark_to_name(&se->se_bookmark, buf, sizeof (buf)); + const char *name = se->se_name ? se->se_name : ""; + (void) zap_update(spa->spa_meta_objset, *obj, buf, 1, + strlen(name) + 1, name, tx); + } + } else { + for (se = avl_first(t); se != NULL; se = AVL_NEXT(t, se)) { + zbookmark_err_phys_t zep; + zep.zb_object = se->se_bookmark.zb_object; + zep.zb_level = se->se_bookmark.zb_level; + zep.zb_blkid = se->se_bookmark.zb_blkid; + + /* + * If we cannot find out the head dataset and birth txg + * of the present error block, we simply continue. + * Reinserting that error block to the error lists, + * even if we are not syncing the final txg, results + * in duplicate posting of errors. + */ + uint64_t head_dataset_obj; + int error = get_head_and_birth_txg(spa, &zep, + se->se_bookmark.zb_objset, &head_dataset_obj); + if (error != 0) + continue; + + uint64_t err_obj; + error = zap_lookup_int_key(spa->spa_meta_objset, + *obj, head_dataset_obj, &err_obj); + + if (error == ENOENT) { + err_obj = zap_create(spa->spa_meta_objset, + DMU_OT_ERROR_LOG, DMU_OT_NONE, 0, tx); + + (void) zap_update_int_key(spa->spa_meta_objset, + *obj, head_dataset_obj, err_obj, tx); + } + errphys_to_name(&zep, buf, sizeof (buf)); + + const char *name = se->se_name ? se->se_name : ""; (void) zap_update(spa->spa_meta_objset, - *obj, buf, 1, strlen(name) + 1, name, tx); + err_obj, buf, 1, strlen(name) + 1, name, tx); } + } + /* purge the error list */ + cookie = NULL; + while ((se = avl_destroy_nodes(t, &cookie)) != NULL) + kmem_free(se, sizeof (spa_error_entry_t)); +} - /* purge the error list */ - cookie = NULL; - while ((se = avl_destroy_nodes(t, &cookie)) != NULL) - kmem_free(se, sizeof (spa_error_entry_t)); +static void +delete_errlog(spa_t *spa, uint64_t spa_err_obj, dmu_tx_t *tx) +{ + if (spa_feature_is_enabled(spa, SPA_FEATURE_HEAD_ERRLOG)) { + zap_cursor_t zc; + zap_attribute_t za; + for (zap_cursor_init(&zc, spa->spa_meta_objset, spa_err_obj); + zap_cursor_retrieve(&zc, &za) == 0; + zap_cursor_advance(&zc)) { + VERIFY0(dmu_object_free(spa->spa_meta_objset, + za.za_first_integer, tx)); + } + zap_cursor_fini(&zc); } + VERIFY0(dmu_object_free(spa->spa_meta_objset, spa_err_obj, tx)); } /* @@ -354,6 +1092,7 @@ spa_errlog_sync(spa_t *spa, uint64_t txg) */ if (avl_numnodes(&spa->spa_errlist_scrub) == 0 && avl_numnodes(&spa->spa_errlist_last) == 0 && + avl_numnodes(&spa->spa_errlist_healed) == 0 && !spa->spa_scrub_finished) { mutex_exit(&spa->spa_errlist_lock); return; @@ -368,6 +1107,11 @@ spa_errlog_sync(spa_t *spa, uint64_t txg) tx = dmu_tx_create_assigned(spa->spa_dsl_pool, txg); + /* + * Remove healed errors from errors. + */ + spa_remove_healed_errors(spa, &last, &scrub, tx); + /* * Sync out the current list of errors. */ @@ -378,8 +1122,7 @@ spa_errlog_sync(spa_t *spa, uint64_t txg) */ if (scrub_finished) { if (spa->spa_errlog_last != 0) - VERIFY(dmu_object_free(spa->spa_meta_objset, - spa->spa_errlog_last, tx) == 0); + delete_errlog(spa, spa->spa_errlog_last, tx); spa->spa_errlog_last = spa->spa_errlog_scrub; spa->spa_errlog_scrub = 0; @@ -406,6 +1149,137 @@ spa_errlog_sync(spa_t *spa, uint64_t txg) mutex_exit(&spa->spa_errlog_lock); } +static void +delete_dataset_errlog(spa_t *spa, uint64_t spa_err_obj, uint64_t ds, + dmu_tx_t *tx) +{ + if (spa_err_obj == 0) + return; + + zap_cursor_t zc; + zap_attribute_t za; + for (zap_cursor_init(&zc, spa->spa_meta_objset, spa_err_obj); + zap_cursor_retrieve(&zc, &za) == 0; zap_cursor_advance(&zc)) { + uint64_t head_ds; + name_to_object(za.za_name, &head_ds); + if (head_ds == ds) { + (void) zap_remove(spa->spa_meta_objset, spa_err_obj, + za.za_name, tx); + VERIFY0(dmu_object_free(spa->spa_meta_objset, + za.za_first_integer, tx)); + break; + } + } + zap_cursor_fini(&zc); +} + +void +spa_delete_dataset_errlog(spa_t *spa, uint64_t ds, dmu_tx_t *tx) +{ + mutex_enter(&spa->spa_errlog_lock); + delete_dataset_errlog(spa, spa->spa_errlog_scrub, ds, tx); + delete_dataset_errlog(spa, spa->spa_errlog_last, ds, tx); + mutex_exit(&spa->spa_errlog_lock); +} + +static int +find_txg_ancestor_snapshot(spa_t *spa, uint64_t new_head, uint64_t old_head, + uint64_t *txg) +{ + dsl_dataset_t *ds; + dsl_pool_t *dp = spa->spa_dsl_pool; + + int error = dsl_dataset_hold_obj(dp, old_head, FTAG, &ds); + if (error != 0) + return (error); + + uint64_t prev_obj = dsl_dataset_phys(ds)->ds_prev_snap_obj; + uint64_t prev_obj_txg = dsl_dataset_phys(ds)->ds_prev_snap_txg; + + while (prev_obj != 0) { + dsl_dataset_rele(ds, FTAG); + if ((error = dsl_dataset_hold_obj(dp, prev_obj, + FTAG, &ds)) == 0 && + dsl_dir_phys(ds->ds_dir)->dd_head_dataset_obj == new_head) + break; + + if (error != 0) + return (error); + + prev_obj_txg = dsl_dataset_phys(ds)->ds_prev_snap_txg; + prev_obj = dsl_dataset_phys(ds)->ds_prev_snap_obj; + } + dsl_dataset_rele(ds, FTAG); + ASSERT(prev_obj != 0); + *txg = prev_obj_txg; + return (0); +} + +static void +swap_errlog(spa_t *spa, uint64_t spa_err_obj, uint64_t new_head, uint64_t + old_head, dmu_tx_t *tx) +{ + if (spa_err_obj == 0) + return; + + uint64_t old_head_errlog; + int error = zap_lookup_int_key(spa->spa_meta_objset, spa_err_obj, + old_head, &old_head_errlog); + + /* If no error log, then there is nothing to do. */ + if (error != 0) + return; + + uint64_t txg; + error = find_txg_ancestor_snapshot(spa, new_head, old_head, &txg); + if (error != 0) + return; + + /* + * Create an error log if the file system being promoted does not + * already have one. + */ + uint64_t new_head_errlog; + error = zap_lookup_int_key(spa->spa_meta_objset, spa_err_obj, new_head, + &new_head_errlog); + + if (error != 0) { + new_head_errlog = zap_create(spa->spa_meta_objset, + DMU_OT_ERROR_LOG, DMU_OT_NONE, 0, tx); + + (void) zap_update_int_key(spa->spa_meta_objset, spa_err_obj, + new_head, new_head_errlog, tx); + } + + zap_cursor_t zc; + zap_attribute_t za; + zbookmark_err_phys_t err_block; + for (zap_cursor_init(&zc, spa->spa_meta_objset, old_head_errlog); + zap_cursor_retrieve(&zc, &za) == 0; zap_cursor_advance(&zc)) { + + const char *name = ""; + name_to_errphys(za.za_name, &err_block); + if (err_block.zb_birth < txg) { + (void) zap_update(spa->spa_meta_objset, new_head_errlog, + za.za_name, 1, strlen(name) + 1, name, tx); + + (void) zap_remove(spa->spa_meta_objset, old_head_errlog, + za.za_name, tx); + } + } + zap_cursor_fini(&zc); +} + +void +spa_swap_errlog(spa_t *spa, uint64_t new_head_ds, uint64_t old_head_ds, + dmu_tx_t *tx) +{ + mutex_enter(&spa->spa_errlog_lock); + swap_errlog(spa, spa->spa_errlog_scrub, new_head_ds, old_head_ds, tx); + swap_errlog(spa, spa->spa_errlog_last, new_head_ds, old_head_ds, tx); + mutex_exit(&spa->spa_errlog_lock); +} + #if defined(_KERNEL) /* error handling */ EXPORT_SYMBOL(spa_log_error); @@ -415,4 +1289,14 @@ EXPORT_SYMBOL(spa_errlog_rotate); EXPORT_SYMBOL(spa_errlog_drain); EXPORT_SYMBOL(spa_errlog_sync); EXPORT_SYMBOL(spa_get_errlists); +EXPORT_SYMBOL(spa_delete_dataset_errlog); +EXPORT_SYMBOL(spa_swap_errlog); +EXPORT_SYMBOL(sync_error_list); +EXPORT_SYMBOL(spa_upgrade_errlog); #endif + +/* BEGIN CSTYLED */ +ZFS_MODULE_PARAM(zfs_spa, spa_, upgrade_errlog_limit, INT, ZMOD_RW, + "Limit the number of errors which will be upgraded to the new " + "on-disk error log when enabling head_errlog"); +/* END CSTYLED */ diff --git a/module/zfs/spa_history.c b/module/zfs/spa_history.c index dae06e46c316..6d468e716421 100644 --- a/module/zfs/spa_history.c +++ b/module/zfs/spa_history.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -248,7 +248,6 @@ spa_history_log_notify(spa_t *spa, nvlist_t *nvl) /* * Write out a history event. */ -/*ARGSUSED*/ static void spa_history_log_sync(void *arg, dmu_tx_t *tx) { diff --git a/module/zfs/spa_log_spacemap.c b/module/zfs/spa_log_spacemap.c index 110a4eab99f9..19e334916bd0 100644 --- a/module/zfs/spa_log_spacemap.c +++ b/module/zfs/spa_log_spacemap.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -257,7 +257,12 @@ static unsigned long zfs_unflushed_log_block_min = 1000; * terms of performance. Thus we have a hard limit in the size of the log in * terms of blocks. */ -static unsigned long zfs_unflushed_log_block_max = (1ULL << 18); +static unsigned long zfs_unflushed_log_block_max = (1ULL << 17); + +/* + * Also we have a hard limit in the size of the log in terms of dirty TXGs. + */ +static unsigned long zfs_unflushed_log_txg_max = 1000; /* * Max # of rows allowed for the log_summary. The tradeoff here is accuracy and @@ -333,9 +338,13 @@ spa_log_sm_set_blocklimit(spa_t *spa) return; } - uint64_t calculated_limit = - (spa_total_metaslabs(spa) * zfs_unflushed_log_block_pct) / 100; - spa->spa_unflushed_stats.sus_blocklimit = MIN(MAX(calculated_limit, + uint64_t msdcount = 0; + for (log_summary_entry_t *e = list_head(&spa->spa_log_summary); + e; e = list_next(&spa->spa_log_summary, e)) + msdcount += e->lse_msdcount; + + uint64_t limit = msdcount * zfs_unflushed_log_block_pct / 100; + spa->spa_unflushed_stats.sus_blocklimit = MIN(MAX(limit, zfs_unflushed_log_block_min), zfs_unflushed_log_block_max); } @@ -380,8 +389,13 @@ spa_log_summary_verify_counts(spa_t *spa) } static boolean_t -summary_entry_is_full(spa_t *spa, log_summary_entry_t *e) +summary_entry_is_full(spa_t *spa, log_summary_entry_t *e, uint64_t txg) { + if (e->lse_end == txg) + return (0); + if (e->lse_txgcount >= DIV_ROUND_UP(zfs_unflushed_log_txg_max, + zfs_max_logsm_summary_length)) + return (1); uint64_t blocks_per_row = MAX(1, DIV_ROUND_UP(spa_log_sm_blocklimit(spa), zfs_max_logsm_summary_length)); @@ -401,7 +415,7 @@ summary_entry_is_full(spa_t *spa, log_summary_entry_t *e) * the metaslab. */ void -spa_log_summary_decrement_mscount(spa_t *spa, uint64_t txg) +spa_log_summary_decrement_mscount(spa_t *spa, uint64_t txg, boolean_t dirty) { /* * We don't track summary data for read-only pools and this function @@ -429,6 +443,8 @@ spa_log_summary_decrement_mscount(spa_t *spa, uint64_t txg) } target->lse_mscount--; + if (dirty) + target->lse_msdcount--; } /* @@ -490,15 +506,11 @@ spa_log_summary_decrement_mscount(spa_t *spa, uint64_t txg) void spa_log_summary_decrement_blkcount(spa_t *spa, uint64_t blocks_gone) { - for (log_summary_entry_t *e = list_head(&spa->spa_log_summary); - e != NULL; e = list_head(&spa->spa_log_summary)) { + log_summary_entry_t *e = list_head(&spa->spa_log_summary); + if (e->lse_txgcount > 0) + e->lse_txgcount--; + for (; e != NULL; e = list_head(&spa->spa_log_summary)) { if (e->lse_blkcount > blocks_gone) { - /* - * Assert that we stopped at an entry that is not - * obsolete. - */ - ASSERT(e->lse_mscount != 0); - e->lse_blkcount -= blocks_gone; blocks_gone = 0; break; @@ -560,31 +572,52 @@ spa_log_sm_increment_current_mscount(spa_t *spa) static void summary_add_data(spa_t *spa, uint64_t txg, uint64_t metaslabs_flushed, - uint64_t nblocks) + uint64_t metaslabs_dirty, uint64_t nblocks) { log_summary_entry_t *e = list_tail(&spa->spa_log_summary); - if (e == NULL || summary_entry_is_full(spa, e)) { + if (e == NULL || summary_entry_is_full(spa, e, txg)) { e = kmem_zalloc(sizeof (log_summary_entry_t), KM_SLEEP); - e->lse_start = txg; + e->lse_start = e->lse_end = txg; + e->lse_txgcount = 1; list_insert_tail(&spa->spa_log_summary, e); } ASSERT3U(e->lse_start, <=, txg); + if (e->lse_end < txg) { + e->lse_end = txg; + e->lse_txgcount++; + } e->lse_mscount += metaslabs_flushed; + e->lse_msdcount += metaslabs_dirty; e->lse_blkcount += nblocks; } static void spa_log_summary_add_incoming_blocks(spa_t *spa, uint64_t nblocks) { - summary_add_data(spa, spa_syncing_txg(spa), 0, nblocks); + summary_add_data(spa, spa_syncing_txg(spa), 0, 0, nblocks); } void -spa_log_summary_add_flushed_metaslab(spa_t *spa) +spa_log_summary_add_flushed_metaslab(spa_t *spa, boolean_t dirty) { - summary_add_data(spa, spa_syncing_txg(spa), 1, 0); + summary_add_data(spa, spa_syncing_txg(spa), 1, dirty ? 1 : 0, 0); +} + +void +spa_log_summary_dirty_flushed_metaslab(spa_t *spa, uint64_t txg) +{ + log_summary_entry_t *target = NULL; + for (log_summary_entry_t *e = list_head(&spa->spa_log_summary); + e != NULL; e = list_next(&spa->spa_log_summary, e)) { + if (e->lse_start > txg) + break; + target = e; + } + ASSERT3P(target, !=, NULL); + ASSERT3U(target->lse_mscount, !=, 0); + target->lse_msdcount++; } /* @@ -630,6 +663,11 @@ spa_estimate_metaslabs_to_flush(spa_t *spa) int64_t available_blocks = spa_log_sm_blocklimit(spa) - spa_log_sm_nblocks(spa) - incoming; + int64_t available_txgs = zfs_unflushed_log_txg_max; + for (log_summary_entry_t *e = list_head(&spa->spa_log_summary); + e; e = list_next(&spa->spa_log_summary, e)) + available_txgs -= e->lse_txgcount; + /* * This variable tells us the total number of flushes needed to * keep the log size within the limit when we reach txgs_in_future. @@ -637,9 +675,7 @@ spa_estimate_metaslabs_to_flush(spa_t *spa) uint64_t total_flushes = 0; /* Holds the current maximum of our estimates so far. */ - uint64_t max_flushes_pertxg = - MIN(avl_numnodes(&spa->spa_metaslabs_by_flushed), - zfs_min_metaslabs_to_flush); + uint64_t max_flushes_pertxg = zfs_min_metaslabs_to_flush; /* * For our estimations we only look as far in the future @@ -653,11 +689,14 @@ spa_estimate_metaslabs_to_flush(spa_t *spa) * then keep skipping TXGs accumulating more blocks * based on the incoming rate until we exceed it. */ - if (available_blocks >= 0) { - uint64_t skip_txgs = (available_blocks / incoming) + 1; + if (available_blocks >= 0 && available_txgs >= 0) { + uint64_t skip_txgs = MIN(available_txgs + 1, + (available_blocks / incoming) + 1); available_blocks -= (skip_txgs * incoming); + available_txgs -= skip_txgs; txgs_in_future += skip_txgs; ASSERT3S(available_blocks, >=, -incoming); + ASSERT3S(available_txgs, >=, -1); } /* @@ -666,9 +705,10 @@ spa_estimate_metaslabs_to_flush(spa_t *spa) * based on the current entry in the summary, updating * our available_blocks. */ - ASSERT3S(available_blocks, <, 0); + ASSERT(available_blocks < 0 || available_txgs < 0); available_blocks += e->lse_blkcount; - total_flushes += e->lse_mscount; + available_txgs += e->lse_txgcount; + total_flushes += e->lse_msdcount; /* * Keep the running maximum of the total_flushes that @@ -680,8 +720,6 @@ spa_estimate_metaslabs_to_flush(spa_t *spa) */ max_flushes_pertxg = MAX(max_flushes_pertxg, DIV_ROUND_UP(total_flushes, txgs_in_future)); - ASSERT3U(avl_numnodes(&spa->spa_metaslabs_by_flushed), >=, - max_flushes_pertxg); } return (max_flushes_pertxg); } @@ -771,14 +809,11 @@ spa_flush_metaslabs(spa_t *spa, dmu_tx_t *tx) uint64_t want_to_flush; if (spa_flush_all_logs_requested(spa)) { ASSERT3S(spa_state(spa), ==, POOL_STATE_EXPORTED); - want_to_flush = avl_numnodes(&spa->spa_metaslabs_by_flushed); + want_to_flush = UINT64_MAX; } else { want_to_flush = spa_estimate_metaslabs_to_flush(spa); } - ASSERT3U(avl_numnodes(&spa->spa_metaslabs_by_flushed), >=, - want_to_flush); - /* Used purely for verification purposes */ uint64_t visited = 0; @@ -809,31 +844,22 @@ spa_flush_metaslabs(spa_t *spa, dmu_tx_t *tx) if (want_to_flush == 0 && !spa_log_exceeds_memlimit(spa)) break; - mutex_enter(&curr->ms_sync_lock); - mutex_enter(&curr->ms_lock); - boolean_t flushed = metaslab_flush(curr, tx); - mutex_exit(&curr->ms_lock); - mutex_exit(&curr->ms_sync_lock); - - /* - * If we failed to flush a metaslab (because it was loading), - * then we are done with the block heuristic as it's not - * possible to destroy any log space maps once you've skipped - * a metaslab. In that case we just set our counter to 0 but - * we continue looping in case there is still memory pressure - * due to unflushed changes. Note that, flushing a metaslab - * that is not the oldest flushed in the pool, will never - * destroy any log space maps [see spa_cleanup_old_sm_logs()]. - */ - if (!flushed) { - want_to_flush = 0; - } else if (want_to_flush > 0) { - want_to_flush--; - } + if (metaslab_unflushed_dirty(curr)) { + mutex_enter(&curr->ms_sync_lock); + mutex_enter(&curr->ms_lock); + metaslab_flush(curr, tx); + mutex_exit(&curr->ms_lock); + mutex_exit(&curr->ms_sync_lock); + if (want_to_flush > 0) + want_to_flush--; + } else + metaslab_unflushed_bump(curr, tx, B_FALSE); visited++; } ASSERT3U(avl_numnodes(&spa->spa_metaslabs_by_flushed), >=, visited); + + spa_log_sm_set_blocklimit(spa); } /* @@ -904,6 +930,7 @@ spa_cleanup_old_sm_logs(spa_t *spa, dmu_tx_t *tx) avl_remove(&spa->spa_sm_logs_by_txg, sls); space_map_free_obj(mos, sls->sls_sm_obj, tx); VERIFY0(zap_remove_int(mos, spacemap_zap, sls->sls_txg, tx)); + spa_log_summary_decrement_blkcount(spa, sls->sls_nblocks); spa->spa_unflushed_stats.sus_nblocks -= sls->sls_nblocks; kmem_free(sls, sizeof (spa_log_sm_t)); } @@ -963,12 +990,7 @@ spa_generate_syncing_log_sm(spa_t *spa, dmu_tx_t *tx) VERIFY0(space_map_open(&spa->spa_syncing_log_sm, mos, sm_obj, 0, UINT64_MAX, SPA_MINBLOCKSHIFT)); - /* - * If the log space map feature was just enabled, the blocklimit - * has not yet been set. - */ - if (spa_log_sm_blocklimit(spa) == 0) - spa_log_sm_set_blocklimit(spa); + spa_log_sm_set_blocklimit(spa); } /* @@ -1094,12 +1116,18 @@ spa_ld_log_sm_cb(space_map_entry_t *sme, void *arg) panic("invalid maptype_t"); break; } + if (!metaslab_unflushed_dirty(ms)) { + metaslab_set_unflushed_dirty(ms, B_TRUE); + spa_log_summary_dirty_flushed_metaslab(spa, + metaslab_unflushed_txg(ms)); + } return (0); } static int spa_ld_log_sm_data(spa_t *spa) { + spa_log_sm_t *sls, *psls; int error = 0; /* @@ -1113,41 +1141,71 @@ spa_ld_log_sm_data(spa_t *spa) ASSERT0(spa->spa_unflushed_stats.sus_memused); hrtime_t read_logs_starttime = gethrtime(); - /* this is a no-op when we don't have space map logs */ - for (spa_log_sm_t *sls = avl_first(&spa->spa_sm_logs_by_txg); - sls; sls = AVL_NEXT(&spa->spa_sm_logs_by_txg, sls)) { - space_map_t *sm = NULL; - error = space_map_open(&sm, spa_meta_objset(spa), - sls->sls_sm_obj, 0, UINT64_MAX, SPA_MINBLOCKSHIFT); - if (error != 0) { - spa_load_failed(spa, "spa_ld_log_sm_data(): failed at " - "space_map_open(obj=%llu) [error %d]", - (u_longlong_t)sls->sls_sm_obj, error); - goto out; + + /* Prefetch log spacemaps dnodes. */ + for (sls = avl_first(&spa->spa_sm_logs_by_txg); sls; + sls = AVL_NEXT(&spa->spa_sm_logs_by_txg, sls)) { + dmu_prefetch(spa_meta_objset(spa), sls->sls_sm_obj, + 0, 0, 0, ZIO_PRIORITY_SYNC_READ); + } + + uint_t pn = 0; + uint64_t ps = 0; + psls = sls = avl_first(&spa->spa_sm_logs_by_txg); + while (sls != NULL) { + /* Prefetch log spacemaps up to 16 TXGs or MBs ahead. */ + if (psls != NULL && pn < 16 && + (pn < 2 || ps < 2 * dmu_prefetch_max)) { + error = space_map_open(&psls->sls_sm, + spa_meta_objset(spa), psls->sls_sm_obj, 0, + UINT64_MAX, SPA_MINBLOCKSHIFT); + if (error != 0) { + spa_load_failed(spa, "spa_ld_log_sm_data(): " + "failed at space_map_open(obj=%llu) " + "[error %d]", + (u_longlong_t)sls->sls_sm_obj, error); + goto out; + } + dmu_prefetch(spa_meta_objset(spa), psls->sls_sm_obj, + 0, 0, space_map_length(psls->sls_sm), + ZIO_PRIORITY_ASYNC_READ); + pn++; + ps += space_map_length(psls->sls_sm); + psls = AVL_NEXT(&spa->spa_sm_logs_by_txg, psls); + continue; } + /* Load TXG log spacemap into ms_unflushed_allocs/frees. */ + cond_resched(); + ASSERT0(sls->sls_nblocks); + sls->sls_nblocks = space_map_nblocks(sls->sls_sm); + spa->spa_unflushed_stats.sus_nblocks += sls->sls_nblocks; + summary_add_data(spa, sls->sls_txg, + sls->sls_mscount, 0, sls->sls_nblocks); + struct spa_ld_log_sm_arg vla = { .slls_spa = spa, .slls_txg = sls->sls_txg }; - error = space_map_iterate(sm, space_map_length(sm), - spa_ld_log_sm_cb, &vla); + error = space_map_iterate(sls->sls_sm, + space_map_length(sls->sls_sm), spa_ld_log_sm_cb, &vla); if (error != 0) { - space_map_close(sm); spa_load_failed(spa, "spa_ld_log_sm_data(): failed " "at space_map_iterate(obj=%llu) [error %d]", (u_longlong_t)sls->sls_sm_obj, error); goto out; } - ASSERT0(sls->sls_nblocks); - sls->sls_nblocks = space_map_nblocks(sm); - spa->spa_unflushed_stats.sus_nblocks += sls->sls_nblocks; - summary_add_data(spa, sls->sls_txg, - sls->sls_mscount, sls->sls_nblocks); + pn--; + ps -= space_map_length(sls->sls_sm); + space_map_close(sls->sls_sm); + sls->sls_sm = NULL; + sls = AVL_NEXT(&spa->spa_sm_logs_by_txg, sls); - space_map_close(sm); + /* Update log block limits considering just loaded. */ + spa_log_sm_set_blocklimit(spa); } + hrtime_t read_logs_endtime = gethrtime(); spa_load_note(spa, "read %llu log space maps (%llu total blocks - blksz = %llu bytes) " @@ -1157,6 +1215,18 @@ spa_ld_log_sm_data(spa_t *spa) (longlong_t)((read_logs_endtime - read_logs_starttime) / 1000000)); out: + if (error != 0) { + for (spa_log_sm_t *sls = avl_first(&spa->spa_sm_logs_by_txg); + sls; sls = AVL_NEXT(&spa->spa_sm_logs_by_txg, sls)) { + if (sls->sls_sm) { + space_map_close(sls->sls_sm); + sls->sls_sm = NULL; + } + } + } else { + ASSERT0(pn); + ASSERT0(ps); + } /* * Now that the metaslabs contain their unflushed changes: * [1] recalculate their actual allocated space @@ -1237,6 +1307,9 @@ spa_ld_unflushed_txgs(vdev_t *vd) } ms->ms_unflushed_txg = entry.msp_unflushed_txg; + ms->ms_unflushed_dirty = B_FALSE; + ASSERT(range_tree_is_empty(ms->ms_unflushed_allocs)); + ASSERT(range_tree_is_empty(ms->ms_unflushed_frees)); if (ms->ms_unflushed_txg != 0) { mutex_enter(&spa->spa_flushed_ms_lock); avl_add(&spa->spa_metaslabs_by_flushed, ms); @@ -1300,6 +1373,10 @@ ZFS_MODULE_PARAM(zfs, zfs_, unflushed_log_block_min, ULONG, ZMOD_RW, "Lower-bound limit for the maximum amount of blocks allowed in " "log spacemap (see zfs_unflushed_log_block_max)"); +ZFS_MODULE_PARAM(zfs, zfs_, unflushed_log_txg_max, ULONG, ZMOD_RW, + "Hard limit (upper-bound) in the size of the space map log " + "in terms of dirty TXGs."); + ZFS_MODULE_PARAM(zfs, zfs_, unflushed_log_block_pct, ULONG, ZMOD_RW, "Tunable used to determine the number of blocks that can be used for " "the spacemap log, expressed as a percentage of the total number of " diff --git a/module/zfs/spa_misc.c b/module/zfs/spa_misc.c index a04766e7e33b..decf4ddae6af 100644 --- a/module/zfs/spa_misc.c +++ b/module/zfs/spa_misc.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -30,6 +30,7 @@ */ #include +#include #include #include #include @@ -461,7 +462,7 @@ spa_config_lock_destroy(spa_t *spa) } int -spa_config_tryenter(spa_t *spa, int locks, void *tag, krw_t rw) +spa_config_tryenter(spa_t *spa, int locks, const void *tag, krw_t rw) { for (int i = 0; i < SCL_LOCKS; i++) { spa_config_lock_t *scl = &spa->spa_config_lock[i]; @@ -876,7 +877,7 @@ spa_next(spa_t *prev) * have the namespace lock held. */ void -spa_open_ref(spa_t *spa, void *tag) +spa_open_ref(spa_t *spa, const void *tag) { ASSERT(zfs_refcount_count(&spa->spa_refcount) >= spa->spa_minref || MUTEX_HELD(&spa_namespace_lock)); @@ -888,7 +889,7 @@ spa_open_ref(spa_t *spa, void *tag) * have the namespace lock held. */ void -spa_close(spa_t *spa, void *tag) +spa_close(spa_t *spa, const void *tag) { ASSERT(zfs_refcount_count(&spa->spa_refcount) > spa->spa_minref || MUTEX_HELD(&spa_namespace_lock)); @@ -904,7 +905,7 @@ spa_close(spa_t *spa, void *tag) * so the asserts in spa_close() do not apply. */ void -spa_async_close(spa_t *spa, void *tag) +spa_async_close(spa_t *spa, const void *tag) { (void) zfs_refcount_remove(&spa->spa_refcount, tag); } @@ -1214,7 +1215,8 @@ spa_vdev_config_enter(spa_t *spa) * of multiple transactions without releasing the spa_namespace_lock. */ void -spa_vdev_config_exit(spa_t *spa, vdev_t *vd, uint64_t txg, int error, char *tag) +spa_vdev_config_exit(spa_t *spa, vdev_t *vd, uint64_t txg, int error, + const char *tag) { ASSERT(MUTEX_HELD(&spa_namespace_lock)); @@ -1477,8 +1479,7 @@ spa_strdup(const char *s) len = strlen(s); new = kmem_alloc(len + 1, KM_SLEEP); - bcopy(s, new, len); - new[len] = '\0'; + memcpy(new, s, len + 1); return (new); } @@ -1513,8 +1514,8 @@ void snprintf_blkptr(char *buf, size_t buflen, const blkptr_t *bp) { char type[256]; - char *checksum = NULL; - char *compress = NULL; + const char *checksum = NULL; + const char *compress = NULL; if (bp != NULL) { if (BP_GET_TYPE(bp) & DMU_OT_NEWTYPE) { @@ -2418,6 +2419,7 @@ spa_init(spa_mode_t mode) vdev_raidz_math_init(); vdev_file_init(); zfs_prop_init(); + chksum_init(); zpool_prop_init(); zpool_feature_init(); spa_config_load(); @@ -2439,6 +2441,7 @@ spa_fini(void) vdev_cache_stat_fini(); vdev_mirror_stat_fini(); vdev_raidz_math_fini(); + chksum_fini(); zil_fini(); dmu_fini(); zio_fini(); @@ -2566,7 +2569,7 @@ spa_scan_get_stats(spa_t *spa, pool_scan_stat_t *ps) if (scn == NULL || scn->scn_phys.scn_func == POOL_SCAN_NONE) return (SET_ERROR(ENOENT)); - bzero(ps, sizeof (pool_scan_stat_t)); + memset(ps, 0, sizeof (pool_scan_stat_t)); /* data stored on disk */ ps->pss_func = scn->scn_phys.scn_func; diff --git a/module/zfs/spa_stats.c b/module/zfs/spa_stats.c index 2a75b37f020e..59844d5d6491 100644 --- a/module/zfs/spa_stats.c +++ b/module/zfs/spa_stats.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -819,6 +819,41 @@ spa_state_init(spa_t *spa) kmem_strfree(name); } +static int +spa_guid_data(char *buf, size_t size, void *data) +{ + spa_t *spa = (spa_t *)data; + (void) snprintf(buf, size, "%llu\n", (u_longlong_t)spa_guid(spa)); + return (0); +} + +static void +spa_guid_init(spa_t *spa) +{ + spa_history_kstat_t *shk = &spa->spa_stats.guid; + char *name; + kstat_t *ksp; + + mutex_init(&shk->lock, NULL, MUTEX_DEFAULT, NULL); + + name = kmem_asprintf("zfs/%s", spa_name(spa)); + + ksp = kstat_create(name, 0, "guid", "misc", + KSTAT_TYPE_RAW, 0, KSTAT_FLAG_VIRTUAL); + + shk->kstat = ksp; + if (ksp) { + ksp->ks_lock = &shk->lock; + ksp->ks_data = NULL; + ksp->ks_private = spa; + ksp->ks_flags |= KSTAT_FLAG_NO_HEADERS; + kstat_set_raw_ops(ksp, NULL, spa_guid_data, spa_state_addr); + kstat_install(ksp); + } + + kmem_strfree(name); +} + static void spa_health_destroy(spa_t *spa) { @@ -830,6 +865,17 @@ spa_health_destroy(spa_t *spa) mutex_destroy(&shk->lock); } +static void +spa_guid_destroy(spa_t *spa) +{ + spa_history_kstat_t *shk = &spa->spa_stats.guid; + kstat_t *ksp = shk->kstat; + if (ksp) + kstat_delete(ksp); + + mutex_destroy(&shk->lock); +} + static const spa_iostats_t spa_iostats_template = { { "trim_extents_written", KSTAT_DATA_UINT64 }, { "trim_bytes_written", KSTAT_DATA_UINT64 }, @@ -950,6 +996,7 @@ spa_stats_init(spa_t *spa) spa_tx_assign_init(spa); spa_mmp_history_init(spa); spa_state_init(spa); + spa_guid_init(spa); spa_iostats_init(spa); } @@ -962,6 +1009,7 @@ spa_stats_destroy(spa_t *spa) spa_txg_history_destroy(spa); spa_read_history_destroy(spa); spa_mmp_history_destroy(spa); + spa_guid_destroy(spa); } ZFS_MODULE_PARAM(zfs, zfs_, read_history, INT, ZMOD_RW, diff --git a/module/zfs/space_map.c b/module/zfs/space_map.c index 11d4798925e4..a336ff41eadb 100644 --- a/module/zfs/space_map.c +++ b/module/zfs/space_map.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -455,7 +455,8 @@ space_map_histogram_clear(space_map_t *sm) if (sm->sm_dbuf->db_size != sizeof (space_map_phys_t)) return; - bzero(sm->sm_phys->smp_histogram, sizeof (sm->sm_phys->smp_histogram)); + memset(sm->sm_phys->smp_histogram, 0, + sizeof (sm->sm_phys->smp_histogram)); } boolean_t @@ -548,7 +549,7 @@ space_map_write_intro_debug(space_map_t *sm, maptype_t maptype, dmu_tx_t *tx) static void space_map_write_seg(space_map_t *sm, uint64_t rstart, uint64_t rend, maptype_t maptype, uint64_t vdev_id, uint8_t words, dmu_buf_t **dbp, - void *tag, dmu_tx_t *tx) + const void *tag, dmu_tx_t *tx) { ASSERT3U(words, !=, 0); ASSERT3U(words, <=, 2); @@ -896,7 +897,7 @@ space_map_truncate(space_map_t *sm, int blocksize, dmu_tx_t *tx) * will be reset. Do the same in the common case so that * bugs related to the uncommon case do not go unnoticed. */ - bzero(sm->sm_phys->smp_histogram, + memset(sm->sm_phys->smp_histogram, 0, sizeof (sm->sm_phys->smp_histogram)); } diff --git a/module/zfs/space_reftree.c b/module/zfs/space_reftree.c index 080fc6646512..ee11e162dd5b 100644 --- a/module/zfs/space_reftree.c +++ b/module/zfs/space_reftree.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/txg.c b/module/zfs/txg.c index 9655efdc4813..6e2b8b01053f 100644 --- a/module/zfs/txg.c +++ b/module/zfs/txg.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -108,8 +108,8 @@ * now transition to the syncing state. */ -static void txg_sync_thread(void *arg); -static void txg_quiesce_thread(void *arg); +static __attribute__((noreturn)) void txg_sync_thread(void *arg); +static __attribute__((noreturn)) void txg_quiesce_thread(void *arg); int zfs_txg_timeout = 5; /* max seconds worth of delta per txg */ @@ -121,7 +121,7 @@ txg_init(dsl_pool_t *dp, uint64_t txg) { tx_state_t *tx = &dp->dp_tx; int c; - bzero(tx, sizeof (tx_state_t)); + memset(tx, 0, sizeof (tx_state_t)); tx->tx_cpu = vmem_zalloc(max_ncpus * sizeof (tx_cpu_t), KM_SLEEP); @@ -186,7 +186,7 @@ txg_fini(dsl_pool_t *dp) vmem_free(tx->tx_cpu, max_ncpus * sizeof (tx_cpu_t)); - bzero(tx, sizeof (tx_state_t)); + memset(tx, 0, sizeof (tx_state_t)); } /* @@ -514,7 +514,7 @@ txg_has_quiesced_to_sync(dsl_pool_t *dp) return (tx->tx_quiesced_txg != 0); } -static void +static __attribute__((noreturn)) void txg_sync_thread(void *arg) { dsl_pool_t *dp = arg; @@ -605,7 +605,7 @@ txg_sync_thread(void *arg) } } -static void +static __attribute__((noreturn)) void txg_quiesce_thread(void *arg) { dsl_pool_t *dp = arg; diff --git a/module/zfs/uberblock.c b/module/zfs/uberblock.c index b8857d74d810..1921be107660 100644 --- a/module/zfs/uberblock.c +++ b/module/zfs/uberblock.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/unique.c b/module/zfs/unique.c index 0e076797a002..799e4095db33 100644 --- a/module/zfs/unique.c +++ b/module/zfs/unique.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev.c b/module/zfs/vdev.c index a844ee2a73c5..ea0245610fb7 100644 --- a/module/zfs/vdev.c +++ b/module/zfs/vdev.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -475,7 +475,7 @@ vdev_add_child(vdev_t *pvd, vdev_t *cvd) newchild = kmem_alloc(newsize, KM_SLEEP); if (pvd->vdev_child != NULL) { - bcopy(pvd->vdev_child, newchild, oldsize); + memcpy(newchild, pvd->vdev_child, oldsize); kmem_free(pvd->vdev_child, oldsize); } @@ -1426,7 +1426,7 @@ vdev_metaslab_init(vdev_t *vd, uint64_t txg) mspp = vmem_zalloc(newc * sizeof (*mspp), KM_SLEEP); if (expanding) { - bcopy(vd->vdev_ms, mspp, oldc * sizeof (*mspp)); + memcpy(mspp, vd->vdev_ms, oldc * sizeof (*mspp)); vmem_free(vd->vdev_ms, oldc * sizeof (*mspp)); } @@ -1523,13 +1523,6 @@ vdev_metaslab_init(vdev_t *vd, uint64_t txg) if (txg == 0) spa_config_exit(spa, SCL_ALLOC, FTAG); - /* - * Regardless whether this vdev was just added or it is being - * expanded, the metaslab count has changed. Recalculate the - * block limit. - */ - spa_log_sm_set_blocklimit(spa); - return (0); } @@ -4418,12 +4411,13 @@ vdev_get_stats_ex(vdev_t *vd, vdev_stat_t *vs, vdev_stat_ex_t *vsx) vdev_t *tvd = vd->vdev_top; mutex_enter(&vd->vdev_stat_lock); if (vs) { - bcopy(&vd->vdev_stat, vs, sizeof (*vs)); + memcpy(vs, &vd->vdev_stat, sizeof (*vs)); vs->vs_timestamp = gethrtime() - vs->vs_timestamp; vs->vs_state = vd->vdev_state; vs->vs_rsize = vdev_get_min_asize(vd); if (vd->vdev_ops->vdev_op_leaf) { + vs->vs_pspace = vd->vdev_psize; vs->vs_rsize += VDEV_LABEL_START_SIZE + VDEV_LABEL_END_SIZE; /* @@ -5239,7 +5233,7 @@ vdev_split(vdev_t *vd) } void -vdev_deadman(vdev_t *vd, char *tag) +vdev_deadman(vdev_t *vd, const char *tag) { for (int c = 0; c < vd->vdev_children; c++) { vdev_t *cvd = vd->vdev_child[c]; @@ -5502,7 +5496,7 @@ vdev_props_set_sync(void *arg, dmu_tx_t *tx) } switch (prop = vdev_name_to_prop(propname)) { - case VDEV_PROP_USER: + case VDEV_PROP_USERPROP: if (vdev_prop_user(propname)) { strval = fnvpair_value_string(elem); if (strlen(strval) == 0) { @@ -5586,7 +5580,7 @@ vdev_prop_set(vdev_t *vd, nvlist_t *innvl, nvlist_t *outnvl) uint64_t intval = 0; char *strval = NULL; - if (prop == VDEV_PROP_USER && !vdev_prop_user(propname)) { + if (prop == VDEV_PROP_USERPROP && !vdev_prop_user(propname)) { error = EINVAL; goto end; } @@ -5796,7 +5790,7 @@ vdev_prop_get(vdev_t *vd, nvlist_t *innvl, nvlist_t *outnvl) KM_SLEEP); for (uint64_t i = 0; i < vd->vdev_children; i++) { - char *vname; + const char *vname; vname = vdev_name(vd->vdev_child[i], namebuf, sizeof (namebuf)); @@ -5943,7 +5937,7 @@ vdev_prop_get(vdev_t *vd, nvlist_t *innvl, nvlist_t *outnvl) case VDEV_PROP_COMMENT: /* Exists in the ZAP below */ /* FALLTHRU */ - case VDEV_PROP_USER: + case VDEV_PROP_USERPROP: /* User Properites */ src = ZPROP_SRC_LOCAL; diff --git a/module/zfs/vdev_cache.c b/module/zfs/vdev_cache.c index a3433311b381..d1b96b5e2eb4 100644 --- a/module/zfs/vdev_cache.c +++ b/module/zfs/vdev_cache.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_draid.c b/module/zfs/vdev_draid.c index 6c765d984585..24034d9d9313 100644 --- a/module/zfs/vdev_draid.c +++ b/module/zfs/vdev_draid.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -1725,7 +1725,7 @@ vdev_draid_spare_create(nvlist_t *nvroot, vdev_t *vd, uint64_t *ndraidp, uint64_t nparity = vdc->vdc_nparity; for (uint64_t spare_id = 0; spare_id < nspares; spare_id++) { - bzero(path, sizeof (path)); + memset(path, 0, sizeof (path)); (void) snprintf(path, sizeof (path) - 1, "%s%llu-%llu-%llu", VDEV_TYPE_DRAID, (u_longlong_t)nparity, diff --git a/module/zfs/vdev_indirect.c b/module/zfs/vdev_indirect.c index aeba1e99e6e5..9189d3f31241 100644 --- a/module/zfs/vdev_indirect.c +++ b/module/zfs/vdev_indirect.c @@ -48,8 +48,8 @@ * "vdev_remap" operation that executes a callback on each contiguous * segment of the new location. This function is used in multiple ways: * - * - i/os to this vdev use the callback to determine where the - * data is now located, and issue child i/os for each segment's new + * - I/Os to this vdev use the callback to determine where the + * data is now located, and issue child I/Os for each segment's new * location. * * - frees and claims to this vdev use the callback to free or claim @@ -1021,7 +1021,7 @@ vdev_indirect_mapping_duplicate_adjacent_entries(vdev_t *vd, uint64_t offset, size_t copy_length = entries * sizeof (*first_mapping); duplicate_mappings = kmem_alloc(copy_length, KM_SLEEP); - bcopy(first_mapping, duplicate_mappings, copy_length); + memcpy(duplicate_mappings, first_mapping, copy_length); *copied_entries = entries; return (duplicate_mappings); diff --git a/module/zfs/vdev_indirect_births.c b/module/zfs/vdev_indirect_births.c index e8f925628d04..65a57e73604f 100644 --- a/module/zfs/vdev_indirect_births.c +++ b/module/zfs/vdev_indirect_births.c @@ -152,7 +152,7 @@ vdev_indirect_births_add_entry(vdev_indirect_births_t *vib, new_entries = vmem_alloc(new_size, KM_SLEEP); if (old_size > 0) { - bcopy(vib->vib_entries, new_entries, old_size); + memcpy(new_entries, vib->vib_entries, old_size); vmem_free(vib->vib_entries, old_size); } new_entries[vib->vib_phys->vib_count - 1] = vibe; diff --git a/module/zfs/vdev_indirect_mapping.c b/module/zfs/vdev_indirect_mapping.c index 4ade56e062f7..e92495f2dd34 100644 --- a/module/zfs/vdev_indirect_mapping.c +++ b/module/zfs/vdev_indirect_mapping.c @@ -482,7 +482,7 @@ vdev_indirect_mapping_add_entries(vdev_indirect_mapping_t *vim, entries_written * sizeof (vdev_indirect_mapping_entry_phys_t)); vim->vim_entries = vmem_alloc(new_size, KM_SLEEP); if (old_size > 0) { - bcopy(old_entries, vim->vim_entries, old_size); + memcpy(vim->vim_entries, old_entries, old_size); vmem_free(old_entries, old_size); } VERIFY0(dmu_read(vim->vim_objset, vim->vim_object, old_size, @@ -584,7 +584,7 @@ vdev_indirect_mapping_load_obsolete_counts(vdev_indirect_mapping_t *vim) 0, counts_size, counts, DMU_READ_PREFETCH)); } else { - bzero(counts, counts_size); + memset(counts, 0, counts_size); } return (counts); } diff --git a/module/zfs/vdev_initialize.c b/module/zfs/vdev_initialize.c index ce1385d5aab1..965fb7ef0593 100644 --- a/module/zfs/vdev_initialize.c +++ b/module/zfs/vdev_initialize.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -488,7 +488,7 @@ vdev_initialize_range_add(void *arg, uint64_t start, uint64_t size) vdev_xlate_walk(vd, &logical_rs, vdev_initialize_xlate_range_add, arg); } -static void +static __attribute__((noreturn)) void vdev_initialize_thread(void *arg) { vdev_t *vd = arg; diff --git a/module/zfs/vdev_label.c b/module/zfs/vdev_label.c index 29391af820ed..6e47c8cb69e2 100644 --- a/module/zfs/vdev_label.c +++ b/module/zfs/vdev_label.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -1565,7 +1565,7 @@ vdev_uberblock_load(vdev_t *rvd, uberblock_t *ub, nvlist_t **config) ASSERT(ub); ASSERT(config); - bzero(ub, sizeof (uberblock_t)); + memset(ub, 0, sizeof (uberblock_t)); *config = NULL; cb.ubl_ubbest = ub; diff --git a/module/zfs/vdev_mirror.c b/module/zfs/vdev_mirror.c index 25abe99e0749..3879de680453 100644 --- a/module/zfs/vdev_mirror.c +++ b/module/zfs/vdev_mirror.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -35,6 +35,7 @@ #include #include #include +#include #include #include @@ -102,6 +103,7 @@ vdev_mirror_stat_fini(void) */ typedef struct mirror_child { vdev_t *mc_vd; + abd_t *mc_abd; uint64_t mc_offset; int mc_error; int mc_load; @@ -439,32 +441,6 @@ vdev_mirror_child_done(zio_t *zio) mc->mc_skipped = 0; } -static void -vdev_mirror_scrub_done(zio_t *zio) -{ - mirror_child_t *mc = zio->io_private; - - if (zio->io_error == 0) { - zio_t *pio; - zio_link_t *zl = NULL; - - mutex_enter(&zio->io_lock); - while ((pio = zio_walk_parents(zio, &zl)) != NULL) { - mutex_enter(&pio->io_lock); - ASSERT3U(zio->io_size, >=, pio->io_size); - abd_copy(pio->io_abd, zio->io_abd, pio->io_size); - mutex_exit(&pio->io_lock); - } - mutex_exit(&zio->io_lock); - } - - abd_free(zio->io_abd); - - mc->mc_error = zio->io_error; - mc->mc_tried = 1; - mc->mc_skipped = 0; -} - /* * Check the other, lower-index DVAs to see if they're on the same * vdev as the child we picked. If they are, use them since they @@ -637,16 +613,15 @@ vdev_mirror_io_start(zio_t *zio) } if (zio->io_type == ZIO_TYPE_READ) { - if (zio->io_bp != NULL && - (zio->io_flags & ZIO_FLAG_SCRUB) && !mm->mm_resilvering) { + if ((zio->io_flags & ZIO_FLAG_SCRUB) && !mm->mm_resilvering) { /* - * For scrubbing reads (if we can verify the - * checksum here, as indicated by io_bp being - * non-NULL) we need to allocate a read buffer for - * each child and issue reads to all children. If - * any child succeeds, it will copy its data into - * zio->io_data in vdev_mirror_scrub_done. + * For scrubbing reads we need to issue reads to all + * children. One child can reuse parent buffer, but + * for others we have to allocate separate ones to + * verify checksums if io_bp is non-NULL, or compare + * them in vdev_mirror_io_done() otherwise. */ + boolean_t first = B_TRUE; for (c = 0; c < mm->mm_children; c++) { mc = &mm->mm_child[c]; @@ -658,12 +633,15 @@ vdev_mirror_io_start(zio_t *zio) continue; } - zio_nowait(zio_vdev_child_io(zio, zio->io_bp, - mc->mc_vd, mc->mc_offset, + mc->mc_abd = first ? zio->io_abd : abd_alloc_sametype(zio->io_abd, - zio->io_size), zio->io_size, - zio->io_type, zio->io_priority, 0, - vdev_mirror_scrub_done, mc)); + zio->io_size); + zio_nowait(zio_vdev_child_io(zio, zio->io_bp, + mc->mc_vd, mc->mc_offset, mc->mc_abd, + zio->io_size, zio->io_type, + zio->io_priority, 0, + vdev_mirror_child_done, mc)); + first = B_FALSE; } zio_execute(zio); return; @@ -731,6 +709,7 @@ vdev_mirror_io_done(zio_t *zio) int c; int good_copies = 0; int unexpected_errors = 0; + int last_good_copy = -1; if (mm == NULL) return; @@ -742,6 +721,7 @@ vdev_mirror_io_done(zio_t *zio) if (!mc->mc_skipped) unexpected_errors++; } else if (mc->mc_tried) { + last_good_copy = c; good_copies++; } } @@ -755,7 +735,6 @@ vdev_mirror_io_done(zio_t *zio) * no non-degraded top-level vdevs left, and not update DTLs * if we intend to reallocate. */ - /* XXPOLICY */ if (good_copies != mm->mm_children) { /* * Always require at least one good copy. @@ -782,7 +761,6 @@ vdev_mirror_io_done(zio_t *zio) /* * If we don't have a good copy yet, keep trying other children. */ - /* XXPOLICY */ if (good_copies == 0 && (c = vdev_mirror_child_select(zio)) != -1) { ASSERT(c >= 0 && c < mm->mm_children); mc = &mm->mm_child[c]; @@ -794,7 +772,80 @@ vdev_mirror_io_done(zio_t *zio) return; } - /* XXPOLICY */ + if (zio->io_flags & ZIO_FLAG_SCRUB && !mm->mm_resilvering) { + abd_t *best_abd = NULL; + if (last_good_copy >= 0) + best_abd = mm->mm_child[last_good_copy].mc_abd; + + /* + * If we're scrubbing but don't have a BP available (because + * this vdev is under a raidz or draid vdev) then the best we + * can do is compare all of the copies read. If they're not + * identical then return a checksum error and the most likely + * correct data. The raidz code will issue a repair I/O if + * possible. + */ + if (zio->io_bp == NULL) { + ASSERT(zio->io_vd->vdev_ops == &vdev_replacing_ops || + zio->io_vd->vdev_ops == &vdev_spare_ops); + + abd_t *pref_abd = NULL; + for (c = 0; c < last_good_copy; c++) { + mc = &mm->mm_child[c]; + if (mc->mc_error || !mc->mc_tried) + continue; + + if (abd_cmp(mc->mc_abd, best_abd) != 0) + zio->io_error = SET_ERROR(ECKSUM); + + /* + * The distributed spare is always prefered + * by vdev_mirror_child_select() so it's + * considered to be the best candidate. + */ + if (pref_abd == NULL && + mc->mc_vd->vdev_ops == + &vdev_draid_spare_ops) + pref_abd = mc->mc_abd; + + /* + * In the absence of a preferred copy, use + * the parent pointer to avoid a memory copy. + */ + if (mc->mc_abd == zio->io_abd) + best_abd = mc->mc_abd; + } + if (pref_abd) + best_abd = pref_abd; + } else { + + /* + * If we have a BP available, then checksums are + * already verified and we just need a buffer + * with valid data, preferring parent one to + * avoid a memory copy. + */ + for (c = 0; c < last_good_copy; c++) { + mc = &mm->mm_child[c]; + if (mc->mc_error || !mc->mc_tried) + continue; + if (mc->mc_abd == zio->io_abd) { + best_abd = mc->mc_abd; + break; + } + } + } + + if (best_abd && best_abd != zio->io_abd) + abd_copy(zio->io_abd, best_abd, zio->io_size); + for (c = 0; c < mm->mm_children; c++) { + mc = &mm->mm_child[c]; + if (mc->mc_abd != zio->io_abd) + abd_free(mc->mc_abd); + mc->mc_abd = NULL; + } + } + if (good_copies == 0) { zio->io_error = vdev_mirror_worst_error(mm); ASSERT(zio->io_error != 0); diff --git a/module/zfs/vdev_missing.c b/module/zfs/vdev_missing.c index 505df23c1fb2..d3580882c3e0 100644 --- a/module/zfs/vdev_missing.c +++ b/module/zfs/vdev_missing.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_queue.c b/module/zfs/vdev_queue.c index 0cad5839bb34..9a805f2c3181 100644 --- a/module/zfs/vdev_queue.c +++ b/module/zfs/vdev_queue.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_raidz.c b/module/zfs/vdev_raidz.c index 1d691c81b5d5..b4daf642ed20 100644 --- a/module/zfs/vdev_raidz.c +++ b/module/zfs/vdev_raidz.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -1779,11 +1779,9 @@ vdev_raidz_checksum_error(zio_t *zio, raidz_col_t *rc, abd_t *bad_data) static int raidz_checksum_verify(zio_t *zio) { - zio_bad_cksum_t zbc; + zio_bad_cksum_t zbc = {{{0}}}; raidz_map_t *rm = zio->io_vsd; - bzero(&zbc, sizeof (zio_bad_cksum_t)); - int ret = zio_checksum_error(zio, &zbc); if (ret != 0 && zbc.zbc_injected != 0) rm->rm_ecksuminjected = 1; @@ -1817,8 +1815,9 @@ raidz_parity_verify(zio_t *zio, raidz_row_t *rr) if (!rc->rc_tried || rc->rc_error != 0) continue; - orig[c] = abd_alloc_sametype(rc->rc_abd, rc->rc_size); - abd_copy(orig[c], rc->rc_abd, rc->rc_size); + orig[c] = rc->rc_abd; + ASSERT3U(abd_get_size(rc->rc_abd), ==, rc->rc_size); + rc->rc_abd = abd_alloc_linear(rc->rc_size, B_FALSE); } /* @@ -1887,6 +1886,9 @@ vdev_raidz_io_done_verified(zio_t *zio, raidz_row_t *rr) } else if (c < rr->rr_firstdatacol && !rc->rc_tried) { parity_untried++; } + + if (rc->rc_force_repair) + unexpected_errors++; } /* @@ -2251,9 +2253,20 @@ vdev_raidz_io_done_reconstruct_known_missing(zio_t *zio, raidz_map_t *rm, for (int c = 0; c < rr->rr_cols; c++) { raidz_col_t *rc = &rr->rr_col[c]; - if (rc->rc_error) { - ASSERT(rc->rc_error != ECKSUM); /* child has no bp */ + /* + * If scrubbing and a replacing/sparing child vdev determined + * that not all of its children have an identical copy of the + * data, then clear the error so the column is treated like + * any other read and force a repair to correct the damage. + */ + if (rc->rc_error == ECKSUM) { + ASSERT(zio->io_flags & ZIO_FLAG_SCRUB); + vdev_raidz_checksum_error(zio, rc, rc->rc_abd); + rc->rc_force_repair = 1; + rc->rc_error = 0; + } + if (rc->rc_error) { if (c < rr->rr_firstdatacol) parity_errors++; else diff --git a/module/zfs/vdev_raidz_math.c b/module/zfs/vdev_raidz_math.c index 50b8dab74848..f74a76a8d5ba 100644 --- a/module/zfs/vdev_raidz_math.c +++ b/module/zfs/vdev_raidz_math.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -563,7 +563,7 @@ vdev_raidz_math_fini(void) } static const struct { - char *name; + const char *name; uint32_t sel; } math_impl_opts[] = { { "cycle", IMPL_CYCLE }, diff --git a/module/zfs/vdev_raidz_math_aarch64_neon.c b/module/zfs/vdev_raidz_math_aarch64_neon.c index 0a67ceb84920..4aa7bc2b9708 100644 --- a/module/zfs/vdev_raidz_math_aarch64_neon.c +++ b/module/zfs/vdev_raidz_math_aarch64_neon.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_raidz_math_aarch64_neon_common.h b/module/zfs/vdev_raidz_math_aarch64_neon_common.h index e46b2536546c..f0f6546f7f71 100644 --- a/module/zfs/vdev_raidz_math_aarch64_neon_common.h +++ b/module/zfs/vdev_raidz_math_aarch64_neon_common.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_raidz_math_aarch64_neonx2.c b/module/zfs/vdev_raidz_math_aarch64_neonx2.c index e072f51cd635..0a1f05fd6664 100644 --- a/module/zfs/vdev_raidz_math_aarch64_neonx2.c +++ b/module/zfs/vdev_raidz_math_aarch64_neonx2.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_raidz_math_avx2.c b/module/zfs/vdev_raidz_math_avx2.c index 65e4bebce8fa..e5bbc7decbfa 100644 --- a/module/zfs/vdev_raidz_math_avx2.c +++ b/module/zfs/vdev_raidz_math_avx2.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_raidz_math_avx512bw.c b/module/zfs/vdev_raidz_math_avx512bw.c index f06b469023eb..3b709ed34fc4 100644 --- a/module/zfs/vdev_raidz_math_avx512bw.c +++ b/module/zfs/vdev_raidz_math_avx512bw.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_raidz_math_avx512f.c b/module/zfs/vdev_raidz_math_avx512f.c index aab653b77491..5ec71a04133a 100644 --- a/module/zfs/vdev_raidz_math_avx512f.c +++ b/module/zfs/vdev_raidz_math_avx512f.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_raidz_math_impl.h b/module/zfs/vdev_raidz_math_impl.h index 35e016fc65a5..2d96f602314c 100644 --- a/module/zfs/vdev_raidz_math_impl.h +++ b/module/zfs/vdev_raidz_math_impl.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_raidz_math_powerpc_altivec.c b/module/zfs/vdev_raidz_math_powerpc_altivec.c index 1db2c4cd3a47..ff493b8b7bc0 100644 --- a/module/zfs/vdev_raidz_math_powerpc_altivec.c +++ b/module/zfs/vdev_raidz_math_powerpc_altivec.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_raidz_math_powerpc_altivec_common.h b/module/zfs/vdev_raidz_math_powerpc_altivec_common.h index 3842f5fd637c..46d42c5e2417 100644 --- a/module/zfs/vdev_raidz_math_powerpc_altivec_common.h +++ b/module/zfs/vdev_raidz_math_powerpc_altivec_common.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_raidz_math_scalar.c b/module/zfs/vdev_raidz_math_scalar.c index 35fafe230ca1..b51352b4e90b 100644 --- a/module/zfs/vdev_raidz_math_scalar.c +++ b/module/zfs/vdev_raidz_math_scalar.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_raidz_math_sse2.c b/module/zfs/vdev_raidz_math_sse2.c index 56a0b123d952..02b5d6a609ab 100644 --- a/module/zfs/vdev_raidz_math_sse2.c +++ b/module/zfs/vdev_raidz_math_sse2.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_raidz_math_ssse3.c b/module/zfs/vdev_raidz_math_ssse3.c index 5ddc079a4f5d..244f137b3d09 100644 --- a/module/zfs/vdev_raidz_math_ssse3.c +++ b/module/zfs/vdev_raidz_math_ssse3.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_rebuild.c b/module/zfs/vdev_rebuild.c index a6866a8b263c..1ce578e228d8 100644 --- a/module/zfs/vdev_rebuild.c +++ b/module/zfs/vdev_rebuild.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -133,7 +133,7 @@ static int zfs_rebuild_scrub_enabled = 1; /* * For vdev_rebuild_initiate_sync() and vdev_rebuild_reset_sync(). */ -static void vdev_rebuild_thread(void *arg); +static __attribute__((noreturn)) void vdev_rebuild_thread(void *arg); /* * Clear the per-vdev rebuild bytes value for a vdev tree. @@ -227,7 +227,7 @@ vdev_rebuild_initiate_sync(void *arg, dmu_tx_t *tx) spa_feature_incr(vd->vdev_spa, SPA_FEATURE_DEVICE_REBUILD, tx); mutex_enter(&vd->vdev_rebuild_lock); - bzero(vrp, sizeof (uint64_t) * REBUILD_PHYS_ENTRIES); + memset(vrp, 0, sizeof (uint64_t) * REBUILD_PHYS_ENTRIES); vrp->vrp_rebuild_state = VDEV_REBUILD_ACTIVE; vrp->vrp_min_txg = 0; vrp->vrp_max_txg = dmu_tx_get_txg(tx); @@ -260,7 +260,7 @@ vdev_rebuild_initiate_sync(void *arg, dmu_tx_t *tx) } static void -vdev_rebuild_log_notify(spa_t *spa, vdev_t *vd, char *name) +vdev_rebuild_log_notify(spa_t *spa, vdev_t *vd, const char *name) { nvlist_t *aux = fnvlist_alloc(); @@ -448,7 +448,7 @@ vdev_rebuild_clear_sync(void *arg, dmu_tx_t *tx) } clear_rebuild_bytes(vd); - bzero(vrp, sizeof (uint64_t) * REBUILD_PHYS_ENTRIES); + memset(vrp, 0, sizeof (uint64_t) * REBUILD_PHYS_ENTRIES); if (vd->vdev_top_zap != 0 && zap_contains(mos, vd->vdev_top_zap, VDEV_TOP_ZAP_VDEV_REBUILD_PHYS) == 0) { @@ -701,7 +701,7 @@ vdev_rebuild_load(vdev_t *vd) vd->vdev_rebuilding = B_FALSE; if (!spa_feature_is_enabled(spa, SPA_FEATURE_DEVICE_REBUILD)) { - bzero(vrp, sizeof (uint64_t) * REBUILD_PHYS_ENTRIES); + memset(vrp, 0, sizeof (uint64_t) * REBUILD_PHYS_ENTRIES); mutex_exit(&vd->vdev_rebuild_lock); return (SET_ERROR(ENOTSUP)); } @@ -718,7 +718,7 @@ vdev_rebuild_load(vdev_t *vd) * status allowing a new resilver/rebuild to be started. */ if (err == ENOENT || err == EOVERFLOW || err == ECKSUM) { - bzero(vrp, sizeof (uint64_t) * REBUILD_PHYS_ENTRIES); + memset(vrp, 0, sizeof (uint64_t) * REBUILD_PHYS_ENTRIES); } else if (err) { mutex_exit(&vd->vdev_rebuild_lock); return (err); @@ -736,7 +736,7 @@ vdev_rebuild_load(vdev_t *vd) * Each scan thread is responsible for rebuilding a top-level vdev. The * rebuild progress in tracked on-disk in VDEV_TOP_ZAP_VDEV_REBUILD_PHYS. */ -static void +static __attribute__((noreturn)) void vdev_rebuild_thread(void *arg) { vdev_t *vd = arg; @@ -1111,7 +1111,7 @@ vdev_rebuild_get_stats(vdev_t *tvd, vdev_rebuild_stat_t *vrs) tvd->vdev_top_zap, VDEV_TOP_ZAP_VDEV_REBUILD_PHYS); if (error == ENOENT) { - bzero(vrs, sizeof (vdev_rebuild_stat_t)); + memset(vrs, 0, sizeof (vdev_rebuild_stat_t)); vrs->vrs_state = VDEV_REBUILD_NONE; error = 0; } else if (error == 0) { diff --git a/module/zfs/vdev_removal.c b/module/zfs/vdev_removal.c index 64be84edd8f5..a48b47a50540 100644 --- a/module/zfs/vdev_removal.c +++ b/module/zfs/vdev_removal.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -140,7 +140,7 @@ int zfs_removal_suspend_progress = 0; #define VDEV_REMOVAL_ZAP_OBJS "lzap" -static void spa_vdev_remove_thread(void *arg); +static __attribute__((noreturn)) void spa_vdev_remove_thread(void *arg); static int spa_vdev_remove_cancel_impl(spa_t *spa); static void @@ -338,8 +338,8 @@ spa_vdev_alloc(spa_t *spa, uint64_t guid) } static void -spa_vdev_remove_aux(nvlist_t *config, char *name, nvlist_t **dev, int count, - nvlist_t *dev_to_remove) +spa_vdev_remove_aux(nvlist_t *config, const char *name, nvlist_t **dev, + int count, nvlist_t *dev_to_remove) { nvlist_t **newdev = NULL; @@ -1364,6 +1364,8 @@ vdev_remove_complete(spa_t *spa) ASSERT3P(vd->vdev_initialize_thread, ==, NULL); ASSERT3P(vd->vdev_trim_thread, ==, NULL); ASSERT3P(vd->vdev_autotrim_thread, ==, NULL); + vdev_rebuild_stop_wait(vd); + ASSERT3P(vd->vdev_rebuild_thread, ==, NULL); uint64_t vdev_space = spa_deflate(spa) ? vd->vdev_stat.vs_dspace : vd->vdev_stat.vs_space; @@ -1386,7 +1388,6 @@ vdev_remove_complete(spa_t *spa) vdev_metaslab_fini(vd); metaslab_group_destroy(vd->vdev_mg); vd->vdev_mg = NULL; - spa_log_sm_set_blocklimit(spa); } if (vd->vdev_log_mg != NULL) { ASSERT0(vd->vdev_ms_count); @@ -1589,7 +1590,7 @@ spa_remove_max_segment(spa_t *spa) * TXG have completed (see spa_txg_zio) and writes the new mappings to disk * (see vdev_mapping_sync()). */ -static void +static __attribute__((noreturn)) void spa_vdev_remove_thread(void *arg) { spa_t *spa = arg; @@ -2131,7 +2132,6 @@ spa_vdev_remove_log(vdev_t *vd, uint64_t *txg) * metaslab_class_histogram_verify() */ vdev_metaslab_fini(vd); - spa_log_sm_set_blocklimit(spa); spa_vdev_config_exit(spa, NULL, *txg, 0, FTAG); *txg = spa_vdev_config_enter(spa); @@ -2251,7 +2251,6 @@ spa_vdev_remove_top_check(vdev_t *vd) * and not be raidz or draid. */ vdev_t *rvd = spa->spa_root_vdev; - int num_indirect = 0; for (uint64_t id = 0; id < rvd->vdev_children; id++) { vdev_t *cvd = rvd->vdev_child[id]; @@ -2267,8 +2266,6 @@ spa_vdev_remove_top_check(vdev_t *vd) if (cvd->vdev_ashift != 0 && cvd->vdev_alloc_bias == VDEV_BIAS_NONE) ASSERT3U(cvd->vdev_ashift, ==, spa->spa_max_ashift); - if (cvd->vdev_ops == &vdev_indirect_ops) - num_indirect++; if (!vdev_is_concrete(cvd)) continue; if (vdev_get_nparity(cvd) != 0) @@ -2387,7 +2384,8 @@ spa_vdev_remove(spa_t *spa, uint64_t guid, boolean_t unspare) int error = 0, error_log; boolean_t locked = MUTEX_HELD(&spa_namespace_lock); sysevent_t *ev = NULL; - char *vd_type = NULL, *vd_path = NULL; + const char *vd_type = NULL; + char *vd_path = NULL; ASSERT(spa_writeable(spa)); diff --git a/module/zfs/vdev_root.c b/module/zfs/vdev_root.c index 45ddc2f71927..e132643dc330 100644 --- a/module/zfs/vdev_root.c +++ b/module/zfs/vdev_root.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/vdev_trim.c b/module/zfs/vdev_trim.c index 39aee3786984..5905d9a07571 100644 --- a/module/zfs/vdev_trim.c +++ b/module/zfs/vdev_trim.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -834,7 +834,7 @@ vdev_trim_range_add(void *arg, uint64_t start, uint64_t size) * by its ms_allocatable. While a metaslab is undergoing trimming it is * not eligible for new allocations. */ -static void +static __attribute__((noreturn)) void vdev_trim_thread(void *arg) { vdev_t *vd = arg; @@ -1175,7 +1175,7 @@ vdev_trim_range_verify(void *arg, uint64_t start, uint64_t size) * N.B. This behavior is different from a manual TRIM where a thread * is created for each leaf vdev, instead of each top-level vdev. */ -static void +static __attribute__((noreturn)) void vdev_autotrim_thread(void *arg) { vdev_t *vd = arg; @@ -1514,13 +1514,13 @@ vdev_autotrim_restart(spa_t *spa) vdev_autotrim(spa); } -static void +static __attribute__((noreturn)) void vdev_trim_l2arc_thread(void *arg) { vdev_t *vd = arg; spa_t *spa = vd->vdev_spa; l2arc_dev_t *dev = l2arc_vdev_get(vd); - trim_args_t ta; + trim_args_t ta = {0}; range_seg64_t physical_rs; ASSERT(vdev_is_concrete(vd)); @@ -1531,7 +1531,6 @@ vdev_trim_l2arc_thread(void *arg) vd->vdev_trim_partial = 0; vd->vdev_trim_secure = 0; - bzero(&ta, sizeof (ta)); ta.trim_vdev = vd; ta.trim_tree = range_tree_create(NULL, RANGE_SEG64, NULL, 0, 0); ta.trim_type = TRIM_TYPE_MANUAL; @@ -1591,7 +1590,7 @@ vdev_trim_l2arc_thread(void *arg) */ spa_config_enter(vd->vdev_spa, SCL_L2ARC, vd, RW_READER); - bzero(dev->l2ad_dev_hdr, dev->l2ad_dev_hdr_asize); + memset(dev->l2ad_dev_hdr, 0, dev->l2ad_dev_hdr_asize); l2arc_dev_hdr_update(dev); spa_config_exit(vd->vdev_spa, SCL_L2ARC, vd); @@ -1655,9 +1654,9 @@ vdev_trim_l2arc(spa_t *spa) int vdev_trim_simple(vdev_t *vd, uint64_t start, uint64_t size) { - trim_args_t ta; - range_seg64_t physical_rs; - int error; + trim_args_t ta = {0}; + range_seg64_t physical_rs; + int error; physical_rs.rs_start = start; physical_rs.rs_end = start + size; @@ -1666,7 +1665,6 @@ vdev_trim_simple(vdev_t *vd, uint64_t start, uint64_t size) ASSERT(!vd->vdev_detached); ASSERT(!vd->vdev_top->vdev_removing); - bzero(&ta, sizeof (ta)); ta.trim_vdev = vd; ta.trim_tree = range_tree_create(NULL, RANGE_SEG64, NULL, 0, 0); ta.trim_type = TRIM_TYPE_SIMPLE; diff --git a/module/zfs/zap.c b/module/zfs/zap.c index d1d07f9fc804..7ba5ad9ea5af 100644 --- a/module/zfs/zap.c +++ b/module/zfs/zap.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -112,7 +112,7 @@ fzap_upgrade(zap_t *zap, dmu_tx_t *tx, zap_flags_t flags) * explicitly zero it since it might be coming from an * initialized microzap */ - bzero(zap->zap_dbuf->db_data, zap->zap_dbuf->db_size); + memset(zap->zap_dbuf->db_data, 0, zap->zap_dbuf->db_size); zp->zap_block_type = ZBT_HEADER; zp->zap_magic = ZAP_MAGIC; @@ -626,7 +626,7 @@ zap_deref_leaf(zap_t *zap, uint64_t h, dmu_tx_t *tx, krw_t lt, zap_leaf_t **lp) static int zap_expand_leaf(zap_name_t *zn, zap_leaf_t *l, - void *tag, dmu_tx_t *tx, zap_leaf_t **lp) + const void *tag, dmu_tx_t *tx, zap_leaf_t **lp) { zap_t *zap = zn->zn_zap; uint64_t hash = zn->zn_hash; @@ -715,7 +715,7 @@ zap_expand_leaf(zap_name_t *zn, zap_leaf_t *l, static void zap_put_leaf_maybe_grow_ptrtbl(zap_name_t *zn, zap_leaf_t *l, - void *tag, dmu_tx_t *tx) + const void *tag, dmu_tx_t *tx) { zap_t *zap = zn->zn_zap; int shift = zap_f_phys(zap)->zap_ptrtbl.zt_shift; @@ -824,7 +824,7 @@ fzap_lookup(zap_name_t *zn, int fzap_add_cd(zap_name_t *zn, uint64_t integer_size, uint64_t num_integers, - const void *val, uint32_t cd, void *tag, dmu_tx_t *tx) + const void *val, uint32_t cd, const void *tag, dmu_tx_t *tx) { zap_leaf_t *l; int err; @@ -876,7 +876,7 @@ fzap_add_cd(zap_name_t *zn, int fzap_add(zap_name_t *zn, uint64_t integer_size, uint64_t num_integers, - const void *val, void *tag, dmu_tx_t *tx) + const void *val, const void *tag, dmu_tx_t *tx) { int err = fzap_check(zn, integer_size, num_integers); if (err != 0) @@ -889,7 +889,7 @@ fzap_add(zap_name_t *zn, int fzap_update(zap_name_t *zn, int integer_size, uint64_t num_integers, const void *val, - void *tag, dmu_tx_t *tx) + const void *tag, dmu_tx_t *tx) { zap_leaf_t *l; int err; diff --git a/module/zfs/zap_leaf.c b/module/zfs/zap_leaf.c index aad923d512df..25c2d5163a26 100644 --- a/module/zfs/zap_leaf.c +++ b/module/zfs/zap_leaf.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -207,7 +207,7 @@ zap_leaf_chunk_free(zap_leaf_t *l, uint16_t chunk) zlf->lf_type = ZAP_CHUNK_FREE; zlf->lf_next = zap_leaf_phys(l)->l_hdr.lh_freelist; - bzero(zlf->lf_pad, sizeof (zlf->lf_pad)); /* help it to compress */ + memset(zlf->lf_pad, 0, sizeof (zlf->lf_pad)); /* help it to compress */ zap_leaf_phys(l)->l_hdr.lh_freelist = chunk; zap_leaf_phys(l)->l_hdr.lh_nfree++; @@ -304,7 +304,7 @@ zap_leaf_array_read(zap_leaf_t *l, uint16_t chunk, while (chunk != CHAIN_END) { struct zap_leaf_array *la = &ZAP_LEAF_CHUNK(l, chunk).l_array; - bcopy(la->la_array, p, ZAP_LEAF_ARRAY_BYTES); + memcpy(p, la->la_array, ZAP_LEAF_ARRAY_BYTES); p += ZAP_LEAF_ARRAY_BYTES; chunk = la->la_next; } @@ -344,7 +344,7 @@ zap_leaf_array_match(zap_leaf_t *l, zap_name_t *zn, zap_leaf_array_read(l, chunk, sizeof (*thiskey), array_numints, sizeof (*thiskey), array_numints, thiskey); - boolean_t match = bcmp(thiskey, zn->zn_key_orig, + boolean_t match = memcmp(thiskey, zn->zn_key_orig, array_numints * sizeof (*thiskey)) == 0; kmem_free(thiskey, array_numints * sizeof (*thiskey)); return (match); @@ -372,7 +372,8 @@ zap_leaf_array_match(zap_leaf_t *l, zap_name_t *zn, struct zap_leaf_array *la = &ZAP_LEAF_CHUNK(l, chunk).l_array; int toread = MIN(array_numints - bseen, ZAP_LEAF_ARRAY_BYTES); ASSERT3U(chunk, <, ZAP_LEAF_NUMCHUNKS(l)); - if (bcmp(la->la_array, (char *)zn->zn_key_orig + bseen, toread)) + if (memcmp(la->la_array, (char *)zn->zn_key_orig + bseen, + toread)) break; chunk = la->la_next; bseen += toread; diff --git a/module/zfs/zap_micro.c b/module/zfs/zap_micro.c index 1f32e4450522..58a5c9f600b7 100644 --- a/module/zfs/zap_micro.c +++ b/module/zfs/zap_micro.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -42,7 +42,7 @@ #endif static int mzap_upgrade(zap_t **zapp, - void *tag, dmu_tx_t *tx, zap_flags_t flags); + const void *tag, dmu_tx_t *tx, zap_flags_t flags); uint64_t zap_getflags(zap_t *zap) @@ -503,7 +503,7 @@ mzap_open(objset_t *os, uint64_t obj, dmu_buf_t *db) * have the specified tag. */ static int -zap_lockdir_impl(dmu_buf_t *db, void *tag, dmu_tx_t *tx, +zap_lockdir_impl(dmu_buf_t *db, const void *tag, dmu_tx_t *tx, krw_t lti, boolean_t fatreader, boolean_t adding, zap_t **zapp) { ASSERT0(db->db_offset); @@ -579,7 +579,8 @@ zap_lockdir_impl(dmu_buf_t *db, void *tag, dmu_tx_t *tx, static int zap_lockdir_by_dnode(dnode_t *dn, dmu_tx_t *tx, - krw_t lti, boolean_t fatreader, boolean_t adding, void *tag, zap_t **zapp) + krw_t lti, boolean_t fatreader, boolean_t adding, const void *tag, + zap_t **zapp) { dmu_buf_t *db; @@ -604,7 +605,8 @@ zap_lockdir_by_dnode(dnode_t *dn, dmu_tx_t *tx, int zap_lockdir(objset_t *os, uint64_t obj, dmu_tx_t *tx, - krw_t lti, boolean_t fatreader, boolean_t adding, void *tag, zap_t **zapp) + krw_t lti, boolean_t fatreader, boolean_t adding, const void *tag, + zap_t **zapp) { dmu_buf_t *db; @@ -625,14 +627,14 @@ zap_lockdir(objset_t *os, uint64_t obj, dmu_tx_t *tx, } void -zap_unlockdir(zap_t *zap, void *tag) +zap_unlockdir(zap_t *zap, const void *tag) { rw_exit(&zap->zap_rwlock); dmu_buf_rele(zap->zap_dbuf, tag); } static int -mzap_upgrade(zap_t **zapp, void *tag, dmu_tx_t *tx, zap_flags_t flags) +mzap_upgrade(zap_t **zapp, const void *tag, dmu_tx_t *tx, zap_flags_t flags) { int err = 0; zap_t *zap = *zapp; @@ -641,7 +643,7 @@ mzap_upgrade(zap_t **zapp, void *tag, dmu_tx_t *tx, zap_flags_t flags) int sz = zap->zap_dbuf->db_size; mzap_phys_t *mzp = vmem_alloc(sz, KM_SLEEP); - bcopy(zap->zap_dbuf->db_data, mzp, sz); + memcpy(mzp, zap->zap_dbuf->db_data, sz); int nchunks = zap->zap_m.zap_num_chunks; if (!flags) { @@ -725,7 +727,7 @@ static uint64_t zap_create_impl(objset_t *os, int normflags, zap_flags_t flags, dmu_object_type_t ot, int leaf_blockshift, int indirect_blockshift, dmu_object_type_t bonustype, int bonuslen, int dnodesize, - dnode_t **allocated_dnode, void *tag, dmu_tx_t *tx) + dnode_t **allocated_dnode, const void *tag, dmu_tx_t *tx) { uint64_t obj; @@ -857,7 +859,7 @@ uint64_t zap_create_hold(objset_t *os, int normflags, zap_flags_t flags, dmu_object_type_t ot, int leaf_blockshift, int indirect_blockshift, dmu_object_type_t bonustype, int bonuslen, int dnodesize, - dnode_t **allocated_dnode, void *tag, dmu_tx_t *tx) + dnode_t **allocated_dnode, const void *tag, dmu_tx_t *tx) { return (zap_create_impl(os, normflags, flags, ot, leaf_blockshift, indirect_blockshift, bonustype, bonuslen, dnodesize, @@ -1222,7 +1224,7 @@ mzap_addent(zap_name_t *zn, uint64_t value) static int zap_add_impl(zap_t *zap, const char *key, int integer_size, uint64_t num_integers, - const void *val, dmu_tx_t *tx, void *tag) + const void *val, dmu_tx_t *tx, const void *tag) { const uint64_t *intval = val; int err = 0; @@ -1407,7 +1409,7 @@ zap_remove_impl(zap_t *zap, const char *name, err = SET_ERROR(ENOENT); } else { zap->zap_m.zap_num_entries--; - bzero(&zap_m_phys(zap)->mz_chunk[mze->mze_chunkid], + memset(&zap_m_phys(zap)->mz_chunk[mze->mze_chunkid], 0, sizeof (mzap_ent_phys_t)); mze_remove(zap, mze); } @@ -1632,7 +1634,7 @@ zap_get_stats(objset_t *os, uint64_t zapobj, zap_stats_t *zs) if (err != 0) return (err); - bzero(zs, sizeof (zap_stats_t)); + memset(zs, 0, sizeof (zap_stats_t)); if (zap->zap_ismicro) { zs->zs_blocksize = zap->zap_dbuf->db_size; diff --git a/module/zfs/zcp.c b/module/zfs/zcp.c index dcfb55d3b3c4..fe90242ca40d 100644 --- a/module/zfs/zcp.c +++ b/module/zfs/zcp.c @@ -622,7 +622,7 @@ zcp_dataset_hold_error(lua_State *state, dsl_pool_t *dp, const char *dsname, */ dsl_dataset_t * zcp_dataset_hold(lua_State *state, dsl_pool_t *dp, const char *dsname, - void *tag) + const void *tag) { dsl_dataset_t *ds; int error = dsl_dataset_hold(dp, dsname, tag, &ds); diff --git a/module/zfs/zcp_get.c b/module/zfs/zcp_get.c index eb2c606163e1..af495ce85140 100644 --- a/module/zfs/zcp_get.c +++ b/module/zfs/zcp_get.c @@ -76,9 +76,8 @@ get_objset_type(dsl_dataset_t *ds, zfs_type_t *type) static int get_objset_type_name(dsl_dataset_t *ds, char *str) { - int error; - zfs_type_t type; - error = get_objset_type(ds, &type); + zfs_type_t type = ZFS_TYPE_INVALID; + int error = get_objset_type(ds, &type); if (error != 0) return (error); switch (type) { @@ -230,7 +229,7 @@ get_special_prop(lua_State *state, dsl_dataset_t *ds, const char *dsname, char *strval = kmem_alloc(ZAP_MAXVALUELEN, KM_SLEEP); char setpoint[ZFS_MAX_DATASET_NAME_LEN] = "Internal error - setpoint not determined"; - zfs_type_t ds_type; + zfs_type_t ds_type = ZFS_TYPE_INVALID; zprop_type_t prop_type = zfs_prop_get_type(zfs_prop); (void) get_objset_type(ds, &ds_type); @@ -497,8 +496,7 @@ get_zap_prop(lua_State *state, dsl_dataset_t *ds, zfs_prop_t zfs_prop) boolean_t prop_valid_for_ds(dsl_dataset_t *ds, zfs_prop_t zfs_prop) { - int error; - zfs_type_t zfs_type; + zfs_type_t zfs_type = ZFS_TYPE_INVALID; /* properties not supported */ if ((zfs_prop == ZFS_PROP_ISCSIOPTIONS) || @@ -509,7 +507,7 @@ prop_valid_for_ds(dsl_dataset_t *ds, zfs_prop_t zfs_prop) if ((zfs_prop == ZFS_PROP_ORIGIN) && (!dsl_dir_is_clone(ds->ds_dir))) return (B_FALSE); - error = get_objset_type(ds, &zfs_type); + int error = get_objset_type(ds, &zfs_type); if (error != 0) return (B_FALSE); return (zfs_prop_valid_for_type(zfs_prop, zfs_type, B_FALSE)); diff --git a/module/zfs/zcp_synctask.c b/module/zfs/zcp_synctask.c index 403856ae3571..24210117eca0 100644 --- a/module/zfs/zcp_synctask.c +++ b/module/zfs/zcp_synctask.c @@ -325,7 +325,7 @@ zcp_synctask_inherit_prop_check(void *arg, dmu_tx_t *tx) zcp_inherit_prop_arg_t *args = arg; zfs_prop_t prop = zfs_name_to_prop(args->zipa_prop); - if (prop == ZPROP_INVAL) { + if (prop == ZPROP_USERPROP) { if (zfs_prop_user(args->zipa_prop)) return (0); diff --git a/module/zfs/zfeature.c b/module/zfs/zfeature.c index 9d16fff81d0a..1d25bc406866 100644 --- a/module/zfs/zfeature.c +++ b/module/zfs/zfeature.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -389,6 +389,13 @@ feature_enable_sync(spa_t *spa, zfeature_info_t *feature, dmu_tx_t *tx) !spa_feature_is_active(spa, SPA_FEATURE_ENCRYPTION) && feature->fi_feature == SPA_FEATURE_BOOKMARK_V2) spa->spa_errata = 0; + + /* + * Convert the old on-disk error log to the new format when activating + * the head_errlog feature. + */ + if (feature->fi_feature == SPA_FEATURE_HEAD_ERRLOG) + spa_upgrade_errlog(spa, tx); } static void diff --git a/module/zfs/zfs_byteswap.c b/module/zfs/zfs_byteswap.c index cd35849c3f37..8666883f09a2 100644 --- a/module/zfs/zfs_byteswap.c +++ b/module/zfs/zfs_byteswap.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -36,9 +36,7 @@ static void zfs_oldace_byteswap(ace_t *ace, int ace_cnt) { - int i; - - for (i = 0; i != ace_cnt; i++, ace++) { + for (int i = 0; i != ace_cnt; i++, ace++) { ace->a_who = BSWAP_32(ace->a_who); ace->a_access_mask = BSWAP_32(ace->a_access_mask); ace->a_flags = BSWAP_16(ace->a_flags); @@ -138,23 +136,16 @@ zfs_ace_byteswap(void *buf, size_t size, boolean_t zfs_layout) } } -/* ARGSUSED */ void zfs_oldacl_byteswap(void *buf, size_t size) { - int cnt; - /* * Arggh, since we don't know how many ACEs are in * the array, we have to swap the entire block */ - - cnt = size / sizeof (ace_t); - - zfs_oldace_byteswap((ace_t *)buf, cnt); + zfs_oldace_byteswap((ace_t *)buf, size / sizeof (ace_t)); } -/* ARGSUSED */ void zfs_acl_byteswap(void *buf, size_t size) { diff --git a/module/zfs/zfs_chksum.c b/module/zfs/zfs_chksum.c new file mode 100644 index 000000000000..f890b1103934 --- /dev/null +++ b/module/zfs/zfs_chksum.c @@ -0,0 +1,334 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2021-2022 Tino Reichardt + */ + +#include +#include +#include +#include +#include + +#include + +static kstat_t *chksum_kstat = NULL; + +typedef struct { + const char *name; + const char *impl; + uint64_t bs1k; + uint64_t bs4k; + uint64_t bs16k; + uint64_t bs64k; + uint64_t bs256k; + uint64_t bs1m; + uint64_t bs4m; + uint64_t bs16m; + zio_cksum_salt_t salt; + zio_checksum_t *(func); + zio_checksum_tmpl_init_t *(init); + zio_checksum_tmpl_free_t *(free); +} chksum_stat_t; + +static int chksum_stat_cnt = 0; +static chksum_stat_t *chksum_stat_data = 0; + +/* + * i3-1005G1 test output: + * + * implementation 1k 4k 16k 64k 256k 1m 4m + * fletcher-4 5421 15001 26468 32555 34720 32801 18847 + * edonr-generic 1196 1602 1761 1749 1762 1759 1751 + * skein-generic 546 591 608 615 619 612 616 + * sha256-generic 246 270 274 274 277 275 276 + * sha256-avx 262 296 304 307 307 307 306 + * sha256-sha-ni 769 1072 1172 1220 1219 1232 1228 + * sha256-openssl 240 300 316 314 304 285 276 + * sha512-generic 333 374 385 392 391 393 392 + * sha512-openssl 353 441 467 476 472 467 426 + * sha512-avx 362 444 473 475 479 476 478 + * sha512-avx2 394 500 530 538 543 545 542 + * blake3-generic 308 313 313 313 312 313 312 + * blake3-sse2 402 1289 1423 1446 1432 1458 1413 + * blake3-sse41 427 1470 1625 1704 1679 1607 1629 + * blake3-avx2 428 1920 3095 3343 3356 3318 3204 + * blake3-avx512 473 2687 4905 5836 5844 5643 5374 + */ +static int +chksum_stat_kstat_headers(char *buf, size_t size) +{ + ssize_t off = 0; + + off += snprintf(buf + off, size, "%-23s", "implementation"); + off += snprintf(buf + off, size - off, "%8s", "1k"); + off += snprintf(buf + off, size - off, "%8s", "4k"); + off += snprintf(buf + off, size - off, "%8s", "16k"); + off += snprintf(buf + off, size - off, "%8s", "64k"); + off += snprintf(buf + off, size - off, "%8s", "256k"); + off += snprintf(buf + off, size - off, "%8s", "1m"); + off += snprintf(buf + off, size - off, "%8s", "4m"); + (void) snprintf(buf + off, size - off, "%8s\n", "16m"); + + return (0); +} + +static int +chksum_stat_kstat_data(char *buf, size_t size, void *data) +{ + chksum_stat_t *cs; + ssize_t off = 0; + char b[24]; + + cs = (chksum_stat_t *)data; + snprintf(b, 23, "%s-%s", cs->name, cs->impl); + off += snprintf(buf + off, size - off, "%-23s", b); + off += snprintf(buf + off, size - off, "%8llu", + (u_longlong_t)cs->bs1k); + off += snprintf(buf + off, size - off, "%8llu", + (u_longlong_t)cs->bs4k); + off += snprintf(buf + off, size - off, "%8llu", + (u_longlong_t)cs->bs16k); + off += snprintf(buf + off, size - off, "%8llu", + (u_longlong_t)cs->bs64k); + off += snprintf(buf + off, size - off, "%8llu", + (u_longlong_t)cs->bs256k); + off += snprintf(buf + off, size - off, "%8llu", + (u_longlong_t)cs->bs1m); + off += snprintf(buf + off, size - off, "%8llu", + (u_longlong_t)cs->bs4m); + (void) snprintf(buf + off, size - off, "%8llu\n", + (u_longlong_t)cs->bs16m); + + return (0); +} + +static void * +chksum_stat_kstat_addr(kstat_t *ksp, loff_t n) +{ + if (n < chksum_stat_cnt) + ksp->ks_private = (void *)(chksum_stat_data + n); + else + ksp->ks_private = NULL; + + return (ksp->ks_private); +} + +static void +chksum_run(chksum_stat_t *cs, abd_t *abd, void *ctx, int round, + uint64_t *result) +{ + hrtime_t start; + uint64_t run_bw, run_time_ns, run_count = 0, size = 0; + uint32_t l, loops = 0; + zio_cksum_t zcp; + + switch (round) { + case 1: /* 1k */ + size = 1<<10; loops = 128; break; + case 2: /* 2k */ + size = 1<<12; loops = 64; break; + case 3: /* 4k */ + size = 1<<14; loops = 32; break; + case 4: /* 16k */ + size = 1<<16; loops = 16; break; + case 5: /* 256k */ + size = 1<<18; loops = 8; break; + case 6: /* 1m */ + size = 1<<20; loops = 4; break; + case 7: /* 4m */ + size = 1<<22; loops = 1; break; + case 8: /* 16m */ + size = 1<<24; loops = 1; break; + } + + kpreempt_disable(); + start = gethrtime(); + do { + for (l = 0; l < loops; l++, run_count++) + cs->func(abd, size, ctx, &zcp); + + run_time_ns = gethrtime() - start; + } while (run_time_ns < MSEC2NSEC(1)); + kpreempt_enable(); + + run_bw = size * run_count * NANOSEC; + run_bw /= run_time_ns; /* B/s */ + *result = run_bw/1024/1024; /* MiB/s */ +} + +static void +chksum_benchit(chksum_stat_t *cs) +{ + abd_t *abd; + void *ctx = 0; + void *salt = &cs->salt.zcs_bytes; + + memset(salt, 0, sizeof (cs->salt.zcs_bytes)); + if (cs->init) { + ctx = cs->init(&cs->salt); + } + + /* allocate test memory via abd linear interface */ + abd = abd_alloc_linear(1<<20, B_FALSE); + chksum_run(cs, abd, ctx, 1, &cs->bs1k); + chksum_run(cs, abd, ctx, 2, &cs->bs4k); + chksum_run(cs, abd, ctx, 3, &cs->bs16k); + chksum_run(cs, abd, ctx, 4, &cs->bs64k); + chksum_run(cs, abd, ctx, 5, &cs->bs256k); + chksum_run(cs, abd, ctx, 6, &cs->bs1m); + abd_free(abd); + + /* allocate test memory via abd non linear interface */ + abd = abd_alloc(1<<24, B_FALSE); + chksum_run(cs, abd, ctx, 7, &cs->bs4m); + chksum_run(cs, abd, ctx, 8, &cs->bs16m); + abd_free(abd); + + /* free up temp memory */ + if (cs->free) { + cs->free(ctx); + } +} + +/* + * Initialize and benchmark all supported implementations. + */ +static void +chksum_benchmark(void) +{ + +#ifndef _KERNEL + /* we need the benchmark only for the kernel module */ + return; +#endif + + chksum_stat_t *cs; + int cbid = 0, id; + uint64_t max = 0; + + /* space for the benchmark times */ + chksum_stat_cnt = 4; + chksum_stat_cnt += blake3_get_impl_count(); + chksum_stat_data = (chksum_stat_t *)kmem_zalloc( + sizeof (chksum_stat_t) * chksum_stat_cnt, KM_SLEEP); + + /* edonr */ + cs = &chksum_stat_data[cbid++]; + cs->init = abd_checksum_edonr_tmpl_init; + cs->func = abd_checksum_edonr_native; + cs->free = abd_checksum_edonr_tmpl_free; + cs->name = "edonr"; + cs->impl = "generic"; + chksum_benchit(cs); + + /* skein */ + cs = &chksum_stat_data[cbid++]; + cs->init = abd_checksum_skein_tmpl_init; + cs->func = abd_checksum_skein_native; + cs->free = abd_checksum_skein_tmpl_free; + cs->name = "skein"; + cs->impl = "generic"; + chksum_benchit(cs); + + /* sha256 */ + cs = &chksum_stat_data[cbid++]; + cs->init = 0; + cs->func = abd_checksum_SHA256; + cs->free = 0; + cs->name = "sha256"; + cs->impl = "generic"; + chksum_benchit(cs); + + /* sha512 */ + cs = &chksum_stat_data[cbid++]; + cs->init = 0; + cs->func = abd_checksum_SHA512_native; + cs->free = 0; + cs->name = "sha512"; + cs->impl = "generic"; + chksum_benchit(cs); + + /* blake3 */ + for (id = 0; id < blake3_get_impl_count(); id++) { + blake3_set_impl_id(id); + cs = &chksum_stat_data[cbid++]; + cs->init = abd_checksum_blake3_tmpl_init; + cs->func = abd_checksum_blake3_native; + cs->free = abd_checksum_blake3_tmpl_free; + cs->name = "blake3"; + cs->impl = blake3_get_impl_name(); + chksum_benchit(cs); + if (cs->bs256k > max) { + max = cs->bs256k; + blake3_set_impl_fastest(id); + } + } +} + +void +chksum_init(void) +{ +#ifdef _KERNEL + blake3_per_cpu_ctx_init(); +#endif + + /* Benchmark supported implementations */ + chksum_benchmark(); + + /* Install kstats for all implementations */ + chksum_kstat = kstat_create("zfs", 0, "chksum_bench", "misc", + KSTAT_TYPE_RAW, 0, KSTAT_FLAG_VIRTUAL); + + if (chksum_kstat != NULL) { + chksum_kstat->ks_data = NULL; + chksum_kstat->ks_ndata = UINT32_MAX; + kstat_set_raw_ops(chksum_kstat, + chksum_stat_kstat_headers, + chksum_stat_kstat_data, + chksum_stat_kstat_addr); + kstat_install(chksum_kstat); + } + + /* setup implementations */ + blake3_setup_impl(); +} + +void +chksum_fini(void) +{ + if (chksum_kstat != NULL) { + kstat_delete(chksum_kstat); + chksum_kstat = NULL; + } + + if (chksum_stat_cnt) { + kmem_free(chksum_stat_data, + sizeof (chksum_stat_t) * chksum_stat_cnt); + chksum_stat_cnt = 0; + chksum_stat_data = 0; + } + +#ifdef _KERNEL + blake3_per_cpu_ctx_fini(); +#endif +} diff --git a/module/zfs/zfs_fm.c b/module/zfs/zfs_fm.c index 828385b430b4..47bb79a5d21a 100644 --- a/module/zfs/zfs_fm.c +++ b/module/zfs/zfs_fm.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -1150,7 +1150,7 @@ zfs_ereport_start_checksum(spa_t *spa, vdev_t *vd, const zbookmark_phys_t *zb, /* copy the checksum failure information if it was provided */ if (info != NULL) { report->zcr_ckinfo = kmem_zalloc(sizeof (*info), KM_SLEEP); - bcopy(info, report->zcr_ckinfo, sizeof (*info)); + memcpy(report->zcr_ckinfo, info, sizeof (*info)); } report->zcr_sector = 1ULL << vd->vdev_top->vdev_ashift; diff --git a/module/zfs/zfs_fuid.c b/module/zfs/zfs_fuid.c index 3aa60034d42a..35466c486ab3 100644 --- a/module/zfs/zfs_fuid.c +++ b/module/zfs/zfs_fuid.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -61,7 +61,7 @@ typedef struct fuid_domain { uint64_t f_idx; } fuid_domain_t; -static char *nulldomain = ""; +static const char *const nulldomain = ""; /* * Compare two indexes. @@ -171,7 +171,7 @@ zfs_fuid_table_destroy(avl_tree_t *idx_tree, avl_tree_t *domain_tree) avl_destroy(idx_tree); } -char * +const char * zfs_fuid_idx_domain(avl_tree_t *idx_tree, uint32_t idx) { fuid_domain_t searchnode, *findnode; @@ -290,9 +290,9 @@ zfs_fuid_sync(zfsvfs_t *zfsvfs, dmu_tx_t *tx) * necessary for the caller or another thread to detect the dirty table * and sync out the changes. */ -int +static int zfs_fuid_find_by_domain(zfsvfs_t *zfsvfs, const char *domain, - char **retdomain, boolean_t addok) + const char **retdomain, boolean_t addok) { fuid_domain_t searchnode, *findnode; avl_index_t loc; @@ -358,7 +358,7 @@ zfs_fuid_find_by_domain(zfsvfs_t *zfsvfs, const char *domain, const char * zfs_fuid_find_by_idx(zfsvfs_t *zfsvfs, uint32_t idx) { - char *domain; + const char *domain; if (idx == 0 || !zfsvfs->z_use_fuids) return (NULL); @@ -518,8 +518,7 @@ zfs_fuid_create_cred(zfsvfs_t *zfsvfs, zfs_fuid_type_t type, uint64_t idx; ksid_t *ksid; uint32_t rid; - char *kdomain; - const char *domain; + const char *kdomain, *domain; uid_t id; VERIFY(type == ZFS_OWNER || type == ZFS_GROUP); @@ -574,8 +573,7 @@ zfs_fuid_create(zfsvfs_t *zfsvfs, uint64_t id, cred_t *cr, zfs_fuid_type_t type, zfs_fuid_info_t **fuidpp) { #ifdef HAVE_KSID - const char *domain; - char *kdomain; + const char *domain, *kdomain; uint32_t fuid_idx = FUID_INDEX(id); uint32_t rid = 0; idmap_stat status; diff --git a/module/zfs/zfs_ioctl.c b/module/zfs/zfs_ioctl.c index 17eb7d497d48..382975208b97 100644 --- a/module/zfs/zfs_ioctl.c +++ b/module/zfs/zfs_ioctl.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -373,10 +373,10 @@ zfs_log_history(zfs_cmd_t *zc) * Policy for top-level read operations (list pools). Requires no privileges, * and can be used in the local zone, as there is no associated dataset. */ -/* ARGSUSED */ static int zfs_secpolicy_none(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) zc, (void) innvl, (void) cr; return (0); } @@ -384,10 +384,10 @@ zfs_secpolicy_none(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) * Policy for dataset read operations (list children, get statistics). Requires * no privileges, but must be visible in the local zone. */ -/* ARGSUSED */ static int zfs_secpolicy_read(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) innvl, (void) cr; if (INGLOBALZONE(curproc) || zone_dataset_visible(zc->zc_name, NULL)) return (0); @@ -656,35 +656,29 @@ zfs_secpolicy_setprop(const char *dsname, zfs_prop_t prop, nvpair_t *propval, return (zfs_secpolicy_write_perms(dsname, zfs_prop_to_name(prop), cr)); } -/* ARGSUSED */ static int zfs_secpolicy_set_fsacl(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { - int error; - - error = zfs_dozonecheck(zc->zc_name, cr); - if (error != 0) - return (error); - /* * permission to set permissions will be evaluated later in * dsl_deleg_can_allow() */ - return (0); + (void) innvl; + return (zfs_dozonecheck(zc->zc_name, cr)); } -/* ARGSUSED */ static int zfs_secpolicy_rollback(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) innvl; return (zfs_secpolicy_write_perms(zc->zc_name, ZFS_DELEG_PERM_ROLLBACK, cr)); } -/* ARGSUSED */ static int zfs_secpolicy_send(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) innvl; dsl_pool_t *dp; dsl_dataset_t *ds; const char *cp; @@ -717,10 +711,10 @@ zfs_secpolicy_send(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) return (error); } -/* ARGSUSED */ static int zfs_secpolicy_send_new(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) innvl; return (zfs_secpolicy_write_perms(zc->zc_name, ZFS_DELEG_PERM_SEND, cr)); } @@ -728,12 +722,14 @@ zfs_secpolicy_send_new(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) static int zfs_secpolicy_share(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) zc, (void) innvl, (void) cr; return (SET_ERROR(ENOTSUP)); } static int zfs_secpolicy_smb_acl(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) zc, (void) innvl, (void) cr; return (SET_ERROR(ENOTSUP)); } @@ -771,10 +767,10 @@ zfs_secpolicy_destroy_perms(const char *name, cred_t *cr) return (zfs_secpolicy_write_perms(name, ZFS_DELEG_PERM_DESTROY, cr)); } -/* ARGSUSED */ static int zfs_secpolicy_destroy(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) innvl; return (zfs_secpolicy_destroy_perms(zc->zc_name, cr)); } @@ -782,10 +778,10 @@ zfs_secpolicy_destroy(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) * Destroying snapshots with delegated permissions requires * descendant mount and destroy permissions. */ -/* ARGSUSED */ static int zfs_secpolicy_destroy_snaps(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) zc; nvlist_t *snaps; nvpair_t *pair, *nextpair; int error = 0; @@ -844,17 +840,17 @@ zfs_secpolicy_rename_perms(const char *from, const char *to, cred_t *cr) return (error); } -/* ARGSUSED */ static int zfs_secpolicy_rename(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) innvl; return (zfs_secpolicy_rename_perms(zc->zc_name, zc->zc_value, cr)); } -/* ARGSUSED */ static int zfs_secpolicy_promote(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) innvl; dsl_pool_t *dp; dsl_dataset_t *clone; int error; @@ -899,10 +895,10 @@ zfs_secpolicy_promote(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) return (error); } -/* ARGSUSED */ static int zfs_secpolicy_recv(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) innvl; int error; if ((error = zfs_secpolicy_write_perms(zc->zc_name, @@ -917,13 +913,6 @@ zfs_secpolicy_recv(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) ZFS_DELEG_PERM_CREATE, cr)); } -/* ARGSUSED */ -static int -zfs_secpolicy_recv_new(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) -{ - return (zfs_secpolicy_recv(zc, innvl, cr)); -} - int zfs_secpolicy_snapshot_perms(const char *name, cred_t *cr) { @@ -934,10 +923,10 @@ zfs_secpolicy_snapshot_perms(const char *name, cred_t *cr) /* * Check for permission to create each snapshot in the nvlist. */ -/* ARGSUSED */ static int zfs_secpolicy_snapshot(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) zc; nvlist_t *snaps; int error = 0; nvpair_t *pair; @@ -965,10 +954,10 @@ zfs_secpolicy_snapshot(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) /* * Check for permission to create each bookmark in the nvlist. */ -/* ARGSUSED */ static int zfs_secpolicy_bookmark(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) zc; int error = 0; for (nvpair_t *pair = nvlist_next_nvpair(innvl, NULL); @@ -990,10 +979,10 @@ zfs_secpolicy_bookmark(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) return (error); } -/* ARGSUSED */ static int zfs_secpolicy_destroy_bookmarks(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) zc; nvpair_t *pair, *nextpair; int error = 0; @@ -1031,10 +1020,10 @@ zfs_secpolicy_destroy_bookmarks(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) return (error); } -/* ARGSUSED */ static int zfs_secpolicy_log_history(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) zc, (void) innvl, (void) cr; /* * Even root must have a proper TSD so that we know what pool * to log to. @@ -1072,10 +1061,11 @@ zfs_secpolicy_create_clone(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) * Policy for pool operations - create/destroy pools, add vdevs, etc. Requires * SYS_CONFIG privilege, which is not available in a local zone. */ -/* ARGSUSED */ int zfs_secpolicy_config(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) zc, (void) innvl; + if (secpolicy_sys_config(cr, B_FALSE) != 0) return (SET_ERROR(EPERM)); @@ -1085,10 +1075,10 @@ zfs_secpolicy_config(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) /* * Policy for object to name lookups. */ -/* ARGSUSED */ static int zfs_secpolicy_diff(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) innvl; int error; if ((error = secpolicy_sys_config(cr, B_FALSE)) == 0) @@ -1101,20 +1091,20 @@ zfs_secpolicy_diff(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) /* * Policy for fault injection. Requires all privileges. */ -/* ARGSUSED */ static int zfs_secpolicy_inject(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) zc, (void) innvl; return (secpolicy_zinject(cr)); } -/* ARGSUSED */ static int zfs_secpolicy_inherit_prop(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) innvl; zfs_prop_t prop = zfs_name_to_prop(zc->zc_value); - if (prop == ZPROP_INVAL) { + if (prop == ZPROP_USERPROP) { if (!zfs_prop_user(zc->zc_value)) return (SET_ERROR(EINVAL)); return (zfs_secpolicy_write_perms(zc->zc_name, @@ -1174,18 +1164,18 @@ zfs_secpolicy_userspace_many(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) userquota_perms[zc->zc_objset_type], cr)); } -/* ARGSUSED */ static int zfs_secpolicy_userspace_upgrade(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) innvl; return (zfs_secpolicy_setprop(zc->zc_name, ZFS_PROP_VERSION, NULL, cr)); } -/* ARGSUSED */ static int zfs_secpolicy_hold(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) zc; nvpair_t *pair; nvlist_t *holds; int error; @@ -1206,10 +1196,10 @@ zfs_secpolicy_hold(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) return (0); } -/* ARGSUSED */ static int zfs_secpolicy_release(zfs_cmd_t *zc, nvlist_t *innvl, cred_t *cr) { + (void) zc; nvpair_t *pair; int error; @@ -1407,7 +1397,8 @@ getzfsvfs(const char *dsname, zfsvfs_t **zfvp) * which prevents all inode ops from running. */ static int -zfsvfs_hold(const char *name, void *tag, zfsvfs_t **zfvp, boolean_t writer) +zfsvfs_hold(const char *name, const void *tag, zfsvfs_t **zfvp, + boolean_t writer) { int error = 0; @@ -1432,7 +1423,7 @@ zfsvfs_hold(const char *name, void *tag, zfsvfs_t **zfvp, boolean_t writer) } static void -zfsvfs_rele(zfsvfs_t *zfsvfs, void *tag) +zfsvfs_rele(zfsvfs_t *zfsvfs, const void *tag) { ZFS_TEARDOWN_EXIT(zfsvfs, tag); @@ -2331,6 +2322,8 @@ zfs_ioc_snapshot_list_next(zfs_cmd_t *zc) } if (zc->zc_simple) { + zc->zc_objset_stats.dds_creation_txg = + dsl_get_creationtxg(ds); dsl_dataset_rele(ds, FTAG); break; } @@ -2416,7 +2409,7 @@ zfs_prop_set_special(const char *dsname, zprop_source_t source, const char *strval = NULL; int err = -1; - if (prop == ZPROP_INVAL) { + if (prop == ZPROP_USERPROP) { if (zfs_prop_userquota(propname)) return (zfs_prop_set_userquota(dsname, pair)); return (-1); @@ -2587,7 +2580,7 @@ zfs_set_prop_nvlist(const char *dsname, zprop_source_t source, nvlist_t *nvl, /* inherited properties are expected to be booleans */ if (nvpair_type(propval) != DATA_TYPE_BOOLEAN) err = SET_ERROR(EINVAL); - } else if (err == 0 && prop == ZPROP_INVAL) { + } else if (err == 0 && prop == ZPROP_USERPROP) { if (zfs_prop_user(propname)) { if (nvpair_type(propval) != DATA_TYPE_STRING) err = SET_ERROR(EINVAL); @@ -2863,11 +2856,11 @@ zfs_ioc_inherit_prop(zfs_cmd_t *zc) * and reservation to the received or default values even though * they are not considered inheritable. */ - if (prop != ZPROP_INVAL && !zfs_prop_inheritable(prop)) + if (prop != ZPROP_USERPROP && !zfs_prop_inheritable(prop)) return (SET_ERROR(EINVAL)); } - if (prop == ZPROP_INVAL) { + if (prop == ZPROP_USERPROP) { if (!zfs_prop_user(propname)) return (SET_ERROR(EINVAL)); @@ -3149,7 +3142,6 @@ zfs_ioc_get_fsacl(zfs_cmd_t *zc) return (error); } -/* ARGSUSED */ static void zfs_create_cb(objset_t *os, void *arg, cred_t *cr, dmu_tx_t *tx) { @@ -3511,11 +3503,11 @@ static const zfs_ioc_key_t zfs_keys_remap[] = { /* no nvl keys */ }; -/* ARGSUSED */ static int zfs_ioc_remap(const char *fsname, nvlist_t *innvl, nvlist_t *outnvl) { /* This IOCTL is no longer supported. */ + (void) fsname, (void) innvl, (void) outnvl; return (0); } @@ -3603,10 +3595,10 @@ static const zfs_ioc_key_t zfs_keys_log_history[] = { {"message", DATA_TYPE_STRING, 0}, }; -/* ARGSUSED */ static int zfs_ioc_log_history(const char *unused, nvlist_t *innvl, nvlist_t *outnvl) { + (void) unused, (void) outnvl; const char *message; char *poolname; spa_t *spa; @@ -3709,10 +3701,10 @@ zfs_unmount_snap(const char *snapname) (void) zfsctl_snapshot_unmount(snapname, MNT_FORCE); } -/* ARGSUSED */ static int zfs_unmount_snap_cb(const char *snapname, void *arg) { + (void) arg; zfs_unmount_snap(snapname); return (0); } @@ -3756,7 +3748,6 @@ static const zfs_ioc_key_t zfs_keys_destroy_snaps[] = { {"defer", DATA_TYPE_BOOLEAN, ZK_OPTIONAL}, }; -/* ARGSUSED */ static int zfs_ioc_destroy_snaps(const char *poolname, nvlist_t *innvl, nvlist_t *outnvl) { @@ -3809,10 +3800,10 @@ static const zfs_ioc_key_t zfs_keys_bookmark[] = { {"...", DATA_TYPE_STRING, ZK_WILDCARDLIST}, }; -/* ARGSUSED */ static int zfs_ioc_bookmark(const char *poolname, nvlist_t *innvl, nvlist_t *outnvl) { + (void) poolname; return (dsl_bookmark_create(innvl, outnvl)); } @@ -3849,11 +3840,11 @@ static const zfs_ioc_key_t zfs_keys_get_bookmark_props[] = { /* no nvl keys */ }; -/* ARGSUSED */ static int zfs_ioc_get_bookmark_props(const char *bookmark, nvlist_t *innvl, nvlist_t *outnvl) { + (void) innvl; char fsname[ZFS_MAX_DATASET_NAME_LEN]; char *bmname; @@ -3958,10 +3949,10 @@ static const zfs_ioc_key_t zfs_keys_pool_checkpoint[] = { /* no nvl keys */ }; -/* ARGSUSED */ static int zfs_ioc_pool_checkpoint(const char *poolname, nvlist_t *innvl, nvlist_t *outnvl) { + (void) innvl, (void) outnvl; return (spa_checkpoint(poolname)); } @@ -3973,11 +3964,11 @@ static const zfs_ioc_key_t zfs_keys_pool_discard_checkpoint[] = { /* no nvl keys */ }; -/* ARGSUSED */ static int zfs_ioc_pool_discard_checkpoint(const char *poolname, nvlist_t *innvl, nvlist_t *outnvl) { + (void) innvl, (void) outnvl; return (spa_checkpoint_discard(poolname)); } @@ -4339,7 +4330,6 @@ static const zfs_ioc_key_t zfs_keys_rollback[] = { {"target", DATA_TYPE_STRING, ZK_OPTIONAL}, }; -/* ARGSUSED */ static int zfs_ioc_rollback(const char *fsname, nvlist_t *innvl, nvlist_t *outnvl) { @@ -4411,14 +4401,15 @@ recursive_unmount(const char *fsname, void *arg) * outnvl is unused */ -/* ARGSUSED */ static const zfs_ioc_key_t zfs_keys_redact[] = { {"bookname", DATA_TYPE_STRING, 0}, {"snapnv", DATA_TYPE_NVLIST, 0}, }; + static int zfs_ioc_redact(const char *snapname, nvlist_t *innvl, nvlist_t *outnvl) { + (void) outnvl; nvlist_t *redactnvl = NULL; char *redactbook = NULL; @@ -4500,7 +4491,7 @@ zfs_check_settable(const char *dsname, nvpair_t *pair, cred_t *cr) uint64_t intval, compval; int err; - if (prop == ZPROP_INVAL) { + if (prop == ZPROP_USERPROP) { if (zfs_prop_user(propname)) { if ((err = zfs_secpolicy_write_perms(dsname, ZFS_DELEG_PERM_USERPROP, cr))) @@ -4937,7 +4928,7 @@ static boolean_t zfs_ioc_recv_inject_err; static int zfs_ioc_recv_impl(char *tofs, char *tosnap, char *origin, nvlist_t *recvprops, nvlist_t *localprops, nvlist_t *hidden_args, boolean_t force, - boolean_t resumable, int input_fd, + boolean_t heal, boolean_t resumable, int input_fd, dmu_replay_record_t *begin_record, uint64_t *read_bytes, uint64_t *errflags, nvlist_t **errors) { @@ -4962,7 +4953,7 @@ zfs_ioc_recv_impl(char *tofs, char *tosnap, char *origin, nvlist_t *recvprops, return (SET_ERROR(EBADF)); noff = off = zfs_file_off(input_fp); - error = dmu_recv_begin(tofs, tosnap, begin_record, force, + error = dmu_recv_begin(tofs, tosnap, begin_record, force, heal, resumable, localprops, hidden_args, origin, &drc, input_fp, &off); if (error != 0) @@ -5046,7 +5037,7 @@ zfs_ioc_recv_impl(char *tofs, char *tosnap, char *origin, nvlist_t *recvprops, /* -x property */ const char *name = nvpair_name(nvp); zfs_prop_t prop = zfs_name_to_prop(name); - if (prop != ZPROP_INVAL) { + if (prop != ZPROP_USERPROP) { if (!zfs_prop_inheritable(prop)) continue; } else if (!zfs_prop_user(name)) @@ -5305,7 +5296,7 @@ zfs_ioc_recv(zfs_cmd_t *zc) begin_record.drr_u.drr_begin = zc->zc_begin_record; error = zfs_ioc_recv_impl(tofs, tosnap, origin, recvdprops, localprops, - NULL, zc->zc_guid, B_FALSE, zc->zc_cookie, &begin_record, + NULL, zc->zc_guid, B_FALSE, B_FALSE, zc->zc_cookie, &begin_record, &zc->zc_cookie, &zc->zc_obj, &errors); nvlist_free(recvdprops); nvlist_free(localprops); @@ -5338,6 +5329,7 @@ zfs_ioc_recv(zfs_cmd_t *zc) * "begin_record" -> non-byteswapped dmu_replay_record_t * "input_fd" -> file descriptor to read stream from (int32) * (optional) "force" -> force flag (value ignored) + * (optional) "heal" -> use send stream to heal data corruption * (optional) "resumable" -> resumable flag (value ignored) * (optional) "cleanup_fd" -> unused * (optional) "action_handle" -> unused @@ -5358,6 +5350,7 @@ static const zfs_ioc_key_t zfs_keys_recv_new[] = { {"begin_record", DATA_TYPE_BYTE_ARRAY, 0}, {"input_fd", DATA_TYPE_INT32, 0}, {"force", DATA_TYPE_BOOLEAN, ZK_OPTIONAL}, + {"heal", DATA_TYPE_BOOLEAN, ZK_OPTIONAL}, {"resumable", DATA_TYPE_BOOLEAN, ZK_OPTIONAL}, {"cleanup_fd", DATA_TYPE_INT32, ZK_OPTIONAL}, {"action_handle", DATA_TYPE_UINT64, ZK_OPTIONAL}, @@ -5378,6 +5371,7 @@ zfs_ioc_recv_new(const char *fsname, nvlist_t *innvl, nvlist_t *outnvl) char *tosnap; char tofs[ZFS_MAX_DATASET_NAME_LEN]; boolean_t force; + boolean_t heal; boolean_t resumable; uint64_t read_bytes = 0; uint64_t errflags = 0; @@ -5407,6 +5401,7 @@ zfs_ioc_recv_new(const char *fsname, nvlist_t *innvl, nvlist_t *outnvl) input_fd = fnvlist_lookup_int32(innvl, "input_fd"); force = nvlist_exists(innvl, "force"); + heal = nvlist_exists(innvl, "heal"); resumable = nvlist_exists(innvl, "resumable"); /* we still use "props" here for backwards compatibility */ @@ -5423,7 +5418,7 @@ zfs_ioc_recv_new(const char *fsname, nvlist_t *innvl, nvlist_t *outnvl) return (error); error = zfs_ioc_recv_impl(tofs, tosnap, origin, recvprops, localprops, - hidden_args, force, resumable, input_fd, begin_record, + hidden_args, force, heal, resumable, input_fd, begin_record, &read_bytes, &errflags, &errors); fnvlist_add_uint64(outnvl, "read_bytes", read_bytes); @@ -5682,7 +5677,7 @@ zfs_ioc_error_log(zfs_cmd_t *zc) { spa_t *spa; int error; - size_t count = (size_t)zc->zc_nvlist_dst_size; + uint64_t count = zc->zc_nvlist_dst_size; if ((error = spa_open(zc->zc_name, &spa, FTAG)) != 0) return (error); @@ -5800,10 +5795,10 @@ static const zfs_ioc_key_t zfs_keys_pool_reopen[] = { {"scrub_restart", DATA_TYPE_BOOLEAN_VALUE, ZK_OPTIONAL}, }; -/* ARGSUSED */ static int zfs_ioc_pool_reopen(const char *pool, nvlist_t *innvl, nvlist_t *outnvl) { + (void) outnvl; spa_t *spa; int error; boolean_t rc, scrub_restart = B_TRUE; @@ -6210,10 +6205,10 @@ static const zfs_ioc_key_t zfs_keys_hold[] = { {"cleanup_fd", DATA_TYPE_INT32, ZK_OPTIONAL}, }; -/* ARGSUSED */ static int zfs_ioc_hold(const char *pool, nvlist_t *args, nvlist_t *errlist) { + (void) pool; nvpair_t *pair; nvlist_t *holds; int cleanup_fd = -1; @@ -6262,10 +6257,10 @@ static const zfs_ioc_key_t zfs_keys_get_holds[] = { /* no nvl keys */ }; -/* ARGSUSED */ static int zfs_ioc_get_holds(const char *snapname, nvlist_t *args, nvlist_t *outnvl) { + (void) args; return (dsl_dataset_get_holds(snapname, outnvl)); } @@ -6284,10 +6279,10 @@ static const zfs_ioc_key_t zfs_keys_release[] = { {"...", DATA_TYPE_NVLIST, ZK_WILDCARDLIST}, }; -/* ARGSUSED */ static int zfs_ioc_release(const char *pool, nvlist_t *holds, nvlist_t *errlist) { + (void) pool; return (dsl_dataset_user_release(holds, errlist)); } @@ -6521,10 +6516,10 @@ static const zfs_ioc_key_t zfs_keys_send_new[] = { {"redactbook", DATA_TYPE_STRING, ZK_OPTIONAL}, }; -/* ARGSUSED */ static int zfs_ioc_send_new(const char *snapname, nvlist_t *innvl, nvlist_t *outnvl) { + (void) outnvl; int error; offset_t off; char *fromname = NULL; @@ -6571,11 +6566,12 @@ zfs_ioc_send_new(const char *snapname, nvlist_t *innvl, nvlist_t *outnvl) return (error); } -/* ARGSUSED */ static int send_space_sum(objset_t *os, void *buf, int len, void *arg) { + (void) os, (void) buf; uint64_t *size = arg; + *size += len; return (0); } @@ -6767,10 +6763,10 @@ static const zfs_ioc_key_t zfs_keys_pool_sync[] = { {"force", DATA_TYPE_BOOLEAN_VALUE, 0}, }; -/* ARGSUSED */ static int zfs_ioc_pool_sync(const char *pool, nvlist_t *innvl, nvlist_t *onvl) { + (void) onvl; int err; boolean_t rc, force = B_FALSE; spa_t *spa; @@ -6810,10 +6806,10 @@ static const zfs_ioc_key_t zfs_keys_load_key[] = { {"noop", DATA_TYPE_BOOLEAN, ZK_OPTIONAL}, }; -/* ARGSUSED */ static int zfs_ioc_load_key(const char *dsname, nvlist_t *innvl, nvlist_t *outnvl) { + (void) outnvl; int ret; dsl_crypto_params_t *dcp = NULL; nvlist_t *hidden_args; @@ -6852,10 +6848,10 @@ static const zfs_ioc_key_t zfs_keys_unload_key[] = { /* no nvl keys */ }; -/* ARGSUSED */ static int zfs_ioc_unload_key(const char *dsname, nvlist_t *innvl, nvlist_t *outnvl) { + (void) innvl, (void) outnvl; int ret = 0; if (strchr(dsname, '@') != NULL || strchr(dsname, '%') != NULL) { @@ -6873,7 +6869,7 @@ zfs_ioc_unload_key(const char *dsname, nvlist_t *innvl, nvlist_t *outnvl) /* * Changes a user's wrapping key used to decrypt a dataset. The keyformat, - * keylocation, pbkdf2salt, and pbkdf2iters properties can also be specified + * keylocation, pbkdf2salt, and pbkdf2iters properties can also be specified * here to change how the key is derived in userspace. * * innvl: { @@ -6890,10 +6886,10 @@ static const zfs_ioc_key_t zfs_keys_change_key[] = { {"props", DATA_TYPE_NVLIST, ZK_OPTIONAL}, }; -/* ARGSUSED */ static int zfs_ioc_change_key(const char *dsname, nvlist_t *innvl, nvlist_t *outnvl) { + (void) outnvl; int ret; uint64_t cmd = DCP_CMD_NONE; dsl_crypto_params_t *dcp = NULL; @@ -7122,7 +7118,7 @@ zfs_ioctl_init(void) ARRAY_SIZE(zfs_keys_destroy_bookmarks)); zfs_ioctl_register("receive", ZFS_IOC_RECV_NEW, - zfs_ioc_recv_new, zfs_secpolicy_recv_new, DATASET_NAME, + zfs_ioc_recv_new, zfs_secpolicy_recv, DATASET_NAME, POOL_CHECK_SUSPENDED | POOL_CHECK_READONLY, B_TRUE, B_TRUE, zfs_keys_recv_new, ARRAY_SIZE(zfs_keys_recv_new)); zfs_ioctl_register("load-key", ZFS_IOC_LOAD_KEY, diff --git a/module/zfs/zfs_log.c b/module/zfs/zfs_log.c index babf0406eba5..c92044337bce 100644 --- a/module/zfs/zfs_log.c +++ b/module/zfs/zfs_log.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -107,86 +107,81 @@ zfs_log_create_txtype(zil_create_t type, vsecattr_t *vsecp, vattr_t *vap) static void zfs_log_xvattr(lr_attr_t *lrattr, xvattr_t *xvap) { - uint32_t *bitmap; - uint64_t *attrs; - uint64_t *crtime; - xoptattr_t *xoap; - void *scanstamp; - int i; + xoptattr_t *xoap; xoap = xva_getxoptattr(xvap); ASSERT(xoap); lrattr->lr_attr_masksize = xvap->xva_mapsize; - bitmap = &lrattr->lr_attr_bitmap; - for (i = 0; i != xvap->xva_mapsize; i++, bitmap++) { + uint32_t *bitmap = &lrattr->lr_attr_bitmap; + for (int i = 0; i != xvap->xva_mapsize; i++, bitmap++) *bitmap = xvap->xva_reqattrmap[i]; - } - /* Now pack the attributes up in a single uint64_t */ - attrs = (uint64_t *)bitmap; - *attrs = 0; - crtime = attrs + 1; - bzero(crtime, 2 * sizeof (uint64_t)); - scanstamp = (caddr_t)(crtime + 2); - bzero(scanstamp, AV_SCANSTAMP_SZ); + lr_attr_end_t *end = (lr_attr_end_t *)bitmap; + end->lr_attr_attrs = 0; + end->lr_attr_crtime[0] = 0; + end->lr_attr_crtime[1] = 0; + memset(end->lr_attr_scanstamp, 0, AV_SCANSTAMP_SZ); + if (XVA_ISSET_REQ(xvap, XAT_READONLY)) - *attrs |= (xoap->xoa_readonly == 0) ? 0 : + end->lr_attr_attrs |= (xoap->xoa_readonly == 0) ? 0 : XAT0_READONLY; if (XVA_ISSET_REQ(xvap, XAT_HIDDEN)) - *attrs |= (xoap->xoa_hidden == 0) ? 0 : + end->lr_attr_attrs |= (xoap->xoa_hidden == 0) ? 0 : XAT0_HIDDEN; if (XVA_ISSET_REQ(xvap, XAT_SYSTEM)) - *attrs |= (xoap->xoa_system == 0) ? 0 : + end->lr_attr_attrs |= (xoap->xoa_system == 0) ? 0 : XAT0_SYSTEM; if (XVA_ISSET_REQ(xvap, XAT_ARCHIVE)) - *attrs |= (xoap->xoa_archive == 0) ? 0 : + end->lr_attr_attrs |= (xoap->xoa_archive == 0) ? 0 : XAT0_ARCHIVE; if (XVA_ISSET_REQ(xvap, XAT_IMMUTABLE)) - *attrs |= (xoap->xoa_immutable == 0) ? 0 : + end->lr_attr_attrs |= (xoap->xoa_immutable == 0) ? 0 : XAT0_IMMUTABLE; if (XVA_ISSET_REQ(xvap, XAT_NOUNLINK)) - *attrs |= (xoap->xoa_nounlink == 0) ? 0 : + end->lr_attr_attrs |= (xoap->xoa_nounlink == 0) ? 0 : XAT0_NOUNLINK; if (XVA_ISSET_REQ(xvap, XAT_APPENDONLY)) - *attrs |= (xoap->xoa_appendonly == 0) ? 0 : + end->lr_attr_attrs |= (xoap->xoa_appendonly == 0) ? 0 : XAT0_APPENDONLY; if (XVA_ISSET_REQ(xvap, XAT_OPAQUE)) - *attrs |= (xoap->xoa_opaque == 0) ? 0 : + end->lr_attr_attrs |= (xoap->xoa_opaque == 0) ? 0 : XAT0_APPENDONLY; if (XVA_ISSET_REQ(xvap, XAT_NODUMP)) - *attrs |= (xoap->xoa_nodump == 0) ? 0 : + end->lr_attr_attrs |= (xoap->xoa_nodump == 0) ? 0 : XAT0_NODUMP; if (XVA_ISSET_REQ(xvap, XAT_AV_QUARANTINED)) - *attrs |= (xoap->xoa_av_quarantined == 0) ? 0 : + end->lr_attr_attrs |= (xoap->xoa_av_quarantined == 0) ? 0 : XAT0_AV_QUARANTINED; if (XVA_ISSET_REQ(xvap, XAT_AV_MODIFIED)) - *attrs |= (xoap->xoa_av_modified == 0) ? 0 : + end->lr_attr_attrs |= (xoap->xoa_av_modified == 0) ? 0 : XAT0_AV_MODIFIED; if (XVA_ISSET_REQ(xvap, XAT_CREATETIME)) - ZFS_TIME_ENCODE(&xoap->xoa_createtime, crtime); + ZFS_TIME_ENCODE(&xoap->xoa_createtime, end->lr_attr_crtime); if (XVA_ISSET_REQ(xvap, XAT_AV_SCANSTAMP)) { ASSERT(!XVA_ISSET_REQ(xvap, XAT_PROJID)); - bcopy(xoap->xoa_av_scanstamp, scanstamp, AV_SCANSTAMP_SZ); + memcpy(end->lr_attr_scanstamp, xoap->xoa_av_scanstamp, + AV_SCANSTAMP_SZ); } else if (XVA_ISSET_REQ(xvap, XAT_PROJID)) { /* * XAT_PROJID and XAT_AV_SCANSTAMP will never be valid * at the same time, so we can share the same space. */ - bcopy(&xoap->xoa_projid, scanstamp, sizeof (uint64_t)); + memcpy(end->lr_attr_scanstamp, &xoap->xoa_projid, + sizeof (uint64_t)); } if (XVA_ISSET_REQ(xvap, XAT_REPARSE)) - *attrs |= (xoap->xoa_reparse == 0) ? 0 : + end->lr_attr_attrs |= (xoap->xoa_reparse == 0) ? 0 : XAT0_REPARSE; if (XVA_ISSET_REQ(xvap, XAT_OFFLINE)) - *attrs |= (xoap->xoa_offline == 0) ? 0 : + end->lr_attr_attrs |= (xoap->xoa_offline == 0) ? 0 : XAT0_OFFLINE; if (XVA_ISSET_REQ(xvap, XAT_SPARSE)) - *attrs |= (xoap->xoa_sparse == 0) ? 0 : + end->lr_attr_attrs |= (xoap->xoa_sparse == 0) ? 0 : XAT0_SPARSE; if (XVA_ISSET_REQ(xvap, XAT_PROJINHERIT)) - *attrs |= (xoap->xoa_projinherit == 0) ? 0 : + end->lr_attr_attrs |= (xoap->xoa_projinherit == 0) ? 0 : XAT0_PROJINHERIT; } @@ -214,7 +209,7 @@ zfs_log_fuid_domains(zfs_fuid_info_t *fuidp, void *start) if (fuidp->z_domain_str_sz != 0) { for (zdomain = list_head(&fuidp->z_domains); zdomain; zdomain = list_next(&fuidp->z_domains, zdomain)) { - bcopy((void *)zdomain->z_domain, start, + memcpy(start, zdomain->z_domain, strlen(zdomain->z_domain) + 1); start = (caddr_t)start + strlen(zdomain->z_domain) + 1; @@ -392,7 +387,7 @@ zfs_log_create(zilog_t *zilog, dmu_tx_t *tx, uint64_t txtype, else lracl->lr_acl_flags = 0; - bcopy(vsecp->vsa_aclentp, end, aclsize); + memcpy(end, vsecp->vsa_aclentp, aclsize); end = (caddr_t)end + ZIL_ACE_LENGTH(aclsize); } @@ -404,7 +399,7 @@ zfs_log_create(zilog_t *zilog, dmu_tx_t *tx, uint64_t txtype, /* * Now place file name in log record */ - bcopy(name, end, namesize); + memcpy(end, name, namesize); zil_itx_assign(zilog, itx, tx); } @@ -426,7 +421,7 @@ zfs_log_remove(zilog_t *zilog, dmu_tx_t *tx, uint64_t txtype, itx = zil_itx_create(txtype, sizeof (*lr) + namesize); lr = (lr_remove_t *)&itx->itx_lr; lr->lr_doid = dzp->z_id; - bcopy(name, (char *)(lr + 1), namesize); + memcpy(lr + 1, name, namesize); itx->itx_oid = foid; @@ -462,7 +457,7 @@ zfs_log_link(zilog_t *zilog, dmu_tx_t *tx, uint64_t txtype, lr = (lr_link_t *)&itx->itx_lr; lr->lr_doid = dzp->z_id; lr->lr_link_obj = zp->z_id; - bcopy(name, (char *)(lr + 1), namesize); + memcpy(lr + 1, name, namesize); zil_itx_assign(zilog, itx, tx); } @@ -493,8 +488,8 @@ zfs_log_symlink(zilog_t *zilog, dmu_tx_t *tx, uint64_t txtype, sizeof (uint64_t)); (void) sa_lookup(zp->z_sa_hdl, SA_ZPL_CRTIME(ZTOZSB(zp)), lr->lr_crtime, sizeof (uint64_t) * 2); - bcopy(name, (char *)(lr + 1), namesize); - bcopy(link, (char *)(lr + 1) + namesize, linksize); + memcpy((char *)(lr + 1), name, namesize); + memcpy((char *)(lr + 1) + namesize, link, linksize); zil_itx_assign(zilog, itx, tx); } @@ -518,8 +513,8 @@ zfs_log_rename(zilog_t *zilog, dmu_tx_t *tx, uint64_t txtype, znode_t *sdzp, lr = (lr_rename_t *)&itx->itx_lr; lr->lr_sdoid = sdzp->z_id; lr->lr_tdoid = tdzp->z_id; - bcopy(sname, (char *)(lr + 1), snamesize); - bcopy(dname, (char *)(lr + 1) + snamesize, dnamesize); + memcpy((char *)(lr + 1), sname, snamesize); + memcpy((char *)(lr + 1) + snamesize, dname, dnamesize); itx->itx_oid = szp->z_id; zil_itx_assign(zilog, itx, tx); @@ -720,6 +715,40 @@ zfs_log_setattr(zilog_t *zilog, dmu_tx_t *tx, int txtype, zil_itx_assign(zilog, itx, tx); } +/* + * Handles TX_SETSAXATTR transactions. + */ +void +zfs_log_setsaxattr(zilog_t *zilog, dmu_tx_t *tx, int txtype, + znode_t *zp, const char *name, const void *value, size_t size) +{ + itx_t *itx; + lr_setsaxattr_t *lr; + size_t recsize = sizeof (lr_setsaxattr_t); + void *xattrstart; + int namelen; + + if (zil_replaying(zilog, tx) || zp->z_unlinked) + return; + + namelen = strlen(name) + 1; + recsize += (namelen + size); + itx = zil_itx_create(txtype, recsize); + lr = (lr_setsaxattr_t *)&itx->itx_lr; + lr->lr_foid = zp->z_id; + xattrstart = (char *)(lr + 1); + memcpy(xattrstart, name, namelen); + if (value != NULL) { + memcpy((char *)xattrstart + namelen, value, size); + lr->lr_size = size; + } else { + lr->lr_size = 0; + } + + itx->itx_sync = (zp->z_sync_cnt != 0); + zil_itx_assign(zilog, itx, tx); +} + /* * Handles TX_ACL transactions. */ @@ -768,11 +797,11 @@ zfs_log_acl(zilog_t *zilog, dmu_tx_t *tx, znode_t *zp, if (txtype == TX_ACL_V0) { lrv0 = (lr_acl_v0_t *)lr; - bcopy(vsecp->vsa_aclentp, (ace_t *)(lrv0 + 1), aclbytes); + memcpy(lrv0 + 1, vsecp->vsa_aclentp, aclbytes); } else { void *start = (ace_t *)(lr + 1); - bcopy(vsecp->vsa_aclentp, start, aclbytes); + memcpy(start, vsecp->vsa_aclentp, aclbytes); start = (caddr_t)start + ZIL_ACE_LENGTH(aclbytes); diff --git a/module/zfs/zfs_onexit.c b/module/zfs/zfs_onexit.c index 7c56dd9c97f5..dfcdeeb5b46f 100644 --- a/module/zfs/zfs_onexit.c +++ b/module/zfs/zfs_onexit.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/zfs_quota.c b/module/zfs/zfs_quota.c index e61db5c7ab83..a5dc5c399b5d 100644 --- a/module/zfs/zfs_quota.c +++ b/module/zfs/zfs_quota.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/zfs_ratelimit.c b/module/zfs/zfs_ratelimit.c index b18b480ce527..091562ca6852 100644 --- a/module/zfs/zfs_ratelimit.c +++ b/module/zfs/zfs_ratelimit.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/zfs_replay.c b/module/zfs/zfs_replay.c index b8288b50f2c6..379e1d1a7b57 100644 --- a/module/zfs/zfs_replay.c +++ b/module/zfs/zfs_replay.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -47,6 +47,8 @@ #include #include #include +#include +#include /* * NB: FreeBSD expects to be able to do vnode locking in lookup and @@ -68,7 +70,7 @@ static void zfs_init_vattr(vattr_t *vap, uint64_t mask, uint64_t mode, uint64_t uid, uint64_t gid, uint64_t rdev, uint64_t nodeid) { - bzero(vap, sizeof (*vap)); + memset(vap, 0, sizeof (*vap)); vap->va_mask = (uint_t)mask; vap->va_mode = mode; #if defined(__FreeBSD__) || defined(__APPLE__) @@ -80,10 +82,10 @@ zfs_init_vattr(vattr_t *vap, uint64_t mask, uint64_t mode, vap->va_nodeid = nodeid; } -/* ARGSUSED */ static int zfs_replay_error(void *arg1, void *arg2, boolean_t byteswap) { + (void) arg1, (void) arg2, (void) byteswap; return (SET_ERROR(ENOTSUP)); } @@ -141,13 +143,13 @@ zfs_replay_xvattr(lr_attr_t *lrattr, xvattr_t *xvap) if (XVA_ISSET_REQ(xvap, XAT_AV_SCANSTAMP)) { ASSERT(!XVA_ISSET_REQ(xvap, XAT_PROJID)); - bcopy(scanstamp, xoap->xoa_av_scanstamp, AV_SCANSTAMP_SZ); + memcpy(xoap->xoa_av_scanstamp, scanstamp, AV_SCANSTAMP_SZ); } else if (XVA_ISSET_REQ(xvap, XAT_PROJID)) { /* * XAT_PROJID and XAT_AV_SCANSTAMP will never be valid * at the same time, so we can share the same space. */ - bcopy(scanstamp, &xoap->xoa_projid, sizeof (uint64_t)); + memcpy(&xoap->xoa_projid, scanstamp, sizeof (uint64_t)); } if (XVA_ISSET_REQ(xvap, XAT_REPARSE)) xoap->xoa_reparse = ((*attrs & XAT0_REPARSE) != 0); @@ -789,7 +791,7 @@ zfs_replay_truncate(void *arg1, void *arg2, boolean_t byteswap) zfsvfs_t *zfsvfs = arg1; lr_truncate_t *lr = arg2; znode_t *zp; - flock64_t fl; + flock64_t fl = {0}; int error; if (byteswap) @@ -798,7 +800,6 @@ zfs_replay_truncate(void *arg1, void *arg2, boolean_t byteswap) if ((error = zfs_zget(zfsvfs, lr->lr_foid, &zp)) != 0) return (error); - bzero(&fl, sizeof (fl)); fl.l_type = F_WRLCK; fl.l_whence = SEEK_SET; fl.l_start = lr->lr_offset; @@ -868,13 +869,93 @@ zfs_replay_setattr(void *arg1, void *arg2, boolean_t byteswap) return (error); } +static int +zfs_replay_setsaxattr(void *arg1, void *arg2, boolean_t byteswap) +{ + zfsvfs_t *zfsvfs = arg1; + lr_setsaxattr_t *lr = arg2; + znode_t *zp; + nvlist_t *nvl; + size_t sa_size; + char *name; + char *value; + size_t size; + int error = 0; + + ASSERT(spa_feature_is_active(zfsvfs->z_os->os_spa, + SPA_FEATURE_ZILSAXATTR)); + if (byteswap) + byteswap_uint64_array(lr, sizeof (*lr)); + + if ((error = zfs_zget(zfsvfs, lr->lr_foid, &zp)) != 0) + return (error); + + rw_enter(&zp->z_xattr_lock, RW_WRITER); + mutex_enter(&zp->z_lock); + if (zp->z_xattr_cached == NULL) + error = zfs_sa_get_xattr(zp); + mutex_exit(&zp->z_lock); + + if (error) + goto out; + + ASSERT(zp->z_xattr_cached); + nvl = zp->z_xattr_cached; + + /* Get xattr name, value and size from log record */ + size = lr->lr_size; + name = (char *)(lr + 1); + if (size == 0) { + value = NULL; + error = nvlist_remove(nvl, name, DATA_TYPE_BYTE_ARRAY); + } else { + value = name + strlen(name) + 1; + /* Limited to 32k to keep nvpair memory allocations small */ + if (size > DXATTR_MAX_ENTRY_SIZE) { + error = SET_ERROR(EFBIG); + goto out; + } + + /* Prevent the DXATTR SA from consuming the entire SA region */ + error = nvlist_size(nvl, &sa_size, NV_ENCODE_XDR); + if (error) + goto out; + + if (sa_size > DXATTR_MAX_SA_SIZE) { + error = SET_ERROR(EFBIG); + goto out; + } + + error = nvlist_add_byte_array(nvl, name, (uchar_t *)value, + size); + } + + /* + * Update the SA for additions, modifications, and removals. On + * error drop the inconsistent cached version of the nvlist, it + * will be reconstructed from the ARC when next accessed. + */ + if (error == 0) + error = zfs_sa_set_xattr(zp, name, value, size); + + if (error) { + nvlist_free(nvl); + zp->z_xattr_cached = NULL; + } + +out: + rw_exit(&zp->z_xattr_lock); + zrele(zp); + return (error); +} + static int zfs_replay_acl_v0(void *arg1, void *arg2, boolean_t byteswap) { zfsvfs_t *zfsvfs = arg1; lr_acl_v0_t *lr = arg2; ace_t *ace = (ace_t *)(lr + 1); /* ace array follows lr_acl_t */ - vsecattr_t vsa; + vsecattr_t vsa = {0}; znode_t *zp; int error; @@ -886,7 +967,6 @@ zfs_replay_acl_v0(void *arg1, void *arg2, boolean_t byteswap) if ((error = zfs_zget(zfsvfs, lr->lr_foid, &zp)) != 0) return (error); - bzero(&vsa, sizeof (vsa)); vsa.vsa_mask = VSA_ACE | VSA_ACECNT; vsa.vsa_aclcnt = lr->lr_aclcnt; vsa.vsa_aclentsz = sizeof (ace_t) * vsa.vsa_aclcnt; @@ -920,7 +1000,7 @@ zfs_replay_acl(void *arg1, void *arg2, boolean_t byteswap) zfsvfs_t *zfsvfs = arg1; lr_acl_t *lr = arg2; ace_t *ace = (ace_t *)(lr + 1); - vsecattr_t vsa; + vsecattr_t vsa = {0}; znode_t *zp; int error; @@ -937,7 +1017,6 @@ zfs_replay_acl(void *arg1, void *arg2, boolean_t byteswap) if ((error = zfs_zget(zfsvfs, lr->lr_foid, &zp)) != 0) return (error); - bzero(&vsa, sizeof (vsa)); vsa.vsa_mask = VSA_ACE | VSA_ACECNT | VSA_ACE_ACLFLAGS; vsa.vsa_aclcnt = lr->lr_aclcnt; vsa.vsa_aclentp = ace; @@ -989,4 +1068,5 @@ zil_replay_func_t *const zfs_replay_vector[TX_MAX_TYPE] = { zfs_replay_create, /* TX_MKDIR_ATTR */ zfs_replay_create_acl, /* TX_MKDIR_ACL_ATTR */ zfs_replay_write2, /* TX_WRITE2 */ + zfs_replay_setsaxattr, /* TX_SETSAXATTR */ }; diff --git a/module/zfs/zfs_rlock.c b/module/zfs/zfs_rlock.c index 06a5e031a7df..f42661df82e4 100644 --- a/module/zfs/zfs_rlock.c +++ b/module/zfs/zfs_rlock.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/zfs_sa.c b/module/zfs/zfs_sa.c index 817f63048789..fb2443b756f8 100644 --- a/module/zfs/zfs_sa.c +++ b/module/zfs/zfs_sa.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -29,6 +29,7 @@ #include #include #include +#include /* * ZPL attribute registration table. @@ -69,7 +70,10 @@ const sa_attr_reg_t zfs_attr_table[ZPL_END+1] = { {NULL, 0, 0, 0} }; + #ifdef _KERNEL +static int zfs_zil_saxattr = 1; + int zfs_sa_readlink(znode_t *zp, zfs_uio_t *uio) { @@ -103,8 +107,8 @@ zfs_sa_symlink(znode_t *zp, char *link, int len, dmu_tx_t *tx) if (ZFS_OLD_ZNODE_PHYS_SIZE + len <= dmu_bonus_max()) { VERIFY0(dmu_set_bonus(db, len + ZFS_OLD_ZNODE_PHYS_SIZE, tx)); if (len) { - bcopy(link, (caddr_t)db->db_data + - ZFS_OLD_ZNODE_PHYS_SIZE, len); + memcpy((caddr_t)db->db_data + + ZFS_OLD_ZNODE_PHYS_SIZE, link, len); } } else { dmu_buf_t *dbp; @@ -116,7 +120,7 @@ zfs_sa_symlink(znode_t *zp, char *link, int len, dmu_tx_t *tx) dmu_buf_will_dirty(dbp, tx); ASSERT3U(len, <=, dbp->db_size); - bcopy(link, dbp->db_data, len); + memcpy(dbp->db_data, link, len); dmu_buf_rele(dbp, FTAG); } } @@ -219,13 +223,14 @@ zfs_sa_get_xattr(znode_t *zp) } int -zfs_sa_set_xattr(znode_t *zp) +zfs_sa_set_xattr(znode_t *zp, const char *name, const void *value, size_t vsize) { zfsvfs_t *zfsvfs = ZTOZSB(zp); + zilog_t *zilog; dmu_tx_t *tx; char *obj; size_t size; - int error; + int error, logsaxattr = 0; ASSERT(RW_WRITE_HELD(&zp->z_xattr_lock)); ASSERT(zp->z_xattr_cached); @@ -244,6 +249,17 @@ zfs_sa_set_xattr(znode_t *zp) if (error) goto out_free; + zilog = zfsvfs->z_log; + + /* + * Users enable ZIL logging of xattr=sa operations by enabling the + * SPA_FEATURE_ZILSAXATTR feature on the pool. Feature is activated + * during zil_process_commit_list/zil_create, if enabled. + */ + if (spa_feature_is_enabled(zfsvfs->z_os->os_spa, + SPA_FEATURE_ZILSAXATTR) && zfs_zil_saxattr) + logsaxattr = 1; + tx = dmu_tx_create(zfsvfs->z_os); dmu_tx_hold_sa_create(tx, size); dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_TRUE); @@ -256,6 +272,10 @@ zfs_sa_set_xattr(znode_t *zp) sa_bulk_attr_t bulk[2]; uint64_t ctime[2]; + if (logsaxattr) + zfs_log_setsaxattr(zilog, tx, TX_SETSAXATTR, zp, name, + value, vsize); + zfs_tstamp_update_setup(zp, STATE_CHANGED, NULL, ctime); SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_DXATTR(zfsvfs), NULL, obj, size); @@ -264,6 +284,8 @@ zfs_sa_set_xattr(znode_t *zp) VERIFY0(sa_bulk_update(zp->z_sa_hdl, bulk, count, tx)); dmu_tx_commit(tx); + if (logsaxattr && zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS) + zil_commit(zilog, 0); } out_free: vmem_free(obj, size); @@ -396,8 +418,9 @@ zfs_sa_upgrade(sa_handle_t *hdl, dmu_tx_t *tx) /* if scanstamp then add scanstamp */ if (zp->z_pflags & ZFS_BONUS_SCANSTAMP) { - bcopy((caddr_t)db->db_data + ZFS_OLD_ZNODE_PHYS_SIZE, - scanstamp, AV_SCANSTAMP_SZ); + memcpy(scanstamp, + (caddr_t)db->db_data + ZFS_OLD_ZNODE_PHYS_SIZE, + AV_SCANSTAMP_SZ); SA_ADD_BULK_ATTR(sa_attrs, count, SA_ZPL_SCANSTAMP(zfsvfs), NULL, scanstamp, AV_SCANSTAMP_SZ); zp->z_pflags &= ~ZFS_BONUS_SCANSTAMP; @@ -433,6 +456,9 @@ zfs_sa_upgrade_txholds(dmu_tx_t *tx, znode_t *zp) } } +ZFS_MODULE_PARAM(zfs, zfs_, zil_saxattr, INT, ZMOD_RW, + "Disable xattr=sa extended attribute logging in ZIL by settng 0."); + EXPORT_SYMBOL(zfs_attr_table); EXPORT_SYMBOL(zfs_sa_readlink); EXPORT_SYMBOL(zfs_sa_symlink); diff --git a/module/zfs/zfs_vnops.c b/module/zfs/zfs_vnops.c index 918938d62823..b02e8283c77d 100644 --- a/module/zfs/zfs_vnops.c +++ b/module/zfs/zfs_vnops.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -68,7 +68,9 @@ zfs_fsync(znode_t *zp, int syncflag, cred_t *cr) if (zfsvfs->z_os->os_sync != ZFS_SYNC_DISABLED) { ZFS_ENTER(zfsvfs); ZFS_VERIFY_ZP(zp); + atomic_inc_32(&zp->z_sync_writes_cnt); zil_commit(zfsvfs->z_log, zp->z_id); + atomic_dec_32(&zp->z_sync_writes_cnt); ZFS_EXIT(zfsvfs); } tsd_set(zfs_fsyncer_key, NULL); @@ -154,7 +156,6 @@ zfs_holey(znode_t *zp, ulong_t cmd, loff_t *off) } #endif /* SEEK_HOLE && SEEK_DATA */ -/*ARGSUSED*/ int zfs_access(znode_t *zp, int mode, int flag, cred_t *cr) { @@ -192,10 +193,10 @@ static unsigned long zfs_vnops_read_chunk_size = 1024 * 1024; /* Tunable */ * Side Effects: * inode - atime updated if byte count > 0 */ -/* ARGSUSED */ int zfs_read(struct znode *zp, zfs_uio_t *uio, int ioflag, cred_t *cr) { + (void) cr; int error = 0; boolean_t frsync = B_FALSE; @@ -356,14 +357,13 @@ zfs_clear_setid_bits_if_necessary(zfsvfs_t *zfsvfs, znode_t *zp, cred_t *cr, * than one TX_SETATTR per transaction group. */ if (*clear_setid_bits_txgp != dmu_tx_get_txg(tx)) { - vattr_t va; + vattr_t va = {0}; - bzero(&va, sizeof (va)); - va.va_mask = AT_MODE; + va.va_mask = ATTR_MODE; va.va_nodeid = zp->z_id; va.va_mode = newmode; - zfs_log_setattr(zilog, tx, TX_SETATTR, zp, &va, AT_MODE, - NULL); + zfs_log_setattr(zilog, tx, TX_SETATTR, zp, &va, + ATTR_MODE, NULL); *clear_setid_bits_txgp = dmu_tx_get_txg(tx); } } else { @@ -389,8 +389,6 @@ zfs_clear_setid_bits_if_necessary(zfsvfs_t *zfsvfs, znode_t *zp, cred_t *cr, * Timestamps: * ip - ctime|mtime updated if byte count > 0 */ - -/* ARGSUSED */ int zfs_write(znode_t *zp, zfs_uio_t *uio, int ioflag, cred_t *cr) { @@ -779,7 +777,6 @@ zfs_write(znode_t *zp, zfs_uio_t *uio, int ioflag, cred_t *cr) return (0); } -/*ARGSUSED*/ int zfs_getsecattr(znode_t *zp, vsecattr_t *vsecp, int flag, cred_t *cr) { @@ -795,7 +792,6 @@ zfs_getsecattr(znode_t *zp, vsecattr_t *vsecp, int flag, cred_t *cr) return (error); } -/*ARGSUSED*/ int zfs_setsecattr(znode_t *zp, vsecattr_t *vsecp, int flag, cred_t *cr) { @@ -965,10 +961,10 @@ zfs_get_data(void *arg, uint64_t gen, lr_write_t *lr, char *buf, } -/* ARGSUSED */ static void zfs_get_done(zgd_t *zgd, int error) { + (void) error; znode_t *zp = zgd->zgd_private; if (zgd->zgd_db) diff --git a/module/zfs/zil.c b/module/zfs/zil.c index 87a50a5c4950..4864e0ccad53 100644 --- a/module/zfs/zil.c +++ b/module/zfs/zil.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -43,6 +43,7 @@ #include #include #include +#include /* * The ZFS Intent Log (ZIL) saves "transaction records" (itxs) of system @@ -94,7 +95,7 @@ static int zfs_commit_timeout_pct = 5; /* * See zil.h for more information about these fields. */ -static zil_stats_t zil_stats = { +static zil_kstat_values_t zil_stats = { { "zil_commit_count", KSTAT_DATA_UINT64 }, { "zil_commit_writer_count", KSTAT_DATA_UINT64 }, { "zil_itx_count", KSTAT_DATA_UINT64 }, @@ -110,7 +111,8 @@ static zil_stats_t zil_stats = { { "zil_itx_metaslab_slog_bytes", KSTAT_DATA_UINT64 }, }; -static kstat_t *zil_ksp; +static zil_sums_t zil_sums_global; +static kstat_t *zil_kstats_global; /* * Disable intent logging replay. This global ZIL switch affects all pools. @@ -213,6 +215,21 @@ zil_init_log_chain(zilog_t *zilog, blkptr_t *bp) zc->zc_word[ZIL_ZC_SEQ] = 1ULL; } +static int +zil_kstats_global_update(kstat_t *ksp, int rw) +{ + zil_kstat_values_t *zs = ksp->ks_data; + ASSERT3P(&zil_stats, ==, zs); + + if (rw == KSTAT_WRITE) { + return (SET_ERROR(EACCES)); + } + + zil_kstat_values_update(zs, &zil_sums_global); + + return (0); +} + /* * Read a log block and make sure it's valid. */ @@ -259,12 +276,12 @@ zil_read_log_block(zilog_t *zilog, boolean_t decrypt, const blkptr_t *bp, char *lr = (char *)(zilc + 1); uint64_t len = zilc->zc_nused - sizeof (zil_chain_t); - if (bcmp(&cksum, &zilc->zc_next_blk.blk_cksum, + if (memcmp(&cksum, &zilc->zc_next_blk.blk_cksum, sizeof (cksum)) || BP_IS_HOLE(&zilc->zc_next_blk)) { error = SET_ERROR(ECKSUM); } else { ASSERT3U(len, <=, SPA_OLD_MAXBLOCKSIZE); - bcopy(lr, dst, len); + memcpy(dst, lr, len); *end = (char *)dst + len; *nbp = zilc->zc_next_blk; } @@ -273,14 +290,14 @@ zil_read_log_block(zilog_t *zilog, boolean_t decrypt, const blkptr_t *bp, uint64_t size = BP_GET_LSIZE(bp); zil_chain_t *zilc = (zil_chain_t *)(lr + size) - 1; - if (bcmp(&cksum, &zilc->zc_next_blk.blk_cksum, + if (memcmp(&cksum, &zilc->zc_next_blk.blk_cksum, sizeof (cksum)) || BP_IS_HOLE(&zilc->zc_next_blk) || (zilc->zc_nused > (size - sizeof (*zilc)))) { error = SET_ERROR(ECKSUM); } else { ASSERT3U(zilc->zc_nused, <=, SPA_OLD_MAXBLOCKSIZE); - bcopy(lr, dst, zilc->zc_nused); + memcpy(dst, lr, zilc->zc_nused); *end = (char *)dst + zilc->zc_nused; *nbp = zilc->zc_next_blk; } @@ -307,7 +324,7 @@ zil_read_log_data(zilog_t *zilog, const lr_write_t *lr, void *wbuf) if (BP_IS_HOLE(bp)) { if (wbuf != NULL) - bzero(wbuf, MAX(BP_GET_LSIZE(bp), lr->lr_length)); + memset(wbuf, 0, MAX(BP_GET_LSIZE(bp), lr->lr_length)); return (0); } @@ -330,13 +347,80 @@ zil_read_log_data(zilog_t *zilog, const lr_write_t *lr, void *wbuf) if (error == 0) { if (wbuf != NULL) - bcopy(abuf->b_data, wbuf, arc_buf_size(abuf)); + memcpy(wbuf, abuf->b_data, arc_buf_size(abuf)); arc_buf_destroy(abuf, &abuf); } return (error); } +void +zil_sums_init(zil_sums_t *zs) +{ + wmsum_init(&zs->zil_commit_count, 0); + wmsum_init(&zs->zil_commit_writer_count, 0); + wmsum_init(&zs->zil_itx_count, 0); + wmsum_init(&zs->zil_itx_indirect_count, 0); + wmsum_init(&zs->zil_itx_indirect_bytes, 0); + wmsum_init(&zs->zil_itx_copied_count, 0); + wmsum_init(&zs->zil_itx_copied_bytes, 0); + wmsum_init(&zs->zil_itx_needcopy_count, 0); + wmsum_init(&zs->zil_itx_needcopy_bytes, 0); + wmsum_init(&zs->zil_itx_metaslab_normal_count, 0); + wmsum_init(&zs->zil_itx_metaslab_normal_bytes, 0); + wmsum_init(&zs->zil_itx_metaslab_slog_count, 0); + wmsum_init(&zs->zil_itx_metaslab_slog_bytes, 0); +} + +void +zil_sums_fini(zil_sums_t *zs) +{ + wmsum_fini(&zs->zil_commit_count); + wmsum_fini(&zs->zil_commit_writer_count); + wmsum_fini(&zs->zil_itx_count); + wmsum_fini(&zs->zil_itx_indirect_count); + wmsum_fini(&zs->zil_itx_indirect_bytes); + wmsum_fini(&zs->zil_itx_copied_count); + wmsum_fini(&zs->zil_itx_copied_bytes); + wmsum_fini(&zs->zil_itx_needcopy_count); + wmsum_fini(&zs->zil_itx_needcopy_bytes); + wmsum_fini(&zs->zil_itx_metaslab_normal_count); + wmsum_fini(&zs->zil_itx_metaslab_normal_bytes); + wmsum_fini(&zs->zil_itx_metaslab_slog_count); + wmsum_fini(&zs->zil_itx_metaslab_slog_bytes); +} + +void +zil_kstat_values_update(zil_kstat_values_t *zs, zil_sums_t *zil_sums) +{ + zs->zil_commit_count.value.ui64 = + wmsum_value(&zil_sums->zil_commit_count); + zs->zil_commit_writer_count.value.ui64 = + wmsum_value(&zil_sums->zil_commit_writer_count); + zs->zil_itx_count.value.ui64 = + wmsum_value(&zil_sums->zil_itx_count); + zs->zil_itx_indirect_count.value.ui64 = + wmsum_value(&zil_sums->zil_itx_indirect_count); + zs->zil_itx_indirect_bytes.value.ui64 = + wmsum_value(&zil_sums->zil_itx_indirect_bytes); + zs->zil_itx_copied_count.value.ui64 = + wmsum_value(&zil_sums->zil_itx_copied_count); + zs->zil_itx_copied_bytes.value.ui64 = + wmsum_value(&zil_sums->zil_itx_copied_bytes); + zs->zil_itx_needcopy_count.value.ui64 = + wmsum_value(&zil_sums->zil_itx_needcopy_count); + zs->zil_itx_needcopy_bytes.value.ui64 = + wmsum_value(&zil_sums->zil_itx_needcopy_bytes); + zs->zil_itx_metaslab_normal_count.value.ui64 = + wmsum_value(&zil_sums->zil_itx_metaslab_normal_count); + zs->zil_itx_metaslab_normal_bytes.value.ui64 = + wmsum_value(&zil_sums->zil_itx_metaslab_normal_bytes); + zs->zil_itx_metaslab_slog_count.value.ui64 = + wmsum_value(&zil_sums->zil_itx_metaslab_slog_count); + zs->zil_itx_metaslab_slog_bytes.value.ui64 = + wmsum_value(&zil_sums->zil_itx_metaslab_slog_bytes); +} + /* * Parse the intent log, and call parse_func for each valid record within. */ @@ -353,12 +437,10 @@ zil_parse(zilog_t *zilog, zil_parse_blk_func_t *parse_blk_func, uint64_t max_lr_seq = 0; uint64_t blk_count = 0; uint64_t lr_count = 0; - blkptr_t blk, next_blk; + blkptr_t blk, next_blk = {{{{0}}}}; char *lrbuf, *lrp; int error = 0; - bzero(&next_blk, sizeof (blkptr_t)); - /* * Old logs didn't record the maximum zh_claim_lr_seq. */ @@ -561,8 +643,8 @@ zil_alloc_lwb(zilog_t *zilog, blkptr_t *bp, boolean_t slog, uint64_t txg, lwb->lwb_max_txg = txg; lwb->lwb_write_zio = NULL; lwb->lwb_root_zio = NULL; - lwb->lwb_tx = NULL; lwb->lwb_issued_timestamp = 0; + lwb->lwb_issued_txg = 0; if (BP_GET_CHECKSUM(bp) == ZIO_CHECKSUM_ZILOG2) { lwb->lwb_nused = sizeof (zil_chain_t); lwb->lwb_sz = BP_GET_LSIZE(bp); @@ -663,6 +745,38 @@ zilog_is_dirty(zilog_t *zilog) return (B_FALSE); } +/* + * Its called in zil_commit context (zil_process_commit_list()/zil_create()). + * It activates SPA_FEATURE_ZILSAXATTR feature, if its enabled. + * Check dsl_dataset_feature_is_active to avoid txg_wait_synced() on every + * zil_commit. + */ +static void +zil_commit_activate_saxattr_feature(zilog_t *zilog) +{ + dsl_dataset_t *ds = dmu_objset_ds(zilog->zl_os); + uint64_t txg = 0; + dmu_tx_t *tx = NULL; + + if (spa_feature_is_enabled(zilog->zl_spa, + SPA_FEATURE_ZILSAXATTR) && + dmu_objset_type(zilog->zl_os) != DMU_OST_ZVOL && + !dsl_dataset_feature_is_active(ds, + SPA_FEATURE_ZILSAXATTR)) { + tx = dmu_tx_create(zilog->zl_os); + VERIFY0(dmu_tx_assign(tx, TXG_WAIT)); + dsl_dataset_dirty(ds, tx); + txg = dmu_tx_get_txg(tx); + + mutex_enter(&ds->ds_lock); + ds->ds_feature_activation[SPA_FEATURE_ZILSAXATTR] = + (void *)B_TRUE; + mutex_exit(&ds->ds_lock); + dmu_tx_commit(tx); + txg_wait_synced(zilog->zl_dmu_pool, txg); + } +} + /* * Create an on-disk intent log. */ @@ -677,6 +791,8 @@ zil_create(zilog_t *zilog) int error = 0; boolean_t fastwrite = FALSE; boolean_t slog = FALSE; + dsl_dataset_t *ds = dmu_objset_ds(zilog->zl_os); + /* * Wait for any previous destroy to complete. @@ -724,11 +840,35 @@ zil_create(zilog_t *zilog) * (zh is part of the MOS, so we cannot modify it in open context.) */ if (tx != NULL) { + /* + * If "zilsaxattr" feature is enabled on zpool, then activate + * it now when we're creating the ZIL chain. We can't wait with + * this until we write the first xattr log record because we + * need to wait for the feature activation to sync out. + */ + if (spa_feature_is_enabled(zilog->zl_spa, + SPA_FEATURE_ZILSAXATTR) && dmu_objset_type(zilog->zl_os) != + DMU_OST_ZVOL) { + mutex_enter(&ds->ds_lock); + ds->ds_feature_activation[SPA_FEATURE_ZILSAXATTR] = + (void *)B_TRUE; + mutex_exit(&ds->ds_lock); + } + dmu_tx_commit(tx); txg_wait_synced(zilog->zl_dmu_pool, txg); + } else { + /* + * This branch covers the case where we enable the feature on a + * zpool that has existing ZIL headers. + */ + zil_commit_activate_saxattr_feature(zilog); } + IMPLY(spa_feature_is_enabled(zilog->zl_spa, SPA_FEATURE_ZILSAXATTR) && + dmu_objset_type(zilog->zl_os) != DMU_OST_ZVOL, + dsl_dataset_feature_is_active(ds, SPA_FEATURE_ZILSAXATTR)); - ASSERT(error != 0 || bcmp(&blk, &zh->zh_log, sizeof (blk)) == 0); + ASSERT(error != 0 || memcmp(&blk, &zh->zh_log, sizeof (blk)) == 0); IMPLY(error == 0, lwb != NULL); return (lwb); @@ -1127,9 +1267,9 @@ zil_lwb_flush_vdevs_done(zio_t *zio) { lwb_t *lwb = zio->io_private; zilog_t *zilog = lwb->lwb_zilog; - dmu_tx_t *tx = lwb->lwb_tx; zil_commit_waiter_t *zcw; itx_t *itx; + uint64_t txg; spa_config_exit(zilog->zl_spa, SCL_STATE, lwb); @@ -1138,15 +1278,13 @@ zil_lwb_flush_vdevs_done(zio_t *zio) mutex_enter(&zilog->zl_lock); /* - * Ensure the lwb buffer pointer is cleared before releasing the - * txg. If we have had an allocation failure and the txg is + * If we have had an allocation failure and the txg is * waiting to sync then we want zil_sync() to remove the lwb so * that it's not picked up as the next new one in * zil_process_commit_list(). zil_sync() will only remove the * lwb if lwb_buf is null. */ lwb->lwb_buf = NULL; - lwb->lwb_tx = NULL; ASSERT3U(lwb->lwb_issued_timestamp, >, 0); zilog->zl_last_lwb_latency = gethrtime() - lwb->lwb_issued_timestamp; @@ -1205,12 +1343,47 @@ zil_lwb_flush_vdevs_done(zio_t *zio) mutex_exit(&zilog->zl_lock); - /* - * Now that we've written this log block, we have a stable pointer - * to the next block in the chain, so it's OK to let the txg in - * which we allocated the next block sync. - */ - dmu_tx_commit(tx); + mutex_enter(&zilog->zl_lwb_io_lock); + txg = lwb->lwb_issued_txg; + ASSERT3U(zilog->zl_lwb_inflight[txg & TXG_MASK], >, 0); + zilog->zl_lwb_inflight[txg & TXG_MASK]--; + if (zilog->zl_lwb_inflight[txg & TXG_MASK] == 0) + cv_broadcast(&zilog->zl_lwb_io_cv); + mutex_exit(&zilog->zl_lwb_io_lock); +} + +/* + * Wait for the completion of all issued write/flush of that txg provided. + * It guarantees zil_lwb_flush_vdevs_done() is called and returned. + */ +static void +zil_lwb_flush_wait_all(zilog_t *zilog, uint64_t txg) +{ + ASSERT3U(txg, ==, spa_syncing_txg(zilog->zl_spa)); + + mutex_enter(&zilog->zl_lwb_io_lock); + while (zilog->zl_lwb_inflight[txg & TXG_MASK] > 0) + cv_wait(&zilog->zl_lwb_io_cv, &zilog->zl_lwb_io_lock); + mutex_exit(&zilog->zl_lwb_io_lock); + +#ifdef ZFS_DEBUG + mutex_enter(&zilog->zl_lock); + mutex_enter(&zilog->zl_lwb_io_lock); + lwb_t *lwb = list_head(&zilog->zl_lwb_list); + while (lwb != NULL && lwb->lwb_max_txg <= txg) { + if (lwb->lwb_issued_txg <= txg) { + ASSERT(lwb->lwb_state != LWB_STATE_ISSUED); + ASSERT(lwb->lwb_state != LWB_STATE_WRITE_DONE); + IMPLY(lwb->lwb_issued_txg > 0, + lwb->lwb_state == LWB_STATE_FLUSH_DONE); + } + IMPLY(lwb->lwb_state == LWB_STATE_FLUSH_DONE, + lwb->lwb_buf == NULL); + lwb = list_next(&zilog->zl_lwb_list, lwb); + } + mutex_exit(&zilog->zl_lwb_io_lock); + mutex_exit(&zilog->zl_lock); +#endif } /* @@ -1506,11 +1679,6 @@ zil_lwb_write_issue(zilog_t *zilog, lwb_t *lwb) /* * Allocate the next block and save its address in this block * before writing it in order to establish the log chain. - * Note that if the allocation of nlwb synced before we wrote - * the block that points at it (lwb), we'd leak it if we crashed. - * Therefore, we don't do dmu_tx_commit() until zil_lwb_write_done(). - * We dirty the dataset to ensure that zil_sync() will be called - * to clean up in the event of allocation failure or I/O failure. */ tx = dmu_tx_create(zilog->zl_os); @@ -1526,7 +1694,11 @@ zil_lwb_write_issue(zilog_t *zilog, lwb_t *lwb) dsl_dataset_dirty(dmu_objset_ds(zilog->zl_os), tx); txg = dmu_tx_get_txg(tx); - lwb->lwb_tx = tx; + mutex_enter(&zilog->zl_lwb_io_lock); + lwb->lwb_issued_txg = txg; + zilog->zl_lwb_inflight[txg & TXG_MASK]++; + zilog->zl_lwb_max_issued_txg = MAX(txg, zilog->zl_lwb_max_issued_txg); + mutex_exit(&zilog->zl_lwb_io_lock); /* * Log blocks are pre-allocated. Here we select the size of the next @@ -1556,11 +1728,13 @@ zil_lwb_write_issue(zilog_t *zilog, lwb_t *lwb) BP_ZERO(bp); error = zio_alloc_zil(spa, zilog->zl_os, txg, bp, zil_blksz, &slog); if (slog) { - ZIL_STAT_BUMP(zil_itx_metaslab_slog_count); - ZIL_STAT_INCR(zil_itx_metaslab_slog_bytes, lwb->lwb_nused); + ZIL_STAT_BUMP(zilog, zil_itx_metaslab_slog_count); + ZIL_STAT_INCR(zilog, zil_itx_metaslab_slog_bytes, + lwb->lwb_nused); } else { - ZIL_STAT_BUMP(zil_itx_metaslab_normal_count); - ZIL_STAT_INCR(zil_itx_metaslab_normal_bytes, lwb->lwb_nused); + ZIL_STAT_BUMP(zilog, zil_itx_metaslab_normal_count); + ZIL_STAT_INCR(zilog, zil_itx_metaslab_normal_bytes, + lwb->lwb_nused); } if (error == 0) { ASSERT3U(bp->blk_birth, ==, txg); @@ -1590,7 +1764,7 @@ zil_lwb_write_issue(zilog_t *zilog, lwb_t *lwb) /* * clear unused data for security */ - bzero(lwb->lwb_buf + lwb->lwb_nused, wsz - lwb->lwb_nused); + memset(lwb->lwb_buf + lwb->lwb_nused, 0, wsz - lwb->lwb_nused); spa_config_enter(zilog->zl_spa, SCL_STATE, lwb, RW_READER); @@ -1601,6 +1775,8 @@ zil_lwb_write_issue(zilog_t *zilog, lwb_t *lwb) zio_nowait(lwb->lwb_root_zio); zio_nowait(lwb->lwb_write_zio); + dmu_tx_commit(tx); + /* * If there was an allocation failure then nlwb will be null which * forces a txg_wait_synced(). @@ -1724,11 +1900,11 @@ zil_lwb_commit(zilog_t *zilog, itx_t *itx, lwb_t *lwb) dnow = MIN(dlen, lwb_sp - reclen); lr_buf = lwb->lwb_buf + lwb->lwb_nused; - bcopy(lrc, lr_buf, reclen); + memcpy(lr_buf, lrc, reclen); lrcb = (lr_t *)lr_buf; /* Like lrc, but inside lwb. */ lrwb = (lr_write_t *)lrcb; /* Like lrw, but inside lwb. */ - ZIL_STAT_BUMP(zil_itx_count); + ZIL_STAT_BUMP(zilog, zil_itx_count); /* * If it's a write, fetch the data or get its blkptr as appropriate. @@ -1737,8 +1913,9 @@ zil_lwb_commit(zilog_t *zilog, itx_t *itx, lwb_t *lwb) if (txg > spa_freeze_txg(zilog->zl_spa)) txg_wait_synced(zilog->zl_dmu_pool, txg); if (itx->itx_wr_state == WR_COPIED) { - ZIL_STAT_BUMP(zil_itx_copied_count); - ZIL_STAT_INCR(zil_itx_copied_bytes, lrw->lr_length); + ZIL_STAT_BUMP(zilog, zil_itx_copied_count); + ZIL_STAT_INCR(zilog, zil_itx_copied_bytes, + lrw->lr_length); } else { char *dbuf; int error; @@ -1750,13 +1927,14 @@ zil_lwb_commit(zilog_t *zilog, itx_t *itx, lwb_t *lwb) lrwb->lr_length = dnow; lrw->lr_offset += dnow; lrw->lr_length -= dnow; - ZIL_STAT_BUMP(zil_itx_needcopy_count); - ZIL_STAT_INCR(zil_itx_needcopy_bytes, dnow); + ZIL_STAT_BUMP(zilog, zil_itx_needcopy_count); + ZIL_STAT_INCR(zilog, zil_itx_needcopy_bytes, + dnow); } else { ASSERT3S(itx->itx_wr_state, ==, WR_INDIRECT); dbuf = NULL; - ZIL_STAT_BUMP(zil_itx_indirect_count); - ZIL_STAT_INCR(zil_itx_indirect_bytes, + ZIL_STAT_BUMP(zilog, zil_itx_indirect_count); + ZIL_STAT_INCR(zilog, zil_itx_indirect_bytes, lrw->lr_length); } @@ -1780,7 +1958,7 @@ zil_lwb_commit(zilog_t *zilog, itx_t *itx, lwb_t *lwb) lwb->lwb_write_zio); if (dbuf != NULL && error == 0 && dnow == dlen) /* Zero any padding bytes in the last block. */ - bzero((char *)dbuf + lrwb->lr_length, dpad); + memset((char *)dbuf + lrwb->lr_length, 0, dpad); if (error == EIO) { txg_wait_synced(zilog->zl_dmu_pool, txg); @@ -1830,7 +2008,7 @@ zil_itx_create(uint64_t txtype, size_t olrsize) itx->itx_lr.lrc_txtype = txtype; itx->itx_lr.lrc_reclen = lrsize; itx->itx_lr.lrc_seq = 0; /* defensive */ - bzero((char *)&itx->itx_lr + olrsize, lrsize - olrsize); + memset((char *)&itx->itx_lr + olrsize, 0, lrsize - olrsize); itx->itx_sync = B_TRUE; /* default is synchronous */ itx->itx_callback = NULL; itx->itx_callback_data = NULL; @@ -2297,6 +2475,11 @@ zil_process_commit_list(zilog_t *zilog) if (lwb == NULL) { lwb = zil_create(zilog); } else { + /* + * Activate SPA_FEATURE_ZILSAXATTR for the cases where ZIL will + * have already been created (zl_lwb_list not empty). + */ + zil_commit_activate_saxattr_feature(zilog); ASSERT3S(lwb->lwb_state, !=, LWB_STATE_ISSUED); ASSERT3S(lwb->lwb_state, !=, LWB_STATE_WRITE_DONE); ASSERT3S(lwb->lwb_state, !=, LWB_STATE_FLUSH_DONE); @@ -2516,7 +2699,7 @@ zil_commit_writer(zilog_t *zilog, zil_commit_waiter_t *zcw) goto out; } - ZIL_STAT_BUMP(zil_commit_writer_count); + ZIL_STAT_BUMP(zilog, zil_commit_writer_count); zil_get_commit_list(zilog); zil_prune_commit_list(zilog); @@ -2870,7 +3053,7 @@ zil_commit_itx_assign(zilog_t *zilog, zil_commit_waiter_t *zcw) * queue prior to zil_commit() having been called, and which itxs were * added after zil_commit() was called. * - * The commit it is special; it doesn't have any on-disk representation. + * The commit itx is special; it doesn't have any on-disk representation. * When a commit itx is "committed" to an lwb, the waiter associated * with it is linked onto the lwb's list of waiters. Then, when that lwb * completes, each waiter on the lwb's list is marked done and signaled @@ -2993,7 +3176,7 @@ zil_commit(zilog_t *zilog, uint64_t foid) void zil_commit_impl(zilog_t *zilog, uint64_t foid) { - ZIL_STAT_BUMP(zil_commit_count); + ZIL_STAT_BUMP(zilog, zil_commit_count); /* * Move the "async" itxs for the specified foid to the "sync" @@ -3063,6 +3246,8 @@ zil_sync(zilog_t *zilog, dmu_tx_t *tx) if (spa_sync_pass(spa) != 1) return; + zil_lwb_flush_wait_all(zilog, txg); + mutex_enter(&zilog->zl_lock); ASSERT(zilog->zl_stop_sync == 0); @@ -3075,11 +3260,13 @@ zil_sync(zilog_t *zilog, dmu_tx_t *tx) if (zilog->zl_destroy_txg == txg) { blkptr_t blk = zh->zh_log; + dsl_dataset_t *ds = dmu_objset_ds(zilog->zl_os); ASSERT(list_head(&zilog->zl_lwb_list) == NULL); - bzero(zh, sizeof (zil_header_t)); - bzero(zilog->zl_replayed_seq, sizeof (zilog->zl_replayed_seq)); + memset(zh, 0, sizeof (zil_header_t)); + memset(zilog->zl_replayed_seq, 0, + sizeof (zilog->zl_replayed_seq)); if (zilog->zl_keep_first) { /* @@ -3092,6 +3279,16 @@ zil_sync(zilog_t *zilog, dmu_tx_t *tx) */ zil_init_log_chain(zilog, &blk); zh->zh_log = blk; + } else { + /* + * A destroyed ZIL chain can't contain any TX_SETSAXATTR + * records. So, deactivate the feature for this dataset. + * We activate it again when we start a new ZIL chain. + */ + if (dsl_dataset_feature_is_active(ds, + SPA_FEATURE_ZILSAXATTR)) + dsl_dataset_deactivate_feature(ds, + SPA_FEATURE_ZILSAXATTR, tx); } } @@ -3162,13 +3359,16 @@ zil_init(void) zil_zcw_cache = kmem_cache_create("zil_zcw_cache", sizeof (zil_commit_waiter_t), 0, NULL, NULL, NULL, NULL, NULL, 0); - zil_ksp = kstat_create("zfs", 0, "zil", "misc", + zil_sums_init(&zil_sums_global); + zil_kstats_global = kstat_create("zfs", 0, "zil", "misc", KSTAT_TYPE_NAMED, sizeof (zil_stats) / sizeof (kstat_named_t), KSTAT_FLAG_VIRTUAL); - if (zil_ksp != NULL) { - zil_ksp->ks_data = &zil_stats; - kstat_install(zil_ksp); + if (zil_kstats_global != NULL) { + zil_kstats_global->ks_data = &zil_stats; + zil_kstats_global->ks_update = zil_kstats_global_update; + zil_kstats_global->ks_private = NULL; + kstat_install(zil_kstats_global); } } @@ -3178,10 +3378,12 @@ zil_fini(void) kmem_cache_destroy(zil_zcw_cache); kmem_cache_destroy(zil_lwb_cache); - if (zil_ksp != NULL) { - kstat_delete(zil_ksp); - zil_ksp = NULL; + if (zil_kstats_global != NULL) { + kstat_delete(zil_kstats_global); + zil_kstats_global = NULL; } + + zil_sums_fini(&zil_sums_global); } void @@ -3217,6 +3419,7 @@ zil_alloc(objset_t *os, zil_header_t *zh_phys) mutex_init(&zilog->zl_lock, NULL, MUTEX_DEFAULT, NULL); mutex_init(&zilog->zl_issuer_lock, NULL, MUTEX_DEFAULT, NULL); + mutex_init(&zilog->zl_lwb_io_lock, NULL, MUTEX_DEFAULT, NULL); for (int i = 0; i < TXG_SIZE; i++) { mutex_init(&zilog->zl_itxg[i].itxg_lock, NULL, @@ -3230,6 +3433,7 @@ zil_alloc(objset_t *os, zil_header_t *zh_phys) offsetof(itx_t, itx_node)); cv_init(&zilog->zl_cv_suspend, NULL, CV_DEFAULT, NULL); + cv_init(&zilog->zl_lwb_io_cv, NULL, CV_DEFAULT, NULL); return (zilog); } @@ -3265,8 +3469,10 @@ zil_free(zilog_t *zilog) mutex_destroy(&zilog->zl_issuer_lock); mutex_destroy(&zilog->zl_lock); + mutex_destroy(&zilog->zl_lwb_io_lock); cv_destroy(&zilog->zl_cv_suspend); + cv_destroy(&zilog->zl_lwb_io_cv); kmem_free(zilog, sizeof (zilog_t)); } @@ -3275,7 +3481,7 @@ zil_free(zilog_t *zilog) * Open an intent log. */ zilog_t * -zil_open(objset_t *os, zil_get_data_t *get_data) +zil_open(objset_t *os, zil_get_data_t *get_data, zil_sums_t *zil_sums) { zilog_t *zilog = dmu_objset_zil(os); @@ -3284,6 +3490,7 @@ zil_open(objset_t *os, zil_get_data_t *get_data) ASSERT(list_is_empty(&zilog->zl_lwb_list)); zilog->zl_get_data = get_data; + zilog->zl_sums = zil_sums; return (zilog); } @@ -3314,9 +3521,18 @@ zil_close(zilog_t *zilog) mutex_exit(&zilog->zl_lock); /* - * We need to use txg_wait_synced() to wait long enough for the - * ZIL to be clean, and to wait for all pending lwbs to be - * written out. + * zl_lwb_max_issued_txg may be larger than lwb_max_txg. It depends + * on the time when the dmu_tx transaction is assigned in + * zil_lwb_write_issue(). + */ + mutex_enter(&zilog->zl_lwb_io_lock); + txg = MAX(zilog->zl_lwb_max_issued_txg, txg); + mutex_exit(&zilog->zl_lwb_io_lock); + + /* + * We need to use txg_wait_synced() to wait until that txg is synced. + * zil_sync() will guarantee all lwbs up to that txg have been + * written out, flushed, and cleaned. */ if (txg != 0) txg_wait_synced(zilog->zl_dmu_pool, txg); @@ -3348,7 +3564,7 @@ zil_close(zilog_t *zilog) mutex_exit(&zilog->zl_lock); } -static char *suspend_tag = "zil suspending"; +static const char *suspend_tag = "zil suspending"; /* * Suspend an intent log. While in suspended mode, we still honor @@ -3571,7 +3787,7 @@ zil_replay_log_record(zilog_t *zilog, const lr_t *lr, void *zra, /* * Make a copy of the data so we can revise and extend it. */ - bcopy(lr, zr->zr_lr, reclen); + memcpy(zr->zr_lr, lr, reclen); /* * If this is a TX_WRITE with a blkptr, suck in the data. @@ -3716,6 +3932,9 @@ EXPORT_SYMBOL(zil_lwb_add_block); EXPORT_SYMBOL(zil_bp_tree_add); EXPORT_SYMBOL(zil_set_sync); EXPORT_SYMBOL(zil_set_logbias); +EXPORT_SYMBOL(zil_sums_init); +EXPORT_SYMBOL(zil_sums_fini); +EXPORT_SYMBOL(zil_kstat_values_update); ZFS_MODULE_PARAM(zfs, zfs_, commit_timeout_pct, INT, ZMOD_RW, "ZIL block open timeout percentage"); diff --git a/module/zfs/zio.c b/module/zfs/zio.c index b0d5c6885b94..7b55450ca906 100644 --- a/module/zfs/zio.c +++ b/module/zfs/zio.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -166,15 +166,6 @@ zio_init(void) cflags = (zio_exclude_metadata || size > zio_buf_debug_limit) ? KMC_NODEBUG : 0; -#if defined(_ILP32) && defined(_KERNEL) - /* - * Cache size limited to 1M on 32-bit platforms until ARC - * buffers no longer require virtual address space. - */ - if (size > zfs_max_recordsize) - break; -#endif - while (!ISP2(p2)) p2 &= p2 - 1; @@ -822,7 +813,7 @@ zio_create(zio_t *pio, spa_t *spa, uint64_t txg, const blkptr_t *bp, IMPLY(lsize != psize, (flags & ZIO_FLAG_RAW_COMPRESS) != 0); zio = kmem_cache_alloc(zio_cache, KM_SLEEP); - bzero(zio, sizeof (zio_t)); + memset(zio, 0, sizeof (zio_t)); mutex_init(&zio->io_lock, NULL, MUTEX_NOLOCKDEP, NULL); cv_init(&zio->io_cv, NULL, CV_DEFAULT, NULL); @@ -891,7 +882,7 @@ zio_create(zio_t *pio, spa_t *spa, uint64_t txg, const blkptr_t *bp, return (zio); } -static void +void zio_destroy(zio_t *zio) { metaslab_trace_fini(&zio->io_alloc_list); @@ -971,14 +962,12 @@ zfs_blkptr_verify(spa_t *spa, const blkptr_t *bp, boolean_t config_held, "blkptr at %p has invalid TYPE %llu", bp, (longlong_t)BP_GET_TYPE(bp)); } - if (BP_GET_CHECKSUM(bp) >= ZIO_CHECKSUM_FUNCTIONS || - BP_GET_CHECKSUM(bp) <= ZIO_CHECKSUM_ON) { + if (BP_GET_CHECKSUM(bp) >= ZIO_CHECKSUM_FUNCTIONS) { errors += zfs_blkptr_verify_log(spa, bp, blk_verify, "blkptr at %p has invalid CHECKSUM %llu", bp, (longlong_t)BP_GET_CHECKSUM(bp)); } - if (BP_GET_COMPRESS(bp) >= ZIO_COMPRESS_FUNCTIONS || - BP_GET_COMPRESS(bp) <= ZIO_COMPRESS_ON) { + if (BP_GET_COMPRESS(bp) >= ZIO_COMPRESS_FUNCTIONS) { errors += zfs_blkptr_verify_log(spa, bp, blk_verify, "blkptr at %p has invalid COMPRESS %llu", bp, (longlong_t)BP_GET_COMPRESS(bp)); @@ -1773,7 +1762,13 @@ zio_write_compress(zio_t *zio) zio->io_abd, NULL, lsize, zp->zp_complevel); if (psize == 0 || psize >= lsize) compress = ZIO_COMPRESS_OFF; - } else if (zio->io_flags & ZIO_FLAG_RAW_COMPRESS) { + } else if (zio->io_flags & ZIO_FLAG_RAW_COMPRESS && + !(zio->io_flags & ZIO_FLAG_RAW_ENCRYPT)) { + /* + * If we are raw receiving an encrypted dataset we should not + * take this codepath because it will change the on-disk block + * and decryption will fail. + */ size_t rounded = MIN((size_t)roundup(psize, spa->spa_min_alloc), lsize); @@ -2067,7 +2062,7 @@ zio_deadman_impl(zio_t *pio, int ziodepth) * using the zfs_dbgmsg() interface then post deadman event for the ZED. */ void -zio_deadman(zio_t *pio, char *tag) +zio_deadman(zio_t *pio, const char *tag) { spa_t *spa = pio->io_spa; char *name = spa_name(spa); @@ -2877,7 +2872,7 @@ zio_write_gang_block(zio_t *pio, metaslab_class_t *mc) gn = zio_gang_node_alloc(gnpp); gbh = gn->gn_gbh; - bzero(gbh, SPA_GANGBLOCKSIZE); + memset(gbh, 0, SPA_GANGBLOCKSIZE); gbh_abd = abd_get_from_buf(gbh, SPA_GANGBLOCKSIZE); /* @@ -2906,9 +2901,9 @@ zio_write_gang_block(zio_t *pio, metaslab_class_t *mc) zp.zp_nopwrite = B_FALSE; zp.zp_encrypt = gio->io_prop.zp_encrypt; zp.zp_byteorder = gio->io_prop.zp_byteorder; - bzero(zp.zp_salt, ZIO_DATA_SALT_LEN); - bzero(zp.zp_iv, ZIO_DATA_IV_LEN); - bzero(zp.zp_mac, ZIO_DATA_MAC_LEN); + memset(zp.zp_salt, 0, ZIO_DATA_SALT_LEN); + memset(zp.zp_iv, 0, ZIO_DATA_IV_LEN); + memset(zp.zp_mac, 0, ZIO_DATA_MAC_LEN); zio_t *cio = zio_write(zio, spa, txg, &gbh->zg_blkptr[g], has_data ? abd_get_offset(pio->io_abd, pio->io_size - @@ -3005,7 +3000,7 @@ zio_nop_write(zio_t *zio) ASSERT3U(BP_GET_PSIZE(bp), ==, BP_GET_PSIZE(bp_orig)); ASSERT3U(BP_GET_LSIZE(bp), ==, BP_GET_LSIZE(bp_orig)); ASSERT(zp->zp_compress != ZIO_COMPRESS_OFF); - ASSERT(bcmp(&bp->blk_prop, &bp_orig->blk_prop, + ASSERT(memcmp(&bp->blk_prop, &bp_orig->blk_prop, sizeof (uint64_t)) == 0); /* @@ -4555,7 +4550,7 @@ zio_done(zio_t *zio) if (zio->io_bp != NULL && !BP_IS_EMBEDDED(zio->io_bp)) { ASSERT(zio->io_bp->blk_pad[0] == 0); ASSERT(zio->io_bp->blk_pad[1] == 0); - ASSERT(bcmp(zio->io_bp, &zio->io_bp_copy, + ASSERT(memcmp(zio->io_bp, &zio->io_bp_copy, sizeof (blkptr_t)) == 0 || (zio->io_bp == zio_unique_parent(zio)->io_bp)); if (zio->io_type == ZIO_TYPE_WRITE && !BP_IS_HOLE(zio->io_bp) && @@ -5002,7 +4997,7 @@ zbookmark_subtree_completed(const dnode_phys_t *dnp, { zbookmark_phys_t mod_zb = *subtree_root; mod_zb.zb_blkid++; - ASSERT(last_block->zb_level == 0); + ASSERT0(last_block->zb_level); /* The objset_phys_t isn't before anything. */ if (dnp == NULL) @@ -5028,6 +5023,22 @@ zbookmark_subtree_completed(const dnode_phys_t *dnp, last_block) <= 0); } +/* + * This function is similar to zbookmark_subtree_completed(), but returns true + * if subtree_root is equal or ahead of last_block, i.e. still to be done. + */ +boolean_t +zbookmark_subtree_tbd(const dnode_phys_t *dnp, + const zbookmark_phys_t *subtree_root, const zbookmark_phys_t *last_block) +{ + ASSERT0(last_block->zb_level); + if (dnp == NULL) + return (B_FALSE); + return (zbookmark_compare(dnp->dn_datablkszsec, dnp->dn_indblkshift, + 1ULL << (DNODE_BLOCK_SHIFT - SPA_MINBLOCKSHIFT), 0, subtree_root, + last_block) >= 0); +} + EXPORT_SYMBOL(zio_type_name); EXPORT_SYMBOL(zio_buf_alloc); EXPORT_SYMBOL(zio_data_buf_alloc); diff --git a/module/zfs/zio_checksum.c b/module/zfs/zio_checksum.c index 4dbab68dd7aa..b3c5fbbd8bba 100644 --- a/module/zfs/zio_checksum.c +++ b/module/zfs/zio_checksum.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -160,7 +160,7 @@ abd_fletcher_4_byteswap(abd_t *abd, uint64_t size, abd_fletcher_4_impl(abd, size, &acd); } -zio_checksum_info_t zio_checksum_table[ZIO_CHECKSUM_FUNCTIONS] = { +const zio_checksum_info_t zio_checksum_table[ZIO_CHECKSUM_FUNCTIONS] = { {{NULL, NULL}, NULL, NULL, 0, "inherit"}, {{NULL, NULL}, NULL, NULL, 0, "on"}, {{abd_checksum_off, abd_checksum_off}, @@ -195,6 +195,10 @@ zio_checksum_info_t zio_checksum_table[ZIO_CHECKSUM_FUNCTIONS] = { abd_checksum_edonr_tmpl_init, abd_checksum_edonr_tmpl_free, ZCHECKSUM_FLAG_METADATA | ZCHECKSUM_FLAG_SALTED | ZCHECKSUM_FLAG_NOPWRITE, "edonr"}, + {{abd_checksum_blake3_native, abd_checksum_blake3_byteswap}, + abd_checksum_blake3_tmpl_init, abd_checksum_blake3_tmpl_free, + ZCHECKSUM_FLAG_METADATA | ZCHECKSUM_FLAG_DEDUP | + ZCHECKSUM_FLAG_SALTED | ZCHECKSUM_FLAG_NOPWRITE, "blake3"}, }; /* @@ -207,6 +211,8 @@ zio_checksum_to_feature(enum zio_checksum cksum) VERIFY((cksum & ~ZIO_CHECKSUM_MASK) == 0); switch (cksum) { + case ZIO_CHECKSUM_BLAKE3: + return (SPA_FEATURE_BLAKE3); case ZIO_CHECKSUM_SHA512: return (SPA_FEATURE_SHA512); case ZIO_CHECKSUM_SKEIN: @@ -351,7 +357,7 @@ zio_checksum_compute(zio_t *zio, enum zio_checksum checksum, zio_eck_t eck; size_t eck_offset; - bzero(&saved, sizeof (zio_cksum_t)); + memset(&saved, 0, sizeof (zio_cksum_t)); if (checksum == ZIO_CHECKSUM_ZILOG2) { zil_chain_t zilc; diff --git a/module/zfs/zio_compress.c b/module/zfs/zio_compress.c index cded11f4cbd5..6527e33ca37d 100644 --- a/module/zfs/zio_compress.c +++ b/module/zfs/zio_compress.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -49,7 +49,7 @@ unsigned long zio_decompress_fail_fraction = 0; /* * Compression vectors. */ -zio_compress_info_t zio_compress_table[ZIO_COMPRESS_FUNCTIONS] = { +const zio_compress_info_t zio_compress_table[ZIO_COMPRESS_FUNCTIONS] = { {"inherit", 0, NULL, NULL, NULL}, {"on", 0, NULL, NULL, NULL}, {"uncompressed", 0, NULL, NULL, NULL}, @@ -66,7 +66,7 @@ zio_compress_info_t zio_compress_table[ZIO_COMPRESS_FUNCTIONS] = { {"gzip-9", 9, gzip_compress, gzip_decompress, NULL}, {"zle", 64, zle_compress, zle_decompress, NULL}, {"lz4", 0, lz4_compress_zfs, lz4_decompress_zfs, NULL}, - {"zstd", ZIO_ZSTD_LEVEL_DEFAULT, zfs_zstd_compress, + {"zstd", ZIO_ZSTD_LEVEL_DEFAULT, zfs_zstd_compress_wrap, zfs_zstd_decompress, zfs_zstd_decompress_level}, }; diff --git a/module/zfs/zio_inject.c b/module/zfs/zio_inject.c index feaf41dc65e3..4f7cb8430d3e 100644 --- a/module/zfs/zio_inject.c +++ b/module/zfs/zio_inject.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -148,7 +148,8 @@ zio_match_handler(const zbookmark_phys_t *zb, uint64_t type, int dva, zb->zb_level == record->zi_level && zb->zb_blkid >= record->zi_start && zb->zb_blkid <= record->zi_end && - (record->zi_dvas == 0 || (record->zi_dvas & (1ULL << dva))) && + (record->zi_dvas == 0 || + (dva != ZI_NO_DVA && (record->zi_dvas & (1ULL << dva)))) && error == record->zi_error) { return (freq_triggered(record->zi_freq)); } @@ -161,7 +162,7 @@ zio_match_handler(const zbookmark_phys_t *zb, uint64_t type, int dva, * specified by tag. */ void -zio_handle_panic_injection(spa_t *spa, char *tag, uint64_t type) +zio_handle_panic_injection(spa_t *spa, const char *tag, uint64_t type) { inject_handler_t *handler; @@ -341,15 +342,14 @@ zio_handle_label_injection(zio_t *zio, int error) return (ret); } -/*ARGSUSED*/ static int zio_inject_bitflip_cb(void *data, size_t len, void *private) { - zio_t *zio __maybe_unused = private; + zio_t *zio = private; uint8_t *buffer = data; uint_t byte = random_in_range(len); - ASSERT(zio->io_type == ZIO_TYPE_READ); + ASSERT3U(zio->io_type, ==, ZIO_TYPE_READ); /* flip a single random bit in an abd data buffer */ buffer[byte] ^= 1 << random_in_range(8); diff --git a/module/zfs/zle.c b/module/zfs/zle.c index 0decebb13ca7..1483a65af803 100644 --- a/module/zfs/zle.c +++ b/module/zfs/zle.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/zrlock.c b/module/zfs/zrlock.c index a4def6053622..3de2bde4cfeb 100644 --- a/module/zfs/zrlock.c +++ b/module/zfs/zrlock.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/module/zfs/zthr.c b/module/zfs/zthr.c index 52ddffae7aaa..02b9f0805dd7 100644 --- a/module/zfs/zthr.c +++ b/module/zfs/zthr.c @@ -231,7 +231,7 @@ struct zthr { const char *zthr_name; }; -static void +static __attribute__((noreturn)) void zthr_procedure(void *arg) { zthr_t *t = arg; diff --git a/module/zfs/zvol.c b/module/zfs/zvol.c index 943be01d47a6..3a1870f568b7 100644 --- a/module/zfs/zvol.c +++ b/module/zfs/zvol.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -513,6 +513,7 @@ zil_replay_func_t *const zvol_replay_vector[TX_MAX_TYPE] = { zvol_replay_err, /* TX_MKDIR_ATTR */ zvol_replay_err, /* TX_MKDIR_ACL_ATTR */ zvol_replay_err, /* TX_WRITE2 */ + zvol_replay_err, /* TX_SETSAXATTR */ }; /* @@ -613,10 +614,10 @@ zvol_log_truncate(zvol_state_t *zv, dmu_tx_t *tx, uint64_t off, uint64_t len, } -/* ARGSUSED */ static void zvol_get_done(zgd_t *zgd, int error) { + (void) error; if (zgd->zgd_db) dmu_buf_rele(zgd->zgd_db, zgd); @@ -1511,10 +1512,10 @@ zvol_set_snapdev_check(void *arg, dmu_tx_t *tx) return (error); } -/* ARGSUSED */ static int zvol_set_snapdev_sync_cb(dsl_pool_t *dp, dsl_dataset_t *ds, void *arg) { + (void) arg; char dsname[MAXNAMELEN]; zvol_task_t *task; uint64_t snapdev; @@ -1597,10 +1598,10 @@ zvol_set_volmode_check(void *arg, dmu_tx_t *tx) return (error); } -/* ARGSUSED */ static int zvol_set_volmode_sync_cb(dsl_pool_t *dp, dsl_dataset_t *ds, void *arg) { + (void) arg; char dsname[MAXNAMELEN]; zvol_task_t *task; uint64_t volmode; diff --git a/module/zstd/Makefile.in b/module/zstd/Makefile.in deleted file mode 100644 index 091f7cea3639..000000000000 --- a/module/zstd/Makefile.in +++ /dev/null @@ -1,39 +0,0 @@ -ifneq ($(KBUILD_EXTMOD),) -src = @abs_srcdir@ -obj = @abs_builddir@ -zstd_include = $(src)/include -else -zstd_include = $(srctree)/$(src)/include -endif - -MODULE := zzstd - -obj-$(CONFIG_ZFS) := $(MODULE).o - -asflags-y := -I$(zstd_include) -ccflags-y := -I$(zstd_include) - -# Zstd uses -O3 by default, so we should follow -ccflags-y += -O3 - -# -fno-tree-vectorize gets set for gcc in zstd/common/compiler.h -# Set it for other compilers, too. -$(obj)/lib/zstd.o: c_flags += -fno-tree-vectorize - -# SSE register return with SSE disabled if -march=znverX is passed -$(obj)/lib/zstd.o: c_flags += -U__BMI__ - -# Quiet warnings about frame size due to unused code in unmodified zstd lib -$(obj)/lib/zstd.o: c_flags += -Wframe-larger-than=20480 - -# Disable aarch64 neon SIMD instructions for kernel mode -$(obj)/lib/zstd.o: c_flags += -include $(zstd_include)/aarch64_compat.h -include $(zstd_include)/zstd_compat_wrapper.h -Wp,-w - -$(obj)/zfs_zstd.o: c_flags += -include $(zstd_include)/zstd_compat_wrapper.h - -$(MODULE)-objs += zfs_zstd.o -$(MODULE)-objs += lib/zstd.o -$(MODULE)-objs += zstd_sparc.o - -all: - mkdir -p lib diff --git a/module/zstd/README.md b/module/zstd/README.md index eed229e2f78f..7ad00e0bd804 100644 --- a/module/zstd/README.md +++ b/module/zstd/README.md @@ -9,10 +9,9 @@ library, besides upgrading to a newer ZSTD release. Tree structure: -* `zfs_zstd.c` is the actual `zzstd` kernel module. -* `lib/` contains the unmodified, [_"amalgamated"_](https://github.com/facebook/zstd/blob/dev/contrib/single_file_libs/README.md) - version of the `Zstandard` library, generated from our template file -* `zstd-in.c` is our template file for generating the library +* `zfs_zstd.c` are the actual `zfs` kernel module hooks. +* `lib/` contains the unmodified version of the `Zstandard` library +* `zstd-in.c` is our template file for generating the single-file library * `include/`: This directory contains supplemental includes for platform compatibility, which are not expected to be used by ZFS elsewhere in the future. Thus we keep them private to ZSTD. @@ -22,37 +21,11 @@ Tree structure: To update ZSTD the following steps need to be taken: 1. Grab the latest release of [ZSTD](https://github.com/facebook/zstd/releases). -2. Update `module/zstd/zstd-in.c` if required. (see - `zstd/contrib/single_file_libs/zstd-in.c` in the zstd repository) -3. Generate the "single-file-library" and put it to `module/zstd/lib/`. -4. Copy the following files to `module/zstd/lib/`: - - `zstd/lib/zstd.h` - - `zstd/lib/common/zstd_errors.h` - -This can be done using a few shell commands from inside the zfs repo: - -~~~sh -cd PATH/TO/ZFS - -url="https://github.com/facebook/zstd" -release="$(curl -s "${url}"/releases/latest | grep -oP '(?<=v)[\d\.]+')" -zstd="/tmp/zstd-${release}/" - -wget -O /tmp/zstd.tar.gz \ - "${url}/releases/download/v${release}/zstd-${release}.tar.gz" -tar -C /tmp -xzf /tmp/zstd.tar.gz - -cp ${zstd}/lib/zstd.h module/zstd/lib/ -cp ${zstd}/lib/zstd_errors.h module/zstd/lib/ -${zstd}/contrib/single_file_libs/combine.sh \ - -r ${zstd}/lib -o module/zstd/lib/zstd.c module/zstd/zstd-in.c -~~~ - -Note: if the zstd library for zfs is updated to a newer version, -the macro list in include/zstd_compat_wrapper.h usually needs to be updated. -this can be done with some hand crafting of the output of the following -script: nm zstd.o | awk '{print "#define "$3 " zfs_" $3}' > macrotable - +2. Copy the files output by the following script to `module/zstd/lib/`: +`grep include [path to zstd]/contrib/single_file_libs/zstd-in.c | awk '{ print $2 }'` +3. Remove debug.c, threading.c, and zstdmt_compress.c. +4. Update Makefiles with resulting file lists. +5. Follow symbol renaming notes in `include/zstd_compat_wrapper.h` ## Altering ZSTD and breaking changes diff --git a/module/zstd/include/string.h b/module/zstd/include/string.h index 78998d3c4655..7474e7f1af0f 100644 --- a/module/zstd/include/string.h +++ b/module/zstd/include/string.h @@ -44,6 +44,7 @@ extern "C" { #ifdef _KERNEL #if defined(__FreeBSD__) +#include /* u_int, u_char */ #include /* memcpy, memset */ #elif defined(__linux__) #include /* memcpy, memset */ diff --git a/module/zstd/include/zstd_compat_wrapper.h b/module/zstd/include/zstd_compat_wrapper.h index 339713590f96..2c4baad27d4e 100644 --- a/module/zstd/include/zstd_compat_wrapper.h +++ b/module/zstd/include/zstd_compat_wrapper.h @@ -36,426 +36,389 @@ * This wrapper fixes a problem, in case the ZFS filesystem driver, is compiled * statically into the kernel. * This will cause a symbol collision with the older in-kernel zstd library. - * The following macros will simply rename all local zstd symbols and references * - * Note: if the zstd library for zfs is updated to a newer version, this macro - * list usually needs to be updated. - * this can be done with some hand crafting of the output of the following - * script - * nm zstd.o | awk '{print "#define "$3 " zfs_" $3}' > macrotable + * On update, truncate this file at the scissor line, rebuild the module, + * and make gen-zstd-symbols. */ -#define BIT_initDStream zfs_BIT_initDStream -#define BIT_mask zfs_BIT_mask -#define BIT_reloadDStream zfs_BIT_reloadDStream +#define MEM_MODULE +#define XXH_NAMESPACE ZSTD_ +#define XXH_PRIVATE_API +#define XXH_INLINE_ALL +#define ZSTD_LEGACY_SUPPORT 0 +#define ZSTD_LIB_DICTBUILDER 0 +#define ZSTD_LIB_DEPRECATED 0 +#define ZSTD_NOBENCH +#define DEBUGLEVEL 0 +#ifdef _KERNEL +#define ZSTD_DEPS_ASSERT +#endif + + +/* -- >8 -- */ + +/* lib/common/entropy_common.o: */ +#define FSE_getErrorName zfs_FSE_getErrorName +#define FSE_isError zfs_FSE_isError +#define FSE_readNCount zfs_FSE_readNCount +#define FSE_versionNumber zfs_FSE_versionNumber +#define HUF_getErrorName zfs_HUF_getErrorName +#define HUF_isError zfs_HUF_isError +#define HUF_readStats zfs_HUF_readStats + +/* lib/common/error_private.o: */ #define ERR_getErrorString zfs_ERR_getErrorString -#define FSE_NCountWriteBound zfs_FSE_NCountWriteBound -#define FSE_buildCTable zfs_FSE_buildCTable + +/* lib/common/fse_decompress.o: */ +#define FSE_buildDTable_raw zfs_FSE_buildDTable_raw +#define FSE_buildDTable_rle zfs_FSE_buildDTable_rle +#define FSE_buildDTable zfs_FSE_buildDTable +#define FSE_createDTable zfs_FSE_createDTable +#define FSE_decompress_usingDTable zfs_FSE_decompress_usingDTable +#define FSE_decompress_wksp zfs_FSE_decompress_wksp +#define FSE_decompress zfs_FSE_decompress +#define FSE_freeDTable zfs_FSE_freeDTable + +/* lib/common/pool.o: */ +#define POOL_add zfs_POOL_add +#define POOL_create_advanced zfs_POOL_create_advanced +#define POOL_create zfs_POOL_create +#define POOL_free zfs_POOL_free +#define POOL_resize zfs_POOL_resize +#define POOL_sizeof zfs_POOL_sizeof +#define POOL_tryAdd zfs_POOL_tryAdd + +/* lib/common/zstd_common.o: */ +#define ZSTD_calloc zfs_ZSTD_calloc +#define ZSTD_free zfs_ZSTD_free +#define ZSTD_getErrorCode zfs_ZSTD_getErrorCode +#define ZSTD_getErrorName zfs_ZSTD_getErrorName +#define ZSTD_getErrorString zfs_ZSTD_getErrorString +#define ZSTD_isError zfs_ZSTD_isError +#define ZSTD_malloc zfs_ZSTD_malloc +#define ZSTD_versionNumber zfs_ZSTD_versionNumber +#define ZSTD_versionString zfs_ZSTD_versionString + +/* lib/compress/fse_compress.o: */ #define FSE_buildCTable_raw zfs_FSE_buildCTable_raw #define FSE_buildCTable_rle zfs_FSE_buildCTable_rle #define FSE_buildCTable_wksp zfs_FSE_buildCTable_wksp -#define FSE_buildDTable zfs_FSE_buildDTable -#define FSE_buildDTable_raw zfs_FSE_buildDTable_raw -#define FSE_buildDTable_rle zfs_FSE_buildDTable_rle -#define FSE_compress zfs_FSE_compress +#define FSE_buildCTable zfs_FSE_buildCTable #define FSE_compress2 zfs_FSE_compress2 #define FSE_compressBound zfs_FSE_compressBound #define FSE_compress_usingCTable zfs_FSE_compress_usingCTable -#define FSE_compress_usingCTable_generic zfs_FSE_compress_usingCTable_generic #define FSE_compress_wksp zfs_FSE_compress_wksp +#define FSE_compress zfs_FSE_compress #define FSE_createCTable zfs_FSE_createCTable -#define FSE_createDTable zfs_FSE_createDTable -#define FSE_decompress zfs_FSE_decompress -#define FSE_decompress_usingDTable zfs_FSE_decompress_usingDTable -#define FSE_decompress_wksp zfs_FSE_decompress_wksp #define FSE_freeCTable zfs_FSE_freeCTable -#define FSE_freeDTable zfs_FSE_freeDTable -#define FSE_getErrorName zfs_FSE_getErrorName +#define FSE_NCountWriteBound zfs_FSE_NCountWriteBound #define FSE_normalizeCount zfs_FSE_normalizeCount -#define FSE_optimalTableLog zfs_FSE_optimalTableLog #define FSE_optimalTableLog_internal zfs_FSE_optimalTableLog_internal -#define FSE_readNCount zfs_FSE_readNCount -#define FSE_versionNumber zfs_FSE_versionNumber +#define FSE_optimalTableLog zfs_FSE_optimalTableLog #define FSE_writeNCount zfs_FSE_writeNCount -#define HIST_count zfs_HIST_count -#define HIST_countFast zfs_HIST_countFast + +/* lib/compress/hist.o: */ #define HIST_countFast_wksp zfs_HIST_countFast_wksp -#define HIST_count_parallel_wksp zfs_HIST_count_parallel_wksp +#define HIST_countFast zfs_HIST_countFast #define HIST_count_simple zfs_HIST_count_simple #define HIST_count_wksp zfs_HIST_count_wksp -#define HUF_buildCTable zfs_HUF_buildCTable +#define HIST_count zfs_HIST_count +#define HIST_isError zfs_HIST_isError + +/* lib/compress/huf_compress.o: */ #define HUF_buildCTable_wksp zfs_HUF_buildCTable_wksp -#define HUF_compress zfs_HUF_compress -#define HUF_compress1X zfs_HUF_compress1X +#define HUF_buildCTable zfs_HUF_buildCTable #define HUF_compress1X_repeat zfs_HUF_compress1X_repeat #define HUF_compress1X_usingCTable zfs_HUF_compress1X_usingCTable #define HUF_compress1X_wksp zfs_HUF_compress1X_wksp +#define HUF_compress1X zfs_HUF_compress1X #define HUF_compress2 zfs_HUF_compress2 #define HUF_compress4X_repeat zfs_HUF_compress4X_repeat #define HUF_compress4X_usingCTable zfs_HUF_compress4X_usingCTable #define HUF_compress4X_wksp zfs_HUF_compress4X_wksp #define HUF_compressBound zfs_HUF_compressBound -#define HUF_compressWeights zfs_HUF_compressWeights -#define HUF_decompress zfs_HUF_decompress -#define HUF_decompress1X1 zfs_HUF_decompress1X1 -#define HUF_decompress1X1_DCtx zfs_HUF_decompress1X1_DCtx -#define HUF_decompress1X1_DCtx_wksp zfs_HUF_decompress1X1_DCtx_wksp -#define HUF_decompress1X1_DCtx_wksp_bmi2 zfs_HUF_decompress1X1_DCtx_wksp_bmi2 -#define HUF_decompress1X1_usingDTable zfs_HUF_decompress1X1_usingDTable -#define HUF_decompress1X2 zfs_HUF_decompress1X2 -#define HUF_decompress1X2_DCtx zfs_HUF_decompress1X2_DCtx -#define HUF_decompress1X2_DCtx_wksp zfs_HUF_decompress1X2_DCtx_wksp -#define HUF_decompress1X2_usingDTable zfs_HUF_decompress1X2_usingDTable -#define HUF_decompress1X_DCtx zfs_HUF_decompress1X_DCtx -#define HUF_decompress1X_DCtx_wksp zfs_HUF_decompress1X_DCtx_wksp -#define HUF_decompress1X_usingDTable zfs_HUF_decompress1X_usingDTable -#define HUF_decompress1X_usingDTable_bmi2 zfs_HUF_decompress1X_usingDTable_bmi2 -#define HUF_decompress4X1 zfs_HUF_decompress4X1 -#define HUF_decompress4X1_DCtx zfs_HUF_decompress4X1_DCtx -#define HUF_decompress4X1_DCtx_wksp zfs_HUF_decompress4X1_DCtx_wksp -#define HUF_decompress4X1_usingDTable zfs_HUF_decompress4X1_usingDTable -#define HUF_decompress4X2 zfs_HUF_decompress4X2 -#define HUF_decompress4X2_DCtx zfs_HUF_decompress4X2_DCtx -#define HUF_decompress4X2_DCtx_wksp zfs_HUF_decompress4X2_DCtx_wksp -#define HUF_decompress4X2_usingDTable zfs_HUF_decompress4X2_usingDTable -#define HUF_decompress4X_DCtx zfs_HUF_decompress4X_DCtx -#define HUF_decompress4X_hufOnly zfs_HUF_decompress4X_hufOnly -#define HUF_decompress4X_hufOnly_wksp zfs_HUF_decompress4X_hufOnly_wksp -#define HUF_decompress4X_hufOnly_wksp_bmi2 \ - zfs_HUF_decompress4X_hufOnly_wksp_bmi2 -#define HUF_decompress4X_usingDTable zfs_HUF_decompress4X_usingDTable -#define HUF_decompress4X_usingDTable_bmi2 zfs_HUF_decompress4X_usingDTable_bmi2 +#define HUF_compress zfs_HUF_compress #define HUF_estimateCompressedSize zfs_HUF_estimateCompressedSize -#define HUF_fillDTableX2Level2 zfs_HUF_fillDTableX2Level2 -#define HUF_getErrorName zfs_HUF_getErrorName #define HUF_getNbBits zfs_HUF_getNbBits #define HUF_optimalTableLog zfs_HUF_optimalTableLog #define HUF_readCTable zfs_HUF_readCTable -#define HUF_readDTableX1 zfs_HUF_readDTableX1 -#define HUF_readDTableX1_wksp zfs_HUF_readDTableX1_wksp -#define HUF_readDTableX2 zfs_HUF_readDTableX2 -#define HUF_readDTableX2_wksp zfs_HUF_readDTableX2_wksp -#define HUF_readStats zfs_HUF_readStats -#define HUF_selectDecoder zfs_HUF_selectDecoder -#define HUF_setMaxHeight zfs_HUF_setMaxHeight #define HUF_validateCTable zfs_HUF_validateCTable #define HUF_writeCTable zfs_HUF_writeCTable -#define LL_base zfs_LL_base -#define LL_bits zfs_LL_bits -#define LL_defaultDTable zfs_LL_defaultDTable -#define LL_defaultNorm zfs_LL_defaultNorm -#define ML_base zfs_ML_base -#define ML_bits zfs_ML_bits -#define ML_defaultDTable zfs_ML_defaultDTable -#define ML_defaultNorm zfs_ML_defaultNorm -#define OF_base zfs_OF_base -#define OF_bits zfs_OF_bits -#define OF_defaultDTable zfs_OF_defaultDTable -#define OF_defaultNorm zfs_OF_defaultNorm -#define POOL_add zfs_POOL_add -#define POOL_create zfs_POOL_create -#define POOL_create_advanced zfs_POOL_create_advanced -#define POOL_free zfs_POOL_free -#define POOL_resize zfs_POOL_resize -#define POOL_sizeof zfs_POOL_sizeof -#define POOL_tryAdd zfs_POOL_tryAdd + +/* lib/compress/zstd_compress_literals.o: */ +#define ZSTD_compressLiterals zfs_ZSTD_compressLiterals +#define ZSTD_compressRleLiteralsBlock zfs_ZSTD_compressRleLiteralsBlock +#define ZSTD_noCompressLiterals zfs_ZSTD_noCompressLiterals + +/* lib/compress/zstd_compress_sequences.o: */ +#define ZSTD_buildCTable zfs_ZSTD_buildCTable +#define ZSTD_crossEntropyCost zfs_ZSTD_crossEntropyCost +#define ZSTD_encodeSequences zfs_ZSTD_encodeSequences +#define ZSTD_fseBitCost zfs_ZSTD_fseBitCost +#define ZSTD_selectEncodingType zfs_ZSTD_selectEncodingType + +/* lib/compress/zstd_compress_superblock.o: */ +#define ZSTD_compressSuperBlock zfs_ZSTD_compressSuperBlock + +/* lib/compress/zstd_compress.o: */ +#define ZSTD_adjustCParams zfs_ZSTD_adjustCParams +#define ZSTD_CCtx_getParameter zfs_ZSTD_CCtx_getParameter +#define ZSTD_CCtx_loadDictionary_advanced zfs_ZSTD_CCtx_loadDictionary_advanced +#define ZSTD_CCtx_loadDictionary_byReference zfs_ZSTD_CCtx_loadDictionary_byReference +#define ZSTD_CCtx_loadDictionary zfs_ZSTD_CCtx_loadDictionary #define ZSTD_CCtxParams_getParameter zfs_ZSTD_CCtxParams_getParameter -#define ZSTD_CCtxParams_init zfs_ZSTD_CCtxParams_init #define ZSTD_CCtxParams_init_advanced zfs_ZSTD_CCtxParams_init_advanced +#define ZSTD_CCtxParams_init zfs_ZSTD_CCtxParams_init #define ZSTD_CCtxParams_reset zfs_ZSTD_CCtxParams_reset #define ZSTD_CCtxParams_setParameter zfs_ZSTD_CCtxParams_setParameter -#define ZSTD_CCtx_getParameter zfs_ZSTD_CCtx_getParameter -#define ZSTD_CCtx_loadDictionary zfs_ZSTD_CCtx_loadDictionary -#define ZSTD_CCtx_loadDictionary_advanced zfs_ZSTD_CCtx_loadDictionary_advanced -#define ZSTD_CCtx_loadDictionary_byReference \ - zfs_ZSTD_CCtx_loadDictionary_byReference #define ZSTD_CCtx_refCDict zfs_ZSTD_CCtx_refCDict -#define ZSTD_CCtx_refPrefix zfs_ZSTD_CCtx_refPrefix #define ZSTD_CCtx_refPrefix_advanced zfs_ZSTD_CCtx_refPrefix_advanced +#define ZSTD_CCtx_refPrefix zfs_ZSTD_CCtx_refPrefix #define ZSTD_CCtx_reset zfs_ZSTD_CCtx_reset +#define ZSTD_CCtx_setParametersUsingCCtxParams zfs_ZSTD_CCtx_setParametersUsingCCtxParams #define ZSTD_CCtx_setParameter zfs_ZSTD_CCtx_setParameter -#define ZSTD_CCtx_setParametersUsingCCtxParams \ - zfs_ZSTD_CCtx_setParametersUsingCCtxParams #define ZSTD_CCtx_setPledgedSrcSize zfs_ZSTD_CCtx_setPledgedSrcSize -#define ZSTD_CStreamInSize zfs_ZSTD_CStreamInSize -#define ZSTD_CStreamOutSize zfs_ZSTD_CStreamOutSize -#define ZSTD_DCtx_loadDictionary zfs_ZSTD_DCtx_loadDictionary -#define ZSTD_DCtx_loadDictionary_advanced zfs_ZSTD_DCtx_loadDictionary_advanced -#define ZSTD_DCtx_loadDictionary_byReference \ - zfs_ZSTD_DCtx_loadDictionary_byReference -#define ZSTD_DCtx_refDDict zfs_ZSTD_DCtx_refDDict -#define ZSTD_DCtx_refPrefix zfs_ZSTD_DCtx_refPrefix -#define ZSTD_DCtx_refPrefix_advanced zfs_ZSTD_DCtx_refPrefix_advanced -#define ZSTD_DCtx_reset zfs_ZSTD_DCtx_reset -#define ZSTD_DCtx_setFormat zfs_ZSTD_DCtx_setFormat -#define ZSTD_DCtx_setMaxWindowSize zfs_ZSTD_DCtx_setMaxWindowSize -#define ZSTD_DCtx_setParameter zfs_ZSTD_DCtx_setParameter -#define ZSTD_DDict_dictContent zfs_ZSTD_DDict_dictContent -#define ZSTD_DDict_dictSize zfs_ZSTD_DDict_dictSize -#define ZSTD_DStreamInSize zfs_ZSTD_DStreamInSize -#define ZSTD_DStreamOutSize zfs_ZSTD_DStreamOutSize -#define ZSTD_DUBT_findBestMatch zfs_ZSTD_DUBT_findBestMatch -#define ZSTD_NCountCost zfs_ZSTD_NCountCost -#define ZSTD_XXH64_digest zfs_ZSTD_XXH64_digest -#define ZSTD_adjustCParams zfs_ZSTD_adjustCParams -#define ZSTD_assignParamsToCCtxParams zfs_ZSTD_assignParamsToCCtxParams -#define ZSTD_buildCTable zfs_ZSTD_buildCTable -#define ZSTD_buildFSETable zfs_ZSTD_buildFSETable -#define ZSTD_buildSeqStore zfs_ZSTD_buildSeqStore -#define ZSTD_buildSeqTable zfs_ZSTD_buildSeqTable -#define ZSTD_cParam_getBounds zfs_ZSTD_cParam_getBounds -#define ZSTD_cParam_withinBounds zfs_ZSTD_cParam_withinBounds -#define ZSTD_calloc zfs_ZSTD_calloc #define ZSTD_checkCParams zfs_ZSTD_checkCParams -#define ZSTD_checkContinuity zfs_ZSTD_checkContinuity -#define ZSTD_compress zfs_ZSTD_compress #define ZSTD_compress2 zfs_ZSTD_compress2 -#define ZSTD_compressBegin zfs_ZSTD_compressBegin +#define ZSTD_compress_advanced_internal zfs_ZSTD_compress_advanced_internal +#define ZSTD_compress_advanced zfs_ZSTD_compress_advanced +#define ZSTD_compressBegin_advanced_internal zfs_ZSTD_compressBegin_advanced_internal #define ZSTD_compressBegin_advanced zfs_ZSTD_compressBegin_advanced -#define ZSTD_compressBegin_advanced_internal \ - zfs_ZSTD_compressBegin_advanced_internal +#define ZSTD_compressBegin_usingCDict_advanced zfs_ZSTD_compressBegin_usingCDict_advanced #define ZSTD_compressBegin_usingCDict zfs_ZSTD_compressBegin_usingCDict -#define ZSTD_compressBegin_usingCDict_advanced \ - zfs_ZSTD_compressBegin_usingCDict_advanced #define ZSTD_compressBegin_usingDict zfs_ZSTD_compressBegin_usingDict +#define ZSTD_compressBegin zfs_ZSTD_compressBegin #define ZSTD_compressBlock zfs_ZSTD_compressBlock -#define ZSTD_compressBlock_btlazy2 zfs_ZSTD_compressBlock_btlazy2 -#define ZSTD_compressBlock_btlazy2_dictMatchState \ - zfs_ZSTD_compressBlock_btlazy2_dictMatchState -#define ZSTD_compressBlock_btlazy2_extDict \ - zfs_ZSTD_compressBlock_btlazy2_extDict -#define ZSTD_compressBlock_btopt zfs_ZSTD_compressBlock_btopt -#define ZSTD_compressBlock_btopt_dictMatchState \ - zfs_ZSTD_compressBlock_btopt_dictMatchState -#define ZSTD_compressBlock_btopt_extDict zfs_ZSTD_compressBlock_btopt_extDict -#define ZSTD_compressBlock_btultra zfs_ZSTD_compressBlock_btultra -#define ZSTD_compressBlock_btultra2 zfs_ZSTD_compressBlock_btultra2 -#define ZSTD_compressBlock_btultra_dictMatchState \ - zfs_ZSTD_compressBlock_btultra_dictMatchState -#define ZSTD_compressBlock_btultra_extDict \ - zfs_ZSTD_compressBlock_btultra_extDict -#define ZSTD_compressBlock_doubleFast zfs_ZSTD_compressBlock_doubleFast -#define ZSTD_compressBlock_doubleFast_dictMatchState \ - zfs_ZSTD_compressBlock_doubleFast_dictMatchState -#define ZSTD_compressBlock_doubleFast_extDict \ - zfs_ZSTD_compressBlock_doubleFast_extDict -#define ZSTD_compressBlock_doubleFast_extDict_generic \ - zfs_ZSTD_compressBlock_doubleFast_extDict_generic -#define ZSTD_compressBlock_fast zfs_ZSTD_compressBlock_fast -#define ZSTD_compressBlock_fast_dictMatchState \ - zfs_ZSTD_compressBlock_fast_dictMatchState -#define ZSTD_compressBlock_fast_extDict zfs_ZSTD_compressBlock_fast_extDict -#define ZSTD_compressBlock_fast_extDict_generic \ - zfs_ZSTD_compressBlock_fast_extDict_generic -#define ZSTD_compressBlock_greedy zfs_ZSTD_compressBlock_greedy -#define ZSTD_compressBlock_greedy_dictMatchState \ - zfs_ZSTD_compressBlock_greedy_dictMatchState -#define ZSTD_compressBlock_greedy_extDict zfs_ZSTD_compressBlock_greedy_extDict -#define ZSTD_compressBlock_internal zfs_ZSTD_compressBlock_internal -#define ZSTD_compressBlock_lazy zfs_ZSTD_compressBlock_lazy -#define ZSTD_compressBlock_lazy2 zfs_ZSTD_compressBlock_lazy2 -#define ZSTD_compressBlock_lazy2_dictMatchState \ - zfs_ZSTD_compressBlock_lazy2_dictMatchState -#define ZSTD_compressBlock_lazy2_extDict zfs_ZSTD_compressBlock_lazy2_extDict -#define ZSTD_compressBlock_lazy_dictMatchState \ - zfs_ZSTD_compressBlock_lazy_dictMatchState -#define ZSTD_compressBlock_lazy_extDict zfs_ZSTD_compressBlock_lazy_extDict #define ZSTD_compressBound zfs_ZSTD_compressBound #define ZSTD_compressCCtx zfs_ZSTD_compressCCtx #define ZSTD_compressContinue zfs_ZSTD_compressContinue -#define ZSTD_compressContinue_internal zfs_ZSTD_compressContinue_internal #define ZSTD_compressEnd zfs_ZSTD_compressEnd -#define ZSTD_compressLiterals zfs_ZSTD_compressLiterals -#define ZSTD_compressRleLiteralsBlock zfs_ZSTD_compressRleLiteralsBlock -#define ZSTD_compressStream zfs_ZSTD_compressStream -#define ZSTD_compressStream2 zfs_ZSTD_compressStream2 #define ZSTD_compressStream2_simpleArgs zfs_ZSTD_compressStream2_simpleArgs -#define ZSTD_compressSuperBlock zfs_ZSTD_compressSuperBlock -#define ZSTD_compress_advanced zfs_ZSTD_compress_advanced -#define ZSTD_compress_advanced_internal zfs_ZSTD_compress_advanced_internal -#define ZSTD_compress_internal zfs_ZSTD_compress_internal -#define ZSTD_compress_usingCDict zfs_ZSTD_compress_usingCDict +#define ZSTD_compressStream2 zfs_ZSTD_compressStream2 +#define ZSTD_compressStream zfs_ZSTD_compressStream #define ZSTD_compress_usingCDict_advanced zfs_ZSTD_compress_usingCDict_advanced +#define ZSTD_compress_usingCDict zfs_ZSTD_compress_usingCDict #define ZSTD_compress_usingDict zfs_ZSTD_compress_usingDict +#define ZSTD_compress zfs_ZSTD_compress #define ZSTD_copyCCtx zfs_ZSTD_copyCCtx -#define ZSTD_copyDCtx zfs_ZSTD_copyDCtx -#define ZSTD_copyDDictParameters zfs_ZSTD_copyDDictParameters -#define ZSTD_count zfs_ZSTD_count -#define ZSTD_count_2segments zfs_ZSTD_count_2segments -#define ZSTD_createCCtx zfs_ZSTD_createCCtx -#define ZSTD_createCCtxParams zfs_ZSTD_createCCtxParams +#define ZSTD_cParam_getBounds zfs_ZSTD_cParam_getBounds #define ZSTD_createCCtx_advanced zfs_ZSTD_createCCtx_advanced -#define ZSTD_createCDict zfs_ZSTD_createCDict +#define ZSTD_createCCtxParams zfs_ZSTD_createCCtxParams +#define ZSTD_createCCtx zfs_ZSTD_createCCtx #define ZSTD_createCDict_advanced zfs_ZSTD_createCDict_advanced #define ZSTD_createCDict_byReference zfs_ZSTD_createCDict_byReference -#define ZSTD_createCStream zfs_ZSTD_createCStream +#define ZSTD_createCDict zfs_ZSTD_createCDict #define ZSTD_createCStream_advanced zfs_ZSTD_createCStream_advanced -#define ZSTD_createDCtx zfs_ZSTD_createDCtx -#define ZSTD_createDCtx_advanced zfs_ZSTD_createDCtx_advanced -#define ZSTD_createDDict zfs_ZSTD_createDDict -#define ZSTD_createDDict_advanced zfs_ZSTD_createDDict_advanced -#define ZSTD_createDDict_byReference zfs_ZSTD_createDDict_byReference -#define ZSTD_createDStream zfs_ZSTD_createDStream -#define ZSTD_createDStream_advanced zfs_ZSTD_createDStream_advanced -#define ZSTD_crossEntropyCost zfs_ZSTD_crossEntropyCost +#define ZSTD_createCStream zfs_ZSTD_createCStream +#define ZSTD_CStreamInSize zfs_ZSTD_CStreamInSize +#define ZSTD_CStreamOutSize zfs_ZSTD_CStreamOutSize #define ZSTD_cycleLog zfs_ZSTD_cycleLog -#define ZSTD_dParam_getBounds zfs_ZSTD_dParam_getBounds -#define ZSTD_decodeLiteralsBlock zfs_ZSTD_decodeLiteralsBlock -#define ZSTD_decodeSeqHeaders zfs_ZSTD_decodeSeqHeaders -#define ZSTD_decodingBufferSize_min zfs_ZSTD_decodingBufferSize_min -#define ZSTD_decompress zfs_ZSTD_decompress -#define ZSTD_decompressBegin zfs_ZSTD_decompressBegin -#define ZSTD_decompressBegin_usingDDict zfs_ZSTD_decompressBegin_usingDDict -#define ZSTD_decompressBegin_usingDict zfs_ZSTD_decompressBegin_usingDict -#define ZSTD_decompressBlock zfs_ZSTD_decompressBlock -#define ZSTD_decompressBlock_internal zfs_ZSTD_decompressBlock_internal -#define ZSTD_decompressBound zfs_ZSTD_decompressBound -#define ZSTD_decompressContinue zfs_ZSTD_decompressContinue -#define ZSTD_decompressContinueStream zfs_ZSTD_decompressContinueStream -#define ZSTD_decompressDCtx zfs_ZSTD_decompressDCtx -#define ZSTD_decompressMultiFrame zfs_ZSTD_decompressMultiFrame -#define ZSTD_decompressStream zfs_ZSTD_decompressStream -#define ZSTD_decompressStream_simpleArgs zfs_ZSTD_decompressStream_simpleArgs -#define ZSTD_decompress_usingDDict zfs_ZSTD_decompress_usingDDict -#define ZSTD_decompress_usingDict zfs_ZSTD_decompress_usingDict -#define ZSTD_defaultCParameters zfs_ZSTD_defaultCParameters -#define ZSTD_did_fieldSize zfs_ZSTD_did_fieldSize -#define ZSTD_encodeSequences zfs_ZSTD_encodeSequences -#define ZSTD_encodeSequences_default zfs_ZSTD_encodeSequences_default #define ZSTD_endStream zfs_ZSTD_endStream +#define ZSTD_estimateCCtxSize_usingCCtxParams zfs_ZSTD_estimateCCtxSize_usingCCtxParams +#define ZSTD_estimateCCtxSize_usingCParams zfs_ZSTD_estimateCCtxSize_usingCParams #define ZSTD_estimateCCtxSize zfs_ZSTD_estimateCCtxSize -#define ZSTD_estimateCCtxSize_usingCCtxParams \ - zfs_ZSTD_estimateCCtxSize_usingCCtxParams -#define ZSTD_estimateCCtxSize_usingCParams \ - zfs_ZSTD_estimateCCtxSize_usingCParams -#define ZSTD_estimateCDictSize zfs_ZSTD_estimateCDictSize #define ZSTD_estimateCDictSize_advanced zfs_ZSTD_estimateCDictSize_advanced +#define ZSTD_estimateCDictSize zfs_ZSTD_estimateCDictSize +#define ZSTD_estimateCStreamSize_usingCCtxParams zfs_ZSTD_estimateCStreamSize_usingCCtxParams +#define ZSTD_estimateCStreamSize_usingCParams zfs_ZSTD_estimateCStreamSize_usingCParams #define ZSTD_estimateCStreamSize zfs_ZSTD_estimateCStreamSize -#define ZSTD_estimateCStreamSize_usingCCtxParams \ - zfs_ZSTD_estimateCStreamSize_usingCCtxParams -#define ZSTD_estimateCStreamSize_usingCParams \ - zfs_ZSTD_estimateCStreamSize_usingCParams -#define ZSTD_estimateDCtxSize zfs_ZSTD_estimateDCtxSize -#define ZSTD_estimateDDictSize zfs_ZSTD_estimateDDictSize -#define ZSTD_estimateDStreamSize zfs_ZSTD_estimateDStreamSize -#define ZSTD_estimateDStreamSize_fromFrame \ - zfs_ZSTD_estimateDStreamSize_fromFrame -#define ZSTD_fcs_fieldSize zfs_ZSTD_fcs_fieldSize -#define ZSTD_fillDoubleHashTable zfs_ZSTD_fillDoubleHashTable -#define ZSTD_fillHashTable zfs_ZSTD_fillHashTable -#define ZSTD_findDecompressedSize zfs_ZSTD_findDecompressedSize -#define ZSTD_findFrameCompressedSize zfs_ZSTD_findFrameCompressedSize -#define ZSTD_findFrameSizeInfo zfs_ZSTD_findFrameSizeInfo #define ZSTD_flushStream zfs_ZSTD_flushStream -#define ZSTD_frameHeaderSize zfs_ZSTD_frameHeaderSize -#define ZSTD_free zfs_ZSTD_free -#define ZSTD_freeCCtx zfs_ZSTD_freeCCtx #define ZSTD_freeCCtxParams zfs_ZSTD_freeCCtxParams +#define ZSTD_freeCCtx zfs_ZSTD_freeCCtx #define ZSTD_freeCDict zfs_ZSTD_freeCDict #define ZSTD_freeCStream zfs_ZSTD_freeCStream -#define ZSTD_freeDCtx zfs_ZSTD_freeDCtx -#define ZSTD_freeDDict zfs_ZSTD_freeDDict -#define ZSTD_freeDStream zfs_ZSTD_freeDStream -#define ZSTD_fseBitCost zfs_ZSTD_fseBitCost #define ZSTD_getBlockSize zfs_ZSTD_getBlockSize -#define ZSTD_getCParams zfs_ZSTD_getCParams #define ZSTD_getCParamsFromCCtxParams zfs_ZSTD_getCParamsFromCCtxParams #define ZSTD_getCParamsFromCDict zfs_ZSTD_getCParamsFromCDict -#define ZSTD_getCParams_internal zfs_ZSTD_getCParams_internal -#define ZSTD_getDDict zfs_ZSTD_getDDict -#define ZSTD_getDecompressedSize zfs_ZSTD_getDecompressedSize -#define ZSTD_getDictID_fromDDict zfs_ZSTD_getDictID_fromDDict -#define ZSTD_getDictID_fromDict zfs_ZSTD_getDictID_fromDict -#define ZSTD_getDictID_fromFrame zfs_ZSTD_getDictID_fromFrame -#define ZSTD_getErrorCode zfs_ZSTD_getErrorCode -#define ZSTD_getErrorName zfs_ZSTD_getErrorName -#define ZSTD_getErrorString zfs_ZSTD_getErrorString -#define ZSTD_getFrameContentSize zfs_ZSTD_getFrameContentSize -#define ZSTD_getFrameHeader zfs_ZSTD_getFrameHeader -#define ZSTD_getFrameHeader_advanced zfs_ZSTD_getFrameHeader_advanced +#define ZSTD_getCParams zfs_ZSTD_getCParams #define ZSTD_getFrameProgression zfs_ZSTD_getFrameProgression #define ZSTD_getParams zfs_ZSTD_getParams #define ZSTD_getSeqStore zfs_ZSTD_getSeqStore #define ZSTD_getSequences zfs_ZSTD_getSequences -#define ZSTD_getcBlockSize zfs_ZSTD_getcBlockSize -#define ZSTD_hashPtr zfs_ZSTD_hashPtr -#define ZSTD_initCDict_internal zfs_ZSTD_initCDict_internal -#define ZSTD_initCStream zfs_ZSTD_initCStream #define ZSTD_initCStream_advanced zfs_ZSTD_initCStream_advanced #define ZSTD_initCStream_internal zfs_ZSTD_initCStream_internal #define ZSTD_initCStream_srcSize zfs_ZSTD_initCStream_srcSize +#define ZSTD_initCStream_usingCDict_advanced zfs_ZSTD_initCStream_usingCDict_advanced #define ZSTD_initCStream_usingCDict zfs_ZSTD_initCStream_usingCDict -#define ZSTD_initCStream_usingCDict_advanced \ - zfs_ZSTD_initCStream_usingCDict_advanced #define ZSTD_initCStream_usingDict zfs_ZSTD_initCStream_usingDict -#define ZSTD_initDDict_internal zfs_ZSTD_initDDict_internal -#define ZSTD_initDStream zfs_ZSTD_initDStream -#define ZSTD_initDStream_usingDDict zfs_ZSTD_initDStream_usingDDict -#define ZSTD_initDStream_usingDict zfs_ZSTD_initDStream_usingDict -#define ZSTD_initFseState zfs_ZSTD_initFseState +#define ZSTD_initCStream zfs_ZSTD_initCStream #define ZSTD_initStaticCCtx zfs_ZSTD_initStaticCCtx #define ZSTD_initStaticCDict zfs_ZSTD_initStaticCDict #define ZSTD_initStaticCStream zfs_ZSTD_initStaticCStream -#define ZSTD_initStaticDCtx zfs_ZSTD_initStaticDCtx -#define ZSTD_initStaticDDict zfs_ZSTD_initStaticDDict -#define ZSTD_initStaticDStream zfs_ZSTD_initStaticDStream -#define ZSTD_initStats_ultra zfs_ZSTD_initStats_ultra -#define ZSTD_insertAndFindFirstIndex zfs_ZSTD_insertAndFindFirstIndex -#define ZSTD_insertAndFindFirstIndexHash3 zfs_ZSTD_insertAndFindFirstIndexHash3 -#define ZSTD_insertAndFindFirstIndex_internal \ - zfs_ZSTD_insertAndFindFirstIndex_internal -#define ZSTD_insertBlock zfs_ZSTD_insertBlock #define ZSTD_invalidateRepCodes zfs_ZSTD_invalidateRepCodes -#define ZSTD_isError zfs_ZSTD_isError -#define ZSTD_isFrame zfs_ZSTD_isFrame -#define ZSTD_ldm_adjustParameters zfs_ZSTD_ldm_adjustParameters -#define ZSTD_ldm_blockCompress zfs_ZSTD_ldm_blockCompress -#define ZSTD_ldm_fillHashTable zfs_ZSTD_ldm_fillHashTable -#define ZSTD_ldm_generateSequences zfs_ZSTD_ldm_generateSequences -#define ZSTD_ldm_getMaxNbSeq zfs_ZSTD_ldm_getMaxNbSeq -#define ZSTD_ldm_getTableSize zfs_ZSTD_ldm_getTableSize -#define ZSTD_ldm_skipSequences zfs_ZSTD_ldm_skipSequences #define ZSTD_loadCEntropy zfs_ZSTD_loadCEntropy -#define ZSTD_loadDEntropy zfs_ZSTD_loadDEntropy -#define ZSTD_loadDictionaryContent zfs_ZSTD_loadDictionaryContent -#define ZSTD_makeCCtxParamsFromCParams zfs_ZSTD_makeCCtxParamsFromCParams -#define ZSTD_malloc zfs_ZSTD_malloc #define ZSTD_maxCLevel zfs_ZSTD_maxCLevel #define ZSTD_minCLevel zfs_ZSTD_minCLevel -#define ZSTD_nextInputType zfs_ZSTD_nextInputType -#define ZSTD_nextSrcSizeToDecompress zfs_ZSTD_nextSrcSizeToDecompress -#define ZSTD_noCompressLiterals zfs_ZSTD_noCompressLiterals #define ZSTD_referenceExternalSequences zfs_ZSTD_referenceExternalSequences -#define ZSTD_rescaleFreqs zfs_ZSTD_rescaleFreqs -#define ZSTD_resetCCtx_internal zfs_ZSTD_resetCCtx_internal -#define ZSTD_resetCCtx_usingCDict zfs_ZSTD_resetCCtx_usingCDict +#define ZSTD_reset_compressedBlockState zfs_ZSTD_reset_compressedBlockState #define ZSTD_resetCStream zfs_ZSTD_resetCStream -#define ZSTD_resetDStream zfs_ZSTD_resetDStream #define ZSTD_resetSeqStore zfs_ZSTD_resetSeqStore -#define ZSTD_reset_compressedBlockState zfs_ZSTD_reset_compressedBlockState -#define ZSTD_safecopy zfs_ZSTD_safecopy #define ZSTD_selectBlockCompressor zfs_ZSTD_selectBlockCompressor -#define ZSTD_selectEncodingType zfs_ZSTD_selectEncodingType #define ZSTD_seqToCodes zfs_ZSTD_seqToCodes #define ZSTD_sizeof_CCtx zfs_ZSTD_sizeof_CCtx #define ZSTD_sizeof_CDict zfs_ZSTD_sizeof_CDict #define ZSTD_sizeof_CStream zfs_ZSTD_sizeof_CStream -#define ZSTD_sizeof_DCtx zfs_ZSTD_sizeof_DCtx -#define ZSTD_sizeof_DDict zfs_ZSTD_sizeof_DDict -#define ZSTD_sizeof_DStream zfs_ZSTD_sizeof_DStream #define ZSTD_toFlushNow zfs_ZSTD_toFlushNow -#define ZSTD_updateRep zfs_ZSTD_updateRep -#define ZSTD_updateStats zfs_ZSTD_updateStats -#define ZSTD_updateTree zfs_ZSTD_updateTree -#define ZSTD_versionNumber zfs_ZSTD_versionNumber -#define ZSTD_versionString zfs_ZSTD_versionString -#define ZSTD_writeFrameHeader zfs_ZSTD_writeFrameHeader #define ZSTD_writeLastEmptyBlock zfs_ZSTD_writeLastEmptyBlock -#define algoTime zfs_algoTime -#define attachDictSizeCutoffs zfs_attachDictSizeCutoffs -#define g_ctx zfs_g_ctx -#define g_debuglevel zfs_g_debuglevel -#define kInverseProbabilityLog256 zfs_kInverseProbabilityLog256 -#define repStartValue zfs_repStartValue -#define FSE_isError zfs_FSE_isError -#define HUF_isError zfs_HUF_isError + +/* lib/compress/zstd_double_fast.o: */ +#define ZSTD_compressBlock_doubleFast_dictMatchState zfs_ZSTD_compressBlock_doubleFast_dictMatchState +#define ZSTD_compressBlock_doubleFast_extDict zfs_ZSTD_compressBlock_doubleFast_extDict +#define ZSTD_compressBlock_doubleFast zfs_ZSTD_compressBlock_doubleFast +#define ZSTD_fillDoubleHashTable zfs_ZSTD_fillDoubleHashTable + +/* lib/compress/zstd_fast.o: */ +#define ZSTD_compressBlock_fast_dictMatchState zfs_ZSTD_compressBlock_fast_dictMatchState +#define ZSTD_compressBlock_fast_extDict zfs_ZSTD_compressBlock_fast_extDict +#define ZSTD_compressBlock_fast zfs_ZSTD_compressBlock_fast +#define ZSTD_fillHashTable zfs_ZSTD_fillHashTable + +/* lib/compress/zstd_lazy.o: */ +#define ZSTD_compressBlock_btlazy2_dictMatchState zfs_ZSTD_compressBlock_btlazy2_dictMatchState +#define ZSTD_compressBlock_btlazy2_extDict zfs_ZSTD_compressBlock_btlazy2_extDict +#define ZSTD_compressBlock_btlazy2 zfs_ZSTD_compressBlock_btlazy2 +#define ZSTD_compressBlock_greedy_dictMatchState zfs_ZSTD_compressBlock_greedy_dictMatchState +#define ZSTD_compressBlock_greedy_extDict zfs_ZSTD_compressBlock_greedy_extDict +#define ZSTD_compressBlock_greedy zfs_ZSTD_compressBlock_greedy +#define ZSTD_compressBlock_lazy2_dictMatchState zfs_ZSTD_compressBlock_lazy2_dictMatchState +#define ZSTD_compressBlock_lazy2_extDict zfs_ZSTD_compressBlock_lazy2_extDict +#define ZSTD_compressBlock_lazy2 zfs_ZSTD_compressBlock_lazy2 +#define ZSTD_compressBlock_lazy_dictMatchState zfs_ZSTD_compressBlock_lazy_dictMatchState +#define ZSTD_compressBlock_lazy_extDict zfs_ZSTD_compressBlock_lazy_extDict +#define ZSTD_compressBlock_lazy zfs_ZSTD_compressBlock_lazy +#define ZSTD_insertAndFindFirstIndex zfs_ZSTD_insertAndFindFirstIndex + +/* lib/compress/zstd_ldm.o: */ +#define ZSTD_ldm_adjustParameters zfs_ZSTD_ldm_adjustParameters +#define ZSTD_ldm_blockCompress zfs_ZSTD_ldm_blockCompress +#define ZSTD_ldm_fillHashTable zfs_ZSTD_ldm_fillHashTable +#define ZSTD_ldm_generateSequences zfs_ZSTD_ldm_generateSequences +#define ZSTD_ldm_getMaxNbSeq zfs_ZSTD_ldm_getMaxNbSeq +#define ZSTD_ldm_getTableSize zfs_ZSTD_ldm_getTableSize +#define ZSTD_ldm_skipSequences zfs_ZSTD_ldm_skipSequences + +/* lib/compress/zstd_opt.o: */ +#define ZSTD_compressBlock_btopt_dictMatchState zfs_ZSTD_compressBlock_btopt_dictMatchState +#define ZSTD_compressBlock_btopt_extDict zfs_ZSTD_compressBlock_btopt_extDict +#define ZSTD_compressBlock_btopt zfs_ZSTD_compressBlock_btopt +#define ZSTD_compressBlock_btultra2 zfs_ZSTD_compressBlock_btultra2 +#define ZSTD_compressBlock_btultra_dictMatchState zfs_ZSTD_compressBlock_btultra_dictMatchState +#define ZSTD_compressBlock_btultra_extDict zfs_ZSTD_compressBlock_btultra_extDict +#define ZSTD_compressBlock_btultra zfs_ZSTD_compressBlock_btultra +#define ZSTD_updateTree zfs_ZSTD_updateTree + +/* lib/decompress/huf_decompress.o: */ +#define HUF_decompress1X1_DCtx_wksp_bmi2 zfs_HUF_decompress1X1_DCtx_wksp_bmi2 +#define HUF_decompress1X1_DCtx_wksp zfs_HUF_decompress1X1_DCtx_wksp +#define HUF_decompress1X1_DCtx zfs_HUF_decompress1X1_DCtx +#define HUF_decompress1X1_usingDTable zfs_HUF_decompress1X1_usingDTable +#define HUF_decompress1X1 zfs_HUF_decompress1X1 +#define HUF_decompress1X2_DCtx_wksp zfs_HUF_decompress1X2_DCtx_wksp +#define HUF_decompress1X2_DCtx zfs_HUF_decompress1X2_DCtx +#define HUF_decompress1X2_usingDTable zfs_HUF_decompress1X2_usingDTable +#define HUF_decompress1X2 zfs_HUF_decompress1X2 +#define HUF_decompress1X_DCtx_wksp zfs_HUF_decompress1X_DCtx_wksp +#define HUF_decompress1X_DCtx zfs_HUF_decompress1X_DCtx +#define HUF_decompress1X_usingDTable_bmi2 zfs_HUF_decompress1X_usingDTable_bmi2 +#define HUF_decompress1X_usingDTable zfs_HUF_decompress1X_usingDTable +#define HUF_decompress4X1_DCtx_wksp zfs_HUF_decompress4X1_DCtx_wksp +#define HUF_decompress4X1_DCtx zfs_HUF_decompress4X1_DCtx +#define HUF_decompress4X1_usingDTable zfs_HUF_decompress4X1_usingDTable +#define HUF_decompress4X1 zfs_HUF_decompress4X1 +#define HUF_decompress4X2_DCtx_wksp zfs_HUF_decompress4X2_DCtx_wksp +#define HUF_decompress4X2_DCtx zfs_HUF_decompress4X2_DCtx +#define HUF_decompress4X2_usingDTable zfs_HUF_decompress4X2_usingDTable +#define HUF_decompress4X2 zfs_HUF_decompress4X2 +#define HUF_decompress4X_DCtx zfs_HUF_decompress4X_DCtx +#define HUF_decompress4X_hufOnly_wksp_bmi2 zfs_HUF_decompress4X_hufOnly_wksp_bmi2 +#define HUF_decompress4X_hufOnly_wksp zfs_HUF_decompress4X_hufOnly_wksp +#define HUF_decompress4X_hufOnly zfs_HUF_decompress4X_hufOnly +#define HUF_decompress4X_usingDTable_bmi2 zfs_HUF_decompress4X_usingDTable_bmi2 +#define HUF_decompress4X_usingDTable zfs_HUF_decompress4X_usingDTable +#define HUF_decompress zfs_HUF_decompress +#define HUF_readDTableX1_wksp zfs_HUF_readDTableX1_wksp +#define HUF_readDTableX1 zfs_HUF_readDTableX1 +#define HUF_readDTableX2_wksp zfs_HUF_readDTableX2_wksp +#define HUF_readDTableX2 zfs_HUF_readDTableX2 +#define HUF_selectDecoder zfs_HUF_selectDecoder + +/* lib/decompress/zstd_ddict.o: */ +#define ZSTD_copyDDictParameters zfs_ZSTD_copyDDictParameters +#define ZSTD_createDDict_advanced zfs_ZSTD_createDDict_advanced +#define ZSTD_createDDict_byReference zfs_ZSTD_createDDict_byReference +#define ZSTD_createDDict zfs_ZSTD_createDDict +#define ZSTD_DDict_dictContent zfs_ZSTD_DDict_dictContent +#define ZSTD_DDict_dictSize zfs_ZSTD_DDict_dictSize +#define ZSTD_estimateDDictSize zfs_ZSTD_estimateDDictSize +#define ZSTD_freeDDict zfs_ZSTD_freeDDict +#define ZSTD_getDictID_fromDDict zfs_ZSTD_getDictID_fromDDict +#define ZSTD_initStaticDDict zfs_ZSTD_initStaticDDict +#define ZSTD_sizeof_DDict zfs_ZSTD_sizeof_DDict + +/* lib/decompress/zstd_decompress.o: */ +#define ZSTD_copyDCtx zfs_ZSTD_copyDCtx +#define ZSTD_createDCtx_advanced zfs_ZSTD_createDCtx_advanced +#define ZSTD_createDCtx zfs_ZSTD_createDCtx +#define ZSTD_createDStream_advanced zfs_ZSTD_createDStream_advanced +#define ZSTD_createDStream zfs_ZSTD_createDStream +#define ZSTD_DCtx_loadDictionary_advanced zfs_ZSTD_DCtx_loadDictionary_advanced +#define ZSTD_DCtx_loadDictionary_byReference zfs_ZSTD_DCtx_loadDictionary_byReference +#define ZSTD_DCtx_loadDictionary zfs_ZSTD_DCtx_loadDictionary +#define ZSTD_DCtx_refDDict zfs_ZSTD_DCtx_refDDict +#define ZSTD_DCtx_refPrefix_advanced zfs_ZSTD_DCtx_refPrefix_advanced +#define ZSTD_DCtx_refPrefix zfs_ZSTD_DCtx_refPrefix +#define ZSTD_DCtx_reset zfs_ZSTD_DCtx_reset +#define ZSTD_DCtx_setFormat zfs_ZSTD_DCtx_setFormat +#define ZSTD_DCtx_setMaxWindowSize zfs_ZSTD_DCtx_setMaxWindowSize +#define ZSTD_DCtx_setParameter zfs_ZSTD_DCtx_setParameter +#define ZSTD_decodingBufferSize_min zfs_ZSTD_decodingBufferSize_min +#define ZSTD_decompressBegin_usingDDict zfs_ZSTD_decompressBegin_usingDDict +#define ZSTD_decompressBegin_usingDict zfs_ZSTD_decompressBegin_usingDict +#define ZSTD_decompressBegin zfs_ZSTD_decompressBegin +#define ZSTD_decompressBound zfs_ZSTD_decompressBound +#define ZSTD_decompressContinue zfs_ZSTD_decompressContinue +#define ZSTD_decompressDCtx zfs_ZSTD_decompressDCtx +#define ZSTD_decompressStream_simpleArgs zfs_ZSTD_decompressStream_simpleArgs +#define ZSTD_decompressStream zfs_ZSTD_decompressStream +#define ZSTD_decompress_usingDDict zfs_ZSTD_decompress_usingDDict +#define ZSTD_decompress_usingDict zfs_ZSTD_decompress_usingDict +#define ZSTD_decompress zfs_ZSTD_decompress +#define ZSTD_dParam_getBounds zfs_ZSTD_dParam_getBounds +#define ZSTD_DStreamInSize zfs_ZSTD_DStreamInSize +#define ZSTD_DStreamOutSize zfs_ZSTD_DStreamOutSize +#define ZSTD_estimateDCtxSize zfs_ZSTD_estimateDCtxSize +#define ZSTD_estimateDStreamSize_fromFrame zfs_ZSTD_estimateDStreamSize_fromFrame +#define ZSTD_estimateDStreamSize zfs_ZSTD_estimateDStreamSize +#define ZSTD_findDecompressedSize zfs_ZSTD_findDecompressedSize +#define ZSTD_findFrameCompressedSize zfs_ZSTD_findFrameCompressedSize +#define ZSTD_frameHeaderSize zfs_ZSTD_frameHeaderSize +#define ZSTD_freeDCtx zfs_ZSTD_freeDCtx +#define ZSTD_freeDStream zfs_ZSTD_freeDStream +#define ZSTD_getDecompressedSize zfs_ZSTD_getDecompressedSize +#define ZSTD_getDictID_fromDict zfs_ZSTD_getDictID_fromDict +#define ZSTD_getDictID_fromFrame zfs_ZSTD_getDictID_fromFrame +#define ZSTD_getFrameContentSize zfs_ZSTD_getFrameContentSize +#define ZSTD_getFrameHeader_advanced zfs_ZSTD_getFrameHeader_advanced +#define ZSTD_getFrameHeader zfs_ZSTD_getFrameHeader +#define ZSTD_initDStream_usingDDict zfs_ZSTD_initDStream_usingDDict +#define ZSTD_initDStream_usingDict zfs_ZSTD_initDStream_usingDict +#define ZSTD_initDStream zfs_ZSTD_initDStream +#define ZSTD_initStaticDCtx zfs_ZSTD_initStaticDCtx +#define ZSTD_initStaticDStream zfs_ZSTD_initStaticDStream +#define ZSTD_insertBlock zfs_ZSTD_insertBlock +#define ZSTD_isFrame zfs_ZSTD_isFrame +#define ZSTD_loadDEntropy zfs_ZSTD_loadDEntropy +#define ZSTD_nextInputType zfs_ZSTD_nextInputType +#define ZSTD_nextSrcSizeToDecompress zfs_ZSTD_nextSrcSizeToDecompress +#define ZSTD_resetDStream zfs_ZSTD_resetDStream +#define ZSTD_sizeof_DCtx zfs_ZSTD_sizeof_DCtx +#define ZSTD_sizeof_DStream zfs_ZSTD_sizeof_DStream + +/* lib/decompress/zstd_decompress_block.o: */ +#define ZSTD_buildFSETable zfs_ZSTD_buildFSETable +#define ZSTD_checkContinuity zfs_ZSTD_checkContinuity +#define ZSTD_decodeLiteralsBlock zfs_ZSTD_decodeLiteralsBlock +#define ZSTD_decodeSeqHeaders zfs_ZSTD_decodeSeqHeaders +#define ZSTD_decompressBlock_internal zfs_ZSTD_decompressBlock_internal +#define ZSTD_decompressBlock zfs_ZSTD_decompressBlock +#define ZSTD_getcBlockSize zfs_ZSTD_getcBlockSize diff --git a/module/zstd/lib/common/bitstream.h b/module/zstd/lib/common/bitstream.h new file mode 100644 index 000000000000..37b99c01eed3 --- /dev/null +++ b/module/zstd/lib/common/bitstream.h @@ -0,0 +1,454 @@ +/* ****************************************************************** + * bitstream + * Part of FSE library + * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. + * + * You can contact the author at : + * - Source repository : https://github.com/Cyan4973/FiniteStateEntropy + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. +****************************************************************** */ +#ifndef BITSTREAM_H_MODULE +#define BITSTREAM_H_MODULE + +#if defined (__cplusplus) +extern "C" { +#endif + +/* +* This API consists of small unitary functions, which must be inlined for best performance. +* Since link-time-optimization is not available for all compilers, +* these functions are defined into a .h to be included. +*/ + +/*-**************************************** +* Dependencies +******************************************/ +#include "mem.h" /* unaligned access routines */ +#include "compiler.h" /* UNLIKELY() */ +#include "debug.h" /* assert(), DEBUGLOG(), RAWLOG() */ +#include "error_private.h" /* error codes and messages */ + + +/*========================================= +* Target specific +=========================================*/ +#if defined(__BMI__) && defined(__GNUC__) +# include /* support for bextr (experimental) */ +#elif defined(__ICCARM__) +# include +#endif + +#define STREAM_ACCUMULATOR_MIN_32 25 +#define STREAM_ACCUMULATOR_MIN_64 57 +#define STREAM_ACCUMULATOR_MIN ((U32)(MEM_32bits() ? STREAM_ACCUMULATOR_MIN_32 : STREAM_ACCUMULATOR_MIN_64)) + + +/*-****************************************** +* bitStream encoding API (write forward) +********************************************/ +/* bitStream can mix input from multiple sources. + * A critical property of these streams is that they encode and decode in **reverse** direction. + * So the first bit sequence you add will be the last to be read, like a LIFO stack. + */ +typedef struct { + size_t bitContainer; + unsigned bitPos; + char* startPtr; + char* ptr; + char* endPtr; +} BIT_CStream_t; + +MEM_STATIC size_t BIT_initCStream(BIT_CStream_t* bitC, void* dstBuffer, size_t dstCapacity); +MEM_STATIC void BIT_addBits(BIT_CStream_t* bitC, size_t value, unsigned nbBits); +MEM_STATIC void BIT_flushBits(BIT_CStream_t* bitC); +MEM_STATIC size_t BIT_closeCStream(BIT_CStream_t* bitC); + +/* Start with initCStream, providing the size of buffer to write into. +* bitStream will never write outside of this buffer. +* `dstCapacity` must be >= sizeof(bitD->bitContainer), otherwise @return will be an error code. +* +* bits are first added to a local register. +* Local register is size_t, hence 64-bits on 64-bits systems, or 32-bits on 32-bits systems. +* Writing data into memory is an explicit operation, performed by the flushBits function. +* Hence keep track how many bits are potentially stored into local register to avoid register overflow. +* After a flushBits, a maximum of 7 bits might still be stored into local register. +* +* Avoid storing elements of more than 24 bits if you want compatibility with 32-bits bitstream readers. +* +* Last operation is to close the bitStream. +* The function returns the final size of CStream in bytes. +* If data couldn't fit into `dstBuffer`, it will return a 0 ( == not storable) +*/ + + +/*-******************************************** +* bitStream decoding API (read backward) +**********************************************/ +typedef struct { + size_t bitContainer; + unsigned bitsConsumed; + const char* ptr; + const char* start; + const char* limitPtr; +} BIT_DStream_t; + +typedef enum { BIT_DStream_unfinished = 0, + BIT_DStream_endOfBuffer = 1, + BIT_DStream_completed = 2, + BIT_DStream_overflow = 3 } BIT_DStream_status; /* result of BIT_reloadDStream() */ + /* 1,2,4,8 would be better for bitmap combinations, but slows down performance a bit ... :( */ + +MEM_STATIC size_t BIT_initDStream(BIT_DStream_t* bitD, const void* srcBuffer, size_t srcSize); +MEM_STATIC size_t BIT_readBits(BIT_DStream_t* bitD, unsigned nbBits); +MEM_STATIC BIT_DStream_status BIT_reloadDStream(BIT_DStream_t* bitD); +MEM_STATIC unsigned BIT_endOfDStream(const BIT_DStream_t* bitD); + + +/* Start by invoking BIT_initDStream(). +* A chunk of the bitStream is then stored into a local register. +* Local register size is 64-bits on 64-bits systems, 32-bits on 32-bits systems (size_t). +* You can then retrieve bitFields stored into the local register, **in reverse order**. +* Local register is explicitly reloaded from memory by the BIT_reloadDStream() method. +* A reload guarantee a minimum of ((8*sizeof(bitD->bitContainer))-7) bits when its result is BIT_DStream_unfinished. +* Otherwise, it can be less than that, so proceed accordingly. +* Checking if DStream has reached its end can be performed with BIT_endOfDStream(). +*/ + + +/*-**************************************** +* unsafe API +******************************************/ +MEM_STATIC void BIT_addBitsFast(BIT_CStream_t* bitC, size_t value, unsigned nbBits); +/* faster, but works only if value is "clean", meaning all high bits above nbBits are 0 */ + +MEM_STATIC void BIT_flushBitsFast(BIT_CStream_t* bitC); +/* unsafe version; does not check buffer overflow */ + +MEM_STATIC size_t BIT_readBitsFast(BIT_DStream_t* bitD, unsigned nbBits); +/* faster, but works only if nbBits >= 1 */ + + + +/*-************************************************************** +* Internal functions +****************************************************************/ +MEM_STATIC unsigned BIT_highbit32 (U32 val) +{ + assert(val != 0); + { +# if defined(_MSC_VER) /* Visual */ + unsigned long r=0; + return _BitScanReverse ( &r, val ) ? (unsigned)r : 0; +# elif defined(__GNUC__) && (__GNUC__ >= 3) /* Use GCC Intrinsic */ + return __builtin_clz (val) ^ 31; +# elif defined(__ICCARM__) /* IAR Intrinsic */ + return 31 - __CLZ(val); +# else /* Software version */ + static const unsigned DeBruijnClz[32] = { 0, 9, 1, 10, 13, 21, 2, 29, + 11, 14, 16, 18, 22, 25, 3, 30, + 8, 12, 20, 28, 15, 17, 24, 7, + 19, 27, 23, 6, 26, 5, 4, 31 }; + U32 v = val; + v |= v >> 1; + v |= v >> 2; + v |= v >> 4; + v |= v >> 8; + v |= v >> 16; + return DeBruijnClz[ (U32) (v * 0x07C4ACDDU) >> 27]; +# endif + } +} + +/*===== Local Constants =====*/ +static const unsigned BIT_mask[] = { + 0, 1, 3, 7, 0xF, 0x1F, + 0x3F, 0x7F, 0xFF, 0x1FF, 0x3FF, 0x7FF, + 0xFFF, 0x1FFF, 0x3FFF, 0x7FFF, 0xFFFF, 0x1FFFF, + 0x3FFFF, 0x7FFFF, 0xFFFFF, 0x1FFFFF, 0x3FFFFF, 0x7FFFFF, + 0xFFFFFF, 0x1FFFFFF, 0x3FFFFFF, 0x7FFFFFF, 0xFFFFFFF, 0x1FFFFFFF, + 0x3FFFFFFF, 0x7FFFFFFF}; /* up to 31 bits */ +#define BIT_MASK_SIZE (sizeof(BIT_mask) / sizeof(BIT_mask[0])) + +/*-************************************************************** +* bitStream encoding +****************************************************************/ +/*! BIT_initCStream() : + * `dstCapacity` must be > sizeof(size_t) + * @return : 0 if success, + * otherwise an error code (can be tested using ERR_isError()) */ +MEM_STATIC size_t BIT_initCStream(BIT_CStream_t* bitC, + void* startPtr, size_t dstCapacity) +{ + bitC->bitContainer = 0; + bitC->bitPos = 0; + bitC->startPtr = (char*)startPtr; + bitC->ptr = bitC->startPtr; + bitC->endPtr = bitC->startPtr + dstCapacity - sizeof(bitC->bitContainer); + if (dstCapacity <= sizeof(bitC->bitContainer)) return ERROR(dstSize_tooSmall); + return 0; +} + +/*! BIT_addBits() : + * can add up to 31 bits into `bitC`. + * Note : does not check for register overflow ! */ +MEM_STATIC void BIT_addBits(BIT_CStream_t* bitC, + size_t value, unsigned nbBits) +{ + MEM_STATIC_ASSERT(BIT_MASK_SIZE == 32); + assert(nbBits < BIT_MASK_SIZE); + assert(nbBits + bitC->bitPos < sizeof(bitC->bitContainer) * 8); + bitC->bitContainer |= (value & BIT_mask[nbBits]) << bitC->bitPos; + bitC->bitPos += nbBits; +} + +/*! BIT_addBitsFast() : + * works only if `value` is _clean_, + * meaning all high bits above nbBits are 0 */ +MEM_STATIC void BIT_addBitsFast(BIT_CStream_t* bitC, + size_t value, unsigned nbBits) +{ + assert((value>>nbBits) == 0); + assert(nbBits + bitC->bitPos < sizeof(bitC->bitContainer) * 8); + bitC->bitContainer |= value << bitC->bitPos; + bitC->bitPos += nbBits; +} + +/*! BIT_flushBitsFast() : + * assumption : bitContainer has not overflowed + * unsafe version; does not check buffer overflow */ +MEM_STATIC void BIT_flushBitsFast(BIT_CStream_t* bitC) +{ + size_t const nbBytes = bitC->bitPos >> 3; + assert(bitC->bitPos < sizeof(bitC->bitContainer) * 8); + assert(bitC->ptr <= bitC->endPtr); + MEM_writeLEST(bitC->ptr, bitC->bitContainer); + bitC->ptr += nbBytes; + bitC->bitPos &= 7; + bitC->bitContainer >>= nbBytes*8; +} + +/*! BIT_flushBits() : + * assumption : bitContainer has not overflowed + * safe version; check for buffer overflow, and prevents it. + * note : does not signal buffer overflow. + * overflow will be revealed later on using BIT_closeCStream() */ +MEM_STATIC void BIT_flushBits(BIT_CStream_t* bitC) +{ + size_t const nbBytes = bitC->bitPos >> 3; + assert(bitC->bitPos < sizeof(bitC->bitContainer) * 8); + assert(bitC->ptr <= bitC->endPtr); + MEM_writeLEST(bitC->ptr, bitC->bitContainer); + bitC->ptr += nbBytes; + if (bitC->ptr > bitC->endPtr) bitC->ptr = bitC->endPtr; + bitC->bitPos &= 7; + bitC->bitContainer >>= nbBytes*8; +} + +/*! BIT_closeCStream() : + * @return : size of CStream, in bytes, + * or 0 if it could not fit into dstBuffer */ +MEM_STATIC size_t BIT_closeCStream(BIT_CStream_t* bitC) +{ + BIT_addBitsFast(bitC, 1, 1); /* endMark */ + BIT_flushBits(bitC); + if (bitC->ptr >= bitC->endPtr) return 0; /* overflow detected */ + return (bitC->ptr - bitC->startPtr) + (bitC->bitPos > 0); +} + + +/*-******************************************************** +* bitStream decoding +**********************************************************/ +/*! BIT_initDStream() : + * Initialize a BIT_DStream_t. + * `bitD` : a pointer to an already allocated BIT_DStream_t structure. + * `srcSize` must be the *exact* size of the bitStream, in bytes. + * @return : size of stream (== srcSize), or an errorCode if a problem is detected + */ +MEM_STATIC size_t BIT_initDStream(BIT_DStream_t* bitD, const void* srcBuffer, size_t srcSize) +{ + if (srcSize < 1) { memset(bitD, 0, sizeof(*bitD)); return ERROR(srcSize_wrong); } + + bitD->start = (const char*)srcBuffer; + bitD->limitPtr = bitD->start + sizeof(bitD->bitContainer); + + if (srcSize >= sizeof(bitD->bitContainer)) { /* normal case */ + bitD->ptr = (const char*)srcBuffer + srcSize - sizeof(bitD->bitContainer); + bitD->bitContainer = MEM_readLEST(bitD->ptr); + { BYTE const lastByte = ((const BYTE*)srcBuffer)[srcSize-1]; + bitD->bitsConsumed = lastByte ? 8 - BIT_highbit32(lastByte) : 0; /* ensures bitsConsumed is always set */ + if (lastByte == 0) return ERROR(GENERIC); /* endMark not present */ } + } else { + bitD->ptr = bitD->start; + bitD->bitContainer = *(const BYTE*)(bitD->start); + switch(srcSize) + { + case 7: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[6]) << (sizeof(bitD->bitContainer)*8 - 16); + /* fall-through */ + + case 6: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[5]) << (sizeof(bitD->bitContainer)*8 - 24); + /* fall-through */ + + case 5: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[4]) << (sizeof(bitD->bitContainer)*8 - 32); + /* fall-through */ + + case 4: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[3]) << 24; + /* fall-through */ + + case 3: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[2]) << 16; + /* fall-through */ + + case 2: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[1]) << 8; + /* fall-through */ + + default: break; + } + { BYTE const lastByte = ((const BYTE*)srcBuffer)[srcSize-1]; + bitD->bitsConsumed = lastByte ? 8 - BIT_highbit32(lastByte) : 0; + if (lastByte == 0) return ERROR(corruption_detected); /* endMark not present */ + } + bitD->bitsConsumed += (U32)(sizeof(bitD->bitContainer) - srcSize)*8; + } + + return srcSize; +} + +MEM_STATIC size_t BIT_getUpperBits(size_t bitContainer, U32 const start) +{ + return bitContainer >> start; +} + +MEM_STATIC size_t BIT_getMiddleBits(size_t bitContainer, U32 const start, U32 const nbBits) +{ + U32 const regMask = sizeof(bitContainer)*8 - 1; + /* if start > regMask, bitstream is corrupted, and result is undefined */ + assert(nbBits < BIT_MASK_SIZE); + return (bitContainer >> (start & regMask)) & BIT_mask[nbBits]; +} + +MEM_STATIC size_t BIT_getLowerBits(size_t bitContainer, U32 const nbBits) +{ + assert(nbBits < BIT_MASK_SIZE); + return bitContainer & BIT_mask[nbBits]; +} + +/*! BIT_lookBits() : + * Provides next n bits from local register. + * local register is not modified. + * On 32-bits, maxNbBits==24. + * On 64-bits, maxNbBits==56. + * @return : value extracted */ +MEM_STATIC size_t BIT_lookBits(const BIT_DStream_t* bitD, U32 nbBits) +{ + /* arbitrate between double-shift and shift+mask */ +#if 1 + /* if bitD->bitsConsumed + nbBits > sizeof(bitD->bitContainer)*8, + * bitstream is likely corrupted, and result is undefined */ + return BIT_getMiddleBits(bitD->bitContainer, (sizeof(bitD->bitContainer)*8) - bitD->bitsConsumed - nbBits, nbBits); +#else + /* this code path is slower on my os-x laptop */ + U32 const regMask = sizeof(bitD->bitContainer)*8 - 1; + return ((bitD->bitContainer << (bitD->bitsConsumed & regMask)) >> 1) >> ((regMask-nbBits) & regMask); +#endif +} + +/*! BIT_lookBitsFast() : + * unsafe version; only works if nbBits >= 1 */ +MEM_STATIC size_t BIT_lookBitsFast(const BIT_DStream_t* bitD, U32 nbBits) +{ + U32 const regMask = sizeof(bitD->bitContainer)*8 - 1; + assert(nbBits >= 1); + return (bitD->bitContainer << (bitD->bitsConsumed & regMask)) >> (((regMask+1)-nbBits) & regMask); +} + +MEM_STATIC void BIT_skipBits(BIT_DStream_t* bitD, U32 nbBits) +{ + bitD->bitsConsumed += nbBits; +} + +/*! BIT_readBits() : + * Read (consume) next n bits from local register and update. + * Pay attention to not read more than nbBits contained into local register. + * @return : extracted value. */ +MEM_STATIC size_t BIT_readBits(BIT_DStream_t* bitD, unsigned nbBits) +{ + size_t const value = BIT_lookBits(bitD, nbBits); + BIT_skipBits(bitD, nbBits); + return value; +} + +/*! BIT_readBitsFast() : + * unsafe version; only works only if nbBits >= 1 */ +MEM_STATIC size_t BIT_readBitsFast(BIT_DStream_t* bitD, unsigned nbBits) +{ + size_t const value = BIT_lookBitsFast(bitD, nbBits); + assert(nbBits >= 1); + BIT_skipBits(bitD, nbBits); + return value; +} + +/*! BIT_reloadDStreamFast() : + * Similar to BIT_reloadDStream(), but with two differences: + * 1. bitsConsumed <= sizeof(bitD->bitContainer)*8 must hold! + * 2. Returns BIT_DStream_overflow when bitD->ptr < bitD->limitPtr, at this + * point you must use BIT_reloadDStream() to reload. + */ +MEM_STATIC BIT_DStream_status BIT_reloadDStreamFast(BIT_DStream_t* bitD) +{ + if (UNLIKELY(bitD->ptr < bitD->limitPtr)) + return BIT_DStream_overflow; + assert(bitD->bitsConsumed <= sizeof(bitD->bitContainer)*8); + bitD->ptr -= bitD->bitsConsumed >> 3; + bitD->bitsConsumed &= 7; + bitD->bitContainer = MEM_readLEST(bitD->ptr); + return BIT_DStream_unfinished; +} + +/*! BIT_reloadDStream() : + * Refill `bitD` from buffer previously set in BIT_initDStream() . + * This function is safe, it guarantees it will not read beyond src buffer. + * @return : status of `BIT_DStream_t` internal register. + * when status == BIT_DStream_unfinished, internal register is filled with at least 25 or 57 bits */ +MEM_STATIC BIT_DStream_status BIT_reloadDStream(BIT_DStream_t* bitD) +{ + if (bitD->bitsConsumed > (sizeof(bitD->bitContainer)*8)) /* overflow detected, like end of stream */ + return BIT_DStream_overflow; + + if (bitD->ptr >= bitD->limitPtr) { + return BIT_reloadDStreamFast(bitD); + } + if (bitD->ptr == bitD->start) { + if (bitD->bitsConsumed < sizeof(bitD->bitContainer)*8) return BIT_DStream_endOfBuffer; + return BIT_DStream_completed; + } + /* start < ptr < limitPtr */ + { U32 nbBytes = bitD->bitsConsumed >> 3; + BIT_DStream_status result = BIT_DStream_unfinished; + if (bitD->ptr - nbBytes < bitD->start) { + nbBytes = (U32)(bitD->ptr - bitD->start); /* ptr > start */ + result = BIT_DStream_endOfBuffer; + } + bitD->ptr -= nbBytes; + bitD->bitsConsumed -= nbBytes*8; + bitD->bitContainer = MEM_readLEST(bitD->ptr); /* reminder : srcSize > sizeof(bitD->bitContainer), otherwise bitD->ptr == bitD->start */ + return result; + } +} + +/*! BIT_endOfDStream() : + * @return : 1 if DStream has _exactly_ reached its end (all bits consumed). + */ +MEM_STATIC unsigned BIT_endOfDStream(const BIT_DStream_t* DStream) +{ + return ((DStream->ptr == DStream->start) && (DStream->bitsConsumed == sizeof(DStream->bitContainer)*8)); +} + +#if defined (__cplusplus) +} +#endif + +#endif /* BITSTREAM_H_MODULE */ diff --git a/module/zstd/lib/common/compiler.h b/module/zstd/lib/common/compiler.h new file mode 100644 index 000000000000..95e9483521d4 --- /dev/null +++ b/module/zstd/lib/common/compiler.h @@ -0,0 +1,175 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#ifndef ZSTD_COMPILER_H +#define ZSTD_COMPILER_H + +/*-******************************************************* +* Compiler specifics +*********************************************************/ +/* force inlining */ + +#if !defined(ZSTD_NO_INLINE) +#if (defined(__GNUC__) && !defined(__STRICT_ANSI__)) || defined(__cplusplus) || defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L /* C99 */ +# define INLINE_KEYWORD inline +#else +# define INLINE_KEYWORD +#endif + +#if defined(__GNUC__) || defined(__ICCARM__) +# define FORCE_INLINE_ATTR __attribute__((always_inline)) +#elif defined(_MSC_VER) +# define FORCE_INLINE_ATTR __forceinline +#else +# define FORCE_INLINE_ATTR +#endif + +#else + +#define INLINE_KEYWORD +#define FORCE_INLINE_ATTR + +#endif + +/** + * FORCE_INLINE_TEMPLATE is used to define C "templates", which take constant + * parameters. They must be inlined for the compiler to eliminate the constant + * branches. + */ +#define FORCE_INLINE_TEMPLATE static INLINE_KEYWORD FORCE_INLINE_ATTR +/** + * HINT_INLINE is used to help the compiler generate better code. It is *not* + * used for "templates", so it can be tweaked based on the compilers + * performance. + * + * gcc-4.8 and gcc-4.9 have been shown to benefit from leaving off the + * always_inline attribute. + * + * clang up to 5.0.0 (trunk) benefit tremendously from the always_inline + * attribute. + */ +#if !defined(__clang__) && defined(__GNUC__) && __GNUC__ >= 4 && __GNUC_MINOR__ >= 8 && __GNUC__ < 5 +# define HINT_INLINE static INLINE_KEYWORD +#else +# define HINT_INLINE static INLINE_KEYWORD FORCE_INLINE_ATTR +#endif + +/* UNUSED_ATTR tells the compiler it is okay if the function is unused. */ +#if defined(__GNUC__) +# define UNUSED_ATTR __attribute__((unused)) +#else +# define UNUSED_ATTR +#endif + +/* force no inlining */ +#ifdef _MSC_VER +# define FORCE_NOINLINE static __declspec(noinline) +#else +# if defined(__GNUC__) || defined(__ICCARM__) +# define FORCE_NOINLINE static __attribute__((__noinline__)) +# else +# define FORCE_NOINLINE static +# endif +#endif + +/* target attribute */ +#ifndef __has_attribute + #define __has_attribute(x) 0 /* Compatibility with non-clang compilers. */ +#endif +#if defined(__GNUC__) || defined(__ICCARM__) +# define TARGET_ATTRIBUTE(target) __attribute__((__target__(target))) +#else +# define TARGET_ATTRIBUTE(target) +#endif + +/* Enable runtime BMI2 dispatch based on the CPU. + * Enabled for clang & gcc >=4.8 on x86 when BMI2 isn't enabled by default. + */ +#ifndef DYNAMIC_BMI2 + #if ((defined(__clang__) && __has_attribute(__target__)) \ + || (defined(__GNUC__) \ + && (__GNUC__ >= 5 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 8)))) \ + && (defined(__x86_64__) || defined(_M_X86)) \ + && !defined(__BMI2__) + # define DYNAMIC_BMI2 1 + #else + # define DYNAMIC_BMI2 0 + #endif +#endif + +/* prefetch + * can be disabled, by declaring NO_PREFETCH build macro */ +#if defined(NO_PREFETCH) +# define PREFETCH_L1(ptr) (void)(ptr) /* disabled */ +# define PREFETCH_L2(ptr) (void)(ptr) /* disabled */ +#else +# if defined(_MSC_VER) && (defined(_M_X64) || defined(_M_I86)) /* _mm_prefetch() is not defined outside of x86/x64 */ +# include /* https://msdn.microsoft.com/fr-fr/library/84szxsww(v=vs.90).aspx */ +# define PREFETCH_L1(ptr) _mm_prefetch((const char*)(ptr), _MM_HINT_T0) +# define PREFETCH_L2(ptr) _mm_prefetch((const char*)(ptr), _MM_HINT_T1) +# elif defined(__aarch64__) +# define PREFETCH_L1(ptr) __asm__ __volatile__("prfm pldl1keep, %0" ::"Q"(*(ptr))) +# define PREFETCH_L2(ptr) __asm__ __volatile__("prfm pldl2keep, %0" ::"Q"(*(ptr))) +# elif defined(__GNUC__) && ( (__GNUC__ >= 4) || ( (__GNUC__ == 3) && (__GNUC_MINOR__ >= 1) ) ) +# define PREFETCH_L1(ptr) __builtin_prefetch((ptr), 0 /* rw==read */, 3 /* locality */) +# define PREFETCH_L2(ptr) __builtin_prefetch((ptr), 0 /* rw==read */, 2 /* locality */) +# else +# define PREFETCH_L1(ptr) (void)(ptr) /* disabled */ +# define PREFETCH_L2(ptr) (void)(ptr) /* disabled */ +# endif +#endif /* NO_PREFETCH */ + +#define CACHELINE_SIZE 64 + +#define PREFETCH_AREA(p, s) { \ + const char* const _ptr = (const char*)(p); \ + size_t const _size = (size_t)(s); \ + size_t _pos; \ + for (_pos=0; _pos<_size; _pos+=CACHELINE_SIZE) { \ + PREFETCH_L2(_ptr + _pos); \ + } \ +} + +/* vectorization + * older GCC (pre gcc-4.3 picked as the cutoff) uses a different syntax */ +#if !defined(__INTEL_COMPILER) && !defined(__clang__) && defined(__GNUC__) +# if (__GNUC__ == 4 && __GNUC_MINOR__ > 3) || (__GNUC__ >= 5) +# define DONT_VECTORIZE __attribute__((optimize("no-tree-vectorize"))) +# else +# define DONT_VECTORIZE _Pragma("GCC optimize(\"no-tree-vectorize\")") +# endif +#else +# define DONT_VECTORIZE +#endif + +/* Tell the compiler that a branch is likely or unlikely. + * Only use these macros if it causes the compiler to generate better code. + * If you can remove a LIKELY/UNLIKELY annotation without speed changes in gcc + * and clang, please do. + */ +#if defined(__GNUC__) +#define LIKELY(x) (__builtin_expect((x), 1)) +#define UNLIKELY(x) (__builtin_expect((x), 0)) +#else +#define LIKELY(x) (x) +#define UNLIKELY(x) (x) +#endif + +/* disable warnings */ +#ifdef _MSC_VER /* Visual Studio */ +# include /* For Visual 2005 */ +# pragma warning(disable : 4100) /* disable: C4100: unreferenced formal parameter */ +# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */ +# pragma warning(disable : 4204) /* disable: C4204: non-constant aggregate initializer */ +# pragma warning(disable : 4214) /* disable: C4214: non-int bitfields */ +# pragma warning(disable : 4324) /* disable: C4324: padded structure */ +#endif + +#endif /* ZSTD_COMPILER_H */ diff --git a/module/zstd/lib/common/cpu.h b/module/zstd/lib/common/cpu.h new file mode 100644 index 000000000000..6e8a974f62d7 --- /dev/null +++ b/module/zstd/lib/common/cpu.h @@ -0,0 +1,215 @@ +/* + * Copyright (c) 2018-2020, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#ifndef ZSTD_COMMON_CPU_H +#define ZSTD_COMMON_CPU_H + +/** + * Implementation taken from folly/CpuId.h + * https://github.com/facebook/folly/blob/master/folly/CpuId.h + */ + +#include + +#include "mem.h" + +#ifdef _MSC_VER +#include +#endif + +typedef struct { + U32 f1c; + U32 f1d; + U32 f7b; + U32 f7c; +} ZSTD_cpuid_t; + +MEM_STATIC ZSTD_cpuid_t ZSTD_cpuid(void) { + U32 f1c = 0; + U32 f1d = 0; + U32 f7b = 0; + U32 f7c = 0; +#if defined(_MSC_VER) && (defined(_M_X64) || defined(_M_IX86)) + int reg[4]; + __cpuid((int*)reg, 0); + { + int const n = reg[0]; + if (n >= 1) { + __cpuid((int*)reg, 1); + f1c = (U32)reg[2]; + f1d = (U32)reg[3]; + } + if (n >= 7) { + __cpuidex((int*)reg, 7, 0); + f7b = (U32)reg[1]; + f7c = (U32)reg[2]; + } + } +#elif defined(__i386__) && defined(__PIC__) && !defined(__clang__) && defined(__GNUC__) + /* The following block like the normal cpuid branch below, but gcc + * reserves ebx for use of its pic register so we must specially + * handle the save and restore to avoid clobbering the register + */ + U32 n; + __asm__( + "pushl %%ebx\n\t" + "cpuid\n\t" + "popl %%ebx\n\t" + : "=a"(n) + : "a"(0) + : "ecx", "edx"); + if (n >= 1) { + U32 f1a; + __asm__( + "pushl %%ebx\n\t" + "cpuid\n\t" + "popl %%ebx\n\t" + : "=a"(f1a), "=c"(f1c), "=d"(f1d) + : "a"(1)); + } + if (n >= 7) { + __asm__( + "pushl %%ebx\n\t" + "cpuid\n\t" + "movl %%ebx, %%eax\n\t" + "popl %%ebx" + : "=a"(f7b), "=c"(f7c) + : "a"(7), "c"(0) + : "edx"); + } +#elif defined(__x86_64__) || defined(_M_X64) || defined(__i386__) + U32 n; + __asm__("cpuid" : "=a"(n) : "a"(0) : "ebx", "ecx", "edx"); + if (n >= 1) { + U32 f1a; + __asm__("cpuid" : "=a"(f1a), "=c"(f1c), "=d"(f1d) : "a"(1) : "ebx"); + } + if (n >= 7) { + U32 f7a; + __asm__("cpuid" + : "=a"(f7a), "=b"(f7b), "=c"(f7c) + : "a"(7), "c"(0) + : "edx"); + } +#endif + { + ZSTD_cpuid_t cpuid; + cpuid.f1c = f1c; + cpuid.f1d = f1d; + cpuid.f7b = f7b; + cpuid.f7c = f7c; + return cpuid; + } +} + +#define X(name, r, bit) \ + MEM_STATIC int ZSTD_cpuid_##name(ZSTD_cpuid_t const cpuid) { \ + return ((cpuid.r) & (1U << bit)) != 0; \ + } + +/* cpuid(1): Processor Info and Feature Bits. */ +#define C(name, bit) X(name, f1c, bit) + C(sse3, 0) + C(pclmuldq, 1) + C(dtes64, 2) + C(monitor, 3) + C(dscpl, 4) + C(vmx, 5) + C(smx, 6) + C(eist, 7) + C(tm2, 8) + C(ssse3, 9) + C(cnxtid, 10) + C(fma, 12) + C(cx16, 13) + C(xtpr, 14) + C(pdcm, 15) + C(pcid, 17) + C(dca, 18) + C(sse41, 19) + C(sse42, 20) + C(x2apic, 21) + C(movbe, 22) + C(popcnt, 23) + C(tscdeadline, 24) + C(aes, 25) + C(xsave, 26) + C(osxsave, 27) + C(avx, 28) + C(f16c, 29) + C(rdrand, 30) +#undef C +#define D(name, bit) X(name, f1d, bit) + D(fpu, 0) + D(vme, 1) + D(de, 2) + D(pse, 3) + D(tsc, 4) + D(msr, 5) + D(pae, 6) + D(mce, 7) + D(cx8, 8) + D(apic, 9) + D(sep, 11) + D(mtrr, 12) + D(pge, 13) + D(mca, 14) + D(cmov, 15) + D(pat, 16) + D(pse36, 17) + D(psn, 18) + D(clfsh, 19) + D(ds, 21) + D(acpi, 22) + D(mmx, 23) + D(fxsr, 24) + D(sse, 25) + D(sse2, 26) + D(ss, 27) + D(htt, 28) + D(tm, 29) + D(pbe, 31) +#undef D + +/* cpuid(7): Extended Features. */ +#define B(name, bit) X(name, f7b, bit) + B(bmi1, 3) + B(hle, 4) + B(avx2, 5) + B(smep, 7) + B(bmi2, 8) + B(erms, 9) + B(invpcid, 10) + B(rtm, 11) + B(mpx, 14) + B(avx512f, 16) + B(avx512dq, 17) + B(rdseed, 18) + B(adx, 19) + B(smap, 20) + B(avx512ifma, 21) + B(pcommit, 22) + B(clflushopt, 23) + B(clwb, 24) + B(avx512pf, 26) + B(avx512er, 27) + B(avx512cd, 28) + B(sha, 29) + B(avx512bw, 30) + B(avx512vl, 31) +#undef B +#define C(name, bit) X(name, f7c, bit) + C(prefetchwt1, 0) + C(avx512vbmi, 1) +#undef C + +#undef X + +#endif /* ZSTD_COMMON_CPU_H */ diff --git a/module/zstd/lib/common/debug.h b/module/zstd/lib/common/debug.h new file mode 100644 index 000000000000..ac6224888d8b --- /dev/null +++ b/module/zstd/lib/common/debug.h @@ -0,0 +1,114 @@ +/* ****************************************************************** + * debug + * Part of FSE library + * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. + * + * You can contact the author at : + * - Source repository : https://github.com/Cyan4973/FiniteStateEntropy + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. +****************************************************************** */ + + +/* + * The purpose of this header is to enable debug functions. + * They regroup assert(), DEBUGLOG() and RAWLOG() for run-time, + * and DEBUG_STATIC_ASSERT() for compile-time. + * + * By default, DEBUGLEVEL==0, which means run-time debug is disabled. + * + * Level 1 enables assert() only. + * Starting level 2, traces can be generated and pushed to stderr. + * The higher the level, the more verbose the traces. + * + * It's possible to dynamically adjust level using variable g_debug_level, + * which is only declared if DEBUGLEVEL>=2, + * and is a global variable, not multi-thread protected (use with care) + */ + +#ifndef DEBUG_H_12987983217 +#define DEBUG_H_12987983217 + +#if defined (__cplusplus) +extern "C" { +#endif + + +/* static assert is triggered at compile time, leaving no runtime artefact. + * static assert only works with compile-time constants. + * Also, this variant can only be used inside a function. */ +#define DEBUG_STATIC_ASSERT(c) (void)sizeof(char[(c) ? 1 : -1]) + + +/* DEBUGLEVEL is expected to be defined externally, + * typically through compiler command line. + * Value must be a number. */ +#ifndef DEBUGLEVEL +# define DEBUGLEVEL 0 +#endif + + +/* DEBUGFILE can be defined externally, + * typically through compiler command line. + * note : currently useless. + * Value must be stderr or stdout */ +#ifndef DEBUGFILE +# define DEBUGFILE stderr +#endif + + +/* recommended values for DEBUGLEVEL : + * 0 : release mode, no debug, all run-time checks disabled + * 1 : enables assert() only, no display + * 2 : reserved, for currently active debug path + * 3 : events once per object lifetime (CCtx, CDict, etc.) + * 4 : events once per frame + * 5 : events once per block + * 6 : events once per sequence (verbose) + * 7+: events at every position (*very* verbose) + * + * It's generally inconvenient to output traces > 5. + * In which case, it's possible to selectively trigger high verbosity levels + * by modifying g_debug_level. + */ + +#if (DEBUGLEVEL>=1) +# include +#else +# ifndef assert /* assert may be already defined, due to prior #include */ +# define assert(condition) ((void)0) /* disable assert (default) */ +# endif +#endif + +#if (DEBUGLEVEL>=2) +# include +extern int g_debuglevel; /* the variable is only declared, + it actually lives in debug.c, + and is shared by the whole process. + It's not thread-safe. + It's useful when enabling very verbose levels + on selective conditions (such as position in src) */ + +# define RAWLOG(l, ...) { \ + if (l<=g_debuglevel) { \ + fprintf(stderr, __VA_ARGS__); \ + } } +# define DEBUGLOG(l, ...) { \ + if (l<=g_debuglevel) { \ + fprintf(stderr, __FILE__ ": " __VA_ARGS__); \ + fprintf(stderr, " \n"); \ + } } +#else +# define RAWLOG(l, ...) {} /* disabled */ +# define DEBUGLOG(l, ...) {} /* disabled */ +#endif + + +#if defined (__cplusplus) +} +#endif + +#endif /* DEBUG_H_12987983217 */ diff --git a/module/zstd/lib/common/entropy_common.c b/module/zstd/lib/common/entropy_common.c new file mode 100644 index 000000000000..9d3e4e8e36ab --- /dev/null +++ b/module/zstd/lib/common/entropy_common.c @@ -0,0 +1,216 @@ +/* ****************************************************************** + * Common functions of New Generation Entropy library + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * + * You can contact the author at : + * - FSE+HUF source repository : https://github.com/Cyan4973/FiniteStateEntropy + * - Public forum : https://groups.google.com/forum/#!forum/lz4c + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. +****************************************************************** */ + +/* ************************************* +* Dependencies +***************************************/ +#include "mem.h" +#include "error_private.h" /* ERR_*, ERROR */ +#define FSE_STATIC_LINKING_ONLY /* FSE_MIN_TABLELOG */ +#include "fse.h" +#define HUF_STATIC_LINKING_ONLY /* HUF_TABLELOG_ABSOLUTEMAX */ +#include "huf.h" + + +/*=== Version ===*/ +unsigned FSE_versionNumber(void) { return FSE_VERSION_NUMBER; } + + +/*=== Error Management ===*/ +unsigned FSE_isError(size_t code) { return ERR_isError(code); } +const char* FSE_getErrorName(size_t code) { return ERR_getErrorName(code); } + +unsigned HUF_isError(size_t code) { return ERR_isError(code); } +const char* HUF_getErrorName(size_t code) { return ERR_getErrorName(code); } + + +/*-************************************************************** +* FSE NCount encoding-decoding +****************************************************************/ +size_t FSE_readNCount (short* normalizedCounter, unsigned* maxSVPtr, unsigned* tableLogPtr, + const void* headerBuffer, size_t hbSize) +{ + const BYTE* const istart = (const BYTE*) headerBuffer; + const BYTE* const iend = istart + hbSize; + const BYTE* ip = istart; + int nbBits; + int remaining; + int threshold; + U32 bitStream; + int bitCount; + unsigned charnum = 0; + int previous0 = 0; + + if (hbSize < 4) { + /* This function only works when hbSize >= 4 */ + char buffer[4]; + memset(buffer, 0, sizeof(buffer)); + memcpy(buffer, headerBuffer, hbSize); + { size_t const countSize = FSE_readNCount(normalizedCounter, maxSVPtr, tableLogPtr, + buffer, sizeof(buffer)); + if (FSE_isError(countSize)) return countSize; + if (countSize > hbSize) return ERROR(corruption_detected); + return countSize; + } } + assert(hbSize >= 4); + + /* init */ + memset(normalizedCounter, 0, (*maxSVPtr+1) * sizeof(normalizedCounter[0])); /* all symbols not present in NCount have a frequency of 0 */ + bitStream = MEM_readLE32(ip); + nbBits = (bitStream & 0xF) + FSE_MIN_TABLELOG; /* extract tableLog */ + if (nbBits > FSE_TABLELOG_ABSOLUTE_MAX) return ERROR(tableLog_tooLarge); + bitStream >>= 4; + bitCount = 4; + *tableLogPtr = nbBits; + remaining = (1<1) & (charnum<=*maxSVPtr)) { + if (previous0) { + unsigned n0 = charnum; + while ((bitStream & 0xFFFF) == 0xFFFF) { + n0 += 24; + if (ip < iend-5) { + ip += 2; + bitStream = MEM_readLE32(ip) >> bitCount; + } else { + bitStream >>= 16; + bitCount += 16; + } } + while ((bitStream & 3) == 3) { + n0 += 3; + bitStream >>= 2; + bitCount += 2; + } + n0 += bitStream & 3; + bitCount += 2; + if (n0 > *maxSVPtr) return ERROR(maxSymbolValue_tooSmall); + while (charnum < n0) normalizedCounter[charnum++] = 0; + if ((ip <= iend-7) || (ip + (bitCount>>3) <= iend-4)) { + assert((bitCount >> 3) <= 3); /* For first condition to work */ + ip += bitCount>>3; + bitCount &= 7; + bitStream = MEM_readLE32(ip) >> bitCount; + } else { + bitStream >>= 2; + } } + { int const max = (2*threshold-1) - remaining; + int count; + + if ((bitStream & (threshold-1)) < (U32)max) { + count = bitStream & (threshold-1); + bitCount += nbBits-1; + } else { + count = bitStream & (2*threshold-1); + if (count >= threshold) count -= max; + bitCount += nbBits; + } + + count--; /* extra accuracy */ + remaining -= count < 0 ? -count : count; /* -1 means +1 */ + normalizedCounter[charnum++] = (short)count; + previous0 = !count; + while (remaining < threshold) { + nbBits--; + threshold >>= 1; + } + + if ((ip <= iend-7) || (ip + (bitCount>>3) <= iend-4)) { + ip += bitCount>>3; + bitCount &= 7; + } else { + bitCount -= (int)(8 * (iend - 4 - ip)); + ip = iend - 4; + } + bitStream = MEM_readLE32(ip) >> (bitCount & 31); + } } /* while ((remaining>1) & (charnum<=*maxSVPtr)) */ + if (remaining != 1) return ERROR(corruption_detected); + if (bitCount > 32) return ERROR(corruption_detected); + *maxSVPtr = charnum-1; + + ip += (bitCount+7)>>3; + return ip-istart; +} + + +/*! HUF_readStats() : + Read compact Huffman tree, saved by HUF_writeCTable(). + `huffWeight` is destination buffer. + `rankStats` is assumed to be a table of at least HUF_TABLELOG_MAX U32. + @return : size read from `src` , or an error Code . + Note : Needed by HUF_readCTable() and HUF_readDTableX?() . +*/ +size_t HUF_readStats(BYTE* huffWeight, size_t hwSize, U32* rankStats, + U32* nbSymbolsPtr, U32* tableLogPtr, + const void* src, size_t srcSize) +{ + U32 weightTotal; + const BYTE* ip = (const BYTE*) src; + size_t iSize; + size_t oSize; + + if (!srcSize) return ERROR(srcSize_wrong); + iSize = ip[0]; + /* memset(huffWeight, 0, hwSize); *//* is not necessary, even though some analyzer complain ... */ + + if (iSize >= 128) { /* special header */ + oSize = iSize - 127; + iSize = ((oSize+1)/2); + if (iSize+1 > srcSize) return ERROR(srcSize_wrong); + if (oSize >= hwSize) return ERROR(corruption_detected); + ip += 1; + { U32 n; + for (n=0; n> 4; + huffWeight[n+1] = ip[n/2] & 15; + } } } + else { /* header compressed with FSE (normal case) */ + FSE_DTable fseWorkspace[FSE_DTABLE_SIZE_U32(6)]; /* 6 is max possible tableLog for HUF header (maybe even 5, to be tested) */ + if (iSize+1 > srcSize) return ERROR(srcSize_wrong); + oSize = FSE_decompress_wksp(huffWeight, hwSize-1, ip+1, iSize, fseWorkspace, 6); /* max (hwSize-1) values decoded, as last one is implied */ + if (FSE_isError(oSize)) return oSize; + } + + /* collect weight stats */ + memset(rankStats, 0, (HUF_TABLELOG_MAX + 1) * sizeof(U32)); + weightTotal = 0; + { U32 n; for (n=0; n= HUF_TABLELOG_MAX) return ERROR(corruption_detected); + rankStats[huffWeight[n]]++; + weightTotal += (1 << huffWeight[n]) >> 1; + } } + if (weightTotal == 0) return ERROR(corruption_detected); + + /* get last non-null symbol weight (implied, total must be 2^n) */ + { U32 const tableLog = BIT_highbit32(weightTotal) + 1; + if (tableLog > HUF_TABLELOG_MAX) return ERROR(corruption_detected); + *tableLogPtr = tableLog; + /* determine last weight */ + { U32 const total = 1 << tableLog; + U32 const rest = total - weightTotal; + U32 const verif = 1 << BIT_highbit32(rest); + U32 const lastWeight = BIT_highbit32(rest) + 1; + if (verif != rest) return ERROR(corruption_detected); /* last value must be a clean power of 2 */ + huffWeight[oSize] = (BYTE)lastWeight; + rankStats[lastWeight]++; + } } + + /* check tree construction validity */ + if ((rankStats[1] < 2) || (rankStats[1] & 1)) return ERROR(corruption_detected); /* by construction : at least 2 elts of rank 1, must be even */ + + /* results */ + *nbSymbolsPtr = (U32)(oSize+1); + return iSize+1; +} diff --git a/module/zstd/lib/common/error_private.c b/module/zstd/lib/common/error_private.c new file mode 100644 index 000000000000..cd437529c12b --- /dev/null +++ b/module/zstd/lib/common/error_private.c @@ -0,0 +1,55 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +/* The purpose of this file is to have a single list of error strings embedded in binary */ + +#include "error_private.h" + +const char* ERR_getErrorString(ERR_enum code) +{ +#ifdef ZSTD_STRIP_ERROR_STRINGS + (void)code; + return "Error strings stripped"; +#else + static const char* const notErrorCode = "Unspecified error code"; + switch( code ) + { + case PREFIX(no_error): return "No error detected"; + case PREFIX(GENERIC): return "Error (generic)"; + case PREFIX(prefix_unknown): return "Unknown frame descriptor"; + case PREFIX(version_unsupported): return "Version not supported"; + case PREFIX(frameParameter_unsupported): return "Unsupported frame parameter"; + case PREFIX(frameParameter_windowTooLarge): return "Frame requires too much memory for decoding"; + case PREFIX(corruption_detected): return "Corrupted block detected"; + case PREFIX(checksum_wrong): return "Restored data doesn't match checksum"; + case PREFIX(parameter_unsupported): return "Unsupported parameter"; + case PREFIX(parameter_outOfBound): return "Parameter is out of bound"; + case PREFIX(init_missing): return "Context should be init first"; + case PREFIX(memory_allocation): return "Allocation error : not enough memory"; + case PREFIX(workSpace_tooSmall): return "workSpace buffer is not large enough"; + case PREFIX(stage_wrong): return "Operation not authorized at current processing stage"; + case PREFIX(tableLog_tooLarge): return "tableLog requires too much memory : unsupported"; + case PREFIX(maxSymbolValue_tooLarge): return "Unsupported max Symbol Value : too large"; + case PREFIX(maxSymbolValue_tooSmall): return "Specified maxSymbolValue is too small"; + case PREFIX(dictionary_corrupted): return "Dictionary is corrupted"; + case PREFIX(dictionary_wrong): return "Dictionary mismatch"; + case PREFIX(dictionaryCreation_failed): return "Cannot create Dictionary from provided samples"; + case PREFIX(dstSize_tooSmall): return "Destination buffer is too small"; + case PREFIX(srcSize_wrong): return "Src size is incorrect"; + case PREFIX(dstBuffer_null): return "Operation on NULL destination buffer"; + /* following error codes are not stable and may be removed or changed in a future version */ + case PREFIX(frameIndex_tooLarge): return "Frame index is too large"; + case PREFIX(seekableIO): return "An I/O error occurred when reading/seeking"; + case PREFIX(dstBuffer_wrong): return "Destination buffer is wrong"; + case PREFIX(maxCode): + default: return notErrorCode; + } +#endif +} diff --git a/module/zstd/lib/common/error_private.h b/module/zstd/lib/common/error_private.h new file mode 100644 index 000000000000..982cf8e9fe6f --- /dev/null +++ b/module/zstd/lib/common/error_private.h @@ -0,0 +1,80 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +/* Note : this module is expected to remain private, do not expose it */ + +#ifndef ERROR_H_MODULE +#define ERROR_H_MODULE + +#if defined (__cplusplus) +extern "C" { +#endif + + +/* **************************************** +* Dependencies +******************************************/ +#include /* size_t */ +#include "zstd_errors.h" /* enum list */ + + +/* **************************************** +* Compiler-specific +******************************************/ +#if defined(__GNUC__) +# define ERR_STATIC static __attribute__((unused)) +#elif defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) +# define ERR_STATIC static inline +#elif defined(_MSC_VER) +# define ERR_STATIC static __inline +#else +# define ERR_STATIC static /* this version may generate warnings for unused static functions; disable the relevant warning */ +#endif + + +/*-**************************************** +* Customization (error_public.h) +******************************************/ +typedef ZSTD_ErrorCode ERR_enum; +#define PREFIX(name) ZSTD_error_##name + + +/*-**************************************** +* Error codes handling +******************************************/ +#undef ERROR /* already defined on Visual Studio */ +#define ERROR(name) ZSTD_ERROR(name) +#define ZSTD_ERROR(name) ((size_t)-PREFIX(name)) + +ERR_STATIC unsigned ERR_isError(size_t code) { return (code > ERROR(maxCode)); } + +ERR_STATIC ERR_enum ERR_getErrorCode(size_t code) { if (!ERR_isError(code)) return (ERR_enum)0; return (ERR_enum) (0-code); } + +/* check and forward error code */ +#define CHECK_V_F(e, f) size_t const e = f; if (ERR_isError(e)) return e +#define CHECK_F(f) { CHECK_V_F(_var_err__, f); } + + +/*-**************************************** +* Error Strings +******************************************/ + +const char* ERR_getErrorString(ERR_enum code); /* error_private.c */ + +ERR_STATIC const char* ERR_getErrorName(size_t code) +{ + return ERR_getErrorString(ERR_getErrorCode(code)); +} + +#if defined (__cplusplus) +} +#endif + +#endif /* ERROR_H_MODULE */ diff --git a/module/zstd/lib/common/fse.h b/module/zstd/lib/common/fse.h new file mode 100644 index 000000000000..ff54e70ea75c --- /dev/null +++ b/module/zstd/lib/common/fse.h @@ -0,0 +1,688 @@ +/* ****************************************************************** + * FSE : Finite State Entropy codec + * Public Prototypes declaration + * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. + * + * You can contact the author at : + * - Source repository : https://github.com/Cyan4973/FiniteStateEntropy + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. +****************************************************************** */ + +#if defined (__cplusplus) +extern "C" { +#endif + +#ifndef FSE_H +#define FSE_H + + +/*-***************************************** +* Dependencies +******************************************/ +#include /* size_t, ptrdiff_t */ + + +/*-***************************************** +* FSE_PUBLIC_API : control library symbols visibility +******************************************/ +#if defined(FSE_DLL_EXPORT) && (FSE_DLL_EXPORT==1) && defined(__GNUC__) && (__GNUC__ >= 4) +# define FSE_PUBLIC_API __attribute__ ((visibility ("default"))) +#elif defined(FSE_DLL_EXPORT) && (FSE_DLL_EXPORT==1) /* Visual expected */ +# define FSE_PUBLIC_API __declspec(dllexport) +#elif defined(FSE_DLL_IMPORT) && (FSE_DLL_IMPORT==1) +# define FSE_PUBLIC_API __declspec(dllimport) /* It isn't required but allows to generate better code, saving a function pointer load from the IAT and an indirect jump.*/ +#else +# define FSE_PUBLIC_API +#endif + +/*------ Version ------*/ +#define FSE_VERSION_MAJOR 0 +#define FSE_VERSION_MINOR 9 +#define FSE_VERSION_RELEASE 0 + +#define FSE_LIB_VERSION FSE_VERSION_MAJOR.FSE_VERSION_MINOR.FSE_VERSION_RELEASE +#define FSE_QUOTE(str) #str +#define FSE_EXPAND_AND_QUOTE(str) FSE_QUOTE(str) +#define FSE_VERSION_STRING FSE_EXPAND_AND_QUOTE(FSE_LIB_VERSION) + +#define FSE_VERSION_NUMBER (FSE_VERSION_MAJOR *100*100 + FSE_VERSION_MINOR *100 + FSE_VERSION_RELEASE) +FSE_PUBLIC_API unsigned FSE_versionNumber(void); /**< library version number; to be used when checking dll version */ + + +/*-**************************************** +* FSE simple functions +******************************************/ +/*! FSE_compress() : + Compress content of buffer 'src', of size 'srcSize', into destination buffer 'dst'. + 'dst' buffer must be already allocated. Compression runs faster is dstCapacity >= FSE_compressBound(srcSize). + @return : size of compressed data (<= dstCapacity). + Special values : if return == 0, srcData is not compressible => Nothing is stored within dst !!! + if return == 1, srcData is a single byte symbol * srcSize times. Use RLE compression instead. + if FSE_isError(return), compression failed (more details using FSE_getErrorName()) +*/ +FSE_PUBLIC_API size_t FSE_compress(void* dst, size_t dstCapacity, + const void* src, size_t srcSize); + +/*! FSE_decompress(): + Decompress FSE data from buffer 'cSrc', of size 'cSrcSize', + into already allocated destination buffer 'dst', of size 'dstCapacity'. + @return : size of regenerated data (<= maxDstSize), + or an error code, which can be tested using FSE_isError() . + + ** Important ** : FSE_decompress() does not decompress non-compressible nor RLE data !!! + Why ? : making this distinction requires a header. + Header management is intentionally delegated to the user layer, which can better manage special cases. +*/ +FSE_PUBLIC_API size_t FSE_decompress(void* dst, size_t dstCapacity, + const void* cSrc, size_t cSrcSize); + + +/*-***************************************** +* Tool functions +******************************************/ +FSE_PUBLIC_API size_t FSE_compressBound(size_t size); /* maximum compressed size */ + +/* Error Management */ +FSE_PUBLIC_API unsigned FSE_isError(size_t code); /* tells if a return value is an error code */ +FSE_PUBLIC_API const char* FSE_getErrorName(size_t code); /* provides error code string (useful for debugging) */ + + +/*-***************************************** +* FSE advanced functions +******************************************/ +/*! FSE_compress2() : + Same as FSE_compress(), but allows the selection of 'maxSymbolValue' and 'tableLog' + Both parameters can be defined as '0' to mean : use default value + @return : size of compressed data + Special values : if return == 0, srcData is not compressible => Nothing is stored within cSrc !!! + if return == 1, srcData is a single byte symbol * srcSize times. Use RLE compression. + if FSE_isError(return), it's an error code. +*/ +FSE_PUBLIC_API size_t FSE_compress2 (void* dst, size_t dstSize, const void* src, size_t srcSize, unsigned maxSymbolValue, unsigned tableLog); + + +/*-***************************************** +* FSE detailed API +******************************************/ +/*! +FSE_compress() does the following: +1. count symbol occurrence from source[] into table count[] (see hist.h) +2. normalize counters so that sum(count[]) == Power_of_2 (2^tableLog) +3. save normalized counters to memory buffer using writeNCount() +4. build encoding table 'CTable' from normalized counters +5. encode the data stream using encoding table 'CTable' + +FSE_decompress() does the following: +1. read normalized counters with readNCount() +2. build decoding table 'DTable' from normalized counters +3. decode the data stream using decoding table 'DTable' + +The following API allows targeting specific sub-functions for advanced tasks. +For example, it's possible to compress several blocks using the same 'CTable', +or to save and provide normalized distribution using external method. +*/ + +/* *** COMPRESSION *** */ + +/*! FSE_optimalTableLog(): + dynamically downsize 'tableLog' when conditions are met. + It saves CPU time, by using smaller tables, while preserving or even improving compression ratio. + @return : recommended tableLog (necessarily <= 'maxTableLog') */ +FSE_PUBLIC_API unsigned FSE_optimalTableLog(unsigned maxTableLog, size_t srcSize, unsigned maxSymbolValue); + +/*! FSE_normalizeCount(): + normalize counts so that sum(count[]) == Power_of_2 (2^tableLog) + 'normalizedCounter' is a table of short, of minimum size (maxSymbolValue+1). + @return : tableLog, + or an errorCode, which can be tested using FSE_isError() */ +FSE_PUBLIC_API size_t FSE_normalizeCount(short* normalizedCounter, unsigned tableLog, + const unsigned* count, size_t srcSize, unsigned maxSymbolValue); + +/*! FSE_NCountWriteBound(): + Provides the maximum possible size of an FSE normalized table, given 'maxSymbolValue' and 'tableLog'. + Typically useful for allocation purpose. */ +FSE_PUBLIC_API size_t FSE_NCountWriteBound(unsigned maxSymbolValue, unsigned tableLog); + +/*! FSE_writeNCount(): + Compactly save 'normalizedCounter' into 'buffer'. + @return : size of the compressed table, + or an errorCode, which can be tested using FSE_isError(). */ +FSE_PUBLIC_API size_t FSE_writeNCount (void* buffer, size_t bufferSize, + const short* normalizedCounter, + unsigned maxSymbolValue, unsigned tableLog); + +/*! Constructor and Destructor of FSE_CTable. + Note that FSE_CTable size depends on 'tableLog' and 'maxSymbolValue' */ +typedef unsigned FSE_CTable; /* don't allocate that. It's only meant to be more restrictive than void* */ +FSE_PUBLIC_API FSE_CTable* FSE_createCTable (unsigned maxSymbolValue, unsigned tableLog); +FSE_PUBLIC_API void FSE_freeCTable (FSE_CTable* ct); + +/*! FSE_buildCTable(): + Builds `ct`, which must be already allocated, using FSE_createCTable(). + @return : 0, or an errorCode, which can be tested using FSE_isError() */ +FSE_PUBLIC_API size_t FSE_buildCTable(FSE_CTable* ct, const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog); + +/*! FSE_compress_usingCTable(): + Compress `src` using `ct` into `dst` which must be already allocated. + @return : size of compressed data (<= `dstCapacity`), + or 0 if compressed data could not fit into `dst`, + or an errorCode, which can be tested using FSE_isError() */ +FSE_PUBLIC_API size_t FSE_compress_usingCTable (void* dst, size_t dstCapacity, const void* src, size_t srcSize, const FSE_CTable* ct); + +/*! +Tutorial : +---------- +The first step is to count all symbols. FSE_count() does this job very fast. +Result will be saved into 'count', a table of unsigned int, which must be already allocated, and have 'maxSymbolValuePtr[0]+1' cells. +'src' is a table of bytes of size 'srcSize'. All values within 'src' MUST be <= maxSymbolValuePtr[0] +maxSymbolValuePtr[0] will be updated, with its real value (necessarily <= original value) +FSE_count() will return the number of occurrence of the most frequent symbol. +This can be used to know if there is a single symbol within 'src', and to quickly evaluate its compressibility. +If there is an error, the function will return an ErrorCode (which can be tested using FSE_isError()). + +The next step is to normalize the frequencies. +FSE_normalizeCount() will ensure that sum of frequencies is == 2 ^'tableLog'. +It also guarantees a minimum of 1 to any Symbol with frequency >= 1. +You can use 'tableLog'==0 to mean "use default tableLog value". +If you are unsure of which tableLog value to use, you can ask FSE_optimalTableLog(), +which will provide the optimal valid tableLog given sourceSize, maxSymbolValue, and a user-defined maximum (0 means "default"). + +The result of FSE_normalizeCount() will be saved into a table, +called 'normalizedCounter', which is a table of signed short. +'normalizedCounter' must be already allocated, and have at least 'maxSymbolValue+1' cells. +The return value is tableLog if everything proceeded as expected. +It is 0 if there is a single symbol within distribution. +If there is an error (ex: invalid tableLog value), the function will return an ErrorCode (which can be tested using FSE_isError()). + +'normalizedCounter' can be saved in a compact manner to a memory area using FSE_writeNCount(). +'buffer' must be already allocated. +For guaranteed success, buffer size must be at least FSE_headerBound(). +The result of the function is the number of bytes written into 'buffer'. +If there is an error, the function will return an ErrorCode (which can be tested using FSE_isError(); ex : buffer size too small). + +'normalizedCounter' can then be used to create the compression table 'CTable'. +The space required by 'CTable' must be already allocated, using FSE_createCTable(). +You can then use FSE_buildCTable() to fill 'CTable'. +If there is an error, both functions will return an ErrorCode (which can be tested using FSE_isError()). + +'CTable' can then be used to compress 'src', with FSE_compress_usingCTable(). +Similar to FSE_count(), the convention is that 'src' is assumed to be a table of char of size 'srcSize' +The function returns the size of compressed data (without header), necessarily <= `dstCapacity`. +If it returns '0', compressed data could not fit into 'dst'. +If there is an error, the function will return an ErrorCode (which can be tested using FSE_isError()). +*/ + + +/* *** DECOMPRESSION *** */ + +/*! FSE_readNCount(): + Read compactly saved 'normalizedCounter' from 'rBuffer'. + @return : size read from 'rBuffer', + or an errorCode, which can be tested using FSE_isError(). + maxSymbolValuePtr[0] and tableLogPtr[0] will also be updated with their respective values */ +FSE_PUBLIC_API size_t FSE_readNCount (short* normalizedCounter, + unsigned* maxSymbolValuePtr, unsigned* tableLogPtr, + const void* rBuffer, size_t rBuffSize); + +/*! Constructor and Destructor of FSE_DTable. + Note that its size depends on 'tableLog' */ +typedef unsigned FSE_DTable; /* don't allocate that. It's just a way to be more restrictive than void* */ +FSE_PUBLIC_API FSE_DTable* FSE_createDTable(unsigned tableLog); +FSE_PUBLIC_API void FSE_freeDTable(FSE_DTable* dt); + +/*! FSE_buildDTable(): + Builds 'dt', which must be already allocated, using FSE_createDTable(). + return : 0, or an errorCode, which can be tested using FSE_isError() */ +FSE_PUBLIC_API size_t FSE_buildDTable (FSE_DTable* dt, const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog); + +/*! FSE_decompress_usingDTable(): + Decompress compressed source `cSrc` of size `cSrcSize` using `dt` + into `dst` which must be already allocated. + @return : size of regenerated data (necessarily <= `dstCapacity`), + or an errorCode, which can be tested using FSE_isError() */ +FSE_PUBLIC_API size_t FSE_decompress_usingDTable(void* dst, size_t dstCapacity, const void* cSrc, size_t cSrcSize, const FSE_DTable* dt); + +/*! +Tutorial : +---------- +(Note : these functions only decompress FSE-compressed blocks. + If block is uncompressed, use memcpy() instead + If block is a single repeated byte, use memset() instead ) + +The first step is to obtain the normalized frequencies of symbols. +This can be performed by FSE_readNCount() if it was saved using FSE_writeNCount(). +'normalizedCounter' must be already allocated, and have at least 'maxSymbolValuePtr[0]+1' cells of signed short. +In practice, that means it's necessary to know 'maxSymbolValue' beforehand, +or size the table to handle worst case situations (typically 256). +FSE_readNCount() will provide 'tableLog' and 'maxSymbolValue'. +The result of FSE_readNCount() is the number of bytes read from 'rBuffer'. +Note that 'rBufferSize' must be at least 4 bytes, even if useful information is less than that. +If there is an error, the function will return an error code, which can be tested using FSE_isError(). + +The next step is to build the decompression tables 'FSE_DTable' from 'normalizedCounter'. +This is performed by the function FSE_buildDTable(). +The space required by 'FSE_DTable' must be already allocated using FSE_createDTable(). +If there is an error, the function will return an error code, which can be tested using FSE_isError(). + +`FSE_DTable` can then be used to decompress `cSrc`, with FSE_decompress_usingDTable(). +`cSrcSize` must be strictly correct, otherwise decompression will fail. +FSE_decompress_usingDTable() result will tell how many bytes were regenerated (<=`dstCapacity`). +If there is an error, the function will return an error code, which can be tested using FSE_isError(). (ex: dst buffer too small) +*/ + +#endif /* FSE_H */ + +#if defined(FSE_STATIC_LINKING_ONLY) && !defined(FSE_H_FSE_STATIC_LINKING_ONLY) +#define FSE_H_FSE_STATIC_LINKING_ONLY + +/* *** Dependency *** */ +#include "bitstream.h" + + +/* ***************************************** +* Static allocation +*******************************************/ +/* FSE buffer bounds */ +#define FSE_NCOUNTBOUND 512 +#define FSE_BLOCKBOUND(size) (size + (size>>7) + 4 /* fse states */ + sizeof(size_t) /* bitContainer */) +#define FSE_COMPRESSBOUND(size) (FSE_NCOUNTBOUND + FSE_BLOCKBOUND(size)) /* Macro version, useful for static allocation */ + +/* It is possible to statically allocate FSE CTable/DTable as a table of FSE_CTable/FSE_DTable using below macros */ +#define FSE_CTABLE_SIZE_U32(maxTableLog, maxSymbolValue) (1 + (1<<(maxTableLog-1)) + ((maxSymbolValue+1)*2)) +#define FSE_DTABLE_SIZE_U32(maxTableLog) (1 + (1< 12) ? (1 << (maxTableLog - 2)) : 1024) ) +size_t FSE_compress_wksp (void* dst, size_t dstSize, const void* src, size_t srcSize, unsigned maxSymbolValue, unsigned tableLog, void* workSpace, size_t wkspSize); + +size_t FSE_buildCTable_raw (FSE_CTable* ct, unsigned nbBits); +/**< build a fake FSE_CTable, designed for a flat distribution, where each symbol uses nbBits */ + +size_t FSE_buildCTable_rle (FSE_CTable* ct, unsigned char symbolValue); +/**< build a fake FSE_CTable, designed to compress always the same symbolValue */ + +/* FSE_buildCTable_wksp() : + * Same as FSE_buildCTable(), but using an externally allocated scratch buffer (`workSpace`). + * `wkspSize` must be >= `(1<= BIT_DStream_completed + +When it's done, verify decompression is fully completed, by checking both DStream and the relevant states. +Checking if DStream has reached its end is performed by : + BIT_endOfDStream(&DStream); +Check also the states. There might be some symbols left there, if some high probability ones (>50%) are possible. + FSE_endOfDState(&DState); +*/ + + +/* ***************************************** +* FSE unsafe API +*******************************************/ +static unsigned char FSE_decodeSymbolFast(FSE_DState_t* DStatePtr, BIT_DStream_t* bitD); +/* faster, but works only if nbBits is always >= 1 (otherwise, result will be corrupted) */ + + +/* ***************************************** +* Implementation of inlined functions +*******************************************/ +typedef struct { + int deltaFindState; + U32 deltaNbBits; +} FSE_symbolCompressionTransform; /* total 8 bytes */ + +MEM_STATIC void FSE_initCState(FSE_CState_t* statePtr, const FSE_CTable* ct) +{ + const void* ptr = ct; + const U16* u16ptr = (const U16*) ptr; + const U32 tableLog = MEM_read16(ptr); + statePtr->value = (ptrdiff_t)1<stateTable = u16ptr+2; + statePtr->symbolTT = ct + 1 + (tableLog ? (1<<(tableLog-1)) : 1); + statePtr->stateLog = tableLog; +} + + +/*! FSE_initCState2() : +* Same as FSE_initCState(), but the first symbol to include (which will be the last to be read) +* uses the smallest state value possible, saving the cost of this symbol */ +MEM_STATIC void FSE_initCState2(FSE_CState_t* statePtr, const FSE_CTable* ct, U32 symbol) +{ + FSE_initCState(statePtr, ct); + { const FSE_symbolCompressionTransform symbolTT = ((const FSE_symbolCompressionTransform*)(statePtr->symbolTT))[symbol]; + const U16* stateTable = (const U16*)(statePtr->stateTable); + U32 nbBitsOut = (U32)((symbolTT.deltaNbBits + (1<<15)) >> 16); + statePtr->value = (nbBitsOut << 16) - symbolTT.deltaNbBits; + statePtr->value = stateTable[(statePtr->value >> nbBitsOut) + symbolTT.deltaFindState]; + } +} + +MEM_STATIC void FSE_encodeSymbol(BIT_CStream_t* bitC, FSE_CState_t* statePtr, unsigned symbol) +{ + FSE_symbolCompressionTransform const symbolTT = ((const FSE_symbolCompressionTransform*)(statePtr->symbolTT))[symbol]; + const U16* const stateTable = (const U16*)(statePtr->stateTable); + U32 const nbBitsOut = (U32)((statePtr->value + symbolTT.deltaNbBits) >> 16); + BIT_addBits(bitC, statePtr->value, nbBitsOut); + statePtr->value = stateTable[ (statePtr->value >> nbBitsOut) + symbolTT.deltaFindState]; +} + +MEM_STATIC void FSE_flushCState(BIT_CStream_t* bitC, const FSE_CState_t* statePtr) +{ + BIT_addBits(bitC, statePtr->value, statePtr->stateLog); + BIT_flushBits(bitC); +} + + +/* FSE_getMaxNbBits() : + * Approximate maximum cost of a symbol, in bits. + * Fractional get rounded up (i.e : a symbol with a normalized frequency of 3 gives the same result as a frequency of 2) + * note 1 : assume symbolValue is valid (<= maxSymbolValue) + * note 2 : if freq[symbolValue]==0, @return a fake cost of tableLog+1 bits */ +MEM_STATIC U32 FSE_getMaxNbBits(const void* symbolTTPtr, U32 symbolValue) +{ + const FSE_symbolCompressionTransform* symbolTT = (const FSE_symbolCompressionTransform*) symbolTTPtr; + return (symbolTT[symbolValue].deltaNbBits + ((1<<16)-1)) >> 16; +} + +/* FSE_bitCost() : + * Approximate symbol cost, as fractional value, using fixed-point format (accuracyLog fractional bits) + * note 1 : assume symbolValue is valid (<= maxSymbolValue) + * note 2 : if freq[symbolValue]==0, @return a fake cost of tableLog+1 bits */ +MEM_STATIC U32 FSE_bitCost(const void* symbolTTPtr, U32 tableLog, U32 symbolValue, U32 accuracyLog) +{ + const FSE_symbolCompressionTransform* symbolTT = (const FSE_symbolCompressionTransform*) symbolTTPtr; + U32 const minNbBits = symbolTT[symbolValue].deltaNbBits >> 16; + U32 const threshold = (minNbBits+1) << 16; + assert(tableLog < 16); + assert(accuracyLog < 31-tableLog); /* ensure enough room for renormalization double shift */ + { U32 const tableSize = 1 << tableLog; + U32 const deltaFromThreshold = threshold - (symbolTT[symbolValue].deltaNbBits + tableSize); + U32 const normalizedDeltaFromThreshold = (deltaFromThreshold << accuracyLog) >> tableLog; /* linear interpolation (very approximate) */ + U32 const bitMultiplier = 1 << accuracyLog; + assert(symbolTT[symbolValue].deltaNbBits + tableSize <= threshold); + assert(normalizedDeltaFromThreshold <= bitMultiplier); + return (minNbBits+1)*bitMultiplier - normalizedDeltaFromThreshold; + } +} + + +/* ====== Decompression ====== */ + +typedef struct { + U16 tableLog; + U16 fastMode; +} FSE_DTableHeader; /* sizeof U32 */ + +typedef struct +{ + unsigned short newState; + unsigned char symbol; + unsigned char nbBits; +} FSE_decode_t; /* size == U32 */ + +MEM_STATIC void FSE_initDState(FSE_DState_t* DStatePtr, BIT_DStream_t* bitD, const FSE_DTable* dt) +{ + const void* ptr = dt; + const FSE_DTableHeader* const DTableH = (const FSE_DTableHeader*)ptr; + DStatePtr->state = BIT_readBits(bitD, DTableH->tableLog); + BIT_reloadDStream(bitD); + DStatePtr->table = dt + 1; +} + +MEM_STATIC BYTE FSE_peekSymbol(const FSE_DState_t* DStatePtr) +{ + FSE_decode_t const DInfo = ((const FSE_decode_t*)(DStatePtr->table))[DStatePtr->state]; + return DInfo.symbol; +} + +MEM_STATIC void FSE_updateState(FSE_DState_t* DStatePtr, BIT_DStream_t* bitD) +{ + FSE_decode_t const DInfo = ((const FSE_decode_t*)(DStatePtr->table))[DStatePtr->state]; + U32 const nbBits = DInfo.nbBits; + size_t const lowBits = BIT_readBits(bitD, nbBits); + DStatePtr->state = DInfo.newState + lowBits; +} + +MEM_STATIC BYTE FSE_decodeSymbol(FSE_DState_t* DStatePtr, BIT_DStream_t* bitD) +{ + FSE_decode_t const DInfo = ((const FSE_decode_t*)(DStatePtr->table))[DStatePtr->state]; + U32 const nbBits = DInfo.nbBits; + BYTE const symbol = DInfo.symbol; + size_t const lowBits = BIT_readBits(bitD, nbBits); + + DStatePtr->state = DInfo.newState + lowBits; + return symbol; +} + +/*! FSE_decodeSymbolFast() : + unsafe, only works if no symbol has a probability > 50% */ +MEM_STATIC BYTE FSE_decodeSymbolFast(FSE_DState_t* DStatePtr, BIT_DStream_t* bitD) +{ + FSE_decode_t const DInfo = ((const FSE_decode_t*)(DStatePtr->table))[DStatePtr->state]; + U32 const nbBits = DInfo.nbBits; + BYTE const symbol = DInfo.symbol; + size_t const lowBits = BIT_readBitsFast(bitD, nbBits); + + DStatePtr->state = DInfo.newState + lowBits; + return symbol; +} + +MEM_STATIC unsigned FSE_endOfDState(const FSE_DState_t* DStatePtr) +{ + return DStatePtr->state == 0; +} + + + +#ifndef FSE_COMMONDEFS_ONLY + +/* ************************************************************** +* Tuning parameters +****************************************************************/ +/*!MEMORY_USAGE : +* Memory usage formula : N->2^N Bytes (examples : 10 -> 1KB; 12 -> 4KB ; 16 -> 64KB; 20 -> 1MB; etc.) +* Increasing memory usage improves compression ratio +* Reduced memory usage can improve speed, due to cache effect +* Recommended max value is 14, for 16KB, which nicely fits into Intel x86 L1 cache */ +#ifndef FSE_MAX_MEMORY_USAGE +# define FSE_MAX_MEMORY_USAGE 14 +#endif +#ifndef FSE_DEFAULT_MEMORY_USAGE +# define FSE_DEFAULT_MEMORY_USAGE 13 +#endif + +/*!FSE_MAX_SYMBOL_VALUE : +* Maximum symbol value authorized. +* Required for proper stack allocation */ +#ifndef FSE_MAX_SYMBOL_VALUE +# define FSE_MAX_SYMBOL_VALUE 255 +#endif + +/* ************************************************************** +* template functions type & suffix +****************************************************************/ +#define FSE_FUNCTION_TYPE BYTE +#define FSE_FUNCTION_EXTENSION +#define FSE_DECODE_TYPE FSE_decode_t + + +#endif /* !FSE_COMMONDEFS_ONLY */ + + +/* *************************************************************** +* Constants +*****************************************************************/ +#define FSE_MAX_TABLELOG (FSE_MAX_MEMORY_USAGE-2) +#define FSE_MAX_TABLESIZE (1U< FSE_TABLELOG_ABSOLUTE_MAX +# error "FSE_MAX_TABLELOG > FSE_TABLELOG_ABSOLUTE_MAX is not supported" +#endif + +#define FSE_TABLESTEP(tableSize) ((tableSize>>1) + (tableSize>>3) + 3) + + +#endif /* FSE_STATIC_LINKING_ONLY */ + + +#if defined (__cplusplus) +} +#endif diff --git a/module/zstd/lib/common/fse_decompress.c b/module/zstd/lib/common/fse_decompress.c new file mode 100644 index 000000000000..bcc2223ccc65 --- /dev/null +++ b/module/zstd/lib/common/fse_decompress.c @@ -0,0 +1,286 @@ +/* ****************************************************************** + * FSE : Finite State Entropy decoder + * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. + * + * You can contact the author at : + * - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy + * - Public forum : https://groups.google.com/forum/#!forum/lz4c + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. +****************************************************************** */ + + +/* ************************************************************** +* Includes +****************************************************************/ +#include /* malloc, free, qsort */ +#include /* memcpy, memset */ +#include "bitstream.h" +#include "compiler.h" +#define FSE_STATIC_LINKING_ONLY +#include "fse.h" +#include "error_private.h" + + +/* ************************************************************** +* Error Management +****************************************************************/ +#define FSE_isError ERR_isError +#define FSE_STATIC_ASSERT(c) DEBUG_STATIC_ASSERT(c) /* use only *after* variable declarations */ + + +/* ************************************************************** +* Templates +****************************************************************/ +/* + designed to be included + for type-specific functions (template emulation in C) + Objective is to write these functions only once, for improved maintenance +*/ + +/* safety checks */ +#ifndef FSE_FUNCTION_EXTENSION +# error "FSE_FUNCTION_EXTENSION must be defined" +#endif +#ifndef FSE_FUNCTION_TYPE +# error "FSE_FUNCTION_TYPE must be defined" +#endif + +/* Function names */ +#define FSE_CAT(X,Y) X##Y +#define FSE_FUNCTION_NAME(X,Y) FSE_CAT(X,Y) +#define FSE_TYPE_NAME(X,Y) FSE_CAT(X,Y) + + +/* Function templates */ +FSE_DTable* FSE_createDTable (unsigned tableLog) +{ + if (tableLog > FSE_TABLELOG_ABSOLUTE_MAX) tableLog = FSE_TABLELOG_ABSOLUTE_MAX; + return (FSE_DTable*)malloc( FSE_DTABLE_SIZE_U32(tableLog) * sizeof (U32) ); +} + +void FSE_freeDTable (FSE_DTable* dt) +{ + free(dt); +} + +size_t FSE_buildDTable(FSE_DTable* dt, const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog) +{ + void* const tdPtr = dt+1; /* because *dt is unsigned, 32-bits aligned on 32-bits */ + FSE_DECODE_TYPE* const tableDecode = (FSE_DECODE_TYPE*) (tdPtr); + U16 symbolNext[FSE_MAX_SYMBOL_VALUE+1]; + + U32 const maxSV1 = maxSymbolValue + 1; + U32 const tableSize = 1 << tableLog; + U32 highThreshold = tableSize-1; + + /* Sanity Checks */ + if (maxSymbolValue > FSE_MAX_SYMBOL_VALUE) return ERROR(maxSymbolValue_tooLarge); + if (tableLog > FSE_MAX_TABLELOG) return ERROR(tableLog_tooLarge); + + /* Init, lay down lowprob symbols */ + { FSE_DTableHeader DTableH; + DTableH.tableLog = (U16)tableLog; + DTableH.fastMode = 1; + { S16 const largeLimit= (S16)(1 << (tableLog-1)); + U32 s; + for (s=0; s= largeLimit) DTableH.fastMode=0; + symbolNext[s] = normalizedCounter[s]; + } } } + memcpy(dt, &DTableH, sizeof(DTableH)); + } + + /* Spread symbols */ + { U32 const tableMask = tableSize-1; + U32 const step = FSE_TABLESTEP(tableSize); + U32 s, position = 0; + for (s=0; s highThreshold) position = (position + step) & tableMask; /* lowprob area */ + } } + if (position!=0) return ERROR(GENERIC); /* position must reach all cells once, otherwise normalizedCounter is incorrect */ + } + + /* Build Decoding table */ + { U32 u; + for (u=0; utableLog = 0; + DTableH->fastMode = 0; + + cell->newState = 0; + cell->symbol = symbolValue; + cell->nbBits = 0; + + return 0; +} + + +size_t FSE_buildDTable_raw (FSE_DTable* dt, unsigned nbBits) +{ + void* ptr = dt; + FSE_DTableHeader* const DTableH = (FSE_DTableHeader*)ptr; + void* dPtr = dt + 1; + FSE_decode_t* const dinfo = (FSE_decode_t*)dPtr; + const unsigned tableSize = 1 << nbBits; + const unsigned tableMask = tableSize - 1; + const unsigned maxSV1 = tableMask+1; + unsigned s; + + /* Sanity checks */ + if (nbBits < 1) return ERROR(GENERIC); /* min size */ + + /* Build Decoding Table */ + DTableH->tableLog = (U16)nbBits; + DTableH->fastMode = 1; + for (s=0; s sizeof(bitD.bitContainer)*8) /* This test must be static */ + BIT_reloadDStream(&bitD); + + op[1] = FSE_GETSYMBOL(&state2); + + if (FSE_MAX_TABLELOG*4+7 > sizeof(bitD.bitContainer)*8) /* This test must be static */ + { if (BIT_reloadDStream(&bitD) > BIT_DStream_unfinished) { op+=2; break; } } + + op[2] = FSE_GETSYMBOL(&state1); + + if (FSE_MAX_TABLELOG*2+7 > sizeof(bitD.bitContainer)*8) /* This test must be static */ + BIT_reloadDStream(&bitD); + + op[3] = FSE_GETSYMBOL(&state2); + } + + /* tail */ + /* note : BIT_reloadDStream(&bitD) >= FSE_DStream_partiallyFilled; Ends at exactly BIT_DStream_completed */ + while (1) { + if (op>(omax-2)) return ERROR(dstSize_tooSmall); + *op++ = FSE_GETSYMBOL(&state1); + if (BIT_reloadDStream(&bitD)==BIT_DStream_overflow) { + *op++ = FSE_GETSYMBOL(&state2); + break; + } + + if (op>(omax-2)) return ERROR(dstSize_tooSmall); + *op++ = FSE_GETSYMBOL(&state2); + if (BIT_reloadDStream(&bitD)==BIT_DStream_overflow) { + *op++ = FSE_GETSYMBOL(&state1); + break; + } } + + return op-ostart; +} + + +size_t FSE_decompress_usingDTable(void* dst, size_t originalSize, + const void* cSrc, size_t cSrcSize, + const FSE_DTable* dt) +{ + const void* ptr = dt; + const FSE_DTableHeader* DTableH = (const FSE_DTableHeader*)ptr; + const U32 fastMode = DTableH->fastMode; + + /* select fast mode (static) */ + if (fastMode) return FSE_decompress_usingDTable_generic(dst, originalSize, cSrc, cSrcSize, dt, 1); + return FSE_decompress_usingDTable_generic(dst, originalSize, cSrc, cSrcSize, dt, 0); +} + + +size_t FSE_decompress_wksp(void* dst, size_t dstCapacity, const void* cSrc, size_t cSrcSize, FSE_DTable* workSpace, unsigned maxLog) +{ + const BYTE* const istart = (const BYTE*)cSrc; + const BYTE* ip = istart; + short counting[FSE_MAX_SYMBOL_VALUE+1]; + unsigned tableLog; + unsigned maxSymbolValue = FSE_MAX_SYMBOL_VALUE; + + /* normal FSE decoding mode */ + size_t const NCountLength = FSE_readNCount (counting, &maxSymbolValue, &tableLog, istart, cSrcSize); + if (FSE_isError(NCountLength)) return NCountLength; + /* if (NCountLength >= cSrcSize) return ERROR(srcSize_wrong); */ /* too small input size; supposed to be already checked in NCountLength, only remaining case : NCountLength==cSrcSize */ + if (tableLog > maxLog) return ERROR(tableLog_tooLarge); + ip += NCountLength; + cSrcSize -= NCountLength; + + CHECK_F( FSE_buildDTable (workSpace, counting, maxSymbolValue, tableLog) ); + + return FSE_decompress_usingDTable (dst, dstCapacity, ip, cSrcSize, workSpace); /* always return, even if it is an error code */ +} + + +typedef FSE_DTable DTable_max_t[FSE_DTABLE_SIZE_U32(FSE_MAX_TABLELOG)]; + +size_t FSE_decompress(void* dst, size_t dstCapacity, const void* cSrc, size_t cSrcSize) +{ + DTable_max_t dt; /* Static analyzer seems unable to understand this table will be properly initialized later */ + return FSE_decompress_wksp(dst, dstCapacity, cSrc, cSrcSize, dt, FSE_MAX_TABLELOG); +} + + + +#endif /* FSE_COMMONDEFS_ONLY */ diff --git a/module/zstd/lib/common/huf.h b/module/zstd/lib/common/huf.h new file mode 100644 index 000000000000..ef432685dac8 --- /dev/null +++ b/module/zstd/lib/common/huf.h @@ -0,0 +1,340 @@ +/* ****************************************************************** + * huff0 huffman codec, + * part of Finite State Entropy library + * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. + * + * You can contact the author at : + * - Source repository : https://github.com/Cyan4973/FiniteStateEntropy + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. +****************************************************************** */ + +#if defined (__cplusplus) +extern "C" { +#endif + +#ifndef HUF_H_298734234 +#define HUF_H_298734234 + +/* *** Dependencies *** */ +#include /* size_t */ + + +/* *** library symbols visibility *** */ +/* Note : when linking with -fvisibility=hidden on gcc, or by default on Visual, + * HUF symbols remain "private" (internal symbols for library only). + * Set macro FSE_DLL_EXPORT to 1 if you want HUF symbols visible on DLL interface */ +#if defined(FSE_DLL_EXPORT) && (FSE_DLL_EXPORT==1) && defined(__GNUC__) && (__GNUC__ >= 4) +# define HUF_PUBLIC_API __attribute__ ((visibility ("default"))) +#elif defined(FSE_DLL_EXPORT) && (FSE_DLL_EXPORT==1) /* Visual expected */ +# define HUF_PUBLIC_API __declspec(dllexport) +#elif defined(FSE_DLL_IMPORT) && (FSE_DLL_IMPORT==1) +# define HUF_PUBLIC_API __declspec(dllimport) /* not required, just to generate faster code (saves a function pointer load from IAT and an indirect jump) */ +#else +# define HUF_PUBLIC_API +#endif + + +/* ========================== */ +/* *** simple functions *** */ +/* ========================== */ + +/** HUF_compress() : + * Compress content from buffer 'src', of size 'srcSize', into buffer 'dst'. + * 'dst' buffer must be already allocated. + * Compression runs faster if `dstCapacity` >= HUF_compressBound(srcSize). + * `srcSize` must be <= `HUF_BLOCKSIZE_MAX` == 128 KB. + * @return : size of compressed data (<= `dstCapacity`). + * Special values : if return == 0, srcData is not compressible => Nothing is stored within dst !!! + * if HUF_isError(return), compression failed (more details using HUF_getErrorName()) + */ +HUF_PUBLIC_API size_t HUF_compress(void* dst, size_t dstCapacity, + const void* src, size_t srcSize); + +/** HUF_decompress() : + * Decompress HUF data from buffer 'cSrc', of size 'cSrcSize', + * into already allocated buffer 'dst', of minimum size 'dstSize'. + * `originalSize` : **must** be the ***exact*** size of original (uncompressed) data. + * Note : in contrast with FSE, HUF_decompress can regenerate + * RLE (cSrcSize==1) and uncompressed (cSrcSize==dstSize) data, + * because it knows size to regenerate (originalSize). + * @return : size of regenerated data (== originalSize), + * or an error code, which can be tested using HUF_isError() + */ +HUF_PUBLIC_API size_t HUF_decompress(void* dst, size_t originalSize, + const void* cSrc, size_t cSrcSize); + + +/* *** Tool functions *** */ +#define HUF_BLOCKSIZE_MAX (128 * 1024) /**< maximum input size for a single block compressed with HUF_compress */ +HUF_PUBLIC_API size_t HUF_compressBound(size_t size); /**< maximum compressed size (worst case) */ + +/* Error Management */ +HUF_PUBLIC_API unsigned HUF_isError(size_t code); /**< tells if a return value is an error code */ +HUF_PUBLIC_API const char* HUF_getErrorName(size_t code); /**< provides error code string (useful for debugging) */ + + +/* *** Advanced function *** */ + +/** HUF_compress2() : + * Same as HUF_compress(), but offers control over `maxSymbolValue` and `tableLog`. + * `maxSymbolValue` must be <= HUF_SYMBOLVALUE_MAX . + * `tableLog` must be `<= HUF_TABLELOG_MAX` . */ +HUF_PUBLIC_API size_t HUF_compress2 (void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + unsigned maxSymbolValue, unsigned tableLog); + +/** HUF_compress4X_wksp() : + * Same as HUF_compress2(), but uses externally allocated `workSpace`. + * `workspace` must have minimum alignment of 4, and be at least as large as HUF_WORKSPACE_SIZE */ +#define HUF_WORKSPACE_SIZE ((6 << 10) + 256) +#define HUF_WORKSPACE_SIZE_U32 (HUF_WORKSPACE_SIZE / sizeof(U32)) +HUF_PUBLIC_API size_t HUF_compress4X_wksp (void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + unsigned maxSymbolValue, unsigned tableLog, + void* workSpace, size_t wkspSize); + +#endif /* HUF_H_298734234 */ + +/* ****************************************************************** + * WARNING !! + * The following section contains advanced and experimental definitions + * which shall never be used in the context of a dynamic library, + * because they are not guaranteed to remain stable in the future. + * Only consider them in association with static linking. + * *****************************************************************/ +#if defined(HUF_STATIC_LINKING_ONLY) && !defined(HUF_H_HUF_STATIC_LINKING_ONLY) +#define HUF_H_HUF_STATIC_LINKING_ONLY + +/* *** Dependencies *** */ +#include "mem.h" /* U32 */ + + +/* *** Constants *** */ +#define HUF_TABLELOG_MAX 12 /* max runtime value of tableLog (due to static allocation); can be modified up to HUF_ABSOLUTEMAX_TABLELOG */ +#define HUF_TABLELOG_DEFAULT 11 /* default tableLog value when none specified */ +#define HUF_SYMBOLVALUE_MAX 255 + +#define HUF_TABLELOG_ABSOLUTEMAX 15 /* absolute limit of HUF_MAX_TABLELOG. Beyond that value, code does not work */ +#if (HUF_TABLELOG_MAX > HUF_TABLELOG_ABSOLUTEMAX) +# error "HUF_TABLELOG_MAX is too large !" +#endif + + +/* **************************************** +* Static allocation +******************************************/ +/* HUF buffer bounds */ +#define HUF_CTABLEBOUND 129 +#define HUF_BLOCKBOUND(size) (size + (size>>8) + 8) /* only true when incompressible is pre-filtered with fast heuristic */ +#define HUF_COMPRESSBOUND(size) (HUF_CTABLEBOUND + HUF_BLOCKBOUND(size)) /* Macro version, useful for static allocation */ + +/* static allocation of HUF's Compression Table */ +#define HUF_CTABLE_SIZE_U32(maxSymbolValue) ((maxSymbolValue)+1) /* Use tables of U32, for proper alignment */ +#define HUF_CTABLE_SIZE(maxSymbolValue) (HUF_CTABLE_SIZE_U32(maxSymbolValue) * sizeof(U32)) +#define HUF_CREATE_STATIC_CTABLE(name, maxSymbolValue) \ + U32 name##hb[HUF_CTABLE_SIZE_U32(maxSymbolValue)]; \ + void* name##hv = &(name##hb); \ + HUF_CElt* name = (HUF_CElt*)(name##hv) /* no final ; */ + +/* static allocation of HUF's DTable */ +typedef U32 HUF_DTable; +#define HUF_DTABLE_SIZE(maxTableLog) (1 + (1<<(maxTableLog))) +#define HUF_CREATE_STATIC_DTABLEX1(DTable, maxTableLog) \ + HUF_DTable DTable[HUF_DTABLE_SIZE((maxTableLog)-1)] = { ((U32)((maxTableLog)-1) * 0x01000001) } +#define HUF_CREATE_STATIC_DTABLEX2(DTable, maxTableLog) \ + HUF_DTable DTable[HUF_DTABLE_SIZE(maxTableLog)] = { ((U32)(maxTableLog) * 0x01000001) } + + +/* **************************************** +* Advanced decompression functions +******************************************/ +size_t HUF_decompress4X1 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< single-symbol decoder */ +#ifndef HUF_FORCE_DECOMPRESS_X1 +size_t HUF_decompress4X2 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< double-symbols decoder */ +#endif + +size_t HUF_decompress4X_DCtx (HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< decodes RLE and uncompressed */ +size_t HUF_decompress4X_hufOnly(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< considers RLE and uncompressed as errors */ +size_t HUF_decompress4X_hufOnly_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize); /**< considers RLE and uncompressed as errors */ +size_t HUF_decompress4X1_DCtx(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< single-symbol decoder */ +size_t HUF_decompress4X1_DCtx_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize); /**< single-symbol decoder */ +#ifndef HUF_FORCE_DECOMPRESS_X1 +size_t HUF_decompress4X2_DCtx(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< double-symbols decoder */ +size_t HUF_decompress4X2_DCtx_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize); /**< double-symbols decoder */ +#endif + + +/* **************************************** + * HUF detailed API + * ****************************************/ + +/*! HUF_compress() does the following: + * 1. count symbol occurrence from source[] into table count[] using FSE_count() (exposed within "fse.h") + * 2. (optional) refine tableLog using HUF_optimalTableLog() + * 3. build Huffman table from count using HUF_buildCTable() + * 4. save Huffman table to memory buffer using HUF_writeCTable() + * 5. encode the data stream using HUF_compress4X_usingCTable() + * + * The following API allows targeting specific sub-functions for advanced tasks. + * For example, it's possible to compress several blocks using the same 'CTable', + * or to save and regenerate 'CTable' using external methods. + */ +unsigned HUF_optimalTableLog(unsigned maxTableLog, size_t srcSize, unsigned maxSymbolValue); +typedef struct HUF_CElt_s HUF_CElt; /* incomplete type */ +size_t HUF_buildCTable (HUF_CElt* CTable, const unsigned* count, unsigned maxSymbolValue, unsigned maxNbBits); /* @return : maxNbBits; CTable and count can overlap. In which case, CTable will overwrite count content */ +size_t HUF_writeCTable (void* dst, size_t maxDstSize, const HUF_CElt* CTable, unsigned maxSymbolValue, unsigned huffLog); +size_t HUF_compress4X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable); +size_t HUF_estimateCompressedSize(const HUF_CElt* CTable, const unsigned* count, unsigned maxSymbolValue); +int HUF_validateCTable(const HUF_CElt* CTable, const unsigned* count, unsigned maxSymbolValue); + +typedef enum { + HUF_repeat_none, /**< Cannot use the previous table */ + HUF_repeat_check, /**< Can use the previous table but it must be checked. Note : The previous table must have been constructed by HUF_compress{1, 4}X_repeat */ + HUF_repeat_valid /**< Can use the previous table and it is assumed to be valid */ + } HUF_repeat; +/** HUF_compress4X_repeat() : + * Same as HUF_compress4X_wksp(), but considers using hufTable if *repeat != HUF_repeat_none. + * If it uses hufTable it does not modify hufTable or repeat. + * If it doesn't, it sets *repeat = HUF_repeat_none, and it sets hufTable to the table used. + * If preferRepeat then the old table will always be used if valid. */ +size_t HUF_compress4X_repeat(void* dst, size_t dstSize, + const void* src, size_t srcSize, + unsigned maxSymbolValue, unsigned tableLog, + void* workSpace, size_t wkspSize, /**< `workSpace` must be aligned on 4-bytes boundaries, `wkspSize` must be >= HUF_WORKSPACE_SIZE */ + HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat, int bmi2); + +/** HUF_buildCTable_wksp() : + * Same as HUF_buildCTable(), but using externally allocated scratch buffer. + * `workSpace` must be aligned on 4-bytes boundaries, and its size must be >= HUF_CTABLE_WORKSPACE_SIZE. + */ +#define HUF_CTABLE_WORKSPACE_SIZE_U32 (2*HUF_SYMBOLVALUE_MAX +1 +1) +#define HUF_CTABLE_WORKSPACE_SIZE (HUF_CTABLE_WORKSPACE_SIZE_U32 * sizeof(unsigned)) +size_t HUF_buildCTable_wksp (HUF_CElt* tree, + const unsigned* count, U32 maxSymbolValue, U32 maxNbBits, + void* workSpace, size_t wkspSize); + +/*! HUF_readStats() : + * Read compact Huffman tree, saved by HUF_writeCTable(). + * `huffWeight` is destination buffer. + * @return : size read from `src` , or an error Code . + * Note : Needed by HUF_readCTable() and HUF_readDTableXn() . */ +size_t HUF_readStats(BYTE* huffWeight, size_t hwSize, + U32* rankStats, U32* nbSymbolsPtr, U32* tableLogPtr, + const void* src, size_t srcSize); + +/** HUF_readCTable() : + * Loading a CTable saved with HUF_writeCTable() */ +size_t HUF_readCTable (HUF_CElt* CTable, unsigned* maxSymbolValuePtr, const void* src, size_t srcSize, unsigned *hasZeroWeights); + +/** HUF_getNbBits() : + * Read nbBits from CTable symbolTable, for symbol `symbolValue` presumed <= HUF_SYMBOLVALUE_MAX + * Note 1 : is not inlined, as HUF_CElt definition is private + * Note 2 : const void* used, so that it can provide a statically allocated table as argument (which uses type U32) */ +U32 HUF_getNbBits(const void* symbolTable, U32 symbolValue); + +/* + * HUF_decompress() does the following: + * 1. select the decompression algorithm (X1, X2) based on pre-computed heuristics + * 2. build Huffman table from save, using HUF_readDTableX?() + * 3. decode 1 or 4 segments in parallel using HUF_decompress?X?_usingDTable() + */ + +/** HUF_selectDecoder() : + * Tells which decoder is likely to decode faster, + * based on a set of pre-computed metrics. + * @return : 0==HUF_decompress4X1, 1==HUF_decompress4X2 . + * Assumption : 0 < dstSize <= 128 KB */ +U32 HUF_selectDecoder (size_t dstSize, size_t cSrcSize); + +/** + * The minimum workspace size for the `workSpace` used in + * HUF_readDTableX1_wksp() and HUF_readDTableX2_wksp(). + * + * The space used depends on HUF_TABLELOG_MAX, ranging from ~1500 bytes when + * HUF_TABLE_LOG_MAX=12 to ~1850 bytes when HUF_TABLE_LOG_MAX=15. + * Buffer overflow errors may potentially occur if code modifications result in + * a required workspace size greater than that specified in the following + * macro. + */ +#define HUF_DECOMPRESS_WORKSPACE_SIZE (2 << 10) +#define HUF_DECOMPRESS_WORKSPACE_SIZE_U32 (HUF_DECOMPRESS_WORKSPACE_SIZE / sizeof(U32)) + +#ifndef HUF_FORCE_DECOMPRESS_X2 +size_t HUF_readDTableX1 (HUF_DTable* DTable, const void* src, size_t srcSize); +size_t HUF_readDTableX1_wksp (HUF_DTable* DTable, const void* src, size_t srcSize, void* workSpace, size_t wkspSize); +#endif +#ifndef HUF_FORCE_DECOMPRESS_X1 +size_t HUF_readDTableX2 (HUF_DTable* DTable, const void* src, size_t srcSize); +size_t HUF_readDTableX2_wksp (HUF_DTable* DTable, const void* src, size_t srcSize, void* workSpace, size_t wkspSize); +#endif + +size_t HUF_decompress4X_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable); +#ifndef HUF_FORCE_DECOMPRESS_X2 +size_t HUF_decompress4X1_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable); +#endif +#ifndef HUF_FORCE_DECOMPRESS_X1 +size_t HUF_decompress4X2_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable); +#endif + + +/* ====================== */ +/* single stream variants */ +/* ====================== */ + +size_t HUF_compress1X (void* dst, size_t dstSize, const void* src, size_t srcSize, unsigned maxSymbolValue, unsigned tableLog); +size_t HUF_compress1X_wksp (void* dst, size_t dstSize, const void* src, size_t srcSize, unsigned maxSymbolValue, unsigned tableLog, void* workSpace, size_t wkspSize); /**< `workSpace` must be a table of at least HUF_WORKSPACE_SIZE_U32 unsigned */ +size_t HUF_compress1X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable); +/** HUF_compress1X_repeat() : + * Same as HUF_compress1X_wksp(), but considers using hufTable if *repeat != HUF_repeat_none. + * If it uses hufTable it does not modify hufTable or repeat. + * If it doesn't, it sets *repeat = HUF_repeat_none, and it sets hufTable to the table used. + * If preferRepeat then the old table will always be used if valid. */ +size_t HUF_compress1X_repeat(void* dst, size_t dstSize, + const void* src, size_t srcSize, + unsigned maxSymbolValue, unsigned tableLog, + void* workSpace, size_t wkspSize, /**< `workSpace` must be aligned on 4-bytes boundaries, `wkspSize` must be >= HUF_WORKSPACE_SIZE */ + HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat, int bmi2); + +size_t HUF_decompress1X1 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* single-symbol decoder */ +#ifndef HUF_FORCE_DECOMPRESS_X1 +size_t HUF_decompress1X2 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* double-symbol decoder */ +#endif + +size_t HUF_decompress1X_DCtx (HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); +size_t HUF_decompress1X_DCtx_wksp (HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize); +#ifndef HUF_FORCE_DECOMPRESS_X2 +size_t HUF_decompress1X1_DCtx(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< single-symbol decoder */ +size_t HUF_decompress1X1_DCtx_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize); /**< single-symbol decoder */ +#endif +#ifndef HUF_FORCE_DECOMPRESS_X1 +size_t HUF_decompress1X2_DCtx(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< double-symbols decoder */ +size_t HUF_decompress1X2_DCtx_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize); /**< double-symbols decoder */ +#endif + +size_t HUF_decompress1X_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable); /**< automatic selection of sing or double symbol decoder, based on DTable */ +#ifndef HUF_FORCE_DECOMPRESS_X2 +size_t HUF_decompress1X1_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable); +#endif +#ifndef HUF_FORCE_DECOMPRESS_X1 +size_t HUF_decompress1X2_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable); +#endif + +/* BMI2 variants. + * If the CPU has BMI2 support, pass bmi2=1, otherwise pass bmi2=0. + */ +size_t HUF_decompress1X_usingDTable_bmi2(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable, int bmi2); +#ifndef HUF_FORCE_DECOMPRESS_X2 +size_t HUF_decompress1X1_DCtx_wksp_bmi2(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize, int bmi2); +#endif +size_t HUF_decompress4X_usingDTable_bmi2(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable, int bmi2); +size_t HUF_decompress4X_hufOnly_wksp_bmi2(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize, int bmi2); + +#endif /* HUF_STATIC_LINKING_ONLY */ + +#if defined (__cplusplus) +} +#endif diff --git a/module/zstd/lib/common/mem.h b/module/zstd/lib/common/mem.h new file mode 100644 index 000000000000..7fbb2e98215e --- /dev/null +++ b/module/zstd/lib/common/mem.h @@ -0,0 +1,450 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#ifndef MEM_H_MODULE +#define MEM_H_MODULE + +#if defined (__cplusplus) +extern "C" { +#endif + +/*-**************************************** +* Dependencies +******************************************/ +#include /* size_t, ptrdiff_t */ +#include /* memcpy */ + + +/*-**************************************** +* Compiler specifics +******************************************/ +#if defined(_MSC_VER) /* Visual Studio */ +# include /* _byteswap_ulong */ +# include /* _byteswap_* */ +#endif +#if defined(__GNUC__) +# define MEM_STATIC static __inline __attribute__((unused)) +#elif defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) +# define MEM_STATIC static inline +#elif defined(_MSC_VER) +# define MEM_STATIC static __inline +#else +# define MEM_STATIC static /* this version may generate warnings for unused static functions; disable the relevant warning */ +#endif + +#ifndef __has_builtin +# define __has_builtin(x) 0 /* compat. with non-clang compilers */ +#endif + +/* code only tested on 32 and 64 bits systems */ +#define MEM_STATIC_ASSERT(c) { enum { MEM_static_assert = 1/(int)(!!(c)) }; } +MEM_STATIC void MEM_check(void) { MEM_STATIC_ASSERT((sizeof(size_t)==4) || (sizeof(size_t)==8)); } + +/* detects whether we are being compiled under msan */ +#if defined (__has_feature) +# if __has_feature(memory_sanitizer) +# define MEMORY_SANITIZER 1 +# endif +#endif + +#if defined (MEMORY_SANITIZER) +/* Not all platforms that support msan provide sanitizers/msan_interface.h. + * We therefore declare the functions we need ourselves, rather than trying to + * include the header file... */ + +#include /* intptr_t */ + +/* Make memory region fully initialized (without changing its contents). */ +void __msan_unpoison(const volatile void *a, size_t size); + +/* Make memory region fully uninitialized (without changing its contents). + This is a legacy interface that does not update origin information. Use + __msan_allocated_memory() instead. */ +void __msan_poison(const volatile void *a, size_t size); + +/* Returns the offset of the first (at least partially) poisoned byte in the + memory range, or -1 if the whole range is good. */ +intptr_t __msan_test_shadow(const volatile void *x, size_t size); +#endif + +/* detects whether we are being compiled under asan */ +#if defined (ZFS_ASAN_ENABLED) +# define ADDRESS_SANITIZER 1 +# define ZSTD_ASAN_DONT_POISON_WORKSPACE +#endif + +#if defined (ADDRESS_SANITIZER) +/* Not all platforms that support asan provide sanitizers/asan_interface.h. + * We therefore declare the functions we need ourselves, rather than trying to + * include the header file... */ + +/** + * Marks a memory region ([addr, addr+size)) as unaddressable. + * + * This memory must be previously allocated by your program. Instrumented + * code is forbidden from accessing addresses in this region until it is + * unpoisoned. This function is not guaranteed to poison the entire region - + * it could poison only a subregion of [addr, addr+size) due to ASan + * alignment restrictions. + * + * \note This function is not thread-safe because no two threads can poison or + * unpoison memory in the same memory region simultaneously. + * + * \param addr Start of memory region. + * \param size Size of memory region. */ +void __asan_poison_memory_region(void const volatile *addr, size_t size); + +/** + * Marks a memory region ([addr, addr+size)) as addressable. + * + * This memory must be previously allocated by your program. Accessing + * addresses in this region is allowed until this region is poisoned again. + * This function could unpoison a super-region of [addr, addr+size) due + * to ASan alignment restrictions. + * + * \note This function is not thread-safe because no two threads can + * poison or unpoison memory in the same memory region simultaneously. + * + * \param addr Start of memory region. + * \param size Size of memory region. */ +void __asan_unpoison_memory_region(void const volatile *addr, size_t size); +#endif + + +/*-************************************************************** +* Basic Types +*****************************************************************/ +#if !defined (__VMS) && (defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) ) +# include + typedef uint8_t BYTE; + typedef uint16_t U16; + typedef int16_t S16; + typedef uint32_t U32; + typedef int32_t S32; + typedef uint64_t U64; + typedef int64_t S64; +#else +# include +#if CHAR_BIT != 8 +# error "this implementation requires char to be exactly 8-bit type" +#endif + typedef unsigned char BYTE; +#if USHRT_MAX != 65535 +# error "this implementation requires short to be exactly 16-bit type" +#endif + typedef unsigned short U16; + typedef signed short S16; +#if UINT_MAX != 4294967295 +# error "this implementation requires int to be exactly 32-bit type" +#endif + typedef unsigned int U32; + typedef signed int S32; +/* note : there are no limits defined for long long type in C90. + * limits exist in C99, however, in such case, is preferred */ + typedef unsigned long long U64; + typedef signed long long S64; +#endif + + +/*-************************************************************** +* Memory I/O +*****************************************************************/ +/* MEM_FORCE_MEMORY_ACCESS : + * By default, access to unaligned memory is controlled by `memcpy()`, which is safe and portable. + * Unfortunately, on some target/compiler combinations, the generated assembly is sub-optimal. + * The below switch allow to select different access method for improved performance. + * Method 0 (default) : use `memcpy()`. Safe and portable. + * Method 1 : `__packed` statement. It depends on compiler extension (i.e., not portable). + * This method is safe if your compiler supports it, and *generally* as fast or faster than `memcpy`. + * Method 2 : direct access. This method is portable but violate C standard. + * It can generate buggy code on targets depending on alignment. + * In some circumstances, it's the only known way to get the most performance (i.e. GCC + ARMv6) + * See http://fastcompression.blogspot.fr/2015/08/accessing-unaligned-memory.html for details. + * Prefer these methods in priority order (0 > 1 > 2) + */ +#ifndef MEM_FORCE_MEMORY_ACCESS /* can be defined externally, on command line for example */ +# if defined(__GNUC__) && ( defined(__ARM_ARCH_6__) || defined(__ARM_ARCH_6J__) || defined(__ARM_ARCH_6K__) || defined(__ARM_ARCH_6Z__) || defined(__ARM_ARCH_6ZK__) || defined(__ARM_ARCH_6T2__) ) +# define MEM_FORCE_MEMORY_ACCESS 2 +# elif defined(__INTEL_COMPILER) || defined(__GNUC__) || defined(__ICCARM__) +# define MEM_FORCE_MEMORY_ACCESS 1 +# endif +#endif + +MEM_STATIC unsigned MEM_32bits(void) { return sizeof(size_t)==4; } +MEM_STATIC unsigned MEM_64bits(void) { return sizeof(size_t)==8; } + +MEM_STATIC unsigned MEM_isLittleEndian(void) +{ + const union { U32 u; BYTE c[4]; } one = { 1 }; /* don't use static : performance detrimental */ + return one.c[0]; +} + +#if defined(MEM_FORCE_MEMORY_ACCESS) && (MEM_FORCE_MEMORY_ACCESS==2) + +/* violates C standard, by lying on structure alignment. +Only use if no other choice to achieve best performance on target platform */ +MEM_STATIC U16 MEM_read16(const void* memPtr) { return *(const U16*) memPtr; } +MEM_STATIC U32 MEM_read32(const void* memPtr) { return *(const U32*) memPtr; } +MEM_STATIC U64 MEM_read64(const void* memPtr) { return *(const U64*) memPtr; } +MEM_STATIC size_t MEM_readST(const void* memPtr) { return *(const size_t*) memPtr; } + +MEM_STATIC void MEM_write16(void* memPtr, U16 value) { *(U16*)memPtr = value; } +MEM_STATIC void MEM_write32(void* memPtr, U32 value) { *(U32*)memPtr = value; } +MEM_STATIC void MEM_write64(void* memPtr, U64 value) { *(U64*)memPtr = value; } + +#elif defined(MEM_FORCE_MEMORY_ACCESS) && (MEM_FORCE_MEMORY_ACCESS==1) + +/* __pack instructions are safer, but compiler specific, hence potentially problematic for some compilers */ +/* currently only defined for gcc and icc */ +#if defined(_MSC_VER) || (defined(__INTEL_COMPILER) && defined(WIN32)) + __pragma( pack(push, 1) ) + typedef struct { U16 v; } unalign16; + typedef struct { U32 v; } unalign32; + typedef struct { U64 v; } unalign64; + typedef struct { size_t v; } unalignArch; + __pragma( pack(pop) ) +#else + typedef struct { U16 v; } __attribute__((packed)) unalign16; + typedef struct { U32 v; } __attribute__((packed)) unalign32; + typedef struct { U64 v; } __attribute__((packed)) unalign64; + typedef struct { size_t v; } __attribute__((packed)) unalignArch; +#endif + +MEM_STATIC U16 MEM_read16(const void* ptr) { return ((const unalign16*)ptr)->v; } +MEM_STATIC U32 MEM_read32(const void* ptr) { return ((const unalign32*)ptr)->v; } +MEM_STATIC U64 MEM_read64(const void* ptr) { return ((const unalign64*)ptr)->v; } +MEM_STATIC size_t MEM_readST(const void* ptr) { return ((const unalignArch*)ptr)->v; } + +MEM_STATIC void MEM_write16(void* memPtr, U16 value) { ((unalign16*)memPtr)->v = value; } +MEM_STATIC void MEM_write32(void* memPtr, U32 value) { ((unalign32*)memPtr)->v = value; } +MEM_STATIC void MEM_write64(void* memPtr, U64 value) { ((unalign64*)memPtr)->v = value; } + +#else + +/* default method, safe and standard. + can sometimes prove slower */ + +MEM_STATIC U16 MEM_read16(const void* memPtr) +{ + U16 val; memcpy(&val, memPtr, sizeof(val)); return val; +} + +MEM_STATIC U32 MEM_read32(const void* memPtr) +{ + U32 val; memcpy(&val, memPtr, sizeof(val)); return val; +} + +MEM_STATIC U64 MEM_read64(const void* memPtr) +{ + U64 val; memcpy(&val, memPtr, sizeof(val)); return val; +} + +MEM_STATIC size_t MEM_readST(const void* memPtr) +{ + size_t val; memcpy(&val, memPtr, sizeof(val)); return val; +} + +MEM_STATIC void MEM_write16(void* memPtr, U16 value) +{ + memcpy(memPtr, &value, sizeof(value)); +} + +MEM_STATIC void MEM_write32(void* memPtr, U32 value) +{ + memcpy(memPtr, &value, sizeof(value)); +} + +MEM_STATIC void MEM_write64(void* memPtr, U64 value) +{ + memcpy(memPtr, &value, sizeof(value)); +} + +#endif /* MEM_FORCE_MEMORY_ACCESS */ + +MEM_STATIC U32 MEM_swap32(U32 in) +{ +#if defined(_MSC_VER) /* Visual Studio */ + return _byteswap_ulong(in); +#elif (defined (__GNUC__) && (__GNUC__ * 100 + __GNUC_MINOR__ >= 403)) \ + || (defined(__clang__) && __has_builtin(__builtin_bswap32)) + return __builtin_bswap32(in); +#else + return ((in << 24) & 0xff000000 ) | + ((in << 8) & 0x00ff0000 ) | + ((in >> 8) & 0x0000ff00 ) | + ((in >> 24) & 0x000000ff ); +#endif +} + +MEM_STATIC U64 MEM_swap64(U64 in) +{ +#if defined(_MSC_VER) /* Visual Studio */ + return _byteswap_uint64(in); +#elif (defined (__GNUC__) && (__GNUC__ * 100 + __GNUC_MINOR__ >= 403)) \ + || (defined(__clang__) && __has_builtin(__builtin_bswap64)) + return __builtin_bswap64(in); +#else + return ((in << 56) & 0xff00000000000000ULL) | + ((in << 40) & 0x00ff000000000000ULL) | + ((in << 24) & 0x0000ff0000000000ULL) | + ((in << 8) & 0x000000ff00000000ULL) | + ((in >> 8) & 0x00000000ff000000ULL) | + ((in >> 24) & 0x0000000000ff0000ULL) | + ((in >> 40) & 0x000000000000ff00ULL) | + ((in >> 56) & 0x00000000000000ffULL); +#endif +} + +MEM_STATIC size_t MEM_swapST(size_t in) +{ + if (MEM_32bits()) + return (size_t)MEM_swap32((U32)in); + else + return (size_t)MEM_swap64((U64)in); +} + +/*=== Little endian r/w ===*/ + +MEM_STATIC U16 MEM_readLE16(const void* memPtr) +{ + if (MEM_isLittleEndian()) + return MEM_read16(memPtr); + else { + const BYTE* p = (const BYTE*)memPtr; + return (U16)(p[0] + (p[1]<<8)); + } +} + +MEM_STATIC void MEM_writeLE16(void* memPtr, U16 val) +{ + if (MEM_isLittleEndian()) { + MEM_write16(memPtr, val); + } else { + BYTE* p = (BYTE*)memPtr; + p[0] = (BYTE)val; + p[1] = (BYTE)(val>>8); + } +} + +MEM_STATIC U32 MEM_readLE24(const void* memPtr) +{ + return MEM_readLE16(memPtr) + (((const BYTE*)memPtr)[2] << 16); +} + +MEM_STATIC void MEM_writeLE24(void* memPtr, U32 val) +{ + MEM_writeLE16(memPtr, (U16)val); + ((BYTE*)memPtr)[2] = (BYTE)(val>>16); +} + +MEM_STATIC U32 MEM_readLE32(const void* memPtr) +{ + if (MEM_isLittleEndian()) + return MEM_read32(memPtr); + else + return MEM_swap32(MEM_read32(memPtr)); +} + +MEM_STATIC void MEM_writeLE32(void* memPtr, U32 val32) +{ + if (MEM_isLittleEndian()) + MEM_write32(memPtr, val32); + else + MEM_write32(memPtr, MEM_swap32(val32)); +} + +MEM_STATIC U64 MEM_readLE64(const void* memPtr) +{ + if (MEM_isLittleEndian()) + return MEM_read64(memPtr); + else + return MEM_swap64(MEM_read64(memPtr)); +} + +MEM_STATIC void MEM_writeLE64(void* memPtr, U64 val64) +{ + if (MEM_isLittleEndian()) + MEM_write64(memPtr, val64); + else + MEM_write64(memPtr, MEM_swap64(val64)); +} + +MEM_STATIC size_t MEM_readLEST(const void* memPtr) +{ + if (MEM_32bits()) + return (size_t)MEM_readLE32(memPtr); + else + return (size_t)MEM_readLE64(memPtr); +} + +MEM_STATIC void MEM_writeLEST(void* memPtr, size_t val) +{ + if (MEM_32bits()) + MEM_writeLE32(memPtr, (U32)val); + else + MEM_writeLE64(memPtr, (U64)val); +} + +/*=== Big endian r/w ===*/ + +MEM_STATIC U32 MEM_readBE32(const void* memPtr) +{ + if (MEM_isLittleEndian()) + return MEM_swap32(MEM_read32(memPtr)); + else + return MEM_read32(memPtr); +} + +MEM_STATIC void MEM_writeBE32(void* memPtr, U32 val32) +{ + if (MEM_isLittleEndian()) + MEM_write32(memPtr, MEM_swap32(val32)); + else + MEM_write32(memPtr, val32); +} + +MEM_STATIC U64 MEM_readBE64(const void* memPtr) +{ + if (MEM_isLittleEndian()) + return MEM_swap64(MEM_read64(memPtr)); + else + return MEM_read64(memPtr); +} + +MEM_STATIC void MEM_writeBE64(void* memPtr, U64 val64) +{ + if (MEM_isLittleEndian()) + MEM_write64(memPtr, MEM_swap64(val64)); + else + MEM_write64(memPtr, val64); +} + +MEM_STATIC size_t MEM_readBEST(const void* memPtr) +{ + if (MEM_32bits()) + return (size_t)MEM_readBE32(memPtr); + else + return (size_t)MEM_readBE64(memPtr); +} + +MEM_STATIC void MEM_writeBEST(void* memPtr, size_t val) +{ + if (MEM_32bits()) + MEM_writeBE32(memPtr, (U32)val); + else + MEM_writeBE64(memPtr, (U64)val); +} + + +#if defined (__cplusplus) +} +#endif + +#endif /* MEM_H_MODULE */ diff --git a/module/zstd/lib/common/pool.c b/module/zstd/lib/common/pool.c new file mode 100644 index 000000000000..aa4b4de0d3f6 --- /dev/null +++ b/module/zstd/lib/common/pool.c @@ -0,0 +1,344 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + + +/* ====== Dependencies ======= */ +#include /* size_t */ +#include "debug.h" /* assert */ +#include "zstd_internal.h" /* ZSTD_malloc, ZSTD_free */ +#include "pool.h" + +/* ====== Compiler specifics ====== */ +#if defined(_MSC_VER) +# pragma warning(disable : 4204) /* disable: C4204: non-constant aggregate initializer */ +#endif + + +#ifdef ZSTD_MULTITHREAD + +#include "threading.h" /* pthread adaptation */ + +/* A job is a function and an opaque argument */ +typedef struct POOL_job_s { + POOL_function function; + void *opaque; +} POOL_job; + +struct POOL_ctx_s { + ZSTD_customMem customMem; + /* Keep track of the threads */ + ZSTD_pthread_t* threads; + size_t threadCapacity; + size_t threadLimit; + + /* The queue is a circular buffer */ + POOL_job *queue; + size_t queueHead; + size_t queueTail; + size_t queueSize; + + /* The number of threads working on jobs */ + size_t numThreadsBusy; + /* Indicates if the queue is empty */ + int queueEmpty; + + /* The mutex protects the queue */ + ZSTD_pthread_mutex_t queueMutex; + /* Condition variable for pushers to wait on when the queue is full */ + ZSTD_pthread_cond_t queuePushCond; + /* Condition variables for poppers to wait on when the queue is empty */ + ZSTD_pthread_cond_t queuePopCond; + /* Indicates if the queue is shutting down */ + int shutdown; +}; + +/* POOL_thread() : + * Work thread for the thread pool. + * Waits for jobs and executes them. + * @returns : NULL on failure else non-null. + */ +static void* POOL_thread(void* opaque) { + POOL_ctx* const ctx = (POOL_ctx*)opaque; + if (!ctx) { return NULL; } + for (;;) { + /* Lock the mutex and wait for a non-empty queue or until shutdown */ + ZSTD_pthread_mutex_lock(&ctx->queueMutex); + + while ( ctx->queueEmpty + || (ctx->numThreadsBusy >= ctx->threadLimit) ) { + if (ctx->shutdown) { + /* even if !queueEmpty, (possible if numThreadsBusy >= threadLimit), + * a few threads will be shutdown while !queueEmpty, + * but enough threads will remain active to finish the queue */ + ZSTD_pthread_mutex_unlock(&ctx->queueMutex); + return opaque; + } + ZSTD_pthread_cond_wait(&ctx->queuePopCond, &ctx->queueMutex); + } + /* Pop a job off the queue */ + { POOL_job const job = ctx->queue[ctx->queueHead]; + ctx->queueHead = (ctx->queueHead + 1) % ctx->queueSize; + ctx->numThreadsBusy++; + ctx->queueEmpty = ctx->queueHead == ctx->queueTail; + /* Unlock the mutex, signal a pusher, and run the job */ + ZSTD_pthread_cond_signal(&ctx->queuePushCond); + ZSTD_pthread_mutex_unlock(&ctx->queueMutex); + + job.function(job.opaque); + + /* If the intended queue size was 0, signal after finishing job */ + ZSTD_pthread_mutex_lock(&ctx->queueMutex); + ctx->numThreadsBusy--; + if (ctx->queueSize == 1) { + ZSTD_pthread_cond_signal(&ctx->queuePushCond); + } + ZSTD_pthread_mutex_unlock(&ctx->queueMutex); + } + } /* for (;;) */ + assert(0); /* Unreachable */ +} + +POOL_ctx* POOL_create(size_t numThreads, size_t queueSize) { + return POOL_create_advanced(numThreads, queueSize, ZSTD_defaultCMem); +} + +POOL_ctx* POOL_create_advanced(size_t numThreads, size_t queueSize, + ZSTD_customMem customMem) { + POOL_ctx* ctx; + /* Check parameters */ + if (!numThreads) { return NULL; } + /* Allocate the context and zero initialize */ + ctx = (POOL_ctx*)ZSTD_calloc(sizeof(POOL_ctx), customMem); + if (!ctx) { return NULL; } + /* Initialize the job queue. + * It needs one extra space since one space is wasted to differentiate + * empty and full queues. + */ + ctx->queueSize = queueSize + 1; + ctx->queue = (POOL_job*)ZSTD_malloc(ctx->queueSize * sizeof(POOL_job), customMem); + ctx->queueHead = 0; + ctx->queueTail = 0; + ctx->numThreadsBusy = 0; + ctx->queueEmpty = 1; + { + int error = 0; + error |= ZSTD_pthread_mutex_init(&ctx->queueMutex, NULL); + error |= ZSTD_pthread_cond_init(&ctx->queuePushCond, NULL); + error |= ZSTD_pthread_cond_init(&ctx->queuePopCond, NULL); + if (error) { POOL_free(ctx); return NULL; } + } + ctx->shutdown = 0; + /* Allocate space for the thread handles */ + ctx->threads = (ZSTD_pthread_t*)ZSTD_malloc(numThreads * sizeof(ZSTD_pthread_t), customMem); + ctx->threadCapacity = 0; + ctx->customMem = customMem; + /* Check for errors */ + if (!ctx->threads || !ctx->queue) { POOL_free(ctx); return NULL; } + /* Initialize the threads */ + { size_t i; + for (i = 0; i < numThreads; ++i) { + if (ZSTD_pthread_create(&ctx->threads[i], NULL, &POOL_thread, ctx)) { + ctx->threadCapacity = i; + POOL_free(ctx); + return NULL; + } } + ctx->threadCapacity = numThreads; + ctx->threadLimit = numThreads; + } + return ctx; +} + +/*! POOL_join() : + Shutdown the queue, wake any sleeping threads, and join all of the threads. +*/ +static void POOL_join(POOL_ctx* ctx) { + /* Shut down the queue */ + ZSTD_pthread_mutex_lock(&ctx->queueMutex); + ctx->shutdown = 1; + ZSTD_pthread_mutex_unlock(&ctx->queueMutex); + /* Wake up sleeping threads */ + ZSTD_pthread_cond_broadcast(&ctx->queuePushCond); + ZSTD_pthread_cond_broadcast(&ctx->queuePopCond); + /* Join all of the threads */ + { size_t i; + for (i = 0; i < ctx->threadCapacity; ++i) { + ZSTD_pthread_join(ctx->threads[i], NULL); /* note : could fail */ + } } +} + +void POOL_free(POOL_ctx *ctx) { + if (!ctx) { return; } + POOL_join(ctx); + ZSTD_pthread_mutex_destroy(&ctx->queueMutex); + ZSTD_pthread_cond_destroy(&ctx->queuePushCond); + ZSTD_pthread_cond_destroy(&ctx->queuePopCond); + ZSTD_free(ctx->queue, ctx->customMem); + ZSTD_free(ctx->threads, ctx->customMem); + ZSTD_free(ctx, ctx->customMem); +} + + + +size_t POOL_sizeof(POOL_ctx *ctx) { + if (ctx==NULL) return 0; /* supports sizeof NULL */ + return sizeof(*ctx) + + ctx->queueSize * sizeof(POOL_job) + + ctx->threadCapacity * sizeof(ZSTD_pthread_t); +} + + +/* @return : 0 on success, 1 on error */ +static int POOL_resize_internal(POOL_ctx* ctx, size_t numThreads) +{ + if (numThreads <= ctx->threadCapacity) { + if (!numThreads) return 1; + ctx->threadLimit = numThreads; + return 0; + } + /* numThreads > threadCapacity */ + { ZSTD_pthread_t* const threadPool = (ZSTD_pthread_t*)ZSTD_malloc(numThreads * sizeof(ZSTD_pthread_t), ctx->customMem); + if (!threadPool) return 1; + /* replace existing thread pool */ + memcpy(threadPool, ctx->threads, ctx->threadCapacity * sizeof(*threadPool)); + ZSTD_free(ctx->threads, ctx->customMem); + ctx->threads = threadPool; + /* Initialize additional threads */ + { size_t threadId; + for (threadId = ctx->threadCapacity; threadId < numThreads; ++threadId) { + if (ZSTD_pthread_create(&threadPool[threadId], NULL, &POOL_thread, ctx)) { + ctx->threadCapacity = threadId; + return 1; + } } + } } + /* successfully expanded */ + ctx->threadCapacity = numThreads; + ctx->threadLimit = numThreads; + return 0; +} + +/* @return : 0 on success, 1 on error */ +int POOL_resize(POOL_ctx* ctx, size_t numThreads) +{ + int result; + if (ctx==NULL) return 1; + ZSTD_pthread_mutex_lock(&ctx->queueMutex); + result = POOL_resize_internal(ctx, numThreads); + ZSTD_pthread_cond_broadcast(&ctx->queuePopCond); + ZSTD_pthread_mutex_unlock(&ctx->queueMutex); + return result; +} + +/** + * Returns 1 if the queue is full and 0 otherwise. + * + * When queueSize is 1 (pool was created with an intended queueSize of 0), + * then a queue is empty if there is a thread free _and_ no job is waiting. + */ +static int isQueueFull(POOL_ctx const* ctx) { + if (ctx->queueSize > 1) { + return ctx->queueHead == ((ctx->queueTail + 1) % ctx->queueSize); + } else { + return (ctx->numThreadsBusy == ctx->threadLimit) || + !ctx->queueEmpty; + } +} + + +static void POOL_add_internal(POOL_ctx* ctx, POOL_function function, void *opaque) +{ + POOL_job const job = {function, opaque}; + assert(ctx != NULL); + if (ctx->shutdown) return; + + ctx->queueEmpty = 0; + ctx->queue[ctx->queueTail] = job; + ctx->queueTail = (ctx->queueTail + 1) % ctx->queueSize; + ZSTD_pthread_cond_signal(&ctx->queuePopCond); +} + +void POOL_add(POOL_ctx* ctx, POOL_function function, void* opaque) +{ + assert(ctx != NULL); + ZSTD_pthread_mutex_lock(&ctx->queueMutex); + /* Wait until there is space in the queue for the new job */ + while (isQueueFull(ctx) && (!ctx->shutdown)) { + ZSTD_pthread_cond_wait(&ctx->queuePushCond, &ctx->queueMutex); + } + POOL_add_internal(ctx, function, opaque); + ZSTD_pthread_mutex_unlock(&ctx->queueMutex); +} + + +int POOL_tryAdd(POOL_ctx* ctx, POOL_function function, void* opaque) +{ + assert(ctx != NULL); + ZSTD_pthread_mutex_lock(&ctx->queueMutex); + if (isQueueFull(ctx)) { + ZSTD_pthread_mutex_unlock(&ctx->queueMutex); + return 0; + } + POOL_add_internal(ctx, function, opaque); + ZSTD_pthread_mutex_unlock(&ctx->queueMutex); + return 1; +} + + +#else /* ZSTD_MULTITHREAD not defined */ + +/* ========================== */ +/* No multi-threading support */ +/* ========================== */ + + +/* We don't need any data, but if it is empty, malloc() might return NULL. */ +struct POOL_ctx_s { + int dummy; +}; +static POOL_ctx g_ctx; + +POOL_ctx* POOL_create(size_t numThreads, size_t queueSize) { + return POOL_create_advanced(numThreads, queueSize, ZSTD_defaultCMem); +} + +POOL_ctx* POOL_create_advanced(size_t numThreads, size_t queueSize, ZSTD_customMem customMem) { + (void)numThreads; + (void)queueSize; + (void)customMem; + return &g_ctx; +} + +void POOL_free(POOL_ctx* ctx) { + assert(!ctx || ctx == &g_ctx); + (void)ctx; +} + +int POOL_resize(POOL_ctx* ctx, size_t numThreads) { + (void)ctx; (void)numThreads; + return 0; +} + +void POOL_add(POOL_ctx* ctx, POOL_function function, void* opaque) { + (void)ctx; + function(opaque); +} + +int POOL_tryAdd(POOL_ctx* ctx, POOL_function function, void* opaque) { + (void)ctx; + function(opaque); + return 1; +} + +size_t POOL_sizeof(POOL_ctx* ctx) { + if (ctx==NULL) return 0; /* supports sizeof NULL */ + assert(ctx == &g_ctx); + return sizeof(*ctx); +} + +#endif /* ZSTD_MULTITHREAD */ diff --git a/module/zstd/lib/common/pool.h b/module/zstd/lib/common/pool.h new file mode 100644 index 000000000000..259bafc97570 --- /dev/null +++ b/module/zstd/lib/common/pool.h @@ -0,0 +1,84 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#ifndef POOL_H +#define POOL_H + +#if defined (__cplusplus) +extern "C" { +#endif + + +#include /* size_t */ +#define ZSTD_STATIC_LINKING_ONLY /* ZSTD_customMem */ +#include "../zstd.h" + +typedef struct POOL_ctx_s POOL_ctx; + +/*! POOL_create() : + * Create a thread pool with at most `numThreads` threads. + * `numThreads` must be at least 1. + * The maximum number of queued jobs before blocking is `queueSize`. + * @return : POOL_ctx pointer on success, else NULL. +*/ +POOL_ctx* POOL_create(size_t numThreads, size_t queueSize); + +POOL_ctx* POOL_create_advanced(size_t numThreads, size_t queueSize, + ZSTD_customMem customMem); + +/*! POOL_free() : + * Free a thread pool returned by POOL_create(). + */ +void POOL_free(POOL_ctx* ctx); + +/*! POOL_resize() : + * Expands or shrinks pool's number of threads. + * This is more efficient than releasing + creating a new context, + * since it tries to preserve and re-use existing threads. + * `numThreads` must be at least 1. + * @return : 0 when resize was successful, + * !0 (typically 1) if there is an error. + * note : only numThreads can be resized, queueSize remains unchanged. + */ +int POOL_resize(POOL_ctx* ctx, size_t numThreads); + +/*! POOL_sizeof() : + * @return threadpool memory usage + * note : compatible with NULL (returns 0 in this case) + */ +size_t POOL_sizeof(POOL_ctx* ctx); + +/*! POOL_function : + * The function type that can be added to a thread pool. + */ +typedef void (*POOL_function)(void*); + +/*! POOL_add() : + * Add the job `function(opaque)` to the thread pool. `ctx` must be valid. + * Possibly blocks until there is room in the queue. + * Note : The function may be executed asynchronously, + * therefore, `opaque` must live until function has been completed. + */ +void POOL_add(POOL_ctx* ctx, POOL_function function, void* opaque); + + +/*! POOL_tryAdd() : + * Add the job `function(opaque)` to thread pool _if_ a worker is available. + * Returns immediately even if not (does not block). + * @return : 1 if successful, 0 if not. + */ +int POOL_tryAdd(POOL_ctx* ctx, POOL_function function, void* opaque); + + +#if defined (__cplusplus) +} +#endif + +#endif diff --git a/module/zstd/lib/common/xxhash.c b/module/zstd/lib/common/xxhash.c new file mode 100644 index 000000000000..597de18fc895 --- /dev/null +++ b/module/zstd/lib/common/xxhash.c @@ -0,0 +1,864 @@ +/* + * xxHash - Fast Hash algorithm + * Copyright (c) 2012-2020, Yann Collet, Facebook, Inc. + * + * You can contact the author at : + * - xxHash homepage: http://www.xxhash.com + * - xxHash source repository : https://github.com/Cyan4973/xxHash + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. +*/ + + +/* ************************************* +* Tuning parameters +***************************************/ +/*!XXH_FORCE_MEMORY_ACCESS : + * By default, access to unaligned memory is controlled by `memcpy()`, which is safe and portable. + * Unfortunately, on some target/compiler combinations, the generated assembly is sub-optimal. + * The below switch allow to select different access method for improved performance. + * Method 0 (default) : use `memcpy()`. Safe and portable. + * Method 1 : `__packed` statement. It depends on compiler extension (ie, not portable). + * This method is safe if your compiler supports it, and *generally* as fast or faster than `memcpy`. + * Method 2 : direct access. This method doesn't depend on compiler but violate C standard. + * It can generate buggy code on targets which do not support unaligned memory accesses. + * But in some circumstances, it's the only known way to get the most performance (ie GCC + ARMv6) + * See http://stackoverflow.com/a/32095106/646947 for details. + * Prefer these methods in priority order (0 > 1 > 2) + */ +#ifndef XXH_FORCE_MEMORY_ACCESS /* can be defined externally, on command line for example */ +# if defined(__GNUC__) && ( defined(__ARM_ARCH_6__) || defined(__ARM_ARCH_6J__) || defined(__ARM_ARCH_6K__) || defined(__ARM_ARCH_6Z__) || defined(__ARM_ARCH_6ZK__) || defined(__ARM_ARCH_6T2__) ) +# define XXH_FORCE_MEMORY_ACCESS 2 +# elif (defined(__INTEL_COMPILER) && !defined(WIN32)) || \ + (defined(__GNUC__) && ( defined(__ARM_ARCH_7__) || defined(__ARM_ARCH_7A__) || defined(__ARM_ARCH_7R__) || defined(__ARM_ARCH_7M__) || defined(__ARM_ARCH_7S__) )) || \ + defined(__ICCARM__) +# define XXH_FORCE_MEMORY_ACCESS 1 +# endif +#endif + +/*!XXH_ACCEPT_NULL_INPUT_POINTER : + * If the input pointer is a null pointer, xxHash default behavior is to trigger a memory access error, since it is a bad pointer. + * When this option is enabled, xxHash output for null input pointers will be the same as a null-length input. + * By default, this option is disabled. To enable it, uncomment below define : + */ +/* #define XXH_ACCEPT_NULL_INPUT_POINTER 1 */ + +/*!XXH_FORCE_NATIVE_FORMAT : + * By default, xxHash library provides endian-independent Hash values, based on little-endian convention. + * Results are therefore identical for little-endian and big-endian CPU. + * This comes at a performance cost for big-endian CPU, since some swapping is required to emulate little-endian format. + * Should endian-independence be of no importance for your application, you may set the #define below to 1, + * to improve speed for Big-endian CPU. + * This option has no impact on Little_Endian CPU. + */ +#ifndef XXH_FORCE_NATIVE_FORMAT /* can be defined externally */ +# define XXH_FORCE_NATIVE_FORMAT 0 +#endif + +/*!XXH_FORCE_ALIGN_CHECK : + * This is a minor performance trick, only useful with lots of very small keys. + * It means : check for aligned/unaligned input. + * The check costs one initial branch per hash; set to 0 when the input data + * is guaranteed to be aligned. + */ +#ifndef XXH_FORCE_ALIGN_CHECK /* can be defined externally */ +# if defined(__i386) || defined(_M_IX86) || defined(__x86_64__) || defined(_M_X64) +# define XXH_FORCE_ALIGN_CHECK 0 +# else +# define XXH_FORCE_ALIGN_CHECK 1 +# endif +#endif + + +/* ************************************* +* Includes & Memory related functions +***************************************/ +/* Modify the local functions below should you wish to use some other memory routines */ +/* for malloc(), free() */ +#include +#include /* size_t */ +static void* XXH_malloc(size_t s) { return malloc(s); } +static void XXH_free (void* p) { free(p); } +/* for memcpy() */ +#include +static void* XXH_memcpy(void* dest, const void* src, size_t size) { return memcpy(dest,src,size); } + +#ifndef XXH_STATIC_LINKING_ONLY +# define XXH_STATIC_LINKING_ONLY +#endif +#include "xxhash.h" + + +/* ************************************* +* Compiler Specific Options +***************************************/ +#if (defined(__GNUC__) && !defined(__STRICT_ANSI__)) || defined(__cplusplus) || defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L /* C99 */ +# define INLINE_KEYWORD inline +#else +# define INLINE_KEYWORD +#endif + +#if defined(__GNUC__) || defined(__ICCARM__) +# define FORCE_INLINE_ATTR __attribute__((always_inline)) +#elif defined(_MSC_VER) +# define FORCE_INLINE_ATTR __forceinline +#else +# define FORCE_INLINE_ATTR +#endif + +#define FORCE_INLINE_TEMPLATE static INLINE_KEYWORD FORCE_INLINE_ATTR + + +#ifdef _MSC_VER +# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */ +#endif + + +/* ************************************* +* Basic Types +***************************************/ +#ifndef MEM_MODULE +# define MEM_MODULE +# if !defined (__VMS) && (defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) ) +# include + typedef uint8_t BYTE; + typedef uint16_t U16; + typedef uint32_t U32; + typedef int32_t S32; + typedef uint64_t U64; +# else + typedef unsigned char BYTE; + typedef unsigned short U16; + typedef unsigned int U32; + typedef signed int S32; + typedef unsigned long long U64; /* if your compiler doesn't support unsigned long long, replace by another 64-bit type here. Note that xxhash.h will also need to be updated. */ +# endif +#endif + + +#if (defined(XXH_FORCE_MEMORY_ACCESS) && (XXH_FORCE_MEMORY_ACCESS==2)) + +/* Force direct memory access. Only works on CPU which support unaligned memory access in hardware */ +static U32 XXH_read32(const void* memPtr) { return *(const U32*) memPtr; } +static U64 XXH_read64(const void* memPtr) { return *(const U64*) memPtr; } + +#elif (defined(XXH_FORCE_MEMORY_ACCESS) && (XXH_FORCE_MEMORY_ACCESS==1)) + +/* __pack instructions are safer, but compiler specific, hence potentially problematic for some compilers */ +/* currently only defined for gcc and icc */ +typedef union { U32 u32; U64 u64; } __attribute__((packed)) unalign; + +static U32 XXH_read32(const void* ptr) { return ((const unalign*)ptr)->u32; } +static U64 XXH_read64(const void* ptr) { return ((const unalign*)ptr)->u64; } + +#else + +/* portable and safe solution. Generally efficient. + * see : http://stackoverflow.com/a/32095106/646947 + */ + +static U32 XXH_read32(const void* memPtr) +{ + U32 val; + memcpy(&val, memPtr, sizeof(val)); + return val; +} + +static U64 XXH_read64(const void* memPtr) +{ + U64 val; + memcpy(&val, memPtr, sizeof(val)); + return val; +} + +#endif /* XXH_FORCE_DIRECT_MEMORY_ACCESS */ + + +/* **************************************** +* Compiler-specific Functions and Macros +******************************************/ +#define GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__) + +/* Note : although _rotl exists for minGW (GCC under windows), performance seems poor */ +#if defined(_MSC_VER) +# define XXH_rotl32(x,r) _rotl(x,r) +# define XXH_rotl64(x,r) _rotl64(x,r) +#else +#if defined(__ICCARM__) +# include +# define XXH_rotl32(x,r) __ROR(x,(32 - r)) +#else +# define XXH_rotl32(x,r) ((x << r) | (x >> (32 - r))) +#endif +# define XXH_rotl64(x,r) ((x << r) | (x >> (64 - r))) +#endif + +#if defined(_MSC_VER) /* Visual Studio */ +# define XXH_swap32 _byteswap_ulong +# define XXH_swap64 _byteswap_uint64 +#elif GCC_VERSION >= 403 +# define XXH_swap32 __builtin_bswap32 +# define XXH_swap64 __builtin_bswap64 +#else +static U32 XXH_swap32 (U32 x) +{ + return ((x << 24) & 0xff000000 ) | + ((x << 8) & 0x00ff0000 ) | + ((x >> 8) & 0x0000ff00 ) | + ((x >> 24) & 0x000000ff ); +} +static U64 XXH_swap64 (U64 x) +{ + return ((x << 56) & 0xff00000000000000ULL) | + ((x << 40) & 0x00ff000000000000ULL) | + ((x << 24) & 0x0000ff0000000000ULL) | + ((x << 8) & 0x000000ff00000000ULL) | + ((x >> 8) & 0x00000000ff000000ULL) | + ((x >> 24) & 0x0000000000ff0000ULL) | + ((x >> 40) & 0x000000000000ff00ULL) | + ((x >> 56) & 0x00000000000000ffULL); +} +#endif + + +/* ************************************* +* Architecture Macros +***************************************/ +typedef enum { XXH_bigEndian=0, XXH_littleEndian=1 } XXH_endianess; + +/* XXH_CPU_LITTLE_ENDIAN can be defined externally, for example on the compiler command line */ +#ifndef XXH_CPU_LITTLE_ENDIAN + static const int g_one = 1; +# define XXH_CPU_LITTLE_ENDIAN (*(const char*)(&g_one)) +#endif + + +/* *************************** +* Memory reads +*****************************/ +typedef enum { XXH_aligned, XXH_unaligned } XXH_alignment; + +FORCE_INLINE_TEMPLATE U32 XXH_readLE32_align(const void* ptr, XXH_endianess endian, XXH_alignment align) +{ + if (align==XXH_unaligned) + return endian==XXH_littleEndian ? XXH_read32(ptr) : XXH_swap32(XXH_read32(ptr)); + else + return endian==XXH_littleEndian ? *(const U32*)ptr : XXH_swap32(*(const U32*)ptr); +} + +FORCE_INLINE_TEMPLATE U32 XXH_readLE32(const void* ptr, XXH_endianess endian) +{ + return XXH_readLE32_align(ptr, endian, XXH_unaligned); +} + +static U32 XXH_readBE32(const void* ptr) +{ + return XXH_CPU_LITTLE_ENDIAN ? XXH_swap32(XXH_read32(ptr)) : XXH_read32(ptr); +} + +FORCE_INLINE_TEMPLATE U64 XXH_readLE64_align(const void* ptr, XXH_endianess endian, XXH_alignment align) +{ + if (align==XXH_unaligned) + return endian==XXH_littleEndian ? XXH_read64(ptr) : XXH_swap64(XXH_read64(ptr)); + else + return endian==XXH_littleEndian ? *(const U64*)ptr : XXH_swap64(*(const U64*)ptr); +} + +FORCE_INLINE_TEMPLATE U64 XXH_readLE64(const void* ptr, XXH_endianess endian) +{ + return XXH_readLE64_align(ptr, endian, XXH_unaligned); +} + +static U64 XXH_readBE64(const void* ptr) +{ + return XXH_CPU_LITTLE_ENDIAN ? XXH_swap64(XXH_read64(ptr)) : XXH_read64(ptr); +} + + +/* ************************************* +* Macros +***************************************/ +#define XXH_STATIC_ASSERT(c) { enum { XXH_static_assert = 1/(int)(!!(c)) }; } /* use only *after* variable declarations */ + + +/* ************************************* +* Constants +***************************************/ +static const U32 PRIME32_1 = 2654435761U; +static const U32 PRIME32_2 = 2246822519U; +static const U32 PRIME32_3 = 3266489917U; +static const U32 PRIME32_4 = 668265263U; +static const U32 PRIME32_5 = 374761393U; + +static const U64 PRIME64_1 = 11400714785074694791ULL; +static const U64 PRIME64_2 = 14029467366897019727ULL; +static const U64 PRIME64_3 = 1609587929392839161ULL; +static const U64 PRIME64_4 = 9650029242287828579ULL; +static const U64 PRIME64_5 = 2870177450012600261ULL; + +XXH_PUBLIC_API unsigned XXH_versionNumber (void) { return XXH_VERSION_NUMBER; } + + +/* ************************** +* Utils +****************************/ +XXH_PUBLIC_API void XXH32_copyState(XXH32_state_t* restrict dstState, const XXH32_state_t* restrict srcState) +{ + memcpy(dstState, srcState, sizeof(*dstState)); +} + +XXH_PUBLIC_API void XXH64_copyState(XXH64_state_t* restrict dstState, const XXH64_state_t* restrict srcState) +{ + memcpy(dstState, srcState, sizeof(*dstState)); +} + + +/* *************************** +* Simple Hash Functions +*****************************/ + +static U32 XXH32_round(U32 seed, U32 input) +{ + seed += input * PRIME32_2; + seed = XXH_rotl32(seed, 13); + seed *= PRIME32_1; + return seed; +} + +FORCE_INLINE_TEMPLATE U32 XXH32_endian_align(const void* input, size_t len, U32 seed, XXH_endianess endian, XXH_alignment align) +{ + const BYTE* p = (const BYTE*)input; + const BYTE* bEnd = p + len; + U32 h32; +#define XXH_get32bits(p) XXH_readLE32_align(p, endian, align) + +#ifdef XXH_ACCEPT_NULL_INPUT_POINTER + if (p==NULL) { + len=0; + bEnd=p=(const BYTE*)(size_t)16; + } +#endif + + if (len>=16) { + const BYTE* const limit = bEnd - 16; + U32 v1 = seed + PRIME32_1 + PRIME32_2; + U32 v2 = seed + PRIME32_2; + U32 v3 = seed + 0; + U32 v4 = seed - PRIME32_1; + + do { + v1 = XXH32_round(v1, XXH_get32bits(p)); p+=4; + v2 = XXH32_round(v2, XXH_get32bits(p)); p+=4; + v3 = XXH32_round(v3, XXH_get32bits(p)); p+=4; + v4 = XXH32_round(v4, XXH_get32bits(p)); p+=4; + } while (p<=limit); + + h32 = XXH_rotl32(v1, 1) + XXH_rotl32(v2, 7) + XXH_rotl32(v3, 12) + XXH_rotl32(v4, 18); + } else { + h32 = seed + PRIME32_5; + } + + h32 += (U32) len; + + while (p+4<=bEnd) { + h32 += XXH_get32bits(p) * PRIME32_3; + h32 = XXH_rotl32(h32, 17) * PRIME32_4 ; + p+=4; + } + + while (p> 15; + h32 *= PRIME32_2; + h32 ^= h32 >> 13; + h32 *= PRIME32_3; + h32 ^= h32 >> 16; + + return h32; +} + + +XXH_PUBLIC_API unsigned int XXH32 (const void* input, size_t len, unsigned int seed) +{ +#if 0 + /* Simple version, good for code maintenance, but unfortunately slow for small inputs */ + XXH32_CREATESTATE_STATIC(state); + XXH32_reset(state, seed); + XXH32_update(state, input, len); + return XXH32_digest(state); +#else + XXH_endianess endian_detected = (XXH_endianess)XXH_CPU_LITTLE_ENDIAN; + + if (XXH_FORCE_ALIGN_CHECK) { + if ((((size_t)input) & 3) == 0) { /* Input is 4-bytes aligned, leverage the speed benefit */ + if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) + return XXH32_endian_align(input, len, seed, XXH_littleEndian, XXH_aligned); + else + return XXH32_endian_align(input, len, seed, XXH_bigEndian, XXH_aligned); + } } + + if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) + return XXH32_endian_align(input, len, seed, XXH_littleEndian, XXH_unaligned); + else + return XXH32_endian_align(input, len, seed, XXH_bigEndian, XXH_unaligned); +#endif +} + + +static U64 XXH64_round(U64 acc, U64 input) +{ + acc += input * PRIME64_2; + acc = XXH_rotl64(acc, 31); + acc *= PRIME64_1; + return acc; +} + +static U64 XXH64_mergeRound(U64 acc, U64 val) +{ + val = XXH64_round(0, val); + acc ^= val; + acc = acc * PRIME64_1 + PRIME64_4; + return acc; +} + +FORCE_INLINE_TEMPLATE U64 XXH64_endian_align(const void* input, size_t len, U64 seed, XXH_endianess endian, XXH_alignment align) +{ + const BYTE* p = (const BYTE*)input; + const BYTE* const bEnd = p + len; + U64 h64; +#define XXH_get64bits(p) XXH_readLE64_align(p, endian, align) + +#ifdef XXH_ACCEPT_NULL_INPUT_POINTER + if (p==NULL) { + len=0; + bEnd=p=(const BYTE*)(size_t)32; + } +#endif + + if (len>=32) { + const BYTE* const limit = bEnd - 32; + U64 v1 = seed + PRIME64_1 + PRIME64_2; + U64 v2 = seed + PRIME64_2; + U64 v3 = seed + 0; + U64 v4 = seed - PRIME64_1; + + do { + v1 = XXH64_round(v1, XXH_get64bits(p)); p+=8; + v2 = XXH64_round(v2, XXH_get64bits(p)); p+=8; + v3 = XXH64_round(v3, XXH_get64bits(p)); p+=8; + v4 = XXH64_round(v4, XXH_get64bits(p)); p+=8; + } while (p<=limit); + + h64 = XXH_rotl64(v1, 1) + XXH_rotl64(v2, 7) + XXH_rotl64(v3, 12) + XXH_rotl64(v4, 18); + h64 = XXH64_mergeRound(h64, v1); + h64 = XXH64_mergeRound(h64, v2); + h64 = XXH64_mergeRound(h64, v3); + h64 = XXH64_mergeRound(h64, v4); + + } else { + h64 = seed + PRIME64_5; + } + + h64 += (U64) len; + + while (p+8<=bEnd) { + U64 const k1 = XXH64_round(0, XXH_get64bits(p)); + h64 ^= k1; + h64 = XXH_rotl64(h64,27) * PRIME64_1 + PRIME64_4; + p+=8; + } + + if (p+4<=bEnd) { + h64 ^= (U64)(XXH_get32bits(p)) * PRIME64_1; + h64 = XXH_rotl64(h64, 23) * PRIME64_2 + PRIME64_3; + p+=4; + } + + while (p> 33; + h64 *= PRIME64_2; + h64 ^= h64 >> 29; + h64 *= PRIME64_3; + h64 ^= h64 >> 32; + + return h64; +} + + +XXH_PUBLIC_API unsigned long long XXH64 (const void* input, size_t len, unsigned long long seed) +{ +#if 0 + /* Simple version, good for code maintenance, but unfortunately slow for small inputs */ + XXH64_CREATESTATE_STATIC(state); + XXH64_reset(state, seed); + XXH64_update(state, input, len); + return XXH64_digest(state); +#else + XXH_endianess endian_detected = (XXH_endianess)XXH_CPU_LITTLE_ENDIAN; + + if (XXH_FORCE_ALIGN_CHECK) { + if ((((size_t)input) & 7)==0) { /* Input is aligned, let's leverage the speed advantage */ + if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) + return XXH64_endian_align(input, len, seed, XXH_littleEndian, XXH_aligned); + else + return XXH64_endian_align(input, len, seed, XXH_bigEndian, XXH_aligned); + } } + + if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) + return XXH64_endian_align(input, len, seed, XXH_littleEndian, XXH_unaligned); + else + return XXH64_endian_align(input, len, seed, XXH_bigEndian, XXH_unaligned); +#endif +} + + +/* ************************************************** +* Advanced Hash Functions +****************************************************/ + +XXH_PUBLIC_API XXH32_state_t* XXH32_createState(void) +{ + return (XXH32_state_t*)XXH_malloc(sizeof(XXH32_state_t)); +} +XXH_PUBLIC_API XXH_errorcode XXH32_freeState(XXH32_state_t* statePtr) +{ + XXH_free(statePtr); + return XXH_OK; +} + +XXH_PUBLIC_API XXH64_state_t* XXH64_createState(void) +{ + return (XXH64_state_t*)XXH_malloc(sizeof(XXH64_state_t)); +} +XXH_PUBLIC_API XXH_errorcode XXH64_freeState(XXH64_state_t* statePtr) +{ + XXH_free(statePtr); + return XXH_OK; +} + + +/*** Hash feed ***/ + +XXH_PUBLIC_API XXH_errorcode XXH32_reset(XXH32_state_t* statePtr, unsigned int seed) +{ + XXH32_state_t state; /* using a local state to memcpy() in order to avoid strict-aliasing warnings */ + memset(&state, 0, sizeof(state)-4); /* do not write into reserved, for future removal */ + state.v1 = seed + PRIME32_1 + PRIME32_2; + state.v2 = seed + PRIME32_2; + state.v3 = seed + 0; + state.v4 = seed - PRIME32_1; + memcpy(statePtr, &state, sizeof(state)); + return XXH_OK; +} + + +XXH_PUBLIC_API XXH_errorcode XXH64_reset(XXH64_state_t* statePtr, unsigned long long seed) +{ + XXH64_state_t state; /* using a local state to memcpy() in order to avoid strict-aliasing warnings */ + memset(&state, 0, sizeof(state)-8); /* do not write into reserved, for future removal */ + state.v1 = seed + PRIME64_1 + PRIME64_2; + state.v2 = seed + PRIME64_2; + state.v3 = seed + 0; + state.v4 = seed - PRIME64_1; + memcpy(statePtr, &state, sizeof(state)); + return XXH_OK; +} + + +FORCE_INLINE_TEMPLATE XXH_errorcode XXH32_update_endian (XXH32_state_t* state, const void* input, size_t len, XXH_endianess endian) +{ + const BYTE* p = (const BYTE*)input; + const BYTE* const bEnd = p + len; + +#ifdef XXH_ACCEPT_NULL_INPUT_POINTER + if (input==NULL) return XXH_ERROR; +#endif + + state->total_len_32 += (unsigned)len; + state->large_len |= (len>=16) | (state->total_len_32>=16); + + if (state->memsize + len < 16) { /* fill in tmp buffer */ + XXH_memcpy((BYTE*)(state->mem32) + state->memsize, input, len); + state->memsize += (unsigned)len; + return XXH_OK; + } + + if (state->memsize) { /* some data left from previous update */ + XXH_memcpy((BYTE*)(state->mem32) + state->memsize, input, 16-state->memsize); + { const U32* p32 = state->mem32; + state->v1 = XXH32_round(state->v1, XXH_readLE32(p32, endian)); p32++; + state->v2 = XXH32_round(state->v2, XXH_readLE32(p32, endian)); p32++; + state->v3 = XXH32_round(state->v3, XXH_readLE32(p32, endian)); p32++; + state->v4 = XXH32_round(state->v4, XXH_readLE32(p32, endian)); p32++; + } + p += 16-state->memsize; + state->memsize = 0; + } + + if (p <= bEnd-16) { + const BYTE* const limit = bEnd - 16; + U32 v1 = state->v1; + U32 v2 = state->v2; + U32 v3 = state->v3; + U32 v4 = state->v4; + + do { + v1 = XXH32_round(v1, XXH_readLE32(p, endian)); p+=4; + v2 = XXH32_round(v2, XXH_readLE32(p, endian)); p+=4; + v3 = XXH32_round(v3, XXH_readLE32(p, endian)); p+=4; + v4 = XXH32_round(v4, XXH_readLE32(p, endian)); p+=4; + } while (p<=limit); + + state->v1 = v1; + state->v2 = v2; + state->v3 = v3; + state->v4 = v4; + } + + if (p < bEnd) { + XXH_memcpy(state->mem32, p, (size_t)(bEnd-p)); + state->memsize = (unsigned)(bEnd-p); + } + + return XXH_OK; +} + +XXH_PUBLIC_API XXH_errorcode XXH32_update (XXH32_state_t* state_in, const void* input, size_t len) +{ + XXH_endianess endian_detected = (XXH_endianess)XXH_CPU_LITTLE_ENDIAN; + + if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) + return XXH32_update_endian(state_in, input, len, XXH_littleEndian); + else + return XXH32_update_endian(state_in, input, len, XXH_bigEndian); +} + + + +FORCE_INLINE_TEMPLATE U32 XXH32_digest_endian (const XXH32_state_t* state, XXH_endianess endian) +{ + const BYTE * p = (const BYTE*)state->mem32; + const BYTE* const bEnd = (const BYTE*)(state->mem32) + state->memsize; + U32 h32; + + if (state->large_len) { + h32 = XXH_rotl32(state->v1, 1) + XXH_rotl32(state->v2, 7) + XXH_rotl32(state->v3, 12) + XXH_rotl32(state->v4, 18); + } else { + h32 = state->v3 /* == seed */ + PRIME32_5; + } + + h32 += state->total_len_32; + + while (p+4<=bEnd) { + h32 += XXH_readLE32(p, endian) * PRIME32_3; + h32 = XXH_rotl32(h32, 17) * PRIME32_4; + p+=4; + } + + while (p> 15; + h32 *= PRIME32_2; + h32 ^= h32 >> 13; + h32 *= PRIME32_3; + h32 ^= h32 >> 16; + + return h32; +} + + +XXH_PUBLIC_API unsigned int XXH32_digest (const XXH32_state_t* state_in) +{ + XXH_endianess endian_detected = (XXH_endianess)XXH_CPU_LITTLE_ENDIAN; + + if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) + return XXH32_digest_endian(state_in, XXH_littleEndian); + else + return XXH32_digest_endian(state_in, XXH_bigEndian); +} + + + +/* **** XXH64 **** */ + +FORCE_INLINE_TEMPLATE XXH_errorcode XXH64_update_endian (XXH64_state_t* state, const void* input, size_t len, XXH_endianess endian) +{ + const BYTE* p = (const BYTE*)input; + const BYTE* const bEnd = p + len; + +#ifdef XXH_ACCEPT_NULL_INPUT_POINTER + if (input==NULL) return XXH_ERROR; +#endif + + state->total_len += len; + + if (state->memsize + len < 32) { /* fill in tmp buffer */ + if (input != NULL) { + XXH_memcpy(((BYTE*)state->mem64) + state->memsize, input, len); + } + state->memsize += (U32)len; + return XXH_OK; + } + + if (state->memsize) { /* tmp buffer is full */ + XXH_memcpy(((BYTE*)state->mem64) + state->memsize, input, 32-state->memsize); + state->v1 = XXH64_round(state->v1, XXH_readLE64(state->mem64+0, endian)); + state->v2 = XXH64_round(state->v2, XXH_readLE64(state->mem64+1, endian)); + state->v3 = XXH64_round(state->v3, XXH_readLE64(state->mem64+2, endian)); + state->v4 = XXH64_round(state->v4, XXH_readLE64(state->mem64+3, endian)); + p += 32-state->memsize; + state->memsize = 0; + } + + if (p+32 <= bEnd) { + const BYTE* const limit = bEnd - 32; + U64 v1 = state->v1; + U64 v2 = state->v2; + U64 v3 = state->v3; + U64 v4 = state->v4; + + do { + v1 = XXH64_round(v1, XXH_readLE64(p, endian)); p+=8; + v2 = XXH64_round(v2, XXH_readLE64(p, endian)); p+=8; + v3 = XXH64_round(v3, XXH_readLE64(p, endian)); p+=8; + v4 = XXH64_round(v4, XXH_readLE64(p, endian)); p+=8; + } while (p<=limit); + + state->v1 = v1; + state->v2 = v2; + state->v3 = v3; + state->v4 = v4; + } + + if (p < bEnd) { + XXH_memcpy(state->mem64, p, (size_t)(bEnd-p)); + state->memsize = (unsigned)(bEnd-p); + } + + return XXH_OK; +} + +XXH_PUBLIC_API XXH_errorcode XXH64_update (XXH64_state_t* state_in, const void* input, size_t len) +{ + XXH_endianess endian_detected = (XXH_endianess)XXH_CPU_LITTLE_ENDIAN; + + if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) + return XXH64_update_endian(state_in, input, len, XXH_littleEndian); + else + return XXH64_update_endian(state_in, input, len, XXH_bigEndian); +} + + + +FORCE_INLINE_TEMPLATE U64 XXH64_digest_endian (const XXH64_state_t* state, XXH_endianess endian) +{ + const BYTE * p = (const BYTE*)state->mem64; + const BYTE* const bEnd = (const BYTE*)state->mem64 + state->memsize; + U64 h64; + + if (state->total_len >= 32) { + U64 const v1 = state->v1; + U64 const v2 = state->v2; + U64 const v3 = state->v3; + U64 const v4 = state->v4; + + h64 = XXH_rotl64(v1, 1) + XXH_rotl64(v2, 7) + XXH_rotl64(v3, 12) + XXH_rotl64(v4, 18); + h64 = XXH64_mergeRound(h64, v1); + h64 = XXH64_mergeRound(h64, v2); + h64 = XXH64_mergeRound(h64, v3); + h64 = XXH64_mergeRound(h64, v4); + } else { + h64 = state->v3 + PRIME64_5; + } + + h64 += (U64) state->total_len; + + while (p+8<=bEnd) { + U64 const k1 = XXH64_round(0, XXH_readLE64(p, endian)); + h64 ^= k1; + h64 = XXH_rotl64(h64,27) * PRIME64_1 + PRIME64_4; + p+=8; + } + + if (p+4<=bEnd) { + h64 ^= (U64)(XXH_readLE32(p, endian)) * PRIME64_1; + h64 = XXH_rotl64(h64, 23) * PRIME64_2 + PRIME64_3; + p+=4; + } + + while (p> 33; + h64 *= PRIME64_2; + h64 ^= h64 >> 29; + h64 *= PRIME64_3; + h64 ^= h64 >> 32; + + return h64; +} + + +XXH_PUBLIC_API unsigned long long XXH64_digest (const XXH64_state_t* state_in) +{ + XXH_endianess endian_detected = (XXH_endianess)XXH_CPU_LITTLE_ENDIAN; + + if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) + return XXH64_digest_endian(state_in, XXH_littleEndian); + else + return XXH64_digest_endian(state_in, XXH_bigEndian); +} + + +/* ************************** +* Canonical representation +****************************/ + +/*! Default XXH result types are basic unsigned 32 and 64 bits. +* The canonical representation follows human-readable write convention, aka big-endian (large digits first). +* These functions allow transformation of hash result into and from its canonical format. +* This way, hash values can be written into a file or buffer, and remain comparable across different systems and programs. +*/ + +XXH_PUBLIC_API void XXH32_canonicalFromHash(XXH32_canonical_t* dst, XXH32_hash_t hash) +{ + XXH_STATIC_ASSERT(sizeof(XXH32_canonical_t) == sizeof(XXH32_hash_t)); + if (XXH_CPU_LITTLE_ENDIAN) hash = XXH_swap32(hash); + memcpy(dst, &hash, sizeof(*dst)); +} + +XXH_PUBLIC_API void XXH64_canonicalFromHash(XXH64_canonical_t* dst, XXH64_hash_t hash) +{ + XXH_STATIC_ASSERT(sizeof(XXH64_canonical_t) == sizeof(XXH64_hash_t)); + if (XXH_CPU_LITTLE_ENDIAN) hash = XXH_swap64(hash); + memcpy(dst, &hash, sizeof(*dst)); +} + +XXH_PUBLIC_API XXH32_hash_t XXH32_hashFromCanonical(const XXH32_canonical_t* src) +{ + return XXH_readBE32(src); +} + +XXH_PUBLIC_API XXH64_hash_t XXH64_hashFromCanonical(const XXH64_canonical_t* src) +{ + return XXH_readBE64(src); +} diff --git a/module/zstd/lib/common/xxhash.h b/module/zstd/lib/common/xxhash.h new file mode 100644 index 000000000000..4207eba8328f --- /dev/null +++ b/module/zstd/lib/common/xxhash.h @@ -0,0 +1,285 @@ +/* + * xxHash - Extremely Fast Hash algorithm + * Header File + * Copyright (c) 2012-2020, Yann Collet, Facebook, Inc. + * + * You can contact the author at : + * - xxHash source repository : https://github.com/Cyan4973/xxHash + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. +*/ + +/* Notice extracted from xxHash homepage : + +xxHash is an extremely fast Hash algorithm, running at RAM speed limits. +It also successfully passes all tests from the SMHasher suite. + +Comparison (single thread, Windows Seven 32 bits, using SMHasher on a Core 2 Duo @3GHz) + +Name Speed Q.Score Author +xxHash 5.4 GB/s 10 +CrapWow 3.2 GB/s 2 Andrew +MumurHash 3a 2.7 GB/s 10 Austin Appleby +SpookyHash 2.0 GB/s 10 Bob Jenkins +SBox 1.4 GB/s 9 Bret Mulvey +Lookup3 1.2 GB/s 9 Bob Jenkins +SuperFastHash 1.2 GB/s 1 Paul Hsieh +CityHash64 1.05 GB/s 10 Pike & Alakuijala +FNV 0.55 GB/s 5 Fowler, Noll, Vo +CRC32 0.43 GB/s 9 +MD5-32 0.33 GB/s 10 Ronald L. Rivest +SHA1-32 0.28 GB/s 10 + +Q.Score is a measure of quality of the hash function. +It depends on successfully passing SMHasher test set. +10 is a perfect score. + +A 64-bits version, named XXH64, is available since r35. +It offers much better speed, but for 64-bits applications only. +Name Speed on 64 bits Speed on 32 bits +XXH64 13.8 GB/s 1.9 GB/s +XXH32 6.8 GB/s 6.0 GB/s +*/ + +#if defined (__cplusplus) +extern "C" { +#endif + +#ifndef XXHASH_H_5627135585666179 +#define XXHASH_H_5627135585666179 1 + + +/* **************************** +* Definitions +******************************/ +#include /* size_t */ +typedef enum { XXH_OK=0, XXH_ERROR } XXH_errorcode; + + +/* **************************** +* API modifier +******************************/ +/** XXH_PRIVATE_API +* This is useful if you want to include xxhash functions in `static` mode +* in order to inline them, and remove their symbol from the public list. +* Methodology : +* #define XXH_PRIVATE_API +* #include "xxhash.h" +* `xxhash.c` is automatically included. +* It's not useful to compile and link it as a separate module anymore. +*/ +#ifdef XXH_PRIVATE_API +# ifndef XXH_STATIC_LINKING_ONLY +# define XXH_STATIC_LINKING_ONLY +# endif +# if defined(__GNUC__) +# define XXH_PUBLIC_API static __inline __attribute__((unused)) +# elif defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) +# define XXH_PUBLIC_API static inline +# elif defined(_MSC_VER) +# define XXH_PUBLIC_API static __inline +# else +# define XXH_PUBLIC_API static /* this version may generate warnings for unused static functions; disable the relevant warning */ +# endif +#else +# define XXH_PUBLIC_API /* do nothing */ +#endif /* XXH_PRIVATE_API */ + +/*!XXH_NAMESPACE, aka Namespace Emulation : + +If you want to include _and expose_ xxHash functions from within your own library, +but also want to avoid symbol collisions with another library which also includes xxHash, + +you can use XXH_NAMESPACE, to automatically prefix any public symbol from xxhash library +with the value of XXH_NAMESPACE (so avoid to keep it NULL and avoid numeric values). + +Note that no change is required within the calling program as long as it includes `xxhash.h` : +regular symbol name will be automatically translated by this header. +*/ +#ifdef XXH_NAMESPACE +# define XXH_CAT(A,B) A##B +# define XXH_NAME2(A,B) XXH_CAT(A,B) +# define XXH32 XXH_NAME2(XXH_NAMESPACE, XXH32) +# define XXH64 XXH_NAME2(XXH_NAMESPACE, XXH64) +# define XXH_versionNumber XXH_NAME2(XXH_NAMESPACE, XXH_versionNumber) +# define XXH32_createState XXH_NAME2(XXH_NAMESPACE, XXH32_createState) +# define XXH64_createState XXH_NAME2(XXH_NAMESPACE, XXH64_createState) +# define XXH32_freeState XXH_NAME2(XXH_NAMESPACE, XXH32_freeState) +# define XXH64_freeState XXH_NAME2(XXH_NAMESPACE, XXH64_freeState) +# define XXH32_reset XXH_NAME2(XXH_NAMESPACE, XXH32_reset) +# define XXH64_reset XXH_NAME2(XXH_NAMESPACE, XXH64_reset) +# define XXH32_update XXH_NAME2(XXH_NAMESPACE, XXH32_update) +# define XXH64_update XXH_NAME2(XXH_NAMESPACE, XXH64_update) +# define XXH32_digest XXH_NAME2(XXH_NAMESPACE, XXH32_digest) +# define XXH64_digest XXH_NAME2(XXH_NAMESPACE, XXH64_digest) +# define XXH32_copyState XXH_NAME2(XXH_NAMESPACE, XXH32_copyState) +# define XXH64_copyState XXH_NAME2(XXH_NAMESPACE, XXH64_copyState) +# define XXH32_canonicalFromHash XXH_NAME2(XXH_NAMESPACE, XXH32_canonicalFromHash) +# define XXH64_canonicalFromHash XXH_NAME2(XXH_NAMESPACE, XXH64_canonicalFromHash) +# define XXH32_hashFromCanonical XXH_NAME2(XXH_NAMESPACE, XXH32_hashFromCanonical) +# define XXH64_hashFromCanonical XXH_NAME2(XXH_NAMESPACE, XXH64_hashFromCanonical) +#endif + + +/* ************************************* +* Version +***************************************/ +#define XXH_VERSION_MAJOR 0 +#define XXH_VERSION_MINOR 6 +#define XXH_VERSION_RELEASE 2 +#define XXH_VERSION_NUMBER (XXH_VERSION_MAJOR *100*100 + XXH_VERSION_MINOR *100 + XXH_VERSION_RELEASE) +XXH_PUBLIC_API unsigned XXH_versionNumber (void); + + +/* **************************** +* Simple Hash Functions +******************************/ +typedef unsigned int XXH32_hash_t; +typedef unsigned long long XXH64_hash_t; + +XXH_PUBLIC_API XXH32_hash_t XXH32 (const void* input, size_t length, unsigned int seed); +XXH_PUBLIC_API XXH64_hash_t XXH64 (const void* input, size_t length, unsigned long long seed); + +/*! +XXH32() : + Calculate the 32-bits hash of sequence "length" bytes stored at memory address "input". + The memory between input & input+length must be valid (allocated and read-accessible). + "seed" can be used to alter the result predictably. + Speed on Core 2 Duo @ 3 GHz (single thread, SMHasher benchmark) : 5.4 GB/s +XXH64() : + Calculate the 64-bits hash of sequence of length "len" stored at memory address "input". + "seed" can be used to alter the result predictably. + This function runs 2x faster on 64-bits systems, but slower on 32-bits systems (see benchmark). +*/ + + +/* **************************** +* Streaming Hash Functions +******************************/ +typedef struct XXH32_state_s XXH32_state_t; /* incomplete type */ +typedef struct XXH64_state_s XXH64_state_t; /* incomplete type */ + +/*! State allocation, compatible with dynamic libraries */ + +XXH_PUBLIC_API XXH32_state_t* XXH32_createState(void); +XXH_PUBLIC_API XXH_errorcode XXH32_freeState(XXH32_state_t* statePtr); + +XXH_PUBLIC_API XXH64_state_t* XXH64_createState(void); +XXH_PUBLIC_API XXH_errorcode XXH64_freeState(XXH64_state_t* statePtr); + + +/* hash streaming */ + +XXH_PUBLIC_API XXH_errorcode XXH32_reset (XXH32_state_t* statePtr, unsigned int seed); +XXH_PUBLIC_API XXH_errorcode XXH32_update (XXH32_state_t* statePtr, const void* input, size_t length); +XXH_PUBLIC_API XXH32_hash_t XXH32_digest (const XXH32_state_t* statePtr); + +XXH_PUBLIC_API XXH_errorcode XXH64_reset (XXH64_state_t* statePtr, unsigned long long seed); +XXH_PUBLIC_API XXH_errorcode XXH64_update (XXH64_state_t* statePtr, const void* input, size_t length); +XXH_PUBLIC_API XXH64_hash_t XXH64_digest (const XXH64_state_t* statePtr); + +/* +These functions generate the xxHash of an input provided in multiple segments. +Note that, for small input, they are slower than single-call functions, due to state management. +For small input, prefer `XXH32()` and `XXH64()` . + +XXH state must first be allocated, using XXH*_createState() . + +Start a new hash by initializing state with a seed, using XXH*_reset(). + +Then, feed the hash state by calling XXH*_update() as many times as necessary. +Obviously, input must be allocated and read accessible. +The function returns an error code, with 0 meaning OK, and any other value meaning there is an error. + +Finally, a hash value can be produced anytime, by using XXH*_digest(). +This function returns the nn-bits hash as an int or long long. + +It's still possible to continue inserting input into the hash state after a digest, +and generate some new hashes later on, by calling again XXH*_digest(). + +When done, free XXH state space if it was allocated dynamically. +*/ + + +/* ************************** +* Utils +****************************/ +#if !(defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L)) /* ! C99 */ +# define restrict /* disable restrict */ +#endif + +XXH_PUBLIC_API void XXH32_copyState(XXH32_state_t* restrict dst_state, const XXH32_state_t* restrict src_state); +XXH_PUBLIC_API void XXH64_copyState(XXH64_state_t* restrict dst_state, const XXH64_state_t* restrict src_state); + + +/* ************************** +* Canonical representation +****************************/ +/* Default result type for XXH functions are primitive unsigned 32 and 64 bits. +* The canonical representation uses human-readable write convention, aka big-endian (large digits first). +* These functions allow transformation of hash result into and from its canonical format. +* This way, hash values can be written into a file / memory, and remain comparable on different systems and programs. +*/ +typedef struct { unsigned char digest[4]; } XXH32_canonical_t; +typedef struct { unsigned char digest[8]; } XXH64_canonical_t; + +XXH_PUBLIC_API void XXH32_canonicalFromHash(XXH32_canonical_t* dst, XXH32_hash_t hash); +XXH_PUBLIC_API void XXH64_canonicalFromHash(XXH64_canonical_t* dst, XXH64_hash_t hash); + +XXH_PUBLIC_API XXH32_hash_t XXH32_hashFromCanonical(const XXH32_canonical_t* src); +XXH_PUBLIC_API XXH64_hash_t XXH64_hashFromCanonical(const XXH64_canonical_t* src); + +#endif /* XXHASH_H_5627135585666179 */ + + + +/* ================================================================================================ + This section contains definitions which are not guaranteed to remain stable. + They may change in future versions, becoming incompatible with a different version of the library. + They shall only be used with static linking. + Never use these definitions in association with dynamic linking ! +=================================================================================================== */ +#if defined(XXH_STATIC_LINKING_ONLY) && !defined(XXH_STATIC_H_3543687687345) +#define XXH_STATIC_H_3543687687345 + +/* These definitions are only meant to allow allocation of XXH state + statically, on stack, or in a struct for example. + Do not use members directly. */ + + struct XXH32_state_s { + unsigned total_len_32; + unsigned large_len; + unsigned v1; + unsigned v2; + unsigned v3; + unsigned v4; + unsigned mem32[4]; /* buffer defined as U32 for alignment */ + unsigned memsize; + unsigned reserved; /* never read nor write, will be removed in a future version */ + }; /* typedef'd to XXH32_state_t */ + + struct XXH64_state_s { + unsigned long long total_len; + unsigned long long v1; + unsigned long long v2; + unsigned long long v3; + unsigned long long v4; + unsigned long long mem64[4]; /* buffer defined as U64 for alignment */ + unsigned memsize; + unsigned reserved[2]; /* never read nor write, will be removed in a future version */ + }; /* typedef'd to XXH64_state_t */ + + +# ifdef XXH_PRIVATE_API +# include "xxhash.c" /* include xxhash functions as `static`, for inlining */ +# endif + +#endif /* XXH_STATIC_LINKING_ONLY && XXH_STATIC_H_3543687687345 */ + + +#if defined (__cplusplus) +} +#endif diff --git a/module/zstd/lib/common/zstd_common.c b/module/zstd/lib/common/zstd_common.c new file mode 100644 index 000000000000..935670b1dce3 --- /dev/null +++ b/module/zstd/lib/common/zstd_common.c @@ -0,0 +1,83 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + + + +/*-************************************* +* Dependencies +***************************************/ +#include /* malloc, calloc, free */ +#include /* memset */ +#include "error_private.h" +#include "zstd_internal.h" + + +/*-**************************************** +* Version +******************************************/ +unsigned ZSTD_versionNumber(void) { return ZSTD_VERSION_NUMBER; } + +const char* ZSTD_versionString(void) { return ZSTD_VERSION_STRING; } + + +/*-**************************************** +* ZSTD Error Management +******************************************/ + +/*! ZSTD_isError() : + * tells if a return value is an error code + * symbol is required for external callers */ +unsigned ZSTD_isError(size_t code) { return ERR_isError(code); } + +/*! ZSTD_getErrorName() : + * provides error code string from function result (useful for debugging) */ +const char* ZSTD_getErrorName(size_t code) { return ERR_getErrorName(code); } + +/*! ZSTD_getError() : + * convert a `size_t` function result into a proper ZSTD_errorCode enum */ +ZSTD_ErrorCode ZSTD_getErrorCode(size_t code) { return ERR_getErrorCode(code); } + +/*! ZSTD_getErrorString() : + * provides error code string from enum */ +const char* ZSTD_getErrorString(ZSTD_ErrorCode code) { return ERR_getErrorString(code); } + + + +/*=************************************************************** +* Custom allocator +****************************************************************/ +void* ZSTD_malloc(size_t size, ZSTD_customMem customMem) +{ + if (customMem.customAlloc) + return customMem.customAlloc(customMem.opaque, size); + return malloc(size); +} + +void* ZSTD_calloc(size_t size, ZSTD_customMem customMem) +{ + if (customMem.customAlloc) { + /* calloc implemented as malloc+memset; + * not as efficient as calloc, but next best guess for custom malloc */ + void* const ptr = customMem.customAlloc(customMem.opaque, size); + memset(ptr, 0, size); + return ptr; + } + return calloc(1, size); +} + +void ZSTD_free(void* ptr, ZSTD_customMem customMem) +{ + if (ptr!=NULL) { + if (customMem.customFree) + customMem.customFree(customMem.opaque, ptr); + else + free(ptr); + } +} diff --git a/module/zstd/lib/zstd_errors.h b/module/zstd/lib/common/zstd_errors.h similarity index 100% rename from module/zstd/lib/zstd_errors.h rename to module/zstd/lib/common/zstd_errors.h diff --git a/module/zstd/lib/common/zstd_internal.h b/module/zstd/lib/common/zstd_internal.h new file mode 100644 index 000000000000..4a86d186a967 --- /dev/null +++ b/module/zstd/lib/common/zstd_internal.h @@ -0,0 +1,446 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#ifndef ZSTD_CCOMMON_H_MODULE +#define ZSTD_CCOMMON_H_MODULE + +/* this module contains definitions which must be identical + * across compression, decompression and dictBuilder. + * It also contains a few functions useful to at least 2 of them + * and which benefit from being inlined */ + +/*-************************************* +* Dependencies +***************************************/ +#if !defined(ZSTD_NO_INTRINSICS) && defined(__ARM_NEON) +#include +#endif +#include "compiler.h" +#include "mem.h" +#include "debug.h" /* assert, DEBUGLOG, RAWLOG, g_debuglevel */ +#include "error_private.h" +#define ZSTD_STATIC_LINKING_ONLY +#include "../zstd.h" +#define FSE_STATIC_LINKING_ONLY +#include "fse.h" +#define HUF_STATIC_LINKING_ONLY +#include "huf.h" +#ifndef XXH_STATIC_LINKING_ONLY +# define XXH_STATIC_LINKING_ONLY /* XXH64_state_t */ +#endif +#include "xxhash.h" /* XXH_reset, update, digest */ + +#if defined (__cplusplus) +extern "C" { +#endif + +/* ---- static assert (debug) --- */ +#define ZSTD_STATIC_ASSERT(c) DEBUG_STATIC_ASSERT(c) +#define FSE_isError ERR_isError +#define HUF_isError ERR_isError + + +/*-************************************* +* shared macros +***************************************/ +#undef MIN +#undef MAX +#define MIN(a,b) ((a)<(b) ? (a) : (b)) +#define MAX(a,b) ((a)>(b) ? (a) : (b)) + +/** + * Ignore: this is an internal helper. + * + * This is a helper function to help force C99-correctness during compilation. + * Under strict compilation modes, variadic macro arguments can't be empty. + * However, variadic function arguments can be. Using a function therefore lets + * us statically check that at least one (string) argument was passed, + * independent of the compilation flags. + */ +static INLINE_KEYWORD UNUSED_ATTR +void _force_has_format_string(const char *format, ...) { + (void)format; +} + +/** + * Ignore: this is an internal helper. + * + * We want to force this function invocation to be syntactically correct, but + * we don't want to force runtime evaluation of its arguments. + */ +#define _FORCE_HAS_FORMAT_STRING(...) \ + if (0) { \ + _force_has_format_string(__VA_ARGS__); \ + } + +/** + * Return the specified error if the condition evaluates to true. + * + * In debug modes, prints additional information. + * In order to do that (particularly, printing the conditional that failed), + * this can't just wrap RETURN_ERROR(). + */ +#define RETURN_ERROR_IF(cond, err, ...) \ + if (cond) { \ + RAWLOG(3, "%s:%d: ERROR!: check %s failed, returning %s", \ + __FILE__, __LINE__, ZSTD_QUOTE(cond), ZSTD_QUOTE(ERROR(err))); \ + _FORCE_HAS_FORMAT_STRING(__VA_ARGS__); \ + RAWLOG(3, ": " __VA_ARGS__); \ + RAWLOG(3, "\n"); \ + return ERROR(err); \ + } + +/** + * Unconditionally return the specified error. + * + * In debug modes, prints additional information. + */ +#define RETURN_ERROR(err, ...) \ + do { \ + RAWLOG(3, "%s:%d: ERROR!: unconditional check failed, returning %s", \ + __FILE__, __LINE__, ZSTD_QUOTE(ERROR(err))); \ + _FORCE_HAS_FORMAT_STRING(__VA_ARGS__); \ + RAWLOG(3, ": " __VA_ARGS__); \ + RAWLOG(3, "\n"); \ + return ERROR(err); \ + } while(0); + +/** + * If the provided expression evaluates to an error code, returns that error code. + * + * In debug modes, prints additional information. + */ +#define FORWARD_IF_ERROR(err, ...) \ + do { \ + size_t const err_code = (err); \ + if (ERR_isError(err_code)) { \ + RAWLOG(3, "%s:%d: ERROR!: forwarding error in %s: %s", \ + __FILE__, __LINE__, ZSTD_QUOTE(err), ERR_getErrorName(err_code)); \ + _FORCE_HAS_FORMAT_STRING(__VA_ARGS__); \ + RAWLOG(3, ": " __VA_ARGS__); \ + RAWLOG(3, "\n"); \ + return err_code; \ + } \ + } while(0); + + +/*-************************************* +* Common constants +***************************************/ +#define ZSTD_OPT_NUM (1<<12) + +#define ZSTD_REP_NUM 3 /* number of repcodes */ +#define ZSTD_REP_MOVE (ZSTD_REP_NUM-1) +static const U32 repStartValue[ZSTD_REP_NUM] = { 1, 4, 8 }; + +#define KB *(1 <<10) +#define MB *(1 <<20) +#define GB *(1U<<30) + +#define BIT7 128 +#define BIT6 64 +#define BIT5 32 +#define BIT4 16 +#define BIT1 2 +#define BIT0 1 + +#define ZSTD_WINDOWLOG_ABSOLUTEMIN 10 +static const size_t ZSTD_fcs_fieldSize[4] = { 0, 2, 4, 8 }; +static const size_t ZSTD_did_fieldSize[4] = { 0, 1, 2, 4 }; + +#define ZSTD_FRAMEIDSIZE 4 /* magic number size */ + +#define ZSTD_BLOCKHEADERSIZE 3 /* C standard doesn't allow `static const` variable to be init using another `static const` variable */ +static const size_t ZSTD_blockHeaderSize = ZSTD_BLOCKHEADERSIZE; +typedef enum { bt_raw, bt_rle, bt_compressed, bt_reserved } blockType_e; + +#define ZSTD_FRAMECHECKSUMSIZE 4 + +#define MIN_SEQUENCES_SIZE 1 /* nbSeq==0 */ +#define MIN_CBLOCK_SIZE (1 /*litCSize*/ + 1 /* RLE or RAW */ + MIN_SEQUENCES_SIZE /* nbSeq==0 */) /* for a non-null block */ + +#define HufLog 12 +typedef enum { set_basic, set_rle, set_compressed, set_repeat } symbolEncodingType_e; + +#define LONGNBSEQ 0x7F00 + +#define MINMATCH 3 + +#define Litbits 8 +#define MaxLit ((1<= 8 || (ovtype == ZSTD_no_overlap && diff <= -WILDCOPY_VECLEN)); + + if (ovtype == ZSTD_overlap_src_before_dst && diff < WILDCOPY_VECLEN) { + /* Handle short offset copies. */ + do { + COPY8(op, ip) + } while (op < oend); + } else { + assert(diff >= WILDCOPY_VECLEN || diff <= -WILDCOPY_VECLEN); + /* Separate out the first COPY16() call because the copy length is + * almost certain to be short, so the branches have different + * probabilities. Since it is almost certain to be short, only do + * one COPY16() in the first call. Then, do two calls per loop since + * at that point it is more likely to have a high trip count. + */ +#ifndef __aarch64__ + do { + COPY16(op, ip); + } + while (op < oend); +#else + COPY16(op, ip); + if (op >= oend) return; + do { + COPY16(op, ip); + COPY16(op, ip); + } + while (op < oend); +#endif + } +} + +MEM_STATIC size_t ZSTD_limitCopy(void* dst, size_t dstCapacity, const void* src, size_t srcSize) +{ + size_t const length = MIN(dstCapacity, srcSize); + if (length > 0) { + memcpy(dst, src, length); + } + return length; +} + +/* define "workspace is too large" as this number of times larger than needed */ +#define ZSTD_WORKSPACETOOLARGE_FACTOR 3 + +/* when workspace is continuously too large + * during at least this number of times, + * context's memory usage is considered wasteful, + * because it's sized to handle a worst case scenario which rarely happens. + * In which case, resize it down to free some memory */ +#define ZSTD_WORKSPACETOOLARGE_MAXDURATION 128 + + +/*-******************************************* +* Private declarations +*********************************************/ +typedef struct seqDef_s { + U32 offset; + U16 litLength; + U16 matchLength; +} seqDef; + +typedef struct { + seqDef* sequencesStart; + seqDef* sequences; + BYTE* litStart; + BYTE* lit; + BYTE* llCode; + BYTE* mlCode; + BYTE* ofCode; + size_t maxNbSeq; + size_t maxNbLit; + U32 longLengthID; /* 0 == no longLength; 1 == Lit.longLength; 2 == Match.longLength; */ + U32 longLengthPos; +} seqStore_t; + +typedef struct { + U32 litLength; + U32 matchLength; +} ZSTD_sequenceLength; + +/** + * Returns the ZSTD_sequenceLength for the given sequences. It handles the decoding of long sequences + * indicated by longLengthPos and longLengthID, and adds MINMATCH back to matchLength. + */ +MEM_STATIC ZSTD_sequenceLength ZSTD_getSequenceLength(seqStore_t const* seqStore, seqDef const* seq) +{ + ZSTD_sequenceLength seqLen; + seqLen.litLength = seq->litLength; + seqLen.matchLength = seq->matchLength + MINMATCH; + if (seqStore->longLengthPos == (U32)(seq - seqStore->sequencesStart)) { + if (seqStore->longLengthID == 1) { + seqLen.litLength += 0xFFFF; + } + if (seqStore->longLengthID == 2) { + seqLen.matchLength += 0xFFFF; + } + } + return seqLen; +} + +/** + * Contains the compressed frame size and an upper-bound for the decompressed frame size. + * Note: before using `compressedSize`, check for errors using ZSTD_isError(). + * similarly, before using `decompressedBound`, check for errors using: + * `decompressedBound != ZSTD_CONTENTSIZE_ERROR` + */ +typedef struct { + size_t compressedSize; + unsigned long long decompressedBound; +} ZSTD_frameSizeInfo; /* decompress & legacy */ + +const seqStore_t* ZSTD_getSeqStore(const ZSTD_CCtx* ctx); /* compress & dictBuilder */ +void ZSTD_seqToCodes(const seqStore_t* seqStorePtr); /* compress, dictBuilder, decodeCorpus (shouldn't get its definition from here) */ + +/* custom memory allocation functions */ +void* ZSTD_malloc(size_t size, ZSTD_customMem customMem); +void* ZSTD_calloc(size_t size, ZSTD_customMem customMem); +void ZSTD_free(void* ptr, ZSTD_customMem customMem); + + +MEM_STATIC U32 ZSTD_highbit32(U32 val) /* compress, dictBuilder, decodeCorpus */ +{ + assert(val != 0); + { +# if defined(_MSC_VER) /* Visual */ + unsigned long r=0; + return _BitScanReverse(&r, val) ? (unsigned)r : 0; +# elif defined(__GNUC__) && (__GNUC__ >= 3) /* GCC Intrinsic */ + return __builtin_clz (val) ^ 31; +# elif defined(__ICCARM__) /* IAR Intrinsic */ + return 31 - __CLZ(val); +# else /* Software version */ + static const U32 DeBruijnClz[32] = { 0, 9, 1, 10, 13, 21, 2, 29, 11, 14, 16, 18, 22, 25, 3, 30, 8, 12, 20, 28, 15, 17, 24, 7, 19, 27, 23, 6, 26, 5, 4, 31 }; + U32 v = val; + v |= v >> 1; + v |= v >> 2; + v |= v >> 4; + v |= v >> 8; + v |= v >> 16; + return DeBruijnClz[(v * 0x07C4ACDDU) >> 27]; +# endif + } +} + + +/* ZSTD_invalidateRepCodes() : + * ensures next compression will not use repcodes from previous block. + * Note : only works with regular variant; + * do not use with extDict variant ! */ +void ZSTD_invalidateRepCodes(ZSTD_CCtx* cctx); /* zstdmt, adaptive_compression (shouldn't get this definition from here) */ + + +typedef struct { + blockType_e blockType; + U32 lastBlock; + U32 origSize; +} blockProperties_t; /* declared here for decompress and fullbench */ + +/*! ZSTD_getcBlockSize() : + * Provides the size of compressed block from block header `src` */ +/* Used by: decompress, fullbench (does not get its definition from here) */ +size_t ZSTD_getcBlockSize(const void* src, size_t srcSize, + blockProperties_t* bpPtr); + +/*! ZSTD_decodeSeqHeaders() : + * decode sequence header from src */ +/* Used by: decompress, fullbench (does not get its definition from here) */ +size_t ZSTD_decodeSeqHeaders(ZSTD_DCtx* dctx, int* nbSeqPtr, + const void* src, size_t srcSize); + + +#if defined (__cplusplus) +} +#endif + +#endif /* ZSTD_CCOMMON_H_MODULE */ diff --git a/module/zstd/lib/compress/fse_compress.c b/module/zstd/lib/compress/fse_compress.c new file mode 100644 index 000000000000..e27414ccbbcd --- /dev/null +++ b/module/zstd/lib/compress/fse_compress.c @@ -0,0 +1,698 @@ +/* ****************************************************************** + * FSE : Finite State Entropy encoder + * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. + * + * You can contact the author at : + * - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy + * - Public forum : https://groups.google.com/forum/#!forum/lz4c + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. +****************************************************************** */ + +/* ************************************************************** +* Includes +****************************************************************/ +#include /* malloc, free, qsort */ +#include /* memcpy, memset */ +#include "../common/compiler.h" +#include "../common/mem.h" /* U32, U16, etc. */ +#include "../common/debug.h" /* assert, DEBUGLOG */ +#include "hist.h" /* HIST_count_wksp */ +#include "../common/bitstream.h" +#define FSE_STATIC_LINKING_ONLY +#include "../common/fse.h" +#include "../common/error_private.h" + + +/* ************************************************************** +* Error Management +****************************************************************/ +#define FSE_isError ERR_isError + + +/* ************************************************************** +* Templates +****************************************************************/ +/* + designed to be included + for type-specific functions (template emulation in C) + Objective is to write these functions only once, for improved maintenance +*/ + +/* safety checks */ +#ifndef FSE_FUNCTION_EXTENSION +# error "FSE_FUNCTION_EXTENSION must be defined" +#endif +#ifndef FSE_FUNCTION_TYPE +# error "FSE_FUNCTION_TYPE must be defined" +#endif + +/* Function names */ +#define FSE_CAT(X,Y) X##Y +#define FSE_FUNCTION_NAME(X,Y) FSE_CAT(X,Y) +#define FSE_TYPE_NAME(X,Y) FSE_CAT(X,Y) + + +/* Function templates */ + +/* FSE_buildCTable_wksp() : + * Same as FSE_buildCTable(), but using an externally allocated scratch buffer (`workSpace`). + * wkspSize should be sized to handle worst case situation, which is `1<>1 : 1) ; + FSE_symbolCompressionTransform* const symbolTT = (FSE_symbolCompressionTransform*) (FSCT); + U32 const step = FSE_TABLESTEP(tableSize); + U32 cumul[FSE_MAX_SYMBOL_VALUE+2]; + + FSE_FUNCTION_TYPE* const tableSymbol = (FSE_FUNCTION_TYPE*)workSpace; + U32 highThreshold = tableSize-1; + + /* CTable header */ + if (((size_t)1 << tableLog) * sizeof(FSE_FUNCTION_TYPE) > wkspSize) return ERROR(tableLog_tooLarge); + tableU16[-2] = (U16) tableLog; + tableU16[-1] = (U16) maxSymbolValue; + assert(tableLog < 16); /* required for threshold strategy to work */ + + /* For explanations on how to distribute symbol values over the table : + * http://fastcompression.blogspot.fr/2014/02/fse-distributing-symbol-values.html */ + + #ifdef __clang_analyzer__ + memset(tableSymbol, 0, sizeof(*tableSymbol) * tableSize); /* useless initialization, just to keep scan-build happy */ + #endif + + /* symbol start positions */ + { U32 u; + cumul[0] = 0; + for (u=1; u <= maxSymbolValue+1; u++) { + if (normalizedCounter[u-1]==-1) { /* Low proba symbol */ + cumul[u] = cumul[u-1] + 1; + tableSymbol[highThreshold--] = (FSE_FUNCTION_TYPE)(u-1); + } else { + cumul[u] = cumul[u-1] + normalizedCounter[u-1]; + } } + cumul[maxSymbolValue+1] = tableSize+1; + } + + /* Spread symbols */ + { U32 position = 0; + U32 symbol; + for (symbol=0; symbol<=maxSymbolValue; symbol++) { + int nbOccurrences; + int const freq = normalizedCounter[symbol]; + for (nbOccurrences=0; nbOccurrences highThreshold) + position = (position + step) & tableMask; /* Low proba area */ + } } + + assert(position==0); /* Must have initialized all positions */ + } + + /* Build table */ + { U32 u; for (u=0; u> 3) + 3; + return maxSymbolValue ? maxHeaderSize : FSE_NCOUNTBOUND; /* maxSymbolValue==0 ? use default */ +} + +static size_t +FSE_writeNCount_generic (void* header, size_t headerBufferSize, + const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog, + unsigned writeIsSafe) +{ + BYTE* const ostart = (BYTE*) header; + BYTE* out = ostart; + BYTE* const oend = ostart + headerBufferSize; + int nbBits; + const int tableSize = 1 << tableLog; + int remaining; + int threshold; + U32 bitStream = 0; + int bitCount = 0; + unsigned symbol = 0; + unsigned const alphabetSize = maxSymbolValue + 1; + int previousIs0 = 0; + + /* Table Size */ + bitStream += (tableLog-FSE_MIN_TABLELOG) << bitCount; + bitCount += 4; + + /* Init */ + remaining = tableSize+1; /* +1 for extra accuracy */ + threshold = tableSize; + nbBits = tableLog+1; + + while ((symbol < alphabetSize) && (remaining>1)) { /* stops at 1 */ + if (previousIs0) { + unsigned start = symbol; + while ((symbol < alphabetSize) && !normalizedCounter[symbol]) symbol++; + if (symbol == alphabetSize) break; /* incorrect distribution */ + while (symbol >= start+24) { + start+=24; + bitStream += 0xFFFFU << bitCount; + if ((!writeIsSafe) && (out > oend-2)) + return ERROR(dstSize_tooSmall); /* Buffer overflow */ + out[0] = (BYTE) bitStream; + out[1] = (BYTE)(bitStream>>8); + out+=2; + bitStream>>=16; + } + while (symbol >= start+3) { + start+=3; + bitStream += 3 << bitCount; + bitCount += 2; + } + bitStream += (symbol-start) << bitCount; + bitCount += 2; + if (bitCount>16) { + if ((!writeIsSafe) && (out > oend - 2)) + return ERROR(dstSize_tooSmall); /* Buffer overflow */ + out[0] = (BYTE)bitStream; + out[1] = (BYTE)(bitStream>>8); + out += 2; + bitStream >>= 16; + bitCount -= 16; + } } + { int count = normalizedCounter[symbol++]; + int const max = (2*threshold-1) - remaining; + remaining -= count < 0 ? -count : count; + count++; /* +1 for extra accuracy */ + if (count>=threshold) + count += max; /* [0..max[ [max..threshold[ (...) [threshold+max 2*threshold[ */ + bitStream += count << bitCount; + bitCount += nbBits; + bitCount -= (count>=1; } + } + if (bitCount>16) { + if ((!writeIsSafe) && (out > oend - 2)) + return ERROR(dstSize_tooSmall); /* Buffer overflow */ + out[0] = (BYTE)bitStream; + out[1] = (BYTE)(bitStream>>8); + out += 2; + bitStream >>= 16; + bitCount -= 16; + } } + + if (remaining != 1) + return ERROR(GENERIC); /* incorrect normalized distribution */ + assert(symbol <= alphabetSize); + + /* flush remaining bitStream */ + if ((!writeIsSafe) && (out > oend - 2)) + return ERROR(dstSize_tooSmall); /* Buffer overflow */ + out[0] = (BYTE)bitStream; + out[1] = (BYTE)(bitStream>>8); + out+= (bitCount+7) /8; + + return (out-ostart); +} + + +size_t FSE_writeNCount (void* buffer, size_t bufferSize, + const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog) +{ + if (tableLog > FSE_MAX_TABLELOG) return ERROR(tableLog_tooLarge); /* Unsupported */ + if (tableLog < FSE_MIN_TABLELOG) return ERROR(GENERIC); /* Unsupported */ + + if (bufferSize < FSE_NCountWriteBound(maxSymbolValue, tableLog)) + return FSE_writeNCount_generic(buffer, bufferSize, normalizedCounter, maxSymbolValue, tableLog, 0); + + return FSE_writeNCount_generic(buffer, bufferSize, normalizedCounter, maxSymbolValue, tableLog, 1 /* write in buffer is safe */); +} + + +/*-************************************************************** +* FSE Compression Code +****************************************************************/ + +FSE_CTable* FSE_createCTable (unsigned maxSymbolValue, unsigned tableLog) +{ + size_t size __attribute__ ((unused)); + if (tableLog > FSE_TABLELOG_ABSOLUTE_MAX) tableLog = FSE_TABLELOG_ABSOLUTE_MAX; + size = FSE_CTABLE_SIZE_U32 (tableLog, maxSymbolValue) * sizeof(U32); + return (FSE_CTable*)malloc(size); +} + +void FSE_freeCTable (FSE_CTable* ct) { free(ct); } + +/* provides the minimum logSize to safely represent a distribution */ +static unsigned FSE_minTableLog(size_t srcSize, unsigned maxSymbolValue) +{ + U32 minBitsSrc = BIT_highbit32((U32)(srcSize)) + 1; + U32 minBitsSymbols = BIT_highbit32(maxSymbolValue) + 2; + U32 minBits = minBitsSrc < minBitsSymbols ? minBitsSrc : minBitsSymbols; + assert(srcSize > 1); /* Not supported, RLE should be used instead */ + return minBits; +} + +unsigned FSE_optimalTableLog_internal(unsigned maxTableLog, size_t srcSize, unsigned maxSymbolValue, unsigned minus) +{ + U32 maxBitsSrc = BIT_highbit32((U32)(srcSize - 1)) - minus; + U32 tableLog = maxTableLog; + U32 minBits = FSE_minTableLog(srcSize, maxSymbolValue); + assert(srcSize > 1); /* Not supported, RLE should be used instead */ + if (tableLog==0) tableLog = FSE_DEFAULT_TABLELOG; + if (maxBitsSrc < tableLog) tableLog = maxBitsSrc; /* Accuracy can be reduced */ + if (minBits > tableLog) tableLog = minBits; /* Need a minimum to safely represent all symbol values */ + if (tableLog < FSE_MIN_TABLELOG) tableLog = FSE_MIN_TABLELOG; + if (tableLog > FSE_MAX_TABLELOG) tableLog = FSE_MAX_TABLELOG; + return tableLog; +} + +unsigned FSE_optimalTableLog(unsigned maxTableLog, size_t srcSize, unsigned maxSymbolValue) +{ + return FSE_optimalTableLog_internal(maxTableLog, srcSize, maxSymbolValue, 2); +} + + +/* Secondary normalization method. + To be used when primary method fails. */ + +static size_t FSE_normalizeM2(short* norm, U32 tableLog, const unsigned* count, size_t total, U32 maxSymbolValue) +{ + short const NOT_YET_ASSIGNED = -2; + U32 s; + U32 distributed = 0; + U32 ToDistribute; + + /* Init */ + U32 const lowThreshold = (U32)(total >> tableLog); + U32 lowOne = (U32)((total * 3) >> (tableLog + 1)); + + for (s=0; s<=maxSymbolValue; s++) { + if (count[s] == 0) { + norm[s]=0; + continue; + } + if (count[s] <= lowThreshold) { + norm[s] = -1; + distributed++; + total -= count[s]; + continue; + } + if (count[s] <= lowOne) { + norm[s] = 1; + distributed++; + total -= count[s]; + continue; + } + + norm[s]=NOT_YET_ASSIGNED; + } + ToDistribute = (1 << tableLog) - distributed; + + if (ToDistribute == 0) + return 0; + + if ((total / ToDistribute) > lowOne) { + /* risk of rounding to zero */ + lowOne = (U32)((total * 3) / (ToDistribute * 2)); + for (s=0; s<=maxSymbolValue; s++) { + if ((norm[s] == NOT_YET_ASSIGNED) && (count[s] <= lowOne)) { + norm[s] = 1; + distributed++; + total -= count[s]; + continue; + } } + ToDistribute = (1 << tableLog) - distributed; + } + + if (distributed == maxSymbolValue+1) { + /* all values are pretty poor; + probably incompressible data (should have already been detected); + find max, then give all remaining points to max */ + U32 maxV = 0, maxC = 0; + for (s=0; s<=maxSymbolValue; s++) + if (count[s] > maxC) { maxV=s; maxC=count[s]; } + norm[maxV] += (short)ToDistribute; + return 0; + } + + if (total == 0) { + /* all of the symbols were low enough for the lowOne or lowThreshold */ + for (s=0; ToDistribute > 0; s = (s+1)%(maxSymbolValue+1)) + if (norm[s] > 0) { ToDistribute--; norm[s]++; } + return 0; + } + + { U64 const vStepLog = 62 - tableLog; + U64 const mid = (1ULL << (vStepLog-1)) - 1; + U64 const rStep = ((((U64)1<> vStepLog); + U32 const sEnd = (U32)(end >> vStepLog); + U32 const weight = sEnd - sStart; + if (weight < 1) + return ERROR(GENERIC); + norm[s] = (short)weight; + tmpTotal = end; + } } } + + return 0; +} + + +size_t FSE_normalizeCount (short* normalizedCounter, unsigned tableLog, + const unsigned* count, size_t total, + unsigned maxSymbolValue) +{ + /* Sanity checks */ + if (tableLog==0) tableLog = FSE_DEFAULT_TABLELOG; + if (tableLog < FSE_MIN_TABLELOG) return ERROR(GENERIC); /* Unsupported size */ + if (tableLog > FSE_MAX_TABLELOG) return ERROR(tableLog_tooLarge); /* Unsupported size */ + if (tableLog < FSE_minTableLog(total, maxSymbolValue)) return ERROR(GENERIC); /* Too small tableLog, compression potentially impossible */ + + { static U32 const rtbTable[] = { 0, 473195, 504333, 520860, 550000, 700000, 750000, 830000 }; + U64 const scale = 62 - tableLog; + U64 const step = ((U64)1<<62) / total; /* <== here, one division ! */ + U64 const vStep = 1ULL<<(scale-20); + int stillToDistribute = 1<> tableLog); + + for (s=0; s<=maxSymbolValue; s++) { + if (count[s] == total) return 0; /* rle special case */ + if (count[s] == 0) { normalizedCounter[s]=0; continue; } + if (count[s] <= lowThreshold) { + normalizedCounter[s] = -1; + stillToDistribute--; + } else { + short proba = (short)((count[s]*step) >> scale); + if (proba<8) { + U64 restToBeat = vStep * rtbTable[proba]; + proba += (count[s]*step) - ((U64)proba< restToBeat; + } + if (proba > largestP) { largestP=proba; largest=s; } + normalizedCounter[s] = proba; + stillToDistribute -= proba; + } } + if (-stillToDistribute >= (normalizedCounter[largest] >> 1)) { + /* corner case, need another normalization method */ + size_t const errorCode = FSE_normalizeM2(normalizedCounter, tableLog, count, total, maxSymbolValue); + if (FSE_isError(errorCode)) return errorCode; + } + else normalizedCounter[largest] += (short)stillToDistribute; + } + +#if 0 + { /* Print Table (debug) */ + U32 s; + U32 nTotal = 0; + for (s=0; s<=maxSymbolValue; s++) + RAWLOG(2, "%3i: %4i \n", s, normalizedCounter[s]); + for (s=0; s<=maxSymbolValue; s++) + nTotal += abs(normalizedCounter[s]); + if (nTotal != (1U<>1); /* assumption : tableLog >= 1 */ + FSE_symbolCompressionTransform* const symbolTT = (FSE_symbolCompressionTransform*) (FSCT); + unsigned s; + + /* Sanity checks */ + if (nbBits < 1) return ERROR(GENERIC); /* min size */ + + /* header */ + tableU16[-2] = (U16) nbBits; + tableU16[-1] = (U16) maxSymbolValue; + + /* Build table */ + for (s=0; s FSE_MAX_TABLELOG*4+7 ) && (srcSize & 2)) { /* test bit 2 */ + FSE_encodeSymbol(&bitC, &CState2, *--ip); + FSE_encodeSymbol(&bitC, &CState1, *--ip); + FSE_FLUSHBITS(&bitC); + } + + /* 2 or 4 encoding per loop */ + while ( ip>istart ) { + + FSE_encodeSymbol(&bitC, &CState2, *--ip); + + if (sizeof(bitC.bitContainer)*8 < FSE_MAX_TABLELOG*2+7 ) /* this test must be static */ + FSE_FLUSHBITS(&bitC); + + FSE_encodeSymbol(&bitC, &CState1, *--ip); + + if (sizeof(bitC.bitContainer)*8 > FSE_MAX_TABLELOG*4+7 ) { /* this test must be static */ + FSE_encodeSymbol(&bitC, &CState2, *--ip); + FSE_encodeSymbol(&bitC, &CState1, *--ip); + } + + FSE_FLUSHBITS(&bitC); + } + + FSE_flushCState(&bitC, &CState2); + FSE_flushCState(&bitC, &CState1); + return BIT_closeCStream(&bitC); +} + +size_t FSE_compress_usingCTable (void* dst, size_t dstSize, + const void* src, size_t srcSize, + const FSE_CTable* ct) +{ + unsigned const fast = (dstSize >= FSE_BLOCKBOUND(srcSize)); + + if (fast) + return FSE_compress_usingCTable_generic(dst, dstSize, src, srcSize, ct, 1); + else + return FSE_compress_usingCTable_generic(dst, dstSize, src, srcSize, ct, 0); +} + + +size_t FSE_compressBound(size_t size) { return FSE_COMPRESSBOUND(size); } + +/* FSE_compress_wksp() : + * Same as FSE_compress2(), but using an externally allocated scratch buffer (`workSpace`). + * `wkspSize` size must be `(1< not compressible */ + if (maxCount < (srcSize >> 7)) return 0; /* Heuristic : not compressible enough */ + } + + tableLog = FSE_optimalTableLog(tableLog, srcSize, maxSymbolValue); + CHECK_F( FSE_normalizeCount(norm, tableLog, count, srcSize, maxSymbolValue) ); + + /* Write table description header */ + { CHECK_V_F(nc_err, FSE_writeNCount(op, oend-op, norm, maxSymbolValue, tableLog) ); + op += nc_err; + } + + /* Compress */ + CHECK_F( FSE_buildCTable_wksp(CTable, norm, maxSymbolValue, tableLog, scratchBuffer, scratchBufferSize) ); + { CHECK_V_F(cSize, FSE_compress_usingCTable(op, oend - op, src, srcSize, CTable) ); + if (cSize == 0) return 0; /* not enough space for compressed data */ + op += cSize; + } + + /* check compressibility */ + if ( (size_t)(op-ostart) >= srcSize-1 ) return 0; + + return op-ostart; +} + +typedef struct { + FSE_CTable CTable_max[FSE_CTABLE_SIZE_U32(FSE_MAX_TABLELOG, FSE_MAX_SYMBOL_VALUE)]; + BYTE scratchBuffer[1 << FSE_MAX_TABLELOG]; +} fseWkspMax_t; + +size_t FSE_compress2 (void* dst, size_t dstCapacity, const void* src, size_t srcSize, unsigned maxSymbolValue, unsigned tableLog) +{ + fseWkspMax_t scratchBuffer; + DEBUG_STATIC_ASSERT(sizeof(scratchBuffer) >= FSE_WKSP_SIZE_U32(FSE_MAX_TABLELOG, FSE_MAX_SYMBOL_VALUE)); /* compilation failures here means scratchBuffer is not large enough */ + if (tableLog > FSE_MAX_TABLELOG) return ERROR(tableLog_tooLarge); + return FSE_compress_wksp(dst, dstCapacity, src, srcSize, maxSymbolValue, tableLog, &scratchBuffer, sizeof(scratchBuffer)); +} + +size_t FSE_compress (void* dst, size_t dstCapacity, const void* src, size_t srcSize) +{ + return FSE_compress2(dst, dstCapacity, src, srcSize, FSE_MAX_SYMBOL_VALUE, FSE_DEFAULT_TABLELOG); +} + + +#endif /* FSE_COMMONDEFS_ONLY */ diff --git a/module/zstd/lib/compress/hist.c b/module/zstd/lib/compress/hist.c new file mode 100644 index 000000000000..61e08c7968be --- /dev/null +++ b/module/zstd/lib/compress/hist.c @@ -0,0 +1,183 @@ +/* ****************************************************************** + * hist : Histogram functions + * part of Finite State Entropy project + * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. + * + * You can contact the author at : + * - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy + * - Public forum : https://groups.google.com/forum/#!forum/lz4c + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. +****************************************************************** */ + +/* --- dependencies --- */ +#include "../common/mem.h" /* U32, BYTE, etc. */ +#include "../common/debug.h" /* assert, DEBUGLOG */ +#include "../common/error_private.h" /* ERROR */ +#include "hist.h" + + +/* --- Error management --- */ +unsigned HIST_isError(size_t code) { return ERR_isError(code); } + +/*-************************************************************** + * Histogram functions + ****************************************************************/ +unsigned HIST_count_simple(unsigned* count, unsigned* maxSymbolValuePtr, + const void* src, size_t srcSize) +{ + const BYTE* ip = (const BYTE*)src; + const BYTE* const end = ip + srcSize; + unsigned maxSymbolValue = *maxSymbolValuePtr; + unsigned largestCount=0; + + memset(count, 0, (maxSymbolValue+1) * sizeof(*count)); + if (srcSize==0) { *maxSymbolValuePtr = 0; return 0; } + + while (ip largestCount) largestCount = count[s]; + } + + return largestCount; +} + +typedef enum { trustInput, checkMaxSymbolValue } HIST_checkInput_e; + +/* HIST_count_parallel_wksp() : + * store histogram into 4 intermediate tables, recombined at the end. + * this design makes better use of OoO cpus, + * and is noticeably faster when some values are heavily repeated. + * But it needs some additional workspace for intermediate tables. + * `workSpace` size must be a table of size >= HIST_WKSP_SIZE_U32. + * @return : largest histogram frequency, + * or an error code (notably when histogram would be larger than *maxSymbolValuePtr). */ +static size_t HIST_count_parallel_wksp( + unsigned* count, unsigned* maxSymbolValuePtr, + const void* source, size_t sourceSize, + HIST_checkInput_e check, + U32* const workSpace) +{ + const BYTE* ip = (const BYTE*)source; + const BYTE* const iend = ip+sourceSize; + unsigned maxSymbolValue = *maxSymbolValuePtr; + unsigned max=0; + U32* const Counting1 = workSpace; + U32* const Counting2 = Counting1 + 256; + U32* const Counting3 = Counting2 + 256; + U32* const Counting4 = Counting3 + 256; + + memset(workSpace, 0, 4*256*sizeof(unsigned)); + + /* safety checks */ + if (!sourceSize) { + memset(count, 0, maxSymbolValue + 1); + *maxSymbolValuePtr = 0; + return 0; + } + if (!maxSymbolValue) maxSymbolValue = 255; /* 0 == default */ + + /* by stripes of 16 bytes */ + { U32 cached = MEM_read32(ip); ip += 4; + while (ip < iend-15) { + U32 c = cached; cached = MEM_read32(ip); ip += 4; + Counting1[(BYTE) c ]++; + Counting2[(BYTE)(c>>8) ]++; + Counting3[(BYTE)(c>>16)]++; + Counting4[ c>>24 ]++; + c = cached; cached = MEM_read32(ip); ip += 4; + Counting1[(BYTE) c ]++; + Counting2[(BYTE)(c>>8) ]++; + Counting3[(BYTE)(c>>16)]++; + Counting4[ c>>24 ]++; + c = cached; cached = MEM_read32(ip); ip += 4; + Counting1[(BYTE) c ]++; + Counting2[(BYTE)(c>>8) ]++; + Counting3[(BYTE)(c>>16)]++; + Counting4[ c>>24 ]++; + c = cached; cached = MEM_read32(ip); ip += 4; + Counting1[(BYTE) c ]++; + Counting2[(BYTE)(c>>8) ]++; + Counting3[(BYTE)(c>>16)]++; + Counting4[ c>>24 ]++; + } + ip-=4; + } + + /* finish last symbols */ + while (ipmaxSymbolValue; s--) { + Counting1[s] += Counting2[s] + Counting3[s] + Counting4[s]; + if (Counting1[s]) return ERROR(maxSymbolValue_tooSmall); + } } + + { U32 s; + if (maxSymbolValue > 255) maxSymbolValue = 255; + for (s=0; s<=maxSymbolValue; s++) { + count[s] = Counting1[s] + Counting2[s] + Counting3[s] + Counting4[s]; + if (count[s] > max) max = count[s]; + } } + + while (!count[maxSymbolValue]) maxSymbolValue--; + *maxSymbolValuePtr = maxSymbolValue; + return (size_t)max; +} + +/* HIST_countFast_wksp() : + * Same as HIST_countFast(), but using an externally provided scratch buffer. + * `workSpace` is a writable buffer which must be 4-bytes aligned, + * `workSpaceSize` must be >= HIST_WKSP_SIZE + */ +size_t HIST_countFast_wksp(unsigned* count, unsigned* maxSymbolValuePtr, + const void* source, size_t sourceSize, + void* workSpace, size_t workSpaceSize) +{ + if (sourceSize < 1500) /* heuristic threshold */ + return HIST_count_simple(count, maxSymbolValuePtr, source, sourceSize); + if ((size_t)workSpace & 3) return ERROR(GENERIC); /* must be aligned on 4-bytes boundaries */ + if (workSpaceSize < HIST_WKSP_SIZE) return ERROR(workSpace_tooSmall); + return HIST_count_parallel_wksp(count, maxSymbolValuePtr, source, sourceSize, trustInput, (U32*)workSpace); +} + +/* fast variant (unsafe : won't check if src contains values beyond count[] limit) */ +size_t HIST_countFast(unsigned* count, unsigned* maxSymbolValuePtr, + const void* source, size_t sourceSize) +{ + unsigned tmpCounters[HIST_WKSP_SIZE_U32]; + return HIST_countFast_wksp(count, maxSymbolValuePtr, source, sourceSize, tmpCounters, sizeof(tmpCounters)); +} + +/* HIST_count_wksp() : + * Same as HIST_count(), but using an externally provided scratch buffer. + * `workSpace` size must be table of >= HIST_WKSP_SIZE_U32 unsigned */ +size_t HIST_count_wksp(unsigned* count, unsigned* maxSymbolValuePtr, + const void* source, size_t sourceSize, + void* workSpace, size_t workSpaceSize) +{ + if ((size_t)workSpace & 3) return ERROR(GENERIC); /* must be aligned on 4-bytes boundaries */ + if (workSpaceSize < HIST_WKSP_SIZE) return ERROR(workSpace_tooSmall); + if (*maxSymbolValuePtr < 255) + return HIST_count_parallel_wksp(count, maxSymbolValuePtr, source, sourceSize, checkMaxSymbolValue, (U32*)workSpace); + *maxSymbolValuePtr = 255; + return HIST_countFast_wksp(count, maxSymbolValuePtr, source, sourceSize, workSpace, workSpaceSize); +} + +size_t HIST_count(unsigned* count, unsigned* maxSymbolValuePtr, + const void* src, size_t srcSize) +{ + unsigned tmpCounters[HIST_WKSP_SIZE_U32]; + return HIST_count_wksp(count, maxSymbolValuePtr, src, srcSize, tmpCounters, sizeof(tmpCounters)); +} diff --git a/module/zstd/lib/compress/hist.h b/module/zstd/lib/compress/hist.h new file mode 100644 index 000000000000..77e3ec4fb192 --- /dev/null +++ b/module/zstd/lib/compress/hist.h @@ -0,0 +1,75 @@ +/* ****************************************************************** + * hist : Histogram functions + * part of Finite State Entropy project + * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. + * + * You can contact the author at : + * - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy + * - Public forum : https://groups.google.com/forum/#!forum/lz4c + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. +****************************************************************** */ + +/* --- dependencies --- */ +#include /* size_t */ + + +/* --- simple histogram functions --- */ + +/*! HIST_count(): + * Provides the precise count of each byte within a table 'count'. + * 'count' is a table of unsigned int, of minimum size (*maxSymbolValuePtr+1). + * Updates *maxSymbolValuePtr with actual largest symbol value detected. + * @return : count of the most frequent symbol (which isn't identified). + * or an error code, which can be tested using HIST_isError(). + * note : if return == srcSize, there is only one symbol. + */ +size_t HIST_count(unsigned* count, unsigned* maxSymbolValuePtr, + const void* src, size_t srcSize); + +unsigned HIST_isError(size_t code); /**< tells if a return value is an error code */ + + +/* --- advanced histogram functions --- */ + +#define HIST_WKSP_SIZE_U32 1024 +#define HIST_WKSP_SIZE (HIST_WKSP_SIZE_U32 * sizeof(unsigned)) +/** HIST_count_wksp() : + * Same as HIST_count(), but using an externally provided scratch buffer. + * Benefit is this function will use very little stack space. + * `workSpace` is a writable buffer which must be 4-bytes aligned, + * `workSpaceSize` must be >= HIST_WKSP_SIZE + */ +size_t HIST_count_wksp(unsigned* count, unsigned* maxSymbolValuePtr, + const void* src, size_t srcSize, + void* workSpace, size_t workSpaceSize); + +/** HIST_countFast() : + * same as HIST_count(), but blindly trusts that all byte values within src are <= *maxSymbolValuePtr. + * This function is unsafe, and will segfault if any value within `src` is `> *maxSymbolValuePtr` + */ +size_t HIST_countFast(unsigned* count, unsigned* maxSymbolValuePtr, + const void* src, size_t srcSize); + +/** HIST_countFast_wksp() : + * Same as HIST_countFast(), but using an externally provided scratch buffer. + * `workSpace` is a writable buffer which must be 4-bytes aligned, + * `workSpaceSize` must be >= HIST_WKSP_SIZE + */ +size_t HIST_countFast_wksp(unsigned* count, unsigned* maxSymbolValuePtr, + const void* src, size_t srcSize, + void* workSpace, size_t workSpaceSize); + +/*! HIST_count_simple() : + * Same as HIST_countFast(), this function is unsafe, + * and will segfault if any value within `src` is `> *maxSymbolValuePtr`. + * It is also a bit slower for large inputs. + * However, it does not need any additional memory (not even on stack). + * @return : count of the most frequent symbol. + * Note this function doesn't produce any error (i.e. it must succeed). + */ +unsigned HIST_count_simple(unsigned* count, unsigned* maxSymbolValuePtr, + const void* src, size_t srcSize); diff --git a/module/zstd/lib/compress/huf_compress.c b/module/zstd/lib/compress/huf_compress.c new file mode 100644 index 000000000000..546879868a53 --- /dev/null +++ b/module/zstd/lib/compress/huf_compress.c @@ -0,0 +1,798 @@ +/* ****************************************************************** + * Huffman encoder, part of New Generation Entropy library + * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. + * + * You can contact the author at : + * - FSE+HUF source repository : https://github.com/Cyan4973/FiniteStateEntropy + * - Public forum : https://groups.google.com/forum/#!forum/lz4c + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. +****************************************************************** */ + +/* ************************************************************** +* Compiler specifics +****************************************************************/ +#ifdef _MSC_VER /* Visual Studio */ +# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */ +#endif + + +/* ************************************************************** +* Includes +****************************************************************/ +#include /* memcpy, memset */ +#include /* printf (debug) */ +#include "../common/compiler.h" +#include "../common/bitstream.h" +#include "hist.h" +#define FSE_STATIC_LINKING_ONLY /* FSE_optimalTableLog_internal */ +#include "../common/fse.h" /* header compression */ +#define HUF_STATIC_LINKING_ONLY +#include "../common/huf.h" +#include "../common/error_private.h" + + +/* ************************************************************** +* Error Management +****************************************************************/ +#define HUF_isError ERR_isError +#define HUF_STATIC_ASSERT(c) DEBUG_STATIC_ASSERT(c) /* use only *after* variable declarations */ + + +/* ************************************************************** +* Utils +****************************************************************/ +unsigned HUF_optimalTableLog(unsigned maxTableLog, size_t srcSize, unsigned maxSymbolValue) +{ + return FSE_optimalTableLog_internal(maxTableLog, srcSize, maxSymbolValue, 1); +} + + +/* ******************************************************* +* HUF : Huffman block compression +*********************************************************/ +/* HUF_compressWeights() : + * Same as FSE_compress(), but dedicated to huff0's weights compression. + * The use case needs much less stack memory. + * Note : all elements within weightTable are supposed to be <= HUF_TABLELOG_MAX. + */ +#define MAX_FSE_TABLELOG_FOR_HUFF_HEADER 6 +static size_t HUF_compressWeights (void* dst, size_t dstSize, const void* weightTable, size_t wtSize) +{ + BYTE* const ostart = (BYTE*) dst; + BYTE* op = ostart; + BYTE* const oend = ostart + dstSize; + + unsigned maxSymbolValue = HUF_TABLELOG_MAX; + U32 tableLog = MAX_FSE_TABLELOG_FOR_HUFF_HEADER; + + FSE_CTable CTable[FSE_CTABLE_SIZE_U32(MAX_FSE_TABLELOG_FOR_HUFF_HEADER, HUF_TABLELOG_MAX)]; + BYTE scratchBuffer[1< not compressible */ + } + + tableLog = FSE_optimalTableLog(tableLog, wtSize, maxSymbolValue); + CHECK_F( FSE_normalizeCount(norm, tableLog, count, wtSize, maxSymbolValue) ); + + /* Write table description header */ + { CHECK_V_F(hSize, FSE_writeNCount(op, (size_t)(oend-op), norm, maxSymbolValue, tableLog) ); + op += hSize; + } + + /* Compress */ + CHECK_F( FSE_buildCTable_wksp(CTable, norm, maxSymbolValue, tableLog, scratchBuffer, sizeof(scratchBuffer)) ); + { CHECK_V_F(cSize, FSE_compress_usingCTable(op, (size_t)(oend - op), weightTable, wtSize, CTable) ); + if (cSize == 0) return 0; /* not enough space for compressed data */ + op += cSize; + } + + return (size_t)(op-ostart); +} + + +struct HUF_CElt_s { + U16 val; + BYTE nbBits; +}; /* typedef'd to HUF_CElt within "huf.h" */ + +/*! HUF_writeCTable() : + `CTable` : Huffman tree to save, using huf representation. + @return : size of saved CTable */ +size_t HUF_writeCTable (void* dst, size_t maxDstSize, + const HUF_CElt* CTable, unsigned maxSymbolValue, unsigned huffLog) +{ + BYTE bitsToWeight[HUF_TABLELOG_MAX + 1]; /* precomputed conversion table */ + BYTE huffWeight[HUF_SYMBOLVALUE_MAX]; + BYTE* op = (BYTE*)dst; + U32 n; + + /* check conditions */ + if (maxSymbolValue > HUF_SYMBOLVALUE_MAX) return ERROR(maxSymbolValue_tooLarge); + + /* convert to weight */ + bitsToWeight[0] = 0; + for (n=1; n1) & (hSize < maxSymbolValue/2)) { /* FSE compressed */ + op[0] = (BYTE)hSize; + return hSize+1; + } } + + /* write raw values as 4-bits (max : 15) */ + if (maxSymbolValue > (256-128)) return ERROR(GENERIC); /* should not happen : likely means source cannot be compressed */ + if (((maxSymbolValue+1)/2) + 1 > maxDstSize) return ERROR(dstSize_tooSmall); /* not enough space within dst buffer */ + op[0] = (BYTE)(128 /*special case*/ + (maxSymbolValue-1)); + huffWeight[maxSymbolValue] = 0; /* to be sure it doesn't cause msan issue in final combination */ + for (n=0; n HUF_TABLELOG_MAX) return ERROR(tableLog_tooLarge); + if (nbSymbols > *maxSymbolValuePtr+1) return ERROR(maxSymbolValue_tooSmall); + + /* Prepare base value per rank */ + { U32 n, nextRankStart = 0; + for (n=1; n<=tableLog; n++) { + U32 current = nextRankStart; + nextRankStart += (rankVal[n] << (n-1)); + rankVal[n] = current; + } } + + /* fill nbBits */ + *hasZeroWeights = 0; + { U32 n; for (n=0; nn=tableLog+1 */ + U16 valPerRank[HUF_TABLELOG_MAX+2] = {0}; + { U32 n; for (n=0; n0; n--) { /* start at n=tablelog <-> w=1 */ + valPerRank[n] = min; /* get starting value within each rank */ + min += nbPerRank[n]; + min >>= 1; + } } + /* assign value within rank, symbol order */ + { U32 n; for (n=0; n maxNbBits */ + + /* there are several too large elements (at least >= 2) */ + { int totalCost = 0; + const U32 baseCost = 1 << (largestBits - maxNbBits); + int n = (int)lastNonNull; + + while (huffNode[n].nbBits > maxNbBits) { + totalCost += baseCost - (1 << (largestBits - huffNode[n].nbBits)); + huffNode[n].nbBits = (BYTE)maxNbBits; + n --; + } /* n stops at huffNode[n].nbBits <= maxNbBits */ + while (huffNode[n].nbBits == maxNbBits) n--; /* n end at index of smallest symbol using < maxNbBits */ + + /* renorm totalCost */ + totalCost >>= (largestBits - maxNbBits); /* note : totalCost is necessarily a multiple of baseCost */ + + /* repay normalized cost */ + { U32 const noSymbol = 0xF0F0F0F0; + U32 rankLast[HUF_TABLELOG_MAX+2]; + + /* Get pos of last (smallest) symbol per rank */ + memset(rankLast, 0xF0, sizeof(rankLast)); + { U32 currentNbBits = maxNbBits; + int pos; + for (pos=n ; pos >= 0; pos--) { + if (huffNode[pos].nbBits >= currentNbBits) continue; + currentNbBits = huffNode[pos].nbBits; /* < maxNbBits */ + rankLast[maxNbBits-currentNbBits] = (U32)pos; + } } + + while (totalCost > 0) { + U32 nBitsToDecrease = BIT_highbit32((U32)totalCost) + 1; + for ( ; nBitsToDecrease > 1; nBitsToDecrease--) { + U32 const highPos = rankLast[nBitsToDecrease]; + U32 const lowPos = rankLast[nBitsToDecrease-1]; + if (highPos == noSymbol) continue; + if (lowPos == noSymbol) break; + { U32 const highTotal = huffNode[highPos].count; + U32 const lowTotal = 2 * huffNode[lowPos].count; + if (highTotal <= lowTotal) break; + } } + /* only triggered when no more rank 1 symbol left => find closest one (note : there is necessarily at least one !) */ + /* HUF_MAX_TABLELOG test just to please gcc 5+; but it should not be necessary */ + while ((nBitsToDecrease<=HUF_TABLELOG_MAX) && (rankLast[nBitsToDecrease] == noSymbol)) + nBitsToDecrease ++; + totalCost -= 1 << (nBitsToDecrease-1); + if (rankLast[nBitsToDecrease-1] == noSymbol) + rankLast[nBitsToDecrease-1] = rankLast[nBitsToDecrease]; /* this rank is no longer empty */ + huffNode[rankLast[nBitsToDecrease]].nbBits ++; + if (rankLast[nBitsToDecrease] == 0) /* special case, reached largest symbol */ + rankLast[nBitsToDecrease] = noSymbol; + else { + rankLast[nBitsToDecrease]--; + if (huffNode[rankLast[nBitsToDecrease]].nbBits != maxNbBits-nBitsToDecrease) + rankLast[nBitsToDecrease] = noSymbol; /* this rank is now empty */ + } } /* while (totalCost > 0) */ + + while (totalCost < 0) { /* Sometimes, cost correction overshoot */ + if (rankLast[1] == noSymbol) { /* special case : no rank 1 symbol (using maxNbBits-1); let's create one from largest rank 0 (using maxNbBits) */ + while (huffNode[n].nbBits == maxNbBits) n--; + huffNode[n+1].nbBits--; + assert(n >= 0); + rankLast[1] = (U32)(n+1); + totalCost++; + continue; + } + huffNode[ rankLast[1] + 1 ].nbBits--; + rankLast[1]++; + totalCost ++; + } } } /* there are several too large elements (at least >= 2) */ + + return maxNbBits; +} + +typedef struct { + U32 base; + U32 current; +} rankPos; + +typedef nodeElt huffNodeTable[HUF_CTABLE_WORKSPACE_SIZE_U32]; + +#define RANK_POSITION_TABLE_SIZE 32 + +typedef struct { + huffNodeTable huffNodeTbl; + rankPos rankPosition[RANK_POSITION_TABLE_SIZE]; +} HUF_buildCTable_wksp_tables; + +static void HUF_sort(nodeElt* huffNode, const unsigned* count, U32 maxSymbolValue, rankPos* rankPosition) +{ + U32 n; + + memset(rankPosition, 0, sizeof(*rankPosition) * RANK_POSITION_TABLE_SIZE); + for (n=0; n<=maxSymbolValue; n++) { + U32 r = BIT_highbit32(count[n] + 1); + rankPosition[r].base ++; + } + for (n=30; n>0; n--) rankPosition[n-1].base += rankPosition[n].base; + for (n=0; n<32; n++) rankPosition[n].current = rankPosition[n].base; + for (n=0; n<=maxSymbolValue; n++) { + U32 const c = count[n]; + U32 const r = BIT_highbit32(c+1) + 1; + U32 pos = rankPosition[r].current++; + while ((pos > rankPosition[r].base) && (c > huffNode[pos-1].count)) { + huffNode[pos] = huffNode[pos-1]; + pos--; + } + huffNode[pos].count = c; + huffNode[pos].byte = (BYTE)n; + } +} + + +/** HUF_buildCTable_wksp() : + * Same as HUF_buildCTable(), but using externally allocated scratch buffer. + * `workSpace` must be aligned on 4-bytes boundaries, and be at least as large as sizeof(HUF_buildCTable_wksp_tables). + */ +#define STARTNODE (HUF_SYMBOLVALUE_MAX+1) + +size_t HUF_buildCTable_wksp (HUF_CElt* tree, const unsigned* count, U32 maxSymbolValue, U32 maxNbBits, void* workSpace, size_t wkspSize) +{ + HUF_buildCTable_wksp_tables* const wksp_tables = (HUF_buildCTable_wksp_tables*)workSpace; + nodeElt* const huffNode0 = wksp_tables->huffNodeTbl; + nodeElt* const huffNode = huffNode0+1; + int nonNullRank; + int lowS, lowN; + int nodeNb = STARTNODE; + int n, nodeRoot; + + /* safety checks */ + if (((size_t)workSpace & 3) != 0) return ERROR(GENERIC); /* must be aligned on 4-bytes boundaries */ + if (wkspSize < sizeof(HUF_buildCTable_wksp_tables)) + return ERROR(workSpace_tooSmall); + if (maxNbBits == 0) maxNbBits = HUF_TABLELOG_DEFAULT; + if (maxSymbolValue > HUF_SYMBOLVALUE_MAX) + return ERROR(maxSymbolValue_tooLarge); + memset(huffNode0, 0, sizeof(huffNodeTable)); + + /* sort, decreasing order */ + HUF_sort(huffNode, count, maxSymbolValue, wksp_tables->rankPosition); + + /* init for parents */ + nonNullRank = (int)maxSymbolValue; + while(huffNode[nonNullRank].count == 0) nonNullRank--; + lowS = nonNullRank; nodeRoot = nodeNb + lowS - 1; lowN = nodeNb; + huffNode[nodeNb].count = huffNode[lowS].count + huffNode[lowS-1].count; + huffNode[lowS].parent = huffNode[lowS-1].parent = (U16)nodeNb; + nodeNb++; lowS-=2; + for (n=nodeNb; n<=nodeRoot; n++) huffNode[n].count = (U32)(1U<<30); + huffNode0[0].count = (U32)(1U<<31); /* fake entry, strong barrier */ + + /* create parents */ + while (nodeNb <= nodeRoot) { + int const n1 = (huffNode[lowS].count < huffNode[lowN].count) ? lowS-- : lowN++; + int const n2 = (huffNode[lowS].count < huffNode[lowN].count) ? lowS-- : lowN++; + huffNode[nodeNb].count = huffNode[n1].count + huffNode[n2].count; + huffNode[n1].parent = huffNode[n2].parent = (U16)nodeNb; + nodeNb++; + } + + /* distribute weights (unlimited tree height) */ + huffNode[nodeRoot].nbBits = 0; + for (n=nodeRoot-1; n>=STARTNODE; n--) + huffNode[n].nbBits = huffNode[ huffNode[n].parent ].nbBits + 1; + for (n=0; n<=nonNullRank; n++) + huffNode[n].nbBits = huffNode[ huffNode[n].parent ].nbBits + 1; + + /* enforce maxTableLog */ + maxNbBits = HUF_setMaxHeight(huffNode, (U32)nonNullRank, maxNbBits); + + /* fill result into tree (val, nbBits) */ + { U16 nbPerRank[HUF_TABLELOG_MAX+1] = {0}; + U16 valPerRank[HUF_TABLELOG_MAX+1] = {0}; + int const alphabetSize = (int)(maxSymbolValue + 1); + if (maxNbBits > HUF_TABLELOG_MAX) return ERROR(GENERIC); /* check fit into table */ + for (n=0; n<=nonNullRank; n++) + nbPerRank[huffNode[n].nbBits]++; + /* determine stating value per rank */ + { U16 min = 0; + for (n=(int)maxNbBits; n>0; n--) { + valPerRank[n] = min; /* get starting value within each rank */ + min += nbPerRank[n]; + min >>= 1; + } } + for (n=0; n> 3; +} + +int HUF_validateCTable(const HUF_CElt* CTable, const unsigned* count, unsigned maxSymbolValue) { + int bad = 0; + int s; + for (s = 0; s <= (int)maxSymbolValue; ++s) { + bad |= (count[s] != 0) & (CTable[s].nbBits == 0); + } + return !bad; +} + +size_t HUF_compressBound(size_t size) { return HUF_COMPRESSBOUND(size); } + +FORCE_INLINE_TEMPLATE void +HUF_encodeSymbol(BIT_CStream_t* bitCPtr, U32 symbol, const HUF_CElt* CTable) +{ + BIT_addBitsFast(bitCPtr, CTable[symbol].val, CTable[symbol].nbBits); +} + +#define HUF_FLUSHBITS(s) BIT_flushBits(s) + +#define HUF_FLUSHBITS_1(stream) \ + if (sizeof((stream)->bitContainer)*8 < HUF_TABLELOG_MAX*2+7) HUF_FLUSHBITS(stream) + +#define HUF_FLUSHBITS_2(stream) \ + if (sizeof((stream)->bitContainer)*8 < HUF_TABLELOG_MAX*4+7) HUF_FLUSHBITS(stream) + +FORCE_INLINE_TEMPLATE size_t +HUF_compress1X_usingCTable_internal_body(void* dst, size_t dstSize, + const void* src, size_t srcSize, + const HUF_CElt* CTable) +{ + const BYTE* ip = (const BYTE*) src; + BYTE* const ostart = (BYTE*)dst; + BYTE* const oend = ostart + dstSize; + BYTE* op = ostart; + size_t n; + BIT_CStream_t bitC; + + /* init */ + if (dstSize < 8) return 0; /* not enough space to compress */ + { size_t const initErr = BIT_initCStream(&bitC, op, (size_t)(oend-op)); + if (HUF_isError(initErr)) return 0; } + + n = srcSize & ~3; /* join to mod 4 */ + switch (srcSize & 3) + { + case 3 : HUF_encodeSymbol(&bitC, ip[n+ 2], CTable); + HUF_FLUSHBITS_2(&bitC); + /* fall-through */ + case 2 : HUF_encodeSymbol(&bitC, ip[n+ 1], CTable); + HUF_FLUSHBITS_1(&bitC); + /* fall-through */ + case 1 : HUF_encodeSymbol(&bitC, ip[n+ 0], CTable); + HUF_FLUSHBITS(&bitC); + /* fall-through */ + case 0 : /* fall-through */ + default: break; + } + + for (; n>0; n-=4) { /* note : n&3==0 at this stage */ + HUF_encodeSymbol(&bitC, ip[n- 1], CTable); + HUF_FLUSHBITS_1(&bitC); + HUF_encodeSymbol(&bitC, ip[n- 2], CTable); + HUF_FLUSHBITS_2(&bitC); + HUF_encodeSymbol(&bitC, ip[n- 3], CTable); + HUF_FLUSHBITS_1(&bitC); + HUF_encodeSymbol(&bitC, ip[n- 4], CTable); + HUF_FLUSHBITS(&bitC); + } + + return BIT_closeCStream(&bitC); +} + +#if DYNAMIC_BMI2 + +static TARGET_ATTRIBUTE("bmi2") size_t +HUF_compress1X_usingCTable_internal_bmi2(void* dst, size_t dstSize, + const void* src, size_t srcSize, + const HUF_CElt* CTable) +{ + return HUF_compress1X_usingCTable_internal_body(dst, dstSize, src, srcSize, CTable); +} + +static size_t +HUF_compress1X_usingCTable_internal_default(void* dst, size_t dstSize, + const void* src, size_t srcSize, + const HUF_CElt* CTable) +{ + return HUF_compress1X_usingCTable_internal_body(dst, dstSize, src, srcSize, CTable); +} + +static size_t +HUF_compress1X_usingCTable_internal(void* dst, size_t dstSize, + const void* src, size_t srcSize, + const HUF_CElt* CTable, const int bmi2) +{ + if (bmi2) { + return HUF_compress1X_usingCTable_internal_bmi2(dst, dstSize, src, srcSize, CTable); + } + return HUF_compress1X_usingCTable_internal_default(dst, dstSize, src, srcSize, CTable); +} + +#else + +static size_t +HUF_compress1X_usingCTable_internal(void* dst, size_t dstSize, + const void* src, size_t srcSize, + const HUF_CElt* CTable, const int bmi2) +{ + (void)bmi2; + return HUF_compress1X_usingCTable_internal_body(dst, dstSize, src, srcSize, CTable); +} + +#endif + +size_t HUF_compress1X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable) +{ + return HUF_compress1X_usingCTable_internal(dst, dstSize, src, srcSize, CTable, /* bmi2 */ 0); +} + + +static size_t +HUF_compress4X_usingCTable_internal(void* dst, size_t dstSize, + const void* src, size_t srcSize, + const HUF_CElt* CTable, int bmi2) +{ + size_t const segmentSize = (srcSize+3)/4; /* first 3 segments */ + const BYTE* ip = (const BYTE*) src; + const BYTE* const iend = ip + srcSize; + BYTE* const ostart = (BYTE*) dst; + BYTE* const oend = ostart + dstSize; + BYTE* op = ostart; + + if (dstSize < 6 + 1 + 1 + 1 + 8) return 0; /* minimum space to compress successfully */ + if (srcSize < 12) return 0; /* no saving possible : too small input */ + op += 6; /* jumpTable */ + + assert(op <= oend); + { CHECK_V_F(cSize, HUF_compress1X_usingCTable_internal(op, (size_t)(oend-op), ip, segmentSize, CTable, bmi2) ); + if (cSize==0) return 0; + assert(cSize <= 65535); + MEM_writeLE16(ostart, (U16)cSize); + op += cSize; + } + + ip += segmentSize; + assert(op <= oend); + { CHECK_V_F(cSize, HUF_compress1X_usingCTable_internal(op, (size_t)(oend-op), ip, segmentSize, CTable, bmi2) ); + if (cSize==0) return 0; + assert(cSize <= 65535); + MEM_writeLE16(ostart+2, (U16)cSize); + op += cSize; + } + + ip += segmentSize; + assert(op <= oend); + { CHECK_V_F(cSize, HUF_compress1X_usingCTable_internal(op, (size_t)(oend-op), ip, segmentSize, CTable, bmi2) ); + if (cSize==0) return 0; + assert(cSize <= 65535); + MEM_writeLE16(ostart+4, (U16)cSize); + op += cSize; + } + + ip += segmentSize; + assert(op <= oend); + assert(ip <= iend); + { CHECK_V_F(cSize, HUF_compress1X_usingCTable_internal(op, (size_t)(oend-op), ip, (size_t)(iend-ip), CTable, bmi2) ); + if (cSize==0) return 0; + op += cSize; + } + + return (size_t)(op-ostart); +} + +size_t HUF_compress4X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable) +{ + return HUF_compress4X_usingCTable_internal(dst, dstSize, src, srcSize, CTable, /* bmi2 */ 0); +} + +typedef enum { HUF_singleStream, HUF_fourStreams } HUF_nbStreams_e; + +static size_t HUF_compressCTable_internal( + BYTE* const ostart, BYTE* op, BYTE* const oend, + const void* src, size_t srcSize, + HUF_nbStreams_e nbStreams, const HUF_CElt* CTable, const int bmi2) +{ + size_t const cSize = (nbStreams==HUF_singleStream) ? + HUF_compress1X_usingCTable_internal(op, (size_t)(oend - op), src, srcSize, CTable, bmi2) : + HUF_compress4X_usingCTable_internal(op, (size_t)(oend - op), src, srcSize, CTable, bmi2); + if (HUF_isError(cSize)) { return cSize; } + if (cSize==0) { return 0; } /* uncompressible */ + op += cSize; + /* check compressibility */ + assert(op >= ostart); + if ((size_t)(op-ostart) >= srcSize-1) { return 0; } + return (size_t)(op-ostart); +} + +typedef struct { + unsigned count[HUF_SYMBOLVALUE_MAX + 1]; + HUF_CElt CTable[HUF_SYMBOLVALUE_MAX + 1]; + HUF_buildCTable_wksp_tables buildCTable_wksp; +} HUF_compress_tables_t; + +/* HUF_compress_internal() : + * `workSpace` must a table of at least HUF_WORKSPACE_SIZE_U32 unsigned */ +static size_t +HUF_compress_internal (void* dst, size_t dstSize, + const void* src, size_t srcSize, + unsigned maxSymbolValue, unsigned huffLog, + HUF_nbStreams_e nbStreams, + void* workSpace, size_t wkspSize, + HUF_CElt* oldHufTable, HUF_repeat* repeat, int preferRepeat, + const int bmi2) +{ + HUF_compress_tables_t* const table = (HUF_compress_tables_t*)workSpace; + BYTE* const ostart = (BYTE*)dst; + BYTE* const oend = ostart + dstSize; + BYTE* op = ostart; + + HUF_STATIC_ASSERT(sizeof(*table) <= HUF_WORKSPACE_SIZE); + + /* checks & inits */ + if (((size_t)workSpace & 3) != 0) return ERROR(GENERIC); /* must be aligned on 4-bytes boundaries */ + if (wkspSize < HUF_WORKSPACE_SIZE) return ERROR(workSpace_tooSmall); + if (!srcSize) return 0; /* Uncompressed */ + if (!dstSize) return 0; /* cannot fit anything within dst budget */ + if (srcSize > HUF_BLOCKSIZE_MAX) return ERROR(srcSize_wrong); /* current block size limit */ + if (huffLog > HUF_TABLELOG_MAX) return ERROR(tableLog_tooLarge); + if (maxSymbolValue > HUF_SYMBOLVALUE_MAX) return ERROR(maxSymbolValue_tooLarge); + if (!maxSymbolValue) maxSymbolValue = HUF_SYMBOLVALUE_MAX; + if (!huffLog) huffLog = HUF_TABLELOG_DEFAULT; + + /* Heuristic : If old table is valid, use it for small inputs */ + if (preferRepeat && repeat && *repeat == HUF_repeat_valid) { + return HUF_compressCTable_internal(ostart, op, oend, + src, srcSize, + nbStreams, oldHufTable, bmi2); + } + + /* Scan input and build symbol stats */ + { CHECK_V_F(largest, HIST_count_wksp (table->count, &maxSymbolValue, (const BYTE*)src, srcSize, workSpace, wkspSize) ); + if (largest == srcSize) { *ostart = ((const BYTE*)src)[0]; return 1; } /* single symbol, rle */ + if (largest <= (srcSize >> 7)+4) return 0; /* heuristic : probably not compressible enough */ + } + + /* Check validity of previous table */ + if ( repeat + && *repeat == HUF_repeat_check + && !HUF_validateCTable(oldHufTable, table->count, maxSymbolValue)) { + *repeat = HUF_repeat_none; + } + /* Heuristic : use existing table for small inputs */ + if (preferRepeat && repeat && *repeat != HUF_repeat_none) { + return HUF_compressCTable_internal(ostart, op, oend, + src, srcSize, + nbStreams, oldHufTable, bmi2); + } + + /* Build Huffman Tree */ + huffLog = HUF_optimalTableLog(huffLog, srcSize, maxSymbolValue); + { size_t const maxBits = HUF_buildCTable_wksp(table->CTable, table->count, + maxSymbolValue, huffLog, + &table->buildCTable_wksp, sizeof(table->buildCTable_wksp)); + CHECK_F(maxBits); + huffLog = (U32)maxBits; + /* Zero unused symbols in CTable, so we can check it for validity */ + memset(table->CTable + (maxSymbolValue + 1), 0, + sizeof(table->CTable) - ((maxSymbolValue + 1) * sizeof(HUF_CElt))); + } + + /* Write table description header */ + { CHECK_V_F(hSize, HUF_writeCTable (op, dstSize, table->CTable, maxSymbolValue, huffLog) ); + /* Check if using previous huffman table is beneficial */ + if (repeat && *repeat != HUF_repeat_none) { + size_t const oldSize = HUF_estimateCompressedSize(oldHufTable, table->count, maxSymbolValue); + size_t const newSize = HUF_estimateCompressedSize(table->CTable, table->count, maxSymbolValue); + if (oldSize <= hSize + newSize || hSize + 12 >= srcSize) { + return HUF_compressCTable_internal(ostart, op, oend, + src, srcSize, + nbStreams, oldHufTable, bmi2); + } } + + /* Use the new huffman table */ + if (hSize + 12ul >= srcSize) { return 0; } + op += hSize; + if (repeat) { *repeat = HUF_repeat_none; } + if (oldHufTable) + memcpy(oldHufTable, table->CTable, sizeof(table->CTable)); /* Save new table */ + } + return HUF_compressCTable_internal(ostart, op, oend, + src, srcSize, + nbStreams, table->CTable, bmi2); +} + + +size_t HUF_compress1X_wksp (void* dst, size_t dstSize, + const void* src, size_t srcSize, + unsigned maxSymbolValue, unsigned huffLog, + void* workSpace, size_t wkspSize) +{ + return HUF_compress_internal(dst, dstSize, src, srcSize, + maxSymbolValue, huffLog, HUF_singleStream, + workSpace, wkspSize, + NULL, NULL, 0, 0 /*bmi2*/); +} + +size_t HUF_compress1X_repeat (void* dst, size_t dstSize, + const void* src, size_t srcSize, + unsigned maxSymbolValue, unsigned huffLog, + void* workSpace, size_t wkspSize, + HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat, int bmi2) +{ + return HUF_compress_internal(dst, dstSize, src, srcSize, + maxSymbolValue, huffLog, HUF_singleStream, + workSpace, wkspSize, hufTable, + repeat, preferRepeat, bmi2); +} + +size_t HUF_compress1X (void* dst, size_t dstSize, + const void* src, size_t srcSize, + unsigned maxSymbolValue, unsigned huffLog) +{ + unsigned workSpace[HUF_WORKSPACE_SIZE_U32]; + return HUF_compress1X_wksp(dst, dstSize, src, srcSize, maxSymbolValue, huffLog, workSpace, sizeof(workSpace)); +} + +/* HUF_compress4X_repeat(): + * compress input using 4 streams. + * provide workspace to generate compression tables */ +size_t HUF_compress4X_wksp (void* dst, size_t dstSize, + const void* src, size_t srcSize, + unsigned maxSymbolValue, unsigned huffLog, + void* workSpace, size_t wkspSize) +{ + return HUF_compress_internal(dst, dstSize, src, srcSize, + maxSymbolValue, huffLog, HUF_fourStreams, + workSpace, wkspSize, + NULL, NULL, 0, 0 /*bmi2*/); +} + +/* HUF_compress4X_repeat(): + * compress input using 4 streams. + * re-use an existing huffman compression table */ +size_t HUF_compress4X_repeat (void* dst, size_t dstSize, + const void* src, size_t srcSize, + unsigned maxSymbolValue, unsigned huffLog, + void* workSpace, size_t wkspSize, + HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat, int bmi2) +{ + return HUF_compress_internal(dst, dstSize, src, srcSize, + maxSymbolValue, huffLog, HUF_fourStreams, + workSpace, wkspSize, + hufTable, repeat, preferRepeat, bmi2); +} + +size_t HUF_compress2 (void* dst, size_t dstSize, + const void* src, size_t srcSize, + unsigned maxSymbolValue, unsigned huffLog) +{ + unsigned workSpace[HUF_WORKSPACE_SIZE_U32]; + return HUF_compress4X_wksp(dst, dstSize, src, srcSize, maxSymbolValue, huffLog, workSpace, sizeof(workSpace)); +} + +size_t HUF_compress (void* dst, size_t maxDstSize, const void* src, size_t srcSize) +{ + return HUF_compress2(dst, maxDstSize, src, srcSize, 255, HUF_TABLELOG_DEFAULT); +} diff --git a/module/zstd/lib/compress/zstd_compress.c b/module/zstd/lib/compress/zstd_compress.c new file mode 100644 index 000000000000..3f963b1cfff8 --- /dev/null +++ b/module/zstd/lib/compress/zstd_compress.c @@ -0,0 +1,4278 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +/*-************************************* +* Dependencies +***************************************/ +#include /* INT_MAX */ +#include /* memset */ +#include "../common/cpu.h" +#include "../common/mem.h" +#include "hist.h" /* HIST_countFast_wksp */ +#define FSE_STATIC_LINKING_ONLY /* FSE_encodeSymbol */ +#include "../common/fse.h" +#define HUF_STATIC_LINKING_ONLY +#include "../common/huf.h" +#include "zstd_compress_internal.h" +#include "zstd_compress_sequences.h" +#include "zstd_compress_literals.h" +#include "zstd_fast.h" +#include "zstd_double_fast.h" +#include "zstd_lazy.h" +#include "zstd_opt.h" +#include "zstd_ldm.h" +#include "zstd_compress_superblock.h" + + +/*-************************************* +* Helper functions +***************************************/ +/* ZSTD_compressBound() + * Note that the result from this function is only compatible with the "normal" + * full-block strategy. + * When there are a lot of small blocks due to frequent flush in streaming mode + * the overhead of headers can make the compressed data to be larger than the + * return value of ZSTD_compressBound(). + */ +size_t ZSTD_compressBound(size_t srcSize) { + return ZSTD_COMPRESSBOUND(srcSize); +} + + +/*-************************************* +* Context memory management +***************************************/ +struct ZSTD_CDict_s { + const void* dictContent; + size_t dictContentSize; + U32* entropyWorkspace; /* entropy workspace of HUF_WORKSPACE_SIZE bytes */ + ZSTD_cwksp workspace; + ZSTD_matchState_t matchState; + ZSTD_compressedBlockState_t cBlockState; + ZSTD_customMem customMem; + U32 dictID; + int compressionLevel; /* 0 indicates that advanced API was used to select CDict params */ +}; /* typedef'd to ZSTD_CDict within "zstd.h" */ + +ZSTD_CCtx* ZSTD_createCCtx(void) +{ + return ZSTD_createCCtx_advanced(ZSTD_defaultCMem); +} + +static void ZSTD_initCCtx(ZSTD_CCtx* cctx, ZSTD_customMem memManager) +{ + assert(cctx != NULL); + memset(cctx, 0, sizeof(*cctx)); + cctx->customMem = memManager; + cctx->bmi2 = ZSTD_cpuid_bmi2(ZSTD_cpuid()); + { size_t const err = ZSTD_CCtx_reset(cctx, ZSTD_reset_parameters); + assert(!ZSTD_isError(err)); + (void)err; + } +} + +ZSTD_CCtx* ZSTD_createCCtx_advanced(ZSTD_customMem customMem) +{ + ZSTD_STATIC_ASSERT(zcss_init==0); + ZSTD_STATIC_ASSERT(ZSTD_CONTENTSIZE_UNKNOWN==(0ULL - 1)); + if (!customMem.customAlloc ^ !customMem.customFree) return NULL; + { ZSTD_CCtx* const cctx = (ZSTD_CCtx*)ZSTD_malloc(sizeof(ZSTD_CCtx), customMem); + if (!cctx) return NULL; + ZSTD_initCCtx(cctx, customMem); + return cctx; + } +} + +ZSTD_CCtx* ZSTD_initStaticCCtx(void* workspace, size_t workspaceSize) +{ + ZSTD_cwksp ws; + ZSTD_CCtx* cctx; + if (workspaceSize <= sizeof(ZSTD_CCtx)) return NULL; /* minimum size */ + if ((size_t)workspace & 7) return NULL; /* must be 8-aligned */ + ZSTD_cwksp_init(&ws, workspace, workspaceSize); + + cctx = (ZSTD_CCtx*)ZSTD_cwksp_reserve_object(&ws, sizeof(ZSTD_CCtx)); + if (cctx == NULL) return NULL; + + memset(cctx, 0, sizeof(ZSTD_CCtx)); + ZSTD_cwksp_move(&cctx->workspace, &ws); + cctx->staticSize = workspaceSize; + + /* statically sized space. entropyWorkspace never moves (but prev/next block swap places) */ + if (!ZSTD_cwksp_check_available(&cctx->workspace, HUF_WORKSPACE_SIZE + 2 * sizeof(ZSTD_compressedBlockState_t))) return NULL; + cctx->blockState.prevCBlock = (ZSTD_compressedBlockState_t*)ZSTD_cwksp_reserve_object(&cctx->workspace, sizeof(ZSTD_compressedBlockState_t)); + cctx->blockState.nextCBlock = (ZSTD_compressedBlockState_t*)ZSTD_cwksp_reserve_object(&cctx->workspace, sizeof(ZSTD_compressedBlockState_t)); + cctx->entropyWorkspace = (U32*)ZSTD_cwksp_reserve_object(&cctx->workspace, HUF_WORKSPACE_SIZE); + cctx->bmi2 = ZSTD_cpuid_bmi2(ZSTD_cpuid()); + return cctx; +} + +/** + * Clears and frees all of the dictionaries in the CCtx. + */ +static void ZSTD_clearAllDicts(ZSTD_CCtx* cctx) +{ + ZSTD_free(cctx->localDict.dictBuffer, cctx->customMem); + ZSTD_freeCDict(cctx->localDict.cdict); + memset(&cctx->localDict, 0, sizeof(cctx->localDict)); + memset(&cctx->prefixDict, 0, sizeof(cctx->prefixDict)); + cctx->cdict = NULL; +} + +static size_t ZSTD_sizeof_localDict(ZSTD_localDict dict) +{ + size_t const bufferSize = dict.dictBuffer != NULL ? dict.dictSize : 0; + size_t const cdictSize = ZSTD_sizeof_CDict(dict.cdict); + return bufferSize + cdictSize; +} + +static void ZSTD_freeCCtxContent(ZSTD_CCtx* cctx) +{ + assert(cctx != NULL); + assert(cctx->staticSize == 0); + ZSTD_clearAllDicts(cctx); +#ifdef ZSTD_MULTITHREAD + ZSTDMT_freeCCtx(cctx->mtctx); cctx->mtctx = NULL; +#endif + ZSTD_cwksp_free(&cctx->workspace, cctx->customMem); +} + +size_t ZSTD_freeCCtx(ZSTD_CCtx* cctx) +{ + if (cctx==NULL) return 0; /* support free on NULL */ + RETURN_ERROR_IF(cctx->staticSize, memory_allocation, + "not compatible with static CCtx"); + { + int cctxInWorkspace = ZSTD_cwksp_owns_buffer(&cctx->workspace, cctx); + ZSTD_freeCCtxContent(cctx); + if (!cctxInWorkspace) { + ZSTD_free(cctx, cctx->customMem); + } + } + return 0; +} + + +static size_t ZSTD_sizeof_mtctx(const ZSTD_CCtx* cctx) +{ +#ifdef ZSTD_MULTITHREAD + return ZSTDMT_sizeof_CCtx(cctx->mtctx); +#else + (void)cctx; + return 0; +#endif +} + + +size_t ZSTD_sizeof_CCtx(const ZSTD_CCtx* cctx) +{ + if (cctx==NULL) return 0; /* support sizeof on NULL */ + /* cctx may be in the workspace */ + return (cctx->workspace.workspace == cctx ? 0 : sizeof(*cctx)) + + ZSTD_cwksp_sizeof(&cctx->workspace) + + ZSTD_sizeof_localDict(cctx->localDict) + + ZSTD_sizeof_mtctx(cctx); +} + +size_t ZSTD_sizeof_CStream(const ZSTD_CStream* zcs) +{ + return ZSTD_sizeof_CCtx(zcs); /* same object */ +} + +/* private API call, for dictBuilder only */ +const seqStore_t* ZSTD_getSeqStore(const ZSTD_CCtx* ctx) { return &(ctx->seqStore); } + +static ZSTD_CCtx_params ZSTD_makeCCtxParamsFromCParams( + ZSTD_compressionParameters cParams) +{ + ZSTD_CCtx_params cctxParams; + memset(&cctxParams, 0, sizeof(cctxParams)); + cctxParams.cParams = cParams; + cctxParams.compressionLevel = ZSTD_CLEVEL_DEFAULT; /* should not matter, as all cParams are presumed properly defined */ + assert(!ZSTD_checkCParams(cParams)); + cctxParams.fParams.contentSizeFlag = 1; + return cctxParams; +} + +static ZSTD_CCtx_params* ZSTD_createCCtxParams_advanced( + ZSTD_customMem customMem) +{ + ZSTD_CCtx_params* params; + if (!customMem.customAlloc ^ !customMem.customFree) return NULL; + params = (ZSTD_CCtx_params*)ZSTD_calloc( + sizeof(ZSTD_CCtx_params), customMem); + if (!params) { return NULL; } + params->customMem = customMem; + params->compressionLevel = ZSTD_CLEVEL_DEFAULT; + params->fParams.contentSizeFlag = 1; + return params; +} + +ZSTD_CCtx_params* ZSTD_createCCtxParams(void) +{ + return ZSTD_createCCtxParams_advanced(ZSTD_defaultCMem); +} + +size_t ZSTD_freeCCtxParams(ZSTD_CCtx_params* params) +{ + if (params == NULL) { return 0; } + ZSTD_free(params, params->customMem); + return 0; +} + +size_t ZSTD_CCtxParams_reset(ZSTD_CCtx_params* params) +{ + return ZSTD_CCtxParams_init(params, ZSTD_CLEVEL_DEFAULT); +} + +size_t ZSTD_CCtxParams_init(ZSTD_CCtx_params* cctxParams, int compressionLevel) { + RETURN_ERROR_IF(!cctxParams, GENERIC, "NULL pointer!"); + memset(cctxParams, 0, sizeof(*cctxParams)); + cctxParams->compressionLevel = compressionLevel; + cctxParams->fParams.contentSizeFlag = 1; + return 0; +} + +size_t ZSTD_CCtxParams_init_advanced(ZSTD_CCtx_params* cctxParams, ZSTD_parameters params) +{ + RETURN_ERROR_IF(!cctxParams, GENERIC, "NULL pointer!"); + FORWARD_IF_ERROR( ZSTD_checkCParams(params.cParams) , ""); + memset(cctxParams, 0, sizeof(*cctxParams)); + assert(!ZSTD_checkCParams(params.cParams)); + cctxParams->cParams = params.cParams; + cctxParams->fParams = params.fParams; + cctxParams->compressionLevel = ZSTD_CLEVEL_DEFAULT; /* should not matter, as all cParams are presumed properly defined */ + return 0; +} + +/* ZSTD_assignParamsToCCtxParams() : + * params is presumed valid at this stage */ +static ZSTD_CCtx_params ZSTD_assignParamsToCCtxParams( + const ZSTD_CCtx_params* cctxParams, const ZSTD_parameters* params) +{ + ZSTD_CCtx_params ret = *cctxParams; + assert(!ZSTD_checkCParams(params->cParams)); + ret.cParams = params->cParams; + ret.fParams = params->fParams; + ret.compressionLevel = ZSTD_CLEVEL_DEFAULT; /* should not matter, as all cParams are presumed properly defined */ + return ret; +} + +ZSTD_bounds ZSTD_cParam_getBounds(ZSTD_cParameter param) +{ + ZSTD_bounds bounds = { 0, 0, 0 }; + + switch(param) + { + case ZSTD_c_compressionLevel: + bounds.lowerBound = ZSTD_minCLevel(); + bounds.upperBound = ZSTD_maxCLevel(); + return bounds; + + case ZSTD_c_windowLog: + bounds.lowerBound = ZSTD_WINDOWLOG_MIN; + bounds.upperBound = ZSTD_WINDOWLOG_MAX; + return bounds; + + case ZSTD_c_hashLog: + bounds.lowerBound = ZSTD_HASHLOG_MIN; + bounds.upperBound = ZSTD_HASHLOG_MAX; + return bounds; + + case ZSTD_c_chainLog: + bounds.lowerBound = ZSTD_CHAINLOG_MIN; + bounds.upperBound = ZSTD_CHAINLOG_MAX; + return bounds; + + case ZSTD_c_searchLog: + bounds.lowerBound = ZSTD_SEARCHLOG_MIN; + bounds.upperBound = ZSTD_SEARCHLOG_MAX; + return bounds; + + case ZSTD_c_minMatch: + bounds.lowerBound = ZSTD_MINMATCH_MIN; + bounds.upperBound = ZSTD_MINMATCH_MAX; + return bounds; + + case ZSTD_c_targetLength: + bounds.lowerBound = ZSTD_TARGETLENGTH_MIN; + bounds.upperBound = ZSTD_TARGETLENGTH_MAX; + return bounds; + + case ZSTD_c_strategy: + bounds.lowerBound = ZSTD_STRATEGY_MIN; + bounds.upperBound = ZSTD_STRATEGY_MAX; + return bounds; + + case ZSTD_c_contentSizeFlag: + bounds.lowerBound = 0; + bounds.upperBound = 1; + return bounds; + + case ZSTD_c_checksumFlag: + bounds.lowerBound = 0; + bounds.upperBound = 1; + return bounds; + + case ZSTD_c_dictIDFlag: + bounds.lowerBound = 0; + bounds.upperBound = 1; + return bounds; + + case ZSTD_c_nbWorkers: + bounds.lowerBound = 0; +#ifdef ZSTD_MULTITHREAD + bounds.upperBound = ZSTDMT_NBWORKERS_MAX; +#else + bounds.upperBound = 0; +#endif + return bounds; + + case ZSTD_c_jobSize: + bounds.lowerBound = 0; +#ifdef ZSTD_MULTITHREAD + bounds.upperBound = ZSTDMT_JOBSIZE_MAX; +#else + bounds.upperBound = 0; +#endif + return bounds; + + case ZSTD_c_overlapLog: +#ifdef ZSTD_MULTITHREAD + bounds.lowerBound = ZSTD_OVERLAPLOG_MIN; + bounds.upperBound = ZSTD_OVERLAPLOG_MAX; +#else + bounds.lowerBound = 0; + bounds.upperBound = 0; +#endif + return bounds; + + case ZSTD_c_enableLongDistanceMatching: + bounds.lowerBound = 0; + bounds.upperBound = 1; + return bounds; + + case ZSTD_c_ldmHashLog: + bounds.lowerBound = ZSTD_LDM_HASHLOG_MIN; + bounds.upperBound = ZSTD_LDM_HASHLOG_MAX; + return bounds; + + case ZSTD_c_ldmMinMatch: + bounds.lowerBound = ZSTD_LDM_MINMATCH_MIN; + bounds.upperBound = ZSTD_LDM_MINMATCH_MAX; + return bounds; + + case ZSTD_c_ldmBucketSizeLog: + bounds.lowerBound = ZSTD_LDM_BUCKETSIZELOG_MIN; + bounds.upperBound = ZSTD_LDM_BUCKETSIZELOG_MAX; + return bounds; + + case ZSTD_c_ldmHashRateLog: + bounds.lowerBound = ZSTD_LDM_HASHRATELOG_MIN; + bounds.upperBound = ZSTD_LDM_HASHRATELOG_MAX; + return bounds; + + /* experimental parameters */ + case ZSTD_c_rsyncable: + bounds.lowerBound = 0; + bounds.upperBound = 1; + return bounds; + + case ZSTD_c_forceMaxWindow : + bounds.lowerBound = 0; + bounds.upperBound = 1; + return bounds; + + case ZSTD_c_format: + ZSTD_STATIC_ASSERT(ZSTD_f_zstd1 < ZSTD_f_zstd1_magicless); + bounds.lowerBound = ZSTD_f_zstd1; + bounds.upperBound = ZSTD_f_zstd1_magicless; /* note : how to ensure at compile time that this is the highest value enum ? */ + return bounds; + + case ZSTD_c_forceAttachDict: + ZSTD_STATIC_ASSERT(ZSTD_dictDefaultAttach < ZSTD_dictForceCopy); + bounds.lowerBound = ZSTD_dictDefaultAttach; + bounds.upperBound = ZSTD_dictForceLoad; /* note : how to ensure at compile time that this is the highest value enum ? */ + return bounds; + + case ZSTD_c_literalCompressionMode: + ZSTD_STATIC_ASSERT(ZSTD_lcm_auto < ZSTD_lcm_huffman && ZSTD_lcm_huffman < ZSTD_lcm_uncompressed); + bounds.lowerBound = ZSTD_lcm_auto; + bounds.upperBound = ZSTD_lcm_uncompressed; + return bounds; + + case ZSTD_c_targetCBlockSize: + bounds.lowerBound = ZSTD_TARGETCBLOCKSIZE_MIN; + bounds.upperBound = ZSTD_TARGETCBLOCKSIZE_MAX; + return bounds; + + case ZSTD_c_srcSizeHint: + bounds.lowerBound = ZSTD_SRCSIZEHINT_MIN; + bounds.upperBound = ZSTD_SRCSIZEHINT_MAX; + return bounds; + + default: + bounds.error = ERROR(parameter_unsupported); + return bounds; + } +} + +/* ZSTD_cParam_clampBounds: + * Clamps the value into the bounded range. + */ +static size_t ZSTD_cParam_clampBounds(ZSTD_cParameter cParam, int* value) +{ + ZSTD_bounds const bounds = ZSTD_cParam_getBounds(cParam); + if (ZSTD_isError(bounds.error)) return bounds.error; + if (*value < bounds.lowerBound) *value = bounds.lowerBound; + if (*value > bounds.upperBound) *value = bounds.upperBound; + return 0; +} + +#define BOUNDCHECK(cParam, val) { \ + RETURN_ERROR_IF(!ZSTD_cParam_withinBounds(cParam,val), \ + parameter_outOfBound, "Param out of bounds"); \ +} + + +static int ZSTD_isUpdateAuthorized(ZSTD_cParameter param) +{ + switch(param) + { + case ZSTD_c_compressionLevel: + case ZSTD_c_hashLog: + case ZSTD_c_chainLog: + case ZSTD_c_searchLog: + case ZSTD_c_minMatch: + case ZSTD_c_targetLength: + case ZSTD_c_strategy: + return 1; + + case ZSTD_c_format: + case ZSTD_c_windowLog: + case ZSTD_c_contentSizeFlag: + case ZSTD_c_checksumFlag: + case ZSTD_c_dictIDFlag: + case ZSTD_c_forceMaxWindow : + case ZSTD_c_nbWorkers: + case ZSTD_c_jobSize: + case ZSTD_c_overlapLog: + case ZSTD_c_rsyncable: + case ZSTD_c_enableLongDistanceMatching: + case ZSTD_c_ldmHashLog: + case ZSTD_c_ldmMinMatch: + case ZSTD_c_ldmBucketSizeLog: + case ZSTD_c_ldmHashRateLog: + case ZSTD_c_forceAttachDict: + case ZSTD_c_literalCompressionMode: + case ZSTD_c_targetCBlockSize: + case ZSTD_c_srcSizeHint: + default: + return 0; + } +} + +size_t ZSTD_CCtx_setParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param, int value) +{ + DEBUGLOG(4, "ZSTD_CCtx_setParameter (%i, %i)", (int)param, value); + if (cctx->streamStage != zcss_init) { + if (ZSTD_isUpdateAuthorized(param)) { + cctx->cParamsChanged = 1; + } else { + RETURN_ERROR(stage_wrong, "can only set params in ctx init stage"); + } } + + switch(param) + { + case ZSTD_c_nbWorkers: + RETURN_ERROR_IF((value!=0) && cctx->staticSize, parameter_unsupported, + "MT not compatible with static alloc"); + break; + + case ZSTD_c_compressionLevel: + case ZSTD_c_windowLog: + case ZSTD_c_hashLog: + case ZSTD_c_chainLog: + case ZSTD_c_searchLog: + case ZSTD_c_minMatch: + case ZSTD_c_targetLength: + case ZSTD_c_strategy: + case ZSTD_c_ldmHashRateLog: + case ZSTD_c_format: + case ZSTD_c_contentSizeFlag: + case ZSTD_c_checksumFlag: + case ZSTD_c_dictIDFlag: + case ZSTD_c_forceMaxWindow: + case ZSTD_c_forceAttachDict: + case ZSTD_c_literalCompressionMode: + case ZSTD_c_jobSize: + case ZSTD_c_overlapLog: + case ZSTD_c_rsyncable: + case ZSTD_c_enableLongDistanceMatching: + case ZSTD_c_ldmHashLog: + case ZSTD_c_ldmMinMatch: + case ZSTD_c_ldmBucketSizeLog: + case ZSTD_c_targetCBlockSize: + case ZSTD_c_srcSizeHint: + break; + + default: RETURN_ERROR(parameter_unsupported, "unknown parameter"); + } + return ZSTD_CCtxParams_setParameter(&cctx->requestedParams, param, value); +} + +size_t ZSTD_CCtxParams_setParameter(ZSTD_CCtx_params* CCtxParams, + ZSTD_cParameter param, int value) +{ + DEBUGLOG(4, "ZSTD_CCtxParams_setParameter (%i, %i)", (int)param, value); + switch(param) + { + case ZSTD_c_format : + BOUNDCHECK(ZSTD_c_format, value); + CCtxParams->format = (ZSTD_format_e)value; + return (size_t)CCtxParams->format; + + case ZSTD_c_compressionLevel : { + FORWARD_IF_ERROR(ZSTD_cParam_clampBounds(param, &value), ""); + if (value) { /* 0 : does not change current level */ + CCtxParams->compressionLevel = value; + } + if (CCtxParams->compressionLevel >= 0) return (size_t)CCtxParams->compressionLevel; + return 0; /* return type (size_t) cannot represent negative values */ + } + + case ZSTD_c_windowLog : + if (value!=0) /* 0 => use default */ + BOUNDCHECK(ZSTD_c_windowLog, value); + CCtxParams->cParams.windowLog = (U32)value; + return CCtxParams->cParams.windowLog; + + case ZSTD_c_hashLog : + if (value!=0) /* 0 => use default */ + BOUNDCHECK(ZSTD_c_hashLog, value); + CCtxParams->cParams.hashLog = (U32)value; + return CCtxParams->cParams.hashLog; + + case ZSTD_c_chainLog : + if (value!=0) /* 0 => use default */ + BOUNDCHECK(ZSTD_c_chainLog, value); + CCtxParams->cParams.chainLog = (U32)value; + return CCtxParams->cParams.chainLog; + + case ZSTD_c_searchLog : + if (value!=0) /* 0 => use default */ + BOUNDCHECK(ZSTD_c_searchLog, value); + CCtxParams->cParams.searchLog = (U32)value; + return (size_t)value; + + case ZSTD_c_minMatch : + if (value!=0) /* 0 => use default */ + BOUNDCHECK(ZSTD_c_minMatch, value); + CCtxParams->cParams.minMatch = value; + return CCtxParams->cParams.minMatch; + + case ZSTD_c_targetLength : + BOUNDCHECK(ZSTD_c_targetLength, value); + CCtxParams->cParams.targetLength = value; + return CCtxParams->cParams.targetLength; + + case ZSTD_c_strategy : + if (value!=0) /* 0 => use default */ + BOUNDCHECK(ZSTD_c_strategy, value); + CCtxParams->cParams.strategy = (ZSTD_strategy)value; + return (size_t)CCtxParams->cParams.strategy; + + case ZSTD_c_contentSizeFlag : + /* Content size written in frame header _when known_ (default:1) */ + DEBUGLOG(4, "set content size flag = %u", (value!=0)); + CCtxParams->fParams.contentSizeFlag = value != 0; + return CCtxParams->fParams.contentSizeFlag; + + case ZSTD_c_checksumFlag : + /* A 32-bits content checksum will be calculated and written at end of frame (default:0) */ + CCtxParams->fParams.checksumFlag = value != 0; + return CCtxParams->fParams.checksumFlag; + + case ZSTD_c_dictIDFlag : /* When applicable, dictionary's dictID is provided in frame header (default:1) */ + DEBUGLOG(4, "set dictIDFlag = %u", (value!=0)); + CCtxParams->fParams.noDictIDFlag = !value; + return !CCtxParams->fParams.noDictIDFlag; + + case ZSTD_c_forceMaxWindow : + CCtxParams->forceWindow = (value != 0); + return CCtxParams->forceWindow; + + case ZSTD_c_forceAttachDict : { + const ZSTD_dictAttachPref_e pref = (ZSTD_dictAttachPref_e)value; + BOUNDCHECK(ZSTD_c_forceAttachDict, pref); + CCtxParams->attachDictPref = pref; + return CCtxParams->attachDictPref; + } + + case ZSTD_c_literalCompressionMode : { + const ZSTD_literalCompressionMode_e lcm = (ZSTD_literalCompressionMode_e)value; + BOUNDCHECK(ZSTD_c_literalCompressionMode, lcm); + CCtxParams->literalCompressionMode = lcm; + return CCtxParams->literalCompressionMode; + } + + case ZSTD_c_nbWorkers : +#ifndef ZSTD_MULTITHREAD + RETURN_ERROR_IF(value!=0, parameter_unsupported, "not compiled with multithreading"); + return 0; +#else + FORWARD_IF_ERROR(ZSTD_cParam_clampBounds(param, &value), ""); + CCtxParams->nbWorkers = value; + return CCtxParams->nbWorkers; +#endif + + case ZSTD_c_jobSize : +#ifndef ZSTD_MULTITHREAD + RETURN_ERROR_IF(value!=0, parameter_unsupported, "not compiled with multithreading"); + return 0; +#else + /* Adjust to the minimum non-default value. */ + if (value != 0 && value < ZSTDMT_JOBSIZE_MIN) + value = ZSTDMT_JOBSIZE_MIN; + FORWARD_IF_ERROR(ZSTD_cParam_clampBounds(param, &value), ""); + assert(value >= 0); + CCtxParams->jobSize = value; + return CCtxParams->jobSize; +#endif + + case ZSTD_c_overlapLog : +#ifndef ZSTD_MULTITHREAD + RETURN_ERROR_IF(value!=0, parameter_unsupported, "not compiled with multithreading"); + return 0; +#else + FORWARD_IF_ERROR(ZSTD_cParam_clampBounds(ZSTD_c_overlapLog, &value), ""); + CCtxParams->overlapLog = value; + return CCtxParams->overlapLog; +#endif + + case ZSTD_c_rsyncable : +#ifndef ZSTD_MULTITHREAD + RETURN_ERROR_IF(value!=0, parameter_unsupported, "not compiled with multithreading"); + return 0; +#else + FORWARD_IF_ERROR(ZSTD_cParam_clampBounds(ZSTD_c_overlapLog, &value), ""); + CCtxParams->rsyncable = value; + return CCtxParams->rsyncable; +#endif + + case ZSTD_c_enableLongDistanceMatching : + CCtxParams->ldmParams.enableLdm = (value!=0); + return CCtxParams->ldmParams.enableLdm; + + case ZSTD_c_ldmHashLog : + if (value!=0) /* 0 ==> auto */ + BOUNDCHECK(ZSTD_c_ldmHashLog, value); + CCtxParams->ldmParams.hashLog = value; + return CCtxParams->ldmParams.hashLog; + + case ZSTD_c_ldmMinMatch : + if (value!=0) /* 0 ==> default */ + BOUNDCHECK(ZSTD_c_ldmMinMatch, value); + CCtxParams->ldmParams.minMatchLength = value; + return CCtxParams->ldmParams.minMatchLength; + + case ZSTD_c_ldmBucketSizeLog : + if (value!=0) /* 0 ==> default */ + BOUNDCHECK(ZSTD_c_ldmBucketSizeLog, value); + CCtxParams->ldmParams.bucketSizeLog = value; + return CCtxParams->ldmParams.bucketSizeLog; + + case ZSTD_c_ldmHashRateLog : + RETURN_ERROR_IF(value > ZSTD_WINDOWLOG_MAX - ZSTD_HASHLOG_MIN, + parameter_outOfBound, "Param out of bounds!"); + CCtxParams->ldmParams.hashRateLog = value; + return CCtxParams->ldmParams.hashRateLog; + + case ZSTD_c_targetCBlockSize : + if (value!=0) /* 0 ==> default */ + BOUNDCHECK(ZSTD_c_targetCBlockSize, value); + CCtxParams->targetCBlockSize = value; + return CCtxParams->targetCBlockSize; + + case ZSTD_c_srcSizeHint : + if (value!=0) /* 0 ==> default */ + BOUNDCHECK(ZSTD_c_srcSizeHint, value); + CCtxParams->srcSizeHint = value; + return CCtxParams->srcSizeHint; + + default: RETURN_ERROR(parameter_unsupported, "unknown parameter"); + } +} + +size_t ZSTD_CCtx_getParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param, int* value) +{ + return ZSTD_CCtxParams_getParameter(&cctx->requestedParams, param, value); +} + +size_t ZSTD_CCtxParams_getParameter( + ZSTD_CCtx_params* CCtxParams, ZSTD_cParameter param, int* value) +{ + switch(param) + { + case ZSTD_c_format : + *value = CCtxParams->format; + break; + case ZSTD_c_compressionLevel : + *value = CCtxParams->compressionLevel; + break; + case ZSTD_c_windowLog : + *value = (int)CCtxParams->cParams.windowLog; + break; + case ZSTD_c_hashLog : + *value = (int)CCtxParams->cParams.hashLog; + break; + case ZSTD_c_chainLog : + *value = (int)CCtxParams->cParams.chainLog; + break; + case ZSTD_c_searchLog : + *value = CCtxParams->cParams.searchLog; + break; + case ZSTD_c_minMatch : + *value = CCtxParams->cParams.minMatch; + break; + case ZSTD_c_targetLength : + *value = CCtxParams->cParams.targetLength; + break; + case ZSTD_c_strategy : + *value = (unsigned)CCtxParams->cParams.strategy; + break; + case ZSTD_c_contentSizeFlag : + *value = CCtxParams->fParams.contentSizeFlag; + break; + case ZSTD_c_checksumFlag : + *value = CCtxParams->fParams.checksumFlag; + break; + case ZSTD_c_dictIDFlag : + *value = !CCtxParams->fParams.noDictIDFlag; + break; + case ZSTD_c_forceMaxWindow : + *value = CCtxParams->forceWindow; + break; + case ZSTD_c_forceAttachDict : + *value = CCtxParams->attachDictPref; + break; + case ZSTD_c_literalCompressionMode : + *value = CCtxParams->literalCompressionMode; + break; + case ZSTD_c_nbWorkers : +#ifndef ZSTD_MULTITHREAD + assert(CCtxParams->nbWorkers == 0); +#endif + *value = CCtxParams->nbWorkers; + break; + case ZSTD_c_jobSize : +#ifndef ZSTD_MULTITHREAD + RETURN_ERROR(parameter_unsupported, "not compiled with multithreading"); +#else + assert(CCtxParams->jobSize <= INT_MAX); + *value = (int)CCtxParams->jobSize; + break; +#endif + case ZSTD_c_overlapLog : +#ifndef ZSTD_MULTITHREAD + RETURN_ERROR(parameter_unsupported, "not compiled with multithreading"); +#else + *value = CCtxParams->overlapLog; + break; +#endif + case ZSTD_c_rsyncable : +#ifndef ZSTD_MULTITHREAD + RETURN_ERROR(parameter_unsupported, "not compiled with multithreading"); +#else + *value = CCtxParams->rsyncable; + break; +#endif + case ZSTD_c_enableLongDistanceMatching : + *value = CCtxParams->ldmParams.enableLdm; + break; + case ZSTD_c_ldmHashLog : + *value = CCtxParams->ldmParams.hashLog; + break; + case ZSTD_c_ldmMinMatch : + *value = CCtxParams->ldmParams.minMatchLength; + break; + case ZSTD_c_ldmBucketSizeLog : + *value = CCtxParams->ldmParams.bucketSizeLog; + break; + case ZSTD_c_ldmHashRateLog : + *value = CCtxParams->ldmParams.hashRateLog; + break; + case ZSTD_c_targetCBlockSize : + *value = (int)CCtxParams->targetCBlockSize; + break; + case ZSTD_c_srcSizeHint : + *value = (int)CCtxParams->srcSizeHint; + break; + default: RETURN_ERROR(parameter_unsupported, "unknown parameter"); + } + return 0; +} + +/** ZSTD_CCtx_setParametersUsingCCtxParams() : + * just applies `params` into `cctx` + * no action is performed, parameters are merely stored. + * If ZSTDMT is enabled, parameters are pushed to cctx->mtctx. + * This is possible even if a compression is ongoing. + * In which case, new parameters will be applied on the fly, starting with next compression job. + */ +size_t ZSTD_CCtx_setParametersUsingCCtxParams( + ZSTD_CCtx* cctx, const ZSTD_CCtx_params* params) +{ + DEBUGLOG(4, "ZSTD_CCtx_setParametersUsingCCtxParams"); + RETURN_ERROR_IF(cctx->streamStage != zcss_init, stage_wrong, + "The context is in the wrong stage!"); + RETURN_ERROR_IF(cctx->cdict, stage_wrong, + "Can't override parameters with cdict attached (some must " + "be inherited from the cdict)."); + + cctx->requestedParams = *params; + return 0; +} + +ZSTDLIB_API size_t ZSTD_CCtx_setPledgedSrcSize(ZSTD_CCtx* cctx, unsigned long long pledgedSrcSize) +{ + DEBUGLOG(4, "ZSTD_CCtx_setPledgedSrcSize to %u bytes", (U32)pledgedSrcSize); + RETURN_ERROR_IF(cctx->streamStage != zcss_init, stage_wrong, + "Can't set pledgedSrcSize when not in init stage."); + cctx->pledgedSrcSizePlusOne = pledgedSrcSize+1; + return 0; +} + +/** + * Initializes the local dict using the requested parameters. + * NOTE: This does not use the pledged src size, because it may be used for more + * than one compression. + */ +static size_t ZSTD_initLocalDict(ZSTD_CCtx* cctx) +{ + ZSTD_localDict* const dl = &cctx->localDict; + ZSTD_compressionParameters const cParams = ZSTD_getCParamsFromCCtxParams( + &cctx->requestedParams, ZSTD_CONTENTSIZE_UNKNOWN, dl->dictSize); + if (dl->dict == NULL) { + /* No local dictionary. */ + assert(dl->dictBuffer == NULL); + assert(dl->cdict == NULL); + assert(dl->dictSize == 0); + return 0; + } + if (dl->cdict != NULL) { + assert(cctx->cdict == dl->cdict); + /* Local dictionary already initialized. */ + return 0; + } + assert(dl->dictSize > 0); + assert(cctx->cdict == NULL); + assert(cctx->prefixDict.dict == NULL); + + dl->cdict = ZSTD_createCDict_advanced( + dl->dict, + dl->dictSize, + ZSTD_dlm_byRef, + dl->dictContentType, + cParams, + cctx->customMem); + RETURN_ERROR_IF(!dl->cdict, memory_allocation, "ZSTD_createCDict_advanced failed"); + cctx->cdict = dl->cdict; + return 0; +} + +size_t ZSTD_CCtx_loadDictionary_advanced( + ZSTD_CCtx* cctx, const void* dict, size_t dictSize, + ZSTD_dictLoadMethod_e dictLoadMethod, ZSTD_dictContentType_e dictContentType) +{ + RETURN_ERROR_IF(cctx->streamStage != zcss_init, stage_wrong, + "Can't load a dictionary when ctx is not in init stage."); + RETURN_ERROR_IF(cctx->staticSize, memory_allocation, + "no malloc for static CCtx"); + DEBUGLOG(4, "ZSTD_CCtx_loadDictionary_advanced (size: %u)", (U32)dictSize); + ZSTD_clearAllDicts(cctx); /* in case one already exists */ + if (dict == NULL || dictSize == 0) /* no dictionary mode */ + return 0; + if (dictLoadMethod == ZSTD_dlm_byRef) { + cctx->localDict.dict = dict; + } else { + void* dictBuffer = ZSTD_malloc(dictSize, cctx->customMem); + RETURN_ERROR_IF(!dictBuffer, memory_allocation, "NULL pointer!"); + memcpy(dictBuffer, dict, dictSize); + cctx->localDict.dictBuffer = dictBuffer; + cctx->localDict.dict = dictBuffer; + } + cctx->localDict.dictSize = dictSize; + cctx->localDict.dictContentType = dictContentType; + return 0; +} + +ZSTDLIB_API size_t ZSTD_CCtx_loadDictionary_byReference( + ZSTD_CCtx* cctx, const void* dict, size_t dictSize) +{ + return ZSTD_CCtx_loadDictionary_advanced( + cctx, dict, dictSize, ZSTD_dlm_byRef, ZSTD_dct_auto); +} + +ZSTDLIB_API size_t ZSTD_CCtx_loadDictionary(ZSTD_CCtx* cctx, const void* dict, size_t dictSize) +{ + return ZSTD_CCtx_loadDictionary_advanced( + cctx, dict, dictSize, ZSTD_dlm_byCopy, ZSTD_dct_auto); +} + + +size_t ZSTD_CCtx_refCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict) +{ + RETURN_ERROR_IF(cctx->streamStage != zcss_init, stage_wrong, + "Can't ref a dict when ctx not in init stage."); + /* Free the existing local cdict (if any) to save memory. */ + ZSTD_clearAllDicts(cctx); + cctx->cdict = cdict; + return 0; +} + +size_t ZSTD_CCtx_refPrefix(ZSTD_CCtx* cctx, const void* prefix, size_t prefixSize) +{ + return ZSTD_CCtx_refPrefix_advanced(cctx, prefix, prefixSize, ZSTD_dct_rawContent); +} + +size_t ZSTD_CCtx_refPrefix_advanced( + ZSTD_CCtx* cctx, const void* prefix, size_t prefixSize, ZSTD_dictContentType_e dictContentType) +{ + RETURN_ERROR_IF(cctx->streamStage != zcss_init, stage_wrong, + "Can't ref a prefix when ctx not in init stage."); + ZSTD_clearAllDicts(cctx); + if (prefix != NULL && prefixSize > 0) { + cctx->prefixDict.dict = prefix; + cctx->prefixDict.dictSize = prefixSize; + cctx->prefixDict.dictContentType = dictContentType; + } + return 0; +} + +/*! ZSTD_CCtx_reset() : + * Also dumps dictionary */ +size_t ZSTD_CCtx_reset(ZSTD_CCtx* cctx, ZSTD_ResetDirective reset) +{ + if ( (reset == ZSTD_reset_session_only) + || (reset == ZSTD_reset_session_and_parameters) ) { + cctx->streamStage = zcss_init; + cctx->pledgedSrcSizePlusOne = 0; + } + if ( (reset == ZSTD_reset_parameters) + || (reset == ZSTD_reset_session_and_parameters) ) { + RETURN_ERROR_IF(cctx->streamStage != zcss_init, stage_wrong, + "Can't reset parameters only when not in init stage."); + ZSTD_clearAllDicts(cctx); + return ZSTD_CCtxParams_reset(&cctx->requestedParams); + } + return 0; +} + + +/** ZSTD_checkCParams() : + control CParam values remain within authorized range. + @return : 0, or an error code if one value is beyond authorized range */ +size_t ZSTD_checkCParams(ZSTD_compressionParameters cParams) +{ + BOUNDCHECK(ZSTD_c_windowLog, (int)cParams.windowLog); + BOUNDCHECK(ZSTD_c_chainLog, (int)cParams.chainLog); + BOUNDCHECK(ZSTD_c_hashLog, (int)cParams.hashLog); + BOUNDCHECK(ZSTD_c_searchLog, (int)cParams.searchLog); + BOUNDCHECK(ZSTD_c_minMatch, (int)cParams.minMatch); + BOUNDCHECK(ZSTD_c_targetLength,(int)cParams.targetLength); + BOUNDCHECK(ZSTD_c_strategy, cParams.strategy); + return 0; +} + +/** ZSTD_clampCParams() : + * make CParam values within valid range. + * @return : valid CParams */ +static ZSTD_compressionParameters +ZSTD_clampCParams(ZSTD_compressionParameters cParams) +{ +# define CLAMP_TYPE(cParam, val, type) { \ + ZSTD_bounds const bounds = ZSTD_cParam_getBounds(cParam); \ + if ((int)valbounds.upperBound) val=(type)bounds.upperBound; \ + } +# define CLAMP(cParam, val) CLAMP_TYPE(cParam, val, unsigned) + CLAMP(ZSTD_c_windowLog, cParams.windowLog); + CLAMP(ZSTD_c_chainLog, cParams.chainLog); + CLAMP(ZSTD_c_hashLog, cParams.hashLog); + CLAMP(ZSTD_c_searchLog, cParams.searchLog); + CLAMP(ZSTD_c_minMatch, cParams.minMatch); + CLAMP(ZSTD_c_targetLength,cParams.targetLength); + CLAMP_TYPE(ZSTD_c_strategy,cParams.strategy, ZSTD_strategy); + return cParams; +} + +/** ZSTD_cycleLog() : + * condition for correct operation : hashLog > 1 */ +U32 ZSTD_cycleLog(U32 hashLog, ZSTD_strategy strat) +{ + U32 const btScale = ((U32)strat >= (U32)ZSTD_btlazy2); + return hashLog - btScale; +} + +/** ZSTD_adjustCParams_internal() : + * optimize `cPar` for a specified input (`srcSize` and `dictSize`). + * mostly downsize to reduce memory consumption and initialization latency. + * `srcSize` can be ZSTD_CONTENTSIZE_UNKNOWN when not known. + * note : `srcSize==0` means 0! + * condition : cPar is presumed validated (can be checked using ZSTD_checkCParams()). */ +static ZSTD_compressionParameters +ZSTD_adjustCParams_internal(ZSTD_compressionParameters cPar, + unsigned long long srcSize, + size_t dictSize) +{ + static const U64 minSrcSize = 513; /* (1<<9) + 1 */ + static const U64 maxWindowResize = 1ULL << (ZSTD_WINDOWLOG_MAX-1); + assert(ZSTD_checkCParams(cPar)==0); + + if (dictSize && srcSize == ZSTD_CONTENTSIZE_UNKNOWN) + srcSize = minSrcSize; + + /* resize windowLog if input is small enough, to use less memory */ + if ( (srcSize < maxWindowResize) + && (dictSize < maxWindowResize) ) { + U32 const tSize = (U32)(srcSize + dictSize); + static U32 const hashSizeMin = 1 << ZSTD_HASHLOG_MIN; + U32 const srcLog = (tSize < hashSizeMin) ? ZSTD_HASHLOG_MIN : + ZSTD_highbit32(tSize-1) + 1; + if (cPar.windowLog > srcLog) cPar.windowLog = srcLog; + } + if (cPar.hashLog > cPar.windowLog+1) cPar.hashLog = cPar.windowLog+1; + { U32 const cycleLog = ZSTD_cycleLog(cPar.chainLog, cPar.strategy); + if (cycleLog > cPar.windowLog) + cPar.chainLog -= (cycleLog - cPar.windowLog); + } + + if (cPar.windowLog < ZSTD_WINDOWLOG_ABSOLUTEMIN) + cPar.windowLog = ZSTD_WINDOWLOG_ABSOLUTEMIN; /* minimum wlog required for valid frame header */ + + return cPar; +} + +ZSTD_compressionParameters +ZSTD_adjustCParams(ZSTD_compressionParameters cPar, + unsigned long long srcSize, + size_t dictSize) +{ + cPar = ZSTD_clampCParams(cPar); /* resulting cPar is necessarily valid (all parameters within range) */ + if (srcSize == 0) srcSize = ZSTD_CONTENTSIZE_UNKNOWN; + return ZSTD_adjustCParams_internal(cPar, srcSize, dictSize); +} + +static ZSTD_compressionParameters ZSTD_getCParams_internal(int compressionLevel, unsigned long long srcSizeHint, size_t dictSize); +static ZSTD_parameters ZSTD_getParams_internal(int compressionLevel, unsigned long long srcSizeHint, size_t dictSize); + +ZSTD_compressionParameters ZSTD_getCParamsFromCCtxParams( + const ZSTD_CCtx_params* CCtxParams, U64 srcSizeHint, size_t dictSize) +{ + ZSTD_compressionParameters cParams; + if (srcSizeHint == ZSTD_CONTENTSIZE_UNKNOWN && CCtxParams->srcSizeHint > 0) { + srcSizeHint = CCtxParams->srcSizeHint; + } + cParams = ZSTD_getCParams_internal(CCtxParams->compressionLevel, srcSizeHint, dictSize); + if (CCtxParams->ldmParams.enableLdm) cParams.windowLog = ZSTD_LDM_DEFAULT_WINDOW_LOG; + if (CCtxParams->cParams.windowLog) cParams.windowLog = CCtxParams->cParams.windowLog; + if (CCtxParams->cParams.hashLog) cParams.hashLog = CCtxParams->cParams.hashLog; + if (CCtxParams->cParams.chainLog) cParams.chainLog = CCtxParams->cParams.chainLog; + if (CCtxParams->cParams.searchLog) cParams.searchLog = CCtxParams->cParams.searchLog; + if (CCtxParams->cParams.minMatch) cParams.minMatch = CCtxParams->cParams.minMatch; + if (CCtxParams->cParams.targetLength) cParams.targetLength = CCtxParams->cParams.targetLength; + if (CCtxParams->cParams.strategy) cParams.strategy = CCtxParams->cParams.strategy; + assert(!ZSTD_checkCParams(cParams)); + /* srcSizeHint == 0 means 0 */ + return ZSTD_adjustCParams_internal(cParams, srcSizeHint, dictSize); +} + +static size_t +ZSTD_sizeof_matchState(const ZSTD_compressionParameters* const cParams, + const U32 forCCtx) +{ + size_t const chainSize = (cParams->strategy == ZSTD_fast) ? 0 : ((size_t)1 << cParams->chainLog); + size_t const hSize = ((size_t)1) << cParams->hashLog; + U32 const hashLog3 = (forCCtx && cParams->minMatch==3) ? MIN(ZSTD_HASHLOG3_MAX, cParams->windowLog) : 0; + size_t const h3Size = hashLog3 ? ((size_t)1) << hashLog3 : 0; + /* We don't use ZSTD_cwksp_alloc_size() here because the tables aren't + * surrounded by redzones in ASAN. */ + size_t const tableSpace = chainSize * sizeof(U32) + + hSize * sizeof(U32) + + h3Size * sizeof(U32); + size_t const optPotentialSpace = + ZSTD_cwksp_alloc_size((MaxML+1) * sizeof(U32)) + + ZSTD_cwksp_alloc_size((MaxLL+1) * sizeof(U32)) + + ZSTD_cwksp_alloc_size((MaxOff+1) * sizeof(U32)) + + ZSTD_cwksp_alloc_size((1<strategy >= ZSTD_btopt)) + ? optPotentialSpace + : 0; + DEBUGLOG(4, "chainSize: %u - hSize: %u - h3Size: %u", + (U32)chainSize, (U32)hSize, (U32)h3Size); + return tableSpace + optSpace; +} + +size_t ZSTD_estimateCCtxSize_usingCCtxParams(const ZSTD_CCtx_params* params) +{ + RETURN_ERROR_IF(params->nbWorkers > 0, GENERIC, "Estimate CCtx size is supported for single-threaded compression only."); + { ZSTD_compressionParameters const cParams = + ZSTD_getCParamsFromCCtxParams(params, ZSTD_CONTENTSIZE_UNKNOWN, 0); + size_t const blockSize = MIN(ZSTD_BLOCKSIZE_MAX, (size_t)1 << cParams.windowLog); + U32 const divider = (cParams.minMatch==3) ? 3 : 4; + size_t const maxNbSeq = blockSize / divider; + size_t const tokenSpace = ZSTD_cwksp_alloc_size(WILDCOPY_OVERLENGTH + blockSize) + + ZSTD_cwksp_alloc_size(maxNbSeq * sizeof(seqDef)) + + 3 * ZSTD_cwksp_alloc_size(maxNbSeq * sizeof(BYTE)); + size_t const entropySpace = ZSTD_cwksp_alloc_size(HUF_WORKSPACE_SIZE); + size_t const blockStateSpace = 2 * ZSTD_cwksp_alloc_size(sizeof(ZSTD_compressedBlockState_t)); + size_t const matchStateSize = ZSTD_sizeof_matchState(&cParams, /* forCCtx */ 1); + + size_t const ldmSpace = ZSTD_ldm_getTableSize(params->ldmParams); + size_t const ldmSeqSpace = ZSTD_cwksp_alloc_size(ZSTD_ldm_getMaxNbSeq(params->ldmParams, blockSize) * sizeof(rawSeq)); + + /* estimateCCtxSize is for one-shot compression. So no buffers should + * be needed. However, we still allocate two 0-sized buffers, which can + * take space under ASAN. */ + size_t const bufferSpace = ZSTD_cwksp_alloc_size(0) + + ZSTD_cwksp_alloc_size(0); + + size_t const cctxSpace = ZSTD_cwksp_alloc_size(sizeof(ZSTD_CCtx)); + + size_t const neededSpace = + cctxSpace + + entropySpace + + blockStateSpace + + ldmSpace + + ldmSeqSpace + + matchStateSize + + tokenSpace + + bufferSpace; + + DEBUGLOG(5, "estimate workspace : %u", (U32)neededSpace); + return neededSpace; + } +} + +size_t ZSTD_estimateCCtxSize_usingCParams(ZSTD_compressionParameters cParams) +{ + ZSTD_CCtx_params const params = ZSTD_makeCCtxParamsFromCParams(cParams); + return ZSTD_estimateCCtxSize_usingCCtxParams(¶ms); +} + +static size_t ZSTD_estimateCCtxSize_internal(int compressionLevel) +{ + ZSTD_compressionParameters const cParams = ZSTD_getCParams_internal(compressionLevel, ZSTD_CONTENTSIZE_UNKNOWN, 0); + return ZSTD_estimateCCtxSize_usingCParams(cParams); +} + +size_t ZSTD_estimateCCtxSize(int compressionLevel) +{ + int level; + size_t memBudget = 0; + for (level=MIN(compressionLevel, 1); level<=compressionLevel; level++) { + size_t const newMB = ZSTD_estimateCCtxSize_internal(level); + if (newMB > memBudget) memBudget = newMB; + } + return memBudget; +} + +size_t ZSTD_estimateCStreamSize_usingCCtxParams(const ZSTD_CCtx_params* params) +{ + RETURN_ERROR_IF(params->nbWorkers > 0, GENERIC, "Estimate CCtx size is supported for single-threaded compression only."); + { ZSTD_compressionParameters const cParams = + ZSTD_getCParamsFromCCtxParams(params, ZSTD_CONTENTSIZE_UNKNOWN, 0); + size_t const CCtxSize = ZSTD_estimateCCtxSize_usingCCtxParams(params); + size_t const blockSize = MIN(ZSTD_BLOCKSIZE_MAX, (size_t)1 << cParams.windowLog); + size_t const inBuffSize = ((size_t)1 << cParams.windowLog) + blockSize; + size_t const outBuffSize = ZSTD_compressBound(blockSize) + 1; + size_t const streamingSize = ZSTD_cwksp_alloc_size(inBuffSize) + + ZSTD_cwksp_alloc_size(outBuffSize); + + return CCtxSize + streamingSize; + } +} + +size_t ZSTD_estimateCStreamSize_usingCParams(ZSTD_compressionParameters cParams) +{ + ZSTD_CCtx_params const params = ZSTD_makeCCtxParamsFromCParams(cParams); + return ZSTD_estimateCStreamSize_usingCCtxParams(¶ms); +} + +static size_t ZSTD_estimateCStreamSize_internal(int compressionLevel) +{ + ZSTD_compressionParameters const cParams = ZSTD_getCParams_internal(compressionLevel, ZSTD_CONTENTSIZE_UNKNOWN, 0); + return ZSTD_estimateCStreamSize_usingCParams(cParams); +} + +size_t ZSTD_estimateCStreamSize(int compressionLevel) +{ + int level; + size_t memBudget = 0; + for (level=MIN(compressionLevel, 1); level<=compressionLevel; level++) { + size_t const newMB = ZSTD_estimateCStreamSize_internal(level); + if (newMB > memBudget) memBudget = newMB; + } + return memBudget; +} + +/* ZSTD_getFrameProgression(): + * tells how much data has been consumed (input) and produced (output) for current frame. + * able to count progression inside worker threads (non-blocking mode). + */ +ZSTD_frameProgression ZSTD_getFrameProgression(const ZSTD_CCtx* cctx) +{ +#ifdef ZSTD_MULTITHREAD + if (cctx->appliedParams.nbWorkers > 0) { + return ZSTDMT_getFrameProgression(cctx->mtctx); + } +#endif + { ZSTD_frameProgression fp; + size_t const buffered = (cctx->inBuff == NULL) ? 0 : + cctx->inBuffPos - cctx->inToCompress; + if (buffered) assert(cctx->inBuffPos >= cctx->inToCompress); + assert(buffered <= ZSTD_BLOCKSIZE_MAX); + fp.ingested = cctx->consumedSrcSize + buffered; + fp.consumed = cctx->consumedSrcSize; + fp.produced = cctx->producedCSize; + fp.flushed = cctx->producedCSize; /* simplified; some data might still be left within streaming output buffer */ + fp.currentJobID = 0; + fp.nbActiveWorkers = 0; + return fp; +} } + +/*! ZSTD_toFlushNow() + * Only useful for multithreading scenarios currently (nbWorkers >= 1). + */ +size_t ZSTD_toFlushNow(ZSTD_CCtx* cctx) +{ +#ifdef ZSTD_MULTITHREAD + if (cctx->appliedParams.nbWorkers > 0) { + return ZSTDMT_toFlushNow(cctx->mtctx); + } +#endif + (void)cctx; + return 0; /* over-simplification; could also check if context is currently running in streaming mode, and in which case, report how many bytes are left to be flushed within output buffer */ +} + +static void ZSTD_assertEqualCParams(ZSTD_compressionParameters cParams1, + ZSTD_compressionParameters cParams2) +{ + (void)cParams1; + (void)cParams2; + assert(cParams1.windowLog == cParams2.windowLog); + assert(cParams1.chainLog == cParams2.chainLog); + assert(cParams1.hashLog == cParams2.hashLog); + assert(cParams1.searchLog == cParams2.searchLog); + assert(cParams1.minMatch == cParams2.minMatch); + assert(cParams1.targetLength == cParams2.targetLength); + assert(cParams1.strategy == cParams2.strategy); +} + +void ZSTD_reset_compressedBlockState(ZSTD_compressedBlockState_t* bs) +{ + int i; + for (i = 0; i < ZSTD_REP_NUM; ++i) + bs->rep[i] = repStartValue[i]; + bs->entropy.huf.repeatMode = HUF_repeat_none; + bs->entropy.fse.offcode_repeatMode = FSE_repeat_none; + bs->entropy.fse.matchlength_repeatMode = FSE_repeat_none; + bs->entropy.fse.litlength_repeatMode = FSE_repeat_none; +} + +/*! ZSTD_invalidateMatchState() + * Invalidate all the matches in the match finder tables. + * Requires nextSrc and base to be set (can be NULL). + */ +static void ZSTD_invalidateMatchState(ZSTD_matchState_t* ms) +{ + ZSTD_window_clear(&ms->window); + + ms->nextToUpdate = ms->window.dictLimit; + ms->loadedDictEnd = 0; + ms->opt.litLengthSum = 0; /* force reset of btopt stats */ + ms->dictMatchState = NULL; +} + +/** + * Indicates whether this compression proceeds directly from user-provided + * source buffer to user-provided destination buffer (ZSTDb_not_buffered), or + * whether the context needs to buffer the input/output (ZSTDb_buffered). + */ +typedef enum { + ZSTDb_not_buffered, + ZSTDb_buffered +} ZSTD_buffered_policy_e; + +/** + * Controls, for this matchState reset, whether the tables need to be cleared / + * prepared for the coming compression (ZSTDcrp_makeClean), or whether the + * tables can be left unclean (ZSTDcrp_leaveDirty), because we know that a + * subsequent operation will overwrite the table space anyways (e.g., copying + * the matchState contents in from a CDict). + */ +typedef enum { + ZSTDcrp_makeClean, + ZSTDcrp_leaveDirty +} ZSTD_compResetPolicy_e; + +/** + * Controls, for this matchState reset, whether indexing can continue where it + * left off (ZSTDirp_continue), or whether it needs to be restarted from zero + * (ZSTDirp_reset). + */ +typedef enum { + ZSTDirp_continue, + ZSTDirp_reset +} ZSTD_indexResetPolicy_e; + +typedef enum { + ZSTD_resetTarget_CDict, + ZSTD_resetTarget_CCtx +} ZSTD_resetTarget_e; + +static size_t +ZSTD_reset_matchState(ZSTD_matchState_t* ms, + ZSTD_cwksp* ws, + const ZSTD_compressionParameters* cParams, + const ZSTD_compResetPolicy_e crp, + const ZSTD_indexResetPolicy_e forceResetIndex, + const ZSTD_resetTarget_e forWho) +{ + size_t const chainSize = (cParams->strategy == ZSTD_fast) ? 0 : ((size_t)1 << cParams->chainLog); + size_t const hSize = ((size_t)1) << cParams->hashLog; + U32 const hashLog3 = ((forWho == ZSTD_resetTarget_CCtx) && cParams->minMatch==3) ? MIN(ZSTD_HASHLOG3_MAX, cParams->windowLog) : 0; + size_t const h3Size = hashLog3 ? ((size_t)1) << hashLog3 : 0; + + DEBUGLOG(4, "reset indices : %u", forceResetIndex == ZSTDirp_reset); + if (forceResetIndex == ZSTDirp_reset) { + ZSTD_window_init(&ms->window); + ZSTD_cwksp_mark_tables_dirty(ws); + } + + ms->hashLog3 = hashLog3; + + ZSTD_invalidateMatchState(ms); + + assert(!ZSTD_cwksp_reserve_failed(ws)); /* check that allocation hasn't already failed */ + + ZSTD_cwksp_clear_tables(ws); + + DEBUGLOG(5, "reserving table space"); + /* table Space */ + ms->hashTable = (U32*)ZSTD_cwksp_reserve_table(ws, hSize * sizeof(U32)); + ms->chainTable = (U32*)ZSTD_cwksp_reserve_table(ws, chainSize * sizeof(U32)); + ms->hashTable3 = (U32*)ZSTD_cwksp_reserve_table(ws, h3Size * sizeof(U32)); + RETURN_ERROR_IF(ZSTD_cwksp_reserve_failed(ws), memory_allocation, + "failed a workspace allocation in ZSTD_reset_matchState"); + + DEBUGLOG(4, "reset table : %u", crp!=ZSTDcrp_leaveDirty); + if (crp!=ZSTDcrp_leaveDirty) { + /* reset tables only */ + ZSTD_cwksp_clean_tables(ws); + } + + /* opt parser space */ + if ((forWho == ZSTD_resetTarget_CCtx) && (cParams->strategy >= ZSTD_btopt)) { + DEBUGLOG(4, "reserving optimal parser space"); + ms->opt.litFreq = (unsigned*)ZSTD_cwksp_reserve_aligned(ws, (1<opt.litLengthFreq = (unsigned*)ZSTD_cwksp_reserve_aligned(ws, (MaxLL+1) * sizeof(unsigned)); + ms->opt.matchLengthFreq = (unsigned*)ZSTD_cwksp_reserve_aligned(ws, (MaxML+1) * sizeof(unsigned)); + ms->opt.offCodeFreq = (unsigned*)ZSTD_cwksp_reserve_aligned(ws, (MaxOff+1) * sizeof(unsigned)); + ms->opt.matchTable = (ZSTD_match_t*)ZSTD_cwksp_reserve_aligned(ws, (ZSTD_OPT_NUM+1) * sizeof(ZSTD_match_t)); + ms->opt.priceTable = (ZSTD_optimal_t*)ZSTD_cwksp_reserve_aligned(ws, (ZSTD_OPT_NUM+1) * sizeof(ZSTD_optimal_t)); + } + + ms->cParams = *cParams; + + RETURN_ERROR_IF(ZSTD_cwksp_reserve_failed(ws), memory_allocation, + "failed a workspace allocation in ZSTD_reset_matchState"); + + return 0; +} + +/* ZSTD_indexTooCloseToMax() : + * minor optimization : prefer memset() rather than reduceIndex() + * which is measurably slow in some circumstances (reported for Visual Studio). + * Works when re-using a context for a lot of smallish inputs : + * if all inputs are smaller than ZSTD_INDEXOVERFLOW_MARGIN, + * memset() will be triggered before reduceIndex(). + */ +#define ZSTD_INDEXOVERFLOW_MARGIN (16 MB) +static int ZSTD_indexTooCloseToMax(ZSTD_window_t w) +{ + return (size_t)(w.nextSrc - w.base) > (ZSTD_CURRENT_MAX - ZSTD_INDEXOVERFLOW_MARGIN); +} + +/*! ZSTD_resetCCtx_internal() : + note : `params` are assumed fully validated at this stage */ +static size_t ZSTD_resetCCtx_internal(ZSTD_CCtx* zc, + ZSTD_CCtx_params params, + U64 const pledgedSrcSize, + ZSTD_compResetPolicy_e const crp, + ZSTD_buffered_policy_e const zbuff) +{ + ZSTD_cwksp* const ws = &zc->workspace; + DEBUGLOG(4, "ZSTD_resetCCtx_internal: pledgedSrcSize=%u, wlog=%u", + (U32)pledgedSrcSize, params.cParams.windowLog); + assert(!ZSTD_isError(ZSTD_checkCParams(params.cParams))); + + zc->isFirstBlock = 1; + + if (params.ldmParams.enableLdm) { + /* Adjust long distance matching parameters */ + ZSTD_ldm_adjustParameters(¶ms.ldmParams, ¶ms.cParams); + assert(params.ldmParams.hashLog >= params.ldmParams.bucketSizeLog); + assert(params.ldmParams.hashRateLog < 32); + zc->ldmState.hashPower = ZSTD_rollingHash_primePower(params.ldmParams.minMatchLength); + } + + { size_t const windowSize = MAX(1, (size_t)MIN(((U64)1 << params.cParams.windowLog), pledgedSrcSize)); + size_t const blockSize = MIN(ZSTD_BLOCKSIZE_MAX, windowSize); + U32 const divider = (params.cParams.minMatch==3) ? 3 : 4; + size_t const maxNbSeq = blockSize / divider; + size_t const tokenSpace = ZSTD_cwksp_alloc_size(WILDCOPY_OVERLENGTH + blockSize) + + ZSTD_cwksp_alloc_size(maxNbSeq * sizeof(seqDef)) + + 3 * ZSTD_cwksp_alloc_size(maxNbSeq * sizeof(BYTE)); + size_t const buffOutSize = (zbuff==ZSTDb_buffered) ? ZSTD_compressBound(blockSize)+1 : 0; + size_t const buffInSize = (zbuff==ZSTDb_buffered) ? windowSize + blockSize : 0; + size_t const matchStateSize = ZSTD_sizeof_matchState(¶ms.cParams, /* forCCtx */ 1); + size_t const maxNbLdmSeq = ZSTD_ldm_getMaxNbSeq(params.ldmParams, blockSize); + + ZSTD_indexResetPolicy_e needsIndexReset = zc->initialized ? ZSTDirp_continue : ZSTDirp_reset; + + if (ZSTD_indexTooCloseToMax(zc->blockState.matchState.window)) { + needsIndexReset = ZSTDirp_reset; + } + + if (!zc->staticSize) ZSTD_cwksp_bump_oversized_duration(ws, 0); + + /* Check if workspace is large enough, alloc a new one if needed */ + { size_t const cctxSpace = zc->staticSize ? ZSTD_cwksp_alloc_size(sizeof(ZSTD_CCtx)) : 0; + size_t const entropySpace = ZSTD_cwksp_alloc_size(HUF_WORKSPACE_SIZE); + size_t const blockStateSpace = 2 * ZSTD_cwksp_alloc_size(sizeof(ZSTD_compressedBlockState_t)); + size_t const bufferSpace = ZSTD_cwksp_alloc_size(buffInSize) + ZSTD_cwksp_alloc_size(buffOutSize); + size_t const ldmSpace = ZSTD_ldm_getTableSize(params.ldmParams); + size_t const ldmSeqSpace = ZSTD_cwksp_alloc_size(maxNbLdmSeq * sizeof(rawSeq)); + + size_t const neededSpace = + cctxSpace + + entropySpace + + blockStateSpace + + ldmSpace + + ldmSeqSpace + + matchStateSize + + tokenSpace + + bufferSpace; + + int const workspaceTooSmall = ZSTD_cwksp_sizeof(ws) < neededSpace; + int const workspaceWasteful = ZSTD_cwksp_check_wasteful(ws, neededSpace); + + DEBUGLOG(4, "Need %zuKB workspace, including %zuKB for match state, and %zuKB for buffers", + neededSpace>>10, matchStateSize>>10, bufferSpace>>10); + DEBUGLOG(4, "windowSize: %zu - blockSize: %zu", windowSize, blockSize); + + if (workspaceTooSmall || workspaceWasteful) { + DEBUGLOG(4, "Resize workspaceSize from %zuKB to %zuKB", + ZSTD_cwksp_sizeof(ws) >> 10, + neededSpace >> 10); + + RETURN_ERROR_IF(zc->staticSize, memory_allocation, "static cctx : no resize"); + + needsIndexReset = ZSTDirp_reset; + + ZSTD_cwksp_free(ws, zc->customMem); + FORWARD_IF_ERROR(ZSTD_cwksp_create(ws, neededSpace, zc->customMem), ""); + + DEBUGLOG(5, "reserving object space"); + /* Statically sized space. + * entropyWorkspace never moves, + * though prev/next block swap places */ + assert(ZSTD_cwksp_check_available(ws, 2 * sizeof(ZSTD_compressedBlockState_t))); + zc->blockState.prevCBlock = (ZSTD_compressedBlockState_t*) ZSTD_cwksp_reserve_object(ws, sizeof(ZSTD_compressedBlockState_t)); + RETURN_ERROR_IF(zc->blockState.prevCBlock == NULL, memory_allocation, "couldn't allocate prevCBlock"); + zc->blockState.nextCBlock = (ZSTD_compressedBlockState_t*) ZSTD_cwksp_reserve_object(ws, sizeof(ZSTD_compressedBlockState_t)); + RETURN_ERROR_IF(zc->blockState.nextCBlock == NULL, memory_allocation, "couldn't allocate nextCBlock"); + zc->entropyWorkspace = (U32*) ZSTD_cwksp_reserve_object(ws, HUF_WORKSPACE_SIZE); + RETURN_ERROR_IF(zc->blockState.nextCBlock == NULL, memory_allocation, "couldn't allocate entropyWorkspace"); + } } + + ZSTD_cwksp_clear(ws); + + /* init params */ + zc->appliedParams = params; + zc->blockState.matchState.cParams = params.cParams; + zc->pledgedSrcSizePlusOne = pledgedSrcSize+1; + zc->consumedSrcSize = 0; + zc->producedCSize = 0; + if (pledgedSrcSize == ZSTD_CONTENTSIZE_UNKNOWN) + zc->appliedParams.fParams.contentSizeFlag = 0; + DEBUGLOG(4, "pledged content size : %u ; flag : %u", + (unsigned)pledgedSrcSize, zc->appliedParams.fParams.contentSizeFlag); + zc->blockSize = blockSize; + + XXH64_reset(&zc->xxhState, 0); + zc->stage = ZSTDcs_init; + zc->dictID = 0; + + ZSTD_reset_compressedBlockState(zc->blockState.prevCBlock); + + /* ZSTD_wildcopy() is used to copy into the literals buffer, + * so we have to oversize the buffer by WILDCOPY_OVERLENGTH bytes. + */ + zc->seqStore.litStart = ZSTD_cwksp_reserve_buffer(ws, blockSize + WILDCOPY_OVERLENGTH); + zc->seqStore.maxNbLit = blockSize; + + /* buffers */ + zc->inBuffSize = buffInSize; + zc->inBuff = (char*)ZSTD_cwksp_reserve_buffer(ws, buffInSize); + zc->outBuffSize = buffOutSize; + zc->outBuff = (char*)ZSTD_cwksp_reserve_buffer(ws, buffOutSize); + + /* ldm bucketOffsets table */ + if (params.ldmParams.enableLdm) { + /* TODO: avoid memset? */ + size_t const ldmBucketSize = + ((size_t)1) << (params.ldmParams.hashLog - + params.ldmParams.bucketSizeLog); + zc->ldmState.bucketOffsets = ZSTD_cwksp_reserve_buffer(ws, ldmBucketSize); + memset(zc->ldmState.bucketOffsets, 0, ldmBucketSize); + } + + /* sequences storage */ + ZSTD_referenceExternalSequences(zc, NULL, 0); + zc->seqStore.maxNbSeq = maxNbSeq; + zc->seqStore.llCode = ZSTD_cwksp_reserve_buffer(ws, maxNbSeq * sizeof(BYTE)); + zc->seqStore.mlCode = ZSTD_cwksp_reserve_buffer(ws, maxNbSeq * sizeof(BYTE)); + zc->seqStore.ofCode = ZSTD_cwksp_reserve_buffer(ws, maxNbSeq * sizeof(BYTE)); + zc->seqStore.sequencesStart = (seqDef*)ZSTD_cwksp_reserve_aligned(ws, maxNbSeq * sizeof(seqDef)); + + FORWARD_IF_ERROR(ZSTD_reset_matchState( + &zc->blockState.matchState, + ws, + ¶ms.cParams, + crp, + needsIndexReset, + ZSTD_resetTarget_CCtx), ""); + + /* ldm hash table */ + if (params.ldmParams.enableLdm) { + /* TODO: avoid memset? */ + size_t const ldmHSize = ((size_t)1) << params.ldmParams.hashLog; + zc->ldmState.hashTable = (ldmEntry_t*)ZSTD_cwksp_reserve_aligned(ws, ldmHSize * sizeof(ldmEntry_t)); + memset(zc->ldmState.hashTable, 0, ldmHSize * sizeof(ldmEntry_t)); + zc->ldmSequences = (rawSeq*)ZSTD_cwksp_reserve_aligned(ws, maxNbLdmSeq * sizeof(rawSeq)); + zc->maxNbLdmSequences = maxNbLdmSeq; + + ZSTD_window_init(&zc->ldmState.window); + ZSTD_window_clear(&zc->ldmState.window); + zc->ldmState.loadedDictEnd = 0; + } + + DEBUGLOG(3, "wksp: finished allocating, %zd bytes remain available", ZSTD_cwksp_available_space(ws)); + zc->initialized = 1; + + return 0; + } +} + +/* ZSTD_invalidateRepCodes() : + * ensures next compression will not use repcodes from previous block. + * Note : only works with regular variant; + * do not use with extDict variant ! */ +void ZSTD_invalidateRepCodes(ZSTD_CCtx* cctx) { + int i; + for (i=0; iblockState.prevCBlock->rep[i] = 0; + assert(!ZSTD_window_hasExtDict(cctx->blockState.matchState.window)); +} + +/* These are the approximate sizes for each strategy past which copying the + * dictionary tables into the working context is faster than using them + * in-place. + */ +static const size_t attachDictSizeCutoffs[ZSTD_STRATEGY_MAX+1] = { + 8 KB, /* unused */ + 8 KB, /* ZSTD_fast */ + 16 KB, /* ZSTD_dfast */ + 32 KB, /* ZSTD_greedy */ + 32 KB, /* ZSTD_lazy */ + 32 KB, /* ZSTD_lazy2 */ + 32 KB, /* ZSTD_btlazy2 */ + 32 KB, /* ZSTD_btopt */ + 8 KB, /* ZSTD_btultra */ + 8 KB /* ZSTD_btultra2 */ +}; + +static int ZSTD_shouldAttachDict(const ZSTD_CDict* cdict, + const ZSTD_CCtx_params* params, + U64 pledgedSrcSize) +{ + size_t cutoff = attachDictSizeCutoffs[cdict->matchState.cParams.strategy]; + return ( pledgedSrcSize <= cutoff + || pledgedSrcSize == ZSTD_CONTENTSIZE_UNKNOWN + || params->attachDictPref == ZSTD_dictForceAttach ) + && params->attachDictPref != ZSTD_dictForceCopy + && !params->forceWindow; /* dictMatchState isn't correctly + * handled in _enforceMaxDist */ +} + +static size_t +ZSTD_resetCCtx_byAttachingCDict(ZSTD_CCtx* cctx, + const ZSTD_CDict* cdict, + ZSTD_CCtx_params params, + U64 pledgedSrcSize, + ZSTD_buffered_policy_e zbuff) +{ + { const ZSTD_compressionParameters* const cdict_cParams = &cdict->matchState.cParams; + unsigned const windowLog = params.cParams.windowLog; + assert(windowLog != 0); + /* Resize working context table params for input only, since the dict + * has its own tables. */ + /* pledgeSrcSize == 0 means 0! */ + params.cParams = ZSTD_adjustCParams_internal(*cdict_cParams, pledgedSrcSize, 0); + params.cParams.windowLog = windowLog; + FORWARD_IF_ERROR(ZSTD_resetCCtx_internal(cctx, params, pledgedSrcSize, + ZSTDcrp_makeClean, zbuff), ""); + assert(cctx->appliedParams.cParams.strategy == cdict_cParams->strategy); + } + + { const U32 cdictEnd = (U32)( cdict->matchState.window.nextSrc + - cdict->matchState.window.base); + const U32 cdictLen = cdictEnd - cdict->matchState.window.dictLimit; + if (cdictLen == 0) { + /* don't even attach dictionaries with no contents */ + DEBUGLOG(4, "skipping attaching empty dictionary"); + } else { + DEBUGLOG(4, "attaching dictionary into context"); + cctx->blockState.matchState.dictMatchState = &cdict->matchState; + + /* prep working match state so dict matches never have negative indices + * when they are translated to the working context's index space. */ + if (cctx->blockState.matchState.window.dictLimit < cdictEnd) { + cctx->blockState.matchState.window.nextSrc = + cctx->blockState.matchState.window.base + cdictEnd; + ZSTD_window_clear(&cctx->blockState.matchState.window); + } + /* loadedDictEnd is expressed within the referential of the active context */ + cctx->blockState.matchState.loadedDictEnd = cctx->blockState.matchState.window.dictLimit; + } } + + cctx->dictID = cdict->dictID; + + /* copy block state */ + memcpy(cctx->blockState.prevCBlock, &cdict->cBlockState, sizeof(cdict->cBlockState)); + + return 0; +} + +static size_t ZSTD_resetCCtx_byCopyingCDict(ZSTD_CCtx* cctx, + const ZSTD_CDict* cdict, + ZSTD_CCtx_params params, + U64 pledgedSrcSize, + ZSTD_buffered_policy_e zbuff) +{ + const ZSTD_compressionParameters *cdict_cParams = &cdict->matchState.cParams; + + DEBUGLOG(4, "copying dictionary into context"); + + { unsigned const windowLog = params.cParams.windowLog; + assert(windowLog != 0); + /* Copy only compression parameters related to tables. */ + params.cParams = *cdict_cParams; + params.cParams.windowLog = windowLog; + FORWARD_IF_ERROR(ZSTD_resetCCtx_internal(cctx, params, pledgedSrcSize, + ZSTDcrp_leaveDirty, zbuff), ""); + assert(cctx->appliedParams.cParams.strategy == cdict_cParams->strategy); + assert(cctx->appliedParams.cParams.hashLog == cdict_cParams->hashLog); + assert(cctx->appliedParams.cParams.chainLog == cdict_cParams->chainLog); + } + + ZSTD_cwksp_mark_tables_dirty(&cctx->workspace); + + /* copy tables */ + { size_t const chainSize = (cdict_cParams->strategy == ZSTD_fast) ? 0 : ((size_t)1 << cdict_cParams->chainLog); + size_t const hSize = (size_t)1 << cdict_cParams->hashLog; + + memcpy(cctx->blockState.matchState.hashTable, + cdict->matchState.hashTable, + hSize * sizeof(U32)); + memcpy(cctx->blockState.matchState.chainTable, + cdict->matchState.chainTable, + chainSize * sizeof(U32)); + } + + /* Zero the hashTable3, since the cdict never fills it */ + { int const h3log = cctx->blockState.matchState.hashLog3; + size_t const h3Size = h3log ? ((size_t)1 << h3log) : 0; + assert(cdict->matchState.hashLog3 == 0); + memset(cctx->blockState.matchState.hashTable3, 0, h3Size * sizeof(U32)); + } + + ZSTD_cwksp_mark_tables_clean(&cctx->workspace); + + /* copy dictionary offsets */ + { ZSTD_matchState_t const* srcMatchState = &cdict->matchState; + ZSTD_matchState_t* dstMatchState = &cctx->blockState.matchState; + dstMatchState->window = srcMatchState->window; + dstMatchState->nextToUpdate = srcMatchState->nextToUpdate; + dstMatchState->loadedDictEnd= srcMatchState->loadedDictEnd; + } + + cctx->dictID = cdict->dictID; + + /* copy block state */ + memcpy(cctx->blockState.prevCBlock, &cdict->cBlockState, sizeof(cdict->cBlockState)); + + return 0; +} + +/* We have a choice between copying the dictionary context into the working + * context, or referencing the dictionary context from the working context + * in-place. We decide here which strategy to use. */ +static size_t ZSTD_resetCCtx_usingCDict(ZSTD_CCtx* cctx, + const ZSTD_CDict* cdict, + const ZSTD_CCtx_params* params, + U64 pledgedSrcSize, + ZSTD_buffered_policy_e zbuff) +{ + + DEBUGLOG(4, "ZSTD_resetCCtx_usingCDict (pledgedSrcSize=%u)", + (unsigned)pledgedSrcSize); + + if (ZSTD_shouldAttachDict(cdict, params, pledgedSrcSize)) { + return ZSTD_resetCCtx_byAttachingCDict( + cctx, cdict, *params, pledgedSrcSize, zbuff); + } else { + return ZSTD_resetCCtx_byCopyingCDict( + cctx, cdict, *params, pledgedSrcSize, zbuff); + } +} + +/*! ZSTD_copyCCtx_internal() : + * Duplicate an existing context `srcCCtx` into another one `dstCCtx`. + * Only works during stage ZSTDcs_init (i.e. after creation, but before first call to ZSTD_compressContinue()). + * The "context", in this case, refers to the hash and chain tables, + * entropy tables, and dictionary references. + * `windowLog` value is enforced if != 0, otherwise value is copied from srcCCtx. + * @return : 0, or an error code */ +static size_t ZSTD_copyCCtx_internal(ZSTD_CCtx* dstCCtx, + const ZSTD_CCtx* srcCCtx, + ZSTD_frameParameters fParams, + U64 pledgedSrcSize, + ZSTD_buffered_policy_e zbuff) +{ + DEBUGLOG(5, "ZSTD_copyCCtx_internal"); + RETURN_ERROR_IF(srcCCtx->stage!=ZSTDcs_init, stage_wrong, + "Can't copy a ctx that's not in init stage."); + + memcpy(&dstCCtx->customMem, &srcCCtx->customMem, sizeof(ZSTD_customMem)); + { ZSTD_CCtx_params params = dstCCtx->requestedParams; + /* Copy only compression parameters related to tables. */ + params.cParams = srcCCtx->appliedParams.cParams; + params.fParams = fParams; + ZSTD_resetCCtx_internal(dstCCtx, params, pledgedSrcSize, + ZSTDcrp_leaveDirty, zbuff); + assert(dstCCtx->appliedParams.cParams.windowLog == srcCCtx->appliedParams.cParams.windowLog); + assert(dstCCtx->appliedParams.cParams.strategy == srcCCtx->appliedParams.cParams.strategy); + assert(dstCCtx->appliedParams.cParams.hashLog == srcCCtx->appliedParams.cParams.hashLog); + assert(dstCCtx->appliedParams.cParams.chainLog == srcCCtx->appliedParams.cParams.chainLog); + assert(dstCCtx->blockState.matchState.hashLog3 == srcCCtx->blockState.matchState.hashLog3); + } + + ZSTD_cwksp_mark_tables_dirty(&dstCCtx->workspace); + + /* copy tables */ + { size_t const chainSize = (srcCCtx->appliedParams.cParams.strategy == ZSTD_fast) ? 0 : ((size_t)1 << srcCCtx->appliedParams.cParams.chainLog); + size_t const hSize = (size_t)1 << srcCCtx->appliedParams.cParams.hashLog; + int const h3log = srcCCtx->blockState.matchState.hashLog3; + size_t const h3Size = h3log ? ((size_t)1 << h3log) : 0; + + memcpy(dstCCtx->blockState.matchState.hashTable, + srcCCtx->blockState.matchState.hashTable, + hSize * sizeof(U32)); + memcpy(dstCCtx->blockState.matchState.chainTable, + srcCCtx->blockState.matchState.chainTable, + chainSize * sizeof(U32)); + memcpy(dstCCtx->blockState.matchState.hashTable3, + srcCCtx->blockState.matchState.hashTable3, + h3Size * sizeof(U32)); + } + + ZSTD_cwksp_mark_tables_clean(&dstCCtx->workspace); + + /* copy dictionary offsets */ + { + const ZSTD_matchState_t* srcMatchState = &srcCCtx->blockState.matchState; + ZSTD_matchState_t* dstMatchState = &dstCCtx->blockState.matchState; + dstMatchState->window = srcMatchState->window; + dstMatchState->nextToUpdate = srcMatchState->nextToUpdate; + dstMatchState->loadedDictEnd= srcMatchState->loadedDictEnd; + } + dstCCtx->dictID = srcCCtx->dictID; + + /* copy block state */ + memcpy(dstCCtx->blockState.prevCBlock, srcCCtx->blockState.prevCBlock, sizeof(*srcCCtx->blockState.prevCBlock)); + + return 0; +} + +/*! ZSTD_copyCCtx() : + * Duplicate an existing context `srcCCtx` into another one `dstCCtx`. + * Only works during stage ZSTDcs_init (i.e. after creation, but before first call to ZSTD_compressContinue()). + * pledgedSrcSize==0 means "unknown". +* @return : 0, or an error code */ +size_t ZSTD_copyCCtx(ZSTD_CCtx* dstCCtx, const ZSTD_CCtx* srcCCtx, unsigned long long pledgedSrcSize) +{ + ZSTD_frameParameters fParams = { 1 /*content*/, 0 /*checksum*/, 0 /*noDictID*/ }; + ZSTD_buffered_policy_e const zbuff = (ZSTD_buffered_policy_e)(srcCCtx->inBuffSize>0); + ZSTD_STATIC_ASSERT((U32)ZSTDb_buffered==1); + if (pledgedSrcSize==0) pledgedSrcSize = ZSTD_CONTENTSIZE_UNKNOWN; + fParams.contentSizeFlag = (pledgedSrcSize != ZSTD_CONTENTSIZE_UNKNOWN); + + return ZSTD_copyCCtx_internal(dstCCtx, srcCCtx, + fParams, pledgedSrcSize, + zbuff); +} + + +#define ZSTD_ROWSIZE 16 +/*! ZSTD_reduceTable() : + * reduce table indexes by `reducerValue`, or squash to zero. + * PreserveMark preserves "unsorted mark" for btlazy2 strategy. + * It must be set to a clear 0/1 value, to remove branch during inlining. + * Presume table size is a multiple of ZSTD_ROWSIZE + * to help auto-vectorization */ +FORCE_INLINE_TEMPLATE void +ZSTD_reduceTable_internal (U32* const table, U32 const size, U32 const reducerValue, int const preserveMark) +{ + int const nbRows = (int)size / ZSTD_ROWSIZE; + int cellNb = 0; + int rowNb; + assert((size & (ZSTD_ROWSIZE-1)) == 0); /* multiple of ZSTD_ROWSIZE */ + assert(size < (1U<<31)); /* can be casted to int */ + +#if defined (MEMORY_SANITIZER) && !defined (ZSTD_MSAN_DONT_POISON_WORKSPACE) + /* To validate that the table re-use logic is sound, and that we don't + * access table space that we haven't cleaned, we re-"poison" the table + * space every time we mark it dirty. + * + * This function however is intended to operate on those dirty tables and + * re-clean them. So when this function is used correctly, we can unpoison + * the memory it operated on. This introduces a blind spot though, since + * if we now try to operate on __actually__ poisoned memory, we will not + * detect that. */ + __msan_unpoison(table, size * sizeof(U32)); +#endif + + for (rowNb=0 ; rowNb < nbRows ; rowNb++) { + int column; + for (column=0; columncParams.hashLog; + ZSTD_reduceTable(ms->hashTable, hSize, reducerValue); + } + + if (params->cParams.strategy != ZSTD_fast) { + U32 const chainSize = (U32)1 << params->cParams.chainLog; + if (params->cParams.strategy == ZSTD_btlazy2) + ZSTD_reduceTable_btlazy2(ms->chainTable, chainSize, reducerValue); + else + ZSTD_reduceTable(ms->chainTable, chainSize, reducerValue); + } + + if (ms->hashLog3) { + U32 const h3Size = (U32)1 << ms->hashLog3; + ZSTD_reduceTable(ms->hashTable3, h3Size, reducerValue); + } +} + + +/*-******************************************************* +* Block entropic compression +*********************************************************/ + +/* See doc/zstd_compression_format.md for detailed format description */ + +void ZSTD_seqToCodes(const seqStore_t* seqStorePtr) +{ + const seqDef* const sequences = seqStorePtr->sequencesStart; + BYTE* const llCodeTable = seqStorePtr->llCode; + BYTE* const ofCodeTable = seqStorePtr->ofCode; + BYTE* const mlCodeTable = seqStorePtr->mlCode; + U32 const nbSeq = (U32)(seqStorePtr->sequences - seqStorePtr->sequencesStart); + U32 u; + assert(nbSeq <= seqStorePtr->maxNbSeq); + for (u=0; ulongLengthID==1) + llCodeTable[seqStorePtr->longLengthPos] = MaxLL; + if (seqStorePtr->longLengthID==2) + mlCodeTable[seqStorePtr->longLengthPos] = MaxML; +} + +/* ZSTD_useTargetCBlockSize(): + * Returns if target compressed block size param is being used. + * If used, compression will do best effort to make a compressed block size to be around targetCBlockSize. + * Returns 1 if true, 0 otherwise. */ +static int ZSTD_useTargetCBlockSize(const ZSTD_CCtx_params* cctxParams) +{ + DEBUGLOG(5, "ZSTD_useTargetCBlockSize (targetCBlockSize=%zu)", cctxParams->targetCBlockSize); + return (cctxParams->targetCBlockSize != 0); +} + +/* ZSTD_compressSequences_internal(): + * actually compresses both literals and sequences */ +MEM_STATIC size_t +ZSTD_compressSequences_internal(seqStore_t* seqStorePtr, + const ZSTD_entropyCTables_t* prevEntropy, + ZSTD_entropyCTables_t* nextEntropy, + const ZSTD_CCtx_params* cctxParams, + void* dst, size_t dstCapacity, + void* entropyWorkspace, size_t entropyWkspSize, + const int bmi2) +{ + const int longOffsets = cctxParams->cParams.windowLog > STREAM_ACCUMULATOR_MIN; + ZSTD_strategy const strategy = cctxParams->cParams.strategy; + unsigned count[MaxSeq+1]; + FSE_CTable* CTable_LitLength = nextEntropy->fse.litlengthCTable; + FSE_CTable* CTable_OffsetBits = nextEntropy->fse.offcodeCTable; + FSE_CTable* CTable_MatchLength = nextEntropy->fse.matchlengthCTable; + U32 LLtype, Offtype, MLtype; /* compressed, raw or rle */ + const seqDef* const sequences = seqStorePtr->sequencesStart; + const BYTE* const ofCodeTable = seqStorePtr->ofCode; + const BYTE* const llCodeTable = seqStorePtr->llCode; + const BYTE* const mlCodeTable = seqStorePtr->mlCode; + BYTE* const ostart = (BYTE*)dst; + BYTE* const oend = ostart + dstCapacity; + BYTE* op = ostart; + size_t const nbSeq = (size_t)(seqStorePtr->sequences - seqStorePtr->sequencesStart); + BYTE* seqHead; + BYTE* lastNCount = NULL; + + DEBUGLOG(5, "ZSTD_compressSequences_internal (nbSeq=%zu)", nbSeq); + ZSTD_STATIC_ASSERT(HUF_WORKSPACE_SIZE >= (1<litStart; + size_t const litSize = (size_t)(seqStorePtr->lit - literals); + size_t const cSize = ZSTD_compressLiterals( + &prevEntropy->huf, &nextEntropy->huf, + cctxParams->cParams.strategy, + ZSTD_disableLiteralsCompression(cctxParams), + op, dstCapacity, + literals, litSize, + entropyWorkspace, entropyWkspSize, + bmi2); + FORWARD_IF_ERROR(cSize, "ZSTD_compressLiterals failed"); + assert(cSize <= dstCapacity); + op += cSize; + } + + /* Sequences Header */ + RETURN_ERROR_IF((oend-op) < 3 /*max nbSeq Size*/ + 1 /*seqHead*/, + dstSize_tooSmall, "Can't fit seq hdr in output buf!"); + if (nbSeq < 128) { + *op++ = (BYTE)nbSeq; + } else if (nbSeq < LONGNBSEQ) { + op[0] = (BYTE)((nbSeq>>8) + 0x80); + op[1] = (BYTE)nbSeq; + op+=2; + } else { + op[0]=0xFF; + MEM_writeLE16(op+1, (U16)(nbSeq - LONGNBSEQ)); + op+=3; + } + assert(op <= oend); + if (nbSeq==0) { + /* Copy the old tables over as if we repeated them */ + memcpy(&nextEntropy->fse, &prevEntropy->fse, sizeof(prevEntropy->fse)); + return (size_t)(op - ostart); + } + + /* seqHead : flags for FSE encoding type */ + seqHead = op++; + assert(op <= oend); + + /* convert length/distances into codes */ + ZSTD_seqToCodes(seqStorePtr); + /* build CTable for Literal Lengths */ + { unsigned max = MaxLL; + size_t const mostFrequent = HIST_countFast_wksp(count, &max, llCodeTable, nbSeq, entropyWorkspace, entropyWkspSize); /* can't fail */ + DEBUGLOG(5, "Building LL table"); + nextEntropy->fse.litlength_repeatMode = prevEntropy->fse.litlength_repeatMode; + LLtype = ZSTD_selectEncodingType(&nextEntropy->fse.litlength_repeatMode, + count, max, mostFrequent, nbSeq, + LLFSELog, prevEntropy->fse.litlengthCTable, + LL_defaultNorm, LL_defaultNormLog, + ZSTD_defaultAllowed, strategy); + assert(set_basic < set_compressed && set_rle < set_compressed); + assert(!(LLtype < set_compressed && nextEntropy->fse.litlength_repeatMode != FSE_repeat_none)); /* We don't copy tables */ + { size_t const countSize = ZSTD_buildCTable( + op, (size_t)(oend - op), + CTable_LitLength, LLFSELog, (symbolEncodingType_e)LLtype, + count, max, llCodeTable, nbSeq, + LL_defaultNorm, LL_defaultNormLog, MaxLL, + prevEntropy->fse.litlengthCTable, + sizeof(prevEntropy->fse.litlengthCTable), + entropyWorkspace, entropyWkspSize); + FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for LitLens failed"); + if (LLtype == set_compressed) + lastNCount = op; + op += countSize; + assert(op <= oend); + } } + /* build CTable for Offsets */ + { unsigned max = MaxOff; + size_t const mostFrequent = HIST_countFast_wksp( + count, &max, ofCodeTable, nbSeq, entropyWorkspace, entropyWkspSize); /* can't fail */ + /* We can only use the basic table if max <= DefaultMaxOff, otherwise the offsets are too large */ + ZSTD_defaultPolicy_e const defaultPolicy = (max <= DefaultMaxOff) ? ZSTD_defaultAllowed : ZSTD_defaultDisallowed; + DEBUGLOG(5, "Building OF table"); + nextEntropy->fse.offcode_repeatMode = prevEntropy->fse.offcode_repeatMode; + Offtype = ZSTD_selectEncodingType(&nextEntropy->fse.offcode_repeatMode, + count, max, mostFrequent, nbSeq, + OffFSELog, prevEntropy->fse.offcodeCTable, + OF_defaultNorm, OF_defaultNormLog, + defaultPolicy, strategy); + assert(!(Offtype < set_compressed && nextEntropy->fse.offcode_repeatMode != FSE_repeat_none)); /* We don't copy tables */ + { size_t const countSize = ZSTD_buildCTable( + op, (size_t)(oend - op), + CTable_OffsetBits, OffFSELog, (symbolEncodingType_e)Offtype, + count, max, ofCodeTable, nbSeq, + OF_defaultNorm, OF_defaultNormLog, DefaultMaxOff, + prevEntropy->fse.offcodeCTable, + sizeof(prevEntropy->fse.offcodeCTable), + entropyWorkspace, entropyWkspSize); + FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for Offsets failed"); + if (Offtype == set_compressed) + lastNCount = op; + op += countSize; + assert(op <= oend); + } } + /* build CTable for MatchLengths */ + { unsigned max = MaxML; + size_t const mostFrequent = HIST_countFast_wksp( + count, &max, mlCodeTable, nbSeq, entropyWorkspace, entropyWkspSize); /* can't fail */ + DEBUGLOG(5, "Building ML table (remaining space : %i)", (int)(oend-op)); + nextEntropy->fse.matchlength_repeatMode = prevEntropy->fse.matchlength_repeatMode; + MLtype = ZSTD_selectEncodingType(&nextEntropy->fse.matchlength_repeatMode, + count, max, mostFrequent, nbSeq, + MLFSELog, prevEntropy->fse.matchlengthCTable, + ML_defaultNorm, ML_defaultNormLog, + ZSTD_defaultAllowed, strategy); + assert(!(MLtype < set_compressed && nextEntropy->fse.matchlength_repeatMode != FSE_repeat_none)); /* We don't copy tables */ + { size_t const countSize = ZSTD_buildCTable( + op, (size_t)(oend - op), + CTable_MatchLength, MLFSELog, (symbolEncodingType_e)MLtype, + count, max, mlCodeTable, nbSeq, + ML_defaultNorm, ML_defaultNormLog, MaxML, + prevEntropy->fse.matchlengthCTable, + sizeof(prevEntropy->fse.matchlengthCTable), + entropyWorkspace, entropyWkspSize); + FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for MatchLengths failed"); + if (MLtype == set_compressed) + lastNCount = op; + op += countSize; + assert(op <= oend); + } } + + *seqHead = (BYTE)((LLtype<<6) + (Offtype<<4) + (MLtype<<2)); + + { size_t const bitstreamSize = ZSTD_encodeSequences( + op, (size_t)(oend - op), + CTable_MatchLength, mlCodeTable, + CTable_OffsetBits, ofCodeTable, + CTable_LitLength, llCodeTable, + sequences, nbSeq, + longOffsets, bmi2); + FORWARD_IF_ERROR(bitstreamSize, "ZSTD_encodeSequences failed"); + op += bitstreamSize; + assert(op <= oend); + /* zstd versions <= 1.3.4 mistakenly report corruption when + * FSE_readNCount() receives a buffer < 4 bytes. + * Fixed by https://github.com/facebook/zstd/pull/1146. + * This can happen when the last set_compressed table present is 2 + * bytes and the bitstream is only one byte. + * In this exceedingly rare case, we will simply emit an uncompressed + * block, since it isn't worth optimizing. + */ + if (lastNCount && (op - lastNCount) < 4) { + /* NCountSize >= 2 && bitstreamSize > 0 ==> lastCountSize == 3 */ + assert(op - lastNCount == 3); + DEBUGLOG(5, "Avoiding bug in zstd decoder in versions <= 1.3.4 by " + "emitting an uncompressed block."); + return 0; + } + } + + DEBUGLOG(5, "compressed block size : %u", (unsigned)(op - ostart)); + return (size_t)(op - ostart); +} + +MEM_STATIC size_t +ZSTD_compressSequences(seqStore_t* seqStorePtr, + const ZSTD_entropyCTables_t* prevEntropy, + ZSTD_entropyCTables_t* nextEntropy, + const ZSTD_CCtx_params* cctxParams, + void* dst, size_t dstCapacity, + size_t srcSize, + void* entropyWorkspace, size_t entropyWkspSize, + int bmi2) +{ + size_t const cSize = ZSTD_compressSequences_internal( + seqStorePtr, prevEntropy, nextEntropy, cctxParams, + dst, dstCapacity, + entropyWorkspace, entropyWkspSize, bmi2); + if (cSize == 0) return 0; + /* When srcSize <= dstCapacity, there is enough space to write a raw uncompressed block. + * Since we ran out of space, block must be not compressible, so fall back to raw uncompressed block. + */ + if ((cSize == ERROR(dstSize_tooSmall)) & (srcSize <= dstCapacity)) + return 0; /* block not compressed */ + FORWARD_IF_ERROR(cSize, "ZSTD_compressSequences_internal failed"); + + /* Check compressibility */ + { size_t const maxCSize = srcSize - ZSTD_minGain(srcSize, cctxParams->cParams.strategy); + if (cSize >= maxCSize) return 0; /* block not compressed */ + } + + return cSize; +} + +/* ZSTD_selectBlockCompressor() : + * Not static, but internal use only (used by long distance matcher) + * assumption : strat is a valid strategy */ +ZSTD_blockCompressor ZSTD_selectBlockCompressor(ZSTD_strategy strat, ZSTD_dictMode_e dictMode) +{ + static const ZSTD_blockCompressor blockCompressor[3][ZSTD_STRATEGY_MAX+1] = { + { ZSTD_compressBlock_fast /* default for 0 */, + ZSTD_compressBlock_fast, + ZSTD_compressBlock_doubleFast, + ZSTD_compressBlock_greedy, + ZSTD_compressBlock_lazy, + ZSTD_compressBlock_lazy2, + ZSTD_compressBlock_btlazy2, + ZSTD_compressBlock_btopt, + ZSTD_compressBlock_btultra, + ZSTD_compressBlock_btultra2 }, + { ZSTD_compressBlock_fast_extDict /* default for 0 */, + ZSTD_compressBlock_fast_extDict, + ZSTD_compressBlock_doubleFast_extDict, + ZSTD_compressBlock_greedy_extDict, + ZSTD_compressBlock_lazy_extDict, + ZSTD_compressBlock_lazy2_extDict, + ZSTD_compressBlock_btlazy2_extDict, + ZSTD_compressBlock_btopt_extDict, + ZSTD_compressBlock_btultra_extDict, + ZSTD_compressBlock_btultra_extDict }, + { ZSTD_compressBlock_fast_dictMatchState /* default for 0 */, + ZSTD_compressBlock_fast_dictMatchState, + ZSTD_compressBlock_doubleFast_dictMatchState, + ZSTD_compressBlock_greedy_dictMatchState, + ZSTD_compressBlock_lazy_dictMatchState, + ZSTD_compressBlock_lazy2_dictMatchState, + ZSTD_compressBlock_btlazy2_dictMatchState, + ZSTD_compressBlock_btopt_dictMatchState, + ZSTD_compressBlock_btultra_dictMatchState, + ZSTD_compressBlock_btultra_dictMatchState } + }; + ZSTD_blockCompressor selectedCompressor; + ZSTD_STATIC_ASSERT((unsigned)ZSTD_fast == 1); + + assert(ZSTD_cParam_withinBounds(ZSTD_c_strategy, strat)); + selectedCompressor = blockCompressor[(int)dictMode][(int)strat]; + assert(selectedCompressor != NULL); + return selectedCompressor; +} + +static void ZSTD_storeLastLiterals(seqStore_t* seqStorePtr, + const BYTE* anchor, size_t lastLLSize) +{ + memcpy(seqStorePtr->lit, anchor, lastLLSize); + seqStorePtr->lit += lastLLSize; +} + +void ZSTD_resetSeqStore(seqStore_t* ssPtr) +{ + ssPtr->lit = ssPtr->litStart; + ssPtr->sequences = ssPtr->sequencesStart; + ssPtr->longLengthID = 0; +} + +typedef enum { ZSTDbss_compress, ZSTDbss_noCompress } ZSTD_buildSeqStore_e; + +static size_t ZSTD_buildSeqStore(ZSTD_CCtx* zc, const void* src, size_t srcSize) +{ + ZSTD_matchState_t* const ms = &zc->blockState.matchState; + DEBUGLOG(5, "ZSTD_buildSeqStore (srcSize=%zu)", srcSize); + assert(srcSize <= ZSTD_BLOCKSIZE_MAX); + /* Assert that we have correctly flushed the ctx params into the ms's copy */ + ZSTD_assertEqualCParams(zc->appliedParams.cParams, ms->cParams); + if (srcSize < MIN_CBLOCK_SIZE+ZSTD_blockHeaderSize+1) { + ZSTD_ldm_skipSequences(&zc->externSeqStore, srcSize, zc->appliedParams.cParams.minMatch); + return ZSTDbss_noCompress; /* don't even attempt compression below a certain srcSize */ + } + ZSTD_resetSeqStore(&(zc->seqStore)); + /* required for optimal parser to read stats from dictionary */ + ms->opt.symbolCosts = &zc->blockState.prevCBlock->entropy; + /* tell the optimal parser how we expect to compress literals */ + ms->opt.literalCompressionMode = zc->appliedParams.literalCompressionMode; + /* a gap between an attached dict and the current window is not safe, + * they must remain adjacent, + * and when that stops being the case, the dict must be unset */ + assert(ms->dictMatchState == NULL || ms->loadedDictEnd == ms->window.dictLimit); + + /* limited update after a very long match */ + { const BYTE* const base = ms->window.base; + const BYTE* const istart = (const BYTE*)src; + const U32 current = (U32)(istart-base); + if (sizeof(ptrdiff_t)==8) assert(istart - base < (ptrdiff_t)(U32)(-1)); /* ensure no overflow */ + if (current > ms->nextToUpdate + 384) + ms->nextToUpdate = current - MIN(192, (U32)(current - ms->nextToUpdate - 384)); + } + + /* select and store sequences */ + { ZSTD_dictMode_e const dictMode = ZSTD_matchState_dictMode(ms); + size_t lastLLSize; + { int i; + for (i = 0; i < ZSTD_REP_NUM; ++i) + zc->blockState.nextCBlock->rep[i] = zc->blockState.prevCBlock->rep[i]; + } + if (zc->externSeqStore.pos < zc->externSeqStore.size) { + assert(!zc->appliedParams.ldmParams.enableLdm); + /* Updates ldmSeqStore.pos */ + lastLLSize = + ZSTD_ldm_blockCompress(&zc->externSeqStore, + ms, &zc->seqStore, + zc->blockState.nextCBlock->rep, + src, srcSize); + assert(zc->externSeqStore.pos <= zc->externSeqStore.size); + } else if (zc->appliedParams.ldmParams.enableLdm) { + rawSeqStore_t ldmSeqStore = {NULL, 0, 0, 0}; + + ldmSeqStore.seq = zc->ldmSequences; + ldmSeqStore.capacity = zc->maxNbLdmSequences; + /* Updates ldmSeqStore.size */ + FORWARD_IF_ERROR(ZSTD_ldm_generateSequences(&zc->ldmState, &ldmSeqStore, + &zc->appliedParams.ldmParams, + src, srcSize), ""); + /* Updates ldmSeqStore.pos */ + lastLLSize = + ZSTD_ldm_blockCompress(&ldmSeqStore, + ms, &zc->seqStore, + zc->blockState.nextCBlock->rep, + src, srcSize); + assert(ldmSeqStore.pos == ldmSeqStore.size); + } else { /* not long range mode */ + ZSTD_blockCompressor const blockCompressor = ZSTD_selectBlockCompressor(zc->appliedParams.cParams.strategy, dictMode); + lastLLSize = blockCompressor(ms, &zc->seqStore, zc->blockState.nextCBlock->rep, src, srcSize); + } + { const BYTE* const lastLiterals = (const BYTE*)src + srcSize - lastLLSize; + ZSTD_storeLastLiterals(&zc->seqStore, lastLiterals, lastLLSize); + } } + return ZSTDbss_compress; +} + +static void ZSTD_copyBlockSequences(ZSTD_CCtx* zc) +{ + const seqStore_t* seqStore = ZSTD_getSeqStore(zc); + const seqDef* seqs = seqStore->sequencesStart; + size_t seqsSize = seqStore->sequences - seqs; + + ZSTD_Sequence* outSeqs = &zc->seqCollector.seqStart[zc->seqCollector.seqIndex]; + size_t i; size_t position; int repIdx; + + assert(zc->seqCollector.seqIndex + 1 < zc->seqCollector.maxSequences); + for (i = 0, position = 0; i < seqsSize; ++i) { + outSeqs[i].offset = seqs[i].offset; + outSeqs[i].litLength = seqs[i].litLength; + outSeqs[i].matchLength = seqs[i].matchLength + MINMATCH; + + if (i == seqStore->longLengthPos) { + if (seqStore->longLengthID == 1) { + outSeqs[i].litLength += 0x10000; + } else if (seqStore->longLengthID == 2) { + outSeqs[i].matchLength += 0x10000; + } + } + + if (outSeqs[i].offset <= ZSTD_REP_NUM) { + outSeqs[i].rep = outSeqs[i].offset; + repIdx = (unsigned int)i - outSeqs[i].offset; + + if (outSeqs[i].litLength == 0) { + if (outSeqs[i].offset < 3) { + --repIdx; + } else { + repIdx = (unsigned int)i - 1; + } + ++outSeqs[i].rep; + } + assert(repIdx >= -3); + outSeqs[i].offset = repIdx >= 0 ? outSeqs[repIdx].offset : repStartValue[-repIdx - 1]; + if (outSeqs[i].rep == 4) { + --outSeqs[i].offset; + } + } else { + outSeqs[i].offset -= ZSTD_REP_NUM; + } + + position += outSeqs[i].litLength; + outSeqs[i].matchPos = (unsigned int)position; + position += outSeqs[i].matchLength; + } + zc->seqCollector.seqIndex += seqsSize; +} + +size_t ZSTD_getSequences(ZSTD_CCtx* zc, ZSTD_Sequence* outSeqs, + size_t outSeqsSize, const void* src, size_t srcSize) +{ + const size_t dstCapacity = ZSTD_compressBound(srcSize); + void* dst = ZSTD_malloc(dstCapacity, ZSTD_defaultCMem); + SeqCollector seqCollector; + + RETURN_ERROR_IF(dst == NULL, memory_allocation, "NULL pointer!"); + + seqCollector.collectSequences = 1; + seqCollector.seqStart = outSeqs; + seqCollector.seqIndex = 0; + seqCollector.maxSequences = outSeqsSize; + zc->seqCollector = seqCollector; + + ZSTD_compress2(zc, dst, dstCapacity, src, srcSize); + ZSTD_free(dst, ZSTD_defaultCMem); + return zc->seqCollector.seqIndex; +} + +/* Returns true if the given block is a RLE block */ +static int ZSTD_isRLE(const BYTE *ip, size_t length) { + size_t i; + if (length < 2) return 1; + for (i = 1; i < length; ++i) { + if (ip[0] != ip[i]) return 0; + } + return 1; +} + +/* Returns true if the given block may be RLE. + * This is just a heuristic based on the compressibility. + * It may return both false positives and false negatives. + */ +static int ZSTD_maybeRLE(seqStore_t const* seqStore) +{ + size_t const nbSeqs = (size_t)(seqStore->sequences - seqStore->sequencesStart); + size_t const nbLits = (size_t)(seqStore->lit - seqStore->litStart); + + return nbSeqs < 4 && nbLits < 10; +} + +static void ZSTD_confirmRepcodesAndEntropyTables(ZSTD_CCtx* zc) +{ + ZSTD_compressedBlockState_t* const tmp = zc->blockState.prevCBlock; + zc->blockState.prevCBlock = zc->blockState.nextCBlock; + zc->blockState.nextCBlock = tmp; +} + +static size_t ZSTD_compressBlock_internal(ZSTD_CCtx* zc, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, U32 frame) +{ + /* This the upper bound for the length of an rle block. + * This isn't the actual upper bound. Finding the real threshold + * needs further investigation. + */ + const U32 rleMaxLength = 25; + size_t cSize; + const BYTE* ip = (const BYTE*)src; + BYTE* op = (BYTE*)dst; + DEBUGLOG(5, "ZSTD_compressBlock_internal (dstCapacity=%u, dictLimit=%u, nextToUpdate=%u)", + (unsigned)dstCapacity, (unsigned)zc->blockState.matchState.window.dictLimit, + (unsigned)zc->blockState.matchState.nextToUpdate); + + { const size_t bss = ZSTD_buildSeqStore(zc, src, srcSize); + FORWARD_IF_ERROR(bss, "ZSTD_buildSeqStore failed"); + if (bss == ZSTDbss_noCompress) { cSize = 0; goto out; } + } + + if (zc->seqCollector.collectSequences) { + ZSTD_copyBlockSequences(zc); + return 0; + } + + /* encode sequences and literals */ + cSize = ZSTD_compressSequences(&zc->seqStore, + &zc->blockState.prevCBlock->entropy, &zc->blockState.nextCBlock->entropy, + &zc->appliedParams, + dst, dstCapacity, + srcSize, + zc->entropyWorkspace, HUF_WORKSPACE_SIZE /* statically allocated in resetCCtx */, + zc->bmi2); + + if (frame && + /* We don't want to emit our first block as a RLE even if it qualifies because + * doing so will cause the decoder (cli only) to throw a "should consume all input error." + * This is only an issue for zstd <= v1.4.3 + */ + !zc->isFirstBlock && + cSize < rleMaxLength && + ZSTD_isRLE(ip, srcSize)) + { + cSize = 1; + op[0] = ip[0]; + } + +out: + if (!ZSTD_isError(cSize) && cSize > 1) { + ZSTD_confirmRepcodesAndEntropyTables(zc); + } + /* We check that dictionaries have offset codes available for the first + * block. After the first block, the offcode table might not have large + * enough codes to represent the offsets in the data. + */ + if (zc->blockState.prevCBlock->entropy.fse.offcode_repeatMode == FSE_repeat_valid) + zc->blockState.prevCBlock->entropy.fse.offcode_repeatMode = FSE_repeat_check; + + return cSize; +} + +static size_t ZSTD_compressBlock_targetCBlockSize_body(ZSTD_CCtx* zc, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + const size_t bss, U32 lastBlock) +{ + DEBUGLOG(6, "Attempting ZSTD_compressSuperBlock()"); + if (bss == ZSTDbss_compress) { + if (/* We don't want to emit our first block as a RLE even if it qualifies because + * doing so will cause the decoder (cli only) to throw a "should consume all input error." + * This is only an issue for zstd <= v1.4.3 + */ + !zc->isFirstBlock && + ZSTD_maybeRLE(&zc->seqStore) && + ZSTD_isRLE((BYTE const*)src, srcSize)) + { + return ZSTD_rleCompressBlock(dst, dstCapacity, *(BYTE const*)src, srcSize, lastBlock); + } + /* Attempt superblock compression. + * + * Note that compressed size of ZSTD_compressSuperBlock() is not bound by the + * standard ZSTD_compressBound(). This is a problem, because even if we have + * space now, taking an extra byte now could cause us to run out of space later + * and violate ZSTD_compressBound(). + * + * Define blockBound(blockSize) = blockSize + ZSTD_blockHeaderSize. + * + * In order to respect ZSTD_compressBound() we must attempt to emit a raw + * uncompressed block in these cases: + * * cSize == 0: Return code for an uncompressed block. + * * cSize == dstSize_tooSmall: We may have expanded beyond blockBound(srcSize). + * ZSTD_noCompressBlock() will return dstSize_tooSmall if we are really out of + * output space. + * * cSize >= blockBound(srcSize): We have expanded the block too much so + * emit an uncompressed block. + */ + { + size_t const cSize = ZSTD_compressSuperBlock(zc, dst, dstCapacity, src, srcSize, lastBlock); + if (cSize != ERROR(dstSize_tooSmall)) { + size_t const maxCSize = srcSize - ZSTD_minGain(srcSize, zc->appliedParams.cParams.strategy); + FORWARD_IF_ERROR(cSize, "ZSTD_compressSuperBlock failed"); + if (cSize != 0 && cSize < maxCSize + ZSTD_blockHeaderSize) { + ZSTD_confirmRepcodesAndEntropyTables(zc); + return cSize; + } + } + } + } + + DEBUGLOG(6, "Resorting to ZSTD_noCompressBlock()"); + /* Superblock compression failed, attempt to emit a single no compress block. + * The decoder will be able to stream this block since it is uncompressed. + */ + return ZSTD_noCompressBlock(dst, dstCapacity, src, srcSize, lastBlock); +} + +static size_t ZSTD_compressBlock_targetCBlockSize(ZSTD_CCtx* zc, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + U32 lastBlock) +{ + size_t cSize = 0; + const size_t bss = ZSTD_buildSeqStore(zc, src, srcSize); + DEBUGLOG(5, "ZSTD_compressBlock_targetCBlockSize (dstCapacity=%u, dictLimit=%u, nextToUpdate=%u, srcSize=%zu)", + (unsigned)dstCapacity, (unsigned)zc->blockState.matchState.window.dictLimit, (unsigned)zc->blockState.matchState.nextToUpdate, srcSize); + FORWARD_IF_ERROR(bss, "ZSTD_buildSeqStore failed"); + + cSize = ZSTD_compressBlock_targetCBlockSize_body(zc, dst, dstCapacity, src, srcSize, bss, lastBlock); + FORWARD_IF_ERROR(cSize, "ZSTD_compressBlock_targetCBlockSize_body failed"); + + if (zc->blockState.prevCBlock->entropy.fse.offcode_repeatMode == FSE_repeat_valid) + zc->blockState.prevCBlock->entropy.fse.offcode_repeatMode = FSE_repeat_check; + + return cSize; +} + +static void ZSTD_overflowCorrectIfNeeded(ZSTD_matchState_t* ms, + ZSTD_cwksp* ws, + ZSTD_CCtx_params const* params, + void const* ip, + void const* iend) +{ + if (ZSTD_window_needOverflowCorrection(ms->window, iend)) { + U32 const maxDist = (U32)1 << params->cParams.windowLog; + U32 const cycleLog = ZSTD_cycleLog(params->cParams.chainLog, params->cParams.strategy); + U32 const correction = ZSTD_window_correctOverflow(&ms->window, cycleLog, maxDist, ip); + ZSTD_STATIC_ASSERT(ZSTD_CHAINLOG_MAX <= 30); + ZSTD_STATIC_ASSERT(ZSTD_WINDOWLOG_MAX_32 <= 30); + ZSTD_STATIC_ASSERT(ZSTD_WINDOWLOG_MAX <= 31); + ZSTD_cwksp_mark_tables_dirty(ws); + ZSTD_reduceIndex(ms, params, correction); + ZSTD_cwksp_mark_tables_clean(ws); + if (ms->nextToUpdate < correction) ms->nextToUpdate = 0; + else ms->nextToUpdate -= correction; + /* invalidate dictionaries on overflow correction */ + ms->loadedDictEnd = 0; + ms->dictMatchState = NULL; + } +} + +/*! ZSTD_compress_frameChunk() : +* Compress a chunk of data into one or multiple blocks. +* All blocks will be terminated, all input will be consumed. +* Function will issue an error if there is not enough `dstCapacity` to hold the compressed content. +* Frame is supposed already started (header already produced) +* @return : compressed size, or an error code +*/ +static size_t ZSTD_compress_frameChunk (ZSTD_CCtx* cctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + U32 lastFrameChunk) +{ + size_t blockSize = cctx->blockSize; + size_t remaining = srcSize; + const BYTE* ip = (const BYTE*)src; + BYTE* const ostart = (BYTE*)dst; + BYTE* op = ostart; + U32 const maxDist = (U32)1 << cctx->appliedParams.cParams.windowLog; + + assert(cctx->appliedParams.cParams.windowLog <= ZSTD_WINDOWLOG_MAX); + + DEBUGLOG(5, "ZSTD_compress_frameChunk (blockSize=%u)", (unsigned)blockSize); + if (cctx->appliedParams.fParams.checksumFlag && srcSize) + XXH64_update(&cctx->xxhState, src, srcSize); + + while (remaining) { + ZSTD_matchState_t* const ms = &cctx->blockState.matchState; + U32 const lastBlock = lastFrameChunk & (blockSize >= remaining); + + RETURN_ERROR_IF(dstCapacity < ZSTD_blockHeaderSize + MIN_CBLOCK_SIZE, + dstSize_tooSmall, + "not enough space to store compressed block"); + if (remaining < blockSize) blockSize = remaining; + + ZSTD_overflowCorrectIfNeeded( + ms, &cctx->workspace, &cctx->appliedParams, ip, ip + blockSize); + ZSTD_checkDictValidity(&ms->window, ip + blockSize, maxDist, &ms->loadedDictEnd, &ms->dictMatchState); + + /* Ensure hash/chain table insertion resumes no sooner than lowlimit */ + if (ms->nextToUpdate < ms->window.lowLimit) ms->nextToUpdate = ms->window.lowLimit; + + { size_t cSize; + if (ZSTD_useTargetCBlockSize(&cctx->appliedParams)) { + cSize = ZSTD_compressBlock_targetCBlockSize(cctx, op, dstCapacity, ip, blockSize, lastBlock); + FORWARD_IF_ERROR(cSize, "ZSTD_compressBlock_targetCBlockSize failed"); + assert(cSize > 0); + assert(cSize <= blockSize + ZSTD_blockHeaderSize); + } else { + cSize = ZSTD_compressBlock_internal(cctx, + op+ZSTD_blockHeaderSize, dstCapacity-ZSTD_blockHeaderSize, + ip, blockSize, 1 /* frame */); + FORWARD_IF_ERROR(cSize, "ZSTD_compressBlock_internal failed"); + + if (cSize == 0) { /* block is not compressible */ + cSize = ZSTD_noCompressBlock(op, dstCapacity, ip, blockSize, lastBlock); + FORWARD_IF_ERROR(cSize, "ZSTD_noCompressBlock failed"); + } else { + U32 const cBlockHeader = cSize == 1 ? + lastBlock + (((U32)bt_rle)<<1) + (U32)(blockSize << 3) : + lastBlock + (((U32)bt_compressed)<<1) + (U32)(cSize << 3); + MEM_writeLE24(op, cBlockHeader); + cSize += ZSTD_blockHeaderSize; + } + } + + + ip += blockSize; + assert(remaining >= blockSize); + remaining -= blockSize; + op += cSize; + assert(dstCapacity >= cSize); + dstCapacity -= cSize; + cctx->isFirstBlock = 0; + DEBUGLOG(5, "ZSTD_compress_frameChunk: adding a block of size %u", + (unsigned)cSize); + } } + + if (lastFrameChunk && (op>ostart)) cctx->stage = ZSTDcs_ending; + return (size_t)(op-ostart); +} + + +static size_t ZSTD_writeFrameHeader(void* dst, size_t dstCapacity, + const ZSTD_CCtx_params* params, U64 pledgedSrcSize, U32 dictID) +{ BYTE* const op = (BYTE*)dst; + U32 const dictIDSizeCodeLength = (dictID>0) + (dictID>=256) + (dictID>=65536); /* 0-3 */ + U32 const dictIDSizeCode = params->fParams.noDictIDFlag ? 0 : dictIDSizeCodeLength; /* 0-3 */ + U32 const checksumFlag = params->fParams.checksumFlag>0; + U32 const windowSize = (U32)1 << params->cParams.windowLog; + U32 const singleSegment = params->fParams.contentSizeFlag && (windowSize >= pledgedSrcSize); + BYTE const windowLogByte = (BYTE)((params->cParams.windowLog - ZSTD_WINDOWLOG_ABSOLUTEMIN) << 3); + U32 const fcsCode = params->fParams.contentSizeFlag ? + (pledgedSrcSize>=256) + (pledgedSrcSize>=65536+256) + (pledgedSrcSize>=0xFFFFFFFFU) : 0; /* 0-3 */ + BYTE const frameHeaderDescriptionByte = (BYTE)(dictIDSizeCode + (checksumFlag<<2) + (singleSegment<<5) + (fcsCode<<6) ); + size_t pos=0; + + assert(!(params->fParams.contentSizeFlag && pledgedSrcSize == ZSTD_CONTENTSIZE_UNKNOWN)); + RETURN_ERROR_IF(dstCapacity < ZSTD_FRAMEHEADERSIZE_MAX, dstSize_tooSmall, + "dst buf is too small to fit worst-case frame header size."); + DEBUGLOG(4, "ZSTD_writeFrameHeader : dictIDFlag : %u ; dictID : %u ; dictIDSizeCode : %u", + !params->fParams.noDictIDFlag, (unsigned)dictID, (unsigned)dictIDSizeCode); + + if (params->format == ZSTD_f_zstd1) { + MEM_writeLE32(dst, ZSTD_MAGICNUMBER); + pos = 4; + } + op[pos++] = frameHeaderDescriptionByte; + if (!singleSegment) op[pos++] = windowLogByte; + switch(dictIDSizeCode) + { + default: assert(0); /* impossible */ + case 0 : break; + case 1 : op[pos] = (BYTE)(dictID); pos++; break; + case 2 : MEM_writeLE16(op+pos, (U16)dictID); pos+=2; break; + case 3 : MEM_writeLE32(op+pos, dictID); pos+=4; break; + } + switch(fcsCode) + { + default: assert(0); /* impossible */ + case 0 : if (singleSegment) op[pos++] = (BYTE)(pledgedSrcSize); break; + case 1 : MEM_writeLE16(op+pos, (U16)(pledgedSrcSize-256)); pos+=2; break; + case 2 : MEM_writeLE32(op+pos, (U32)(pledgedSrcSize)); pos+=4; break; + case 3 : MEM_writeLE64(op+pos, (U64)(pledgedSrcSize)); pos+=8; break; + } + return pos; +} + +/* ZSTD_writeLastEmptyBlock() : + * output an empty Block with end-of-frame mark to complete a frame + * @return : size of data written into `dst` (== ZSTD_blockHeaderSize (defined in zstd_internal.h)) + * or an error code if `dstCapacity` is too small (stage != ZSTDcs_init, stage_wrong, + "wrong cctx stage"); + RETURN_ERROR_IF(cctx->appliedParams.ldmParams.enableLdm, + parameter_unsupported, + "incompatible with ldm"); + cctx->externSeqStore.seq = seq; + cctx->externSeqStore.size = nbSeq; + cctx->externSeqStore.capacity = nbSeq; + cctx->externSeqStore.pos = 0; + return 0; +} + + +static size_t ZSTD_compressContinue_internal (ZSTD_CCtx* cctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + U32 frame, U32 lastFrameChunk) +{ + ZSTD_matchState_t* const ms = &cctx->blockState.matchState; + size_t fhSize = 0; + + DEBUGLOG(5, "ZSTD_compressContinue_internal, stage: %u, srcSize: %u", + cctx->stage, (unsigned)srcSize); + RETURN_ERROR_IF(cctx->stage==ZSTDcs_created, stage_wrong, + "missing init (ZSTD_compressBegin)"); + + if (frame && (cctx->stage==ZSTDcs_init)) { + fhSize = ZSTD_writeFrameHeader(dst, dstCapacity, &cctx->appliedParams, + cctx->pledgedSrcSizePlusOne-1, cctx->dictID); + FORWARD_IF_ERROR(fhSize, "ZSTD_writeFrameHeader failed"); + assert(fhSize <= dstCapacity); + dstCapacity -= fhSize; + dst = (char*)dst + fhSize; + cctx->stage = ZSTDcs_ongoing; + } + + if (!srcSize) return fhSize; /* do not generate an empty block if no input */ + + if (!ZSTD_window_update(&ms->window, src, srcSize)) { + ms->nextToUpdate = ms->window.dictLimit; + } + if (cctx->appliedParams.ldmParams.enableLdm) { + ZSTD_window_update(&cctx->ldmState.window, src, srcSize); + } + + if (!frame) { + /* overflow check and correction for block mode */ + ZSTD_overflowCorrectIfNeeded( + ms, &cctx->workspace, &cctx->appliedParams, + src, (BYTE const*)src + srcSize); + } + + DEBUGLOG(5, "ZSTD_compressContinue_internal (blockSize=%u)", (unsigned)cctx->blockSize); + { size_t const cSize = frame ? + ZSTD_compress_frameChunk (cctx, dst, dstCapacity, src, srcSize, lastFrameChunk) : + ZSTD_compressBlock_internal (cctx, dst, dstCapacity, src, srcSize, 0 /* frame */); + FORWARD_IF_ERROR(cSize, "%s", frame ? "ZSTD_compress_frameChunk failed" : "ZSTD_compressBlock_internal failed"); + cctx->consumedSrcSize += srcSize; + cctx->producedCSize += (cSize + fhSize); + assert(!(cctx->appliedParams.fParams.contentSizeFlag && cctx->pledgedSrcSizePlusOne == 0)); + if (cctx->pledgedSrcSizePlusOne != 0) { /* control src size */ + ZSTD_STATIC_ASSERT(ZSTD_CONTENTSIZE_UNKNOWN == (unsigned long long)-1); + RETURN_ERROR_IF( + cctx->consumedSrcSize+1 > cctx->pledgedSrcSizePlusOne, + srcSize_wrong, + "error : pledgedSrcSize = %u, while realSrcSize >= %u", + (unsigned)cctx->pledgedSrcSizePlusOne-1, + (unsigned)cctx->consumedSrcSize); + } + return cSize + fhSize; + } +} + +size_t ZSTD_compressContinue (ZSTD_CCtx* cctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize) +{ + DEBUGLOG(5, "ZSTD_compressContinue (srcSize=%u)", (unsigned)srcSize); + return ZSTD_compressContinue_internal(cctx, dst, dstCapacity, src, srcSize, 1 /* frame mode */, 0 /* last chunk */); +} + + +size_t ZSTD_getBlockSize(const ZSTD_CCtx* cctx) +{ + ZSTD_compressionParameters const cParams = cctx->appliedParams.cParams; + assert(!ZSTD_checkCParams(cParams)); + return MIN (ZSTD_BLOCKSIZE_MAX, (U32)1 << cParams.windowLog); +} + +size_t ZSTD_compressBlock(ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize) +{ + DEBUGLOG(5, "ZSTD_compressBlock: srcSize = %u", (unsigned)srcSize); + { size_t const blockSizeMax = ZSTD_getBlockSize(cctx); + RETURN_ERROR_IF(srcSize > blockSizeMax, srcSize_wrong, "input is larger than a block"); } + + return ZSTD_compressContinue_internal(cctx, dst, dstCapacity, src, srcSize, 0 /* frame mode */, 0 /* last chunk */); +} + +/*! ZSTD_loadDictionaryContent() : + * @return : 0, or an error code + */ +static size_t ZSTD_loadDictionaryContent(ZSTD_matchState_t* ms, + ldmState_t* ls, + ZSTD_cwksp* ws, + ZSTD_CCtx_params const* params, + const void* src, size_t srcSize, + ZSTD_dictTableLoadMethod_e dtlm) +{ + const BYTE* ip = (const BYTE*) src; + const BYTE* const iend = ip + srcSize; + + ZSTD_window_update(&ms->window, src, srcSize); + ms->loadedDictEnd = params->forceWindow ? 0 : (U32)(iend - ms->window.base); + + if (params->ldmParams.enableLdm && ls != NULL) { + ZSTD_window_update(&ls->window, src, srcSize); + ls->loadedDictEnd = params->forceWindow ? 0 : (U32)(iend - ls->window.base); + } + + /* Assert that we the ms params match the params we're being given */ + ZSTD_assertEqualCParams(params->cParams, ms->cParams); + + if (srcSize <= HASH_READ_SIZE) return 0; + + while (iend - ip > HASH_READ_SIZE) { + size_t const remaining = (size_t)(iend - ip); + size_t const chunk = MIN(remaining, ZSTD_CHUNKSIZE_MAX); + const BYTE* const ichunk = ip + chunk; + + ZSTD_overflowCorrectIfNeeded(ms, ws, params, ip, ichunk); + + if (params->ldmParams.enableLdm && ls != NULL) + ZSTD_ldm_fillHashTable(ls, (const BYTE*)src, (const BYTE*)src + srcSize, ¶ms->ldmParams); + + switch(params->cParams.strategy) + { + case ZSTD_fast: + ZSTD_fillHashTable(ms, ichunk, dtlm); + break; + case ZSTD_dfast: + ZSTD_fillDoubleHashTable(ms, ichunk, dtlm); + break; + + case ZSTD_greedy: + case ZSTD_lazy: + case ZSTD_lazy2: + if (chunk >= HASH_READ_SIZE) + ZSTD_insertAndFindFirstIndex(ms, ichunk-HASH_READ_SIZE); + break; + + case ZSTD_btlazy2: /* we want the dictionary table fully sorted */ + case ZSTD_btopt: + case ZSTD_btultra: + case ZSTD_btultra2: + if (chunk >= HASH_READ_SIZE) + ZSTD_updateTree(ms, ichunk-HASH_READ_SIZE, ichunk); + break; + + default: + assert(0); /* not possible : not a valid strategy id */ + } + + ip = ichunk; + } + + ms->nextToUpdate = (U32)(iend - ms->window.base); + return 0; +} + + +/* Dictionaries that assign zero probability to symbols that show up causes problems + when FSE encoding. Refuse dictionaries that assign zero probability to symbols + that we may encounter during compression. + NOTE: This behavior is not standard and could be improved in the future. */ +static size_t ZSTD_checkDictNCount(short* normalizedCounter, unsigned dictMaxSymbolValue, unsigned maxSymbolValue) { + U32 s; + RETURN_ERROR_IF(dictMaxSymbolValue < maxSymbolValue, dictionary_corrupted, "dict fse tables don't have all symbols"); + for (s = 0; s <= maxSymbolValue; ++s) { + RETURN_ERROR_IF(normalizedCounter[s] == 0, dictionary_corrupted, "dict fse tables don't have all symbols"); + } + return 0; +} + +size_t ZSTD_loadCEntropy(ZSTD_compressedBlockState_t* bs, void* workspace, + short* offcodeNCount, unsigned* offcodeMaxValue, + const void* const dict, size_t dictSize) +{ + const BYTE* dictPtr = (const BYTE*)dict; /* skip magic num and dict ID */ + const BYTE* const dictEnd = dictPtr + dictSize; + dictPtr += 8; + bs->entropy.huf.repeatMode = HUF_repeat_check; + + { unsigned maxSymbolValue = 255; + unsigned hasZeroWeights = 1; + size_t const hufHeaderSize = HUF_readCTable((HUF_CElt*)bs->entropy.huf.CTable, &maxSymbolValue, dictPtr, + dictEnd-dictPtr, &hasZeroWeights); + + /* We only set the loaded table as valid if it contains all non-zero + * weights. Otherwise, we set it to check */ + if (!hasZeroWeights) + bs->entropy.huf.repeatMode = HUF_repeat_valid; + + RETURN_ERROR_IF(HUF_isError(hufHeaderSize), dictionary_corrupted, ""); + RETURN_ERROR_IF(maxSymbolValue < 255, dictionary_corrupted, ""); + dictPtr += hufHeaderSize; + } + + { unsigned offcodeLog; + size_t const offcodeHeaderSize = FSE_readNCount(offcodeNCount, offcodeMaxValue, &offcodeLog, dictPtr, dictEnd-dictPtr); + RETURN_ERROR_IF(FSE_isError(offcodeHeaderSize), dictionary_corrupted, ""); + RETURN_ERROR_IF(offcodeLog > OffFSELog, dictionary_corrupted, ""); + /* Defer checking offcodeMaxValue because we need to know the size of the dictionary content */ + /* fill all offset symbols to avoid garbage at end of table */ + RETURN_ERROR_IF(FSE_isError(FSE_buildCTable_wksp( + bs->entropy.fse.offcodeCTable, + offcodeNCount, MaxOff, offcodeLog, + workspace, HUF_WORKSPACE_SIZE)), + dictionary_corrupted, ""); + dictPtr += offcodeHeaderSize; + } + + { short matchlengthNCount[MaxML+1]; + unsigned matchlengthMaxValue = MaxML, matchlengthLog; + size_t const matchlengthHeaderSize = FSE_readNCount(matchlengthNCount, &matchlengthMaxValue, &matchlengthLog, dictPtr, dictEnd-dictPtr); + RETURN_ERROR_IF(FSE_isError(matchlengthHeaderSize), dictionary_corrupted, ""); + RETURN_ERROR_IF(matchlengthLog > MLFSELog, dictionary_corrupted, ""); + /* Every match length code must have non-zero probability */ + FORWARD_IF_ERROR( ZSTD_checkDictNCount(matchlengthNCount, matchlengthMaxValue, MaxML), ""); + RETURN_ERROR_IF(FSE_isError(FSE_buildCTable_wksp( + bs->entropy.fse.matchlengthCTable, + matchlengthNCount, matchlengthMaxValue, matchlengthLog, + workspace, HUF_WORKSPACE_SIZE)), + dictionary_corrupted, ""); + dictPtr += matchlengthHeaderSize; + } + + { short litlengthNCount[MaxLL+1]; + unsigned litlengthMaxValue = MaxLL, litlengthLog; + size_t const litlengthHeaderSize = FSE_readNCount(litlengthNCount, &litlengthMaxValue, &litlengthLog, dictPtr, dictEnd-dictPtr); + RETURN_ERROR_IF(FSE_isError(litlengthHeaderSize), dictionary_corrupted, ""); + RETURN_ERROR_IF(litlengthLog > LLFSELog, dictionary_corrupted, ""); + /* Every literal length code must have non-zero probability */ + FORWARD_IF_ERROR( ZSTD_checkDictNCount(litlengthNCount, litlengthMaxValue, MaxLL), ""); + RETURN_ERROR_IF(FSE_isError(FSE_buildCTable_wksp( + bs->entropy.fse.litlengthCTable, + litlengthNCount, litlengthMaxValue, litlengthLog, + workspace, HUF_WORKSPACE_SIZE)), + dictionary_corrupted, ""); + dictPtr += litlengthHeaderSize; + } + + RETURN_ERROR_IF(dictPtr+12 > dictEnd, dictionary_corrupted, ""); + bs->rep[0] = MEM_readLE32(dictPtr+0); + bs->rep[1] = MEM_readLE32(dictPtr+4); + bs->rep[2] = MEM_readLE32(dictPtr+8); + dictPtr += 12; + + return dictPtr - (const BYTE*)dict; +} + +/* Dictionary format : + * See : + * https://github.com/facebook/zstd/blob/master/doc/zstd_compression_format.md#dictionary-format + */ +/*! ZSTD_loadZstdDictionary() : + * @return : dictID, or an error code + * assumptions : magic number supposed already checked + * dictSize supposed >= 8 + */ +static size_t ZSTD_loadZstdDictionary(ZSTD_compressedBlockState_t* bs, + ZSTD_matchState_t* ms, + ZSTD_cwksp* ws, + ZSTD_CCtx_params const* params, + const void* dict, size_t dictSize, + ZSTD_dictTableLoadMethod_e dtlm, + void* workspace) +{ + const BYTE* dictPtr = (const BYTE*)dict; + const BYTE* const dictEnd = dictPtr + dictSize; + short offcodeNCount[MaxOff+1]; + unsigned offcodeMaxValue = MaxOff; + size_t dictID; + size_t eSize; + + ZSTD_STATIC_ASSERT(HUF_WORKSPACE_SIZE >= (1<= 8); + assert(MEM_readLE32(dictPtr) == ZSTD_MAGIC_DICTIONARY); + + dictID = params->fParams.noDictIDFlag ? 0 : MEM_readLE32(dictPtr + 4 /* skip magic number */ ); + eSize = ZSTD_loadCEntropy(bs, workspace, offcodeNCount, &offcodeMaxValue, dict, dictSize); + FORWARD_IF_ERROR(eSize, "ZSTD_loadCEntropy failed"); + dictPtr += eSize; + + { size_t const dictContentSize = (size_t)(dictEnd - dictPtr); + U32 offcodeMax = MaxOff; + if (dictContentSize <= ((U32)-1) - 128 KB) { + U32 const maxOffset = (U32)dictContentSize + 128 KB; /* The maximum offset that must be supported */ + offcodeMax = ZSTD_highbit32(maxOffset); /* Calculate minimum offset code required to represent maxOffset */ + } + /* All offset values <= dictContentSize + 128 KB must be representable */ + FORWARD_IF_ERROR(ZSTD_checkDictNCount(offcodeNCount, offcodeMaxValue, MIN(offcodeMax, MaxOff)), ""); + /* All repCodes must be <= dictContentSize and != 0*/ + { U32 u; + for (u=0; u<3; u++) { + RETURN_ERROR_IF(bs->rep[u] == 0, dictionary_corrupted, ""); + RETURN_ERROR_IF(bs->rep[u] > dictContentSize, dictionary_corrupted, ""); + } } + + bs->entropy.fse.offcode_repeatMode = FSE_repeat_valid; + bs->entropy.fse.matchlength_repeatMode = FSE_repeat_valid; + bs->entropy.fse.litlength_repeatMode = FSE_repeat_valid; + FORWARD_IF_ERROR(ZSTD_loadDictionaryContent( + ms, NULL, ws, params, dictPtr, dictContentSize, dtlm), ""); + return dictID; + } +} + +/** ZSTD_compress_insertDictionary() : +* @return : dictID, or an error code */ +static size_t +ZSTD_compress_insertDictionary(ZSTD_compressedBlockState_t* bs, + ZSTD_matchState_t* ms, + ldmState_t* ls, + ZSTD_cwksp* ws, + const ZSTD_CCtx_params* params, + const void* dict, size_t dictSize, + ZSTD_dictContentType_e dictContentType, + ZSTD_dictTableLoadMethod_e dtlm, + void* workspace) +{ + DEBUGLOG(4, "ZSTD_compress_insertDictionary (dictSize=%u)", (U32)dictSize); + if ((dict==NULL) || (dictSize<8)) { + RETURN_ERROR_IF(dictContentType == ZSTD_dct_fullDict, dictionary_wrong, ""); + return 0; + } + + ZSTD_reset_compressedBlockState(bs); + + /* dict restricted modes */ + if (dictContentType == ZSTD_dct_rawContent) + return ZSTD_loadDictionaryContent(ms, ls, ws, params, dict, dictSize, dtlm); + + if (MEM_readLE32(dict) != ZSTD_MAGIC_DICTIONARY) { + if (dictContentType == ZSTD_dct_auto) { + DEBUGLOG(4, "raw content dictionary detected"); + return ZSTD_loadDictionaryContent( + ms, ls, ws, params, dict, dictSize, dtlm); + } + RETURN_ERROR_IF(dictContentType == ZSTD_dct_fullDict, dictionary_wrong, ""); + assert(0); /* impossible */ + } + + /* dict as full zstd dictionary */ + return ZSTD_loadZstdDictionary( + bs, ms, ws, params, dict, dictSize, dtlm, workspace); +} + +#define ZSTD_USE_CDICT_PARAMS_SRCSIZE_CUTOFF (128 KB) +#define ZSTD_USE_CDICT_PARAMS_DICTSIZE_MULTIPLIER (6) + +/*! ZSTD_compressBegin_internal() : + * @return : 0, or an error code */ +static size_t ZSTD_compressBegin_internal(ZSTD_CCtx* cctx, + const void* dict, size_t dictSize, + ZSTD_dictContentType_e dictContentType, + ZSTD_dictTableLoadMethod_e dtlm, + const ZSTD_CDict* cdict, + const ZSTD_CCtx_params* params, U64 pledgedSrcSize, + ZSTD_buffered_policy_e zbuff) +{ + DEBUGLOG(4, "ZSTD_compressBegin_internal: wlog=%u", params->cParams.windowLog); + /* params are supposed to be fully validated at this point */ + assert(!ZSTD_isError(ZSTD_checkCParams(params->cParams))); + assert(!((dict) && (cdict))); /* either dict or cdict, not both */ + if ( (cdict) + && (cdict->dictContentSize > 0) + && ( pledgedSrcSize < ZSTD_USE_CDICT_PARAMS_SRCSIZE_CUTOFF + || pledgedSrcSize < cdict->dictContentSize * ZSTD_USE_CDICT_PARAMS_DICTSIZE_MULTIPLIER + || pledgedSrcSize == ZSTD_CONTENTSIZE_UNKNOWN + || cdict->compressionLevel == 0) + && (params->attachDictPref != ZSTD_dictForceLoad) ) { + return ZSTD_resetCCtx_usingCDict(cctx, cdict, params, pledgedSrcSize, zbuff); + } + + FORWARD_IF_ERROR( ZSTD_resetCCtx_internal(cctx, *params, pledgedSrcSize, + ZSTDcrp_makeClean, zbuff) , ""); + { size_t const dictID = cdict ? + ZSTD_compress_insertDictionary( + cctx->blockState.prevCBlock, &cctx->blockState.matchState, + &cctx->ldmState, &cctx->workspace, &cctx->appliedParams, cdict->dictContent, + cdict->dictContentSize, dictContentType, dtlm, + cctx->entropyWorkspace) + : ZSTD_compress_insertDictionary( + cctx->blockState.prevCBlock, &cctx->blockState.matchState, + &cctx->ldmState, &cctx->workspace, &cctx->appliedParams, dict, dictSize, + dictContentType, dtlm, cctx->entropyWorkspace); + FORWARD_IF_ERROR(dictID, "ZSTD_compress_insertDictionary failed"); + assert(dictID <= UINT_MAX); + cctx->dictID = (U32)dictID; + } + return 0; +} + +size_t ZSTD_compressBegin_advanced_internal(ZSTD_CCtx* cctx, + const void* dict, size_t dictSize, + ZSTD_dictContentType_e dictContentType, + ZSTD_dictTableLoadMethod_e dtlm, + const ZSTD_CDict* cdict, + const ZSTD_CCtx_params* params, + unsigned long long pledgedSrcSize) +{ + DEBUGLOG(4, "ZSTD_compressBegin_advanced_internal: wlog=%u", params->cParams.windowLog); + /* compression parameters verification and optimization */ + FORWARD_IF_ERROR( ZSTD_checkCParams(params->cParams) , ""); + return ZSTD_compressBegin_internal(cctx, + dict, dictSize, dictContentType, dtlm, + cdict, + params, pledgedSrcSize, + ZSTDb_not_buffered); +} + +/*! ZSTD_compressBegin_advanced() : +* @return : 0, or an error code */ +size_t ZSTD_compressBegin_advanced(ZSTD_CCtx* cctx, + const void* dict, size_t dictSize, + ZSTD_parameters params, unsigned long long pledgedSrcSize) +{ + ZSTD_CCtx_params const cctxParams = + ZSTD_assignParamsToCCtxParams(&cctx->requestedParams, ¶ms); + return ZSTD_compressBegin_advanced_internal(cctx, + dict, dictSize, ZSTD_dct_auto, ZSTD_dtlm_fast, + NULL /*cdict*/, + &cctxParams, pledgedSrcSize); +} + +size_t ZSTD_compressBegin_usingDict(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, int compressionLevel) +{ + ZSTD_parameters const params = ZSTD_getParams_internal(compressionLevel, ZSTD_CONTENTSIZE_UNKNOWN, dictSize); + ZSTD_CCtx_params const cctxParams = + ZSTD_assignParamsToCCtxParams(&cctx->requestedParams, ¶ms); + DEBUGLOG(4, "ZSTD_compressBegin_usingDict (dictSize=%u)", (unsigned)dictSize); + return ZSTD_compressBegin_internal(cctx, dict, dictSize, ZSTD_dct_auto, ZSTD_dtlm_fast, NULL, + &cctxParams, ZSTD_CONTENTSIZE_UNKNOWN, ZSTDb_not_buffered); +} + +size_t ZSTD_compressBegin(ZSTD_CCtx* cctx, int compressionLevel) +{ + return ZSTD_compressBegin_usingDict(cctx, NULL, 0, compressionLevel); +} + + +/*! ZSTD_writeEpilogue() : +* Ends a frame. +* @return : nb of bytes written into dst (or an error code) */ +static size_t ZSTD_writeEpilogue(ZSTD_CCtx* cctx, void* dst, size_t dstCapacity) +{ + BYTE* const ostart = (BYTE*)dst; + BYTE* op = ostart; + size_t fhSize = 0; + + DEBUGLOG(4, "ZSTD_writeEpilogue"); + RETURN_ERROR_IF(cctx->stage == ZSTDcs_created, stage_wrong, "init missing"); + + /* special case : empty frame */ + if (cctx->stage == ZSTDcs_init) { + fhSize = ZSTD_writeFrameHeader(dst, dstCapacity, &cctx->appliedParams, 0, 0); + FORWARD_IF_ERROR(fhSize, "ZSTD_writeFrameHeader failed"); + dstCapacity -= fhSize; + op += fhSize; + cctx->stage = ZSTDcs_ongoing; + } + + if (cctx->stage != ZSTDcs_ending) { + /* write one last empty block, make it the "last" block */ + U32 const cBlockHeader24 = 1 /* last block */ + (((U32)bt_raw)<<1) + 0; + RETURN_ERROR_IF(dstCapacity<4, dstSize_tooSmall, "no room for epilogue"); + MEM_writeLE32(op, cBlockHeader24); + op += ZSTD_blockHeaderSize; + dstCapacity -= ZSTD_blockHeaderSize; + } + + if (cctx->appliedParams.fParams.checksumFlag) { + U32 const checksum = (U32) XXH64_digest(&cctx->xxhState); + RETURN_ERROR_IF(dstCapacity<4, dstSize_tooSmall, "no room for checksum"); + DEBUGLOG(4, "ZSTD_writeEpilogue: write checksum : %08X", (unsigned)checksum); + MEM_writeLE32(op, checksum); + op += 4; + } + + cctx->stage = ZSTDcs_created; /* return to "created but no init" status */ + return op-ostart; +} + +size_t ZSTD_compressEnd (ZSTD_CCtx* cctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize) +{ + size_t endResult; + size_t const cSize = ZSTD_compressContinue_internal(cctx, + dst, dstCapacity, src, srcSize, + 1 /* frame mode */, 1 /* last chunk */); + FORWARD_IF_ERROR(cSize, "ZSTD_compressContinue_internal failed"); + endResult = ZSTD_writeEpilogue(cctx, (char*)dst + cSize, dstCapacity-cSize); + FORWARD_IF_ERROR(endResult, "ZSTD_writeEpilogue failed"); + assert(!(cctx->appliedParams.fParams.contentSizeFlag && cctx->pledgedSrcSizePlusOne == 0)); + if (cctx->pledgedSrcSizePlusOne != 0) { /* control src size */ + ZSTD_STATIC_ASSERT(ZSTD_CONTENTSIZE_UNKNOWN == (unsigned long long)-1); + DEBUGLOG(4, "end of frame : controlling src size"); + RETURN_ERROR_IF( + cctx->pledgedSrcSizePlusOne != cctx->consumedSrcSize+1, + srcSize_wrong, + "error : pledgedSrcSize = %u, while realSrcSize = %u", + (unsigned)cctx->pledgedSrcSizePlusOne-1, + (unsigned)cctx->consumedSrcSize); + } + return cSize + endResult; +} + + +static size_t ZSTD_compress_internal (ZSTD_CCtx* cctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + const void* dict,size_t dictSize, + const ZSTD_parameters* params) +{ + ZSTD_CCtx_params const cctxParams = + ZSTD_assignParamsToCCtxParams(&cctx->requestedParams, params); + DEBUGLOG(4, "ZSTD_compress_internal"); + return ZSTD_compress_advanced_internal(cctx, + dst, dstCapacity, + src, srcSize, + dict, dictSize, + &cctxParams); +} + +size_t ZSTD_compress_advanced (ZSTD_CCtx* cctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + const void* dict,size_t dictSize, + ZSTD_parameters params) +{ + DEBUGLOG(4, "ZSTD_compress_advanced"); + FORWARD_IF_ERROR(ZSTD_checkCParams(params.cParams), ""); + return ZSTD_compress_internal(cctx, + dst, dstCapacity, + src, srcSize, + dict, dictSize, + ¶ms); +} + +/* Internal */ +size_t ZSTD_compress_advanced_internal( + ZSTD_CCtx* cctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + const void* dict,size_t dictSize, + const ZSTD_CCtx_params* params) +{ + DEBUGLOG(4, "ZSTD_compress_advanced_internal (srcSize:%u)", (unsigned)srcSize); + FORWARD_IF_ERROR( ZSTD_compressBegin_internal(cctx, + dict, dictSize, ZSTD_dct_auto, ZSTD_dtlm_fast, NULL, + params, srcSize, ZSTDb_not_buffered) , ""); + return ZSTD_compressEnd(cctx, dst, dstCapacity, src, srcSize); +} + +size_t ZSTD_compress_usingDict(ZSTD_CCtx* cctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + const void* dict, size_t dictSize, + int compressionLevel) +{ + ZSTD_parameters const params = ZSTD_getParams_internal(compressionLevel, srcSize, dict ? dictSize : 0); + ZSTD_CCtx_params cctxParams = ZSTD_assignParamsToCCtxParams(&cctx->requestedParams, ¶ms); + DEBUGLOG(4, "ZSTD_compress_usingDict (srcSize=%u)", (unsigned)srcSize); + assert(params.fParams.contentSizeFlag == 1); + return ZSTD_compress_advanced_internal(cctx, dst, dstCapacity, src, srcSize, dict, dictSize, &cctxParams); +} + +size_t ZSTD_compressCCtx(ZSTD_CCtx* cctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + int compressionLevel) +{ + DEBUGLOG(4, "ZSTD_compressCCtx (srcSize=%u)", (unsigned)srcSize); + assert(cctx != NULL); + return ZSTD_compress_usingDict(cctx, dst, dstCapacity, src, srcSize, NULL, 0, compressionLevel); +} + +size_t ZSTD_compress(void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + int compressionLevel) +{ + size_t result; + ZSTD_CCtx ctxBody; + ZSTD_initCCtx(&ctxBody, ZSTD_defaultCMem); + result = ZSTD_compressCCtx(&ctxBody, dst, dstCapacity, src, srcSize, compressionLevel); + ZSTD_freeCCtxContent(&ctxBody); /* can't free ctxBody itself, as it's on stack; free only heap content */ + return result; +} + + +/* ===== Dictionary API ===== */ + +/*! ZSTD_estimateCDictSize_advanced() : + * Estimate amount of memory that will be needed to create a dictionary with following arguments */ +size_t ZSTD_estimateCDictSize_advanced( + size_t dictSize, ZSTD_compressionParameters cParams, + ZSTD_dictLoadMethod_e dictLoadMethod) +{ + DEBUGLOG(5, "sizeof(ZSTD_CDict) : %u", (unsigned)sizeof(ZSTD_CDict)); + return ZSTD_cwksp_alloc_size(sizeof(ZSTD_CDict)) + + ZSTD_cwksp_alloc_size(HUF_WORKSPACE_SIZE) + + ZSTD_sizeof_matchState(&cParams, /* forCCtx */ 0) + + (dictLoadMethod == ZSTD_dlm_byRef ? 0 + : ZSTD_cwksp_alloc_size(ZSTD_cwksp_align(dictSize, sizeof(void *)))); +} + +size_t ZSTD_estimateCDictSize(size_t dictSize, int compressionLevel) +{ + ZSTD_compressionParameters const cParams = ZSTD_getCParams_internal(compressionLevel, ZSTD_CONTENTSIZE_UNKNOWN, dictSize); + return ZSTD_estimateCDictSize_advanced(dictSize, cParams, ZSTD_dlm_byCopy); +} + +size_t ZSTD_sizeof_CDict(const ZSTD_CDict* cdict) +{ + if (cdict==NULL) return 0; /* support sizeof on NULL */ + DEBUGLOG(5, "sizeof(*cdict) : %u", (unsigned)sizeof(*cdict)); + /* cdict may be in the workspace */ + return (cdict->workspace.workspace == cdict ? 0 : sizeof(*cdict)) + + ZSTD_cwksp_sizeof(&cdict->workspace); +} + +static size_t ZSTD_initCDict_internal( + ZSTD_CDict* cdict, + const void* dictBuffer, size_t dictSize, + ZSTD_dictLoadMethod_e dictLoadMethod, + ZSTD_dictContentType_e dictContentType, + ZSTD_compressionParameters cParams) +{ + DEBUGLOG(3, "ZSTD_initCDict_internal (dictContentType:%u)", (unsigned)dictContentType); + assert(!ZSTD_checkCParams(cParams)); + cdict->matchState.cParams = cParams; + if ((dictLoadMethod == ZSTD_dlm_byRef) || (!dictBuffer) || (!dictSize)) { + cdict->dictContent = dictBuffer; + } else { + void *internalBuffer = ZSTD_cwksp_reserve_object(&cdict->workspace, ZSTD_cwksp_align(dictSize, sizeof(void*))); + RETURN_ERROR_IF(!internalBuffer, memory_allocation, "NULL pointer!"); + cdict->dictContent = internalBuffer; + memcpy(internalBuffer, dictBuffer, dictSize); + } + cdict->dictContentSize = dictSize; + + cdict->entropyWorkspace = (U32*)ZSTD_cwksp_reserve_object(&cdict->workspace, HUF_WORKSPACE_SIZE); + + + /* Reset the state to no dictionary */ + ZSTD_reset_compressedBlockState(&cdict->cBlockState); + FORWARD_IF_ERROR(ZSTD_reset_matchState( + &cdict->matchState, + &cdict->workspace, + &cParams, + ZSTDcrp_makeClean, + ZSTDirp_reset, + ZSTD_resetTarget_CDict), ""); + /* (Maybe) load the dictionary + * Skips loading the dictionary if it is < 8 bytes. + */ + { ZSTD_CCtx_params params; + memset(¶ms, 0, sizeof(params)); + params.compressionLevel = ZSTD_CLEVEL_DEFAULT; + params.fParams.contentSizeFlag = 1; + params.cParams = cParams; + { size_t const dictID = ZSTD_compress_insertDictionary( + &cdict->cBlockState, &cdict->matchState, NULL, &cdict->workspace, + ¶ms, cdict->dictContent, cdict->dictContentSize, + dictContentType, ZSTD_dtlm_full, cdict->entropyWorkspace); + FORWARD_IF_ERROR(dictID, "ZSTD_compress_insertDictionary failed"); + assert(dictID <= (size_t)(U32)-1); + cdict->dictID = (U32)dictID; + } + } + + return 0; +} + +ZSTD_CDict* ZSTD_createCDict_advanced(const void* dictBuffer, size_t dictSize, + ZSTD_dictLoadMethod_e dictLoadMethod, + ZSTD_dictContentType_e dictContentType, + ZSTD_compressionParameters cParams, ZSTD_customMem customMem) +{ + DEBUGLOG(3, "ZSTD_createCDict_advanced, mode %u", (unsigned)dictContentType); + if (!customMem.customAlloc ^ !customMem.customFree) return NULL; + + { size_t const workspaceSize = + ZSTD_cwksp_alloc_size(sizeof(ZSTD_CDict)) + + ZSTD_cwksp_alloc_size(HUF_WORKSPACE_SIZE) + + ZSTD_sizeof_matchState(&cParams, /* forCCtx */ 0) + + (dictLoadMethod == ZSTD_dlm_byRef ? 0 + : ZSTD_cwksp_alloc_size(ZSTD_cwksp_align(dictSize, sizeof(void*)))); + void* const workspace = ZSTD_malloc(workspaceSize, customMem); + ZSTD_cwksp ws; + ZSTD_CDict* cdict; + + if (!workspace) { + ZSTD_free(workspace, customMem); + return NULL; + } + + ZSTD_cwksp_init(&ws, workspace, workspaceSize); + + cdict = (ZSTD_CDict*)ZSTD_cwksp_reserve_object(&ws, sizeof(ZSTD_CDict)); + assert(cdict != NULL); + ZSTD_cwksp_move(&cdict->workspace, &ws); + cdict->customMem = customMem; + cdict->compressionLevel = 0; /* signals advanced API usage */ + + if (ZSTD_isError( ZSTD_initCDict_internal(cdict, + dictBuffer, dictSize, + dictLoadMethod, dictContentType, + cParams) )) { + ZSTD_freeCDict(cdict); + return NULL; + } + + return cdict; + } +} + +ZSTD_CDict* ZSTD_createCDict(const void* dict, size_t dictSize, int compressionLevel) +{ + ZSTD_compressionParameters cParams = ZSTD_getCParams_internal(compressionLevel, ZSTD_CONTENTSIZE_UNKNOWN, dictSize); + ZSTD_CDict* cdict = ZSTD_createCDict_advanced(dict, dictSize, + ZSTD_dlm_byCopy, ZSTD_dct_auto, + cParams, ZSTD_defaultCMem); + if (cdict) + cdict->compressionLevel = compressionLevel == 0 ? ZSTD_CLEVEL_DEFAULT : compressionLevel; + return cdict; +} + +ZSTD_CDict* ZSTD_createCDict_byReference(const void* dict, size_t dictSize, int compressionLevel) +{ + ZSTD_compressionParameters cParams = ZSTD_getCParams_internal(compressionLevel, ZSTD_CONTENTSIZE_UNKNOWN, dictSize); + return ZSTD_createCDict_advanced(dict, dictSize, + ZSTD_dlm_byRef, ZSTD_dct_auto, + cParams, ZSTD_defaultCMem); +} + +size_t ZSTD_freeCDict(ZSTD_CDict* cdict) +{ + if (cdict==NULL) return 0; /* support free on NULL */ + { ZSTD_customMem const cMem = cdict->customMem; + int cdictInWorkspace = ZSTD_cwksp_owns_buffer(&cdict->workspace, cdict); + ZSTD_cwksp_free(&cdict->workspace, cMem); + if (!cdictInWorkspace) { + ZSTD_free(cdict, cMem); + } + return 0; + } +} + +/*! ZSTD_initStaticCDict_advanced() : + * Generate a digested dictionary in provided memory area. + * workspace: The memory area to emplace the dictionary into. + * Provided pointer must 8-bytes aligned. + * It must outlive dictionary usage. + * workspaceSize: Use ZSTD_estimateCDictSize() + * to determine how large workspace must be. + * cParams : use ZSTD_getCParams() to transform a compression level + * into its relevants cParams. + * @return : pointer to ZSTD_CDict*, or NULL if error (size too small) + * Note : there is no corresponding "free" function. + * Since workspace was allocated externally, it must be freed externally. + */ +const ZSTD_CDict* ZSTD_initStaticCDict( + void* workspace, size_t workspaceSize, + const void* dict, size_t dictSize, + ZSTD_dictLoadMethod_e dictLoadMethod, + ZSTD_dictContentType_e dictContentType, + ZSTD_compressionParameters cParams) +{ + size_t const matchStateSize = ZSTD_sizeof_matchState(&cParams, /* forCCtx */ 0); + size_t const neededSize = ZSTD_cwksp_alloc_size(sizeof(ZSTD_CDict)) + + (dictLoadMethod == ZSTD_dlm_byRef ? 0 + : ZSTD_cwksp_alloc_size(ZSTD_cwksp_align(dictSize, sizeof(void*)))) + + ZSTD_cwksp_alloc_size(HUF_WORKSPACE_SIZE) + + matchStateSize; + ZSTD_CDict* cdict; + + if ((size_t)workspace & 7) return NULL; /* 8-aligned */ + + { + ZSTD_cwksp ws; + ZSTD_cwksp_init(&ws, workspace, workspaceSize); + cdict = (ZSTD_CDict*)ZSTD_cwksp_reserve_object(&ws, sizeof(ZSTD_CDict)); + if (cdict == NULL) return NULL; + ZSTD_cwksp_move(&cdict->workspace, &ws); + } + + DEBUGLOG(4, "(workspaceSize < neededSize) : (%u < %u) => %u", + (unsigned)workspaceSize, (unsigned)neededSize, (unsigned)(workspaceSize < neededSize)); + if (workspaceSize < neededSize) return NULL; + + if (ZSTD_isError( ZSTD_initCDict_internal(cdict, + dict, dictSize, + dictLoadMethod, dictContentType, + cParams) )) + return NULL; + + return cdict; +} + +ZSTD_compressionParameters ZSTD_getCParamsFromCDict(const ZSTD_CDict* cdict) +{ + assert(cdict != NULL); + return cdict->matchState.cParams; +} + +/* ZSTD_compressBegin_usingCDict_advanced() : + * cdict must be != NULL */ +size_t ZSTD_compressBegin_usingCDict_advanced( + ZSTD_CCtx* const cctx, const ZSTD_CDict* const cdict, + ZSTD_frameParameters const fParams, unsigned long long const pledgedSrcSize) +{ + DEBUGLOG(4, "ZSTD_compressBegin_usingCDict_advanced"); + RETURN_ERROR_IF(cdict==NULL, dictionary_wrong, "NULL pointer!"); + { ZSTD_CCtx_params params = cctx->requestedParams; + params.cParams = ( pledgedSrcSize < ZSTD_USE_CDICT_PARAMS_SRCSIZE_CUTOFF + || pledgedSrcSize < cdict->dictContentSize * ZSTD_USE_CDICT_PARAMS_DICTSIZE_MULTIPLIER + || pledgedSrcSize == ZSTD_CONTENTSIZE_UNKNOWN + || cdict->compressionLevel == 0 ) + && (params.attachDictPref != ZSTD_dictForceLoad) ? + ZSTD_getCParamsFromCDict(cdict) + : ZSTD_getCParams(cdict->compressionLevel, + pledgedSrcSize, + cdict->dictContentSize); + /* Increase window log to fit the entire dictionary and source if the + * source size is known. Limit the increase to 19, which is the + * window log for compression level 1 with the largest source size. + */ + if (pledgedSrcSize != ZSTD_CONTENTSIZE_UNKNOWN) { + U32 const limitedSrcSize = (U32)MIN(pledgedSrcSize, 1U << 19); + U32 const limitedSrcLog = limitedSrcSize > 1 ? ZSTD_highbit32(limitedSrcSize - 1) + 1 : 1; + params.cParams.windowLog = MAX(params.cParams.windowLog, limitedSrcLog); + } + params.fParams = fParams; + return ZSTD_compressBegin_internal(cctx, + NULL, 0, ZSTD_dct_auto, ZSTD_dtlm_fast, + cdict, + ¶ms, pledgedSrcSize, + ZSTDb_not_buffered); + } +} + +/* ZSTD_compressBegin_usingCDict() : + * pledgedSrcSize=0 means "unknown" + * if pledgedSrcSize>0, it will enable contentSizeFlag */ +size_t ZSTD_compressBegin_usingCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict) +{ + ZSTD_frameParameters const fParams = { 0 /*content*/, 0 /*checksum*/, 0 /*noDictID*/ }; + DEBUGLOG(4, "ZSTD_compressBegin_usingCDict : dictIDFlag == %u", !fParams.noDictIDFlag); + return ZSTD_compressBegin_usingCDict_advanced(cctx, cdict, fParams, ZSTD_CONTENTSIZE_UNKNOWN); +} + +size_t ZSTD_compress_usingCDict_advanced(ZSTD_CCtx* cctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + const ZSTD_CDict* cdict, ZSTD_frameParameters fParams) +{ + FORWARD_IF_ERROR(ZSTD_compressBegin_usingCDict_advanced(cctx, cdict, fParams, srcSize), ""); /* will check if cdict != NULL */ + return ZSTD_compressEnd(cctx, dst, dstCapacity, src, srcSize); +} + +/*! ZSTD_compress_usingCDict() : + * Compression using a digested Dictionary. + * Faster startup than ZSTD_compress_usingDict(), recommended when same dictionary is used multiple times. + * Note that compression parameters are decided at CDict creation time + * while frame parameters are hardcoded */ +size_t ZSTD_compress_usingCDict(ZSTD_CCtx* cctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + const ZSTD_CDict* cdict) +{ + ZSTD_frameParameters const fParams = { 1 /*content*/, 0 /*checksum*/, 0 /*noDictID*/ }; + return ZSTD_compress_usingCDict_advanced(cctx, dst, dstCapacity, src, srcSize, cdict, fParams); +} + + + +/* ****************************************************************** +* Streaming +********************************************************************/ + +ZSTD_CStream* ZSTD_createCStream(void) +{ + DEBUGLOG(3, "ZSTD_createCStream"); + return ZSTD_createCStream_advanced(ZSTD_defaultCMem); +} + +ZSTD_CStream* ZSTD_initStaticCStream(void *workspace, size_t workspaceSize) +{ + return ZSTD_initStaticCCtx(workspace, workspaceSize); +} + +ZSTD_CStream* ZSTD_createCStream_advanced(ZSTD_customMem customMem) +{ /* CStream and CCtx are now same object */ + return ZSTD_createCCtx_advanced(customMem); +} + +size_t ZSTD_freeCStream(ZSTD_CStream* zcs) +{ + return ZSTD_freeCCtx(zcs); /* same object */ +} + + + +/*====== Initialization ======*/ + +size_t ZSTD_CStreamInSize(void) { return ZSTD_BLOCKSIZE_MAX; } + +size_t ZSTD_CStreamOutSize(void) +{ + return ZSTD_compressBound(ZSTD_BLOCKSIZE_MAX) + ZSTD_blockHeaderSize + 4 /* 32-bits hash */ ; +} + +static size_t ZSTD_resetCStream_internal(ZSTD_CStream* cctx, + const void* const dict, size_t const dictSize, ZSTD_dictContentType_e const dictContentType, + const ZSTD_CDict* const cdict, + ZSTD_CCtx_params params, unsigned long long const pledgedSrcSize) +{ + DEBUGLOG(4, "ZSTD_resetCStream_internal"); + /* Finalize the compression parameters */ + params.cParams = ZSTD_getCParamsFromCCtxParams(¶ms, pledgedSrcSize, dictSize); + /* params are supposed to be fully validated at this point */ + assert(!ZSTD_isError(ZSTD_checkCParams(params.cParams))); + assert(!((dict) && (cdict))); /* either dict or cdict, not both */ + + FORWARD_IF_ERROR( ZSTD_compressBegin_internal(cctx, + dict, dictSize, dictContentType, ZSTD_dtlm_fast, + cdict, + ¶ms, pledgedSrcSize, + ZSTDb_buffered) , ""); + + cctx->inToCompress = 0; + cctx->inBuffPos = 0; + cctx->inBuffTarget = cctx->blockSize + + (cctx->blockSize == pledgedSrcSize); /* for small input: avoid automatic flush on reaching end of block, since it would require to add a 3-bytes null block to end frame */ + cctx->outBuffContentSize = cctx->outBuffFlushedSize = 0; + cctx->streamStage = zcss_load; + cctx->frameEnded = 0; + return 0; /* ready to go */ +} + +/* ZSTD_resetCStream(): + * pledgedSrcSize == 0 means "unknown" */ +size_t ZSTD_resetCStream(ZSTD_CStream* zcs, unsigned long long pss) +{ + /* temporary : 0 interpreted as "unknown" during transition period. + * Users willing to specify "unknown" **must** use ZSTD_CONTENTSIZE_UNKNOWN. + * 0 will be interpreted as "empty" in the future. + */ + U64 const pledgedSrcSize = (pss==0) ? ZSTD_CONTENTSIZE_UNKNOWN : pss; + DEBUGLOG(4, "ZSTD_resetCStream: pledgedSrcSize = %u", (unsigned)pledgedSrcSize); + FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); + FORWARD_IF_ERROR( ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize) , ""); + return 0; +} + +/*! ZSTD_initCStream_internal() : + * Note : for lib/compress only. Used by zstdmt_compress.c. + * Assumption 1 : params are valid + * Assumption 2 : either dict, or cdict, is defined, not both */ +size_t ZSTD_initCStream_internal(ZSTD_CStream* zcs, + const void* dict, size_t dictSize, const ZSTD_CDict* cdict, + const ZSTD_CCtx_params* params, + unsigned long long pledgedSrcSize) +{ + DEBUGLOG(4, "ZSTD_initCStream_internal"); + FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); + FORWARD_IF_ERROR( ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize) , ""); + assert(!ZSTD_isError(ZSTD_checkCParams(params->cParams))); + zcs->requestedParams = *params; + assert(!((dict) && (cdict))); /* either dict or cdict, not both */ + if (dict) { + FORWARD_IF_ERROR( ZSTD_CCtx_loadDictionary(zcs, dict, dictSize) , ""); + } else { + /* Dictionary is cleared if !cdict */ + FORWARD_IF_ERROR( ZSTD_CCtx_refCDict(zcs, cdict) , ""); + } + return 0; +} + +/* ZSTD_initCStream_usingCDict_advanced() : + * same as ZSTD_initCStream_usingCDict(), with control over frame parameters */ +size_t ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs, + const ZSTD_CDict* cdict, + ZSTD_frameParameters fParams, + unsigned long long pledgedSrcSize) +{ + DEBUGLOG(4, "ZSTD_initCStream_usingCDict_advanced"); + FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); + FORWARD_IF_ERROR( ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize) , ""); + zcs->requestedParams.fParams = fParams; + FORWARD_IF_ERROR( ZSTD_CCtx_refCDict(zcs, cdict) , ""); + return 0; +} + +/* note : cdict must outlive compression session */ +size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict) +{ + DEBUGLOG(4, "ZSTD_initCStream_usingCDict"); + FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); + FORWARD_IF_ERROR( ZSTD_CCtx_refCDict(zcs, cdict) , ""); + return 0; +} + + +/* ZSTD_initCStream_advanced() : + * pledgedSrcSize must be exact. + * if srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. + * dict is loaded with default parameters ZSTD_dct_auto and ZSTD_dlm_byCopy. */ +size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs, + const void* dict, size_t dictSize, + ZSTD_parameters params, unsigned long long pss) +{ + /* for compatibility with older programs relying on this behavior. + * Users should now specify ZSTD_CONTENTSIZE_UNKNOWN. + * This line will be removed in the future. + */ + U64 const pledgedSrcSize = (pss==0 && params.fParams.contentSizeFlag==0) ? ZSTD_CONTENTSIZE_UNKNOWN : pss; + DEBUGLOG(4, "ZSTD_initCStream_advanced"); + FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); + FORWARD_IF_ERROR( ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize) , ""); + FORWARD_IF_ERROR( ZSTD_checkCParams(params.cParams) , ""); + zcs->requestedParams = ZSTD_assignParamsToCCtxParams(&zcs->requestedParams, ¶ms); + FORWARD_IF_ERROR( ZSTD_CCtx_loadDictionary(zcs, dict, dictSize) , ""); + return 0; +} + +size_t ZSTD_initCStream_usingDict(ZSTD_CStream* zcs, const void* dict, size_t dictSize, int compressionLevel) +{ + DEBUGLOG(4, "ZSTD_initCStream_usingDict"); + FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); + FORWARD_IF_ERROR( ZSTD_CCtx_setParameter(zcs, ZSTD_c_compressionLevel, compressionLevel) , ""); + FORWARD_IF_ERROR( ZSTD_CCtx_loadDictionary(zcs, dict, dictSize) , ""); + return 0; +} + +size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pss) +{ + /* temporary : 0 interpreted as "unknown" during transition period. + * Users willing to specify "unknown" **must** use ZSTD_CONTENTSIZE_UNKNOWN. + * 0 will be interpreted as "empty" in the future. + */ + U64 const pledgedSrcSize = (pss==0) ? ZSTD_CONTENTSIZE_UNKNOWN : pss; + DEBUGLOG(4, "ZSTD_initCStream_srcSize"); + FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); + FORWARD_IF_ERROR( ZSTD_CCtx_refCDict(zcs, NULL) , ""); + FORWARD_IF_ERROR( ZSTD_CCtx_setParameter(zcs, ZSTD_c_compressionLevel, compressionLevel) , ""); + FORWARD_IF_ERROR( ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize) , ""); + return 0; +} + +size_t ZSTD_initCStream(ZSTD_CStream* zcs, int compressionLevel) +{ + DEBUGLOG(4, "ZSTD_initCStream"); + FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); + FORWARD_IF_ERROR( ZSTD_CCtx_refCDict(zcs, NULL) , ""); + FORWARD_IF_ERROR( ZSTD_CCtx_setParameter(zcs, ZSTD_c_compressionLevel, compressionLevel) , ""); + return 0; +} + +/*====== Compression ======*/ + +static size_t ZSTD_nextInputSizeHint(const ZSTD_CCtx* cctx) +{ + size_t hintInSize = cctx->inBuffTarget - cctx->inBuffPos; + if (hintInSize==0) hintInSize = cctx->blockSize; + return hintInSize; +} + +/** ZSTD_compressStream_generic(): + * internal function for all *compressStream*() variants + * non-static, because can be called from zstdmt_compress.c + * @return : hint size for next input */ +static size_t ZSTD_compressStream_generic(ZSTD_CStream* zcs, + ZSTD_outBuffer* output, + ZSTD_inBuffer* input, + ZSTD_EndDirective const flushMode) +{ + const char* const istart = (const char*)input->src; + const char* const iend = input->size != 0 ? istart + input->size : istart; + const char* ip = input->pos != 0 ? istart + input->pos : istart; + char* const ostart = (char*)output->dst; + char* const oend = output->size != 0 ? ostart + output->size : ostart; + char* op = output->pos != 0 ? ostart + output->pos : ostart; + U32 someMoreWork = 1; + + /* check expectations */ + DEBUGLOG(5, "ZSTD_compressStream_generic, flush=%u", (unsigned)flushMode); + assert(zcs->inBuff != NULL); + assert(zcs->inBuffSize > 0); + assert(zcs->outBuff != NULL); + assert(zcs->outBuffSize > 0); + assert(output->pos <= output->size); + assert(input->pos <= input->size); + + while (someMoreWork) { + switch(zcs->streamStage) + { + case zcss_init: + RETURN_ERROR(init_missing, "call ZSTD_initCStream() first!"); + + case zcss_load: + if ( (flushMode == ZSTD_e_end) + && ((size_t)(oend-op) >= ZSTD_compressBound(iend-ip)) /* enough dstCapacity */ + && (zcs->inBuffPos == 0) ) { + /* shortcut to compression pass directly into output buffer */ + size_t const cSize = ZSTD_compressEnd(zcs, + op, oend-op, ip, iend-ip); + DEBUGLOG(4, "ZSTD_compressEnd : cSize=%u", (unsigned)cSize); + FORWARD_IF_ERROR(cSize, "ZSTD_compressEnd failed"); + ip = iend; + op += cSize; + zcs->frameEnded = 1; + ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only); + someMoreWork = 0; break; + } + /* complete loading into inBuffer */ + { size_t const toLoad = zcs->inBuffTarget - zcs->inBuffPos; + size_t const loaded = ZSTD_limitCopy( + zcs->inBuff + zcs->inBuffPos, toLoad, + ip, iend-ip); + zcs->inBuffPos += loaded; + if (loaded != 0) + ip += loaded; + if ( (flushMode == ZSTD_e_continue) + && (zcs->inBuffPos < zcs->inBuffTarget) ) { + /* not enough input to fill full block : stop here */ + someMoreWork = 0; break; + } + if ( (flushMode == ZSTD_e_flush) + && (zcs->inBuffPos == zcs->inToCompress) ) { + /* empty */ + someMoreWork = 0; break; + } + } + /* compress current block (note : this stage cannot be stopped in the middle) */ + DEBUGLOG(5, "stream compression stage (flushMode==%u)", flushMode); + { void* cDst; + size_t cSize; + size_t const iSize = zcs->inBuffPos - zcs->inToCompress; + size_t oSize = oend-op; + unsigned const lastBlock = (flushMode == ZSTD_e_end) && (ip==iend); + if (oSize >= ZSTD_compressBound(iSize)) + cDst = op; /* compress into output buffer, to skip flush stage */ + else + cDst = zcs->outBuff, oSize = zcs->outBuffSize; + cSize = lastBlock ? + ZSTD_compressEnd(zcs, cDst, oSize, + zcs->inBuff + zcs->inToCompress, iSize) : + ZSTD_compressContinue(zcs, cDst, oSize, + zcs->inBuff + zcs->inToCompress, iSize); + FORWARD_IF_ERROR(cSize, "%s", lastBlock ? "ZSTD_compressEnd failed" : "ZSTD_compressContinue failed"); + zcs->frameEnded = lastBlock; + /* prepare next block */ + zcs->inBuffTarget = zcs->inBuffPos + zcs->blockSize; + if (zcs->inBuffTarget > zcs->inBuffSize) + zcs->inBuffPos = 0, zcs->inBuffTarget = zcs->blockSize; + DEBUGLOG(5, "inBuffTarget:%u / inBuffSize:%u", + (unsigned)zcs->inBuffTarget, (unsigned)zcs->inBuffSize); + if (!lastBlock) + assert(zcs->inBuffTarget <= zcs->inBuffSize); + zcs->inToCompress = zcs->inBuffPos; + if (cDst == op) { /* no need to flush */ + op += cSize; + if (zcs->frameEnded) { + DEBUGLOG(5, "Frame completed directly in outBuffer"); + someMoreWork = 0; + ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only); + } + break; + } + zcs->outBuffContentSize = cSize; + zcs->outBuffFlushedSize = 0; + zcs->streamStage = zcss_flush; /* pass-through to flush stage */ + } + /* fall-through */ + case zcss_flush: + DEBUGLOG(5, "flush stage"); + { size_t const toFlush = zcs->outBuffContentSize - zcs->outBuffFlushedSize; + size_t const flushed = ZSTD_limitCopy(op, (size_t)(oend-op), + zcs->outBuff + zcs->outBuffFlushedSize, toFlush); + DEBUGLOG(5, "toFlush: %u into %u ==> flushed: %u", + (unsigned)toFlush, (unsigned)(oend-op), (unsigned)flushed); + if (flushed) + op += flushed; + zcs->outBuffFlushedSize += flushed; + if (toFlush!=flushed) { + /* flush not fully completed, presumably because dst is too small */ + assert(op==oend); + someMoreWork = 0; + break; + } + zcs->outBuffContentSize = zcs->outBuffFlushedSize = 0; + if (zcs->frameEnded) { + DEBUGLOG(5, "Frame completed on flush"); + someMoreWork = 0; + ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only); + break; + } + zcs->streamStage = zcss_load; + break; + } + + default: /* impossible */ + assert(0); + } + } + + input->pos = ip - istart; + output->pos = op - ostart; + if (zcs->frameEnded) return 0; + return ZSTD_nextInputSizeHint(zcs); +} + +static size_t ZSTD_nextInputSizeHint_MTorST(const ZSTD_CCtx* cctx) +{ +#ifdef ZSTD_MULTITHREAD + if (cctx->appliedParams.nbWorkers >= 1) { + assert(cctx->mtctx != NULL); + return ZSTDMT_nextInputSizeHint(cctx->mtctx); + } +#endif + return ZSTD_nextInputSizeHint(cctx); + +} + +size_t ZSTD_compressStream(ZSTD_CStream* zcs, ZSTD_outBuffer* output, ZSTD_inBuffer* input) +{ + FORWARD_IF_ERROR( ZSTD_compressStream2(zcs, output, input, ZSTD_e_continue) , ""); + return ZSTD_nextInputSizeHint_MTorST(zcs); +} + + +size_t ZSTD_compressStream2( ZSTD_CCtx* cctx, + ZSTD_outBuffer* output, + ZSTD_inBuffer* input, + ZSTD_EndDirective endOp) +{ + DEBUGLOG(5, "ZSTD_compressStream2, endOp=%u ", (unsigned)endOp); + /* check conditions */ + RETURN_ERROR_IF(output->pos > output->size, GENERIC, "invalid buffer"); + RETURN_ERROR_IF(input->pos > input->size, GENERIC, "invalid buffer"); + assert(cctx!=NULL); + + /* transparent initialization stage */ + if (cctx->streamStage == zcss_init) { + ZSTD_CCtx_params params = cctx->requestedParams; + ZSTD_prefixDict const prefixDict = cctx->prefixDict; + FORWARD_IF_ERROR( ZSTD_initLocalDict(cctx) , ""); /* Init the local dict if present. */ + memset(&cctx->prefixDict, 0, sizeof(cctx->prefixDict)); /* single usage */ + assert(prefixDict.dict==NULL || cctx->cdict==NULL); /* only one can be set */ + DEBUGLOG(4, "ZSTD_compressStream2 : transparent init stage"); + if (endOp == ZSTD_e_end) cctx->pledgedSrcSizePlusOne = input->size + 1; /* auto-fix pledgedSrcSize */ + params.cParams = ZSTD_getCParamsFromCCtxParams( + &cctx->requestedParams, cctx->pledgedSrcSizePlusOne-1, 0 /*dictSize*/); + + +#ifdef ZSTD_MULTITHREAD + if ((cctx->pledgedSrcSizePlusOne-1) <= ZSTDMT_JOBSIZE_MIN) { + params.nbWorkers = 0; /* do not invoke multi-threading when src size is too small */ + } + if (params.nbWorkers > 0) { + /* mt context creation */ + if (cctx->mtctx == NULL) { + DEBUGLOG(4, "ZSTD_compressStream2: creating new mtctx for nbWorkers=%u", + params.nbWorkers); + cctx->mtctx = ZSTDMT_createCCtx_advanced((U32)params.nbWorkers, cctx->customMem); + RETURN_ERROR_IF(cctx->mtctx == NULL, memory_allocation, "NULL pointer!"); + } + /* mt compression */ + DEBUGLOG(4, "call ZSTDMT_initCStream_internal as nbWorkers=%u", params.nbWorkers); + FORWARD_IF_ERROR( ZSTDMT_initCStream_internal( + cctx->mtctx, + prefixDict.dict, prefixDict.dictSize, prefixDict.dictContentType, + cctx->cdict, params, cctx->pledgedSrcSizePlusOne-1) , ""); + cctx->streamStage = zcss_load; + cctx->appliedParams.nbWorkers = params.nbWorkers; + } else +#endif + { FORWARD_IF_ERROR( ZSTD_resetCStream_internal(cctx, + prefixDict.dict, prefixDict.dictSize, prefixDict.dictContentType, + cctx->cdict, + params, cctx->pledgedSrcSizePlusOne-1) , ""); + assert(cctx->streamStage == zcss_load); + assert(cctx->appliedParams.nbWorkers == 0); + } } + /* end of transparent initialization stage */ + + /* compression stage */ +#ifdef ZSTD_MULTITHREAD + if (cctx->appliedParams.nbWorkers > 0) { + int const forceMaxProgress = (endOp == ZSTD_e_flush || endOp == ZSTD_e_end); + size_t flushMin; + assert(forceMaxProgress || endOp == ZSTD_e_continue /* Protection for a new flush type */); + if (cctx->cParamsChanged) { + ZSTDMT_updateCParams_whileCompressing(cctx->mtctx, &cctx->requestedParams); + cctx->cParamsChanged = 0; + } + do { + flushMin = ZSTDMT_compressStream_generic(cctx->mtctx, output, input, endOp); + if ( ZSTD_isError(flushMin) + || (endOp == ZSTD_e_end && flushMin == 0) ) { /* compression completed */ + ZSTD_CCtx_reset(cctx, ZSTD_reset_session_only); + } + FORWARD_IF_ERROR(flushMin, "ZSTDMT_compressStream_generic failed"); + } while (forceMaxProgress && flushMin != 0 && output->pos < output->size); + DEBUGLOG(5, "completed ZSTD_compressStream2 delegating to ZSTDMT_compressStream_generic"); + /* Either we don't require maximum forward progress, we've finished the + * flush, or we are out of output space. + */ + assert(!forceMaxProgress || flushMin == 0 || output->pos == output->size); + return flushMin; + } +#endif + FORWARD_IF_ERROR( ZSTD_compressStream_generic(cctx, output, input, endOp) , ""); + DEBUGLOG(5, "completed ZSTD_compressStream2"); + return cctx->outBuffContentSize - cctx->outBuffFlushedSize; /* remaining to flush */ +} + +size_t ZSTD_compressStream2_simpleArgs ( + ZSTD_CCtx* cctx, + void* dst, size_t dstCapacity, size_t* dstPos, + const void* src, size_t srcSize, size_t* srcPos, + ZSTD_EndDirective endOp) +{ + ZSTD_outBuffer output = { dst, dstCapacity, *dstPos }; + ZSTD_inBuffer input = { src, srcSize, *srcPos }; + /* ZSTD_compressStream2() will check validity of dstPos and srcPos */ + size_t const cErr = ZSTD_compressStream2(cctx, &output, &input, endOp); + *dstPos = output.pos; + *srcPos = input.pos; + return cErr; +} + +size_t ZSTD_compress2(ZSTD_CCtx* cctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize) +{ + DEBUGLOG(4, "ZSTD_compress2 (srcSize=%u)", (unsigned)srcSize); + ZSTD_CCtx_reset(cctx, ZSTD_reset_session_only); + { size_t oPos = 0; + size_t iPos = 0; + size_t const result = ZSTD_compressStream2_simpleArgs(cctx, + dst, dstCapacity, &oPos, + src, srcSize, &iPos, + ZSTD_e_end); + FORWARD_IF_ERROR(result, "ZSTD_compressStream2_simpleArgs failed"); + if (result != 0) { /* compression not completed, due to lack of output space */ + assert(oPos == dstCapacity); + RETURN_ERROR(dstSize_tooSmall, ""); + } + assert(iPos == srcSize); /* all input is expected consumed */ + return oPos; + } +} + +/*====== Finalize ======*/ + +/*! ZSTD_flushStream() : + * @return : amount of data remaining to flush */ +size_t ZSTD_flushStream(ZSTD_CStream* zcs, ZSTD_outBuffer* output) +{ + ZSTD_inBuffer input = { NULL, 0, 0 }; + return ZSTD_compressStream2(zcs, output, &input, ZSTD_e_flush); +} + + +size_t ZSTD_endStream(ZSTD_CStream* zcs, ZSTD_outBuffer* output) +{ + ZSTD_inBuffer input = { NULL, 0, 0 }; + size_t const remainingToFlush = ZSTD_compressStream2(zcs, output, &input, ZSTD_e_end); + FORWARD_IF_ERROR( remainingToFlush , "ZSTD_compressStream2 failed"); + if (zcs->appliedParams.nbWorkers > 0) return remainingToFlush; /* minimal estimation */ + /* single thread mode : attempt to calculate remaining to flush more precisely */ + { size_t const lastBlockSize = zcs->frameEnded ? 0 : ZSTD_BLOCKHEADERSIZE; + size_t const checksumSize = (size_t)(zcs->frameEnded ? 0 : zcs->appliedParams.fParams.checksumFlag * 4); + size_t const toFlush = remainingToFlush + lastBlockSize + checksumSize; + DEBUGLOG(4, "ZSTD_endStream : remaining to flush : %u", (unsigned)toFlush); + return toFlush; + } +} + + +/*-===== Pre-defined compression levels =====-*/ + +#define ZSTD_MAX_CLEVEL 22 +int ZSTD_maxCLevel(void) { return ZSTD_MAX_CLEVEL; } +int ZSTD_minCLevel(void) { return (int)-ZSTD_TARGETLENGTH_MAX; } + +static const ZSTD_compressionParameters ZSTD_defaultCParameters[4][ZSTD_MAX_CLEVEL+1] = { +{ /* "default" - for any srcSize > 256 KB */ + /* W, C, H, S, L, TL, strat */ + { 19, 12, 13, 1, 6, 1, ZSTD_fast }, /* base for negative levels */ + { 19, 13, 14, 1, 7, 0, ZSTD_fast }, /* level 1 */ + { 20, 15, 16, 1, 6, 0, ZSTD_fast }, /* level 2 */ + { 21, 16, 17, 1, 5, 0, ZSTD_dfast }, /* level 3 */ + { 21, 18, 18, 1, 5, 0, ZSTD_dfast }, /* level 4 */ + { 21, 18, 19, 2, 5, 2, ZSTD_greedy }, /* level 5 */ + { 21, 19, 19, 3, 5, 4, ZSTD_greedy }, /* level 6 */ + { 21, 19, 19, 3, 5, 8, ZSTD_lazy }, /* level 7 */ + { 21, 19, 19, 3, 5, 16, ZSTD_lazy2 }, /* level 8 */ + { 21, 19, 20, 4, 5, 16, ZSTD_lazy2 }, /* level 9 */ + { 22, 20, 21, 4, 5, 16, ZSTD_lazy2 }, /* level 10 */ + { 22, 21, 22, 4, 5, 16, ZSTD_lazy2 }, /* level 11 */ + { 22, 21, 22, 5, 5, 16, ZSTD_lazy2 }, /* level 12 */ + { 22, 21, 22, 5, 5, 32, ZSTD_btlazy2 }, /* level 13 */ + { 22, 22, 23, 5, 5, 32, ZSTD_btlazy2 }, /* level 14 */ + { 22, 23, 23, 6, 5, 32, ZSTD_btlazy2 }, /* level 15 */ + { 22, 22, 22, 5, 5, 48, ZSTD_btopt }, /* level 16 */ + { 23, 23, 22, 5, 4, 64, ZSTD_btopt }, /* level 17 */ + { 23, 23, 22, 6, 3, 64, ZSTD_btultra }, /* level 18 */ + { 23, 24, 22, 7, 3,256, ZSTD_btultra2}, /* level 19 */ + { 25, 25, 23, 7, 3,256, ZSTD_btultra2}, /* level 20 */ + { 26, 26, 24, 7, 3,512, ZSTD_btultra2}, /* level 21 */ + { 27, 27, 25, 9, 3,999, ZSTD_btultra2}, /* level 22 */ +}, +{ /* for srcSize <= 256 KB */ + /* W, C, H, S, L, T, strat */ + { 18, 12, 13, 1, 5, 1, ZSTD_fast }, /* base for negative levels */ + { 18, 13, 14, 1, 6, 0, ZSTD_fast }, /* level 1 */ + { 18, 14, 14, 1, 5, 0, ZSTD_dfast }, /* level 2 */ + { 18, 16, 16, 1, 4, 0, ZSTD_dfast }, /* level 3 */ + { 18, 16, 17, 2, 5, 2, ZSTD_greedy }, /* level 4.*/ + { 18, 18, 18, 3, 5, 2, ZSTD_greedy }, /* level 5.*/ + { 18, 18, 19, 3, 5, 4, ZSTD_lazy }, /* level 6.*/ + { 18, 18, 19, 4, 4, 4, ZSTD_lazy }, /* level 7 */ + { 18, 18, 19, 4, 4, 8, ZSTD_lazy2 }, /* level 8 */ + { 18, 18, 19, 5, 4, 8, ZSTD_lazy2 }, /* level 9 */ + { 18, 18, 19, 6, 4, 8, ZSTD_lazy2 }, /* level 10 */ + { 18, 18, 19, 5, 4, 12, ZSTD_btlazy2 }, /* level 11.*/ + { 18, 19, 19, 7, 4, 12, ZSTD_btlazy2 }, /* level 12.*/ + { 18, 18, 19, 4, 4, 16, ZSTD_btopt }, /* level 13 */ + { 18, 18, 19, 4, 3, 32, ZSTD_btopt }, /* level 14.*/ + { 18, 18, 19, 6, 3,128, ZSTD_btopt }, /* level 15.*/ + { 18, 19, 19, 6, 3,128, ZSTD_btultra }, /* level 16.*/ + { 18, 19, 19, 8, 3,256, ZSTD_btultra }, /* level 17.*/ + { 18, 19, 19, 6, 3,128, ZSTD_btultra2}, /* level 18.*/ + { 18, 19, 19, 8, 3,256, ZSTD_btultra2}, /* level 19.*/ + { 18, 19, 19, 10, 3,512, ZSTD_btultra2}, /* level 20.*/ + { 18, 19, 19, 12, 3,512, ZSTD_btultra2}, /* level 21.*/ + { 18, 19, 19, 13, 3,999, ZSTD_btultra2}, /* level 22.*/ +}, +{ /* for srcSize <= 128 KB */ + /* W, C, H, S, L, T, strat */ + { 17, 12, 12, 1, 5, 1, ZSTD_fast }, /* base for negative levels */ + { 17, 12, 13, 1, 6, 0, ZSTD_fast }, /* level 1 */ + { 17, 13, 15, 1, 5, 0, ZSTD_fast }, /* level 2 */ + { 17, 15, 16, 2, 5, 0, ZSTD_dfast }, /* level 3 */ + { 17, 17, 17, 2, 4, 0, ZSTD_dfast }, /* level 4 */ + { 17, 16, 17, 3, 4, 2, ZSTD_greedy }, /* level 5 */ + { 17, 17, 17, 3, 4, 4, ZSTD_lazy }, /* level 6 */ + { 17, 17, 17, 3, 4, 8, ZSTD_lazy2 }, /* level 7 */ + { 17, 17, 17, 4, 4, 8, ZSTD_lazy2 }, /* level 8 */ + { 17, 17, 17, 5, 4, 8, ZSTD_lazy2 }, /* level 9 */ + { 17, 17, 17, 6, 4, 8, ZSTD_lazy2 }, /* level 10 */ + { 17, 17, 17, 5, 4, 8, ZSTD_btlazy2 }, /* level 11 */ + { 17, 18, 17, 7, 4, 12, ZSTD_btlazy2 }, /* level 12 */ + { 17, 18, 17, 3, 4, 12, ZSTD_btopt }, /* level 13.*/ + { 17, 18, 17, 4, 3, 32, ZSTD_btopt }, /* level 14.*/ + { 17, 18, 17, 6, 3,256, ZSTD_btopt }, /* level 15.*/ + { 17, 18, 17, 6, 3,128, ZSTD_btultra }, /* level 16.*/ + { 17, 18, 17, 8, 3,256, ZSTD_btultra }, /* level 17.*/ + { 17, 18, 17, 10, 3,512, ZSTD_btultra }, /* level 18.*/ + { 17, 18, 17, 5, 3,256, ZSTD_btultra2}, /* level 19.*/ + { 17, 18, 17, 7, 3,512, ZSTD_btultra2}, /* level 20.*/ + { 17, 18, 17, 9, 3,512, ZSTD_btultra2}, /* level 21.*/ + { 17, 18, 17, 11, 3,999, ZSTD_btultra2}, /* level 22.*/ +}, +{ /* for srcSize <= 16 KB */ + /* W, C, H, S, L, T, strat */ + { 14, 12, 13, 1, 5, 1, ZSTD_fast }, /* base for negative levels */ + { 14, 14, 15, 1, 5, 0, ZSTD_fast }, /* level 1 */ + { 14, 14, 15, 1, 4, 0, ZSTD_fast }, /* level 2 */ + { 14, 14, 15, 2, 4, 0, ZSTD_dfast }, /* level 3 */ + { 14, 14, 14, 4, 4, 2, ZSTD_greedy }, /* level 4 */ + { 14, 14, 14, 3, 4, 4, ZSTD_lazy }, /* level 5.*/ + { 14, 14, 14, 4, 4, 8, ZSTD_lazy2 }, /* level 6 */ + { 14, 14, 14, 6, 4, 8, ZSTD_lazy2 }, /* level 7 */ + { 14, 14, 14, 8, 4, 8, ZSTD_lazy2 }, /* level 8.*/ + { 14, 15, 14, 5, 4, 8, ZSTD_btlazy2 }, /* level 9.*/ + { 14, 15, 14, 9, 4, 8, ZSTD_btlazy2 }, /* level 10.*/ + { 14, 15, 14, 3, 4, 12, ZSTD_btopt }, /* level 11.*/ + { 14, 15, 14, 4, 3, 24, ZSTD_btopt }, /* level 12.*/ + { 14, 15, 14, 5, 3, 32, ZSTD_btultra }, /* level 13.*/ + { 14, 15, 15, 6, 3, 64, ZSTD_btultra }, /* level 14.*/ + { 14, 15, 15, 7, 3,256, ZSTD_btultra }, /* level 15.*/ + { 14, 15, 15, 5, 3, 48, ZSTD_btultra2}, /* level 16.*/ + { 14, 15, 15, 6, 3,128, ZSTD_btultra2}, /* level 17.*/ + { 14, 15, 15, 7, 3,256, ZSTD_btultra2}, /* level 18.*/ + { 14, 15, 15, 8, 3,256, ZSTD_btultra2}, /* level 19.*/ + { 14, 15, 15, 8, 3,512, ZSTD_btultra2}, /* level 20.*/ + { 14, 15, 15, 9, 3,512, ZSTD_btultra2}, /* level 21.*/ + { 14, 15, 15, 10, 3,999, ZSTD_btultra2}, /* level 22.*/ +}, +}; + +/*! ZSTD_getCParams_internal() : + * @return ZSTD_compressionParameters structure for a selected compression level, srcSize and dictSize. + * Note: srcSizeHint 0 means 0, use ZSTD_CONTENTSIZE_UNKNOWN for unknown. + * Use dictSize == 0 for unknown or unused. */ +static ZSTD_compressionParameters ZSTD_getCParams_internal(int compressionLevel, unsigned long long srcSizeHint, size_t dictSize) +{ + int const unknown = srcSizeHint == ZSTD_CONTENTSIZE_UNKNOWN; + size_t const addedSize = unknown && dictSize > 0 ? 500 : 0; + U64 const rSize = unknown && dictSize == 0 ? ZSTD_CONTENTSIZE_UNKNOWN : srcSizeHint+dictSize+addedSize; + U32 const tableID = (rSize <= 256 KB) + (rSize <= 128 KB) + (rSize <= 16 KB); + int row = compressionLevel; + DEBUGLOG(5, "ZSTD_getCParams_internal (cLevel=%i)", compressionLevel); + if (compressionLevel == 0) row = ZSTD_CLEVEL_DEFAULT; /* 0 == default */ + if (compressionLevel < 0) row = 0; /* entry 0 is baseline for fast mode */ + if (compressionLevel > ZSTD_MAX_CLEVEL) row = ZSTD_MAX_CLEVEL; + { ZSTD_compressionParameters cp = ZSTD_defaultCParameters[tableID][row]; + if (compressionLevel < 0) cp.targetLength = (unsigned)(-compressionLevel); /* acceleration factor */ + /* refine parameters based on srcSize & dictSize */ + return ZSTD_adjustCParams_internal(cp, srcSizeHint, dictSize); + } +} + +/*! ZSTD_getCParams() : + * @return ZSTD_compressionParameters structure for a selected compression level, srcSize and dictSize. + * Size values are optional, provide 0 if not known or unused */ +ZSTD_compressionParameters ZSTD_getCParams(int compressionLevel, unsigned long long srcSizeHint, size_t dictSize) +{ + if (srcSizeHint == 0) srcSizeHint = ZSTD_CONTENTSIZE_UNKNOWN; + return ZSTD_getCParams_internal(compressionLevel, srcSizeHint, dictSize); +} + +/*! ZSTD_getParams() : + * same idea as ZSTD_getCParams() + * @return a `ZSTD_parameters` structure (instead of `ZSTD_compressionParameters`). + * Fields of `ZSTD_frameParameters` are set to default values */ +static ZSTD_parameters ZSTD_getParams_internal(int compressionLevel, unsigned long long srcSizeHint, size_t dictSize) { + ZSTD_parameters params; + ZSTD_compressionParameters const cParams = ZSTD_getCParams_internal(compressionLevel, srcSizeHint, dictSize); + DEBUGLOG(5, "ZSTD_getParams (cLevel=%i)", compressionLevel); + memset(¶ms, 0, sizeof(params)); + params.cParams = cParams; + params.fParams.contentSizeFlag = 1; + return params; +} + +/*! ZSTD_getParams() : + * same idea as ZSTD_getCParams() + * @return a `ZSTD_parameters` structure (instead of `ZSTD_compressionParameters`). + * Fields of `ZSTD_frameParameters` are set to default values */ +ZSTD_parameters ZSTD_getParams(int compressionLevel, unsigned long long srcSizeHint, size_t dictSize) { + if (srcSizeHint == 0) srcSizeHint = ZSTD_CONTENTSIZE_UNKNOWN; + return ZSTD_getParams_internal(compressionLevel, srcSizeHint, dictSize); +} diff --git a/module/zstd/lib/compress/zstd_compress_internal.h b/module/zstd/lib/compress/zstd_compress_internal.h new file mode 100644 index 000000000000..db73f6ce21f2 --- /dev/null +++ b/module/zstd/lib/compress/zstd_compress_internal.h @@ -0,0 +1,1125 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +/* This header contains definitions + * that shall **only** be used by modules within lib/compress. + */ + +#ifndef ZSTD_COMPRESS_H +#define ZSTD_COMPRESS_H + +/*-************************************* +* Dependencies +***************************************/ +#include "../common/zstd_internal.h" +#include "zstd_cwksp.h" +#ifdef ZSTD_MULTITHREAD +# include "zstdmt_compress.h" +#endif + +#if defined (__cplusplus) +extern "C" { +#endif + + +/*-************************************* +* Constants +***************************************/ +#define kSearchStrength 8 +#define HASH_READ_SIZE 8 +#define ZSTD_DUBT_UNSORTED_MARK 1 /* For btlazy2 strategy, index ZSTD_DUBT_UNSORTED_MARK==1 means "unsorted". + It could be confused for a real successor at index "1", if sorted as larger than its predecessor. + It's not a big deal though : candidate will just be sorted again. + Additionally, candidate position 1 will be lost. + But candidate 1 cannot hide a large tree of candidates, so it's a minimal loss. + The benefit is that ZSTD_DUBT_UNSORTED_MARK cannot be mishandled after table re-use with a different strategy. + This constant is required by ZSTD_compressBlock_btlazy2() and ZSTD_reduceTable_internal() */ + + +/*-************************************* +* Context memory management +***************************************/ +typedef enum { ZSTDcs_created=0, ZSTDcs_init, ZSTDcs_ongoing, ZSTDcs_ending } ZSTD_compressionStage_e; +typedef enum { zcss_init=0, zcss_load, zcss_flush } ZSTD_cStreamStage; + +typedef struct ZSTD_prefixDict_s { + const void* dict; + size_t dictSize; + ZSTD_dictContentType_e dictContentType; +} ZSTD_prefixDict; + +typedef struct { + void* dictBuffer; + void const* dict; + size_t dictSize; + ZSTD_dictContentType_e dictContentType; + ZSTD_CDict* cdict; +} ZSTD_localDict; + +typedef struct { + U32 CTable[HUF_CTABLE_SIZE_U32(255)]; + HUF_repeat repeatMode; +} ZSTD_hufCTables_t; + +typedef struct { + FSE_CTable offcodeCTable[FSE_CTABLE_SIZE_U32(OffFSELog, MaxOff)]; + FSE_CTable matchlengthCTable[FSE_CTABLE_SIZE_U32(MLFSELog, MaxML)]; + FSE_CTable litlengthCTable[FSE_CTABLE_SIZE_U32(LLFSELog, MaxLL)]; + FSE_repeat offcode_repeatMode; + FSE_repeat matchlength_repeatMode; + FSE_repeat litlength_repeatMode; +} ZSTD_fseCTables_t; + +typedef struct { + ZSTD_hufCTables_t huf; + ZSTD_fseCTables_t fse; +} ZSTD_entropyCTables_t; + +typedef struct { + U32 off; + U32 len; +} ZSTD_match_t; + +typedef struct { + int price; + U32 off; + U32 mlen; + U32 litlen; + U32 rep[ZSTD_REP_NUM]; +} ZSTD_optimal_t; + +typedef enum { zop_dynamic=0, zop_predef } ZSTD_OptPrice_e; + +typedef struct { + /* All tables are allocated inside cctx->workspace by ZSTD_resetCCtx_internal() */ + unsigned* litFreq; /* table of literals statistics, of size 256 */ + unsigned* litLengthFreq; /* table of litLength statistics, of size (MaxLL+1) */ + unsigned* matchLengthFreq; /* table of matchLength statistics, of size (MaxML+1) */ + unsigned* offCodeFreq; /* table of offCode statistics, of size (MaxOff+1) */ + ZSTD_match_t* matchTable; /* list of found matches, of size ZSTD_OPT_NUM+1 */ + ZSTD_optimal_t* priceTable; /* All positions tracked by optimal parser, of size ZSTD_OPT_NUM+1 */ + + U32 litSum; /* nb of literals */ + U32 litLengthSum; /* nb of litLength codes */ + U32 matchLengthSum; /* nb of matchLength codes */ + U32 offCodeSum; /* nb of offset codes */ + U32 litSumBasePrice; /* to compare to log2(litfreq) */ + U32 litLengthSumBasePrice; /* to compare to log2(llfreq) */ + U32 matchLengthSumBasePrice;/* to compare to log2(mlfreq) */ + U32 offCodeSumBasePrice; /* to compare to log2(offreq) */ + ZSTD_OptPrice_e priceType; /* prices can be determined dynamically, or follow a pre-defined cost structure */ + const ZSTD_entropyCTables_t* symbolCosts; /* pre-calculated dictionary statistics */ + ZSTD_literalCompressionMode_e literalCompressionMode; +} optState_t; + +typedef struct { + ZSTD_entropyCTables_t entropy; + U32 rep[ZSTD_REP_NUM]; +} ZSTD_compressedBlockState_t; + +typedef struct { + BYTE const* nextSrc; /* next block here to continue on current prefix */ + BYTE const* base; /* All regular indexes relative to this position */ + BYTE const* dictBase; /* extDict indexes relative to this position */ + U32 dictLimit; /* below that point, need extDict */ + U32 lowLimit; /* below that point, no more valid data */ +} ZSTD_window_t; + +typedef struct ZSTD_matchState_t ZSTD_matchState_t; +struct ZSTD_matchState_t { + ZSTD_window_t window; /* State for window round buffer management */ + U32 loadedDictEnd; /* index of end of dictionary, within context's referential. + * When loadedDictEnd != 0, a dictionary is in use, and still valid. + * This relies on a mechanism to set loadedDictEnd=0 when dictionary is no longer within distance. + * Such mechanism is provided within ZSTD_window_enforceMaxDist() and ZSTD_checkDictValidity(). + * When dict referential is copied into active context (i.e. not attached), + * loadedDictEnd == dictSize, since referential starts from zero. + */ + U32 nextToUpdate; /* index from which to continue table update */ + U32 hashLog3; /* dispatch table for matches of len==3 : larger == faster, more memory */ + U32* hashTable; + U32* hashTable3; + U32* chainTable; + optState_t opt; /* optimal parser state */ + const ZSTD_matchState_t* dictMatchState; + ZSTD_compressionParameters cParams; +}; + +typedef struct { + ZSTD_compressedBlockState_t* prevCBlock; + ZSTD_compressedBlockState_t* nextCBlock; + ZSTD_matchState_t matchState; +} ZSTD_blockState_t; + +typedef struct { + U32 offset; + U32 checksum; +} ldmEntry_t; + +typedef struct { + ZSTD_window_t window; /* State for the window round buffer management */ + ldmEntry_t* hashTable; + U32 loadedDictEnd; + BYTE* bucketOffsets; /* Next position in bucket to insert entry */ + U64 hashPower; /* Used to compute the rolling hash. + * Depends on ldmParams.minMatchLength */ +} ldmState_t; + +typedef struct { + U32 enableLdm; /* 1 if enable long distance matching */ + U32 hashLog; /* Log size of hashTable */ + U32 bucketSizeLog; /* Log bucket size for collision resolution, at most 8 */ + U32 minMatchLength; /* Minimum match length */ + U32 hashRateLog; /* Log number of entries to skip */ + U32 windowLog; /* Window log for the LDM */ +} ldmParams_t; + +typedef struct { + U32 offset; + U32 litLength; + U32 matchLength; +} rawSeq; + +typedef struct { + rawSeq* seq; /* The start of the sequences */ + size_t pos; /* The position where reading stopped. <= size. */ + size_t size; /* The number of sequences. <= capacity. */ + size_t capacity; /* The capacity starting from `seq` pointer */ +} rawSeqStore_t; + +typedef struct { + int collectSequences; + ZSTD_Sequence* seqStart; + size_t seqIndex; + size_t maxSequences; +} SeqCollector; + +struct ZSTD_CCtx_params_s { + ZSTD_format_e format; + ZSTD_compressionParameters cParams; + ZSTD_frameParameters fParams; + + int compressionLevel; + int forceWindow; /* force back-references to respect limit of + * 1< 63) ? ZSTD_highbit32(litLength) + LL_deltaCode : LL_Code[litLength]; +} + +/* ZSTD_MLcode() : + * note : mlBase = matchLength - MINMATCH; + * because it's the format it's stored in seqStore->sequences */ +MEM_STATIC U32 ZSTD_MLcode(U32 mlBase) +{ + static const BYTE ML_Code[128] = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, + 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, + 32, 32, 33, 33, 34, 34, 35, 35, 36, 36, 36, 36, 37, 37, 37, 37, + 38, 38, 38, 38, 38, 38, 38, 38, 39, 39, 39, 39, 39, 39, 39, 39, + 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, + 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, + 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, + 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42 }; + static const U32 ML_deltaCode = 36; + return (mlBase > 127) ? ZSTD_highbit32(mlBase) + ML_deltaCode : ML_Code[mlBase]; +} + +typedef struct repcodes_s { + U32 rep[3]; +} repcodes_t; + +MEM_STATIC repcodes_t ZSTD_updateRep(U32 const rep[3], U32 const offset, U32 const ll0) +{ + repcodes_t newReps; + if (offset >= ZSTD_REP_NUM) { /* full offset */ + newReps.rep[2] = rep[1]; + newReps.rep[1] = rep[0]; + newReps.rep[0] = offset - ZSTD_REP_MOVE; + } else { /* repcode */ + U32 const repCode = offset + ll0; + if (repCode > 0) { /* note : if repCode==0, no change */ + U32 const currentOffset = (repCode==ZSTD_REP_NUM) ? (rep[0] - 1) : rep[repCode]; + newReps.rep[2] = (repCode >= 2) ? rep[1] : rep[2]; + newReps.rep[1] = rep[0]; + newReps.rep[0] = currentOffset; + } else { /* repCode == 0 */ + memcpy(&newReps, rep, sizeof(newReps)); + } + } + return newReps; +} + +/* ZSTD_cParam_withinBounds: + * @return 1 if value is within cParam bounds, + * 0 otherwise */ +MEM_STATIC int ZSTD_cParam_withinBounds(ZSTD_cParameter cParam, int value) +{ + ZSTD_bounds const bounds = ZSTD_cParam_getBounds(cParam); + if (ZSTD_isError(bounds.error)) return 0; + if (value < bounds.lowerBound) return 0; + if (value > bounds.upperBound) return 0; + return 1; +} + +/* ZSTD_noCompressBlock() : + * Writes uncompressed block to dst buffer from given src. + * Returns the size of the block */ +MEM_STATIC size_t ZSTD_noCompressBlock (void* dst, size_t dstCapacity, const void* src, size_t srcSize, U32 lastBlock) +{ + U32 const cBlockHeader24 = lastBlock + (((U32)bt_raw)<<1) + (U32)(srcSize << 3); + RETURN_ERROR_IF(srcSize + ZSTD_blockHeaderSize > dstCapacity, + dstSize_tooSmall, "dst buf too small for uncompressed block"); + MEM_writeLE24(dst, cBlockHeader24); + memcpy((BYTE*)dst + ZSTD_blockHeaderSize, src, srcSize); + return ZSTD_blockHeaderSize + srcSize; +} + +MEM_STATIC size_t ZSTD_rleCompressBlock (void* dst, size_t dstCapacity, BYTE src, size_t srcSize, U32 lastBlock) +{ + BYTE* const op = (BYTE*)dst; + U32 const cBlockHeader = lastBlock + (((U32)bt_rle)<<1) + (U32)(srcSize << 3); + RETURN_ERROR_IF(dstCapacity < 4, dstSize_tooSmall, ""); + MEM_writeLE24(op, cBlockHeader); + op[3] = src; + return 4; +} + + +/* ZSTD_minGain() : + * minimum compression required + * to generate a compress block or a compressed literals section. + * note : use same formula for both situations */ +MEM_STATIC size_t ZSTD_minGain(size_t srcSize, ZSTD_strategy strat) +{ + U32 const minlog = (strat>=ZSTD_btultra) ? (U32)(strat) - 1 : 6; + ZSTD_STATIC_ASSERT(ZSTD_btultra == 8); + assert(ZSTD_cParam_withinBounds(ZSTD_c_strategy, strat)); + return (srcSize >> minlog) + 2; +} + +MEM_STATIC int ZSTD_disableLiteralsCompression(const ZSTD_CCtx_params* cctxParams) +{ + switch (cctxParams->literalCompressionMode) { + case ZSTD_lcm_huffman: + return 0; + case ZSTD_lcm_uncompressed: + return 1; + default: + assert(0 /* impossible: pre-validated */); + /* fall-through */ + case ZSTD_lcm_auto: + return (cctxParams->cParams.strategy == ZSTD_fast) && (cctxParams->cParams.targetLength > 0); + } +} + +/*! ZSTD_safecopyLiterals() : + * memcpy() function that won't read beyond more than WILDCOPY_OVERLENGTH bytes past ilimit_w. + * Only called when the sequence ends past ilimit_w, so it only needs to be optimized for single + * large copies. + */ +static void ZSTD_safecopyLiterals(BYTE* op, BYTE const* ip, BYTE const* const iend, BYTE const* ilimit_w) { + assert(iend > ilimit_w); + if (ip <= ilimit_w) { + ZSTD_wildcopy(op, ip, ilimit_w - ip, ZSTD_no_overlap); + op += ilimit_w - ip; + ip = ilimit_w; + } + while (ip < iend) *op++ = *ip++; +} + +/*! ZSTD_storeSeq() : + * Store a sequence (litlen, litPtr, offCode and mlBase) into seqStore_t. + * `offCode` : distance to match + ZSTD_REP_MOVE (values <= ZSTD_REP_MOVE are repCodes). + * `mlBase` : matchLength - MINMATCH + * Allowed to overread literals up to litLimit. +*/ +HINT_INLINE UNUSED_ATTR +void ZSTD_storeSeq(seqStore_t* seqStorePtr, size_t litLength, const BYTE* literals, const BYTE* litLimit, U32 offCode, size_t mlBase) +{ + BYTE const* const litLimit_w = litLimit - WILDCOPY_OVERLENGTH; + BYTE const* const litEnd = literals + litLength; +#if defined(DEBUGLEVEL) && (DEBUGLEVEL >= 6) + static const BYTE* g_start = NULL; + if (g_start==NULL) g_start = (const BYTE*)literals; /* note : index only works for compression within a single segment */ + { U32 const pos = (U32)((const BYTE*)literals - g_start); + DEBUGLOG(6, "Cpos%7u :%3u literals, match%4u bytes at offCode%7u", + pos, (U32)litLength, (U32)mlBase+MINMATCH, (U32)offCode); + } +#endif + assert((size_t)(seqStorePtr->sequences - seqStorePtr->sequencesStart) < seqStorePtr->maxNbSeq); + /* copy Literals */ + assert(seqStorePtr->maxNbLit <= 128 KB); + assert(seqStorePtr->lit + litLength <= seqStorePtr->litStart + seqStorePtr->maxNbLit); + assert(literals + litLength <= litLimit); + if (litEnd <= litLimit_w) { + /* Common case we can use wildcopy. + * First copy 16 bytes, because literals are likely short. + */ + assert(WILDCOPY_OVERLENGTH >= 16); + ZSTD_copy16(seqStorePtr->lit, literals); + if (litLength > 16) { + ZSTD_wildcopy(seqStorePtr->lit+16, literals+16, (ptrdiff_t)litLength-16, ZSTD_no_overlap); + } + } else { + ZSTD_safecopyLiterals(seqStorePtr->lit, literals, litEnd, litLimit_w); + } + seqStorePtr->lit += litLength; + + /* literal Length */ + if (litLength>0xFFFF) { + assert(seqStorePtr->longLengthID == 0); /* there can only be a single long length */ + seqStorePtr->longLengthID = 1; + seqStorePtr->longLengthPos = (U32)(seqStorePtr->sequences - seqStorePtr->sequencesStart); + } + seqStorePtr->sequences[0].litLength = (U16)litLength; + + /* match offset */ + seqStorePtr->sequences[0].offset = offCode + 1; + + /* match Length */ + if (mlBase>0xFFFF) { + assert(seqStorePtr->longLengthID == 0); /* there can only be a single long length */ + seqStorePtr->longLengthID = 2; + seqStorePtr->longLengthPos = (U32)(seqStorePtr->sequences - seqStorePtr->sequencesStart); + } + seqStorePtr->sequences[0].matchLength = (U16)mlBase; + + seqStorePtr->sequences++; +} + + +/*-************************************* +* Match length counter +***************************************/ +static unsigned ZSTD_NbCommonBytes (size_t val) +{ + if (MEM_isLittleEndian()) { + if (MEM_64bits()) { +# if defined(_MSC_VER) && defined(_WIN64) + unsigned long r = 0; + return _BitScanForward64( &r, (U64)val ) ? (unsigned)(r >> 3) : 0; +# elif defined(__GNUC__) && (__GNUC__ >= 4) + return (__builtin_ctzll((U64)val) >> 3); +# else + static const int DeBruijnBytePos[64] = { 0, 0, 0, 0, 0, 1, 1, 2, + 0, 3, 1, 3, 1, 4, 2, 7, + 0, 2, 3, 6, 1, 5, 3, 5, + 1, 3, 4, 4, 2, 5, 6, 7, + 7, 0, 1, 2, 3, 3, 4, 6, + 2, 6, 5, 5, 3, 4, 5, 6, + 7, 1, 2, 4, 6, 4, 4, 5, + 7, 2, 6, 5, 7, 6, 7, 7 }; + return DeBruijnBytePos[((U64)((val & -(long long)val) * 0x0218A392CDABBD3FULL)) >> 58]; +# endif + } else { /* 32 bits */ +# if defined(_MSC_VER) + unsigned long r=0; + return _BitScanForward( &r, (U32)val ) ? (unsigned)(r >> 3) : 0; +# elif defined(__GNUC__) && (__GNUC__ >= 3) + return (__builtin_ctz((U32)val) >> 3); +# else + static const int DeBruijnBytePos[32] = { 0, 0, 3, 0, 3, 1, 3, 0, + 3, 2, 2, 1, 3, 2, 0, 1, + 3, 3, 1, 2, 2, 2, 2, 0, + 3, 1, 2, 0, 1, 0, 1, 1 }; + return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >> 27]; +# endif + } + } else { /* Big Endian CPU */ + if (MEM_64bits()) { +# if defined(_MSC_VER) && defined(_WIN64) + unsigned long r = 0; + return _BitScanReverse64( &r, val ) ? (unsigned)(r >> 3) : 0; +# elif defined(__GNUC__) && (__GNUC__ >= 4) + return (__builtin_clzll(val) >> 3); +# else + unsigned r; + const unsigned n32 = sizeof(size_t)*4; /* calculate this way due to compiler complaining in 32-bits mode */ + if (!(val>>n32)) { r=4; } else { r=0; val>>=n32; } + if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; } + r += (!val); + return r; +# endif + } else { /* 32 bits */ +# if defined(_MSC_VER) + unsigned long r = 0; + return _BitScanReverse( &r, (unsigned long)val ) ? (unsigned)(r >> 3) : 0; +# elif defined(__GNUC__) && (__GNUC__ >= 3) + return (__builtin_clz((U32)val) >> 3); +# else + unsigned r; + if (!(val>>16)) { r=2; val>>=8; } else { r=0; val>>=24; } + r += (!val); + return r; +# endif + } } +} + + +MEM_STATIC size_t ZSTD_count(const BYTE* pIn, const BYTE* pMatch, const BYTE* const pInLimit) +{ + const BYTE* const pStart = pIn; + const BYTE* const pInLoopLimit = pInLimit - (sizeof(size_t)-1); + + if (pIn < pInLoopLimit) { + { size_t const diff = MEM_readST(pMatch) ^ MEM_readST(pIn); + if (diff) return ZSTD_NbCommonBytes(diff); } + pIn+=sizeof(size_t); pMatch+=sizeof(size_t); + while (pIn < pInLoopLimit) { + size_t const diff = MEM_readST(pMatch) ^ MEM_readST(pIn); + if (!diff) { pIn+=sizeof(size_t); pMatch+=sizeof(size_t); continue; } + pIn += ZSTD_NbCommonBytes(diff); + return (size_t)(pIn - pStart); + } } + if (MEM_64bits() && (pIn<(pInLimit-3)) && (MEM_read32(pMatch) == MEM_read32(pIn))) { pIn+=4; pMatch+=4; } + if ((pIn<(pInLimit-1)) && (MEM_read16(pMatch) == MEM_read16(pIn))) { pIn+=2; pMatch+=2; } + if ((pIn> (32-h) ; } +MEM_STATIC size_t ZSTD_hash3Ptr(const void* ptr, U32 h) { return ZSTD_hash3(MEM_readLE32(ptr), h); } /* only in zstd_opt.h */ + +static const U32 prime4bytes = 2654435761U; +static U32 ZSTD_hash4(U32 u, U32 h) { return (u * prime4bytes) >> (32-h) ; } +static size_t ZSTD_hash4Ptr(const void* ptr, U32 h) { return ZSTD_hash4(MEM_read32(ptr), h); } + +static const U64 prime5bytes = 889523592379ULL; +static size_t ZSTD_hash5(U64 u, U32 h) { return (size_t)(((u << (64-40)) * prime5bytes) >> (64-h)) ; } +static size_t ZSTD_hash5Ptr(const void* p, U32 h) { return ZSTD_hash5(MEM_readLE64(p), h); } + +static const U64 prime6bytes = 227718039650203ULL; +static size_t ZSTD_hash6(U64 u, U32 h) { return (size_t)(((u << (64-48)) * prime6bytes) >> (64-h)) ; } +static size_t ZSTD_hash6Ptr(const void* p, U32 h) { return ZSTD_hash6(MEM_readLE64(p), h); } + +static const U64 prime7bytes = 58295818150454627ULL; +static size_t ZSTD_hash7(U64 u, U32 h) { return (size_t)(((u << (64-56)) * prime7bytes) >> (64-h)) ; } +static size_t ZSTD_hash7Ptr(const void* p, U32 h) { return ZSTD_hash7(MEM_readLE64(p), h); } + +static const U64 prime8bytes = 0xCF1BBCDCB7A56463ULL; +static size_t ZSTD_hash8(U64 u, U32 h) { return (size_t)(((u) * prime8bytes) >> (64-h)) ; } +static size_t ZSTD_hash8Ptr(const void* p, U32 h) { return ZSTD_hash8(MEM_readLE64(p), h); } + +MEM_STATIC size_t ZSTD_hashPtr(const void* p, U32 hBits, U32 mls) +{ + switch(mls) + { + default: + case 4: return ZSTD_hash4Ptr(p, hBits); + case 5: return ZSTD_hash5Ptr(p, hBits); + case 6: return ZSTD_hash6Ptr(p, hBits); + case 7: return ZSTD_hash7Ptr(p, hBits); + case 8: return ZSTD_hash8Ptr(p, hBits); + } +} + +/** ZSTD_ipow() : + * Return base^exponent. + */ +static U64 ZSTD_ipow(U64 base, U64 exponent) +{ + U64 power = 1; + while (exponent) { + if (exponent & 1) power *= base; + exponent >>= 1; + base *= base; + } + return power; +} + +#define ZSTD_ROLL_HASH_CHAR_OFFSET 10 + +/** ZSTD_rollingHash_append() : + * Add the buffer to the hash value. + */ +static U64 ZSTD_rollingHash_append(U64 hash, void const* buf, size_t size) +{ + BYTE const* istart = (BYTE const*)buf; + size_t pos; + for (pos = 0; pos < size; ++pos) { + hash *= prime8bytes; + hash += istart[pos] + ZSTD_ROLL_HASH_CHAR_OFFSET; + } + return hash; +} + +/** ZSTD_rollingHash_compute() : + * Compute the rolling hash value of the buffer. + */ +MEM_STATIC U64 ZSTD_rollingHash_compute(void const* buf, size_t size) +{ + return ZSTD_rollingHash_append(0, buf, size); +} + +/** ZSTD_rollingHash_primePower() : + * Compute the primePower to be passed to ZSTD_rollingHash_rotate() for a hash + * over a window of length bytes. + */ +MEM_STATIC U64 ZSTD_rollingHash_primePower(U32 length) +{ + return ZSTD_ipow(prime8bytes, length - 1); +} + +/** ZSTD_rollingHash_rotate() : + * Rotate the rolling hash by one byte. + */ +MEM_STATIC U64 ZSTD_rollingHash_rotate(U64 hash, BYTE toRemove, BYTE toAdd, U64 primePower) +{ + hash -= (toRemove + ZSTD_ROLL_HASH_CHAR_OFFSET) * primePower; + hash *= prime8bytes; + hash += toAdd + ZSTD_ROLL_HASH_CHAR_OFFSET; + return hash; +} + +/*-************************************* +* Round buffer management +***************************************/ +#if (ZSTD_WINDOWLOG_MAX_64 > 31) +# error "ZSTD_WINDOWLOG_MAX is too large : would overflow ZSTD_CURRENT_MAX" +#endif +/* Max current allowed */ +#define ZSTD_CURRENT_MAX ((3U << 29) + (1U << ZSTD_WINDOWLOG_MAX)) +/* Maximum chunk size before overflow correction needs to be called again */ +#define ZSTD_CHUNKSIZE_MAX \ + ( ((U32)-1) /* Maximum ending current index */ \ + - ZSTD_CURRENT_MAX) /* Maximum beginning lowLimit */ + +/** + * ZSTD_window_clear(): + * Clears the window containing the history by simply setting it to empty. + */ +MEM_STATIC void ZSTD_window_clear(ZSTD_window_t* window) +{ + size_t const endT = (size_t)(window->nextSrc - window->base); + U32 const end = (U32)endT; + + window->lowLimit = end; + window->dictLimit = end; +} + +/** + * ZSTD_window_hasExtDict(): + * Returns non-zero if the window has a non-empty extDict. + */ +MEM_STATIC U32 ZSTD_window_hasExtDict(ZSTD_window_t const window) +{ + return window.lowLimit < window.dictLimit; +} + +/** + * ZSTD_matchState_dictMode(): + * Inspects the provided matchState and figures out what dictMode should be + * passed to the compressor. + */ +MEM_STATIC ZSTD_dictMode_e ZSTD_matchState_dictMode(const ZSTD_matchState_t *ms) +{ + return ZSTD_window_hasExtDict(ms->window) ? + ZSTD_extDict : + ms->dictMatchState != NULL ? + ZSTD_dictMatchState : + ZSTD_noDict; +} + +/** + * ZSTD_window_needOverflowCorrection(): + * Returns non-zero if the indices are getting too large and need overflow + * protection. + */ +MEM_STATIC U32 ZSTD_window_needOverflowCorrection(ZSTD_window_t const window, + void const* srcEnd) +{ + U32 const current = (U32)((BYTE const*)srcEnd - window.base); + return current > ZSTD_CURRENT_MAX; +} + +/** + * ZSTD_window_correctOverflow(): + * Reduces the indices to protect from index overflow. + * Returns the correction made to the indices, which must be applied to every + * stored index. + * + * The least significant cycleLog bits of the indices must remain the same, + * which may be 0. Every index up to maxDist in the past must be valid. + * NOTE: (maxDist & cycleMask) must be zero. + */ +MEM_STATIC U32 ZSTD_window_correctOverflow(ZSTD_window_t* window, U32 cycleLog, + U32 maxDist, void const* src) +{ + /* preemptive overflow correction: + * 1. correction is large enough: + * lowLimit > (3<<29) ==> current > 3<<29 + 1< (3<<29 + 1< (3<<29) - (1< (3<<29) - (1<<30) (NOTE: chainLog <= 30) + * > 1<<29 + * + * 2. (ip+ZSTD_CHUNKSIZE_MAX - cctx->base) doesn't overflow: + * After correction, current is less than (1<base < 1<<32. + * 3. (cctx->lowLimit + 1< 3<<29 + 1<base); + U32 const currentCycle0 = current & cycleMask; + /* Exclude zero so that newCurrent - maxDist >= 1. */ + U32 const currentCycle1 = currentCycle0 == 0 ? (1U << cycleLog) : currentCycle0; + U32 const newCurrent = currentCycle1 + maxDist; + U32 const correction = current - newCurrent; + assert((maxDist & cycleMask) == 0); + assert(current > newCurrent); + /* Loose bound, should be around 1<<29 (see above) */ + assert(correction > 1<<28); + + window->base += correction; + window->dictBase += correction; + if (window->lowLimit <= correction) window->lowLimit = 1; + else window->lowLimit -= correction; + if (window->dictLimit <= correction) window->dictLimit = 1; + else window->dictLimit -= correction; + + /* Ensure we can still reference the full window. */ + assert(newCurrent >= maxDist); + assert(newCurrent - maxDist >= 1); + /* Ensure that lowLimit and dictLimit didn't underflow. */ + assert(window->lowLimit <= newCurrent); + assert(window->dictLimit <= newCurrent); + + DEBUGLOG(4, "Correction of 0x%x bytes to lowLimit=0x%x", correction, + window->lowLimit); + return correction; +} + +/** + * ZSTD_window_enforceMaxDist(): + * Updates lowLimit so that: + * (srcEnd - base) - lowLimit == maxDist + loadedDictEnd + * + * It ensures index is valid as long as index >= lowLimit. + * This must be called before a block compression call. + * + * loadedDictEnd is only defined if a dictionary is in use for current compression. + * As the name implies, loadedDictEnd represents the index at end of dictionary. + * The value lies within context's referential, it can be directly compared to blockEndIdx. + * + * If loadedDictEndPtr is NULL, no dictionary is in use, and we use loadedDictEnd == 0. + * If loadedDictEndPtr is not NULL, we set it to zero after updating lowLimit. + * This is because dictionaries are allowed to be referenced fully + * as long as the last byte of the dictionary is in the window. + * Once input has progressed beyond window size, dictionary cannot be referenced anymore. + * + * In normal dict mode, the dictionary lies between lowLimit and dictLimit. + * In dictMatchState mode, lowLimit and dictLimit are the same, + * and the dictionary is below them. + * forceWindow and dictMatchState are therefore incompatible. + */ +MEM_STATIC void +ZSTD_window_enforceMaxDist(ZSTD_window_t* window, + const void* blockEnd, + U32 maxDist, + U32* loadedDictEndPtr, + const ZSTD_matchState_t** dictMatchStatePtr) +{ + U32 const blockEndIdx = (U32)((BYTE const*)blockEnd - window->base); + U32 const loadedDictEnd = (loadedDictEndPtr != NULL) ? *loadedDictEndPtr : 0; + DEBUGLOG(5, "ZSTD_window_enforceMaxDist: blockEndIdx=%u, maxDist=%u, loadedDictEnd=%u", + (unsigned)blockEndIdx, (unsigned)maxDist, (unsigned)loadedDictEnd); + + /* - When there is no dictionary : loadedDictEnd == 0. + In which case, the test (blockEndIdx > maxDist) is merely to avoid + overflowing next operation `newLowLimit = blockEndIdx - maxDist`. + - When there is a standard dictionary : + Index referential is copied from the dictionary, + which means it starts from 0. + In which case, loadedDictEnd == dictSize, + and it makes sense to compare `blockEndIdx > maxDist + dictSize` + since `blockEndIdx` also starts from zero. + - When there is an attached dictionary : + loadedDictEnd is expressed within the referential of the context, + so it can be directly compared against blockEndIdx. + */ + if (blockEndIdx > maxDist + loadedDictEnd) { + U32 const newLowLimit = blockEndIdx - maxDist; + if (window->lowLimit < newLowLimit) window->lowLimit = newLowLimit; + if (window->dictLimit < window->lowLimit) { + DEBUGLOG(5, "Update dictLimit to match lowLimit, from %u to %u", + (unsigned)window->dictLimit, (unsigned)window->lowLimit); + window->dictLimit = window->lowLimit; + } + /* On reaching window size, dictionaries are invalidated */ + if (loadedDictEndPtr) *loadedDictEndPtr = 0; + if (dictMatchStatePtr) *dictMatchStatePtr = NULL; + } +} + +/* Similar to ZSTD_window_enforceMaxDist(), + * but only invalidates dictionary + * when input progresses beyond window size. + * assumption : loadedDictEndPtr and dictMatchStatePtr are valid (non NULL) + * loadedDictEnd uses same referential as window->base + * maxDist is the window size */ +MEM_STATIC void +ZSTD_checkDictValidity(const ZSTD_window_t* window, + const void* blockEnd, + U32 maxDist, + U32* loadedDictEndPtr, + const ZSTD_matchState_t** dictMatchStatePtr) +{ + assert(loadedDictEndPtr != NULL); + assert(dictMatchStatePtr != NULL); + { U32 const blockEndIdx = (U32)((BYTE const*)blockEnd - window->base); + U32 const loadedDictEnd = *loadedDictEndPtr; + DEBUGLOG(5, "ZSTD_checkDictValidity: blockEndIdx=%u, maxDist=%u, loadedDictEnd=%u", + (unsigned)blockEndIdx, (unsigned)maxDist, (unsigned)loadedDictEnd); + assert(blockEndIdx >= loadedDictEnd); + + if (blockEndIdx > loadedDictEnd + maxDist) { + /* On reaching window size, dictionaries are invalidated. + * For simplification, if window size is reached anywhere within next block, + * the dictionary is invalidated for the full block. + */ + DEBUGLOG(6, "invalidating dictionary for current block (distance > windowSize)"); + *loadedDictEndPtr = 0; + *dictMatchStatePtr = NULL; + } else { + if (*loadedDictEndPtr != 0) { + DEBUGLOG(6, "dictionary considered valid for current block"); + } } } +} + +MEM_STATIC void ZSTD_window_init(ZSTD_window_t* window) { + memset(window, 0, sizeof(*window)); + window->base = (BYTE const*)""; + window->dictBase = (BYTE const*)""; + window->dictLimit = 1; /* start from 1, so that 1st position is valid */ + window->lowLimit = 1; /* it ensures first and later CCtx usages compress the same */ + window->nextSrc = window->base + 1; /* see issue #1241 */ +} + +/** + * ZSTD_window_update(): + * Updates the window by appending [src, src + srcSize) to the window. + * If it is not contiguous, the current prefix becomes the extDict, and we + * forget about the extDict. Handles overlap of the prefix and extDict. + * Returns non-zero if the segment is contiguous. + */ +MEM_STATIC U32 ZSTD_window_update(ZSTD_window_t* window, + void const* src, size_t srcSize) +{ + BYTE const* const ip = (BYTE const*)src; + U32 contiguous = 1; + DEBUGLOG(5, "ZSTD_window_update"); + if (srcSize == 0) + return contiguous; + assert(window->base != NULL); + assert(window->dictBase != NULL); + /* Check if blocks follow each other */ + if (src != window->nextSrc) { + /* not contiguous */ + size_t const distanceFromBase = (size_t)(window->nextSrc - window->base); + DEBUGLOG(5, "Non contiguous blocks, new segment starts at %u", window->dictLimit); + window->lowLimit = window->dictLimit; + assert(distanceFromBase == (size_t)(U32)distanceFromBase); /* should never overflow */ + window->dictLimit = (U32)distanceFromBase; + window->dictBase = window->base; + window->base = ip - distanceFromBase; + /* ms->nextToUpdate = window->dictLimit; */ + if (window->dictLimit - window->lowLimit < HASH_READ_SIZE) window->lowLimit = window->dictLimit; /* too small extDict */ + contiguous = 0; + } + window->nextSrc = ip + srcSize; + /* if input and dictionary overlap : reduce dictionary (area presumed modified by input) */ + if ( (ip+srcSize > window->dictBase + window->lowLimit) + & (ip < window->dictBase + window->dictLimit)) { + ptrdiff_t const highInputIdx = (ip + srcSize) - window->dictBase; + U32 const lowLimitMax = (highInputIdx > (ptrdiff_t)window->dictLimit) ? window->dictLimit : (U32)highInputIdx; + window->lowLimit = lowLimitMax; + DEBUGLOG(5, "Overlapping extDict and input : new lowLimit = %u", window->lowLimit); + } + return contiguous; +} + +/** + * Returns the lowest allowed match index. It may either be in the ext-dict or the prefix. + */ +MEM_STATIC U32 ZSTD_getLowestMatchIndex(const ZSTD_matchState_t* ms, U32 current, unsigned windowLog) +{ + U32 const maxDistance = 1U << windowLog; + U32 const lowestValid = ms->window.lowLimit; + U32 const withinWindow = (current - lowestValid > maxDistance) ? current - maxDistance : lowestValid; + U32 const isDictionary = (ms->loadedDictEnd != 0); + U32 const matchLowest = isDictionary ? lowestValid : withinWindow; + return matchLowest; +} + +/** + * Returns the lowest allowed match index in the prefix. + */ +MEM_STATIC U32 ZSTD_getLowestPrefixIndex(const ZSTD_matchState_t* ms, U32 current, unsigned windowLog) +{ + U32 const maxDistance = 1U << windowLog; + U32 const lowestValid = ms->window.dictLimit; + U32 const withinWindow = (current - lowestValid > maxDistance) ? current - maxDistance : lowestValid; + U32 const isDictionary = (ms->loadedDictEnd != 0); + U32 const matchLowest = isDictionary ? lowestValid : withinWindow; + return matchLowest; +} + + + +/* debug functions */ +#if (DEBUGLEVEL>=2) + +MEM_STATIC double ZSTD_fWeight(U32 rawStat) +{ + U32 const fp_accuracy = 8; + U32 const fp_multiplier = (1 << fp_accuracy); + U32 const newStat = rawStat + 1; + U32 const hb = ZSTD_highbit32(newStat); + U32 const BWeight = hb * fp_multiplier; + U32 const FWeight = (newStat << fp_accuracy) >> hb; + U32 const weight = BWeight + FWeight; + assert(hb + fp_accuracy < 31); + return (double)weight / fp_multiplier; +} + +/* display a table content, + * listing each element, its frequency, and its predicted bit cost */ +MEM_STATIC void ZSTD_debugTable(const U32* table, U32 max) +{ + unsigned u, sum; + for (u=0, sum=0; u<=max; u++) sum += table[u]; + DEBUGLOG(2, "total nb elts: %u", sum); + for (u=0; u<=max; u++) { + DEBUGLOG(2, "%2u: %5u (%.2f)", + u, table[u], ZSTD_fWeight(sum) - ZSTD_fWeight(table[u]) ); + } +} + +#endif + + +#if defined (__cplusplus) +} +#endif + +/* =============================================================== + * Shared internal declarations + * These prototypes may be called from sources not in lib/compress + * =============================================================== */ + +/* ZSTD_loadCEntropy() : + * dict : must point at beginning of a valid zstd dictionary. + * return : size of dictionary header (size of magic number + dict ID + entropy tables) + * assumptions : magic number supposed already checked + * and dictSize >= 8 */ +size_t ZSTD_loadCEntropy(ZSTD_compressedBlockState_t* bs, void* workspace, + short* offcodeNCount, unsigned* offcodeMaxValue, + const void* const dict, size_t dictSize); + +void ZSTD_reset_compressedBlockState(ZSTD_compressedBlockState_t* bs); + +/* ============================================================== + * Private declarations + * These prototypes shall only be called from within lib/compress + * ============================================================== */ + +/* ZSTD_getCParamsFromCCtxParams() : + * cParams are built depending on compressionLevel, src size hints, + * LDM and manually set compression parameters. + * Note: srcSizeHint == 0 means 0! + */ +ZSTD_compressionParameters ZSTD_getCParamsFromCCtxParams( + const ZSTD_CCtx_params* CCtxParams, U64 srcSizeHint, size_t dictSize); + +/*! ZSTD_initCStream_internal() : + * Private use only. Init streaming operation. + * expects params to be valid. + * must receive dict, or cdict, or none, but not both. + * @return : 0, or an error code */ +size_t ZSTD_initCStream_internal(ZSTD_CStream* zcs, + const void* dict, size_t dictSize, + const ZSTD_CDict* cdict, + const ZSTD_CCtx_params* params, unsigned long long pledgedSrcSize); + +void ZSTD_resetSeqStore(seqStore_t* ssPtr); + +/*! ZSTD_getCParamsFromCDict() : + * as the name implies */ +ZSTD_compressionParameters ZSTD_getCParamsFromCDict(const ZSTD_CDict* cdict); + +/* ZSTD_compressBegin_advanced_internal() : + * Private use only. To be called from zstdmt_compress.c. */ +size_t ZSTD_compressBegin_advanced_internal(ZSTD_CCtx* cctx, + const void* dict, size_t dictSize, + ZSTD_dictContentType_e dictContentType, + ZSTD_dictTableLoadMethod_e dtlm, + const ZSTD_CDict* cdict, + const ZSTD_CCtx_params* params, + unsigned long long pledgedSrcSize); + +/* ZSTD_compress_advanced_internal() : + * Private use only. To be called from zstdmt_compress.c. */ +size_t ZSTD_compress_advanced_internal(ZSTD_CCtx* cctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + const void* dict,size_t dictSize, + const ZSTD_CCtx_params* params); + + +/* ZSTD_writeLastEmptyBlock() : + * output an empty Block with end-of-frame mark to complete a frame + * @return : size of data written into `dst` (== ZSTD_blockHeaderSize (defined in zstd_internal.h)) + * or an error code if `dstCapacity` is too small ( 1 */ +U32 ZSTD_cycleLog(U32 hashLog, ZSTD_strategy strat); + +#endif /* ZSTD_COMPRESS_H */ diff --git a/module/zstd/lib/compress/zstd_compress_literals.c b/module/zstd/lib/compress/zstd_compress_literals.c new file mode 100644 index 000000000000..17e7168d8936 --- /dev/null +++ b/module/zstd/lib/compress/zstd_compress_literals.c @@ -0,0 +1,158 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + + /*-************************************* + * Dependencies + ***************************************/ +#include "zstd_compress_literals.h" + +size_t ZSTD_noCompressLiterals (void* dst, size_t dstCapacity, const void* src, size_t srcSize) +{ + BYTE* const ostart = (BYTE* const)dst; + U32 const flSize = 1 + (srcSize>31) + (srcSize>4095); + + RETURN_ERROR_IF(srcSize + flSize > dstCapacity, dstSize_tooSmall, ""); + + switch(flSize) + { + case 1: /* 2 - 1 - 5 */ + ostart[0] = (BYTE)((U32)set_basic + (srcSize<<3)); + break; + case 2: /* 2 - 2 - 12 */ + MEM_writeLE16(ostart, (U16)((U32)set_basic + (1<<2) + (srcSize<<4))); + break; + case 3: /* 2 - 2 - 20 */ + MEM_writeLE32(ostart, (U32)((U32)set_basic + (3<<2) + (srcSize<<4))); + break; + default: /* not necessary : flSize is {1,2,3} */ + assert(0); + } + + memcpy(ostart + flSize, src, srcSize); + DEBUGLOG(5, "Raw literals: %u -> %u", (U32)srcSize, (U32)(srcSize + flSize)); + return srcSize + flSize; +} + +size_t ZSTD_compressRleLiteralsBlock (void* dst, size_t dstCapacity, const void* src, size_t srcSize) +{ + BYTE* const ostart = (BYTE* const)dst; + U32 const flSize = 1 + (srcSize>31) + (srcSize>4095); + + (void)dstCapacity; /* dstCapacity already guaranteed to be >=4, hence large enough */ + + switch(flSize) + { + case 1: /* 2 - 1 - 5 */ + ostart[0] = (BYTE)((U32)set_rle + (srcSize<<3)); + break; + case 2: /* 2 - 2 - 12 */ + MEM_writeLE16(ostart, (U16)((U32)set_rle + (1<<2) + (srcSize<<4))); + break; + case 3: /* 2 - 2 - 20 */ + MEM_writeLE32(ostart, (U32)((U32)set_rle + (3<<2) + (srcSize<<4))); + break; + default: /* not necessary : flSize is {1,2,3} */ + assert(0); + } + + ostart[flSize] = *(const BYTE*)src; + DEBUGLOG(5, "RLE literals: %u -> %u", (U32)srcSize, (U32)flSize + 1); + return flSize+1; +} + +size_t ZSTD_compressLiterals (ZSTD_hufCTables_t const* prevHuf, + ZSTD_hufCTables_t* nextHuf, + ZSTD_strategy strategy, int disableLiteralCompression, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + void* entropyWorkspace, size_t entropyWorkspaceSize, + const int bmi2) +{ + size_t const minGain = ZSTD_minGain(srcSize, strategy); + size_t const lhSize = 3 + (srcSize >= 1 KB) + (srcSize >= 16 KB); + BYTE* const ostart = (BYTE*)dst; + U32 singleStream = srcSize < 256; + symbolEncodingType_e hType = set_compressed; + size_t cLitSize; + + DEBUGLOG(5,"ZSTD_compressLiterals (disableLiteralCompression=%i srcSize=%u)", + disableLiteralCompression, (U32)srcSize); + + /* Prepare nextEntropy assuming reusing the existing table */ + memcpy(nextHuf, prevHuf, sizeof(*prevHuf)); + + if (disableLiteralCompression) + return ZSTD_noCompressLiterals(dst, dstCapacity, src, srcSize); + + /* small ? don't even attempt compression (speed opt) */ +# define COMPRESS_LITERALS_SIZE_MIN 63 + { size_t const minLitSize = (prevHuf->repeatMode == HUF_repeat_valid) ? 6 : COMPRESS_LITERALS_SIZE_MIN; + if (srcSize <= minLitSize) return ZSTD_noCompressLiterals(dst, dstCapacity, src, srcSize); + } + + RETURN_ERROR_IF(dstCapacity < lhSize+1, dstSize_tooSmall, "not enough space for compression"); + { HUF_repeat repeat = prevHuf->repeatMode; + int const preferRepeat = strategy < ZSTD_lazy ? srcSize <= 1024 : 0; + if (repeat == HUF_repeat_valid && lhSize == 3) singleStream = 1; + cLitSize = singleStream ? + HUF_compress1X_repeat( + ostart+lhSize, dstCapacity-lhSize, src, srcSize, + HUF_SYMBOLVALUE_MAX, HUF_TABLELOG_DEFAULT, entropyWorkspace, entropyWorkspaceSize, + (HUF_CElt*)nextHuf->CTable, &repeat, preferRepeat, bmi2) : + HUF_compress4X_repeat( + ostart+lhSize, dstCapacity-lhSize, src, srcSize, + HUF_SYMBOLVALUE_MAX, HUF_TABLELOG_DEFAULT, entropyWorkspace, entropyWorkspaceSize, + (HUF_CElt*)nextHuf->CTable, &repeat, preferRepeat, bmi2); + if (repeat != HUF_repeat_none) { + /* reused the existing table */ + DEBUGLOG(5, "Reusing previous huffman table"); + hType = set_repeat; + } + } + + if ((cLitSize==0) | (cLitSize >= srcSize - minGain) | ERR_isError(cLitSize)) { + memcpy(nextHuf, prevHuf, sizeof(*prevHuf)); + return ZSTD_noCompressLiterals(dst, dstCapacity, src, srcSize); + } + if (cLitSize==1) { + memcpy(nextHuf, prevHuf, sizeof(*prevHuf)); + return ZSTD_compressRleLiteralsBlock(dst, dstCapacity, src, srcSize); + } + + if (hType == set_compressed) { + /* using a newly constructed table */ + nextHuf->repeatMode = HUF_repeat_check; + } + + /* Build header */ + switch(lhSize) + { + case 3: /* 2 - 2 - 10 - 10 */ + { U32 const lhc = hType + ((!singleStream) << 2) + ((U32)srcSize<<4) + ((U32)cLitSize<<14); + MEM_writeLE24(ostart, lhc); + break; + } + case 4: /* 2 - 2 - 14 - 14 */ + { U32 const lhc = hType + (2 << 2) + ((U32)srcSize<<4) + ((U32)cLitSize<<18); + MEM_writeLE32(ostart, lhc); + break; + } + case 5: /* 2 - 2 - 18 - 18 */ + { U32 const lhc = hType + (3 << 2) + ((U32)srcSize<<4) + ((U32)cLitSize<<22); + MEM_writeLE32(ostart, lhc); + ostart[4] = (BYTE)(cLitSize >> 10); + break; + } + default: /* not possible : lhSize is {3,4,5} */ + assert(0); + } + DEBUGLOG(5, "Compressed literals: %u -> %u", (U32)srcSize, (U32)(lhSize+cLitSize)); + return lhSize+cLitSize; +} diff --git a/module/zstd/lib/compress/zstd_compress_literals.h b/module/zstd/lib/compress/zstd_compress_literals.h new file mode 100644 index 000000000000..8b0870574326 --- /dev/null +++ b/module/zstd/lib/compress/zstd_compress_literals.h @@ -0,0 +1,29 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#ifndef ZSTD_COMPRESS_LITERALS_H +#define ZSTD_COMPRESS_LITERALS_H + +#include "zstd_compress_internal.h" /* ZSTD_hufCTables_t, ZSTD_minGain() */ + + +size_t ZSTD_noCompressLiterals (void* dst, size_t dstCapacity, const void* src, size_t srcSize); + +size_t ZSTD_compressRleLiteralsBlock (void* dst, size_t dstCapacity, const void* src, size_t srcSize); + +size_t ZSTD_compressLiterals (ZSTD_hufCTables_t const* prevHuf, + ZSTD_hufCTables_t* nextHuf, + ZSTD_strategy strategy, int disableLiteralCompression, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + void* entropyWorkspace, size_t entropyWorkspaceSize, + const int bmi2); + +#endif /* ZSTD_COMPRESS_LITERALS_H */ diff --git a/module/zstd/lib/compress/zstd_compress_sequences.c b/module/zstd/lib/compress/zstd_compress_sequences.c new file mode 100644 index 000000000000..f9f8097c839b --- /dev/null +++ b/module/zstd/lib/compress/zstd_compress_sequences.c @@ -0,0 +1,419 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + + /*-************************************* + * Dependencies + ***************************************/ +#include "zstd_compress_sequences.h" + +/** + * -log2(x / 256) lookup table for x in [0, 256). + * If x == 0: Return 0 + * Else: Return floor(-log2(x / 256) * 256) + */ +static unsigned const kInverseProbabilityLog256[256] = { + 0, 2048, 1792, 1642, 1536, 1453, 1386, 1329, 1280, 1236, 1197, 1162, + 1130, 1100, 1073, 1047, 1024, 1001, 980, 960, 941, 923, 906, 889, + 874, 859, 844, 830, 817, 804, 791, 779, 768, 756, 745, 734, + 724, 714, 704, 694, 685, 676, 667, 658, 650, 642, 633, 626, + 618, 610, 603, 595, 588, 581, 574, 567, 561, 554, 548, 542, + 535, 529, 523, 517, 512, 506, 500, 495, 489, 484, 478, 473, + 468, 463, 458, 453, 448, 443, 438, 434, 429, 424, 420, 415, + 411, 407, 402, 398, 394, 390, 386, 382, 377, 373, 370, 366, + 362, 358, 354, 350, 347, 343, 339, 336, 332, 329, 325, 322, + 318, 315, 311, 308, 305, 302, 298, 295, 292, 289, 286, 282, + 279, 276, 273, 270, 267, 264, 261, 258, 256, 253, 250, 247, + 244, 241, 239, 236, 233, 230, 228, 225, 222, 220, 217, 215, + 212, 209, 207, 204, 202, 199, 197, 194, 192, 190, 187, 185, + 182, 180, 178, 175, 173, 171, 168, 166, 164, 162, 159, 157, + 155, 153, 151, 149, 146, 144, 142, 140, 138, 136, 134, 132, + 130, 128, 126, 123, 121, 119, 117, 115, 114, 112, 110, 108, + 106, 104, 102, 100, 98, 96, 94, 93, 91, 89, 87, 85, + 83, 82, 80, 78, 76, 74, 73, 71, 69, 67, 66, 64, + 62, 61, 59, 57, 55, 54, 52, 50, 49, 47, 46, 44, + 42, 41, 39, 37, 36, 34, 33, 31, 30, 28, 26, 25, + 23, 22, 20, 19, 17, 16, 14, 13, 11, 10, 8, 7, + 5, 4, 2, 1, +}; + +static unsigned ZSTD_getFSEMaxSymbolValue(FSE_CTable const* ctable) { + void const* ptr = ctable; + U16 const* u16ptr = (U16 const*)ptr; + U32 const maxSymbolValue = MEM_read16(u16ptr + 1); + return maxSymbolValue; +} + +/** + * Returns the cost in bytes of encoding the normalized count header. + * Returns an error if any of the helper functions return an error. + */ +static size_t ZSTD_NCountCost(unsigned const* count, unsigned const max, + size_t const nbSeq, unsigned const FSELog) +{ + BYTE wksp[FSE_NCOUNTBOUND]; + S16 norm[MaxSeq + 1]; + const U32 tableLog = FSE_optimalTableLog(FSELog, nbSeq, max); + FORWARD_IF_ERROR(FSE_normalizeCount(norm, tableLog, count, nbSeq, max), ""); + return FSE_writeNCount(wksp, sizeof(wksp), norm, max, tableLog); +} + +/** + * Returns the cost in bits of encoding the distribution described by count + * using the entropy bound. + */ +static size_t ZSTD_entropyCost(unsigned const* count, unsigned const max, size_t const total) +{ + unsigned cost = 0; + unsigned s; + for (s = 0; s <= max; ++s) { + unsigned norm = (unsigned)((256 * count[s]) / total); + if (count[s] != 0 && norm == 0) + norm = 1; + assert(count[s] < total); + cost += count[s] * kInverseProbabilityLog256[norm]; + } + return cost >> 8; +} + +/** + * Returns the cost in bits of encoding the distribution in count using ctable. + * Returns an error if ctable cannot represent all the symbols in count. + */ +size_t ZSTD_fseBitCost( + FSE_CTable const* ctable, + unsigned const* count, + unsigned const max) +{ + unsigned const kAccuracyLog = 8; + size_t cost = 0; + unsigned s; + FSE_CState_t cstate; + FSE_initCState(&cstate, ctable); + if (ZSTD_getFSEMaxSymbolValue(ctable) < max) { + DEBUGLOG(5, "Repeat FSE_CTable has maxSymbolValue %u < %u", + ZSTD_getFSEMaxSymbolValue(ctable), max); + return ERROR(GENERIC); + } + for (s = 0; s <= max; ++s) { + unsigned const tableLog = cstate.stateLog; + unsigned const badCost = (tableLog + 1) << kAccuracyLog; + unsigned const bitCost = FSE_bitCost(cstate.symbolTT, tableLog, s, kAccuracyLog); + if (count[s] == 0) + continue; + if (bitCost >= badCost) { + DEBUGLOG(5, "Repeat FSE_CTable has Prob[%u] == 0", s); + return ERROR(GENERIC); + } + cost += (size_t)count[s] * bitCost; + } + return cost >> kAccuracyLog; +} + +/** + * Returns the cost in bits of encoding the distribution in count using the + * table described by norm. The max symbol support by norm is assumed >= max. + * norm must be valid for every symbol with non-zero probability in count. + */ +size_t ZSTD_crossEntropyCost(short const* norm, unsigned accuracyLog, + unsigned const* count, unsigned const max) +{ + unsigned const shift = 8 - accuracyLog; + size_t cost = 0; + unsigned s; + assert(accuracyLog <= 8); + for (s = 0; s <= max; ++s) { + unsigned const normAcc = (norm[s] != -1) ? (unsigned)norm[s] : 1; + unsigned const norm256 = normAcc << shift; + assert(norm256 > 0); + assert(norm256 < 256); + cost += count[s] * kInverseProbabilityLog256[norm256]; + } + return cost >> 8; +} + +symbolEncodingType_e +ZSTD_selectEncodingType( + FSE_repeat* repeatMode, unsigned const* count, unsigned const max, + size_t const mostFrequent, size_t nbSeq, unsigned const FSELog, + FSE_CTable const* prevCTable, + short const* defaultNorm, U32 defaultNormLog, + ZSTD_defaultPolicy_e const isDefaultAllowed, + ZSTD_strategy const strategy) +{ + ZSTD_STATIC_ASSERT(ZSTD_defaultDisallowed == 0 && ZSTD_defaultAllowed != 0); + if (mostFrequent == nbSeq) { + *repeatMode = FSE_repeat_none; + if (isDefaultAllowed && nbSeq <= 2) { + /* Prefer set_basic over set_rle when there are 2 or less symbols, + * since RLE uses 1 byte, but set_basic uses 5-6 bits per symbol. + * If basic encoding isn't possible, always choose RLE. + */ + DEBUGLOG(5, "Selected set_basic"); + return set_basic; + } + DEBUGLOG(5, "Selected set_rle"); + return set_rle; + } + if (strategy < ZSTD_lazy) { + if (isDefaultAllowed) { + size_t const staticFse_nbSeq_max = 1000; + size_t const mult = 10 - strategy; + size_t const baseLog = 3; + size_t const dynamicFse_nbSeq_min = (((size_t)1 << defaultNormLog) * mult) >> baseLog; /* 28-36 for offset, 56-72 for lengths */ + assert(defaultNormLog >= 5 && defaultNormLog <= 6); /* xx_DEFAULTNORMLOG */ + assert(mult <= 9 && mult >= 7); + if ( (*repeatMode == FSE_repeat_valid) + && (nbSeq < staticFse_nbSeq_max) ) { + DEBUGLOG(5, "Selected set_repeat"); + return set_repeat; + } + if ( (nbSeq < dynamicFse_nbSeq_min) + || (mostFrequent < (nbSeq >> (defaultNormLog-1))) ) { + DEBUGLOG(5, "Selected set_basic"); + /* The format allows default tables to be repeated, but it isn't useful. + * When using simple heuristics to select encoding type, we don't want + * to confuse these tables with dictionaries. When running more careful + * analysis, we don't need to waste time checking both repeating tables + * and default tables. + */ + *repeatMode = FSE_repeat_none; + return set_basic; + } + } + } else { + size_t const basicCost = isDefaultAllowed ? ZSTD_crossEntropyCost(defaultNorm, defaultNormLog, count, max) : ERROR(GENERIC); + size_t const repeatCost = *repeatMode != FSE_repeat_none ? ZSTD_fseBitCost(prevCTable, count, max) : ERROR(GENERIC); + size_t const NCountCost = ZSTD_NCountCost(count, max, nbSeq, FSELog); + size_t const compressedCost = (NCountCost << 3) + ZSTD_entropyCost(count, max, nbSeq); + + if (isDefaultAllowed) { + assert(!ZSTD_isError(basicCost)); + assert(!(*repeatMode == FSE_repeat_valid && ZSTD_isError(repeatCost))); + } + assert(!ZSTD_isError(NCountCost)); + assert(compressedCost < ERROR(maxCode)); + DEBUGLOG(5, "Estimated bit costs: basic=%u\trepeat=%u\tcompressed=%u", + (unsigned)basicCost, (unsigned)repeatCost, (unsigned)compressedCost); + if (basicCost <= repeatCost && basicCost <= compressedCost) { + DEBUGLOG(5, "Selected set_basic"); + assert(isDefaultAllowed); + *repeatMode = FSE_repeat_none; + return set_basic; + } + if (repeatCost <= compressedCost) { + DEBUGLOG(5, "Selected set_repeat"); + assert(!ZSTD_isError(repeatCost)); + return set_repeat; + } + assert(compressedCost < basicCost && compressedCost < repeatCost); + } + DEBUGLOG(5, "Selected set_compressed"); + *repeatMode = FSE_repeat_check; + return set_compressed; +} + +size_t +ZSTD_buildCTable(void* dst, size_t dstCapacity, + FSE_CTable* nextCTable, U32 FSELog, symbolEncodingType_e type, + unsigned* count, U32 max, + const BYTE* codeTable, size_t nbSeq, + const S16* defaultNorm, U32 defaultNormLog, U32 defaultMax, + const FSE_CTable* prevCTable, size_t prevCTableSize, + void* entropyWorkspace, size_t entropyWorkspaceSize) +{ + BYTE* op = (BYTE*)dst; + const BYTE* const oend = op + dstCapacity; + DEBUGLOG(6, "ZSTD_buildCTable (dstCapacity=%u)", (unsigned)dstCapacity); + + switch (type) { + case set_rle: + FORWARD_IF_ERROR(FSE_buildCTable_rle(nextCTable, (BYTE)max), ""); + RETURN_ERROR_IF(dstCapacity==0, dstSize_tooSmall, "not enough space"); + *op = codeTable[0]; + return 1; + case set_repeat: + memcpy(nextCTable, prevCTable, prevCTableSize); + return 0; + case set_basic: + FORWARD_IF_ERROR(FSE_buildCTable_wksp(nextCTable, defaultNorm, defaultMax, defaultNormLog, entropyWorkspace, entropyWorkspaceSize), ""); /* note : could be pre-calculated */ + return 0; + case set_compressed: { + S16 norm[MaxSeq + 1]; + size_t nbSeq_1 = nbSeq; + const U32 tableLog = FSE_optimalTableLog(FSELog, nbSeq, max); + if (count[codeTable[nbSeq-1]] > 1) { + count[codeTable[nbSeq-1]]--; + nbSeq_1--; + } + assert(nbSeq_1 > 1); + FORWARD_IF_ERROR(FSE_normalizeCount(norm, tableLog, count, nbSeq_1, max), ""); + { size_t const NCountSize = FSE_writeNCount(op, oend - op, norm, max, tableLog); /* overflow protected */ + FORWARD_IF_ERROR(NCountSize, "FSE_writeNCount failed"); + FORWARD_IF_ERROR(FSE_buildCTable_wksp(nextCTable, norm, max, tableLog, entropyWorkspace, entropyWorkspaceSize), ""); + return NCountSize; + } + } + default: assert(0); RETURN_ERROR(GENERIC, "impossible to reach"); + } +} + +FORCE_INLINE_TEMPLATE size_t +ZSTD_encodeSequences_body( + void* dst, size_t dstCapacity, + FSE_CTable const* CTable_MatchLength, BYTE const* mlCodeTable, + FSE_CTable const* CTable_OffsetBits, BYTE const* ofCodeTable, + FSE_CTable const* CTable_LitLength, BYTE const* llCodeTable, + seqDef const* sequences, size_t nbSeq, int longOffsets) +{ + BIT_CStream_t blockStream; + FSE_CState_t stateMatchLength; + FSE_CState_t stateOffsetBits; + FSE_CState_t stateLitLength; + + RETURN_ERROR_IF( + ERR_isError(BIT_initCStream(&blockStream, dst, dstCapacity)), + dstSize_tooSmall, "not enough space remaining"); + DEBUGLOG(6, "available space for bitstream : %i (dstCapacity=%u)", + (int)(blockStream.endPtr - blockStream.startPtr), + (unsigned)dstCapacity); + + /* first symbols */ + FSE_initCState2(&stateMatchLength, CTable_MatchLength, mlCodeTable[nbSeq-1]); + FSE_initCState2(&stateOffsetBits, CTable_OffsetBits, ofCodeTable[nbSeq-1]); + FSE_initCState2(&stateLitLength, CTable_LitLength, llCodeTable[nbSeq-1]); + BIT_addBits(&blockStream, sequences[nbSeq-1].litLength, LL_bits[llCodeTable[nbSeq-1]]); + if (MEM_32bits()) BIT_flushBits(&blockStream); + BIT_addBits(&blockStream, sequences[nbSeq-1].matchLength, ML_bits[mlCodeTable[nbSeq-1]]); + if (MEM_32bits()) BIT_flushBits(&blockStream); + if (longOffsets) { + U32 const ofBits = ofCodeTable[nbSeq-1]; + unsigned const extraBits = ofBits - MIN(ofBits, STREAM_ACCUMULATOR_MIN-1); + if (extraBits) { + BIT_addBits(&blockStream, sequences[nbSeq-1].offset, extraBits); + BIT_flushBits(&blockStream); + } + BIT_addBits(&blockStream, sequences[nbSeq-1].offset >> extraBits, + ofBits - extraBits); + } else { + BIT_addBits(&blockStream, sequences[nbSeq-1].offset, ofCodeTable[nbSeq-1]); + } + BIT_flushBits(&blockStream); + + { size_t n; + for (n=nbSeq-2 ; n= 64-7-(LLFSELog+MLFSELog+OffFSELog))) + BIT_flushBits(&blockStream); /* (7)*/ + BIT_addBits(&blockStream, sequences[n].litLength, llBits); + if (MEM_32bits() && ((llBits+mlBits)>24)) BIT_flushBits(&blockStream); + BIT_addBits(&blockStream, sequences[n].matchLength, mlBits); + if (MEM_32bits() || (ofBits+mlBits+llBits > 56)) BIT_flushBits(&blockStream); + if (longOffsets) { + unsigned const extraBits = ofBits - MIN(ofBits, STREAM_ACCUMULATOR_MIN-1); + if (extraBits) { + BIT_addBits(&blockStream, sequences[n].offset, extraBits); + BIT_flushBits(&blockStream); /* (7)*/ + } + BIT_addBits(&blockStream, sequences[n].offset >> extraBits, + ofBits - extraBits); /* 31 */ + } else { + BIT_addBits(&blockStream, sequences[n].offset, ofBits); /* 31 */ + } + BIT_flushBits(&blockStream); /* (7)*/ + DEBUGLOG(7, "remaining space : %i", (int)(blockStream.endPtr - blockStream.ptr)); + } } + + DEBUGLOG(6, "ZSTD_encodeSequences: flushing ML state with %u bits", stateMatchLength.stateLog); + FSE_flushCState(&blockStream, &stateMatchLength); + DEBUGLOG(6, "ZSTD_encodeSequences: flushing Off state with %u bits", stateOffsetBits.stateLog); + FSE_flushCState(&blockStream, &stateOffsetBits); + DEBUGLOG(6, "ZSTD_encodeSequences: flushing LL state with %u bits", stateLitLength.stateLog); + FSE_flushCState(&blockStream, &stateLitLength); + + { size_t const streamSize = BIT_closeCStream(&blockStream); + RETURN_ERROR_IF(streamSize==0, dstSize_tooSmall, "not enough space"); + return streamSize; + } +} + +static size_t +ZSTD_encodeSequences_default( + void* dst, size_t dstCapacity, + FSE_CTable const* CTable_MatchLength, BYTE const* mlCodeTable, + FSE_CTable const* CTable_OffsetBits, BYTE const* ofCodeTable, + FSE_CTable const* CTable_LitLength, BYTE const* llCodeTable, + seqDef const* sequences, size_t nbSeq, int longOffsets) +{ + return ZSTD_encodeSequences_body(dst, dstCapacity, + CTable_MatchLength, mlCodeTable, + CTable_OffsetBits, ofCodeTable, + CTable_LitLength, llCodeTable, + sequences, nbSeq, longOffsets); +} + + +#if DYNAMIC_BMI2 + +static TARGET_ATTRIBUTE("bmi2") size_t +ZSTD_encodeSequences_bmi2( + void* dst, size_t dstCapacity, + FSE_CTable const* CTable_MatchLength, BYTE const* mlCodeTable, + FSE_CTable const* CTable_OffsetBits, BYTE const* ofCodeTable, + FSE_CTable const* CTable_LitLength, BYTE const* llCodeTable, + seqDef const* sequences, size_t nbSeq, int longOffsets) +{ + return ZSTD_encodeSequences_body(dst, dstCapacity, + CTable_MatchLength, mlCodeTable, + CTable_OffsetBits, ofCodeTable, + CTable_LitLength, llCodeTable, + sequences, nbSeq, longOffsets); +} + +#endif + +size_t ZSTD_encodeSequences( + void* dst, size_t dstCapacity, + FSE_CTable const* CTable_MatchLength, BYTE const* mlCodeTable, + FSE_CTable const* CTable_OffsetBits, BYTE const* ofCodeTable, + FSE_CTable const* CTable_LitLength, BYTE const* llCodeTable, + seqDef const* sequences, size_t nbSeq, int longOffsets, int bmi2) +{ + DEBUGLOG(5, "ZSTD_encodeSequences: dstCapacity = %u", (unsigned)dstCapacity); +#if DYNAMIC_BMI2 + if (bmi2) { + return ZSTD_encodeSequences_bmi2(dst, dstCapacity, + CTable_MatchLength, mlCodeTable, + CTable_OffsetBits, ofCodeTable, + CTable_LitLength, llCodeTable, + sequences, nbSeq, longOffsets); + } +#endif + (void)bmi2; + return ZSTD_encodeSequences_default(dst, dstCapacity, + CTable_MatchLength, mlCodeTable, + CTable_OffsetBits, ofCodeTable, + CTable_LitLength, llCodeTable, + sequences, nbSeq, longOffsets); +} diff --git a/module/zstd/lib/compress/zstd_compress_sequences.h b/module/zstd/lib/compress/zstd_compress_sequences.h new file mode 100644 index 000000000000..68c6f9a5acd8 --- /dev/null +++ b/module/zstd/lib/compress/zstd_compress_sequences.h @@ -0,0 +1,54 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#ifndef ZSTD_COMPRESS_SEQUENCES_H +#define ZSTD_COMPRESS_SEQUENCES_H + +#include "../common/fse.h" /* FSE_repeat, FSE_CTable */ +#include "../common/zstd_internal.h" /* symbolEncodingType_e, ZSTD_strategy */ + +typedef enum { + ZSTD_defaultDisallowed = 0, + ZSTD_defaultAllowed = 1 +} ZSTD_defaultPolicy_e; + +symbolEncodingType_e +ZSTD_selectEncodingType( + FSE_repeat* repeatMode, unsigned const* count, unsigned const max, + size_t const mostFrequent, size_t nbSeq, unsigned const FSELog, + FSE_CTable const* prevCTable, + short const* defaultNorm, U32 defaultNormLog, + ZSTD_defaultPolicy_e const isDefaultAllowed, + ZSTD_strategy const strategy); + +size_t +ZSTD_buildCTable(void* dst, size_t dstCapacity, + FSE_CTable* nextCTable, U32 FSELog, symbolEncodingType_e type, + unsigned* count, U32 max, + const BYTE* codeTable, size_t nbSeq, + const S16* defaultNorm, U32 defaultNormLog, U32 defaultMax, + const FSE_CTable* prevCTable, size_t prevCTableSize, + void* entropyWorkspace, size_t entropyWorkspaceSize); + +size_t ZSTD_encodeSequences( + void* dst, size_t dstCapacity, + FSE_CTable const* CTable_MatchLength, BYTE const* mlCodeTable, + FSE_CTable const* CTable_OffsetBits, BYTE const* ofCodeTable, + FSE_CTable const* CTable_LitLength, BYTE const* llCodeTable, + seqDef const* sequences, size_t nbSeq, int longOffsets, int bmi2); + +size_t ZSTD_fseBitCost( + FSE_CTable const* ctable, + unsigned const* count, + unsigned const max); + +size_t ZSTD_crossEntropyCost(short const* norm, unsigned accuracyLog, + unsigned const* count, unsigned const max); +#endif /* ZSTD_COMPRESS_SEQUENCES_H */ diff --git a/module/zstd/lib/compress/zstd_compress_superblock.c b/module/zstd/lib/compress/zstd_compress_superblock.c new file mode 100644 index 000000000000..ffa4bb67597f --- /dev/null +++ b/module/zstd/lib/compress/zstd_compress_superblock.c @@ -0,0 +1,845 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + + /*-************************************* + * Dependencies + ***************************************/ +#include "zstd_compress_superblock.h" + +#include "../common/zstd_internal.h" /* ZSTD_getSequenceLength */ +#include "hist.h" /* HIST_countFast_wksp */ +#include "zstd_compress_internal.h" +#include "zstd_compress_sequences.h" +#include "zstd_compress_literals.h" + +/*-************************************* +* Superblock entropy buffer structs +***************************************/ +/** ZSTD_hufCTablesMetadata_t : + * Stores Literals Block Type for a super-block in hType, and + * huffman tree description in hufDesBuffer. + * hufDesSize refers to the size of huffman tree description in bytes. + * This metadata is populated in ZSTD_buildSuperBlockEntropy_literal() */ +typedef struct { + symbolEncodingType_e hType; + BYTE hufDesBuffer[500]; /* TODO give name to this value */ + size_t hufDesSize; +} ZSTD_hufCTablesMetadata_t; + +/** ZSTD_fseCTablesMetadata_t : + * Stores symbol compression modes for a super-block in {ll, ol, ml}Type, and + * fse tables in fseTablesBuffer. + * fseTablesSize refers to the size of fse tables in bytes. + * This metadata is populated in ZSTD_buildSuperBlockEntropy_sequences() */ +typedef struct { + symbolEncodingType_e llType; + symbolEncodingType_e ofType; + symbolEncodingType_e mlType; + BYTE fseTablesBuffer[500]; /* TODO give name to this value */ + size_t fseTablesSize; + size_t lastCountSize; /* This is to account for bug in 1.3.4. More detail in ZSTD_compressSubBlock_sequences() */ +} ZSTD_fseCTablesMetadata_t; + +typedef struct { + ZSTD_hufCTablesMetadata_t hufMetadata; + ZSTD_fseCTablesMetadata_t fseMetadata; +} ZSTD_entropyCTablesMetadata_t; + + +/** ZSTD_buildSuperBlockEntropy_literal() : + * Builds entropy for the super-block literals. + * Stores literals block type (raw, rle, compressed, repeat) and + * huffman description table to hufMetadata. + * @return : size of huffman description table or error code */ +static size_t ZSTD_buildSuperBlockEntropy_literal(void* const src, size_t srcSize, + const ZSTD_hufCTables_t* prevHuf, + ZSTD_hufCTables_t* nextHuf, + ZSTD_hufCTablesMetadata_t* hufMetadata, + const int disableLiteralsCompression, + void* workspace, size_t wkspSize) +{ + BYTE* const wkspStart = (BYTE*)workspace; + BYTE* const wkspEnd = wkspStart + wkspSize; + BYTE* const countWkspStart = wkspStart; + unsigned* const countWksp = (unsigned*)workspace; + const size_t countWkspSize = (HUF_SYMBOLVALUE_MAX + 1) * sizeof(unsigned); + BYTE* const nodeWksp = countWkspStart + countWkspSize; + const size_t nodeWkspSize = wkspEnd-nodeWksp; + unsigned maxSymbolValue = 255; + unsigned huffLog = HUF_TABLELOG_DEFAULT; + HUF_repeat repeat = prevHuf->repeatMode; + + DEBUGLOG(5, "ZSTD_buildSuperBlockEntropy_literal (srcSize=%zu)", srcSize); + + /* Prepare nextEntropy assuming reusing the existing table */ + memcpy(nextHuf, prevHuf, sizeof(*prevHuf)); + + if (disableLiteralsCompression) { + DEBUGLOG(5, "set_basic - disabled"); + hufMetadata->hType = set_basic; + return 0; + } + + /* small ? don't even attempt compression (speed opt) */ +# define COMPRESS_LITERALS_SIZE_MIN 63 + { size_t const minLitSize = (prevHuf->repeatMode == HUF_repeat_valid) ? 6 : COMPRESS_LITERALS_SIZE_MIN; + if (srcSize <= minLitSize) { + DEBUGLOG(5, "set_basic - too small"); + hufMetadata->hType = set_basic; + return 0; + } + } + + /* Scan input and build symbol stats */ + { size_t const largest = HIST_count_wksp (countWksp, &maxSymbolValue, (const BYTE*)src, srcSize, workspace, wkspSize); + FORWARD_IF_ERROR(largest, "HIST_count_wksp failed"); + if (largest == srcSize) { + DEBUGLOG(5, "set_rle"); + hufMetadata->hType = set_rle; + return 0; + } + if (largest <= (srcSize >> 7)+4) { + DEBUGLOG(5, "set_basic - no gain"); + hufMetadata->hType = set_basic; + return 0; + } + } + + /* Validate the previous Huffman table */ + if (repeat == HUF_repeat_check && !HUF_validateCTable((HUF_CElt const*)prevHuf->CTable, countWksp, maxSymbolValue)) { + repeat = HUF_repeat_none; + } + + /* Build Huffman Tree */ + memset(nextHuf->CTable, 0, sizeof(nextHuf->CTable)); + huffLog = HUF_optimalTableLog(huffLog, srcSize, maxSymbolValue); + { size_t const maxBits = HUF_buildCTable_wksp((HUF_CElt*)nextHuf->CTable, countWksp, + maxSymbolValue, huffLog, + nodeWksp, nodeWkspSize); + FORWARD_IF_ERROR(maxBits, "HUF_buildCTable_wksp"); + huffLog = (U32)maxBits; + { /* Build and write the CTable */ + size_t const newCSize = HUF_estimateCompressedSize( + (HUF_CElt*)nextHuf->CTable, countWksp, maxSymbolValue); + size_t const hSize = HUF_writeCTable( + hufMetadata->hufDesBuffer, sizeof(hufMetadata->hufDesBuffer), + (HUF_CElt*)nextHuf->CTable, maxSymbolValue, huffLog); + /* Check against repeating the previous CTable */ + if (repeat != HUF_repeat_none) { + size_t const oldCSize = HUF_estimateCompressedSize( + (HUF_CElt const*)prevHuf->CTable, countWksp, maxSymbolValue); + if (oldCSize < srcSize && (oldCSize <= hSize + newCSize || hSize + 12 >= srcSize)) { + DEBUGLOG(5, "set_repeat - smaller"); + memcpy(nextHuf, prevHuf, sizeof(*prevHuf)); + hufMetadata->hType = set_repeat; + return 0; + } + } + if (newCSize + hSize >= srcSize) { + DEBUGLOG(5, "set_basic - no gains"); + memcpy(nextHuf, prevHuf, sizeof(*prevHuf)); + hufMetadata->hType = set_basic; + return 0; + } + DEBUGLOG(5, "set_compressed (hSize=%u)", (U32)hSize); + hufMetadata->hType = set_compressed; + nextHuf->repeatMode = HUF_repeat_check; + return hSize; + } + } +} + +/** ZSTD_buildSuperBlockEntropy_sequences() : + * Builds entropy for the super-block sequences. + * Stores symbol compression modes and fse table to fseMetadata. + * @return : size of fse tables or error code */ +static size_t ZSTD_buildSuperBlockEntropy_sequences(seqStore_t* seqStorePtr, + const ZSTD_fseCTables_t* prevEntropy, + ZSTD_fseCTables_t* nextEntropy, + const ZSTD_CCtx_params* cctxParams, + ZSTD_fseCTablesMetadata_t* fseMetadata, + void* workspace, size_t wkspSize) +{ + BYTE* const wkspStart = (BYTE*)workspace; + BYTE* const wkspEnd = wkspStart + wkspSize; + BYTE* const countWkspStart = wkspStart; + unsigned* const countWksp = (unsigned*)workspace; + const size_t countWkspSize = (MaxSeq + 1) * sizeof(unsigned); + BYTE* const cTableWksp = countWkspStart + countWkspSize; + const size_t cTableWkspSize = wkspEnd-cTableWksp; + ZSTD_strategy const strategy = cctxParams->cParams.strategy; + FSE_CTable* CTable_LitLength = nextEntropy->litlengthCTable; + FSE_CTable* CTable_OffsetBits = nextEntropy->offcodeCTable; + FSE_CTable* CTable_MatchLength = nextEntropy->matchlengthCTable; + const BYTE* const ofCodeTable = seqStorePtr->ofCode; + const BYTE* const llCodeTable = seqStorePtr->llCode; + const BYTE* const mlCodeTable = seqStorePtr->mlCode; + size_t const nbSeq = seqStorePtr->sequences - seqStorePtr->sequencesStart; + BYTE* const ostart = fseMetadata->fseTablesBuffer; + BYTE* const oend = ostart + sizeof(fseMetadata->fseTablesBuffer); + BYTE* op = ostart; + + assert(cTableWkspSize >= (1 << MaxFSELog) * sizeof(FSE_FUNCTION_TYPE)); + DEBUGLOG(5, "ZSTD_buildSuperBlockEntropy_sequences (nbSeq=%zu)", nbSeq); + memset(workspace, 0, wkspSize); + + fseMetadata->lastCountSize = 0; + /* convert length/distances into codes */ + ZSTD_seqToCodes(seqStorePtr); + /* build CTable for Literal Lengths */ + { U32 LLtype; + unsigned max = MaxLL; + size_t const mostFrequent = HIST_countFast_wksp(countWksp, &max, llCodeTable, nbSeq, workspace, wkspSize); /* can't fail */ + DEBUGLOG(5, "Building LL table"); + nextEntropy->litlength_repeatMode = prevEntropy->litlength_repeatMode; + LLtype = ZSTD_selectEncodingType(&nextEntropy->litlength_repeatMode, + countWksp, max, mostFrequent, nbSeq, + LLFSELog, prevEntropy->litlengthCTable, + LL_defaultNorm, LL_defaultNormLog, + ZSTD_defaultAllowed, strategy); + assert(set_basic < set_compressed && set_rle < set_compressed); + assert(!(LLtype < set_compressed && nextEntropy->litlength_repeatMode != FSE_repeat_none)); /* We don't copy tables */ + { size_t const countSize = ZSTD_buildCTable(op, oend - op, CTable_LitLength, LLFSELog, (symbolEncodingType_e)LLtype, + countWksp, max, llCodeTable, nbSeq, LL_defaultNorm, LL_defaultNormLog, MaxLL, + prevEntropy->litlengthCTable, sizeof(prevEntropy->litlengthCTable), + cTableWksp, cTableWkspSize); + FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for LitLens failed"); + if (LLtype == set_compressed) + fseMetadata->lastCountSize = countSize; + op += countSize; + fseMetadata->llType = (symbolEncodingType_e) LLtype; + } } + /* build CTable for Offsets */ + { U32 Offtype; + unsigned max = MaxOff; + size_t const mostFrequent = HIST_countFast_wksp(countWksp, &max, ofCodeTable, nbSeq, workspace, wkspSize); /* can't fail */ + /* We can only use the basic table if max <= DefaultMaxOff, otherwise the offsets are too large */ + ZSTD_defaultPolicy_e const defaultPolicy = (max <= DefaultMaxOff) ? ZSTD_defaultAllowed : ZSTD_defaultDisallowed; + DEBUGLOG(5, "Building OF table"); + nextEntropy->offcode_repeatMode = prevEntropy->offcode_repeatMode; + Offtype = ZSTD_selectEncodingType(&nextEntropy->offcode_repeatMode, + countWksp, max, mostFrequent, nbSeq, + OffFSELog, prevEntropy->offcodeCTable, + OF_defaultNorm, OF_defaultNormLog, + defaultPolicy, strategy); + assert(!(Offtype < set_compressed && nextEntropy->offcode_repeatMode != FSE_repeat_none)); /* We don't copy tables */ + { size_t const countSize = ZSTD_buildCTable(op, oend - op, CTable_OffsetBits, OffFSELog, (symbolEncodingType_e)Offtype, + countWksp, max, ofCodeTable, nbSeq, OF_defaultNorm, OF_defaultNormLog, DefaultMaxOff, + prevEntropy->offcodeCTable, sizeof(prevEntropy->offcodeCTable), + cTableWksp, cTableWkspSize); + FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for Offsets failed"); + if (Offtype == set_compressed) + fseMetadata->lastCountSize = countSize; + op += countSize; + fseMetadata->ofType = (symbolEncodingType_e) Offtype; + } } + /* build CTable for MatchLengths */ + { U32 MLtype; + unsigned max = MaxML; + size_t const mostFrequent = HIST_countFast_wksp(countWksp, &max, mlCodeTable, nbSeq, workspace, wkspSize); /* can't fail */ + DEBUGLOG(5, "Building ML table (remaining space : %i)", (int)(oend-op)); + nextEntropy->matchlength_repeatMode = prevEntropy->matchlength_repeatMode; + MLtype = ZSTD_selectEncodingType(&nextEntropy->matchlength_repeatMode, + countWksp, max, mostFrequent, nbSeq, + MLFSELog, prevEntropy->matchlengthCTable, + ML_defaultNorm, ML_defaultNormLog, + ZSTD_defaultAllowed, strategy); + assert(!(MLtype < set_compressed && nextEntropy->matchlength_repeatMode != FSE_repeat_none)); /* We don't copy tables */ + { size_t const countSize = ZSTD_buildCTable(op, oend - op, CTable_MatchLength, MLFSELog, (symbolEncodingType_e)MLtype, + countWksp, max, mlCodeTable, nbSeq, ML_defaultNorm, ML_defaultNormLog, MaxML, + prevEntropy->matchlengthCTable, sizeof(prevEntropy->matchlengthCTable), + cTableWksp, cTableWkspSize); + FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for MatchLengths failed"); + if (MLtype == set_compressed) + fseMetadata->lastCountSize = countSize; + op += countSize; + fseMetadata->mlType = (symbolEncodingType_e) MLtype; + } } + assert((size_t) (op-ostart) <= sizeof(fseMetadata->fseTablesBuffer)); + return op-ostart; +} + + +/** ZSTD_buildSuperBlockEntropy() : + * Builds entropy for the super-block. + * @return : 0 on success or error code */ +static size_t +ZSTD_buildSuperBlockEntropy(seqStore_t* seqStorePtr, + const ZSTD_entropyCTables_t* prevEntropy, + ZSTD_entropyCTables_t* nextEntropy, + const ZSTD_CCtx_params* cctxParams, + ZSTD_entropyCTablesMetadata_t* entropyMetadata, + void* workspace, size_t wkspSize) +{ + size_t const litSize = seqStorePtr->lit - seqStorePtr->litStart; + DEBUGLOG(5, "ZSTD_buildSuperBlockEntropy"); + entropyMetadata->hufMetadata.hufDesSize = + ZSTD_buildSuperBlockEntropy_literal(seqStorePtr->litStart, litSize, + &prevEntropy->huf, &nextEntropy->huf, + &entropyMetadata->hufMetadata, + ZSTD_disableLiteralsCompression(cctxParams), + workspace, wkspSize); + FORWARD_IF_ERROR(entropyMetadata->hufMetadata.hufDesSize, "ZSTD_buildSuperBlockEntropy_literal failed"); + entropyMetadata->fseMetadata.fseTablesSize = + ZSTD_buildSuperBlockEntropy_sequences(seqStorePtr, + &prevEntropy->fse, &nextEntropy->fse, + cctxParams, + &entropyMetadata->fseMetadata, + workspace, wkspSize); + FORWARD_IF_ERROR(entropyMetadata->fseMetadata.fseTablesSize, "ZSTD_buildSuperBlockEntropy_sequences failed"); + return 0; +} + +/** ZSTD_compressSubBlock_literal() : + * Compresses literals section for a sub-block. + * When we have to write the Huffman table we will sometimes choose a header + * size larger than necessary. This is because we have to pick the header size + * before we know the table size + compressed size, so we have a bound on the + * table size. If we guessed incorrectly, we fall back to uncompressed literals. + * + * We write the header when writeEntropy=1 and set entropyWrriten=1 when we succeeded + * in writing the header, otherwise it is set to 0. + * + * hufMetadata->hType has literals block type info. + * If it is set_basic, all sub-blocks literals section will be Raw_Literals_Block. + * If it is set_rle, all sub-blocks literals section will be RLE_Literals_Block. + * If it is set_compressed, first sub-block's literals section will be Compressed_Literals_Block + * If it is set_compressed, first sub-block's literals section will be Treeless_Literals_Block + * and the following sub-blocks' literals sections will be Treeless_Literals_Block. + * @return : compressed size of literals section of a sub-block + * Or 0 if it unable to compress. + * Or error code */ +static size_t ZSTD_compressSubBlock_literal(const HUF_CElt* hufTable, + const ZSTD_hufCTablesMetadata_t* hufMetadata, + const BYTE* literals, size_t litSize, + void* dst, size_t dstSize, + const int bmi2, int writeEntropy, int* entropyWritten) +{ + size_t const header = writeEntropy ? 200 : 0; + size_t const lhSize = 3 + (litSize >= (1 KB - header)) + (litSize >= (16 KB - header)); + BYTE* const ostart = (BYTE*)dst; + BYTE* const oend = ostart + dstSize; + BYTE* op = ostart + lhSize; + U32 const singleStream = lhSize == 3; + symbolEncodingType_e hType = writeEntropy ? hufMetadata->hType : set_repeat; + size_t cLitSize = 0; + + (void)bmi2; /* TODO bmi2... */ + + DEBUGLOG(5, "ZSTD_compressSubBlock_literal (litSize=%zu, lhSize=%zu, writeEntropy=%d)", litSize, lhSize, writeEntropy); + + *entropyWritten = 0; + if (litSize == 0 || hufMetadata->hType == set_basic) { + DEBUGLOG(5, "ZSTD_compressSubBlock_literal using raw literal"); + return ZSTD_noCompressLiterals(dst, dstSize, literals, litSize); + } else if (hufMetadata->hType == set_rle) { + DEBUGLOG(5, "ZSTD_compressSubBlock_literal using rle literal"); + return ZSTD_compressRleLiteralsBlock(dst, dstSize, literals, litSize); + } + + assert(litSize > 0); + assert(hufMetadata->hType == set_compressed || hufMetadata->hType == set_repeat); + + if (writeEntropy && hufMetadata->hType == set_compressed) { + memcpy(op, hufMetadata->hufDesBuffer, hufMetadata->hufDesSize); + op += hufMetadata->hufDesSize; + cLitSize += hufMetadata->hufDesSize; + DEBUGLOG(5, "ZSTD_compressSubBlock_literal (hSize=%zu)", hufMetadata->hufDesSize); + } + + /* TODO bmi2 */ + { const size_t cSize = singleStream ? HUF_compress1X_usingCTable(op, oend-op, literals, litSize, hufTable) + : HUF_compress4X_usingCTable(op, oend-op, literals, litSize, hufTable); + op += cSize; + cLitSize += cSize; + if (cSize == 0 || ERR_isError(cSize)) { + DEBUGLOG(5, "Failed to write entropy tables %s", ZSTD_getErrorName(cSize)); + return 0; + } + /* If we expand and we aren't writing a header then emit uncompressed */ + if (!writeEntropy && cLitSize >= litSize) { + DEBUGLOG(5, "ZSTD_compressSubBlock_literal using raw literal because uncompressible"); + return ZSTD_noCompressLiterals(dst, dstSize, literals, litSize); + } + /* If we are writing headers then allow expansion that doesn't change our header size. */ + if (lhSize < (size_t)(3 + (cLitSize >= 1 KB) + (cLitSize >= 16 KB))) { + assert(cLitSize > litSize); + DEBUGLOG(5, "Literals expanded beyond allowed header size"); + return ZSTD_noCompressLiterals(dst, dstSize, literals, litSize); + } + DEBUGLOG(5, "ZSTD_compressSubBlock_literal (cSize=%zu)", cSize); + } + + /* Build header */ + switch(lhSize) + { + case 3: /* 2 - 2 - 10 - 10 */ + { U32 const lhc = hType + ((!singleStream) << 2) + ((U32)litSize<<4) + ((U32)cLitSize<<14); + MEM_writeLE24(ostart, lhc); + break; + } + case 4: /* 2 - 2 - 14 - 14 */ + { U32 const lhc = hType + (2 << 2) + ((U32)litSize<<4) + ((U32)cLitSize<<18); + MEM_writeLE32(ostart, lhc); + break; + } + case 5: /* 2 - 2 - 18 - 18 */ + { U32 const lhc = hType + (3 << 2) + ((U32)litSize<<4) + ((U32)cLitSize<<22); + MEM_writeLE32(ostart, lhc); + ostart[4] = (BYTE)(cLitSize >> 10); + break; + } + default: /* not possible : lhSize is {3,4,5} */ + assert(0); + } + *entropyWritten = 1; + DEBUGLOG(5, "Compressed literals: %u -> %u", (U32)litSize, (U32)(op-ostart)); + return op-ostart; +} + +static size_t ZSTD_seqDecompressedSize(seqStore_t const* seqStore, const seqDef* sequences, size_t nbSeq, size_t litSize, int lastSequence) { + const seqDef* const sstart = sequences; + const seqDef* const send = sequences + nbSeq; + const seqDef* sp = sstart; + size_t matchLengthSum = 0; + size_t litLengthSum __attribute__ ((unused)) = 0; + while (send-sp > 0) { + ZSTD_sequenceLength const seqLen = ZSTD_getSequenceLength(seqStore, sp); + litLengthSum += seqLen.litLength; + matchLengthSum += seqLen.matchLength; + sp++; + } + assert(litLengthSum <= litSize); + if (!lastSequence) { + assert(litLengthSum == litSize); + } + return matchLengthSum + litSize; +} + +/** ZSTD_compressSubBlock_sequences() : + * Compresses sequences section for a sub-block. + * fseMetadata->llType, fseMetadata->ofType, and fseMetadata->mlType have + * symbol compression modes for the super-block. + * The first successfully compressed block will have these in its header. + * We set entropyWritten=1 when we succeed in compressing the sequences. + * The following sub-blocks will always have repeat mode. + * @return : compressed size of sequences section of a sub-block + * Or 0 if it is unable to compress + * Or error code. */ +static size_t ZSTD_compressSubBlock_sequences(const ZSTD_fseCTables_t* fseTables, + const ZSTD_fseCTablesMetadata_t* fseMetadata, + const seqDef* sequences, size_t nbSeq, + const BYTE* llCode, const BYTE* mlCode, const BYTE* ofCode, + const ZSTD_CCtx_params* cctxParams, + void* dst, size_t dstCapacity, + const int bmi2, int writeEntropy, int* entropyWritten) +{ + const int longOffsets = cctxParams->cParams.windowLog > STREAM_ACCUMULATOR_MIN; + BYTE* const ostart = (BYTE*)dst; + BYTE* const oend = ostart + dstCapacity; + BYTE* op = ostart; + BYTE* seqHead; + + DEBUGLOG(5, "ZSTD_compressSubBlock_sequences (nbSeq=%zu, writeEntropy=%d, longOffsets=%d)", nbSeq, writeEntropy, longOffsets); + + *entropyWritten = 0; + /* Sequences Header */ + RETURN_ERROR_IF((oend-op) < 3 /*max nbSeq Size*/ + 1 /*seqHead*/, + dstSize_tooSmall, ""); + if (nbSeq < 0x7F) + *op++ = (BYTE)nbSeq; + else if (nbSeq < LONGNBSEQ) + op[0] = (BYTE)((nbSeq>>8) + 0x80), op[1] = (BYTE)nbSeq, op+=2; + else + op[0]=0xFF, MEM_writeLE16(op+1, (U16)(nbSeq - LONGNBSEQ)), op+=3; + if (nbSeq==0) { + return op - ostart; + } + + /* seqHead : flags for FSE encoding type */ + seqHead = op++; + + DEBUGLOG(5, "ZSTD_compressSubBlock_sequences (seqHeadSize=%u)", (unsigned)(op-ostart)); + + if (writeEntropy) { + const U32 LLtype = fseMetadata->llType; + const U32 Offtype = fseMetadata->ofType; + const U32 MLtype = fseMetadata->mlType; + DEBUGLOG(5, "ZSTD_compressSubBlock_sequences (fseTablesSize=%zu)", fseMetadata->fseTablesSize); + *seqHead = (BYTE)((LLtype<<6) + (Offtype<<4) + (MLtype<<2)); + memcpy(op, fseMetadata->fseTablesBuffer, fseMetadata->fseTablesSize); + op += fseMetadata->fseTablesSize; + } else { + const U32 repeat = set_repeat; + *seqHead = (BYTE)((repeat<<6) + (repeat<<4) + (repeat<<2)); + } + + { size_t const bitstreamSize = ZSTD_encodeSequences( + op, oend - op, + fseTables->matchlengthCTable, mlCode, + fseTables->offcodeCTable, ofCode, + fseTables->litlengthCTable, llCode, + sequences, nbSeq, + longOffsets, bmi2); + FORWARD_IF_ERROR(bitstreamSize, "ZSTD_encodeSequences failed"); + op += bitstreamSize; + /* zstd versions <= 1.3.4 mistakenly report corruption when + * FSE_readNCount() receives a buffer < 4 bytes. + * Fixed by https://github.com/facebook/zstd/pull/1146. + * This can happen when the last set_compressed table present is 2 + * bytes and the bitstream is only one byte. + * In this exceedingly rare case, we will simply emit an uncompressed + * block, since it isn't worth optimizing. + */ +#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION + if (writeEntropy && fseMetadata->lastCountSize && fseMetadata->lastCountSize + bitstreamSize < 4) { + /* NCountSize >= 2 && bitstreamSize > 0 ==> lastCountSize == 3 */ + assert(fseMetadata->lastCountSize + bitstreamSize == 3); + DEBUGLOG(5, "Avoiding bug in zstd decoder in versions <= 1.3.4 by " + "emitting an uncompressed block."); + return 0; + } +#endif + DEBUGLOG(5, "ZSTD_compressSubBlock_sequences (bitstreamSize=%zu)", bitstreamSize); + } + + /* zstd versions <= 1.4.0 mistakenly report error when + * sequences section body size is less than 3 bytes. + * Fixed by https://github.com/facebook/zstd/pull/1664. + * This can happen when the previous sequences section block is compressed + * with rle mode and the current block's sequences section is compressed + * with repeat mode where sequences section body size can be 1 byte. + */ +#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION + if (op-seqHead < 4) { + DEBUGLOG(5, "Avoiding bug in zstd decoder in versions <= 1.4.0 by emitting " + "an uncompressed block when sequences are < 4 bytes"); + return 0; + } +#endif + + *entropyWritten = 1; + return op - ostart; +} + +/** ZSTD_compressSubBlock() : + * Compresses a single sub-block. + * @return : compressed size of the sub-block + * Or 0 if it failed to compress. */ +static size_t ZSTD_compressSubBlock(const ZSTD_entropyCTables_t* entropy, + const ZSTD_entropyCTablesMetadata_t* entropyMetadata, + const seqDef* sequences, size_t nbSeq, + const BYTE* literals, size_t litSize, + const BYTE* llCode, const BYTE* mlCode, const BYTE* ofCode, + const ZSTD_CCtx_params* cctxParams, + void* dst, size_t dstCapacity, + const int bmi2, + int writeLitEntropy, int writeSeqEntropy, + int* litEntropyWritten, int* seqEntropyWritten, + U32 lastBlock) +{ + BYTE* const ostart = (BYTE*)dst; + BYTE* const oend = ostart + dstCapacity; + BYTE* op = ostart + ZSTD_blockHeaderSize; + DEBUGLOG(5, "ZSTD_compressSubBlock (litSize=%zu, nbSeq=%zu, writeLitEntropy=%d, writeSeqEntropy=%d, lastBlock=%d)", + litSize, nbSeq, writeLitEntropy, writeSeqEntropy, lastBlock); + { size_t cLitSize = ZSTD_compressSubBlock_literal((const HUF_CElt*)entropy->huf.CTable, + &entropyMetadata->hufMetadata, literals, litSize, + op, oend-op, bmi2, writeLitEntropy, litEntropyWritten); + FORWARD_IF_ERROR(cLitSize, "ZSTD_compressSubBlock_literal failed"); + if (cLitSize == 0) return 0; + op += cLitSize; + } + { size_t cSeqSize = ZSTD_compressSubBlock_sequences(&entropy->fse, + &entropyMetadata->fseMetadata, + sequences, nbSeq, + llCode, mlCode, ofCode, + cctxParams, + op, oend-op, + bmi2, writeSeqEntropy, seqEntropyWritten); + FORWARD_IF_ERROR(cSeqSize, "ZSTD_compressSubBlock_sequences failed"); + if (cSeqSize == 0) return 0; + op += cSeqSize; + } + /* Write block header */ + { size_t cSize = (op-ostart)-ZSTD_blockHeaderSize; + U32 const cBlockHeader24 = lastBlock + (((U32)bt_compressed)<<1) + (U32)(cSize << 3); + MEM_writeLE24(ostart, cBlockHeader24); + } + return op-ostart; +} + +static size_t ZSTD_estimateSubBlockSize_literal(const BYTE* literals, size_t litSize, + const ZSTD_hufCTables_t* huf, + const ZSTD_hufCTablesMetadata_t* hufMetadata, + void* workspace, size_t wkspSize, + int writeEntropy) +{ + unsigned* const countWksp = (unsigned*)workspace; + unsigned maxSymbolValue = 255; + size_t literalSectionHeaderSize = 3; /* Use hard coded size of 3 bytes */ + + if (hufMetadata->hType == set_basic) return litSize; + else if (hufMetadata->hType == set_rle) return 1; + else if (hufMetadata->hType == set_compressed || hufMetadata->hType == set_repeat) { + size_t const largest = HIST_count_wksp (countWksp, &maxSymbolValue, (const BYTE*)literals, litSize, workspace, wkspSize); + if (ZSTD_isError(largest)) return litSize; + { size_t cLitSizeEstimate = HUF_estimateCompressedSize((const HUF_CElt*)huf->CTable, countWksp, maxSymbolValue); + if (writeEntropy) cLitSizeEstimate += hufMetadata->hufDesSize; + return cLitSizeEstimate + literalSectionHeaderSize; + } } + assert(0); /* impossible */ + return 0; +} + +static size_t ZSTD_estimateSubBlockSize_symbolType(symbolEncodingType_e type, + const BYTE* codeTable, unsigned maxCode, + size_t nbSeq, const FSE_CTable* fseCTable, + const U32* additionalBits, + short const* defaultNorm, U32 defaultNormLog, + void* workspace, size_t wkspSize) +{ + unsigned* const countWksp = (unsigned*)workspace; + const BYTE* ctp = codeTable; + const BYTE* const ctStart = ctp; + const BYTE* const ctEnd = ctStart + nbSeq; + size_t cSymbolTypeSizeEstimateInBits = 0; + unsigned max = maxCode; + + HIST_countFast_wksp(countWksp, &max, codeTable, nbSeq, workspace, wkspSize); /* can't fail */ + if (type == set_basic) { + cSymbolTypeSizeEstimateInBits = ZSTD_crossEntropyCost(defaultNorm, defaultNormLog, countWksp, max); + } else if (type == set_rle) { + cSymbolTypeSizeEstimateInBits = 0; + } else if (type == set_compressed || type == set_repeat) { + cSymbolTypeSizeEstimateInBits = ZSTD_fseBitCost(fseCTable, countWksp, max); + } + if (ZSTD_isError(cSymbolTypeSizeEstimateInBits)) return nbSeq * 10; + while (ctp < ctEnd) { + if (additionalBits) cSymbolTypeSizeEstimateInBits += additionalBits[*ctp]; + else cSymbolTypeSizeEstimateInBits += *ctp; /* for offset, offset code is also the number of additional bits */ + ctp++; + } + return cSymbolTypeSizeEstimateInBits / 8; +} + +static size_t ZSTD_estimateSubBlockSize_sequences(const BYTE* ofCodeTable, + const BYTE* llCodeTable, + const BYTE* mlCodeTable, + size_t nbSeq, + const ZSTD_fseCTables_t* fseTables, + const ZSTD_fseCTablesMetadata_t* fseMetadata, + void* workspace, size_t wkspSize, + int writeEntropy) +{ + size_t sequencesSectionHeaderSize = 3; /* Use hard coded size of 3 bytes */ + size_t cSeqSizeEstimate = 0; + cSeqSizeEstimate += ZSTD_estimateSubBlockSize_symbolType(fseMetadata->ofType, ofCodeTable, MaxOff, + nbSeq, fseTables->offcodeCTable, NULL, + OF_defaultNorm, OF_defaultNormLog, + workspace, wkspSize); + cSeqSizeEstimate += ZSTD_estimateSubBlockSize_symbolType(fseMetadata->llType, llCodeTable, MaxLL, + nbSeq, fseTables->litlengthCTable, LL_bits, + LL_defaultNorm, LL_defaultNormLog, + workspace, wkspSize); + cSeqSizeEstimate += ZSTD_estimateSubBlockSize_symbolType(fseMetadata->mlType, mlCodeTable, MaxML, + nbSeq, fseTables->matchlengthCTable, ML_bits, + ML_defaultNorm, ML_defaultNormLog, + workspace, wkspSize); + if (writeEntropy) cSeqSizeEstimate += fseMetadata->fseTablesSize; + return cSeqSizeEstimate + sequencesSectionHeaderSize; +} + +static size_t ZSTD_estimateSubBlockSize(const BYTE* literals, size_t litSize, + const BYTE* ofCodeTable, + const BYTE* llCodeTable, + const BYTE* mlCodeTable, + size_t nbSeq, + const ZSTD_entropyCTables_t* entropy, + const ZSTD_entropyCTablesMetadata_t* entropyMetadata, + void* workspace, size_t wkspSize, + int writeLitEntropy, int writeSeqEntropy) { + size_t cSizeEstimate = 0; + cSizeEstimate += ZSTD_estimateSubBlockSize_literal(literals, litSize, + &entropy->huf, &entropyMetadata->hufMetadata, + workspace, wkspSize, writeLitEntropy); + cSizeEstimate += ZSTD_estimateSubBlockSize_sequences(ofCodeTable, llCodeTable, mlCodeTable, + nbSeq, &entropy->fse, &entropyMetadata->fseMetadata, + workspace, wkspSize, writeSeqEntropy); + return cSizeEstimate + ZSTD_blockHeaderSize; +} + +static int ZSTD_needSequenceEntropyTables(ZSTD_fseCTablesMetadata_t const* fseMetadata) +{ + if (fseMetadata->llType == set_compressed || fseMetadata->llType == set_rle) + return 1; + if (fseMetadata->mlType == set_compressed || fseMetadata->mlType == set_rle) + return 1; + if (fseMetadata->ofType == set_compressed || fseMetadata->ofType == set_rle) + return 1; + return 0; +} + +/** ZSTD_compressSubBlock_multi() : + * Breaks super-block into multiple sub-blocks and compresses them. + * Entropy will be written to the first block. + * The following blocks will use repeat mode to compress. + * All sub-blocks are compressed blocks (no raw or rle blocks). + * @return : compressed size of the super block (which is multiple ZSTD blocks) + * Or 0 if it failed to compress. */ +static size_t ZSTD_compressSubBlock_multi(const seqStore_t* seqStorePtr, + const ZSTD_compressedBlockState_t* prevCBlock, + ZSTD_compressedBlockState_t* nextCBlock, + const ZSTD_entropyCTablesMetadata_t* entropyMetadata, + const ZSTD_CCtx_params* cctxParams, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + const int bmi2, U32 lastBlock, + void* workspace, size_t wkspSize) +{ + const seqDef* const sstart = seqStorePtr->sequencesStart; + const seqDef* const send = seqStorePtr->sequences; + const seqDef* sp = sstart; + const BYTE* const lstart = seqStorePtr->litStart; + const BYTE* const lend = seqStorePtr->lit; + const BYTE* lp = lstart; + BYTE const* ip = (BYTE const*)src; + BYTE const* const iend = ip + srcSize; + BYTE* const ostart = (BYTE*)dst; + BYTE* const oend = ostart + dstCapacity; + BYTE* op = ostart; + const BYTE* llCodePtr = seqStorePtr->llCode; + const BYTE* mlCodePtr = seqStorePtr->mlCode; + const BYTE* ofCodePtr = seqStorePtr->ofCode; + size_t targetCBlockSize = cctxParams->targetCBlockSize; + size_t litSize, seqCount; + int writeLitEntropy = entropyMetadata->hufMetadata.hType == set_compressed; + int writeSeqEntropy = 1; + int lastSequence = 0; + + DEBUGLOG(5, "ZSTD_compressSubBlock_multi (litSize=%u, nbSeq=%u)", + (unsigned)(lend-lp), (unsigned)(send-sstart)); + + litSize = 0; + seqCount = 0; + do { + size_t cBlockSizeEstimate = 0; + if (sstart == send) { + lastSequence = 1; + } else { + const seqDef* const sequence = sp + seqCount; + lastSequence = sequence == send - 1; + litSize += ZSTD_getSequenceLength(seqStorePtr, sequence).litLength; + seqCount++; + } + if (lastSequence) { + assert(lp <= lend); + assert(litSize <= (size_t)(lend - lp)); + litSize = (size_t)(lend - lp); + } + /* I think there is an optimization opportunity here. + * Calling ZSTD_estimateSubBlockSize for every sequence can be wasteful + * since it recalculates estimate from scratch. + * For example, it would recount literal distribution and symbol codes everytime. + */ + cBlockSizeEstimate = ZSTD_estimateSubBlockSize(lp, litSize, ofCodePtr, llCodePtr, mlCodePtr, seqCount, + &nextCBlock->entropy, entropyMetadata, + workspace, wkspSize, writeLitEntropy, writeSeqEntropy); + if (cBlockSizeEstimate > targetCBlockSize || lastSequence) { + int litEntropyWritten = 0; + int seqEntropyWritten = 0; + const size_t decompressedSize = ZSTD_seqDecompressedSize(seqStorePtr, sp, seqCount, litSize, lastSequence); + const size_t cSize = ZSTD_compressSubBlock(&nextCBlock->entropy, entropyMetadata, + sp, seqCount, + lp, litSize, + llCodePtr, mlCodePtr, ofCodePtr, + cctxParams, + op, oend-op, + bmi2, writeLitEntropy, writeSeqEntropy, + &litEntropyWritten, &seqEntropyWritten, + lastBlock && lastSequence); + FORWARD_IF_ERROR(cSize, "ZSTD_compressSubBlock failed"); + if (cSize > 0 && cSize < decompressedSize) { + DEBUGLOG(5, "Committed the sub-block"); + assert(ip + decompressedSize <= iend); + ip += decompressedSize; + sp += seqCount; + lp += litSize; + op += cSize; + llCodePtr += seqCount; + mlCodePtr += seqCount; + ofCodePtr += seqCount; + litSize = 0; + seqCount = 0; + /* Entropy only needs to be written once */ + if (litEntropyWritten) { + writeLitEntropy = 0; + } + if (seqEntropyWritten) { + writeSeqEntropy = 0; + } + } + } + } while (!lastSequence); + if (writeLitEntropy) { + DEBUGLOG(5, "ZSTD_compressSubBlock_multi has literal entropy tables unwritten"); + memcpy(&nextCBlock->entropy.huf, &prevCBlock->entropy.huf, sizeof(prevCBlock->entropy.huf)); + } + if (writeSeqEntropy && ZSTD_needSequenceEntropyTables(&entropyMetadata->fseMetadata)) { + /* If we haven't written our entropy tables, then we've violated our contract and + * must emit an uncompressed block. + */ + DEBUGLOG(5, "ZSTD_compressSubBlock_multi has sequence entropy tables unwritten"); + return 0; + } + if (ip < iend) { + size_t const cSize = ZSTD_noCompressBlock(op, oend - op, ip, iend - ip, lastBlock); + DEBUGLOG(5, "ZSTD_compressSubBlock_multi last sub-block uncompressed, %zu bytes", (size_t)(iend - ip)); + FORWARD_IF_ERROR(cSize, "ZSTD_noCompressBlock failed"); + assert(cSize != 0); + op += cSize; + /* We have to regenerate the repcodes because we've skipped some sequences */ + if (sp < send) { + seqDef const* seq; + repcodes_t rep; + memcpy(&rep, prevCBlock->rep, sizeof(rep)); + for (seq = sstart; seq < sp; ++seq) { + rep = ZSTD_updateRep(rep.rep, seq->offset - 1, ZSTD_getSequenceLength(seqStorePtr, seq).litLength == 0); + } + memcpy(nextCBlock->rep, &rep, sizeof(rep)); + } + } + DEBUGLOG(5, "ZSTD_compressSubBlock_multi compressed"); + return op-ostart; +} + +size_t ZSTD_compressSuperBlock(ZSTD_CCtx* zc, + void* dst, size_t dstCapacity, + void const* src, size_t srcSize, + unsigned lastBlock) { + ZSTD_entropyCTablesMetadata_t entropyMetadata; + + FORWARD_IF_ERROR(ZSTD_buildSuperBlockEntropy(&zc->seqStore, + &zc->blockState.prevCBlock->entropy, + &zc->blockState.nextCBlock->entropy, + &zc->appliedParams, + &entropyMetadata, + zc->entropyWorkspace, HUF_WORKSPACE_SIZE /* statically allocated in resetCCtx */), ""); + + return ZSTD_compressSubBlock_multi(&zc->seqStore, + zc->blockState.prevCBlock, + zc->blockState.nextCBlock, + &entropyMetadata, + &zc->appliedParams, + dst, dstCapacity, + src, srcSize, + zc->bmi2, lastBlock, + zc->entropyWorkspace, HUF_WORKSPACE_SIZE /* statically allocated in resetCCtx */); +} diff --git a/module/zstd/lib/compress/zstd_compress_superblock.h b/module/zstd/lib/compress/zstd_compress_superblock.h new file mode 100644 index 000000000000..07f4cb1dc646 --- /dev/null +++ b/module/zstd/lib/compress/zstd_compress_superblock.h @@ -0,0 +1,32 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#ifndef ZSTD_COMPRESS_ADVANCED_H +#define ZSTD_COMPRESS_ADVANCED_H + +/*-************************************* +* Dependencies +***************************************/ + +#include "../zstd.h" /* ZSTD_CCtx */ + +/*-************************************* +* Target Compressed Block Size +***************************************/ + +/* ZSTD_compressSuperBlock() : + * Used to compress a super block when targetCBlockSize is being used. + * The given block will be compressed into multiple sub blocks that are around targetCBlockSize. */ +size_t ZSTD_compressSuperBlock(ZSTD_CCtx* zc, + void* dst, size_t dstCapacity, + void const* src, size_t srcSize, + unsigned lastBlock); + +#endif /* ZSTD_COMPRESS_ADVANCED_H */ diff --git a/module/zstd/lib/compress/zstd_cwksp.h b/module/zstd/lib/compress/zstd_cwksp.h new file mode 100644 index 000000000000..a25c9263b7d7 --- /dev/null +++ b/module/zstd/lib/compress/zstd_cwksp.h @@ -0,0 +1,525 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#ifndef ZSTD_CWKSP_H +#define ZSTD_CWKSP_H + +/*-************************************* +* Dependencies +***************************************/ +#include "../common/zstd_internal.h" + +#if defined (__cplusplus) +extern "C" { +#endif + +/*-************************************* +* Constants +***************************************/ + +/* Since the workspace is effectively its own little malloc implementation / + * arena, when we run under ASAN, we should similarly insert redzones between + * each internal element of the workspace, so ASAN will catch overruns that + * reach outside an object but that stay inside the workspace. + * + * This defines the size of that redzone. + */ +#ifndef ZSTD_CWKSP_ASAN_REDZONE_SIZE +#define ZSTD_CWKSP_ASAN_REDZONE_SIZE 128 +#endif + +/*-************************************* +* Structures +***************************************/ +typedef enum { + ZSTD_cwksp_alloc_objects, + ZSTD_cwksp_alloc_buffers, + ZSTD_cwksp_alloc_aligned +} ZSTD_cwksp_alloc_phase_e; + +/** + * Zstd fits all its internal datastructures into a single continuous buffer, + * so that it only needs to perform a single OS allocation (or so that a buffer + * can be provided to it and it can perform no allocations at all). This buffer + * is called the workspace. + * + * Several optimizations complicate that process of allocating memory ranges + * from this workspace for each internal datastructure: + * + * - These different internal datastructures have different setup requirements: + * + * - The static objects need to be cleared once and can then be trivially + * reused for each compression. + * + * - Various buffers don't need to be initialized at all--they are always + * written into before they're read. + * + * - The matchstate tables have a unique requirement that they don't need + * their memory to be totally cleared, but they do need the memory to have + * some bound, i.e., a guarantee that all values in the memory they've been + * allocated is less than some maximum value (which is the starting value + * for the indices that they will then use for compression). When this + * guarantee is provided to them, they can use the memory without any setup + * work. When it can't, they have to clear the area. + * + * - These buffers also have different alignment requirements. + * + * - We would like to reuse the objects in the workspace for multiple + * compressions without having to perform any expensive reallocation or + * reinitialization work. + * + * - We would like to be able to efficiently reuse the workspace across + * multiple compressions **even when the compression parameters change** and + * we need to resize some of the objects (where possible). + * + * To attempt to manage this buffer, given these constraints, the ZSTD_cwksp + * abstraction was created. It works as follows: + * + * Workspace Layout: + * + * [ ... workspace ... ] + * [objects][tables ... ->] free space [<- ... aligned][<- ... buffers] + * + * The various objects that live in the workspace are divided into the + * following categories, and are allocated separately: + * + * - Static objects: this is optionally the enclosing ZSTD_CCtx or ZSTD_CDict, + * so that literally everything fits in a single buffer. Note: if present, + * this must be the first object in the workspace, since ZSTD_free{CCtx, + * CDict}() rely on a pointer comparison to see whether one or two frees are + * required. + * + * - Fixed size objects: these are fixed-size, fixed-count objects that are + * nonetheless "dynamically" allocated in the workspace so that we can + * control how they're initialized separately from the broader ZSTD_CCtx. + * Examples: + * - Entropy Workspace + * - 2 x ZSTD_compressedBlockState_t + * - CDict dictionary contents + * + * - Tables: these are any of several different datastructures (hash tables, + * chain tables, binary trees) that all respect a common format: they are + * uint32_t arrays, all of whose values are between 0 and (nextSrc - base). + * Their sizes depend on the cparams. + * + * - Aligned: these buffers are used for various purposes that require 4 byte + * alignment, but don't require any initialization before they're used. + * + * - Buffers: these buffers are used for various purposes that don't require + * any alignment or initialization before they're used. This means they can + * be moved around at no cost for a new compression. + * + * Allocating Memory: + * + * The various types of objects must be allocated in order, so they can be + * correctly packed into the workspace buffer. That order is: + * + * 1. Objects + * 2. Buffers + * 3. Aligned + * 4. Tables + * + * Attempts to reserve objects of different types out of order will fail. + */ +typedef struct { + void* workspace; + void* workspaceEnd; + + void* objectEnd; + void* tableEnd; + void* tableValidEnd; + void* allocStart; + + int allocFailed; + int workspaceOversizedDuration; + ZSTD_cwksp_alloc_phase_e phase; +} ZSTD_cwksp; + +/*-************************************* +* Functions +***************************************/ + +MEM_STATIC size_t ZSTD_cwksp_available_space(ZSTD_cwksp* ws); + +MEM_STATIC void ZSTD_cwksp_assert_internal_consistency(ZSTD_cwksp* ws) { + (void)ws; + assert(ws->workspace <= ws->objectEnd); + assert(ws->objectEnd <= ws->tableEnd); + assert(ws->objectEnd <= ws->tableValidEnd); + assert(ws->tableEnd <= ws->allocStart); + assert(ws->tableValidEnd <= ws->allocStart); + assert(ws->allocStart <= ws->workspaceEnd); +} + +/** + * Align must be a power of 2. + */ +MEM_STATIC size_t ZSTD_cwksp_align(size_t size, size_t const align) { + size_t const mask = align - 1; + assert((align & mask) == 0); + return (size + mask) & ~mask; +} + +/** + * Use this to determine how much space in the workspace we will consume to + * allocate this object. (Normally it should be exactly the size of the object, + * but under special conditions, like ASAN, where we pad each object, it might + * be larger.) + * + * Since tables aren't currently redzoned, you don't need to call through this + * to figure out how much space you need for the matchState tables. Everything + * else is though. + */ +MEM_STATIC size_t ZSTD_cwksp_alloc_size(size_t size) { +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) + return size + 2 * ZSTD_CWKSP_ASAN_REDZONE_SIZE; +#else + return size; +#endif +} + +MEM_STATIC void ZSTD_cwksp_internal_advance_phase( + ZSTD_cwksp* ws, ZSTD_cwksp_alloc_phase_e phase) { + assert(phase >= ws->phase); + if (phase > ws->phase) { + if (ws->phase < ZSTD_cwksp_alloc_buffers && + phase >= ZSTD_cwksp_alloc_buffers) { + ws->tableValidEnd = ws->objectEnd; + } + if (ws->phase < ZSTD_cwksp_alloc_aligned && + phase >= ZSTD_cwksp_alloc_aligned) { + /* If unaligned allocations down from a too-large top have left us + * unaligned, we need to realign our alloc ptr. Technically, this + * can consume space that is unaccounted for in the neededSpace + * calculation. However, I believe this can only happen when the + * workspace is too large, and specifically when it is too large + * by a larger margin than the space that will be consumed. */ + /* TODO: cleaner, compiler warning friendly way to do this??? */ + ws->allocStart = (BYTE*)ws->allocStart - ((size_t)ws->allocStart & (sizeof(U32)-1)); + if (ws->allocStart < ws->tableValidEnd) { + ws->tableValidEnd = ws->allocStart; + } + } + ws->phase = phase; + } +} + +/** + * Returns whether this object/buffer/etc was allocated in this workspace. + */ +MEM_STATIC int ZSTD_cwksp_owns_buffer(const ZSTD_cwksp* ws, const void* ptr) { + return (ptr != NULL) && (ws->workspace <= ptr) && (ptr <= ws->workspaceEnd); +} + +/** + * Internal function. Do not use directly. + */ +MEM_STATIC void* ZSTD_cwksp_reserve_internal( + ZSTD_cwksp* ws, size_t bytes, ZSTD_cwksp_alloc_phase_e phase) { + void* alloc; + void* bottom = ws->tableEnd; + ZSTD_cwksp_internal_advance_phase(ws, phase); + alloc = (BYTE *)ws->allocStart - bytes; + +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) + /* over-reserve space */ + alloc = (BYTE *)alloc - 2 * ZSTD_CWKSP_ASAN_REDZONE_SIZE; +#endif + + DEBUGLOG(5, "cwksp: reserving %p %zd bytes, %zd bytes remaining", + alloc, bytes, ZSTD_cwksp_available_space(ws) - bytes); + ZSTD_cwksp_assert_internal_consistency(ws); + assert(alloc >= bottom); + if (alloc < bottom) { + DEBUGLOG(4, "cwksp: alloc failed!"); + ws->allocFailed = 1; + return NULL; + } + if (alloc < ws->tableValidEnd) { + ws->tableValidEnd = alloc; + } + ws->allocStart = alloc; + +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) + /* Move alloc so there's ZSTD_CWKSP_ASAN_REDZONE_SIZE unused space on + * either size. */ + alloc = (BYTE *)alloc + ZSTD_CWKSP_ASAN_REDZONE_SIZE; + __asan_unpoison_memory_region(alloc, bytes); +#endif + + return alloc; +} + +/** + * Reserves and returns unaligned memory. + */ +MEM_STATIC BYTE* ZSTD_cwksp_reserve_buffer(ZSTD_cwksp* ws, size_t bytes) { + return (BYTE*)ZSTD_cwksp_reserve_internal(ws, bytes, ZSTD_cwksp_alloc_buffers); +} + +/** + * Reserves and returns memory sized on and aligned on sizeof(unsigned). + */ +MEM_STATIC void* ZSTD_cwksp_reserve_aligned(ZSTD_cwksp* ws, size_t bytes) { + assert((bytes & (sizeof(U32)-1)) == 0); + return ZSTD_cwksp_reserve_internal(ws, ZSTD_cwksp_align(bytes, sizeof(U32)), ZSTD_cwksp_alloc_aligned); +} + +/** + * Aligned on sizeof(unsigned). These buffers have the special property that + * their values remain constrained, allowing us to re-use them without + * memset()-ing them. + */ +MEM_STATIC void* ZSTD_cwksp_reserve_table(ZSTD_cwksp* ws, size_t bytes) { + const ZSTD_cwksp_alloc_phase_e phase = ZSTD_cwksp_alloc_aligned; + void* alloc = ws->tableEnd; + void* end = (BYTE *)alloc + bytes; + void* top = ws->allocStart; + + DEBUGLOG(5, "cwksp: reserving %p table %zd bytes, %zd bytes remaining", + alloc, bytes, ZSTD_cwksp_available_space(ws) - bytes); + assert((bytes & (sizeof(U32)-1)) == 0); + ZSTD_cwksp_internal_advance_phase(ws, phase); + ZSTD_cwksp_assert_internal_consistency(ws); + assert(end <= top); + if (end > top) { + DEBUGLOG(4, "cwksp: table alloc failed!"); + ws->allocFailed = 1; + return NULL; + } + ws->tableEnd = end; + +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) + __asan_unpoison_memory_region(alloc, bytes); +#endif + + return alloc; +} + +/** + * Aligned on sizeof(void*). + */ +MEM_STATIC void* ZSTD_cwksp_reserve_object(ZSTD_cwksp* ws, size_t bytes) { + size_t roundedBytes = ZSTD_cwksp_align(bytes, sizeof(void*)); + void* alloc = ws->objectEnd; + void* end = (BYTE*)alloc + roundedBytes; + +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) + /* over-reserve space */ + end = (BYTE *)end + 2 * ZSTD_CWKSP_ASAN_REDZONE_SIZE; +#endif + + DEBUGLOG(5, + "cwksp: reserving %p object %zd bytes (rounded to %zd), %zd bytes remaining", + alloc, bytes, roundedBytes, ZSTD_cwksp_available_space(ws) - roundedBytes); + assert(((size_t)alloc & (sizeof(void*)-1)) == 0); + assert((bytes & (sizeof(void*)-1)) == 0); + ZSTD_cwksp_assert_internal_consistency(ws); + /* we must be in the first phase, no advance is possible */ + if (ws->phase != ZSTD_cwksp_alloc_objects || end > ws->workspaceEnd) { + DEBUGLOG(4, "cwksp: object alloc failed!"); + ws->allocFailed = 1; + return NULL; + } + ws->objectEnd = end; + ws->tableEnd = end; + ws->tableValidEnd = end; + +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) + /* Move alloc so there's ZSTD_CWKSP_ASAN_REDZONE_SIZE unused space on + * either size. */ + alloc = (BYTE *)alloc + ZSTD_CWKSP_ASAN_REDZONE_SIZE; + __asan_unpoison_memory_region(alloc, bytes); +#endif + + return alloc; +} + +MEM_STATIC void ZSTD_cwksp_mark_tables_dirty(ZSTD_cwksp* ws) { + DEBUGLOG(4, "cwksp: ZSTD_cwksp_mark_tables_dirty"); + +#if defined (MEMORY_SANITIZER) && !defined (ZSTD_MSAN_DONT_POISON_WORKSPACE) + /* To validate that the table re-use logic is sound, and that we don't + * access table space that we haven't cleaned, we re-"poison" the table + * space every time we mark it dirty. */ + { + size_t size = (BYTE*)ws->tableValidEnd - (BYTE*)ws->objectEnd; + assert(__msan_test_shadow(ws->objectEnd, size) == -1); + __msan_poison(ws->objectEnd, size); + } +#endif + + assert(ws->tableValidEnd >= ws->objectEnd); + assert(ws->tableValidEnd <= ws->allocStart); + ws->tableValidEnd = ws->objectEnd; + ZSTD_cwksp_assert_internal_consistency(ws); +} + +MEM_STATIC void ZSTD_cwksp_mark_tables_clean(ZSTD_cwksp* ws) { + DEBUGLOG(4, "cwksp: ZSTD_cwksp_mark_tables_clean"); + assert(ws->tableValidEnd >= ws->objectEnd); + assert(ws->tableValidEnd <= ws->allocStart); + if (ws->tableValidEnd < ws->tableEnd) { + ws->tableValidEnd = ws->tableEnd; + } + ZSTD_cwksp_assert_internal_consistency(ws); +} + +/** + * Zero the part of the allocated tables not already marked clean. + */ +MEM_STATIC void ZSTD_cwksp_clean_tables(ZSTD_cwksp* ws) { + DEBUGLOG(4, "cwksp: ZSTD_cwksp_clean_tables"); + assert(ws->tableValidEnd >= ws->objectEnd); + assert(ws->tableValidEnd <= ws->allocStart); + if (ws->tableValidEnd < ws->tableEnd) { + memset(ws->tableValidEnd, 0, (BYTE*)ws->tableEnd - (BYTE*)ws->tableValidEnd); + } + ZSTD_cwksp_mark_tables_clean(ws); +} + +/** + * Invalidates table allocations. + * All other allocations remain valid. + */ +MEM_STATIC void ZSTD_cwksp_clear_tables(ZSTD_cwksp* ws) { + DEBUGLOG(4, "cwksp: clearing tables!"); + +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) + { + size_t size = (BYTE*)ws->tableValidEnd - (BYTE*)ws->objectEnd; + __asan_poison_memory_region(ws->objectEnd, size); + } +#endif + + ws->tableEnd = ws->objectEnd; + ZSTD_cwksp_assert_internal_consistency(ws); +} + +/** + * Invalidates all buffer, aligned, and table allocations. + * Object allocations remain valid. + */ +MEM_STATIC void ZSTD_cwksp_clear(ZSTD_cwksp* ws) { + DEBUGLOG(4, "cwksp: clearing!"); + +#if defined (MEMORY_SANITIZER) && !defined (ZSTD_MSAN_DONT_POISON_WORKSPACE) + /* To validate that the context re-use logic is sound, and that we don't + * access stuff that this compression hasn't initialized, we re-"poison" + * the workspace (or at least the non-static, non-table parts of it) + * every time we start a new compression. */ + { + size_t size = (BYTE*)ws->workspaceEnd - (BYTE*)ws->tableValidEnd; + __msan_poison(ws->tableValidEnd, size); + } +#endif + +#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) + { + size_t size = (BYTE*)ws->workspaceEnd - (BYTE*)ws->objectEnd; + __asan_poison_memory_region(ws->objectEnd, size); + } +#endif + + ws->tableEnd = ws->objectEnd; + ws->allocStart = ws->workspaceEnd; + ws->allocFailed = 0; + if (ws->phase > ZSTD_cwksp_alloc_buffers) { + ws->phase = ZSTD_cwksp_alloc_buffers; + } + ZSTD_cwksp_assert_internal_consistency(ws); +} + +/** + * The provided workspace takes ownership of the buffer [start, start+size). + * Any existing values in the workspace are ignored (the previously managed + * buffer, if present, must be separately freed). + */ +MEM_STATIC void ZSTD_cwksp_init(ZSTD_cwksp* ws, void* start, size_t size) { + DEBUGLOG(4, "cwksp: init'ing workspace with %zd bytes", size); + assert(((size_t)start & (sizeof(void*)-1)) == 0); /* ensure correct alignment */ + ws->workspace = start; + ws->workspaceEnd = (BYTE*)start + size; + ws->objectEnd = ws->workspace; + ws->tableValidEnd = ws->objectEnd; + ws->phase = ZSTD_cwksp_alloc_objects; + ZSTD_cwksp_clear(ws); + ws->workspaceOversizedDuration = 0; + ZSTD_cwksp_assert_internal_consistency(ws); +} + +MEM_STATIC size_t ZSTD_cwksp_create(ZSTD_cwksp* ws, size_t size, ZSTD_customMem customMem) { + void* workspace = ZSTD_malloc(size, customMem); + DEBUGLOG(4, "cwksp: creating new workspace with %zd bytes", size); + RETURN_ERROR_IF(workspace == NULL, memory_allocation, "NULL pointer!"); + ZSTD_cwksp_init(ws, workspace, size); + return 0; +} + +MEM_STATIC void ZSTD_cwksp_free(ZSTD_cwksp* ws, ZSTD_customMem customMem) { + void *ptr = ws->workspace; + DEBUGLOG(4, "cwksp: freeing workspace"); + memset(ws, 0, sizeof(ZSTD_cwksp)); + ZSTD_free(ptr, customMem); +} + +/** + * Moves the management of a workspace from one cwksp to another. The src cwksp + * is left in an invalid state (src must be re-init()'ed before its used again). + */ +MEM_STATIC void ZSTD_cwksp_move(ZSTD_cwksp* dst, ZSTD_cwksp* src) { + *dst = *src; + memset(src, 0, sizeof(ZSTD_cwksp)); +} + +MEM_STATIC size_t ZSTD_cwksp_sizeof(const ZSTD_cwksp* ws) { + return (size_t)((BYTE*)ws->workspaceEnd - (BYTE*)ws->workspace); +} + +MEM_STATIC int ZSTD_cwksp_reserve_failed(const ZSTD_cwksp* ws) { + return ws->allocFailed; +} + +/*-************************************* +* Functions Checking Free Space +***************************************/ + +MEM_STATIC size_t ZSTD_cwksp_available_space(ZSTD_cwksp* ws) { + return (size_t)((BYTE*)ws->allocStart - (BYTE*)ws->tableEnd); +} + +MEM_STATIC int ZSTD_cwksp_check_available(ZSTD_cwksp* ws, size_t additionalNeededSpace) { + return ZSTD_cwksp_available_space(ws) >= additionalNeededSpace; +} + +MEM_STATIC int ZSTD_cwksp_check_too_large(ZSTD_cwksp* ws, size_t additionalNeededSpace) { + return ZSTD_cwksp_check_available( + ws, additionalNeededSpace * ZSTD_WORKSPACETOOLARGE_FACTOR); +} + +MEM_STATIC int ZSTD_cwksp_check_wasteful(ZSTD_cwksp* ws, size_t additionalNeededSpace) { + return ZSTD_cwksp_check_too_large(ws, additionalNeededSpace) + && ws->workspaceOversizedDuration > ZSTD_WORKSPACETOOLARGE_MAXDURATION; +} + +MEM_STATIC void ZSTD_cwksp_bump_oversized_duration( + ZSTD_cwksp* ws, size_t additionalNeededSpace) { + if (ZSTD_cwksp_check_too_large(ws, additionalNeededSpace)) { + ws->workspaceOversizedDuration++; + } else { + ws->workspaceOversizedDuration = 0; + } +} + +#if defined (__cplusplus) +} +#endif + +#endif /* ZSTD_CWKSP_H */ diff --git a/module/zstd/lib/compress/zstd_double_fast.c b/module/zstd/lib/compress/zstd_double_fast.c new file mode 100644 index 000000000000..27eed66cfedd --- /dev/null +++ b/module/zstd/lib/compress/zstd_double_fast.c @@ -0,0 +1,521 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#include "zstd_compress_internal.h" +#include "zstd_double_fast.h" + + +void ZSTD_fillDoubleHashTable(ZSTD_matchState_t* ms, + void const* end, ZSTD_dictTableLoadMethod_e dtlm) +{ + const ZSTD_compressionParameters* const cParams = &ms->cParams; + U32* const hashLarge = ms->hashTable; + U32 const hBitsL = cParams->hashLog; + U32 const mls = cParams->minMatch; + U32* const hashSmall = ms->chainTable; + U32 const hBitsS = cParams->chainLog; + const BYTE* const base = ms->window.base; + const BYTE* ip = base + ms->nextToUpdate; + const BYTE* const iend = ((const BYTE*)end) - HASH_READ_SIZE; + const U32 fastHashFillStep = 3; + + /* Always insert every fastHashFillStep position into the hash tables. + * Insert the other positions into the large hash table if their entry + * is empty. + */ + for (; ip + fastHashFillStep - 1 <= iend; ip += fastHashFillStep) { + U32 const current = (U32)(ip - base); + U32 i; + for (i = 0; i < fastHashFillStep; ++i) { + size_t const smHash = ZSTD_hashPtr(ip + i, hBitsS, mls); + size_t const lgHash = ZSTD_hashPtr(ip + i, hBitsL, 8); + if (i == 0) + hashSmall[smHash] = current + i; + if (i == 0 || hashLarge[lgHash] == 0) + hashLarge[lgHash] = current + i; + /* Only load extra positions for ZSTD_dtlm_full */ + if (dtlm == ZSTD_dtlm_fast) + break; + } } +} + + +FORCE_INLINE_TEMPLATE +size_t ZSTD_compressBlock_doubleFast_generic( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize, + U32 const mls /* template */, ZSTD_dictMode_e const dictMode) +{ + ZSTD_compressionParameters const* cParams = &ms->cParams; + U32* const hashLong = ms->hashTable; + const U32 hBitsL = cParams->hashLog; + U32* const hashSmall = ms->chainTable; + const U32 hBitsS = cParams->chainLog; + const BYTE* const base = ms->window.base; + const BYTE* const istart = (const BYTE*)src; + const BYTE* ip = istart; + const BYTE* anchor = istart; + const U32 endIndex = (U32)((size_t)(istart - base) + srcSize); + /* presumes that, if there is a dictionary, it must be using Attach mode */ + const U32 prefixLowestIndex = ZSTD_getLowestPrefixIndex(ms, endIndex, cParams->windowLog); + const BYTE* const prefixLowest = base + prefixLowestIndex; + const BYTE* const iend = istart + srcSize; + const BYTE* const ilimit = iend - HASH_READ_SIZE; + U32 offset_1=rep[0], offset_2=rep[1]; + U32 offsetSaved = 0; + + const ZSTD_matchState_t* const dms = ms->dictMatchState; + const ZSTD_compressionParameters* const dictCParams = + dictMode == ZSTD_dictMatchState ? + &dms->cParams : NULL; + const U32* const dictHashLong = dictMode == ZSTD_dictMatchState ? + dms->hashTable : NULL; + const U32* const dictHashSmall = dictMode == ZSTD_dictMatchState ? + dms->chainTable : NULL; + const U32 dictStartIndex = dictMode == ZSTD_dictMatchState ? + dms->window.dictLimit : 0; + const BYTE* const dictBase = dictMode == ZSTD_dictMatchState ? + dms->window.base : NULL; + const BYTE* const dictStart = dictMode == ZSTD_dictMatchState ? + dictBase + dictStartIndex : NULL; + const BYTE* const dictEnd = dictMode == ZSTD_dictMatchState ? + dms->window.nextSrc : NULL; + const U32 dictIndexDelta = dictMode == ZSTD_dictMatchState ? + prefixLowestIndex - (U32)(dictEnd - dictBase) : + 0; + const U32 dictHBitsL = dictMode == ZSTD_dictMatchState ? + dictCParams->hashLog : hBitsL; + const U32 dictHBitsS = dictMode == ZSTD_dictMatchState ? + dictCParams->chainLog : hBitsS; + const U32 dictAndPrefixLength = (U32)((ip - prefixLowest) + (dictEnd - dictStart)); + + DEBUGLOG(5, "ZSTD_compressBlock_doubleFast_generic"); + + assert(dictMode == ZSTD_noDict || dictMode == ZSTD_dictMatchState); + + /* if a dictionary is attached, it must be within window range */ + if (dictMode == ZSTD_dictMatchState) { + assert(ms->window.dictLimit + (1U << cParams->windowLog) >= endIndex); + } + + /* init */ + ip += (dictAndPrefixLength == 0); + if (dictMode == ZSTD_noDict) { + U32 const current = (U32)(ip - base); + U32 const windowLow = ZSTD_getLowestPrefixIndex(ms, current, cParams->windowLog); + U32 const maxRep = current - windowLow; + if (offset_2 > maxRep) offsetSaved = offset_2, offset_2 = 0; + if (offset_1 > maxRep) offsetSaved = offset_1, offset_1 = 0; + } + if (dictMode == ZSTD_dictMatchState) { + /* dictMatchState repCode checks don't currently handle repCode == 0 + * disabling. */ + assert(offset_1 <= dictAndPrefixLength); + assert(offset_2 <= dictAndPrefixLength); + } + + /* Main Search Loop */ + while (ip < ilimit) { /* < instead of <=, because repcode check at (ip+1) */ + size_t mLength; + U32 offset; + size_t const h2 = ZSTD_hashPtr(ip, hBitsL, 8); + size_t const h = ZSTD_hashPtr(ip, hBitsS, mls); + size_t const dictHL = ZSTD_hashPtr(ip, dictHBitsL, 8); + size_t const dictHS = ZSTD_hashPtr(ip, dictHBitsS, mls); + U32 const current = (U32)(ip-base); + U32 const matchIndexL = hashLong[h2]; + U32 matchIndexS = hashSmall[h]; + const BYTE* matchLong = base + matchIndexL; + const BYTE* match = base + matchIndexS; + const U32 repIndex = current + 1 - offset_1; + const BYTE* repMatch = (dictMode == ZSTD_dictMatchState + && repIndex < prefixLowestIndex) ? + dictBase + (repIndex - dictIndexDelta) : + base + repIndex; + hashLong[h2] = hashSmall[h] = current; /* update hash tables */ + + /* check dictMatchState repcode */ + if (dictMode == ZSTD_dictMatchState + && ((U32)((prefixLowestIndex-1) - repIndex) >= 3 /* intentional underflow */) + && (MEM_read32(repMatch) == MEM_read32(ip+1)) ) { + const BYTE* repMatchEnd = repIndex < prefixLowestIndex ? dictEnd : iend; + mLength = ZSTD_count_2segments(ip+1+4, repMatch+4, iend, repMatchEnd, prefixLowest) + 4; + ip++; + ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, mLength-MINMATCH); + goto _match_stored; + } + + /* check noDict repcode */ + if ( dictMode == ZSTD_noDict + && ((offset_1 > 0) & (MEM_read32(ip+1-offset_1) == MEM_read32(ip+1)))) { + mLength = ZSTD_count(ip+1+4, ip+1+4-offset_1, iend) + 4; + ip++; + ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, mLength-MINMATCH); + goto _match_stored; + } + + if (matchIndexL > prefixLowestIndex) { + /* check prefix long match */ + if (MEM_read64(matchLong) == MEM_read64(ip)) { + mLength = ZSTD_count(ip+8, matchLong+8, iend) + 8; + offset = (U32)(ip-matchLong); + while (((ip>anchor) & (matchLong>prefixLowest)) && (ip[-1] == matchLong[-1])) { ip--; matchLong--; mLength++; } /* catch up */ + goto _match_found; + } + } else if (dictMode == ZSTD_dictMatchState) { + /* check dictMatchState long match */ + U32 const dictMatchIndexL = dictHashLong[dictHL]; + const BYTE* dictMatchL = dictBase + dictMatchIndexL; + assert(dictMatchL < dictEnd); + + if (dictMatchL > dictStart && MEM_read64(dictMatchL) == MEM_read64(ip)) { + mLength = ZSTD_count_2segments(ip+8, dictMatchL+8, iend, dictEnd, prefixLowest) + 8; + offset = (U32)(current - dictMatchIndexL - dictIndexDelta); + while (((ip>anchor) & (dictMatchL>dictStart)) && (ip[-1] == dictMatchL[-1])) { ip--; dictMatchL--; mLength++; } /* catch up */ + goto _match_found; + } } + + if (matchIndexS > prefixLowestIndex) { + /* check prefix short match */ + if (MEM_read32(match) == MEM_read32(ip)) { + goto _search_next_long; + } + } else if (dictMode == ZSTD_dictMatchState) { + /* check dictMatchState short match */ + U32 const dictMatchIndexS = dictHashSmall[dictHS]; + match = dictBase + dictMatchIndexS; + matchIndexS = dictMatchIndexS + dictIndexDelta; + + if (match > dictStart && MEM_read32(match) == MEM_read32(ip)) { + goto _search_next_long; + } } + + ip += ((ip-anchor) >> kSearchStrength) + 1; +#if defined(__aarch64__) + PREFETCH_L1(ip+256); +#endif + continue; + +_search_next_long: + + { size_t const hl3 = ZSTD_hashPtr(ip+1, hBitsL, 8); + size_t const dictHLNext = ZSTD_hashPtr(ip+1, dictHBitsL, 8); + U32 const matchIndexL3 = hashLong[hl3]; + const BYTE* matchL3 = base + matchIndexL3; + hashLong[hl3] = current + 1; + + /* check prefix long +1 match */ + if (matchIndexL3 > prefixLowestIndex) { + if (MEM_read64(matchL3) == MEM_read64(ip+1)) { + mLength = ZSTD_count(ip+9, matchL3+8, iend) + 8; + ip++; + offset = (U32)(ip-matchL3); + while (((ip>anchor) & (matchL3>prefixLowest)) && (ip[-1] == matchL3[-1])) { ip--; matchL3--; mLength++; } /* catch up */ + goto _match_found; + } + } else if (dictMode == ZSTD_dictMatchState) { + /* check dict long +1 match */ + U32 const dictMatchIndexL3 = dictHashLong[dictHLNext]; + const BYTE* dictMatchL3 = dictBase + dictMatchIndexL3; + assert(dictMatchL3 < dictEnd); + if (dictMatchL3 > dictStart && MEM_read64(dictMatchL3) == MEM_read64(ip+1)) { + mLength = ZSTD_count_2segments(ip+1+8, dictMatchL3+8, iend, dictEnd, prefixLowest) + 8; + ip++; + offset = (U32)(current + 1 - dictMatchIndexL3 - dictIndexDelta); + while (((ip>anchor) & (dictMatchL3>dictStart)) && (ip[-1] == dictMatchL3[-1])) { ip--; dictMatchL3--; mLength++; } /* catch up */ + goto _match_found; + } } } + + /* if no long +1 match, explore the short match we found */ + if (dictMode == ZSTD_dictMatchState && matchIndexS < prefixLowestIndex) { + mLength = ZSTD_count_2segments(ip+4, match+4, iend, dictEnd, prefixLowest) + 4; + offset = (U32)(current - matchIndexS); + while (((ip>anchor) & (match>dictStart)) && (ip[-1] == match[-1])) { ip--; match--; mLength++; } /* catch up */ + } else { + mLength = ZSTD_count(ip+4, match+4, iend) + 4; + offset = (U32)(ip - match); + while (((ip>anchor) & (match>prefixLowest)) && (ip[-1] == match[-1])) { ip--; match--; mLength++; } /* catch up */ + } + + /* fall-through */ + +_match_found: + offset_2 = offset_1; + offset_1 = offset; + + ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH); + +_match_stored: + /* match found */ + ip += mLength; + anchor = ip; + + if (ip <= ilimit) { + /* Complementary insertion */ + /* done after iLimit test, as candidates could be > iend-8 */ + { U32 const indexToInsert = current+2; + hashLong[ZSTD_hashPtr(base+indexToInsert, hBitsL, 8)] = indexToInsert; + hashLong[ZSTD_hashPtr(ip-2, hBitsL, 8)] = (U32)(ip-2-base); + hashSmall[ZSTD_hashPtr(base+indexToInsert, hBitsS, mls)] = indexToInsert; + hashSmall[ZSTD_hashPtr(ip-1, hBitsS, mls)] = (U32)(ip-1-base); + } + + /* check immediate repcode */ + if (dictMode == ZSTD_dictMatchState) { + while (ip <= ilimit) { + U32 const current2 = (U32)(ip-base); + U32 const repIndex2 = current2 - offset_2; + const BYTE* repMatch2 = dictMode == ZSTD_dictMatchState + && repIndex2 < prefixLowestIndex ? + dictBase + repIndex2 - dictIndexDelta : + base + repIndex2; + if ( ((U32)((prefixLowestIndex-1) - (U32)repIndex2) >= 3 /* intentional overflow */) + && (MEM_read32(repMatch2) == MEM_read32(ip)) ) { + const BYTE* const repEnd2 = repIndex2 < prefixLowestIndex ? dictEnd : iend; + size_t const repLength2 = ZSTD_count_2segments(ip+4, repMatch2+4, iend, repEnd2, prefixLowest) + 4; + U32 tmpOffset = offset_2; offset_2 = offset_1; offset_1 = tmpOffset; /* swap offset_2 <=> offset_1 */ + ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, repLength2-MINMATCH); + hashSmall[ZSTD_hashPtr(ip, hBitsS, mls)] = current2; + hashLong[ZSTD_hashPtr(ip, hBitsL, 8)] = current2; + ip += repLength2; + anchor = ip; + continue; + } + break; + } } + + if (dictMode == ZSTD_noDict) { + while ( (ip <= ilimit) + && ( (offset_2>0) + & (MEM_read32(ip) == MEM_read32(ip - offset_2)) )) { + /* store sequence */ + size_t const rLength = ZSTD_count(ip+4, ip+4-offset_2, iend) + 4; + U32 const tmpOff = offset_2; offset_2 = offset_1; offset_1 = tmpOff; /* swap offset_2 <=> offset_1 */ + hashSmall[ZSTD_hashPtr(ip, hBitsS, mls)] = (U32)(ip-base); + hashLong[ZSTD_hashPtr(ip, hBitsL, 8)] = (U32)(ip-base); + ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, rLength-MINMATCH); + ip += rLength; + anchor = ip; + continue; /* faster when present ... (?) */ + } } } + } /* while (ip < ilimit) */ + + /* save reps for next block */ + rep[0] = offset_1 ? offset_1 : offsetSaved; + rep[1] = offset_2 ? offset_2 : offsetSaved; + + /* Return the last literals size */ + return (size_t)(iend - anchor); +} + + +size_t ZSTD_compressBlock_doubleFast( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + const U32 mls = ms->cParams.minMatch; + switch(mls) + { + default: /* includes case 3 */ + case 4 : + return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 4, ZSTD_noDict); + case 5 : + return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 5, ZSTD_noDict); + case 6 : + return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 6, ZSTD_noDict); + case 7 : + return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 7, ZSTD_noDict); + } +} + + +size_t ZSTD_compressBlock_doubleFast_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + const U32 mls = ms->cParams.minMatch; + switch(mls) + { + default: /* includes case 3 */ + case 4 : + return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 4, ZSTD_dictMatchState); + case 5 : + return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 5, ZSTD_dictMatchState); + case 6 : + return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 6, ZSTD_dictMatchState); + case 7 : + return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 7, ZSTD_dictMatchState); + } +} + + +static size_t ZSTD_compressBlock_doubleFast_extDict_generic( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize, + U32 const mls /* template */) +{ + ZSTD_compressionParameters const* cParams = &ms->cParams; + U32* const hashLong = ms->hashTable; + U32 const hBitsL = cParams->hashLog; + U32* const hashSmall = ms->chainTable; + U32 const hBitsS = cParams->chainLog; + const BYTE* const istart = (const BYTE*)src; + const BYTE* ip = istart; + const BYTE* anchor = istart; + const BYTE* const iend = istart + srcSize; + const BYTE* const ilimit = iend - 8; + const BYTE* const base = ms->window.base; + const U32 endIndex = (U32)((size_t)(istart - base) + srcSize); + const U32 lowLimit = ZSTD_getLowestMatchIndex(ms, endIndex, cParams->windowLog); + const U32 dictStartIndex = lowLimit; + const U32 dictLimit = ms->window.dictLimit; + const U32 prefixStartIndex = (dictLimit > lowLimit) ? dictLimit : lowLimit; + const BYTE* const prefixStart = base + prefixStartIndex; + const BYTE* const dictBase = ms->window.dictBase; + const BYTE* const dictStart = dictBase + dictStartIndex; + const BYTE* const dictEnd = dictBase + prefixStartIndex; + U32 offset_1=rep[0], offset_2=rep[1]; + + DEBUGLOG(5, "ZSTD_compressBlock_doubleFast_extDict_generic (srcSize=%zu)", srcSize); + + /* if extDict is invalidated due to maxDistance, switch to "regular" variant */ + if (prefixStartIndex == dictStartIndex) + return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, mls, ZSTD_noDict); + + /* Search Loop */ + while (ip < ilimit) { /* < instead of <=, because (ip+1) */ + const size_t hSmall = ZSTD_hashPtr(ip, hBitsS, mls); + const U32 matchIndex = hashSmall[hSmall]; + const BYTE* const matchBase = matchIndex < prefixStartIndex ? dictBase : base; + const BYTE* match = matchBase + matchIndex; + + const size_t hLong = ZSTD_hashPtr(ip, hBitsL, 8); + const U32 matchLongIndex = hashLong[hLong]; + const BYTE* const matchLongBase = matchLongIndex < prefixStartIndex ? dictBase : base; + const BYTE* matchLong = matchLongBase + matchLongIndex; + + const U32 current = (U32)(ip-base); + const U32 repIndex = current + 1 - offset_1; /* offset_1 expected <= current +1 */ + const BYTE* const repBase = repIndex < prefixStartIndex ? dictBase : base; + const BYTE* const repMatch = repBase + repIndex; + size_t mLength; + hashSmall[hSmall] = hashLong[hLong] = current; /* update hash table */ + + if ((((U32)((prefixStartIndex-1) - repIndex) >= 3) /* intentional underflow : ensure repIndex doesn't overlap dict + prefix */ + & (repIndex > dictStartIndex)) + && (MEM_read32(repMatch) == MEM_read32(ip+1)) ) { + const BYTE* repMatchEnd = repIndex < prefixStartIndex ? dictEnd : iend; + mLength = ZSTD_count_2segments(ip+1+4, repMatch+4, iend, repMatchEnd, prefixStart) + 4; + ip++; + ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, mLength-MINMATCH); + } else { + if ((matchLongIndex > dictStartIndex) && (MEM_read64(matchLong) == MEM_read64(ip))) { + const BYTE* const matchEnd = matchLongIndex < prefixStartIndex ? dictEnd : iend; + const BYTE* const lowMatchPtr = matchLongIndex < prefixStartIndex ? dictStart : prefixStart; + U32 offset; + mLength = ZSTD_count_2segments(ip+8, matchLong+8, iend, matchEnd, prefixStart) + 8; + offset = current - matchLongIndex; + while (((ip>anchor) & (matchLong>lowMatchPtr)) && (ip[-1] == matchLong[-1])) { ip--; matchLong--; mLength++; } /* catch up */ + offset_2 = offset_1; + offset_1 = offset; + ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH); + + } else if ((matchIndex > dictStartIndex) && (MEM_read32(match) == MEM_read32(ip))) { + size_t const h3 = ZSTD_hashPtr(ip+1, hBitsL, 8); + U32 const matchIndex3 = hashLong[h3]; + const BYTE* const match3Base = matchIndex3 < prefixStartIndex ? dictBase : base; + const BYTE* match3 = match3Base + matchIndex3; + U32 offset; + hashLong[h3] = current + 1; + if ( (matchIndex3 > dictStartIndex) && (MEM_read64(match3) == MEM_read64(ip+1)) ) { + const BYTE* const matchEnd = matchIndex3 < prefixStartIndex ? dictEnd : iend; + const BYTE* const lowMatchPtr = matchIndex3 < prefixStartIndex ? dictStart : prefixStart; + mLength = ZSTD_count_2segments(ip+9, match3+8, iend, matchEnd, prefixStart) + 8; + ip++; + offset = current+1 - matchIndex3; + while (((ip>anchor) & (match3>lowMatchPtr)) && (ip[-1] == match3[-1])) { ip--; match3--; mLength++; } /* catch up */ + } else { + const BYTE* const matchEnd = matchIndex < prefixStartIndex ? dictEnd : iend; + const BYTE* const lowMatchPtr = matchIndex < prefixStartIndex ? dictStart : prefixStart; + mLength = ZSTD_count_2segments(ip+4, match+4, iend, matchEnd, prefixStart) + 4; + offset = current - matchIndex; + while (((ip>anchor) & (match>lowMatchPtr)) && (ip[-1] == match[-1])) { ip--; match--; mLength++; } /* catch up */ + } + offset_2 = offset_1; + offset_1 = offset; + ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH); + + } else { + ip += ((ip-anchor) >> kSearchStrength) + 1; + continue; + } } + + /* move to next sequence start */ + ip += mLength; + anchor = ip; + + if (ip <= ilimit) { + /* Complementary insertion */ + /* done after iLimit test, as candidates could be > iend-8 */ + { U32 const indexToInsert = current+2; + hashLong[ZSTD_hashPtr(base+indexToInsert, hBitsL, 8)] = indexToInsert; + hashLong[ZSTD_hashPtr(ip-2, hBitsL, 8)] = (U32)(ip-2-base); + hashSmall[ZSTD_hashPtr(base+indexToInsert, hBitsS, mls)] = indexToInsert; + hashSmall[ZSTD_hashPtr(ip-1, hBitsS, mls)] = (U32)(ip-1-base); + } + + /* check immediate repcode */ + while (ip <= ilimit) { + U32 const current2 = (U32)(ip-base); + U32 const repIndex2 = current2 - offset_2; + const BYTE* repMatch2 = repIndex2 < prefixStartIndex ? dictBase + repIndex2 : base + repIndex2; + if ( (((U32)((prefixStartIndex-1) - repIndex2) >= 3) /* intentional overflow : ensure repIndex2 doesn't overlap dict + prefix */ + & (repIndex2 > dictStartIndex)) + && (MEM_read32(repMatch2) == MEM_read32(ip)) ) { + const BYTE* const repEnd2 = repIndex2 < prefixStartIndex ? dictEnd : iend; + size_t const repLength2 = ZSTD_count_2segments(ip+4, repMatch2+4, iend, repEnd2, prefixStart) + 4; + U32 const tmpOffset = offset_2; offset_2 = offset_1; offset_1 = tmpOffset; /* swap offset_2 <=> offset_1 */ + ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, repLength2-MINMATCH); + hashSmall[ZSTD_hashPtr(ip, hBitsS, mls)] = current2; + hashLong[ZSTD_hashPtr(ip, hBitsL, 8)] = current2; + ip += repLength2; + anchor = ip; + continue; + } + break; + } } } + + /* save reps for next block */ + rep[0] = offset_1; + rep[1] = offset_2; + + /* Return the last literals size */ + return (size_t)(iend - anchor); +} + + +size_t ZSTD_compressBlock_doubleFast_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + U32 const mls = ms->cParams.minMatch; + switch(mls) + { + default: /* includes case 3 */ + case 4 : + return ZSTD_compressBlock_doubleFast_extDict_generic(ms, seqStore, rep, src, srcSize, 4); + case 5 : + return ZSTD_compressBlock_doubleFast_extDict_generic(ms, seqStore, rep, src, srcSize, 5); + case 6 : + return ZSTD_compressBlock_doubleFast_extDict_generic(ms, seqStore, rep, src, srcSize, 6); + case 7 : + return ZSTD_compressBlock_doubleFast_extDict_generic(ms, seqStore, rep, src, srcSize, 7); + } +} diff --git a/module/zstd/lib/compress/zstd_double_fast.h b/module/zstd/lib/compress/zstd_double_fast.h new file mode 100644 index 000000000000..14d944d69bc1 --- /dev/null +++ b/module/zstd/lib/compress/zstd_double_fast.h @@ -0,0 +1,38 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#ifndef ZSTD_DOUBLE_FAST_H +#define ZSTD_DOUBLE_FAST_H + +#if defined (__cplusplus) +extern "C" { +#endif + +#include "../common/mem.h" /* U32 */ +#include "zstd_compress_internal.h" /* ZSTD_CCtx, size_t */ + +void ZSTD_fillDoubleHashTable(ZSTD_matchState_t* ms, + void const* end, ZSTD_dictTableLoadMethod_e dtlm); +size_t ZSTD_compressBlock_doubleFast( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_doubleFast_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_doubleFast_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); + + +#if defined (__cplusplus) +} +#endif + +#endif /* ZSTD_DOUBLE_FAST_H */ diff --git a/module/zstd/lib/compress/zstd_fast.c b/module/zstd/lib/compress/zstd_fast.c new file mode 100644 index 000000000000..85a3a7a91e49 --- /dev/null +++ b/module/zstd/lib/compress/zstd_fast.c @@ -0,0 +1,496 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#include "zstd_compress_internal.h" /* ZSTD_hashPtr, ZSTD_count, ZSTD_storeSeq */ +#include "zstd_fast.h" + + +void ZSTD_fillHashTable(ZSTD_matchState_t* ms, + const void* const end, + ZSTD_dictTableLoadMethod_e dtlm) +{ + const ZSTD_compressionParameters* const cParams = &ms->cParams; + U32* const hashTable = ms->hashTable; + U32 const hBits = cParams->hashLog; + U32 const mls = cParams->minMatch; + const BYTE* const base = ms->window.base; + const BYTE* ip = base + ms->nextToUpdate; + const BYTE* const iend = ((const BYTE*)end) - HASH_READ_SIZE; + const U32 fastHashFillStep = 3; + + /* Always insert every fastHashFillStep position into the hash table. + * Insert the other positions if their hash entry is empty. + */ + for ( ; ip + fastHashFillStep < iend + 2; ip += fastHashFillStep) { + U32 const current = (U32)(ip - base); + size_t const hash0 = ZSTD_hashPtr(ip, hBits, mls); + hashTable[hash0] = current; + if (dtlm == ZSTD_dtlm_fast) continue; + /* Only load extra positions for ZSTD_dtlm_full */ + { U32 p; + for (p = 1; p < fastHashFillStep; ++p) { + size_t const hash = ZSTD_hashPtr(ip + p, hBits, mls); + if (hashTable[hash] == 0) { /* not yet filled */ + hashTable[hash] = current + p; + } } } } +} + + +FORCE_INLINE_TEMPLATE size_t +ZSTD_compressBlock_fast_generic( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize, + U32 const mls) +{ + const ZSTD_compressionParameters* const cParams = &ms->cParams; + U32* const hashTable = ms->hashTable; + U32 const hlog = cParams->hashLog; + /* support stepSize of 0 */ + size_t const stepSize = cParams->targetLength + !(cParams->targetLength) + 1; + const BYTE* const base = ms->window.base; + const BYTE* const istart = (const BYTE*)src; + /* We check ip0 (ip + 0) and ip1 (ip + 1) each loop */ + const BYTE* ip0 = istart; + const BYTE* ip1; + const BYTE* anchor = istart; + const U32 endIndex = (U32)((size_t)(istart - base) + srcSize); + const U32 prefixStartIndex = ZSTD_getLowestPrefixIndex(ms, endIndex, cParams->windowLog); + const BYTE* const prefixStart = base + prefixStartIndex; + const BYTE* const iend = istart + srcSize; + const BYTE* const ilimit = iend - HASH_READ_SIZE; + U32 offset_1=rep[0], offset_2=rep[1]; + U32 offsetSaved = 0; + + /* init */ + DEBUGLOG(5, "ZSTD_compressBlock_fast_generic"); + ip0 += (ip0 == prefixStart); + ip1 = ip0 + 1; + { U32 const current = (U32)(ip0 - base); + U32 const windowLow = ZSTD_getLowestPrefixIndex(ms, current, cParams->windowLog); + U32 const maxRep = current - windowLow; + if (offset_2 > maxRep) offsetSaved = offset_2, offset_2 = 0; + if (offset_1 > maxRep) offsetSaved = offset_1, offset_1 = 0; + } + + /* Main Search Loop */ +#ifdef __INTEL_COMPILER + /* From intel 'The vector pragma indicates that the loop should be + * vectorized if it is legal to do so'. Can be used together with + * #pragma ivdep (but have opted to exclude that because intel + * warns against using it).*/ + #pragma vector always +#endif + while (ip1 < ilimit) { /* < instead of <=, because check at ip0+2 */ + size_t mLength; + BYTE const* ip2 = ip0 + 2; + size_t const h0 = ZSTD_hashPtr(ip0, hlog, mls); + U32 const val0 = MEM_read32(ip0); + size_t const h1 = ZSTD_hashPtr(ip1, hlog, mls); + U32 const val1 = MEM_read32(ip1); + U32 const current0 = (U32)(ip0-base); + U32 const current1 = (U32)(ip1-base); + U32 const matchIndex0 = hashTable[h0]; + U32 const matchIndex1 = hashTable[h1]; + BYTE const* repMatch = ip2 - offset_1; + const BYTE* match0 = base + matchIndex0; + const BYTE* match1 = base + matchIndex1; + U32 offcode; + +#if defined(__aarch64__) + PREFETCH_L1(ip0+256); +#endif + + hashTable[h0] = current0; /* update hash table */ + hashTable[h1] = current1; /* update hash table */ + + assert(ip0 + 1 == ip1); + + if ((offset_1 > 0) & (MEM_read32(repMatch) == MEM_read32(ip2))) { + mLength = (ip2[-1] == repMatch[-1]) ? 1 : 0; + ip0 = ip2 - mLength; + match0 = repMatch - mLength; + mLength += 4; + offcode = 0; + goto _match; + } + if ((matchIndex0 > prefixStartIndex) && MEM_read32(match0) == val0) { + /* found a regular match */ + goto _offset; + } + if ((matchIndex1 > prefixStartIndex) && MEM_read32(match1) == val1) { + /* found a regular match after one literal */ + ip0 = ip1; + match0 = match1; + goto _offset; + } + { size_t const step = ((size_t)(ip0-anchor) >> (kSearchStrength - 1)) + stepSize; + assert(step >= 2); + ip0 += step; + ip1 += step; + continue; + } +_offset: /* Requires: ip0, match0 */ + /* Compute the offset code */ + offset_2 = offset_1; + offset_1 = (U32)(ip0-match0); + offcode = offset_1 + ZSTD_REP_MOVE; + mLength = 4; + /* Count the backwards match length */ + while (((ip0>anchor) & (match0>prefixStart)) + && (ip0[-1] == match0[-1])) { ip0--; match0--; mLength++; } /* catch up */ + +_match: /* Requires: ip0, match0, offcode */ + /* Count the forward length */ + mLength += ZSTD_count(ip0+mLength, match0+mLength, iend); + ZSTD_storeSeq(seqStore, (size_t)(ip0-anchor), anchor, iend, offcode, mLength-MINMATCH); + /* match found */ + ip0 += mLength; + anchor = ip0; + + if (ip0 <= ilimit) { + /* Fill Table */ + assert(base+current0+2 > istart); /* check base overflow */ + hashTable[ZSTD_hashPtr(base+current0+2, hlog, mls)] = current0+2; /* here because current+2 could be > iend-8 */ + hashTable[ZSTD_hashPtr(ip0-2, hlog, mls)] = (U32)(ip0-2-base); + + if (offset_2 > 0) { /* offset_2==0 means offset_2 is invalidated */ + while ( (ip0 <= ilimit) && (MEM_read32(ip0) == MEM_read32(ip0 - offset_2)) ) { + /* store sequence */ + size_t const rLength = ZSTD_count(ip0+4, ip0+4-offset_2, iend) + 4; + { U32 const tmpOff = offset_2; offset_2 = offset_1; offset_1 = tmpOff; } /* swap offset_2 <=> offset_1 */ + hashTable[ZSTD_hashPtr(ip0, hlog, mls)] = (U32)(ip0-base); + ip0 += rLength; + ZSTD_storeSeq(seqStore, 0 /*litLen*/, anchor, iend, 0 /*offCode*/, rLength-MINMATCH); + anchor = ip0; + continue; /* faster when present (confirmed on gcc-8) ... (?) */ + } } } + ip1 = ip0 + 1; + } + + /* save reps for next block */ + rep[0] = offset_1 ? offset_1 : offsetSaved; + rep[1] = offset_2 ? offset_2 : offsetSaved; + + /* Return the last literals size */ + return (size_t)(iend - anchor); +} + + +size_t ZSTD_compressBlock_fast( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + U32 const mls = ms->cParams.minMatch; + assert(ms->dictMatchState == NULL); + switch(mls) + { + default: /* includes case 3 */ + case 4 : + return ZSTD_compressBlock_fast_generic(ms, seqStore, rep, src, srcSize, 4); + case 5 : + return ZSTD_compressBlock_fast_generic(ms, seqStore, rep, src, srcSize, 5); + case 6 : + return ZSTD_compressBlock_fast_generic(ms, seqStore, rep, src, srcSize, 6); + case 7 : + return ZSTD_compressBlock_fast_generic(ms, seqStore, rep, src, srcSize, 7); + } +} + +FORCE_INLINE_TEMPLATE +size_t ZSTD_compressBlock_fast_dictMatchState_generic( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize, U32 const mls) +{ + const ZSTD_compressionParameters* const cParams = &ms->cParams; + U32* const hashTable = ms->hashTable; + U32 const hlog = cParams->hashLog; + /* support stepSize of 0 */ + U32 const stepSize = cParams->targetLength + !(cParams->targetLength); + const BYTE* const base = ms->window.base; + const BYTE* const istart = (const BYTE*)src; + const BYTE* ip = istart; + const BYTE* anchor = istart; + const U32 prefixStartIndex = ms->window.dictLimit; + const BYTE* const prefixStart = base + prefixStartIndex; + const BYTE* const iend = istart + srcSize; + const BYTE* const ilimit = iend - HASH_READ_SIZE; + U32 offset_1=rep[0], offset_2=rep[1]; + U32 offsetSaved = 0; + + const ZSTD_matchState_t* const dms = ms->dictMatchState; + const ZSTD_compressionParameters* const dictCParams = &dms->cParams ; + const U32* const dictHashTable = dms->hashTable; + const U32 dictStartIndex = dms->window.dictLimit; + const BYTE* const dictBase = dms->window.base; + const BYTE* const dictStart = dictBase + dictStartIndex; + const BYTE* const dictEnd = dms->window.nextSrc; + const U32 dictIndexDelta = prefixStartIndex - (U32)(dictEnd - dictBase); + const U32 dictAndPrefixLength = (U32)(ip - prefixStart + dictEnd - dictStart); + const U32 dictHLog = dictCParams->hashLog; + + /* if a dictionary is still attached, it necessarily means that + * it is within window size. So we just check it. */ + const U32 maxDistance = 1U << cParams->windowLog; + const U32 endIndex = (U32)((size_t)(ip - base) + srcSize); + assert(endIndex - prefixStartIndex <= maxDistance); + (void)maxDistance; (void)endIndex; /* these variables are not used when assert() is disabled */ + + /* ensure there will be no no underflow + * when translating a dict index into a local index */ + assert(prefixStartIndex >= (U32)(dictEnd - dictBase)); + + /* init */ + DEBUGLOG(5, "ZSTD_compressBlock_fast_dictMatchState_generic"); + ip += (dictAndPrefixLength == 0); + /* dictMatchState repCode checks don't currently handle repCode == 0 + * disabling. */ + assert(offset_1 <= dictAndPrefixLength); + assert(offset_2 <= dictAndPrefixLength); + + /* Main Search Loop */ + while (ip < ilimit) { /* < instead of <=, because repcode check at (ip+1) */ + size_t mLength; + size_t const h = ZSTD_hashPtr(ip, hlog, mls); + U32 const current = (U32)(ip-base); + U32 const matchIndex = hashTable[h]; + const BYTE* match = base + matchIndex; + const U32 repIndex = current + 1 - offset_1; + const BYTE* repMatch = (repIndex < prefixStartIndex) ? + dictBase + (repIndex - dictIndexDelta) : + base + repIndex; + hashTable[h] = current; /* update hash table */ + + if ( ((U32)((prefixStartIndex-1) - repIndex) >= 3) /* intentional underflow : ensure repIndex isn't overlapping dict + prefix */ + && (MEM_read32(repMatch) == MEM_read32(ip+1)) ) { + const BYTE* const repMatchEnd = repIndex < prefixStartIndex ? dictEnd : iend; + mLength = ZSTD_count_2segments(ip+1+4, repMatch+4, iend, repMatchEnd, prefixStart) + 4; + ip++; + ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, mLength-MINMATCH); + } else if ( (matchIndex <= prefixStartIndex) ) { + size_t const dictHash = ZSTD_hashPtr(ip, dictHLog, mls); + U32 const dictMatchIndex = dictHashTable[dictHash]; + const BYTE* dictMatch = dictBase + dictMatchIndex; + if (dictMatchIndex <= dictStartIndex || + MEM_read32(dictMatch) != MEM_read32(ip)) { + assert(stepSize >= 1); + ip += ((ip-anchor) >> kSearchStrength) + stepSize; + continue; + } else { + /* found a dict match */ + U32 const offset = (U32)(current-dictMatchIndex-dictIndexDelta); + mLength = ZSTD_count_2segments(ip+4, dictMatch+4, iend, dictEnd, prefixStart) + 4; + while (((ip>anchor) & (dictMatch>dictStart)) + && (ip[-1] == dictMatch[-1])) { + ip--; dictMatch--; mLength++; + } /* catch up */ + offset_2 = offset_1; + offset_1 = offset; + ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH); + } + } else if (MEM_read32(match) != MEM_read32(ip)) { + /* it's not a match, and we're not going to check the dictionary */ + assert(stepSize >= 1); + ip += ((ip-anchor) >> kSearchStrength) + stepSize; + continue; + } else { + /* found a regular match */ + U32 const offset = (U32)(ip-match); + mLength = ZSTD_count(ip+4, match+4, iend) + 4; + while (((ip>anchor) & (match>prefixStart)) + && (ip[-1] == match[-1])) { ip--; match--; mLength++; } /* catch up */ + offset_2 = offset_1; + offset_1 = offset; + ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH); + } + + /* match found */ + ip += mLength; + anchor = ip; + + if (ip <= ilimit) { + /* Fill Table */ + assert(base+current+2 > istart); /* check base overflow */ + hashTable[ZSTD_hashPtr(base+current+2, hlog, mls)] = current+2; /* here because current+2 could be > iend-8 */ + hashTable[ZSTD_hashPtr(ip-2, hlog, mls)] = (U32)(ip-2-base); + + /* check immediate repcode */ + while (ip <= ilimit) { + U32 const current2 = (U32)(ip-base); + U32 const repIndex2 = current2 - offset_2; + const BYTE* repMatch2 = repIndex2 < prefixStartIndex ? + dictBase - dictIndexDelta + repIndex2 : + base + repIndex2; + if ( ((U32)((prefixStartIndex-1) - (U32)repIndex2) >= 3 /* intentional overflow */) + && (MEM_read32(repMatch2) == MEM_read32(ip)) ) { + const BYTE* const repEnd2 = repIndex2 < prefixStartIndex ? dictEnd : iend; + size_t const repLength2 = ZSTD_count_2segments(ip+4, repMatch2+4, iend, repEnd2, prefixStart) + 4; + U32 tmpOffset = offset_2; offset_2 = offset_1; offset_1 = tmpOffset; /* swap offset_2 <=> offset_1 */ + ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, repLength2-MINMATCH); + hashTable[ZSTD_hashPtr(ip, hlog, mls)] = current2; + ip += repLength2; + anchor = ip; + continue; + } + break; + } + } + } + + /* save reps for next block */ + rep[0] = offset_1 ? offset_1 : offsetSaved; + rep[1] = offset_2 ? offset_2 : offsetSaved; + + /* Return the last literals size */ + return (size_t)(iend - anchor); +} + +size_t ZSTD_compressBlock_fast_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + U32 const mls = ms->cParams.minMatch; + assert(ms->dictMatchState != NULL); + switch(mls) + { + default: /* includes case 3 */ + case 4 : + return ZSTD_compressBlock_fast_dictMatchState_generic(ms, seqStore, rep, src, srcSize, 4); + case 5 : + return ZSTD_compressBlock_fast_dictMatchState_generic(ms, seqStore, rep, src, srcSize, 5); + case 6 : + return ZSTD_compressBlock_fast_dictMatchState_generic(ms, seqStore, rep, src, srcSize, 6); + case 7 : + return ZSTD_compressBlock_fast_dictMatchState_generic(ms, seqStore, rep, src, srcSize, 7); + } +} + + +static size_t ZSTD_compressBlock_fast_extDict_generic( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize, U32 const mls) +{ + const ZSTD_compressionParameters* const cParams = &ms->cParams; + U32* const hashTable = ms->hashTable; + U32 const hlog = cParams->hashLog; + /* support stepSize of 0 */ + U32 const stepSize = cParams->targetLength + !(cParams->targetLength); + const BYTE* const base = ms->window.base; + const BYTE* const dictBase = ms->window.dictBase; + const BYTE* const istart = (const BYTE*)src; + const BYTE* ip = istart; + const BYTE* anchor = istart; + const U32 endIndex = (U32)((size_t)(istart - base) + srcSize); + const U32 lowLimit = ZSTD_getLowestMatchIndex(ms, endIndex, cParams->windowLog); + const U32 dictStartIndex = lowLimit; + const BYTE* const dictStart = dictBase + dictStartIndex; + const U32 dictLimit = ms->window.dictLimit; + const U32 prefixStartIndex = dictLimit < lowLimit ? lowLimit : dictLimit; + const BYTE* const prefixStart = base + prefixStartIndex; + const BYTE* const dictEnd = dictBase + prefixStartIndex; + const BYTE* const iend = istart + srcSize; + const BYTE* const ilimit = iend - 8; + U32 offset_1=rep[0], offset_2=rep[1]; + + DEBUGLOG(5, "ZSTD_compressBlock_fast_extDict_generic (offset_1=%u)", offset_1); + + /* switch to "regular" variant if extDict is invalidated due to maxDistance */ + if (prefixStartIndex == dictStartIndex) + return ZSTD_compressBlock_fast_generic(ms, seqStore, rep, src, srcSize, mls); + + /* Search Loop */ + while (ip < ilimit) { /* < instead of <=, because (ip+1) */ + const size_t h = ZSTD_hashPtr(ip, hlog, mls); + const U32 matchIndex = hashTable[h]; + const BYTE* const matchBase = matchIndex < prefixStartIndex ? dictBase : base; + const BYTE* match = matchBase + matchIndex; + const U32 current = (U32)(ip-base); + const U32 repIndex = current + 1 - offset_1; + const BYTE* const repBase = repIndex < prefixStartIndex ? dictBase : base; + const BYTE* const repMatch = repBase + repIndex; + hashTable[h] = current; /* update hash table */ + DEBUGLOG(7, "offset_1 = %u , current = %u", offset_1, current); + assert(offset_1 <= current +1); /* check repIndex */ + + if ( (((U32)((prefixStartIndex-1) - repIndex) >= 3) /* intentional underflow */ & (repIndex > dictStartIndex)) + && (MEM_read32(repMatch) == MEM_read32(ip+1)) ) { + const BYTE* const repMatchEnd = repIndex < prefixStartIndex ? dictEnd : iend; + size_t const rLength = ZSTD_count_2segments(ip+1 +4, repMatch +4, iend, repMatchEnd, prefixStart) + 4; + ip++; + ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, rLength-MINMATCH); + ip += rLength; + anchor = ip; + } else { + if ( (matchIndex < dictStartIndex) || + (MEM_read32(match) != MEM_read32(ip)) ) { + assert(stepSize >= 1); + ip += ((ip-anchor) >> kSearchStrength) + stepSize; + continue; + } + { const BYTE* const matchEnd = matchIndex < prefixStartIndex ? dictEnd : iend; + const BYTE* const lowMatchPtr = matchIndex < prefixStartIndex ? dictStart : prefixStart; + U32 const offset = current - matchIndex; + size_t mLength = ZSTD_count_2segments(ip+4, match+4, iend, matchEnd, prefixStart) + 4; + while (((ip>anchor) & (match>lowMatchPtr)) && (ip[-1] == match[-1])) { ip--; match--; mLength++; } /* catch up */ + offset_2 = offset_1; offset_1 = offset; /* update offset history */ + ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH); + ip += mLength; + anchor = ip; + } } + + if (ip <= ilimit) { + /* Fill Table */ + hashTable[ZSTD_hashPtr(base+current+2, hlog, mls)] = current+2; + hashTable[ZSTD_hashPtr(ip-2, hlog, mls)] = (U32)(ip-2-base); + /* check immediate repcode */ + while (ip <= ilimit) { + U32 const current2 = (U32)(ip-base); + U32 const repIndex2 = current2 - offset_2; + const BYTE* const repMatch2 = repIndex2 < prefixStartIndex ? dictBase + repIndex2 : base + repIndex2; + if ( (((U32)((prefixStartIndex-1) - repIndex2) >= 3) & (repIndex2 > dictStartIndex)) /* intentional overflow */ + && (MEM_read32(repMatch2) == MEM_read32(ip)) ) { + const BYTE* const repEnd2 = repIndex2 < prefixStartIndex ? dictEnd : iend; + size_t const repLength2 = ZSTD_count_2segments(ip+4, repMatch2+4, iend, repEnd2, prefixStart) + 4; + { U32 const tmpOffset = offset_2; offset_2 = offset_1; offset_1 = tmpOffset; } /* swap offset_2 <=> offset_1 */ + ZSTD_storeSeq(seqStore, 0 /*litlen*/, anchor, iend, 0 /*offcode*/, repLength2-MINMATCH); + hashTable[ZSTD_hashPtr(ip, hlog, mls)] = current2; + ip += repLength2; + anchor = ip; + continue; + } + break; + } } } + + /* save reps for next block */ + rep[0] = offset_1; + rep[1] = offset_2; + + /* Return the last literals size */ + return (size_t)(iend - anchor); +} + + +size_t ZSTD_compressBlock_fast_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + U32 const mls = ms->cParams.minMatch; + switch(mls) + { + default: /* includes case 3 */ + case 4 : + return ZSTD_compressBlock_fast_extDict_generic(ms, seqStore, rep, src, srcSize, 4); + case 5 : + return ZSTD_compressBlock_fast_extDict_generic(ms, seqStore, rep, src, srcSize, 5); + case 6 : + return ZSTD_compressBlock_fast_extDict_generic(ms, seqStore, rep, src, srcSize, 6); + case 7 : + return ZSTD_compressBlock_fast_extDict_generic(ms, seqStore, rep, src, srcSize, 7); + } +} diff --git a/module/zstd/lib/compress/zstd_fast.h b/module/zstd/lib/compress/zstd_fast.h new file mode 100644 index 000000000000..cf6aaa8e6750 --- /dev/null +++ b/module/zstd/lib/compress/zstd_fast.h @@ -0,0 +1,37 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#ifndef ZSTD_FAST_H +#define ZSTD_FAST_H + +#if defined (__cplusplus) +extern "C" { +#endif + +#include "../common/mem.h" /* U32 */ +#include "zstd_compress_internal.h" + +void ZSTD_fillHashTable(ZSTD_matchState_t* ms, + void const* end, ZSTD_dictTableLoadMethod_e dtlm); +size_t ZSTD_compressBlock_fast( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_fast_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_fast_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); + +#if defined (__cplusplus) +} +#endif + +#endif /* ZSTD_FAST_H */ diff --git a/module/zstd/lib/compress/zstd_lazy.c b/module/zstd/lib/compress/zstd_lazy.c new file mode 100644 index 000000000000..4cf5c88b5325 --- /dev/null +++ b/module/zstd/lib/compress/zstd_lazy.c @@ -0,0 +1,1138 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#include "zstd_compress_internal.h" +#include "zstd_lazy.h" + + +/*-************************************* +* Binary Tree search +***************************************/ + +static void +ZSTD_updateDUBT(ZSTD_matchState_t* ms, + const BYTE* ip, const BYTE* iend, + U32 mls) +{ + const ZSTD_compressionParameters* const cParams = &ms->cParams; + U32* const hashTable = ms->hashTable; + U32 const hashLog = cParams->hashLog; + + U32* const bt = ms->chainTable; + U32 const btLog = cParams->chainLog - 1; + U32 const btMask = (1 << btLog) - 1; + + const BYTE* const base = ms->window.base; + U32 const target = (U32)(ip - base); + U32 idx = ms->nextToUpdate; + + if (idx != target) + DEBUGLOG(7, "ZSTD_updateDUBT, from %u to %u (dictLimit:%u)", + idx, target, ms->window.dictLimit); + assert(ip + 8 <= iend); /* condition for ZSTD_hashPtr */ + (void)iend; + + assert(idx >= ms->window.dictLimit); /* condition for valid base+idx */ + for ( ; idx < target ; idx++) { + size_t const h = ZSTD_hashPtr(base + idx, hashLog, mls); /* assumption : ip + 8 <= iend */ + U32 const matchIndex = hashTable[h]; + + U32* const nextCandidatePtr = bt + 2*(idx&btMask); + U32* const sortMarkPtr = nextCandidatePtr + 1; + + DEBUGLOG(8, "ZSTD_updateDUBT: insert %u", idx); + hashTable[h] = idx; /* Update Hash Table */ + *nextCandidatePtr = matchIndex; /* update BT like a chain */ + *sortMarkPtr = ZSTD_DUBT_UNSORTED_MARK; + } + ms->nextToUpdate = target; +} + + +/** ZSTD_insertDUBT1() : + * sort one already inserted but unsorted position + * assumption : current >= btlow == (current - btmask) + * doesn't fail */ +static void +ZSTD_insertDUBT1(ZSTD_matchState_t* ms, + U32 current, const BYTE* inputEnd, + U32 nbCompares, U32 btLow, + const ZSTD_dictMode_e dictMode) +{ + const ZSTD_compressionParameters* const cParams = &ms->cParams; + U32* const bt = ms->chainTable; + U32 const btLog = cParams->chainLog - 1; + U32 const btMask = (1 << btLog) - 1; + size_t commonLengthSmaller=0, commonLengthLarger=0; + const BYTE* const base = ms->window.base; + const BYTE* const dictBase = ms->window.dictBase; + const U32 dictLimit = ms->window.dictLimit; + const BYTE* const ip = (current>=dictLimit) ? base + current : dictBase + current; + const BYTE* const iend = (current>=dictLimit) ? inputEnd : dictBase + dictLimit; + const BYTE* const dictEnd = dictBase + dictLimit; + const BYTE* const prefixStart = base + dictLimit; + const BYTE* match; + U32* smallerPtr = bt + 2*(current&btMask); + U32* largerPtr = smallerPtr + 1; + U32 matchIndex = *smallerPtr; /* this candidate is unsorted : next sorted candidate is reached through *smallerPtr, while *largerPtr contains previous unsorted candidate (which is already saved and can be overwritten) */ + U32 dummy32; /* to be nullified at the end */ + U32 const windowValid = ms->window.lowLimit; + U32 const maxDistance = 1U << cParams->windowLog; + U32 const windowLow = (current - windowValid > maxDistance) ? current - maxDistance : windowValid; + + + DEBUGLOG(8, "ZSTD_insertDUBT1(%u) (dictLimit=%u, lowLimit=%u)", + current, dictLimit, windowLow); + assert(current >= btLow); + assert(ip < iend); /* condition for ZSTD_count */ + + while (nbCompares-- && (matchIndex > windowLow)) { + U32* const nextPtr = bt + 2*(matchIndex & btMask); + size_t matchLength = MIN(commonLengthSmaller, commonLengthLarger); /* guaranteed minimum nb of common bytes */ + assert(matchIndex < current); + /* note : all candidates are now supposed sorted, + * but it's still possible to have nextPtr[1] == ZSTD_DUBT_UNSORTED_MARK + * when a real index has the same value as ZSTD_DUBT_UNSORTED_MARK */ + + if ( (dictMode != ZSTD_extDict) + || (matchIndex+matchLength >= dictLimit) /* both in current segment*/ + || (current < dictLimit) /* both in extDict */) { + const BYTE* const mBase = ( (dictMode != ZSTD_extDict) + || (matchIndex+matchLength >= dictLimit)) ? + base : dictBase; + assert( (matchIndex+matchLength >= dictLimit) /* might be wrong if extDict is incorrectly set to 0 */ + || (current < dictLimit) ); + match = mBase + matchIndex; + matchLength += ZSTD_count(ip+matchLength, match+matchLength, iend); + } else { + match = dictBase + matchIndex; + matchLength += ZSTD_count_2segments(ip+matchLength, match+matchLength, iend, dictEnd, prefixStart); + if (matchIndex+matchLength >= dictLimit) + match = base + matchIndex; /* preparation for next read of match[matchLength] */ + } + + DEBUGLOG(8, "ZSTD_insertDUBT1: comparing %u with %u : found %u common bytes ", + current, matchIndex, (U32)matchLength); + + if (ip+matchLength == iend) { /* equal : no way to know if inf or sup */ + break; /* drop , to guarantee consistency ; miss a bit of compression, but other solutions can corrupt tree */ + } + + if (match[matchLength] < ip[matchLength]) { /* necessarily within buffer */ + /* match is smaller than current */ + *smallerPtr = matchIndex; /* update smaller idx */ + commonLengthSmaller = matchLength; /* all smaller will now have at least this guaranteed common length */ + if (matchIndex <= btLow) { smallerPtr=&dummy32; break; } /* beyond tree size, stop searching */ + DEBUGLOG(8, "ZSTD_insertDUBT1: %u (>btLow=%u) is smaller : next => %u", + matchIndex, btLow, nextPtr[1]); + smallerPtr = nextPtr+1; /* new "candidate" => larger than match, which was smaller than target */ + matchIndex = nextPtr[1]; /* new matchIndex, larger than previous and closer to current */ + } else { + /* match is larger than current */ + *largerPtr = matchIndex; + commonLengthLarger = matchLength; + if (matchIndex <= btLow) { largerPtr=&dummy32; break; } /* beyond tree size, stop searching */ + DEBUGLOG(8, "ZSTD_insertDUBT1: %u (>btLow=%u) is larger => %u", + matchIndex, btLow, nextPtr[0]); + largerPtr = nextPtr; + matchIndex = nextPtr[0]; + } } + + *smallerPtr = *largerPtr = 0; +} + + +static size_t +ZSTD_DUBT_findBetterDictMatch ( + ZSTD_matchState_t* ms, + const BYTE* const ip, const BYTE* const iend, + size_t* offsetPtr, + size_t bestLength, + U32 nbCompares, + U32 const mls, + const ZSTD_dictMode_e dictMode) +{ + const ZSTD_matchState_t * const dms = ms->dictMatchState; + const ZSTD_compressionParameters* const dmsCParams = &dms->cParams; + const U32 * const dictHashTable = dms->hashTable; + U32 const hashLog = dmsCParams->hashLog; + size_t const h = ZSTD_hashPtr(ip, hashLog, mls); + U32 dictMatchIndex = dictHashTable[h]; + + const BYTE* const base = ms->window.base; + const BYTE* const prefixStart = base + ms->window.dictLimit; + U32 const current = (U32)(ip-base); + const BYTE* const dictBase = dms->window.base; + const BYTE* const dictEnd = dms->window.nextSrc; + U32 const dictHighLimit = (U32)(dms->window.nextSrc - dms->window.base); + U32 const dictLowLimit = dms->window.lowLimit; + U32 const dictIndexDelta = ms->window.lowLimit - dictHighLimit; + + U32* const dictBt = dms->chainTable; + U32 const btLog = dmsCParams->chainLog - 1; + U32 const btMask = (1 << btLog) - 1; + U32 const btLow = (btMask >= dictHighLimit - dictLowLimit) ? dictLowLimit : dictHighLimit - btMask; + + size_t commonLengthSmaller=0, commonLengthLarger=0; + + (void)dictMode; + assert(dictMode == ZSTD_dictMatchState); + + while (nbCompares-- && (dictMatchIndex > dictLowLimit)) { + U32* const nextPtr = dictBt + 2*(dictMatchIndex & btMask); + size_t matchLength = MIN(commonLengthSmaller, commonLengthLarger); /* guaranteed minimum nb of common bytes */ + const BYTE* match = dictBase + dictMatchIndex; + matchLength += ZSTD_count_2segments(ip+matchLength, match+matchLength, iend, dictEnd, prefixStart); + if (dictMatchIndex+matchLength >= dictHighLimit) + match = base + dictMatchIndex + dictIndexDelta; /* to prepare for next usage of match[matchLength] */ + + if (matchLength > bestLength) { + U32 matchIndex = dictMatchIndex + dictIndexDelta; + if ( (4*(int)(matchLength-bestLength)) > (int)(ZSTD_highbit32(current-matchIndex+1) - ZSTD_highbit32((U32)offsetPtr[0]+1)) ) { + DEBUGLOG(9, "ZSTD_DUBT_findBetterDictMatch(%u) : found better match length %u -> %u and offsetCode %u -> %u (dictMatchIndex %u, matchIndex %u)", + current, (U32)bestLength, (U32)matchLength, (U32)*offsetPtr, ZSTD_REP_MOVE + current - matchIndex, dictMatchIndex, matchIndex); + bestLength = matchLength, *offsetPtr = ZSTD_REP_MOVE + current - matchIndex; + } + if (ip+matchLength == iend) { /* reached end of input : ip[matchLength] is not valid, no way to know if it's larger or smaller than match */ + break; /* drop, to guarantee consistency (miss a little bit of compression) */ + } + } + + if (match[matchLength] < ip[matchLength]) { + if (dictMatchIndex <= btLow) { break; } /* beyond tree size, stop the search */ + commonLengthSmaller = matchLength; /* all smaller will now have at least this guaranteed common length */ + dictMatchIndex = nextPtr[1]; /* new matchIndex larger than previous (closer to current) */ + } else { + /* match is larger than current */ + if (dictMatchIndex <= btLow) { break; } /* beyond tree size, stop the search */ + commonLengthLarger = matchLength; + dictMatchIndex = nextPtr[0]; + } + } + + if (bestLength >= MINMATCH) { + U32 const mIndex = current - ((U32)*offsetPtr - ZSTD_REP_MOVE); (void)mIndex; + DEBUGLOG(8, "ZSTD_DUBT_findBetterDictMatch(%u) : found match of length %u and offsetCode %u (pos %u)", + current, (U32)bestLength, (U32)*offsetPtr, mIndex); + } + return bestLength; + +} + + +static size_t +ZSTD_DUBT_findBestMatch(ZSTD_matchState_t* ms, + const BYTE* const ip, const BYTE* const iend, + size_t* offsetPtr, + U32 const mls, + const ZSTD_dictMode_e dictMode) +{ + const ZSTD_compressionParameters* const cParams = &ms->cParams; + U32* const hashTable = ms->hashTable; + U32 const hashLog = cParams->hashLog; + size_t const h = ZSTD_hashPtr(ip, hashLog, mls); + U32 matchIndex = hashTable[h]; + + const BYTE* const base = ms->window.base; + U32 const current = (U32)(ip-base); + U32 const windowLow = ZSTD_getLowestMatchIndex(ms, current, cParams->windowLog); + + U32* const bt = ms->chainTable; + U32 const btLog = cParams->chainLog - 1; + U32 const btMask = (1 << btLog) - 1; + U32 const btLow = (btMask >= current) ? 0 : current - btMask; + U32 const unsortLimit = MAX(btLow, windowLow); + + U32* nextCandidate = bt + 2*(matchIndex&btMask); + U32* unsortedMark = bt + 2*(matchIndex&btMask) + 1; + U32 nbCompares = 1U << cParams->searchLog; + U32 nbCandidates = nbCompares; + U32 previousCandidate = 0; + + DEBUGLOG(7, "ZSTD_DUBT_findBestMatch (%u) ", current); + assert(ip <= iend-8); /* required for h calculation */ + + /* reach end of unsorted candidates list */ + while ( (matchIndex > unsortLimit) + && (*unsortedMark == ZSTD_DUBT_UNSORTED_MARK) + && (nbCandidates > 1) ) { + DEBUGLOG(8, "ZSTD_DUBT_findBestMatch: candidate %u is unsorted", + matchIndex); + *unsortedMark = previousCandidate; /* the unsortedMark becomes a reversed chain, to move up back to original position */ + previousCandidate = matchIndex; + matchIndex = *nextCandidate; + nextCandidate = bt + 2*(matchIndex&btMask); + unsortedMark = bt + 2*(matchIndex&btMask) + 1; + nbCandidates --; + } + + /* nullify last candidate if it's still unsorted + * simplification, detrimental to compression ratio, beneficial for speed */ + if ( (matchIndex > unsortLimit) + && (*unsortedMark==ZSTD_DUBT_UNSORTED_MARK) ) { + DEBUGLOG(7, "ZSTD_DUBT_findBestMatch: nullify last unsorted candidate %u", + matchIndex); + *nextCandidate = *unsortedMark = 0; + } + + /* batch sort stacked candidates */ + matchIndex = previousCandidate; + while (matchIndex) { /* will end on matchIndex == 0 */ + U32* const nextCandidateIdxPtr = bt + 2*(matchIndex&btMask) + 1; + U32 const nextCandidateIdx = *nextCandidateIdxPtr; + ZSTD_insertDUBT1(ms, matchIndex, iend, + nbCandidates, unsortLimit, dictMode); + matchIndex = nextCandidateIdx; + nbCandidates++; + } + + /* find longest match */ + { size_t commonLengthSmaller = 0, commonLengthLarger = 0; + const BYTE* const dictBase = ms->window.dictBase; + const U32 dictLimit = ms->window.dictLimit; + const BYTE* const dictEnd = dictBase + dictLimit; + const BYTE* const prefixStart = base + dictLimit; + U32* smallerPtr = bt + 2*(current&btMask); + U32* largerPtr = bt + 2*(current&btMask) + 1; + U32 matchEndIdx = current + 8 + 1; + U32 dummy32; /* to be nullified at the end */ + size_t bestLength = 0; + + matchIndex = hashTable[h]; + hashTable[h] = current; /* Update Hash Table */ + + while (nbCompares-- && (matchIndex > windowLow)) { + U32* const nextPtr = bt + 2*(matchIndex & btMask); + size_t matchLength = MIN(commonLengthSmaller, commonLengthLarger); /* guaranteed minimum nb of common bytes */ + const BYTE* match; + + if ((dictMode != ZSTD_extDict) || (matchIndex+matchLength >= dictLimit)) { + match = base + matchIndex; + matchLength += ZSTD_count(ip+matchLength, match+matchLength, iend); + } else { + match = dictBase + matchIndex; + matchLength += ZSTD_count_2segments(ip+matchLength, match+matchLength, iend, dictEnd, prefixStart); + if (matchIndex+matchLength >= dictLimit) + match = base + matchIndex; /* to prepare for next usage of match[matchLength] */ + } + + if (matchLength > bestLength) { + if (matchLength > matchEndIdx - matchIndex) + matchEndIdx = matchIndex + (U32)matchLength; + if ( (4*(int)(matchLength-bestLength)) > (int)(ZSTD_highbit32(current-matchIndex+1) - ZSTD_highbit32((U32)offsetPtr[0]+1)) ) + bestLength = matchLength, *offsetPtr = ZSTD_REP_MOVE + current - matchIndex; + if (ip+matchLength == iend) { /* equal : no way to know if inf or sup */ + if (dictMode == ZSTD_dictMatchState) { + nbCompares = 0; /* in addition to avoiding checking any + * further in this loop, make sure we + * skip checking in the dictionary. */ + } + break; /* drop, to guarantee consistency (miss a little bit of compression) */ + } + } + + if (match[matchLength] < ip[matchLength]) { + /* match is smaller than current */ + *smallerPtr = matchIndex; /* update smaller idx */ + commonLengthSmaller = matchLength; /* all smaller will now have at least this guaranteed common length */ + if (matchIndex <= btLow) { smallerPtr=&dummy32; break; } /* beyond tree size, stop the search */ + smallerPtr = nextPtr+1; /* new "smaller" => larger of match */ + matchIndex = nextPtr[1]; /* new matchIndex larger than previous (closer to current) */ + } else { + /* match is larger than current */ + *largerPtr = matchIndex; + commonLengthLarger = matchLength; + if (matchIndex <= btLow) { largerPtr=&dummy32; break; } /* beyond tree size, stop the search */ + largerPtr = nextPtr; + matchIndex = nextPtr[0]; + } } + + *smallerPtr = *largerPtr = 0; + + if (dictMode == ZSTD_dictMatchState && nbCompares) { + bestLength = ZSTD_DUBT_findBetterDictMatch( + ms, ip, iend, + offsetPtr, bestLength, nbCompares, + mls, dictMode); + } + + assert(matchEndIdx > current+8); /* ensure nextToUpdate is increased */ + ms->nextToUpdate = matchEndIdx - 8; /* skip repetitive patterns */ + if (bestLength >= MINMATCH) { + U32 const mIndex = current - ((U32)*offsetPtr - ZSTD_REP_MOVE); (void)mIndex; + DEBUGLOG(8, "ZSTD_DUBT_findBestMatch(%u) : found match of length %u and offsetCode %u (pos %u)", + current, (U32)bestLength, (U32)*offsetPtr, mIndex); + } + return bestLength; + } +} + + +/** ZSTD_BtFindBestMatch() : Tree updater, providing best match */ +FORCE_INLINE_TEMPLATE size_t +ZSTD_BtFindBestMatch( ZSTD_matchState_t* ms, + const BYTE* const ip, const BYTE* const iLimit, + size_t* offsetPtr, + const U32 mls /* template */, + const ZSTD_dictMode_e dictMode) +{ + DEBUGLOG(7, "ZSTD_BtFindBestMatch"); + if (ip < ms->window.base + ms->nextToUpdate) return 0; /* skipped area */ + ZSTD_updateDUBT(ms, ip, iLimit, mls); + return ZSTD_DUBT_findBestMatch(ms, ip, iLimit, offsetPtr, mls, dictMode); +} + + +static size_t +ZSTD_BtFindBestMatch_selectMLS ( ZSTD_matchState_t* ms, + const BYTE* ip, const BYTE* const iLimit, + size_t* offsetPtr) +{ + switch(ms->cParams.minMatch) + { + default : /* includes case 3 */ + case 4 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 4, ZSTD_noDict); + case 5 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 5, ZSTD_noDict); + case 7 : + case 6 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 6, ZSTD_noDict); + } +} + + +static size_t ZSTD_BtFindBestMatch_dictMatchState_selectMLS ( + ZSTD_matchState_t* ms, + const BYTE* ip, const BYTE* const iLimit, + size_t* offsetPtr) +{ + switch(ms->cParams.minMatch) + { + default : /* includes case 3 */ + case 4 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 4, ZSTD_dictMatchState); + case 5 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 5, ZSTD_dictMatchState); + case 7 : + case 6 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 6, ZSTD_dictMatchState); + } +} + + +static size_t ZSTD_BtFindBestMatch_extDict_selectMLS ( + ZSTD_matchState_t* ms, + const BYTE* ip, const BYTE* const iLimit, + size_t* offsetPtr) +{ + switch(ms->cParams.minMatch) + { + default : /* includes case 3 */ + case 4 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 4, ZSTD_extDict); + case 5 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 5, ZSTD_extDict); + case 7 : + case 6 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 6, ZSTD_extDict); + } +} + + + +/* ********************************* +* Hash Chain +***********************************/ +#define NEXT_IN_CHAIN(d, mask) chainTable[(d) & (mask)] + +/* Update chains up to ip (excluded) + Assumption : always within prefix (i.e. not within extDict) */ +static U32 ZSTD_insertAndFindFirstIndex_internal( + ZSTD_matchState_t* ms, + const ZSTD_compressionParameters* const cParams, + const BYTE* ip, U32 const mls) +{ + U32* const hashTable = ms->hashTable; + const U32 hashLog = cParams->hashLog; + U32* const chainTable = ms->chainTable; + const U32 chainMask = (1 << cParams->chainLog) - 1; + const BYTE* const base = ms->window.base; + const U32 target = (U32)(ip - base); + U32 idx = ms->nextToUpdate; + + while(idx < target) { /* catch up */ + size_t const h = ZSTD_hashPtr(base+idx, hashLog, mls); + NEXT_IN_CHAIN(idx, chainMask) = hashTable[h]; + hashTable[h] = idx; + idx++; + } + + ms->nextToUpdate = target; + return hashTable[ZSTD_hashPtr(ip, hashLog, mls)]; +} + +U32 ZSTD_insertAndFindFirstIndex(ZSTD_matchState_t* ms, const BYTE* ip) { + const ZSTD_compressionParameters* const cParams = &ms->cParams; + return ZSTD_insertAndFindFirstIndex_internal(ms, cParams, ip, ms->cParams.minMatch); +} + + +/* inlining is important to hardwire a hot branch (template emulation) */ +FORCE_INLINE_TEMPLATE +size_t ZSTD_HcFindBestMatch_generic ( + ZSTD_matchState_t* ms, + const BYTE* const ip, const BYTE* const iLimit, + size_t* offsetPtr, + const U32 mls, const ZSTD_dictMode_e dictMode) +{ + const ZSTD_compressionParameters* const cParams = &ms->cParams; + U32* const chainTable = ms->chainTable; + const U32 chainSize = (1 << cParams->chainLog); + const U32 chainMask = chainSize-1; + const BYTE* const base = ms->window.base; + const BYTE* const dictBase = ms->window.dictBase; + const U32 dictLimit = ms->window.dictLimit; + const BYTE* const prefixStart = base + dictLimit; + const BYTE* const dictEnd = dictBase + dictLimit; + const U32 current = (U32)(ip-base); + const U32 maxDistance = 1U << cParams->windowLog; + const U32 lowestValid = ms->window.lowLimit; + const U32 withinMaxDistance = (current - lowestValid > maxDistance) ? current - maxDistance : lowestValid; + const U32 isDictionary = (ms->loadedDictEnd != 0); + const U32 lowLimit = isDictionary ? lowestValid : withinMaxDistance; + const U32 minChain = current > chainSize ? current - chainSize : 0; + U32 nbAttempts = 1U << cParams->searchLog; + size_t ml=4-1; + + /* HC4 match finder */ + U32 matchIndex = ZSTD_insertAndFindFirstIndex_internal(ms, cParams, ip, mls); + + for ( ; (matchIndex>lowLimit) & (nbAttempts>0) ; nbAttempts--) { + size_t currentMl=0; + if ((dictMode != ZSTD_extDict) || matchIndex >= dictLimit) { + const BYTE* const match = base + matchIndex; + assert(matchIndex >= dictLimit); /* ensures this is true if dictMode != ZSTD_extDict */ + if (match[ml] == ip[ml]) /* potentially better */ + currentMl = ZSTD_count(ip, match, iLimit); + } else { + const BYTE* const match = dictBase + matchIndex; + assert(match+4 <= dictEnd); + if (MEM_read32(match) == MEM_read32(ip)) /* assumption : matchIndex <= dictLimit-4 (by table construction) */ + currentMl = ZSTD_count_2segments(ip+4, match+4, iLimit, dictEnd, prefixStart) + 4; + } + + /* save best solution */ + if (currentMl > ml) { + ml = currentMl; + *offsetPtr = current - matchIndex + ZSTD_REP_MOVE; + if (ip+currentMl == iLimit) break; /* best possible, avoids read overflow on next attempt */ + } + + if (matchIndex <= minChain) break; + matchIndex = NEXT_IN_CHAIN(matchIndex, chainMask); + } + + if (dictMode == ZSTD_dictMatchState) { + const ZSTD_matchState_t* const dms = ms->dictMatchState; + const U32* const dmsChainTable = dms->chainTable; + const U32 dmsChainSize = (1 << dms->cParams.chainLog); + const U32 dmsChainMask = dmsChainSize - 1; + const U32 dmsLowestIndex = dms->window.dictLimit; + const BYTE* const dmsBase = dms->window.base; + const BYTE* const dmsEnd = dms->window.nextSrc; + const U32 dmsSize = (U32)(dmsEnd - dmsBase); + const U32 dmsIndexDelta = dictLimit - dmsSize; + const U32 dmsMinChain = dmsSize > dmsChainSize ? dmsSize - dmsChainSize : 0; + + matchIndex = dms->hashTable[ZSTD_hashPtr(ip, dms->cParams.hashLog, mls)]; + + for ( ; (matchIndex>dmsLowestIndex) & (nbAttempts>0) ; nbAttempts--) { + size_t currentMl=0; + const BYTE* const match = dmsBase + matchIndex; + assert(match+4 <= dmsEnd); + if (MEM_read32(match) == MEM_read32(ip)) /* assumption : matchIndex <= dictLimit-4 (by table construction) */ + currentMl = ZSTD_count_2segments(ip+4, match+4, iLimit, dmsEnd, prefixStart) + 4; + + /* save best solution */ + if (currentMl > ml) { + ml = currentMl; + *offsetPtr = current - (matchIndex + dmsIndexDelta) + ZSTD_REP_MOVE; + if (ip+currentMl == iLimit) break; /* best possible, avoids read overflow on next attempt */ + } + + if (matchIndex <= dmsMinChain) break; + matchIndex = dmsChainTable[matchIndex & dmsChainMask]; + } + } + + return ml; +} + + +FORCE_INLINE_TEMPLATE size_t ZSTD_HcFindBestMatch_selectMLS ( + ZSTD_matchState_t* ms, + const BYTE* ip, const BYTE* const iLimit, + size_t* offsetPtr) +{ + switch(ms->cParams.minMatch) + { + default : /* includes case 3 */ + case 4 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 4, ZSTD_noDict); + case 5 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 5, ZSTD_noDict); + case 7 : + case 6 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 6, ZSTD_noDict); + } +} + + +static size_t ZSTD_HcFindBestMatch_dictMatchState_selectMLS ( + ZSTD_matchState_t* ms, + const BYTE* ip, const BYTE* const iLimit, + size_t* offsetPtr) +{ + switch(ms->cParams.minMatch) + { + default : /* includes case 3 */ + case 4 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 4, ZSTD_dictMatchState); + case 5 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 5, ZSTD_dictMatchState); + case 7 : + case 6 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 6, ZSTD_dictMatchState); + } +} + + +FORCE_INLINE_TEMPLATE size_t ZSTD_HcFindBestMatch_extDict_selectMLS ( + ZSTD_matchState_t* ms, + const BYTE* ip, const BYTE* const iLimit, + size_t* offsetPtr) +{ + switch(ms->cParams.minMatch) + { + default : /* includes case 3 */ + case 4 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 4, ZSTD_extDict); + case 5 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 5, ZSTD_extDict); + case 7 : + case 6 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 6, ZSTD_extDict); + } +} + + +/* ******************************* +* Common parser - lazy strategy +*********************************/ +typedef enum { search_hashChain, search_binaryTree } searchMethod_e; + +FORCE_INLINE_TEMPLATE size_t +ZSTD_compressBlock_lazy_generic( + ZSTD_matchState_t* ms, seqStore_t* seqStore, + U32 rep[ZSTD_REP_NUM], + const void* src, size_t srcSize, + const searchMethod_e searchMethod, const U32 depth, + ZSTD_dictMode_e const dictMode) +{ + const BYTE* const istart = (const BYTE*)src; + const BYTE* ip = istart; + const BYTE* anchor = istart; + const BYTE* const iend = istart + srcSize; + const BYTE* const ilimit = iend - 8; + const BYTE* const base = ms->window.base; + const U32 prefixLowestIndex = ms->window.dictLimit; + const BYTE* const prefixLowest = base + prefixLowestIndex; + + typedef size_t (*searchMax_f)( + ZSTD_matchState_t* ms, + const BYTE* ip, const BYTE* iLimit, size_t* offsetPtr); + searchMax_f const searchMax = dictMode == ZSTD_dictMatchState ? + (searchMethod==search_binaryTree ? ZSTD_BtFindBestMatch_dictMatchState_selectMLS + : ZSTD_HcFindBestMatch_dictMatchState_selectMLS) : + (searchMethod==search_binaryTree ? ZSTD_BtFindBestMatch_selectMLS + : ZSTD_HcFindBestMatch_selectMLS); + U32 offset_1 = rep[0], offset_2 = rep[1], savedOffset=0; + + const ZSTD_matchState_t* const dms = ms->dictMatchState; + const U32 dictLowestIndex = dictMode == ZSTD_dictMatchState ? + dms->window.dictLimit : 0; + const BYTE* const dictBase = dictMode == ZSTD_dictMatchState ? + dms->window.base : NULL; + const BYTE* const dictLowest = dictMode == ZSTD_dictMatchState ? + dictBase + dictLowestIndex : NULL; + const BYTE* const dictEnd = dictMode == ZSTD_dictMatchState ? + dms->window.nextSrc : NULL; + const U32 dictIndexDelta = dictMode == ZSTD_dictMatchState ? + prefixLowestIndex - (U32)(dictEnd - dictBase) : + 0; + const U32 dictAndPrefixLength = (U32)((ip - prefixLowest) + (dictEnd - dictLowest)); + + DEBUGLOG(5, "ZSTD_compressBlock_lazy_generic (dictMode=%u)", (U32)dictMode); + + /* init */ + ip += (dictAndPrefixLength == 0); + if (dictMode == ZSTD_noDict) { + U32 const current = (U32)(ip - base); + U32 const windowLow = ZSTD_getLowestPrefixIndex(ms, current, ms->cParams.windowLog); + U32 const maxRep = current - windowLow; + if (offset_2 > maxRep) savedOffset = offset_2, offset_2 = 0; + if (offset_1 > maxRep) savedOffset = offset_1, offset_1 = 0; + } + if (dictMode == ZSTD_dictMatchState) { + /* dictMatchState repCode checks don't currently handle repCode == 0 + * disabling. */ + assert(offset_1 <= dictAndPrefixLength); + assert(offset_2 <= dictAndPrefixLength); + } + + /* Match Loop */ +#if defined(__GNUC__) && defined(__x86_64__) + /* I've measured random a 5% speed loss on levels 5 & 6 (greedy) when the + * code alignment is perturbed. To fix the instability align the loop on 32-bytes. + */ + __asm__(".p2align 5"); +#endif + while (ip < ilimit) { + size_t matchLength=0; + size_t offset=0; + const BYTE* start=ip+1; + + /* check repCode */ + if (dictMode == ZSTD_dictMatchState) { + const U32 repIndex = (U32)(ip - base) + 1 - offset_1; + const BYTE* repMatch = (dictMode == ZSTD_dictMatchState + && repIndex < prefixLowestIndex) ? + dictBase + (repIndex - dictIndexDelta) : + base + repIndex; + if (((U32)((prefixLowestIndex-1) - repIndex) >= 3 /* intentional underflow */) + && (MEM_read32(repMatch) == MEM_read32(ip+1)) ) { + const BYTE* repMatchEnd = repIndex < prefixLowestIndex ? dictEnd : iend; + matchLength = ZSTD_count_2segments(ip+1+4, repMatch+4, iend, repMatchEnd, prefixLowest) + 4; + if (depth==0) goto _storeSequence; + } + } + if ( dictMode == ZSTD_noDict + && ((offset_1 > 0) & (MEM_read32(ip+1-offset_1) == MEM_read32(ip+1)))) { + matchLength = ZSTD_count(ip+1+4, ip+1+4-offset_1, iend) + 4; + if (depth==0) goto _storeSequence; + } + + /* first search (depth 0) */ + { size_t offsetFound = 999999999; + size_t const ml2 = searchMax(ms, ip, iend, &offsetFound); + if (ml2 > matchLength) + matchLength = ml2, start = ip, offset=offsetFound; + } + + if (matchLength < 4) { + ip += ((ip-anchor) >> kSearchStrength) + 1; /* jump faster over incompressible sections */ + continue; + } + + /* let's try to find a better solution */ + if (depth>=1) + while (ip0) & (MEM_read32(ip) == MEM_read32(ip - offset_1)))) { + size_t const mlRep = ZSTD_count(ip+4, ip+4-offset_1, iend) + 4; + int const gain2 = (int)(mlRep * 3); + int const gain1 = (int)(matchLength*3 - ZSTD_highbit32((U32)offset+1) + 1); + if ((mlRep >= 4) && (gain2 > gain1)) + matchLength = mlRep, offset = 0, start = ip; + } + if (dictMode == ZSTD_dictMatchState) { + const U32 repIndex = (U32)(ip - base) - offset_1; + const BYTE* repMatch = repIndex < prefixLowestIndex ? + dictBase + (repIndex - dictIndexDelta) : + base + repIndex; + if (((U32)((prefixLowestIndex-1) - repIndex) >= 3 /* intentional underflow */) + && (MEM_read32(repMatch) == MEM_read32(ip)) ) { + const BYTE* repMatchEnd = repIndex < prefixLowestIndex ? dictEnd : iend; + size_t const mlRep = ZSTD_count_2segments(ip+4, repMatch+4, iend, repMatchEnd, prefixLowest) + 4; + int const gain2 = (int)(mlRep * 3); + int const gain1 = (int)(matchLength*3 - ZSTD_highbit32((U32)offset+1) + 1); + if ((mlRep >= 4) && (gain2 > gain1)) + matchLength = mlRep, offset = 0, start = ip; + } + } + { size_t offset2=999999999; + size_t const ml2 = searchMax(ms, ip, iend, &offset2); + int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)offset2+1)); /* raw approx */ + int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 4); + if ((ml2 >= 4) && (gain2 > gain1)) { + matchLength = ml2, offset = offset2, start = ip; + continue; /* search a better one */ + } } + + /* let's find an even better one */ + if ((depth==2) && (ip0) & (MEM_read32(ip) == MEM_read32(ip - offset_1)))) { + size_t const mlRep = ZSTD_count(ip+4, ip+4-offset_1, iend) + 4; + int const gain2 = (int)(mlRep * 4); + int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 1); + if ((mlRep >= 4) && (gain2 > gain1)) + matchLength = mlRep, offset = 0, start = ip; + } + if (dictMode == ZSTD_dictMatchState) { + const U32 repIndex = (U32)(ip - base) - offset_1; + const BYTE* repMatch = repIndex < prefixLowestIndex ? + dictBase + (repIndex - dictIndexDelta) : + base + repIndex; + if (((U32)((prefixLowestIndex-1) - repIndex) >= 3 /* intentional underflow */) + && (MEM_read32(repMatch) == MEM_read32(ip)) ) { + const BYTE* repMatchEnd = repIndex < prefixLowestIndex ? dictEnd : iend; + size_t const mlRep = ZSTD_count_2segments(ip+4, repMatch+4, iend, repMatchEnd, prefixLowest) + 4; + int const gain2 = (int)(mlRep * 4); + int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 1); + if ((mlRep >= 4) && (gain2 > gain1)) + matchLength = mlRep, offset = 0, start = ip; + } + } + { size_t offset2=999999999; + size_t const ml2 = searchMax(ms, ip, iend, &offset2); + int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)offset2+1)); /* raw approx */ + int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 7); + if ((ml2 >= 4) && (gain2 > gain1)) { + matchLength = ml2, offset = offset2, start = ip; + continue; + } } } + break; /* nothing found : store previous solution */ + } + + /* NOTE: + * start[-offset+ZSTD_REP_MOVE-1] is undefined behavior. + * (-offset+ZSTD_REP_MOVE-1) is unsigned, and is added to start, which + * overflows the pointer, which is undefined behavior. + */ + /* catch up */ + if (offset) { + if (dictMode == ZSTD_noDict) { + while ( ((start > anchor) & (start - (offset-ZSTD_REP_MOVE) > prefixLowest)) + && (start[-1] == (start-(offset-ZSTD_REP_MOVE))[-1]) ) /* only search for offset within prefix */ + { start--; matchLength++; } + } + if (dictMode == ZSTD_dictMatchState) { + U32 const matchIndex = (U32)((start-base) - (offset - ZSTD_REP_MOVE)); + const BYTE* match = (matchIndex < prefixLowestIndex) ? dictBase + matchIndex - dictIndexDelta : base + matchIndex; + const BYTE* const mStart = (matchIndex < prefixLowestIndex) ? dictLowest : prefixLowest; + while ((start>anchor) && (match>mStart) && (start[-1] == match[-1])) { start--; match--; matchLength++; } /* catch up */ + } + offset_2 = offset_1; offset_1 = (U32)(offset - ZSTD_REP_MOVE); + } + /* store sequence */ +_storeSequence: + { size_t const litLength = start - anchor; + ZSTD_storeSeq(seqStore, litLength, anchor, iend, (U32)offset, matchLength-MINMATCH); + anchor = ip = start + matchLength; + } + + /* check immediate repcode */ + if (dictMode == ZSTD_dictMatchState) { + while (ip <= ilimit) { + U32 const current2 = (U32)(ip-base); + U32 const repIndex = current2 - offset_2; + const BYTE* repMatch = dictMode == ZSTD_dictMatchState + && repIndex < prefixLowestIndex ? + dictBase - dictIndexDelta + repIndex : + base + repIndex; + if ( ((U32)((prefixLowestIndex-1) - (U32)repIndex) >= 3 /* intentional overflow */) + && (MEM_read32(repMatch) == MEM_read32(ip)) ) { + const BYTE* const repEnd2 = repIndex < prefixLowestIndex ? dictEnd : iend; + matchLength = ZSTD_count_2segments(ip+4, repMatch+4, iend, repEnd2, prefixLowest) + 4; + offset = offset_2; offset_2 = offset_1; offset_1 = (U32)offset; /* swap offset_2 <=> offset_1 */ + ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, matchLength-MINMATCH); + ip += matchLength; + anchor = ip; + continue; + } + break; + } + } + + if (dictMode == ZSTD_noDict) { + while ( ((ip <= ilimit) & (offset_2>0)) + && (MEM_read32(ip) == MEM_read32(ip - offset_2)) ) { + /* store sequence */ + matchLength = ZSTD_count(ip+4, ip+4-offset_2, iend) + 4; + offset = offset_2; offset_2 = offset_1; offset_1 = (U32)offset; /* swap repcodes */ + ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, matchLength-MINMATCH); + ip += matchLength; + anchor = ip; + continue; /* faster when present ... (?) */ + } } } + + /* Save reps for next block */ + rep[0] = offset_1 ? offset_1 : savedOffset; + rep[1] = offset_2 ? offset_2 : savedOffset; + + /* Return the last literals size */ + return (size_t)(iend - anchor); +} + + +size_t ZSTD_compressBlock_btlazy2( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_binaryTree, 2, ZSTD_noDict); +} + +size_t ZSTD_compressBlock_lazy2( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 2, ZSTD_noDict); +} + +size_t ZSTD_compressBlock_lazy( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 1, ZSTD_noDict); +} + +size_t ZSTD_compressBlock_greedy( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 0, ZSTD_noDict); +} + +size_t ZSTD_compressBlock_btlazy2_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_binaryTree, 2, ZSTD_dictMatchState); +} + +size_t ZSTD_compressBlock_lazy2_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 2, ZSTD_dictMatchState); +} + +size_t ZSTD_compressBlock_lazy_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 1, ZSTD_dictMatchState); +} + +size_t ZSTD_compressBlock_greedy_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 0, ZSTD_dictMatchState); +} + + +FORCE_INLINE_TEMPLATE +size_t ZSTD_compressBlock_lazy_extDict_generic( + ZSTD_matchState_t* ms, seqStore_t* seqStore, + U32 rep[ZSTD_REP_NUM], + const void* src, size_t srcSize, + const searchMethod_e searchMethod, const U32 depth) +{ + const BYTE* const istart = (const BYTE*)src; + const BYTE* ip = istart; + const BYTE* anchor = istart; + const BYTE* const iend = istart + srcSize; + const BYTE* const ilimit = iend - 8; + const BYTE* const base = ms->window.base; + const U32 dictLimit = ms->window.dictLimit; + const BYTE* const prefixStart = base + dictLimit; + const BYTE* const dictBase = ms->window.dictBase; + const BYTE* const dictEnd = dictBase + dictLimit; + const BYTE* const dictStart = dictBase + ms->window.lowLimit; + const U32 windowLog = ms->cParams.windowLog; + + typedef size_t (*searchMax_f)( + ZSTD_matchState_t* ms, + const BYTE* ip, const BYTE* iLimit, size_t* offsetPtr); + searchMax_f searchMax = searchMethod==search_binaryTree ? ZSTD_BtFindBestMatch_extDict_selectMLS : ZSTD_HcFindBestMatch_extDict_selectMLS; + + U32 offset_1 = rep[0], offset_2 = rep[1]; + + DEBUGLOG(5, "ZSTD_compressBlock_lazy_extDict_generic"); + + /* init */ + ip += (ip == prefixStart); + + /* Match Loop */ +#if defined(__GNUC__) && defined(__x86_64__) + /* I've measured random a 5% speed loss on levels 5 & 6 (greedy) when the + * code alignment is perturbed. To fix the instability align the loop on 32-bytes. + */ + __asm__(".p2align 5"); +#endif + while (ip < ilimit) { + size_t matchLength=0; + size_t offset=0; + const BYTE* start=ip+1; + U32 current = (U32)(ip-base); + + /* check repCode */ + { const U32 windowLow = ZSTD_getLowestMatchIndex(ms, current+1, windowLog); + const U32 repIndex = (U32)(current+1 - offset_1); + const BYTE* const repBase = repIndex < dictLimit ? dictBase : base; + const BYTE* const repMatch = repBase + repIndex; + if (((U32)((dictLimit-1) - repIndex) >= 3) & (repIndex > windowLow)) /* intentional overflow */ + if (MEM_read32(ip+1) == MEM_read32(repMatch)) { + /* repcode detected we should take it */ + const BYTE* const repEnd = repIndex < dictLimit ? dictEnd : iend; + matchLength = ZSTD_count_2segments(ip+1+4, repMatch+4, iend, repEnd, prefixStart) + 4; + if (depth==0) goto _storeSequence; + } } + + /* first search (depth 0) */ + { size_t offsetFound = 999999999; + size_t const ml2 = searchMax(ms, ip, iend, &offsetFound); + if (ml2 > matchLength) + matchLength = ml2, start = ip, offset=offsetFound; + } + + if (matchLength < 4) { + ip += ((ip-anchor) >> kSearchStrength) + 1; /* jump faster over incompressible sections */ + continue; + } + + /* let's try to find a better solution */ + if (depth>=1) + while (ip= 3) & (repIndex > windowLow)) /* intentional overflow */ + if (MEM_read32(ip) == MEM_read32(repMatch)) { + /* repcode detected */ + const BYTE* const repEnd = repIndex < dictLimit ? dictEnd : iend; + size_t const repLength = ZSTD_count_2segments(ip+4, repMatch+4, iend, repEnd, prefixStart) + 4; + int const gain2 = (int)(repLength * 3); + int const gain1 = (int)(matchLength*3 - ZSTD_highbit32((U32)offset+1) + 1); + if ((repLength >= 4) && (gain2 > gain1)) + matchLength = repLength, offset = 0, start = ip; + } } + + /* search match, depth 1 */ + { size_t offset2=999999999; + size_t const ml2 = searchMax(ms, ip, iend, &offset2); + int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)offset2+1)); /* raw approx */ + int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 4); + if ((ml2 >= 4) && (gain2 > gain1)) { + matchLength = ml2, offset = offset2, start = ip; + continue; /* search a better one */ + } } + + /* let's find an even better one */ + if ((depth==2) && (ip= 3) & (repIndex > windowLow)) /* intentional overflow */ + if (MEM_read32(ip) == MEM_read32(repMatch)) { + /* repcode detected */ + const BYTE* const repEnd = repIndex < dictLimit ? dictEnd : iend; + size_t const repLength = ZSTD_count_2segments(ip+4, repMatch+4, iend, repEnd, prefixStart) + 4; + int const gain2 = (int)(repLength * 4); + int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 1); + if ((repLength >= 4) && (gain2 > gain1)) + matchLength = repLength, offset = 0, start = ip; + } } + + /* search match, depth 2 */ + { size_t offset2=999999999; + size_t const ml2 = searchMax(ms, ip, iend, &offset2); + int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)offset2+1)); /* raw approx */ + int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 7); + if ((ml2 >= 4) && (gain2 > gain1)) { + matchLength = ml2, offset = offset2, start = ip; + continue; + } } } + break; /* nothing found : store previous solution */ + } + + /* catch up */ + if (offset) { + U32 const matchIndex = (U32)((start-base) - (offset - ZSTD_REP_MOVE)); + const BYTE* match = (matchIndex < dictLimit) ? dictBase + matchIndex : base + matchIndex; + const BYTE* const mStart = (matchIndex < dictLimit) ? dictStart : prefixStart; + while ((start>anchor) && (match>mStart) && (start[-1] == match[-1])) { start--; match--; matchLength++; } /* catch up */ + offset_2 = offset_1; offset_1 = (U32)(offset - ZSTD_REP_MOVE); + } + + /* store sequence */ +_storeSequence: + { size_t const litLength = start - anchor; + ZSTD_storeSeq(seqStore, litLength, anchor, iend, (U32)offset, matchLength-MINMATCH); + anchor = ip = start + matchLength; + } + + /* check immediate repcode */ + while (ip <= ilimit) { + const U32 repCurrent = (U32)(ip-base); + const U32 windowLow = ZSTD_getLowestMatchIndex(ms, repCurrent, windowLog); + const U32 repIndex = repCurrent - offset_2; + const BYTE* const repBase = repIndex < dictLimit ? dictBase : base; + const BYTE* const repMatch = repBase + repIndex; + if (((U32)((dictLimit-1) - repIndex) >= 3) & (repIndex > windowLow)) /* intentional overflow */ + if (MEM_read32(ip) == MEM_read32(repMatch)) { + /* repcode detected we should take it */ + const BYTE* const repEnd = repIndex < dictLimit ? dictEnd : iend; + matchLength = ZSTD_count_2segments(ip+4, repMatch+4, iend, repEnd, prefixStart) + 4; + offset = offset_2; offset_2 = offset_1; offset_1 = (U32)offset; /* swap offset history */ + ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, matchLength-MINMATCH); + ip += matchLength; + anchor = ip; + continue; /* faster when present ... (?) */ + } + break; + } } + + /* Save reps for next block */ + rep[0] = offset_1; + rep[1] = offset_2; + + /* Return the last literals size */ + return (size_t)(iend - anchor); +} + + +size_t ZSTD_compressBlock_greedy_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + return ZSTD_compressBlock_lazy_extDict_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 0); +} + +size_t ZSTD_compressBlock_lazy_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) + +{ + return ZSTD_compressBlock_lazy_extDict_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 1); +} + +size_t ZSTD_compressBlock_lazy2_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) + +{ + return ZSTD_compressBlock_lazy_extDict_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 2); +} + +size_t ZSTD_compressBlock_btlazy2_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) + +{ + return ZSTD_compressBlock_lazy_extDict_generic(ms, seqStore, rep, src, srcSize, search_binaryTree, 2); +} diff --git a/module/zstd/lib/compress/zstd_lazy.h b/module/zstd/lib/compress/zstd_lazy.h new file mode 100644 index 000000000000..581936f03bd4 --- /dev/null +++ b/module/zstd/lib/compress/zstd_lazy.h @@ -0,0 +1,67 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#ifndef ZSTD_LAZY_H +#define ZSTD_LAZY_H + +#if defined (__cplusplus) +extern "C" { +#endif + +#include "zstd_compress_internal.h" + +U32 ZSTD_insertAndFindFirstIndex(ZSTD_matchState_t* ms, const BYTE* ip); + +void ZSTD_preserveUnsortedMark (U32* const table, U32 const size, U32 const reducerValue); /*! used in ZSTD_reduceIndex(). preemptively increase value of ZSTD_DUBT_UNSORTED_MARK */ + +size_t ZSTD_compressBlock_btlazy2( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_lazy2( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_lazy( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_greedy( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); + +size_t ZSTD_compressBlock_btlazy2_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_lazy2_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_lazy_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_greedy_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); + +size_t ZSTD_compressBlock_greedy_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_lazy_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_lazy2_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_btlazy2_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); + +#if defined (__cplusplus) +} +#endif + +#endif /* ZSTD_LAZY_H */ diff --git a/module/zstd/lib/compress/zstd_ldm.c b/module/zstd/lib/compress/zstd_ldm.c new file mode 100644 index 000000000000..8c479483581c --- /dev/null +++ b/module/zstd/lib/compress/zstd_ldm.c @@ -0,0 +1,619 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#include "zstd_ldm.h" + +#include "../common/debug.h" +#include "zstd_fast.h" /* ZSTD_fillHashTable() */ +#include "zstd_double_fast.h" /* ZSTD_fillDoubleHashTable() */ + +#define LDM_BUCKET_SIZE_LOG 3 +#define LDM_MIN_MATCH_LENGTH 64 +#define LDM_HASH_RLOG 7 +#define LDM_HASH_CHAR_OFFSET 10 + +void ZSTD_ldm_adjustParameters(ldmParams_t* params, + ZSTD_compressionParameters const* cParams) +{ + params->windowLog = cParams->windowLog; + ZSTD_STATIC_ASSERT(LDM_BUCKET_SIZE_LOG <= ZSTD_LDM_BUCKETSIZELOG_MAX); + DEBUGLOG(4, "ZSTD_ldm_adjustParameters"); + if (!params->bucketSizeLog) params->bucketSizeLog = LDM_BUCKET_SIZE_LOG; + if (!params->minMatchLength) params->minMatchLength = LDM_MIN_MATCH_LENGTH; + if (cParams->strategy >= ZSTD_btopt) { + /* Get out of the way of the optimal parser */ + U32 const minMatch = MAX(cParams->targetLength, params->minMatchLength); + assert(minMatch >= ZSTD_LDM_MINMATCH_MIN); + assert(minMatch <= ZSTD_LDM_MINMATCH_MAX); + params->minMatchLength = minMatch; + } + if (params->hashLog == 0) { + params->hashLog = MAX(ZSTD_HASHLOG_MIN, params->windowLog - LDM_HASH_RLOG); + assert(params->hashLog <= ZSTD_HASHLOG_MAX); + } + if (params->hashRateLog == 0) { + params->hashRateLog = params->windowLog < params->hashLog + ? 0 + : params->windowLog - params->hashLog; + } + params->bucketSizeLog = MIN(params->bucketSizeLog, params->hashLog); +} + +size_t ZSTD_ldm_getTableSize(ldmParams_t params) +{ + size_t const ldmHSize = ((size_t)1) << params.hashLog; + size_t const ldmBucketSizeLog = MIN(params.bucketSizeLog, params.hashLog); + size_t const ldmBucketSize = ((size_t)1) << (params.hashLog - ldmBucketSizeLog); + size_t const totalSize = ZSTD_cwksp_alloc_size(ldmBucketSize) + + ZSTD_cwksp_alloc_size(ldmHSize * sizeof(ldmEntry_t)); + return params.enableLdm ? totalSize : 0; +} + +size_t ZSTD_ldm_getMaxNbSeq(ldmParams_t params, size_t maxChunkSize) +{ + return params.enableLdm ? (maxChunkSize / params.minMatchLength) : 0; +} + +/** ZSTD_ldm_getSmallHash() : + * numBits should be <= 32 + * If numBits==0, returns 0. + * @return : the most significant numBits of value. */ +static U32 ZSTD_ldm_getSmallHash(U64 value, U32 numBits) +{ + assert(numBits <= 32); + return numBits == 0 ? 0 : (U32)(value >> (64 - numBits)); +} + +/** ZSTD_ldm_getChecksum() : + * numBitsToDiscard should be <= 32 + * @return : the next most significant 32 bits after numBitsToDiscard */ +static U32 ZSTD_ldm_getChecksum(U64 hash, U32 numBitsToDiscard) +{ + assert(numBitsToDiscard <= 32); + return (hash >> (64 - 32 - numBitsToDiscard)) & 0xFFFFFFFF; +} + +/** ZSTD_ldm_getTag() ; + * Given the hash, returns the most significant numTagBits bits + * after (32 + hbits) bits. + * + * If there are not enough bits remaining, return the last + * numTagBits bits. */ +static U32 ZSTD_ldm_getTag(U64 hash, U32 hbits, U32 numTagBits) +{ + assert(numTagBits < 32 && hbits <= 32); + if (32 - hbits < numTagBits) { + return hash & (((U32)1 << numTagBits) - 1); + } else { + return (hash >> (32 - hbits - numTagBits)) & (((U32)1 << numTagBits) - 1); + } +} + +/** ZSTD_ldm_getBucket() : + * Returns a pointer to the start of the bucket associated with hash. */ +static ldmEntry_t* ZSTD_ldm_getBucket( + ldmState_t* ldmState, size_t hash, ldmParams_t const ldmParams) +{ + return ldmState->hashTable + (hash << ldmParams.bucketSizeLog); +} + +/** ZSTD_ldm_insertEntry() : + * Insert the entry with corresponding hash into the hash table */ +static void ZSTD_ldm_insertEntry(ldmState_t* ldmState, + size_t const hash, const ldmEntry_t entry, + ldmParams_t const ldmParams) +{ + BYTE* const bucketOffsets = ldmState->bucketOffsets; + *(ZSTD_ldm_getBucket(ldmState, hash, ldmParams) + bucketOffsets[hash]) = entry; + bucketOffsets[hash]++; + bucketOffsets[hash] &= ((U32)1 << ldmParams.bucketSizeLog) - 1; +} + +/** ZSTD_ldm_makeEntryAndInsertByTag() : + * + * Gets the small hash, checksum, and tag from the rollingHash. + * + * If the tag matches (1 << ldmParams.hashRateLog)-1, then + * creates an ldmEntry from the offset, and inserts it into the hash table. + * + * hBits is the length of the small hash, which is the most significant hBits + * of rollingHash. The checksum is the next 32 most significant bits, followed + * by ldmParams.hashRateLog bits that make up the tag. */ +static void ZSTD_ldm_makeEntryAndInsertByTag(ldmState_t* ldmState, + U64 const rollingHash, + U32 const hBits, + U32 const offset, + ldmParams_t const ldmParams) +{ + U32 const tag = ZSTD_ldm_getTag(rollingHash, hBits, ldmParams.hashRateLog); + U32 const tagMask = ((U32)1 << ldmParams.hashRateLog) - 1; + if (tag == tagMask) { + U32 const hash = ZSTD_ldm_getSmallHash(rollingHash, hBits); + U32 const checksum = ZSTD_ldm_getChecksum(rollingHash, hBits); + ldmEntry_t entry; + entry.offset = offset; + entry.checksum = checksum; + ZSTD_ldm_insertEntry(ldmState, hash, entry, ldmParams); + } +} + +/** ZSTD_ldm_countBackwardsMatch() : + * Returns the number of bytes that match backwards before pIn and pMatch. + * + * We count only bytes where pMatch >= pBase and pIn >= pAnchor. */ +static size_t ZSTD_ldm_countBackwardsMatch( + const BYTE* pIn, const BYTE* pAnchor, + const BYTE* pMatch, const BYTE* pBase) +{ + size_t matchLength = 0; + while (pIn > pAnchor && pMatch > pBase && pIn[-1] == pMatch[-1]) { + pIn--; + pMatch--; + matchLength++; + } + return matchLength; +} + +/** ZSTD_ldm_fillFastTables() : + * + * Fills the relevant tables for the ZSTD_fast and ZSTD_dfast strategies. + * This is similar to ZSTD_loadDictionaryContent. + * + * The tables for the other strategies are filled within their + * block compressors. */ +static size_t ZSTD_ldm_fillFastTables(ZSTD_matchState_t* ms, + void const* end) +{ + const BYTE* const iend = (const BYTE*)end; + + switch(ms->cParams.strategy) + { + case ZSTD_fast: + ZSTD_fillHashTable(ms, iend, ZSTD_dtlm_fast); + break; + + case ZSTD_dfast: + ZSTD_fillDoubleHashTable(ms, iend, ZSTD_dtlm_fast); + break; + + case ZSTD_greedy: + case ZSTD_lazy: + case ZSTD_lazy2: + case ZSTD_btlazy2: + case ZSTD_btopt: + case ZSTD_btultra: + case ZSTD_btultra2: + break; + default: + assert(0); /* not possible : not a valid strategy id */ + } + + return 0; +} + +/** ZSTD_ldm_fillLdmHashTable() : + * + * Fills hashTable from (lastHashed + 1) to iend (non-inclusive). + * lastHash is the rolling hash that corresponds to lastHashed. + * + * Returns the rolling hash corresponding to position iend-1. */ +static U64 ZSTD_ldm_fillLdmHashTable(ldmState_t* state, + U64 lastHash, const BYTE* lastHashed, + const BYTE* iend, const BYTE* base, + U32 hBits, ldmParams_t const ldmParams) +{ + U64 rollingHash = lastHash; + const BYTE* cur = lastHashed + 1; + + while (cur < iend) { + rollingHash = ZSTD_rollingHash_rotate(rollingHash, cur[-1], + cur[ldmParams.minMatchLength-1], + state->hashPower); + ZSTD_ldm_makeEntryAndInsertByTag(state, + rollingHash, hBits, + (U32)(cur - base), ldmParams); + ++cur; + } + return rollingHash; +} + +void ZSTD_ldm_fillHashTable( + ldmState_t* state, const BYTE* ip, + const BYTE* iend, ldmParams_t const* params) +{ + DEBUGLOG(5, "ZSTD_ldm_fillHashTable"); + if ((size_t)(iend - ip) >= params->minMatchLength) { + U64 startingHash = ZSTD_rollingHash_compute(ip, params->minMatchLength); + ZSTD_ldm_fillLdmHashTable( + state, startingHash, ip, iend - params->minMatchLength, state->window.base, + params->hashLog - params->bucketSizeLog, + *params); + } +} + + +/** ZSTD_ldm_limitTableUpdate() : + * + * Sets cctx->nextToUpdate to a position corresponding closer to anchor + * if it is far way + * (after a long match, only update tables a limited amount). */ +static void ZSTD_ldm_limitTableUpdate(ZSTD_matchState_t* ms, const BYTE* anchor) +{ + U32 const current = (U32)(anchor - ms->window.base); + if (current > ms->nextToUpdate + 1024) { + ms->nextToUpdate = + current - MIN(512, current - ms->nextToUpdate - 1024); + } +} + +static size_t ZSTD_ldm_generateSequences_internal( + ldmState_t* ldmState, rawSeqStore_t* rawSeqStore, + ldmParams_t const* params, void const* src, size_t srcSize) +{ + /* LDM parameters */ + int const extDict = ZSTD_window_hasExtDict(ldmState->window); + U32 const minMatchLength = params->minMatchLength; + U64 const hashPower = ldmState->hashPower; + U32 const hBits = params->hashLog - params->bucketSizeLog; + U32 const ldmBucketSize = 1U << params->bucketSizeLog; + U32 const hashRateLog = params->hashRateLog; + U32 const ldmTagMask = (1U << params->hashRateLog) - 1; + /* Prefix and extDict parameters */ + U32 const dictLimit = ldmState->window.dictLimit; + U32 const lowestIndex = extDict ? ldmState->window.lowLimit : dictLimit; + BYTE const* const base = ldmState->window.base; + BYTE const* const dictBase = extDict ? ldmState->window.dictBase : NULL; + BYTE const* const dictStart = extDict ? dictBase + lowestIndex : NULL; + BYTE const* const dictEnd = extDict ? dictBase + dictLimit : NULL; + BYTE const* const lowPrefixPtr = base + dictLimit; + /* Input bounds */ + BYTE const* const istart = (BYTE const*)src; + BYTE const* const iend = istart + srcSize; + BYTE const* const ilimit = iend - MAX(minMatchLength, HASH_READ_SIZE); + /* Input positions */ + BYTE const* anchor = istart; + BYTE const* ip = istart; + /* Rolling hash */ + BYTE const* lastHashed = NULL; + U64 rollingHash = 0; + + while (ip <= ilimit) { + size_t mLength; + U32 const current = (U32)(ip - base); + size_t forwardMatchLength = 0, backwardMatchLength = 0; + ldmEntry_t* bestEntry = NULL; + if (ip != istart) { + rollingHash = ZSTD_rollingHash_rotate(rollingHash, lastHashed[0], + lastHashed[minMatchLength], + hashPower); + } else { + rollingHash = ZSTD_rollingHash_compute(ip, minMatchLength); + } + lastHashed = ip; + + /* Do not insert and do not look for a match */ + if (ZSTD_ldm_getTag(rollingHash, hBits, hashRateLog) != ldmTagMask) { + ip++; + continue; + } + + /* Get the best entry and compute the match lengths */ + { + ldmEntry_t* const bucket = + ZSTD_ldm_getBucket(ldmState, + ZSTD_ldm_getSmallHash(rollingHash, hBits), + *params); + ldmEntry_t* cur; + size_t bestMatchLength = 0; + U32 const checksum = ZSTD_ldm_getChecksum(rollingHash, hBits); + + for (cur = bucket; cur < bucket + ldmBucketSize; ++cur) { + size_t curForwardMatchLength, curBackwardMatchLength, + curTotalMatchLength; + if (cur->checksum != checksum || cur->offset <= lowestIndex) { + continue; + } + if (extDict) { + BYTE const* const curMatchBase = + cur->offset < dictLimit ? dictBase : base; + BYTE const* const pMatch = curMatchBase + cur->offset; + BYTE const* const matchEnd = + cur->offset < dictLimit ? dictEnd : iend; + BYTE const* const lowMatchPtr = + cur->offset < dictLimit ? dictStart : lowPrefixPtr; + + curForwardMatchLength = ZSTD_count_2segments( + ip, pMatch, iend, + matchEnd, lowPrefixPtr); + if (curForwardMatchLength < minMatchLength) { + continue; + } + curBackwardMatchLength = + ZSTD_ldm_countBackwardsMatch(ip, anchor, pMatch, + lowMatchPtr); + curTotalMatchLength = curForwardMatchLength + + curBackwardMatchLength; + } else { /* !extDict */ + BYTE const* const pMatch = base + cur->offset; + curForwardMatchLength = ZSTD_count(ip, pMatch, iend); + if (curForwardMatchLength < minMatchLength) { + continue; + } + curBackwardMatchLength = + ZSTD_ldm_countBackwardsMatch(ip, anchor, pMatch, + lowPrefixPtr); + curTotalMatchLength = curForwardMatchLength + + curBackwardMatchLength; + } + + if (curTotalMatchLength > bestMatchLength) { + bestMatchLength = curTotalMatchLength; + forwardMatchLength = curForwardMatchLength; + backwardMatchLength = curBackwardMatchLength; + bestEntry = cur; + } + } + } + + /* No match found -- continue searching */ + if (bestEntry == NULL) { + ZSTD_ldm_makeEntryAndInsertByTag(ldmState, rollingHash, + hBits, current, + *params); + ip++; + continue; + } + + /* Match found */ + mLength = forwardMatchLength + backwardMatchLength; + ip -= backwardMatchLength; + + { + /* Store the sequence: + * ip = current - backwardMatchLength + * The match is at (bestEntry->offset - backwardMatchLength) + */ + U32 const matchIndex = bestEntry->offset; + U32 const offset = current - matchIndex; + rawSeq* const seq = rawSeqStore->seq + rawSeqStore->size; + + /* Out of sequence storage */ + if (rawSeqStore->size == rawSeqStore->capacity) + return ERROR(dstSize_tooSmall); + seq->litLength = (U32)(ip - anchor); + seq->matchLength = (U32)mLength; + seq->offset = offset; + rawSeqStore->size++; + } + + /* Insert the current entry into the hash table */ + ZSTD_ldm_makeEntryAndInsertByTag(ldmState, rollingHash, hBits, + (U32)(lastHashed - base), + *params); + + assert(ip + backwardMatchLength == lastHashed); + + /* Fill the hash table from lastHashed+1 to ip+mLength*/ + /* Heuristic: don't need to fill the entire table at end of block */ + if (ip + mLength <= ilimit) { + rollingHash = ZSTD_ldm_fillLdmHashTable( + ldmState, rollingHash, lastHashed, + ip + mLength, base, hBits, *params); + lastHashed = ip + mLength - 1; + } + ip += mLength; + anchor = ip; + } + return iend - anchor; +} + +/*! ZSTD_ldm_reduceTable() : + * reduce table indexes by `reducerValue` */ +static void ZSTD_ldm_reduceTable(ldmEntry_t* const table, U32 const size, + U32 const reducerValue) +{ + U32 u; + for (u = 0; u < size; u++) { + if (table[u].offset < reducerValue) table[u].offset = 0; + else table[u].offset -= reducerValue; + } +} + +size_t ZSTD_ldm_generateSequences( + ldmState_t* ldmState, rawSeqStore_t* sequences, + ldmParams_t const* params, void const* src, size_t srcSize) +{ + U32 const maxDist = 1U << params->windowLog; + BYTE const* const istart = (BYTE const*)src; + BYTE const* const iend = istart + srcSize; + size_t const kMaxChunkSize = 1 << 20; + size_t const nbChunks = (srcSize / kMaxChunkSize) + ((srcSize % kMaxChunkSize) != 0); + size_t chunk; + size_t leftoverSize = 0; + + assert(ZSTD_CHUNKSIZE_MAX >= kMaxChunkSize); + /* Check that ZSTD_window_update() has been called for this chunk prior + * to passing it to this function. + */ + assert(ldmState->window.nextSrc >= (BYTE const*)src + srcSize); + /* The input could be very large (in zstdmt), so it must be broken up into + * chunks to enforce the maximum distance and handle overflow correction. + */ + assert(sequences->pos <= sequences->size); + assert(sequences->size <= sequences->capacity); + for (chunk = 0; chunk < nbChunks && sequences->size < sequences->capacity; ++chunk) { + BYTE const* const chunkStart = istart + chunk * kMaxChunkSize; + size_t const remaining = (size_t)(iend - chunkStart); + BYTE const *const chunkEnd = + (remaining < kMaxChunkSize) ? iend : chunkStart + kMaxChunkSize; + size_t const chunkSize = chunkEnd - chunkStart; + size_t newLeftoverSize; + size_t const prevSize = sequences->size; + + assert(chunkStart < iend); + /* 1. Perform overflow correction if necessary. */ + if (ZSTD_window_needOverflowCorrection(ldmState->window, chunkEnd)) { + U32 const ldmHSize = 1U << params->hashLog; + U32 const correction = ZSTD_window_correctOverflow( + &ldmState->window, /* cycleLog */ 0, maxDist, chunkStart); + ZSTD_ldm_reduceTable(ldmState->hashTable, ldmHSize, correction); + /* invalidate dictionaries on overflow correction */ + ldmState->loadedDictEnd = 0; + } + /* 2. We enforce the maximum offset allowed. + * + * kMaxChunkSize should be small enough that we don't lose too much of + * the window through early invalidation. + * TODO: * Test the chunk size. + * * Try invalidation after the sequence generation and test the + * the offset against maxDist directly. + * + * NOTE: Because of dictionaries + sequence splitting we MUST make sure + * that any offset used is valid at the END of the sequence, since it may + * be split into two sequences. This condition holds when using + * ZSTD_window_enforceMaxDist(), but if we move to checking offsets + * against maxDist directly, we'll have to carefully handle that case. + */ + ZSTD_window_enforceMaxDist(&ldmState->window, chunkEnd, maxDist, &ldmState->loadedDictEnd, NULL); + /* 3. Generate the sequences for the chunk, and get newLeftoverSize. */ + newLeftoverSize = ZSTD_ldm_generateSequences_internal( + ldmState, sequences, params, chunkStart, chunkSize); + if (ZSTD_isError(newLeftoverSize)) + return newLeftoverSize; + /* 4. We add the leftover literals from previous iterations to the first + * newly generated sequence, or add the `newLeftoverSize` if none are + * generated. + */ + /* Prepend the leftover literals from the last call */ + if (prevSize < sequences->size) { + sequences->seq[prevSize].litLength += (U32)leftoverSize; + leftoverSize = newLeftoverSize; + } else { + assert(newLeftoverSize == chunkSize); + leftoverSize += chunkSize; + } + } + return 0; +} + +void ZSTD_ldm_skipSequences(rawSeqStore_t* rawSeqStore, size_t srcSize, U32 const minMatch) { + while (srcSize > 0 && rawSeqStore->pos < rawSeqStore->size) { + rawSeq* seq = rawSeqStore->seq + rawSeqStore->pos; + if (srcSize <= seq->litLength) { + /* Skip past srcSize literals */ + seq->litLength -= (U32)srcSize; + return; + } + srcSize -= seq->litLength; + seq->litLength = 0; + if (srcSize < seq->matchLength) { + /* Skip past the first srcSize of the match */ + seq->matchLength -= (U32)srcSize; + if (seq->matchLength < minMatch) { + /* The match is too short, omit it */ + if (rawSeqStore->pos + 1 < rawSeqStore->size) { + seq[1].litLength += seq[0].matchLength; + } + rawSeqStore->pos++; + } + return; + } + srcSize -= seq->matchLength; + seq->matchLength = 0; + rawSeqStore->pos++; + } +} + +/** + * If the sequence length is longer than remaining then the sequence is split + * between this block and the next. + * + * Returns the current sequence to handle, or if the rest of the block should + * be literals, it returns a sequence with offset == 0. + */ +static rawSeq maybeSplitSequence(rawSeqStore_t* rawSeqStore, + U32 const remaining, U32 const minMatch) +{ + rawSeq sequence = rawSeqStore->seq[rawSeqStore->pos]; + assert(sequence.offset > 0); + /* Likely: No partial sequence */ + if (remaining >= sequence.litLength + sequence.matchLength) { + rawSeqStore->pos++; + return sequence; + } + /* Cut the sequence short (offset == 0 ==> rest is literals). */ + if (remaining <= sequence.litLength) { + sequence.offset = 0; + } else if (remaining < sequence.litLength + sequence.matchLength) { + sequence.matchLength = remaining - sequence.litLength; + if (sequence.matchLength < minMatch) { + sequence.offset = 0; + } + } + /* Skip past `remaining` bytes for the future sequences. */ + ZSTD_ldm_skipSequences(rawSeqStore, remaining, minMatch); + return sequence; +} + +size_t ZSTD_ldm_blockCompress(rawSeqStore_t* rawSeqStore, + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize) +{ + const ZSTD_compressionParameters* const cParams = &ms->cParams; + unsigned const minMatch = cParams->minMatch; + ZSTD_blockCompressor const blockCompressor = + ZSTD_selectBlockCompressor(cParams->strategy, ZSTD_matchState_dictMode(ms)); + /* Input bounds */ + BYTE const* const istart = (BYTE const*)src; + BYTE const* const iend = istart + srcSize; + /* Input positions */ + BYTE const* ip = istart; + + DEBUGLOG(5, "ZSTD_ldm_blockCompress: srcSize=%zu", srcSize); + assert(rawSeqStore->pos <= rawSeqStore->size); + assert(rawSeqStore->size <= rawSeqStore->capacity); + /* Loop through each sequence and apply the block compressor to the lits */ + while (rawSeqStore->pos < rawSeqStore->size && ip < iend) { + /* maybeSplitSequence updates rawSeqStore->pos */ + rawSeq const sequence = maybeSplitSequence(rawSeqStore, + (U32)(iend - ip), minMatch); + int i; + /* End signal */ + if (sequence.offset == 0) + break; + + assert(ip + sequence.litLength + sequence.matchLength <= iend); + + /* Fill tables for block compressor */ + ZSTD_ldm_limitTableUpdate(ms, ip); + ZSTD_ldm_fillFastTables(ms, ip); + /* Run the block compressor */ + DEBUGLOG(5, "pos %u : calling block compressor on segment of size %u", (unsigned)(ip-istart), sequence.litLength); + { + size_t const newLitLength = + blockCompressor(ms, seqStore, rep, ip, sequence.litLength); + ip += sequence.litLength; + /* Update the repcodes */ + for (i = ZSTD_REP_NUM - 1; i > 0; i--) + rep[i] = rep[i-1]; + rep[0] = sequence.offset; + /* Store the sequence */ + ZSTD_storeSeq(seqStore, newLitLength, ip - newLitLength, iend, + sequence.offset + ZSTD_REP_MOVE, + sequence.matchLength - MINMATCH); + ip += sequence.matchLength; + } + } + /* Fill the tables for the block compressor */ + ZSTD_ldm_limitTableUpdate(ms, ip); + ZSTD_ldm_fillFastTables(ms, ip); + /* Compress the last literals */ + return blockCompressor(ms, seqStore, rep, ip, iend - ip); +} diff --git a/module/zstd/lib/compress/zstd_ldm.h b/module/zstd/lib/compress/zstd_ldm.h new file mode 100644 index 000000000000..229ea05a9e1e --- /dev/null +++ b/module/zstd/lib/compress/zstd_ldm.h @@ -0,0 +1,110 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#ifndef ZSTD_LDM_H +#define ZSTD_LDM_H + +#if defined (__cplusplus) +extern "C" { +#endif + +#include "zstd_compress_internal.h" /* ldmParams_t, U32 */ +#include "../zstd.h" /* ZSTD_CCtx, size_t */ + +/*-************************************* +* Long distance matching +***************************************/ + +#define ZSTD_LDM_DEFAULT_WINDOW_LOG ZSTD_WINDOWLOG_LIMIT_DEFAULT + +void ZSTD_ldm_fillHashTable( + ldmState_t* state, const BYTE* ip, + const BYTE* iend, ldmParams_t const* params); + +/** + * ZSTD_ldm_generateSequences(): + * + * Generates the sequences using the long distance match finder. + * Generates long range matching sequences in `sequences`, which parse a prefix + * of the source. `sequences` must be large enough to store every sequence, + * which can be checked with `ZSTD_ldm_getMaxNbSeq()`. + * @returns 0 or an error code. + * + * NOTE: The user must have called ZSTD_window_update() for all of the input + * they have, even if they pass it to ZSTD_ldm_generateSequences() in chunks. + * NOTE: This function returns an error if it runs out of space to store + * sequences. + */ +size_t ZSTD_ldm_generateSequences( + ldmState_t* ldms, rawSeqStore_t* sequences, + ldmParams_t const* params, void const* src, size_t srcSize); + +/** + * ZSTD_ldm_blockCompress(): + * + * Compresses a block using the predefined sequences, along with a secondary + * block compressor. The literals section of every sequence is passed to the + * secondary block compressor, and those sequences are interspersed with the + * predefined sequences. Returns the length of the last literals. + * Updates `rawSeqStore.pos` to indicate how many sequences have been consumed. + * `rawSeqStore.seq` may also be updated to split the last sequence between two + * blocks. + * @return The length of the last literals. + * + * NOTE: The source must be at most the maximum block size, but the predefined + * sequences can be any size, and may be longer than the block. In the case that + * they are longer than the block, the last sequences may need to be split into + * two. We handle that case correctly, and update `rawSeqStore` appropriately. + * NOTE: This function does not return any errors. + */ +size_t ZSTD_ldm_blockCompress(rawSeqStore_t* rawSeqStore, + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); + +/** + * ZSTD_ldm_skipSequences(): + * + * Skip past `srcSize` bytes worth of sequences in `rawSeqStore`. + * Avoids emitting matches less than `minMatch` bytes. + * Must be called for data with is not passed to ZSTD_ldm_blockCompress(). + */ +void ZSTD_ldm_skipSequences(rawSeqStore_t* rawSeqStore, size_t srcSize, + U32 const minMatch); + + +/** ZSTD_ldm_getTableSize() : + * Estimate the space needed for long distance matching tables or 0 if LDM is + * disabled. + */ +size_t ZSTD_ldm_getTableSize(ldmParams_t params); + +/** ZSTD_ldm_getSeqSpace() : + * Return an upper bound on the number of sequences that can be produced by + * the long distance matcher, or 0 if LDM is disabled. + */ +size_t ZSTD_ldm_getMaxNbSeq(ldmParams_t params, size_t maxChunkSize); + +/** ZSTD_ldm_adjustParameters() : + * If the params->hashRateLog is not set, set it to its default value based on + * windowLog and params->hashLog. + * + * Ensures that params->bucketSizeLog is <= params->hashLog (setting it to + * params->hashLog if it is not). + * + * Ensures that the minMatchLength >= targetLength during optimal parsing. + */ +void ZSTD_ldm_adjustParameters(ldmParams_t* params, + ZSTD_compressionParameters const* cParams); + +#if defined (__cplusplus) +} +#endif + +#endif /* ZSTD_FAST_H */ diff --git a/module/zstd/lib/compress/zstd_opt.c b/module/zstd/lib/compress/zstd_opt.c new file mode 100644 index 000000000000..36fff050cf5a --- /dev/null +++ b/module/zstd/lib/compress/zstd_opt.c @@ -0,0 +1,1200 @@ +/* + * Copyright (c) 2016-2020, Przemyslaw Skibinski, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#include "zstd_compress_internal.h" +#include "hist.h" +#include "zstd_opt.h" + + +#define ZSTD_LITFREQ_ADD 2 /* scaling factor for litFreq, so that frequencies adapt faster to new stats */ +#define ZSTD_FREQ_DIV 4 /* log factor when using previous stats to init next stats */ +#define ZSTD_MAX_PRICE (1<<30) + +#define ZSTD_PREDEF_THRESHOLD 1024 /* if srcSize < ZSTD_PREDEF_THRESHOLD, symbols' cost is assumed static, directly determined by pre-defined distributions */ + + +/*-************************************* +* Price functions for optimal parser +***************************************/ + +#if 0 /* approximation at bit level */ +# define BITCOST_ACCURACY 0 +# define BITCOST_MULTIPLIER (1 << BITCOST_ACCURACY) +# define WEIGHT(stat) ((void)opt, ZSTD_bitWeight(stat)) +#elif 0 /* fractional bit accuracy */ +# define BITCOST_ACCURACY 8 +# define BITCOST_MULTIPLIER (1 << BITCOST_ACCURACY) +# define WEIGHT(stat,opt) ((void)opt, ZSTD_fracWeight(stat)) +#else /* opt==approx, ultra==accurate */ +# define BITCOST_ACCURACY 8 +# define BITCOST_MULTIPLIER (1 << BITCOST_ACCURACY) +# define WEIGHT(stat,opt) (opt ? ZSTD_fracWeight(stat) : ZSTD_bitWeight(stat)) +#endif + +MEM_STATIC U32 ZSTD_bitWeight(U32 stat) +{ + return (ZSTD_highbit32(stat+1) * BITCOST_MULTIPLIER); +} + +MEM_STATIC U32 ZSTD_fracWeight(U32 rawStat) +{ + U32 const stat = rawStat + 1; + U32 const hb = ZSTD_highbit32(stat); + U32 const BWeight = hb * BITCOST_MULTIPLIER; + U32 const FWeight = (stat << BITCOST_ACCURACY) >> hb; + U32 const weight = BWeight + FWeight; + assert(hb + BITCOST_ACCURACY < 31); + return weight; +} + +#if (DEBUGLEVEL>=2) +/* debugging function, + * @return price in bytes as fractional value + * for debug messages only */ +MEM_STATIC double ZSTD_fCost(U32 price) +{ + return (double)price / (BITCOST_MULTIPLIER*8); +} +#endif + +static int ZSTD_compressedLiterals(optState_t const* const optPtr) +{ + return optPtr->literalCompressionMode != ZSTD_lcm_uncompressed; +} + +static void ZSTD_setBasePrices(optState_t* optPtr, int optLevel) +{ + if (ZSTD_compressedLiterals(optPtr)) + optPtr->litSumBasePrice = WEIGHT(optPtr->litSum, optLevel); + optPtr->litLengthSumBasePrice = WEIGHT(optPtr->litLengthSum, optLevel); + optPtr->matchLengthSumBasePrice = WEIGHT(optPtr->matchLengthSum, optLevel); + optPtr->offCodeSumBasePrice = WEIGHT(optPtr->offCodeSum, optLevel); +} + + +/* ZSTD_downscaleStat() : + * reduce all elements in table by a factor 2^(ZSTD_FREQ_DIV+malus) + * return the resulting sum of elements */ +static U32 ZSTD_downscaleStat(unsigned* table, U32 lastEltIndex, int malus) +{ + U32 s, sum=0; + DEBUGLOG(5, "ZSTD_downscaleStat (nbElts=%u)", (unsigned)lastEltIndex+1); + assert(ZSTD_FREQ_DIV+malus > 0 && ZSTD_FREQ_DIV+malus < 31); + for (s=0; s> (ZSTD_FREQ_DIV+malus)); + sum += table[s]; + } + return sum; +} + +/* ZSTD_rescaleFreqs() : + * if first block (detected by optPtr->litLengthSum == 0) : init statistics + * take hints from dictionary if there is one + * or init from zero, using src for literals stats, or flat 1 for match symbols + * otherwise downscale existing stats, to be used as seed for next block. + */ +static void +ZSTD_rescaleFreqs(optState_t* const optPtr, + const BYTE* const src, size_t const srcSize, + int const optLevel) +{ + int const compressedLiterals = ZSTD_compressedLiterals(optPtr); + DEBUGLOG(5, "ZSTD_rescaleFreqs (srcSize=%u)", (unsigned)srcSize); + optPtr->priceType = zop_dynamic; + + if (optPtr->litLengthSum == 0) { /* first block : init */ + if (srcSize <= ZSTD_PREDEF_THRESHOLD) { /* heuristic */ + DEBUGLOG(5, "(srcSize <= ZSTD_PREDEF_THRESHOLD) => zop_predef"); + optPtr->priceType = zop_predef; + } + + assert(optPtr->symbolCosts != NULL); + if (optPtr->symbolCosts->huf.repeatMode == HUF_repeat_valid) { + /* huffman table presumed generated by dictionary */ + optPtr->priceType = zop_dynamic; + + if (compressedLiterals) { + unsigned lit; + assert(optPtr->litFreq != NULL); + optPtr->litSum = 0; + for (lit=0; lit<=MaxLit; lit++) { + U32 const scaleLog = 11; /* scale to 2K */ + U32 const bitCost = HUF_getNbBits(optPtr->symbolCosts->huf.CTable, lit); + assert(bitCost <= scaleLog); + optPtr->litFreq[lit] = bitCost ? 1 << (scaleLog-bitCost) : 1 /*minimum to calculate cost*/; + optPtr->litSum += optPtr->litFreq[lit]; + } } + + { unsigned ll; + FSE_CState_t llstate; + FSE_initCState(&llstate, optPtr->symbolCosts->fse.litlengthCTable); + optPtr->litLengthSum = 0; + for (ll=0; ll<=MaxLL; ll++) { + U32 const scaleLog = 10; /* scale to 1K */ + U32 const bitCost = FSE_getMaxNbBits(llstate.symbolTT, ll); + assert(bitCost < scaleLog); + optPtr->litLengthFreq[ll] = bitCost ? 1 << (scaleLog-bitCost) : 1 /*minimum to calculate cost*/; + optPtr->litLengthSum += optPtr->litLengthFreq[ll]; + } } + + { unsigned ml; + FSE_CState_t mlstate; + FSE_initCState(&mlstate, optPtr->symbolCosts->fse.matchlengthCTable); + optPtr->matchLengthSum = 0; + for (ml=0; ml<=MaxML; ml++) { + U32 const scaleLog = 10; + U32 const bitCost = FSE_getMaxNbBits(mlstate.symbolTT, ml); + assert(bitCost < scaleLog); + optPtr->matchLengthFreq[ml] = bitCost ? 1 << (scaleLog-bitCost) : 1 /*minimum to calculate cost*/; + optPtr->matchLengthSum += optPtr->matchLengthFreq[ml]; + } } + + { unsigned of; + FSE_CState_t ofstate; + FSE_initCState(&ofstate, optPtr->symbolCosts->fse.offcodeCTable); + optPtr->offCodeSum = 0; + for (of=0; of<=MaxOff; of++) { + U32 const scaleLog = 10; + U32 const bitCost = FSE_getMaxNbBits(ofstate.symbolTT, of); + assert(bitCost < scaleLog); + optPtr->offCodeFreq[of] = bitCost ? 1 << (scaleLog-bitCost) : 1 /*minimum to calculate cost*/; + optPtr->offCodeSum += optPtr->offCodeFreq[of]; + } } + + } else { /* not a dictionary */ + + assert(optPtr->litFreq != NULL); + if (compressedLiterals) { + unsigned lit = MaxLit; + HIST_count_simple(optPtr->litFreq, &lit, src, srcSize); /* use raw first block to init statistics */ + optPtr->litSum = ZSTD_downscaleStat(optPtr->litFreq, MaxLit, 1); + } + + { unsigned ll; + for (ll=0; ll<=MaxLL; ll++) + optPtr->litLengthFreq[ll] = 1; + } + optPtr->litLengthSum = MaxLL+1; + + { unsigned ml; + for (ml=0; ml<=MaxML; ml++) + optPtr->matchLengthFreq[ml] = 1; + } + optPtr->matchLengthSum = MaxML+1; + + { unsigned of; + for (of=0; of<=MaxOff; of++) + optPtr->offCodeFreq[of] = 1; + } + optPtr->offCodeSum = MaxOff+1; + + } + + } else { /* new block : re-use previous statistics, scaled down */ + + if (compressedLiterals) + optPtr->litSum = ZSTD_downscaleStat(optPtr->litFreq, MaxLit, 1); + optPtr->litLengthSum = ZSTD_downscaleStat(optPtr->litLengthFreq, MaxLL, 0); + optPtr->matchLengthSum = ZSTD_downscaleStat(optPtr->matchLengthFreq, MaxML, 0); + optPtr->offCodeSum = ZSTD_downscaleStat(optPtr->offCodeFreq, MaxOff, 0); + } + + ZSTD_setBasePrices(optPtr, optLevel); +} + +/* ZSTD_rawLiteralsCost() : + * price of literals (only) in specified segment (which length can be 0). + * does not include price of literalLength symbol */ +static U32 ZSTD_rawLiteralsCost(const BYTE* const literals, U32 const litLength, + const optState_t* const optPtr, + int optLevel) +{ + if (litLength == 0) return 0; + + if (!ZSTD_compressedLiterals(optPtr)) + return (litLength << 3) * BITCOST_MULTIPLIER; /* Uncompressed - 8 bytes per literal. */ + + if (optPtr->priceType == zop_predef) + return (litLength*6) * BITCOST_MULTIPLIER; /* 6 bit per literal - no statistic used */ + + /* dynamic statistics */ + { U32 price = litLength * optPtr->litSumBasePrice; + U32 u; + for (u=0; u < litLength; u++) { + assert(WEIGHT(optPtr->litFreq[literals[u]], optLevel) <= optPtr->litSumBasePrice); /* literal cost should never be negative */ + price -= WEIGHT(optPtr->litFreq[literals[u]], optLevel); + } + return price; + } +} + +/* ZSTD_litLengthPrice() : + * cost of literalLength symbol */ +static U32 ZSTD_litLengthPrice(U32 const litLength, const optState_t* const optPtr, int optLevel) +{ + if (optPtr->priceType == zop_predef) return WEIGHT(litLength, optLevel); + + /* dynamic statistics */ + { U32 const llCode = ZSTD_LLcode(litLength); + return (LL_bits[llCode] * BITCOST_MULTIPLIER) + + optPtr->litLengthSumBasePrice + - WEIGHT(optPtr->litLengthFreq[llCode], optLevel); + } +} + +/* ZSTD_getMatchPrice() : + * Provides the cost of the match part (offset + matchLength) of a sequence + * Must be combined with ZSTD_fullLiteralsCost() to get the full cost of a sequence. + * optLevel: when <2, favors small offset for decompression speed (improved cache efficiency) */ +FORCE_INLINE_TEMPLATE U32 +ZSTD_getMatchPrice(U32 const offset, + U32 const matchLength, + const optState_t* const optPtr, + int const optLevel) +{ + U32 price; + U32 const offCode = ZSTD_highbit32(offset+1); + U32 const mlBase = matchLength - MINMATCH; + assert(matchLength >= MINMATCH); + + if (optPtr->priceType == zop_predef) /* fixed scheme, do not use statistics */ + return WEIGHT(mlBase, optLevel) + ((16 + offCode) * BITCOST_MULTIPLIER); + + /* dynamic statistics */ + price = (offCode * BITCOST_MULTIPLIER) + (optPtr->offCodeSumBasePrice - WEIGHT(optPtr->offCodeFreq[offCode], optLevel)); + if ((optLevel<2) /*static*/ && offCode >= 20) + price += (offCode-19)*2 * BITCOST_MULTIPLIER; /* handicap for long distance offsets, favor decompression speed */ + + /* match Length */ + { U32 const mlCode = ZSTD_MLcode(mlBase); + price += (ML_bits[mlCode] * BITCOST_MULTIPLIER) + (optPtr->matchLengthSumBasePrice - WEIGHT(optPtr->matchLengthFreq[mlCode], optLevel)); + } + + price += BITCOST_MULTIPLIER / 5; /* heuristic : make matches a bit more costly to favor less sequences -> faster decompression speed */ + + DEBUGLOG(8, "ZSTD_getMatchPrice(ml:%u) = %u", matchLength, price); + return price; +} + +/* ZSTD_updateStats() : + * assumption : literals + litLengtn <= iend */ +static void ZSTD_updateStats(optState_t* const optPtr, + U32 litLength, const BYTE* literals, + U32 offsetCode, U32 matchLength) +{ + /* literals */ + if (ZSTD_compressedLiterals(optPtr)) { + U32 u; + for (u=0; u < litLength; u++) + optPtr->litFreq[literals[u]] += ZSTD_LITFREQ_ADD; + optPtr->litSum += litLength*ZSTD_LITFREQ_ADD; + } + + /* literal Length */ + { U32 const llCode = ZSTD_LLcode(litLength); + optPtr->litLengthFreq[llCode]++; + optPtr->litLengthSum++; + } + + /* match offset code (0-2=>repCode; 3+=>offset+2) */ + { U32 const offCode = ZSTD_highbit32(offsetCode+1); + assert(offCode <= MaxOff); + optPtr->offCodeFreq[offCode]++; + optPtr->offCodeSum++; + } + + /* match Length */ + { U32 const mlBase = matchLength - MINMATCH; + U32 const mlCode = ZSTD_MLcode(mlBase); + optPtr->matchLengthFreq[mlCode]++; + optPtr->matchLengthSum++; + } +} + + +/* ZSTD_readMINMATCH() : + * function safe only for comparisons + * assumption : memPtr must be at least 4 bytes before end of buffer */ +MEM_STATIC U32 ZSTD_readMINMATCH(const void* memPtr, U32 length) +{ + switch (length) + { + default : + case 4 : return MEM_read32(memPtr); + case 3 : if (MEM_isLittleEndian()) + return MEM_read32(memPtr)<<8; + else + return MEM_read32(memPtr)>>8; + } +} + + +/* Update hashTable3 up to ip (excluded) + Assumption : always within prefix (i.e. not within extDict) */ +static U32 ZSTD_insertAndFindFirstIndexHash3 (ZSTD_matchState_t* ms, + U32* nextToUpdate3, + const BYTE* const ip) +{ + U32* const hashTable3 = ms->hashTable3; + U32 const hashLog3 = ms->hashLog3; + const BYTE* const base = ms->window.base; + U32 idx = *nextToUpdate3; + U32 const target = (U32)(ip - base); + size_t const hash3 = ZSTD_hash3Ptr(ip, hashLog3); + assert(hashLog3 > 0); + + while(idx < target) { + hashTable3[ZSTD_hash3Ptr(base+idx, hashLog3)] = idx; + idx++; + } + + *nextToUpdate3 = target; + return hashTable3[hash3]; +} + + +/*-************************************* +* Binary Tree search +***************************************/ +/** ZSTD_insertBt1() : add one or multiple positions to tree. + * ip : assumed <= iend-8 . + * @return : nb of positions added */ +static U32 ZSTD_insertBt1( + ZSTD_matchState_t* ms, + const BYTE* const ip, const BYTE* const iend, + U32 const mls, const int extDict) +{ + const ZSTD_compressionParameters* const cParams = &ms->cParams; + U32* const hashTable = ms->hashTable; + U32 const hashLog = cParams->hashLog; + size_t const h = ZSTD_hashPtr(ip, hashLog, mls); + U32* const bt = ms->chainTable; + U32 const btLog = cParams->chainLog - 1; + U32 const btMask = (1 << btLog) - 1; + U32 matchIndex = hashTable[h]; + size_t commonLengthSmaller=0, commonLengthLarger=0; + const BYTE* const base = ms->window.base; + const BYTE* const dictBase = ms->window.dictBase; + const U32 dictLimit = ms->window.dictLimit; + const BYTE* const dictEnd = dictBase + dictLimit; + const BYTE* const prefixStart = base + dictLimit; + const BYTE* match; + const U32 current = (U32)(ip-base); + const U32 btLow = btMask >= current ? 0 : current - btMask; + U32* smallerPtr = bt + 2*(current&btMask); + U32* largerPtr = smallerPtr + 1; + U32 dummy32; /* to be nullified at the end */ + U32 const windowLow = ms->window.lowLimit; + U32 matchEndIdx = current+8+1; + size_t bestLength = 8; + U32 nbCompares = 1U << cParams->searchLog; +#ifdef ZSTD_C_PREDICT + U32 predictedSmall = *(bt + 2*((current-1)&btMask) + 0); + U32 predictedLarge = *(bt + 2*((current-1)&btMask) + 1); + predictedSmall += (predictedSmall>0); + predictedLarge += (predictedLarge>0); +#endif /* ZSTD_C_PREDICT */ + + DEBUGLOG(8, "ZSTD_insertBt1 (%u)", current); + + assert(ip <= iend-8); /* required for h calculation */ + hashTable[h] = current; /* Update Hash Table */ + + assert(windowLow > 0); + while (nbCompares-- && (matchIndex >= windowLow)) { + U32* const nextPtr = bt + 2*(matchIndex & btMask); + size_t matchLength = MIN(commonLengthSmaller, commonLengthLarger); /* guaranteed minimum nb of common bytes */ + assert(matchIndex < current); + +#ifdef ZSTD_C_PREDICT /* note : can create issues when hlog small <= 11 */ + const U32* predictPtr = bt + 2*((matchIndex-1) & btMask); /* written this way, as bt is a roll buffer */ + if (matchIndex == predictedSmall) { + /* no need to check length, result known */ + *smallerPtr = matchIndex; + if (matchIndex <= btLow) { smallerPtr=&dummy32; break; } /* beyond tree size, stop the search */ + smallerPtr = nextPtr+1; /* new "smaller" => larger of match */ + matchIndex = nextPtr[1]; /* new matchIndex larger than previous (closer to current) */ + predictedSmall = predictPtr[1] + (predictPtr[1]>0); + continue; + } + if (matchIndex == predictedLarge) { + *largerPtr = matchIndex; + if (matchIndex <= btLow) { largerPtr=&dummy32; break; } /* beyond tree size, stop the search */ + largerPtr = nextPtr; + matchIndex = nextPtr[0]; + predictedLarge = predictPtr[0] + (predictPtr[0]>0); + continue; + } +#endif + + if (!extDict || (matchIndex+matchLength >= dictLimit)) { + assert(matchIndex+matchLength >= dictLimit); /* might be wrong if actually extDict */ + match = base + matchIndex; + matchLength += ZSTD_count(ip+matchLength, match+matchLength, iend); + } else { + match = dictBase + matchIndex; + matchLength += ZSTD_count_2segments(ip+matchLength, match+matchLength, iend, dictEnd, prefixStart); + if (matchIndex+matchLength >= dictLimit) + match = base + matchIndex; /* to prepare for next usage of match[matchLength] */ + } + + if (matchLength > bestLength) { + bestLength = matchLength; + if (matchLength > matchEndIdx - matchIndex) + matchEndIdx = matchIndex + (U32)matchLength; + } + + if (ip+matchLength == iend) { /* equal : no way to know if inf or sup */ + break; /* drop , to guarantee consistency ; miss a bit of compression, but other solutions can corrupt tree */ + } + + if (match[matchLength] < ip[matchLength]) { /* necessarily within buffer */ + /* match is smaller than current */ + *smallerPtr = matchIndex; /* update smaller idx */ + commonLengthSmaller = matchLength; /* all smaller will now have at least this guaranteed common length */ + if (matchIndex <= btLow) { smallerPtr=&dummy32; break; } /* beyond tree size, stop searching */ + smallerPtr = nextPtr+1; /* new "candidate" => larger than match, which was smaller than target */ + matchIndex = nextPtr[1]; /* new matchIndex, larger than previous and closer to current */ + } else { + /* match is larger than current */ + *largerPtr = matchIndex; + commonLengthLarger = matchLength; + if (matchIndex <= btLow) { largerPtr=&dummy32; break; } /* beyond tree size, stop searching */ + largerPtr = nextPtr; + matchIndex = nextPtr[0]; + } } + + *smallerPtr = *largerPtr = 0; + { U32 positions = 0; + if (bestLength > 384) positions = MIN(192, (U32)(bestLength - 384)); /* speed optimization */ + assert(matchEndIdx > current + 8); + return MAX(positions, matchEndIdx - (current + 8)); + } +} + +FORCE_INLINE_TEMPLATE +void ZSTD_updateTree_internal( + ZSTD_matchState_t* ms, + const BYTE* const ip, const BYTE* const iend, + const U32 mls, const ZSTD_dictMode_e dictMode) +{ + const BYTE* const base = ms->window.base; + U32 const target = (U32)(ip - base); + U32 idx = ms->nextToUpdate; + DEBUGLOG(6, "ZSTD_updateTree_internal, from %u to %u (dictMode:%u)", + idx, target, dictMode); + + while(idx < target) { + U32 const forward = ZSTD_insertBt1(ms, base+idx, iend, mls, dictMode == ZSTD_extDict); + assert(idx < (U32)(idx + forward)); + idx += forward; + } + assert((size_t)(ip - base) <= (size_t)(U32)(-1)); + assert((size_t)(iend - base) <= (size_t)(U32)(-1)); + ms->nextToUpdate = target; +} + +void ZSTD_updateTree(ZSTD_matchState_t* ms, const BYTE* ip, const BYTE* iend) { + ZSTD_updateTree_internal(ms, ip, iend, ms->cParams.minMatch, ZSTD_noDict); +} + +FORCE_INLINE_TEMPLATE +U32 ZSTD_insertBtAndGetAllMatches ( + ZSTD_match_t* matches, /* store result (found matches) in this table (presumed large enough) */ + ZSTD_matchState_t* ms, + U32* nextToUpdate3, + const BYTE* const ip, const BYTE* const iLimit, const ZSTD_dictMode_e dictMode, + const U32 rep[ZSTD_REP_NUM], + U32 const ll0, /* tells if associated literal length is 0 or not. This value must be 0 or 1 */ + const U32 lengthToBeat, + U32 const mls /* template */) +{ + const ZSTD_compressionParameters* const cParams = &ms->cParams; + U32 const sufficient_len = MIN(cParams->targetLength, ZSTD_OPT_NUM -1); + const BYTE* const base = ms->window.base; + U32 const current = (U32)(ip-base); + U32 const hashLog = cParams->hashLog; + U32 const minMatch = (mls==3) ? 3 : 4; + U32* const hashTable = ms->hashTable; + size_t const h = ZSTD_hashPtr(ip, hashLog, mls); + U32 matchIndex = hashTable[h]; + U32* const bt = ms->chainTable; + U32 const btLog = cParams->chainLog - 1; + U32 const btMask= (1U << btLog) - 1; + size_t commonLengthSmaller=0, commonLengthLarger=0; + const BYTE* const dictBase = ms->window.dictBase; + U32 const dictLimit = ms->window.dictLimit; + const BYTE* const dictEnd = dictBase + dictLimit; + const BYTE* const prefixStart = base + dictLimit; + U32 const btLow = (btMask >= current) ? 0 : current - btMask; + U32 const windowLow = ZSTD_getLowestMatchIndex(ms, current, cParams->windowLog); + U32 const matchLow = windowLow ? windowLow : 1; + U32* smallerPtr = bt + 2*(current&btMask); + U32* largerPtr = bt + 2*(current&btMask) + 1; + U32 matchEndIdx = current+8+1; /* farthest referenced position of any match => detects repetitive patterns */ + U32 dummy32; /* to be nullified at the end */ + U32 mnum = 0; + U32 nbCompares = 1U << cParams->searchLog; + + const ZSTD_matchState_t* dms = dictMode == ZSTD_dictMatchState ? ms->dictMatchState : NULL; + const ZSTD_compressionParameters* const dmsCParams = + dictMode == ZSTD_dictMatchState ? &dms->cParams : NULL; + const BYTE* const dmsBase = dictMode == ZSTD_dictMatchState ? dms->window.base : NULL; + const BYTE* const dmsEnd = dictMode == ZSTD_dictMatchState ? dms->window.nextSrc : NULL; + U32 const dmsHighLimit = dictMode == ZSTD_dictMatchState ? (U32)(dmsEnd - dmsBase) : 0; + U32 const dmsLowLimit = dictMode == ZSTD_dictMatchState ? dms->window.lowLimit : 0; + U32 const dmsIndexDelta = dictMode == ZSTD_dictMatchState ? windowLow - dmsHighLimit : 0; + U32 const dmsHashLog = dictMode == ZSTD_dictMatchState ? dmsCParams->hashLog : hashLog; + U32 const dmsBtLog = dictMode == ZSTD_dictMatchState ? dmsCParams->chainLog - 1 : btLog; + U32 const dmsBtMask = dictMode == ZSTD_dictMatchState ? (1U << dmsBtLog) - 1 : 0; + U32 const dmsBtLow = dictMode == ZSTD_dictMatchState && dmsBtMask < dmsHighLimit - dmsLowLimit ? dmsHighLimit - dmsBtMask : dmsLowLimit; + + size_t bestLength = lengthToBeat-1; + DEBUGLOG(8, "ZSTD_insertBtAndGetAllMatches: current=%u", current); + + /* check repCode */ + assert(ll0 <= 1); /* necessarily 1 or 0 */ + { U32 const lastR = ZSTD_REP_NUM + ll0; + U32 repCode; + for (repCode = ll0; repCode < lastR; repCode++) { + U32 const repOffset = (repCode==ZSTD_REP_NUM) ? (rep[0] - 1) : rep[repCode]; + U32 const repIndex = current - repOffset; + U32 repLen = 0; + assert(current >= dictLimit); + if (repOffset-1 /* intentional overflow, discards 0 and -1 */ < current-dictLimit) { /* equivalent to `current > repIndex >= dictLimit` */ + /* We must validate the repcode offset because when we're using a dictionary the + * valid offset range shrinks when the dictionary goes out of bounds. + */ + if ((repIndex >= windowLow) & (ZSTD_readMINMATCH(ip, minMatch) == ZSTD_readMINMATCH(ip - repOffset, minMatch))) { + repLen = (U32)ZSTD_count(ip+minMatch, ip+minMatch-repOffset, iLimit) + minMatch; + } + } else { /* repIndex < dictLimit || repIndex >= current */ + const BYTE* const repMatch = dictMode == ZSTD_dictMatchState ? + dmsBase + repIndex - dmsIndexDelta : + dictBase + repIndex; + assert(current >= windowLow); + if ( dictMode == ZSTD_extDict + && ( ((repOffset-1) /*intentional overflow*/ < current - windowLow) /* equivalent to `current > repIndex >= windowLow` */ + & (((U32)((dictLimit-1) - repIndex) >= 3) ) /* intentional overflow : do not test positions overlapping 2 memory segments */) + && (ZSTD_readMINMATCH(ip, minMatch) == ZSTD_readMINMATCH(repMatch, minMatch)) ) { + repLen = (U32)ZSTD_count_2segments(ip+minMatch, repMatch+minMatch, iLimit, dictEnd, prefixStart) + minMatch; + } + if (dictMode == ZSTD_dictMatchState + && ( ((repOffset-1) /*intentional overflow*/ < current - (dmsLowLimit + dmsIndexDelta)) /* equivalent to `current > repIndex >= dmsLowLimit` */ + & ((U32)((dictLimit-1) - repIndex) >= 3) ) /* intentional overflow : do not test positions overlapping 2 memory segments */ + && (ZSTD_readMINMATCH(ip, minMatch) == ZSTD_readMINMATCH(repMatch, minMatch)) ) { + repLen = (U32)ZSTD_count_2segments(ip+minMatch, repMatch+minMatch, iLimit, dmsEnd, prefixStart) + minMatch; + } } + /* save longer solution */ + if (repLen > bestLength) { + DEBUGLOG(8, "found repCode %u (ll0:%u, offset:%u) of length %u", + repCode, ll0, repOffset, repLen); + bestLength = repLen; + matches[mnum].off = repCode - ll0; + matches[mnum].len = (U32)repLen; + mnum++; + if ( (repLen > sufficient_len) + | (ip+repLen == iLimit) ) { /* best possible */ + return mnum; + } } } } + + /* HC3 match finder */ + if ((mls == 3) /*static*/ && (bestLength < mls)) { + U32 const matchIndex3 = ZSTD_insertAndFindFirstIndexHash3(ms, nextToUpdate3, ip); + if ((matchIndex3 >= matchLow) + & (current - matchIndex3 < (1<<18)) /*heuristic : longer distance likely too expensive*/ ) { + size_t mlen; + if ((dictMode == ZSTD_noDict) /*static*/ || (dictMode == ZSTD_dictMatchState) /*static*/ || (matchIndex3 >= dictLimit)) { + const BYTE* const match = base + matchIndex3; + mlen = ZSTD_count(ip, match, iLimit); + } else { + const BYTE* const match = dictBase + matchIndex3; + mlen = ZSTD_count_2segments(ip, match, iLimit, dictEnd, prefixStart); + } + + /* save best solution */ + if (mlen >= mls /* == 3 > bestLength */) { + DEBUGLOG(8, "found small match with hlog3, of length %u", + (U32)mlen); + bestLength = mlen; + assert(current > matchIndex3); + assert(mnum==0); /* no prior solution */ + matches[0].off = (current - matchIndex3) + ZSTD_REP_MOVE; + matches[0].len = (U32)mlen; + mnum = 1; + if ( (mlen > sufficient_len) | + (ip+mlen == iLimit) ) { /* best possible length */ + ms->nextToUpdate = current+1; /* skip insertion */ + return 1; + } } } + /* no dictMatchState lookup: dicts don't have a populated HC3 table */ + } + + hashTable[h] = current; /* Update Hash Table */ + + while (nbCompares-- && (matchIndex >= matchLow)) { + U32* const nextPtr = bt + 2*(matchIndex & btMask); + const BYTE* match; + size_t matchLength = MIN(commonLengthSmaller, commonLengthLarger); /* guaranteed minimum nb of common bytes */ + assert(current > matchIndex); + + if ((dictMode == ZSTD_noDict) || (dictMode == ZSTD_dictMatchState) || (matchIndex+matchLength >= dictLimit)) { + assert(matchIndex+matchLength >= dictLimit); /* ensure the condition is correct when !extDict */ + match = base + matchIndex; + if (matchIndex >= dictLimit) assert(memcmp(match, ip, matchLength) == 0); /* ensure early section of match is equal as expected */ + matchLength += ZSTD_count(ip+matchLength, match+matchLength, iLimit); + } else { + match = dictBase + matchIndex; + assert(memcmp(match, ip, matchLength) == 0); /* ensure early section of match is equal as expected */ + matchLength += ZSTD_count_2segments(ip+matchLength, match+matchLength, iLimit, dictEnd, prefixStart); + if (matchIndex+matchLength >= dictLimit) + match = base + matchIndex; /* prepare for match[matchLength] read */ + } + + if (matchLength > bestLength) { + DEBUGLOG(8, "found match of length %u at distance %u (offCode=%u)", + (U32)matchLength, current - matchIndex, current - matchIndex + ZSTD_REP_MOVE); + assert(matchEndIdx > matchIndex); + if (matchLength > matchEndIdx - matchIndex) + matchEndIdx = matchIndex + (U32)matchLength; + bestLength = matchLength; + matches[mnum].off = (current - matchIndex) + ZSTD_REP_MOVE; + matches[mnum].len = (U32)matchLength; + mnum++; + if ( (matchLength > ZSTD_OPT_NUM) + | (ip+matchLength == iLimit) /* equal : no way to know if inf or sup */) { + if (dictMode == ZSTD_dictMatchState) nbCompares = 0; /* break should also skip searching dms */ + break; /* drop, to preserve bt consistency (miss a little bit of compression) */ + } + } + + if (match[matchLength] < ip[matchLength]) { + /* match smaller than current */ + *smallerPtr = matchIndex; /* update smaller idx */ + commonLengthSmaller = matchLength; /* all smaller will now have at least this guaranteed common length */ + if (matchIndex <= btLow) { smallerPtr=&dummy32; break; } /* beyond tree size, stop the search */ + smallerPtr = nextPtr+1; /* new candidate => larger than match, which was smaller than current */ + matchIndex = nextPtr[1]; /* new matchIndex, larger than previous, closer to current */ + } else { + *largerPtr = matchIndex; + commonLengthLarger = matchLength; + if (matchIndex <= btLow) { largerPtr=&dummy32; break; } /* beyond tree size, stop the search */ + largerPtr = nextPtr; + matchIndex = nextPtr[0]; + } } + + *smallerPtr = *largerPtr = 0; + + if (dictMode == ZSTD_dictMatchState && nbCompares) { + size_t const dmsH = ZSTD_hashPtr(ip, dmsHashLog, mls); + U32 dictMatchIndex = dms->hashTable[dmsH]; + const U32* const dmsBt = dms->chainTable; + commonLengthSmaller = commonLengthLarger = 0; + while (nbCompares-- && (dictMatchIndex > dmsLowLimit)) { + const U32* const nextPtr = dmsBt + 2*(dictMatchIndex & dmsBtMask); + size_t matchLength = MIN(commonLengthSmaller, commonLengthLarger); /* guaranteed minimum nb of common bytes */ + const BYTE* match = dmsBase + dictMatchIndex; + matchLength += ZSTD_count_2segments(ip+matchLength, match+matchLength, iLimit, dmsEnd, prefixStart); + if (dictMatchIndex+matchLength >= dmsHighLimit) + match = base + dictMatchIndex + dmsIndexDelta; /* to prepare for next usage of match[matchLength] */ + + if (matchLength > bestLength) { + matchIndex = dictMatchIndex + dmsIndexDelta; + DEBUGLOG(8, "found dms match of length %u at distance %u (offCode=%u)", + (U32)matchLength, current - matchIndex, current - matchIndex + ZSTD_REP_MOVE); + if (matchLength > matchEndIdx - matchIndex) + matchEndIdx = matchIndex + (U32)matchLength; + bestLength = matchLength; + matches[mnum].off = (current - matchIndex) + ZSTD_REP_MOVE; + matches[mnum].len = (U32)matchLength; + mnum++; + if ( (matchLength > ZSTD_OPT_NUM) + | (ip+matchLength == iLimit) /* equal : no way to know if inf or sup */) { + break; /* drop, to guarantee consistency (miss a little bit of compression) */ + } + } + + if (dictMatchIndex <= dmsBtLow) { break; } /* beyond tree size, stop the search */ + if (match[matchLength] < ip[matchLength]) { + commonLengthSmaller = matchLength; /* all smaller will now have at least this guaranteed common length */ + dictMatchIndex = nextPtr[1]; /* new matchIndex larger than previous (closer to current) */ + } else { + /* match is larger than current */ + commonLengthLarger = matchLength; + dictMatchIndex = nextPtr[0]; + } + } + } + + assert(matchEndIdx > current+8); + ms->nextToUpdate = matchEndIdx - 8; /* skip repetitive patterns */ + return mnum; +} + + +FORCE_INLINE_TEMPLATE U32 ZSTD_BtGetAllMatches ( + ZSTD_match_t* matches, /* store result (match found, increasing size) in this table */ + ZSTD_matchState_t* ms, + U32* nextToUpdate3, + const BYTE* ip, const BYTE* const iHighLimit, const ZSTD_dictMode_e dictMode, + const U32 rep[ZSTD_REP_NUM], + U32 const ll0, + U32 const lengthToBeat) +{ + const ZSTD_compressionParameters* const cParams = &ms->cParams; + U32 const matchLengthSearch = cParams->minMatch; + DEBUGLOG(8, "ZSTD_BtGetAllMatches"); + if (ip < ms->window.base + ms->nextToUpdate) return 0; /* skipped area */ + ZSTD_updateTree_internal(ms, ip, iHighLimit, matchLengthSearch, dictMode); + switch(matchLengthSearch) + { + case 3 : return ZSTD_insertBtAndGetAllMatches(matches, ms, nextToUpdate3, ip, iHighLimit, dictMode, rep, ll0, lengthToBeat, 3); + default : + case 4 : return ZSTD_insertBtAndGetAllMatches(matches, ms, nextToUpdate3, ip, iHighLimit, dictMode, rep, ll0, lengthToBeat, 4); + case 5 : return ZSTD_insertBtAndGetAllMatches(matches, ms, nextToUpdate3, ip, iHighLimit, dictMode, rep, ll0, lengthToBeat, 5); + case 7 : + case 6 : return ZSTD_insertBtAndGetAllMatches(matches, ms, nextToUpdate3, ip, iHighLimit, dictMode, rep, ll0, lengthToBeat, 6); + } +} + + +/*-******************************* +* Optimal parser +*********************************/ + + +static U32 ZSTD_totalLen(ZSTD_optimal_t sol) +{ + return sol.litlen + sol.mlen; +} + +#if 0 /* debug */ + +static void +listStats(const U32* table, int lastEltID) +{ + int const nbElts = lastEltID + 1; + int enb; + for (enb=0; enb < nbElts; enb++) { + (void)table; + /* RAWLOG(2, "%3i:%3i, ", enb, table[enb]); */ + RAWLOG(2, "%4i,", table[enb]); + } + RAWLOG(2, " \n"); +} + +#endif + +FORCE_INLINE_TEMPLATE size_t +ZSTD_compressBlock_opt_generic(ZSTD_matchState_t* ms, + seqStore_t* seqStore, + U32 rep[ZSTD_REP_NUM], + const void* src, size_t srcSize, + const int optLevel, + const ZSTD_dictMode_e dictMode) +{ + optState_t* const optStatePtr = &ms->opt; + const BYTE* const istart = (const BYTE*)src; + const BYTE* ip = istart; + const BYTE* anchor = istart; + const BYTE* const iend = istart + srcSize; + const BYTE* const ilimit = iend - 8; + const BYTE* const base = ms->window.base; + const BYTE* const prefixStart = base + ms->window.dictLimit; + const ZSTD_compressionParameters* const cParams = &ms->cParams; + + U32 const sufficient_len = MIN(cParams->targetLength, ZSTD_OPT_NUM -1); + U32 const minMatch = (cParams->minMatch == 3) ? 3 : 4; + U32 nextToUpdate3 = ms->nextToUpdate; + + ZSTD_optimal_t* const opt = optStatePtr->priceTable; + ZSTD_match_t* const matches = optStatePtr->matchTable; + ZSTD_optimal_t lastSequence; + + /* init */ + DEBUGLOG(5, "ZSTD_compressBlock_opt_generic: current=%u, prefix=%u, nextToUpdate=%u", + (U32)(ip - base), ms->window.dictLimit, ms->nextToUpdate); + assert(optLevel <= 2); + ZSTD_rescaleFreqs(optStatePtr, (const BYTE*)src, srcSize, optLevel); + ip += (ip==prefixStart); + + /* Match Loop */ + while (ip < ilimit) { + U32 cur, last_pos = 0; + + /* find first match */ + { U32 const litlen = (U32)(ip - anchor); + U32 const ll0 = !litlen; + U32 const nbMatches = ZSTD_BtGetAllMatches(matches, ms, &nextToUpdate3, ip, iend, dictMode, rep, ll0, minMatch); + if (!nbMatches) { ip++; continue; } + + /* initialize opt[0] */ + { U32 i ; for (i=0; i immediate encoding */ + { U32 const maxML = matches[nbMatches-1].len; + U32 const maxOffset = matches[nbMatches-1].off; + DEBUGLOG(6, "found %u matches of maxLength=%u and maxOffCode=%u at cPos=%u => start new series", + nbMatches, maxML, maxOffset, (U32)(ip-prefixStart)); + + if (maxML > sufficient_len) { + lastSequence.litlen = litlen; + lastSequence.mlen = maxML; + lastSequence.off = maxOffset; + DEBUGLOG(6, "large match (%u>%u), immediate encoding", + maxML, sufficient_len); + cur = 0; + last_pos = ZSTD_totalLen(lastSequence); + goto _shortestPath; + } } + + /* set prices for first matches starting position == 0 */ + { U32 const literalsPrice = opt[0].price + ZSTD_litLengthPrice(0, optStatePtr, optLevel); + U32 pos; + U32 matchNb; + for (pos = 1; pos < minMatch; pos++) { + opt[pos].price = ZSTD_MAX_PRICE; /* mlen, litlen and price will be fixed during forward scanning */ + } + for (matchNb = 0; matchNb < nbMatches; matchNb++) { + U32 const offset = matches[matchNb].off; + U32 const end = matches[matchNb].len; + for ( ; pos <= end ; pos++ ) { + U32 const matchPrice = ZSTD_getMatchPrice(offset, pos, optStatePtr, optLevel); + U32 const sequencePrice = literalsPrice + matchPrice; + DEBUGLOG(7, "rPos:%u => set initial price : %.2f", + pos, ZSTD_fCost(sequencePrice)); + opt[pos].mlen = pos; + opt[pos].off = offset; + opt[pos].litlen = litlen; + opt[pos].price = sequencePrice; + } } + last_pos = pos-1; + } + } + + /* check further positions */ + for (cur = 1; cur <= last_pos; cur++) { + const BYTE* const inr = ip + cur; + assert(cur < ZSTD_OPT_NUM); + DEBUGLOG(7, "cPos:%zi==rPos:%u", inr-istart, cur) + + /* Fix current position with one literal if cheaper */ + { U32 const litlen = (opt[cur-1].mlen == 0) ? opt[cur-1].litlen + 1 : 1; + int const price = opt[cur-1].price + + ZSTD_rawLiteralsCost(ip+cur-1, 1, optStatePtr, optLevel) + + ZSTD_litLengthPrice(litlen, optStatePtr, optLevel) + - ZSTD_litLengthPrice(litlen-1, optStatePtr, optLevel); + assert(price < 1000000000); /* overflow check */ + if (price <= opt[cur].price) { + DEBUGLOG(7, "cPos:%zi==rPos:%u : better price (%.2f<=%.2f) using literal (ll==%u) (hist:%u,%u,%u)", + inr-istart, cur, ZSTD_fCost(price), ZSTD_fCost(opt[cur].price), litlen, + opt[cur-1].rep[0], opt[cur-1].rep[1], opt[cur-1].rep[2]); + opt[cur].mlen = 0; + opt[cur].off = 0; + opt[cur].litlen = litlen; + opt[cur].price = price; + } else { + DEBUGLOG(7, "cPos:%zi==rPos:%u : literal would cost more (%.2f>%.2f) (hist:%u,%u,%u)", + inr-istart, cur, ZSTD_fCost(price), ZSTD_fCost(opt[cur].price), + opt[cur].rep[0], opt[cur].rep[1], opt[cur].rep[2]); + } + } + + /* Set the repcodes of the current position. We must do it here + * because we rely on the repcodes of the 2nd to last sequence being + * correct to set the next chunks repcodes during the backward + * traversal. + */ + ZSTD_STATIC_ASSERT(sizeof(opt[cur].rep) == sizeof(repcodes_t)); + assert(cur >= opt[cur].mlen); + if (opt[cur].mlen != 0) { + U32 const prev = cur - opt[cur].mlen; + repcodes_t newReps = ZSTD_updateRep(opt[prev].rep, opt[cur].off, opt[cur].litlen==0); + memcpy(opt[cur].rep, &newReps, sizeof(repcodes_t)); + } else { + memcpy(opt[cur].rep, opt[cur - 1].rep, sizeof(repcodes_t)); + } + + /* last match must start at a minimum distance of 8 from oend */ + if (inr > ilimit) continue; + + if (cur == last_pos) break; + + if ( (optLevel==0) /*static_test*/ + && (opt[cur+1].price <= opt[cur].price + (BITCOST_MULTIPLIER/2)) ) { + DEBUGLOG(7, "move to next rPos:%u : price is <=", cur+1); + continue; /* skip unpromising positions; about ~+6% speed, -0.01 ratio */ + } + + { U32 const ll0 = (opt[cur].mlen != 0); + U32 const litlen = (opt[cur].mlen == 0) ? opt[cur].litlen : 0; + U32 const previousPrice = opt[cur].price; + U32 const basePrice = previousPrice + ZSTD_litLengthPrice(0, optStatePtr, optLevel); + U32 const nbMatches = ZSTD_BtGetAllMatches(matches, ms, &nextToUpdate3, inr, iend, dictMode, opt[cur].rep, ll0, minMatch); + U32 matchNb; + if (!nbMatches) { + DEBUGLOG(7, "rPos:%u : no match found", cur); + continue; + } + + { U32 const maxML = matches[nbMatches-1].len; + DEBUGLOG(7, "cPos:%zi==rPos:%u, found %u matches, of maxLength=%u", + inr-istart, cur, nbMatches, maxML); + + if ( (maxML > sufficient_len) + || (cur + maxML >= ZSTD_OPT_NUM) ) { + lastSequence.mlen = maxML; + lastSequence.off = matches[nbMatches-1].off; + lastSequence.litlen = litlen; + cur -= (opt[cur].mlen==0) ? opt[cur].litlen : 0; /* last sequence is actually only literals, fix cur to last match - note : may underflow, in which case, it's first sequence, and it's okay */ + last_pos = cur + ZSTD_totalLen(lastSequence); + if (cur > ZSTD_OPT_NUM) cur = 0; /* underflow => first match */ + goto _shortestPath; + } } + + /* set prices using matches found at position == cur */ + for (matchNb = 0; matchNb < nbMatches; matchNb++) { + U32 const offset = matches[matchNb].off; + U32 const lastML = matches[matchNb].len; + U32 const startML = (matchNb>0) ? matches[matchNb-1].len+1 : minMatch; + U32 mlen; + + DEBUGLOG(7, "testing match %u => offCode=%4u, mlen=%2u, llen=%2u", + matchNb, matches[matchNb].off, lastML, litlen); + + for (mlen = lastML; mlen >= startML; mlen--) { /* scan downward */ + U32 const pos = cur + mlen; + int const price = basePrice + ZSTD_getMatchPrice(offset, mlen, optStatePtr, optLevel); + + if ((pos > last_pos) || (price < opt[pos].price)) { + DEBUGLOG(7, "rPos:%u (ml=%2u) => new better price (%.2f<%.2f)", + pos, mlen, ZSTD_fCost(price), ZSTD_fCost(opt[pos].price)); + while (last_pos < pos) { opt[last_pos+1].price = ZSTD_MAX_PRICE; last_pos++; } /* fill empty positions */ + opt[pos].mlen = mlen; + opt[pos].off = offset; + opt[pos].litlen = litlen; + opt[pos].price = price; + } else { + DEBUGLOG(7, "rPos:%u (ml=%2u) => new price is worse (%.2f>=%.2f)", + pos, mlen, ZSTD_fCost(price), ZSTD_fCost(opt[pos].price)); + if (optLevel==0) break; /* early update abort; gets ~+10% speed for about -0.01 ratio loss */ + } + } } } + } /* for (cur = 1; cur <= last_pos; cur++) */ + + lastSequence = opt[last_pos]; + cur = last_pos > ZSTD_totalLen(lastSequence) ? last_pos - ZSTD_totalLen(lastSequence) : 0; /* single sequence, and it starts before `ip` */ + assert(cur < ZSTD_OPT_NUM); /* control overflow*/ + +_shortestPath: /* cur, last_pos, best_mlen, best_off have to be set */ + assert(opt[0].mlen == 0); + + /* Set the next chunk's repcodes based on the repcodes of the beginning + * of the last match, and the last sequence. This avoids us having to + * update them while traversing the sequences. + */ + if (lastSequence.mlen != 0) { + repcodes_t reps = ZSTD_updateRep(opt[cur].rep, lastSequence.off, lastSequence.litlen==0); + memcpy(rep, &reps, sizeof(reps)); + } else { + memcpy(rep, opt[cur].rep, sizeof(repcodes_t)); + } + + { U32 const storeEnd = cur + 1; + U32 storeStart = storeEnd; + U32 seqPos = cur; + + DEBUGLOG(6, "start reverse traversal (last_pos:%u, cur:%u)", + last_pos, cur); (void)last_pos; + assert(storeEnd < ZSTD_OPT_NUM); + DEBUGLOG(6, "last sequence copied into pos=%u (llen=%u,mlen=%u,ofc=%u)", + storeEnd, lastSequence.litlen, lastSequence.mlen, lastSequence.off); + opt[storeEnd] = lastSequence; + while (seqPos > 0) { + U32 const backDist = ZSTD_totalLen(opt[seqPos]); + storeStart--; + DEBUGLOG(6, "sequence from rPos=%u copied into pos=%u (llen=%u,mlen=%u,ofc=%u)", + seqPos, storeStart, opt[seqPos].litlen, opt[seqPos].mlen, opt[seqPos].off); + opt[storeStart] = opt[seqPos]; + seqPos = (seqPos > backDist) ? seqPos - backDist : 0; + } + + /* save sequences */ + DEBUGLOG(6, "sending selected sequences into seqStore") + { U32 storePos; + for (storePos=storeStart; storePos <= storeEnd; storePos++) { + U32 const llen = opt[storePos].litlen; + U32 const mlen = opt[storePos].mlen; + U32 const offCode = opt[storePos].off; + U32 const advance = llen + mlen; + DEBUGLOG(6, "considering seq starting at %zi, llen=%u, mlen=%u", + anchor - istart, (unsigned)llen, (unsigned)mlen); + + if (mlen==0) { /* only literals => must be last "sequence", actually starting a new stream of sequences */ + assert(storePos == storeEnd); /* must be last sequence */ + ip = anchor + llen; /* last "sequence" is a bunch of literals => don't progress anchor */ + continue; /* will finish */ + } + + assert(anchor + llen <= iend); + ZSTD_updateStats(optStatePtr, llen, anchor, offCode, mlen); + ZSTD_storeSeq(seqStore, llen, anchor, iend, offCode, mlen-MINMATCH); + anchor += advance; + ip = anchor; + } } + ZSTD_setBasePrices(optStatePtr, optLevel); + } + } /* while (ip < ilimit) */ + + /* Return the last literals size */ + return (size_t)(iend - anchor); +} + + +size_t ZSTD_compressBlock_btopt( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + const void* src, size_t srcSize) +{ + DEBUGLOG(5, "ZSTD_compressBlock_btopt"); + return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 0 /*optLevel*/, ZSTD_noDict); +} + + +/* used in 2-pass strategy */ +static U32 ZSTD_upscaleStat(unsigned* table, U32 lastEltIndex, int bonus) +{ + U32 s, sum=0; + assert(ZSTD_FREQ_DIV+bonus >= 0); + for (s=0; slitSum = ZSTD_upscaleStat(optPtr->litFreq, MaxLit, 0); + optPtr->litLengthSum = ZSTD_upscaleStat(optPtr->litLengthFreq, MaxLL, 0); + optPtr->matchLengthSum = ZSTD_upscaleStat(optPtr->matchLengthFreq, MaxML, 0); + optPtr->offCodeSum = ZSTD_upscaleStat(optPtr->offCodeFreq, MaxOff, 0); +} + +/* ZSTD_initStats_ultra(): + * make a first compression pass, just to seed stats with more accurate starting values. + * only works on first block, with no dictionary and no ldm. + * this function cannot error, hence its contract must be respected. + */ +static void +ZSTD_initStats_ultra(ZSTD_matchState_t* ms, + seqStore_t* seqStore, + U32 rep[ZSTD_REP_NUM], + const void* src, size_t srcSize) +{ + U32 tmpRep[ZSTD_REP_NUM]; /* updated rep codes will sink here */ + memcpy(tmpRep, rep, sizeof(tmpRep)); + + DEBUGLOG(4, "ZSTD_initStats_ultra (srcSize=%zu)", srcSize); + assert(ms->opt.litLengthSum == 0); /* first block */ + assert(seqStore->sequences == seqStore->sequencesStart); /* no ldm */ + assert(ms->window.dictLimit == ms->window.lowLimit); /* no dictionary */ + assert(ms->window.dictLimit - ms->nextToUpdate <= 1); /* no prefix (note: intentional overflow, defined as 2-complement) */ + + ZSTD_compressBlock_opt_generic(ms, seqStore, tmpRep, src, srcSize, 2 /*optLevel*/, ZSTD_noDict); /* generate stats into ms->opt*/ + + /* invalidate first scan from history */ + ZSTD_resetSeqStore(seqStore); + ms->window.base -= srcSize; + ms->window.dictLimit += (U32)srcSize; + ms->window.lowLimit = ms->window.dictLimit; + ms->nextToUpdate = ms->window.dictLimit; + + /* re-inforce weight of collected statistics */ + ZSTD_upscaleStats(&ms->opt); +} + +size_t ZSTD_compressBlock_btultra( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + const void* src, size_t srcSize) +{ + DEBUGLOG(5, "ZSTD_compressBlock_btultra (srcSize=%zu)", srcSize); + return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 2 /*optLevel*/, ZSTD_noDict); +} + +size_t ZSTD_compressBlock_btultra2( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + const void* src, size_t srcSize) +{ + U32 const current = (U32)((const BYTE*)src - ms->window.base); + DEBUGLOG(5, "ZSTD_compressBlock_btultra2 (srcSize=%zu)", srcSize); + + /* 2-pass strategy: + * this strategy makes a first pass over first block to collect statistics + * and seed next round's statistics with it. + * After 1st pass, function forgets everything, and starts a new block. + * Consequently, this can only work if no data has been previously loaded in tables, + * aka, no dictionary, no prefix, no ldm preprocessing. + * The compression ratio gain is generally small (~0.5% on first block), + * the cost is 2x cpu time on first block. */ + assert(srcSize <= ZSTD_BLOCKSIZE_MAX); + if ( (ms->opt.litLengthSum==0) /* first block */ + && (seqStore->sequences == seqStore->sequencesStart) /* no ldm */ + && (ms->window.dictLimit == ms->window.lowLimit) /* no dictionary */ + && (current == ms->window.dictLimit) /* start of frame, nothing already loaded nor skipped */ + && (srcSize > ZSTD_PREDEF_THRESHOLD) + ) { + ZSTD_initStats_ultra(ms, seqStore, rep, src, srcSize); + } + + return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 2 /*optLevel*/, ZSTD_noDict); +} + +size_t ZSTD_compressBlock_btopt_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + const void* src, size_t srcSize) +{ + return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 0 /*optLevel*/, ZSTD_dictMatchState); +} + +size_t ZSTD_compressBlock_btultra_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + const void* src, size_t srcSize) +{ + return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 2 /*optLevel*/, ZSTD_dictMatchState); +} + +size_t ZSTD_compressBlock_btopt_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + const void* src, size_t srcSize) +{ + return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 0 /*optLevel*/, ZSTD_extDict); +} + +size_t ZSTD_compressBlock_btultra_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + const void* src, size_t srcSize) +{ + return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 2 /*optLevel*/, ZSTD_extDict); +} + +/* note : no btultra2 variant for extDict nor dictMatchState, + * because btultra2 is not meant to work with dictionaries + * and is only specific for the first block (no prefix) */ diff --git a/module/zstd/lib/compress/zstd_opt.h b/module/zstd/lib/compress/zstd_opt.h new file mode 100644 index 000000000000..9aba8a9018c5 --- /dev/null +++ b/module/zstd/lib/compress/zstd_opt.h @@ -0,0 +1,56 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +#ifndef ZSTD_OPT_H +#define ZSTD_OPT_H + +#if defined (__cplusplus) +extern "C" { +#endif + +#include "zstd_compress_internal.h" + +/* used in ZSTD_loadDictionaryContent() */ +void ZSTD_updateTree(ZSTD_matchState_t* ms, const BYTE* ip, const BYTE* iend); + +size_t ZSTD_compressBlock_btopt( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_btultra( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_btultra2( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); + + +size_t ZSTD_compressBlock_btopt_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_btultra_dictMatchState( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); + +size_t ZSTD_compressBlock_btopt_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); +size_t ZSTD_compressBlock_btultra_extDict( + ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], + void const* src, size_t srcSize); + + /* note : no btultra2 variant for extDict nor dictMatchState, + * because btultra2 is not meant to work with dictionaries + * and is only specific for the first block (no prefix) */ + +#if defined (__cplusplus) +} +#endif + +#endif /* ZSTD_OPT_H */ diff --git a/module/zstd/lib/decompress/huf_decompress.c b/module/zstd/lib/decompress/huf_decompress.c new file mode 100644 index 000000000000..68293a13096a --- /dev/null +++ b/module/zstd/lib/decompress/huf_decompress.c @@ -0,0 +1,1248 @@ +/* ****************************************************************** + * huff0 huffman decoder, + * part of Finite State Entropy library + * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. + * + * You can contact the author at : + * - FSE+HUF source repository : https://github.com/Cyan4973/FiniteStateEntropy + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. +****************************************************************** */ + +/* ************************************************************** +* Dependencies +****************************************************************/ +#include /* memcpy, memset */ +#include "../common/compiler.h" +#include "../common/bitstream.h" /* BIT_* */ +#include "../common/fse.h" /* to compress headers */ +#define HUF_STATIC_LINKING_ONLY +#include "../common/huf.h" +#include "../common/error_private.h" + +/* ************************************************************** +* Macros +****************************************************************/ + +/* These two optional macros force the use one way or another of the two + * Huffman decompression implementations. You can't force in both directions + * at the same time. + */ +#if defined(HUF_FORCE_DECOMPRESS_X1) && \ + defined(HUF_FORCE_DECOMPRESS_X2) +#error "Cannot force the use of the X1 and X2 decoders at the same time!" +#endif + + +/* ************************************************************** +* Error Management +****************************************************************/ +#define HUF_isError ERR_isError + + +/* ************************************************************** +* Byte alignment for workSpace management +****************************************************************/ +#define HUF_ALIGN(x, a) HUF_ALIGN_MASK((x), (a) - 1) +#define HUF_ALIGN_MASK(x, mask) (((x) + (mask)) & ~(mask)) + + +/* ************************************************************** +* BMI2 Variant Wrappers +****************************************************************/ +#if DYNAMIC_BMI2 + +#define HUF_DGEN(fn) \ + \ + static size_t fn##_default( \ + void* dst, size_t dstSize, \ + const void* cSrc, size_t cSrcSize, \ + const HUF_DTable* DTable) \ + { \ + return fn##_body(dst, dstSize, cSrc, cSrcSize, DTable); \ + } \ + \ + static TARGET_ATTRIBUTE("bmi2") size_t fn##_bmi2( \ + void* dst, size_t dstSize, \ + const void* cSrc, size_t cSrcSize, \ + const HUF_DTable* DTable) \ + { \ + return fn##_body(dst, dstSize, cSrc, cSrcSize, DTable); \ + } \ + \ + static size_t fn(void* dst, size_t dstSize, void const* cSrc, \ + size_t cSrcSize, HUF_DTable const* DTable, int bmi2) \ + { \ + if (bmi2) { \ + return fn##_bmi2(dst, dstSize, cSrc, cSrcSize, DTable); \ + } \ + return fn##_default(dst, dstSize, cSrc, cSrcSize, DTable); \ + } + +#else + +#define HUF_DGEN(fn) \ + static size_t fn(void* dst, size_t dstSize, void const* cSrc, \ + size_t cSrcSize, HUF_DTable const* DTable, int bmi2) \ + { \ + (void)bmi2; \ + return fn##_body(dst, dstSize, cSrc, cSrcSize, DTable); \ + } + +#endif + + +/*-***************************/ +/* generic DTableDesc */ +/*-***************************/ +typedef struct { BYTE maxTableLog; BYTE tableType; BYTE tableLog; BYTE reserved; } DTableDesc; + +static DTableDesc HUF_getDTableDesc(const HUF_DTable* table) +{ + DTableDesc dtd; + memcpy(&dtd, table, sizeof(dtd)); + return dtd; +} + + +#ifndef HUF_FORCE_DECOMPRESS_X2 + +/*-***************************/ +/* single-symbol decoding */ +/*-***************************/ +typedef struct { BYTE byte; BYTE nbBits; } HUF_DEltX1; /* single-symbol decoding */ + +size_t HUF_readDTableX1_wksp(HUF_DTable* DTable, const void* src, size_t srcSize, void* workSpace, size_t wkspSize) +{ + U32 tableLog = 0; + U32 nbSymbols = 0; + size_t iSize; + void* const dtPtr = DTable + 1; + HUF_DEltX1* const dt = (HUF_DEltX1*)dtPtr; + + U32* rankVal; + BYTE* huffWeight; + size_t spaceUsed32 = 0; + + rankVal = (U32 *)workSpace + spaceUsed32; + spaceUsed32 += HUF_TABLELOG_ABSOLUTEMAX + 1; + huffWeight = (BYTE *)((U32 *)workSpace + spaceUsed32); + spaceUsed32 += HUF_ALIGN(HUF_SYMBOLVALUE_MAX + 1, sizeof(U32)) >> 2; + + if ((spaceUsed32 << 2) > wkspSize) return ERROR(tableLog_tooLarge); + + DEBUG_STATIC_ASSERT(sizeof(DTableDesc) == sizeof(HUF_DTable)); + /* memset(huffWeight, 0, sizeof(huffWeight)); */ /* is not necessary, even though some analyzer complain ... */ + + iSize = HUF_readStats(huffWeight, HUF_SYMBOLVALUE_MAX + 1, rankVal, &nbSymbols, &tableLog, src, srcSize); + if (HUF_isError(iSize)) return iSize; + + /* Table header */ + { DTableDesc dtd = HUF_getDTableDesc(DTable); + if (tableLog > (U32)(dtd.maxTableLog+1)) return ERROR(tableLog_tooLarge); /* DTable too small, Huffman tree cannot fit in */ + dtd.tableType = 0; + dtd.tableLog = (BYTE)tableLog; + memcpy(DTable, &dtd, sizeof(dtd)); + } + + /* Calculate starting value for each rank */ + { U32 n, nextRankStart = 0; + for (n=1; n> 1; + size_t const uStart = rankVal[w]; + size_t const uEnd = uStart + length; + size_t u; + HUF_DEltX1 D; + D.byte = (BYTE)n; + D.nbBits = (BYTE)(tableLog + 1 - w); + rankVal[w] = (U32)uEnd; + if (length < 4) { + /* Use length in the loop bound so the compiler knows it is short. */ + for (u = 0; u < length; ++u) + dt[uStart + u] = D; + } else { + /* Unroll the loop 4 times, we know it is a power of 2. */ + for (u = uStart; u < uEnd; u += 4) { + dt[u + 0] = D; + dt[u + 1] = D; + dt[u + 2] = D; + dt[u + 3] = D; + } } } } + return iSize; +} + +size_t HUF_readDTableX1(HUF_DTable* DTable, const void* src, size_t srcSize) +{ + U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; + return HUF_readDTableX1_wksp(DTable, src, srcSize, + workSpace, sizeof(workSpace)); +} + +FORCE_INLINE_TEMPLATE BYTE +HUF_decodeSymbolX1(BIT_DStream_t* Dstream, const HUF_DEltX1* dt, const U32 dtLog) +{ + size_t const val = BIT_lookBitsFast(Dstream, dtLog); /* note : dtLog >= 1 */ + BYTE const c = dt[val].byte; + BIT_skipBits(Dstream, dt[val].nbBits); + return c; +} + +#define HUF_DECODE_SYMBOLX1_0(ptr, DStreamPtr) \ + *ptr++ = HUF_decodeSymbolX1(DStreamPtr, dt, dtLog) + +#define HUF_DECODE_SYMBOLX1_1(ptr, DStreamPtr) \ + if (MEM_64bits() || (HUF_TABLELOG_MAX<=12)) \ + HUF_DECODE_SYMBOLX1_0(ptr, DStreamPtr) + +#define HUF_DECODE_SYMBOLX1_2(ptr, DStreamPtr) \ + if (MEM_64bits()) \ + HUF_DECODE_SYMBOLX1_0(ptr, DStreamPtr) + +HINT_INLINE size_t +HUF_decodeStreamX1(BYTE* p, BIT_DStream_t* const bitDPtr, BYTE* const pEnd, const HUF_DEltX1* const dt, const U32 dtLog) +{ + BYTE* const pStart = p; + + /* up to 4 symbols at a time */ + while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) & (p < pEnd-3)) { + HUF_DECODE_SYMBOLX1_2(p, bitDPtr); + HUF_DECODE_SYMBOLX1_1(p, bitDPtr); + HUF_DECODE_SYMBOLX1_2(p, bitDPtr); + HUF_DECODE_SYMBOLX1_0(p, bitDPtr); + } + + /* [0-3] symbols remaining */ + if (MEM_32bits()) + while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) & (p < pEnd)) + HUF_DECODE_SYMBOLX1_0(p, bitDPtr); + + /* no more data to retrieve from bitstream, no need to reload */ + while (p < pEnd) + HUF_DECODE_SYMBOLX1_0(p, bitDPtr); + + return pEnd-pStart; +} + +FORCE_INLINE_TEMPLATE size_t +HUF_decompress1X1_usingDTable_internal_body( + void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize, + const HUF_DTable* DTable) +{ + BYTE* op = (BYTE*)dst; + BYTE* const oend = op + dstSize; + const void* dtPtr = DTable + 1; + const HUF_DEltX1* const dt = (const HUF_DEltX1*)dtPtr; + BIT_DStream_t bitD; + DTableDesc const dtd = HUF_getDTableDesc(DTable); + U32 const dtLog = dtd.tableLog; + + CHECK_F( BIT_initDStream(&bitD, cSrc, cSrcSize) ); + + HUF_decodeStreamX1(op, &bitD, oend, dt, dtLog); + + if (!BIT_endOfDStream(&bitD)) return ERROR(corruption_detected); + + return dstSize; +} + +FORCE_INLINE_TEMPLATE size_t +HUF_decompress4X1_usingDTable_internal_body( + void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize, + const HUF_DTable* DTable) +{ + /* Check */ + if (cSrcSize < 10) return ERROR(corruption_detected); /* strict minimum : jump table + 1 byte per stream */ + + { const BYTE* const istart = (const BYTE*) cSrc; + BYTE* const ostart = (BYTE*) dst; + BYTE* const oend = ostart + dstSize; + BYTE* const olimit = oend - 3; + const void* const dtPtr = DTable + 1; + const HUF_DEltX1* const dt = (const HUF_DEltX1*)dtPtr; + + /* Init */ + BIT_DStream_t bitD1; + BIT_DStream_t bitD2; + BIT_DStream_t bitD3; + BIT_DStream_t bitD4; + size_t const length1 = MEM_readLE16(istart); + size_t const length2 = MEM_readLE16(istart+2); + size_t const length3 = MEM_readLE16(istart+4); + size_t const length4 = cSrcSize - (length1 + length2 + length3 + 6); + const BYTE* const istart1 = istart + 6; /* jumpTable */ + const BYTE* const istart2 = istart1 + length1; + const BYTE* const istart3 = istart2 + length2; + const BYTE* const istart4 = istart3 + length3; + const size_t segmentSize = (dstSize+3) / 4; + BYTE* const opStart2 = ostart + segmentSize; + BYTE* const opStart3 = opStart2 + segmentSize; + BYTE* const opStart4 = opStart3 + segmentSize; + BYTE* op1 = ostart; + BYTE* op2 = opStart2; + BYTE* op3 = opStart3; + BYTE* op4 = opStart4; + DTableDesc const dtd = HUF_getDTableDesc(DTable); + U32 const dtLog = dtd.tableLog; + U32 endSignal = 1; + + if (length4 > cSrcSize) return ERROR(corruption_detected); /* overflow */ + CHECK_F( BIT_initDStream(&bitD1, istart1, length1) ); + CHECK_F( BIT_initDStream(&bitD2, istart2, length2) ); + CHECK_F( BIT_initDStream(&bitD3, istart3, length3) ); + CHECK_F( BIT_initDStream(&bitD4, istart4, length4) ); + + /* up to 16 symbols per loop (4 symbols per stream) in 64-bit mode */ + for ( ; (endSignal) & (op4 < olimit) ; ) { + HUF_DECODE_SYMBOLX1_2(op1, &bitD1); + HUF_DECODE_SYMBOLX1_2(op2, &bitD2); + HUF_DECODE_SYMBOLX1_2(op3, &bitD3); + HUF_DECODE_SYMBOLX1_2(op4, &bitD4); + HUF_DECODE_SYMBOLX1_1(op1, &bitD1); + HUF_DECODE_SYMBOLX1_1(op2, &bitD2); + HUF_DECODE_SYMBOLX1_1(op3, &bitD3); + HUF_DECODE_SYMBOLX1_1(op4, &bitD4); + HUF_DECODE_SYMBOLX1_2(op1, &bitD1); + HUF_DECODE_SYMBOLX1_2(op2, &bitD2); + HUF_DECODE_SYMBOLX1_2(op3, &bitD3); + HUF_DECODE_SYMBOLX1_2(op4, &bitD4); + HUF_DECODE_SYMBOLX1_0(op1, &bitD1); + HUF_DECODE_SYMBOLX1_0(op2, &bitD2); + HUF_DECODE_SYMBOLX1_0(op3, &bitD3); + HUF_DECODE_SYMBOLX1_0(op4, &bitD4); + endSignal &= BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished; + endSignal &= BIT_reloadDStreamFast(&bitD2) == BIT_DStream_unfinished; + endSignal &= BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished; + endSignal &= BIT_reloadDStreamFast(&bitD4) == BIT_DStream_unfinished; + } + + /* check corruption */ + /* note : should not be necessary : op# advance in lock step, and we control op4. + * but curiously, binary generated by gcc 7.2 & 7.3 with -mbmi2 runs faster when >=1 test is present */ + if (op1 > opStart2) return ERROR(corruption_detected); + if (op2 > opStart3) return ERROR(corruption_detected); + if (op3 > opStart4) return ERROR(corruption_detected); + /* note : op4 supposed already verified within main loop */ + + /* finish bitStreams one by one */ + HUF_decodeStreamX1(op1, &bitD1, opStart2, dt, dtLog); + HUF_decodeStreamX1(op2, &bitD2, opStart3, dt, dtLog); + HUF_decodeStreamX1(op3, &bitD3, opStart4, dt, dtLog); + HUF_decodeStreamX1(op4, &bitD4, oend, dt, dtLog); + + /* check */ + { U32 const endCheck = BIT_endOfDStream(&bitD1) & BIT_endOfDStream(&bitD2) & BIT_endOfDStream(&bitD3) & BIT_endOfDStream(&bitD4); + if (!endCheck) return ERROR(corruption_detected); } + + /* decoded size */ + return dstSize; + } +} + + +typedef size_t (*HUF_decompress_usingDTable_t)(void *dst, size_t dstSize, + const void *cSrc, + size_t cSrcSize, + const HUF_DTable *DTable); + +HUF_DGEN(HUF_decompress1X1_usingDTable_internal) +HUF_DGEN(HUF_decompress4X1_usingDTable_internal) + + + +size_t HUF_decompress1X1_usingDTable( + void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize, + const HUF_DTable* DTable) +{ + DTableDesc dtd = HUF_getDTableDesc(DTable); + if (dtd.tableType != 0) return ERROR(GENERIC); + return HUF_decompress1X1_usingDTable_internal(dst, dstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); +} + +size_t HUF_decompress1X1_DCtx_wksp(HUF_DTable* DCtx, void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize, + void* workSpace, size_t wkspSize) +{ + const BYTE* ip = (const BYTE*) cSrc; + + size_t const hSize = HUF_readDTableX1_wksp(DCtx, cSrc, cSrcSize, workSpace, wkspSize); + if (HUF_isError(hSize)) return hSize; + if (hSize >= cSrcSize) return ERROR(srcSize_wrong); + ip += hSize; cSrcSize -= hSize; + + return HUF_decompress1X1_usingDTable_internal(dst, dstSize, ip, cSrcSize, DCtx, /* bmi2 */ 0); +} + + +size_t HUF_decompress1X1_DCtx(HUF_DTable* DCtx, void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize) +{ + U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; + return HUF_decompress1X1_DCtx_wksp(DCtx, dst, dstSize, cSrc, cSrcSize, + workSpace, sizeof(workSpace)); +} + +size_t HUF_decompress1X1 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) +{ + HUF_CREATE_STATIC_DTABLEX1(DTable, HUF_TABLELOG_MAX); + return HUF_decompress1X1_DCtx (DTable, dst, dstSize, cSrc, cSrcSize); +} + +size_t HUF_decompress4X1_usingDTable( + void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize, + const HUF_DTable* DTable) +{ + DTableDesc dtd = HUF_getDTableDesc(DTable); + if (dtd.tableType != 0) return ERROR(GENERIC); + return HUF_decompress4X1_usingDTable_internal(dst, dstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); +} + +static size_t HUF_decompress4X1_DCtx_wksp_bmi2(HUF_DTable* dctx, void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize, + void* workSpace, size_t wkspSize, int bmi2) +{ + const BYTE* ip = (const BYTE*) cSrc; + + size_t const hSize = HUF_readDTableX1_wksp (dctx, cSrc, cSrcSize, + workSpace, wkspSize); + if (HUF_isError(hSize)) return hSize; + if (hSize >= cSrcSize) return ERROR(srcSize_wrong); + ip += hSize; cSrcSize -= hSize; + + return HUF_decompress4X1_usingDTable_internal(dst, dstSize, ip, cSrcSize, dctx, bmi2); +} + +size_t HUF_decompress4X1_DCtx_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize, + void* workSpace, size_t wkspSize) +{ + return HUF_decompress4X1_DCtx_wksp_bmi2(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize, 0); +} + + +size_t HUF_decompress4X1_DCtx (HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) +{ + U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; + return HUF_decompress4X1_DCtx_wksp(dctx, dst, dstSize, cSrc, cSrcSize, + workSpace, sizeof(workSpace)); +} +size_t HUF_decompress4X1 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) +{ + HUF_CREATE_STATIC_DTABLEX1(DTable, HUF_TABLELOG_MAX); + return HUF_decompress4X1_DCtx(DTable, dst, dstSize, cSrc, cSrcSize); +} + +#endif /* HUF_FORCE_DECOMPRESS_X2 */ + + +#ifndef HUF_FORCE_DECOMPRESS_X1 + +/* *************************/ +/* double-symbols decoding */ +/* *************************/ + +typedef struct { U16 sequence; BYTE nbBits; BYTE length; } HUF_DEltX2; /* double-symbols decoding */ +typedef struct { BYTE symbol; BYTE weight; } sortedSymbol_t; +typedef U32 rankValCol_t[HUF_TABLELOG_MAX + 1]; +typedef rankValCol_t rankVal_t[HUF_TABLELOG_MAX]; + + +/* HUF_fillDTableX2Level2() : + * `rankValOrigin` must be a table of at least (HUF_TABLELOG_MAX + 1) U32 */ +static void HUF_fillDTableX2Level2(HUF_DEltX2* DTable, U32 sizeLog, const U32 consumed, + const U32* rankValOrigin, const int minWeight, + const sortedSymbol_t* sortedSymbols, const U32 sortedListSize, + U32 nbBitsBaseline, U16 baseSeq) +{ + HUF_DEltX2 DElt; + U32 rankVal[HUF_TABLELOG_MAX + 1]; + + /* get pre-calculated rankVal */ + memcpy(rankVal, rankValOrigin, sizeof(rankVal)); + + /* fill skipped values */ + if (minWeight>1) { + U32 i, skipSize = rankVal[minWeight]; + MEM_writeLE16(&(DElt.sequence), baseSeq); + DElt.nbBits = (BYTE)(consumed); + DElt.length = 1; + for (i = 0; i < skipSize; i++) + DTable[i] = DElt; + } + + /* fill DTable */ + { U32 s; for (s=0; s= 1 */ + + rankVal[weight] += length; + } } +} + + +static void HUF_fillDTableX2(HUF_DEltX2* DTable, const U32 targetLog, + const sortedSymbol_t* sortedList, const U32 sortedListSize, + const U32* rankStart, rankVal_t rankValOrigin, const U32 maxWeight, + const U32 nbBitsBaseline) +{ + U32 rankVal[HUF_TABLELOG_MAX + 1]; + const int scaleLog = nbBitsBaseline - targetLog; /* note : targetLog >= srcLog, hence scaleLog <= 1 */ + const U32 minBits = nbBitsBaseline - maxWeight; + U32 s; + + memcpy(rankVal, rankValOrigin, sizeof(rankVal)); + + /* fill DTable */ + for (s=0; s= minBits) { /* enough room for a second symbol */ + U32 sortedRank; + int minWeight = nbBits + scaleLog; + if (minWeight < 1) minWeight = 1; + sortedRank = rankStart[minWeight]; + HUF_fillDTableX2Level2(DTable+start, targetLog-nbBits, nbBits, + rankValOrigin[nbBits], minWeight, + sortedList+sortedRank, sortedListSize-sortedRank, + nbBitsBaseline, symbol); + } else { + HUF_DEltX2 DElt; + MEM_writeLE16(&(DElt.sequence), symbol); + DElt.nbBits = (BYTE)(nbBits); + DElt.length = 1; + { U32 const end = start + length; + U32 u; + for (u = start; u < end; u++) DTable[u] = DElt; + } } + rankVal[weight] += length; + } +} + +size_t HUF_readDTableX2_wksp(HUF_DTable* DTable, + const void* src, size_t srcSize, + void* workSpace, size_t wkspSize) +{ + U32 tableLog, maxW, sizeOfSort, nbSymbols; + DTableDesc dtd = HUF_getDTableDesc(DTable); + U32 const maxTableLog = dtd.maxTableLog; + size_t iSize; + void* dtPtr = DTable+1; /* force compiler to avoid strict-aliasing */ + HUF_DEltX2* const dt = (HUF_DEltX2*)dtPtr; + U32 *rankStart; + + rankValCol_t* rankVal; + U32* rankStats; + U32* rankStart0; + sortedSymbol_t* sortedSymbol; + BYTE* weightList; + size_t spaceUsed32 = 0; + + rankVal = (rankValCol_t *)((U32 *)workSpace + spaceUsed32); + spaceUsed32 += (sizeof(rankValCol_t) * HUF_TABLELOG_MAX) >> 2; + rankStats = (U32 *)workSpace + spaceUsed32; + spaceUsed32 += HUF_TABLELOG_MAX + 1; + rankStart0 = (U32 *)workSpace + spaceUsed32; + spaceUsed32 += HUF_TABLELOG_MAX + 2; + sortedSymbol = (sortedSymbol_t *)workSpace + (spaceUsed32 * sizeof(U32)) / sizeof(sortedSymbol_t); + spaceUsed32 += HUF_ALIGN(sizeof(sortedSymbol_t) * (HUF_SYMBOLVALUE_MAX + 1), sizeof(U32)) >> 2; + weightList = (BYTE *)((U32 *)workSpace + spaceUsed32); + spaceUsed32 += HUF_ALIGN(HUF_SYMBOLVALUE_MAX + 1, sizeof(U32)) >> 2; + + if ((spaceUsed32 << 2) > wkspSize) return ERROR(tableLog_tooLarge); + + rankStart = rankStart0 + 1; + memset(rankStats, 0, sizeof(U32) * (2 * HUF_TABLELOG_MAX + 2 + 1)); + + DEBUG_STATIC_ASSERT(sizeof(HUF_DEltX2) == sizeof(HUF_DTable)); /* if compiler fails here, assertion is wrong */ + if (maxTableLog > HUF_TABLELOG_MAX) return ERROR(tableLog_tooLarge); + /* memset(weightList, 0, sizeof(weightList)); */ /* is not necessary, even though some analyzer complain ... */ + + iSize = HUF_readStats(weightList, HUF_SYMBOLVALUE_MAX + 1, rankStats, &nbSymbols, &tableLog, src, srcSize); + if (HUF_isError(iSize)) return iSize; + + /* check result */ + if (tableLog > maxTableLog) return ERROR(tableLog_tooLarge); /* DTable can't fit code depth */ + + /* find maxWeight */ + for (maxW = tableLog; rankStats[maxW]==0; maxW--) {} /* necessarily finds a solution before 0 */ + + /* Get start index of each weight */ + { U32 w, nextRankStart = 0; + for (w=1; w> consumed; + } } } } + + HUF_fillDTableX2(dt, maxTableLog, + sortedSymbol, sizeOfSort, + rankStart0, rankVal, maxW, + tableLog+1); + + dtd.tableLog = (BYTE)maxTableLog; + dtd.tableType = 1; + memcpy(DTable, &dtd, sizeof(dtd)); + return iSize; +} + +size_t HUF_readDTableX2(HUF_DTable* DTable, const void* src, size_t srcSize) +{ + U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; + return HUF_readDTableX2_wksp(DTable, src, srcSize, + workSpace, sizeof(workSpace)); +} + + +FORCE_INLINE_TEMPLATE U32 +HUF_decodeSymbolX2(void* op, BIT_DStream_t* DStream, const HUF_DEltX2* dt, const U32 dtLog) +{ + size_t const val = BIT_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */ + memcpy(op, dt+val, 2); + BIT_skipBits(DStream, dt[val].nbBits); + return dt[val].length; +} + +FORCE_INLINE_TEMPLATE U32 +HUF_decodeLastSymbolX2(void* op, BIT_DStream_t* DStream, const HUF_DEltX2* dt, const U32 dtLog) +{ + size_t const val = BIT_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */ + memcpy(op, dt+val, 1); + if (dt[val].length==1) BIT_skipBits(DStream, dt[val].nbBits); + else { + if (DStream->bitsConsumed < (sizeof(DStream->bitContainer)*8)) { + BIT_skipBits(DStream, dt[val].nbBits); + if (DStream->bitsConsumed > (sizeof(DStream->bitContainer)*8)) + /* ugly hack; works only because it's the last symbol. Note : can't easily extract nbBits from just this symbol */ + DStream->bitsConsumed = (sizeof(DStream->bitContainer)*8); + } } + return 1; +} + +#define HUF_DECODE_SYMBOLX2_0(ptr, DStreamPtr) \ + ptr += HUF_decodeSymbolX2(ptr, DStreamPtr, dt, dtLog) + +#define HUF_DECODE_SYMBOLX2_1(ptr, DStreamPtr) \ + if (MEM_64bits() || (HUF_TABLELOG_MAX<=12)) \ + ptr += HUF_decodeSymbolX2(ptr, DStreamPtr, dt, dtLog) + +#define HUF_DECODE_SYMBOLX2_2(ptr, DStreamPtr) \ + if (MEM_64bits()) \ + ptr += HUF_decodeSymbolX2(ptr, DStreamPtr, dt, dtLog) + +HINT_INLINE size_t +HUF_decodeStreamX2(BYTE* p, BIT_DStream_t* bitDPtr, BYTE* const pEnd, + const HUF_DEltX2* const dt, const U32 dtLog) +{ + BYTE* const pStart = p; + + /* up to 8 symbols at a time */ + while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) & (p < pEnd-(sizeof(bitDPtr->bitContainer)-1))) { + HUF_DECODE_SYMBOLX2_2(p, bitDPtr); + HUF_DECODE_SYMBOLX2_1(p, bitDPtr); + HUF_DECODE_SYMBOLX2_2(p, bitDPtr); + HUF_DECODE_SYMBOLX2_0(p, bitDPtr); + } + + /* closer to end : up to 2 symbols at a time */ + while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) & (p <= pEnd-2)) + HUF_DECODE_SYMBOLX2_0(p, bitDPtr); + + while (p <= pEnd-2) + HUF_DECODE_SYMBOLX2_0(p, bitDPtr); /* no need to reload : reached the end of DStream */ + + if (p < pEnd) + p += HUF_decodeLastSymbolX2(p, bitDPtr, dt, dtLog); + + return p-pStart; +} + +FORCE_INLINE_TEMPLATE size_t +HUF_decompress1X2_usingDTable_internal_body( + void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize, + const HUF_DTable* DTable) +{ + BIT_DStream_t bitD; + + /* Init */ + CHECK_F( BIT_initDStream(&bitD, cSrc, cSrcSize) ); + + /* decode */ + { BYTE* const ostart = (BYTE*) dst; + BYTE* const oend = ostart + dstSize; + const void* const dtPtr = DTable+1; /* force compiler to not use strict-aliasing */ + const HUF_DEltX2* const dt = (const HUF_DEltX2*)dtPtr; + DTableDesc const dtd = HUF_getDTableDesc(DTable); + HUF_decodeStreamX2(ostart, &bitD, oend, dt, dtd.tableLog); + } + + /* check */ + if (!BIT_endOfDStream(&bitD)) return ERROR(corruption_detected); + + /* decoded size */ + return dstSize; +} + +FORCE_INLINE_TEMPLATE size_t +HUF_decompress4X2_usingDTable_internal_body( + void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize, + const HUF_DTable* DTable) +{ + if (cSrcSize < 10) return ERROR(corruption_detected); /* strict minimum : jump table + 1 byte per stream */ + + { const BYTE* const istart = (const BYTE*) cSrc; + BYTE* const ostart = (BYTE*) dst; + BYTE* const oend = ostart + dstSize; + BYTE* const olimit = oend - (sizeof(size_t)-1); + const void* const dtPtr = DTable+1; + const HUF_DEltX2* const dt = (const HUF_DEltX2*)dtPtr; + + /* Init */ + BIT_DStream_t bitD1; + BIT_DStream_t bitD2; + BIT_DStream_t bitD3; + BIT_DStream_t bitD4; + size_t const length1 = MEM_readLE16(istart); + size_t const length2 = MEM_readLE16(istart+2); + size_t const length3 = MEM_readLE16(istart+4); + size_t const length4 = cSrcSize - (length1 + length2 + length3 + 6); + const BYTE* const istart1 = istart + 6; /* jumpTable */ + const BYTE* const istart2 = istart1 + length1; + const BYTE* const istart3 = istart2 + length2; + const BYTE* const istart4 = istart3 + length3; + size_t const segmentSize = (dstSize+3) / 4; + BYTE* const opStart2 = ostart + segmentSize; + BYTE* const opStart3 = opStart2 + segmentSize; + BYTE* const opStart4 = opStart3 + segmentSize; + BYTE* op1 = ostart; + BYTE* op2 = opStart2; + BYTE* op3 = opStart3; + BYTE* op4 = opStart4; + U32 endSignal = 1; + DTableDesc const dtd = HUF_getDTableDesc(DTable); + U32 const dtLog = dtd.tableLog; + + if (length4 > cSrcSize) return ERROR(corruption_detected); /* overflow */ + CHECK_F( BIT_initDStream(&bitD1, istart1, length1) ); + CHECK_F( BIT_initDStream(&bitD2, istart2, length2) ); + CHECK_F( BIT_initDStream(&bitD3, istart3, length3) ); + CHECK_F( BIT_initDStream(&bitD4, istart4, length4) ); + + /* 16-32 symbols per loop (4-8 symbols per stream) */ + for ( ; (endSignal) & (op4 < olimit); ) { +#if defined(__clang__) && (defined(__x86_64__) || defined(__i386__)) + HUF_DECODE_SYMBOLX2_2(op1, &bitD1); + HUF_DECODE_SYMBOLX2_1(op1, &bitD1); + HUF_DECODE_SYMBOLX2_2(op1, &bitD1); + HUF_DECODE_SYMBOLX2_0(op1, &bitD1); + HUF_DECODE_SYMBOLX2_2(op2, &bitD2); + HUF_DECODE_SYMBOLX2_1(op2, &bitD2); + HUF_DECODE_SYMBOLX2_2(op2, &bitD2); + HUF_DECODE_SYMBOLX2_0(op2, &bitD2); + endSignal &= BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished; + endSignal &= BIT_reloadDStreamFast(&bitD2) == BIT_DStream_unfinished; + HUF_DECODE_SYMBOLX2_2(op3, &bitD3); + HUF_DECODE_SYMBOLX2_1(op3, &bitD3); + HUF_DECODE_SYMBOLX2_2(op3, &bitD3); + HUF_DECODE_SYMBOLX2_0(op3, &bitD3); + HUF_DECODE_SYMBOLX2_2(op4, &bitD4); + HUF_DECODE_SYMBOLX2_1(op4, &bitD4); + HUF_DECODE_SYMBOLX2_2(op4, &bitD4); + HUF_DECODE_SYMBOLX2_0(op4, &bitD4); + endSignal &= BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished; + endSignal &= BIT_reloadDStreamFast(&bitD4) == BIT_DStream_unfinished; +#else + HUF_DECODE_SYMBOLX2_2(op1, &bitD1); + HUF_DECODE_SYMBOLX2_2(op2, &bitD2); + HUF_DECODE_SYMBOLX2_2(op3, &bitD3); + HUF_DECODE_SYMBOLX2_2(op4, &bitD4); + HUF_DECODE_SYMBOLX2_1(op1, &bitD1); + HUF_DECODE_SYMBOLX2_1(op2, &bitD2); + HUF_DECODE_SYMBOLX2_1(op3, &bitD3); + HUF_DECODE_SYMBOLX2_1(op4, &bitD4); + HUF_DECODE_SYMBOLX2_2(op1, &bitD1); + HUF_DECODE_SYMBOLX2_2(op2, &bitD2); + HUF_DECODE_SYMBOLX2_2(op3, &bitD3); + HUF_DECODE_SYMBOLX2_2(op4, &bitD4); + HUF_DECODE_SYMBOLX2_0(op1, &bitD1); + HUF_DECODE_SYMBOLX2_0(op2, &bitD2); + HUF_DECODE_SYMBOLX2_0(op3, &bitD3); + HUF_DECODE_SYMBOLX2_0(op4, &bitD4); + endSignal = (U32)LIKELY( + (BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished) + & (BIT_reloadDStreamFast(&bitD2) == BIT_DStream_unfinished) + & (BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished) + & (BIT_reloadDStreamFast(&bitD4) == BIT_DStream_unfinished)); +#endif + } + + /* check corruption */ + if (op1 > opStart2) return ERROR(corruption_detected); + if (op2 > opStart3) return ERROR(corruption_detected); + if (op3 > opStart4) return ERROR(corruption_detected); + /* note : op4 already verified within main loop */ + + /* finish bitStreams one by one */ + HUF_decodeStreamX2(op1, &bitD1, opStart2, dt, dtLog); + HUF_decodeStreamX2(op2, &bitD2, opStart3, dt, dtLog); + HUF_decodeStreamX2(op3, &bitD3, opStart4, dt, dtLog); + HUF_decodeStreamX2(op4, &bitD4, oend, dt, dtLog); + + /* check */ + { U32 const endCheck = BIT_endOfDStream(&bitD1) & BIT_endOfDStream(&bitD2) & BIT_endOfDStream(&bitD3) & BIT_endOfDStream(&bitD4); + if (!endCheck) return ERROR(corruption_detected); } + + /* decoded size */ + return dstSize; + } +} + +HUF_DGEN(HUF_decompress1X2_usingDTable_internal) +HUF_DGEN(HUF_decompress4X2_usingDTable_internal) + +size_t HUF_decompress1X2_usingDTable( + void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize, + const HUF_DTable* DTable) +{ + DTableDesc dtd = HUF_getDTableDesc(DTable); + if (dtd.tableType != 1) return ERROR(GENERIC); + return HUF_decompress1X2_usingDTable_internal(dst, dstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); +} + +size_t HUF_decompress1X2_DCtx_wksp(HUF_DTable* DCtx, void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize, + void* workSpace, size_t wkspSize) +{ + const BYTE* ip = (const BYTE*) cSrc; + + size_t const hSize = HUF_readDTableX2_wksp(DCtx, cSrc, cSrcSize, + workSpace, wkspSize); + if (HUF_isError(hSize)) return hSize; + if (hSize >= cSrcSize) return ERROR(srcSize_wrong); + ip += hSize; cSrcSize -= hSize; + + return HUF_decompress1X2_usingDTable_internal(dst, dstSize, ip, cSrcSize, DCtx, /* bmi2 */ 0); +} + + +size_t HUF_decompress1X2_DCtx(HUF_DTable* DCtx, void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize) +{ + U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; + return HUF_decompress1X2_DCtx_wksp(DCtx, dst, dstSize, cSrc, cSrcSize, + workSpace, sizeof(workSpace)); +} + +size_t HUF_decompress1X2 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) +{ + HUF_CREATE_STATIC_DTABLEX2(DTable, HUF_TABLELOG_MAX); + return HUF_decompress1X2_DCtx(DTable, dst, dstSize, cSrc, cSrcSize); +} + +size_t HUF_decompress4X2_usingDTable( + void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize, + const HUF_DTable* DTable) +{ + DTableDesc dtd = HUF_getDTableDesc(DTable); + if (dtd.tableType != 1) return ERROR(GENERIC); + return HUF_decompress4X2_usingDTable_internal(dst, dstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); +} + +static size_t HUF_decompress4X2_DCtx_wksp_bmi2(HUF_DTable* dctx, void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize, + void* workSpace, size_t wkspSize, int bmi2) +{ + const BYTE* ip = (const BYTE*) cSrc; + + size_t hSize = HUF_readDTableX2_wksp(dctx, cSrc, cSrcSize, + workSpace, wkspSize); + if (HUF_isError(hSize)) return hSize; + if (hSize >= cSrcSize) return ERROR(srcSize_wrong); + ip += hSize; cSrcSize -= hSize; + + return HUF_decompress4X2_usingDTable_internal(dst, dstSize, ip, cSrcSize, dctx, bmi2); +} + +size_t HUF_decompress4X2_DCtx_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize, + void* workSpace, size_t wkspSize) +{ + return HUF_decompress4X2_DCtx_wksp_bmi2(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize, /* bmi2 */ 0); +} + + +size_t HUF_decompress4X2_DCtx(HUF_DTable* dctx, void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize) +{ + U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; + return HUF_decompress4X2_DCtx_wksp(dctx, dst, dstSize, cSrc, cSrcSize, + workSpace, sizeof(workSpace)); +} + +size_t HUF_decompress4X2 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) +{ + HUF_CREATE_STATIC_DTABLEX2(DTable, HUF_TABLELOG_MAX); + return HUF_decompress4X2_DCtx(DTable, dst, dstSize, cSrc, cSrcSize); +} + +#endif /* HUF_FORCE_DECOMPRESS_X1 */ + + +/* ***********************************/ +/* Universal decompression selectors */ +/* ***********************************/ + +size_t HUF_decompress1X_usingDTable(void* dst, size_t maxDstSize, + const void* cSrc, size_t cSrcSize, + const HUF_DTable* DTable) +{ + DTableDesc const dtd = HUF_getDTableDesc(DTable); +#if defined(HUF_FORCE_DECOMPRESS_X1) + (void)dtd; + assert(dtd.tableType == 0); + return HUF_decompress1X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); +#elif defined(HUF_FORCE_DECOMPRESS_X2) + (void)dtd; + assert(dtd.tableType == 1); + return HUF_decompress1X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); +#else + return dtd.tableType ? HUF_decompress1X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0) : + HUF_decompress1X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); +#endif +} + +size_t HUF_decompress4X_usingDTable(void* dst, size_t maxDstSize, + const void* cSrc, size_t cSrcSize, + const HUF_DTable* DTable) +{ + DTableDesc const dtd = HUF_getDTableDesc(DTable); +#if defined(HUF_FORCE_DECOMPRESS_X1) + (void)dtd; + assert(dtd.tableType == 0); + return HUF_decompress4X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); +#elif defined(HUF_FORCE_DECOMPRESS_X2) + (void)dtd; + assert(dtd.tableType == 1); + return HUF_decompress4X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); +#else + return dtd.tableType ? HUF_decompress4X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0) : + HUF_decompress4X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); +#endif +} + + +#if !defined(HUF_FORCE_DECOMPRESS_X1) && !defined(HUF_FORCE_DECOMPRESS_X2) +typedef struct { U32 tableTime; U32 decode256Time; } algo_time_t; +static const algo_time_t algoTime[16 /* Quantization */][3 /* single, double, quad */] = +{ + /* single, double, quad */ + {{0,0}, {1,1}, {2,2}}, /* Q==0 : impossible */ + {{0,0}, {1,1}, {2,2}}, /* Q==1 : impossible */ + {{ 38,130}, {1313, 74}, {2151, 38}}, /* Q == 2 : 12-18% */ + {{ 448,128}, {1353, 74}, {2238, 41}}, /* Q == 3 : 18-25% */ + {{ 556,128}, {1353, 74}, {2238, 47}}, /* Q == 4 : 25-32% */ + {{ 714,128}, {1418, 74}, {2436, 53}}, /* Q == 5 : 32-38% */ + {{ 883,128}, {1437, 74}, {2464, 61}}, /* Q == 6 : 38-44% */ + {{ 897,128}, {1515, 75}, {2622, 68}}, /* Q == 7 : 44-50% */ + {{ 926,128}, {1613, 75}, {2730, 75}}, /* Q == 8 : 50-56% */ + {{ 947,128}, {1729, 77}, {3359, 77}}, /* Q == 9 : 56-62% */ + {{1107,128}, {2083, 81}, {4006, 84}}, /* Q ==10 : 62-69% */ + {{1177,128}, {2379, 87}, {4785, 88}}, /* Q ==11 : 69-75% */ + {{1242,128}, {2415, 93}, {5155, 84}}, /* Q ==12 : 75-81% */ + {{1349,128}, {2644,106}, {5260,106}}, /* Q ==13 : 81-87% */ + {{1455,128}, {2422,124}, {4174,124}}, /* Q ==14 : 87-93% */ + {{ 722,128}, {1891,145}, {1936,146}}, /* Q ==15 : 93-99% */ +}; +#endif + +/** HUF_selectDecoder() : + * Tells which decoder is likely to decode faster, + * based on a set of pre-computed metrics. + * @return : 0==HUF_decompress4X1, 1==HUF_decompress4X2 . + * Assumption : 0 < dstSize <= 128 KB */ +U32 HUF_selectDecoder (size_t dstSize, size_t cSrcSize) +{ + assert(dstSize > 0); + assert(dstSize <= 128*1024); +#if defined(HUF_FORCE_DECOMPRESS_X1) + (void)dstSize; + (void)cSrcSize; + return 0; +#elif defined(HUF_FORCE_DECOMPRESS_X2) + (void)dstSize; + (void)cSrcSize; + return 1; +#else + /* decoder timing evaluation */ + { U32 const Q = (cSrcSize >= dstSize) ? 15 : (U32)(cSrcSize * 16 / dstSize); /* Q < 16 */ + U32 const D256 = (U32)(dstSize >> 8); + U32 const DTime0 = algoTime[Q][0].tableTime + (algoTime[Q][0].decode256Time * D256); + U32 DTime1 = algoTime[Q][1].tableTime + (algoTime[Q][1].decode256Time * D256); + DTime1 += DTime1 >> 3; /* advantage to algorithm using less memory, to reduce cache eviction */ + return DTime1 < DTime0; + } +#endif +} + + +typedef size_t (*decompressionAlgo)(void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); + +size_t HUF_decompress (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) +{ +#if !defined(HUF_FORCE_DECOMPRESS_X1) && !defined(HUF_FORCE_DECOMPRESS_X2) + static const decompressionAlgo decompress[2] = { HUF_decompress4X1, HUF_decompress4X2 }; +#endif + + /* validation checks */ + if (dstSize == 0) return ERROR(dstSize_tooSmall); + if (cSrcSize > dstSize) return ERROR(corruption_detected); /* invalid */ + if (cSrcSize == dstSize) { memcpy(dst, cSrc, dstSize); return dstSize; } /* not compressed */ + if (cSrcSize == 1) { memset(dst, *(const BYTE*)cSrc, dstSize); return dstSize; } /* RLE */ + + { U32 const algoNb = HUF_selectDecoder(dstSize, cSrcSize); +#if defined(HUF_FORCE_DECOMPRESS_X1) + (void)algoNb; + assert(algoNb == 0); + return HUF_decompress4X1(dst, dstSize, cSrc, cSrcSize); +#elif defined(HUF_FORCE_DECOMPRESS_X2) + (void)algoNb; + assert(algoNb == 1); + return HUF_decompress4X2(dst, dstSize, cSrc, cSrcSize); +#else + return decompress[algoNb](dst, dstSize, cSrc, cSrcSize); +#endif + } +} + +size_t HUF_decompress4X_DCtx (HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) +{ + /* validation checks */ + if (dstSize == 0) return ERROR(dstSize_tooSmall); + if (cSrcSize > dstSize) return ERROR(corruption_detected); /* invalid */ + if (cSrcSize == dstSize) { memcpy(dst, cSrc, dstSize); return dstSize; } /* not compressed */ + if (cSrcSize == 1) { memset(dst, *(const BYTE*)cSrc, dstSize); return dstSize; } /* RLE */ + + { U32 const algoNb = HUF_selectDecoder(dstSize, cSrcSize); +#if defined(HUF_FORCE_DECOMPRESS_X1) + (void)algoNb; + assert(algoNb == 0); + return HUF_decompress4X1_DCtx(dctx, dst, dstSize, cSrc, cSrcSize); +#elif defined(HUF_FORCE_DECOMPRESS_X2) + (void)algoNb; + assert(algoNb == 1); + return HUF_decompress4X2_DCtx(dctx, dst, dstSize, cSrc, cSrcSize); +#else + return algoNb ? HUF_decompress4X2_DCtx(dctx, dst, dstSize, cSrc, cSrcSize) : + HUF_decompress4X1_DCtx(dctx, dst, dstSize, cSrc, cSrcSize) ; +#endif + } +} + +size_t HUF_decompress4X_hufOnly(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) +{ + U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; + return HUF_decompress4X_hufOnly_wksp(dctx, dst, dstSize, cSrc, cSrcSize, + workSpace, sizeof(workSpace)); +} + + +size_t HUF_decompress4X_hufOnly_wksp(HUF_DTable* dctx, void* dst, + size_t dstSize, const void* cSrc, + size_t cSrcSize, void* workSpace, + size_t wkspSize) +{ + /* validation checks */ + if (dstSize == 0) return ERROR(dstSize_tooSmall); + if (cSrcSize == 0) return ERROR(corruption_detected); + + { U32 const algoNb = HUF_selectDecoder(dstSize, cSrcSize); +#if defined(HUF_FORCE_DECOMPRESS_X1) + (void)algoNb; + assert(algoNb == 0); + return HUF_decompress4X1_DCtx_wksp(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize); +#elif defined(HUF_FORCE_DECOMPRESS_X2) + (void)algoNb; + assert(algoNb == 1); + return HUF_decompress4X2_DCtx_wksp(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize); +#else + return algoNb ? HUF_decompress4X2_DCtx_wksp(dctx, dst, dstSize, cSrc, + cSrcSize, workSpace, wkspSize): + HUF_decompress4X1_DCtx_wksp(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize); +#endif + } +} + +size_t HUF_decompress1X_DCtx_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize, + void* workSpace, size_t wkspSize) +{ + /* validation checks */ + if (dstSize == 0) return ERROR(dstSize_tooSmall); + if (cSrcSize > dstSize) return ERROR(corruption_detected); /* invalid */ + if (cSrcSize == dstSize) { memcpy(dst, cSrc, dstSize); return dstSize; } /* not compressed */ + if (cSrcSize == 1) { memset(dst, *(const BYTE*)cSrc, dstSize); return dstSize; } /* RLE */ + + { U32 const algoNb = HUF_selectDecoder(dstSize, cSrcSize); +#if defined(HUF_FORCE_DECOMPRESS_X1) + (void)algoNb; + assert(algoNb == 0); + return HUF_decompress1X1_DCtx_wksp(dctx, dst, dstSize, cSrc, + cSrcSize, workSpace, wkspSize); +#elif defined(HUF_FORCE_DECOMPRESS_X2) + (void)algoNb; + assert(algoNb == 1); + return HUF_decompress1X2_DCtx_wksp(dctx, dst, dstSize, cSrc, + cSrcSize, workSpace, wkspSize); +#else + return algoNb ? HUF_decompress1X2_DCtx_wksp(dctx, dst, dstSize, cSrc, + cSrcSize, workSpace, wkspSize): + HUF_decompress1X1_DCtx_wksp(dctx, dst, dstSize, cSrc, + cSrcSize, workSpace, wkspSize); +#endif + } +} + +size_t HUF_decompress1X_DCtx(HUF_DTable* dctx, void* dst, size_t dstSize, + const void* cSrc, size_t cSrcSize) +{ + U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; + return HUF_decompress1X_DCtx_wksp(dctx, dst, dstSize, cSrc, cSrcSize, + workSpace, sizeof(workSpace)); +} + + +size_t HUF_decompress1X_usingDTable_bmi2(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable, int bmi2) +{ + DTableDesc const dtd = HUF_getDTableDesc(DTable); +#if defined(HUF_FORCE_DECOMPRESS_X1) + (void)dtd; + assert(dtd.tableType == 0); + return HUF_decompress1X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2); +#elif defined(HUF_FORCE_DECOMPRESS_X2) + (void)dtd; + assert(dtd.tableType == 1); + return HUF_decompress1X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2); +#else + return dtd.tableType ? HUF_decompress1X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2) : + HUF_decompress1X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2); +#endif +} + +#ifndef HUF_FORCE_DECOMPRESS_X2 +size_t HUF_decompress1X1_DCtx_wksp_bmi2(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize, int bmi2) +{ + const BYTE* ip = (const BYTE*) cSrc; + + size_t const hSize = HUF_readDTableX1_wksp(dctx, cSrc, cSrcSize, workSpace, wkspSize); + if (HUF_isError(hSize)) return hSize; + if (hSize >= cSrcSize) return ERROR(srcSize_wrong); + ip += hSize; cSrcSize -= hSize; + + return HUF_decompress1X1_usingDTable_internal(dst, dstSize, ip, cSrcSize, dctx, bmi2); +} +#endif + +size_t HUF_decompress4X_usingDTable_bmi2(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable, int bmi2) +{ + DTableDesc const dtd = HUF_getDTableDesc(DTable); +#if defined(HUF_FORCE_DECOMPRESS_X1) + (void)dtd; + assert(dtd.tableType == 0); + return HUF_decompress4X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2); +#elif defined(HUF_FORCE_DECOMPRESS_X2) + (void)dtd; + assert(dtd.tableType == 1); + return HUF_decompress4X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2); +#else + return dtd.tableType ? HUF_decompress4X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2) : + HUF_decompress4X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2); +#endif +} + +size_t HUF_decompress4X_hufOnly_wksp_bmi2(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize, int bmi2) +{ + /* validation checks */ + if (dstSize == 0) return ERROR(dstSize_tooSmall); + if (cSrcSize == 0) return ERROR(corruption_detected); + + { U32 const algoNb = HUF_selectDecoder(dstSize, cSrcSize); +#if defined(HUF_FORCE_DECOMPRESS_X1) + (void)algoNb; + assert(algoNb == 0); + return HUF_decompress4X1_DCtx_wksp_bmi2(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize, bmi2); +#elif defined(HUF_FORCE_DECOMPRESS_X2) + (void)algoNb; + assert(algoNb == 1); + return HUF_decompress4X2_DCtx_wksp_bmi2(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize, bmi2); +#else + return algoNb ? HUF_decompress4X2_DCtx_wksp_bmi2(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize, bmi2) : + HUF_decompress4X1_DCtx_wksp_bmi2(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize, bmi2); +#endif + } +} diff --git a/module/zstd/lib/decompress/zstd_ddict.c b/module/zstd/lib/decompress/zstd_ddict.c new file mode 100644 index 000000000000..c8cb8ecc9524 --- /dev/null +++ b/module/zstd/lib/decompress/zstd_ddict.c @@ -0,0 +1,244 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +/* zstd_ddict.c : + * concentrates all logic that needs to know the internals of ZSTD_DDict object */ + +/*-******************************************************* +* Dependencies +*********************************************************/ +#include /* memcpy, memmove, memset */ +#include "../common/cpu.h" /* bmi2 */ +#include "../common/mem.h" /* low level memory routines */ +#define FSE_STATIC_LINKING_ONLY +#include "../common/fse.h" +#define HUF_STATIC_LINKING_ONLY +#include "../common/huf.h" +#include "zstd_decompress_internal.h" +#include "zstd_ddict.h" + +#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT>=1) +# include "../legacy/zstd_legacy.h" +#endif + + + +/*-******************************************************* +* Types +*********************************************************/ +struct ZSTD_DDict_s { + void* dictBuffer; + const void* dictContent; + size_t dictSize; + ZSTD_entropyDTables_t entropy; + U32 dictID; + U32 entropyPresent; + ZSTD_customMem cMem; +}; /* typedef'd to ZSTD_DDict within "zstd.h" */ + +const void* ZSTD_DDict_dictContent(const ZSTD_DDict* ddict) +{ + assert(ddict != NULL); + return ddict->dictContent; +} + +size_t ZSTD_DDict_dictSize(const ZSTD_DDict* ddict) +{ + assert(ddict != NULL); + return ddict->dictSize; +} + +void ZSTD_copyDDictParameters(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict) +{ + DEBUGLOG(4, "ZSTD_copyDDictParameters"); + assert(dctx != NULL); + assert(ddict != NULL); + dctx->dictID = ddict->dictID; + dctx->prefixStart = ddict->dictContent; + dctx->virtualStart = ddict->dictContent; + dctx->dictEnd = (const BYTE*)ddict->dictContent + ddict->dictSize; + dctx->previousDstEnd = dctx->dictEnd; +#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION + dctx->dictContentBeginForFuzzing = dctx->prefixStart; + dctx->dictContentEndForFuzzing = dctx->previousDstEnd; +#endif + if (ddict->entropyPresent) { + dctx->litEntropy = 1; + dctx->fseEntropy = 1; + dctx->LLTptr = ddict->entropy.LLTable; + dctx->MLTptr = ddict->entropy.MLTable; + dctx->OFTptr = ddict->entropy.OFTable; + dctx->HUFptr = ddict->entropy.hufTable; + dctx->entropy.rep[0] = ddict->entropy.rep[0]; + dctx->entropy.rep[1] = ddict->entropy.rep[1]; + dctx->entropy.rep[2] = ddict->entropy.rep[2]; + } else { + dctx->litEntropy = 0; + dctx->fseEntropy = 0; + } +} + + +static size_t +ZSTD_loadEntropy_intoDDict(ZSTD_DDict* ddict, + ZSTD_dictContentType_e dictContentType) +{ + ddict->dictID = 0; + ddict->entropyPresent = 0; + if (dictContentType == ZSTD_dct_rawContent) return 0; + + if (ddict->dictSize < 8) { + if (dictContentType == ZSTD_dct_fullDict) + return ERROR(dictionary_corrupted); /* only accept specified dictionaries */ + return 0; /* pure content mode */ + } + { U32 const magic = MEM_readLE32(ddict->dictContent); + if (magic != ZSTD_MAGIC_DICTIONARY) { + if (dictContentType == ZSTD_dct_fullDict) + return ERROR(dictionary_corrupted); /* only accept specified dictionaries */ + return 0; /* pure content mode */ + } + } + ddict->dictID = MEM_readLE32((const char*)ddict->dictContent + ZSTD_FRAMEIDSIZE); + + /* load entropy tables */ + RETURN_ERROR_IF(ZSTD_isError(ZSTD_loadDEntropy( + &ddict->entropy, ddict->dictContent, ddict->dictSize)), + dictionary_corrupted, ""); + ddict->entropyPresent = 1; + return 0; +} + + +static size_t ZSTD_initDDict_internal(ZSTD_DDict* ddict, + const void* dict, size_t dictSize, + ZSTD_dictLoadMethod_e dictLoadMethod, + ZSTD_dictContentType_e dictContentType) +{ + if ((dictLoadMethod == ZSTD_dlm_byRef) || (!dict) || (!dictSize)) { + ddict->dictBuffer = NULL; + ddict->dictContent = dict; + if (!dict) dictSize = 0; + } else { + void* const internalBuffer = ZSTD_malloc(dictSize, ddict->cMem); + ddict->dictBuffer = internalBuffer; + ddict->dictContent = internalBuffer; + if (!internalBuffer) return ERROR(memory_allocation); + memcpy(internalBuffer, dict, dictSize); + } + ddict->dictSize = dictSize; + ddict->entropy.hufTable[0] = (HUF_DTable)((HufLog)*0x1000001); /* cover both little and big endian */ + + /* parse dictionary content */ + FORWARD_IF_ERROR( ZSTD_loadEntropy_intoDDict(ddict, dictContentType) , ""); + + return 0; +} + +ZSTD_DDict* ZSTD_createDDict_advanced(const void* dict, size_t dictSize, + ZSTD_dictLoadMethod_e dictLoadMethod, + ZSTD_dictContentType_e dictContentType, + ZSTD_customMem customMem) +{ + if (!customMem.customAlloc ^ !customMem.customFree) return NULL; + + { ZSTD_DDict* const ddict = (ZSTD_DDict*) ZSTD_malloc(sizeof(ZSTD_DDict), customMem); + if (ddict == NULL) return NULL; + ddict->cMem = customMem; + { size_t const initResult = ZSTD_initDDict_internal(ddict, + dict, dictSize, + dictLoadMethod, dictContentType); + if (ZSTD_isError(initResult)) { + ZSTD_freeDDict(ddict); + return NULL; + } } + return ddict; + } +} + +/*! ZSTD_createDDict() : +* Create a digested dictionary, to start decompression without startup delay. +* `dict` content is copied inside DDict. +* Consequently, `dict` can be released after `ZSTD_DDict` creation */ +ZSTD_DDict* ZSTD_createDDict(const void* dict, size_t dictSize) +{ + ZSTD_customMem const allocator = { NULL, NULL, NULL }; + return ZSTD_createDDict_advanced(dict, dictSize, ZSTD_dlm_byCopy, ZSTD_dct_auto, allocator); +} + +/*! ZSTD_createDDict_byReference() : + * Create a digested dictionary, to start decompression without startup delay. + * Dictionary content is simply referenced, it will be accessed during decompression. + * Warning : dictBuffer must outlive DDict (DDict must be freed before dictBuffer) */ +ZSTD_DDict* ZSTD_createDDict_byReference(const void* dictBuffer, size_t dictSize) +{ + ZSTD_customMem const allocator = { NULL, NULL, NULL }; + return ZSTD_createDDict_advanced(dictBuffer, dictSize, ZSTD_dlm_byRef, ZSTD_dct_auto, allocator); +} + + +const ZSTD_DDict* ZSTD_initStaticDDict( + void* sBuffer, size_t sBufferSize, + const void* dict, size_t dictSize, + ZSTD_dictLoadMethod_e dictLoadMethod, + ZSTD_dictContentType_e dictContentType) +{ + size_t const neededSpace = sizeof(ZSTD_DDict) + + (dictLoadMethod == ZSTD_dlm_byRef ? 0 : dictSize); + ZSTD_DDict* const ddict = (ZSTD_DDict*)sBuffer; + assert(sBuffer != NULL); + assert(dict != NULL); + if ((size_t)sBuffer & 7) return NULL; /* 8-aligned */ + if (sBufferSize < neededSpace) return NULL; + if (dictLoadMethod == ZSTD_dlm_byCopy) { + memcpy(ddict+1, dict, dictSize); /* local copy */ + dict = ddict+1; + } + if (ZSTD_isError( ZSTD_initDDict_internal(ddict, + dict, dictSize, + ZSTD_dlm_byRef, dictContentType) )) + return NULL; + return ddict; +} + + +size_t ZSTD_freeDDict(ZSTD_DDict* ddict) +{ + if (ddict==NULL) return 0; /* support free on NULL */ + { ZSTD_customMem const cMem = ddict->cMem; + ZSTD_free(ddict->dictBuffer, cMem); + ZSTD_free(ddict, cMem); + return 0; + } +} + +/*! ZSTD_estimateDDictSize() : + * Estimate amount of memory that will be needed to create a dictionary for decompression. + * Note : dictionary created by reference using ZSTD_dlm_byRef are smaller */ +size_t ZSTD_estimateDDictSize(size_t dictSize, ZSTD_dictLoadMethod_e dictLoadMethod) +{ + return sizeof(ZSTD_DDict) + (dictLoadMethod == ZSTD_dlm_byRef ? 0 : dictSize); +} + +size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict) +{ + if (ddict==NULL) return 0; /* support sizeof on NULL */ + return sizeof(*ddict) + (ddict->dictBuffer ? ddict->dictSize : 0) ; +} + +/*! ZSTD_getDictID_fromDDict() : + * Provides the dictID of the dictionary loaded into `ddict`. + * If @return == 0, the dictionary is not conformant to Zstandard specification, or empty. + * Non-conformant dictionaries can still be loaded, but as content-only dictionaries. */ +unsigned ZSTD_getDictID_fromDDict(const ZSTD_DDict* ddict) +{ + if (ddict==NULL) return 0; + return ZSTD_getDictID_fromDict(ddict->dictContent, ddict->dictSize); +} diff --git a/module/zstd/lib/decompress/zstd_ddict.h b/module/zstd/lib/decompress/zstd_ddict.h new file mode 100644 index 000000000000..af307efd3d76 --- /dev/null +++ b/module/zstd/lib/decompress/zstd_ddict.h @@ -0,0 +1,44 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + + +#ifndef ZSTD_DDICT_H +#define ZSTD_DDICT_H + +/*-******************************************************* + * Dependencies + *********************************************************/ +#include /* size_t */ +#include "../zstd.h" /* ZSTD_DDict, and several public functions */ + + +/*-******************************************************* + * Interface + *********************************************************/ + +/* note: several prototypes are already published in `zstd.h` : + * ZSTD_createDDict() + * ZSTD_createDDict_byReference() + * ZSTD_createDDict_advanced() + * ZSTD_freeDDict() + * ZSTD_initStaticDDict() + * ZSTD_sizeof_DDict() + * ZSTD_estimateDDictSize() + * ZSTD_getDictID_fromDict() + */ + +const void* ZSTD_DDict_dictContent(const ZSTD_DDict* ddict); +size_t ZSTD_DDict_dictSize(const ZSTD_DDict* ddict); + +void ZSTD_copyDDictParameters(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict); + + + +#endif /* ZSTD_DDICT_H */ diff --git a/module/zstd/lib/decompress/zstd_decompress.c b/module/zstd/lib/decompress/zstd_decompress.c new file mode 100644 index 000000000000..be5c7cfc3347 --- /dev/null +++ b/module/zstd/lib/decompress/zstd_decompress.c @@ -0,0 +1,1885 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + + +/* *************************************************************** +* Tuning parameters +*****************************************************************/ +/*! + * HEAPMODE : + * Select how default decompression function ZSTD_decompress() allocates its context, + * on stack (0), or into heap (1, default; requires malloc()). + * Note that functions with explicit context such as ZSTD_decompressDCtx() are unaffected. + */ +#ifndef ZSTD_HEAPMODE +# define ZSTD_HEAPMODE 1 +#endif + +/*! +* LEGACY_SUPPORT : +* if set to 1+, ZSTD_decompress() can decode older formats (v0.1+) +*/ +#ifndef ZSTD_LEGACY_SUPPORT +# define ZSTD_LEGACY_SUPPORT 0 +#endif + +/*! + * MAXWINDOWSIZE_DEFAULT : + * maximum window size accepted by DStream __by default__. + * Frames requiring more memory will be rejected. + * It's possible to set a different limit using ZSTD_DCtx_setMaxWindowSize(). + */ +#ifndef ZSTD_MAXWINDOWSIZE_DEFAULT +# define ZSTD_MAXWINDOWSIZE_DEFAULT (((U32)1 << ZSTD_WINDOWLOG_LIMIT_DEFAULT) + 1) +#endif + +/*! + * NO_FORWARD_PROGRESS_MAX : + * maximum allowed nb of calls to ZSTD_decompressStream() + * without any forward progress + * (defined as: no byte read from input, and no byte flushed to output) + * before triggering an error. + */ +#ifndef ZSTD_NO_FORWARD_PROGRESS_MAX +# define ZSTD_NO_FORWARD_PROGRESS_MAX 16 +#endif + + +/*-******************************************************* +* Dependencies +*********************************************************/ +#include /* memcpy, memmove, memset */ +#include "../common/cpu.h" /* bmi2 */ +#include "../common/mem.h" /* low level memory routines */ +#define FSE_STATIC_LINKING_ONLY +#include "../common/fse.h" +#define HUF_STATIC_LINKING_ONLY +#include "../common/huf.h" +#include "../common/zstd_internal.h" /* blockProperties_t */ +#include "zstd_decompress_internal.h" /* ZSTD_DCtx */ +#include "zstd_ddict.h" /* ZSTD_DDictDictContent */ +#include "zstd_decompress_block.h" /* ZSTD_decompressBlock_internal */ + +#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT>=1) +# include "../legacy/zstd_legacy.h" +#endif + + +/*-************************************************************* +* Context management +***************************************************************/ +size_t ZSTD_sizeof_DCtx (const ZSTD_DCtx* dctx) +{ + if (dctx==NULL) return 0; /* support sizeof NULL */ + return sizeof(*dctx) + + ZSTD_sizeof_DDict(dctx->ddictLocal) + + dctx->inBuffSize + dctx->outBuffSize; +} + +size_t ZSTD_estimateDCtxSize(void) { return sizeof(ZSTD_DCtx); } + + +static size_t ZSTD_startingInputLength(ZSTD_format_e format) +{ + size_t const startingInputLength = ZSTD_FRAMEHEADERSIZE_PREFIX(format); + /* only supports formats ZSTD_f_zstd1 and ZSTD_f_zstd1_magicless */ + assert( (format == ZSTD_f_zstd1) || (format == ZSTD_f_zstd1_magicless) ); + return startingInputLength; +} + +static void ZSTD_initDCtx_internal(ZSTD_DCtx* dctx) +{ + dctx->format = ZSTD_f_zstd1; /* ZSTD_decompressBegin() invokes ZSTD_startingInputLength() with argument dctx->format */ + dctx->staticSize = 0; + dctx->maxWindowSize = ZSTD_MAXWINDOWSIZE_DEFAULT; + dctx->ddict = NULL; + dctx->ddictLocal = NULL; + dctx->dictEnd = NULL; + dctx->ddictIsCold = 0; + dctx->dictUses = ZSTD_dont_use; + dctx->inBuff = NULL; + dctx->inBuffSize = 0; + dctx->outBuffSize = 0; + dctx->streamStage = zdss_init; + dctx->legacyContext = NULL; + dctx->previousLegacyVersion = 0; + dctx->noForwardProgress = 0; + dctx->oversizedDuration = 0; + dctx->bmi2 = ZSTD_cpuid_bmi2(ZSTD_cpuid()); + dctx->outBufferMode = ZSTD_obm_buffered; +#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION + dctx->dictContentEndForFuzzing = NULL; +#endif +} + +ZSTD_DCtx* ZSTD_initStaticDCtx(void *workspace, size_t workspaceSize) +{ + ZSTD_DCtx* const dctx = (ZSTD_DCtx*) workspace; + + if ((size_t)workspace & 7) return NULL; /* 8-aligned */ + if (workspaceSize < sizeof(ZSTD_DCtx)) return NULL; /* minimum size */ + + ZSTD_initDCtx_internal(dctx); + dctx->staticSize = workspaceSize; + dctx->inBuff = (char*)(dctx+1); + return dctx; +} + +ZSTD_DCtx* ZSTD_createDCtx_advanced(ZSTD_customMem customMem) +{ + if (!customMem.customAlloc ^ !customMem.customFree) return NULL; + + { ZSTD_DCtx* const dctx = (ZSTD_DCtx*)ZSTD_malloc(sizeof(*dctx), customMem); + if (!dctx) return NULL; + dctx->customMem = customMem; + ZSTD_initDCtx_internal(dctx); + return dctx; + } +} + +ZSTD_DCtx* ZSTD_createDCtx(void) +{ + DEBUGLOG(3, "ZSTD_createDCtx"); + return ZSTD_createDCtx_advanced(ZSTD_defaultCMem); +} + +static void ZSTD_clearDict(ZSTD_DCtx* dctx) +{ + ZSTD_freeDDict(dctx->ddictLocal); + dctx->ddictLocal = NULL; + dctx->ddict = NULL; + dctx->dictUses = ZSTD_dont_use; +} + +size_t ZSTD_freeDCtx(ZSTD_DCtx* dctx) +{ + if (dctx==NULL) return 0; /* support free on NULL */ + RETURN_ERROR_IF(dctx->staticSize, memory_allocation, "not compatible with static DCtx"); + { ZSTD_customMem const cMem = dctx->customMem; + ZSTD_clearDict(dctx); + ZSTD_free(dctx->inBuff, cMem); + dctx->inBuff = NULL; +#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT >= 1) + if (dctx->legacyContext) + ZSTD_freeLegacyStreamContext(dctx->legacyContext, dctx->previousLegacyVersion); +#endif + ZSTD_free(dctx, cMem); + return 0; + } +} + +/* no longer useful */ +void ZSTD_copyDCtx(ZSTD_DCtx* dstDCtx, const ZSTD_DCtx* srcDCtx) +{ + size_t const toCopy = (size_t)((char*)(&dstDCtx->inBuff) - (char*)dstDCtx); + memcpy(dstDCtx, srcDCtx, toCopy); /* no need to copy workspace */ +} + + +/*-************************************************************* + * Frame header decoding + ***************************************************************/ + +/*! ZSTD_isFrame() : + * Tells if the content of `buffer` starts with a valid Frame Identifier. + * Note : Frame Identifier is 4 bytes. If `size < 4`, @return will always be 0. + * Note 2 : Legacy Frame Identifiers are considered valid only if Legacy Support is enabled. + * Note 3 : Skippable Frame Identifiers are considered valid. */ +unsigned ZSTD_isFrame(const void* buffer, size_t size) +{ + if (size < ZSTD_FRAMEIDSIZE) return 0; + { U32 const magic = MEM_readLE32(buffer); + if (magic == ZSTD_MAGICNUMBER) return 1; + if ((magic & ZSTD_MAGIC_SKIPPABLE_MASK) == ZSTD_MAGIC_SKIPPABLE_START) return 1; + } +#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT >= 1) + if (ZSTD_isLegacy(buffer, size)) return 1; +#endif + return 0; +} + +/** ZSTD_frameHeaderSize_internal() : + * srcSize must be large enough to reach header size fields. + * note : only works for formats ZSTD_f_zstd1 and ZSTD_f_zstd1_magicless. + * @return : size of the Frame Header + * or an error code, which can be tested with ZSTD_isError() */ +static size_t ZSTD_frameHeaderSize_internal(const void* src, size_t srcSize, ZSTD_format_e format) +{ + size_t const minInputSize = ZSTD_startingInputLength(format); + RETURN_ERROR_IF(srcSize < minInputSize, srcSize_wrong, ""); + + { BYTE const fhd = ((const BYTE*)src)[minInputSize-1]; + U32 const dictID= fhd & 3; + U32 const singleSegment = (fhd >> 5) & 1; + U32 const fcsId = fhd >> 6; + return minInputSize + !singleSegment + + ZSTD_did_fieldSize[dictID] + ZSTD_fcs_fieldSize[fcsId] + + (singleSegment && !fcsId); + } +} + +/** ZSTD_frameHeaderSize() : + * srcSize must be >= ZSTD_frameHeaderSize_prefix. + * @return : size of the Frame Header, + * or an error code (if srcSize is too small) */ +size_t ZSTD_frameHeaderSize(const void* src, size_t srcSize) +{ + return ZSTD_frameHeaderSize_internal(src, srcSize, ZSTD_f_zstd1); +} + + +/** ZSTD_getFrameHeader_advanced() : + * decode Frame Header, or require larger `srcSize`. + * note : only works for formats ZSTD_f_zstd1 and ZSTD_f_zstd1_magicless + * @return : 0, `zfhPtr` is correctly filled, + * >0, `srcSize` is too small, value is wanted `srcSize` amount, + * or an error code, which can be tested using ZSTD_isError() */ +size_t ZSTD_getFrameHeader_advanced(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize, ZSTD_format_e format) +{ + const BYTE* ip = (const BYTE*)src; + size_t const minInputSize = ZSTD_startingInputLength(format); + + memset(zfhPtr, 0, sizeof(*zfhPtr)); /* not strictly necessary, but static analyzer do not understand that zfhPtr is only going to be read only if return value is zero, since they are 2 different signals */ + if (srcSize < minInputSize) return minInputSize; + RETURN_ERROR_IF(src==NULL, GENERIC, "invalid parameter"); + + if ( (format != ZSTD_f_zstd1_magicless) + && (MEM_readLE32(src) != ZSTD_MAGICNUMBER) ) { + if ((MEM_readLE32(src) & ZSTD_MAGIC_SKIPPABLE_MASK) == ZSTD_MAGIC_SKIPPABLE_START) { + /* skippable frame */ + if (srcSize < ZSTD_SKIPPABLEHEADERSIZE) + return ZSTD_SKIPPABLEHEADERSIZE; /* magic number + frame length */ + memset(zfhPtr, 0, sizeof(*zfhPtr)); + zfhPtr->frameContentSize = MEM_readLE32((const char *)src + ZSTD_FRAMEIDSIZE); + zfhPtr->frameType = ZSTD_skippableFrame; + return 0; + } + RETURN_ERROR(prefix_unknown, ""); + } + + /* ensure there is enough `srcSize` to fully read/decode frame header */ + { size_t const fhsize = ZSTD_frameHeaderSize_internal(src, srcSize, format); + if (srcSize < fhsize) return fhsize; + zfhPtr->headerSize = (U32)fhsize; + } + + { BYTE const fhdByte = ip[minInputSize-1]; + size_t pos = minInputSize; + U32 const dictIDSizeCode = fhdByte&3; + U32 const checksumFlag = (fhdByte>>2)&1; + U32 const singleSegment = (fhdByte>>5)&1; + U32 const fcsID = fhdByte>>6; + U64 windowSize = 0; + U32 dictID = 0; + U64 frameContentSize = ZSTD_CONTENTSIZE_UNKNOWN; + RETURN_ERROR_IF((fhdByte & 0x08) != 0, frameParameter_unsupported, + "reserved bits, must be zero"); + + if (!singleSegment) { + BYTE const wlByte = ip[pos++]; + U32 const windowLog = (wlByte >> 3) + ZSTD_WINDOWLOG_ABSOLUTEMIN; + RETURN_ERROR_IF(windowLog > ZSTD_WINDOWLOG_MAX, frameParameter_windowTooLarge, ""); + windowSize = (1ULL << windowLog); + windowSize += (windowSize >> 3) * (wlByte&7); + } + switch(dictIDSizeCode) + { + default: assert(0); /* impossible */ + case 0 : break; + case 1 : dictID = ip[pos]; pos++; break; + case 2 : dictID = MEM_readLE16(ip+pos); pos+=2; break; + case 3 : dictID = MEM_readLE32(ip+pos); pos+=4; break; + } + switch(fcsID) + { + default: assert(0); /* impossible */ + case 0 : if (singleSegment) frameContentSize = ip[pos]; break; + case 1 : frameContentSize = MEM_readLE16(ip+pos)+256; break; + case 2 : frameContentSize = MEM_readLE32(ip+pos); break; + case 3 : frameContentSize = MEM_readLE64(ip+pos); break; + } + if (singleSegment) windowSize = frameContentSize; + + zfhPtr->frameType = ZSTD_frame; + zfhPtr->frameContentSize = frameContentSize; + zfhPtr->windowSize = windowSize; + zfhPtr->blockSizeMax = (unsigned) MIN(windowSize, ZSTD_BLOCKSIZE_MAX); + zfhPtr->dictID = dictID; + zfhPtr->checksumFlag = checksumFlag; + } + return 0; +} + +/** ZSTD_getFrameHeader() : + * decode Frame Header, or require larger `srcSize`. + * note : this function does not consume input, it only reads it. + * @return : 0, `zfhPtr` is correctly filled, + * >0, `srcSize` is too small, value is wanted `srcSize` amount, + * or an error code, which can be tested using ZSTD_isError() */ +size_t ZSTD_getFrameHeader(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize) +{ + return ZSTD_getFrameHeader_advanced(zfhPtr, src, srcSize, ZSTD_f_zstd1); +} + + +/** ZSTD_getFrameContentSize() : + * compatible with legacy mode + * @return : decompressed size of the single frame pointed to be `src` if known, otherwise + * - ZSTD_CONTENTSIZE_UNKNOWN if the size cannot be determined + * - ZSTD_CONTENTSIZE_ERROR if an error occurred (e.g. invalid magic number, srcSize too small) */ +unsigned long long ZSTD_getFrameContentSize(const void *src, size_t srcSize) +{ +#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT >= 1) + if (ZSTD_isLegacy(src, srcSize)) { + unsigned long long const ret = ZSTD_getDecompressedSize_legacy(src, srcSize); + return ret == 0 ? ZSTD_CONTENTSIZE_UNKNOWN : ret; + } +#endif + { ZSTD_frameHeader zfh; + if (ZSTD_getFrameHeader(&zfh, src, srcSize) != 0) + return ZSTD_CONTENTSIZE_ERROR; + if (zfh.frameType == ZSTD_skippableFrame) { + return 0; + } else { + return zfh.frameContentSize; + } } +} + +static size_t readSkippableFrameSize(void const* src, size_t srcSize) +{ + size_t const skippableHeaderSize = ZSTD_SKIPPABLEHEADERSIZE; + U32 sizeU32; + + RETURN_ERROR_IF(srcSize < ZSTD_SKIPPABLEHEADERSIZE, srcSize_wrong, ""); + + sizeU32 = MEM_readLE32((BYTE const*)src + ZSTD_FRAMEIDSIZE); + RETURN_ERROR_IF((U32)(sizeU32 + ZSTD_SKIPPABLEHEADERSIZE) < sizeU32, + frameParameter_unsupported, ""); + { + size_t const skippableSize = skippableHeaderSize + sizeU32; + RETURN_ERROR_IF(skippableSize > srcSize, srcSize_wrong, ""); + return skippableSize; + } +} + +/** ZSTD_findDecompressedSize() : + * compatible with legacy mode + * `srcSize` must be the exact length of some number of ZSTD compressed and/or + * skippable frames + * @return : decompressed size of the frames contained */ +unsigned long long ZSTD_findDecompressedSize(const void* src, size_t srcSize) +{ + unsigned long long totalDstSize = 0; + + while (srcSize >= ZSTD_startingInputLength(ZSTD_f_zstd1)) { + U32 const magicNumber = MEM_readLE32(src); + + if ((magicNumber & ZSTD_MAGIC_SKIPPABLE_MASK) == ZSTD_MAGIC_SKIPPABLE_START) { + size_t const skippableSize = readSkippableFrameSize(src, srcSize); + if (ZSTD_isError(skippableSize)) { + return ZSTD_CONTENTSIZE_ERROR; + } + assert(skippableSize <= srcSize); + + src = (const BYTE *)src + skippableSize; + srcSize -= skippableSize; + continue; + } + + { unsigned long long const ret = ZSTD_getFrameContentSize(src, srcSize); + if (ret >= ZSTD_CONTENTSIZE_ERROR) return ret; + + /* check for overflow */ + if (totalDstSize + ret < totalDstSize) return ZSTD_CONTENTSIZE_ERROR; + totalDstSize += ret; + } + { size_t const frameSrcSize = ZSTD_findFrameCompressedSize(src, srcSize); + if (ZSTD_isError(frameSrcSize)) { + return ZSTD_CONTENTSIZE_ERROR; + } + + src = (const BYTE *)src + frameSrcSize; + srcSize -= frameSrcSize; + } + } /* while (srcSize >= ZSTD_frameHeaderSize_prefix) */ + + if (srcSize) return ZSTD_CONTENTSIZE_ERROR; + + return totalDstSize; +} + +/** ZSTD_getDecompressedSize() : + * compatible with legacy mode + * @return : decompressed size if known, 0 otherwise + note : 0 can mean any of the following : + - frame content is empty + - decompressed size field is not present in frame header + - frame header unknown / not supported + - frame header not complete (`srcSize` too small) */ +unsigned long long ZSTD_getDecompressedSize(const void* src, size_t srcSize) +{ + unsigned long long const ret = ZSTD_getFrameContentSize(src, srcSize); + ZSTD_STATIC_ASSERT(ZSTD_CONTENTSIZE_ERROR < ZSTD_CONTENTSIZE_UNKNOWN); + return (ret >= ZSTD_CONTENTSIZE_ERROR) ? 0 : ret; +} + + +/** ZSTD_decodeFrameHeader() : + * `headerSize` must be the size provided by ZSTD_frameHeaderSize(). + * @return : 0 if success, or an error code, which can be tested using ZSTD_isError() */ +static size_t ZSTD_decodeFrameHeader(ZSTD_DCtx* dctx, const void* src, size_t headerSize) +{ + size_t const result = ZSTD_getFrameHeader_advanced(&(dctx->fParams), src, headerSize, dctx->format); + if (ZSTD_isError(result)) return result; /* invalid header */ + RETURN_ERROR_IF(result>0, srcSize_wrong, "headerSize too small"); +#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION + /* Skip the dictID check in fuzzing mode, because it makes the search + * harder. + */ + RETURN_ERROR_IF(dctx->fParams.dictID && (dctx->dictID != dctx->fParams.dictID), + dictionary_wrong, ""); +#endif + if (dctx->fParams.checksumFlag) XXH64_reset(&dctx->xxhState, 0); + return 0; +} + +static ZSTD_frameSizeInfo ZSTD_errorFrameSizeInfo(size_t ret) +{ + ZSTD_frameSizeInfo frameSizeInfo; + frameSizeInfo.compressedSize = ret; + frameSizeInfo.decompressedBound = ZSTD_CONTENTSIZE_ERROR; + return frameSizeInfo; +} + +static ZSTD_frameSizeInfo ZSTD_findFrameSizeInfo(const void* src, size_t srcSize) +{ + ZSTD_frameSizeInfo frameSizeInfo; + memset(&frameSizeInfo, 0, sizeof(ZSTD_frameSizeInfo)); + +#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT >= 1) + if (ZSTD_isLegacy(src, srcSize)) + return ZSTD_findFrameSizeInfoLegacy(src, srcSize); +#endif + + if ((srcSize >= ZSTD_SKIPPABLEHEADERSIZE) + && (MEM_readLE32(src) & ZSTD_MAGIC_SKIPPABLE_MASK) == ZSTD_MAGIC_SKIPPABLE_START) { + frameSizeInfo.compressedSize = readSkippableFrameSize(src, srcSize); + assert(ZSTD_isError(frameSizeInfo.compressedSize) || + frameSizeInfo.compressedSize <= srcSize); + return frameSizeInfo; + } else { + const BYTE* ip = (const BYTE*)src; + const BYTE* const ipstart = ip; + size_t remainingSize = srcSize; + size_t nbBlocks = 0; + ZSTD_frameHeader zfh; + + /* Extract Frame Header */ + { size_t const ret = ZSTD_getFrameHeader(&zfh, src, srcSize); + if (ZSTD_isError(ret)) + return ZSTD_errorFrameSizeInfo(ret); + if (ret > 0) + return ZSTD_errorFrameSizeInfo(ERROR(srcSize_wrong)); + } + + ip += zfh.headerSize; + remainingSize -= zfh.headerSize; + + /* Iterate over each block */ + while (1) { + blockProperties_t blockProperties; + size_t const cBlockSize = ZSTD_getcBlockSize(ip, remainingSize, &blockProperties); + if (ZSTD_isError(cBlockSize)) + return ZSTD_errorFrameSizeInfo(cBlockSize); + + if (ZSTD_blockHeaderSize + cBlockSize > remainingSize) + return ZSTD_errorFrameSizeInfo(ERROR(srcSize_wrong)); + + ip += ZSTD_blockHeaderSize + cBlockSize; + remainingSize -= ZSTD_blockHeaderSize + cBlockSize; + nbBlocks++; + + if (blockProperties.lastBlock) break; + } + + /* Final frame content checksum */ + if (zfh.checksumFlag) { + if (remainingSize < 4) + return ZSTD_errorFrameSizeInfo(ERROR(srcSize_wrong)); + ip += 4; + } + + frameSizeInfo.compressedSize = ip - ipstart; + frameSizeInfo.decompressedBound = (zfh.frameContentSize != ZSTD_CONTENTSIZE_UNKNOWN) + ? zfh.frameContentSize + : nbBlocks * zfh.blockSizeMax; + return frameSizeInfo; + } +} + +/** ZSTD_findFrameCompressedSize() : + * compatible with legacy mode + * `src` must point to the start of a ZSTD frame, ZSTD legacy frame, or skippable frame + * `srcSize` must be at least as large as the frame contained + * @return : the compressed size of the frame starting at `src` */ +size_t ZSTD_findFrameCompressedSize(const void *src, size_t srcSize) +{ + ZSTD_frameSizeInfo const frameSizeInfo = ZSTD_findFrameSizeInfo(src, srcSize); + return frameSizeInfo.compressedSize; +} + +/** ZSTD_decompressBound() : + * compatible with legacy mode + * `src` must point to the start of a ZSTD frame or a skippeable frame + * `srcSize` must be at least as large as the frame contained + * @return : the maximum decompressed size of the compressed source + */ +unsigned long long ZSTD_decompressBound(const void* src, size_t srcSize) +{ + unsigned long long bound = 0; + /* Iterate over each frame */ + while (srcSize > 0) { + ZSTD_frameSizeInfo const frameSizeInfo = ZSTD_findFrameSizeInfo(src, srcSize); + size_t const compressedSize = frameSizeInfo.compressedSize; + unsigned long long const decompressedBound = frameSizeInfo.decompressedBound; + if (ZSTD_isError(compressedSize) || decompressedBound == ZSTD_CONTENTSIZE_ERROR) + return ZSTD_CONTENTSIZE_ERROR; + assert(srcSize >= compressedSize); + src = (const BYTE*)src + compressedSize; + srcSize -= compressedSize; + bound += decompressedBound; + } + return bound; +} + + +/*-************************************************************* + * Frame decoding + ***************************************************************/ + +/** ZSTD_insertBlock() : + * insert `src` block into `dctx` history. Useful to track uncompressed blocks. */ +size_t ZSTD_insertBlock(ZSTD_DCtx* dctx, const void* blockStart, size_t blockSize) +{ + DEBUGLOG(5, "ZSTD_insertBlock: %u bytes", (unsigned)blockSize); + ZSTD_checkContinuity(dctx, blockStart); + dctx->previousDstEnd = (const char*)blockStart + blockSize; + return blockSize; +} + + +static size_t ZSTD_copyRawBlock(void* dst, size_t dstCapacity, + const void* src, size_t srcSize) +{ + DEBUGLOG(5, "ZSTD_copyRawBlock"); + if (dst == NULL) { + if (srcSize == 0) return 0; + RETURN_ERROR(dstBuffer_null, ""); + } + RETURN_ERROR_IF(srcSize > dstCapacity, dstSize_tooSmall, ""); + memcpy(dst, src, srcSize); + return srcSize; +} + +static size_t ZSTD_setRleBlock(void* dst, size_t dstCapacity, + BYTE b, + size_t regenSize) +{ + if (dst == NULL) { + if (regenSize == 0) return 0; + RETURN_ERROR(dstBuffer_null, ""); + } + RETURN_ERROR_IF(regenSize > dstCapacity, dstSize_tooSmall, ""); + memset(dst, b, regenSize); + return regenSize; +} + + +/*! ZSTD_decompressFrame() : + * @dctx must be properly initialized + * will update *srcPtr and *srcSizePtr, + * to make *srcPtr progress by one frame. */ +static size_t ZSTD_decompressFrame(ZSTD_DCtx* dctx, + void* dst, size_t dstCapacity, + const void** srcPtr, size_t *srcSizePtr) +{ + const BYTE* ip = (const BYTE*)(*srcPtr); + BYTE* const ostart = (BYTE* const)dst; + BYTE* const oend = dstCapacity != 0 ? ostart + dstCapacity : ostart; + BYTE* op = ostart; + size_t remainingSrcSize = *srcSizePtr; + + DEBUGLOG(4, "ZSTD_decompressFrame (srcSize:%i)", (int)*srcSizePtr); + + /* check */ + RETURN_ERROR_IF( + remainingSrcSize < ZSTD_FRAMEHEADERSIZE_MIN(dctx->format)+ZSTD_blockHeaderSize, + srcSize_wrong, ""); + + /* Frame Header */ + { size_t const frameHeaderSize = ZSTD_frameHeaderSize_internal( + ip, ZSTD_FRAMEHEADERSIZE_PREFIX(dctx->format), dctx->format); + if (ZSTD_isError(frameHeaderSize)) return frameHeaderSize; + RETURN_ERROR_IF(remainingSrcSize < frameHeaderSize+ZSTD_blockHeaderSize, + srcSize_wrong, ""); + FORWARD_IF_ERROR( ZSTD_decodeFrameHeader(dctx, ip, frameHeaderSize) , ""); + ip += frameHeaderSize; remainingSrcSize -= frameHeaderSize; + } + + /* Loop on each block */ + while (1) { + size_t decodedSize; + blockProperties_t blockProperties; + size_t const cBlockSize = ZSTD_getcBlockSize(ip, remainingSrcSize, &blockProperties); + if (ZSTD_isError(cBlockSize)) return cBlockSize; + + ip += ZSTD_blockHeaderSize; + remainingSrcSize -= ZSTD_blockHeaderSize; + RETURN_ERROR_IF(cBlockSize > remainingSrcSize, srcSize_wrong, ""); + + switch(blockProperties.blockType) + { + case bt_compressed: + decodedSize = ZSTD_decompressBlock_internal(dctx, op, oend-op, ip, cBlockSize, /* frame */ 1); + break; + case bt_raw : + decodedSize = ZSTD_copyRawBlock(op, oend-op, ip, cBlockSize); + break; + case bt_rle : + decodedSize = ZSTD_setRleBlock(op, oend-op, *ip, blockProperties.origSize); + break; + case bt_reserved : + default: + RETURN_ERROR(corruption_detected, "invalid block type"); + } + + if (ZSTD_isError(decodedSize)) return decodedSize; + if (dctx->fParams.checksumFlag) + XXH64_update(&dctx->xxhState, op, decodedSize); + if (decodedSize != 0) + op += decodedSize; + assert(ip != NULL); + ip += cBlockSize; + remainingSrcSize -= cBlockSize; + if (blockProperties.lastBlock) break; + } + + if (dctx->fParams.frameContentSize != ZSTD_CONTENTSIZE_UNKNOWN) { + RETURN_ERROR_IF((U64)(op-ostart) != dctx->fParams.frameContentSize, + corruption_detected, ""); + } + if (dctx->fParams.checksumFlag) { /* Frame content checksum verification */ + U32 const checkCalc = (U32)XXH64_digest(&dctx->xxhState); + U32 checkRead; + RETURN_ERROR_IF(remainingSrcSize<4, checksum_wrong, ""); + checkRead = MEM_readLE32(ip); + RETURN_ERROR_IF(checkRead != checkCalc, checksum_wrong, ""); + ip += 4; + remainingSrcSize -= 4; + } + + /* Allow caller to get size read */ + *srcPtr = ip; + *srcSizePtr = remainingSrcSize; + return op-ostart; +} + +static size_t ZSTD_decompressMultiFrame(ZSTD_DCtx* dctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + const void* dict, size_t dictSize, + const ZSTD_DDict* ddict) +{ + void* const dststart = dst; + int moreThan1Frame = 0; + + DEBUGLOG(5, "ZSTD_decompressMultiFrame"); + assert(dict==NULL || ddict==NULL); /* either dict or ddict set, not both */ + + if (ddict) { + dict = ZSTD_DDict_dictContent(ddict); + dictSize = ZSTD_DDict_dictSize(ddict); + } + + while (srcSize >= ZSTD_startingInputLength(dctx->format)) { + +#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT >= 1) + if (ZSTD_isLegacy(src, srcSize)) { + size_t decodedSize; + size_t const frameSize = ZSTD_findFrameCompressedSizeLegacy(src, srcSize); + if (ZSTD_isError(frameSize)) return frameSize; + RETURN_ERROR_IF(dctx->staticSize, memory_allocation, + "legacy support is not compatible with static dctx"); + + decodedSize = ZSTD_decompressLegacy(dst, dstCapacity, src, frameSize, dict, dictSize); + if (ZSTD_isError(decodedSize)) return decodedSize; + + assert(decodedSize <=- dstCapacity); + dst = (BYTE*)dst + decodedSize; + dstCapacity -= decodedSize; + + src = (const BYTE*)src + frameSize; + srcSize -= frameSize; + + continue; + } +#endif + + { U32 const magicNumber = MEM_readLE32(src); + DEBUGLOG(4, "reading magic number %08X (expecting %08X)", + (unsigned)magicNumber, ZSTD_MAGICNUMBER); + if ((magicNumber & ZSTD_MAGIC_SKIPPABLE_MASK) == ZSTD_MAGIC_SKIPPABLE_START) { + size_t const skippableSize = readSkippableFrameSize(src, srcSize); + FORWARD_IF_ERROR(skippableSize, "readSkippableFrameSize failed"); + assert(skippableSize <= srcSize); + + src = (const BYTE *)src + skippableSize; + srcSize -= skippableSize; + continue; + } } + + if (ddict) { + /* we were called from ZSTD_decompress_usingDDict */ + FORWARD_IF_ERROR(ZSTD_decompressBegin_usingDDict(dctx, ddict), ""); + } else { + /* this will initialize correctly with no dict if dict == NULL, so + * use this in all cases but ddict */ + FORWARD_IF_ERROR(ZSTD_decompressBegin_usingDict(dctx, dict, dictSize), ""); + } + ZSTD_checkContinuity(dctx, dst); + + { const size_t res = ZSTD_decompressFrame(dctx, dst, dstCapacity, + &src, &srcSize); + RETURN_ERROR_IF( + (ZSTD_getErrorCode(res) == ZSTD_error_prefix_unknown) + && (moreThan1Frame==1), + srcSize_wrong, + "at least one frame successfully completed, but following " + "bytes are garbage: it's more likely to be a srcSize error, " + "specifying more bytes than compressed size of frame(s). This " + "error message replaces ERROR(prefix_unknown), which would be " + "confusing, as the first header is actually correct. Note that " + "one could be unlucky, it might be a corruption error instead, " + "happening right at the place where we expect zstd magic " + "bytes. But this is _much_ less likely than a srcSize field " + "error."); + if (ZSTD_isError(res)) return res; + assert(res <= dstCapacity); + if (res != 0) + dst = (BYTE*)dst + res; + dstCapacity -= res; + } + moreThan1Frame = 1; + } /* while (srcSize >= ZSTD_frameHeaderSize_prefix) */ + + RETURN_ERROR_IF(srcSize, srcSize_wrong, "input not entirely consumed"); + + return (BYTE*)dst - (BYTE*)dststart; +} + +size_t ZSTD_decompress_usingDict(ZSTD_DCtx* dctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + const void* dict, size_t dictSize) +{ + return ZSTD_decompressMultiFrame(dctx, dst, dstCapacity, src, srcSize, dict, dictSize, NULL); +} + + +static ZSTD_DDict const* ZSTD_getDDict(ZSTD_DCtx* dctx) +{ + switch (dctx->dictUses) { + default: + assert(0 /* Impossible */); + /* fall-through */ + case ZSTD_dont_use: + ZSTD_clearDict(dctx); + return NULL; + case ZSTD_use_indefinitely: + return dctx->ddict; + case ZSTD_use_once: + dctx->dictUses = ZSTD_dont_use; + return dctx->ddict; + } +} + +size_t ZSTD_decompressDCtx(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize) +{ + return ZSTD_decompress_usingDDict(dctx, dst, dstCapacity, src, srcSize, ZSTD_getDDict(dctx)); +} + + +size_t ZSTD_decompress(void* dst, size_t dstCapacity, const void* src, size_t srcSize) +{ +#if defined(ZSTD_HEAPMODE) && (ZSTD_HEAPMODE>=1) + size_t regenSize; + ZSTD_DCtx* const dctx = ZSTD_createDCtx(); + RETURN_ERROR_IF(dctx==NULL, memory_allocation, "NULL pointer!"); + regenSize = ZSTD_decompressDCtx(dctx, dst, dstCapacity, src, srcSize); + ZSTD_freeDCtx(dctx); + return regenSize; +#else /* stack mode */ + ZSTD_DCtx dctx; + ZSTD_initDCtx_internal(&dctx); + return ZSTD_decompressDCtx(&dctx, dst, dstCapacity, src, srcSize); +#endif +} + + +/*-************************************** +* Advanced Streaming Decompression API +* Bufferless and synchronous +****************************************/ +size_t ZSTD_nextSrcSizeToDecompress(ZSTD_DCtx* dctx) { return dctx->expected; } + +/** + * Similar to ZSTD_nextSrcSizeToDecompress(), but when when a block input can be streamed, + * we allow taking a partial block as the input. Currently only raw uncompressed blocks can + * be streamed. + * + * For blocks that can be streamed, this allows us to reduce the latency until we produce + * output, and avoid copying the input. + * + * @param inputSize - The total amount of input that the caller currently has. + */ +static size_t ZSTD_nextSrcSizeToDecompressWithInputSize(ZSTD_DCtx* dctx, size_t inputSize) { + if (!(dctx->stage == ZSTDds_decompressBlock || dctx->stage == ZSTDds_decompressLastBlock)) + return dctx->expected; + if (dctx->bType != bt_raw) + return dctx->expected; + return MIN(MAX(inputSize, 1), dctx->expected); +} + +ZSTD_nextInputType_e ZSTD_nextInputType(ZSTD_DCtx* dctx) { + switch(dctx->stage) + { + default: /* should not happen */ + assert(0); + case ZSTDds_getFrameHeaderSize: + case ZSTDds_decodeFrameHeader: + return ZSTDnit_frameHeader; + case ZSTDds_decodeBlockHeader: + return ZSTDnit_blockHeader; + case ZSTDds_decompressBlock: + return ZSTDnit_block; + case ZSTDds_decompressLastBlock: + return ZSTDnit_lastBlock; + case ZSTDds_checkChecksum: + return ZSTDnit_checksum; + case ZSTDds_decodeSkippableHeader: + case ZSTDds_skipFrame: + return ZSTDnit_skippableFrame; + } +} + +static int ZSTD_isSkipFrame(ZSTD_DCtx* dctx) { return dctx->stage == ZSTDds_skipFrame; } + +/** ZSTD_decompressContinue() : + * srcSize : must be the exact nb of bytes expected (see ZSTD_nextSrcSizeToDecompress()) + * @return : nb of bytes generated into `dst` (necessarily <= `dstCapacity) + * or an error code, which can be tested using ZSTD_isError() */ +size_t ZSTD_decompressContinue(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize) +{ + DEBUGLOG(5, "ZSTD_decompressContinue (srcSize:%u)", (unsigned)srcSize); + /* Sanity check */ + RETURN_ERROR_IF(srcSize != ZSTD_nextSrcSizeToDecompressWithInputSize(dctx, srcSize), srcSize_wrong, "not allowed"); + if (dstCapacity) ZSTD_checkContinuity(dctx, dst); + + switch (dctx->stage) + { + case ZSTDds_getFrameHeaderSize : + assert(src != NULL); + if (dctx->format == ZSTD_f_zstd1) { /* allows header */ + assert(srcSize >= ZSTD_FRAMEIDSIZE); /* to read skippable magic number */ + if ((MEM_readLE32(src) & ZSTD_MAGIC_SKIPPABLE_MASK) == ZSTD_MAGIC_SKIPPABLE_START) { /* skippable frame */ + memcpy(dctx->headerBuffer, src, srcSize); + dctx->expected = ZSTD_SKIPPABLEHEADERSIZE - srcSize; /* remaining to load to get full skippable frame header */ + dctx->stage = ZSTDds_decodeSkippableHeader; + return 0; + } } + dctx->headerSize = ZSTD_frameHeaderSize_internal(src, srcSize, dctx->format); + if (ZSTD_isError(dctx->headerSize)) return dctx->headerSize; + memcpy(dctx->headerBuffer, src, srcSize); + dctx->expected = dctx->headerSize - srcSize; + dctx->stage = ZSTDds_decodeFrameHeader; + return 0; + + case ZSTDds_decodeFrameHeader: + assert(src != NULL); + memcpy(dctx->headerBuffer + (dctx->headerSize - srcSize), src, srcSize); + FORWARD_IF_ERROR(ZSTD_decodeFrameHeader(dctx, dctx->headerBuffer, dctx->headerSize), ""); + dctx->expected = ZSTD_blockHeaderSize; + dctx->stage = ZSTDds_decodeBlockHeader; + return 0; + + case ZSTDds_decodeBlockHeader: + { blockProperties_t bp; + size_t const cBlockSize = ZSTD_getcBlockSize(src, ZSTD_blockHeaderSize, &bp); + if (ZSTD_isError(cBlockSize)) return cBlockSize; + RETURN_ERROR_IF(cBlockSize > dctx->fParams.blockSizeMax, corruption_detected, "Block Size Exceeds Maximum"); + dctx->expected = cBlockSize; + dctx->bType = bp.blockType; + dctx->rleSize = bp.origSize; + if (cBlockSize) { + dctx->stage = bp.lastBlock ? ZSTDds_decompressLastBlock : ZSTDds_decompressBlock; + return 0; + } + /* empty block */ + if (bp.lastBlock) { + if (dctx->fParams.checksumFlag) { + dctx->expected = 4; + dctx->stage = ZSTDds_checkChecksum; + } else { + dctx->expected = 0; /* end of frame */ + dctx->stage = ZSTDds_getFrameHeaderSize; + } + } else { + dctx->expected = ZSTD_blockHeaderSize; /* jump to next header */ + dctx->stage = ZSTDds_decodeBlockHeader; + } + return 0; + } + + case ZSTDds_decompressLastBlock: + case ZSTDds_decompressBlock: + DEBUGLOG(5, "ZSTD_decompressContinue: case ZSTDds_decompressBlock"); + { size_t rSize; + switch(dctx->bType) + { + case bt_compressed: + DEBUGLOG(5, "ZSTD_decompressContinue: case bt_compressed"); + rSize = ZSTD_decompressBlock_internal(dctx, dst, dstCapacity, src, srcSize, /* frame */ 1); + dctx->expected = 0; /* Streaming not supported */ + break; + case bt_raw : + assert(srcSize <= dctx->expected); + rSize = ZSTD_copyRawBlock(dst, dstCapacity, src, srcSize); + FORWARD_IF_ERROR(rSize, "ZSTD_copyRawBlock failed"); + assert(rSize == srcSize); + dctx->expected -= rSize; + break; + case bt_rle : + rSize = ZSTD_setRleBlock(dst, dstCapacity, *(const BYTE*)src, dctx->rleSize); + dctx->expected = 0; /* Streaming not supported */ + break; + case bt_reserved : /* should never happen */ + default: + RETURN_ERROR(corruption_detected, "invalid block type"); + } + FORWARD_IF_ERROR(rSize, ""); + RETURN_ERROR_IF(rSize > dctx->fParams.blockSizeMax, corruption_detected, "Decompressed Block Size Exceeds Maximum"); + DEBUGLOG(5, "ZSTD_decompressContinue: decoded size from block : %u", (unsigned)rSize); + dctx->decodedSize += rSize; + if (dctx->fParams.checksumFlag) XXH64_update(&dctx->xxhState, dst, rSize); + dctx->previousDstEnd = (char*)dst + rSize; + + /* Stay on the same stage until we are finished streaming the block. */ + if (dctx->expected > 0) { + return rSize; + } + + if (dctx->stage == ZSTDds_decompressLastBlock) { /* end of frame */ + DEBUGLOG(4, "ZSTD_decompressContinue: decoded size from frame : %u", (unsigned)dctx->decodedSize); + RETURN_ERROR_IF( + dctx->fParams.frameContentSize != ZSTD_CONTENTSIZE_UNKNOWN + && dctx->decodedSize != dctx->fParams.frameContentSize, + corruption_detected, ""); + if (dctx->fParams.checksumFlag) { /* another round for frame checksum */ + dctx->expected = 4; + dctx->stage = ZSTDds_checkChecksum; + } else { + dctx->expected = 0; /* ends here */ + dctx->stage = ZSTDds_getFrameHeaderSize; + } + } else { + dctx->stage = ZSTDds_decodeBlockHeader; + dctx->expected = ZSTD_blockHeaderSize; + } + return rSize; + } + + case ZSTDds_checkChecksum: + assert(srcSize == 4); /* guaranteed by dctx->expected */ + { U32 const h32 = (U32)XXH64_digest(&dctx->xxhState); + U32 const check32 = MEM_readLE32(src); + DEBUGLOG(4, "ZSTD_decompressContinue: checksum : calculated %08X :: %08X read", (unsigned)h32, (unsigned)check32); + RETURN_ERROR_IF(check32 != h32, checksum_wrong, ""); + dctx->expected = 0; + dctx->stage = ZSTDds_getFrameHeaderSize; + return 0; + } + + case ZSTDds_decodeSkippableHeader: + assert(src != NULL); + assert(srcSize <= ZSTD_SKIPPABLEHEADERSIZE); + memcpy(dctx->headerBuffer + (ZSTD_SKIPPABLEHEADERSIZE - srcSize), src, srcSize); /* complete skippable header */ + dctx->expected = MEM_readLE32(dctx->headerBuffer + ZSTD_FRAMEIDSIZE); /* note : dctx->expected can grow seriously large, beyond local buffer size */ + dctx->stage = ZSTDds_skipFrame; + return 0; + + case ZSTDds_skipFrame: + dctx->expected = 0; + dctx->stage = ZSTDds_getFrameHeaderSize; + return 0; + + default: + assert(0); /* impossible */ + RETURN_ERROR(GENERIC, "impossible to reach"); /* some compiler require default to do something */ + } +} + + +static size_t ZSTD_refDictContent(ZSTD_DCtx* dctx, const void* dict, size_t dictSize) +{ + dctx->dictEnd = dctx->previousDstEnd; + dctx->virtualStart = (const char*)dict - ((const char*)(dctx->previousDstEnd) - (const char*)(dctx->prefixStart)); + dctx->prefixStart = dict; + dctx->previousDstEnd = (const char*)dict + dictSize; +#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION + dctx->dictContentBeginForFuzzing = dctx->prefixStart; + dctx->dictContentEndForFuzzing = dctx->previousDstEnd; +#endif + return 0; +} + +/*! ZSTD_loadDEntropy() : + * dict : must point at beginning of a valid zstd dictionary. + * @return : size of entropy tables read */ +size_t +ZSTD_loadDEntropy(ZSTD_entropyDTables_t* entropy, + const void* const dict, size_t const dictSize) +{ + const BYTE* dictPtr = (const BYTE*)dict; + const BYTE* const dictEnd = dictPtr + dictSize; + + RETURN_ERROR_IF(dictSize <= 8, dictionary_corrupted, "dict is too small"); + assert(MEM_readLE32(dict) == ZSTD_MAGIC_DICTIONARY); /* dict must be valid */ + dictPtr += 8; /* skip header = magic + dictID */ + + ZSTD_STATIC_ASSERT(offsetof(ZSTD_entropyDTables_t, OFTable) == offsetof(ZSTD_entropyDTables_t, LLTable) + sizeof(entropy->LLTable)); + ZSTD_STATIC_ASSERT(offsetof(ZSTD_entropyDTables_t, MLTable) == offsetof(ZSTD_entropyDTables_t, OFTable) + sizeof(entropy->OFTable)); + ZSTD_STATIC_ASSERT(sizeof(entropy->LLTable) + sizeof(entropy->OFTable) + sizeof(entropy->MLTable) >= HUF_DECOMPRESS_WORKSPACE_SIZE); + { void* const workspace = &entropy->LLTable; /* use fse tables as temporary workspace; implies fse tables are grouped together */ + size_t const workspaceSize = sizeof(entropy->LLTable) + sizeof(entropy->OFTable) + sizeof(entropy->MLTable); +#ifdef HUF_FORCE_DECOMPRESS_X1 + /* in minimal huffman, we always use X1 variants */ + size_t const hSize = HUF_readDTableX1_wksp(entropy->hufTable, + dictPtr, dictEnd - dictPtr, + workspace, workspaceSize); +#else + size_t const hSize = HUF_readDTableX2_wksp(entropy->hufTable, + dictPtr, dictEnd - dictPtr, + workspace, workspaceSize); +#endif + RETURN_ERROR_IF(HUF_isError(hSize), dictionary_corrupted, ""); + dictPtr += hSize; + } + + { short offcodeNCount[MaxOff+1]; + unsigned offcodeMaxValue = MaxOff, offcodeLog; + size_t const offcodeHeaderSize = FSE_readNCount(offcodeNCount, &offcodeMaxValue, &offcodeLog, dictPtr, dictEnd-dictPtr); + RETURN_ERROR_IF(FSE_isError(offcodeHeaderSize), dictionary_corrupted, ""); + RETURN_ERROR_IF(offcodeMaxValue > MaxOff, dictionary_corrupted, ""); + RETURN_ERROR_IF(offcodeLog > OffFSELog, dictionary_corrupted, ""); + ZSTD_buildFSETable( entropy->OFTable, + offcodeNCount, offcodeMaxValue, + OF_base, OF_bits, + offcodeLog); + dictPtr += offcodeHeaderSize; + } + + { short matchlengthNCount[MaxML+1]; + unsigned matchlengthMaxValue = MaxML, matchlengthLog; + size_t const matchlengthHeaderSize = FSE_readNCount(matchlengthNCount, &matchlengthMaxValue, &matchlengthLog, dictPtr, dictEnd-dictPtr); + RETURN_ERROR_IF(FSE_isError(matchlengthHeaderSize), dictionary_corrupted, ""); + RETURN_ERROR_IF(matchlengthMaxValue > MaxML, dictionary_corrupted, ""); + RETURN_ERROR_IF(matchlengthLog > MLFSELog, dictionary_corrupted, ""); + ZSTD_buildFSETable( entropy->MLTable, + matchlengthNCount, matchlengthMaxValue, + ML_base, ML_bits, + matchlengthLog); + dictPtr += matchlengthHeaderSize; + } + + { short litlengthNCount[MaxLL+1]; + unsigned litlengthMaxValue = MaxLL, litlengthLog; + size_t const litlengthHeaderSize = FSE_readNCount(litlengthNCount, &litlengthMaxValue, &litlengthLog, dictPtr, dictEnd-dictPtr); + RETURN_ERROR_IF(FSE_isError(litlengthHeaderSize), dictionary_corrupted, ""); + RETURN_ERROR_IF(litlengthMaxValue > MaxLL, dictionary_corrupted, ""); + RETURN_ERROR_IF(litlengthLog > LLFSELog, dictionary_corrupted, ""); + ZSTD_buildFSETable( entropy->LLTable, + litlengthNCount, litlengthMaxValue, + LL_base, LL_bits, + litlengthLog); + dictPtr += litlengthHeaderSize; + } + + RETURN_ERROR_IF(dictPtr+12 > dictEnd, dictionary_corrupted, ""); + { int i; + size_t const dictContentSize = (size_t)(dictEnd - (dictPtr+12)); + for (i=0; i<3; i++) { + U32 const rep = MEM_readLE32(dictPtr); dictPtr += 4; + RETURN_ERROR_IF(rep==0 || rep > dictContentSize, + dictionary_corrupted, ""); + entropy->rep[i] = rep; + } } + + return dictPtr - (const BYTE*)dict; +} + +static size_t ZSTD_decompress_insertDictionary(ZSTD_DCtx* dctx, const void* dict, size_t dictSize) +{ + if (dictSize < 8) return ZSTD_refDictContent(dctx, dict, dictSize); + { U32 const magic = MEM_readLE32(dict); + if (magic != ZSTD_MAGIC_DICTIONARY) { + return ZSTD_refDictContent(dctx, dict, dictSize); /* pure content mode */ + } } + dctx->dictID = MEM_readLE32((const char*)dict + ZSTD_FRAMEIDSIZE); + + /* load entropy tables */ + { size_t const eSize = ZSTD_loadDEntropy(&dctx->entropy, dict, dictSize); + RETURN_ERROR_IF(ZSTD_isError(eSize), dictionary_corrupted, ""); + dict = (const char*)dict + eSize; + dictSize -= eSize; + } + dctx->litEntropy = dctx->fseEntropy = 1; + + /* reference dictionary content */ + return ZSTD_refDictContent(dctx, dict, dictSize); +} + +size_t ZSTD_decompressBegin(ZSTD_DCtx* dctx) +{ + assert(dctx != NULL); + dctx->expected = ZSTD_startingInputLength(dctx->format); /* dctx->format must be properly set */ + dctx->stage = ZSTDds_getFrameHeaderSize; + dctx->decodedSize = 0; + dctx->previousDstEnd = NULL; + dctx->prefixStart = NULL; + dctx->virtualStart = NULL; + dctx->dictEnd = NULL; + dctx->entropy.hufTable[0] = (HUF_DTable)((HufLog)*0x1000001); /* cover both little and big endian */ + dctx->litEntropy = dctx->fseEntropy = 0; + dctx->dictID = 0; + dctx->bType = bt_reserved; + ZSTD_STATIC_ASSERT(sizeof(dctx->entropy.rep) == sizeof(repStartValue)); + memcpy(dctx->entropy.rep, repStartValue, sizeof(repStartValue)); /* initial repcodes */ + dctx->LLTptr = dctx->entropy.LLTable; + dctx->MLTptr = dctx->entropy.MLTable; + dctx->OFTptr = dctx->entropy.OFTable; + dctx->HUFptr = dctx->entropy.hufTable; + return 0; +} + +size_t ZSTD_decompressBegin_usingDict(ZSTD_DCtx* dctx, const void* dict, size_t dictSize) +{ + FORWARD_IF_ERROR( ZSTD_decompressBegin(dctx) , ""); + if (dict && dictSize) + RETURN_ERROR_IF( + ZSTD_isError(ZSTD_decompress_insertDictionary(dctx, dict, dictSize)), + dictionary_corrupted, ""); + return 0; +} + + +/* ====== ZSTD_DDict ====== */ + +size_t ZSTD_decompressBegin_usingDDict(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict) +{ + DEBUGLOG(4, "ZSTD_decompressBegin_usingDDict"); + assert(dctx != NULL); + if (ddict) { + const char* const dictStart = (const char*)ZSTD_DDict_dictContent(ddict); + size_t const dictSize = ZSTD_DDict_dictSize(ddict); + const void* const dictEnd = dictStart + dictSize; + dctx->ddictIsCold = (dctx->dictEnd != dictEnd); + DEBUGLOG(4, "DDict is %s", + dctx->ddictIsCold ? "~cold~" : "hot!"); + } + FORWARD_IF_ERROR( ZSTD_decompressBegin(dctx) , ""); + if (ddict) { /* NULL ddict is equivalent to no dictionary */ + ZSTD_copyDDictParameters(dctx, ddict); + } + return 0; +} + +/*! ZSTD_getDictID_fromDict() : + * Provides the dictID stored within dictionary. + * if @return == 0, the dictionary is not conformant with Zstandard specification. + * It can still be loaded, but as a content-only dictionary. */ +unsigned ZSTD_getDictID_fromDict(const void* dict, size_t dictSize) +{ + if (dictSize < 8) return 0; + if (MEM_readLE32(dict) != ZSTD_MAGIC_DICTIONARY) return 0; + return MEM_readLE32((const char*)dict + ZSTD_FRAMEIDSIZE); +} + +/*! ZSTD_getDictID_fromFrame() : + * Provides the dictID required to decompress frame stored within `src`. + * If @return == 0, the dictID could not be decoded. + * This could for one of the following reasons : + * - The frame does not require a dictionary (most common case). + * - The frame was built with dictID intentionally removed. + * Needed dictionary is a hidden information. + * Note : this use case also happens when using a non-conformant dictionary. + * - `srcSize` is too small, and as a result, frame header could not be decoded. + * Note : possible if `srcSize < ZSTD_FRAMEHEADERSIZE_MAX`. + * - This is not a Zstandard frame. + * When identifying the exact failure cause, it's possible to use + * ZSTD_getFrameHeader(), which will provide a more precise error code. */ +unsigned ZSTD_getDictID_fromFrame(const void* src, size_t srcSize) +{ + ZSTD_frameHeader zfp = { 0, 0, 0, ZSTD_frame, 0, 0, 0 }; + size_t const hError = ZSTD_getFrameHeader(&zfp, src, srcSize); + if (ZSTD_isError(hError)) return 0; + return zfp.dictID; +} + + +/*! ZSTD_decompress_usingDDict() : +* Decompression using a pre-digested Dictionary +* Use dictionary without significant overhead. */ +size_t ZSTD_decompress_usingDDict(ZSTD_DCtx* dctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, + const ZSTD_DDict* ddict) +{ + /* pass content and size in case legacy frames are encountered */ + return ZSTD_decompressMultiFrame(dctx, dst, dstCapacity, src, srcSize, + NULL, 0, + ddict); +} + + +/*===================================== +* Streaming decompression +*====================================*/ + +ZSTD_DStream* ZSTD_createDStream(void) +{ + DEBUGLOG(3, "ZSTD_createDStream"); + return ZSTD_createDStream_advanced(ZSTD_defaultCMem); +} + +ZSTD_DStream* ZSTD_initStaticDStream(void *workspace, size_t workspaceSize) +{ + return ZSTD_initStaticDCtx(workspace, workspaceSize); +} + +ZSTD_DStream* ZSTD_createDStream_advanced(ZSTD_customMem customMem) +{ + return ZSTD_createDCtx_advanced(customMem); +} + +size_t ZSTD_freeDStream(ZSTD_DStream* zds) +{ + return ZSTD_freeDCtx(zds); +} + + +/* *** Initialization *** */ + +size_t ZSTD_DStreamInSize(void) { return ZSTD_BLOCKSIZE_MAX + ZSTD_blockHeaderSize; } +size_t ZSTD_DStreamOutSize(void) { return ZSTD_BLOCKSIZE_MAX; } + +size_t ZSTD_DCtx_loadDictionary_advanced(ZSTD_DCtx* dctx, + const void* dict, size_t dictSize, + ZSTD_dictLoadMethod_e dictLoadMethod, + ZSTD_dictContentType_e dictContentType) +{ + RETURN_ERROR_IF(dctx->streamStage != zdss_init, stage_wrong, ""); + ZSTD_clearDict(dctx); + if (dict && dictSize != 0) { + dctx->ddictLocal = ZSTD_createDDict_advanced(dict, dictSize, dictLoadMethod, dictContentType, dctx->customMem); + RETURN_ERROR_IF(dctx->ddictLocal == NULL, memory_allocation, "NULL pointer!"); + dctx->ddict = dctx->ddictLocal; + dctx->dictUses = ZSTD_use_indefinitely; + } + return 0; +} + +size_t ZSTD_DCtx_loadDictionary_byReference(ZSTD_DCtx* dctx, const void* dict, size_t dictSize) +{ + return ZSTD_DCtx_loadDictionary_advanced(dctx, dict, dictSize, ZSTD_dlm_byRef, ZSTD_dct_auto); +} + +size_t ZSTD_DCtx_loadDictionary(ZSTD_DCtx* dctx, const void* dict, size_t dictSize) +{ + return ZSTD_DCtx_loadDictionary_advanced(dctx, dict, dictSize, ZSTD_dlm_byCopy, ZSTD_dct_auto); +} + +size_t ZSTD_DCtx_refPrefix_advanced(ZSTD_DCtx* dctx, const void* prefix, size_t prefixSize, ZSTD_dictContentType_e dictContentType) +{ + FORWARD_IF_ERROR(ZSTD_DCtx_loadDictionary_advanced(dctx, prefix, prefixSize, ZSTD_dlm_byRef, dictContentType), ""); + dctx->dictUses = ZSTD_use_once; + return 0; +} + +size_t ZSTD_DCtx_refPrefix(ZSTD_DCtx* dctx, const void* prefix, size_t prefixSize) +{ + return ZSTD_DCtx_refPrefix_advanced(dctx, prefix, prefixSize, ZSTD_dct_rawContent); +} + + +/* ZSTD_initDStream_usingDict() : + * return : expected size, aka ZSTD_startingInputLength(). + * this function cannot fail */ +size_t ZSTD_initDStream_usingDict(ZSTD_DStream* zds, const void* dict, size_t dictSize) +{ + DEBUGLOG(4, "ZSTD_initDStream_usingDict"); + FORWARD_IF_ERROR( ZSTD_DCtx_reset(zds, ZSTD_reset_session_only) , ""); + FORWARD_IF_ERROR( ZSTD_DCtx_loadDictionary(zds, dict, dictSize) , ""); + return ZSTD_startingInputLength(zds->format); +} + +/* note : this variant can't fail */ +size_t ZSTD_initDStream(ZSTD_DStream* zds) +{ + DEBUGLOG(4, "ZSTD_initDStream"); + return ZSTD_initDStream_usingDDict(zds, NULL); +} + +/* ZSTD_initDStream_usingDDict() : + * ddict will just be referenced, and must outlive decompression session + * this function cannot fail */ +size_t ZSTD_initDStream_usingDDict(ZSTD_DStream* dctx, const ZSTD_DDict* ddict) +{ + FORWARD_IF_ERROR( ZSTD_DCtx_reset(dctx, ZSTD_reset_session_only) , ""); + FORWARD_IF_ERROR( ZSTD_DCtx_refDDict(dctx, ddict) , ""); + return ZSTD_startingInputLength(dctx->format); +} + +/* ZSTD_resetDStream() : + * return : expected size, aka ZSTD_startingInputLength(). + * this function cannot fail */ +size_t ZSTD_resetDStream(ZSTD_DStream* dctx) +{ + FORWARD_IF_ERROR(ZSTD_DCtx_reset(dctx, ZSTD_reset_session_only), ""); + return ZSTD_startingInputLength(dctx->format); +} + + +size_t ZSTD_DCtx_refDDict(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict) +{ + RETURN_ERROR_IF(dctx->streamStage != zdss_init, stage_wrong, ""); + ZSTD_clearDict(dctx); + if (ddict) { + dctx->ddict = ddict; + dctx->dictUses = ZSTD_use_indefinitely; + } + return 0; +} + +/* ZSTD_DCtx_setMaxWindowSize() : + * note : no direct equivalence in ZSTD_DCtx_setParameter, + * since this version sets windowSize, and the other sets windowLog */ +size_t ZSTD_DCtx_setMaxWindowSize(ZSTD_DCtx* dctx, size_t maxWindowSize) +{ + ZSTD_bounds const bounds = ZSTD_dParam_getBounds(ZSTD_d_windowLogMax); + size_t const min = (size_t)1 << bounds.lowerBound; + size_t const max = (size_t)1 << bounds.upperBound; + RETURN_ERROR_IF(dctx->streamStage != zdss_init, stage_wrong, ""); + RETURN_ERROR_IF(maxWindowSize < min, parameter_outOfBound, ""); + RETURN_ERROR_IF(maxWindowSize > max, parameter_outOfBound, ""); + dctx->maxWindowSize = maxWindowSize; + return 0; +} + +size_t ZSTD_DCtx_setFormat(ZSTD_DCtx* dctx, ZSTD_format_e format) +{ + return ZSTD_DCtx_setParameter(dctx, ZSTD_d_format, format); +} + +ZSTD_bounds ZSTD_dParam_getBounds(ZSTD_dParameter dParam) +{ + ZSTD_bounds bounds = { 0, 0, 0 }; + switch(dParam) { + case ZSTD_d_windowLogMax: + bounds.lowerBound = ZSTD_WINDOWLOG_ABSOLUTEMIN; + bounds.upperBound = ZSTD_WINDOWLOG_MAX; + return bounds; + case ZSTD_d_format: + bounds.lowerBound = (int)ZSTD_f_zstd1; + bounds.upperBound = (int)ZSTD_f_zstd1_magicless; + ZSTD_STATIC_ASSERT(ZSTD_f_zstd1 < ZSTD_f_zstd1_magicless); + return bounds; + case ZSTD_d_stableOutBuffer: + bounds.lowerBound = (int)ZSTD_obm_buffered; + bounds.upperBound = (int)ZSTD_obm_stable; + return bounds; + default:; + } + bounds.error = ERROR(parameter_unsupported); + return bounds; +} + +/* ZSTD_dParam_withinBounds: + * @return 1 if value is within dParam bounds, + * 0 otherwise */ +static int ZSTD_dParam_withinBounds(ZSTD_dParameter dParam, int value) +{ + ZSTD_bounds const bounds = ZSTD_dParam_getBounds(dParam); + if (ZSTD_isError(bounds.error)) return 0; + if (value < bounds.lowerBound) return 0; + if (value > bounds.upperBound) return 0; + return 1; +} + +#define CHECK_DBOUNDS(p,v) { \ + RETURN_ERROR_IF(!ZSTD_dParam_withinBounds(p, v), parameter_outOfBound, ""); \ +} + +size_t ZSTD_DCtx_setParameter(ZSTD_DCtx* dctx, ZSTD_dParameter dParam, int value) +{ + RETURN_ERROR_IF(dctx->streamStage != zdss_init, stage_wrong, ""); + switch(dParam) { + case ZSTD_d_windowLogMax: + if (value == 0) value = ZSTD_WINDOWLOG_LIMIT_DEFAULT; + CHECK_DBOUNDS(ZSTD_d_windowLogMax, value); + dctx->maxWindowSize = ((size_t)1) << value; + return 0; + case ZSTD_d_format: + CHECK_DBOUNDS(ZSTD_d_format, value); + dctx->format = (ZSTD_format_e)value; + return 0; + case ZSTD_d_stableOutBuffer: + CHECK_DBOUNDS(ZSTD_d_stableOutBuffer, value); + dctx->outBufferMode = (ZSTD_outBufferMode_e)value; + return 0; + default:; + } + RETURN_ERROR(parameter_unsupported, ""); +} + +size_t ZSTD_DCtx_reset(ZSTD_DCtx* dctx, ZSTD_ResetDirective reset) +{ + if ( (reset == ZSTD_reset_session_only) + || (reset == ZSTD_reset_session_and_parameters) ) { + dctx->streamStage = zdss_init; + dctx->noForwardProgress = 0; + } + if ( (reset == ZSTD_reset_parameters) + || (reset == ZSTD_reset_session_and_parameters) ) { + RETURN_ERROR_IF(dctx->streamStage != zdss_init, stage_wrong, ""); + ZSTD_clearDict(dctx); + dctx->format = ZSTD_f_zstd1; + dctx->maxWindowSize = ZSTD_MAXWINDOWSIZE_DEFAULT; + } + return 0; +} + + +size_t ZSTD_sizeof_DStream(const ZSTD_DStream* dctx) +{ + return ZSTD_sizeof_DCtx(dctx); +} + +size_t ZSTD_decodingBufferSize_min(unsigned long long windowSize, unsigned long long frameContentSize) +{ + size_t const blockSize = (size_t) MIN(windowSize, ZSTD_BLOCKSIZE_MAX); + unsigned long long const neededRBSize = windowSize + blockSize + (WILDCOPY_OVERLENGTH * 2); + unsigned long long const neededSize = MIN(frameContentSize, neededRBSize); + size_t const minRBSize = (size_t) neededSize; + RETURN_ERROR_IF((unsigned long long)minRBSize != neededSize, + frameParameter_windowTooLarge, ""); + return minRBSize; +} + +size_t ZSTD_estimateDStreamSize(size_t windowSize) +{ + size_t const blockSize = MIN(windowSize, ZSTD_BLOCKSIZE_MAX); + size_t const inBuffSize = blockSize; /* no block can be larger */ + size_t const outBuffSize = ZSTD_decodingBufferSize_min(windowSize, ZSTD_CONTENTSIZE_UNKNOWN); + return ZSTD_estimateDCtxSize() + inBuffSize + outBuffSize; +} + +size_t ZSTD_estimateDStreamSize_fromFrame(const void* src, size_t srcSize) +{ + U32 const windowSizeMax = 1U << ZSTD_WINDOWLOG_MAX; /* note : should be user-selectable, but requires an additional parameter (or a dctx) */ + ZSTD_frameHeader zfh; + size_t const err = ZSTD_getFrameHeader(&zfh, src, srcSize); + if (ZSTD_isError(err)) return err; + RETURN_ERROR_IF(err>0, srcSize_wrong, ""); + RETURN_ERROR_IF(zfh.windowSize > windowSizeMax, + frameParameter_windowTooLarge, ""); + return ZSTD_estimateDStreamSize((size_t)zfh.windowSize); +} + + +/* ***** Decompression ***** */ + +static int ZSTD_DCtx_isOverflow(ZSTD_DStream* zds, size_t const neededInBuffSize, size_t const neededOutBuffSize) +{ + return (zds->inBuffSize + zds->outBuffSize) >= (neededInBuffSize + neededOutBuffSize) * ZSTD_WORKSPACETOOLARGE_FACTOR; +} + +static void ZSTD_DCtx_updateOversizedDuration(ZSTD_DStream* zds, size_t const neededInBuffSize, size_t const neededOutBuffSize) +{ + if (ZSTD_DCtx_isOverflow(zds, neededInBuffSize, neededOutBuffSize)) + zds->oversizedDuration++; + else + zds->oversizedDuration = 0; +} + +static int ZSTD_DCtx_isOversizedTooLong(ZSTD_DStream* zds) +{ + return zds->oversizedDuration >= ZSTD_WORKSPACETOOLARGE_MAXDURATION; +} + +/* Checks that the output buffer hasn't changed if ZSTD_obm_stable is used. */ +static size_t ZSTD_checkOutBuffer(ZSTD_DStream const* zds, ZSTD_outBuffer const* output) +{ + ZSTD_outBuffer const expect = zds->expectedOutBuffer; + /* No requirement when ZSTD_obm_stable is not enabled. */ + if (zds->outBufferMode != ZSTD_obm_stable) + return 0; + /* Any buffer is allowed in zdss_init, this must be the same for every other call until + * the context is reset. + */ + if (zds->streamStage == zdss_init) + return 0; + /* The buffer must match our expectation exactly. */ + if (expect.dst == output->dst && expect.pos == output->pos && expect.size == output->size) + return 0; + RETURN_ERROR(dstBuffer_wrong, "ZSTD_obm_stable enabled but output differs!"); +} + +/* Calls ZSTD_decompressContinue() with the right parameters for ZSTD_decompressStream() + * and updates the stage and the output buffer state. This call is extracted so it can be + * used both when reading directly from the ZSTD_inBuffer, and in buffered input mode. + * NOTE: You must break after calling this function since the streamStage is modified. + */ +static size_t ZSTD_decompressContinueStream( + ZSTD_DStream* zds, char** op, char* oend, + void const* src, size_t srcSize) { + int const isSkipFrame = ZSTD_isSkipFrame(zds); + if (zds->outBufferMode == ZSTD_obm_buffered) { + size_t const dstSize = isSkipFrame ? 0 : zds->outBuffSize - zds->outStart; + size_t const decodedSize = ZSTD_decompressContinue(zds, + zds->outBuff + zds->outStart, dstSize, src, srcSize); + FORWARD_IF_ERROR(decodedSize, ""); + if (!decodedSize && !isSkipFrame) { + zds->streamStage = zdss_read; + } else { + zds->outEnd = zds->outStart + decodedSize; + zds->streamStage = zdss_flush; + } + } else { + /* Write directly into the output buffer */ + size_t const dstSize = isSkipFrame ? 0 : oend - *op; + size_t const decodedSize = ZSTD_decompressContinue(zds, *op, dstSize, src, srcSize); + FORWARD_IF_ERROR(decodedSize, ""); + *op += decodedSize; + /* Flushing is not needed. */ + zds->streamStage = zdss_read; + assert(*op <= oend); + assert(zds->outBufferMode == ZSTD_obm_stable); + } + return 0; +} + +size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inBuffer* input) +{ + const char* const src = (const char*)input->src; + const char* const istart = input->pos != 0 ? src + input->pos : src; + const char* const iend = input->size != 0 ? src + input->size : src; + const char* ip = istart; + char* const dst = (char*)output->dst; + char* const ostart = output->pos != 0 ? dst + output->pos : dst; + char* const oend = output->size != 0 ? dst + output->size : dst; + char* op = ostart; + U32 someMoreWork = 1; + + DEBUGLOG(5, "ZSTD_decompressStream"); + RETURN_ERROR_IF( + input->pos > input->size, + srcSize_wrong, + "forbidden. in: pos: %u vs size: %u", + (U32)input->pos, (U32)input->size); + RETURN_ERROR_IF( + output->pos > output->size, + dstSize_tooSmall, + "forbidden. out: pos: %u vs size: %u", + (U32)output->pos, (U32)output->size); + DEBUGLOG(5, "input size : %u", (U32)(input->size - input->pos)); + FORWARD_IF_ERROR(ZSTD_checkOutBuffer(zds, output), ""); + + while (someMoreWork) { + switch(zds->streamStage) + { + case zdss_init : + DEBUGLOG(5, "stage zdss_init => transparent reset "); + zds->streamStage = zdss_loadHeader; + zds->lhSize = zds->inPos = zds->outStart = zds->outEnd = 0; + zds->legacyVersion = 0; + zds->hostageByte = 0; + zds->expectedOutBuffer = *output; + /* fall-through */ + + case zdss_loadHeader : + DEBUGLOG(5, "stage zdss_loadHeader (srcSize : %u)", (U32)(iend - ip)); +#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT>=1) + if (zds->legacyVersion) { + RETURN_ERROR_IF(zds->staticSize, memory_allocation, + "legacy support is incompatible with static dctx"); + { size_t const hint = ZSTD_decompressLegacyStream(zds->legacyContext, zds->legacyVersion, output, input); + if (hint==0) zds->streamStage = zdss_init; + return hint; + } } +#endif + { size_t const hSize = ZSTD_getFrameHeader_advanced(&zds->fParams, zds->headerBuffer, zds->lhSize, zds->format); + DEBUGLOG(5, "header size : %u", (U32)hSize); + if (ZSTD_isError(hSize)) { +#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT>=1) + U32 const legacyVersion = ZSTD_isLegacy(istart, iend-istart); + if (legacyVersion) { + ZSTD_DDict const* const ddict = ZSTD_getDDict(zds); + const void* const dict = ddict ? ZSTD_DDict_dictContent(ddict) : NULL; + size_t const dictSize = ddict ? ZSTD_DDict_dictSize(ddict) : 0; + DEBUGLOG(5, "ZSTD_decompressStream: detected legacy version v0.%u", legacyVersion); + RETURN_ERROR_IF(zds->staticSize, memory_allocation, + "legacy support is incompatible with static dctx"); + FORWARD_IF_ERROR(ZSTD_initLegacyStream(&zds->legacyContext, + zds->previousLegacyVersion, legacyVersion, + dict, dictSize), ""); + zds->legacyVersion = zds->previousLegacyVersion = legacyVersion; + { size_t const hint = ZSTD_decompressLegacyStream(zds->legacyContext, legacyVersion, output, input); + if (hint==0) zds->streamStage = zdss_init; /* or stay in stage zdss_loadHeader */ + return hint; + } } +#endif + return hSize; /* error */ + } + if (hSize != 0) { /* need more input */ + size_t const toLoad = hSize - zds->lhSize; /* if hSize!=0, hSize > zds->lhSize */ + size_t const remainingInput = (size_t)(iend-ip); + assert(iend >= ip); + if (toLoad > remainingInput) { /* not enough input to load full header */ + if (remainingInput > 0) { + memcpy(zds->headerBuffer + zds->lhSize, ip, remainingInput); + zds->lhSize += remainingInput; + } + input->pos = input->size; + return (MAX((size_t)ZSTD_FRAMEHEADERSIZE_MIN(zds->format), hSize) - zds->lhSize) + ZSTD_blockHeaderSize; /* remaining header bytes + next block header */ + } + assert(ip != NULL); + memcpy(zds->headerBuffer + zds->lhSize, ip, toLoad); zds->lhSize = hSize; ip += toLoad; + break; + } } + + /* check for single-pass mode opportunity */ + if (zds->fParams.frameContentSize != ZSTD_CONTENTSIZE_UNKNOWN + && zds->fParams.frameType != ZSTD_skippableFrame + && (U64)(size_t)(oend-op) >= zds->fParams.frameContentSize) { + size_t const cSize = ZSTD_findFrameCompressedSize(istart, iend-istart); + if (cSize <= (size_t)(iend-istart)) { + /* shortcut : using single-pass mode */ + size_t const decompressedSize = ZSTD_decompress_usingDDict(zds, op, oend-op, istart, cSize, ZSTD_getDDict(zds)); + if (ZSTD_isError(decompressedSize)) return decompressedSize; + DEBUGLOG(4, "shortcut to single-pass ZSTD_decompress_usingDDict()") + ip = istart + cSize; + op += decompressedSize; + zds->expected = 0; + zds->streamStage = zdss_init; + someMoreWork = 0; + break; + } } + + /* Check output buffer is large enough for ZSTD_odm_stable. */ + if (zds->outBufferMode == ZSTD_obm_stable + && zds->fParams.frameType != ZSTD_skippableFrame + && zds->fParams.frameContentSize != ZSTD_CONTENTSIZE_UNKNOWN + && (U64)(size_t)(oend-op) < zds->fParams.frameContentSize) { + RETURN_ERROR(dstSize_tooSmall, "ZSTD_obm_stable passed but ZSTD_outBuffer is too small"); + } + + /* Consume header (see ZSTDds_decodeFrameHeader) */ + DEBUGLOG(4, "Consume header"); + FORWARD_IF_ERROR(ZSTD_decompressBegin_usingDDict(zds, ZSTD_getDDict(zds)), ""); + + if ((MEM_readLE32(zds->headerBuffer) & ZSTD_MAGIC_SKIPPABLE_MASK) == ZSTD_MAGIC_SKIPPABLE_START) { /* skippable frame */ + zds->expected = MEM_readLE32(zds->headerBuffer + ZSTD_FRAMEIDSIZE); + zds->stage = ZSTDds_skipFrame; + } else { + FORWARD_IF_ERROR(ZSTD_decodeFrameHeader(zds, zds->headerBuffer, zds->lhSize), ""); + zds->expected = ZSTD_blockHeaderSize; + zds->stage = ZSTDds_decodeBlockHeader; + } + + /* control buffer memory usage */ + DEBUGLOG(4, "Control max memory usage (%u KB <= max %u KB)", + (U32)(zds->fParams.windowSize >>10), + (U32)(zds->maxWindowSize >> 10) ); + zds->fParams.windowSize = MAX(zds->fParams.windowSize, 1U << ZSTD_WINDOWLOG_ABSOLUTEMIN); + RETURN_ERROR_IF(zds->fParams.windowSize > zds->maxWindowSize, + frameParameter_windowTooLarge, ""); + + /* Adapt buffer sizes to frame header instructions */ + { size_t const neededInBuffSize = MAX(zds->fParams.blockSizeMax, 4 /* frame checksum */); + size_t const neededOutBuffSize = zds->outBufferMode == ZSTD_obm_buffered + ? ZSTD_decodingBufferSize_min(zds->fParams.windowSize, zds->fParams.frameContentSize) + : 0; + + ZSTD_DCtx_updateOversizedDuration(zds, neededInBuffSize, neededOutBuffSize); + + { int const tooSmall = (zds->inBuffSize < neededInBuffSize) || (zds->outBuffSize < neededOutBuffSize); + int const tooLarge = ZSTD_DCtx_isOversizedTooLong(zds); + + if (tooSmall || tooLarge) { + size_t const bufferSize = neededInBuffSize + neededOutBuffSize; + DEBUGLOG(4, "inBuff : from %u to %u", + (U32)zds->inBuffSize, (U32)neededInBuffSize); + DEBUGLOG(4, "outBuff : from %u to %u", + (U32)zds->outBuffSize, (U32)neededOutBuffSize); + if (zds->staticSize) { /* static DCtx */ + DEBUGLOG(4, "staticSize : %u", (U32)zds->staticSize); + assert(zds->staticSize >= sizeof(ZSTD_DCtx)); /* controlled at init */ + RETURN_ERROR_IF( + bufferSize > zds->staticSize - sizeof(ZSTD_DCtx), + memory_allocation, ""); + } else { + ZSTD_free(zds->inBuff, zds->customMem); + zds->inBuffSize = 0; + zds->outBuffSize = 0; + zds->inBuff = (char*)ZSTD_malloc(bufferSize, zds->customMem); + RETURN_ERROR_IF(zds->inBuff == NULL, memory_allocation, ""); + } + zds->inBuffSize = neededInBuffSize; + zds->outBuff = zds->inBuff + zds->inBuffSize; + zds->outBuffSize = neededOutBuffSize; + } } } + zds->streamStage = zdss_read; + /* fall-through */ + + case zdss_read: + DEBUGLOG(5, "stage zdss_read"); + { size_t const neededInSize = ZSTD_nextSrcSizeToDecompressWithInputSize(zds, iend - ip); + DEBUGLOG(5, "neededInSize = %u", (U32)neededInSize); + if (neededInSize==0) { /* end of frame */ + zds->streamStage = zdss_init; + someMoreWork = 0; + break; + } + if ((size_t)(iend-ip) >= neededInSize) { /* decode directly from src */ + FORWARD_IF_ERROR(ZSTD_decompressContinueStream(zds, &op, oend, ip, neededInSize), ""); + ip += neededInSize; + /* Function modifies the stage so we must break */ + break; + } } + if (ip==iend) { someMoreWork = 0; break; } /* no more input */ + zds->streamStage = zdss_load; + /* fall-through */ + + case zdss_load: + { size_t const neededInSize = ZSTD_nextSrcSizeToDecompress(zds); + size_t const toLoad = neededInSize - zds->inPos; + int const isSkipFrame = ZSTD_isSkipFrame(zds); + size_t loadedSize; + /* At this point we shouldn't be decompressing a block that we can stream. */ + assert(neededInSize == ZSTD_nextSrcSizeToDecompressWithInputSize(zds, iend - ip)); + if (isSkipFrame) { + loadedSize = MIN(toLoad, (size_t)(iend-ip)); + } else { + RETURN_ERROR_IF(toLoad > zds->inBuffSize - zds->inPos, + corruption_detected, + "should never happen"); + loadedSize = ZSTD_limitCopy(zds->inBuff + zds->inPos, toLoad, ip, iend-ip); + } + ip += loadedSize; + zds->inPos += loadedSize; + if (loadedSize < toLoad) { someMoreWork = 0; break; } /* not enough input, wait for more */ + + /* decode loaded input */ + zds->inPos = 0; /* input is consumed */ + FORWARD_IF_ERROR(ZSTD_decompressContinueStream(zds, &op, oend, zds->inBuff, neededInSize), ""); + /* Function modifies the stage so we must break */ + break; + } + case zdss_flush: + { size_t const toFlushSize = zds->outEnd - zds->outStart; + size_t const flushedSize = ZSTD_limitCopy(op, oend-op, zds->outBuff + zds->outStart, toFlushSize); + op += flushedSize; + zds->outStart += flushedSize; + if (flushedSize == toFlushSize) { /* flush completed */ + zds->streamStage = zdss_read; + if ( (zds->outBuffSize < zds->fParams.frameContentSize) + && (zds->outStart + zds->fParams.blockSizeMax > zds->outBuffSize) ) { + DEBUGLOG(5, "restart filling outBuff from beginning (left:%i, needed:%u)", + (int)(zds->outBuffSize - zds->outStart), + (U32)zds->fParams.blockSizeMax); + zds->outStart = zds->outEnd = 0; + } + break; + } } + /* cannot complete flush */ + someMoreWork = 0; + break; + + default: + assert(0); /* impossible */ + RETURN_ERROR(GENERIC, "impossible to reach"); /* some compiler require default to do something */ + } } + + /* result */ + input->pos = (size_t)(ip - (const char*)(input->src)); + output->pos = (size_t)(op - (char*)(output->dst)); + + /* Update the expected output buffer for ZSTD_obm_stable. */ + zds->expectedOutBuffer = *output; + + if ((ip==istart) && (op==ostart)) { /* no forward progress */ + zds->noForwardProgress ++; + if (zds->noForwardProgress >= ZSTD_NO_FORWARD_PROGRESS_MAX) { + RETURN_ERROR_IF(op==oend, dstSize_tooSmall, ""); + RETURN_ERROR_IF(ip==iend, srcSize_wrong, ""); + assert(0); + } + } else { + zds->noForwardProgress = 0; + } + { size_t nextSrcSizeHint = ZSTD_nextSrcSizeToDecompress(zds); + if (!nextSrcSizeHint) { /* frame fully decoded */ + if (zds->outEnd == zds->outStart) { /* output fully flushed */ + if (zds->hostageByte) { + if (input->pos >= input->size) { + /* can't release hostage (not present) */ + zds->streamStage = zdss_read; + return 1; + } + input->pos++; /* release hostage */ + } /* zds->hostageByte */ + return 0; + } /* zds->outEnd == zds->outStart */ + if (!zds->hostageByte) { /* output not fully flushed; keep last byte as hostage; will be released when all output is flushed */ + input->pos--; /* note : pos > 0, otherwise, impossible to finish reading last block */ + zds->hostageByte=1; + } + return 1; + } /* nextSrcSizeHint==0 */ + nextSrcSizeHint += ZSTD_blockHeaderSize * (ZSTD_nextInputType(zds) == ZSTDnit_block); /* preload header of next block */ + assert(zds->inPos <= nextSrcSizeHint); + nextSrcSizeHint -= zds->inPos; /* part already loaded*/ + return nextSrcSizeHint; + } +} + +size_t ZSTD_decompressStream_simpleArgs ( + ZSTD_DCtx* dctx, + void* dst, size_t dstCapacity, size_t* dstPos, + const void* src, size_t srcSize, size_t* srcPos) +{ + ZSTD_outBuffer output = { dst, dstCapacity, *dstPos }; + ZSTD_inBuffer input = { src, srcSize, *srcPos }; + /* ZSTD_compress_generic() will check validity of dstPos and srcPos */ + size_t const cErr = ZSTD_decompressStream(dctx, &output, &input); + *dstPos = output.pos; + *srcPos = input.pos; + return cErr; +} diff --git a/module/zstd/lib/decompress/zstd_decompress_block.c b/module/zstd/lib/decompress/zstd_decompress_block.c new file mode 100644 index 000000000000..ad3b3d8dbd0a --- /dev/null +++ b/module/zstd/lib/decompress/zstd_decompress_block.c @@ -0,0 +1,1432 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + +/* zstd_decompress_block : + * this module takes care of decompressing _compressed_ block */ + +/*-******************************************************* +* Dependencies +*********************************************************/ +#include /* memcpy, memmove, memset */ +#include "../common/compiler.h" /* prefetch */ +#include "../common/cpu.h" /* bmi2 */ +#include "../common/mem.h" /* low level memory routines */ +#define FSE_STATIC_LINKING_ONLY +#include "../common/fse.h" +#define HUF_STATIC_LINKING_ONLY +#include "../common/huf.h" +#include "../common/zstd_internal.h" +#include "zstd_decompress_internal.h" /* ZSTD_DCtx */ +#include "zstd_ddict.h" /* ZSTD_DDictDictContent */ +#include "zstd_decompress_block.h" + +/*_******************************************************* +* Macros +**********************************************************/ + +/* These two optional macros force the use one way or another of the two + * ZSTD_decompressSequences implementations. You can't force in both directions + * at the same time. + */ +#if defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT) && \ + defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG) +#error "Cannot force the use of the short and the long ZSTD_decompressSequences variants!" +#endif + + +/*_******************************************************* +* Memory operations +**********************************************************/ +static void ZSTD_copy4(void* dst, const void* src) { memcpy(dst, src, 4); } + + +/*-************************************************************* + * Block decoding + ***************************************************************/ + +/*! ZSTD_getcBlockSize() : + * Provides the size of compressed block from block header `src` */ +size_t ZSTD_getcBlockSize(const void* src, size_t srcSize, + blockProperties_t* bpPtr) +{ + RETURN_ERROR_IF(srcSize < ZSTD_blockHeaderSize, srcSize_wrong, ""); + + { U32 const cBlockHeader = MEM_readLE24(src); + U32 const cSize = cBlockHeader >> 3; + bpPtr->lastBlock = cBlockHeader & 1; + bpPtr->blockType = (blockType_e)((cBlockHeader >> 1) & 3); + bpPtr->origSize = cSize; /* only useful for RLE */ + if (bpPtr->blockType == bt_rle) return 1; + RETURN_ERROR_IF(bpPtr->blockType == bt_reserved, corruption_detected, ""); + return cSize; + } +} + + +/* Hidden declaration for fullbench */ +size_t ZSTD_decodeLiteralsBlock(ZSTD_DCtx* dctx, + const void* src, size_t srcSize); +/*! ZSTD_decodeLiteralsBlock() : + * @return : nb of bytes read from src (< srcSize ) + * note : symbol not declared but exposed for fullbench */ +size_t ZSTD_decodeLiteralsBlock(ZSTD_DCtx* dctx, + const void* src, size_t srcSize) /* note : srcSize < BLOCKSIZE */ +{ + DEBUGLOG(5, "ZSTD_decodeLiteralsBlock"); + RETURN_ERROR_IF(srcSize < MIN_CBLOCK_SIZE, corruption_detected, ""); + + { const BYTE* const istart = (const BYTE*) src; + symbolEncodingType_e const litEncType = (symbolEncodingType_e)(istart[0] & 3); + + switch(litEncType) + { + case set_repeat: + DEBUGLOG(5, "set_repeat flag : re-using stats from previous compressed literals block"); + RETURN_ERROR_IF(dctx->litEntropy==0, dictionary_corrupted, ""); + /* fall-through */ + + case set_compressed: + RETURN_ERROR_IF(srcSize < 5, corruption_detected, "srcSize >= MIN_CBLOCK_SIZE == 3; here we need up to 5 for case 3"); + { size_t lhSize, litSize, litCSize; + U32 singleStream=0; + U32 const lhlCode = (istart[0] >> 2) & 3; + U32 const lhc = MEM_readLE32(istart); + size_t hufSuccess; + switch(lhlCode) + { + case 0: case 1: default: /* note : default is impossible, since lhlCode into [0..3] */ + /* 2 - 2 - 10 - 10 */ + singleStream = !lhlCode; + lhSize = 3; + litSize = (lhc >> 4) & 0x3FF; + litCSize = (lhc >> 14) & 0x3FF; + break; + case 2: + /* 2 - 2 - 14 - 14 */ + lhSize = 4; + litSize = (lhc >> 4) & 0x3FFF; + litCSize = lhc >> 18; + break; + case 3: + /* 2 - 2 - 18 - 18 */ + lhSize = 5; + litSize = (lhc >> 4) & 0x3FFFF; + litCSize = (lhc >> 22) + ((size_t)istart[4] << 10); + break; + } + RETURN_ERROR_IF(litSize > ZSTD_BLOCKSIZE_MAX, corruption_detected, ""); + RETURN_ERROR_IF(litCSize + lhSize > srcSize, corruption_detected, ""); + + /* prefetch huffman table if cold */ + if (dctx->ddictIsCold && (litSize > 768 /* heuristic */)) { + PREFETCH_AREA(dctx->HUFptr, sizeof(dctx->entropy.hufTable)); + } + + if (litEncType==set_repeat) { + if (singleStream) { + hufSuccess = HUF_decompress1X_usingDTable_bmi2( + dctx->litBuffer, litSize, istart+lhSize, litCSize, + dctx->HUFptr, dctx->bmi2); + } else { + hufSuccess = HUF_decompress4X_usingDTable_bmi2( + dctx->litBuffer, litSize, istart+lhSize, litCSize, + dctx->HUFptr, dctx->bmi2); + } + } else { + if (singleStream) { +#if defined(HUF_FORCE_DECOMPRESS_X2) + hufSuccess = HUF_decompress1X_DCtx_wksp( + dctx->entropy.hufTable, dctx->litBuffer, litSize, + istart+lhSize, litCSize, dctx->workspace, + sizeof(dctx->workspace)); +#else + hufSuccess = HUF_decompress1X1_DCtx_wksp_bmi2( + dctx->entropy.hufTable, dctx->litBuffer, litSize, + istart+lhSize, litCSize, dctx->workspace, + sizeof(dctx->workspace), dctx->bmi2); +#endif + } else { + hufSuccess = HUF_decompress4X_hufOnly_wksp_bmi2( + dctx->entropy.hufTable, dctx->litBuffer, litSize, + istart+lhSize, litCSize, dctx->workspace, + sizeof(dctx->workspace), dctx->bmi2); + } + } + + RETURN_ERROR_IF(HUF_isError(hufSuccess), corruption_detected, ""); + + dctx->litPtr = dctx->litBuffer; + dctx->litSize = litSize; + dctx->litEntropy = 1; + if (litEncType==set_compressed) dctx->HUFptr = dctx->entropy.hufTable; + memset(dctx->litBuffer + dctx->litSize, 0, WILDCOPY_OVERLENGTH); + return litCSize + lhSize; + } + + case set_basic: + { size_t litSize, lhSize; + U32 const lhlCode = ((istart[0]) >> 2) & 3; + switch(lhlCode) + { + case 0: case 2: default: /* note : default is impossible, since lhlCode into [0..3] */ + lhSize = 1; + litSize = istart[0] >> 3; + break; + case 1: + lhSize = 2; + litSize = MEM_readLE16(istart) >> 4; + break; + case 3: + lhSize = 3; + litSize = MEM_readLE24(istart) >> 4; + break; + } + + if (lhSize+litSize+WILDCOPY_OVERLENGTH > srcSize) { /* risk reading beyond src buffer with wildcopy */ + RETURN_ERROR_IF(litSize+lhSize > srcSize, corruption_detected, ""); + memcpy(dctx->litBuffer, istart+lhSize, litSize); + dctx->litPtr = dctx->litBuffer; + dctx->litSize = litSize; + memset(dctx->litBuffer + dctx->litSize, 0, WILDCOPY_OVERLENGTH); + return lhSize+litSize; + } + /* direct reference into compressed stream */ + dctx->litPtr = istart+lhSize; + dctx->litSize = litSize; + return lhSize+litSize; + } + + case set_rle: + { U32 const lhlCode = ((istart[0]) >> 2) & 3; + size_t litSize, lhSize; + switch(lhlCode) + { + case 0: case 2: default: /* note : default is impossible, since lhlCode into [0..3] */ + lhSize = 1; + litSize = istart[0] >> 3; + break; + case 1: + lhSize = 2; + litSize = MEM_readLE16(istart) >> 4; + break; + case 3: + lhSize = 3; + litSize = MEM_readLE24(istart) >> 4; + RETURN_ERROR_IF(srcSize<4, corruption_detected, "srcSize >= MIN_CBLOCK_SIZE == 3; here we need lhSize+1 = 4"); + break; + } + RETURN_ERROR_IF(litSize > ZSTD_BLOCKSIZE_MAX, corruption_detected, ""); + memset(dctx->litBuffer, istart[lhSize], litSize + WILDCOPY_OVERLENGTH); + dctx->litPtr = dctx->litBuffer; + dctx->litSize = litSize; + return lhSize+1; + } + default: + RETURN_ERROR(corruption_detected, "impossible"); + } + } +} + +/* Default FSE distribution tables. + * These are pre-calculated FSE decoding tables using default distributions as defined in specification : + * https://github.com/facebook/zstd/blob/master/doc/zstd_compression_format.md#default-distributions + * They were generated programmatically with following method : + * - start from default distributions, present in /lib/common/zstd_internal.h + * - generate tables normally, using ZSTD_buildFSETable() + * - printout the content of tables + * - pretify output, report below, test with fuzzer to ensure it's correct */ + +/* Default FSE distribution table for Literal Lengths */ +static const ZSTD_seqSymbol LL_defaultDTable[(1<tableLog = 0; + DTableH->fastMode = 0; + + cell->nbBits = 0; + cell->nextState = 0; + assert(nbAddBits < 255); + cell->nbAdditionalBits = (BYTE)nbAddBits; + cell->baseValue = baseValue; +} + + +/* ZSTD_buildFSETable() : + * generate FSE decoding table for one symbol (ll, ml or off) + * cannot fail if input is valid => + * all inputs are presumed validated at this stage */ +void +ZSTD_buildFSETable(ZSTD_seqSymbol* dt, + const short* normalizedCounter, unsigned maxSymbolValue, + const U32* baseValue, const U32* nbAdditionalBits, + unsigned tableLog) +{ + ZSTD_seqSymbol* const tableDecode = dt+1; + U16 symbolNext[MaxSeq+1]; + + U32 const maxSV1 = maxSymbolValue + 1; + U32 const tableSize = 1 << tableLog; + U32 highThreshold = tableSize-1; + + /* Sanity Checks */ + assert(maxSymbolValue <= MaxSeq); + assert(tableLog <= MaxFSELog); + + /* Init, lay down lowprob symbols */ + { ZSTD_seqSymbol_header DTableH; + DTableH.tableLog = tableLog; + DTableH.fastMode = 1; + { S16 const largeLimit= (S16)(1 << (tableLog-1)); + U32 s; + for (s=0; s= largeLimit) DTableH.fastMode=0; + assert(normalizedCounter[s]>=0); + symbolNext[s] = (U16)normalizedCounter[s]; + } } } + memcpy(dt, &DTableH, sizeof(DTableH)); + } + + /* Spread symbols */ + { U32 const tableMask = tableSize-1; + U32 const step = FSE_TABLESTEP(tableSize); + U32 s, position = 0; + for (s=0; s highThreshold) position = (position + step) & tableMask; /* lowprob area */ + } } + assert(position == 0); /* position must reach all cells once, otherwise normalizedCounter is incorrect */ + } + + /* Build Decoding table */ + { U32 u; + for (u=0; u max, corruption_detected, ""); + { U32 const symbol = *(const BYTE*)src; + U32 const baseline = baseValue[symbol]; + U32 const nbBits = nbAdditionalBits[symbol]; + ZSTD_buildSeqTable_rle(DTableSpace, baseline, nbBits); + } + *DTablePtr = DTableSpace; + return 1; + case set_basic : + *DTablePtr = defaultTable; + return 0; + case set_repeat: + RETURN_ERROR_IF(!flagRepeatTable, corruption_detected, ""); + /* prefetch FSE table if used */ + if (ddictIsCold && (nbSeq > 24 /* heuristic */)) { + const void* const pStart = *DTablePtr; + size_t const pSize = sizeof(ZSTD_seqSymbol) * (SEQSYMBOL_TABLE_SIZE(maxLog)); + PREFETCH_AREA(pStart, pSize); + } + return 0; + case set_compressed : + { unsigned tableLog; + S16 norm[MaxSeq+1]; + size_t const headerSize = FSE_readNCount(norm, &max, &tableLog, src, srcSize); + RETURN_ERROR_IF(FSE_isError(headerSize), corruption_detected, ""); + RETURN_ERROR_IF(tableLog > maxLog, corruption_detected, ""); + ZSTD_buildFSETable(DTableSpace, norm, max, baseValue, nbAdditionalBits, tableLog); + *DTablePtr = DTableSpace; + return headerSize; + } + default : + assert(0); + RETURN_ERROR(GENERIC, "impossible"); + } +} + +size_t ZSTD_decodeSeqHeaders(ZSTD_DCtx* dctx, int* nbSeqPtr, + const void* src, size_t srcSize) +{ + const BYTE* const istart = (const BYTE* const)src; + const BYTE* const iend = istart + srcSize; + const BYTE* ip = istart; + int nbSeq; + DEBUGLOG(5, "ZSTD_decodeSeqHeaders"); + + /* check */ + RETURN_ERROR_IF(srcSize < MIN_SEQUENCES_SIZE, srcSize_wrong, ""); + + /* SeqHead */ + nbSeq = *ip++; + if (!nbSeq) { + *nbSeqPtr=0; + RETURN_ERROR_IF(srcSize != 1, srcSize_wrong, ""); + return 1; + } + if (nbSeq > 0x7F) { + if (nbSeq == 0xFF) { + RETURN_ERROR_IF(ip+2 > iend, srcSize_wrong, ""); + nbSeq = MEM_readLE16(ip) + LONGNBSEQ, ip+=2; + } else { + RETURN_ERROR_IF(ip >= iend, srcSize_wrong, ""); + nbSeq = ((nbSeq-0x80)<<8) + *ip++; + } + } + *nbSeqPtr = nbSeq; + + /* FSE table descriptors */ + RETURN_ERROR_IF(ip+1 > iend, srcSize_wrong, ""); /* minimum possible size: 1 byte for symbol encoding types */ + { symbolEncodingType_e const LLtype = (symbolEncodingType_e)(*ip >> 6); + symbolEncodingType_e const OFtype = (symbolEncodingType_e)((*ip >> 4) & 3); + symbolEncodingType_e const MLtype = (symbolEncodingType_e)((*ip >> 2) & 3); + ip++; + + /* Build DTables */ + { size_t const llhSize = ZSTD_buildSeqTable(dctx->entropy.LLTable, &dctx->LLTptr, + LLtype, MaxLL, LLFSELog, + ip, iend-ip, + LL_base, LL_bits, + LL_defaultDTable, dctx->fseEntropy, + dctx->ddictIsCold, nbSeq); + RETURN_ERROR_IF(ZSTD_isError(llhSize), corruption_detected, "ZSTD_buildSeqTable failed"); + ip += llhSize; + } + + { size_t const ofhSize = ZSTD_buildSeqTable(dctx->entropy.OFTable, &dctx->OFTptr, + OFtype, MaxOff, OffFSELog, + ip, iend-ip, + OF_base, OF_bits, + OF_defaultDTable, dctx->fseEntropy, + dctx->ddictIsCold, nbSeq); + RETURN_ERROR_IF(ZSTD_isError(ofhSize), corruption_detected, "ZSTD_buildSeqTable failed"); + ip += ofhSize; + } + + { size_t const mlhSize = ZSTD_buildSeqTable(dctx->entropy.MLTable, &dctx->MLTptr, + MLtype, MaxML, MLFSELog, + ip, iend-ip, + ML_base, ML_bits, + ML_defaultDTable, dctx->fseEntropy, + dctx->ddictIsCold, nbSeq); + RETURN_ERROR_IF(ZSTD_isError(mlhSize), corruption_detected, "ZSTD_buildSeqTable failed"); + ip += mlhSize; + } + } + + return ip-istart; +} + + +typedef struct { + size_t litLength; + size_t matchLength; + size_t offset; + const BYTE* match; +} seq_t; + +typedef struct { + size_t state; + const ZSTD_seqSymbol* table; +} ZSTD_fseState; + +typedef struct { + BIT_DStream_t DStream; + ZSTD_fseState stateLL; + ZSTD_fseState stateOffb; + ZSTD_fseState stateML; + size_t prevOffset[ZSTD_REP_NUM]; + const BYTE* prefixStart; + const BYTE* dictEnd; + size_t pos; +} seqState_t; + +/*! ZSTD_overlapCopy8() : + * Copies 8 bytes from ip to op and updates op and ip where ip <= op. + * If the offset is < 8 then the offset is spread to at least 8 bytes. + * + * Precondition: *ip <= *op + * Postcondition: *op - *op >= 8 + */ +HINT_INLINE void ZSTD_overlapCopy8(BYTE** op, BYTE const** ip, size_t offset) { + assert(*ip <= *op); + if (offset < 8) { + /* close range match, overlap */ + static const U32 dec32table[] = { 0, 1, 2, 1, 4, 4, 4, 4 }; /* added */ + static const int dec64table[] = { 8, 8, 8, 7, 8, 9,10,11 }; /* subtracted */ + int const sub2 = dec64table[offset]; + (*op)[0] = (*ip)[0]; + (*op)[1] = (*ip)[1]; + (*op)[2] = (*ip)[2]; + (*op)[3] = (*ip)[3]; + *ip += dec32table[offset]; + ZSTD_copy4(*op+4, *ip); + *ip -= sub2; + } else { + ZSTD_copy8(*op, *ip); + } + *ip += 8; + *op += 8; + assert(*op - *ip >= 8); +} + +/*! ZSTD_safecopy() : + * Specialized version of memcpy() that is allowed to READ up to WILDCOPY_OVERLENGTH past the input buffer + * and write up to 16 bytes past oend_w (op >= oend_w is allowed). + * This function is only called in the uncommon case where the sequence is near the end of the block. It + * should be fast for a single long sequence, but can be slow for several short sequences. + * + * @param ovtype controls the overlap detection + * - ZSTD_no_overlap: The source and destination are guaranteed to be at least WILDCOPY_VECLEN bytes apart. + * - ZSTD_overlap_src_before_dst: The src and dst may overlap and may be any distance apart. + * The src buffer must be before the dst buffer. + */ +static void ZSTD_safecopy(BYTE* op, BYTE* const oend_w, BYTE const* ip, ptrdiff_t length, ZSTD_overlap_e ovtype) { + ptrdiff_t const diff = op - ip; + BYTE* const oend = op + length; + + assert((ovtype == ZSTD_no_overlap && (diff <= -8 || diff >= 8 || op >= oend_w)) || + (ovtype == ZSTD_overlap_src_before_dst && diff >= 0)); + + if (length < 8) { + /* Handle short lengths. */ + while (op < oend) *op++ = *ip++; + return; + } + if (ovtype == ZSTD_overlap_src_before_dst) { + /* Copy 8 bytes and ensure the offset >= 8 when there can be overlap. */ + assert(length >= 8); + ZSTD_overlapCopy8(&op, &ip, diff); + assert(op - ip >= 8); + assert(op <= oend); + } + + if (oend <= oend_w) { + /* No risk of overwrite. */ + ZSTD_wildcopy(op, ip, length, ovtype); + return; + } + if (op <= oend_w) { + /* Wildcopy until we get close to the end. */ + assert(oend > oend_w); + ZSTD_wildcopy(op, ip, oend_w - op, ovtype); + ip += oend_w - op; + op = oend_w; + } + /* Handle the leftovers. */ + while (op < oend) *op++ = *ip++; +} + +/* ZSTD_execSequenceEnd(): + * This version handles cases that are near the end of the output buffer. It requires + * more careful checks to make sure there is no overflow. By separating out these hard + * and unlikely cases, we can speed up the common cases. + * + * NOTE: This function needs to be fast for a single long sequence, but doesn't need + * to be optimized for many small sequences, since those fall into ZSTD_execSequence(). + */ +FORCE_NOINLINE +size_t ZSTD_execSequenceEnd(BYTE* op, + BYTE* const oend, seq_t sequence, + const BYTE** litPtr, const BYTE* const litLimit, + const BYTE* const prefixStart, const BYTE* const virtualStart, const BYTE* const dictEnd) +{ + BYTE* const oLitEnd = op + sequence.litLength; + size_t const sequenceLength = sequence.litLength + sequence.matchLength; + const BYTE* const iLitEnd = *litPtr + sequence.litLength; + const BYTE* match = oLitEnd - sequence.offset; + BYTE* const oend_w = oend - WILDCOPY_OVERLENGTH; + + /* bounds checks : careful of address space overflow in 32-bit mode */ + RETURN_ERROR_IF(sequenceLength > (size_t)(oend - op), dstSize_tooSmall, "last match must fit within dstBuffer"); + RETURN_ERROR_IF(sequence.litLength > (size_t)(litLimit - *litPtr), corruption_detected, "try to read beyond literal buffer"); + assert(op < op + sequenceLength); + assert(oLitEnd < op + sequenceLength); + + /* copy literals */ + ZSTD_safecopy(op, oend_w, *litPtr, sequence.litLength, ZSTD_no_overlap); + op = oLitEnd; + *litPtr = iLitEnd; + + /* copy Match */ + if (sequence.offset > (size_t)(oLitEnd - prefixStart)) { + /* offset beyond prefix */ + RETURN_ERROR_IF(sequence.offset > (size_t)(oLitEnd - virtualStart), corruption_detected, ""); + match = dictEnd - (prefixStart-match); + if (match + sequence.matchLength <= dictEnd) { + memmove(oLitEnd, match, sequence.matchLength); + return sequenceLength; + } + /* span extDict & currentPrefixSegment */ + { size_t const length1 = dictEnd - match; + memmove(oLitEnd, match, length1); + op = oLitEnd + length1; + sequence.matchLength -= length1; + match = prefixStart; + } } + ZSTD_safecopy(op, oend_w, match, sequence.matchLength, ZSTD_overlap_src_before_dst); + return sequenceLength; +} + +HINT_INLINE +size_t ZSTD_execSequence(BYTE* op, + BYTE* const oend, seq_t sequence, + const BYTE** litPtr, const BYTE* const litLimit, + const BYTE* const prefixStart, const BYTE* const virtualStart, const BYTE* const dictEnd) +{ + BYTE* const oLitEnd = op + sequence.litLength; + size_t const sequenceLength = sequence.litLength + sequence.matchLength; + BYTE* const oMatchEnd = op + sequenceLength; /* risk : address space overflow (32-bits) */ + BYTE* const oend_w = oend - WILDCOPY_OVERLENGTH; /* risk : address space underflow on oend=NULL */ + const BYTE* const iLitEnd = *litPtr + sequence.litLength; + const BYTE* match = oLitEnd - sequence.offset; + + assert(op != NULL /* Precondition */); + assert(oend_w < oend /* No underflow */); + /* Handle edge cases in a slow path: + * - Read beyond end of literals + * - Match end is within WILDCOPY_OVERLIMIT of oend + * - 32-bit mode and the match length overflows + */ + if (UNLIKELY( + iLitEnd > litLimit || + oMatchEnd > oend_w || + (MEM_32bits() && (size_t)(oend - op) < sequenceLength + WILDCOPY_OVERLENGTH))) + return ZSTD_execSequenceEnd(op, oend, sequence, litPtr, litLimit, prefixStart, virtualStart, dictEnd); + + /* Assumptions (everything else goes into ZSTD_execSequenceEnd()) */ + assert(op <= oLitEnd /* No overflow */); + assert(oLitEnd < oMatchEnd /* Non-zero match & no overflow */); + assert(oMatchEnd <= oend /* No underflow */); + assert(iLitEnd <= litLimit /* Literal length is in bounds */); + assert(oLitEnd <= oend_w /* Can wildcopy literals */); + assert(oMatchEnd <= oend_w /* Can wildcopy matches */); + + /* Copy Literals: + * Split out litLength <= 16 since it is nearly always true. +1.6% on gcc-9. + * We likely don't need the full 32-byte wildcopy. + */ + assert(WILDCOPY_OVERLENGTH >= 16); + ZSTD_copy16(op, (*litPtr)); + if (UNLIKELY(sequence.litLength > 16)) { + ZSTD_wildcopy(op+16, (*litPtr)+16, sequence.litLength-16, ZSTD_no_overlap); + } + op = oLitEnd; + *litPtr = iLitEnd; /* update for next sequence */ + + /* Copy Match */ + if (sequence.offset > (size_t)(oLitEnd - prefixStart)) { + /* offset beyond prefix -> go into extDict */ + RETURN_ERROR_IF(UNLIKELY(sequence.offset > (size_t)(oLitEnd - virtualStart)), corruption_detected, ""); + match = dictEnd + (match - prefixStart); + if (match + sequence.matchLength <= dictEnd) { + memmove(oLitEnd, match, sequence.matchLength); + return sequenceLength; + } + /* span extDict & currentPrefixSegment */ + { size_t const length1 = dictEnd - match; + memmove(oLitEnd, match, length1); + op = oLitEnd + length1; + sequence.matchLength -= length1; + match = prefixStart; + } } + /* Match within prefix of 1 or more bytes */ + assert(op <= oMatchEnd); + assert(oMatchEnd <= oend_w); + assert(match >= prefixStart); + assert(sequence.matchLength >= 1); + + /* Nearly all offsets are >= WILDCOPY_VECLEN bytes, which means we can use wildcopy + * without overlap checking. + */ + if (LIKELY(sequence.offset >= WILDCOPY_VECLEN)) { + /* We bet on a full wildcopy for matches, since we expect matches to be + * longer than literals (in general). In silesia, ~10% of matches are longer + * than 16 bytes. + */ + ZSTD_wildcopy(op, match, (ptrdiff_t)sequence.matchLength, ZSTD_no_overlap); + return sequenceLength; + } + assert(sequence.offset < WILDCOPY_VECLEN); + + /* Copy 8 bytes and spread the offset to be >= 8. */ + ZSTD_overlapCopy8(&op, &match, sequence.offset); + + /* If the match length is > 8 bytes, then continue with the wildcopy. */ + if (sequence.matchLength > 8) { + assert(op < oMatchEnd); + ZSTD_wildcopy(op, match, (ptrdiff_t)sequence.matchLength-8, ZSTD_overlap_src_before_dst); + } + return sequenceLength; +} + +static void +ZSTD_initFseState(ZSTD_fseState* DStatePtr, BIT_DStream_t* bitD, const ZSTD_seqSymbol* dt) +{ + const void* ptr = dt; + const ZSTD_seqSymbol_header* const DTableH = (const ZSTD_seqSymbol_header*)ptr; + DStatePtr->state = BIT_readBits(bitD, DTableH->tableLog); + DEBUGLOG(6, "ZSTD_initFseState : val=%u using %u bits", + (U32)DStatePtr->state, DTableH->tableLog); + BIT_reloadDStream(bitD); + DStatePtr->table = dt + 1; +} + +FORCE_INLINE_TEMPLATE void +ZSTD_updateFseState(ZSTD_fseState* DStatePtr, BIT_DStream_t* bitD) +{ + ZSTD_seqSymbol const DInfo = DStatePtr->table[DStatePtr->state]; + U32 const nbBits = DInfo.nbBits; + size_t const lowBits = BIT_readBits(bitD, nbBits); + DStatePtr->state = DInfo.nextState + lowBits; +} + +FORCE_INLINE_TEMPLATE void +ZSTD_updateFseStateWithDInfo(ZSTD_fseState* DStatePtr, BIT_DStream_t* bitD, ZSTD_seqSymbol const DInfo) +{ + U32 const nbBits = DInfo.nbBits; + size_t const lowBits = BIT_readBits(bitD, nbBits); + DStatePtr->state = DInfo.nextState + lowBits; +} + +/* We need to add at most (ZSTD_WINDOWLOG_MAX_32 - 1) bits to read the maximum + * offset bits. But we can only read at most (STREAM_ACCUMULATOR_MIN_32 - 1) + * bits before reloading. This value is the maximum number of bytes we read + * after reloading when we are decoding long offsets. + */ +#define LONG_OFFSETS_MAX_EXTRA_BITS_32 \ + (ZSTD_WINDOWLOG_MAX_32 > STREAM_ACCUMULATOR_MIN_32 \ + ? ZSTD_WINDOWLOG_MAX_32 - STREAM_ACCUMULATOR_MIN_32 \ + : 0) + +typedef enum { ZSTD_lo_isRegularOffset, ZSTD_lo_isLongOffset=1 } ZSTD_longOffset_e; +typedef enum { ZSTD_p_noPrefetch=0, ZSTD_p_prefetch=1 } ZSTD_prefetch_e; + +FORCE_INLINE_TEMPLATE seq_t +ZSTD_decodeSequence(seqState_t* seqState, const ZSTD_longOffset_e longOffsets, const ZSTD_prefetch_e prefetch) +{ + seq_t seq; + ZSTD_seqSymbol const llDInfo = seqState->stateLL.table[seqState->stateLL.state]; + ZSTD_seqSymbol const mlDInfo = seqState->stateML.table[seqState->stateML.state]; + ZSTD_seqSymbol const ofDInfo = seqState->stateOffb.table[seqState->stateOffb.state]; + U32 const llBase = llDInfo.baseValue; + U32 const mlBase = mlDInfo.baseValue; + U32 const ofBase = ofDInfo.baseValue; + BYTE const llBits = llDInfo.nbAdditionalBits; + BYTE const mlBits = mlDInfo.nbAdditionalBits; + BYTE const ofBits = ofDInfo.nbAdditionalBits; + BYTE const totalBits = llBits+mlBits+ofBits; + + /* sequence */ + { size_t offset; + if (ofBits > 1) { + ZSTD_STATIC_ASSERT(ZSTD_lo_isLongOffset == 1); + ZSTD_STATIC_ASSERT(LONG_OFFSETS_MAX_EXTRA_BITS_32 == 5); + assert(ofBits <= MaxOff); + if (MEM_32bits() && longOffsets && (ofBits >= STREAM_ACCUMULATOR_MIN_32)) { + U32 const extraBits = ofBits - MIN(ofBits, 32 - seqState->DStream.bitsConsumed); + offset = ofBase + (BIT_readBitsFast(&seqState->DStream, ofBits - extraBits) << extraBits); + BIT_reloadDStream(&seqState->DStream); + if (extraBits) offset += BIT_readBitsFast(&seqState->DStream, extraBits); + assert(extraBits <= LONG_OFFSETS_MAX_EXTRA_BITS_32); /* to avoid another reload */ + } else { + offset = ofBase + BIT_readBitsFast(&seqState->DStream, ofBits/*>0*/); /* <= (ZSTD_WINDOWLOG_MAX-1) bits */ + if (MEM_32bits()) BIT_reloadDStream(&seqState->DStream); + } + seqState->prevOffset[2] = seqState->prevOffset[1]; + seqState->prevOffset[1] = seqState->prevOffset[0]; + seqState->prevOffset[0] = offset; + } else { + U32 const ll0 = (llBase == 0); + if (LIKELY((ofBits == 0))) { + if (LIKELY(!ll0)) + offset = seqState->prevOffset[0]; + else { + offset = seqState->prevOffset[1]; + seqState->prevOffset[1] = seqState->prevOffset[0]; + seqState->prevOffset[0] = offset; + } + } else { + offset = ofBase + ll0 + BIT_readBitsFast(&seqState->DStream, 1); + { size_t temp = (offset==3) ? seqState->prevOffset[0] - 1 : seqState->prevOffset[offset]; + temp += !temp; /* 0 is not valid; input is corrupted; force offset to 1 */ + if (offset != 1) seqState->prevOffset[2] = seqState->prevOffset[1]; + seqState->prevOffset[1] = seqState->prevOffset[0]; + seqState->prevOffset[0] = offset = temp; + } } } + seq.offset = offset; + } + + seq.matchLength = mlBase; + if (mlBits > 0) + seq.matchLength += BIT_readBitsFast(&seqState->DStream, mlBits/*>0*/); + + if (MEM_32bits() && (mlBits+llBits >= STREAM_ACCUMULATOR_MIN_32-LONG_OFFSETS_MAX_EXTRA_BITS_32)) + BIT_reloadDStream(&seqState->DStream); + if (MEM_64bits() && UNLIKELY(totalBits >= STREAM_ACCUMULATOR_MIN_64-(LLFSELog+MLFSELog+OffFSELog))) + BIT_reloadDStream(&seqState->DStream); + /* Ensure there are enough bits to read the rest of data in 64-bit mode. */ + ZSTD_STATIC_ASSERT(16+LLFSELog+MLFSELog+OffFSELog < STREAM_ACCUMULATOR_MIN_64); + + seq.litLength = llBase; + if (llBits > 0) + seq.litLength += BIT_readBitsFast(&seqState->DStream, llBits/*>0*/); + + if (MEM_32bits()) + BIT_reloadDStream(&seqState->DStream); + + DEBUGLOG(6, "seq: litL=%u, matchL=%u, offset=%u", + (U32)seq.litLength, (U32)seq.matchLength, (U32)seq.offset); + + if (prefetch == ZSTD_p_prefetch) { + size_t const pos = seqState->pos + seq.litLength; + const BYTE* const matchBase = (seq.offset > pos) ? seqState->dictEnd : seqState->prefixStart; + seq.match = matchBase + pos - seq.offset; /* note : this operation can overflow when seq.offset is really too large, which can only happen when input is corrupted. + * No consequence though : no memory access will occur, offset is only used for prefetching */ + seqState->pos = pos + seq.matchLength; + } + + /* ANS state update + * gcc-9.0.0 does 2.5% worse with ZSTD_updateFseStateWithDInfo(). + * clang-9.2.0 does 7% worse with ZSTD_updateFseState(). + * Naturally it seems like ZSTD_updateFseStateWithDInfo() should be the + * better option, so it is the default for other compilers. But, if you + * measure that it is worse, please put up a pull request. + */ + { +#if defined(__GNUC__) && !defined(__clang__) + const int kUseUpdateFseState = 1; +#else + const int kUseUpdateFseState = 0; +#endif + if (kUseUpdateFseState) { + ZSTD_updateFseState(&seqState->stateLL, &seqState->DStream); /* <= 9 bits */ + ZSTD_updateFseState(&seqState->stateML, &seqState->DStream); /* <= 9 bits */ + if (MEM_32bits()) BIT_reloadDStream(&seqState->DStream); /* <= 18 bits */ + ZSTD_updateFseState(&seqState->stateOffb, &seqState->DStream); /* <= 8 bits */ + } else { + ZSTD_updateFseStateWithDInfo(&seqState->stateLL, &seqState->DStream, llDInfo); /* <= 9 bits */ + ZSTD_updateFseStateWithDInfo(&seqState->stateML, &seqState->DStream, mlDInfo); /* <= 9 bits */ + if (MEM_32bits()) BIT_reloadDStream(&seqState->DStream); /* <= 18 bits */ + ZSTD_updateFseStateWithDInfo(&seqState->stateOffb, &seqState->DStream, ofDInfo); /* <= 8 bits */ + } + } + + return seq; +} + +#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION +static int ZSTD_dictionaryIsActive(ZSTD_DCtx const* dctx, BYTE const* prefixStart, BYTE const* oLitEnd) +{ + size_t const windowSize = dctx->fParams.windowSize; + /* No dictionary used. */ + if (dctx->dictContentEndForFuzzing == NULL) return 0; + /* Dictionary is our prefix. */ + if (prefixStart == dctx->dictContentBeginForFuzzing) return 1; + /* Dictionary is not our ext-dict. */ + if (dctx->dictEnd != dctx->dictContentEndForFuzzing) return 0; + /* Dictionary is not within our window size. */ + if ((size_t)(oLitEnd - prefixStart) >= windowSize) return 0; + /* Dictionary is active. */ + return 1; +} + +MEM_STATIC void ZSTD_assertValidSequence( + ZSTD_DCtx const* dctx, + BYTE const* op, BYTE const* oend, + seq_t const seq, + BYTE const* prefixStart, BYTE const* virtualStart) +{ + size_t const windowSize = dctx->fParams.windowSize; + size_t const sequenceSize = seq.litLength + seq.matchLength; + BYTE const* const oLitEnd = op + seq.litLength; + DEBUGLOG(6, "Checking sequence: litL=%u matchL=%u offset=%u", + (U32)seq.litLength, (U32)seq.matchLength, (U32)seq.offset); + assert(op <= oend); + assert((size_t)(oend - op) >= sequenceSize); + assert(sequenceSize <= ZSTD_BLOCKSIZE_MAX); + if (ZSTD_dictionaryIsActive(dctx, prefixStart, oLitEnd)) { + size_t const dictSize = (size_t)((char const*)dctx->dictContentEndForFuzzing - (char const*)dctx->dictContentBeginForFuzzing); + /* Offset must be within the dictionary. */ + assert(seq.offset <= (size_t)(oLitEnd - virtualStart)); + assert(seq.offset <= windowSize + dictSize); + } else { + /* Offset must be within our window. */ + assert(seq.offset <= windowSize); + } +} +#endif + +#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG +FORCE_INLINE_TEMPLATE size_t +DONT_VECTORIZE +ZSTD_decompressSequences_body( ZSTD_DCtx* dctx, + void* dst, size_t maxDstSize, + const void* seqStart, size_t seqSize, int nbSeq, + const ZSTD_longOffset_e isLongOffset, + const int frame) +{ + const BYTE* ip = (const BYTE*)seqStart; + const BYTE* const iend = ip + seqSize; + BYTE* const ostart = (BYTE* const)dst; + BYTE* const oend = ostart + maxDstSize; + BYTE* op = ostart; + const BYTE* litPtr = dctx->litPtr; + const BYTE* const litEnd = litPtr + dctx->litSize; + const BYTE* const prefixStart = (const BYTE*) (dctx->prefixStart); + const BYTE* const vBase = (const BYTE*) (dctx->virtualStart); + const BYTE* const dictEnd = (const BYTE*) (dctx->dictEnd); + DEBUGLOG(5, "ZSTD_decompressSequences_body"); + (void)frame; + + /* Regen sequences */ + if (nbSeq) { + seqState_t seqState; + size_t error = 0; + dctx->fseEntropy = 1; + { U32 i; for (i=0; ientropy.rep[i]; } + RETURN_ERROR_IF( + ERR_isError(BIT_initDStream(&seqState.DStream, ip, iend-ip)), + corruption_detected, ""); + ZSTD_initFseState(&seqState.stateLL, &seqState.DStream, dctx->LLTptr); + ZSTD_initFseState(&seqState.stateOffb, &seqState.DStream, dctx->OFTptr); + ZSTD_initFseState(&seqState.stateML, &seqState.DStream, dctx->MLTptr); + assert(dst != NULL); + + ZSTD_STATIC_ASSERT( + BIT_DStream_unfinished < BIT_DStream_completed && + BIT_DStream_endOfBuffer < BIT_DStream_completed && + BIT_DStream_completed < BIT_DStream_overflow); + +#if defined(__GNUC__) && defined(__x86_64__) + /* Align the decompression loop to 32 + 16 bytes. + * + * zstd compiled with gcc-9 on an Intel i9-9900k shows 10% decompression + * speed swings based on the alignment of the decompression loop. This + * performance swing is caused by parts of the decompression loop falling + * out of the DSB. The entire decompression loop should fit in the DSB, + * when it can't we get much worse performance. You can measure if you've + * hit the good case or the bad case with this perf command for some + * compressed file test.zst: + * + * perf stat -e cycles -e instructions -e idq.all_dsb_cycles_any_uops \ + * -e idq.all_mite_cycles_any_uops -- ./zstd -tq test.zst + * + * If you see most cycles served out of the MITE you've hit the bad case. + * If you see most cycles served out of the DSB you've hit the good case. + * If it is pretty even then you may be in an okay case. + * + * I've been able to reproduce this issue on the following CPUs: + * - Kabylake: Macbook Pro (15-inch, 2019) 2.4 GHz Intel Core i9 + * Use Instruments->Counters to get DSB/MITE cycles. + * I never got performance swings, but I was able to + * go from the good case of mostly DSB to half of the + * cycles served from MITE. + * - Coffeelake: Intel i9-9900k + * + * I haven't been able to reproduce the instability or DSB misses on any + * of the following CPUS: + * - Haswell + * - Broadwell: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GH + * - Skylake + * + * If you are seeing performance stability this script can help test. + * It tests on 4 commits in zstd where I saw performance change. + * + * https://gist.github.com/terrelln/9889fc06a423fd5ca6e99351564473f4 + */ + __asm__(".p2align 5"); + __asm__("nop"); + __asm__(".p2align 4"); +#endif + for ( ; ; ) { + seq_t const sequence = ZSTD_decodeSequence(&seqState, isLongOffset, ZSTD_p_noPrefetch); + size_t const oneSeqSize = ZSTD_execSequence(op, oend, sequence, &litPtr, litEnd, prefixStart, vBase, dictEnd); +#if defined(FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION) && defined(FUZZING_ASSERT_VALID_SEQUENCE) + assert(!ZSTD_isError(oneSeqSize)); + if (frame) ZSTD_assertValidSequence(dctx, op, oend, sequence, prefixStart, vBase); +#endif + DEBUGLOG(6, "regenerated sequence size : %u", (U32)oneSeqSize); + BIT_reloadDStream(&(seqState.DStream)); + /* gcc and clang both don't like early returns in this loop. + * gcc doesn't like early breaks either. + * Instead save an error and report it at the end. + * When there is an error, don't increment op, so we don't + * overwrite. + */ + if (UNLIKELY(ZSTD_isError(oneSeqSize))) error = oneSeqSize; + else op += oneSeqSize; + if (UNLIKELY(!--nbSeq)) break; + } + + /* check if reached exact end */ + DEBUGLOG(5, "ZSTD_decompressSequences_body: after decode loop, remaining nbSeq : %i", nbSeq); + if (ZSTD_isError(error)) return error; + RETURN_ERROR_IF(nbSeq, corruption_detected, ""); + RETURN_ERROR_IF(BIT_reloadDStream(&seqState.DStream) < BIT_DStream_completed, corruption_detected, ""); + /* save reps for next block */ + { U32 i; for (i=0; ientropy.rep[i] = (U32)(seqState.prevOffset[i]); } + } + + /* last literal segment */ + { size_t const lastLLSize = litEnd - litPtr; + RETURN_ERROR_IF(lastLLSize > (size_t)(oend-op), dstSize_tooSmall, ""); + if (op != NULL) { + memcpy(op, litPtr, lastLLSize); + op += lastLLSize; + } + } + + return op-ostart; +} + +static size_t +ZSTD_decompressSequences_default(ZSTD_DCtx* dctx, + void* dst, size_t maxDstSize, + const void* seqStart, size_t seqSize, int nbSeq, + const ZSTD_longOffset_e isLongOffset, + const int frame) +{ + return ZSTD_decompressSequences_body(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); +} +#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG */ + +#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT +FORCE_INLINE_TEMPLATE size_t +ZSTD_decompressSequencesLong_body( + ZSTD_DCtx* dctx, + void* dst, size_t maxDstSize, + const void* seqStart, size_t seqSize, int nbSeq, + const ZSTD_longOffset_e isLongOffset, + const int frame) +{ + const BYTE* ip = (const BYTE*)seqStart; + const BYTE* const iend = ip + seqSize; + BYTE* const ostart = (BYTE* const)dst; + BYTE* const oend = ostart + maxDstSize; + BYTE* op = ostart; + const BYTE* litPtr = dctx->litPtr; + const BYTE* const litEnd = litPtr + dctx->litSize; + const BYTE* const prefixStart = (const BYTE*) (dctx->prefixStart); + const BYTE* const dictStart = (const BYTE*) (dctx->virtualStart); + const BYTE* const dictEnd = (const BYTE*) (dctx->dictEnd); + (void)frame; + + /* Regen sequences */ + if (nbSeq) { +#define STORED_SEQS 4 +#define STORED_SEQS_MASK (STORED_SEQS-1) +#define ADVANCED_SEQS 4 + seq_t sequences[STORED_SEQS]; + int const seqAdvance = MIN(nbSeq, ADVANCED_SEQS); + seqState_t seqState; + int seqNb; + dctx->fseEntropy = 1; + { int i; for (i=0; ientropy.rep[i]; } + seqState.prefixStart = prefixStart; + seqState.pos = (size_t)(op-prefixStart); + seqState.dictEnd = dictEnd; + assert(dst != NULL); + assert(iend >= ip); + RETURN_ERROR_IF( + ERR_isError(BIT_initDStream(&seqState.DStream, ip, iend-ip)), + corruption_detected, ""); + ZSTD_initFseState(&seqState.stateLL, &seqState.DStream, dctx->LLTptr); + ZSTD_initFseState(&seqState.stateOffb, &seqState.DStream, dctx->OFTptr); + ZSTD_initFseState(&seqState.stateML, &seqState.DStream, dctx->MLTptr); + + /* prepare in advance */ + for (seqNb=0; (BIT_reloadDStream(&seqState.DStream) <= BIT_DStream_completed) && (seqNbentropy.rep[i] = (U32)(seqState.prevOffset[i]); } + } + + /* last literal segment */ + { size_t const lastLLSize = litEnd - litPtr; + RETURN_ERROR_IF(lastLLSize > (size_t)(oend-op), dstSize_tooSmall, ""); + if (op != NULL) { + memcpy(op, litPtr, lastLLSize); + op += lastLLSize; + } + } + + return op-ostart; +} + +static size_t +ZSTD_decompressSequencesLong_default(ZSTD_DCtx* dctx, + void* dst, size_t maxDstSize, + const void* seqStart, size_t seqSize, int nbSeq, + const ZSTD_longOffset_e isLongOffset, + const int frame) +{ + return ZSTD_decompressSequencesLong_body(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); +} +#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT */ + + + +#if DYNAMIC_BMI2 + +#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG +static TARGET_ATTRIBUTE("bmi2") size_t +DONT_VECTORIZE +ZSTD_decompressSequences_bmi2(ZSTD_DCtx* dctx, + void* dst, size_t maxDstSize, + const void* seqStart, size_t seqSize, int nbSeq, + const ZSTD_longOffset_e isLongOffset, + const int frame) +{ + return ZSTD_decompressSequences_body(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); +} +#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG */ + +#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT +static TARGET_ATTRIBUTE("bmi2") size_t +ZSTD_decompressSequencesLong_bmi2(ZSTD_DCtx* dctx, + void* dst, size_t maxDstSize, + const void* seqStart, size_t seqSize, int nbSeq, + const ZSTD_longOffset_e isLongOffset, + const int frame) +{ + return ZSTD_decompressSequencesLong_body(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); +} +#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT */ + +#endif /* DYNAMIC_BMI2 */ + +typedef size_t (*ZSTD_decompressSequences_t)( + ZSTD_DCtx* dctx, + void* dst, size_t maxDstSize, + const void* seqStart, size_t seqSize, int nbSeq, + const ZSTD_longOffset_e isLongOffset, + const int frame); + +#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG +static size_t +ZSTD_decompressSequences(ZSTD_DCtx* dctx, void* dst, size_t maxDstSize, + const void* seqStart, size_t seqSize, int nbSeq, + const ZSTD_longOffset_e isLongOffset, + const int frame) +{ + DEBUGLOG(5, "ZSTD_decompressSequences"); +#if DYNAMIC_BMI2 + if (dctx->bmi2) { + return ZSTD_decompressSequences_bmi2(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); + } +#endif + return ZSTD_decompressSequences_default(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); +} +#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG */ + + +#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT +/* ZSTD_decompressSequencesLong() : + * decompression function triggered when a minimum share of offsets is considered "long", + * aka out of cache. + * note : "long" definition seems overloaded here, sometimes meaning "wider than bitstream register", and sometimes meaning "farther than memory cache distance". + * This function will try to mitigate main memory latency through the use of prefetching */ +static size_t +ZSTD_decompressSequencesLong(ZSTD_DCtx* dctx, + void* dst, size_t maxDstSize, + const void* seqStart, size_t seqSize, int nbSeq, + const ZSTD_longOffset_e isLongOffset, + const int frame) +{ + DEBUGLOG(5, "ZSTD_decompressSequencesLong"); +#if DYNAMIC_BMI2 + if (dctx->bmi2) { + return ZSTD_decompressSequencesLong_bmi2(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); + } +#endif + return ZSTD_decompressSequencesLong_default(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); +} +#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT */ + + + +#if !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT) && \ + !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG) +/* ZSTD_getLongOffsetsShare() : + * condition : offTable must be valid + * @return : "share" of long offsets (arbitrarily defined as > (1<<23)) + * compared to maximum possible of (1< 22) total += 1; + } + + assert(tableLog <= OffFSELog); + total <<= (OffFSELog - tableLog); /* scale to OffFSELog */ + + return total; +} +#endif + +size_t +ZSTD_decompressBlock_internal(ZSTD_DCtx* dctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, const int frame) +{ /* blockType == blockCompressed */ + const BYTE* ip = (const BYTE*)src; + /* isLongOffset must be true if there are long offsets. + * Offsets are long if they are larger than 2^STREAM_ACCUMULATOR_MIN. + * We don't expect that to be the case in 64-bit mode. + * In block mode, window size is not known, so we have to be conservative. + * (note: but it could be evaluated from current-lowLimit) + */ + ZSTD_longOffset_e const isLongOffset = (ZSTD_longOffset_e)(MEM_32bits() && (!frame || (dctx->fParams.windowSize > (1ULL << STREAM_ACCUMULATOR_MIN)))); + DEBUGLOG(5, "ZSTD_decompressBlock_internal (size : %u)", (U32)srcSize); + + RETURN_ERROR_IF(srcSize >= ZSTD_BLOCKSIZE_MAX, srcSize_wrong, ""); + + /* Decode literals section */ + { size_t const litCSize = ZSTD_decodeLiteralsBlock(dctx, src, srcSize); + DEBUGLOG(5, "ZSTD_decodeLiteralsBlock : %u", (U32)litCSize); + if (ZSTD_isError(litCSize)) return litCSize; + ip += litCSize; + srcSize -= litCSize; + } + + /* Build Decoding Tables */ + { + /* These macros control at build-time which decompressor implementation + * we use. If neither is defined, we do some inspection and dispatch at + * runtime. + */ +#if !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT) && \ + !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG) + int usePrefetchDecoder = dctx->ddictIsCold; +#endif + int nbSeq; + size_t const seqHSize = ZSTD_decodeSeqHeaders(dctx, &nbSeq, ip, srcSize); + if (ZSTD_isError(seqHSize)) return seqHSize; + ip += seqHSize; + srcSize -= seqHSize; + + RETURN_ERROR_IF(dst == NULL && nbSeq > 0, dstSize_tooSmall, "NULL not handled"); + +#if !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT) && \ + !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG) + if ( !usePrefetchDecoder + && (!frame || (dctx->fParams.windowSize > (1<<24))) + && (nbSeq>ADVANCED_SEQS) ) { /* could probably use a larger nbSeq limit */ + U32 const shareLongOffsets = ZSTD_getLongOffsetsShare(dctx->OFTptr); + U32 const minShare = MEM_64bits() ? 7 : 20; /* heuristic values, correspond to 2.73% and 7.81% */ + usePrefetchDecoder = (shareLongOffsets >= minShare); + } +#endif + + dctx->ddictIsCold = 0; + +#if !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT) && \ + !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG) + if (usePrefetchDecoder) +#endif +#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT + return ZSTD_decompressSequencesLong(dctx, dst, dstCapacity, ip, srcSize, nbSeq, isLongOffset, frame); +#endif + +#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG + /* else */ + return ZSTD_decompressSequences(dctx, dst, dstCapacity, ip, srcSize, nbSeq, isLongOffset, frame); +#endif + } +} + + +void ZSTD_checkContinuity(ZSTD_DCtx* dctx, const void* dst) +{ + if (dst != dctx->previousDstEnd) { /* not contiguous */ + dctx->dictEnd = dctx->previousDstEnd; + dctx->virtualStart = (const char*)dst - ((const char*)(dctx->previousDstEnd) - (const char*)(dctx->prefixStart)); + dctx->prefixStart = dst; + dctx->previousDstEnd = dst; + } +} + + +size_t ZSTD_decompressBlock(ZSTD_DCtx* dctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize) +{ + size_t dSize; + ZSTD_checkContinuity(dctx, dst); + dSize = ZSTD_decompressBlock_internal(dctx, dst, dstCapacity, src, srcSize, /* frame */ 0); + dctx->previousDstEnd = (char*)dst + dSize; + return dSize; +} diff --git a/module/zstd/lib/decompress/zstd_decompress_block.h b/module/zstd/lib/decompress/zstd_decompress_block.h new file mode 100644 index 000000000000..bf39b7350cf9 --- /dev/null +++ b/module/zstd/lib/decompress/zstd_decompress_block.h @@ -0,0 +1,59 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + + +#ifndef ZSTD_DEC_BLOCK_H +#define ZSTD_DEC_BLOCK_H + +/*-******************************************************* + * Dependencies + *********************************************************/ +#include /* size_t */ +#include "../zstd.h" /* DCtx, and some public functions */ +#include "../common/zstd_internal.h" /* blockProperties_t, and some public functions */ +#include "zstd_decompress_internal.h" /* ZSTD_seqSymbol */ + + +/* === Prototypes === */ + +/* note: prototypes already published within `zstd.h` : + * ZSTD_decompressBlock() + */ + +/* note: prototypes already published within `zstd_internal.h` : + * ZSTD_getcBlockSize() + * ZSTD_decodeSeqHeaders() + */ + + +/* ZSTD_decompressBlock_internal() : + * decompress block, starting at `src`, + * into destination buffer `dst`. + * @return : decompressed block size, + * or an error code (which can be tested using ZSTD_isError()) + */ +size_t ZSTD_decompressBlock_internal(ZSTD_DCtx* dctx, + void* dst, size_t dstCapacity, + const void* src, size_t srcSize, const int frame); + +/* ZSTD_buildFSETable() : + * generate FSE decoding table for one symbol (ll, ml or off) + * this function must be called with valid parameters only + * (dt is large enough, normalizedCounter distribution total is a power of 2, max is within range, etc.) + * in which case it cannot fail. + * Internal use only. + */ +void ZSTD_buildFSETable(ZSTD_seqSymbol* dt, + const short* normalizedCounter, unsigned maxSymbolValue, + const U32* baseValue, const U32* nbAdditionalBits, + unsigned tableLog); + + +#endif /* ZSTD_DEC_BLOCK_H */ diff --git a/module/zstd/lib/decompress/zstd_decompress_internal.h b/module/zstd/lib/decompress/zstd_decompress_internal.h new file mode 100644 index 000000000000..9ad96c554885 --- /dev/null +++ b/module/zstd/lib/decompress/zstd_decompress_internal.h @@ -0,0 +1,189 @@ +/* + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. + */ + + +/* zstd_decompress_internal: + * objects and definitions shared within lib/decompress modules */ + + #ifndef ZSTD_DECOMPRESS_INTERNAL_H + #define ZSTD_DECOMPRESS_INTERNAL_H + + +/*-******************************************************* + * Dependencies + *********************************************************/ +#include "../common/mem.h" /* BYTE, U16, U32 */ +#include "../common/zstd_internal.h" /* ZSTD_seqSymbol */ + + + +/*-******************************************************* + * Constants + *********************************************************/ +static const U32 LL_base[MaxLL+1] = { + 0, 1, 2, 3, 4, 5, 6, 7, + 8, 9, 10, 11, 12, 13, 14, 15, + 16, 18, 20, 22, 24, 28, 32, 40, + 48, 64, 0x80, 0x100, 0x200, 0x400, 0x800, 0x1000, + 0x2000, 0x4000, 0x8000, 0x10000 }; + +static const U32 OF_base[MaxOff+1] = { + 0, 1, 1, 5, 0xD, 0x1D, 0x3D, 0x7D, + 0xFD, 0x1FD, 0x3FD, 0x7FD, 0xFFD, 0x1FFD, 0x3FFD, 0x7FFD, + 0xFFFD, 0x1FFFD, 0x3FFFD, 0x7FFFD, 0xFFFFD, 0x1FFFFD, 0x3FFFFD, 0x7FFFFD, + 0xFFFFFD, 0x1FFFFFD, 0x3FFFFFD, 0x7FFFFFD, 0xFFFFFFD, 0x1FFFFFFD, 0x3FFFFFFD, 0x7FFFFFFD }; + +static const U32 OF_bits[MaxOff+1] = { + 0, 1, 2, 3, 4, 5, 6, 7, + 8, 9, 10, 11, 12, 13, 14, 15, + 16, 17, 18, 19, 20, 21, 22, 23, + 24, 25, 26, 27, 28, 29, 30, 31 }; + +static const U32 ML_base[MaxML+1] = { + 3, 4, 5, 6, 7, 8, 9, 10, + 11, 12, 13, 14, 15, 16, 17, 18, + 19, 20, 21, 22, 23, 24, 25, 26, + 27, 28, 29, 30, 31, 32, 33, 34, + 35, 37, 39, 41, 43, 47, 51, 59, + 67, 83, 99, 0x83, 0x103, 0x203, 0x403, 0x803, + 0x1003, 0x2003, 0x4003, 0x8003, 0x10003 }; + + +/*-******************************************************* + * Decompression types + *********************************************************/ + typedef struct { + U32 fastMode; + U32 tableLog; + } ZSTD_seqSymbol_header; + + typedef struct { + U16 nextState; + BYTE nbAdditionalBits; + BYTE nbBits; + U32 baseValue; + } ZSTD_seqSymbol; + + #define SEQSYMBOL_TABLE_SIZE(log) (1 + (1 << (log))) + +typedef struct { + ZSTD_seqSymbol LLTable[SEQSYMBOL_TABLE_SIZE(LLFSELog)]; /* Note : Space reserved for FSE Tables */ + ZSTD_seqSymbol OFTable[SEQSYMBOL_TABLE_SIZE(OffFSELog)]; /* is also used as temporary workspace while building hufTable during DDict creation */ + ZSTD_seqSymbol MLTable[SEQSYMBOL_TABLE_SIZE(MLFSELog)]; /* and therefore must be at least HUF_DECOMPRESS_WORKSPACE_SIZE large */ + HUF_DTable hufTable[HUF_DTABLE_SIZE(HufLog)]; /* can accommodate HUF_decompress4X */ + U32 rep[ZSTD_REP_NUM]; +} ZSTD_entropyDTables_t; + +typedef enum { ZSTDds_getFrameHeaderSize, ZSTDds_decodeFrameHeader, + ZSTDds_decodeBlockHeader, ZSTDds_decompressBlock, + ZSTDds_decompressLastBlock, ZSTDds_checkChecksum, + ZSTDds_decodeSkippableHeader, ZSTDds_skipFrame } ZSTD_dStage; + +typedef enum { zdss_init=0, zdss_loadHeader, + zdss_read, zdss_load, zdss_flush } ZSTD_dStreamStage; + +typedef enum { + ZSTD_use_indefinitely = -1, /* Use the dictionary indefinitely */ + ZSTD_dont_use = 0, /* Do not use the dictionary (if one exists free it) */ + ZSTD_use_once = 1 /* Use the dictionary once and set to ZSTD_dont_use */ +} ZSTD_dictUses_e; + +typedef enum { + ZSTD_obm_buffered = 0, /* Buffer the output */ + ZSTD_obm_stable = 1 /* ZSTD_outBuffer is stable */ +} ZSTD_outBufferMode_e; + +struct ZSTD_DCtx_s +{ + const ZSTD_seqSymbol* LLTptr; + const ZSTD_seqSymbol* MLTptr; + const ZSTD_seqSymbol* OFTptr; + const HUF_DTable* HUFptr; + ZSTD_entropyDTables_t entropy; + U32 workspace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; /* space needed when building huffman tables */ + const void* previousDstEnd; /* detect continuity */ + const void* prefixStart; /* start of current segment */ + const void* virtualStart; /* virtual start of previous segment if it was just before current one */ + const void* dictEnd; /* end of previous segment */ + size_t expected; + ZSTD_frameHeader fParams; + U64 decodedSize; + blockType_e bType; /* used in ZSTD_decompressContinue(), store blockType between block header decoding and block decompression stages */ + ZSTD_dStage stage; + U32 litEntropy; + U32 fseEntropy; + XXH64_state_t xxhState; + size_t headerSize; + ZSTD_format_e format; + const BYTE* litPtr; + ZSTD_customMem customMem; + size_t litSize; + size_t rleSize; + size_t staticSize; + int bmi2; /* == 1 if the CPU supports BMI2 and 0 otherwise. CPU support is determined dynamically once per context lifetime. */ + + /* dictionary */ + ZSTD_DDict* ddictLocal; + const ZSTD_DDict* ddict; /* set by ZSTD_initDStream_usingDDict(), or ZSTD_DCtx_refDDict() */ + U32 dictID; + int ddictIsCold; /* if == 1 : dictionary is "new" for working context, and presumed "cold" (not in cpu cache) */ + ZSTD_dictUses_e dictUses; + + /* streaming */ + ZSTD_dStreamStage streamStage; + char* inBuff; + size_t inBuffSize; + size_t inPos; + size_t maxWindowSize; + char* outBuff; + size_t outBuffSize; + size_t outStart; + size_t outEnd; + size_t lhSize; + void* legacyContext; + U32 previousLegacyVersion; + U32 legacyVersion; + U32 hostageByte; + int noForwardProgress; + ZSTD_outBufferMode_e outBufferMode; + ZSTD_outBuffer expectedOutBuffer; + + /* workspace */ + BYTE litBuffer[ZSTD_BLOCKSIZE_MAX + WILDCOPY_OVERLENGTH]; + BYTE headerBuffer[ZSTD_FRAMEHEADERSIZE_MAX]; + + size_t oversizedDuration; + +#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION + void const* dictContentBeginForFuzzing; + void const* dictContentEndForFuzzing; +#endif +}; /* typedef'd to ZSTD_DCtx within "zstd.h" */ + + +/*-******************************************************* + * Shared internal functions + *********************************************************/ + +/*! ZSTD_loadDEntropy() : + * dict : must point at beginning of a valid zstd dictionary. + * @return : size of dictionary header (size of magic number + dict ID + entropy tables) */ +size_t ZSTD_loadDEntropy(ZSTD_entropyDTables_t* entropy, + const void* const dict, size_t const dictSize); + +/*! ZSTD_checkContinuity() : + * check if next `dst` follows previous position, where decompression ended. + * If yes, do nothing (continue on current segment). + * If not, classify previous segment as "external dictionary", and start a new segment. + * This function cannot fail. */ +void ZSTD_checkContinuity(ZSTD_DCtx* dctx, const void* dst); + + +#endif /* ZSTD_DECOMPRESS_INTERNAL_H */ diff --git a/module/zstd/lib/zstd.c b/module/zstd/lib/zstd.c deleted file mode 100644 index b57a9ef40d7b..000000000000 --- a/module/zstd/lib/zstd.c +++ /dev/null @@ -1,27821 +0,0 @@ -/* - * BSD 3-Clause Clear License - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * 1. Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * - * 2. Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * - * 3. Neither the name of the copyright holder nor the names of its - * contributors may be used to endorse or promote products derived from this - * software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" - * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR - * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS - * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN - * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE - * POSSIBILITY OF SUCH DAMAGE. - */ - -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. All rights reserved. - * Copyright (c) 2019-2020, Michael Niewöhner. All rights reserved. - */ - -#define MEM_MODULE -#define XXH_NAMESPACE ZSTD_ -#define XXH_PRIVATE_API -#define XXH_INLINE_ALL -#define ZSTD_LEGACY_SUPPORT 0 -#define ZSTD_LIB_DICTBUILDER 0 -#define ZSTD_LIB_DEPRECATED 0 -#define ZSTD_NOBENCH - -/**** start inlining common/debug.c ****/ -/* ****************************************************************** - * debug - * Part of FSE library - * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. - * - * You can contact the author at : - * - Source repository : https://github.com/Cyan4973/FiniteStateEntropy - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. -****************************************************************** */ - - -/* - * This module only hosts one global variable - * which can be used to dynamically influence the verbosity of traces, - * such as DEBUGLOG and RAWLOG - */ - -/**** start inlining debug.h ****/ -/* ****************************************************************** - * debug - * Part of FSE library - * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. - * - * You can contact the author at : - * - Source repository : https://github.com/Cyan4973/FiniteStateEntropy - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. -****************************************************************** */ - - -/* - * The purpose of this header is to enable debug functions. - * They regroup assert(), DEBUGLOG() and RAWLOG() for run-time, - * and DEBUG_STATIC_ASSERT() for compile-time. - * - * By default, DEBUGLEVEL==0, which means run-time debug is disabled. - * - * Level 1 enables assert() only. - * Starting level 2, traces can be generated and pushed to stderr. - * The higher the level, the more verbose the traces. - * - * It's possible to dynamically adjust level using variable g_debug_level, - * which is only declared if DEBUGLEVEL>=2, - * and is a global variable, not multi-thread protected (use with care) - */ - -#ifndef DEBUG_H_12987983217 -#define DEBUG_H_12987983217 - -#if defined (__cplusplus) -extern "C" { -#endif - - -/* static assert is triggered at compile time, leaving no runtime artefact. - * static assert only works with compile-time constants. - * Also, this variant can only be used inside a function. */ -#define DEBUG_STATIC_ASSERT(c) (void)sizeof(char[(c) ? 1 : -1]) - - -/* DEBUGLEVEL is expected to be defined externally, - * typically through compiler command line. - * Value must be a number. */ -#ifndef DEBUGLEVEL -# define DEBUGLEVEL 0 -#endif - - -/* DEBUGFILE can be defined externally, - * typically through compiler command line. - * note : currently useless. - * Value must be stderr or stdout */ -#ifndef DEBUGFILE -# define DEBUGFILE stderr -#endif - - -/* recommended values for DEBUGLEVEL : - * 0 : release mode, no debug, all run-time checks disabled - * 1 : enables assert() only, no display - * 2 : reserved, for currently active debug path - * 3 : events once per object lifetime (CCtx, CDict, etc.) - * 4 : events once per frame - * 5 : events once per block - * 6 : events once per sequence (verbose) - * 7+: events at every position (*very* verbose) - * - * It's generally inconvenient to output traces > 5. - * In which case, it's possible to selectively trigger high verbosity levels - * by modifying g_debug_level. - */ - -#if (DEBUGLEVEL>=1) -# include -#else -# ifndef assert /* assert may be already defined, due to prior #include */ -# define assert(condition) ((void)0) /* disable assert (default) */ -# endif -#endif - -#if (DEBUGLEVEL>=2) -# include -extern int g_debuglevel; /* the variable is only declared, - it actually lives in debug.c, - and is shared by the whole process. - It's not thread-safe. - It's useful when enabling very verbose levels - on selective conditions (such as position in src) */ - -# define RAWLOG(l, ...) { \ - if (l<=g_debuglevel) { \ - fprintf(stderr, __VA_ARGS__); \ - } } -# define DEBUGLOG(l, ...) { \ - if (l<=g_debuglevel) { \ - fprintf(stderr, __FILE__ ": " __VA_ARGS__); \ - fprintf(stderr, " \n"); \ - } } -#else -# define RAWLOG(l, ...) {} /* disabled */ -# define DEBUGLOG(l, ...) {} /* disabled */ -#endif - - -#if defined (__cplusplus) -} -#endif - -#endif /* DEBUG_H_12987983217 */ -/**** ended inlining debug.h ****/ - -int g_debuglevel = DEBUGLEVEL; -/**** ended inlining common/debug.c ****/ -/**** start inlining common/entropy_common.c ****/ -/* ****************************************************************** - * Common functions of New Generation Entropy library - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * - * You can contact the author at : - * - FSE+HUF source repository : https://github.com/Cyan4973/FiniteStateEntropy - * - Public forum : https://groups.google.com/forum/#!forum/lz4c - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. -****************************************************************** */ - -/* ************************************* -* Dependencies -***************************************/ -/**** start inlining mem.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef MEM_H_MODULE -#define MEM_H_MODULE - -#if defined (__cplusplus) -extern "C" { -#endif - -/*-**************************************** -* Dependencies -******************************************/ -#include /* size_t, ptrdiff_t */ -#include /* memcpy */ - - -/*-**************************************** -* Compiler specifics -******************************************/ -#if defined(_MSC_VER) /* Visual Studio */ -# include /* _byteswap_ulong */ -# include /* _byteswap_* */ -#endif -#if defined(__GNUC__) -# define MEM_STATIC static __inline __attribute__((unused)) -#elif defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) -# define MEM_STATIC static inline -#elif defined(_MSC_VER) -# define MEM_STATIC static __inline -#else -# define MEM_STATIC static /* this version may generate warnings for unused static functions; disable the relevant warning */ -#endif - -#ifndef __has_builtin -# define __has_builtin(x) 0 /* compat. with non-clang compilers */ -#endif - -/* code only tested on 32 and 64 bits systems */ -#define MEM_STATIC_ASSERT(c) { enum { MEM_static_assert = 1/(int)(!!(c)) }; } -MEM_STATIC void MEM_check(void) { MEM_STATIC_ASSERT((sizeof(size_t)==4) || (sizeof(size_t)==8)); } - -/* detects whether we are being compiled under msan */ -#if defined (__has_feature) -# if __has_feature(memory_sanitizer) -# define MEMORY_SANITIZER 1 -# endif -#endif - -#if defined (MEMORY_SANITIZER) -/* Not all platforms that support msan provide sanitizers/msan_interface.h. - * We therefore declare the functions we need ourselves, rather than trying to - * include the header file... */ - -#include /* intptr_t */ - -/* Make memory region fully initialized (without changing its contents). */ -void __msan_unpoison(const volatile void *a, size_t size); - -/* Make memory region fully uninitialized (without changing its contents). - This is a legacy interface that does not update origin information. Use - __msan_allocated_memory() instead. */ -void __msan_poison(const volatile void *a, size_t size); - -/* Returns the offset of the first (at least partially) poisoned byte in the - memory range, or -1 if the whole range is good. */ -intptr_t __msan_test_shadow(const volatile void *x, size_t size); -#endif - -/* detects whether we are being compiled under asan */ -#if defined (ZFS_ASAN_ENABLED) -# define ADDRESS_SANITIZER 1 -# define ZSTD_ASAN_DONT_POISON_WORKSPACE -#endif - -#if defined (ADDRESS_SANITIZER) -/* Not all platforms that support asan provide sanitizers/asan_interface.h. - * We therefore declare the functions we need ourselves, rather than trying to - * include the header file... */ - -/** - * Marks a memory region ([addr, addr+size)) as unaddressable. - * - * This memory must be previously allocated by your program. Instrumented - * code is forbidden from accessing addresses in this region until it is - * unpoisoned. This function is not guaranteed to poison the entire region - - * it could poison only a subregion of [addr, addr+size) due to ASan - * alignment restrictions. - * - * \note This function is not thread-safe because no two threads can poison or - * unpoison memory in the same memory region simultaneously. - * - * \param addr Start of memory region. - * \param size Size of memory region. */ -void __asan_poison_memory_region(void const volatile *addr, size_t size); - -/** - * Marks a memory region ([addr, addr+size)) as addressable. - * - * This memory must be previously allocated by your program. Accessing - * addresses in this region is allowed until this region is poisoned again. - * This function could unpoison a super-region of [addr, addr+size) due - * to ASan alignment restrictions. - * - * \note This function is not thread-safe because no two threads can - * poison or unpoison memory in the same memory region simultaneously. - * - * \param addr Start of memory region. - * \param size Size of memory region. */ -void __asan_unpoison_memory_region(void const volatile *addr, size_t size); -#endif - - -/*-************************************************************** -* Basic Types -*****************************************************************/ -#if !defined (__VMS) && (defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) ) -# include - typedef uint8_t BYTE; - typedef uint16_t U16; - typedef int16_t S16; - typedef uint32_t U32; - typedef int32_t S32; - typedef uint64_t U64; - typedef int64_t S64; -#else -# include -#if CHAR_BIT != 8 -# error "this implementation requires char to be exactly 8-bit type" -#endif - typedef unsigned char BYTE; -#if USHRT_MAX != 65535 -# error "this implementation requires short to be exactly 16-bit type" -#endif - typedef unsigned short U16; - typedef signed short S16; -#if UINT_MAX != 4294967295 -# error "this implementation requires int to be exactly 32-bit type" -#endif - typedef unsigned int U32; - typedef signed int S32; -/* note : there are no limits defined for long long type in C90. - * limits exist in C99, however, in such case, is preferred */ - typedef unsigned long long U64; - typedef signed long long S64; -#endif - - -/*-************************************************************** -* Memory I/O -*****************************************************************/ -/* MEM_FORCE_MEMORY_ACCESS : - * By default, access to unaligned memory is controlled by `memcpy()`, which is safe and portable. - * Unfortunately, on some target/compiler combinations, the generated assembly is sub-optimal. - * The below switch allow to select different access method for improved performance. - * Method 0 (default) : use `memcpy()`. Safe and portable. - * Method 1 : `__packed` statement. It depends on compiler extension (i.e., not portable). - * This method is safe if your compiler supports it, and *generally* as fast or faster than `memcpy`. - * Method 2 : direct access. This method is portable but violate C standard. - * It can generate buggy code on targets depending on alignment. - * In some circumstances, it's the only known way to get the most performance (i.e. GCC + ARMv6) - * See http://fastcompression.blogspot.fr/2015/08/accessing-unaligned-memory.html for details. - * Prefer these methods in priority order (0 > 1 > 2) - */ -#ifndef MEM_FORCE_MEMORY_ACCESS /* can be defined externally, on command line for example */ -# if defined(__GNUC__) && ( defined(__ARM_ARCH_6__) || defined(__ARM_ARCH_6J__) || defined(__ARM_ARCH_6K__) || defined(__ARM_ARCH_6Z__) || defined(__ARM_ARCH_6ZK__) || defined(__ARM_ARCH_6T2__) ) -# define MEM_FORCE_MEMORY_ACCESS 2 -# elif defined(__INTEL_COMPILER) || defined(__GNUC__) || defined(__ICCARM__) -# define MEM_FORCE_MEMORY_ACCESS 1 -# endif -#endif - -MEM_STATIC unsigned MEM_32bits(void) { return sizeof(size_t)==4; } -MEM_STATIC unsigned MEM_64bits(void) { return sizeof(size_t)==8; } - -MEM_STATIC unsigned MEM_isLittleEndian(void) -{ - const union { U32 u; BYTE c[4]; } one = { 1 }; /* don't use static : performance detrimental */ - return one.c[0]; -} - -#if defined(MEM_FORCE_MEMORY_ACCESS) && (MEM_FORCE_MEMORY_ACCESS==2) - -/* violates C standard, by lying on structure alignment. -Only use if no other choice to achieve best performance on target platform */ -MEM_STATIC U16 MEM_read16(const void* memPtr) { return *(const U16*) memPtr; } -MEM_STATIC U32 MEM_read32(const void* memPtr) { return *(const U32*) memPtr; } -MEM_STATIC U64 MEM_read64(const void* memPtr) { return *(const U64*) memPtr; } -MEM_STATIC size_t MEM_readST(const void* memPtr) { return *(const size_t*) memPtr; } - -MEM_STATIC void MEM_write16(void* memPtr, U16 value) { *(U16*)memPtr = value; } -MEM_STATIC void MEM_write32(void* memPtr, U32 value) { *(U32*)memPtr = value; } -MEM_STATIC void MEM_write64(void* memPtr, U64 value) { *(U64*)memPtr = value; } - -#elif defined(MEM_FORCE_MEMORY_ACCESS) && (MEM_FORCE_MEMORY_ACCESS==1) - -/* __pack instructions are safer, but compiler specific, hence potentially problematic for some compilers */ -/* currently only defined for gcc and icc */ -#if defined(_MSC_VER) || (defined(__INTEL_COMPILER) && defined(WIN32)) - __pragma( pack(push, 1) ) - typedef struct { U16 v; } unalign16; - typedef struct { U32 v; } unalign32; - typedef struct { U64 v; } unalign64; - typedef struct { size_t v; } unalignArch; - __pragma( pack(pop) ) -#else - typedef struct { U16 v; } __attribute__((packed)) unalign16; - typedef struct { U32 v; } __attribute__((packed)) unalign32; - typedef struct { U64 v; } __attribute__((packed)) unalign64; - typedef struct { size_t v; } __attribute__((packed)) unalignArch; -#endif - -MEM_STATIC U16 MEM_read16(const void* ptr) { return ((const unalign16*)ptr)->v; } -MEM_STATIC U32 MEM_read32(const void* ptr) { return ((const unalign32*)ptr)->v; } -MEM_STATIC U64 MEM_read64(const void* ptr) { return ((const unalign64*)ptr)->v; } -MEM_STATIC size_t MEM_readST(const void* ptr) { return ((const unalignArch*)ptr)->v; } - -MEM_STATIC void MEM_write16(void* memPtr, U16 value) { ((unalign16*)memPtr)->v = value; } -MEM_STATIC void MEM_write32(void* memPtr, U32 value) { ((unalign32*)memPtr)->v = value; } -MEM_STATIC void MEM_write64(void* memPtr, U64 value) { ((unalign64*)memPtr)->v = value; } - -#else - -/* default method, safe and standard. - can sometimes prove slower */ - -MEM_STATIC U16 MEM_read16(const void* memPtr) -{ - U16 val; memcpy(&val, memPtr, sizeof(val)); return val; -} - -MEM_STATIC U32 MEM_read32(const void* memPtr) -{ - U32 val; memcpy(&val, memPtr, sizeof(val)); return val; -} - -MEM_STATIC U64 MEM_read64(const void* memPtr) -{ - U64 val; memcpy(&val, memPtr, sizeof(val)); return val; -} - -MEM_STATIC size_t MEM_readST(const void* memPtr) -{ - size_t val; memcpy(&val, memPtr, sizeof(val)); return val; -} - -MEM_STATIC void MEM_write16(void* memPtr, U16 value) -{ - memcpy(memPtr, &value, sizeof(value)); -} - -MEM_STATIC void MEM_write32(void* memPtr, U32 value) -{ - memcpy(memPtr, &value, sizeof(value)); -} - -MEM_STATIC void MEM_write64(void* memPtr, U64 value) -{ - memcpy(memPtr, &value, sizeof(value)); -} - -#endif /* MEM_FORCE_MEMORY_ACCESS */ - -MEM_STATIC U32 MEM_swap32(U32 in) -{ -#if defined(_MSC_VER) /* Visual Studio */ - return _byteswap_ulong(in); -#elif (defined (__GNUC__) && (__GNUC__ * 100 + __GNUC_MINOR__ >= 403)) \ - || (defined(__clang__) && __has_builtin(__builtin_bswap32)) - return __builtin_bswap32(in); -#else - return ((in << 24) & 0xff000000 ) | - ((in << 8) & 0x00ff0000 ) | - ((in >> 8) & 0x0000ff00 ) | - ((in >> 24) & 0x000000ff ); -#endif -} - -MEM_STATIC U64 MEM_swap64(U64 in) -{ -#if defined(_MSC_VER) /* Visual Studio */ - return _byteswap_uint64(in); -#elif (defined (__GNUC__) && (__GNUC__ * 100 + __GNUC_MINOR__ >= 403)) \ - || (defined(__clang__) && __has_builtin(__builtin_bswap64)) - return __builtin_bswap64(in); -#else - return ((in << 56) & 0xff00000000000000ULL) | - ((in << 40) & 0x00ff000000000000ULL) | - ((in << 24) & 0x0000ff0000000000ULL) | - ((in << 8) & 0x000000ff00000000ULL) | - ((in >> 8) & 0x00000000ff000000ULL) | - ((in >> 24) & 0x0000000000ff0000ULL) | - ((in >> 40) & 0x000000000000ff00ULL) | - ((in >> 56) & 0x00000000000000ffULL); -#endif -} - -MEM_STATIC size_t MEM_swapST(size_t in) -{ - if (MEM_32bits()) - return (size_t)MEM_swap32((U32)in); - else - return (size_t)MEM_swap64((U64)in); -} - -/*=== Little endian r/w ===*/ - -MEM_STATIC U16 MEM_readLE16(const void* memPtr) -{ - if (MEM_isLittleEndian()) - return MEM_read16(memPtr); - else { - const BYTE* p = (const BYTE*)memPtr; - return (U16)(p[0] + (p[1]<<8)); - } -} - -MEM_STATIC void MEM_writeLE16(void* memPtr, U16 val) -{ - if (MEM_isLittleEndian()) { - MEM_write16(memPtr, val); - } else { - BYTE* p = (BYTE*)memPtr; - p[0] = (BYTE)val; - p[1] = (BYTE)(val>>8); - } -} - -MEM_STATIC U32 MEM_readLE24(const void* memPtr) -{ - return MEM_readLE16(memPtr) + (((const BYTE*)memPtr)[2] << 16); -} - -MEM_STATIC void MEM_writeLE24(void* memPtr, U32 val) -{ - MEM_writeLE16(memPtr, (U16)val); - ((BYTE*)memPtr)[2] = (BYTE)(val>>16); -} - -MEM_STATIC U32 MEM_readLE32(const void* memPtr) -{ - if (MEM_isLittleEndian()) - return MEM_read32(memPtr); - else - return MEM_swap32(MEM_read32(memPtr)); -} - -MEM_STATIC void MEM_writeLE32(void* memPtr, U32 val32) -{ - if (MEM_isLittleEndian()) - MEM_write32(memPtr, val32); - else - MEM_write32(memPtr, MEM_swap32(val32)); -} - -MEM_STATIC U64 MEM_readLE64(const void* memPtr) -{ - if (MEM_isLittleEndian()) - return MEM_read64(memPtr); - else - return MEM_swap64(MEM_read64(memPtr)); -} - -MEM_STATIC void MEM_writeLE64(void* memPtr, U64 val64) -{ - if (MEM_isLittleEndian()) - MEM_write64(memPtr, val64); - else - MEM_write64(memPtr, MEM_swap64(val64)); -} - -MEM_STATIC size_t MEM_readLEST(const void* memPtr) -{ - if (MEM_32bits()) - return (size_t)MEM_readLE32(memPtr); - else - return (size_t)MEM_readLE64(memPtr); -} - -MEM_STATIC void MEM_writeLEST(void* memPtr, size_t val) -{ - if (MEM_32bits()) - MEM_writeLE32(memPtr, (U32)val); - else - MEM_writeLE64(memPtr, (U64)val); -} - -/*=== Big endian r/w ===*/ - -MEM_STATIC U32 MEM_readBE32(const void* memPtr) -{ - if (MEM_isLittleEndian()) - return MEM_swap32(MEM_read32(memPtr)); - else - return MEM_read32(memPtr); -} - -MEM_STATIC void MEM_writeBE32(void* memPtr, U32 val32) -{ - if (MEM_isLittleEndian()) - MEM_write32(memPtr, MEM_swap32(val32)); - else - MEM_write32(memPtr, val32); -} - -MEM_STATIC U64 MEM_readBE64(const void* memPtr) -{ - if (MEM_isLittleEndian()) - return MEM_swap64(MEM_read64(memPtr)); - else - return MEM_read64(memPtr); -} - -MEM_STATIC void MEM_writeBE64(void* memPtr, U64 val64) -{ - if (MEM_isLittleEndian()) - MEM_write64(memPtr, MEM_swap64(val64)); - else - MEM_write64(memPtr, val64); -} - -MEM_STATIC size_t MEM_readBEST(const void* memPtr) -{ - if (MEM_32bits()) - return (size_t)MEM_readBE32(memPtr); - else - return (size_t)MEM_readBE64(memPtr); -} - -MEM_STATIC void MEM_writeBEST(void* memPtr, size_t val) -{ - if (MEM_32bits()) - MEM_writeBE32(memPtr, (U32)val); - else - MEM_writeBE64(memPtr, (U64)val); -} - - -#if defined (__cplusplus) -} -#endif - -#endif /* MEM_H_MODULE */ -/**** ended inlining mem.h ****/ -/**** start inlining error_private.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -/* Note : this module is expected to remain private, do not expose it */ - -#ifndef ERROR_H_MODULE -#define ERROR_H_MODULE - -#if defined (__cplusplus) -extern "C" { -#endif - - -/* **************************************** -* Dependencies -******************************************/ -#include /* size_t */ -/**** start inlining zstd_errors.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_ERRORS_H_398273423 -#define ZSTD_ERRORS_H_398273423 - -#if defined (__cplusplus) -extern "C" { -#endif - -/*===== dependency =====*/ -#include /* size_t */ - - -/* ===== ZSTDERRORLIB_API : control library symbols visibility ===== */ -#ifndef ZSTDERRORLIB_VISIBILITY -# if defined(__GNUC__) && (__GNUC__ >= 4) -# define ZSTDERRORLIB_VISIBILITY __attribute__ ((visibility ("default"))) -# else -# define ZSTDERRORLIB_VISIBILITY -# endif -#endif -#if defined(ZSTD_DLL_EXPORT) && (ZSTD_DLL_EXPORT==1) -# define ZSTDERRORLIB_API __declspec(dllexport) ZSTDERRORLIB_VISIBILITY -#elif defined(ZSTD_DLL_IMPORT) && (ZSTD_DLL_IMPORT==1) -# define ZSTDERRORLIB_API __declspec(dllimport) ZSTDERRORLIB_VISIBILITY /* It isn't required but allows to generate better code, saving a function pointer load from the IAT and an indirect jump.*/ -#else -# define ZSTDERRORLIB_API ZSTDERRORLIB_VISIBILITY -#endif - -/*-********************************************* - * Error codes list - *-********************************************* - * Error codes _values_ are pinned down since v1.3.1 only. - * Therefore, don't rely on values if you may link to any version < v1.3.1. - * - * Only values < 100 are considered stable. - * - * note 1 : this API shall be used with static linking only. - * dynamic linking is not yet officially supported. - * note 2 : Prefer relying on the enum than on its value whenever possible - * This is the only supported way to use the error list < v1.3.1 - * note 3 : ZSTD_isError() is always correct, whatever the library version. - **********************************************/ -typedef enum { - ZSTD_error_no_error = 0, - ZSTD_error_GENERIC = 1, - ZSTD_error_prefix_unknown = 10, - ZSTD_error_version_unsupported = 12, - ZSTD_error_frameParameter_unsupported = 14, - ZSTD_error_frameParameter_windowTooLarge = 16, - ZSTD_error_corruption_detected = 20, - ZSTD_error_checksum_wrong = 22, - ZSTD_error_dictionary_corrupted = 30, - ZSTD_error_dictionary_wrong = 32, - ZSTD_error_dictionaryCreation_failed = 34, - ZSTD_error_parameter_unsupported = 40, - ZSTD_error_parameter_outOfBound = 42, - ZSTD_error_tableLog_tooLarge = 44, - ZSTD_error_maxSymbolValue_tooLarge = 46, - ZSTD_error_maxSymbolValue_tooSmall = 48, - ZSTD_error_stage_wrong = 60, - ZSTD_error_init_missing = 62, - ZSTD_error_memory_allocation = 64, - ZSTD_error_workSpace_tooSmall= 66, - ZSTD_error_dstSize_tooSmall = 70, - ZSTD_error_srcSize_wrong = 72, - ZSTD_error_dstBuffer_null = 74, - /* following error codes are __NOT STABLE__, they can be removed or changed in future versions */ - ZSTD_error_frameIndex_tooLarge = 100, - ZSTD_error_seekableIO = 102, - ZSTD_error_dstBuffer_wrong = 104, - ZSTD_error_maxCode = 120 /* never EVER use this value directly, it can change in future versions! Use ZSTD_isError() instead */ -} ZSTD_ErrorCode; - -/*! ZSTD_getErrorCode() : - convert a `size_t` function result into a `ZSTD_ErrorCode` enum type, - which can be used to compare with enum list published above */ -ZSTDERRORLIB_API ZSTD_ErrorCode ZSTD_getErrorCode(size_t functionResult); -ZSTDERRORLIB_API const char* ZSTD_getErrorString(ZSTD_ErrorCode code); /**< Same as ZSTD_getErrorName, but using a `ZSTD_ErrorCode` enum argument */ - - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTD_ERRORS_H_398273423 */ -/**** ended inlining zstd_errors.h ****/ - - -/* **************************************** -* Compiler-specific -******************************************/ -#if defined(__GNUC__) -# define ERR_STATIC static __attribute__((unused)) -#elif defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) -# define ERR_STATIC static inline -#elif defined(_MSC_VER) -# define ERR_STATIC static __inline -#else -# define ERR_STATIC static /* this version may generate warnings for unused static functions; disable the relevant warning */ -#endif - - -/*-**************************************** -* Customization (error_public.h) -******************************************/ -typedef ZSTD_ErrorCode ERR_enum; -#define PREFIX(name) ZSTD_error_##name - - -/*-**************************************** -* Error codes handling -******************************************/ -#undef ERROR /* already defined on Visual Studio */ -#define ERROR(name) ZSTD_ERROR(name) -#define ZSTD_ERROR(name) ((size_t)-PREFIX(name)) - -ERR_STATIC unsigned ERR_isError(size_t code) { return (code > ERROR(maxCode)); } - -ERR_STATIC ERR_enum ERR_getErrorCode(size_t code) { if (!ERR_isError(code)) return (ERR_enum)0; return (ERR_enum) (0-code); } - -/* check and forward error code */ -#define CHECK_V_F(e, f) size_t const e = f; if (ERR_isError(e)) return e -#define CHECK_F(f) { CHECK_V_F(_var_err__, f); } - - -/*-**************************************** -* Error Strings -******************************************/ - -const char* ERR_getErrorString(ERR_enum code); /* error_private.c */ - -ERR_STATIC const char* ERR_getErrorName(size_t code) -{ - return ERR_getErrorString(ERR_getErrorCode(code)); -} - -#if defined (__cplusplus) -} -#endif - -#endif /* ERROR_H_MODULE */ -/**** ended inlining error_private.h ****/ -#define FSE_STATIC_LINKING_ONLY /* FSE_MIN_TABLELOG */ -/**** start inlining fse.h ****/ -/* ****************************************************************** - * FSE : Finite State Entropy codec - * Public Prototypes declaration - * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. - * - * You can contact the author at : - * - Source repository : https://github.com/Cyan4973/FiniteStateEntropy - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. -****************************************************************** */ - -#if defined (__cplusplus) -extern "C" { -#endif - -#ifndef FSE_H -#define FSE_H - - -/*-***************************************** -* Dependencies -******************************************/ -#include /* size_t, ptrdiff_t */ - - -/*-***************************************** -* FSE_PUBLIC_API : control library symbols visibility -******************************************/ -#if defined(FSE_DLL_EXPORT) && (FSE_DLL_EXPORT==1) && defined(__GNUC__) && (__GNUC__ >= 4) -# define FSE_PUBLIC_API __attribute__ ((visibility ("default"))) -#elif defined(FSE_DLL_EXPORT) && (FSE_DLL_EXPORT==1) /* Visual expected */ -# define FSE_PUBLIC_API __declspec(dllexport) -#elif defined(FSE_DLL_IMPORT) && (FSE_DLL_IMPORT==1) -# define FSE_PUBLIC_API __declspec(dllimport) /* It isn't required but allows to generate better code, saving a function pointer load from the IAT and an indirect jump.*/ -#else -# define FSE_PUBLIC_API -#endif - -/*------ Version ------*/ -#define FSE_VERSION_MAJOR 0 -#define FSE_VERSION_MINOR 9 -#define FSE_VERSION_RELEASE 0 - -#define FSE_LIB_VERSION FSE_VERSION_MAJOR.FSE_VERSION_MINOR.FSE_VERSION_RELEASE -#define FSE_QUOTE(str) #str -#define FSE_EXPAND_AND_QUOTE(str) FSE_QUOTE(str) -#define FSE_VERSION_STRING FSE_EXPAND_AND_QUOTE(FSE_LIB_VERSION) - -#define FSE_VERSION_NUMBER (FSE_VERSION_MAJOR *100*100 + FSE_VERSION_MINOR *100 + FSE_VERSION_RELEASE) -FSE_PUBLIC_API unsigned FSE_versionNumber(void); /**< library version number; to be used when checking dll version */ - - -/*-**************************************** -* FSE simple functions -******************************************/ -/*! FSE_compress() : - Compress content of buffer 'src', of size 'srcSize', into destination buffer 'dst'. - 'dst' buffer must be already allocated. Compression runs faster is dstCapacity >= FSE_compressBound(srcSize). - @return : size of compressed data (<= dstCapacity). - Special values : if return == 0, srcData is not compressible => Nothing is stored within dst !!! - if return == 1, srcData is a single byte symbol * srcSize times. Use RLE compression instead. - if FSE_isError(return), compression failed (more details using FSE_getErrorName()) -*/ -FSE_PUBLIC_API size_t FSE_compress(void* dst, size_t dstCapacity, - const void* src, size_t srcSize); - -/*! FSE_decompress(): - Decompress FSE data from buffer 'cSrc', of size 'cSrcSize', - into already allocated destination buffer 'dst', of size 'dstCapacity'. - @return : size of regenerated data (<= maxDstSize), - or an error code, which can be tested using FSE_isError() . - - ** Important ** : FSE_decompress() does not decompress non-compressible nor RLE data !!! - Why ? : making this distinction requires a header. - Header management is intentionally delegated to the user layer, which can better manage special cases. -*/ -FSE_PUBLIC_API size_t FSE_decompress(void* dst, size_t dstCapacity, - const void* cSrc, size_t cSrcSize); - - -/*-***************************************** -* Tool functions -******************************************/ -FSE_PUBLIC_API size_t FSE_compressBound(size_t size); /* maximum compressed size */ - -/* Error Management */ -FSE_PUBLIC_API unsigned FSE_isError(size_t code); /* tells if a return value is an error code */ -FSE_PUBLIC_API const char* FSE_getErrorName(size_t code); /* provides error code string (useful for debugging) */ - - -/*-***************************************** -* FSE advanced functions -******************************************/ -/*! FSE_compress2() : - Same as FSE_compress(), but allows the selection of 'maxSymbolValue' and 'tableLog' - Both parameters can be defined as '0' to mean : use default value - @return : size of compressed data - Special values : if return == 0, srcData is not compressible => Nothing is stored within cSrc !!! - if return == 1, srcData is a single byte symbol * srcSize times. Use RLE compression. - if FSE_isError(return), it's an error code. -*/ -FSE_PUBLIC_API size_t FSE_compress2 (void* dst, size_t dstSize, const void* src, size_t srcSize, unsigned maxSymbolValue, unsigned tableLog); - - -/*-***************************************** -* FSE detailed API -******************************************/ -/*! -FSE_compress() does the following: -1. count symbol occurrence from source[] into table count[] (see hist.h) -2. normalize counters so that sum(count[]) == Power_of_2 (2^tableLog) -3. save normalized counters to memory buffer using writeNCount() -4. build encoding table 'CTable' from normalized counters -5. encode the data stream using encoding table 'CTable' - -FSE_decompress() does the following: -1. read normalized counters with readNCount() -2. build decoding table 'DTable' from normalized counters -3. decode the data stream using decoding table 'DTable' - -The following API allows targeting specific sub-functions for advanced tasks. -For example, it's possible to compress several blocks using the same 'CTable', -or to save and provide normalized distribution using external method. -*/ - -/* *** COMPRESSION *** */ - -/*! FSE_optimalTableLog(): - dynamically downsize 'tableLog' when conditions are met. - It saves CPU time, by using smaller tables, while preserving or even improving compression ratio. - @return : recommended tableLog (necessarily <= 'maxTableLog') */ -FSE_PUBLIC_API unsigned FSE_optimalTableLog(unsigned maxTableLog, size_t srcSize, unsigned maxSymbolValue); - -/*! FSE_normalizeCount(): - normalize counts so that sum(count[]) == Power_of_2 (2^tableLog) - 'normalizedCounter' is a table of short, of minimum size (maxSymbolValue+1). - @return : tableLog, - or an errorCode, which can be tested using FSE_isError() */ -FSE_PUBLIC_API size_t FSE_normalizeCount(short* normalizedCounter, unsigned tableLog, - const unsigned* count, size_t srcSize, unsigned maxSymbolValue); - -/*! FSE_NCountWriteBound(): - Provides the maximum possible size of an FSE normalized table, given 'maxSymbolValue' and 'tableLog'. - Typically useful for allocation purpose. */ -FSE_PUBLIC_API size_t FSE_NCountWriteBound(unsigned maxSymbolValue, unsigned tableLog); - -/*! FSE_writeNCount(): - Compactly save 'normalizedCounter' into 'buffer'. - @return : size of the compressed table, - or an errorCode, which can be tested using FSE_isError(). */ -FSE_PUBLIC_API size_t FSE_writeNCount (void* buffer, size_t bufferSize, - const short* normalizedCounter, - unsigned maxSymbolValue, unsigned tableLog); - -/*! Constructor and Destructor of FSE_CTable. - Note that FSE_CTable size depends on 'tableLog' and 'maxSymbolValue' */ -typedef unsigned FSE_CTable; /* don't allocate that. It's only meant to be more restrictive than void* */ -FSE_PUBLIC_API FSE_CTable* FSE_createCTable (unsigned maxSymbolValue, unsigned tableLog); -FSE_PUBLIC_API void FSE_freeCTable (FSE_CTable* ct); - -/*! FSE_buildCTable(): - Builds `ct`, which must be already allocated, using FSE_createCTable(). - @return : 0, or an errorCode, which can be tested using FSE_isError() */ -FSE_PUBLIC_API size_t FSE_buildCTable(FSE_CTable* ct, const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog); - -/*! FSE_compress_usingCTable(): - Compress `src` using `ct` into `dst` which must be already allocated. - @return : size of compressed data (<= `dstCapacity`), - or 0 if compressed data could not fit into `dst`, - or an errorCode, which can be tested using FSE_isError() */ -FSE_PUBLIC_API size_t FSE_compress_usingCTable (void* dst, size_t dstCapacity, const void* src, size_t srcSize, const FSE_CTable* ct); - -/*! -Tutorial : ----------- -The first step is to count all symbols. FSE_count() does this job very fast. -Result will be saved into 'count', a table of unsigned int, which must be already allocated, and have 'maxSymbolValuePtr[0]+1' cells. -'src' is a table of bytes of size 'srcSize'. All values within 'src' MUST be <= maxSymbolValuePtr[0] -maxSymbolValuePtr[0] will be updated, with its real value (necessarily <= original value) -FSE_count() will return the number of occurrence of the most frequent symbol. -This can be used to know if there is a single symbol within 'src', and to quickly evaluate its compressibility. -If there is an error, the function will return an ErrorCode (which can be tested using FSE_isError()). - -The next step is to normalize the frequencies. -FSE_normalizeCount() will ensure that sum of frequencies is == 2 ^'tableLog'. -It also guarantees a minimum of 1 to any Symbol with frequency >= 1. -You can use 'tableLog'==0 to mean "use default tableLog value". -If you are unsure of which tableLog value to use, you can ask FSE_optimalTableLog(), -which will provide the optimal valid tableLog given sourceSize, maxSymbolValue, and a user-defined maximum (0 means "default"). - -The result of FSE_normalizeCount() will be saved into a table, -called 'normalizedCounter', which is a table of signed short. -'normalizedCounter' must be already allocated, and have at least 'maxSymbolValue+1' cells. -The return value is tableLog if everything proceeded as expected. -It is 0 if there is a single symbol within distribution. -If there is an error (ex: invalid tableLog value), the function will return an ErrorCode (which can be tested using FSE_isError()). - -'normalizedCounter' can be saved in a compact manner to a memory area using FSE_writeNCount(). -'buffer' must be already allocated. -For guaranteed success, buffer size must be at least FSE_headerBound(). -The result of the function is the number of bytes written into 'buffer'. -If there is an error, the function will return an ErrorCode (which can be tested using FSE_isError(); ex : buffer size too small). - -'normalizedCounter' can then be used to create the compression table 'CTable'. -The space required by 'CTable' must be already allocated, using FSE_createCTable(). -You can then use FSE_buildCTable() to fill 'CTable'. -If there is an error, both functions will return an ErrorCode (which can be tested using FSE_isError()). - -'CTable' can then be used to compress 'src', with FSE_compress_usingCTable(). -Similar to FSE_count(), the convention is that 'src' is assumed to be a table of char of size 'srcSize' -The function returns the size of compressed data (without header), necessarily <= `dstCapacity`. -If it returns '0', compressed data could not fit into 'dst'. -If there is an error, the function will return an ErrorCode (which can be tested using FSE_isError()). -*/ - - -/* *** DECOMPRESSION *** */ - -/*! FSE_readNCount(): - Read compactly saved 'normalizedCounter' from 'rBuffer'. - @return : size read from 'rBuffer', - or an errorCode, which can be tested using FSE_isError(). - maxSymbolValuePtr[0] and tableLogPtr[0] will also be updated with their respective values */ -FSE_PUBLIC_API size_t FSE_readNCount (short* normalizedCounter, - unsigned* maxSymbolValuePtr, unsigned* tableLogPtr, - const void* rBuffer, size_t rBuffSize); - -/*! Constructor and Destructor of FSE_DTable. - Note that its size depends on 'tableLog' */ -typedef unsigned FSE_DTable; /* don't allocate that. It's just a way to be more restrictive than void* */ -FSE_PUBLIC_API FSE_DTable* FSE_createDTable(unsigned tableLog); -FSE_PUBLIC_API void FSE_freeDTable(FSE_DTable* dt); - -/*! FSE_buildDTable(): - Builds 'dt', which must be already allocated, using FSE_createDTable(). - return : 0, or an errorCode, which can be tested using FSE_isError() */ -FSE_PUBLIC_API size_t FSE_buildDTable (FSE_DTable* dt, const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog); - -/*! FSE_decompress_usingDTable(): - Decompress compressed source `cSrc` of size `cSrcSize` using `dt` - into `dst` which must be already allocated. - @return : size of regenerated data (necessarily <= `dstCapacity`), - or an errorCode, which can be tested using FSE_isError() */ -FSE_PUBLIC_API size_t FSE_decompress_usingDTable(void* dst, size_t dstCapacity, const void* cSrc, size_t cSrcSize, const FSE_DTable* dt); - -/*! -Tutorial : ----------- -(Note : these functions only decompress FSE-compressed blocks. - If block is uncompressed, use memcpy() instead - If block is a single repeated byte, use memset() instead ) - -The first step is to obtain the normalized frequencies of symbols. -This can be performed by FSE_readNCount() if it was saved using FSE_writeNCount(). -'normalizedCounter' must be already allocated, and have at least 'maxSymbolValuePtr[0]+1' cells of signed short. -In practice, that means it's necessary to know 'maxSymbolValue' beforehand, -or size the table to handle worst case situations (typically 256). -FSE_readNCount() will provide 'tableLog' and 'maxSymbolValue'. -The result of FSE_readNCount() is the number of bytes read from 'rBuffer'. -Note that 'rBufferSize' must be at least 4 bytes, even if useful information is less than that. -If there is an error, the function will return an error code, which can be tested using FSE_isError(). - -The next step is to build the decompression tables 'FSE_DTable' from 'normalizedCounter'. -This is performed by the function FSE_buildDTable(). -The space required by 'FSE_DTable' must be already allocated using FSE_createDTable(). -If there is an error, the function will return an error code, which can be tested using FSE_isError(). - -`FSE_DTable` can then be used to decompress `cSrc`, with FSE_decompress_usingDTable(). -`cSrcSize` must be strictly correct, otherwise decompression will fail. -FSE_decompress_usingDTable() result will tell how many bytes were regenerated (<=`dstCapacity`). -If there is an error, the function will return an error code, which can be tested using FSE_isError(). (ex: dst buffer too small) -*/ - -#endif /* FSE_H */ - -#if defined(FSE_STATIC_LINKING_ONLY) && !defined(FSE_H_FSE_STATIC_LINKING_ONLY) -#define FSE_H_FSE_STATIC_LINKING_ONLY - -/* *** Dependency *** */ -/**** start inlining bitstream.h ****/ -/* ****************************************************************** - * bitstream - * Part of FSE library - * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. - * - * You can contact the author at : - * - Source repository : https://github.com/Cyan4973/FiniteStateEntropy - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. -****************************************************************** */ -#ifndef BITSTREAM_H_MODULE -#define BITSTREAM_H_MODULE - -#if defined (__cplusplus) -extern "C" { -#endif - -/* -* This API consists of small unitary functions, which must be inlined for best performance. -* Since link-time-optimization is not available for all compilers, -* these functions are defined into a .h to be included. -*/ - -/*-**************************************** -* Dependencies -******************************************/ -/**** skipping file: mem.h ****/ -/**** start inlining compiler.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_COMPILER_H -#define ZSTD_COMPILER_H - -/*-******************************************************* -* Compiler specifics -*********************************************************/ -/* force inlining */ - -#if !defined(ZSTD_NO_INLINE) -#if (defined(__GNUC__) && !defined(__STRICT_ANSI__)) || defined(__cplusplus) || defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L /* C99 */ -# define INLINE_KEYWORD inline -#else -# define INLINE_KEYWORD -#endif - -#if defined(__GNUC__) || defined(__ICCARM__) -# define FORCE_INLINE_ATTR __attribute__((always_inline)) -#elif defined(_MSC_VER) -# define FORCE_INLINE_ATTR __forceinline -#else -# define FORCE_INLINE_ATTR -#endif - -#else - -#define INLINE_KEYWORD -#define FORCE_INLINE_ATTR - -#endif - -/** - * FORCE_INLINE_TEMPLATE is used to define C "templates", which take constant - * parameters. They must be inlined for the compiler to eliminate the constant - * branches. - */ -#define FORCE_INLINE_TEMPLATE static INLINE_KEYWORD FORCE_INLINE_ATTR -/** - * HINT_INLINE is used to help the compiler generate better code. It is *not* - * used for "templates", so it can be tweaked based on the compilers - * performance. - * - * gcc-4.8 and gcc-4.9 have been shown to benefit from leaving off the - * always_inline attribute. - * - * clang up to 5.0.0 (trunk) benefit tremendously from the always_inline - * attribute. - */ -#if !defined(__clang__) && defined(__GNUC__) && __GNUC__ >= 4 && __GNUC_MINOR__ >= 8 && __GNUC__ < 5 -# define HINT_INLINE static INLINE_KEYWORD -#else -# define HINT_INLINE static INLINE_KEYWORD FORCE_INLINE_ATTR -#endif - -/* UNUSED_ATTR tells the compiler it is okay if the function is unused. */ -#if defined(__GNUC__) -# define UNUSED_ATTR __attribute__((unused)) -#else -# define UNUSED_ATTR -#endif - -/* force no inlining */ -#ifdef _MSC_VER -# define FORCE_NOINLINE static __declspec(noinline) -#else -# if defined(__GNUC__) || defined(__ICCARM__) -# define FORCE_NOINLINE static __attribute__((__noinline__)) -# else -# define FORCE_NOINLINE static -# endif -#endif - -/* target attribute */ -#ifndef __has_attribute - #define __has_attribute(x) 0 /* Compatibility with non-clang compilers. */ -#endif -#if defined(__GNUC__) || defined(__ICCARM__) -# define TARGET_ATTRIBUTE(target) __attribute__((__target__(target))) -#else -# define TARGET_ATTRIBUTE(target) -#endif - -/* Enable runtime BMI2 dispatch based on the CPU. - * Enabled for clang & gcc >=4.8 on x86 when BMI2 isn't enabled by default. - */ -#ifndef DYNAMIC_BMI2 - #if ((defined(__clang__) && __has_attribute(__target__)) \ - || (defined(__GNUC__) \ - && (__GNUC__ >= 5 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 8)))) \ - && (defined(__x86_64__) || defined(_M_X86)) \ - && !defined(__BMI2__) - # define DYNAMIC_BMI2 1 - #else - # define DYNAMIC_BMI2 0 - #endif -#endif - -/* prefetch - * can be disabled, by declaring NO_PREFETCH build macro */ -#if defined(NO_PREFETCH) -# define PREFETCH_L1(ptr) (void)(ptr) /* disabled */ -# define PREFETCH_L2(ptr) (void)(ptr) /* disabled */ -#else -# if defined(_MSC_VER) && (defined(_M_X64) || defined(_M_I86)) /* _mm_prefetch() is not defined outside of x86/x64 */ -# include /* https://msdn.microsoft.com/fr-fr/library/84szxsww(v=vs.90).aspx */ -# define PREFETCH_L1(ptr) _mm_prefetch((const char*)(ptr), _MM_HINT_T0) -# define PREFETCH_L2(ptr) _mm_prefetch((const char*)(ptr), _MM_HINT_T1) -# elif defined(__aarch64__) -# define PREFETCH_L1(ptr) __asm__ __volatile__("prfm pldl1keep, %0" ::"Q"(*(ptr))) -# define PREFETCH_L2(ptr) __asm__ __volatile__("prfm pldl2keep, %0" ::"Q"(*(ptr))) -# elif defined(__GNUC__) && ( (__GNUC__ >= 4) || ( (__GNUC__ == 3) && (__GNUC_MINOR__ >= 1) ) ) -# define PREFETCH_L1(ptr) __builtin_prefetch((ptr), 0 /* rw==read */, 3 /* locality */) -# define PREFETCH_L2(ptr) __builtin_prefetch((ptr), 0 /* rw==read */, 2 /* locality */) -# else -# define PREFETCH_L1(ptr) (void)(ptr) /* disabled */ -# define PREFETCH_L2(ptr) (void)(ptr) /* disabled */ -# endif -#endif /* NO_PREFETCH */ - -#define CACHELINE_SIZE 64 - -#define PREFETCH_AREA(p, s) { \ - const char* const _ptr = (const char*)(p); \ - size_t const _size = (size_t)(s); \ - size_t _pos; \ - for (_pos=0; _pos<_size; _pos+=CACHELINE_SIZE) { \ - PREFETCH_L2(_ptr + _pos); \ - } \ -} - -/* vectorization - * older GCC (pre gcc-4.3 picked as the cutoff) uses a different syntax */ -#if !defined(__INTEL_COMPILER) && !defined(__clang__) && defined(__GNUC__) -# if (__GNUC__ == 4 && __GNUC_MINOR__ > 3) || (__GNUC__ >= 5) -# define DONT_VECTORIZE __attribute__((optimize("no-tree-vectorize"))) -# else -# define DONT_VECTORIZE _Pragma("GCC optimize(\"no-tree-vectorize\")") -# endif -#else -# define DONT_VECTORIZE -#endif - -/* Tell the compiler that a branch is likely or unlikely. - * Only use these macros if it causes the compiler to generate better code. - * If you can remove a LIKELY/UNLIKELY annotation without speed changes in gcc - * and clang, please do. - */ -#if defined(__GNUC__) -#define LIKELY(x) (__builtin_expect((x), 1)) -#define UNLIKELY(x) (__builtin_expect((x), 0)) -#else -#define LIKELY(x) (x) -#define UNLIKELY(x) (x) -#endif - -/* disable warnings */ -#ifdef _MSC_VER /* Visual Studio */ -# include /* For Visual 2005 */ -# pragma warning(disable : 4100) /* disable: C4100: unreferenced formal parameter */ -# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */ -# pragma warning(disable : 4204) /* disable: C4204: non-constant aggregate initializer */ -# pragma warning(disable : 4214) /* disable: C4214: non-int bitfields */ -# pragma warning(disable : 4324) /* disable: C4324: padded structure */ -#endif - -#endif /* ZSTD_COMPILER_H */ -/**** ended inlining compiler.h ****/ -/**** skipping file: debug.h ****/ -/**** skipping file: error_private.h ****/ - - -/*========================================= -* Target specific -=========================================*/ -#if defined(__BMI__) && defined(__GNUC__) -# include /* support for bextr (experimental) */ -#elif defined(__ICCARM__) -# include -#endif - -#define STREAM_ACCUMULATOR_MIN_32 25 -#define STREAM_ACCUMULATOR_MIN_64 57 -#define STREAM_ACCUMULATOR_MIN ((U32)(MEM_32bits() ? STREAM_ACCUMULATOR_MIN_32 : STREAM_ACCUMULATOR_MIN_64)) - - -/*-****************************************** -* bitStream encoding API (write forward) -********************************************/ -/* bitStream can mix input from multiple sources. - * A critical property of these streams is that they encode and decode in **reverse** direction. - * So the first bit sequence you add will be the last to be read, like a LIFO stack. - */ -typedef struct { - size_t bitContainer; - unsigned bitPos; - char* startPtr; - char* ptr; - char* endPtr; -} BIT_CStream_t; - -MEM_STATIC size_t BIT_initCStream(BIT_CStream_t* bitC, void* dstBuffer, size_t dstCapacity); -MEM_STATIC void BIT_addBits(BIT_CStream_t* bitC, size_t value, unsigned nbBits); -MEM_STATIC void BIT_flushBits(BIT_CStream_t* bitC); -MEM_STATIC size_t BIT_closeCStream(BIT_CStream_t* bitC); - -/* Start with initCStream, providing the size of buffer to write into. -* bitStream will never write outside of this buffer. -* `dstCapacity` must be >= sizeof(bitD->bitContainer), otherwise @return will be an error code. -* -* bits are first added to a local register. -* Local register is size_t, hence 64-bits on 64-bits systems, or 32-bits on 32-bits systems. -* Writing data into memory is an explicit operation, performed by the flushBits function. -* Hence keep track how many bits are potentially stored into local register to avoid register overflow. -* After a flushBits, a maximum of 7 bits might still be stored into local register. -* -* Avoid storing elements of more than 24 bits if you want compatibility with 32-bits bitstream readers. -* -* Last operation is to close the bitStream. -* The function returns the final size of CStream in bytes. -* If data couldn't fit into `dstBuffer`, it will return a 0 ( == not storable) -*/ - - -/*-******************************************** -* bitStream decoding API (read backward) -**********************************************/ -typedef struct { - size_t bitContainer; - unsigned bitsConsumed; - const char* ptr; - const char* start; - const char* limitPtr; -} BIT_DStream_t; - -typedef enum { BIT_DStream_unfinished = 0, - BIT_DStream_endOfBuffer = 1, - BIT_DStream_completed = 2, - BIT_DStream_overflow = 3 } BIT_DStream_status; /* result of BIT_reloadDStream() */ - /* 1,2,4,8 would be better for bitmap combinations, but slows down performance a bit ... :( */ - -MEM_STATIC size_t BIT_initDStream(BIT_DStream_t* bitD, const void* srcBuffer, size_t srcSize); -MEM_STATIC size_t BIT_readBits(BIT_DStream_t* bitD, unsigned nbBits); -MEM_STATIC BIT_DStream_status BIT_reloadDStream(BIT_DStream_t* bitD); -MEM_STATIC unsigned BIT_endOfDStream(const BIT_DStream_t* bitD); - - -/* Start by invoking BIT_initDStream(). -* A chunk of the bitStream is then stored into a local register. -* Local register size is 64-bits on 64-bits systems, 32-bits on 32-bits systems (size_t). -* You can then retrieve bitFields stored into the local register, **in reverse order**. -* Local register is explicitly reloaded from memory by the BIT_reloadDStream() method. -* A reload guarantee a minimum of ((8*sizeof(bitD->bitContainer))-7) bits when its result is BIT_DStream_unfinished. -* Otherwise, it can be less than that, so proceed accordingly. -* Checking if DStream has reached its end can be performed with BIT_endOfDStream(). -*/ - - -/*-**************************************** -* unsafe API -******************************************/ -MEM_STATIC void BIT_addBitsFast(BIT_CStream_t* bitC, size_t value, unsigned nbBits); -/* faster, but works only if value is "clean", meaning all high bits above nbBits are 0 */ - -MEM_STATIC void BIT_flushBitsFast(BIT_CStream_t* bitC); -/* unsafe version; does not check buffer overflow */ - -MEM_STATIC size_t BIT_readBitsFast(BIT_DStream_t* bitD, unsigned nbBits); -/* faster, but works only if nbBits >= 1 */ - - - -/*-************************************************************** -* Internal functions -****************************************************************/ -MEM_STATIC unsigned BIT_highbit32 (U32 val) -{ - assert(val != 0); - { -# if defined(_MSC_VER) /* Visual */ - unsigned long r=0; - return _BitScanReverse ( &r, val ) ? (unsigned)r : 0; -# elif defined(__GNUC__) && (__GNUC__ >= 3) /* Use GCC Intrinsic */ - return __builtin_clz (val) ^ 31; -# elif defined(__ICCARM__) /* IAR Intrinsic */ - return 31 - __CLZ(val); -# else /* Software version */ - static const unsigned DeBruijnClz[32] = { 0, 9, 1, 10, 13, 21, 2, 29, - 11, 14, 16, 18, 22, 25, 3, 30, - 8, 12, 20, 28, 15, 17, 24, 7, - 19, 27, 23, 6, 26, 5, 4, 31 }; - U32 v = val; - v |= v >> 1; - v |= v >> 2; - v |= v >> 4; - v |= v >> 8; - v |= v >> 16; - return DeBruijnClz[ (U32) (v * 0x07C4ACDDU) >> 27]; -# endif - } -} - -/*===== Local Constants =====*/ -static const unsigned BIT_mask[] = { - 0, 1, 3, 7, 0xF, 0x1F, - 0x3F, 0x7F, 0xFF, 0x1FF, 0x3FF, 0x7FF, - 0xFFF, 0x1FFF, 0x3FFF, 0x7FFF, 0xFFFF, 0x1FFFF, - 0x3FFFF, 0x7FFFF, 0xFFFFF, 0x1FFFFF, 0x3FFFFF, 0x7FFFFF, - 0xFFFFFF, 0x1FFFFFF, 0x3FFFFFF, 0x7FFFFFF, 0xFFFFFFF, 0x1FFFFFFF, - 0x3FFFFFFF, 0x7FFFFFFF}; /* up to 31 bits */ -#define BIT_MASK_SIZE (sizeof(BIT_mask) / sizeof(BIT_mask[0])) - -/*-************************************************************** -* bitStream encoding -****************************************************************/ -/*! BIT_initCStream() : - * `dstCapacity` must be > sizeof(size_t) - * @return : 0 if success, - * otherwise an error code (can be tested using ERR_isError()) */ -MEM_STATIC size_t BIT_initCStream(BIT_CStream_t* bitC, - void* startPtr, size_t dstCapacity) -{ - bitC->bitContainer = 0; - bitC->bitPos = 0; - bitC->startPtr = (char*)startPtr; - bitC->ptr = bitC->startPtr; - bitC->endPtr = bitC->startPtr + dstCapacity - sizeof(bitC->bitContainer); - if (dstCapacity <= sizeof(bitC->bitContainer)) return ERROR(dstSize_tooSmall); - return 0; -} - -/*! BIT_addBits() : - * can add up to 31 bits into `bitC`. - * Note : does not check for register overflow ! */ -MEM_STATIC void BIT_addBits(BIT_CStream_t* bitC, - size_t value, unsigned nbBits) -{ - MEM_STATIC_ASSERT(BIT_MASK_SIZE == 32); - assert(nbBits < BIT_MASK_SIZE); - assert(nbBits + bitC->bitPos < sizeof(bitC->bitContainer) * 8); - bitC->bitContainer |= (value & BIT_mask[nbBits]) << bitC->bitPos; - bitC->bitPos += nbBits; -} - -/*! BIT_addBitsFast() : - * works only if `value` is _clean_, - * meaning all high bits above nbBits are 0 */ -MEM_STATIC void BIT_addBitsFast(BIT_CStream_t* bitC, - size_t value, unsigned nbBits) -{ - assert((value>>nbBits) == 0); - assert(nbBits + bitC->bitPos < sizeof(bitC->bitContainer) * 8); - bitC->bitContainer |= value << bitC->bitPos; - bitC->bitPos += nbBits; -} - -/*! BIT_flushBitsFast() : - * assumption : bitContainer has not overflowed - * unsafe version; does not check buffer overflow */ -MEM_STATIC void BIT_flushBitsFast(BIT_CStream_t* bitC) -{ - size_t const nbBytes = bitC->bitPos >> 3; - assert(bitC->bitPos < sizeof(bitC->bitContainer) * 8); - assert(bitC->ptr <= bitC->endPtr); - MEM_writeLEST(bitC->ptr, bitC->bitContainer); - bitC->ptr += nbBytes; - bitC->bitPos &= 7; - bitC->bitContainer >>= nbBytes*8; -} - -/*! BIT_flushBits() : - * assumption : bitContainer has not overflowed - * safe version; check for buffer overflow, and prevents it. - * note : does not signal buffer overflow. - * overflow will be revealed later on using BIT_closeCStream() */ -MEM_STATIC void BIT_flushBits(BIT_CStream_t* bitC) -{ - size_t const nbBytes = bitC->bitPos >> 3; - assert(bitC->bitPos < sizeof(bitC->bitContainer) * 8); - assert(bitC->ptr <= bitC->endPtr); - MEM_writeLEST(bitC->ptr, bitC->bitContainer); - bitC->ptr += nbBytes; - if (bitC->ptr > bitC->endPtr) bitC->ptr = bitC->endPtr; - bitC->bitPos &= 7; - bitC->bitContainer >>= nbBytes*8; -} - -/*! BIT_closeCStream() : - * @return : size of CStream, in bytes, - * or 0 if it could not fit into dstBuffer */ -MEM_STATIC size_t BIT_closeCStream(BIT_CStream_t* bitC) -{ - BIT_addBitsFast(bitC, 1, 1); /* endMark */ - BIT_flushBits(bitC); - if (bitC->ptr >= bitC->endPtr) return 0; /* overflow detected */ - return (bitC->ptr - bitC->startPtr) + (bitC->bitPos > 0); -} - - -/*-******************************************************** -* bitStream decoding -**********************************************************/ -/*! BIT_initDStream() : - * Initialize a BIT_DStream_t. - * `bitD` : a pointer to an already allocated BIT_DStream_t structure. - * `srcSize` must be the *exact* size of the bitStream, in bytes. - * @return : size of stream (== srcSize), or an errorCode if a problem is detected - */ -MEM_STATIC size_t BIT_initDStream(BIT_DStream_t* bitD, const void* srcBuffer, size_t srcSize) -{ - if (srcSize < 1) { memset(bitD, 0, sizeof(*bitD)); return ERROR(srcSize_wrong); } - - bitD->start = (const char*)srcBuffer; - bitD->limitPtr = bitD->start + sizeof(bitD->bitContainer); - - if (srcSize >= sizeof(bitD->bitContainer)) { /* normal case */ - bitD->ptr = (const char*)srcBuffer + srcSize - sizeof(bitD->bitContainer); - bitD->bitContainer = MEM_readLEST(bitD->ptr); - { BYTE const lastByte = ((const BYTE*)srcBuffer)[srcSize-1]; - bitD->bitsConsumed = lastByte ? 8 - BIT_highbit32(lastByte) : 0; /* ensures bitsConsumed is always set */ - if (lastByte == 0) return ERROR(GENERIC); /* endMark not present */ } - } else { - bitD->ptr = bitD->start; - bitD->bitContainer = *(const BYTE*)(bitD->start); - switch(srcSize) - { - case 7: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[6]) << (sizeof(bitD->bitContainer)*8 - 16); - /* fall-through */ - - case 6: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[5]) << (sizeof(bitD->bitContainer)*8 - 24); - /* fall-through */ - - case 5: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[4]) << (sizeof(bitD->bitContainer)*8 - 32); - /* fall-through */ - - case 4: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[3]) << 24; - /* fall-through */ - - case 3: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[2]) << 16; - /* fall-through */ - - case 2: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[1]) << 8; - /* fall-through */ - - default: break; - } - { BYTE const lastByte = ((const BYTE*)srcBuffer)[srcSize-1]; - bitD->bitsConsumed = lastByte ? 8 - BIT_highbit32(lastByte) : 0; - if (lastByte == 0) return ERROR(corruption_detected); /* endMark not present */ - } - bitD->bitsConsumed += (U32)(sizeof(bitD->bitContainer) - srcSize)*8; - } - - return srcSize; -} - -MEM_STATIC size_t BIT_getUpperBits(size_t bitContainer, U32 const start) -{ - return bitContainer >> start; -} - -MEM_STATIC size_t BIT_getMiddleBits(size_t bitContainer, U32 const start, U32 const nbBits) -{ - U32 const regMask = sizeof(bitContainer)*8 - 1; - /* if start > regMask, bitstream is corrupted, and result is undefined */ - assert(nbBits < BIT_MASK_SIZE); - return (bitContainer >> (start & regMask)) & BIT_mask[nbBits]; -} - -MEM_STATIC size_t BIT_getLowerBits(size_t bitContainer, U32 const nbBits) -{ - assert(nbBits < BIT_MASK_SIZE); - return bitContainer & BIT_mask[nbBits]; -} - -/*! BIT_lookBits() : - * Provides next n bits from local register. - * local register is not modified. - * On 32-bits, maxNbBits==24. - * On 64-bits, maxNbBits==56. - * @return : value extracted */ -MEM_STATIC size_t BIT_lookBits(const BIT_DStream_t* bitD, U32 nbBits) -{ - /* arbitrate between double-shift and shift+mask */ -#if 1 - /* if bitD->bitsConsumed + nbBits > sizeof(bitD->bitContainer)*8, - * bitstream is likely corrupted, and result is undefined */ - return BIT_getMiddleBits(bitD->bitContainer, (sizeof(bitD->bitContainer)*8) - bitD->bitsConsumed - nbBits, nbBits); -#else - /* this code path is slower on my os-x laptop */ - U32 const regMask = sizeof(bitD->bitContainer)*8 - 1; - return ((bitD->bitContainer << (bitD->bitsConsumed & regMask)) >> 1) >> ((regMask-nbBits) & regMask); -#endif -} - -/*! BIT_lookBitsFast() : - * unsafe version; only works if nbBits >= 1 */ -MEM_STATIC size_t BIT_lookBitsFast(const BIT_DStream_t* bitD, U32 nbBits) -{ - U32 const regMask = sizeof(bitD->bitContainer)*8 - 1; - assert(nbBits >= 1); - return (bitD->bitContainer << (bitD->bitsConsumed & regMask)) >> (((regMask+1)-nbBits) & regMask); -} - -MEM_STATIC void BIT_skipBits(BIT_DStream_t* bitD, U32 nbBits) -{ - bitD->bitsConsumed += nbBits; -} - -/*! BIT_readBits() : - * Read (consume) next n bits from local register and update. - * Pay attention to not read more than nbBits contained into local register. - * @return : extracted value. */ -MEM_STATIC size_t BIT_readBits(BIT_DStream_t* bitD, unsigned nbBits) -{ - size_t const value = BIT_lookBits(bitD, nbBits); - BIT_skipBits(bitD, nbBits); - return value; -} - -/*! BIT_readBitsFast() : - * unsafe version; only works only if nbBits >= 1 */ -MEM_STATIC size_t BIT_readBitsFast(BIT_DStream_t* bitD, unsigned nbBits) -{ - size_t const value = BIT_lookBitsFast(bitD, nbBits); - assert(nbBits >= 1); - BIT_skipBits(bitD, nbBits); - return value; -} - -/*! BIT_reloadDStreamFast() : - * Similar to BIT_reloadDStream(), but with two differences: - * 1. bitsConsumed <= sizeof(bitD->bitContainer)*8 must hold! - * 2. Returns BIT_DStream_overflow when bitD->ptr < bitD->limitPtr, at this - * point you must use BIT_reloadDStream() to reload. - */ -MEM_STATIC BIT_DStream_status BIT_reloadDStreamFast(BIT_DStream_t* bitD) -{ - if (UNLIKELY(bitD->ptr < bitD->limitPtr)) - return BIT_DStream_overflow; - assert(bitD->bitsConsumed <= sizeof(bitD->bitContainer)*8); - bitD->ptr -= bitD->bitsConsumed >> 3; - bitD->bitsConsumed &= 7; - bitD->bitContainer = MEM_readLEST(bitD->ptr); - return BIT_DStream_unfinished; -} - -/*! BIT_reloadDStream() : - * Refill `bitD` from buffer previously set in BIT_initDStream() . - * This function is safe, it guarantees it will not read beyond src buffer. - * @return : status of `BIT_DStream_t` internal register. - * when status == BIT_DStream_unfinished, internal register is filled with at least 25 or 57 bits */ -MEM_STATIC BIT_DStream_status BIT_reloadDStream(BIT_DStream_t* bitD) -{ - if (bitD->bitsConsumed > (sizeof(bitD->bitContainer)*8)) /* overflow detected, like end of stream */ - return BIT_DStream_overflow; - - if (bitD->ptr >= bitD->limitPtr) { - return BIT_reloadDStreamFast(bitD); - } - if (bitD->ptr == bitD->start) { - if (bitD->bitsConsumed < sizeof(bitD->bitContainer)*8) return BIT_DStream_endOfBuffer; - return BIT_DStream_completed; - } - /* start < ptr < limitPtr */ - { U32 nbBytes = bitD->bitsConsumed >> 3; - BIT_DStream_status result = BIT_DStream_unfinished; - if (bitD->ptr - nbBytes < bitD->start) { - nbBytes = (U32)(bitD->ptr - bitD->start); /* ptr > start */ - result = BIT_DStream_endOfBuffer; - } - bitD->ptr -= nbBytes; - bitD->bitsConsumed -= nbBytes*8; - bitD->bitContainer = MEM_readLEST(bitD->ptr); /* reminder : srcSize > sizeof(bitD->bitContainer), otherwise bitD->ptr == bitD->start */ - return result; - } -} - -/*! BIT_endOfDStream() : - * @return : 1 if DStream has _exactly_ reached its end (all bits consumed). - */ -MEM_STATIC unsigned BIT_endOfDStream(const BIT_DStream_t* DStream) -{ - return ((DStream->ptr == DStream->start) && (DStream->bitsConsumed == sizeof(DStream->bitContainer)*8)); -} - -#if defined (__cplusplus) -} -#endif - -#endif /* BITSTREAM_H_MODULE */ -/**** ended inlining bitstream.h ****/ - - -/* ***************************************** -* Static allocation -*******************************************/ -/* FSE buffer bounds */ -#define FSE_NCOUNTBOUND 512 -#define FSE_BLOCKBOUND(size) (size + (size>>7) + 4 /* fse states */ + sizeof(size_t) /* bitContainer */) -#define FSE_COMPRESSBOUND(size) (FSE_NCOUNTBOUND + FSE_BLOCKBOUND(size)) /* Macro version, useful for static allocation */ - -/* It is possible to statically allocate FSE CTable/DTable as a table of FSE_CTable/FSE_DTable using below macros */ -#define FSE_CTABLE_SIZE_U32(maxTableLog, maxSymbolValue) (1 + (1<<(maxTableLog-1)) + ((maxSymbolValue+1)*2)) -#define FSE_DTABLE_SIZE_U32(maxTableLog) (1 + (1< 12) ? (1 << (maxTableLog - 2)) : 1024) ) -size_t FSE_compress_wksp (void* dst, size_t dstSize, const void* src, size_t srcSize, unsigned maxSymbolValue, unsigned tableLog, void* workSpace, size_t wkspSize); - -size_t FSE_buildCTable_raw (FSE_CTable* ct, unsigned nbBits); -/**< build a fake FSE_CTable, designed for a flat distribution, where each symbol uses nbBits */ - -size_t FSE_buildCTable_rle (FSE_CTable* ct, unsigned char symbolValue); -/**< build a fake FSE_CTable, designed to compress always the same symbolValue */ - -/* FSE_buildCTable_wksp() : - * Same as FSE_buildCTable(), but using an externally allocated scratch buffer (`workSpace`). - * `wkspSize` must be >= `(1<= BIT_DStream_completed - -When it's done, verify decompression is fully completed, by checking both DStream and the relevant states. -Checking if DStream has reached its end is performed by : - BIT_endOfDStream(&DStream); -Check also the states. There might be some symbols left there, if some high probability ones (>50%) are possible. - FSE_endOfDState(&DState); -*/ - - -/* ***************************************** -* FSE unsafe API -*******************************************/ -static unsigned char FSE_decodeSymbolFast(FSE_DState_t* DStatePtr, BIT_DStream_t* bitD); -/* faster, but works only if nbBits is always >= 1 (otherwise, result will be corrupted) */ - - -/* ***************************************** -* Implementation of inlined functions -*******************************************/ -typedef struct { - int deltaFindState; - U32 deltaNbBits; -} FSE_symbolCompressionTransform; /* total 8 bytes */ - -MEM_STATIC void FSE_initCState(FSE_CState_t* statePtr, const FSE_CTable* ct) -{ - const void* ptr = ct; - const U16* u16ptr = (const U16*) ptr; - const U32 tableLog = MEM_read16(ptr); - statePtr->value = (ptrdiff_t)1<stateTable = u16ptr+2; - statePtr->symbolTT = ct + 1 + (tableLog ? (1<<(tableLog-1)) : 1); - statePtr->stateLog = tableLog; -} - - -/*! FSE_initCState2() : -* Same as FSE_initCState(), but the first symbol to include (which will be the last to be read) -* uses the smallest state value possible, saving the cost of this symbol */ -MEM_STATIC void FSE_initCState2(FSE_CState_t* statePtr, const FSE_CTable* ct, U32 symbol) -{ - FSE_initCState(statePtr, ct); - { const FSE_symbolCompressionTransform symbolTT = ((const FSE_symbolCompressionTransform*)(statePtr->symbolTT))[symbol]; - const U16* stateTable = (const U16*)(statePtr->stateTable); - U32 nbBitsOut = (U32)((symbolTT.deltaNbBits + (1<<15)) >> 16); - statePtr->value = (nbBitsOut << 16) - symbolTT.deltaNbBits; - statePtr->value = stateTable[(statePtr->value >> nbBitsOut) + symbolTT.deltaFindState]; - } -} - -MEM_STATIC void FSE_encodeSymbol(BIT_CStream_t* bitC, FSE_CState_t* statePtr, unsigned symbol) -{ - FSE_symbolCompressionTransform const symbolTT = ((const FSE_symbolCompressionTransform*)(statePtr->symbolTT))[symbol]; - const U16* const stateTable = (const U16*)(statePtr->stateTable); - U32 const nbBitsOut = (U32)((statePtr->value + symbolTT.deltaNbBits) >> 16); - BIT_addBits(bitC, statePtr->value, nbBitsOut); - statePtr->value = stateTable[ (statePtr->value >> nbBitsOut) + symbolTT.deltaFindState]; -} - -MEM_STATIC void FSE_flushCState(BIT_CStream_t* bitC, const FSE_CState_t* statePtr) -{ - BIT_addBits(bitC, statePtr->value, statePtr->stateLog); - BIT_flushBits(bitC); -} - - -/* FSE_getMaxNbBits() : - * Approximate maximum cost of a symbol, in bits. - * Fractional get rounded up (i.e : a symbol with a normalized frequency of 3 gives the same result as a frequency of 2) - * note 1 : assume symbolValue is valid (<= maxSymbolValue) - * note 2 : if freq[symbolValue]==0, @return a fake cost of tableLog+1 bits */ -MEM_STATIC U32 FSE_getMaxNbBits(const void* symbolTTPtr, U32 symbolValue) -{ - const FSE_symbolCompressionTransform* symbolTT = (const FSE_symbolCompressionTransform*) symbolTTPtr; - return (symbolTT[symbolValue].deltaNbBits + ((1<<16)-1)) >> 16; -} - -/* FSE_bitCost() : - * Approximate symbol cost, as fractional value, using fixed-point format (accuracyLog fractional bits) - * note 1 : assume symbolValue is valid (<= maxSymbolValue) - * note 2 : if freq[symbolValue]==0, @return a fake cost of tableLog+1 bits */ -MEM_STATIC U32 FSE_bitCost(const void* symbolTTPtr, U32 tableLog, U32 symbolValue, U32 accuracyLog) -{ - const FSE_symbolCompressionTransform* symbolTT = (const FSE_symbolCompressionTransform*) symbolTTPtr; - U32 const minNbBits = symbolTT[symbolValue].deltaNbBits >> 16; - U32 const threshold = (minNbBits+1) << 16; - assert(tableLog < 16); - assert(accuracyLog < 31-tableLog); /* ensure enough room for renormalization double shift */ - { U32 const tableSize = 1 << tableLog; - U32 const deltaFromThreshold = threshold - (symbolTT[symbolValue].deltaNbBits + tableSize); - U32 const normalizedDeltaFromThreshold = (deltaFromThreshold << accuracyLog) >> tableLog; /* linear interpolation (very approximate) */ - U32 const bitMultiplier = 1 << accuracyLog; - assert(symbolTT[symbolValue].deltaNbBits + tableSize <= threshold); - assert(normalizedDeltaFromThreshold <= bitMultiplier); - return (minNbBits+1)*bitMultiplier - normalizedDeltaFromThreshold; - } -} - - -/* ====== Decompression ====== */ - -typedef struct { - U16 tableLog; - U16 fastMode; -} FSE_DTableHeader; /* sizeof U32 */ - -typedef struct -{ - unsigned short newState; - unsigned char symbol; - unsigned char nbBits; -} FSE_decode_t; /* size == U32 */ - -MEM_STATIC void FSE_initDState(FSE_DState_t* DStatePtr, BIT_DStream_t* bitD, const FSE_DTable* dt) -{ - const void* ptr = dt; - const FSE_DTableHeader* const DTableH = (const FSE_DTableHeader*)ptr; - DStatePtr->state = BIT_readBits(bitD, DTableH->tableLog); - BIT_reloadDStream(bitD); - DStatePtr->table = dt + 1; -} - -MEM_STATIC BYTE FSE_peekSymbol(const FSE_DState_t* DStatePtr) -{ - FSE_decode_t const DInfo = ((const FSE_decode_t*)(DStatePtr->table))[DStatePtr->state]; - return DInfo.symbol; -} - -MEM_STATIC void FSE_updateState(FSE_DState_t* DStatePtr, BIT_DStream_t* bitD) -{ - FSE_decode_t const DInfo = ((const FSE_decode_t*)(DStatePtr->table))[DStatePtr->state]; - U32 const nbBits = DInfo.nbBits; - size_t const lowBits = BIT_readBits(bitD, nbBits); - DStatePtr->state = DInfo.newState + lowBits; -} - -MEM_STATIC BYTE FSE_decodeSymbol(FSE_DState_t* DStatePtr, BIT_DStream_t* bitD) -{ - FSE_decode_t const DInfo = ((const FSE_decode_t*)(DStatePtr->table))[DStatePtr->state]; - U32 const nbBits = DInfo.nbBits; - BYTE const symbol = DInfo.symbol; - size_t const lowBits = BIT_readBits(bitD, nbBits); - - DStatePtr->state = DInfo.newState + lowBits; - return symbol; -} - -/*! FSE_decodeSymbolFast() : - unsafe, only works if no symbol has a probability > 50% */ -MEM_STATIC BYTE FSE_decodeSymbolFast(FSE_DState_t* DStatePtr, BIT_DStream_t* bitD) -{ - FSE_decode_t const DInfo = ((const FSE_decode_t*)(DStatePtr->table))[DStatePtr->state]; - U32 const nbBits = DInfo.nbBits; - BYTE const symbol = DInfo.symbol; - size_t const lowBits = BIT_readBitsFast(bitD, nbBits); - - DStatePtr->state = DInfo.newState + lowBits; - return symbol; -} - -MEM_STATIC unsigned FSE_endOfDState(const FSE_DState_t* DStatePtr) -{ - return DStatePtr->state == 0; -} - - - -#ifndef FSE_COMMONDEFS_ONLY - -/* ************************************************************** -* Tuning parameters -****************************************************************/ -/*!MEMORY_USAGE : -* Memory usage formula : N->2^N Bytes (examples : 10 -> 1KB; 12 -> 4KB ; 16 -> 64KB; 20 -> 1MB; etc.) -* Increasing memory usage improves compression ratio -* Reduced memory usage can improve speed, due to cache effect -* Recommended max value is 14, for 16KB, which nicely fits into Intel x86 L1 cache */ -#ifndef FSE_MAX_MEMORY_USAGE -# define FSE_MAX_MEMORY_USAGE 14 -#endif -#ifndef FSE_DEFAULT_MEMORY_USAGE -# define FSE_DEFAULT_MEMORY_USAGE 13 -#endif - -/*!FSE_MAX_SYMBOL_VALUE : -* Maximum symbol value authorized. -* Required for proper stack allocation */ -#ifndef FSE_MAX_SYMBOL_VALUE -# define FSE_MAX_SYMBOL_VALUE 255 -#endif - -/* ************************************************************** -* template functions type & suffix -****************************************************************/ -#define FSE_FUNCTION_TYPE BYTE -#define FSE_FUNCTION_EXTENSION -#define FSE_DECODE_TYPE FSE_decode_t - - -#endif /* !FSE_COMMONDEFS_ONLY */ - - -/* *************************************************************** -* Constants -*****************************************************************/ -#define FSE_MAX_TABLELOG (FSE_MAX_MEMORY_USAGE-2) -#define FSE_MAX_TABLESIZE (1U< FSE_TABLELOG_ABSOLUTE_MAX -# error "FSE_MAX_TABLELOG > FSE_TABLELOG_ABSOLUTE_MAX is not supported" -#endif - -#define FSE_TABLESTEP(tableSize) ((tableSize>>1) + (tableSize>>3) + 3) - - -#endif /* FSE_STATIC_LINKING_ONLY */ - - -#if defined (__cplusplus) -} -#endif -/**** ended inlining fse.h ****/ -#define HUF_STATIC_LINKING_ONLY /* HUF_TABLELOG_ABSOLUTEMAX */ -/**** start inlining huf.h ****/ -/* ****************************************************************** - * huff0 huffman codec, - * part of Finite State Entropy library - * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. - * - * You can contact the author at : - * - Source repository : https://github.com/Cyan4973/FiniteStateEntropy - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. -****************************************************************** */ - -#if defined (__cplusplus) -extern "C" { -#endif - -#ifndef HUF_H_298734234 -#define HUF_H_298734234 - -/* *** Dependencies *** */ -#include /* size_t */ - - -/* *** library symbols visibility *** */ -/* Note : when linking with -fvisibility=hidden on gcc, or by default on Visual, - * HUF symbols remain "private" (internal symbols for library only). - * Set macro FSE_DLL_EXPORT to 1 if you want HUF symbols visible on DLL interface */ -#if defined(FSE_DLL_EXPORT) && (FSE_DLL_EXPORT==1) && defined(__GNUC__) && (__GNUC__ >= 4) -# define HUF_PUBLIC_API __attribute__ ((visibility ("default"))) -#elif defined(FSE_DLL_EXPORT) && (FSE_DLL_EXPORT==1) /* Visual expected */ -# define HUF_PUBLIC_API __declspec(dllexport) -#elif defined(FSE_DLL_IMPORT) && (FSE_DLL_IMPORT==1) -# define HUF_PUBLIC_API __declspec(dllimport) /* not required, just to generate faster code (saves a function pointer load from IAT and an indirect jump) */ -#else -# define HUF_PUBLIC_API -#endif - - -/* ========================== */ -/* *** simple functions *** */ -/* ========================== */ - -/** HUF_compress() : - * Compress content from buffer 'src', of size 'srcSize', into buffer 'dst'. - * 'dst' buffer must be already allocated. - * Compression runs faster if `dstCapacity` >= HUF_compressBound(srcSize). - * `srcSize` must be <= `HUF_BLOCKSIZE_MAX` == 128 KB. - * @return : size of compressed data (<= `dstCapacity`). - * Special values : if return == 0, srcData is not compressible => Nothing is stored within dst !!! - * if HUF_isError(return), compression failed (more details using HUF_getErrorName()) - */ -HUF_PUBLIC_API size_t HUF_compress(void* dst, size_t dstCapacity, - const void* src, size_t srcSize); - -/** HUF_decompress() : - * Decompress HUF data from buffer 'cSrc', of size 'cSrcSize', - * into already allocated buffer 'dst', of minimum size 'dstSize'. - * `originalSize` : **must** be the ***exact*** size of original (uncompressed) data. - * Note : in contrast with FSE, HUF_decompress can regenerate - * RLE (cSrcSize==1) and uncompressed (cSrcSize==dstSize) data, - * because it knows size to regenerate (originalSize). - * @return : size of regenerated data (== originalSize), - * or an error code, which can be tested using HUF_isError() - */ -HUF_PUBLIC_API size_t HUF_decompress(void* dst, size_t originalSize, - const void* cSrc, size_t cSrcSize); - - -/* *** Tool functions *** */ -#define HUF_BLOCKSIZE_MAX (128 * 1024) /**< maximum input size for a single block compressed with HUF_compress */ -HUF_PUBLIC_API size_t HUF_compressBound(size_t size); /**< maximum compressed size (worst case) */ - -/* Error Management */ -HUF_PUBLIC_API unsigned HUF_isError(size_t code); /**< tells if a return value is an error code */ -HUF_PUBLIC_API const char* HUF_getErrorName(size_t code); /**< provides error code string (useful for debugging) */ - - -/* *** Advanced function *** */ - -/** HUF_compress2() : - * Same as HUF_compress(), but offers control over `maxSymbolValue` and `tableLog`. - * `maxSymbolValue` must be <= HUF_SYMBOLVALUE_MAX . - * `tableLog` must be `<= HUF_TABLELOG_MAX` . */ -HUF_PUBLIC_API size_t HUF_compress2 (void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - unsigned maxSymbolValue, unsigned tableLog); - -/** HUF_compress4X_wksp() : - * Same as HUF_compress2(), but uses externally allocated `workSpace`. - * `workspace` must have minimum alignment of 4, and be at least as large as HUF_WORKSPACE_SIZE */ -#define HUF_WORKSPACE_SIZE ((6 << 10) + 256) -#define HUF_WORKSPACE_SIZE_U32 (HUF_WORKSPACE_SIZE / sizeof(U32)) -HUF_PUBLIC_API size_t HUF_compress4X_wksp (void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - unsigned maxSymbolValue, unsigned tableLog, - void* workSpace, size_t wkspSize); - -#endif /* HUF_H_298734234 */ - -/* ****************************************************************** - * WARNING !! - * The following section contains advanced and experimental definitions - * which shall never be used in the context of a dynamic library, - * because they are not guaranteed to remain stable in the future. - * Only consider them in association with static linking. - * *****************************************************************/ -#if defined(HUF_STATIC_LINKING_ONLY) && !defined(HUF_H_HUF_STATIC_LINKING_ONLY) -#define HUF_H_HUF_STATIC_LINKING_ONLY - -/* *** Dependencies *** */ -/**** skipping file: mem.h ****/ - - -/* *** Constants *** */ -#define HUF_TABLELOG_MAX 12 /* max runtime value of tableLog (due to static allocation); can be modified up to HUF_ABSOLUTEMAX_TABLELOG */ -#define HUF_TABLELOG_DEFAULT 11 /* default tableLog value when none specified */ -#define HUF_SYMBOLVALUE_MAX 255 - -#define HUF_TABLELOG_ABSOLUTEMAX 15 /* absolute limit of HUF_MAX_TABLELOG. Beyond that value, code does not work */ -#if (HUF_TABLELOG_MAX > HUF_TABLELOG_ABSOLUTEMAX) -# error "HUF_TABLELOG_MAX is too large !" -#endif - - -/* **************************************** -* Static allocation -******************************************/ -/* HUF buffer bounds */ -#define HUF_CTABLEBOUND 129 -#define HUF_BLOCKBOUND(size) (size + (size>>8) + 8) /* only true when incompressible is pre-filtered with fast heuristic */ -#define HUF_COMPRESSBOUND(size) (HUF_CTABLEBOUND + HUF_BLOCKBOUND(size)) /* Macro version, useful for static allocation */ - -/* static allocation of HUF's Compression Table */ -#define HUF_CTABLE_SIZE_U32(maxSymbolValue) ((maxSymbolValue)+1) /* Use tables of U32, for proper alignment */ -#define HUF_CTABLE_SIZE(maxSymbolValue) (HUF_CTABLE_SIZE_U32(maxSymbolValue) * sizeof(U32)) -#define HUF_CREATE_STATIC_CTABLE(name, maxSymbolValue) \ - U32 name##hb[HUF_CTABLE_SIZE_U32(maxSymbolValue)]; \ - void* name##hv = &(name##hb); \ - HUF_CElt* name = (HUF_CElt*)(name##hv) /* no final ; */ - -/* static allocation of HUF's DTable */ -typedef U32 HUF_DTable; -#define HUF_DTABLE_SIZE(maxTableLog) (1 + (1<<(maxTableLog))) -#define HUF_CREATE_STATIC_DTABLEX1(DTable, maxTableLog) \ - HUF_DTable DTable[HUF_DTABLE_SIZE((maxTableLog)-1)] = { ((U32)((maxTableLog)-1) * 0x01000001) } -#define HUF_CREATE_STATIC_DTABLEX2(DTable, maxTableLog) \ - HUF_DTable DTable[HUF_DTABLE_SIZE(maxTableLog)] = { ((U32)(maxTableLog) * 0x01000001) } - - -/* **************************************** -* Advanced decompression functions -******************************************/ -size_t HUF_decompress4X1 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< single-symbol decoder */ -#ifndef HUF_FORCE_DECOMPRESS_X1 -size_t HUF_decompress4X2 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< double-symbols decoder */ -#endif - -size_t HUF_decompress4X_DCtx (HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< decodes RLE and uncompressed */ -size_t HUF_decompress4X_hufOnly(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< considers RLE and uncompressed as errors */ -size_t HUF_decompress4X_hufOnly_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize); /**< considers RLE and uncompressed as errors */ -size_t HUF_decompress4X1_DCtx(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< single-symbol decoder */ -size_t HUF_decompress4X1_DCtx_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize); /**< single-symbol decoder */ -#ifndef HUF_FORCE_DECOMPRESS_X1 -size_t HUF_decompress4X2_DCtx(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< double-symbols decoder */ -size_t HUF_decompress4X2_DCtx_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize); /**< double-symbols decoder */ -#endif - - -/* **************************************** - * HUF detailed API - * ****************************************/ - -/*! HUF_compress() does the following: - * 1. count symbol occurrence from source[] into table count[] using FSE_count() (exposed within "fse.h") - * 2. (optional) refine tableLog using HUF_optimalTableLog() - * 3. build Huffman table from count using HUF_buildCTable() - * 4. save Huffman table to memory buffer using HUF_writeCTable() - * 5. encode the data stream using HUF_compress4X_usingCTable() - * - * The following API allows targeting specific sub-functions for advanced tasks. - * For example, it's possible to compress several blocks using the same 'CTable', - * or to save and regenerate 'CTable' using external methods. - */ -unsigned HUF_optimalTableLog(unsigned maxTableLog, size_t srcSize, unsigned maxSymbolValue); -typedef struct HUF_CElt_s HUF_CElt; /* incomplete type */ -size_t HUF_buildCTable (HUF_CElt* CTable, const unsigned* count, unsigned maxSymbolValue, unsigned maxNbBits); /* @return : maxNbBits; CTable and count can overlap. In which case, CTable will overwrite count content */ -size_t HUF_writeCTable (void* dst, size_t maxDstSize, const HUF_CElt* CTable, unsigned maxSymbolValue, unsigned huffLog); -size_t HUF_compress4X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable); -size_t HUF_estimateCompressedSize(const HUF_CElt* CTable, const unsigned* count, unsigned maxSymbolValue); -int HUF_validateCTable(const HUF_CElt* CTable, const unsigned* count, unsigned maxSymbolValue); - -typedef enum { - HUF_repeat_none, /**< Cannot use the previous table */ - HUF_repeat_check, /**< Can use the previous table but it must be checked. Note : The previous table must have been constructed by HUF_compress{1, 4}X_repeat */ - HUF_repeat_valid /**< Can use the previous table and it is assumed to be valid */ - } HUF_repeat; -/** HUF_compress4X_repeat() : - * Same as HUF_compress4X_wksp(), but considers using hufTable if *repeat != HUF_repeat_none. - * If it uses hufTable it does not modify hufTable or repeat. - * If it doesn't, it sets *repeat = HUF_repeat_none, and it sets hufTable to the table used. - * If preferRepeat then the old table will always be used if valid. */ -size_t HUF_compress4X_repeat(void* dst, size_t dstSize, - const void* src, size_t srcSize, - unsigned maxSymbolValue, unsigned tableLog, - void* workSpace, size_t wkspSize, /**< `workSpace` must be aligned on 4-bytes boundaries, `wkspSize` must be >= HUF_WORKSPACE_SIZE */ - HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat, int bmi2); - -/** HUF_buildCTable_wksp() : - * Same as HUF_buildCTable(), but using externally allocated scratch buffer. - * `workSpace` must be aligned on 4-bytes boundaries, and its size must be >= HUF_CTABLE_WORKSPACE_SIZE. - */ -#define HUF_CTABLE_WORKSPACE_SIZE_U32 (2*HUF_SYMBOLVALUE_MAX +1 +1) -#define HUF_CTABLE_WORKSPACE_SIZE (HUF_CTABLE_WORKSPACE_SIZE_U32 * sizeof(unsigned)) -size_t HUF_buildCTable_wksp (HUF_CElt* tree, - const unsigned* count, U32 maxSymbolValue, U32 maxNbBits, - void* workSpace, size_t wkspSize); - -/*! HUF_readStats() : - * Read compact Huffman tree, saved by HUF_writeCTable(). - * `huffWeight` is destination buffer. - * @return : size read from `src` , or an error Code . - * Note : Needed by HUF_readCTable() and HUF_readDTableXn() . */ -size_t HUF_readStats(BYTE* huffWeight, size_t hwSize, - U32* rankStats, U32* nbSymbolsPtr, U32* tableLogPtr, - const void* src, size_t srcSize); - -/** HUF_readCTable() : - * Loading a CTable saved with HUF_writeCTable() */ -size_t HUF_readCTable (HUF_CElt* CTable, unsigned* maxSymbolValuePtr, const void* src, size_t srcSize, unsigned *hasZeroWeights); - -/** HUF_getNbBits() : - * Read nbBits from CTable symbolTable, for symbol `symbolValue` presumed <= HUF_SYMBOLVALUE_MAX - * Note 1 : is not inlined, as HUF_CElt definition is private - * Note 2 : const void* used, so that it can provide a statically allocated table as argument (which uses type U32) */ -U32 HUF_getNbBits(const void* symbolTable, U32 symbolValue); - -/* - * HUF_decompress() does the following: - * 1. select the decompression algorithm (X1, X2) based on pre-computed heuristics - * 2. build Huffman table from save, using HUF_readDTableX?() - * 3. decode 1 or 4 segments in parallel using HUF_decompress?X?_usingDTable() - */ - -/** HUF_selectDecoder() : - * Tells which decoder is likely to decode faster, - * based on a set of pre-computed metrics. - * @return : 0==HUF_decompress4X1, 1==HUF_decompress4X2 . - * Assumption : 0 < dstSize <= 128 KB */ -U32 HUF_selectDecoder (size_t dstSize, size_t cSrcSize); - -/** - * The minimum workspace size for the `workSpace` used in - * HUF_readDTableX1_wksp() and HUF_readDTableX2_wksp(). - * - * The space used depends on HUF_TABLELOG_MAX, ranging from ~1500 bytes when - * HUF_TABLE_LOG_MAX=12 to ~1850 bytes when HUF_TABLE_LOG_MAX=15. - * Buffer overflow errors may potentially occur if code modifications result in - * a required workspace size greater than that specified in the following - * macro. - */ -#define HUF_DECOMPRESS_WORKSPACE_SIZE (2 << 10) -#define HUF_DECOMPRESS_WORKSPACE_SIZE_U32 (HUF_DECOMPRESS_WORKSPACE_SIZE / sizeof(U32)) - -#ifndef HUF_FORCE_DECOMPRESS_X2 -size_t HUF_readDTableX1 (HUF_DTable* DTable, const void* src, size_t srcSize); -size_t HUF_readDTableX1_wksp (HUF_DTable* DTable, const void* src, size_t srcSize, void* workSpace, size_t wkspSize); -#endif -#ifndef HUF_FORCE_DECOMPRESS_X1 -size_t HUF_readDTableX2 (HUF_DTable* DTable, const void* src, size_t srcSize); -size_t HUF_readDTableX2_wksp (HUF_DTable* DTable, const void* src, size_t srcSize, void* workSpace, size_t wkspSize); -#endif - -size_t HUF_decompress4X_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable); -#ifndef HUF_FORCE_DECOMPRESS_X2 -size_t HUF_decompress4X1_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable); -#endif -#ifndef HUF_FORCE_DECOMPRESS_X1 -size_t HUF_decompress4X2_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable); -#endif - - -/* ====================== */ -/* single stream variants */ -/* ====================== */ - -size_t HUF_compress1X (void* dst, size_t dstSize, const void* src, size_t srcSize, unsigned maxSymbolValue, unsigned tableLog); -size_t HUF_compress1X_wksp (void* dst, size_t dstSize, const void* src, size_t srcSize, unsigned maxSymbolValue, unsigned tableLog, void* workSpace, size_t wkspSize); /**< `workSpace` must be a table of at least HUF_WORKSPACE_SIZE_U32 unsigned */ -size_t HUF_compress1X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable); -/** HUF_compress1X_repeat() : - * Same as HUF_compress1X_wksp(), but considers using hufTable if *repeat != HUF_repeat_none. - * If it uses hufTable it does not modify hufTable or repeat. - * If it doesn't, it sets *repeat = HUF_repeat_none, and it sets hufTable to the table used. - * If preferRepeat then the old table will always be used if valid. */ -size_t HUF_compress1X_repeat(void* dst, size_t dstSize, - const void* src, size_t srcSize, - unsigned maxSymbolValue, unsigned tableLog, - void* workSpace, size_t wkspSize, /**< `workSpace` must be aligned on 4-bytes boundaries, `wkspSize` must be >= HUF_WORKSPACE_SIZE */ - HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat, int bmi2); - -size_t HUF_decompress1X1 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* single-symbol decoder */ -#ifndef HUF_FORCE_DECOMPRESS_X1 -size_t HUF_decompress1X2 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* double-symbol decoder */ -#endif - -size_t HUF_decompress1X_DCtx (HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); -size_t HUF_decompress1X_DCtx_wksp (HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize); -#ifndef HUF_FORCE_DECOMPRESS_X2 -size_t HUF_decompress1X1_DCtx(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< single-symbol decoder */ -size_t HUF_decompress1X1_DCtx_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize); /**< single-symbol decoder */ -#endif -#ifndef HUF_FORCE_DECOMPRESS_X1 -size_t HUF_decompress1X2_DCtx(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /**< double-symbols decoder */ -size_t HUF_decompress1X2_DCtx_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize); /**< double-symbols decoder */ -#endif - -size_t HUF_decompress1X_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable); /**< automatic selection of sing or double symbol decoder, based on DTable */ -#ifndef HUF_FORCE_DECOMPRESS_X2 -size_t HUF_decompress1X1_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable); -#endif -#ifndef HUF_FORCE_DECOMPRESS_X1 -size_t HUF_decompress1X2_usingDTable(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable); -#endif - -/* BMI2 variants. - * If the CPU has BMI2 support, pass bmi2=1, otherwise pass bmi2=0. - */ -size_t HUF_decompress1X_usingDTable_bmi2(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable, int bmi2); -#ifndef HUF_FORCE_DECOMPRESS_X2 -size_t HUF_decompress1X1_DCtx_wksp_bmi2(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize, int bmi2); -#endif -size_t HUF_decompress4X_usingDTable_bmi2(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable, int bmi2); -size_t HUF_decompress4X_hufOnly_wksp_bmi2(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize, int bmi2); - -#endif /* HUF_STATIC_LINKING_ONLY */ - -#if defined (__cplusplus) -} -#endif -/**** ended inlining huf.h ****/ - - -/*=== Version ===*/ -unsigned FSE_versionNumber(void) { return FSE_VERSION_NUMBER; } - - -/*=== Error Management ===*/ -unsigned FSE_isError(size_t code) { return ERR_isError(code); } -const char* FSE_getErrorName(size_t code) { return ERR_getErrorName(code); } - -unsigned HUF_isError(size_t code) { return ERR_isError(code); } -const char* HUF_getErrorName(size_t code) { return ERR_getErrorName(code); } - - -/*-************************************************************** -* FSE NCount encoding-decoding -****************************************************************/ -size_t FSE_readNCount (short* normalizedCounter, unsigned* maxSVPtr, unsigned* tableLogPtr, - const void* headerBuffer, size_t hbSize) -{ - const BYTE* const istart = (const BYTE*) headerBuffer; - const BYTE* const iend = istart + hbSize; - const BYTE* ip = istart; - int nbBits; - int remaining; - int threshold; - U32 bitStream; - int bitCount; - unsigned charnum = 0; - int previous0 = 0; - - if (hbSize < 4) { - /* This function only works when hbSize >= 4 */ - char buffer[4]; - memset(buffer, 0, sizeof(buffer)); - memcpy(buffer, headerBuffer, hbSize); - { size_t const countSize = FSE_readNCount(normalizedCounter, maxSVPtr, tableLogPtr, - buffer, sizeof(buffer)); - if (FSE_isError(countSize)) return countSize; - if (countSize > hbSize) return ERROR(corruption_detected); - return countSize; - } } - assert(hbSize >= 4); - - /* init */ - memset(normalizedCounter, 0, (*maxSVPtr+1) * sizeof(normalizedCounter[0])); /* all symbols not present in NCount have a frequency of 0 */ - bitStream = MEM_readLE32(ip); - nbBits = (bitStream & 0xF) + FSE_MIN_TABLELOG; /* extract tableLog */ - if (nbBits > FSE_TABLELOG_ABSOLUTE_MAX) return ERROR(tableLog_tooLarge); - bitStream >>= 4; - bitCount = 4; - *tableLogPtr = nbBits; - remaining = (1<1) & (charnum<=*maxSVPtr)) { - if (previous0) { - unsigned n0 = charnum; - while ((bitStream & 0xFFFF) == 0xFFFF) { - n0 += 24; - if (ip < iend-5) { - ip += 2; - bitStream = MEM_readLE32(ip) >> bitCount; - } else { - bitStream >>= 16; - bitCount += 16; - } } - while ((bitStream & 3) == 3) { - n0 += 3; - bitStream >>= 2; - bitCount += 2; - } - n0 += bitStream & 3; - bitCount += 2; - if (n0 > *maxSVPtr) return ERROR(maxSymbolValue_tooSmall); - while (charnum < n0) normalizedCounter[charnum++] = 0; - if ((ip <= iend-7) || (ip + (bitCount>>3) <= iend-4)) { - assert((bitCount >> 3) <= 3); /* For first condition to work */ - ip += bitCount>>3; - bitCount &= 7; - bitStream = MEM_readLE32(ip) >> bitCount; - } else { - bitStream >>= 2; - } } - { int const max = (2*threshold-1) - remaining; - int count; - - if ((bitStream & (threshold-1)) < (U32)max) { - count = bitStream & (threshold-1); - bitCount += nbBits-1; - } else { - count = bitStream & (2*threshold-1); - if (count >= threshold) count -= max; - bitCount += nbBits; - } - - count--; /* extra accuracy */ - remaining -= count < 0 ? -count : count; /* -1 means +1 */ - normalizedCounter[charnum++] = (short)count; - previous0 = !count; - while (remaining < threshold) { - nbBits--; - threshold >>= 1; - } - - if ((ip <= iend-7) || (ip + (bitCount>>3) <= iend-4)) { - ip += bitCount>>3; - bitCount &= 7; - } else { - bitCount -= (int)(8 * (iend - 4 - ip)); - ip = iend - 4; - } - bitStream = MEM_readLE32(ip) >> (bitCount & 31); - } } /* while ((remaining>1) & (charnum<=*maxSVPtr)) */ - if (remaining != 1) return ERROR(corruption_detected); - if (bitCount > 32) return ERROR(corruption_detected); - *maxSVPtr = charnum-1; - - ip += (bitCount+7)>>3; - return ip-istart; -} - - -/*! HUF_readStats() : - Read compact Huffman tree, saved by HUF_writeCTable(). - `huffWeight` is destination buffer. - `rankStats` is assumed to be a table of at least HUF_TABLELOG_MAX U32. - @return : size read from `src` , or an error Code . - Note : Needed by HUF_readCTable() and HUF_readDTableX?() . -*/ -size_t HUF_readStats(BYTE* huffWeight, size_t hwSize, U32* rankStats, - U32* nbSymbolsPtr, U32* tableLogPtr, - const void* src, size_t srcSize) -{ - U32 weightTotal; - const BYTE* ip = (const BYTE*) src; - size_t iSize; - size_t oSize; - - if (!srcSize) return ERROR(srcSize_wrong); - iSize = ip[0]; - /* memset(huffWeight, 0, hwSize); *//* is not necessary, even though some analyzer complain ... */ - - if (iSize >= 128) { /* special header */ - oSize = iSize - 127; - iSize = ((oSize+1)/2); - if (iSize+1 > srcSize) return ERROR(srcSize_wrong); - if (oSize >= hwSize) return ERROR(corruption_detected); - ip += 1; - { U32 n; - for (n=0; n> 4; - huffWeight[n+1] = ip[n/2] & 15; - } } } - else { /* header compressed with FSE (normal case) */ - FSE_DTable fseWorkspace[FSE_DTABLE_SIZE_U32(6)]; /* 6 is max possible tableLog for HUF header (maybe even 5, to be tested) */ - if (iSize+1 > srcSize) return ERROR(srcSize_wrong); - oSize = FSE_decompress_wksp(huffWeight, hwSize-1, ip+1, iSize, fseWorkspace, 6); /* max (hwSize-1) values decoded, as last one is implied */ - if (FSE_isError(oSize)) return oSize; - } - - /* collect weight stats */ - memset(rankStats, 0, (HUF_TABLELOG_MAX + 1) * sizeof(U32)); - weightTotal = 0; - { U32 n; for (n=0; n= HUF_TABLELOG_MAX) return ERROR(corruption_detected); - rankStats[huffWeight[n]]++; - weightTotal += (1 << huffWeight[n]) >> 1; - } } - if (weightTotal == 0) return ERROR(corruption_detected); - - /* get last non-null symbol weight (implied, total must be 2^n) */ - { U32 const tableLog = BIT_highbit32(weightTotal) + 1; - if (tableLog > HUF_TABLELOG_MAX) return ERROR(corruption_detected); - *tableLogPtr = tableLog; - /* determine last weight */ - { U32 const total = 1 << tableLog; - U32 const rest = total - weightTotal; - U32 const verif = 1 << BIT_highbit32(rest); - U32 const lastWeight = BIT_highbit32(rest) + 1; - if (verif != rest) return ERROR(corruption_detected); /* last value must be a clean power of 2 */ - huffWeight[oSize] = (BYTE)lastWeight; - rankStats[lastWeight]++; - } } - - /* check tree construction validity */ - if ((rankStats[1] < 2) || (rankStats[1] & 1)) return ERROR(corruption_detected); /* by construction : at least 2 elts of rank 1, must be even */ - - /* results */ - *nbSymbolsPtr = (U32)(oSize+1); - return iSize+1; -} -/**** ended inlining common/entropy_common.c ****/ -/**** start inlining common/error_private.c ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -/* The purpose of this file is to have a single list of error strings embedded in binary */ - -/**** skipping file: error_private.h ****/ - -const char* ERR_getErrorString(ERR_enum code) -{ -#ifdef ZSTD_STRIP_ERROR_STRINGS - (void)code; - return "Error strings stripped"; -#else - static const char* const notErrorCode = "Unspecified error code"; - switch( code ) - { - case PREFIX(no_error): return "No error detected"; - case PREFIX(GENERIC): return "Error (generic)"; - case PREFIX(prefix_unknown): return "Unknown frame descriptor"; - case PREFIX(version_unsupported): return "Version not supported"; - case PREFIX(frameParameter_unsupported): return "Unsupported frame parameter"; - case PREFIX(frameParameter_windowTooLarge): return "Frame requires too much memory for decoding"; - case PREFIX(corruption_detected): return "Corrupted block detected"; - case PREFIX(checksum_wrong): return "Restored data doesn't match checksum"; - case PREFIX(parameter_unsupported): return "Unsupported parameter"; - case PREFIX(parameter_outOfBound): return "Parameter is out of bound"; - case PREFIX(init_missing): return "Context should be init first"; - case PREFIX(memory_allocation): return "Allocation error : not enough memory"; - case PREFIX(workSpace_tooSmall): return "workSpace buffer is not large enough"; - case PREFIX(stage_wrong): return "Operation not authorized at current processing stage"; - case PREFIX(tableLog_tooLarge): return "tableLog requires too much memory : unsupported"; - case PREFIX(maxSymbolValue_tooLarge): return "Unsupported max Symbol Value : too large"; - case PREFIX(maxSymbolValue_tooSmall): return "Specified maxSymbolValue is too small"; - case PREFIX(dictionary_corrupted): return "Dictionary is corrupted"; - case PREFIX(dictionary_wrong): return "Dictionary mismatch"; - case PREFIX(dictionaryCreation_failed): return "Cannot create Dictionary from provided samples"; - case PREFIX(dstSize_tooSmall): return "Destination buffer is too small"; - case PREFIX(srcSize_wrong): return "Src size is incorrect"; - case PREFIX(dstBuffer_null): return "Operation on NULL destination buffer"; - /* following error codes are not stable and may be removed or changed in a future version */ - case PREFIX(frameIndex_tooLarge): return "Frame index is too large"; - case PREFIX(seekableIO): return "An I/O error occurred when reading/seeking"; - case PREFIX(dstBuffer_wrong): return "Destination buffer is wrong"; - case PREFIX(maxCode): - default: return notErrorCode; - } -#endif -} -/**** ended inlining common/error_private.c ****/ -/**** start inlining common/fse_decompress.c ****/ -/* ****************************************************************** - * FSE : Finite State Entropy decoder - * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. - * - * You can contact the author at : - * - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy - * - Public forum : https://groups.google.com/forum/#!forum/lz4c - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. -****************************************************************** */ - - -/* ************************************************************** -* Includes -****************************************************************/ -#include /* malloc, free, qsort */ -#include /* memcpy, memset */ -/**** skipping file: bitstream.h ****/ -/**** skipping file: compiler.h ****/ -#define FSE_STATIC_LINKING_ONLY -/**** skipping file: fse.h ****/ -/**** skipping file: error_private.h ****/ - - -/* ************************************************************** -* Error Management -****************************************************************/ -#define FSE_isError ERR_isError -#define FSE_STATIC_ASSERT(c) DEBUG_STATIC_ASSERT(c) /* use only *after* variable declarations */ - - -/* ************************************************************** -* Templates -****************************************************************/ -/* - designed to be included - for type-specific functions (template emulation in C) - Objective is to write these functions only once, for improved maintenance -*/ - -/* safety checks */ -#ifndef FSE_FUNCTION_EXTENSION -# error "FSE_FUNCTION_EXTENSION must be defined" -#endif -#ifndef FSE_FUNCTION_TYPE -# error "FSE_FUNCTION_TYPE must be defined" -#endif - -/* Function names */ -#define FSE_CAT(X,Y) X##Y -#define FSE_FUNCTION_NAME(X,Y) FSE_CAT(X,Y) -#define FSE_TYPE_NAME(X,Y) FSE_CAT(X,Y) - - -/* Function templates */ -FSE_DTable* FSE_createDTable (unsigned tableLog) -{ - if (tableLog > FSE_TABLELOG_ABSOLUTE_MAX) tableLog = FSE_TABLELOG_ABSOLUTE_MAX; - return (FSE_DTable*)malloc( FSE_DTABLE_SIZE_U32(tableLog) * sizeof (U32) ); -} - -void FSE_freeDTable (FSE_DTable* dt) -{ - free(dt); -} - -size_t FSE_buildDTable(FSE_DTable* dt, const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog) -{ - void* const tdPtr = dt+1; /* because *dt is unsigned, 32-bits aligned on 32-bits */ - FSE_DECODE_TYPE* const tableDecode = (FSE_DECODE_TYPE*) (tdPtr); - U16 symbolNext[FSE_MAX_SYMBOL_VALUE+1]; - - U32 const maxSV1 = maxSymbolValue + 1; - U32 const tableSize = 1 << tableLog; - U32 highThreshold = tableSize-1; - - /* Sanity Checks */ - if (maxSymbolValue > FSE_MAX_SYMBOL_VALUE) return ERROR(maxSymbolValue_tooLarge); - if (tableLog > FSE_MAX_TABLELOG) return ERROR(tableLog_tooLarge); - - /* Init, lay down lowprob symbols */ - { FSE_DTableHeader DTableH; - DTableH.tableLog = (U16)tableLog; - DTableH.fastMode = 1; - { S16 const largeLimit= (S16)(1 << (tableLog-1)); - U32 s; - for (s=0; s= largeLimit) DTableH.fastMode=0; - symbolNext[s] = normalizedCounter[s]; - } } } - memcpy(dt, &DTableH, sizeof(DTableH)); - } - - /* Spread symbols */ - { U32 const tableMask = tableSize-1; - U32 const step = FSE_TABLESTEP(tableSize); - U32 s, position = 0; - for (s=0; s highThreshold) position = (position + step) & tableMask; /* lowprob area */ - } } - if (position!=0) return ERROR(GENERIC); /* position must reach all cells once, otherwise normalizedCounter is incorrect */ - } - - /* Build Decoding table */ - { U32 u; - for (u=0; utableLog = 0; - DTableH->fastMode = 0; - - cell->newState = 0; - cell->symbol = symbolValue; - cell->nbBits = 0; - - return 0; -} - - -size_t FSE_buildDTable_raw (FSE_DTable* dt, unsigned nbBits) -{ - void* ptr = dt; - FSE_DTableHeader* const DTableH = (FSE_DTableHeader*)ptr; - void* dPtr = dt + 1; - FSE_decode_t* const dinfo = (FSE_decode_t*)dPtr; - const unsigned tableSize = 1 << nbBits; - const unsigned tableMask = tableSize - 1; - const unsigned maxSV1 = tableMask+1; - unsigned s; - - /* Sanity checks */ - if (nbBits < 1) return ERROR(GENERIC); /* min size */ - - /* Build Decoding Table */ - DTableH->tableLog = (U16)nbBits; - DTableH->fastMode = 1; - for (s=0; s sizeof(bitD.bitContainer)*8) /* This test must be static */ - BIT_reloadDStream(&bitD); - - op[1] = FSE_GETSYMBOL(&state2); - - if (FSE_MAX_TABLELOG*4+7 > sizeof(bitD.bitContainer)*8) /* This test must be static */ - { if (BIT_reloadDStream(&bitD) > BIT_DStream_unfinished) { op+=2; break; } } - - op[2] = FSE_GETSYMBOL(&state1); - - if (FSE_MAX_TABLELOG*2+7 > sizeof(bitD.bitContainer)*8) /* This test must be static */ - BIT_reloadDStream(&bitD); - - op[3] = FSE_GETSYMBOL(&state2); - } - - /* tail */ - /* note : BIT_reloadDStream(&bitD) >= FSE_DStream_partiallyFilled; Ends at exactly BIT_DStream_completed */ - while (1) { - if (op>(omax-2)) return ERROR(dstSize_tooSmall); - *op++ = FSE_GETSYMBOL(&state1); - if (BIT_reloadDStream(&bitD)==BIT_DStream_overflow) { - *op++ = FSE_GETSYMBOL(&state2); - break; - } - - if (op>(omax-2)) return ERROR(dstSize_tooSmall); - *op++ = FSE_GETSYMBOL(&state2); - if (BIT_reloadDStream(&bitD)==BIT_DStream_overflow) { - *op++ = FSE_GETSYMBOL(&state1); - break; - } } - - return op-ostart; -} - - -size_t FSE_decompress_usingDTable(void* dst, size_t originalSize, - const void* cSrc, size_t cSrcSize, - const FSE_DTable* dt) -{ - const void* ptr = dt; - const FSE_DTableHeader* DTableH = (const FSE_DTableHeader*)ptr; - const U32 fastMode = DTableH->fastMode; - - /* select fast mode (static) */ - if (fastMode) return FSE_decompress_usingDTable_generic(dst, originalSize, cSrc, cSrcSize, dt, 1); - return FSE_decompress_usingDTable_generic(dst, originalSize, cSrc, cSrcSize, dt, 0); -} - - -size_t FSE_decompress_wksp(void* dst, size_t dstCapacity, const void* cSrc, size_t cSrcSize, FSE_DTable* workSpace, unsigned maxLog) -{ - const BYTE* const istart = (const BYTE*)cSrc; - const BYTE* ip = istart; - short counting[FSE_MAX_SYMBOL_VALUE+1]; - unsigned tableLog; - unsigned maxSymbolValue = FSE_MAX_SYMBOL_VALUE; - - /* normal FSE decoding mode */ - size_t const NCountLength = FSE_readNCount (counting, &maxSymbolValue, &tableLog, istart, cSrcSize); - if (FSE_isError(NCountLength)) return NCountLength; - /* if (NCountLength >= cSrcSize) return ERROR(srcSize_wrong); */ /* too small input size; supposed to be already checked in NCountLength, only remaining case : NCountLength==cSrcSize */ - if (tableLog > maxLog) return ERROR(tableLog_tooLarge); - ip += NCountLength; - cSrcSize -= NCountLength; - - CHECK_F( FSE_buildDTable (workSpace, counting, maxSymbolValue, tableLog) ); - - return FSE_decompress_usingDTable (dst, dstCapacity, ip, cSrcSize, workSpace); /* always return, even if it is an error code */ -} - - -typedef FSE_DTable DTable_max_t[FSE_DTABLE_SIZE_U32(FSE_MAX_TABLELOG)]; - -size_t FSE_decompress(void* dst, size_t dstCapacity, const void* cSrc, size_t cSrcSize) -{ - DTable_max_t dt; /* Static analyzer seems unable to understand this table will be properly initialized later */ - return FSE_decompress_wksp(dst, dstCapacity, cSrc, cSrcSize, dt, FSE_MAX_TABLELOG); -} - - - -#endif /* FSE_COMMONDEFS_ONLY */ -/**** ended inlining common/fse_decompress.c ****/ -/**** start inlining common/pool.c ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - - -/* ====== Dependencies ======= */ -#include /* size_t */ -/**** skipping file: debug.h ****/ -/**** start inlining zstd_internal.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_CCOMMON_H_MODULE -#define ZSTD_CCOMMON_H_MODULE - -/* this module contains definitions which must be identical - * across compression, decompression and dictBuilder. - * It also contains a few functions useful to at least 2 of them - * and which benefit from being inlined */ - -/*-************************************* -* Dependencies -***************************************/ -#if !defined(ZSTD_NO_INTRINSICS) && defined(__ARM_NEON) -#include -#endif -/**** skipping file: compiler.h ****/ -/**** skipping file: mem.h ****/ -/**** skipping file: debug.h ****/ -/**** skipping file: error_private.h ****/ -#define ZSTD_STATIC_LINKING_ONLY -/**** start inlining ../zstd.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ -#if defined (__cplusplus) -extern "C" { -#endif - -#ifndef ZSTD_H_235446 -#define ZSTD_H_235446 - -/* ====== Dependency ======*/ -#include /* INT_MAX */ -#include /* size_t */ - - -/* ===== ZSTDLIB_API : control library symbols visibility ===== */ -#ifndef ZSTDLIB_VISIBILITY -# if defined(__GNUC__) && (__GNUC__ >= 4) -# define ZSTDLIB_VISIBILITY __attribute__ ((visibility ("default"))) -# else -# define ZSTDLIB_VISIBILITY -# endif -#endif -#if defined(ZSTD_DLL_EXPORT) && (ZSTD_DLL_EXPORT==1) -# define ZSTDLIB_API __declspec(dllexport) ZSTDLIB_VISIBILITY -#elif defined(ZSTD_DLL_IMPORT) && (ZSTD_DLL_IMPORT==1) -# define ZSTDLIB_API __declspec(dllimport) ZSTDLIB_VISIBILITY /* It isn't required but allows to generate better code, saving a function pointer load from the IAT and an indirect jump.*/ -#else -# define ZSTDLIB_API ZSTDLIB_VISIBILITY -#endif - - -/******************************************************************************* - Introduction - - zstd, short for Zstandard, is a fast lossless compression algorithm, targeting - real-time compression scenarios at zlib-level and better compression ratios. - The zstd compression library provides in-memory compression and decompression - functions. - - The library supports regular compression levels from 1 up to ZSTD_maxCLevel(), - which is currently 22. Levels >= 20, labeled `--ultra`, should be used with - caution, as they require more memory. The library also offers negative - compression levels, which extend the range of speed vs. ratio preferences. - The lower the level, the faster the speed (at the cost of compression). - - Compression can be done in: - - a single step (described as Simple API) - - a single step, reusing a context (described as Explicit context) - - unbounded multiple steps (described as Streaming compression) - - The compression ratio achievable on small data can be highly improved using - a dictionary. Dictionary compression can be performed in: - - a single step (described as Simple dictionary API) - - a single step, reusing a dictionary (described as Bulk-processing - dictionary API) - - Advanced experimental functions can be accessed using - `#define ZSTD_STATIC_LINKING_ONLY` before including zstd.h. - - Advanced experimental APIs should never be used with a dynamically-linked - library. They are not "stable"; their definitions or signatures may change in - the future. Only static linking is allowed. -*******************************************************************************/ - -/*------ Version ------*/ -#define ZSTD_VERSION_MAJOR 1 -#define ZSTD_VERSION_MINOR 4 -#define ZSTD_VERSION_RELEASE 5 - -#define ZSTD_VERSION_NUMBER (ZSTD_VERSION_MAJOR *100*100 + ZSTD_VERSION_MINOR *100 + ZSTD_VERSION_RELEASE) -ZSTDLIB_API unsigned ZSTD_versionNumber(void); /**< to check runtime library version */ - -#define ZSTD_LIB_VERSION ZSTD_VERSION_MAJOR.ZSTD_VERSION_MINOR.ZSTD_VERSION_RELEASE -#define ZSTD_QUOTE(str) #str -#define ZSTD_EXPAND_AND_QUOTE(str) ZSTD_QUOTE(str) -#define ZSTD_VERSION_STRING ZSTD_EXPAND_AND_QUOTE(ZSTD_LIB_VERSION) -ZSTDLIB_API const char* ZSTD_versionString(void); /* requires v1.3.0+ */ - -/* ************************************* - * Default constant - ***************************************/ -#ifndef ZSTD_CLEVEL_DEFAULT -# define ZSTD_CLEVEL_DEFAULT 3 -#endif - -/* ************************************* - * Constants - ***************************************/ - -/* All magic numbers are supposed read/written to/from files/memory using little-endian convention */ -#define ZSTD_MAGICNUMBER 0xFD2FB528 /* valid since v0.8.0 */ -#define ZSTD_MAGIC_DICTIONARY 0xEC30A437 /* valid since v0.7.0 */ -#define ZSTD_MAGIC_SKIPPABLE_START 0x184D2A50 /* all 16 values, from 0x184D2A50 to 0x184D2A5F, signal the beginning of a skippable frame */ -#define ZSTD_MAGIC_SKIPPABLE_MASK 0xFFFFFFF0 - -#define ZSTD_BLOCKSIZELOG_MAX 17 -#define ZSTD_BLOCKSIZE_MAX (1<= `ZSTD_compressBound(srcSize)`. - * @return : compressed size written into `dst` (<= `dstCapacity), - * or an error code if it fails (which can be tested using ZSTD_isError()). */ -ZSTDLIB_API size_t ZSTD_compress( void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - int compressionLevel); - -/*! ZSTD_decompress() : - * `compressedSize` : must be the _exact_ size of some number of compressed and/or skippable frames. - * `dstCapacity` is an upper bound of originalSize to regenerate. - * If user cannot imply a maximum upper bound, it's better to use streaming mode to decompress data. - * @return : the number of bytes decompressed into `dst` (<= `dstCapacity`), - * or an errorCode if it fails (which can be tested using ZSTD_isError()). */ -ZSTDLIB_API size_t ZSTD_decompress( void* dst, size_t dstCapacity, - const void* src, size_t compressedSize); - -/*! ZSTD_getFrameContentSize() : requires v1.3.0+ - * `src` should point to the start of a ZSTD encoded frame. - * `srcSize` must be at least as large as the frame header. - * hint : any size >= `ZSTD_frameHeaderSize_max` is large enough. - * @return : - decompressed size of `src` frame content, if known - * - ZSTD_CONTENTSIZE_UNKNOWN if the size cannot be determined - * - ZSTD_CONTENTSIZE_ERROR if an error occurred (e.g. invalid magic number, srcSize too small) - * note 1 : a 0 return value means the frame is valid but "empty". - * note 2 : decompressed size is an optional field, it may not be present, typically in streaming mode. - * When `return==ZSTD_CONTENTSIZE_UNKNOWN`, data to decompress could be any size. - * In which case, it's necessary to use streaming mode to decompress data. - * Optionally, application can rely on some implicit limit, - * as ZSTD_decompress() only needs an upper bound of decompressed size. - * (For example, data could be necessarily cut into blocks <= 16 KB). - * note 3 : decompressed size is always present when compression is completed using single-pass functions, - * such as ZSTD_compress(), ZSTD_compressCCtx() ZSTD_compress_usingDict() or ZSTD_compress_usingCDict(). - * note 4 : decompressed size can be very large (64-bits value), - * potentially larger than what local system can handle as a single memory segment. - * In which case, it's necessary to use streaming mode to decompress data. - * note 5 : If source is untrusted, decompressed size could be wrong or intentionally modified. - * Always ensure return value fits within application's authorized limits. - * Each application can set its own limits. - * note 6 : This function replaces ZSTD_getDecompressedSize() */ -#define ZSTD_CONTENTSIZE_UNKNOWN (0ULL - 1) -#define ZSTD_CONTENTSIZE_ERROR (0ULL - 2) -ZSTDLIB_API unsigned long long ZSTD_getFrameContentSize(const void *src, size_t srcSize); - -/*! ZSTD_getDecompressedSize() : - * NOTE: This function is now obsolete, in favor of ZSTD_getFrameContentSize(). - * Both functions work the same way, but ZSTD_getDecompressedSize() blends - * "empty", "unknown" and "error" results to the same return value (0), - * while ZSTD_getFrameContentSize() gives them separate return values. - * @return : decompressed size of `src` frame content _if known and not empty_, 0 otherwise. */ -ZSTDLIB_API unsigned long long ZSTD_getDecompressedSize(const void* src, size_t srcSize); - -/*! ZSTD_findFrameCompressedSize() : - * `src` should point to the start of a ZSTD frame or skippable frame. - * `srcSize` must be >= first frame size - * @return : the compressed size of the first frame starting at `src`, - * suitable to pass as `srcSize` to `ZSTD_decompress` or similar, - * or an error code if input is invalid */ -ZSTDLIB_API size_t ZSTD_findFrameCompressedSize(const void* src, size_t srcSize); - - -/*====== Helper functions ======*/ -#define ZSTD_COMPRESSBOUND(srcSize) ((srcSize) + ((srcSize)>>8) + (((srcSize) < (128<<10)) ? (((128<<10) - (srcSize)) >> 11) /* margin, from 64 to 0 */ : 0)) /* this formula ensures that bound(A) + bound(B) <= bound(A+B) as long as A and B >= 128 KB */ -ZSTDLIB_API size_t ZSTD_compressBound(size_t srcSize); /*!< maximum compressed size in worst case single-pass scenario */ -ZSTDLIB_API unsigned ZSTD_isError(size_t code); /*!< tells if a `size_t` function result is an error code */ -ZSTDLIB_API const char* ZSTD_getErrorName(size_t code); /*!< provides readable string from an error code */ -ZSTDLIB_API int ZSTD_minCLevel(void); /*!< minimum negative compression level allowed */ -ZSTDLIB_API int ZSTD_maxCLevel(void); /*!< maximum compression level available */ - - -/*************************************** -* Explicit context -***************************************/ -/*= Compression context - * When compressing many times, - * it is recommended to allocate a context just once, - * and re-use it for each successive compression operation. - * This will make workload friendlier for system's memory. - * Note : re-using context is just a speed / resource optimization. - * It doesn't change the compression ratio, which remains identical. - * Note 2 : In multi-threaded environments, - * use one different context per thread for parallel execution. - */ -typedef struct ZSTD_CCtx_s ZSTD_CCtx; -ZSTDLIB_API ZSTD_CCtx* ZSTD_createCCtx(void); -ZSTDLIB_API size_t ZSTD_freeCCtx(ZSTD_CCtx* cctx); - -/*! ZSTD_compressCCtx() : - * Same as ZSTD_compress(), using an explicit ZSTD_CCtx. - * Important : in order to behave similarly to `ZSTD_compress()`, - * this function compresses at requested compression level, - * __ignoring any other parameter__ . - * If any advanced parameter was set using the advanced API, - * they will all be reset. Only `compressionLevel` remains. - */ -ZSTDLIB_API size_t ZSTD_compressCCtx(ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - int compressionLevel); - -/*= Decompression context - * When decompressing many times, - * it is recommended to allocate a context only once, - * and re-use it for each successive compression operation. - * This will make workload friendlier for system's memory. - * Use one context per thread for parallel execution. */ -typedef struct ZSTD_DCtx_s ZSTD_DCtx; -ZSTDLIB_API ZSTD_DCtx* ZSTD_createDCtx(void); -ZSTDLIB_API size_t ZSTD_freeDCtx(ZSTD_DCtx* dctx); - -/*! ZSTD_decompressDCtx() : - * Same as ZSTD_decompress(), - * requires an allocated ZSTD_DCtx. - * Compatible with sticky parameters. - */ -ZSTDLIB_API size_t ZSTD_decompressDCtx(ZSTD_DCtx* dctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize); - - -/*************************************** -* Advanced compression API -***************************************/ - -/* API design : - * Parameters are pushed one by one into an existing context, - * using ZSTD_CCtx_set*() functions. - * Pushed parameters are sticky : they are valid for next compressed frame, and any subsequent frame. - * "sticky" parameters are applicable to `ZSTD_compress2()` and `ZSTD_compressStream*()` ! - * __They do not apply to "simple" one-shot variants such as ZSTD_compressCCtx()__ . - * - * It's possible to reset all parameters to "default" using ZSTD_CCtx_reset(). - * - * This API supercedes all other "advanced" API entry points in the experimental section. - * In the future, we expect to remove from experimental API entry points which are redundant with this API. - */ - - -/* Compression strategies, listed from fastest to strongest */ -typedef enum { ZSTD_fast=1, - ZSTD_dfast=2, - ZSTD_greedy=3, - ZSTD_lazy=4, - ZSTD_lazy2=5, - ZSTD_btlazy2=6, - ZSTD_btopt=7, - ZSTD_btultra=8, - ZSTD_btultra2=9 - /* note : new strategies _might_ be added in the future. - Only the order (from fast to strong) is guaranteed */ -} ZSTD_strategy; - - -typedef enum { - - /* compression parameters - * Note: When compressing with a ZSTD_CDict these parameters are superseded - * by the parameters used to construct the ZSTD_CDict. - * See ZSTD_CCtx_refCDict() for more info (superseded-by-cdict). */ - ZSTD_c_compressionLevel=100, /* Set compression parameters according to pre-defined cLevel table. - * Note that exact compression parameters are dynamically determined, - * depending on both compression level and srcSize (when known). - * Default level is ZSTD_CLEVEL_DEFAULT==3. - * Special: value 0 means default, which is controlled by ZSTD_CLEVEL_DEFAULT. - * Note 1 : it's possible to pass a negative compression level. - * Note 2 : setting a level does not automatically set all other compression parameters - * to default. Setting this will however eventually dynamically impact the compression - * parameters which have not been manually set. The manually set - * ones will 'stick'. */ - /* Advanced compression parameters : - * It's possible to pin down compression parameters to some specific values. - * In which case, these values are no longer dynamically selected by the compressor */ - ZSTD_c_windowLog=101, /* Maximum allowed back-reference distance, expressed as power of 2. - * This will set a memory budget for streaming decompression, - * with larger values requiring more memory - * and typically compressing more. - * Must be clamped between ZSTD_WINDOWLOG_MIN and ZSTD_WINDOWLOG_MAX. - * Special: value 0 means "use default windowLog". - * Note: Using a windowLog greater than ZSTD_WINDOWLOG_LIMIT_DEFAULT - * requires explicitly allowing such size at streaming decompression stage. */ - ZSTD_c_hashLog=102, /* Size of the initial probe table, as a power of 2. - * Resulting memory usage is (1 << (hashLog+2)). - * Must be clamped between ZSTD_HASHLOG_MIN and ZSTD_HASHLOG_MAX. - * Larger tables improve compression ratio of strategies <= dFast, - * and improve speed of strategies > dFast. - * Special: value 0 means "use default hashLog". */ - ZSTD_c_chainLog=103, /* Size of the multi-probe search table, as a power of 2. - * Resulting memory usage is (1 << (chainLog+2)). - * Must be clamped between ZSTD_CHAINLOG_MIN and ZSTD_CHAINLOG_MAX. - * Larger tables result in better and slower compression. - * This parameter is useless for "fast" strategy. - * It's still useful when using "dfast" strategy, - * in which case it defines a secondary probe table. - * Special: value 0 means "use default chainLog". */ - ZSTD_c_searchLog=104, /* Number of search attempts, as a power of 2. - * More attempts result in better and slower compression. - * This parameter is useless for "fast" and "dFast" strategies. - * Special: value 0 means "use default searchLog". */ - ZSTD_c_minMatch=105, /* Minimum size of searched matches. - * Note that Zstandard can still find matches of smaller size, - * it just tweaks its search algorithm to look for this size and larger. - * Larger values increase compression and decompression speed, but decrease ratio. - * Must be clamped between ZSTD_MINMATCH_MIN and ZSTD_MINMATCH_MAX. - * Note that currently, for all strategies < btopt, effective minimum is 4. - * , for all strategies > fast, effective maximum is 6. - * Special: value 0 means "use default minMatchLength". */ - ZSTD_c_targetLength=106, /* Impact of this field depends on strategy. - * For strategies btopt, btultra & btultra2: - * Length of Match considered "good enough" to stop search. - * Larger values make compression stronger, and slower. - * For strategy fast: - * Distance between match sampling. - * Larger values make compression faster, and weaker. - * Special: value 0 means "use default targetLength". */ - ZSTD_c_strategy=107, /* See ZSTD_strategy enum definition. - * The higher the value of selected strategy, the more complex it is, - * resulting in stronger and slower compression. - * Special: value 0 means "use default strategy". */ - - /* LDM mode parameters */ - ZSTD_c_enableLongDistanceMatching=160, /* Enable long distance matching. - * This parameter is designed to improve compression ratio - * for large inputs, by finding large matches at long distance. - * It increases memory usage and window size. - * Note: enabling this parameter increases default ZSTD_c_windowLog to 128 MB - * except when expressly set to a different value. */ - ZSTD_c_ldmHashLog=161, /* Size of the table for long distance matching, as a power of 2. - * Larger values increase memory usage and compression ratio, - * but decrease compression speed. - * Must be clamped between ZSTD_HASHLOG_MIN and ZSTD_HASHLOG_MAX - * default: windowlog - 7. - * Special: value 0 means "automatically determine hashlog". */ - ZSTD_c_ldmMinMatch=162, /* Minimum match size for long distance matcher. - * Larger/too small values usually decrease compression ratio. - * Must be clamped between ZSTD_LDM_MINMATCH_MIN and ZSTD_LDM_MINMATCH_MAX. - * Special: value 0 means "use default value" (default: 64). */ - ZSTD_c_ldmBucketSizeLog=163, /* Log size of each bucket in the LDM hash table for collision resolution. - * Larger values improve collision resolution but decrease compression speed. - * The maximum value is ZSTD_LDM_BUCKETSIZELOG_MAX. - * Special: value 0 means "use default value" (default: 3). */ - ZSTD_c_ldmHashRateLog=164, /* Frequency of inserting/looking up entries into the LDM hash table. - * Must be clamped between 0 and (ZSTD_WINDOWLOG_MAX - ZSTD_HASHLOG_MIN). - * Default is MAX(0, (windowLog - ldmHashLog)), optimizing hash table usage. - * Larger values improve compression speed. - * Deviating far from default value will likely result in a compression ratio decrease. - * Special: value 0 means "automatically determine hashRateLog". */ - - /* frame parameters */ - ZSTD_c_contentSizeFlag=200, /* Content size will be written into frame header _whenever known_ (default:1) - * Content size must be known at the beginning of compression. - * This is automatically the case when using ZSTD_compress2(), - * For streaming scenarios, content size must be provided with ZSTD_CCtx_setPledgedSrcSize() */ - ZSTD_c_checksumFlag=201, /* A 32-bits checksum of content is written at end of frame (default:0) */ - ZSTD_c_dictIDFlag=202, /* When applicable, dictionary's ID is written into frame header (default:1) */ - - /* multi-threading parameters */ - /* These parameters are only useful if multi-threading is enabled (compiled with build macro ZSTD_MULTITHREAD). - * They return an error otherwise. */ - ZSTD_c_nbWorkers=400, /* Select how many threads will be spawned to compress in parallel. - * When nbWorkers >= 1, triggers asynchronous mode when used with ZSTD_compressStream*() : - * ZSTD_compressStream*() consumes input and flush output if possible, but immediately gives back control to caller, - * while compression work is performed in parallel, within worker threads. - * (note : a strong exception to this rule is when first invocation of ZSTD_compressStream2() sets ZSTD_e_end : - * in which case, ZSTD_compressStream2() delegates to ZSTD_compress2(), which is always a blocking call). - * More workers improve speed, but also increase memory usage. - * Default value is `0`, aka "single-threaded mode" : no worker is spawned, compression is performed inside Caller's thread, all invocations are blocking */ - ZSTD_c_jobSize=401, /* Size of a compression job. This value is enforced only when nbWorkers >= 1. - * Each compression job is completed in parallel, so this value can indirectly impact the nb of active threads. - * 0 means default, which is dynamically determined based on compression parameters. - * Job size must be a minimum of overlap size, or 1 MB, whichever is largest. - * The minimum size is automatically and transparently enforced. */ - ZSTD_c_overlapLog=402, /* Control the overlap size, as a fraction of window size. - * The overlap size is an amount of data reloaded from previous job at the beginning of a new job. - * It helps preserve compression ratio, while each job is compressed in parallel. - * This value is enforced only when nbWorkers >= 1. - * Larger values increase compression ratio, but decrease speed. - * Possible values range from 0 to 9 : - * - 0 means "default" : value will be determined by the library, depending on strategy - * - 1 means "no overlap" - * - 9 means "full overlap", using a full window size. - * Each intermediate rank increases/decreases load size by a factor 2 : - * 9: full window; 8: w/2; 7: w/4; 6: w/8; 5:w/16; 4: w/32; 3:w/64; 2:w/128; 1:no overlap; 0:default - * default value varies between 6 and 9, depending on strategy */ - - /* note : additional experimental parameters are also available - * within the experimental section of the API. - * At the time of this writing, they include : - * ZSTD_c_rsyncable - * ZSTD_c_format - * ZSTD_c_forceMaxWindow - * ZSTD_c_forceAttachDict - * ZSTD_c_literalCompressionMode - * ZSTD_c_targetCBlockSize - * ZSTD_c_srcSizeHint - * Because they are not stable, it's necessary to define ZSTD_STATIC_LINKING_ONLY to access them. - * note : never ever use experimentalParam? names directly; - * also, the enums values themselves are unstable and can still change. - */ - ZSTD_c_experimentalParam1=500, - ZSTD_c_experimentalParam2=10, - ZSTD_c_experimentalParam3=1000, - ZSTD_c_experimentalParam4=1001, - ZSTD_c_experimentalParam5=1002, - ZSTD_c_experimentalParam6=1003, - ZSTD_c_experimentalParam7=1004 -} ZSTD_cParameter; - -typedef struct { - size_t error; - int lowerBound; - int upperBound; -} ZSTD_bounds; - -/*! ZSTD_cParam_getBounds() : - * All parameters must belong to an interval with lower and upper bounds, - * otherwise they will either trigger an error or be automatically clamped. - * @return : a structure, ZSTD_bounds, which contains - * - an error status field, which must be tested using ZSTD_isError() - * - lower and upper bounds, both inclusive - */ -ZSTDLIB_API ZSTD_bounds ZSTD_cParam_getBounds(ZSTD_cParameter cParam); - -/*! ZSTD_CCtx_setParameter() : - * Set one compression parameter, selected by enum ZSTD_cParameter. - * All parameters have valid bounds. Bounds can be queried using ZSTD_cParam_getBounds(). - * Providing a value beyond bound will either clamp it, or trigger an error (depending on parameter). - * Setting a parameter is generally only possible during frame initialization (before starting compression). - * Exception : when using multi-threading mode (nbWorkers >= 1), - * the following parameters can be updated _during_ compression (within same frame): - * => compressionLevel, hashLog, chainLog, searchLog, minMatch, targetLength and strategy. - * new parameters will be active for next job only (after a flush()). - * @return : an error code (which can be tested using ZSTD_isError()). - */ -ZSTDLIB_API size_t ZSTD_CCtx_setParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param, int value); - -/*! ZSTD_CCtx_setPledgedSrcSize() : - * Total input data size to be compressed as a single frame. - * Value will be written in frame header, unless if explicitly forbidden using ZSTD_c_contentSizeFlag. - * This value will also be controlled at end of frame, and trigger an error if not respected. - * @result : 0, or an error code (which can be tested with ZSTD_isError()). - * Note 1 : pledgedSrcSize==0 actually means zero, aka an empty frame. - * In order to mean "unknown content size", pass constant ZSTD_CONTENTSIZE_UNKNOWN. - * ZSTD_CONTENTSIZE_UNKNOWN is default value for any new frame. - * Note 2 : pledgedSrcSize is only valid once, for the next frame. - * It's discarded at the end of the frame, and replaced by ZSTD_CONTENTSIZE_UNKNOWN. - * Note 3 : Whenever all input data is provided and consumed in a single round, - * for example with ZSTD_compress2(), - * or invoking immediately ZSTD_compressStream2(,,,ZSTD_e_end), - * this value is automatically overridden by srcSize instead. - */ -ZSTDLIB_API size_t ZSTD_CCtx_setPledgedSrcSize(ZSTD_CCtx* cctx, unsigned long long pledgedSrcSize); - -typedef enum { - ZSTD_reset_session_only = 1, - ZSTD_reset_parameters = 2, - ZSTD_reset_session_and_parameters = 3 -} ZSTD_ResetDirective; - -/*! ZSTD_CCtx_reset() : - * There are 2 different things that can be reset, independently or jointly : - * - The session : will stop compressing current frame, and make CCtx ready to start a new one. - * Useful after an error, or to interrupt any ongoing compression. - * Any internal data not yet flushed is cancelled. - * Compression parameters and dictionary remain unchanged. - * They will be used to compress next frame. - * Resetting session never fails. - * - The parameters : changes all parameters back to "default". - * This removes any reference to any dictionary too. - * Parameters can only be changed between 2 sessions (i.e. no compression is currently ongoing) - * otherwise the reset fails, and function returns an error value (which can be tested using ZSTD_isError()) - * - Both : similar to resetting the session, followed by resetting parameters. - */ -ZSTDLIB_API size_t ZSTD_CCtx_reset(ZSTD_CCtx* cctx, ZSTD_ResetDirective reset); - -/*! ZSTD_compress2() : - * Behave the same as ZSTD_compressCCtx(), but compression parameters are set using the advanced API. - * ZSTD_compress2() always starts a new frame. - * Should cctx hold data from a previously unfinished frame, everything about it is forgotten. - * - Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*() - * - The function is always blocking, returns when compression is completed. - * Hint : compression runs faster if `dstCapacity` >= `ZSTD_compressBound(srcSize)`. - * @return : compressed size written into `dst` (<= `dstCapacity), - * or an error code if it fails (which can be tested using ZSTD_isError()). - */ -ZSTDLIB_API size_t ZSTD_compress2( ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize); - - -/*************************************** -* Advanced decompression API -***************************************/ - -/* The advanced API pushes parameters one by one into an existing DCtx context. - * Parameters are sticky, and remain valid for all following frames - * using the same DCtx context. - * It's possible to reset parameters to default values using ZSTD_DCtx_reset(). - * Note : This API is compatible with existing ZSTD_decompressDCtx() and ZSTD_decompressStream(). - * Therefore, no new decompression function is necessary. - */ - -typedef enum { - - ZSTD_d_windowLogMax=100, /* Select a size limit (in power of 2) beyond which - * the streaming API will refuse to allocate memory buffer - * in order to protect the host from unreasonable memory requirements. - * This parameter is only useful in streaming mode, since no internal buffer is allocated in single-pass mode. - * By default, a decompression context accepts window sizes <= (1 << ZSTD_WINDOWLOG_LIMIT_DEFAULT). - * Special: value 0 means "use default maximum windowLog". */ - - /* note : additional experimental parameters are also available - * within the experimental section of the API. - * At the time of this writing, they include : - * ZSTD_d_format - * ZSTD_d_stableOutBuffer - * Because they are not stable, it's necessary to define ZSTD_STATIC_LINKING_ONLY to access them. - * note : never ever use experimentalParam? names directly - */ - ZSTD_d_experimentalParam1=1000, - ZSTD_d_experimentalParam2=1001 - -} ZSTD_dParameter; - -/*! ZSTD_dParam_getBounds() : - * All parameters must belong to an interval with lower and upper bounds, - * otherwise they will either trigger an error or be automatically clamped. - * @return : a structure, ZSTD_bounds, which contains - * - an error status field, which must be tested using ZSTD_isError() - * - both lower and upper bounds, inclusive - */ -ZSTDLIB_API ZSTD_bounds ZSTD_dParam_getBounds(ZSTD_dParameter dParam); - -/*! ZSTD_DCtx_setParameter() : - * Set one compression parameter, selected by enum ZSTD_dParameter. - * All parameters have valid bounds. Bounds can be queried using ZSTD_dParam_getBounds(). - * Providing a value beyond bound will either clamp it, or trigger an error (depending on parameter). - * Setting a parameter is only possible during frame initialization (before starting decompression). - * @return : 0, or an error code (which can be tested using ZSTD_isError()). - */ -ZSTDLIB_API size_t ZSTD_DCtx_setParameter(ZSTD_DCtx* dctx, ZSTD_dParameter param, int value); - -/*! ZSTD_DCtx_reset() : - * Return a DCtx to clean state. - * Session and parameters can be reset jointly or separately. - * Parameters can only be reset when no active frame is being decompressed. - * @return : 0, or an error code, which can be tested with ZSTD_isError() - */ -ZSTDLIB_API size_t ZSTD_DCtx_reset(ZSTD_DCtx* dctx, ZSTD_ResetDirective reset); - - -/**************************** -* Streaming -****************************/ - -typedef struct ZSTD_inBuffer_s { - const void* src; /**< start of input buffer */ - size_t size; /**< size of input buffer */ - size_t pos; /**< position where reading stopped. Will be updated. Necessarily 0 <= pos <= size */ -} ZSTD_inBuffer; - -typedef struct ZSTD_outBuffer_s { - void* dst; /**< start of output buffer */ - size_t size; /**< size of output buffer */ - size_t pos; /**< position where writing stopped. Will be updated. Necessarily 0 <= pos <= size */ -} ZSTD_outBuffer; - - - -/*-*********************************************************************** -* Streaming compression - HowTo -* -* A ZSTD_CStream object is required to track streaming operation. -* Use ZSTD_createCStream() and ZSTD_freeCStream() to create/release resources. -* ZSTD_CStream objects can be reused multiple times on consecutive compression operations. -* It is recommended to re-use ZSTD_CStream since it will play nicer with system's memory, by re-using already allocated memory. -* -* For parallel execution, use one separate ZSTD_CStream per thread. -* -* note : since v1.3.0, ZSTD_CStream and ZSTD_CCtx are the same thing. -* -* Parameters are sticky : when starting a new compression on the same context, -* it will re-use the same sticky parameters as previous compression session. -* When in doubt, it's recommended to fully initialize the context before usage. -* Use ZSTD_CCtx_reset() to reset the context and ZSTD_CCtx_setParameter(), -* ZSTD_CCtx_setPledgedSrcSize(), or ZSTD_CCtx_loadDictionary() and friends to -* set more specific parameters, the pledged source size, or load a dictionary. -* -* Use ZSTD_compressStream2() with ZSTD_e_continue as many times as necessary to -* consume input stream. The function will automatically update both `pos` -* fields within `input` and `output`. -* Note that the function may not consume the entire input, for example, because -* the output buffer is already full, in which case `input.pos < input.size`. -* The caller must check if input has been entirely consumed. -* If not, the caller must make some room to receive more compressed data, -* and then present again remaining input data. -* note: ZSTD_e_continue is guaranteed to make some forward progress when called, -* but doesn't guarantee maximal forward progress. This is especially relevant -* when compressing with multiple threads. The call won't block if it can -* consume some input, but if it can't it will wait for some, but not all, -* output to be flushed. -* @return : provides a minimum amount of data remaining to be flushed from internal buffers -* or an error code, which can be tested using ZSTD_isError(). -* -* At any moment, it's possible to flush whatever data might remain stuck within internal buffer, -* using ZSTD_compressStream2() with ZSTD_e_flush. `output->pos` will be updated. -* Note that, if `output->size` is too small, a single invocation with ZSTD_e_flush might not be enough (return code > 0). -* In which case, make some room to receive more compressed data, and call again ZSTD_compressStream2() with ZSTD_e_flush. -* You must continue calling ZSTD_compressStream2() with ZSTD_e_flush until it returns 0, at which point you can change the -* operation. -* note: ZSTD_e_flush will flush as much output as possible, meaning when compressing with multiple threads, it will -* block until the flush is complete or the output buffer is full. -* @return : 0 if internal buffers are entirely flushed, -* >0 if some data still present within internal buffer (the value is minimal estimation of remaining size), -* or an error code, which can be tested using ZSTD_isError(). -* -* Calling ZSTD_compressStream2() with ZSTD_e_end instructs to finish a frame. -* It will perform a flush and write frame epilogue. -* The epilogue is required for decoders to consider a frame completed. -* flush operation is the same, and follows same rules as calling ZSTD_compressStream2() with ZSTD_e_flush. -* You must continue calling ZSTD_compressStream2() with ZSTD_e_end until it returns 0, at which point you are free to -* start a new frame. -* note: ZSTD_e_end will flush as much output as possible, meaning when compressing with multiple threads, it will -* block until the flush is complete or the output buffer is full. -* @return : 0 if frame fully completed and fully flushed, -* >0 if some data still present within internal buffer (the value is minimal estimation of remaining size), -* or an error code, which can be tested using ZSTD_isError(). -* -* *******************************************************************/ - -typedef ZSTD_CCtx ZSTD_CStream; /**< CCtx and CStream are now effectively same object (>= v1.3.0) */ - /* Continue to distinguish them for compatibility with older versions <= v1.2.0 */ -/*===== ZSTD_CStream management functions =====*/ -ZSTDLIB_API ZSTD_CStream* ZSTD_createCStream(void); -ZSTDLIB_API size_t ZSTD_freeCStream(ZSTD_CStream* zcs); - -/*===== Streaming compression functions =====*/ -typedef enum { - ZSTD_e_continue=0, /* collect more data, encoder decides when to output compressed result, for optimal compression ratio */ - ZSTD_e_flush=1, /* flush any data provided so far, - * it creates (at least) one new block, that can be decoded immediately on reception; - * frame will continue: any future data can still reference previously compressed data, improving compression. - * note : multithreaded compression will block to flush as much output as possible. */ - ZSTD_e_end=2 /* flush any remaining data _and_ close current frame. - * note that frame is only closed after compressed data is fully flushed (return value == 0). - * After that point, any additional data starts a new frame. - * note : each frame is independent (does not reference any content from previous frame). - : note : multithreaded compression will block to flush as much output as possible. */ -} ZSTD_EndDirective; - -/*! ZSTD_compressStream2() : - * Behaves about the same as ZSTD_compressStream, with additional control on end directive. - * - Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*() - * - Compression parameters cannot be changed once compression is started (save a list of exceptions in multi-threading mode) - * - output->pos must be <= dstCapacity, input->pos must be <= srcSize - * - output->pos and input->pos will be updated. They are guaranteed to remain below their respective limit. - * - When nbWorkers==0 (default), function is blocking : it completes its job before returning to caller. - * - When nbWorkers>=1, function is non-blocking : it just acquires a copy of input, and distributes jobs to internal worker threads, flush whatever is available, - * and then immediately returns, just indicating that there is some data remaining to be flushed. - * The function nonetheless guarantees forward progress : it will return only after it reads or write at least 1+ byte. - * - Exception : if the first call requests a ZSTD_e_end directive and provides enough dstCapacity, the function delegates to ZSTD_compress2() which is always blocking. - * - @return provides a minimum amount of data remaining to be flushed from internal buffers - * or an error code, which can be tested using ZSTD_isError(). - * if @return != 0, flush is not fully completed, there is still some data left within internal buffers. - * This is useful for ZSTD_e_flush, since in this case more flushes are necessary to empty all buffers. - * For ZSTD_e_end, @return == 0 when internal buffers are fully flushed and frame is completed. - * - after a ZSTD_e_end directive, if internal buffer is not fully flushed (@return != 0), - * only ZSTD_e_end or ZSTD_e_flush operations are allowed. - * Before starting a new compression job, or changing compression parameters, - * it is required to fully flush internal buffers. - */ -ZSTDLIB_API size_t ZSTD_compressStream2( ZSTD_CCtx* cctx, - ZSTD_outBuffer* output, - ZSTD_inBuffer* input, - ZSTD_EndDirective endOp); - - -/* These buffer sizes are softly recommended. - * They are not required : ZSTD_compressStream*() happily accepts any buffer size, for both input and output. - * Respecting the recommended size just makes it a bit easier for ZSTD_compressStream*(), - * reducing the amount of memory shuffling and buffering, resulting in minor performance savings. - * - * However, note that these recommendations are from the perspective of a C caller program. - * If the streaming interface is invoked from some other language, - * especially managed ones such as Java or Go, through a foreign function interface such as jni or cgo, - * a major performance rule is to reduce crossing such interface to an absolute minimum. - * It's not rare that performance ends being spent more into the interface, rather than compression itself. - * In which cases, prefer using large buffers, as large as practical, - * for both input and output, to reduce the nb of roundtrips. - */ -ZSTDLIB_API size_t ZSTD_CStreamInSize(void); /**< recommended size for input buffer */ -ZSTDLIB_API size_t ZSTD_CStreamOutSize(void); /**< recommended size for output buffer. Guarantee to successfully flush at least one complete compressed block. */ - - -/* ***************************************************************************** - * This following is a legacy streaming API. - * It can be replaced by ZSTD_CCtx_reset() and ZSTD_compressStream2(). - * It is redundant, but remains fully supported. - * Advanced parameters and dictionary compression can only be used through the - * new API. - ******************************************************************************/ - -/*! - * Equivalent to: - * - * ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only); - * ZSTD_CCtx_refCDict(zcs, NULL); // clear the dictionary (if any) - * ZSTD_CCtx_setParameter(zcs, ZSTD_c_compressionLevel, compressionLevel); - */ -ZSTDLIB_API size_t ZSTD_initCStream(ZSTD_CStream* zcs, int compressionLevel); -/*! - * Alternative for ZSTD_compressStream2(zcs, output, input, ZSTD_e_continue). - * NOTE: The return value is different. ZSTD_compressStream() returns a hint for - * the next read size (if non-zero and not an error). ZSTD_compressStream2() - * returns the minimum nb of bytes left to flush (if non-zero and not an error). - */ -ZSTDLIB_API size_t ZSTD_compressStream(ZSTD_CStream* zcs, ZSTD_outBuffer* output, ZSTD_inBuffer* input); -/*! Equivalent to ZSTD_compressStream2(zcs, output, &emptyInput, ZSTD_e_flush). */ -ZSTDLIB_API size_t ZSTD_flushStream(ZSTD_CStream* zcs, ZSTD_outBuffer* output); -/*! Equivalent to ZSTD_compressStream2(zcs, output, &emptyInput, ZSTD_e_end). */ -ZSTDLIB_API size_t ZSTD_endStream(ZSTD_CStream* zcs, ZSTD_outBuffer* output); - - -/*-*************************************************************************** -* Streaming decompression - HowTo -* -* A ZSTD_DStream object is required to track streaming operations. -* Use ZSTD_createDStream() and ZSTD_freeDStream() to create/release resources. -* ZSTD_DStream objects can be re-used multiple times. -* -* Use ZSTD_initDStream() to start a new decompression operation. -* @return : recommended first input size -* Alternatively, use advanced API to set specific properties. -* -* Use ZSTD_decompressStream() repetitively to consume your input. -* The function will update both `pos` fields. -* If `input.pos < input.size`, some input has not been consumed. -* It's up to the caller to present again remaining data. -* The function tries to flush all data decoded immediately, respecting output buffer size. -* If `output.pos < output.size`, decoder has flushed everything it could. -* But if `output.pos == output.size`, there might be some data left within internal buffers., -* In which case, call ZSTD_decompressStream() again to flush whatever remains in the buffer. -* Note : with no additional input provided, amount of data flushed is necessarily <= ZSTD_BLOCKSIZE_MAX. -* @return : 0 when a frame is completely decoded and fully flushed, -* or an error code, which can be tested using ZSTD_isError(), -* or any other value > 0, which means there is still some decoding or flushing to do to complete current frame : -* the return value is a suggested next input size (just a hint for better latency) -* that will never request more than the remaining frame size. -* *******************************************************************************/ - -typedef ZSTD_DCtx ZSTD_DStream; /**< DCtx and DStream are now effectively same object (>= v1.3.0) */ - /* For compatibility with versions <= v1.2.0, prefer differentiating them. */ -/*===== ZSTD_DStream management functions =====*/ -ZSTDLIB_API ZSTD_DStream* ZSTD_createDStream(void); -ZSTDLIB_API size_t ZSTD_freeDStream(ZSTD_DStream* zds); - -/*===== Streaming decompression functions =====*/ - -/* This function is redundant with the advanced API and equivalent to: - * - * ZSTD_DCtx_reset(zds, ZSTD_reset_session_only); - * ZSTD_DCtx_refDDict(zds, NULL); - */ -ZSTDLIB_API size_t ZSTD_initDStream(ZSTD_DStream* zds); - -ZSTDLIB_API size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inBuffer* input); - -ZSTDLIB_API size_t ZSTD_DStreamInSize(void); /*!< recommended size for input buffer */ -ZSTDLIB_API size_t ZSTD_DStreamOutSize(void); /*!< recommended size for output buffer. Guarantee to successfully flush at least one complete block in all circumstances. */ - - -/************************** -* Simple dictionary API -***************************/ -/*! ZSTD_compress_usingDict() : - * Compression at an explicit compression level using a Dictionary. - * A dictionary can be any arbitrary data segment (also called a prefix), - * or a buffer with specified information (see dictBuilder/zdict.h). - * Note : This function loads the dictionary, resulting in significant startup delay. - * It's intended for a dictionary used only once. - * Note 2 : When `dict == NULL || dictSize < 8` no dictionary is used. */ -ZSTDLIB_API size_t ZSTD_compress_usingDict(ZSTD_CCtx* ctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const void* dict,size_t dictSize, - int compressionLevel); - -/*! ZSTD_decompress_usingDict() : - * Decompression using a known Dictionary. - * Dictionary must be identical to the one used during compression. - * Note : This function loads the dictionary, resulting in significant startup delay. - * It's intended for a dictionary used only once. - * Note : When `dict == NULL || dictSize < 8` no dictionary is used. */ -ZSTDLIB_API size_t ZSTD_decompress_usingDict(ZSTD_DCtx* dctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const void* dict,size_t dictSize); - - -/*********************************** - * Bulk processing dictionary API - **********************************/ -typedef struct ZSTD_CDict_s ZSTD_CDict; - -/*! ZSTD_createCDict() : - * When compressing multiple messages or blocks using the same dictionary, - * it's recommended to digest the dictionary only once, since it's a costly operation. - * ZSTD_createCDict() will create a state from digesting a dictionary. - * The resulting state can be used for future compression operations with very limited startup cost. - * ZSTD_CDict can be created once and shared by multiple threads concurrently, since its usage is read-only. - * @dictBuffer can be released after ZSTD_CDict creation, because its content is copied within CDict. - * Note 1 : Consider experimental function `ZSTD_createCDict_byReference()` if you prefer to not duplicate @dictBuffer content. - * Note 2 : A ZSTD_CDict can be created from an empty @dictBuffer, - * in which case the only thing that it transports is the @compressionLevel. - * This can be useful in a pipeline featuring ZSTD_compress_usingCDict() exclusively, - * expecting a ZSTD_CDict parameter with any data, including those without a known dictionary. */ -ZSTDLIB_API ZSTD_CDict* ZSTD_createCDict(const void* dictBuffer, size_t dictSize, - int compressionLevel); - -/*! ZSTD_freeCDict() : - * Function frees memory allocated by ZSTD_createCDict(). */ -ZSTDLIB_API size_t ZSTD_freeCDict(ZSTD_CDict* CDict); - -/*! ZSTD_compress_usingCDict() : - * Compression using a digested Dictionary. - * Recommended when same dictionary is used multiple times. - * Note : compression level is _decided at dictionary creation time_, - * and frame parameters are hardcoded (dictID=yes, contentSize=yes, checksum=no) */ -ZSTDLIB_API size_t ZSTD_compress_usingCDict(ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const ZSTD_CDict* cdict); - - -typedef struct ZSTD_DDict_s ZSTD_DDict; - -/*! ZSTD_createDDict() : - * Create a digested dictionary, ready to start decompression operation without startup delay. - * dictBuffer can be released after DDict creation, as its content is copied inside DDict. */ -ZSTDLIB_API ZSTD_DDict* ZSTD_createDDict(const void* dictBuffer, size_t dictSize); - -/*! ZSTD_freeDDict() : - * Function frees memory allocated with ZSTD_createDDict() */ -ZSTDLIB_API size_t ZSTD_freeDDict(ZSTD_DDict* ddict); - -/*! ZSTD_decompress_usingDDict() : - * Decompression using a digested Dictionary. - * Recommended when same dictionary is used multiple times. */ -ZSTDLIB_API size_t ZSTD_decompress_usingDDict(ZSTD_DCtx* dctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const ZSTD_DDict* ddict); - - -/******************************** - * Dictionary helper functions - *******************************/ - -/*! ZSTD_getDictID_fromDict() : - * Provides the dictID stored within dictionary. - * if @return == 0, the dictionary is not conformant with Zstandard specification. - * It can still be loaded, but as a content-only dictionary. */ -ZSTDLIB_API unsigned ZSTD_getDictID_fromDict(const void* dict, size_t dictSize); - -/*! ZSTD_getDictID_fromDDict() : - * Provides the dictID of the dictionary loaded into `ddict`. - * If @return == 0, the dictionary is not conformant to Zstandard specification, or empty. - * Non-conformant dictionaries can still be loaded, but as content-only dictionaries. */ -ZSTDLIB_API unsigned ZSTD_getDictID_fromDDict(const ZSTD_DDict* ddict); - -/*! ZSTD_getDictID_fromFrame() : - * Provides the dictID required to decompressed the frame stored within `src`. - * If @return == 0, the dictID could not be decoded. - * This could for one of the following reasons : - * - The frame does not require a dictionary to be decoded (most common case). - * - The frame was built with dictID intentionally removed. Whatever dictionary is necessary is a hidden information. - * Note : this use case also happens when using a non-conformant dictionary. - * - `srcSize` is too small, and as a result, the frame header could not be decoded (only possible if `srcSize < ZSTD_FRAMEHEADERSIZE_MAX`). - * - This is not a Zstandard frame. - * When identifying the exact failure cause, it's possible to use ZSTD_getFrameHeader(), which will provide a more precise error code. */ -ZSTDLIB_API unsigned ZSTD_getDictID_fromFrame(const void* src, size_t srcSize); - - -/******************************************************************************* - * Advanced dictionary and prefix API - * - * This API allows dictionaries to be used with ZSTD_compress2(), - * ZSTD_compressStream2(), and ZSTD_decompress(). Dictionaries are sticky, and - * only reset with the context is reset with ZSTD_reset_parameters or - * ZSTD_reset_session_and_parameters. Prefixes are single-use. - ******************************************************************************/ - - -/*! ZSTD_CCtx_loadDictionary() : - * Create an internal CDict from `dict` buffer. - * Decompression will have to use same dictionary. - * @result : 0, or an error code (which can be tested with ZSTD_isError()). - * Special: Loading a NULL (or 0-size) dictionary invalidates previous dictionary, - * meaning "return to no-dictionary mode". - * Note 1 : Dictionary is sticky, it will be used for all future compressed frames. - * To return to "no-dictionary" situation, load a NULL dictionary (or reset parameters). - * Note 2 : Loading a dictionary involves building tables. - * It's also a CPU consuming operation, with non-negligible impact on latency. - * Tables are dependent on compression parameters, and for this reason, - * compression parameters can no longer be changed after loading a dictionary. - * Note 3 :`dict` content will be copied internally. - * Use experimental ZSTD_CCtx_loadDictionary_byReference() to reference content instead. - * In such a case, dictionary buffer must outlive its users. - * Note 4 : Use ZSTD_CCtx_loadDictionary_advanced() - * to precisely select how dictionary content must be interpreted. */ -ZSTDLIB_API size_t ZSTD_CCtx_loadDictionary(ZSTD_CCtx* cctx, const void* dict, size_t dictSize); - -/*! ZSTD_CCtx_refCDict() : - * Reference a prepared dictionary, to be used for all next compressed frames. - * Note that compression parameters are enforced from within CDict, - * and supersede any compression parameter previously set within CCtx. - * The parameters ignored are labled as "superseded-by-cdict" in the ZSTD_cParameter enum docs. - * The ignored parameters will be used again if the CCtx is returned to no-dictionary mode. - * The dictionary will remain valid for future compressed frames using same CCtx. - * @result : 0, or an error code (which can be tested with ZSTD_isError()). - * Special : Referencing a NULL CDict means "return to no-dictionary mode". - * Note 1 : Currently, only one dictionary can be managed. - * Referencing a new dictionary effectively "discards" any previous one. - * Note 2 : CDict is just referenced, its lifetime must outlive its usage within CCtx. */ -ZSTDLIB_API size_t ZSTD_CCtx_refCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict); - -/*! ZSTD_CCtx_refPrefix() : - * Reference a prefix (single-usage dictionary) for next compressed frame. - * A prefix is **only used once**. Tables are discarded at end of frame (ZSTD_e_end). - * Decompression will need same prefix to properly regenerate data. - * Compressing with a prefix is similar in outcome as performing a diff and compressing it, - * but performs much faster, especially during decompression (compression speed is tunable with compression level). - * @result : 0, or an error code (which can be tested with ZSTD_isError()). - * Special: Adding any prefix (including NULL) invalidates any previous prefix or dictionary - * Note 1 : Prefix buffer is referenced. It **must** outlive compression. - * Its content must remain unmodified during compression. - * Note 2 : If the intention is to diff some large src data blob with some prior version of itself, - * ensure that the window size is large enough to contain the entire source. - * See ZSTD_c_windowLog. - * Note 3 : Referencing a prefix involves building tables, which are dependent on compression parameters. - * It's a CPU consuming operation, with non-negligible impact on latency. - * If there is a need to use the same prefix multiple times, consider loadDictionary instead. - * Note 4 : By default, the prefix is interpreted as raw content (ZSTD_dct_rawContent). - * Use experimental ZSTD_CCtx_refPrefix_advanced() to alter dictionary interpretation. */ -ZSTDLIB_API size_t ZSTD_CCtx_refPrefix(ZSTD_CCtx* cctx, - const void* prefix, size_t prefixSize); - -/*! ZSTD_DCtx_loadDictionary() : - * Create an internal DDict from dict buffer, - * to be used to decompress next frames. - * The dictionary remains valid for all future frames, until explicitly invalidated. - * @result : 0, or an error code (which can be tested with ZSTD_isError()). - * Special : Adding a NULL (or 0-size) dictionary invalidates any previous dictionary, - * meaning "return to no-dictionary mode". - * Note 1 : Loading a dictionary involves building tables, - * which has a non-negligible impact on CPU usage and latency. - * It's recommended to "load once, use many times", to amortize the cost - * Note 2 :`dict` content will be copied internally, so `dict` can be released after loading. - * Use ZSTD_DCtx_loadDictionary_byReference() to reference dictionary content instead. - * Note 3 : Use ZSTD_DCtx_loadDictionary_advanced() to take control of - * how dictionary content is loaded and interpreted. - */ -ZSTDLIB_API size_t ZSTD_DCtx_loadDictionary(ZSTD_DCtx* dctx, const void* dict, size_t dictSize); - -/*! ZSTD_DCtx_refDDict() : - * Reference a prepared dictionary, to be used to decompress next frames. - * The dictionary remains active for decompression of future frames using same DCtx. - * @result : 0, or an error code (which can be tested with ZSTD_isError()). - * Note 1 : Currently, only one dictionary can be managed. - * Referencing a new dictionary effectively "discards" any previous one. - * Special: referencing a NULL DDict means "return to no-dictionary mode". - * Note 2 : DDict is just referenced, its lifetime must outlive its usage from DCtx. - */ -ZSTDLIB_API size_t ZSTD_DCtx_refDDict(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict); - -/*! ZSTD_DCtx_refPrefix() : - * Reference a prefix (single-usage dictionary) to decompress next frame. - * This is the reverse operation of ZSTD_CCtx_refPrefix(), - * and must use the same prefix as the one used during compression. - * Prefix is **only used once**. Reference is discarded at end of frame. - * End of frame is reached when ZSTD_decompressStream() returns 0. - * @result : 0, or an error code (which can be tested with ZSTD_isError()). - * Note 1 : Adding any prefix (including NULL) invalidates any previously set prefix or dictionary - * Note 2 : Prefix buffer is referenced. It **must** outlive decompression. - * Prefix buffer must remain unmodified up to the end of frame, - * reached when ZSTD_decompressStream() returns 0. - * Note 3 : By default, the prefix is treated as raw content (ZSTD_dct_rawContent). - * Use ZSTD_CCtx_refPrefix_advanced() to alter dictMode (Experimental section) - * Note 4 : Referencing a raw content prefix has almost no cpu nor memory cost. - * A full dictionary is more costly, as it requires building tables. - */ -ZSTDLIB_API size_t ZSTD_DCtx_refPrefix(ZSTD_DCtx* dctx, - const void* prefix, size_t prefixSize); - -/* === Memory management === */ - -/*! ZSTD_sizeof_*() : - * These functions give the _current_ memory usage of selected object. - * Note that object memory usage can evolve (increase or decrease) over time. */ -ZSTDLIB_API size_t ZSTD_sizeof_CCtx(const ZSTD_CCtx* cctx); -ZSTDLIB_API size_t ZSTD_sizeof_DCtx(const ZSTD_DCtx* dctx); -ZSTDLIB_API size_t ZSTD_sizeof_CStream(const ZSTD_CStream* zcs); -ZSTDLIB_API size_t ZSTD_sizeof_DStream(const ZSTD_DStream* zds); -ZSTDLIB_API size_t ZSTD_sizeof_CDict(const ZSTD_CDict* cdict); -ZSTDLIB_API size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict); - -#endif /* ZSTD_H_235446 */ - - -/* ************************************************************************************** - * ADVANCED AND EXPERIMENTAL FUNCTIONS - **************************************************************************************** - * The definitions in the following section are considered experimental. - * They are provided for advanced scenarios. - * They should never be used with a dynamic library, as prototypes may change in the future. - * Use them only in association with static linking. - * ***************************************************************************************/ - -#if defined(ZSTD_STATIC_LINKING_ONLY) && !defined(ZSTD_H_ZSTD_STATIC_LINKING_ONLY) -#define ZSTD_H_ZSTD_STATIC_LINKING_ONLY - -/**************************************************************************************** - * experimental API (static linking only) - **************************************************************************************** - * The following symbols and constants - * are not planned to join "stable API" status in the near future. - * They can still change in future versions. - * Some of them are planned to remain in the static_only section indefinitely. - * Some of them might be removed in the future (especially when redundant with existing stable functions) - * ***************************************************************************************/ - -#define ZSTD_FRAMEHEADERSIZE_PREFIX(format) ((format) == ZSTD_f_zstd1 ? 5 : 1) /* minimum input size required to query frame header size */ -#define ZSTD_FRAMEHEADERSIZE_MIN(format) ((format) == ZSTD_f_zstd1 ? 6 : 2) -#define ZSTD_FRAMEHEADERSIZE_MAX 18 /* can be useful for static allocation */ -#define ZSTD_SKIPPABLEHEADERSIZE 8 - -/* compression parameter bounds */ -#define ZSTD_WINDOWLOG_MAX_32 30 -#define ZSTD_WINDOWLOG_MAX_64 31 -#define ZSTD_WINDOWLOG_MAX ((int)(sizeof(size_t) == 4 ? ZSTD_WINDOWLOG_MAX_32 : ZSTD_WINDOWLOG_MAX_64)) -#define ZSTD_WINDOWLOG_MIN 10 -#define ZSTD_HASHLOG_MAX ((ZSTD_WINDOWLOG_MAX < 30) ? ZSTD_WINDOWLOG_MAX : 30) -#define ZSTD_HASHLOG_MIN 6 -#define ZSTD_CHAINLOG_MAX_32 29 -#define ZSTD_CHAINLOG_MAX_64 30 -#define ZSTD_CHAINLOG_MAX ((int)(sizeof(size_t) == 4 ? ZSTD_CHAINLOG_MAX_32 : ZSTD_CHAINLOG_MAX_64)) -#define ZSTD_CHAINLOG_MIN ZSTD_HASHLOG_MIN -#define ZSTD_SEARCHLOG_MAX (ZSTD_WINDOWLOG_MAX-1) -#define ZSTD_SEARCHLOG_MIN 1 -#define ZSTD_MINMATCH_MAX 7 /* only for ZSTD_fast, other strategies are limited to 6 */ -#define ZSTD_MINMATCH_MIN 3 /* only for ZSTD_btopt+, faster strategies are limited to 4 */ -#define ZSTD_TARGETLENGTH_MAX ZSTD_BLOCKSIZE_MAX -#define ZSTD_TARGETLENGTH_MIN 0 /* note : comparing this constant to an unsigned results in a tautological test */ -#define ZSTD_STRATEGY_MIN ZSTD_fast -#define ZSTD_STRATEGY_MAX ZSTD_btultra2 - - -#define ZSTD_OVERLAPLOG_MIN 0 -#define ZSTD_OVERLAPLOG_MAX 9 - -#define ZSTD_WINDOWLOG_LIMIT_DEFAULT 27 /* by default, the streaming decoder will refuse any frame - * requiring larger than (1< 3, then this is seqDef.offset - 3 - * If seqDef.offset < 3, then this is the corresponding repeat offset - * But if seqDef.offset < 3 and litLength == 0, this is the - * repeat offset before the corresponding repeat offset - * And if seqDef.offset == 3 and litLength == 0, this is the - * most recent repeat offset - 1 - */ - unsigned int offset; - unsigned int litLength; /* Literal length */ - unsigned int matchLength; /* Match length */ - /* 0 when seq not rep and seqDef.offset otherwise - * when litLength == 0 this will be <= 4, otherwise <= 3 like normal - */ - unsigned int rep; -} ZSTD_Sequence; - -typedef struct { - unsigned windowLog; /**< largest match distance : larger == more compression, more memory needed during decompression */ - unsigned chainLog; /**< fully searched segment : larger == more compression, slower, more memory (useless for fast) */ - unsigned hashLog; /**< dispatch table : larger == faster, more memory */ - unsigned searchLog; /**< nb of searches : larger == more compression, slower */ - unsigned minMatch; /**< match length searched : larger == faster decompression, sometimes less compression */ - unsigned targetLength; /**< acceptable match size for optimal parser (only) : larger == more compression, slower */ - ZSTD_strategy strategy; /**< see ZSTD_strategy definition above */ -} ZSTD_compressionParameters; - -typedef struct { - int contentSizeFlag; /**< 1: content size will be in frame header (when known) */ - int checksumFlag; /**< 1: generate a 32-bits checksum using XXH64 algorithm at end of frame, for error detection */ - int noDictIDFlag; /**< 1: no dictID will be saved into frame header (dictID is only useful for dictionary compression) */ -} ZSTD_frameParameters; - -typedef struct { - ZSTD_compressionParameters cParams; - ZSTD_frameParameters fParams; -} ZSTD_parameters; - -typedef enum { - ZSTD_dct_auto = 0, /* dictionary is "full" when starting with ZSTD_MAGIC_DICTIONARY, otherwise it is "rawContent" */ - ZSTD_dct_rawContent = 1, /* ensures dictionary is always loaded as rawContent, even if it starts with ZSTD_MAGIC_DICTIONARY */ - ZSTD_dct_fullDict = 2 /* refuses to load a dictionary if it does not respect Zstandard's specification, starting with ZSTD_MAGIC_DICTIONARY */ -} ZSTD_dictContentType_e; - -typedef enum { - ZSTD_dlm_byCopy = 0, /**< Copy dictionary content internally */ - ZSTD_dlm_byRef = 1 /**< Reference dictionary content -- the dictionary buffer must outlive its users. */ -} ZSTD_dictLoadMethod_e; - -typedef enum { - ZSTD_f_zstd1 = 0, /* zstd frame format, specified in zstd_compression_format.md (default) */ - ZSTD_f_zstd1_magicless = 1 /* Variant of zstd frame format, without initial 4-bytes magic number. - * Useful to save 4 bytes per generated frame. - * Decoder cannot recognise automatically this format, requiring this instruction. */ -} ZSTD_format_e; - -typedef enum { - /* Note: this enum and the behavior it controls are effectively internal - * implementation details of the compressor. They are expected to continue - * to evolve and should be considered only in the context of extremely - * advanced performance tuning. - * - * Zstd currently supports the use of a CDict in three ways: - * - * - The contents of the CDict can be copied into the working context. This - * means that the compression can search both the dictionary and input - * while operating on a single set of internal tables. This makes - * the compression faster per-byte of input. However, the initial copy of - * the CDict's tables incurs a fixed cost at the beginning of the - * compression. For small compressions (< 8 KB), that copy can dominate - * the cost of the compression. - * - * - The CDict's tables can be used in-place. In this model, compression is - * slower per input byte, because the compressor has to search two sets of - * tables. However, this model incurs no start-up cost (as long as the - * working context's tables can be reused). For small inputs, this can be - * faster than copying the CDict's tables. - * - * - The CDict's tables are not used at all, and instead we use the working - * context alone to reload the dictionary and use params based on the source - * size. See ZSTD_compress_insertDictionary() and ZSTD_compress_usingDict(). - * This method is effective when the dictionary sizes are very small relative - * to the input size, and the input size is fairly large to begin with. - * - * Zstd has a simple internal heuristic that selects which strategy to use - * at the beginning of a compression. However, if experimentation shows that - * Zstd is making poor choices, it is possible to override that choice with - * this enum. - */ - ZSTD_dictDefaultAttach = 0, /* Use the default heuristic. */ - ZSTD_dictForceAttach = 1, /* Never copy the dictionary. */ - ZSTD_dictForceCopy = 2, /* Always copy the dictionary. */ - ZSTD_dictForceLoad = 3 /* Always reload the dictionary */ -} ZSTD_dictAttachPref_e; - -typedef enum { - ZSTD_lcm_auto = 0, /**< Automatically determine the compression mode based on the compression level. - * Negative compression levels will be uncompressed, and positive compression - * levels will be compressed. */ - ZSTD_lcm_huffman = 1, /**< Always attempt Huffman compression. Uncompressed literals will still be - * emitted if Huffman compression is not profitable. */ - ZSTD_lcm_uncompressed = 2 /**< Always emit uncompressed literals. */ -} ZSTD_literalCompressionMode_e; - - -/*************************************** -* Frame size functions -***************************************/ - -/*! ZSTD_findDecompressedSize() : - * `src` should point to the start of a series of ZSTD encoded and/or skippable frames - * `srcSize` must be the _exact_ size of this series - * (i.e. there should be a frame boundary at `src + srcSize`) - * @return : - decompressed size of all data in all successive frames - * - if the decompressed size cannot be determined: ZSTD_CONTENTSIZE_UNKNOWN - * - if an error occurred: ZSTD_CONTENTSIZE_ERROR - * - * note 1 : decompressed size is an optional field, that may not be present, especially in streaming mode. - * When `return==ZSTD_CONTENTSIZE_UNKNOWN`, data to decompress could be any size. - * In which case, it's necessary to use streaming mode to decompress data. - * note 2 : decompressed size is always present when compression is done with ZSTD_compress() - * note 3 : decompressed size can be very large (64-bits value), - * potentially larger than what local system can handle as a single memory segment. - * In which case, it's necessary to use streaming mode to decompress data. - * note 4 : If source is untrusted, decompressed size could be wrong or intentionally modified. - * Always ensure result fits within application's authorized limits. - * Each application can set its own limits. - * note 5 : ZSTD_findDecompressedSize handles multiple frames, and so it must traverse the input to - * read each contained frame header. This is fast as most of the data is skipped, - * however it does mean that all frame data must be present and valid. */ -ZSTDLIB_API unsigned long long ZSTD_findDecompressedSize(const void* src, size_t srcSize); - -/*! ZSTD_decompressBound() : - * `src` should point to the start of a series of ZSTD encoded and/or skippable frames - * `srcSize` must be the _exact_ size of this series - * (i.e. there should be a frame boundary at `src + srcSize`) - * @return : - upper-bound for the decompressed size of all data in all successive frames - * - if an error occured: ZSTD_CONTENTSIZE_ERROR - * - * note 1 : an error can occur if `src` contains an invalid or incorrectly formatted frame. - * note 2 : the upper-bound is exact when the decompressed size field is available in every ZSTD encoded frame of `src`. - * in this case, `ZSTD_findDecompressedSize` and `ZSTD_decompressBound` return the same value. - * note 3 : when the decompressed size field isn't available, the upper-bound for that frame is calculated by: - * upper-bound = # blocks * min(128 KB, Window_Size) - */ -ZSTDLIB_API unsigned long long ZSTD_decompressBound(const void* src, size_t srcSize); - -/*! ZSTD_frameHeaderSize() : - * srcSize must be >= ZSTD_FRAMEHEADERSIZE_PREFIX. - * @return : size of the Frame Header, - * or an error code (if srcSize is too small) */ -ZSTDLIB_API size_t ZSTD_frameHeaderSize(const void* src, size_t srcSize); - -/*! ZSTD_getSequences() : - * Extract sequences from the sequence store - * zc can be used to insert custom compression params. - * This function invokes ZSTD_compress2 - * @return : number of sequences extracted - */ -ZSTDLIB_API size_t ZSTD_getSequences(ZSTD_CCtx* zc, ZSTD_Sequence* outSeqs, - size_t outSeqsSize, const void* src, size_t srcSize); - - -/*************************************** -* Memory management -***************************************/ - -/*! ZSTD_estimate*() : - * These functions make it possible to estimate memory usage - * of a future {D,C}Ctx, before its creation. - * - * ZSTD_estimateCCtxSize() will provide a memory budget large enough - * for any compression level up to selected one. - * Note : Unlike ZSTD_estimateCStreamSize*(), this estimate - * does not include space for a window buffer. - * Therefore, the estimation is only guaranteed for single-shot compressions, not streaming. - * The estimate will assume the input may be arbitrarily large, - * which is the worst case. - * - * When srcSize can be bound by a known and rather "small" value, - * this fact can be used to provide a tighter estimation - * because the CCtx compression context will need less memory. - * This tighter estimation can be provided by more advanced functions - * ZSTD_estimateCCtxSize_usingCParams(), which can be used in tandem with ZSTD_getCParams(), - * and ZSTD_estimateCCtxSize_usingCCtxParams(), which can be used in tandem with ZSTD_CCtxParams_setParameter(). - * Both can be used to estimate memory using custom compression parameters and arbitrary srcSize limits. - * - * Note 2 : only single-threaded compression is supported. - * ZSTD_estimateCCtxSize_usingCCtxParams() will return an error code if ZSTD_c_nbWorkers is >= 1. - */ -ZSTDLIB_API size_t ZSTD_estimateCCtxSize(int compressionLevel); -ZSTDLIB_API size_t ZSTD_estimateCCtxSize_usingCParams(ZSTD_compressionParameters cParams); -ZSTDLIB_API size_t ZSTD_estimateCCtxSize_usingCCtxParams(const ZSTD_CCtx_params* params); -ZSTDLIB_API size_t ZSTD_estimateDCtxSize(void); - -/*! ZSTD_estimateCStreamSize() : - * ZSTD_estimateCStreamSize() will provide a budget large enough for any compression level up to selected one. - * It will also consider src size to be arbitrarily "large", which is worst case. - * If srcSize is known to always be small, ZSTD_estimateCStreamSize_usingCParams() can provide a tighter estimation. - * ZSTD_estimateCStreamSize_usingCParams() can be used in tandem with ZSTD_getCParams() to create cParams from compressionLevel. - * ZSTD_estimateCStreamSize_usingCCtxParams() can be used in tandem with ZSTD_CCtxParams_setParameter(). Only single-threaded compression is supported. This function will return an error code if ZSTD_c_nbWorkers is >= 1. - * Note : CStream size estimation is only correct for single-threaded compression. - * ZSTD_DStream memory budget depends on window Size. - * This information can be passed manually, using ZSTD_estimateDStreamSize, - * or deducted from a valid frame Header, using ZSTD_estimateDStreamSize_fromFrame(); - * Note : if streaming is init with function ZSTD_init?Stream_usingDict(), - * an internal ?Dict will be created, which additional size is not estimated here. - * In this case, get total size by adding ZSTD_estimate?DictSize */ -ZSTDLIB_API size_t ZSTD_estimateCStreamSize(int compressionLevel); -ZSTDLIB_API size_t ZSTD_estimateCStreamSize_usingCParams(ZSTD_compressionParameters cParams); -ZSTDLIB_API size_t ZSTD_estimateCStreamSize_usingCCtxParams(const ZSTD_CCtx_params* params); -ZSTDLIB_API size_t ZSTD_estimateDStreamSize(size_t windowSize); -ZSTDLIB_API size_t ZSTD_estimateDStreamSize_fromFrame(const void* src, size_t srcSize); - -/*! ZSTD_estimate?DictSize() : - * ZSTD_estimateCDictSize() will bet that src size is relatively "small", and content is copied, like ZSTD_createCDict(). - * ZSTD_estimateCDictSize_advanced() makes it possible to control compression parameters precisely, like ZSTD_createCDict_advanced(). - * Note : dictionaries created by reference (`ZSTD_dlm_byRef`) are logically smaller. - */ -ZSTDLIB_API size_t ZSTD_estimateCDictSize(size_t dictSize, int compressionLevel); -ZSTDLIB_API size_t ZSTD_estimateCDictSize_advanced(size_t dictSize, ZSTD_compressionParameters cParams, ZSTD_dictLoadMethod_e dictLoadMethod); -ZSTDLIB_API size_t ZSTD_estimateDDictSize(size_t dictSize, ZSTD_dictLoadMethod_e dictLoadMethod); - -/*! ZSTD_initStatic*() : - * Initialize an object using a pre-allocated fixed-size buffer. - * workspace: The memory area to emplace the object into. - * Provided pointer *must be 8-bytes aligned*. - * Buffer must outlive object. - * workspaceSize: Use ZSTD_estimate*Size() to determine - * how large workspace must be to support target scenario. - * @return : pointer to object (same address as workspace, just different type), - * or NULL if error (size too small, incorrect alignment, etc.) - * Note : zstd will never resize nor malloc() when using a static buffer. - * If the object requires more memory than available, - * zstd will just error out (typically ZSTD_error_memory_allocation). - * Note 2 : there is no corresponding "free" function. - * Since workspace is allocated externally, it must be freed externally too. - * Note 3 : cParams : use ZSTD_getCParams() to convert a compression level - * into its associated cParams. - * Limitation 1 : currently not compatible with internal dictionary creation, triggered by - * ZSTD_CCtx_loadDictionary(), ZSTD_initCStream_usingDict() or ZSTD_initDStream_usingDict(). - * Limitation 2 : static cctx currently not compatible with multi-threading. - * Limitation 3 : static dctx is incompatible with legacy support. - */ -ZSTDLIB_API ZSTD_CCtx* ZSTD_initStaticCCtx(void* workspace, size_t workspaceSize); -ZSTDLIB_API ZSTD_CStream* ZSTD_initStaticCStream(void* workspace, size_t workspaceSize); /**< same as ZSTD_initStaticCCtx() */ - -ZSTDLIB_API ZSTD_DCtx* ZSTD_initStaticDCtx(void* workspace, size_t workspaceSize); -ZSTDLIB_API ZSTD_DStream* ZSTD_initStaticDStream(void* workspace, size_t workspaceSize); /**< same as ZSTD_initStaticDCtx() */ - -ZSTDLIB_API const ZSTD_CDict* ZSTD_initStaticCDict( - void* workspace, size_t workspaceSize, - const void* dict, size_t dictSize, - ZSTD_dictLoadMethod_e dictLoadMethod, - ZSTD_dictContentType_e dictContentType, - ZSTD_compressionParameters cParams); - -ZSTDLIB_API const ZSTD_DDict* ZSTD_initStaticDDict( - void* workspace, size_t workspaceSize, - const void* dict, size_t dictSize, - ZSTD_dictLoadMethod_e dictLoadMethod, - ZSTD_dictContentType_e dictContentType); - - -/*! Custom memory allocation : - * These prototypes make it possible to pass your own allocation/free functions. - * ZSTD_customMem is provided at creation time, using ZSTD_create*_advanced() variants listed below. - * All allocation/free operations will be completed using these custom variants instead of regular ones. - */ -typedef void* (*ZSTD_allocFunction) (void* opaque, size_t size); -typedef void (*ZSTD_freeFunction) (void* opaque, void* address); -typedef struct { ZSTD_allocFunction customAlloc; ZSTD_freeFunction customFree; void* opaque; } ZSTD_customMem; -static ZSTD_customMem const ZSTD_defaultCMem = { NULL, NULL, NULL }; /**< this constant defers to stdlib's functions */ - -ZSTDLIB_API ZSTD_CCtx* ZSTD_createCCtx_advanced(ZSTD_customMem customMem); -ZSTDLIB_API ZSTD_CStream* ZSTD_createCStream_advanced(ZSTD_customMem customMem); -ZSTDLIB_API ZSTD_DCtx* ZSTD_createDCtx_advanced(ZSTD_customMem customMem); -ZSTDLIB_API ZSTD_DStream* ZSTD_createDStream_advanced(ZSTD_customMem customMem); - -ZSTDLIB_API ZSTD_CDict* ZSTD_createCDict_advanced(const void* dict, size_t dictSize, - ZSTD_dictLoadMethod_e dictLoadMethod, - ZSTD_dictContentType_e dictContentType, - ZSTD_compressionParameters cParams, - ZSTD_customMem customMem); - -ZSTDLIB_API ZSTD_DDict* ZSTD_createDDict_advanced(const void* dict, size_t dictSize, - ZSTD_dictLoadMethod_e dictLoadMethod, - ZSTD_dictContentType_e dictContentType, - ZSTD_customMem customMem); - - - -/*************************************** -* Advanced compression functions -***************************************/ - -/*! ZSTD_createCDict_byReference() : - * Create a digested dictionary for compression - * Dictionary content is just referenced, not duplicated. - * As a consequence, `dictBuffer` **must** outlive CDict, - * and its content must remain unmodified throughout the lifetime of CDict. - * note: equivalent to ZSTD_createCDict_advanced(), with dictLoadMethod==ZSTD_dlm_byRef */ -ZSTDLIB_API ZSTD_CDict* ZSTD_createCDict_byReference(const void* dictBuffer, size_t dictSize, int compressionLevel); - -/*! ZSTD_getCParams() : - * @return ZSTD_compressionParameters structure for a selected compression level and estimated srcSize. - * `estimatedSrcSize` value is optional, select 0 if not known */ -ZSTDLIB_API ZSTD_compressionParameters ZSTD_getCParams(int compressionLevel, unsigned long long estimatedSrcSize, size_t dictSize); - -/*! ZSTD_getParams() : - * same as ZSTD_getCParams(), but @return a full `ZSTD_parameters` object instead of sub-component `ZSTD_compressionParameters`. - * All fields of `ZSTD_frameParameters` are set to default : contentSize=1, checksum=0, noDictID=0 */ -ZSTDLIB_API ZSTD_parameters ZSTD_getParams(int compressionLevel, unsigned long long estimatedSrcSize, size_t dictSize); - -/*! ZSTD_checkCParams() : - * Ensure param values remain within authorized range. - * @return 0 on success, or an error code (can be checked with ZSTD_isError()) */ -ZSTDLIB_API size_t ZSTD_checkCParams(ZSTD_compressionParameters params); - -/*! ZSTD_adjustCParams() : - * optimize params for a given `srcSize` and `dictSize`. - * `srcSize` can be unknown, in which case use ZSTD_CONTENTSIZE_UNKNOWN. - * `dictSize` must be `0` when there is no dictionary. - * cPar can be invalid : all parameters will be clamped within valid range in the @return struct. - * This function never fails (wide contract) */ -ZSTDLIB_API ZSTD_compressionParameters ZSTD_adjustCParams(ZSTD_compressionParameters cPar, unsigned long long srcSize, size_t dictSize); - -/*! ZSTD_compress_advanced() : - * Note : this function is now DEPRECATED. - * It can be replaced by ZSTD_compress2(), in combination with ZSTD_CCtx_setParameter() and other parameter setters. - * This prototype will be marked as deprecated and generate compilation warning on reaching v1.5.x */ -ZSTDLIB_API size_t ZSTD_compress_advanced(ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const void* dict,size_t dictSize, - ZSTD_parameters params); - -/*! ZSTD_compress_usingCDict_advanced() : - * Note : this function is now REDUNDANT. - * It can be replaced by ZSTD_compress2(), in combination with ZSTD_CCtx_loadDictionary() and other parameter setters. - * This prototype will be marked as deprecated and generate compilation warning in some future version */ -ZSTDLIB_API size_t ZSTD_compress_usingCDict_advanced(ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const ZSTD_CDict* cdict, - ZSTD_frameParameters fParams); - - -/*! ZSTD_CCtx_loadDictionary_byReference() : - * Same as ZSTD_CCtx_loadDictionary(), but dictionary content is referenced, instead of being copied into CCtx. - * It saves some memory, but also requires that `dict` outlives its usage within `cctx` */ -ZSTDLIB_API size_t ZSTD_CCtx_loadDictionary_byReference(ZSTD_CCtx* cctx, const void* dict, size_t dictSize); - -/*! ZSTD_CCtx_loadDictionary_advanced() : - * Same as ZSTD_CCtx_loadDictionary(), but gives finer control over - * how to load the dictionary (by copy ? by reference ?) - * and how to interpret it (automatic ? force raw mode ? full mode only ?) */ -ZSTDLIB_API size_t ZSTD_CCtx_loadDictionary_advanced(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, ZSTD_dictLoadMethod_e dictLoadMethod, ZSTD_dictContentType_e dictContentType); - -/*! ZSTD_CCtx_refPrefix_advanced() : - * Same as ZSTD_CCtx_refPrefix(), but gives finer control over - * how to interpret prefix content (automatic ? force raw mode (default) ? full mode only ?) */ -ZSTDLIB_API size_t ZSTD_CCtx_refPrefix_advanced(ZSTD_CCtx* cctx, const void* prefix, size_t prefixSize, ZSTD_dictContentType_e dictContentType); - -/* === experimental parameters === */ -/* these parameters can be used with ZSTD_setParameter() - * they are not guaranteed to remain supported in the future */ - - /* Enables rsyncable mode, - * which makes compressed files more rsync friendly - * by adding periodic synchronization points to the compressed data. - * The target average block size is ZSTD_c_jobSize / 2. - * It's possible to modify the job size to increase or decrease - * the granularity of the synchronization point. - * Once the jobSize is smaller than the window size, - * it will result in compression ratio degradation. - * NOTE 1: rsyncable mode only works when multithreading is enabled. - * NOTE 2: rsyncable performs poorly in combination with long range mode, - * since it will decrease the effectiveness of synchronization points, - * though mileage may vary. - * NOTE 3: Rsyncable mode limits maximum compression speed to ~400 MB/s. - * If the selected compression level is already running significantly slower, - * the overall speed won't be significantly impacted. - */ - #define ZSTD_c_rsyncable ZSTD_c_experimentalParam1 - -/* Select a compression format. - * The value must be of type ZSTD_format_e. - * See ZSTD_format_e enum definition for details */ -#define ZSTD_c_format ZSTD_c_experimentalParam2 - -/* Force back-reference distances to remain < windowSize, - * even when referencing into Dictionary content (default:0) */ -#define ZSTD_c_forceMaxWindow ZSTD_c_experimentalParam3 - -/* Controls whether the contents of a CDict - * are used in place, or copied into the working context. - * Accepts values from the ZSTD_dictAttachPref_e enum. - * See the comments on that enum for an explanation of the feature. */ -#define ZSTD_c_forceAttachDict ZSTD_c_experimentalParam4 - -/* Controls how the literals are compressed (default is auto). - * The value must be of type ZSTD_literalCompressionMode_e. - * See ZSTD_literalCompressionMode_t enum definition for details. - */ -#define ZSTD_c_literalCompressionMode ZSTD_c_experimentalParam5 - -/* Tries to fit compressed block size to be around targetCBlockSize. - * No target when targetCBlockSize == 0. - * There is no guarantee on compressed block size (default:0) */ -#define ZSTD_c_targetCBlockSize ZSTD_c_experimentalParam6 - -/* User's best guess of source size. - * Hint is not valid when srcSizeHint == 0. - * There is no guarantee that hint is close to actual source size, - * but compression ratio may regress significantly if guess considerably underestimates */ -#define ZSTD_c_srcSizeHint ZSTD_c_experimentalParam7 - -/*! ZSTD_CCtx_getParameter() : - * Get the requested compression parameter value, selected by enum ZSTD_cParameter, - * and store it into int* value. - * @return : 0, or an error code (which can be tested with ZSTD_isError()). - */ -ZSTDLIB_API size_t ZSTD_CCtx_getParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param, int* value); - - -/*! ZSTD_CCtx_params : - * Quick howto : - * - ZSTD_createCCtxParams() : Create a ZSTD_CCtx_params structure - * - ZSTD_CCtxParams_setParameter() : Push parameters one by one into - * an existing ZSTD_CCtx_params structure. - * This is similar to - * ZSTD_CCtx_setParameter(). - * - ZSTD_CCtx_setParametersUsingCCtxParams() : Apply parameters to - * an existing CCtx. - * These parameters will be applied to - * all subsequent frames. - * - ZSTD_compressStream2() : Do compression using the CCtx. - * - ZSTD_freeCCtxParams() : Free the memory. - * - * This can be used with ZSTD_estimateCCtxSize_advanced_usingCCtxParams() - * for static allocation of CCtx for single-threaded compression. - */ -ZSTDLIB_API ZSTD_CCtx_params* ZSTD_createCCtxParams(void); -ZSTDLIB_API size_t ZSTD_freeCCtxParams(ZSTD_CCtx_params* params); - -/*! ZSTD_CCtxParams_reset() : - * Reset params to default values. - */ -ZSTDLIB_API size_t ZSTD_CCtxParams_reset(ZSTD_CCtx_params* params); - -/*! ZSTD_CCtxParams_init() : - * Initializes the compression parameters of cctxParams according to - * compression level. All other parameters are reset to their default values. - */ -ZSTDLIB_API size_t ZSTD_CCtxParams_init(ZSTD_CCtx_params* cctxParams, int compressionLevel); - -/*! ZSTD_CCtxParams_init_advanced() : - * Initializes the compression and frame parameters of cctxParams according to - * params. All other parameters are reset to their default values. - */ -ZSTDLIB_API size_t ZSTD_CCtxParams_init_advanced(ZSTD_CCtx_params* cctxParams, ZSTD_parameters params); - -/*! ZSTD_CCtxParams_setParameter() : - * Similar to ZSTD_CCtx_setParameter. - * Set one compression parameter, selected by enum ZSTD_cParameter. - * Parameters must be applied to a ZSTD_CCtx using ZSTD_CCtx_setParametersUsingCCtxParams(). - * @result : 0, or an error code (which can be tested with ZSTD_isError()). - */ -ZSTDLIB_API size_t ZSTD_CCtxParams_setParameter(ZSTD_CCtx_params* params, ZSTD_cParameter param, int value); - -/*! ZSTD_CCtxParams_getParameter() : - * Similar to ZSTD_CCtx_getParameter. - * Get the requested value of one compression parameter, selected by enum ZSTD_cParameter. - * @result : 0, or an error code (which can be tested with ZSTD_isError()). - */ -ZSTDLIB_API size_t ZSTD_CCtxParams_getParameter(ZSTD_CCtx_params* params, ZSTD_cParameter param, int* value); - -/*! ZSTD_CCtx_setParametersUsingCCtxParams() : - * Apply a set of ZSTD_CCtx_params to the compression context. - * This can be done even after compression is started, - * if nbWorkers==0, this will have no impact until a new compression is started. - * if nbWorkers>=1, new parameters will be picked up at next job, - * with a few restrictions (windowLog, pledgedSrcSize, nbWorkers, jobSize, and overlapLog are not updated). - */ -ZSTDLIB_API size_t ZSTD_CCtx_setParametersUsingCCtxParams( - ZSTD_CCtx* cctx, const ZSTD_CCtx_params* params); - -/*! ZSTD_compressStream2_simpleArgs() : - * Same as ZSTD_compressStream2(), - * but using only integral types as arguments. - * This variant might be helpful for binders from dynamic languages - * which have troubles handling structures containing memory pointers. - */ -ZSTDLIB_API size_t ZSTD_compressStream2_simpleArgs ( - ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, size_t* dstPos, - const void* src, size_t srcSize, size_t* srcPos, - ZSTD_EndDirective endOp); - - -/*************************************** -* Advanced decompression functions -***************************************/ - -/*! ZSTD_isFrame() : - * Tells if the content of `buffer` starts with a valid Frame Identifier. - * Note : Frame Identifier is 4 bytes. If `size < 4`, @return will always be 0. - * Note 2 : Legacy Frame Identifiers are considered valid only if Legacy Support is enabled. - * Note 3 : Skippable Frame Identifiers are considered valid. */ -ZSTDLIB_API unsigned ZSTD_isFrame(const void* buffer, size_t size); - -/*! ZSTD_createDDict_byReference() : - * Create a digested dictionary, ready to start decompression operation without startup delay. - * Dictionary content is referenced, and therefore stays in dictBuffer. - * It is important that dictBuffer outlives DDict, - * it must remain read accessible throughout the lifetime of DDict */ -ZSTDLIB_API ZSTD_DDict* ZSTD_createDDict_byReference(const void* dictBuffer, size_t dictSize); - -/*! ZSTD_DCtx_loadDictionary_byReference() : - * Same as ZSTD_DCtx_loadDictionary(), - * but references `dict` content instead of copying it into `dctx`. - * This saves memory if `dict` remains around., - * However, it's imperative that `dict` remains accessible (and unmodified) while being used, so it must outlive decompression. */ -ZSTDLIB_API size_t ZSTD_DCtx_loadDictionary_byReference(ZSTD_DCtx* dctx, const void* dict, size_t dictSize); - -/*! ZSTD_DCtx_loadDictionary_advanced() : - * Same as ZSTD_DCtx_loadDictionary(), - * but gives direct control over - * how to load the dictionary (by copy ? by reference ?) - * and how to interpret it (automatic ? force raw mode ? full mode only ?). */ -ZSTDLIB_API size_t ZSTD_DCtx_loadDictionary_advanced(ZSTD_DCtx* dctx, const void* dict, size_t dictSize, ZSTD_dictLoadMethod_e dictLoadMethod, ZSTD_dictContentType_e dictContentType); - -/*! ZSTD_DCtx_refPrefix_advanced() : - * Same as ZSTD_DCtx_refPrefix(), but gives finer control over - * how to interpret prefix content (automatic ? force raw mode (default) ? full mode only ?) */ -ZSTDLIB_API size_t ZSTD_DCtx_refPrefix_advanced(ZSTD_DCtx* dctx, const void* prefix, size_t prefixSize, ZSTD_dictContentType_e dictContentType); - -/*! ZSTD_DCtx_setMaxWindowSize() : - * Refuses allocating internal buffers for frames requiring a window size larger than provided limit. - * This protects a decoder context from reserving too much memory for itself (potential attack scenario). - * This parameter is only useful in streaming mode, since no internal buffer is allocated in single-pass mode. - * By default, a decompression context accepts all window sizes <= (1 << ZSTD_WINDOWLOG_LIMIT_DEFAULT) - * @return : 0, or an error code (which can be tested using ZSTD_isError()). - */ -ZSTDLIB_API size_t ZSTD_DCtx_setMaxWindowSize(ZSTD_DCtx* dctx, size_t maxWindowSize); - -/* ZSTD_d_format - * experimental parameter, - * allowing selection between ZSTD_format_e input compression formats - */ -#define ZSTD_d_format ZSTD_d_experimentalParam1 -/* ZSTD_d_stableOutBuffer - * Experimental parameter. - * Default is 0 == disabled. Set to 1 to enable. - * - * Tells the decompressor that the ZSTD_outBuffer will ALWAYS be the same - * between calls, except for the modifications that zstd makes to pos (the - * caller must not modify pos). This is checked by the decompressor, and - * decompression will fail if it ever changes. Therefore the ZSTD_outBuffer - * MUST be large enough to fit the entire decompressed frame. This will be - * checked when the frame content size is known. The data in the ZSTD_outBuffer - * in the range [dst, dst + pos) MUST not be modified during decompression - * or you will get data corruption. - * - * When this flags is enabled zstd won't allocate an output buffer, because - * it can write directly to the ZSTD_outBuffer, but it will still allocate - * an input buffer large enough to fit any compressed block. This will also - * avoid the memcpy() from the internal output buffer to the ZSTD_outBuffer. - * If you need to avoid the input buffer allocation use the buffer-less - * streaming API. - * - * NOTE: So long as the ZSTD_outBuffer always points to valid memory, using - * this flag is ALWAYS memory safe, and will never access out-of-bounds - * memory. However, decompression WILL fail if you violate the preconditions. - * - * WARNING: The data in the ZSTD_outBuffer in the range [dst, dst + pos) MUST - * not be modified during decompression or you will get data corruption. This - * is because zstd needs to reference data in the ZSTD_outBuffer to regenerate - * matches. Normally zstd maintains its own buffer for this purpose, but passing - * this flag tells zstd to use the user provided buffer. - */ -#define ZSTD_d_stableOutBuffer ZSTD_d_experimentalParam2 - -/*! ZSTD_DCtx_setFormat() : - * Instruct the decoder context about what kind of data to decode next. - * This instruction is mandatory to decode data without a fully-formed header, - * such ZSTD_f_zstd1_magicless for example. - * @return : 0, or an error code (which can be tested using ZSTD_isError()). */ -ZSTDLIB_API size_t ZSTD_DCtx_setFormat(ZSTD_DCtx* dctx, ZSTD_format_e format); - -/*! ZSTD_decompressStream_simpleArgs() : - * Same as ZSTD_decompressStream(), - * but using only integral types as arguments. - * This can be helpful for binders from dynamic languages - * which have troubles handling structures containing memory pointers. - */ -ZSTDLIB_API size_t ZSTD_decompressStream_simpleArgs ( - ZSTD_DCtx* dctx, - void* dst, size_t dstCapacity, size_t* dstPos, - const void* src, size_t srcSize, size_t* srcPos); - - -/******************************************************************** -* Advanced streaming functions -* Warning : most of these functions are now redundant with the Advanced API. -* Once Advanced API reaches "stable" status, -* redundant functions will be deprecated, and then at some point removed. -********************************************************************/ - -/*===== Advanced Streaming compression functions =====*/ -/**! ZSTD_initCStream_srcSize() : - * This function is deprecated, and equivalent to: - * ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only); - * ZSTD_CCtx_refCDict(zcs, NULL); // clear the dictionary (if any) - * ZSTD_CCtx_setParameter(zcs, ZSTD_c_compressionLevel, compressionLevel); - * ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize); - * - * pledgedSrcSize must be correct. If it is not known at init time, use - * ZSTD_CONTENTSIZE_UNKNOWN. Note that, for compatibility with older programs, - * "0" also disables frame content size field. It may be enabled in the future. - * Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x - */ -ZSTDLIB_API size_t -ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, - int compressionLevel, - unsigned long long pledgedSrcSize); - -/**! ZSTD_initCStream_usingDict() : - * This function is deprecated, and is equivalent to: - * ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only); - * ZSTD_CCtx_setParameter(zcs, ZSTD_c_compressionLevel, compressionLevel); - * ZSTD_CCtx_loadDictionary(zcs, dict, dictSize); - * - * Creates of an internal CDict (incompatible with static CCtx), except if - * dict == NULL or dictSize < 8, in which case no dict is used. - * Note: dict is loaded with ZSTD_dct_auto (treated as a full zstd dictionary if - * it begins with ZSTD_MAGIC_DICTIONARY, else as raw content) and ZSTD_dlm_byCopy. - * Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x - */ -ZSTDLIB_API size_t -ZSTD_initCStream_usingDict(ZSTD_CStream* zcs, - const void* dict, size_t dictSize, - int compressionLevel); - -/**! ZSTD_initCStream_advanced() : - * This function is deprecated, and is approximately equivalent to: - * ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only); - * // Pseudocode: Set each zstd parameter and leave the rest as-is. - * for ((param, value) : params) { - * ZSTD_CCtx_setParameter(zcs, param, value); - * } - * ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize); - * ZSTD_CCtx_loadDictionary(zcs, dict, dictSize); - * - * dict is loaded with ZSTD_dct_auto and ZSTD_dlm_byCopy. - * pledgedSrcSize must be correct. - * If srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. - * Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x - */ -ZSTDLIB_API size_t -ZSTD_initCStream_advanced(ZSTD_CStream* zcs, - const void* dict, size_t dictSize, - ZSTD_parameters params, - unsigned long long pledgedSrcSize); - -/**! ZSTD_initCStream_usingCDict() : - * This function is deprecated, and equivalent to: - * ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only); - * ZSTD_CCtx_refCDict(zcs, cdict); - * - * note : cdict will just be referenced, and must outlive compression session - * Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x - */ -ZSTDLIB_API size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict); - -/**! ZSTD_initCStream_usingCDict_advanced() : - * This function is DEPRECATED, and is approximately equivalent to: - * ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only); - * // Pseudocode: Set each zstd frame parameter and leave the rest as-is. - * for ((fParam, value) : fParams) { - * ZSTD_CCtx_setParameter(zcs, fParam, value); - * } - * ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize); - * ZSTD_CCtx_refCDict(zcs, cdict); - * - * same as ZSTD_initCStream_usingCDict(), with control over frame parameters. - * pledgedSrcSize must be correct. If srcSize is not known at init time, use - * value ZSTD_CONTENTSIZE_UNKNOWN. - * Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x - */ -ZSTDLIB_API size_t -ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs, - const ZSTD_CDict* cdict, - ZSTD_frameParameters fParams, - unsigned long long pledgedSrcSize); - -/*! ZSTD_resetCStream() : - * This function is deprecated, and is equivalent to: - * ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only); - * ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize); - * - * start a new frame, using same parameters from previous frame. - * This is typically useful to skip dictionary loading stage, since it will re-use it in-place. - * Note that zcs must be init at least once before using ZSTD_resetCStream(). - * If pledgedSrcSize is not known at reset time, use macro ZSTD_CONTENTSIZE_UNKNOWN. - * If pledgedSrcSize > 0, its value must be correct, as it will be written in header, and controlled at the end. - * For the time being, pledgedSrcSize==0 is interpreted as "srcSize unknown" for compatibility with older programs, - * but it will change to mean "empty" in future version, so use macro ZSTD_CONTENTSIZE_UNKNOWN instead. - * @return : 0, or an error code (which can be tested using ZSTD_isError()) - * Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x - */ -ZSTDLIB_API size_t ZSTD_resetCStream(ZSTD_CStream* zcs, unsigned long long pledgedSrcSize); - - -typedef struct { - unsigned long long ingested; /* nb input bytes read and buffered */ - unsigned long long consumed; /* nb input bytes actually compressed */ - unsigned long long produced; /* nb of compressed bytes generated and buffered */ - unsigned long long flushed; /* nb of compressed bytes flushed : not provided; can be tracked from caller side */ - unsigned currentJobID; /* MT only : latest started job nb */ - unsigned nbActiveWorkers; /* MT only : nb of workers actively compressing at probe time */ -} ZSTD_frameProgression; - -/* ZSTD_getFrameProgression() : - * tells how much data has been ingested (read from input) - * consumed (input actually compressed) and produced (output) for current frame. - * Note : (ingested - consumed) is amount of input data buffered internally, not yet compressed. - * Aggregates progression inside active worker threads. - */ -ZSTDLIB_API ZSTD_frameProgression ZSTD_getFrameProgression(const ZSTD_CCtx* cctx); - -/*! ZSTD_toFlushNow() : - * Tell how many bytes are ready to be flushed immediately. - * Useful for multithreading scenarios (nbWorkers >= 1). - * Probe the oldest active job, defined as oldest job not yet entirely flushed, - * and check its output buffer. - * @return : amount of data stored in oldest job and ready to be flushed immediately. - * if @return == 0, it means either : - * + there is no active job (could be checked with ZSTD_frameProgression()), or - * + oldest job is still actively compressing data, - * but everything it has produced has also been flushed so far, - * therefore flush speed is limited by production speed of oldest job - * irrespective of the speed of concurrent (and newer) jobs. - */ -ZSTDLIB_API size_t ZSTD_toFlushNow(ZSTD_CCtx* cctx); - - -/*===== Advanced Streaming decompression functions =====*/ -/** - * This function is deprecated, and is equivalent to: - * - * ZSTD_DCtx_reset(zds, ZSTD_reset_session_only); - * ZSTD_DCtx_loadDictionary(zds, dict, dictSize); - * - * note: no dictionary will be used if dict == NULL or dictSize < 8 - * Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x - */ -ZSTDLIB_API size_t ZSTD_initDStream_usingDict(ZSTD_DStream* zds, const void* dict, size_t dictSize); - -/** - * This function is deprecated, and is equivalent to: - * - * ZSTD_DCtx_reset(zds, ZSTD_reset_session_only); - * ZSTD_DCtx_refDDict(zds, ddict); - * - * note : ddict is referenced, it must outlive decompression session - * Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x - */ -ZSTDLIB_API size_t ZSTD_initDStream_usingDDict(ZSTD_DStream* zds, const ZSTD_DDict* ddict); - -/** - * This function is deprecated, and is equivalent to: - * - * ZSTD_DCtx_reset(zds, ZSTD_reset_session_only); - * - * re-use decompression parameters from previous init; saves dictionary loading - * Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x - */ -ZSTDLIB_API size_t ZSTD_resetDStream(ZSTD_DStream* zds); - - -/********************************************************************* -* Buffer-less and synchronous inner streaming functions -* -* This is an advanced API, giving full control over buffer management, for users which need direct control over memory. -* But it's also a complex one, with several restrictions, documented below. -* Prefer normal streaming API for an easier experience. -********************************************************************* */ - -/** - Buffer-less streaming compression (synchronous mode) - - A ZSTD_CCtx object is required to track streaming operations. - Use ZSTD_createCCtx() / ZSTD_freeCCtx() to manage resource. - ZSTD_CCtx object can be re-used multiple times within successive compression operations. - - Start by initializing a context. - Use ZSTD_compressBegin(), or ZSTD_compressBegin_usingDict() for dictionary compression, - or ZSTD_compressBegin_advanced(), for finer parameter control. - It's also possible to duplicate a reference context which has already been initialized, using ZSTD_copyCCtx() - - Then, consume your input using ZSTD_compressContinue(). - There are some important considerations to keep in mind when using this advanced function : - - ZSTD_compressContinue() has no internal buffer. It uses externally provided buffers only. - - Interface is synchronous : input is consumed entirely and produces 1+ compressed blocks. - - Caller must ensure there is enough space in `dst` to store compressed data under worst case scenario. - Worst case evaluation is provided by ZSTD_compressBound(). - ZSTD_compressContinue() doesn't guarantee recover after a failed compression. - - ZSTD_compressContinue() presumes prior input ***is still accessible and unmodified*** (up to maximum distance size, see WindowLog). - It remembers all previous contiguous blocks, plus one separated memory segment (which can itself consists of multiple contiguous blocks) - - ZSTD_compressContinue() detects that prior input has been overwritten when `src` buffer overlaps. - In which case, it will "discard" the relevant memory section from its history. - - Finish a frame with ZSTD_compressEnd(), which will write the last block(s) and optional checksum. - It's possible to use srcSize==0, in which case, it will write a final empty block to end the frame. - Without last block mark, frames are considered unfinished (hence corrupted) by compliant decoders. - - `ZSTD_CCtx` object can be re-used (ZSTD_compressBegin()) to compress again. -*/ - -/*===== Buffer-less streaming compression functions =====*/ -ZSTDLIB_API size_t ZSTD_compressBegin(ZSTD_CCtx* cctx, int compressionLevel); -ZSTDLIB_API size_t ZSTD_compressBegin_usingDict(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, int compressionLevel); -ZSTDLIB_API size_t ZSTD_compressBegin_advanced(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, ZSTD_parameters params, unsigned long long pledgedSrcSize); /**< pledgedSrcSize : If srcSize is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN */ -ZSTDLIB_API size_t ZSTD_compressBegin_usingCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict); /**< note: fails if cdict==NULL */ -ZSTDLIB_API size_t ZSTD_compressBegin_usingCDict_advanced(ZSTD_CCtx* const cctx, const ZSTD_CDict* const cdict, ZSTD_frameParameters const fParams, unsigned long long const pledgedSrcSize); /* compression parameters are already set within cdict. pledgedSrcSize must be correct. If srcSize is not known, use macro ZSTD_CONTENTSIZE_UNKNOWN */ -ZSTDLIB_API size_t ZSTD_copyCCtx(ZSTD_CCtx* cctx, const ZSTD_CCtx* preparedCCtx, unsigned long long pledgedSrcSize); /**< note: if pledgedSrcSize is not known, use ZSTD_CONTENTSIZE_UNKNOWN */ - -ZSTDLIB_API size_t ZSTD_compressContinue(ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize); -ZSTDLIB_API size_t ZSTD_compressEnd(ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize); - - -/*- - Buffer-less streaming decompression (synchronous mode) - - A ZSTD_DCtx object is required to track streaming operations. - Use ZSTD_createDCtx() / ZSTD_freeDCtx() to manage it. - A ZSTD_DCtx object can be re-used multiple times. - - First typical operation is to retrieve frame parameters, using ZSTD_getFrameHeader(). - Frame header is extracted from the beginning of compressed frame, so providing only the frame's beginning is enough. - Data fragment must be large enough to ensure successful decoding. - `ZSTD_frameHeaderSize_max` bytes is guaranteed to always be large enough. - @result : 0 : successful decoding, the `ZSTD_frameHeader` structure is correctly filled. - >0 : `srcSize` is too small, please provide at least @result bytes on next attempt. - errorCode, which can be tested using ZSTD_isError(). - - It fills a ZSTD_frameHeader structure with important information to correctly decode the frame, - such as the dictionary ID, content size, or maximum back-reference distance (`windowSize`). - Note that these values could be wrong, either because of data corruption, or because a 3rd party deliberately spoofs false information. - As a consequence, check that values remain within valid application range. - For example, do not allocate memory blindly, check that `windowSize` is within expectation. - Each application can set its own limits, depending on local restrictions. - For extended interoperability, it is recommended to support `windowSize` of at least 8 MB. - - ZSTD_decompressContinue() needs previous data blocks during decompression, up to `windowSize` bytes. - ZSTD_decompressContinue() is very sensitive to contiguity, - if 2 blocks don't follow each other, make sure that either the compressor breaks contiguity at the same place, - or that previous contiguous segment is large enough to properly handle maximum back-reference distance. - There are multiple ways to guarantee this condition. - - The most memory efficient way is to use a round buffer of sufficient size. - Sufficient size is determined by invoking ZSTD_decodingBufferSize_min(), - which can @return an error code if required value is too large for current system (in 32-bits mode). - In a round buffer methodology, ZSTD_decompressContinue() decompresses each block next to previous one, - up to the moment there is not enough room left in the buffer to guarantee decoding another full block, - which maximum size is provided in `ZSTD_frameHeader` structure, field `blockSizeMax`. - At which point, decoding can resume from the beginning of the buffer. - Note that already decoded data stored in the buffer should be flushed before being overwritten. - - There are alternatives possible, for example using two or more buffers of size `windowSize` each, though they consume more memory. - - Finally, if you control the compression process, you can also ignore all buffer size rules, - as long as the encoder and decoder progress in "lock-step", - aka use exactly the same buffer sizes, break contiguity at the same place, etc. - - Once buffers are setup, start decompression, with ZSTD_decompressBegin(). - If decompression requires a dictionary, use ZSTD_decompressBegin_usingDict() or ZSTD_decompressBegin_usingDDict(). - - Then use ZSTD_nextSrcSizeToDecompress() and ZSTD_decompressContinue() alternatively. - ZSTD_nextSrcSizeToDecompress() tells how many bytes to provide as 'srcSize' to ZSTD_decompressContinue(). - ZSTD_decompressContinue() requires this _exact_ amount of bytes, or it will fail. - - @result of ZSTD_decompressContinue() is the number of bytes regenerated within 'dst' (necessarily <= dstCapacity). - It can be zero : it just means ZSTD_decompressContinue() has decoded some metadata item. - It can also be an error code, which can be tested with ZSTD_isError(). - - A frame is fully decoded when ZSTD_nextSrcSizeToDecompress() returns zero. - Context can then be reset to start a new decompression. - - Note : it's possible to know if next input to present is a header or a block, using ZSTD_nextInputType(). - This information is not required to properly decode a frame. - - == Special case : skippable frames == - - Skippable frames allow integration of user-defined data into a flow of concatenated frames. - Skippable frames will be ignored (skipped) by decompressor. - The format of skippable frames is as follows : - a) Skippable frame ID - 4 Bytes, Little endian format, any value from 0x184D2A50 to 0x184D2A5F - b) Frame Size - 4 Bytes, Little endian format, unsigned 32-bits - c) Frame Content - any content (User Data) of length equal to Frame Size - For skippable frames ZSTD_getFrameHeader() returns zfhPtr->frameType==ZSTD_skippableFrame. - For skippable frames ZSTD_decompressContinue() always returns 0 : it only skips the content. -*/ - -/*===== Buffer-less streaming decompression functions =====*/ -typedef enum { ZSTD_frame, ZSTD_skippableFrame } ZSTD_frameType_e; -typedef struct { - unsigned long long frameContentSize; /* if == ZSTD_CONTENTSIZE_UNKNOWN, it means this field is not available. 0 means "empty" */ - unsigned long long windowSize; /* can be very large, up to <= frameContentSize */ - unsigned blockSizeMax; - ZSTD_frameType_e frameType; /* if == ZSTD_skippableFrame, frameContentSize is the size of skippable content */ - unsigned headerSize; - unsigned dictID; - unsigned checksumFlag; -} ZSTD_frameHeader; - -/*! ZSTD_getFrameHeader() : - * decode Frame Header, or requires larger `srcSize`. - * @return : 0, `zfhPtr` is correctly filled, - * >0, `srcSize` is too small, value is wanted `srcSize` amount, - * or an error code, which can be tested using ZSTD_isError() */ -ZSTDLIB_API size_t ZSTD_getFrameHeader(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize); /**< doesn't consume input */ -/*! ZSTD_getFrameHeader_advanced() : - * same as ZSTD_getFrameHeader(), - * with added capability to select a format (like ZSTD_f_zstd1_magicless) */ -ZSTDLIB_API size_t ZSTD_getFrameHeader_advanced(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize, ZSTD_format_e format); -ZSTDLIB_API size_t ZSTD_decodingBufferSize_min(unsigned long long windowSize, unsigned long long frameContentSize); /**< when frame content size is not known, pass in frameContentSize == ZSTD_CONTENTSIZE_UNKNOWN */ - -ZSTDLIB_API size_t ZSTD_decompressBegin(ZSTD_DCtx* dctx); -ZSTDLIB_API size_t ZSTD_decompressBegin_usingDict(ZSTD_DCtx* dctx, const void* dict, size_t dictSize); -ZSTDLIB_API size_t ZSTD_decompressBegin_usingDDict(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict); - -ZSTDLIB_API size_t ZSTD_nextSrcSizeToDecompress(ZSTD_DCtx* dctx); -ZSTDLIB_API size_t ZSTD_decompressContinue(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize); - -/* misc */ -ZSTDLIB_API void ZSTD_copyDCtx(ZSTD_DCtx* dctx, const ZSTD_DCtx* preparedDCtx); -typedef enum { ZSTDnit_frameHeader, ZSTDnit_blockHeader, ZSTDnit_block, ZSTDnit_lastBlock, ZSTDnit_checksum, ZSTDnit_skippableFrame } ZSTD_nextInputType_e; -ZSTDLIB_API ZSTD_nextInputType_e ZSTD_nextInputType(ZSTD_DCtx* dctx); - - - - -/* ============================ */ -/** Block level API */ -/* ============================ */ - -/*! - Block functions produce and decode raw zstd blocks, without frame metadata. - Frame metadata cost is typically ~12 bytes, which can be non-negligible for very small blocks (< 100 bytes). - But users will have to take in charge needed metadata to regenerate data, such as compressed and content sizes. - - A few rules to respect : - - Compressing and decompressing require a context structure - + Use ZSTD_createCCtx() and ZSTD_createDCtx() - - It is necessary to init context before starting - + compression : any ZSTD_compressBegin*() variant, including with dictionary - + decompression : any ZSTD_decompressBegin*() variant, including with dictionary - + copyCCtx() and copyDCtx() can be used too - - Block size is limited, it must be <= ZSTD_getBlockSize() <= ZSTD_BLOCKSIZE_MAX == 128 KB - + If input is larger than a block size, it's necessary to split input data into multiple blocks - + For inputs larger than a single block, consider using regular ZSTD_compress() instead. - Frame metadata is not that costly, and quickly becomes negligible as source size grows larger than a block. - - When a block is considered not compressible enough, ZSTD_compressBlock() result will be 0 (zero) ! - ===> In which case, nothing is produced into `dst` ! - + User __must__ test for such outcome and deal directly with uncompressed data - + A block cannot be declared incompressible if ZSTD_compressBlock() return value was != 0. - Doing so would mess up with statistics history, leading to potential data corruption. - + ZSTD_decompressBlock() _doesn't accept uncompressed data as input_ !! - + In case of multiple successive blocks, should some of them be uncompressed, - decoder must be informed of their existence in order to follow proper history. - Use ZSTD_insertBlock() for such a case. -*/ - -/*===== Raw zstd block functions =====*/ -ZSTDLIB_API size_t ZSTD_getBlockSize (const ZSTD_CCtx* cctx); -ZSTDLIB_API size_t ZSTD_compressBlock (ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize); -ZSTDLIB_API size_t ZSTD_decompressBlock(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize); -ZSTDLIB_API size_t ZSTD_insertBlock (ZSTD_DCtx* dctx, const void* blockStart, size_t blockSize); /**< insert uncompressed block into `dctx` history. Useful for multi-blocks decompression. */ - - -#endif /* ZSTD_H_ZSTD_STATIC_LINKING_ONLY */ - -#if defined (__cplusplus) -} -#endif -/**** ended inlining ../zstd.h ****/ -#define FSE_STATIC_LINKING_ONLY -/**** skipping file: fse.h ****/ -#define HUF_STATIC_LINKING_ONLY -/**** skipping file: huf.h ****/ -#ifndef XXH_STATIC_LINKING_ONLY -# define XXH_STATIC_LINKING_ONLY /* XXH64_state_t */ -#endif -/**** start inlining xxhash.h ****/ -/* - * xxHash - Extremely Fast Hash algorithm - * Header File - * Copyright (c) 2012-2020, Yann Collet, Facebook, Inc. - * - * You can contact the author at : - * - xxHash source repository : https://github.com/Cyan4973/xxHash - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. -*/ - -/* Notice extracted from xxHash homepage : - -xxHash is an extremely fast Hash algorithm, running at RAM speed limits. -It also successfully passes all tests from the SMHasher suite. - -Comparison (single thread, Windows Seven 32 bits, using SMHasher on a Core 2 Duo @3GHz) - -Name Speed Q.Score Author -xxHash 5.4 GB/s 10 -CrapWow 3.2 GB/s 2 Andrew -MumurHash 3a 2.7 GB/s 10 Austin Appleby -SpookyHash 2.0 GB/s 10 Bob Jenkins -SBox 1.4 GB/s 9 Bret Mulvey -Lookup3 1.2 GB/s 9 Bob Jenkins -SuperFastHash 1.2 GB/s 1 Paul Hsieh -CityHash64 1.05 GB/s 10 Pike & Alakuijala -FNV 0.55 GB/s 5 Fowler, Noll, Vo -CRC32 0.43 GB/s 9 -MD5-32 0.33 GB/s 10 Ronald L. Rivest -SHA1-32 0.28 GB/s 10 - -Q.Score is a measure of quality of the hash function. -It depends on successfully passing SMHasher test set. -10 is a perfect score. - -A 64-bits version, named XXH64, is available since r35. -It offers much better speed, but for 64-bits applications only. -Name Speed on 64 bits Speed on 32 bits -XXH64 13.8 GB/s 1.9 GB/s -XXH32 6.8 GB/s 6.0 GB/s -*/ - -#if defined (__cplusplus) -extern "C" { -#endif - -#ifndef XXHASH_H_5627135585666179 -#define XXHASH_H_5627135585666179 1 - - -/* **************************** -* Definitions -******************************/ -#include /* size_t */ -typedef enum { XXH_OK=0, XXH_ERROR } XXH_errorcode; - - -/* **************************** -* API modifier -******************************/ -/** XXH_PRIVATE_API -* This is useful if you want to include xxhash functions in `static` mode -* in order to inline them, and remove their symbol from the public list. -* Methodology : -* #define XXH_PRIVATE_API -* #include "xxhash.h" -* `xxhash.c` is automatically included. -* It's not useful to compile and link it as a separate module anymore. -*/ -#ifdef XXH_PRIVATE_API -# ifndef XXH_STATIC_LINKING_ONLY -# define XXH_STATIC_LINKING_ONLY -# endif -# if defined(__GNUC__) -# define XXH_PUBLIC_API static __inline __attribute__((unused)) -# elif defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) -# define XXH_PUBLIC_API static inline -# elif defined(_MSC_VER) -# define XXH_PUBLIC_API static __inline -# else -# define XXH_PUBLIC_API static /* this version may generate warnings for unused static functions; disable the relevant warning */ -# endif -#else -# define XXH_PUBLIC_API /* do nothing */ -#endif /* XXH_PRIVATE_API */ - -/*!XXH_NAMESPACE, aka Namespace Emulation : - -If you want to include _and expose_ xxHash functions from within your own library, -but also want to avoid symbol collisions with another library which also includes xxHash, - -you can use XXH_NAMESPACE, to automatically prefix any public symbol from xxhash library -with the value of XXH_NAMESPACE (so avoid to keep it NULL and avoid numeric values). - -Note that no change is required within the calling program as long as it includes `xxhash.h` : -regular symbol name will be automatically translated by this header. -*/ -#ifdef XXH_NAMESPACE -# define XXH_CAT(A,B) A##B -# define XXH_NAME2(A,B) XXH_CAT(A,B) -# define XXH32 XXH_NAME2(XXH_NAMESPACE, XXH32) -# define XXH64 XXH_NAME2(XXH_NAMESPACE, XXH64) -# define XXH_versionNumber XXH_NAME2(XXH_NAMESPACE, XXH_versionNumber) -# define XXH32_createState XXH_NAME2(XXH_NAMESPACE, XXH32_createState) -# define XXH64_createState XXH_NAME2(XXH_NAMESPACE, XXH64_createState) -# define XXH32_freeState XXH_NAME2(XXH_NAMESPACE, XXH32_freeState) -# define XXH64_freeState XXH_NAME2(XXH_NAMESPACE, XXH64_freeState) -# define XXH32_reset XXH_NAME2(XXH_NAMESPACE, XXH32_reset) -# define XXH64_reset XXH_NAME2(XXH_NAMESPACE, XXH64_reset) -# define XXH32_update XXH_NAME2(XXH_NAMESPACE, XXH32_update) -# define XXH64_update XXH_NAME2(XXH_NAMESPACE, XXH64_update) -# define XXH32_digest XXH_NAME2(XXH_NAMESPACE, XXH32_digest) -# define XXH64_digest XXH_NAME2(XXH_NAMESPACE, XXH64_digest) -# define XXH32_copyState XXH_NAME2(XXH_NAMESPACE, XXH32_copyState) -# define XXH64_copyState XXH_NAME2(XXH_NAMESPACE, XXH64_copyState) -# define XXH32_canonicalFromHash XXH_NAME2(XXH_NAMESPACE, XXH32_canonicalFromHash) -# define XXH64_canonicalFromHash XXH_NAME2(XXH_NAMESPACE, XXH64_canonicalFromHash) -# define XXH32_hashFromCanonical XXH_NAME2(XXH_NAMESPACE, XXH32_hashFromCanonical) -# define XXH64_hashFromCanonical XXH_NAME2(XXH_NAMESPACE, XXH64_hashFromCanonical) -#endif - - -/* ************************************* -* Version -***************************************/ -#define XXH_VERSION_MAJOR 0 -#define XXH_VERSION_MINOR 6 -#define XXH_VERSION_RELEASE 2 -#define XXH_VERSION_NUMBER (XXH_VERSION_MAJOR *100*100 + XXH_VERSION_MINOR *100 + XXH_VERSION_RELEASE) -XXH_PUBLIC_API unsigned XXH_versionNumber (void); - - -/* **************************** -* Simple Hash Functions -******************************/ -typedef unsigned int XXH32_hash_t; -typedef unsigned long long XXH64_hash_t; - -XXH_PUBLIC_API XXH32_hash_t XXH32 (const void* input, size_t length, unsigned int seed); -XXH_PUBLIC_API XXH64_hash_t XXH64 (const void* input, size_t length, unsigned long long seed); - -/*! -XXH32() : - Calculate the 32-bits hash of sequence "length" bytes stored at memory address "input". - The memory between input & input+length must be valid (allocated and read-accessible). - "seed" can be used to alter the result predictably. - Speed on Core 2 Duo @ 3 GHz (single thread, SMHasher benchmark) : 5.4 GB/s -XXH64() : - Calculate the 64-bits hash of sequence of length "len" stored at memory address "input". - "seed" can be used to alter the result predictably. - This function runs 2x faster on 64-bits systems, but slower on 32-bits systems (see benchmark). -*/ - - -/* **************************** -* Streaming Hash Functions -******************************/ -typedef struct XXH32_state_s XXH32_state_t; /* incomplete type */ -typedef struct XXH64_state_s XXH64_state_t; /* incomplete type */ - -/*! State allocation, compatible with dynamic libraries */ - -XXH_PUBLIC_API XXH32_state_t* XXH32_createState(void); -XXH_PUBLIC_API XXH_errorcode XXH32_freeState(XXH32_state_t* statePtr); - -XXH_PUBLIC_API XXH64_state_t* XXH64_createState(void); -XXH_PUBLIC_API XXH_errorcode XXH64_freeState(XXH64_state_t* statePtr); - - -/* hash streaming */ - -XXH_PUBLIC_API XXH_errorcode XXH32_reset (XXH32_state_t* statePtr, unsigned int seed); -XXH_PUBLIC_API XXH_errorcode XXH32_update (XXH32_state_t* statePtr, const void* input, size_t length); -XXH_PUBLIC_API XXH32_hash_t XXH32_digest (const XXH32_state_t* statePtr); - -XXH_PUBLIC_API XXH_errorcode XXH64_reset (XXH64_state_t* statePtr, unsigned long long seed); -XXH_PUBLIC_API XXH_errorcode XXH64_update (XXH64_state_t* statePtr, const void* input, size_t length); -XXH_PUBLIC_API XXH64_hash_t XXH64_digest (const XXH64_state_t* statePtr); - -/* -These functions generate the xxHash of an input provided in multiple segments. -Note that, for small input, they are slower than single-call functions, due to state management. -For small input, prefer `XXH32()` and `XXH64()` . - -XXH state must first be allocated, using XXH*_createState() . - -Start a new hash by initializing state with a seed, using XXH*_reset(). - -Then, feed the hash state by calling XXH*_update() as many times as necessary. -Obviously, input must be allocated and read accessible. -The function returns an error code, with 0 meaning OK, and any other value meaning there is an error. - -Finally, a hash value can be produced anytime, by using XXH*_digest(). -This function returns the nn-bits hash as an int or long long. - -It's still possible to continue inserting input into the hash state after a digest, -and generate some new hashes later on, by calling again XXH*_digest(). - -When done, free XXH state space if it was allocated dynamically. -*/ - - -/* ************************** -* Utils -****************************/ -#if !(defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L)) /* ! C99 */ -# define restrict /* disable restrict */ -#endif - -XXH_PUBLIC_API void XXH32_copyState(XXH32_state_t* restrict dst_state, const XXH32_state_t* restrict src_state); -XXH_PUBLIC_API void XXH64_copyState(XXH64_state_t* restrict dst_state, const XXH64_state_t* restrict src_state); - - -/* ************************** -* Canonical representation -****************************/ -/* Default result type for XXH functions are primitive unsigned 32 and 64 bits. -* The canonical representation uses human-readable write convention, aka big-endian (large digits first). -* These functions allow transformation of hash result into and from its canonical format. -* This way, hash values can be written into a file / memory, and remain comparable on different systems and programs. -*/ -typedef struct { unsigned char digest[4]; } XXH32_canonical_t; -typedef struct { unsigned char digest[8]; } XXH64_canonical_t; - -XXH_PUBLIC_API void XXH32_canonicalFromHash(XXH32_canonical_t* dst, XXH32_hash_t hash); -XXH_PUBLIC_API void XXH64_canonicalFromHash(XXH64_canonical_t* dst, XXH64_hash_t hash); - -XXH_PUBLIC_API XXH32_hash_t XXH32_hashFromCanonical(const XXH32_canonical_t* src); -XXH_PUBLIC_API XXH64_hash_t XXH64_hashFromCanonical(const XXH64_canonical_t* src); - -#endif /* XXHASH_H_5627135585666179 */ - - - -/* ================================================================================================ - This section contains definitions which are not guaranteed to remain stable. - They may change in future versions, becoming incompatible with a different version of the library. - They shall only be used with static linking. - Never use these definitions in association with dynamic linking ! -=================================================================================================== */ -#if defined(XXH_STATIC_LINKING_ONLY) && !defined(XXH_STATIC_H_3543687687345) -#define XXH_STATIC_H_3543687687345 - -/* These definitions are only meant to allow allocation of XXH state - statically, on stack, or in a struct for example. - Do not use members directly. */ - - struct XXH32_state_s { - unsigned total_len_32; - unsigned large_len; - unsigned v1; - unsigned v2; - unsigned v3; - unsigned v4; - unsigned mem32[4]; /* buffer defined as U32 for alignment */ - unsigned memsize; - unsigned reserved; /* never read nor write, will be removed in a future version */ - }; /* typedef'd to XXH32_state_t */ - - struct XXH64_state_s { - unsigned long long total_len; - unsigned long long v1; - unsigned long long v2; - unsigned long long v3; - unsigned long long v4; - unsigned long long mem64[4]; /* buffer defined as U64 for alignment */ - unsigned memsize; - unsigned reserved[2]; /* never read nor write, will be removed in a future version */ - }; /* typedef'd to XXH64_state_t */ - - -# ifdef XXH_PRIVATE_API -/**** start inlining xxhash.c ****/ -/* - * xxHash - Fast Hash algorithm - * Copyright (c) 2012-2020, Yann Collet, Facebook, Inc. - * - * You can contact the author at : - * - xxHash homepage: http://www.xxhash.com - * - xxHash source repository : https://github.com/Cyan4973/xxHash - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. -*/ - - -/* ************************************* -* Tuning parameters -***************************************/ -/*!XXH_FORCE_MEMORY_ACCESS : - * By default, access to unaligned memory is controlled by `memcpy()`, which is safe and portable. - * Unfortunately, on some target/compiler combinations, the generated assembly is sub-optimal. - * The below switch allow to select different access method for improved performance. - * Method 0 (default) : use `memcpy()`. Safe and portable. - * Method 1 : `__packed` statement. It depends on compiler extension (ie, not portable). - * This method is safe if your compiler supports it, and *generally* as fast or faster than `memcpy`. - * Method 2 : direct access. This method doesn't depend on compiler but violate C standard. - * It can generate buggy code on targets which do not support unaligned memory accesses. - * But in some circumstances, it's the only known way to get the most performance (ie GCC + ARMv6) - * See http://stackoverflow.com/a/32095106/646947 for details. - * Prefer these methods in priority order (0 > 1 > 2) - */ -#ifndef XXH_FORCE_MEMORY_ACCESS /* can be defined externally, on command line for example */ -# if defined(__GNUC__) && ( defined(__ARM_ARCH_6__) || defined(__ARM_ARCH_6J__) || defined(__ARM_ARCH_6K__) || defined(__ARM_ARCH_6Z__) || defined(__ARM_ARCH_6ZK__) || defined(__ARM_ARCH_6T2__) ) -# define XXH_FORCE_MEMORY_ACCESS 2 -# elif (defined(__INTEL_COMPILER) && !defined(WIN32)) || \ - (defined(__GNUC__) && ( defined(__ARM_ARCH_7__) || defined(__ARM_ARCH_7A__) || defined(__ARM_ARCH_7R__) || defined(__ARM_ARCH_7M__) || defined(__ARM_ARCH_7S__) )) || \ - defined(__ICCARM__) -# define XXH_FORCE_MEMORY_ACCESS 1 -# endif -#endif - -/*!XXH_ACCEPT_NULL_INPUT_POINTER : - * If the input pointer is a null pointer, xxHash default behavior is to trigger a memory access error, since it is a bad pointer. - * When this option is enabled, xxHash output for null input pointers will be the same as a null-length input. - * By default, this option is disabled. To enable it, uncomment below define : - */ -/* #define XXH_ACCEPT_NULL_INPUT_POINTER 1 */ - -/*!XXH_FORCE_NATIVE_FORMAT : - * By default, xxHash library provides endian-independent Hash values, based on little-endian convention. - * Results are therefore identical for little-endian and big-endian CPU. - * This comes at a performance cost for big-endian CPU, since some swapping is required to emulate little-endian format. - * Should endian-independence be of no importance for your application, you may set the #define below to 1, - * to improve speed for Big-endian CPU. - * This option has no impact on Little_Endian CPU. - */ -#ifndef XXH_FORCE_NATIVE_FORMAT /* can be defined externally */ -# define XXH_FORCE_NATIVE_FORMAT 0 -#endif - -/*!XXH_FORCE_ALIGN_CHECK : - * This is a minor performance trick, only useful with lots of very small keys. - * It means : check for aligned/unaligned input. - * The check costs one initial branch per hash; set to 0 when the input data - * is guaranteed to be aligned. - */ -#ifndef XXH_FORCE_ALIGN_CHECK /* can be defined externally */ -# if defined(__i386) || defined(_M_IX86) || defined(__x86_64__) || defined(_M_X64) -# define XXH_FORCE_ALIGN_CHECK 0 -# else -# define XXH_FORCE_ALIGN_CHECK 1 -# endif -#endif - - -/* ************************************* -* Includes & Memory related functions -***************************************/ -/* Modify the local functions below should you wish to use some other memory routines */ -/* for malloc(), free() */ -#include -#include /* size_t */ -static void* XXH_malloc(size_t s) { return malloc(s); } -static void XXH_free (void* p) { free(p); } -/* for memcpy() */ -#include -static void* XXH_memcpy(void* dest, const void* src, size_t size) { return memcpy(dest,src,size); } - -#ifndef XXH_STATIC_LINKING_ONLY -# define XXH_STATIC_LINKING_ONLY -#endif -/**** skipping file: xxhash.h ****/ - - -/* ************************************* -* Compiler Specific Options -***************************************/ -#if (defined(__GNUC__) && !defined(__STRICT_ANSI__)) || defined(__cplusplus) || defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L /* C99 */ -# define INLINE_KEYWORD inline -#else -# define INLINE_KEYWORD -#endif - -#if defined(__GNUC__) || defined(__ICCARM__) -# define FORCE_INLINE_ATTR __attribute__((always_inline)) -#elif defined(_MSC_VER) -# define FORCE_INLINE_ATTR __forceinline -#else -# define FORCE_INLINE_ATTR -#endif - -#define FORCE_INLINE_TEMPLATE static INLINE_KEYWORD FORCE_INLINE_ATTR - - -#ifdef _MSC_VER -# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */ -#endif - - -/* ************************************* -* Basic Types -***************************************/ -#ifndef MEM_MODULE -# define MEM_MODULE -# if !defined (__VMS) && (defined (__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) ) -# include - typedef uint8_t BYTE; - typedef uint16_t U16; - typedef uint32_t U32; - typedef int32_t S32; - typedef uint64_t U64; -# else - typedef unsigned char BYTE; - typedef unsigned short U16; - typedef unsigned int U32; - typedef signed int S32; - typedef unsigned long long U64; /* if your compiler doesn't support unsigned long long, replace by another 64-bit type here. Note that xxhash.h will also need to be updated. */ -# endif -#endif - - -#if (defined(XXH_FORCE_MEMORY_ACCESS) && (XXH_FORCE_MEMORY_ACCESS==2)) - -/* Force direct memory access. Only works on CPU which support unaligned memory access in hardware */ -static U32 XXH_read32(const void* memPtr) { return *(const U32*) memPtr; } -static U64 XXH_read64(const void* memPtr) { return *(const U64*) memPtr; } - -#elif (defined(XXH_FORCE_MEMORY_ACCESS) && (XXH_FORCE_MEMORY_ACCESS==1)) - -/* __pack instructions are safer, but compiler specific, hence potentially problematic for some compilers */ -/* currently only defined for gcc and icc */ -typedef union { U32 u32; U64 u64; } __attribute__((packed)) unalign; - -static U32 XXH_read32(const void* ptr) { return ((const unalign*)ptr)->u32; } -static U64 XXH_read64(const void* ptr) { return ((const unalign*)ptr)->u64; } - -#else - -/* portable and safe solution. Generally efficient. - * see : http://stackoverflow.com/a/32095106/646947 - */ - -static U32 XXH_read32(const void* memPtr) -{ - U32 val; - memcpy(&val, memPtr, sizeof(val)); - return val; -} - -static U64 XXH_read64(const void* memPtr) -{ - U64 val; - memcpy(&val, memPtr, sizeof(val)); - return val; -} - -#endif /* XXH_FORCE_DIRECT_MEMORY_ACCESS */ - - -/* **************************************** -* Compiler-specific Functions and Macros -******************************************/ -#define GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__) - -/* Note : although _rotl exists for minGW (GCC under windows), performance seems poor */ -#if defined(_MSC_VER) -# define XXH_rotl32(x,r) _rotl(x,r) -# define XXH_rotl64(x,r) _rotl64(x,r) -#else -#if defined(__ICCARM__) -# include -# define XXH_rotl32(x,r) __ROR(x,(32 - r)) -#else -# define XXH_rotl32(x,r) ((x << r) | (x >> (32 - r))) -#endif -# define XXH_rotl64(x,r) ((x << r) | (x >> (64 - r))) -#endif - -#if defined(_MSC_VER) /* Visual Studio */ -# define XXH_swap32 _byteswap_ulong -# define XXH_swap64 _byteswap_uint64 -#elif GCC_VERSION >= 403 -# define XXH_swap32 __builtin_bswap32 -# define XXH_swap64 __builtin_bswap64 -#else -static U32 XXH_swap32 (U32 x) -{ - return ((x << 24) & 0xff000000 ) | - ((x << 8) & 0x00ff0000 ) | - ((x >> 8) & 0x0000ff00 ) | - ((x >> 24) & 0x000000ff ); -} -static U64 XXH_swap64 (U64 x) -{ - return ((x << 56) & 0xff00000000000000ULL) | - ((x << 40) & 0x00ff000000000000ULL) | - ((x << 24) & 0x0000ff0000000000ULL) | - ((x << 8) & 0x000000ff00000000ULL) | - ((x >> 8) & 0x00000000ff000000ULL) | - ((x >> 24) & 0x0000000000ff0000ULL) | - ((x >> 40) & 0x000000000000ff00ULL) | - ((x >> 56) & 0x00000000000000ffULL); -} -#endif - - -/* ************************************* -* Architecture Macros -***************************************/ -typedef enum { XXH_bigEndian=0, XXH_littleEndian=1 } XXH_endianess; - -/* XXH_CPU_LITTLE_ENDIAN can be defined externally, for example on the compiler command line */ -#ifndef XXH_CPU_LITTLE_ENDIAN - static const int g_one = 1; -# define XXH_CPU_LITTLE_ENDIAN (*(const char*)(&g_one)) -#endif - - -/* *************************** -* Memory reads -*****************************/ -typedef enum { XXH_aligned, XXH_unaligned } XXH_alignment; - -FORCE_INLINE_TEMPLATE U32 XXH_readLE32_align(const void* ptr, XXH_endianess endian, XXH_alignment align) -{ - if (align==XXH_unaligned) - return endian==XXH_littleEndian ? XXH_read32(ptr) : XXH_swap32(XXH_read32(ptr)); - else - return endian==XXH_littleEndian ? *(const U32*)ptr : XXH_swap32(*(const U32*)ptr); -} - -FORCE_INLINE_TEMPLATE U32 XXH_readLE32(const void* ptr, XXH_endianess endian) -{ - return XXH_readLE32_align(ptr, endian, XXH_unaligned); -} - -static U32 XXH_readBE32(const void* ptr) -{ - return XXH_CPU_LITTLE_ENDIAN ? XXH_swap32(XXH_read32(ptr)) : XXH_read32(ptr); -} - -FORCE_INLINE_TEMPLATE U64 XXH_readLE64_align(const void* ptr, XXH_endianess endian, XXH_alignment align) -{ - if (align==XXH_unaligned) - return endian==XXH_littleEndian ? XXH_read64(ptr) : XXH_swap64(XXH_read64(ptr)); - else - return endian==XXH_littleEndian ? *(const U64*)ptr : XXH_swap64(*(const U64*)ptr); -} - -FORCE_INLINE_TEMPLATE U64 XXH_readLE64(const void* ptr, XXH_endianess endian) -{ - return XXH_readLE64_align(ptr, endian, XXH_unaligned); -} - -static U64 XXH_readBE64(const void* ptr) -{ - return XXH_CPU_LITTLE_ENDIAN ? XXH_swap64(XXH_read64(ptr)) : XXH_read64(ptr); -} - - -/* ************************************* -* Macros -***************************************/ -#define XXH_STATIC_ASSERT(c) { enum { XXH_static_assert = 1/(int)(!!(c)) }; } /* use only *after* variable declarations */ - - -/* ************************************* -* Constants -***************************************/ -static const U32 PRIME32_1 = 2654435761U; -static const U32 PRIME32_2 = 2246822519U; -static const U32 PRIME32_3 = 3266489917U; -static const U32 PRIME32_4 = 668265263U; -static const U32 PRIME32_5 = 374761393U; - -static const U64 PRIME64_1 = 11400714785074694791ULL; -static const U64 PRIME64_2 = 14029467366897019727ULL; -static const U64 PRIME64_3 = 1609587929392839161ULL; -static const U64 PRIME64_4 = 9650029242287828579ULL; -static const U64 PRIME64_5 = 2870177450012600261ULL; - -XXH_PUBLIC_API unsigned XXH_versionNumber (void) { return XXH_VERSION_NUMBER; } - - -/* ************************** -* Utils -****************************/ -XXH_PUBLIC_API void XXH32_copyState(XXH32_state_t* restrict dstState, const XXH32_state_t* restrict srcState) -{ - memcpy(dstState, srcState, sizeof(*dstState)); -} - -XXH_PUBLIC_API void XXH64_copyState(XXH64_state_t* restrict dstState, const XXH64_state_t* restrict srcState) -{ - memcpy(dstState, srcState, sizeof(*dstState)); -} - - -/* *************************** -* Simple Hash Functions -*****************************/ - -static U32 XXH32_round(U32 seed, U32 input) -{ - seed += input * PRIME32_2; - seed = XXH_rotl32(seed, 13); - seed *= PRIME32_1; - return seed; -} - -FORCE_INLINE_TEMPLATE U32 XXH32_endian_align(const void* input, size_t len, U32 seed, XXH_endianess endian, XXH_alignment align) -{ - const BYTE* p = (const BYTE*)input; - const BYTE* bEnd = p + len; - U32 h32; -#define XXH_get32bits(p) XXH_readLE32_align(p, endian, align) - -#ifdef XXH_ACCEPT_NULL_INPUT_POINTER - if (p==NULL) { - len=0; - bEnd=p=(const BYTE*)(size_t)16; - } -#endif - - if (len>=16) { - const BYTE* const limit = bEnd - 16; - U32 v1 = seed + PRIME32_1 + PRIME32_2; - U32 v2 = seed + PRIME32_2; - U32 v3 = seed + 0; - U32 v4 = seed - PRIME32_1; - - do { - v1 = XXH32_round(v1, XXH_get32bits(p)); p+=4; - v2 = XXH32_round(v2, XXH_get32bits(p)); p+=4; - v3 = XXH32_round(v3, XXH_get32bits(p)); p+=4; - v4 = XXH32_round(v4, XXH_get32bits(p)); p+=4; - } while (p<=limit); - - h32 = XXH_rotl32(v1, 1) + XXH_rotl32(v2, 7) + XXH_rotl32(v3, 12) + XXH_rotl32(v4, 18); - } else { - h32 = seed + PRIME32_5; - } - - h32 += (U32) len; - - while (p+4<=bEnd) { - h32 += XXH_get32bits(p) * PRIME32_3; - h32 = XXH_rotl32(h32, 17) * PRIME32_4 ; - p+=4; - } - - while (p> 15; - h32 *= PRIME32_2; - h32 ^= h32 >> 13; - h32 *= PRIME32_3; - h32 ^= h32 >> 16; - - return h32; -} - - -XXH_PUBLIC_API unsigned int XXH32 (const void* input, size_t len, unsigned int seed) -{ -#if 0 - /* Simple version, good for code maintenance, but unfortunately slow for small inputs */ - XXH32_CREATESTATE_STATIC(state); - XXH32_reset(state, seed); - XXH32_update(state, input, len); - return XXH32_digest(state); -#else - XXH_endianess endian_detected = (XXH_endianess)XXH_CPU_LITTLE_ENDIAN; - - if (XXH_FORCE_ALIGN_CHECK) { - if ((((size_t)input) & 3) == 0) { /* Input is 4-bytes aligned, leverage the speed benefit */ - if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) - return XXH32_endian_align(input, len, seed, XXH_littleEndian, XXH_aligned); - else - return XXH32_endian_align(input, len, seed, XXH_bigEndian, XXH_aligned); - } } - - if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) - return XXH32_endian_align(input, len, seed, XXH_littleEndian, XXH_unaligned); - else - return XXH32_endian_align(input, len, seed, XXH_bigEndian, XXH_unaligned); -#endif -} - - -static U64 XXH64_round(U64 acc, U64 input) -{ - acc += input * PRIME64_2; - acc = XXH_rotl64(acc, 31); - acc *= PRIME64_1; - return acc; -} - -static U64 XXH64_mergeRound(U64 acc, U64 val) -{ - val = XXH64_round(0, val); - acc ^= val; - acc = acc * PRIME64_1 + PRIME64_4; - return acc; -} - -FORCE_INLINE_TEMPLATE U64 XXH64_endian_align(const void* input, size_t len, U64 seed, XXH_endianess endian, XXH_alignment align) -{ - const BYTE* p = (const BYTE*)input; - const BYTE* const bEnd = p + len; - U64 h64; -#define XXH_get64bits(p) XXH_readLE64_align(p, endian, align) - -#ifdef XXH_ACCEPT_NULL_INPUT_POINTER - if (p==NULL) { - len=0; - bEnd=p=(const BYTE*)(size_t)32; - } -#endif - - if (len>=32) { - const BYTE* const limit = bEnd - 32; - U64 v1 = seed + PRIME64_1 + PRIME64_2; - U64 v2 = seed + PRIME64_2; - U64 v3 = seed + 0; - U64 v4 = seed - PRIME64_1; - - do { - v1 = XXH64_round(v1, XXH_get64bits(p)); p+=8; - v2 = XXH64_round(v2, XXH_get64bits(p)); p+=8; - v3 = XXH64_round(v3, XXH_get64bits(p)); p+=8; - v4 = XXH64_round(v4, XXH_get64bits(p)); p+=8; - } while (p<=limit); - - h64 = XXH_rotl64(v1, 1) + XXH_rotl64(v2, 7) + XXH_rotl64(v3, 12) + XXH_rotl64(v4, 18); - h64 = XXH64_mergeRound(h64, v1); - h64 = XXH64_mergeRound(h64, v2); - h64 = XXH64_mergeRound(h64, v3); - h64 = XXH64_mergeRound(h64, v4); - - } else { - h64 = seed + PRIME64_5; - } - - h64 += (U64) len; - - while (p+8<=bEnd) { - U64 const k1 = XXH64_round(0, XXH_get64bits(p)); - h64 ^= k1; - h64 = XXH_rotl64(h64,27) * PRIME64_1 + PRIME64_4; - p+=8; - } - - if (p+4<=bEnd) { - h64 ^= (U64)(XXH_get32bits(p)) * PRIME64_1; - h64 = XXH_rotl64(h64, 23) * PRIME64_2 + PRIME64_3; - p+=4; - } - - while (p> 33; - h64 *= PRIME64_2; - h64 ^= h64 >> 29; - h64 *= PRIME64_3; - h64 ^= h64 >> 32; - - return h64; -} - - -XXH_PUBLIC_API unsigned long long XXH64 (const void* input, size_t len, unsigned long long seed) -{ -#if 0 - /* Simple version, good for code maintenance, but unfortunately slow for small inputs */ - XXH64_CREATESTATE_STATIC(state); - XXH64_reset(state, seed); - XXH64_update(state, input, len); - return XXH64_digest(state); -#else - XXH_endianess endian_detected = (XXH_endianess)XXH_CPU_LITTLE_ENDIAN; - - if (XXH_FORCE_ALIGN_CHECK) { - if ((((size_t)input) & 7)==0) { /* Input is aligned, let's leverage the speed advantage */ - if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) - return XXH64_endian_align(input, len, seed, XXH_littleEndian, XXH_aligned); - else - return XXH64_endian_align(input, len, seed, XXH_bigEndian, XXH_aligned); - } } - - if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) - return XXH64_endian_align(input, len, seed, XXH_littleEndian, XXH_unaligned); - else - return XXH64_endian_align(input, len, seed, XXH_bigEndian, XXH_unaligned); -#endif -} - - -/* ************************************************** -* Advanced Hash Functions -****************************************************/ - -XXH_PUBLIC_API XXH32_state_t* XXH32_createState(void) -{ - return (XXH32_state_t*)XXH_malloc(sizeof(XXH32_state_t)); -} -XXH_PUBLIC_API XXH_errorcode XXH32_freeState(XXH32_state_t* statePtr) -{ - XXH_free(statePtr); - return XXH_OK; -} - -XXH_PUBLIC_API XXH64_state_t* XXH64_createState(void) -{ - return (XXH64_state_t*)XXH_malloc(sizeof(XXH64_state_t)); -} -XXH_PUBLIC_API XXH_errorcode XXH64_freeState(XXH64_state_t* statePtr) -{ - XXH_free(statePtr); - return XXH_OK; -} - - -/*** Hash feed ***/ - -XXH_PUBLIC_API XXH_errorcode XXH32_reset(XXH32_state_t* statePtr, unsigned int seed) -{ - XXH32_state_t state; /* using a local state to memcpy() in order to avoid strict-aliasing warnings */ - memset(&state, 0, sizeof(state)-4); /* do not write into reserved, for future removal */ - state.v1 = seed + PRIME32_1 + PRIME32_2; - state.v2 = seed + PRIME32_2; - state.v3 = seed + 0; - state.v4 = seed - PRIME32_1; - memcpy(statePtr, &state, sizeof(state)); - return XXH_OK; -} - - -XXH_PUBLIC_API XXH_errorcode XXH64_reset(XXH64_state_t* statePtr, unsigned long long seed) -{ - XXH64_state_t state; /* using a local state to memcpy() in order to avoid strict-aliasing warnings */ - memset(&state, 0, sizeof(state)-8); /* do not write into reserved, for future removal */ - state.v1 = seed + PRIME64_1 + PRIME64_2; - state.v2 = seed + PRIME64_2; - state.v3 = seed + 0; - state.v4 = seed - PRIME64_1; - memcpy(statePtr, &state, sizeof(state)); - return XXH_OK; -} - - -FORCE_INLINE_TEMPLATE XXH_errorcode XXH32_update_endian (XXH32_state_t* state, const void* input, size_t len, XXH_endianess endian) -{ - const BYTE* p = (const BYTE*)input; - const BYTE* const bEnd = p + len; - -#ifdef XXH_ACCEPT_NULL_INPUT_POINTER - if (input==NULL) return XXH_ERROR; -#endif - - state->total_len_32 += (unsigned)len; - state->large_len |= (len>=16) | (state->total_len_32>=16); - - if (state->memsize + len < 16) { /* fill in tmp buffer */ - XXH_memcpy((BYTE*)(state->mem32) + state->memsize, input, len); - state->memsize += (unsigned)len; - return XXH_OK; - } - - if (state->memsize) { /* some data left from previous update */ - XXH_memcpy((BYTE*)(state->mem32) + state->memsize, input, 16-state->memsize); - { const U32* p32 = state->mem32; - state->v1 = XXH32_round(state->v1, XXH_readLE32(p32, endian)); p32++; - state->v2 = XXH32_round(state->v2, XXH_readLE32(p32, endian)); p32++; - state->v3 = XXH32_round(state->v3, XXH_readLE32(p32, endian)); p32++; - state->v4 = XXH32_round(state->v4, XXH_readLE32(p32, endian)); p32++; - } - p += 16-state->memsize; - state->memsize = 0; - } - - if (p <= bEnd-16) { - const BYTE* const limit = bEnd - 16; - U32 v1 = state->v1; - U32 v2 = state->v2; - U32 v3 = state->v3; - U32 v4 = state->v4; - - do { - v1 = XXH32_round(v1, XXH_readLE32(p, endian)); p+=4; - v2 = XXH32_round(v2, XXH_readLE32(p, endian)); p+=4; - v3 = XXH32_round(v3, XXH_readLE32(p, endian)); p+=4; - v4 = XXH32_round(v4, XXH_readLE32(p, endian)); p+=4; - } while (p<=limit); - - state->v1 = v1; - state->v2 = v2; - state->v3 = v3; - state->v4 = v4; - } - - if (p < bEnd) { - XXH_memcpy(state->mem32, p, (size_t)(bEnd-p)); - state->memsize = (unsigned)(bEnd-p); - } - - return XXH_OK; -} - -XXH_PUBLIC_API XXH_errorcode XXH32_update (XXH32_state_t* state_in, const void* input, size_t len) -{ - XXH_endianess endian_detected = (XXH_endianess)XXH_CPU_LITTLE_ENDIAN; - - if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) - return XXH32_update_endian(state_in, input, len, XXH_littleEndian); - else - return XXH32_update_endian(state_in, input, len, XXH_bigEndian); -} - - - -FORCE_INLINE_TEMPLATE U32 XXH32_digest_endian (const XXH32_state_t* state, XXH_endianess endian) -{ - const BYTE * p = (const BYTE*)state->mem32; - const BYTE* const bEnd = (const BYTE*)(state->mem32) + state->memsize; - U32 h32; - - if (state->large_len) { - h32 = XXH_rotl32(state->v1, 1) + XXH_rotl32(state->v2, 7) + XXH_rotl32(state->v3, 12) + XXH_rotl32(state->v4, 18); - } else { - h32 = state->v3 /* == seed */ + PRIME32_5; - } - - h32 += state->total_len_32; - - while (p+4<=bEnd) { - h32 += XXH_readLE32(p, endian) * PRIME32_3; - h32 = XXH_rotl32(h32, 17) * PRIME32_4; - p+=4; - } - - while (p> 15; - h32 *= PRIME32_2; - h32 ^= h32 >> 13; - h32 *= PRIME32_3; - h32 ^= h32 >> 16; - - return h32; -} - - -XXH_PUBLIC_API unsigned int XXH32_digest (const XXH32_state_t* state_in) -{ - XXH_endianess endian_detected = (XXH_endianess)XXH_CPU_LITTLE_ENDIAN; - - if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) - return XXH32_digest_endian(state_in, XXH_littleEndian); - else - return XXH32_digest_endian(state_in, XXH_bigEndian); -} - - - -/* **** XXH64 **** */ - -FORCE_INLINE_TEMPLATE XXH_errorcode XXH64_update_endian (XXH64_state_t* state, const void* input, size_t len, XXH_endianess endian) -{ - const BYTE* p = (const BYTE*)input; - const BYTE* const bEnd = p + len; - -#ifdef XXH_ACCEPT_NULL_INPUT_POINTER - if (input==NULL) return XXH_ERROR; -#endif - - state->total_len += len; - - if (state->memsize + len < 32) { /* fill in tmp buffer */ - if (input != NULL) { - XXH_memcpy(((BYTE*)state->mem64) + state->memsize, input, len); - } - state->memsize += (U32)len; - return XXH_OK; - } - - if (state->memsize) { /* tmp buffer is full */ - XXH_memcpy(((BYTE*)state->mem64) + state->memsize, input, 32-state->memsize); - state->v1 = XXH64_round(state->v1, XXH_readLE64(state->mem64+0, endian)); - state->v2 = XXH64_round(state->v2, XXH_readLE64(state->mem64+1, endian)); - state->v3 = XXH64_round(state->v3, XXH_readLE64(state->mem64+2, endian)); - state->v4 = XXH64_round(state->v4, XXH_readLE64(state->mem64+3, endian)); - p += 32-state->memsize; - state->memsize = 0; - } - - if (p+32 <= bEnd) { - const BYTE* const limit = bEnd - 32; - U64 v1 = state->v1; - U64 v2 = state->v2; - U64 v3 = state->v3; - U64 v4 = state->v4; - - do { - v1 = XXH64_round(v1, XXH_readLE64(p, endian)); p+=8; - v2 = XXH64_round(v2, XXH_readLE64(p, endian)); p+=8; - v3 = XXH64_round(v3, XXH_readLE64(p, endian)); p+=8; - v4 = XXH64_round(v4, XXH_readLE64(p, endian)); p+=8; - } while (p<=limit); - - state->v1 = v1; - state->v2 = v2; - state->v3 = v3; - state->v4 = v4; - } - - if (p < bEnd) { - XXH_memcpy(state->mem64, p, (size_t)(bEnd-p)); - state->memsize = (unsigned)(bEnd-p); - } - - return XXH_OK; -} - -XXH_PUBLIC_API XXH_errorcode XXH64_update (XXH64_state_t* state_in, const void* input, size_t len) -{ - XXH_endianess endian_detected = (XXH_endianess)XXH_CPU_LITTLE_ENDIAN; - - if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) - return XXH64_update_endian(state_in, input, len, XXH_littleEndian); - else - return XXH64_update_endian(state_in, input, len, XXH_bigEndian); -} - - - -FORCE_INLINE_TEMPLATE U64 XXH64_digest_endian (const XXH64_state_t* state, XXH_endianess endian) -{ - const BYTE * p = (const BYTE*)state->mem64; - const BYTE* const bEnd = (const BYTE*)state->mem64 + state->memsize; - U64 h64; - - if (state->total_len >= 32) { - U64 const v1 = state->v1; - U64 const v2 = state->v2; - U64 const v3 = state->v3; - U64 const v4 = state->v4; - - h64 = XXH_rotl64(v1, 1) + XXH_rotl64(v2, 7) + XXH_rotl64(v3, 12) + XXH_rotl64(v4, 18); - h64 = XXH64_mergeRound(h64, v1); - h64 = XXH64_mergeRound(h64, v2); - h64 = XXH64_mergeRound(h64, v3); - h64 = XXH64_mergeRound(h64, v4); - } else { - h64 = state->v3 + PRIME64_5; - } - - h64 += (U64) state->total_len; - - while (p+8<=bEnd) { - U64 const k1 = XXH64_round(0, XXH_readLE64(p, endian)); - h64 ^= k1; - h64 = XXH_rotl64(h64,27) * PRIME64_1 + PRIME64_4; - p+=8; - } - - if (p+4<=bEnd) { - h64 ^= (U64)(XXH_readLE32(p, endian)) * PRIME64_1; - h64 = XXH_rotl64(h64, 23) * PRIME64_2 + PRIME64_3; - p+=4; - } - - while (p> 33; - h64 *= PRIME64_2; - h64 ^= h64 >> 29; - h64 *= PRIME64_3; - h64 ^= h64 >> 32; - - return h64; -} - - -XXH_PUBLIC_API unsigned long long XXH64_digest (const XXH64_state_t* state_in) -{ - XXH_endianess endian_detected = (XXH_endianess)XXH_CPU_LITTLE_ENDIAN; - - if ((endian_detected==XXH_littleEndian) || XXH_FORCE_NATIVE_FORMAT) - return XXH64_digest_endian(state_in, XXH_littleEndian); - else - return XXH64_digest_endian(state_in, XXH_bigEndian); -} - - -/* ************************** -* Canonical representation -****************************/ - -/*! Default XXH result types are basic unsigned 32 and 64 bits. -* The canonical representation follows human-readable write convention, aka big-endian (large digits first). -* These functions allow transformation of hash result into and from its canonical format. -* This way, hash values can be written into a file or buffer, and remain comparable across different systems and programs. -*/ - -XXH_PUBLIC_API void XXH32_canonicalFromHash(XXH32_canonical_t* dst, XXH32_hash_t hash) -{ - XXH_STATIC_ASSERT(sizeof(XXH32_canonical_t) == sizeof(XXH32_hash_t)); - if (XXH_CPU_LITTLE_ENDIAN) hash = XXH_swap32(hash); - memcpy(dst, &hash, sizeof(*dst)); -} - -XXH_PUBLIC_API void XXH64_canonicalFromHash(XXH64_canonical_t* dst, XXH64_hash_t hash) -{ - XXH_STATIC_ASSERT(sizeof(XXH64_canonical_t) == sizeof(XXH64_hash_t)); - if (XXH_CPU_LITTLE_ENDIAN) hash = XXH_swap64(hash); - memcpy(dst, &hash, sizeof(*dst)); -} - -XXH_PUBLIC_API XXH32_hash_t XXH32_hashFromCanonical(const XXH32_canonical_t* src) -{ - return XXH_readBE32(src); -} - -XXH_PUBLIC_API XXH64_hash_t XXH64_hashFromCanonical(const XXH64_canonical_t* src) -{ - return XXH_readBE64(src); -} -/**** ended inlining xxhash.c ****/ -# endif - -#endif /* XXH_STATIC_LINKING_ONLY && XXH_STATIC_H_3543687687345 */ - - -#if defined (__cplusplus) -} -#endif -/**** ended inlining xxhash.h ****/ - -#if defined (__cplusplus) -extern "C" { -#endif - -/* ---- static assert (debug) --- */ -#define ZSTD_STATIC_ASSERT(c) DEBUG_STATIC_ASSERT(c) -#define FSE_isError ERR_isError -#define HUF_isError ERR_isError - - -/*-************************************* -* shared macros -***************************************/ -#undef MIN -#undef MAX -#define MIN(a,b) ((a)<(b) ? (a) : (b)) -#define MAX(a,b) ((a)>(b) ? (a) : (b)) - -/** - * Ignore: this is an internal helper. - * - * This is a helper function to help force C99-correctness during compilation. - * Under strict compilation modes, variadic macro arguments can't be empty. - * However, variadic function arguments can be. Using a function therefore lets - * us statically check that at least one (string) argument was passed, - * independent of the compilation flags. - */ -static INLINE_KEYWORD UNUSED_ATTR -void _force_has_format_string(const char *format, ...) { - (void)format; -} - -/** - * Ignore: this is an internal helper. - * - * We want to force this function invocation to be syntactically correct, but - * we don't want to force runtime evaluation of its arguments. - */ -#define _FORCE_HAS_FORMAT_STRING(...) \ - if (0) { \ - _force_has_format_string(__VA_ARGS__); \ - } - -/** - * Return the specified error if the condition evaluates to true. - * - * In debug modes, prints additional information. - * In order to do that (particularly, printing the conditional that failed), - * this can't just wrap RETURN_ERROR(). - */ -#define RETURN_ERROR_IF(cond, err, ...) \ - if (cond) { \ - RAWLOG(3, "%s:%d: ERROR!: check %s failed, returning %s", \ - __FILE__, __LINE__, ZSTD_QUOTE(cond), ZSTD_QUOTE(ERROR(err))); \ - _FORCE_HAS_FORMAT_STRING(__VA_ARGS__); \ - RAWLOG(3, ": " __VA_ARGS__); \ - RAWLOG(3, "\n"); \ - return ERROR(err); \ - } - -/** - * Unconditionally return the specified error. - * - * In debug modes, prints additional information. - */ -#define RETURN_ERROR(err, ...) \ - do { \ - RAWLOG(3, "%s:%d: ERROR!: unconditional check failed, returning %s", \ - __FILE__, __LINE__, ZSTD_QUOTE(ERROR(err))); \ - _FORCE_HAS_FORMAT_STRING(__VA_ARGS__); \ - RAWLOG(3, ": " __VA_ARGS__); \ - RAWLOG(3, "\n"); \ - return ERROR(err); \ - } while(0); - -/** - * If the provided expression evaluates to an error code, returns that error code. - * - * In debug modes, prints additional information. - */ -#define FORWARD_IF_ERROR(err, ...) \ - do { \ - size_t const err_code = (err); \ - if (ERR_isError(err_code)) { \ - RAWLOG(3, "%s:%d: ERROR!: forwarding error in %s: %s", \ - __FILE__, __LINE__, ZSTD_QUOTE(err), ERR_getErrorName(err_code)); \ - _FORCE_HAS_FORMAT_STRING(__VA_ARGS__); \ - RAWLOG(3, ": " __VA_ARGS__); \ - RAWLOG(3, "\n"); \ - return err_code; \ - } \ - } while(0); - - -/*-************************************* -* Common constants -***************************************/ -#define ZSTD_OPT_NUM (1<<12) - -#define ZSTD_REP_NUM 3 /* number of repcodes */ -#define ZSTD_REP_MOVE (ZSTD_REP_NUM-1) -static const U32 repStartValue[ZSTD_REP_NUM] = { 1, 4, 8 }; - -#define KB *(1 <<10) -#define MB *(1 <<20) -#define GB *(1U<<30) - -#define BIT7 128 -#define BIT6 64 -#define BIT5 32 -#define BIT4 16 -#define BIT1 2 -#define BIT0 1 - -#define ZSTD_WINDOWLOG_ABSOLUTEMIN 10 -static const size_t ZSTD_fcs_fieldSize[4] = { 0, 2, 4, 8 }; -static const size_t ZSTD_did_fieldSize[4] = { 0, 1, 2, 4 }; - -#define ZSTD_FRAMEIDSIZE 4 /* magic number size */ - -#define ZSTD_BLOCKHEADERSIZE 3 /* C standard doesn't allow `static const` variable to be init using another `static const` variable */ -static const size_t ZSTD_blockHeaderSize = ZSTD_BLOCKHEADERSIZE; -typedef enum { bt_raw, bt_rle, bt_compressed, bt_reserved } blockType_e; - -#define ZSTD_FRAMECHECKSUMSIZE 4 - -#define MIN_SEQUENCES_SIZE 1 /* nbSeq==0 */ -#define MIN_CBLOCK_SIZE (1 /*litCSize*/ + 1 /* RLE or RAW */ + MIN_SEQUENCES_SIZE /* nbSeq==0 */) /* for a non-null block */ - -#define HufLog 12 -typedef enum { set_basic, set_rle, set_compressed, set_repeat } symbolEncodingType_e; - -#define LONGNBSEQ 0x7F00 - -#define MINMATCH 3 - -#define Litbits 8 -#define MaxLit ((1<= 8 || (ovtype == ZSTD_no_overlap && diff <= -WILDCOPY_VECLEN)); - - if (ovtype == ZSTD_overlap_src_before_dst && diff < WILDCOPY_VECLEN) { - /* Handle short offset copies. */ - do { - COPY8(op, ip) - } while (op < oend); - } else { - assert(diff >= WILDCOPY_VECLEN || diff <= -WILDCOPY_VECLEN); - /* Separate out the first COPY16() call because the copy length is - * almost certain to be short, so the branches have different - * probabilities. Since it is almost certain to be short, only do - * one COPY16() in the first call. Then, do two calls per loop since - * at that point it is more likely to have a high trip count. - */ -#ifndef __aarch64__ - do { - COPY16(op, ip); - } - while (op < oend); -#else - COPY16(op, ip); - if (op >= oend) return; - do { - COPY16(op, ip); - COPY16(op, ip); - } - while (op < oend); -#endif - } -} - -MEM_STATIC size_t ZSTD_limitCopy(void* dst, size_t dstCapacity, const void* src, size_t srcSize) -{ - size_t const length = MIN(dstCapacity, srcSize); - if (length > 0) { - memcpy(dst, src, length); - } - return length; -} - -/* define "workspace is too large" as this number of times larger than needed */ -#define ZSTD_WORKSPACETOOLARGE_FACTOR 3 - -/* when workspace is continuously too large - * during at least this number of times, - * context's memory usage is considered wasteful, - * because it's sized to handle a worst case scenario which rarely happens. - * In which case, resize it down to free some memory */ -#define ZSTD_WORKSPACETOOLARGE_MAXDURATION 128 - - -/*-******************************************* -* Private declarations -*********************************************/ -typedef struct seqDef_s { - U32 offset; - U16 litLength; - U16 matchLength; -} seqDef; - -typedef struct { - seqDef* sequencesStart; - seqDef* sequences; - BYTE* litStart; - BYTE* lit; - BYTE* llCode; - BYTE* mlCode; - BYTE* ofCode; - size_t maxNbSeq; - size_t maxNbLit; - U32 longLengthID; /* 0 == no longLength; 1 == Lit.longLength; 2 == Match.longLength; */ - U32 longLengthPos; -} seqStore_t; - -typedef struct { - U32 litLength; - U32 matchLength; -} ZSTD_sequenceLength; - -/** - * Returns the ZSTD_sequenceLength for the given sequences. It handles the decoding of long sequences - * indicated by longLengthPos and longLengthID, and adds MINMATCH back to matchLength. - */ -MEM_STATIC ZSTD_sequenceLength ZSTD_getSequenceLength(seqStore_t const* seqStore, seqDef const* seq) -{ - ZSTD_sequenceLength seqLen; - seqLen.litLength = seq->litLength; - seqLen.matchLength = seq->matchLength + MINMATCH; - if (seqStore->longLengthPos == (U32)(seq - seqStore->sequencesStart)) { - if (seqStore->longLengthID == 1) { - seqLen.litLength += 0xFFFF; - } - if (seqStore->longLengthID == 2) { - seqLen.matchLength += 0xFFFF; - } - } - return seqLen; -} - -/** - * Contains the compressed frame size and an upper-bound for the decompressed frame size. - * Note: before using `compressedSize`, check for errors using ZSTD_isError(). - * similarly, before using `decompressedBound`, check for errors using: - * `decompressedBound != ZSTD_CONTENTSIZE_ERROR` - */ -typedef struct { - size_t compressedSize; - unsigned long long decompressedBound; -} ZSTD_frameSizeInfo; /* decompress & legacy */ - -const seqStore_t* ZSTD_getSeqStore(const ZSTD_CCtx* ctx); /* compress & dictBuilder */ -void ZSTD_seqToCodes(const seqStore_t* seqStorePtr); /* compress, dictBuilder, decodeCorpus (shouldn't get its definition from here) */ - -/* custom memory allocation functions */ -void* ZSTD_malloc(size_t size, ZSTD_customMem customMem); -void* ZSTD_calloc(size_t size, ZSTD_customMem customMem); -void ZSTD_free(void* ptr, ZSTD_customMem customMem); - - -MEM_STATIC U32 ZSTD_highbit32(U32 val) /* compress, dictBuilder, decodeCorpus */ -{ - assert(val != 0); - { -# if defined(_MSC_VER) /* Visual */ - unsigned long r=0; - return _BitScanReverse(&r, val) ? (unsigned)r : 0; -# elif defined(__GNUC__) && (__GNUC__ >= 3) /* GCC Intrinsic */ - return __builtin_clz (val) ^ 31; -# elif defined(__ICCARM__) /* IAR Intrinsic */ - return 31 - __CLZ(val); -# else /* Software version */ - static const U32 DeBruijnClz[32] = { 0, 9, 1, 10, 13, 21, 2, 29, 11, 14, 16, 18, 22, 25, 3, 30, 8, 12, 20, 28, 15, 17, 24, 7, 19, 27, 23, 6, 26, 5, 4, 31 }; - U32 v = val; - v |= v >> 1; - v |= v >> 2; - v |= v >> 4; - v |= v >> 8; - v |= v >> 16; - return DeBruijnClz[(v * 0x07C4ACDDU) >> 27]; -# endif - } -} - - -/* ZSTD_invalidateRepCodes() : - * ensures next compression will not use repcodes from previous block. - * Note : only works with regular variant; - * do not use with extDict variant ! */ -void ZSTD_invalidateRepCodes(ZSTD_CCtx* cctx); /* zstdmt, adaptive_compression (shouldn't get this definition from here) */ - - -typedef struct { - blockType_e blockType; - U32 lastBlock; - U32 origSize; -} blockProperties_t; /* declared here for decompress and fullbench */ - -/*! ZSTD_getcBlockSize() : - * Provides the size of compressed block from block header `src` */ -/* Used by: decompress, fullbench (does not get its definition from here) */ -size_t ZSTD_getcBlockSize(const void* src, size_t srcSize, - blockProperties_t* bpPtr); - -/*! ZSTD_decodeSeqHeaders() : - * decode sequence header from src */ -/* Used by: decompress, fullbench (does not get its definition from here) */ -size_t ZSTD_decodeSeqHeaders(ZSTD_DCtx* dctx, int* nbSeqPtr, - const void* src, size_t srcSize); - - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTD_CCOMMON_H_MODULE */ -/**** ended inlining zstd_internal.h ****/ -/**** start inlining pool.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef POOL_H -#define POOL_H - -#if defined (__cplusplus) -extern "C" { -#endif - - -#include /* size_t */ -#define ZSTD_STATIC_LINKING_ONLY /* ZSTD_customMem */ -/**** skipping file: ../zstd.h ****/ - -typedef struct POOL_ctx_s POOL_ctx; - -/*! POOL_create() : - * Create a thread pool with at most `numThreads` threads. - * `numThreads` must be at least 1. - * The maximum number of queued jobs before blocking is `queueSize`. - * @return : POOL_ctx pointer on success, else NULL. -*/ -POOL_ctx* POOL_create(size_t numThreads, size_t queueSize); - -POOL_ctx* POOL_create_advanced(size_t numThreads, size_t queueSize, - ZSTD_customMem customMem); - -/*! POOL_free() : - * Free a thread pool returned by POOL_create(). - */ -void POOL_free(POOL_ctx* ctx); - -/*! POOL_resize() : - * Expands or shrinks pool's number of threads. - * This is more efficient than releasing + creating a new context, - * since it tries to preserve and re-use existing threads. - * `numThreads` must be at least 1. - * @return : 0 when resize was successful, - * !0 (typically 1) if there is an error. - * note : only numThreads can be resized, queueSize remains unchanged. - */ -int POOL_resize(POOL_ctx* ctx, size_t numThreads); - -/*! POOL_sizeof() : - * @return threadpool memory usage - * note : compatible with NULL (returns 0 in this case) - */ -size_t POOL_sizeof(POOL_ctx* ctx); - -/*! POOL_function : - * The function type that can be added to a thread pool. - */ -typedef void (*POOL_function)(void*); - -/*! POOL_add() : - * Add the job `function(opaque)` to the thread pool. `ctx` must be valid. - * Possibly blocks until there is room in the queue. - * Note : The function may be executed asynchronously, - * therefore, `opaque` must live until function has been completed. - */ -void POOL_add(POOL_ctx* ctx, POOL_function function, void* opaque); - - -/*! POOL_tryAdd() : - * Add the job `function(opaque)` to thread pool _if_ a worker is available. - * Returns immediately even if not (does not block). - * @return : 1 if successful, 0 if not. - */ -int POOL_tryAdd(POOL_ctx* ctx, POOL_function function, void* opaque); - - -#if defined (__cplusplus) -} -#endif - -#endif -/**** ended inlining pool.h ****/ - -/* ====== Compiler specifics ====== */ -#if defined(_MSC_VER) -# pragma warning(disable : 4204) /* disable: C4204: non-constant aggregate initializer */ -#endif - - -#ifdef ZSTD_MULTITHREAD - -/**** start inlining threading.h ****/ -/** - * Copyright (c) 2016 Tino Reichardt - * All rights reserved. - * - * You can contact the author at: - * - zstdmt source repository: https://github.com/mcmilk/zstdmt - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef THREADING_H_938743 -#define THREADING_H_938743 - -/**** skipping file: debug.h ****/ - -#if defined (__cplusplus) -extern "C" { -#endif - -#if defined(ZSTD_MULTITHREAD) && defined(_WIN32) - -/** - * Windows minimalist Pthread Wrapper, based on : - * http://www.cse.wustl.edu/~schmidt/win32-cv-1.html - */ -#ifdef WINVER -# undef WINVER -#endif -#define WINVER 0x0600 - -#ifdef _WIN32_WINNT -# undef _WIN32_WINNT -#endif -#define _WIN32_WINNT 0x0600 - -#ifndef WIN32_LEAN_AND_MEAN -# define WIN32_LEAN_AND_MEAN -#endif - -#undef ERROR /* reported already defined on VS 2015 (Rich Geldreich) */ -#include -#undef ERROR -#define ERROR(name) ZSTD_ERROR(name) - - -/* mutex */ -#define ZSTD_pthread_mutex_t CRITICAL_SECTION -#define ZSTD_pthread_mutex_init(a, b) ((void)(b), InitializeCriticalSection((a)), 0) -#define ZSTD_pthread_mutex_destroy(a) DeleteCriticalSection((a)) -#define ZSTD_pthread_mutex_lock(a) EnterCriticalSection((a)) -#define ZSTD_pthread_mutex_unlock(a) LeaveCriticalSection((a)) - -/* condition variable */ -#define ZSTD_pthread_cond_t CONDITION_VARIABLE -#define ZSTD_pthread_cond_init(a, b) ((void)(b), InitializeConditionVariable((a)), 0) -#define ZSTD_pthread_cond_destroy(a) ((void)(a)) -#define ZSTD_pthread_cond_wait(a, b) SleepConditionVariableCS((a), (b), INFINITE) -#define ZSTD_pthread_cond_signal(a) WakeConditionVariable((a)) -#define ZSTD_pthread_cond_broadcast(a) WakeAllConditionVariable((a)) - -/* ZSTD_pthread_create() and ZSTD_pthread_join() */ -typedef struct { - HANDLE handle; - void* (*start_routine)(void*); - void* arg; -} ZSTD_pthread_t; - -int ZSTD_pthread_create(ZSTD_pthread_t* thread, const void* unused, - void* (*start_routine) (void*), void* arg); - -int ZSTD_pthread_join(ZSTD_pthread_t thread, void** value_ptr); - -/** - * add here more wrappers as required - */ - - -#elif defined(ZSTD_MULTITHREAD) /* posix assumed ; need a better detection method */ -/* === POSIX Systems === */ -# include - -#if DEBUGLEVEL < 1 - -#define ZSTD_pthread_mutex_t pthread_mutex_t -#define ZSTD_pthread_mutex_init(a, b) pthread_mutex_init((a), (b)) -#define ZSTD_pthread_mutex_destroy(a) pthread_mutex_destroy((a)) -#define ZSTD_pthread_mutex_lock(a) pthread_mutex_lock((a)) -#define ZSTD_pthread_mutex_unlock(a) pthread_mutex_unlock((a)) - -#define ZSTD_pthread_cond_t pthread_cond_t -#define ZSTD_pthread_cond_init(a, b) pthread_cond_init((a), (b)) -#define ZSTD_pthread_cond_destroy(a) pthread_cond_destroy((a)) -#define ZSTD_pthread_cond_wait(a, b) pthread_cond_wait((a), (b)) -#define ZSTD_pthread_cond_signal(a) pthread_cond_signal((a)) -#define ZSTD_pthread_cond_broadcast(a) pthread_cond_broadcast((a)) - -#define ZSTD_pthread_t pthread_t -#define ZSTD_pthread_create(a, b, c, d) pthread_create((a), (b), (c), (d)) -#define ZSTD_pthread_join(a, b) pthread_join((a),(b)) - -#else /* DEBUGLEVEL >= 1 */ - -/* Debug implementation of threading. - * In this implementation we use pointers for mutexes and condition variables. - * This way, if we forget to init/destroy them the program will crash or ASAN - * will report leaks. - */ - -#define ZSTD_pthread_mutex_t pthread_mutex_t* -int ZSTD_pthread_mutex_init(ZSTD_pthread_mutex_t* mutex, pthread_mutexattr_t const* attr); -int ZSTD_pthread_mutex_destroy(ZSTD_pthread_mutex_t* mutex); -#define ZSTD_pthread_mutex_lock(a) pthread_mutex_lock(*(a)) -#define ZSTD_pthread_mutex_unlock(a) pthread_mutex_unlock(*(a)) - -#define ZSTD_pthread_cond_t pthread_cond_t* -int ZSTD_pthread_cond_init(ZSTD_pthread_cond_t* cond, pthread_condattr_t const* attr); -int ZSTD_pthread_cond_destroy(ZSTD_pthread_cond_t* cond); -#define ZSTD_pthread_cond_wait(a, b) pthread_cond_wait(*(a), *(b)) -#define ZSTD_pthread_cond_signal(a) pthread_cond_signal(*(a)) -#define ZSTD_pthread_cond_broadcast(a) pthread_cond_broadcast(*(a)) - -#define ZSTD_pthread_t pthread_t -#define ZSTD_pthread_create(a, b, c, d) pthread_create((a), (b), (c), (d)) -#define ZSTD_pthread_join(a, b) pthread_join((a),(b)) - -#endif - -#else /* ZSTD_MULTITHREAD not defined */ -/* No multithreading support */ - -typedef int ZSTD_pthread_mutex_t; -#define ZSTD_pthread_mutex_init(a, b) ((void)(a), (void)(b), 0) -#define ZSTD_pthread_mutex_destroy(a) ((void)(a)) -#define ZSTD_pthread_mutex_lock(a) ((void)(a)) -#define ZSTD_pthread_mutex_unlock(a) ((void)(a)) - -typedef int ZSTD_pthread_cond_t; -#define ZSTD_pthread_cond_init(a, b) ((void)(a), (void)(b), 0) -#define ZSTD_pthread_cond_destroy(a) ((void)(a)) -#define ZSTD_pthread_cond_wait(a, b) ((void)(a), (void)(b)) -#define ZSTD_pthread_cond_signal(a) ((void)(a)) -#define ZSTD_pthread_cond_broadcast(a) ((void)(a)) - -/* do not use ZSTD_pthread_t */ - -#endif /* ZSTD_MULTITHREAD */ - -#if defined (__cplusplus) -} -#endif - -#endif /* THREADING_H_938743 */ -/**** ended inlining threading.h ****/ - -/* A job is a function and an opaque argument */ -typedef struct POOL_job_s { - POOL_function function; - void *opaque; -} POOL_job; - -struct POOL_ctx_s { - ZSTD_customMem customMem; - /* Keep track of the threads */ - ZSTD_pthread_t* threads; - size_t threadCapacity; - size_t threadLimit; - - /* The queue is a circular buffer */ - POOL_job *queue; - size_t queueHead; - size_t queueTail; - size_t queueSize; - - /* The number of threads working on jobs */ - size_t numThreadsBusy; - /* Indicates if the queue is empty */ - int queueEmpty; - - /* The mutex protects the queue */ - ZSTD_pthread_mutex_t queueMutex; - /* Condition variable for pushers to wait on when the queue is full */ - ZSTD_pthread_cond_t queuePushCond; - /* Condition variables for poppers to wait on when the queue is empty */ - ZSTD_pthread_cond_t queuePopCond; - /* Indicates if the queue is shutting down */ - int shutdown; -}; - -/* POOL_thread() : - * Work thread for the thread pool. - * Waits for jobs and executes them. - * @returns : NULL on failure else non-null. - */ -static void* POOL_thread(void* opaque) { - POOL_ctx* const ctx = (POOL_ctx*)opaque; - if (!ctx) { return NULL; } - for (;;) { - /* Lock the mutex and wait for a non-empty queue or until shutdown */ - ZSTD_pthread_mutex_lock(&ctx->queueMutex); - - while ( ctx->queueEmpty - || (ctx->numThreadsBusy >= ctx->threadLimit) ) { - if (ctx->shutdown) { - /* even if !queueEmpty, (possible if numThreadsBusy >= threadLimit), - * a few threads will be shutdown while !queueEmpty, - * but enough threads will remain active to finish the queue */ - ZSTD_pthread_mutex_unlock(&ctx->queueMutex); - return opaque; - } - ZSTD_pthread_cond_wait(&ctx->queuePopCond, &ctx->queueMutex); - } - /* Pop a job off the queue */ - { POOL_job const job = ctx->queue[ctx->queueHead]; - ctx->queueHead = (ctx->queueHead + 1) % ctx->queueSize; - ctx->numThreadsBusy++; - ctx->queueEmpty = ctx->queueHead == ctx->queueTail; - /* Unlock the mutex, signal a pusher, and run the job */ - ZSTD_pthread_cond_signal(&ctx->queuePushCond); - ZSTD_pthread_mutex_unlock(&ctx->queueMutex); - - job.function(job.opaque); - - /* If the intended queue size was 0, signal after finishing job */ - ZSTD_pthread_mutex_lock(&ctx->queueMutex); - ctx->numThreadsBusy--; - if (ctx->queueSize == 1) { - ZSTD_pthread_cond_signal(&ctx->queuePushCond); - } - ZSTD_pthread_mutex_unlock(&ctx->queueMutex); - } - } /* for (;;) */ - assert(0); /* Unreachable */ -} - -POOL_ctx* POOL_create(size_t numThreads, size_t queueSize) { - return POOL_create_advanced(numThreads, queueSize, ZSTD_defaultCMem); -} - -POOL_ctx* POOL_create_advanced(size_t numThreads, size_t queueSize, - ZSTD_customMem customMem) { - POOL_ctx* ctx; - /* Check parameters */ - if (!numThreads) { return NULL; } - /* Allocate the context and zero initialize */ - ctx = (POOL_ctx*)ZSTD_calloc(sizeof(POOL_ctx), customMem); - if (!ctx) { return NULL; } - /* Initialize the job queue. - * It needs one extra space since one space is wasted to differentiate - * empty and full queues. - */ - ctx->queueSize = queueSize + 1; - ctx->queue = (POOL_job*)ZSTD_malloc(ctx->queueSize * sizeof(POOL_job), customMem); - ctx->queueHead = 0; - ctx->queueTail = 0; - ctx->numThreadsBusy = 0; - ctx->queueEmpty = 1; - { - int error = 0; - error |= ZSTD_pthread_mutex_init(&ctx->queueMutex, NULL); - error |= ZSTD_pthread_cond_init(&ctx->queuePushCond, NULL); - error |= ZSTD_pthread_cond_init(&ctx->queuePopCond, NULL); - if (error) { POOL_free(ctx); return NULL; } - } - ctx->shutdown = 0; - /* Allocate space for the thread handles */ - ctx->threads = (ZSTD_pthread_t*)ZSTD_malloc(numThreads * sizeof(ZSTD_pthread_t), customMem); - ctx->threadCapacity = 0; - ctx->customMem = customMem; - /* Check for errors */ - if (!ctx->threads || !ctx->queue) { POOL_free(ctx); return NULL; } - /* Initialize the threads */ - { size_t i; - for (i = 0; i < numThreads; ++i) { - if (ZSTD_pthread_create(&ctx->threads[i], NULL, &POOL_thread, ctx)) { - ctx->threadCapacity = i; - POOL_free(ctx); - return NULL; - } } - ctx->threadCapacity = numThreads; - ctx->threadLimit = numThreads; - } - return ctx; -} - -/*! POOL_join() : - Shutdown the queue, wake any sleeping threads, and join all of the threads. -*/ -static void POOL_join(POOL_ctx* ctx) { - /* Shut down the queue */ - ZSTD_pthread_mutex_lock(&ctx->queueMutex); - ctx->shutdown = 1; - ZSTD_pthread_mutex_unlock(&ctx->queueMutex); - /* Wake up sleeping threads */ - ZSTD_pthread_cond_broadcast(&ctx->queuePushCond); - ZSTD_pthread_cond_broadcast(&ctx->queuePopCond); - /* Join all of the threads */ - { size_t i; - for (i = 0; i < ctx->threadCapacity; ++i) { - ZSTD_pthread_join(ctx->threads[i], NULL); /* note : could fail */ - } } -} - -void POOL_free(POOL_ctx *ctx) { - if (!ctx) { return; } - POOL_join(ctx); - ZSTD_pthread_mutex_destroy(&ctx->queueMutex); - ZSTD_pthread_cond_destroy(&ctx->queuePushCond); - ZSTD_pthread_cond_destroy(&ctx->queuePopCond); - ZSTD_free(ctx->queue, ctx->customMem); - ZSTD_free(ctx->threads, ctx->customMem); - ZSTD_free(ctx, ctx->customMem); -} - - - -size_t POOL_sizeof(POOL_ctx *ctx) { - if (ctx==NULL) return 0; /* supports sizeof NULL */ - return sizeof(*ctx) - + ctx->queueSize * sizeof(POOL_job) - + ctx->threadCapacity * sizeof(ZSTD_pthread_t); -} - - -/* @return : 0 on success, 1 on error */ -static int POOL_resize_internal(POOL_ctx* ctx, size_t numThreads) -{ - if (numThreads <= ctx->threadCapacity) { - if (!numThreads) return 1; - ctx->threadLimit = numThreads; - return 0; - } - /* numThreads > threadCapacity */ - { ZSTD_pthread_t* const threadPool = (ZSTD_pthread_t*)ZSTD_malloc(numThreads * sizeof(ZSTD_pthread_t), ctx->customMem); - if (!threadPool) return 1; - /* replace existing thread pool */ - memcpy(threadPool, ctx->threads, ctx->threadCapacity * sizeof(*threadPool)); - ZSTD_free(ctx->threads, ctx->customMem); - ctx->threads = threadPool; - /* Initialize additional threads */ - { size_t threadId; - for (threadId = ctx->threadCapacity; threadId < numThreads; ++threadId) { - if (ZSTD_pthread_create(&threadPool[threadId], NULL, &POOL_thread, ctx)) { - ctx->threadCapacity = threadId; - return 1; - } } - } } - /* successfully expanded */ - ctx->threadCapacity = numThreads; - ctx->threadLimit = numThreads; - return 0; -} - -/* @return : 0 on success, 1 on error */ -int POOL_resize(POOL_ctx* ctx, size_t numThreads) -{ - int result; - if (ctx==NULL) return 1; - ZSTD_pthread_mutex_lock(&ctx->queueMutex); - result = POOL_resize_internal(ctx, numThreads); - ZSTD_pthread_cond_broadcast(&ctx->queuePopCond); - ZSTD_pthread_mutex_unlock(&ctx->queueMutex); - return result; -} - -/** - * Returns 1 if the queue is full and 0 otherwise. - * - * When queueSize is 1 (pool was created with an intended queueSize of 0), - * then a queue is empty if there is a thread free _and_ no job is waiting. - */ -static int isQueueFull(POOL_ctx const* ctx) { - if (ctx->queueSize > 1) { - return ctx->queueHead == ((ctx->queueTail + 1) % ctx->queueSize); - } else { - return (ctx->numThreadsBusy == ctx->threadLimit) || - !ctx->queueEmpty; - } -} - - -static void POOL_add_internal(POOL_ctx* ctx, POOL_function function, void *opaque) -{ - POOL_job const job = {function, opaque}; - assert(ctx != NULL); - if (ctx->shutdown) return; - - ctx->queueEmpty = 0; - ctx->queue[ctx->queueTail] = job; - ctx->queueTail = (ctx->queueTail + 1) % ctx->queueSize; - ZSTD_pthread_cond_signal(&ctx->queuePopCond); -} - -void POOL_add(POOL_ctx* ctx, POOL_function function, void* opaque) -{ - assert(ctx != NULL); - ZSTD_pthread_mutex_lock(&ctx->queueMutex); - /* Wait until there is space in the queue for the new job */ - while (isQueueFull(ctx) && (!ctx->shutdown)) { - ZSTD_pthread_cond_wait(&ctx->queuePushCond, &ctx->queueMutex); - } - POOL_add_internal(ctx, function, opaque); - ZSTD_pthread_mutex_unlock(&ctx->queueMutex); -} - - -int POOL_tryAdd(POOL_ctx* ctx, POOL_function function, void* opaque) -{ - assert(ctx != NULL); - ZSTD_pthread_mutex_lock(&ctx->queueMutex); - if (isQueueFull(ctx)) { - ZSTD_pthread_mutex_unlock(&ctx->queueMutex); - return 0; - } - POOL_add_internal(ctx, function, opaque); - ZSTD_pthread_mutex_unlock(&ctx->queueMutex); - return 1; -} - - -#else /* ZSTD_MULTITHREAD not defined */ - -/* ========================== */ -/* No multi-threading support */ -/* ========================== */ - - -/* We don't need any data, but if it is empty, malloc() might return NULL. */ -struct POOL_ctx_s { - int dummy; -}; -static POOL_ctx g_ctx; - -POOL_ctx* POOL_create(size_t numThreads, size_t queueSize) { - return POOL_create_advanced(numThreads, queueSize, ZSTD_defaultCMem); -} - -POOL_ctx* POOL_create_advanced(size_t numThreads, size_t queueSize, ZSTD_customMem customMem) { - (void)numThreads; - (void)queueSize; - (void)customMem; - return &g_ctx; -} - -void POOL_free(POOL_ctx* ctx) { - assert(!ctx || ctx == &g_ctx); - (void)ctx; -} - -int POOL_resize(POOL_ctx* ctx, size_t numThreads) { - (void)ctx; (void)numThreads; - return 0; -} - -void POOL_add(POOL_ctx* ctx, POOL_function function, void* opaque) { - (void)ctx; - function(opaque); -} - -int POOL_tryAdd(POOL_ctx* ctx, POOL_function function, void* opaque) { - (void)ctx; - function(opaque); - return 1; -} - -size_t POOL_sizeof(POOL_ctx* ctx) { - if (ctx==NULL) return 0; /* supports sizeof NULL */ - assert(ctx == &g_ctx); - return sizeof(*ctx); -} - -#endif /* ZSTD_MULTITHREAD */ -/**** ended inlining common/pool.c ****/ -/**** start inlining common/zstd_common.c ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - - - -/*-************************************* -* Dependencies -***************************************/ -#include /* malloc, calloc, free */ -#include /* memset */ -/**** skipping file: error_private.h ****/ -/**** skipping file: zstd_internal.h ****/ - - -/*-**************************************** -* Version -******************************************/ -unsigned ZSTD_versionNumber(void) { return ZSTD_VERSION_NUMBER; } - -const char* ZSTD_versionString(void) { return ZSTD_VERSION_STRING; } - - -/*-**************************************** -* ZSTD Error Management -******************************************/ -/*! ZSTD_isError() : - * tells if a return value is an error code - * symbol is required for external callers */ -unsigned ZSTD_isError(size_t code) { return ERR_isError(code); } - -/*! ZSTD_getErrorName() : - * provides error code string from function result (useful for debugging) */ -const char* ZSTD_getErrorName(size_t code) { return ERR_getErrorName(code); } - -/*! ZSTD_getError() : - * convert a `size_t` function result into a proper ZSTD_errorCode enum */ -ZSTD_ErrorCode ZSTD_getErrorCode(size_t code) { return ERR_getErrorCode(code); } - -/*! ZSTD_getErrorString() : - * provides error code string from enum */ -const char* ZSTD_getErrorString(ZSTD_ErrorCode code) { return ERR_getErrorString(code); } - - - -/*=************************************************************** -* Custom allocator -****************************************************************/ -void* ZSTD_malloc(size_t size, ZSTD_customMem customMem) -{ - if (customMem.customAlloc) - return customMem.customAlloc(customMem.opaque, size); - return malloc(size); -} - -void* ZSTD_calloc(size_t size, ZSTD_customMem customMem) -{ - if (customMem.customAlloc) { - /* calloc implemented as malloc+memset; - * not as efficient as calloc, but next best guess for custom malloc */ - void* const ptr = customMem.customAlloc(customMem.opaque, size); - memset(ptr, 0, size); - return ptr; - } - return calloc(1, size); -} - -void ZSTD_free(void* ptr, ZSTD_customMem customMem) -{ - if (ptr!=NULL) { - if (customMem.customFree) - customMem.customFree(customMem.opaque, ptr); - else - free(ptr); - } -} -/**** ended inlining common/zstd_common.c ****/ - -/**** start inlining compress/fse_compress.c ****/ -/* ****************************************************************** - * FSE : Finite State Entropy encoder - * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. - * - * You can contact the author at : - * - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy - * - Public forum : https://groups.google.com/forum/#!forum/lz4c - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. -****************************************************************** */ - -/* ************************************************************** -* Includes -****************************************************************/ -#include /* malloc, free, qsort */ -#include /* memcpy, memset */ -/**** skipping file: ../common/compiler.h ****/ -/**** skipping file: ../common/mem.h ****/ -/**** skipping file: ../common/debug.h ****/ -/**** start inlining hist.h ****/ -/* ****************************************************************** - * hist : Histogram functions - * part of Finite State Entropy project - * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. - * - * You can contact the author at : - * - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy - * - Public forum : https://groups.google.com/forum/#!forum/lz4c - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. -****************************************************************** */ - -/* --- dependencies --- */ -#include /* size_t */ - - -/* --- simple histogram functions --- */ - -/*! HIST_count(): - * Provides the precise count of each byte within a table 'count'. - * 'count' is a table of unsigned int, of minimum size (*maxSymbolValuePtr+1). - * Updates *maxSymbolValuePtr with actual largest symbol value detected. - * @return : count of the most frequent symbol (which isn't identified). - * or an error code, which can be tested using HIST_isError(). - * note : if return == srcSize, there is only one symbol. - */ -size_t HIST_count(unsigned* count, unsigned* maxSymbolValuePtr, - const void* src, size_t srcSize); - -unsigned HIST_isError(size_t code); /**< tells if a return value is an error code */ - - -/* --- advanced histogram functions --- */ - -#define HIST_WKSP_SIZE_U32 1024 -#define HIST_WKSP_SIZE (HIST_WKSP_SIZE_U32 * sizeof(unsigned)) -/** HIST_count_wksp() : - * Same as HIST_count(), but using an externally provided scratch buffer. - * Benefit is this function will use very little stack space. - * `workSpace` is a writable buffer which must be 4-bytes aligned, - * `workSpaceSize` must be >= HIST_WKSP_SIZE - */ -size_t HIST_count_wksp(unsigned* count, unsigned* maxSymbolValuePtr, - const void* src, size_t srcSize, - void* workSpace, size_t workSpaceSize); - -/** HIST_countFast() : - * same as HIST_count(), but blindly trusts that all byte values within src are <= *maxSymbolValuePtr. - * This function is unsafe, and will segfault if any value within `src` is `> *maxSymbolValuePtr` - */ -size_t HIST_countFast(unsigned* count, unsigned* maxSymbolValuePtr, - const void* src, size_t srcSize); - -/** HIST_countFast_wksp() : - * Same as HIST_countFast(), but using an externally provided scratch buffer. - * `workSpace` is a writable buffer which must be 4-bytes aligned, - * `workSpaceSize` must be >= HIST_WKSP_SIZE - */ -size_t HIST_countFast_wksp(unsigned* count, unsigned* maxSymbolValuePtr, - const void* src, size_t srcSize, - void* workSpace, size_t workSpaceSize); - -/*! HIST_count_simple() : - * Same as HIST_countFast(), this function is unsafe, - * and will segfault if any value within `src` is `> *maxSymbolValuePtr`. - * It is also a bit slower for large inputs. - * However, it does not need any additional memory (not even on stack). - * @return : count of the most frequent symbol. - * Note this function doesn't produce any error (i.e. it must succeed). - */ -unsigned HIST_count_simple(unsigned* count, unsigned* maxSymbolValuePtr, - const void* src, size_t srcSize); -/**** ended inlining hist.h ****/ -/**** skipping file: ../common/bitstream.h ****/ -#define FSE_STATIC_LINKING_ONLY -/**** skipping file: ../common/fse.h ****/ -/**** skipping file: ../common/error_private.h ****/ - - -/* ************************************************************** -* Error Management -****************************************************************/ -#define FSE_isError ERR_isError - - -/* ************************************************************** -* Templates -****************************************************************/ -/* - designed to be included - for type-specific functions (template emulation in C) - Objective is to write these functions only once, for improved maintenance -*/ - -/* safety checks */ -#ifndef FSE_FUNCTION_EXTENSION -# error "FSE_FUNCTION_EXTENSION must be defined" -#endif -#ifndef FSE_FUNCTION_TYPE -# error "FSE_FUNCTION_TYPE must be defined" -#endif - -/* Function names */ -#define FSE_CAT(X,Y) X##Y -#define FSE_FUNCTION_NAME(X,Y) FSE_CAT(X,Y) -#define FSE_TYPE_NAME(X,Y) FSE_CAT(X,Y) - - -/* Function templates */ - -/* FSE_buildCTable_wksp() : - * Same as FSE_buildCTable(), but using an externally allocated scratch buffer (`workSpace`). - * wkspSize should be sized to handle worst case situation, which is `1<>1 : 1) ; - FSE_symbolCompressionTransform* const symbolTT = (FSE_symbolCompressionTransform*) (FSCT); - U32 const step = FSE_TABLESTEP(tableSize); - U32 cumul[FSE_MAX_SYMBOL_VALUE+2]; - - FSE_FUNCTION_TYPE* const tableSymbol = (FSE_FUNCTION_TYPE*)workSpace; - U32 highThreshold = tableSize-1; - - /* CTable header */ - if (((size_t)1 << tableLog) * sizeof(FSE_FUNCTION_TYPE) > wkspSize) return ERROR(tableLog_tooLarge); - tableU16[-2] = (U16) tableLog; - tableU16[-1] = (U16) maxSymbolValue; - assert(tableLog < 16); /* required for threshold strategy to work */ - - /* For explanations on how to distribute symbol values over the table : - * http://fastcompression.blogspot.fr/2014/02/fse-distributing-symbol-values.html */ - - #ifdef __clang_analyzer__ - memset(tableSymbol, 0, sizeof(*tableSymbol) * tableSize); /* useless initialization, just to keep scan-build happy */ - #endif - - /* symbol start positions */ - { U32 u; - cumul[0] = 0; - for (u=1; u <= maxSymbolValue+1; u++) { - if (normalizedCounter[u-1]==-1) { /* Low proba symbol */ - cumul[u] = cumul[u-1] + 1; - tableSymbol[highThreshold--] = (FSE_FUNCTION_TYPE)(u-1); - } else { - cumul[u] = cumul[u-1] + normalizedCounter[u-1]; - } } - cumul[maxSymbolValue+1] = tableSize+1; - } - - /* Spread symbols */ - { U32 position = 0; - U32 symbol; - for (symbol=0; symbol<=maxSymbolValue; symbol++) { - int nbOccurrences; - int const freq = normalizedCounter[symbol]; - for (nbOccurrences=0; nbOccurrences highThreshold) - position = (position + step) & tableMask; /* Low proba area */ - } } - - assert(position==0); /* Must have initialized all positions */ - } - - /* Build table */ - { U32 u; for (u=0; u> 3) + 3; - return maxSymbolValue ? maxHeaderSize : FSE_NCOUNTBOUND; /* maxSymbolValue==0 ? use default */ -} - -static size_t -FSE_writeNCount_generic (void* header, size_t headerBufferSize, - const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog, - unsigned writeIsSafe) -{ - BYTE* const ostart = (BYTE*) header; - BYTE* out = ostart; - BYTE* const oend = ostart + headerBufferSize; - int nbBits; - const int tableSize = 1 << tableLog; - int remaining; - int threshold; - U32 bitStream = 0; - int bitCount = 0; - unsigned symbol = 0; - unsigned const alphabetSize = maxSymbolValue + 1; - int previousIs0 = 0; - - /* Table Size */ - bitStream += (tableLog-FSE_MIN_TABLELOG) << bitCount; - bitCount += 4; - - /* Init */ - remaining = tableSize+1; /* +1 for extra accuracy */ - threshold = tableSize; - nbBits = tableLog+1; - - while ((symbol < alphabetSize) && (remaining>1)) { /* stops at 1 */ - if (previousIs0) { - unsigned start = symbol; - while ((symbol < alphabetSize) && !normalizedCounter[symbol]) symbol++; - if (symbol == alphabetSize) break; /* incorrect distribution */ - while (symbol >= start+24) { - start+=24; - bitStream += 0xFFFFU << bitCount; - if ((!writeIsSafe) && (out > oend-2)) - return ERROR(dstSize_tooSmall); /* Buffer overflow */ - out[0] = (BYTE) bitStream; - out[1] = (BYTE)(bitStream>>8); - out+=2; - bitStream>>=16; - } - while (symbol >= start+3) { - start+=3; - bitStream += 3 << bitCount; - bitCount += 2; - } - bitStream += (symbol-start) << bitCount; - bitCount += 2; - if (bitCount>16) { - if ((!writeIsSafe) && (out > oend - 2)) - return ERROR(dstSize_tooSmall); /* Buffer overflow */ - out[0] = (BYTE)bitStream; - out[1] = (BYTE)(bitStream>>8); - out += 2; - bitStream >>= 16; - bitCount -= 16; - } } - { int count = normalizedCounter[symbol++]; - int const max = (2*threshold-1) - remaining; - remaining -= count < 0 ? -count : count; - count++; /* +1 for extra accuracy */ - if (count>=threshold) - count += max; /* [0..max[ [max..threshold[ (...) [threshold+max 2*threshold[ */ - bitStream += count << bitCount; - bitCount += nbBits; - bitCount -= (count>=1; } - } - if (bitCount>16) { - if ((!writeIsSafe) && (out > oend - 2)) - return ERROR(dstSize_tooSmall); /* Buffer overflow */ - out[0] = (BYTE)bitStream; - out[1] = (BYTE)(bitStream>>8); - out += 2; - bitStream >>= 16; - bitCount -= 16; - } } - - if (remaining != 1) - return ERROR(GENERIC); /* incorrect normalized distribution */ - assert(symbol <= alphabetSize); - - /* flush remaining bitStream */ - if ((!writeIsSafe) && (out > oend - 2)) - return ERROR(dstSize_tooSmall); /* Buffer overflow */ - out[0] = (BYTE)bitStream; - out[1] = (BYTE)(bitStream>>8); - out+= (bitCount+7) /8; - - return (out-ostart); -} - - -size_t FSE_writeNCount (void* buffer, size_t bufferSize, - const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog) -{ - if (tableLog > FSE_MAX_TABLELOG) return ERROR(tableLog_tooLarge); /* Unsupported */ - if (tableLog < FSE_MIN_TABLELOG) return ERROR(GENERIC); /* Unsupported */ - - if (bufferSize < FSE_NCountWriteBound(maxSymbolValue, tableLog)) - return FSE_writeNCount_generic(buffer, bufferSize, normalizedCounter, maxSymbolValue, tableLog, 0); - - return FSE_writeNCount_generic(buffer, bufferSize, normalizedCounter, maxSymbolValue, tableLog, 1 /* write in buffer is safe */); -} - - -/*-************************************************************** -* FSE Compression Code -****************************************************************/ - -FSE_CTable* FSE_createCTable (unsigned maxSymbolValue, unsigned tableLog) -{ - size_t size; - if (tableLog > FSE_TABLELOG_ABSOLUTE_MAX) tableLog = FSE_TABLELOG_ABSOLUTE_MAX; - size = FSE_CTABLE_SIZE_U32 (tableLog, maxSymbolValue) * sizeof(U32); - return (FSE_CTable*)malloc(size); -} - -void FSE_freeCTable (FSE_CTable* ct) { free(ct); } - -/* provides the minimum logSize to safely represent a distribution */ -static unsigned FSE_minTableLog(size_t srcSize, unsigned maxSymbolValue) -{ - U32 minBitsSrc = BIT_highbit32((U32)(srcSize)) + 1; - U32 minBitsSymbols = BIT_highbit32(maxSymbolValue) + 2; - U32 minBits = minBitsSrc < minBitsSymbols ? minBitsSrc : minBitsSymbols; - assert(srcSize > 1); /* Not supported, RLE should be used instead */ - return minBits; -} - -unsigned FSE_optimalTableLog_internal(unsigned maxTableLog, size_t srcSize, unsigned maxSymbolValue, unsigned minus) -{ - U32 maxBitsSrc = BIT_highbit32((U32)(srcSize - 1)) - minus; - U32 tableLog = maxTableLog; - U32 minBits = FSE_minTableLog(srcSize, maxSymbolValue); - assert(srcSize > 1); /* Not supported, RLE should be used instead */ - if (tableLog==0) tableLog = FSE_DEFAULT_TABLELOG; - if (maxBitsSrc < tableLog) tableLog = maxBitsSrc; /* Accuracy can be reduced */ - if (minBits > tableLog) tableLog = minBits; /* Need a minimum to safely represent all symbol values */ - if (tableLog < FSE_MIN_TABLELOG) tableLog = FSE_MIN_TABLELOG; - if (tableLog > FSE_MAX_TABLELOG) tableLog = FSE_MAX_TABLELOG; - return tableLog; -} - -unsigned FSE_optimalTableLog(unsigned maxTableLog, size_t srcSize, unsigned maxSymbolValue) -{ - return FSE_optimalTableLog_internal(maxTableLog, srcSize, maxSymbolValue, 2); -} - - -/* Secondary normalization method. - To be used when primary method fails. */ - -static size_t FSE_normalizeM2(short* norm, U32 tableLog, const unsigned* count, size_t total, U32 maxSymbolValue) -{ - short const NOT_YET_ASSIGNED = -2; - U32 s; - U32 distributed = 0; - U32 ToDistribute; - - /* Init */ - U32 const lowThreshold = (U32)(total >> tableLog); - U32 lowOne = (U32)((total * 3) >> (tableLog + 1)); - - for (s=0; s<=maxSymbolValue; s++) { - if (count[s] == 0) { - norm[s]=0; - continue; - } - if (count[s] <= lowThreshold) { - norm[s] = -1; - distributed++; - total -= count[s]; - continue; - } - if (count[s] <= lowOne) { - norm[s] = 1; - distributed++; - total -= count[s]; - continue; - } - - norm[s]=NOT_YET_ASSIGNED; - } - ToDistribute = (1 << tableLog) - distributed; - - if (ToDistribute == 0) - return 0; - - if ((total / ToDistribute) > lowOne) { - /* risk of rounding to zero */ - lowOne = (U32)((total * 3) / (ToDistribute * 2)); - for (s=0; s<=maxSymbolValue; s++) { - if ((norm[s] == NOT_YET_ASSIGNED) && (count[s] <= lowOne)) { - norm[s] = 1; - distributed++; - total -= count[s]; - continue; - } } - ToDistribute = (1 << tableLog) - distributed; - } - - if (distributed == maxSymbolValue+1) { - /* all values are pretty poor; - probably incompressible data (should have already been detected); - find max, then give all remaining points to max */ - U32 maxV = 0, maxC = 0; - for (s=0; s<=maxSymbolValue; s++) - if (count[s] > maxC) { maxV=s; maxC=count[s]; } - norm[maxV] += (short)ToDistribute; - return 0; - } - - if (total == 0) { - /* all of the symbols were low enough for the lowOne or lowThreshold */ - for (s=0; ToDistribute > 0; s = (s+1)%(maxSymbolValue+1)) - if (norm[s] > 0) { ToDistribute--; norm[s]++; } - return 0; - } - - { U64 const vStepLog = 62 - tableLog; - U64 const mid = (1ULL << (vStepLog-1)) - 1; - U64 const rStep = ((((U64)1<> vStepLog); - U32 const sEnd = (U32)(end >> vStepLog); - U32 const weight = sEnd - sStart; - if (weight < 1) - return ERROR(GENERIC); - norm[s] = (short)weight; - tmpTotal = end; - } } } - - return 0; -} - - -size_t FSE_normalizeCount (short* normalizedCounter, unsigned tableLog, - const unsigned* count, size_t total, - unsigned maxSymbolValue) -{ - /* Sanity checks */ - if (tableLog==0) tableLog = FSE_DEFAULT_TABLELOG; - if (tableLog < FSE_MIN_TABLELOG) return ERROR(GENERIC); /* Unsupported size */ - if (tableLog > FSE_MAX_TABLELOG) return ERROR(tableLog_tooLarge); /* Unsupported size */ - if (tableLog < FSE_minTableLog(total, maxSymbolValue)) return ERROR(GENERIC); /* Too small tableLog, compression potentially impossible */ - - { static U32 const rtbTable[] = { 0, 473195, 504333, 520860, 550000, 700000, 750000, 830000 }; - U64 const scale = 62 - tableLog; - U64 const step = ((U64)1<<62) / total; /* <== here, one division ! */ - U64 const vStep = 1ULL<<(scale-20); - int stillToDistribute = 1<> tableLog); - - for (s=0; s<=maxSymbolValue; s++) { - if (count[s] == total) return 0; /* rle special case */ - if (count[s] == 0) { normalizedCounter[s]=0; continue; } - if (count[s] <= lowThreshold) { - normalizedCounter[s] = -1; - stillToDistribute--; - } else { - short proba = (short)((count[s]*step) >> scale); - if (proba<8) { - U64 restToBeat = vStep * rtbTable[proba]; - proba += (count[s]*step) - ((U64)proba< restToBeat; - } - if (proba > largestP) { largestP=proba; largest=s; } - normalizedCounter[s] = proba; - stillToDistribute -= proba; - } } - if (-stillToDistribute >= (normalizedCounter[largest] >> 1)) { - /* corner case, need another normalization method */ - size_t const errorCode = FSE_normalizeM2(normalizedCounter, tableLog, count, total, maxSymbolValue); - if (FSE_isError(errorCode)) return errorCode; - } - else normalizedCounter[largest] += (short)stillToDistribute; - } - -#if 0 - { /* Print Table (debug) */ - U32 s; - U32 nTotal = 0; - for (s=0; s<=maxSymbolValue; s++) - RAWLOG(2, "%3i: %4i \n", s, normalizedCounter[s]); - for (s=0; s<=maxSymbolValue; s++) - nTotal += abs(normalizedCounter[s]); - if (nTotal != (1U<>1); /* assumption : tableLog >= 1 */ - FSE_symbolCompressionTransform* const symbolTT = (FSE_symbolCompressionTransform*) (FSCT); - unsigned s; - - /* Sanity checks */ - if (nbBits < 1) return ERROR(GENERIC); /* min size */ - - /* header */ - tableU16[-2] = (U16) nbBits; - tableU16[-1] = (U16) maxSymbolValue; - - /* Build table */ - for (s=0; s FSE_MAX_TABLELOG*4+7 ) && (srcSize & 2)) { /* test bit 2 */ - FSE_encodeSymbol(&bitC, &CState2, *--ip); - FSE_encodeSymbol(&bitC, &CState1, *--ip); - FSE_FLUSHBITS(&bitC); - } - - /* 2 or 4 encoding per loop */ - while ( ip>istart ) { - - FSE_encodeSymbol(&bitC, &CState2, *--ip); - - if (sizeof(bitC.bitContainer)*8 < FSE_MAX_TABLELOG*2+7 ) /* this test must be static */ - FSE_FLUSHBITS(&bitC); - - FSE_encodeSymbol(&bitC, &CState1, *--ip); - - if (sizeof(bitC.bitContainer)*8 > FSE_MAX_TABLELOG*4+7 ) { /* this test must be static */ - FSE_encodeSymbol(&bitC, &CState2, *--ip); - FSE_encodeSymbol(&bitC, &CState1, *--ip); - } - - FSE_FLUSHBITS(&bitC); - } - - FSE_flushCState(&bitC, &CState2); - FSE_flushCState(&bitC, &CState1); - return BIT_closeCStream(&bitC); -} - -size_t FSE_compress_usingCTable (void* dst, size_t dstSize, - const void* src, size_t srcSize, - const FSE_CTable* ct) -{ - unsigned const fast = (dstSize >= FSE_BLOCKBOUND(srcSize)); - - if (fast) - return FSE_compress_usingCTable_generic(dst, dstSize, src, srcSize, ct, 1); - else - return FSE_compress_usingCTable_generic(dst, dstSize, src, srcSize, ct, 0); -} - - -size_t FSE_compressBound(size_t size) { return FSE_COMPRESSBOUND(size); } - -/* FSE_compress_wksp() : - * Same as FSE_compress2(), but using an externally allocated scratch buffer (`workSpace`). - * `wkspSize` size must be `(1< not compressible */ - if (maxCount < (srcSize >> 7)) return 0; /* Heuristic : not compressible enough */ - } - - tableLog = FSE_optimalTableLog(tableLog, srcSize, maxSymbolValue); - CHECK_F( FSE_normalizeCount(norm, tableLog, count, srcSize, maxSymbolValue) ); - - /* Write table description header */ - { CHECK_V_F(nc_err, FSE_writeNCount(op, oend-op, norm, maxSymbolValue, tableLog) ); - op += nc_err; - } - - /* Compress */ - CHECK_F( FSE_buildCTable_wksp(CTable, norm, maxSymbolValue, tableLog, scratchBuffer, scratchBufferSize) ); - { CHECK_V_F(cSize, FSE_compress_usingCTable(op, oend - op, src, srcSize, CTable) ); - if (cSize == 0) return 0; /* not enough space for compressed data */ - op += cSize; - } - - /* check compressibility */ - if ( (size_t)(op-ostart) >= srcSize-1 ) return 0; - - return op-ostart; -} - -typedef struct { - FSE_CTable CTable_max[FSE_CTABLE_SIZE_U32(FSE_MAX_TABLELOG, FSE_MAX_SYMBOL_VALUE)]; - BYTE scratchBuffer[1 << FSE_MAX_TABLELOG]; -} fseWkspMax_t; - -size_t FSE_compress2 (void* dst, size_t dstCapacity, const void* src, size_t srcSize, unsigned maxSymbolValue, unsigned tableLog) -{ - fseWkspMax_t scratchBuffer; - DEBUG_STATIC_ASSERT(sizeof(scratchBuffer) >= FSE_WKSP_SIZE_U32(FSE_MAX_TABLELOG, FSE_MAX_SYMBOL_VALUE)); /* compilation failures here means scratchBuffer is not large enough */ - if (tableLog > FSE_MAX_TABLELOG) return ERROR(tableLog_tooLarge); - return FSE_compress_wksp(dst, dstCapacity, src, srcSize, maxSymbolValue, tableLog, &scratchBuffer, sizeof(scratchBuffer)); -} - -size_t FSE_compress (void* dst, size_t dstCapacity, const void* src, size_t srcSize) -{ - return FSE_compress2(dst, dstCapacity, src, srcSize, FSE_MAX_SYMBOL_VALUE, FSE_DEFAULT_TABLELOG); -} - - -#endif /* FSE_COMMONDEFS_ONLY */ -/**** ended inlining compress/fse_compress.c ****/ -/**** start inlining compress/hist.c ****/ -/* ****************************************************************** - * hist : Histogram functions - * part of Finite State Entropy project - * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. - * - * You can contact the author at : - * - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy - * - Public forum : https://groups.google.com/forum/#!forum/lz4c - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. -****************************************************************** */ - -/* --- dependencies --- */ -/**** skipping file: ../common/mem.h ****/ -/**** skipping file: ../common/debug.h ****/ -/**** skipping file: ../common/error_private.h ****/ -/**** skipping file: hist.h ****/ - - -/* --- Error management --- */ -unsigned HIST_isError(size_t code) { return ERR_isError(code); } - -/*-************************************************************** - * Histogram functions - ****************************************************************/ -unsigned HIST_count_simple(unsigned* count, unsigned* maxSymbolValuePtr, - const void* src, size_t srcSize) -{ - const BYTE* ip = (const BYTE*)src; - const BYTE* const end = ip + srcSize; - unsigned maxSymbolValue = *maxSymbolValuePtr; - unsigned largestCount=0; - - memset(count, 0, (maxSymbolValue+1) * sizeof(*count)); - if (srcSize==0) { *maxSymbolValuePtr = 0; return 0; } - - while (ip largestCount) largestCount = count[s]; - } - - return largestCount; -} - -typedef enum { trustInput, checkMaxSymbolValue } HIST_checkInput_e; - -/* HIST_count_parallel_wksp() : - * store histogram into 4 intermediate tables, recombined at the end. - * this design makes better use of OoO cpus, - * and is noticeably faster when some values are heavily repeated. - * But it needs some additional workspace for intermediate tables. - * `workSpace` size must be a table of size >= HIST_WKSP_SIZE_U32. - * @return : largest histogram frequency, - * or an error code (notably when histogram would be larger than *maxSymbolValuePtr). */ -static size_t HIST_count_parallel_wksp( - unsigned* count, unsigned* maxSymbolValuePtr, - const void* source, size_t sourceSize, - HIST_checkInput_e check, - U32* const workSpace) -{ - const BYTE* ip = (const BYTE*)source; - const BYTE* const iend = ip+sourceSize; - unsigned maxSymbolValue = *maxSymbolValuePtr; - unsigned max=0; - U32* const Counting1 = workSpace; - U32* const Counting2 = Counting1 + 256; - U32* const Counting3 = Counting2 + 256; - U32* const Counting4 = Counting3 + 256; - - memset(workSpace, 0, 4*256*sizeof(unsigned)); - - /* safety checks */ - if (!sourceSize) { - memset(count, 0, maxSymbolValue + 1); - *maxSymbolValuePtr = 0; - return 0; - } - if (!maxSymbolValue) maxSymbolValue = 255; /* 0 == default */ - - /* by stripes of 16 bytes */ - { U32 cached = MEM_read32(ip); ip += 4; - while (ip < iend-15) { - U32 c = cached; cached = MEM_read32(ip); ip += 4; - Counting1[(BYTE) c ]++; - Counting2[(BYTE)(c>>8) ]++; - Counting3[(BYTE)(c>>16)]++; - Counting4[ c>>24 ]++; - c = cached; cached = MEM_read32(ip); ip += 4; - Counting1[(BYTE) c ]++; - Counting2[(BYTE)(c>>8) ]++; - Counting3[(BYTE)(c>>16)]++; - Counting4[ c>>24 ]++; - c = cached; cached = MEM_read32(ip); ip += 4; - Counting1[(BYTE) c ]++; - Counting2[(BYTE)(c>>8) ]++; - Counting3[(BYTE)(c>>16)]++; - Counting4[ c>>24 ]++; - c = cached; cached = MEM_read32(ip); ip += 4; - Counting1[(BYTE) c ]++; - Counting2[(BYTE)(c>>8) ]++; - Counting3[(BYTE)(c>>16)]++; - Counting4[ c>>24 ]++; - } - ip-=4; - } - - /* finish last symbols */ - while (ipmaxSymbolValue; s--) { - Counting1[s] += Counting2[s] + Counting3[s] + Counting4[s]; - if (Counting1[s]) return ERROR(maxSymbolValue_tooSmall); - } } - - { U32 s; - if (maxSymbolValue > 255) maxSymbolValue = 255; - for (s=0; s<=maxSymbolValue; s++) { - count[s] = Counting1[s] + Counting2[s] + Counting3[s] + Counting4[s]; - if (count[s] > max) max = count[s]; - } } - - while (!count[maxSymbolValue]) maxSymbolValue--; - *maxSymbolValuePtr = maxSymbolValue; - return (size_t)max; -} - -/* HIST_countFast_wksp() : - * Same as HIST_countFast(), but using an externally provided scratch buffer. - * `workSpace` is a writable buffer which must be 4-bytes aligned, - * `workSpaceSize` must be >= HIST_WKSP_SIZE - */ -size_t HIST_countFast_wksp(unsigned* count, unsigned* maxSymbolValuePtr, - const void* source, size_t sourceSize, - void* workSpace, size_t workSpaceSize) -{ - if (sourceSize < 1500) /* heuristic threshold */ - return HIST_count_simple(count, maxSymbolValuePtr, source, sourceSize); - if ((size_t)workSpace & 3) return ERROR(GENERIC); /* must be aligned on 4-bytes boundaries */ - if (workSpaceSize < HIST_WKSP_SIZE) return ERROR(workSpace_tooSmall); - return HIST_count_parallel_wksp(count, maxSymbolValuePtr, source, sourceSize, trustInput, (U32*)workSpace); -} - -/* fast variant (unsafe : won't check if src contains values beyond count[] limit) */ -size_t HIST_countFast(unsigned* count, unsigned* maxSymbolValuePtr, - const void* source, size_t sourceSize) -{ - unsigned tmpCounters[HIST_WKSP_SIZE_U32]; - return HIST_countFast_wksp(count, maxSymbolValuePtr, source, sourceSize, tmpCounters, sizeof(tmpCounters)); -} - -/* HIST_count_wksp() : - * Same as HIST_count(), but using an externally provided scratch buffer. - * `workSpace` size must be table of >= HIST_WKSP_SIZE_U32 unsigned */ -size_t HIST_count_wksp(unsigned* count, unsigned* maxSymbolValuePtr, - const void* source, size_t sourceSize, - void* workSpace, size_t workSpaceSize) -{ - if ((size_t)workSpace & 3) return ERROR(GENERIC); /* must be aligned on 4-bytes boundaries */ - if (workSpaceSize < HIST_WKSP_SIZE) return ERROR(workSpace_tooSmall); - if (*maxSymbolValuePtr < 255) - return HIST_count_parallel_wksp(count, maxSymbolValuePtr, source, sourceSize, checkMaxSymbolValue, (U32*)workSpace); - *maxSymbolValuePtr = 255; - return HIST_countFast_wksp(count, maxSymbolValuePtr, source, sourceSize, workSpace, workSpaceSize); -} - -size_t HIST_count(unsigned* count, unsigned* maxSymbolValuePtr, - const void* src, size_t srcSize) -{ - unsigned tmpCounters[HIST_WKSP_SIZE_U32]; - return HIST_count_wksp(count, maxSymbolValuePtr, src, srcSize, tmpCounters, sizeof(tmpCounters)); -} -/**** ended inlining compress/hist.c ****/ -/**** start inlining compress/huf_compress.c ****/ -/* ****************************************************************** - * Huffman encoder, part of New Generation Entropy library - * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. - * - * You can contact the author at : - * - FSE+HUF source repository : https://github.com/Cyan4973/FiniteStateEntropy - * - Public forum : https://groups.google.com/forum/#!forum/lz4c - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. -****************************************************************** */ - -/* ************************************************************** -* Compiler specifics -****************************************************************/ -#ifdef _MSC_VER /* Visual Studio */ -# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */ -#endif - - -/* ************************************************************** -* Includes -****************************************************************/ -#include /* memcpy, memset */ -#include /* printf (debug) */ -/**** skipping file: ../common/compiler.h ****/ -/**** skipping file: ../common/bitstream.h ****/ -/**** skipping file: hist.h ****/ -#define FSE_STATIC_LINKING_ONLY /* FSE_optimalTableLog_internal */ -/**** skipping file: ../common/fse.h ****/ -#define HUF_STATIC_LINKING_ONLY -/**** skipping file: ../common/huf.h ****/ -/**** skipping file: ../common/error_private.h ****/ - - -/* ************************************************************** -* Error Management -****************************************************************/ -#define HUF_isError ERR_isError -#define HUF_STATIC_ASSERT(c) DEBUG_STATIC_ASSERT(c) /* use only *after* variable declarations */ - - -/* ************************************************************** -* Utils -****************************************************************/ -unsigned HUF_optimalTableLog(unsigned maxTableLog, size_t srcSize, unsigned maxSymbolValue) -{ - return FSE_optimalTableLog_internal(maxTableLog, srcSize, maxSymbolValue, 1); -} - - -/* ******************************************************* -* HUF : Huffman block compression -*********************************************************/ -/* HUF_compressWeights() : - * Same as FSE_compress(), but dedicated to huff0's weights compression. - * The use case needs much less stack memory. - * Note : all elements within weightTable are supposed to be <= HUF_TABLELOG_MAX. - */ -#define MAX_FSE_TABLELOG_FOR_HUFF_HEADER 6 -static size_t HUF_compressWeights (void* dst, size_t dstSize, const void* weightTable, size_t wtSize) -{ - BYTE* const ostart = (BYTE*) dst; - BYTE* op = ostart; - BYTE* const oend = ostart + dstSize; - - unsigned maxSymbolValue = HUF_TABLELOG_MAX; - U32 tableLog = MAX_FSE_TABLELOG_FOR_HUFF_HEADER; - - FSE_CTable CTable[FSE_CTABLE_SIZE_U32(MAX_FSE_TABLELOG_FOR_HUFF_HEADER, HUF_TABLELOG_MAX)]; - BYTE scratchBuffer[1< not compressible */ - } - - tableLog = FSE_optimalTableLog(tableLog, wtSize, maxSymbolValue); - CHECK_F( FSE_normalizeCount(norm, tableLog, count, wtSize, maxSymbolValue) ); - - /* Write table description header */ - { CHECK_V_F(hSize, FSE_writeNCount(op, (size_t)(oend-op), norm, maxSymbolValue, tableLog) ); - op += hSize; - } - - /* Compress */ - CHECK_F( FSE_buildCTable_wksp(CTable, norm, maxSymbolValue, tableLog, scratchBuffer, sizeof(scratchBuffer)) ); - { CHECK_V_F(cSize, FSE_compress_usingCTable(op, (size_t)(oend - op), weightTable, wtSize, CTable) ); - if (cSize == 0) return 0; /* not enough space for compressed data */ - op += cSize; - } - - return (size_t)(op-ostart); -} - - -struct HUF_CElt_s { - U16 val; - BYTE nbBits; -}; /* typedef'd to HUF_CElt within "huf.h" */ - -/*! HUF_writeCTable() : - `CTable` : Huffman tree to save, using huf representation. - @return : size of saved CTable */ -size_t HUF_writeCTable (void* dst, size_t maxDstSize, - const HUF_CElt* CTable, unsigned maxSymbolValue, unsigned huffLog) -{ - BYTE bitsToWeight[HUF_TABLELOG_MAX + 1]; /* precomputed conversion table */ - BYTE huffWeight[HUF_SYMBOLVALUE_MAX]; - BYTE* op = (BYTE*)dst; - U32 n; - - /* check conditions */ - if (maxSymbolValue > HUF_SYMBOLVALUE_MAX) return ERROR(maxSymbolValue_tooLarge); - - /* convert to weight */ - bitsToWeight[0] = 0; - for (n=1; n1) & (hSize < maxSymbolValue/2)) { /* FSE compressed */ - op[0] = (BYTE)hSize; - return hSize+1; - } } - - /* write raw values as 4-bits (max : 15) */ - if (maxSymbolValue > (256-128)) return ERROR(GENERIC); /* should not happen : likely means source cannot be compressed */ - if (((maxSymbolValue+1)/2) + 1 > maxDstSize) return ERROR(dstSize_tooSmall); /* not enough space within dst buffer */ - op[0] = (BYTE)(128 /*special case*/ + (maxSymbolValue-1)); - huffWeight[maxSymbolValue] = 0; /* to be sure it doesn't cause msan issue in final combination */ - for (n=0; n HUF_TABLELOG_MAX) return ERROR(tableLog_tooLarge); - if (nbSymbols > *maxSymbolValuePtr+1) return ERROR(maxSymbolValue_tooSmall); - - /* Prepare base value per rank */ - { U32 n, nextRankStart = 0; - for (n=1; n<=tableLog; n++) { - U32 current = nextRankStart; - nextRankStart += (rankVal[n] << (n-1)); - rankVal[n] = current; - } } - - /* fill nbBits */ - *hasZeroWeights = 0; - { U32 n; for (n=0; nn=tableLog+1 */ - U16 valPerRank[HUF_TABLELOG_MAX+2] = {0}; - { U32 n; for (n=0; n0; n--) { /* start at n=tablelog <-> w=1 */ - valPerRank[n] = min; /* get starting value within each rank */ - min += nbPerRank[n]; - min >>= 1; - } } - /* assign value within rank, symbol order */ - { U32 n; for (n=0; n maxNbBits */ - - /* there are several too large elements (at least >= 2) */ - { int totalCost = 0; - const U32 baseCost = 1 << (largestBits - maxNbBits); - int n = (int)lastNonNull; - - while (huffNode[n].nbBits > maxNbBits) { - totalCost += baseCost - (1 << (largestBits - huffNode[n].nbBits)); - huffNode[n].nbBits = (BYTE)maxNbBits; - n --; - } /* n stops at huffNode[n].nbBits <= maxNbBits */ - while (huffNode[n].nbBits == maxNbBits) n--; /* n end at index of smallest symbol using < maxNbBits */ - - /* renorm totalCost */ - totalCost >>= (largestBits - maxNbBits); /* note : totalCost is necessarily a multiple of baseCost */ - - /* repay normalized cost */ - { U32 const noSymbol = 0xF0F0F0F0; - U32 rankLast[HUF_TABLELOG_MAX+2]; - - /* Get pos of last (smallest) symbol per rank */ - memset(rankLast, 0xF0, sizeof(rankLast)); - { U32 currentNbBits = maxNbBits; - int pos; - for (pos=n ; pos >= 0; pos--) { - if (huffNode[pos].nbBits >= currentNbBits) continue; - currentNbBits = huffNode[pos].nbBits; /* < maxNbBits */ - rankLast[maxNbBits-currentNbBits] = (U32)pos; - } } - - while (totalCost > 0) { - U32 nBitsToDecrease = BIT_highbit32((U32)totalCost) + 1; - for ( ; nBitsToDecrease > 1; nBitsToDecrease--) { - U32 const highPos = rankLast[nBitsToDecrease]; - U32 const lowPos = rankLast[nBitsToDecrease-1]; - if (highPos == noSymbol) continue; - if (lowPos == noSymbol) break; - { U32 const highTotal = huffNode[highPos].count; - U32 const lowTotal = 2 * huffNode[lowPos].count; - if (highTotal <= lowTotal) break; - } } - /* only triggered when no more rank 1 symbol left => find closest one (note : there is necessarily at least one !) */ - /* HUF_MAX_TABLELOG test just to please gcc 5+; but it should not be necessary */ - while ((nBitsToDecrease<=HUF_TABLELOG_MAX) && (rankLast[nBitsToDecrease] == noSymbol)) - nBitsToDecrease ++; - totalCost -= 1 << (nBitsToDecrease-1); - if (rankLast[nBitsToDecrease-1] == noSymbol) - rankLast[nBitsToDecrease-1] = rankLast[nBitsToDecrease]; /* this rank is no longer empty */ - huffNode[rankLast[nBitsToDecrease]].nbBits ++; - if (rankLast[nBitsToDecrease] == 0) /* special case, reached largest symbol */ - rankLast[nBitsToDecrease] = noSymbol; - else { - rankLast[nBitsToDecrease]--; - if (huffNode[rankLast[nBitsToDecrease]].nbBits != maxNbBits-nBitsToDecrease) - rankLast[nBitsToDecrease] = noSymbol; /* this rank is now empty */ - } } /* while (totalCost > 0) */ - - while (totalCost < 0) { /* Sometimes, cost correction overshoot */ - if (rankLast[1] == noSymbol) { /* special case : no rank 1 symbol (using maxNbBits-1); let's create one from largest rank 0 (using maxNbBits) */ - while (huffNode[n].nbBits == maxNbBits) n--; - huffNode[n+1].nbBits--; - assert(n >= 0); - rankLast[1] = (U32)(n+1); - totalCost++; - continue; - } - huffNode[ rankLast[1] + 1 ].nbBits--; - rankLast[1]++; - totalCost ++; - } } } /* there are several too large elements (at least >= 2) */ - - return maxNbBits; -} - -typedef struct { - U32 base; - U32 current; -} rankPos; - -typedef nodeElt huffNodeTable[HUF_CTABLE_WORKSPACE_SIZE_U32]; - -#define RANK_POSITION_TABLE_SIZE 32 - -typedef struct { - huffNodeTable huffNodeTbl; - rankPos rankPosition[RANK_POSITION_TABLE_SIZE]; -} HUF_buildCTable_wksp_tables; - -static void HUF_sort(nodeElt* huffNode, const unsigned* count, U32 maxSymbolValue, rankPos* rankPosition) -{ - U32 n; - - memset(rankPosition, 0, sizeof(*rankPosition) * RANK_POSITION_TABLE_SIZE); - for (n=0; n<=maxSymbolValue; n++) { - U32 r = BIT_highbit32(count[n] + 1); - rankPosition[r].base ++; - } - for (n=30; n>0; n--) rankPosition[n-1].base += rankPosition[n].base; - for (n=0; n<32; n++) rankPosition[n].current = rankPosition[n].base; - for (n=0; n<=maxSymbolValue; n++) { - U32 const c = count[n]; - U32 const r = BIT_highbit32(c+1) + 1; - U32 pos = rankPosition[r].current++; - while ((pos > rankPosition[r].base) && (c > huffNode[pos-1].count)) { - huffNode[pos] = huffNode[pos-1]; - pos--; - } - huffNode[pos].count = c; - huffNode[pos].byte = (BYTE)n; - } -} - - -/** HUF_buildCTable_wksp() : - * Same as HUF_buildCTable(), but using externally allocated scratch buffer. - * `workSpace` must be aligned on 4-bytes boundaries, and be at least as large as sizeof(HUF_buildCTable_wksp_tables). - */ -#define STARTNODE (HUF_SYMBOLVALUE_MAX+1) - -size_t HUF_buildCTable_wksp (HUF_CElt* tree, const unsigned* count, U32 maxSymbolValue, U32 maxNbBits, void* workSpace, size_t wkspSize) -{ - HUF_buildCTable_wksp_tables* const wksp_tables = (HUF_buildCTable_wksp_tables*)workSpace; - nodeElt* const huffNode0 = wksp_tables->huffNodeTbl; - nodeElt* const huffNode = huffNode0+1; - int nonNullRank; - int lowS, lowN; - int nodeNb = STARTNODE; - int n, nodeRoot; - - /* safety checks */ - if (((size_t)workSpace & 3) != 0) return ERROR(GENERIC); /* must be aligned on 4-bytes boundaries */ - if (wkspSize < sizeof(HUF_buildCTable_wksp_tables)) - return ERROR(workSpace_tooSmall); - if (maxNbBits == 0) maxNbBits = HUF_TABLELOG_DEFAULT; - if (maxSymbolValue > HUF_SYMBOLVALUE_MAX) - return ERROR(maxSymbolValue_tooLarge); - memset(huffNode0, 0, sizeof(huffNodeTable)); - - /* sort, decreasing order */ - HUF_sort(huffNode, count, maxSymbolValue, wksp_tables->rankPosition); - - /* init for parents */ - nonNullRank = (int)maxSymbolValue; - while(huffNode[nonNullRank].count == 0) nonNullRank--; - lowS = nonNullRank; nodeRoot = nodeNb + lowS - 1; lowN = nodeNb; - huffNode[nodeNb].count = huffNode[lowS].count + huffNode[lowS-1].count; - huffNode[lowS].parent = huffNode[lowS-1].parent = (U16)nodeNb; - nodeNb++; lowS-=2; - for (n=nodeNb; n<=nodeRoot; n++) huffNode[n].count = (U32)(1U<<30); - huffNode0[0].count = (U32)(1U<<31); /* fake entry, strong barrier */ - - /* create parents */ - while (nodeNb <= nodeRoot) { - int const n1 = (huffNode[lowS].count < huffNode[lowN].count) ? lowS-- : lowN++; - int const n2 = (huffNode[lowS].count < huffNode[lowN].count) ? lowS-- : lowN++; - huffNode[nodeNb].count = huffNode[n1].count + huffNode[n2].count; - huffNode[n1].parent = huffNode[n2].parent = (U16)nodeNb; - nodeNb++; - } - - /* distribute weights (unlimited tree height) */ - huffNode[nodeRoot].nbBits = 0; - for (n=nodeRoot-1; n>=STARTNODE; n--) - huffNode[n].nbBits = huffNode[ huffNode[n].parent ].nbBits + 1; - for (n=0; n<=nonNullRank; n++) - huffNode[n].nbBits = huffNode[ huffNode[n].parent ].nbBits + 1; - - /* enforce maxTableLog */ - maxNbBits = HUF_setMaxHeight(huffNode, (U32)nonNullRank, maxNbBits); - - /* fill result into tree (val, nbBits) */ - { U16 nbPerRank[HUF_TABLELOG_MAX+1] = {0}; - U16 valPerRank[HUF_TABLELOG_MAX+1] = {0}; - int const alphabetSize = (int)(maxSymbolValue + 1); - if (maxNbBits > HUF_TABLELOG_MAX) return ERROR(GENERIC); /* check fit into table */ - for (n=0; n<=nonNullRank; n++) - nbPerRank[huffNode[n].nbBits]++; - /* determine stating value per rank */ - { U16 min = 0; - for (n=(int)maxNbBits; n>0; n--) { - valPerRank[n] = min; /* get starting value within each rank */ - min += nbPerRank[n]; - min >>= 1; - } } - for (n=0; n> 3; -} - -int HUF_validateCTable(const HUF_CElt* CTable, const unsigned* count, unsigned maxSymbolValue) { - int bad = 0; - int s; - for (s = 0; s <= (int)maxSymbolValue; ++s) { - bad |= (count[s] != 0) & (CTable[s].nbBits == 0); - } - return !bad; -} - -size_t HUF_compressBound(size_t size) { return HUF_COMPRESSBOUND(size); } - -FORCE_INLINE_TEMPLATE void -HUF_encodeSymbol(BIT_CStream_t* bitCPtr, U32 symbol, const HUF_CElt* CTable) -{ - BIT_addBitsFast(bitCPtr, CTable[symbol].val, CTable[symbol].nbBits); -} - -#define HUF_FLUSHBITS(s) BIT_flushBits(s) - -#define HUF_FLUSHBITS_1(stream) \ - if (sizeof((stream)->bitContainer)*8 < HUF_TABLELOG_MAX*2+7) HUF_FLUSHBITS(stream) - -#define HUF_FLUSHBITS_2(stream) \ - if (sizeof((stream)->bitContainer)*8 < HUF_TABLELOG_MAX*4+7) HUF_FLUSHBITS(stream) - -FORCE_INLINE_TEMPLATE size_t -HUF_compress1X_usingCTable_internal_body(void* dst, size_t dstSize, - const void* src, size_t srcSize, - const HUF_CElt* CTable) -{ - const BYTE* ip = (const BYTE*) src; - BYTE* const ostart = (BYTE*)dst; - BYTE* const oend = ostart + dstSize; - BYTE* op = ostart; - size_t n; - BIT_CStream_t bitC; - - /* init */ - if (dstSize < 8) return 0; /* not enough space to compress */ - { size_t const initErr = BIT_initCStream(&bitC, op, (size_t)(oend-op)); - if (HUF_isError(initErr)) return 0; } - - n = srcSize & ~3; /* join to mod 4 */ - switch (srcSize & 3) - { - case 3 : HUF_encodeSymbol(&bitC, ip[n+ 2], CTable); - HUF_FLUSHBITS_2(&bitC); - /* fall-through */ - case 2 : HUF_encodeSymbol(&bitC, ip[n+ 1], CTable); - HUF_FLUSHBITS_1(&bitC); - /* fall-through */ - case 1 : HUF_encodeSymbol(&bitC, ip[n+ 0], CTable); - HUF_FLUSHBITS(&bitC); - /* fall-through */ - case 0 : /* fall-through */ - default: break; - } - - for (; n>0; n-=4) { /* note : n&3==0 at this stage */ - HUF_encodeSymbol(&bitC, ip[n- 1], CTable); - HUF_FLUSHBITS_1(&bitC); - HUF_encodeSymbol(&bitC, ip[n- 2], CTable); - HUF_FLUSHBITS_2(&bitC); - HUF_encodeSymbol(&bitC, ip[n- 3], CTable); - HUF_FLUSHBITS_1(&bitC); - HUF_encodeSymbol(&bitC, ip[n- 4], CTable); - HUF_FLUSHBITS(&bitC); - } - - return BIT_closeCStream(&bitC); -} - -#if DYNAMIC_BMI2 - -static TARGET_ATTRIBUTE("bmi2") size_t -HUF_compress1X_usingCTable_internal_bmi2(void* dst, size_t dstSize, - const void* src, size_t srcSize, - const HUF_CElt* CTable) -{ - return HUF_compress1X_usingCTable_internal_body(dst, dstSize, src, srcSize, CTable); -} - -static size_t -HUF_compress1X_usingCTable_internal_default(void* dst, size_t dstSize, - const void* src, size_t srcSize, - const HUF_CElt* CTable) -{ - return HUF_compress1X_usingCTable_internal_body(dst, dstSize, src, srcSize, CTable); -} - -static size_t -HUF_compress1X_usingCTable_internal(void* dst, size_t dstSize, - const void* src, size_t srcSize, - const HUF_CElt* CTable, const int bmi2) -{ - if (bmi2) { - return HUF_compress1X_usingCTable_internal_bmi2(dst, dstSize, src, srcSize, CTable); - } - return HUF_compress1X_usingCTable_internal_default(dst, dstSize, src, srcSize, CTable); -} - -#else - -static size_t -HUF_compress1X_usingCTable_internal(void* dst, size_t dstSize, - const void* src, size_t srcSize, - const HUF_CElt* CTable, const int bmi2) -{ - (void)bmi2; - return HUF_compress1X_usingCTable_internal_body(dst, dstSize, src, srcSize, CTable); -} - -#endif - -size_t HUF_compress1X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable) -{ - return HUF_compress1X_usingCTable_internal(dst, dstSize, src, srcSize, CTable, /* bmi2 */ 0); -} - - -static size_t -HUF_compress4X_usingCTable_internal(void* dst, size_t dstSize, - const void* src, size_t srcSize, - const HUF_CElt* CTable, int bmi2) -{ - size_t const segmentSize = (srcSize+3)/4; /* first 3 segments */ - const BYTE* ip = (const BYTE*) src; - const BYTE* const iend = ip + srcSize; - BYTE* const ostart = (BYTE*) dst; - BYTE* const oend = ostart + dstSize; - BYTE* op = ostart; - - if (dstSize < 6 + 1 + 1 + 1 + 8) return 0; /* minimum space to compress successfully */ - if (srcSize < 12) return 0; /* no saving possible : too small input */ - op += 6; /* jumpTable */ - - assert(op <= oend); - { CHECK_V_F(cSize, HUF_compress1X_usingCTable_internal(op, (size_t)(oend-op), ip, segmentSize, CTable, bmi2) ); - if (cSize==0) return 0; - assert(cSize <= 65535); - MEM_writeLE16(ostart, (U16)cSize); - op += cSize; - } - - ip += segmentSize; - assert(op <= oend); - { CHECK_V_F(cSize, HUF_compress1X_usingCTable_internal(op, (size_t)(oend-op), ip, segmentSize, CTable, bmi2) ); - if (cSize==0) return 0; - assert(cSize <= 65535); - MEM_writeLE16(ostart+2, (U16)cSize); - op += cSize; - } - - ip += segmentSize; - assert(op <= oend); - { CHECK_V_F(cSize, HUF_compress1X_usingCTable_internal(op, (size_t)(oend-op), ip, segmentSize, CTable, bmi2) ); - if (cSize==0) return 0; - assert(cSize <= 65535); - MEM_writeLE16(ostart+4, (U16)cSize); - op += cSize; - } - - ip += segmentSize; - assert(op <= oend); - assert(ip <= iend); - { CHECK_V_F(cSize, HUF_compress1X_usingCTable_internal(op, (size_t)(oend-op), ip, (size_t)(iend-ip), CTable, bmi2) ); - if (cSize==0) return 0; - op += cSize; - } - - return (size_t)(op-ostart); -} - -size_t HUF_compress4X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable) -{ - return HUF_compress4X_usingCTable_internal(dst, dstSize, src, srcSize, CTable, /* bmi2 */ 0); -} - -typedef enum { HUF_singleStream, HUF_fourStreams } HUF_nbStreams_e; - -static size_t HUF_compressCTable_internal( - BYTE* const ostart, BYTE* op, BYTE* const oend, - const void* src, size_t srcSize, - HUF_nbStreams_e nbStreams, const HUF_CElt* CTable, const int bmi2) -{ - size_t const cSize = (nbStreams==HUF_singleStream) ? - HUF_compress1X_usingCTable_internal(op, (size_t)(oend - op), src, srcSize, CTable, bmi2) : - HUF_compress4X_usingCTable_internal(op, (size_t)(oend - op), src, srcSize, CTable, bmi2); - if (HUF_isError(cSize)) { return cSize; } - if (cSize==0) { return 0; } /* uncompressible */ - op += cSize; - /* check compressibility */ - assert(op >= ostart); - if ((size_t)(op-ostart) >= srcSize-1) { return 0; } - return (size_t)(op-ostart); -} - -typedef struct { - unsigned count[HUF_SYMBOLVALUE_MAX + 1]; - HUF_CElt CTable[HUF_SYMBOLVALUE_MAX + 1]; - HUF_buildCTable_wksp_tables buildCTable_wksp; -} HUF_compress_tables_t; - -/* HUF_compress_internal() : - * `workSpace` must a table of at least HUF_WORKSPACE_SIZE_U32 unsigned */ -static size_t -HUF_compress_internal (void* dst, size_t dstSize, - const void* src, size_t srcSize, - unsigned maxSymbolValue, unsigned huffLog, - HUF_nbStreams_e nbStreams, - void* workSpace, size_t wkspSize, - HUF_CElt* oldHufTable, HUF_repeat* repeat, int preferRepeat, - const int bmi2) -{ - HUF_compress_tables_t* const table = (HUF_compress_tables_t*)workSpace; - BYTE* const ostart = (BYTE*)dst; - BYTE* const oend = ostart + dstSize; - BYTE* op = ostart; - - HUF_STATIC_ASSERT(sizeof(*table) <= HUF_WORKSPACE_SIZE); - - /* checks & inits */ - if (((size_t)workSpace & 3) != 0) return ERROR(GENERIC); /* must be aligned on 4-bytes boundaries */ - if (wkspSize < HUF_WORKSPACE_SIZE) return ERROR(workSpace_tooSmall); - if (!srcSize) return 0; /* Uncompressed */ - if (!dstSize) return 0; /* cannot fit anything within dst budget */ - if (srcSize > HUF_BLOCKSIZE_MAX) return ERROR(srcSize_wrong); /* current block size limit */ - if (huffLog > HUF_TABLELOG_MAX) return ERROR(tableLog_tooLarge); - if (maxSymbolValue > HUF_SYMBOLVALUE_MAX) return ERROR(maxSymbolValue_tooLarge); - if (!maxSymbolValue) maxSymbolValue = HUF_SYMBOLVALUE_MAX; - if (!huffLog) huffLog = HUF_TABLELOG_DEFAULT; - - /* Heuristic : If old table is valid, use it for small inputs */ - if (preferRepeat && repeat && *repeat == HUF_repeat_valid) { - return HUF_compressCTable_internal(ostart, op, oend, - src, srcSize, - nbStreams, oldHufTable, bmi2); - } - - /* Scan input and build symbol stats */ - { CHECK_V_F(largest, HIST_count_wksp (table->count, &maxSymbolValue, (const BYTE*)src, srcSize, workSpace, wkspSize) ); - if (largest == srcSize) { *ostart = ((const BYTE*)src)[0]; return 1; } /* single symbol, rle */ - if (largest <= (srcSize >> 7)+4) return 0; /* heuristic : probably not compressible enough */ - } - - /* Check validity of previous table */ - if ( repeat - && *repeat == HUF_repeat_check - && !HUF_validateCTable(oldHufTable, table->count, maxSymbolValue)) { - *repeat = HUF_repeat_none; - } - /* Heuristic : use existing table for small inputs */ - if (preferRepeat && repeat && *repeat != HUF_repeat_none) { - return HUF_compressCTable_internal(ostart, op, oend, - src, srcSize, - nbStreams, oldHufTable, bmi2); - } - - /* Build Huffman Tree */ - huffLog = HUF_optimalTableLog(huffLog, srcSize, maxSymbolValue); - { size_t const maxBits = HUF_buildCTable_wksp(table->CTable, table->count, - maxSymbolValue, huffLog, - &table->buildCTable_wksp, sizeof(table->buildCTable_wksp)); - CHECK_F(maxBits); - huffLog = (U32)maxBits; - /* Zero unused symbols in CTable, so we can check it for validity */ - memset(table->CTable + (maxSymbolValue + 1), 0, - sizeof(table->CTable) - ((maxSymbolValue + 1) * sizeof(HUF_CElt))); - } - - /* Write table description header */ - { CHECK_V_F(hSize, HUF_writeCTable (op, dstSize, table->CTable, maxSymbolValue, huffLog) ); - /* Check if using previous huffman table is beneficial */ - if (repeat && *repeat != HUF_repeat_none) { - size_t const oldSize = HUF_estimateCompressedSize(oldHufTable, table->count, maxSymbolValue); - size_t const newSize = HUF_estimateCompressedSize(table->CTable, table->count, maxSymbolValue); - if (oldSize <= hSize + newSize || hSize + 12 >= srcSize) { - return HUF_compressCTable_internal(ostart, op, oend, - src, srcSize, - nbStreams, oldHufTable, bmi2); - } } - - /* Use the new huffman table */ - if (hSize + 12ul >= srcSize) { return 0; } - op += hSize; - if (repeat) { *repeat = HUF_repeat_none; } - if (oldHufTable) - memcpy(oldHufTable, table->CTable, sizeof(table->CTable)); /* Save new table */ - } - return HUF_compressCTable_internal(ostart, op, oend, - src, srcSize, - nbStreams, table->CTable, bmi2); -} - - -size_t HUF_compress1X_wksp (void* dst, size_t dstSize, - const void* src, size_t srcSize, - unsigned maxSymbolValue, unsigned huffLog, - void* workSpace, size_t wkspSize) -{ - return HUF_compress_internal(dst, dstSize, src, srcSize, - maxSymbolValue, huffLog, HUF_singleStream, - workSpace, wkspSize, - NULL, NULL, 0, 0 /*bmi2*/); -} - -size_t HUF_compress1X_repeat (void* dst, size_t dstSize, - const void* src, size_t srcSize, - unsigned maxSymbolValue, unsigned huffLog, - void* workSpace, size_t wkspSize, - HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat, int bmi2) -{ - return HUF_compress_internal(dst, dstSize, src, srcSize, - maxSymbolValue, huffLog, HUF_singleStream, - workSpace, wkspSize, hufTable, - repeat, preferRepeat, bmi2); -} - -size_t HUF_compress1X (void* dst, size_t dstSize, - const void* src, size_t srcSize, - unsigned maxSymbolValue, unsigned huffLog) -{ - unsigned workSpace[HUF_WORKSPACE_SIZE_U32]; - return HUF_compress1X_wksp(dst, dstSize, src, srcSize, maxSymbolValue, huffLog, workSpace, sizeof(workSpace)); -} - -/* HUF_compress4X_repeat(): - * compress input using 4 streams. - * provide workspace to generate compression tables */ -size_t HUF_compress4X_wksp (void* dst, size_t dstSize, - const void* src, size_t srcSize, - unsigned maxSymbolValue, unsigned huffLog, - void* workSpace, size_t wkspSize) -{ - return HUF_compress_internal(dst, dstSize, src, srcSize, - maxSymbolValue, huffLog, HUF_fourStreams, - workSpace, wkspSize, - NULL, NULL, 0, 0 /*bmi2*/); -} - -/* HUF_compress4X_repeat(): - * compress input using 4 streams. - * re-use an existing huffman compression table */ -size_t HUF_compress4X_repeat (void* dst, size_t dstSize, - const void* src, size_t srcSize, - unsigned maxSymbolValue, unsigned huffLog, - void* workSpace, size_t wkspSize, - HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat, int bmi2) -{ - return HUF_compress_internal(dst, dstSize, src, srcSize, - maxSymbolValue, huffLog, HUF_fourStreams, - workSpace, wkspSize, - hufTable, repeat, preferRepeat, bmi2); -} - -size_t HUF_compress2 (void* dst, size_t dstSize, - const void* src, size_t srcSize, - unsigned maxSymbolValue, unsigned huffLog) -{ - unsigned workSpace[HUF_WORKSPACE_SIZE_U32]; - return HUF_compress4X_wksp(dst, dstSize, src, srcSize, maxSymbolValue, huffLog, workSpace, sizeof(workSpace)); -} - -size_t HUF_compress (void* dst, size_t maxDstSize, const void* src, size_t srcSize) -{ - return HUF_compress2(dst, maxDstSize, src, srcSize, 255, HUF_TABLELOG_DEFAULT); -} -/**** ended inlining compress/huf_compress.c ****/ -/**** start inlining compress/zstd_compress_literals.c ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - - /*-************************************* - * Dependencies - ***************************************/ -/**** start inlining zstd_compress_literals.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_COMPRESS_LITERALS_H -#define ZSTD_COMPRESS_LITERALS_H - -/**** start inlining zstd_compress_internal.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -/* This header contains definitions - * that shall **only** be used by modules within lib/compress. - */ - -#ifndef ZSTD_COMPRESS_H -#define ZSTD_COMPRESS_H - -/*-************************************* -* Dependencies -***************************************/ -/**** skipping file: ../common/zstd_internal.h ****/ -/**** start inlining zstd_cwksp.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_CWKSP_H -#define ZSTD_CWKSP_H - -/*-************************************* -* Dependencies -***************************************/ -/**** skipping file: ../common/zstd_internal.h ****/ - -#if defined (__cplusplus) -extern "C" { -#endif - -/*-************************************* -* Constants -***************************************/ - -/* Since the workspace is effectively its own little malloc implementation / - * arena, when we run under ASAN, we should similarly insert redzones between - * each internal element of the workspace, so ASAN will catch overruns that - * reach outside an object but that stay inside the workspace. - * - * This defines the size of that redzone. - */ -#ifndef ZSTD_CWKSP_ASAN_REDZONE_SIZE -#define ZSTD_CWKSP_ASAN_REDZONE_SIZE 128 -#endif - -/*-************************************* -* Structures -***************************************/ -typedef enum { - ZSTD_cwksp_alloc_objects, - ZSTD_cwksp_alloc_buffers, - ZSTD_cwksp_alloc_aligned -} ZSTD_cwksp_alloc_phase_e; - -/** - * Zstd fits all its internal datastructures into a single continuous buffer, - * so that it only needs to perform a single OS allocation (or so that a buffer - * can be provided to it and it can perform no allocations at all). This buffer - * is called the workspace. - * - * Several optimizations complicate that process of allocating memory ranges - * from this workspace for each internal datastructure: - * - * - These different internal datastructures have different setup requirements: - * - * - The static objects need to be cleared once and can then be trivially - * reused for each compression. - * - * - Various buffers don't need to be initialized at all--they are always - * written into before they're read. - * - * - The matchstate tables have a unique requirement that they don't need - * their memory to be totally cleared, but they do need the memory to have - * some bound, i.e., a guarantee that all values in the memory they've been - * allocated is less than some maximum value (which is the starting value - * for the indices that they will then use for compression). When this - * guarantee is provided to them, they can use the memory without any setup - * work. When it can't, they have to clear the area. - * - * - These buffers also have different alignment requirements. - * - * - We would like to reuse the objects in the workspace for multiple - * compressions without having to perform any expensive reallocation or - * reinitialization work. - * - * - We would like to be able to efficiently reuse the workspace across - * multiple compressions **even when the compression parameters change** and - * we need to resize some of the objects (where possible). - * - * To attempt to manage this buffer, given these constraints, the ZSTD_cwksp - * abstraction was created. It works as follows: - * - * Workspace Layout: - * - * [ ... workspace ... ] - * [objects][tables ... ->] free space [<- ... aligned][<- ... buffers] - * - * The various objects that live in the workspace are divided into the - * following categories, and are allocated separately: - * - * - Static objects: this is optionally the enclosing ZSTD_CCtx or ZSTD_CDict, - * so that literally everything fits in a single buffer. Note: if present, - * this must be the first object in the workspace, since ZSTD_free{CCtx, - * CDict}() rely on a pointer comparison to see whether one or two frees are - * required. - * - * - Fixed size objects: these are fixed-size, fixed-count objects that are - * nonetheless "dynamically" allocated in the workspace so that we can - * control how they're initialized separately from the broader ZSTD_CCtx. - * Examples: - * - Entropy Workspace - * - 2 x ZSTD_compressedBlockState_t - * - CDict dictionary contents - * - * - Tables: these are any of several different datastructures (hash tables, - * chain tables, binary trees) that all respect a common format: they are - * uint32_t arrays, all of whose values are between 0 and (nextSrc - base). - * Their sizes depend on the cparams. - * - * - Aligned: these buffers are used for various purposes that require 4 byte - * alignment, but don't require any initialization before they're used. - * - * - Buffers: these buffers are used for various purposes that don't require - * any alignment or initialization before they're used. This means they can - * be moved around at no cost for a new compression. - * - * Allocating Memory: - * - * The various types of objects must be allocated in order, so they can be - * correctly packed into the workspace buffer. That order is: - * - * 1. Objects - * 2. Buffers - * 3. Aligned - * 4. Tables - * - * Attempts to reserve objects of different types out of order will fail. - */ -typedef struct { - void* workspace; - void* workspaceEnd; - - void* objectEnd; - void* tableEnd; - void* tableValidEnd; - void* allocStart; - - int allocFailed; - int workspaceOversizedDuration; - ZSTD_cwksp_alloc_phase_e phase; -} ZSTD_cwksp; - -/*-************************************* -* Functions -***************************************/ - -MEM_STATIC size_t ZSTD_cwksp_available_space(ZSTD_cwksp* ws); - -MEM_STATIC void ZSTD_cwksp_assert_internal_consistency(ZSTD_cwksp* ws) { - (void)ws; - assert(ws->workspace <= ws->objectEnd); - assert(ws->objectEnd <= ws->tableEnd); - assert(ws->objectEnd <= ws->tableValidEnd); - assert(ws->tableEnd <= ws->allocStart); - assert(ws->tableValidEnd <= ws->allocStart); - assert(ws->allocStart <= ws->workspaceEnd); -} - -/** - * Align must be a power of 2. - */ -MEM_STATIC size_t ZSTD_cwksp_align(size_t size, size_t const align) { - size_t const mask = align - 1; - assert((align & mask) == 0); - return (size + mask) & ~mask; -} - -/** - * Use this to determine how much space in the workspace we will consume to - * allocate this object. (Normally it should be exactly the size of the object, - * but under special conditions, like ASAN, where we pad each object, it might - * be larger.) - * - * Since tables aren't currently redzoned, you don't need to call through this - * to figure out how much space you need for the matchState tables. Everything - * else is though. - */ -MEM_STATIC size_t ZSTD_cwksp_alloc_size(size_t size) { -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) - return size + 2 * ZSTD_CWKSP_ASAN_REDZONE_SIZE; -#else - return size; -#endif -} - -MEM_STATIC void ZSTD_cwksp_internal_advance_phase( - ZSTD_cwksp* ws, ZSTD_cwksp_alloc_phase_e phase) { - assert(phase >= ws->phase); - if (phase > ws->phase) { - if (ws->phase < ZSTD_cwksp_alloc_buffers && - phase >= ZSTD_cwksp_alloc_buffers) { - ws->tableValidEnd = ws->objectEnd; - } - if (ws->phase < ZSTD_cwksp_alloc_aligned && - phase >= ZSTD_cwksp_alloc_aligned) { - /* If unaligned allocations down from a too-large top have left us - * unaligned, we need to realign our alloc ptr. Technically, this - * can consume space that is unaccounted for in the neededSpace - * calculation. However, I believe this can only happen when the - * workspace is too large, and specifically when it is too large - * by a larger margin than the space that will be consumed. */ - /* TODO: cleaner, compiler warning friendly way to do this??? */ - ws->allocStart = (BYTE*)ws->allocStart - ((size_t)ws->allocStart & (sizeof(U32)-1)); - if (ws->allocStart < ws->tableValidEnd) { - ws->tableValidEnd = ws->allocStart; - } - } - ws->phase = phase; - } -} - -/** - * Returns whether this object/buffer/etc was allocated in this workspace. - */ -MEM_STATIC int ZSTD_cwksp_owns_buffer(const ZSTD_cwksp* ws, const void* ptr) { - return (ptr != NULL) && (ws->workspace <= ptr) && (ptr <= ws->workspaceEnd); -} - -/** - * Internal function. Do not use directly. - */ -MEM_STATIC void* ZSTD_cwksp_reserve_internal( - ZSTD_cwksp* ws, size_t bytes, ZSTD_cwksp_alloc_phase_e phase) { - void* alloc; - void* bottom = ws->tableEnd; - ZSTD_cwksp_internal_advance_phase(ws, phase); - alloc = (BYTE *)ws->allocStart - bytes; - -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) - /* over-reserve space */ - alloc = (BYTE *)alloc - 2 * ZSTD_CWKSP_ASAN_REDZONE_SIZE; -#endif - - DEBUGLOG(5, "cwksp: reserving %p %zd bytes, %zd bytes remaining", - alloc, bytes, ZSTD_cwksp_available_space(ws) - bytes); - ZSTD_cwksp_assert_internal_consistency(ws); - assert(alloc >= bottom); - if (alloc < bottom) { - DEBUGLOG(4, "cwksp: alloc failed!"); - ws->allocFailed = 1; - return NULL; - } - if (alloc < ws->tableValidEnd) { - ws->tableValidEnd = alloc; - } - ws->allocStart = alloc; - -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) - /* Move alloc so there's ZSTD_CWKSP_ASAN_REDZONE_SIZE unused space on - * either size. */ - alloc = (BYTE *)alloc + ZSTD_CWKSP_ASAN_REDZONE_SIZE; - __asan_unpoison_memory_region(alloc, bytes); -#endif - - return alloc; -} - -/** - * Reserves and returns unaligned memory. - */ -MEM_STATIC BYTE* ZSTD_cwksp_reserve_buffer(ZSTD_cwksp* ws, size_t bytes) { - return (BYTE*)ZSTD_cwksp_reserve_internal(ws, bytes, ZSTD_cwksp_alloc_buffers); -} - -/** - * Reserves and returns memory sized on and aligned on sizeof(unsigned). - */ -MEM_STATIC void* ZSTD_cwksp_reserve_aligned(ZSTD_cwksp* ws, size_t bytes) { - assert((bytes & (sizeof(U32)-1)) == 0); - return ZSTD_cwksp_reserve_internal(ws, ZSTD_cwksp_align(bytes, sizeof(U32)), ZSTD_cwksp_alloc_aligned); -} - -/** - * Aligned on sizeof(unsigned). These buffers have the special property that - * their values remain constrained, allowing us to re-use them without - * memset()-ing them. - */ -MEM_STATIC void* ZSTD_cwksp_reserve_table(ZSTD_cwksp* ws, size_t bytes) { - const ZSTD_cwksp_alloc_phase_e phase = ZSTD_cwksp_alloc_aligned; - void* alloc = ws->tableEnd; - void* end = (BYTE *)alloc + bytes; - void* top = ws->allocStart; - - DEBUGLOG(5, "cwksp: reserving %p table %zd bytes, %zd bytes remaining", - alloc, bytes, ZSTD_cwksp_available_space(ws) - bytes); - assert((bytes & (sizeof(U32)-1)) == 0); - ZSTD_cwksp_internal_advance_phase(ws, phase); - ZSTD_cwksp_assert_internal_consistency(ws); - assert(end <= top); - if (end > top) { - DEBUGLOG(4, "cwksp: table alloc failed!"); - ws->allocFailed = 1; - return NULL; - } - ws->tableEnd = end; - -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) - __asan_unpoison_memory_region(alloc, bytes); -#endif - - return alloc; -} - -/** - * Aligned on sizeof(void*). - */ -MEM_STATIC void* ZSTD_cwksp_reserve_object(ZSTD_cwksp* ws, size_t bytes) { - size_t roundedBytes = ZSTD_cwksp_align(bytes, sizeof(void*)); - void* alloc = ws->objectEnd; - void* end = (BYTE*)alloc + roundedBytes; - -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) - /* over-reserve space */ - end = (BYTE *)end + 2 * ZSTD_CWKSP_ASAN_REDZONE_SIZE; -#endif - - DEBUGLOG(5, - "cwksp: reserving %p object %zd bytes (rounded to %zd), %zd bytes remaining", - alloc, bytes, roundedBytes, ZSTD_cwksp_available_space(ws) - roundedBytes); - assert(((size_t)alloc & (sizeof(void*)-1)) == 0); - assert((bytes & (sizeof(void*)-1)) == 0); - ZSTD_cwksp_assert_internal_consistency(ws); - /* we must be in the first phase, no advance is possible */ - if (ws->phase != ZSTD_cwksp_alloc_objects || end > ws->workspaceEnd) { - DEBUGLOG(4, "cwksp: object alloc failed!"); - ws->allocFailed = 1; - return NULL; - } - ws->objectEnd = end; - ws->tableEnd = end; - ws->tableValidEnd = end; - -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) - /* Move alloc so there's ZSTD_CWKSP_ASAN_REDZONE_SIZE unused space on - * either size. */ - alloc = (BYTE *)alloc + ZSTD_CWKSP_ASAN_REDZONE_SIZE; - __asan_unpoison_memory_region(alloc, bytes); -#endif - - return alloc; -} - -MEM_STATIC void ZSTD_cwksp_mark_tables_dirty(ZSTD_cwksp* ws) { - DEBUGLOG(4, "cwksp: ZSTD_cwksp_mark_tables_dirty"); - -#if defined (MEMORY_SANITIZER) && !defined (ZSTD_MSAN_DONT_POISON_WORKSPACE) - /* To validate that the table re-use logic is sound, and that we don't - * access table space that we haven't cleaned, we re-"poison" the table - * space every time we mark it dirty. */ - { - size_t size = (BYTE*)ws->tableValidEnd - (BYTE*)ws->objectEnd; - assert(__msan_test_shadow(ws->objectEnd, size) == -1); - __msan_poison(ws->objectEnd, size); - } -#endif - - assert(ws->tableValidEnd >= ws->objectEnd); - assert(ws->tableValidEnd <= ws->allocStart); - ws->tableValidEnd = ws->objectEnd; - ZSTD_cwksp_assert_internal_consistency(ws); -} - -MEM_STATIC void ZSTD_cwksp_mark_tables_clean(ZSTD_cwksp* ws) { - DEBUGLOG(4, "cwksp: ZSTD_cwksp_mark_tables_clean"); - assert(ws->tableValidEnd >= ws->objectEnd); - assert(ws->tableValidEnd <= ws->allocStart); - if (ws->tableValidEnd < ws->tableEnd) { - ws->tableValidEnd = ws->tableEnd; - } - ZSTD_cwksp_assert_internal_consistency(ws); -} - -/** - * Zero the part of the allocated tables not already marked clean. - */ -MEM_STATIC void ZSTD_cwksp_clean_tables(ZSTD_cwksp* ws) { - DEBUGLOG(4, "cwksp: ZSTD_cwksp_clean_tables"); - assert(ws->tableValidEnd >= ws->objectEnd); - assert(ws->tableValidEnd <= ws->allocStart); - if (ws->tableValidEnd < ws->tableEnd) { - memset(ws->tableValidEnd, 0, (BYTE*)ws->tableEnd - (BYTE*)ws->tableValidEnd); - } - ZSTD_cwksp_mark_tables_clean(ws); -} - -/** - * Invalidates table allocations. - * All other allocations remain valid. - */ -MEM_STATIC void ZSTD_cwksp_clear_tables(ZSTD_cwksp* ws) { - DEBUGLOG(4, "cwksp: clearing tables!"); - -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) - { - size_t size = (BYTE*)ws->tableValidEnd - (BYTE*)ws->objectEnd; - __asan_poison_memory_region(ws->objectEnd, size); - } -#endif - - ws->tableEnd = ws->objectEnd; - ZSTD_cwksp_assert_internal_consistency(ws); -} - -/** - * Invalidates all buffer, aligned, and table allocations. - * Object allocations remain valid. - */ -MEM_STATIC void ZSTD_cwksp_clear(ZSTD_cwksp* ws) { - DEBUGLOG(4, "cwksp: clearing!"); - -#if defined (MEMORY_SANITIZER) && !defined (ZSTD_MSAN_DONT_POISON_WORKSPACE) - /* To validate that the context re-use logic is sound, and that we don't - * access stuff that this compression hasn't initialized, we re-"poison" - * the workspace (or at least the non-static, non-table parts of it) - * every time we start a new compression. */ - { - size_t size = (BYTE*)ws->workspaceEnd - (BYTE*)ws->tableValidEnd; - __msan_poison(ws->tableValidEnd, size); - } -#endif - -#if defined (ADDRESS_SANITIZER) && !defined (ZSTD_ASAN_DONT_POISON_WORKSPACE) - { - size_t size = (BYTE*)ws->workspaceEnd - (BYTE*)ws->objectEnd; - __asan_poison_memory_region(ws->objectEnd, size); - } -#endif - - ws->tableEnd = ws->objectEnd; - ws->allocStart = ws->workspaceEnd; - ws->allocFailed = 0; - if (ws->phase > ZSTD_cwksp_alloc_buffers) { - ws->phase = ZSTD_cwksp_alloc_buffers; - } - ZSTD_cwksp_assert_internal_consistency(ws); -} - -/** - * The provided workspace takes ownership of the buffer [start, start+size). - * Any existing values in the workspace are ignored (the previously managed - * buffer, if present, must be separately freed). - */ -MEM_STATIC void ZSTD_cwksp_init(ZSTD_cwksp* ws, void* start, size_t size) { - DEBUGLOG(4, "cwksp: init'ing workspace with %zd bytes", size); - assert(((size_t)start & (sizeof(void*)-1)) == 0); /* ensure correct alignment */ - ws->workspace = start; - ws->workspaceEnd = (BYTE*)start + size; - ws->objectEnd = ws->workspace; - ws->tableValidEnd = ws->objectEnd; - ws->phase = ZSTD_cwksp_alloc_objects; - ZSTD_cwksp_clear(ws); - ws->workspaceOversizedDuration = 0; - ZSTD_cwksp_assert_internal_consistency(ws); -} - -MEM_STATIC size_t ZSTD_cwksp_create(ZSTD_cwksp* ws, size_t size, ZSTD_customMem customMem) { - void* workspace = ZSTD_malloc(size, customMem); - DEBUGLOG(4, "cwksp: creating new workspace with %zd bytes", size); - RETURN_ERROR_IF(workspace == NULL, memory_allocation, "NULL pointer!"); - ZSTD_cwksp_init(ws, workspace, size); - return 0; -} - -MEM_STATIC void ZSTD_cwksp_free(ZSTD_cwksp* ws, ZSTD_customMem customMem) { - void *ptr = ws->workspace; - DEBUGLOG(4, "cwksp: freeing workspace"); - memset(ws, 0, sizeof(ZSTD_cwksp)); - ZSTD_free(ptr, customMem); -} - -/** - * Moves the management of a workspace from one cwksp to another. The src cwksp - * is left in an invalid state (src must be re-init()'ed before its used again). - */ -MEM_STATIC void ZSTD_cwksp_move(ZSTD_cwksp* dst, ZSTD_cwksp* src) { - *dst = *src; - memset(src, 0, sizeof(ZSTD_cwksp)); -} - -MEM_STATIC size_t ZSTD_cwksp_sizeof(const ZSTD_cwksp* ws) { - return (size_t)((BYTE*)ws->workspaceEnd - (BYTE*)ws->workspace); -} - -MEM_STATIC int ZSTD_cwksp_reserve_failed(const ZSTD_cwksp* ws) { - return ws->allocFailed; -} - -/*-************************************* -* Functions Checking Free Space -***************************************/ - -MEM_STATIC size_t ZSTD_cwksp_available_space(ZSTD_cwksp* ws) { - return (size_t)((BYTE*)ws->allocStart - (BYTE*)ws->tableEnd); -} - -MEM_STATIC int ZSTD_cwksp_check_available(ZSTD_cwksp* ws, size_t additionalNeededSpace) { - return ZSTD_cwksp_available_space(ws) >= additionalNeededSpace; -} - -MEM_STATIC int ZSTD_cwksp_check_too_large(ZSTD_cwksp* ws, size_t additionalNeededSpace) { - return ZSTD_cwksp_check_available( - ws, additionalNeededSpace * ZSTD_WORKSPACETOOLARGE_FACTOR); -} - -MEM_STATIC int ZSTD_cwksp_check_wasteful(ZSTD_cwksp* ws, size_t additionalNeededSpace) { - return ZSTD_cwksp_check_too_large(ws, additionalNeededSpace) - && ws->workspaceOversizedDuration > ZSTD_WORKSPACETOOLARGE_MAXDURATION; -} - -MEM_STATIC void ZSTD_cwksp_bump_oversized_duration( - ZSTD_cwksp* ws, size_t additionalNeededSpace) { - if (ZSTD_cwksp_check_too_large(ws, additionalNeededSpace)) { - ws->workspaceOversizedDuration++; - } else { - ws->workspaceOversizedDuration = 0; - } -} - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTD_CWKSP_H */ -/**** ended inlining zstd_cwksp.h ****/ -#ifdef ZSTD_MULTITHREAD -/**** start inlining zstdmt_compress.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - - #ifndef ZSTDMT_COMPRESS_H - #define ZSTDMT_COMPRESS_H - - #if defined (__cplusplus) - extern "C" { - #endif - - -/* Note : This is an internal API. - * These APIs used to be exposed with ZSTDLIB_API, - * because it used to be the only way to invoke MT compression. - * Now, it's recommended to use ZSTD_compress2 and ZSTD_compressStream2() - * instead. - * - * If you depend on these APIs and can't switch, then define - * ZSTD_LEGACY_MULTITHREADED_API when making the dynamic library. - * However, we may completely remove these functions in a future - * release, so please switch soon. - * - * This API requires ZSTD_MULTITHREAD to be defined during compilation, - * otherwise ZSTDMT_createCCtx*() will fail. - */ - -#ifdef ZSTD_LEGACY_MULTITHREADED_API -# define ZSTDMT_API ZSTDLIB_API -#else -# define ZSTDMT_API -#endif - -/* === Dependencies === */ -#include /* size_t */ -#define ZSTD_STATIC_LINKING_ONLY /* ZSTD_parameters */ -/**** skipping file: ../zstd.h ****/ - - -/* === Constants === */ -#ifndef ZSTDMT_NBWORKERS_MAX -# define ZSTDMT_NBWORKERS_MAX 200 -#endif -#ifndef ZSTDMT_JOBSIZE_MIN -# define ZSTDMT_JOBSIZE_MIN (1 MB) -#endif -#define ZSTDMT_JOBLOG_MAX (MEM_32bits() ? 29 : 30) -#define ZSTDMT_JOBSIZE_MAX (MEM_32bits() ? (512 MB) : (1024 MB)) - - -/* === Memory management === */ -typedef struct ZSTDMT_CCtx_s ZSTDMT_CCtx; -/* Requires ZSTD_MULTITHREAD to be defined during compilation, otherwise it will return NULL. */ -ZSTDMT_API ZSTDMT_CCtx* ZSTDMT_createCCtx(unsigned nbWorkers); -/* Requires ZSTD_MULTITHREAD to be defined during compilation, otherwise it will return NULL. */ -ZSTDMT_API ZSTDMT_CCtx* ZSTDMT_createCCtx_advanced(unsigned nbWorkers, - ZSTD_customMem cMem); -ZSTDMT_API size_t ZSTDMT_freeCCtx(ZSTDMT_CCtx* mtctx); - -ZSTDMT_API size_t ZSTDMT_sizeof_CCtx(ZSTDMT_CCtx* mtctx); - - -/* === Simple one-pass compression function === */ - -ZSTDMT_API size_t ZSTDMT_compressCCtx(ZSTDMT_CCtx* mtctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - int compressionLevel); - - - -/* === Streaming functions === */ - -ZSTDMT_API size_t ZSTDMT_initCStream(ZSTDMT_CCtx* mtctx, int compressionLevel); -ZSTDMT_API size_t ZSTDMT_resetCStream(ZSTDMT_CCtx* mtctx, unsigned long long pledgedSrcSize); /**< if srcSize is not known at reset time, use ZSTD_CONTENTSIZE_UNKNOWN. Note: for compatibility with older programs, 0 means the same as ZSTD_CONTENTSIZE_UNKNOWN, but it will change in the future to mean "empty" */ - -ZSTDMT_API size_t ZSTDMT_nextInputSizeHint(const ZSTDMT_CCtx* mtctx); -ZSTDMT_API size_t ZSTDMT_compressStream(ZSTDMT_CCtx* mtctx, ZSTD_outBuffer* output, ZSTD_inBuffer* input); - -ZSTDMT_API size_t ZSTDMT_flushStream(ZSTDMT_CCtx* mtctx, ZSTD_outBuffer* output); /**< @return : 0 == all flushed; >0 : still some data to be flushed; or an error code (ZSTD_isError()) */ -ZSTDMT_API size_t ZSTDMT_endStream(ZSTDMT_CCtx* mtctx, ZSTD_outBuffer* output); /**< @return : 0 == all flushed; >0 : still some data to be flushed; or an error code (ZSTD_isError()) */ - - -/* === Advanced functions and parameters === */ - -ZSTDMT_API size_t ZSTDMT_compress_advanced(ZSTDMT_CCtx* mtctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const ZSTD_CDict* cdict, - ZSTD_parameters params, - int overlapLog); - -ZSTDMT_API size_t ZSTDMT_initCStream_advanced(ZSTDMT_CCtx* mtctx, - const void* dict, size_t dictSize, /* dict can be released after init, a local copy is preserved within zcs */ - ZSTD_parameters params, - unsigned long long pledgedSrcSize); /* pledgedSrcSize is optional and can be zero == unknown */ - -ZSTDMT_API size_t ZSTDMT_initCStream_usingCDict(ZSTDMT_CCtx* mtctx, - const ZSTD_CDict* cdict, - ZSTD_frameParameters fparams, - unsigned long long pledgedSrcSize); /* note : zero means empty */ - -/* ZSTDMT_parameter : - * List of parameters that can be set using ZSTDMT_setMTCtxParameter() */ -typedef enum { - ZSTDMT_p_jobSize, /* Each job is compressed in parallel. By default, this value is dynamically determined depending on compression parameters. Can be set explicitly here. */ - ZSTDMT_p_overlapLog, /* Each job may reload a part of previous job to enhance compression ratio; 0 == no overlap, 6(default) == use 1/8th of window, >=9 == use full window. This is a "sticky" parameter : its value will be re-used on next compression job */ - ZSTDMT_p_rsyncable /* Enables rsyncable mode. */ -} ZSTDMT_parameter; - -/* ZSTDMT_setMTCtxParameter() : - * allow setting individual parameters, one at a time, among a list of enums defined in ZSTDMT_parameter. - * The function must be called typically after ZSTD_createCCtx() but __before ZSTDMT_init*() !__ - * Parameters not explicitly reset by ZSTDMT_init*() remain the same in consecutive compression sessions. - * @return : 0, or an error code (which can be tested using ZSTD_isError()) */ -ZSTDMT_API size_t ZSTDMT_setMTCtxParameter(ZSTDMT_CCtx* mtctx, ZSTDMT_parameter parameter, int value); - -/* ZSTDMT_getMTCtxParameter() : - * Query the ZSTDMT_CCtx for a parameter value. - * @return : 0, or an error code (which can be tested using ZSTD_isError()) */ -ZSTDMT_API size_t ZSTDMT_getMTCtxParameter(ZSTDMT_CCtx* mtctx, ZSTDMT_parameter parameter, int* value); - - -/*! ZSTDMT_compressStream_generic() : - * Combines ZSTDMT_compressStream() with optional ZSTDMT_flushStream() or ZSTDMT_endStream() - * depending on flush directive. - * @return : minimum amount of data still to be flushed - * 0 if fully flushed - * or an error code - * note : needs to be init using any ZSTD_initCStream*() variant */ -ZSTDMT_API size_t ZSTDMT_compressStream_generic(ZSTDMT_CCtx* mtctx, - ZSTD_outBuffer* output, - ZSTD_inBuffer* input, - ZSTD_EndDirective endOp); - - -/* ======================================================== - * === Private interface, for use by ZSTD_compress.c === - * === Not exposed in libzstd. Never invoke directly === - * ======================================================== */ - - /*! ZSTDMT_toFlushNow() - * Tell how many bytes are ready to be flushed immediately. - * Probe the oldest active job (not yet entirely flushed) and check its output buffer. - * If return 0, it means there is no active job, - * or, it means oldest job is still active, but everything produced has been flushed so far, - * therefore flushing is limited by speed of oldest job. */ -size_t ZSTDMT_toFlushNow(ZSTDMT_CCtx* mtctx); - -/*! ZSTDMT_CCtxParam_setMTCtxParameter() - * like ZSTDMT_setMTCtxParameter(), but into a ZSTD_CCtx_Params */ -size_t ZSTDMT_CCtxParam_setMTCtxParameter(ZSTD_CCtx_params* params, ZSTDMT_parameter parameter, int value); - -/*! ZSTDMT_CCtxParam_setNbWorkers() - * Set nbWorkers, and clamp it. - * Also reset jobSize and overlapLog */ -size_t ZSTDMT_CCtxParam_setNbWorkers(ZSTD_CCtx_params* params, unsigned nbWorkers); - -/*! ZSTDMT_updateCParams_whileCompressing() : - * Updates only a selected set of compression parameters, to remain compatible with current frame. - * New parameters will be applied to next compression job. */ -void ZSTDMT_updateCParams_whileCompressing(ZSTDMT_CCtx* mtctx, const ZSTD_CCtx_params* cctxParams); - -/*! ZSTDMT_getFrameProgression(): - * tells how much data has been consumed (input) and produced (output) for current frame. - * able to count progression inside worker threads. - */ -ZSTD_frameProgression ZSTDMT_getFrameProgression(ZSTDMT_CCtx* mtctx); - - -/*! ZSTDMT_initCStream_internal() : - * Private use only. Init streaming operation. - * expects params to be valid. - * must receive dict, or cdict, or none, but not both. - * @return : 0, or an error code */ -size_t ZSTDMT_initCStream_internal(ZSTDMT_CCtx* zcs, - const void* dict, size_t dictSize, ZSTD_dictContentType_e dictContentType, - const ZSTD_CDict* cdict, - ZSTD_CCtx_params params, unsigned long long pledgedSrcSize); - - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTDMT_COMPRESS_H */ -/**** ended inlining zstdmt_compress.h ****/ -#endif - -#if defined (__cplusplus) -extern "C" { -#endif - - -/*-************************************* -* Constants -***************************************/ -#define kSearchStrength 8 -#define HASH_READ_SIZE 8 -#define ZSTD_DUBT_UNSORTED_MARK 1 /* For btlazy2 strategy, index ZSTD_DUBT_UNSORTED_MARK==1 means "unsorted". - It could be confused for a real successor at index "1", if sorted as larger than its predecessor. - It's not a big deal though : candidate will just be sorted again. - Additionally, candidate position 1 will be lost. - But candidate 1 cannot hide a large tree of candidates, so it's a minimal loss. - The benefit is that ZSTD_DUBT_UNSORTED_MARK cannot be mishandled after table re-use with a different strategy. - This constant is required by ZSTD_compressBlock_btlazy2() and ZSTD_reduceTable_internal() */ - - -/*-************************************* -* Context memory management -***************************************/ -typedef enum { ZSTDcs_created=0, ZSTDcs_init, ZSTDcs_ongoing, ZSTDcs_ending } ZSTD_compressionStage_e; -typedef enum { zcss_init=0, zcss_load, zcss_flush } ZSTD_cStreamStage; - -typedef struct ZSTD_prefixDict_s { - const void* dict; - size_t dictSize; - ZSTD_dictContentType_e dictContentType; -} ZSTD_prefixDict; - -typedef struct { - void* dictBuffer; - void const* dict; - size_t dictSize; - ZSTD_dictContentType_e dictContentType; - ZSTD_CDict* cdict; -} ZSTD_localDict; - -typedef struct { - U32 CTable[HUF_CTABLE_SIZE_U32(255)]; - HUF_repeat repeatMode; -} ZSTD_hufCTables_t; - -typedef struct { - FSE_CTable offcodeCTable[FSE_CTABLE_SIZE_U32(OffFSELog, MaxOff)]; - FSE_CTable matchlengthCTable[FSE_CTABLE_SIZE_U32(MLFSELog, MaxML)]; - FSE_CTable litlengthCTable[FSE_CTABLE_SIZE_U32(LLFSELog, MaxLL)]; - FSE_repeat offcode_repeatMode; - FSE_repeat matchlength_repeatMode; - FSE_repeat litlength_repeatMode; -} ZSTD_fseCTables_t; - -typedef struct { - ZSTD_hufCTables_t huf; - ZSTD_fseCTables_t fse; -} ZSTD_entropyCTables_t; - -typedef struct { - U32 off; - U32 len; -} ZSTD_match_t; - -typedef struct { - int price; - U32 off; - U32 mlen; - U32 litlen; - U32 rep[ZSTD_REP_NUM]; -} ZSTD_optimal_t; - -typedef enum { zop_dynamic=0, zop_predef } ZSTD_OptPrice_e; - -typedef struct { - /* All tables are allocated inside cctx->workspace by ZSTD_resetCCtx_internal() */ - unsigned* litFreq; /* table of literals statistics, of size 256 */ - unsigned* litLengthFreq; /* table of litLength statistics, of size (MaxLL+1) */ - unsigned* matchLengthFreq; /* table of matchLength statistics, of size (MaxML+1) */ - unsigned* offCodeFreq; /* table of offCode statistics, of size (MaxOff+1) */ - ZSTD_match_t* matchTable; /* list of found matches, of size ZSTD_OPT_NUM+1 */ - ZSTD_optimal_t* priceTable; /* All positions tracked by optimal parser, of size ZSTD_OPT_NUM+1 */ - - U32 litSum; /* nb of literals */ - U32 litLengthSum; /* nb of litLength codes */ - U32 matchLengthSum; /* nb of matchLength codes */ - U32 offCodeSum; /* nb of offset codes */ - U32 litSumBasePrice; /* to compare to log2(litfreq) */ - U32 litLengthSumBasePrice; /* to compare to log2(llfreq) */ - U32 matchLengthSumBasePrice;/* to compare to log2(mlfreq) */ - U32 offCodeSumBasePrice; /* to compare to log2(offreq) */ - ZSTD_OptPrice_e priceType; /* prices can be determined dynamically, or follow a pre-defined cost structure */ - const ZSTD_entropyCTables_t* symbolCosts; /* pre-calculated dictionary statistics */ - ZSTD_literalCompressionMode_e literalCompressionMode; -} optState_t; - -typedef struct { - ZSTD_entropyCTables_t entropy; - U32 rep[ZSTD_REP_NUM]; -} ZSTD_compressedBlockState_t; - -typedef struct { - BYTE const* nextSrc; /* next block here to continue on current prefix */ - BYTE const* base; /* All regular indexes relative to this position */ - BYTE const* dictBase; /* extDict indexes relative to this position */ - U32 dictLimit; /* below that point, need extDict */ - U32 lowLimit; /* below that point, no more valid data */ -} ZSTD_window_t; - -typedef struct ZSTD_matchState_t ZSTD_matchState_t; -struct ZSTD_matchState_t { - ZSTD_window_t window; /* State for window round buffer management */ - U32 loadedDictEnd; /* index of end of dictionary, within context's referential. - * When loadedDictEnd != 0, a dictionary is in use, and still valid. - * This relies on a mechanism to set loadedDictEnd=0 when dictionary is no longer within distance. - * Such mechanism is provided within ZSTD_window_enforceMaxDist() and ZSTD_checkDictValidity(). - * When dict referential is copied into active context (i.e. not attached), - * loadedDictEnd == dictSize, since referential starts from zero. - */ - U32 nextToUpdate; /* index from which to continue table update */ - U32 hashLog3; /* dispatch table for matches of len==3 : larger == faster, more memory */ - U32* hashTable; - U32* hashTable3; - U32* chainTable; - optState_t opt; /* optimal parser state */ - const ZSTD_matchState_t* dictMatchState; - ZSTD_compressionParameters cParams; -}; - -typedef struct { - ZSTD_compressedBlockState_t* prevCBlock; - ZSTD_compressedBlockState_t* nextCBlock; - ZSTD_matchState_t matchState; -} ZSTD_blockState_t; - -typedef struct { - U32 offset; - U32 checksum; -} ldmEntry_t; - -typedef struct { - ZSTD_window_t window; /* State for the window round buffer management */ - ldmEntry_t* hashTable; - U32 loadedDictEnd; - BYTE* bucketOffsets; /* Next position in bucket to insert entry */ - U64 hashPower; /* Used to compute the rolling hash. - * Depends on ldmParams.minMatchLength */ -} ldmState_t; - -typedef struct { - U32 enableLdm; /* 1 if enable long distance matching */ - U32 hashLog; /* Log size of hashTable */ - U32 bucketSizeLog; /* Log bucket size for collision resolution, at most 8 */ - U32 minMatchLength; /* Minimum match length */ - U32 hashRateLog; /* Log number of entries to skip */ - U32 windowLog; /* Window log for the LDM */ -} ldmParams_t; - -typedef struct { - U32 offset; - U32 litLength; - U32 matchLength; -} rawSeq; - -typedef struct { - rawSeq* seq; /* The start of the sequences */ - size_t pos; /* The position where reading stopped. <= size. */ - size_t size; /* The number of sequences. <= capacity. */ - size_t capacity; /* The capacity starting from `seq` pointer */ -} rawSeqStore_t; - -typedef struct { - int collectSequences; - ZSTD_Sequence* seqStart; - size_t seqIndex; - size_t maxSequences; -} SeqCollector; - -struct ZSTD_CCtx_params_s { - ZSTD_format_e format; - ZSTD_compressionParameters cParams; - ZSTD_frameParameters fParams; - - int compressionLevel; - int forceWindow; /* force back-references to respect limit of - * 1< 63) ? ZSTD_highbit32(litLength) + LL_deltaCode : LL_Code[litLength]; -} - -/* ZSTD_MLcode() : - * note : mlBase = matchLength - MINMATCH; - * because it's the format it's stored in seqStore->sequences */ -MEM_STATIC U32 ZSTD_MLcode(U32 mlBase) -{ - static const BYTE ML_Code[128] = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, - 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, - 32, 32, 33, 33, 34, 34, 35, 35, 36, 36, 36, 36, 37, 37, 37, 37, - 38, 38, 38, 38, 38, 38, 38, 38, 39, 39, 39, 39, 39, 39, 39, 39, - 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, - 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, - 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, - 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42 }; - static const U32 ML_deltaCode = 36; - return (mlBase > 127) ? ZSTD_highbit32(mlBase) + ML_deltaCode : ML_Code[mlBase]; -} - -typedef struct repcodes_s { - U32 rep[3]; -} repcodes_t; - -MEM_STATIC repcodes_t ZSTD_updateRep(U32 const rep[3], U32 const offset, U32 const ll0) -{ - repcodes_t newReps; - if (offset >= ZSTD_REP_NUM) { /* full offset */ - newReps.rep[2] = rep[1]; - newReps.rep[1] = rep[0]; - newReps.rep[0] = offset - ZSTD_REP_MOVE; - } else { /* repcode */ - U32 const repCode = offset + ll0; - if (repCode > 0) { /* note : if repCode==0, no change */ - U32 const currentOffset = (repCode==ZSTD_REP_NUM) ? (rep[0] - 1) : rep[repCode]; - newReps.rep[2] = (repCode >= 2) ? rep[1] : rep[2]; - newReps.rep[1] = rep[0]; - newReps.rep[0] = currentOffset; - } else { /* repCode == 0 */ - memcpy(&newReps, rep, sizeof(newReps)); - } - } - return newReps; -} - -/* ZSTD_cParam_withinBounds: - * @return 1 if value is within cParam bounds, - * 0 otherwise */ -MEM_STATIC int ZSTD_cParam_withinBounds(ZSTD_cParameter cParam, int value) -{ - ZSTD_bounds const bounds = ZSTD_cParam_getBounds(cParam); - if (ZSTD_isError(bounds.error)) return 0; - if (value < bounds.lowerBound) return 0; - if (value > bounds.upperBound) return 0; - return 1; -} - -/* ZSTD_noCompressBlock() : - * Writes uncompressed block to dst buffer from given src. - * Returns the size of the block */ -MEM_STATIC size_t ZSTD_noCompressBlock (void* dst, size_t dstCapacity, const void* src, size_t srcSize, U32 lastBlock) -{ - U32 const cBlockHeader24 = lastBlock + (((U32)bt_raw)<<1) + (U32)(srcSize << 3); - RETURN_ERROR_IF(srcSize + ZSTD_blockHeaderSize > dstCapacity, - dstSize_tooSmall, "dst buf too small for uncompressed block"); - MEM_writeLE24(dst, cBlockHeader24); - memcpy((BYTE*)dst + ZSTD_blockHeaderSize, src, srcSize); - return ZSTD_blockHeaderSize + srcSize; -} - -MEM_STATIC size_t ZSTD_rleCompressBlock (void* dst, size_t dstCapacity, BYTE src, size_t srcSize, U32 lastBlock) -{ - BYTE* const op = (BYTE*)dst; - U32 const cBlockHeader = lastBlock + (((U32)bt_rle)<<1) + (U32)(srcSize << 3); - RETURN_ERROR_IF(dstCapacity < 4, dstSize_tooSmall, ""); - MEM_writeLE24(op, cBlockHeader); - op[3] = src; - return 4; -} - - -/* ZSTD_minGain() : - * minimum compression required - * to generate a compress block or a compressed literals section. - * note : use same formula for both situations */ -MEM_STATIC size_t ZSTD_minGain(size_t srcSize, ZSTD_strategy strat) -{ - U32 const minlog = (strat>=ZSTD_btultra) ? (U32)(strat) - 1 : 6; - ZSTD_STATIC_ASSERT(ZSTD_btultra == 8); - assert(ZSTD_cParam_withinBounds(ZSTD_c_strategy, strat)); - return (srcSize >> minlog) + 2; -} - -MEM_STATIC int ZSTD_disableLiteralsCompression(const ZSTD_CCtx_params* cctxParams) -{ - switch (cctxParams->literalCompressionMode) { - case ZSTD_lcm_huffman: - return 0; - case ZSTD_lcm_uncompressed: - return 1; - default: - assert(0 /* impossible: pre-validated */); - /* fall-through */ - case ZSTD_lcm_auto: - return (cctxParams->cParams.strategy == ZSTD_fast) && (cctxParams->cParams.targetLength > 0); - } -} - -/*! ZSTD_safecopyLiterals() : - * memcpy() function that won't read beyond more than WILDCOPY_OVERLENGTH bytes past ilimit_w. - * Only called when the sequence ends past ilimit_w, so it only needs to be optimized for single - * large copies. - */ -static void ZSTD_safecopyLiterals(BYTE* op, BYTE const* ip, BYTE const* const iend, BYTE const* ilimit_w) { - assert(iend > ilimit_w); - if (ip <= ilimit_w) { - ZSTD_wildcopy(op, ip, ilimit_w - ip, ZSTD_no_overlap); - op += ilimit_w - ip; - ip = ilimit_w; - } - while (ip < iend) *op++ = *ip++; -} - -/*! ZSTD_storeSeq() : - * Store a sequence (litlen, litPtr, offCode and mlBase) into seqStore_t. - * `offCode` : distance to match + ZSTD_REP_MOVE (values <= ZSTD_REP_MOVE are repCodes). - * `mlBase` : matchLength - MINMATCH - * Allowed to overread literals up to litLimit. -*/ -HINT_INLINE UNUSED_ATTR -void ZSTD_storeSeq(seqStore_t* seqStorePtr, size_t litLength, const BYTE* literals, const BYTE* litLimit, U32 offCode, size_t mlBase) -{ - BYTE const* const litLimit_w = litLimit - WILDCOPY_OVERLENGTH; - BYTE const* const litEnd = literals + litLength; -#if defined(DEBUGLEVEL) && (DEBUGLEVEL >= 6) - static const BYTE* g_start = NULL; - if (g_start==NULL) g_start = (const BYTE*)literals; /* note : index only works for compression within a single segment */ - { U32 const pos = (U32)((const BYTE*)literals - g_start); - DEBUGLOG(6, "Cpos%7u :%3u literals, match%4u bytes at offCode%7u", - pos, (U32)litLength, (U32)mlBase+MINMATCH, (U32)offCode); - } -#endif - assert((size_t)(seqStorePtr->sequences - seqStorePtr->sequencesStart) < seqStorePtr->maxNbSeq); - /* copy Literals */ - assert(seqStorePtr->maxNbLit <= 128 KB); - assert(seqStorePtr->lit + litLength <= seqStorePtr->litStart + seqStorePtr->maxNbLit); - assert(literals + litLength <= litLimit); - if (litEnd <= litLimit_w) { - /* Common case we can use wildcopy. - * First copy 16 bytes, because literals are likely short. - */ - assert(WILDCOPY_OVERLENGTH >= 16); - ZSTD_copy16(seqStorePtr->lit, literals); - if (litLength > 16) { - ZSTD_wildcopy(seqStorePtr->lit+16, literals+16, (ptrdiff_t)litLength-16, ZSTD_no_overlap); - } - } else { - ZSTD_safecopyLiterals(seqStorePtr->lit, literals, litEnd, litLimit_w); - } - seqStorePtr->lit += litLength; - - /* literal Length */ - if (litLength>0xFFFF) { - assert(seqStorePtr->longLengthID == 0); /* there can only be a single long length */ - seqStorePtr->longLengthID = 1; - seqStorePtr->longLengthPos = (U32)(seqStorePtr->sequences - seqStorePtr->sequencesStart); - } - seqStorePtr->sequences[0].litLength = (U16)litLength; - - /* match offset */ - seqStorePtr->sequences[0].offset = offCode + 1; - - /* match Length */ - if (mlBase>0xFFFF) { - assert(seqStorePtr->longLengthID == 0); /* there can only be a single long length */ - seqStorePtr->longLengthID = 2; - seqStorePtr->longLengthPos = (U32)(seqStorePtr->sequences - seqStorePtr->sequencesStart); - } - seqStorePtr->sequences[0].matchLength = (U16)mlBase; - - seqStorePtr->sequences++; -} - - -/*-************************************* -* Match length counter -***************************************/ -static unsigned ZSTD_NbCommonBytes (size_t val) -{ - if (MEM_isLittleEndian()) { - if (MEM_64bits()) { -# if defined(_MSC_VER) && defined(_WIN64) - unsigned long r = 0; - return _BitScanForward64( &r, (U64)val ) ? (unsigned)(r >> 3) : 0; -# elif defined(__GNUC__) && (__GNUC__ >= 4) - return (__builtin_ctzll((U64)val) >> 3); -# else - static const int DeBruijnBytePos[64] = { 0, 0, 0, 0, 0, 1, 1, 2, - 0, 3, 1, 3, 1, 4, 2, 7, - 0, 2, 3, 6, 1, 5, 3, 5, - 1, 3, 4, 4, 2, 5, 6, 7, - 7, 0, 1, 2, 3, 3, 4, 6, - 2, 6, 5, 5, 3, 4, 5, 6, - 7, 1, 2, 4, 6, 4, 4, 5, - 7, 2, 6, 5, 7, 6, 7, 7 }; - return DeBruijnBytePos[((U64)((val & -(long long)val) * 0x0218A392CDABBD3FULL)) >> 58]; -# endif - } else { /* 32 bits */ -# if defined(_MSC_VER) - unsigned long r=0; - return _BitScanForward( &r, (U32)val ) ? (unsigned)(r >> 3) : 0; -# elif defined(__GNUC__) && (__GNUC__ >= 3) - return (__builtin_ctz((U32)val) >> 3); -# else - static const int DeBruijnBytePos[32] = { 0, 0, 3, 0, 3, 1, 3, 0, - 3, 2, 2, 1, 3, 2, 0, 1, - 3, 3, 1, 2, 2, 2, 2, 0, - 3, 1, 2, 0, 1, 0, 1, 1 }; - return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >> 27]; -# endif - } - } else { /* Big Endian CPU */ - if (MEM_64bits()) { -# if defined(_MSC_VER) && defined(_WIN64) - unsigned long r = 0; - return _BitScanReverse64( &r, val ) ? (unsigned)(r >> 3) : 0; -# elif defined(__GNUC__) && (__GNUC__ >= 4) - return (__builtin_clzll(val) >> 3); -# else - unsigned r; - const unsigned n32 = sizeof(size_t)*4; /* calculate this way due to compiler complaining in 32-bits mode */ - if (!(val>>n32)) { r=4; } else { r=0; val>>=n32; } - if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; } - r += (!val); - return r; -# endif - } else { /* 32 bits */ -# if defined(_MSC_VER) - unsigned long r = 0; - return _BitScanReverse( &r, (unsigned long)val ) ? (unsigned)(r >> 3) : 0; -# elif defined(__GNUC__) && (__GNUC__ >= 3) - return (__builtin_clz((U32)val) >> 3); -# else - unsigned r; - if (!(val>>16)) { r=2; val>>=8; } else { r=0; val>>=24; } - r += (!val); - return r; -# endif - } } -} - - -MEM_STATIC size_t ZSTD_count(const BYTE* pIn, const BYTE* pMatch, const BYTE* const pInLimit) -{ - const BYTE* const pStart = pIn; - const BYTE* const pInLoopLimit = pInLimit - (sizeof(size_t)-1); - - if (pIn < pInLoopLimit) { - { size_t const diff = MEM_readST(pMatch) ^ MEM_readST(pIn); - if (diff) return ZSTD_NbCommonBytes(diff); } - pIn+=sizeof(size_t); pMatch+=sizeof(size_t); - while (pIn < pInLoopLimit) { - size_t const diff = MEM_readST(pMatch) ^ MEM_readST(pIn); - if (!diff) { pIn+=sizeof(size_t); pMatch+=sizeof(size_t); continue; } - pIn += ZSTD_NbCommonBytes(diff); - return (size_t)(pIn - pStart); - } } - if (MEM_64bits() && (pIn<(pInLimit-3)) && (MEM_read32(pMatch) == MEM_read32(pIn))) { pIn+=4; pMatch+=4; } - if ((pIn<(pInLimit-1)) && (MEM_read16(pMatch) == MEM_read16(pIn))) { pIn+=2; pMatch+=2; } - if ((pIn> (32-h) ; } -MEM_STATIC size_t ZSTD_hash3Ptr(const void* ptr, U32 h) { return ZSTD_hash3(MEM_readLE32(ptr), h); } /* only in zstd_opt.h */ - -static const U32 prime4bytes = 2654435761U; -static U32 ZSTD_hash4(U32 u, U32 h) { return (u * prime4bytes) >> (32-h) ; } -static size_t ZSTD_hash4Ptr(const void* ptr, U32 h) { return ZSTD_hash4(MEM_read32(ptr), h); } - -static const U64 prime5bytes = 889523592379ULL; -static size_t ZSTD_hash5(U64 u, U32 h) { return (size_t)(((u << (64-40)) * prime5bytes) >> (64-h)) ; } -static size_t ZSTD_hash5Ptr(const void* p, U32 h) { return ZSTD_hash5(MEM_readLE64(p), h); } - -static const U64 prime6bytes = 227718039650203ULL; -static size_t ZSTD_hash6(U64 u, U32 h) { return (size_t)(((u << (64-48)) * prime6bytes) >> (64-h)) ; } -static size_t ZSTD_hash6Ptr(const void* p, U32 h) { return ZSTD_hash6(MEM_readLE64(p), h); } - -static const U64 prime7bytes = 58295818150454627ULL; -static size_t ZSTD_hash7(U64 u, U32 h) { return (size_t)(((u << (64-56)) * prime7bytes) >> (64-h)) ; } -static size_t ZSTD_hash7Ptr(const void* p, U32 h) { return ZSTD_hash7(MEM_readLE64(p), h); } - -static const U64 prime8bytes = 0xCF1BBCDCB7A56463ULL; -static size_t ZSTD_hash8(U64 u, U32 h) { return (size_t)(((u) * prime8bytes) >> (64-h)) ; } -static size_t ZSTD_hash8Ptr(const void* p, U32 h) { return ZSTD_hash8(MEM_readLE64(p), h); } - -MEM_STATIC size_t ZSTD_hashPtr(const void* p, U32 hBits, U32 mls) -{ - switch(mls) - { - default: - case 4: return ZSTD_hash4Ptr(p, hBits); - case 5: return ZSTD_hash5Ptr(p, hBits); - case 6: return ZSTD_hash6Ptr(p, hBits); - case 7: return ZSTD_hash7Ptr(p, hBits); - case 8: return ZSTD_hash8Ptr(p, hBits); - } -} - -/** ZSTD_ipow() : - * Return base^exponent. - */ -static U64 ZSTD_ipow(U64 base, U64 exponent) -{ - U64 power = 1; - while (exponent) { - if (exponent & 1) power *= base; - exponent >>= 1; - base *= base; - } - return power; -} - -#define ZSTD_ROLL_HASH_CHAR_OFFSET 10 - -/** ZSTD_rollingHash_append() : - * Add the buffer to the hash value. - */ -static U64 ZSTD_rollingHash_append(U64 hash, void const* buf, size_t size) -{ - BYTE const* istart = (BYTE const*)buf; - size_t pos; - for (pos = 0; pos < size; ++pos) { - hash *= prime8bytes; - hash += istart[pos] + ZSTD_ROLL_HASH_CHAR_OFFSET; - } - return hash; -} - -/** ZSTD_rollingHash_compute() : - * Compute the rolling hash value of the buffer. - */ -MEM_STATIC U64 ZSTD_rollingHash_compute(void const* buf, size_t size) -{ - return ZSTD_rollingHash_append(0, buf, size); -} - -/** ZSTD_rollingHash_primePower() : - * Compute the primePower to be passed to ZSTD_rollingHash_rotate() for a hash - * over a window of length bytes. - */ -MEM_STATIC U64 ZSTD_rollingHash_primePower(U32 length) -{ - return ZSTD_ipow(prime8bytes, length - 1); -} - -/** ZSTD_rollingHash_rotate() : - * Rotate the rolling hash by one byte. - */ -MEM_STATIC U64 ZSTD_rollingHash_rotate(U64 hash, BYTE toRemove, BYTE toAdd, U64 primePower) -{ - hash -= (toRemove + ZSTD_ROLL_HASH_CHAR_OFFSET) * primePower; - hash *= prime8bytes; - hash += toAdd + ZSTD_ROLL_HASH_CHAR_OFFSET; - return hash; -} - -/*-************************************* -* Round buffer management -***************************************/ -#if (ZSTD_WINDOWLOG_MAX_64 > 31) -# error "ZSTD_WINDOWLOG_MAX is too large : would overflow ZSTD_CURRENT_MAX" -#endif -/* Max current allowed */ -#define ZSTD_CURRENT_MAX ((3U << 29) + (1U << ZSTD_WINDOWLOG_MAX)) -/* Maximum chunk size before overflow correction needs to be called again */ -#define ZSTD_CHUNKSIZE_MAX \ - ( ((U32)-1) /* Maximum ending current index */ \ - - ZSTD_CURRENT_MAX) /* Maximum beginning lowLimit */ - -/** - * ZSTD_window_clear(): - * Clears the window containing the history by simply setting it to empty. - */ -MEM_STATIC void ZSTD_window_clear(ZSTD_window_t* window) -{ - size_t const endT = (size_t)(window->nextSrc - window->base); - U32 const end = (U32)endT; - - window->lowLimit = end; - window->dictLimit = end; -} - -/** - * ZSTD_window_hasExtDict(): - * Returns non-zero if the window has a non-empty extDict. - */ -MEM_STATIC U32 ZSTD_window_hasExtDict(ZSTD_window_t const window) -{ - return window.lowLimit < window.dictLimit; -} - -/** - * ZSTD_matchState_dictMode(): - * Inspects the provided matchState and figures out what dictMode should be - * passed to the compressor. - */ -MEM_STATIC ZSTD_dictMode_e ZSTD_matchState_dictMode(const ZSTD_matchState_t *ms) -{ - return ZSTD_window_hasExtDict(ms->window) ? - ZSTD_extDict : - ms->dictMatchState != NULL ? - ZSTD_dictMatchState : - ZSTD_noDict; -} - -/** - * ZSTD_window_needOverflowCorrection(): - * Returns non-zero if the indices are getting too large and need overflow - * protection. - */ -MEM_STATIC U32 ZSTD_window_needOverflowCorrection(ZSTD_window_t const window, - void const* srcEnd) -{ - U32 const current = (U32)((BYTE const*)srcEnd - window.base); - return current > ZSTD_CURRENT_MAX; -} - -/** - * ZSTD_window_correctOverflow(): - * Reduces the indices to protect from index overflow. - * Returns the correction made to the indices, which must be applied to every - * stored index. - * - * The least significant cycleLog bits of the indices must remain the same, - * which may be 0. Every index up to maxDist in the past must be valid. - * NOTE: (maxDist & cycleMask) must be zero. - */ -MEM_STATIC U32 ZSTD_window_correctOverflow(ZSTD_window_t* window, U32 cycleLog, - U32 maxDist, void const* src) -{ - /* preemptive overflow correction: - * 1. correction is large enough: - * lowLimit > (3<<29) ==> current > 3<<29 + 1< (3<<29 + 1< (3<<29) - (1< (3<<29) - (1<<30) (NOTE: chainLog <= 30) - * > 1<<29 - * - * 2. (ip+ZSTD_CHUNKSIZE_MAX - cctx->base) doesn't overflow: - * After correction, current is less than (1<base < 1<<32. - * 3. (cctx->lowLimit + 1< 3<<29 + 1<base); - U32 const currentCycle0 = current & cycleMask; - /* Exclude zero so that newCurrent - maxDist >= 1. */ - U32 const currentCycle1 = currentCycle0 == 0 ? (1U << cycleLog) : currentCycle0; - U32 const newCurrent = currentCycle1 + maxDist; - U32 const correction = current - newCurrent; - assert((maxDist & cycleMask) == 0); - assert(current > newCurrent); - /* Loose bound, should be around 1<<29 (see above) */ - assert(correction > 1<<28); - - window->base += correction; - window->dictBase += correction; - if (window->lowLimit <= correction) window->lowLimit = 1; - else window->lowLimit -= correction; - if (window->dictLimit <= correction) window->dictLimit = 1; - else window->dictLimit -= correction; - - /* Ensure we can still reference the full window. */ - assert(newCurrent >= maxDist); - assert(newCurrent - maxDist >= 1); - /* Ensure that lowLimit and dictLimit didn't underflow. */ - assert(window->lowLimit <= newCurrent); - assert(window->dictLimit <= newCurrent); - - DEBUGLOG(4, "Correction of 0x%x bytes to lowLimit=0x%x", correction, - window->lowLimit); - return correction; -} - -/** - * ZSTD_window_enforceMaxDist(): - * Updates lowLimit so that: - * (srcEnd - base) - lowLimit == maxDist + loadedDictEnd - * - * It ensures index is valid as long as index >= lowLimit. - * This must be called before a block compression call. - * - * loadedDictEnd is only defined if a dictionary is in use for current compression. - * As the name implies, loadedDictEnd represents the index at end of dictionary. - * The value lies within context's referential, it can be directly compared to blockEndIdx. - * - * If loadedDictEndPtr is NULL, no dictionary is in use, and we use loadedDictEnd == 0. - * If loadedDictEndPtr is not NULL, we set it to zero after updating lowLimit. - * This is because dictionaries are allowed to be referenced fully - * as long as the last byte of the dictionary is in the window. - * Once input has progressed beyond window size, dictionary cannot be referenced anymore. - * - * In normal dict mode, the dictionary lies between lowLimit and dictLimit. - * In dictMatchState mode, lowLimit and dictLimit are the same, - * and the dictionary is below them. - * forceWindow and dictMatchState are therefore incompatible. - */ -MEM_STATIC void -ZSTD_window_enforceMaxDist(ZSTD_window_t* window, - const void* blockEnd, - U32 maxDist, - U32* loadedDictEndPtr, - const ZSTD_matchState_t** dictMatchStatePtr) -{ - U32 const blockEndIdx = (U32)((BYTE const*)blockEnd - window->base); - U32 const loadedDictEnd = (loadedDictEndPtr != NULL) ? *loadedDictEndPtr : 0; - DEBUGLOG(5, "ZSTD_window_enforceMaxDist: blockEndIdx=%u, maxDist=%u, loadedDictEnd=%u", - (unsigned)blockEndIdx, (unsigned)maxDist, (unsigned)loadedDictEnd); - - /* - When there is no dictionary : loadedDictEnd == 0. - In which case, the test (blockEndIdx > maxDist) is merely to avoid - overflowing next operation `newLowLimit = blockEndIdx - maxDist`. - - When there is a standard dictionary : - Index referential is copied from the dictionary, - which means it starts from 0. - In which case, loadedDictEnd == dictSize, - and it makes sense to compare `blockEndIdx > maxDist + dictSize` - since `blockEndIdx` also starts from zero. - - When there is an attached dictionary : - loadedDictEnd is expressed within the referential of the context, - so it can be directly compared against blockEndIdx. - */ - if (blockEndIdx > maxDist + loadedDictEnd) { - U32 const newLowLimit = blockEndIdx - maxDist; - if (window->lowLimit < newLowLimit) window->lowLimit = newLowLimit; - if (window->dictLimit < window->lowLimit) { - DEBUGLOG(5, "Update dictLimit to match lowLimit, from %u to %u", - (unsigned)window->dictLimit, (unsigned)window->lowLimit); - window->dictLimit = window->lowLimit; - } - /* On reaching window size, dictionaries are invalidated */ - if (loadedDictEndPtr) *loadedDictEndPtr = 0; - if (dictMatchStatePtr) *dictMatchStatePtr = NULL; - } -} - -/* Similar to ZSTD_window_enforceMaxDist(), - * but only invalidates dictionary - * when input progresses beyond window size. - * assumption : loadedDictEndPtr and dictMatchStatePtr are valid (non NULL) - * loadedDictEnd uses same referential as window->base - * maxDist is the window size */ -MEM_STATIC void -ZSTD_checkDictValidity(const ZSTD_window_t* window, - const void* blockEnd, - U32 maxDist, - U32* loadedDictEndPtr, - const ZSTD_matchState_t** dictMatchStatePtr) -{ - assert(loadedDictEndPtr != NULL); - assert(dictMatchStatePtr != NULL); - { U32 const blockEndIdx = (U32)((BYTE const*)blockEnd - window->base); - U32 const loadedDictEnd = *loadedDictEndPtr; - DEBUGLOG(5, "ZSTD_checkDictValidity: blockEndIdx=%u, maxDist=%u, loadedDictEnd=%u", - (unsigned)blockEndIdx, (unsigned)maxDist, (unsigned)loadedDictEnd); - assert(blockEndIdx >= loadedDictEnd); - - if (blockEndIdx > loadedDictEnd + maxDist) { - /* On reaching window size, dictionaries are invalidated. - * For simplification, if window size is reached anywhere within next block, - * the dictionary is invalidated for the full block. - */ - DEBUGLOG(6, "invalidating dictionary for current block (distance > windowSize)"); - *loadedDictEndPtr = 0; - *dictMatchStatePtr = NULL; - } else { - if (*loadedDictEndPtr != 0) { - DEBUGLOG(6, "dictionary considered valid for current block"); - } } } -} - -MEM_STATIC void ZSTD_window_init(ZSTD_window_t* window) { - memset(window, 0, sizeof(*window)); - window->base = (BYTE const*)""; - window->dictBase = (BYTE const*)""; - window->dictLimit = 1; /* start from 1, so that 1st position is valid */ - window->lowLimit = 1; /* it ensures first and later CCtx usages compress the same */ - window->nextSrc = window->base + 1; /* see issue #1241 */ -} - -/** - * ZSTD_window_update(): - * Updates the window by appending [src, src + srcSize) to the window. - * If it is not contiguous, the current prefix becomes the extDict, and we - * forget about the extDict. Handles overlap of the prefix and extDict. - * Returns non-zero if the segment is contiguous. - */ -MEM_STATIC U32 ZSTD_window_update(ZSTD_window_t* window, - void const* src, size_t srcSize) -{ - BYTE const* const ip = (BYTE const*)src; - U32 contiguous = 1; - DEBUGLOG(5, "ZSTD_window_update"); - if (srcSize == 0) - return contiguous; - assert(window->base != NULL); - assert(window->dictBase != NULL); - /* Check if blocks follow each other */ - if (src != window->nextSrc) { - /* not contiguous */ - size_t const distanceFromBase = (size_t)(window->nextSrc - window->base); - DEBUGLOG(5, "Non contiguous blocks, new segment starts at %u", window->dictLimit); - window->lowLimit = window->dictLimit; - assert(distanceFromBase == (size_t)(U32)distanceFromBase); /* should never overflow */ - window->dictLimit = (U32)distanceFromBase; - window->dictBase = window->base; - window->base = ip - distanceFromBase; - /* ms->nextToUpdate = window->dictLimit; */ - if (window->dictLimit - window->lowLimit < HASH_READ_SIZE) window->lowLimit = window->dictLimit; /* too small extDict */ - contiguous = 0; - } - window->nextSrc = ip + srcSize; - /* if input and dictionary overlap : reduce dictionary (area presumed modified by input) */ - if ( (ip+srcSize > window->dictBase + window->lowLimit) - & (ip < window->dictBase + window->dictLimit)) { - ptrdiff_t const highInputIdx = (ip + srcSize) - window->dictBase; - U32 const lowLimitMax = (highInputIdx > (ptrdiff_t)window->dictLimit) ? window->dictLimit : (U32)highInputIdx; - window->lowLimit = lowLimitMax; - DEBUGLOG(5, "Overlapping extDict and input : new lowLimit = %u", window->lowLimit); - } - return contiguous; -} - -/** - * Returns the lowest allowed match index. It may either be in the ext-dict or the prefix. - */ -MEM_STATIC U32 ZSTD_getLowestMatchIndex(const ZSTD_matchState_t* ms, U32 current, unsigned windowLog) -{ - U32 const maxDistance = 1U << windowLog; - U32 const lowestValid = ms->window.lowLimit; - U32 const withinWindow = (current - lowestValid > maxDistance) ? current - maxDistance : lowestValid; - U32 const isDictionary = (ms->loadedDictEnd != 0); - U32 const matchLowest = isDictionary ? lowestValid : withinWindow; - return matchLowest; -} - -/** - * Returns the lowest allowed match index in the prefix. - */ -MEM_STATIC U32 ZSTD_getLowestPrefixIndex(const ZSTD_matchState_t* ms, U32 current, unsigned windowLog) -{ - U32 const maxDistance = 1U << windowLog; - U32 const lowestValid = ms->window.dictLimit; - U32 const withinWindow = (current - lowestValid > maxDistance) ? current - maxDistance : lowestValid; - U32 const isDictionary = (ms->loadedDictEnd != 0); - U32 const matchLowest = isDictionary ? lowestValid : withinWindow; - return matchLowest; -} - - - -/* debug functions */ -#if (DEBUGLEVEL>=2) - -MEM_STATIC double ZSTD_fWeight(U32 rawStat) -{ - U32 const fp_accuracy = 8; - U32 const fp_multiplier = (1 << fp_accuracy); - U32 const newStat = rawStat + 1; - U32 const hb = ZSTD_highbit32(newStat); - U32 const BWeight = hb * fp_multiplier; - U32 const FWeight = (newStat << fp_accuracy) >> hb; - U32 const weight = BWeight + FWeight; - assert(hb + fp_accuracy < 31); - return (double)weight / fp_multiplier; -} - -/* display a table content, - * listing each element, its frequency, and its predicted bit cost */ -MEM_STATIC void ZSTD_debugTable(const U32* table, U32 max) -{ - unsigned u, sum; - for (u=0, sum=0; u<=max; u++) sum += table[u]; - DEBUGLOG(2, "total nb elts: %u", sum); - for (u=0; u<=max; u++) { - DEBUGLOG(2, "%2u: %5u (%.2f)", - u, table[u], ZSTD_fWeight(sum) - ZSTD_fWeight(table[u]) ); - } -} - -#endif - - -#if defined (__cplusplus) -} -#endif - -/* =============================================================== - * Shared internal declarations - * These prototypes may be called from sources not in lib/compress - * =============================================================== */ - -/* ZSTD_loadCEntropy() : - * dict : must point at beginning of a valid zstd dictionary. - * return : size of dictionary header (size of magic number + dict ID + entropy tables) - * assumptions : magic number supposed already checked - * and dictSize >= 8 */ -size_t ZSTD_loadCEntropy(ZSTD_compressedBlockState_t* bs, void* workspace, - short* offcodeNCount, unsigned* offcodeMaxValue, - const void* const dict, size_t dictSize); - -void ZSTD_reset_compressedBlockState(ZSTD_compressedBlockState_t* bs); - -/* ============================================================== - * Private declarations - * These prototypes shall only be called from within lib/compress - * ============================================================== */ - -/* ZSTD_getCParamsFromCCtxParams() : - * cParams are built depending on compressionLevel, src size hints, - * LDM and manually set compression parameters. - * Note: srcSizeHint == 0 means 0! - */ -ZSTD_compressionParameters ZSTD_getCParamsFromCCtxParams( - const ZSTD_CCtx_params* CCtxParams, U64 srcSizeHint, size_t dictSize); - -/*! ZSTD_initCStream_internal() : - * Private use only. Init streaming operation. - * expects params to be valid. - * must receive dict, or cdict, or none, but not both. - * @return : 0, or an error code */ -size_t ZSTD_initCStream_internal(ZSTD_CStream* zcs, - const void* dict, size_t dictSize, - const ZSTD_CDict* cdict, - const ZSTD_CCtx_params* params, unsigned long long pledgedSrcSize); - -void ZSTD_resetSeqStore(seqStore_t* ssPtr); - -/*! ZSTD_getCParamsFromCDict() : - * as the name implies */ -ZSTD_compressionParameters ZSTD_getCParamsFromCDict(const ZSTD_CDict* cdict); - -/* ZSTD_compressBegin_advanced_internal() : - * Private use only. To be called from zstdmt_compress.c. */ -size_t ZSTD_compressBegin_advanced_internal(ZSTD_CCtx* cctx, - const void* dict, size_t dictSize, - ZSTD_dictContentType_e dictContentType, - ZSTD_dictTableLoadMethod_e dtlm, - const ZSTD_CDict* cdict, - const ZSTD_CCtx_params* params, - unsigned long long pledgedSrcSize); - -/* ZSTD_compress_advanced_internal() : - * Private use only. To be called from zstdmt_compress.c. */ -size_t ZSTD_compress_advanced_internal(ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const void* dict,size_t dictSize, - const ZSTD_CCtx_params* params); - - -/* ZSTD_writeLastEmptyBlock() : - * output an empty Block with end-of-frame mark to complete a frame - * @return : size of data written into `dst` (== ZSTD_blockHeaderSize (defined in zstd_internal.h)) - * or an error code if `dstCapacity` is too small ( 1 */ -U32 ZSTD_cycleLog(U32 hashLog, ZSTD_strategy strat); - -#endif /* ZSTD_COMPRESS_H */ -/**** ended inlining zstd_compress_internal.h ****/ - - -size_t ZSTD_noCompressLiterals (void* dst, size_t dstCapacity, const void* src, size_t srcSize); - -size_t ZSTD_compressRleLiteralsBlock (void* dst, size_t dstCapacity, const void* src, size_t srcSize); - -size_t ZSTD_compressLiterals (ZSTD_hufCTables_t const* prevHuf, - ZSTD_hufCTables_t* nextHuf, - ZSTD_strategy strategy, int disableLiteralCompression, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - void* entropyWorkspace, size_t entropyWorkspaceSize, - const int bmi2); - -#endif /* ZSTD_COMPRESS_LITERALS_H */ -/**** ended inlining zstd_compress_literals.h ****/ - -size_t ZSTD_noCompressLiterals (void* dst, size_t dstCapacity, const void* src, size_t srcSize) -{ - BYTE* const ostart = (BYTE* const)dst; - U32 const flSize = 1 + (srcSize>31) + (srcSize>4095); - - RETURN_ERROR_IF(srcSize + flSize > dstCapacity, dstSize_tooSmall, ""); - - switch(flSize) - { - case 1: /* 2 - 1 - 5 */ - ostart[0] = (BYTE)((U32)set_basic + (srcSize<<3)); - break; - case 2: /* 2 - 2 - 12 */ - MEM_writeLE16(ostart, (U16)((U32)set_basic + (1<<2) + (srcSize<<4))); - break; - case 3: /* 2 - 2 - 20 */ - MEM_writeLE32(ostart, (U32)((U32)set_basic + (3<<2) + (srcSize<<4))); - break; - default: /* not necessary : flSize is {1,2,3} */ - assert(0); - } - - memcpy(ostart + flSize, src, srcSize); - DEBUGLOG(5, "Raw literals: %u -> %u", (U32)srcSize, (U32)(srcSize + flSize)); - return srcSize + flSize; -} - -size_t ZSTD_compressRleLiteralsBlock (void* dst, size_t dstCapacity, const void* src, size_t srcSize) -{ - BYTE* const ostart = (BYTE* const)dst; - U32 const flSize = 1 + (srcSize>31) + (srcSize>4095); - - (void)dstCapacity; /* dstCapacity already guaranteed to be >=4, hence large enough */ - - switch(flSize) - { - case 1: /* 2 - 1 - 5 */ - ostart[0] = (BYTE)((U32)set_rle + (srcSize<<3)); - break; - case 2: /* 2 - 2 - 12 */ - MEM_writeLE16(ostart, (U16)((U32)set_rle + (1<<2) + (srcSize<<4))); - break; - case 3: /* 2 - 2 - 20 */ - MEM_writeLE32(ostart, (U32)((U32)set_rle + (3<<2) + (srcSize<<4))); - break; - default: /* not necessary : flSize is {1,2,3} */ - assert(0); - } - - ostart[flSize] = *(const BYTE*)src; - DEBUGLOG(5, "RLE literals: %u -> %u", (U32)srcSize, (U32)flSize + 1); - return flSize+1; -} - -size_t ZSTD_compressLiterals (ZSTD_hufCTables_t const* prevHuf, - ZSTD_hufCTables_t* nextHuf, - ZSTD_strategy strategy, int disableLiteralCompression, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - void* entropyWorkspace, size_t entropyWorkspaceSize, - const int bmi2) -{ - size_t const minGain = ZSTD_minGain(srcSize, strategy); - size_t const lhSize = 3 + (srcSize >= 1 KB) + (srcSize >= 16 KB); - BYTE* const ostart = (BYTE*)dst; - U32 singleStream = srcSize < 256; - symbolEncodingType_e hType = set_compressed; - size_t cLitSize; - - DEBUGLOG(5,"ZSTD_compressLiterals (disableLiteralCompression=%i srcSize=%u)", - disableLiteralCompression, (U32)srcSize); - - /* Prepare nextEntropy assuming reusing the existing table */ - memcpy(nextHuf, prevHuf, sizeof(*prevHuf)); - - if (disableLiteralCompression) - return ZSTD_noCompressLiterals(dst, dstCapacity, src, srcSize); - - /* small ? don't even attempt compression (speed opt) */ -# define COMPRESS_LITERALS_SIZE_MIN 63 - { size_t const minLitSize = (prevHuf->repeatMode == HUF_repeat_valid) ? 6 : COMPRESS_LITERALS_SIZE_MIN; - if (srcSize <= minLitSize) return ZSTD_noCompressLiterals(dst, dstCapacity, src, srcSize); - } - - RETURN_ERROR_IF(dstCapacity < lhSize+1, dstSize_tooSmall, "not enough space for compression"); - { HUF_repeat repeat = prevHuf->repeatMode; - int const preferRepeat = strategy < ZSTD_lazy ? srcSize <= 1024 : 0; - if (repeat == HUF_repeat_valid && lhSize == 3) singleStream = 1; - cLitSize = singleStream ? - HUF_compress1X_repeat( - ostart+lhSize, dstCapacity-lhSize, src, srcSize, - HUF_SYMBOLVALUE_MAX, HUF_TABLELOG_DEFAULT, entropyWorkspace, entropyWorkspaceSize, - (HUF_CElt*)nextHuf->CTable, &repeat, preferRepeat, bmi2) : - HUF_compress4X_repeat( - ostart+lhSize, dstCapacity-lhSize, src, srcSize, - HUF_SYMBOLVALUE_MAX, HUF_TABLELOG_DEFAULT, entropyWorkspace, entropyWorkspaceSize, - (HUF_CElt*)nextHuf->CTable, &repeat, preferRepeat, bmi2); - if (repeat != HUF_repeat_none) { - /* reused the existing table */ - DEBUGLOG(5, "Reusing previous huffman table"); - hType = set_repeat; - } - } - - if ((cLitSize==0) | (cLitSize >= srcSize - minGain) | ERR_isError(cLitSize)) { - memcpy(nextHuf, prevHuf, sizeof(*prevHuf)); - return ZSTD_noCompressLiterals(dst, dstCapacity, src, srcSize); - } - if (cLitSize==1) { - memcpy(nextHuf, prevHuf, sizeof(*prevHuf)); - return ZSTD_compressRleLiteralsBlock(dst, dstCapacity, src, srcSize); - } - - if (hType == set_compressed) { - /* using a newly constructed table */ - nextHuf->repeatMode = HUF_repeat_check; - } - - /* Build header */ - switch(lhSize) - { - case 3: /* 2 - 2 - 10 - 10 */ - { U32 const lhc = hType + ((!singleStream) << 2) + ((U32)srcSize<<4) + ((U32)cLitSize<<14); - MEM_writeLE24(ostart, lhc); - break; - } - case 4: /* 2 - 2 - 14 - 14 */ - { U32 const lhc = hType + (2 << 2) + ((U32)srcSize<<4) + ((U32)cLitSize<<18); - MEM_writeLE32(ostart, lhc); - break; - } - case 5: /* 2 - 2 - 18 - 18 */ - { U32 const lhc = hType + (3 << 2) + ((U32)srcSize<<4) + ((U32)cLitSize<<22); - MEM_writeLE32(ostart, lhc); - ostart[4] = (BYTE)(cLitSize >> 10); - break; - } - default: /* not possible : lhSize is {3,4,5} */ - assert(0); - } - DEBUGLOG(5, "Compressed literals: %u -> %u", (U32)srcSize, (U32)(lhSize+cLitSize)); - return lhSize+cLitSize; -} -/**** ended inlining compress/zstd_compress_literals.c ****/ -/**** start inlining compress/zstd_compress_sequences.c ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - - /*-************************************* - * Dependencies - ***************************************/ -/**** start inlining zstd_compress_sequences.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_COMPRESS_SEQUENCES_H -#define ZSTD_COMPRESS_SEQUENCES_H - -/**** skipping file: ../common/fse.h ****/ -/**** skipping file: ../common/zstd_internal.h ****/ - -typedef enum { - ZSTD_defaultDisallowed = 0, - ZSTD_defaultAllowed = 1 -} ZSTD_defaultPolicy_e; - -symbolEncodingType_e -ZSTD_selectEncodingType( - FSE_repeat* repeatMode, unsigned const* count, unsigned const max, - size_t const mostFrequent, size_t nbSeq, unsigned const FSELog, - FSE_CTable const* prevCTable, - short const* defaultNorm, U32 defaultNormLog, - ZSTD_defaultPolicy_e const isDefaultAllowed, - ZSTD_strategy const strategy); - -size_t -ZSTD_buildCTable(void* dst, size_t dstCapacity, - FSE_CTable* nextCTable, U32 FSELog, symbolEncodingType_e type, - unsigned* count, U32 max, - const BYTE* codeTable, size_t nbSeq, - const S16* defaultNorm, U32 defaultNormLog, U32 defaultMax, - const FSE_CTable* prevCTable, size_t prevCTableSize, - void* entropyWorkspace, size_t entropyWorkspaceSize); - -size_t ZSTD_encodeSequences( - void* dst, size_t dstCapacity, - FSE_CTable const* CTable_MatchLength, BYTE const* mlCodeTable, - FSE_CTable const* CTable_OffsetBits, BYTE const* ofCodeTable, - FSE_CTable const* CTable_LitLength, BYTE const* llCodeTable, - seqDef const* sequences, size_t nbSeq, int longOffsets, int bmi2); - -size_t ZSTD_fseBitCost( - FSE_CTable const* ctable, - unsigned const* count, - unsigned const max); - -size_t ZSTD_crossEntropyCost(short const* norm, unsigned accuracyLog, - unsigned const* count, unsigned const max); -#endif /* ZSTD_COMPRESS_SEQUENCES_H */ -/**** ended inlining zstd_compress_sequences.h ****/ - -/** - * -log2(x / 256) lookup table for x in [0, 256). - * If x == 0: Return 0 - * Else: Return floor(-log2(x / 256) * 256) - */ -static unsigned const kInverseProbabilityLog256[256] = { - 0, 2048, 1792, 1642, 1536, 1453, 1386, 1329, 1280, 1236, 1197, 1162, - 1130, 1100, 1073, 1047, 1024, 1001, 980, 960, 941, 923, 906, 889, - 874, 859, 844, 830, 817, 804, 791, 779, 768, 756, 745, 734, - 724, 714, 704, 694, 685, 676, 667, 658, 650, 642, 633, 626, - 618, 610, 603, 595, 588, 581, 574, 567, 561, 554, 548, 542, - 535, 529, 523, 517, 512, 506, 500, 495, 489, 484, 478, 473, - 468, 463, 458, 453, 448, 443, 438, 434, 429, 424, 420, 415, - 411, 407, 402, 398, 394, 390, 386, 382, 377, 373, 370, 366, - 362, 358, 354, 350, 347, 343, 339, 336, 332, 329, 325, 322, - 318, 315, 311, 308, 305, 302, 298, 295, 292, 289, 286, 282, - 279, 276, 273, 270, 267, 264, 261, 258, 256, 253, 250, 247, - 244, 241, 239, 236, 233, 230, 228, 225, 222, 220, 217, 215, - 212, 209, 207, 204, 202, 199, 197, 194, 192, 190, 187, 185, - 182, 180, 178, 175, 173, 171, 168, 166, 164, 162, 159, 157, - 155, 153, 151, 149, 146, 144, 142, 140, 138, 136, 134, 132, - 130, 128, 126, 123, 121, 119, 117, 115, 114, 112, 110, 108, - 106, 104, 102, 100, 98, 96, 94, 93, 91, 89, 87, 85, - 83, 82, 80, 78, 76, 74, 73, 71, 69, 67, 66, 64, - 62, 61, 59, 57, 55, 54, 52, 50, 49, 47, 46, 44, - 42, 41, 39, 37, 36, 34, 33, 31, 30, 28, 26, 25, - 23, 22, 20, 19, 17, 16, 14, 13, 11, 10, 8, 7, - 5, 4, 2, 1, -}; - -static unsigned ZSTD_getFSEMaxSymbolValue(FSE_CTable const* ctable) { - void const* ptr = ctable; - U16 const* u16ptr = (U16 const*)ptr; - U32 const maxSymbolValue = MEM_read16(u16ptr + 1); - return maxSymbolValue; -} - -/** - * Returns the cost in bytes of encoding the normalized count header. - * Returns an error if any of the helper functions return an error. - */ -static size_t ZSTD_NCountCost(unsigned const* count, unsigned const max, - size_t const nbSeq, unsigned const FSELog) -{ - BYTE wksp[FSE_NCOUNTBOUND]; - S16 norm[MaxSeq + 1]; - const U32 tableLog = FSE_optimalTableLog(FSELog, nbSeq, max); - FORWARD_IF_ERROR(FSE_normalizeCount(norm, tableLog, count, nbSeq, max), ""); - return FSE_writeNCount(wksp, sizeof(wksp), norm, max, tableLog); -} - -/** - * Returns the cost in bits of encoding the distribution described by count - * using the entropy bound. - */ -static size_t ZSTD_entropyCost(unsigned const* count, unsigned const max, size_t const total) -{ - unsigned cost = 0; - unsigned s; - for (s = 0; s <= max; ++s) { - unsigned norm = (unsigned)((256 * count[s]) / total); - if (count[s] != 0 && norm == 0) - norm = 1; - assert(count[s] < total); - cost += count[s] * kInverseProbabilityLog256[norm]; - } - return cost >> 8; -} - -/** - * Returns the cost in bits of encoding the distribution in count using ctable. - * Returns an error if ctable cannot represent all the symbols in count. - */ -size_t ZSTD_fseBitCost( - FSE_CTable const* ctable, - unsigned const* count, - unsigned const max) -{ - unsigned const kAccuracyLog = 8; - size_t cost = 0; - unsigned s; - FSE_CState_t cstate; - FSE_initCState(&cstate, ctable); - if (ZSTD_getFSEMaxSymbolValue(ctable) < max) { - DEBUGLOG(5, "Repeat FSE_CTable has maxSymbolValue %u < %u", - ZSTD_getFSEMaxSymbolValue(ctable), max); - return ERROR(GENERIC); - } - for (s = 0; s <= max; ++s) { - unsigned const tableLog = cstate.stateLog; - unsigned const badCost = (tableLog + 1) << kAccuracyLog; - unsigned const bitCost = FSE_bitCost(cstate.symbolTT, tableLog, s, kAccuracyLog); - if (count[s] == 0) - continue; - if (bitCost >= badCost) { - DEBUGLOG(5, "Repeat FSE_CTable has Prob[%u] == 0", s); - return ERROR(GENERIC); - } - cost += (size_t)count[s] * bitCost; - } - return cost >> kAccuracyLog; -} - -/** - * Returns the cost in bits of encoding the distribution in count using the - * table described by norm. The max symbol support by norm is assumed >= max. - * norm must be valid for every symbol with non-zero probability in count. - */ -size_t ZSTD_crossEntropyCost(short const* norm, unsigned accuracyLog, - unsigned const* count, unsigned const max) -{ - unsigned const shift = 8 - accuracyLog; - size_t cost = 0; - unsigned s; - assert(accuracyLog <= 8); - for (s = 0; s <= max; ++s) { - unsigned const normAcc = (norm[s] != -1) ? (unsigned)norm[s] : 1; - unsigned const norm256 = normAcc << shift; - assert(norm256 > 0); - assert(norm256 < 256); - cost += count[s] * kInverseProbabilityLog256[norm256]; - } - return cost >> 8; -} - -symbolEncodingType_e -ZSTD_selectEncodingType( - FSE_repeat* repeatMode, unsigned const* count, unsigned const max, - size_t const mostFrequent, size_t nbSeq, unsigned const FSELog, - FSE_CTable const* prevCTable, - short const* defaultNorm, U32 defaultNormLog, - ZSTD_defaultPolicy_e const isDefaultAllowed, - ZSTD_strategy const strategy) -{ - ZSTD_STATIC_ASSERT(ZSTD_defaultDisallowed == 0 && ZSTD_defaultAllowed != 0); - if (mostFrequent == nbSeq) { - *repeatMode = FSE_repeat_none; - if (isDefaultAllowed && nbSeq <= 2) { - /* Prefer set_basic over set_rle when there are 2 or less symbols, - * since RLE uses 1 byte, but set_basic uses 5-6 bits per symbol. - * If basic encoding isn't possible, always choose RLE. - */ - DEBUGLOG(5, "Selected set_basic"); - return set_basic; - } - DEBUGLOG(5, "Selected set_rle"); - return set_rle; - } - if (strategy < ZSTD_lazy) { - if (isDefaultAllowed) { - size_t const staticFse_nbSeq_max = 1000; - size_t const mult = 10 - strategy; - size_t const baseLog = 3; - size_t const dynamicFse_nbSeq_min = (((size_t)1 << defaultNormLog) * mult) >> baseLog; /* 28-36 for offset, 56-72 for lengths */ - assert(defaultNormLog >= 5 && defaultNormLog <= 6); /* xx_DEFAULTNORMLOG */ - assert(mult <= 9 && mult >= 7); - if ( (*repeatMode == FSE_repeat_valid) - && (nbSeq < staticFse_nbSeq_max) ) { - DEBUGLOG(5, "Selected set_repeat"); - return set_repeat; - } - if ( (nbSeq < dynamicFse_nbSeq_min) - || (mostFrequent < (nbSeq >> (defaultNormLog-1))) ) { - DEBUGLOG(5, "Selected set_basic"); - /* The format allows default tables to be repeated, but it isn't useful. - * When using simple heuristics to select encoding type, we don't want - * to confuse these tables with dictionaries. When running more careful - * analysis, we don't need to waste time checking both repeating tables - * and default tables. - */ - *repeatMode = FSE_repeat_none; - return set_basic; - } - } - } else { - size_t const basicCost = isDefaultAllowed ? ZSTD_crossEntropyCost(defaultNorm, defaultNormLog, count, max) : ERROR(GENERIC); - size_t const repeatCost = *repeatMode != FSE_repeat_none ? ZSTD_fseBitCost(prevCTable, count, max) : ERROR(GENERIC); - size_t const NCountCost = ZSTD_NCountCost(count, max, nbSeq, FSELog); - size_t const compressedCost = (NCountCost << 3) + ZSTD_entropyCost(count, max, nbSeq); - - if (isDefaultAllowed) { - assert(!ZSTD_isError(basicCost)); - assert(!(*repeatMode == FSE_repeat_valid && ZSTD_isError(repeatCost))); - } - assert(!ZSTD_isError(NCountCost)); - assert(compressedCost < ERROR(maxCode)); - DEBUGLOG(5, "Estimated bit costs: basic=%u\trepeat=%u\tcompressed=%u", - (unsigned)basicCost, (unsigned)repeatCost, (unsigned)compressedCost); - if (basicCost <= repeatCost && basicCost <= compressedCost) { - DEBUGLOG(5, "Selected set_basic"); - assert(isDefaultAllowed); - *repeatMode = FSE_repeat_none; - return set_basic; - } - if (repeatCost <= compressedCost) { - DEBUGLOG(5, "Selected set_repeat"); - assert(!ZSTD_isError(repeatCost)); - return set_repeat; - } - assert(compressedCost < basicCost && compressedCost < repeatCost); - } - DEBUGLOG(5, "Selected set_compressed"); - *repeatMode = FSE_repeat_check; - return set_compressed; -} - -size_t -ZSTD_buildCTable(void* dst, size_t dstCapacity, - FSE_CTable* nextCTable, U32 FSELog, symbolEncodingType_e type, - unsigned* count, U32 max, - const BYTE* codeTable, size_t nbSeq, - const S16* defaultNorm, U32 defaultNormLog, U32 defaultMax, - const FSE_CTable* prevCTable, size_t prevCTableSize, - void* entropyWorkspace, size_t entropyWorkspaceSize) -{ - BYTE* op = (BYTE*)dst; - const BYTE* const oend = op + dstCapacity; - DEBUGLOG(6, "ZSTD_buildCTable (dstCapacity=%u)", (unsigned)dstCapacity); - - switch (type) { - case set_rle: - FORWARD_IF_ERROR(FSE_buildCTable_rle(nextCTable, (BYTE)max), ""); - RETURN_ERROR_IF(dstCapacity==0, dstSize_tooSmall, "not enough space"); - *op = codeTable[0]; - return 1; - case set_repeat: - memcpy(nextCTable, prevCTable, prevCTableSize); - return 0; - case set_basic: - FORWARD_IF_ERROR(FSE_buildCTable_wksp(nextCTable, defaultNorm, defaultMax, defaultNormLog, entropyWorkspace, entropyWorkspaceSize), ""); /* note : could be pre-calculated */ - return 0; - case set_compressed: { - S16 norm[MaxSeq + 1]; - size_t nbSeq_1 = nbSeq; - const U32 tableLog = FSE_optimalTableLog(FSELog, nbSeq, max); - if (count[codeTable[nbSeq-1]] > 1) { - count[codeTable[nbSeq-1]]--; - nbSeq_1--; - } - assert(nbSeq_1 > 1); - FORWARD_IF_ERROR(FSE_normalizeCount(norm, tableLog, count, nbSeq_1, max), ""); - { size_t const NCountSize = FSE_writeNCount(op, oend - op, norm, max, tableLog); /* overflow protected */ - FORWARD_IF_ERROR(NCountSize, "FSE_writeNCount failed"); - FORWARD_IF_ERROR(FSE_buildCTable_wksp(nextCTable, norm, max, tableLog, entropyWorkspace, entropyWorkspaceSize), ""); - return NCountSize; - } - } - default: assert(0); RETURN_ERROR(GENERIC, "impossible to reach"); - } -} - -FORCE_INLINE_TEMPLATE size_t -ZSTD_encodeSequences_body( - void* dst, size_t dstCapacity, - FSE_CTable const* CTable_MatchLength, BYTE const* mlCodeTable, - FSE_CTable const* CTable_OffsetBits, BYTE const* ofCodeTable, - FSE_CTable const* CTable_LitLength, BYTE const* llCodeTable, - seqDef const* sequences, size_t nbSeq, int longOffsets) -{ - BIT_CStream_t blockStream; - FSE_CState_t stateMatchLength; - FSE_CState_t stateOffsetBits; - FSE_CState_t stateLitLength; - - RETURN_ERROR_IF( - ERR_isError(BIT_initCStream(&blockStream, dst, dstCapacity)), - dstSize_tooSmall, "not enough space remaining"); - DEBUGLOG(6, "available space for bitstream : %i (dstCapacity=%u)", - (int)(blockStream.endPtr - blockStream.startPtr), - (unsigned)dstCapacity); - - /* first symbols */ - FSE_initCState2(&stateMatchLength, CTable_MatchLength, mlCodeTable[nbSeq-1]); - FSE_initCState2(&stateOffsetBits, CTable_OffsetBits, ofCodeTable[nbSeq-1]); - FSE_initCState2(&stateLitLength, CTable_LitLength, llCodeTable[nbSeq-1]); - BIT_addBits(&blockStream, sequences[nbSeq-1].litLength, LL_bits[llCodeTable[nbSeq-1]]); - if (MEM_32bits()) BIT_flushBits(&blockStream); - BIT_addBits(&blockStream, sequences[nbSeq-1].matchLength, ML_bits[mlCodeTable[nbSeq-1]]); - if (MEM_32bits()) BIT_flushBits(&blockStream); - if (longOffsets) { - U32 const ofBits = ofCodeTable[nbSeq-1]; - unsigned const extraBits = ofBits - MIN(ofBits, STREAM_ACCUMULATOR_MIN-1); - if (extraBits) { - BIT_addBits(&blockStream, sequences[nbSeq-1].offset, extraBits); - BIT_flushBits(&blockStream); - } - BIT_addBits(&blockStream, sequences[nbSeq-1].offset >> extraBits, - ofBits - extraBits); - } else { - BIT_addBits(&blockStream, sequences[nbSeq-1].offset, ofCodeTable[nbSeq-1]); - } - BIT_flushBits(&blockStream); - - { size_t n; - for (n=nbSeq-2 ; n= 64-7-(LLFSELog+MLFSELog+OffFSELog))) - BIT_flushBits(&blockStream); /* (7)*/ - BIT_addBits(&blockStream, sequences[n].litLength, llBits); - if (MEM_32bits() && ((llBits+mlBits)>24)) BIT_flushBits(&blockStream); - BIT_addBits(&blockStream, sequences[n].matchLength, mlBits); - if (MEM_32bits() || (ofBits+mlBits+llBits > 56)) BIT_flushBits(&blockStream); - if (longOffsets) { - unsigned const extraBits = ofBits - MIN(ofBits, STREAM_ACCUMULATOR_MIN-1); - if (extraBits) { - BIT_addBits(&blockStream, sequences[n].offset, extraBits); - BIT_flushBits(&blockStream); /* (7)*/ - } - BIT_addBits(&blockStream, sequences[n].offset >> extraBits, - ofBits - extraBits); /* 31 */ - } else { - BIT_addBits(&blockStream, sequences[n].offset, ofBits); /* 31 */ - } - BIT_flushBits(&blockStream); /* (7)*/ - DEBUGLOG(7, "remaining space : %i", (int)(blockStream.endPtr - blockStream.ptr)); - } } - - DEBUGLOG(6, "ZSTD_encodeSequences: flushing ML state with %u bits", stateMatchLength.stateLog); - FSE_flushCState(&blockStream, &stateMatchLength); - DEBUGLOG(6, "ZSTD_encodeSequences: flushing Off state with %u bits", stateOffsetBits.stateLog); - FSE_flushCState(&blockStream, &stateOffsetBits); - DEBUGLOG(6, "ZSTD_encodeSequences: flushing LL state with %u bits", stateLitLength.stateLog); - FSE_flushCState(&blockStream, &stateLitLength); - - { size_t const streamSize = BIT_closeCStream(&blockStream); - RETURN_ERROR_IF(streamSize==0, dstSize_tooSmall, "not enough space"); - return streamSize; - } -} - -static size_t -ZSTD_encodeSequences_default( - void* dst, size_t dstCapacity, - FSE_CTable const* CTable_MatchLength, BYTE const* mlCodeTable, - FSE_CTable const* CTable_OffsetBits, BYTE const* ofCodeTable, - FSE_CTable const* CTable_LitLength, BYTE const* llCodeTable, - seqDef const* sequences, size_t nbSeq, int longOffsets) -{ - return ZSTD_encodeSequences_body(dst, dstCapacity, - CTable_MatchLength, mlCodeTable, - CTable_OffsetBits, ofCodeTable, - CTable_LitLength, llCodeTable, - sequences, nbSeq, longOffsets); -} - - -#if DYNAMIC_BMI2 - -static TARGET_ATTRIBUTE("bmi2") size_t -ZSTD_encodeSequences_bmi2( - void* dst, size_t dstCapacity, - FSE_CTable const* CTable_MatchLength, BYTE const* mlCodeTable, - FSE_CTable const* CTable_OffsetBits, BYTE const* ofCodeTable, - FSE_CTable const* CTable_LitLength, BYTE const* llCodeTable, - seqDef const* sequences, size_t nbSeq, int longOffsets) -{ - return ZSTD_encodeSequences_body(dst, dstCapacity, - CTable_MatchLength, mlCodeTable, - CTable_OffsetBits, ofCodeTable, - CTable_LitLength, llCodeTable, - sequences, nbSeq, longOffsets); -} - -#endif - -size_t ZSTD_encodeSequences( - void* dst, size_t dstCapacity, - FSE_CTable const* CTable_MatchLength, BYTE const* mlCodeTable, - FSE_CTable const* CTable_OffsetBits, BYTE const* ofCodeTable, - FSE_CTable const* CTable_LitLength, BYTE const* llCodeTable, - seqDef const* sequences, size_t nbSeq, int longOffsets, int bmi2) -{ - DEBUGLOG(5, "ZSTD_encodeSequences: dstCapacity = %u", (unsigned)dstCapacity); -#if DYNAMIC_BMI2 - if (bmi2) { - return ZSTD_encodeSequences_bmi2(dst, dstCapacity, - CTable_MatchLength, mlCodeTable, - CTable_OffsetBits, ofCodeTable, - CTable_LitLength, llCodeTable, - sequences, nbSeq, longOffsets); - } -#endif - (void)bmi2; - return ZSTD_encodeSequences_default(dst, dstCapacity, - CTable_MatchLength, mlCodeTable, - CTable_OffsetBits, ofCodeTable, - CTable_LitLength, llCodeTable, - sequences, nbSeq, longOffsets); -} -/**** ended inlining compress/zstd_compress_sequences.c ****/ -/**** start inlining compress/zstd_compress_superblock.c ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - - /*-************************************* - * Dependencies - ***************************************/ -/**** start inlining zstd_compress_superblock.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_COMPRESS_ADVANCED_H -#define ZSTD_COMPRESS_ADVANCED_H - -/*-************************************* -* Dependencies -***************************************/ - -/**** skipping file: ../zstd.h ****/ - -/*-************************************* -* Target Compressed Block Size -***************************************/ - -/* ZSTD_compressSuperBlock() : - * Used to compress a super block when targetCBlockSize is being used. - * The given block will be compressed into multiple sub blocks that are around targetCBlockSize. */ -size_t ZSTD_compressSuperBlock(ZSTD_CCtx* zc, - void* dst, size_t dstCapacity, - void const* src, size_t srcSize, - unsigned lastBlock); - -#endif /* ZSTD_COMPRESS_ADVANCED_H */ -/**** ended inlining zstd_compress_superblock.h ****/ - -/**** skipping file: ../common/zstd_internal.h ****/ -/**** skipping file: hist.h ****/ -/**** skipping file: zstd_compress_internal.h ****/ -/**** skipping file: zstd_compress_sequences.h ****/ -/**** skipping file: zstd_compress_literals.h ****/ - -/*-************************************* -* Superblock entropy buffer structs -***************************************/ -/** ZSTD_hufCTablesMetadata_t : - * Stores Literals Block Type for a super-block in hType, and - * huffman tree description in hufDesBuffer. - * hufDesSize refers to the size of huffman tree description in bytes. - * This metadata is populated in ZSTD_buildSuperBlockEntropy_literal() */ -typedef struct { - symbolEncodingType_e hType; - BYTE hufDesBuffer[500]; /* TODO give name to this value */ - size_t hufDesSize; -} ZSTD_hufCTablesMetadata_t; - -/** ZSTD_fseCTablesMetadata_t : - * Stores symbol compression modes for a super-block in {ll, ol, ml}Type, and - * fse tables in fseTablesBuffer. - * fseTablesSize refers to the size of fse tables in bytes. - * This metadata is populated in ZSTD_buildSuperBlockEntropy_sequences() */ -typedef struct { - symbolEncodingType_e llType; - symbolEncodingType_e ofType; - symbolEncodingType_e mlType; - BYTE fseTablesBuffer[500]; /* TODO give name to this value */ - size_t fseTablesSize; - size_t lastCountSize; /* This is to account for bug in 1.3.4. More detail in ZSTD_compressSubBlock_sequences() */ -} ZSTD_fseCTablesMetadata_t; - -typedef struct { - ZSTD_hufCTablesMetadata_t hufMetadata; - ZSTD_fseCTablesMetadata_t fseMetadata; -} ZSTD_entropyCTablesMetadata_t; - - -/** ZSTD_buildSuperBlockEntropy_literal() : - * Builds entropy for the super-block literals. - * Stores literals block type (raw, rle, compressed, repeat) and - * huffman description table to hufMetadata. - * @return : size of huffman description table or error code */ -static size_t ZSTD_buildSuperBlockEntropy_literal(void* const src, size_t srcSize, - const ZSTD_hufCTables_t* prevHuf, - ZSTD_hufCTables_t* nextHuf, - ZSTD_hufCTablesMetadata_t* hufMetadata, - const int disableLiteralsCompression, - void* workspace, size_t wkspSize) -{ - BYTE* const wkspStart = (BYTE*)workspace; - BYTE* const wkspEnd = wkspStart + wkspSize; - BYTE* const countWkspStart = wkspStart; - unsigned* const countWksp = (unsigned*)workspace; - const size_t countWkspSize = (HUF_SYMBOLVALUE_MAX + 1) * sizeof(unsigned); - BYTE* const nodeWksp = countWkspStart + countWkspSize; - const size_t nodeWkspSize = wkspEnd-nodeWksp; - unsigned maxSymbolValue = 255; - unsigned huffLog = HUF_TABLELOG_DEFAULT; - HUF_repeat repeat = prevHuf->repeatMode; - - DEBUGLOG(5, "ZSTD_buildSuperBlockEntropy_literal (srcSize=%zu)", srcSize); - - /* Prepare nextEntropy assuming reusing the existing table */ - memcpy(nextHuf, prevHuf, sizeof(*prevHuf)); - - if (disableLiteralsCompression) { - DEBUGLOG(5, "set_basic - disabled"); - hufMetadata->hType = set_basic; - return 0; - } - - /* small ? don't even attempt compression (speed opt) */ -# define COMPRESS_LITERALS_SIZE_MIN 63 - { size_t const minLitSize = (prevHuf->repeatMode == HUF_repeat_valid) ? 6 : COMPRESS_LITERALS_SIZE_MIN; - if (srcSize <= minLitSize) { - DEBUGLOG(5, "set_basic - too small"); - hufMetadata->hType = set_basic; - return 0; - } - } - - /* Scan input and build symbol stats */ - { size_t const largest = HIST_count_wksp (countWksp, &maxSymbolValue, (const BYTE*)src, srcSize, workspace, wkspSize); - FORWARD_IF_ERROR(largest, "HIST_count_wksp failed"); - if (largest == srcSize) { - DEBUGLOG(5, "set_rle"); - hufMetadata->hType = set_rle; - return 0; - } - if (largest <= (srcSize >> 7)+4) { - DEBUGLOG(5, "set_basic - no gain"); - hufMetadata->hType = set_basic; - return 0; - } - } - - /* Validate the previous Huffman table */ - if (repeat == HUF_repeat_check && !HUF_validateCTable((HUF_CElt const*)prevHuf->CTable, countWksp, maxSymbolValue)) { - repeat = HUF_repeat_none; - } - - /* Build Huffman Tree */ - memset(nextHuf->CTable, 0, sizeof(nextHuf->CTable)); - huffLog = HUF_optimalTableLog(huffLog, srcSize, maxSymbolValue); - { size_t const maxBits = HUF_buildCTable_wksp((HUF_CElt*)nextHuf->CTable, countWksp, - maxSymbolValue, huffLog, - nodeWksp, nodeWkspSize); - FORWARD_IF_ERROR(maxBits, "HUF_buildCTable_wksp"); - huffLog = (U32)maxBits; - { /* Build and write the CTable */ - size_t const newCSize = HUF_estimateCompressedSize( - (HUF_CElt*)nextHuf->CTable, countWksp, maxSymbolValue); - size_t const hSize = HUF_writeCTable( - hufMetadata->hufDesBuffer, sizeof(hufMetadata->hufDesBuffer), - (HUF_CElt*)nextHuf->CTable, maxSymbolValue, huffLog); - /* Check against repeating the previous CTable */ - if (repeat != HUF_repeat_none) { - size_t const oldCSize = HUF_estimateCompressedSize( - (HUF_CElt const*)prevHuf->CTable, countWksp, maxSymbolValue); - if (oldCSize < srcSize && (oldCSize <= hSize + newCSize || hSize + 12 >= srcSize)) { - DEBUGLOG(5, "set_repeat - smaller"); - memcpy(nextHuf, prevHuf, sizeof(*prevHuf)); - hufMetadata->hType = set_repeat; - return 0; - } - } - if (newCSize + hSize >= srcSize) { - DEBUGLOG(5, "set_basic - no gains"); - memcpy(nextHuf, prevHuf, sizeof(*prevHuf)); - hufMetadata->hType = set_basic; - return 0; - } - DEBUGLOG(5, "set_compressed (hSize=%u)", (U32)hSize); - hufMetadata->hType = set_compressed; - nextHuf->repeatMode = HUF_repeat_check; - return hSize; - } - } -} - -/** ZSTD_buildSuperBlockEntropy_sequences() : - * Builds entropy for the super-block sequences. - * Stores symbol compression modes and fse table to fseMetadata. - * @return : size of fse tables or error code */ -static size_t ZSTD_buildSuperBlockEntropy_sequences(seqStore_t* seqStorePtr, - const ZSTD_fseCTables_t* prevEntropy, - ZSTD_fseCTables_t* nextEntropy, - const ZSTD_CCtx_params* cctxParams, - ZSTD_fseCTablesMetadata_t* fseMetadata, - void* workspace, size_t wkspSize) -{ - BYTE* const wkspStart = (BYTE*)workspace; - BYTE* const wkspEnd = wkspStart + wkspSize; - BYTE* const countWkspStart = wkspStart; - unsigned* const countWksp = (unsigned*)workspace; - const size_t countWkspSize = (MaxSeq + 1) * sizeof(unsigned); - BYTE* const cTableWksp = countWkspStart + countWkspSize; - const size_t cTableWkspSize = wkspEnd-cTableWksp; - ZSTD_strategy const strategy = cctxParams->cParams.strategy; - FSE_CTable* CTable_LitLength = nextEntropy->litlengthCTable; - FSE_CTable* CTable_OffsetBits = nextEntropy->offcodeCTable; - FSE_CTable* CTable_MatchLength = nextEntropy->matchlengthCTable; - const BYTE* const ofCodeTable = seqStorePtr->ofCode; - const BYTE* const llCodeTable = seqStorePtr->llCode; - const BYTE* const mlCodeTable = seqStorePtr->mlCode; - size_t const nbSeq = seqStorePtr->sequences - seqStorePtr->sequencesStart; - BYTE* const ostart = fseMetadata->fseTablesBuffer; - BYTE* const oend = ostart + sizeof(fseMetadata->fseTablesBuffer); - BYTE* op = ostart; - - assert(cTableWkspSize >= (1 << MaxFSELog) * sizeof(FSE_FUNCTION_TYPE)); - DEBUGLOG(5, "ZSTD_buildSuperBlockEntropy_sequences (nbSeq=%zu)", nbSeq); - memset(workspace, 0, wkspSize); - - fseMetadata->lastCountSize = 0; - /* convert length/distances into codes */ - ZSTD_seqToCodes(seqStorePtr); - /* build CTable for Literal Lengths */ - { U32 LLtype; - unsigned max = MaxLL; - size_t const mostFrequent = HIST_countFast_wksp(countWksp, &max, llCodeTable, nbSeq, workspace, wkspSize); /* can't fail */ - DEBUGLOG(5, "Building LL table"); - nextEntropy->litlength_repeatMode = prevEntropy->litlength_repeatMode; - LLtype = ZSTD_selectEncodingType(&nextEntropy->litlength_repeatMode, - countWksp, max, mostFrequent, nbSeq, - LLFSELog, prevEntropy->litlengthCTable, - LL_defaultNorm, LL_defaultNormLog, - ZSTD_defaultAllowed, strategy); - assert(set_basic < set_compressed && set_rle < set_compressed); - assert(!(LLtype < set_compressed && nextEntropy->litlength_repeatMode != FSE_repeat_none)); /* We don't copy tables */ - { size_t const countSize = ZSTD_buildCTable(op, oend - op, CTable_LitLength, LLFSELog, (symbolEncodingType_e)LLtype, - countWksp, max, llCodeTable, nbSeq, LL_defaultNorm, LL_defaultNormLog, MaxLL, - prevEntropy->litlengthCTable, sizeof(prevEntropy->litlengthCTable), - cTableWksp, cTableWkspSize); - FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for LitLens failed"); - if (LLtype == set_compressed) - fseMetadata->lastCountSize = countSize; - op += countSize; - fseMetadata->llType = (symbolEncodingType_e) LLtype; - } } - /* build CTable for Offsets */ - { U32 Offtype; - unsigned max = MaxOff; - size_t const mostFrequent = HIST_countFast_wksp(countWksp, &max, ofCodeTable, nbSeq, workspace, wkspSize); /* can't fail */ - /* We can only use the basic table if max <= DefaultMaxOff, otherwise the offsets are too large */ - ZSTD_defaultPolicy_e const defaultPolicy = (max <= DefaultMaxOff) ? ZSTD_defaultAllowed : ZSTD_defaultDisallowed; - DEBUGLOG(5, "Building OF table"); - nextEntropy->offcode_repeatMode = prevEntropy->offcode_repeatMode; - Offtype = ZSTD_selectEncodingType(&nextEntropy->offcode_repeatMode, - countWksp, max, mostFrequent, nbSeq, - OffFSELog, prevEntropy->offcodeCTable, - OF_defaultNorm, OF_defaultNormLog, - defaultPolicy, strategy); - assert(!(Offtype < set_compressed && nextEntropy->offcode_repeatMode != FSE_repeat_none)); /* We don't copy tables */ - { size_t const countSize = ZSTD_buildCTable(op, oend - op, CTable_OffsetBits, OffFSELog, (symbolEncodingType_e)Offtype, - countWksp, max, ofCodeTable, nbSeq, OF_defaultNorm, OF_defaultNormLog, DefaultMaxOff, - prevEntropy->offcodeCTable, sizeof(prevEntropy->offcodeCTable), - cTableWksp, cTableWkspSize); - FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for Offsets failed"); - if (Offtype == set_compressed) - fseMetadata->lastCountSize = countSize; - op += countSize; - fseMetadata->ofType = (symbolEncodingType_e) Offtype; - } } - /* build CTable for MatchLengths */ - { U32 MLtype; - unsigned max = MaxML; - size_t const mostFrequent = HIST_countFast_wksp(countWksp, &max, mlCodeTable, nbSeq, workspace, wkspSize); /* can't fail */ - DEBUGLOG(5, "Building ML table (remaining space : %i)", (int)(oend-op)); - nextEntropy->matchlength_repeatMode = prevEntropy->matchlength_repeatMode; - MLtype = ZSTD_selectEncodingType(&nextEntropy->matchlength_repeatMode, - countWksp, max, mostFrequent, nbSeq, - MLFSELog, prevEntropy->matchlengthCTable, - ML_defaultNorm, ML_defaultNormLog, - ZSTD_defaultAllowed, strategy); - assert(!(MLtype < set_compressed && nextEntropy->matchlength_repeatMode != FSE_repeat_none)); /* We don't copy tables */ - { size_t const countSize = ZSTD_buildCTable(op, oend - op, CTable_MatchLength, MLFSELog, (symbolEncodingType_e)MLtype, - countWksp, max, mlCodeTable, nbSeq, ML_defaultNorm, ML_defaultNormLog, MaxML, - prevEntropy->matchlengthCTable, sizeof(prevEntropy->matchlengthCTable), - cTableWksp, cTableWkspSize); - FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for MatchLengths failed"); - if (MLtype == set_compressed) - fseMetadata->lastCountSize = countSize; - op += countSize; - fseMetadata->mlType = (symbolEncodingType_e) MLtype; - } } - assert((size_t) (op-ostart) <= sizeof(fseMetadata->fseTablesBuffer)); - return op-ostart; -} - - -/** ZSTD_buildSuperBlockEntropy() : - * Builds entropy for the super-block. - * @return : 0 on success or error code */ -static size_t -ZSTD_buildSuperBlockEntropy(seqStore_t* seqStorePtr, - const ZSTD_entropyCTables_t* prevEntropy, - ZSTD_entropyCTables_t* nextEntropy, - const ZSTD_CCtx_params* cctxParams, - ZSTD_entropyCTablesMetadata_t* entropyMetadata, - void* workspace, size_t wkspSize) -{ - size_t const litSize = seqStorePtr->lit - seqStorePtr->litStart; - DEBUGLOG(5, "ZSTD_buildSuperBlockEntropy"); - entropyMetadata->hufMetadata.hufDesSize = - ZSTD_buildSuperBlockEntropy_literal(seqStorePtr->litStart, litSize, - &prevEntropy->huf, &nextEntropy->huf, - &entropyMetadata->hufMetadata, - ZSTD_disableLiteralsCompression(cctxParams), - workspace, wkspSize); - FORWARD_IF_ERROR(entropyMetadata->hufMetadata.hufDesSize, "ZSTD_buildSuperBlockEntropy_literal failed"); - entropyMetadata->fseMetadata.fseTablesSize = - ZSTD_buildSuperBlockEntropy_sequences(seqStorePtr, - &prevEntropy->fse, &nextEntropy->fse, - cctxParams, - &entropyMetadata->fseMetadata, - workspace, wkspSize); - FORWARD_IF_ERROR(entropyMetadata->fseMetadata.fseTablesSize, "ZSTD_buildSuperBlockEntropy_sequences failed"); - return 0; -} - -/** ZSTD_compressSubBlock_literal() : - * Compresses literals section for a sub-block. - * When we have to write the Huffman table we will sometimes choose a header - * size larger than necessary. This is because we have to pick the header size - * before we know the table size + compressed size, so we have a bound on the - * table size. If we guessed incorrectly, we fall back to uncompressed literals. - * - * We write the header when writeEntropy=1 and set entropyWrriten=1 when we succeeded - * in writing the header, otherwise it is set to 0. - * - * hufMetadata->hType has literals block type info. - * If it is set_basic, all sub-blocks literals section will be Raw_Literals_Block. - * If it is set_rle, all sub-blocks literals section will be RLE_Literals_Block. - * If it is set_compressed, first sub-block's literals section will be Compressed_Literals_Block - * If it is set_compressed, first sub-block's literals section will be Treeless_Literals_Block - * and the following sub-blocks' literals sections will be Treeless_Literals_Block. - * @return : compressed size of literals section of a sub-block - * Or 0 if it unable to compress. - * Or error code */ -static size_t ZSTD_compressSubBlock_literal(const HUF_CElt* hufTable, - const ZSTD_hufCTablesMetadata_t* hufMetadata, - const BYTE* literals, size_t litSize, - void* dst, size_t dstSize, - const int bmi2, int writeEntropy, int* entropyWritten) -{ - size_t const header = writeEntropy ? 200 : 0; - size_t const lhSize = 3 + (litSize >= (1 KB - header)) + (litSize >= (16 KB - header)); - BYTE* const ostart = (BYTE*)dst; - BYTE* const oend = ostart + dstSize; - BYTE* op = ostart + lhSize; - U32 const singleStream = lhSize == 3; - symbolEncodingType_e hType = writeEntropy ? hufMetadata->hType : set_repeat; - size_t cLitSize = 0; - - (void)bmi2; /* TODO bmi2... */ - - DEBUGLOG(5, "ZSTD_compressSubBlock_literal (litSize=%zu, lhSize=%zu, writeEntropy=%d)", litSize, lhSize, writeEntropy); - - *entropyWritten = 0; - if (litSize == 0 || hufMetadata->hType == set_basic) { - DEBUGLOG(5, "ZSTD_compressSubBlock_literal using raw literal"); - return ZSTD_noCompressLiterals(dst, dstSize, literals, litSize); - } else if (hufMetadata->hType == set_rle) { - DEBUGLOG(5, "ZSTD_compressSubBlock_literal using rle literal"); - return ZSTD_compressRleLiteralsBlock(dst, dstSize, literals, litSize); - } - - assert(litSize > 0); - assert(hufMetadata->hType == set_compressed || hufMetadata->hType == set_repeat); - - if (writeEntropy && hufMetadata->hType == set_compressed) { - memcpy(op, hufMetadata->hufDesBuffer, hufMetadata->hufDesSize); - op += hufMetadata->hufDesSize; - cLitSize += hufMetadata->hufDesSize; - DEBUGLOG(5, "ZSTD_compressSubBlock_literal (hSize=%zu)", hufMetadata->hufDesSize); - } - - /* TODO bmi2 */ - { const size_t cSize = singleStream ? HUF_compress1X_usingCTable(op, oend-op, literals, litSize, hufTable) - : HUF_compress4X_usingCTable(op, oend-op, literals, litSize, hufTable); - op += cSize; - cLitSize += cSize; - if (cSize == 0 || ERR_isError(cSize)) { - DEBUGLOG(5, "Failed to write entropy tables %s", ZSTD_getErrorName(cSize)); - return 0; - } - /* If we expand and we aren't writing a header then emit uncompressed */ - if (!writeEntropy && cLitSize >= litSize) { - DEBUGLOG(5, "ZSTD_compressSubBlock_literal using raw literal because uncompressible"); - return ZSTD_noCompressLiterals(dst, dstSize, literals, litSize); - } - /* If we are writing headers then allow expansion that doesn't change our header size. */ - if (lhSize < (size_t)(3 + (cLitSize >= 1 KB) + (cLitSize >= 16 KB))) { - assert(cLitSize > litSize); - DEBUGLOG(5, "Literals expanded beyond allowed header size"); - return ZSTD_noCompressLiterals(dst, dstSize, literals, litSize); - } - DEBUGLOG(5, "ZSTD_compressSubBlock_literal (cSize=%zu)", cSize); - } - - /* Build header */ - switch(lhSize) - { - case 3: /* 2 - 2 - 10 - 10 */ - { U32 const lhc = hType + ((!singleStream) << 2) + ((U32)litSize<<4) + ((U32)cLitSize<<14); - MEM_writeLE24(ostart, lhc); - break; - } - case 4: /* 2 - 2 - 14 - 14 */ - { U32 const lhc = hType + (2 << 2) + ((U32)litSize<<4) + ((U32)cLitSize<<18); - MEM_writeLE32(ostart, lhc); - break; - } - case 5: /* 2 - 2 - 18 - 18 */ - { U32 const lhc = hType + (3 << 2) + ((U32)litSize<<4) + ((U32)cLitSize<<22); - MEM_writeLE32(ostart, lhc); - ostart[4] = (BYTE)(cLitSize >> 10); - break; - } - default: /* not possible : lhSize is {3,4,5} */ - assert(0); - } - *entropyWritten = 1; - DEBUGLOG(5, "Compressed literals: %u -> %u", (U32)litSize, (U32)(op-ostart)); - return op-ostart; -} - -static size_t ZSTD_seqDecompressedSize(seqStore_t const* seqStore, const seqDef* sequences, size_t nbSeq, size_t litSize, int lastSequence) { - const seqDef* const sstart = sequences; - const seqDef* const send = sequences + nbSeq; - const seqDef* sp = sstart; - size_t matchLengthSum = 0; - size_t litLengthSum = 0; - while (send-sp > 0) { - ZSTD_sequenceLength const seqLen = ZSTD_getSequenceLength(seqStore, sp); - litLengthSum += seqLen.litLength; - matchLengthSum += seqLen.matchLength; - sp++; - } - assert(litLengthSum <= litSize); - if (!lastSequence) { - assert(litLengthSum == litSize); - } - return matchLengthSum + litSize; -} - -/** ZSTD_compressSubBlock_sequences() : - * Compresses sequences section for a sub-block. - * fseMetadata->llType, fseMetadata->ofType, and fseMetadata->mlType have - * symbol compression modes for the super-block. - * The first successfully compressed block will have these in its header. - * We set entropyWritten=1 when we succeed in compressing the sequences. - * The following sub-blocks will always have repeat mode. - * @return : compressed size of sequences section of a sub-block - * Or 0 if it is unable to compress - * Or error code. */ -static size_t ZSTD_compressSubBlock_sequences(const ZSTD_fseCTables_t* fseTables, - const ZSTD_fseCTablesMetadata_t* fseMetadata, - const seqDef* sequences, size_t nbSeq, - const BYTE* llCode, const BYTE* mlCode, const BYTE* ofCode, - const ZSTD_CCtx_params* cctxParams, - void* dst, size_t dstCapacity, - const int bmi2, int writeEntropy, int* entropyWritten) -{ - const int longOffsets = cctxParams->cParams.windowLog > STREAM_ACCUMULATOR_MIN; - BYTE* const ostart = (BYTE*)dst; - BYTE* const oend = ostart + dstCapacity; - BYTE* op = ostart; - BYTE* seqHead; - - DEBUGLOG(5, "ZSTD_compressSubBlock_sequences (nbSeq=%zu, writeEntropy=%d, longOffsets=%d)", nbSeq, writeEntropy, longOffsets); - - *entropyWritten = 0; - /* Sequences Header */ - RETURN_ERROR_IF((oend-op) < 3 /*max nbSeq Size*/ + 1 /*seqHead*/, - dstSize_tooSmall, ""); - if (nbSeq < 0x7F) - *op++ = (BYTE)nbSeq; - else if (nbSeq < LONGNBSEQ) - op[0] = (BYTE)((nbSeq>>8) + 0x80), op[1] = (BYTE)nbSeq, op+=2; - else - op[0]=0xFF, MEM_writeLE16(op+1, (U16)(nbSeq - LONGNBSEQ)), op+=3; - if (nbSeq==0) { - return op - ostart; - } - - /* seqHead : flags for FSE encoding type */ - seqHead = op++; - - DEBUGLOG(5, "ZSTD_compressSubBlock_sequences (seqHeadSize=%u)", (unsigned)(op-ostart)); - - if (writeEntropy) { - const U32 LLtype = fseMetadata->llType; - const U32 Offtype = fseMetadata->ofType; - const U32 MLtype = fseMetadata->mlType; - DEBUGLOG(5, "ZSTD_compressSubBlock_sequences (fseTablesSize=%zu)", fseMetadata->fseTablesSize); - *seqHead = (BYTE)((LLtype<<6) + (Offtype<<4) + (MLtype<<2)); - memcpy(op, fseMetadata->fseTablesBuffer, fseMetadata->fseTablesSize); - op += fseMetadata->fseTablesSize; - } else { - const U32 repeat = set_repeat; - *seqHead = (BYTE)((repeat<<6) + (repeat<<4) + (repeat<<2)); - } - - { size_t const bitstreamSize = ZSTD_encodeSequences( - op, oend - op, - fseTables->matchlengthCTable, mlCode, - fseTables->offcodeCTable, ofCode, - fseTables->litlengthCTable, llCode, - sequences, nbSeq, - longOffsets, bmi2); - FORWARD_IF_ERROR(bitstreamSize, "ZSTD_encodeSequences failed"); - op += bitstreamSize; - /* zstd versions <= 1.3.4 mistakenly report corruption when - * FSE_readNCount() receives a buffer < 4 bytes. - * Fixed by https://github.com/facebook/zstd/pull/1146. - * This can happen when the last set_compressed table present is 2 - * bytes and the bitstream is only one byte. - * In this exceedingly rare case, we will simply emit an uncompressed - * block, since it isn't worth optimizing. - */ -#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION - if (writeEntropy && fseMetadata->lastCountSize && fseMetadata->lastCountSize + bitstreamSize < 4) { - /* NCountSize >= 2 && bitstreamSize > 0 ==> lastCountSize == 3 */ - assert(fseMetadata->lastCountSize + bitstreamSize == 3); - DEBUGLOG(5, "Avoiding bug in zstd decoder in versions <= 1.3.4 by " - "emitting an uncompressed block."); - return 0; - } -#endif - DEBUGLOG(5, "ZSTD_compressSubBlock_sequences (bitstreamSize=%zu)", bitstreamSize); - } - - /* zstd versions <= 1.4.0 mistakenly report error when - * sequences section body size is less than 3 bytes. - * Fixed by https://github.com/facebook/zstd/pull/1664. - * This can happen when the previous sequences section block is compressed - * with rle mode and the current block's sequences section is compressed - * with repeat mode where sequences section body size can be 1 byte. - */ -#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION - if (op-seqHead < 4) { - DEBUGLOG(5, "Avoiding bug in zstd decoder in versions <= 1.4.0 by emitting " - "an uncompressed block when sequences are < 4 bytes"); - return 0; - } -#endif - - *entropyWritten = 1; - return op - ostart; -} - -/** ZSTD_compressSubBlock() : - * Compresses a single sub-block. - * @return : compressed size of the sub-block - * Or 0 if it failed to compress. */ -static size_t ZSTD_compressSubBlock(const ZSTD_entropyCTables_t* entropy, - const ZSTD_entropyCTablesMetadata_t* entropyMetadata, - const seqDef* sequences, size_t nbSeq, - const BYTE* literals, size_t litSize, - const BYTE* llCode, const BYTE* mlCode, const BYTE* ofCode, - const ZSTD_CCtx_params* cctxParams, - void* dst, size_t dstCapacity, - const int bmi2, - int writeLitEntropy, int writeSeqEntropy, - int* litEntropyWritten, int* seqEntropyWritten, - U32 lastBlock) -{ - BYTE* const ostart = (BYTE*)dst; - BYTE* const oend = ostart + dstCapacity; - BYTE* op = ostart + ZSTD_blockHeaderSize; - DEBUGLOG(5, "ZSTD_compressSubBlock (litSize=%zu, nbSeq=%zu, writeLitEntropy=%d, writeSeqEntropy=%d, lastBlock=%d)", - litSize, nbSeq, writeLitEntropy, writeSeqEntropy, lastBlock); - { size_t cLitSize = ZSTD_compressSubBlock_literal((const HUF_CElt*)entropy->huf.CTable, - &entropyMetadata->hufMetadata, literals, litSize, - op, oend-op, bmi2, writeLitEntropy, litEntropyWritten); - FORWARD_IF_ERROR(cLitSize, "ZSTD_compressSubBlock_literal failed"); - if (cLitSize == 0) return 0; - op += cLitSize; - } - { size_t cSeqSize = ZSTD_compressSubBlock_sequences(&entropy->fse, - &entropyMetadata->fseMetadata, - sequences, nbSeq, - llCode, mlCode, ofCode, - cctxParams, - op, oend-op, - bmi2, writeSeqEntropy, seqEntropyWritten); - FORWARD_IF_ERROR(cSeqSize, "ZSTD_compressSubBlock_sequences failed"); - if (cSeqSize == 0) return 0; - op += cSeqSize; - } - /* Write block header */ - { size_t cSize = (op-ostart)-ZSTD_blockHeaderSize; - U32 const cBlockHeader24 = lastBlock + (((U32)bt_compressed)<<1) + (U32)(cSize << 3); - MEM_writeLE24(ostart, cBlockHeader24); - } - return op-ostart; -} - -static size_t ZSTD_estimateSubBlockSize_literal(const BYTE* literals, size_t litSize, - const ZSTD_hufCTables_t* huf, - const ZSTD_hufCTablesMetadata_t* hufMetadata, - void* workspace, size_t wkspSize, - int writeEntropy) -{ - unsigned* const countWksp = (unsigned*)workspace; - unsigned maxSymbolValue = 255; - size_t literalSectionHeaderSize = 3; /* Use hard coded size of 3 bytes */ - - if (hufMetadata->hType == set_basic) return litSize; - else if (hufMetadata->hType == set_rle) return 1; - else if (hufMetadata->hType == set_compressed || hufMetadata->hType == set_repeat) { - size_t const largest = HIST_count_wksp (countWksp, &maxSymbolValue, (const BYTE*)literals, litSize, workspace, wkspSize); - if (ZSTD_isError(largest)) return litSize; - { size_t cLitSizeEstimate = HUF_estimateCompressedSize((const HUF_CElt*)huf->CTable, countWksp, maxSymbolValue); - if (writeEntropy) cLitSizeEstimate += hufMetadata->hufDesSize; - return cLitSizeEstimate + literalSectionHeaderSize; - } } - assert(0); /* impossible */ - return 0; -} - -static size_t ZSTD_estimateSubBlockSize_symbolType(symbolEncodingType_e type, - const BYTE* codeTable, unsigned maxCode, - size_t nbSeq, const FSE_CTable* fseCTable, - const U32* additionalBits, - short const* defaultNorm, U32 defaultNormLog, - void* workspace, size_t wkspSize) -{ - unsigned* const countWksp = (unsigned*)workspace; - const BYTE* ctp = codeTable; - const BYTE* const ctStart = ctp; - const BYTE* const ctEnd = ctStart + nbSeq; - size_t cSymbolTypeSizeEstimateInBits = 0; - unsigned max = maxCode; - - HIST_countFast_wksp(countWksp, &max, codeTable, nbSeq, workspace, wkspSize); /* can't fail */ - if (type == set_basic) { - cSymbolTypeSizeEstimateInBits = ZSTD_crossEntropyCost(defaultNorm, defaultNormLog, countWksp, max); - } else if (type == set_rle) { - cSymbolTypeSizeEstimateInBits = 0; - } else if (type == set_compressed || type == set_repeat) { - cSymbolTypeSizeEstimateInBits = ZSTD_fseBitCost(fseCTable, countWksp, max); - } - if (ZSTD_isError(cSymbolTypeSizeEstimateInBits)) return nbSeq * 10; - while (ctp < ctEnd) { - if (additionalBits) cSymbolTypeSizeEstimateInBits += additionalBits[*ctp]; - else cSymbolTypeSizeEstimateInBits += *ctp; /* for offset, offset code is also the number of additional bits */ - ctp++; - } - return cSymbolTypeSizeEstimateInBits / 8; -} - -static size_t ZSTD_estimateSubBlockSize_sequences(const BYTE* ofCodeTable, - const BYTE* llCodeTable, - const BYTE* mlCodeTable, - size_t nbSeq, - const ZSTD_fseCTables_t* fseTables, - const ZSTD_fseCTablesMetadata_t* fseMetadata, - void* workspace, size_t wkspSize, - int writeEntropy) -{ - size_t sequencesSectionHeaderSize = 3; /* Use hard coded size of 3 bytes */ - size_t cSeqSizeEstimate = 0; - cSeqSizeEstimate += ZSTD_estimateSubBlockSize_symbolType(fseMetadata->ofType, ofCodeTable, MaxOff, - nbSeq, fseTables->offcodeCTable, NULL, - OF_defaultNorm, OF_defaultNormLog, - workspace, wkspSize); - cSeqSizeEstimate += ZSTD_estimateSubBlockSize_symbolType(fseMetadata->llType, llCodeTable, MaxLL, - nbSeq, fseTables->litlengthCTable, LL_bits, - LL_defaultNorm, LL_defaultNormLog, - workspace, wkspSize); - cSeqSizeEstimate += ZSTD_estimateSubBlockSize_symbolType(fseMetadata->mlType, mlCodeTable, MaxML, - nbSeq, fseTables->matchlengthCTable, ML_bits, - ML_defaultNorm, ML_defaultNormLog, - workspace, wkspSize); - if (writeEntropy) cSeqSizeEstimate += fseMetadata->fseTablesSize; - return cSeqSizeEstimate + sequencesSectionHeaderSize; -} - -static size_t ZSTD_estimateSubBlockSize(const BYTE* literals, size_t litSize, - const BYTE* ofCodeTable, - const BYTE* llCodeTable, - const BYTE* mlCodeTable, - size_t nbSeq, - const ZSTD_entropyCTables_t* entropy, - const ZSTD_entropyCTablesMetadata_t* entropyMetadata, - void* workspace, size_t wkspSize, - int writeLitEntropy, int writeSeqEntropy) { - size_t cSizeEstimate = 0; - cSizeEstimate += ZSTD_estimateSubBlockSize_literal(literals, litSize, - &entropy->huf, &entropyMetadata->hufMetadata, - workspace, wkspSize, writeLitEntropy); - cSizeEstimate += ZSTD_estimateSubBlockSize_sequences(ofCodeTable, llCodeTable, mlCodeTable, - nbSeq, &entropy->fse, &entropyMetadata->fseMetadata, - workspace, wkspSize, writeSeqEntropy); - return cSizeEstimate + ZSTD_blockHeaderSize; -} - -static int ZSTD_needSequenceEntropyTables(ZSTD_fseCTablesMetadata_t const* fseMetadata) -{ - if (fseMetadata->llType == set_compressed || fseMetadata->llType == set_rle) - return 1; - if (fseMetadata->mlType == set_compressed || fseMetadata->mlType == set_rle) - return 1; - if (fseMetadata->ofType == set_compressed || fseMetadata->ofType == set_rle) - return 1; - return 0; -} - -/** ZSTD_compressSubBlock_multi() : - * Breaks super-block into multiple sub-blocks and compresses them. - * Entropy will be written to the first block. - * The following blocks will use repeat mode to compress. - * All sub-blocks are compressed blocks (no raw or rle blocks). - * @return : compressed size of the super block (which is multiple ZSTD blocks) - * Or 0 if it failed to compress. */ -static size_t ZSTD_compressSubBlock_multi(const seqStore_t* seqStorePtr, - const ZSTD_compressedBlockState_t* prevCBlock, - ZSTD_compressedBlockState_t* nextCBlock, - const ZSTD_entropyCTablesMetadata_t* entropyMetadata, - const ZSTD_CCtx_params* cctxParams, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const int bmi2, U32 lastBlock, - void* workspace, size_t wkspSize) -{ - const seqDef* const sstart = seqStorePtr->sequencesStart; - const seqDef* const send = seqStorePtr->sequences; - const seqDef* sp = sstart; - const BYTE* const lstart = seqStorePtr->litStart; - const BYTE* const lend = seqStorePtr->lit; - const BYTE* lp = lstart; - BYTE const* ip = (BYTE const*)src; - BYTE const* const iend = ip + srcSize; - BYTE* const ostart = (BYTE*)dst; - BYTE* const oend = ostart + dstCapacity; - BYTE* op = ostart; - const BYTE* llCodePtr = seqStorePtr->llCode; - const BYTE* mlCodePtr = seqStorePtr->mlCode; - const BYTE* ofCodePtr = seqStorePtr->ofCode; - size_t targetCBlockSize = cctxParams->targetCBlockSize; - size_t litSize, seqCount; - int writeLitEntropy = entropyMetadata->hufMetadata.hType == set_compressed; - int writeSeqEntropy = 1; - int lastSequence = 0; - - DEBUGLOG(5, "ZSTD_compressSubBlock_multi (litSize=%u, nbSeq=%u)", - (unsigned)(lend-lp), (unsigned)(send-sstart)); - - litSize = 0; - seqCount = 0; - do { - size_t cBlockSizeEstimate = 0; - if (sstart == send) { - lastSequence = 1; - } else { - const seqDef* const sequence = sp + seqCount; - lastSequence = sequence == send - 1; - litSize += ZSTD_getSequenceLength(seqStorePtr, sequence).litLength; - seqCount++; - } - if (lastSequence) { - assert(lp <= lend); - assert(litSize <= (size_t)(lend - lp)); - litSize = (size_t)(lend - lp); - } - /* I think there is an optimization opportunity here. - * Calling ZSTD_estimateSubBlockSize for every sequence can be wasteful - * since it recalculates estimate from scratch. - * For example, it would recount literal distribution and symbol codes everytime. - */ - cBlockSizeEstimate = ZSTD_estimateSubBlockSize(lp, litSize, ofCodePtr, llCodePtr, mlCodePtr, seqCount, - &nextCBlock->entropy, entropyMetadata, - workspace, wkspSize, writeLitEntropy, writeSeqEntropy); - if (cBlockSizeEstimate > targetCBlockSize || lastSequence) { - int litEntropyWritten = 0; - int seqEntropyWritten = 0; - const size_t decompressedSize = ZSTD_seqDecompressedSize(seqStorePtr, sp, seqCount, litSize, lastSequence); - const size_t cSize = ZSTD_compressSubBlock(&nextCBlock->entropy, entropyMetadata, - sp, seqCount, - lp, litSize, - llCodePtr, mlCodePtr, ofCodePtr, - cctxParams, - op, oend-op, - bmi2, writeLitEntropy, writeSeqEntropy, - &litEntropyWritten, &seqEntropyWritten, - lastBlock && lastSequence); - FORWARD_IF_ERROR(cSize, "ZSTD_compressSubBlock failed"); - if (cSize > 0 && cSize < decompressedSize) { - DEBUGLOG(5, "Committed the sub-block"); - assert(ip + decompressedSize <= iend); - ip += decompressedSize; - sp += seqCount; - lp += litSize; - op += cSize; - llCodePtr += seqCount; - mlCodePtr += seqCount; - ofCodePtr += seqCount; - litSize = 0; - seqCount = 0; - /* Entropy only needs to be written once */ - if (litEntropyWritten) { - writeLitEntropy = 0; - } - if (seqEntropyWritten) { - writeSeqEntropy = 0; - } - } - } - } while (!lastSequence); - if (writeLitEntropy) { - DEBUGLOG(5, "ZSTD_compressSubBlock_multi has literal entropy tables unwritten"); - memcpy(&nextCBlock->entropy.huf, &prevCBlock->entropy.huf, sizeof(prevCBlock->entropy.huf)); - } - if (writeSeqEntropy && ZSTD_needSequenceEntropyTables(&entropyMetadata->fseMetadata)) { - /* If we haven't written our entropy tables, then we've violated our contract and - * must emit an uncompressed block. - */ - DEBUGLOG(5, "ZSTD_compressSubBlock_multi has sequence entropy tables unwritten"); - return 0; - } - if (ip < iend) { - size_t const cSize = ZSTD_noCompressBlock(op, oend - op, ip, iend - ip, lastBlock); - DEBUGLOG(5, "ZSTD_compressSubBlock_multi last sub-block uncompressed, %zu bytes", (size_t)(iend - ip)); - FORWARD_IF_ERROR(cSize, "ZSTD_noCompressBlock failed"); - assert(cSize != 0); - op += cSize; - /* We have to regenerate the repcodes because we've skipped some sequences */ - if (sp < send) { - seqDef const* seq; - repcodes_t rep; - memcpy(&rep, prevCBlock->rep, sizeof(rep)); - for (seq = sstart; seq < sp; ++seq) { - rep = ZSTD_updateRep(rep.rep, seq->offset - 1, ZSTD_getSequenceLength(seqStorePtr, seq).litLength == 0); - } - memcpy(nextCBlock->rep, &rep, sizeof(rep)); - } - } - DEBUGLOG(5, "ZSTD_compressSubBlock_multi compressed"); - return op-ostart; -} - -size_t ZSTD_compressSuperBlock(ZSTD_CCtx* zc, - void* dst, size_t dstCapacity, - void const* src, size_t srcSize, - unsigned lastBlock) { - ZSTD_entropyCTablesMetadata_t entropyMetadata; - - FORWARD_IF_ERROR(ZSTD_buildSuperBlockEntropy(&zc->seqStore, - &zc->blockState.prevCBlock->entropy, - &zc->blockState.nextCBlock->entropy, - &zc->appliedParams, - &entropyMetadata, - zc->entropyWorkspace, HUF_WORKSPACE_SIZE /* statically allocated in resetCCtx */), ""); - - return ZSTD_compressSubBlock_multi(&zc->seqStore, - zc->blockState.prevCBlock, - zc->blockState.nextCBlock, - &entropyMetadata, - &zc->appliedParams, - dst, dstCapacity, - src, srcSize, - zc->bmi2, lastBlock, - zc->entropyWorkspace, HUF_WORKSPACE_SIZE /* statically allocated in resetCCtx */); -} -/**** ended inlining compress/zstd_compress_superblock.c ****/ -/**** start inlining compress/zstd_compress.c ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -/*-************************************* -* Dependencies -***************************************/ -#include /* INT_MAX */ -#include /* memset */ -/**** start inlining ../common/cpu.h ****/ -/* - * Copyright (c) 2018-2020, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_COMMON_CPU_H -#define ZSTD_COMMON_CPU_H - -/** - * Implementation taken from folly/CpuId.h - * https://github.com/facebook/folly/blob/master/folly/CpuId.h - */ - -#include - -/**** skipping file: mem.h ****/ - -#ifdef _MSC_VER -#include -#endif - -typedef struct { - U32 f1c; - U32 f1d; - U32 f7b; - U32 f7c; -} ZSTD_cpuid_t; - -MEM_STATIC ZSTD_cpuid_t ZSTD_cpuid(void) { - U32 f1c = 0; - U32 f1d = 0; - U32 f7b = 0; - U32 f7c = 0; -#if defined(_MSC_VER) && (defined(_M_X64) || defined(_M_IX86)) - int reg[4]; - __cpuid((int*)reg, 0); - { - int const n = reg[0]; - if (n >= 1) { - __cpuid((int*)reg, 1); - f1c = (U32)reg[2]; - f1d = (U32)reg[3]; - } - if (n >= 7) { - __cpuidex((int*)reg, 7, 0); - f7b = (U32)reg[1]; - f7c = (U32)reg[2]; - } - } -#elif defined(__i386__) && defined(__PIC__) && !defined(__clang__) && defined(__GNUC__) - /* The following block like the normal cpuid branch below, but gcc - * reserves ebx for use of its pic register so we must specially - * handle the save and restore to avoid clobbering the register - */ - U32 n; - __asm__( - "pushl %%ebx\n\t" - "cpuid\n\t" - "popl %%ebx\n\t" - : "=a"(n) - : "a"(0) - : "ecx", "edx"); - if (n >= 1) { - U32 f1a; - __asm__( - "pushl %%ebx\n\t" - "cpuid\n\t" - "popl %%ebx\n\t" - : "=a"(f1a), "=c"(f1c), "=d"(f1d) - : "a"(1)); - } - if (n >= 7) { - __asm__( - "pushl %%ebx\n\t" - "cpuid\n\t" - "movl %%ebx, %%eax\n\t" - "popl %%ebx" - : "=a"(f7b), "=c"(f7c) - : "a"(7), "c"(0) - : "edx"); - } -#elif defined(__x86_64__) || defined(_M_X64) || defined(__i386__) - U32 n; - __asm__("cpuid" : "=a"(n) : "a"(0) : "ebx", "ecx", "edx"); - if (n >= 1) { - U32 f1a; - __asm__("cpuid" : "=a"(f1a), "=c"(f1c), "=d"(f1d) : "a"(1) : "ebx"); - } - if (n >= 7) { - U32 f7a; - __asm__("cpuid" - : "=a"(f7a), "=b"(f7b), "=c"(f7c) - : "a"(7), "c"(0) - : "edx"); - } -#endif - { - ZSTD_cpuid_t cpuid; - cpuid.f1c = f1c; - cpuid.f1d = f1d; - cpuid.f7b = f7b; - cpuid.f7c = f7c; - return cpuid; - } -} - -#define X(name, r, bit) \ - MEM_STATIC int ZSTD_cpuid_##name(ZSTD_cpuid_t const cpuid) { \ - return ((cpuid.r) & (1U << bit)) != 0; \ - } - -/* cpuid(1): Processor Info and Feature Bits. */ -#define C(name, bit) X(name, f1c, bit) - C(sse3, 0) - C(pclmuldq, 1) - C(dtes64, 2) - C(monitor, 3) - C(dscpl, 4) - C(vmx, 5) - C(smx, 6) - C(eist, 7) - C(tm2, 8) - C(ssse3, 9) - C(cnxtid, 10) - C(fma, 12) - C(cx16, 13) - C(xtpr, 14) - C(pdcm, 15) - C(pcid, 17) - C(dca, 18) - C(sse41, 19) - C(sse42, 20) - C(x2apic, 21) - C(movbe, 22) - C(popcnt, 23) - C(tscdeadline, 24) - C(aes, 25) - C(xsave, 26) - C(osxsave, 27) - C(avx, 28) - C(f16c, 29) - C(rdrand, 30) -#undef C -#define D(name, bit) X(name, f1d, bit) - D(fpu, 0) - D(vme, 1) - D(de, 2) - D(pse, 3) - D(tsc, 4) - D(msr, 5) - D(pae, 6) - D(mce, 7) - D(cx8, 8) - D(apic, 9) - D(sep, 11) - D(mtrr, 12) - D(pge, 13) - D(mca, 14) - D(cmov, 15) - D(pat, 16) - D(pse36, 17) - D(psn, 18) - D(clfsh, 19) - D(ds, 21) - D(acpi, 22) - D(mmx, 23) - D(fxsr, 24) - D(sse, 25) - D(sse2, 26) - D(ss, 27) - D(htt, 28) - D(tm, 29) - D(pbe, 31) -#undef D - -/* cpuid(7): Extended Features. */ -#define B(name, bit) X(name, f7b, bit) - B(bmi1, 3) - B(hle, 4) - B(avx2, 5) - B(smep, 7) - B(bmi2, 8) - B(erms, 9) - B(invpcid, 10) - B(rtm, 11) - B(mpx, 14) - B(avx512f, 16) - B(avx512dq, 17) - B(rdseed, 18) - B(adx, 19) - B(smap, 20) - B(avx512ifma, 21) - B(pcommit, 22) - B(clflushopt, 23) - B(clwb, 24) - B(avx512pf, 26) - B(avx512er, 27) - B(avx512cd, 28) - B(sha, 29) - B(avx512bw, 30) - B(avx512vl, 31) -#undef B -#define C(name, bit) X(name, f7c, bit) - C(prefetchwt1, 0) - C(avx512vbmi, 1) -#undef C - -#undef X - -#endif /* ZSTD_COMMON_CPU_H */ -/**** ended inlining ../common/cpu.h ****/ -/**** skipping file: ../common/mem.h ****/ -/**** skipping file: hist.h ****/ -#define FSE_STATIC_LINKING_ONLY /* FSE_encodeSymbol */ -/**** skipping file: ../common/fse.h ****/ -#define HUF_STATIC_LINKING_ONLY -/**** skipping file: ../common/huf.h ****/ -/**** skipping file: zstd_compress_internal.h ****/ -/**** skipping file: zstd_compress_sequences.h ****/ -/**** skipping file: zstd_compress_literals.h ****/ -/**** start inlining zstd_fast.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_FAST_H -#define ZSTD_FAST_H - -#if defined (__cplusplus) -extern "C" { -#endif - -/**** skipping file: ../common/mem.h ****/ -/**** skipping file: zstd_compress_internal.h ****/ - -void ZSTD_fillHashTable(ZSTD_matchState_t* ms, - void const* end, ZSTD_dictTableLoadMethod_e dtlm); -size_t ZSTD_compressBlock_fast( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_fast_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_fast_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTD_FAST_H */ -/**** ended inlining zstd_fast.h ****/ -/**** start inlining zstd_double_fast.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_DOUBLE_FAST_H -#define ZSTD_DOUBLE_FAST_H - -#if defined (__cplusplus) -extern "C" { -#endif - -/**** skipping file: ../common/mem.h ****/ -/**** skipping file: zstd_compress_internal.h ****/ - -void ZSTD_fillDoubleHashTable(ZSTD_matchState_t* ms, - void const* end, ZSTD_dictTableLoadMethod_e dtlm); -size_t ZSTD_compressBlock_doubleFast( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_doubleFast_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_doubleFast_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); - - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTD_DOUBLE_FAST_H */ -/**** ended inlining zstd_double_fast.h ****/ -/**** start inlining zstd_lazy.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_LAZY_H -#define ZSTD_LAZY_H - -#if defined (__cplusplus) -extern "C" { -#endif - -/**** skipping file: zstd_compress_internal.h ****/ - -U32 ZSTD_insertAndFindFirstIndex(ZSTD_matchState_t* ms, const BYTE* ip); - -void ZSTD_preserveUnsortedMark (U32* const table, U32 const size, U32 const reducerValue); /*! used in ZSTD_reduceIndex(). preemptively increase value of ZSTD_DUBT_UNSORTED_MARK */ - -size_t ZSTD_compressBlock_btlazy2( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_lazy2( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_lazy( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_greedy( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); - -size_t ZSTD_compressBlock_btlazy2_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_lazy2_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_lazy_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_greedy_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); - -size_t ZSTD_compressBlock_greedy_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_lazy_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_lazy2_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_btlazy2_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTD_LAZY_H */ -/**** ended inlining zstd_lazy.h ****/ -/**** start inlining zstd_opt.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_OPT_H -#define ZSTD_OPT_H - -#if defined (__cplusplus) -extern "C" { -#endif - -/**** skipping file: zstd_compress_internal.h ****/ - -/* used in ZSTD_loadDictionaryContent() */ -void ZSTD_updateTree(ZSTD_matchState_t* ms, const BYTE* ip, const BYTE* iend); - -size_t ZSTD_compressBlock_btopt( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_btultra( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_btultra2( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); - - -size_t ZSTD_compressBlock_btopt_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_btultra_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); - -size_t ZSTD_compressBlock_btopt_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); -size_t ZSTD_compressBlock_btultra_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); - - /* note : no btultra2 variant for extDict nor dictMatchState, - * because btultra2 is not meant to work with dictionaries - * and is only specific for the first block (no prefix) */ - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTD_OPT_H */ -/**** ended inlining zstd_opt.h ****/ -/**** start inlining zstd_ldm.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_LDM_H -#define ZSTD_LDM_H - -#if defined (__cplusplus) -extern "C" { -#endif - -/**** skipping file: zstd_compress_internal.h ****/ -/**** skipping file: ../zstd.h ****/ - -/*-************************************* -* Long distance matching -***************************************/ - -#define ZSTD_LDM_DEFAULT_WINDOW_LOG ZSTD_WINDOWLOG_LIMIT_DEFAULT - -void ZSTD_ldm_fillHashTable( - ldmState_t* state, const BYTE* ip, - const BYTE* iend, ldmParams_t const* params); - -/** - * ZSTD_ldm_generateSequences(): - * - * Generates the sequences using the long distance match finder. - * Generates long range matching sequences in `sequences`, which parse a prefix - * of the source. `sequences` must be large enough to store every sequence, - * which can be checked with `ZSTD_ldm_getMaxNbSeq()`. - * @returns 0 or an error code. - * - * NOTE: The user must have called ZSTD_window_update() for all of the input - * they have, even if they pass it to ZSTD_ldm_generateSequences() in chunks. - * NOTE: This function returns an error if it runs out of space to store - * sequences. - */ -size_t ZSTD_ldm_generateSequences( - ldmState_t* ldms, rawSeqStore_t* sequences, - ldmParams_t const* params, void const* src, size_t srcSize); - -/** - * ZSTD_ldm_blockCompress(): - * - * Compresses a block using the predefined sequences, along with a secondary - * block compressor. The literals section of every sequence is passed to the - * secondary block compressor, and those sequences are interspersed with the - * predefined sequences. Returns the length of the last literals. - * Updates `rawSeqStore.pos` to indicate how many sequences have been consumed. - * `rawSeqStore.seq` may also be updated to split the last sequence between two - * blocks. - * @return The length of the last literals. - * - * NOTE: The source must be at most the maximum block size, but the predefined - * sequences can be any size, and may be longer than the block. In the case that - * they are longer than the block, the last sequences may need to be split into - * two. We handle that case correctly, and update `rawSeqStore` appropriately. - * NOTE: This function does not return any errors. - */ -size_t ZSTD_ldm_blockCompress(rawSeqStore_t* rawSeqStore, - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize); - -/** - * ZSTD_ldm_skipSequences(): - * - * Skip past `srcSize` bytes worth of sequences in `rawSeqStore`. - * Avoids emitting matches less than `minMatch` bytes. - * Must be called for data with is not passed to ZSTD_ldm_blockCompress(). - */ -void ZSTD_ldm_skipSequences(rawSeqStore_t* rawSeqStore, size_t srcSize, - U32 const minMatch); - - -/** ZSTD_ldm_getTableSize() : - * Estimate the space needed for long distance matching tables or 0 if LDM is - * disabled. - */ -size_t ZSTD_ldm_getTableSize(ldmParams_t params); - -/** ZSTD_ldm_getSeqSpace() : - * Return an upper bound on the number of sequences that can be produced by - * the long distance matcher, or 0 if LDM is disabled. - */ -size_t ZSTD_ldm_getMaxNbSeq(ldmParams_t params, size_t maxChunkSize); - -/** ZSTD_ldm_adjustParameters() : - * If the params->hashRateLog is not set, set it to its default value based on - * windowLog and params->hashLog. - * - * Ensures that params->bucketSizeLog is <= params->hashLog (setting it to - * params->hashLog if it is not). - * - * Ensures that the minMatchLength >= targetLength during optimal parsing. - */ -void ZSTD_ldm_adjustParameters(ldmParams_t* params, - ZSTD_compressionParameters const* cParams); - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTD_FAST_H */ -/**** ended inlining zstd_ldm.h ****/ -/**** skipping file: zstd_compress_superblock.h ****/ - - -/*-************************************* -* Helper functions -***************************************/ -/* ZSTD_compressBound() - * Note that the result from this function is only compatible with the "normal" - * full-block strategy. - * When there are a lot of small blocks due to frequent flush in streaming mode - * the overhead of headers can make the compressed data to be larger than the - * return value of ZSTD_compressBound(). - */ -size_t ZSTD_compressBound(size_t srcSize) { - return ZSTD_COMPRESSBOUND(srcSize); -} - - -/*-************************************* -* Context memory management -***************************************/ -struct ZSTD_CDict_s { - const void* dictContent; - size_t dictContentSize; - U32* entropyWorkspace; /* entropy workspace of HUF_WORKSPACE_SIZE bytes */ - ZSTD_cwksp workspace; - ZSTD_matchState_t matchState; - ZSTD_compressedBlockState_t cBlockState; - ZSTD_customMem customMem; - U32 dictID; - int compressionLevel; /* 0 indicates that advanced API was used to select CDict params */ -}; /* typedef'd to ZSTD_CDict within "zstd.h" */ - -ZSTD_CCtx* ZSTD_createCCtx(void) -{ - return ZSTD_createCCtx_advanced(ZSTD_defaultCMem); -} - -static void ZSTD_initCCtx(ZSTD_CCtx* cctx, ZSTD_customMem memManager) -{ - assert(cctx != NULL); - memset(cctx, 0, sizeof(*cctx)); - cctx->customMem = memManager; - cctx->bmi2 = ZSTD_cpuid_bmi2(ZSTD_cpuid()); - { size_t const err = ZSTD_CCtx_reset(cctx, ZSTD_reset_parameters); - assert(!ZSTD_isError(err)); - (void)err; - } -} - -ZSTD_CCtx* ZSTD_createCCtx_advanced(ZSTD_customMem customMem) -{ - ZSTD_STATIC_ASSERT(zcss_init==0); - ZSTD_STATIC_ASSERT(ZSTD_CONTENTSIZE_UNKNOWN==(0ULL - 1)); - if (!customMem.customAlloc ^ !customMem.customFree) return NULL; - { ZSTD_CCtx* const cctx = (ZSTD_CCtx*)ZSTD_malloc(sizeof(ZSTD_CCtx), customMem); - if (!cctx) return NULL; - ZSTD_initCCtx(cctx, customMem); - return cctx; - } -} - -ZSTD_CCtx* ZSTD_initStaticCCtx(void* workspace, size_t workspaceSize) -{ - ZSTD_cwksp ws; - ZSTD_CCtx* cctx; - if (workspaceSize <= sizeof(ZSTD_CCtx)) return NULL; /* minimum size */ - if ((size_t)workspace & 7) return NULL; /* must be 8-aligned */ - ZSTD_cwksp_init(&ws, workspace, workspaceSize); - - cctx = (ZSTD_CCtx*)ZSTD_cwksp_reserve_object(&ws, sizeof(ZSTD_CCtx)); - if (cctx == NULL) return NULL; - - memset(cctx, 0, sizeof(ZSTD_CCtx)); - ZSTD_cwksp_move(&cctx->workspace, &ws); - cctx->staticSize = workspaceSize; - - /* statically sized space. entropyWorkspace never moves (but prev/next block swap places) */ - if (!ZSTD_cwksp_check_available(&cctx->workspace, HUF_WORKSPACE_SIZE + 2 * sizeof(ZSTD_compressedBlockState_t))) return NULL; - cctx->blockState.prevCBlock = (ZSTD_compressedBlockState_t*)ZSTD_cwksp_reserve_object(&cctx->workspace, sizeof(ZSTD_compressedBlockState_t)); - cctx->blockState.nextCBlock = (ZSTD_compressedBlockState_t*)ZSTD_cwksp_reserve_object(&cctx->workspace, sizeof(ZSTD_compressedBlockState_t)); - cctx->entropyWorkspace = (U32*)ZSTD_cwksp_reserve_object(&cctx->workspace, HUF_WORKSPACE_SIZE); - cctx->bmi2 = ZSTD_cpuid_bmi2(ZSTD_cpuid()); - return cctx; -} - -/** - * Clears and frees all of the dictionaries in the CCtx. - */ -static void ZSTD_clearAllDicts(ZSTD_CCtx* cctx) -{ - ZSTD_free(cctx->localDict.dictBuffer, cctx->customMem); - ZSTD_freeCDict(cctx->localDict.cdict); - memset(&cctx->localDict, 0, sizeof(cctx->localDict)); - memset(&cctx->prefixDict, 0, sizeof(cctx->prefixDict)); - cctx->cdict = NULL; -} - -static size_t ZSTD_sizeof_localDict(ZSTD_localDict dict) -{ - size_t const bufferSize = dict.dictBuffer != NULL ? dict.dictSize : 0; - size_t const cdictSize = ZSTD_sizeof_CDict(dict.cdict); - return bufferSize + cdictSize; -} - -static void ZSTD_freeCCtxContent(ZSTD_CCtx* cctx) -{ - assert(cctx != NULL); - assert(cctx->staticSize == 0); - ZSTD_clearAllDicts(cctx); -#ifdef ZSTD_MULTITHREAD - ZSTDMT_freeCCtx(cctx->mtctx); cctx->mtctx = NULL; -#endif - ZSTD_cwksp_free(&cctx->workspace, cctx->customMem); -} - -size_t ZSTD_freeCCtx(ZSTD_CCtx* cctx) -{ - if (cctx==NULL) return 0; /* support free on NULL */ - RETURN_ERROR_IF(cctx->staticSize, memory_allocation, - "not compatible with static CCtx"); - { - int cctxInWorkspace = ZSTD_cwksp_owns_buffer(&cctx->workspace, cctx); - ZSTD_freeCCtxContent(cctx); - if (!cctxInWorkspace) { - ZSTD_free(cctx, cctx->customMem); - } - } - return 0; -} - - -static size_t ZSTD_sizeof_mtctx(const ZSTD_CCtx* cctx) -{ -#ifdef ZSTD_MULTITHREAD - return ZSTDMT_sizeof_CCtx(cctx->mtctx); -#else - (void)cctx; - return 0; -#endif -} - - -size_t ZSTD_sizeof_CCtx(const ZSTD_CCtx* cctx) -{ - if (cctx==NULL) return 0; /* support sizeof on NULL */ - /* cctx may be in the workspace */ - return (cctx->workspace.workspace == cctx ? 0 : sizeof(*cctx)) - + ZSTD_cwksp_sizeof(&cctx->workspace) - + ZSTD_sizeof_localDict(cctx->localDict) - + ZSTD_sizeof_mtctx(cctx); -} - -size_t ZSTD_sizeof_CStream(const ZSTD_CStream* zcs) -{ - return ZSTD_sizeof_CCtx(zcs); /* same object */ -} - -/* private API call, for dictBuilder only */ -const seqStore_t* ZSTD_getSeqStore(const ZSTD_CCtx* ctx) { return &(ctx->seqStore); } - -static ZSTD_CCtx_params ZSTD_makeCCtxParamsFromCParams( - ZSTD_compressionParameters cParams) -{ - ZSTD_CCtx_params cctxParams; - memset(&cctxParams, 0, sizeof(cctxParams)); - cctxParams.cParams = cParams; - cctxParams.compressionLevel = ZSTD_CLEVEL_DEFAULT; /* should not matter, as all cParams are presumed properly defined */ - assert(!ZSTD_checkCParams(cParams)); - cctxParams.fParams.contentSizeFlag = 1; - return cctxParams; -} - -static ZSTD_CCtx_params* ZSTD_createCCtxParams_advanced( - ZSTD_customMem customMem) -{ - ZSTD_CCtx_params* params; - if (!customMem.customAlloc ^ !customMem.customFree) return NULL; - params = (ZSTD_CCtx_params*)ZSTD_calloc( - sizeof(ZSTD_CCtx_params), customMem); - if (!params) { return NULL; } - params->customMem = customMem; - params->compressionLevel = ZSTD_CLEVEL_DEFAULT; - params->fParams.contentSizeFlag = 1; - return params; -} - -ZSTD_CCtx_params* ZSTD_createCCtxParams(void) -{ - return ZSTD_createCCtxParams_advanced(ZSTD_defaultCMem); -} - -size_t ZSTD_freeCCtxParams(ZSTD_CCtx_params* params) -{ - if (params == NULL) { return 0; } - ZSTD_free(params, params->customMem); - return 0; -} - -size_t ZSTD_CCtxParams_reset(ZSTD_CCtx_params* params) -{ - return ZSTD_CCtxParams_init(params, ZSTD_CLEVEL_DEFAULT); -} - -size_t ZSTD_CCtxParams_init(ZSTD_CCtx_params* cctxParams, int compressionLevel) { - RETURN_ERROR_IF(!cctxParams, GENERIC, "NULL pointer!"); - memset(cctxParams, 0, sizeof(*cctxParams)); - cctxParams->compressionLevel = compressionLevel; - cctxParams->fParams.contentSizeFlag = 1; - return 0; -} - -size_t ZSTD_CCtxParams_init_advanced(ZSTD_CCtx_params* cctxParams, ZSTD_parameters params) -{ - RETURN_ERROR_IF(!cctxParams, GENERIC, "NULL pointer!"); - FORWARD_IF_ERROR( ZSTD_checkCParams(params.cParams) , ""); - memset(cctxParams, 0, sizeof(*cctxParams)); - assert(!ZSTD_checkCParams(params.cParams)); - cctxParams->cParams = params.cParams; - cctxParams->fParams = params.fParams; - cctxParams->compressionLevel = ZSTD_CLEVEL_DEFAULT; /* should not matter, as all cParams are presumed properly defined */ - return 0; -} - -/* ZSTD_assignParamsToCCtxParams() : - * params is presumed valid at this stage */ -static ZSTD_CCtx_params ZSTD_assignParamsToCCtxParams( - const ZSTD_CCtx_params* cctxParams, const ZSTD_parameters* params) -{ - ZSTD_CCtx_params ret = *cctxParams; - assert(!ZSTD_checkCParams(params->cParams)); - ret.cParams = params->cParams; - ret.fParams = params->fParams; - ret.compressionLevel = ZSTD_CLEVEL_DEFAULT; /* should not matter, as all cParams are presumed properly defined */ - return ret; -} - -ZSTD_bounds ZSTD_cParam_getBounds(ZSTD_cParameter param) -{ - ZSTD_bounds bounds = { 0, 0, 0 }; - - switch(param) - { - case ZSTD_c_compressionLevel: - bounds.lowerBound = ZSTD_minCLevel(); - bounds.upperBound = ZSTD_maxCLevel(); - return bounds; - - case ZSTD_c_windowLog: - bounds.lowerBound = ZSTD_WINDOWLOG_MIN; - bounds.upperBound = ZSTD_WINDOWLOG_MAX; - return bounds; - - case ZSTD_c_hashLog: - bounds.lowerBound = ZSTD_HASHLOG_MIN; - bounds.upperBound = ZSTD_HASHLOG_MAX; - return bounds; - - case ZSTD_c_chainLog: - bounds.lowerBound = ZSTD_CHAINLOG_MIN; - bounds.upperBound = ZSTD_CHAINLOG_MAX; - return bounds; - - case ZSTD_c_searchLog: - bounds.lowerBound = ZSTD_SEARCHLOG_MIN; - bounds.upperBound = ZSTD_SEARCHLOG_MAX; - return bounds; - - case ZSTD_c_minMatch: - bounds.lowerBound = ZSTD_MINMATCH_MIN; - bounds.upperBound = ZSTD_MINMATCH_MAX; - return bounds; - - case ZSTD_c_targetLength: - bounds.lowerBound = ZSTD_TARGETLENGTH_MIN; - bounds.upperBound = ZSTD_TARGETLENGTH_MAX; - return bounds; - - case ZSTD_c_strategy: - bounds.lowerBound = ZSTD_STRATEGY_MIN; - bounds.upperBound = ZSTD_STRATEGY_MAX; - return bounds; - - case ZSTD_c_contentSizeFlag: - bounds.lowerBound = 0; - bounds.upperBound = 1; - return bounds; - - case ZSTD_c_checksumFlag: - bounds.lowerBound = 0; - bounds.upperBound = 1; - return bounds; - - case ZSTD_c_dictIDFlag: - bounds.lowerBound = 0; - bounds.upperBound = 1; - return bounds; - - case ZSTD_c_nbWorkers: - bounds.lowerBound = 0; -#ifdef ZSTD_MULTITHREAD - bounds.upperBound = ZSTDMT_NBWORKERS_MAX; -#else - bounds.upperBound = 0; -#endif - return bounds; - - case ZSTD_c_jobSize: - bounds.lowerBound = 0; -#ifdef ZSTD_MULTITHREAD - bounds.upperBound = ZSTDMT_JOBSIZE_MAX; -#else - bounds.upperBound = 0; -#endif - return bounds; - - case ZSTD_c_overlapLog: -#ifdef ZSTD_MULTITHREAD - bounds.lowerBound = ZSTD_OVERLAPLOG_MIN; - bounds.upperBound = ZSTD_OVERLAPLOG_MAX; -#else - bounds.lowerBound = 0; - bounds.upperBound = 0; -#endif - return bounds; - - case ZSTD_c_enableLongDistanceMatching: - bounds.lowerBound = 0; - bounds.upperBound = 1; - return bounds; - - case ZSTD_c_ldmHashLog: - bounds.lowerBound = ZSTD_LDM_HASHLOG_MIN; - bounds.upperBound = ZSTD_LDM_HASHLOG_MAX; - return bounds; - - case ZSTD_c_ldmMinMatch: - bounds.lowerBound = ZSTD_LDM_MINMATCH_MIN; - bounds.upperBound = ZSTD_LDM_MINMATCH_MAX; - return bounds; - - case ZSTD_c_ldmBucketSizeLog: - bounds.lowerBound = ZSTD_LDM_BUCKETSIZELOG_MIN; - bounds.upperBound = ZSTD_LDM_BUCKETSIZELOG_MAX; - return bounds; - - case ZSTD_c_ldmHashRateLog: - bounds.lowerBound = ZSTD_LDM_HASHRATELOG_MIN; - bounds.upperBound = ZSTD_LDM_HASHRATELOG_MAX; - return bounds; - - /* experimental parameters */ - case ZSTD_c_rsyncable: - bounds.lowerBound = 0; - bounds.upperBound = 1; - return bounds; - - case ZSTD_c_forceMaxWindow : - bounds.lowerBound = 0; - bounds.upperBound = 1; - return bounds; - - case ZSTD_c_format: - ZSTD_STATIC_ASSERT(ZSTD_f_zstd1 < ZSTD_f_zstd1_magicless); - bounds.lowerBound = ZSTD_f_zstd1; - bounds.upperBound = ZSTD_f_zstd1_magicless; /* note : how to ensure at compile time that this is the highest value enum ? */ - return bounds; - - case ZSTD_c_forceAttachDict: - ZSTD_STATIC_ASSERT(ZSTD_dictDefaultAttach < ZSTD_dictForceCopy); - bounds.lowerBound = ZSTD_dictDefaultAttach; - bounds.upperBound = ZSTD_dictForceLoad; /* note : how to ensure at compile time that this is the highest value enum ? */ - return bounds; - - case ZSTD_c_literalCompressionMode: - ZSTD_STATIC_ASSERT(ZSTD_lcm_auto < ZSTD_lcm_huffman && ZSTD_lcm_huffman < ZSTD_lcm_uncompressed); - bounds.lowerBound = ZSTD_lcm_auto; - bounds.upperBound = ZSTD_lcm_uncompressed; - return bounds; - - case ZSTD_c_targetCBlockSize: - bounds.lowerBound = ZSTD_TARGETCBLOCKSIZE_MIN; - bounds.upperBound = ZSTD_TARGETCBLOCKSIZE_MAX; - return bounds; - - case ZSTD_c_srcSizeHint: - bounds.lowerBound = ZSTD_SRCSIZEHINT_MIN; - bounds.upperBound = ZSTD_SRCSIZEHINT_MAX; - return bounds; - - default: - bounds.error = ERROR(parameter_unsupported); - return bounds; - } -} - -/* ZSTD_cParam_clampBounds: - * Clamps the value into the bounded range. - */ -static size_t ZSTD_cParam_clampBounds(ZSTD_cParameter cParam, int* value) -{ - ZSTD_bounds const bounds = ZSTD_cParam_getBounds(cParam); - if (ZSTD_isError(bounds.error)) return bounds.error; - if (*value < bounds.lowerBound) *value = bounds.lowerBound; - if (*value > bounds.upperBound) *value = bounds.upperBound; - return 0; -} - -#define BOUNDCHECK(cParam, val) { \ - RETURN_ERROR_IF(!ZSTD_cParam_withinBounds(cParam,val), \ - parameter_outOfBound, "Param out of bounds"); \ -} - - -static int ZSTD_isUpdateAuthorized(ZSTD_cParameter param) -{ - switch(param) - { - case ZSTD_c_compressionLevel: - case ZSTD_c_hashLog: - case ZSTD_c_chainLog: - case ZSTD_c_searchLog: - case ZSTD_c_minMatch: - case ZSTD_c_targetLength: - case ZSTD_c_strategy: - return 1; - - case ZSTD_c_format: - case ZSTD_c_windowLog: - case ZSTD_c_contentSizeFlag: - case ZSTD_c_checksumFlag: - case ZSTD_c_dictIDFlag: - case ZSTD_c_forceMaxWindow : - case ZSTD_c_nbWorkers: - case ZSTD_c_jobSize: - case ZSTD_c_overlapLog: - case ZSTD_c_rsyncable: - case ZSTD_c_enableLongDistanceMatching: - case ZSTD_c_ldmHashLog: - case ZSTD_c_ldmMinMatch: - case ZSTD_c_ldmBucketSizeLog: - case ZSTD_c_ldmHashRateLog: - case ZSTD_c_forceAttachDict: - case ZSTD_c_literalCompressionMode: - case ZSTD_c_targetCBlockSize: - case ZSTD_c_srcSizeHint: - default: - return 0; - } -} - -size_t ZSTD_CCtx_setParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param, int value) -{ - DEBUGLOG(4, "ZSTD_CCtx_setParameter (%i, %i)", (int)param, value); - if (cctx->streamStage != zcss_init) { - if (ZSTD_isUpdateAuthorized(param)) { - cctx->cParamsChanged = 1; - } else { - RETURN_ERROR(stage_wrong, "can only set params in ctx init stage"); - } } - - switch(param) - { - case ZSTD_c_nbWorkers: - RETURN_ERROR_IF((value!=0) && cctx->staticSize, parameter_unsupported, - "MT not compatible with static alloc"); - break; - - case ZSTD_c_compressionLevel: - case ZSTD_c_windowLog: - case ZSTD_c_hashLog: - case ZSTD_c_chainLog: - case ZSTD_c_searchLog: - case ZSTD_c_minMatch: - case ZSTD_c_targetLength: - case ZSTD_c_strategy: - case ZSTD_c_ldmHashRateLog: - case ZSTD_c_format: - case ZSTD_c_contentSizeFlag: - case ZSTD_c_checksumFlag: - case ZSTD_c_dictIDFlag: - case ZSTD_c_forceMaxWindow: - case ZSTD_c_forceAttachDict: - case ZSTD_c_literalCompressionMode: - case ZSTD_c_jobSize: - case ZSTD_c_overlapLog: - case ZSTD_c_rsyncable: - case ZSTD_c_enableLongDistanceMatching: - case ZSTD_c_ldmHashLog: - case ZSTD_c_ldmMinMatch: - case ZSTD_c_ldmBucketSizeLog: - case ZSTD_c_targetCBlockSize: - case ZSTD_c_srcSizeHint: - break; - - default: RETURN_ERROR(parameter_unsupported, "unknown parameter"); - } - return ZSTD_CCtxParams_setParameter(&cctx->requestedParams, param, value); -} - -size_t ZSTD_CCtxParams_setParameter(ZSTD_CCtx_params* CCtxParams, - ZSTD_cParameter param, int value) -{ - DEBUGLOG(4, "ZSTD_CCtxParams_setParameter (%i, %i)", (int)param, value); - switch(param) - { - case ZSTD_c_format : - BOUNDCHECK(ZSTD_c_format, value); - CCtxParams->format = (ZSTD_format_e)value; - return (size_t)CCtxParams->format; - - case ZSTD_c_compressionLevel : { - FORWARD_IF_ERROR(ZSTD_cParam_clampBounds(param, &value), ""); - if (value) { /* 0 : does not change current level */ - CCtxParams->compressionLevel = value; - } - if (CCtxParams->compressionLevel >= 0) return (size_t)CCtxParams->compressionLevel; - return 0; /* return type (size_t) cannot represent negative values */ - } - - case ZSTD_c_windowLog : - if (value!=0) /* 0 => use default */ - BOUNDCHECK(ZSTD_c_windowLog, value); - CCtxParams->cParams.windowLog = (U32)value; - return CCtxParams->cParams.windowLog; - - case ZSTD_c_hashLog : - if (value!=0) /* 0 => use default */ - BOUNDCHECK(ZSTD_c_hashLog, value); - CCtxParams->cParams.hashLog = (U32)value; - return CCtxParams->cParams.hashLog; - - case ZSTD_c_chainLog : - if (value!=0) /* 0 => use default */ - BOUNDCHECK(ZSTD_c_chainLog, value); - CCtxParams->cParams.chainLog = (U32)value; - return CCtxParams->cParams.chainLog; - - case ZSTD_c_searchLog : - if (value!=0) /* 0 => use default */ - BOUNDCHECK(ZSTD_c_searchLog, value); - CCtxParams->cParams.searchLog = (U32)value; - return (size_t)value; - - case ZSTD_c_minMatch : - if (value!=0) /* 0 => use default */ - BOUNDCHECK(ZSTD_c_minMatch, value); - CCtxParams->cParams.minMatch = value; - return CCtxParams->cParams.minMatch; - - case ZSTD_c_targetLength : - BOUNDCHECK(ZSTD_c_targetLength, value); - CCtxParams->cParams.targetLength = value; - return CCtxParams->cParams.targetLength; - - case ZSTD_c_strategy : - if (value!=0) /* 0 => use default */ - BOUNDCHECK(ZSTD_c_strategy, value); - CCtxParams->cParams.strategy = (ZSTD_strategy)value; - return (size_t)CCtxParams->cParams.strategy; - - case ZSTD_c_contentSizeFlag : - /* Content size written in frame header _when known_ (default:1) */ - DEBUGLOG(4, "set content size flag = %u", (value!=0)); - CCtxParams->fParams.contentSizeFlag = value != 0; - return CCtxParams->fParams.contentSizeFlag; - - case ZSTD_c_checksumFlag : - /* A 32-bits content checksum will be calculated and written at end of frame (default:0) */ - CCtxParams->fParams.checksumFlag = value != 0; - return CCtxParams->fParams.checksumFlag; - - case ZSTD_c_dictIDFlag : /* When applicable, dictionary's dictID is provided in frame header (default:1) */ - DEBUGLOG(4, "set dictIDFlag = %u", (value!=0)); - CCtxParams->fParams.noDictIDFlag = !value; - return !CCtxParams->fParams.noDictIDFlag; - - case ZSTD_c_forceMaxWindow : - CCtxParams->forceWindow = (value != 0); - return CCtxParams->forceWindow; - - case ZSTD_c_forceAttachDict : { - const ZSTD_dictAttachPref_e pref = (ZSTD_dictAttachPref_e)value; - BOUNDCHECK(ZSTD_c_forceAttachDict, pref); - CCtxParams->attachDictPref = pref; - return CCtxParams->attachDictPref; - } - - case ZSTD_c_literalCompressionMode : { - const ZSTD_literalCompressionMode_e lcm = (ZSTD_literalCompressionMode_e)value; - BOUNDCHECK(ZSTD_c_literalCompressionMode, lcm); - CCtxParams->literalCompressionMode = lcm; - return CCtxParams->literalCompressionMode; - } - - case ZSTD_c_nbWorkers : -#ifndef ZSTD_MULTITHREAD - RETURN_ERROR_IF(value!=0, parameter_unsupported, "not compiled with multithreading"); - return 0; -#else - FORWARD_IF_ERROR(ZSTD_cParam_clampBounds(param, &value), ""); - CCtxParams->nbWorkers = value; - return CCtxParams->nbWorkers; -#endif - - case ZSTD_c_jobSize : -#ifndef ZSTD_MULTITHREAD - RETURN_ERROR_IF(value!=0, parameter_unsupported, "not compiled with multithreading"); - return 0; -#else - /* Adjust to the minimum non-default value. */ - if (value != 0 && value < ZSTDMT_JOBSIZE_MIN) - value = ZSTDMT_JOBSIZE_MIN; - FORWARD_IF_ERROR(ZSTD_cParam_clampBounds(param, &value), ""); - assert(value >= 0); - CCtxParams->jobSize = value; - return CCtxParams->jobSize; -#endif - - case ZSTD_c_overlapLog : -#ifndef ZSTD_MULTITHREAD - RETURN_ERROR_IF(value!=0, parameter_unsupported, "not compiled with multithreading"); - return 0; -#else - FORWARD_IF_ERROR(ZSTD_cParam_clampBounds(ZSTD_c_overlapLog, &value), ""); - CCtxParams->overlapLog = value; - return CCtxParams->overlapLog; -#endif - - case ZSTD_c_rsyncable : -#ifndef ZSTD_MULTITHREAD - RETURN_ERROR_IF(value!=0, parameter_unsupported, "not compiled with multithreading"); - return 0; -#else - FORWARD_IF_ERROR(ZSTD_cParam_clampBounds(ZSTD_c_overlapLog, &value), ""); - CCtxParams->rsyncable = value; - return CCtxParams->rsyncable; -#endif - - case ZSTD_c_enableLongDistanceMatching : - CCtxParams->ldmParams.enableLdm = (value!=0); - return CCtxParams->ldmParams.enableLdm; - - case ZSTD_c_ldmHashLog : - if (value!=0) /* 0 ==> auto */ - BOUNDCHECK(ZSTD_c_ldmHashLog, value); - CCtxParams->ldmParams.hashLog = value; - return CCtxParams->ldmParams.hashLog; - - case ZSTD_c_ldmMinMatch : - if (value!=0) /* 0 ==> default */ - BOUNDCHECK(ZSTD_c_ldmMinMatch, value); - CCtxParams->ldmParams.minMatchLength = value; - return CCtxParams->ldmParams.minMatchLength; - - case ZSTD_c_ldmBucketSizeLog : - if (value!=0) /* 0 ==> default */ - BOUNDCHECK(ZSTD_c_ldmBucketSizeLog, value); - CCtxParams->ldmParams.bucketSizeLog = value; - return CCtxParams->ldmParams.bucketSizeLog; - - case ZSTD_c_ldmHashRateLog : - RETURN_ERROR_IF(value > ZSTD_WINDOWLOG_MAX - ZSTD_HASHLOG_MIN, - parameter_outOfBound, "Param out of bounds!"); - CCtxParams->ldmParams.hashRateLog = value; - return CCtxParams->ldmParams.hashRateLog; - - case ZSTD_c_targetCBlockSize : - if (value!=0) /* 0 ==> default */ - BOUNDCHECK(ZSTD_c_targetCBlockSize, value); - CCtxParams->targetCBlockSize = value; - return CCtxParams->targetCBlockSize; - - case ZSTD_c_srcSizeHint : - if (value!=0) /* 0 ==> default */ - BOUNDCHECK(ZSTD_c_srcSizeHint, value); - CCtxParams->srcSizeHint = value; - return CCtxParams->srcSizeHint; - - default: RETURN_ERROR(parameter_unsupported, "unknown parameter"); - } -} - -size_t ZSTD_CCtx_getParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param, int* value) -{ - return ZSTD_CCtxParams_getParameter(&cctx->requestedParams, param, value); -} - -size_t ZSTD_CCtxParams_getParameter( - ZSTD_CCtx_params* CCtxParams, ZSTD_cParameter param, int* value) -{ - switch(param) - { - case ZSTD_c_format : - *value = CCtxParams->format; - break; - case ZSTD_c_compressionLevel : - *value = CCtxParams->compressionLevel; - break; - case ZSTD_c_windowLog : - *value = (int)CCtxParams->cParams.windowLog; - break; - case ZSTD_c_hashLog : - *value = (int)CCtxParams->cParams.hashLog; - break; - case ZSTD_c_chainLog : - *value = (int)CCtxParams->cParams.chainLog; - break; - case ZSTD_c_searchLog : - *value = CCtxParams->cParams.searchLog; - break; - case ZSTD_c_minMatch : - *value = CCtxParams->cParams.minMatch; - break; - case ZSTD_c_targetLength : - *value = CCtxParams->cParams.targetLength; - break; - case ZSTD_c_strategy : - *value = (unsigned)CCtxParams->cParams.strategy; - break; - case ZSTD_c_contentSizeFlag : - *value = CCtxParams->fParams.contentSizeFlag; - break; - case ZSTD_c_checksumFlag : - *value = CCtxParams->fParams.checksumFlag; - break; - case ZSTD_c_dictIDFlag : - *value = !CCtxParams->fParams.noDictIDFlag; - break; - case ZSTD_c_forceMaxWindow : - *value = CCtxParams->forceWindow; - break; - case ZSTD_c_forceAttachDict : - *value = CCtxParams->attachDictPref; - break; - case ZSTD_c_literalCompressionMode : - *value = CCtxParams->literalCompressionMode; - break; - case ZSTD_c_nbWorkers : -#ifndef ZSTD_MULTITHREAD - assert(CCtxParams->nbWorkers == 0); -#endif - *value = CCtxParams->nbWorkers; - break; - case ZSTD_c_jobSize : -#ifndef ZSTD_MULTITHREAD - RETURN_ERROR(parameter_unsupported, "not compiled with multithreading"); -#else - assert(CCtxParams->jobSize <= INT_MAX); - *value = (int)CCtxParams->jobSize; - break; -#endif - case ZSTD_c_overlapLog : -#ifndef ZSTD_MULTITHREAD - RETURN_ERROR(parameter_unsupported, "not compiled with multithreading"); -#else - *value = CCtxParams->overlapLog; - break; -#endif - case ZSTD_c_rsyncable : -#ifndef ZSTD_MULTITHREAD - RETURN_ERROR(parameter_unsupported, "not compiled with multithreading"); -#else - *value = CCtxParams->rsyncable; - break; -#endif - case ZSTD_c_enableLongDistanceMatching : - *value = CCtxParams->ldmParams.enableLdm; - break; - case ZSTD_c_ldmHashLog : - *value = CCtxParams->ldmParams.hashLog; - break; - case ZSTD_c_ldmMinMatch : - *value = CCtxParams->ldmParams.minMatchLength; - break; - case ZSTD_c_ldmBucketSizeLog : - *value = CCtxParams->ldmParams.bucketSizeLog; - break; - case ZSTD_c_ldmHashRateLog : - *value = CCtxParams->ldmParams.hashRateLog; - break; - case ZSTD_c_targetCBlockSize : - *value = (int)CCtxParams->targetCBlockSize; - break; - case ZSTD_c_srcSizeHint : - *value = (int)CCtxParams->srcSizeHint; - break; - default: RETURN_ERROR(parameter_unsupported, "unknown parameter"); - } - return 0; -} - -/** ZSTD_CCtx_setParametersUsingCCtxParams() : - * just applies `params` into `cctx` - * no action is performed, parameters are merely stored. - * If ZSTDMT is enabled, parameters are pushed to cctx->mtctx. - * This is possible even if a compression is ongoing. - * In which case, new parameters will be applied on the fly, starting with next compression job. - */ -size_t ZSTD_CCtx_setParametersUsingCCtxParams( - ZSTD_CCtx* cctx, const ZSTD_CCtx_params* params) -{ - DEBUGLOG(4, "ZSTD_CCtx_setParametersUsingCCtxParams"); - RETURN_ERROR_IF(cctx->streamStage != zcss_init, stage_wrong, - "The context is in the wrong stage!"); - RETURN_ERROR_IF(cctx->cdict, stage_wrong, - "Can't override parameters with cdict attached (some must " - "be inherited from the cdict)."); - - cctx->requestedParams = *params; - return 0; -} - -ZSTDLIB_API size_t ZSTD_CCtx_setPledgedSrcSize(ZSTD_CCtx* cctx, unsigned long long pledgedSrcSize) -{ - DEBUGLOG(4, "ZSTD_CCtx_setPledgedSrcSize to %u bytes", (U32)pledgedSrcSize); - RETURN_ERROR_IF(cctx->streamStage != zcss_init, stage_wrong, - "Can't set pledgedSrcSize when not in init stage."); - cctx->pledgedSrcSizePlusOne = pledgedSrcSize+1; - return 0; -} - -/** - * Initializes the local dict using the requested parameters. - * NOTE: This does not use the pledged src size, because it may be used for more - * than one compression. - */ -static size_t ZSTD_initLocalDict(ZSTD_CCtx* cctx) -{ - ZSTD_localDict* const dl = &cctx->localDict; - ZSTD_compressionParameters const cParams = ZSTD_getCParamsFromCCtxParams( - &cctx->requestedParams, ZSTD_CONTENTSIZE_UNKNOWN, dl->dictSize); - if (dl->dict == NULL) { - /* No local dictionary. */ - assert(dl->dictBuffer == NULL); - assert(dl->cdict == NULL); - assert(dl->dictSize == 0); - return 0; - } - if (dl->cdict != NULL) { - assert(cctx->cdict == dl->cdict); - /* Local dictionary already initialized. */ - return 0; - } - assert(dl->dictSize > 0); - assert(cctx->cdict == NULL); - assert(cctx->prefixDict.dict == NULL); - - dl->cdict = ZSTD_createCDict_advanced( - dl->dict, - dl->dictSize, - ZSTD_dlm_byRef, - dl->dictContentType, - cParams, - cctx->customMem); - RETURN_ERROR_IF(!dl->cdict, memory_allocation, "ZSTD_createCDict_advanced failed"); - cctx->cdict = dl->cdict; - return 0; -} - -size_t ZSTD_CCtx_loadDictionary_advanced( - ZSTD_CCtx* cctx, const void* dict, size_t dictSize, - ZSTD_dictLoadMethod_e dictLoadMethod, ZSTD_dictContentType_e dictContentType) -{ - RETURN_ERROR_IF(cctx->streamStage != zcss_init, stage_wrong, - "Can't load a dictionary when ctx is not in init stage."); - RETURN_ERROR_IF(cctx->staticSize, memory_allocation, - "no malloc for static CCtx"); - DEBUGLOG(4, "ZSTD_CCtx_loadDictionary_advanced (size: %u)", (U32)dictSize); - ZSTD_clearAllDicts(cctx); /* in case one already exists */ - if (dict == NULL || dictSize == 0) /* no dictionary mode */ - return 0; - if (dictLoadMethod == ZSTD_dlm_byRef) { - cctx->localDict.dict = dict; - } else { - void* dictBuffer = ZSTD_malloc(dictSize, cctx->customMem); - RETURN_ERROR_IF(!dictBuffer, memory_allocation, "NULL pointer!"); - memcpy(dictBuffer, dict, dictSize); - cctx->localDict.dictBuffer = dictBuffer; - cctx->localDict.dict = dictBuffer; - } - cctx->localDict.dictSize = dictSize; - cctx->localDict.dictContentType = dictContentType; - return 0; -} - -ZSTDLIB_API size_t ZSTD_CCtx_loadDictionary_byReference( - ZSTD_CCtx* cctx, const void* dict, size_t dictSize) -{ - return ZSTD_CCtx_loadDictionary_advanced( - cctx, dict, dictSize, ZSTD_dlm_byRef, ZSTD_dct_auto); -} - -ZSTDLIB_API size_t ZSTD_CCtx_loadDictionary(ZSTD_CCtx* cctx, const void* dict, size_t dictSize) -{ - return ZSTD_CCtx_loadDictionary_advanced( - cctx, dict, dictSize, ZSTD_dlm_byCopy, ZSTD_dct_auto); -} - - -size_t ZSTD_CCtx_refCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict) -{ - RETURN_ERROR_IF(cctx->streamStage != zcss_init, stage_wrong, - "Can't ref a dict when ctx not in init stage."); - /* Free the existing local cdict (if any) to save memory. */ - ZSTD_clearAllDicts(cctx); - cctx->cdict = cdict; - return 0; -} - -size_t ZSTD_CCtx_refPrefix(ZSTD_CCtx* cctx, const void* prefix, size_t prefixSize) -{ - return ZSTD_CCtx_refPrefix_advanced(cctx, prefix, prefixSize, ZSTD_dct_rawContent); -} - -size_t ZSTD_CCtx_refPrefix_advanced( - ZSTD_CCtx* cctx, const void* prefix, size_t prefixSize, ZSTD_dictContentType_e dictContentType) -{ - RETURN_ERROR_IF(cctx->streamStage != zcss_init, stage_wrong, - "Can't ref a prefix when ctx not in init stage."); - ZSTD_clearAllDicts(cctx); - if (prefix != NULL && prefixSize > 0) { - cctx->prefixDict.dict = prefix; - cctx->prefixDict.dictSize = prefixSize; - cctx->prefixDict.dictContentType = dictContentType; - } - return 0; -} - -/*! ZSTD_CCtx_reset() : - * Also dumps dictionary */ -size_t ZSTD_CCtx_reset(ZSTD_CCtx* cctx, ZSTD_ResetDirective reset) -{ - if ( (reset == ZSTD_reset_session_only) - || (reset == ZSTD_reset_session_and_parameters) ) { - cctx->streamStage = zcss_init; - cctx->pledgedSrcSizePlusOne = 0; - } - if ( (reset == ZSTD_reset_parameters) - || (reset == ZSTD_reset_session_and_parameters) ) { - RETURN_ERROR_IF(cctx->streamStage != zcss_init, stage_wrong, - "Can't reset parameters only when not in init stage."); - ZSTD_clearAllDicts(cctx); - return ZSTD_CCtxParams_reset(&cctx->requestedParams); - } - return 0; -} - - -/** ZSTD_checkCParams() : - control CParam values remain within authorized range. - @return : 0, or an error code if one value is beyond authorized range */ -size_t ZSTD_checkCParams(ZSTD_compressionParameters cParams) -{ - BOUNDCHECK(ZSTD_c_windowLog, (int)cParams.windowLog); - BOUNDCHECK(ZSTD_c_chainLog, (int)cParams.chainLog); - BOUNDCHECK(ZSTD_c_hashLog, (int)cParams.hashLog); - BOUNDCHECK(ZSTD_c_searchLog, (int)cParams.searchLog); - BOUNDCHECK(ZSTD_c_minMatch, (int)cParams.minMatch); - BOUNDCHECK(ZSTD_c_targetLength,(int)cParams.targetLength); - BOUNDCHECK(ZSTD_c_strategy, cParams.strategy); - return 0; -} - -/** ZSTD_clampCParams() : - * make CParam values within valid range. - * @return : valid CParams */ -static ZSTD_compressionParameters -ZSTD_clampCParams(ZSTD_compressionParameters cParams) -{ -# define CLAMP_TYPE(cParam, val, type) { \ - ZSTD_bounds const bounds = ZSTD_cParam_getBounds(cParam); \ - if ((int)valbounds.upperBound) val=(type)bounds.upperBound; \ - } -# define CLAMP(cParam, val) CLAMP_TYPE(cParam, val, unsigned) - CLAMP(ZSTD_c_windowLog, cParams.windowLog); - CLAMP(ZSTD_c_chainLog, cParams.chainLog); - CLAMP(ZSTD_c_hashLog, cParams.hashLog); - CLAMP(ZSTD_c_searchLog, cParams.searchLog); - CLAMP(ZSTD_c_minMatch, cParams.minMatch); - CLAMP(ZSTD_c_targetLength,cParams.targetLength); - CLAMP_TYPE(ZSTD_c_strategy,cParams.strategy, ZSTD_strategy); - return cParams; -} - -/** ZSTD_cycleLog() : - * condition for correct operation : hashLog > 1 */ -U32 ZSTD_cycleLog(U32 hashLog, ZSTD_strategy strat) -{ - U32 const btScale = ((U32)strat >= (U32)ZSTD_btlazy2); - return hashLog - btScale; -} - -/** ZSTD_adjustCParams_internal() : - * optimize `cPar` for a specified input (`srcSize` and `dictSize`). - * mostly downsize to reduce memory consumption and initialization latency. - * `srcSize` can be ZSTD_CONTENTSIZE_UNKNOWN when not known. - * note : `srcSize==0` means 0! - * condition : cPar is presumed validated (can be checked using ZSTD_checkCParams()). */ -static ZSTD_compressionParameters -ZSTD_adjustCParams_internal(ZSTD_compressionParameters cPar, - unsigned long long srcSize, - size_t dictSize) -{ - static const U64 minSrcSize = 513; /* (1<<9) + 1 */ - static const U64 maxWindowResize = 1ULL << (ZSTD_WINDOWLOG_MAX-1); - assert(ZSTD_checkCParams(cPar)==0); - - if (dictSize && srcSize == ZSTD_CONTENTSIZE_UNKNOWN) - srcSize = minSrcSize; - - /* resize windowLog if input is small enough, to use less memory */ - if ( (srcSize < maxWindowResize) - && (dictSize < maxWindowResize) ) { - U32 const tSize = (U32)(srcSize + dictSize); - static U32 const hashSizeMin = 1 << ZSTD_HASHLOG_MIN; - U32 const srcLog = (tSize < hashSizeMin) ? ZSTD_HASHLOG_MIN : - ZSTD_highbit32(tSize-1) + 1; - if (cPar.windowLog > srcLog) cPar.windowLog = srcLog; - } - if (cPar.hashLog > cPar.windowLog+1) cPar.hashLog = cPar.windowLog+1; - { U32 const cycleLog = ZSTD_cycleLog(cPar.chainLog, cPar.strategy); - if (cycleLog > cPar.windowLog) - cPar.chainLog -= (cycleLog - cPar.windowLog); - } - - if (cPar.windowLog < ZSTD_WINDOWLOG_ABSOLUTEMIN) - cPar.windowLog = ZSTD_WINDOWLOG_ABSOLUTEMIN; /* minimum wlog required for valid frame header */ - - return cPar; -} - -ZSTD_compressionParameters -ZSTD_adjustCParams(ZSTD_compressionParameters cPar, - unsigned long long srcSize, - size_t dictSize) -{ - cPar = ZSTD_clampCParams(cPar); /* resulting cPar is necessarily valid (all parameters within range) */ - if (srcSize == 0) srcSize = ZSTD_CONTENTSIZE_UNKNOWN; - return ZSTD_adjustCParams_internal(cPar, srcSize, dictSize); -} - -static ZSTD_compressionParameters ZSTD_getCParams_internal(int compressionLevel, unsigned long long srcSizeHint, size_t dictSize); -static ZSTD_parameters ZSTD_getParams_internal(int compressionLevel, unsigned long long srcSizeHint, size_t dictSize); - -ZSTD_compressionParameters ZSTD_getCParamsFromCCtxParams( - const ZSTD_CCtx_params* CCtxParams, U64 srcSizeHint, size_t dictSize) -{ - ZSTD_compressionParameters cParams; - if (srcSizeHint == ZSTD_CONTENTSIZE_UNKNOWN && CCtxParams->srcSizeHint > 0) { - srcSizeHint = CCtxParams->srcSizeHint; - } - cParams = ZSTD_getCParams_internal(CCtxParams->compressionLevel, srcSizeHint, dictSize); - if (CCtxParams->ldmParams.enableLdm) cParams.windowLog = ZSTD_LDM_DEFAULT_WINDOW_LOG; - if (CCtxParams->cParams.windowLog) cParams.windowLog = CCtxParams->cParams.windowLog; - if (CCtxParams->cParams.hashLog) cParams.hashLog = CCtxParams->cParams.hashLog; - if (CCtxParams->cParams.chainLog) cParams.chainLog = CCtxParams->cParams.chainLog; - if (CCtxParams->cParams.searchLog) cParams.searchLog = CCtxParams->cParams.searchLog; - if (CCtxParams->cParams.minMatch) cParams.minMatch = CCtxParams->cParams.minMatch; - if (CCtxParams->cParams.targetLength) cParams.targetLength = CCtxParams->cParams.targetLength; - if (CCtxParams->cParams.strategy) cParams.strategy = CCtxParams->cParams.strategy; - assert(!ZSTD_checkCParams(cParams)); - /* srcSizeHint == 0 means 0 */ - return ZSTD_adjustCParams_internal(cParams, srcSizeHint, dictSize); -} - -static size_t -ZSTD_sizeof_matchState(const ZSTD_compressionParameters* const cParams, - const U32 forCCtx) -{ - size_t const chainSize = (cParams->strategy == ZSTD_fast) ? 0 : ((size_t)1 << cParams->chainLog); - size_t const hSize = ((size_t)1) << cParams->hashLog; - U32 const hashLog3 = (forCCtx && cParams->minMatch==3) ? MIN(ZSTD_HASHLOG3_MAX, cParams->windowLog) : 0; - size_t const h3Size = hashLog3 ? ((size_t)1) << hashLog3 : 0; - /* We don't use ZSTD_cwksp_alloc_size() here because the tables aren't - * surrounded by redzones in ASAN. */ - size_t const tableSpace = chainSize * sizeof(U32) - + hSize * sizeof(U32) - + h3Size * sizeof(U32); - size_t const optPotentialSpace = - ZSTD_cwksp_alloc_size((MaxML+1) * sizeof(U32)) - + ZSTD_cwksp_alloc_size((MaxLL+1) * sizeof(U32)) - + ZSTD_cwksp_alloc_size((MaxOff+1) * sizeof(U32)) - + ZSTD_cwksp_alloc_size((1<strategy >= ZSTD_btopt)) - ? optPotentialSpace - : 0; - DEBUGLOG(4, "chainSize: %u - hSize: %u - h3Size: %u", - (U32)chainSize, (U32)hSize, (U32)h3Size); - return tableSpace + optSpace; -} - -size_t ZSTD_estimateCCtxSize_usingCCtxParams(const ZSTD_CCtx_params* params) -{ - RETURN_ERROR_IF(params->nbWorkers > 0, GENERIC, "Estimate CCtx size is supported for single-threaded compression only."); - { ZSTD_compressionParameters const cParams = - ZSTD_getCParamsFromCCtxParams(params, ZSTD_CONTENTSIZE_UNKNOWN, 0); - size_t const blockSize = MIN(ZSTD_BLOCKSIZE_MAX, (size_t)1 << cParams.windowLog); - U32 const divider = (cParams.minMatch==3) ? 3 : 4; - size_t const maxNbSeq = blockSize / divider; - size_t const tokenSpace = ZSTD_cwksp_alloc_size(WILDCOPY_OVERLENGTH + blockSize) - + ZSTD_cwksp_alloc_size(maxNbSeq * sizeof(seqDef)) - + 3 * ZSTD_cwksp_alloc_size(maxNbSeq * sizeof(BYTE)); - size_t const entropySpace = ZSTD_cwksp_alloc_size(HUF_WORKSPACE_SIZE); - size_t const blockStateSpace = 2 * ZSTD_cwksp_alloc_size(sizeof(ZSTD_compressedBlockState_t)); - size_t const matchStateSize = ZSTD_sizeof_matchState(&cParams, /* forCCtx */ 1); - - size_t const ldmSpace = ZSTD_ldm_getTableSize(params->ldmParams); - size_t const ldmSeqSpace = ZSTD_cwksp_alloc_size(ZSTD_ldm_getMaxNbSeq(params->ldmParams, blockSize) * sizeof(rawSeq)); - - /* estimateCCtxSize is for one-shot compression. So no buffers should - * be needed. However, we still allocate two 0-sized buffers, which can - * take space under ASAN. */ - size_t const bufferSpace = ZSTD_cwksp_alloc_size(0) - + ZSTD_cwksp_alloc_size(0); - - size_t const cctxSpace = ZSTD_cwksp_alloc_size(sizeof(ZSTD_CCtx)); - - size_t const neededSpace = - cctxSpace + - entropySpace + - blockStateSpace + - ldmSpace + - ldmSeqSpace + - matchStateSize + - tokenSpace + - bufferSpace; - - DEBUGLOG(5, "estimate workspace : %u", (U32)neededSpace); - return neededSpace; - } -} - -size_t ZSTD_estimateCCtxSize_usingCParams(ZSTD_compressionParameters cParams) -{ - ZSTD_CCtx_params const params = ZSTD_makeCCtxParamsFromCParams(cParams); - return ZSTD_estimateCCtxSize_usingCCtxParams(¶ms); -} - -static size_t ZSTD_estimateCCtxSize_internal(int compressionLevel) -{ - ZSTD_compressionParameters const cParams = ZSTD_getCParams_internal(compressionLevel, ZSTD_CONTENTSIZE_UNKNOWN, 0); - return ZSTD_estimateCCtxSize_usingCParams(cParams); -} - -size_t ZSTD_estimateCCtxSize(int compressionLevel) -{ - int level; - size_t memBudget = 0; - for (level=MIN(compressionLevel, 1); level<=compressionLevel; level++) { - size_t const newMB = ZSTD_estimateCCtxSize_internal(level); - if (newMB > memBudget) memBudget = newMB; - } - return memBudget; -} - -size_t ZSTD_estimateCStreamSize_usingCCtxParams(const ZSTD_CCtx_params* params) -{ - RETURN_ERROR_IF(params->nbWorkers > 0, GENERIC, "Estimate CCtx size is supported for single-threaded compression only."); - { ZSTD_compressionParameters const cParams = - ZSTD_getCParamsFromCCtxParams(params, ZSTD_CONTENTSIZE_UNKNOWN, 0); - size_t const CCtxSize = ZSTD_estimateCCtxSize_usingCCtxParams(params); - size_t const blockSize = MIN(ZSTD_BLOCKSIZE_MAX, (size_t)1 << cParams.windowLog); - size_t const inBuffSize = ((size_t)1 << cParams.windowLog) + blockSize; - size_t const outBuffSize = ZSTD_compressBound(blockSize) + 1; - size_t const streamingSize = ZSTD_cwksp_alloc_size(inBuffSize) - + ZSTD_cwksp_alloc_size(outBuffSize); - - return CCtxSize + streamingSize; - } -} - -size_t ZSTD_estimateCStreamSize_usingCParams(ZSTD_compressionParameters cParams) -{ - ZSTD_CCtx_params const params = ZSTD_makeCCtxParamsFromCParams(cParams); - return ZSTD_estimateCStreamSize_usingCCtxParams(¶ms); -} - -static size_t ZSTD_estimateCStreamSize_internal(int compressionLevel) -{ - ZSTD_compressionParameters const cParams = ZSTD_getCParams_internal(compressionLevel, ZSTD_CONTENTSIZE_UNKNOWN, 0); - return ZSTD_estimateCStreamSize_usingCParams(cParams); -} - -size_t ZSTD_estimateCStreamSize(int compressionLevel) -{ - int level; - size_t memBudget = 0; - for (level=MIN(compressionLevel, 1); level<=compressionLevel; level++) { - size_t const newMB = ZSTD_estimateCStreamSize_internal(level); - if (newMB > memBudget) memBudget = newMB; - } - return memBudget; -} - -/* ZSTD_getFrameProgression(): - * tells how much data has been consumed (input) and produced (output) for current frame. - * able to count progression inside worker threads (non-blocking mode). - */ -ZSTD_frameProgression ZSTD_getFrameProgression(const ZSTD_CCtx* cctx) -{ -#ifdef ZSTD_MULTITHREAD - if (cctx->appliedParams.nbWorkers > 0) { - return ZSTDMT_getFrameProgression(cctx->mtctx); - } -#endif - { ZSTD_frameProgression fp; - size_t const buffered = (cctx->inBuff == NULL) ? 0 : - cctx->inBuffPos - cctx->inToCompress; - if (buffered) assert(cctx->inBuffPos >= cctx->inToCompress); - assert(buffered <= ZSTD_BLOCKSIZE_MAX); - fp.ingested = cctx->consumedSrcSize + buffered; - fp.consumed = cctx->consumedSrcSize; - fp.produced = cctx->producedCSize; - fp.flushed = cctx->producedCSize; /* simplified; some data might still be left within streaming output buffer */ - fp.currentJobID = 0; - fp.nbActiveWorkers = 0; - return fp; -} } - -/*! ZSTD_toFlushNow() - * Only useful for multithreading scenarios currently (nbWorkers >= 1). - */ -size_t ZSTD_toFlushNow(ZSTD_CCtx* cctx) -{ -#ifdef ZSTD_MULTITHREAD - if (cctx->appliedParams.nbWorkers > 0) { - return ZSTDMT_toFlushNow(cctx->mtctx); - } -#endif - (void)cctx; - return 0; /* over-simplification; could also check if context is currently running in streaming mode, and in which case, report how many bytes are left to be flushed within output buffer */ -} - -static void ZSTD_assertEqualCParams(ZSTD_compressionParameters cParams1, - ZSTD_compressionParameters cParams2) -{ - (void)cParams1; - (void)cParams2; - assert(cParams1.windowLog == cParams2.windowLog); - assert(cParams1.chainLog == cParams2.chainLog); - assert(cParams1.hashLog == cParams2.hashLog); - assert(cParams1.searchLog == cParams2.searchLog); - assert(cParams1.minMatch == cParams2.minMatch); - assert(cParams1.targetLength == cParams2.targetLength); - assert(cParams1.strategy == cParams2.strategy); -} - -void ZSTD_reset_compressedBlockState(ZSTD_compressedBlockState_t* bs) -{ - int i; - for (i = 0; i < ZSTD_REP_NUM; ++i) - bs->rep[i] = repStartValue[i]; - bs->entropy.huf.repeatMode = HUF_repeat_none; - bs->entropy.fse.offcode_repeatMode = FSE_repeat_none; - bs->entropy.fse.matchlength_repeatMode = FSE_repeat_none; - bs->entropy.fse.litlength_repeatMode = FSE_repeat_none; -} - -/*! ZSTD_invalidateMatchState() - * Invalidate all the matches in the match finder tables. - * Requires nextSrc and base to be set (can be NULL). - */ -static void ZSTD_invalidateMatchState(ZSTD_matchState_t* ms) -{ - ZSTD_window_clear(&ms->window); - - ms->nextToUpdate = ms->window.dictLimit; - ms->loadedDictEnd = 0; - ms->opt.litLengthSum = 0; /* force reset of btopt stats */ - ms->dictMatchState = NULL; -} - -/** - * Indicates whether this compression proceeds directly from user-provided - * source buffer to user-provided destination buffer (ZSTDb_not_buffered), or - * whether the context needs to buffer the input/output (ZSTDb_buffered). - */ -typedef enum { - ZSTDb_not_buffered, - ZSTDb_buffered -} ZSTD_buffered_policy_e; - -/** - * Controls, for this matchState reset, whether the tables need to be cleared / - * prepared for the coming compression (ZSTDcrp_makeClean), or whether the - * tables can be left unclean (ZSTDcrp_leaveDirty), because we know that a - * subsequent operation will overwrite the table space anyways (e.g., copying - * the matchState contents in from a CDict). - */ -typedef enum { - ZSTDcrp_makeClean, - ZSTDcrp_leaveDirty -} ZSTD_compResetPolicy_e; - -/** - * Controls, for this matchState reset, whether indexing can continue where it - * left off (ZSTDirp_continue), or whether it needs to be restarted from zero - * (ZSTDirp_reset). - */ -typedef enum { - ZSTDirp_continue, - ZSTDirp_reset -} ZSTD_indexResetPolicy_e; - -typedef enum { - ZSTD_resetTarget_CDict, - ZSTD_resetTarget_CCtx -} ZSTD_resetTarget_e; - -static size_t -ZSTD_reset_matchState(ZSTD_matchState_t* ms, - ZSTD_cwksp* ws, - const ZSTD_compressionParameters* cParams, - const ZSTD_compResetPolicy_e crp, - const ZSTD_indexResetPolicy_e forceResetIndex, - const ZSTD_resetTarget_e forWho) -{ - size_t const chainSize = (cParams->strategy == ZSTD_fast) ? 0 : ((size_t)1 << cParams->chainLog); - size_t const hSize = ((size_t)1) << cParams->hashLog; - U32 const hashLog3 = ((forWho == ZSTD_resetTarget_CCtx) && cParams->minMatch==3) ? MIN(ZSTD_HASHLOG3_MAX, cParams->windowLog) : 0; - size_t const h3Size = hashLog3 ? ((size_t)1) << hashLog3 : 0; - - DEBUGLOG(4, "reset indices : %u", forceResetIndex == ZSTDirp_reset); - if (forceResetIndex == ZSTDirp_reset) { - ZSTD_window_init(&ms->window); - ZSTD_cwksp_mark_tables_dirty(ws); - } - - ms->hashLog3 = hashLog3; - - ZSTD_invalidateMatchState(ms); - - assert(!ZSTD_cwksp_reserve_failed(ws)); /* check that allocation hasn't already failed */ - - ZSTD_cwksp_clear_tables(ws); - - DEBUGLOG(5, "reserving table space"); - /* table Space */ - ms->hashTable = (U32*)ZSTD_cwksp_reserve_table(ws, hSize * sizeof(U32)); - ms->chainTable = (U32*)ZSTD_cwksp_reserve_table(ws, chainSize * sizeof(U32)); - ms->hashTable3 = (U32*)ZSTD_cwksp_reserve_table(ws, h3Size * sizeof(U32)); - RETURN_ERROR_IF(ZSTD_cwksp_reserve_failed(ws), memory_allocation, - "failed a workspace allocation in ZSTD_reset_matchState"); - - DEBUGLOG(4, "reset table : %u", crp!=ZSTDcrp_leaveDirty); - if (crp!=ZSTDcrp_leaveDirty) { - /* reset tables only */ - ZSTD_cwksp_clean_tables(ws); - } - - /* opt parser space */ - if ((forWho == ZSTD_resetTarget_CCtx) && (cParams->strategy >= ZSTD_btopt)) { - DEBUGLOG(4, "reserving optimal parser space"); - ms->opt.litFreq = (unsigned*)ZSTD_cwksp_reserve_aligned(ws, (1<opt.litLengthFreq = (unsigned*)ZSTD_cwksp_reserve_aligned(ws, (MaxLL+1) * sizeof(unsigned)); - ms->opt.matchLengthFreq = (unsigned*)ZSTD_cwksp_reserve_aligned(ws, (MaxML+1) * sizeof(unsigned)); - ms->opt.offCodeFreq = (unsigned*)ZSTD_cwksp_reserve_aligned(ws, (MaxOff+1) * sizeof(unsigned)); - ms->opt.matchTable = (ZSTD_match_t*)ZSTD_cwksp_reserve_aligned(ws, (ZSTD_OPT_NUM+1) * sizeof(ZSTD_match_t)); - ms->opt.priceTable = (ZSTD_optimal_t*)ZSTD_cwksp_reserve_aligned(ws, (ZSTD_OPT_NUM+1) * sizeof(ZSTD_optimal_t)); - } - - ms->cParams = *cParams; - - RETURN_ERROR_IF(ZSTD_cwksp_reserve_failed(ws), memory_allocation, - "failed a workspace allocation in ZSTD_reset_matchState"); - - return 0; -} - -/* ZSTD_indexTooCloseToMax() : - * minor optimization : prefer memset() rather than reduceIndex() - * which is measurably slow in some circumstances (reported for Visual Studio). - * Works when re-using a context for a lot of smallish inputs : - * if all inputs are smaller than ZSTD_INDEXOVERFLOW_MARGIN, - * memset() will be triggered before reduceIndex(). - */ -#define ZSTD_INDEXOVERFLOW_MARGIN (16 MB) -static int ZSTD_indexTooCloseToMax(ZSTD_window_t w) -{ - return (size_t)(w.nextSrc - w.base) > (ZSTD_CURRENT_MAX - ZSTD_INDEXOVERFLOW_MARGIN); -} - -/*! ZSTD_resetCCtx_internal() : - note : `params` are assumed fully validated at this stage */ -static size_t ZSTD_resetCCtx_internal(ZSTD_CCtx* zc, - ZSTD_CCtx_params params, - U64 const pledgedSrcSize, - ZSTD_compResetPolicy_e const crp, - ZSTD_buffered_policy_e const zbuff) -{ - ZSTD_cwksp* const ws = &zc->workspace; - DEBUGLOG(4, "ZSTD_resetCCtx_internal: pledgedSrcSize=%u, wlog=%u", - (U32)pledgedSrcSize, params.cParams.windowLog); - assert(!ZSTD_isError(ZSTD_checkCParams(params.cParams))); - - zc->isFirstBlock = 1; - - if (params.ldmParams.enableLdm) { - /* Adjust long distance matching parameters */ - ZSTD_ldm_adjustParameters(¶ms.ldmParams, ¶ms.cParams); - assert(params.ldmParams.hashLog >= params.ldmParams.bucketSizeLog); - assert(params.ldmParams.hashRateLog < 32); - zc->ldmState.hashPower = ZSTD_rollingHash_primePower(params.ldmParams.minMatchLength); - } - - { size_t const windowSize = MAX(1, (size_t)MIN(((U64)1 << params.cParams.windowLog), pledgedSrcSize)); - size_t const blockSize = MIN(ZSTD_BLOCKSIZE_MAX, windowSize); - U32 const divider = (params.cParams.minMatch==3) ? 3 : 4; - size_t const maxNbSeq = blockSize / divider; - size_t const tokenSpace = ZSTD_cwksp_alloc_size(WILDCOPY_OVERLENGTH + blockSize) - + ZSTD_cwksp_alloc_size(maxNbSeq * sizeof(seqDef)) - + 3 * ZSTD_cwksp_alloc_size(maxNbSeq * sizeof(BYTE)); - size_t const buffOutSize = (zbuff==ZSTDb_buffered) ? ZSTD_compressBound(blockSize)+1 : 0; - size_t const buffInSize = (zbuff==ZSTDb_buffered) ? windowSize + blockSize : 0; - size_t const matchStateSize = ZSTD_sizeof_matchState(¶ms.cParams, /* forCCtx */ 1); - size_t const maxNbLdmSeq = ZSTD_ldm_getMaxNbSeq(params.ldmParams, blockSize); - - ZSTD_indexResetPolicy_e needsIndexReset = zc->initialized ? ZSTDirp_continue : ZSTDirp_reset; - - if (ZSTD_indexTooCloseToMax(zc->blockState.matchState.window)) { - needsIndexReset = ZSTDirp_reset; - } - - if (!zc->staticSize) ZSTD_cwksp_bump_oversized_duration(ws, 0); - - /* Check if workspace is large enough, alloc a new one if needed */ - { size_t const cctxSpace = zc->staticSize ? ZSTD_cwksp_alloc_size(sizeof(ZSTD_CCtx)) : 0; - size_t const entropySpace = ZSTD_cwksp_alloc_size(HUF_WORKSPACE_SIZE); - size_t const blockStateSpace = 2 * ZSTD_cwksp_alloc_size(sizeof(ZSTD_compressedBlockState_t)); - size_t const bufferSpace = ZSTD_cwksp_alloc_size(buffInSize) + ZSTD_cwksp_alloc_size(buffOutSize); - size_t const ldmSpace = ZSTD_ldm_getTableSize(params.ldmParams); - size_t const ldmSeqSpace = ZSTD_cwksp_alloc_size(maxNbLdmSeq * sizeof(rawSeq)); - - size_t const neededSpace = - cctxSpace + - entropySpace + - blockStateSpace + - ldmSpace + - ldmSeqSpace + - matchStateSize + - tokenSpace + - bufferSpace; - - int const workspaceTooSmall = ZSTD_cwksp_sizeof(ws) < neededSpace; - int const workspaceWasteful = ZSTD_cwksp_check_wasteful(ws, neededSpace); - - DEBUGLOG(4, "Need %zuKB workspace, including %zuKB for match state, and %zuKB for buffers", - neededSpace>>10, matchStateSize>>10, bufferSpace>>10); - DEBUGLOG(4, "windowSize: %zu - blockSize: %zu", windowSize, blockSize); - - if (workspaceTooSmall || workspaceWasteful) { - DEBUGLOG(4, "Resize workspaceSize from %zuKB to %zuKB", - ZSTD_cwksp_sizeof(ws) >> 10, - neededSpace >> 10); - - RETURN_ERROR_IF(zc->staticSize, memory_allocation, "static cctx : no resize"); - - needsIndexReset = ZSTDirp_reset; - - ZSTD_cwksp_free(ws, zc->customMem); - FORWARD_IF_ERROR(ZSTD_cwksp_create(ws, neededSpace, zc->customMem), ""); - - DEBUGLOG(5, "reserving object space"); - /* Statically sized space. - * entropyWorkspace never moves, - * though prev/next block swap places */ - assert(ZSTD_cwksp_check_available(ws, 2 * sizeof(ZSTD_compressedBlockState_t))); - zc->blockState.prevCBlock = (ZSTD_compressedBlockState_t*) ZSTD_cwksp_reserve_object(ws, sizeof(ZSTD_compressedBlockState_t)); - RETURN_ERROR_IF(zc->blockState.prevCBlock == NULL, memory_allocation, "couldn't allocate prevCBlock"); - zc->blockState.nextCBlock = (ZSTD_compressedBlockState_t*) ZSTD_cwksp_reserve_object(ws, sizeof(ZSTD_compressedBlockState_t)); - RETURN_ERROR_IF(zc->blockState.nextCBlock == NULL, memory_allocation, "couldn't allocate nextCBlock"); - zc->entropyWorkspace = (U32*) ZSTD_cwksp_reserve_object(ws, HUF_WORKSPACE_SIZE); - RETURN_ERROR_IF(zc->blockState.nextCBlock == NULL, memory_allocation, "couldn't allocate entropyWorkspace"); - } } - - ZSTD_cwksp_clear(ws); - - /* init params */ - zc->appliedParams = params; - zc->blockState.matchState.cParams = params.cParams; - zc->pledgedSrcSizePlusOne = pledgedSrcSize+1; - zc->consumedSrcSize = 0; - zc->producedCSize = 0; - if (pledgedSrcSize == ZSTD_CONTENTSIZE_UNKNOWN) - zc->appliedParams.fParams.contentSizeFlag = 0; - DEBUGLOG(4, "pledged content size : %u ; flag : %u", - (unsigned)pledgedSrcSize, zc->appliedParams.fParams.contentSizeFlag); - zc->blockSize = blockSize; - - XXH64_reset(&zc->xxhState, 0); - zc->stage = ZSTDcs_init; - zc->dictID = 0; - - ZSTD_reset_compressedBlockState(zc->blockState.prevCBlock); - - /* ZSTD_wildcopy() is used to copy into the literals buffer, - * so we have to oversize the buffer by WILDCOPY_OVERLENGTH bytes. - */ - zc->seqStore.litStart = ZSTD_cwksp_reserve_buffer(ws, blockSize + WILDCOPY_OVERLENGTH); - zc->seqStore.maxNbLit = blockSize; - - /* buffers */ - zc->inBuffSize = buffInSize; - zc->inBuff = (char*)ZSTD_cwksp_reserve_buffer(ws, buffInSize); - zc->outBuffSize = buffOutSize; - zc->outBuff = (char*)ZSTD_cwksp_reserve_buffer(ws, buffOutSize); - - /* ldm bucketOffsets table */ - if (params.ldmParams.enableLdm) { - /* TODO: avoid memset? */ - size_t const ldmBucketSize = - ((size_t)1) << (params.ldmParams.hashLog - - params.ldmParams.bucketSizeLog); - zc->ldmState.bucketOffsets = ZSTD_cwksp_reserve_buffer(ws, ldmBucketSize); - memset(zc->ldmState.bucketOffsets, 0, ldmBucketSize); - } - - /* sequences storage */ - ZSTD_referenceExternalSequences(zc, NULL, 0); - zc->seqStore.maxNbSeq = maxNbSeq; - zc->seqStore.llCode = ZSTD_cwksp_reserve_buffer(ws, maxNbSeq * sizeof(BYTE)); - zc->seqStore.mlCode = ZSTD_cwksp_reserve_buffer(ws, maxNbSeq * sizeof(BYTE)); - zc->seqStore.ofCode = ZSTD_cwksp_reserve_buffer(ws, maxNbSeq * sizeof(BYTE)); - zc->seqStore.sequencesStart = (seqDef*)ZSTD_cwksp_reserve_aligned(ws, maxNbSeq * sizeof(seqDef)); - - FORWARD_IF_ERROR(ZSTD_reset_matchState( - &zc->blockState.matchState, - ws, - ¶ms.cParams, - crp, - needsIndexReset, - ZSTD_resetTarget_CCtx), ""); - - /* ldm hash table */ - if (params.ldmParams.enableLdm) { - /* TODO: avoid memset? */ - size_t const ldmHSize = ((size_t)1) << params.ldmParams.hashLog; - zc->ldmState.hashTable = (ldmEntry_t*)ZSTD_cwksp_reserve_aligned(ws, ldmHSize * sizeof(ldmEntry_t)); - memset(zc->ldmState.hashTable, 0, ldmHSize * sizeof(ldmEntry_t)); - zc->ldmSequences = (rawSeq*)ZSTD_cwksp_reserve_aligned(ws, maxNbLdmSeq * sizeof(rawSeq)); - zc->maxNbLdmSequences = maxNbLdmSeq; - - ZSTD_window_init(&zc->ldmState.window); - ZSTD_window_clear(&zc->ldmState.window); - zc->ldmState.loadedDictEnd = 0; - } - - DEBUGLOG(3, "wksp: finished allocating, %zd bytes remain available", ZSTD_cwksp_available_space(ws)); - zc->initialized = 1; - - return 0; - } -} - -/* ZSTD_invalidateRepCodes() : - * ensures next compression will not use repcodes from previous block. - * Note : only works with regular variant; - * do not use with extDict variant ! */ -void ZSTD_invalidateRepCodes(ZSTD_CCtx* cctx) { - int i; - for (i=0; iblockState.prevCBlock->rep[i] = 0; - assert(!ZSTD_window_hasExtDict(cctx->blockState.matchState.window)); -} - -/* These are the approximate sizes for each strategy past which copying the - * dictionary tables into the working context is faster than using them - * in-place. - */ -static const size_t attachDictSizeCutoffs[ZSTD_STRATEGY_MAX+1] = { - 8 KB, /* unused */ - 8 KB, /* ZSTD_fast */ - 16 KB, /* ZSTD_dfast */ - 32 KB, /* ZSTD_greedy */ - 32 KB, /* ZSTD_lazy */ - 32 KB, /* ZSTD_lazy2 */ - 32 KB, /* ZSTD_btlazy2 */ - 32 KB, /* ZSTD_btopt */ - 8 KB, /* ZSTD_btultra */ - 8 KB /* ZSTD_btultra2 */ -}; - -static int ZSTD_shouldAttachDict(const ZSTD_CDict* cdict, - const ZSTD_CCtx_params* params, - U64 pledgedSrcSize) -{ - size_t cutoff = attachDictSizeCutoffs[cdict->matchState.cParams.strategy]; - return ( pledgedSrcSize <= cutoff - || pledgedSrcSize == ZSTD_CONTENTSIZE_UNKNOWN - || params->attachDictPref == ZSTD_dictForceAttach ) - && params->attachDictPref != ZSTD_dictForceCopy - && !params->forceWindow; /* dictMatchState isn't correctly - * handled in _enforceMaxDist */ -} - -static size_t -ZSTD_resetCCtx_byAttachingCDict(ZSTD_CCtx* cctx, - const ZSTD_CDict* cdict, - ZSTD_CCtx_params params, - U64 pledgedSrcSize, - ZSTD_buffered_policy_e zbuff) -{ - { const ZSTD_compressionParameters* const cdict_cParams = &cdict->matchState.cParams; - unsigned const windowLog = params.cParams.windowLog; - assert(windowLog != 0); - /* Resize working context table params for input only, since the dict - * has its own tables. */ - /* pledgeSrcSize == 0 means 0! */ - params.cParams = ZSTD_adjustCParams_internal(*cdict_cParams, pledgedSrcSize, 0); - params.cParams.windowLog = windowLog; - FORWARD_IF_ERROR(ZSTD_resetCCtx_internal(cctx, params, pledgedSrcSize, - ZSTDcrp_makeClean, zbuff), ""); - assert(cctx->appliedParams.cParams.strategy == cdict_cParams->strategy); - } - - { const U32 cdictEnd = (U32)( cdict->matchState.window.nextSrc - - cdict->matchState.window.base); - const U32 cdictLen = cdictEnd - cdict->matchState.window.dictLimit; - if (cdictLen == 0) { - /* don't even attach dictionaries with no contents */ - DEBUGLOG(4, "skipping attaching empty dictionary"); - } else { - DEBUGLOG(4, "attaching dictionary into context"); - cctx->blockState.matchState.dictMatchState = &cdict->matchState; - - /* prep working match state so dict matches never have negative indices - * when they are translated to the working context's index space. */ - if (cctx->blockState.matchState.window.dictLimit < cdictEnd) { - cctx->blockState.matchState.window.nextSrc = - cctx->blockState.matchState.window.base + cdictEnd; - ZSTD_window_clear(&cctx->blockState.matchState.window); - } - /* loadedDictEnd is expressed within the referential of the active context */ - cctx->blockState.matchState.loadedDictEnd = cctx->blockState.matchState.window.dictLimit; - } } - - cctx->dictID = cdict->dictID; - - /* copy block state */ - memcpy(cctx->blockState.prevCBlock, &cdict->cBlockState, sizeof(cdict->cBlockState)); - - return 0; -} - -static size_t ZSTD_resetCCtx_byCopyingCDict(ZSTD_CCtx* cctx, - const ZSTD_CDict* cdict, - ZSTD_CCtx_params params, - U64 pledgedSrcSize, - ZSTD_buffered_policy_e zbuff) -{ - const ZSTD_compressionParameters *cdict_cParams = &cdict->matchState.cParams; - - DEBUGLOG(4, "copying dictionary into context"); - - { unsigned const windowLog = params.cParams.windowLog; - assert(windowLog != 0); - /* Copy only compression parameters related to tables. */ - params.cParams = *cdict_cParams; - params.cParams.windowLog = windowLog; - FORWARD_IF_ERROR(ZSTD_resetCCtx_internal(cctx, params, pledgedSrcSize, - ZSTDcrp_leaveDirty, zbuff), ""); - assert(cctx->appliedParams.cParams.strategy == cdict_cParams->strategy); - assert(cctx->appliedParams.cParams.hashLog == cdict_cParams->hashLog); - assert(cctx->appliedParams.cParams.chainLog == cdict_cParams->chainLog); - } - - ZSTD_cwksp_mark_tables_dirty(&cctx->workspace); - - /* copy tables */ - { size_t const chainSize = (cdict_cParams->strategy == ZSTD_fast) ? 0 : ((size_t)1 << cdict_cParams->chainLog); - size_t const hSize = (size_t)1 << cdict_cParams->hashLog; - - memcpy(cctx->blockState.matchState.hashTable, - cdict->matchState.hashTable, - hSize * sizeof(U32)); - memcpy(cctx->blockState.matchState.chainTable, - cdict->matchState.chainTable, - chainSize * sizeof(U32)); - } - - /* Zero the hashTable3, since the cdict never fills it */ - { int const h3log = cctx->blockState.matchState.hashLog3; - size_t const h3Size = h3log ? ((size_t)1 << h3log) : 0; - assert(cdict->matchState.hashLog3 == 0); - memset(cctx->blockState.matchState.hashTable3, 0, h3Size * sizeof(U32)); - } - - ZSTD_cwksp_mark_tables_clean(&cctx->workspace); - - /* copy dictionary offsets */ - { ZSTD_matchState_t const* srcMatchState = &cdict->matchState; - ZSTD_matchState_t* dstMatchState = &cctx->blockState.matchState; - dstMatchState->window = srcMatchState->window; - dstMatchState->nextToUpdate = srcMatchState->nextToUpdate; - dstMatchState->loadedDictEnd= srcMatchState->loadedDictEnd; - } - - cctx->dictID = cdict->dictID; - - /* copy block state */ - memcpy(cctx->blockState.prevCBlock, &cdict->cBlockState, sizeof(cdict->cBlockState)); - - return 0; -} - -/* We have a choice between copying the dictionary context into the working - * context, or referencing the dictionary context from the working context - * in-place. We decide here which strategy to use. */ -static size_t ZSTD_resetCCtx_usingCDict(ZSTD_CCtx* cctx, - const ZSTD_CDict* cdict, - const ZSTD_CCtx_params* params, - U64 pledgedSrcSize, - ZSTD_buffered_policy_e zbuff) -{ - - DEBUGLOG(4, "ZSTD_resetCCtx_usingCDict (pledgedSrcSize=%u)", - (unsigned)pledgedSrcSize); - - if (ZSTD_shouldAttachDict(cdict, params, pledgedSrcSize)) { - return ZSTD_resetCCtx_byAttachingCDict( - cctx, cdict, *params, pledgedSrcSize, zbuff); - } else { - return ZSTD_resetCCtx_byCopyingCDict( - cctx, cdict, *params, pledgedSrcSize, zbuff); - } -} - -/*! ZSTD_copyCCtx_internal() : - * Duplicate an existing context `srcCCtx` into another one `dstCCtx`. - * Only works during stage ZSTDcs_init (i.e. after creation, but before first call to ZSTD_compressContinue()). - * The "context", in this case, refers to the hash and chain tables, - * entropy tables, and dictionary references. - * `windowLog` value is enforced if != 0, otherwise value is copied from srcCCtx. - * @return : 0, or an error code */ -static size_t ZSTD_copyCCtx_internal(ZSTD_CCtx* dstCCtx, - const ZSTD_CCtx* srcCCtx, - ZSTD_frameParameters fParams, - U64 pledgedSrcSize, - ZSTD_buffered_policy_e zbuff) -{ - DEBUGLOG(5, "ZSTD_copyCCtx_internal"); - RETURN_ERROR_IF(srcCCtx->stage!=ZSTDcs_init, stage_wrong, - "Can't copy a ctx that's not in init stage."); - - memcpy(&dstCCtx->customMem, &srcCCtx->customMem, sizeof(ZSTD_customMem)); - { ZSTD_CCtx_params params = dstCCtx->requestedParams; - /* Copy only compression parameters related to tables. */ - params.cParams = srcCCtx->appliedParams.cParams; - params.fParams = fParams; - ZSTD_resetCCtx_internal(dstCCtx, params, pledgedSrcSize, - ZSTDcrp_leaveDirty, zbuff); - assert(dstCCtx->appliedParams.cParams.windowLog == srcCCtx->appliedParams.cParams.windowLog); - assert(dstCCtx->appliedParams.cParams.strategy == srcCCtx->appliedParams.cParams.strategy); - assert(dstCCtx->appliedParams.cParams.hashLog == srcCCtx->appliedParams.cParams.hashLog); - assert(dstCCtx->appliedParams.cParams.chainLog == srcCCtx->appliedParams.cParams.chainLog); - assert(dstCCtx->blockState.matchState.hashLog3 == srcCCtx->blockState.matchState.hashLog3); - } - - ZSTD_cwksp_mark_tables_dirty(&dstCCtx->workspace); - - /* copy tables */ - { size_t const chainSize = (srcCCtx->appliedParams.cParams.strategy == ZSTD_fast) ? 0 : ((size_t)1 << srcCCtx->appliedParams.cParams.chainLog); - size_t const hSize = (size_t)1 << srcCCtx->appliedParams.cParams.hashLog; - int const h3log = srcCCtx->blockState.matchState.hashLog3; - size_t const h3Size = h3log ? ((size_t)1 << h3log) : 0; - - memcpy(dstCCtx->blockState.matchState.hashTable, - srcCCtx->blockState.matchState.hashTable, - hSize * sizeof(U32)); - memcpy(dstCCtx->blockState.matchState.chainTable, - srcCCtx->blockState.matchState.chainTable, - chainSize * sizeof(U32)); - memcpy(dstCCtx->blockState.matchState.hashTable3, - srcCCtx->blockState.matchState.hashTable3, - h3Size * sizeof(U32)); - } - - ZSTD_cwksp_mark_tables_clean(&dstCCtx->workspace); - - /* copy dictionary offsets */ - { - const ZSTD_matchState_t* srcMatchState = &srcCCtx->blockState.matchState; - ZSTD_matchState_t* dstMatchState = &dstCCtx->blockState.matchState; - dstMatchState->window = srcMatchState->window; - dstMatchState->nextToUpdate = srcMatchState->nextToUpdate; - dstMatchState->loadedDictEnd= srcMatchState->loadedDictEnd; - } - dstCCtx->dictID = srcCCtx->dictID; - - /* copy block state */ - memcpy(dstCCtx->blockState.prevCBlock, srcCCtx->blockState.prevCBlock, sizeof(*srcCCtx->blockState.prevCBlock)); - - return 0; -} - -/*! ZSTD_copyCCtx() : - * Duplicate an existing context `srcCCtx` into another one `dstCCtx`. - * Only works during stage ZSTDcs_init (i.e. after creation, but before first call to ZSTD_compressContinue()). - * pledgedSrcSize==0 means "unknown". -* @return : 0, or an error code */ -size_t ZSTD_copyCCtx(ZSTD_CCtx* dstCCtx, const ZSTD_CCtx* srcCCtx, unsigned long long pledgedSrcSize) -{ - ZSTD_frameParameters fParams = { 1 /*content*/, 0 /*checksum*/, 0 /*noDictID*/ }; - ZSTD_buffered_policy_e const zbuff = (ZSTD_buffered_policy_e)(srcCCtx->inBuffSize>0); - ZSTD_STATIC_ASSERT((U32)ZSTDb_buffered==1); - if (pledgedSrcSize==0) pledgedSrcSize = ZSTD_CONTENTSIZE_UNKNOWN; - fParams.contentSizeFlag = (pledgedSrcSize != ZSTD_CONTENTSIZE_UNKNOWN); - - return ZSTD_copyCCtx_internal(dstCCtx, srcCCtx, - fParams, pledgedSrcSize, - zbuff); -} - - -#define ZSTD_ROWSIZE 16 -/*! ZSTD_reduceTable() : - * reduce table indexes by `reducerValue`, or squash to zero. - * PreserveMark preserves "unsorted mark" for btlazy2 strategy. - * It must be set to a clear 0/1 value, to remove branch during inlining. - * Presume table size is a multiple of ZSTD_ROWSIZE - * to help auto-vectorization */ -FORCE_INLINE_TEMPLATE void -ZSTD_reduceTable_internal (U32* const table, U32 const size, U32 const reducerValue, int const preserveMark) -{ - int const nbRows = (int)size / ZSTD_ROWSIZE; - int cellNb = 0; - int rowNb; - assert((size & (ZSTD_ROWSIZE-1)) == 0); /* multiple of ZSTD_ROWSIZE */ - assert(size < (1U<<31)); /* can be casted to int */ - -#if defined (MEMORY_SANITIZER) && !defined (ZSTD_MSAN_DONT_POISON_WORKSPACE) - /* To validate that the table re-use logic is sound, and that we don't - * access table space that we haven't cleaned, we re-"poison" the table - * space every time we mark it dirty. - * - * This function however is intended to operate on those dirty tables and - * re-clean them. So when this function is used correctly, we can unpoison - * the memory it operated on. This introduces a blind spot though, since - * if we now try to operate on __actually__ poisoned memory, we will not - * detect that. */ - __msan_unpoison(table, size * sizeof(U32)); -#endif - - for (rowNb=0 ; rowNb < nbRows ; rowNb++) { - int column; - for (column=0; columncParams.hashLog; - ZSTD_reduceTable(ms->hashTable, hSize, reducerValue); - } - - if (params->cParams.strategy != ZSTD_fast) { - U32 const chainSize = (U32)1 << params->cParams.chainLog; - if (params->cParams.strategy == ZSTD_btlazy2) - ZSTD_reduceTable_btlazy2(ms->chainTable, chainSize, reducerValue); - else - ZSTD_reduceTable(ms->chainTable, chainSize, reducerValue); - } - - if (ms->hashLog3) { - U32 const h3Size = (U32)1 << ms->hashLog3; - ZSTD_reduceTable(ms->hashTable3, h3Size, reducerValue); - } -} - - -/*-******************************************************* -* Block entropic compression -*********************************************************/ - -/* See doc/zstd_compression_format.md for detailed format description */ - -void ZSTD_seqToCodes(const seqStore_t* seqStorePtr) -{ - const seqDef* const sequences = seqStorePtr->sequencesStart; - BYTE* const llCodeTable = seqStorePtr->llCode; - BYTE* const ofCodeTable = seqStorePtr->ofCode; - BYTE* const mlCodeTable = seqStorePtr->mlCode; - U32 const nbSeq = (U32)(seqStorePtr->sequences - seqStorePtr->sequencesStart); - U32 u; - assert(nbSeq <= seqStorePtr->maxNbSeq); - for (u=0; ulongLengthID==1) - llCodeTable[seqStorePtr->longLengthPos] = MaxLL; - if (seqStorePtr->longLengthID==2) - mlCodeTable[seqStorePtr->longLengthPos] = MaxML; -} - -/* ZSTD_useTargetCBlockSize(): - * Returns if target compressed block size param is being used. - * If used, compression will do best effort to make a compressed block size to be around targetCBlockSize. - * Returns 1 if true, 0 otherwise. */ -static int ZSTD_useTargetCBlockSize(const ZSTD_CCtx_params* cctxParams) -{ - DEBUGLOG(5, "ZSTD_useTargetCBlockSize (targetCBlockSize=%zu)", cctxParams->targetCBlockSize); - return (cctxParams->targetCBlockSize != 0); -} - -/* ZSTD_compressSequences_internal(): - * actually compresses both literals and sequences */ -MEM_STATIC size_t -ZSTD_compressSequences_internal(seqStore_t* seqStorePtr, - const ZSTD_entropyCTables_t* prevEntropy, - ZSTD_entropyCTables_t* nextEntropy, - const ZSTD_CCtx_params* cctxParams, - void* dst, size_t dstCapacity, - void* entropyWorkspace, size_t entropyWkspSize, - const int bmi2) -{ - const int longOffsets = cctxParams->cParams.windowLog > STREAM_ACCUMULATOR_MIN; - ZSTD_strategy const strategy = cctxParams->cParams.strategy; - unsigned count[MaxSeq+1]; - FSE_CTable* CTable_LitLength = nextEntropy->fse.litlengthCTable; - FSE_CTable* CTable_OffsetBits = nextEntropy->fse.offcodeCTable; - FSE_CTable* CTable_MatchLength = nextEntropy->fse.matchlengthCTable; - U32 LLtype, Offtype, MLtype; /* compressed, raw or rle */ - const seqDef* const sequences = seqStorePtr->sequencesStart; - const BYTE* const ofCodeTable = seqStorePtr->ofCode; - const BYTE* const llCodeTable = seqStorePtr->llCode; - const BYTE* const mlCodeTable = seqStorePtr->mlCode; - BYTE* const ostart = (BYTE*)dst; - BYTE* const oend = ostart + dstCapacity; - BYTE* op = ostart; - size_t const nbSeq = (size_t)(seqStorePtr->sequences - seqStorePtr->sequencesStart); - BYTE* seqHead; - BYTE* lastNCount = NULL; - - DEBUGLOG(5, "ZSTD_compressSequences_internal (nbSeq=%zu)", nbSeq); - ZSTD_STATIC_ASSERT(HUF_WORKSPACE_SIZE >= (1<litStart; - size_t const litSize = (size_t)(seqStorePtr->lit - literals); - size_t const cSize = ZSTD_compressLiterals( - &prevEntropy->huf, &nextEntropy->huf, - cctxParams->cParams.strategy, - ZSTD_disableLiteralsCompression(cctxParams), - op, dstCapacity, - literals, litSize, - entropyWorkspace, entropyWkspSize, - bmi2); - FORWARD_IF_ERROR(cSize, "ZSTD_compressLiterals failed"); - assert(cSize <= dstCapacity); - op += cSize; - } - - /* Sequences Header */ - RETURN_ERROR_IF((oend-op) < 3 /*max nbSeq Size*/ + 1 /*seqHead*/, - dstSize_tooSmall, "Can't fit seq hdr in output buf!"); - if (nbSeq < 128) { - *op++ = (BYTE)nbSeq; - } else if (nbSeq < LONGNBSEQ) { - op[0] = (BYTE)((nbSeq>>8) + 0x80); - op[1] = (BYTE)nbSeq; - op+=2; - } else { - op[0]=0xFF; - MEM_writeLE16(op+1, (U16)(nbSeq - LONGNBSEQ)); - op+=3; - } - assert(op <= oend); - if (nbSeq==0) { - /* Copy the old tables over as if we repeated them */ - memcpy(&nextEntropy->fse, &prevEntropy->fse, sizeof(prevEntropy->fse)); - return (size_t)(op - ostart); - } - - /* seqHead : flags for FSE encoding type */ - seqHead = op++; - assert(op <= oend); - - /* convert length/distances into codes */ - ZSTD_seqToCodes(seqStorePtr); - /* build CTable for Literal Lengths */ - { unsigned max = MaxLL; - size_t const mostFrequent = HIST_countFast_wksp(count, &max, llCodeTable, nbSeq, entropyWorkspace, entropyWkspSize); /* can't fail */ - DEBUGLOG(5, "Building LL table"); - nextEntropy->fse.litlength_repeatMode = prevEntropy->fse.litlength_repeatMode; - LLtype = ZSTD_selectEncodingType(&nextEntropy->fse.litlength_repeatMode, - count, max, mostFrequent, nbSeq, - LLFSELog, prevEntropy->fse.litlengthCTable, - LL_defaultNorm, LL_defaultNormLog, - ZSTD_defaultAllowed, strategy); - assert(set_basic < set_compressed && set_rle < set_compressed); - assert(!(LLtype < set_compressed && nextEntropy->fse.litlength_repeatMode != FSE_repeat_none)); /* We don't copy tables */ - { size_t const countSize = ZSTD_buildCTable( - op, (size_t)(oend - op), - CTable_LitLength, LLFSELog, (symbolEncodingType_e)LLtype, - count, max, llCodeTable, nbSeq, - LL_defaultNorm, LL_defaultNormLog, MaxLL, - prevEntropy->fse.litlengthCTable, - sizeof(prevEntropy->fse.litlengthCTable), - entropyWorkspace, entropyWkspSize); - FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for LitLens failed"); - if (LLtype == set_compressed) - lastNCount = op; - op += countSize; - assert(op <= oend); - } } - /* build CTable for Offsets */ - { unsigned max = MaxOff; - size_t const mostFrequent = HIST_countFast_wksp( - count, &max, ofCodeTable, nbSeq, entropyWorkspace, entropyWkspSize); /* can't fail */ - /* We can only use the basic table if max <= DefaultMaxOff, otherwise the offsets are too large */ - ZSTD_defaultPolicy_e const defaultPolicy = (max <= DefaultMaxOff) ? ZSTD_defaultAllowed : ZSTD_defaultDisallowed; - DEBUGLOG(5, "Building OF table"); - nextEntropy->fse.offcode_repeatMode = prevEntropy->fse.offcode_repeatMode; - Offtype = ZSTD_selectEncodingType(&nextEntropy->fse.offcode_repeatMode, - count, max, mostFrequent, nbSeq, - OffFSELog, prevEntropy->fse.offcodeCTable, - OF_defaultNorm, OF_defaultNormLog, - defaultPolicy, strategy); - assert(!(Offtype < set_compressed && nextEntropy->fse.offcode_repeatMode != FSE_repeat_none)); /* We don't copy tables */ - { size_t const countSize = ZSTD_buildCTable( - op, (size_t)(oend - op), - CTable_OffsetBits, OffFSELog, (symbolEncodingType_e)Offtype, - count, max, ofCodeTable, nbSeq, - OF_defaultNorm, OF_defaultNormLog, DefaultMaxOff, - prevEntropy->fse.offcodeCTable, - sizeof(prevEntropy->fse.offcodeCTable), - entropyWorkspace, entropyWkspSize); - FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for Offsets failed"); - if (Offtype == set_compressed) - lastNCount = op; - op += countSize; - assert(op <= oend); - } } - /* build CTable for MatchLengths */ - { unsigned max = MaxML; - size_t const mostFrequent = HIST_countFast_wksp( - count, &max, mlCodeTable, nbSeq, entropyWorkspace, entropyWkspSize); /* can't fail */ - DEBUGLOG(5, "Building ML table (remaining space : %i)", (int)(oend-op)); - nextEntropy->fse.matchlength_repeatMode = prevEntropy->fse.matchlength_repeatMode; - MLtype = ZSTD_selectEncodingType(&nextEntropy->fse.matchlength_repeatMode, - count, max, mostFrequent, nbSeq, - MLFSELog, prevEntropy->fse.matchlengthCTable, - ML_defaultNorm, ML_defaultNormLog, - ZSTD_defaultAllowed, strategy); - assert(!(MLtype < set_compressed && nextEntropy->fse.matchlength_repeatMode != FSE_repeat_none)); /* We don't copy tables */ - { size_t const countSize = ZSTD_buildCTable( - op, (size_t)(oend - op), - CTable_MatchLength, MLFSELog, (symbolEncodingType_e)MLtype, - count, max, mlCodeTable, nbSeq, - ML_defaultNorm, ML_defaultNormLog, MaxML, - prevEntropy->fse.matchlengthCTable, - sizeof(prevEntropy->fse.matchlengthCTable), - entropyWorkspace, entropyWkspSize); - FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for MatchLengths failed"); - if (MLtype == set_compressed) - lastNCount = op; - op += countSize; - assert(op <= oend); - } } - - *seqHead = (BYTE)((LLtype<<6) + (Offtype<<4) + (MLtype<<2)); - - { size_t const bitstreamSize = ZSTD_encodeSequences( - op, (size_t)(oend - op), - CTable_MatchLength, mlCodeTable, - CTable_OffsetBits, ofCodeTable, - CTable_LitLength, llCodeTable, - sequences, nbSeq, - longOffsets, bmi2); - FORWARD_IF_ERROR(bitstreamSize, "ZSTD_encodeSequences failed"); - op += bitstreamSize; - assert(op <= oend); - /* zstd versions <= 1.3.4 mistakenly report corruption when - * FSE_readNCount() receives a buffer < 4 bytes. - * Fixed by https://github.com/facebook/zstd/pull/1146. - * This can happen when the last set_compressed table present is 2 - * bytes and the bitstream is only one byte. - * In this exceedingly rare case, we will simply emit an uncompressed - * block, since it isn't worth optimizing. - */ - if (lastNCount && (op - lastNCount) < 4) { - /* NCountSize >= 2 && bitstreamSize > 0 ==> lastCountSize == 3 */ - assert(op - lastNCount == 3); - DEBUGLOG(5, "Avoiding bug in zstd decoder in versions <= 1.3.4 by " - "emitting an uncompressed block."); - return 0; - } - } - - DEBUGLOG(5, "compressed block size : %u", (unsigned)(op - ostart)); - return (size_t)(op - ostart); -} - -MEM_STATIC size_t -ZSTD_compressSequences(seqStore_t* seqStorePtr, - const ZSTD_entropyCTables_t* prevEntropy, - ZSTD_entropyCTables_t* nextEntropy, - const ZSTD_CCtx_params* cctxParams, - void* dst, size_t dstCapacity, - size_t srcSize, - void* entropyWorkspace, size_t entropyWkspSize, - int bmi2) -{ - size_t const cSize = ZSTD_compressSequences_internal( - seqStorePtr, prevEntropy, nextEntropy, cctxParams, - dst, dstCapacity, - entropyWorkspace, entropyWkspSize, bmi2); - if (cSize == 0) return 0; - /* When srcSize <= dstCapacity, there is enough space to write a raw uncompressed block. - * Since we ran out of space, block must be not compressible, so fall back to raw uncompressed block. - */ - if ((cSize == ERROR(dstSize_tooSmall)) & (srcSize <= dstCapacity)) - return 0; /* block not compressed */ - FORWARD_IF_ERROR(cSize, "ZSTD_compressSequences_internal failed"); - - /* Check compressibility */ - { size_t const maxCSize = srcSize - ZSTD_minGain(srcSize, cctxParams->cParams.strategy); - if (cSize >= maxCSize) return 0; /* block not compressed */ - } - - return cSize; -} - -/* ZSTD_selectBlockCompressor() : - * Not static, but internal use only (used by long distance matcher) - * assumption : strat is a valid strategy */ -ZSTD_blockCompressor ZSTD_selectBlockCompressor(ZSTD_strategy strat, ZSTD_dictMode_e dictMode) -{ - static const ZSTD_blockCompressor blockCompressor[3][ZSTD_STRATEGY_MAX+1] = { - { ZSTD_compressBlock_fast /* default for 0 */, - ZSTD_compressBlock_fast, - ZSTD_compressBlock_doubleFast, - ZSTD_compressBlock_greedy, - ZSTD_compressBlock_lazy, - ZSTD_compressBlock_lazy2, - ZSTD_compressBlock_btlazy2, - ZSTD_compressBlock_btopt, - ZSTD_compressBlock_btultra, - ZSTD_compressBlock_btultra2 }, - { ZSTD_compressBlock_fast_extDict /* default for 0 */, - ZSTD_compressBlock_fast_extDict, - ZSTD_compressBlock_doubleFast_extDict, - ZSTD_compressBlock_greedy_extDict, - ZSTD_compressBlock_lazy_extDict, - ZSTD_compressBlock_lazy2_extDict, - ZSTD_compressBlock_btlazy2_extDict, - ZSTD_compressBlock_btopt_extDict, - ZSTD_compressBlock_btultra_extDict, - ZSTD_compressBlock_btultra_extDict }, - { ZSTD_compressBlock_fast_dictMatchState /* default for 0 */, - ZSTD_compressBlock_fast_dictMatchState, - ZSTD_compressBlock_doubleFast_dictMatchState, - ZSTD_compressBlock_greedy_dictMatchState, - ZSTD_compressBlock_lazy_dictMatchState, - ZSTD_compressBlock_lazy2_dictMatchState, - ZSTD_compressBlock_btlazy2_dictMatchState, - ZSTD_compressBlock_btopt_dictMatchState, - ZSTD_compressBlock_btultra_dictMatchState, - ZSTD_compressBlock_btultra_dictMatchState } - }; - ZSTD_blockCompressor selectedCompressor; - ZSTD_STATIC_ASSERT((unsigned)ZSTD_fast == 1); - - assert(ZSTD_cParam_withinBounds(ZSTD_c_strategy, strat)); - selectedCompressor = blockCompressor[(int)dictMode][(int)strat]; - assert(selectedCompressor != NULL); - return selectedCompressor; -} - -static void ZSTD_storeLastLiterals(seqStore_t* seqStorePtr, - const BYTE* anchor, size_t lastLLSize) -{ - memcpy(seqStorePtr->lit, anchor, lastLLSize); - seqStorePtr->lit += lastLLSize; -} - -void ZSTD_resetSeqStore(seqStore_t* ssPtr) -{ - ssPtr->lit = ssPtr->litStart; - ssPtr->sequences = ssPtr->sequencesStart; - ssPtr->longLengthID = 0; -} - -typedef enum { ZSTDbss_compress, ZSTDbss_noCompress } ZSTD_buildSeqStore_e; - -static size_t ZSTD_buildSeqStore(ZSTD_CCtx* zc, const void* src, size_t srcSize) -{ - ZSTD_matchState_t* const ms = &zc->blockState.matchState; - DEBUGLOG(5, "ZSTD_buildSeqStore (srcSize=%zu)", srcSize); - assert(srcSize <= ZSTD_BLOCKSIZE_MAX); - /* Assert that we have correctly flushed the ctx params into the ms's copy */ - ZSTD_assertEqualCParams(zc->appliedParams.cParams, ms->cParams); - if (srcSize < MIN_CBLOCK_SIZE+ZSTD_blockHeaderSize+1) { - ZSTD_ldm_skipSequences(&zc->externSeqStore, srcSize, zc->appliedParams.cParams.minMatch); - return ZSTDbss_noCompress; /* don't even attempt compression below a certain srcSize */ - } - ZSTD_resetSeqStore(&(zc->seqStore)); - /* required for optimal parser to read stats from dictionary */ - ms->opt.symbolCosts = &zc->blockState.prevCBlock->entropy; - /* tell the optimal parser how we expect to compress literals */ - ms->opt.literalCompressionMode = zc->appliedParams.literalCompressionMode; - /* a gap between an attached dict and the current window is not safe, - * they must remain adjacent, - * and when that stops being the case, the dict must be unset */ - assert(ms->dictMatchState == NULL || ms->loadedDictEnd == ms->window.dictLimit); - - /* limited update after a very long match */ - { const BYTE* const base = ms->window.base; - const BYTE* const istart = (const BYTE*)src; - const U32 current = (U32)(istart-base); - if (sizeof(ptrdiff_t)==8) assert(istart - base < (ptrdiff_t)(U32)(-1)); /* ensure no overflow */ - if (current > ms->nextToUpdate + 384) - ms->nextToUpdate = current - MIN(192, (U32)(current - ms->nextToUpdate - 384)); - } - - /* select and store sequences */ - { ZSTD_dictMode_e const dictMode = ZSTD_matchState_dictMode(ms); - size_t lastLLSize; - { int i; - for (i = 0; i < ZSTD_REP_NUM; ++i) - zc->blockState.nextCBlock->rep[i] = zc->blockState.prevCBlock->rep[i]; - } - if (zc->externSeqStore.pos < zc->externSeqStore.size) { - assert(!zc->appliedParams.ldmParams.enableLdm); - /* Updates ldmSeqStore.pos */ - lastLLSize = - ZSTD_ldm_blockCompress(&zc->externSeqStore, - ms, &zc->seqStore, - zc->blockState.nextCBlock->rep, - src, srcSize); - assert(zc->externSeqStore.pos <= zc->externSeqStore.size); - } else if (zc->appliedParams.ldmParams.enableLdm) { - rawSeqStore_t ldmSeqStore = {NULL, 0, 0, 0}; - - ldmSeqStore.seq = zc->ldmSequences; - ldmSeqStore.capacity = zc->maxNbLdmSequences; - /* Updates ldmSeqStore.size */ - FORWARD_IF_ERROR(ZSTD_ldm_generateSequences(&zc->ldmState, &ldmSeqStore, - &zc->appliedParams.ldmParams, - src, srcSize), ""); - /* Updates ldmSeqStore.pos */ - lastLLSize = - ZSTD_ldm_blockCompress(&ldmSeqStore, - ms, &zc->seqStore, - zc->blockState.nextCBlock->rep, - src, srcSize); - assert(ldmSeqStore.pos == ldmSeqStore.size); - } else { /* not long range mode */ - ZSTD_blockCompressor const blockCompressor = ZSTD_selectBlockCompressor(zc->appliedParams.cParams.strategy, dictMode); - lastLLSize = blockCompressor(ms, &zc->seqStore, zc->blockState.nextCBlock->rep, src, srcSize); - } - { const BYTE* const lastLiterals = (const BYTE*)src + srcSize - lastLLSize; - ZSTD_storeLastLiterals(&zc->seqStore, lastLiterals, lastLLSize); - } } - return ZSTDbss_compress; -} - -static void ZSTD_copyBlockSequences(ZSTD_CCtx* zc) -{ - const seqStore_t* seqStore = ZSTD_getSeqStore(zc); - const seqDef* seqs = seqStore->sequencesStart; - size_t seqsSize = seqStore->sequences - seqs; - - ZSTD_Sequence* outSeqs = &zc->seqCollector.seqStart[zc->seqCollector.seqIndex]; - size_t i; size_t position; int repIdx; - - assert(zc->seqCollector.seqIndex + 1 < zc->seqCollector.maxSequences); - for (i = 0, position = 0; i < seqsSize; ++i) { - outSeqs[i].offset = seqs[i].offset; - outSeqs[i].litLength = seqs[i].litLength; - outSeqs[i].matchLength = seqs[i].matchLength + MINMATCH; - - if (i == seqStore->longLengthPos) { - if (seqStore->longLengthID == 1) { - outSeqs[i].litLength += 0x10000; - } else if (seqStore->longLengthID == 2) { - outSeqs[i].matchLength += 0x10000; - } - } - - if (outSeqs[i].offset <= ZSTD_REP_NUM) { - outSeqs[i].rep = outSeqs[i].offset; - repIdx = (unsigned int)i - outSeqs[i].offset; - - if (outSeqs[i].litLength == 0) { - if (outSeqs[i].offset < 3) { - --repIdx; - } else { - repIdx = (unsigned int)i - 1; - } - ++outSeqs[i].rep; - } - assert(repIdx >= -3); - outSeqs[i].offset = repIdx >= 0 ? outSeqs[repIdx].offset : repStartValue[-repIdx - 1]; - if (outSeqs[i].rep == 4) { - --outSeqs[i].offset; - } - } else { - outSeqs[i].offset -= ZSTD_REP_NUM; - } - - position += outSeqs[i].litLength; - outSeqs[i].matchPos = (unsigned int)position; - position += outSeqs[i].matchLength; - } - zc->seqCollector.seqIndex += seqsSize; -} - -size_t ZSTD_getSequences(ZSTD_CCtx* zc, ZSTD_Sequence* outSeqs, - size_t outSeqsSize, const void* src, size_t srcSize) -{ - const size_t dstCapacity = ZSTD_compressBound(srcSize); - void* dst = ZSTD_malloc(dstCapacity, ZSTD_defaultCMem); - SeqCollector seqCollector; - - RETURN_ERROR_IF(dst == NULL, memory_allocation, "NULL pointer!"); - - seqCollector.collectSequences = 1; - seqCollector.seqStart = outSeqs; - seqCollector.seqIndex = 0; - seqCollector.maxSequences = outSeqsSize; - zc->seqCollector = seqCollector; - - ZSTD_compress2(zc, dst, dstCapacity, src, srcSize); - ZSTD_free(dst, ZSTD_defaultCMem); - return zc->seqCollector.seqIndex; -} - -/* Returns true if the given block is a RLE block */ -static int ZSTD_isRLE(const BYTE *ip, size_t length) { - size_t i; - if (length < 2) return 1; - for (i = 1; i < length; ++i) { - if (ip[0] != ip[i]) return 0; - } - return 1; -} - -/* Returns true if the given block may be RLE. - * This is just a heuristic based on the compressibility. - * It may return both false positives and false negatives. - */ -static int ZSTD_maybeRLE(seqStore_t const* seqStore) -{ - size_t const nbSeqs = (size_t)(seqStore->sequences - seqStore->sequencesStart); - size_t const nbLits = (size_t)(seqStore->lit - seqStore->litStart); - - return nbSeqs < 4 && nbLits < 10; -} - -static void ZSTD_confirmRepcodesAndEntropyTables(ZSTD_CCtx* zc) -{ - ZSTD_compressedBlockState_t* const tmp = zc->blockState.prevCBlock; - zc->blockState.prevCBlock = zc->blockState.nextCBlock; - zc->blockState.nextCBlock = tmp; -} - -static size_t ZSTD_compressBlock_internal(ZSTD_CCtx* zc, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, U32 frame) -{ - /* This the upper bound for the length of an rle block. - * This isn't the actual upper bound. Finding the real threshold - * needs further investigation. - */ - const U32 rleMaxLength = 25; - size_t cSize; - const BYTE* ip = (const BYTE*)src; - BYTE* op = (BYTE*)dst; - DEBUGLOG(5, "ZSTD_compressBlock_internal (dstCapacity=%u, dictLimit=%u, nextToUpdate=%u)", - (unsigned)dstCapacity, (unsigned)zc->blockState.matchState.window.dictLimit, - (unsigned)zc->blockState.matchState.nextToUpdate); - - { const size_t bss = ZSTD_buildSeqStore(zc, src, srcSize); - FORWARD_IF_ERROR(bss, "ZSTD_buildSeqStore failed"); - if (bss == ZSTDbss_noCompress) { cSize = 0; goto out; } - } - - if (zc->seqCollector.collectSequences) { - ZSTD_copyBlockSequences(zc); - return 0; - } - - /* encode sequences and literals */ - cSize = ZSTD_compressSequences(&zc->seqStore, - &zc->blockState.prevCBlock->entropy, &zc->blockState.nextCBlock->entropy, - &zc->appliedParams, - dst, dstCapacity, - srcSize, - zc->entropyWorkspace, HUF_WORKSPACE_SIZE /* statically allocated in resetCCtx */, - zc->bmi2); - - if (frame && - /* We don't want to emit our first block as a RLE even if it qualifies because - * doing so will cause the decoder (cli only) to throw a "should consume all input error." - * This is only an issue for zstd <= v1.4.3 - */ - !zc->isFirstBlock && - cSize < rleMaxLength && - ZSTD_isRLE(ip, srcSize)) - { - cSize = 1; - op[0] = ip[0]; - } - -out: - if (!ZSTD_isError(cSize) && cSize > 1) { - ZSTD_confirmRepcodesAndEntropyTables(zc); - } - /* We check that dictionaries have offset codes available for the first - * block. After the first block, the offcode table might not have large - * enough codes to represent the offsets in the data. - */ - if (zc->blockState.prevCBlock->entropy.fse.offcode_repeatMode == FSE_repeat_valid) - zc->blockState.prevCBlock->entropy.fse.offcode_repeatMode = FSE_repeat_check; - - return cSize; -} - -static size_t ZSTD_compressBlock_targetCBlockSize_body(ZSTD_CCtx* zc, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const size_t bss, U32 lastBlock) -{ - DEBUGLOG(6, "Attempting ZSTD_compressSuperBlock()"); - if (bss == ZSTDbss_compress) { - if (/* We don't want to emit our first block as a RLE even if it qualifies because - * doing so will cause the decoder (cli only) to throw a "should consume all input error." - * This is only an issue for zstd <= v1.4.3 - */ - !zc->isFirstBlock && - ZSTD_maybeRLE(&zc->seqStore) && - ZSTD_isRLE((BYTE const*)src, srcSize)) - { - return ZSTD_rleCompressBlock(dst, dstCapacity, *(BYTE const*)src, srcSize, lastBlock); - } - /* Attempt superblock compression. - * - * Note that compressed size of ZSTD_compressSuperBlock() is not bound by the - * standard ZSTD_compressBound(). This is a problem, because even if we have - * space now, taking an extra byte now could cause us to run out of space later - * and violate ZSTD_compressBound(). - * - * Define blockBound(blockSize) = blockSize + ZSTD_blockHeaderSize. - * - * In order to respect ZSTD_compressBound() we must attempt to emit a raw - * uncompressed block in these cases: - * * cSize == 0: Return code for an uncompressed block. - * * cSize == dstSize_tooSmall: We may have expanded beyond blockBound(srcSize). - * ZSTD_noCompressBlock() will return dstSize_tooSmall if we are really out of - * output space. - * * cSize >= blockBound(srcSize): We have expanded the block too much so - * emit an uncompressed block. - */ - { - size_t const cSize = ZSTD_compressSuperBlock(zc, dst, dstCapacity, src, srcSize, lastBlock); - if (cSize != ERROR(dstSize_tooSmall)) { - size_t const maxCSize = srcSize - ZSTD_minGain(srcSize, zc->appliedParams.cParams.strategy); - FORWARD_IF_ERROR(cSize, "ZSTD_compressSuperBlock failed"); - if (cSize != 0 && cSize < maxCSize + ZSTD_blockHeaderSize) { - ZSTD_confirmRepcodesAndEntropyTables(zc); - return cSize; - } - } - } - } - - DEBUGLOG(6, "Resorting to ZSTD_noCompressBlock()"); - /* Superblock compression failed, attempt to emit a single no compress block. - * The decoder will be able to stream this block since it is uncompressed. - */ - return ZSTD_noCompressBlock(dst, dstCapacity, src, srcSize, lastBlock); -} - -static size_t ZSTD_compressBlock_targetCBlockSize(ZSTD_CCtx* zc, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - U32 lastBlock) -{ - size_t cSize = 0; - const size_t bss = ZSTD_buildSeqStore(zc, src, srcSize); - DEBUGLOG(5, "ZSTD_compressBlock_targetCBlockSize (dstCapacity=%u, dictLimit=%u, nextToUpdate=%u, srcSize=%zu)", - (unsigned)dstCapacity, (unsigned)zc->blockState.matchState.window.dictLimit, (unsigned)zc->blockState.matchState.nextToUpdate, srcSize); - FORWARD_IF_ERROR(bss, "ZSTD_buildSeqStore failed"); - - cSize = ZSTD_compressBlock_targetCBlockSize_body(zc, dst, dstCapacity, src, srcSize, bss, lastBlock); - FORWARD_IF_ERROR(cSize, "ZSTD_compressBlock_targetCBlockSize_body failed"); - - if (zc->blockState.prevCBlock->entropy.fse.offcode_repeatMode == FSE_repeat_valid) - zc->blockState.prevCBlock->entropy.fse.offcode_repeatMode = FSE_repeat_check; - - return cSize; -} - -static void ZSTD_overflowCorrectIfNeeded(ZSTD_matchState_t* ms, - ZSTD_cwksp* ws, - ZSTD_CCtx_params const* params, - void const* ip, - void const* iend) -{ - if (ZSTD_window_needOverflowCorrection(ms->window, iend)) { - U32 const maxDist = (U32)1 << params->cParams.windowLog; - U32 const cycleLog = ZSTD_cycleLog(params->cParams.chainLog, params->cParams.strategy); - U32 const correction = ZSTD_window_correctOverflow(&ms->window, cycleLog, maxDist, ip); - ZSTD_STATIC_ASSERT(ZSTD_CHAINLOG_MAX <= 30); - ZSTD_STATIC_ASSERT(ZSTD_WINDOWLOG_MAX_32 <= 30); - ZSTD_STATIC_ASSERT(ZSTD_WINDOWLOG_MAX <= 31); - ZSTD_cwksp_mark_tables_dirty(ws); - ZSTD_reduceIndex(ms, params, correction); - ZSTD_cwksp_mark_tables_clean(ws); - if (ms->nextToUpdate < correction) ms->nextToUpdate = 0; - else ms->nextToUpdate -= correction; - /* invalidate dictionaries on overflow correction */ - ms->loadedDictEnd = 0; - ms->dictMatchState = NULL; - } -} - -/*! ZSTD_compress_frameChunk() : -* Compress a chunk of data into one or multiple blocks. -* All blocks will be terminated, all input will be consumed. -* Function will issue an error if there is not enough `dstCapacity` to hold the compressed content. -* Frame is supposed already started (header already produced) -* @return : compressed size, or an error code -*/ -static size_t ZSTD_compress_frameChunk (ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - U32 lastFrameChunk) -{ - size_t blockSize = cctx->blockSize; - size_t remaining = srcSize; - const BYTE* ip = (const BYTE*)src; - BYTE* const ostart = (BYTE*)dst; - BYTE* op = ostart; - U32 const maxDist = (U32)1 << cctx->appliedParams.cParams.windowLog; - - assert(cctx->appliedParams.cParams.windowLog <= ZSTD_WINDOWLOG_MAX); - - DEBUGLOG(5, "ZSTD_compress_frameChunk (blockSize=%u)", (unsigned)blockSize); - if (cctx->appliedParams.fParams.checksumFlag && srcSize) - XXH64_update(&cctx->xxhState, src, srcSize); - - while (remaining) { - ZSTD_matchState_t* const ms = &cctx->blockState.matchState; - U32 const lastBlock = lastFrameChunk & (blockSize >= remaining); - - RETURN_ERROR_IF(dstCapacity < ZSTD_blockHeaderSize + MIN_CBLOCK_SIZE, - dstSize_tooSmall, - "not enough space to store compressed block"); - if (remaining < blockSize) blockSize = remaining; - - ZSTD_overflowCorrectIfNeeded( - ms, &cctx->workspace, &cctx->appliedParams, ip, ip + blockSize); - ZSTD_checkDictValidity(&ms->window, ip + blockSize, maxDist, &ms->loadedDictEnd, &ms->dictMatchState); - - /* Ensure hash/chain table insertion resumes no sooner than lowlimit */ - if (ms->nextToUpdate < ms->window.lowLimit) ms->nextToUpdate = ms->window.lowLimit; - - { size_t cSize; - if (ZSTD_useTargetCBlockSize(&cctx->appliedParams)) { - cSize = ZSTD_compressBlock_targetCBlockSize(cctx, op, dstCapacity, ip, blockSize, lastBlock); - FORWARD_IF_ERROR(cSize, "ZSTD_compressBlock_targetCBlockSize failed"); - assert(cSize > 0); - assert(cSize <= blockSize + ZSTD_blockHeaderSize); - } else { - cSize = ZSTD_compressBlock_internal(cctx, - op+ZSTD_blockHeaderSize, dstCapacity-ZSTD_blockHeaderSize, - ip, blockSize, 1 /* frame */); - FORWARD_IF_ERROR(cSize, "ZSTD_compressBlock_internal failed"); - - if (cSize == 0) { /* block is not compressible */ - cSize = ZSTD_noCompressBlock(op, dstCapacity, ip, blockSize, lastBlock); - FORWARD_IF_ERROR(cSize, "ZSTD_noCompressBlock failed"); - } else { - U32 const cBlockHeader = cSize == 1 ? - lastBlock + (((U32)bt_rle)<<1) + (U32)(blockSize << 3) : - lastBlock + (((U32)bt_compressed)<<1) + (U32)(cSize << 3); - MEM_writeLE24(op, cBlockHeader); - cSize += ZSTD_blockHeaderSize; - } - } - - - ip += blockSize; - assert(remaining >= blockSize); - remaining -= blockSize; - op += cSize; - assert(dstCapacity >= cSize); - dstCapacity -= cSize; - cctx->isFirstBlock = 0; - DEBUGLOG(5, "ZSTD_compress_frameChunk: adding a block of size %u", - (unsigned)cSize); - } } - - if (lastFrameChunk && (op>ostart)) cctx->stage = ZSTDcs_ending; - return (size_t)(op-ostart); -} - - -static size_t ZSTD_writeFrameHeader(void* dst, size_t dstCapacity, - const ZSTD_CCtx_params* params, U64 pledgedSrcSize, U32 dictID) -{ BYTE* const op = (BYTE*)dst; - U32 const dictIDSizeCodeLength = (dictID>0) + (dictID>=256) + (dictID>=65536); /* 0-3 */ - U32 const dictIDSizeCode = params->fParams.noDictIDFlag ? 0 : dictIDSizeCodeLength; /* 0-3 */ - U32 const checksumFlag = params->fParams.checksumFlag>0; - U32 const windowSize = (U32)1 << params->cParams.windowLog; - U32 const singleSegment = params->fParams.contentSizeFlag && (windowSize >= pledgedSrcSize); - BYTE const windowLogByte = (BYTE)((params->cParams.windowLog - ZSTD_WINDOWLOG_ABSOLUTEMIN) << 3); - U32 const fcsCode = params->fParams.contentSizeFlag ? - (pledgedSrcSize>=256) + (pledgedSrcSize>=65536+256) + (pledgedSrcSize>=0xFFFFFFFFU) : 0; /* 0-3 */ - BYTE const frameHeaderDescriptionByte = (BYTE)(dictIDSizeCode + (checksumFlag<<2) + (singleSegment<<5) + (fcsCode<<6) ); - size_t pos=0; - - assert(!(params->fParams.contentSizeFlag && pledgedSrcSize == ZSTD_CONTENTSIZE_UNKNOWN)); - RETURN_ERROR_IF(dstCapacity < ZSTD_FRAMEHEADERSIZE_MAX, dstSize_tooSmall, - "dst buf is too small to fit worst-case frame header size."); - DEBUGLOG(4, "ZSTD_writeFrameHeader : dictIDFlag : %u ; dictID : %u ; dictIDSizeCode : %u", - !params->fParams.noDictIDFlag, (unsigned)dictID, (unsigned)dictIDSizeCode); - - if (params->format == ZSTD_f_zstd1) { - MEM_writeLE32(dst, ZSTD_MAGICNUMBER); - pos = 4; - } - op[pos++] = frameHeaderDescriptionByte; - if (!singleSegment) op[pos++] = windowLogByte; - switch(dictIDSizeCode) - { - default: assert(0); /* impossible */ - case 0 : break; - case 1 : op[pos] = (BYTE)(dictID); pos++; break; - case 2 : MEM_writeLE16(op+pos, (U16)dictID); pos+=2; break; - case 3 : MEM_writeLE32(op+pos, dictID); pos+=4; break; - } - switch(fcsCode) - { - default: assert(0); /* impossible */ - case 0 : if (singleSegment) op[pos++] = (BYTE)(pledgedSrcSize); break; - case 1 : MEM_writeLE16(op+pos, (U16)(pledgedSrcSize-256)); pos+=2; break; - case 2 : MEM_writeLE32(op+pos, (U32)(pledgedSrcSize)); pos+=4; break; - case 3 : MEM_writeLE64(op+pos, (U64)(pledgedSrcSize)); pos+=8; break; - } - return pos; -} - -/* ZSTD_writeLastEmptyBlock() : - * output an empty Block with end-of-frame mark to complete a frame - * @return : size of data written into `dst` (== ZSTD_blockHeaderSize (defined in zstd_internal.h)) - * or an error code if `dstCapacity` is too small (stage != ZSTDcs_init, stage_wrong, - "wrong cctx stage"); - RETURN_ERROR_IF(cctx->appliedParams.ldmParams.enableLdm, - parameter_unsupported, - "incompatible with ldm"); - cctx->externSeqStore.seq = seq; - cctx->externSeqStore.size = nbSeq; - cctx->externSeqStore.capacity = nbSeq; - cctx->externSeqStore.pos = 0; - return 0; -} - - -static size_t ZSTD_compressContinue_internal (ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - U32 frame, U32 lastFrameChunk) -{ - ZSTD_matchState_t* const ms = &cctx->blockState.matchState; - size_t fhSize = 0; - - DEBUGLOG(5, "ZSTD_compressContinue_internal, stage: %u, srcSize: %u", - cctx->stage, (unsigned)srcSize); - RETURN_ERROR_IF(cctx->stage==ZSTDcs_created, stage_wrong, - "missing init (ZSTD_compressBegin)"); - - if (frame && (cctx->stage==ZSTDcs_init)) { - fhSize = ZSTD_writeFrameHeader(dst, dstCapacity, &cctx->appliedParams, - cctx->pledgedSrcSizePlusOne-1, cctx->dictID); - FORWARD_IF_ERROR(fhSize, "ZSTD_writeFrameHeader failed"); - assert(fhSize <= dstCapacity); - dstCapacity -= fhSize; - dst = (char*)dst + fhSize; - cctx->stage = ZSTDcs_ongoing; - } - - if (!srcSize) return fhSize; /* do not generate an empty block if no input */ - - if (!ZSTD_window_update(&ms->window, src, srcSize)) { - ms->nextToUpdate = ms->window.dictLimit; - } - if (cctx->appliedParams.ldmParams.enableLdm) { - ZSTD_window_update(&cctx->ldmState.window, src, srcSize); - } - - if (!frame) { - /* overflow check and correction for block mode */ - ZSTD_overflowCorrectIfNeeded( - ms, &cctx->workspace, &cctx->appliedParams, - src, (BYTE const*)src + srcSize); - } - - DEBUGLOG(5, "ZSTD_compressContinue_internal (blockSize=%u)", (unsigned)cctx->blockSize); - { size_t const cSize = frame ? - ZSTD_compress_frameChunk (cctx, dst, dstCapacity, src, srcSize, lastFrameChunk) : - ZSTD_compressBlock_internal (cctx, dst, dstCapacity, src, srcSize, 0 /* frame */); - FORWARD_IF_ERROR(cSize, "%s", frame ? "ZSTD_compress_frameChunk failed" : "ZSTD_compressBlock_internal failed"); - cctx->consumedSrcSize += srcSize; - cctx->producedCSize += (cSize + fhSize); - assert(!(cctx->appliedParams.fParams.contentSizeFlag && cctx->pledgedSrcSizePlusOne == 0)); - if (cctx->pledgedSrcSizePlusOne != 0) { /* control src size */ - ZSTD_STATIC_ASSERT(ZSTD_CONTENTSIZE_UNKNOWN == (unsigned long long)-1); - RETURN_ERROR_IF( - cctx->consumedSrcSize+1 > cctx->pledgedSrcSizePlusOne, - srcSize_wrong, - "error : pledgedSrcSize = %u, while realSrcSize >= %u", - (unsigned)cctx->pledgedSrcSizePlusOne-1, - (unsigned)cctx->consumedSrcSize); - } - return cSize + fhSize; - } -} - -size_t ZSTD_compressContinue (ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize) -{ - DEBUGLOG(5, "ZSTD_compressContinue (srcSize=%u)", (unsigned)srcSize); - return ZSTD_compressContinue_internal(cctx, dst, dstCapacity, src, srcSize, 1 /* frame mode */, 0 /* last chunk */); -} - - -size_t ZSTD_getBlockSize(const ZSTD_CCtx* cctx) -{ - ZSTD_compressionParameters const cParams = cctx->appliedParams.cParams; - assert(!ZSTD_checkCParams(cParams)); - return MIN (ZSTD_BLOCKSIZE_MAX, (U32)1 << cParams.windowLog); -} - -size_t ZSTD_compressBlock(ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize) -{ - DEBUGLOG(5, "ZSTD_compressBlock: srcSize = %u", (unsigned)srcSize); - { size_t const blockSizeMax = ZSTD_getBlockSize(cctx); - RETURN_ERROR_IF(srcSize > blockSizeMax, srcSize_wrong, "input is larger than a block"); } - - return ZSTD_compressContinue_internal(cctx, dst, dstCapacity, src, srcSize, 0 /* frame mode */, 0 /* last chunk */); -} - -/*! ZSTD_loadDictionaryContent() : - * @return : 0, or an error code - */ -static size_t ZSTD_loadDictionaryContent(ZSTD_matchState_t* ms, - ldmState_t* ls, - ZSTD_cwksp* ws, - ZSTD_CCtx_params const* params, - const void* src, size_t srcSize, - ZSTD_dictTableLoadMethod_e dtlm) -{ - const BYTE* ip = (const BYTE*) src; - const BYTE* const iend = ip + srcSize; - - ZSTD_window_update(&ms->window, src, srcSize); - ms->loadedDictEnd = params->forceWindow ? 0 : (U32)(iend - ms->window.base); - - if (params->ldmParams.enableLdm && ls != NULL) { - ZSTD_window_update(&ls->window, src, srcSize); - ls->loadedDictEnd = params->forceWindow ? 0 : (U32)(iend - ls->window.base); - } - - /* Assert that we the ms params match the params we're being given */ - ZSTD_assertEqualCParams(params->cParams, ms->cParams); - - if (srcSize <= HASH_READ_SIZE) return 0; - - while (iend - ip > HASH_READ_SIZE) { - size_t const remaining = (size_t)(iend - ip); - size_t const chunk = MIN(remaining, ZSTD_CHUNKSIZE_MAX); - const BYTE* const ichunk = ip + chunk; - - ZSTD_overflowCorrectIfNeeded(ms, ws, params, ip, ichunk); - - if (params->ldmParams.enableLdm && ls != NULL) - ZSTD_ldm_fillHashTable(ls, (const BYTE*)src, (const BYTE*)src + srcSize, ¶ms->ldmParams); - - switch(params->cParams.strategy) - { - case ZSTD_fast: - ZSTD_fillHashTable(ms, ichunk, dtlm); - break; - case ZSTD_dfast: - ZSTD_fillDoubleHashTable(ms, ichunk, dtlm); - break; - - case ZSTD_greedy: - case ZSTD_lazy: - case ZSTD_lazy2: - if (chunk >= HASH_READ_SIZE) - ZSTD_insertAndFindFirstIndex(ms, ichunk-HASH_READ_SIZE); - break; - - case ZSTD_btlazy2: /* we want the dictionary table fully sorted */ - case ZSTD_btopt: - case ZSTD_btultra: - case ZSTD_btultra2: - if (chunk >= HASH_READ_SIZE) - ZSTD_updateTree(ms, ichunk-HASH_READ_SIZE, ichunk); - break; - - default: - assert(0); /* not possible : not a valid strategy id */ - } - - ip = ichunk; - } - - ms->nextToUpdate = (U32)(iend - ms->window.base); - return 0; -} - - -/* Dictionaries that assign zero probability to symbols that show up causes problems - when FSE encoding. Refuse dictionaries that assign zero probability to symbols - that we may encounter during compression. - NOTE: This behavior is not standard and could be improved in the future. */ -static size_t ZSTD_checkDictNCount(short* normalizedCounter, unsigned dictMaxSymbolValue, unsigned maxSymbolValue) { - U32 s; - RETURN_ERROR_IF(dictMaxSymbolValue < maxSymbolValue, dictionary_corrupted, "dict fse tables don't have all symbols"); - for (s = 0; s <= maxSymbolValue; ++s) { - RETURN_ERROR_IF(normalizedCounter[s] == 0, dictionary_corrupted, "dict fse tables don't have all symbols"); - } - return 0; -} - -size_t ZSTD_loadCEntropy(ZSTD_compressedBlockState_t* bs, void* workspace, - short* offcodeNCount, unsigned* offcodeMaxValue, - const void* const dict, size_t dictSize) -{ - const BYTE* dictPtr = (const BYTE*)dict; /* skip magic num and dict ID */ - const BYTE* const dictEnd = dictPtr + dictSize; - dictPtr += 8; - bs->entropy.huf.repeatMode = HUF_repeat_check; - - { unsigned maxSymbolValue = 255; - unsigned hasZeroWeights = 1; - size_t const hufHeaderSize = HUF_readCTable((HUF_CElt*)bs->entropy.huf.CTable, &maxSymbolValue, dictPtr, - dictEnd-dictPtr, &hasZeroWeights); - - /* We only set the loaded table as valid if it contains all non-zero - * weights. Otherwise, we set it to check */ - if (!hasZeroWeights) - bs->entropy.huf.repeatMode = HUF_repeat_valid; - - RETURN_ERROR_IF(HUF_isError(hufHeaderSize), dictionary_corrupted, ""); - RETURN_ERROR_IF(maxSymbolValue < 255, dictionary_corrupted, ""); - dictPtr += hufHeaderSize; - } - - { unsigned offcodeLog; - size_t const offcodeHeaderSize = FSE_readNCount(offcodeNCount, offcodeMaxValue, &offcodeLog, dictPtr, dictEnd-dictPtr); - RETURN_ERROR_IF(FSE_isError(offcodeHeaderSize), dictionary_corrupted, ""); - RETURN_ERROR_IF(offcodeLog > OffFSELog, dictionary_corrupted, ""); - /* Defer checking offcodeMaxValue because we need to know the size of the dictionary content */ - /* fill all offset symbols to avoid garbage at end of table */ - RETURN_ERROR_IF(FSE_isError(FSE_buildCTable_wksp( - bs->entropy.fse.offcodeCTable, - offcodeNCount, MaxOff, offcodeLog, - workspace, HUF_WORKSPACE_SIZE)), - dictionary_corrupted, ""); - dictPtr += offcodeHeaderSize; - } - - { short matchlengthNCount[MaxML+1]; - unsigned matchlengthMaxValue = MaxML, matchlengthLog; - size_t const matchlengthHeaderSize = FSE_readNCount(matchlengthNCount, &matchlengthMaxValue, &matchlengthLog, dictPtr, dictEnd-dictPtr); - RETURN_ERROR_IF(FSE_isError(matchlengthHeaderSize), dictionary_corrupted, ""); - RETURN_ERROR_IF(matchlengthLog > MLFSELog, dictionary_corrupted, ""); - /* Every match length code must have non-zero probability */ - FORWARD_IF_ERROR( ZSTD_checkDictNCount(matchlengthNCount, matchlengthMaxValue, MaxML), ""); - RETURN_ERROR_IF(FSE_isError(FSE_buildCTable_wksp( - bs->entropy.fse.matchlengthCTable, - matchlengthNCount, matchlengthMaxValue, matchlengthLog, - workspace, HUF_WORKSPACE_SIZE)), - dictionary_corrupted, ""); - dictPtr += matchlengthHeaderSize; - } - - { short litlengthNCount[MaxLL+1]; - unsigned litlengthMaxValue = MaxLL, litlengthLog; - size_t const litlengthHeaderSize = FSE_readNCount(litlengthNCount, &litlengthMaxValue, &litlengthLog, dictPtr, dictEnd-dictPtr); - RETURN_ERROR_IF(FSE_isError(litlengthHeaderSize), dictionary_corrupted, ""); - RETURN_ERROR_IF(litlengthLog > LLFSELog, dictionary_corrupted, ""); - /* Every literal length code must have non-zero probability */ - FORWARD_IF_ERROR( ZSTD_checkDictNCount(litlengthNCount, litlengthMaxValue, MaxLL), ""); - RETURN_ERROR_IF(FSE_isError(FSE_buildCTable_wksp( - bs->entropy.fse.litlengthCTable, - litlengthNCount, litlengthMaxValue, litlengthLog, - workspace, HUF_WORKSPACE_SIZE)), - dictionary_corrupted, ""); - dictPtr += litlengthHeaderSize; - } - - RETURN_ERROR_IF(dictPtr+12 > dictEnd, dictionary_corrupted, ""); - bs->rep[0] = MEM_readLE32(dictPtr+0); - bs->rep[1] = MEM_readLE32(dictPtr+4); - bs->rep[2] = MEM_readLE32(dictPtr+8); - dictPtr += 12; - - return dictPtr - (const BYTE*)dict; -} - -/* Dictionary format : - * See : - * https://github.com/facebook/zstd/blob/master/doc/zstd_compression_format.md#dictionary-format - */ -/*! ZSTD_loadZstdDictionary() : - * @return : dictID, or an error code - * assumptions : magic number supposed already checked - * dictSize supposed >= 8 - */ -static size_t ZSTD_loadZstdDictionary(ZSTD_compressedBlockState_t* bs, - ZSTD_matchState_t* ms, - ZSTD_cwksp* ws, - ZSTD_CCtx_params const* params, - const void* dict, size_t dictSize, - ZSTD_dictTableLoadMethod_e dtlm, - void* workspace) -{ - const BYTE* dictPtr = (const BYTE*)dict; - const BYTE* const dictEnd = dictPtr + dictSize; - short offcodeNCount[MaxOff+1]; - unsigned offcodeMaxValue = MaxOff; - size_t dictID; - size_t eSize; - - ZSTD_STATIC_ASSERT(HUF_WORKSPACE_SIZE >= (1<= 8); - assert(MEM_readLE32(dictPtr) == ZSTD_MAGIC_DICTIONARY); - - dictID = params->fParams.noDictIDFlag ? 0 : MEM_readLE32(dictPtr + 4 /* skip magic number */ ); - eSize = ZSTD_loadCEntropy(bs, workspace, offcodeNCount, &offcodeMaxValue, dict, dictSize); - FORWARD_IF_ERROR(eSize, "ZSTD_loadCEntropy failed"); - dictPtr += eSize; - - { size_t const dictContentSize = (size_t)(dictEnd - dictPtr); - U32 offcodeMax = MaxOff; - if (dictContentSize <= ((U32)-1) - 128 KB) { - U32 const maxOffset = (U32)dictContentSize + 128 KB; /* The maximum offset that must be supported */ - offcodeMax = ZSTD_highbit32(maxOffset); /* Calculate minimum offset code required to represent maxOffset */ - } - /* All offset values <= dictContentSize + 128 KB must be representable */ - FORWARD_IF_ERROR(ZSTD_checkDictNCount(offcodeNCount, offcodeMaxValue, MIN(offcodeMax, MaxOff)), ""); - /* All repCodes must be <= dictContentSize and != 0*/ - { U32 u; - for (u=0; u<3; u++) { - RETURN_ERROR_IF(bs->rep[u] == 0, dictionary_corrupted, ""); - RETURN_ERROR_IF(bs->rep[u] > dictContentSize, dictionary_corrupted, ""); - } } - - bs->entropy.fse.offcode_repeatMode = FSE_repeat_valid; - bs->entropy.fse.matchlength_repeatMode = FSE_repeat_valid; - bs->entropy.fse.litlength_repeatMode = FSE_repeat_valid; - FORWARD_IF_ERROR(ZSTD_loadDictionaryContent( - ms, NULL, ws, params, dictPtr, dictContentSize, dtlm), ""); - return dictID; - } -} - -/** ZSTD_compress_insertDictionary() : -* @return : dictID, or an error code */ -static size_t -ZSTD_compress_insertDictionary(ZSTD_compressedBlockState_t* bs, - ZSTD_matchState_t* ms, - ldmState_t* ls, - ZSTD_cwksp* ws, - const ZSTD_CCtx_params* params, - const void* dict, size_t dictSize, - ZSTD_dictContentType_e dictContentType, - ZSTD_dictTableLoadMethod_e dtlm, - void* workspace) -{ - DEBUGLOG(4, "ZSTD_compress_insertDictionary (dictSize=%u)", (U32)dictSize); - if ((dict==NULL) || (dictSize<8)) { - RETURN_ERROR_IF(dictContentType == ZSTD_dct_fullDict, dictionary_wrong, ""); - return 0; - } - - ZSTD_reset_compressedBlockState(bs); - - /* dict restricted modes */ - if (dictContentType == ZSTD_dct_rawContent) - return ZSTD_loadDictionaryContent(ms, ls, ws, params, dict, dictSize, dtlm); - - if (MEM_readLE32(dict) != ZSTD_MAGIC_DICTIONARY) { - if (dictContentType == ZSTD_dct_auto) { - DEBUGLOG(4, "raw content dictionary detected"); - return ZSTD_loadDictionaryContent( - ms, ls, ws, params, dict, dictSize, dtlm); - } - RETURN_ERROR_IF(dictContentType == ZSTD_dct_fullDict, dictionary_wrong, ""); - assert(0); /* impossible */ - } - - /* dict as full zstd dictionary */ - return ZSTD_loadZstdDictionary( - bs, ms, ws, params, dict, dictSize, dtlm, workspace); -} - -#define ZSTD_USE_CDICT_PARAMS_SRCSIZE_CUTOFF (128 KB) -#define ZSTD_USE_CDICT_PARAMS_DICTSIZE_MULTIPLIER (6) - -/*! ZSTD_compressBegin_internal() : - * @return : 0, or an error code */ -static size_t ZSTD_compressBegin_internal(ZSTD_CCtx* cctx, - const void* dict, size_t dictSize, - ZSTD_dictContentType_e dictContentType, - ZSTD_dictTableLoadMethod_e dtlm, - const ZSTD_CDict* cdict, - const ZSTD_CCtx_params* params, U64 pledgedSrcSize, - ZSTD_buffered_policy_e zbuff) -{ - DEBUGLOG(4, "ZSTD_compressBegin_internal: wlog=%u", params->cParams.windowLog); - /* params are supposed to be fully validated at this point */ - assert(!ZSTD_isError(ZSTD_checkCParams(params->cParams))); - assert(!((dict) && (cdict))); /* either dict or cdict, not both */ - if ( (cdict) - && (cdict->dictContentSize > 0) - && ( pledgedSrcSize < ZSTD_USE_CDICT_PARAMS_SRCSIZE_CUTOFF - || pledgedSrcSize < cdict->dictContentSize * ZSTD_USE_CDICT_PARAMS_DICTSIZE_MULTIPLIER - || pledgedSrcSize == ZSTD_CONTENTSIZE_UNKNOWN - || cdict->compressionLevel == 0) - && (params->attachDictPref != ZSTD_dictForceLoad) ) { - return ZSTD_resetCCtx_usingCDict(cctx, cdict, params, pledgedSrcSize, zbuff); - } - - FORWARD_IF_ERROR( ZSTD_resetCCtx_internal(cctx, *params, pledgedSrcSize, - ZSTDcrp_makeClean, zbuff) , ""); - { size_t const dictID = cdict ? - ZSTD_compress_insertDictionary( - cctx->blockState.prevCBlock, &cctx->blockState.matchState, - &cctx->ldmState, &cctx->workspace, &cctx->appliedParams, cdict->dictContent, - cdict->dictContentSize, dictContentType, dtlm, - cctx->entropyWorkspace) - : ZSTD_compress_insertDictionary( - cctx->blockState.prevCBlock, &cctx->blockState.matchState, - &cctx->ldmState, &cctx->workspace, &cctx->appliedParams, dict, dictSize, - dictContentType, dtlm, cctx->entropyWorkspace); - FORWARD_IF_ERROR(dictID, "ZSTD_compress_insertDictionary failed"); - assert(dictID <= UINT_MAX); - cctx->dictID = (U32)dictID; - } - return 0; -} - -size_t ZSTD_compressBegin_advanced_internal(ZSTD_CCtx* cctx, - const void* dict, size_t dictSize, - ZSTD_dictContentType_e dictContentType, - ZSTD_dictTableLoadMethod_e dtlm, - const ZSTD_CDict* cdict, - const ZSTD_CCtx_params* params, - unsigned long long pledgedSrcSize) -{ - DEBUGLOG(4, "ZSTD_compressBegin_advanced_internal: wlog=%u", params->cParams.windowLog); - /* compression parameters verification and optimization */ - FORWARD_IF_ERROR( ZSTD_checkCParams(params->cParams) , ""); - return ZSTD_compressBegin_internal(cctx, - dict, dictSize, dictContentType, dtlm, - cdict, - params, pledgedSrcSize, - ZSTDb_not_buffered); -} - -/*! ZSTD_compressBegin_advanced() : -* @return : 0, or an error code */ -size_t ZSTD_compressBegin_advanced(ZSTD_CCtx* cctx, - const void* dict, size_t dictSize, - ZSTD_parameters params, unsigned long long pledgedSrcSize) -{ - ZSTD_CCtx_params const cctxParams = - ZSTD_assignParamsToCCtxParams(&cctx->requestedParams, ¶ms); - return ZSTD_compressBegin_advanced_internal(cctx, - dict, dictSize, ZSTD_dct_auto, ZSTD_dtlm_fast, - NULL /*cdict*/, - &cctxParams, pledgedSrcSize); -} - -size_t ZSTD_compressBegin_usingDict(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, int compressionLevel) -{ - ZSTD_parameters const params = ZSTD_getParams_internal(compressionLevel, ZSTD_CONTENTSIZE_UNKNOWN, dictSize); - ZSTD_CCtx_params const cctxParams = - ZSTD_assignParamsToCCtxParams(&cctx->requestedParams, ¶ms); - DEBUGLOG(4, "ZSTD_compressBegin_usingDict (dictSize=%u)", (unsigned)dictSize); - return ZSTD_compressBegin_internal(cctx, dict, dictSize, ZSTD_dct_auto, ZSTD_dtlm_fast, NULL, - &cctxParams, ZSTD_CONTENTSIZE_UNKNOWN, ZSTDb_not_buffered); -} - -size_t ZSTD_compressBegin(ZSTD_CCtx* cctx, int compressionLevel) -{ - return ZSTD_compressBegin_usingDict(cctx, NULL, 0, compressionLevel); -} - - -/*! ZSTD_writeEpilogue() : -* Ends a frame. -* @return : nb of bytes written into dst (or an error code) */ -static size_t ZSTD_writeEpilogue(ZSTD_CCtx* cctx, void* dst, size_t dstCapacity) -{ - BYTE* const ostart = (BYTE*)dst; - BYTE* op = ostart; - size_t fhSize = 0; - - DEBUGLOG(4, "ZSTD_writeEpilogue"); - RETURN_ERROR_IF(cctx->stage == ZSTDcs_created, stage_wrong, "init missing"); - - /* special case : empty frame */ - if (cctx->stage == ZSTDcs_init) { - fhSize = ZSTD_writeFrameHeader(dst, dstCapacity, &cctx->appliedParams, 0, 0); - FORWARD_IF_ERROR(fhSize, "ZSTD_writeFrameHeader failed"); - dstCapacity -= fhSize; - op += fhSize; - cctx->stage = ZSTDcs_ongoing; - } - - if (cctx->stage != ZSTDcs_ending) { - /* write one last empty block, make it the "last" block */ - U32 const cBlockHeader24 = 1 /* last block */ + (((U32)bt_raw)<<1) + 0; - RETURN_ERROR_IF(dstCapacity<4, dstSize_tooSmall, "no room for epilogue"); - MEM_writeLE32(op, cBlockHeader24); - op += ZSTD_blockHeaderSize; - dstCapacity -= ZSTD_blockHeaderSize; - } - - if (cctx->appliedParams.fParams.checksumFlag) { - U32 const checksum = (U32) XXH64_digest(&cctx->xxhState); - RETURN_ERROR_IF(dstCapacity<4, dstSize_tooSmall, "no room for checksum"); - DEBUGLOG(4, "ZSTD_writeEpilogue: write checksum : %08X", (unsigned)checksum); - MEM_writeLE32(op, checksum); - op += 4; - } - - cctx->stage = ZSTDcs_created; /* return to "created but no init" status */ - return op-ostart; -} - -size_t ZSTD_compressEnd (ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize) -{ - size_t endResult; - size_t const cSize = ZSTD_compressContinue_internal(cctx, - dst, dstCapacity, src, srcSize, - 1 /* frame mode */, 1 /* last chunk */); - FORWARD_IF_ERROR(cSize, "ZSTD_compressContinue_internal failed"); - endResult = ZSTD_writeEpilogue(cctx, (char*)dst + cSize, dstCapacity-cSize); - FORWARD_IF_ERROR(endResult, "ZSTD_writeEpilogue failed"); - assert(!(cctx->appliedParams.fParams.contentSizeFlag && cctx->pledgedSrcSizePlusOne == 0)); - if (cctx->pledgedSrcSizePlusOne != 0) { /* control src size */ - ZSTD_STATIC_ASSERT(ZSTD_CONTENTSIZE_UNKNOWN == (unsigned long long)-1); - DEBUGLOG(4, "end of frame : controlling src size"); - RETURN_ERROR_IF( - cctx->pledgedSrcSizePlusOne != cctx->consumedSrcSize+1, - srcSize_wrong, - "error : pledgedSrcSize = %u, while realSrcSize = %u", - (unsigned)cctx->pledgedSrcSizePlusOne-1, - (unsigned)cctx->consumedSrcSize); - } - return cSize + endResult; -} - - -static size_t ZSTD_compress_internal (ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const void* dict,size_t dictSize, - const ZSTD_parameters* params) -{ - ZSTD_CCtx_params const cctxParams = - ZSTD_assignParamsToCCtxParams(&cctx->requestedParams, params); - DEBUGLOG(4, "ZSTD_compress_internal"); - return ZSTD_compress_advanced_internal(cctx, - dst, dstCapacity, - src, srcSize, - dict, dictSize, - &cctxParams); -} - -size_t ZSTD_compress_advanced (ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const void* dict,size_t dictSize, - ZSTD_parameters params) -{ - DEBUGLOG(4, "ZSTD_compress_advanced"); - FORWARD_IF_ERROR(ZSTD_checkCParams(params.cParams), ""); - return ZSTD_compress_internal(cctx, - dst, dstCapacity, - src, srcSize, - dict, dictSize, - ¶ms); -} - -/* Internal */ -size_t ZSTD_compress_advanced_internal( - ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const void* dict,size_t dictSize, - const ZSTD_CCtx_params* params) -{ - DEBUGLOG(4, "ZSTD_compress_advanced_internal (srcSize:%u)", (unsigned)srcSize); - FORWARD_IF_ERROR( ZSTD_compressBegin_internal(cctx, - dict, dictSize, ZSTD_dct_auto, ZSTD_dtlm_fast, NULL, - params, srcSize, ZSTDb_not_buffered) , ""); - return ZSTD_compressEnd(cctx, dst, dstCapacity, src, srcSize); -} - -size_t ZSTD_compress_usingDict(ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const void* dict, size_t dictSize, - int compressionLevel) -{ - ZSTD_parameters const params = ZSTD_getParams_internal(compressionLevel, srcSize, dict ? dictSize : 0); - ZSTD_CCtx_params cctxParams = ZSTD_assignParamsToCCtxParams(&cctx->requestedParams, ¶ms); - DEBUGLOG(4, "ZSTD_compress_usingDict (srcSize=%u)", (unsigned)srcSize); - assert(params.fParams.contentSizeFlag == 1); - return ZSTD_compress_advanced_internal(cctx, dst, dstCapacity, src, srcSize, dict, dictSize, &cctxParams); -} - -size_t ZSTD_compressCCtx(ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - int compressionLevel) -{ - DEBUGLOG(4, "ZSTD_compressCCtx (srcSize=%u)", (unsigned)srcSize); - assert(cctx != NULL); - return ZSTD_compress_usingDict(cctx, dst, dstCapacity, src, srcSize, NULL, 0, compressionLevel); -} - -size_t ZSTD_compress(void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - int compressionLevel) -{ - size_t result; - ZSTD_CCtx ctxBody; - ZSTD_initCCtx(&ctxBody, ZSTD_defaultCMem); - result = ZSTD_compressCCtx(&ctxBody, dst, dstCapacity, src, srcSize, compressionLevel); - ZSTD_freeCCtxContent(&ctxBody); /* can't free ctxBody itself, as it's on stack; free only heap content */ - return result; -} - - -/* ===== Dictionary API ===== */ - -/*! ZSTD_estimateCDictSize_advanced() : - * Estimate amount of memory that will be needed to create a dictionary with following arguments */ -size_t ZSTD_estimateCDictSize_advanced( - size_t dictSize, ZSTD_compressionParameters cParams, - ZSTD_dictLoadMethod_e dictLoadMethod) -{ - DEBUGLOG(5, "sizeof(ZSTD_CDict) : %u", (unsigned)sizeof(ZSTD_CDict)); - return ZSTD_cwksp_alloc_size(sizeof(ZSTD_CDict)) - + ZSTD_cwksp_alloc_size(HUF_WORKSPACE_SIZE) - + ZSTD_sizeof_matchState(&cParams, /* forCCtx */ 0) - + (dictLoadMethod == ZSTD_dlm_byRef ? 0 - : ZSTD_cwksp_alloc_size(ZSTD_cwksp_align(dictSize, sizeof(void *)))); -} - -size_t ZSTD_estimateCDictSize(size_t dictSize, int compressionLevel) -{ - ZSTD_compressionParameters const cParams = ZSTD_getCParams_internal(compressionLevel, ZSTD_CONTENTSIZE_UNKNOWN, dictSize); - return ZSTD_estimateCDictSize_advanced(dictSize, cParams, ZSTD_dlm_byCopy); -} - -size_t ZSTD_sizeof_CDict(const ZSTD_CDict* cdict) -{ - if (cdict==NULL) return 0; /* support sizeof on NULL */ - DEBUGLOG(5, "sizeof(*cdict) : %u", (unsigned)sizeof(*cdict)); - /* cdict may be in the workspace */ - return (cdict->workspace.workspace == cdict ? 0 : sizeof(*cdict)) - + ZSTD_cwksp_sizeof(&cdict->workspace); -} - -static size_t ZSTD_initCDict_internal( - ZSTD_CDict* cdict, - const void* dictBuffer, size_t dictSize, - ZSTD_dictLoadMethod_e dictLoadMethod, - ZSTD_dictContentType_e dictContentType, - ZSTD_compressionParameters cParams) -{ - DEBUGLOG(3, "ZSTD_initCDict_internal (dictContentType:%u)", (unsigned)dictContentType); - assert(!ZSTD_checkCParams(cParams)); - cdict->matchState.cParams = cParams; - if ((dictLoadMethod == ZSTD_dlm_byRef) || (!dictBuffer) || (!dictSize)) { - cdict->dictContent = dictBuffer; - } else { - void *internalBuffer = ZSTD_cwksp_reserve_object(&cdict->workspace, ZSTD_cwksp_align(dictSize, sizeof(void*))); - RETURN_ERROR_IF(!internalBuffer, memory_allocation, "NULL pointer!"); - cdict->dictContent = internalBuffer; - memcpy(internalBuffer, dictBuffer, dictSize); - } - cdict->dictContentSize = dictSize; - - cdict->entropyWorkspace = (U32*)ZSTD_cwksp_reserve_object(&cdict->workspace, HUF_WORKSPACE_SIZE); - - - /* Reset the state to no dictionary */ - ZSTD_reset_compressedBlockState(&cdict->cBlockState); - FORWARD_IF_ERROR(ZSTD_reset_matchState( - &cdict->matchState, - &cdict->workspace, - &cParams, - ZSTDcrp_makeClean, - ZSTDirp_reset, - ZSTD_resetTarget_CDict), ""); - /* (Maybe) load the dictionary - * Skips loading the dictionary if it is < 8 bytes. - */ - { ZSTD_CCtx_params params; - memset(¶ms, 0, sizeof(params)); - params.compressionLevel = ZSTD_CLEVEL_DEFAULT; - params.fParams.contentSizeFlag = 1; - params.cParams = cParams; - { size_t const dictID = ZSTD_compress_insertDictionary( - &cdict->cBlockState, &cdict->matchState, NULL, &cdict->workspace, - ¶ms, cdict->dictContent, cdict->dictContentSize, - dictContentType, ZSTD_dtlm_full, cdict->entropyWorkspace); - FORWARD_IF_ERROR(dictID, "ZSTD_compress_insertDictionary failed"); - assert(dictID <= (size_t)(U32)-1); - cdict->dictID = (U32)dictID; - } - } - - return 0; -} - -ZSTD_CDict* ZSTD_createCDict_advanced(const void* dictBuffer, size_t dictSize, - ZSTD_dictLoadMethod_e dictLoadMethod, - ZSTD_dictContentType_e dictContentType, - ZSTD_compressionParameters cParams, ZSTD_customMem customMem) -{ - DEBUGLOG(3, "ZSTD_createCDict_advanced, mode %u", (unsigned)dictContentType); - if (!customMem.customAlloc ^ !customMem.customFree) return NULL; - - { size_t const workspaceSize = - ZSTD_cwksp_alloc_size(sizeof(ZSTD_CDict)) + - ZSTD_cwksp_alloc_size(HUF_WORKSPACE_SIZE) + - ZSTD_sizeof_matchState(&cParams, /* forCCtx */ 0) + - (dictLoadMethod == ZSTD_dlm_byRef ? 0 - : ZSTD_cwksp_alloc_size(ZSTD_cwksp_align(dictSize, sizeof(void*)))); - void* const workspace = ZSTD_malloc(workspaceSize, customMem); - ZSTD_cwksp ws; - ZSTD_CDict* cdict; - - if (!workspace) { - ZSTD_free(workspace, customMem); - return NULL; - } - - ZSTD_cwksp_init(&ws, workspace, workspaceSize); - - cdict = (ZSTD_CDict*)ZSTD_cwksp_reserve_object(&ws, sizeof(ZSTD_CDict)); - assert(cdict != NULL); - ZSTD_cwksp_move(&cdict->workspace, &ws); - cdict->customMem = customMem; - cdict->compressionLevel = 0; /* signals advanced API usage */ - - if (ZSTD_isError( ZSTD_initCDict_internal(cdict, - dictBuffer, dictSize, - dictLoadMethod, dictContentType, - cParams) )) { - ZSTD_freeCDict(cdict); - return NULL; - } - - return cdict; - } -} - -ZSTD_CDict* ZSTD_createCDict(const void* dict, size_t dictSize, int compressionLevel) -{ - ZSTD_compressionParameters cParams = ZSTD_getCParams_internal(compressionLevel, ZSTD_CONTENTSIZE_UNKNOWN, dictSize); - ZSTD_CDict* cdict = ZSTD_createCDict_advanced(dict, dictSize, - ZSTD_dlm_byCopy, ZSTD_dct_auto, - cParams, ZSTD_defaultCMem); - if (cdict) - cdict->compressionLevel = compressionLevel == 0 ? ZSTD_CLEVEL_DEFAULT : compressionLevel; - return cdict; -} - -ZSTD_CDict* ZSTD_createCDict_byReference(const void* dict, size_t dictSize, int compressionLevel) -{ - ZSTD_compressionParameters cParams = ZSTD_getCParams_internal(compressionLevel, ZSTD_CONTENTSIZE_UNKNOWN, dictSize); - return ZSTD_createCDict_advanced(dict, dictSize, - ZSTD_dlm_byRef, ZSTD_dct_auto, - cParams, ZSTD_defaultCMem); -} - -size_t ZSTD_freeCDict(ZSTD_CDict* cdict) -{ - if (cdict==NULL) return 0; /* support free on NULL */ - { ZSTD_customMem const cMem = cdict->customMem; - int cdictInWorkspace = ZSTD_cwksp_owns_buffer(&cdict->workspace, cdict); - ZSTD_cwksp_free(&cdict->workspace, cMem); - if (!cdictInWorkspace) { - ZSTD_free(cdict, cMem); - } - return 0; - } -} - -/*! ZSTD_initStaticCDict_advanced() : - * Generate a digested dictionary in provided memory area. - * workspace: The memory area to emplace the dictionary into. - * Provided pointer must 8-bytes aligned. - * It must outlive dictionary usage. - * workspaceSize: Use ZSTD_estimateCDictSize() - * to determine how large workspace must be. - * cParams : use ZSTD_getCParams() to transform a compression level - * into its relevants cParams. - * @return : pointer to ZSTD_CDict*, or NULL if error (size too small) - * Note : there is no corresponding "free" function. - * Since workspace was allocated externally, it must be freed externally. - */ -const ZSTD_CDict* ZSTD_initStaticCDict( - void* workspace, size_t workspaceSize, - const void* dict, size_t dictSize, - ZSTD_dictLoadMethod_e dictLoadMethod, - ZSTD_dictContentType_e dictContentType, - ZSTD_compressionParameters cParams) -{ - size_t const matchStateSize = ZSTD_sizeof_matchState(&cParams, /* forCCtx */ 0); - size_t const neededSize = ZSTD_cwksp_alloc_size(sizeof(ZSTD_CDict)) - + (dictLoadMethod == ZSTD_dlm_byRef ? 0 - : ZSTD_cwksp_alloc_size(ZSTD_cwksp_align(dictSize, sizeof(void*)))) - + ZSTD_cwksp_alloc_size(HUF_WORKSPACE_SIZE) - + matchStateSize; - ZSTD_CDict* cdict; - - if ((size_t)workspace & 7) return NULL; /* 8-aligned */ - - { - ZSTD_cwksp ws; - ZSTD_cwksp_init(&ws, workspace, workspaceSize); - cdict = (ZSTD_CDict*)ZSTD_cwksp_reserve_object(&ws, sizeof(ZSTD_CDict)); - if (cdict == NULL) return NULL; - ZSTD_cwksp_move(&cdict->workspace, &ws); - } - - DEBUGLOG(4, "(workspaceSize < neededSize) : (%u < %u) => %u", - (unsigned)workspaceSize, (unsigned)neededSize, (unsigned)(workspaceSize < neededSize)); - if (workspaceSize < neededSize) return NULL; - - if (ZSTD_isError( ZSTD_initCDict_internal(cdict, - dict, dictSize, - dictLoadMethod, dictContentType, - cParams) )) - return NULL; - - return cdict; -} - -ZSTD_compressionParameters ZSTD_getCParamsFromCDict(const ZSTD_CDict* cdict) -{ - assert(cdict != NULL); - return cdict->matchState.cParams; -} - -/* ZSTD_compressBegin_usingCDict_advanced() : - * cdict must be != NULL */ -size_t ZSTD_compressBegin_usingCDict_advanced( - ZSTD_CCtx* const cctx, const ZSTD_CDict* const cdict, - ZSTD_frameParameters const fParams, unsigned long long const pledgedSrcSize) -{ - DEBUGLOG(4, "ZSTD_compressBegin_usingCDict_advanced"); - RETURN_ERROR_IF(cdict==NULL, dictionary_wrong, "NULL pointer!"); - { ZSTD_CCtx_params params = cctx->requestedParams; - params.cParams = ( pledgedSrcSize < ZSTD_USE_CDICT_PARAMS_SRCSIZE_CUTOFF - || pledgedSrcSize < cdict->dictContentSize * ZSTD_USE_CDICT_PARAMS_DICTSIZE_MULTIPLIER - || pledgedSrcSize == ZSTD_CONTENTSIZE_UNKNOWN - || cdict->compressionLevel == 0 ) - && (params.attachDictPref != ZSTD_dictForceLoad) ? - ZSTD_getCParamsFromCDict(cdict) - : ZSTD_getCParams(cdict->compressionLevel, - pledgedSrcSize, - cdict->dictContentSize); - /* Increase window log to fit the entire dictionary and source if the - * source size is known. Limit the increase to 19, which is the - * window log for compression level 1 with the largest source size. - */ - if (pledgedSrcSize != ZSTD_CONTENTSIZE_UNKNOWN) { - U32 const limitedSrcSize = (U32)MIN(pledgedSrcSize, 1U << 19); - U32 const limitedSrcLog = limitedSrcSize > 1 ? ZSTD_highbit32(limitedSrcSize - 1) + 1 : 1; - params.cParams.windowLog = MAX(params.cParams.windowLog, limitedSrcLog); - } - params.fParams = fParams; - return ZSTD_compressBegin_internal(cctx, - NULL, 0, ZSTD_dct_auto, ZSTD_dtlm_fast, - cdict, - ¶ms, pledgedSrcSize, - ZSTDb_not_buffered); - } -} - -/* ZSTD_compressBegin_usingCDict() : - * pledgedSrcSize=0 means "unknown" - * if pledgedSrcSize>0, it will enable contentSizeFlag */ -size_t ZSTD_compressBegin_usingCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict) -{ - ZSTD_frameParameters const fParams = { 0 /*content*/, 0 /*checksum*/, 0 /*noDictID*/ }; - DEBUGLOG(4, "ZSTD_compressBegin_usingCDict : dictIDFlag == %u", !fParams.noDictIDFlag); - return ZSTD_compressBegin_usingCDict_advanced(cctx, cdict, fParams, ZSTD_CONTENTSIZE_UNKNOWN); -} - -size_t ZSTD_compress_usingCDict_advanced(ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const ZSTD_CDict* cdict, ZSTD_frameParameters fParams) -{ - FORWARD_IF_ERROR(ZSTD_compressBegin_usingCDict_advanced(cctx, cdict, fParams, srcSize), ""); /* will check if cdict != NULL */ - return ZSTD_compressEnd(cctx, dst, dstCapacity, src, srcSize); -} - -/*! ZSTD_compress_usingCDict() : - * Compression using a digested Dictionary. - * Faster startup than ZSTD_compress_usingDict(), recommended when same dictionary is used multiple times. - * Note that compression parameters are decided at CDict creation time - * while frame parameters are hardcoded */ -size_t ZSTD_compress_usingCDict(ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const ZSTD_CDict* cdict) -{ - ZSTD_frameParameters const fParams = { 1 /*content*/, 0 /*checksum*/, 0 /*noDictID*/ }; - return ZSTD_compress_usingCDict_advanced(cctx, dst, dstCapacity, src, srcSize, cdict, fParams); -} - - - -/* ****************************************************************** -* Streaming -********************************************************************/ - -ZSTD_CStream* ZSTD_createCStream(void) -{ - DEBUGLOG(3, "ZSTD_createCStream"); - return ZSTD_createCStream_advanced(ZSTD_defaultCMem); -} - -ZSTD_CStream* ZSTD_initStaticCStream(void *workspace, size_t workspaceSize) -{ - return ZSTD_initStaticCCtx(workspace, workspaceSize); -} - -ZSTD_CStream* ZSTD_createCStream_advanced(ZSTD_customMem customMem) -{ /* CStream and CCtx are now same object */ - return ZSTD_createCCtx_advanced(customMem); -} - -size_t ZSTD_freeCStream(ZSTD_CStream* zcs) -{ - return ZSTD_freeCCtx(zcs); /* same object */ -} - - - -/*====== Initialization ======*/ - -size_t ZSTD_CStreamInSize(void) { return ZSTD_BLOCKSIZE_MAX; } - -size_t ZSTD_CStreamOutSize(void) -{ - return ZSTD_compressBound(ZSTD_BLOCKSIZE_MAX) + ZSTD_blockHeaderSize + 4 /* 32-bits hash */ ; -} - -static size_t ZSTD_resetCStream_internal(ZSTD_CStream* cctx, - const void* const dict, size_t const dictSize, ZSTD_dictContentType_e const dictContentType, - const ZSTD_CDict* const cdict, - ZSTD_CCtx_params params, unsigned long long const pledgedSrcSize) -{ - DEBUGLOG(4, "ZSTD_resetCStream_internal"); - /* Finalize the compression parameters */ - params.cParams = ZSTD_getCParamsFromCCtxParams(¶ms, pledgedSrcSize, dictSize); - /* params are supposed to be fully validated at this point */ - assert(!ZSTD_isError(ZSTD_checkCParams(params.cParams))); - assert(!((dict) && (cdict))); /* either dict or cdict, not both */ - - FORWARD_IF_ERROR( ZSTD_compressBegin_internal(cctx, - dict, dictSize, dictContentType, ZSTD_dtlm_fast, - cdict, - ¶ms, pledgedSrcSize, - ZSTDb_buffered) , ""); - - cctx->inToCompress = 0; - cctx->inBuffPos = 0; - cctx->inBuffTarget = cctx->blockSize - + (cctx->blockSize == pledgedSrcSize); /* for small input: avoid automatic flush on reaching end of block, since it would require to add a 3-bytes null block to end frame */ - cctx->outBuffContentSize = cctx->outBuffFlushedSize = 0; - cctx->streamStage = zcss_load; - cctx->frameEnded = 0; - return 0; /* ready to go */ -} - -/* ZSTD_resetCStream(): - * pledgedSrcSize == 0 means "unknown" */ -size_t ZSTD_resetCStream(ZSTD_CStream* zcs, unsigned long long pss) -{ - /* temporary : 0 interpreted as "unknown" during transition period. - * Users willing to specify "unknown" **must** use ZSTD_CONTENTSIZE_UNKNOWN. - * 0 will be interpreted as "empty" in the future. - */ - U64 const pledgedSrcSize = (pss==0) ? ZSTD_CONTENTSIZE_UNKNOWN : pss; - DEBUGLOG(4, "ZSTD_resetCStream: pledgedSrcSize = %u", (unsigned)pledgedSrcSize); - FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); - FORWARD_IF_ERROR( ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize) , ""); - return 0; -} - -/*! ZSTD_initCStream_internal() : - * Note : for lib/compress only. Used by zstdmt_compress.c. - * Assumption 1 : params are valid - * Assumption 2 : either dict, or cdict, is defined, not both */ -size_t ZSTD_initCStream_internal(ZSTD_CStream* zcs, - const void* dict, size_t dictSize, const ZSTD_CDict* cdict, - const ZSTD_CCtx_params* params, - unsigned long long pledgedSrcSize) -{ - DEBUGLOG(4, "ZSTD_initCStream_internal"); - FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); - FORWARD_IF_ERROR( ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize) , ""); - assert(!ZSTD_isError(ZSTD_checkCParams(params->cParams))); - zcs->requestedParams = *params; - assert(!((dict) && (cdict))); /* either dict or cdict, not both */ - if (dict) { - FORWARD_IF_ERROR( ZSTD_CCtx_loadDictionary(zcs, dict, dictSize) , ""); - } else { - /* Dictionary is cleared if !cdict */ - FORWARD_IF_ERROR( ZSTD_CCtx_refCDict(zcs, cdict) , ""); - } - return 0; -} - -/* ZSTD_initCStream_usingCDict_advanced() : - * same as ZSTD_initCStream_usingCDict(), with control over frame parameters */ -size_t ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs, - const ZSTD_CDict* cdict, - ZSTD_frameParameters fParams, - unsigned long long pledgedSrcSize) -{ - DEBUGLOG(4, "ZSTD_initCStream_usingCDict_advanced"); - FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); - FORWARD_IF_ERROR( ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize) , ""); - zcs->requestedParams.fParams = fParams; - FORWARD_IF_ERROR( ZSTD_CCtx_refCDict(zcs, cdict) , ""); - return 0; -} - -/* note : cdict must outlive compression session */ -size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict) -{ - DEBUGLOG(4, "ZSTD_initCStream_usingCDict"); - FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); - FORWARD_IF_ERROR( ZSTD_CCtx_refCDict(zcs, cdict) , ""); - return 0; -} - - -/* ZSTD_initCStream_advanced() : - * pledgedSrcSize must be exact. - * if srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN. - * dict is loaded with default parameters ZSTD_dct_auto and ZSTD_dlm_byCopy. */ -size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs, - const void* dict, size_t dictSize, - ZSTD_parameters params, unsigned long long pss) -{ - /* for compatibility with older programs relying on this behavior. - * Users should now specify ZSTD_CONTENTSIZE_UNKNOWN. - * This line will be removed in the future. - */ - U64 const pledgedSrcSize = (pss==0 && params.fParams.contentSizeFlag==0) ? ZSTD_CONTENTSIZE_UNKNOWN : pss; - DEBUGLOG(4, "ZSTD_initCStream_advanced"); - FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); - FORWARD_IF_ERROR( ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize) , ""); - FORWARD_IF_ERROR( ZSTD_checkCParams(params.cParams) , ""); - zcs->requestedParams = ZSTD_assignParamsToCCtxParams(&zcs->requestedParams, ¶ms); - FORWARD_IF_ERROR( ZSTD_CCtx_loadDictionary(zcs, dict, dictSize) , ""); - return 0; -} - -size_t ZSTD_initCStream_usingDict(ZSTD_CStream* zcs, const void* dict, size_t dictSize, int compressionLevel) -{ - DEBUGLOG(4, "ZSTD_initCStream_usingDict"); - FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); - FORWARD_IF_ERROR( ZSTD_CCtx_setParameter(zcs, ZSTD_c_compressionLevel, compressionLevel) , ""); - FORWARD_IF_ERROR( ZSTD_CCtx_loadDictionary(zcs, dict, dictSize) , ""); - return 0; -} - -size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pss) -{ - /* temporary : 0 interpreted as "unknown" during transition period. - * Users willing to specify "unknown" **must** use ZSTD_CONTENTSIZE_UNKNOWN. - * 0 will be interpreted as "empty" in the future. - */ - U64 const pledgedSrcSize = (pss==0) ? ZSTD_CONTENTSIZE_UNKNOWN : pss; - DEBUGLOG(4, "ZSTD_initCStream_srcSize"); - FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); - FORWARD_IF_ERROR( ZSTD_CCtx_refCDict(zcs, NULL) , ""); - FORWARD_IF_ERROR( ZSTD_CCtx_setParameter(zcs, ZSTD_c_compressionLevel, compressionLevel) , ""); - FORWARD_IF_ERROR( ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize) , ""); - return 0; -} - -size_t ZSTD_initCStream(ZSTD_CStream* zcs, int compressionLevel) -{ - DEBUGLOG(4, "ZSTD_initCStream"); - FORWARD_IF_ERROR( ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only) , ""); - FORWARD_IF_ERROR( ZSTD_CCtx_refCDict(zcs, NULL) , ""); - FORWARD_IF_ERROR( ZSTD_CCtx_setParameter(zcs, ZSTD_c_compressionLevel, compressionLevel) , ""); - return 0; -} - -/*====== Compression ======*/ - -static size_t ZSTD_nextInputSizeHint(const ZSTD_CCtx* cctx) -{ - size_t hintInSize = cctx->inBuffTarget - cctx->inBuffPos; - if (hintInSize==0) hintInSize = cctx->blockSize; - return hintInSize; -} - -/** ZSTD_compressStream_generic(): - * internal function for all *compressStream*() variants - * non-static, because can be called from zstdmt_compress.c - * @return : hint size for next input */ -static size_t ZSTD_compressStream_generic(ZSTD_CStream* zcs, - ZSTD_outBuffer* output, - ZSTD_inBuffer* input, - ZSTD_EndDirective const flushMode) -{ - const char* const istart = (const char*)input->src; - const char* const iend = input->size != 0 ? istart + input->size : istart; - const char* ip = input->pos != 0 ? istart + input->pos : istart; - char* const ostart = (char*)output->dst; - char* const oend = output->size != 0 ? ostart + output->size : ostart; - char* op = output->pos != 0 ? ostart + output->pos : ostart; - U32 someMoreWork = 1; - - /* check expectations */ - DEBUGLOG(5, "ZSTD_compressStream_generic, flush=%u", (unsigned)flushMode); - assert(zcs->inBuff != NULL); - assert(zcs->inBuffSize > 0); - assert(zcs->outBuff != NULL); - assert(zcs->outBuffSize > 0); - assert(output->pos <= output->size); - assert(input->pos <= input->size); - - while (someMoreWork) { - switch(zcs->streamStage) - { - case zcss_init: - RETURN_ERROR(init_missing, "call ZSTD_initCStream() first!"); - - case zcss_load: - if ( (flushMode == ZSTD_e_end) - && ((size_t)(oend-op) >= ZSTD_compressBound(iend-ip)) /* enough dstCapacity */ - && (zcs->inBuffPos == 0) ) { - /* shortcut to compression pass directly into output buffer */ - size_t const cSize = ZSTD_compressEnd(zcs, - op, oend-op, ip, iend-ip); - DEBUGLOG(4, "ZSTD_compressEnd : cSize=%u", (unsigned)cSize); - FORWARD_IF_ERROR(cSize, "ZSTD_compressEnd failed"); - ip = iend; - op += cSize; - zcs->frameEnded = 1; - ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only); - someMoreWork = 0; break; - } - /* complete loading into inBuffer */ - { size_t const toLoad = zcs->inBuffTarget - zcs->inBuffPos; - size_t const loaded = ZSTD_limitCopy( - zcs->inBuff + zcs->inBuffPos, toLoad, - ip, iend-ip); - zcs->inBuffPos += loaded; - if (loaded != 0) - ip += loaded; - if ( (flushMode == ZSTD_e_continue) - && (zcs->inBuffPos < zcs->inBuffTarget) ) { - /* not enough input to fill full block : stop here */ - someMoreWork = 0; break; - } - if ( (flushMode == ZSTD_e_flush) - && (zcs->inBuffPos == zcs->inToCompress) ) { - /* empty */ - someMoreWork = 0; break; - } - } - /* compress current block (note : this stage cannot be stopped in the middle) */ - DEBUGLOG(5, "stream compression stage (flushMode==%u)", flushMode); - { void* cDst; - size_t cSize; - size_t const iSize = zcs->inBuffPos - zcs->inToCompress; - size_t oSize = oend-op; - unsigned const lastBlock = (flushMode == ZSTD_e_end) && (ip==iend); - if (oSize >= ZSTD_compressBound(iSize)) - cDst = op; /* compress into output buffer, to skip flush stage */ - else - cDst = zcs->outBuff, oSize = zcs->outBuffSize; - cSize = lastBlock ? - ZSTD_compressEnd(zcs, cDst, oSize, - zcs->inBuff + zcs->inToCompress, iSize) : - ZSTD_compressContinue(zcs, cDst, oSize, - zcs->inBuff + zcs->inToCompress, iSize); - FORWARD_IF_ERROR(cSize, "%s", lastBlock ? "ZSTD_compressEnd failed" : "ZSTD_compressContinue failed"); - zcs->frameEnded = lastBlock; - /* prepare next block */ - zcs->inBuffTarget = zcs->inBuffPos + zcs->blockSize; - if (zcs->inBuffTarget > zcs->inBuffSize) - zcs->inBuffPos = 0, zcs->inBuffTarget = zcs->blockSize; - DEBUGLOG(5, "inBuffTarget:%u / inBuffSize:%u", - (unsigned)zcs->inBuffTarget, (unsigned)zcs->inBuffSize); - if (!lastBlock) - assert(zcs->inBuffTarget <= zcs->inBuffSize); - zcs->inToCompress = zcs->inBuffPos; - if (cDst == op) { /* no need to flush */ - op += cSize; - if (zcs->frameEnded) { - DEBUGLOG(5, "Frame completed directly in outBuffer"); - someMoreWork = 0; - ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only); - } - break; - } - zcs->outBuffContentSize = cSize; - zcs->outBuffFlushedSize = 0; - zcs->streamStage = zcss_flush; /* pass-through to flush stage */ - } - /* fall-through */ - case zcss_flush: - DEBUGLOG(5, "flush stage"); - { size_t const toFlush = zcs->outBuffContentSize - zcs->outBuffFlushedSize; - size_t const flushed = ZSTD_limitCopy(op, (size_t)(oend-op), - zcs->outBuff + zcs->outBuffFlushedSize, toFlush); - DEBUGLOG(5, "toFlush: %u into %u ==> flushed: %u", - (unsigned)toFlush, (unsigned)(oend-op), (unsigned)flushed); - if (flushed) - op += flushed; - zcs->outBuffFlushedSize += flushed; - if (toFlush!=flushed) { - /* flush not fully completed, presumably because dst is too small */ - assert(op==oend); - someMoreWork = 0; - break; - } - zcs->outBuffContentSize = zcs->outBuffFlushedSize = 0; - if (zcs->frameEnded) { - DEBUGLOG(5, "Frame completed on flush"); - someMoreWork = 0; - ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only); - break; - } - zcs->streamStage = zcss_load; - break; - } - - default: /* impossible */ - assert(0); - } - } - - input->pos = ip - istart; - output->pos = op - ostart; - if (zcs->frameEnded) return 0; - return ZSTD_nextInputSizeHint(zcs); -} - -static size_t ZSTD_nextInputSizeHint_MTorST(const ZSTD_CCtx* cctx) -{ -#ifdef ZSTD_MULTITHREAD - if (cctx->appliedParams.nbWorkers >= 1) { - assert(cctx->mtctx != NULL); - return ZSTDMT_nextInputSizeHint(cctx->mtctx); - } -#endif - return ZSTD_nextInputSizeHint(cctx); - -} - -size_t ZSTD_compressStream(ZSTD_CStream* zcs, ZSTD_outBuffer* output, ZSTD_inBuffer* input) -{ - FORWARD_IF_ERROR( ZSTD_compressStream2(zcs, output, input, ZSTD_e_continue) , ""); - return ZSTD_nextInputSizeHint_MTorST(zcs); -} - - -size_t ZSTD_compressStream2( ZSTD_CCtx* cctx, - ZSTD_outBuffer* output, - ZSTD_inBuffer* input, - ZSTD_EndDirective endOp) -{ - DEBUGLOG(5, "ZSTD_compressStream2, endOp=%u ", (unsigned)endOp); - /* check conditions */ - RETURN_ERROR_IF(output->pos > output->size, GENERIC, "invalid buffer"); - RETURN_ERROR_IF(input->pos > input->size, GENERIC, "invalid buffer"); - assert(cctx!=NULL); - - /* transparent initialization stage */ - if (cctx->streamStage == zcss_init) { - ZSTD_CCtx_params params = cctx->requestedParams; - ZSTD_prefixDict const prefixDict = cctx->prefixDict; - FORWARD_IF_ERROR( ZSTD_initLocalDict(cctx) , ""); /* Init the local dict if present. */ - memset(&cctx->prefixDict, 0, sizeof(cctx->prefixDict)); /* single usage */ - assert(prefixDict.dict==NULL || cctx->cdict==NULL); /* only one can be set */ - DEBUGLOG(4, "ZSTD_compressStream2 : transparent init stage"); - if (endOp == ZSTD_e_end) cctx->pledgedSrcSizePlusOne = input->size + 1; /* auto-fix pledgedSrcSize */ - params.cParams = ZSTD_getCParamsFromCCtxParams( - &cctx->requestedParams, cctx->pledgedSrcSizePlusOne-1, 0 /*dictSize*/); - - -#ifdef ZSTD_MULTITHREAD - if ((cctx->pledgedSrcSizePlusOne-1) <= ZSTDMT_JOBSIZE_MIN) { - params.nbWorkers = 0; /* do not invoke multi-threading when src size is too small */ - } - if (params.nbWorkers > 0) { - /* mt context creation */ - if (cctx->mtctx == NULL) { - DEBUGLOG(4, "ZSTD_compressStream2: creating new mtctx for nbWorkers=%u", - params.nbWorkers); - cctx->mtctx = ZSTDMT_createCCtx_advanced((U32)params.nbWorkers, cctx->customMem); - RETURN_ERROR_IF(cctx->mtctx == NULL, memory_allocation, "NULL pointer!"); - } - /* mt compression */ - DEBUGLOG(4, "call ZSTDMT_initCStream_internal as nbWorkers=%u", params.nbWorkers); - FORWARD_IF_ERROR( ZSTDMT_initCStream_internal( - cctx->mtctx, - prefixDict.dict, prefixDict.dictSize, prefixDict.dictContentType, - cctx->cdict, params, cctx->pledgedSrcSizePlusOne-1) , ""); - cctx->streamStage = zcss_load; - cctx->appliedParams.nbWorkers = params.nbWorkers; - } else -#endif - { FORWARD_IF_ERROR( ZSTD_resetCStream_internal(cctx, - prefixDict.dict, prefixDict.dictSize, prefixDict.dictContentType, - cctx->cdict, - params, cctx->pledgedSrcSizePlusOne-1) , ""); - assert(cctx->streamStage == zcss_load); - assert(cctx->appliedParams.nbWorkers == 0); - } } - /* end of transparent initialization stage */ - - /* compression stage */ -#ifdef ZSTD_MULTITHREAD - if (cctx->appliedParams.nbWorkers > 0) { - int const forceMaxProgress = (endOp == ZSTD_e_flush || endOp == ZSTD_e_end); - size_t flushMin; - assert(forceMaxProgress || endOp == ZSTD_e_continue /* Protection for a new flush type */); - if (cctx->cParamsChanged) { - ZSTDMT_updateCParams_whileCompressing(cctx->mtctx, &cctx->requestedParams); - cctx->cParamsChanged = 0; - } - do { - flushMin = ZSTDMT_compressStream_generic(cctx->mtctx, output, input, endOp); - if ( ZSTD_isError(flushMin) - || (endOp == ZSTD_e_end && flushMin == 0) ) { /* compression completed */ - ZSTD_CCtx_reset(cctx, ZSTD_reset_session_only); - } - FORWARD_IF_ERROR(flushMin, "ZSTDMT_compressStream_generic failed"); - } while (forceMaxProgress && flushMin != 0 && output->pos < output->size); - DEBUGLOG(5, "completed ZSTD_compressStream2 delegating to ZSTDMT_compressStream_generic"); - /* Either we don't require maximum forward progress, we've finished the - * flush, or we are out of output space. - */ - assert(!forceMaxProgress || flushMin == 0 || output->pos == output->size); - return flushMin; - } -#endif - FORWARD_IF_ERROR( ZSTD_compressStream_generic(cctx, output, input, endOp) , ""); - DEBUGLOG(5, "completed ZSTD_compressStream2"); - return cctx->outBuffContentSize - cctx->outBuffFlushedSize; /* remaining to flush */ -} - -size_t ZSTD_compressStream2_simpleArgs ( - ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, size_t* dstPos, - const void* src, size_t srcSize, size_t* srcPos, - ZSTD_EndDirective endOp) -{ - ZSTD_outBuffer output = { dst, dstCapacity, *dstPos }; - ZSTD_inBuffer input = { src, srcSize, *srcPos }; - /* ZSTD_compressStream2() will check validity of dstPos and srcPos */ - size_t const cErr = ZSTD_compressStream2(cctx, &output, &input, endOp); - *dstPos = output.pos; - *srcPos = input.pos; - return cErr; -} - -size_t ZSTD_compress2(ZSTD_CCtx* cctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize) -{ - DEBUGLOG(4, "ZSTD_compress2 (srcSize=%u)", (unsigned)srcSize); - ZSTD_CCtx_reset(cctx, ZSTD_reset_session_only); - { size_t oPos = 0; - size_t iPos = 0; - size_t const result = ZSTD_compressStream2_simpleArgs(cctx, - dst, dstCapacity, &oPos, - src, srcSize, &iPos, - ZSTD_e_end); - FORWARD_IF_ERROR(result, "ZSTD_compressStream2_simpleArgs failed"); - if (result != 0) { /* compression not completed, due to lack of output space */ - assert(oPos == dstCapacity); - RETURN_ERROR(dstSize_tooSmall, ""); - } - assert(iPos == srcSize); /* all input is expected consumed */ - return oPos; - } -} - -/*====== Finalize ======*/ - -/*! ZSTD_flushStream() : - * @return : amount of data remaining to flush */ -size_t ZSTD_flushStream(ZSTD_CStream* zcs, ZSTD_outBuffer* output) -{ - ZSTD_inBuffer input = { NULL, 0, 0 }; - return ZSTD_compressStream2(zcs, output, &input, ZSTD_e_flush); -} - - -size_t ZSTD_endStream(ZSTD_CStream* zcs, ZSTD_outBuffer* output) -{ - ZSTD_inBuffer input = { NULL, 0, 0 }; - size_t const remainingToFlush = ZSTD_compressStream2(zcs, output, &input, ZSTD_e_end); - FORWARD_IF_ERROR( remainingToFlush , "ZSTD_compressStream2 failed"); - if (zcs->appliedParams.nbWorkers > 0) return remainingToFlush; /* minimal estimation */ - /* single thread mode : attempt to calculate remaining to flush more precisely */ - { size_t const lastBlockSize = zcs->frameEnded ? 0 : ZSTD_BLOCKHEADERSIZE; - size_t const checksumSize = (size_t)(zcs->frameEnded ? 0 : zcs->appliedParams.fParams.checksumFlag * 4); - size_t const toFlush = remainingToFlush + lastBlockSize + checksumSize; - DEBUGLOG(4, "ZSTD_endStream : remaining to flush : %u", (unsigned)toFlush); - return toFlush; - } -} - - -/*-===== Pre-defined compression levels =====-*/ - -#define ZSTD_MAX_CLEVEL 22 -int ZSTD_maxCLevel(void) { return ZSTD_MAX_CLEVEL; } -int ZSTD_minCLevel(void) { return (int)-ZSTD_TARGETLENGTH_MAX; } - -static const ZSTD_compressionParameters ZSTD_defaultCParameters[4][ZSTD_MAX_CLEVEL+1] = { -{ /* "default" - for any srcSize > 256 KB */ - /* W, C, H, S, L, TL, strat */ - { 19, 12, 13, 1, 6, 1, ZSTD_fast }, /* base for negative levels */ - { 19, 13, 14, 1, 7, 0, ZSTD_fast }, /* level 1 */ - { 20, 15, 16, 1, 6, 0, ZSTD_fast }, /* level 2 */ - { 21, 16, 17, 1, 5, 0, ZSTD_dfast }, /* level 3 */ - { 21, 18, 18, 1, 5, 0, ZSTD_dfast }, /* level 4 */ - { 21, 18, 19, 2, 5, 2, ZSTD_greedy }, /* level 5 */ - { 21, 19, 19, 3, 5, 4, ZSTD_greedy }, /* level 6 */ - { 21, 19, 19, 3, 5, 8, ZSTD_lazy }, /* level 7 */ - { 21, 19, 19, 3, 5, 16, ZSTD_lazy2 }, /* level 8 */ - { 21, 19, 20, 4, 5, 16, ZSTD_lazy2 }, /* level 9 */ - { 22, 20, 21, 4, 5, 16, ZSTD_lazy2 }, /* level 10 */ - { 22, 21, 22, 4, 5, 16, ZSTD_lazy2 }, /* level 11 */ - { 22, 21, 22, 5, 5, 16, ZSTD_lazy2 }, /* level 12 */ - { 22, 21, 22, 5, 5, 32, ZSTD_btlazy2 }, /* level 13 */ - { 22, 22, 23, 5, 5, 32, ZSTD_btlazy2 }, /* level 14 */ - { 22, 23, 23, 6, 5, 32, ZSTD_btlazy2 }, /* level 15 */ - { 22, 22, 22, 5, 5, 48, ZSTD_btopt }, /* level 16 */ - { 23, 23, 22, 5, 4, 64, ZSTD_btopt }, /* level 17 */ - { 23, 23, 22, 6, 3, 64, ZSTD_btultra }, /* level 18 */ - { 23, 24, 22, 7, 3,256, ZSTD_btultra2}, /* level 19 */ - { 25, 25, 23, 7, 3,256, ZSTD_btultra2}, /* level 20 */ - { 26, 26, 24, 7, 3,512, ZSTD_btultra2}, /* level 21 */ - { 27, 27, 25, 9, 3,999, ZSTD_btultra2}, /* level 22 */ -}, -{ /* for srcSize <= 256 KB */ - /* W, C, H, S, L, T, strat */ - { 18, 12, 13, 1, 5, 1, ZSTD_fast }, /* base for negative levels */ - { 18, 13, 14, 1, 6, 0, ZSTD_fast }, /* level 1 */ - { 18, 14, 14, 1, 5, 0, ZSTD_dfast }, /* level 2 */ - { 18, 16, 16, 1, 4, 0, ZSTD_dfast }, /* level 3 */ - { 18, 16, 17, 2, 5, 2, ZSTD_greedy }, /* level 4.*/ - { 18, 18, 18, 3, 5, 2, ZSTD_greedy }, /* level 5.*/ - { 18, 18, 19, 3, 5, 4, ZSTD_lazy }, /* level 6.*/ - { 18, 18, 19, 4, 4, 4, ZSTD_lazy }, /* level 7 */ - { 18, 18, 19, 4, 4, 8, ZSTD_lazy2 }, /* level 8 */ - { 18, 18, 19, 5, 4, 8, ZSTD_lazy2 }, /* level 9 */ - { 18, 18, 19, 6, 4, 8, ZSTD_lazy2 }, /* level 10 */ - { 18, 18, 19, 5, 4, 12, ZSTD_btlazy2 }, /* level 11.*/ - { 18, 19, 19, 7, 4, 12, ZSTD_btlazy2 }, /* level 12.*/ - { 18, 18, 19, 4, 4, 16, ZSTD_btopt }, /* level 13 */ - { 18, 18, 19, 4, 3, 32, ZSTD_btopt }, /* level 14.*/ - { 18, 18, 19, 6, 3,128, ZSTD_btopt }, /* level 15.*/ - { 18, 19, 19, 6, 3,128, ZSTD_btultra }, /* level 16.*/ - { 18, 19, 19, 8, 3,256, ZSTD_btultra }, /* level 17.*/ - { 18, 19, 19, 6, 3,128, ZSTD_btultra2}, /* level 18.*/ - { 18, 19, 19, 8, 3,256, ZSTD_btultra2}, /* level 19.*/ - { 18, 19, 19, 10, 3,512, ZSTD_btultra2}, /* level 20.*/ - { 18, 19, 19, 12, 3,512, ZSTD_btultra2}, /* level 21.*/ - { 18, 19, 19, 13, 3,999, ZSTD_btultra2}, /* level 22.*/ -}, -{ /* for srcSize <= 128 KB */ - /* W, C, H, S, L, T, strat */ - { 17, 12, 12, 1, 5, 1, ZSTD_fast }, /* base for negative levels */ - { 17, 12, 13, 1, 6, 0, ZSTD_fast }, /* level 1 */ - { 17, 13, 15, 1, 5, 0, ZSTD_fast }, /* level 2 */ - { 17, 15, 16, 2, 5, 0, ZSTD_dfast }, /* level 3 */ - { 17, 17, 17, 2, 4, 0, ZSTD_dfast }, /* level 4 */ - { 17, 16, 17, 3, 4, 2, ZSTD_greedy }, /* level 5 */ - { 17, 17, 17, 3, 4, 4, ZSTD_lazy }, /* level 6 */ - { 17, 17, 17, 3, 4, 8, ZSTD_lazy2 }, /* level 7 */ - { 17, 17, 17, 4, 4, 8, ZSTD_lazy2 }, /* level 8 */ - { 17, 17, 17, 5, 4, 8, ZSTD_lazy2 }, /* level 9 */ - { 17, 17, 17, 6, 4, 8, ZSTD_lazy2 }, /* level 10 */ - { 17, 17, 17, 5, 4, 8, ZSTD_btlazy2 }, /* level 11 */ - { 17, 18, 17, 7, 4, 12, ZSTD_btlazy2 }, /* level 12 */ - { 17, 18, 17, 3, 4, 12, ZSTD_btopt }, /* level 13.*/ - { 17, 18, 17, 4, 3, 32, ZSTD_btopt }, /* level 14.*/ - { 17, 18, 17, 6, 3,256, ZSTD_btopt }, /* level 15.*/ - { 17, 18, 17, 6, 3,128, ZSTD_btultra }, /* level 16.*/ - { 17, 18, 17, 8, 3,256, ZSTD_btultra }, /* level 17.*/ - { 17, 18, 17, 10, 3,512, ZSTD_btultra }, /* level 18.*/ - { 17, 18, 17, 5, 3,256, ZSTD_btultra2}, /* level 19.*/ - { 17, 18, 17, 7, 3,512, ZSTD_btultra2}, /* level 20.*/ - { 17, 18, 17, 9, 3,512, ZSTD_btultra2}, /* level 21.*/ - { 17, 18, 17, 11, 3,999, ZSTD_btultra2}, /* level 22.*/ -}, -{ /* for srcSize <= 16 KB */ - /* W, C, H, S, L, T, strat */ - { 14, 12, 13, 1, 5, 1, ZSTD_fast }, /* base for negative levels */ - { 14, 14, 15, 1, 5, 0, ZSTD_fast }, /* level 1 */ - { 14, 14, 15, 1, 4, 0, ZSTD_fast }, /* level 2 */ - { 14, 14, 15, 2, 4, 0, ZSTD_dfast }, /* level 3 */ - { 14, 14, 14, 4, 4, 2, ZSTD_greedy }, /* level 4 */ - { 14, 14, 14, 3, 4, 4, ZSTD_lazy }, /* level 5.*/ - { 14, 14, 14, 4, 4, 8, ZSTD_lazy2 }, /* level 6 */ - { 14, 14, 14, 6, 4, 8, ZSTD_lazy2 }, /* level 7 */ - { 14, 14, 14, 8, 4, 8, ZSTD_lazy2 }, /* level 8.*/ - { 14, 15, 14, 5, 4, 8, ZSTD_btlazy2 }, /* level 9.*/ - { 14, 15, 14, 9, 4, 8, ZSTD_btlazy2 }, /* level 10.*/ - { 14, 15, 14, 3, 4, 12, ZSTD_btopt }, /* level 11.*/ - { 14, 15, 14, 4, 3, 24, ZSTD_btopt }, /* level 12.*/ - { 14, 15, 14, 5, 3, 32, ZSTD_btultra }, /* level 13.*/ - { 14, 15, 15, 6, 3, 64, ZSTD_btultra }, /* level 14.*/ - { 14, 15, 15, 7, 3,256, ZSTD_btultra }, /* level 15.*/ - { 14, 15, 15, 5, 3, 48, ZSTD_btultra2}, /* level 16.*/ - { 14, 15, 15, 6, 3,128, ZSTD_btultra2}, /* level 17.*/ - { 14, 15, 15, 7, 3,256, ZSTD_btultra2}, /* level 18.*/ - { 14, 15, 15, 8, 3,256, ZSTD_btultra2}, /* level 19.*/ - { 14, 15, 15, 8, 3,512, ZSTD_btultra2}, /* level 20.*/ - { 14, 15, 15, 9, 3,512, ZSTD_btultra2}, /* level 21.*/ - { 14, 15, 15, 10, 3,999, ZSTD_btultra2}, /* level 22.*/ -}, -}; - -/*! ZSTD_getCParams_internal() : - * @return ZSTD_compressionParameters structure for a selected compression level, srcSize and dictSize. - * Note: srcSizeHint 0 means 0, use ZSTD_CONTENTSIZE_UNKNOWN for unknown. - * Use dictSize == 0 for unknown or unused. */ -static ZSTD_compressionParameters ZSTD_getCParams_internal(int compressionLevel, unsigned long long srcSizeHint, size_t dictSize) -{ - int const unknown = srcSizeHint == ZSTD_CONTENTSIZE_UNKNOWN; - size_t const addedSize = unknown && dictSize > 0 ? 500 : 0; - U64 const rSize = unknown && dictSize == 0 ? ZSTD_CONTENTSIZE_UNKNOWN : srcSizeHint+dictSize+addedSize; - U32 const tableID = (rSize <= 256 KB) + (rSize <= 128 KB) + (rSize <= 16 KB); - int row = compressionLevel; - DEBUGLOG(5, "ZSTD_getCParams_internal (cLevel=%i)", compressionLevel); - if (compressionLevel == 0) row = ZSTD_CLEVEL_DEFAULT; /* 0 == default */ - if (compressionLevel < 0) row = 0; /* entry 0 is baseline for fast mode */ - if (compressionLevel > ZSTD_MAX_CLEVEL) row = ZSTD_MAX_CLEVEL; - { ZSTD_compressionParameters cp = ZSTD_defaultCParameters[tableID][row]; - if (compressionLevel < 0) cp.targetLength = (unsigned)(-compressionLevel); /* acceleration factor */ - /* refine parameters based on srcSize & dictSize */ - return ZSTD_adjustCParams_internal(cp, srcSizeHint, dictSize); - } -} - -/*! ZSTD_getCParams() : - * @return ZSTD_compressionParameters structure for a selected compression level, srcSize and dictSize. - * Size values are optional, provide 0 if not known or unused */ -ZSTD_compressionParameters ZSTD_getCParams(int compressionLevel, unsigned long long srcSizeHint, size_t dictSize) -{ - if (srcSizeHint == 0) srcSizeHint = ZSTD_CONTENTSIZE_UNKNOWN; - return ZSTD_getCParams_internal(compressionLevel, srcSizeHint, dictSize); -} - -/*! ZSTD_getParams() : - * same idea as ZSTD_getCParams() - * @return a `ZSTD_parameters` structure (instead of `ZSTD_compressionParameters`). - * Fields of `ZSTD_frameParameters` are set to default values */ -static ZSTD_parameters ZSTD_getParams_internal(int compressionLevel, unsigned long long srcSizeHint, size_t dictSize) { - ZSTD_parameters params; - ZSTD_compressionParameters const cParams = ZSTD_getCParams_internal(compressionLevel, srcSizeHint, dictSize); - DEBUGLOG(5, "ZSTD_getParams (cLevel=%i)", compressionLevel); - memset(¶ms, 0, sizeof(params)); - params.cParams = cParams; - params.fParams.contentSizeFlag = 1; - return params; -} - -/*! ZSTD_getParams() : - * same idea as ZSTD_getCParams() - * @return a `ZSTD_parameters` structure (instead of `ZSTD_compressionParameters`). - * Fields of `ZSTD_frameParameters` are set to default values */ -ZSTD_parameters ZSTD_getParams(int compressionLevel, unsigned long long srcSizeHint, size_t dictSize) { - if (srcSizeHint == 0) srcSizeHint = ZSTD_CONTENTSIZE_UNKNOWN; - return ZSTD_getParams_internal(compressionLevel, srcSizeHint, dictSize); -} -/**** ended inlining compress/zstd_compress.c ****/ -/**** start inlining compress/zstd_double_fast.c ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -/**** skipping file: zstd_compress_internal.h ****/ -/**** skipping file: zstd_double_fast.h ****/ - - -void ZSTD_fillDoubleHashTable(ZSTD_matchState_t* ms, - void const* end, ZSTD_dictTableLoadMethod_e dtlm) -{ - const ZSTD_compressionParameters* const cParams = &ms->cParams; - U32* const hashLarge = ms->hashTable; - U32 const hBitsL = cParams->hashLog; - U32 const mls = cParams->minMatch; - U32* const hashSmall = ms->chainTable; - U32 const hBitsS = cParams->chainLog; - const BYTE* const base = ms->window.base; - const BYTE* ip = base + ms->nextToUpdate; - const BYTE* const iend = ((const BYTE*)end) - HASH_READ_SIZE; - const U32 fastHashFillStep = 3; - - /* Always insert every fastHashFillStep position into the hash tables. - * Insert the other positions into the large hash table if their entry - * is empty. - */ - for (; ip + fastHashFillStep - 1 <= iend; ip += fastHashFillStep) { - U32 const current = (U32)(ip - base); - U32 i; - for (i = 0; i < fastHashFillStep; ++i) { - size_t const smHash = ZSTD_hashPtr(ip + i, hBitsS, mls); - size_t const lgHash = ZSTD_hashPtr(ip + i, hBitsL, 8); - if (i == 0) - hashSmall[smHash] = current + i; - if (i == 0 || hashLarge[lgHash] == 0) - hashLarge[lgHash] = current + i; - /* Only load extra positions for ZSTD_dtlm_full */ - if (dtlm == ZSTD_dtlm_fast) - break; - } } -} - - -FORCE_INLINE_TEMPLATE -size_t ZSTD_compressBlock_doubleFast_generic( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize, - U32 const mls /* template */, ZSTD_dictMode_e const dictMode) -{ - ZSTD_compressionParameters const* cParams = &ms->cParams; - U32* const hashLong = ms->hashTable; - const U32 hBitsL = cParams->hashLog; - U32* const hashSmall = ms->chainTable; - const U32 hBitsS = cParams->chainLog; - const BYTE* const base = ms->window.base; - const BYTE* const istart = (const BYTE*)src; - const BYTE* ip = istart; - const BYTE* anchor = istart; - const U32 endIndex = (U32)((size_t)(istart - base) + srcSize); - /* presumes that, if there is a dictionary, it must be using Attach mode */ - const U32 prefixLowestIndex = ZSTD_getLowestPrefixIndex(ms, endIndex, cParams->windowLog); - const BYTE* const prefixLowest = base + prefixLowestIndex; - const BYTE* const iend = istart + srcSize; - const BYTE* const ilimit = iend - HASH_READ_SIZE; - U32 offset_1=rep[0], offset_2=rep[1]; - U32 offsetSaved = 0; - - const ZSTD_matchState_t* const dms = ms->dictMatchState; - const ZSTD_compressionParameters* const dictCParams = - dictMode == ZSTD_dictMatchState ? - &dms->cParams : NULL; - const U32* const dictHashLong = dictMode == ZSTD_dictMatchState ? - dms->hashTable : NULL; - const U32* const dictHashSmall = dictMode == ZSTD_dictMatchState ? - dms->chainTable : NULL; - const U32 dictStartIndex = dictMode == ZSTD_dictMatchState ? - dms->window.dictLimit : 0; - const BYTE* const dictBase = dictMode == ZSTD_dictMatchState ? - dms->window.base : NULL; - const BYTE* const dictStart = dictMode == ZSTD_dictMatchState ? - dictBase + dictStartIndex : NULL; - const BYTE* const dictEnd = dictMode == ZSTD_dictMatchState ? - dms->window.nextSrc : NULL; - const U32 dictIndexDelta = dictMode == ZSTD_dictMatchState ? - prefixLowestIndex - (U32)(dictEnd - dictBase) : - 0; - const U32 dictHBitsL = dictMode == ZSTD_dictMatchState ? - dictCParams->hashLog : hBitsL; - const U32 dictHBitsS = dictMode == ZSTD_dictMatchState ? - dictCParams->chainLog : hBitsS; - const U32 dictAndPrefixLength = (U32)((ip - prefixLowest) + (dictEnd - dictStart)); - - DEBUGLOG(5, "ZSTD_compressBlock_doubleFast_generic"); - - assert(dictMode == ZSTD_noDict || dictMode == ZSTD_dictMatchState); - - /* if a dictionary is attached, it must be within window range */ - if (dictMode == ZSTD_dictMatchState) { - assert(ms->window.dictLimit + (1U << cParams->windowLog) >= endIndex); - } - - /* init */ - ip += (dictAndPrefixLength == 0); - if (dictMode == ZSTD_noDict) { - U32 const current = (U32)(ip - base); - U32 const windowLow = ZSTD_getLowestPrefixIndex(ms, current, cParams->windowLog); - U32 const maxRep = current - windowLow; - if (offset_2 > maxRep) offsetSaved = offset_2, offset_2 = 0; - if (offset_1 > maxRep) offsetSaved = offset_1, offset_1 = 0; - } - if (dictMode == ZSTD_dictMatchState) { - /* dictMatchState repCode checks don't currently handle repCode == 0 - * disabling. */ - assert(offset_1 <= dictAndPrefixLength); - assert(offset_2 <= dictAndPrefixLength); - } - - /* Main Search Loop */ - while (ip < ilimit) { /* < instead of <=, because repcode check at (ip+1) */ - size_t mLength; - U32 offset; - size_t const h2 = ZSTD_hashPtr(ip, hBitsL, 8); - size_t const h = ZSTD_hashPtr(ip, hBitsS, mls); - size_t const dictHL = ZSTD_hashPtr(ip, dictHBitsL, 8); - size_t const dictHS = ZSTD_hashPtr(ip, dictHBitsS, mls); - U32 const current = (U32)(ip-base); - U32 const matchIndexL = hashLong[h2]; - U32 matchIndexS = hashSmall[h]; - const BYTE* matchLong = base + matchIndexL; - const BYTE* match = base + matchIndexS; - const U32 repIndex = current + 1 - offset_1; - const BYTE* repMatch = (dictMode == ZSTD_dictMatchState - && repIndex < prefixLowestIndex) ? - dictBase + (repIndex - dictIndexDelta) : - base + repIndex; - hashLong[h2] = hashSmall[h] = current; /* update hash tables */ - - /* check dictMatchState repcode */ - if (dictMode == ZSTD_dictMatchState - && ((U32)((prefixLowestIndex-1) - repIndex) >= 3 /* intentional underflow */) - && (MEM_read32(repMatch) == MEM_read32(ip+1)) ) { - const BYTE* repMatchEnd = repIndex < prefixLowestIndex ? dictEnd : iend; - mLength = ZSTD_count_2segments(ip+1+4, repMatch+4, iend, repMatchEnd, prefixLowest) + 4; - ip++; - ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, mLength-MINMATCH); - goto _match_stored; - } - - /* check noDict repcode */ - if ( dictMode == ZSTD_noDict - && ((offset_1 > 0) & (MEM_read32(ip+1-offset_1) == MEM_read32(ip+1)))) { - mLength = ZSTD_count(ip+1+4, ip+1+4-offset_1, iend) + 4; - ip++; - ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, mLength-MINMATCH); - goto _match_stored; - } - - if (matchIndexL > prefixLowestIndex) { - /* check prefix long match */ - if (MEM_read64(matchLong) == MEM_read64(ip)) { - mLength = ZSTD_count(ip+8, matchLong+8, iend) + 8; - offset = (U32)(ip-matchLong); - while (((ip>anchor) & (matchLong>prefixLowest)) && (ip[-1] == matchLong[-1])) { ip--; matchLong--; mLength++; } /* catch up */ - goto _match_found; - } - } else if (dictMode == ZSTD_dictMatchState) { - /* check dictMatchState long match */ - U32 const dictMatchIndexL = dictHashLong[dictHL]; - const BYTE* dictMatchL = dictBase + dictMatchIndexL; - assert(dictMatchL < dictEnd); - - if (dictMatchL > dictStart && MEM_read64(dictMatchL) == MEM_read64(ip)) { - mLength = ZSTD_count_2segments(ip+8, dictMatchL+8, iend, dictEnd, prefixLowest) + 8; - offset = (U32)(current - dictMatchIndexL - dictIndexDelta); - while (((ip>anchor) & (dictMatchL>dictStart)) && (ip[-1] == dictMatchL[-1])) { ip--; dictMatchL--; mLength++; } /* catch up */ - goto _match_found; - } } - - if (matchIndexS > prefixLowestIndex) { - /* check prefix short match */ - if (MEM_read32(match) == MEM_read32(ip)) { - goto _search_next_long; - } - } else if (dictMode == ZSTD_dictMatchState) { - /* check dictMatchState short match */ - U32 const dictMatchIndexS = dictHashSmall[dictHS]; - match = dictBase + dictMatchIndexS; - matchIndexS = dictMatchIndexS + dictIndexDelta; - - if (match > dictStart && MEM_read32(match) == MEM_read32(ip)) { - goto _search_next_long; - } } - - ip += ((ip-anchor) >> kSearchStrength) + 1; -#if defined(__aarch64__) - PREFETCH_L1(ip+256); -#endif - continue; - -_search_next_long: - - { size_t const hl3 = ZSTD_hashPtr(ip+1, hBitsL, 8); - size_t const dictHLNext = ZSTD_hashPtr(ip+1, dictHBitsL, 8); - U32 const matchIndexL3 = hashLong[hl3]; - const BYTE* matchL3 = base + matchIndexL3; - hashLong[hl3] = current + 1; - - /* check prefix long +1 match */ - if (matchIndexL3 > prefixLowestIndex) { - if (MEM_read64(matchL3) == MEM_read64(ip+1)) { - mLength = ZSTD_count(ip+9, matchL3+8, iend) + 8; - ip++; - offset = (U32)(ip-matchL3); - while (((ip>anchor) & (matchL3>prefixLowest)) && (ip[-1] == matchL3[-1])) { ip--; matchL3--; mLength++; } /* catch up */ - goto _match_found; - } - } else if (dictMode == ZSTD_dictMatchState) { - /* check dict long +1 match */ - U32 const dictMatchIndexL3 = dictHashLong[dictHLNext]; - const BYTE* dictMatchL3 = dictBase + dictMatchIndexL3; - assert(dictMatchL3 < dictEnd); - if (dictMatchL3 > dictStart && MEM_read64(dictMatchL3) == MEM_read64(ip+1)) { - mLength = ZSTD_count_2segments(ip+1+8, dictMatchL3+8, iend, dictEnd, prefixLowest) + 8; - ip++; - offset = (U32)(current + 1 - dictMatchIndexL3 - dictIndexDelta); - while (((ip>anchor) & (dictMatchL3>dictStart)) && (ip[-1] == dictMatchL3[-1])) { ip--; dictMatchL3--; mLength++; } /* catch up */ - goto _match_found; - } } } - - /* if no long +1 match, explore the short match we found */ - if (dictMode == ZSTD_dictMatchState && matchIndexS < prefixLowestIndex) { - mLength = ZSTD_count_2segments(ip+4, match+4, iend, dictEnd, prefixLowest) + 4; - offset = (U32)(current - matchIndexS); - while (((ip>anchor) & (match>dictStart)) && (ip[-1] == match[-1])) { ip--; match--; mLength++; } /* catch up */ - } else { - mLength = ZSTD_count(ip+4, match+4, iend) + 4; - offset = (U32)(ip - match); - while (((ip>anchor) & (match>prefixLowest)) && (ip[-1] == match[-1])) { ip--; match--; mLength++; } /* catch up */ - } - - /* fall-through */ - -_match_found: - offset_2 = offset_1; - offset_1 = offset; - - ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH); - -_match_stored: - /* match found */ - ip += mLength; - anchor = ip; - - if (ip <= ilimit) { - /* Complementary insertion */ - /* done after iLimit test, as candidates could be > iend-8 */ - { U32 const indexToInsert = current+2; - hashLong[ZSTD_hashPtr(base+indexToInsert, hBitsL, 8)] = indexToInsert; - hashLong[ZSTD_hashPtr(ip-2, hBitsL, 8)] = (U32)(ip-2-base); - hashSmall[ZSTD_hashPtr(base+indexToInsert, hBitsS, mls)] = indexToInsert; - hashSmall[ZSTD_hashPtr(ip-1, hBitsS, mls)] = (U32)(ip-1-base); - } - - /* check immediate repcode */ - if (dictMode == ZSTD_dictMatchState) { - while (ip <= ilimit) { - U32 const current2 = (U32)(ip-base); - U32 const repIndex2 = current2 - offset_2; - const BYTE* repMatch2 = dictMode == ZSTD_dictMatchState - && repIndex2 < prefixLowestIndex ? - dictBase + repIndex2 - dictIndexDelta : - base + repIndex2; - if ( ((U32)((prefixLowestIndex-1) - (U32)repIndex2) >= 3 /* intentional overflow */) - && (MEM_read32(repMatch2) == MEM_read32(ip)) ) { - const BYTE* const repEnd2 = repIndex2 < prefixLowestIndex ? dictEnd : iend; - size_t const repLength2 = ZSTD_count_2segments(ip+4, repMatch2+4, iend, repEnd2, prefixLowest) + 4; - U32 tmpOffset = offset_2; offset_2 = offset_1; offset_1 = tmpOffset; /* swap offset_2 <=> offset_1 */ - ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, repLength2-MINMATCH); - hashSmall[ZSTD_hashPtr(ip, hBitsS, mls)] = current2; - hashLong[ZSTD_hashPtr(ip, hBitsL, 8)] = current2; - ip += repLength2; - anchor = ip; - continue; - } - break; - } } - - if (dictMode == ZSTD_noDict) { - while ( (ip <= ilimit) - && ( (offset_2>0) - & (MEM_read32(ip) == MEM_read32(ip - offset_2)) )) { - /* store sequence */ - size_t const rLength = ZSTD_count(ip+4, ip+4-offset_2, iend) + 4; - U32 const tmpOff = offset_2; offset_2 = offset_1; offset_1 = tmpOff; /* swap offset_2 <=> offset_1 */ - hashSmall[ZSTD_hashPtr(ip, hBitsS, mls)] = (U32)(ip-base); - hashLong[ZSTD_hashPtr(ip, hBitsL, 8)] = (U32)(ip-base); - ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, rLength-MINMATCH); - ip += rLength; - anchor = ip; - continue; /* faster when present ... (?) */ - } } } - } /* while (ip < ilimit) */ - - /* save reps for next block */ - rep[0] = offset_1 ? offset_1 : offsetSaved; - rep[1] = offset_2 ? offset_2 : offsetSaved; - - /* Return the last literals size */ - return (size_t)(iend - anchor); -} - - -size_t ZSTD_compressBlock_doubleFast( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - const U32 mls = ms->cParams.minMatch; - switch(mls) - { - default: /* includes case 3 */ - case 4 : - return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 4, ZSTD_noDict); - case 5 : - return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 5, ZSTD_noDict); - case 6 : - return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 6, ZSTD_noDict); - case 7 : - return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 7, ZSTD_noDict); - } -} - - -size_t ZSTD_compressBlock_doubleFast_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - const U32 mls = ms->cParams.minMatch; - switch(mls) - { - default: /* includes case 3 */ - case 4 : - return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 4, ZSTD_dictMatchState); - case 5 : - return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 5, ZSTD_dictMatchState); - case 6 : - return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 6, ZSTD_dictMatchState); - case 7 : - return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 7, ZSTD_dictMatchState); - } -} - - -static size_t ZSTD_compressBlock_doubleFast_extDict_generic( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize, - U32 const mls /* template */) -{ - ZSTD_compressionParameters const* cParams = &ms->cParams; - U32* const hashLong = ms->hashTable; - U32 const hBitsL = cParams->hashLog; - U32* const hashSmall = ms->chainTable; - U32 const hBitsS = cParams->chainLog; - const BYTE* const istart = (const BYTE*)src; - const BYTE* ip = istart; - const BYTE* anchor = istart; - const BYTE* const iend = istart + srcSize; - const BYTE* const ilimit = iend - 8; - const BYTE* const base = ms->window.base; - const U32 endIndex = (U32)((size_t)(istart - base) + srcSize); - const U32 lowLimit = ZSTD_getLowestMatchIndex(ms, endIndex, cParams->windowLog); - const U32 dictStartIndex = lowLimit; - const U32 dictLimit = ms->window.dictLimit; - const U32 prefixStartIndex = (dictLimit > lowLimit) ? dictLimit : lowLimit; - const BYTE* const prefixStart = base + prefixStartIndex; - const BYTE* const dictBase = ms->window.dictBase; - const BYTE* const dictStart = dictBase + dictStartIndex; - const BYTE* const dictEnd = dictBase + prefixStartIndex; - U32 offset_1=rep[0], offset_2=rep[1]; - - DEBUGLOG(5, "ZSTD_compressBlock_doubleFast_extDict_generic (srcSize=%zu)", srcSize); - - /* if extDict is invalidated due to maxDistance, switch to "regular" variant */ - if (prefixStartIndex == dictStartIndex) - return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, mls, ZSTD_noDict); - - /* Search Loop */ - while (ip < ilimit) { /* < instead of <=, because (ip+1) */ - const size_t hSmall = ZSTD_hashPtr(ip, hBitsS, mls); - const U32 matchIndex = hashSmall[hSmall]; - const BYTE* const matchBase = matchIndex < prefixStartIndex ? dictBase : base; - const BYTE* match = matchBase + matchIndex; - - const size_t hLong = ZSTD_hashPtr(ip, hBitsL, 8); - const U32 matchLongIndex = hashLong[hLong]; - const BYTE* const matchLongBase = matchLongIndex < prefixStartIndex ? dictBase : base; - const BYTE* matchLong = matchLongBase + matchLongIndex; - - const U32 current = (U32)(ip-base); - const U32 repIndex = current + 1 - offset_1; /* offset_1 expected <= current +1 */ - const BYTE* const repBase = repIndex < prefixStartIndex ? dictBase : base; - const BYTE* const repMatch = repBase + repIndex; - size_t mLength; - hashSmall[hSmall] = hashLong[hLong] = current; /* update hash table */ - - if ((((U32)((prefixStartIndex-1) - repIndex) >= 3) /* intentional underflow : ensure repIndex doesn't overlap dict + prefix */ - & (repIndex > dictStartIndex)) - && (MEM_read32(repMatch) == MEM_read32(ip+1)) ) { - const BYTE* repMatchEnd = repIndex < prefixStartIndex ? dictEnd : iend; - mLength = ZSTD_count_2segments(ip+1+4, repMatch+4, iend, repMatchEnd, prefixStart) + 4; - ip++; - ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, mLength-MINMATCH); - } else { - if ((matchLongIndex > dictStartIndex) && (MEM_read64(matchLong) == MEM_read64(ip))) { - const BYTE* const matchEnd = matchLongIndex < prefixStartIndex ? dictEnd : iend; - const BYTE* const lowMatchPtr = matchLongIndex < prefixStartIndex ? dictStart : prefixStart; - U32 offset; - mLength = ZSTD_count_2segments(ip+8, matchLong+8, iend, matchEnd, prefixStart) + 8; - offset = current - matchLongIndex; - while (((ip>anchor) & (matchLong>lowMatchPtr)) && (ip[-1] == matchLong[-1])) { ip--; matchLong--; mLength++; } /* catch up */ - offset_2 = offset_1; - offset_1 = offset; - ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH); - - } else if ((matchIndex > dictStartIndex) && (MEM_read32(match) == MEM_read32(ip))) { - size_t const h3 = ZSTD_hashPtr(ip+1, hBitsL, 8); - U32 const matchIndex3 = hashLong[h3]; - const BYTE* const match3Base = matchIndex3 < prefixStartIndex ? dictBase : base; - const BYTE* match3 = match3Base + matchIndex3; - U32 offset; - hashLong[h3] = current + 1; - if ( (matchIndex3 > dictStartIndex) && (MEM_read64(match3) == MEM_read64(ip+1)) ) { - const BYTE* const matchEnd = matchIndex3 < prefixStartIndex ? dictEnd : iend; - const BYTE* const lowMatchPtr = matchIndex3 < prefixStartIndex ? dictStart : prefixStart; - mLength = ZSTD_count_2segments(ip+9, match3+8, iend, matchEnd, prefixStart) + 8; - ip++; - offset = current+1 - matchIndex3; - while (((ip>anchor) & (match3>lowMatchPtr)) && (ip[-1] == match3[-1])) { ip--; match3--; mLength++; } /* catch up */ - } else { - const BYTE* const matchEnd = matchIndex < prefixStartIndex ? dictEnd : iend; - const BYTE* const lowMatchPtr = matchIndex < prefixStartIndex ? dictStart : prefixStart; - mLength = ZSTD_count_2segments(ip+4, match+4, iend, matchEnd, prefixStart) + 4; - offset = current - matchIndex; - while (((ip>anchor) & (match>lowMatchPtr)) && (ip[-1] == match[-1])) { ip--; match--; mLength++; } /* catch up */ - } - offset_2 = offset_1; - offset_1 = offset; - ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH); - - } else { - ip += ((ip-anchor) >> kSearchStrength) + 1; - continue; - } } - - /* move to next sequence start */ - ip += mLength; - anchor = ip; - - if (ip <= ilimit) { - /* Complementary insertion */ - /* done after iLimit test, as candidates could be > iend-8 */ - { U32 const indexToInsert = current+2; - hashLong[ZSTD_hashPtr(base+indexToInsert, hBitsL, 8)] = indexToInsert; - hashLong[ZSTD_hashPtr(ip-2, hBitsL, 8)] = (U32)(ip-2-base); - hashSmall[ZSTD_hashPtr(base+indexToInsert, hBitsS, mls)] = indexToInsert; - hashSmall[ZSTD_hashPtr(ip-1, hBitsS, mls)] = (U32)(ip-1-base); - } - - /* check immediate repcode */ - while (ip <= ilimit) { - U32 const current2 = (U32)(ip-base); - U32 const repIndex2 = current2 - offset_2; - const BYTE* repMatch2 = repIndex2 < prefixStartIndex ? dictBase + repIndex2 : base + repIndex2; - if ( (((U32)((prefixStartIndex-1) - repIndex2) >= 3) /* intentional overflow : ensure repIndex2 doesn't overlap dict + prefix */ - & (repIndex2 > dictStartIndex)) - && (MEM_read32(repMatch2) == MEM_read32(ip)) ) { - const BYTE* const repEnd2 = repIndex2 < prefixStartIndex ? dictEnd : iend; - size_t const repLength2 = ZSTD_count_2segments(ip+4, repMatch2+4, iend, repEnd2, prefixStart) + 4; - U32 const tmpOffset = offset_2; offset_2 = offset_1; offset_1 = tmpOffset; /* swap offset_2 <=> offset_1 */ - ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, repLength2-MINMATCH); - hashSmall[ZSTD_hashPtr(ip, hBitsS, mls)] = current2; - hashLong[ZSTD_hashPtr(ip, hBitsL, 8)] = current2; - ip += repLength2; - anchor = ip; - continue; - } - break; - } } } - - /* save reps for next block */ - rep[0] = offset_1; - rep[1] = offset_2; - - /* Return the last literals size */ - return (size_t)(iend - anchor); -} - - -size_t ZSTD_compressBlock_doubleFast_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - U32 const mls = ms->cParams.minMatch; - switch(mls) - { - default: /* includes case 3 */ - case 4 : - return ZSTD_compressBlock_doubleFast_extDict_generic(ms, seqStore, rep, src, srcSize, 4); - case 5 : - return ZSTD_compressBlock_doubleFast_extDict_generic(ms, seqStore, rep, src, srcSize, 5); - case 6 : - return ZSTD_compressBlock_doubleFast_extDict_generic(ms, seqStore, rep, src, srcSize, 6); - case 7 : - return ZSTD_compressBlock_doubleFast_extDict_generic(ms, seqStore, rep, src, srcSize, 7); - } -} -/**** ended inlining compress/zstd_double_fast.c ****/ -/**** start inlining compress/zstd_fast.c ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -/**** skipping file: zstd_compress_internal.h ****/ -/**** skipping file: zstd_fast.h ****/ - - -void ZSTD_fillHashTable(ZSTD_matchState_t* ms, - const void* const end, - ZSTD_dictTableLoadMethod_e dtlm) -{ - const ZSTD_compressionParameters* const cParams = &ms->cParams; - U32* const hashTable = ms->hashTable; - U32 const hBits = cParams->hashLog; - U32 const mls = cParams->minMatch; - const BYTE* const base = ms->window.base; - const BYTE* ip = base + ms->nextToUpdate; - const BYTE* const iend = ((const BYTE*)end) - HASH_READ_SIZE; - const U32 fastHashFillStep = 3; - - /* Always insert every fastHashFillStep position into the hash table. - * Insert the other positions if their hash entry is empty. - */ - for ( ; ip + fastHashFillStep < iend + 2; ip += fastHashFillStep) { - U32 const current = (U32)(ip - base); - size_t const hash0 = ZSTD_hashPtr(ip, hBits, mls); - hashTable[hash0] = current; - if (dtlm == ZSTD_dtlm_fast) continue; - /* Only load extra positions for ZSTD_dtlm_full */ - { U32 p; - for (p = 1; p < fastHashFillStep; ++p) { - size_t const hash = ZSTD_hashPtr(ip + p, hBits, mls); - if (hashTable[hash] == 0) { /* not yet filled */ - hashTable[hash] = current + p; - } } } } -} - - -FORCE_INLINE_TEMPLATE size_t -ZSTD_compressBlock_fast_generic( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize, - U32 const mls) -{ - const ZSTD_compressionParameters* const cParams = &ms->cParams; - U32* const hashTable = ms->hashTable; - U32 const hlog = cParams->hashLog; - /* support stepSize of 0 */ - size_t const stepSize = cParams->targetLength + !(cParams->targetLength) + 1; - const BYTE* const base = ms->window.base; - const BYTE* const istart = (const BYTE*)src; - /* We check ip0 (ip + 0) and ip1 (ip + 1) each loop */ - const BYTE* ip0 = istart; - const BYTE* ip1; - const BYTE* anchor = istart; - const U32 endIndex = (U32)((size_t)(istart - base) + srcSize); - const U32 prefixStartIndex = ZSTD_getLowestPrefixIndex(ms, endIndex, cParams->windowLog); - const BYTE* const prefixStart = base + prefixStartIndex; - const BYTE* const iend = istart + srcSize; - const BYTE* const ilimit = iend - HASH_READ_SIZE; - U32 offset_1=rep[0], offset_2=rep[1]; - U32 offsetSaved = 0; - - /* init */ - DEBUGLOG(5, "ZSTD_compressBlock_fast_generic"); - ip0 += (ip0 == prefixStart); - ip1 = ip0 + 1; - { U32 const current = (U32)(ip0 - base); - U32 const windowLow = ZSTD_getLowestPrefixIndex(ms, current, cParams->windowLog); - U32 const maxRep = current - windowLow; - if (offset_2 > maxRep) offsetSaved = offset_2, offset_2 = 0; - if (offset_1 > maxRep) offsetSaved = offset_1, offset_1 = 0; - } - - /* Main Search Loop */ -#ifdef __INTEL_COMPILER - /* From intel 'The vector pragma indicates that the loop should be - * vectorized if it is legal to do so'. Can be used together with - * #pragma ivdep (but have opted to exclude that because intel - * warns against using it).*/ - #pragma vector always -#endif - while (ip1 < ilimit) { /* < instead of <=, because check at ip0+2 */ - size_t mLength; - BYTE const* ip2 = ip0 + 2; - size_t const h0 = ZSTD_hashPtr(ip0, hlog, mls); - U32 const val0 = MEM_read32(ip0); - size_t const h1 = ZSTD_hashPtr(ip1, hlog, mls); - U32 const val1 = MEM_read32(ip1); - U32 const current0 = (U32)(ip0-base); - U32 const current1 = (U32)(ip1-base); - U32 const matchIndex0 = hashTable[h0]; - U32 const matchIndex1 = hashTable[h1]; - BYTE const* repMatch = ip2 - offset_1; - const BYTE* match0 = base + matchIndex0; - const BYTE* match1 = base + matchIndex1; - U32 offcode; - -#if defined(__aarch64__) - PREFETCH_L1(ip0+256); -#endif - - hashTable[h0] = current0; /* update hash table */ - hashTable[h1] = current1; /* update hash table */ - - assert(ip0 + 1 == ip1); - - if ((offset_1 > 0) & (MEM_read32(repMatch) == MEM_read32(ip2))) { - mLength = (ip2[-1] == repMatch[-1]) ? 1 : 0; - ip0 = ip2 - mLength; - match0 = repMatch - mLength; - mLength += 4; - offcode = 0; - goto _match; - } - if ((matchIndex0 > prefixStartIndex) && MEM_read32(match0) == val0) { - /* found a regular match */ - goto _offset; - } - if ((matchIndex1 > prefixStartIndex) && MEM_read32(match1) == val1) { - /* found a regular match after one literal */ - ip0 = ip1; - match0 = match1; - goto _offset; - } - { size_t const step = ((size_t)(ip0-anchor) >> (kSearchStrength - 1)) + stepSize; - assert(step >= 2); - ip0 += step; - ip1 += step; - continue; - } -_offset: /* Requires: ip0, match0 */ - /* Compute the offset code */ - offset_2 = offset_1; - offset_1 = (U32)(ip0-match0); - offcode = offset_1 + ZSTD_REP_MOVE; - mLength = 4; - /* Count the backwards match length */ - while (((ip0>anchor) & (match0>prefixStart)) - && (ip0[-1] == match0[-1])) { ip0--; match0--; mLength++; } /* catch up */ - -_match: /* Requires: ip0, match0, offcode */ - /* Count the forward length */ - mLength += ZSTD_count(ip0+mLength, match0+mLength, iend); - ZSTD_storeSeq(seqStore, (size_t)(ip0-anchor), anchor, iend, offcode, mLength-MINMATCH); - /* match found */ - ip0 += mLength; - anchor = ip0; - - if (ip0 <= ilimit) { - /* Fill Table */ - assert(base+current0+2 > istart); /* check base overflow */ - hashTable[ZSTD_hashPtr(base+current0+2, hlog, mls)] = current0+2; /* here because current+2 could be > iend-8 */ - hashTable[ZSTD_hashPtr(ip0-2, hlog, mls)] = (U32)(ip0-2-base); - - if (offset_2 > 0) { /* offset_2==0 means offset_2 is invalidated */ - while ( (ip0 <= ilimit) && (MEM_read32(ip0) == MEM_read32(ip0 - offset_2)) ) { - /* store sequence */ - size_t const rLength = ZSTD_count(ip0+4, ip0+4-offset_2, iend) + 4; - { U32 const tmpOff = offset_2; offset_2 = offset_1; offset_1 = tmpOff; } /* swap offset_2 <=> offset_1 */ - hashTable[ZSTD_hashPtr(ip0, hlog, mls)] = (U32)(ip0-base); - ip0 += rLength; - ZSTD_storeSeq(seqStore, 0 /*litLen*/, anchor, iend, 0 /*offCode*/, rLength-MINMATCH); - anchor = ip0; - continue; /* faster when present (confirmed on gcc-8) ... (?) */ - } } } - ip1 = ip0 + 1; - } - - /* save reps for next block */ - rep[0] = offset_1 ? offset_1 : offsetSaved; - rep[1] = offset_2 ? offset_2 : offsetSaved; - - /* Return the last literals size */ - return (size_t)(iend - anchor); -} - - -size_t ZSTD_compressBlock_fast( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - U32 const mls = ms->cParams.minMatch; - assert(ms->dictMatchState == NULL); - switch(mls) - { - default: /* includes case 3 */ - case 4 : - return ZSTD_compressBlock_fast_generic(ms, seqStore, rep, src, srcSize, 4); - case 5 : - return ZSTD_compressBlock_fast_generic(ms, seqStore, rep, src, srcSize, 5); - case 6 : - return ZSTD_compressBlock_fast_generic(ms, seqStore, rep, src, srcSize, 6); - case 7 : - return ZSTD_compressBlock_fast_generic(ms, seqStore, rep, src, srcSize, 7); - } -} - -FORCE_INLINE_TEMPLATE -size_t ZSTD_compressBlock_fast_dictMatchState_generic( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize, U32 const mls) -{ - const ZSTD_compressionParameters* const cParams = &ms->cParams; - U32* const hashTable = ms->hashTable; - U32 const hlog = cParams->hashLog; - /* support stepSize of 0 */ - U32 const stepSize = cParams->targetLength + !(cParams->targetLength); - const BYTE* const base = ms->window.base; - const BYTE* const istart = (const BYTE*)src; - const BYTE* ip = istart; - const BYTE* anchor = istart; - const U32 prefixStartIndex = ms->window.dictLimit; - const BYTE* const prefixStart = base + prefixStartIndex; - const BYTE* const iend = istart + srcSize; - const BYTE* const ilimit = iend - HASH_READ_SIZE; - U32 offset_1=rep[0], offset_2=rep[1]; - U32 offsetSaved = 0; - - const ZSTD_matchState_t* const dms = ms->dictMatchState; - const ZSTD_compressionParameters* const dictCParams = &dms->cParams ; - const U32* const dictHashTable = dms->hashTable; - const U32 dictStartIndex = dms->window.dictLimit; - const BYTE* const dictBase = dms->window.base; - const BYTE* const dictStart = dictBase + dictStartIndex; - const BYTE* const dictEnd = dms->window.nextSrc; - const U32 dictIndexDelta = prefixStartIndex - (U32)(dictEnd - dictBase); - const U32 dictAndPrefixLength = (U32)(ip - prefixStart + dictEnd - dictStart); - const U32 dictHLog = dictCParams->hashLog; - - /* if a dictionary is still attached, it necessarily means that - * it is within window size. So we just check it. */ - const U32 maxDistance = 1U << cParams->windowLog; - const U32 endIndex = (U32)((size_t)(ip - base) + srcSize); - assert(endIndex - prefixStartIndex <= maxDistance); - (void)maxDistance; (void)endIndex; /* these variables are not used when assert() is disabled */ - - /* ensure there will be no no underflow - * when translating a dict index into a local index */ - assert(prefixStartIndex >= (U32)(dictEnd - dictBase)); - - /* init */ - DEBUGLOG(5, "ZSTD_compressBlock_fast_dictMatchState_generic"); - ip += (dictAndPrefixLength == 0); - /* dictMatchState repCode checks don't currently handle repCode == 0 - * disabling. */ - assert(offset_1 <= dictAndPrefixLength); - assert(offset_2 <= dictAndPrefixLength); - - /* Main Search Loop */ - while (ip < ilimit) { /* < instead of <=, because repcode check at (ip+1) */ - size_t mLength; - size_t const h = ZSTD_hashPtr(ip, hlog, mls); - U32 const current = (U32)(ip-base); - U32 const matchIndex = hashTable[h]; - const BYTE* match = base + matchIndex; - const U32 repIndex = current + 1 - offset_1; - const BYTE* repMatch = (repIndex < prefixStartIndex) ? - dictBase + (repIndex - dictIndexDelta) : - base + repIndex; - hashTable[h] = current; /* update hash table */ - - if ( ((U32)((prefixStartIndex-1) - repIndex) >= 3) /* intentional underflow : ensure repIndex isn't overlapping dict + prefix */ - && (MEM_read32(repMatch) == MEM_read32(ip+1)) ) { - const BYTE* const repMatchEnd = repIndex < prefixStartIndex ? dictEnd : iend; - mLength = ZSTD_count_2segments(ip+1+4, repMatch+4, iend, repMatchEnd, prefixStart) + 4; - ip++; - ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, mLength-MINMATCH); - } else if ( (matchIndex <= prefixStartIndex) ) { - size_t const dictHash = ZSTD_hashPtr(ip, dictHLog, mls); - U32 const dictMatchIndex = dictHashTable[dictHash]; - const BYTE* dictMatch = dictBase + dictMatchIndex; - if (dictMatchIndex <= dictStartIndex || - MEM_read32(dictMatch) != MEM_read32(ip)) { - assert(stepSize >= 1); - ip += ((ip-anchor) >> kSearchStrength) + stepSize; - continue; - } else { - /* found a dict match */ - U32 const offset = (U32)(current-dictMatchIndex-dictIndexDelta); - mLength = ZSTD_count_2segments(ip+4, dictMatch+4, iend, dictEnd, prefixStart) + 4; - while (((ip>anchor) & (dictMatch>dictStart)) - && (ip[-1] == dictMatch[-1])) { - ip--; dictMatch--; mLength++; - } /* catch up */ - offset_2 = offset_1; - offset_1 = offset; - ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH); - } - } else if (MEM_read32(match) != MEM_read32(ip)) { - /* it's not a match, and we're not going to check the dictionary */ - assert(stepSize >= 1); - ip += ((ip-anchor) >> kSearchStrength) + stepSize; - continue; - } else { - /* found a regular match */ - U32 const offset = (U32)(ip-match); - mLength = ZSTD_count(ip+4, match+4, iend) + 4; - while (((ip>anchor) & (match>prefixStart)) - && (ip[-1] == match[-1])) { ip--; match--; mLength++; } /* catch up */ - offset_2 = offset_1; - offset_1 = offset; - ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH); - } - - /* match found */ - ip += mLength; - anchor = ip; - - if (ip <= ilimit) { - /* Fill Table */ - assert(base+current+2 > istart); /* check base overflow */ - hashTable[ZSTD_hashPtr(base+current+2, hlog, mls)] = current+2; /* here because current+2 could be > iend-8 */ - hashTable[ZSTD_hashPtr(ip-2, hlog, mls)] = (U32)(ip-2-base); - - /* check immediate repcode */ - while (ip <= ilimit) { - U32 const current2 = (U32)(ip-base); - U32 const repIndex2 = current2 - offset_2; - const BYTE* repMatch2 = repIndex2 < prefixStartIndex ? - dictBase - dictIndexDelta + repIndex2 : - base + repIndex2; - if ( ((U32)((prefixStartIndex-1) - (U32)repIndex2) >= 3 /* intentional overflow */) - && (MEM_read32(repMatch2) == MEM_read32(ip)) ) { - const BYTE* const repEnd2 = repIndex2 < prefixStartIndex ? dictEnd : iend; - size_t const repLength2 = ZSTD_count_2segments(ip+4, repMatch2+4, iend, repEnd2, prefixStart) + 4; - U32 tmpOffset = offset_2; offset_2 = offset_1; offset_1 = tmpOffset; /* swap offset_2 <=> offset_1 */ - ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, repLength2-MINMATCH); - hashTable[ZSTD_hashPtr(ip, hlog, mls)] = current2; - ip += repLength2; - anchor = ip; - continue; - } - break; - } - } - } - - /* save reps for next block */ - rep[0] = offset_1 ? offset_1 : offsetSaved; - rep[1] = offset_2 ? offset_2 : offsetSaved; - - /* Return the last literals size */ - return (size_t)(iend - anchor); -} - -size_t ZSTD_compressBlock_fast_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - U32 const mls = ms->cParams.minMatch; - assert(ms->dictMatchState != NULL); - switch(mls) - { - default: /* includes case 3 */ - case 4 : - return ZSTD_compressBlock_fast_dictMatchState_generic(ms, seqStore, rep, src, srcSize, 4); - case 5 : - return ZSTD_compressBlock_fast_dictMatchState_generic(ms, seqStore, rep, src, srcSize, 5); - case 6 : - return ZSTD_compressBlock_fast_dictMatchState_generic(ms, seqStore, rep, src, srcSize, 6); - case 7 : - return ZSTD_compressBlock_fast_dictMatchState_generic(ms, seqStore, rep, src, srcSize, 7); - } -} - - -static size_t ZSTD_compressBlock_fast_extDict_generic( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize, U32 const mls) -{ - const ZSTD_compressionParameters* const cParams = &ms->cParams; - U32* const hashTable = ms->hashTable; - U32 const hlog = cParams->hashLog; - /* support stepSize of 0 */ - U32 const stepSize = cParams->targetLength + !(cParams->targetLength); - const BYTE* const base = ms->window.base; - const BYTE* const dictBase = ms->window.dictBase; - const BYTE* const istart = (const BYTE*)src; - const BYTE* ip = istart; - const BYTE* anchor = istart; - const U32 endIndex = (U32)((size_t)(istart - base) + srcSize); - const U32 lowLimit = ZSTD_getLowestMatchIndex(ms, endIndex, cParams->windowLog); - const U32 dictStartIndex = lowLimit; - const BYTE* const dictStart = dictBase + dictStartIndex; - const U32 dictLimit = ms->window.dictLimit; - const U32 prefixStartIndex = dictLimit < lowLimit ? lowLimit : dictLimit; - const BYTE* const prefixStart = base + prefixStartIndex; - const BYTE* const dictEnd = dictBase + prefixStartIndex; - const BYTE* const iend = istart + srcSize; - const BYTE* const ilimit = iend - 8; - U32 offset_1=rep[0], offset_2=rep[1]; - - DEBUGLOG(5, "ZSTD_compressBlock_fast_extDict_generic (offset_1=%u)", offset_1); - - /* switch to "regular" variant if extDict is invalidated due to maxDistance */ - if (prefixStartIndex == dictStartIndex) - return ZSTD_compressBlock_fast_generic(ms, seqStore, rep, src, srcSize, mls); - - /* Search Loop */ - while (ip < ilimit) { /* < instead of <=, because (ip+1) */ - const size_t h = ZSTD_hashPtr(ip, hlog, mls); - const U32 matchIndex = hashTable[h]; - const BYTE* const matchBase = matchIndex < prefixStartIndex ? dictBase : base; - const BYTE* match = matchBase + matchIndex; - const U32 current = (U32)(ip-base); - const U32 repIndex = current + 1 - offset_1; - const BYTE* const repBase = repIndex < prefixStartIndex ? dictBase : base; - const BYTE* const repMatch = repBase + repIndex; - hashTable[h] = current; /* update hash table */ - DEBUGLOG(7, "offset_1 = %u , current = %u", offset_1, current); - assert(offset_1 <= current +1); /* check repIndex */ - - if ( (((U32)((prefixStartIndex-1) - repIndex) >= 3) /* intentional underflow */ & (repIndex > dictStartIndex)) - && (MEM_read32(repMatch) == MEM_read32(ip+1)) ) { - const BYTE* const repMatchEnd = repIndex < prefixStartIndex ? dictEnd : iend; - size_t const rLength = ZSTD_count_2segments(ip+1 +4, repMatch +4, iend, repMatchEnd, prefixStart) + 4; - ip++; - ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, rLength-MINMATCH); - ip += rLength; - anchor = ip; - } else { - if ( (matchIndex < dictStartIndex) || - (MEM_read32(match) != MEM_read32(ip)) ) { - assert(stepSize >= 1); - ip += ((ip-anchor) >> kSearchStrength) + stepSize; - continue; - } - { const BYTE* const matchEnd = matchIndex < prefixStartIndex ? dictEnd : iend; - const BYTE* const lowMatchPtr = matchIndex < prefixStartIndex ? dictStart : prefixStart; - U32 const offset = current - matchIndex; - size_t mLength = ZSTD_count_2segments(ip+4, match+4, iend, matchEnd, prefixStart) + 4; - while (((ip>anchor) & (match>lowMatchPtr)) && (ip[-1] == match[-1])) { ip--; match--; mLength++; } /* catch up */ - offset_2 = offset_1; offset_1 = offset; /* update offset history */ - ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH); - ip += mLength; - anchor = ip; - } } - - if (ip <= ilimit) { - /* Fill Table */ - hashTable[ZSTD_hashPtr(base+current+2, hlog, mls)] = current+2; - hashTable[ZSTD_hashPtr(ip-2, hlog, mls)] = (U32)(ip-2-base); - /* check immediate repcode */ - while (ip <= ilimit) { - U32 const current2 = (U32)(ip-base); - U32 const repIndex2 = current2 - offset_2; - const BYTE* const repMatch2 = repIndex2 < prefixStartIndex ? dictBase + repIndex2 : base + repIndex2; - if ( (((U32)((prefixStartIndex-1) - repIndex2) >= 3) & (repIndex2 > dictStartIndex)) /* intentional overflow */ - && (MEM_read32(repMatch2) == MEM_read32(ip)) ) { - const BYTE* const repEnd2 = repIndex2 < prefixStartIndex ? dictEnd : iend; - size_t const repLength2 = ZSTD_count_2segments(ip+4, repMatch2+4, iend, repEnd2, prefixStart) + 4; - { U32 const tmpOffset = offset_2; offset_2 = offset_1; offset_1 = tmpOffset; } /* swap offset_2 <=> offset_1 */ - ZSTD_storeSeq(seqStore, 0 /*litlen*/, anchor, iend, 0 /*offcode*/, repLength2-MINMATCH); - hashTable[ZSTD_hashPtr(ip, hlog, mls)] = current2; - ip += repLength2; - anchor = ip; - continue; - } - break; - } } } - - /* save reps for next block */ - rep[0] = offset_1; - rep[1] = offset_2; - - /* Return the last literals size */ - return (size_t)(iend - anchor); -} - - -size_t ZSTD_compressBlock_fast_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - U32 const mls = ms->cParams.minMatch; - switch(mls) - { - default: /* includes case 3 */ - case 4 : - return ZSTD_compressBlock_fast_extDict_generic(ms, seqStore, rep, src, srcSize, 4); - case 5 : - return ZSTD_compressBlock_fast_extDict_generic(ms, seqStore, rep, src, srcSize, 5); - case 6 : - return ZSTD_compressBlock_fast_extDict_generic(ms, seqStore, rep, src, srcSize, 6); - case 7 : - return ZSTD_compressBlock_fast_extDict_generic(ms, seqStore, rep, src, srcSize, 7); - } -} -/**** ended inlining compress/zstd_fast.c ****/ -/**** start inlining compress/zstd_lazy.c ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -/**** skipping file: zstd_compress_internal.h ****/ -/**** skipping file: zstd_lazy.h ****/ - - -/*-************************************* -* Binary Tree search -***************************************/ - -static void -ZSTD_updateDUBT(ZSTD_matchState_t* ms, - const BYTE* ip, const BYTE* iend, - U32 mls) -{ - const ZSTD_compressionParameters* const cParams = &ms->cParams; - U32* const hashTable = ms->hashTable; - U32 const hashLog = cParams->hashLog; - - U32* const bt = ms->chainTable; - U32 const btLog = cParams->chainLog - 1; - U32 const btMask = (1 << btLog) - 1; - - const BYTE* const base = ms->window.base; - U32 const target = (U32)(ip - base); - U32 idx = ms->nextToUpdate; - - if (idx != target) - DEBUGLOG(7, "ZSTD_updateDUBT, from %u to %u (dictLimit:%u)", - idx, target, ms->window.dictLimit); - assert(ip + 8 <= iend); /* condition for ZSTD_hashPtr */ - (void)iend; - - assert(idx >= ms->window.dictLimit); /* condition for valid base+idx */ - for ( ; idx < target ; idx++) { - size_t const h = ZSTD_hashPtr(base + idx, hashLog, mls); /* assumption : ip + 8 <= iend */ - U32 const matchIndex = hashTable[h]; - - U32* const nextCandidatePtr = bt + 2*(idx&btMask); - U32* const sortMarkPtr = nextCandidatePtr + 1; - - DEBUGLOG(8, "ZSTD_updateDUBT: insert %u", idx); - hashTable[h] = idx; /* Update Hash Table */ - *nextCandidatePtr = matchIndex; /* update BT like a chain */ - *sortMarkPtr = ZSTD_DUBT_UNSORTED_MARK; - } - ms->nextToUpdate = target; -} - - -/** ZSTD_insertDUBT1() : - * sort one already inserted but unsorted position - * assumption : current >= btlow == (current - btmask) - * doesn't fail */ -static void -ZSTD_insertDUBT1(ZSTD_matchState_t* ms, - U32 current, const BYTE* inputEnd, - U32 nbCompares, U32 btLow, - const ZSTD_dictMode_e dictMode) -{ - const ZSTD_compressionParameters* const cParams = &ms->cParams; - U32* const bt = ms->chainTable; - U32 const btLog = cParams->chainLog - 1; - U32 const btMask = (1 << btLog) - 1; - size_t commonLengthSmaller=0, commonLengthLarger=0; - const BYTE* const base = ms->window.base; - const BYTE* const dictBase = ms->window.dictBase; - const U32 dictLimit = ms->window.dictLimit; - const BYTE* const ip = (current>=dictLimit) ? base + current : dictBase + current; - const BYTE* const iend = (current>=dictLimit) ? inputEnd : dictBase + dictLimit; - const BYTE* const dictEnd = dictBase + dictLimit; - const BYTE* const prefixStart = base + dictLimit; - const BYTE* match; - U32* smallerPtr = bt + 2*(current&btMask); - U32* largerPtr = smallerPtr + 1; - U32 matchIndex = *smallerPtr; /* this candidate is unsorted : next sorted candidate is reached through *smallerPtr, while *largerPtr contains previous unsorted candidate (which is already saved and can be overwritten) */ - U32 dummy32; /* to be nullified at the end */ - U32 const windowValid = ms->window.lowLimit; - U32 const maxDistance = 1U << cParams->windowLog; - U32 const windowLow = (current - windowValid > maxDistance) ? current - maxDistance : windowValid; - - - DEBUGLOG(8, "ZSTD_insertDUBT1(%u) (dictLimit=%u, lowLimit=%u)", - current, dictLimit, windowLow); - assert(current >= btLow); - assert(ip < iend); /* condition for ZSTD_count */ - - while (nbCompares-- && (matchIndex > windowLow)) { - U32* const nextPtr = bt + 2*(matchIndex & btMask); - size_t matchLength = MIN(commonLengthSmaller, commonLengthLarger); /* guaranteed minimum nb of common bytes */ - assert(matchIndex < current); - /* note : all candidates are now supposed sorted, - * but it's still possible to have nextPtr[1] == ZSTD_DUBT_UNSORTED_MARK - * when a real index has the same value as ZSTD_DUBT_UNSORTED_MARK */ - - if ( (dictMode != ZSTD_extDict) - || (matchIndex+matchLength >= dictLimit) /* both in current segment*/ - || (current < dictLimit) /* both in extDict */) { - const BYTE* const mBase = ( (dictMode != ZSTD_extDict) - || (matchIndex+matchLength >= dictLimit)) ? - base : dictBase; - assert( (matchIndex+matchLength >= dictLimit) /* might be wrong if extDict is incorrectly set to 0 */ - || (current < dictLimit) ); - match = mBase + matchIndex; - matchLength += ZSTD_count(ip+matchLength, match+matchLength, iend); - } else { - match = dictBase + matchIndex; - matchLength += ZSTD_count_2segments(ip+matchLength, match+matchLength, iend, dictEnd, prefixStart); - if (matchIndex+matchLength >= dictLimit) - match = base + matchIndex; /* preparation for next read of match[matchLength] */ - } - - DEBUGLOG(8, "ZSTD_insertDUBT1: comparing %u with %u : found %u common bytes ", - current, matchIndex, (U32)matchLength); - - if (ip+matchLength == iend) { /* equal : no way to know if inf or sup */ - break; /* drop , to guarantee consistency ; miss a bit of compression, but other solutions can corrupt tree */ - } - - if (match[matchLength] < ip[matchLength]) { /* necessarily within buffer */ - /* match is smaller than current */ - *smallerPtr = matchIndex; /* update smaller idx */ - commonLengthSmaller = matchLength; /* all smaller will now have at least this guaranteed common length */ - if (matchIndex <= btLow) { smallerPtr=&dummy32; break; } /* beyond tree size, stop searching */ - DEBUGLOG(8, "ZSTD_insertDUBT1: %u (>btLow=%u) is smaller : next => %u", - matchIndex, btLow, nextPtr[1]); - smallerPtr = nextPtr+1; /* new "candidate" => larger than match, which was smaller than target */ - matchIndex = nextPtr[1]; /* new matchIndex, larger than previous and closer to current */ - } else { - /* match is larger than current */ - *largerPtr = matchIndex; - commonLengthLarger = matchLength; - if (matchIndex <= btLow) { largerPtr=&dummy32; break; } /* beyond tree size, stop searching */ - DEBUGLOG(8, "ZSTD_insertDUBT1: %u (>btLow=%u) is larger => %u", - matchIndex, btLow, nextPtr[0]); - largerPtr = nextPtr; - matchIndex = nextPtr[0]; - } } - - *smallerPtr = *largerPtr = 0; -} - - -static size_t -ZSTD_DUBT_findBetterDictMatch ( - ZSTD_matchState_t* ms, - const BYTE* const ip, const BYTE* const iend, - size_t* offsetPtr, - size_t bestLength, - U32 nbCompares, - U32 const mls, - const ZSTD_dictMode_e dictMode) -{ - const ZSTD_matchState_t * const dms = ms->dictMatchState; - const ZSTD_compressionParameters* const dmsCParams = &dms->cParams; - const U32 * const dictHashTable = dms->hashTable; - U32 const hashLog = dmsCParams->hashLog; - size_t const h = ZSTD_hashPtr(ip, hashLog, mls); - U32 dictMatchIndex = dictHashTable[h]; - - const BYTE* const base = ms->window.base; - const BYTE* const prefixStart = base + ms->window.dictLimit; - U32 const current = (U32)(ip-base); - const BYTE* const dictBase = dms->window.base; - const BYTE* const dictEnd = dms->window.nextSrc; - U32 const dictHighLimit = (U32)(dms->window.nextSrc - dms->window.base); - U32 const dictLowLimit = dms->window.lowLimit; - U32 const dictIndexDelta = ms->window.lowLimit - dictHighLimit; - - U32* const dictBt = dms->chainTable; - U32 const btLog = dmsCParams->chainLog - 1; - U32 const btMask = (1 << btLog) - 1; - U32 const btLow = (btMask >= dictHighLimit - dictLowLimit) ? dictLowLimit : dictHighLimit - btMask; - - size_t commonLengthSmaller=0, commonLengthLarger=0; - - (void)dictMode; - assert(dictMode == ZSTD_dictMatchState); - - while (nbCompares-- && (dictMatchIndex > dictLowLimit)) { - U32* const nextPtr = dictBt + 2*(dictMatchIndex & btMask); - size_t matchLength = MIN(commonLengthSmaller, commonLengthLarger); /* guaranteed minimum nb of common bytes */ - const BYTE* match = dictBase + dictMatchIndex; - matchLength += ZSTD_count_2segments(ip+matchLength, match+matchLength, iend, dictEnd, prefixStart); - if (dictMatchIndex+matchLength >= dictHighLimit) - match = base + dictMatchIndex + dictIndexDelta; /* to prepare for next usage of match[matchLength] */ - - if (matchLength > bestLength) { - U32 matchIndex = dictMatchIndex + dictIndexDelta; - if ( (4*(int)(matchLength-bestLength)) > (int)(ZSTD_highbit32(current-matchIndex+1) - ZSTD_highbit32((U32)offsetPtr[0]+1)) ) { - DEBUGLOG(9, "ZSTD_DUBT_findBetterDictMatch(%u) : found better match length %u -> %u and offsetCode %u -> %u (dictMatchIndex %u, matchIndex %u)", - current, (U32)bestLength, (U32)matchLength, (U32)*offsetPtr, ZSTD_REP_MOVE + current - matchIndex, dictMatchIndex, matchIndex); - bestLength = matchLength, *offsetPtr = ZSTD_REP_MOVE + current - matchIndex; - } - if (ip+matchLength == iend) { /* reached end of input : ip[matchLength] is not valid, no way to know if it's larger or smaller than match */ - break; /* drop, to guarantee consistency (miss a little bit of compression) */ - } - } - - if (match[matchLength] < ip[matchLength]) { - if (dictMatchIndex <= btLow) { break; } /* beyond tree size, stop the search */ - commonLengthSmaller = matchLength; /* all smaller will now have at least this guaranteed common length */ - dictMatchIndex = nextPtr[1]; /* new matchIndex larger than previous (closer to current) */ - } else { - /* match is larger than current */ - if (dictMatchIndex <= btLow) { break; } /* beyond tree size, stop the search */ - commonLengthLarger = matchLength; - dictMatchIndex = nextPtr[0]; - } - } - - if (bestLength >= MINMATCH) { - U32 const mIndex = current - ((U32)*offsetPtr - ZSTD_REP_MOVE); (void)mIndex; - DEBUGLOG(8, "ZSTD_DUBT_findBetterDictMatch(%u) : found match of length %u and offsetCode %u (pos %u)", - current, (U32)bestLength, (U32)*offsetPtr, mIndex); - } - return bestLength; - -} - - -static size_t -ZSTD_DUBT_findBestMatch(ZSTD_matchState_t* ms, - const BYTE* const ip, const BYTE* const iend, - size_t* offsetPtr, - U32 const mls, - const ZSTD_dictMode_e dictMode) -{ - const ZSTD_compressionParameters* const cParams = &ms->cParams; - U32* const hashTable = ms->hashTable; - U32 const hashLog = cParams->hashLog; - size_t const h = ZSTD_hashPtr(ip, hashLog, mls); - U32 matchIndex = hashTable[h]; - - const BYTE* const base = ms->window.base; - U32 const current = (U32)(ip-base); - U32 const windowLow = ZSTD_getLowestMatchIndex(ms, current, cParams->windowLog); - - U32* const bt = ms->chainTable; - U32 const btLog = cParams->chainLog - 1; - U32 const btMask = (1 << btLog) - 1; - U32 const btLow = (btMask >= current) ? 0 : current - btMask; - U32 const unsortLimit = MAX(btLow, windowLow); - - U32* nextCandidate = bt + 2*(matchIndex&btMask); - U32* unsortedMark = bt + 2*(matchIndex&btMask) + 1; - U32 nbCompares = 1U << cParams->searchLog; - U32 nbCandidates = nbCompares; - U32 previousCandidate = 0; - - DEBUGLOG(7, "ZSTD_DUBT_findBestMatch (%u) ", current); - assert(ip <= iend-8); /* required for h calculation */ - - /* reach end of unsorted candidates list */ - while ( (matchIndex > unsortLimit) - && (*unsortedMark == ZSTD_DUBT_UNSORTED_MARK) - && (nbCandidates > 1) ) { - DEBUGLOG(8, "ZSTD_DUBT_findBestMatch: candidate %u is unsorted", - matchIndex); - *unsortedMark = previousCandidate; /* the unsortedMark becomes a reversed chain, to move up back to original position */ - previousCandidate = matchIndex; - matchIndex = *nextCandidate; - nextCandidate = bt + 2*(matchIndex&btMask); - unsortedMark = bt + 2*(matchIndex&btMask) + 1; - nbCandidates --; - } - - /* nullify last candidate if it's still unsorted - * simplification, detrimental to compression ratio, beneficial for speed */ - if ( (matchIndex > unsortLimit) - && (*unsortedMark==ZSTD_DUBT_UNSORTED_MARK) ) { - DEBUGLOG(7, "ZSTD_DUBT_findBestMatch: nullify last unsorted candidate %u", - matchIndex); - *nextCandidate = *unsortedMark = 0; - } - - /* batch sort stacked candidates */ - matchIndex = previousCandidate; - while (matchIndex) { /* will end on matchIndex == 0 */ - U32* const nextCandidateIdxPtr = bt + 2*(matchIndex&btMask) + 1; - U32 const nextCandidateIdx = *nextCandidateIdxPtr; - ZSTD_insertDUBT1(ms, matchIndex, iend, - nbCandidates, unsortLimit, dictMode); - matchIndex = nextCandidateIdx; - nbCandidates++; - } - - /* find longest match */ - { size_t commonLengthSmaller = 0, commonLengthLarger = 0; - const BYTE* const dictBase = ms->window.dictBase; - const U32 dictLimit = ms->window.dictLimit; - const BYTE* const dictEnd = dictBase + dictLimit; - const BYTE* const prefixStart = base + dictLimit; - U32* smallerPtr = bt + 2*(current&btMask); - U32* largerPtr = bt + 2*(current&btMask) + 1; - U32 matchEndIdx = current + 8 + 1; - U32 dummy32; /* to be nullified at the end */ - size_t bestLength = 0; - - matchIndex = hashTable[h]; - hashTable[h] = current; /* Update Hash Table */ - - while (nbCompares-- && (matchIndex > windowLow)) { - U32* const nextPtr = bt + 2*(matchIndex & btMask); - size_t matchLength = MIN(commonLengthSmaller, commonLengthLarger); /* guaranteed minimum nb of common bytes */ - const BYTE* match; - - if ((dictMode != ZSTD_extDict) || (matchIndex+matchLength >= dictLimit)) { - match = base + matchIndex; - matchLength += ZSTD_count(ip+matchLength, match+matchLength, iend); - } else { - match = dictBase + matchIndex; - matchLength += ZSTD_count_2segments(ip+matchLength, match+matchLength, iend, dictEnd, prefixStart); - if (matchIndex+matchLength >= dictLimit) - match = base + matchIndex; /* to prepare for next usage of match[matchLength] */ - } - - if (matchLength > bestLength) { - if (matchLength > matchEndIdx - matchIndex) - matchEndIdx = matchIndex + (U32)matchLength; - if ( (4*(int)(matchLength-bestLength)) > (int)(ZSTD_highbit32(current-matchIndex+1) - ZSTD_highbit32((U32)offsetPtr[0]+1)) ) - bestLength = matchLength, *offsetPtr = ZSTD_REP_MOVE + current - matchIndex; - if (ip+matchLength == iend) { /* equal : no way to know if inf or sup */ - if (dictMode == ZSTD_dictMatchState) { - nbCompares = 0; /* in addition to avoiding checking any - * further in this loop, make sure we - * skip checking in the dictionary. */ - } - break; /* drop, to guarantee consistency (miss a little bit of compression) */ - } - } - - if (match[matchLength] < ip[matchLength]) { - /* match is smaller than current */ - *smallerPtr = matchIndex; /* update smaller idx */ - commonLengthSmaller = matchLength; /* all smaller will now have at least this guaranteed common length */ - if (matchIndex <= btLow) { smallerPtr=&dummy32; break; } /* beyond tree size, stop the search */ - smallerPtr = nextPtr+1; /* new "smaller" => larger of match */ - matchIndex = nextPtr[1]; /* new matchIndex larger than previous (closer to current) */ - } else { - /* match is larger than current */ - *largerPtr = matchIndex; - commonLengthLarger = matchLength; - if (matchIndex <= btLow) { largerPtr=&dummy32; break; } /* beyond tree size, stop the search */ - largerPtr = nextPtr; - matchIndex = nextPtr[0]; - } } - - *smallerPtr = *largerPtr = 0; - - if (dictMode == ZSTD_dictMatchState && nbCompares) { - bestLength = ZSTD_DUBT_findBetterDictMatch( - ms, ip, iend, - offsetPtr, bestLength, nbCompares, - mls, dictMode); - } - - assert(matchEndIdx > current+8); /* ensure nextToUpdate is increased */ - ms->nextToUpdate = matchEndIdx - 8; /* skip repetitive patterns */ - if (bestLength >= MINMATCH) { - U32 const mIndex = current - ((U32)*offsetPtr - ZSTD_REP_MOVE); (void)mIndex; - DEBUGLOG(8, "ZSTD_DUBT_findBestMatch(%u) : found match of length %u and offsetCode %u (pos %u)", - current, (U32)bestLength, (U32)*offsetPtr, mIndex); - } - return bestLength; - } -} - - -/** ZSTD_BtFindBestMatch() : Tree updater, providing best match */ -FORCE_INLINE_TEMPLATE size_t -ZSTD_BtFindBestMatch( ZSTD_matchState_t* ms, - const BYTE* const ip, const BYTE* const iLimit, - size_t* offsetPtr, - const U32 mls /* template */, - const ZSTD_dictMode_e dictMode) -{ - DEBUGLOG(7, "ZSTD_BtFindBestMatch"); - if (ip < ms->window.base + ms->nextToUpdate) return 0; /* skipped area */ - ZSTD_updateDUBT(ms, ip, iLimit, mls); - return ZSTD_DUBT_findBestMatch(ms, ip, iLimit, offsetPtr, mls, dictMode); -} - - -static size_t -ZSTD_BtFindBestMatch_selectMLS ( ZSTD_matchState_t* ms, - const BYTE* ip, const BYTE* const iLimit, - size_t* offsetPtr) -{ - switch(ms->cParams.minMatch) - { - default : /* includes case 3 */ - case 4 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 4, ZSTD_noDict); - case 5 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 5, ZSTD_noDict); - case 7 : - case 6 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 6, ZSTD_noDict); - } -} - - -static size_t ZSTD_BtFindBestMatch_dictMatchState_selectMLS ( - ZSTD_matchState_t* ms, - const BYTE* ip, const BYTE* const iLimit, - size_t* offsetPtr) -{ - switch(ms->cParams.minMatch) - { - default : /* includes case 3 */ - case 4 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 4, ZSTD_dictMatchState); - case 5 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 5, ZSTD_dictMatchState); - case 7 : - case 6 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 6, ZSTD_dictMatchState); - } -} - - -static size_t ZSTD_BtFindBestMatch_extDict_selectMLS ( - ZSTD_matchState_t* ms, - const BYTE* ip, const BYTE* const iLimit, - size_t* offsetPtr) -{ - switch(ms->cParams.minMatch) - { - default : /* includes case 3 */ - case 4 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 4, ZSTD_extDict); - case 5 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 5, ZSTD_extDict); - case 7 : - case 6 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 6, ZSTD_extDict); - } -} - - - -/* ********************************* -* Hash Chain -***********************************/ -#define NEXT_IN_CHAIN(d, mask) chainTable[(d) & (mask)] - -/* Update chains up to ip (excluded) - Assumption : always within prefix (i.e. not within extDict) */ -static U32 ZSTD_insertAndFindFirstIndex_internal( - ZSTD_matchState_t* ms, - const ZSTD_compressionParameters* const cParams, - const BYTE* ip, U32 const mls) -{ - U32* const hashTable = ms->hashTable; - const U32 hashLog = cParams->hashLog; - U32* const chainTable = ms->chainTable; - const U32 chainMask = (1 << cParams->chainLog) - 1; - const BYTE* const base = ms->window.base; - const U32 target = (U32)(ip - base); - U32 idx = ms->nextToUpdate; - - while(idx < target) { /* catch up */ - size_t const h = ZSTD_hashPtr(base+idx, hashLog, mls); - NEXT_IN_CHAIN(idx, chainMask) = hashTable[h]; - hashTable[h] = idx; - idx++; - } - - ms->nextToUpdate = target; - return hashTable[ZSTD_hashPtr(ip, hashLog, mls)]; -} - -U32 ZSTD_insertAndFindFirstIndex(ZSTD_matchState_t* ms, const BYTE* ip) { - const ZSTD_compressionParameters* const cParams = &ms->cParams; - return ZSTD_insertAndFindFirstIndex_internal(ms, cParams, ip, ms->cParams.minMatch); -} - - -/* inlining is important to hardwire a hot branch (template emulation) */ -FORCE_INLINE_TEMPLATE -size_t ZSTD_HcFindBestMatch_generic ( - ZSTD_matchState_t* ms, - const BYTE* const ip, const BYTE* const iLimit, - size_t* offsetPtr, - const U32 mls, const ZSTD_dictMode_e dictMode) -{ - const ZSTD_compressionParameters* const cParams = &ms->cParams; - U32* const chainTable = ms->chainTable; - const U32 chainSize = (1 << cParams->chainLog); - const U32 chainMask = chainSize-1; - const BYTE* const base = ms->window.base; - const BYTE* const dictBase = ms->window.dictBase; - const U32 dictLimit = ms->window.dictLimit; - const BYTE* const prefixStart = base + dictLimit; - const BYTE* const dictEnd = dictBase + dictLimit; - const U32 current = (U32)(ip-base); - const U32 maxDistance = 1U << cParams->windowLog; - const U32 lowestValid = ms->window.lowLimit; - const U32 withinMaxDistance = (current - lowestValid > maxDistance) ? current - maxDistance : lowestValid; - const U32 isDictionary = (ms->loadedDictEnd != 0); - const U32 lowLimit = isDictionary ? lowestValid : withinMaxDistance; - const U32 minChain = current > chainSize ? current - chainSize : 0; - U32 nbAttempts = 1U << cParams->searchLog; - size_t ml=4-1; - - /* HC4 match finder */ - U32 matchIndex = ZSTD_insertAndFindFirstIndex_internal(ms, cParams, ip, mls); - - for ( ; (matchIndex>lowLimit) & (nbAttempts>0) ; nbAttempts--) { - size_t currentMl=0; - if ((dictMode != ZSTD_extDict) || matchIndex >= dictLimit) { - const BYTE* const match = base + matchIndex; - assert(matchIndex >= dictLimit); /* ensures this is true if dictMode != ZSTD_extDict */ - if (match[ml] == ip[ml]) /* potentially better */ - currentMl = ZSTD_count(ip, match, iLimit); - } else { - const BYTE* const match = dictBase + matchIndex; - assert(match+4 <= dictEnd); - if (MEM_read32(match) == MEM_read32(ip)) /* assumption : matchIndex <= dictLimit-4 (by table construction) */ - currentMl = ZSTD_count_2segments(ip+4, match+4, iLimit, dictEnd, prefixStart) + 4; - } - - /* save best solution */ - if (currentMl > ml) { - ml = currentMl; - *offsetPtr = current - matchIndex + ZSTD_REP_MOVE; - if (ip+currentMl == iLimit) break; /* best possible, avoids read overflow on next attempt */ - } - - if (matchIndex <= minChain) break; - matchIndex = NEXT_IN_CHAIN(matchIndex, chainMask); - } - - if (dictMode == ZSTD_dictMatchState) { - const ZSTD_matchState_t* const dms = ms->dictMatchState; - const U32* const dmsChainTable = dms->chainTable; - const U32 dmsChainSize = (1 << dms->cParams.chainLog); - const U32 dmsChainMask = dmsChainSize - 1; - const U32 dmsLowestIndex = dms->window.dictLimit; - const BYTE* const dmsBase = dms->window.base; - const BYTE* const dmsEnd = dms->window.nextSrc; - const U32 dmsSize = (U32)(dmsEnd - dmsBase); - const U32 dmsIndexDelta = dictLimit - dmsSize; - const U32 dmsMinChain = dmsSize > dmsChainSize ? dmsSize - dmsChainSize : 0; - - matchIndex = dms->hashTable[ZSTD_hashPtr(ip, dms->cParams.hashLog, mls)]; - - for ( ; (matchIndex>dmsLowestIndex) & (nbAttempts>0) ; nbAttempts--) { - size_t currentMl=0; - const BYTE* const match = dmsBase + matchIndex; - assert(match+4 <= dmsEnd); - if (MEM_read32(match) == MEM_read32(ip)) /* assumption : matchIndex <= dictLimit-4 (by table construction) */ - currentMl = ZSTD_count_2segments(ip+4, match+4, iLimit, dmsEnd, prefixStart) + 4; - - /* save best solution */ - if (currentMl > ml) { - ml = currentMl; - *offsetPtr = current - (matchIndex + dmsIndexDelta) + ZSTD_REP_MOVE; - if (ip+currentMl == iLimit) break; /* best possible, avoids read overflow on next attempt */ - } - - if (matchIndex <= dmsMinChain) break; - matchIndex = dmsChainTable[matchIndex & dmsChainMask]; - } - } - - return ml; -} - - -FORCE_INLINE_TEMPLATE size_t ZSTD_HcFindBestMatch_selectMLS ( - ZSTD_matchState_t* ms, - const BYTE* ip, const BYTE* const iLimit, - size_t* offsetPtr) -{ - switch(ms->cParams.minMatch) - { - default : /* includes case 3 */ - case 4 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 4, ZSTD_noDict); - case 5 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 5, ZSTD_noDict); - case 7 : - case 6 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 6, ZSTD_noDict); - } -} - - -static size_t ZSTD_HcFindBestMatch_dictMatchState_selectMLS ( - ZSTD_matchState_t* ms, - const BYTE* ip, const BYTE* const iLimit, - size_t* offsetPtr) -{ - switch(ms->cParams.minMatch) - { - default : /* includes case 3 */ - case 4 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 4, ZSTD_dictMatchState); - case 5 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 5, ZSTD_dictMatchState); - case 7 : - case 6 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 6, ZSTD_dictMatchState); - } -} - - -FORCE_INLINE_TEMPLATE size_t ZSTD_HcFindBestMatch_extDict_selectMLS ( - ZSTD_matchState_t* ms, - const BYTE* ip, const BYTE* const iLimit, - size_t* offsetPtr) -{ - switch(ms->cParams.minMatch) - { - default : /* includes case 3 */ - case 4 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 4, ZSTD_extDict); - case 5 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 5, ZSTD_extDict); - case 7 : - case 6 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 6, ZSTD_extDict); - } -} - - -/* ******************************* -* Common parser - lazy strategy -*********************************/ -typedef enum { search_hashChain, search_binaryTree } searchMethod_e; - -FORCE_INLINE_TEMPLATE size_t -ZSTD_compressBlock_lazy_generic( - ZSTD_matchState_t* ms, seqStore_t* seqStore, - U32 rep[ZSTD_REP_NUM], - const void* src, size_t srcSize, - const searchMethod_e searchMethod, const U32 depth, - ZSTD_dictMode_e const dictMode) -{ - const BYTE* const istart = (const BYTE*)src; - const BYTE* ip = istart; - const BYTE* anchor = istart; - const BYTE* const iend = istart + srcSize; - const BYTE* const ilimit = iend - 8; - const BYTE* const base = ms->window.base; - const U32 prefixLowestIndex = ms->window.dictLimit; - const BYTE* const prefixLowest = base + prefixLowestIndex; - - typedef size_t (*searchMax_f)( - ZSTD_matchState_t* ms, - const BYTE* ip, const BYTE* iLimit, size_t* offsetPtr); - searchMax_f const searchMax = dictMode == ZSTD_dictMatchState ? - (searchMethod==search_binaryTree ? ZSTD_BtFindBestMatch_dictMatchState_selectMLS - : ZSTD_HcFindBestMatch_dictMatchState_selectMLS) : - (searchMethod==search_binaryTree ? ZSTD_BtFindBestMatch_selectMLS - : ZSTD_HcFindBestMatch_selectMLS); - U32 offset_1 = rep[0], offset_2 = rep[1], savedOffset=0; - - const ZSTD_matchState_t* const dms = ms->dictMatchState; - const U32 dictLowestIndex = dictMode == ZSTD_dictMatchState ? - dms->window.dictLimit : 0; - const BYTE* const dictBase = dictMode == ZSTD_dictMatchState ? - dms->window.base : NULL; - const BYTE* const dictLowest = dictMode == ZSTD_dictMatchState ? - dictBase + dictLowestIndex : NULL; - const BYTE* const dictEnd = dictMode == ZSTD_dictMatchState ? - dms->window.nextSrc : NULL; - const U32 dictIndexDelta = dictMode == ZSTD_dictMatchState ? - prefixLowestIndex - (U32)(dictEnd - dictBase) : - 0; - const U32 dictAndPrefixLength = (U32)((ip - prefixLowest) + (dictEnd - dictLowest)); - - DEBUGLOG(5, "ZSTD_compressBlock_lazy_generic (dictMode=%u)", (U32)dictMode); - - /* init */ - ip += (dictAndPrefixLength == 0); - if (dictMode == ZSTD_noDict) { - U32 const current = (U32)(ip - base); - U32 const windowLow = ZSTD_getLowestPrefixIndex(ms, current, ms->cParams.windowLog); - U32 const maxRep = current - windowLow; - if (offset_2 > maxRep) savedOffset = offset_2, offset_2 = 0; - if (offset_1 > maxRep) savedOffset = offset_1, offset_1 = 0; - } - if (dictMode == ZSTD_dictMatchState) { - /* dictMatchState repCode checks don't currently handle repCode == 0 - * disabling. */ - assert(offset_1 <= dictAndPrefixLength); - assert(offset_2 <= dictAndPrefixLength); - } - - /* Match Loop */ -#if defined(__GNUC__) && defined(__x86_64__) - /* I've measured random a 5% speed loss on levels 5 & 6 (greedy) when the - * code alignment is perturbed. To fix the instability align the loop on 32-bytes. - */ - __asm__(".p2align 5"); -#endif - while (ip < ilimit) { - size_t matchLength=0; - size_t offset=0; - const BYTE* start=ip+1; - - /* check repCode */ - if (dictMode == ZSTD_dictMatchState) { - const U32 repIndex = (U32)(ip - base) + 1 - offset_1; - const BYTE* repMatch = (dictMode == ZSTD_dictMatchState - && repIndex < prefixLowestIndex) ? - dictBase + (repIndex - dictIndexDelta) : - base + repIndex; - if (((U32)((prefixLowestIndex-1) - repIndex) >= 3 /* intentional underflow */) - && (MEM_read32(repMatch) == MEM_read32(ip+1)) ) { - const BYTE* repMatchEnd = repIndex < prefixLowestIndex ? dictEnd : iend; - matchLength = ZSTD_count_2segments(ip+1+4, repMatch+4, iend, repMatchEnd, prefixLowest) + 4; - if (depth==0) goto _storeSequence; - } - } - if ( dictMode == ZSTD_noDict - && ((offset_1 > 0) & (MEM_read32(ip+1-offset_1) == MEM_read32(ip+1)))) { - matchLength = ZSTD_count(ip+1+4, ip+1+4-offset_1, iend) + 4; - if (depth==0) goto _storeSequence; - } - - /* first search (depth 0) */ - { size_t offsetFound = 999999999; - size_t const ml2 = searchMax(ms, ip, iend, &offsetFound); - if (ml2 > matchLength) - matchLength = ml2, start = ip, offset=offsetFound; - } - - if (matchLength < 4) { - ip += ((ip-anchor) >> kSearchStrength) + 1; /* jump faster over incompressible sections */ - continue; - } - - /* let's try to find a better solution */ - if (depth>=1) - while (ip0) & (MEM_read32(ip) == MEM_read32(ip - offset_1)))) { - size_t const mlRep = ZSTD_count(ip+4, ip+4-offset_1, iend) + 4; - int const gain2 = (int)(mlRep * 3); - int const gain1 = (int)(matchLength*3 - ZSTD_highbit32((U32)offset+1) + 1); - if ((mlRep >= 4) && (gain2 > gain1)) - matchLength = mlRep, offset = 0, start = ip; - } - if (dictMode == ZSTD_dictMatchState) { - const U32 repIndex = (U32)(ip - base) - offset_1; - const BYTE* repMatch = repIndex < prefixLowestIndex ? - dictBase + (repIndex - dictIndexDelta) : - base + repIndex; - if (((U32)((prefixLowestIndex-1) - repIndex) >= 3 /* intentional underflow */) - && (MEM_read32(repMatch) == MEM_read32(ip)) ) { - const BYTE* repMatchEnd = repIndex < prefixLowestIndex ? dictEnd : iend; - size_t const mlRep = ZSTD_count_2segments(ip+4, repMatch+4, iend, repMatchEnd, prefixLowest) + 4; - int const gain2 = (int)(mlRep * 3); - int const gain1 = (int)(matchLength*3 - ZSTD_highbit32((U32)offset+1) + 1); - if ((mlRep >= 4) && (gain2 > gain1)) - matchLength = mlRep, offset = 0, start = ip; - } - } - { size_t offset2=999999999; - size_t const ml2 = searchMax(ms, ip, iend, &offset2); - int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)offset2+1)); /* raw approx */ - int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 4); - if ((ml2 >= 4) && (gain2 > gain1)) { - matchLength = ml2, offset = offset2, start = ip; - continue; /* search a better one */ - } } - - /* let's find an even better one */ - if ((depth==2) && (ip0) & (MEM_read32(ip) == MEM_read32(ip - offset_1)))) { - size_t const mlRep = ZSTD_count(ip+4, ip+4-offset_1, iend) + 4; - int const gain2 = (int)(mlRep * 4); - int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 1); - if ((mlRep >= 4) && (gain2 > gain1)) - matchLength = mlRep, offset = 0, start = ip; - } - if (dictMode == ZSTD_dictMatchState) { - const U32 repIndex = (U32)(ip - base) - offset_1; - const BYTE* repMatch = repIndex < prefixLowestIndex ? - dictBase + (repIndex - dictIndexDelta) : - base + repIndex; - if (((U32)((prefixLowestIndex-1) - repIndex) >= 3 /* intentional underflow */) - && (MEM_read32(repMatch) == MEM_read32(ip)) ) { - const BYTE* repMatchEnd = repIndex < prefixLowestIndex ? dictEnd : iend; - size_t const mlRep = ZSTD_count_2segments(ip+4, repMatch+4, iend, repMatchEnd, prefixLowest) + 4; - int const gain2 = (int)(mlRep * 4); - int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 1); - if ((mlRep >= 4) && (gain2 > gain1)) - matchLength = mlRep, offset = 0, start = ip; - } - } - { size_t offset2=999999999; - size_t const ml2 = searchMax(ms, ip, iend, &offset2); - int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)offset2+1)); /* raw approx */ - int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 7); - if ((ml2 >= 4) && (gain2 > gain1)) { - matchLength = ml2, offset = offset2, start = ip; - continue; - } } } - break; /* nothing found : store previous solution */ - } - - /* NOTE: - * start[-offset+ZSTD_REP_MOVE-1] is undefined behavior. - * (-offset+ZSTD_REP_MOVE-1) is unsigned, and is added to start, which - * overflows the pointer, which is undefined behavior. - */ - /* catch up */ - if (offset) { - if (dictMode == ZSTD_noDict) { - while ( ((start > anchor) & (start - (offset-ZSTD_REP_MOVE) > prefixLowest)) - && (start[-1] == (start-(offset-ZSTD_REP_MOVE))[-1]) ) /* only search for offset within prefix */ - { start--; matchLength++; } - } - if (dictMode == ZSTD_dictMatchState) { - U32 const matchIndex = (U32)((start-base) - (offset - ZSTD_REP_MOVE)); - const BYTE* match = (matchIndex < prefixLowestIndex) ? dictBase + matchIndex - dictIndexDelta : base + matchIndex; - const BYTE* const mStart = (matchIndex < prefixLowestIndex) ? dictLowest : prefixLowest; - while ((start>anchor) && (match>mStart) && (start[-1] == match[-1])) { start--; match--; matchLength++; } /* catch up */ - } - offset_2 = offset_1; offset_1 = (U32)(offset - ZSTD_REP_MOVE); - } - /* store sequence */ -_storeSequence: - { size_t const litLength = start - anchor; - ZSTD_storeSeq(seqStore, litLength, anchor, iend, (U32)offset, matchLength-MINMATCH); - anchor = ip = start + matchLength; - } - - /* check immediate repcode */ - if (dictMode == ZSTD_dictMatchState) { - while (ip <= ilimit) { - U32 const current2 = (U32)(ip-base); - U32 const repIndex = current2 - offset_2; - const BYTE* repMatch = dictMode == ZSTD_dictMatchState - && repIndex < prefixLowestIndex ? - dictBase - dictIndexDelta + repIndex : - base + repIndex; - if ( ((U32)((prefixLowestIndex-1) - (U32)repIndex) >= 3 /* intentional overflow */) - && (MEM_read32(repMatch) == MEM_read32(ip)) ) { - const BYTE* const repEnd2 = repIndex < prefixLowestIndex ? dictEnd : iend; - matchLength = ZSTD_count_2segments(ip+4, repMatch+4, iend, repEnd2, prefixLowest) + 4; - offset = offset_2; offset_2 = offset_1; offset_1 = (U32)offset; /* swap offset_2 <=> offset_1 */ - ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, matchLength-MINMATCH); - ip += matchLength; - anchor = ip; - continue; - } - break; - } - } - - if (dictMode == ZSTD_noDict) { - while ( ((ip <= ilimit) & (offset_2>0)) - && (MEM_read32(ip) == MEM_read32(ip - offset_2)) ) { - /* store sequence */ - matchLength = ZSTD_count(ip+4, ip+4-offset_2, iend) + 4; - offset = offset_2; offset_2 = offset_1; offset_1 = (U32)offset; /* swap repcodes */ - ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, matchLength-MINMATCH); - ip += matchLength; - anchor = ip; - continue; /* faster when present ... (?) */ - } } } - - /* Save reps for next block */ - rep[0] = offset_1 ? offset_1 : savedOffset; - rep[1] = offset_2 ? offset_2 : savedOffset; - - /* Return the last literals size */ - return (size_t)(iend - anchor); -} - - -size_t ZSTD_compressBlock_btlazy2( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_binaryTree, 2, ZSTD_noDict); -} - -size_t ZSTD_compressBlock_lazy2( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 2, ZSTD_noDict); -} - -size_t ZSTD_compressBlock_lazy( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 1, ZSTD_noDict); -} - -size_t ZSTD_compressBlock_greedy( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 0, ZSTD_noDict); -} - -size_t ZSTD_compressBlock_btlazy2_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_binaryTree, 2, ZSTD_dictMatchState); -} - -size_t ZSTD_compressBlock_lazy2_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 2, ZSTD_dictMatchState); -} - -size_t ZSTD_compressBlock_lazy_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 1, ZSTD_dictMatchState); -} - -size_t ZSTD_compressBlock_greedy_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 0, ZSTD_dictMatchState); -} - - -FORCE_INLINE_TEMPLATE -size_t ZSTD_compressBlock_lazy_extDict_generic( - ZSTD_matchState_t* ms, seqStore_t* seqStore, - U32 rep[ZSTD_REP_NUM], - const void* src, size_t srcSize, - const searchMethod_e searchMethod, const U32 depth) -{ - const BYTE* const istart = (const BYTE*)src; - const BYTE* ip = istart; - const BYTE* anchor = istart; - const BYTE* const iend = istart + srcSize; - const BYTE* const ilimit = iend - 8; - const BYTE* const base = ms->window.base; - const U32 dictLimit = ms->window.dictLimit; - const BYTE* const prefixStart = base + dictLimit; - const BYTE* const dictBase = ms->window.dictBase; - const BYTE* const dictEnd = dictBase + dictLimit; - const BYTE* const dictStart = dictBase + ms->window.lowLimit; - const U32 windowLog = ms->cParams.windowLog; - - typedef size_t (*searchMax_f)( - ZSTD_matchState_t* ms, - const BYTE* ip, const BYTE* iLimit, size_t* offsetPtr); - searchMax_f searchMax = searchMethod==search_binaryTree ? ZSTD_BtFindBestMatch_extDict_selectMLS : ZSTD_HcFindBestMatch_extDict_selectMLS; - - U32 offset_1 = rep[0], offset_2 = rep[1]; - - DEBUGLOG(5, "ZSTD_compressBlock_lazy_extDict_generic"); - - /* init */ - ip += (ip == prefixStart); - - /* Match Loop */ -#if defined(__GNUC__) && defined(__x86_64__) - /* I've measured random a 5% speed loss on levels 5 & 6 (greedy) when the - * code alignment is perturbed. To fix the instability align the loop on 32-bytes. - */ - __asm__(".p2align 5"); -#endif - while (ip < ilimit) { - size_t matchLength=0; - size_t offset=0; - const BYTE* start=ip+1; - U32 current = (U32)(ip-base); - - /* check repCode */ - { const U32 windowLow = ZSTD_getLowestMatchIndex(ms, current+1, windowLog); - const U32 repIndex = (U32)(current+1 - offset_1); - const BYTE* const repBase = repIndex < dictLimit ? dictBase : base; - const BYTE* const repMatch = repBase + repIndex; - if (((U32)((dictLimit-1) - repIndex) >= 3) & (repIndex > windowLow)) /* intentional overflow */ - if (MEM_read32(ip+1) == MEM_read32(repMatch)) { - /* repcode detected we should take it */ - const BYTE* const repEnd = repIndex < dictLimit ? dictEnd : iend; - matchLength = ZSTD_count_2segments(ip+1+4, repMatch+4, iend, repEnd, prefixStart) + 4; - if (depth==0) goto _storeSequence; - } } - - /* first search (depth 0) */ - { size_t offsetFound = 999999999; - size_t const ml2 = searchMax(ms, ip, iend, &offsetFound); - if (ml2 > matchLength) - matchLength = ml2, start = ip, offset=offsetFound; - } - - if (matchLength < 4) { - ip += ((ip-anchor) >> kSearchStrength) + 1; /* jump faster over incompressible sections */ - continue; - } - - /* let's try to find a better solution */ - if (depth>=1) - while (ip= 3) & (repIndex > windowLow)) /* intentional overflow */ - if (MEM_read32(ip) == MEM_read32(repMatch)) { - /* repcode detected */ - const BYTE* const repEnd = repIndex < dictLimit ? dictEnd : iend; - size_t const repLength = ZSTD_count_2segments(ip+4, repMatch+4, iend, repEnd, prefixStart) + 4; - int const gain2 = (int)(repLength * 3); - int const gain1 = (int)(matchLength*3 - ZSTD_highbit32((U32)offset+1) + 1); - if ((repLength >= 4) && (gain2 > gain1)) - matchLength = repLength, offset = 0, start = ip; - } } - - /* search match, depth 1 */ - { size_t offset2=999999999; - size_t const ml2 = searchMax(ms, ip, iend, &offset2); - int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)offset2+1)); /* raw approx */ - int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 4); - if ((ml2 >= 4) && (gain2 > gain1)) { - matchLength = ml2, offset = offset2, start = ip; - continue; /* search a better one */ - } } - - /* let's find an even better one */ - if ((depth==2) && (ip= 3) & (repIndex > windowLow)) /* intentional overflow */ - if (MEM_read32(ip) == MEM_read32(repMatch)) { - /* repcode detected */ - const BYTE* const repEnd = repIndex < dictLimit ? dictEnd : iend; - size_t const repLength = ZSTD_count_2segments(ip+4, repMatch+4, iend, repEnd, prefixStart) + 4; - int const gain2 = (int)(repLength * 4); - int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 1); - if ((repLength >= 4) && (gain2 > gain1)) - matchLength = repLength, offset = 0, start = ip; - } } - - /* search match, depth 2 */ - { size_t offset2=999999999; - size_t const ml2 = searchMax(ms, ip, iend, &offset2); - int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)offset2+1)); /* raw approx */ - int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 7); - if ((ml2 >= 4) && (gain2 > gain1)) { - matchLength = ml2, offset = offset2, start = ip; - continue; - } } } - break; /* nothing found : store previous solution */ - } - - /* catch up */ - if (offset) { - U32 const matchIndex = (U32)((start-base) - (offset - ZSTD_REP_MOVE)); - const BYTE* match = (matchIndex < dictLimit) ? dictBase + matchIndex : base + matchIndex; - const BYTE* const mStart = (matchIndex < dictLimit) ? dictStart : prefixStart; - while ((start>anchor) && (match>mStart) && (start[-1] == match[-1])) { start--; match--; matchLength++; } /* catch up */ - offset_2 = offset_1; offset_1 = (U32)(offset - ZSTD_REP_MOVE); - } - - /* store sequence */ -_storeSequence: - { size_t const litLength = start - anchor; - ZSTD_storeSeq(seqStore, litLength, anchor, iend, (U32)offset, matchLength-MINMATCH); - anchor = ip = start + matchLength; - } - - /* check immediate repcode */ - while (ip <= ilimit) { - const U32 repCurrent = (U32)(ip-base); - const U32 windowLow = ZSTD_getLowestMatchIndex(ms, repCurrent, windowLog); - const U32 repIndex = repCurrent - offset_2; - const BYTE* const repBase = repIndex < dictLimit ? dictBase : base; - const BYTE* const repMatch = repBase + repIndex; - if (((U32)((dictLimit-1) - repIndex) >= 3) & (repIndex > windowLow)) /* intentional overflow */ - if (MEM_read32(ip) == MEM_read32(repMatch)) { - /* repcode detected we should take it */ - const BYTE* const repEnd = repIndex < dictLimit ? dictEnd : iend; - matchLength = ZSTD_count_2segments(ip+4, repMatch+4, iend, repEnd, prefixStart) + 4; - offset = offset_2; offset_2 = offset_1; offset_1 = (U32)offset; /* swap offset history */ - ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, matchLength-MINMATCH); - ip += matchLength; - anchor = ip; - continue; /* faster when present ... (?) */ - } - break; - } } - - /* Save reps for next block */ - rep[0] = offset_1; - rep[1] = offset_2; - - /* Return the last literals size */ - return (size_t)(iend - anchor); -} - - -size_t ZSTD_compressBlock_greedy_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - return ZSTD_compressBlock_lazy_extDict_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 0); -} - -size_t ZSTD_compressBlock_lazy_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) - -{ - return ZSTD_compressBlock_lazy_extDict_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 1); -} - -size_t ZSTD_compressBlock_lazy2_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) - -{ - return ZSTD_compressBlock_lazy_extDict_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 2); -} - -size_t ZSTD_compressBlock_btlazy2_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) - -{ - return ZSTD_compressBlock_lazy_extDict_generic(ms, seqStore, rep, src, srcSize, search_binaryTree, 2); -} -/**** ended inlining compress/zstd_lazy.c ****/ -/**** start inlining compress/zstd_ldm.c ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -/**** skipping file: zstd_ldm.h ****/ - -/**** skipping file: ../common/debug.h ****/ -/**** skipping file: zstd_fast.h ****/ -/**** skipping file: zstd_double_fast.h ****/ - -#define LDM_BUCKET_SIZE_LOG 3 -#define LDM_MIN_MATCH_LENGTH 64 -#define LDM_HASH_RLOG 7 -#define LDM_HASH_CHAR_OFFSET 10 - -void ZSTD_ldm_adjustParameters(ldmParams_t* params, - ZSTD_compressionParameters const* cParams) -{ - params->windowLog = cParams->windowLog; - ZSTD_STATIC_ASSERT(LDM_BUCKET_SIZE_LOG <= ZSTD_LDM_BUCKETSIZELOG_MAX); - DEBUGLOG(4, "ZSTD_ldm_adjustParameters"); - if (!params->bucketSizeLog) params->bucketSizeLog = LDM_BUCKET_SIZE_LOG; - if (!params->minMatchLength) params->minMatchLength = LDM_MIN_MATCH_LENGTH; - if (cParams->strategy >= ZSTD_btopt) { - /* Get out of the way of the optimal parser */ - U32 const minMatch = MAX(cParams->targetLength, params->minMatchLength); - assert(minMatch >= ZSTD_LDM_MINMATCH_MIN); - assert(minMatch <= ZSTD_LDM_MINMATCH_MAX); - params->minMatchLength = minMatch; - } - if (params->hashLog == 0) { - params->hashLog = MAX(ZSTD_HASHLOG_MIN, params->windowLog - LDM_HASH_RLOG); - assert(params->hashLog <= ZSTD_HASHLOG_MAX); - } - if (params->hashRateLog == 0) { - params->hashRateLog = params->windowLog < params->hashLog - ? 0 - : params->windowLog - params->hashLog; - } - params->bucketSizeLog = MIN(params->bucketSizeLog, params->hashLog); -} - -size_t ZSTD_ldm_getTableSize(ldmParams_t params) -{ - size_t const ldmHSize = ((size_t)1) << params.hashLog; - size_t const ldmBucketSizeLog = MIN(params.bucketSizeLog, params.hashLog); - size_t const ldmBucketSize = ((size_t)1) << (params.hashLog - ldmBucketSizeLog); - size_t const totalSize = ZSTD_cwksp_alloc_size(ldmBucketSize) - + ZSTD_cwksp_alloc_size(ldmHSize * sizeof(ldmEntry_t)); - return params.enableLdm ? totalSize : 0; -} - -size_t ZSTD_ldm_getMaxNbSeq(ldmParams_t params, size_t maxChunkSize) -{ - return params.enableLdm ? (maxChunkSize / params.minMatchLength) : 0; -} - -/** ZSTD_ldm_getSmallHash() : - * numBits should be <= 32 - * If numBits==0, returns 0. - * @return : the most significant numBits of value. */ -static U32 ZSTD_ldm_getSmallHash(U64 value, U32 numBits) -{ - assert(numBits <= 32); - return numBits == 0 ? 0 : (U32)(value >> (64 - numBits)); -} - -/** ZSTD_ldm_getChecksum() : - * numBitsToDiscard should be <= 32 - * @return : the next most significant 32 bits after numBitsToDiscard */ -static U32 ZSTD_ldm_getChecksum(U64 hash, U32 numBitsToDiscard) -{ - assert(numBitsToDiscard <= 32); - return (hash >> (64 - 32 - numBitsToDiscard)) & 0xFFFFFFFF; -} - -/** ZSTD_ldm_getTag() ; - * Given the hash, returns the most significant numTagBits bits - * after (32 + hbits) bits. - * - * If there are not enough bits remaining, return the last - * numTagBits bits. */ -static U32 ZSTD_ldm_getTag(U64 hash, U32 hbits, U32 numTagBits) -{ - assert(numTagBits < 32 && hbits <= 32); - if (32 - hbits < numTagBits) { - return hash & (((U32)1 << numTagBits) - 1); - } else { - return (hash >> (32 - hbits - numTagBits)) & (((U32)1 << numTagBits) - 1); - } -} - -/** ZSTD_ldm_getBucket() : - * Returns a pointer to the start of the bucket associated with hash. */ -static ldmEntry_t* ZSTD_ldm_getBucket( - ldmState_t* ldmState, size_t hash, ldmParams_t const ldmParams) -{ - return ldmState->hashTable + (hash << ldmParams.bucketSizeLog); -} - -/** ZSTD_ldm_insertEntry() : - * Insert the entry with corresponding hash into the hash table */ -static void ZSTD_ldm_insertEntry(ldmState_t* ldmState, - size_t const hash, const ldmEntry_t entry, - ldmParams_t const ldmParams) -{ - BYTE* const bucketOffsets = ldmState->bucketOffsets; - *(ZSTD_ldm_getBucket(ldmState, hash, ldmParams) + bucketOffsets[hash]) = entry; - bucketOffsets[hash]++; - bucketOffsets[hash] &= ((U32)1 << ldmParams.bucketSizeLog) - 1; -} - -/** ZSTD_ldm_makeEntryAndInsertByTag() : - * - * Gets the small hash, checksum, and tag from the rollingHash. - * - * If the tag matches (1 << ldmParams.hashRateLog)-1, then - * creates an ldmEntry from the offset, and inserts it into the hash table. - * - * hBits is the length of the small hash, which is the most significant hBits - * of rollingHash. The checksum is the next 32 most significant bits, followed - * by ldmParams.hashRateLog bits that make up the tag. */ -static void ZSTD_ldm_makeEntryAndInsertByTag(ldmState_t* ldmState, - U64 const rollingHash, - U32 const hBits, - U32 const offset, - ldmParams_t const ldmParams) -{ - U32 const tag = ZSTD_ldm_getTag(rollingHash, hBits, ldmParams.hashRateLog); - U32 const tagMask = ((U32)1 << ldmParams.hashRateLog) - 1; - if (tag == tagMask) { - U32 const hash = ZSTD_ldm_getSmallHash(rollingHash, hBits); - U32 const checksum = ZSTD_ldm_getChecksum(rollingHash, hBits); - ldmEntry_t entry; - entry.offset = offset; - entry.checksum = checksum; - ZSTD_ldm_insertEntry(ldmState, hash, entry, ldmParams); - } -} - -/** ZSTD_ldm_countBackwardsMatch() : - * Returns the number of bytes that match backwards before pIn and pMatch. - * - * We count only bytes where pMatch >= pBase and pIn >= pAnchor. */ -static size_t ZSTD_ldm_countBackwardsMatch( - const BYTE* pIn, const BYTE* pAnchor, - const BYTE* pMatch, const BYTE* pBase) -{ - size_t matchLength = 0; - while (pIn > pAnchor && pMatch > pBase && pIn[-1] == pMatch[-1]) { - pIn--; - pMatch--; - matchLength++; - } - return matchLength; -} - -/** ZSTD_ldm_fillFastTables() : - * - * Fills the relevant tables for the ZSTD_fast and ZSTD_dfast strategies. - * This is similar to ZSTD_loadDictionaryContent. - * - * The tables for the other strategies are filled within their - * block compressors. */ -static size_t ZSTD_ldm_fillFastTables(ZSTD_matchState_t* ms, - void const* end) -{ - const BYTE* const iend = (const BYTE*)end; - - switch(ms->cParams.strategy) - { - case ZSTD_fast: - ZSTD_fillHashTable(ms, iend, ZSTD_dtlm_fast); - break; - - case ZSTD_dfast: - ZSTD_fillDoubleHashTable(ms, iend, ZSTD_dtlm_fast); - break; - - case ZSTD_greedy: - case ZSTD_lazy: - case ZSTD_lazy2: - case ZSTD_btlazy2: - case ZSTD_btopt: - case ZSTD_btultra: - case ZSTD_btultra2: - break; - default: - assert(0); /* not possible : not a valid strategy id */ - } - - return 0; -} - -/** ZSTD_ldm_fillLdmHashTable() : - * - * Fills hashTable from (lastHashed + 1) to iend (non-inclusive). - * lastHash is the rolling hash that corresponds to lastHashed. - * - * Returns the rolling hash corresponding to position iend-1. */ -static U64 ZSTD_ldm_fillLdmHashTable(ldmState_t* state, - U64 lastHash, const BYTE* lastHashed, - const BYTE* iend, const BYTE* base, - U32 hBits, ldmParams_t const ldmParams) -{ - U64 rollingHash = lastHash; - const BYTE* cur = lastHashed + 1; - - while (cur < iend) { - rollingHash = ZSTD_rollingHash_rotate(rollingHash, cur[-1], - cur[ldmParams.minMatchLength-1], - state->hashPower); - ZSTD_ldm_makeEntryAndInsertByTag(state, - rollingHash, hBits, - (U32)(cur - base), ldmParams); - ++cur; - } - return rollingHash; -} - -void ZSTD_ldm_fillHashTable( - ldmState_t* state, const BYTE* ip, - const BYTE* iend, ldmParams_t const* params) -{ - DEBUGLOG(5, "ZSTD_ldm_fillHashTable"); - if ((size_t)(iend - ip) >= params->minMatchLength) { - U64 startingHash = ZSTD_rollingHash_compute(ip, params->minMatchLength); - ZSTD_ldm_fillLdmHashTable( - state, startingHash, ip, iend - params->minMatchLength, state->window.base, - params->hashLog - params->bucketSizeLog, - *params); - } -} - - -/** ZSTD_ldm_limitTableUpdate() : - * - * Sets cctx->nextToUpdate to a position corresponding closer to anchor - * if it is far way - * (after a long match, only update tables a limited amount). */ -static void ZSTD_ldm_limitTableUpdate(ZSTD_matchState_t* ms, const BYTE* anchor) -{ - U32 const current = (U32)(anchor - ms->window.base); - if (current > ms->nextToUpdate + 1024) { - ms->nextToUpdate = - current - MIN(512, current - ms->nextToUpdate - 1024); - } -} - -static size_t ZSTD_ldm_generateSequences_internal( - ldmState_t* ldmState, rawSeqStore_t* rawSeqStore, - ldmParams_t const* params, void const* src, size_t srcSize) -{ - /* LDM parameters */ - int const extDict = ZSTD_window_hasExtDict(ldmState->window); - U32 const minMatchLength = params->minMatchLength; - U64 const hashPower = ldmState->hashPower; - U32 const hBits = params->hashLog - params->bucketSizeLog; - U32 const ldmBucketSize = 1U << params->bucketSizeLog; - U32 const hashRateLog = params->hashRateLog; - U32 const ldmTagMask = (1U << params->hashRateLog) - 1; - /* Prefix and extDict parameters */ - U32 const dictLimit = ldmState->window.dictLimit; - U32 const lowestIndex = extDict ? ldmState->window.lowLimit : dictLimit; - BYTE const* const base = ldmState->window.base; - BYTE const* const dictBase = extDict ? ldmState->window.dictBase : NULL; - BYTE const* const dictStart = extDict ? dictBase + lowestIndex : NULL; - BYTE const* const dictEnd = extDict ? dictBase + dictLimit : NULL; - BYTE const* const lowPrefixPtr = base + dictLimit; - /* Input bounds */ - BYTE const* const istart = (BYTE const*)src; - BYTE const* const iend = istart + srcSize; - BYTE const* const ilimit = iend - MAX(minMatchLength, HASH_READ_SIZE); - /* Input positions */ - BYTE const* anchor = istart; - BYTE const* ip = istart; - /* Rolling hash */ - BYTE const* lastHashed = NULL; - U64 rollingHash = 0; - - while (ip <= ilimit) { - size_t mLength; - U32 const current = (U32)(ip - base); - size_t forwardMatchLength = 0, backwardMatchLength = 0; - ldmEntry_t* bestEntry = NULL; - if (ip != istart) { - rollingHash = ZSTD_rollingHash_rotate(rollingHash, lastHashed[0], - lastHashed[minMatchLength], - hashPower); - } else { - rollingHash = ZSTD_rollingHash_compute(ip, minMatchLength); - } - lastHashed = ip; - - /* Do not insert and do not look for a match */ - if (ZSTD_ldm_getTag(rollingHash, hBits, hashRateLog) != ldmTagMask) { - ip++; - continue; - } - - /* Get the best entry and compute the match lengths */ - { - ldmEntry_t* const bucket = - ZSTD_ldm_getBucket(ldmState, - ZSTD_ldm_getSmallHash(rollingHash, hBits), - *params); - ldmEntry_t* cur; - size_t bestMatchLength = 0; - U32 const checksum = ZSTD_ldm_getChecksum(rollingHash, hBits); - - for (cur = bucket; cur < bucket + ldmBucketSize; ++cur) { - size_t curForwardMatchLength, curBackwardMatchLength, - curTotalMatchLength; - if (cur->checksum != checksum || cur->offset <= lowestIndex) { - continue; - } - if (extDict) { - BYTE const* const curMatchBase = - cur->offset < dictLimit ? dictBase : base; - BYTE const* const pMatch = curMatchBase + cur->offset; - BYTE const* const matchEnd = - cur->offset < dictLimit ? dictEnd : iend; - BYTE const* const lowMatchPtr = - cur->offset < dictLimit ? dictStart : lowPrefixPtr; - - curForwardMatchLength = ZSTD_count_2segments( - ip, pMatch, iend, - matchEnd, lowPrefixPtr); - if (curForwardMatchLength < minMatchLength) { - continue; - } - curBackwardMatchLength = - ZSTD_ldm_countBackwardsMatch(ip, anchor, pMatch, - lowMatchPtr); - curTotalMatchLength = curForwardMatchLength + - curBackwardMatchLength; - } else { /* !extDict */ - BYTE const* const pMatch = base + cur->offset; - curForwardMatchLength = ZSTD_count(ip, pMatch, iend); - if (curForwardMatchLength < minMatchLength) { - continue; - } - curBackwardMatchLength = - ZSTD_ldm_countBackwardsMatch(ip, anchor, pMatch, - lowPrefixPtr); - curTotalMatchLength = curForwardMatchLength + - curBackwardMatchLength; - } - - if (curTotalMatchLength > bestMatchLength) { - bestMatchLength = curTotalMatchLength; - forwardMatchLength = curForwardMatchLength; - backwardMatchLength = curBackwardMatchLength; - bestEntry = cur; - } - } - } - - /* No match found -- continue searching */ - if (bestEntry == NULL) { - ZSTD_ldm_makeEntryAndInsertByTag(ldmState, rollingHash, - hBits, current, - *params); - ip++; - continue; - } - - /* Match found */ - mLength = forwardMatchLength + backwardMatchLength; - ip -= backwardMatchLength; - - { - /* Store the sequence: - * ip = current - backwardMatchLength - * The match is at (bestEntry->offset - backwardMatchLength) - */ - U32 const matchIndex = bestEntry->offset; - U32 const offset = current - matchIndex; - rawSeq* const seq = rawSeqStore->seq + rawSeqStore->size; - - /* Out of sequence storage */ - if (rawSeqStore->size == rawSeqStore->capacity) - return ERROR(dstSize_tooSmall); - seq->litLength = (U32)(ip - anchor); - seq->matchLength = (U32)mLength; - seq->offset = offset; - rawSeqStore->size++; - } - - /* Insert the current entry into the hash table */ - ZSTD_ldm_makeEntryAndInsertByTag(ldmState, rollingHash, hBits, - (U32)(lastHashed - base), - *params); - - assert(ip + backwardMatchLength == lastHashed); - - /* Fill the hash table from lastHashed+1 to ip+mLength*/ - /* Heuristic: don't need to fill the entire table at end of block */ - if (ip + mLength <= ilimit) { - rollingHash = ZSTD_ldm_fillLdmHashTable( - ldmState, rollingHash, lastHashed, - ip + mLength, base, hBits, *params); - lastHashed = ip + mLength - 1; - } - ip += mLength; - anchor = ip; - } - return iend - anchor; -} - -/*! ZSTD_ldm_reduceTable() : - * reduce table indexes by `reducerValue` */ -static void ZSTD_ldm_reduceTable(ldmEntry_t* const table, U32 const size, - U32 const reducerValue) -{ - U32 u; - for (u = 0; u < size; u++) { - if (table[u].offset < reducerValue) table[u].offset = 0; - else table[u].offset -= reducerValue; - } -} - -size_t ZSTD_ldm_generateSequences( - ldmState_t* ldmState, rawSeqStore_t* sequences, - ldmParams_t const* params, void const* src, size_t srcSize) -{ - U32 const maxDist = 1U << params->windowLog; - BYTE const* const istart = (BYTE const*)src; - BYTE const* const iend = istart + srcSize; - size_t const kMaxChunkSize = 1 << 20; - size_t const nbChunks = (srcSize / kMaxChunkSize) + ((srcSize % kMaxChunkSize) != 0); - size_t chunk; - size_t leftoverSize = 0; - - assert(ZSTD_CHUNKSIZE_MAX >= kMaxChunkSize); - /* Check that ZSTD_window_update() has been called for this chunk prior - * to passing it to this function. - */ - assert(ldmState->window.nextSrc >= (BYTE const*)src + srcSize); - /* The input could be very large (in zstdmt), so it must be broken up into - * chunks to enforce the maximum distance and handle overflow correction. - */ - assert(sequences->pos <= sequences->size); - assert(sequences->size <= sequences->capacity); - for (chunk = 0; chunk < nbChunks && sequences->size < sequences->capacity; ++chunk) { - BYTE const* const chunkStart = istart + chunk * kMaxChunkSize; - size_t const remaining = (size_t)(iend - chunkStart); - BYTE const *const chunkEnd = - (remaining < kMaxChunkSize) ? iend : chunkStart + kMaxChunkSize; - size_t const chunkSize = chunkEnd - chunkStart; - size_t newLeftoverSize; - size_t const prevSize = sequences->size; - - assert(chunkStart < iend); - /* 1. Perform overflow correction if necessary. */ - if (ZSTD_window_needOverflowCorrection(ldmState->window, chunkEnd)) { - U32 const ldmHSize = 1U << params->hashLog; - U32 const correction = ZSTD_window_correctOverflow( - &ldmState->window, /* cycleLog */ 0, maxDist, chunkStart); - ZSTD_ldm_reduceTable(ldmState->hashTable, ldmHSize, correction); - /* invalidate dictionaries on overflow correction */ - ldmState->loadedDictEnd = 0; - } - /* 2. We enforce the maximum offset allowed. - * - * kMaxChunkSize should be small enough that we don't lose too much of - * the window through early invalidation. - * TODO: * Test the chunk size. - * * Try invalidation after the sequence generation and test the - * the offset against maxDist directly. - * - * NOTE: Because of dictionaries + sequence splitting we MUST make sure - * that any offset used is valid at the END of the sequence, since it may - * be split into two sequences. This condition holds when using - * ZSTD_window_enforceMaxDist(), but if we move to checking offsets - * against maxDist directly, we'll have to carefully handle that case. - */ - ZSTD_window_enforceMaxDist(&ldmState->window, chunkEnd, maxDist, &ldmState->loadedDictEnd, NULL); - /* 3. Generate the sequences for the chunk, and get newLeftoverSize. */ - newLeftoverSize = ZSTD_ldm_generateSequences_internal( - ldmState, sequences, params, chunkStart, chunkSize); - if (ZSTD_isError(newLeftoverSize)) - return newLeftoverSize; - /* 4. We add the leftover literals from previous iterations to the first - * newly generated sequence, or add the `newLeftoverSize` if none are - * generated. - */ - /* Prepend the leftover literals from the last call */ - if (prevSize < sequences->size) { - sequences->seq[prevSize].litLength += (U32)leftoverSize; - leftoverSize = newLeftoverSize; - } else { - assert(newLeftoverSize == chunkSize); - leftoverSize += chunkSize; - } - } - return 0; -} - -void ZSTD_ldm_skipSequences(rawSeqStore_t* rawSeqStore, size_t srcSize, U32 const minMatch) { - while (srcSize > 0 && rawSeqStore->pos < rawSeqStore->size) { - rawSeq* seq = rawSeqStore->seq + rawSeqStore->pos; - if (srcSize <= seq->litLength) { - /* Skip past srcSize literals */ - seq->litLength -= (U32)srcSize; - return; - } - srcSize -= seq->litLength; - seq->litLength = 0; - if (srcSize < seq->matchLength) { - /* Skip past the first srcSize of the match */ - seq->matchLength -= (U32)srcSize; - if (seq->matchLength < minMatch) { - /* The match is too short, omit it */ - if (rawSeqStore->pos + 1 < rawSeqStore->size) { - seq[1].litLength += seq[0].matchLength; - } - rawSeqStore->pos++; - } - return; - } - srcSize -= seq->matchLength; - seq->matchLength = 0; - rawSeqStore->pos++; - } -} - -/** - * If the sequence length is longer than remaining then the sequence is split - * between this block and the next. - * - * Returns the current sequence to handle, or if the rest of the block should - * be literals, it returns a sequence with offset == 0. - */ -static rawSeq maybeSplitSequence(rawSeqStore_t* rawSeqStore, - U32 const remaining, U32 const minMatch) -{ - rawSeq sequence = rawSeqStore->seq[rawSeqStore->pos]; - assert(sequence.offset > 0); - /* Likely: No partial sequence */ - if (remaining >= sequence.litLength + sequence.matchLength) { - rawSeqStore->pos++; - return sequence; - } - /* Cut the sequence short (offset == 0 ==> rest is literals). */ - if (remaining <= sequence.litLength) { - sequence.offset = 0; - } else if (remaining < sequence.litLength + sequence.matchLength) { - sequence.matchLength = remaining - sequence.litLength; - if (sequence.matchLength < minMatch) { - sequence.offset = 0; - } - } - /* Skip past `remaining` bytes for the future sequences. */ - ZSTD_ldm_skipSequences(rawSeqStore, remaining, minMatch); - return sequence; -} - -size_t ZSTD_ldm_blockCompress(rawSeqStore_t* rawSeqStore, - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - void const* src, size_t srcSize) -{ - const ZSTD_compressionParameters* const cParams = &ms->cParams; - unsigned const minMatch = cParams->minMatch; - ZSTD_blockCompressor const blockCompressor = - ZSTD_selectBlockCompressor(cParams->strategy, ZSTD_matchState_dictMode(ms)); - /* Input bounds */ - BYTE const* const istart = (BYTE const*)src; - BYTE const* const iend = istart + srcSize; - /* Input positions */ - BYTE const* ip = istart; - - DEBUGLOG(5, "ZSTD_ldm_blockCompress: srcSize=%zu", srcSize); - assert(rawSeqStore->pos <= rawSeqStore->size); - assert(rawSeqStore->size <= rawSeqStore->capacity); - /* Loop through each sequence and apply the block compressor to the lits */ - while (rawSeqStore->pos < rawSeqStore->size && ip < iend) { - /* maybeSplitSequence updates rawSeqStore->pos */ - rawSeq const sequence = maybeSplitSequence(rawSeqStore, - (U32)(iend - ip), minMatch); - int i; - /* End signal */ - if (sequence.offset == 0) - break; - - assert(ip + sequence.litLength + sequence.matchLength <= iend); - - /* Fill tables for block compressor */ - ZSTD_ldm_limitTableUpdate(ms, ip); - ZSTD_ldm_fillFastTables(ms, ip); - /* Run the block compressor */ - DEBUGLOG(5, "pos %u : calling block compressor on segment of size %u", (unsigned)(ip-istart), sequence.litLength); - { - size_t const newLitLength = - blockCompressor(ms, seqStore, rep, ip, sequence.litLength); - ip += sequence.litLength; - /* Update the repcodes */ - for (i = ZSTD_REP_NUM - 1; i > 0; i--) - rep[i] = rep[i-1]; - rep[0] = sequence.offset; - /* Store the sequence */ - ZSTD_storeSeq(seqStore, newLitLength, ip - newLitLength, iend, - sequence.offset + ZSTD_REP_MOVE, - sequence.matchLength - MINMATCH); - ip += sequence.matchLength; - } - } - /* Fill the tables for the block compressor */ - ZSTD_ldm_limitTableUpdate(ms, ip); - ZSTD_ldm_fillFastTables(ms, ip); - /* Compress the last literals */ - return blockCompressor(ms, seqStore, rep, ip, iend - ip); -} -/**** ended inlining compress/zstd_ldm.c ****/ -/**** start inlining compress/zstd_opt.c ****/ -/* - * Copyright (c) 2016-2020, Przemyslaw Skibinski, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -/**** skipping file: zstd_compress_internal.h ****/ -/**** skipping file: hist.h ****/ -/**** skipping file: zstd_opt.h ****/ - - -#define ZSTD_LITFREQ_ADD 2 /* scaling factor for litFreq, so that frequencies adapt faster to new stats */ -#define ZSTD_FREQ_DIV 4 /* log factor when using previous stats to init next stats */ -#define ZSTD_MAX_PRICE (1<<30) - -#define ZSTD_PREDEF_THRESHOLD 1024 /* if srcSize < ZSTD_PREDEF_THRESHOLD, symbols' cost is assumed static, directly determined by pre-defined distributions */ - - -/*-************************************* -* Price functions for optimal parser -***************************************/ - -#if 0 /* approximation at bit level */ -# define BITCOST_ACCURACY 0 -# define BITCOST_MULTIPLIER (1 << BITCOST_ACCURACY) -# define WEIGHT(stat) ((void)opt, ZSTD_bitWeight(stat)) -#elif 0 /* fractional bit accuracy */ -# define BITCOST_ACCURACY 8 -# define BITCOST_MULTIPLIER (1 << BITCOST_ACCURACY) -# define WEIGHT(stat,opt) ((void)opt, ZSTD_fracWeight(stat)) -#else /* opt==approx, ultra==accurate */ -# define BITCOST_ACCURACY 8 -# define BITCOST_MULTIPLIER (1 << BITCOST_ACCURACY) -# define WEIGHT(stat,opt) (opt ? ZSTD_fracWeight(stat) : ZSTD_bitWeight(stat)) -#endif - -MEM_STATIC U32 ZSTD_bitWeight(U32 stat) -{ - return (ZSTD_highbit32(stat+1) * BITCOST_MULTIPLIER); -} - -MEM_STATIC U32 ZSTD_fracWeight(U32 rawStat) -{ - U32 const stat = rawStat + 1; - U32 const hb = ZSTD_highbit32(stat); - U32 const BWeight = hb * BITCOST_MULTIPLIER; - U32 const FWeight = (stat << BITCOST_ACCURACY) >> hb; - U32 const weight = BWeight + FWeight; - assert(hb + BITCOST_ACCURACY < 31); - return weight; -} - -#if (DEBUGLEVEL>=2) -/* debugging function, - * @return price in bytes as fractional value - * for debug messages only */ -MEM_STATIC double ZSTD_fCost(U32 price) -{ - return (double)price / (BITCOST_MULTIPLIER*8); -} -#endif - -static int ZSTD_compressedLiterals(optState_t const* const optPtr) -{ - return optPtr->literalCompressionMode != ZSTD_lcm_uncompressed; -} - -static void ZSTD_setBasePrices(optState_t* optPtr, int optLevel) -{ - if (ZSTD_compressedLiterals(optPtr)) - optPtr->litSumBasePrice = WEIGHT(optPtr->litSum, optLevel); - optPtr->litLengthSumBasePrice = WEIGHT(optPtr->litLengthSum, optLevel); - optPtr->matchLengthSumBasePrice = WEIGHT(optPtr->matchLengthSum, optLevel); - optPtr->offCodeSumBasePrice = WEIGHT(optPtr->offCodeSum, optLevel); -} - - -/* ZSTD_downscaleStat() : - * reduce all elements in table by a factor 2^(ZSTD_FREQ_DIV+malus) - * return the resulting sum of elements */ -static U32 ZSTD_downscaleStat(unsigned* table, U32 lastEltIndex, int malus) -{ - U32 s, sum=0; - DEBUGLOG(5, "ZSTD_downscaleStat (nbElts=%u)", (unsigned)lastEltIndex+1); - assert(ZSTD_FREQ_DIV+malus > 0 && ZSTD_FREQ_DIV+malus < 31); - for (s=0; s> (ZSTD_FREQ_DIV+malus)); - sum += table[s]; - } - return sum; -} - -/* ZSTD_rescaleFreqs() : - * if first block (detected by optPtr->litLengthSum == 0) : init statistics - * take hints from dictionary if there is one - * or init from zero, using src for literals stats, or flat 1 for match symbols - * otherwise downscale existing stats, to be used as seed for next block. - */ -static void -ZSTD_rescaleFreqs(optState_t* const optPtr, - const BYTE* const src, size_t const srcSize, - int const optLevel) -{ - int const compressedLiterals = ZSTD_compressedLiterals(optPtr); - DEBUGLOG(5, "ZSTD_rescaleFreqs (srcSize=%u)", (unsigned)srcSize); - optPtr->priceType = zop_dynamic; - - if (optPtr->litLengthSum == 0) { /* first block : init */ - if (srcSize <= ZSTD_PREDEF_THRESHOLD) { /* heuristic */ - DEBUGLOG(5, "(srcSize <= ZSTD_PREDEF_THRESHOLD) => zop_predef"); - optPtr->priceType = zop_predef; - } - - assert(optPtr->symbolCosts != NULL); - if (optPtr->symbolCosts->huf.repeatMode == HUF_repeat_valid) { - /* huffman table presumed generated by dictionary */ - optPtr->priceType = zop_dynamic; - - if (compressedLiterals) { - unsigned lit; - assert(optPtr->litFreq != NULL); - optPtr->litSum = 0; - for (lit=0; lit<=MaxLit; lit++) { - U32 const scaleLog = 11; /* scale to 2K */ - U32 const bitCost = HUF_getNbBits(optPtr->symbolCosts->huf.CTable, lit); - assert(bitCost <= scaleLog); - optPtr->litFreq[lit] = bitCost ? 1 << (scaleLog-bitCost) : 1 /*minimum to calculate cost*/; - optPtr->litSum += optPtr->litFreq[lit]; - } } - - { unsigned ll; - FSE_CState_t llstate; - FSE_initCState(&llstate, optPtr->symbolCosts->fse.litlengthCTable); - optPtr->litLengthSum = 0; - for (ll=0; ll<=MaxLL; ll++) { - U32 const scaleLog = 10; /* scale to 1K */ - U32 const bitCost = FSE_getMaxNbBits(llstate.symbolTT, ll); - assert(bitCost < scaleLog); - optPtr->litLengthFreq[ll] = bitCost ? 1 << (scaleLog-bitCost) : 1 /*minimum to calculate cost*/; - optPtr->litLengthSum += optPtr->litLengthFreq[ll]; - } } - - { unsigned ml; - FSE_CState_t mlstate; - FSE_initCState(&mlstate, optPtr->symbolCosts->fse.matchlengthCTable); - optPtr->matchLengthSum = 0; - for (ml=0; ml<=MaxML; ml++) { - U32 const scaleLog = 10; - U32 const bitCost = FSE_getMaxNbBits(mlstate.symbolTT, ml); - assert(bitCost < scaleLog); - optPtr->matchLengthFreq[ml] = bitCost ? 1 << (scaleLog-bitCost) : 1 /*minimum to calculate cost*/; - optPtr->matchLengthSum += optPtr->matchLengthFreq[ml]; - } } - - { unsigned of; - FSE_CState_t ofstate; - FSE_initCState(&ofstate, optPtr->symbolCosts->fse.offcodeCTable); - optPtr->offCodeSum = 0; - for (of=0; of<=MaxOff; of++) { - U32 const scaleLog = 10; - U32 const bitCost = FSE_getMaxNbBits(ofstate.symbolTT, of); - assert(bitCost < scaleLog); - optPtr->offCodeFreq[of] = bitCost ? 1 << (scaleLog-bitCost) : 1 /*minimum to calculate cost*/; - optPtr->offCodeSum += optPtr->offCodeFreq[of]; - } } - - } else { /* not a dictionary */ - - assert(optPtr->litFreq != NULL); - if (compressedLiterals) { - unsigned lit = MaxLit; - HIST_count_simple(optPtr->litFreq, &lit, src, srcSize); /* use raw first block to init statistics */ - optPtr->litSum = ZSTD_downscaleStat(optPtr->litFreq, MaxLit, 1); - } - - { unsigned ll; - for (ll=0; ll<=MaxLL; ll++) - optPtr->litLengthFreq[ll] = 1; - } - optPtr->litLengthSum = MaxLL+1; - - { unsigned ml; - for (ml=0; ml<=MaxML; ml++) - optPtr->matchLengthFreq[ml] = 1; - } - optPtr->matchLengthSum = MaxML+1; - - { unsigned of; - for (of=0; of<=MaxOff; of++) - optPtr->offCodeFreq[of] = 1; - } - optPtr->offCodeSum = MaxOff+1; - - } - - } else { /* new block : re-use previous statistics, scaled down */ - - if (compressedLiterals) - optPtr->litSum = ZSTD_downscaleStat(optPtr->litFreq, MaxLit, 1); - optPtr->litLengthSum = ZSTD_downscaleStat(optPtr->litLengthFreq, MaxLL, 0); - optPtr->matchLengthSum = ZSTD_downscaleStat(optPtr->matchLengthFreq, MaxML, 0); - optPtr->offCodeSum = ZSTD_downscaleStat(optPtr->offCodeFreq, MaxOff, 0); - } - - ZSTD_setBasePrices(optPtr, optLevel); -} - -/* ZSTD_rawLiteralsCost() : - * price of literals (only) in specified segment (which length can be 0). - * does not include price of literalLength symbol */ -static U32 ZSTD_rawLiteralsCost(const BYTE* const literals, U32 const litLength, - const optState_t* const optPtr, - int optLevel) -{ - if (litLength == 0) return 0; - - if (!ZSTD_compressedLiterals(optPtr)) - return (litLength << 3) * BITCOST_MULTIPLIER; /* Uncompressed - 8 bytes per literal. */ - - if (optPtr->priceType == zop_predef) - return (litLength*6) * BITCOST_MULTIPLIER; /* 6 bit per literal - no statistic used */ - - /* dynamic statistics */ - { U32 price = litLength * optPtr->litSumBasePrice; - U32 u; - for (u=0; u < litLength; u++) { - assert(WEIGHT(optPtr->litFreq[literals[u]], optLevel) <= optPtr->litSumBasePrice); /* literal cost should never be negative */ - price -= WEIGHT(optPtr->litFreq[literals[u]], optLevel); - } - return price; - } -} - -/* ZSTD_litLengthPrice() : - * cost of literalLength symbol */ -static U32 ZSTD_litLengthPrice(U32 const litLength, const optState_t* const optPtr, int optLevel) -{ - if (optPtr->priceType == zop_predef) return WEIGHT(litLength, optLevel); - - /* dynamic statistics */ - { U32 const llCode = ZSTD_LLcode(litLength); - return (LL_bits[llCode] * BITCOST_MULTIPLIER) - + optPtr->litLengthSumBasePrice - - WEIGHT(optPtr->litLengthFreq[llCode], optLevel); - } -} - -/* ZSTD_getMatchPrice() : - * Provides the cost of the match part (offset + matchLength) of a sequence - * Must be combined with ZSTD_fullLiteralsCost() to get the full cost of a sequence. - * optLevel: when <2, favors small offset for decompression speed (improved cache efficiency) */ -FORCE_INLINE_TEMPLATE U32 -ZSTD_getMatchPrice(U32 const offset, - U32 const matchLength, - const optState_t* const optPtr, - int const optLevel) -{ - U32 price; - U32 const offCode = ZSTD_highbit32(offset+1); - U32 const mlBase = matchLength - MINMATCH; - assert(matchLength >= MINMATCH); - - if (optPtr->priceType == zop_predef) /* fixed scheme, do not use statistics */ - return WEIGHT(mlBase, optLevel) + ((16 + offCode) * BITCOST_MULTIPLIER); - - /* dynamic statistics */ - price = (offCode * BITCOST_MULTIPLIER) + (optPtr->offCodeSumBasePrice - WEIGHT(optPtr->offCodeFreq[offCode], optLevel)); - if ((optLevel<2) /*static*/ && offCode >= 20) - price += (offCode-19)*2 * BITCOST_MULTIPLIER; /* handicap for long distance offsets, favor decompression speed */ - - /* match Length */ - { U32 const mlCode = ZSTD_MLcode(mlBase); - price += (ML_bits[mlCode] * BITCOST_MULTIPLIER) + (optPtr->matchLengthSumBasePrice - WEIGHT(optPtr->matchLengthFreq[mlCode], optLevel)); - } - - price += BITCOST_MULTIPLIER / 5; /* heuristic : make matches a bit more costly to favor less sequences -> faster decompression speed */ - - DEBUGLOG(8, "ZSTD_getMatchPrice(ml:%u) = %u", matchLength, price); - return price; -} - -/* ZSTD_updateStats() : - * assumption : literals + litLengtn <= iend */ -static void ZSTD_updateStats(optState_t* const optPtr, - U32 litLength, const BYTE* literals, - U32 offsetCode, U32 matchLength) -{ - /* literals */ - if (ZSTD_compressedLiterals(optPtr)) { - U32 u; - for (u=0; u < litLength; u++) - optPtr->litFreq[literals[u]] += ZSTD_LITFREQ_ADD; - optPtr->litSum += litLength*ZSTD_LITFREQ_ADD; - } - - /* literal Length */ - { U32 const llCode = ZSTD_LLcode(litLength); - optPtr->litLengthFreq[llCode]++; - optPtr->litLengthSum++; - } - - /* match offset code (0-2=>repCode; 3+=>offset+2) */ - { U32 const offCode = ZSTD_highbit32(offsetCode+1); - assert(offCode <= MaxOff); - optPtr->offCodeFreq[offCode]++; - optPtr->offCodeSum++; - } - - /* match Length */ - { U32 const mlBase = matchLength - MINMATCH; - U32 const mlCode = ZSTD_MLcode(mlBase); - optPtr->matchLengthFreq[mlCode]++; - optPtr->matchLengthSum++; - } -} - - -/* ZSTD_readMINMATCH() : - * function safe only for comparisons - * assumption : memPtr must be at least 4 bytes before end of buffer */ -MEM_STATIC U32 ZSTD_readMINMATCH(const void* memPtr, U32 length) -{ - switch (length) - { - default : - case 4 : return MEM_read32(memPtr); - case 3 : if (MEM_isLittleEndian()) - return MEM_read32(memPtr)<<8; - else - return MEM_read32(memPtr)>>8; - } -} - - -/* Update hashTable3 up to ip (excluded) - Assumption : always within prefix (i.e. not within extDict) */ -static U32 ZSTD_insertAndFindFirstIndexHash3 (ZSTD_matchState_t* ms, - U32* nextToUpdate3, - const BYTE* const ip) -{ - U32* const hashTable3 = ms->hashTable3; - U32 const hashLog3 = ms->hashLog3; - const BYTE* const base = ms->window.base; - U32 idx = *nextToUpdate3; - U32 const target = (U32)(ip - base); - size_t const hash3 = ZSTD_hash3Ptr(ip, hashLog3); - assert(hashLog3 > 0); - - while(idx < target) { - hashTable3[ZSTD_hash3Ptr(base+idx, hashLog3)] = idx; - idx++; - } - - *nextToUpdate3 = target; - return hashTable3[hash3]; -} - - -/*-************************************* -* Binary Tree search -***************************************/ -/** ZSTD_insertBt1() : add one or multiple positions to tree. - * ip : assumed <= iend-8 . - * @return : nb of positions added */ -static U32 ZSTD_insertBt1( - ZSTD_matchState_t* ms, - const BYTE* const ip, const BYTE* const iend, - U32 const mls, const int extDict) -{ - const ZSTD_compressionParameters* const cParams = &ms->cParams; - U32* const hashTable = ms->hashTable; - U32 const hashLog = cParams->hashLog; - size_t const h = ZSTD_hashPtr(ip, hashLog, mls); - U32* const bt = ms->chainTable; - U32 const btLog = cParams->chainLog - 1; - U32 const btMask = (1 << btLog) - 1; - U32 matchIndex = hashTable[h]; - size_t commonLengthSmaller=0, commonLengthLarger=0; - const BYTE* const base = ms->window.base; - const BYTE* const dictBase = ms->window.dictBase; - const U32 dictLimit = ms->window.dictLimit; - const BYTE* const dictEnd = dictBase + dictLimit; - const BYTE* const prefixStart = base + dictLimit; - const BYTE* match; - const U32 current = (U32)(ip-base); - const U32 btLow = btMask >= current ? 0 : current - btMask; - U32* smallerPtr = bt + 2*(current&btMask); - U32* largerPtr = smallerPtr + 1; - U32 dummy32; /* to be nullified at the end */ - U32 const windowLow = ms->window.lowLimit; - U32 matchEndIdx = current+8+1; - size_t bestLength = 8; - U32 nbCompares = 1U << cParams->searchLog; -#ifdef ZSTD_C_PREDICT - U32 predictedSmall = *(bt + 2*((current-1)&btMask) + 0); - U32 predictedLarge = *(bt + 2*((current-1)&btMask) + 1); - predictedSmall += (predictedSmall>0); - predictedLarge += (predictedLarge>0); -#endif /* ZSTD_C_PREDICT */ - - DEBUGLOG(8, "ZSTD_insertBt1 (%u)", current); - - assert(ip <= iend-8); /* required for h calculation */ - hashTable[h] = current; /* Update Hash Table */ - - assert(windowLow > 0); - while (nbCompares-- && (matchIndex >= windowLow)) { - U32* const nextPtr = bt + 2*(matchIndex & btMask); - size_t matchLength = MIN(commonLengthSmaller, commonLengthLarger); /* guaranteed minimum nb of common bytes */ - assert(matchIndex < current); - -#ifdef ZSTD_C_PREDICT /* note : can create issues when hlog small <= 11 */ - const U32* predictPtr = bt + 2*((matchIndex-1) & btMask); /* written this way, as bt is a roll buffer */ - if (matchIndex == predictedSmall) { - /* no need to check length, result known */ - *smallerPtr = matchIndex; - if (matchIndex <= btLow) { smallerPtr=&dummy32; break; } /* beyond tree size, stop the search */ - smallerPtr = nextPtr+1; /* new "smaller" => larger of match */ - matchIndex = nextPtr[1]; /* new matchIndex larger than previous (closer to current) */ - predictedSmall = predictPtr[1] + (predictPtr[1]>0); - continue; - } - if (matchIndex == predictedLarge) { - *largerPtr = matchIndex; - if (matchIndex <= btLow) { largerPtr=&dummy32; break; } /* beyond tree size, stop the search */ - largerPtr = nextPtr; - matchIndex = nextPtr[0]; - predictedLarge = predictPtr[0] + (predictPtr[0]>0); - continue; - } -#endif - - if (!extDict || (matchIndex+matchLength >= dictLimit)) { - assert(matchIndex+matchLength >= dictLimit); /* might be wrong if actually extDict */ - match = base + matchIndex; - matchLength += ZSTD_count(ip+matchLength, match+matchLength, iend); - } else { - match = dictBase + matchIndex; - matchLength += ZSTD_count_2segments(ip+matchLength, match+matchLength, iend, dictEnd, prefixStart); - if (matchIndex+matchLength >= dictLimit) - match = base + matchIndex; /* to prepare for next usage of match[matchLength] */ - } - - if (matchLength > bestLength) { - bestLength = matchLength; - if (matchLength > matchEndIdx - matchIndex) - matchEndIdx = matchIndex + (U32)matchLength; - } - - if (ip+matchLength == iend) { /* equal : no way to know if inf or sup */ - break; /* drop , to guarantee consistency ; miss a bit of compression, but other solutions can corrupt tree */ - } - - if (match[matchLength] < ip[matchLength]) { /* necessarily within buffer */ - /* match is smaller than current */ - *smallerPtr = matchIndex; /* update smaller idx */ - commonLengthSmaller = matchLength; /* all smaller will now have at least this guaranteed common length */ - if (matchIndex <= btLow) { smallerPtr=&dummy32; break; } /* beyond tree size, stop searching */ - smallerPtr = nextPtr+1; /* new "candidate" => larger than match, which was smaller than target */ - matchIndex = nextPtr[1]; /* new matchIndex, larger than previous and closer to current */ - } else { - /* match is larger than current */ - *largerPtr = matchIndex; - commonLengthLarger = matchLength; - if (matchIndex <= btLow) { largerPtr=&dummy32; break; } /* beyond tree size, stop searching */ - largerPtr = nextPtr; - matchIndex = nextPtr[0]; - } } - - *smallerPtr = *largerPtr = 0; - { U32 positions = 0; - if (bestLength > 384) positions = MIN(192, (U32)(bestLength - 384)); /* speed optimization */ - assert(matchEndIdx > current + 8); - return MAX(positions, matchEndIdx - (current + 8)); - } -} - -FORCE_INLINE_TEMPLATE -void ZSTD_updateTree_internal( - ZSTD_matchState_t* ms, - const BYTE* const ip, const BYTE* const iend, - const U32 mls, const ZSTD_dictMode_e dictMode) -{ - const BYTE* const base = ms->window.base; - U32 const target = (U32)(ip - base); - U32 idx = ms->nextToUpdate; - DEBUGLOG(6, "ZSTD_updateTree_internal, from %u to %u (dictMode:%u)", - idx, target, dictMode); - - while(idx < target) { - U32 const forward = ZSTD_insertBt1(ms, base+idx, iend, mls, dictMode == ZSTD_extDict); - assert(idx < (U32)(idx + forward)); - idx += forward; - } - assert((size_t)(ip - base) <= (size_t)(U32)(-1)); - assert((size_t)(iend - base) <= (size_t)(U32)(-1)); - ms->nextToUpdate = target; -} - -void ZSTD_updateTree(ZSTD_matchState_t* ms, const BYTE* ip, const BYTE* iend) { - ZSTD_updateTree_internal(ms, ip, iend, ms->cParams.minMatch, ZSTD_noDict); -} - -FORCE_INLINE_TEMPLATE -U32 ZSTD_insertBtAndGetAllMatches ( - ZSTD_match_t* matches, /* store result (found matches) in this table (presumed large enough) */ - ZSTD_matchState_t* ms, - U32* nextToUpdate3, - const BYTE* const ip, const BYTE* const iLimit, const ZSTD_dictMode_e dictMode, - const U32 rep[ZSTD_REP_NUM], - U32 const ll0, /* tells if associated literal length is 0 or not. This value must be 0 or 1 */ - const U32 lengthToBeat, - U32 const mls /* template */) -{ - const ZSTD_compressionParameters* const cParams = &ms->cParams; - U32 const sufficient_len = MIN(cParams->targetLength, ZSTD_OPT_NUM -1); - const BYTE* const base = ms->window.base; - U32 const current = (U32)(ip-base); - U32 const hashLog = cParams->hashLog; - U32 const minMatch = (mls==3) ? 3 : 4; - U32* const hashTable = ms->hashTable; - size_t const h = ZSTD_hashPtr(ip, hashLog, mls); - U32 matchIndex = hashTable[h]; - U32* const bt = ms->chainTable; - U32 const btLog = cParams->chainLog - 1; - U32 const btMask= (1U << btLog) - 1; - size_t commonLengthSmaller=0, commonLengthLarger=0; - const BYTE* const dictBase = ms->window.dictBase; - U32 const dictLimit = ms->window.dictLimit; - const BYTE* const dictEnd = dictBase + dictLimit; - const BYTE* const prefixStart = base + dictLimit; - U32 const btLow = (btMask >= current) ? 0 : current - btMask; - U32 const windowLow = ZSTD_getLowestMatchIndex(ms, current, cParams->windowLog); - U32 const matchLow = windowLow ? windowLow : 1; - U32* smallerPtr = bt + 2*(current&btMask); - U32* largerPtr = bt + 2*(current&btMask) + 1; - U32 matchEndIdx = current+8+1; /* farthest referenced position of any match => detects repetitive patterns */ - U32 dummy32; /* to be nullified at the end */ - U32 mnum = 0; - U32 nbCompares = 1U << cParams->searchLog; - - const ZSTD_matchState_t* dms = dictMode == ZSTD_dictMatchState ? ms->dictMatchState : NULL; - const ZSTD_compressionParameters* const dmsCParams = - dictMode == ZSTD_dictMatchState ? &dms->cParams : NULL; - const BYTE* const dmsBase = dictMode == ZSTD_dictMatchState ? dms->window.base : NULL; - const BYTE* const dmsEnd = dictMode == ZSTD_dictMatchState ? dms->window.nextSrc : NULL; - U32 const dmsHighLimit = dictMode == ZSTD_dictMatchState ? (U32)(dmsEnd - dmsBase) : 0; - U32 const dmsLowLimit = dictMode == ZSTD_dictMatchState ? dms->window.lowLimit : 0; - U32 const dmsIndexDelta = dictMode == ZSTD_dictMatchState ? windowLow - dmsHighLimit : 0; - U32 const dmsHashLog = dictMode == ZSTD_dictMatchState ? dmsCParams->hashLog : hashLog; - U32 const dmsBtLog = dictMode == ZSTD_dictMatchState ? dmsCParams->chainLog - 1 : btLog; - U32 const dmsBtMask = dictMode == ZSTD_dictMatchState ? (1U << dmsBtLog) - 1 : 0; - U32 const dmsBtLow = dictMode == ZSTD_dictMatchState && dmsBtMask < dmsHighLimit - dmsLowLimit ? dmsHighLimit - dmsBtMask : dmsLowLimit; - - size_t bestLength = lengthToBeat-1; - DEBUGLOG(8, "ZSTD_insertBtAndGetAllMatches: current=%u", current); - - /* check repCode */ - assert(ll0 <= 1); /* necessarily 1 or 0 */ - { U32 const lastR = ZSTD_REP_NUM + ll0; - U32 repCode; - for (repCode = ll0; repCode < lastR; repCode++) { - U32 const repOffset = (repCode==ZSTD_REP_NUM) ? (rep[0] - 1) : rep[repCode]; - U32 const repIndex = current - repOffset; - U32 repLen = 0; - assert(current >= dictLimit); - if (repOffset-1 /* intentional overflow, discards 0 and -1 */ < current-dictLimit) { /* equivalent to `current > repIndex >= dictLimit` */ - /* We must validate the repcode offset because when we're using a dictionary the - * valid offset range shrinks when the dictionary goes out of bounds. - */ - if ((repIndex >= windowLow) & (ZSTD_readMINMATCH(ip, minMatch) == ZSTD_readMINMATCH(ip - repOffset, minMatch))) { - repLen = (U32)ZSTD_count(ip+minMatch, ip+minMatch-repOffset, iLimit) + minMatch; - } - } else { /* repIndex < dictLimit || repIndex >= current */ - const BYTE* const repMatch = dictMode == ZSTD_dictMatchState ? - dmsBase + repIndex - dmsIndexDelta : - dictBase + repIndex; - assert(current >= windowLow); - if ( dictMode == ZSTD_extDict - && ( ((repOffset-1) /*intentional overflow*/ < current - windowLow) /* equivalent to `current > repIndex >= windowLow` */ - & (((U32)((dictLimit-1) - repIndex) >= 3) ) /* intentional overflow : do not test positions overlapping 2 memory segments */) - && (ZSTD_readMINMATCH(ip, minMatch) == ZSTD_readMINMATCH(repMatch, minMatch)) ) { - repLen = (U32)ZSTD_count_2segments(ip+minMatch, repMatch+minMatch, iLimit, dictEnd, prefixStart) + minMatch; - } - if (dictMode == ZSTD_dictMatchState - && ( ((repOffset-1) /*intentional overflow*/ < current - (dmsLowLimit + dmsIndexDelta)) /* equivalent to `current > repIndex >= dmsLowLimit` */ - & ((U32)((dictLimit-1) - repIndex) >= 3) ) /* intentional overflow : do not test positions overlapping 2 memory segments */ - && (ZSTD_readMINMATCH(ip, minMatch) == ZSTD_readMINMATCH(repMatch, minMatch)) ) { - repLen = (U32)ZSTD_count_2segments(ip+minMatch, repMatch+minMatch, iLimit, dmsEnd, prefixStart) + minMatch; - } } - /* save longer solution */ - if (repLen > bestLength) { - DEBUGLOG(8, "found repCode %u (ll0:%u, offset:%u) of length %u", - repCode, ll0, repOffset, repLen); - bestLength = repLen; - matches[mnum].off = repCode - ll0; - matches[mnum].len = (U32)repLen; - mnum++; - if ( (repLen > sufficient_len) - | (ip+repLen == iLimit) ) { /* best possible */ - return mnum; - } } } } - - /* HC3 match finder */ - if ((mls == 3) /*static*/ && (bestLength < mls)) { - U32 const matchIndex3 = ZSTD_insertAndFindFirstIndexHash3(ms, nextToUpdate3, ip); - if ((matchIndex3 >= matchLow) - & (current - matchIndex3 < (1<<18)) /*heuristic : longer distance likely too expensive*/ ) { - size_t mlen; - if ((dictMode == ZSTD_noDict) /*static*/ || (dictMode == ZSTD_dictMatchState) /*static*/ || (matchIndex3 >= dictLimit)) { - const BYTE* const match = base + matchIndex3; - mlen = ZSTD_count(ip, match, iLimit); - } else { - const BYTE* const match = dictBase + matchIndex3; - mlen = ZSTD_count_2segments(ip, match, iLimit, dictEnd, prefixStart); - } - - /* save best solution */ - if (mlen >= mls /* == 3 > bestLength */) { - DEBUGLOG(8, "found small match with hlog3, of length %u", - (U32)mlen); - bestLength = mlen; - assert(current > matchIndex3); - assert(mnum==0); /* no prior solution */ - matches[0].off = (current - matchIndex3) + ZSTD_REP_MOVE; - matches[0].len = (U32)mlen; - mnum = 1; - if ( (mlen > sufficient_len) | - (ip+mlen == iLimit) ) { /* best possible length */ - ms->nextToUpdate = current+1; /* skip insertion */ - return 1; - } } } - /* no dictMatchState lookup: dicts don't have a populated HC3 table */ - } - - hashTable[h] = current; /* Update Hash Table */ - - while (nbCompares-- && (matchIndex >= matchLow)) { - U32* const nextPtr = bt + 2*(matchIndex & btMask); - const BYTE* match; - size_t matchLength = MIN(commonLengthSmaller, commonLengthLarger); /* guaranteed minimum nb of common bytes */ - assert(current > matchIndex); - - if ((dictMode == ZSTD_noDict) || (dictMode == ZSTD_dictMatchState) || (matchIndex+matchLength >= dictLimit)) { - assert(matchIndex+matchLength >= dictLimit); /* ensure the condition is correct when !extDict */ - match = base + matchIndex; - if (matchIndex >= dictLimit) assert(memcmp(match, ip, matchLength) == 0); /* ensure early section of match is equal as expected */ - matchLength += ZSTD_count(ip+matchLength, match+matchLength, iLimit); - } else { - match = dictBase + matchIndex; - assert(memcmp(match, ip, matchLength) == 0); /* ensure early section of match is equal as expected */ - matchLength += ZSTD_count_2segments(ip+matchLength, match+matchLength, iLimit, dictEnd, prefixStart); - if (matchIndex+matchLength >= dictLimit) - match = base + matchIndex; /* prepare for match[matchLength] read */ - } - - if (matchLength > bestLength) { - DEBUGLOG(8, "found match of length %u at distance %u (offCode=%u)", - (U32)matchLength, current - matchIndex, current - matchIndex + ZSTD_REP_MOVE); - assert(matchEndIdx > matchIndex); - if (matchLength > matchEndIdx - matchIndex) - matchEndIdx = matchIndex + (U32)matchLength; - bestLength = matchLength; - matches[mnum].off = (current - matchIndex) + ZSTD_REP_MOVE; - matches[mnum].len = (U32)matchLength; - mnum++; - if ( (matchLength > ZSTD_OPT_NUM) - | (ip+matchLength == iLimit) /* equal : no way to know if inf or sup */) { - if (dictMode == ZSTD_dictMatchState) nbCompares = 0; /* break should also skip searching dms */ - break; /* drop, to preserve bt consistency (miss a little bit of compression) */ - } - } - - if (match[matchLength] < ip[matchLength]) { - /* match smaller than current */ - *smallerPtr = matchIndex; /* update smaller idx */ - commonLengthSmaller = matchLength; /* all smaller will now have at least this guaranteed common length */ - if (matchIndex <= btLow) { smallerPtr=&dummy32; break; } /* beyond tree size, stop the search */ - smallerPtr = nextPtr+1; /* new candidate => larger than match, which was smaller than current */ - matchIndex = nextPtr[1]; /* new matchIndex, larger than previous, closer to current */ - } else { - *largerPtr = matchIndex; - commonLengthLarger = matchLength; - if (matchIndex <= btLow) { largerPtr=&dummy32; break; } /* beyond tree size, stop the search */ - largerPtr = nextPtr; - matchIndex = nextPtr[0]; - } } - - *smallerPtr = *largerPtr = 0; - - if (dictMode == ZSTD_dictMatchState && nbCompares) { - size_t const dmsH = ZSTD_hashPtr(ip, dmsHashLog, mls); - U32 dictMatchIndex = dms->hashTable[dmsH]; - const U32* const dmsBt = dms->chainTable; - commonLengthSmaller = commonLengthLarger = 0; - while (nbCompares-- && (dictMatchIndex > dmsLowLimit)) { - const U32* const nextPtr = dmsBt + 2*(dictMatchIndex & dmsBtMask); - size_t matchLength = MIN(commonLengthSmaller, commonLengthLarger); /* guaranteed minimum nb of common bytes */ - const BYTE* match = dmsBase + dictMatchIndex; - matchLength += ZSTD_count_2segments(ip+matchLength, match+matchLength, iLimit, dmsEnd, prefixStart); - if (dictMatchIndex+matchLength >= dmsHighLimit) - match = base + dictMatchIndex + dmsIndexDelta; /* to prepare for next usage of match[matchLength] */ - - if (matchLength > bestLength) { - matchIndex = dictMatchIndex + dmsIndexDelta; - DEBUGLOG(8, "found dms match of length %u at distance %u (offCode=%u)", - (U32)matchLength, current - matchIndex, current - matchIndex + ZSTD_REP_MOVE); - if (matchLength > matchEndIdx - matchIndex) - matchEndIdx = matchIndex + (U32)matchLength; - bestLength = matchLength; - matches[mnum].off = (current - matchIndex) + ZSTD_REP_MOVE; - matches[mnum].len = (U32)matchLength; - mnum++; - if ( (matchLength > ZSTD_OPT_NUM) - | (ip+matchLength == iLimit) /* equal : no way to know if inf or sup */) { - break; /* drop, to guarantee consistency (miss a little bit of compression) */ - } - } - - if (dictMatchIndex <= dmsBtLow) { break; } /* beyond tree size, stop the search */ - if (match[matchLength] < ip[matchLength]) { - commonLengthSmaller = matchLength; /* all smaller will now have at least this guaranteed common length */ - dictMatchIndex = nextPtr[1]; /* new matchIndex larger than previous (closer to current) */ - } else { - /* match is larger than current */ - commonLengthLarger = matchLength; - dictMatchIndex = nextPtr[0]; - } - } - } - - assert(matchEndIdx > current+8); - ms->nextToUpdate = matchEndIdx - 8; /* skip repetitive patterns */ - return mnum; -} - - -FORCE_INLINE_TEMPLATE U32 ZSTD_BtGetAllMatches ( - ZSTD_match_t* matches, /* store result (match found, increasing size) in this table */ - ZSTD_matchState_t* ms, - U32* nextToUpdate3, - const BYTE* ip, const BYTE* const iHighLimit, const ZSTD_dictMode_e dictMode, - const U32 rep[ZSTD_REP_NUM], - U32 const ll0, - U32 const lengthToBeat) -{ - const ZSTD_compressionParameters* const cParams = &ms->cParams; - U32 const matchLengthSearch = cParams->minMatch; - DEBUGLOG(8, "ZSTD_BtGetAllMatches"); - if (ip < ms->window.base + ms->nextToUpdate) return 0; /* skipped area */ - ZSTD_updateTree_internal(ms, ip, iHighLimit, matchLengthSearch, dictMode); - switch(matchLengthSearch) - { - case 3 : return ZSTD_insertBtAndGetAllMatches(matches, ms, nextToUpdate3, ip, iHighLimit, dictMode, rep, ll0, lengthToBeat, 3); - default : - case 4 : return ZSTD_insertBtAndGetAllMatches(matches, ms, nextToUpdate3, ip, iHighLimit, dictMode, rep, ll0, lengthToBeat, 4); - case 5 : return ZSTD_insertBtAndGetAllMatches(matches, ms, nextToUpdate3, ip, iHighLimit, dictMode, rep, ll0, lengthToBeat, 5); - case 7 : - case 6 : return ZSTD_insertBtAndGetAllMatches(matches, ms, nextToUpdate3, ip, iHighLimit, dictMode, rep, ll0, lengthToBeat, 6); - } -} - - -/*-******************************* -* Optimal parser -*********************************/ - - -static U32 ZSTD_totalLen(ZSTD_optimal_t sol) -{ - return sol.litlen + sol.mlen; -} - -#if 0 /* debug */ - -static void -listStats(const U32* table, int lastEltID) -{ - int const nbElts = lastEltID + 1; - int enb; - for (enb=0; enb < nbElts; enb++) { - (void)table; - /* RAWLOG(2, "%3i:%3i, ", enb, table[enb]); */ - RAWLOG(2, "%4i,", table[enb]); - } - RAWLOG(2, " \n"); -} - -#endif - -FORCE_INLINE_TEMPLATE size_t -ZSTD_compressBlock_opt_generic(ZSTD_matchState_t* ms, - seqStore_t* seqStore, - U32 rep[ZSTD_REP_NUM], - const void* src, size_t srcSize, - const int optLevel, - const ZSTD_dictMode_e dictMode) -{ - optState_t* const optStatePtr = &ms->opt; - const BYTE* const istart = (const BYTE*)src; - const BYTE* ip = istart; - const BYTE* anchor = istart; - const BYTE* const iend = istart + srcSize; - const BYTE* const ilimit = iend - 8; - const BYTE* const base = ms->window.base; - const BYTE* const prefixStart = base + ms->window.dictLimit; - const ZSTD_compressionParameters* const cParams = &ms->cParams; - - U32 const sufficient_len = MIN(cParams->targetLength, ZSTD_OPT_NUM -1); - U32 const minMatch = (cParams->minMatch == 3) ? 3 : 4; - U32 nextToUpdate3 = ms->nextToUpdate; - - ZSTD_optimal_t* const opt = optStatePtr->priceTable; - ZSTD_match_t* const matches = optStatePtr->matchTable; - ZSTD_optimal_t lastSequence; - - /* init */ - DEBUGLOG(5, "ZSTD_compressBlock_opt_generic: current=%u, prefix=%u, nextToUpdate=%u", - (U32)(ip - base), ms->window.dictLimit, ms->nextToUpdate); - assert(optLevel <= 2); - ZSTD_rescaleFreqs(optStatePtr, (const BYTE*)src, srcSize, optLevel); - ip += (ip==prefixStart); - - /* Match Loop */ - while (ip < ilimit) { - U32 cur, last_pos = 0; - - /* find first match */ - { U32 const litlen = (U32)(ip - anchor); - U32 const ll0 = !litlen; - U32 const nbMatches = ZSTD_BtGetAllMatches(matches, ms, &nextToUpdate3, ip, iend, dictMode, rep, ll0, minMatch); - if (!nbMatches) { ip++; continue; } - - /* initialize opt[0] */ - { U32 i ; for (i=0; i immediate encoding */ - { U32 const maxML = matches[nbMatches-1].len; - U32 const maxOffset = matches[nbMatches-1].off; - DEBUGLOG(6, "found %u matches of maxLength=%u and maxOffCode=%u at cPos=%u => start new series", - nbMatches, maxML, maxOffset, (U32)(ip-prefixStart)); - - if (maxML > sufficient_len) { - lastSequence.litlen = litlen; - lastSequence.mlen = maxML; - lastSequence.off = maxOffset; - DEBUGLOG(6, "large match (%u>%u), immediate encoding", - maxML, sufficient_len); - cur = 0; - last_pos = ZSTD_totalLen(lastSequence); - goto _shortestPath; - } } - - /* set prices for first matches starting position == 0 */ - { U32 const literalsPrice = opt[0].price + ZSTD_litLengthPrice(0, optStatePtr, optLevel); - U32 pos; - U32 matchNb; - for (pos = 1; pos < minMatch; pos++) { - opt[pos].price = ZSTD_MAX_PRICE; /* mlen, litlen and price will be fixed during forward scanning */ - } - for (matchNb = 0; matchNb < nbMatches; matchNb++) { - U32 const offset = matches[matchNb].off; - U32 const end = matches[matchNb].len; - for ( ; pos <= end ; pos++ ) { - U32 const matchPrice = ZSTD_getMatchPrice(offset, pos, optStatePtr, optLevel); - U32 const sequencePrice = literalsPrice + matchPrice; - DEBUGLOG(7, "rPos:%u => set initial price : %.2f", - pos, ZSTD_fCost(sequencePrice)); - opt[pos].mlen = pos; - opt[pos].off = offset; - opt[pos].litlen = litlen; - opt[pos].price = sequencePrice; - } } - last_pos = pos-1; - } - } - - /* check further positions */ - for (cur = 1; cur <= last_pos; cur++) { - const BYTE* const inr = ip + cur; - assert(cur < ZSTD_OPT_NUM); - DEBUGLOG(7, "cPos:%zi==rPos:%u", inr-istart, cur) - - /* Fix current position with one literal if cheaper */ - { U32 const litlen = (opt[cur-1].mlen == 0) ? opt[cur-1].litlen + 1 : 1; - int const price = opt[cur-1].price - + ZSTD_rawLiteralsCost(ip+cur-1, 1, optStatePtr, optLevel) - + ZSTD_litLengthPrice(litlen, optStatePtr, optLevel) - - ZSTD_litLengthPrice(litlen-1, optStatePtr, optLevel); - assert(price < 1000000000); /* overflow check */ - if (price <= opt[cur].price) { - DEBUGLOG(7, "cPos:%zi==rPos:%u : better price (%.2f<=%.2f) using literal (ll==%u) (hist:%u,%u,%u)", - inr-istart, cur, ZSTD_fCost(price), ZSTD_fCost(opt[cur].price), litlen, - opt[cur-1].rep[0], opt[cur-1].rep[1], opt[cur-1].rep[2]); - opt[cur].mlen = 0; - opt[cur].off = 0; - opt[cur].litlen = litlen; - opt[cur].price = price; - } else { - DEBUGLOG(7, "cPos:%zi==rPos:%u : literal would cost more (%.2f>%.2f) (hist:%u,%u,%u)", - inr-istart, cur, ZSTD_fCost(price), ZSTD_fCost(opt[cur].price), - opt[cur].rep[0], opt[cur].rep[1], opt[cur].rep[2]); - } - } - - /* Set the repcodes of the current position. We must do it here - * because we rely on the repcodes of the 2nd to last sequence being - * correct to set the next chunks repcodes during the backward - * traversal. - */ - ZSTD_STATIC_ASSERT(sizeof(opt[cur].rep) == sizeof(repcodes_t)); - assert(cur >= opt[cur].mlen); - if (opt[cur].mlen != 0) { - U32 const prev = cur - opt[cur].mlen; - repcodes_t newReps = ZSTD_updateRep(opt[prev].rep, opt[cur].off, opt[cur].litlen==0); - memcpy(opt[cur].rep, &newReps, sizeof(repcodes_t)); - } else { - memcpy(opt[cur].rep, opt[cur - 1].rep, sizeof(repcodes_t)); - } - - /* last match must start at a minimum distance of 8 from oend */ - if (inr > ilimit) continue; - - if (cur == last_pos) break; - - if ( (optLevel==0) /*static_test*/ - && (opt[cur+1].price <= opt[cur].price + (BITCOST_MULTIPLIER/2)) ) { - DEBUGLOG(7, "move to next rPos:%u : price is <=", cur+1); - continue; /* skip unpromising positions; about ~+6% speed, -0.01 ratio */ - } - - { U32 const ll0 = (opt[cur].mlen != 0); - U32 const litlen = (opt[cur].mlen == 0) ? opt[cur].litlen : 0; - U32 const previousPrice = opt[cur].price; - U32 const basePrice = previousPrice + ZSTD_litLengthPrice(0, optStatePtr, optLevel); - U32 const nbMatches = ZSTD_BtGetAllMatches(matches, ms, &nextToUpdate3, inr, iend, dictMode, opt[cur].rep, ll0, minMatch); - U32 matchNb; - if (!nbMatches) { - DEBUGLOG(7, "rPos:%u : no match found", cur); - continue; - } - - { U32 const maxML = matches[nbMatches-1].len; - DEBUGLOG(7, "cPos:%zi==rPos:%u, found %u matches, of maxLength=%u", - inr-istart, cur, nbMatches, maxML); - - if ( (maxML > sufficient_len) - || (cur + maxML >= ZSTD_OPT_NUM) ) { - lastSequence.mlen = maxML; - lastSequence.off = matches[nbMatches-1].off; - lastSequence.litlen = litlen; - cur -= (opt[cur].mlen==0) ? opt[cur].litlen : 0; /* last sequence is actually only literals, fix cur to last match - note : may underflow, in which case, it's first sequence, and it's okay */ - last_pos = cur + ZSTD_totalLen(lastSequence); - if (cur > ZSTD_OPT_NUM) cur = 0; /* underflow => first match */ - goto _shortestPath; - } } - - /* set prices using matches found at position == cur */ - for (matchNb = 0; matchNb < nbMatches; matchNb++) { - U32 const offset = matches[matchNb].off; - U32 const lastML = matches[matchNb].len; - U32 const startML = (matchNb>0) ? matches[matchNb-1].len+1 : minMatch; - U32 mlen; - - DEBUGLOG(7, "testing match %u => offCode=%4u, mlen=%2u, llen=%2u", - matchNb, matches[matchNb].off, lastML, litlen); - - for (mlen = lastML; mlen >= startML; mlen--) { /* scan downward */ - U32 const pos = cur + mlen; - int const price = basePrice + ZSTD_getMatchPrice(offset, mlen, optStatePtr, optLevel); - - if ((pos > last_pos) || (price < opt[pos].price)) { - DEBUGLOG(7, "rPos:%u (ml=%2u) => new better price (%.2f<%.2f)", - pos, mlen, ZSTD_fCost(price), ZSTD_fCost(opt[pos].price)); - while (last_pos < pos) { opt[last_pos+1].price = ZSTD_MAX_PRICE; last_pos++; } /* fill empty positions */ - opt[pos].mlen = mlen; - opt[pos].off = offset; - opt[pos].litlen = litlen; - opt[pos].price = price; - } else { - DEBUGLOG(7, "rPos:%u (ml=%2u) => new price is worse (%.2f>=%.2f)", - pos, mlen, ZSTD_fCost(price), ZSTD_fCost(opt[pos].price)); - if (optLevel==0) break; /* early update abort; gets ~+10% speed for about -0.01 ratio loss */ - } - } } } - } /* for (cur = 1; cur <= last_pos; cur++) */ - - lastSequence = opt[last_pos]; - cur = last_pos > ZSTD_totalLen(lastSequence) ? last_pos - ZSTD_totalLen(lastSequence) : 0; /* single sequence, and it starts before `ip` */ - assert(cur < ZSTD_OPT_NUM); /* control overflow*/ - -_shortestPath: /* cur, last_pos, best_mlen, best_off have to be set */ - assert(opt[0].mlen == 0); - - /* Set the next chunk's repcodes based on the repcodes of the beginning - * of the last match, and the last sequence. This avoids us having to - * update them while traversing the sequences. - */ - if (lastSequence.mlen != 0) { - repcodes_t reps = ZSTD_updateRep(opt[cur].rep, lastSequence.off, lastSequence.litlen==0); - memcpy(rep, &reps, sizeof(reps)); - } else { - memcpy(rep, opt[cur].rep, sizeof(repcodes_t)); - } - - { U32 const storeEnd = cur + 1; - U32 storeStart = storeEnd; - U32 seqPos = cur; - - DEBUGLOG(6, "start reverse traversal (last_pos:%u, cur:%u)", - last_pos, cur); (void)last_pos; - assert(storeEnd < ZSTD_OPT_NUM); - DEBUGLOG(6, "last sequence copied into pos=%u (llen=%u,mlen=%u,ofc=%u)", - storeEnd, lastSequence.litlen, lastSequence.mlen, lastSequence.off); - opt[storeEnd] = lastSequence; - while (seqPos > 0) { - U32 const backDist = ZSTD_totalLen(opt[seqPos]); - storeStart--; - DEBUGLOG(6, "sequence from rPos=%u copied into pos=%u (llen=%u,mlen=%u,ofc=%u)", - seqPos, storeStart, opt[seqPos].litlen, opt[seqPos].mlen, opt[seqPos].off); - opt[storeStart] = opt[seqPos]; - seqPos = (seqPos > backDist) ? seqPos - backDist : 0; - } - - /* save sequences */ - DEBUGLOG(6, "sending selected sequences into seqStore") - { U32 storePos; - for (storePos=storeStart; storePos <= storeEnd; storePos++) { - U32 const llen = opt[storePos].litlen; - U32 const mlen = opt[storePos].mlen; - U32 const offCode = opt[storePos].off; - U32 const advance = llen + mlen; - DEBUGLOG(6, "considering seq starting at %zi, llen=%u, mlen=%u", - anchor - istart, (unsigned)llen, (unsigned)mlen); - - if (mlen==0) { /* only literals => must be last "sequence", actually starting a new stream of sequences */ - assert(storePos == storeEnd); /* must be last sequence */ - ip = anchor + llen; /* last "sequence" is a bunch of literals => don't progress anchor */ - continue; /* will finish */ - } - - assert(anchor + llen <= iend); - ZSTD_updateStats(optStatePtr, llen, anchor, offCode, mlen); - ZSTD_storeSeq(seqStore, llen, anchor, iend, offCode, mlen-MINMATCH); - anchor += advance; - ip = anchor; - } } - ZSTD_setBasePrices(optStatePtr, optLevel); - } - } /* while (ip < ilimit) */ - - /* Return the last literals size */ - return (size_t)(iend - anchor); -} - - -size_t ZSTD_compressBlock_btopt( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - const void* src, size_t srcSize) -{ - DEBUGLOG(5, "ZSTD_compressBlock_btopt"); - return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 0 /*optLevel*/, ZSTD_noDict); -} - - -/* used in 2-pass strategy */ -static U32 ZSTD_upscaleStat(unsigned* table, U32 lastEltIndex, int bonus) -{ - U32 s, sum=0; - assert(ZSTD_FREQ_DIV+bonus >= 0); - for (s=0; slitSum = ZSTD_upscaleStat(optPtr->litFreq, MaxLit, 0); - optPtr->litLengthSum = ZSTD_upscaleStat(optPtr->litLengthFreq, MaxLL, 0); - optPtr->matchLengthSum = ZSTD_upscaleStat(optPtr->matchLengthFreq, MaxML, 0); - optPtr->offCodeSum = ZSTD_upscaleStat(optPtr->offCodeFreq, MaxOff, 0); -} - -/* ZSTD_initStats_ultra(): - * make a first compression pass, just to seed stats with more accurate starting values. - * only works on first block, with no dictionary and no ldm. - * this function cannot error, hence its contract must be respected. - */ -static void -ZSTD_initStats_ultra(ZSTD_matchState_t* ms, - seqStore_t* seqStore, - U32 rep[ZSTD_REP_NUM], - const void* src, size_t srcSize) -{ - U32 tmpRep[ZSTD_REP_NUM]; /* updated rep codes will sink here */ - memcpy(tmpRep, rep, sizeof(tmpRep)); - - DEBUGLOG(4, "ZSTD_initStats_ultra (srcSize=%zu)", srcSize); - assert(ms->opt.litLengthSum == 0); /* first block */ - assert(seqStore->sequences == seqStore->sequencesStart); /* no ldm */ - assert(ms->window.dictLimit == ms->window.lowLimit); /* no dictionary */ - assert(ms->window.dictLimit - ms->nextToUpdate <= 1); /* no prefix (note: intentional overflow, defined as 2-complement) */ - - ZSTD_compressBlock_opt_generic(ms, seqStore, tmpRep, src, srcSize, 2 /*optLevel*/, ZSTD_noDict); /* generate stats into ms->opt*/ - - /* invalidate first scan from history */ - ZSTD_resetSeqStore(seqStore); - ms->window.base -= srcSize; - ms->window.dictLimit += (U32)srcSize; - ms->window.lowLimit = ms->window.dictLimit; - ms->nextToUpdate = ms->window.dictLimit; - - /* re-inforce weight of collected statistics */ - ZSTD_upscaleStats(&ms->opt); -} - -size_t ZSTD_compressBlock_btultra( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - const void* src, size_t srcSize) -{ - DEBUGLOG(5, "ZSTD_compressBlock_btultra (srcSize=%zu)", srcSize); - return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 2 /*optLevel*/, ZSTD_noDict); -} - -size_t ZSTD_compressBlock_btultra2( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - const void* src, size_t srcSize) -{ - U32 const current = (U32)((const BYTE*)src - ms->window.base); - DEBUGLOG(5, "ZSTD_compressBlock_btultra2 (srcSize=%zu)", srcSize); - - /* 2-pass strategy: - * this strategy makes a first pass over first block to collect statistics - * and seed next round's statistics with it. - * After 1st pass, function forgets everything, and starts a new block. - * Consequently, this can only work if no data has been previously loaded in tables, - * aka, no dictionary, no prefix, no ldm preprocessing. - * The compression ratio gain is generally small (~0.5% on first block), - * the cost is 2x cpu time on first block. */ - assert(srcSize <= ZSTD_BLOCKSIZE_MAX); - if ( (ms->opt.litLengthSum==0) /* first block */ - && (seqStore->sequences == seqStore->sequencesStart) /* no ldm */ - && (ms->window.dictLimit == ms->window.lowLimit) /* no dictionary */ - && (current == ms->window.dictLimit) /* start of frame, nothing already loaded nor skipped */ - && (srcSize > ZSTD_PREDEF_THRESHOLD) - ) { - ZSTD_initStats_ultra(ms, seqStore, rep, src, srcSize); - } - - return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 2 /*optLevel*/, ZSTD_noDict); -} - -size_t ZSTD_compressBlock_btopt_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - const void* src, size_t srcSize) -{ - return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 0 /*optLevel*/, ZSTD_dictMatchState); -} - -size_t ZSTD_compressBlock_btultra_dictMatchState( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - const void* src, size_t srcSize) -{ - return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 2 /*optLevel*/, ZSTD_dictMatchState); -} - -size_t ZSTD_compressBlock_btopt_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - const void* src, size_t srcSize) -{ - return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 0 /*optLevel*/, ZSTD_extDict); -} - -size_t ZSTD_compressBlock_btultra_extDict( - ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], - const void* src, size_t srcSize) -{ - return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 2 /*optLevel*/, ZSTD_extDict); -} - -/* note : no btultra2 variant for extDict nor dictMatchState, - * because btultra2 is not meant to work with dictionaries - * and is only specific for the first block (no prefix) */ -/**** ended inlining compress/zstd_opt.c ****/ - -/**** start inlining decompress/huf_decompress.c ****/ -/* ****************************************************************** - * huff0 huffman decoder, - * part of Finite State Entropy library - * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc. - * - * You can contact the author at : - * - FSE+HUF source repository : https://github.com/Cyan4973/FiniteStateEntropy - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. -****************************************************************** */ - -/* ************************************************************** -* Dependencies -****************************************************************/ -#include /* memcpy, memset */ -/**** skipping file: ../common/compiler.h ****/ -/**** skipping file: ../common/bitstream.h ****/ -/**** skipping file: ../common/fse.h ****/ -#define HUF_STATIC_LINKING_ONLY -/**** skipping file: ../common/huf.h ****/ -/**** skipping file: ../common/error_private.h ****/ - -/* ************************************************************** -* Macros -****************************************************************/ - -/* These two optional macros force the use one way or another of the two - * Huffman decompression implementations. You can't force in both directions - * at the same time. - */ -#if defined(HUF_FORCE_DECOMPRESS_X1) && \ - defined(HUF_FORCE_DECOMPRESS_X2) -#error "Cannot force the use of the X1 and X2 decoders at the same time!" -#endif - - -/* ************************************************************** -* Error Management -****************************************************************/ -#define HUF_isError ERR_isError - - -/* ************************************************************** -* Byte alignment for workSpace management -****************************************************************/ -#define HUF_ALIGN(x, a) HUF_ALIGN_MASK((x), (a) - 1) -#define HUF_ALIGN_MASK(x, mask) (((x) + (mask)) & ~(mask)) - - -/* ************************************************************** -* BMI2 Variant Wrappers -****************************************************************/ -#if DYNAMIC_BMI2 - -#define HUF_DGEN(fn) \ - \ - static size_t fn##_default( \ - void* dst, size_t dstSize, \ - const void* cSrc, size_t cSrcSize, \ - const HUF_DTable* DTable) \ - { \ - return fn##_body(dst, dstSize, cSrc, cSrcSize, DTable); \ - } \ - \ - static TARGET_ATTRIBUTE("bmi2") size_t fn##_bmi2( \ - void* dst, size_t dstSize, \ - const void* cSrc, size_t cSrcSize, \ - const HUF_DTable* DTable) \ - { \ - return fn##_body(dst, dstSize, cSrc, cSrcSize, DTable); \ - } \ - \ - static size_t fn(void* dst, size_t dstSize, void const* cSrc, \ - size_t cSrcSize, HUF_DTable const* DTable, int bmi2) \ - { \ - if (bmi2) { \ - return fn##_bmi2(dst, dstSize, cSrc, cSrcSize, DTable); \ - } \ - return fn##_default(dst, dstSize, cSrc, cSrcSize, DTable); \ - } - -#else - -#define HUF_DGEN(fn) \ - static size_t fn(void* dst, size_t dstSize, void const* cSrc, \ - size_t cSrcSize, HUF_DTable const* DTable, int bmi2) \ - { \ - (void)bmi2; \ - return fn##_body(dst, dstSize, cSrc, cSrcSize, DTable); \ - } - -#endif - - -/*-***************************/ -/* generic DTableDesc */ -/*-***************************/ -typedef struct { BYTE maxTableLog; BYTE tableType; BYTE tableLog; BYTE reserved; } DTableDesc; - -static DTableDesc HUF_getDTableDesc(const HUF_DTable* table) -{ - DTableDesc dtd; - memcpy(&dtd, table, sizeof(dtd)); - return dtd; -} - - -#ifndef HUF_FORCE_DECOMPRESS_X2 - -/*-***************************/ -/* single-symbol decoding */ -/*-***************************/ -typedef struct { BYTE byte; BYTE nbBits; } HUF_DEltX1; /* single-symbol decoding */ - -size_t HUF_readDTableX1_wksp(HUF_DTable* DTable, const void* src, size_t srcSize, void* workSpace, size_t wkspSize) -{ - U32 tableLog = 0; - U32 nbSymbols = 0; - size_t iSize; - void* const dtPtr = DTable + 1; - HUF_DEltX1* const dt = (HUF_DEltX1*)dtPtr; - - U32* rankVal; - BYTE* huffWeight; - size_t spaceUsed32 = 0; - - rankVal = (U32 *)workSpace + spaceUsed32; - spaceUsed32 += HUF_TABLELOG_ABSOLUTEMAX + 1; - huffWeight = (BYTE *)((U32 *)workSpace + spaceUsed32); - spaceUsed32 += HUF_ALIGN(HUF_SYMBOLVALUE_MAX + 1, sizeof(U32)) >> 2; - - if ((spaceUsed32 << 2) > wkspSize) return ERROR(tableLog_tooLarge); - - DEBUG_STATIC_ASSERT(sizeof(DTableDesc) == sizeof(HUF_DTable)); - /* memset(huffWeight, 0, sizeof(huffWeight)); */ /* is not necessary, even though some analyzer complain ... */ - - iSize = HUF_readStats(huffWeight, HUF_SYMBOLVALUE_MAX + 1, rankVal, &nbSymbols, &tableLog, src, srcSize); - if (HUF_isError(iSize)) return iSize; - - /* Table header */ - { DTableDesc dtd = HUF_getDTableDesc(DTable); - if (tableLog > (U32)(dtd.maxTableLog+1)) return ERROR(tableLog_tooLarge); /* DTable too small, Huffman tree cannot fit in */ - dtd.tableType = 0; - dtd.tableLog = (BYTE)tableLog; - memcpy(DTable, &dtd, sizeof(dtd)); - } - - /* Calculate starting value for each rank */ - { U32 n, nextRankStart = 0; - for (n=1; n> 1; - size_t const uStart = rankVal[w]; - size_t const uEnd = uStart + length; - size_t u; - HUF_DEltX1 D; - D.byte = (BYTE)n; - D.nbBits = (BYTE)(tableLog + 1 - w); - rankVal[w] = (U32)uEnd; - if (length < 4) { - /* Use length in the loop bound so the compiler knows it is short. */ - for (u = 0; u < length; ++u) - dt[uStart + u] = D; - } else { - /* Unroll the loop 4 times, we know it is a power of 2. */ - for (u = uStart; u < uEnd; u += 4) { - dt[u + 0] = D; - dt[u + 1] = D; - dt[u + 2] = D; - dt[u + 3] = D; - } } } } - return iSize; -} - -size_t HUF_readDTableX1(HUF_DTable* DTable, const void* src, size_t srcSize) -{ - U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; - return HUF_readDTableX1_wksp(DTable, src, srcSize, - workSpace, sizeof(workSpace)); -} - -FORCE_INLINE_TEMPLATE BYTE -HUF_decodeSymbolX1(BIT_DStream_t* Dstream, const HUF_DEltX1* dt, const U32 dtLog) -{ - size_t const val = BIT_lookBitsFast(Dstream, dtLog); /* note : dtLog >= 1 */ - BYTE const c = dt[val].byte; - BIT_skipBits(Dstream, dt[val].nbBits); - return c; -} - -#define HUF_DECODE_SYMBOLX1_0(ptr, DStreamPtr) \ - *ptr++ = HUF_decodeSymbolX1(DStreamPtr, dt, dtLog) - -#define HUF_DECODE_SYMBOLX1_1(ptr, DStreamPtr) \ - if (MEM_64bits() || (HUF_TABLELOG_MAX<=12)) \ - HUF_DECODE_SYMBOLX1_0(ptr, DStreamPtr) - -#define HUF_DECODE_SYMBOLX1_2(ptr, DStreamPtr) \ - if (MEM_64bits()) \ - HUF_DECODE_SYMBOLX1_0(ptr, DStreamPtr) - -HINT_INLINE size_t -HUF_decodeStreamX1(BYTE* p, BIT_DStream_t* const bitDPtr, BYTE* const pEnd, const HUF_DEltX1* const dt, const U32 dtLog) -{ - BYTE* const pStart = p; - - /* up to 4 symbols at a time */ - while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) & (p < pEnd-3)) { - HUF_DECODE_SYMBOLX1_2(p, bitDPtr); - HUF_DECODE_SYMBOLX1_1(p, bitDPtr); - HUF_DECODE_SYMBOLX1_2(p, bitDPtr); - HUF_DECODE_SYMBOLX1_0(p, bitDPtr); - } - - /* [0-3] symbols remaining */ - if (MEM_32bits()) - while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) & (p < pEnd)) - HUF_DECODE_SYMBOLX1_0(p, bitDPtr); - - /* no more data to retrieve from bitstream, no need to reload */ - while (p < pEnd) - HUF_DECODE_SYMBOLX1_0(p, bitDPtr); - - return pEnd-pStart; -} - -FORCE_INLINE_TEMPLATE size_t -HUF_decompress1X1_usingDTable_internal_body( - void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize, - const HUF_DTable* DTable) -{ - BYTE* op = (BYTE*)dst; - BYTE* const oend = op + dstSize; - const void* dtPtr = DTable + 1; - const HUF_DEltX1* const dt = (const HUF_DEltX1*)dtPtr; - BIT_DStream_t bitD; - DTableDesc const dtd = HUF_getDTableDesc(DTable); - U32 const dtLog = dtd.tableLog; - - CHECK_F( BIT_initDStream(&bitD, cSrc, cSrcSize) ); - - HUF_decodeStreamX1(op, &bitD, oend, dt, dtLog); - - if (!BIT_endOfDStream(&bitD)) return ERROR(corruption_detected); - - return dstSize; -} - -FORCE_INLINE_TEMPLATE size_t -HUF_decompress4X1_usingDTable_internal_body( - void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize, - const HUF_DTable* DTable) -{ - /* Check */ - if (cSrcSize < 10) return ERROR(corruption_detected); /* strict minimum : jump table + 1 byte per stream */ - - { const BYTE* const istart = (const BYTE*) cSrc; - BYTE* const ostart = (BYTE*) dst; - BYTE* const oend = ostart + dstSize; - BYTE* const olimit = oend - 3; - const void* const dtPtr = DTable + 1; - const HUF_DEltX1* const dt = (const HUF_DEltX1*)dtPtr; - - /* Init */ - BIT_DStream_t bitD1; - BIT_DStream_t bitD2; - BIT_DStream_t bitD3; - BIT_DStream_t bitD4; - size_t const length1 = MEM_readLE16(istart); - size_t const length2 = MEM_readLE16(istart+2); - size_t const length3 = MEM_readLE16(istart+4); - size_t const length4 = cSrcSize - (length1 + length2 + length3 + 6); - const BYTE* const istart1 = istart + 6; /* jumpTable */ - const BYTE* const istart2 = istart1 + length1; - const BYTE* const istart3 = istart2 + length2; - const BYTE* const istart4 = istart3 + length3; - const size_t segmentSize = (dstSize+3) / 4; - BYTE* const opStart2 = ostart + segmentSize; - BYTE* const opStart3 = opStart2 + segmentSize; - BYTE* const opStart4 = opStart3 + segmentSize; - BYTE* op1 = ostart; - BYTE* op2 = opStart2; - BYTE* op3 = opStart3; - BYTE* op4 = opStart4; - DTableDesc const dtd = HUF_getDTableDesc(DTable); - U32 const dtLog = dtd.tableLog; - U32 endSignal = 1; - - if (length4 > cSrcSize) return ERROR(corruption_detected); /* overflow */ - CHECK_F( BIT_initDStream(&bitD1, istart1, length1) ); - CHECK_F( BIT_initDStream(&bitD2, istart2, length2) ); - CHECK_F( BIT_initDStream(&bitD3, istart3, length3) ); - CHECK_F( BIT_initDStream(&bitD4, istart4, length4) ); - - /* up to 16 symbols per loop (4 symbols per stream) in 64-bit mode */ - for ( ; (endSignal) & (op4 < olimit) ; ) { - HUF_DECODE_SYMBOLX1_2(op1, &bitD1); - HUF_DECODE_SYMBOLX1_2(op2, &bitD2); - HUF_DECODE_SYMBOLX1_2(op3, &bitD3); - HUF_DECODE_SYMBOLX1_2(op4, &bitD4); - HUF_DECODE_SYMBOLX1_1(op1, &bitD1); - HUF_DECODE_SYMBOLX1_1(op2, &bitD2); - HUF_DECODE_SYMBOLX1_1(op3, &bitD3); - HUF_DECODE_SYMBOLX1_1(op4, &bitD4); - HUF_DECODE_SYMBOLX1_2(op1, &bitD1); - HUF_DECODE_SYMBOLX1_2(op2, &bitD2); - HUF_DECODE_SYMBOLX1_2(op3, &bitD3); - HUF_DECODE_SYMBOLX1_2(op4, &bitD4); - HUF_DECODE_SYMBOLX1_0(op1, &bitD1); - HUF_DECODE_SYMBOLX1_0(op2, &bitD2); - HUF_DECODE_SYMBOLX1_0(op3, &bitD3); - HUF_DECODE_SYMBOLX1_0(op4, &bitD4); - endSignal &= BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished; - endSignal &= BIT_reloadDStreamFast(&bitD2) == BIT_DStream_unfinished; - endSignal &= BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished; - endSignal &= BIT_reloadDStreamFast(&bitD4) == BIT_DStream_unfinished; - } - - /* check corruption */ - /* note : should not be necessary : op# advance in lock step, and we control op4. - * but curiously, binary generated by gcc 7.2 & 7.3 with -mbmi2 runs faster when >=1 test is present */ - if (op1 > opStart2) return ERROR(corruption_detected); - if (op2 > opStart3) return ERROR(corruption_detected); - if (op3 > opStart4) return ERROR(corruption_detected); - /* note : op4 supposed already verified within main loop */ - - /* finish bitStreams one by one */ - HUF_decodeStreamX1(op1, &bitD1, opStart2, dt, dtLog); - HUF_decodeStreamX1(op2, &bitD2, opStart3, dt, dtLog); - HUF_decodeStreamX1(op3, &bitD3, opStart4, dt, dtLog); - HUF_decodeStreamX1(op4, &bitD4, oend, dt, dtLog); - - /* check */ - { U32 const endCheck = BIT_endOfDStream(&bitD1) & BIT_endOfDStream(&bitD2) & BIT_endOfDStream(&bitD3) & BIT_endOfDStream(&bitD4); - if (!endCheck) return ERROR(corruption_detected); } - - /* decoded size */ - return dstSize; - } -} - - -typedef size_t (*HUF_decompress_usingDTable_t)(void *dst, size_t dstSize, - const void *cSrc, - size_t cSrcSize, - const HUF_DTable *DTable); - -HUF_DGEN(HUF_decompress1X1_usingDTable_internal) -HUF_DGEN(HUF_decompress4X1_usingDTable_internal) - - - -size_t HUF_decompress1X1_usingDTable( - void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize, - const HUF_DTable* DTable) -{ - DTableDesc dtd = HUF_getDTableDesc(DTable); - if (dtd.tableType != 0) return ERROR(GENERIC); - return HUF_decompress1X1_usingDTable_internal(dst, dstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); -} - -size_t HUF_decompress1X1_DCtx_wksp(HUF_DTable* DCtx, void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize, - void* workSpace, size_t wkspSize) -{ - const BYTE* ip = (const BYTE*) cSrc; - - size_t const hSize = HUF_readDTableX1_wksp(DCtx, cSrc, cSrcSize, workSpace, wkspSize); - if (HUF_isError(hSize)) return hSize; - if (hSize >= cSrcSize) return ERROR(srcSize_wrong); - ip += hSize; cSrcSize -= hSize; - - return HUF_decompress1X1_usingDTable_internal(dst, dstSize, ip, cSrcSize, DCtx, /* bmi2 */ 0); -} - - -size_t HUF_decompress1X1_DCtx(HUF_DTable* DCtx, void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize) -{ - U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; - return HUF_decompress1X1_DCtx_wksp(DCtx, dst, dstSize, cSrc, cSrcSize, - workSpace, sizeof(workSpace)); -} - -size_t HUF_decompress1X1 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) -{ - HUF_CREATE_STATIC_DTABLEX1(DTable, HUF_TABLELOG_MAX); - return HUF_decompress1X1_DCtx (DTable, dst, dstSize, cSrc, cSrcSize); -} - -size_t HUF_decompress4X1_usingDTable( - void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize, - const HUF_DTable* DTable) -{ - DTableDesc dtd = HUF_getDTableDesc(DTable); - if (dtd.tableType != 0) return ERROR(GENERIC); - return HUF_decompress4X1_usingDTable_internal(dst, dstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); -} - -static size_t HUF_decompress4X1_DCtx_wksp_bmi2(HUF_DTable* dctx, void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize, - void* workSpace, size_t wkspSize, int bmi2) -{ - const BYTE* ip = (const BYTE*) cSrc; - - size_t const hSize = HUF_readDTableX1_wksp (dctx, cSrc, cSrcSize, - workSpace, wkspSize); - if (HUF_isError(hSize)) return hSize; - if (hSize >= cSrcSize) return ERROR(srcSize_wrong); - ip += hSize; cSrcSize -= hSize; - - return HUF_decompress4X1_usingDTable_internal(dst, dstSize, ip, cSrcSize, dctx, bmi2); -} - -size_t HUF_decompress4X1_DCtx_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize, - void* workSpace, size_t wkspSize) -{ - return HUF_decompress4X1_DCtx_wksp_bmi2(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize, 0); -} - - -size_t HUF_decompress4X1_DCtx (HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) -{ - U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; - return HUF_decompress4X1_DCtx_wksp(dctx, dst, dstSize, cSrc, cSrcSize, - workSpace, sizeof(workSpace)); -} -size_t HUF_decompress4X1 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) -{ - HUF_CREATE_STATIC_DTABLEX1(DTable, HUF_TABLELOG_MAX); - return HUF_decompress4X1_DCtx(DTable, dst, dstSize, cSrc, cSrcSize); -} - -#endif /* HUF_FORCE_DECOMPRESS_X2 */ - - -#ifndef HUF_FORCE_DECOMPRESS_X1 - -/* *************************/ -/* double-symbols decoding */ -/* *************************/ - -typedef struct { U16 sequence; BYTE nbBits; BYTE length; } HUF_DEltX2; /* double-symbols decoding */ -typedef struct { BYTE symbol; BYTE weight; } sortedSymbol_t; -typedef U32 rankValCol_t[HUF_TABLELOG_MAX + 1]; -typedef rankValCol_t rankVal_t[HUF_TABLELOG_MAX]; - - -/* HUF_fillDTableX2Level2() : - * `rankValOrigin` must be a table of at least (HUF_TABLELOG_MAX + 1) U32 */ -static void HUF_fillDTableX2Level2(HUF_DEltX2* DTable, U32 sizeLog, const U32 consumed, - const U32* rankValOrigin, const int minWeight, - const sortedSymbol_t* sortedSymbols, const U32 sortedListSize, - U32 nbBitsBaseline, U16 baseSeq) -{ - HUF_DEltX2 DElt; - U32 rankVal[HUF_TABLELOG_MAX + 1]; - - /* get pre-calculated rankVal */ - memcpy(rankVal, rankValOrigin, sizeof(rankVal)); - - /* fill skipped values */ - if (minWeight>1) { - U32 i, skipSize = rankVal[minWeight]; - MEM_writeLE16(&(DElt.sequence), baseSeq); - DElt.nbBits = (BYTE)(consumed); - DElt.length = 1; - for (i = 0; i < skipSize; i++) - DTable[i] = DElt; - } - - /* fill DTable */ - { U32 s; for (s=0; s= 1 */ - - rankVal[weight] += length; - } } -} - - -static void HUF_fillDTableX2(HUF_DEltX2* DTable, const U32 targetLog, - const sortedSymbol_t* sortedList, const U32 sortedListSize, - const U32* rankStart, rankVal_t rankValOrigin, const U32 maxWeight, - const U32 nbBitsBaseline) -{ - U32 rankVal[HUF_TABLELOG_MAX + 1]; - const int scaleLog = nbBitsBaseline - targetLog; /* note : targetLog >= srcLog, hence scaleLog <= 1 */ - const U32 minBits = nbBitsBaseline - maxWeight; - U32 s; - - memcpy(rankVal, rankValOrigin, sizeof(rankVal)); - - /* fill DTable */ - for (s=0; s= minBits) { /* enough room for a second symbol */ - U32 sortedRank; - int minWeight = nbBits + scaleLog; - if (minWeight < 1) minWeight = 1; - sortedRank = rankStart[minWeight]; - HUF_fillDTableX2Level2(DTable+start, targetLog-nbBits, nbBits, - rankValOrigin[nbBits], minWeight, - sortedList+sortedRank, sortedListSize-sortedRank, - nbBitsBaseline, symbol); - } else { - HUF_DEltX2 DElt; - MEM_writeLE16(&(DElt.sequence), symbol); - DElt.nbBits = (BYTE)(nbBits); - DElt.length = 1; - { U32 const end = start + length; - U32 u; - for (u = start; u < end; u++) DTable[u] = DElt; - } } - rankVal[weight] += length; - } -} - -size_t HUF_readDTableX2_wksp(HUF_DTable* DTable, - const void* src, size_t srcSize, - void* workSpace, size_t wkspSize) -{ - U32 tableLog, maxW, sizeOfSort, nbSymbols; - DTableDesc dtd = HUF_getDTableDesc(DTable); - U32 const maxTableLog = dtd.maxTableLog; - size_t iSize; - void* dtPtr = DTable+1; /* force compiler to avoid strict-aliasing */ - HUF_DEltX2* const dt = (HUF_DEltX2*)dtPtr; - U32 *rankStart; - - rankValCol_t* rankVal; - U32* rankStats; - U32* rankStart0; - sortedSymbol_t* sortedSymbol; - BYTE* weightList; - size_t spaceUsed32 = 0; - - rankVal = (rankValCol_t *)((U32 *)workSpace + spaceUsed32); - spaceUsed32 += (sizeof(rankValCol_t) * HUF_TABLELOG_MAX) >> 2; - rankStats = (U32 *)workSpace + spaceUsed32; - spaceUsed32 += HUF_TABLELOG_MAX + 1; - rankStart0 = (U32 *)workSpace + spaceUsed32; - spaceUsed32 += HUF_TABLELOG_MAX + 2; - sortedSymbol = (sortedSymbol_t *)workSpace + (spaceUsed32 * sizeof(U32)) / sizeof(sortedSymbol_t); - spaceUsed32 += HUF_ALIGN(sizeof(sortedSymbol_t) * (HUF_SYMBOLVALUE_MAX + 1), sizeof(U32)) >> 2; - weightList = (BYTE *)((U32 *)workSpace + spaceUsed32); - spaceUsed32 += HUF_ALIGN(HUF_SYMBOLVALUE_MAX + 1, sizeof(U32)) >> 2; - - if ((spaceUsed32 << 2) > wkspSize) return ERROR(tableLog_tooLarge); - - rankStart = rankStart0 + 1; - memset(rankStats, 0, sizeof(U32) * (2 * HUF_TABLELOG_MAX + 2 + 1)); - - DEBUG_STATIC_ASSERT(sizeof(HUF_DEltX2) == sizeof(HUF_DTable)); /* if compiler fails here, assertion is wrong */ - if (maxTableLog > HUF_TABLELOG_MAX) return ERROR(tableLog_tooLarge); - /* memset(weightList, 0, sizeof(weightList)); */ /* is not necessary, even though some analyzer complain ... */ - - iSize = HUF_readStats(weightList, HUF_SYMBOLVALUE_MAX + 1, rankStats, &nbSymbols, &tableLog, src, srcSize); - if (HUF_isError(iSize)) return iSize; - - /* check result */ - if (tableLog > maxTableLog) return ERROR(tableLog_tooLarge); /* DTable can't fit code depth */ - - /* find maxWeight */ - for (maxW = tableLog; rankStats[maxW]==0; maxW--) {} /* necessarily finds a solution before 0 */ - - /* Get start index of each weight */ - { U32 w, nextRankStart = 0; - for (w=1; w> consumed; - } } } } - - HUF_fillDTableX2(dt, maxTableLog, - sortedSymbol, sizeOfSort, - rankStart0, rankVal, maxW, - tableLog+1); - - dtd.tableLog = (BYTE)maxTableLog; - dtd.tableType = 1; - memcpy(DTable, &dtd, sizeof(dtd)); - return iSize; -} - -size_t HUF_readDTableX2(HUF_DTable* DTable, const void* src, size_t srcSize) -{ - U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; - return HUF_readDTableX2_wksp(DTable, src, srcSize, - workSpace, sizeof(workSpace)); -} - - -FORCE_INLINE_TEMPLATE U32 -HUF_decodeSymbolX2(void* op, BIT_DStream_t* DStream, const HUF_DEltX2* dt, const U32 dtLog) -{ - size_t const val = BIT_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */ - memcpy(op, dt+val, 2); - BIT_skipBits(DStream, dt[val].nbBits); - return dt[val].length; -} - -FORCE_INLINE_TEMPLATE U32 -HUF_decodeLastSymbolX2(void* op, BIT_DStream_t* DStream, const HUF_DEltX2* dt, const U32 dtLog) -{ - size_t const val = BIT_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */ - memcpy(op, dt+val, 1); - if (dt[val].length==1) BIT_skipBits(DStream, dt[val].nbBits); - else { - if (DStream->bitsConsumed < (sizeof(DStream->bitContainer)*8)) { - BIT_skipBits(DStream, dt[val].nbBits); - if (DStream->bitsConsumed > (sizeof(DStream->bitContainer)*8)) - /* ugly hack; works only because it's the last symbol. Note : can't easily extract nbBits from just this symbol */ - DStream->bitsConsumed = (sizeof(DStream->bitContainer)*8); - } } - return 1; -} - -#define HUF_DECODE_SYMBOLX2_0(ptr, DStreamPtr) \ - ptr += HUF_decodeSymbolX2(ptr, DStreamPtr, dt, dtLog) - -#define HUF_DECODE_SYMBOLX2_1(ptr, DStreamPtr) \ - if (MEM_64bits() || (HUF_TABLELOG_MAX<=12)) \ - ptr += HUF_decodeSymbolX2(ptr, DStreamPtr, dt, dtLog) - -#define HUF_DECODE_SYMBOLX2_2(ptr, DStreamPtr) \ - if (MEM_64bits()) \ - ptr += HUF_decodeSymbolX2(ptr, DStreamPtr, dt, dtLog) - -HINT_INLINE size_t -HUF_decodeStreamX2(BYTE* p, BIT_DStream_t* bitDPtr, BYTE* const pEnd, - const HUF_DEltX2* const dt, const U32 dtLog) -{ - BYTE* const pStart = p; - - /* up to 8 symbols at a time */ - while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) & (p < pEnd-(sizeof(bitDPtr->bitContainer)-1))) { - HUF_DECODE_SYMBOLX2_2(p, bitDPtr); - HUF_DECODE_SYMBOLX2_1(p, bitDPtr); - HUF_DECODE_SYMBOLX2_2(p, bitDPtr); - HUF_DECODE_SYMBOLX2_0(p, bitDPtr); - } - - /* closer to end : up to 2 symbols at a time */ - while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) & (p <= pEnd-2)) - HUF_DECODE_SYMBOLX2_0(p, bitDPtr); - - while (p <= pEnd-2) - HUF_DECODE_SYMBOLX2_0(p, bitDPtr); /* no need to reload : reached the end of DStream */ - - if (p < pEnd) - p += HUF_decodeLastSymbolX2(p, bitDPtr, dt, dtLog); - - return p-pStart; -} - -FORCE_INLINE_TEMPLATE size_t -HUF_decompress1X2_usingDTable_internal_body( - void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize, - const HUF_DTable* DTable) -{ - BIT_DStream_t bitD; - - /* Init */ - CHECK_F( BIT_initDStream(&bitD, cSrc, cSrcSize) ); - - /* decode */ - { BYTE* const ostart = (BYTE*) dst; - BYTE* const oend = ostart + dstSize; - const void* const dtPtr = DTable+1; /* force compiler to not use strict-aliasing */ - const HUF_DEltX2* const dt = (const HUF_DEltX2*)dtPtr; - DTableDesc const dtd = HUF_getDTableDesc(DTable); - HUF_decodeStreamX2(ostart, &bitD, oend, dt, dtd.tableLog); - } - - /* check */ - if (!BIT_endOfDStream(&bitD)) return ERROR(corruption_detected); - - /* decoded size */ - return dstSize; -} - -FORCE_INLINE_TEMPLATE size_t -HUF_decompress4X2_usingDTable_internal_body( - void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize, - const HUF_DTable* DTable) -{ - if (cSrcSize < 10) return ERROR(corruption_detected); /* strict minimum : jump table + 1 byte per stream */ - - { const BYTE* const istart = (const BYTE*) cSrc; - BYTE* const ostart = (BYTE*) dst; - BYTE* const oend = ostart + dstSize; - BYTE* const olimit = oend - (sizeof(size_t)-1); - const void* const dtPtr = DTable+1; - const HUF_DEltX2* const dt = (const HUF_DEltX2*)dtPtr; - - /* Init */ - BIT_DStream_t bitD1; - BIT_DStream_t bitD2; - BIT_DStream_t bitD3; - BIT_DStream_t bitD4; - size_t const length1 = MEM_readLE16(istart); - size_t const length2 = MEM_readLE16(istart+2); - size_t const length3 = MEM_readLE16(istart+4); - size_t const length4 = cSrcSize - (length1 + length2 + length3 + 6); - const BYTE* const istart1 = istart + 6; /* jumpTable */ - const BYTE* const istart2 = istart1 + length1; - const BYTE* const istart3 = istart2 + length2; - const BYTE* const istart4 = istart3 + length3; - size_t const segmentSize = (dstSize+3) / 4; - BYTE* const opStart2 = ostart + segmentSize; - BYTE* const opStart3 = opStart2 + segmentSize; - BYTE* const opStart4 = opStart3 + segmentSize; - BYTE* op1 = ostart; - BYTE* op2 = opStart2; - BYTE* op3 = opStart3; - BYTE* op4 = opStart4; - U32 endSignal = 1; - DTableDesc const dtd = HUF_getDTableDesc(DTable); - U32 const dtLog = dtd.tableLog; - - if (length4 > cSrcSize) return ERROR(corruption_detected); /* overflow */ - CHECK_F( BIT_initDStream(&bitD1, istart1, length1) ); - CHECK_F( BIT_initDStream(&bitD2, istart2, length2) ); - CHECK_F( BIT_initDStream(&bitD3, istart3, length3) ); - CHECK_F( BIT_initDStream(&bitD4, istart4, length4) ); - - /* 16-32 symbols per loop (4-8 symbols per stream) */ - for ( ; (endSignal) & (op4 < olimit); ) { -#if defined(__clang__) && (defined(__x86_64__) || defined(__i386__)) - HUF_DECODE_SYMBOLX2_2(op1, &bitD1); - HUF_DECODE_SYMBOLX2_1(op1, &bitD1); - HUF_DECODE_SYMBOLX2_2(op1, &bitD1); - HUF_DECODE_SYMBOLX2_0(op1, &bitD1); - HUF_DECODE_SYMBOLX2_2(op2, &bitD2); - HUF_DECODE_SYMBOLX2_1(op2, &bitD2); - HUF_DECODE_SYMBOLX2_2(op2, &bitD2); - HUF_DECODE_SYMBOLX2_0(op2, &bitD2); - endSignal &= BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished; - endSignal &= BIT_reloadDStreamFast(&bitD2) == BIT_DStream_unfinished; - HUF_DECODE_SYMBOLX2_2(op3, &bitD3); - HUF_DECODE_SYMBOLX2_1(op3, &bitD3); - HUF_DECODE_SYMBOLX2_2(op3, &bitD3); - HUF_DECODE_SYMBOLX2_0(op3, &bitD3); - HUF_DECODE_SYMBOLX2_2(op4, &bitD4); - HUF_DECODE_SYMBOLX2_1(op4, &bitD4); - HUF_DECODE_SYMBOLX2_2(op4, &bitD4); - HUF_DECODE_SYMBOLX2_0(op4, &bitD4); - endSignal &= BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished; - endSignal &= BIT_reloadDStreamFast(&bitD4) == BIT_DStream_unfinished; -#else - HUF_DECODE_SYMBOLX2_2(op1, &bitD1); - HUF_DECODE_SYMBOLX2_2(op2, &bitD2); - HUF_DECODE_SYMBOLX2_2(op3, &bitD3); - HUF_DECODE_SYMBOLX2_2(op4, &bitD4); - HUF_DECODE_SYMBOLX2_1(op1, &bitD1); - HUF_DECODE_SYMBOLX2_1(op2, &bitD2); - HUF_DECODE_SYMBOLX2_1(op3, &bitD3); - HUF_DECODE_SYMBOLX2_1(op4, &bitD4); - HUF_DECODE_SYMBOLX2_2(op1, &bitD1); - HUF_DECODE_SYMBOLX2_2(op2, &bitD2); - HUF_DECODE_SYMBOLX2_2(op3, &bitD3); - HUF_DECODE_SYMBOLX2_2(op4, &bitD4); - HUF_DECODE_SYMBOLX2_0(op1, &bitD1); - HUF_DECODE_SYMBOLX2_0(op2, &bitD2); - HUF_DECODE_SYMBOLX2_0(op3, &bitD3); - HUF_DECODE_SYMBOLX2_0(op4, &bitD4); - endSignal = (U32)LIKELY( - (BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished) - & (BIT_reloadDStreamFast(&bitD2) == BIT_DStream_unfinished) - & (BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished) - & (BIT_reloadDStreamFast(&bitD4) == BIT_DStream_unfinished)); -#endif - } - - /* check corruption */ - if (op1 > opStart2) return ERROR(corruption_detected); - if (op2 > opStart3) return ERROR(corruption_detected); - if (op3 > opStart4) return ERROR(corruption_detected); - /* note : op4 already verified within main loop */ - - /* finish bitStreams one by one */ - HUF_decodeStreamX2(op1, &bitD1, opStart2, dt, dtLog); - HUF_decodeStreamX2(op2, &bitD2, opStart3, dt, dtLog); - HUF_decodeStreamX2(op3, &bitD3, opStart4, dt, dtLog); - HUF_decodeStreamX2(op4, &bitD4, oend, dt, dtLog); - - /* check */ - { U32 const endCheck = BIT_endOfDStream(&bitD1) & BIT_endOfDStream(&bitD2) & BIT_endOfDStream(&bitD3) & BIT_endOfDStream(&bitD4); - if (!endCheck) return ERROR(corruption_detected); } - - /* decoded size */ - return dstSize; - } -} - -HUF_DGEN(HUF_decompress1X2_usingDTable_internal) -HUF_DGEN(HUF_decompress4X2_usingDTable_internal) - -size_t HUF_decompress1X2_usingDTable( - void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize, - const HUF_DTable* DTable) -{ - DTableDesc dtd = HUF_getDTableDesc(DTable); - if (dtd.tableType != 1) return ERROR(GENERIC); - return HUF_decompress1X2_usingDTable_internal(dst, dstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); -} - -size_t HUF_decompress1X2_DCtx_wksp(HUF_DTable* DCtx, void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize, - void* workSpace, size_t wkspSize) -{ - const BYTE* ip = (const BYTE*) cSrc; - - size_t const hSize = HUF_readDTableX2_wksp(DCtx, cSrc, cSrcSize, - workSpace, wkspSize); - if (HUF_isError(hSize)) return hSize; - if (hSize >= cSrcSize) return ERROR(srcSize_wrong); - ip += hSize; cSrcSize -= hSize; - - return HUF_decompress1X2_usingDTable_internal(dst, dstSize, ip, cSrcSize, DCtx, /* bmi2 */ 0); -} - - -size_t HUF_decompress1X2_DCtx(HUF_DTable* DCtx, void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize) -{ - U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; - return HUF_decompress1X2_DCtx_wksp(DCtx, dst, dstSize, cSrc, cSrcSize, - workSpace, sizeof(workSpace)); -} - -size_t HUF_decompress1X2 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) -{ - HUF_CREATE_STATIC_DTABLEX2(DTable, HUF_TABLELOG_MAX); - return HUF_decompress1X2_DCtx(DTable, dst, dstSize, cSrc, cSrcSize); -} - -size_t HUF_decompress4X2_usingDTable( - void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize, - const HUF_DTable* DTable) -{ - DTableDesc dtd = HUF_getDTableDesc(DTable); - if (dtd.tableType != 1) return ERROR(GENERIC); - return HUF_decompress4X2_usingDTable_internal(dst, dstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); -} - -static size_t HUF_decompress4X2_DCtx_wksp_bmi2(HUF_DTable* dctx, void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize, - void* workSpace, size_t wkspSize, int bmi2) -{ - const BYTE* ip = (const BYTE*) cSrc; - - size_t hSize = HUF_readDTableX2_wksp(dctx, cSrc, cSrcSize, - workSpace, wkspSize); - if (HUF_isError(hSize)) return hSize; - if (hSize >= cSrcSize) return ERROR(srcSize_wrong); - ip += hSize; cSrcSize -= hSize; - - return HUF_decompress4X2_usingDTable_internal(dst, dstSize, ip, cSrcSize, dctx, bmi2); -} - -size_t HUF_decompress4X2_DCtx_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize, - void* workSpace, size_t wkspSize) -{ - return HUF_decompress4X2_DCtx_wksp_bmi2(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize, /* bmi2 */ 0); -} - - -size_t HUF_decompress4X2_DCtx(HUF_DTable* dctx, void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize) -{ - U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; - return HUF_decompress4X2_DCtx_wksp(dctx, dst, dstSize, cSrc, cSrcSize, - workSpace, sizeof(workSpace)); -} - -size_t HUF_decompress4X2 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) -{ - HUF_CREATE_STATIC_DTABLEX2(DTable, HUF_TABLELOG_MAX); - return HUF_decompress4X2_DCtx(DTable, dst, dstSize, cSrc, cSrcSize); -} - -#endif /* HUF_FORCE_DECOMPRESS_X1 */ - - -/* ***********************************/ -/* Universal decompression selectors */ -/* ***********************************/ - -size_t HUF_decompress1X_usingDTable(void* dst, size_t maxDstSize, - const void* cSrc, size_t cSrcSize, - const HUF_DTable* DTable) -{ - DTableDesc const dtd = HUF_getDTableDesc(DTable); -#if defined(HUF_FORCE_DECOMPRESS_X1) - (void)dtd; - assert(dtd.tableType == 0); - return HUF_decompress1X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); -#elif defined(HUF_FORCE_DECOMPRESS_X2) - (void)dtd; - assert(dtd.tableType == 1); - return HUF_decompress1X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); -#else - return dtd.tableType ? HUF_decompress1X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0) : - HUF_decompress1X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); -#endif -} - -size_t HUF_decompress4X_usingDTable(void* dst, size_t maxDstSize, - const void* cSrc, size_t cSrcSize, - const HUF_DTable* DTable) -{ - DTableDesc const dtd = HUF_getDTableDesc(DTable); -#if defined(HUF_FORCE_DECOMPRESS_X1) - (void)dtd; - assert(dtd.tableType == 0); - return HUF_decompress4X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); -#elif defined(HUF_FORCE_DECOMPRESS_X2) - (void)dtd; - assert(dtd.tableType == 1); - return HUF_decompress4X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); -#else - return dtd.tableType ? HUF_decompress4X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0) : - HUF_decompress4X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, /* bmi2 */ 0); -#endif -} - - -#if !defined(HUF_FORCE_DECOMPRESS_X1) && !defined(HUF_FORCE_DECOMPRESS_X2) -typedef struct { U32 tableTime; U32 decode256Time; } algo_time_t; -static const algo_time_t algoTime[16 /* Quantization */][3 /* single, double, quad */] = -{ - /* single, double, quad */ - {{0,0}, {1,1}, {2,2}}, /* Q==0 : impossible */ - {{0,0}, {1,1}, {2,2}}, /* Q==1 : impossible */ - {{ 38,130}, {1313, 74}, {2151, 38}}, /* Q == 2 : 12-18% */ - {{ 448,128}, {1353, 74}, {2238, 41}}, /* Q == 3 : 18-25% */ - {{ 556,128}, {1353, 74}, {2238, 47}}, /* Q == 4 : 25-32% */ - {{ 714,128}, {1418, 74}, {2436, 53}}, /* Q == 5 : 32-38% */ - {{ 883,128}, {1437, 74}, {2464, 61}}, /* Q == 6 : 38-44% */ - {{ 897,128}, {1515, 75}, {2622, 68}}, /* Q == 7 : 44-50% */ - {{ 926,128}, {1613, 75}, {2730, 75}}, /* Q == 8 : 50-56% */ - {{ 947,128}, {1729, 77}, {3359, 77}}, /* Q == 9 : 56-62% */ - {{1107,128}, {2083, 81}, {4006, 84}}, /* Q ==10 : 62-69% */ - {{1177,128}, {2379, 87}, {4785, 88}}, /* Q ==11 : 69-75% */ - {{1242,128}, {2415, 93}, {5155, 84}}, /* Q ==12 : 75-81% */ - {{1349,128}, {2644,106}, {5260,106}}, /* Q ==13 : 81-87% */ - {{1455,128}, {2422,124}, {4174,124}}, /* Q ==14 : 87-93% */ - {{ 722,128}, {1891,145}, {1936,146}}, /* Q ==15 : 93-99% */ -}; -#endif - -/** HUF_selectDecoder() : - * Tells which decoder is likely to decode faster, - * based on a set of pre-computed metrics. - * @return : 0==HUF_decompress4X1, 1==HUF_decompress4X2 . - * Assumption : 0 < dstSize <= 128 KB */ -U32 HUF_selectDecoder (size_t dstSize, size_t cSrcSize) -{ - assert(dstSize > 0); - assert(dstSize <= 128*1024); -#if defined(HUF_FORCE_DECOMPRESS_X1) - (void)dstSize; - (void)cSrcSize; - return 0; -#elif defined(HUF_FORCE_DECOMPRESS_X2) - (void)dstSize; - (void)cSrcSize; - return 1; -#else - /* decoder timing evaluation */ - { U32 const Q = (cSrcSize >= dstSize) ? 15 : (U32)(cSrcSize * 16 / dstSize); /* Q < 16 */ - U32 const D256 = (U32)(dstSize >> 8); - U32 const DTime0 = algoTime[Q][0].tableTime + (algoTime[Q][0].decode256Time * D256); - U32 DTime1 = algoTime[Q][1].tableTime + (algoTime[Q][1].decode256Time * D256); - DTime1 += DTime1 >> 3; /* advantage to algorithm using less memory, to reduce cache eviction */ - return DTime1 < DTime0; - } -#endif -} - - -typedef size_t (*decompressionAlgo)(void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); - -size_t HUF_decompress (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) -{ -#if !defined(HUF_FORCE_DECOMPRESS_X1) && !defined(HUF_FORCE_DECOMPRESS_X2) - static const decompressionAlgo decompress[2] = { HUF_decompress4X1, HUF_decompress4X2 }; -#endif - - /* validation checks */ - if (dstSize == 0) return ERROR(dstSize_tooSmall); - if (cSrcSize > dstSize) return ERROR(corruption_detected); /* invalid */ - if (cSrcSize == dstSize) { memcpy(dst, cSrc, dstSize); return dstSize; } /* not compressed */ - if (cSrcSize == 1) { memset(dst, *(const BYTE*)cSrc, dstSize); return dstSize; } /* RLE */ - - { U32 const algoNb = HUF_selectDecoder(dstSize, cSrcSize); -#if defined(HUF_FORCE_DECOMPRESS_X1) - (void)algoNb; - assert(algoNb == 0); - return HUF_decompress4X1(dst, dstSize, cSrc, cSrcSize); -#elif defined(HUF_FORCE_DECOMPRESS_X2) - (void)algoNb; - assert(algoNb == 1); - return HUF_decompress4X2(dst, dstSize, cSrc, cSrcSize); -#else - return decompress[algoNb](dst, dstSize, cSrc, cSrcSize); -#endif - } -} - -size_t HUF_decompress4X_DCtx (HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) -{ - /* validation checks */ - if (dstSize == 0) return ERROR(dstSize_tooSmall); - if (cSrcSize > dstSize) return ERROR(corruption_detected); /* invalid */ - if (cSrcSize == dstSize) { memcpy(dst, cSrc, dstSize); return dstSize; } /* not compressed */ - if (cSrcSize == 1) { memset(dst, *(const BYTE*)cSrc, dstSize); return dstSize; } /* RLE */ - - { U32 const algoNb = HUF_selectDecoder(dstSize, cSrcSize); -#if defined(HUF_FORCE_DECOMPRESS_X1) - (void)algoNb; - assert(algoNb == 0); - return HUF_decompress4X1_DCtx(dctx, dst, dstSize, cSrc, cSrcSize); -#elif defined(HUF_FORCE_DECOMPRESS_X2) - (void)algoNb; - assert(algoNb == 1); - return HUF_decompress4X2_DCtx(dctx, dst, dstSize, cSrc, cSrcSize); -#else - return algoNb ? HUF_decompress4X2_DCtx(dctx, dst, dstSize, cSrc, cSrcSize) : - HUF_decompress4X1_DCtx(dctx, dst, dstSize, cSrc, cSrcSize) ; -#endif - } -} - -size_t HUF_decompress4X_hufOnly(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize) -{ - U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; - return HUF_decompress4X_hufOnly_wksp(dctx, dst, dstSize, cSrc, cSrcSize, - workSpace, sizeof(workSpace)); -} - - -size_t HUF_decompress4X_hufOnly_wksp(HUF_DTable* dctx, void* dst, - size_t dstSize, const void* cSrc, - size_t cSrcSize, void* workSpace, - size_t wkspSize) -{ - /* validation checks */ - if (dstSize == 0) return ERROR(dstSize_tooSmall); - if (cSrcSize == 0) return ERROR(corruption_detected); - - { U32 const algoNb = HUF_selectDecoder(dstSize, cSrcSize); -#if defined(HUF_FORCE_DECOMPRESS_X1) - (void)algoNb; - assert(algoNb == 0); - return HUF_decompress4X1_DCtx_wksp(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize); -#elif defined(HUF_FORCE_DECOMPRESS_X2) - (void)algoNb; - assert(algoNb == 1); - return HUF_decompress4X2_DCtx_wksp(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize); -#else - return algoNb ? HUF_decompress4X2_DCtx_wksp(dctx, dst, dstSize, cSrc, - cSrcSize, workSpace, wkspSize): - HUF_decompress4X1_DCtx_wksp(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize); -#endif - } -} - -size_t HUF_decompress1X_DCtx_wksp(HUF_DTable* dctx, void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize, - void* workSpace, size_t wkspSize) -{ - /* validation checks */ - if (dstSize == 0) return ERROR(dstSize_tooSmall); - if (cSrcSize > dstSize) return ERROR(corruption_detected); /* invalid */ - if (cSrcSize == dstSize) { memcpy(dst, cSrc, dstSize); return dstSize; } /* not compressed */ - if (cSrcSize == 1) { memset(dst, *(const BYTE*)cSrc, dstSize); return dstSize; } /* RLE */ - - { U32 const algoNb = HUF_selectDecoder(dstSize, cSrcSize); -#if defined(HUF_FORCE_DECOMPRESS_X1) - (void)algoNb; - assert(algoNb == 0); - return HUF_decompress1X1_DCtx_wksp(dctx, dst, dstSize, cSrc, - cSrcSize, workSpace, wkspSize); -#elif defined(HUF_FORCE_DECOMPRESS_X2) - (void)algoNb; - assert(algoNb == 1); - return HUF_decompress1X2_DCtx_wksp(dctx, dst, dstSize, cSrc, - cSrcSize, workSpace, wkspSize); -#else - return algoNb ? HUF_decompress1X2_DCtx_wksp(dctx, dst, dstSize, cSrc, - cSrcSize, workSpace, wkspSize): - HUF_decompress1X1_DCtx_wksp(dctx, dst, dstSize, cSrc, - cSrcSize, workSpace, wkspSize); -#endif - } -} - -size_t HUF_decompress1X_DCtx(HUF_DTable* dctx, void* dst, size_t dstSize, - const void* cSrc, size_t cSrcSize) -{ - U32 workSpace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; - return HUF_decompress1X_DCtx_wksp(dctx, dst, dstSize, cSrc, cSrcSize, - workSpace, sizeof(workSpace)); -} - - -size_t HUF_decompress1X_usingDTable_bmi2(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable, int bmi2) -{ - DTableDesc const dtd = HUF_getDTableDesc(DTable); -#if defined(HUF_FORCE_DECOMPRESS_X1) - (void)dtd; - assert(dtd.tableType == 0); - return HUF_decompress1X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2); -#elif defined(HUF_FORCE_DECOMPRESS_X2) - (void)dtd; - assert(dtd.tableType == 1); - return HUF_decompress1X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2); -#else - return dtd.tableType ? HUF_decompress1X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2) : - HUF_decompress1X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2); -#endif -} - -#ifndef HUF_FORCE_DECOMPRESS_X2 -size_t HUF_decompress1X1_DCtx_wksp_bmi2(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize, int bmi2) -{ - const BYTE* ip = (const BYTE*) cSrc; - - size_t const hSize = HUF_readDTableX1_wksp(dctx, cSrc, cSrcSize, workSpace, wkspSize); - if (HUF_isError(hSize)) return hSize; - if (hSize >= cSrcSize) return ERROR(srcSize_wrong); - ip += hSize; cSrcSize -= hSize; - - return HUF_decompress1X1_usingDTable_internal(dst, dstSize, ip, cSrcSize, dctx, bmi2); -} -#endif - -size_t HUF_decompress4X_usingDTable_bmi2(void* dst, size_t maxDstSize, const void* cSrc, size_t cSrcSize, const HUF_DTable* DTable, int bmi2) -{ - DTableDesc const dtd = HUF_getDTableDesc(DTable); -#if defined(HUF_FORCE_DECOMPRESS_X1) - (void)dtd; - assert(dtd.tableType == 0); - return HUF_decompress4X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2); -#elif defined(HUF_FORCE_DECOMPRESS_X2) - (void)dtd; - assert(dtd.tableType == 1); - return HUF_decompress4X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2); -#else - return dtd.tableType ? HUF_decompress4X2_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2) : - HUF_decompress4X1_usingDTable_internal(dst, maxDstSize, cSrc, cSrcSize, DTable, bmi2); -#endif -} - -size_t HUF_decompress4X_hufOnly_wksp_bmi2(HUF_DTable* dctx, void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize, void* workSpace, size_t wkspSize, int bmi2) -{ - /* validation checks */ - if (dstSize == 0) return ERROR(dstSize_tooSmall); - if (cSrcSize == 0) return ERROR(corruption_detected); - - { U32 const algoNb = HUF_selectDecoder(dstSize, cSrcSize); -#if defined(HUF_FORCE_DECOMPRESS_X1) - (void)algoNb; - assert(algoNb == 0); - return HUF_decompress4X1_DCtx_wksp_bmi2(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize, bmi2); -#elif defined(HUF_FORCE_DECOMPRESS_X2) - (void)algoNb; - assert(algoNb == 1); - return HUF_decompress4X2_DCtx_wksp_bmi2(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize, bmi2); -#else - return algoNb ? HUF_decompress4X2_DCtx_wksp_bmi2(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize, bmi2) : - HUF_decompress4X1_DCtx_wksp_bmi2(dctx, dst, dstSize, cSrc, cSrcSize, workSpace, wkspSize, bmi2); -#endif - } -} -/**** ended inlining decompress/huf_decompress.c ****/ -/**** start inlining decompress/zstd_ddict.c ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -/* zstd_ddict.c : - * concentrates all logic that needs to know the internals of ZSTD_DDict object */ - -/*-******************************************************* -* Dependencies -*********************************************************/ -#include /* memcpy, memmove, memset */ -/**** skipping file: ../common/cpu.h ****/ -/**** skipping file: ../common/mem.h ****/ -#define FSE_STATIC_LINKING_ONLY -/**** skipping file: ../common/fse.h ****/ -#define HUF_STATIC_LINKING_ONLY -/**** skipping file: ../common/huf.h ****/ -/**** start inlining zstd_decompress_internal.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - - -/* zstd_decompress_internal: - * objects and definitions shared within lib/decompress modules */ - - #ifndef ZSTD_DECOMPRESS_INTERNAL_H - #define ZSTD_DECOMPRESS_INTERNAL_H - - -/*-******************************************************* - * Dependencies - *********************************************************/ -/**** skipping file: ../common/mem.h ****/ -/**** skipping file: ../common/zstd_internal.h ****/ - - - -/*-******************************************************* - * Constants - *********************************************************/ -static const U32 LL_base[MaxLL+1] = { - 0, 1, 2, 3, 4, 5, 6, 7, - 8, 9, 10, 11, 12, 13, 14, 15, - 16, 18, 20, 22, 24, 28, 32, 40, - 48, 64, 0x80, 0x100, 0x200, 0x400, 0x800, 0x1000, - 0x2000, 0x4000, 0x8000, 0x10000 }; - -static const U32 OF_base[MaxOff+1] = { - 0, 1, 1, 5, 0xD, 0x1D, 0x3D, 0x7D, - 0xFD, 0x1FD, 0x3FD, 0x7FD, 0xFFD, 0x1FFD, 0x3FFD, 0x7FFD, - 0xFFFD, 0x1FFFD, 0x3FFFD, 0x7FFFD, 0xFFFFD, 0x1FFFFD, 0x3FFFFD, 0x7FFFFD, - 0xFFFFFD, 0x1FFFFFD, 0x3FFFFFD, 0x7FFFFFD, 0xFFFFFFD, 0x1FFFFFFD, 0x3FFFFFFD, 0x7FFFFFFD }; - -static const U32 OF_bits[MaxOff+1] = { - 0, 1, 2, 3, 4, 5, 6, 7, - 8, 9, 10, 11, 12, 13, 14, 15, - 16, 17, 18, 19, 20, 21, 22, 23, - 24, 25, 26, 27, 28, 29, 30, 31 }; - -static const U32 ML_base[MaxML+1] = { - 3, 4, 5, 6, 7, 8, 9, 10, - 11, 12, 13, 14, 15, 16, 17, 18, - 19, 20, 21, 22, 23, 24, 25, 26, - 27, 28, 29, 30, 31, 32, 33, 34, - 35, 37, 39, 41, 43, 47, 51, 59, - 67, 83, 99, 0x83, 0x103, 0x203, 0x403, 0x803, - 0x1003, 0x2003, 0x4003, 0x8003, 0x10003 }; - - -/*-******************************************************* - * Decompression types - *********************************************************/ - typedef struct { - U32 fastMode; - U32 tableLog; - } ZSTD_seqSymbol_header; - - typedef struct { - U16 nextState; - BYTE nbAdditionalBits; - BYTE nbBits; - U32 baseValue; - } ZSTD_seqSymbol; - - #define SEQSYMBOL_TABLE_SIZE(log) (1 + (1 << (log))) - -typedef struct { - ZSTD_seqSymbol LLTable[SEQSYMBOL_TABLE_SIZE(LLFSELog)]; /* Note : Space reserved for FSE Tables */ - ZSTD_seqSymbol OFTable[SEQSYMBOL_TABLE_SIZE(OffFSELog)]; /* is also used as temporary workspace while building hufTable during DDict creation */ - ZSTD_seqSymbol MLTable[SEQSYMBOL_TABLE_SIZE(MLFSELog)]; /* and therefore must be at least HUF_DECOMPRESS_WORKSPACE_SIZE large */ - HUF_DTable hufTable[HUF_DTABLE_SIZE(HufLog)]; /* can accommodate HUF_decompress4X */ - U32 rep[ZSTD_REP_NUM]; -} ZSTD_entropyDTables_t; - -typedef enum { ZSTDds_getFrameHeaderSize, ZSTDds_decodeFrameHeader, - ZSTDds_decodeBlockHeader, ZSTDds_decompressBlock, - ZSTDds_decompressLastBlock, ZSTDds_checkChecksum, - ZSTDds_decodeSkippableHeader, ZSTDds_skipFrame } ZSTD_dStage; - -typedef enum { zdss_init=0, zdss_loadHeader, - zdss_read, zdss_load, zdss_flush } ZSTD_dStreamStage; - -typedef enum { - ZSTD_use_indefinitely = -1, /* Use the dictionary indefinitely */ - ZSTD_dont_use = 0, /* Do not use the dictionary (if one exists free it) */ - ZSTD_use_once = 1 /* Use the dictionary once and set to ZSTD_dont_use */ -} ZSTD_dictUses_e; - -typedef enum { - ZSTD_obm_buffered = 0, /* Buffer the output */ - ZSTD_obm_stable = 1 /* ZSTD_outBuffer is stable */ -} ZSTD_outBufferMode_e; - -struct ZSTD_DCtx_s -{ - const ZSTD_seqSymbol* LLTptr; - const ZSTD_seqSymbol* MLTptr; - const ZSTD_seqSymbol* OFTptr; - const HUF_DTable* HUFptr; - ZSTD_entropyDTables_t entropy; - U32 workspace[HUF_DECOMPRESS_WORKSPACE_SIZE_U32]; /* space needed when building huffman tables */ - const void* previousDstEnd; /* detect continuity */ - const void* prefixStart; /* start of current segment */ - const void* virtualStart; /* virtual start of previous segment if it was just before current one */ - const void* dictEnd; /* end of previous segment */ - size_t expected; - ZSTD_frameHeader fParams; - U64 decodedSize; - blockType_e bType; /* used in ZSTD_decompressContinue(), store blockType between block header decoding and block decompression stages */ - ZSTD_dStage stage; - U32 litEntropy; - U32 fseEntropy; - XXH64_state_t xxhState; - size_t headerSize; - ZSTD_format_e format; - const BYTE* litPtr; - ZSTD_customMem customMem; - size_t litSize; - size_t rleSize; - size_t staticSize; - int bmi2; /* == 1 if the CPU supports BMI2 and 0 otherwise. CPU support is determined dynamically once per context lifetime. */ - - /* dictionary */ - ZSTD_DDict* ddictLocal; - const ZSTD_DDict* ddict; /* set by ZSTD_initDStream_usingDDict(), or ZSTD_DCtx_refDDict() */ - U32 dictID; - int ddictIsCold; /* if == 1 : dictionary is "new" for working context, and presumed "cold" (not in cpu cache) */ - ZSTD_dictUses_e dictUses; - - /* streaming */ - ZSTD_dStreamStage streamStage; - char* inBuff; - size_t inBuffSize; - size_t inPos; - size_t maxWindowSize; - char* outBuff; - size_t outBuffSize; - size_t outStart; - size_t outEnd; - size_t lhSize; - void* legacyContext; - U32 previousLegacyVersion; - U32 legacyVersion; - U32 hostageByte; - int noForwardProgress; - ZSTD_outBufferMode_e outBufferMode; - ZSTD_outBuffer expectedOutBuffer; - - /* workspace */ - BYTE litBuffer[ZSTD_BLOCKSIZE_MAX + WILDCOPY_OVERLENGTH]; - BYTE headerBuffer[ZSTD_FRAMEHEADERSIZE_MAX]; - - size_t oversizedDuration; - -#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION - void const* dictContentBeginForFuzzing; - void const* dictContentEndForFuzzing; -#endif -}; /* typedef'd to ZSTD_DCtx within "zstd.h" */ - - -/*-******************************************************* - * Shared internal functions - *********************************************************/ - -/*! ZSTD_loadDEntropy() : - * dict : must point at beginning of a valid zstd dictionary. - * @return : size of dictionary header (size of magic number + dict ID + entropy tables) */ -size_t ZSTD_loadDEntropy(ZSTD_entropyDTables_t* entropy, - const void* const dict, size_t const dictSize); - -/*! ZSTD_checkContinuity() : - * check if next `dst` follows previous position, where decompression ended. - * If yes, do nothing (continue on current segment). - * If not, classify previous segment as "external dictionary", and start a new segment. - * This function cannot fail. */ -void ZSTD_checkContinuity(ZSTD_DCtx* dctx, const void* dst); - - -#endif /* ZSTD_DECOMPRESS_INTERNAL_H */ -/**** ended inlining zstd_decompress_internal.h ****/ -/**** start inlining zstd_ddict.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - - -#ifndef ZSTD_DDICT_H -#define ZSTD_DDICT_H - -/*-******************************************************* - * Dependencies - *********************************************************/ -#include /* size_t */ -/**** skipping file: ../zstd.h ****/ - - -/*-******************************************************* - * Interface - *********************************************************/ - -/* note: several prototypes are already published in `zstd.h` : - * ZSTD_createDDict() - * ZSTD_createDDict_byReference() - * ZSTD_createDDict_advanced() - * ZSTD_freeDDict() - * ZSTD_initStaticDDict() - * ZSTD_sizeof_DDict() - * ZSTD_estimateDDictSize() - * ZSTD_getDictID_fromDict() - */ - -const void* ZSTD_DDict_dictContent(const ZSTD_DDict* ddict); -size_t ZSTD_DDict_dictSize(const ZSTD_DDict* ddict); - -void ZSTD_copyDDictParameters(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict); - - - -#endif /* ZSTD_DDICT_H */ -/**** ended inlining zstd_ddict.h ****/ - -#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT>=1) -/**** start inlining ../legacy/zstd_legacy.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_LEGACY_H -#define ZSTD_LEGACY_H - -#if defined (__cplusplus) -extern "C" { -#endif - -/* ************************************* -* Includes -***************************************/ -/**** skipping file: ../common/mem.h ****/ -/**** skipping file: ../common/error_private.h ****/ -/**** skipping file: ../common/zstd_internal.h ****/ - -#if !defined (ZSTD_LEGACY_SUPPORT) || (ZSTD_LEGACY_SUPPORT == 0) -# undef ZSTD_LEGACY_SUPPORT -# define ZSTD_LEGACY_SUPPORT 8 -#endif - -#if (ZSTD_LEGACY_SUPPORT <= 1) -/**** start inlining zstd_v01.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_V01_H_28739879432 -#define ZSTD_V01_H_28739879432 - -#if defined (__cplusplus) -extern "C" { -#endif - -/* ************************************* -* Includes -***************************************/ -#include /* size_t */ - - -/* ************************************* -* Simple one-step function -***************************************/ -/** -ZSTDv01_decompress() : decompress ZSTD frames compliant with v0.1.x format - compressedSize : is the exact source size - maxOriginalSize : is the size of the 'dst' buffer, which must be already allocated. - It must be equal or larger than originalSize, otherwise decompression will fail. - return : the number of bytes decompressed into destination buffer (originalSize) - or an errorCode if it fails (which can be tested using ZSTDv01_isError()) -*/ -size_t ZSTDv01_decompress( void* dst, size_t maxOriginalSize, - const void* src, size_t compressedSize); - - /** - ZSTDv01_findFrameSizeInfoLegacy() : get the source length and decompressed bound of a ZSTD frame compliant with v0.1.x format - srcSize : The size of the 'src' buffer, at least as large as the frame pointed to by 'src' - cSize (output parameter) : the number of bytes that would be read to decompress this frame - or an error code if it fails (which can be tested using ZSTDv01_isError()) - dBound (output parameter) : an upper-bound for the decompressed size of the data in the frame - or ZSTD_CONTENTSIZE_ERROR if an error occurs - - note : assumes `cSize` and `dBound` are _not_ NULL. - */ -void ZSTDv01_findFrameSizeInfoLegacy(const void *src, size_t srcSize, - size_t* cSize, unsigned long long* dBound); - -/** -ZSTDv01_isError() : tells if the result of ZSTDv01_decompress() is an error -*/ -unsigned ZSTDv01_isError(size_t code); - - -/* ************************************* -* Advanced functions -***************************************/ -typedef struct ZSTDv01_Dctx_s ZSTDv01_Dctx; -ZSTDv01_Dctx* ZSTDv01_createDCtx(void); -size_t ZSTDv01_freeDCtx(ZSTDv01_Dctx* dctx); - -size_t ZSTDv01_decompressDCtx(void* ctx, - void* dst, size_t maxOriginalSize, - const void* src, size_t compressedSize); - -/* ************************************* -* Streaming functions -***************************************/ -size_t ZSTDv01_resetDCtx(ZSTDv01_Dctx* dctx); - -size_t ZSTDv01_nextSrcSizeToDecompress(ZSTDv01_Dctx* dctx); -size_t ZSTDv01_decompressContinue(ZSTDv01_Dctx* dctx, void* dst, size_t maxDstSize, const void* src, size_t srcSize); -/** - Use above functions alternatively. - ZSTD_nextSrcSizeToDecompress() tells how much bytes to provide as 'srcSize' to ZSTD_decompressContinue(). - ZSTD_decompressContinue() will use previous data blocks to improve compression if they are located prior to current block. - Result is the number of bytes regenerated within 'dst'. - It can be zero, which is not an error; it just means ZSTD_decompressContinue() has decoded some header. -*/ - -/* ************************************* -* Prefix - version detection -***************************************/ -#define ZSTDv01_magicNumber 0xFD2FB51E /* Big Endian version */ -#define ZSTDv01_magicNumberLE 0x1EB52FFD /* Little Endian version */ - - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTD_V01_H_28739879432 */ -/**** ended inlining zstd_v01.h ****/ -#endif -#if (ZSTD_LEGACY_SUPPORT <= 2) -/**** start inlining zstd_v02.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_V02_H_4174539423 -#define ZSTD_V02_H_4174539423 - -#if defined (__cplusplus) -extern "C" { -#endif - -/* ************************************* -* Includes -***************************************/ -#include /* size_t */ - - -/* ************************************* -* Simple one-step function -***************************************/ -/** -ZSTDv02_decompress() : decompress ZSTD frames compliant with v0.2.x format - compressedSize : is the exact source size - maxOriginalSize : is the size of the 'dst' buffer, which must be already allocated. - It must be equal or larger than originalSize, otherwise decompression will fail. - return : the number of bytes decompressed into destination buffer (originalSize) - or an errorCode if it fails (which can be tested using ZSTDv01_isError()) -*/ -size_t ZSTDv02_decompress( void* dst, size_t maxOriginalSize, - const void* src, size_t compressedSize); - - /** - ZSTDv02_findFrameSizeInfoLegacy() : get the source length and decompressed bound of a ZSTD frame compliant with v0.2.x format - srcSize : The size of the 'src' buffer, at least as large as the frame pointed to by 'src' - cSize (output parameter) : the number of bytes that would be read to decompress this frame - or an error code if it fails (which can be tested using ZSTDv01_isError()) - dBound (output parameter) : an upper-bound for the decompressed size of the data in the frame - or ZSTD_CONTENTSIZE_ERROR if an error occurs - - note : assumes `cSize` and `dBound` are _not_ NULL. - */ -void ZSTDv02_findFrameSizeInfoLegacy(const void *src, size_t srcSize, - size_t* cSize, unsigned long long* dBound); - -/** -ZSTDv02_isError() : tells if the result of ZSTDv02_decompress() is an error -*/ -unsigned ZSTDv02_isError(size_t code); - - -/* ************************************* -* Advanced functions -***************************************/ -typedef struct ZSTDv02_Dctx_s ZSTDv02_Dctx; -ZSTDv02_Dctx* ZSTDv02_createDCtx(void); -size_t ZSTDv02_freeDCtx(ZSTDv02_Dctx* dctx); - -size_t ZSTDv02_decompressDCtx(void* ctx, - void* dst, size_t maxOriginalSize, - const void* src, size_t compressedSize); - -/* ************************************* -* Streaming functions -***************************************/ -size_t ZSTDv02_resetDCtx(ZSTDv02_Dctx* dctx); - -size_t ZSTDv02_nextSrcSizeToDecompress(ZSTDv02_Dctx* dctx); -size_t ZSTDv02_decompressContinue(ZSTDv02_Dctx* dctx, void* dst, size_t maxDstSize, const void* src, size_t srcSize); -/** - Use above functions alternatively. - ZSTD_nextSrcSizeToDecompress() tells how much bytes to provide as 'srcSize' to ZSTD_decompressContinue(). - ZSTD_decompressContinue() will use previous data blocks to improve compression if they are located prior to current block. - Result is the number of bytes regenerated within 'dst'. - It can be zero, which is not an error; it just means ZSTD_decompressContinue() has decoded some header. -*/ - -/* ************************************* -* Prefix - version detection -***************************************/ -#define ZSTDv02_magicNumber 0xFD2FB522 /* v0.2 */ - - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTD_V02_H_4174539423 */ -/**** ended inlining zstd_v02.h ****/ -#endif -#if (ZSTD_LEGACY_SUPPORT <= 3) -/**** start inlining zstd_v03.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_V03_H_298734209782 -#define ZSTD_V03_H_298734209782 - -#if defined (__cplusplus) -extern "C" { -#endif - -/* ************************************* -* Includes -***************************************/ -#include /* size_t */ - - -/* ************************************* -* Simple one-step function -***************************************/ -/** -ZSTDv03_decompress() : decompress ZSTD frames compliant with v0.3.x format - compressedSize : is the exact source size - maxOriginalSize : is the size of the 'dst' buffer, which must be already allocated. - It must be equal or larger than originalSize, otherwise decompression will fail. - return : the number of bytes decompressed into destination buffer (originalSize) - or an errorCode if it fails (which can be tested using ZSTDv01_isError()) -*/ -size_t ZSTDv03_decompress( void* dst, size_t maxOriginalSize, - const void* src, size_t compressedSize); - - /** - ZSTDv03_findFrameSizeInfoLegacy() : get the source length and decompressed bound of a ZSTD frame compliant with v0.3.x format - srcSize : The size of the 'src' buffer, at least as large as the frame pointed to by 'src' - cSize (output parameter) : the number of bytes that would be read to decompress this frame - or an error code if it fails (which can be tested using ZSTDv01_isError()) - dBound (output parameter) : an upper-bound for the decompressed size of the data in the frame - or ZSTD_CONTENTSIZE_ERROR if an error occurs - - note : assumes `cSize` and `dBound` are _not_ NULL. - */ - void ZSTDv03_findFrameSizeInfoLegacy(const void *src, size_t srcSize, - size_t* cSize, unsigned long long* dBound); - - /** -ZSTDv03_isError() : tells if the result of ZSTDv03_decompress() is an error -*/ -unsigned ZSTDv03_isError(size_t code); - - -/* ************************************* -* Advanced functions -***************************************/ -typedef struct ZSTDv03_Dctx_s ZSTDv03_Dctx; -ZSTDv03_Dctx* ZSTDv03_createDCtx(void); -size_t ZSTDv03_freeDCtx(ZSTDv03_Dctx* dctx); - -size_t ZSTDv03_decompressDCtx(void* ctx, - void* dst, size_t maxOriginalSize, - const void* src, size_t compressedSize); - -/* ************************************* -* Streaming functions -***************************************/ -size_t ZSTDv03_resetDCtx(ZSTDv03_Dctx* dctx); - -size_t ZSTDv03_nextSrcSizeToDecompress(ZSTDv03_Dctx* dctx); -size_t ZSTDv03_decompressContinue(ZSTDv03_Dctx* dctx, void* dst, size_t maxDstSize, const void* src, size_t srcSize); -/** - Use above functions alternatively. - ZSTD_nextSrcSizeToDecompress() tells how much bytes to provide as 'srcSize' to ZSTD_decompressContinue(). - ZSTD_decompressContinue() will use previous data blocks to improve compression if they are located prior to current block. - Result is the number of bytes regenerated within 'dst'. - It can be zero, which is not an error; it just means ZSTD_decompressContinue() has decoded some header. -*/ - -/* ************************************* -* Prefix - version detection -***************************************/ -#define ZSTDv03_magicNumber 0xFD2FB523 /* v0.3 */ - - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTD_V03_H_298734209782 */ -/**** ended inlining zstd_v03.h ****/ -#endif -#if (ZSTD_LEGACY_SUPPORT <= 4) -/**** start inlining zstd_v04.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTD_V04_H_91868324769238 -#define ZSTD_V04_H_91868324769238 - -#if defined (__cplusplus) -extern "C" { -#endif - -/* ************************************* -* Includes -***************************************/ -#include /* size_t */ - - -/* ************************************* -* Simple one-step function -***************************************/ -/** -ZSTDv04_decompress() : decompress ZSTD frames compliant with v0.4.x format - compressedSize : is the exact source size - maxOriginalSize : is the size of the 'dst' buffer, which must be already allocated. - It must be equal or larger than originalSize, otherwise decompression will fail. - return : the number of bytes decompressed into destination buffer (originalSize) - or an errorCode if it fails (which can be tested using ZSTDv01_isError()) -*/ -size_t ZSTDv04_decompress( void* dst, size_t maxOriginalSize, - const void* src, size_t compressedSize); - - /** - ZSTDv04_findFrameSizeInfoLegacy() : get the source length and decompressed bound of a ZSTD frame compliant with v0.4.x format - srcSize : The size of the 'src' buffer, at least as large as the frame pointed to by 'src' - cSize (output parameter) : the number of bytes that would be read to decompress this frame - or an error code if it fails (which can be tested using ZSTDv01_isError()) - dBound (output parameter) : an upper-bound for the decompressed size of the data in the frame - or ZSTD_CONTENTSIZE_ERROR if an error occurs - - note : assumes `cSize` and `dBound` are _not_ NULL. - */ - void ZSTDv04_findFrameSizeInfoLegacy(const void *src, size_t srcSize, - size_t* cSize, unsigned long long* dBound); - -/** -ZSTDv04_isError() : tells if the result of ZSTDv04_decompress() is an error -*/ -unsigned ZSTDv04_isError(size_t code); - - -/* ************************************* -* Advanced functions -***************************************/ -typedef struct ZSTDv04_Dctx_s ZSTDv04_Dctx; -ZSTDv04_Dctx* ZSTDv04_createDCtx(void); -size_t ZSTDv04_freeDCtx(ZSTDv04_Dctx* dctx); - -size_t ZSTDv04_decompressDCtx(ZSTDv04_Dctx* dctx, - void* dst, size_t maxOriginalSize, - const void* src, size_t compressedSize); - - -/* ************************************* -* Direct Streaming -***************************************/ -size_t ZSTDv04_resetDCtx(ZSTDv04_Dctx* dctx); - -size_t ZSTDv04_nextSrcSizeToDecompress(ZSTDv04_Dctx* dctx); -size_t ZSTDv04_decompressContinue(ZSTDv04_Dctx* dctx, void* dst, size_t maxDstSize, const void* src, size_t srcSize); -/** - Use above functions alternatively. - ZSTD_nextSrcSizeToDecompress() tells how much bytes to provide as 'srcSize' to ZSTD_decompressContinue(). - ZSTD_decompressContinue() will use previous data blocks to improve compression if they are located prior to current block. - Result is the number of bytes regenerated within 'dst'. - It can be zero, which is not an error; it just means ZSTD_decompressContinue() has decoded some header. -*/ - - -/* ************************************* -* Buffered Streaming -***************************************/ -typedef struct ZBUFFv04_DCtx_s ZBUFFv04_DCtx; -ZBUFFv04_DCtx* ZBUFFv04_createDCtx(void); -size_t ZBUFFv04_freeDCtx(ZBUFFv04_DCtx* dctx); - -size_t ZBUFFv04_decompressInit(ZBUFFv04_DCtx* dctx); -size_t ZBUFFv04_decompressWithDictionary(ZBUFFv04_DCtx* dctx, const void* dict, size_t dictSize); - -size_t ZBUFFv04_decompressContinue(ZBUFFv04_DCtx* dctx, void* dst, size_t* maxDstSizePtr, const void* src, size_t* srcSizePtr); - -/** ************************************************ -* Streaming decompression -* -* A ZBUFF_DCtx object is required to track streaming operation. -* Use ZBUFF_createDCtx() and ZBUFF_freeDCtx() to create/release resources. -* Use ZBUFF_decompressInit() to start a new decompression operation. -* ZBUFF_DCtx objects can be reused multiple times. -* -* Optionally, a reference to a static dictionary can be set, using ZBUFF_decompressWithDictionary() -* It must be the same content as the one set during compression phase. -* Dictionary content must remain accessible during the decompression process. -* -* Use ZBUFF_decompressContinue() repetitively to consume your input. -* *srcSizePtr and *maxDstSizePtr can be any size. -* The function will report how many bytes were read or written by modifying *srcSizePtr and *maxDstSizePtr. -* Note that it may not consume the entire input, in which case it's up to the caller to present remaining input again. -* The content of dst will be overwritten (up to *maxDstSizePtr) at each function call, so save its content if it matters or change dst. -* @return : a hint to preferred nb of bytes to use as input for next function call (it's only a hint, to improve latency) -* or 0 when a frame is completely decoded -* or an error code, which can be tested using ZBUFF_isError(). -* -* Hint : recommended buffer sizes (not compulsory) : ZBUFF_recommendedDInSize / ZBUFF_recommendedDOutSize -* output : ZBUFF_recommendedDOutSize==128 KB block size is the internal unit, it ensures it's always possible to write a full block when it's decoded. -* input : ZBUFF_recommendedDInSize==128Kb+3; just follow indications from ZBUFF_decompressContinue() to minimize latency. It should always be <= 128 KB + 3 . -* **************************************************/ -unsigned ZBUFFv04_isError(size_t errorCode); -const char* ZBUFFv04_getErrorName(size_t errorCode); - - -/** The below functions provide recommended buffer sizes for Compression or Decompression operations. -* These sizes are not compulsory, they just tend to offer better latency */ -size_t ZBUFFv04_recommendedDInSize(void); -size_t ZBUFFv04_recommendedDOutSize(void); - - -/* ************************************* -* Prefix - version detection -***************************************/ -#define ZSTDv04_magicNumber 0xFD2FB524 /* v0.4 */ - - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTD_V04_H_91868324769238 */ -/**** ended inlining zstd_v04.h ****/ -#endif -#if (ZSTD_LEGACY_SUPPORT <= 5) -/**** start inlining zstd_v05.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTDv05_H -#define ZSTDv05_H - -#if defined (__cplusplus) -extern "C" { -#endif - -/*-************************************* -* Dependencies -***************************************/ -#include /* size_t */ -/**** skipping file: ../common/mem.h ****/ - - -/* ************************************* -* Simple functions -***************************************/ -/*! ZSTDv05_decompress() : - `compressedSize` : is the _exact_ size of the compressed blob, otherwise decompression will fail. - `dstCapacity` must be large enough, equal or larger than originalSize. - @return : the number of bytes decompressed into `dst` (<= `dstCapacity`), - or an errorCode if it fails (which can be tested using ZSTDv05_isError()) */ -size_t ZSTDv05_decompress( void* dst, size_t dstCapacity, - const void* src, size_t compressedSize); - - /** - ZSTDv05_findFrameSizeInfoLegacy() : get the source length and decompressed bound of a ZSTD frame compliant with v0.5.x format - srcSize : The size of the 'src' buffer, at least as large as the frame pointed to by 'src' - cSize (output parameter) : the number of bytes that would be read to decompress this frame - or an error code if it fails (which can be tested using ZSTDv01_isError()) - dBound (output parameter) : an upper-bound for the decompressed size of the data in the frame - or ZSTD_CONTENTSIZE_ERROR if an error occurs - - note : assumes `cSize` and `dBound` are _not_ NULL. - */ -void ZSTDv05_findFrameSizeInfoLegacy(const void *src, size_t srcSize, - size_t* cSize, unsigned long long* dBound); - -/* ************************************* -* Helper functions -***************************************/ -/* Error Management */ -unsigned ZSTDv05_isError(size_t code); /*!< tells if a `size_t` function result is an error code */ -const char* ZSTDv05_getErrorName(size_t code); /*!< provides readable string for an error code */ - - -/* ************************************* -* Explicit memory management -***************************************/ -/** Decompression context */ -typedef struct ZSTDv05_DCtx_s ZSTDv05_DCtx; -ZSTDv05_DCtx* ZSTDv05_createDCtx(void); -size_t ZSTDv05_freeDCtx(ZSTDv05_DCtx* dctx); /*!< @return : errorCode */ - -/** ZSTDv05_decompressDCtx() : -* Same as ZSTDv05_decompress(), but requires an already allocated ZSTDv05_DCtx (see ZSTDv05_createDCtx()) */ -size_t ZSTDv05_decompressDCtx(ZSTDv05_DCtx* ctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize); - - -/*-*********************** -* Simple Dictionary API -*************************/ -/*! ZSTDv05_decompress_usingDict() : -* Decompression using a pre-defined Dictionary content (see dictBuilder). -* Dictionary must be identical to the one used during compression, otherwise regenerated data will be corrupted. -* Note : dict can be NULL, in which case, it's equivalent to ZSTDv05_decompressDCtx() */ -size_t ZSTDv05_decompress_usingDict(ZSTDv05_DCtx* dctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const void* dict,size_t dictSize); - -/*-************************ -* Advanced Streaming API -***************************/ -typedef enum { ZSTDv05_fast, ZSTDv05_greedy, ZSTDv05_lazy, ZSTDv05_lazy2, ZSTDv05_btlazy2, ZSTDv05_opt, ZSTDv05_btopt } ZSTDv05_strategy; -typedef struct { - U64 srcSize; - U32 windowLog; /* the only useful information to retrieve */ - U32 contentLog; U32 hashLog; U32 searchLog; U32 searchLength; U32 targetLength; ZSTDv05_strategy strategy; -} ZSTDv05_parameters; -size_t ZSTDv05_getFrameParams(ZSTDv05_parameters* params, const void* src, size_t srcSize); - -size_t ZSTDv05_decompressBegin_usingDict(ZSTDv05_DCtx* dctx, const void* dict, size_t dictSize); -void ZSTDv05_copyDCtx(ZSTDv05_DCtx* dstDCtx, const ZSTDv05_DCtx* srcDCtx); -size_t ZSTDv05_nextSrcSizeToDecompress(ZSTDv05_DCtx* dctx); -size_t ZSTDv05_decompressContinue(ZSTDv05_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize); - - -/*-*********************** -* ZBUFF API -*************************/ -typedef struct ZBUFFv05_DCtx_s ZBUFFv05_DCtx; -ZBUFFv05_DCtx* ZBUFFv05_createDCtx(void); -size_t ZBUFFv05_freeDCtx(ZBUFFv05_DCtx* dctx); - -size_t ZBUFFv05_decompressInit(ZBUFFv05_DCtx* dctx); -size_t ZBUFFv05_decompressInitDictionary(ZBUFFv05_DCtx* dctx, const void* dict, size_t dictSize); - -size_t ZBUFFv05_decompressContinue(ZBUFFv05_DCtx* dctx, - void* dst, size_t* dstCapacityPtr, - const void* src, size_t* srcSizePtr); - -/*-*************************************************************************** -* Streaming decompression -* -* A ZBUFFv05_DCtx object is required to track streaming operations. -* Use ZBUFFv05_createDCtx() and ZBUFFv05_freeDCtx() to create/release resources. -* Use ZBUFFv05_decompressInit() to start a new decompression operation, -* or ZBUFFv05_decompressInitDictionary() if decompression requires a dictionary. -* Note that ZBUFFv05_DCtx objects can be reused multiple times. -* -* Use ZBUFFv05_decompressContinue() repetitively to consume your input. -* *srcSizePtr and *dstCapacityPtr can be any size. -* The function will report how many bytes were read or written by modifying *srcSizePtr and *dstCapacityPtr. -* Note that it may not consume the entire input, in which case it's up to the caller to present remaining input again. -* The content of @dst will be overwritten (up to *dstCapacityPtr) at each function call, so save its content if it matters or change @dst. -* @return : a hint to preferred nb of bytes to use as input for next function call (it's only a hint, to help latency) -* or 0 when a frame is completely decoded -* or an error code, which can be tested using ZBUFFv05_isError(). -* -* Hint : recommended buffer sizes (not compulsory) : ZBUFFv05_recommendedDInSize() / ZBUFFv05_recommendedDOutSize() -* output : ZBUFFv05_recommendedDOutSize==128 KB block size is the internal unit, it ensures it's always possible to write a full block when decoded. -* input : ZBUFFv05_recommendedDInSize==128Kb+3; just follow indications from ZBUFFv05_decompressContinue() to minimize latency. It should always be <= 128 KB + 3 . -* *******************************************************************************/ - - -/* ************************************* -* Tool functions -***************************************/ -unsigned ZBUFFv05_isError(size_t errorCode); -const char* ZBUFFv05_getErrorName(size_t errorCode); - -/** Functions below provide recommended buffer sizes for Compression or Decompression operations. -* These sizes are just hints, and tend to offer better latency */ -size_t ZBUFFv05_recommendedDInSize(void); -size_t ZBUFFv05_recommendedDOutSize(void); - - - -/*-************************************* -* Constants -***************************************/ -#define ZSTDv05_MAGICNUMBER 0xFD2FB525 /* v0.5 */ - - - - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTDv0505_H */ -/**** ended inlining zstd_v05.h ****/ -#endif -#if (ZSTD_LEGACY_SUPPORT <= 6) -/**** start inlining zstd_v06.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTDv06_H -#define ZSTDv06_H - -#if defined (__cplusplus) -extern "C" { -#endif - -/*====== Dependency ======*/ -#include /* size_t */ - - -/*====== Export for Windows ======*/ -/*! -* ZSTDv06_DLL_EXPORT : -* Enable exporting of functions when building a Windows DLL -*/ -#if defined(_WIN32) && defined(ZSTDv06_DLL_EXPORT) && (ZSTDv06_DLL_EXPORT==1) -# define ZSTDLIBv06_API __declspec(dllexport) -#else -# define ZSTDLIBv06_API -#endif - - -/* ************************************* -* Simple functions -***************************************/ -/*! ZSTDv06_decompress() : - `compressedSize` : is the _exact_ size of the compressed blob, otherwise decompression will fail. - `dstCapacity` must be large enough, equal or larger than originalSize. - @return : the number of bytes decompressed into `dst` (<= `dstCapacity`), - or an errorCode if it fails (which can be tested using ZSTDv06_isError()) */ -ZSTDLIBv06_API size_t ZSTDv06_decompress( void* dst, size_t dstCapacity, - const void* src, size_t compressedSize); - -/** -ZSTDv06_findFrameSizeInfoLegacy() : get the source length and decompressed bound of a ZSTD frame compliant with v0.6.x format - srcSize : The size of the 'src' buffer, at least as large as the frame pointed to by 'src' - cSize (output parameter) : the number of bytes that would be read to decompress this frame - or an error code if it fails (which can be tested using ZSTDv01_isError()) - dBound (output parameter) : an upper-bound for the decompressed size of the data in the frame - or ZSTD_CONTENTSIZE_ERROR if an error occurs - - note : assumes `cSize` and `dBound` are _not_ NULL. -*/ -void ZSTDv06_findFrameSizeInfoLegacy(const void *src, size_t srcSize, - size_t* cSize, unsigned long long* dBound); - -/* ************************************* -* Helper functions -***************************************/ -ZSTDLIBv06_API size_t ZSTDv06_compressBound(size_t srcSize); /*!< maximum compressed size (worst case scenario) */ - -/* Error Management */ -ZSTDLIBv06_API unsigned ZSTDv06_isError(size_t code); /*!< tells if a `size_t` function result is an error code */ -ZSTDLIBv06_API const char* ZSTDv06_getErrorName(size_t code); /*!< provides readable string for an error code */ - - -/* ************************************* -* Explicit memory management -***************************************/ -/** Decompression context */ -typedef struct ZSTDv06_DCtx_s ZSTDv06_DCtx; -ZSTDLIBv06_API ZSTDv06_DCtx* ZSTDv06_createDCtx(void); -ZSTDLIBv06_API size_t ZSTDv06_freeDCtx(ZSTDv06_DCtx* dctx); /*!< @return : errorCode */ - -/** ZSTDv06_decompressDCtx() : -* Same as ZSTDv06_decompress(), but requires an already allocated ZSTDv06_DCtx (see ZSTDv06_createDCtx()) */ -ZSTDLIBv06_API size_t ZSTDv06_decompressDCtx(ZSTDv06_DCtx* ctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize); - - -/*-*********************** -* Dictionary API -*************************/ -/*! ZSTDv06_decompress_usingDict() : -* Decompression using a pre-defined Dictionary content (see dictBuilder). -* Dictionary must be identical to the one used during compression, otherwise regenerated data will be corrupted. -* Note : dict can be NULL, in which case, it's equivalent to ZSTDv06_decompressDCtx() */ -ZSTDLIBv06_API size_t ZSTDv06_decompress_usingDict(ZSTDv06_DCtx* dctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const void* dict,size_t dictSize); - - -/*-************************ -* Advanced Streaming API -***************************/ -struct ZSTDv06_frameParams_s { unsigned long long frameContentSize; unsigned windowLog; }; -typedef struct ZSTDv06_frameParams_s ZSTDv06_frameParams; - -ZSTDLIBv06_API size_t ZSTDv06_getFrameParams(ZSTDv06_frameParams* fparamsPtr, const void* src, size_t srcSize); /**< doesn't consume input */ -ZSTDLIBv06_API size_t ZSTDv06_decompressBegin_usingDict(ZSTDv06_DCtx* dctx, const void* dict, size_t dictSize); -ZSTDLIBv06_API void ZSTDv06_copyDCtx(ZSTDv06_DCtx* dctx, const ZSTDv06_DCtx* preparedDCtx); - -ZSTDLIBv06_API size_t ZSTDv06_nextSrcSizeToDecompress(ZSTDv06_DCtx* dctx); -ZSTDLIBv06_API size_t ZSTDv06_decompressContinue(ZSTDv06_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize); - - - -/* ************************************* -* ZBUFF API -***************************************/ - -typedef struct ZBUFFv06_DCtx_s ZBUFFv06_DCtx; -ZSTDLIBv06_API ZBUFFv06_DCtx* ZBUFFv06_createDCtx(void); -ZSTDLIBv06_API size_t ZBUFFv06_freeDCtx(ZBUFFv06_DCtx* dctx); - -ZSTDLIBv06_API size_t ZBUFFv06_decompressInit(ZBUFFv06_DCtx* dctx); -ZSTDLIBv06_API size_t ZBUFFv06_decompressInitDictionary(ZBUFFv06_DCtx* dctx, const void* dict, size_t dictSize); - -ZSTDLIBv06_API size_t ZBUFFv06_decompressContinue(ZBUFFv06_DCtx* dctx, - void* dst, size_t* dstCapacityPtr, - const void* src, size_t* srcSizePtr); - -/*-*************************************************************************** -* Streaming decompression howto -* -* A ZBUFFv06_DCtx object is required to track streaming operations. -* Use ZBUFFv06_createDCtx() and ZBUFFv06_freeDCtx() to create/release resources. -* Use ZBUFFv06_decompressInit() to start a new decompression operation, -* or ZBUFFv06_decompressInitDictionary() if decompression requires a dictionary. -* Note that ZBUFFv06_DCtx objects can be re-init multiple times. -* -* Use ZBUFFv06_decompressContinue() repetitively to consume your input. -* *srcSizePtr and *dstCapacityPtr can be any size. -* The function will report how many bytes were read or written by modifying *srcSizePtr and *dstCapacityPtr. -* Note that it may not consume the entire input, in which case it's up to the caller to present remaining input again. -* The content of `dst` will be overwritten (up to *dstCapacityPtr) at each function call, so save its content if it matters, or change `dst`. -* @return : a hint to preferred nb of bytes to use as input for next function call (it's only a hint, to help latency), -* or 0 when a frame is completely decoded, -* or an error code, which can be tested using ZBUFFv06_isError(). -* -* Hint : recommended buffer sizes (not compulsory) : ZBUFFv06_recommendedDInSize() and ZBUFFv06_recommendedDOutSize() -* output : ZBUFFv06_recommendedDOutSize== 128 KB block size is the internal unit, it ensures it's always possible to write a full block when decoded. -* input : ZBUFFv06_recommendedDInSize == 128KB + 3; -* just follow indications from ZBUFFv06_decompressContinue() to minimize latency. It should always be <= 128 KB + 3 . -* *******************************************************************************/ - - -/* ************************************* -* Tool functions -***************************************/ -ZSTDLIBv06_API unsigned ZBUFFv06_isError(size_t errorCode); -ZSTDLIBv06_API const char* ZBUFFv06_getErrorName(size_t errorCode); - -/** Functions below provide recommended buffer sizes for Compression or Decompression operations. -* These sizes are just hints, they tend to offer better latency */ -ZSTDLIBv06_API size_t ZBUFFv06_recommendedDInSize(void); -ZSTDLIBv06_API size_t ZBUFFv06_recommendedDOutSize(void); - - -/*-************************************* -* Constants -***************************************/ -#define ZSTDv06_MAGICNUMBER 0xFD2FB526 /* v0.6 */ - - - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTDv06_BUFFERED_H */ -/**** ended inlining zstd_v06.h ****/ -#endif -#if (ZSTD_LEGACY_SUPPORT <= 7) -/**** start inlining zstd_v07.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -#ifndef ZSTDv07_H_235446 -#define ZSTDv07_H_235446 - -#if defined (__cplusplus) -extern "C" { -#endif - -/*====== Dependency ======*/ -#include /* size_t */ - - -/*====== Export for Windows ======*/ -/*! -* ZSTDv07_DLL_EXPORT : -* Enable exporting of functions when building a Windows DLL -*/ -#if defined(_WIN32) && defined(ZSTDv07_DLL_EXPORT) && (ZSTDv07_DLL_EXPORT==1) -# define ZSTDLIBv07_API __declspec(dllexport) -#else -# define ZSTDLIBv07_API -#endif - - -/* ************************************* -* Simple API -***************************************/ -/*! ZSTDv07_getDecompressedSize() : -* @return : decompressed size if known, 0 otherwise. - note 1 : if `0`, follow up with ZSTDv07_getFrameParams() to know precise failure cause. - note 2 : decompressed size could be wrong or intentionally modified ! - always ensure results fit within application's authorized limits */ -unsigned long long ZSTDv07_getDecompressedSize(const void* src, size_t srcSize); - -/*! ZSTDv07_decompress() : - `compressedSize` : must be _exact_ size of compressed input, otherwise decompression will fail. - `dstCapacity` must be equal or larger than originalSize. - @return : the number of bytes decompressed into `dst` (<= `dstCapacity`), - or an errorCode if it fails (which can be tested using ZSTDv07_isError()) */ -ZSTDLIBv07_API size_t ZSTDv07_decompress( void* dst, size_t dstCapacity, - const void* src, size_t compressedSize); - -/** -ZSTDv07_findFrameSizeInfoLegacy() : get the source length and decompressed bound of a ZSTD frame compliant with v0.7.x format - srcSize : The size of the 'src' buffer, at least as large as the frame pointed to by 'src' - cSize (output parameter) : the number of bytes that would be read to decompress this frame - or an error code if it fails (which can be tested using ZSTDv01_isError()) - dBound (output parameter) : an upper-bound for the decompressed size of the data in the frame - or ZSTD_CONTENTSIZE_ERROR if an error occurs - - note : assumes `cSize` and `dBound` are _not_ NULL. -*/ -void ZSTDv07_findFrameSizeInfoLegacy(const void *src, size_t srcSize, - size_t* cSize, unsigned long long* dBound); - -/*====== Helper functions ======*/ -ZSTDLIBv07_API unsigned ZSTDv07_isError(size_t code); /*!< tells if a `size_t` function result is an error code */ -ZSTDLIBv07_API const char* ZSTDv07_getErrorName(size_t code); /*!< provides readable string from an error code */ - - -/*-************************************* -* Explicit memory management -***************************************/ -/** Decompression context */ -typedef struct ZSTDv07_DCtx_s ZSTDv07_DCtx; -ZSTDLIBv07_API ZSTDv07_DCtx* ZSTDv07_createDCtx(void); -ZSTDLIBv07_API size_t ZSTDv07_freeDCtx(ZSTDv07_DCtx* dctx); /*!< @return : errorCode */ - -/** ZSTDv07_decompressDCtx() : -* Same as ZSTDv07_decompress(), requires an allocated ZSTDv07_DCtx (see ZSTDv07_createDCtx()) */ -ZSTDLIBv07_API size_t ZSTDv07_decompressDCtx(ZSTDv07_DCtx* ctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize); - - -/*-************************ -* Simple dictionary API -***************************/ -/*! ZSTDv07_decompress_usingDict() : -* Decompression using a pre-defined Dictionary content (see dictBuilder). -* Dictionary must be identical to the one used during compression. -* Note : This function load the dictionary, resulting in a significant startup time */ -ZSTDLIBv07_API size_t ZSTDv07_decompress_usingDict(ZSTDv07_DCtx* dctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const void* dict,size_t dictSize); - - -/*-************************** -* Advanced Dictionary API -****************************/ -/*! ZSTDv07_createDDict() : -* Create a digested dictionary, ready to start decompression operation without startup delay. -* `dict` can be released after creation */ -typedef struct ZSTDv07_DDict_s ZSTDv07_DDict; -ZSTDLIBv07_API ZSTDv07_DDict* ZSTDv07_createDDict(const void* dict, size_t dictSize); -ZSTDLIBv07_API size_t ZSTDv07_freeDDict(ZSTDv07_DDict* ddict); - -/*! ZSTDv07_decompress_usingDDict() : -* Decompression using a pre-digested Dictionary -* Faster startup than ZSTDv07_decompress_usingDict(), recommended when same dictionary is used multiple times. */ -ZSTDLIBv07_API size_t ZSTDv07_decompress_usingDDict(ZSTDv07_DCtx* dctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const ZSTDv07_DDict* ddict); - -typedef struct { - unsigned long long frameContentSize; - unsigned windowSize; - unsigned dictID; - unsigned checksumFlag; -} ZSTDv07_frameParams; - -ZSTDLIBv07_API size_t ZSTDv07_getFrameParams(ZSTDv07_frameParams* fparamsPtr, const void* src, size_t srcSize); /**< doesn't consume input */ - - - - -/* ************************************* -* Streaming functions -***************************************/ -typedef struct ZBUFFv07_DCtx_s ZBUFFv07_DCtx; -ZSTDLIBv07_API ZBUFFv07_DCtx* ZBUFFv07_createDCtx(void); -ZSTDLIBv07_API size_t ZBUFFv07_freeDCtx(ZBUFFv07_DCtx* dctx); - -ZSTDLIBv07_API size_t ZBUFFv07_decompressInit(ZBUFFv07_DCtx* dctx); -ZSTDLIBv07_API size_t ZBUFFv07_decompressInitDictionary(ZBUFFv07_DCtx* dctx, const void* dict, size_t dictSize); - -ZSTDLIBv07_API size_t ZBUFFv07_decompressContinue(ZBUFFv07_DCtx* dctx, - void* dst, size_t* dstCapacityPtr, - const void* src, size_t* srcSizePtr); - -/*-*************************************************************************** -* Streaming decompression howto -* -* A ZBUFFv07_DCtx object is required to track streaming operations. -* Use ZBUFFv07_createDCtx() and ZBUFFv07_freeDCtx() to create/release resources. -* Use ZBUFFv07_decompressInit() to start a new decompression operation, -* or ZBUFFv07_decompressInitDictionary() if decompression requires a dictionary. -* Note that ZBUFFv07_DCtx objects can be re-init multiple times. -* -* Use ZBUFFv07_decompressContinue() repetitively to consume your input. -* *srcSizePtr and *dstCapacityPtr can be any size. -* The function will report how many bytes were read or written by modifying *srcSizePtr and *dstCapacityPtr. -* Note that it may not consume the entire input, in which case it's up to the caller to present remaining input again. -* The content of `dst` will be overwritten (up to *dstCapacityPtr) at each function call, so save its content if it matters, or change `dst`. -* @return : a hint to preferred nb of bytes to use as input for next function call (it's only a hint, to help latency), -* or 0 when a frame is completely decoded, -* or an error code, which can be tested using ZBUFFv07_isError(). -* -* Hint : recommended buffer sizes (not compulsory) : ZBUFFv07_recommendedDInSize() and ZBUFFv07_recommendedDOutSize() -* output : ZBUFFv07_recommendedDOutSize== 128 KB block size is the internal unit, it ensures it's always possible to write a full block when decoded. -* input : ZBUFFv07_recommendedDInSize == 128KB + 3; -* just follow indications from ZBUFFv07_decompressContinue() to minimize latency. It should always be <= 128 KB + 3 . -* *******************************************************************************/ - - -/* ************************************* -* Tool functions -***************************************/ -ZSTDLIBv07_API unsigned ZBUFFv07_isError(size_t errorCode); -ZSTDLIBv07_API const char* ZBUFFv07_getErrorName(size_t errorCode); - -/** Functions below provide recommended buffer sizes for Compression or Decompression operations. -* These sizes are just hints, they tend to offer better latency */ -ZSTDLIBv07_API size_t ZBUFFv07_recommendedDInSize(void); -ZSTDLIBv07_API size_t ZBUFFv07_recommendedDOutSize(void); - - -/*-************************************* -* Constants -***************************************/ -#define ZSTDv07_MAGICNUMBER 0xFD2FB527 /* v0.7 */ - - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTDv07_H_235446 */ -/**** ended inlining zstd_v07.h ****/ -#endif - -/** ZSTD_isLegacy() : - @return : > 0 if supported by legacy decoder. 0 otherwise. - return value is the version. -*/ -MEM_STATIC unsigned ZSTD_isLegacy(const void* src, size_t srcSize) -{ - U32 magicNumberLE; - if (srcSize<4) return 0; - magicNumberLE = MEM_readLE32(src); - switch(magicNumberLE) - { -#if (ZSTD_LEGACY_SUPPORT <= 1) - case ZSTDv01_magicNumberLE:return 1; -#endif -#if (ZSTD_LEGACY_SUPPORT <= 2) - case ZSTDv02_magicNumber : return 2; -#endif -#if (ZSTD_LEGACY_SUPPORT <= 3) - case ZSTDv03_magicNumber : return 3; -#endif -#if (ZSTD_LEGACY_SUPPORT <= 4) - case ZSTDv04_magicNumber : return 4; -#endif -#if (ZSTD_LEGACY_SUPPORT <= 5) - case ZSTDv05_MAGICNUMBER : return 5; -#endif -#if (ZSTD_LEGACY_SUPPORT <= 6) - case ZSTDv06_MAGICNUMBER : return 6; -#endif -#if (ZSTD_LEGACY_SUPPORT <= 7) - case ZSTDv07_MAGICNUMBER : return 7; -#endif - default : return 0; - } -} - - -MEM_STATIC unsigned long long ZSTD_getDecompressedSize_legacy(const void* src, size_t srcSize) -{ - U32 const version = ZSTD_isLegacy(src, srcSize); - if (version < 5) return 0; /* no decompressed size in frame header, or not a legacy format */ -#if (ZSTD_LEGACY_SUPPORT <= 5) - if (version==5) { - ZSTDv05_parameters fParams; - size_t const frResult = ZSTDv05_getFrameParams(&fParams, src, srcSize); - if (frResult != 0) return 0; - return fParams.srcSize; - } -#endif -#if (ZSTD_LEGACY_SUPPORT <= 6) - if (version==6) { - ZSTDv06_frameParams fParams; - size_t const frResult = ZSTDv06_getFrameParams(&fParams, src, srcSize); - if (frResult != 0) return 0; - return fParams.frameContentSize; - } -#endif -#if (ZSTD_LEGACY_SUPPORT <= 7) - if (version==7) { - ZSTDv07_frameParams fParams; - size_t const frResult = ZSTDv07_getFrameParams(&fParams, src, srcSize); - if (frResult != 0) return 0; - return fParams.frameContentSize; - } -#endif - return 0; /* should not be possible */ -} - - -MEM_STATIC size_t ZSTD_decompressLegacy( - void* dst, size_t dstCapacity, - const void* src, size_t compressedSize, - const void* dict,size_t dictSize) -{ - U32 const version = ZSTD_isLegacy(src, compressedSize); - (void)dst; (void)dstCapacity; (void)dict; (void)dictSize; /* unused when ZSTD_LEGACY_SUPPORT >= 8 */ - switch(version) - { -#if (ZSTD_LEGACY_SUPPORT <= 1) - case 1 : - return ZSTDv01_decompress(dst, dstCapacity, src, compressedSize); -#endif -#if (ZSTD_LEGACY_SUPPORT <= 2) - case 2 : - return ZSTDv02_decompress(dst, dstCapacity, src, compressedSize); -#endif -#if (ZSTD_LEGACY_SUPPORT <= 3) - case 3 : - return ZSTDv03_decompress(dst, dstCapacity, src, compressedSize); -#endif -#if (ZSTD_LEGACY_SUPPORT <= 4) - case 4 : - return ZSTDv04_decompress(dst, dstCapacity, src, compressedSize); -#endif -#if (ZSTD_LEGACY_SUPPORT <= 5) - case 5 : - { size_t result; - ZSTDv05_DCtx* const zd = ZSTDv05_createDCtx(); - if (zd==NULL) return ERROR(memory_allocation); - result = ZSTDv05_decompress_usingDict(zd, dst, dstCapacity, src, compressedSize, dict, dictSize); - ZSTDv05_freeDCtx(zd); - return result; - } -#endif -#if (ZSTD_LEGACY_SUPPORT <= 6) - case 6 : - { size_t result; - ZSTDv06_DCtx* const zd = ZSTDv06_createDCtx(); - if (zd==NULL) return ERROR(memory_allocation); - result = ZSTDv06_decompress_usingDict(zd, dst, dstCapacity, src, compressedSize, dict, dictSize); - ZSTDv06_freeDCtx(zd); - return result; - } -#endif -#if (ZSTD_LEGACY_SUPPORT <= 7) - case 7 : - { size_t result; - ZSTDv07_DCtx* const zd = ZSTDv07_createDCtx(); - if (zd==NULL) return ERROR(memory_allocation); - result = ZSTDv07_decompress_usingDict(zd, dst, dstCapacity, src, compressedSize, dict, dictSize); - ZSTDv07_freeDCtx(zd); - return result; - } -#endif - default : - return ERROR(prefix_unknown); - } -} - -MEM_STATIC ZSTD_frameSizeInfo ZSTD_findFrameSizeInfoLegacy(const void *src, size_t srcSize) -{ - ZSTD_frameSizeInfo frameSizeInfo; - U32 const version = ZSTD_isLegacy(src, srcSize); - switch(version) - { -#if (ZSTD_LEGACY_SUPPORT <= 1) - case 1 : - ZSTDv01_findFrameSizeInfoLegacy(src, srcSize, - &frameSizeInfo.compressedSize, - &frameSizeInfo.decompressedBound); - break; -#endif -#if (ZSTD_LEGACY_SUPPORT <= 2) - case 2 : - ZSTDv02_findFrameSizeInfoLegacy(src, srcSize, - &frameSizeInfo.compressedSize, - &frameSizeInfo.decompressedBound); - break; -#endif -#if (ZSTD_LEGACY_SUPPORT <= 3) - case 3 : - ZSTDv03_findFrameSizeInfoLegacy(src, srcSize, - &frameSizeInfo.compressedSize, - &frameSizeInfo.decompressedBound); - break; -#endif -#if (ZSTD_LEGACY_SUPPORT <= 4) - case 4 : - ZSTDv04_findFrameSizeInfoLegacy(src, srcSize, - &frameSizeInfo.compressedSize, - &frameSizeInfo.decompressedBound); - break; -#endif -#if (ZSTD_LEGACY_SUPPORT <= 5) - case 5 : - ZSTDv05_findFrameSizeInfoLegacy(src, srcSize, - &frameSizeInfo.compressedSize, - &frameSizeInfo.decompressedBound); - break; -#endif -#if (ZSTD_LEGACY_SUPPORT <= 6) - case 6 : - ZSTDv06_findFrameSizeInfoLegacy(src, srcSize, - &frameSizeInfo.compressedSize, - &frameSizeInfo.decompressedBound); - break; -#endif -#if (ZSTD_LEGACY_SUPPORT <= 7) - case 7 : - ZSTDv07_findFrameSizeInfoLegacy(src, srcSize, - &frameSizeInfo.compressedSize, - &frameSizeInfo.decompressedBound); - break; -#endif - default : - frameSizeInfo.compressedSize = ERROR(prefix_unknown); - frameSizeInfo.decompressedBound = ZSTD_CONTENTSIZE_ERROR; - break; - } - if (!ZSTD_isError(frameSizeInfo.compressedSize) && frameSizeInfo.compressedSize > srcSize) { - frameSizeInfo.compressedSize = ERROR(srcSize_wrong); - frameSizeInfo.decompressedBound = ZSTD_CONTENTSIZE_ERROR; - } - return frameSizeInfo; -} - -MEM_STATIC size_t ZSTD_findFrameCompressedSizeLegacy(const void *src, size_t srcSize) -{ - ZSTD_frameSizeInfo frameSizeInfo = ZSTD_findFrameSizeInfoLegacy(src, srcSize); - return frameSizeInfo.compressedSize; -} - -MEM_STATIC size_t ZSTD_freeLegacyStreamContext(void* legacyContext, U32 version) -{ - switch(version) - { - default : - case 1 : - case 2 : - case 3 : - (void)legacyContext; - return ERROR(version_unsupported); -#if (ZSTD_LEGACY_SUPPORT <= 4) - case 4 : return ZBUFFv04_freeDCtx((ZBUFFv04_DCtx*)legacyContext); -#endif -#if (ZSTD_LEGACY_SUPPORT <= 5) - case 5 : return ZBUFFv05_freeDCtx((ZBUFFv05_DCtx*)legacyContext); -#endif -#if (ZSTD_LEGACY_SUPPORT <= 6) - case 6 : return ZBUFFv06_freeDCtx((ZBUFFv06_DCtx*)legacyContext); -#endif -#if (ZSTD_LEGACY_SUPPORT <= 7) - case 7 : return ZBUFFv07_freeDCtx((ZBUFFv07_DCtx*)legacyContext); -#endif - } -} - - -MEM_STATIC size_t ZSTD_initLegacyStream(void** legacyContext, U32 prevVersion, U32 newVersion, - const void* dict, size_t dictSize) -{ - DEBUGLOG(5, "ZSTD_initLegacyStream for v0.%u", newVersion); - if (prevVersion != newVersion) ZSTD_freeLegacyStreamContext(*legacyContext, prevVersion); - switch(newVersion) - { - default : - case 1 : - case 2 : - case 3 : - (void)dict; (void)dictSize; - return 0; -#if (ZSTD_LEGACY_SUPPORT <= 4) - case 4 : - { - ZBUFFv04_DCtx* dctx = (prevVersion != newVersion) ? ZBUFFv04_createDCtx() : (ZBUFFv04_DCtx*)*legacyContext; - if (dctx==NULL) return ERROR(memory_allocation); - ZBUFFv04_decompressInit(dctx); - ZBUFFv04_decompressWithDictionary(dctx, dict, dictSize); - *legacyContext = dctx; - return 0; - } -#endif -#if (ZSTD_LEGACY_SUPPORT <= 5) - case 5 : - { - ZBUFFv05_DCtx* dctx = (prevVersion != newVersion) ? ZBUFFv05_createDCtx() : (ZBUFFv05_DCtx*)*legacyContext; - if (dctx==NULL) return ERROR(memory_allocation); - ZBUFFv05_decompressInitDictionary(dctx, dict, dictSize); - *legacyContext = dctx; - return 0; - } -#endif -#if (ZSTD_LEGACY_SUPPORT <= 6) - case 6 : - { - ZBUFFv06_DCtx* dctx = (prevVersion != newVersion) ? ZBUFFv06_createDCtx() : (ZBUFFv06_DCtx*)*legacyContext; - if (dctx==NULL) return ERROR(memory_allocation); - ZBUFFv06_decompressInitDictionary(dctx, dict, dictSize); - *legacyContext = dctx; - return 0; - } -#endif -#if (ZSTD_LEGACY_SUPPORT <= 7) - case 7 : - { - ZBUFFv07_DCtx* dctx = (prevVersion != newVersion) ? ZBUFFv07_createDCtx() : (ZBUFFv07_DCtx*)*legacyContext; - if (dctx==NULL) return ERROR(memory_allocation); - ZBUFFv07_decompressInitDictionary(dctx, dict, dictSize); - *legacyContext = dctx; - return 0; - } -#endif - } -} - - - -MEM_STATIC size_t ZSTD_decompressLegacyStream(void* legacyContext, U32 version, - ZSTD_outBuffer* output, ZSTD_inBuffer* input) -{ - DEBUGLOG(5, "ZSTD_decompressLegacyStream for v0.%u", version); - switch(version) - { - default : - case 1 : - case 2 : - case 3 : - (void)legacyContext; (void)output; (void)input; - return ERROR(version_unsupported); -#if (ZSTD_LEGACY_SUPPORT <= 4) - case 4 : - { - ZBUFFv04_DCtx* dctx = (ZBUFFv04_DCtx*) legacyContext; - const void* src = (const char*)input->src + input->pos; - size_t readSize = input->size - input->pos; - void* dst = (char*)output->dst + output->pos; - size_t decodedSize = output->size - output->pos; - size_t const hintSize = ZBUFFv04_decompressContinue(dctx, dst, &decodedSize, src, &readSize); - output->pos += decodedSize; - input->pos += readSize; - return hintSize; - } -#endif -#if (ZSTD_LEGACY_SUPPORT <= 5) - case 5 : - { - ZBUFFv05_DCtx* dctx = (ZBUFFv05_DCtx*) legacyContext; - const void* src = (const char*)input->src + input->pos; - size_t readSize = input->size - input->pos; - void* dst = (char*)output->dst + output->pos; - size_t decodedSize = output->size - output->pos; - size_t const hintSize = ZBUFFv05_decompressContinue(dctx, dst, &decodedSize, src, &readSize); - output->pos += decodedSize; - input->pos += readSize; - return hintSize; - } -#endif -#if (ZSTD_LEGACY_SUPPORT <= 6) - case 6 : - { - ZBUFFv06_DCtx* dctx = (ZBUFFv06_DCtx*) legacyContext; - const void* src = (const char*)input->src + input->pos; - size_t readSize = input->size - input->pos; - void* dst = (char*)output->dst + output->pos; - size_t decodedSize = output->size - output->pos; - size_t const hintSize = ZBUFFv06_decompressContinue(dctx, dst, &decodedSize, src, &readSize); - output->pos += decodedSize; - input->pos += readSize; - return hintSize; - } -#endif -#if (ZSTD_LEGACY_SUPPORT <= 7) - case 7 : - { - ZBUFFv07_DCtx* dctx = (ZBUFFv07_DCtx*) legacyContext; - const void* src = (const char*)input->src + input->pos; - size_t readSize = input->size - input->pos; - void* dst = (char*)output->dst + output->pos; - size_t decodedSize = output->size - output->pos; - size_t const hintSize = ZBUFFv07_decompressContinue(dctx, dst, &decodedSize, src, &readSize); - output->pos += decodedSize; - input->pos += readSize; - return hintSize; - } -#endif - } -} - - -#if defined (__cplusplus) -} -#endif - -#endif /* ZSTD_LEGACY_H */ -/**** ended inlining ../legacy/zstd_legacy.h ****/ -#endif - - - -/*-******************************************************* -* Types -*********************************************************/ -struct ZSTD_DDict_s { - void* dictBuffer; - const void* dictContent; - size_t dictSize; - ZSTD_entropyDTables_t entropy; - U32 dictID; - U32 entropyPresent; - ZSTD_customMem cMem; -}; /* typedef'd to ZSTD_DDict within "zstd.h" */ - -const void* ZSTD_DDict_dictContent(const ZSTD_DDict* ddict) -{ - assert(ddict != NULL); - return ddict->dictContent; -} - -size_t ZSTD_DDict_dictSize(const ZSTD_DDict* ddict) -{ - assert(ddict != NULL); - return ddict->dictSize; -} - -void ZSTD_copyDDictParameters(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict) -{ - DEBUGLOG(4, "ZSTD_copyDDictParameters"); - assert(dctx != NULL); - assert(ddict != NULL); - dctx->dictID = ddict->dictID; - dctx->prefixStart = ddict->dictContent; - dctx->virtualStart = ddict->dictContent; - dctx->dictEnd = (const BYTE*)ddict->dictContent + ddict->dictSize; - dctx->previousDstEnd = dctx->dictEnd; -#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION - dctx->dictContentBeginForFuzzing = dctx->prefixStart; - dctx->dictContentEndForFuzzing = dctx->previousDstEnd; -#endif - if (ddict->entropyPresent) { - dctx->litEntropy = 1; - dctx->fseEntropy = 1; - dctx->LLTptr = ddict->entropy.LLTable; - dctx->MLTptr = ddict->entropy.MLTable; - dctx->OFTptr = ddict->entropy.OFTable; - dctx->HUFptr = ddict->entropy.hufTable; - dctx->entropy.rep[0] = ddict->entropy.rep[0]; - dctx->entropy.rep[1] = ddict->entropy.rep[1]; - dctx->entropy.rep[2] = ddict->entropy.rep[2]; - } else { - dctx->litEntropy = 0; - dctx->fseEntropy = 0; - } -} - - -static size_t -ZSTD_loadEntropy_intoDDict(ZSTD_DDict* ddict, - ZSTD_dictContentType_e dictContentType) -{ - ddict->dictID = 0; - ddict->entropyPresent = 0; - if (dictContentType == ZSTD_dct_rawContent) return 0; - - if (ddict->dictSize < 8) { - if (dictContentType == ZSTD_dct_fullDict) - return ERROR(dictionary_corrupted); /* only accept specified dictionaries */ - return 0; /* pure content mode */ - } - { U32 const magic = MEM_readLE32(ddict->dictContent); - if (magic != ZSTD_MAGIC_DICTIONARY) { - if (dictContentType == ZSTD_dct_fullDict) - return ERROR(dictionary_corrupted); /* only accept specified dictionaries */ - return 0; /* pure content mode */ - } - } - ddict->dictID = MEM_readLE32((const char*)ddict->dictContent + ZSTD_FRAMEIDSIZE); - - /* load entropy tables */ - RETURN_ERROR_IF(ZSTD_isError(ZSTD_loadDEntropy( - &ddict->entropy, ddict->dictContent, ddict->dictSize)), - dictionary_corrupted, ""); - ddict->entropyPresent = 1; - return 0; -} - - -static size_t ZSTD_initDDict_internal(ZSTD_DDict* ddict, - const void* dict, size_t dictSize, - ZSTD_dictLoadMethod_e dictLoadMethod, - ZSTD_dictContentType_e dictContentType) -{ - if ((dictLoadMethod == ZSTD_dlm_byRef) || (!dict) || (!dictSize)) { - ddict->dictBuffer = NULL; - ddict->dictContent = dict; - if (!dict) dictSize = 0; - } else { - void* const internalBuffer = ZSTD_malloc(dictSize, ddict->cMem); - ddict->dictBuffer = internalBuffer; - ddict->dictContent = internalBuffer; - if (!internalBuffer) return ERROR(memory_allocation); - memcpy(internalBuffer, dict, dictSize); - } - ddict->dictSize = dictSize; - ddict->entropy.hufTable[0] = (HUF_DTable)((HufLog)*0x1000001); /* cover both little and big endian */ - - /* parse dictionary content */ - FORWARD_IF_ERROR( ZSTD_loadEntropy_intoDDict(ddict, dictContentType) , ""); - - return 0; -} - -ZSTD_DDict* ZSTD_createDDict_advanced(const void* dict, size_t dictSize, - ZSTD_dictLoadMethod_e dictLoadMethod, - ZSTD_dictContentType_e dictContentType, - ZSTD_customMem customMem) -{ - if (!customMem.customAlloc ^ !customMem.customFree) return NULL; - - { ZSTD_DDict* const ddict = (ZSTD_DDict*) ZSTD_malloc(sizeof(ZSTD_DDict), customMem); - if (ddict == NULL) return NULL; - ddict->cMem = customMem; - { size_t const initResult = ZSTD_initDDict_internal(ddict, - dict, dictSize, - dictLoadMethod, dictContentType); - if (ZSTD_isError(initResult)) { - ZSTD_freeDDict(ddict); - return NULL; - } } - return ddict; - } -} - -/*! ZSTD_createDDict() : -* Create a digested dictionary, to start decompression without startup delay. -* `dict` content is copied inside DDict. -* Consequently, `dict` can be released after `ZSTD_DDict` creation */ -ZSTD_DDict* ZSTD_createDDict(const void* dict, size_t dictSize) -{ - ZSTD_customMem const allocator = { NULL, NULL, NULL }; - return ZSTD_createDDict_advanced(dict, dictSize, ZSTD_dlm_byCopy, ZSTD_dct_auto, allocator); -} - -/*! ZSTD_createDDict_byReference() : - * Create a digested dictionary, to start decompression without startup delay. - * Dictionary content is simply referenced, it will be accessed during decompression. - * Warning : dictBuffer must outlive DDict (DDict must be freed before dictBuffer) */ -ZSTD_DDict* ZSTD_createDDict_byReference(const void* dictBuffer, size_t dictSize) -{ - ZSTD_customMem const allocator = { NULL, NULL, NULL }; - return ZSTD_createDDict_advanced(dictBuffer, dictSize, ZSTD_dlm_byRef, ZSTD_dct_auto, allocator); -} - - -const ZSTD_DDict* ZSTD_initStaticDDict( - void* sBuffer, size_t sBufferSize, - const void* dict, size_t dictSize, - ZSTD_dictLoadMethod_e dictLoadMethod, - ZSTD_dictContentType_e dictContentType) -{ - size_t const neededSpace = sizeof(ZSTD_DDict) - + (dictLoadMethod == ZSTD_dlm_byRef ? 0 : dictSize); - ZSTD_DDict* const ddict = (ZSTD_DDict*)sBuffer; - assert(sBuffer != NULL); - assert(dict != NULL); - if ((size_t)sBuffer & 7) return NULL; /* 8-aligned */ - if (sBufferSize < neededSpace) return NULL; - if (dictLoadMethod == ZSTD_dlm_byCopy) { - memcpy(ddict+1, dict, dictSize); /* local copy */ - dict = ddict+1; - } - if (ZSTD_isError( ZSTD_initDDict_internal(ddict, - dict, dictSize, - ZSTD_dlm_byRef, dictContentType) )) - return NULL; - return ddict; -} - - -size_t ZSTD_freeDDict(ZSTD_DDict* ddict) -{ - if (ddict==NULL) return 0; /* support free on NULL */ - { ZSTD_customMem const cMem = ddict->cMem; - ZSTD_free(ddict->dictBuffer, cMem); - ZSTD_free(ddict, cMem); - return 0; - } -} - -/*! ZSTD_estimateDDictSize() : - * Estimate amount of memory that will be needed to create a dictionary for decompression. - * Note : dictionary created by reference using ZSTD_dlm_byRef are smaller */ -size_t ZSTD_estimateDDictSize(size_t dictSize, ZSTD_dictLoadMethod_e dictLoadMethod) -{ - return sizeof(ZSTD_DDict) + (dictLoadMethod == ZSTD_dlm_byRef ? 0 : dictSize); -} - -size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict) -{ - if (ddict==NULL) return 0; /* support sizeof on NULL */ - return sizeof(*ddict) + (ddict->dictBuffer ? ddict->dictSize : 0) ; -} - -/*! ZSTD_getDictID_fromDDict() : - * Provides the dictID of the dictionary loaded into `ddict`. - * If @return == 0, the dictionary is not conformant to Zstandard specification, or empty. - * Non-conformant dictionaries can still be loaded, but as content-only dictionaries. */ -unsigned ZSTD_getDictID_fromDDict(const ZSTD_DDict* ddict) -{ - if (ddict==NULL) return 0; - return ZSTD_getDictID_fromDict(ddict->dictContent, ddict->dictSize); -} -/**** ended inlining decompress/zstd_ddict.c ****/ -/**** start inlining decompress/zstd_decompress.c ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - - -/* *************************************************************** -* Tuning parameters -*****************************************************************/ -/*! - * HEAPMODE : - * Select how default decompression function ZSTD_decompress() allocates its context, - * on stack (0), or into heap (1, default; requires malloc()). - * Note that functions with explicit context such as ZSTD_decompressDCtx() are unaffected. - */ -#ifndef ZSTD_HEAPMODE -# define ZSTD_HEAPMODE 1 -#endif - -/*! -* LEGACY_SUPPORT : -* if set to 1+, ZSTD_decompress() can decode older formats (v0.1+) -*/ -#ifndef ZSTD_LEGACY_SUPPORT -# define ZSTD_LEGACY_SUPPORT 0 -#endif - -/*! - * MAXWINDOWSIZE_DEFAULT : - * maximum window size accepted by DStream __by default__. - * Frames requiring more memory will be rejected. - * It's possible to set a different limit using ZSTD_DCtx_setMaxWindowSize(). - */ -#ifndef ZSTD_MAXWINDOWSIZE_DEFAULT -# define ZSTD_MAXWINDOWSIZE_DEFAULT (((U32)1 << ZSTD_WINDOWLOG_LIMIT_DEFAULT) + 1) -#endif - -/*! - * NO_FORWARD_PROGRESS_MAX : - * maximum allowed nb of calls to ZSTD_decompressStream() - * without any forward progress - * (defined as: no byte read from input, and no byte flushed to output) - * before triggering an error. - */ -#ifndef ZSTD_NO_FORWARD_PROGRESS_MAX -# define ZSTD_NO_FORWARD_PROGRESS_MAX 16 -#endif - - -/*-******************************************************* -* Dependencies -*********************************************************/ -#include /* memcpy, memmove, memset */ -/**** skipping file: ../common/cpu.h ****/ -/**** skipping file: ../common/mem.h ****/ -#define FSE_STATIC_LINKING_ONLY -/**** skipping file: ../common/fse.h ****/ -#define HUF_STATIC_LINKING_ONLY -/**** skipping file: ../common/huf.h ****/ -/**** skipping file: ../common/zstd_internal.h ****/ -/**** skipping file: zstd_decompress_internal.h ****/ -/**** skipping file: zstd_ddict.h ****/ -/**** start inlining zstd_decompress_block.h ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - - -#ifndef ZSTD_DEC_BLOCK_H -#define ZSTD_DEC_BLOCK_H - -/*-******************************************************* - * Dependencies - *********************************************************/ -#include /* size_t */ -/**** skipping file: ../zstd.h ****/ -/**** skipping file: ../common/zstd_internal.h ****/ -/**** skipping file: zstd_decompress_internal.h ****/ - - -/* === Prototypes === */ - -/* note: prototypes already published within `zstd.h` : - * ZSTD_decompressBlock() - */ - -/* note: prototypes already published within `zstd_internal.h` : - * ZSTD_getcBlockSize() - * ZSTD_decodeSeqHeaders() - */ - - -/* ZSTD_decompressBlock_internal() : - * decompress block, starting at `src`, - * into destination buffer `dst`. - * @return : decompressed block size, - * or an error code (which can be tested using ZSTD_isError()) - */ -size_t ZSTD_decompressBlock_internal(ZSTD_DCtx* dctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, const int frame); - -/* ZSTD_buildFSETable() : - * generate FSE decoding table for one symbol (ll, ml or off) - * this function must be called with valid parameters only - * (dt is large enough, normalizedCounter distribution total is a power of 2, max is within range, etc.) - * in which case it cannot fail. - * Internal use only. - */ -void ZSTD_buildFSETable(ZSTD_seqSymbol* dt, - const short* normalizedCounter, unsigned maxSymbolValue, - const U32* baseValue, const U32* nbAdditionalBits, - unsigned tableLog); - - -#endif /* ZSTD_DEC_BLOCK_H */ -/**** ended inlining zstd_decompress_block.h ****/ - -#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT>=1) -/**** skipping file: ../legacy/zstd_legacy.h ****/ -#endif - - -/*-************************************************************* -* Context management -***************************************************************/ -size_t ZSTD_sizeof_DCtx (const ZSTD_DCtx* dctx) -{ - if (dctx==NULL) return 0; /* support sizeof NULL */ - return sizeof(*dctx) - + ZSTD_sizeof_DDict(dctx->ddictLocal) - + dctx->inBuffSize + dctx->outBuffSize; -} - -size_t ZSTD_estimateDCtxSize(void) { return sizeof(ZSTD_DCtx); } - - -static size_t ZSTD_startingInputLength(ZSTD_format_e format) -{ - size_t const startingInputLength = ZSTD_FRAMEHEADERSIZE_PREFIX(format); - /* only supports formats ZSTD_f_zstd1 and ZSTD_f_zstd1_magicless */ - assert( (format == ZSTD_f_zstd1) || (format == ZSTD_f_zstd1_magicless) ); - return startingInputLength; -} - -static void ZSTD_initDCtx_internal(ZSTD_DCtx* dctx) -{ - dctx->format = ZSTD_f_zstd1; /* ZSTD_decompressBegin() invokes ZSTD_startingInputLength() with argument dctx->format */ - dctx->staticSize = 0; - dctx->maxWindowSize = ZSTD_MAXWINDOWSIZE_DEFAULT; - dctx->ddict = NULL; - dctx->ddictLocal = NULL; - dctx->dictEnd = NULL; - dctx->ddictIsCold = 0; - dctx->dictUses = ZSTD_dont_use; - dctx->inBuff = NULL; - dctx->inBuffSize = 0; - dctx->outBuffSize = 0; - dctx->streamStage = zdss_init; - dctx->legacyContext = NULL; - dctx->previousLegacyVersion = 0; - dctx->noForwardProgress = 0; - dctx->oversizedDuration = 0; - dctx->bmi2 = ZSTD_cpuid_bmi2(ZSTD_cpuid()); - dctx->outBufferMode = ZSTD_obm_buffered; -#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION - dctx->dictContentEndForFuzzing = NULL; -#endif -} - -ZSTD_DCtx* ZSTD_initStaticDCtx(void *workspace, size_t workspaceSize) -{ - ZSTD_DCtx* const dctx = (ZSTD_DCtx*) workspace; - - if ((size_t)workspace & 7) return NULL; /* 8-aligned */ - if (workspaceSize < sizeof(ZSTD_DCtx)) return NULL; /* minimum size */ - - ZSTD_initDCtx_internal(dctx); - dctx->staticSize = workspaceSize; - dctx->inBuff = (char*)(dctx+1); - return dctx; -} - -ZSTD_DCtx* ZSTD_createDCtx_advanced(ZSTD_customMem customMem) -{ - if (!customMem.customAlloc ^ !customMem.customFree) return NULL; - - { ZSTD_DCtx* const dctx = (ZSTD_DCtx*)ZSTD_malloc(sizeof(*dctx), customMem); - if (!dctx) return NULL; - dctx->customMem = customMem; - ZSTD_initDCtx_internal(dctx); - return dctx; - } -} - -ZSTD_DCtx* ZSTD_createDCtx(void) -{ - DEBUGLOG(3, "ZSTD_createDCtx"); - return ZSTD_createDCtx_advanced(ZSTD_defaultCMem); -} - -static void ZSTD_clearDict(ZSTD_DCtx* dctx) -{ - ZSTD_freeDDict(dctx->ddictLocal); - dctx->ddictLocal = NULL; - dctx->ddict = NULL; - dctx->dictUses = ZSTD_dont_use; -} - -size_t ZSTD_freeDCtx(ZSTD_DCtx* dctx) -{ - if (dctx==NULL) return 0; /* support free on NULL */ - RETURN_ERROR_IF(dctx->staticSize, memory_allocation, "not compatible with static DCtx"); - { ZSTD_customMem const cMem = dctx->customMem; - ZSTD_clearDict(dctx); - ZSTD_free(dctx->inBuff, cMem); - dctx->inBuff = NULL; -#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT >= 1) - if (dctx->legacyContext) - ZSTD_freeLegacyStreamContext(dctx->legacyContext, dctx->previousLegacyVersion); -#endif - ZSTD_free(dctx, cMem); - return 0; - } -} - -/* no longer useful */ -void ZSTD_copyDCtx(ZSTD_DCtx* dstDCtx, const ZSTD_DCtx* srcDCtx) -{ - size_t const toCopy = (size_t)((char*)(&dstDCtx->inBuff) - (char*)dstDCtx); - memcpy(dstDCtx, srcDCtx, toCopy); /* no need to copy workspace */ -} - - -/*-************************************************************* - * Frame header decoding - ***************************************************************/ - -/*! ZSTD_isFrame() : - * Tells if the content of `buffer` starts with a valid Frame Identifier. - * Note : Frame Identifier is 4 bytes. If `size < 4`, @return will always be 0. - * Note 2 : Legacy Frame Identifiers are considered valid only if Legacy Support is enabled. - * Note 3 : Skippable Frame Identifiers are considered valid. */ -unsigned ZSTD_isFrame(const void* buffer, size_t size) -{ - if (size < ZSTD_FRAMEIDSIZE) return 0; - { U32 const magic = MEM_readLE32(buffer); - if (magic == ZSTD_MAGICNUMBER) return 1; - if ((magic & ZSTD_MAGIC_SKIPPABLE_MASK) == ZSTD_MAGIC_SKIPPABLE_START) return 1; - } -#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT >= 1) - if (ZSTD_isLegacy(buffer, size)) return 1; -#endif - return 0; -} - -/** ZSTD_frameHeaderSize_internal() : - * srcSize must be large enough to reach header size fields. - * note : only works for formats ZSTD_f_zstd1 and ZSTD_f_zstd1_magicless. - * @return : size of the Frame Header - * or an error code, which can be tested with ZSTD_isError() */ -static size_t ZSTD_frameHeaderSize_internal(const void* src, size_t srcSize, ZSTD_format_e format) -{ - size_t const minInputSize = ZSTD_startingInputLength(format); - RETURN_ERROR_IF(srcSize < minInputSize, srcSize_wrong, ""); - - { BYTE const fhd = ((const BYTE*)src)[minInputSize-1]; - U32 const dictID= fhd & 3; - U32 const singleSegment = (fhd >> 5) & 1; - U32 const fcsId = fhd >> 6; - return minInputSize + !singleSegment - + ZSTD_did_fieldSize[dictID] + ZSTD_fcs_fieldSize[fcsId] - + (singleSegment && !fcsId); - } -} - -/** ZSTD_frameHeaderSize() : - * srcSize must be >= ZSTD_frameHeaderSize_prefix. - * @return : size of the Frame Header, - * or an error code (if srcSize is too small) */ -size_t ZSTD_frameHeaderSize(const void* src, size_t srcSize) -{ - return ZSTD_frameHeaderSize_internal(src, srcSize, ZSTD_f_zstd1); -} - - -/** ZSTD_getFrameHeader_advanced() : - * decode Frame Header, or require larger `srcSize`. - * note : only works for formats ZSTD_f_zstd1 and ZSTD_f_zstd1_magicless - * @return : 0, `zfhPtr` is correctly filled, - * >0, `srcSize` is too small, value is wanted `srcSize` amount, - * or an error code, which can be tested using ZSTD_isError() */ -size_t ZSTD_getFrameHeader_advanced(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize, ZSTD_format_e format) -{ - const BYTE* ip = (const BYTE*)src; - size_t const minInputSize = ZSTD_startingInputLength(format); - - memset(zfhPtr, 0, sizeof(*zfhPtr)); /* not strictly necessary, but static analyzer do not understand that zfhPtr is only going to be read only if return value is zero, since they are 2 different signals */ - if (srcSize < minInputSize) return minInputSize; - RETURN_ERROR_IF(src==NULL, GENERIC, "invalid parameter"); - - if ( (format != ZSTD_f_zstd1_magicless) - && (MEM_readLE32(src) != ZSTD_MAGICNUMBER) ) { - if ((MEM_readLE32(src) & ZSTD_MAGIC_SKIPPABLE_MASK) == ZSTD_MAGIC_SKIPPABLE_START) { - /* skippable frame */ - if (srcSize < ZSTD_SKIPPABLEHEADERSIZE) - return ZSTD_SKIPPABLEHEADERSIZE; /* magic number + frame length */ - memset(zfhPtr, 0, sizeof(*zfhPtr)); - zfhPtr->frameContentSize = MEM_readLE32((const char *)src + ZSTD_FRAMEIDSIZE); - zfhPtr->frameType = ZSTD_skippableFrame; - return 0; - } - RETURN_ERROR(prefix_unknown, ""); - } - - /* ensure there is enough `srcSize` to fully read/decode frame header */ - { size_t const fhsize = ZSTD_frameHeaderSize_internal(src, srcSize, format); - if (srcSize < fhsize) return fhsize; - zfhPtr->headerSize = (U32)fhsize; - } - - { BYTE const fhdByte = ip[minInputSize-1]; - size_t pos = minInputSize; - U32 const dictIDSizeCode = fhdByte&3; - U32 const checksumFlag = (fhdByte>>2)&1; - U32 const singleSegment = (fhdByte>>5)&1; - U32 const fcsID = fhdByte>>6; - U64 windowSize = 0; - U32 dictID = 0; - U64 frameContentSize = ZSTD_CONTENTSIZE_UNKNOWN; - RETURN_ERROR_IF((fhdByte & 0x08) != 0, frameParameter_unsupported, - "reserved bits, must be zero"); - - if (!singleSegment) { - BYTE const wlByte = ip[pos++]; - U32 const windowLog = (wlByte >> 3) + ZSTD_WINDOWLOG_ABSOLUTEMIN; - RETURN_ERROR_IF(windowLog > ZSTD_WINDOWLOG_MAX, frameParameter_windowTooLarge, ""); - windowSize = (1ULL << windowLog); - windowSize += (windowSize >> 3) * (wlByte&7); - } - switch(dictIDSizeCode) - { - default: assert(0); /* impossible */ - case 0 : break; - case 1 : dictID = ip[pos]; pos++; break; - case 2 : dictID = MEM_readLE16(ip+pos); pos+=2; break; - case 3 : dictID = MEM_readLE32(ip+pos); pos+=4; break; - } - switch(fcsID) - { - default: assert(0); /* impossible */ - case 0 : if (singleSegment) frameContentSize = ip[pos]; break; - case 1 : frameContentSize = MEM_readLE16(ip+pos)+256; break; - case 2 : frameContentSize = MEM_readLE32(ip+pos); break; - case 3 : frameContentSize = MEM_readLE64(ip+pos); break; - } - if (singleSegment) windowSize = frameContentSize; - - zfhPtr->frameType = ZSTD_frame; - zfhPtr->frameContentSize = frameContentSize; - zfhPtr->windowSize = windowSize; - zfhPtr->blockSizeMax = (unsigned) MIN(windowSize, ZSTD_BLOCKSIZE_MAX); - zfhPtr->dictID = dictID; - zfhPtr->checksumFlag = checksumFlag; - } - return 0; -} - -/** ZSTD_getFrameHeader() : - * decode Frame Header, or require larger `srcSize`. - * note : this function does not consume input, it only reads it. - * @return : 0, `zfhPtr` is correctly filled, - * >0, `srcSize` is too small, value is wanted `srcSize` amount, - * or an error code, which can be tested using ZSTD_isError() */ -size_t ZSTD_getFrameHeader(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize) -{ - return ZSTD_getFrameHeader_advanced(zfhPtr, src, srcSize, ZSTD_f_zstd1); -} - - -/** ZSTD_getFrameContentSize() : - * compatible with legacy mode - * @return : decompressed size of the single frame pointed to be `src` if known, otherwise - * - ZSTD_CONTENTSIZE_UNKNOWN if the size cannot be determined - * - ZSTD_CONTENTSIZE_ERROR if an error occurred (e.g. invalid magic number, srcSize too small) */ -unsigned long long ZSTD_getFrameContentSize(const void *src, size_t srcSize) -{ -#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT >= 1) - if (ZSTD_isLegacy(src, srcSize)) { - unsigned long long const ret = ZSTD_getDecompressedSize_legacy(src, srcSize); - return ret == 0 ? ZSTD_CONTENTSIZE_UNKNOWN : ret; - } -#endif - { ZSTD_frameHeader zfh; - if (ZSTD_getFrameHeader(&zfh, src, srcSize) != 0) - return ZSTD_CONTENTSIZE_ERROR; - if (zfh.frameType == ZSTD_skippableFrame) { - return 0; - } else { - return zfh.frameContentSize; - } } -} - -static size_t readSkippableFrameSize(void const* src, size_t srcSize) -{ - size_t const skippableHeaderSize = ZSTD_SKIPPABLEHEADERSIZE; - U32 sizeU32; - - RETURN_ERROR_IF(srcSize < ZSTD_SKIPPABLEHEADERSIZE, srcSize_wrong, ""); - - sizeU32 = MEM_readLE32((BYTE const*)src + ZSTD_FRAMEIDSIZE); - RETURN_ERROR_IF((U32)(sizeU32 + ZSTD_SKIPPABLEHEADERSIZE) < sizeU32, - frameParameter_unsupported, ""); - { - size_t const skippableSize = skippableHeaderSize + sizeU32; - RETURN_ERROR_IF(skippableSize > srcSize, srcSize_wrong, ""); - return skippableSize; - } -} - -/** ZSTD_findDecompressedSize() : - * compatible with legacy mode - * `srcSize` must be the exact length of some number of ZSTD compressed and/or - * skippable frames - * @return : decompressed size of the frames contained */ -unsigned long long ZSTD_findDecompressedSize(const void* src, size_t srcSize) -{ - unsigned long long totalDstSize = 0; - - while (srcSize >= ZSTD_startingInputLength(ZSTD_f_zstd1)) { - U32 const magicNumber = MEM_readLE32(src); - - if ((magicNumber & ZSTD_MAGIC_SKIPPABLE_MASK) == ZSTD_MAGIC_SKIPPABLE_START) { - size_t const skippableSize = readSkippableFrameSize(src, srcSize); - if (ZSTD_isError(skippableSize)) { - return ZSTD_CONTENTSIZE_ERROR; - } - assert(skippableSize <= srcSize); - - src = (const BYTE *)src + skippableSize; - srcSize -= skippableSize; - continue; - } - - { unsigned long long const ret = ZSTD_getFrameContentSize(src, srcSize); - if (ret >= ZSTD_CONTENTSIZE_ERROR) return ret; - - /* check for overflow */ - if (totalDstSize + ret < totalDstSize) return ZSTD_CONTENTSIZE_ERROR; - totalDstSize += ret; - } - { size_t const frameSrcSize = ZSTD_findFrameCompressedSize(src, srcSize); - if (ZSTD_isError(frameSrcSize)) { - return ZSTD_CONTENTSIZE_ERROR; - } - - src = (const BYTE *)src + frameSrcSize; - srcSize -= frameSrcSize; - } - } /* while (srcSize >= ZSTD_frameHeaderSize_prefix) */ - - if (srcSize) return ZSTD_CONTENTSIZE_ERROR; - - return totalDstSize; -} - -/** ZSTD_getDecompressedSize() : - * compatible with legacy mode - * @return : decompressed size if known, 0 otherwise - note : 0 can mean any of the following : - - frame content is empty - - decompressed size field is not present in frame header - - frame header unknown / not supported - - frame header not complete (`srcSize` too small) */ -unsigned long long ZSTD_getDecompressedSize(const void* src, size_t srcSize) -{ - unsigned long long const ret = ZSTD_getFrameContentSize(src, srcSize); - ZSTD_STATIC_ASSERT(ZSTD_CONTENTSIZE_ERROR < ZSTD_CONTENTSIZE_UNKNOWN); - return (ret >= ZSTD_CONTENTSIZE_ERROR) ? 0 : ret; -} - - -/** ZSTD_decodeFrameHeader() : - * `headerSize` must be the size provided by ZSTD_frameHeaderSize(). - * @return : 0 if success, or an error code, which can be tested using ZSTD_isError() */ -static size_t ZSTD_decodeFrameHeader(ZSTD_DCtx* dctx, const void* src, size_t headerSize) -{ - size_t const result = ZSTD_getFrameHeader_advanced(&(dctx->fParams), src, headerSize, dctx->format); - if (ZSTD_isError(result)) return result; /* invalid header */ - RETURN_ERROR_IF(result>0, srcSize_wrong, "headerSize too small"); -#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION - /* Skip the dictID check in fuzzing mode, because it makes the search - * harder. - */ - RETURN_ERROR_IF(dctx->fParams.dictID && (dctx->dictID != dctx->fParams.dictID), - dictionary_wrong, ""); -#endif - if (dctx->fParams.checksumFlag) XXH64_reset(&dctx->xxhState, 0); - return 0; -} - -static ZSTD_frameSizeInfo ZSTD_errorFrameSizeInfo(size_t ret) -{ - ZSTD_frameSizeInfo frameSizeInfo; - frameSizeInfo.compressedSize = ret; - frameSizeInfo.decompressedBound = ZSTD_CONTENTSIZE_ERROR; - return frameSizeInfo; -} - -static ZSTD_frameSizeInfo ZSTD_findFrameSizeInfo(const void* src, size_t srcSize) -{ - ZSTD_frameSizeInfo frameSizeInfo; - memset(&frameSizeInfo, 0, sizeof(ZSTD_frameSizeInfo)); - -#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT >= 1) - if (ZSTD_isLegacy(src, srcSize)) - return ZSTD_findFrameSizeInfoLegacy(src, srcSize); -#endif - - if ((srcSize >= ZSTD_SKIPPABLEHEADERSIZE) - && (MEM_readLE32(src) & ZSTD_MAGIC_SKIPPABLE_MASK) == ZSTD_MAGIC_SKIPPABLE_START) { - frameSizeInfo.compressedSize = readSkippableFrameSize(src, srcSize); - assert(ZSTD_isError(frameSizeInfo.compressedSize) || - frameSizeInfo.compressedSize <= srcSize); - return frameSizeInfo; - } else { - const BYTE* ip = (const BYTE*)src; - const BYTE* const ipstart = ip; - size_t remainingSize = srcSize; - size_t nbBlocks = 0; - ZSTD_frameHeader zfh; - - /* Extract Frame Header */ - { size_t const ret = ZSTD_getFrameHeader(&zfh, src, srcSize); - if (ZSTD_isError(ret)) - return ZSTD_errorFrameSizeInfo(ret); - if (ret > 0) - return ZSTD_errorFrameSizeInfo(ERROR(srcSize_wrong)); - } - - ip += zfh.headerSize; - remainingSize -= zfh.headerSize; - - /* Iterate over each block */ - while (1) { - blockProperties_t blockProperties; - size_t const cBlockSize = ZSTD_getcBlockSize(ip, remainingSize, &blockProperties); - if (ZSTD_isError(cBlockSize)) - return ZSTD_errorFrameSizeInfo(cBlockSize); - - if (ZSTD_blockHeaderSize + cBlockSize > remainingSize) - return ZSTD_errorFrameSizeInfo(ERROR(srcSize_wrong)); - - ip += ZSTD_blockHeaderSize + cBlockSize; - remainingSize -= ZSTD_blockHeaderSize + cBlockSize; - nbBlocks++; - - if (blockProperties.lastBlock) break; - } - - /* Final frame content checksum */ - if (zfh.checksumFlag) { - if (remainingSize < 4) - return ZSTD_errorFrameSizeInfo(ERROR(srcSize_wrong)); - ip += 4; - } - - frameSizeInfo.compressedSize = ip - ipstart; - frameSizeInfo.decompressedBound = (zfh.frameContentSize != ZSTD_CONTENTSIZE_UNKNOWN) - ? zfh.frameContentSize - : nbBlocks * zfh.blockSizeMax; - return frameSizeInfo; - } -} - -/** ZSTD_findFrameCompressedSize() : - * compatible with legacy mode - * `src` must point to the start of a ZSTD frame, ZSTD legacy frame, or skippable frame - * `srcSize` must be at least as large as the frame contained - * @return : the compressed size of the frame starting at `src` */ -size_t ZSTD_findFrameCompressedSize(const void *src, size_t srcSize) -{ - ZSTD_frameSizeInfo const frameSizeInfo = ZSTD_findFrameSizeInfo(src, srcSize); - return frameSizeInfo.compressedSize; -} - -/** ZSTD_decompressBound() : - * compatible with legacy mode - * `src` must point to the start of a ZSTD frame or a skippeable frame - * `srcSize` must be at least as large as the frame contained - * @return : the maximum decompressed size of the compressed source - */ -unsigned long long ZSTD_decompressBound(const void* src, size_t srcSize) -{ - unsigned long long bound = 0; - /* Iterate over each frame */ - while (srcSize > 0) { - ZSTD_frameSizeInfo const frameSizeInfo = ZSTD_findFrameSizeInfo(src, srcSize); - size_t const compressedSize = frameSizeInfo.compressedSize; - unsigned long long const decompressedBound = frameSizeInfo.decompressedBound; - if (ZSTD_isError(compressedSize) || decompressedBound == ZSTD_CONTENTSIZE_ERROR) - return ZSTD_CONTENTSIZE_ERROR; - assert(srcSize >= compressedSize); - src = (const BYTE*)src + compressedSize; - srcSize -= compressedSize; - bound += decompressedBound; - } - return bound; -} - - -/*-************************************************************* - * Frame decoding - ***************************************************************/ - -/** ZSTD_insertBlock() : - * insert `src` block into `dctx` history. Useful to track uncompressed blocks. */ -size_t ZSTD_insertBlock(ZSTD_DCtx* dctx, const void* blockStart, size_t blockSize) -{ - DEBUGLOG(5, "ZSTD_insertBlock: %u bytes", (unsigned)blockSize); - ZSTD_checkContinuity(dctx, blockStart); - dctx->previousDstEnd = (const char*)blockStart + blockSize; - return blockSize; -} - - -static size_t ZSTD_copyRawBlock(void* dst, size_t dstCapacity, - const void* src, size_t srcSize) -{ - DEBUGLOG(5, "ZSTD_copyRawBlock"); - if (dst == NULL) { - if (srcSize == 0) return 0; - RETURN_ERROR(dstBuffer_null, ""); - } - RETURN_ERROR_IF(srcSize > dstCapacity, dstSize_tooSmall, ""); - memcpy(dst, src, srcSize); - return srcSize; -} - -static size_t ZSTD_setRleBlock(void* dst, size_t dstCapacity, - BYTE b, - size_t regenSize) -{ - if (dst == NULL) { - if (regenSize == 0) return 0; - RETURN_ERROR(dstBuffer_null, ""); - } - RETURN_ERROR_IF(regenSize > dstCapacity, dstSize_tooSmall, ""); - memset(dst, b, regenSize); - return regenSize; -} - - -/*! ZSTD_decompressFrame() : - * @dctx must be properly initialized - * will update *srcPtr and *srcSizePtr, - * to make *srcPtr progress by one frame. */ -static size_t ZSTD_decompressFrame(ZSTD_DCtx* dctx, - void* dst, size_t dstCapacity, - const void** srcPtr, size_t *srcSizePtr) -{ - const BYTE* ip = (const BYTE*)(*srcPtr); - BYTE* const ostart = (BYTE* const)dst; - BYTE* const oend = dstCapacity != 0 ? ostart + dstCapacity : ostart; - BYTE* op = ostart; - size_t remainingSrcSize = *srcSizePtr; - - DEBUGLOG(4, "ZSTD_decompressFrame (srcSize:%i)", (int)*srcSizePtr); - - /* check */ - RETURN_ERROR_IF( - remainingSrcSize < ZSTD_FRAMEHEADERSIZE_MIN(dctx->format)+ZSTD_blockHeaderSize, - srcSize_wrong, ""); - - /* Frame Header */ - { size_t const frameHeaderSize = ZSTD_frameHeaderSize_internal( - ip, ZSTD_FRAMEHEADERSIZE_PREFIX(dctx->format), dctx->format); - if (ZSTD_isError(frameHeaderSize)) return frameHeaderSize; - RETURN_ERROR_IF(remainingSrcSize < frameHeaderSize+ZSTD_blockHeaderSize, - srcSize_wrong, ""); - FORWARD_IF_ERROR( ZSTD_decodeFrameHeader(dctx, ip, frameHeaderSize) , ""); - ip += frameHeaderSize; remainingSrcSize -= frameHeaderSize; - } - - /* Loop on each block */ - while (1) { - size_t decodedSize; - blockProperties_t blockProperties; - size_t const cBlockSize = ZSTD_getcBlockSize(ip, remainingSrcSize, &blockProperties); - if (ZSTD_isError(cBlockSize)) return cBlockSize; - - ip += ZSTD_blockHeaderSize; - remainingSrcSize -= ZSTD_blockHeaderSize; - RETURN_ERROR_IF(cBlockSize > remainingSrcSize, srcSize_wrong, ""); - - switch(blockProperties.blockType) - { - case bt_compressed: - decodedSize = ZSTD_decompressBlock_internal(dctx, op, oend-op, ip, cBlockSize, /* frame */ 1); - break; - case bt_raw : - decodedSize = ZSTD_copyRawBlock(op, oend-op, ip, cBlockSize); - break; - case bt_rle : - decodedSize = ZSTD_setRleBlock(op, oend-op, *ip, blockProperties.origSize); - break; - case bt_reserved : - default: - RETURN_ERROR(corruption_detected, "invalid block type"); - } - - if (ZSTD_isError(decodedSize)) return decodedSize; - if (dctx->fParams.checksumFlag) - XXH64_update(&dctx->xxhState, op, decodedSize); - if (decodedSize != 0) - op += decodedSize; - assert(ip != NULL); - ip += cBlockSize; - remainingSrcSize -= cBlockSize; - if (blockProperties.lastBlock) break; - } - - if (dctx->fParams.frameContentSize != ZSTD_CONTENTSIZE_UNKNOWN) { - RETURN_ERROR_IF((U64)(op-ostart) != dctx->fParams.frameContentSize, - corruption_detected, ""); - } - if (dctx->fParams.checksumFlag) { /* Frame content checksum verification */ - U32 const checkCalc = (U32)XXH64_digest(&dctx->xxhState); - U32 checkRead; - RETURN_ERROR_IF(remainingSrcSize<4, checksum_wrong, ""); - checkRead = MEM_readLE32(ip); - RETURN_ERROR_IF(checkRead != checkCalc, checksum_wrong, ""); - ip += 4; - remainingSrcSize -= 4; - } - - /* Allow caller to get size read */ - *srcPtr = ip; - *srcSizePtr = remainingSrcSize; - return op-ostart; -} - -static size_t ZSTD_decompressMultiFrame(ZSTD_DCtx* dctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const void* dict, size_t dictSize, - const ZSTD_DDict* ddict) -{ - void* const dststart = dst; - int moreThan1Frame = 0; - - DEBUGLOG(5, "ZSTD_decompressMultiFrame"); - assert(dict==NULL || ddict==NULL); /* either dict or ddict set, not both */ - - if (ddict) { - dict = ZSTD_DDict_dictContent(ddict); - dictSize = ZSTD_DDict_dictSize(ddict); - } - - while (srcSize >= ZSTD_startingInputLength(dctx->format)) { - -#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT >= 1) - if (ZSTD_isLegacy(src, srcSize)) { - size_t decodedSize; - size_t const frameSize = ZSTD_findFrameCompressedSizeLegacy(src, srcSize); - if (ZSTD_isError(frameSize)) return frameSize; - RETURN_ERROR_IF(dctx->staticSize, memory_allocation, - "legacy support is not compatible with static dctx"); - - decodedSize = ZSTD_decompressLegacy(dst, dstCapacity, src, frameSize, dict, dictSize); - if (ZSTD_isError(decodedSize)) return decodedSize; - - assert(decodedSize <=- dstCapacity); - dst = (BYTE*)dst + decodedSize; - dstCapacity -= decodedSize; - - src = (const BYTE*)src + frameSize; - srcSize -= frameSize; - - continue; - } -#endif - - { U32 const magicNumber = MEM_readLE32(src); - DEBUGLOG(4, "reading magic number %08X (expecting %08X)", - (unsigned)magicNumber, ZSTD_MAGICNUMBER); - if ((magicNumber & ZSTD_MAGIC_SKIPPABLE_MASK) == ZSTD_MAGIC_SKIPPABLE_START) { - size_t const skippableSize = readSkippableFrameSize(src, srcSize); - FORWARD_IF_ERROR(skippableSize, "readSkippableFrameSize failed"); - assert(skippableSize <= srcSize); - - src = (const BYTE *)src + skippableSize; - srcSize -= skippableSize; - continue; - } } - - if (ddict) { - /* we were called from ZSTD_decompress_usingDDict */ - FORWARD_IF_ERROR(ZSTD_decompressBegin_usingDDict(dctx, ddict), ""); - } else { - /* this will initialize correctly with no dict if dict == NULL, so - * use this in all cases but ddict */ - FORWARD_IF_ERROR(ZSTD_decompressBegin_usingDict(dctx, dict, dictSize), ""); - } - ZSTD_checkContinuity(dctx, dst); - - { const size_t res = ZSTD_decompressFrame(dctx, dst, dstCapacity, - &src, &srcSize); - RETURN_ERROR_IF( - (ZSTD_getErrorCode(res) == ZSTD_error_prefix_unknown) - && (moreThan1Frame==1), - srcSize_wrong, - "at least one frame successfully completed, but following " - "bytes are garbage: it's more likely to be a srcSize error, " - "specifying more bytes than compressed size of frame(s). This " - "error message replaces ERROR(prefix_unknown), which would be " - "confusing, as the first header is actually correct. Note that " - "one could be unlucky, it might be a corruption error instead, " - "happening right at the place where we expect zstd magic " - "bytes. But this is _much_ less likely than a srcSize field " - "error."); - if (ZSTD_isError(res)) return res; - assert(res <= dstCapacity); - if (res != 0) - dst = (BYTE*)dst + res; - dstCapacity -= res; - } - moreThan1Frame = 1; - } /* while (srcSize >= ZSTD_frameHeaderSize_prefix) */ - - RETURN_ERROR_IF(srcSize, srcSize_wrong, "input not entirely consumed"); - - return (BYTE*)dst - (BYTE*)dststart; -} - -size_t ZSTD_decompress_usingDict(ZSTD_DCtx* dctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const void* dict, size_t dictSize) -{ - return ZSTD_decompressMultiFrame(dctx, dst, dstCapacity, src, srcSize, dict, dictSize, NULL); -} - - -static ZSTD_DDict const* ZSTD_getDDict(ZSTD_DCtx* dctx) -{ - switch (dctx->dictUses) { - default: - assert(0 /* Impossible */); - /* fall-through */ - case ZSTD_dont_use: - ZSTD_clearDict(dctx); - return NULL; - case ZSTD_use_indefinitely: - return dctx->ddict; - case ZSTD_use_once: - dctx->dictUses = ZSTD_dont_use; - return dctx->ddict; - } -} - -size_t ZSTD_decompressDCtx(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize) -{ - return ZSTD_decompress_usingDDict(dctx, dst, dstCapacity, src, srcSize, ZSTD_getDDict(dctx)); -} - - -size_t ZSTD_decompress(void* dst, size_t dstCapacity, const void* src, size_t srcSize) -{ -#if defined(ZSTD_HEAPMODE) && (ZSTD_HEAPMODE>=1) - size_t regenSize; - ZSTD_DCtx* const dctx = ZSTD_createDCtx(); - RETURN_ERROR_IF(dctx==NULL, memory_allocation, "NULL pointer!"); - regenSize = ZSTD_decompressDCtx(dctx, dst, dstCapacity, src, srcSize); - ZSTD_freeDCtx(dctx); - return regenSize; -#else /* stack mode */ - ZSTD_DCtx dctx; - ZSTD_initDCtx_internal(&dctx); - return ZSTD_decompressDCtx(&dctx, dst, dstCapacity, src, srcSize); -#endif -} - - -/*-************************************** -* Advanced Streaming Decompression API -* Bufferless and synchronous -****************************************/ -size_t ZSTD_nextSrcSizeToDecompress(ZSTD_DCtx* dctx) { return dctx->expected; } - -/** - * Similar to ZSTD_nextSrcSizeToDecompress(), but when when a block input can be streamed, - * we allow taking a partial block as the input. Currently only raw uncompressed blocks can - * be streamed. - * - * For blocks that can be streamed, this allows us to reduce the latency until we produce - * output, and avoid copying the input. - * - * @param inputSize - The total amount of input that the caller currently has. - */ -static size_t ZSTD_nextSrcSizeToDecompressWithInputSize(ZSTD_DCtx* dctx, size_t inputSize) { - if (!(dctx->stage == ZSTDds_decompressBlock || dctx->stage == ZSTDds_decompressLastBlock)) - return dctx->expected; - if (dctx->bType != bt_raw) - return dctx->expected; - return MIN(MAX(inputSize, 1), dctx->expected); -} - -ZSTD_nextInputType_e ZSTD_nextInputType(ZSTD_DCtx* dctx) { - switch(dctx->stage) - { - default: /* should not happen */ - assert(0); - case ZSTDds_getFrameHeaderSize: - case ZSTDds_decodeFrameHeader: - return ZSTDnit_frameHeader; - case ZSTDds_decodeBlockHeader: - return ZSTDnit_blockHeader; - case ZSTDds_decompressBlock: - return ZSTDnit_block; - case ZSTDds_decompressLastBlock: - return ZSTDnit_lastBlock; - case ZSTDds_checkChecksum: - return ZSTDnit_checksum; - case ZSTDds_decodeSkippableHeader: - case ZSTDds_skipFrame: - return ZSTDnit_skippableFrame; - } -} - -static int ZSTD_isSkipFrame(ZSTD_DCtx* dctx) { return dctx->stage == ZSTDds_skipFrame; } - -/** ZSTD_decompressContinue() : - * srcSize : must be the exact nb of bytes expected (see ZSTD_nextSrcSizeToDecompress()) - * @return : nb of bytes generated into `dst` (necessarily <= `dstCapacity) - * or an error code, which can be tested using ZSTD_isError() */ -size_t ZSTD_decompressContinue(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize) -{ - DEBUGLOG(5, "ZSTD_decompressContinue (srcSize:%u)", (unsigned)srcSize); - /* Sanity check */ - RETURN_ERROR_IF(srcSize != ZSTD_nextSrcSizeToDecompressWithInputSize(dctx, srcSize), srcSize_wrong, "not allowed"); - if (dstCapacity) ZSTD_checkContinuity(dctx, dst); - - switch (dctx->stage) - { - case ZSTDds_getFrameHeaderSize : - assert(src != NULL); - if (dctx->format == ZSTD_f_zstd1) { /* allows header */ - assert(srcSize >= ZSTD_FRAMEIDSIZE); /* to read skippable magic number */ - if ((MEM_readLE32(src) & ZSTD_MAGIC_SKIPPABLE_MASK) == ZSTD_MAGIC_SKIPPABLE_START) { /* skippable frame */ - memcpy(dctx->headerBuffer, src, srcSize); - dctx->expected = ZSTD_SKIPPABLEHEADERSIZE - srcSize; /* remaining to load to get full skippable frame header */ - dctx->stage = ZSTDds_decodeSkippableHeader; - return 0; - } } - dctx->headerSize = ZSTD_frameHeaderSize_internal(src, srcSize, dctx->format); - if (ZSTD_isError(dctx->headerSize)) return dctx->headerSize; - memcpy(dctx->headerBuffer, src, srcSize); - dctx->expected = dctx->headerSize - srcSize; - dctx->stage = ZSTDds_decodeFrameHeader; - return 0; - - case ZSTDds_decodeFrameHeader: - assert(src != NULL); - memcpy(dctx->headerBuffer + (dctx->headerSize - srcSize), src, srcSize); - FORWARD_IF_ERROR(ZSTD_decodeFrameHeader(dctx, dctx->headerBuffer, dctx->headerSize), ""); - dctx->expected = ZSTD_blockHeaderSize; - dctx->stage = ZSTDds_decodeBlockHeader; - return 0; - - case ZSTDds_decodeBlockHeader: - { blockProperties_t bp; - size_t const cBlockSize = ZSTD_getcBlockSize(src, ZSTD_blockHeaderSize, &bp); - if (ZSTD_isError(cBlockSize)) return cBlockSize; - RETURN_ERROR_IF(cBlockSize > dctx->fParams.blockSizeMax, corruption_detected, "Block Size Exceeds Maximum"); - dctx->expected = cBlockSize; - dctx->bType = bp.blockType; - dctx->rleSize = bp.origSize; - if (cBlockSize) { - dctx->stage = bp.lastBlock ? ZSTDds_decompressLastBlock : ZSTDds_decompressBlock; - return 0; - } - /* empty block */ - if (bp.lastBlock) { - if (dctx->fParams.checksumFlag) { - dctx->expected = 4; - dctx->stage = ZSTDds_checkChecksum; - } else { - dctx->expected = 0; /* end of frame */ - dctx->stage = ZSTDds_getFrameHeaderSize; - } - } else { - dctx->expected = ZSTD_blockHeaderSize; /* jump to next header */ - dctx->stage = ZSTDds_decodeBlockHeader; - } - return 0; - } - - case ZSTDds_decompressLastBlock: - case ZSTDds_decompressBlock: - DEBUGLOG(5, "ZSTD_decompressContinue: case ZSTDds_decompressBlock"); - { size_t rSize; - switch(dctx->bType) - { - case bt_compressed: - DEBUGLOG(5, "ZSTD_decompressContinue: case bt_compressed"); - rSize = ZSTD_decompressBlock_internal(dctx, dst, dstCapacity, src, srcSize, /* frame */ 1); - dctx->expected = 0; /* Streaming not supported */ - break; - case bt_raw : - assert(srcSize <= dctx->expected); - rSize = ZSTD_copyRawBlock(dst, dstCapacity, src, srcSize); - FORWARD_IF_ERROR(rSize, "ZSTD_copyRawBlock failed"); - assert(rSize == srcSize); - dctx->expected -= rSize; - break; - case bt_rle : - rSize = ZSTD_setRleBlock(dst, dstCapacity, *(const BYTE*)src, dctx->rleSize); - dctx->expected = 0; /* Streaming not supported */ - break; - case bt_reserved : /* should never happen */ - default: - RETURN_ERROR(corruption_detected, "invalid block type"); - } - FORWARD_IF_ERROR(rSize, ""); - RETURN_ERROR_IF(rSize > dctx->fParams.blockSizeMax, corruption_detected, "Decompressed Block Size Exceeds Maximum"); - DEBUGLOG(5, "ZSTD_decompressContinue: decoded size from block : %u", (unsigned)rSize); - dctx->decodedSize += rSize; - if (dctx->fParams.checksumFlag) XXH64_update(&dctx->xxhState, dst, rSize); - dctx->previousDstEnd = (char*)dst + rSize; - - /* Stay on the same stage until we are finished streaming the block. */ - if (dctx->expected > 0) { - return rSize; - } - - if (dctx->stage == ZSTDds_decompressLastBlock) { /* end of frame */ - DEBUGLOG(4, "ZSTD_decompressContinue: decoded size from frame : %u", (unsigned)dctx->decodedSize); - RETURN_ERROR_IF( - dctx->fParams.frameContentSize != ZSTD_CONTENTSIZE_UNKNOWN - && dctx->decodedSize != dctx->fParams.frameContentSize, - corruption_detected, ""); - if (dctx->fParams.checksumFlag) { /* another round for frame checksum */ - dctx->expected = 4; - dctx->stage = ZSTDds_checkChecksum; - } else { - dctx->expected = 0; /* ends here */ - dctx->stage = ZSTDds_getFrameHeaderSize; - } - } else { - dctx->stage = ZSTDds_decodeBlockHeader; - dctx->expected = ZSTD_blockHeaderSize; - } - return rSize; - } - - case ZSTDds_checkChecksum: - assert(srcSize == 4); /* guaranteed by dctx->expected */ - { U32 const h32 = (U32)XXH64_digest(&dctx->xxhState); - U32 const check32 = MEM_readLE32(src); - DEBUGLOG(4, "ZSTD_decompressContinue: checksum : calculated %08X :: %08X read", (unsigned)h32, (unsigned)check32); - RETURN_ERROR_IF(check32 != h32, checksum_wrong, ""); - dctx->expected = 0; - dctx->stage = ZSTDds_getFrameHeaderSize; - return 0; - } - - case ZSTDds_decodeSkippableHeader: - assert(src != NULL); - assert(srcSize <= ZSTD_SKIPPABLEHEADERSIZE); - memcpy(dctx->headerBuffer + (ZSTD_SKIPPABLEHEADERSIZE - srcSize), src, srcSize); /* complete skippable header */ - dctx->expected = MEM_readLE32(dctx->headerBuffer + ZSTD_FRAMEIDSIZE); /* note : dctx->expected can grow seriously large, beyond local buffer size */ - dctx->stage = ZSTDds_skipFrame; - return 0; - - case ZSTDds_skipFrame: - dctx->expected = 0; - dctx->stage = ZSTDds_getFrameHeaderSize; - return 0; - - default: - assert(0); /* impossible */ - RETURN_ERROR(GENERIC, "impossible to reach"); /* some compiler require default to do something */ - } -} - - -static size_t ZSTD_refDictContent(ZSTD_DCtx* dctx, const void* dict, size_t dictSize) -{ - dctx->dictEnd = dctx->previousDstEnd; - dctx->virtualStart = (const char*)dict - ((const char*)(dctx->previousDstEnd) - (const char*)(dctx->prefixStart)); - dctx->prefixStart = dict; - dctx->previousDstEnd = (const char*)dict + dictSize; -#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION - dctx->dictContentBeginForFuzzing = dctx->prefixStart; - dctx->dictContentEndForFuzzing = dctx->previousDstEnd; -#endif - return 0; -} - -/*! ZSTD_loadDEntropy() : - * dict : must point at beginning of a valid zstd dictionary. - * @return : size of entropy tables read */ -size_t -ZSTD_loadDEntropy(ZSTD_entropyDTables_t* entropy, - const void* const dict, size_t const dictSize) -{ - const BYTE* dictPtr = (const BYTE*)dict; - const BYTE* const dictEnd = dictPtr + dictSize; - - RETURN_ERROR_IF(dictSize <= 8, dictionary_corrupted, "dict is too small"); - assert(MEM_readLE32(dict) == ZSTD_MAGIC_DICTIONARY); /* dict must be valid */ - dictPtr += 8; /* skip header = magic + dictID */ - - ZSTD_STATIC_ASSERT(offsetof(ZSTD_entropyDTables_t, OFTable) == offsetof(ZSTD_entropyDTables_t, LLTable) + sizeof(entropy->LLTable)); - ZSTD_STATIC_ASSERT(offsetof(ZSTD_entropyDTables_t, MLTable) == offsetof(ZSTD_entropyDTables_t, OFTable) + sizeof(entropy->OFTable)); - ZSTD_STATIC_ASSERT(sizeof(entropy->LLTable) + sizeof(entropy->OFTable) + sizeof(entropy->MLTable) >= HUF_DECOMPRESS_WORKSPACE_SIZE); - { void* const workspace = &entropy->LLTable; /* use fse tables as temporary workspace; implies fse tables are grouped together */ - size_t const workspaceSize = sizeof(entropy->LLTable) + sizeof(entropy->OFTable) + sizeof(entropy->MLTable); -#ifdef HUF_FORCE_DECOMPRESS_X1 - /* in minimal huffman, we always use X1 variants */ - size_t const hSize = HUF_readDTableX1_wksp(entropy->hufTable, - dictPtr, dictEnd - dictPtr, - workspace, workspaceSize); -#else - size_t const hSize = HUF_readDTableX2_wksp(entropy->hufTable, - dictPtr, dictEnd - dictPtr, - workspace, workspaceSize); -#endif - RETURN_ERROR_IF(HUF_isError(hSize), dictionary_corrupted, ""); - dictPtr += hSize; - } - - { short offcodeNCount[MaxOff+1]; - unsigned offcodeMaxValue = MaxOff, offcodeLog; - size_t const offcodeHeaderSize = FSE_readNCount(offcodeNCount, &offcodeMaxValue, &offcodeLog, dictPtr, dictEnd-dictPtr); - RETURN_ERROR_IF(FSE_isError(offcodeHeaderSize), dictionary_corrupted, ""); - RETURN_ERROR_IF(offcodeMaxValue > MaxOff, dictionary_corrupted, ""); - RETURN_ERROR_IF(offcodeLog > OffFSELog, dictionary_corrupted, ""); - ZSTD_buildFSETable( entropy->OFTable, - offcodeNCount, offcodeMaxValue, - OF_base, OF_bits, - offcodeLog); - dictPtr += offcodeHeaderSize; - } - - { short matchlengthNCount[MaxML+1]; - unsigned matchlengthMaxValue = MaxML, matchlengthLog; - size_t const matchlengthHeaderSize = FSE_readNCount(matchlengthNCount, &matchlengthMaxValue, &matchlengthLog, dictPtr, dictEnd-dictPtr); - RETURN_ERROR_IF(FSE_isError(matchlengthHeaderSize), dictionary_corrupted, ""); - RETURN_ERROR_IF(matchlengthMaxValue > MaxML, dictionary_corrupted, ""); - RETURN_ERROR_IF(matchlengthLog > MLFSELog, dictionary_corrupted, ""); - ZSTD_buildFSETable( entropy->MLTable, - matchlengthNCount, matchlengthMaxValue, - ML_base, ML_bits, - matchlengthLog); - dictPtr += matchlengthHeaderSize; - } - - { short litlengthNCount[MaxLL+1]; - unsigned litlengthMaxValue = MaxLL, litlengthLog; - size_t const litlengthHeaderSize = FSE_readNCount(litlengthNCount, &litlengthMaxValue, &litlengthLog, dictPtr, dictEnd-dictPtr); - RETURN_ERROR_IF(FSE_isError(litlengthHeaderSize), dictionary_corrupted, ""); - RETURN_ERROR_IF(litlengthMaxValue > MaxLL, dictionary_corrupted, ""); - RETURN_ERROR_IF(litlengthLog > LLFSELog, dictionary_corrupted, ""); - ZSTD_buildFSETable( entropy->LLTable, - litlengthNCount, litlengthMaxValue, - LL_base, LL_bits, - litlengthLog); - dictPtr += litlengthHeaderSize; - } - - RETURN_ERROR_IF(dictPtr+12 > dictEnd, dictionary_corrupted, ""); - { int i; - size_t const dictContentSize = (size_t)(dictEnd - (dictPtr+12)); - for (i=0; i<3; i++) { - U32 const rep = MEM_readLE32(dictPtr); dictPtr += 4; - RETURN_ERROR_IF(rep==0 || rep > dictContentSize, - dictionary_corrupted, ""); - entropy->rep[i] = rep; - } } - - return dictPtr - (const BYTE*)dict; -} - -static size_t ZSTD_decompress_insertDictionary(ZSTD_DCtx* dctx, const void* dict, size_t dictSize) -{ - if (dictSize < 8) return ZSTD_refDictContent(dctx, dict, dictSize); - { U32 const magic = MEM_readLE32(dict); - if (magic != ZSTD_MAGIC_DICTIONARY) { - return ZSTD_refDictContent(dctx, dict, dictSize); /* pure content mode */ - } } - dctx->dictID = MEM_readLE32((const char*)dict + ZSTD_FRAMEIDSIZE); - - /* load entropy tables */ - { size_t const eSize = ZSTD_loadDEntropy(&dctx->entropy, dict, dictSize); - RETURN_ERROR_IF(ZSTD_isError(eSize), dictionary_corrupted, ""); - dict = (const char*)dict + eSize; - dictSize -= eSize; - } - dctx->litEntropy = dctx->fseEntropy = 1; - - /* reference dictionary content */ - return ZSTD_refDictContent(dctx, dict, dictSize); -} - -size_t ZSTD_decompressBegin(ZSTD_DCtx* dctx) -{ - assert(dctx != NULL); - dctx->expected = ZSTD_startingInputLength(dctx->format); /* dctx->format must be properly set */ - dctx->stage = ZSTDds_getFrameHeaderSize; - dctx->decodedSize = 0; - dctx->previousDstEnd = NULL; - dctx->prefixStart = NULL; - dctx->virtualStart = NULL; - dctx->dictEnd = NULL; - dctx->entropy.hufTable[0] = (HUF_DTable)((HufLog)*0x1000001); /* cover both little and big endian */ - dctx->litEntropy = dctx->fseEntropy = 0; - dctx->dictID = 0; - dctx->bType = bt_reserved; - ZSTD_STATIC_ASSERT(sizeof(dctx->entropy.rep) == sizeof(repStartValue)); - memcpy(dctx->entropy.rep, repStartValue, sizeof(repStartValue)); /* initial repcodes */ - dctx->LLTptr = dctx->entropy.LLTable; - dctx->MLTptr = dctx->entropy.MLTable; - dctx->OFTptr = dctx->entropy.OFTable; - dctx->HUFptr = dctx->entropy.hufTable; - return 0; -} - -size_t ZSTD_decompressBegin_usingDict(ZSTD_DCtx* dctx, const void* dict, size_t dictSize) -{ - FORWARD_IF_ERROR( ZSTD_decompressBegin(dctx) , ""); - if (dict && dictSize) - RETURN_ERROR_IF( - ZSTD_isError(ZSTD_decompress_insertDictionary(dctx, dict, dictSize)), - dictionary_corrupted, ""); - return 0; -} - - -/* ====== ZSTD_DDict ====== */ - -size_t ZSTD_decompressBegin_usingDDict(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict) -{ - DEBUGLOG(4, "ZSTD_decompressBegin_usingDDict"); - assert(dctx != NULL); - if (ddict) { - const char* const dictStart = (const char*)ZSTD_DDict_dictContent(ddict); - size_t const dictSize = ZSTD_DDict_dictSize(ddict); - const void* const dictEnd = dictStart + dictSize; - dctx->ddictIsCold = (dctx->dictEnd != dictEnd); - DEBUGLOG(4, "DDict is %s", - dctx->ddictIsCold ? "~cold~" : "hot!"); - } - FORWARD_IF_ERROR( ZSTD_decompressBegin(dctx) , ""); - if (ddict) { /* NULL ddict is equivalent to no dictionary */ - ZSTD_copyDDictParameters(dctx, ddict); - } - return 0; -} - -/*! ZSTD_getDictID_fromDict() : - * Provides the dictID stored within dictionary. - * if @return == 0, the dictionary is not conformant with Zstandard specification. - * It can still be loaded, but as a content-only dictionary. */ -unsigned ZSTD_getDictID_fromDict(const void* dict, size_t dictSize) -{ - if (dictSize < 8) return 0; - if (MEM_readLE32(dict) != ZSTD_MAGIC_DICTIONARY) return 0; - return MEM_readLE32((const char*)dict + ZSTD_FRAMEIDSIZE); -} - -/*! ZSTD_getDictID_fromFrame() : - * Provides the dictID required to decompress frame stored within `src`. - * If @return == 0, the dictID could not be decoded. - * This could for one of the following reasons : - * - The frame does not require a dictionary (most common case). - * - The frame was built with dictID intentionally removed. - * Needed dictionary is a hidden information. - * Note : this use case also happens when using a non-conformant dictionary. - * - `srcSize` is too small, and as a result, frame header could not be decoded. - * Note : possible if `srcSize < ZSTD_FRAMEHEADERSIZE_MAX`. - * - This is not a Zstandard frame. - * When identifying the exact failure cause, it's possible to use - * ZSTD_getFrameHeader(), which will provide a more precise error code. */ -unsigned ZSTD_getDictID_fromFrame(const void* src, size_t srcSize) -{ - ZSTD_frameHeader zfp = { 0, 0, 0, ZSTD_frame, 0, 0, 0 }; - size_t const hError = ZSTD_getFrameHeader(&zfp, src, srcSize); - if (ZSTD_isError(hError)) return 0; - return zfp.dictID; -} - - -/*! ZSTD_decompress_usingDDict() : -* Decompression using a pre-digested Dictionary -* Use dictionary without significant overhead. */ -size_t ZSTD_decompress_usingDDict(ZSTD_DCtx* dctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, - const ZSTD_DDict* ddict) -{ - /* pass content and size in case legacy frames are encountered */ - return ZSTD_decompressMultiFrame(dctx, dst, dstCapacity, src, srcSize, - NULL, 0, - ddict); -} - - -/*===================================== -* Streaming decompression -*====================================*/ - -ZSTD_DStream* ZSTD_createDStream(void) -{ - DEBUGLOG(3, "ZSTD_createDStream"); - return ZSTD_createDStream_advanced(ZSTD_defaultCMem); -} - -ZSTD_DStream* ZSTD_initStaticDStream(void *workspace, size_t workspaceSize) -{ - return ZSTD_initStaticDCtx(workspace, workspaceSize); -} - -ZSTD_DStream* ZSTD_createDStream_advanced(ZSTD_customMem customMem) -{ - return ZSTD_createDCtx_advanced(customMem); -} - -size_t ZSTD_freeDStream(ZSTD_DStream* zds) -{ - return ZSTD_freeDCtx(zds); -} - - -/* *** Initialization *** */ - -size_t ZSTD_DStreamInSize(void) { return ZSTD_BLOCKSIZE_MAX + ZSTD_blockHeaderSize; } -size_t ZSTD_DStreamOutSize(void) { return ZSTD_BLOCKSIZE_MAX; } - -size_t ZSTD_DCtx_loadDictionary_advanced(ZSTD_DCtx* dctx, - const void* dict, size_t dictSize, - ZSTD_dictLoadMethod_e dictLoadMethod, - ZSTD_dictContentType_e dictContentType) -{ - RETURN_ERROR_IF(dctx->streamStage != zdss_init, stage_wrong, ""); - ZSTD_clearDict(dctx); - if (dict && dictSize != 0) { - dctx->ddictLocal = ZSTD_createDDict_advanced(dict, dictSize, dictLoadMethod, dictContentType, dctx->customMem); - RETURN_ERROR_IF(dctx->ddictLocal == NULL, memory_allocation, "NULL pointer!"); - dctx->ddict = dctx->ddictLocal; - dctx->dictUses = ZSTD_use_indefinitely; - } - return 0; -} - -size_t ZSTD_DCtx_loadDictionary_byReference(ZSTD_DCtx* dctx, const void* dict, size_t dictSize) -{ - return ZSTD_DCtx_loadDictionary_advanced(dctx, dict, dictSize, ZSTD_dlm_byRef, ZSTD_dct_auto); -} - -size_t ZSTD_DCtx_loadDictionary(ZSTD_DCtx* dctx, const void* dict, size_t dictSize) -{ - return ZSTD_DCtx_loadDictionary_advanced(dctx, dict, dictSize, ZSTD_dlm_byCopy, ZSTD_dct_auto); -} - -size_t ZSTD_DCtx_refPrefix_advanced(ZSTD_DCtx* dctx, const void* prefix, size_t prefixSize, ZSTD_dictContentType_e dictContentType) -{ - FORWARD_IF_ERROR(ZSTD_DCtx_loadDictionary_advanced(dctx, prefix, prefixSize, ZSTD_dlm_byRef, dictContentType), ""); - dctx->dictUses = ZSTD_use_once; - return 0; -} - -size_t ZSTD_DCtx_refPrefix(ZSTD_DCtx* dctx, const void* prefix, size_t prefixSize) -{ - return ZSTD_DCtx_refPrefix_advanced(dctx, prefix, prefixSize, ZSTD_dct_rawContent); -} - - -/* ZSTD_initDStream_usingDict() : - * return : expected size, aka ZSTD_startingInputLength(). - * this function cannot fail */ -size_t ZSTD_initDStream_usingDict(ZSTD_DStream* zds, const void* dict, size_t dictSize) -{ - DEBUGLOG(4, "ZSTD_initDStream_usingDict"); - FORWARD_IF_ERROR( ZSTD_DCtx_reset(zds, ZSTD_reset_session_only) , ""); - FORWARD_IF_ERROR( ZSTD_DCtx_loadDictionary(zds, dict, dictSize) , ""); - return ZSTD_startingInputLength(zds->format); -} - -/* note : this variant can't fail */ -size_t ZSTD_initDStream(ZSTD_DStream* zds) -{ - DEBUGLOG(4, "ZSTD_initDStream"); - return ZSTD_initDStream_usingDDict(zds, NULL); -} - -/* ZSTD_initDStream_usingDDict() : - * ddict will just be referenced, and must outlive decompression session - * this function cannot fail */ -size_t ZSTD_initDStream_usingDDict(ZSTD_DStream* dctx, const ZSTD_DDict* ddict) -{ - FORWARD_IF_ERROR( ZSTD_DCtx_reset(dctx, ZSTD_reset_session_only) , ""); - FORWARD_IF_ERROR( ZSTD_DCtx_refDDict(dctx, ddict) , ""); - return ZSTD_startingInputLength(dctx->format); -} - -/* ZSTD_resetDStream() : - * return : expected size, aka ZSTD_startingInputLength(). - * this function cannot fail */ -size_t ZSTD_resetDStream(ZSTD_DStream* dctx) -{ - FORWARD_IF_ERROR(ZSTD_DCtx_reset(dctx, ZSTD_reset_session_only), ""); - return ZSTD_startingInputLength(dctx->format); -} - - -size_t ZSTD_DCtx_refDDict(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict) -{ - RETURN_ERROR_IF(dctx->streamStage != zdss_init, stage_wrong, ""); - ZSTD_clearDict(dctx); - if (ddict) { - dctx->ddict = ddict; - dctx->dictUses = ZSTD_use_indefinitely; - } - return 0; -} - -/* ZSTD_DCtx_setMaxWindowSize() : - * note : no direct equivalence in ZSTD_DCtx_setParameter, - * since this version sets windowSize, and the other sets windowLog */ -size_t ZSTD_DCtx_setMaxWindowSize(ZSTD_DCtx* dctx, size_t maxWindowSize) -{ - ZSTD_bounds const bounds = ZSTD_dParam_getBounds(ZSTD_d_windowLogMax); - size_t const min = (size_t)1 << bounds.lowerBound; - size_t const max = (size_t)1 << bounds.upperBound; - RETURN_ERROR_IF(dctx->streamStage != zdss_init, stage_wrong, ""); - RETURN_ERROR_IF(maxWindowSize < min, parameter_outOfBound, ""); - RETURN_ERROR_IF(maxWindowSize > max, parameter_outOfBound, ""); - dctx->maxWindowSize = maxWindowSize; - return 0; -} - -size_t ZSTD_DCtx_setFormat(ZSTD_DCtx* dctx, ZSTD_format_e format) -{ - return ZSTD_DCtx_setParameter(dctx, ZSTD_d_format, format); -} - -ZSTD_bounds ZSTD_dParam_getBounds(ZSTD_dParameter dParam) -{ - ZSTD_bounds bounds = { 0, 0, 0 }; - switch(dParam) { - case ZSTD_d_windowLogMax: - bounds.lowerBound = ZSTD_WINDOWLOG_ABSOLUTEMIN; - bounds.upperBound = ZSTD_WINDOWLOG_MAX; - return bounds; - case ZSTD_d_format: - bounds.lowerBound = (int)ZSTD_f_zstd1; - bounds.upperBound = (int)ZSTD_f_zstd1_magicless; - ZSTD_STATIC_ASSERT(ZSTD_f_zstd1 < ZSTD_f_zstd1_magicless); - return bounds; - case ZSTD_d_stableOutBuffer: - bounds.lowerBound = (int)ZSTD_obm_buffered; - bounds.upperBound = (int)ZSTD_obm_stable; - return bounds; - default:; - } - bounds.error = ERROR(parameter_unsupported); - return bounds; -} - -/* ZSTD_dParam_withinBounds: - * @return 1 if value is within dParam bounds, - * 0 otherwise */ -static int ZSTD_dParam_withinBounds(ZSTD_dParameter dParam, int value) -{ - ZSTD_bounds const bounds = ZSTD_dParam_getBounds(dParam); - if (ZSTD_isError(bounds.error)) return 0; - if (value < bounds.lowerBound) return 0; - if (value > bounds.upperBound) return 0; - return 1; -} - -#define CHECK_DBOUNDS(p,v) { \ - RETURN_ERROR_IF(!ZSTD_dParam_withinBounds(p, v), parameter_outOfBound, ""); \ -} - -size_t ZSTD_DCtx_setParameter(ZSTD_DCtx* dctx, ZSTD_dParameter dParam, int value) -{ - RETURN_ERROR_IF(dctx->streamStage != zdss_init, stage_wrong, ""); - switch(dParam) { - case ZSTD_d_windowLogMax: - if (value == 0) value = ZSTD_WINDOWLOG_LIMIT_DEFAULT; - CHECK_DBOUNDS(ZSTD_d_windowLogMax, value); - dctx->maxWindowSize = ((size_t)1) << value; - return 0; - case ZSTD_d_format: - CHECK_DBOUNDS(ZSTD_d_format, value); - dctx->format = (ZSTD_format_e)value; - return 0; - case ZSTD_d_stableOutBuffer: - CHECK_DBOUNDS(ZSTD_d_stableOutBuffer, value); - dctx->outBufferMode = (ZSTD_outBufferMode_e)value; - return 0; - default:; - } - RETURN_ERROR(parameter_unsupported, ""); -} - -size_t ZSTD_DCtx_reset(ZSTD_DCtx* dctx, ZSTD_ResetDirective reset) -{ - if ( (reset == ZSTD_reset_session_only) - || (reset == ZSTD_reset_session_and_parameters) ) { - dctx->streamStage = zdss_init; - dctx->noForwardProgress = 0; - } - if ( (reset == ZSTD_reset_parameters) - || (reset == ZSTD_reset_session_and_parameters) ) { - RETURN_ERROR_IF(dctx->streamStage != zdss_init, stage_wrong, ""); - ZSTD_clearDict(dctx); - dctx->format = ZSTD_f_zstd1; - dctx->maxWindowSize = ZSTD_MAXWINDOWSIZE_DEFAULT; - } - return 0; -} - - -size_t ZSTD_sizeof_DStream(const ZSTD_DStream* dctx) -{ - return ZSTD_sizeof_DCtx(dctx); -} - -size_t ZSTD_decodingBufferSize_min(unsigned long long windowSize, unsigned long long frameContentSize) -{ - size_t const blockSize = (size_t) MIN(windowSize, ZSTD_BLOCKSIZE_MAX); - unsigned long long const neededRBSize = windowSize + blockSize + (WILDCOPY_OVERLENGTH * 2); - unsigned long long const neededSize = MIN(frameContentSize, neededRBSize); - size_t const minRBSize = (size_t) neededSize; - RETURN_ERROR_IF((unsigned long long)minRBSize != neededSize, - frameParameter_windowTooLarge, ""); - return minRBSize; -} - -size_t ZSTD_estimateDStreamSize(size_t windowSize) -{ - size_t const blockSize = MIN(windowSize, ZSTD_BLOCKSIZE_MAX); - size_t const inBuffSize = blockSize; /* no block can be larger */ - size_t const outBuffSize = ZSTD_decodingBufferSize_min(windowSize, ZSTD_CONTENTSIZE_UNKNOWN); - return ZSTD_estimateDCtxSize() + inBuffSize + outBuffSize; -} - -size_t ZSTD_estimateDStreamSize_fromFrame(const void* src, size_t srcSize) -{ - U32 const windowSizeMax = 1U << ZSTD_WINDOWLOG_MAX; /* note : should be user-selectable, but requires an additional parameter (or a dctx) */ - ZSTD_frameHeader zfh; - size_t const err = ZSTD_getFrameHeader(&zfh, src, srcSize); - if (ZSTD_isError(err)) return err; - RETURN_ERROR_IF(err>0, srcSize_wrong, ""); - RETURN_ERROR_IF(zfh.windowSize > windowSizeMax, - frameParameter_windowTooLarge, ""); - return ZSTD_estimateDStreamSize((size_t)zfh.windowSize); -} - - -/* ***** Decompression ***** */ - -static int ZSTD_DCtx_isOverflow(ZSTD_DStream* zds, size_t const neededInBuffSize, size_t const neededOutBuffSize) -{ - return (zds->inBuffSize + zds->outBuffSize) >= (neededInBuffSize + neededOutBuffSize) * ZSTD_WORKSPACETOOLARGE_FACTOR; -} - -static void ZSTD_DCtx_updateOversizedDuration(ZSTD_DStream* zds, size_t const neededInBuffSize, size_t const neededOutBuffSize) -{ - if (ZSTD_DCtx_isOverflow(zds, neededInBuffSize, neededOutBuffSize)) - zds->oversizedDuration++; - else - zds->oversizedDuration = 0; -} - -static int ZSTD_DCtx_isOversizedTooLong(ZSTD_DStream* zds) -{ - return zds->oversizedDuration >= ZSTD_WORKSPACETOOLARGE_MAXDURATION; -} - -/* Checks that the output buffer hasn't changed if ZSTD_obm_stable is used. */ -static size_t ZSTD_checkOutBuffer(ZSTD_DStream const* zds, ZSTD_outBuffer const* output) -{ - ZSTD_outBuffer const expect = zds->expectedOutBuffer; - /* No requirement when ZSTD_obm_stable is not enabled. */ - if (zds->outBufferMode != ZSTD_obm_stable) - return 0; - /* Any buffer is allowed in zdss_init, this must be the same for every other call until - * the context is reset. - */ - if (zds->streamStage == zdss_init) - return 0; - /* The buffer must match our expectation exactly. */ - if (expect.dst == output->dst && expect.pos == output->pos && expect.size == output->size) - return 0; - RETURN_ERROR(dstBuffer_wrong, "ZSTD_obm_stable enabled but output differs!"); -} - -/* Calls ZSTD_decompressContinue() with the right parameters for ZSTD_decompressStream() - * and updates the stage and the output buffer state. This call is extracted so it can be - * used both when reading directly from the ZSTD_inBuffer, and in buffered input mode. - * NOTE: You must break after calling this function since the streamStage is modified. - */ -static size_t ZSTD_decompressContinueStream( - ZSTD_DStream* zds, char** op, char* oend, - void const* src, size_t srcSize) { - int const isSkipFrame = ZSTD_isSkipFrame(zds); - if (zds->outBufferMode == ZSTD_obm_buffered) { - size_t const dstSize = isSkipFrame ? 0 : zds->outBuffSize - zds->outStart; - size_t const decodedSize = ZSTD_decompressContinue(zds, - zds->outBuff + zds->outStart, dstSize, src, srcSize); - FORWARD_IF_ERROR(decodedSize, ""); - if (!decodedSize && !isSkipFrame) { - zds->streamStage = zdss_read; - } else { - zds->outEnd = zds->outStart + decodedSize; - zds->streamStage = zdss_flush; - } - } else { - /* Write directly into the output buffer */ - size_t const dstSize = isSkipFrame ? 0 : oend - *op; - size_t const decodedSize = ZSTD_decompressContinue(zds, *op, dstSize, src, srcSize); - FORWARD_IF_ERROR(decodedSize, ""); - *op += decodedSize; - /* Flushing is not needed. */ - zds->streamStage = zdss_read; - assert(*op <= oend); - assert(zds->outBufferMode == ZSTD_obm_stable); - } - return 0; -} - -size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inBuffer* input) -{ - const char* const src = (const char*)input->src; - const char* const istart = input->pos != 0 ? src + input->pos : src; - const char* const iend = input->size != 0 ? src + input->size : src; - const char* ip = istart; - char* const dst = (char*)output->dst; - char* const ostart = output->pos != 0 ? dst + output->pos : dst; - char* const oend = output->size != 0 ? dst + output->size : dst; - char* op = ostart; - U32 someMoreWork = 1; - - DEBUGLOG(5, "ZSTD_decompressStream"); - RETURN_ERROR_IF( - input->pos > input->size, - srcSize_wrong, - "forbidden. in: pos: %u vs size: %u", - (U32)input->pos, (U32)input->size); - RETURN_ERROR_IF( - output->pos > output->size, - dstSize_tooSmall, - "forbidden. out: pos: %u vs size: %u", - (U32)output->pos, (U32)output->size); - DEBUGLOG(5, "input size : %u", (U32)(input->size - input->pos)); - FORWARD_IF_ERROR(ZSTD_checkOutBuffer(zds, output), ""); - - while (someMoreWork) { - switch(zds->streamStage) - { - case zdss_init : - DEBUGLOG(5, "stage zdss_init => transparent reset "); - zds->streamStage = zdss_loadHeader; - zds->lhSize = zds->inPos = zds->outStart = zds->outEnd = 0; - zds->legacyVersion = 0; - zds->hostageByte = 0; - zds->expectedOutBuffer = *output; - /* fall-through */ - - case zdss_loadHeader : - DEBUGLOG(5, "stage zdss_loadHeader (srcSize : %u)", (U32)(iend - ip)); -#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT>=1) - if (zds->legacyVersion) { - RETURN_ERROR_IF(zds->staticSize, memory_allocation, - "legacy support is incompatible with static dctx"); - { size_t const hint = ZSTD_decompressLegacyStream(zds->legacyContext, zds->legacyVersion, output, input); - if (hint==0) zds->streamStage = zdss_init; - return hint; - } } -#endif - { size_t const hSize = ZSTD_getFrameHeader_advanced(&zds->fParams, zds->headerBuffer, zds->lhSize, zds->format); - DEBUGLOG(5, "header size : %u", (U32)hSize); - if (ZSTD_isError(hSize)) { -#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT>=1) - U32 const legacyVersion = ZSTD_isLegacy(istart, iend-istart); - if (legacyVersion) { - ZSTD_DDict const* const ddict = ZSTD_getDDict(zds); - const void* const dict = ddict ? ZSTD_DDict_dictContent(ddict) : NULL; - size_t const dictSize = ddict ? ZSTD_DDict_dictSize(ddict) : 0; - DEBUGLOG(5, "ZSTD_decompressStream: detected legacy version v0.%u", legacyVersion); - RETURN_ERROR_IF(zds->staticSize, memory_allocation, - "legacy support is incompatible with static dctx"); - FORWARD_IF_ERROR(ZSTD_initLegacyStream(&zds->legacyContext, - zds->previousLegacyVersion, legacyVersion, - dict, dictSize), ""); - zds->legacyVersion = zds->previousLegacyVersion = legacyVersion; - { size_t const hint = ZSTD_decompressLegacyStream(zds->legacyContext, legacyVersion, output, input); - if (hint==0) zds->streamStage = zdss_init; /* or stay in stage zdss_loadHeader */ - return hint; - } } -#endif - return hSize; /* error */ - } - if (hSize != 0) { /* need more input */ - size_t const toLoad = hSize - zds->lhSize; /* if hSize!=0, hSize > zds->lhSize */ - size_t const remainingInput = (size_t)(iend-ip); - assert(iend >= ip); - if (toLoad > remainingInput) { /* not enough input to load full header */ - if (remainingInput > 0) { - memcpy(zds->headerBuffer + zds->lhSize, ip, remainingInput); - zds->lhSize += remainingInput; - } - input->pos = input->size; - return (MAX((size_t)ZSTD_FRAMEHEADERSIZE_MIN(zds->format), hSize) - zds->lhSize) + ZSTD_blockHeaderSize; /* remaining header bytes + next block header */ - } - assert(ip != NULL); - memcpy(zds->headerBuffer + zds->lhSize, ip, toLoad); zds->lhSize = hSize; ip += toLoad; - break; - } } - - /* check for single-pass mode opportunity */ - if (zds->fParams.frameContentSize != ZSTD_CONTENTSIZE_UNKNOWN - && zds->fParams.frameType != ZSTD_skippableFrame - && (U64)(size_t)(oend-op) >= zds->fParams.frameContentSize) { - size_t const cSize = ZSTD_findFrameCompressedSize(istart, iend-istart); - if (cSize <= (size_t)(iend-istart)) { - /* shortcut : using single-pass mode */ - size_t const decompressedSize = ZSTD_decompress_usingDDict(zds, op, oend-op, istart, cSize, ZSTD_getDDict(zds)); - if (ZSTD_isError(decompressedSize)) return decompressedSize; - DEBUGLOG(4, "shortcut to single-pass ZSTD_decompress_usingDDict()") - ip = istart + cSize; - op += decompressedSize; - zds->expected = 0; - zds->streamStage = zdss_init; - someMoreWork = 0; - break; - } } - - /* Check output buffer is large enough for ZSTD_odm_stable. */ - if (zds->outBufferMode == ZSTD_obm_stable - && zds->fParams.frameType != ZSTD_skippableFrame - && zds->fParams.frameContentSize != ZSTD_CONTENTSIZE_UNKNOWN - && (U64)(size_t)(oend-op) < zds->fParams.frameContentSize) { - RETURN_ERROR(dstSize_tooSmall, "ZSTD_obm_stable passed but ZSTD_outBuffer is too small"); - } - - /* Consume header (see ZSTDds_decodeFrameHeader) */ - DEBUGLOG(4, "Consume header"); - FORWARD_IF_ERROR(ZSTD_decompressBegin_usingDDict(zds, ZSTD_getDDict(zds)), ""); - - if ((MEM_readLE32(zds->headerBuffer) & ZSTD_MAGIC_SKIPPABLE_MASK) == ZSTD_MAGIC_SKIPPABLE_START) { /* skippable frame */ - zds->expected = MEM_readLE32(zds->headerBuffer + ZSTD_FRAMEIDSIZE); - zds->stage = ZSTDds_skipFrame; - } else { - FORWARD_IF_ERROR(ZSTD_decodeFrameHeader(zds, zds->headerBuffer, zds->lhSize), ""); - zds->expected = ZSTD_blockHeaderSize; - zds->stage = ZSTDds_decodeBlockHeader; - } - - /* control buffer memory usage */ - DEBUGLOG(4, "Control max memory usage (%u KB <= max %u KB)", - (U32)(zds->fParams.windowSize >>10), - (U32)(zds->maxWindowSize >> 10) ); - zds->fParams.windowSize = MAX(zds->fParams.windowSize, 1U << ZSTD_WINDOWLOG_ABSOLUTEMIN); - RETURN_ERROR_IF(zds->fParams.windowSize > zds->maxWindowSize, - frameParameter_windowTooLarge, ""); - - /* Adapt buffer sizes to frame header instructions */ - { size_t const neededInBuffSize = MAX(zds->fParams.blockSizeMax, 4 /* frame checksum */); - size_t const neededOutBuffSize = zds->outBufferMode == ZSTD_obm_buffered - ? ZSTD_decodingBufferSize_min(zds->fParams.windowSize, zds->fParams.frameContentSize) - : 0; - - ZSTD_DCtx_updateOversizedDuration(zds, neededInBuffSize, neededOutBuffSize); - - { int const tooSmall = (zds->inBuffSize < neededInBuffSize) || (zds->outBuffSize < neededOutBuffSize); - int const tooLarge = ZSTD_DCtx_isOversizedTooLong(zds); - - if (tooSmall || tooLarge) { - size_t const bufferSize = neededInBuffSize + neededOutBuffSize; - DEBUGLOG(4, "inBuff : from %u to %u", - (U32)zds->inBuffSize, (U32)neededInBuffSize); - DEBUGLOG(4, "outBuff : from %u to %u", - (U32)zds->outBuffSize, (U32)neededOutBuffSize); - if (zds->staticSize) { /* static DCtx */ - DEBUGLOG(4, "staticSize : %u", (U32)zds->staticSize); - assert(zds->staticSize >= sizeof(ZSTD_DCtx)); /* controlled at init */ - RETURN_ERROR_IF( - bufferSize > zds->staticSize - sizeof(ZSTD_DCtx), - memory_allocation, ""); - } else { - ZSTD_free(zds->inBuff, zds->customMem); - zds->inBuffSize = 0; - zds->outBuffSize = 0; - zds->inBuff = (char*)ZSTD_malloc(bufferSize, zds->customMem); - RETURN_ERROR_IF(zds->inBuff == NULL, memory_allocation, ""); - } - zds->inBuffSize = neededInBuffSize; - zds->outBuff = zds->inBuff + zds->inBuffSize; - zds->outBuffSize = neededOutBuffSize; - } } } - zds->streamStage = zdss_read; - /* fall-through */ - - case zdss_read: - DEBUGLOG(5, "stage zdss_read"); - { size_t const neededInSize = ZSTD_nextSrcSizeToDecompressWithInputSize(zds, iend - ip); - DEBUGLOG(5, "neededInSize = %u", (U32)neededInSize); - if (neededInSize==0) { /* end of frame */ - zds->streamStage = zdss_init; - someMoreWork = 0; - break; - } - if ((size_t)(iend-ip) >= neededInSize) { /* decode directly from src */ - FORWARD_IF_ERROR(ZSTD_decompressContinueStream(zds, &op, oend, ip, neededInSize), ""); - ip += neededInSize; - /* Function modifies the stage so we must break */ - break; - } } - if (ip==iend) { someMoreWork = 0; break; } /* no more input */ - zds->streamStage = zdss_load; - /* fall-through */ - - case zdss_load: - { size_t const neededInSize = ZSTD_nextSrcSizeToDecompress(zds); - size_t const toLoad = neededInSize - zds->inPos; - int const isSkipFrame = ZSTD_isSkipFrame(zds); - size_t loadedSize; - /* At this point we shouldn't be decompressing a block that we can stream. */ - assert(neededInSize == ZSTD_nextSrcSizeToDecompressWithInputSize(zds, iend - ip)); - if (isSkipFrame) { - loadedSize = MIN(toLoad, (size_t)(iend-ip)); - } else { - RETURN_ERROR_IF(toLoad > zds->inBuffSize - zds->inPos, - corruption_detected, - "should never happen"); - loadedSize = ZSTD_limitCopy(zds->inBuff + zds->inPos, toLoad, ip, iend-ip); - } - ip += loadedSize; - zds->inPos += loadedSize; - if (loadedSize < toLoad) { someMoreWork = 0; break; } /* not enough input, wait for more */ - - /* decode loaded input */ - zds->inPos = 0; /* input is consumed */ - FORWARD_IF_ERROR(ZSTD_decompressContinueStream(zds, &op, oend, zds->inBuff, neededInSize), ""); - /* Function modifies the stage so we must break */ - break; - } - case zdss_flush: - { size_t const toFlushSize = zds->outEnd - zds->outStart; - size_t const flushedSize = ZSTD_limitCopy(op, oend-op, zds->outBuff + zds->outStart, toFlushSize); - op += flushedSize; - zds->outStart += flushedSize; - if (flushedSize == toFlushSize) { /* flush completed */ - zds->streamStage = zdss_read; - if ( (zds->outBuffSize < zds->fParams.frameContentSize) - && (zds->outStart + zds->fParams.blockSizeMax > zds->outBuffSize) ) { - DEBUGLOG(5, "restart filling outBuff from beginning (left:%i, needed:%u)", - (int)(zds->outBuffSize - zds->outStart), - (U32)zds->fParams.blockSizeMax); - zds->outStart = zds->outEnd = 0; - } - break; - } } - /* cannot complete flush */ - someMoreWork = 0; - break; - - default: - assert(0); /* impossible */ - RETURN_ERROR(GENERIC, "impossible to reach"); /* some compiler require default to do something */ - } } - - /* result */ - input->pos = (size_t)(ip - (const char*)(input->src)); - output->pos = (size_t)(op - (char*)(output->dst)); - - /* Update the expected output buffer for ZSTD_obm_stable. */ - zds->expectedOutBuffer = *output; - - if ((ip==istart) && (op==ostart)) { /* no forward progress */ - zds->noForwardProgress ++; - if (zds->noForwardProgress >= ZSTD_NO_FORWARD_PROGRESS_MAX) { - RETURN_ERROR_IF(op==oend, dstSize_tooSmall, ""); - RETURN_ERROR_IF(ip==iend, srcSize_wrong, ""); - assert(0); - } - } else { - zds->noForwardProgress = 0; - } - { size_t nextSrcSizeHint = ZSTD_nextSrcSizeToDecompress(zds); - if (!nextSrcSizeHint) { /* frame fully decoded */ - if (zds->outEnd == zds->outStart) { /* output fully flushed */ - if (zds->hostageByte) { - if (input->pos >= input->size) { - /* can't release hostage (not present) */ - zds->streamStage = zdss_read; - return 1; - } - input->pos++; /* release hostage */ - } /* zds->hostageByte */ - return 0; - } /* zds->outEnd == zds->outStart */ - if (!zds->hostageByte) { /* output not fully flushed; keep last byte as hostage; will be released when all output is flushed */ - input->pos--; /* note : pos > 0, otherwise, impossible to finish reading last block */ - zds->hostageByte=1; - } - return 1; - } /* nextSrcSizeHint==0 */ - nextSrcSizeHint += ZSTD_blockHeaderSize * (ZSTD_nextInputType(zds) == ZSTDnit_block); /* preload header of next block */ - assert(zds->inPos <= nextSrcSizeHint); - nextSrcSizeHint -= zds->inPos; /* part already loaded*/ - return nextSrcSizeHint; - } -} - -size_t ZSTD_decompressStream_simpleArgs ( - ZSTD_DCtx* dctx, - void* dst, size_t dstCapacity, size_t* dstPos, - const void* src, size_t srcSize, size_t* srcPos) -{ - ZSTD_outBuffer output = { dst, dstCapacity, *dstPos }; - ZSTD_inBuffer input = { src, srcSize, *srcPos }; - /* ZSTD_compress_generic() will check validity of dstPos and srcPos */ - size_t const cErr = ZSTD_decompressStream(dctx, &output, &input); - *dstPos = output.pos; - *srcPos = input.pos; - return cErr; -} -/**** ended inlining decompress/zstd_decompress.c ****/ -/**** start inlining decompress/zstd_decompress_block.c ****/ -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under both the BSD-style license (found in the - * LICENSE file in the root directory of this source tree) and the GPLv2 (found - * in the COPYING file in the root directory of this source tree). - * You may select, at your option, one of the above-listed licenses. - */ - -/* zstd_decompress_block : - * this module takes care of decompressing _compressed_ block */ - -/*-******************************************************* -* Dependencies -*********************************************************/ -#include /* memcpy, memmove, memset */ -/**** skipping file: ../common/compiler.h ****/ -/**** skipping file: ../common/cpu.h ****/ -/**** skipping file: ../common/mem.h ****/ -#define FSE_STATIC_LINKING_ONLY -/**** skipping file: ../common/fse.h ****/ -#define HUF_STATIC_LINKING_ONLY -/**** skipping file: ../common/huf.h ****/ -/**** skipping file: ../common/zstd_internal.h ****/ -/**** skipping file: zstd_decompress_internal.h ****/ -/**** skipping file: zstd_ddict.h ****/ -/**** skipping file: zstd_decompress_block.h ****/ - -/*_******************************************************* -* Macros -**********************************************************/ - -/* These two optional macros force the use one way or another of the two - * ZSTD_decompressSequences implementations. You can't force in both directions - * at the same time. - */ -#if defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT) && \ - defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG) -#error "Cannot force the use of the short and the long ZSTD_decompressSequences variants!" -#endif - - -/*_******************************************************* -* Memory operations -**********************************************************/ -static void ZSTD_copy4(void* dst, const void* src) { memcpy(dst, src, 4); } - - -/*-************************************************************* - * Block decoding - ***************************************************************/ - -/*! ZSTD_getcBlockSize() : - * Provides the size of compressed block from block header `src` */ -size_t ZSTD_getcBlockSize(const void* src, size_t srcSize, - blockProperties_t* bpPtr) -{ - RETURN_ERROR_IF(srcSize < ZSTD_blockHeaderSize, srcSize_wrong, ""); - - { U32 const cBlockHeader = MEM_readLE24(src); - U32 const cSize = cBlockHeader >> 3; - bpPtr->lastBlock = cBlockHeader & 1; - bpPtr->blockType = (blockType_e)((cBlockHeader >> 1) & 3); - bpPtr->origSize = cSize; /* only useful for RLE */ - if (bpPtr->blockType == bt_rle) return 1; - RETURN_ERROR_IF(bpPtr->blockType == bt_reserved, corruption_detected, ""); - return cSize; - } -} - - -/* Hidden declaration for fullbench */ -size_t ZSTD_decodeLiteralsBlock(ZSTD_DCtx* dctx, - const void* src, size_t srcSize); -/*! ZSTD_decodeLiteralsBlock() : - * @return : nb of bytes read from src (< srcSize ) - * note : symbol not declared but exposed for fullbench */ -size_t ZSTD_decodeLiteralsBlock(ZSTD_DCtx* dctx, - const void* src, size_t srcSize) /* note : srcSize < BLOCKSIZE */ -{ - DEBUGLOG(5, "ZSTD_decodeLiteralsBlock"); - RETURN_ERROR_IF(srcSize < MIN_CBLOCK_SIZE, corruption_detected, ""); - - { const BYTE* const istart = (const BYTE*) src; - symbolEncodingType_e const litEncType = (symbolEncodingType_e)(istart[0] & 3); - - switch(litEncType) - { - case set_repeat: - DEBUGLOG(5, "set_repeat flag : re-using stats from previous compressed literals block"); - RETURN_ERROR_IF(dctx->litEntropy==0, dictionary_corrupted, ""); - /* fall-through */ - - case set_compressed: - RETURN_ERROR_IF(srcSize < 5, corruption_detected, "srcSize >= MIN_CBLOCK_SIZE == 3; here we need up to 5 for case 3"); - { size_t lhSize, litSize, litCSize; - U32 singleStream=0; - U32 const lhlCode = (istart[0] >> 2) & 3; - U32 const lhc = MEM_readLE32(istart); - size_t hufSuccess; - switch(lhlCode) - { - case 0: case 1: default: /* note : default is impossible, since lhlCode into [0..3] */ - /* 2 - 2 - 10 - 10 */ - singleStream = !lhlCode; - lhSize = 3; - litSize = (lhc >> 4) & 0x3FF; - litCSize = (lhc >> 14) & 0x3FF; - break; - case 2: - /* 2 - 2 - 14 - 14 */ - lhSize = 4; - litSize = (lhc >> 4) & 0x3FFF; - litCSize = lhc >> 18; - break; - case 3: - /* 2 - 2 - 18 - 18 */ - lhSize = 5; - litSize = (lhc >> 4) & 0x3FFFF; - litCSize = (lhc >> 22) + ((size_t)istart[4] << 10); - break; - } - RETURN_ERROR_IF(litSize > ZSTD_BLOCKSIZE_MAX, corruption_detected, ""); - RETURN_ERROR_IF(litCSize + lhSize > srcSize, corruption_detected, ""); - - /* prefetch huffman table if cold */ - if (dctx->ddictIsCold && (litSize > 768 /* heuristic */)) { - PREFETCH_AREA(dctx->HUFptr, sizeof(dctx->entropy.hufTable)); - } - - if (litEncType==set_repeat) { - if (singleStream) { - hufSuccess = HUF_decompress1X_usingDTable_bmi2( - dctx->litBuffer, litSize, istart+lhSize, litCSize, - dctx->HUFptr, dctx->bmi2); - } else { - hufSuccess = HUF_decompress4X_usingDTable_bmi2( - dctx->litBuffer, litSize, istart+lhSize, litCSize, - dctx->HUFptr, dctx->bmi2); - } - } else { - if (singleStream) { -#if defined(HUF_FORCE_DECOMPRESS_X2) - hufSuccess = HUF_decompress1X_DCtx_wksp( - dctx->entropy.hufTable, dctx->litBuffer, litSize, - istart+lhSize, litCSize, dctx->workspace, - sizeof(dctx->workspace)); -#else - hufSuccess = HUF_decompress1X1_DCtx_wksp_bmi2( - dctx->entropy.hufTable, dctx->litBuffer, litSize, - istart+lhSize, litCSize, dctx->workspace, - sizeof(dctx->workspace), dctx->bmi2); -#endif - } else { - hufSuccess = HUF_decompress4X_hufOnly_wksp_bmi2( - dctx->entropy.hufTable, dctx->litBuffer, litSize, - istart+lhSize, litCSize, dctx->workspace, - sizeof(dctx->workspace), dctx->bmi2); - } - } - - RETURN_ERROR_IF(HUF_isError(hufSuccess), corruption_detected, ""); - - dctx->litPtr = dctx->litBuffer; - dctx->litSize = litSize; - dctx->litEntropy = 1; - if (litEncType==set_compressed) dctx->HUFptr = dctx->entropy.hufTable; - memset(dctx->litBuffer + dctx->litSize, 0, WILDCOPY_OVERLENGTH); - return litCSize + lhSize; - } - - case set_basic: - { size_t litSize, lhSize; - U32 const lhlCode = ((istart[0]) >> 2) & 3; - switch(lhlCode) - { - case 0: case 2: default: /* note : default is impossible, since lhlCode into [0..3] */ - lhSize = 1; - litSize = istart[0] >> 3; - break; - case 1: - lhSize = 2; - litSize = MEM_readLE16(istart) >> 4; - break; - case 3: - lhSize = 3; - litSize = MEM_readLE24(istart) >> 4; - break; - } - - if (lhSize+litSize+WILDCOPY_OVERLENGTH > srcSize) { /* risk reading beyond src buffer with wildcopy */ - RETURN_ERROR_IF(litSize+lhSize > srcSize, corruption_detected, ""); - memcpy(dctx->litBuffer, istart+lhSize, litSize); - dctx->litPtr = dctx->litBuffer; - dctx->litSize = litSize; - memset(dctx->litBuffer + dctx->litSize, 0, WILDCOPY_OVERLENGTH); - return lhSize+litSize; - } - /* direct reference into compressed stream */ - dctx->litPtr = istart+lhSize; - dctx->litSize = litSize; - return lhSize+litSize; - } - - case set_rle: - { U32 const lhlCode = ((istart[0]) >> 2) & 3; - size_t litSize, lhSize; - switch(lhlCode) - { - case 0: case 2: default: /* note : default is impossible, since lhlCode into [0..3] */ - lhSize = 1; - litSize = istart[0] >> 3; - break; - case 1: - lhSize = 2; - litSize = MEM_readLE16(istart) >> 4; - break; - case 3: - lhSize = 3; - litSize = MEM_readLE24(istart) >> 4; - RETURN_ERROR_IF(srcSize<4, corruption_detected, "srcSize >= MIN_CBLOCK_SIZE == 3; here we need lhSize+1 = 4"); - break; - } - RETURN_ERROR_IF(litSize > ZSTD_BLOCKSIZE_MAX, corruption_detected, ""); - memset(dctx->litBuffer, istart[lhSize], litSize + WILDCOPY_OVERLENGTH); - dctx->litPtr = dctx->litBuffer; - dctx->litSize = litSize; - return lhSize+1; - } - default: - RETURN_ERROR(corruption_detected, "impossible"); - } - } -} - -/* Default FSE distribution tables. - * These are pre-calculated FSE decoding tables using default distributions as defined in specification : - * https://github.com/facebook/zstd/blob/master/doc/zstd_compression_format.md#default-distributions - * They were generated programmatically with following method : - * - start from default distributions, present in /lib/common/zstd_internal.h - * - generate tables normally, using ZSTD_buildFSETable() - * - printout the content of tables - * - pretify output, report below, test with fuzzer to ensure it's correct */ - -/* Default FSE distribution table for Literal Lengths */ -static const ZSTD_seqSymbol LL_defaultDTable[(1<tableLog = 0; - DTableH->fastMode = 0; - - cell->nbBits = 0; - cell->nextState = 0; - assert(nbAddBits < 255); - cell->nbAdditionalBits = (BYTE)nbAddBits; - cell->baseValue = baseValue; -} - - -/* ZSTD_buildFSETable() : - * generate FSE decoding table for one symbol (ll, ml or off) - * cannot fail if input is valid => - * all inputs are presumed validated at this stage */ -void -ZSTD_buildFSETable(ZSTD_seqSymbol* dt, - const short* normalizedCounter, unsigned maxSymbolValue, - const U32* baseValue, const U32* nbAdditionalBits, - unsigned tableLog) -{ - ZSTD_seqSymbol* const tableDecode = dt+1; - U16 symbolNext[MaxSeq+1]; - - U32 const maxSV1 = maxSymbolValue + 1; - U32 const tableSize = 1 << tableLog; - U32 highThreshold = tableSize-1; - - /* Sanity Checks */ - assert(maxSymbolValue <= MaxSeq); - assert(tableLog <= MaxFSELog); - - /* Init, lay down lowprob symbols */ - { ZSTD_seqSymbol_header DTableH; - DTableH.tableLog = tableLog; - DTableH.fastMode = 1; - { S16 const largeLimit= (S16)(1 << (tableLog-1)); - U32 s; - for (s=0; s= largeLimit) DTableH.fastMode=0; - assert(normalizedCounter[s]>=0); - symbolNext[s] = (U16)normalizedCounter[s]; - } } } - memcpy(dt, &DTableH, sizeof(DTableH)); - } - - /* Spread symbols */ - { U32 const tableMask = tableSize-1; - U32 const step = FSE_TABLESTEP(tableSize); - U32 s, position = 0; - for (s=0; s highThreshold) position = (position + step) & tableMask; /* lowprob area */ - } } - assert(position == 0); /* position must reach all cells once, otherwise normalizedCounter is incorrect */ - } - - /* Build Decoding table */ - { U32 u; - for (u=0; u max, corruption_detected, ""); - { U32 const symbol = *(const BYTE*)src; - U32 const baseline = baseValue[symbol]; - U32 const nbBits = nbAdditionalBits[symbol]; - ZSTD_buildSeqTable_rle(DTableSpace, baseline, nbBits); - } - *DTablePtr = DTableSpace; - return 1; - case set_basic : - *DTablePtr = defaultTable; - return 0; - case set_repeat: - RETURN_ERROR_IF(!flagRepeatTable, corruption_detected, ""); - /* prefetch FSE table if used */ - if (ddictIsCold && (nbSeq > 24 /* heuristic */)) { - const void* const pStart = *DTablePtr; - size_t const pSize = sizeof(ZSTD_seqSymbol) * (SEQSYMBOL_TABLE_SIZE(maxLog)); - PREFETCH_AREA(pStart, pSize); - } - return 0; - case set_compressed : - { unsigned tableLog; - S16 norm[MaxSeq+1]; - size_t const headerSize = FSE_readNCount(norm, &max, &tableLog, src, srcSize); - RETURN_ERROR_IF(FSE_isError(headerSize), corruption_detected, ""); - RETURN_ERROR_IF(tableLog > maxLog, corruption_detected, ""); - ZSTD_buildFSETable(DTableSpace, norm, max, baseValue, nbAdditionalBits, tableLog); - *DTablePtr = DTableSpace; - return headerSize; - } - default : - assert(0); - RETURN_ERROR(GENERIC, "impossible"); - } -} - -size_t ZSTD_decodeSeqHeaders(ZSTD_DCtx* dctx, int* nbSeqPtr, - const void* src, size_t srcSize) -{ - const BYTE* const istart = (const BYTE* const)src; - const BYTE* const iend = istart + srcSize; - const BYTE* ip = istart; - int nbSeq; - DEBUGLOG(5, "ZSTD_decodeSeqHeaders"); - - /* check */ - RETURN_ERROR_IF(srcSize < MIN_SEQUENCES_SIZE, srcSize_wrong, ""); - - /* SeqHead */ - nbSeq = *ip++; - if (!nbSeq) { - *nbSeqPtr=0; - RETURN_ERROR_IF(srcSize != 1, srcSize_wrong, ""); - return 1; - } - if (nbSeq > 0x7F) { - if (nbSeq == 0xFF) { - RETURN_ERROR_IF(ip+2 > iend, srcSize_wrong, ""); - nbSeq = MEM_readLE16(ip) + LONGNBSEQ, ip+=2; - } else { - RETURN_ERROR_IF(ip >= iend, srcSize_wrong, ""); - nbSeq = ((nbSeq-0x80)<<8) + *ip++; - } - } - *nbSeqPtr = nbSeq; - - /* FSE table descriptors */ - RETURN_ERROR_IF(ip+1 > iend, srcSize_wrong, ""); /* minimum possible size: 1 byte for symbol encoding types */ - { symbolEncodingType_e const LLtype = (symbolEncodingType_e)(*ip >> 6); - symbolEncodingType_e const OFtype = (symbolEncodingType_e)((*ip >> 4) & 3); - symbolEncodingType_e const MLtype = (symbolEncodingType_e)((*ip >> 2) & 3); - ip++; - - /* Build DTables */ - { size_t const llhSize = ZSTD_buildSeqTable(dctx->entropy.LLTable, &dctx->LLTptr, - LLtype, MaxLL, LLFSELog, - ip, iend-ip, - LL_base, LL_bits, - LL_defaultDTable, dctx->fseEntropy, - dctx->ddictIsCold, nbSeq); - RETURN_ERROR_IF(ZSTD_isError(llhSize), corruption_detected, "ZSTD_buildSeqTable failed"); - ip += llhSize; - } - - { size_t const ofhSize = ZSTD_buildSeqTable(dctx->entropy.OFTable, &dctx->OFTptr, - OFtype, MaxOff, OffFSELog, - ip, iend-ip, - OF_base, OF_bits, - OF_defaultDTable, dctx->fseEntropy, - dctx->ddictIsCold, nbSeq); - RETURN_ERROR_IF(ZSTD_isError(ofhSize), corruption_detected, "ZSTD_buildSeqTable failed"); - ip += ofhSize; - } - - { size_t const mlhSize = ZSTD_buildSeqTable(dctx->entropy.MLTable, &dctx->MLTptr, - MLtype, MaxML, MLFSELog, - ip, iend-ip, - ML_base, ML_bits, - ML_defaultDTable, dctx->fseEntropy, - dctx->ddictIsCold, nbSeq); - RETURN_ERROR_IF(ZSTD_isError(mlhSize), corruption_detected, "ZSTD_buildSeqTable failed"); - ip += mlhSize; - } - } - - return ip-istart; -} - - -typedef struct { - size_t litLength; - size_t matchLength; - size_t offset; - const BYTE* match; -} seq_t; - -typedef struct { - size_t state; - const ZSTD_seqSymbol* table; -} ZSTD_fseState; - -typedef struct { - BIT_DStream_t DStream; - ZSTD_fseState stateLL; - ZSTD_fseState stateOffb; - ZSTD_fseState stateML; - size_t prevOffset[ZSTD_REP_NUM]; - const BYTE* prefixStart; - const BYTE* dictEnd; - size_t pos; -} seqState_t; - -/*! ZSTD_overlapCopy8() : - * Copies 8 bytes from ip to op and updates op and ip where ip <= op. - * If the offset is < 8 then the offset is spread to at least 8 bytes. - * - * Precondition: *ip <= *op - * Postcondition: *op - *op >= 8 - */ -HINT_INLINE void ZSTD_overlapCopy8(BYTE** op, BYTE const** ip, size_t offset) { - assert(*ip <= *op); - if (offset < 8) { - /* close range match, overlap */ - static const U32 dec32table[] = { 0, 1, 2, 1, 4, 4, 4, 4 }; /* added */ - static const int dec64table[] = { 8, 8, 8, 7, 8, 9,10,11 }; /* subtracted */ - int const sub2 = dec64table[offset]; - (*op)[0] = (*ip)[0]; - (*op)[1] = (*ip)[1]; - (*op)[2] = (*ip)[2]; - (*op)[3] = (*ip)[3]; - *ip += dec32table[offset]; - ZSTD_copy4(*op+4, *ip); - *ip -= sub2; - } else { - ZSTD_copy8(*op, *ip); - } - *ip += 8; - *op += 8; - assert(*op - *ip >= 8); -} - -/*! ZSTD_safecopy() : - * Specialized version of memcpy() that is allowed to READ up to WILDCOPY_OVERLENGTH past the input buffer - * and write up to 16 bytes past oend_w (op >= oend_w is allowed). - * This function is only called in the uncommon case where the sequence is near the end of the block. It - * should be fast for a single long sequence, but can be slow for several short sequences. - * - * @param ovtype controls the overlap detection - * - ZSTD_no_overlap: The source and destination are guaranteed to be at least WILDCOPY_VECLEN bytes apart. - * - ZSTD_overlap_src_before_dst: The src and dst may overlap and may be any distance apart. - * The src buffer must be before the dst buffer. - */ -static void ZSTD_safecopy(BYTE* op, BYTE* const oend_w, BYTE const* ip, ptrdiff_t length, ZSTD_overlap_e ovtype) { - ptrdiff_t const diff = op - ip; - BYTE* const oend = op + length; - - assert((ovtype == ZSTD_no_overlap && (diff <= -8 || diff >= 8 || op >= oend_w)) || - (ovtype == ZSTD_overlap_src_before_dst && diff >= 0)); - - if (length < 8) { - /* Handle short lengths. */ - while (op < oend) *op++ = *ip++; - return; - } - if (ovtype == ZSTD_overlap_src_before_dst) { - /* Copy 8 bytes and ensure the offset >= 8 when there can be overlap. */ - assert(length >= 8); - ZSTD_overlapCopy8(&op, &ip, diff); - assert(op - ip >= 8); - assert(op <= oend); - } - - if (oend <= oend_w) { - /* No risk of overwrite. */ - ZSTD_wildcopy(op, ip, length, ovtype); - return; - } - if (op <= oend_w) { - /* Wildcopy until we get close to the end. */ - assert(oend > oend_w); - ZSTD_wildcopy(op, ip, oend_w - op, ovtype); - ip += oend_w - op; - op = oend_w; - } - /* Handle the leftovers. */ - while (op < oend) *op++ = *ip++; -} - -/* ZSTD_execSequenceEnd(): - * This version handles cases that are near the end of the output buffer. It requires - * more careful checks to make sure there is no overflow. By separating out these hard - * and unlikely cases, we can speed up the common cases. - * - * NOTE: This function needs to be fast for a single long sequence, but doesn't need - * to be optimized for many small sequences, since those fall into ZSTD_execSequence(). - */ -FORCE_NOINLINE -size_t ZSTD_execSequenceEnd(BYTE* op, - BYTE* const oend, seq_t sequence, - const BYTE** litPtr, const BYTE* const litLimit, - const BYTE* const prefixStart, const BYTE* const virtualStart, const BYTE* const dictEnd) -{ - BYTE* const oLitEnd = op + sequence.litLength; - size_t const sequenceLength = sequence.litLength + sequence.matchLength; - const BYTE* const iLitEnd = *litPtr + sequence.litLength; - const BYTE* match = oLitEnd - sequence.offset; - BYTE* const oend_w = oend - WILDCOPY_OVERLENGTH; - - /* bounds checks : careful of address space overflow in 32-bit mode */ - RETURN_ERROR_IF(sequenceLength > (size_t)(oend - op), dstSize_tooSmall, "last match must fit within dstBuffer"); - RETURN_ERROR_IF(sequence.litLength > (size_t)(litLimit - *litPtr), corruption_detected, "try to read beyond literal buffer"); - assert(op < op + sequenceLength); - assert(oLitEnd < op + sequenceLength); - - /* copy literals */ - ZSTD_safecopy(op, oend_w, *litPtr, sequence.litLength, ZSTD_no_overlap); - op = oLitEnd; - *litPtr = iLitEnd; - - /* copy Match */ - if (sequence.offset > (size_t)(oLitEnd - prefixStart)) { - /* offset beyond prefix */ - RETURN_ERROR_IF(sequence.offset > (size_t)(oLitEnd - virtualStart), corruption_detected, ""); - match = dictEnd - (prefixStart-match); - if (match + sequence.matchLength <= dictEnd) { - memmove(oLitEnd, match, sequence.matchLength); - return sequenceLength; - } - /* span extDict & currentPrefixSegment */ - { size_t const length1 = dictEnd - match; - memmove(oLitEnd, match, length1); - op = oLitEnd + length1; - sequence.matchLength -= length1; - match = prefixStart; - } } - ZSTD_safecopy(op, oend_w, match, sequence.matchLength, ZSTD_overlap_src_before_dst); - return sequenceLength; -} - -HINT_INLINE -size_t ZSTD_execSequence(BYTE* op, - BYTE* const oend, seq_t sequence, - const BYTE** litPtr, const BYTE* const litLimit, - const BYTE* const prefixStart, const BYTE* const virtualStart, const BYTE* const dictEnd) -{ - BYTE* const oLitEnd = op + sequence.litLength; - size_t const sequenceLength = sequence.litLength + sequence.matchLength; - BYTE* const oMatchEnd = op + sequenceLength; /* risk : address space overflow (32-bits) */ - BYTE* const oend_w = oend - WILDCOPY_OVERLENGTH; /* risk : address space underflow on oend=NULL */ - const BYTE* const iLitEnd = *litPtr + sequence.litLength; - const BYTE* match = oLitEnd - sequence.offset; - - assert(op != NULL /* Precondition */); - assert(oend_w < oend /* No underflow */); - /* Handle edge cases in a slow path: - * - Read beyond end of literals - * - Match end is within WILDCOPY_OVERLIMIT of oend - * - 32-bit mode and the match length overflows - */ - if (UNLIKELY( - iLitEnd > litLimit || - oMatchEnd > oend_w || - (MEM_32bits() && (size_t)(oend - op) < sequenceLength + WILDCOPY_OVERLENGTH))) - return ZSTD_execSequenceEnd(op, oend, sequence, litPtr, litLimit, prefixStart, virtualStart, dictEnd); - - /* Assumptions (everything else goes into ZSTD_execSequenceEnd()) */ - assert(op <= oLitEnd /* No overflow */); - assert(oLitEnd < oMatchEnd /* Non-zero match & no overflow */); - assert(oMatchEnd <= oend /* No underflow */); - assert(iLitEnd <= litLimit /* Literal length is in bounds */); - assert(oLitEnd <= oend_w /* Can wildcopy literals */); - assert(oMatchEnd <= oend_w /* Can wildcopy matches */); - - /* Copy Literals: - * Split out litLength <= 16 since it is nearly always true. +1.6% on gcc-9. - * We likely don't need the full 32-byte wildcopy. - */ - assert(WILDCOPY_OVERLENGTH >= 16); - ZSTD_copy16(op, (*litPtr)); - if (UNLIKELY(sequence.litLength > 16)) { - ZSTD_wildcopy(op+16, (*litPtr)+16, sequence.litLength-16, ZSTD_no_overlap); - } - op = oLitEnd; - *litPtr = iLitEnd; /* update for next sequence */ - - /* Copy Match */ - if (sequence.offset > (size_t)(oLitEnd - prefixStart)) { - /* offset beyond prefix -> go into extDict */ - RETURN_ERROR_IF(UNLIKELY(sequence.offset > (size_t)(oLitEnd - virtualStart)), corruption_detected, ""); - match = dictEnd + (match - prefixStart); - if (match + sequence.matchLength <= dictEnd) { - memmove(oLitEnd, match, sequence.matchLength); - return sequenceLength; - } - /* span extDict & currentPrefixSegment */ - { size_t const length1 = dictEnd - match; - memmove(oLitEnd, match, length1); - op = oLitEnd + length1; - sequence.matchLength -= length1; - match = prefixStart; - } } - /* Match within prefix of 1 or more bytes */ - assert(op <= oMatchEnd); - assert(oMatchEnd <= oend_w); - assert(match >= prefixStart); - assert(sequence.matchLength >= 1); - - /* Nearly all offsets are >= WILDCOPY_VECLEN bytes, which means we can use wildcopy - * without overlap checking. - */ - if (LIKELY(sequence.offset >= WILDCOPY_VECLEN)) { - /* We bet on a full wildcopy for matches, since we expect matches to be - * longer than literals (in general). In silesia, ~10% of matches are longer - * than 16 bytes. - */ - ZSTD_wildcopy(op, match, (ptrdiff_t)sequence.matchLength, ZSTD_no_overlap); - return sequenceLength; - } - assert(sequence.offset < WILDCOPY_VECLEN); - - /* Copy 8 bytes and spread the offset to be >= 8. */ - ZSTD_overlapCopy8(&op, &match, sequence.offset); - - /* If the match length is > 8 bytes, then continue with the wildcopy. */ - if (sequence.matchLength > 8) { - assert(op < oMatchEnd); - ZSTD_wildcopy(op, match, (ptrdiff_t)sequence.matchLength-8, ZSTD_overlap_src_before_dst); - } - return sequenceLength; -} - -static void -ZSTD_initFseState(ZSTD_fseState* DStatePtr, BIT_DStream_t* bitD, const ZSTD_seqSymbol* dt) -{ - const void* ptr = dt; - const ZSTD_seqSymbol_header* const DTableH = (const ZSTD_seqSymbol_header*)ptr; - DStatePtr->state = BIT_readBits(bitD, DTableH->tableLog); - DEBUGLOG(6, "ZSTD_initFseState : val=%u using %u bits", - (U32)DStatePtr->state, DTableH->tableLog); - BIT_reloadDStream(bitD); - DStatePtr->table = dt + 1; -} - -FORCE_INLINE_TEMPLATE void -ZSTD_updateFseState(ZSTD_fseState* DStatePtr, BIT_DStream_t* bitD) -{ - ZSTD_seqSymbol const DInfo = DStatePtr->table[DStatePtr->state]; - U32 const nbBits = DInfo.nbBits; - size_t const lowBits = BIT_readBits(bitD, nbBits); - DStatePtr->state = DInfo.nextState + lowBits; -} - -FORCE_INLINE_TEMPLATE void -ZSTD_updateFseStateWithDInfo(ZSTD_fseState* DStatePtr, BIT_DStream_t* bitD, ZSTD_seqSymbol const DInfo) -{ - U32 const nbBits = DInfo.nbBits; - size_t const lowBits = BIT_readBits(bitD, nbBits); - DStatePtr->state = DInfo.nextState + lowBits; -} - -/* We need to add at most (ZSTD_WINDOWLOG_MAX_32 - 1) bits to read the maximum - * offset bits. But we can only read at most (STREAM_ACCUMULATOR_MIN_32 - 1) - * bits before reloading. This value is the maximum number of bytes we read - * after reloading when we are decoding long offsets. - */ -#define LONG_OFFSETS_MAX_EXTRA_BITS_32 \ - (ZSTD_WINDOWLOG_MAX_32 > STREAM_ACCUMULATOR_MIN_32 \ - ? ZSTD_WINDOWLOG_MAX_32 - STREAM_ACCUMULATOR_MIN_32 \ - : 0) - -typedef enum { ZSTD_lo_isRegularOffset, ZSTD_lo_isLongOffset=1 } ZSTD_longOffset_e; -typedef enum { ZSTD_p_noPrefetch=0, ZSTD_p_prefetch=1 } ZSTD_prefetch_e; - -FORCE_INLINE_TEMPLATE seq_t -ZSTD_decodeSequence(seqState_t* seqState, const ZSTD_longOffset_e longOffsets, const ZSTD_prefetch_e prefetch) -{ - seq_t seq; - ZSTD_seqSymbol const llDInfo = seqState->stateLL.table[seqState->stateLL.state]; - ZSTD_seqSymbol const mlDInfo = seqState->stateML.table[seqState->stateML.state]; - ZSTD_seqSymbol const ofDInfo = seqState->stateOffb.table[seqState->stateOffb.state]; - U32 const llBase = llDInfo.baseValue; - U32 const mlBase = mlDInfo.baseValue; - U32 const ofBase = ofDInfo.baseValue; - BYTE const llBits = llDInfo.nbAdditionalBits; - BYTE const mlBits = mlDInfo.nbAdditionalBits; - BYTE const ofBits = ofDInfo.nbAdditionalBits; - BYTE const totalBits = llBits+mlBits+ofBits; - - /* sequence */ - { size_t offset; - if (ofBits > 1) { - ZSTD_STATIC_ASSERT(ZSTD_lo_isLongOffset == 1); - ZSTD_STATIC_ASSERT(LONG_OFFSETS_MAX_EXTRA_BITS_32 == 5); - assert(ofBits <= MaxOff); - if (MEM_32bits() && longOffsets && (ofBits >= STREAM_ACCUMULATOR_MIN_32)) { - U32 const extraBits = ofBits - MIN(ofBits, 32 - seqState->DStream.bitsConsumed); - offset = ofBase + (BIT_readBitsFast(&seqState->DStream, ofBits - extraBits) << extraBits); - BIT_reloadDStream(&seqState->DStream); - if (extraBits) offset += BIT_readBitsFast(&seqState->DStream, extraBits); - assert(extraBits <= LONG_OFFSETS_MAX_EXTRA_BITS_32); /* to avoid another reload */ - } else { - offset = ofBase + BIT_readBitsFast(&seqState->DStream, ofBits/*>0*/); /* <= (ZSTD_WINDOWLOG_MAX-1) bits */ - if (MEM_32bits()) BIT_reloadDStream(&seqState->DStream); - } - seqState->prevOffset[2] = seqState->prevOffset[1]; - seqState->prevOffset[1] = seqState->prevOffset[0]; - seqState->prevOffset[0] = offset; - } else { - U32 const ll0 = (llBase == 0); - if (LIKELY((ofBits == 0))) { - if (LIKELY(!ll0)) - offset = seqState->prevOffset[0]; - else { - offset = seqState->prevOffset[1]; - seqState->prevOffset[1] = seqState->prevOffset[0]; - seqState->prevOffset[0] = offset; - } - } else { - offset = ofBase + ll0 + BIT_readBitsFast(&seqState->DStream, 1); - { size_t temp = (offset==3) ? seqState->prevOffset[0] - 1 : seqState->prevOffset[offset]; - temp += !temp; /* 0 is not valid; input is corrupted; force offset to 1 */ - if (offset != 1) seqState->prevOffset[2] = seqState->prevOffset[1]; - seqState->prevOffset[1] = seqState->prevOffset[0]; - seqState->prevOffset[0] = offset = temp; - } } } - seq.offset = offset; - } - - seq.matchLength = mlBase; - if (mlBits > 0) - seq.matchLength += BIT_readBitsFast(&seqState->DStream, mlBits/*>0*/); - - if (MEM_32bits() && (mlBits+llBits >= STREAM_ACCUMULATOR_MIN_32-LONG_OFFSETS_MAX_EXTRA_BITS_32)) - BIT_reloadDStream(&seqState->DStream); - if (MEM_64bits() && UNLIKELY(totalBits >= STREAM_ACCUMULATOR_MIN_64-(LLFSELog+MLFSELog+OffFSELog))) - BIT_reloadDStream(&seqState->DStream); - /* Ensure there are enough bits to read the rest of data in 64-bit mode. */ - ZSTD_STATIC_ASSERT(16+LLFSELog+MLFSELog+OffFSELog < STREAM_ACCUMULATOR_MIN_64); - - seq.litLength = llBase; - if (llBits > 0) - seq.litLength += BIT_readBitsFast(&seqState->DStream, llBits/*>0*/); - - if (MEM_32bits()) - BIT_reloadDStream(&seqState->DStream); - - DEBUGLOG(6, "seq: litL=%u, matchL=%u, offset=%u", - (U32)seq.litLength, (U32)seq.matchLength, (U32)seq.offset); - - if (prefetch == ZSTD_p_prefetch) { - size_t const pos = seqState->pos + seq.litLength; - const BYTE* const matchBase = (seq.offset > pos) ? seqState->dictEnd : seqState->prefixStart; - seq.match = matchBase + pos - seq.offset; /* note : this operation can overflow when seq.offset is really too large, which can only happen when input is corrupted. - * No consequence though : no memory access will occur, offset is only used for prefetching */ - seqState->pos = pos + seq.matchLength; - } - - /* ANS state update - * gcc-9.0.0 does 2.5% worse with ZSTD_updateFseStateWithDInfo(). - * clang-9.2.0 does 7% worse with ZSTD_updateFseState(). - * Naturally it seems like ZSTD_updateFseStateWithDInfo() should be the - * better option, so it is the default for other compilers. But, if you - * measure that it is worse, please put up a pull request. - */ - { -#if defined(__GNUC__) && !defined(__clang__) - const int kUseUpdateFseState = 1; -#else - const int kUseUpdateFseState = 0; -#endif - if (kUseUpdateFseState) { - ZSTD_updateFseState(&seqState->stateLL, &seqState->DStream); /* <= 9 bits */ - ZSTD_updateFseState(&seqState->stateML, &seqState->DStream); /* <= 9 bits */ - if (MEM_32bits()) BIT_reloadDStream(&seqState->DStream); /* <= 18 bits */ - ZSTD_updateFseState(&seqState->stateOffb, &seqState->DStream); /* <= 8 bits */ - } else { - ZSTD_updateFseStateWithDInfo(&seqState->stateLL, &seqState->DStream, llDInfo); /* <= 9 bits */ - ZSTD_updateFseStateWithDInfo(&seqState->stateML, &seqState->DStream, mlDInfo); /* <= 9 bits */ - if (MEM_32bits()) BIT_reloadDStream(&seqState->DStream); /* <= 18 bits */ - ZSTD_updateFseStateWithDInfo(&seqState->stateOffb, &seqState->DStream, ofDInfo); /* <= 8 bits */ - } - } - - return seq; -} - -#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -static int ZSTD_dictionaryIsActive(ZSTD_DCtx const* dctx, BYTE const* prefixStart, BYTE const* oLitEnd) -{ - size_t const windowSize = dctx->fParams.windowSize; - /* No dictionary used. */ - if (dctx->dictContentEndForFuzzing == NULL) return 0; - /* Dictionary is our prefix. */ - if (prefixStart == dctx->dictContentBeginForFuzzing) return 1; - /* Dictionary is not our ext-dict. */ - if (dctx->dictEnd != dctx->dictContentEndForFuzzing) return 0; - /* Dictionary is not within our window size. */ - if ((size_t)(oLitEnd - prefixStart) >= windowSize) return 0; - /* Dictionary is active. */ - return 1; -} - -MEM_STATIC void ZSTD_assertValidSequence( - ZSTD_DCtx const* dctx, - BYTE const* op, BYTE const* oend, - seq_t const seq, - BYTE const* prefixStart, BYTE const* virtualStart) -{ - size_t const windowSize = dctx->fParams.windowSize; - size_t const sequenceSize = seq.litLength + seq.matchLength; - BYTE const* const oLitEnd = op + seq.litLength; - DEBUGLOG(6, "Checking sequence: litL=%u matchL=%u offset=%u", - (U32)seq.litLength, (U32)seq.matchLength, (U32)seq.offset); - assert(op <= oend); - assert((size_t)(oend - op) >= sequenceSize); - assert(sequenceSize <= ZSTD_BLOCKSIZE_MAX); - if (ZSTD_dictionaryIsActive(dctx, prefixStart, oLitEnd)) { - size_t const dictSize = (size_t)((char const*)dctx->dictContentEndForFuzzing - (char const*)dctx->dictContentBeginForFuzzing); - /* Offset must be within the dictionary. */ - assert(seq.offset <= (size_t)(oLitEnd - virtualStart)); - assert(seq.offset <= windowSize + dictSize); - } else { - /* Offset must be within our window. */ - assert(seq.offset <= windowSize); - } -} -#endif - -#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG -FORCE_INLINE_TEMPLATE size_t -DONT_VECTORIZE -ZSTD_decompressSequences_body( ZSTD_DCtx* dctx, - void* dst, size_t maxDstSize, - const void* seqStart, size_t seqSize, int nbSeq, - const ZSTD_longOffset_e isLongOffset, - const int frame) -{ - const BYTE* ip = (const BYTE*)seqStart; - const BYTE* const iend = ip + seqSize; - BYTE* const ostart = (BYTE* const)dst; - BYTE* const oend = ostart + maxDstSize; - BYTE* op = ostart; - const BYTE* litPtr = dctx->litPtr; - const BYTE* const litEnd = litPtr + dctx->litSize; - const BYTE* const prefixStart = (const BYTE*) (dctx->prefixStart); - const BYTE* const vBase = (const BYTE*) (dctx->virtualStart); - const BYTE* const dictEnd = (const BYTE*) (dctx->dictEnd); - DEBUGLOG(5, "ZSTD_decompressSequences_body"); - (void)frame; - - /* Regen sequences */ - if (nbSeq) { - seqState_t seqState; - size_t error = 0; - dctx->fseEntropy = 1; - { U32 i; for (i=0; ientropy.rep[i]; } - RETURN_ERROR_IF( - ERR_isError(BIT_initDStream(&seqState.DStream, ip, iend-ip)), - corruption_detected, ""); - ZSTD_initFseState(&seqState.stateLL, &seqState.DStream, dctx->LLTptr); - ZSTD_initFseState(&seqState.stateOffb, &seqState.DStream, dctx->OFTptr); - ZSTD_initFseState(&seqState.stateML, &seqState.DStream, dctx->MLTptr); - assert(dst != NULL); - - ZSTD_STATIC_ASSERT( - BIT_DStream_unfinished < BIT_DStream_completed && - BIT_DStream_endOfBuffer < BIT_DStream_completed && - BIT_DStream_completed < BIT_DStream_overflow); - -#if defined(__GNUC__) && defined(__x86_64__) - /* Align the decompression loop to 32 + 16 bytes. - * - * zstd compiled with gcc-9 on an Intel i9-9900k shows 10% decompression - * speed swings based on the alignment of the decompression loop. This - * performance swing is caused by parts of the decompression loop falling - * out of the DSB. The entire decompression loop should fit in the DSB, - * when it can't we get much worse performance. You can measure if you've - * hit the good case or the bad case with this perf command for some - * compressed file test.zst: - * - * perf stat -e cycles -e instructions -e idq.all_dsb_cycles_any_uops \ - * -e idq.all_mite_cycles_any_uops -- ./zstd -tq test.zst - * - * If you see most cycles served out of the MITE you've hit the bad case. - * If you see most cycles served out of the DSB you've hit the good case. - * If it is pretty even then you may be in an okay case. - * - * I've been able to reproduce this issue on the following CPUs: - * - Kabylake: Macbook Pro (15-inch, 2019) 2.4 GHz Intel Core i9 - * Use Instruments->Counters to get DSB/MITE cycles. - * I never got performance swings, but I was able to - * go from the good case of mostly DSB to half of the - * cycles served from MITE. - * - Coffeelake: Intel i9-9900k - * - * I haven't been able to reproduce the instability or DSB misses on any - * of the following CPUS: - * - Haswell - * - Broadwell: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GH - * - Skylake - * - * If you are seeing performance stability this script can help test. - * It tests on 4 commits in zstd where I saw performance change. - * - * https://gist.github.com/terrelln/9889fc06a423fd5ca6e99351564473f4 - */ - __asm__(".p2align 5"); - __asm__("nop"); - __asm__(".p2align 4"); -#endif - for ( ; ; ) { - seq_t const sequence = ZSTD_decodeSequence(&seqState, isLongOffset, ZSTD_p_noPrefetch); - size_t const oneSeqSize = ZSTD_execSequence(op, oend, sequence, &litPtr, litEnd, prefixStart, vBase, dictEnd); -#if defined(FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION) && defined(FUZZING_ASSERT_VALID_SEQUENCE) - assert(!ZSTD_isError(oneSeqSize)); - if (frame) ZSTD_assertValidSequence(dctx, op, oend, sequence, prefixStart, vBase); -#endif - DEBUGLOG(6, "regenerated sequence size : %u", (U32)oneSeqSize); - BIT_reloadDStream(&(seqState.DStream)); - /* gcc and clang both don't like early returns in this loop. - * gcc doesn't like early breaks either. - * Instead save an error and report it at the end. - * When there is an error, don't increment op, so we don't - * overwrite. - */ - if (UNLIKELY(ZSTD_isError(oneSeqSize))) error = oneSeqSize; - else op += oneSeqSize; - if (UNLIKELY(!--nbSeq)) break; - } - - /* check if reached exact end */ - DEBUGLOG(5, "ZSTD_decompressSequences_body: after decode loop, remaining nbSeq : %i", nbSeq); - if (ZSTD_isError(error)) return error; - RETURN_ERROR_IF(nbSeq, corruption_detected, ""); - RETURN_ERROR_IF(BIT_reloadDStream(&seqState.DStream) < BIT_DStream_completed, corruption_detected, ""); - /* save reps for next block */ - { U32 i; for (i=0; ientropy.rep[i] = (U32)(seqState.prevOffset[i]); } - } - - /* last literal segment */ - { size_t const lastLLSize = litEnd - litPtr; - RETURN_ERROR_IF(lastLLSize > (size_t)(oend-op), dstSize_tooSmall, ""); - if (op != NULL) { - memcpy(op, litPtr, lastLLSize); - op += lastLLSize; - } - } - - return op-ostart; -} - -static size_t -ZSTD_decompressSequences_default(ZSTD_DCtx* dctx, - void* dst, size_t maxDstSize, - const void* seqStart, size_t seqSize, int nbSeq, - const ZSTD_longOffset_e isLongOffset, - const int frame) -{ - return ZSTD_decompressSequences_body(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); -} -#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG */ - -#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT -FORCE_INLINE_TEMPLATE size_t -ZSTD_decompressSequencesLong_body( - ZSTD_DCtx* dctx, - void* dst, size_t maxDstSize, - const void* seqStart, size_t seqSize, int nbSeq, - const ZSTD_longOffset_e isLongOffset, - const int frame) -{ - const BYTE* ip = (const BYTE*)seqStart; - const BYTE* const iend = ip + seqSize; - BYTE* const ostart = (BYTE* const)dst; - BYTE* const oend = ostart + maxDstSize; - BYTE* op = ostart; - const BYTE* litPtr = dctx->litPtr; - const BYTE* const litEnd = litPtr + dctx->litSize; - const BYTE* const prefixStart = (const BYTE*) (dctx->prefixStart); - const BYTE* const dictStart = (const BYTE*) (dctx->virtualStart); - const BYTE* const dictEnd = (const BYTE*) (dctx->dictEnd); - (void)frame; - - /* Regen sequences */ - if (nbSeq) { -#define STORED_SEQS 4 -#define STORED_SEQS_MASK (STORED_SEQS-1) -#define ADVANCED_SEQS 4 - seq_t sequences[STORED_SEQS]; - int const seqAdvance = MIN(nbSeq, ADVANCED_SEQS); - seqState_t seqState; - int seqNb; - dctx->fseEntropy = 1; - { int i; for (i=0; ientropy.rep[i]; } - seqState.prefixStart = prefixStart; - seqState.pos = (size_t)(op-prefixStart); - seqState.dictEnd = dictEnd; - assert(dst != NULL); - assert(iend >= ip); - RETURN_ERROR_IF( - ERR_isError(BIT_initDStream(&seqState.DStream, ip, iend-ip)), - corruption_detected, ""); - ZSTD_initFseState(&seqState.stateLL, &seqState.DStream, dctx->LLTptr); - ZSTD_initFseState(&seqState.stateOffb, &seqState.DStream, dctx->OFTptr); - ZSTD_initFseState(&seqState.stateML, &seqState.DStream, dctx->MLTptr); - - /* prepare in advance */ - for (seqNb=0; (BIT_reloadDStream(&seqState.DStream) <= BIT_DStream_completed) && (seqNbentropy.rep[i] = (U32)(seqState.prevOffset[i]); } - } - - /* last literal segment */ - { size_t const lastLLSize = litEnd - litPtr; - RETURN_ERROR_IF(lastLLSize > (size_t)(oend-op), dstSize_tooSmall, ""); - if (op != NULL) { - memcpy(op, litPtr, lastLLSize); - op += lastLLSize; - } - } - - return op-ostart; -} - -static size_t -ZSTD_decompressSequencesLong_default(ZSTD_DCtx* dctx, - void* dst, size_t maxDstSize, - const void* seqStart, size_t seqSize, int nbSeq, - const ZSTD_longOffset_e isLongOffset, - const int frame) -{ - return ZSTD_decompressSequencesLong_body(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); -} -#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT */ - - - -#if DYNAMIC_BMI2 - -#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG -static TARGET_ATTRIBUTE("bmi2") size_t -DONT_VECTORIZE -ZSTD_decompressSequences_bmi2(ZSTD_DCtx* dctx, - void* dst, size_t maxDstSize, - const void* seqStart, size_t seqSize, int nbSeq, - const ZSTD_longOffset_e isLongOffset, - const int frame) -{ - return ZSTD_decompressSequences_body(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); -} -#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG */ - -#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT -static TARGET_ATTRIBUTE("bmi2") size_t -ZSTD_decompressSequencesLong_bmi2(ZSTD_DCtx* dctx, - void* dst, size_t maxDstSize, - const void* seqStart, size_t seqSize, int nbSeq, - const ZSTD_longOffset_e isLongOffset, - const int frame) -{ - return ZSTD_decompressSequencesLong_body(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); -} -#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT */ - -#endif /* DYNAMIC_BMI2 */ - -typedef size_t (*ZSTD_decompressSequences_t)( - ZSTD_DCtx* dctx, - void* dst, size_t maxDstSize, - const void* seqStart, size_t seqSize, int nbSeq, - const ZSTD_longOffset_e isLongOffset, - const int frame); - -#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG -static size_t -ZSTD_decompressSequences(ZSTD_DCtx* dctx, void* dst, size_t maxDstSize, - const void* seqStart, size_t seqSize, int nbSeq, - const ZSTD_longOffset_e isLongOffset, - const int frame) -{ - DEBUGLOG(5, "ZSTD_decompressSequences"); -#if DYNAMIC_BMI2 - if (dctx->bmi2) { - return ZSTD_decompressSequences_bmi2(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); - } -#endif - return ZSTD_decompressSequences_default(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); -} -#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG */ - - -#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT -/* ZSTD_decompressSequencesLong() : - * decompression function triggered when a minimum share of offsets is considered "long", - * aka out of cache. - * note : "long" definition seems overloaded here, sometimes meaning "wider than bitstream register", and sometimes meaning "farther than memory cache distance". - * This function will try to mitigate main memory latency through the use of prefetching */ -static size_t -ZSTD_decompressSequencesLong(ZSTD_DCtx* dctx, - void* dst, size_t maxDstSize, - const void* seqStart, size_t seqSize, int nbSeq, - const ZSTD_longOffset_e isLongOffset, - const int frame) -{ - DEBUGLOG(5, "ZSTD_decompressSequencesLong"); -#if DYNAMIC_BMI2 - if (dctx->bmi2) { - return ZSTD_decompressSequencesLong_bmi2(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); - } -#endif - return ZSTD_decompressSequencesLong_default(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame); -} -#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT */ - - - -#if !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT) && \ - !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG) -/* ZSTD_getLongOffsetsShare() : - * condition : offTable must be valid - * @return : "share" of long offsets (arbitrarily defined as > (1<<23)) - * compared to maximum possible of (1< 22) total += 1; - } - - assert(tableLog <= OffFSELog); - total <<= (OffFSELog - tableLog); /* scale to OffFSELog */ - - return total; -} -#endif - -size_t -ZSTD_decompressBlock_internal(ZSTD_DCtx* dctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize, const int frame) -{ /* blockType == blockCompressed */ - const BYTE* ip = (const BYTE*)src; - /* isLongOffset must be true if there are long offsets. - * Offsets are long if they are larger than 2^STREAM_ACCUMULATOR_MIN. - * We don't expect that to be the case in 64-bit mode. - * In block mode, window size is not known, so we have to be conservative. - * (note: but it could be evaluated from current-lowLimit) - */ - ZSTD_longOffset_e const isLongOffset = (ZSTD_longOffset_e)(MEM_32bits() && (!frame || (dctx->fParams.windowSize > (1ULL << STREAM_ACCUMULATOR_MIN)))); - DEBUGLOG(5, "ZSTD_decompressBlock_internal (size : %u)", (U32)srcSize); - - RETURN_ERROR_IF(srcSize >= ZSTD_BLOCKSIZE_MAX, srcSize_wrong, ""); - - /* Decode literals section */ - { size_t const litCSize = ZSTD_decodeLiteralsBlock(dctx, src, srcSize); - DEBUGLOG(5, "ZSTD_decodeLiteralsBlock : %u", (U32)litCSize); - if (ZSTD_isError(litCSize)) return litCSize; - ip += litCSize; - srcSize -= litCSize; - } - - /* Build Decoding Tables */ - { - /* These macros control at build-time which decompressor implementation - * we use. If neither is defined, we do some inspection and dispatch at - * runtime. - */ -#if !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT) && \ - !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG) - int usePrefetchDecoder = dctx->ddictIsCold; -#endif - int nbSeq; - size_t const seqHSize = ZSTD_decodeSeqHeaders(dctx, &nbSeq, ip, srcSize); - if (ZSTD_isError(seqHSize)) return seqHSize; - ip += seqHSize; - srcSize -= seqHSize; - - RETURN_ERROR_IF(dst == NULL && nbSeq > 0, dstSize_tooSmall, "NULL not handled"); - -#if !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT) && \ - !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG) - if ( !usePrefetchDecoder - && (!frame || (dctx->fParams.windowSize > (1<<24))) - && (nbSeq>ADVANCED_SEQS) ) { /* could probably use a larger nbSeq limit */ - U32 const shareLongOffsets = ZSTD_getLongOffsetsShare(dctx->OFTptr); - U32 const minShare = MEM_64bits() ? 7 : 20; /* heuristic values, correspond to 2.73% and 7.81% */ - usePrefetchDecoder = (shareLongOffsets >= minShare); - } -#endif - - dctx->ddictIsCold = 0; - -#if !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT) && \ - !defined(ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG) - if (usePrefetchDecoder) -#endif -#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT - return ZSTD_decompressSequencesLong(dctx, dst, dstCapacity, ip, srcSize, nbSeq, isLongOffset, frame); -#endif - -#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG - /* else */ - return ZSTD_decompressSequences(dctx, dst, dstCapacity, ip, srcSize, nbSeq, isLongOffset, frame); -#endif - } -} - - -void ZSTD_checkContinuity(ZSTD_DCtx* dctx, const void* dst) -{ - if (dst != dctx->previousDstEnd) { /* not contiguous */ - dctx->dictEnd = dctx->previousDstEnd; - dctx->virtualStart = (const char*)dst - ((const char*)(dctx->previousDstEnd) - (const char*)(dctx->prefixStart)); - dctx->prefixStart = dst; - dctx->previousDstEnd = dst; - } -} - - -size_t ZSTD_decompressBlock(ZSTD_DCtx* dctx, - void* dst, size_t dstCapacity, - const void* src, size_t srcSize) -{ - size_t dSize; - ZSTD_checkContinuity(dctx, dst); - dSize = ZSTD_decompressBlock_internal(dctx, dst, dstCapacity, src, srcSize, /* frame */ 0); - dctx->previousDstEnd = (char*)dst + dSize; - return dSize; -} -/**** ended inlining decompress/zstd_decompress_block.c ****/ diff --git a/module/zstd/lib/zstd.h b/module/zstd/lib/zstd.h index b6772f8818a7..8c6fc6ae90e6 100644 --- a/module/zstd/lib/zstd.h +++ b/module/zstd/lib/zstd.h @@ -1,37 +1,12 @@ /* - * BSD 3-Clause Clear License + * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. + * All rights reserved. * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * 1. Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * - * 2. Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * - * 3. Neither the name of the copyright holder nor the names of its - * contributors may be used to endorse or promote products derived from this - * software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" - * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR - * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS - * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN - * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE - * POSSIBILITY OF SUCH DAMAGE. + * This source code is licensed under both the BSD-style license (found in the + * LICENSE file in the root directory of this source tree) and the GPLv2 (found + * in the COPYING file in the root directory of this source tree). + * You may select, at your option, one of the above-listed licenses. */ - -/* - * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc. All rights reserved. - */ - #if defined (__cplusplus) extern "C" { #endif diff --git a/module/zstd/zfs_zstd.c b/module/zstd/zfs_zstd.c index 9ff9ec3ec73c..41351898981a 100644 --- a/module/zstd/zfs_zstd.c +++ b/module/zstd/zfs_zstd.c @@ -48,9 +48,13 @@ #define ZSTD_STATIC_LINKING_ONLY #include "lib/zstd.h" -#include "lib/zstd_errors.h" +#include "lib/common/zstd_errors.h" -kstat_t *zstd_ksp = NULL; +static int zstd_earlyabort_pass = 1; +static int zstd_cutoff_level = ZIO_ZSTD_LEVEL_3; +static unsigned int zstd_abort_size = (128 * 1024); + +static kstat_t *zstd_ksp = NULL; typedef struct zstd_stats { kstat_named_t zstd_stat_alloc_fail; @@ -62,6 +66,21 @@ typedef struct zstd_stats { kstat_named_t zstd_stat_dec_header_inval; kstat_named_t zstd_stat_com_fail; kstat_named_t zstd_stat_dec_fail; + /* + * LZ4 first-pass early abort verdict + */ + kstat_named_t zstd_stat_lz4pass_allowed; + kstat_named_t zstd_stat_lz4pass_rejected; + /* + * zstd-1 second-pass early abort verdict + */ + kstat_named_t zstd_stat_zstdpass_allowed; + kstat_named_t zstd_stat_zstdpass_rejected; + /* + * We excluded this from early abort for some reason + */ + kstat_named_t zstd_stat_passignored; + kstat_named_t zstd_stat_passignored_size; kstat_named_t zstd_stat_buffers; kstat_named_t zstd_stat_size; } zstd_stats_t; @@ -76,10 +95,44 @@ static zstd_stats_t zstd_stats = { { "decompress_header_invalid", KSTAT_DATA_UINT64 }, { "compress_failed", KSTAT_DATA_UINT64 }, { "decompress_failed", KSTAT_DATA_UINT64 }, + { "lz4pass_allowed", KSTAT_DATA_UINT64 }, + { "lz4pass_rejected", KSTAT_DATA_UINT64 }, + { "zstdpass_allowed", KSTAT_DATA_UINT64 }, + { "zstdpass_rejected", KSTAT_DATA_UINT64 }, + { "passignored", KSTAT_DATA_UINT64 }, + { "passignored_size", KSTAT_DATA_UINT64 }, { "buffers", KSTAT_DATA_UINT64 }, { "size", KSTAT_DATA_UINT64 }, }; +#ifdef _KERNEL +static int +kstat_zstd_update(kstat_t *ksp, int rw) +{ + ASSERT(ksp != NULL); + + if (rw == KSTAT_WRITE && ksp == zstd_ksp) { + ZSTDSTAT_ZERO(zstd_stat_alloc_fail); + ZSTDSTAT_ZERO(zstd_stat_alloc_fallback); + ZSTDSTAT_ZERO(zstd_stat_com_alloc_fail); + ZSTDSTAT_ZERO(zstd_stat_dec_alloc_fail); + ZSTDSTAT_ZERO(zstd_stat_com_inval); + ZSTDSTAT_ZERO(zstd_stat_dec_inval); + ZSTDSTAT_ZERO(zstd_stat_dec_header_inval); + ZSTDSTAT_ZERO(zstd_stat_com_fail); + ZSTDSTAT_ZERO(zstd_stat_dec_fail); + ZSTDSTAT_ZERO(zstd_stat_lz4pass_allowed); + ZSTDSTAT_ZERO(zstd_stat_lz4pass_rejected); + ZSTDSTAT_ZERO(zstd_stat_zstdpass_allowed); + ZSTDSTAT_ZERO(zstd_stat_zstdpass_rejected); + ZSTDSTAT_ZERO(zstd_stat_passignored); + ZSTDSTAT_ZERO(zstd_stat_passignored_size); + } + + return (0); +} +#endif + /* Enums describing the allocator type specified by kmem_type in zstd_kmem */ enum zstd_kmem_type { ZSTD_KMEM_UNKNOWN = 0, @@ -377,6 +430,64 @@ zstd_enum_to_level(enum zio_zstd_levels level, int16_t *zstd_level) } +size_t +zfs_zstd_compress_wrap(void *s_start, void *d_start, size_t s_len, size_t d_len, + int level) +{ + int16_t zstd_level; + if (zstd_enum_to_level(level, &zstd_level)) { + ZSTDSTAT_BUMP(zstd_stat_com_inval); + return (s_len); + } + /* + * A zstd early abort heuristic. + * + * - Zeroth, if this is <= zstd-3, or < zstd_abort_size (currently + * 128k), don't try any of this, just go. + * (because experimentally that was a reasonable cutoff for a perf win + * with tiny ratio change) + * - First, we try LZ4 compression, and if it doesn't early abort, we + * jump directly to whatever compression level we intended to try. + * - Second, we try zstd-1 - if that errors out (usually, but not + * exclusively, if it would overflow), we give up early. + * + * If it works, instead we go on and compress anyway. + * + * Why two passes? LZ4 alone gets you a lot of the way, but on highly + * compressible data, it was losing up to 8.5% of the compressed + * savings versus no early abort, and all the zstd-fast levels are + * worse indications on their own than LZ4, and don't improve the LZ4 + * pass noticably if stacked like this. + */ + size_t actual_abort_size = zstd_abort_size; + if (zstd_earlyabort_pass > 0 && zstd_level >= zstd_cutoff_level && + s_len >= actual_abort_size) { + int pass_len = 1; + pass_len = lz4_compress_zfs(s_start, d_start, s_len, d_len, 0); + if (pass_len < d_len) { + ZSTDSTAT_BUMP(zstd_stat_lz4pass_allowed); + goto keep_trying; + } + ZSTDSTAT_BUMP(zstd_stat_lz4pass_rejected); + + pass_len = zfs_zstd_compress(s_start, d_start, s_len, d_len, + ZIO_ZSTD_LEVEL_1); + if (pass_len == s_len || pass_len <= 0 || pass_len > d_len) { + ZSTDSTAT_BUMP(zstd_stat_zstdpass_rejected); + return (s_len); + } + ZSTDSTAT_BUMP(zstd_stat_zstdpass_allowed); + } else { + ZSTDSTAT_BUMP(zstd_stat_passignored); + if (s_len < actual_abort_size) { + ZSTDSTAT_BUMP(zstd_stat_passignored_size); + } + } +keep_trying: + return (zfs_zstd_compress(s_start, d_start, s_len, d_len, level)); + +} + /* Compress block using zstd */ size_t zfs_zstd_compress(void *s_start, void *d_start, size_t s_len, size_t d_len, @@ -437,8 +548,10 @@ zfs_zstd_compress(void *s_start, void *d_start, size_t s_len, size_t d_len, * too small, that is not a failure. Everything else is a * failure, so increment the compression failure counter. */ - if (ZSTD_getErrorCode(c_len) != ZSTD_error_dstSize_tooSmall) { + int err = ZSTD_getErrorCode(c_len); + if (err != ZSTD_error_dstSize_tooSmall) { ZSTDSTAT_BUMP(zstd_stat_com_fail); + dprintf("Error: %s", ZSTD_getErrorString(err)); } return (s_len); } @@ -702,7 +815,7 @@ zstd_meminit(void) } /* Release object from pool and free memory */ -static void __exit +static void release_pool(struct zstd_pool *pool) { mutex_destroy(&pool->barrier); @@ -712,7 +825,7 @@ release_pool(struct zstd_pool *pool) } /* Release memory pool objects */ -static void __exit +static void zstd_mempool_deinit(void) { for (int i = 0; i < ZSTD_POOL_MAX; i++) { @@ -753,12 +866,15 @@ zstd_init(void) if (zstd_ksp != NULL) { zstd_ksp->ks_data = &zstd_stats; kstat_install(zstd_ksp); +#ifdef _KERNEL + zstd_ksp->ks_update = kstat_zstd_update; +#endif } return (0); } -extern void __exit +extern void zstd_fini(void) { /* Deinitialize kstat */ @@ -776,15 +892,13 @@ zstd_fini(void) } #if defined(_KERNEL) +#ifdef __FreeBSD__ module_init(zstd_init); module_exit(zstd_fini); +#endif -ZFS_MODULE_DESCRIPTION("ZSTD Compression for ZFS"); -ZFS_MODULE_LICENSE("Dual BSD/GPL"); -ZFS_MODULE_VERSION(ZSTD_VERSION_STRING "a"); - -EXPORT_SYMBOL(zfs_zstd_compress); -EXPORT_SYMBOL(zfs_zstd_decompress_level); -EXPORT_SYMBOL(zfs_zstd_decompress); -EXPORT_SYMBOL(zfs_zstd_cache_reap_now); +ZFS_MODULE_PARAM(zfs, zstd_, earlyabort_pass, INT, ZMOD_RW, + "Enable early abort attempts when using zstd"); +ZFS_MODULE_PARAM(zfs, zstd_, abort_size, UINT, ZMOD_RW, + "Minimal size of block to attempt early abort"); #endif diff --git a/rpm/.gitignore b/rpm/.gitignore new file mode 100644 index 000000000000..f83960d1a7c0 --- /dev/null +++ b/rpm/.gitignore @@ -0,0 +1 @@ +/*/*.spec diff --git a/rpm/Makefile.am b/rpm/Makefile.am index f2cf72cef13c..af7b25021eac 100644 --- a/rpm/Makefile.am +++ b/rpm/Makefile.am @@ -1 +1,8 @@ -SUBDIRS = generic redhat +dist_noinst_DATA += \ + %D%/generic/zfs-dkms.spec.in \ + %D%/generic/zfs-kmod.spec.in \ + %D%/generic/zfs.spec.in \ + \ + %D%/redhat/zfs-dkms.spec.in \ + %D%/redhat/zfs-kmod.spec.in \ + %D%/redhat/zfs.spec.in diff --git a/rpm/generic/.gitignore b/rpm/generic/.gitignore deleted file mode 100644 index 7f5daafdd6d4..000000000000 --- a/rpm/generic/.gitignore +++ /dev/null @@ -1,3 +0,0 @@ -/zfs-dkms.spec -/zfs-kmod.spec -/zfs.spec diff --git a/rpm/generic/Makefile.am b/rpm/generic/Makefile.am deleted file mode 100644 index 89b13640d622..000000000000 --- a/rpm/generic/Makefile.am +++ /dev/null @@ -1 +0,0 @@ -EXTRA_DIST = zfs.spec.in zfs-kmod.spec.in zfs-dkms.spec.in diff --git a/rpm/generic/zfs-dkms.spec.in b/rpm/generic/zfs-dkms.spec.in index 02be716aa964..196d9a140b4e 100644 --- a/rpm/generic/zfs-dkms.spec.in +++ b/rpm/generic/zfs-dkms.spec.in @@ -31,12 +31,12 @@ Requires(post): gcc, make, perl, diffutils %if 0%{?rhel}%{?fedora}%{?mageia}%{?suse_version} Requires: kernel-devel >= @ZFS_META_KVER_MIN@, kernel-devel <= @ZFS_META_KVER_MAX@.999 Requires(post): kernel-devel >= @ZFS_META_KVER_MIN@, kernel-devel <= @ZFS_META_KVER_MAX@.999 -Obsoletes: spl-dkms +Obsoletes: spl-dkms <= %{version} %endif Provides: %{module}-kmod = %{version} AutoReqProv: no -%if 0%{?rhel}%{?fedora}%{?suse_version} +%if (0%{?fedora}%{?suse_version}) || (0%{?rhel} && 0%{?rhel} < 9) # We don't directly use it, but if this isn't installed, rpmbuild as root can # crash+corrupt rpmdb # See issue #12071 @@ -68,46 +68,9 @@ fi %defattr(-,root,root) /usr/src/%{module}-%{version} -%post -for POSTINST in /usr/lib/dkms/common.postinst; do - if [ -f $POSTINST ]; then - $POSTINST %{module} %{version} - exit $? - fi - echo "WARNING: $POSTINST does not exist." -done -echo -e "ERROR: DKMS version is too old and %{module} was not" -echo -e "built with legacy DKMS support." -echo -e "You must either rebuild %{module} with legacy postinst" -echo -e "support or upgrade DKMS to a more current version." -exit 1 - %preun -# Are we doing an upgrade? -if [ "$1" = "1" -o "$1" = "upgrade" ] ; then - # Yes we are. Are we upgrading to a new ZFS version? - NEWEST_VER=$(dkms status zfs | tr -d , | sort -r -V | awk '/installed/{print $2; exit}') - if [ "$NEWEST_VER" != "%{version}" ] ; then - # Yes, it's a new ZFS version. We'll uninstall the old module - # later on in this script. - true - else - # No, it's probably an upgrade of the same ZFS version - # to a new distro (zfs-dkms-0.7.12.fc28->zfs-dkms-0.7.12.fc29). - # Don't remove our modules, since the rebuild for the new - # distro will automatically delete the old modules. - exit 0 - fi -fi +dkms remove -m %{module} -v %{version} --all + +%posttrans +/usr/lib/dkms/common.postinst %{module} %{version} -# If we're here then we're doing an uninstall (not upgrade). -CONFIG_H="/var/lib/dkms/%{module}/%{version}/*/*/%{module}_config.h" -SPEC_META_ALIAS="@PACKAGE@-@VERSION@-@RELEASE@" -DKMS_META_ALIAS=`cat $CONFIG_H 2>/dev/null | - awk -F'"' '/META_ALIAS\s+"/ { print $2; exit 0 }'` -if [ "$SPEC_META_ALIAS" = "$DKMS_META_ALIAS" ]; then - echo -e - echo -e "Uninstall of %{module} module ($SPEC_META_ALIAS) beginning:" - dkms remove -m %{module} -v %{version} --all %{!?not_rpm:--rpm_safe_upgrade} -fi -exit 0 diff --git a/rpm/generic/zfs-kmod.spec.in b/rpm/generic/zfs-kmod.spec.in index 53b1e1385159..ae0795427868 100644 --- a/rpm/generic/zfs-kmod.spec.in +++ b/rpm/generic/zfs-kmod.spec.in @@ -57,7 +57,7 @@ BuildRequires: gcc, make BuildRequires: elfutils-libelf-devel %endif -%if 0%{?rhel}%{?fedora}%{?suse_version} +%if (0%{?fedora}%{?suse_version}) || (0%{?rhel} && 0%{?rhel} < 9) # We don't directly use it, but if this isn't installed, rpmbuild as root can # crash+corrupt rpmdb # See issue #12071 @@ -98,7 +98,7 @@ BuildRequires: %{_bindir}/kmodtool # Kmodtool does its magic here. A patched version of kmodtool is shipped # with the source rpm until kmod development packages are supported upstream. # https://bugzilla.rpmfusion.org/show_bug.cgi?id=2714 -%{expand:%(bash %{SOURCE10} --target %{_target_cpu} %{?repo:--repo %{?repo}} --kmodname %{name} %{?buildforkernels:--%{buildforkernels}} --devel %{?prefix:--prefix "%{?prefix}"} %{?kernels:--for-kernels "%{?kernels}"} %{?kernelbuildroot:--buildroot "%{?kernelbuildroot}"} --obsolete-name spl --obsolete-version 0.8 2>/dev/null) } +%{expand:%(bash %{SOURCE10} --target %{_target_cpu} %{?repo:--repo %{?repo}} --kmodname %{name} %{?buildforkernels:--%{buildforkernels}} --devel %{?prefix:--prefix "%{?prefix}"} %{?kernels:--for-kernels "%{?kernels}"} %{?kernelbuildroot:--buildroot "%{?kernelbuildroot}"} 2>/dev/null) } %description @@ -109,7 +109,7 @@ This package contains the ZFS kernel modules. %{?kmodtool_check} # Print kmodtool output for debugging purposes: -bash %{SOURCE10} --target %{_target_cpu} %{?repo:--repo %{?repo}} --kmodname %{name} %{?buildforkernels:--%{buildforkernels}} --devel %{?prefix:--prefix "%{?prefix}"} %{?kernels:--for-kernels "%{?kernels}"} %{?kernelbuildroot:--buildroot "%{?kernelbuildroot}"} --obsolete-name spl --obsolete-version 0.8 2>/dev/null +bash %{SOURCE10} --target %{_target_cpu} %{?repo:--repo %{?repo}} --kmodname %{name} %{?buildforkernels:--%{buildforkernels}} --devel %{?prefix:--prefix "%{?prefix}"} %{?kernels:--for-kernels "%{?kernels}"} %{?kernelbuildroot:--buildroot "%{?kernelbuildroot}"} 2>/dev/null %if %{with debug} %define debug --enable-debug @@ -162,7 +162,7 @@ for kernel_version in %{?kernel_versions}; do cd .. done # find-debuginfo.sh only considers executables -chmod u+x ${RPM_BUILD_ROOT}%{kmodinstdir_prefix}/*/extra/*/*/* +chmod u+x ${RPM_BUILD_ROOT}%{kmodinstdir_prefix}/*/extra/*/* %{?akmod_install} diff --git a/rpm/generic/zfs.spec.in b/rpm/generic/zfs.spec.in index bdd77b43da6f..b1a94fbb7ab2 100644 --- a/rpm/generic/zfs.spec.in +++ b/rpm/generic/zfs.spec.in @@ -90,13 +90,13 @@ License: @ZFS_META_LICENSE@ URL: https://github.com/openzfs/zfs Source0: %{name}-%{version}.tar.gz BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n) -Requires: libzpool5 = %{version} -Requires: libnvpair3 = %{version} -Requires: libuutil3 = %{version} -Requires: libzfs5 = %{version} +Requires: libzpool5%{?_isa} = %{version}-%{release} +Requires: libnvpair3%{?_isa} = %{version}-%{release} +Requires: libuutil3%{?_isa} = %{version}-%{release} +Requires: libzfs5%{?_isa} = %{version}-%{release} Requires: %{name}-kmod = %{version} -Provides: %{name}-kmod-common = %{version} -Obsoletes: spl +Provides: %{name}-kmod-common = %{version}-%{release} +Obsoletes: spl <= %{version} # zfs-fuse provides the same commands and man pages that OpenZFS does. # Renaming those on either side would conflict with all available documentation. @@ -110,16 +110,15 @@ BuildRequires: libblkid-devel BuildRequires: libudev-devel BuildRequires: libattr-devel BuildRequires: openssl-devel -# We don't directly use it, but if this isn't installed, rpmbuild as root can -# crash+corrupt rpmdb -# See issue #12071 -BuildRequires: ncompress %if 0%{?fedora} || 0%{?rhel} >= 8 || 0%{?centos} >= 8 BuildRequires: libtirpc-devel %endif -%if %{with pam} -BuildRequires: pam-devel +%if (0%{?fedora}%{?suse_version}) || (0%{?rhel} && 0%{?rhel} < 9) +# We don't directly use it, but if this isn't installed, rpmbuild as root can +# crash+corrupt rpmdb +# See issue #12071 +BuildRequires: ncompress %endif Requires: openssl @@ -145,8 +144,8 @@ This package contains the core ZFS command line utilities. %package -n libzpool5 Summary: Native ZFS pool library for Linux Group: System Environment/Kernel -Obsoletes: libzpool2 -Obsoletes: libzpool4 +Obsoletes: libzpool2 <= %{version} +Obsoletes: libzpool4 <= %{version} %description -n libzpool5 This package contains the zpool library, which provides support @@ -162,7 +161,7 @@ for managing zpools %package -n libnvpair3 Summary: Solaris name-value library for Linux Group: System Environment/Kernel -Obsoletes: libnvpair1 +Obsoletes: libnvpair1 <= %{version} %description -n libnvpair3 This package contains routines for packing and unpacking name-value @@ -180,7 +179,7 @@ to write self describing data structures on disk. %package -n libuutil3 Summary: Solaris userland utility library for Linux Group: System Environment/Kernel -Obsoletes: libuutil1 +Obsoletes: libuutil1 <= %{version} %description -n libuutil3 This library provides a variety of compatibility functions for OpenZFS: @@ -206,8 +205,8 @@ This library provides a variety of compatibility functions for OpenZFS: %package -n libzfs5 Summary: Native ZFS filesystem library for Linux Group: System Environment/Kernel -Obsoletes: libzfs2 -Obsoletes: libzfs4 +Obsoletes: libzfs2 <= %{version} +Obsoletes: libzfs4 <= %{version} %description -n libzfs5 This package provides support for managing ZFS filesystems @@ -222,16 +221,16 @@ This package provides support for managing ZFS filesystems %package -n libzfs5-devel Summary: Development headers Group: System Environment/Kernel -Requires: libzfs5 = %{version} -Requires: libzpool5 = %{version} -Requires: libnvpair3 = %{version} -Requires: libuutil3 = %{version} -Provides: libzpool5-devel -Provides: libnvpair3-devel -Provides: libuutil3-devel -Obsoletes: zfs-devel -Obsoletes: libzfs2-devel -Obsoletes: libzfs4-devel +Requires: libzfs5%{?_isa} = %{version}-%{release} +Requires: libzpool5%{?_isa} = %{version}-%{release} +Requires: libnvpair3%{?_isa} = %{version}-%{release} +Requires: libuutil3%{?_isa} = %{version}-%{release} +Provides: libzpool5-devel = %{version}-%{release} +Provides: libnvpair3-devel = %{version}-%{release} +Provides: libuutil3-devel = %{version}-%{release} +Obsoletes: zfs-devel <= %{version} +Obsoletes: libzfs2-devel <= %{version} +Obsoletes: libzfs4-devel <= %{version} %description -n libzfs5-devel This package contains the header files needed for building additional @@ -282,8 +281,8 @@ Summary: Python %{python_version} wrapper for libzfs_core Group: Development/Languages/Python License: Apache-2.0 BuildArch: noarch -Requires: libzfs5 = %{version} -Requires: libnvpair3 = %{version} +Requires: libzfs5 = %{version}-%{release} +Requires: libnvpair3 = %{version}-%{release} Requires: libffi Requires: python%{__python_pkg_version} @@ -318,7 +317,6 @@ This package provides a python wrapper for the libzfs_core C library. Summary: Initramfs module Group: System Environment/Kernel Requires: %{name}%{?_isa} = %{version}-%{release} -Requires: %{name} = %{version}-%{release} Requires: initramfs-tools %description initramfs @@ -326,6 +324,19 @@ This package contains a initramfs module used to construct an initramfs image which is ZFS aware. %endif +%if %{with pam} +%package -n pam_zfs_key +Summary: PAM module for encrypted ZFS datasets + +%if 0%{?rhel}%{?centos}%{?fedora}%{?suse_version} +BuildRequires: pam-devel +%endif + +%description -n pam_zfs_key +This package contains the pam_zfs_key PAM module, which provides +support for unlocking datasets on user login. +%endif + %prep %if %{with debug} %define debug --enable-debug @@ -508,10 +519,8 @@ systemctl --system daemon-reload >/dev/null || true %config(noreplace) %{_sysconfdir}/%{name}/zpool.d/* %config(noreplace) %{_sysconfdir}/%{name}/vdev_id.conf.*.example %attr(440, root, root) %config(noreplace) %{_sysconfdir}/sudoers.d/* -%if %{with pam} -%{_libdir}/security/* -%{_datadir}/pam-configs/* -%endif + +%config(noreplace) %{_sysconfdir}/bash_completion.d/zfs %files -n libzpool5 %{_libdir}/libzpool.so.* @@ -561,3 +570,9 @@ systemctl --system daemon-reload >/dev/null || true # ignore those files. %exclude /usr/share/initramfs-tools %endif + +%if %{with pam} +%files -n pam_zfs_key +%{_libdir}/security/* +%{_datadir}/pam-configs/* +%endif diff --git a/rpm/redhat/.gitignore b/rpm/redhat/.gitignore deleted file mode 100644 index 7f5daafdd6d4..000000000000 --- a/rpm/redhat/.gitignore +++ /dev/null @@ -1,3 +0,0 @@ -/zfs-dkms.spec -/zfs-kmod.spec -/zfs.spec diff --git a/rpm/redhat/Makefile.am b/rpm/redhat/Makefile.am deleted file mode 100644 index 89b13640d622..000000000000 --- a/rpm/redhat/Makefile.am +++ /dev/null @@ -1 +0,0 @@ -EXTRA_DIST = zfs.spec.in zfs-kmod.spec.in zfs-dkms.spec.in diff --git a/rpm/redhat/zfs-kmod.spec.in b/rpm/redhat/zfs-kmod.spec.in index 7b74fdc51f43..f59551c0b43a 100644 --- a/rpm/redhat/zfs-kmod.spec.in +++ b/rpm/redhat/zfs-kmod.spec.in @@ -17,9 +17,7 @@ BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n) # by generating a preamble text file which kmodtool can append to the spec file. %(/bin/echo -e "\ Requires: @PACKAGE@ = %{version}\n\ -Conflicts: @PACKAGE@-dkms\n\ -Obsoletes: kmod-spl\n\ -Obsoletes: spl-kmod\n\n" > %{_sourcedir}/kmod-preamble) +Conflicts: @PACKAGE@-dkms) # LDFLAGS are not sanitized by arch/*/Makefile for these architectures. %ifarch ppc ppc64 ppc64le aarch64 @@ -39,7 +37,6 @@ This package contains the ZFS kernel modules. %package -n kmod-%{kmod_name}-devel Summary: ZFS kernel module(s) devel common Group: System Environment/Kernel -Provides: kmod-spl-devel = %{version} %description -n kmod-%{kmod_name}-devel This package provides the header files and objects to build kernel modules. @@ -82,11 +79,10 @@ make install \ %{__rm} -f %{buildroot}/lib/modules/%{kverrel}/modules.* # find-debuginfo.sh only considers executables -%{__chmod} u+x %{buildroot}/lib/modules/%{kverrel}/extra/*/*/* +%{__chmod} u+x %{buildroot}/lib/modules/%{kverrel}/extra/*/* %clean rm -rf $RPM_BUILD_ROOT %files -n kmod-%{kmod_name}-devel %{_usrsrc}/%{kmod_name}-%{version} -%{_usrsrc}/spl-%{version} diff --git a/scripts/Makefile.am b/scripts/Makefile.am index 77b1269a9de4..79719e621b69 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -1,89 +1,88 @@ -include $(top_srcdir)/config/Shellcheck.am +scriptsdir = $(datadir)/$(PACKAGE) +dist_scripts_SCRIPTS = \ + %D%/zfs-helpers.sh \ + %D%/zfs-tests.sh \ + %D%/zfs.sh \ + %D%/zimport.sh \ + %D%/zloop.sh -pkgdatadir = $(datadir)/@PACKAGE@ +dist_noinst_SCRIPTS = \ + %D%/commitcheck.sh \ + %D%/common.sh.in \ + %D%/dkms.mkconf \ + %D%/dkms.postbuild \ + %D%/kmodtool \ + %D%/make_gitrev.sh \ + %D%/man-dates.sh \ + %D%/mancheck.sh \ + %D%/paxcheck.sh \ + %D%/zfs-tests-color.sh -dist_pkgdata_SCRIPTS = \ - zimport.sh \ - zfs.sh \ - zfs-tests.sh \ - zloop.sh \ - zfs-helpers.sh +dist_noinst_DATA += \ + %D%/cstyle.pl \ + %D%/enum-extract.pl \ + %D%/zfs2zol-patch.sed \ + %D%/zol2zfs-patch.sed -EXTRA_SCRIPTS = \ - commitcheck.sh \ - common.sh.in \ - dkms.mkconf \ - dkms.postbuild \ - kmodtool \ - make_gitrev.sh \ - man-dates.sh \ - paxcheck.sh \ - mancheck.sh +SHELLCHECKSCRIPTS += $(dist_scripts_SCRIPTS) $(dist_noinst_SCRIPTS) -EXTRA_DIST = \ - cstyle.pl \ - enum-extract.pl \ - zfs2zol-patch.sed \ - zol2zfs-patch.sed \ - $(EXTRA_SCRIPTS) - -SHELLCHECKSCRIPTS = $(EXTRA_SCRIPTS) - -define EXTRA_ENVIRONMENT +define SCRIPTS_EXTRA_ENVIRONMENT # Only required for in-tree use export INTREE="yes" export GDB="libtool --mode=execute gdb" export LDMOD=/sbin/insmod -export CMD_DIR=@abs_top_builddir@/cmd -export UDEV_RULE_DIR=@abs_top_builddir@/udev/rules.d -export ZEDLET_ETC_DIR=$$CMD_DIR/zed/zed.d -export ZEDLET_LIBEXEC_DIR=$$CMD_DIR/zed/zed.d -export ZPOOL_SCRIPT_DIR=$$CMD_DIR/zpool/zpool.d -export ZPOOL_SCRIPTS_PATH=$$CMD_DIR/zpool/zpool.d -export ZPOOL_COMPAT_DIR=$$CMD_DIR/zpool/compatibility.d -export CONTRIB_DIR=@abs_top_builddir@/contrib -export LIB_DIR=@abs_top_builddir@/lib -export SYSCONF_DIR=@abs_top_builddir@/etc +export CMD_DIR=$(abs_top_builddir) +export UDEV_SCRIPT_DIR=$(abs_top_srcdir)/udev +export UDEV_CMD_DIR=$(abs_top_builddir)/udev +export UDEV_RULE_DIR=$(abs_top_builddir)/udev/rules.d +export ZEDLET_ETC_DIR=$$CMD_DIR/cmd/zed/zed.d +export ZEDLET_LIBEXEC_DIR=$$CMD_DIR/cmd/zed/zed.d +export ZPOOL_SCRIPT_DIR=$$CMD_DIR/cmd/zpool/zpool.d +export ZPOOL_SCRIPTS_PATH=$$CMD_DIR/cmd/zpool/zpool.d +export ZPOOL_COMPAT_DIR=$$CMD_DIR/cmd/zpool/compatibility.d +export CONTRIB_DIR=$(abs_top_builddir)/contrib +export LIB_DIR=$(abs_top_builddir)/.libs +export SYSCONF_DIR=$(abs_top_builddir)/etc -export INSTALL_UDEV_DIR=@udevdir@ -export INSTALL_UDEV_RULE_DIR=@udevruledir@ -export INSTALL_MOUNT_HELPER_DIR=@mounthelperdir@ -export INSTALL_SYSCONF_DIR=@sysconfdir@ -export INSTALL_PYTHON_DIR=@pythonsitedir@ +export INSTALL_UDEV_DIR=$(udevdir) +export INSTALL_UDEV_RULE_DIR=$(udevruledir) +export INSTALL_MOUNT_HELPER_DIR=$(mounthelperdir) +export INSTALL_SYSCONF_DIR=$(sysconfdir) +export INSTALL_PYTHON_DIR=$(pythonsitedir) -export KMOD_SPL=@abs_top_builddir@/module/spl/spl.ko -export KMOD_ZAVL=@abs_top_builddir@/module/avl/zavl.ko -export KMOD_ZNVPAIR=@abs_top_builddir@/module/nvpair/znvpair.ko -export KMOD_ZUNICODE=@abs_top_builddir@/module/unicode/zunicode.ko -export KMOD_ZCOMMON=@abs_top_builddir@/module/zcommon/zcommon.ko -export KMOD_ZLUA=@abs_top_builddir@/module/lua/zlua.ko -export KMOD_ICP=@abs_top_builddir@/module/icp/icp.ko -export KMOD_ZFS=@abs_top_builddir@/module/zfs/zfs.ko -export KMOD_FREEBSD=@abs_top_builddir@/module/openzfs.ko -export KMOD_ZZSTD=@abs_top_builddir@/module/zstd/zzstd.ko +export KMOD_SPL=$(abs_top_builddir)/module/spl.ko +export KMOD_ZFS=$(abs_top_builddir)/module/zfs.ko +export KMOD_FREEBSD=$(abs_top_builddir)/module/openzfs.ko endef -export EXTRA_ENVIRONMENT +export SCRIPTS_EXTRA_ENVIRONMENT + +CLEANFILES += %D%/common.sh +%D%/common.sh: %D%/common.sh.in Makefile + -$(AM_V_at)$(MKDIR_P) $(@D) + -$(AM_V_GEN)$(SED) -e '/^export BIN_DIR=/s|$$|$(abs_top_builddir)/tests/zfs-tests/bin|' \ + -e '/^export SBIN_DIR=/s|$$|$(abs_top_builddir)|' \ + -e '/^export LIBEXEC_DIR=/s|$$|$(abs_top_builddir)|' \ + -e '/^export ZTS_DIR=/s|$$|$(abs_top_srcdir)/tests|' \ + -e '/^export SCRIPT_DIR=/s|$$|$(abs_top_srcdir)/scripts|' \ + $< >$@ + -$(AM_V_at)echo "$$SCRIPTS_EXTRA_ENVIRONMENT" >>$@ -all-local: - -$(SED) -e '\|^export BIN_DIR=|s|$$|@abs_top_builddir@/bin|' \ - -e '\|^export SBIN_DIR=|s|$$|@abs_top_builddir@/bin|' \ - -e '\|^export LIBEXEC_DIR=|s|$$|@abs_top_builddir@/bin|' \ - -e '\|^export ZTS_DIR=|s|$$|@abs_top_srcdir@/tests|' \ - -e '\|^export SCRIPT_DIR=|s|$$|@abs_top_srcdir@/scripts|' \ - $(abs_top_srcdir)/scripts/common.sh.in >common.sh - -echo "$$EXTRA_ENVIRONMENT" >>common.sh +ALL_LOCAL += scripts-all-local +scripts-all-local: %D%/common.sh + -SCRIPT_COMMON=$< $(srcdir)/%D%/zfs-tests.sh -c -clean-local: - -$(RM) common.sh +CLEAN_LOCAL += scripts-clean-local +scripts-clean-local: + -$(RM) -r tests/zfs-tests/bin/ -install-data-hook: - -$(SED) -e '\|^export BIN_DIR=|s|$$|@bindir@|' \ - -e '\|^export SBIN_DIR=|s|$$|@sbindir@|' \ - -e '\|^export LIBEXEC_DIR=|s|$$|@zfsexecdir@|' \ - -e '\|^export ZTS_DIR=|s|$$|@datadir@/@PACKAGE@|' \ - -e '\|^export SCRIPT_DIR=|s|$$|@datadir@/@PACKAGE@|' \ - $(abs_top_srcdir)/scripts/common.sh.in \ - >$(DESTDIR)$(datadir)/@PACKAGE@/common.sh +INSTALL_DATA_HOOKS += scripts-install-data-hook +scripts-install-data-hook: %D%/common.sh.in Makefile + -$(SED) -e '/^export BIN_DIR=/s|$$|$(bindir)|' \ + -e '/^export SBIN_DIR=/s|$$|$(sbindir)|' \ + -e '/^export LIBEXEC_DIR=/s|$$|$(zfsexecdir)|' \ + -e '/^export ZTS_DIR=/s|$$|$(datadir)/$(PACKAGE)|' \ + -e '/^export SCRIPT_DIR=/s|$$|$(datadir)/$(PACKAGE)|' \ + $< >$(DESTDIR)$(datadir)/$(PACKAGE)/common.sh diff --git a/scripts/cstyle.pl b/scripts/cstyle.pl index d19718ecf443..ca7f027051cc 100755 --- a/scripts/cstyle.pl +++ b/scripts/cstyle.pl @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -58,52 +58,27 @@ use strict; my $usage = -"usage: cstyle [-cghpvCP] [-o constructs] file ... +"usage: cstyle [-cgpvP] file... -c check continuation indentation inside functions -g print github actions' workflow commands - -h perform heuristic checks that are sometimes wrong -p perform some of the more picky checks -v verbose - -C don't check anything in header block comments -P check for use of non-POSIX types - -o constructs - allow a comma-separated list of optional constructs: - doxygen allow doxygen-style block comments (/** /*!) - splint allow splint-style lint comments (/*@ ... @*/) "; my %opts; -if (!getopts("cgho:pvCP", \%opts)) { +if (!getopts("cghpvCP", \%opts)) { print $usage; exit 2; } my $check_continuation = $opts{'c'}; my $github_workflow = $opts{'g'} || $ENV{'CI'}; -my $heuristic = $opts{'h'}; my $picky = $opts{'p'}; my $verbose = $opts{'v'}; -my $ignore_hdr_comment = $opts{'C'}; my $check_posix_types = $opts{'P'}; -my $doxygen_comments = 0; -my $splint_comments = 0; - -if (defined($opts{'o'})) { - for my $x (split /,/, $opts{'o'}) { - if ($x eq "doxygen") { - $doxygen_comments = 1; - } elsif ($x eq "splint") { - $splint_comments = 1; - } else { - print "cstyle: unrecognized construct \"$x\"\n"; - print $usage; - exit 2; - } - } -} - my ($filename, $line, $prev); # shared globals my $fmt; @@ -115,12 +90,7 @@ $fmt = "%s: %d: %s\n"; } -if ($doxygen_comments) { - # doxygen comments look like "/*!" or "/**"; allow them. - $hdr_comment_start = qr/^\s*\/\*[\!\*]?$/; -} else { - $hdr_comment_start = qr/^\s*\/\*$/; -} +$hdr_comment_start = qr/^\s*\/\*$/; # Note, following must be in single quotes so that \s and \w work right. my $typename = '(int|char|short|long|unsigned|float|double' . @@ -140,14 +110,12 @@ ); my $lint_re = qr/\/\*(?: - ARGSUSED[0-9]*|NOTREACHED|LINTLIBRARY|VARARGS[0-9]*| + NOTREACHED|LINTLIBRARY|VARARGS[0-9]*| CONSTCOND|CONSTANTCOND|CONSTANTCONDITION|EMPTY| FALLTHRU|FALLTHROUGH|LINTED.*?|PRINTFLIKE[0-9]*| PROTOLIB[0-9]*|SCANFLIKE[0-9]*|CSTYLED.*? )\*\//x; -my $splint_re = qr/\/\*@.*?@\*\//x; - my $warlock_re = qr/\/\*\s*(?: VARIABLES\ PROTECTED\ BY| MEMBERS\ PROTECTED\ BY| @@ -241,7 +209,6 @@ ($$) my $next_in_cpp = 0; my $in_comment = 0; -my $in_header_comment = 0; my $comment_done = 0; my $in_warlock_comment = 0; my $in_function = 0; @@ -385,6 +352,9 @@ ($$) if (/[^ \t(]\/\*/ && !/\w\(\/\*.*\*\/\);/) { err("comment preceded by non-blank"); } + if (/ARGSUSED/) { + err("ARGSUSED directive"); + } # is this the beginning or ending of a function? # (not if "struct foo\n{\n") @@ -469,7 +439,6 @@ ($$) if ($comment_done) { $in_comment = 0; - $in_header_comment = 0; $comment_done = 0; } # does this looks like the start of a block comment? @@ -480,9 +449,6 @@ ($$) $in_comment = 1; /^(\s*)\//; $comment_prefix = $1; - if ($comment_prefix eq "") { - $in_header_comment = 1; - } $prev = $line; next line; } @@ -492,20 +458,13 @@ ($$) $comment_done = 1; } elsif (/\*\//) { $comment_done = 1; - err("improper block comment close") - unless ($ignore_hdr_comment && $in_header_comment); + err("improper block comment close"); } elsif (!/^$comment_prefix \*[ \t]/ && !/^$comment_prefix \*$/) { - err("improper block comment") - unless ($ignore_hdr_comment && $in_header_comment); + err("improper block comment"); } } - if ($in_header_comment && $ignore_hdr_comment) { - $prev = $line; - next line; - } - # check for errors that might occur in comments and in code. # allow spaces to be used to draw pictures in all comments. @@ -533,12 +492,10 @@ ($$) next line; } - if ((/[^(]\/\*\S/ || /^\/\*\S/) && - !(/$lint_re/ || ($splint_comments && /$splint_re/))) { + if ((/[^(]\/\*\S/ || /^\/\*\S/) && !/$lint_re/) { err("missing blank after open comment"); } - if (/\S\*\/[^)]|\S\*\/$/ && - !(/$lint_re/ || ($splint_comments && /$splint_re/))) { + if (/\S\*\/[^)]|\S\*\/$/ && !/$lint_re/) { err("missing blank before close comment"); } if (/\/\/\S/) { # C++ comments @@ -732,19 +689,6 @@ ($$) err("non-POSIX typedef $1 used: use $old2posix{$1} instead"); } } - if ($heuristic) { - # cannot check this everywhere due to "struct {\n...\n} foo;" - if ($in_function && !$in_declaration && - /\}./ && !/\}\s+=/ && !/\{.*\}[;,]$/ && !/\}(\s|)*$/ && - !/\} (else|while)/ && !/\}\}/) { - err("possible bad text following right brace"); - } - # cannot check this because sub-blocks in - # the middle of code are ok - if ($in_function && /^\s+\{/) { - err("possible left brace starting a line"); - } - } if (/^\s*else\W/) { if ($prev =~ /^\s*\}$/) { err_prefix($prev, diff --git a/scripts/dkms.mkconf b/scripts/dkms.mkconf index ebe38e31916a..0bd383420435 100755 --- a/scripts/dkms.mkconf +++ b/scripts/dkms.mkconf @@ -29,22 +29,14 @@ PRE_BUILD="configure --prefix=/usr --with-config=kernel --with-linux=\$( - case \`lsb_release -is\` in - (Debian|Devuan) - if [[ -e \${kernel_source_dir/%build/source} ]] - then - echo \${kernel_source_dir/%build/source} - else - # A kpkg exception for Proxmox 2.0 - echo \${kernel_source_dir} - fi - ;; - (*) - echo \${kernel_source_dir} - ;; - esac + if [ -e "\${kernel_source_dir/%build/source}" ] + then + echo "\${kernel_source_dir/%build/source}" + else + echo "\${kernel_source_dir}" + fi ) - --with-linux-obj=\${kernel_source_dir} + --with-linux-obj="\${kernel_source_dir}" \$( [[ -n \"\${ICP_ROOT}\" ]] && \\ { @@ -85,38 +77,10 @@ STRIP[0]="\$( && echo -n no )" STRIP[1]="\${STRIP[0]}" -STRIP[2]="\${STRIP[0]}" -STRIP[3]="\${STRIP[0]}" -STRIP[4]="\${STRIP[0]}" -STRIP[5]="\${STRIP[0]}" -STRIP[6]="\${STRIP[0]}" -STRIP[7]="\${STRIP[0]}" -STRIP[8]="\${STRIP[0]}" -BUILT_MODULE_NAME[0]="zavl" -BUILT_MODULE_LOCATION[0]="module/avl/" -DEST_MODULE_LOCATION[0]="/extra/avl/avl" -BUILT_MODULE_NAME[1]="znvpair" -BUILT_MODULE_LOCATION[1]="module/nvpair/" -DEST_MODULE_LOCATION[1]="/extra/nvpair/znvpair" -BUILT_MODULE_NAME[2]="zunicode" -BUILT_MODULE_LOCATION[2]="module/unicode/" -DEST_MODULE_LOCATION[2]="/extra/unicode/zunicode" -BUILT_MODULE_NAME[3]="zcommon" -BUILT_MODULE_LOCATION[3]="module/zcommon/" -DEST_MODULE_LOCATION[3]="/extra/zcommon/zcommon" -BUILT_MODULE_NAME[4]="zfs" -BUILT_MODULE_LOCATION[4]="module/zfs/" -DEST_MODULE_LOCATION[4]="/extra/zfs/zfs" -BUILT_MODULE_NAME[5]="icp" -BUILT_MODULE_LOCATION[5]="module/icp/" -DEST_MODULE_LOCATION[5]="/extra/icp/icp" -BUILT_MODULE_NAME[6]="zlua" -BUILT_MODULE_LOCATION[6]="module/lua/" -DEST_MODULE_LOCATION[6]="/extra/lua/zlua" -BUILT_MODULE_NAME[7]="spl" -BUILT_MODULE_LOCATION[7]="module/spl/" -DEST_MODULE_LOCATION[7]="/extra/spl/spl" -BUILT_MODULE_NAME[8]="zzstd" -BUILT_MODULE_LOCATION[8]="module/zstd/" -DEST_MODULE_LOCATION[8]="/extra/zstd/zzstd" +BUILT_MODULE_NAME[0]="zfs" +BUILT_MODULE_LOCATION[0]="module/" +DEST_MODULE_LOCATION[0]="/extra" +BUILT_MODULE_NAME[1]="spl" +BUILT_MODULE_LOCATION[1]="module/" +DEST_MODULE_LOCATION[1]="/extra" EOF diff --git a/scripts/kmodtool b/scripts/kmodtool index afbb6ab3b03c..a79ad0c7a0f3 100755 --- a/scripts/kmodtool +++ b/scripts/kmodtool @@ -72,7 +72,7 @@ LANG=C rpmbuild --define "_sourcedir %{_sourcedir}" \\\ ln -s \$(ls \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/) \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/${kmodname}-kmod.latest %package -n akmod-${kmodname} -Summary: Akmod package for ${kmodname} kernel module(s) +Summary: Akmod package for ${kmodname} kernel module(s) Group: System Environment/Kernel Requires: kmodtool Requires: akmods @@ -82,11 +82,6 @@ Requires: ${kmodname}-kmod-common >= %{?epoch:%{epoch}:}%{version} Provides: ${kmodname}-kmod = %{?epoch:%{epoch}:}%{version}-%{release} EOF - if [ -n "${obsolete_name}" ]; then - echo "Provides: akmod-${obsolete_name} = ${obsolete_version}" - echo "Obsoletes: akmod-${obsolete_name} < ${obsolete_version}" - fi - cat <= %{?epoch:%{epoch}:}%{version}-%{release}" fi - if [ -n "${obsolete_name}" ]; then - echo "Provides: kmod-${obsolete_name}-devel = ${obsolete_version}" - echo "Obsoletes: kmod-${obsolete_name}-devel < ${obsolete_version}" - fi - cat < objects for the newest kernel. %defattr(644,root,root,755) %{_usrsrc}/${kmodname}-%{version} EOF - if [ -n "${obsolete_name}" ]; then - echo "%{_usrsrc}/${obsolete_name}-%{version}" - fi for kernel in ${1}; do local kernel_uname_r=${kernel} echo "%exclude %{_usrsrc}/${kmodname}-%{version}/${kernel_uname_r}" - if [ -n "${obsolete_name}" ]; then - echo "%exclude %{_usrsrc}/${obsolete_name}-%{version}/${kernel_uname_r}" - fi done echo @@ -303,11 +278,6 @@ Provides: ${kmodname}-devel-kmod = %{?epoch:%{epoch}:}%{version}-%{relea Provides: kmod-${kmodname}-devel-uname-r = ${kernel_uname_r} EOF - if [ -n "${obsolete_name}" ]; then - echo "Provides: kmod-${obsolete_name}-devel-${kernel_uname_r} = ${obsolete_version}" - echo "Obsoletes: kmod-${obsolete_name}-devel-${kernel_uname_r} < ${obsolete_version}" - fi - # second part if [ -z "${customkernel}" ]; then cat <= %{?epoch:%{epoch}:}%{vers %{?KmodsMetaRequires:Requires: %{?KmodsMetaRequires}} EOF - if [ -n "${obsolete_name}" ]; then - echo "Provides: kmod-${obsolete_name}${kernel_variant} = ${obsolete_version}" - echo "Obsoletes: kmod-${obsolete_name}${kernel_variant} < ${obsolete_version}" - fi - cat <&2 - elif [ ! -e "${1}" ]; then + elif [ ! -e "${1}" ]; then error_out 2 "Filterfile ${1} not found" >&2 fi filterfile="${1}" @@ -514,22 +476,6 @@ while [ -n "${1}" ] ; do shift noakmod="true" ;; - --obsolete-name) - shift - if [ -z "${1}" ] ; then - error_out 2 "Please provide the name of the kmod to obsolete together with --obsolete-name" >&2 - fi - obsolete_name="${1}" - shift - ;; - --obsolete-version) - shift - if [ -z "${1}" ] ; then - error_out 2 "Please provide the version of the kmod to obsolete together with --obsolete-version" >&2 - fi - obsolete_version="${1}" - shift - ;; --target) shift target="${1}" @@ -583,8 +529,6 @@ elif [ -z "${kmodname}" ]; then error_out 2 "please pass kmodname with --kmodname" elif [ -z "${kernels_known_variants}" ] ; then error_out 2 "could not determine known variants" -elif { [ -n "${obsolete_name}" ] && [ -z "${obsolete_version}" ]; } || { [ -z "${obsolete_name}" ] && [ -n "${obsolete_version}" ]; } ; then - error_out 2 "you need to provide both --obsolete-name and --obsolete-version" fi # go @@ -608,21 +552,17 @@ else # call buildsys-build-${repo}-kerneldevpkgs to get the list of kernels cmdoptions="--target ${target}" - # filterfile to filter list of kernels? + # filterfile to filter list of kernels? if [ -n "${filterfile}" ] ; then cmdoptions="${cmdoptions} --filterfile ${filterfile}" fi - kernel_versions_to_build_for=$(buildsys-build-${repo}-kerneldevpkgs "--${build_kernels}" ${cmdoptions}) - returncode=$? - if [ "$returncode" -ne 0 ]; then - + kernel_versions_to_build_for=$(buildsys-build-${repo}-kerneldevpkgs "--${build_kernels}" ${cmdoptions}) || error_out 2 "buildsys-build-${repo}-kerneldevpkgs failed: ${kernel_versions_to_build_for}" - fi if [ "${build_kernels}" = "current" ] && [ -z "${noakmod}" ]; then print_akmodtemplate fi - print_rpmtemplate + print_rpmtemplate fi diff --git a/scripts/zfs-helpers.sh b/scripts/zfs-helpers.sh index a86a6eb61a60..8dcb0630126a 100755 --- a/scripts/zfs-helpers.sh +++ b/scripts/zfs-helpers.sh @@ -18,7 +18,7 @@ # --sysconfdir=DIR install zfs configuration files [PREFIX/etc] # -BASE_DIR=$(dirname "$0") +BASE_DIR=${0%/*} SCRIPT_COMMON=common.sh if [ -f "${BASE_DIR}/${SCRIPT_COMMON}" ]; then . "${BASE_DIR}/${SCRIPT_COMMON}" @@ -46,7 +46,7 @@ msg() { usage() { cat << EOF USAGE: -$0 [dhirv] +$0 [-dhirv] DESCRIPTION: Install/remove the ZFS helper utilities. @@ -99,7 +99,7 @@ if [ "$INSTALL" = "no" ] && [ "$REMOVE" = "no" ]; then fail "Either -i or -r must be specified" fi -if [ "$(id -u)" != "0" ]; then +if [ "$(id -u)" != "0" ] && [ "$DRYRUN" = "no" ]; then fail "Must run as root" fi @@ -126,13 +126,13 @@ install() { echo "Symlink exists: $dst" elif [ -e "$dst" ]; then echo "File exists: $dst" - elif [ ! -e "$src" ]; then + elif ! [ -e "$src" ]; then echo "Source missing: $src" else msg "ln -s $src $dst" if [ "$DRYRUN" = "no" ]; then - DIR=$(dirname "$dst") + DIR=${dst%/*} mkdir -p "$DIR" >/dev/null 2>&1 ln -s "$src" "$dst" fi @@ -145,7 +145,7 @@ remove() { if [ -h "$dst" ]; then msg "rm $dst" rm "$dst" - DIR=$(dirname "$dst") + DIR=${dst%/*} rmdir "$DIR" >/dev/null 2>&1 elif [ -e "$dst" ]; then echo "Expected symlink: $dst" @@ -153,32 +153,23 @@ remove() { } if [ "${INSTALL}" = "yes" ]; then - install "$CMD_DIR/mount_zfs/mount.zfs" \ - "$INSTALL_MOUNT_HELPER_DIR/mount.zfs" - install "$CMD_DIR/fsck_zfs/fsck.zfs" \ - "$INSTALL_MOUNT_HELPER_DIR/fsck.zfs" - install "$CMD_DIR/zvol_id/zvol_id" \ - "$INSTALL_UDEV_DIR/zvol_id" - install "$CMD_DIR/vdev_id/vdev_id" \ - "$INSTALL_UDEV_DIR/vdev_id" - install "$UDEV_RULE_DIR/60-zvol.rules" \ - "$INSTALL_UDEV_RULE_DIR/60-zvol.rules" - install "$UDEV_RULE_DIR/69-vdev.rules" \ - "$INSTALL_UDEV_RULE_DIR/69-vdev.rules" - install "$UDEV_RULE_DIR/90-zfs.rules" \ - "$INSTALL_UDEV_RULE_DIR/90-zfs.rules" - install "$CMD_DIR/zpool/zpool.d" \ - "$INSTALL_SYSCONF_DIR/zfs/zpool.d" - install "$CONTRIB_DIR/pyzfs/libzfs_core" \ - "$INSTALL_PYTHON_DIR/libzfs_core" + for cmd in "mount.zfs" "fsck.zfs"; do + install "$CMD_DIR/$cmd" "$INSTALL_MOUNT_HELPER_DIR/$cmd" + done + for udev in "$UDEV_CMD_DIR/zvol_id" "$UDEV_SCRIPT_DIR/vdev_id"; do + install "$udev" "$INSTALL_UDEV_DIR/${udev##*/}" + done + for rule in "60-zvol.rules" "69-vdev.rules" "90-zfs.rules"; do + install "$UDEV_RULE_DIR/$rule" "$INSTALL_UDEV_RULE_DIR/$rule" + done + install "$ZPOOL_SCRIPT_DIR" "$INSTALL_SYSCONF_DIR/zfs/zpool.d" + install "$CONTRIB_DIR/pyzfs/libzfs_core" "$INSTALL_PYTHON_DIR/libzfs_core" # Ideally we would install these in the configured ${libdir}, which is # by default "/usr/local/lib and unfortunately not included in the # dynamic linker search path. - install "$(find "$LIB_DIR/libzfs_core" -type f -name 'libzfs_core.so*')" \ - "/lib/libzfs_core.so" - install "$(find "$LIB_DIR/libnvpair" -type f -name 'libnvpair.so*')" \ - "/lib/libnvpair.so" - ldconfig + install "$LIB_DIR"/libzfs_core.so.?.?.? "/lib/libzfs_core.so" + install "$LIB_DIR"/libnvpair.so.?.?.? "/lib/libnvpair.so" + [ "$DRYRUN" = "no" ] && ldconfig else remove "$INSTALL_MOUNT_HELPER_DIR/mount.zfs" remove "$INSTALL_MOUNT_HELPER_DIR/fsck.zfs" diff --git a/scripts/zfs-tests-color.sh b/scripts/zfs-tests-color.sh new file mode 100755 index 000000000000..9098abb62d74 --- /dev/null +++ b/scripts/zfs-tests-color.sh @@ -0,0 +1,27 @@ +#!/bin/sh +# A large mass of sed for coloring zfs-tests.sh output +# Version 2, thanks to наб. +# Just pipe zfs-tests.sh output into this, and watch. + +exec "$(command -v gsed || echo sed)" \ + -e 's/\] \[PASS\]$/] [\x1b[92mPASS\x1b[0m]/' \ + -e 's/\] \[FAIL\]$/] [\x1b[1;91mFAIL\x1b[0m]/' \ + -e 's/\] \[KILLED\]$/] [\x1b[1;101mKILLED\x1b[0m]/' \ + -e 's/\] \[SKIP\]$/] [\x1b[1mSKIP\x1b[0m]/' \ + -e 's/\] \[RERAN\]$/] [\x1b[1;93mRERAN\x1b[0m]/' \ + -e 's/^\(PASS\W\)/\x1b[92m\1\x1b[0m/' \ + -e 's/^\(FAIL\W\)/\x1b[1;91m\1\x1b[0m/' \ + -e 's/^\(KILLED\W\)/\x1b[1;101m\1\x1b[0m/' \ + -e 's/^\(SKIP\W\)/\x1b[1m\1\x1b[0m/' \ + -e 's/^\(RERAN\W\)/\x1b[1;93m\1\x1b[0m/' \ + -e 's/^Tests with result\(.\+\)PASS\(.\+\)$/Tests with result\1\x1b[92mPASS\x1b[0m\2/' \ + -e 's/^\(\W\+\)\(KILLED\)\(\W\)/\1\x1b[1;101m\2\x1b[0m\3/g' \ + -e 's/^\(\W\+\)\(FAIL\)\(\W\)/\1\x1b[1;91m\2\x1b[0m\3/g' \ + -e 's/^\(\W\+\)\(RERUN\)\(\W\)/\1\x1b[1;93m\2\x1b[0m\3/g' \ + -e 's/^\(\W\+\)\(SKIP\)\(\W\)/\1\x1b[1m\2\x1b[0m\3/g' \ + -e 's/expected \(PASS\))$/expected \x1b[92m\1\x1b[0m)/' \ + -e 's/expected \(KILLED\))$/expected \x1b[1;101m\1\x1b[0m)/' \ + -e 's/expected \(FAIL\))$/expected \x1b[1;91m\1\x1b[0m)/' \ + -e 's/expected \(RERUN\))$/expected \x1b[1;93m\1\x1b[0m)/' \ + -e 's/expected \(SKIP\))$/expected \x1b[1m\1\x1b[0m)/' \ + -e 's/^Test\( ([[:alnum:] ]\+)\)\?: \(.\+\) (run as \(.\+\)) \[\([0-9]\+:[0-9]\+\)\] \[\(.\+\)\]$/\x1b[1mTest\1: \x1b[0m\2 (run as \x1b[1m\3\x1b[0m) [\x1b[1m\4\x1b[0m\] [\5\]/' diff --git a/scripts/zfs-tests.sh b/scripts/zfs-tests.sh index fd734aed260a..179e24d7a0ef 100755 --- a/scripts/zfs-tests.sh +++ b/scripts/zfs-tests.sh @@ -9,7 +9,7 @@ # with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -26,19 +26,15 @@ # Copyright 2020 OmniOS Community Edition (OmniOSce) Association. # -BASE_DIR=$(dirname "$0") -SCRIPT_COMMON=common.sh -if [ -f "${BASE_DIR}/${SCRIPT_COMMON}" ]; then -. "${BASE_DIR}/${SCRIPT_COMMON}" -else -echo "Missing helper script ${SCRIPT_COMMON}" && exit 1 -fi +SCRIPT_COMMON=${SCRIPT_COMMON:-${0%/*}/common.sh} +. "${SCRIPT_COMMON}" || exit PROG=zfs-tests.sh VERBOSE="no" QUIET="" CLEANUP="yes" CLEANUPALL="no" +KMSG="" LOOPBACK="yes" STACK_TRACER="no" FILESIZE="4G" @@ -52,8 +48,9 @@ TAGS="" ITERATIONS=1 ZFS_DBGMSG="$STF_SUITE/callbacks/zfs_dbgmsg.ksh" ZFS_DMESG="$STF_SUITE/callbacks/zfs_dmesg.ksh" -UNAME=$(uname -s) +UNAME=$(uname) RERUN="" +KMEMLEAK="" # Override some defaults if on FreeBSD if [ "$UNAME" = "FreeBSD" ] ; then @@ -98,7 +95,7 @@ cleanup_linux_loopback() { for TEST_LOOPBACK in ${LOOPBACKS}; do LOOP_DEV="${TEST_LOOPBACK##*/}" DM_DEV=$(sudo "${DMSETUP}" ls 2>/dev/null | \ - grep "${LOOP_DEV}" | cut -f1) + awk -v l="${LOOP_DEV}" '$0 ~ l {print $1}') if [ -n "$DM_DEV" ]; then sudo "${DMSETUP}" remove "${DM_DEV}" || @@ -131,9 +128,8 @@ cleanup() { fi fi - for TEST_FILE in ${FILES}; do - rm -f "${TEST_FILE}" >/dev/null 2>&1 - done + # shellcheck disable=SC2086 + rm -f ${FILES} >/dev/null 2>&1 if [ "$STF_PATH_REMOVE" = "yes" ] && [ -d "$STF_PATH" ]; then rm -Rf "$STF_PATH" @@ -148,28 +144,30 @@ trap cleanup EXIT # be dangerous and should only be used in a dedicated test environment. # cleanup_all() { - TEST_POOLS=$(sudo env ASAN_OPTIONS=detect_leaks=false "$ZPOOL" list -H -o name | grep testpool) + TEST_POOLS=$(ASAN_OPTIONS=detect_leaks=false "$ZPOOL" list -Ho name | grep testpool) if [ "$UNAME" = "FreeBSD" ] ; then TEST_LOOPBACKS=$(sudo "${LOSETUP}" -l) else - TEST_LOOPBACKS=$(sudo "${LOSETUP}" -a|grep file-vdev|cut -f1 -d:) + TEST_LOOPBACKS=$("${LOSETUP}" -a | awk -F: '/file-vdev/ {print $1}') fi - TEST_FILES=$(ls /var/tmp/file-vdev* 2>/dev/null) + TEST_FILES=$(ls "${FILEDIR}"/file-vdev* /var/tmp/file-vdev* 2>/dev/null) msg msg "--- Cleanup ---" - msg "Removing pool(s): $(echo "${TEST_POOLS}" | tr '\n' ' ')" + # shellcheck disable=2116,2086 + msg "Removing pool(s): $(echo ${TEST_POOLS})" for TEST_POOL in $TEST_POOLS; do sudo env ASAN_OPTIONS=detect_leaks=false "$ZPOOL" destroy "${TEST_POOL}" done if [ "$UNAME" != "FreeBSD" ] ; then - msg "Removing dm(s): $(sudo "${DMSETUP}" ls | + msg "Removing all dm(s): $(sudo "${DMSETUP}" ls | grep loop | tr '\n' ' ')" sudo "${DMSETUP}" remove_all fi - msg "Removing loopback(s): $(echo "${TEST_LOOPBACKS}" | tr '\n' ' ')" + # shellcheck disable=2116,2086 + msg "Removing loopback(s): $(echo ${TEST_LOOPBACKS})" for TEST_LOOPBACK in $TEST_LOOPBACKS; do if [ "$UNAME" = "FreeBSD" ] ; then sudo "${LOSETUP}" -d -u "${TEST_LOOPBACK}" @@ -178,10 +176,10 @@ cleanup_all() { fi done - msg "Removing files(s): $(echo "${TEST_FILES}" | tr '\n' ' ')" - for TEST_FILE in $TEST_FILES; do - sudo rm -f "${TEST_FILE}" - done + # shellcheck disable=2116,2086 + msg "Removing files(s): $(echo ${TEST_FILES})" + # shellcheck disable=2086 + sudo rm -f ${TEST_FILES} } # @@ -195,19 +193,18 @@ cleanup_all() { # find_runfile() { NAME=$1 - RESULT="" if [ -f "$RUNFILE_DIR/$NAME" ]; then - RESULT="$RUNFILE_DIR/$NAME" + echo "$RUNFILE_DIR/$NAME" elif [ -f "$RUNFILE_DIR/$NAME.run" ]; then - RESULT="$RUNFILE_DIR/$NAME.run" + echo "$RUNFILE_DIR/$NAME.run" elif [ -f "$NAME" ]; then - RESULT="$NAME" + echo "$NAME" elif [ -f "$NAME.run" ]; then - RESULT="$NAME.run" + echo "$NAME.run" + else + return 1 fi - - echo "$RESULT" } # @@ -251,7 +248,7 @@ constrain_path() { SYSTEM_DIRS="$SYSTEM_DIRS /usr/bin /usr/sbin /bin /sbin $LIBEXEC_DIR" if [ "$INTREE" = "yes" ]; then - # Constrained path set to ./zfs/bin/ + # Constrained path set to $(top_builddir)/tests/zfs-tests/bin STF_PATH="$BIN_DIR" STF_PATH_REMOVE="no" STF_MISSING_BIN="" @@ -261,14 +258,10 @@ constrain_path() { fi # Special case links for standard zfs utilities - DIRS="$(find "$CMD_DIR" -type d \( ! -name .deps -a \ - ! -name .libs \) -print | tr '\n' ' ')" - create_links "$DIRS" "$ZFS_FILES" + create_links "$CMD_DIR" "$ZFS_FILES" # Special case links for zfs test suite utilities - DIRS="$(find "$STF_SUITE" -type d \( ! -name .deps -a \ - ! -name .libs \) -print | tr '\n' ' ')" - create_links "$DIRS" "$ZFSTEST_FILES" + create_links "$CMD_DIR/tests/zfs-tests/cmd" "$ZFSTEST_FILES" else # Constrained path set to /var/tmp/constrained_path.* SYSTEMDIR=${SYSTEMDIR:-/var/tmp/constrained_path.XXXXXX} @@ -295,14 +288,11 @@ constrain_path() { create_links "$SYSTEM_DIRS" "$SYSTEM_FILES" # Exceptions - ln -fs "$STF_PATH/awk" "$STF_PATH/nawk" if [ "$UNAME" = "Linux" ] ; then ln -fs /sbin/fsck.ext4 "$STF_PATH/fsck" ln -fs /sbin/mkfs.ext4 "$STF_PATH/newfs" ln -fs "$STF_PATH/gzip" "$STF_PATH/compress" ln -fs "$STF_PATH/gunzip" "$STF_PATH/uncompress" - ln -fs "$STF_PATH/exportfs" "$STF_PATH/share" - ln -fs "$STF_PATH/exportfs" "$STF_PATH/unshare" elif [ "$UNAME" = "FreeBSD" ] ; then ln -fs /usr/local/bin/ksh93 "$STF_PATH/ksh" fi @@ -325,13 +315,15 @@ OPTIONS: -q Quiet test-runner output -x Remove all testpools, dm, lo, and files (unsafe) -k Disable cleanup after test failure + -K Log test names to /dev/kmsg -f Use files only, disables block device tests -S Enable stack tracer (negative performance impact) -c Only create and populate constrained path -R Automatically rerun failing tests + -m Enable kmemleak reporting (Linux only) -n NFSFILE Use the nfsfile to determine the NFS configuration -I NUM Number of iterations - -d DIR Use DIR for files and loopback devices + -d DIR Use world-writable DIR for files and loopback devices -s SIZE Use vdevs of SIZE (default: 4G) -r RUNFILES Run tests in RUNFILES (default: ${DEFAULT_RUNFILES}) -t PATH Run single test at PATH relative to test suite @@ -339,7 +331,7 @@ OPTIONS: -u USER Run single test as USER (default: root) EXAMPLES: -# Run the default (linux) suite of tests and output the configuration used. +# Run the default ($(echo "${DEFAULT_RUNFILES}" | sed 's/\.run//')) suite of tests and output the configuration used. $0 -v # Run a smaller suite of tests designed to run more quickly. @@ -349,13 +341,13 @@ $0 -r linux-fast $0 -t tests/functional/cli_root/zfs_bookmark/zfs_bookmark_cliargs.ksh # Cleanup a previous run of the test suite prior to testing, run the -# default (linux) suite of tests and perform no cleanup on exit. +# default ($(echo "${DEFAULT_RUNFILES}" | sed 's/\.run//')) suite of tests and perform no cleanup on exit. $0 -x EOF } -while getopts 'hvqxkfScRn:d:s:r:?t:T:u:I:' OPTION; do +while getopts 'hvqxkKfScRmn:d:s:r:?t:T:u:I:' OPTION; do case $OPTION in h) usage @@ -373,6 +365,9 @@ while getopts 'hvqxkfScRn:d:s:r:?t:T:u:I:' OPTION; do k) CLEANUP="no" ;; + K) + KMSG="yes" + ;; f) LOOPBACK="no" ;; @@ -386,6 +381,9 @@ while getopts 'hvqxkfScRn:d:s:r:?t:T:u:I:' OPTION; do R) RERUN="yes" ;; + m) + KMEMLEAK="yes" + ;; n) nfsfile=$OPTARG [ -f "$nfsfile" ] || fail "Cannot read file: $nfsfile" @@ -439,11 +437,7 @@ if [ -n "$SINGLETEST" ]; then fi RUNFILE_DIR="/var/tmp" RUNFILES="zfs-tests.$$.run" - SINGLEQUIET="False" - - if [ -n "$QUIET" ]; then - SINGLEQUIET="True" - fi + [ -n "$QUIET" ] && SINGLEQUIET="True" || SINGLEQUIET="False" cat >"${RUNFILE_DIR}/${RUNFILES}" << EOF [DEFAULT] @@ -456,19 +450,14 @@ post_user = root post = outputdir = /var/tmp/test_results EOF - SINGLETESTDIR=$(dirname "$SINGLETEST") - SINGLETESTFILE=$(basename "$SINGLETEST") - SETUPSCRIPT= - CLEANUPSCRIPT= + SINGLETESTDIR="${SINGLETEST%/*}" - if [ -f "$STF_SUITE/$SINGLETESTDIR/setup.ksh" ]; then - SETUPSCRIPT="setup" - fi - - if [ -f "$STF_SUITE/$SINGLETESTDIR/cleanup.ksh" ]; then - CLEANUPSCRIPT="cleanup" - fi + SETUPDIR="$SINGLETESTDIR" + [ "${SETUPDIR#/}" = "$SETUPDIR" ] && SETUPDIR="$STF_SUITE/$SINGLETESTDIR" + [ -x "$SETUPDIR/setup.ksh" ] && SETUPSCRIPT="setup" || SETUPSCRIPT= + [ -x "$SETUPDIR/cleanup.ksh" ] && CLEANUPSCRIPT="cleanup" || CLEANUPSCRIPT= + SINGLETESTFILE="${SINGLETEST##*/}" cat >>"${RUNFILE_DIR}/${RUNFILES}" << EOF [$SINGLETESTDIR] @@ -492,8 +481,8 @@ IFS=, for RUNFILE in $RUNFILES; do if [ -n "$RUNFILE" ]; then SAVED_RUNFILE="$RUNFILE" - RUNFILE=$(find_runfile "$RUNFILE") - [ -z "$RUNFILE" ] && fail "Cannot find runfile: $SAVED_RUNFILE" + RUNFILE=$(find_runfile "$RUNFILE") || + fail "Cannot find runfile: $SAVED_RUNFILE" R="$R,$RUNFILE" fi @@ -513,7 +502,7 @@ if [ "$(id -u)" = "0" ]; then fail "This script must not be run as root." fi -if [ "$(sudo whoami)" != "root" ]; then +if [ "$(sudo id -un)" != "root" ]; then fail "Passwordless sudo access required." fi @@ -550,16 +539,14 @@ fi # # By default preserve any existing pools -# NOTE: Since 'zpool list' outputs a newline-delimited list convert $KEEP from -# space-delimited to newline-delimited. # if [ -z "${KEEP}" ]; then - KEEP="$(sudo env ASAN_OPTIONS=detect_leaks=false "$ZPOOL" list -H -o name)" + KEEP="$(ASAN_OPTIONS=detect_leaks=false "$ZPOOL" list -Ho name | tr -s '[:space:]' ' ')" if [ -z "${KEEP}" ]; then KEEP="rpool" fi else - KEEP="$(echo "$KEEP" | tr '[:blank:]' '\n')" + KEEP="$(echo "$KEEP" | tr -s '[:space:]' ' ')" fi # @@ -571,11 +558,7 @@ fi # # See libzfs/libzfs_config.c for more information. # -if [ "$UNAME" = "FreeBSD" ] ; then - __ZFS_POOL_EXCLUDE="$(echo "$KEEP" | tr -s '\n' ' ')" -else - __ZFS_POOL_EXCLUDE="$(echo "$KEEP" | sed ':a;N;s/\n/ /g;ba')" -fi +__ZFS_POOL_EXCLUDE="$KEEP" . "$STF_SUITE/include/default.cfg" @@ -615,8 +598,7 @@ if [ -z "${DISKS}" ]; then DISKS="$DISKS $MDDEVICE" LOOPBACKS="$LOOPBACKS $MDDEVICE" else - TEST_LOOPBACK=$(sudo "${LOSETUP}" -f) - sudo "${LOSETUP}" "${TEST_LOOPBACK}" "${TEST_FILE}" || + TEST_LOOPBACK=$(sudo "${LOSETUP}" --show -f "${TEST_FILE}") || fail "Failed: ${TEST_FILE} -> ${TEST_LOOPBACK}" BASELOOPBACK="${TEST_LOOPBACK##*/}" DISKS="$DISKS $BASELOOPBACK" @@ -642,7 +624,7 @@ fi # # Disable SELinux until the ZFS Test Suite has been updated accordingly. # -if [ -x "$STF_PATH/setenforce" ]; then +if command -v setenforce >/dev/null; then sudo setenforce permissive >/dev/null 2>&1 fi @@ -650,8 +632,8 @@ fi # Enable internal ZFS debug log and clear it. # if [ -e /sys/module/zfs/parameters/zfs_dbgmsg_enable ]; then - sudo /bin/sh -c "echo 1 >/sys/module/zfs/parameters/zfs_dbgmsg_enable" - sudo /bin/sh -c "echo 0 >/proc/spl/kstat/zfs/dbgmsg" + sudo sh -c "echo 1 >/sys/module/zfs/parameters/zfs_dbgmsg_enable" + sudo sh -c "echo 0 >/proc/spl/kstat/zfs/dbgmsg" fi msg @@ -681,26 +663,34 @@ export FILEDIR export KEEP export __ZFS_POOL_EXCLUDE export TESTFAIL_CALLBACKS -export PATH=$STF_PATH -if [ "$UNAME" = "FreeBSD" ] ; then - mkdir -p "$FILEDIR" || true - RESULTS_FILE=$(mktemp -u "${FILEDIR}/zts-results.XXXXXX") - REPORT_FILE=$(mktemp -u "${FILEDIR}/zts-report.XXXXXX") -else - RESULTS_FILE=$(mktemp -u -t zts-results.XXXXXX -p "$FILEDIR") - REPORT_FILE=$(mktemp -u -t zts-report.XXXXXX -p "$FILEDIR") -fi +mktemp_file() { + if [ "$UNAME" = "FreeBSD" ]; then + mktemp -u "${FILEDIR}/$1.XXXXXX" + else + mktemp -ut "$1.XXXXXX" -p "$FILEDIR" + fi +} +mkdir -p "$FILEDIR" || : +RESULTS_FILE=$(mktemp_file zts-results) +REPORT_FILE=$(mktemp_file zts-report) # # Run all the tests as specified. # -msg "${TEST_RUNNER} ${QUIET:+-q}" \ +msg "${TEST_RUNNER}" \ + "${QUIET:+-q}" \ + "${KMEMLEAK:+-m}" \ + "${KMSG:+-K}" \ "-c \"${RUNFILES}\"" \ "-T \"${TAGS}\"" \ "-i \"${STF_SUITE}\"" \ "-I \"${ITERATIONS}\"" -{ ${TEST_RUNNER} ${QUIET:+-q} \ +{ PATH=$STF_PATH \ + ${TEST_RUNNER} \ + ${QUIET:+-q} \ + ${KMEMLEAK:+-m} \ + ${KMSG:+-K} \ -c "${RUNFILES}" \ -T "${TAGS}" \ -i "${STF_SUITE}" \ @@ -716,13 +706,16 @@ RESULT=$? if [ "$RESULT" -eq "2" ] && [ -n "$RERUN" ]; then MAYBES="$($ZTS_REPORT --list-maybes)" - TEMP_RESULTS_FILE=$(mktemp -u -t zts-results-tmp.XXXXX -p "$FILEDIR") - TEST_LIST=$(mktemp -u -t test-list.XXXXX -p "$FILEDIR") + TEMP_RESULTS_FILE=$(mktemp_file zts-results-tmp) + TEST_LIST=$(mktemp_file test-list) grep "^Test:.*\[FAIL\]" "$RESULTS_FILE" >"$TEMP_RESULTS_FILE" for test_name in $MAYBES; do grep "$test_name " "$TEMP_RESULTS_FILE" >>"$TEST_LIST" done - { ${TEST_RUNNER} ${QUIET:+-q} \ + { PATH=$STF_PATH \ + ${TEST_RUNNER} \ + ${QUIET:+-q} \ + ${KMEMLEAK:+-m} \ -c "${RUNFILES}" \ -T "${TAGS}" \ -i "${STF_SUITE}" \ @@ -745,7 +738,7 @@ if [ -d "$RESULTS_DIR" ]; then cat "$RESULTS_FILE" "$REPORT_FILE" >"$RESULTS_DIR/results" fi -rm -f "$RESULTS_FILE" "$REPORT_FILE" +rm -f "$RESULTS_FILE" "$REPORT_FILE" "$TEST_LIST" "$TEMP_RESULTS_FILE" if [ -n "$SINGLETEST" ]; then rm -f "$RUNFILES" >/dev/null 2>&1 diff --git a/scripts/zfs.sh b/scripts/zfs.sh index 0561092a089f..502c5430ab05 100755 --- a/scripts/zfs.sh +++ b/scripts/zfs.sh @@ -3,7 +3,7 @@ # A simple script to load/unload the ZFS module stack. # -BASE_DIR=$(dirname "$0") +BASE_DIR=${0%/*} SCRIPT_COMMON=common.sh if [ -f "${BASE_DIR}/${SCRIPT_COMMON}" ]; then . "${BASE_DIR}/${SCRIPT_COMMON}" @@ -11,7 +11,6 @@ else echo "Missing helper script ${SCRIPT_COMMON}" && exit 1 fi -PROG=zfs.sh VERBOSE="no" UNLOAD="no" LOAD="yes" @@ -19,44 +18,35 @@ STACK_TRACER="no" ZED_PIDFILE=${ZED_PIDFILE:-/var/run/zed.pid} LDMOD=${LDMOD:-/sbin/modprobe} +DELMOD=${DELMOD:-/sbin/rmmod} KMOD_ZLIB_DEFLATE=${KMOD_ZLIB_DEFLATE:-zlib_deflate} KMOD_ZLIB_INFLATE=${KMOD_ZLIB_INFLATE:-zlib_inflate} KMOD_SPL=${KMOD_SPL:-spl} -KMOD_ZAVL=${KMOD_ZAVL:-zavl} -KMOD_ZNVPAIR=${KMOD_ZNVPAIR:-znvpair} -KMOD_ZUNICODE=${KMOD_ZUNICODE:-zunicode} -KMOD_ZCOMMON=${KMOD_ZCOMMON:-zcommon} -KMOD_ZLUA=${KMOD_ZLUA:-zlua} -KMOD_ICP=${KMOD_ICP:-icp} KMOD_ZFS=${KMOD_ZFS:-zfs} KMOD_FREEBSD=${KMOD_FREEBSD:-openzfs} -KMOD_ZZSTD=${KMOD_ZZSTD:-zzstd} usage() { -cat << EOF + cat << EOF USAGE: -$0 [hvudS] [module-options] +$0 [hvudS] DESCRIPTION: Load/unload the ZFS module stack. OPTIONS: - -h Show this message - -v Verbose + -h Show this message + -v Verbose -r Reload modules - -u Unload modules - -S Enable kernel stack tracer + -u Unload modules + -S Enable kernel stack tracer EOF + exit 1 } while getopts 'hvruS' OPTION; do case $OPTION in - h) - usage - exit 1 - ;; v) VERBOSE="yes" ;; @@ -71,18 +61,17 @@ while getopts 'hvruS' OPTION; do S) STACK_TRACER="yes" ;; - ?) - usage - exit - ;; *) + usage ;; esac done +shift $(( OPTIND - 1 )) +[ $# -eq 0 ] || usage kill_zed() { if [ -f "$ZED_PIDFILE" ]; then - PID=$(cat "$ZED_PIDFILE") + read -r PID <"$ZED_PIDFILE" kill "$PID" fi } @@ -91,8 +80,7 @@ check_modules_linux() { LOADED_MODULES="" MISSING_MODULES="" - for KMOD in $KMOD_SPL $KMOD_ZAVL $KMOD_ZNVPAIR $KMOD_ZUNICODE $KMOD_ZCOMMON \ - $KMOD_ZLUA $KMOD_ZZSTD $KMOD_ICP $KMOD_ZFS; do + for KMOD in $KMOD_SPL $KMOD_ZFS; do NAME="${KMOD##*/}" NAME="${NAME%.ko}" @@ -106,7 +94,7 @@ check_modules_linux() { done if [ -n "$LOADED_MODULES" ]; then - printf "Unload the kernel modules by running '%s -u':\n" "$PROG" + printf "Unload the kernel modules by running '%s -u':\n" "$0" printf "%b" "$LOADED_MODULES" exit 1 fi @@ -123,10 +111,11 @@ check_modules_linux() { load_module_linux() { KMOD=$1 - FILE=$(modinfo "$KMOD" | awk '/^filename:/ {print $2}') - VERSION=$(modinfo "$KMOD" | awk '/^version:/ {print $2}') + FILE=$(modinfo "$KMOD" 2>&1 | awk 'NR == 1 && /zlib/ && /not found/ {print "(builtin)"; exit} /^filename:/ {print $2}') + [ "$FILE" = "(builtin)" ] && return if [ "$VERBOSE" = "yes" ]; then + VERSION=$(modinfo "$KMOD" | awk '/^version:/ {print $2}') echo "Loading: $FILE ($VERSION)" fi @@ -151,17 +140,7 @@ load_modules_freebsd() { load_modules_linux() { mkdir -p /etc/zfs - if modinfo "$KMOD_ZLIB_DEFLATE" >/dev/null 2>&1; then - modprobe "$KMOD_ZLIB_DEFLATE" >/dev/null 2>&1 - fi - - if modinfo "$KMOD_ZLIB_INFLATE">/dev/null 2>&1; then - modprobe "$KMOD_ZLIB_INFLATE" >/dev/null 2>&1 - fi - - for KMOD in $KMOD_SPL $KMOD_ZAVL $KMOD_ZNVPAIR \ - $KMOD_ZUNICODE $KMOD_ZCOMMON $KMOD_ZLUA $KMOD_ZZSTD \ - $KMOD_ICP $KMOD_ZFS; do + for KMOD in "$KMOD_ZLIB_DEFLATE" "$KMOD_ZLIB_INFLATE" $KMOD_SPL $KMOD_ZFS; do load_module_linux "$KMOD" || return 1 done @@ -172,23 +151,6 @@ load_modules_linux() { return 0 } -unload_module_linux() { - KMOD=$1 - - NAME="${KMOD##*/}" - NAME="${NAME%.ko}" - FILE=$(modinfo "$KMOD" | awk '/^filename:/ {print $2}') - VERSION=$(modinfo "$KMOD" | awk '/^version:/ {print $2}') - - if [ "$VERBOSE" = "yes" ]; then - echo "Unloading: $KMOD ($VERSION)" - fi - - rmmod "$NAME" || echo "Failed to unload $NAME" - - return 0 -} - unload_modules_freebsd() { kldunload "$KMOD_FREEBSD" || echo "Failed to unload $KMOD_FREEBSD" @@ -200,33 +162,16 @@ unload_modules_freebsd() { } unload_modules_linux() { - for KMOD in $KMOD_ZFS $KMOD_ICP $KMOD_ZZSTD $KMOD_ZLUA $KMOD_ZCOMMON \ - $KMOD_ZUNICODE $KMOD_ZNVPAIR $KMOD_ZAVL $KMOD_SPL; do + legacy_kmods="icp zzstd zlua zcommon zunicode znvpair zavl" + for KMOD in "$KMOD_ZFS" $legacy_kmods "$KMOD_SPL"; do NAME="${KMOD##*/}" NAME="${NAME%.ko}" - USE_COUNT=$(lsmod | awk '/^'"${NAME}"'/ {print $3}') - - if [ "$USE_COUNT" = "0" ] ; then - unload_module_linux "$KMOD" || return 1 - elif [ "$USE_COUNT" != "" ] ; then - echo "Module ${NAME} is still in use!" - return 1 - fi + ! [ -d "/sys/module/$NAME" ] || $DELMOD "$NAME" || return done - if modinfo "$KMOD_ZLIB_DEFLATE" >/dev/null 2>&1; then - modprobe -r "$KMOD_ZLIB_DEFLATE" >/dev/null 2>&1 - fi - - if modinfo "$KMOD_ZLIB_INFLATE">/dev/null 2>&1; then - modprobe -r "$KMOD_ZLIB_INFLATE" >/dev/null 2>&1 - fi - if [ "$VERBOSE" = "yes" ]; then echo "Successfully unloaded ZFS module stack" fi - - return 0 } stack_clear_linux() { @@ -245,8 +190,7 @@ stack_check_linux() { STACK_LIMIT=15362 if [ -e "$STACK_MAX_SIZE" ]; then - STACK_SIZE=$(cat "$STACK_MAX_SIZE") - + read -r STACK_SIZE <"$STACK_MAX_SIZE" if [ "$STACK_SIZE" -ge "$STACK_LIMIT" ]; then echo echo "Warning: max stack size $STACK_SIZE bytes" @@ -260,22 +204,22 @@ if [ "$(id -u)" != 0 ]; then exit 1 fi -UNAME=$(uname -s) +UNAME=$(uname) if [ "$UNLOAD" = "yes" ]; then kill_zed umount -t zfs -a case $UNAME in FreeBSD) - unload_modules_freebsd + unload_modules_freebsd ;; Linux) - stack_check_linux - unload_modules_linux + stack_check_linux + unload_modules_linux ;; *) - echo "unknown system: $UNAME" >&2 - exit 1 + echo "unknown system: $UNAME" >&2 + exit 1 ;; esac fi @@ -287,13 +231,13 @@ if [ "$LOAD" = "yes" ]; then Linux) stack_clear_linux check_modules_linux - load_modules_linux "$@" + load_modules_linux udevadm trigger udevadm settle ;; *) - echo "unknown system: $UNAME" >&2 - exit 1 + echo "unknown system: $UNAME" >&2 + exit 1 ;; esac fi diff --git a/scripts/zfs2zol-patch.sed b/scripts/zfs2zol-patch.sed index 99824d6dd4af..2d744cd5de52 100755 --- a/scripts/zfs2zol-patch.sed +++ b/scripts/zfs2zol-patch.sed @@ -19,7 +19,7 @@ s:usr/src/test/zfs-tests/runfiles:tests/runfiles:g s:usr/src/test/zfs-tests/tests/functional:tests/zfs-tests/tests/functional:g s:usr/src/test/zfs-tests/tests/perf:tests/zfs-tests/tests/perf:g s:usr/src/test/test-runner/cmd/run.py:tests/test-runner/cmd/test-runner.py:g -s/usr\/src\/common\/zfs\/\(.*\)\.c/module\/zcommon\/\1.c/g +s:usr/src/common/zfs/\(.*\)\.c:module/zcommon/\1.c:g # crypto framework s:usr/src/common/crypto:module/icp/algs:g diff --git a/scripts/zimport.sh b/scripts/zimport.sh index 14d2813ce268..595de494e50b 100755 --- a/scripts/zimport.sh +++ b/scripts/zimport.sh @@ -235,8 +235,8 @@ src_set_vars() { ZFS_CMD=$(command -v zfs) ZFS_SH="/usr/share/zfs/zfs.sh" else - ZPOOL_CMD="./cmd/zpool/zpool" - ZFS_CMD="./cmd/zfs/zfs" + ZPOOL_CMD="./zpool" + ZFS_CMD="./zfs" ZFS_SH="./scripts/zfs.sh" fi } diff --git a/scripts/zloop.sh b/scripts/zloop.sh index dd89ff6fdf29..ade2c84c55b7 100755 --- a/scripts/zloop.sh +++ b/scripts/zloop.sh @@ -21,7 +21,7 @@ # Copyright (c) 2017, Intel Corporation. # -BASE_DIR=$(dirname "$0") +BASE_DIR=${0%/*} SCRIPT_COMMON=common.sh if [ -f "${BASE_DIR}/${SCRIPT_COMMON}" ]; then . "${BASE_DIR}/${SCRIPT_COMMON}" @@ -66,8 +66,7 @@ EOF function or_die { - # shellcheck disable=SC2068 - if ! $@; then + if ! "$@"; then echo "Command failed: $*" exit 1 fi @@ -79,7 +78,7 @@ FreeBSD) ;; Linux) # core file helpers - origcorepattern="$(cat /proc/sys/kernel/core_pattern)" + read -r origcorepattern >ztest.zdb dest=$coredir/$coreid - or_die mkdir -p "$dest" or_die mkdir -p "$dest/vdev" if [[ $symlink -ne 0 ]]; then @@ -137,10 +135,8 @@ function store_core echo "*** ztest crash found - moving logs to $dest" - or_die mv ztest.history "$dest/" - or_die mv ztest.zdb "$dest/" - or_die mv ztest.out "$dest/" - or_die mv "$workdir/ztest*" "$dest/vdev/" + or_die mv ztest.history ztest.zdb ztest.out "$dest/" + or_die mv "$workdir/"ztest* "$dest/vdev/" if [[ -e "$workdir/zpool.cache" ]]; then or_die mv "$workdir/zpool.cache" "$dest/vdev/" @@ -235,9 +231,7 @@ if [[ ! -w $coredir ]]; then exit 1 fi -or_die rm -f ztest.history -or_die rm -f ztest.zdb -or_die rm -f ztest.cores +or_die rm -f ztest.history ztest.zdb ztest.cores ztrc=0 # ztest return value foundcrashes=0 # number of crashes found so far @@ -312,9 +306,7 @@ while (( timeout == 0 )) || (( curtime <= (starttime + timeout) )); do zopt="$zopt -f $workdir" cmd="$ZTEST $zopt $*" - desc="$(date '+%m/%d %T') $cmd" - echo "$desc" | tee -a ztest.history - echo "$desc" >>ztest.out + echo "$(date '+%m/%d %T') $cmd" | tee -a ztest.history ztest.out $cmd >>ztest.out 2>&1 ztrc=$? grep -E '===|WARNING' ztest.out >>ztest.history diff --git a/tests/Makefile.am b/tests/Makefile.am index 1dfc2cc5f518..2e633041ab59 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -1,8 +1,30 @@ -include $(top_srcdir)/config/Shellcheck.am +include $(srcdir)/%D%/zfs-tests/Makefile.am -SUBDIRS = runfiles test-runner zfs-tests -EXTRA_DIST = README.md +scripts_test_runner_bindir = $(datadir)/$(PACKAGE)/test-runner/bin +scripts_test_runner_bin_SCRIPTS = \ + %D%/test-runner/bin/test-runner.py \ + %D%/test-runner/bin/zts-report.py -SHELLCHECKSCRIPTS = $$(find . -name '*.sh') -.PHONY: $(SHELLCHECKSCRIPTS) +SUBSTFILES += $(scripts_test_runner_bin_SCRIPTS) + + +scripts_test_runner_includedir = $(datadir)/$(PACKAGE)/test-runner/include +dist_scripts_test_runner_include_DATA = \ + %D%/test-runner/include/logapi.shlib + + +scripts_runfilesdir = $(datadir)/$(PACKAGE)/runfiles +dist_scripts_runfiles_DATA = \ + %D%/runfiles/common.run \ + %D%/runfiles/freebsd.run \ + %D%/runfiles/linux.run \ + %D%/runfiles/longevity.run \ + %D%/runfiles/perf-regression.run \ + %D%/runfiles/sanity.run \ + %D%/runfiles/sunos.run + + +dist_noinst_DATA += %D%/README.md + +SHELLCHECKSCRIPTS += $(shell find $(srcdir)/%D% -name '*.sh') diff --git a/tests/README.md b/tests/README.md index a01ffbe12cf7..624f5efd16e5 100644 --- a/tests/README.md +++ b/tests/README.md @@ -39,6 +39,9 @@ The pre-requisites for running the ZFS Test Suite are: * The ZFS Test Suite will add users and groups to test machine to verify functionality. Therefore it is strongly advised that a dedicated test machine, which can be a VM, be used for testing. + * On FreeBSD, mountd(8) must use `/etc/zfs/exports` + as one of its export files – by default this can be done by setting + `zfs_enable=yes` in `/etc/rc.conf`. Once the pre-requisites are satisfied simply run the zfs-tests.sh script: @@ -88,6 +91,7 @@ The following zfs-tests.sh options are supported: -d DIR Create sparse files for vdevs in the DIR directory. By default these files are created under /var/tmp/. + This directory must be world-writable. -s SIZE Use vdevs of SIZE (default: 4G) diff --git a/tests/runfiles/Makefile.am b/tests/runfiles/Makefile.am deleted file mode 100644 index 278e94934fe1..000000000000 --- a/tests/runfiles/Makefile.am +++ /dev/null @@ -1,9 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/runfiles -dist_pkgdata_DATA = \ - common.run \ - freebsd.run \ - linux.run \ - longevity.run \ - perf-regression.run \ - sanity.run \ - sunos.run diff --git a/tests/runfiles/common.run b/tests/runfiles/common.run index a7c34d0eac33..8055c51932bf 100644 --- a/tests/runfiles/common.run +++ b/tests/runfiles/common.run @@ -29,7 +29,7 @@ outputdir = /var/tmp/test_results tags = ['functional'] [tests/functional/acl/off] -tests = ['posixmode'] +tests = ['dosmode', 'posixmode'] tags = ['functional', 'acl'] [tests/functional/alloc_class] @@ -40,6 +40,10 @@ tests = ['alloc_class_001_pos', 'alloc_class_002_neg', 'alloc_class_003_pos', 'alloc_class_013_pos'] tags = ['functional', 'alloc_class'] +[tests/functional/append] +tests = ['file_append', 'threadsappend_001_pos'] +tags = ['functional', 'append'] + [tests/functional/arc] tests = ['dbufstats_001_pos', 'dbufstats_002_pos', 'dbufstats_003_pos', 'arcstats_runtime_tuning'] @@ -109,8 +113,8 @@ tests = ['tst.destroy_fs', 'tst.destroy_snap', 'tst.get_count_and_limit', tags = ['functional', 'channel_program', 'synctask_core'] [tests/functional/checksum] -tests = ['run_edonr_test', 'run_sha2_test', 'run_skein_test', 'filetest_001_pos', - 'filetest_002_pos'] +tests = ['run_edonr_test', 'run_sha2_test', 'run_skein_test', 'run_blake3_test', + 'filetest_001_pos', 'filetest_002_pos'] tags = ['functional', 'checksum'] [tests/functional/clean_mirror] @@ -237,7 +241,8 @@ tests = ['zfs_receive_001_pos', 'zfs_receive_002_pos', 'zfs_receive_003_pos', 'zfs_receive_from_encrypted', 'zfs_receive_to_encrypted', 'zfs_receive_raw', 'zfs_receive_raw_incremental', 'zfs_receive_-e', 'zfs_receive_raw_-d', 'zfs_receive_from_zstd', 'zfs_receive_new_props', - 'zfs_receive_-wR-encrypted-mix'] + 'zfs_receive_-wR-encrypted-mix', 'zfs_receive_corrective', + 'zfs_receive_compressed_corrective'] tags = ['functional', 'cli_root', 'zfs_receive'] [tests/functional/cli_root/zfs_rename] @@ -305,7 +310,7 @@ tags = ['functional', 'cli_root', 'zfs_unmount'] [tests/functional/cli_root/zfs_unshare] tests = ['zfs_unshare_001_pos', 'zfs_unshare_002_pos', 'zfs_unshare_003_pos', 'zfs_unshare_004_neg', 'zfs_unshare_005_neg', 'zfs_unshare_006_pos', - 'zfs_unshare_007_pos'] + 'zfs_unshare_007_pos', 'zfs_unshare_008_pos'] tags = ['functional', 'cli_root', 'zfs_unshare'] [tests/functional/cli_root/zfs_upgrade] @@ -315,7 +320,7 @@ tests = ['zfs_upgrade_001_pos', 'zfs_upgrade_002_pos', 'zfs_upgrade_003_pos', tags = ['functional', 'cli_root', 'zfs_upgrade'] [tests/functional/cli_root/zfs_wait] -tests = ['zfs_wait_deleteq'] +tests = ['zfs_wait_deleteq', 'zfs_wait_getsubopt'] tags = ['functional', 'cli_root', 'zfs_wait'] [tests/functional/cli_root/zhack] @@ -488,6 +493,7 @@ tags = ['functional', 'cli_root', 'zpool_split'] [tests/functional/cli_root/zpool_status] tests = ['zpool_status_001_pos', 'zpool_status_002_pos', + 'zpool_status_003_pos', 'zpool_status_004_pos', 'zpool_status_features_001_pos'] tags = ['functional', 'cli_root', 'zpool_status'] @@ -551,7 +557,8 @@ tags = ['functional', 'cli_user', 'misc'] [tests/functional/cli_user/zfs_list] tests = ['zfs_list_001_pos', 'zfs_list_002_pos', 'zfs_list_003_pos', - 'zfs_list_004_neg', 'zfs_list_007_pos', 'zfs_list_008_neg'] + 'zfs_list_004_neg', 'zfs_list_005_neg', 'zfs_list_007_pos', + 'zfs_list_008_neg'] user = tags = ['functional', 'cli_user', 'zfs_list'] @@ -639,7 +646,9 @@ tests = ['history_001_pos', 'history_002_pos', 'history_003_pos', tags = ['functional', 'history'] [tests/functional/hkdf] -tests = ['run_hkdf_test'] +pre = +post = +tests = ['hkdf_test'] tags = ['functional', 'hkdf'] [tests/functional/inheritance] @@ -660,12 +669,6 @@ tags = ['functional', 'inuse'] tests = ['large_files_001_pos', 'large_files_002_pos'] tags = ['functional', 'large_files'] -[tests/functional/largest_pool] -tests = ['largest_pool_001_pos'] -pre = -post = -tags = ['functional', 'largest_pool'] - [tests/functional/limits] tests = ['filesystem_count', 'filesystem_limit', 'snapshot_count', 'snapshot_limit'] @@ -683,7 +686,7 @@ tests = ['migration_001_pos', 'migration_002_pos', 'migration_003_pos', tags = ['functional', 'migration'] [tests/functional/mmap] -tests = ['mmap_write_001_pos', 'mmap_read_001_pos', 'mmap_seek_001_pos'] +tests = ['mmap_write_001_pos', 'mmap_read_001_pos', 'mmap_seek_001_pos', 'mmap_sync_001_pos'] tags = ['functional', 'mmap'] [tests/functional/mount] @@ -700,7 +703,7 @@ tags = ['functional', 'nestedfs'] [tests/functional/no_space] tests = ['enospc_001_pos', 'enospc_002_pos', 'enospc_003_pos', - 'enospc_df'] + 'enospc_df', 'enospc_rm'] tags = ['functional', 'no_space'] [tests/functional/nopwrite] @@ -761,7 +764,8 @@ tags = ['functional', 'raidz'] [tests/functional/redundancy] tests = ['redundancy_draid', 'redundancy_draid1', 'redundancy_draid2', - 'redundancy_draid3', 'redundancy_draid_damaged', 'redundancy_draid_spare1', + 'redundancy_draid3', 'redundancy_draid_damaged1', + 'redundancy_draid_damaged2', 'redundancy_draid_spare1', 'redundancy_draid_spare2', 'redundancy_draid_spare3', 'redundancy_mirror', 'redundancy_raidz', 'redundancy_raidz1', 'redundancy_raidz2', 'redundancy_raidz3', 'redundancy_stripe'] @@ -829,7 +833,8 @@ tests = ['recv_dedup', 'recv_dedup_encrypted_zvol', 'rsend_001_pos', 'rsend_006_pos', 'rsend_007_pos', 'rsend_008_pos', 'rsend_009_pos', 'rsend_010_pos', 'rsend_011_pos', 'rsend_012_pos', 'rsend_013_pos', 'rsend_014_pos', 'rsend_016_neg', 'rsend_019_pos', 'rsend_020_pos', - 'rsend_021_pos', 'rsend_022_pos', 'rsend_024_pos', + 'rsend_021_pos', 'rsend_022_pos', 'rsend_024_pos', 'rsend_025_pos', + 'rsend_026_neg', 'rsend_027_pos', 'rsend_028_neg', 'rsend_029_neg', 'send-c_verify_ratio', 'send-c_verify_contents', 'send-c_props', 'send-c_incremental', 'send-c_volume', 'send-c_zstreamdump', 'send-c_lz4_disabled', 'send-c_recv_lz4_disabled', @@ -841,7 +846,7 @@ tests = ['recv_dedup', 'recv_dedup_encrypted_zvol', 'rsend_001_pos', 'send_realloc_encrypted_files', 'send_spill_block', 'send_holds', 'send_hole_birth', 'send_mixed_raw', 'send-wR_encrypted_zvol', 'send_partial_dataset', 'send_invalid', 'send_doall', - 'send_raw_spill_block'] + 'send_raw_spill_block', 'send_raw_ashift'] tags = ['functional', 'rsend'] [tests/functional/scrub_mirror] @@ -854,7 +859,7 @@ tests = ['slog_001_pos', 'slog_002_pos', 'slog_003_pos', 'slog_004_pos', 'slog_005_pos', 'slog_006_pos', 'slog_007_pos', 'slog_008_neg', 'slog_009_neg', 'slog_010_neg', 'slog_011_neg', 'slog_012_neg', 'slog_013_pos', 'slog_014_pos', 'slog_015_neg', 'slog_replay_fs_001', - 'slog_replay_fs_002', 'slog_replay_volume'] + 'slog_replay_fs_002', 'slog_replay_volume', 'slog_016_pos'] tags = ['functional', 'slog'] [tests/functional/snapshot] @@ -885,10 +890,6 @@ tests = ['suid_write_to_suid', 'suid_write_to_sgid', 'suid_write_to_suid_sgid', 'suid_write_to_none', 'suid_write_zil_replay'] tags = ['functional', 'suid'] -[tests/functional/threadsappend] -tests = ['threadsappend_001_pos'] -tags = ['functional', 'threadsappend'] - [tests/functional/trim] tests = ['autotrim_integrity', 'autotrim_config', 'autotrim_trim_integrity', 'trim_integrity', 'trim_config', 'trim_l2arc'] @@ -925,7 +926,7 @@ tags = ['functional', 'write_dirs'] [tests/functional/xattr] tests = ['xattr_001_pos', 'xattr_002_neg', 'xattr_003_neg', 'xattr_004_pos', 'xattr_005_pos', 'xattr_006_pos', 'xattr_007_neg', - 'xattr_011_pos', 'xattr_012_pos', 'xattr_013_pos'] + 'xattr_011_pos', 'xattr_012_pos', 'xattr_013_pos', 'xattr_compat'] tags = ['functional', 'xattr'] [tests/functional/zvol/zvol_ENOSPC] @@ -938,9 +939,13 @@ tags = ['functional', 'zvol', 'zvol_cli'] [tests/functional/zvol/zvol_misc] tests = ['zvol_misc_002_pos', 'zvol_misc_hierarchy', 'zvol_misc_rename_inuse', - 'zvol_misc_snapdev', 'zvol_misc_volmode', 'zvol_misc_zil'] + 'zvol_misc_snapdev', 'zvol_misc_trim', 'zvol_misc_volmode', 'zvol_misc_zil'] tags = ['functional', 'zvol', 'zvol_misc'] +[tests/functional/zvol/zvol_stress] +tests = ['zvol_stress'] +tags = ['functional', 'zvol', 'zvol_stress'] + [tests/functional/zvol/zvol_swap] tests = ['zvol_swap_001_pos', 'zvol_swap_002_pos', 'zvol_swap_004_pos'] tags = ['functional', 'zvol', 'zvol_swap'] diff --git a/tests/runfiles/freebsd.run b/tests/runfiles/freebsd.run index 153b204b4932..c7ca1d769fc3 100644 --- a/tests/runfiles/freebsd.run +++ b/tests/runfiles/freebsd.run @@ -22,10 +22,6 @@ failsafe = callbacks/zfs_failsafe outputdir = /var/tmp/test_results tags = ['functional'] -[tests/functional/acl/off:FreeBSD] -tests = ['dosmode'] -tags = ['functional', 'acl'] - [tests/functional/cli_root/zfs_jail:FreeBSD] tests = ['zfs_jail_001_pos'] tags = ['functional', 'cli_root', 'zfs_jail'] diff --git a/tests/runfiles/linux.run b/tests/runfiles/linux.run index 6ce78a3fc96c..9b32e73afb1e 100644 --- a/tests/runfiles/linux.run +++ b/tests/runfiles/linux.run @@ -90,7 +90,7 @@ tests = ['events_001_pos', 'events_002_pos', 'zed_rc_filter', 'zed_fd_spill'] tags = ['functional', 'events'] [tests/functional/fallocate:Linux] -tests = ['fallocate_prealloc'] +tests = ['fallocate_prealloc', 'fallocate_zero-range'] tags = ['functional', 'fallocate'] [tests/functional/fault:Linux] @@ -109,6 +109,12 @@ tags = ['functional', 'features', 'large_dnode'] tests = ['libaio', 'io_uring'] tags = ['functional', 'io'] +[tests/functional/largest_pool:Linux] +tests = ['largest_pool_001_pos'] +pre = +post = +tags = ['functional', 'largest_pool'] + [tests/functional/mmap:Linux] tests = ['mmap_libaio_001_pos'] tags = ['functional', 'mmap'] @@ -143,6 +149,10 @@ tests = ['projectid_001_pos', 'projectid_002_pos', 'projectid_003_pos', 'projecttree_001_pos', 'projecttree_002_pos', 'projecttree_003_neg'] tags = ['functional', 'projectquota'] +[tests/functional/dos_attributes:Linux] +tests = ['read_dos_attrs_001', 'write_dos_attrs_001'] +tags = ['functional', 'dos_attributes'] + [tests/functional/rsend:Linux] tests = ['send_realloc_dnode_size', 'send_encrypted_files'] tags = ['functional', 'rsend'] @@ -167,10 +177,16 @@ tests = ['upgrade_projectquota_001_pos'] tags = ['functional', 'upgrade'] [tests/functional/user_namespace:Linux] -tests = ['user_namespace_001'] +tests = ['user_namespace_001', 'user_namespace_002', 'user_namespace_003', + 'user_namespace_004'] tags = ['functional', 'user_namespace'] [tests/functional/userquota:Linux] tests = ['groupspace_001_pos', 'groupspace_002_pos', 'groupspace_003_pos', 'userquota_013_pos', 'userspace_003_pos'] tags = ['functional', 'userquota'] + +[tests/functional/zvol/zvol_misc:Linux] +tests = ['zvol_misc_fua'] +tags = ['functional', 'zvol', 'zvol_misc'] + diff --git a/tests/runfiles/sanity.run b/tests/runfiles/sanity.run index 8057d8148c93..f5dcfa5be4b7 100644 --- a/tests/runfiles/sanity.run +++ b/tests/runfiles/sanity.run @@ -236,7 +236,7 @@ tests = ['zfs_upgrade_001_pos', 'zfs_upgrade_002_pos', 'zfs_upgrade_006_neg', tags = ['functional', 'cli_root', 'zfs_upgrade'] [tests/functional/cli_root/zfs_wait] -tests = ['zfs_wait_deleteq'] +tests = ['zfs_wait_deleteq', 'zfs_wait_getsubopt'] tags = ['functional', 'cli_root', 'zfs_wait'] [tests/functional/cli_root/zpool] @@ -438,7 +438,9 @@ tests = ['history_004_pos', 'history_005_neg', 'history_007_pos', tags = ['functional', 'history'] [tests/functional/hkdf] -tests = ['run_hkdf_test'] +pre = +post = +tests = ['hkdf_test'] tags = ['functional', 'hkdf'] [tests/functional/inuse] @@ -582,7 +584,7 @@ tests = ['suid_write_to_suid', 'suid_write_to_sgid', 'suid_write_to_suid_sgid', 'suid_write_to_none'] tags = ['functional', 'suid'] -[tests/functional/threadsappend] +[tests/functional/append] tests = ['threadsappend_001_pos'] tags = ['functional', 'threadsappend'] @@ -602,7 +604,7 @@ tags = ['functional', 'vdev_zaps'] [tests/functional/xattr] tests = ['xattr_001_pos', 'xattr_002_neg', 'xattr_003_neg', 'xattr_004_pos', 'xattr_005_pos', 'xattr_006_pos', 'xattr_007_neg', - 'xattr_011_pos', 'xattr_013_pos'] + 'xattr_011_pos', 'xattr_013_pos', 'xattr_compat'] tags = ['functional', 'xattr'] [tests/functional/zvol/zvol_ENOSPC] diff --git a/tests/test-runner/Makefile.am b/tests/test-runner/Makefile.am deleted file mode 100644 index db3d966142d6..000000000000 --- a/tests/test-runner/Makefile.am +++ /dev/null @@ -1 +0,0 @@ -SUBDIRS = bin include man diff --git a/tests/test-runner/bin/Makefile.am b/tests/test-runner/bin/Makefile.am deleted file mode 100644 index e11e55fffdeb..000000000000 --- a/tests/test-runner/bin/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -include $(top_srcdir)/config/Substfiles.am - -pkgdatadir = $(datadir)/@PACKAGE@/test-runner/bin -pkgdata_SCRIPTS = \ - test-runner.py \ - zts-report.py - -SUBSTFILES += $(pkgdata_SCRIPTS) diff --git a/tests/test-runner/bin/test-runner.py.in b/tests/test-runner/bin/test-runner.py.in index 3c0b82a311bb..cb453b266f3c 100755 --- a/tests/test-runner/bin/test-runner.py.in +++ b/tests/test-runner/bin/test-runner.py.in @@ -31,11 +31,14 @@ from pwd import getpwuid from select import select from subprocess import PIPE from subprocess import Popen +from subprocess import check_output from threading import Timer from time import time, CLOCK_MONOTONIC +from os.path import exists BASEDIR = '/var/tmp/test_results' TESTDIR = '/usr/share/zfs/' +KMEMLEAK_FILE = '/sys/kernel/debug/kmemleak' KILL = 'kill' TRUE = 'true' SUDO = 'sudo' @@ -75,6 +78,7 @@ class Result(object): self.runtime = '' self.stdout = [] self.stderr = [] + self.kmemleak = '' self.result = '' def done(self, proc, killed, reran): @@ -90,6 +94,9 @@ class Result(object): if killed: self.result = 'KILLED' Result.runresults['KILLED'] += 1 + elif len(self.kmemleak) > 0: + self.result = 'FAIL' + Result.runresults['FAIL'] += 1 elif self.returncode == 0: self.result = 'PASS' Result.runresults['PASS'] += 1 @@ -250,7 +257,7 @@ User: %s return out.lines, err.lines - def run(self, dryrun): + def run(self, dryrun, kmemleak, kmsg): """ This is the main function that runs each individual test. Determine whether or not the command requires sudo, and modify it @@ -269,7 +276,24 @@ User: %s except OSError as e: fail('%s' % e) + """ + Log each test we run to /dev/kmsg (on Linux), so if there's a kernel + warning we'll be able to match it up to a particular test. + """ + if kmsg is True and exists("/dev/kmsg"): + try: + kp = Popen([SUDO, "sh", "-c", + f"echo ZTS run {self.pathname} > /dev/kmsg"]) + kp.wait() + except Exception: + pass + self.result.starttime = monotonic_time() + + if kmemleak: + cmd = f'{SUDO} sh -c "echo clear > {KMEMLEAK_FILE}"' + check_output(cmd, shell=True) + proc = Popen(privcmd, stdout=PIPE, stderr=PIPE) # Allow a special timeout value of 0 to mean infinity if int(self.timeout) == 0: @@ -279,6 +303,12 @@ User: %s try: t.start() self.result.stdout, self.result.stderr = self.collect_output(proc) + + if kmemleak: + cmd = f'{SUDO} sh -c "echo scan > {KMEMLEAK_FILE}"' + check_output(cmd, shell=True) + cmd = f'{SUDO} cat {KMEMLEAK_FILE}' + self.result.kmemleak = check_output(cmd, shell=True) except KeyboardInterrupt: self.kill_cmd(proc, True) fail('\nRun terminated at user request.') @@ -355,6 +385,9 @@ User: %s with open(os.path.join(self.outputdir, 'merged'), 'wb') as merged: for _, line in lines: os.write(merged.fileno(), b'%s\n' % line) + if len(self.result.kmemleak): + with open(os.path.join(self.outputdir, 'kmemleak'), 'wb') as kmem: + kmem.write(self.result.kmemleak) class Test(Cmd): @@ -439,14 +472,14 @@ Tags: %s cont = True if len(pretest.pathname): - pretest.run(options.dryrun) + pretest.run(options.dryrun, False, options.kmsg) cont = pretest.result.result == 'PASS' pretest.log(options) if cont: - test.run(options.dryrun) + test.run(options.dryrun, options.kmemleak, options.kmsg) if test.result.result == 'KILLED' and len(failsafe.pathname): - failsafe.run(options.dryrun) + failsafe.run(options.dryrun, False, options.kmsg) failsafe.log(options, suppress_console=True) else: test.skip() @@ -454,7 +487,7 @@ Tags: %s test.log(options) if len(posttest.pathname): - posttest.run(options.dryrun) + posttest.run(options.dryrun, False, options.kmsg) posttest.log(options) @@ -557,7 +590,7 @@ Tags: %s cont = True if len(pretest.pathname): - pretest.run(options.dryrun) + pretest.run(options.dryrun, False, options.kmsg) cont = pretest.result.result == 'PASS' pretest.log(options) @@ -570,9 +603,9 @@ Tags: %s failsafe = Cmd(self.failsafe, outputdir=odir, timeout=self.timeout, user=self.failsafe_user, identifier=self.identifier) if cont: - test.run(options.dryrun) + test.run(options.dryrun, options.kmemleak, options.kmsg) if test.result.result == 'KILLED' and len(failsafe.pathname): - failsafe.run(options.dryrun) + failsafe.run(options.dryrun, False, options.kmsg) failsafe.log(options, suppress_console=True) else: test.skip() @@ -580,7 +613,7 @@ Tags: %s test.log(options) if len(posttest.pathname): - posttest.run(options.dryrun) + posttest.run(options.dryrun, False, options.kmsg) posttest.log(options) @@ -845,6 +878,11 @@ class TestRun(object): else: write_log('Could not make a symlink to directory %s\n' % self.outputdir, LOG_ERR) + + if options.kmemleak: + cmd = f'{SUDO} -c "echo scan=0 > {KMEMLEAK_FILE}"' + check_output(cmd, shell=True) + iteration = 0 while iteration < options.iterations: for test in sorted(self.tests.keys()): @@ -990,6 +1028,14 @@ def fail(retstr, ret=1): exit(ret) +def kmemleak_cb(option, opt_str, value, parser): + if not os.path.exists(KMEMLEAK_FILE): + fail(f"File '{KMEMLEAK_FILE}' doesn't exist. " + + "Enable CONFIG_DEBUG_KMEMLEAK in kernel configuration.") + + setattr(parser.values, option.dest, True) + + def options_cb(option, opt_str, value, parser): path_options = ['outputdir', 'template', 'testdir', 'logfile'] @@ -1027,6 +1073,11 @@ def parse_args(): parser.add_option('-i', action='callback', callback=options_cb, default=TESTDIR, dest='testdir', type='string', metavar='testdir', help='Specify a test directory.') + parser.add_option('-K', action='store_true', default=False, dest='kmsg', + help='Log tests names to /dev/kmsg') + parser.add_option('-m', action='callback', callback=kmemleak_cb, + default=False, dest='kmemleak', + help='Enable kmemleak reporting (Linux only)') parser.add_option('-p', action='callback', callback=options_cb, default='', dest='pre', metavar='script', type='string', help='Specify a pre script.') diff --git a/tests/test-runner/bin/zts-report.py.in b/tests/test-runner/bin/zts-report.py.in index f3a454722bf4..bf7cf22b61fc 100755 --- a/tests/test-runner/bin/zts-report.py.in +++ b/tests/test-runner/bin/zts-report.py.in @@ -67,7 +67,6 @@ exec_reason = 'Test user execute permissions required for utilities' # additional python modules be installed, for example python3-cffi is required # by the pyzfs tests. # -python_reason = 'Python v3.6 or newer required' python_deps_reason = 'Python modules missing: python3-cffi' # @@ -82,11 +81,6 @@ tmpfile_reason = 'Kernel O_TMPFILE support required' # statx_reason = 'Kernel statx(2) system call required on Linux' -# -# Some tests require that the NFS client and server utilities be installed. -# -share_reason = 'NFS client and server utilities required' - # # Some tests require that the lsattr utility support the project id feature. # @@ -106,14 +100,6 @@ user_ns_reason = 'Kernel user namespace support required' # rewind_reason = 'Arbitrary pool rewind is not guaranteed' -# -# Some tests may by structured in a way that relies on exact knowledge -# of how much free space in available in a pool. These tests cannot be -# made completely reliable because the internal details of how free space -# is managed are not exposed to user space. -# -enospc_reason = 'Exact free space reporting is not guaranteed' - # # Some tests require a minimum version of the fio benchmark utility. # Older distributions such as CentOS 6.x only provide fio-2.0.13. @@ -146,11 +132,6 @@ na_reason = "Not applicable" # ci_reason = 'CI runner doesn\'t have all requirements' -summary = { - 'total': float(0), - 'passed': float(0), - 'logfile': "Could not determine logfile location." -} # # These tests are known to fail, thus we use this list to prevent these @@ -164,10 +145,8 @@ summary = { # reasons listed above can be used. # known = { - 'casenorm/mixed_none_lookup_ci': ['FAIL', '7633'], - 'casenorm/mixed_formd_lookup_ci': ['FAIL', '7633'], - 'cli_root/zfs_unshare/zfs_unshare_002_pos': ['SKIP', na_reason], - 'cli_root/zfs_unshare/zfs_unshare_006_pos': ['SKIP', na_reason], + 'casenorm/mixed_none_lookup_ci': ['FAIL', 7633], + 'casenorm/mixed_formd_lookup_ci': ['FAIL', 7633], 'cli_root/zpool_import/import_rewind_device_replaced': ['FAIL', rewind_reason], 'cli_user/misc/zfs_share_001_neg': ['SKIP', na_reason], @@ -175,24 +154,30 @@ known = { 'privilege/setup': ['SKIP', na_reason], 'refreserv/refreserv_004_pos': ['FAIL', known_reason], 'rootpool/setup': ['SKIP', na_reason], - 'rsend/rsend_008_pos': ['SKIP', '6066'], + 'rsend/rsend_008_pos': ['SKIP', 6066], 'vdev_zaps/vdev_zaps_007_pos': ['FAIL', known_reason], } if sys.platform.startswith('freebsd'): known.update({ + 'cli_root/zfs_receive/receive-o-x_props_override': + ['FAIL', known_reason], 'cli_root/zpool_wait/zpool_wait_trim_basic': ['SKIP', trim_reason], 'cli_root/zpool_wait/zpool_wait_trim_cancel': ['SKIP', trim_reason], 'cli_root/zpool_wait/zpool_wait_trim_flag': ['SKIP', trim_reason], + 'cli_root/zfs_unshare/zfs_unshare_008_pos': ['SKIP', na_reason], 'link_count/link_count_001': ['SKIP', na_reason], + 'casenorm/mixed_create_failure': ['FAIL', 13215], + 'mmap/mmap_sync_001_pos': ['SKIP', na_reason], }) elif sys.platform.startswith('linux'): known.update({ - 'casenorm/mixed_formd_lookup': ['FAIL', '7633'], - 'casenorm/mixed_formd_delete': ['FAIL', '7633'], - 'casenorm/sensitive_formd_lookup': ['FAIL', '7633'], - 'casenorm/sensitive_formd_delete': ['FAIL', '7633'], + 'casenorm/mixed_formd_lookup': ['FAIL', 7633], + 'casenorm/mixed_formd_delete': ['FAIL', 7633], + 'casenorm/sensitive_formd_lookup': ['FAIL', 7633], + 'casenorm/sensitive_formd_delete': ['FAIL', 7633], 'removal/removal_with_zdb': ['SKIP', known_reason], + 'cli_root/zfs_unshare/zfs_unshare_002_pos': ['SKIP', na_reason], }) @@ -214,53 +199,47 @@ maybe = { 'cli_root/zfs_destroy/zfs_destroy_dev_removal_condense': ['FAIL', known_reason], 'cli_root/zfs_get/zfs_get_004_pos': ['FAIL', known_reason], - 'cli_root/zfs_get/zfs_get_009_pos': ['SKIP', '5479'], + 'cli_root/zfs_get/zfs_get_009_pos': ['SKIP', 5479], 'cli_root/zfs_rollback/zfs_rollback_001_pos': ['FAIL', known_reason], 'cli_root/zfs_rollback/zfs_rollback_002_pos': ['FAIL', known_reason], - 'cli_root/zfs_share/setup': ['SKIP', share_reason], 'cli_root/zfs_snapshot/zfs_snapshot_002_neg': ['FAIL', known_reason], - 'cli_root/zfs_unshare/setup': ['SKIP', share_reason], + 'cli_root/zfs_unshare/zfs_unshare_006_pos': ['SKIP', na_reason], 'cli_root/zpool_add/zpool_add_004_pos': ['FAIL', known_reason], - 'cli_root/zpool_destroy/zpool_destroy_001_pos': ['SKIP', '6145'], - 'cli_root/zpool_import/import_rewind_config_changed': - ['FAIL', rewind_reason], - 'cli_root/zpool_import/zpool_import_missing_003_pos': ['SKIP', '6839'], + 'cli_root/zpool_destroy/zpool_destroy_001_pos': ['SKIP', 6145], + 'cli_root/zpool_import/zpool_import_missing_003_pos': ['SKIP', 6839], 'cli_root/zpool_initialize/zpool_initialize_import_export': - ['FAIL', '11948'], + ['FAIL', 11948], 'cli_root/zpool_labelclear/zpool_labelclear_removed': ['FAIL', known_reason], 'cli_root/zpool_trim/setup': ['SKIP', trim_reason], - 'cli_root/zpool_upgrade/zpool_upgrade_004_pos': ['FAIL', '6141'], + 'cli_root/zpool_upgrade/zpool_upgrade_004_pos': ['FAIL', 6141], 'delegate/setup': ['SKIP', exec_reason], 'fallocate/fallocate_punch-hole': ['SKIP', fspacectl_reason], - 'history/history_004_pos': ['FAIL', '7026'], - 'history/history_005_neg': ['FAIL', '6680'], - 'history/history_006_neg': ['FAIL', '5657'], + 'history/history_004_pos': ['FAIL', 7026], + 'history/history_005_neg': ['FAIL', 6680], + 'history/history_006_neg': ['FAIL', 5657], 'history/history_008_pos': ['FAIL', known_reason], 'history/history_010_pos': ['SKIP', exec_reason], 'io/mmap': ['SKIP', fio_reason], 'largest_pool/largest_pool_001_pos': ['FAIL', known_reason], 'mmp/mmp_on_uberblocks': ['FAIL', known_reason], 'pyzfs/pyzfs_unittest': ['SKIP', python_deps_reason], - 'pool_checkpoint/checkpoint_discard_busy': ['FAIL', '11946'], + 'pool_checkpoint/checkpoint_discard_busy': ['FAIL', 11946], 'projectquota/setup': ['SKIP', exec_reason], - 'redundancy/redundancy_004_neg': ['FAIL', '7290'], - 'redundancy/redundancy_draid_spare3': ['SKIP', known_reason], 'removal/removal_condense_export': ['FAIL', known_reason], - 'reservation/reservation_008_pos': ['FAIL', '7741'], - 'reservation/reservation_018_pos': ['FAIL', '5642'], + 'reservation/reservation_008_pos': ['FAIL', 7741], + 'reservation/reservation_018_pos': ['FAIL', 5642], 'snapshot/clone_001_pos': ['FAIL', known_reason], - 'snapshot/snapshot_009_pos': ['FAIL', '7961'], - 'snapshot/snapshot_010_pos': ['FAIL', '7961'], - 'snapused/snapused_004_pos': ['FAIL', '5513'], + 'snapshot/snapshot_009_pos': ['FAIL', 7961], + 'snapshot/snapshot_010_pos': ['FAIL', 7961], + 'snapused/snapused_004_pos': ['FAIL', 5513], 'tmpfile/setup': ['SKIP', tmpfile_reason], - 'threadsappend/threadsappend_001_pos': ['FAIL', '6136'], + 'append/threadsappend_001_pos': ['FAIL', 6136], 'trim/setup': ['SKIP', trim_reason], 'upgrade/upgrade_projectquota_001_pos': ['SKIP', project_id_reason], 'user_namespace/setup': ['SKIP', user_ns_reason], 'userquota/setup': ['SKIP', exec_reason], - 'vdev_zaps/vdev_zaps_004_pos': ['FAIL', '6935'], - 'zvol/zvol_ENOSPC/zvol_ENOSPC_001_pos': ['FAIL', '5848'], + 'zvol/zvol_ENOSPC/zvol_ENOSPC_001_pos': ['FAIL', 5848], 'pam/setup': ['SKIP', "pamtester might be not available"], } @@ -268,30 +247,28 @@ if sys.platform.startswith('freebsd'): maybe.update({ 'cli_root/zfs_copies/zfs_copies_002_pos': ['FAIL', known_reason], 'cli_root/zfs_inherit/zfs_inherit_001_neg': ['FAIL', known_reason], - 'cli_root/zfs_receive/receive-o-x_props_override': - ['FAIL', known_reason], - 'cli_root/zfs_share/zfs_share_011_pos': ['FAIL', known_reason], 'cli_root/zfs_share/zfs_share_concurrent_shares': ['FAIL', known_reason], 'cli_root/zpool_import/zpool_import_012_pos': ['FAIL', known_reason], 'delegate/zfs_allow_003_pos': ['FAIL', known_reason], - 'inheritance/inherit_001_pos': ['FAIL', '11829'], + 'inheritance/inherit_001_pos': ['FAIL', 11829], 'resilver/resilver_restart_001': ['FAIL', known_reason], - 'pool_checkpoint/checkpoint_big_rewind': ['FAIL', '12622'], - 'pool_checkpoint/checkpoint_indirect': ['FAIL', '12623'], + 'pool_checkpoint/checkpoint_big_rewind': ['FAIL', 12622], + 'pool_checkpoint/checkpoint_indirect': ['FAIL', 12623], }) elif sys.platform.startswith('linux'): maybe.update({ 'cli_root/zfs_rename/zfs_rename_002_pos': ['FAIL', known_reason], 'cli_root/zpool_reopen/zpool_reopen_003_pos': ['FAIL', known_reason], - 'fault/auto_spare_shared': ['FAIL', '11889'], + 'fault/auto_spare_shared': ['FAIL', 11889], + 'fault/auto_spare_multiple': ['FAIL', 11889], 'io/io_uring': ['SKIP', 'io_uring support required'], 'limits/filesystem_limit': ['SKIP', known_reason], 'limits/snapshot_limit': ['SKIP', known_reason], 'mmp/mmp_active_import': ['FAIL', known_reason], 'mmp/mmp_exported_import': ['FAIL', known_reason], 'mmp/mmp_inactive_import': ['FAIL', known_reason], - 'zvol/zvol_misc/zvol_misc_snapdev': ['FAIL', '12621'], + 'zvol/zvol_misc/zvol_misc_snapdev': ['FAIL', 12621], 'zvol/zvol_misc/zvol_misc_volmode': ['FAIL', known_reason], }) @@ -322,44 +299,37 @@ if os.environ.get('CI') == 'true': }) maybe.update({ - 'events/events_002_pos': ['FAIL', '11546'], + 'events/events_002_pos': ['FAIL', 11546], }) -def usage(s): - print(s) - sys.exit(1) - - def process_results(pathname): try: f = open(pathname) except IOError as e: - print('Error opening file: %s' % e) + print('Error opening file:', e) sys.exit(1) prefix = '/zfs-tests/tests/functional/' pattern = \ r'^Test(?:\s+\(\S+\))?:' + \ - r'\s*\S*%s(\S+)\s*\(run as (\S+)\)\s*\[(\S+)\]\s*\[(\S+)\]' \ - % prefix + rf'\s*\S*{prefix}(\S+)' + \ + r'\s*\(run as (\S+)\)\s*\[(\S+)\]\s*\[(\S+)\]' pattern_log = r'^\s*Log directory:\s*(\S*)' d = {} + logdir = 'Could not determine log directory.' for line in f.readlines(): m = re.match(pattern, line) if m and len(m.groups()) == 4: - summary['total'] += 1 - if m.group(4) == "PASS": - summary['passed'] += 1 d[m.group(1)] = m.group(4) continue m = re.match(pattern_log, line) if m: - summary['logfile'] = m.group(1) + logdir = m.group(1) - return d + return d, logdir class ListMaybesAction(argparse.Action): @@ -388,11 +358,11 @@ if __name__ == "__main__": parser.add_argument('--no-maybes', action='store_false', dest='maybes') args = parser.parse_args() - results = process_results(args.logfile) + results, logdir = process_results(args.logfile) - if summary['total'] == 0: + if not results: print("\n\nNo test results were found.") - print("Log directory: %s" % summary['logfile']) + print("Log directory:", logdir) sys.exit(0) expected = [] @@ -432,13 +402,13 @@ if __name__ == "__main__": if test in known: if known[test][1] == na_reason: continue - elif known[test][1].isdigit(): - expect = issue_url + known[test][1] + elif isinstance(known[test][1], int): + expect = f"{issue_url}{known[test][1]}" else: expect = known[test][1] elif test in maybe: - if maybe[test][1].isdigit(): - expect = issue_url + maybe[test][1] + if isinstance(maybe[test][1], int): + expect = f"{issue_url}{maybe[test][1]}" else: expect = maybe[test][1] elif setup in known and known[setup][0] == "SKIP" and setup != test: @@ -447,7 +417,7 @@ if __name__ == "__main__": continue else: expect = "UNKNOWN REASON" - print(" %s %s (%s)" % (results[test], test, expect)) + print(f" {results[test]} {test} ({expect})") print("\nTests with result of PASS that are unexpected:") for test in sorted(known.keys()): @@ -455,13 +425,12 @@ if __name__ == "__main__": # where "test" is not in "results". if test not in results or results[test] != "PASS": continue - print(" %s %s (expected %s)" % (results[test], test, - known[test][0])) + print(f" {results[test]} {test} (expected {known[test][0]})") print("\nTests with results other than PASS that are unexpected:") for test in sorted(unexpected): expect = "PASS" if test not in known else known[test][0] - print(" %s %s (expected %s)" % (results[test], test, expect)) + print(f" {results[test]} {test} (expected {expect})") if len(unexpected) == 0: sys.exit(0) diff --git a/tests/test-runner/include/Makefile.am b/tests/test-runner/include/Makefile.am deleted file mode 100644 index d3eeb32de914..000000000000 --- a/tests/test-runner/include/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/test-runner/include - -dist_pkgdata_DATA = \ - logapi.shlib \ - stf.shlib diff --git a/tests/test-runner/include/logapi.shlib b/tests/test-runner/include/logapi.shlib index c9c01ab752ea..8a9ca3e87caf 100644 --- a/tests/test-runner/include/logapi.shlib +++ b/tests/test-runner/include/logapi.shlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -26,7 +26,11 @@ # Copyright (c) 2012, 2020 by Delphix. All rights reserved. # -. ${STF_TOOLS}/include/stf.shlib +STF_PASS=0 +STF_FAIL=1 +STF_UNRESOLVED=2 +STF_UNSUPPORTED=4 +STF_UNTESTED=5 # Output an assertion # @@ -54,8 +58,7 @@ function log_note function log_neg { - log_neg_expect "" "$@" - return $? + log_neg_expect "" "$@" } # Execute a positive test and exit $STF_FAIL is test fails @@ -64,8 +67,7 @@ function log_neg function log_must { - log_pos "$@" - (( $? != 0 )) && log_fail + log_pos "$@" || log_fail } # Execute a positive test (expecting no stderr) and exit $STF_FAIL @@ -74,8 +76,7 @@ function log_must function log_must_nostderr { - log_pos_nostderr "$@" - (( $? != 0 )) && log_fail + log_pos_nostderr "$@" || log_fail } # Execute a positive test but retry the command on failure if the output @@ -88,7 +89,6 @@ function log_must_nostderr # function log_must_retry { - typeset out="" typeset logfile="/tmp/log.$$" typeset status=1 typeset expect=$1 @@ -103,14 +103,10 @@ function log_must_retry while (( $retry > 0 )); do "$@" 2>$logfile status=$? - out="cat $logfile" if (( $status == 0 )); then - $out | egrep -i "internal error|assertion failed" \ - > /dev/null 2>&1 - # internal error or assertion failed - if [[ $? -eq 0 ]]; then - print -u2 $($out) + if grep -qEi "internal error|assertion failed" $logfile; then + cat $logfile >&2 _printerror "$@" "internal error or" \ " assertion failure exited $status" status=1 @@ -120,9 +116,8 @@ function log_must_retry fi break else - $out | grep -i "$expect" > /dev/null 2>&1 - if (( $? == 0 )); then - print -u2 $($out) + if grep -qi "$expect" $logfile; then + cat $logfile >&2 _printerror "$@" "Retry in $delay seconds" sleep $delay @@ -135,7 +130,7 @@ function log_must_retry done if (( $status != 0 )) ; then - print -u2 $($out) + cat $logfile >&2 _printerror "$@" "exited $status" fi @@ -149,8 +144,7 @@ function log_must_retry # $@ - command to execute function log_must_busy { - log_must_retry "busy" 5 "$@" - (( $? != 0 )) && log_fail + log_must_retry "busy" 5 "$@" || log_fail } # Execute a negative test and exit $STF_FAIL if test passes @@ -159,8 +153,7 @@ function log_must_busy function log_mustnot { - log_neg "$@" - (( $? != 0 )) && log_fail + log_neg "$@" || log_fail } # Execute a negative test with keyword expected, and exit @@ -171,8 +164,7 @@ function log_mustnot function log_mustnot_expect { - log_neg_expect "$@" - (( $? != 0 )) && log_fail + log_neg_expect "$@" || log_fail } # Signal numbers are platform-dependent @@ -203,7 +195,6 @@ EXIT_SIGSEGV=$((EXIT_SIGNAL + SIGSEGV)) function log_neg_expect { - typeset out="" typeset logfile="/tmp/log.$$" typeset ret=1 typeset expect=$1 @@ -215,38 +206,33 @@ function log_neg_expect "$@" 2>$logfile typeset status=$? - out="cat $logfile" # unexpected status if (( $status == EXIT_SUCCESS )); then - print -u2 $($out) + cat $logfile >&2 _printerror "$@" "unexpectedly exited $status" # missing binary elif (( $status == EXIT_NOTFOUND )); then - print -u2 $($out) + cat $logfile >&2 _printerror "$@" "unexpectedly exited $status (File not found)" # bus error - core dump elif (( $status == EXIT_SIGBUS )); then - print -u2 $($out) + cat $logfile >&2 _printerror "$@" "unexpectedly exited $status (Bus Error)" # segmentation violation - core dump elif (( $status == EXIT_SIGSEGV )); then - print -u2 $($out) + cat $logfile >&2 _printerror "$@" "unexpectedly exited $status (SEGV)" else - $out | egrep -i "internal error|assertion failed" \ - > /dev/null 2>&1 - # internal error or assertion failed - if (( $? == 0 )); then - print -u2 $($out) + if grep -qEi "internal error|assertion failed" $logfile; then + cat $logfile >&2 _printerror "$@" "internal error or assertion failure" \ " exited $status" elif [[ -n $expect ]] ; then - $out | grep -i "$expect" > /dev/null 2>&1 - if (( $? == 0 )); then + if grep -qi "$expect" $logfile; then ret=0 else - print -u2 $($out) + cat $logfile >&2 _printerror "$@" "unexpectedly exited $status" fi else @@ -270,7 +256,6 @@ function log_neg_expect function log_pos { - typeset out="" typeset logfile="/tmp/log.$$" while [[ -e $logfile ]]; do @@ -279,17 +264,13 @@ function log_pos "$@" 2>$logfile typeset status=$? - out="cat $logfile" if (( $status != 0 )) ; then - print -u2 $($out) + cat $logfile >&2 _printerror "$@" "exited $status" else - $out | egrep -i "internal error|assertion failed" \ - > /dev/null 2>&1 - # internal error or assertion failed - if [[ $? -eq 0 ]]; then - print -u2 $($out) + if grep -qEi "internal error|assertion failed" $logfile; then + cat $logfile >&2 _printerror "$@" "internal error or assertion failure" \ " exited $status" status=1 @@ -312,7 +293,6 @@ function log_pos function log_pos_nostderr { - typeset out="" typeset logfile="/tmp/log.$$" while [[ -e $logfile ]]; do @@ -321,15 +301,13 @@ function log_pos_nostderr "$@" 2>$logfile typeset status=$? - out="cat $logfile" - typeset out_msg=$($out) if (( $status != 0 )) ; then - print -u2 $out_msg + cat $logfile >&2 _printerror "$@" "exited $status" else - if [[ ! -z "$out_msg" ]]; then - print -u2 $out_msg + if [ -s "$logfile" ]; then + cat $logfile >&2 _printerror "$@" "message in stderr" \ " exited $status" status=1 @@ -398,15 +376,6 @@ function log_unresolved _endlog $STF_UNRESOLVED "$@" } -# Perform cleanup and exit $STF_NOTINUSE -# -# $@ - message text - -function log_notinuse -{ - _endlog $STF_NOTINUSE "$@" -} - # Perform cleanup and exit $STF_UNSUPPORTED # # $@ - message text @@ -425,51 +394,6 @@ function log_untested _endlog $STF_UNTESTED "$@" } -# Perform cleanup and exit $STF_UNINITIATED -# -# $@ - message text - -function log_uninitiated -{ - _endlog $STF_UNINITIATED "$@" -} - -# Perform cleanup and exit $STF_NORESULT -# -# $@ - message text - -function log_noresult -{ - _endlog $STF_NORESULT "$@" -} - -# Perform cleanup and exit $STF_WARNING -# -# $@ - message text - -function log_warning -{ - _endlog $STF_WARNING "$@" -} - -# Perform cleanup and exit $STF_TIMED_OUT -# -# $@ - message text - -function log_timed_out -{ - _endlog $STF_TIMED_OUT "$@" -} - -# Perform cleanup and exit $STF_OTHER -# -# $@ - message text - -function log_other -{ - _endlog $STF_OTHER "$@" -} - function set_main_pid { _MAINPID=$1 @@ -487,12 +411,12 @@ function _execute_testfail_callbacks { typeset callback - print "$TESTFAIL_CALLBACKS:" | while read -d ":" callback; do + while read -d ":" callback; do if [[ -n "$callback" ]] ; then log_note "Performing test-fail callback ($callback)" $callback fi - done + done <<<"$TESTFAIL_CALLBACKS:" } # Perform cleanup and exit @@ -540,7 +464,7 @@ function _endlog function _printline { - print "$@" + echo "$@" } # Output an error message @@ -576,5 +500,5 @@ function _recursive_output #logfile fi rm -f $logfile logfile="$logfile.$$" - done + done } diff --git a/tests/test-runner/man/Makefile.am b/tests/test-runner/man/Makefile.am deleted file mode 100644 index a7017f5f0535..000000000000 --- a/tests/test-runner/man/Makefile.am +++ /dev/null @@ -1,4 +0,0 @@ -dist_man_MANS = test-runner.1 - -install-data-local: - $(INSTALL) -d -m 0755 "$(DESTDIR)$(mandir)/man1" diff --git a/tests/zfs-tests/.gitignore b/tests/zfs-tests/.gitignore new file mode 100644 index 000000000000..ba077a4031ad --- /dev/null +++ b/tests/zfs-tests/.gitignore @@ -0,0 +1 @@ +bin diff --git a/tests/zfs-tests/Makefile.am b/tests/zfs-tests/Makefile.am index ef4e6be9e980..f8166352489e 100644 --- a/tests/zfs-tests/Makefile.am +++ b/tests/zfs-tests/Makefile.am @@ -1 +1,48 @@ -SUBDIRS = cmd include tests callbacks +SUBDIRS += %D%/tests + +include $(srcdir)/%D%/cmd/Makefile.am + + +scripts_zfs_tests_functional_libzfsdir = $(datadir)/$(PACKAGE)/zfs-tests/tests/functional/libzfs +scripts_zfs_tests_functional_libzfs_PROGRAMS = %D%/tests/functional/libzfs/many_fds +%C%_tests_functional_libzfs_many_fds_LDADD = \ + libzfs.la + +scripts_zfs_tests_functional_hkdfdir = $(datadir)/$(PACKAGE)/zfs-tests/tests/functional/hkdf +scripts_zfs_tests_functional_hkdf_PROGRAMS = %D%/tests/functional/hkdf/hkdf_test +%C%_tests_functional_hkdf_hkdf_test_LDADD = \ + libzpool.la + +if BUILD_LINUX +scripts_zfs_tests_functional_tmpfiledir = $(datadir)/$(PACKAGE)/zfs-tests/tests/functional/tmpfile +scripts_zfs_tests_functional_tmpfile_PROGRAMS = \ + %D%/tests/functional/tmpfile/tmpfile_001_pos \ + %D%/tests/functional/tmpfile/tmpfile_002_pos \ + %D%/tests/functional/tmpfile/tmpfile_003_pos \ + %D%/tests/functional/tmpfile/tmpfile_stat_mode \ + %D%/tests/functional/tmpfile/tmpfile_test +endif + + +scripts_zfs_tests_callbacksdir = $(datadir)/$(PACKAGE)/zfs-tests/callbacks +dist_scripts_zfs_tests_callbacks_SCRIPTS = \ + %D%/callbacks/zfs_dbgmsg.ksh \ + %D%/callbacks/zfs_dmesg.ksh \ + %D%/callbacks/zfs_failsafe.ksh \ + %D%/callbacks/zfs_mmp.ksh + + +scripts_zfs_tests_includedir = $(datadir)/$(PACKAGE)/zfs-tests/include +dist_scripts_zfs_tests_include_DATA = \ + %D%/include/blkdev.shlib \ + %D%/include/commands.cfg \ + %D%/include/libtest.shlib \ + %D%/include/math.shlib \ + %D%/include/properties.shlib \ + %D%/include/tunables.cfg \ + %D%/include/zpool_script.shlib + +nodist_scripts_zfs_tests_include_DATA = \ + %D%/include/default.cfg + +SUBSTFILES += $(nodist_scripts_zfs_tests_include_DATA) diff --git a/tests/zfs-tests/callbacks/Makefile.am b/tests/zfs-tests/callbacks/Makefile.am deleted file mode 100644 index 512a737bb5c9..000000000000 --- a/tests/zfs-tests/callbacks/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/callbacks -dist_pkgdata_SCRIPTS = \ - zfs_failsafe.ksh \ - zfs_dbgmsg.ksh \ - zfs_dmesg.ksh \ - zfs_mmp.ksh diff --git a/tests/zfs-tests/cmd/.gitignore b/tests/zfs-tests/cmd/.gitignore new file mode 100644 index 000000000000..20d1382532bd --- /dev/null +++ b/tests/zfs-tests/cmd/.gitignore @@ -0,0 +1,48 @@ +/badsend +/btree_test +/chg_usr_exec +/devname2devid +/dir_rd_update +/draid +/file_append +/file_check +/file_trunc +/file_write +/get_diff +/getversion +/largest_file +/libzfs_input_check +/mkbusy +/mkfile +/mkfiles +/mktree +/mmap_exec +/mmap_libaio +/mmap_seek +/mmap_sync +/mmapwrite +/nvlist_to_lua +/randfree_file +/randwritecomp +/read_dos_attributes +/readmmap +/rename_dir +/rm_lnkcnt_zero_file +/send_doall +/stride_dd +/threadsappend +/user_ns_exec +/write_dos_attributes +/xattrtest +/zed_fd_spill-zedlet +/suid_write_to_file +/cp_files +/ctime +/truncate_test +/ereports +/zfs_diff-socket +/dosmode_readonly_write +/blake3_test +/edonr_test +/skein_test +/sha2_test diff --git a/tests/zfs-tests/cmd/Makefile.am b/tests/zfs-tests/cmd/Makefile.am index 2470397a90a3..3c8faf5afbbb 100644 --- a/tests/zfs-tests/cmd/Makefile.am +++ b/tests/zfs-tests/cmd/Makefile.am @@ -1,39 +1,131 @@ -EXTRA_DIST = file_common.h - -SUBDIRS = \ - badsend \ - btree_test \ - chg_usr_exec \ - devname2devid \ - dir_rd_update \ - draid \ - file_check \ - file_trunc \ - file_write \ - get_diff \ - largest_file \ - libzfs_input_check \ - mkbusy \ - mkfile \ - mkfiles \ - mktree \ - mmap_exec \ - mmap_libaio \ - mmap_seek \ - mmapwrite \ - nvlist_to_lua \ - randwritecomp \ - readmmap \ - rename_dir \ - rm_lnkcnt_zero_file \ - send_doall \ - stride_dd \ - threadsappend +scripts_zfs_tests_bindir = $(datadir)/$(PACKAGE)/zfs-tests/bin + + +scripts_zfs_tests_bin_PROGRAMS = %D%/chg_usr_exec +scripts_zfs_tests_bin_PROGRAMS += %D%/cp_files +scripts_zfs_tests_bin_PROGRAMS += %D%/ctime +scripts_zfs_tests_bin_PROGRAMS += %D%/dir_rd_update +scripts_zfs_tests_bin_PROGRAMS += %D%/dosmode_readonly_write +scripts_zfs_tests_bin_PROGRAMS += %D%/get_diff +scripts_zfs_tests_bin_PROGRAMS += %D%/rename_dir +scripts_zfs_tests_bin_PROGRAMS += %D%/suid_write_to_file +scripts_zfs_tests_bin_PROGRAMS += %D%/truncate_test +scripts_zfs_tests_bin_PROGRAMS += %D%/zfs_diff-socket + + +scripts_zfs_tests_bin_PROGRAMS += %D%/badsend +%C%_badsend_LDADD = \ + libzfs_core.la \ + libzfs.la \ + libnvpair.la + + +scripts_zfs_tests_bin_PROGRAMS += %D%/btree_test +%C%_btree_test_CPPFLAGS = $(AM_CPPFLAGS) $(FORCEDEBUG_CPPFLAGS) +%C%_btree_test_LDADD = \ + libzpool.la \ + libzfs_core.la + + +if WANT_DEVNAME2DEVID +scripts_zfs_tests_bin_PROGRAMS += %D%/devname2devid +%C%_devname2devid_CFLAGS = $(AM_CFLAGS) $(LIBUDEV_CFLAGS) +%C%_devname2devid_LDADD = $(LIBUDEV_LIBS) +endif + + +scripts_zfs_tests_bin_PROGRAMS += %D%/draid +%C%_draid_CFLAGS = $(AM_CFLAGS) $(ZLIB_CFLAGS) +%C%_draid_LDADD = \ + libzpool.la \ + libnvpair.la +%C%_draid_LDADD += $(ZLIB_LIBS) + +dist_noinst_DATA += %D%/file/file_common.h +scripts_zfs_tests_bin_PROGRAMS += %D%/file_append %D%/file_check %D%/file_trunc %D%/file_write %D%/largest_file %D%/randwritecomp +%C%_file_append_SOURCES = %D%/file/file_append.c +%C%_file_check_SOURCES = %D%/file/file_check.c +%C%_file_trunc_SOURCES = %D%/file/file_trunc.c +%C%_file_write_SOURCES = %D%/file/file_write.c +%C%_largest_file_SOURCES = %D%/file/largest_file.c +%C%_randwritecomp_SOURCES = %D%/file/randwritecomp.c + + +scripts_zfs_tests_bin_PROGRAMS += %D%/libzfs_input_check +%C%_libzfs_input_check_CPPFLAGS = $(AM_CPPFLAGS) -I$(top_srcdir)/include/os/@ac_system_l@/zfs +%C%_libzfs_input_check_LDADD = \ + libzfs_core.la \ + libnvpair.la + + +scripts_zfs_tests_bin_PROGRAMS += %D%/mkbusy %D%/mkfile %D%/mkfiles %D%/mktree +%C%_mkfile_LDADD = $(LTLIBINTL) + + +scripts_zfs_tests_bin_PROGRAMS += %D%/mmap_exec %D%/mmap_seek %D%/mmap_sync %D%/mmapwrite %D%/readmmap +%C%_mmapwrite_LDADD = -lpthread + +if WANT_MMAP_LIBAIO +scripts_zfs_tests_bin_PROGRAMS += %D%/mmap_libaio +%C%_mmap_libaio_CFLAGS = $(AM_CFLAGS) $(LIBAIO_CFLAGS) +%C%_mmap_libaio_LDADD = $(LIBAIO_LIBS) +endif + + +scripts_zfs_tests_bin_PROGRAMS += %D%/nvlist_to_lua +%C%_nvlist_to_lua_LDADD = \ + libzfs_core.la \ + libnvpair.la + +scripts_zfs_tests_bin_PROGRAMS += %D%/rm_lnkcnt_zero_file +%C%_rm_lnkcnt_zero_file_LDADD = -lpthread + +scripts_zfs_tests_bin_PROGRAMS += %D%/send_doall +%C%_send_doall_LDADD = \ + libzfs_core.la \ + libzfs.la \ + libnvpair.la + +scripts_zfs_tests_bin_PROGRAMS += %D%/stride_dd +%C%_stride_dd_LDADD = -lrt + +scripts_zfs_tests_bin_PROGRAMS += %D%/threadsappend +%C%_threadsappend_LDADD = -lpthread + +scripts_zfs_tests_bin_PROGRAMS += %D%/ereports +%C%_ereports_LDADD = \ + libnvpair.la \ + libzfs.la + + +scripts_zfs_tests_bin_PROGRAMS += %D%/edonr_test %D%/skein_test \ + %D%/sha2_test %D%/blake3_test +%C%_skein_test_SOURCES = %D%/checksum/skein_test.c +%C%_sha2_test_SOURCES = %D%/checksum/sha2_test.c +%C%_edonr_test_SOURCES = %D%/checksum/edonr_test.c +%C%_blake3_test_SOURCES = %D%/checksum/blake3_test.c +%C%_skein_test_LDADD = \ + libicp.la \ + libspl.la \ + libspl_assert.la +%C%_sha2_test_LDADD = $(%C%_skein_test_LDADD) +%C%_edonr_test_LDADD = $(%C%_skein_test_LDADD) +%C%_blake3_test_LDADD = $(%C%_skein_test_LDADD) + if BUILD_LINUX -SUBDIRS += \ - getversion \ - randfree_file \ - user_ns_exec \ - xattrtest +scripts_zfs_tests_bin_PROGRAMS += %D%/getversion +scripts_zfs_tests_bin_PROGRAMS += %D%/user_ns_exec +scripts_zfs_tests_bin_PROGRAMS += %D%/xattrtest +scripts_zfs_tests_bin_PROGRAMS += %D%/zed_fd_spill-zedlet + + +dist_noinst_DATA += %D%/linux_dos_attributes/dos_attributes.h +scripts_zfs_tests_bin_PROGRAMS += %D%/read_dos_attributes %D%/write_dos_attributes +%C%_read_dos_attributes_SOURCES = %D%/linux_dos_attributes/read_dos_attributes.c +%C%_write_dos_attributes_SOURCES = %D%/linux_dos_attributes/write_dos_attributes.c + + +scripts_zfs_tests_bin_PROGRAMS += %D%/randfree_file +%C%_randfree_file_SOURCES = %D%/file/randfree_file.c endif diff --git a/tests/zfs-tests/cmd/badsend/badsend.c b/tests/zfs-tests/cmd/badsend.c similarity index 98% rename from tests/zfs-tests/cmd/badsend/badsend.c rename to tests/zfs-tests/cmd/badsend.c index af17bc7255b4..48d05bd3fd4c 100644 --- a/tests/zfs-tests/cmd/badsend/badsend.c +++ b/tests/zfs-tests/cmd/badsend.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/tests/zfs-tests/cmd/badsend/.gitignore b/tests/zfs-tests/cmd/badsend/.gitignore deleted file mode 100644 index d2efa627aa29..000000000000 --- a/tests/zfs-tests/cmd/badsend/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/badsend diff --git a/tests/zfs-tests/cmd/badsend/Makefile.am b/tests/zfs-tests/cmd/badsend/Makefile.am deleted file mode 100644 index 5a8946f0d4bf..000000000000 --- a/tests/zfs-tests/cmd/badsend/Makefile.am +++ /dev/null @@ -1,11 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = badsend - -badsend_SOURCES = badsend.c -badsend_LDADD = \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la \ - $(abs_top_builddir)/lib/libzfs/libzfs.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la diff --git a/tests/zfs-tests/cmd/btree_test/btree_test.c b/tests/zfs-tests/cmd/btree_test.c similarity index 100% rename from tests/zfs-tests/cmd/btree_test/btree_test.c rename to tests/zfs-tests/cmd/btree_test.c diff --git a/tests/zfs-tests/cmd/btree_test/.gitignore b/tests/zfs-tests/cmd/btree_test/.gitignore deleted file mode 100644 index 73777c4c1f4b..000000000000 --- a/tests/zfs-tests/cmd/btree_test/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/btree_test diff --git a/tests/zfs-tests/cmd/btree_test/Makefile.am b/tests/zfs-tests/cmd/btree_test/Makefile.am deleted file mode 100644 index 4c9a1a4cc296..000000000000 --- a/tests/zfs-tests/cmd/btree_test/Makefile.am +++ /dev/null @@ -1,32 +0,0 @@ -# -# This file and its contents are supplied under the terms of the -# Common Development and Distribution License ("CDDL"), version 1.0. -# You may only use this file in accordance with the terms of version -# 1.0 of the CDDL. -# -# A full copy of the text of the CDDL should have accompanied this -# source. A copy of the CDDL is also available via the Internet at -# http://www.illumos.org/license/CDDL. -# - -# -# Copyright (c) 2019 by Delphix. All rights reserved. -# - -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -DEFAULT_INCLUDES += \ - -I$(top_srcdir)/include \ - -I$(top_srcdir)/lib/libspl/include - -# Unconditionally enable ASSERTs -AM_CPPFLAGS += -DDEBUG -UNDEBUG -DZFS_DEBUG - -pkgexec_PROGRAMS = btree_test -btree_test_SOURCES = btree_test.c - -btree_test_LDADD = \ - $(abs_top_builddir)/lib/libzpool/libzpool.la \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la diff --git a/tests/zfs-tests/cmd/checksum/blake3_test.c b/tests/zfs-tests/cmd/checksum/blake3_test.c new file mode 100644 index 000000000000..d57d0e047f01 --- /dev/null +++ b/tests/zfs-tests/cmd/checksum/blake3_test.c @@ -0,0 +1,575 @@ + +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2022 Tino Reichardt + */ + +#include +#include +#include +#include +#include +#include + +/* + * set it to a define for debugging + */ +#undef BLAKE3_DEBUG + +/* + * C version of: + * https://github.com/BLAKE3-team/BLAKE3/tree/master/test_vectors + */ +typedef struct { + /* input length for this entry */ + const int input_len; + + /* hash value */ + const char *hash; + + /* salted hash value */ + const char *shash; +} blake3_test_t; + +/* BLAKE3 is variable here */ +#define TEST_DIGEST_LEN 262 + +/* + * key for the keyed hashing + */ +static const char *salt = "whats the Elvish word for friend"; + +static blake3_test_t TestArray[] = { + { + 0, + "af1349b9f5f9a1a6a0404dea36dcc9499bcb25c9adc112b7cc9a93cae41f3262e0" + "0f03e7b69af26b7faaf09fcd333050338ddfe085b8cc869ca98b206c08243a26f5" + "487789e8f660afe6c99ef9e0c52b92e7393024a80459cf91f476f9ffdbda7001c2" + "2e159b402631f277ca96f2defdf1078282314e763699a31c5363165421cce14d", + "92b2b75604ed3c761f9d6f62392c8a9227ad0ea3f09573e783f1498a4ed60d26b1" + "8171a2f22a4b94822c701f107153dba24918c4bae4d2945c20ece13387627d3b73" + "cbf97b797d5e59948c7ef788f54372df45e45e4293c7dc18c1d41144a9758be589" + "60856be1eabbe22c2653190de560ca3b2ac4aa692a9210694254c371e851bc8f", + }, + { + 1, + "2d3adedff11b61f14c886e35afa036736dcd87a74d27b5c1510225d0f592e213c3" + "a6cb8bf623e20cdb535f8d1a5ffb86342d9c0b64aca3bce1d31f60adfa137b358a" + "d4d79f97b47c3d5e79f179df87a3b9776ef8325f8329886ba42f07fb138bb502f4" + "081cbcec3195c5871e6c23e2cc97d3c69a613eba131e5f1351f3f1da786545e5", + "6d7878dfff2f485635d39013278ae14f1454b8c0a3a2d34bc1ab38228a80c95b65" + "68c0490609413006fbd428eb3fd14e7756d90f73a4725fad147f7bf70fd61c4e0c" + "f7074885e92b0e3f125978b4154986d4fb202a3f331a3fb6cf349a3a70e49990f9" + "8fe4289761c8602c4e6ab1138d31d3b62218078b2f3ba9a88e1d08d0dd4cea11", + }, + { + 2, + "7b7015bb92cf0b318037702a6cdd81dee41224f734684c2c122cd6359cb1ee63d8" + "386b22e2ddc05836b7c1bb693d92af006deb5ffbc4c70fb44d0195d0c6f252faac" + "61659ef86523aa16517f87cb5f1340e723756ab65efb2f91964e14391de2a43226" + "3a6faf1d146937b35a33621c12d00be8223a7f1919cec0acd12097ff3ab00ab1", + "5392ddae0e0a69d5f40160462cbd9bd889375082ff224ac9c758802b7a6fd20a9f" + "fbf7efd13e989a6c246f96d3a96b9d279f2c4e63fb0bdff633957acf50ee1a5f65" + "8be144bab0f6f16500dee4aa5967fc2c586d85a04caddec90fffb7633f46a60786" + "024353b9e5cebe277fcd9514217fee2267dcda8f7b31697b7c54fab6a939bf8f", + }, + { + 3, + "e1be4d7a8ab5560aa4199eea339849ba8e293d55ca0a81006726d184519e647f5b" + "49b82f805a538c68915c1ae8035c900fd1d4b13902920fd05e1450822f36de9454" + "b7e9996de4900c8e723512883f93f4345f8a58bfe64ee38d3ad71ab027765d25cd" + "d0e448328a8e7a683b9a6af8b0af94fa09010d9186890b096a08471e4230a134", + "39e67b76b5a007d4921969779fe666da67b5213b096084ab674742f0d5ec62b9b9" + "142d0fab08e1b161efdbb28d18afc64d8f72160c958e53a950cdecf91c1a1bbab1" + "a9c0f01def762a77e2e8545d4dec241e98a89b6db2e9a5b070fc110caae2622690" + "bd7b76c02ab60750a3ea75426a6bb8803c370ffe465f07fb57def95df772c39f", + }, + { + 4, + "f30f5ab28fe047904037f77b6da4fea1e27241c5d132638d8bedce9d40494f328f" + "603ba4564453e06cdcee6cbe728a4519bbe6f0d41e8a14b5b225174a566dbfa61b" + "56afb1e452dc08c804f8c3143c9e2cc4a31bb738bf8c1917b55830c6e657972117" + "01dc0b98daa1faeaa6ee9e56ab606ce03a1a881e8f14e87a4acf4646272cfd12", + "7671dde590c95d5ac9616651ff5aa0a27bee5913a348e053b8aa9108917fe07011" + "6c0acff3f0d1fa97ab38d813fd46506089118147d83393019b068a55d646251ecf" + "81105f798d76a10ae413f3d925787d6216a7eb444e510fd56916f1d753a5544ecf" + "0072134a146b2615b42f50c179f56b8fae0788008e3e27c67482349e249cb86a", + }, + { + 5, + "b40b44dfd97e7a84a996a91af8b85188c66c126940ba7aad2e7ae6b385402aa2eb" + "cfdac6c5d32c31209e1f81a454751280db64942ce395104e1e4eaca62607de1c2c" + "a748251754ea5bbe8c20150e7f47efd57012c63b3c6a6632dc1c7cd15f3e1c9999" + "04037d60fac2eb9397f2adbe458d7f264e64f1e73aa927b30988e2aed2f03620", + "73ac69eecf286894d8102018a6fc729f4b1f4247d3703f69bdc6a5fe3e0c84616a" + "b199d1f2f3e53bffb17f0a2209fe8b4f7d4c7bae59c2bc7d01f1ff94c67588cc6b" + "38fa6024886f2c078bfe09b5d9e6584cd6c521c3bb52f4de7687b37117a2dbbec0" + "d59e92fa9a8cc3240d4432f91757aabcae03e87431dac003e7d73574bfdd8218", + }, + { + 6, + "06c4e8ffb6872fad96f9aaca5eee1553eb62aed0ad7198cef42e87f6a616c84461" + "1a30c4e4f37fe2fe23c0883cde5cf7059d88b657c7ed2087e3d210925ede716435" + "d6d5d82597a1e52b9553919e804f5656278bd739880692c94bff2824d8e0b48cac" + "1d24682699e4883389dc4f2faa2eb3b4db6e39debd5061ff3609916f3e07529a", + "82d3199d0013035682cc7f2a399d4c212544376a839aa863a0f4c91220ca7a6dc2" + "ffb3aa05f2631f0fa9ac19b6e97eb7e6669e5ec254799350c8b8d189e880780084" + "2a5383c4d907c932f34490aaf00064de8cdb157357bde37c1504d2960034930887" + "603abc5ccb9f5247f79224baff6120a3c622a46d7b1bcaee02c5025460941256", + }, + { + 7, + "3f8770f387faad08faa9d8414e9f449ac68e6ff0417f673f602a646a891419fe66" + "036ef6e6d1a8f54baa9fed1fc11c77cfb9cff65bae915045027046ebe0c01bf5a9" + "41f3bb0f73791d3fc0b84370f9f30af0cd5b0fc334dd61f70feb60dad785f070fe" + "f1f343ed933b49a5ca0d16a503f599a365a4296739248b28d1a20b0e2cc8975c", + "af0a7ec382aedc0cfd626e49e7628bc7a353a4cb108855541a5651bf64fbb28a7c" + "5035ba0f48a9c73dabb2be0533d02e8fd5d0d5639a18b2803ba6bf527e1d145d5f" + "d6406c437b79bcaad6c7bdf1cf4bd56a893c3eb9510335a7a798548c6753f74617" + "bede88bef924ba4b334f8852476d90b26c5dc4c3668a2519266a562c6c8034a6", + }, + { + 8, + "2351207d04fc16ade43ccab08600939c7c1fa70a5c0aaca76063d04c3228eaeb72" + "5d6d46ceed8f785ab9f2f9b06acfe398c6699c6129da084cb531177445a682894f" + "9685eaf836999221d17c9a64a3a057000524cd2823986db378b074290a1a9b93a2" + "2e135ed2c14c7e20c6d045cd00b903400374126676ea78874d79f2dd7883cf5c", + "be2f5495c61cba1bb348a34948c004045e3bd4dae8f0fe82bf44d0da245a060048" + "eb5e68ce6dea1eb0229e144f578b3aa7e9f4f85febd135df8525e6fe40c6f0340d" + "13dd09b255ccd5112a94238f2be3c0b5b7ecde06580426a93e0708555a265305ab" + "f86d874e34b4995b788e37a823491f25127a502fe0704baa6bfdf04e76c13276", + }, + { + 63, + "e9bc37a594daad83be9470df7f7b3798297c3d834ce80ba85d6e207627b7db7b11" + "97012b1e7d9af4d7cb7bdd1f3bb49a90a9b5dec3ea2bbc6eaebce77f4e470cbf46" + "87093b5352f04e4a4570fba233164e6acc36900e35d185886a827f7ea9bdc1e5c3" + "ce88b095a200e62c10c043b3e9bc6cb9b6ac4dfa51794b02ace9f98779040755", + "bb1eb5d4afa793c1ebdd9fb08def6c36d10096986ae0cfe148cd101170ce37aea0" + "5a63d74a840aecd514f654f080e51ac50fd617d22610d91780fe6b07a26b0847ab" + "b38291058c97474ef6ddd190d30fc318185c09ca1589d2024f0a6f16d45f116783" + "77483fa5c005b2a107cb9943e5da634e7046855eaa888663de55d6471371d55d", + }, + { + 64, + "4eed7141ea4a5cd4b788606bd23f46e212af9cacebacdc7d1f4c6dc7f2511b98fc" + "9cc56cb831ffe33ea8e7e1d1df09b26efd2767670066aa82d023b1dfe8ab1b2b7f" + "bb5b97592d46ffe3e05a6a9b592e2949c74160e4674301bc3f97e04903f8c6cf95" + "b863174c33228924cdef7ae47559b10b294acd660666c4538833582b43f82d74", + "ba8ced36f327700d213f120b1a207a3b8c04330528586f414d09f2f7d9ccb7e682" + "44c26010afc3f762615bbac552a1ca909e67c83e2fd5478cf46b9e811efccc93f7" + "7a21b17a152ebaca1695733fdb086e23cd0eb48c41c034d52523fc21236e5d8c92" + "55306e48d52ba40b4dac24256460d56573d1312319afcf3ed39d72d0bfc69acb", + }, + { + 65, + "de1e5fa0be70df6d2be8fffd0e99ceaa8eb6e8c93a63f2d8d1c30ecb6b263dee0e" + "16e0a4749d6811dd1d6d1265c29729b1b75a9ac346cf93f0e1d7296dfcfd4313b3" + "a227faaaaf7757cc95b4e87a49be3b8a270a12020233509b1c3632b3485eef309d" + "0abc4a4a696c9decc6e90454b53b000f456a3f10079072baaf7a981653221f2c", + "c0a4edefa2d2accb9277c371ac12fcdbb52988a86edc54f0716e1591b4326e72d5" + "e795f46a596b02d3d4bfb43abad1e5d19211152722ec1f20fef2cd413e3c22f2fc" + "5da3d73041275be6ede3517b3b9f0fc67ade5956a672b8b75d96cb43294b904149" + "7de92637ed3f2439225e683910cb3ae923374449ca788fb0f9bea92731bc26ad", + }, + { + 127, + "d81293fda863f008c09e92fc382a81f5a0b4a1251cba1634016a0f86a6bd640de3" + "137d477156d1fde56b0cf36f8ef18b44b2d79897bece12227539ac9ae0a5119da4" + "7644d934d26e74dc316145dcb8bb69ac3f2e05c242dd6ee06484fcb0e956dc4435" + "5b452c5e2bbb5e2b66e99f5dd443d0cbcaaafd4beebaed24ae2f8bb672bcef78", + "c64200ae7dfaf35577ac5a9521c47863fb71514a3bcad18819218b818de85818ee" + "7a317aaccc1458f78d6f65f3427ec97d9c0adb0d6dacd4471374b621b7b5f35cd5" + "4663c64dbe0b9e2d95632f84c611313ea5bd90b71ce97b3cf645776f3adc11e27d" + "135cbadb9875c2bf8d3ae6b02f8a0206aba0c35bfe42574011931c9a255ce6dc", + }, + { + 128, + "f17e570564b26578c33bb7f44643f539624b05df1a76c81f30acd548c44b45efa6" + "9faba091427f9c5c4caa873aa07828651f19c55bad85c47d1368b11c6fd99e47ec" + "ba5820a0325984d74fe3e4058494ca12e3f1d3293d0010a9722f7dee64f71246f7" + "5e9361f44cc8e214a100650db1313ff76a9f93ec6e84edb7add1cb4a95019b0c", + "b04fe15577457267ff3b6f3c947d93be581e7e3a4b018679125eaf86f6a628ecd8" + "6bbe0001f10bda47e6077b735016fca8119da11348d93ca302bbd125bde0db2b50" + "edbe728a620bb9d3e6f706286aedea973425c0b9eedf8a38873544cf91badf49ad" + "92a635a93f71ddfcee1eae536c25d1b270956be16588ef1cfef2f1d15f650bd5", + }, + { + 129, + "683aaae9f3c5ba37eaaf072aed0f9e30bac0865137bae68b1fde4ca2aebdcb12f9" + "6ffa7b36dd78ba321be7e842d364a62a42e3746681c8bace18a4a8a79649285c71" + "27bf8febf125be9de39586d251f0d41da20980b70d35e3dac0eee59e468a894fa7" + "e6a07129aaad09855f6ad4801512a116ba2b7841e6cfc99ad77594a8f2d181a7", + "d4a64dae6cdccbac1e5287f54f17c5f985105457c1a2ec1878ebd4b57e20d38f1c" + "9db018541eec241b748f87725665b7b1ace3e0065b29c3bcb232c90e37897fa5aa" + "ee7e1e8a2ecfcd9b51463e42238cfdd7fee1aecb3267fa7f2128079176132a412c" + "d8aaf0791276f6b98ff67359bd8652ef3a203976d5ff1cd41885573487bcd683", + }, + { + 1023, + "10108970eeda3eb932baac1428c7a2163b0e924c9a9e25b35bba72b28f70bd11a1" + "82d27a591b05592b15607500e1e8dd56bc6c7fc063715b7a1d737df5bad3339c56" + "778957d870eb9717b57ea3d9fb68d1b55127bba6a906a4a24bbd5acb2d123a37b2" + "8f9e9a81bbaae360d58f85e5fc9d75f7c370a0cc09b6522d9c8d822f2f28f485", + "c951ecdf03288d0fcc96ee3413563d8a6d3589547f2c2fb36d9786470f1b9d6e89" + "0316d2e6d8b8c25b0a5b2180f94fb1a158ef508c3cde45e2966bd796a696d3e13e" + "fd86259d756387d9becf5c8bf1ce2192b87025152907b6d8cc33d17826d8b7b9bc" + "97e38c3c85108ef09f013e01c229c20a83d9e8efac5b37470da28575fd755a10", + }, + { + 1024, + "42214739f095a406f3fc83deb889744ac00df831c10daa55189b5d121c855af71c" + "f8107265ecdaf8505b95d8fcec83a98a6a96ea5109d2c179c47a387ffbb404756f" + "6eeae7883b446b70ebb144527c2075ab8ab204c0086bb22b7c93d465efc57f8d91" + "7f0b385c6df265e77003b85102967486ed57db5c5ca170ba441427ed9afa684e", + "75c46f6f3d9eb4f55ecaaee480db732e6c2105546f1e675003687c31719c7ba4a7" + "8bc838c72852d4f49c864acb7adafe2478e824afe51c8919d06168414c265f298a" + "8094b1ad813a9b8614acabac321f24ce61c5a5346eb519520d38ecc43e89b50002" + "36df0597243e4d2493fd626730e2ba17ac4d8824d09d1a4a8f57b8227778e2de", + }, + { + 1025, + "d00278ae47eb27b34faecf67b4fe263f82d5412916c1ffd97c8cb7fb814b8444f4" + "c4a22b4b399155358a994e52bf255de60035742ec71bd08ac275a1b51cc6bfe332" + "b0ef84b409108cda080e6269ed4b3e2c3f7d722aa4cdc98d16deb554e5627be8f9" + "55c98e1d5f9565a9194cad0c4285f93700062d9595adb992ae68ff12800ab67a", + "357dc55de0c7e382c900fd6e320acc04146be01db6a8ce7210b7189bd664ea6936" + "2396b77fdc0d2634a552970843722066c3c15902ae5097e00ff53f1e116f1cd535" + "2720113a837ab2452cafbde4d54085d9cf5d21ca613071551b25d52e69d6c81123" + "872b6f19cd3bc1333edf0c52b94de23ba772cf82636cff4542540a7738d5b930", + }, + { + 2048, + "e776b6028c7cd22a4d0ba182a8bf62205d2ef576467e838ed6f2529b85fba24a9a" + "60bf80001410ec9eea6698cd537939fad4749edd484cb541aced55cd9bf54764d0" + "63f23f6f1e32e12958ba5cfeb1bf618ad094266d4fc3c968c2088f677454c288c6" + "7ba0dba337b9d91c7e1ba586dc9a5bc2d5e90c14f53a8863ac75655461cea8f9", + "879cf1fa2ea0e79126cb1063617a05b6ad9d0b696d0d757cf053439f60a99dd101" + "73b961cd574288194b23ece278c330fbb8585485e74967f31352a8183aa782b2b2" + "2f26cdcadb61eed1a5bc144b8198fbb0c13abbf8e3192c145d0a5c21633b0ef860" + "54f42809df823389ee40811a5910dcbd1018af31c3b43aa55201ed4edaac74fe", + }, + { + 2049, + "5f4d72f40d7a5f82b15ca2b2e44b1de3c2ef86c426c95c1af0b687952256303096" + "de31d71d74103403822a2e0bc1eb193e7aecc9643a76b7bbc0c9f9c52e8783aae9" + "8764ca468962b5c2ec92f0c74eb5448d519713e09413719431c802f948dd5d9042" + "5a4ecdadece9eb178d80f26efccae630734dff63340285adec2aed3b51073ad3", + "9f29700902f7c86e514ddc4df1e3049f258b2472b6dd5267f61bf13983b78dd5f9" + "a88abfefdfa1e00b418971f2b39c64ca621e8eb37fceac57fd0c8fc8e117d43b81" + "447be22d5d8186f8f5919ba6bcc6846bd7d50726c06d245672c2ad4f61702c6464" + "99ee1173daa061ffe15bf45a631e2946d616a4c345822f1151284712f76b2b0e", + }, + { + 3072, + "b98cb0ff3623be03326b373de6b9095218513e64f1ee2edd2525c7ad1e5cffd29a" + "3f6b0b978d6608335c09dc94ccf682f9951cdfc501bfe47b9c9189a6fc7b404d12" + "0258506341a6d802857322fbd20d3e5dae05b95c88793fa83db1cb08e7d8008d15" + "99b6209d78336e24839724c191b2a52a80448306e0daa84a3fdb566661a37e11", + "044a0e7b172a312dc02a4c9a818c036ffa2776368d7f528268d2e6b5df19177022" + "f302d0529e4174cc507c463671217975e81dab02b8fdeb0d7ccc7568dd22574c78" + "3a76be215441b32e91b9a904be8ea81f7a0afd14bad8ee7c8efc305ace5d3dd61b" + "996febe8da4f56ca0919359a7533216e2999fc87ff7d8f176fbecb3d6f34278b", + }, + { + 3073, + "7124b49501012f81cc7f11ca069ec9226cecb8a2c850cfe644e327d22d3e1cd39a" + "27ae3b79d68d89da9bf25bc27139ae65a324918a5f9b7828181e52cf373c84f35b" + "639b7fccbb985b6f2fa56aea0c18f531203497b8bbd3a07ceb5926f1cab74d14bd" + "66486d9a91eba99059a98bd1cd25876b2af5a76c3e9eed554ed72ea952b603bf", + "68dede9bef00ba89e43f31a6825f4cf433389fedae75c04ee9f0cf16a427c95a96" + "d6da3fe985054d3478865be9a092250839a697bbda74e279e8a9e69f0025e4cfdd" + "d6cfb434b1cd9543aaf97c635d1b451a4386041e4bb100f5e45407cbbc24fa53ea" + "2de3536ccb329e4eb9466ec37093a42cf62b82903c696a93a50b702c80f3c3c5", + }, + { + 4096, + "015094013f57a5277b59d8475c0501042c0b642e531b0a1c8f58d2163229e96902" + "89e9409ddb1b99768eafe1623da896faf7e1114bebeadc1be30829b6f8af707d85" + "c298f4f0ff4d9438aef948335612ae921e76d411c3a9111df62d27eaf871959ae0" + "062b5492a0feb98ef3ed4af277f5395172dbe5c311918ea0074ce0036454f620", + "befc660aea2f1718884cd8deb9902811d332f4fc4a38cf7c7300d597a081bfc0bb" + "b64a36edb564e01e4b4aaf3b060092a6b838bea44afebd2deb8298fa562b7b597c" + "757b9df4c911c3ca462e2ac89e9a787357aaf74c3b56d5c07bc93ce899568a3eb1" + "7d9250c20f6c5f6c1e792ec9a2dcb715398d5a6ec6d5c54f586a00403a1af1de", + }, + { + 4097, + "9b4052b38f1c5fc8b1f9ff7ac7b27cd242487b3d890d15c96a1c25b8aa0fb99505" + "f91b0b5600a11251652eacfa9497b31cd3c409ce2e45cfe6c0a016967316c426bd" + "26f619eab5d70af9a418b845c608840390f361630bd497b1ab44019316357c61db" + "e091ce72fc16dc340ac3d6e009e050b3adac4b5b2c92e722cffdc46501531956", + "00df940cd36bb9fa7cbbc3556744e0dbc8191401afe70520ba292ee3ca80abbc60" + "6db4976cfdd266ae0abf667d9481831ff12e0caa268e7d3e57260c0824115a54ce" + "595ccc897786d9dcbf495599cfd90157186a46ec800a6763f1c59e36197e9939e9" + "00809f7077c102f888caaf864b253bc41eea812656d46742e4ea42769f89b83f", + }, + { + 5120, + "9cadc15fed8b5d854562b26a9536d9707cadeda9b143978f319ab34230535833ac" + "c61c8fdc114a2010ce8038c853e121e1544985133fccdd0a2d507e8e615e611e9a" + "0ba4f47915f49e53d721816a9198e8b30f12d20ec3689989175f1bf7a300eee0d9" + "321fad8da232ece6efb8e9fd81b42ad161f6b9550a069e66b11b40487a5f5059", + "2c493e48e9b9bf31e0553a22b23503c0a3388f035cece68eb438d22fa1943e209b" + "4dc9209cd80ce7c1f7c9a744658e7e288465717ae6e56d5463d4f80cdb2ef56495" + "f6a4f5487f69749af0c34c2cdfa857f3056bf8d807336a14d7b89bf62bef2fb54f" + "9af6a546f818dc1e98b9e07f8a5834da50fa28fb5874af91bf06020d1bf0120e", + }, + { + 5121, + "628bd2cb2004694adaab7bbd778a25df25c47b9d4155a55f8fbd79f2fe154cff96" + "adaab0613a6146cdaabe498c3a94e529d3fc1da2bd08edf54ed64d40dcd6777647" + "eac51d8277d70219a9694334a68bc8f0f23e20b0ff70ada6f844542dfa32cd4204" + "ca1846ef76d811cdb296f65e260227f477aa7aa008bac878f72257484f2b6c95", + "6ccf1c34753e7a044db80798ecd0782a8f76f33563accaddbfbb2e0ea4b2d0240d" + "07e63f13667a8d1490e5e04f13eb617aea16a8c8a5aaed1ef6fbde1b0515e3c810" + "50b361af6ead126032998290b563e3caddeaebfab592e155f2e161fb7cba939092" + "133f23f9e65245e58ec23457b78a2e8a125588aad6e07d7f11a85b88d375b72d", + }, + { + 6144, + "3e2e5b74e048f3add6d21faab3f83aa44d3b2278afb83b80b3c35164ebeca2054d" + "742022da6fdda444ebc384b04a54c3ac5839b49da7d39f6d8a9db03deab32aade1" + "56c1c0311e9b3435cde0ddba0dce7b26a376cad121294b689193508dd63151603c" + "6ddb866ad16c2ee41585d1633a2cea093bea714f4c5d6b903522045b20395c83", + "3d6b6d21281d0ade5b2b016ae4034c5dec10ca7e475f90f76eac7138e9bc8f1dc3" + "5754060091dc5caf3efabe0603c60f45e415bb3407db67e6beb3d11cf8e4f79075" + "61f05dace0c15807f4b5f389c841eb114d81a82c02a00b57206b1d11fa6e803486" + "b048a5ce87105a686dee041207e095323dfe172df73deb8c9532066d88f9da7e", + }, + { + 6145, + "f1323a8631446cc50536a9f705ee5cb619424d46887f3c376c695b70e0f0507f18" + "a2cfdd73c6e39dd75ce7c1c6e3ef238fd54465f053b25d21044ccb2093beb01501" + "5532b108313b5829c3621ce324b8e14229091b7c93f32db2e4e63126a377d2a63a" + "3597997d4f1cba59309cb4af240ba70cebff9a23d5e3ff0cdae2cfd54e070022", + "9ac301e9e39e45e3250a7e3b3df701aa0fb6889fbd80eeecf28dbc6300fbc539f3" + "c184ca2f59780e27a576c1d1fb9772e99fd17881d02ac7dfd39675aca918453283" + "ed8c3169085ef4a466b91c1649cc341dfdee60e32231fc34c9c4e0b9a2ba87ca8f" + "372589c744c15fd6f985eec15e98136f25beeb4b13c4e43dc84abcc79cd4646c", + }, + { + 7168, + "61da957ec2499a95d6b8023e2b0e604ec7f6b50e80a9678b89d2628e99ada77a57" + "07c321c83361793b9af62a40f43b523df1c8633cecb4cd14d00bdc79c78fca5165" + "b863893f6d38b02ff7236c5a9a8ad2dba87d24c547cab046c29fc5bc1ed142e1de" + "4763613bb162a5a538e6ef05ed05199d751f9eb58d332791b8d73fb74e4fce95", + "b42835e40e9d4a7f42ad8cc04f85a963a76e18198377ed84adddeaecacc6f3fca2" + "f01d5277d69bb681c70fa8d36094f73ec06e452c80d2ff2257ed82e7ba34840098" + "9a65ee8daa7094ae0933e3d2210ac6395c4af24f91c2b590ef87d7788d7066ea3e" + "aebca4c08a4f14b9a27644f99084c3543711b64a070b94f2c9d1d8a90d035d52", + }, + { + 7169, + "a003fc7a51754a9b3c7fae0367ab3d782dccf28855a03d435f8cfe74605e781798" + "a8b20534be1ca9eb2ae2df3fae2ea60e48c6fb0b850b1385b5de0fe460dbe9d9f9" + "b0d8db4435da75c601156df9d047f4ede008732eb17adc05d96180f8a735485228" + "40779e6062d643b79478a6e8dbce68927f36ebf676ffa7d72d5f68f050b119c8", + "ed9b1a922c046fdb3d423ae34e143b05ca1bf28b710432857bf738bcedbfa5113c" + "9e28d72fcbfc020814ce3f5d4fc867f01c8f5b6caf305b3ea8a8ba2da3ab69fabc" + "b438f19ff11f5378ad4484d75c478de425fb8e6ee809b54eec9bdb184315dc8566" + "17c09f5340451bf42fd3270a7b0b6566169f242e533777604c118a6358250f54", + }, + { + 8192, + "aae792484c8efe4f19e2ca7d371d8c467ffb10748d8a5a1ae579948f718a2a635f" + "e51a27db045a567c1ad51be5aa34c01c6651c4d9b5b5ac5d0fd58cf18dd61a4777" + "8566b797a8c67df7b1d60b97b19288d2d877bb2df417ace009dcb0241ca1257d62" + "712b6a4043b4ff33f690d849da91ea3bf711ed583cb7b7a7da2839ba71309bbf", + "dc9637c8845a770b4cbf76b8daec0eebf7dc2eac11498517f08d44c8fc00d58a48" + "34464159dcbc12a0ba0c6d6eb41bac0ed6585cabfe0aca36a375e6c5480c22afdc" + "40785c170f5a6b8a1107dbee282318d00d915ac9ed1143ad40765ec120042ee121" + "cd2baa36250c618adaf9e27260fda2f94dea8fb6f08c04f8f10c78292aa46102", + }, + { + 8193, + "bab6c09cb8ce8cf459261398d2e7aef35700bf488116ceb94a36d0f5f1b7bc3bb2" + "282aa69be089359ea1154b9a9286c4a56af4de975a9aa4a5c497654914d279bea6" + "0bb6d2cf7225a2fa0ff5ef56bbe4b149f3ed15860f78b4e2ad04e158e375c1e0c0" + "b551cd7dfc82f1b155c11b6b3ed51ec9edb30d133653bb5709d1dbd55f4e1ff6", + "954a2a75420c8d6547e3ba5b98d963e6fa6491addc8c023189cc519821b4a1f5f0" + "3228648fd983aef045c2fa8290934b0866b615f585149587dda229903996532883" + "5a2b18f1d63b7e300fc76ff260b571839fe44876a4eae66cbac8c67694411ed7e0" + "9df51068a22c6e67d6d3dd2cca8ff12e3275384006c80f4db68023f24eebba57", + }, + { + 16384, + "f875d6646de28985646f34ee13be9a576fd515f76b5b0a26bb324735041ddde49d" + "764c270176e53e97bdffa58d549073f2c660be0e81293767ed4e4929f9ad34bbb3" + "9a529334c57c4a381ffd2a6d4bfdbf1482651b172aa883cc13408fa67758a3e475" + "03f93f87720a3177325f7823251b85275f64636a8f1d599c2e49722f42e93893", + "9e9fc4eb7cf081ea7c47d1807790ed211bfec56aa25bb7037784c13c4b707b0df9" + "e601b101e4cf63a404dfe50f2e1865bb12edc8fca166579ce0c70dba5a5c0fc960" + "ad6f3772183416a00bd29d4c6e651ea7620bb100c9449858bf14e1ddc9ecd35725" + "581ca5b9160de04060045993d972571c3e8f71e9d0496bfa744656861b169d65", + }, + { + 31744, + "62b6960e1a44bcc1eb1a611a8d6235b6b4b78f32e7abc4fb4c6cdcce94895c4786" + "0cc51f2b0c28a7b77304bd55fe73af663c02d3f52ea053ba43431ca5bab7bfea2f" + "5e9d7121770d88f70ae9649ea713087d1914f7f312147e247f87eb2d4ffef0ac97" + "8bf7b6579d57d533355aa20b8b77b13fd09748728a5cc327a8ec470f4013226f", + "efa53b389ab67c593dba624d898d0f7353ab99e4ac9d42302ee64cbf9939a4193a" + "7258db2d9cd32a7a3ecfce46144114b15c2fcb68a618a976bd74515d47be08b628" + "be420b5e830fade7c080e351a076fbc38641ad80c736c8a18fe3c66ce12f95c61c" + "2462a9770d60d0f77115bbcd3782b593016a4e728d4c06cee4505cb0c08a42ec", + }, + { + 102400, + "bc3e3d41a1146b069abffad3c0d44860cf664390afce4d9661f7902e7943e085e0" + "1c59dab908c04c3342b816941a26d69c2605ebee5ec5291cc55e15b76146e6745f" + "0601156c3596cb75065a9c57f35585a52e1ac70f69131c23d611ce11ee4ab1ec2c" + "009012d236648e77be9295dd0426f29b764d65de58eb7d01dd42248204f45f8e", + "1c35d1a5811083fd7119f5d5d1ba027b4d01c0c6c49fb6ff2cf75393ea5db4a7f9" + "dbdd3e1d81dcbca3ba241bb18760f207710b751846faaeb9dff8262710999a59b2" + "aa1aca298a032d94eacfadf1aa192418eb54808db23b56e34213266aa08499a16b" + "354f018fc4967d05f8b9d2ad87a7278337be9693fc638a3bfdbe314574ee6fc4", + }, + { + 0, 0, 0 + } +}; + +#ifdef BLAKE3_DEBUG +#define dprintf printf +#else +#define dprintf(...) +#endif + +static char fmt_tohex(char c); +static size_t fmt_hexdump(char *dest, const char *src, size_t len); + +static char fmt_tohex(char c) { + return ((char)(c >= 10 ? c-10+'a' : c+'0')); +} + +static size_t fmt_hexdump(char *dest, const char *src, size_t len) { + register const unsigned char *s = (const unsigned char *) src; + size_t written = 0, i; + + if (!dest) + return ((len > ((size_t)-1)/2) ? (size_t)-1 : len*2); + for (i = 0; i < len; ++i) { + dest[written] = fmt_tohex(s[i]>>4); + dest[written+1] = fmt_tohex(s[i]&15); + written += 2; + } + + return (written); +} + +int +main(int argc, char *argv[]) +{ + boolean_t failed = B_FALSE; + uint8_t buffer[102400]; + uint64_t cpu_mhz = 0; + int id, i, j; + + if (argc == 2) + cpu_mhz = atoi(argv[1]); + + /* fill test message */ + for (i = 0, j = 0; i < sizeof (buffer); i++, j++) { + if (j == 251) + j = 0; + buffer[i] = (uint8_t)j; + } + + (void) printf("Running algorithm correctness tests:\n"); + for (id = 0; id < blake3_get_impl_count(); id++) { + blake3_set_impl_id(id); + const char *name = blake3_get_impl_name(); + dprintf("Result for BLAKE3-%s:\n", name); + for (i = 0; TestArray[i].hash; i++) { + blake3_test_t *cur = &TestArray[i]; + + BLAKE3_CTX ctx; + uint8_t digest[TEST_DIGEST_LEN]; + char result[TEST_DIGEST_LEN]; + + /* default hashing */ + Blake3_Init(&ctx); + Blake3_Update(&ctx, buffer, cur->input_len); + Blake3_FinalSeek(&ctx, 0, digest, TEST_DIGEST_LEN); + fmt_hexdump(result, (char *)digest, 131); + if (memcmp(result, cur->hash, 131) != 0) + failed = B_TRUE; + + dprintf("HASH-res: %s\n", result); + dprintf("HASH-ref: %s\n", cur->hash); + + /* salted hashing */ + Blake3_InitKeyed(&ctx, (const uint8_t *)salt); + Blake3_Update(&ctx, buffer, cur->input_len); + Blake3_FinalSeek(&ctx, 0, digest, TEST_DIGEST_LEN); + fmt_hexdump(result, (char *)digest, 131); + if (memcmp(result, cur->shash, 131) != 0) + failed = B_TRUE; + + dprintf("SHASH-res: %s\n", result); + dprintf("SHASH-ref: %s\n", cur->shash); + + printf("BLAKE3-%s Message (inlen=%d)\tResult: %s\n", + name, cur->input_len, failed?"FAILED!":"OK"); + } + } + + if (failed) + return (1); + +#define BLAKE3_PERF_TEST(impl, diglen) \ + do { \ + BLAKE3_CTX ctx; \ + uint8_t digest[diglen / 8]; \ + uint8_t block[131072]; \ + uint64_t delta; \ + double cpb = 0; \ + int i; \ + struct timeval start, end; \ + memset(block, 0, sizeof (block)); \ + (void) gettimeofday(&start, NULL); \ + Blake3_Init(&ctx); \ + for (i = 0; i < 8192; i++) \ + Blake3_Update(&ctx, block, sizeof (block)); \ + Blake3_Final(&ctx, digest); \ + (void) gettimeofday(&end, NULL); \ + delta = (end.tv_sec * 1000000llu + end.tv_usec) - \ + (start.tv_sec * 1000000llu + start.tv_usec); \ + if (cpu_mhz != 0) { \ + cpb = (cpu_mhz * 1e6 * ((double)delta / \ + 1000000)) / (8192 * 128 * 1024); \ + } \ + (void) printf("BLAKE3-%s %llu us (%.02f CPB)\n", impl, \ + (u_longlong_t)delta, cpb); \ + } while (0) + + printf("Running performance tests (hashing 1024 MiB of data):\n"); + for (id = 0; id < blake3_get_impl_count(); id++) { + blake3_set_impl_id(id); + const char *name = blake3_get_impl_name(); + BLAKE3_PERF_TEST(name, 256); + } + + return (0); +} diff --git a/tests/zfs-tests/tests/functional/checksum/edonr_test.c b/tests/zfs-tests/cmd/checksum/edonr_test.c similarity index 98% rename from tests/zfs-tests/tests/functional/checksum/edonr_test.c rename to tests/zfs-tests/cmd/checksum/edonr_test.c index d8585ea4cf7e..3a0a48533c53 100644 --- a/tests/zfs-tests/tests/functional/checksum/edonr_test.c +++ b/tests/zfs-tests/cmd/checksum/edonr_test.c @@ -28,9 +28,6 @@ * gettimeofday due to -D_KERNEL (we can do this since we're actually * running in userspace, but we need -D_KERNEL for the remaining Edon-R code). */ -#ifdef _KERNEL -#undef _KERNEL -#endif #include #include @@ -160,7 +157,7 @@ main(int argc, char *argv[]) EdonRFinal(&ctx, digest); \ (void) printf("Edon-R-%-6sMessage: " #_m \ "\tResult: ", #mode); \ - if (bcmp(digest, testdigest, mode / 8) == 0) { \ + if (memcmp(digest, testdigest, mode / 8) == 0) { \ (void) printf("OK\n"); \ } else { \ (void) printf("FAILED!\n"); \ @@ -177,7 +174,7 @@ main(int argc, char *argv[]) double cpb = 0; \ int i; \ struct timeval start, end; \ - bzero(block, sizeof (block)); \ + memset(block, 0, sizeof (block)); \ (void) gettimeofday(&start, NULL); \ EdonRInit(&ctx, mode); \ for (i = 0; i < 8192; i++) \ diff --git a/tests/zfs-tests/tests/functional/checksum/sha2_test.c b/tests/zfs-tests/cmd/checksum/sha2_test.c similarity index 98% rename from tests/zfs-tests/tests/functional/checksum/sha2_test.c rename to tests/zfs-tests/cmd/checksum/sha2_test.c index c7561b54f29e..bb355311091e 100644 --- a/tests/zfs-tests/tests/functional/checksum/sha2_test.c +++ b/tests/zfs-tests/cmd/checksum/sha2_test.c @@ -28,9 +28,6 @@ * gettimeofday due to -D_KERNEL (we can do this since we're actually * running in userspace, but we need -D_KERNEL for the remaining SHA2 code). */ -#ifdef _KERNEL -#undef _KERNEL -#endif #include #include @@ -189,7 +186,7 @@ main(int argc, char *argv[]) SHA2Final(digest, &ctx); \ (void) printf("SHA%-9sMessage: " #_m \ "\tResult: ", #mode); \ - if (bcmp(digest, testdigest, diglen / 8) == 0) { \ + if (memcmp(digest, testdigest, diglen / 8) == 0) { \ (void) printf("OK\n"); \ } else { \ (void) printf("FAILED!\n"); \ @@ -206,7 +203,7 @@ main(int argc, char *argv[]) double cpb = 0; \ int i; \ struct timeval start, end; \ - bzero(block, sizeof (block)); \ + memset(block, 0, sizeof (block)); \ (void) gettimeofday(&start, NULL); \ SHA2Init(SHA ## mode ## _MECH_INFO_TYPE, &ctx); \ for (i = 0; i < 8192; i++) \ diff --git a/tests/zfs-tests/tests/functional/checksum/skein_test.c b/tests/zfs-tests/cmd/checksum/skein_test.c similarity index 98% rename from tests/zfs-tests/tests/functional/checksum/skein_test.c rename to tests/zfs-tests/cmd/checksum/skein_test.c index 484fad844b73..13611c860c42 100644 --- a/tests/zfs-tests/tests/functional/checksum/skein_test.c +++ b/tests/zfs-tests/cmd/checksum/skein_test.c @@ -28,13 +28,10 @@ * gettimeofday due to -D_KERNEL (we can do this since we're actually * running in userspace, but we need -D_KERNEL for the remaining Skein code). */ -#ifdef _KERNEL -#undef _KERNEL -#endif #include #include -#include +#include #include #include #include @@ -278,7 +275,7 @@ main(int argc, char *argv[]) (void) Skein ## mode ## _Final(&ctx, digest); \ (void) printf("Skein" #mode "/" #diglen \ "\tMessage: " #_m "\tResult: "); \ - if (bcmp(digest, testdigest, diglen / 8) == 0) { \ + if (memcmp(digest, testdigest, diglen / 8) == 0) { \ (void) printf("OK\n"); \ } else { \ (void) printf("FAILED!\n"); \ @@ -295,7 +292,7 @@ main(int argc, char *argv[]) double cpb = 0; \ int i; \ struct timeval start, end; \ - bzero(block, sizeof (block)); \ + memset(block, 0, sizeof (block)); \ (void) gettimeofday(&start, NULL); \ (void) Skein ## mode ## _Init(&ctx, diglen); \ for (i = 0; i < 8192; i++) { \ diff --git a/tests/zfs-tests/cmd/chg_usr_exec/chg_usr_exec.c b/tests/zfs-tests/cmd/chg_usr_exec.c similarity index 97% rename from tests/zfs-tests/cmd/chg_usr_exec/chg_usr_exec.c rename to tests/zfs-tests/cmd/chg_usr_exec.c index 1fa9e88a6fde..911c907b38ac 100644 --- a/tests/zfs-tests/cmd/chg_usr_exec/chg_usr_exec.c +++ b/tests/zfs-tests/cmd/chg_usr_exec.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/tests/zfs-tests/cmd/chg_usr_exec/.gitignore b/tests/zfs-tests/cmd/chg_usr_exec/.gitignore deleted file mode 100644 index a8b44df7c5bf..000000000000 --- a/tests/zfs-tests/cmd/chg_usr_exec/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/chg_usr_exec diff --git a/tests/zfs-tests/cmd/chg_usr_exec/Makefile.am b/tests/zfs-tests/cmd/chg_usr_exec/Makefile.am deleted file mode 100644 index 6f2968f1fac4..000000000000 --- a/tests/zfs-tests/cmd/chg_usr_exec/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = chg_usr_exec -chg_usr_exec_SOURCES = chg_usr_exec.c diff --git a/tests/zfs-tests/tests/functional/cp_files/cp_files.c b/tests/zfs-tests/cmd/cp_files.c similarity index 100% rename from tests/zfs-tests/tests/functional/cp_files/cp_files.c rename to tests/zfs-tests/cmd/cp_files.c diff --git a/tests/zfs-tests/tests/functional/ctime/ctime.c b/tests/zfs-tests/cmd/ctime.c similarity index 98% rename from tests/zfs-tests/tests/functional/ctime/ctime.c rename to tests/zfs-tests/cmd/ctime.c index 41dcf6abc9c2..f0f3d526eb3e 100644 --- a/tests/zfs-tests/tests/functional/ctime/ctime.c +++ b/tests/zfs-tests/cmd/ctime.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -38,7 +38,6 @@ #include #include #include -#include #include #include #include @@ -52,7 +51,7 @@ typedef struct timetest { int type; - char *name; + const char *name; int (*func)(const char *pfile); } timetest_t; @@ -261,7 +260,7 @@ static int do_xattr(const char *pfile) { int ret = 0; - char *value = "user.value"; + const char *value = "user.value"; if (pfile == NULL) { return (-1); @@ -307,7 +306,7 @@ int main(void) { int i, ret, fd; - char *penv[] = {"TESTDIR", "TESTFILE0"}; + const char *penv[] = {"TESTDIR", "TESTFILE0"}; (void) atexit(cleanup); diff --git a/tests/zfs-tests/cmd/devname2devid/devname2devid.c b/tests/zfs-tests/cmd/devname2devid.c similarity index 98% rename from tests/zfs-tests/cmd/devname2devid/devname2devid.c rename to tests/zfs-tests/cmd/devname2devid.c index 91e59c589fd5..0f3655be7143 100644 --- a/tests/zfs-tests/cmd/devname2devid/devname2devid.c +++ b/tests/zfs-tests/cmd/devname2devid.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/tests/zfs-tests/cmd/devname2devid/.gitignore b/tests/zfs-tests/cmd/devname2devid/.gitignore deleted file mode 100644 index fa9fb6c509a7..000000000000 --- a/tests/zfs-tests/cmd/devname2devid/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/devname2devid diff --git a/tests/zfs-tests/cmd/devname2devid/Makefile.am b/tests/zfs-tests/cmd/devname2devid/Makefile.am deleted file mode 100644 index b8b630dc2dd4..000000000000 --- a/tests/zfs-tests/cmd/devname2devid/Makefile.am +++ /dev/null @@ -1,10 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -if WANT_DEVNAME2DEVID -pkgexec_PROGRAMS = devname2devid -devname2devid_SOURCES = devname2devid.c -devname2devid_CFLAGS = $(AM_CFLAGS) $(LIBUDEV_CFLAGS) -devname2devid_LDADD = $(LIBUDEV_LIBS) -endif diff --git a/tests/zfs-tests/cmd/dir_rd_update/dir_rd_update.c b/tests/zfs-tests/cmd/dir_rd_update.c similarity index 97% rename from tests/zfs-tests/cmd/dir_rd_update/dir_rd_update.c rename to tests/zfs-tests/cmd/dir_rd_update.c index ea46f047a35e..80c395cc62d8 100644 --- a/tests/zfs-tests/cmd/dir_rd_update/dir_rd_update.c +++ b/tests/zfs-tests/cmd/dir_rd_update.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -47,7 +47,7 @@ static char dirpath[256]; int main(int argc, char **argv) { - char *cp1 = ""; + const char *cp1 = ""; int i = 0; int ret = 0; int testdd = 0; diff --git a/tests/zfs-tests/cmd/dir_rd_update/.gitignore b/tests/zfs-tests/cmd/dir_rd_update/.gitignore deleted file mode 100644 index ec9a15f17a1d..000000000000 --- a/tests/zfs-tests/cmd/dir_rd_update/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/dir_rd_update diff --git a/tests/zfs-tests/cmd/dir_rd_update/Makefile.am b/tests/zfs-tests/cmd/dir_rd_update/Makefile.am deleted file mode 100644 index 27cc9e97e0e9..000000000000 --- a/tests/zfs-tests/cmd/dir_rd_update/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = dir_rd_update -dir_rd_update_SOURCES = dir_rd_update.c diff --git a/tests/zfs-tests/tests/functional/acl/off/dosmode_readonly_write.c b/tests/zfs-tests/cmd/dosmode_readonly_write.c similarity index 90% rename from tests/zfs-tests/tests/functional/acl/off/dosmode_readonly_write.c rename to tests/zfs-tests/cmd/dosmode_readonly_write.c index 372c3f7f64de..0441d1c7b472 100644 --- a/tests/zfs-tests/tests/functional/acl/off/dosmode_readonly_write.c +++ b/tests/zfs-tests/cmd/dosmode_readonly_write.c @@ -36,6 +36,11 @@ #include #include +#ifdef __linux__ +#include +#include +#endif + int main(int argc, const char *argv[]) { @@ -51,8 +56,14 @@ main(int argc, const char *argv[]) fd = open(path, O_CREAT|O_RDWR, 0777); if (fd == -1) err(EXIT_FAILURE, "%s: open failed", path); +#ifdef __linux__ + uint64_t dosflags = ZFS_READONLY; + if (ioctl(fd, ZFS_IOC_SETDOSFLAGS, &dosflags) == -1) + err(EXIT_FAILURE, "%s: ZFS_IOC_SETDOSFLAGS failed", path); +#else if (chflags(path, UF_READONLY) == -1) err(EXIT_FAILURE, "%s: chflags failed", path); +#endif if (write(fd, buf, strlen(buf)) == -1) err(EXIT_FAILURE, "%s: write failed", path); if (close(fd) == -1) diff --git a/tests/zfs-tests/cmd/draid/draid.c b/tests/zfs-tests/cmd/draid.c similarity index 97% rename from tests/zfs-tests/cmd/draid/draid.c rename to tests/zfs-tests/cmd/draid.c index e12ab84fce59..b995be46a373 100644 --- a/tests/zfs-tests/cmd/draid/draid.c +++ b/tests/zfs-tests/cmd/draid.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -130,7 +130,7 @@ read_map(const char *filename, nvlist_t **allcfgs) * for freeing the configuration returned. */ static int -read_map_key(const char *filename, char *key, nvlist_t **cfg) +read_map_key(const char *filename, const char *key, nvlist_t **cfg) { nvlist_t *allcfgs, *foundcfg = NULL; int error; @@ -320,8 +320,8 @@ write_map_key(const char *filename, char *key, draid_map_t *map, } static void -dump_map(draid_map_t *map, char *key, double worst_ratio, double avg_ratio, - int verbose) +dump_map(draid_map_t *map, const char *key, double worst_ratio, + double avg_ratio, int verbose) { if (verbose == 0) { return; @@ -363,7 +363,7 @@ dump_map(draid_map_t *map, char *key, double worst_ratio, double avg_ratio, } static void -dump_map_nv(char *key, nvlist_t *cfg, int verbose) +dump_map_nv(const char *key, nvlist_t *cfg, int verbose) { draid_map_t map; uint_t c; @@ -385,7 +385,7 @@ dump_map_nv(char *key, nvlist_t *cfg, int verbose) * Print a summary of the mapping. */ static int -dump_map_key(const char *filename, char *key, int verbose) +dump_map_key(const char *filename, const char *key, int verbose) { nvlist_t *cfg; int error; @@ -766,7 +766,7 @@ eval_maps(uint64_t children, int passes, uint64_t *map_seed, static int draid_generate(int argc, char *argv[]) { - char filename[MAXPATHLEN]; + char filename[MAXPATHLEN] = {0}; uint64_t map_seed; int c, fd, error, verbose = 0, passes = 1, continuous = 0; int min_children = VDEV_DRAID_MIN_CHILDREN; @@ -824,10 +824,9 @@ draid_generate(int argc, char *argv[]) } } - if (argc > optind) { - bzero(filename, MAXPATHLEN); + if (argc > optind) strncpy(filename, argv[optind], MAXPATHLEN - 1); - } else { + else { (void) fprintf(stderr, "A FILE must be specified.\n"); return (1); } @@ -926,7 +925,7 @@ draid_generate(int argc, char *argv[]) static int draid_verify(int argc, char *argv[]) { - char filename[MAXPATHLEN]; + char filename[MAXPATHLEN] = {0}; int n = 0, c, error, verbose = 1; int check_ratios = 0; @@ -956,7 +955,6 @@ draid_verify(int argc, char *argv[]) if (abspath == NULL) return (ENOMEM); - bzero(filename, MAXPATHLEN); if (realpath(argv[optind], abspath) != NULL) strncpy(filename, abspath, MAXPATHLEN - 1); else @@ -980,9 +978,8 @@ draid_verify(int argc, char *argv[]) children <= VDEV_DRAID_MAX_CHILDREN; children++) { draid_map_t *map; - char key[8]; + char key[8] = {0}; - bzero(key, 8); snprintf(key, 8, "%llu", (u_longlong_t)children); error = alloc_fixed_map(children, &map); @@ -1126,7 +1123,7 @@ draid_verify(int argc, char *argv[]) static int draid_dump(int argc, char *argv[]) { - char filename[MAXPATHLEN]; + char filename[MAXPATHLEN] = {0}; int c, error, verbose = 1; int min_children = VDEV_DRAID_MIN_CHILDREN; int max_children = VDEV_DRAID_MAX_CHILDREN; @@ -1167,10 +1164,9 @@ draid_dump(int argc, char *argv[]) } } - if (argc > optind) { - bzero(filename, MAXPATHLEN); + if (argc > optind) strncpy(filename, argv[optind], MAXPATHLEN - 1); - } else { + else { (void) fprintf(stderr, "A FILE must be specified.\n"); return (1); } @@ -1202,13 +1198,12 @@ draid_dump(int argc, char *argv[]) static int draid_table(int argc, char *argv[]) { - char filename[MAXPATHLEN]; + char filename[MAXPATHLEN] = {0}; int error; - if (argc > optind) { - bzero(filename, MAXPATHLEN); + if (argc > optind) strncpy(filename, argv[optind], MAXPATHLEN - 1); - } else { + else { (void) fprintf(stderr, "A FILE must be specified.\n"); return (1); } @@ -1221,9 +1216,8 @@ draid_table(int argc, char *argv[]) children++) { uint64_t seed, checksum, nperms, avg_ratio; nvlist_t *cfg; - char key[8]; + char key[8] = {0}; - bzero(key, 8); snprintf(key, 8, "%llu", (u_longlong_t)children); error = read_map_key(filename, key, &cfg); @@ -1317,15 +1311,12 @@ draid_merge_impl(nvlist_t *allcfgs, const char *srcfilename, int *mergedp) static int draid_merge(int argc, char *argv[]) { - char filename[MAXPATHLEN]; - int c, error, total_merged = 0, verbose = 0; + char filename[MAXPATHLEN] = {0}; + int c, error, total_merged = 0; nvlist_t *allcfgs; - while ((c = getopt(argc, argv, ":v")) != -1) { + while ((c = getopt(argc, argv, ":")) != -1) { switch (c) { - case 'v': - verbose++; - break; case ':': (void) fprintf(stderr, "missing argument for '%c' option\n", optopt); @@ -1345,7 +1336,6 @@ draid_merge(int argc, char *argv[]) return (1); } - bzero(filename, MAXPATHLEN); strncpy(filename, argv[optind], MAXPATHLEN - 1); optind++; @@ -1358,10 +1348,9 @@ draid_merge(int argc, char *argv[]) } while (optind < argc) { - char srcfilename[MAXPATHLEN]; + char srcfilename[MAXPATHLEN] = {0}; int merged = 0; - bzero(srcfilename, MAXPATHLEN); strncpy(srcfilename, argv[optind], MAXPATHLEN - 1); error = draid_merge_impl(allcfgs, srcfilename, &merged); diff --git a/tests/zfs-tests/cmd/draid/.gitignore b/tests/zfs-tests/cmd/draid/.gitignore deleted file mode 100644 index 911b9f07785a..000000000000 --- a/tests/zfs-tests/cmd/draid/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/draid diff --git a/tests/zfs-tests/cmd/draid/Makefile.am b/tests/zfs-tests/cmd/draid/Makefile.am deleted file mode 100644 index 69fed7a6bea4..000000000000 --- a/tests/zfs-tests/cmd/draid/Makefile.am +++ /dev/null @@ -1,15 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -AM_CFLAGS += $(ZLIB_CFLAGS) - -pkgexec_PROGRAMS = draid - -draid_SOURCES = draid.c - -draid_LDADD = \ - $(abs_top_builddir)/lib/libzpool/libzpool.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la - -draid_LDADD += $(ZLIB_LIBS) diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_events/ereports.c b/tests/zfs-tests/cmd/ereports.c similarity index 98% rename from tests/zfs-tests/tests/functional/cli_root/zpool_events/ereports.c rename to tests/zfs-tests/cmd/ereports.c index bff3bb1ee808..392f5952d27a 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_events/ereports.c +++ b/tests/zfs-tests/cmd/ereports.c @@ -42,7 +42,7 @@ * When the class and all of these values match, then an ereport is * considered to be a duplicate. */ -static const char *criteria_name[] = { +static const char *const criteria_name[] = { FM_EREPORT_PAYLOAD_ZFS_POOL, FM_EREPORT_PAYLOAD_ZFS_VDEV_GUID, FM_EREPORT_PAYLOAD_ZFS_ZIO_ERR, diff --git a/tests/zfs-tests/cmd/file/file_append.c b/tests/zfs-tests/cmd/file/file_append.c new file mode 100644 index 000000000000..3c2e196121e1 --- /dev/null +++ b/tests/zfs-tests/cmd/file/file_append.c @@ -0,0 +1,206 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2022 by Triad National Security, LLC + */ + +#include "file_common.h" +#include +#include + +static char *filename = NULL; +static int expected_offset = -1; +static int blocksize = 131072; /* 128KiB */ +static int numblocks = 8; +static const char *execname = "file_append"; +static int use_odirect = 0; + +static void +usage(void) +{ + (void) fprintf(stderr, + "usage %s -f filename -e expected_offset [-b blocksize] \n" + " [-n numblocks] [-d use_odirect] [-h help]\n" + "\n" + "Opens a file using O_APPEND and writes numblocks blocksize\n" + "blocks to filename.\n" + "Checks if expected_offst == lseek(fd, 0, SEEK_CUR)).\n" + "\n" + " filename: File to open with O_APPEND and write to.\n" + " expected_offset: Expected file offset after writing\n" + " blocksize numblocks to filename\n" + " blocksize: Size of each block to writei (must be at\n" + " least >= 512). If using use_odirect (-d)\n" + " must be a mutltiple of _SC_PAGE_SIZE\n" + " numblocks: Total number of blocksized blocks to\n" + " write.\n" + " use_odirect: Open file using O_DIRECT.\n" + " help: Print usage information and exit.\n" + "\n" + " Required parameters:\n" + " filename\n" + " expected_offset\n" + "\n" + " Default values:\n" + " blocksize -> 131072 (128 KiB)\n" + " numblocks -> 8\n" + " use_odirect -> False\n", + execname); + (void) exit(1); +} + +static void +parse_options(int argc, char *argv[]) +{ + int c; + int errflag = 0; + extern char *optarg; + extern int optind, optopt; + + while ((c = getopt(argc, argv, "b:de:f:hn:")) != -1) { + switch (c) { + case 'b': + blocksize = atoi(optarg); + break; + case 'd': + use_odirect = 1; + break; + case 'e': + expected_offset = atoi(optarg); + break; + case 'f': + filename = optarg; + break; + case 'h': + (void) usage(); + break; + case 'n': + numblocks = atoi(optarg); + break; + case ':': + (void) fprintf(stderr, + "Option -%c requires an operand\n", + optopt); + errflag++; + break; + case '?': + default: + (void) fprintf(stderr, + "Unrecognized option: -%c\n", optopt); + errflag++; + break; + } + } + + if (errflag) + (void) usage(); + + if (use_odirect && ((blocksize % sysconf(_SC_PAGE_SIZE)) != 0)) { + (void) fprintf(stderr, + "blocksize parameter invalid when using O_DIRECT.\n"); + (void) usage(); + } + + if (blocksize < 512 || expected_offset < 0 || filename == NULL || + numblocks <= 0) { + (void) fprintf(stderr, + "Required parameters(s) missing or invalid value for " + "parameter.\n"); + (void) usage(); + } +} + +int +main(int argc, char *argv[]) +{ + int err; + const char *datapattern = "0xf00ba3"; + int fd = -1; + int fd_flags = O_WRONLY | O_CREAT | O_APPEND; + int buf_offset = 0; + char *buf; + + parse_options(argc, argv); + + if (use_odirect) + fd_flags |= O_DIRECT; + + fd = open(filename, fd_flags, 0666); + if (fd == -1) { + (void) fprintf(stderr, "%s: %s: ", execname, filename); + perror("open"); + (void) exit(2); + } + + err = posix_memalign((void **)&buf, sysconf(_SC_PAGE_SIZE), + blocksize); + + if (err != 0) { + (void) fprintf(stderr, + "%s: %s\n", execname, strerror(err)); + (void) exit(2); + } + + /* Putting known data pattern in buffer */ + int left = blocksize; + while (left) { + size_t amt = MIN(strlen(datapattern), left); + memcpy(&buf[buf_offset], datapattern, amt); + buf_offset += amt; + left -= amt; + } + + for (int i = 0; i < numblocks; i++) { + int wrote = write(fd, buf, blocksize); + + if (wrote != blocksize) { + if (wrote < 0) { + perror("write"); + } else { + (void) fprintf(stderr, + "%s: unexpected short write, wrote %d " + "byte, expected %d\n", execname, wrote, + blocksize); + } + (void) exit(2); + } + } + + /* Getting current file offset */ + off_t off = lseek(fd, 0, SEEK_CUR); + + if (off == -1) { + perror("output seek"); + (void) exit(2); + } else if (off != expected_offset) { + (void) fprintf(stderr, + "%s: expected offset %d but current offset in %s is set " + "to %ld\n", execname, expected_offset, filename, + (long int)off); + (void) exit(2); + } + + (void) close(fd); + free(buf); + + return (0); +} diff --git a/tests/zfs-tests/cmd/file_check/file_check.c b/tests/zfs-tests/cmd/file/file_check.c similarity index 96% rename from tests/zfs-tests/cmd/file_check/file_check.c rename to tests/zfs-tests/cmd/file/file_check.c index 3d3db753f3d7..1976beffaf28 100644 --- a/tests/zfs-tests/cmd/file_check/file_check.c +++ b/tests/zfs-tests/cmd/file/file_check.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -24,7 +24,7 @@ * Use is subject to license terms. */ -#include "../file_common.h" +#include "file_common.h" static unsigned char bigbuffer[BIGBUFFERSIZE]; diff --git a/tests/zfs-tests/cmd/file_common.h b/tests/zfs-tests/cmd/file/file_common.h similarity index 95% rename from tests/zfs-tests/cmd/file_common.h rename to tests/zfs-tests/cmd/file/file_common.h index 64b1777a9ae8..2aedf2905aef 100644 --- a/tests/zfs-tests/cmd/file_common.h +++ b/tests/zfs-tests/cmd/file/file_common.h @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -52,7 +52,7 @@ extern "C" { #include #include #include -#include +#include #define BLOCKSZ 8192 #define DATA 0xa5 diff --git a/tests/zfs-tests/cmd/file_trunc/file_trunc.c b/tests/zfs-tests/cmd/file/file_trunc.c similarity index 98% rename from tests/zfs-tests/cmd/file_trunc/file_trunc.c rename to tests/zfs-tests/cmd/file/file_trunc.c index 69096752efa2..2085f5955ea1 100644 --- a/tests/zfs-tests/cmd/file_trunc/file_trunc.c +++ b/tests/zfs-tests/cmd/file/file_trunc.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/tests/zfs-tests/cmd/file_write/file_write.c b/tests/zfs-tests/cmd/file/file_write.c similarity index 97% rename from tests/zfs-tests/cmd/file_write/file_write.c rename to tests/zfs-tests/cmd/file/file_write.c index 60893c34fbc9..ce1fa5d9bc02 100644 --- a/tests/zfs-tests/cmd/file_write/file_write.c +++ b/tests/zfs-tests/cmd/file/file_write.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -24,7 +24,7 @@ * Use is subject to license terms. */ -#include "../file_common.h" +#include "file_common.h" #include #include #include @@ -251,7 +251,7 @@ usage(char *prog) "\t[-s offset] [-c write_count] [-d data]\n\n" "Where [data] equal to zero causes chars " "0->%d to be repeated throughout, or [data]\n" - "equal to 'R' for psudorandom data.\n", + "equal to 'R' for pseudorandom data.\n", prog, DATA_RANGE); exit(1); diff --git a/tests/zfs-tests/cmd/largest_file/largest_file.c b/tests/zfs-tests/cmd/file/largest_file.c similarity index 97% rename from tests/zfs-tests/cmd/largest_file/largest_file.c rename to tests/zfs-tests/cmd/file/largest_file.c index 912607640965..8545bb7eea42 100644 --- a/tests/zfs-tests/cmd/largest_file/largest_file.c +++ b/tests/zfs-tests/cmd/file/largest_file.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -28,7 +28,7 @@ * Copyright (c) 2012 by Delphix. All rights reserved. */ -#include "../file_common.h" +#include "file_common.h" #include #include #include diff --git a/tests/zfs-tests/cmd/randfree_file/randfree_file.c b/tests/zfs-tests/cmd/file/randfree_file.c similarity index 97% rename from tests/zfs-tests/cmd/randfree_file/randfree_file.c rename to tests/zfs-tests/cmd/file/randfree_file.c index c708d647e8b9..1bb897881830 100644 --- a/tests/zfs-tests/cmd/randfree_file/randfree_file.c +++ b/tests/zfs-tests/cmd/file/randfree_file.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -28,7 +28,7 @@ * Copyright (c) 2012 by Delphix. All rights reserved. */ -#include "../file_common.h" +#include "file_common.h" #include #include #include diff --git a/tests/zfs-tests/cmd/randwritecomp/randwritecomp.c b/tests/zfs-tests/cmd/file/randwritecomp.c similarity index 90% rename from tests/zfs-tests/cmd/randwritecomp/randwritecomp.c rename to tests/zfs-tests/cmd/file/randwritecomp.c index 708d5ee90511..cc70d1212f84 100644 --- a/tests/zfs-tests/cmd/randwritecomp/randwritecomp.c +++ b/tests/zfs-tests/cmd/file/randwritecomp.c @@ -13,21 +13,15 @@ * Copyright (c) 2017 by Delphix. All rights reserved. */ -/* - * The following is defined so the source can use - * lrand48() and srand48(). - */ -#define __EXTENSIONS__ - #include #include -#include "../file_common.h" +#include "file_common.h" /* * The following sample was derived from real-world data * of a production Oracle database. */ -static uint64_t size_distribution[] = { +static const uint64_t size_distribution[] = { 0, 1499018, 352084, @@ -87,11 +81,11 @@ fillbuf(char *buf) break; } - bcopy(randbuf, buf, BLOCKSZ); + memcpy(buf, randbuf, BLOCKSZ); if (i == 0) - bzero(buf, BLOCKSZ - 10); + memset(buf, 0, BLOCKSZ - 10); else if (i < 16) - bzero(buf, BLOCKSZ - i * 512 + 256); + memset(buf, 0, BLOCKSZ - i * 512 + 256); /*LINTED: E_BAD_PTR_CAST_ALIGN*/ ((uint32_t *)buf)[0] = lrand48(); } @@ -99,8 +93,7 @@ fillbuf(char *buf) static void exit_usage(void) { - (void) printf("usage: "); - (void) printf("randwritecomp [-s] [nwrites]\n"); + (void) puts("usage: randwritecomp [-s] file [nwrites]"); exit(EXIT_FAILURE); } diff --git a/tests/zfs-tests/cmd/file_check/.gitignore b/tests/zfs-tests/cmd/file_check/.gitignore deleted file mode 100644 index 24fe113221d2..000000000000 --- a/tests/zfs-tests/cmd/file_check/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/file_check diff --git a/tests/zfs-tests/cmd/file_check/Makefile.am b/tests/zfs-tests/cmd/file_check/Makefile.am deleted file mode 100644 index 13027ef5bd02..000000000000 --- a/tests/zfs-tests/cmd/file_check/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = file_check -file_check_SOURCES = file_check.c diff --git a/tests/zfs-tests/cmd/file_trunc/.gitignore b/tests/zfs-tests/cmd/file_trunc/.gitignore deleted file mode 100644 index 90b149ff51c3..000000000000 --- a/tests/zfs-tests/cmd/file_trunc/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/file_trunc diff --git a/tests/zfs-tests/cmd/file_trunc/Makefile.am b/tests/zfs-tests/cmd/file_trunc/Makefile.am deleted file mode 100644 index 0455eb4a4633..000000000000 --- a/tests/zfs-tests/cmd/file_trunc/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = file_trunc -file_trunc_SOURCES = file_trunc.c diff --git a/tests/zfs-tests/cmd/file_write/.gitignore b/tests/zfs-tests/cmd/file_write/.gitignore deleted file mode 100644 index 9f691d580a57..000000000000 --- a/tests/zfs-tests/cmd/file_write/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/file_write diff --git a/tests/zfs-tests/cmd/file_write/Makefile.am b/tests/zfs-tests/cmd/file_write/Makefile.am deleted file mode 100644 index 60895711e7dc..000000000000 --- a/tests/zfs-tests/cmd/file_write/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = file_write -file_write_SOURCES = file_write.c diff --git a/tests/zfs-tests/cmd/get_diff/get_diff.c b/tests/zfs-tests/cmd/get_diff.c similarity index 95% rename from tests/zfs-tests/cmd/get_diff/get_diff.c rename to tests/zfs-tests/cmd/get_diff.c index 2799f46b0747..3f8fe787f7b9 100644 --- a/tests/zfs-tests/cmd/get_diff/get_diff.c +++ b/tests/zfs-tests/cmd/get_diff.c @@ -25,10 +25,9 @@ #include static void -usage(char *msg, int exit_value) +usage(const char *msg, int exit_value) { - (void) fprintf(stderr, "get_diff file redacted_file\n"); - (void) fprintf(stderr, "%s\n", msg); + (void) fprintf(stderr, "usage: get_diff file redacted_file\n%s\n", msg); exit(exit_value); } diff --git a/tests/zfs-tests/cmd/get_diff/.gitignore b/tests/zfs-tests/cmd/get_diff/.gitignore deleted file mode 100644 index f5fc360a6839..000000000000 --- a/tests/zfs-tests/cmd/get_diff/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/get_diff diff --git a/tests/zfs-tests/cmd/get_diff/Makefile.am b/tests/zfs-tests/cmd/get_diff/Makefile.am deleted file mode 100644 index 06c39ddd81ce..000000000000 --- a/tests/zfs-tests/cmd/get_diff/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = get_diff -get_diff_SOURCES = get_diff.c diff --git a/tests/zfs-tests/cmd/getversion/getversion.c b/tests/zfs-tests/cmd/getversion.c similarity index 100% rename from tests/zfs-tests/cmd/getversion/getversion.c rename to tests/zfs-tests/cmd/getversion.c diff --git a/tests/zfs-tests/cmd/getversion/.gitignore b/tests/zfs-tests/cmd/getversion/.gitignore deleted file mode 100644 index b347c417aa13..000000000000 --- a/tests/zfs-tests/cmd/getversion/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/getversion diff --git a/tests/zfs-tests/cmd/getversion/Makefile.am b/tests/zfs-tests/cmd/getversion/Makefile.am deleted file mode 100644 index d6b5e84082b2..000000000000 --- a/tests/zfs-tests/cmd/getversion/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = getversion -getversion_SOURCES = getversion.c diff --git a/tests/zfs-tests/cmd/largest_file/.gitignore b/tests/zfs-tests/cmd/largest_file/.gitignore deleted file mode 100644 index f8f480d06542..000000000000 --- a/tests/zfs-tests/cmd/largest_file/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/largest_file diff --git a/tests/zfs-tests/cmd/largest_file/Makefile.am b/tests/zfs-tests/cmd/largest_file/Makefile.am deleted file mode 100644 index a3e4e9337cf0..000000000000 --- a/tests/zfs-tests/cmd/largest_file/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = largest_file -largest_file_SOURCES = largest_file.c diff --git a/tests/zfs-tests/cmd/libzfs_input_check/libzfs_input_check.c b/tests/zfs-tests/cmd/libzfs_input_check.c similarity index 98% rename from tests/zfs-tests/cmd/libzfs_input_check/libzfs_input_check.c rename to tests/zfs-tests/cmd/libzfs_input_check.c index fba18068cfc8..434cc863f36c 100644 --- a/tests/zfs-tests/cmd/libzfs_input_check/libzfs_input_check.c +++ b/tests/zfs-tests/cmd/libzfs_input_check.c @@ -20,7 +20,6 @@ #include #include #include -#include #include #include @@ -33,19 +32,19 @@ * Test the nvpair inputs for the non-legacy zfs ioctl commands. */ -boolean_t unexpected_failures; -int zfs_fd; -const char *active_test; +static boolean_t unexpected_failures; +static int zfs_fd; +static const char *active_test; /* * Tracks which zfs_ioc_t commands were tested */ -boolean_t ioc_tested[ZFS_IOC_LAST - ZFS_IOC_FIRST]; +static boolean_t ioc_tested[ZFS_IOC_LAST - ZFS_IOC_FIRST]; /* * Legacy ioctls that are skipped (for now) */ -static unsigned ioc_skip[] = { +static const zfs_ioc_t ioc_skip[] = { ZFS_IOC_POOL_CREATE, ZFS_IOC_POOL_DESTROY, ZFS_IOC_POOL_IMPORT, @@ -546,6 +545,7 @@ test_recv_new(const char *dataset, int fd) fnvlist_add_string(props, "org.openzfs:launch", "September 17th, 2013"); fnvlist_add_nvlist(optional, "localprops", props); fnvlist_add_boolean(optional, "force"); + fnvlist_add_boolean(optional, "heal"); fnvlist_add_int32(optional, "cleanup_fd", cleanup_fd); /* @@ -597,13 +597,13 @@ test_channel_program(const char *pool) "arg = ...\n" "argv = arg[\"argv\"]\n" "return argv[1]"; - char *const argv[1] = { "Hello World!" }; + const char *const argv[1] = { "Hello World!" }; nvlist_t *required = fnvlist_alloc(); nvlist_t *optional = fnvlist_alloc(); nvlist_t *args = fnvlist_alloc(); fnvlist_add_string(required, "program", program); - fnvlist_add_string_array(args, "argv", (const char * const *)argv, 1); + fnvlist_add_string_array(args, "argv", argv, 1); fnvlist_add_nvlist(required, "arg", args); fnvlist_add_boolean_value(optional, "sync", B_TRUE); diff --git a/tests/zfs-tests/cmd/libzfs_input_check/.gitignore b/tests/zfs-tests/cmd/libzfs_input_check/.gitignore deleted file mode 100644 index c8796008483f..000000000000 --- a/tests/zfs-tests/cmd/libzfs_input_check/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/libzfs_input_check diff --git a/tests/zfs-tests/cmd/libzfs_input_check/Makefile.am b/tests/zfs-tests/cmd/libzfs_input_check/Makefile.am deleted file mode 100644 index cd462208957c..000000000000 --- a/tests/zfs-tests/cmd/libzfs_input_check/Makefile.am +++ /dev/null @@ -1,17 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = libzfs_input_check - -if BUILD_FREEBSD -DEFAULT_INCLUDES += -I$(top_srcdir)/include/os/freebsd/zfs -endif -if BUILD_LINUX -DEFAULT_INCLUDES += -I$(top_srcdir)/include/os/linux/zfs -endif - -libzfs_input_check_SOURCES = libzfs_input_check.c -libzfs_input_check_LDADD = \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la diff --git a/tests/zfs-tests/cmd/linux_dos_attributes/dos_attributes.h b/tests/zfs-tests/cmd/linux_dos_attributes/dos_attributes.h new file mode 100644 index 000000000000..dd3a820ff20c --- /dev/null +++ b/tests/zfs-tests/cmd/linux_dos_attributes/dos_attributes.h @@ -0,0 +1,82 @@ +// SPDX-License-Identifier: 0BSD + +#include + + +#define U_APPEND_SHORT "uappnd" +#define U_APPEND_FULL "uappend" +#define SU_APPEND_SHORT "sappnd" +#define SU_APPEND_FULL "sappend" + +#define U_ARCH_SHORT "uarch" +#define U_ARCH_FULL "uarchive" +#define SU_ARCH_SHORT "arch" +#define SU_ARCH_FULL "archived" + +#define U_HIDDEN_SHORT "hidden" +#define U_HIDDEN_FULL "uhidden" + +#define SU_IMMUTABLE_FULL "simmutable" +#define SU_IMMUTABLE_SHORT "schange" +#define SU_IMMUTABLE "schg" +#define U_IMMUTABLE_FULL "uimmutable" +#define U_IMMUTABLE_SHORT "uchange" +#define U_IMMUTABLE "uchg" + +#define SU_NODUMP "nodump" +#define UNSET_NODUMP "dump" + +#define U_UNLINK_SHORT "uunlnk" +#define U_UNLINK_FULL "uunlink" +#define SU_UNLINK_SHORT "sunlnk" +#define SU_UNLINK_FULL "sunlink" + +#define U_OFFLINE_SHORT "offline" +#define U_OFFLINE_FULL "uoffline" + +#define U_RDONLY "rdonly" +#define U_RDONLY_SHORT "urdonly" +#define U_RDONLY_FULL "readonly" + +#define U_REPARSE_SHORT "reparse" +#define U_REPARSE_FULL "ureparse" + +#define U_SPARSE_SHORT "sparse" +#define U_SPARSE_FULL "usparse" + +#define U_SYSTEM_SHORT "system" +#define U_SYSTEM_FULL "usystem" + + +static const uint64_t all_dos_attributes[] = { + ZFS_ARCHIVE, + ZFS_APPENDONLY, + ZFS_IMMUTABLE, + ZFS_NOUNLINK, + ZFS_NODUMP, + ZFS_HIDDEN, + ZFS_OFFLINE, + ZFS_READONLY, + ZFS_SPARSE, + ZFS_SYSTEM, + ZFS_REPARSE, +}; + +static const char *const all_dos_attribute_names[][7] = { + {U_ARCH_SHORT, U_ARCH_FULL, SU_ARCH_SHORT, SU_ARCH_FULL}, + {U_APPEND_SHORT, U_APPEND_FULL, SU_APPEND_SHORT, SU_APPEND_FULL}, + {SU_IMMUTABLE_FULL, SU_IMMUTABLE_SHORT, SU_IMMUTABLE, + U_IMMUTABLE_FULL, U_IMMUTABLE_SHORT, U_IMMUTABLE}, + {U_UNLINK_SHORT, U_UNLINK_FULL, SU_UNLINK_FULL, SU_UNLINK_SHORT}, + {SU_NODUMP, /* UNSET_NODUMP */}, + {U_HIDDEN_SHORT, U_HIDDEN_FULL}, + {U_OFFLINE_SHORT, U_OFFLINE_FULL}, + {U_RDONLY, U_RDONLY_SHORT, U_RDONLY_FULL}, + {U_SPARSE_SHORT, U_SPARSE_FULL}, + {U_SYSTEM_SHORT, U_SYSTEM_FULL}, + {U_REPARSE_SHORT, U_REPARSE_FULL}, +}; + +_Static_assert( + ARRAY_SIZE(all_dos_attributes) == ARRAY_SIZE(all_dos_attribute_names), + "attribute list length mismatch"); diff --git a/tests/zfs-tests/cmd/linux_dos_attributes/read_dos_attributes.c b/tests/zfs-tests/cmd/linux_dos_attributes/read_dos_attributes.c new file mode 100644 index 000000000000..07821140512e --- /dev/null +++ b/tests/zfs-tests/cmd/linux_dos_attributes/read_dos_attributes.c @@ -0,0 +1,60 @@ +/* + * This file and its contents are supplied under the terms of the + * Common Development and Distribution License ("CDDL"), version 1.0. + * You may only use this file in accordance with the terms of version + * 1.0 of the CDDL. + * + * A full copy of the text of the CDDL should have accompanied this + * source. A copy of the CDDL is also available via the Internet at + * http://www.illumos.org/license/CDDL. + */ + +/* + * Copyright 2022 iXsystems, Inc. + */ + +/* + * FreeBSD exposes additional file attributes via ls -o and chflags. + * Under Linux, we provide ZFS_IOC_[GS]ETDOSFLAGS ioctl()s. + * + * This application is the equivalent to FreeBSD ls -lo $1 | awk '{print $5}'. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "dos_attributes.h" + +int +main(int argc, const char *const *argv) +{ + if (argc != 2) + errx(EXIT_FAILURE, "usage: %s file", argv[0]); + + int fd = open(argv[1], O_RDONLY | O_CLOEXEC); + if (fd == -1) + err(EXIT_FAILURE, "%s", argv[1]); + + uint64_t flags; + if (ioctl(fd, ZFS_IOC_GETDOSFLAGS, &flags) == -1) + err(EXIT_FAILURE, "ZFS_IOC_GETDOSFLAGS"); + + bool any = false; + for (size_t i = 0; i < ARRAY_SIZE(all_dos_attributes); ++i) + if (flags & all_dos_attributes[i]) { + if (any) + putchar(','); + (void) fputs(*all_dos_attribute_names[i], stdout); + any = true; + } + if (any) + (void) putchar('\n'); + else + (void) puts("-"); +} diff --git a/tests/zfs-tests/cmd/linux_dos_attributes/write_dos_attributes.c b/tests/zfs-tests/cmd/linux_dos_attributes/write_dos_attributes.c new file mode 100644 index 000000000000..3cea7d4b1c7d --- /dev/null +++ b/tests/zfs-tests/cmd/linux_dos_attributes/write_dos_attributes.c @@ -0,0 +1,95 @@ +/* + * This file and its contents are supplied under the terms of the + * Common Development and Distribution License ("CDDL"), version 1.0. + * You may only use this file in accordance with the terms of version + * 1.0 of the CDDL. + * + * A full copy of the text of the CDDL should have accompanied this + * source. A copy of the CDDL is also available via the Internet at + * http://www.illumos.org/license/CDDL. + */ + +/* + * Copyright 2022 iXsystems, Inc. + */ + +/* + * FreeBSD exposes additional file attributes via ls -o and chflags. + * Under Linux, we provide ZFS_IOC_[GS]ETDOSFLAGS ioctl()s. + * + * This application is equivalent to FreeBSD chflags. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "dos_attributes.h" + +int +main(int argc, const char *const *argv) +{ + if (argc != 3) + errx(EXIT_FAILURE, "usage: %s flag file", argv[0]); + + bool unset = false; + uint64_t attr = 0; + const char *flag = argv[1]; + if (strcmp(flag, "0") == 0) + ; + else if (strcmp(flag, SU_NODUMP) == 0) + attr = ZFS_NODUMP; + else if (strcmp(flag, UNSET_NODUMP) == 0) { + attr = ZFS_NODUMP; + unset = true; + } else { + if (strncmp(flag, "no", 2) == 0) { + unset = true; + flag += 2; + } + for (size_t i = 0; i < ARRAY_SIZE(all_dos_attribute_names); ++i) + for (const char *const *nm = all_dos_attribute_names[i]; + *nm; ++nm) + if (strcmp(flag, *nm) == 0) { + attr = all_dos_attributes[i]; + goto found; + } + + errx(EXIT_FAILURE, "%s: unknown flag", argv[1]); +found:; + } + + int fd = open(argv[2], O_RDWR | O_APPEND | O_CLOEXEC); + if (fd == -1) + err(EXIT_FAILURE, "%s", argv[2]); + + uint64_t flags; + if (ioctl(fd, ZFS_IOC_GETDOSFLAGS, &flags) == -1) + err(EXIT_FAILURE, "ZFS_IOC_GETDOSFLAGS"); + + if (attr == 0) + flags = 0; + else if (unset) + flags &= ~attr; + else + flags |= attr; + + if (ioctl(fd, ZFS_IOC_SETDOSFLAGS, &flags) == -1) + err(EXIT_FAILURE, "ZFS_IOC_SETDOSFLAGS"); + + uint64_t newflags; + if (ioctl(fd, ZFS_IOC_GETDOSFLAGS, &newflags) == -1) + err(EXIT_FAILURE, "second ZFS_IOC_GETDOSFLAGS"); + + if (newflags != flags) + errx(EXIT_FAILURE, "expecting %#" PRIx64 ", got %#" PRIx64 + "; %ssetting %#" PRIx64 "", + flags, newflags, unset ? "un" : "", attr); + + (void) printf("%#" PRIx64 "\n", flags); +} diff --git a/tests/zfs-tests/cmd/mkbusy/mkbusy.c b/tests/zfs-tests/cmd/mkbusy.c similarity index 97% rename from tests/zfs-tests/cmd/mkbusy/mkbusy.c rename to tests/zfs-tests/cmd/mkbusy.c index 50a4f90a226a..cc4a6cfcb98c 100644 --- a/tests/zfs-tests/cmd/mkbusy/mkbusy.c +++ b/tests/zfs-tests/cmd/mkbusy.c @@ -24,7 +24,7 @@ #include #include #include -#include +#include #include #include #include @@ -32,14 +32,14 @@ static __attribute__((noreturn)) void -usage(char *progname) +usage(const char *progname) { (void) fprintf(stderr, "Usage: %s \n", progname); exit(1); } static __attribute__((noreturn)) void -fail(char *err) +fail(const char *err) { perror(err); exit(1); diff --git a/tests/zfs-tests/cmd/mkbusy/.gitignore b/tests/zfs-tests/cmd/mkbusy/.gitignore deleted file mode 100644 index 18d099c08eec..000000000000 --- a/tests/zfs-tests/cmd/mkbusy/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/mkbusy diff --git a/tests/zfs-tests/cmd/mkbusy/Makefile.am b/tests/zfs-tests/cmd/mkbusy/Makefile.am deleted file mode 100644 index abae69dea8c7..000000000000 --- a/tests/zfs-tests/cmd/mkbusy/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = mkbusy -mkbusy_SOURCES = mkbusy.c diff --git a/tests/zfs-tests/cmd/mkfile/mkfile.c b/tests/zfs-tests/cmd/mkfile.c similarity index 98% rename from tests/zfs-tests/cmd/mkfile/mkfile.c rename to tests/zfs-tests/cmd/mkfile.c index 673cbf9e0069..7ce50e6a37c4 100644 --- a/tests/zfs-tests/cmd/mkfile/mkfile.c +++ b/tests/zfs-tests/cmd/mkfile.c @@ -7,7 +7,7 @@ * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -44,7 +44,13 @@ #define FILE_MODE (S_ISVTX + S_IRUSR + S_IWUSR) -static void usage(void) __attribute__((noreturn)); +static __attribute__((noreturn)) void +usage(void) +{ + (void) fprintf(stderr, gettext( + "Usage: mkfile [-nv] [g|k|b|m] [] ...\n")); + exit(1); +} int main(int argc, char **argv) @@ -272,10 +278,3 @@ main(int argc, char **argv) } return (errors); } - -static void usage() -{ - (void) fprintf(stderr, gettext( - "Usage: mkfile [-nv] [g|k|b|m] [] ...\n")); - exit(1); -} diff --git a/tests/zfs-tests/cmd/mkfile/.gitignore b/tests/zfs-tests/cmd/mkfile/.gitignore deleted file mode 100644 index 93e9a8a6ded4..000000000000 --- a/tests/zfs-tests/cmd/mkfile/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/mkfile diff --git a/tests/zfs-tests/cmd/mkfile/Makefile.am b/tests/zfs-tests/cmd/mkfile/Makefile.am deleted file mode 100644 index 5f0e2e03efd9..000000000000 --- a/tests/zfs-tests/cmd/mkfile/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = mkfile -mkfile_SOURCES = mkfile.c - -mkfile_LDADD = $(LTLIBINTL) diff --git a/tests/zfs-tests/cmd/mkfiles/mkfiles.c b/tests/zfs-tests/cmd/mkfiles.c similarity index 81% rename from tests/zfs-tests/cmd/mkfiles/mkfiles.c rename to tests/zfs-tests/cmd/mkfiles.c index 32abfd0c3d67..76be4dadb161 100644 --- a/tests/zfs-tests/cmd/mkfiles/mkfiles.c +++ b/tests/zfs-tests/cmd/mkfiles.c @@ -21,13 +21,11 @@ #include #include -#define MAX_INT_LENGTH 10 - static void -usage(char *msg, int exit_value) +usage(const char *msg, int exit_value) { - (void) fprintf(stderr, "mkfiles basename max_file [min_file]\n"); - (void) fprintf(stderr, "%s\n", msg); + (void) fprintf(stderr, "usage: mkfiles basename max_file [min_file]\n" + "%s\n", msg); exit(exit_value); } @@ -40,13 +38,13 @@ main(int argc, char **argv) char buf[MAXPATHLEN]; if (argc < 3 || argc > 4) - usage("Invalid number of arguments", -1); + usage("Invalid number of arguments", 1); if (sscanf(argv[2], "%u", &numfiles) != 1) - usage("Invalid maximum file", -2); + usage("Invalid maximum file", 2); if (argc == 4 && sscanf(argv[3], "%u", &first_file) != 1) - usage("Invalid first file", -3); + usage("Invalid first file", 3); for (i = first_file; i < first_file + numfiles; i++) { int fd; @@ -54,11 +52,11 @@ main(int argc, char **argv) if ((fd = open(buf, O_CREAT | O_EXCL, O_RDWR)) == -1) { (void) fprintf(stderr, "Failed to create %s %s\n", buf, strerror(errno)); - return (-4); + return (4); } else if (fchown(fd, getuid(), getgid()) < 0) { (void) fprintf(stderr, "Failed to chown %s %s\n", buf, strerror(errno)); - return (-5); + return (5); } (void) close(fd); } diff --git a/tests/zfs-tests/cmd/mkfiles/.gitignore b/tests/zfs-tests/cmd/mkfiles/.gitignore deleted file mode 100644 index cee4858b701b..000000000000 --- a/tests/zfs-tests/cmd/mkfiles/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/mkfiles diff --git a/tests/zfs-tests/cmd/mkfiles/Makefile.am b/tests/zfs-tests/cmd/mkfiles/Makefile.am deleted file mode 100644 index 54c21597f3eb..000000000000 --- a/tests/zfs-tests/cmd/mkfiles/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = mkfiles -mkfiles_SOURCES = mkfiles.c diff --git a/tests/zfs-tests/cmd/mktree/mktree.c b/tests/zfs-tests/cmd/mktree.c similarity index 97% rename from tests/zfs-tests/cmd/mktree/mktree.c rename to tests/zfs-tests/cmd/mktree.c index 25b26c9e151b..8ab38ee83aa8 100644 --- a/tests/zfs-tests/cmd/mktree/mktree.c +++ b/tests/zfs-tests/cmd/mktree.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -153,7 +153,7 @@ crtfile(char *pname) { int fd = -1; int i, size; - char *context = "0123456789ABCDF"; + const char *context = "0123456789ABCDF"; char *pbuf; if (pname == NULL) { diff --git a/tests/zfs-tests/cmd/mktree/.gitignore b/tests/zfs-tests/cmd/mktree/.gitignore deleted file mode 100644 index 588bc6d1cce6..000000000000 --- a/tests/zfs-tests/cmd/mktree/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/mktree diff --git a/tests/zfs-tests/cmd/mktree/Makefile.am b/tests/zfs-tests/cmd/mktree/Makefile.am deleted file mode 100644 index 88c74ae0a346..000000000000 --- a/tests/zfs-tests/cmd/mktree/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = mktree -mktree_SOURCES = mktree.c diff --git a/tests/zfs-tests/cmd/mmap_exec/mmap_exec.c b/tests/zfs-tests/cmd/mmap_exec.c similarity index 97% rename from tests/zfs-tests/cmd/mmap_exec/mmap_exec.c rename to tests/zfs-tests/cmd/mmap_exec.c index db90adbdca10..462f985398b6 100644 --- a/tests/zfs-tests/cmd/mmap_exec/mmap_exec.c +++ b/tests/zfs-tests/cmd/mmap_exec.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/tests/zfs-tests/cmd/mmap_exec/.gitignore b/tests/zfs-tests/cmd/mmap_exec/.gitignore deleted file mode 100644 index 63a68bbc681e..000000000000 --- a/tests/zfs-tests/cmd/mmap_exec/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/mmap_exec diff --git a/tests/zfs-tests/cmd/mmap_exec/Makefile.am b/tests/zfs-tests/cmd/mmap_exec/Makefile.am deleted file mode 100644 index ab9f81be9463..000000000000 --- a/tests/zfs-tests/cmd/mmap_exec/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = mmap_exec -mmap_exec_SOURCES = mmap_exec.c diff --git a/tests/zfs-tests/cmd/mmap_libaio/mmap_libaio.c b/tests/zfs-tests/cmd/mmap_libaio.c similarity index 97% rename from tests/zfs-tests/cmd/mmap_libaio/mmap_libaio.c rename to tests/zfs-tests/cmd/mmap_libaio.c index a02f46d91b30..7d76c9b4eb2f 100644 --- a/tests/zfs-tests/cmd/mmap_libaio/mmap_libaio.c +++ b/tests/zfs-tests/cmd/mmap_libaio.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/tests/zfs-tests/cmd/mmap_libaio/.gitignore b/tests/zfs-tests/cmd/mmap_libaio/.gitignore deleted file mode 100644 index 792c8d3400b0..000000000000 --- a/tests/zfs-tests/cmd/mmap_libaio/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/mmap_libaio diff --git a/tests/zfs-tests/cmd/mmap_libaio/Makefile.am b/tests/zfs-tests/cmd/mmap_libaio/Makefile.am deleted file mode 100644 index 25f9dda2b623..000000000000 --- a/tests/zfs-tests/cmd/mmap_libaio/Makefile.am +++ /dev/null @@ -1,10 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -if WANT_MMAP_LIBAIO -pkgexec_PROGRAMS = mmap_libaio -mmap_libaio_SOURCES = mmap_libaio.c -mmap_libaio_CFLAGS = $(AM_CFLAGS) $(LIBAIO_CFLAGS) -mmap_libaio_LDADD = $(LIBAIO_LIBS) -endif diff --git a/tests/zfs-tests/cmd/mmap_seek/mmap_seek.c b/tests/zfs-tests/cmd/mmap_seek.c similarity index 98% rename from tests/zfs-tests/cmd/mmap_seek/mmap_seek.c rename to tests/zfs-tests/cmd/mmap_seek.c index bb36527aafee..7be92d109565 100644 --- a/tests/zfs-tests/cmd/mmap_seek/mmap_seek.c +++ b/tests/zfs-tests/cmd/mmap_seek.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/tests/zfs-tests/cmd/mmap_seek/.gitignore b/tests/zfs-tests/cmd/mmap_seek/.gitignore deleted file mode 100644 index 6b05a7917500..000000000000 --- a/tests/zfs-tests/cmd/mmap_seek/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/mmap_seek diff --git a/tests/zfs-tests/cmd/mmap_seek/Makefile.am b/tests/zfs-tests/cmd/mmap_seek/Makefile.am deleted file mode 100644 index b938931125f5..000000000000 --- a/tests/zfs-tests/cmd/mmap_seek/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = mmap_seek -mmap_seek_SOURCES = mmap_seek.c diff --git a/tests/zfs-tests/cmd/mmap_sync.c b/tests/zfs-tests/cmd/mmap_sync.c new file mode 100644 index 000000000000..0e4bba37d7be --- /dev/null +++ b/tests/zfs-tests/cmd/mmap_sync.c @@ -0,0 +1,152 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static void +cleanup(char *file) +{ + remove(file); +} + +int +main(int argc, char *argv[]) +{ + char *testdir = getenv("TESTDIR"); + if (!testdir) { + fprintf(stderr, "environment variable TESTDIR not set\n"); + return (1); + } + + struct stat st; + umask(0); + if (stat(testdir, &st) != 0 && + mkdir(testdir, 0777) != 0) { + perror("mkdir"); + return (1); + } + + if (argc > 3) { + fprintf(stderr, "usage: %s " + "[run time in mins] " + "[max msync time in ms]\n", argv[0]); + return (1); + } + + int run_time_mins = 5; + if (argc >= 2) { + run_time_mins = atoi(argv[1]); + } + + int max_msync_time_ms = 1000; + if (argc >= 3) { + max_msync_time_ms = atoi(argv[2]); + } + + char filepath[512]; + filepath[0] = '\0'; + char *file = &filepath[0]; + + strcat(file, testdir); + strcat(file, "/msync_file"); + + const int LEN = 8; + cleanup(file); + + int fd = open(file, O_CREAT | O_RDWR, S_IRUSR | S_IWUSR | + S_IRGRP | S_IROTH); + + if (fd == -1) { + (void) fprintf(stderr, "%s: %s: ", argv[0], file); + perror("open"); + return (1); + } + + if (ftruncate(fd, LEN) != 0) { + perror("ftruncate"); + cleanup(file); + return (1); + } + + void *ptr = mmap(NULL, LEN, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); + + if (ptr == MAP_FAILED) { + perror("mmap"); + cleanup(file); + return (1); + } + + struct timeval tstart; + gettimeofday(&tstart, NULL); + + long long x = 0LL; + + for (;;) { + *((long long *)ptr) = x; + x++; + + struct timeval t1, t2; + gettimeofday(&t1, NULL); + if (msync(ptr, LEN, MS_SYNC|MS_INVALIDATE) != 0) { + perror("msync"); + cleanup(file); + return (1); + } + + gettimeofday(&t2, NULL); + + double elapsed = (t2.tv_sec - t1.tv_sec) * 1000.0; + elapsed += ((t2.tv_usec - t1.tv_usec) / 1000.0); + if (elapsed > max_msync_time_ms) { + fprintf(stderr, "slow msync: %f ms\n", elapsed); + munmap(ptr, LEN); + cleanup(file); + return (1); + } + + double elapsed_start = (t2.tv_sec - tstart.tv_sec) * 1000.0; + elapsed_start += ((t2.tv_usec - tstart.tv_usec) / 1000.0); + if (elapsed_start > run_time_mins * 60 * 1000) { + break; + } + } + + if (munmap(ptr, LEN) != 0) { + perror("munmap"); + cleanup(file); + return (1); + } + + if (close(fd) != 0) { + perror("close"); + } + + cleanup(file); + return (0); +} diff --git a/tests/zfs-tests/cmd/mmapwrite/mmapwrite.c b/tests/zfs-tests/cmd/mmapwrite.c similarity index 98% rename from tests/zfs-tests/cmd/mmapwrite/mmapwrite.c rename to tests/zfs-tests/cmd/mmapwrite.c index 1f344534d53e..0d277d6e3099 100644 --- a/tests/zfs-tests/cmd/mmapwrite/mmapwrite.c +++ b/tests/zfs-tests/cmd/mmapwrite.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/tests/zfs-tests/cmd/mmapwrite/.gitignore b/tests/zfs-tests/cmd/mmapwrite/.gitignore deleted file mode 100644 index 4e7043bbfd58..000000000000 --- a/tests/zfs-tests/cmd/mmapwrite/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/mmapwrite diff --git a/tests/zfs-tests/cmd/mmapwrite/Makefile.am b/tests/zfs-tests/cmd/mmapwrite/Makefile.am deleted file mode 100644 index b21b9e779bf2..000000000000 --- a/tests/zfs-tests/cmd/mmapwrite/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = mmapwrite -mmapwrite_SOURCES = mmapwrite.c -mmapwrite_LDADD = -lpthread diff --git a/tests/zfs-tests/cmd/nvlist_to_lua/nvlist_to_lua.c b/tests/zfs-tests/cmd/nvlist_to_lua.c similarity index 96% rename from tests/zfs-tests/cmd/nvlist_to_lua/nvlist_to_lua.c rename to tests/zfs-tests/cmd/nvlist_to_lua.c index 3072ca97939c..b65b3fd269d9 100644 --- a/tests/zfs-tests/cmd/nvlist_to_lua/nvlist_to_lua.c +++ b/tests/zfs-tests/cmd/nvlist_to_lua.c @@ -20,13 +20,12 @@ #include #include #include -#include #include #include -nvlist_t *nvl; -const char *pool; -boolean_t unexpected_failures; +static nvlist_t *nvl; +static const char *pool; +static boolean_t unexpected_failures; static boolean_t nvlist_equal(nvlist_t *nvla, nvlist_t *nvlb) @@ -85,7 +84,7 @@ nvlist_equal(nvlist_t *nvla, nvlist_t *nvlb) static void test(const char *testname, boolean_t expect_success, boolean_t expect_match) { - char *progstr = "input = ...; return {output=input}"; + const char *progstr = "input = ...; return {output=input}"; nvlist_t *outnvl; @@ -231,8 +230,8 @@ run_tests(void) test("uint64_array", B_FALSE, B_FALSE); } { - char *const val[2] = { "0", "1" }; - fnvlist_add_string_array(nvl, key, (const char **)val, 2); + const char *val[2] = { "0", "1" }; + fnvlist_add_string_array(nvl, key, val, 2); test("string_array", B_TRUE, B_FALSE); } { diff --git a/tests/zfs-tests/cmd/nvlist_to_lua/.gitignore b/tests/zfs-tests/cmd/nvlist_to_lua/.gitignore deleted file mode 100644 index b31db6454dce..000000000000 --- a/tests/zfs-tests/cmd/nvlist_to_lua/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/nvlist_to_lua diff --git a/tests/zfs-tests/cmd/nvlist_to_lua/Makefile.am b/tests/zfs-tests/cmd/nvlist_to_lua/Makefile.am deleted file mode 100644 index 511b6c6913bb..000000000000 --- a/tests/zfs-tests/cmd/nvlist_to_lua/Makefile.am +++ /dev/null @@ -1,10 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = nvlist_to_lua - -nvlist_to_lua_SOURCES = nvlist_to_lua.c -nvlist_to_lua_LDADD = \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la diff --git a/tests/zfs-tests/cmd/randfree_file/.gitignore b/tests/zfs-tests/cmd/randfree_file/.gitignore deleted file mode 100644 index 0f5b394c5fbd..000000000000 --- a/tests/zfs-tests/cmd/randfree_file/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/randfree_file diff --git a/tests/zfs-tests/cmd/randfree_file/Makefile.am b/tests/zfs-tests/cmd/randfree_file/Makefile.am deleted file mode 100644 index 6306e0e75740..000000000000 --- a/tests/zfs-tests/cmd/randfree_file/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = randfree_file -randfree_file_SOURCES = randfree_file.c diff --git a/tests/zfs-tests/cmd/randwritecomp/.gitignore b/tests/zfs-tests/cmd/randwritecomp/.gitignore deleted file mode 100644 index fb231c678cb2..000000000000 --- a/tests/zfs-tests/cmd/randwritecomp/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/randwritecomp diff --git a/tests/zfs-tests/cmd/randwritecomp/Makefile.am b/tests/zfs-tests/cmd/randwritecomp/Makefile.am deleted file mode 100644 index 0002291fa7bf..000000000000 --- a/tests/zfs-tests/cmd/randwritecomp/Makefile.am +++ /dev/null @@ -1,9 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -DEFAULT_INCLUDES += \ - -I$(top_srcdir)/include - -pkgexec_PROGRAMS = randwritecomp -randwritecomp_SOURCES = randwritecomp.c diff --git a/tests/zfs-tests/cmd/readmmap/readmmap.c b/tests/zfs-tests/cmd/readmmap.c similarity index 97% rename from tests/zfs-tests/cmd/readmmap/readmmap.c rename to tests/zfs-tests/cmd/readmmap.c index e21c2c867d9a..c4812b4a259d 100644 --- a/tests/zfs-tests/cmd/readmmap/readmmap.c +++ b/tests/zfs-tests/cmd/readmmap.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -49,7 +49,7 @@ int main(int argc, char **argv) { - char *filename = "badfile"; + const char *filename = "badfile"; size_t size = 4395; size_t idx = 0; char *buf = NULL; diff --git a/tests/zfs-tests/cmd/readmmap/.gitignore b/tests/zfs-tests/cmd/readmmap/.gitignore deleted file mode 100644 index 3799193a92be..000000000000 --- a/tests/zfs-tests/cmd/readmmap/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/readmmap diff --git a/tests/zfs-tests/cmd/readmmap/Makefile.am b/tests/zfs-tests/cmd/readmmap/Makefile.am deleted file mode 100644 index 9b735c287e69..000000000000 --- a/tests/zfs-tests/cmd/readmmap/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = readmmap -readmmap_SOURCES = readmmap.c diff --git a/tests/zfs-tests/cmd/rename_dir/rename_dir.c b/tests/zfs-tests/cmd/rename_dir.c similarity index 92% rename from tests/zfs-tests/cmd/rename_dir/rename_dir.c rename to tests/zfs-tests/cmd/rename_dir.c index 897ec7b84148..568cbfe720b9 100644 --- a/tests/zfs-tests/cmd/rename_dir/rename_dir.c +++ b/tests/zfs-tests/cmd/rename_dir.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -46,7 +46,6 @@ #include #include #include -#include int main(void) @@ -58,6 +57,7 @@ main(void) perror("fork"); exit(1); break; + case 0: while (i > 0) { int c_count = 0; @@ -65,11 +65,11 @@ main(void) c_count++; if (rename("1/2/3/c", "a/b/c") == 0) c_count++; - if (c_count) { + if (c_count) (void) fprintf(stderr, "c_count: %d", c_count); - } } - break; + _exit(0); + default: while (i > 0) { int p_count = 0; @@ -77,12 +77,9 @@ main(void) p_count++; if (rename("a/b/c/d/e/1", "1") == 0) p_count++; - if (p_count) { + if (p_count) (void) fprintf(stderr, "p_count: %d", p_count); - } } - break; + return (0); } - - return (0); } diff --git a/tests/zfs-tests/cmd/rename_dir/.gitignore b/tests/zfs-tests/cmd/rename_dir/.gitignore deleted file mode 100644 index 39a0cb222ad1..000000000000 --- a/tests/zfs-tests/cmd/rename_dir/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/rename_dir diff --git a/tests/zfs-tests/cmd/rename_dir/Makefile.am b/tests/zfs-tests/cmd/rename_dir/Makefile.am deleted file mode 100644 index 21971cd888fb..000000000000 --- a/tests/zfs-tests/cmd/rename_dir/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = rename_dir -rename_dir_SOURCES = rename_dir.c diff --git a/tests/zfs-tests/cmd/rm_lnkcnt_zero_file/rm_lnkcnt_zero_file.c b/tests/zfs-tests/cmd/rm_lnkcnt_zero_file.c similarity index 97% rename from tests/zfs-tests/cmd/rm_lnkcnt_zero_file/rm_lnkcnt_zero_file.c rename to tests/zfs-tests/cmd/rm_lnkcnt_zero_file.c index f8b6e3e808bb..0bf3b54cc090 100644 --- a/tests/zfs-tests/cmd/rm_lnkcnt_zero_file/rm_lnkcnt_zero_file.c +++ b/tests/zfs-tests/cmd/rm_lnkcnt_zero_file.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -45,7 +45,6 @@ #include #include #include -#include static char *filebase; diff --git a/tests/zfs-tests/cmd/rm_lnkcnt_zero_file/.gitignore b/tests/zfs-tests/cmd/rm_lnkcnt_zero_file/.gitignore deleted file mode 100644 index fc6323fb3ff3..000000000000 --- a/tests/zfs-tests/cmd/rm_lnkcnt_zero_file/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/rm_lnkcnt_zero_file diff --git a/tests/zfs-tests/cmd/rm_lnkcnt_zero_file/Makefile.am b/tests/zfs-tests/cmd/rm_lnkcnt_zero_file/Makefile.am deleted file mode 100644 index 90fc8d0541b6..000000000000 --- a/tests/zfs-tests/cmd/rm_lnkcnt_zero_file/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = rm_lnkcnt_zero_file -rm_lnkcnt_zero_file_SOURCES = rm_lnkcnt_zero_file.c -rm_lnkcnt_zero_file_LDADD = -lpthread diff --git a/tests/zfs-tests/cmd/send_doall/send_doall.c b/tests/zfs-tests/cmd/send_doall.c similarity index 97% rename from tests/zfs-tests/cmd/send_doall/send_doall.c rename to tests/zfs-tests/cmd/send_doall.c index 6f47df047478..e0ab6c41812c 100644 --- a/tests/zfs-tests/cmd/send_doall/send_doall.c +++ b/tests/zfs-tests/cmd/send_doall.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/tests/zfs-tests/cmd/send_doall/.gitignore b/tests/zfs-tests/cmd/send_doall/.gitignore deleted file mode 100644 index 6ba2e603f744..000000000000 --- a/tests/zfs-tests/cmd/send_doall/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/send_doall diff --git a/tests/zfs-tests/cmd/send_doall/Makefile.am b/tests/zfs-tests/cmd/send_doall/Makefile.am deleted file mode 100644 index 33a6b83122b8..000000000000 --- a/tests/zfs-tests/cmd/send_doall/Makefile.am +++ /dev/null @@ -1,11 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = send_doall - -send_doall_SOURCES = send_doall.c -send_doall_LDADD = \ - $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la \ - $(abs_top_builddir)/lib/libzfs/libzfs.la \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la diff --git a/tests/zfs-tests/cmd/stride_dd/stride_dd.c b/tests/zfs-tests/cmd/stride_dd.c similarity index 99% rename from tests/zfs-tests/cmd/stride_dd/stride_dd.c rename to tests/zfs-tests/cmd/stride_dd.c index 88bd532923c7..732ac9f47268 100644 --- a/tests/zfs-tests/cmd/stride_dd/stride_dd.c +++ b/tests/zfs-tests/cmd/stride_dd.c @@ -27,7 +27,7 @@ static char *ifile = NULL; static char *ofile = NULL; static int stride = 0; static int seek = 0; -static char *execname = "stride_dd"; +static const char *execname = "stride_dd"; static void usage(void); static void parse_options(int argc, char *argv[]); diff --git a/tests/zfs-tests/cmd/stride_dd/.gitignore b/tests/zfs-tests/cmd/stride_dd/.gitignore deleted file mode 100644 index 7c072ee0dec6..000000000000 --- a/tests/zfs-tests/cmd/stride_dd/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/stride_dd diff --git a/tests/zfs-tests/cmd/stride_dd/Makefile.am b/tests/zfs-tests/cmd/stride_dd/Makefile.am deleted file mode 100644 index d6f1adbac2b7..000000000000 --- a/tests/zfs-tests/cmd/stride_dd/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = stride_dd -stride_dd_SOURCES = stride_dd.c -stride_dd_LDADD = -lrt diff --git a/tests/zfs-tests/tests/functional/suid/suid_write_to_file.c b/tests/zfs-tests/cmd/suid_write_to_file.c similarity index 98% rename from tests/zfs-tests/tests/functional/suid/suid_write_to_file.c rename to tests/zfs-tests/cmd/suid_write_to_file.c index f3febb903b59..1a8157aa564f 100644 --- a/tests/zfs-tests/tests/functional/suid/suid_write_to_file.c +++ b/tests/zfs-tests/cmd/suid_write_to_file.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/tests/zfs-tests/cmd/threadsappend/threadsappend.c b/tests/zfs-tests/cmd/threadsappend.c similarity index 98% rename from tests/zfs-tests/cmd/threadsappend/threadsappend.c rename to tests/zfs-tests/cmd/threadsappend.c index 25710a3c12ef..b0ccd0ff6c42 100644 --- a/tests/zfs-tests/cmd/threadsappend/threadsappend.c +++ b/tests/zfs-tests/cmd/threadsappend.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/tests/zfs-tests/cmd/threadsappend/.gitignore b/tests/zfs-tests/cmd/threadsappend/.gitignore deleted file mode 100644 index 4c8c8cdf34c1..000000000000 --- a/tests/zfs-tests/cmd/threadsappend/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/threadsappend diff --git a/tests/zfs-tests/cmd/threadsappend/Makefile.am b/tests/zfs-tests/cmd/threadsappend/Makefile.am deleted file mode 100644 index f030b42d50fe..000000000000 --- a/tests/zfs-tests/cmd/threadsappend/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = threadsappend -threadsappend_SOURCES = threadsappend.c -threadsappend_LDADD = -lpthread diff --git a/tests/zfs-tests/tests/functional/truncate/truncate_test.c b/tests/zfs-tests/cmd/truncate_test.c similarity index 100% rename from tests/zfs-tests/tests/functional/truncate/truncate_test.c rename to tests/zfs-tests/cmd/truncate_test.c diff --git a/tests/zfs-tests/cmd/user_ns_exec/user_ns_exec.c b/tests/zfs-tests/cmd/user_ns_exec.c similarity index 98% rename from tests/zfs-tests/cmd/user_ns_exec/user_ns_exec.c rename to tests/zfs-tests/cmd/user_ns_exec.c index cd46738bd0bd..86593622399e 100644 --- a/tests/zfs-tests/cmd/user_ns_exec/user_ns_exec.c +++ b/tests/zfs-tests/cmd/user_ns_exec.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * diff --git a/tests/zfs-tests/cmd/user_ns_exec/.gitignore b/tests/zfs-tests/cmd/user_ns_exec/.gitignore deleted file mode 100644 index 655867a640a3..000000000000 --- a/tests/zfs-tests/cmd/user_ns_exec/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/user_ns_exec diff --git a/tests/zfs-tests/cmd/user_ns_exec/Makefile.am b/tests/zfs-tests/cmd/user_ns_exec/Makefile.am deleted file mode 100644 index 5b4bc9aaa683..000000000000 --- a/tests/zfs-tests/cmd/user_ns_exec/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = user_ns_exec -user_ns_exec_SOURCES = user_ns_exec.c diff --git a/tests/zfs-tests/cmd/xattrtest/xattrtest.c b/tests/zfs-tests/cmd/xattrtest.c similarity index 97% rename from tests/zfs-tests/cmd/xattrtest/xattrtest.c rename to tests/zfs-tests/cmd/xattrtest.c index 49b6629ba056..4e72e9182b60 100644 --- a/tests/zfs-tests/cmd/xattrtest/xattrtest.c +++ b/tests/zfs-tests/cmd/xattrtest.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -91,8 +91,8 @@ static int size_is_random = 0; static int value_is_random = 0; static int keep_files = 0; static int phase = PHASE_ALL; -static char path[PATH_MAX] = "/tmp/xattrtest"; -static char script[PATH_MAX] = "/bin/true"; +static const char *path = "/tmp/xattrtest"; +static const char *script = "/bin/true"; static char xattrbytes[XATTR_SIZE_MAX]; static int @@ -161,8 +161,7 @@ parse_args(int argc, char **argv) } break; case 'p': - strncpy(path, optarg, PATH_MAX); - path[PATH_MAX - 1] = '\0'; + path = optarg; break; case 'c': synccaches = 1; @@ -171,8 +170,7 @@ parse_args(int argc, char **argv) dropcaches = 1; break; case 't': - strncpy(script, optarg, PATH_MAX); - script[PATH_MAX - 1] = '\0'; + script = optarg; break; case 'e': seed = strtol(optarg, NULL, 0); @@ -291,9 +289,9 @@ run_process(const char *path, char *argv[]) } static int -post_hook(char *phase) +post_hook(const char *phase) { - char *argv[3] = { script, phase, (char *)0 }; + char *argv[3] = { (char *)script, (char *)phase, NULL }; int rc; if (synccaches) @@ -517,9 +515,9 @@ getxattrs(void) int i, j, rnd_size, shift, rc = 0; char name[XATTR_NAME_MAX]; char *verify_value = NULL; - char *verify_string; + const char *verify_string; char *value = NULL; - char *value_string; + const char *value_string; char *file = NULL; struct timeval start, stop; double seconds; diff --git a/tests/zfs-tests/cmd/xattrtest/.gitignore b/tests/zfs-tests/cmd/xattrtest/.gitignore deleted file mode 100644 index 7d2128383639..000000000000 --- a/tests/zfs-tests/cmd/xattrtest/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/xattrtest diff --git a/tests/zfs-tests/cmd/xattrtest/Makefile.am b/tests/zfs-tests/cmd/xattrtest/Makefile.am deleted file mode 100644 index 7398ae634629..000000000000 --- a/tests/zfs-tests/cmd/xattrtest/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin - -pkgexec_PROGRAMS = xattrtest -xattrtest_SOURCES = xattrtest.c diff --git a/tests/zfs-tests/tests/functional/events/zed_fd_spill-zedlet.c b/tests/zfs-tests/cmd/zed_fd_spill-zedlet.c similarity index 100% rename from tests/zfs-tests/tests/functional/events/zed_fd_spill-zedlet.c rename to tests/zfs-tests/cmd/zed_fd_spill-zedlet.c diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_diff/socket.c b/tests/zfs-tests/cmd/zfs_diff-socket.c similarity index 100% rename from tests/zfs-tests/tests/functional/cli_root/zfs_diff/socket.c rename to tests/zfs-tests/cmd/zfs_diff-socket.c diff --git a/tests/zfs-tests/include/Makefile.am b/tests/zfs-tests/include/Makefile.am deleted file mode 100644 index 16cdf2c81432..000000000000 --- a/tests/zfs-tests/include/Makefile.am +++ /dev/null @@ -1,14 +0,0 @@ -include $(top_srcdir)/config/Substfiles.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/include -dist_pkgdata_DATA = \ - blkdev.shlib \ - commands.cfg \ - libtest.shlib \ - math.shlib \ - properties.shlib \ - tunables.cfg \ - zpool_script.shlib - -nodist_pkgdata_DATA = default.cfg -SUBSTFILES += $(nodist_pkgdata_DATA) diff --git a/tests/zfs-tests/include/blkdev.shlib b/tests/zfs-tests/include/blkdev.shlib index 7159b92c080f..6b83b10d604d 100644 --- a/tests/zfs-tests/include/blkdev.shlib +++ b/tests/zfs-tests/include/blkdev.shlib @@ -45,9 +45,7 @@ function scan_scsi_hosts log_must eval "echo '- - -' > $host/scan" done else - log_must eval \ - "echo /sys/class/scsi_host/host$hostnum/scan" \ - > /dev/null + log_note "/sys/class/scsi_host/host$hostnum/scan" log_must eval \ "echo '- - -' > /sys/class/scsi_host/host$hostnum/scan" fi @@ -115,21 +113,18 @@ function is_physical_device #device if is_linux; then is_disk_device "$DEV_DSKDIR/$device" && \ - [[ -f /sys/module/loop/parameters/max_part ]] - return $? + [ -f /sys/module/loop/parameters/max_part ] elif is_freebsd; then is_disk_device "$DEV_DSKDIR/$device" && \ - echo $device | egrep -q \ + echo $device | grep -qE \ -e '^a?da[0-9]+$' \ -e '^md[0-9]+$' \ -e '^mfid[0-9]+$' \ -e '^nda[0-9]+$' \ -e '^nvd[0-9]+$' \ -e '^vtbd[0-9]+$' - return $? else - echo $device | egrep "^c[0-F]+([td][0-F]+)+$" > /dev/null 2>&1 - return $? + echo $device | grep -qE "^c[0-F]+([td][0-F]+)+$" fi } @@ -143,8 +138,7 @@ function is_real_device #disk if is_linux; then lsblk $DEV_RDSKDIR/$disk -o TYPE 2>/dev/null | \ - egrep disk >/dev/null - return $? + grep -q disk fi } @@ -158,8 +152,7 @@ function is_loop_device #disk if is_linux; then lsblk $DEV_RDSKDIR/$disk -o TYPE 2>/dev/null | \ - egrep loop >/dev/null - return $? + grep -q loop fi } @@ -181,13 +174,11 @@ function is_mpath_device #disk [[ -z $disk ]] && log_fail "No argument for disk given." if is_linux; then - lsblk $DEV_MPATHDIR/$disk -o TYPE 2>/dev/null | \ - egrep mpath >/dev/null - if (($? == 0)); then + if lsblk $DEV_MPATHDIR/$disk -o TYPE 2>/dev/null | \ + grep -q mpath; then readlink $DEV_MPATHDIR/$disk > /dev/null 2>&1 - return $? else - return $? + false fi elif is_freebsd; then is_disk_device $DEV_MPATHDIR/$disk @@ -223,13 +214,11 @@ function set_slice_prefix if is_linux; then while (( i < $DISK_ARRAY_NUM )); do - disk="$(echo $DISKS | nawk '{print $(i + 1)}')" - if ( is_mpath_device $disk ) && [[ -z $(echo $disk | awk 'substr($1,18,1)\ - ~ /^[[:digit:]]+$/') ]] || ( is_real_device $disk ); then + disk="$(echo $DISKS | awk '{print $(i + 1)}')" + if is_mpath_device $disk && ! echo $disk | awk 'substr($1,18,1) ~ /^[[:digit:]]+$/ {exit 1}' || is_real_device $disk; then export SLICE_PREFIX="" return 0 - elif ( is_mpath_device $disk || is_loop_device \ - $disk ); then + elif is_mpath_device $disk || is_loop_device $disk; then export SLICE_PREFIX="p" return 0 else @@ -255,7 +244,7 @@ function set_device_dir if is_linux; then while (( i < $DISK_ARRAY_NUM )); do - disk="$(echo $DISKS | nawk '{print $(i + 1)}')" + disk="$(echo $DISKS | awk '{print $(i + 1)}')" if is_mpath_device $disk; then export DEV_DSKDIR=$DEV_MPATHDIR return 0 @@ -296,20 +285,15 @@ function get_device_dir #device function get_persistent_disk_name #device { typeset device=$1 - typeset dev_id if is_linux; then if is_real_device $device; then - dev_id="$(udevadm info -q all -n $DEV_DSKDIR/$device \ - | egrep disk/by-id | nawk '{print $2; exit}' \ - | nawk -F / '{print $3}')" - echo $dev_id + udevadm info -q all -n $DEV_DSKDIR/$device \ + | awk '/disk\/by-id/ {print $2; exit}' | cut -d/ -f3 elif is_mpath_device $device; then - dev_id="$(udevadm info -q all -n $DEV_DSKDIR/$device \ - | egrep disk/by-id/dm-uuid \ - | nawk '{print $2; exit}' \ - | nawk -F / '{print $3}')" - echo $dev_id + udevadm info -q all -n $DEV_DSKDIR/$device \ + | awk '/disk\/by-id\/dm-uuid/ {print $2; exit}' \ + | cut -d/ -f3 else echo $device fi @@ -336,42 +320,33 @@ function on_off_disk # disk state{online,offline} host if is_linux; then if [[ $state == "offline" ]] && ( is_mpath_device $disk ); then - dm_name="$(readlink $DEV_DSKDIR/$disk \ - | nawk -F / '{print $2}')" - dep="$(ls /sys/block/${dm_name}/slaves \ - | nawk '{print $1}')" + dm_name="$(readlink $DEV_DSKDIR/$disk | cut -d/ -f2)" + dep="$(ls /sys/block/${dm_name}/slaves | awk '{print $1}')" while [[ -n $dep ]]; do #check if disk is online - lsscsi | egrep $dep > /dev/null - if (($? == 0)); then + if lsscsi | grep -qF $dep; then dep_dir="/sys/block/${dm_name}" dep_dir+="/slaves/${dep}/device" ss="${dep_dir}/state" sd="${dep_dir}/delete" log_must eval "echo 'offline' > ${ss}" log_must eval "echo '1' > ${sd}" - lsscsi | egrep $dep > /dev/null - if (($? == 0)); then - log_fail "Offlining" \ - "$disk failed" - fi + if lsscsi | grep -qF $dep; then + log_fail "Offlining $disk failed" + fi fi - dep="$(ls /sys/block/$dm_name/slaves \ - 2>/dev/null | nawk '{print $1}')" + dep="$(ls /sys/block/$dm_name/slaves 2>/dev/null | awk '{print $1}')" done elif [[ $state == "offline" ]] && ( is_real_device $disk ); then #check if disk is online - lsscsi | egrep $disk > /dev/null - if (($? == 0)); then + if lsscsi | grep -qF $disk; then dev_state="/sys/block/$disk/device/state" dev_delete="/sys/block/$disk/device/delete" log_must eval "echo 'offline' > ${dev_state}" log_must eval "echo '1' > ${dev_delete}" - lsscsi | egrep $disk > /dev/null - if (($? == 0)); then - log_fail "Offlining $disk" \ - "failed" - fi + if lsscsi | grep -qF $disk; then + log_fail "Offlining $disk failed" + fi else log_note "$disk is already offline" fi @@ -380,18 +355,15 @@ function on_off_disk # disk state{online,offline} host scan_scsi_hosts $host block_device_wait if is_mpath_device $disk; then - dm_name="$(readlink $DEV_DSKDIR/$disk \ - | nawk -F / '{print $2}')" - dep="$(ls /sys/block/$dm_name/slaves \ - | nawk '{print $1}')" - lsscsi | egrep $dep > /dev/null - if (($? != 0)); then + dm_name="$(readlink $DEV_DSKDIR/$disk | cut -d/ -f2)" + dep="$(ls /sys/block/$dm_name/slaves | awk '{print $1}')" + if lsscsi | grep -qF $dep; then log_fail "Onlining $disk failed" fi elif is_real_device $disk; then block_device_wait typeset -i retries=0 - while ! lsscsi | egrep -q $disk; do + while ! lsscsi | grep -qF $disk; do if (( $retries > 2 )); then log_fail "Onlining $disk failed" break @@ -462,21 +434,16 @@ function load_scsi_debug # dev_size_mb add_host num_tgts max_luns blksz esac if is_linux; then - modprobe -n scsi_debug - if (($? != 0)); then - log_unsupported "Platform does not have scsi_debug" - "module" - fi - lsmod | egrep scsi_debug > /dev/null - if (($? == 0)); then + modprobe -n scsi_debug || + log_unsupported "Platform does not have scsi_debug module" + if lsmod | grep -q scsi_debug; then log_fail "scsi_debug module already installed" else log_must modprobe scsi_debug dev_size_mb=$devsize \ add_host=$hosts num_tgts=$tgts max_luns=$luns \ sector_size=$sector physblk_exp=$blkexp block_device_wait - lsscsi | egrep scsi_debug > /dev/null - if (($? == 1)); then + if ! lsscsi | grep -q scsi_debug; then log_fail "scsi_debug module install failed" fi fi @@ -498,7 +465,7 @@ function unload_scsi_debug function get_debug_device { for i in {1..10} ; do - val=$(lsscsi | nawk '/scsi_debug/ {print $6; exit}' | cut -d / -f3) + val=$(lsscsi | awk '/scsi_debug/ {print $6; exit}' | cut -d/ -f3) # lsscsi can take time to settle if [ "$val" != "-" ] ; then @@ -518,11 +485,11 @@ function get_pool_devices #testpool #devdir typeset devdir=$2 typeset out="" - if is_linux || is_freebsd; then - out=$(zpool status -P $testpool |grep ${devdir} | awk '{print $1}') - out=$(echo $out | sed -e "s|${devdir}/||g" | tr '\n' ' ') - fi - echo $out + case "$UNAME" in + Linux|FreeBSD) + zpool status -P $testpool | awk -v d="$devdir" '$1 ~ d {sub(d "/", ""); printf("%s ", $1)}' + ;; + esac } # diff --git a/tests/zfs-tests/include/commands.cfg b/tests/zfs-tests/include/commands.cfg index 51052e801e87..47357dca57fb 100644 --- a/tests/zfs-tests/include/commands.cfg +++ b/tests/zfs-tests/include/commands.cfg @@ -8,9 +8,7 @@ # Please keep the contents of each variable sorted for ease of reading # and maintenance. # -export SYSTEM_FILES_COMMON='arp - awk - base64 +export SYSTEM_FILES_COMMON='awk basename bc bunzip2 @@ -32,7 +30,6 @@ export SYSTEM_FILES_COMMON='arp dmesg du echo - egrep env expr false @@ -53,10 +50,10 @@ export SYSTEM_FILES_COMMON='arp ksh ldd ln - logname ls mkdir mknod + mkfifo mktemp mount mv @@ -68,15 +65,13 @@ export SYSTEM_FILES_COMMON='arp pgrep ping pkill - printenv printf ps - pwd python3 - quotaon readlink rm rmdir + rsync scp script sed @@ -88,34 +83,26 @@ export SYSTEM_FILES_COMMON='arp ssh stat strings - su sudo - sum swapoff swapon sync tail tar - tee timeout touch tr true truncate - umask umount uname uniq - uuidgen vmstat - wait - wc - which' + wc' export SYSTEM_FILES_FREEBSD='chflags compress diskinfo - dumpon fsck getextattr gpart @@ -125,7 +112,6 @@ export SYSTEM_FILES_FREEBSD='chflags lsextattr md5 mdconfig - mkfifo newfs pw rmextattr @@ -134,17 +120,16 @@ export SYSTEM_FILES_FREEBSD='chflags showmount swapctl sysctl + trim uncompress' export SYSTEM_FILES_LINUX='attr - bash blkid + blkdiscard blockdev chattr - dmidecode exportfs fallocate - fdisk free getfattr groupadd @@ -161,16 +146,19 @@ export SYSTEM_FILES_LINUX='attr mkswap modprobe mpstat - nproc + nsenter parted perf - setenforce setfattr sha256sum udevadm + unshare useradd userdel - usermod' + usermod + + flock + logger' export ZFS_FILES='zdb zfs @@ -195,6 +183,7 @@ export ZFSTEST_FILES='badsend devname2devid dir_rd_update draid + file_append file_check file_trunc file_write @@ -209,15 +198,30 @@ export ZFSTEST_FILES='badsend mmap_exec mmap_libaio mmap_seek + mmap_sync mmapwrite nvlist_to_lua randfree_file randwritecomp readmmap + read_dos_attributes rename_dir rm_lnkcnt_zero_file send_doall threadsappend user_ns_exec + write_dos_attributes xattrtest - stride_dd' + stride_dd + zed_fd_spill-zedlet + suid_write_to_file + cp_files + blake3_test + edonr_test + skein_test + sha2_test + ctime + truncate_test + ereports + zfs_diff-socket + dosmode_readonly_write' diff --git a/tests/zfs-tests/include/default.cfg.in b/tests/zfs-tests/include/default.cfg.in index cf382cfe994c..56111a1473c4 100644 --- a/tests/zfs-tests/include/default.cfg.in +++ b/tests/zfs-tests/include/default.cfg.in @@ -8,7 +8,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -170,8 +170,8 @@ if [ "@UBSAN_ENABLED@" = "yes" ]; then fi -case $(uname -o) in -GNU/Linux) +case $(uname) in +Linux) unpack_opts="--sparse -xf" pack_opts="--sparse -cf" verbose=" -v" @@ -209,7 +209,7 @@ FreeBSD) NEWFS_DEFAULT_FS="ufs" SLICE_PREFIX="p" ;; -illumos) +*) export AUTO_SNAP=$(svcs -a | \ awk '/auto-snapshot/ && /online/ { print $3 }') # finally, if we're running in a local zone diff --git a/tests/zfs-tests/include/libtest.shlib b/tests/zfs-tests/include/libtest.shlib index b229a161518b..6d72f6f20780 100644 --- a/tests/zfs-tests/include/libtest.shlib +++ b/tests/zfs-tests/include/libtest.shlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -31,12 +31,12 @@ # Use is subject to license terms. # +. ${STF_SUITE}/include/tunables.cfg + . ${STF_TOOLS}/include/logapi.shlib . ${STF_SUITE}/include/math.shlib . ${STF_SUITE}/include/blkdev.shlib -. ${STF_SUITE}/include/tunables.cfg - # # Apply constrained path when available. This is required since the # PATH may have been modified by sudo's secure_path behavior. @@ -52,11 +52,7 @@ fi # function compare_version_gte { - if [[ "$(printf "$1\n$2" | sort -V | tail -n1)" == "$1" ]]; then - return 0 - else - return 1 - fi + [ "$(printf "$1\n$2" | sort -V | tail -n1)" = "$1" ] } # Linux kernel version comparison function @@ -69,17 +65,16 @@ function linux_version { typeset ver="$1" - [[ -z "$ver" ]] && ver=$(uname -r | grep -Eo "^[0-9]+\.[0-9]+\.[0-9]+") + [ -z "$ver" ] && ver=$(uname -r | grep -Eo "^[0-9]+\.[0-9]+\.[0-9]+") - typeset version=$(echo $ver | cut -d '.' -f 1) - typeset major=$(echo $ver | cut -d '.' -f 2) - typeset minor=$(echo $ver | cut -d '.' -f 3) + typeset version major minor _ + IFS='.' read -r version major minor _ <<<"$ver" - [[ -z "$version" ]] && version=0 - [[ -z "$major" ]] && major=0 - [[ -z "$minor" ]] && minor=0 + [ -z "$version" ] && version=0 + [ -z "$major" ] && major=0 + [ -z "$minor" ] && minor=0 - echo $((version * 10000 + major * 100 + minor)) + echo $((version * 100000 + major * 1000 + minor)) } # Determine if this is a Linux test system @@ -88,11 +83,7 @@ function linux_version function is_linux { - if [[ $(uname -o) == "GNU/Linux" ]]; then - return 0 - else - return 1 - fi + [ "$UNAME" = "Linux" ] } # Determine if this is an illumos test system @@ -100,11 +91,7 @@ function is_linux # Return 0 if platform illumos, 1 if otherwise function is_illumos { - if [[ $(uname -o) == "illumos" ]]; then - return 0 - else - return 1 - fi + [ "$UNAME" = "illumos" ] } # Determine if this is a FreeBSD test system @@ -113,26 +100,7 @@ function is_illumos function is_freebsd { - if [[ $(uname -o) == "FreeBSD" ]]; then - return 0 - else - return 1 - fi -} - -# Determine if this is a DilOS test system -# -# Return 0 if platform DilOS, 1 if otherwise - -function is_dilos -{ - typeset ID="" - [[ -f /etc/os-release ]] && . /etc/os-release - if [[ $ID == "dilos" ]]; then - return 0 - else - return 1 - fi + [ "$UNAME" = "FreeBSD" ] } # Determine if this is a 32-bit system @@ -141,11 +109,7 @@ function is_dilos function is_32bit { - if [[ $(getconf LONG_BIT) == "32" ]]; then - return 0 - else - return 1 - fi + [ $(getconf LONG_BIT) = "32" ] } # Determine if kmemleak is enabled @@ -154,11 +118,7 @@ function is_32bit function is_kmemleak { - if is_linux && [[ -e /sys/kernel/debug/kmemleak ]]; then - return 0 - else - return 1 - fi + is_linux && [ -e /sys/kernel/debug/kmemleak ] } # Determine whether a dataset is mounted @@ -172,18 +132,14 @@ function ismounted { typeset fstype=$2 [[ -z $fstype ]] && fstype=zfs - typeset out dir name ret + typeset out dir name case $fstype in zfs) if [[ "$1" == "/"* ]] ; then - for out in $(zfs mount | awk '{print $2}'); do - [[ $1 == $out ]] && return 0 - done + ! zfs mount | awk -v fs="$1" '$2 == fs {exit 1}' else - for out in $(zfs mount | awk '{print $1}'); do - [[ $1 == $out ]] && return 0 - done + ! zfs mount | awk -v ds="$1" '$1 == ds {exit 1}' fi ;; ufs|nfs) @@ -192,9 +148,7 @@ function ismounted [[ "$1" == "$dev" || "$1" == "$dir" ]] && return 0 done else - out=$(df -F $fstype $1 2>/dev/null) - ret=$? - (($ret != 0)) && return $ret + out=$(df -F $fstype $1 2>/dev/null) || return dir=${out%%\(*} dir=${dir%% *} @@ -206,8 +160,7 @@ function ismounted fi ;; ext*) - out=$(df -t $fstype $1 2>/dev/null) - return $? + df -t $fstype $1 > /dev/null 2>&1 ;; zvol) if [[ -L "$ZVOL_DEVDIR/$1" ]]; then @@ -217,9 +170,10 @@ function ismounted return 0 fi ;; + *) + false + ;; esac - - return 1 } # Return 0 if a dataset is mounted; 1 otherwise @@ -230,8 +184,6 @@ function ismounted function mounted { ismounted $1 $2 - (($? == 0)) && return 0 - return 1 } # Return 0 if a dataset is unmounted; 1 otherwise @@ -241,18 +193,7 @@ function mounted function unmounted { - ismounted $1 $2 - (($? == 1)) && return 0 - return 1 -} - -# split line on "," -# -# $1 - line to split - -function splitline -{ - echo $1 | tr ',' ' ' + ! ismounted $1 $2 } function default_setup @@ -448,7 +389,7 @@ function create_recv_clone datasetexists $recvfs && log_fail "Recv filesystem must not exist." datasetexists $sendfs && log_fail "Send filesystem must not exist." - log_must zfs create -o mountpoint="$mountpoint" $sendfs + log_must zfs create -o compression=off -o mountpoint="$mountpoint" $sendfs log_must zfs snapshot $snap log_must eval "zfs send $snap | zfs recv -u $recvfs" log_must mkfile 1m "$mountpoint/data" @@ -492,48 +433,6 @@ function default_mirror_setup_noexit log_must zfs set mountpoint=$TESTDIR $TESTPOOL/$TESTFS } -# -# create a number of mirrors. -# We create a number($1) of 2 way mirrors using the pairs of disks named -# on the command line. These mirrors are *not* mounted -# @parameters: $1 the number of mirrors to create -# $... the devices to use to create the mirrors on -# @uses: ZPOOL ZFS TESTPOOL -function setup_mirrors -{ - typeset -i nmirrors=$1 - - shift - while ((nmirrors > 0)); do - log_must test -n "$1" -a -n "$2" - [[ -d /$TESTPOOL$nmirrors ]] && rm -rf /$TESTPOOL$nmirrors - log_must zpool create -f $TESTPOOL$nmirrors mirror $1 $2 - shift 2 - ((nmirrors = nmirrors - 1)) - done -} - -# -# create a number of raidz pools. -# We create a number($1) of 2 raidz pools using the pairs of disks named -# on the command line. These pools are *not* mounted -# @parameters: $1 the number of pools to create -# $... the devices to use to create the pools on -# @uses: ZPOOL ZFS TESTPOOL -function setup_raidzs -{ - typeset -i nraidzs=$1 - - shift - while ((nraidzs > 0)); do - log_must test -n "$1" -a -n "$2" - [[ -d /$TESTPOOL$nraidzs ]] && rm -rf /$TESTPOOL$nraidzs - log_must zpool create -f $TESTPOOL$nraidzs raidz $1 $2 - shift 2 - ((nraidzs = nraidzs - 1)) - done -} - # # Destroy the configured testpool mirrors. # the mirrors are of the form ${TESTPOOL}{number} @@ -545,11 +444,18 @@ function destroy_mirrors log_pass } +function default_raidz_setup +{ + default_raidz_setup_noexit "$*" + + log_pass +} + # # Given a minimum of two disks, set up a storage pool and dataset for the raid-z # $1 the list of disks # -function default_raidz_setup +function default_raidz_setup_noexit { typeset disklist="$*" disks=(${disklist[*]}) @@ -562,8 +468,6 @@ function default_raidz_setup log_must zpool create -f $TESTPOOL raidz $disklist log_must zfs create $TESTPOOL/$TESTFS log_must zfs set mountpoint=$TESTDIR $TESTPOOL/$TESTFS - - log_pass } # @@ -640,10 +544,8 @@ function default_cleanup_noexit log_must zfs set reservation=none $fs log_must zfs set recordsize=128K $fs log_must zfs set mountpoint=/$fs $fs - typeset enc="" - enc=$(get_prop encryption $fs) - if [[ $? -ne 0 ]] || [[ -z "$enc" ]] || \ - [[ "$enc" == "off" ]]; then + typeset enc=$(get_prop encryption $fs) + if [ -z "$enc" ] || [ "$enc" = "off" ]; then log_must zfs set checksum=on $fs fi log_must zfs set compression=off $fs @@ -681,15 +583,14 @@ function default_container_cleanup reexport_pool fi - ismounted $TESTPOOL/$TESTCTR/$TESTFS1 - [[ $? -eq 0 ]] && \ + ismounted $TESTPOOL/$TESTCTR/$TESTFS1 && log_must zfs unmount $TESTPOOL/$TESTCTR/$TESTFS1 destroy_dataset "$TESTPOOL/$TESTCTR/$TESTFS1" "-R" destroy_dataset "$TESTPOOL/$TESTCTR" "-Rf" [[ -e $TESTDIR1 ]] && \ - log_must rm -rf $TESTDIR1 > /dev/null 2>&1 + log_must rm -rf $TESTDIR1 default_cleanup } @@ -716,8 +617,6 @@ function destroy_snapshot typeset mtpt="" if ismounted $snap; then mtpt=$(get_prop mountpoint $snap) - (($? != 0)) && \ - log_fail "get_prop mountpoint $snap failed." fi destroy_dataset "$snap" @@ -742,8 +641,6 @@ function destroy_clone typeset mtpt="" if ismounted $clone; then mtpt=$(get_prop mountpoint $clone) - (($? != 0)) && \ - log_fail "get_prop mountpoint $clone failed." fi destroy_dataset "$clone" @@ -775,7 +672,6 @@ function destroy_bookmark function snapexists { zfs list -H -t snapshot "$1" > /dev/null 2>&1 - return $? } # @@ -786,7 +682,6 @@ function snapexists function bkmarkexists { zfs list -H -t bookmark "$1" > /dev/null 2>&1 - return $? } # @@ -797,8 +692,7 @@ function bkmarkexists # function holdexists { - zfs holds "$2" | awk '{ print $2 }' | grep "$1" > /dev/null 2>&1 - return $? + ! zfs holds "$2" | awk -v t="$1" '$2 ~ t { exit 1 }' } # @@ -834,46 +728,6 @@ function dataset_setprop return 0 } -# -# Assign suite defined dataset properties. -# This function is used to apply the suite's defined default set of -# properties to a dataset. -# @parameters: $1 dataset to use -# @uses: ZFS COMPRESSION_PROP CHECKSUM_PROP -# @returns: -# 0 if the dataset has been altered. -# 1 if no pool name was passed in. -# 2 if the dataset could not be found. -# 3 if the dataset could not have it's properties set. -# -function dataset_set_defaultproperties -{ - typeset dataset="$1" - - [[ -z $dataset ]] && return 1 - - typeset confset= - typeset -i found=0 - for confset in $(zfs list); do - if [[ $dataset = $confset ]]; then - found=1 - break - fi - done - [[ $found -eq 0 ]] && return 2 - if [[ -n $COMPRESSION_PROP ]]; then - dataset_setprop $dataset compression $COMPRESSION_PROP || \ - return 3 - log_note "Compression set to '$COMPRESSION_PROP' on $dataset" - fi - if [[ -n $CHECKSUM_PROP ]]; then - dataset_setprop $dataset checksum $CHECKSUM_PROP || \ - return 3 - log_note "Checksum set to '$CHECKSUM_PROP' on $dataset" - fi - return 0 -} - # # Check a numeric assertion # @parameter: $@ the assertion to check @@ -933,7 +787,7 @@ function set_partition typeset disk=${4#$DEV_DSKDIR/} disk=${disk#$DEV_RDSKDIR/} - case "$(uname)" in + case "$UNAME" in Linux) if [[ -z $size || -z $disk ]]; then log_fail "The size or disk name is unspecified." @@ -951,8 +805,7 @@ function set_partition parted $disk -s -- print 1 >/dev/null typeset ret_val=$? if [[ $slicenum -eq 0 || $ret_val -ne 0 ]]; then - parted $disk -s -- mklabel gpt - if [[ $? -ne 0 ]]; then + if ! parted $disk -s -- mklabel gpt; then log_note "Failed to create GPT partition table on $disk" return 1 fi @@ -966,9 +819,8 @@ function set_partition # Determine the cylinder size for the device and using # that calculate the end offset in cylinders. typeset -i cly_size_kb=0 - cly_size_kb=$(parted -m $disk -s -- \ - unit cyl print | head -3 | tail -1 | \ - awk -F '[:k.]' '{print $4}') + cly_size_kb=$(parted -m $disk -s -- unit cyl print | + awk -F '[:k.]' 'NR == 3 {print $4}') ((end = (size_mb * 1024 / cly_size_kb) + start)) parted $disk -s -- \ @@ -990,8 +842,7 @@ function set_partition if [[ $slicenum -eq 0 ]] || ! gpart show $disk >/dev/null 2>&1; then gpart destroy -F $disk >/dev/null 2>&1 - gpart create -s GPT $disk - if [[ $? -ne 0 ]]; then + if ! gpart create -s GPT $disk; then log_note "Failed to create GPT partition table on $disk" return 1 fi @@ -1089,7 +940,7 @@ function get_endslice # log_fail "The disk name or slice number is unspecified." fi - case "$(uname)" in + case "$UNAME" in Linux) endcyl=$(parted -s $DEV_DSKDIR/$disk -- unit cyl print | \ awk "/part${slice}/"' {sub(/cyl/, "", $3); print $3}') @@ -1109,15 +960,14 @@ function get_endslice # typeset -i ratio=0 ratio=$(prtvtoc /dev/rdsk/${disk}s2 | \ - grep "sectors\/cylinder" | \ - awk '{print $2}') + awk '/sectors\/cylinder/ {print $2}') if ((ratio == 0)); then return fi typeset -i endcyl=$(prtvtoc -h /dev/rdsk/${disk}s2 | - nawk -v token="$slice" '{if ($1==token) print $6}') + awk -v token="$slice" '$1 == token {print $6}') ((endcyl = (endcyl + 1) / ratio)) ;; @@ -1190,61 +1040,26 @@ function fill_fs # destdir dirnum filenum bytes num_writes data mkdir -p $destdir/{1..$dirnum} for f in $destdir/{1..$dirnum}/$TESTFILE{1..$filenum}; do file_write -o create -f $f -b $bytes -c $num_writes -d $data \ - || return $? + || return done - return 0 } -# -# Simple function to get the specified property. If unable to -# get the property then exits. -# -# Note property is in 'parsable' format (-p) -# +# Get the specified dataset property in parsable format or fail function get_prop # property dataset { - typeset prop_val typeset prop=$1 typeset dataset=$2 - prop_val=$(zfs get -pH -o value $prop $dataset 2>/dev/null) - if [[ $? -ne 0 ]]; then - log_note "Unable to get $prop property for dataset " \ - "$dataset" - return 1 - fi - - echo "$prop_val" - return 0 + zfs get -Hpo value "$prop" "$dataset" || log_fail "zfs get $prop $dataset" } -# -# Simple function to get the specified property of pool. If unable to -# get the property then exits. -# -# Note property is in 'parsable' format (-p) -# +# Get the specified pool property in parsable format or fail function get_pool_prop # property pool { - typeset prop_val typeset prop=$1 typeset pool=$2 - if poolexists $pool ; then - prop_val=$(zpool get -pH $prop $pool 2>/dev/null | tail -1 | \ - awk '{print $3}') - if [[ $? -ne 0 ]]; then - log_note "Unable to get $prop property for pool " \ - "$pool" - return 1 - fi - else - log_note "Pool $pool not exists." - return 1 - fi - - echo "$prop_val" - return 0 + zpool get -Hpo value "$prop" "$pool" || log_fail "zpool get $prop $pool" } # Return 0 if a pool exists; $? otherwise @@ -1261,7 +1076,6 @@ function poolexists fi zpool get name "$pool" > /dev/null 2>&1 - return $? } # Return 0 if all the specified datasets exist; $? otherwise @@ -1274,13 +1088,7 @@ function datasetexists return 1 fi - while (($# > 0)); do - zfs get name $1 > /dev/null 2>&1 || \ - return $? - shift - done - - return 0 + zfs get name "$@" > /dev/null 2>&1 } # return 0 if none of the specified datasets exists, otherwise return 1. @@ -1302,11 +1110,23 @@ function datasetnonexists return 0 } +# FreeBSD breaks exports(5) at whitespace and doesn't process escapes +# Solaris just breaks +# +# cf. https://github.com/openzfs/zfs/pull/13165#issuecomment-1059845807 +# +# Linux can have spaces (which are \OOO-escaped), +# but can't have backslashes because they're parsed recursively +function shares_can_have_whitespace +{ + is_linux +} + function is_shared_freebsd { typeset fs=$1 - pgrep -q mountd && showmount -E | grep -qx $fs + pgrep -q mountd && showmount -E | grep -qx "$fs" } function is_shared_illumos @@ -1331,14 +1151,7 @@ function is_shared_illumos function is_shared_linux { typeset fs=$1 - typeset mtpt - - for mtpt in `share | awk '{print $1}'` ; do - if [[ $mtpt == $fs ]] ; then - return 0 - fi - done - return 1 + ! exportfs -s | awk -v fs="${fs//\\/\\\\}" '/^\// && $1 == fs {exit 1}' } # @@ -1356,7 +1169,7 @@ function is_shared return 1 else mtpt=$(get_prop mountpoint "$fs") - case $mtpt in + case "$mtpt" in none|legacy|-) return 1 ;; *) fs=$mtpt @@ -1365,7 +1178,7 @@ function is_shared fi fi - case $(uname) in + case "$UNAME" in FreeBSD) is_shared_freebsd "$fs" ;; Linux) is_shared_linux "$fs" ;; *) is_shared_illumos "$fs" ;; @@ -1375,13 +1188,11 @@ function is_shared function is_exported_illumos { typeset fs=$1 - typeset mtpt + typeset mtpt _ - for mtpt in `awk '{print $1}' /etc/dfs/sharetab` ; do - if [[ $mtpt == $fs ]] ; then - return 0 - fi - done + while read -r mtpt _; do + [ "$mtpt" = "$fs" ] && return + done < /etc/dfs/sharetab return 1 } @@ -1389,13 +1200,11 @@ function is_exported_illumos function is_exported_freebsd { typeset fs=$1 - typeset mtpt + typeset mtpt _ - for mtpt in `awk '{print $1}' /etc/zfs/exports` ; do - if [[ $mtpt == $fs ]] ; then - return 0 - fi - done + while read -r mtpt _; do + [ "$mtpt" = "$fs" ] && return + done < /etc/zfs/exports return 1 } @@ -1403,13 +1212,11 @@ function is_exported_freebsd function is_exported_linux { typeset fs=$1 - typeset mtpt + typeset mtpt _ - for mtpt in `awk '{print $1}' /etc/exports.d/zfs.exports` ; do - if [[ $mtpt == $fs ]] ; then - return 0 - fi - done + while read -r mtpt _; do + [ "$(printf "$mtpt")" = "$fs" ] && return + done < /etc/exports.d/zfs.exports return 1 } @@ -1439,7 +1246,7 @@ function is_exported fi fi - case $(uname) in + case "$UNAME" in FreeBSD) is_exported_freebsd "$fs" ;; Linux) is_exported_linux "$fs" ;; *) is_exported_illumos "$fs" ;; @@ -1454,23 +1261,13 @@ function is_exported function is_shared_smb { typeset fs=$1 - typeset mtpt - if datasetnonexists "$fs" ; then - return 1 - else - fs=$(echo $fs | tr / _) - fi + datasetexists "$fs" || return if is_linux; then - for mtpt in `net usershare list | awk '{print $1}'` ; do - if [[ $mtpt == $fs ]] ; then - return 0 - fi - done - return 1 + net usershare list | grep -xFq "${fs//\//_}" else - log_note "Currently unsupported by the test framework" + log_note "SMB on $UNAME currently unsupported by the test framework" return 1 fi } @@ -1482,14 +1279,7 @@ function is_shared_smb # function not_shared { - typeset fs=$1 - - is_shared $fs - if (($? == 0)); then - return 1 - fi - - return 0 + ! is_shared $1 } # @@ -1499,14 +1289,7 @@ function not_shared # function not_shared_smb { - typeset fs=$1 - - is_shared_smb $fs - if (($? == 0)); then - return 1 - fi - - return 0 + ! is_shared_smb $1 } # @@ -1516,12 +1299,9 @@ function unshare_fs #fs { typeset fs=$1 - is_shared $fs || is_shared_smb $fs - if (($? == 0)); then - zfs unshare $fs || log_fail "zfs unshare $fs failed" + if is_shared $fs || is_shared_smb $fs; then + log_must zfs unshare $fs fi - - return 0 } # @@ -1531,17 +1311,22 @@ function share_nfs #fs { typeset fs=$1 - if is_linux; then - is_shared $fs - if (($? != 0)); then - log_must share "*:$fs" - fi - else - is_shared $fs - if (($? != 0)); then - log_must share -F nfs $fs - fi - fi + is_shared "$fs" && return + + case "$UNAME" in + Linux) + log_must exportfs "*:$fs" + ;; + FreeBSD) + typeset mountd + read -r mountd < /var/run/mountd.pid + log_must eval "printf '%s\t\n' \"$fs\" >> /etc/zfs/exports" + log_must kill -s HUP "$mountd" + ;; + *) + log_must share -F nfs "$fs" + ;; + esac return 0 } @@ -1553,17 +1338,23 @@ function unshare_nfs #fs { typeset fs=$1 - if is_linux; then - is_shared $fs - if (($? == 0)); then - log_must unshare -u "*:$fs" - fi - else - is_shared $fs - if (($? == 0)); then - log_must unshare -F nfs $fs - fi - fi + ! is_shared "$fs" && return + + case "$UNAME" in + Linux) + log_must exportfs -u "*:$fs" + ;; + FreeBSD) + typeset mountd + read -r mountd < /var/run/mountd.pid + awk -v fs="${fs//\\/\\\\}" '$1 != fs' /etc/zfs/exports > /etc/zfs/exports.$$ + log_must mv /etc/zfs/exports.$$ /etc/zfs/exports + log_must kill -s HUP "$mountd" + ;; + *) + log_must unshare -F nfs $fs + ;; + esac return 0 } @@ -1573,42 +1364,32 @@ function unshare_nfs #fs # function showshares_nfs { - if is_linux; then - share -v - else + case "$UNAME" in + Linux) + exportfs -v + ;; + FreeBSD) + showmount + ;; + *) share -F nfs - fi - - return 0 -} - -# -# Helper function to show SMB shares. -# -function showshares_smb -{ - if is_linux; then - net usershare list - else - share -F smb - fi - - return 0 + ;; + esac } function check_nfs { - if is_linux; then - share -s - elif is_freebsd; then + case "$UNAME" in + Linux) + exportfs -s + ;; + FreeBSD) showmount -e - else + ;; + *) log_unsupported "Unknown platform" - fi - - if [[ $? -ne 0 ]]; then - log_unsupported "The NFS utilities are not installed" - fi + ;; + esac || log_unsupported "The NFS utilities are not installed" } # @@ -1628,12 +1409,12 @@ function setup_nfs_server # Re-synchronize /var/lib/nfs/etab with /etc/exports and # /etc/exports.d./* to provide a clean test environment. # - log_must share -r + log_must exportfs -r log_note "NFS server must be started prior to running ZTS." return elif is_freebsd; then - kill -s HUP $(cat /var/run/mountd.pid) + log_must kill -s HUP $(/dev/null) - if [[ $cur_zone != "global" ]]; then - return 1 - fi - return 0 + [ $cur_zone = "global" ] fi } @@ -1836,7 +1614,7 @@ function create_dataset #dataset dataset_options # $2 - custom arguments for zfs destroy # Destroy dataset with the given parameters. -function destroy_dataset #dataset #args +function destroy_dataset # dataset [args] { typeset dataset=$1 typeset mtpt @@ -1852,8 +1630,7 @@ function destroy_dataset #dataset #args mtpt=$(get_prop mountpoint "$dataset") log_must_busy zfs destroy $args $dataset - [[ -d $mtpt ]] && \ - log_must rm -rf $mtpt + [ -d $mtpt ] && log_must rm -rf $mtpt else log_note "Dataset does not exist. ($dataset)" return 1 @@ -1863,118 +1640,6 @@ function destroy_dataset #dataset #args return 0 } -# -# Firstly, create a pool with 5 datasets. Then, create a single zone and -# export the 5 datasets to it. In addition, we also add a ZFS filesystem -# and a zvol device to the zone. -# -# $1 zone name -# $2 zone root directory prefix -# $3 zone ip -# -function zfs_zones_setup #zone_name zone_root zone_ip -{ - typeset zone_name=${1:-$(hostname)-z} - typeset zone_root=${2:-"/zone_root"} - typeset zone_ip=${3:-"10.1.1.10"} - typeset prefix_ctr=$ZONE_CTR - typeset pool_name=$ZONE_POOL - typeset -i cntctr=5 - typeset -i i=0 - - # Create pool and 5 container within it - # - [[ -d /$pool_name ]] && rm -rf /$pool_name - log_must zpool create -f $pool_name $DISKS - while ((i < cntctr)); do - log_must zfs create $pool_name/$prefix_ctr$i - ((i += 1)) - done - - # create a zvol - log_must zfs create -V 1g $pool_name/zone_zvol - block_device_wait - - # - # Add slog device for pool - # - typeset sdevs="$TEST_BASE_DIR/sdev1 $TEST_BASE_DIR/sdev2" - log_must mkfile $MINVDEVSIZE $sdevs - log_must zpool add $pool_name log mirror $sdevs - - # this isn't supported just yet. - # Create a filesystem. In order to add this to - # the zone, it must have it's mountpoint set to 'legacy' - # log_must zfs create $pool_name/zfs_filesystem - # log_must zfs set mountpoint=legacy $pool_name/zfs_filesystem - - [[ -d $zone_root ]] && \ - log_must rm -rf $zone_root/$zone_name - [[ ! -d $zone_root ]] && \ - log_must mkdir -p -m 0700 $zone_root/$zone_name - - # Create zone configure file and configure the zone - # - typeset zone_conf=/tmp/zone_conf.$$ - echo "create" > $zone_conf - echo "set zonepath=$zone_root/$zone_name" >> $zone_conf - echo "set autoboot=true" >> $zone_conf - i=0 - while ((i < cntctr)); do - echo "add dataset" >> $zone_conf - echo "set name=$pool_name/$prefix_ctr$i" >> \ - $zone_conf - echo "end" >> $zone_conf - ((i += 1)) - done - - # add our zvol to the zone - echo "add device" >> $zone_conf - echo "set match=/dev/zvol/dsk/$pool_name/zone_zvol" >> $zone_conf - echo "end" >> $zone_conf - - # add a corresponding zvol rdsk to the zone - echo "add device" >> $zone_conf - echo "set match=$ZVOL_RDEVDIR/$pool_name/zone_zvol" >> $zone_conf - echo "end" >> $zone_conf - - # once it's supported, we'll add our filesystem to the zone - # echo "add fs" >> $zone_conf - # echo "set type=zfs" >> $zone_conf - # echo "set special=$pool_name/zfs_filesystem" >> $zone_conf - # echo "set dir=/export/zfs_filesystem" >> $zone_conf - # echo "end" >> $zone_conf - - echo "verify" >> $zone_conf - echo "commit" >> $zone_conf - log_must zonecfg -z $zone_name -f $zone_conf - log_must rm -f $zone_conf - - # Install the zone - zoneadm -z $zone_name install - if (($? == 0)); then - log_note "SUCCESS: zoneadm -z $zone_name install" - else - log_fail "FAIL: zoneadm -z $zone_name install" - fi - - # Install sysidcfg file - # - typeset sysidcfg=$zone_root/$zone_name/root/etc/sysidcfg - echo "system_locale=C" > $sysidcfg - echo "terminal=dtterm" >> $sysidcfg - echo "network_interface=primary {" >> $sysidcfg - echo "hostname=$zone_name" >> $sysidcfg - echo "}" >> $sysidcfg - echo "name_service=NONE" >> $sysidcfg - echo "root_password=mo791xfZ/SFiw" >> $sysidcfg - echo "security_policy=NONE" >> $sysidcfg - echo "timezone=US/Eastern" >> $sysidcfg - - # Boot this zone - log_must zoneadm -z $zone_name boot -} - # # Reexport TESTPOOL & TESTPOOL(1-4) # @@ -2015,14 +1680,10 @@ function check_state # pool disk state{online,offline,degraded} if [[ -z $disk ]]; then #check pool state only - zpool get -H -o value health $pool \ - | grep -i "$state" > /dev/null 2>&1 + zpool get -H -o value health $pool | grep -qi "$state" else - zpool status -v $pool | grep "$disk" \ - | grep -i "$state" > /dev/null 2>&1 + zpool status -v $pool | grep "$disk" | grep -qi "$state" fi - - return $? } # @@ -2056,19 +1717,16 @@ function verify_ashift # device ashift typeset device="$1" typeset ashift="$2" - zdb -e -lll $device | awk -v ashift=$ashift '/ashift: / { - if (ashift != $2) - exit 1; - else - count++; - } END { - if (count != 4) - exit 1; - else - exit 0; + zdb -e -lll $device | awk -v ashift=$ashift ' + /ashift: / { + if (ashift != $2) + exit 1; + else + count++; + } + END { + exit (count != 4); }' - - return $? } # @@ -2099,10 +1757,10 @@ function verify_filesys # pool filesystem dir log_must zpool import $search_path $pool - zdb -cudi $filesys > $zdbout 2>&1 - if [[ $? != 0 ]]; then + if ! zdb -cudi $filesys > $zdbout 2>&1; then log_note "Output: zdb -cudi $filesys" cat $zdbout + rm -f $zdbout log_fail "zdb detected errors with: '$filesys'" fi @@ -2137,13 +1795,8 @@ function verify_pool # function get_disklist # pool { - typeset disklist="" - - disklist=$(zpool iostat -v $1 | nawk '(NR >4) {print $1}' | \ - grep -v "\-\-\-\-\-" | \ - egrep -v -e "^(mirror|raidz[1-3]|draid[1-3]|spare|log|cache|special|dedup)|\-[0-9]$") - - echo $disklist + echo $(zpool iostat -v $1 | awk '(NR > 4) {print $1}' | \ + grep -vEe '^-----' -e "^(mirror|raidz[1-3]|draid[1-3]|spare|log|cache|special|dedup)|\-[0-9]$") } # @@ -2152,8 +1805,7 @@ function get_disklist # pool # function get_disklist_fullpath # pool { - args="-P $1" - get_disklist $args + get_disklist "-P $1" } @@ -2181,10 +1833,8 @@ function stress_timeout log_note "Killing child processes after ${TIMEOUT} stress timeout." typeset pid for pid in $cpids; do - ps -p $pid > /dev/null 2>&1 - if (($? == 0)); then + ps -p $pid > /dev/null 2>&1 && log_must kill -USR1 $pid - fi done } @@ -2201,10 +1851,7 @@ function check_hotspare_state # pool disk state{inuse,avail} cur_state=$(get_device_state $pool $disk "spares") - if [[ $state != ${cur_state} ]]; then - return 1 - fi - return 0 + [ $state = $cur_state ] } # @@ -2227,28 +1874,9 @@ function wait_hotspare_state # pool disk state timeout i=$((i+1)) sleep 1 - done - - return 1 -} - -# -# Verify a given slog disk is inuse or avail -# -# Return 0 is pool/disk matches expected state, 1 otherwise -# -function check_slog_state # pool disk state{online,offline,unavail} -{ - typeset pool=$1 - typeset disk=${2#$DEV_DSKDIR/} - typeset state=$3 - - cur_state=$(get_device_state $pool $disk "logs") - - if [[ $state != ${cur_state} ]]; then - return 1 - fi - return 0 + done + + return 1 } # @@ -2264,10 +1892,7 @@ function check_vdev_state # pool disk state{online,offline,unavail} cur_state=$(get_device_state $pool $disk) - if [[ $state != ${cur_state} ]]; then - return 1 - fi - return 0 + [ $state = $cur_state ] } # @@ -2308,14 +1933,11 @@ function check_pool_status # pool token keyword typeset keyword=$3 typeset verbose=${4:-false} - scan=$(zpool status -v "$pool" 2>/dev/null | nawk -v token="$token:" ' - ($1==token) {print $0}') + scan=$(zpool status -v "$pool" 2>/dev/null | awk -v token="$token:" '$1==token') if [[ $verbose == true ]]; then log_note $scan fi - echo $scan | egrep -i "$keyword" > /dev/null 2>&1 - - return $? + echo $scan | grep -qi "$keyword" } # @@ -2334,55 +1956,46 @@ function is_pool_resilvering #pool { check_pool_status "$1" "scan" \ "resilver[ ()0-9A-Za-z:_-]* in progress since" $2 - return $? } function is_pool_resilvered #pool { check_pool_status "$1" "scan" "resilvered " $2 - return $? } function is_pool_scrubbing #pool { check_pool_status "$1" "scan" "scrub in progress since " $2 - return $? } function is_pool_scrubbed #pool { check_pool_status "$1" "scan" "scrub repaired" $2 - return $? } function is_pool_scrub_stopped #pool { check_pool_status "$1" "scan" "scrub canceled" $2 - return $? } function is_pool_scrub_paused #pool { check_pool_status "$1" "scan" "scrub paused since " $2 - return $? } function is_pool_removing #pool { check_pool_status "$1" "remove" "in progress since " - return $? } function is_pool_removed #pool { check_pool_status "$1" "remove" "completed on" - return $? } function is_pool_discarding #pool { check_pool_status "$1" "checkpoint" "discarding" - return $? } function wait_for_degraded @@ -2449,55 +2062,39 @@ function find_disks swap -l > $sfi dumpadm > $dmpi 2>/dev/null -# write an awk script that can process the output of format -# to produce a list of disks we know about. Note that we have -# to escape "$2" so that the shell doesn't interpret it while -# we're creating the awk script. -# ------------------- - cat > /tmp/find_disks.awk </dev/null | awk ' +BEGIN { FS="."; } - { - if (searchdisks && \$2 !~ "^$"){ - split(\$2,arr," "); - print arr[1]; - } - } +/^Specify disk/{ + searchdisks=0; +} - /^AVAILABLE DISK SELECTIONS:/{ - searchdisks=1; +{ + if (searchdisks && $2 !~ "^$"){ + split($2,arr," "); + print arr[1]; } -EOF -#--------------------- +} - chmod 755 /tmp/find_disks.awk - disks=${@:-$(echo "" | format -e 2>/dev/null | /tmp/find_disks.awk)} - rm /tmp/find_disks.awk +/^AVAILABLE DISK SELECTIONS:/{ + searchdisks=1; +} +')} unused="" for disk in $disks; do # Check for mounted - grep "${disk}[sp]" /etc/mnttab >/dev/null - (($? == 0)) && continue + grep -q "${disk}[sp]" /etc/mnttab && continue # Check for swap - grep "${disk}[sp]" $sfi >/dev/null - (($? == 0)) && continue + grep -q "${disk}[sp]" $sfi && continue # check for dump device - grep "${disk}[sp]" $dmpi >/dev/null - (($? == 0)) && continue + grep -q "${disk}[sp]" $dmpi && continue # check to see if this disk hasn't been explicitly excluded # by a user-set environment variable - echo "${ZFS_HOST_DEVICES_IGNORE}" | grep "${disk}" > /dev/null - (($? == 0)) && continue + echo "${ZFS_HOST_DEVICES_IGNORE}" | grep -q "${disk}" && continue unused_candidates="$unused_candidates $disk" done - rm $sfi - rm $dmpi + rm $sfi $dmpi # now just check to see if those disks do actually exist # by looking for a device pointing to the first slice in @@ -2530,10 +2127,8 @@ function add_user_freebsd # # Assign 1000 as the base uid typeset -i uid=1000 while true; do - typeset -i ret pw useradd -u $uid -g $group -d $basedir/$user -m -n $user - ret=$? - case $ret in + case $? in 0) break ;; # The uid is not unique 65) ((uid += 1)) ;; @@ -2584,8 +2179,7 @@ function add_group_freebsd # typeset -i gid=1000 while true; do pw groupadd -g $gid -n $group > /dev/null 2>&1 - typeset -i ret=$? - case $ret in + case $? in 0) return 0 ;; # The gid is not unique 65) ((gid += 1)) ;; @@ -2607,8 +2201,7 @@ function del_group_freebsd # typeset group=$1 pw groupdel -n $group > /dev/null 2>&1 - typeset -i ret=$? - case $ret in + case $? in # Group does not exist, or was deleted successfully. 0|6|65) return 0 ;; # Name already exists as a group name @@ -2648,8 +2241,7 @@ function add_group_illumos # typeset -i gid=100 while true; do groupadd -g $gid $group > /dev/null 2>&1 - typeset -i ret=$? - case $ret in + case $? in 0) return 0 ;; # The gid is not unique 4) ((gid += 1)) ;; @@ -2663,8 +2255,7 @@ function del_group_illumos # typeset group=$1 groupmod -n $grp $grp > /dev/null 2>&1 - typeset -i ret=$? - case $ret in + case $? in # Group does not exist. 6) return 0 ;; # Name already exists as a group name @@ -2684,7 +2275,7 @@ function add_user_linux # # Add new users to the same group and the command line utils. # This allows them to be run out of the original users home # directory as long as it permissioned to be group readable. - cmd_group=$(stat --format="%G" $(which zfs)) + cmd_group=$(stat --format="%G" $(command -v zfs)) log_must usermod -a -G $cmd_group $user return 0 @@ -2697,8 +2288,6 @@ function del_user_linux # if id $user > /dev/null 2>&1; then log_must_retry "currently used" 6 userdel $user fi - - return 0 } function add_group_linux # @@ -2709,8 +2298,7 @@ function add_group_linux # # Linux because for many distributions 1000 and under are reserved. while true; do groupadd $group > /dev/null 2>&1 - typeset -i ret=$? - case $ret in + case $? in 0) return 0 ;; *) return 1 ;; esac @@ -2722,8 +2310,7 @@ function del_group_linux # typeset group=$1 getent group $group > /dev/null 2>&1 - typeset -i ret=$? - case $ret in + case $? in # Group does not exist. 2) return 0 ;; # Name already exists as a group name @@ -2751,7 +2338,7 @@ function add_user # log_fail "group name or user name are not defined." fi - case $(uname) in + case "$UNAME" in FreeBSD) add_user_freebsd "$group" "$user" "$basedir" ;; @@ -2781,7 +2368,7 @@ function del_user # log_fail "login name is necessary." fi - case $(uname) in + case "$UNAME" in FreeBSD) del_user_freebsd "$user" ;; @@ -2811,7 +2398,7 @@ function add_group # log_fail "group name is necessary." fi - case $(uname) in + case "$UNAME" in FreeBSD) add_group_freebsd "$group" ;; @@ -2839,7 +2426,7 @@ function del_group # log_fail "group name is necessary." fi - case $(uname) in + case "$UNAME" in FreeBSD) del_group_freebsd "$group" ;; @@ -2875,23 +2462,21 @@ function safe_to_destroy_pool { # $1 the pool name # this is a list of the top-level directories in each of the # files that make up the path to the files the pool is based on - FILEPOOL=$(zpool status -v $pool | grep /$1/ | \ - awk '{print $1}') + FILEPOOL=$(zpool status -v $pool | awk -v pool="/$1/" '$0 ~ pool {print $1}') # this is a list of the zvols that make up the pool - ZVOLPOOL=$(zpool status -v $pool | grep "$ZVOL_DEVDIR/$1$" \ - | awk '{print $1}') + ZVOLPOOL=$(zpool status -v $pool | awk -v zvols="$ZVOL_DEVDIR/$1$" '$0 ~ zvols {print $1}') # also want to determine if it's a file-based pool using an # alternate mountpoint... POOL_FILE_DIRS=$(zpool status -v $pool | \ - grep / | awk '{print $1}' | \ - awk -F/ '{print $2}' | grep -v "dev") + awk '/\// {print $1}' | \ + awk -F/ '!/dev/ {print $2}') for pooldir in $POOL_FILE_DIRS do OUTPUT=$(zfs list -H -r -o mountpoint $1 | \ - grep "${pooldir}$" | awk '{print $1}') + awk -v pd="${pooldir}$" '$0 ~ pd {print $1}') ALTMOUNTPOOL="${ALTMOUNTPOOL}${OUTPUT}" done @@ -3004,26 +2589,15 @@ function get_config { typeset pool=$1 typeset config=$2 - typeset alt_root if ! poolexists "$pool" ; then return 1 fi - alt_root=$(zpool list -H $pool | awk '{print $NF}') - if [[ $alt_root == "-" ]]; then - value=$(zdb -C $pool | grep "$config:" | awk -F: \ - '{print $2}') + if [ "$(get_pool_prop cachefile "$pool")" = "none" ]; then + zdb -e $pool else - value=$(zdb -e $pool | grep "$config:" | awk -F: \ - '{print $2}') - fi - if [[ -n $value ]] ; then - value=${value#'} - value=${value%'} - fi - echo $value - - return 0 + zdb -C $pool + fi | awk -F: -v cfg="$config:" '$0 ~ cfg {sub(/^'\''/, $2); sub(/'\''$/, $2); print $2}' } # @@ -3041,8 +2615,7 @@ function _random_get typeset -i ind ((ind = RANDOM % cnt + 1)) - typeset ret=$(echo "$str" | cut -f $ind -d ' ') - echo $ret + echo "$str" | cut -f $ind -d ' ' } # @@ -3105,20 +2678,7 @@ function datasetcksum typeset cksum sync sync_all_pools - cksum=$(zdb -vvv $1 | grep "^Dataset $1 \[" | grep "cksum" \ - | awk -F= '{print $7}') - echo $cksum -} - -# -# Get cksum of file -# #1 file path -# -function checksum -{ - typeset cksum - cksum=$(cksum $1 | awk '{print $1}') - echo $cksum + zdb -vvv $1 | awk -F= -v ds="^Dataset $1 "'\\[' '$0 ~ ds && /cksum/ {print $7}' } # @@ -3130,99 +2690,14 @@ function get_device_state #pool disk field("", "spares","logs") typeset disk=${2#$DEV_DSKDIR/} typeset field=${3:-$pool} - state=$(zpool status -v "$pool" 2>/dev/null | \ - nawk -v device=$disk -v pool=$pool -v field=$field \ + zpool status -v "$pool" 2>/dev/null | \ + awk -v device=$disk -v pool=$pool -v field=$field \ 'BEGIN {startconfig=0; startfield=0; } /config:/ {startconfig=1} (startconfig==1) && ($1==field) {startfield=1; next;} (startfield==1) && ($1==device) {print $2; exit;} (startfield==1) && - ($1==field || $1 ~ "^spares$" || $1 ~ "^logs$") {startfield=0}') - echo $state -} - - -# -# print the given directory filesystem type -# -# $1 directory name -# -function get_fstype -{ - typeset dir=$1 - - if [[ -z $dir ]]; then - log_fail "Usage: get_fstype " - fi - - # - # $ df -n / - # / : ufs - # - df -n $dir | awk '{print $3}' -} - -# -# Given a disk, label it to VTOC regardless what label was on the disk -# $1 disk -# -function labelvtoc -{ - typeset disk=$1 - if [[ -z $disk ]]; then - log_fail "The disk name is unspecified." - fi - typeset label_file=/var/tmp/labelvtoc.$$ - typeset arch=$(uname -p) - - if is_linux || is_freebsd; then - log_note "Currently unsupported by the test framework" - return 1 - fi - - if [[ $arch == "i386" ]]; then - echo "label" > $label_file - echo "0" >> $label_file - echo "" >> $label_file - echo "q" >> $label_file - echo "q" >> $label_file - - fdisk -B $disk >/dev/null 2>&1 - # wait a while for fdisk finishes - sleep 60 - elif [[ $arch == "sparc" ]]; then - echo "label" > $label_file - echo "0" >> $label_file - echo "" >> $label_file - echo "" >> $label_file - echo "" >> $label_file - echo "q" >> $label_file - else - log_fail "unknown arch type" - fi - - format -e -s -d $disk -f $label_file - typeset -i ret_val=$? - rm -f $label_file - # - # wait the format to finish - # - sleep 60 - if ((ret_val != 0)); then - log_fail "unable to label $disk as VTOC." - fi - - return 0 -} - -# -# check if the system was installed as zfsroot or not -# return: 0 if zfsroot, non-zero if not -# -function is_zfsroot -{ - df -n / | grep zfs > /dev/null 2>&1 - return $? + ($1==field || $1 ~ "^spares$" || $1 ~ "^logs$") {startfield=0}' } # @@ -3236,14 +2711,13 @@ function get_rootfs if is_freebsd; then rootfs=$(mount -p | awk '$2 == "/" && $3 == "zfs" {print $1}') elif ! is_linux; then - rootfs=$(awk '{if ($2 == "/" && $3 == "zfs") print $1}' \ + rootfs=$(awk '$2 == "/" && $3 == "zfs" {print $1}' \ /etc/mnttab) fi if [[ -z "$rootfs" ]]; then log_fail "Can not get rootfs" fi - zfs list $rootfs > /dev/null 2>&1 - if (($? == 0)); then + if datasetexists $rootfs; then echo $rootfs else log_fail "This is not a zfsroot system." @@ -3257,32 +2731,8 @@ function get_rootfs # function get_rootpool { - typeset rootfs="" - typeset rootpool="" - - if is_freebsd; then - rootfs=$(mount -p | awk '$2 == "/" && $3 == "zfs" {print $1}') - elif ! is_linux; then - rootfs=$(awk '{if ($2 == "/" && $3 =="zfs") print $1}' \ - /etc/mnttab) - fi - if [[ -z "$rootfs" ]]; then - log_fail "Can not get rootpool" - fi - zfs list $rootfs > /dev/null 2>&1 - if (($? == 0)); then - echo ${rootfs%%/*} - else - log_fail "This is not a zfsroot system." - fi -} - -# -# Get the word numbers from a string separated by white space -# -function get_word_count -{ - echo $1 | wc -w + typeset rootfs=$(get_rootfs) + echo ${rootfs%%/*} } # @@ -3292,7 +2742,7 @@ function verify_disk_count { typeset -i min=${2:-1} - typeset -i count=$(get_word_count "$1") + typeset -i count=$(echo "$1" | wc -w) if ((count < min)); then log_untested "A minimum of $min disks is required to run." \ @@ -3303,22 +2753,13 @@ function verify_disk_count function ds_is_volume { typeset type=$(get_prop type $1) - [[ $type = "volume" ]] && return 0 - return 1 + [ $type = "volume" ] } function ds_is_filesystem { typeset type=$(get_prop type $1) - [[ $type = "filesystem" ]] && return 0 - return 1 -} - -function ds_is_snapshot -{ - typeset type=$(get_prop type $1) - [[ $type = "snapshot" ]] && return 0 - return 1 + [ $type = "filesystem" ] } # @@ -3326,26 +2767,25 @@ function ds_is_snapshot # function is_te_enabled { - svcs -H -o state labeld 2>/dev/null | grep "enabled" - if (($? != 0)); then - return 1 - else - return 0 - fi + svcs -H -o state labeld 2>/dev/null | grep -q "enabled" } -# Utility function to determine if a system has multiple cpus. -function is_mp +# Return the number of CPUs (cross-platform) +function get_num_cpus { - if is_linux; then - (($(nproc) > 1)) + if is_linux ; then + grep -c '^processor' /proc/cpuinfo elif is_freebsd; then sysctl -n kern.smp.cpus else - (($(psrinfo | wc -l) > 1)) + psrinfo | wc -l fi +} - return $? +# Utility function to determine if a system has multiple cpus. +function is_mp +{ + [[ $(get_num_cpus) -gt 1 ]] } function get_cpu_freq @@ -3404,14 +2844,12 @@ function vdevs_in_pool # therefore we use the 'zpool status' output. typeset tmpfile=$(mktemp) zpool status -v "$pool" | grep -A 1000 "config:" >$tmpfile - for vdev in $@; do - grep -w ${vdev##*/} $tmpfile >/dev/null 2>&1 - [[ $? -ne 0 ]] && return 1 + for vdev in "$@"; do + grep -wq ${vdev##*/} $tmpfile || return 1 done rm -f $tmpfile - - return 0; + return 0 } function get_max @@ -3426,18 +2864,6 @@ function get_max echo $max } -function get_min -{ - typeset -l i min=$1 - shift - - for i in "$@"; do - min=$((min < i ? min : i)) - done - - echo $min -} - # Write data that can be compressed into a directory function write_compressible { @@ -3469,20 +2895,21 @@ function write_compressible done done else - log_must eval "fio \ + command -v fio > /dev/null || log_unsupported "fio missing" + log_must eval fio \ --name=job \ --fallocate=0 \ --minimal \ --randrepeat=0 \ --buffer_compress_percentage=66 \ --buffer_compress_chunk=4096 \ - --directory=$dir \ - --numjobs=$nfiles \ - --nrfiles=$nfiles \ + --directory="$dir" \ + --numjobs="$nfiles" \ + --nrfiles="$nfiles" \ --rw=write \ - --bs=$bs \ - --filesize=$megs \ - --filename_format='$fname.\$jobnum' >/dev/null" + --bs="$bs" \ + --filesize="$megs" \ + "--filename_format='$fname.\$jobnum' >/dev/null" fi } @@ -3560,24 +2987,24 @@ function wait_freeing #pool function wait_replacing #pool { typeset pool=${1:-$TESTPOOL} - while true; do - [[ "" == "$(zpool status $pool | - awk '/replacing-[0-9]+/ {print $1}')" ]] && break + while zpool status $pool | grep -qE 'replacing-[0-9]+'; do log_must sleep 1 done } -# # Wait for a pool to be scrubbed # # $1 pool name +# $2 timeout # -function wait_scrubbed +function wait_scrubbed #pool timeout { - typeset pool=${1:-$TESTPOOL} - while ! is_pool_scrubbed $pool ; do - sleep 1 - done + typeset timeout=${2:-300} + typeset pool=${1:-$TESTPOOL} + for (( timer = 0; timer < $timeout; timer++ )); do + is_pool_scrubbed $pool && break; + sleep 1; + done } # Backup the zed.rc in our test directory so that we can edit it for our test. @@ -3602,7 +3029,7 @@ function zed_rc_restore function zed_setup { if ! is_linux; then - log_unsupported "No zed on $(uname)" + log_unsupported "No zed on $UNAME" fi if [[ ! -d $ZEDLET_DIR ]]; then @@ -3651,24 +3078,26 @@ function zed_cleanup if ! is_linux; then return fi - EXTRA_ZEDLETS=$@ - log_must rm -f ${ZEDLET_DIR}/zed.rc - log_must rm -f ${ZEDLET_DIR}/zed-functions.sh - log_must rm -f ${ZEDLET_DIR}/all-syslog.sh - log_must rm -f ${ZEDLET_DIR}/all-debug.sh - log_must rm -f ${ZEDLET_DIR}/state + for extra_zedlet; do + log_must rm -f ${ZEDLET_DIR}/$extra_zedlet + done + log_must rm -fd ${ZEDLET_DIR}/zed.rc ${ZEDLET_DIR}/zed-functions.sh ${ZEDLET_DIR}/all-syslog.sh ${ZEDLET_DIR}/all-debug.sh ${ZEDLET_DIR}/state \ + $ZED_LOG $ZED_DEBUG_LOG $VDEVID_CONF_ETC $VDEVID_CONF \ + $ZEDLET_DIR +} - if [[ -n "$EXTRA_ZEDLETS" ]] ; then - for i in $EXTRA_ZEDLETS ; do - log_must rm -f ${ZEDLET_DIR}/$i - done +# +# Check if ZED is currently running; if so, returns PIDs +# +function zed_check +{ + if ! is_linux; then + return fi - log_must rm -f $ZED_LOG - log_must rm -f $ZED_DEBUG_LOG - log_must rm -f $VDEVID_CONF_ETC - log_must rm -f $VDEVID_CONF - rmdir $ZEDLET_DIR + zedpids="$(pgrep -x zed)" + zedpids2="$(pgrep -x lt-zed)" + echo ${zedpids} ${zedpids2} } # @@ -3686,9 +3115,14 @@ function zed_start fi # Verify the ZED is not already running. - pgrep -x zed > /dev/null - if (($? == 0)); then - log_note "ZED already running" + zedpids=$(zed_check) + if [ -n "$zedpids" ]; then + # We never, ever, really want it to just keep going if zed + # is already running - usually this implies our test cases + # will break very strangely because whatever we wanted to + # configure zed for won't be listening to our changes in the + # tmpdir + log_fail "ZED already running - ${zedpids}" else log_note "Starting ZED" # run ZED in the background and redirect foreground logging @@ -3707,13 +3141,13 @@ function zed_start function zed_stop { if ! is_linux; then - return + return "" fi log_note "Stopping ZED" while true; do - zedpids="$(pgrep -x zed)" - [ "$?" -ne 0 ] && break + zedpids=$(zed_check) + [ ! -n "$zedpids" ] && break log_must kill $zedpids sleep 1 @@ -3761,15 +3195,17 @@ function is_swap_inuse return 1 fi - if is_linux; then - swapon -s | grep -w $(readlink -f $device) > /dev/null 2>&1 - elif is_freebsd; then - swapctl -l | grep -w $device - else - swap -l | grep -w $device > /dev/null 2>&1 - fi - - return $? + case "$UNAME" in + Linux) + swapon -s | grep -wq $(readlink -f $device) + ;; + FreeBSD) + swapctl -l | grep -wq $device + ;; + *) + swap -l | grep -wq $device + ;; + esac } # @@ -3779,14 +3215,18 @@ function swap_setup { typeset swapdev=$1 - if is_linux; then + case "$UNAME" in + Linux) log_must eval "mkswap $swapdev > /dev/null 2>&1" log_must swapon $swapdev - elif is_freebsd; then + ;; + FreeBSD) log_must swapctl -a $swapdev - else - log_must swap -a $swapdev - fi + ;; + *) + log_must swap -a $swapdev + ;; + esac return 0 } @@ -3838,12 +3278,11 @@ function set_tunable_impl typeset name="$1" typeset value="$2" typeset mdb_cmd="$3" - typeset module="${4:-zfs}" eval "typeset tunable=\$$name" case "$tunable" in UNSUPPORTED) - log_unsupported "Tunable '$name' is unsupported on $(uname)" + log_unsupported "Tunable '$name' is unsupported on $UNAME" ;; "") log_fail "Tunable '$name' must be added to tunables.cfg" @@ -3855,21 +3294,16 @@ function set_tunable_impl [[ -z "$value" ]] && return 1 [[ -z "$mdb_cmd" ]] && return 1 - case "$(uname)" in + case "$UNAME" in Linux) - typeset zfs_tunables="/sys/module/$module/parameters" - [[ -w "$zfs_tunables/$tunable" ]] || return 1 - cat >"$zfs_tunables/$tunable" <<<"$value" - return $? + typeset zfs_tunables="/sys/module/zfs/parameters" + echo "$value" >"$zfs_tunables/$tunable" ;; FreeBSD) sysctl vfs.zfs.$tunable=$value - return "$?" ;; SunOS) - [[ "$module" -eq "zfs" ]] || return 1 echo "${tunable}/${mdb_cmd}0t${value}" | mdb -kw - return $? ;; esac } @@ -3888,25 +3322,32 @@ function get_tunable_impl { typeset name="$1" typeset module="${2:-zfs}" + typeset check_only="$3" eval "typeset tunable=\$$name" case "$tunable" in UNSUPPORTED) - log_unsupported "Tunable '$name' is unsupported on $(uname)" + if [ -z "$check_only" ] ; then + log_unsupported "Tunable '$name' is unsupported on $UNAME" + else + return 1 + fi ;; "") - log_fail "Tunable '$name' must be added to tunables.cfg" + if [ -z "$check_only" ] ; then + log_fail "Tunable '$name' must be added to tunables.cfg" + else + return 1 + fi ;; *) ;; esac - case "$(uname)" in + case "$UNAME" in Linux) typeset zfs_tunables="/sys/module/$module/parameters" - [[ -f "$zfs_tunables/$tunable" ]] || return 1 cat $zfs_tunables/$tunable - return $? ;; FreeBSD) sysctl -n vfs.zfs.$tunable @@ -3915,69 +3356,14 @@ function get_tunable_impl [[ "$module" -eq "zfs" ]] || return 1 ;; esac - - return 1 } +# Does a tunable exist? # -# Prints the current time in seconds since UNIX Epoch. -# -function current_epoch -{ - printf '%(%s)T' -} - -# -# Get decimal value of global uint32_t variable using mdb. -# -function mdb_get_uint32 -{ - typeset variable=$1 - typeset value - - value=$(mdb -k -e "$variable/X | ::eval .=U") - if [[ $? -ne 0 ]]; then - log_fail "Failed to get value of '$variable' from mdb." - return 1 - fi - - echo $value - return 0 -} - -# -# Set global uint32_t variable to a decimal value using mdb. -# -function mdb_set_uint32 +# $1: Tunable name +function tunable_exists { - typeset variable=$1 - typeset value=$2 - - mdb -kw -e "$variable/W 0t$value" > /dev/null - if [[ $? -ne 0 ]]; then - echo "Failed to set '$variable' to '$value' in mdb." - return 1 - fi - - return 0 -} - -# -# Set global scalar integer variable to a hex value using mdb. -# Note: Target should have CTF data loaded. -# -function mdb_ctf_set_int -{ - typeset variable=$1 - typeset value=$2 - - mdb -kw -e "$variable/z $value" > /dev/null - if [[ $? -ne 0 ]]; then - echo "Failed to set '$variable' to '$value' in mdb." - return 1 - fi - - return 0 + get_tunable_impl $1 "zfs" 1 } # @@ -3988,12 +3374,14 @@ function md5digest { typeset file=$1 - case $(uname) in + case "$UNAME" in FreeBSD) md5 -q $file ;; *) - md5sum -b $file | awk '{ print $1 }' + typeset sum _ + read -r sum _ < <(md5sum -b $file) + echo $sum ;; esac } @@ -4006,19 +3394,21 @@ function sha256digest { typeset file=$1 - case $(uname) in + case "$UNAME" in FreeBSD) sha256 -q $file ;; *) - sha256sum -b $file | awk '{ print $1 }' + typeset sum _ + read -r sum _ < <(sha256sum -b $file) + echo $sum ;; esac } function new_fs # { - case $(uname) in + case "$UNAME" in FreeBSD) newfs "$@" ;; @@ -4032,7 +3422,7 @@ function stat_size # { typeset path=$1 - case $(uname) in + case "$UNAME" in FreeBSD) stat -f %z "$path" ;; @@ -4046,7 +3436,7 @@ function stat_ctime # { typeset path=$1 - case $(uname) in + case "$UNAME" in FreeBSD) stat -f %c "$path" ;; @@ -4060,7 +3450,7 @@ function stat_crtime # { typeset path=$1 - case $(uname) in + case "$UNAME" in FreeBSD) stat -f %B "$path" ;; @@ -4074,7 +3464,7 @@ function stat_generation # { typeset path=$1 - case $(uname) in + case "$UNAME" in Linux) getversion "${path}" ;; @@ -4119,7 +3509,7 @@ function get_xattr # name path typeset name=$1 typeset path=$2 - case $(uname) in + case "$UNAME" in FreeBSD) getextattr -qq user "${name}" "${path}" ;; @@ -4135,7 +3525,7 @@ function set_xattr # name value path typeset value=$2 typeset path=$3 - case $(uname) in + case "$UNAME" in FreeBSD) setextattr user "${name}" "${value}" "${path}" ;; @@ -4150,7 +3540,7 @@ function set_xattr_stdin # name value typeset name=$1 typeset path=$2 - case $(uname) in + case "$UNAME" in FreeBSD) setextattr -i user "${name}" "${path}" ;; @@ -4165,7 +3555,7 @@ function rm_xattr # name path typeset name=$1 typeset path=$2 - case $(uname) in + case "$UNAME" in FreeBSD) rmextattr -q user "${name}" "${path}" ;; @@ -4179,7 +3569,7 @@ function ls_xattr # path { typeset path=$1 - case $(uname) in + case "$UNAME" in FreeBSD) lsextattr -qq user "${path}" ;; @@ -4194,14 +3584,12 @@ function kstat # stat flags? typeset stat=$1 typeset flags=${2-"-n"} - case $(uname) in + case "$UNAME" in FreeBSD) sysctl $flags kstat.zfs.misc.$stat ;; Linux) - typeset zfs_kstat="/proc/spl/kstat/zfs/$stat" - [[ -f "$zfs_kstat" ]] || return 1 - cat $zfs_kstat + cat "/proc/spl/kstat/zfs/$stat" 2>/dev/null ;; *) false @@ -4213,12 +3601,12 @@ function get_arcstat # stat { typeset stat=$1 - case $(uname) in + case "$UNAME" in FreeBSD) kstat arcstats.$stat ;; Linux) - kstat arcstats | awk "/$stat/ { print \$3 }" + kstat arcstats | awk "/$stat/"' { print $3 }' ;; *) false @@ -4232,7 +3620,7 @@ function punch_hole # offset length file typeset length=$2 typeset file=$3 - case $(uname) in + case "$UNAME" in FreeBSD) truncate -d -o $offset -l $length "$file" ;; @@ -4245,6 +3633,22 @@ function punch_hole # offset length file esac } +function zero_range # offset length file +{ + typeset offset=$1 + typeset length=$2 + typeset file=$3 + + case "$UNAME" in + Linux) + fallocate --zero-range --offset $offset --length $length "$file" + ;; + *) + false + ;; + esac +} + # # Wait for the specified arcstat to reach non-zero quiescence. # If echo is 1 echo the value after reaching quiescence, otherwise @@ -4303,3 +3707,129 @@ function wait_for_children #children done return $rv } + +# +# Compare two directory trees recursively in a manner similar to diff(1), but +# using rsync. If there are any discrepancies, a summary of the differences are +# output and a non-zero error is returned. +# +# If you're comparing a directory after a ZIL replay, you should set +# LIBTEST_DIFF_ZIL_REPLAY=1 or use replay_directory_diff which will cause +# directory_diff to ignore mtime changes (the ZIL replay won't fix up mtime +# information). +# +function directory_diff # dir_a dir_b +{ + dir_a="$1" + dir_b="$2" + zil_replay="${LIBTEST_DIFF_ZIL_REPLAY:-0}" + + # If one of the directories doesn't exist, return 2. This is to match the + # semantics of diff. + if ! [ -d "$dir_a" -a -d "$dir_b" ]; then + return 2 + fi + + # Run rsync with --dry-run --itemize-changes to get something akin to diff + # output, but rsync is far more thorough in detecting differences (diff + # doesn't compare file metadata, and cannot handle special files). + # + # Also make sure to filter out non-user.* xattrs when comparing. On + # SELinux-enabled systems the copied tree will probably have different + # SELinux labels. + args=("-nicaAHX" '--filter=-x! user.*' "--delete") + + # NOTE: Quite a few rsync builds do not support --crtimes which would be + # necessary to verify that creation times are being maintained properly. + # Unfortunately because of this we cannot use it unconditionally but we can + # check if this rsync build supports it and use it then. This check is + # based on the same check in the rsync test suite (testsuite/crtimes.test). + # + # We check ctimes even with zil_replay=1 because the ZIL does store + # creation times and we should make sure they match (if the creation times + # do not match there is a "c" entry in one of the columns). + if rsync --version | grep -q "[, ] crtimes"; then + args+=("--crtimes") + else + log_note "This rsync package does not support --crtimes (-N)." + fi + + # If we are testing a ZIL replay, we need to ignore timestamp changes. + # Unfortunately --no-times doesn't do what we want -- it will still tell + # you if the timestamps don't match but rsync will set the timestamps to + # the current time (leading to an itemised change entry). It's simpler to + # just filter out those lines. + if [ "$zil_replay" -eq 0 ]; then + filter=("cat") + else + # Different rsync versions have different numbers of columns. So just + # require that aside from the first two, all other columns must be + # blank (literal ".") or a timestamp field ("[tT]"). + filter=("grep" "-v" '^\..[.Tt]\+ ') + fi + + diff="$(rsync "${args[@]}" "$dir_a/" "$dir_b/" | "${filter[@]}")" + rv=0 + if [ -n "$diff" ]; then + echo "$diff" + rv=1 + fi + return $rv +} + +# +# Compare two directory trees recursively, without checking whether the mtimes +# match (creation times will be checked if the available rsync binary supports +# it). This is necessary for ZIL replay checks (because the ZIL does not +# contain mtimes and thus after a ZIL replay, mtimes won't match). +# +# This is shorthand for LIBTEST_DIFF_ZIL_REPLAY=1 directory_diff <...>. +# +function replay_directory_diff # dir_a dir_b +{ + LIBTEST_DIFF_ZIL_REPLAY=1 directory_diff "$@" +} + +# +# Put coredumps into $1/core.{basename} +# +# Output must be saved and passed to pop_coredump_pattern on cleanup +# +function push_coredump_pattern # dir +{ + ulimit -c unlimited + case "$UNAME" in + Linux) + cat /proc/sys/kernel/core_pattern /proc/sys/kernel/core_uses_pid + echo "$1/core.%e" >/proc/sys/kernel/core_pattern && + echo 0 >/proc/sys/kernel/core_uses_pid + ;; + FreeBSD) + sysctl -n kern.corefile + sysctl kern.corefile="$1/core.%N" >/dev/null + ;; + *) + # Nothing to output – set only for this shell + coreadm -p "$1/core.%f" + ;; + esac +} + +# +# Put coredumps back into the default location +# +function pop_coredump_pattern +{ + [ -s "$1" ] || return 0 + case "$UNAME" in + Linux) + typeset pat pid + { read -r pat; read -r pid; } < "$1" + echo "$pat" >/proc/sys/kernel/core_pattern && + echo "$pid" >/proc/sys/kernel/core_uses_pid + ;; + FreeBSD) + sysctl kern.corefile="$(<"$1")" >/dev/null + ;; + esac +} diff --git a/tests/zfs-tests/include/math.shlib b/tests/zfs-tests/include/math.shlib index 7ac59f279604..38d9fecea7cf 100644 --- a/tests/zfs-tests/include/math.shlib +++ b/tests/zfs-tests/include/math.shlib @@ -30,17 +30,14 @@ function within_percent typeset percent=$3 # Set $a or $b to $2 such that a >= b - [[ '1' = $(echo "if ($2 > $a) 1 else 0" | bc) ]] && a=$2 || b=$2 + [ 1 -eq $(echo "$2 > $a" | bc) ] && a=$2 || b=$2 # Prevent division by 0 [[ $a =~ [1-9] ]] || return 1 typeset p=$(echo "scale=2; $b * 100 / $a" | bc) log_note "Comparing $a and $b given $percent% (calculated: $p%)" - [[ '1' = $(echo "scale=2; if ($p >= $percent) 1 else 0" | bc) ]] && \ - return 0 - - return 1 + [ 1 -eq $(echo "scale=2; $p >= $percent" | bc) ] } # @@ -61,9 +58,7 @@ function within_tolerance #value #target #tolerance typeset diff=$((abs(val - target))) log_note "Checking if $val is within +/-$tol of $target (diff: $diff)" - ((diff <= tol)) && return 0 - - return 1 + ((diff <= tol)) } # diff --git a/tests/zfs-tests/include/properties.shlib b/tests/zfs-tests/include/properties.shlib index 6d467b60051d..14b3f4415b7d 100644 --- a/tests/zfs-tests/include/properties.shlib +++ b/tests/zfs-tests/include/properties.shlib @@ -17,7 +17,7 @@ typeset -a compress_prop_vals=('off' 'lzjb' 'lz4' 'gzip' 'zle' 'zstd') typeset -a checksum_prop_vals=('on' 'off' 'fletcher2' 'fletcher4' 'sha256' - 'noparity' 'sha512' 'skein') + 'noparity' 'sha512' 'skein' 'blake3') if ! is_freebsd; then checksum_prop_vals+=('edonr') fi @@ -69,26 +69,6 @@ function get_rand_prop_vals echo $retstr } -function get_rand_checksum -{ - get_rand_prop_vals checksum $1 2 -} - -function get_rand_checksum_any -{ - get_rand_prop_vals checksum $1 0 -} - -function get_rand_recsize -{ - get_rand_prop_vals recsize $1 0 -} - -function get_rand_large_recsize -{ - get_rand_prop_vals recsize $1 9 -} - # # Functions to toggle on/off properties # @@ -100,12 +80,8 @@ else binary_props+=('zoned') fi -if is_linux; then - # Only older kernels support non-blocking mandatory locks - if [[ $(linux_version) -lt $(linux_version "4.4") ]]; then - binary_props+=('nbmand') - fi -else +# Newer Linuxes dropped non-blocking mandatory locks +if ! is_linux || [ $(linux_version) -lt $(linux_version "4.4") ]; then binary_props+=('nbmand') fi @@ -114,7 +90,6 @@ function toggle_prop typeset ds=$1 typeset prop=$2 - datasetexists $ds || log_fail "$ds does not exist" typeset val=$(get_prop $prop $ds) typeset newval='off' @@ -137,7 +112,6 @@ function randomize_ds_props typeset ds=$1 typeset prop proplist val - datasetexists $ds || log_fail "$ds does not exist" if ds_is_volume $ds; then toggle_prop $ds readonly proplist="${vol_props[@]}" diff --git a/tests/zfs-tests/include/tunables.cfg b/tests/zfs-tests/include/tunables.cfg index fff43e469165..d6a2fe5db7c6 100644 --- a/tests/zfs-tests/include/tunables.cfg +++ b/tests/zfs-tests/include/tunables.cfg @@ -87,9 +87,12 @@ VDEV_VALIDATE_SKIP vdev.validate_skip vdev_validate_skip VOL_INHIBIT_DEV UNSUPPORTED zvol_inhibit_dev VOL_MODE vol.mode zvol_volmode VOL_RECURSIVE vol.recursive UNSUPPORTED +VOL_USE_BLK_MQ UNSUPPORTED zvol_use_blk_mq +XATTR_COMPAT xattr_compat zfs_xattr_compat ZEVENT_LEN_MAX zevent.len_max zfs_zevent_len_max ZEVENT_RETAIN_MAX zevent.retain_max zfs_zevent_retain_max ZIO_SLOW_IO_MS zio.slow_io_ms zio_slow_io_ms +ZIL_SAXATTR zil_saxattr zfs_zil_saxattr %%%% while read name FreeBSD Linux; do eval "export ${name}=\$${UNAME}" diff --git a/tests/zfs-tests/include/zpool_script.shlib b/tests/zfs-tests/include/zpool_script.shlib index 10bc0cc26128..cbf1e07803d4 100644 --- a/tests/zfs-tests/include/zpool_script.shlib +++ b/tests/zfs-tests/include/zpool_script.shlib @@ -15,7 +15,7 @@ function test_zpool_script { out="$($wholecmd)" # Default number of columns that get printed without -c - if echo "$cmd" | grep -q iostat ; then + if [ "$cmd" != "${cmd/iostat/_/}" ]; then # iostat dcols=7 else @@ -39,9 +39,9 @@ function test_zpool_script { # zpool iostat -v output is 7 columns, so if the script ran correctly # we should see more than that. if ! newcols=$(echo "$out" | \ - awk '/\/dev/{print NF-'$dcols'; if (NF <= '$dcols') {exit 1}}' | \ - head -n 1) ; \ - then + awk '/\/dev/ {print NF-'$dcols'; if (NF <= '$dcols') {exit 1}}' | \ + head -n 1) + then log_fail "'$wholecmd' didn't create a new column value" else log_note "'$wholecmd' passed ($newcols new columns)" diff --git a/tests/zfs-tests/tests/Makefile.am b/tests/zfs-tests/tests/Makefile.am index f7494791524e..b13f66dc3e1b 100644 --- a/tests/zfs-tests/tests/Makefile.am +++ b/tests/zfs-tests/tests/Makefile.am @@ -1 +1,1992 @@ -SUBDIRS = functional perf stress +CLEANFILES = +dist_noinst_DATA = +include $(top_srcdir)/config/Substfiles.am + + +datadir_zfs_tests_testsdir = $(datadir)/$(PACKAGE)/zfs-tests/tests +nobase_dist_datadir_zfs_tests_tests_DATA = \ + perf/nfs-sample.cfg \ + perf/perf.shlib \ + \ + perf/fio/mkfiles.fio \ + perf/fio/random_reads.fio \ + perf/fio/random_readwrite.fio \ + perf/fio/random_readwrite_fixed.fio \ + perf/fio/random_writes.fio \ + perf/fio/sequential_reads.fio \ + perf/fio/sequential_readwrite.fio \ + perf/fio/sequential_writes.fio + +nobase_dist_datadir_zfs_tests_tests_SCRIPTS = \ + perf/regression/random_reads.ksh \ + perf/regression/random_readwrite.ksh \ + perf/regression/random_readwrite_fixed.ksh \ + perf/regression/random_writes.ksh \ + perf/regression/random_writes_zil.ksh \ + perf/regression/sequential_reads_arc_cached_clone.ksh \ + perf/regression/sequential_reads_arc_cached.ksh \ + perf/regression/sequential_reads_dbuf_cached.ksh \ + perf/regression/sequential_reads.ksh \ + perf/regression/sequential_writes.ksh \ + perf/regression/setup.ksh \ + \ + perf/scripts/prefetch_io.sh + +# These lists can be regenerated by running make regen-tests at the root, or, on a *clean* source: +# find functional/ ! -type d ! -name .gitignore ! -name .dirstamp ! -name '*.Po' ! -executable -name '*.in' | sort | sed 's/\.in$//;s/^/\t/;$!s/$/ \\/' +# find functional/ ! -type d ! -name .gitignore ! -name .dirstamp ! -name '*.Po' -executable -name '*.in' | sort | sed 's/\.in$//;s/^/\t/;$!s/$/ \\/' +# find functional/ ! -type d ! -name .gitignore ! -name .dirstamp ! -name '*.Po' ! -name '*.in' ! -name '*.c' | grep -Fe /simd -e /tmpfile | sort | sed 's/^/\t/;$!s/$/ \\/' +# find functional/ ! -type d ! -name .gitignore ! -name .dirstamp ! -name '*.Po' ! -executable ! -name '*.in' ! -name '*.c' | grep -vFe /simd -e /tmpfile | sort | sed 's/^/\t/;$!s/$/ \\/' +# find functional/ ! -type d ! -name .gitignore ! -name .dirstamp ! -name '*.Po' -executable ! -name '*.in' ! -name '*.c' | grep -vFe /simd -e /tmpfile | sort | sed 's/^/\t/;$!s/$/ \\/' +# +# simd and tmpfile are Linux-only and not installed elsewhere +# +# C programs are specced in ../Makefile.am above as part of the main Makefile + +find_common := find functional/ ! -type d ! -name .gitignore ! -name .dirstamp ! -name '*.Po' +regen: + @$(MAKE) -C $(top_builddir) clean + @$(MAKE) clean + $(SED) $(ac_inplace) '/^# -- >8 --/q' Makefile.am + echo >> Makefile.am + echo 'nobase_nodist_datadir_zfs_tests_tests_DATA = \' >> Makefile.am + $(find_common) ! -executable -name '*.in' | sort | sed 's/\.in$$//;s/^/\t/;$$!s/$$/ \\/' >> Makefile.am + echo 'nobase_nodist_datadir_zfs_tests_tests_SCRIPTS = \' >> Makefile.am + $(find_common) -executable -name '*.in' | sort | sed 's/\.in$$//;s/^/\t/;$$!s/$$/ \\/' >> Makefile.am + echo >> Makefile.am + echo 'SUBSTFILES += $$(nobase_nodist_datadir_zfs_tests_tests_DATA) $$(nobase_nodist_datadir_zfs_tests_tests_SCRIPTS)' >> Makefile.am + echo >> Makefile.am + echo 'if BUILD_LINUX' >> Makefile.am + echo 'nobase_dist_datadir_zfs_tests_tests_SCRIPTS += \' >> Makefile.am + $(find_common) ! -name '*.in' ! -name '*.c' | grep -Fe /simd -e /tmpfile | sort | sed 's/^/\t/;$$!s/$$/ \\/' >> Makefile.am + echo 'endif' >> Makefile.am + echo >> Makefile.am + echo 'nobase_dist_datadir_zfs_tests_tests_DATA += \' >> Makefile.am + $(find_common) ! -executable ! -name '*.in' ! -name '*.c' | grep -vFe /simd -e /tmpfile | sort | sed 's/^/\t/;$$!s/$$/ \\/' >> Makefile.am + echo >> Makefile.am + echo 'nobase_dist_datadir_zfs_tests_tests_SCRIPTS += \' >> Makefile.am + $(find_common) -executable ! -name '*.in' ! -name '*.c' | grep -vFe /simd -e /tmpfile | sort | sed 's/^/\t/;$$!s/$$/ \\/' >> Makefile.am + +# -- >8 -- + +nobase_nodist_datadir_zfs_tests_tests_DATA = \ + functional/pam/utilities.kshlib +nobase_nodist_datadir_zfs_tests_tests_SCRIPTS = \ + functional/pyzfs/pyzfs_unittest.ksh + +SUBSTFILES += $(nobase_nodist_datadir_zfs_tests_tests_DATA) $(nobase_nodist_datadir_zfs_tests_tests_SCRIPTS) + +if BUILD_LINUX +nobase_dist_datadir_zfs_tests_tests_SCRIPTS += \ + functional/simd/simd_supported.ksh \ + functional/tmpfile/cleanup.ksh \ + functional/tmpfile/setup.ksh +endif + +nobase_dist_datadir_zfs_tests_tests_DATA += \ + functional/acl/acl.cfg \ + functional/acl/acl_common.kshlib \ + functional/alloc_class/alloc_class.cfg \ + functional/alloc_class/alloc_class.kshlib \ + functional/atime/atime.cfg \ + functional/atime/atime_common.kshlib \ + functional/cache/cache.cfg \ + functional/cache/cache.kshlib \ + functional/cachefile/cachefile.cfg \ + functional/cachefile/cachefile.kshlib \ + functional/casenorm/casenorm.cfg \ + functional/casenorm/casenorm.kshlib \ + functional/channel_program/channel_common.kshlib \ + functional/channel_program/lua_core/tst.args_to_lua.out \ + functional/channel_program/lua_core/tst.args_to_lua.zcp \ + functional/channel_program/lua_core/tst.divide_by_zero.err \ + functional/channel_program/lua_core/tst.divide_by_zero.zcp \ + functional/channel_program/lua_core/tst.exists.zcp \ + functional/channel_program/lua_core/tst.large_prog.out \ + functional/channel_program/lua_core/tst.large_prog.zcp \ + functional/channel_program/lua_core/tst.lib_base.lua \ + functional/channel_program/lua_core/tst.lib_coroutine.lua \ + functional/channel_program/lua_core/tst.lib_strings.lua \ + functional/channel_program/lua_core/tst.lib_table.lua \ + functional/channel_program/lua_core/tst.nested_neg.zcp \ + functional/channel_program/lua_core/tst.nested_pos.zcp \ + functional/channel_program/lua_core/tst.recursive.zcp \ + functional/channel_program/lua_core/tst.return_large.zcp \ + functional/channel_program/lua_core/tst.return_recursive_table.zcp \ + functional/channel_program/lua_core/tst.stack_gsub.err \ + functional/channel_program/lua_core/tst.stack_gsub.zcp \ + functional/channel_program/lua_core/tst.timeout.zcp \ + functional/channel_program/synctask_core/tst.bookmark.copy.zcp \ + functional/channel_program/synctask_core/tst.bookmark.create.zcp \ + functional/channel_program/synctask_core/tst.get_index_props.out \ + functional/channel_program/synctask_core/tst.get_index_props.zcp \ + functional/channel_program/synctask_core/tst.get_number_props.out \ + functional/channel_program/synctask_core/tst.get_number_props.zcp \ + functional/channel_program/synctask_core/tst.get_string_props.out \ + functional/channel_program/synctask_core/tst.get_string_props.zcp \ + functional/channel_program/synctask_core/tst.promote_conflict.zcp \ + functional/channel_program/synctask_core/tst.set_props.zcp \ + functional/channel_program/synctask_core/tst.snapshot_destroy.zcp \ + functional/channel_program/synctask_core/tst.snapshot_neg.zcp \ + functional/channel_program/synctask_core/tst.snapshot_recursive.zcp \ + functional/channel_program/synctask_core/tst.snapshot_simple.zcp \ + functional/checksum/default.cfg \ + functional/clean_mirror/clean_mirror_common.kshlib \ + functional/clean_mirror/default.cfg \ + functional/cli_root/cli_common.kshlib \ + functional/cli_root/zfs_copies/zfs_copies.cfg \ + functional/cli_root/zfs_copies/zfs_copies.kshlib \ + functional/cli_root/zfs_create/properties.kshlib \ + functional/cli_root/zfs_create/zfs_create.cfg \ + functional/cli_root/zfs_create/zfs_create_common.kshlib \ + functional/cli_root/zfs_destroy/zfs_destroy.cfg \ + functional/cli_root/zfs_destroy/zfs_destroy_common.kshlib \ + functional/cli_root/zfs_get/zfs_get_common.kshlib \ + functional/cli_root/zfs_get/zfs_get_list_d.kshlib \ + functional/cli_root/zfs_jail/jail.conf \ + functional/cli_root/zfs_load-key/HEXKEY \ + functional/cli_root/zfs_load-key/PASSPHRASE \ + functional/cli_root/zfs_load-key/RAWKEY \ + functional/cli_root/zfs_load-key/zfs_load-key.cfg \ + functional/cli_root/zfs_load-key/zfs_load-key_common.kshlib \ + functional/cli_root/zfs_mount/zfs_mount.cfg \ + functional/cli_root/zfs_mount/zfs_mount.kshlib \ + functional/cli_root/zfs_promote/zfs_promote.cfg \ + functional/cli_root/zfs_receive/zstd_test_data.txt \ + functional/cli_root/zfs_rename/zfs_rename.cfg \ + functional/cli_root/zfs_rename/zfs_rename.kshlib \ + functional/cli_root/zfs_rollback/zfs_rollback.cfg \ + functional/cli_root/zfs_rollback/zfs_rollback_common.kshlib \ + functional/cli_root/zfs_send/zfs_send.cfg \ + functional/cli_root/zfs_set/zfs_set_common.kshlib \ + functional/cli_root/zfs_share/zfs_share.cfg \ + functional/cli_root/zfs_snapshot/zfs_snapshot.cfg \ + functional/cli_root/zfs_unmount/zfs_unmount.cfg \ + functional/cli_root/zfs_unmount/zfs_unmount.kshlib \ + functional/cli_root/zfs_upgrade/zfs_upgrade.kshlib \ + functional/cli_root/zfs_wait/zfs_wait.kshlib \ + functional/cli_root/zpool_add/zpool_add.cfg \ + functional/cli_root/zpool_add/zpool_add.kshlib \ + functional/cli_root/zpool_clear/zpool_clear.cfg \ + functional/cli_root/zpool_create/draidcfg.gz \ + functional/cli_root/zpool_create/zpool_create.cfg \ + functional/cli_root/zpool_create/zpool_create.shlib \ + functional/cli_root/zpool_destroy/zpool_destroy.cfg \ + functional/cli_root/zpool_events/zpool_events.cfg \ + functional/cli_root/zpool_events/zpool_events.kshlib \ + functional/cli_root/zpool_expand/zpool_expand.cfg \ + functional/cli_root/zpool_export/zpool_export.cfg \ + functional/cli_root/zpool_export/zpool_export.kshlib \ + functional/cli_root/zpool_get/zpool_get.cfg \ + functional/cli_root/zpool_get/zpool_get_parsable.cfg \ + functional/cli_root/zpool_import/blockfiles/cryptv0.dat.bz2 \ + functional/cli_root/zpool_import/blockfiles/missing_ivset.dat.bz2 \ + functional/cli_root/zpool_import/blockfiles/unclean_export.dat.bz2 \ + functional/cli_root/zpool_import/zpool_import.cfg \ + functional/cli_root/zpool_import/zpool_import.kshlib \ + functional/cli_root/zpool_initialize/zpool_initialize.kshlib \ + functional/cli_root/zpool_labelclear/labelclear.cfg \ + functional/cli_root/zpool_remove/zpool_remove.cfg \ + functional/cli_root/zpool_reopen/zpool_reopen.cfg \ + functional/cli_root/zpool_reopen/zpool_reopen.shlib \ + functional/cli_root/zpool_resilver/zpool_resilver.cfg \ + functional/cli_root/zpool_scrub/zpool_scrub.cfg \ + functional/cli_root/zpool_split/zpool_split.cfg \ + functional/cli_root/zpool_trim/zpool_trim.kshlib \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-broken-mirror1.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-broken-mirror2.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v10.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v11.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v12.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v13.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v14.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v15.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v1.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v1mirror1.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v1mirror2.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v1mirror3.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v1raidz1.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v1raidz2.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v1raidz3.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v1stripe1.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v1stripe2.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v1stripe3.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v2.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v2mirror1.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v2mirror2.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v2mirror3.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v2raidz1.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v2raidz2.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v2raidz3.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v2stripe1.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v2stripe2.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v2stripe3.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3hotspare1.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3hotspare2.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3hotspare3.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3mirror1.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3mirror2.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3mirror3.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3raidz1.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3raidz21.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3raidz22.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3raidz23.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3raidz2.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3raidz3.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3stripe1.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3stripe2.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v3stripe3.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v4.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v5.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v6.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v7.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v8.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v999.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-v9.dat.bz2 \ + functional/cli_root/zpool_upgrade/blockfiles/zfs-pool-vBROKEN.dat.bz2 \ + functional/cli_root/zpool_upgrade/zpool_upgrade.cfg \ + functional/cli_root/zpool_upgrade/zpool_upgrade.kshlib \ + functional/cli_root/zpool_wait/zpool_wait.kshlib \ + functional/cli_user/misc/misc.cfg \ + functional/cli_user/zfs_list/zfs_list.cfg \ + functional/cli_user/zfs_list/zfs_list.kshlib \ + functional/compression/compress.cfg \ + functional/compression/testpool_zstd.tar.gz \ + functional/deadman/deadman.cfg \ + functional/delegate/delegate.cfg \ + functional/delegate/delegate_common.kshlib \ + functional/devices/devices.cfg \ + functional/devices/devices_common.kshlib \ + functional/events/events.cfg \ + functional/events/events_common.kshlib \ + functional/fault/fault.cfg \ + functional/grow/grow.cfg \ + functional/history/history.cfg \ + functional/history/history_common.kshlib \ + functional/history/i386.migratedpool.DAT.Z \ + functional/history/i386.orig_history.txt \ + functional/history/sparc.migratedpool.DAT.Z \ + functional/history/sparc.orig_history.txt \ + functional/history/zfs-pool-v4.dat.Z \ + functional/inheritance/config001.cfg \ + functional/inheritance/config002.cfg \ + functional/inheritance/config003.cfg \ + functional/inheritance/config004.cfg \ + functional/inheritance/config005.cfg \ + functional/inheritance/config006.cfg \ + functional/inheritance/config007.cfg \ + functional/inheritance/config008.cfg \ + functional/inheritance/config009.cfg \ + functional/inheritance/config010.cfg \ + functional/inheritance/config011.cfg \ + functional/inheritance/config012.cfg \ + functional/inheritance/config013.cfg \ + functional/inheritance/config014.cfg \ + functional/inheritance/config015.cfg \ + functional/inheritance/config016.cfg \ + functional/inheritance/config017.cfg \ + functional/inheritance/config018.cfg \ + functional/inheritance/config019.cfg \ + functional/inheritance/config020.cfg \ + functional/inheritance/config021.cfg \ + functional/inheritance/config022.cfg \ + functional/inheritance/config023.cfg \ + functional/inheritance/config024.cfg \ + functional/inheritance/inherit.kshlib \ + functional/inheritance/README.config \ + functional/inheritance/README.state \ + functional/inheritance/state001.cfg \ + functional/inheritance/state002.cfg \ + functional/inheritance/state003.cfg \ + functional/inheritance/state004.cfg \ + functional/inheritance/state005.cfg \ + functional/inheritance/state006.cfg \ + functional/inheritance/state007.cfg \ + functional/inheritance/state008.cfg \ + functional/inheritance/state009.cfg \ + functional/inheritance/state010.cfg \ + functional/inheritance/state011.cfg \ + functional/inheritance/state012.cfg \ + functional/inheritance/state013.cfg \ + functional/inheritance/state014.cfg \ + functional/inheritance/state015.cfg \ + functional/inheritance/state016.cfg \ + functional/inheritance/state017.cfg \ + functional/inheritance/state018.cfg \ + functional/inheritance/state019.cfg \ + functional/inheritance/state020.cfg \ + functional/inheritance/state021.cfg \ + functional/inheritance/state022.cfg \ + functional/inheritance/state023.cfg \ + functional/inheritance/state024.cfg \ + functional/inuse/inuse.cfg \ + functional/io/io.cfg \ + functional/l2arc/l2arc.cfg \ + functional/largest_pool/largest_pool.cfg \ + functional/migration/migration.cfg \ + functional/migration/migration.kshlib \ + functional/mmap/mmap.cfg \ + functional/mmp/mmp.cfg \ + functional/mmp/mmp.kshlib \ + functional/mv_files/mv_files.cfg \ + functional/mv_files/mv_files_common.kshlib \ + functional/nopwrite/nopwrite.shlib \ + functional/no_space/enospc.cfg \ + functional/online_offline/online_offline.cfg \ + functional/pool_checkpoint/pool_checkpoint.kshlib \ + functional/projectquota/projectquota.cfg \ + functional/projectquota/projectquota_common.kshlib \ + functional/quota/quota.cfg \ + functional/quota/quota.kshlib \ + functional/redacted_send/redacted.cfg \ + functional/redacted_send/redacted.kshlib \ + functional/redundancy/redundancy.cfg \ + functional/redundancy/redundancy.kshlib \ + functional/refreserv/refreserv.cfg \ + functional/removal/removal.kshlib \ + functional/replacement/replacement.cfg \ + functional/reservation/reservation.cfg \ + functional/reservation/reservation.shlib \ + functional/rsend/dedup_encrypted_zvol.bz2 \ + functional/rsend/dedup_encrypted_zvol.zsend.bz2 \ + functional/rsend/dedup.zsend.bz2 \ + functional/rsend/fs.tar.gz \ + functional/rsend/rsend.cfg \ + functional/rsend/rsend.kshlib \ + functional/scrub_mirror/default.cfg \ + functional/scrub_mirror/scrub_mirror_common.kshlib \ + functional/slog/slog.cfg \ + functional/slog/slog.kshlib \ + functional/snapshot/snapshot.cfg \ + functional/snapused/snapused.kshlib \ + functional/sparse/sparse.cfg \ + functional/trim/trim.cfg \ + functional/trim/trim.kshlib \ + functional/truncate/truncate.cfg \ + functional/upgrade/upgrade_common.kshlib \ + functional/user_namespace/user_namespace.cfg \ + functional/user_namespace/user_namespace_common.kshlib \ + functional/userquota/userquota.cfg \ + functional/userquota/userquota_common.kshlib \ + functional/vdev_zaps/vdev_zaps.kshlib \ + functional/xattr/xattr.cfg \ + functional/xattr/xattr_common.kshlib \ + functional/zvol/zvol.cfg \ + functional/zvol/zvol_cli/zvol_cli.cfg \ + functional/zvol/zvol_common.shlib \ + functional/zvol/zvol_ENOSPC/zvol_ENOSPC.cfg \ + functional/zvol/zvol_misc/zvol_misc_common.kshlib \ + functional/zvol/zvol_swap/zvol_swap.cfg + +nobase_dist_datadir_zfs_tests_tests_SCRIPTS += \ + functional/acl/off/cleanup.ksh \ + functional/acl/off/dosmode.ksh \ + functional/acl/off/posixmode.ksh \ + functional/acl/off/setup.ksh \ + functional/acl/posix/cleanup.ksh \ + functional/acl/posix/posix_001_pos.ksh \ + functional/acl/posix/posix_002_pos.ksh \ + functional/acl/posix/posix_003_pos.ksh \ + functional/acl/posix/posix_004_pos.ksh \ + functional/acl/posix-sa/cleanup.ksh \ + functional/acl/posix-sa/posix_001_pos.ksh \ + functional/acl/posix-sa/posix_002_pos.ksh \ + functional/acl/posix-sa/posix_003_pos.ksh \ + functional/acl/posix-sa/posix_004_pos.ksh \ + functional/acl/posix-sa/setup.ksh \ + functional/acl/posix/setup.ksh \ + functional/alloc_class/alloc_class_001_pos.ksh \ + functional/alloc_class/alloc_class_002_neg.ksh \ + functional/alloc_class/alloc_class_003_pos.ksh \ + functional/alloc_class/alloc_class_004_pos.ksh \ + functional/alloc_class/alloc_class_005_pos.ksh \ + functional/alloc_class/alloc_class_006_pos.ksh \ + functional/alloc_class/alloc_class_007_pos.ksh \ + functional/alloc_class/alloc_class_008_pos.ksh \ + functional/alloc_class/alloc_class_009_pos.ksh \ + functional/alloc_class/alloc_class_010_pos.ksh \ + functional/alloc_class/alloc_class_011_neg.ksh \ + functional/alloc_class/alloc_class_012_pos.ksh \ + functional/alloc_class/alloc_class_013_pos.ksh \ + functional/alloc_class/cleanup.ksh \ + functional/alloc_class/setup.ksh \ + functional/append/file_append.ksh \ + functional/append/threadsappend_001_pos.ksh \ + functional/append/cleanup.ksh \ + functional/append/setup.ksh \ + functional/arc/arcstats_runtime_tuning.ksh \ + functional/arc/cleanup.ksh \ + functional/arc/dbufstats_001_pos.ksh \ + functional/arc/dbufstats_002_pos.ksh \ + functional/arc/dbufstats_003_pos.ksh \ + functional/arc/setup.ksh \ + functional/atime/atime_001_pos.ksh \ + functional/atime/atime_002_neg.ksh \ + functional/atime/atime_003_pos.ksh \ + functional/atime/cleanup.ksh \ + functional/atime/root_atime_off.ksh \ + functional/atime/root_atime_on.ksh \ + functional/atime/root_relatime_on.ksh \ + functional/atime/setup.ksh \ + functional/bootfs/bootfs_001_pos.ksh \ + functional/bootfs/bootfs_002_neg.ksh \ + functional/bootfs/bootfs_003_pos.ksh \ + functional/bootfs/bootfs_004_neg.ksh \ + functional/bootfs/bootfs_005_neg.ksh \ + functional/bootfs/bootfs_006_pos.ksh \ + functional/bootfs/bootfs_007_pos.ksh \ + functional/bootfs/bootfs_008_pos.ksh \ + functional/bootfs/cleanup.ksh \ + functional/bootfs/setup.ksh \ + functional/btree/btree_negative.ksh \ + functional/btree/btree_positive.ksh \ + functional/cache/cache_001_pos.ksh \ + functional/cache/cache_002_pos.ksh \ + functional/cache/cache_003_pos.ksh \ + functional/cache/cache_004_neg.ksh \ + functional/cache/cache_005_neg.ksh \ + functional/cache/cache_006_pos.ksh \ + functional/cache/cache_007_neg.ksh \ + functional/cache/cache_008_neg.ksh \ + functional/cache/cache_009_pos.ksh \ + functional/cache/cache_010_pos.ksh \ + functional/cache/cache_011_pos.ksh \ + functional/cache/cache_012_pos.ksh \ + functional/cache/cleanup.ksh \ + functional/cachefile/cachefile_001_pos.ksh \ + functional/cachefile/cachefile_002_pos.ksh \ + functional/cachefile/cachefile_003_pos.ksh \ + functional/cachefile/cachefile_004_pos.ksh \ + functional/cachefile/cleanup.ksh \ + functional/cachefile/setup.ksh \ + functional/cache/setup.ksh \ + functional/casenorm/case_all_values.ksh \ + functional/casenorm/cleanup.ksh \ + functional/casenorm/insensitive_formd_delete.ksh \ + functional/casenorm/insensitive_formd_lookup.ksh \ + functional/casenorm/insensitive_none_delete.ksh \ + functional/casenorm/insensitive_none_lookup.ksh \ + functional/casenorm/mixed_create_failure.ksh \ + functional/casenorm/mixed_formd_delete.ksh \ + functional/casenorm/mixed_formd_lookup_ci.ksh \ + functional/casenorm/mixed_formd_lookup.ksh \ + functional/casenorm/mixed_none_delete.ksh \ + functional/casenorm/mixed_none_lookup_ci.ksh \ + functional/casenorm/mixed_none_lookup.ksh \ + functional/casenorm/norm_all_values.ksh \ + functional/casenorm/sensitive_formd_delete.ksh \ + functional/casenorm/sensitive_formd_lookup.ksh \ + functional/casenorm/sensitive_none_delete.ksh \ + functional/casenorm/sensitive_none_lookup.ksh \ + functional/casenorm/setup.ksh \ + functional/channel_program/lua_core/cleanup.ksh \ + functional/channel_program/lua_core/setup.ksh \ + functional/channel_program/lua_core/tst.args_to_lua.ksh \ + functional/channel_program/lua_core/tst.divide_by_zero.ksh \ + functional/channel_program/lua_core/tst.exists.ksh \ + functional/channel_program/lua_core/tst.integer_illegal.ksh \ + functional/channel_program/lua_core/tst.integer_overflow.ksh \ + functional/channel_program/lua_core/tst.language_functions_neg.ksh \ + functional/channel_program/lua_core/tst.language_functions_pos.ksh \ + functional/channel_program/lua_core/tst.large_prog.ksh \ + functional/channel_program/lua_core/tst.libraries.ksh \ + functional/channel_program/lua_core/tst.memory_limit.ksh \ + functional/channel_program/lua_core/tst.nested_neg.ksh \ + functional/channel_program/lua_core/tst.nested_pos.ksh \ + functional/channel_program/lua_core/tst.nvlist_to_lua.ksh \ + functional/channel_program/lua_core/tst.recursive_neg.ksh \ + functional/channel_program/lua_core/tst.recursive_pos.ksh \ + functional/channel_program/lua_core/tst.return_large.ksh \ + functional/channel_program/lua_core/tst.return_nvlist_neg.ksh \ + functional/channel_program/lua_core/tst.return_nvlist_pos.ksh \ + functional/channel_program/lua_core/tst.return_recursive_table.ksh \ + functional/channel_program/lua_core/tst.stack_gsub.ksh \ + functional/channel_program/lua_core/tst.timeout.ksh \ + functional/channel_program/synctask_core/cleanup.ksh \ + functional/channel_program/synctask_core/setup.ksh \ + functional/channel_program/synctask_core/tst.bookmark.copy.ksh \ + functional/channel_program/synctask_core/tst.bookmark.create.ksh \ + functional/channel_program/synctask_core/tst.destroy_fs.ksh \ + functional/channel_program/synctask_core/tst.destroy_snap.ksh \ + functional/channel_program/synctask_core/tst.get_count_and_limit.ksh \ + functional/channel_program/synctask_core/tst.get_index_props.ksh \ + functional/channel_program/synctask_core/tst.get_mountpoint.ksh \ + functional/channel_program/synctask_core/tst.get_neg.ksh \ + functional/channel_program/synctask_core/tst.get_number_props.ksh \ + functional/channel_program/synctask_core/tst.get_string_props.ksh \ + functional/channel_program/synctask_core/tst.get_type.ksh \ + functional/channel_program/synctask_core/tst.get_userquota.ksh \ + functional/channel_program/synctask_core/tst.get_written.ksh \ + functional/channel_program/synctask_core/tst.inherit.ksh \ + functional/channel_program/synctask_core/tst.list_bookmarks.ksh \ + functional/channel_program/synctask_core/tst.list_children.ksh \ + functional/channel_program/synctask_core/tst.list_clones.ksh \ + functional/channel_program/synctask_core/tst.list_holds.ksh \ + functional/channel_program/synctask_core/tst.list_snapshots.ksh \ + functional/channel_program/synctask_core/tst.list_system_props.ksh \ + functional/channel_program/synctask_core/tst.list_user_props.ksh \ + functional/channel_program/synctask_core/tst.parse_args_neg.ksh \ + functional/channel_program/synctask_core/tst.promote_conflict.ksh \ + functional/channel_program/synctask_core/tst.promote_multiple.ksh \ + functional/channel_program/synctask_core/tst.promote_simple.ksh \ + functional/channel_program/synctask_core/tst.rollback_mult.ksh \ + functional/channel_program/synctask_core/tst.rollback_one.ksh \ + functional/channel_program/synctask_core/tst.set_props.ksh \ + functional/channel_program/synctask_core/tst.snapshot_destroy.ksh \ + functional/channel_program/synctask_core/tst.snapshot_neg.ksh \ + functional/channel_program/synctask_core/tst.snapshot_recursive.ksh \ + functional/channel_program/synctask_core/tst.snapshot_simple.ksh \ + functional/channel_program/synctask_core/tst.terminate_by_signal.ksh \ + functional/chattr/chattr_001_pos.ksh \ + functional/chattr/chattr_002_neg.ksh \ + functional/chattr/cleanup.ksh \ + functional/chattr/setup.ksh \ + functional/checksum/cleanup.ksh \ + functional/checksum/filetest_001_pos.ksh \ + functional/checksum/filetest_002_pos.ksh \ + functional/checksum/run_blake3_test.ksh \ + functional/checksum/run_edonr_test.ksh \ + functional/checksum/run_sha2_test.ksh \ + functional/checksum/run_skein_test.ksh \ + functional/checksum/setup.ksh \ + functional/clean_mirror/clean_mirror_001_pos.ksh \ + functional/clean_mirror/clean_mirror_002_pos.ksh \ + functional/clean_mirror/clean_mirror_003_pos.ksh \ + functional/clean_mirror/clean_mirror_004_pos.ksh \ + functional/clean_mirror/cleanup.ksh \ + functional/clean_mirror/setup.ksh \ + functional/cli_root/zdb/zdb_002_pos.ksh \ + functional/cli_root/zdb/zdb_003_pos.ksh \ + functional/cli_root/zdb/zdb_004_pos.ksh \ + functional/cli_root/zdb/zdb_005_pos.ksh \ + functional/cli_root/zdb/zdb_006_pos.ksh \ + functional/cli_root/zdb/zdb_args_neg.ksh \ + functional/cli_root/zdb/zdb_args_pos.ksh \ + functional/cli_root/zdb/zdb_block_size_histogram.ksh \ + functional/cli_root/zdb/zdb_checksum.ksh \ + functional/cli_root/zdb/zdb_decompress.ksh \ + functional/cli_root/zdb/zdb_decompress_zstd.ksh \ + functional/cli_root/zdb/zdb_display_block.ksh \ + functional/cli_root/zdb/zdb_label_checksum.ksh \ + functional/cli_root/zdb/zdb_object_range_neg.ksh \ + functional/cli_root/zdb/zdb_object_range_pos.ksh \ + functional/cli_root/zdb/zdb_objset_id.ksh \ + functional/cli_root/zdb/zdb_recover_2.ksh \ + functional/cli_root/zdb/zdb_recover.ksh \ + functional/cli_root/zfs_bookmark/cleanup.ksh \ + functional/cli_root/zfs_bookmark/setup.ksh \ + functional/cli_root/zfs_bookmark/zfs_bookmark_cliargs.ksh \ + functional/cli_root/zfs_change-key/cleanup.ksh \ + functional/cli_root/zfs_change-key/setup.ksh \ + functional/cli_root/zfs_change-key/zfs_change-key_child.ksh \ + functional/cli_root/zfs_change-key/zfs_change-key_clones.ksh \ + functional/cli_root/zfs_change-key/zfs_change-key_format.ksh \ + functional/cli_root/zfs_change-key/zfs_change-key_inherit.ksh \ + functional/cli_root/zfs_change-key/zfs_change-key.ksh \ + functional/cli_root/zfs_change-key/zfs_change-key_load.ksh \ + functional/cli_root/zfs_change-key/zfs_change-key_location.ksh \ + functional/cli_root/zfs_change-key/zfs_change-key_pbkdf2iters.ksh \ + functional/cli_root/zfs/cleanup.ksh \ + functional/cli_root/zfs_clone/cleanup.ksh \ + functional/cli_root/zfs_clone/setup.ksh \ + functional/cli_root/zfs_clone/zfs_clone_001_neg.ksh \ + functional/cli_root/zfs_clone/zfs_clone_002_pos.ksh \ + functional/cli_root/zfs_clone/zfs_clone_003_pos.ksh \ + functional/cli_root/zfs_clone/zfs_clone_004_pos.ksh \ + functional/cli_root/zfs_clone/zfs_clone_005_pos.ksh \ + functional/cli_root/zfs_clone/zfs_clone_006_pos.ksh \ + functional/cli_root/zfs_clone/zfs_clone_007_pos.ksh \ + functional/cli_root/zfs_clone/zfs_clone_008_neg.ksh \ + functional/cli_root/zfs_clone/zfs_clone_009_neg.ksh \ + functional/cli_root/zfs_clone/zfs_clone_010_pos.ksh \ + functional/cli_root/zfs_clone/zfs_clone_deeply_nested.ksh \ + functional/cli_root/zfs_clone/zfs_clone_encrypted.ksh \ + functional/cli_root/zfs_clone/zfs_clone_rm_nested.ksh \ + functional/cli_root/zfs_copies/cleanup.ksh \ + functional/cli_root/zfs_copies/setup.ksh \ + functional/cli_root/zfs_copies/zfs_copies_001_pos.ksh \ + functional/cli_root/zfs_copies/zfs_copies_002_pos.ksh \ + functional/cli_root/zfs_copies/zfs_copies_003_pos.ksh \ + functional/cli_root/zfs_copies/zfs_copies_004_neg.ksh \ + functional/cli_root/zfs_copies/zfs_copies_005_neg.ksh \ + functional/cli_root/zfs_copies/zfs_copies_006_pos.ksh \ + functional/cli_root/zfs_create/cleanup.ksh \ + functional/cli_root/zfs_create/setup.ksh \ + functional/cli_root/zfs_create/zfs_create_001_pos.ksh \ + functional/cli_root/zfs_create/zfs_create_002_pos.ksh \ + functional/cli_root/zfs_create/zfs_create_003_pos.ksh \ + functional/cli_root/zfs_create/zfs_create_004_pos.ksh \ + functional/cli_root/zfs_create/zfs_create_005_pos.ksh \ + functional/cli_root/zfs_create/zfs_create_006_pos.ksh \ + functional/cli_root/zfs_create/zfs_create_007_pos.ksh \ + functional/cli_root/zfs_create/zfs_create_008_neg.ksh \ + functional/cli_root/zfs_create/zfs_create_009_neg.ksh \ + functional/cli_root/zfs_create/zfs_create_010_neg.ksh \ + functional/cli_root/zfs_create/zfs_create_011_pos.ksh \ + functional/cli_root/zfs_create/zfs_create_012_pos.ksh \ + functional/cli_root/zfs_create/zfs_create_013_pos.ksh \ + functional/cli_root/zfs_create/zfs_create_014_pos.ksh \ + functional/cli_root/zfs_create/zfs_create_crypt_combos.ksh \ + functional/cli_root/zfs_create/zfs_create_dryrun.ksh \ + functional/cli_root/zfs_create/zfs_create_encrypted.ksh \ + functional/cli_root/zfs_create/zfs_create_nomount.ksh \ + functional/cli_root/zfs_create/zfs_create_verbose.ksh \ + functional/cli_root/zfs_destroy/cleanup.ksh \ + functional/cli_root/zfs_destroy/setup.ksh \ + functional/cli_root/zfs_destroy/zfs_clone_livelist_condense_and_disable.ksh \ + functional/cli_root/zfs_destroy/zfs_clone_livelist_condense_races.ksh \ + functional/cli_root/zfs_destroy/zfs_clone_livelist_dedup.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_001_pos.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_002_pos.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_003_pos.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_004_pos.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_005_neg.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_006_neg.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_007_neg.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_008_pos.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_009_pos.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_010_pos.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_011_pos.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_012_pos.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_013_neg.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_014_pos.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_015_pos.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_016_pos.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_clone_livelist.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_dev_removal_condense.ksh \ + functional/cli_root/zfs_destroy/zfs_destroy_dev_removal.ksh \ + functional/cli_root/zfs_diff/cleanup.ksh \ + functional/cli_root/zfs_diff/setup.ksh \ + functional/cli_root/zfs_diff/zfs_diff_changes.ksh \ + functional/cli_root/zfs_diff/zfs_diff_cliargs.ksh \ + functional/cli_root/zfs_diff/zfs_diff_encrypted.ksh \ + functional/cli_root/zfs_diff/zfs_diff_mangle.ksh \ + functional/cli_root/zfs_diff/zfs_diff_timestamp.ksh \ + functional/cli_root/zfs_diff/zfs_diff_types.ksh \ + functional/cli_root/zfs_get/cleanup.ksh \ + functional/cli_root/zfs_get/setup.ksh \ + functional/cli_root/zfs_get/zfs_get_001_pos.ksh \ + functional/cli_root/zfs_get/zfs_get_002_pos.ksh \ + functional/cli_root/zfs_get/zfs_get_003_pos.ksh \ + functional/cli_root/zfs_get/zfs_get_004_pos.ksh \ + functional/cli_root/zfs_get/zfs_get_005_neg.ksh \ + functional/cli_root/zfs_get/zfs_get_006_neg.ksh \ + functional/cli_root/zfs_get/zfs_get_007_neg.ksh \ + functional/cli_root/zfs_get/zfs_get_008_pos.ksh \ + functional/cli_root/zfs_get/zfs_get_009_pos.ksh \ + functional/cli_root/zfs_get/zfs_get_010_neg.ksh \ + functional/cli_root/zfs_ids_to_path/cleanup.ksh \ + functional/cli_root/zfs_ids_to_path/setup.ksh \ + functional/cli_root/zfs_ids_to_path/zfs_ids_to_path_001_pos.ksh \ + functional/cli_root/zfs_inherit/cleanup.ksh \ + functional/cli_root/zfs_inherit/setup.ksh \ + functional/cli_root/zfs_inherit/zfs_inherit_001_neg.ksh \ + functional/cli_root/zfs_inherit/zfs_inherit_002_neg.ksh \ + functional/cli_root/zfs_inherit/zfs_inherit_003_pos.ksh \ + functional/cli_root/zfs_inherit/zfs_inherit_mountpoint.ksh \ + functional/cli_root/zfs_jail/cleanup.ksh \ + functional/cli_root/zfs_jail/setup.ksh \ + functional/cli_root/zfs_jail/zfs_jail_001_pos.ksh \ + functional/cli_root/zfs_load-key/cleanup.ksh \ + functional/cli_root/zfs_load-key/setup.ksh \ + functional/cli_root/zfs_load-key/zfs_load-key_all.ksh \ + functional/cli_root/zfs_load-key/zfs_load-key_file.ksh \ + functional/cli_root/zfs_load-key/zfs_load-key_https.ksh \ + functional/cli_root/zfs_load-key/zfs_load-key.ksh \ + functional/cli_root/zfs_load-key/zfs_load-key_location.ksh \ + functional/cli_root/zfs_load-key/zfs_load-key_noop.ksh \ + functional/cli_root/zfs_load-key/zfs_load-key_recursive.ksh \ + functional/cli_root/zfs_mount/cleanup.ksh \ + functional/cli_root/zfs_mount/setup.ksh \ + functional/cli_root/zfs_mount/zfs_mount_001_pos.ksh \ + functional/cli_root/zfs_mount/zfs_mount_002_pos.ksh \ + functional/cli_root/zfs_mount/zfs_mount_003_pos.ksh \ + functional/cli_root/zfs_mount/zfs_mount_004_pos.ksh \ + functional/cli_root/zfs_mount/zfs_mount_005_pos.ksh \ + functional/cli_root/zfs_mount/zfs_mount_006_pos.ksh \ + functional/cli_root/zfs_mount/zfs_mount_007_pos.ksh \ + functional/cli_root/zfs_mount/zfs_mount_008_pos.ksh \ + functional/cli_root/zfs_mount/zfs_mount_009_neg.ksh \ + functional/cli_root/zfs_mount/zfs_mount_010_neg.ksh \ + functional/cli_root/zfs_mount/zfs_mount_011_neg.ksh \ + functional/cli_root/zfs_mount/zfs_mount_012_pos.ksh \ + functional/cli_root/zfs_mount/zfs_mount_013_pos.ksh \ + functional/cli_root/zfs_mount/zfs_mount_014_neg.ksh \ + functional/cli_root/zfs_mount/zfs_mount_all_001_pos.ksh \ + functional/cli_root/zfs_mount/zfs_mount_all_fail.ksh \ + functional/cli_root/zfs_mount/zfs_mount_all_mountpoints.ksh \ + functional/cli_root/zfs_mount/zfs_mount_encrypted.ksh \ + functional/cli_root/zfs_mount/zfs_mount_remount.ksh \ + functional/cli_root/zfs_mount/zfs_mount_test_race.ksh \ + functional/cli_root/zfs_mount/zfs_multi_mount.ksh \ + functional/cli_root/zfs_program/cleanup.ksh \ + functional/cli_root/zfs_program/setup.ksh \ + functional/cli_root/zfs_program/zfs_program_json.ksh \ + functional/cli_root/zfs_promote/cleanup.ksh \ + functional/cli_root/zfs_promote/setup.ksh \ + functional/cli_root/zfs_promote/zfs_promote_001_pos.ksh \ + functional/cli_root/zfs_promote/zfs_promote_002_pos.ksh \ + functional/cli_root/zfs_promote/zfs_promote_003_pos.ksh \ + functional/cli_root/zfs_promote/zfs_promote_004_pos.ksh \ + functional/cli_root/zfs_promote/zfs_promote_005_pos.ksh \ + functional/cli_root/zfs_promote/zfs_promote_006_neg.ksh \ + functional/cli_root/zfs_promote/zfs_promote_007_neg.ksh \ + functional/cli_root/zfs_promote/zfs_promote_008_pos.ksh \ + functional/cli_root/zfs_promote/zfs_promote_encryptionroot.ksh \ + functional/cli_root/zfs_property/cleanup.ksh \ + functional/cli_root/zfs_property/setup.ksh \ + functional/cli_root/zfs_property/zfs_written_property_001_pos.ksh \ + functional/cli_root/zfs_receive/cleanup.ksh \ + functional/cli_root/zfs_receive/receive-o-x_props_aliases.ksh \ + functional/cli_root/zfs_receive/receive-o-x_props_override.ksh \ + functional/cli_root/zfs_receive/setup.ksh \ + functional/cli_root/zfs_receive/zfs_receive_001_pos.ksh \ + functional/cli_root/zfs_receive/zfs_receive_002_pos.ksh \ + functional/cli_root/zfs_receive/zfs_receive_003_pos.ksh \ + functional/cli_root/zfs_receive/zfs_receive_004_neg.ksh \ + functional/cli_root/zfs_receive/zfs_receive_005_neg.ksh \ + functional/cli_root/zfs_receive/zfs_receive_006_pos.ksh \ + functional/cli_root/zfs_receive/zfs_receive_007_neg.ksh \ + functional/cli_root/zfs_receive/zfs_receive_008_pos.ksh \ + functional/cli_root/zfs_receive/zfs_receive_009_neg.ksh \ + functional/cli_root/zfs_receive/zfs_receive_010_pos.ksh \ + functional/cli_root/zfs_receive/zfs_receive_011_pos.ksh \ + functional/cli_root/zfs_receive/zfs_receive_012_pos.ksh \ + functional/cli_root/zfs_receive/zfs_receive_013_pos.ksh \ + functional/cli_root/zfs_receive/zfs_receive_014_pos.ksh \ + functional/cli_root/zfs_receive/zfs_receive_015_pos.ksh \ + functional/cli_root/zfs_receive/zfs_receive_016_pos.ksh \ + functional/cli_root/zfs_receive/zfs_receive_-e.ksh \ + functional/cli_root/zfs_receive/zfs_receive_from_encrypted.ksh \ + functional/cli_root/zfs_receive/zfs_receive_from_zstd.ksh \ + functional/cli_root/zfs_receive/zfs_receive_new_props.ksh \ + functional/cli_root/zfs_receive/zfs_receive_raw_-d.ksh \ + functional/cli_root/zfs_receive/zfs_receive_raw_incremental.ksh \ + functional/cli_root/zfs_receive/zfs_receive_raw.ksh \ + functional/cli_root/zfs_receive/zfs_receive_to_encrypted.ksh \ + functional/cli_root/zfs_receive/zfs_receive_-wR-encrypted-mix.ksh \ + functional/cli_root/zfs_receive/zfs_receive_corrective.ksh \ + functional/cli_root/zfs_receive/zfs_receive_compressed_corrective.ksh \ + functional/cli_root/zfs_rename/cleanup.ksh \ + functional/cli_root/zfs_rename/setup.ksh \ + functional/cli_root/zfs_rename/zfs_rename_001_pos.ksh \ + functional/cli_root/zfs_rename/zfs_rename_002_pos.ksh \ + functional/cli_root/zfs_rename/zfs_rename_003_pos.ksh \ + functional/cli_root/zfs_rename/zfs_rename_004_neg.ksh \ + functional/cli_root/zfs_rename/zfs_rename_005_neg.ksh \ + functional/cli_root/zfs_rename/zfs_rename_006_pos.ksh \ + functional/cli_root/zfs_rename/zfs_rename_007_pos.ksh \ + functional/cli_root/zfs_rename/zfs_rename_008_pos.ksh \ + functional/cli_root/zfs_rename/zfs_rename_009_neg.ksh \ + functional/cli_root/zfs_rename/zfs_rename_010_neg.ksh \ + functional/cli_root/zfs_rename/zfs_rename_011_pos.ksh \ + functional/cli_root/zfs_rename/zfs_rename_012_neg.ksh \ + functional/cli_root/zfs_rename/zfs_rename_013_pos.ksh \ + functional/cli_root/zfs_rename/zfs_rename_014_neg.ksh \ + functional/cli_root/zfs_rename/zfs_rename_encrypted_child.ksh \ + functional/cli_root/zfs_rename/zfs_rename_mountpoint.ksh \ + functional/cli_root/zfs_rename/zfs_rename_nounmount.ksh \ + functional/cli_root/zfs_rename/zfs_rename_to_encrypted.ksh \ + functional/cli_root/zfs_reservation/cleanup.ksh \ + functional/cli_root/zfs_reservation/setup.ksh \ + functional/cli_root/zfs_reservation/zfs_reservation_001_pos.ksh \ + functional/cli_root/zfs_reservation/zfs_reservation_002_pos.ksh \ + functional/cli_root/zfs_rollback/cleanup.ksh \ + functional/cli_root/zfs_rollback/setup.ksh \ + functional/cli_root/zfs_rollback/zfs_rollback_001_pos.ksh \ + functional/cli_root/zfs_rollback/zfs_rollback_002_pos.ksh \ + functional/cli_root/zfs_rollback/zfs_rollback_003_neg.ksh \ + functional/cli_root/zfs_rollback/zfs_rollback_004_neg.ksh \ + functional/cli_root/zfs_send/cleanup.ksh \ + functional/cli_root/zfs_send/setup.ksh \ + functional/cli_root/zfs_send/zfs_send_001_pos.ksh \ + functional/cli_root/zfs_send/zfs_send_002_pos.ksh \ + functional/cli_root/zfs_send/zfs_send_003_pos.ksh \ + functional/cli_root/zfs_send/zfs_send_004_neg.ksh \ + functional/cli_root/zfs_send/zfs_send_005_pos.ksh \ + functional/cli_root/zfs_send/zfs_send_006_pos.ksh \ + functional/cli_root/zfs_send/zfs_send_007_pos.ksh \ + functional/cli_root/zfs_send/zfs_send-b.ksh \ + functional/cli_root/zfs_send/zfs_send_encrypted.ksh \ + functional/cli_root/zfs_send/zfs_send_encrypted_unloaded.ksh \ + functional/cli_root/zfs_send/zfs_send_raw.ksh \ + functional/cli_root/zfs_send/zfs_send_skip_missing.ksh \ + functional/cli_root/zfs_send/zfs_send_sparse.ksh \ + functional/cli_root/zfs_set/cache_001_pos.ksh \ + functional/cli_root/zfs_set/cache_002_neg.ksh \ + functional/cli_root/zfs_set/canmount_001_pos.ksh \ + functional/cli_root/zfs_set/canmount_002_pos.ksh \ + functional/cli_root/zfs_set/canmount_003_pos.ksh \ + functional/cli_root/zfs_set/canmount_004_pos.ksh \ + functional/cli_root/zfs_set/checksum_001_pos.ksh \ + functional/cli_root/zfs_set/cleanup.ksh \ + functional/cli_root/zfs_set/compression_001_pos.ksh \ + functional/cli_root/zfs_set/mountpoint_001_pos.ksh \ + functional/cli_root/zfs_set/mountpoint_002_pos.ksh \ + functional/cli_root/zfs_set/mountpoint_003_pos.ksh \ + functional/cli_root/zfs_set/onoffs_001_pos.ksh \ + functional/cli_root/zfs_set/property_alias_001_pos.ksh \ + functional/cli_root/zfs_set/readonly_001_pos.ksh \ + functional/cli_root/zfs_set/reservation_001_neg.ksh \ + functional/cli_root/zfs_set/ro_props_001_pos.ksh \ + functional/cli_root/zfs_set/setup.ksh \ + functional/cli_root/zfs_set/share_mount_001_neg.ksh \ + functional/cli_root/zfs_set/snapdir_001_pos.ksh \ + functional/cli_root/zfs/setup.ksh \ + functional/cli_root/zfs_set/user_property_001_pos.ksh \ + functional/cli_root/zfs_set/user_property_002_pos.ksh \ + functional/cli_root/zfs_set/user_property_003_neg.ksh \ + functional/cli_root/zfs_set/user_property_004_pos.ksh \ + functional/cli_root/zfs_set/version_001_neg.ksh \ + functional/cli_root/zfs_set/zfs_set_001_neg.ksh \ + functional/cli_root/zfs_set/zfs_set_002_neg.ksh \ + functional/cli_root/zfs_set/zfs_set_003_neg.ksh \ + functional/cli_root/zfs_set/zfs_set_feature_activation.ksh \ + functional/cli_root/zfs_set/zfs_set_keylocation.ksh \ + functional/cli_root/zfs_share/cleanup.ksh \ + functional/cli_root/zfs_share/setup.ksh \ + functional/cli_root/zfs_share/zfs_share_001_pos.ksh \ + functional/cli_root/zfs_share/zfs_share_002_pos.ksh \ + functional/cli_root/zfs_share/zfs_share_003_pos.ksh \ + functional/cli_root/zfs_share/zfs_share_004_pos.ksh \ + functional/cli_root/zfs_share/zfs_share_005_pos.ksh \ + functional/cli_root/zfs_share/zfs_share_006_pos.ksh \ + functional/cli_root/zfs_share/zfs_share_007_neg.ksh \ + functional/cli_root/zfs_share/zfs_share_008_neg.ksh \ + functional/cli_root/zfs_share/zfs_share_009_neg.ksh \ + functional/cli_root/zfs_share/zfs_share_010_neg.ksh \ + functional/cli_root/zfs_share/zfs_share_011_pos.ksh \ + functional/cli_root/zfs_share/zfs_share_012_pos.ksh \ + functional/cli_root/zfs_share/zfs_share_013_pos.ksh \ + functional/cli_root/zfs_share/zfs_share_concurrent_shares.ksh \ + functional/cli_root/zfs_snapshot/cleanup.ksh \ + functional/cli_root/zfs_snapshot/setup.ksh \ + functional/cli_root/zfs_snapshot/zfs_snapshot_001_neg.ksh \ + functional/cli_root/zfs_snapshot/zfs_snapshot_002_neg.ksh \ + functional/cli_root/zfs_snapshot/zfs_snapshot_003_neg.ksh \ + functional/cli_root/zfs_snapshot/zfs_snapshot_004_neg.ksh \ + functional/cli_root/zfs_snapshot/zfs_snapshot_005_neg.ksh \ + functional/cli_root/zfs_snapshot/zfs_snapshot_006_pos.ksh \ + functional/cli_root/zfs_snapshot/zfs_snapshot_007_neg.ksh \ + functional/cli_root/zfs_snapshot/zfs_snapshot_008_neg.ksh \ + functional/cli_root/zfs_snapshot/zfs_snapshot_009_pos.ksh \ + functional/cli_root/zfs_sysfs/cleanup.ksh \ + functional/cli_root/zfs_sysfs/setup.ksh \ + functional/cli_root/zfs_sysfs/zfeature_set_unsupported.ksh \ + functional/cli_root/zfs_sysfs/zfs_get_unsupported.ksh \ + functional/cli_root/zfs_sysfs/zfs_set_unsupported.ksh \ + functional/cli_root/zfs_sysfs/zfs_sysfs_live.ksh \ + functional/cli_root/zfs_sysfs/zpool_get_unsupported.ksh \ + functional/cli_root/zfs_sysfs/zpool_set_unsupported.ksh \ + functional/cli_root/zfs_unload-key/cleanup.ksh \ + functional/cli_root/zfs_unload-key/setup.ksh \ + functional/cli_root/zfs_unload-key/zfs_unload-key_all.ksh \ + functional/cli_root/zfs_unload-key/zfs_unload-key.ksh \ + functional/cli_root/zfs_unload-key/zfs_unload-key_recursive.ksh \ + functional/cli_root/zfs_unmount/cleanup.ksh \ + functional/cli_root/zfs_unmount/setup.ksh \ + functional/cli_root/zfs_unmount/zfs_unmount_001_pos.ksh \ + functional/cli_root/zfs_unmount/zfs_unmount_002_pos.ksh \ + functional/cli_root/zfs_unmount/zfs_unmount_003_pos.ksh \ + functional/cli_root/zfs_unmount/zfs_unmount_004_pos.ksh \ + functional/cli_root/zfs_unmount/zfs_unmount_005_pos.ksh \ + functional/cli_root/zfs_unmount/zfs_unmount_006_pos.ksh \ + functional/cli_root/zfs_unmount/zfs_unmount_007_neg.ksh \ + functional/cli_root/zfs_unmount/zfs_unmount_008_neg.ksh \ + functional/cli_root/zfs_unmount/zfs_unmount_009_pos.ksh \ + functional/cli_root/zfs_unmount/zfs_unmount_all_001_pos.ksh \ + functional/cli_root/zfs_unmount/zfs_unmount_nested.ksh \ + functional/cli_root/zfs_unmount/zfs_unmount_unload_keys.ksh \ + functional/cli_root/zfs_unshare/cleanup.ksh \ + functional/cli_root/zfs_unshare/setup.ksh \ + functional/cli_root/zfs_unshare/zfs_unshare_001_pos.ksh \ + functional/cli_root/zfs_unshare/zfs_unshare_002_pos.ksh \ + functional/cli_root/zfs_unshare/zfs_unshare_003_pos.ksh \ + functional/cli_root/zfs_unshare/zfs_unshare_004_neg.ksh \ + functional/cli_root/zfs_unshare/zfs_unshare_005_neg.ksh \ + functional/cli_root/zfs_unshare/zfs_unshare_006_pos.ksh \ + functional/cli_root/zfs_unshare/zfs_unshare_007_pos.ksh \ + functional/cli_root/zfs_unshare/zfs_unshare_008_pos.ksh \ + functional/cli_root/zfs_upgrade/cleanup.ksh \ + functional/cli_root/zfs_upgrade/setup.ksh \ + functional/cli_root/zfs_upgrade/zfs_upgrade_001_pos.ksh \ + functional/cli_root/zfs_upgrade/zfs_upgrade_002_pos.ksh \ + functional/cli_root/zfs_upgrade/zfs_upgrade_003_pos.ksh \ + functional/cli_root/zfs_upgrade/zfs_upgrade_004_pos.ksh \ + functional/cli_root/zfs_upgrade/zfs_upgrade_005_pos.ksh \ + functional/cli_root/zfs_upgrade/zfs_upgrade_006_neg.ksh \ + functional/cli_root/zfs_upgrade/zfs_upgrade_007_neg.ksh \ + functional/cli_root/zfs_wait/cleanup.ksh \ + functional/cli_root/zfs_wait/setup.ksh \ + functional/cli_root/zfs_wait/zfs_wait_deleteq.ksh \ + functional/cli_root/zfs_wait/zfs_wait_getsubopt.ksh \ + functional/cli_root/zfs/zfs_001_neg.ksh \ + functional/cli_root/zfs/zfs_002_pos.ksh \ + functional/cli_root/zfs/zfs_003_neg.ksh \ + functional/cli_root/zhack/zhack_label_checksum.ksh \ + functional/cli_root/zpool_add/add_nested_replacing_spare.ksh \ + functional/cli_root/zpool_add/add-o_ashift.ksh \ + functional/cli_root/zpool_add/add_prop_ashift.ksh \ + functional/cli_root/zpool_add/cleanup.ksh \ + functional/cli_root/zpool_add/setup.ksh \ + functional/cli_root/zpool_add/zpool_add_001_pos.ksh \ + functional/cli_root/zpool_add/zpool_add_002_pos.ksh \ + functional/cli_root/zpool_add/zpool_add_003_pos.ksh \ + functional/cli_root/zpool_add/zpool_add_004_pos.ksh \ + functional/cli_root/zpool_add/zpool_add_005_pos.ksh \ + functional/cli_root/zpool_add/zpool_add_006_pos.ksh \ + functional/cli_root/zpool_add/zpool_add_007_neg.ksh \ + functional/cli_root/zpool_add/zpool_add_008_neg.ksh \ + functional/cli_root/zpool_add/zpool_add_009_neg.ksh \ + functional/cli_root/zpool_add/zpool_add_010_pos.ksh \ + functional/cli_root/zpool_add/zpool_add_dryrun_output.ksh \ + functional/cli_root/zpool_attach/attach-o_ashift.ksh \ + functional/cli_root/zpool_attach/cleanup.ksh \ + functional/cli_root/zpool_attach/setup.ksh \ + functional/cli_root/zpool_attach/zpool_attach_001_neg.ksh \ + functional/cli_root/zpool/cleanup.ksh \ + functional/cli_root/zpool_clear/cleanup.ksh \ + functional/cli_root/zpool_clear/setup.ksh \ + functional/cli_root/zpool_clear/zpool_clear_001_pos.ksh \ + functional/cli_root/zpool_clear/zpool_clear_002_neg.ksh \ + functional/cli_root/zpool_clear/zpool_clear_003_neg.ksh \ + functional/cli_root/zpool_clear/zpool_clear_readonly.ksh \ + functional/cli_root/zpool_create/cleanup.ksh \ + functional/cli_root/zpool_create/create-o_ashift.ksh \ + functional/cli_root/zpool_create/setup.ksh \ + functional/cli_root/zpool_create/zpool_create_001_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_002_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_003_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_004_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_005_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_006_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_007_neg.ksh \ + functional/cli_root/zpool_create/zpool_create_008_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_009_neg.ksh \ + functional/cli_root/zpool_create/zpool_create_010_neg.ksh \ + functional/cli_root/zpool_create/zpool_create_011_neg.ksh \ + functional/cli_root/zpool_create/zpool_create_012_neg.ksh \ + functional/cli_root/zpool_create/zpool_create_014_neg.ksh \ + functional/cli_root/zpool_create/zpool_create_015_neg.ksh \ + functional/cli_root/zpool_create/zpool_create_016_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_017_neg.ksh \ + functional/cli_root/zpool_create/zpool_create_018_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_019_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_020_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_021_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_022_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_023_neg.ksh \ + functional/cli_root/zpool_create/zpool_create_024_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_crypt_combos.ksh \ + functional/cli_root/zpool_create/zpool_create_draid_001_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_draid_002_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_draid_003_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_draid_004_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_dryrun_output.ksh \ + functional/cli_root/zpool_create/zpool_create_encrypted.ksh \ + functional/cli_root/zpool_create/zpool_create_features_001_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_features_002_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_features_003_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_features_004_neg.ksh \ + functional/cli_root/zpool_create/zpool_create_features_005_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_features_006_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_features_007_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_features_008_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_features_009_pos.ksh \ + functional/cli_root/zpool_create/zpool_create_tempname.ksh \ + functional/cli_root/zpool_destroy/zpool_destroy_001_pos.ksh \ + functional/cli_root/zpool_destroy/zpool_destroy_002_pos.ksh \ + functional/cli_root/zpool_destroy/zpool_destroy_003_neg.ksh \ + functional/cli_root/zpool_detach/cleanup.ksh \ + functional/cli_root/zpool_detach/setup.ksh \ + functional/cli_root/zpool_detach/zpool_detach_001_neg.ksh \ + functional/cli_root/zpool_events/cleanup.ksh \ + functional/cli_root/zpool_events/setup.ksh \ + functional/cli_root/zpool_events/zpool_events_clear.ksh \ + functional/cli_root/zpool_events/zpool_events_clear_retained.ksh \ + functional/cli_root/zpool_events/zpool_events_cliargs.ksh \ + functional/cli_root/zpool_events/zpool_events_duplicates.ksh \ + functional/cli_root/zpool_events/zpool_events_errors.ksh \ + functional/cli_root/zpool_events/zpool_events_follow.ksh \ + functional/cli_root/zpool_events/zpool_events_poolname.ksh \ + functional/cli_root/zpool_expand/cleanup.ksh \ + functional/cli_root/zpool_expand/setup.ksh \ + functional/cli_root/zpool_expand/zpool_expand_001_pos.ksh \ + functional/cli_root/zpool_expand/zpool_expand_002_pos.ksh \ + functional/cli_root/zpool_expand/zpool_expand_003_neg.ksh \ + functional/cli_root/zpool_expand/zpool_expand_004_pos.ksh \ + functional/cli_root/zpool_expand/zpool_expand_005_pos.ksh \ + functional/cli_root/zpool_export/cleanup.ksh \ + functional/cli_root/zpool_export/setup.ksh \ + functional/cli_root/zpool_export/zpool_export_001_pos.ksh \ + functional/cli_root/zpool_export/zpool_export_002_pos.ksh \ + functional/cli_root/zpool_export/zpool_export_003_neg.ksh \ + functional/cli_root/zpool_export/zpool_export_004_pos.ksh \ + functional/cli_root/zpool_get/cleanup.ksh \ + functional/cli_root/zpool_get/setup.ksh \ + functional/cli_root/zpool_get/zpool_get_001_pos.ksh \ + functional/cli_root/zpool_get/zpool_get_002_pos.ksh \ + functional/cli_root/zpool_get/zpool_get_003_pos.ksh \ + functional/cli_root/zpool_get/zpool_get_004_neg.ksh \ + functional/cli_root/zpool_get/zpool_get_005_pos.ksh \ + functional/cli_root/zpool_history/cleanup.ksh \ + functional/cli_root/zpool_history/setup.ksh \ + functional/cli_root/zpool_history/zpool_history_001_neg.ksh \ + functional/cli_root/zpool_history/zpool_history_002_pos.ksh \ + functional/cli_root/zpool_import/cleanup.ksh \ + functional/cli_root/zpool_import/import_cachefile_device_added.ksh \ + functional/cli_root/zpool_import/import_cachefile_device_removed.ksh \ + functional/cli_root/zpool_import/import_cachefile_device_replaced.ksh \ + functional/cli_root/zpool_import/import_cachefile_mirror_attached.ksh \ + functional/cli_root/zpool_import/import_cachefile_mirror_detached.ksh \ + functional/cli_root/zpool_import/import_cachefile_paths_changed.ksh \ + functional/cli_root/zpool_import/import_cachefile_shared_device.ksh \ + functional/cli_root/zpool_import/import_devices_missing.ksh \ + functional/cli_root/zpool_import/import_paths_changed.ksh \ + functional/cli_root/zpool_import/import_rewind_config_changed.ksh \ + functional/cli_root/zpool_import/import_rewind_device_replaced.ksh \ + functional/cli_root/zpool_import/setup.ksh \ + functional/cli_root/zpool_import/zpool_import_001_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_002_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_003_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_004_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_005_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_006_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_007_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_008_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_009_neg.ksh \ + functional/cli_root/zpool_import/zpool_import_010_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_011_neg.ksh \ + functional/cli_root/zpool_import/zpool_import_012_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_013_neg.ksh \ + functional/cli_root/zpool_import/zpool_import_014_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_015_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_016_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_017_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_all_001_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_encrypted.ksh \ + functional/cli_root/zpool_import/zpool_import_encrypted_load.ksh \ + functional/cli_root/zpool_import/zpool_import_errata3.ksh \ + functional/cli_root/zpool_import/zpool_import_errata4.ksh \ + functional/cli_root/zpool_import/zpool_import_features_001_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_features_002_neg.ksh \ + functional/cli_root/zpool_import/zpool_import_features_003_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_missing_001_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_missing_002_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_missing_003_pos.ksh \ + functional/cli_root/zpool_import/zpool_import_rename_001_pos.ksh \ + functional/cli_root/zpool_initialize/cleanup.ksh \ + functional/cli_root/zpool_initialize/zpool_initialize_attach_detach_add_remove.ksh \ + functional/cli_root/zpool_initialize/zpool_initialize_fault_export_import_online.ksh \ + functional/cli_root/zpool_initialize/zpool_initialize_import_export.ksh \ + functional/cli_root/zpool_initialize/zpool_initialize_offline_export_import_online.ksh \ + functional/cli_root/zpool_initialize/zpool_initialize_online_offline.ksh \ + functional/cli_root/zpool_initialize/zpool_initialize_split.ksh \ + functional/cli_root/zpool_initialize/zpool_initialize_start_and_cancel_neg.ksh \ + functional/cli_root/zpool_initialize/zpool_initialize_start_and_cancel_pos.ksh \ + functional/cli_root/zpool_initialize/zpool_initialize_suspend_resume.ksh \ + functional/cli_root/zpool_initialize/zpool_initialize_unsupported_vdevs.ksh \ + functional/cli_root/zpool_initialize/zpool_initialize_verify_checksums.ksh \ + functional/cli_root/zpool_initialize/zpool_initialize_verify_initialized.ksh \ + functional/cli_root/zpool_labelclear/zpool_labelclear_active.ksh \ + functional/cli_root/zpool_labelclear/zpool_labelclear_exported.ksh \ + functional/cli_root/zpool_labelclear/zpool_labelclear_removed.ksh \ + functional/cli_root/zpool_labelclear/zpool_labelclear_valid.ksh \ + functional/cli_root/zpool_offline/cleanup.ksh \ + functional/cli_root/zpool_offline/setup.ksh \ + functional/cli_root/zpool_offline/zpool_offline_001_pos.ksh \ + functional/cli_root/zpool_offline/zpool_offline_002_neg.ksh \ + functional/cli_root/zpool_offline/zpool_offline_003_pos.ksh \ + functional/cli_root/zpool_online/cleanup.ksh \ + functional/cli_root/zpool_online/setup.ksh \ + functional/cli_root/zpool_online/zpool_online_001_pos.ksh \ + functional/cli_root/zpool_online/zpool_online_002_neg.ksh \ + functional/cli_root/zpool_remove/cleanup.ksh \ + functional/cli_root/zpool_remove/setup.ksh \ + functional/cli_root/zpool_remove/zpool_remove_001_neg.ksh \ + functional/cli_root/zpool_remove/zpool_remove_002_pos.ksh \ + functional/cli_root/zpool_remove/zpool_remove_003_pos.ksh \ + functional/cli_root/zpool_reopen/cleanup.ksh \ + functional/cli_root/zpool_reopen/setup.ksh \ + functional/cli_root/zpool_reopen/zpool_reopen_001_pos.ksh \ + functional/cli_root/zpool_reopen/zpool_reopen_002_pos.ksh \ + functional/cli_root/zpool_reopen/zpool_reopen_003_pos.ksh \ + functional/cli_root/zpool_reopen/zpool_reopen_004_pos.ksh \ + functional/cli_root/zpool_reopen/zpool_reopen_005_pos.ksh \ + functional/cli_root/zpool_reopen/zpool_reopen_006_neg.ksh \ + functional/cli_root/zpool_reopen/zpool_reopen_007_pos.ksh \ + functional/cli_root/zpool_replace/cleanup.ksh \ + functional/cli_root/zpool_replace/replace-o_ashift.ksh \ + functional/cli_root/zpool_replace/replace_prop_ashift.ksh \ + functional/cli_root/zpool_replace/setup.ksh \ + functional/cli_root/zpool_replace/zpool_replace_001_neg.ksh \ + functional/cli_root/zpool_resilver/cleanup.ksh \ + functional/cli_root/zpool_resilver/setup.ksh \ + functional/cli_root/zpool_resilver/zpool_resilver_bad_args.ksh \ + functional/cli_root/zpool_resilver/zpool_resilver_restart.ksh \ + functional/cli_root/zpool_scrub/cleanup.ksh \ + functional/cli_root/zpool_scrub/setup.ksh \ + functional/cli_root/zpool_scrub/zpool_scrub_001_neg.ksh \ + functional/cli_root/zpool_scrub/zpool_scrub_002_pos.ksh \ + functional/cli_root/zpool_scrub/zpool_scrub_003_pos.ksh \ + functional/cli_root/zpool_scrub/zpool_scrub_004_pos.ksh \ + functional/cli_root/zpool_scrub/zpool_scrub_005_pos.ksh \ + functional/cli_root/zpool_scrub/zpool_scrub_encrypted_unloaded.ksh \ + functional/cli_root/zpool_scrub/zpool_scrub_multiple_copies.ksh \ + functional/cli_root/zpool_scrub/zpool_scrub_offline_device.ksh \ + functional/cli_root/zpool_scrub/zpool_scrub_print_repairing.ksh \ + functional/cli_root/zpool_set/cleanup.ksh \ + functional/cli_root/zpool_set/setup.ksh \ + functional/cli_root/zpool/setup.ksh \ + functional/cli_root/zpool_set/zpool_set_001_pos.ksh \ + functional/cli_root/zpool_set/zpool_set_002_neg.ksh \ + functional/cli_root/zpool_set/zpool_set_003_neg.ksh \ + functional/cli_root/zpool_set/zpool_set_ashift.ksh \ + functional/cli_root/zpool_set/zpool_set_features.ksh \ + functional/cli_root/zpool_split/cleanup.ksh \ + functional/cli_root/zpool_split/setup.ksh \ + functional/cli_root/zpool_split/zpool_split_cliargs.ksh \ + functional/cli_root/zpool_split/zpool_split_devices.ksh \ + functional/cli_root/zpool_split/zpool_split_dryrun_output.ksh \ + functional/cli_root/zpool_split/zpool_split_encryption.ksh \ + functional/cli_root/zpool_split/zpool_split_indirect.ksh \ + functional/cli_root/zpool_split/zpool_split_props.ksh \ + functional/cli_root/zpool_split/zpool_split_resilver.ksh \ + functional/cli_root/zpool_split/zpool_split_vdevs.ksh \ + functional/cli_root/zpool_split/zpool_split_wholedisk.ksh \ + functional/cli_root/zpool_status/cleanup.ksh \ + functional/cli_root/zpool_status/setup.ksh \ + functional/cli_root/zpool_status/zpool_status_001_pos.ksh \ + functional/cli_root/zpool_status/zpool_status_002_pos.ksh \ + functional/cli_root/zpool_status/zpool_status_003_pos.ksh \ + functional/cli_root/zpool_status/zpool_status_004_pos.ksh \ + functional/cli_root/zpool_status/zpool_status_features_001_pos.ksh \ + functional/cli_root/zpool_sync/cleanup.ksh \ + functional/cli_root/zpool_sync/setup.ksh \ + functional/cli_root/zpool_sync/zpool_sync_001_pos.ksh \ + functional/cli_root/zpool_sync/zpool_sync_002_neg.ksh \ + functional/cli_root/zpool_trim/cleanup.ksh \ + functional/cli_root/zpool_trim/setup.ksh \ + functional/cli_root/zpool_trim/zpool_trim_attach_detach_add_remove.ksh \ + functional/cli_root/zpool_trim/zpool_trim_fault_export_import_online.ksh \ + functional/cli_root/zpool_trim/zpool_trim_import_export.ksh \ + functional/cli_root/zpool_trim/zpool_trim_multiple.ksh \ + functional/cli_root/zpool_trim/zpool_trim_neg.ksh \ + functional/cli_root/zpool_trim/zpool_trim_offline_export_import_online.ksh \ + functional/cli_root/zpool_trim/zpool_trim_online_offline.ksh \ + functional/cli_root/zpool_trim/zpool_trim_partial.ksh \ + functional/cli_root/zpool_trim/zpool_trim_rate.ksh \ + functional/cli_root/zpool_trim/zpool_trim_rate_neg.ksh \ + functional/cli_root/zpool_trim/zpool_trim_secure.ksh \ + functional/cli_root/zpool_trim/zpool_trim_split.ksh \ + functional/cli_root/zpool_trim/zpool_trim_start_and_cancel_neg.ksh \ + functional/cli_root/zpool_trim/zpool_trim_start_and_cancel_pos.ksh \ + functional/cli_root/zpool_trim/zpool_trim_suspend_resume.ksh \ + functional/cli_root/zpool_trim/zpool_trim_unsupported_vdevs.ksh \ + functional/cli_root/zpool_trim/zpool_trim_verify_checksums.ksh \ + functional/cli_root/zpool_trim/zpool_trim_verify_trimmed.ksh \ + functional/cli_root/zpool_upgrade/cleanup.ksh \ + functional/cli_root/zpool_upgrade/setup.ksh \ + functional/cli_root/zpool_upgrade/zpool_upgrade_001_pos.ksh \ + functional/cli_root/zpool_upgrade/zpool_upgrade_002_pos.ksh \ + functional/cli_root/zpool_upgrade/zpool_upgrade_003_pos.ksh \ + functional/cli_root/zpool_upgrade/zpool_upgrade_004_pos.ksh \ + functional/cli_root/zpool_upgrade/zpool_upgrade_005_neg.ksh \ + functional/cli_root/zpool_upgrade/zpool_upgrade_006_neg.ksh \ + functional/cli_root/zpool_upgrade/zpool_upgrade_007_pos.ksh \ + functional/cli_root/zpool_upgrade/zpool_upgrade_008_pos.ksh \ + functional/cli_root/zpool_upgrade/zpool_upgrade_009_neg.ksh \ + functional/cli_root/zpool_upgrade/zpool_upgrade_features_001_pos.ksh \ + functional/cli_root/zpool_wait/cleanup.ksh \ + functional/cli_root/zpool_wait/scan/cleanup.ksh \ + functional/cli_root/zpool_wait/scan/setup.ksh \ + functional/cli_root/zpool_wait/scan/zpool_wait_rebuild.ksh \ + functional/cli_root/zpool_wait/scan/zpool_wait_replace_cancel.ksh \ + functional/cli_root/zpool_wait/scan/zpool_wait_replace.ksh \ + functional/cli_root/zpool_wait/scan/zpool_wait_resilver.ksh \ + functional/cli_root/zpool_wait/scan/zpool_wait_scrub_basic.ksh \ + functional/cli_root/zpool_wait/scan/zpool_wait_scrub_cancel.ksh \ + functional/cli_root/zpool_wait/scan/zpool_wait_scrub_flag.ksh \ + functional/cli_root/zpool_wait/setup.ksh \ + functional/cli_root/zpool_wait/zpool_wait_discard.ksh \ + functional/cli_root/zpool_wait/zpool_wait_freeing.ksh \ + functional/cli_root/zpool_wait/zpool_wait_initialize_basic.ksh \ + functional/cli_root/zpool_wait/zpool_wait_initialize_cancel.ksh \ + functional/cli_root/zpool_wait/zpool_wait_initialize_flag.ksh \ + functional/cli_root/zpool_wait/zpool_wait_multiple.ksh \ + functional/cli_root/zpool_wait/zpool_wait_no_activity.ksh \ + functional/cli_root/zpool_wait/zpool_wait_remove_cancel.ksh \ + functional/cli_root/zpool_wait/zpool_wait_remove.ksh \ + functional/cli_root/zpool_wait/zpool_wait_trim_basic.ksh \ + functional/cli_root/zpool_wait/zpool_wait_trim_cancel.ksh \ + functional/cli_root/zpool_wait/zpool_wait_trim_flag.ksh \ + functional/cli_root/zpool_wait/zpool_wait_usage.ksh \ + functional/cli_root/zpool/zpool_001_neg.ksh \ + functional/cli_root/zpool/zpool_002_pos.ksh \ + functional/cli_root/zpool/zpool_003_pos.ksh \ + functional/cli_root/zpool/zpool_colors.ksh \ + functional/cli_user/misc/arcstat_001_pos.ksh \ + functional/cli_user/misc/arc_summary_001_pos.ksh \ + functional/cli_user/misc/arc_summary_002_neg.ksh \ + functional/cli_user/misc/cleanup.ksh \ + functional/cli_user/misc/setup.ksh \ + functional/cli_user/misc/zdb_001_neg.ksh \ + functional/cli_user/misc/zfs_001_neg.ksh \ + functional/cli_user/misc/zfs_allow_001_neg.ksh \ + functional/cli_user/misc/zfs_clone_001_neg.ksh \ + functional/cli_user/misc/zfs_create_001_neg.ksh \ + functional/cli_user/misc/zfs_destroy_001_neg.ksh \ + functional/cli_user/misc/zfs_get_001_neg.ksh \ + functional/cli_user/misc/zfs_inherit_001_neg.ksh \ + functional/cli_user/misc/zfs_mount_001_neg.ksh \ + functional/cli_user/misc/zfs_promote_001_neg.ksh \ + functional/cli_user/misc/zfs_receive_001_neg.ksh \ + functional/cli_user/misc/zfs_rename_001_neg.ksh \ + functional/cli_user/misc/zfs_rollback_001_neg.ksh \ + functional/cli_user/misc/zfs_send_001_neg.ksh \ + functional/cli_user/misc/zfs_set_001_neg.ksh \ + functional/cli_user/misc/zfs_share_001_neg.ksh \ + functional/cli_user/misc/zfs_snapshot_001_neg.ksh \ + functional/cli_user/misc/zfs_unallow_001_neg.ksh \ + functional/cli_user/misc/zfs_unmount_001_neg.ksh \ + functional/cli_user/misc/zfs_unshare_001_neg.ksh \ + functional/cli_user/misc/zfs_upgrade_001_neg.ksh \ + functional/cli_user/misc/zpool_001_neg.ksh \ + functional/cli_user/misc/zpool_add_001_neg.ksh \ + functional/cli_user/misc/zpool_attach_001_neg.ksh \ + functional/cli_user/misc/zpool_clear_001_neg.ksh \ + functional/cli_user/misc/zpool_create_001_neg.ksh \ + functional/cli_user/misc/zpool_destroy_001_neg.ksh \ + functional/cli_user/misc/zpool_detach_001_neg.ksh \ + functional/cli_user/misc/zpool_export_001_neg.ksh \ + functional/cli_user/misc/zpool_get_001_neg.ksh \ + functional/cli_user/misc/zpool_history_001_neg.ksh \ + functional/cli_user/misc/zpool_import_001_neg.ksh \ + functional/cli_user/misc/zpool_import_002_neg.ksh \ + functional/cli_user/misc/zpool_offline_001_neg.ksh \ + functional/cli_user/misc/zpool_online_001_neg.ksh \ + functional/cli_user/misc/zpool_remove_001_neg.ksh \ + functional/cli_user/misc/zpool_replace_001_neg.ksh \ + functional/cli_user/misc/zpool_scrub_001_neg.ksh \ + functional/cli_user/misc/zpool_set_001_neg.ksh \ + functional/cli_user/misc/zpool_status_001_neg.ksh \ + functional/cli_user/misc/zpool_upgrade_001_neg.ksh \ + functional/cli_user/misc/zpool_wait_privilege.ksh \ + functional/cli_user/zfs_list/cleanup.ksh \ + functional/cli_user/zfs_list/setup.ksh \ + functional/cli_user/zfs_list/zfs_list_001_pos.ksh \ + functional/cli_user/zfs_list/zfs_list_002_pos.ksh \ + functional/cli_user/zfs_list/zfs_list_003_pos.ksh \ + functional/cli_user/zfs_list/zfs_list_004_neg.ksh \ + functional/cli_user/zfs_list/zfs_list_005_neg.ksh \ + functional/cli_user/zfs_list/zfs_list_007_pos.ksh \ + functional/cli_user/zfs_list/zfs_list_008_neg.ksh \ + functional/cli_user/zpool_iostat/cleanup.ksh \ + functional/cli_user/zpool_iostat/setup.ksh \ + functional/cli_user/zpool_iostat/zpool_iostat_001_neg.ksh \ + functional/cli_user/zpool_iostat/zpool_iostat_002_pos.ksh \ + functional/cli_user/zpool_iostat/zpool_iostat_003_neg.ksh \ + functional/cli_user/zpool_iostat/zpool_iostat_004_pos.ksh \ + functional/cli_user/zpool_iostat/zpool_iostat_005_pos.ksh \ + functional/cli_user/zpool_iostat/zpool_iostat_-c_disable.ksh \ + functional/cli_user/zpool_iostat/zpool_iostat_-c_homedir.ksh \ + functional/cli_user/zpool_iostat/zpool_iostat_-c_searchpath.ksh \ + functional/cli_user/zpool_list/cleanup.ksh \ + functional/cli_user/zpool_list/setup.ksh \ + functional/cli_user/zpool_list/zpool_list_001_pos.ksh \ + functional/cli_user/zpool_list/zpool_list_002_neg.ksh \ + functional/cli_user/zpool_status/cleanup.ksh \ + functional/cli_user/zpool_status/setup.ksh \ + functional/cli_user/zpool_status/zpool_status_003_pos.ksh \ + functional/cli_user/zpool_status/zpool_status_-c_disable.ksh \ + functional/cli_user/zpool_status/zpool_status_-c_homedir.ksh \ + functional/cli_user/zpool_status/zpool_status_-c_searchpath.ksh \ + functional/compression/cleanup.ksh \ + functional/compression/compress_001_pos.ksh \ + functional/compression/compress_002_pos.ksh \ + functional/compression/compress_003_pos.ksh \ + functional/compression/compress_004_pos.ksh \ + functional/compression/compress_zstd_bswap.ksh \ + functional/compression/l2arc_compressed_arc_disabled.ksh \ + functional/compression/l2arc_compressed_arc.ksh \ + functional/compression/l2arc_encrypted.ksh \ + functional/compression/l2arc_encrypted_no_compressed_arc.ksh \ + functional/compression/setup.ksh \ + functional/cp_files/cleanup.ksh \ + functional/cp_files/cp_files_001_pos.ksh \ + functional/cp_files/setup.ksh \ + functional/crtime/cleanup.ksh \ + functional/crtime/crtime_001_pos.ksh \ + functional/crtime/setup.ksh \ + functional/ctime/cleanup.ksh \ + functional/ctime/ctime_001_pos.ksh \ + functional/ctime/setup.ksh \ + functional/deadman/deadman_ratelimit.ksh \ + functional/deadman/deadman_sync.ksh \ + functional/deadman/deadman_zio.ksh \ + functional/delegate/cleanup.ksh \ + functional/delegate/setup.ksh \ + functional/delegate/zfs_allow_001_pos.ksh \ + functional/delegate/zfs_allow_002_pos.ksh \ + functional/delegate/zfs_allow_003_pos.ksh \ + functional/delegate/zfs_allow_004_pos.ksh \ + functional/delegate/zfs_allow_005_pos.ksh \ + functional/delegate/zfs_allow_006_pos.ksh \ + functional/delegate/zfs_allow_007_pos.ksh \ + functional/delegate/zfs_allow_008_pos.ksh \ + functional/delegate/zfs_allow_009_neg.ksh \ + functional/delegate/zfs_allow_010_pos.ksh \ + functional/delegate/zfs_allow_011_neg.ksh \ + functional/delegate/zfs_allow_012_neg.ksh \ + functional/delegate/zfs_unallow_001_pos.ksh \ + functional/delegate/zfs_unallow_002_pos.ksh \ + functional/delegate/zfs_unallow_003_pos.ksh \ + functional/delegate/zfs_unallow_004_pos.ksh \ + functional/delegate/zfs_unallow_005_pos.ksh \ + functional/delegate/zfs_unallow_006_pos.ksh \ + functional/delegate/zfs_unallow_007_neg.ksh \ + functional/delegate/zfs_unallow_008_neg.ksh \ + functional/devices/cleanup.ksh \ + functional/devices/devices_001_pos.ksh \ + functional/devices/devices_002_neg.ksh \ + functional/devices/devices_003_pos.ksh \ + functional/devices/setup.ksh \ + functional/dos_attributes/cleanup.ksh \ + functional/dos_attributes/read_dos_attrs_001.ksh \ + functional/dos_attributes/setup.ksh \ + functional/dos_attributes/write_dos_attrs_001.ksh \ + functional/events/cleanup.ksh \ + functional/events/events_001_pos.ksh \ + functional/events/events_002_pos.ksh \ + functional/events/setup.ksh \ + functional/events/zed_fd_spill.ksh \ + functional/events/zed_rc_filter.ksh \ + functional/exec/cleanup.ksh \ + functional/exec/exec_001_pos.ksh \ + functional/exec/exec_002_neg.ksh \ + functional/exec/setup.ksh \ + functional/fallocate/cleanup.ksh \ + functional/fallocate/fallocate_prealloc.ksh \ + functional/fallocate/fallocate_punch-hole.ksh \ + functional/fallocate/fallocate_zero-range.ksh \ + functional/fallocate/setup.ksh \ + functional/fault/auto_offline_001_pos.ksh \ + functional/fault/auto_online_001_pos.ksh \ + functional/fault/auto_online_002_pos.ksh \ + functional/fault/auto_replace_001_pos.ksh \ + functional/fault/auto_spare_001_pos.ksh \ + functional/fault/auto_spare_002_pos.ksh \ + functional/fault/auto_spare_ashift.ksh \ + functional/fault/auto_spare_multiple.ksh \ + functional/fault/auto_spare_shared.ksh \ + functional/fault/cleanup.ksh \ + functional/fault/decompress_fault.ksh \ + functional/fault/decrypt_fault.ksh \ + functional/fault/scrub_after_resilver.ksh \ + functional/fault/setup.ksh \ + functional/fault/zpool_status_-s.ksh \ + functional/features/async_destroy/async_destroy_001_pos.ksh \ + functional/features/async_destroy/cleanup.ksh \ + functional/features/async_destroy/setup.ksh \ + functional/features/large_dnode/cleanup.ksh \ + functional/features/large_dnode/large_dnode_001_pos.ksh \ + functional/features/large_dnode/large_dnode_002_pos.ksh \ + functional/features/large_dnode/large_dnode_003_pos.ksh \ + functional/features/large_dnode/large_dnode_004_neg.ksh \ + functional/features/large_dnode/large_dnode_005_pos.ksh \ + functional/features/large_dnode/large_dnode_006_pos.ksh \ + functional/features/large_dnode/large_dnode_007_neg.ksh \ + functional/features/large_dnode/large_dnode_008_pos.ksh \ + functional/features/large_dnode/large_dnode_009_pos.ksh \ + functional/features/large_dnode/setup.ksh \ + functional/grow/grow_pool_001_pos.ksh \ + functional/grow/grow_replicas_001_pos.ksh \ + functional/history/cleanup.ksh \ + functional/history/history_001_pos.ksh \ + functional/history/history_002_pos.ksh \ + functional/history/history_003_pos.ksh \ + functional/history/history_004_pos.ksh \ + functional/history/history_005_neg.ksh \ + functional/history/history_006_neg.ksh \ + functional/history/history_007_pos.ksh \ + functional/history/history_008_pos.ksh \ + functional/history/history_009_pos.ksh \ + functional/history/history_010_pos.ksh \ + functional/history/setup.ksh \ + functional/inheritance/cleanup.ksh \ + functional/inheritance/inherit_001_pos.ksh \ + functional/inuse/inuse_001_pos.ksh \ + functional/inuse/inuse_003_pos.ksh \ + functional/inuse/inuse_004_pos.ksh \ + functional/inuse/inuse_005_pos.ksh \ + functional/inuse/inuse_006_pos.ksh \ + functional/inuse/inuse_007_pos.ksh \ + functional/inuse/inuse_008_pos.ksh \ + functional/inuse/inuse_009_pos.ksh \ + functional/inuse/setup.ksh \ + functional/io/cleanup.ksh \ + functional/io/io_uring.ksh \ + functional/io/libaio.ksh \ + functional/io/mmap.ksh \ + functional/io/posixaio.ksh \ + functional/io/psync.ksh \ + functional/io/setup.ksh \ + functional/io/sync.ksh \ + functional/l2arc/cleanup.ksh \ + functional/l2arc/l2arc_arcstats_pos.ksh \ + functional/l2arc/l2arc_l2miss_pos.ksh \ + functional/l2arc/l2arc_mfuonly_pos.ksh \ + functional/l2arc/persist_l2arc_001_pos.ksh \ + functional/l2arc/persist_l2arc_002_pos.ksh \ + functional/l2arc/persist_l2arc_003_neg.ksh \ + functional/l2arc/persist_l2arc_004_pos.ksh \ + functional/l2arc/persist_l2arc_005_pos.ksh \ + functional/l2arc/setup.ksh \ + functional/large_files/cleanup.ksh \ + functional/large_files/large_files_001_pos.ksh \ + functional/large_files/large_files_002_pos.ksh \ + functional/large_files/setup.ksh \ + functional/largest_pool/largest_pool_001_pos.ksh \ + functional/libzfs/cleanup.ksh \ + functional/libzfs/libzfs_input.ksh \ + functional/libzfs/setup.ksh \ + functional/limits/cleanup.ksh \ + functional/limits/filesystem_count.ksh \ + functional/limits/filesystem_limit.ksh \ + functional/limits/setup.ksh \ + functional/limits/snapshot_count.ksh \ + functional/limits/snapshot_limit.ksh \ + functional/link_count/cleanup.ksh \ + functional/link_count/link_count_001.ksh \ + functional/link_count/link_count_root_inode.ksh \ + functional/link_count/setup.ksh \ + functional/log_spacemap/log_spacemap_import_logs.ksh \ + functional/migration/cleanup.ksh \ + functional/migration/migration_001_pos.ksh \ + functional/migration/migration_002_pos.ksh \ + functional/migration/migration_003_pos.ksh \ + functional/migration/migration_004_pos.ksh \ + functional/migration/migration_005_pos.ksh \ + functional/migration/migration_006_pos.ksh \ + functional/migration/migration_007_pos.ksh \ + functional/migration/migration_008_pos.ksh \ + functional/migration/migration_009_pos.ksh \ + functional/migration/migration_010_pos.ksh \ + functional/migration/migration_011_pos.ksh \ + functional/migration/migration_012_pos.ksh \ + functional/migration/setup.ksh \ + functional/mmap/cleanup.ksh \ + functional/mmap/mmap_libaio_001_pos.ksh \ + functional/mmap/mmap_read_001_pos.ksh \ + functional/mmap/mmap_seek_001_pos.ksh \ + functional/mmap/mmap_sync_001_pos.ksh \ + functional/mmap/mmap_write_001_pos.ksh \ + functional/mmap/setup.ksh \ + functional/mmp/cleanup.ksh \ + functional/mmp/mmp_active_import.ksh \ + functional/mmp/mmp_exported_import.ksh \ + functional/mmp/mmp_hostid.ksh \ + functional/mmp/mmp_inactive_import.ksh \ + functional/mmp/mmp_interval.ksh \ + functional/mmp/mmp_on_off.ksh \ + functional/mmp/mmp_on_thread.ksh \ + functional/mmp/mmp_on_uberblocks.ksh \ + functional/mmp/mmp_on_zdb.ksh \ + functional/mmp/mmp_reset_interval.ksh \ + functional/mmp/mmp_write_distribution.ksh \ + functional/mmp/mmp_write_uberblocks.ksh \ + functional/mmp/multihost_history.ksh \ + functional/mmp/setup.ksh \ + functional/mount/cleanup.ksh \ + functional/mount/setup.ksh \ + functional/mount/umount_001.ksh \ + functional/mount/umountall_001.ksh \ + functional/mount/umount_unlinked_drain.ksh \ + functional/mv_files/cleanup.ksh \ + functional/mv_files/mv_files_001_pos.ksh \ + functional/mv_files/mv_files_002_pos.ksh \ + functional/mv_files/random_creation.ksh \ + functional/mv_files/setup.ksh \ + functional/nestedfs/cleanup.ksh \ + functional/nestedfs/nestedfs_001_pos.ksh \ + functional/nestedfs/setup.ksh \ + functional/nopwrite/cleanup.ksh \ + functional/nopwrite/nopwrite_copies.ksh \ + functional/nopwrite/nopwrite_mtime.ksh \ + functional/nopwrite/nopwrite_negative.ksh \ + functional/nopwrite/nopwrite_promoted_clone.ksh \ + functional/nopwrite/nopwrite_recsize.ksh \ + functional/nopwrite/nopwrite_sync.ksh \ + functional/nopwrite/nopwrite_varying_compression.ksh \ + functional/nopwrite/nopwrite_volume.ksh \ + functional/nopwrite/setup.ksh \ + functional/no_space/cleanup.ksh \ + functional/no_space/enospc_001_pos.ksh \ + functional/no_space/enospc_002_pos.ksh \ + functional/no_space/enospc_003_pos.ksh \ + functional/no_space/enospc_df.ksh \ + functional/no_space/enospc_rm.ksh \ + functional/no_space/setup.ksh \ + functional/online_offline/cleanup.ksh \ + functional/online_offline/online_offline_001_pos.ksh \ + functional/online_offline/online_offline_002_neg.ksh \ + functional/online_offline/online_offline_003_neg.ksh \ + functional/online_offline/setup.ksh \ + functional/pam/cleanup.ksh \ + functional/pam/pam_basic.ksh \ + functional/pam/pam_nounmount.ksh \ + functional/pam/pam_short_password.ksh \ + functional/pam/setup.ksh \ + functional/pool_checkpoint/checkpoint_after_rewind.ksh \ + functional/pool_checkpoint/checkpoint_big_rewind.ksh \ + functional/pool_checkpoint/checkpoint_capacity.ksh \ + functional/pool_checkpoint/checkpoint_conf_change.ksh \ + functional/pool_checkpoint/checkpoint_discard_busy.ksh \ + functional/pool_checkpoint/checkpoint_discard.ksh \ + functional/pool_checkpoint/checkpoint_discard_many.ksh \ + functional/pool_checkpoint/checkpoint_indirect.ksh \ + functional/pool_checkpoint/checkpoint_invalid.ksh \ + functional/pool_checkpoint/checkpoint_lun_expsz.ksh \ + functional/pool_checkpoint/checkpoint_open.ksh \ + functional/pool_checkpoint/checkpoint_removal.ksh \ + functional/pool_checkpoint/checkpoint_rewind.ksh \ + functional/pool_checkpoint/checkpoint_ro_rewind.ksh \ + functional/pool_checkpoint/checkpoint_sm_scale.ksh \ + functional/pool_checkpoint/checkpoint_twice.ksh \ + functional/pool_checkpoint/checkpoint_vdev_add.ksh \ + functional/pool_checkpoint/checkpoint_zdb.ksh \ + functional/pool_checkpoint/checkpoint_zhack_feat.ksh \ + functional/pool_checkpoint/cleanup.ksh \ + functional/pool_checkpoint/setup.ksh \ + functional/pool_names/pool_names_001_pos.ksh \ + functional/pool_names/pool_names_002_neg.ksh \ + functional/poolversion/cleanup.ksh \ + functional/poolversion/poolversion_001_pos.ksh \ + functional/poolversion/poolversion_002_pos.ksh \ + functional/poolversion/setup.ksh \ + functional/privilege/cleanup.ksh \ + functional/privilege/privilege_001_pos.ksh \ + functional/privilege/privilege_002_pos.ksh \ + functional/privilege/setup.ksh \ + functional/procfs/cleanup.ksh \ + functional/procfs/pool_state.ksh \ + functional/procfs/procfs_list_basic.ksh \ + functional/procfs/procfs_list_concurrent_readers.ksh \ + functional/procfs/procfs_list_stale_read.ksh \ + functional/procfs/setup.ksh \ + functional/projectquota/cleanup.ksh \ + functional/projectquota/projectid_001_pos.ksh \ + functional/projectquota/projectid_002_pos.ksh \ + functional/projectquota/projectid_003_pos.ksh \ + functional/projectquota/projectquota_001_pos.ksh \ + functional/projectquota/projectquota_002_pos.ksh \ + functional/projectquota/projectquota_003_pos.ksh \ + functional/projectquota/projectquota_004_neg.ksh \ + functional/projectquota/projectquota_005_pos.ksh \ + functional/projectquota/projectquota_006_pos.ksh \ + functional/projectquota/projectquota_007_pos.ksh \ + functional/projectquota/projectquota_008_pos.ksh \ + functional/projectquota/projectquota_009_pos.ksh \ + functional/projectquota/projectspace_001_pos.ksh \ + functional/projectquota/projectspace_002_pos.ksh \ + functional/projectquota/projectspace_003_pos.ksh \ + functional/projectquota/projectspace_004_pos.ksh \ + functional/projectquota/projecttree_001_pos.ksh \ + functional/projectquota/projecttree_002_pos.ksh \ + functional/projectquota/projecttree_003_neg.ksh \ + functional/projectquota/setup.ksh \ + functional/quota/cleanup.ksh \ + functional/quota/quota_001_pos.ksh \ + functional/quota/quota_002_pos.ksh \ + functional/quota/quota_003_pos.ksh \ + functional/quota/quota_004_pos.ksh \ + functional/quota/quota_005_pos.ksh \ + functional/quota/quota_006_neg.ksh \ + functional/quota/setup.ksh \ + functional/raidz/cleanup.ksh \ + functional/raidz/raidz_001_neg.ksh \ + functional/raidz/raidz_002_pos.ksh \ + functional/raidz/raidz_003_pos.ksh \ + functional/raidz/raidz_004_pos.ksh \ + functional/raidz/setup.ksh \ + functional/redacted_send/cleanup.ksh \ + functional/redacted_send/redacted_compressed.ksh \ + functional/redacted_send/redacted_contents.ksh \ + functional/redacted_send/redacted_deleted.ksh \ + functional/redacted_send/redacted_disabled_feature.ksh \ + functional/redacted_send/redacted_embedded.ksh \ + functional/redacted_send/redacted_holes.ksh \ + functional/redacted_send/redacted_incrementals.ksh \ + functional/redacted_send/redacted_largeblocks.ksh \ + functional/redacted_send/redacted_many_clones.ksh \ + functional/redacted_send/redacted_mixed_recsize.ksh \ + functional/redacted_send/redacted_mounts.ksh \ + functional/redacted_send/redacted_negative.ksh \ + functional/redacted_send/redacted_origin.ksh \ + functional/redacted_send/redacted_panic.ksh \ + functional/redacted_send/redacted_props.ksh \ + functional/redacted_send/redacted_resume.ksh \ + functional/redacted_send/redacted_size.ksh \ + functional/redacted_send/redacted_volume.ksh \ + functional/redacted_send/setup.ksh \ + functional/redundancy/cleanup.ksh \ + functional/redundancy/redundancy_draid1.ksh \ + functional/redundancy/redundancy_draid2.ksh \ + functional/redundancy/redundancy_draid3.ksh \ + functional/redundancy/redundancy_draid_damaged1.ksh \ + functional/redundancy/redundancy_draid_damaged2.ksh \ + functional/redundancy/redundancy_draid.ksh \ + functional/redundancy/redundancy_draid_spare1.ksh \ + functional/redundancy/redundancy_draid_spare2.ksh \ + functional/redundancy/redundancy_draid_spare3.ksh \ + functional/redundancy/redundancy_mirror.ksh \ + functional/redundancy/redundancy_raidz1.ksh \ + functional/redundancy/redundancy_raidz2.ksh \ + functional/redundancy/redundancy_raidz3.ksh \ + functional/redundancy/redundancy_raidz.ksh \ + functional/redundancy/redundancy_stripe.ksh \ + functional/redundancy/setup.ksh \ + functional/refquota/cleanup.ksh \ + functional/refquota/refquota_001_pos.ksh \ + functional/refquota/refquota_002_pos.ksh \ + functional/refquota/refquota_003_pos.ksh \ + functional/refquota/refquota_004_pos.ksh \ + functional/refquota/refquota_005_pos.ksh \ + functional/refquota/refquota_006_neg.ksh \ + functional/refquota/refquota_007_neg.ksh \ + functional/refquota/refquota_008_neg.ksh \ + functional/refquota/setup.ksh \ + functional/refreserv/cleanup.ksh \ + functional/refreserv/refreserv_001_pos.ksh \ + functional/refreserv/refreserv_002_pos.ksh \ + functional/refreserv/refreserv_003_pos.ksh \ + functional/refreserv/refreserv_004_pos.ksh \ + functional/refreserv/refreserv_005_pos.ksh \ + functional/refreserv/refreserv_multi_raidz.ksh \ + functional/refreserv/refreserv_raidz.ksh \ + functional/refreserv/setup.ksh \ + functional/removal/cleanup.ksh \ + functional/removal/removal_all_vdev.ksh \ + functional/removal/removal_cancel.ksh \ + functional/removal/removal_check_space.ksh \ + functional/removal/removal_condense_export.ksh \ + functional/removal/removal_multiple_indirection.ksh \ + functional/removal/removal_nopwrite.ksh \ + functional/removal/removal_remap_deadlists.ksh \ + functional/removal/removal_reservation.ksh \ + functional/removal/removal_resume_export.ksh \ + functional/removal/removal_sanity.ksh \ + functional/removal/removal_with_add.ksh \ + functional/removal/removal_with_create_fs.ksh \ + functional/removal/removal_with_dedup.ksh \ + functional/removal/removal_with_errors.ksh \ + functional/removal/removal_with_export.ksh \ + functional/removal/removal_with_faulted.ksh \ + functional/removal/removal_with_ganging.ksh \ + functional/removal/removal_with_remove.ksh \ + functional/removal/removal_with_scrub.ksh \ + functional/removal/removal_with_send.ksh \ + functional/removal/removal_with_send_recv.ksh \ + functional/removal/removal_with_snapshot.ksh \ + functional/removal/removal_with_write.ksh \ + functional/removal/removal_with_zdb.ksh \ + functional/removal/remove_attach_mirror.ksh \ + functional/removal/remove_expanded.ksh \ + functional/removal/remove_indirect.ksh \ + functional/removal/remove_mirror.ksh \ + functional/removal/remove_mirror_sanity.ksh \ + functional/removal/remove_raidz.ksh \ + functional/rename_dirs/cleanup.ksh \ + functional/rename_dirs/rename_dirs_001_pos.ksh \ + functional/rename_dirs/setup.ksh \ + functional/replacement/attach_import.ksh \ + functional/replacement/attach_multiple.ksh \ + functional/replacement/attach_rebuild.ksh \ + functional/replacement/attach_resilver.ksh \ + functional/replacement/cleanup.ksh \ + functional/replacement/detach.ksh \ + functional/replacement/rebuild_disabled_feature.ksh \ + functional/replacement/rebuild_multiple.ksh \ + functional/replacement/rebuild_raidz.ksh \ + functional/replacement/replace_import.ksh \ + functional/replacement/replace_rebuild.ksh \ + functional/replacement/replace_resilver.ksh \ + functional/replacement/resilver_restart_001.ksh \ + functional/replacement/resilver_restart_002.ksh \ + functional/replacement/scrub_cancel.ksh \ + functional/replacement/setup.ksh \ + functional/reservation/cleanup.ksh \ + functional/reservation/reservation_001_pos.ksh \ + functional/reservation/reservation_002_pos.ksh \ + functional/reservation/reservation_003_pos.ksh \ + functional/reservation/reservation_004_pos.ksh \ + functional/reservation/reservation_005_pos.ksh \ + functional/reservation/reservation_006_pos.ksh \ + functional/reservation/reservation_007_pos.ksh \ + functional/reservation/reservation_008_pos.ksh \ + functional/reservation/reservation_009_pos.ksh \ + functional/reservation/reservation_010_pos.ksh \ + functional/reservation/reservation_011_pos.ksh \ + functional/reservation/reservation_012_pos.ksh \ + functional/reservation/reservation_013_pos.ksh \ + functional/reservation/reservation_014_pos.ksh \ + functional/reservation/reservation_015_pos.ksh \ + functional/reservation/reservation_016_pos.ksh \ + functional/reservation/reservation_017_pos.ksh \ + functional/reservation/reservation_018_pos.ksh \ + functional/reservation/reservation_019_pos.ksh \ + functional/reservation/reservation_020_pos.ksh \ + functional/reservation/reservation_021_neg.ksh \ + functional/reservation/reservation_022_pos.ksh \ + functional/reservation/setup.ksh \ + functional/rootpool/cleanup.ksh \ + functional/rootpool/rootpool_002_neg.ksh \ + functional/rootpool/rootpool_003_neg.ksh \ + functional/rootpool/rootpool_007_pos.ksh \ + functional/rootpool/setup.ksh \ + functional/rsend/cleanup.ksh \ + functional/rsend/recv_dedup_encrypted_zvol.ksh \ + functional/rsend/recv_dedup.ksh \ + functional/rsend/rsend_001_pos.ksh \ + functional/rsend/rsend_002_pos.ksh \ + functional/rsend/rsend_003_pos.ksh \ + functional/rsend/rsend_004_pos.ksh \ + functional/rsend/rsend_005_pos.ksh \ + functional/rsend/rsend_006_pos.ksh \ + functional/rsend/rsend_007_pos.ksh \ + functional/rsend/rsend_008_pos.ksh \ + functional/rsend/rsend_009_pos.ksh \ + functional/rsend/rsend_010_pos.ksh \ + functional/rsend/rsend_011_pos.ksh \ + functional/rsend/rsend_012_pos.ksh \ + functional/rsend/rsend_013_pos.ksh \ + functional/rsend/rsend_014_pos.ksh \ + functional/rsend/rsend_016_neg.ksh \ + functional/rsend/rsend_019_pos.ksh \ + functional/rsend/rsend_020_pos.ksh \ + functional/rsend/rsend_021_pos.ksh \ + functional/rsend/rsend_022_pos.ksh \ + functional/rsend/rsend_024_pos.ksh \ + functional/rsend/rsend_025_pos.ksh \ + functional/rsend/rsend_026_neg.ksh \ + functional/rsend/rsend_027_pos.ksh \ + functional/rsend/rsend_028_neg.ksh \ + functional/rsend/rsend_029_neg.ksh \ + functional/rsend/send-c_embedded_blocks.ksh \ + functional/rsend/send-c_incremental.ksh \ + functional/rsend/send-c_lz4_disabled.ksh \ + functional/rsend/send-c_mixed_compression.ksh \ + functional/rsend/send-cpL_varied_recsize.ksh \ + functional/rsend/send-c_props.ksh \ + functional/rsend/send-c_recv_dedup.ksh \ + functional/rsend/send-c_recv_lz4_disabled.ksh \ + functional/rsend/send-c_resume.ksh \ + functional/rsend/send-c_stream_size_estimate.ksh \ + functional/rsend/send-c_verify_contents.ksh \ + functional/rsend/send-c_verify_ratio.ksh \ + functional/rsend/send-c_volume.ksh \ + functional/rsend/send-c_zstreamdump.ksh \ + functional/rsend/send_doall.ksh \ + functional/rsend/send_encrypted_files.ksh \ + functional/rsend/send_encrypted_hierarchy.ksh \ + functional/rsend/send_encrypted_props.ksh \ + functional/rsend/send_encrypted_truncated_files.ksh \ + functional/rsend/send_freeobjects.ksh \ + functional/rsend/send_holds.ksh \ + functional/rsend/send_hole_birth.ksh \ + functional/rsend/send_invalid.ksh \ + functional/rsend/send-L_toggle.ksh \ + functional/rsend/send_mixed_raw.ksh \ + functional/rsend/send_partial_dataset.ksh \ + functional/rsend/send_raw_ashift.ksh \ + functional/rsend/send_raw_spill_block.ksh \ + functional/rsend/send_realloc_dnode_size.ksh \ + functional/rsend/send_realloc_encrypted_files.ksh \ + functional/rsend/send_realloc_files.ksh \ + functional/rsend/send_spill_block.ksh \ + functional/rsend/send-wR_encrypted_zvol.ksh \ + functional/rsend/setup.ksh \ + functional/scrub_mirror/cleanup.ksh \ + functional/scrub_mirror/scrub_mirror_001_pos.ksh \ + functional/scrub_mirror/scrub_mirror_002_pos.ksh \ + functional/scrub_mirror/scrub_mirror_003_pos.ksh \ + functional/scrub_mirror/scrub_mirror_004_pos.ksh \ + functional/scrub_mirror/setup.ksh \ + functional/slog/cleanup.ksh \ + functional/slog/setup.ksh \ + functional/slog/slog_001_pos.ksh \ + functional/slog/slog_002_pos.ksh \ + functional/slog/slog_003_pos.ksh \ + functional/slog/slog_004_pos.ksh \ + functional/slog/slog_005_pos.ksh \ + functional/slog/slog_006_pos.ksh \ + functional/slog/slog_007_pos.ksh \ + functional/slog/slog_008_neg.ksh \ + functional/slog/slog_009_neg.ksh \ + functional/slog/slog_010_neg.ksh \ + functional/slog/slog_011_neg.ksh \ + functional/slog/slog_012_neg.ksh \ + functional/slog/slog_013_pos.ksh \ + functional/slog/slog_014_pos.ksh \ + functional/slog/slog_015_neg.ksh \ + functional/slog/slog_016_pos.ksh \ + functional/slog/slog_replay_fs_001.ksh \ + functional/slog/slog_replay_fs_002.ksh \ + functional/slog/slog_replay_volume.ksh \ + functional/snapshot/cleanup.ksh \ + functional/snapshot/clone_001_pos.ksh \ + functional/snapshot/rollback_001_pos.ksh \ + functional/snapshot/rollback_002_pos.ksh \ + functional/snapshot/rollback_003_pos.ksh \ + functional/snapshot/setup.ksh \ + functional/snapshot/snapshot_001_pos.ksh \ + functional/snapshot/snapshot_002_pos.ksh \ + functional/snapshot/snapshot_003_pos.ksh \ + functional/snapshot/snapshot_004_pos.ksh \ + functional/snapshot/snapshot_005_pos.ksh \ + functional/snapshot/snapshot_006_pos.ksh \ + functional/snapshot/snapshot_007_pos.ksh \ + functional/snapshot/snapshot_008_pos.ksh \ + functional/snapshot/snapshot_009_pos.ksh \ + functional/snapshot/snapshot_010_pos.ksh \ + functional/snapshot/snapshot_011_pos.ksh \ + functional/snapshot/snapshot_012_pos.ksh \ + functional/snapshot/snapshot_013_pos.ksh \ + functional/snapshot/snapshot_014_pos.ksh \ + functional/snapshot/snapshot_015_pos.ksh \ + functional/snapshot/snapshot_016_pos.ksh \ + functional/snapshot/snapshot_017_pos.ksh \ + functional/snapused/cleanup.ksh \ + functional/snapused/setup.ksh \ + functional/snapused/snapused_001_pos.ksh \ + functional/snapused/snapused_002_pos.ksh \ + functional/snapused/snapused_003_pos.ksh \ + functional/snapused/snapused_004_pos.ksh \ + functional/snapused/snapused_005_pos.ksh \ + functional/sparse/cleanup.ksh \ + functional/sparse/setup.ksh \ + functional/sparse/sparse_001_pos.ksh \ + functional/stat/cleanup.ksh \ + functional/stat/setup.ksh \ + functional/stat/stat_001_pos.ksh \ + functional/suid/cleanup.ksh \ + functional/suid/setup.ksh \ + functional/suid/suid_write_to_none.ksh \ + functional/suid/suid_write_to_sgid.ksh \ + functional/suid/suid_write_to_suid.ksh \ + functional/suid/suid_write_to_suid_sgid.ksh \ + functional/suid/suid_write_zil_replay.ksh \ + functional/trim/autotrim_config.ksh \ + functional/trim/autotrim_integrity.ksh \ + functional/trim/autotrim_trim_integrity.ksh \ + functional/trim/cleanup.ksh \ + functional/trim/setup.ksh \ + functional/trim/trim_config.ksh \ + functional/trim/trim_integrity.ksh \ + functional/trim/trim_l2arc.ksh \ + functional/truncate/cleanup.ksh \ + functional/truncate/setup.ksh \ + functional/truncate/truncate_001_pos.ksh \ + functional/truncate/truncate_002_pos.ksh \ + functional/truncate/truncate_timestamps.ksh \ + functional/upgrade/cleanup.ksh \ + functional/upgrade/setup.ksh \ + functional/upgrade/upgrade_projectquota_001_pos.ksh \ + functional/upgrade/upgrade_readonly_pool.ksh \ + functional/upgrade/upgrade_userobj_001_pos.ksh \ + functional/user_namespace/cleanup.ksh \ + functional/user_namespace/setup.ksh \ + functional/user_namespace/user_namespace_001.ksh \ + functional/user_namespace/user_namespace_002.ksh \ + functional/user_namespace/user_namespace_003.ksh \ + functional/user_namespace/user_namespace_004.ksh \ + functional/userquota/cleanup.ksh \ + functional/userquota/groupspace_001_pos.ksh \ + functional/userquota/groupspace_002_pos.ksh \ + functional/userquota/groupspace_003_pos.ksh \ + functional/userquota/setup.ksh \ + functional/userquota/userquota_001_pos.ksh \ + functional/userquota/userquota_002_pos.ksh \ + functional/userquota/userquota_003_pos.ksh \ + functional/userquota/userquota_004_pos.ksh \ + functional/userquota/userquota_005_neg.ksh \ + functional/userquota/userquota_006_pos.ksh \ + functional/userquota/userquota_007_pos.ksh \ + functional/userquota/userquota_008_pos.ksh \ + functional/userquota/userquota_009_pos.ksh \ + functional/userquota/userquota_010_pos.ksh \ + functional/userquota/userquota_011_pos.ksh \ + functional/userquota/userquota_012_neg.ksh \ + functional/userquota/userquota_013_pos.ksh \ + functional/userquota/userspace_001_pos.ksh \ + functional/userquota/userspace_002_pos.ksh \ + functional/userquota/userspace_003_pos.ksh \ + functional/userquota/userspace_encrypted.ksh \ + functional/userquota/userspace_send_encrypted.ksh \ + functional/vdev_zaps/cleanup.ksh \ + functional/vdev_zaps/setup.ksh \ + functional/vdev_zaps/vdev_zaps_001_pos.ksh \ + functional/vdev_zaps/vdev_zaps_002_pos.ksh \ + functional/vdev_zaps/vdev_zaps_003_pos.ksh \ + functional/vdev_zaps/vdev_zaps_004_pos.ksh \ + functional/vdev_zaps/vdev_zaps_005_pos.ksh \ + functional/vdev_zaps/vdev_zaps_006_pos.ksh \ + functional/vdev_zaps/vdev_zaps_007_pos.ksh \ + functional/write_dirs/cleanup.ksh \ + functional/write_dirs/setup.ksh \ + functional/write_dirs/write_dirs_001_pos.ksh \ + functional/write_dirs/write_dirs_002_pos.ksh \ + functional/xattr/cleanup.ksh \ + functional/xattr/setup.ksh \ + functional/xattr/xattr_001_pos.ksh \ + functional/xattr/xattr_002_neg.ksh \ + functional/xattr/xattr_003_neg.ksh \ + functional/xattr/xattr_004_pos.ksh \ + functional/xattr/xattr_005_pos.ksh \ + functional/xattr/xattr_006_pos.ksh \ + functional/xattr/xattr_007_neg.ksh \ + functional/xattr/xattr_008_pos.ksh \ + functional/xattr/xattr_009_neg.ksh \ + functional/xattr/xattr_010_neg.ksh \ + functional/xattr/xattr_011_pos.ksh \ + functional/xattr/xattr_012_pos.ksh \ + functional/xattr/xattr_013_pos.ksh \ + functional/xattr/xattr_compat.ksh \ + functional/zpool_influxdb/cleanup.ksh \ + functional/zpool_influxdb/setup.ksh \ + functional/zpool_influxdb/zpool_influxdb.ksh \ + functional/zvol/zvol_cli/cleanup.ksh \ + functional/zvol/zvol_cli/setup.ksh \ + functional/zvol/zvol_cli/zvol_cli_001_pos.ksh \ + functional/zvol/zvol_cli/zvol_cli_002_pos.ksh \ + functional/zvol/zvol_cli/zvol_cli_003_neg.ksh \ + functional/zvol/zvol_ENOSPC/cleanup.ksh \ + functional/zvol/zvol_ENOSPC/setup.ksh \ + functional/zvol/zvol_ENOSPC/zvol_ENOSPC_001_pos.ksh \ + functional/zvol/zvol_misc/cleanup.ksh \ + functional/zvol/zvol_misc/setup.ksh \ + functional/zvol/zvol_misc/zvol_misc_001_neg.ksh \ + functional/zvol/zvol_misc/zvol_misc_002_pos.ksh \ + functional/zvol/zvol_misc/zvol_misc_003_neg.ksh \ + functional/zvol/zvol_misc/zvol_misc_004_pos.ksh \ + functional/zvol/zvol_misc/zvol_misc_005_neg.ksh \ + functional/zvol/zvol_misc/zvol_misc_006_pos.ksh \ + functional/zvol/zvol_misc/zvol_misc_fua.ksh \ + functional/zvol/zvol_misc/zvol_misc_hierarchy.ksh \ + functional/zvol/zvol_misc/zvol_misc_rename_inuse.ksh \ + functional/zvol/zvol_misc/zvol_misc_snapdev.ksh \ + functional/zvol/zvol_misc/zvol_misc_trim.ksh \ + functional/zvol/zvol_misc/zvol_misc_volmode.ksh \ + functional/zvol/zvol_misc/zvol_misc_zil.ksh \ + functional/zvol/zvol_stress/cleanup.ksh \ + functional/zvol/zvol_stress/setup.ksh \ + functional/zvol/zvol_stress/zvol_stress.ksh \ + functional/zvol/zvol_swap/cleanup.ksh \ + functional/zvol/zvol_swap/setup.ksh \ + functional/zvol/zvol_swap/zvol_swap_001_pos.ksh \ + functional/zvol/zvol_swap/zvol_swap_002_pos.ksh \ + functional/zvol/zvol_swap/zvol_swap_003_pos.ksh \ + functional/zvol/zvol_swap/zvol_swap_004_pos.ksh \ + functional/zvol/zvol_swap/zvol_swap_005_pos.ksh \ + functional/zvol/zvol_swap/zvol_swap_006_pos.ksh diff --git a/tests/zfs-tests/tests/functional/Makefile.am b/tests/zfs-tests/tests/functional/Makefile.am deleted file mode 100644 index e71172b8e86d..000000000000 --- a/tests/zfs-tests/tests/functional/Makefile.am +++ /dev/null @@ -1,94 +0,0 @@ -SUBDIRS = \ - acl \ - alloc_class \ - arc \ - atime \ - bootfs \ - btree \ - cache \ - cachefile \ - casenorm \ - channel_program \ - chattr \ - checksum \ - clean_mirror \ - cli_root \ - cli_user \ - compression \ - cp_files \ - crtime \ - ctime \ - deadman \ - delegate \ - devices \ - events \ - exec \ - fallocate \ - fault \ - features \ - grow \ - history \ - hkdf \ - inheritance \ - inuse \ - io \ - l2arc \ - large_files \ - largest_pool \ - libzfs \ - limits \ - link_count \ - log_spacemap \ - migration \ - mmap \ - mmp \ - mount \ - mv_files \ - nestedfs \ - no_space \ - nopwrite \ - online_offline \ - pam \ - pool_checkpoint \ - pool_names \ - poolversion \ - privilege \ - procfs \ - projectquota \ - pyzfs \ - quota \ - raidz \ - redacted_send \ - redundancy \ - refquota \ - refreserv \ - removal \ - rename_dirs \ - replacement \ - reservation \ - rootpool \ - rsend \ - scrub_mirror \ - slog \ - snapshot \ - snapused \ - sparse \ - stat \ - suid \ - threadsappend \ - trim \ - truncate \ - upgrade \ - user_namespace \ - userquota \ - vdev_zaps \ - write_dirs \ - xattr \ - zpool_influxdb \ - zvol - -if BUILD_LINUX -SUBDIRS += \ - simd \ - tmpfile -endif diff --git a/tests/zfs-tests/tests/functional/acl/Makefile.am b/tests/zfs-tests/tests/functional/acl/Makefile.am deleted file mode 100644 index d752f63744cb..000000000000 --- a/tests/zfs-tests/tests/functional/acl/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/acl -dist_pkgdata_DATA = \ - acl.cfg \ - acl_common.kshlib - -SUBDIRS = off posix posix-sa diff --git a/tests/zfs-tests/tests/functional/acl/acl.cfg b/tests/zfs-tests/tests/functional/acl/acl.cfg index 8fa85d673bb7..e350f18469d6 100644 --- a/tests/zfs-tests/tests/functional/acl/acl.cfg +++ b/tests/zfs-tests/tests/functional/acl/acl.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/acl/acl_common.kshlib b/tests/zfs-tests/tests/functional/acl/acl_common.kshlib index ba08bcb48bef..2844f5dc13dc 100644 --- a/tests/zfs-tests/tests/functional/acl/acl_common.kshlib +++ b/tests/zfs-tests/tests/functional/acl/acl_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -31,21 +31,6 @@ . $STF_SUITE/tests/functional/acl/acl.cfg . $STF_SUITE/include/libtest.shlib -# -# Get the given file/directory access mode -# -# $1 object -- file or directory -# -function get_mode # -{ - typeset obj=$1 - if (( ${#obj} == 0 )); then - return 1 - fi - - ls -ld $obj | awk '{print $1}' -} - # # Get the given file/directory ACL # @@ -58,7 +43,7 @@ function get_acl # return 1 fi - ls -vd $obj | nawk '(NR != 1) {print $0}' + ls -vd $obj | awk '(NR != 1) {print $0}' } # @@ -73,7 +58,7 @@ function get_compact_acl # return 1 fi - ls -Vd $obj | nawk '(NR != 1) {print $0}' + ls -Vd $obj | awk '(NR != 1) {print $0}' } # @@ -98,7 +83,7 @@ function compare_acls # get_acl $src > $tmpsrc get_acl $tgt > $tmptgt typeset -i ret=0 - diff $tmpsrc $tmptgt > /dev/null 2>&1 + cmp $tmpsrc $tmptgt > /dev/null ret=$? rm -f $tmpsrc $tmptgt @@ -108,43 +93,13 @@ function compare_acls # get_compact_acl $src > $tmpsrc get_compact_acl $tgt > $tmptgt - diff $tmpsrc $tmptgt > /dev/null 2>&1 + cmp $tmpsrc $tmptgt > /dev/null ret=$? rm -f $tmpsrc $tmptgt return $ret } -# -# Check that the given two objects have the same modes. -# Return 0, if their modes are equal with each other. Otherwise, return 1. -# -# $1 source object -# $2 target object -# -function compare_modes # -{ - typeset src=$1 - typeset tgt=$2 - typeset -i i=0 - set -A mode - - (( ${#src} == 0 || ${#tgt} == 0 )) && return 1 - [[ $src == $tgt ]] && return 0 - - typeset obj - for obj in $src $tgt - do - mode[i]=$(get_mode $obj) - - (( i = i + 1 )) - done - - [[ ${mode[0]} != ${mode[1]} ]] && return 1 - - return 0 -} - # # Check that the given two objects have the same xattrs. # Return 0, if their xattrs are equal with each other. Otherwise, return 1. @@ -166,69 +121,13 @@ function compare_xattrs # get_xattr $src > $tmpsrc get_xattr $tgt > $tmptgt typeset -i ret=0 - diff $tmpsrc $tmptgt > /dev/null 2>&1 + cmp $tmpsrc $tmptgt > /dev/null ret=$? rm -f $tmpsrc $tmptgt return $ret } -# -# Check '+' is set for a given file/directory with 'ls [-l]' command -# -# $1 object -- file or directory. -# -function plus_sign_check_l # -{ - typeset obj=$1 - if (( ${#obj} == 0 )); then - return 1 - fi - - ls -ld $obj | awk '{print $1}' | grep "+$" > /dev/null - - return $? -} - -# -# Check '+' is set for a given file/directory with 'ls [-v]' command -# -# $1 object -- file or directory. -# -function plus_sign_check_v # -{ - typeset obj=$1 - if (( ${#obj} == 0 )); then - return 1 - fi - - ls -vd $obj | awk '(NR == 1) {print $1}' | grep "+$" > /dev/null - - return $? -} - -# -# A wrapper function of c program -# -# $1 legal login name -# $2-n commands and options -# -function chgusr_exec # [...] -{ - chg_usr_exec $@ - return $? -} - -# -# Export the current user for the following usr_exec operating. -# -# $1 legal login name -# -function set_cur_usr # -{ - export ZFS_ACL_CUR_USER=$1 -} - # # Run commands by $ZFS_ACL_CUR_USER # @@ -237,83 +136,6 @@ function set_cur_usr # function usr_exec # [...] { chg_usr_exec "$ZFS_ACL_CUR_USER" $@ - return $? -} - -# -# Count how many ACEs for the specified file or directory. -# -# $1 file or directory name -# -function count_ACE # -{ - if [[ ! -e $1 ]]; then - log_note "Need input file or directory name." - return 1 - fi - - ls -vd $1 | nawk 'BEGIN {count=0} - (NR != 1)&&(/[0-9]:/) {count++} - END {print count}' - - return 0 -} - -# -# Get specified number ACE content of specified file or directory. -# -# $1 file or directory name -# $2 specified number -# -function get_ACE # -{ - if [[ ! -e $1 || $2 -ge $(count_ACE $1) ]]; then - return 1 - fi - - typeset file=$1 - typeset -i num=$2 - typeset format=${3:-verbose} - typeset -i next_num=-1 - - typeset tmpfile=$TEST_BASE_DIR/tmp_get_ACE.$$ - typeset line="" - typeset args - - case $format in - verbose) args="-vd" - ;; - compact) args="-Vd" - ;; - *) log_fail "Invalid parameter as ($format), " \ - "only verbose|compact is supported." - ;; - esac - - ls $args $file > $tmpfile - (( $? != 0 )) && log_fail "FAIL: ls $args $file > $tmpfile" - while read line; do - [[ -z $line ]] && continue - if [[ $args == -vd ]]; then - if [[ $line == "$num":* ]]; then - (( next_num = num + 1 )) - fi - if [[ $line == "$next_num":* ]]; then - break - fi - if (( next_num != -1 )); then - print -n $line - fi - else - if (( next_num == num )); then - print -n $line - fi - (( next_num += 1 )) - fi - done < $tmpfile - - rm -f $tmpfile - (( $? != 0 )) && log_fail "FAIL: rm -f $tmpfile" } # @@ -345,57 +167,6 @@ function cleanup fi } -# -# According to specified access or acl_spec, do relevant operating by using the -# specified user. -# -# $1 specified user -# $2 node -# $3 acl_spec or access -# -function rwx_node #user node acl_spec|access -{ - typeset user=$1 - typeset node=$2 - typeset acl_spec=$3 - - if [[ $user == "" || $node == "" || $acl_spec == "" ]]; then - log_note "node or acl_spec are not defined." - return 1 - fi - - if [[ -d $node ]]; then - case $acl_spec in - *:read_data:*|read_data) - chgusr_exec $user ls -l $node > /dev/null 2>&1 - return $? ;; - *:write_data:*|write_data) - if [[ -f ${node}/tmpfile ]]; then - log_must rm -f ${node}/tmpfile - fi - chgusr_exec $user touch ${node}/tmpfile > \ - /dev/null 2>&1 - return $? ;; - *"execute:"*|execute) - chgusr_exec $user find $node > /dev/null 2>&1 - return $? ;; - esac - else - case $acl_spec in - *:read_data:*|read_data) - chgusr_exec $user cat $node > /dev/null 2>&1 - return $? ;; - *:write_data:*|write_data) - chgusr_exec $user dd if=/usr/bin/ls of=$node > \ - /dev/null 2>&1 - return $? ;; - *"execute:"*|execute) - ZFS_ACL_ERR_STR=$(chgusr_exec $user $node 2>&1) - return $? ;; - esac - fi -} - # # Get the given file/directory xattr # @@ -409,133 +180,11 @@ function get_xattr # return 1 fi - for xattr in `runat $obj ls | \ - grep -E -v -e SUNWattr_ro -e SUNWattr_rw` ; do - runat $obj sum $xattr + for xattr in $(runat $obj ls | grep -v 'SUNWattr_r[ow]'); do + runat $obj cksum $xattr done } -# -# Get the owner of a file/directory -# -function get_owner #node -{ - typeset node=$1 - typeset value - - if [[ -z $node ]]; then - log_fail "node are not defined." - fi - - if [[ -d $node ]]; then - value=$(ls -dl $node | awk '{print $3}') - elif [[ -e $node ]]; then - value=$(ls -l $node | awk '{print $3}') - fi - - echo $value -} - -# -# Get the group of a file/directory -# -function get_group #node -{ - typeset node=$1 - typeset value - - if [[ -z $node ]]; then - log_fail "node are not defined." - fi - - if [[ -d $node ]]; then - value=$(ls -dl $node | awk '{print $4}') - elif [[ -e $node ]]; then - value=$(ls -l $node | awk '{print $4}') - fi - - echo $value -} - - -# -# Get the group name that a UID belongs to -# -function get_user_group #uid -{ - typeset uid=$1 - typeset value - - if [[ -z $uid ]]; then - log_fail "UID not defined." - fi - - value=$(id $uid) - - if [[ $? -eq 0 ]]; then - value=${value##*\(} - value=${value%%\)*} - echo $value - else - log_fail "Invalid UID (uid)." - fi -} - -# -# Get the specified item of the specified string -# -# $1: Item number, count from 0. -# $2-n: strings -# -function getitem -{ - typeset -i n=$1 - shift - - (( n += 1 )) - eval echo \${$n} -} - -# -# This function calculate the specified directory files checksum and write -# to the specified array. -# -# $1 directory in which the files will be cksum. -# $2 file array name which was used to store file cksum information. -# $3 attribute array name which was used to store attribute information. -# -function cksum_files # -{ - typeset dir=$1 - typeset farr_name=$2 - typeset aarr_name=$3 - - [[ ! -d $dir ]] && return - typeset oldpwd=$PWD - cd $dir - typeset files=$(ls file*) - - typeset -i i=0 - typeset -i n=0 - while (( i < NUM_FILE )); do - typeset f=$(getitem $i $files) - eval $farr_name[$i]=\$\(\cksum $f\) - - typeset -i j=0 - while (( j < NUM_ATTR )); do - eval $aarr_name[$n]=\$\(\runat \$f \cksum \ - attribute.$j\) - - (( j += 1 )) - (( n += 1 )) - done - - (( i += 1 )) - done - - cd $oldpwd -} - # # This function compare two cksum results array. # @@ -563,26 +212,6 @@ function compare_cksum # return 0 } -# -# This function calculate all the files cksum information in current directory -# and output them to the specified file. -# -# $1 directory from which the files will be cksum. -# $2 cksum output file -# -function record_cksum # -{ - typeset dir=$1 - typeset outfile=$2 - - [[ ! -d ${outfile%/*} ]] && usr_exec mkdir -p ${outfile%/*} - - usr_exec cd $dir ; find . -depth -type f -exec cksum {} \\\; | \ - sort > $outfile - usr_exec cd $dir ; find . -depth -type f -xattr -exec runat {} \ - cksum attribute* \\\; | sort >> $outfile -} - # # The function create_files creates the directories and files that the script # will operate on to test extended attribute functionality. diff --git a/tests/zfs-tests/tests/functional/acl/off/.gitignore b/tests/zfs-tests/tests/functional/acl/off/.gitignore deleted file mode 100644 index f3c93191cea9..000000000000 --- a/tests/zfs-tests/tests/functional/acl/off/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/dosmode_readonly_write diff --git a/tests/zfs-tests/tests/functional/acl/off/Makefile.am b/tests/zfs-tests/tests/functional/acl/off/Makefile.am deleted file mode 100644 index 36aa13dd03fd..000000000000 --- a/tests/zfs-tests/tests/functional/acl/off/Makefile.am +++ /dev/null @@ -1,16 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/acl/off - -dist_pkgdata_SCRIPTS = \ - dosmode.ksh \ - posixmode.ksh \ - cleanup.ksh \ - setup.ksh - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/acl/off - -if BUILD_FREEBSD -pkgexec_PROGRAMS = dosmode_readonly_write -dosmode_readonly_write_SOURCES = dosmode_readonly_write.c -endif diff --git a/tests/zfs-tests/tests/functional/acl/off/cleanup.ksh b/tests/zfs-tests/tests/functional/acl/off/cleanup.ksh index bb58a8cf2e7b..58ce22d547a2 100755 --- a/tests/zfs-tests/tests/functional/acl/off/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/acl/off/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/acl/off/dosmode.ksh b/tests/zfs-tests/tests/functional/acl/off/dosmode.ksh index e232dfd525c6..fa304aeb917d 100755 --- a/tests/zfs-tests/tests/functional/acl/off/dosmode.ksh +++ b/tests/zfs-tests/tests/functional/acl/off/dosmode.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -31,8 +31,6 @@ # DESCRIPTION: # Verify that DOS mode flags function correctly. # -# These flags are not currently exposed on Linux, so the test is -# only useful on FreeBSD. # # STRATEGY: # 1. ARCHIVE @@ -56,17 +54,26 @@ function hasflag typeset flag=$1 typeset path=$2 - ls -lo $path | awk '{ gsub(",", "\n", $5); print $5 }' | grep -qxF $flag + if is_linux; then + read_dos_attributes $path + else + ls -lo $path | awk '{ print $5 }' + fi | grep -qwF $flag } log_assert "Verify DOS mode flags function correctly" log_onexit cleanup -tests_base=$STF_SUITE/tests/functional/acl/off testfile=$TESTDIR/testfile owner=$ZFS_ACL_STAFF1 other=$ZFS_ACL_STAFF2 +if is_linux; then + changeflags=write_dos_attributes +else + changeflags=chflags +fi + # # ARCHIVE # @@ -75,36 +82,40 @@ other=$ZFS_ACL_STAFF2 # log_must touch $testfile log_must hasflag uarch $testfile -log_must chflags nouarch $testfile +log_must $changeflags nouarch $testfile log_must hasflag - $testfile log_must touch $testfile -log_must hasflag uarch $testfile +if ! is_linux; then + log_must hasflag uarch $testfile +fi log_must rm $testfile log_must user_run $owner touch $testfile log_must hasflag uarch $testfile -log_must user_run $owner chflags nouarch $testfile -log_mustnot user_run $other chflags uarch $testfile +log_must user_run $owner $changeflags nouarch $testfile +log_mustnot user_run $other $changeflags uarch $testfile log_must hasflag - $testfile log_must user_run $owner touch $testfile -log_mustnot user_run $other chflags nouarch $testfile -log_must hasflag uarch $testfile +log_mustnot user_run $other $changeflags nouarch $testfile +if ! is_linux; then + log_must hasflag uarch $testfile +fi log_must user_run $owner rm $testfile # # HIDDEN # log_must touch $testfile -log_must chflags hidden $testfile +log_must $changeflags hidden $testfile log_must hasflag hidden $testfile -log_must chflags 0 $testfile +log_must $changeflags 0 $testfile log_must hasflag - $testfile log_must rm $testfile log_must user_run $owner touch $testfile -log_must user_run $owner chflags hidden $testfile -log_mustnot user_run $other chflags nohidden $testfile +log_must user_run $owner $changeflags hidden $testfile +log_mustnot user_run $other $changeflags nohidden $testfile log_must hasflag hidden $testfile -log_must user_run $owner chflags 0 $testfile -log_mustnot user_run $other chflags hidden $testfile +log_must user_run $owner $changeflags 0 $testfile +log_mustnot user_run $other $changeflags hidden $testfile log_must hasflag - $testfile log_must user_run $owner rm $testfile @@ -113,17 +124,17 @@ log_must user_run $owner rm $testfile # OFFLINE # log_must touch $testfile -log_must chflags offline $testfile +log_must $changeflags offline $testfile log_must hasflag offline $testfile -log_must chflags 0 $testfile +log_must $changeflags 0 $testfile log_must hasflag - $testfile log_must rm $testfile log_must user_run $owner touch $testfile -log_must user_run $owner chflags offline $testfile -log_mustnot user_run $other chflags nooffline $testfile +log_must user_run $owner $changeflags offline $testfile +log_mustnot user_run $other $changeflags nooffline $testfile log_must hasflag offline $testfile -log_must user_run $owner chflags 0 $testfile -log_mustnot user_run $other chflags offline $testfile +log_must user_run $owner $changeflags 0 $testfile +log_mustnot user_run $other $changeflags offline $testfile log_must hasflag - $testfile log_must user_run $owner rm $testfile @@ -134,21 +145,23 @@ log_must user_run $owner rm $testfile # but root is always allowed the operation. # log_must touch $testfile -log_must chflags rdonly $testfile +log_must $changeflags rdonly $testfile log_must hasflag rdonly $testfile log_must eval "echo 'root write allowed' >> $testfile" log_must cat $testfile -log_must chflags 0 $testfile -log_must hasflag - $tesfile +log_must $changeflags 0 $testfile +log_must hasflag - $testfile log_must rm $testfile # It is required to still be able to write to an fd that was opened RW before # READONLY is set. We have a special test program for that. log_must user_run $owner touch $testfile -log_mustnot user_run $other chflags rdonly $testfile -log_must user_run $owner $tests_base/dosmode_readonly_write $testfile -log_mustnot user_run $other chflags nordonly $testfile +log_mustnot user_run $other $changeflags rdonly $testfile +log_must user_run $owner dosmode_readonly_write $testfile +log_mustnot user_run $other $changeflags nordonly $testfile log_must hasflag rdonly $testfile -log_mustnot user_run $owner "echo 'user write forbidden' >> $testfile" +if ! is_linux; then + log_mustnot user_run $owner "echo 'user write forbidden' >> $testfile" +fi log_must eval "echo 'root write allowed' >> $testfile" # We are still allowed to read and remove the file when READONLY is set. log_must user_run $owner cat $testfile @@ -157,24 +170,23 @@ log_must user_run $owner rm $testfile # # REPARSE # -# FIXME: does not work, not sure if broken or testing wrong -# +# not allowed to be changed # # SPARSE # log_must truncate -s 1m $testfile -log_must chflags sparse $testfile +log_must $changeflags sparse $testfile log_must hasflag sparse $testfile -log_must chflags 0 $testfile +log_must $changeflags 0 $testfile log_must hasflag - $testfile log_must rm $testfile log_must user_run $owner truncate -s 1m $testfile -log_must user_run $owner chflags sparse $testfile -log_mustnot user_run $other chflags nosparse $testfile +log_must user_run $owner $changeflags sparse $testfile +log_mustnot user_run $other $changeflags nosparse $testfile log_must hasflag sparse $testfile -log_must user_run $owner chflags 0 $testfile -log_mustnot user_run $other chflags sparse $testfile +log_must user_run $owner $changeflags 0 $testfile +log_mustnot user_run $other $changeflags sparse $testfile log_must hasflag - $testfile log_must user_run $owner rm $testfile @@ -182,17 +194,17 @@ log_must user_run $owner rm $testfile # SYSTEM # log_must touch $testfile -log_must chflags system $testfile +log_must $changeflags system $testfile log_must hasflag system $testfile -log_must chflags 0 $testfile +log_must $changeflags 0 $testfile log_must hasflag - $testfile log_must rm $testfile log_must user_run $owner touch $testfile -log_must user_run $owner chflags system $testfile -log_mustnot user_run $other chflags nosystem $testfile +log_must user_run $owner $changeflags system $testfile +log_mustnot user_run $other $changeflags nosystem $testfile log_must hasflag system $testfile -log_must user_run $owner chflags 0 $testfile -log_mustnot user_run $other chflags system $testfile +log_must user_run $owner $changeflags 0 $testfile +log_mustnot user_run $other $changeflags system $testfile log_must hasflag - $testfile log_must user_run $owner rm $testfile diff --git a/tests/zfs-tests/tests/functional/acl/off/posixmode.ksh b/tests/zfs-tests/tests/functional/acl/off/posixmode.ksh index 63870caa3234..df278ae2366c 100755 --- a/tests/zfs-tests/tests/functional/acl/off/posixmode.ksh +++ b/tests/zfs-tests/tests/functional/acl/off/posixmode.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/acl/off/setup.ksh b/tests/zfs-tests/tests/functional/acl/off/setup.ksh index 9a5b598a59d7..9a0b949ad938 100755 --- a/tests/zfs-tests/tests/functional/acl/off/setup.ksh +++ b/tests/zfs-tests/tests/functional/acl/off/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/acl/posix-sa/Makefile.am b/tests/zfs-tests/tests/functional/acl/posix-sa/Makefile.am deleted file mode 100644 index 31d1237ce265..000000000000 --- a/tests/zfs-tests/tests/functional/acl/posix-sa/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/acl/posix-sa -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - posix_001_pos.ksh \ - posix_002_pos.ksh \ - posix_003_pos.ksh \ - posix_004_pos.ksh diff --git a/tests/zfs-tests/tests/functional/acl/posix-sa/cleanup.ksh b/tests/zfs-tests/tests/functional/acl/posix-sa/cleanup.ksh index bb58a8cf2e7b..58ce22d547a2 100755 --- a/tests/zfs-tests/tests/functional/acl/posix-sa/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/acl/posix-sa/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/acl/posix-sa/setup.ksh b/tests/zfs-tests/tests/functional/acl/posix-sa/setup.ksh index d8bf8a638e7b..3bb3760507ca 100755 --- a/tests/zfs-tests/tests/functional/acl/posix-sa/setup.ksh +++ b/tests/zfs-tests/tests/functional/acl/posix-sa/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/acl/posix/Makefile.am b/tests/zfs-tests/tests/functional/acl/posix/Makefile.am deleted file mode 100644 index e63f63185afe..000000000000 --- a/tests/zfs-tests/tests/functional/acl/posix/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/acl/posix -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - posix_001_pos.ksh \ - posix_002_pos.ksh \ - posix_003_pos.ksh \ - posix_004_pos.ksh diff --git a/tests/zfs-tests/tests/functional/acl/posix/cleanup.ksh b/tests/zfs-tests/tests/functional/acl/posix/cleanup.ksh index bb58a8cf2e7b..58ce22d547a2 100755 --- a/tests/zfs-tests/tests/functional/acl/posix/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/acl/posix/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/acl/posix/posix_001_pos.ksh b/tests/zfs-tests/tests/functional/acl/posix/posix_001_pos.ksh index d62bf9c346b6..803c7816fedc 100755 --- a/tests/zfs-tests/tests/functional/acl/posix/posix_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/acl/posix/posix_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -57,41 +57,41 @@ log_onexit cleanup log_note "Testing access to FILE" log_must touch $TESTDIR/file.0 log_must setfacl -m g:$ZFS_ACL_STAFF_GROUP:rw $TESTDIR/file.0 -getfacl $TESTDIR/file.0 2> /dev/null | egrep -q \ - "^group:$ZFS_ACL_STAFF_GROUP:rw-$" -if [ "$?" -eq "0" ]; then - # Should be able to write to file - log_must user_run $ZFS_ACL_STAFF1 \ - "echo 'echo test > /dev/null' > $TESTDIR/file.0" +if ! getfacl $TESTDIR/file.0 2> /dev/null | + grep -qFx "group:$ZFS_ACL_STAFF_GROUP:rw-" +then + log_note "$(getfacl $TESTDIR/file.0 2> /dev/null)" + log_fail "Group '$ZFS_ACL_STAFF_GROUP' does not have 'rw' as specified" +fi - # Since $TESTDIR is 777, create a new dir with controlled permissions - # for testing that creating a new file is not allowed. - log_must mkdir $TESTDIR/dir.0 - log_must chmod 700 $TESTDIR/dir.0 - log_must setfacl -m g:$ZFS_ACL_STAFF_GROUP:rw $TESTDIR/dir.0 - # Confirm permissions - ls -l $TESTDIR |grep "dir.0" |grep -q "drwxrw----+" - if [ "$?" -ne "0" ]; then - msk=$(ls -l $TESTDIR |grep "dir.0" | awk '{print $1}') - log_note "expected mask drwxrw----+ but found $msk" - log_fail "Expected permissions were not set." - fi - getfacl $TESTDIR/dir.0 2> /dev/null | egrep -q \ - "^group:$ZFS_ACL_STAFF_GROUP:rw-$" - if [ "$?" -ne "0" ]; then - acl=$(getfacl $TESTDIR/dir.0 2> /dev/null) - log_note $acl - log_fail "ACL group:$ZFS_ACL_STAFF_GROUP:rw- was not set." - fi - # Should NOT be able to create new file - log_mustnot user_run $ZFS_ACL_STAFF1 "touch $TESTDIR/dir.0/file.1" +# Should be able to write to file +log_must user_run $ZFS_ACL_STAFF1 \ + "echo 'echo test > /dev/null' > $TESTDIR/file.0" - # Root should be able to run file, but not user - chmod +x $TESTDIR/file.0 - log_must $TESTDIR/file.0 - log_mustnot user_run $ZFS_ACL_STAFF1 $TESTDIR/file.0 +# Since $TESTDIR is 777, create a new dir with controlled permissions +# for testing that creating a new file is not allowed. +log_must mkdir $TESTDIR/dir.0 +log_must chmod 700 $TESTDIR/dir.0 +log_must setfacl -m g:$ZFS_ACL_STAFF_GROUP:rw $TESTDIR/dir.0 +# Confirm permissions +msk=$(ls -ld $TESTDIR/dir.0 | awk '{print $1}') +if ! [ "$msk" = "drwxrw----+" ]; then + log_note "expected mask drwxrw----+ but found $msk" + log_fail "Expected permissions were not set." +fi - log_pass "POSIX ACL mode works on files" -else - log_fail "Group '$ZFS_ACL_STAFF_GROUP' does not have 'rw' as specified" +if ! getfacl $TESTDIR/dir.0 2> /dev/null | + grep -qFx "group:$ZFS_ACL_STAFF_GROUP:rw-" +then + log_note "$(getfacl $TESTDIR/dir.0 2> /dev/null)" + log_fail "ACL group:$ZFS_ACL_STAFF_GROUP:rw- was not set." fi +# Should NOT be able to create new file +log_mustnot user_run $ZFS_ACL_STAFF1 "touch $TESTDIR/dir.0/file.1" + +# Root should be able to run file, but not user +chmod +x $TESTDIR/file.0 +log_must $TESTDIR/file.0 +log_mustnot user_run $ZFS_ACL_STAFF1 $TESTDIR/file.0 + +log_pass "POSIX ACL mode works on files" diff --git a/tests/zfs-tests/tests/functional/acl/posix/posix_002_pos.ksh b/tests/zfs-tests/tests/functional/acl/posix/posix_002_pos.ksh index d9b5036458f8..4c0bc372b78b 100755 --- a/tests/zfs-tests/tests/functional/acl/posix/posix_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/acl/posix/posix_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -53,15 +53,14 @@ log_must mkdir $TESTDIR/dir.0 log_must chmod 700 $TESTDIR/dir.0 log_must setfacl -m g:$ZFS_ACL_STAFF_GROUP:wx $TESTDIR/dir.0 # Confirm permissions -ls -l $TESTDIR |grep "dir.0" |grep -q "drwx-wx---+" -if [ "$?" -ne "0" ]; then - msk=$(ls -l $TESTDIR |grep "dir.0" | awk '{print $1}') +if ! ls -l $TESTDIR | grep "dir.0" | grep -q "drwx-wx---+"; then + msk=$(ls -l $TESTDIR | awk '/dir.0/ {print $1}') log_note "expected mask drwx-wx---+ but found $msk" log_fail "Expected permissions were not set." fi -getfacl $TESTDIR/dir.0 2> /dev/null | egrep -q \ - "^group:$ZFS_ACL_STAFF_GROUP:-wx$" -if [ "$?" -eq "0" ]; then +if getfacl $TESTDIR/dir.0 2> /dev/null | + grep -q "^group:$ZFS_ACL_STAFF_GROUP:-wx$" +then # Should be able to create file in directory log_must user_run $ZFS_ACL_STAFF1 "touch $TESTDIR/dir.0/file.0" diff --git a/tests/zfs-tests/tests/functional/acl/posix/posix_003_pos.ksh b/tests/zfs-tests/tests/functional/acl/posix/posix_003_pos.ksh index 1b04a024f2ad..2a9c173ccb2c 100755 --- a/tests/zfs-tests/tests/functional/acl/posix/posix_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/acl/posix/posix_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -43,16 +43,14 @@ log_note "Testing access to DIRECTORY" log_must mkdir $ACLDIR log_must setfacl -m g:$ZFS_ACL_STAFF_GROUP:wx $ACLDIR log_must setfacl -d -m g:$ZFS_ACL_STAFF_GROUP:wx $ACLDIR -getfacl $ACLDIR 2> /dev/null | egrep -q "$acl_str1" -if [ "$?" -eq "0" ]; then - getfacl $ACLDIR 2> /dev/null | egrep -q "$acl_str2" -fi -if [ "$?" -eq "0" ]; then +if getfacl $ACLDIR 2> /dev/null | grep -q "$acl_str1" && + getfacl $ACLDIR 2> /dev/null | grep -q "$acl_str2" +then log_must zfs unmount $TESTPOOL/$TESTFS log_must zfs mount $TESTPOOL/$TESTFS - log_must eval "getfacl $ACLDIR 2> /dev/null | egrep -q \"$acl_str1\"" - log_must eval "getfacl $ACLDIR 2> /dev/null | egrep -q \"$acl_str2\"" + log_must eval "getfacl $ACLDIR 2> /dev/null | grep -q \"$acl_str1\"" + log_must eval "getfacl $ACLDIR 2> /dev/null | grep -q \"$acl_str2\"" log_pass "POSIX ACLs survive remount" else log_fail "Group '$ZFS_ACL_STAFF_GROUP' does not have 'rwx'" diff --git a/tests/zfs-tests/tests/functional/acl/posix/posix_004_pos.ksh b/tests/zfs-tests/tests/functional/acl/posix/posix_004_pos.ksh index 6c6b592fbb9e..7906f5063c81 100755 --- a/tests/zfs-tests/tests/functional/acl/posix/posix_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/acl/posix/posix_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/acl/posix/setup.ksh b/tests/zfs-tests/tests/functional/acl/posix/setup.ksh index 526c78e17f1a..8ed9ab7e471e 100755 --- a/tests/zfs-tests/tests/functional/acl/posix/setup.ksh +++ b/tests/zfs-tests/tests/functional/acl/posix/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/alloc_class/Makefile.am b/tests/zfs-tests/tests/functional/alloc_class/Makefile.am deleted file mode 100644 index 7cffb2eac450..000000000000 --- a/tests/zfs-tests/tests/functional/alloc_class/Makefile.am +++ /dev/null @@ -1,21 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/alloc_class -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - alloc_class_001_pos.ksh \ - alloc_class_002_neg.ksh \ - alloc_class_003_pos.ksh \ - alloc_class_004_pos.ksh \ - alloc_class_005_pos.ksh \ - alloc_class_006_pos.ksh \ - alloc_class_007_pos.ksh \ - alloc_class_008_pos.ksh \ - alloc_class_009_pos.ksh \ - alloc_class_010_pos.ksh \ - alloc_class_011_neg.ksh \ - alloc_class_012_pos.ksh \ - alloc_class_013_pos.ksh - -dist_pkgdata_DATA = \ - alloc_class.cfg \ - alloc_class.kshlib diff --git a/tests/zfs-tests/tests/functional/alloc_class/alloc_class_004_pos.ksh b/tests/zfs-tests/tests/functional/alloc_class/alloc_class_004_pos.ksh index 79ac9364c257..04ce486adb83 100755 --- a/tests/zfs-tests/tests/functional/alloc_class/alloc_class_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/alloc_class/alloc_class_004_pos.ksh @@ -52,8 +52,7 @@ do log_must zpool create $TESTPOOL $type $ZPOOL_DISKS \ special $stype $sdisks - ac_value="$(zpool get -H -o property,value all | \ - egrep allocation_classes | nawk '{print $2}')" + ac_value="$(zpool get -H -o property,value all | awk '/allocation_classes/ {print $2}')" if [ "$ac_value" = "active" ]; then log_note "feature@allocation_classes is active" else diff --git a/tests/zfs-tests/tests/functional/alloc_class/alloc_class_005_pos.ksh b/tests/zfs-tests/tests/functional/alloc_class/alloc_class_005_pos.ksh index 337114cdb59e..08c703e21acb 100755 --- a/tests/zfs-tests/tests/functional/alloc_class/alloc_class_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/alloc_class/alloc_class_005_pos.ksh @@ -42,7 +42,7 @@ do log_must zpool create $TESTPOOL $type $ZPOOL_DISKS fi ac_value="$(zpool get -H -o property,value all | \ - egrep allocation_classes | awk '{print $2}')" + awk '/allocation_classes/ {print $2}')" if [ "$ac_value" = "enabled" ]; then log_note "feature@allocation_classes is enabled" else @@ -57,7 +57,7 @@ do $CLASS_DISK0 $CLASS_DISK1 fi ac_value="$(zpool get -H -o property,value all | \ - egrep allocation_classes | awk '{print $2}')" + awk '/allocation_classes/ {print $2}')" if [ "$ac_value" = "active" ]; then log_note "feature@allocation_classes is active" else diff --git a/tests/zfs-tests/tests/functional/alloc_class/alloc_class_010_pos.ksh b/tests/zfs-tests/tests/functional/alloc_class/alloc_class_010_pos.ksh index 2c14c69d8b20..cbf5cbf89bdc 100755 --- a/tests/zfs-tests/tests/functional/alloc_class/alloc_class_010_pos.ksh +++ b/tests/zfs-tests/tests/functional/alloc_class/alloc_class_010_pos.ksh @@ -39,7 +39,7 @@ for value in 0 512 1024 2048 4096 8192 16384 32768 65536 131072 do log_must zfs set special_small_blocks=$value $TESTPOOL ACTUAL=$(zfs get -p special_small_blocks $TESTPOOL | \ - grep special_small_blocks | awk '{print $3}') + awk '/special_small_blocks/ {print $3}') if [ "$ACTUAL" != "$value" ] then log_fail "v. $ACTUAL set for $TESTPOOL, expected v. $value!" diff --git a/tests/zfs-tests/tests/functional/alloc_class/alloc_class_011_neg.ksh b/tests/zfs-tests/tests/functional/alloc_class/alloc_class_011_neg.ksh index d804e5371ebb..0be49b858758 100755 --- a/tests/zfs-tests/tests/functional/alloc_class/alloc_class_011_neg.ksh +++ b/tests/zfs-tests/tests/functional/alloc_class/alloc_class_011_neg.ksh @@ -35,7 +35,7 @@ log_must disk_setup log_must zpool create $TESTPOOL raidz $ZPOOL_DISKS special mirror \ $CLASS_DISK0 $CLASS_DISK1 -for value in 256 1025 2097152 +for value in 256 1025 33554432 do log_mustnot zfs set special_small_blocks=$value $TESTPOOL done diff --git a/tests/zfs-tests/tests/functional/append/cleanup.ksh b/tests/zfs-tests/tests/functional/append/cleanup.ksh new file mode 100755 index 000000000000..7d8f3aafb771 --- /dev/null +++ b/tests/zfs-tests/tests/functional/append/cleanup.ksh @@ -0,0 +1,29 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright (c) 2022 by Triad National Security, LCC +# + +. ${STF_SUITE}/include/libtest.shlib + +default_cleanup diff --git a/tests/zfs-tests/tests/functional/append/file_append.ksh b/tests/zfs-tests/tests/functional/append/file_append.ksh new file mode 100755 index 000000000000..52c3aff17b79 --- /dev/null +++ b/tests/zfs-tests/tests/functional/append/file_append.ksh @@ -0,0 +1,73 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright (c) 2022 by Triad National Security, LCC +# + +. $STF_SUITE/include/libtest.shlib + +# +# DESCRIPTION: +# Tests file offset using O_APPEND. +# +# STRATEGY: +# 1. Open file using O_APPEND +# 2. Write to the file using random number of blocks (1, 2, or 3) +# 3. Verify that the file offset is correct using lseek after the write +# 4. Repeat steps 2 and 3, 5 times +# 5. Close the file. +# 6. Repeat steps 1-5 but also open file with O_DIRECT +# + +verify_runnable "global" + +log_assert "Ensure file offset is updated correctly when opened with O_APPEND" + +mntpt=$(get_prop mountpoint $TESTPOOL/$TESTFS) +filename=$mntpt/append_file.txt +bs=131072 +ITERATIONS=5 +expected=0 + +# First test using buffered writes with O_APPEND +for i in $(seq $ITERATIONS); do + num_blocks=$(random_int_between 1 3) + expected=$((expected + ( bs * num_blocks))) + log_must file_append -f $filename -e $expected -b $bs -n $num_blocks + curr_offset=$expected +done + +log_must rm -f $filename + +expected=0 + +# Repeat same test using O_DIRECT writes with O_APPEND +for i in $(seq $ITERATIONS); do + num_blocks=$(random_int_between 1 3) + expected=$((expected + ( bs * num_blocks))) + log_must file_append -f $filename -e $expected -b $bs -n $num_blocks -d +done + +log_must rm -f $filename + +log_pass "File offset updated correctly when opening a file with O_APPEND." diff --git a/tests/zfs-tests/tests/functional/append/setup.ksh b/tests/zfs-tests/tests/functional/append/setup.ksh new file mode 100755 index 000000000000..4642f3a1925c --- /dev/null +++ b/tests/zfs-tests/tests/functional/append/setup.ksh @@ -0,0 +1,31 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright (c) 2022 by Triad National Security, LCC +# + +. $STF_SUITE/include/libtest.shlib + +DISK=${DISKS%% *} + +default_setup ${DISK} diff --git a/tests/zfs-tests/tests/functional/threadsappend/threadsappend_001_pos.ksh b/tests/zfs-tests/tests/functional/append/threadsappend_001_pos.ksh similarity index 97% rename from tests/zfs-tests/tests/functional/threadsappend/threadsappend_001_pos.ksh rename to tests/zfs-tests/tests/functional/append/threadsappend_001_pos.ksh index 8154214a0ca7..8ce79c12a04a 100755 --- a/tests/zfs-tests/tests/functional/threadsappend/threadsappend_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/append/threadsappend_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/arc/Makefile.am b/tests/zfs-tests/tests/functional/arc/Makefile.am deleted file mode 100644 index 809d0346f872..000000000000 --- a/tests/zfs-tests/tests/functional/arc/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/arc -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - arcstats_runtime_tuning.ksh \ - dbufstats_001_pos.ksh \ - dbufstats_002_pos.ksh \ - dbufstats_003_pos.ksh diff --git a/tests/zfs-tests/tests/functional/arc/cleanup.ksh b/tests/zfs-tests/tests/functional/arc/cleanup.ksh index 63a47ba7daf3..72a091d25157 100755 --- a/tests/zfs-tests/tests/functional/arc/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/arc/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/arc/dbufstats_001_pos.ksh b/tests/zfs-tests/tests/functional/arc/dbufstats_001_pos.ksh index 712309eda72f..e51cf179d8ef 100755 --- a/tests/zfs-tests/tests/functional/arc/dbufstats_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/arc/dbufstats_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -56,8 +56,7 @@ function testdbufstat # stat_name dbufstat_filter [[ -n "$2" ]] && filter="-F $2" if is_linux; then - from_dbufstat=$(grep -w "$name" "$DBUFSTATS_FILE" | - awk '{ print $3 }') + read -r _ _ from_dbufstat _ < <(grep -w "$name" "$DBUFSTATS_FILE") else from_dbufstat=$(awk "/dbufstats\.$name:/ { print \$2 }" \ "$DBUFSTATS_FILE") diff --git a/tests/zfs-tests/tests/functional/arc/dbufstats_002_pos.ksh b/tests/zfs-tests/tests/functional/arc/dbufstats_002_pos.ksh index e253553f07c0..2908895d0c6a 100755 --- a/tests/zfs-tests/tests/functional/arc/dbufstats_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/arc/dbufstats_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/arc/dbufstats_003_pos.ksh b/tests/zfs-tests/tests/functional/arc/dbufstats_003_pos.ksh index 91cec74881a6..7be57bc9f255 100755 --- a/tests/zfs-tests/tests/functional/arc/dbufstats_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/arc/dbufstats_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/arc/setup.ksh b/tests/zfs-tests/tests/functional/arc/setup.ksh index 37b8f352cc64..a499d7415077 100755 --- a/tests/zfs-tests/tests/functional/arc/setup.ksh +++ b/tests/zfs-tests/tests/functional/arc/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -26,5 +26,7 @@ . $STF_SUITE/include/libtest.shlib +is_freebsd && ! python3 -c 'import sysctl' 2>/dev/null && log_unsupported "python3 sysctl module missing" + DISK=${DISKS%% *} default_setup $DISK diff --git a/tests/zfs-tests/tests/functional/atime/Makefile.am b/tests/zfs-tests/tests/functional/atime/Makefile.am deleted file mode 100644 index 63d510b99e75..000000000000 --- a/tests/zfs-tests/tests/functional/atime/Makefile.am +++ /dev/null @@ -1,14 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/atime -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - atime_001_pos.ksh \ - atime_002_neg.ksh \ - atime_003_pos.ksh \ - root_atime_off.ksh \ - root_atime_on.ksh \ - root_relatime_on.ksh - -dist_pkgdata_DATA = \ - atime.cfg \ - atime_common.kshlib diff --git a/tests/zfs-tests/tests/functional/atime/atime.cfg b/tests/zfs-tests/tests/functional/atime/atime.cfg index c1532eb47fd3..eece764248cd 100644 --- a/tests/zfs-tests/tests/functional/atime/atime.cfg +++ b/tests/zfs-tests/tests/functional/atime/atime.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/atime/atime_001_pos.ksh b/tests/zfs-tests/tests/functional/atime/atime_001_pos.ksh index 274a4da0dcb9..136bb6b11c2c 100755 --- a/tests/zfs-tests/tests/functional/atime/atime_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/atime/atime_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/atime/atime_002_neg.ksh b/tests/zfs-tests/tests/functional/atime/atime_002_neg.ksh index c8579337c053..8fde4f23455c 100755 --- a/tests/zfs-tests/tests/functional/atime/atime_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/atime/atime_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/atime/atime_003_pos.ksh b/tests/zfs-tests/tests/functional/atime/atime_003_pos.ksh index d3eec92effbc..f5c1044dcc8e 100755 --- a/tests/zfs-tests/tests/functional/atime/atime_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/atime/atime_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/atime/atime_common.kshlib b/tests/zfs-tests/tests/functional/atime/atime_common.kshlib index fce85c379865..1755f0b49732 100644 --- a/tests/zfs-tests/tests/functional/atime/atime_common.kshlib +++ b/tests/zfs-tests/tests/functional/atime/atime_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/atime/cleanup.ksh b/tests/zfs-tests/tests/functional/atime/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/atime/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/atime/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/atime/root_atime_off.ksh b/tests/zfs-tests/tests/functional/atime/root_atime_off.ksh index 7eb2ed937287..9c1bf3150570 100755 --- a/tests/zfs-tests/tests/functional/atime/root_atime_off.ksh +++ b/tests/zfs-tests/tests/functional/atime/root_atime_off.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/atime/root_atime_on.ksh b/tests/zfs-tests/tests/functional/atime/root_atime_on.ksh index 44d471a2128f..1e3b543a6d42 100755 --- a/tests/zfs-tests/tests/functional/atime/root_atime_on.ksh +++ b/tests/zfs-tests/tests/functional/atime/root_atime_on.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/atime/root_relatime_on.ksh b/tests/zfs-tests/tests/functional/atime/root_relatime_on.ksh index 120129425afa..36dbaaee7793 100755 --- a/tests/zfs-tests/tests/functional/atime/root_relatime_on.ksh +++ b/tests/zfs-tests/tests/functional/atime/root_relatime_on.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/atime/setup.ksh b/tests/zfs-tests/tests/functional/atime/setup.ksh index 3720e7521cc2..f662bffbc374 100755 --- a/tests/zfs-tests/tests/functional/atime/setup.ksh +++ b/tests/zfs-tests/tests/functional/atime/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/bootfs/Makefile.am b/tests/zfs-tests/tests/functional/bootfs/Makefile.am deleted file mode 100644 index 15a6e202c311..000000000000 --- a/tests/zfs-tests/tests/functional/bootfs/Makefile.am +++ /dev/null @@ -1,12 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/bootfs -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - bootfs_001_pos.ksh \ - bootfs_002_neg.ksh \ - bootfs_003_pos.ksh \ - bootfs_004_neg.ksh \ - bootfs_005_neg.ksh \ - bootfs_006_pos.ksh \ - bootfs_007_pos.ksh \ - bootfs_008_pos.ksh diff --git a/tests/zfs-tests/tests/functional/bootfs/bootfs_001_pos.ksh b/tests/zfs-tests/tests/functional/bootfs/bootfs_001_pos.ksh index 3e9357063bed..add455eb13ee 100755 --- a/tests/zfs-tests/tests/functional/bootfs/bootfs_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/bootfs/bootfs_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -59,12 +59,6 @@ function cleanup { fi } -zpool set 2>&1 | grep bootfs > /dev/null -if [ $? -ne 0 ] -then - log_unsupported "bootfs pool property not supported on this release." -fi - log_assert "Valid datasets are accepted as bootfs property values" log_onexit cleanup diff --git a/tests/zfs-tests/tests/functional/bootfs/bootfs_002_neg.ksh b/tests/zfs-tests/tests/functional/bootfs/bootfs_002_neg.ksh index a5bc7753e96e..a4324a63c710 100755 --- a/tests/zfs-tests/tests/functional/bootfs/bootfs_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/bootfs/bootfs_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -60,12 +60,6 @@ function cleanup { } -zpool set 2>&1 | grep bootfs > /dev/null -if [ $? -ne 0 ] -then - log_unsupported "bootfs pool property not supported on this release." -fi - log_assert "Invalid datasets are rejected as boot property values" log_onexit cleanup diff --git a/tests/zfs-tests/tests/functional/bootfs/bootfs_003_pos.ksh b/tests/zfs-tests/tests/functional/bootfs/bootfs_003_pos.ksh index e719b94e2763..e1d27d1cc6df 100755 --- a/tests/zfs-tests/tests/functional/bootfs/bootfs_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/bootfs/bootfs_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -54,12 +54,6 @@ function cleanup { } -zpool set 2>&1 | grep bootfs > /dev/null -if [ $? -ne 0 ] -then - log_unsupported "bootfs pool property not supported on this release." -fi - log_onexit cleanup log_assert "Valid pool names are accepted by zpool set bootfs" @@ -74,7 +68,7 @@ do log_must zfs create $POOL/$TESTFS log_must zpool set bootfs=$POOL/$TESTFS $POOL - RES=$(zpool get bootfs $POOL | tail -1 | awk '{print $3}' ) + RES=$(zpool get bootfs $POOL | awk 'END {print $3}' ) if [ $RES != "$POOL/$TESTFS" ] then log_fail "Expected $RES == $POOL/$TESTFS" diff --git a/tests/zfs-tests/tests/functional/bootfs/bootfs_004_neg.ksh b/tests/zfs-tests/tests/functional/bootfs/bootfs_004_neg.ksh index 97b456aade3e..fa341622f3cf 100755 --- a/tests/zfs-tests/tests/functional/bootfs/bootfs_004_neg.ksh +++ b/tests/zfs-tests/tests/functional/bootfs/bootfs_004_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -55,12 +55,6 @@ function cleanup { } -zpool set 2>&1 | grep bootfs > /dev/null -if [ $? -ne 0 ] -then - log_unsupported "bootfs pool property not supported on this release." -fi - log_assert "Invalid pool names are rejected by zpool set bootfs" log_onexit cleanup diff --git a/tests/zfs-tests/tests/functional/bootfs/bootfs_005_neg.ksh b/tests/zfs-tests/tests/functional/bootfs/bootfs_005_neg.ksh index 42c82b88fba6..39a9a85b44a7 100755 --- a/tests/zfs-tests/tests/functional/bootfs/bootfs_005_neg.ksh +++ b/tests/zfs-tests/tests/functional/bootfs/bootfs_005_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/bootfs/bootfs_006_pos.ksh b/tests/zfs-tests/tests/functional/bootfs/bootfs_006_pos.ksh index d29fe7e89c50..cdd69f4a5b2f 100755 --- a/tests/zfs-tests/tests/functional/bootfs/bootfs_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/bootfs/bootfs_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -44,12 +44,6 @@ verify_runnable "global" -zpool set 2>&1 | grep bootfs > /dev/null -if [ $? -ne 0 ] -then - log_unsupported "bootfs pool property not supported on this release." -fi - VDEV1=$TESTDIR/bootfs_006_pos_a.$$.dat VDEV2=$TESTDIR/bootfs_006_pos_b.$$.dat VDEV3=$TESTDIR/bootfs_006_pos_c.$$.dat @@ -60,7 +54,7 @@ function verify_bootfs { # $POOL log_must zfs create $POOL/$TESTFS log_must zpool set bootfs=$POOL/$TESTFS $POOL - VAL=$(zpool get bootfs $POOL | tail -1 | awk '{print $3}' ) + VAL=$(zpool get bootfs $POOL | awk 'END {print $3}' ) if [ $VAL != "$POOL/$TESTFS" ] then log_must zpool status -v $POOL @@ -74,7 +68,7 @@ function verify_no_bootfs { # $POOL POOL=$1 log_must zfs create $POOL/$TESTFS log_mustnot zpool set bootfs=$POOL/$TESTFS $POOL - VAL=$(zpool get bootfs $POOL | tail -1 | awk '{print $3}' ) + VAL=$(zpool get bootfs $POOL | awk 'END {print $3}' ) if [ $VAL == "$POOL/$TESTFS" ] then log_must zpool status -v $POOL diff --git a/tests/zfs-tests/tests/functional/bootfs/bootfs_007_pos.ksh b/tests/zfs-tests/tests/functional/bootfs/bootfs_007_pos.ksh index 2e71df86c62b..5750fe9338eb 100755 --- a/tests/zfs-tests/tests/functional/bootfs/bootfs_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/bootfs/bootfs_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/bootfs/bootfs_008_pos.ksh b/tests/zfs-tests/tests/functional/bootfs/bootfs_008_pos.ksh index bcf74f2cdee2..270a3d706339 100755 --- a/tests/zfs-tests/tests/functional/bootfs/bootfs_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/bootfs/bootfs_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/bootfs/cleanup.ksh b/tests/zfs-tests/tests/functional/bootfs/cleanup.ksh index 71d8ced18206..eb542c0092fc 100755 --- a/tests/zfs-tests/tests/functional/bootfs/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/bootfs/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/bootfs/setup.ksh b/tests/zfs-tests/tests/functional/bootfs/setup.ksh index 273f239137b0..2f89df767e4d 100755 --- a/tests/zfs-tests/tests/functional/bootfs/setup.ksh +++ b/tests/zfs-tests/tests/functional/bootfs/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/btree/Makefile.am b/tests/zfs-tests/tests/functional/btree/Makefile.am deleted file mode 100644 index 333209d98f2b..000000000000 --- a/tests/zfs-tests/tests/functional/btree/Makefile.am +++ /dev/null @@ -1,20 +0,0 @@ -# -# This file and its contents are supplied under the terms of the -# Common Development and Distribution License ("CDDL"), version 1.0. -# You may only use this file in accordance with the terms of version -# 1.0 of the CDDL. -# -# A full copy of the text of the CDDL should have accompanied this -# source. A copy of the CDDL is also available via the Internet at -# http://www.illumos.org/license/CDDL. -# - -# -# Copyright (c) 2019 by Delphix. All rights reserved. -# - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/btree - -dist_pkgdata_SCRIPTS = \ - btree_positive.ksh \ - btree_negative.ksh diff --git a/tests/zfs-tests/tests/functional/btree/btree_negative.ksh b/tests/zfs-tests/tests/functional/btree/btree_negative.ksh index cefcbc51e796..667ac87b4fea 100755 --- a/tests/zfs-tests/tests/functional/btree/btree_negative.ksh +++ b/tests/zfs-tests/tests/functional/btree/btree_negative.ksh @@ -29,10 +29,7 @@ # looks for return values that correspond to a core dump and cause a test # failure. -btree_test -n insert_duplicate -[[ $? -eq 0 ]] && log_fail "Failure from insert_duplicate" - -btree_test -n remove_missing -[[ $? -eq 0 ]] && log_fail "Failure from remove_missing" +btree_test -n insert_duplicate && log_fail "Failure from insert_duplicate" +btree_test -n remove_missing && log_fail "Failure from remove_missing" log_pass "Btree negative tests passed" diff --git a/tests/zfs-tests/tests/functional/cache/Makefile.am b/tests/zfs-tests/tests/functional/cache/Makefile.am deleted file mode 100644 index f28130ee9e4d..000000000000 --- a/tests/zfs-tests/tests/functional/cache/Makefile.am +++ /dev/null @@ -1,20 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cache -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - cache_001_pos.ksh \ - cache_002_pos.ksh \ - cache_003_pos.ksh \ - cache_004_neg.ksh \ - cache_005_neg.ksh \ - cache_006_pos.ksh \ - cache_007_neg.ksh \ - cache_008_neg.ksh \ - cache_009_pos.ksh \ - cache_010_pos.ksh \ - cache_011_pos.ksh \ - cache_012_pos.ksh - -dist_pkgdata_DATA = \ - cache.cfg \ - cache.kshlib diff --git a/tests/zfs-tests/tests/functional/cache/cache.cfg b/tests/zfs-tests/tests/functional/cache/cache.cfg index 07e482d7df96..84dd78438d61 100644 --- a/tests/zfs-tests/tests/functional/cache/cache.cfg +++ b/tests/zfs-tests/tests/functional/cache/cache.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cache/cache.kshlib b/tests/zfs-tests/tests/functional/cache/cache.kshlib index 2e258e22cd39..63cc0b28ecf1 100644 --- a/tests/zfs-tests/tests/functional/cache/cache.kshlib +++ b/tests/zfs-tests/tests/functional/cache/cache.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -94,7 +94,7 @@ function verify_cache_device # # mirror:/disks/d ONLINE mirror:/disks/e ONLINE stripe:/disks/f ONLINE # - set -A dev_stat_tab $(zpool status -v $pool | nawk 'BEGIN {start=0} \ + set -A dev_stat_tab $(zpool status -v $pool | awk 'BEGIN {start=0} \ /\tcache/ {start=1} /\tmirror/ || /\tspares/ || /^$/ {start=0} (start==1) && /\t (\/|[a-zA-Z])/ \ @@ -146,9 +146,3 @@ function verify_cache_device log_note "Can not find device: $device" return 1 } - -function verify_cache_support -{ - zpool upgrade -v | grep "Cache devices" > /dev/null 2>&1 - return $? -} diff --git a/tests/zfs-tests/tests/functional/cache/cache_001_pos.ksh b/tests/zfs-tests/tests/functional/cache/cache_001_pos.ksh index 955c9f12019f..c77668cf7deb 100755 --- a/tests/zfs-tests/tests/functional/cache/cache_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cache/cache_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cache/cache_002_pos.ksh b/tests/zfs-tests/tests/functional/cache/cache_002_pos.ksh index 888645c353e8..5f1dc75c9919 100755 --- a/tests/zfs-tests/tests/functional/cache/cache_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cache/cache_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cache/cache_003_pos.ksh b/tests/zfs-tests/tests/functional/cache/cache_003_pos.ksh index 566c5c6ac675..f2a2777dc6ce 100755 --- a/tests/zfs-tests/tests/functional/cache/cache_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cache/cache_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cache/cache_004_neg.ksh b/tests/zfs-tests/tests/functional/cache/cache_004_neg.ksh index 38fee608bbc8..fe74cbaf1dae 100755 --- a/tests/zfs-tests/tests/functional/cache/cache_004_neg.ksh +++ b/tests/zfs-tests/tests/functional/cache/cache_004_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cache/cache_005_neg.ksh b/tests/zfs-tests/tests/functional/cache/cache_005_neg.ksh index 35e15abe2602..59afbb0186b6 100755 --- a/tests/zfs-tests/tests/functional/cache/cache_005_neg.ksh +++ b/tests/zfs-tests/tests/functional/cache/cache_005_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cache/cache_006_pos.ksh b/tests/zfs-tests/tests/functional/cache/cache_006_pos.ksh index 46bd5fc8b3ef..2f00b43f3424 100755 --- a/tests/zfs-tests/tests/functional/cache/cache_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/cache/cache_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cache/cache_007_neg.ksh b/tests/zfs-tests/tests/functional/cache/cache_007_neg.ksh index e2cdc0a28d38..721dd2e04837 100755 --- a/tests/zfs-tests/tests/functional/cache/cache_007_neg.ksh +++ b/tests/zfs-tests/tests/functional/cache/cache_007_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cache/cache_008_neg.ksh b/tests/zfs-tests/tests/functional/cache/cache_008_neg.ksh index 93866dd3097f..d0c3f621b756 100755 --- a/tests/zfs-tests/tests/functional/cache/cache_008_neg.ksh +++ b/tests/zfs-tests/tests/functional/cache/cache_008_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cache/cache_009_pos.ksh b/tests/zfs-tests/tests/functional/cache/cache_009_pos.ksh index 7a0586648f84..1d83e144266e 100755 --- a/tests/zfs-tests/tests/functional/cache/cache_009_pos.ksh +++ b/tests/zfs-tests/tests/functional/cache/cache_009_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cache/cache_010_pos.ksh b/tests/zfs-tests/tests/functional/cache/cache_010_pos.ksh index 1d9fc5a8922b..b912eeec8769 100755 --- a/tests/zfs-tests/tests/functional/cache/cache_010_pos.ksh +++ b/tests/zfs-tests/tests/functional/cache/cache_010_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cache/cache_011_pos.ksh b/tests/zfs-tests/tests/functional/cache/cache_011_pos.ksh index caa7e6e900b7..0e48ac5e2a9e 100755 --- a/tests/zfs-tests/tests/functional/cache/cache_011_pos.ksh +++ b/tests/zfs-tests/tests/functional/cache/cache_011_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cache/cache_012_pos.ksh b/tests/zfs-tests/tests/functional/cache/cache_012_pos.ksh index edefe9c1bf68..be250158bf7e 100755 --- a/tests/zfs-tests/tests/functional/cache/cache_012_pos.ksh +++ b/tests/zfs-tests/tests/functional/cache/cache_012_pos.ksh @@ -44,6 +44,8 @@ verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + log_assert "Looping around a cache device succeeds." function cleanup diff --git a/tests/zfs-tests/tests/functional/cache/cleanup.ksh b/tests/zfs-tests/tests/functional/cache/cleanup.ksh index 258f92d19e0e..23d9e2d45286 100755 --- a/tests/zfs-tests/tests/functional/cache/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cache/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cache/setup.ksh b/tests/zfs-tests/tests/functional/cache/setup.ksh index 0493637fcca8..84d11a4d70e6 100755 --- a/tests/zfs-tests/tests/functional/cache/setup.ksh +++ b/tests/zfs-tests/tests/functional/cache/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cachefile/Makefile.am b/tests/zfs-tests/tests/functional/cachefile/Makefile.am deleted file mode 100644 index 53d8c8c6c9d5..000000000000 --- a/tests/zfs-tests/tests/functional/cachefile/Makefile.am +++ /dev/null @@ -1,12 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cachefile -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - cachefile_001_pos.ksh \ - cachefile_002_pos.ksh \ - cachefile_003_pos.ksh \ - cachefile_004_pos.ksh - -dist_pkgdata_DATA = \ - cachefile.cfg \ - cachefile.kshlib diff --git a/tests/zfs-tests/tests/functional/cachefile/cachefile.cfg b/tests/zfs-tests/tests/functional/cachefile/cachefile.cfg index d93ec3ed6f3b..47b0853184e0 100644 --- a/tests/zfs-tests/tests/functional/cachefile/cachefile.cfg +++ b/tests/zfs-tests/tests/functional/cachefile/cachefile.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -33,5 +33,5 @@ export CPATH1=$TEST_BASE_DIR/cachefile.1.$$ export CPATH2=$TEST_BASE_DIR/cachefile.2.$$ export DISKSARRAY=$DISKS -export DISK_ARRAY_NUM=$(echo ${DISKS} | nawk '{print NF}') +export DISK_ARRAY_NUM=$(echo ${DISKS} | awk '{print NF}') set_device_dir diff --git a/tests/zfs-tests/tests/functional/cachefile/cachefile.kshlib b/tests/zfs-tests/tests/functional/cachefile/cachefile.kshlib index 7723e3083f1d..463355371c75 100644 --- a/tests/zfs-tests/tests/functional/cachefile/cachefile.kshlib +++ b/tests/zfs-tests/tests/functional/cachefile/cachefile.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cachefile/cachefile_001_pos.ksh b/tests/zfs-tests/tests/functional/cachefile/cachefile_001_pos.ksh index f164a1fad32f..256c2d38878b 100755 --- a/tests/zfs-tests/tests/functional/cachefile/cachefile_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cachefile/cachefile_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cachefile/cachefile_002_pos.ksh b/tests/zfs-tests/tests/functional/cachefile/cachefile_002_pos.ksh index 9268f9786355..71039c11b2be 100755 --- a/tests/zfs-tests/tests/functional/cachefile/cachefile_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cachefile/cachefile_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cachefile/cachefile_003_pos.ksh b/tests/zfs-tests/tests/functional/cachefile/cachefile_003_pos.ksh index a5f427cb4135..4a6d28f69333 100755 --- a/tests/zfs-tests/tests/functional/cachefile/cachefile_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cachefile/cachefile_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cachefile/cachefile_004_pos.ksh b/tests/zfs-tests/tests/functional/cachefile/cachefile_004_pos.ksh index 841b141e16fc..59a3b7dcf995 100755 --- a/tests/zfs-tests/tests/functional/cachefile/cachefile_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cachefile/cachefile_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cachefile/cleanup.ksh b/tests/zfs-tests/tests/functional/cachefile/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cachefile/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cachefile/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cachefile/setup.ksh b/tests/zfs-tests/tests/functional/cachefile/setup.ksh index 47c7893bfdd6..de6c2be66b7b 100755 --- a/tests/zfs-tests/tests/functional/cachefile/setup.ksh +++ b/tests/zfs-tests/tests/functional/cachefile/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/casenorm/Makefile.am b/tests/zfs-tests/tests/functional/casenorm/Makefile.am deleted file mode 100644 index b284a2560b27..000000000000 --- a/tests/zfs-tests/tests/functional/casenorm/Makefile.am +++ /dev/null @@ -1,25 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/casenorm -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - case_all_values.ksh \ - insensitive_formd_delete.ksh \ - insensitive_formd_lookup.ksh \ - insensitive_none_delete.ksh \ - insensitive_none_lookup.ksh \ - mixed_create_failure.ksh \ - mixed_formd_delete.ksh \ - mixed_formd_lookup_ci.ksh \ - mixed_formd_lookup.ksh \ - mixed_none_delete.ksh \ - mixed_none_lookup_ci.ksh \ - mixed_none_lookup.ksh \ - norm_all_values.ksh \ - sensitive_formd_delete.ksh \ - sensitive_formd_lookup.ksh \ - sensitive_none_delete.ksh \ - sensitive_none_lookup.ksh - -dist_pkgdata_DATA = \ - casenorm.cfg \ - casenorm.kshlib diff --git a/tests/zfs-tests/tests/functional/casenorm/casenorm.kshlib b/tests/zfs-tests/tests/functional/casenorm/casenorm.kshlib index f0fe1bbaa886..ad5b5367ae48 100644 --- a/tests/zfs-tests/tests/functional/casenorm/casenorm.kshlib +++ b/tests/zfs-tests/tests/functional/casenorm/casenorm.kshlib @@ -50,13 +50,8 @@ function delete_file { typeset name=$TESTDIR/$1 - rm $name >/dev/null 2>&1 - - if [[ $? -ne 0 ]] ; then - return 1 - fi - - if [[ -f $name ]] ; then + rm $name >/dev/null 2>&1 || return 1 + if [ -f $name ]; then return 2 fi } @@ -86,10 +81,7 @@ function lookup_file_ci function lookup_any { for name in $NAMES_ALL ; do - lookup_file $name - if [[ $? -eq 0 ]] ; then - return 0 - fi + lookup_file $name && return done return 1 diff --git a/tests/zfs-tests/tests/functional/casenorm/mixed_create_failure.ksh b/tests/zfs-tests/tests/functional/casenorm/mixed_create_failure.ksh index 51b5bb3f6584..208be91dea42 100755 --- a/tests/zfs-tests/tests/functional/casenorm/mixed_create_failure.ksh +++ b/tests/zfs-tests/tests/functional/casenorm/mixed_create_failure.ksh @@ -70,6 +70,7 @@ function test_ops { typeset obj_type=$1 typeset testdir=$2 + typeset save_name= target_obj='target-file' @@ -83,7 +84,7 @@ function test_ops log_note "Created test dir $test_path" if [[ $obj_type = "symlink" || $obj_type = "hardlink" ]]; then - touch $test_path/$target_obj + > $test_path/$target_obj log_note "Created target: $test_path/$target_obj" op="$op $test_path/$target_obj" fi @@ -100,29 +101,24 @@ function test_ops save_name="$test_path/$name" break; else - log_err "$cmd failed with unexpected error : $out" + log_fail "$cmd failed: $out" fi fi done + [ -n "$save_name" ] || log_fail "Didn't ENOSPC!" - log_note 'Test rename \"sample_name\" rename' + log_note 'Test rename "sample_name" rename' TMP_OBJ="$test_path/tmp_obj" cmd="$op $TMP_OBJ" - out=$($cmd 2>&1) - ret=$? - if (($ret != 0)); then - log_err "cmd:$cmd failed out:$out" - fi + log_must $cmd # Now, try to rename the tmp_obj to the name which we failed to add earlier. # This should fail as well. - out=$(mv $TMP_OBJ $save_name 2>&1) - ret=$? - if (($ret != 0)); then + if ! out=$(mv $TMP_OBJ $save_name 2>&1); then if [[ $out = *@(No space left on device)* ]]; then - log_note "$cmd failed as expected : $out" + log_note "$cmd failed as expected: $out" else - log_err "$cmd failed with : $out" + log_fail "$cmd failed: $out" fi fi } diff --git a/tests/zfs-tests/tests/functional/channel_program/Makefile.am b/tests/zfs-tests/tests/functional/channel_program/Makefile.am deleted file mode 100644 index 3886863d1dfb..000000000000 --- a/tests/zfs-tests/tests/functional/channel_program/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -SUBDIRS = \ - lua_core \ - synctask_core - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/channel_program -dist_pkgdata_DATA = channel_common.kshlib diff --git a/tests/zfs-tests/tests/functional/channel_program/channel_common.kshlib b/tests/zfs-tests/tests/functional/channel_program/channel_common.kshlib index a828ba29065e..c937e90614c8 100644 --- a/tests/zfs-tests/tests/functional/channel_program/channel_common.kshlib +++ b/tests/zfs-tests/tests/functional/channel_program/channel_common.kshlib @@ -47,9 +47,10 @@ function log_program zfs program $cmdargs >$tmpout 2>$tmperr typeset ret=$? - log_note "input:\n$(cat $tmpin)" - log_note "output:\n$(cat $tmpout)" - log_note "error:\n$(cat $tmperr)" + log_note $'input:\n'"$(<$tmpin)" + log_note $'output:\n'"$(<$tmpout)" + log_note $'error:\n'"$(<$tmperr)" + log_note "ret: $ret" # # Verify correct return value @@ -64,35 +65,29 @@ function log_program # respectively. # if [[ -f "$basename.out" ]] && [[ $expectexit -eq 0 ]]; then - - outdiff=$(diff "$basename.out" "$tmpout") - if [[ $? -ne 0 ]]; then + if ! outdiff=$(diff "$basename.out" "$tmpout"); then output=$(<$tmpout) rm $tmpout $tmperr $tmpin - log_fail "Output mismatch. Expected:\n" \ - "$(<$basename.out)\nBut got:\n$output\n" \ - "Diff:\n$outdiff" + log_fail $'Output mismatch. Expected:\n' \ + "$(<$basename.out)"$'\nBut got:\n'"$output"$'\n' \ + $'Diff:\n'"$outdiff" fi elif [[ -f "$basename.err" ]] && [[ $expectexit -ne 0 ]]; then - - outdiff=$(diff "$basename.err" "$tmperr") - if [[ $? -ne 0 ]]; then + if ! outdiff=$(diff "$basename.err" "$tmperr"); then outputerror=$(<$tmperr) rm $tmpout $tmperr $tmpin - log_fail "Error mismatch. Expected:\n" \ - "$(<$basename.err)\nBut got:\n$outputerror\n" \ - "Diff:\n$outdiff" + log_fail $'Error mismatch. Expected:\n' \ + "$(<$basename.err)"$'\nBut got:\n'"$outputerror"$'\n' \ + $'Diff:\n'"$outdiff" fi elif [[ -n $expecterror ]] && [[ $expectexit -ne 0 ]]; then - - grep -q "$expecterror" $tmperr - if [[ $? -ne 0 ]]; then + if ! grep -q "$expecterror" $tmperr; then outputerror=$(<$tmperr) rm $tmpout $tmperr $tmpin - log_fail "Error mismatch. Expected to contain:\n" \ - "$expecterror\nBut got:\n$outputerror\n" + log_fail $'Error mismatch. Expected to contain:\n' \ + "$expecterror"$'\nBut got:\n'"$outputerror"$'\n' fi elif [[ $expectexit -ne 0 ]]; then diff --git a/tests/zfs-tests/tests/functional/channel_program/lua_core/Makefile.am b/tests/zfs-tests/tests/functional/channel_program/lua_core/Makefile.am deleted file mode 100644 index fb352081190c..000000000000 --- a/tests/zfs-tests/tests/functional/channel_program/lua_core/Makefile.am +++ /dev/null @@ -1,46 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/channel_program/lua_core -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - tst.args_to_lua.ksh \ - tst.divide_by_zero.ksh \ - tst.exists.ksh \ - tst.integer_illegal.ksh \ - tst.integer_overflow.ksh \ - tst.language_functions_neg.ksh \ - tst.language_functions_pos.ksh \ - tst.large_prog.ksh \ - tst.libraries.ksh \ - tst.memory_limit.ksh \ - tst.nested_neg.ksh \ - tst.nested_pos.ksh \ - tst.nvlist_to_lua.ksh \ - tst.recursive_neg.ksh \ - tst.recursive_pos.ksh \ - tst.return_large.ksh \ - tst.return_nvlist_neg.ksh \ - tst.return_nvlist_pos.ksh \ - tst.return_recursive_table.ksh \ - tst.stack_gsub.ksh \ - tst.timeout.ksh - -dist_pkgdata_DATA = \ - tst.args_to_lua.out \ - tst.args_to_lua.zcp \ - tst.divide_by_zero.err \ - tst.divide_by_zero.zcp \ - tst.exists.zcp \ - tst.large_prog.out \ - tst.large_prog.zcp \ - tst.lib_base.lua \ - tst.lib_coroutine.lua \ - tst.lib_strings.lua \ - tst.lib_table.lua \ - tst.nested_neg.zcp \ - tst.nested_pos.zcp \ - tst.recursive.zcp \ - tst.return_large.zcp \ - tst.return_recursive_table.zcp \ - tst.stack_gsub.err \ - tst.stack_gsub.zcp \ - tst.timeout.zcp diff --git a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.exists.ksh b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.exists.ksh index eba01b17c80e..34f29d9c039d 100755 --- a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.exists.ksh +++ b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.exists.ksh @@ -38,8 +38,6 @@ log_must_program $TESTPOOL $ZCP_ROOT/lua_core/tst.exists.zcp \ $TESTPOOL/$TESTCLONE log_mustnot_checkerror_program "not in the target pool" \ - $TESTPOOL - <<-EOF - return zfs.exists('rpool') -EOF + $TESTPOOL - <<<"return zfs.exists('rpool')" log_pass "zfs.exists() gives correct results" diff --git a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.integer_illegal.ksh b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.integer_illegal.ksh index c34f2afd9e18..1c70bc85920a 100755 --- a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.integer_illegal.ksh +++ b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.integer_illegal.ksh @@ -32,9 +32,7 @@ set -A args "1.0" \ typeset -i i=0 while (( i < ${#args[*]} )); do - log_mustnot_checkerror_program "malformed number" $TESTPOOL - <<-EOF - return ${args[i]} - EOF + log_mustnot_checkerror_program "malformed number" $TESTPOOL - <<<"return ${args[i]}" ((i = i + 1)) done diff --git a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.integer_overflow.ksh b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.integer_overflow.ksh index c129bae51b04..be2129792499 100755 --- a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.integer_overflow.ksh +++ b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.integer_overflow.ksh @@ -25,8 +25,6 @@ verify_runnable "global" log_assert "overflowing a 64-bit integer should wrap around" -log_must_program $TESTPOOL - <<-EOF - assert(18446744073709551615 + 1 == (-18446744073709551616)) -EOF +log_must_program $TESTPOOL - <<<"assert(18446744073709551615 + 1 == (-18446744073709551616))" log_pass "overflowing a 64-bit integer should wrap around" diff --git a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.language_functions_neg.ksh b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.language_functions_neg.ksh index 0125d76c7036..3f91db2be9c3 100755 --- a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.language_functions_neg.ksh +++ b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.language_functions_neg.ksh @@ -43,9 +43,7 @@ log_assert "Runtime errors in lua scripts fail as expected." typeset -i i=0 while (( i < ${#args[*]} )); do - log_mustnot_checkerror_program "execution failed" $TESTPOOL - <<-EOF - ${args[i]} - EOF + log_mustnot_checkerror_program "execution failed" $TESTPOOL - <<<"${args[i]}" ((i = i + 1)) done diff --git a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.language_functions_pos.ksh b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.language_functions_pos.ksh index 924d8e2c6944..019622cf84cc 100755 --- a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.language_functions_pos.ksh +++ b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.language_functions_pos.ksh @@ -33,9 +33,7 @@ log_assert "Simple lua scripts pass." typeset -i i=0 while (( i < ${#args[*]} )); do - log_must_program $TESTPOOL - <<-EOF - ${args[i]} - EOF + log_must_program $TESTPOOL - <<<"${args[i]}" ((i = i + 1)) done diff --git a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.return_large.ksh b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.return_large.ksh index bbaeb54f59a5..ae42665cd9f5 100755 --- a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.return_large.ksh +++ b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.return_large.ksh @@ -41,8 +41,7 @@ log_must zfs create $fs output_lines=$(log_must zfs program $TESTPOOL \ $ZCP_ROOT/lua_core/tst.return_large.zcp | wc -l) -[[ $output_lines -lt 5000 ]] && - log_fail "Expected return of full list but only got $output_lines lines" +log_must [ $output_lines -ge 5000 ] # # Make sure we fail if the return is over the memory limit diff --git a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.return_nvlist_neg.ksh b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.return_nvlist_neg.ksh index 10afa6727847..a8d4dd31d84b 100755 --- a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.return_nvlist_neg.ksh +++ b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.return_nvlist_neg.ksh @@ -45,17 +45,13 @@ set -A args 'function() return 1 end' \ typeset -i last_index=$((${#args[*]} - 1)) for i in $(seq 0 $last_index); do log_note "running program: ${args[i]}" - log_mustnot_checkerror_program "execution failed" $TESTPOOL - <<-EOF - return ${args[i]} - EOF + log_mustnot_checkerror_program "execution failed" $TESTPOOL - <<<"return ${args[i]}" ((i = i + 1)) done for i in $(seq 0 $last_index); do log_note "running program: ${args[i]}" - log_mustnot_checkerror_program "execution failed" $TESTPOOL - <<-EOF - error(${args[i]}) - EOF + log_mustnot_checkerror_program "execution failed" $TESTPOOL - <<<"error(${args[i]})" ((i = i + 1)) done diff --git a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.return_nvlist_pos.ksh b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.return_nvlist_pos.ksh index 8b4ef6e760b0..75ade2b45d12 100755 --- a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.return_nvlist_pos.ksh +++ b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.return_nvlist_pos.ksh @@ -39,18 +39,14 @@ log_assert "Returning valid lua constructs works." typeset -i i=0 while (( i < ${#args[*]} )); do log_note "running program: return ${args[i]}" - log_must_program $TESTPOOL - <<-EOF - return ${args[i]} - EOF + log_must_program $TESTPOOL - <<<"return ${args[i]}" ((i = i + 1)) done typeset -i i=0 while (( i < ${#args[*]} )); do log_note "running program: error(${args[i]})" - log_mustnot_checkerror_program "in function 'error'" $TESTPOOL - <<-EOF - error(${args[i]}) - EOF + log_mustnot_checkerror_program "in function 'error'" $TESTPOOL - <<<"error(${args[i]})" ((i = i + 1)) done diff --git a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.timeout.ksh b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.timeout.ksh index 22ea37548173..905a3c327cce 100755 --- a/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.timeout.ksh +++ b/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.timeout.ksh @@ -34,15 +34,11 @@ function test_instr_limit { typeset lim=$1 - error=$(zfs program -t $lim $TESTPOOL $ZCP_ROOT/lua_core/tst.timeout.zcp 2>&1) - [[ $? -ne 0 ]] || log_fail "Channel program with limit $lim exited 0: $error" - - instrs_run=$(echo $error | awk -F "chunk" '{print $2}' | awk '{print $1}') - if [[ $instrs_run -lt $(( $lim - 100 )) ]]; then - log_fail "Runtime (${instrs_run} instr) < limit (${lim} - 100 instr)" - elif [[ $instrs_run -gt $(( $lim + 100 )) ]]; then - log_fail "Runtime (${instrs_run} instr) > limit (${lim} + 100 instr)" - fi + log_mustnot eval 'error=$(zfs program -t '$lim' $TESTPOOL $ZCP_ROOT/lua_core/tst.timeout.zcp 2>&1)' + + read -r instrs_run _ < <(echo $error | awk -F "chunk" '{print $2}') + log_must [ $instrs_run -ge $(( $lim - 100 )) ] + log_must [ $instrs_run -le $(( $lim + 100 )) ] log_note "With limit $lim the program ended after $instrs_run instructions" } diff --git a/tests/zfs-tests/tests/functional/channel_program/synctask_core/Makefile.am b/tests/zfs-tests/tests/functional/channel_program/synctask_core/Makefile.am deleted file mode 100644 index 4d9aa9cebbfc..000000000000 --- a/tests/zfs-tests/tests/functional/channel_program/synctask_core/Makefile.am +++ /dev/null @@ -1,53 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/channel_program/synctask_core -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - tst.destroy_fs.ksh \ - tst.destroy_snap.ksh \ - tst.get_count_and_limit.ksh \ - tst.get_index_props.ksh \ - tst.get_mountpoint.ksh \ - tst.get_neg.ksh \ - tst.get_number_props.ksh \ - tst.get_string_props.ksh \ - tst.get_type.ksh \ - tst.get_userquota.ksh \ - tst.get_written.ksh \ - tst.inherit.ksh \ - tst.list_bookmarks.ksh \ - tst.list_children.ksh \ - tst.list_clones.ksh \ - tst.list_holds.ksh \ - tst.list_snapshots.ksh \ - tst.list_system_props.ksh \ - tst.list_user_props.ksh \ - tst.parse_args_neg.ksh \ - tst.promote_conflict.ksh \ - tst.promote_multiple.ksh \ - tst.promote_simple.ksh \ - tst.rollback_mult.ksh \ - tst.rollback_one.ksh \ - tst.set_props.ksh \ - tst.snapshot_destroy.ksh \ - tst.snapshot_neg.ksh \ - tst.snapshot_recursive.ksh \ - tst.bookmark.create.ksh \ - tst.bookmark.copy.ksh \ - tst.snapshot_simple.ksh \ - tst.terminate_by_signal.ksh - -dist_pkgdata_DATA = \ - tst.get_index_props.out \ - tst.get_index_props.zcp \ - tst.get_number_props.out \ - tst.get_number_props.zcp \ - tst.get_string_props.out \ - tst.get_string_props.zcp \ - tst.promote_conflict.zcp \ - tst.set_props.zcp \ - tst.snapshot_destroy.zcp \ - tst.snapshot_neg.zcp \ - tst.snapshot_recursive.zcp \ - tst.snapshot_simple.zcp \ - tst.bookmark.create.zcp \ - tst.bookmark.copy.zcp diff --git a/tests/zfs-tests/tests/functional/channel_program/synctask_core/tst.terminate_by_signal.ksh b/tests/zfs-tests/tests/functional/channel_program/synctask_core/tst.terminate_by_signal.ksh index 2c9014a08483..53d5b819daf6 100755 --- a/tests/zfs-tests/tests/functional/channel_program/synctask_core/tst.terminate_by_signal.ksh +++ b/tests/zfs-tests/tests/functional/channel_program/synctask_core/tst.terminate_by_signal.ksh @@ -86,13 +86,10 @@ log_pos kill $CHILD # Make sure the channel program did not fully complete by enforcing # that not all of the snapshots were created. # -snap_count=$(zfs list -t snapshot | grep $TESTPOOL | wc -l) +snap_count=$(zfs list -t snapshot | grep -c $TESTPOOL) log_note "$snap_count snapshots created by ZCP" -if [ "$snap_count" -eq 0 ]; then - log_fail "Channel program failed to run." -elif [ "$snap_count" -gt 90 ]; then - log_fail "Too many snapshots after a cancel ($snap_count)." -else - log_pass "Canceling a long-running channel program works." -fi +log_mustnot [ "$snap_count" -eq 0 ] +log_mustnot [ "$snap_count" -gt 90 ] + +log_pass "Cancelling a long-running channel program works." diff --git a/tests/zfs-tests/tests/functional/chattr/Makefile.am b/tests/zfs-tests/tests/functional/chattr/Makefile.am deleted file mode 100644 index 431208e587ec..000000000000 --- a/tests/zfs-tests/tests/functional/chattr/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/chattr -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - chattr_001_pos.ksh \ - chattr_002_neg.ksh diff --git a/tests/zfs-tests/tests/functional/chattr/chattr_001_pos.ksh b/tests/zfs-tests/tests/functional/chattr/chattr_001_pos.ksh index cb8c2ead593a..c263c0f4fadf 100755 --- a/tests/zfs-tests/tests/functional/chattr/chattr_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/chattr/chattr_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/chattr/chattr_002_neg.ksh b/tests/zfs-tests/tests/functional/chattr/chattr_002_neg.ksh index cad1b0a38a4d..758de3fd277d 100755 --- a/tests/zfs-tests/tests/functional/chattr/chattr_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/chattr/chattr_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/chattr/cleanup.ksh b/tests/zfs-tests/tests/functional/chattr/cleanup.ksh index 59875d609ad9..59a93a652d05 100755 --- a/tests/zfs-tests/tests/functional/chattr/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/chattr/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/chattr/setup.ksh b/tests/zfs-tests/tests/functional/chattr/setup.ksh index d4b3cdcaba92..d9fc7be4e4fe 100755 --- a/tests/zfs-tests/tests/functional/chattr/setup.ksh +++ b/tests/zfs-tests/tests/functional/chattr/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -47,10 +47,8 @@ log_must add_user $QGROUP $QUSER2 # # chmod 0750 $HOME # -user_run $QUSER1 zfs list -if [ $? -ne 0 ]; then +user_run $QUSER1 zfs list || log_unsupported "Test user $QUSER1 cannot execute zfs utilities" -fi DISK=${DISKS%% *} default_setup $DISK diff --git a/tests/zfs-tests/tests/functional/checksum/.gitignore b/tests/zfs-tests/tests/functional/checksum/.gitignore deleted file mode 100644 index 0411d5aa47dc..000000000000 --- a/tests/zfs-tests/tests/functional/checksum/.gitignore +++ /dev/null @@ -1,4 +0,0 @@ -skein_test -edonr_test -sha2_test - diff --git a/tests/zfs-tests/tests/functional/checksum/Makefile.am b/tests/zfs-tests/tests/functional/checksum/Makefile.am deleted file mode 100644 index 717098aa0723..000000000000 --- a/tests/zfs-tests/tests/functional/checksum/Makefile.am +++ /dev/null @@ -1,31 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -LDADD = \ - $(abs_top_builddir)/lib/libicp/libicp.la \ - $(abs_top_builddir)/lib/libspl/libspl_assert.la - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/checksum - -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - run_edonr_test.ksh \ - run_sha2_test.ksh \ - run_skein_test.ksh \ - filetest_001_pos.ksh \ - filetest_002_pos.ksh - -dist_pkgdata_DATA = \ - default.cfg - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/checksum - -pkgexec_PROGRAMS = \ - edonr_test \ - skein_test \ - sha2_test - -skein_test_SOURCES = skein_test.c -sha2_test_SOURCES = sha2_test.c - -edonr_test_SOURCES = edonr_test.c diff --git a/tests/zfs-tests/tests/functional/checksum/cleanup.ksh b/tests/zfs-tests/tests/functional/checksum/cleanup.ksh index 19b0f35cf768..7d2380db5c6d 100755 --- a/tests/zfs-tests/tests/functional/checksum/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/checksum/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/checksum/default.cfg b/tests/zfs-tests/tests/functional/checksum/default.cfg index afb956093d8a..ca13be4e4d5d 100644 --- a/tests/zfs-tests/tests/functional/checksum/default.cfg +++ b/tests/zfs-tests/tests/functional/checksum/default.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -30,4 +30,4 @@ . $STF_SUITE/include/libtest.shlib -set -A CHECKSUM_TYPES "fletcher2" "fletcher4" "sha256" "sha512" "skein" "edonr" +set -A CHECKSUM_TYPES "fletcher2" "fletcher4" "blake3" "sha256" "sha512" "skein" "edonr" diff --git a/tests/zfs-tests/tests/functional/checksum/filetest_001_pos.ksh b/tests/zfs-tests/tests/functional/checksum/filetest_001_pos.ksh index 615b41f312b6..7257da488383 100755 --- a/tests/zfs-tests/tests/functional/checksum/filetest_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/checksum/filetest_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -85,7 +85,7 @@ log_must zpool import $TESTPOOL log_must zpool scrub $TESTPOOL log_must wait_scrubbed $TESTPOOL -cksum=$(zpool status -P -v $TESTPOOL | grep "$firstvdev" | awk '{print $5}') +cksum=$(zpool status -P -v $TESTPOOL | awk -v v="$firstvdev" '$0 ~ v {print $5}') log_assert "Normal file write test saw $cksum checksum errors" log_must [ $cksum -eq 0 ] @@ -105,8 +105,7 @@ while [[ $j -lt ${#CHECKSUM_TYPES[*]} ]]; do log_must zpool scrub $TESTPOOL log_must wait_scrubbed $TESTPOOL - cksum=$(zpool status -P -v $TESTPOOL | grep "$firstvdev" | \ - awk '{print $5}') + cksum=$(zpool status -P -v $TESTPOOL | awk -v v="$firstvdev" '$0 ~ v {print $5}') log_assert "Checksum '$type' caught $cksum checksum errors" log_must [ $cksum -ne 0 ] diff --git a/tests/zfs-tests/tests/functional/checksum/filetest_002_pos.ksh b/tests/zfs-tests/tests/functional/checksum/filetest_002_pos.ksh index 921a4b392a45..a0be1c2050b9 100755 --- a/tests/zfs-tests/tests/functional/checksum/filetest_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/checksum/filetest_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/hkdf/cleanup.ksh b/tests/zfs-tests/tests/functional/checksum/run_blake3_test.ksh similarity index 61% rename from tests/zfs-tests/tests/functional/hkdf/cleanup.ksh rename to tests/zfs-tests/tests/functional/checksum/run_blake3_test.ksh index 2bdca1950d37..cf1ca70328e1 100755 --- a/tests/zfs-tests/tests/functional/hkdf/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/checksum/run_blake3_test.ksh @@ -1,4 +1,4 @@ -#!/bin/ksh +#!/bin/ksh -p # # This file and its contents are supplied under the terms of the @@ -12,11 +12,19 @@ # # -# Copyright (c) 2017 by Datto Inc. All rights reserved. +# Copyright (c) 2015, 2016 by Delphix. All rights reserved. # . $STF_SUITE/include/libtest.shlib -verify_runnable "global" +# +# Description: +# Run the tests for the BLAKE3 hash algorithm. +# + +log_assert "Run the tests for the BLAKE3 hash algorithm." + +freq=$(get_cpu_freq) +log_must blake3_test $freq -log_pass +log_pass "BLAKE3 tests passed." diff --git a/tests/zfs-tests/tests/functional/checksum/run_edonr_test.ksh b/tests/zfs-tests/tests/functional/checksum/run_edonr_test.ksh index 42e88a8c8e3f..de5b21918edb 100755 --- a/tests/zfs-tests/tests/functional/checksum/run_edonr_test.ksh +++ b/tests/zfs-tests/tests/functional/checksum/run_edonr_test.ksh @@ -25,6 +25,6 @@ log_assert "Run the tests for the EdonR hash algorithm." freq=$(get_cpu_freq) -log_must $STF_SUITE/tests/functional/checksum/edonr_test $freq +log_must edonr_test $freq log_pass "EdonR tests passed." diff --git a/tests/zfs-tests/tests/functional/checksum/run_sha2_test.ksh b/tests/zfs-tests/tests/functional/checksum/run_sha2_test.ksh index e238d7a53e6d..23954a5d35fe 100755 --- a/tests/zfs-tests/tests/functional/checksum/run_sha2_test.ksh +++ b/tests/zfs-tests/tests/functional/checksum/run_sha2_test.ksh @@ -25,6 +25,6 @@ log_assert "Run the tests for the SHA-2 hash algorithm." freq=$(get_cpu_freq) -log_must $STF_SUITE/tests/functional/checksum/sha2_test $freq +log_must sha2_test $freq log_pass "SHA-2 tests passed." diff --git a/tests/zfs-tests/tests/functional/checksum/run_skein_test.ksh b/tests/zfs-tests/tests/functional/checksum/run_skein_test.ksh index b3a33c3ca8bc..d59bde206d77 100755 --- a/tests/zfs-tests/tests/functional/checksum/run_skein_test.ksh +++ b/tests/zfs-tests/tests/functional/checksum/run_skein_test.ksh @@ -25,6 +25,6 @@ log_assert "Run the tests for the Skein hash algorithm." freq=$(get_cpu_freq) -log_must $STF_SUITE/tests/functional/checksum/skein_test $freq +log_must skein_test $freq log_pass "Skein tests passed." diff --git a/tests/zfs-tests/tests/functional/checksum/setup.ksh b/tests/zfs-tests/tests/functional/checksum/setup.ksh index 204069de7383..be72cacc0fc2 100755 --- a/tests/zfs-tests/tests/functional/checksum/setup.ksh +++ b/tests/zfs-tests/tests/functional/checksum/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/clean_mirror/Makefile.am b/tests/zfs-tests/tests/functional/clean_mirror/Makefile.am deleted file mode 100644 index 2bc67709fffb..000000000000 --- a/tests/zfs-tests/tests/functional/clean_mirror/Makefile.am +++ /dev/null @@ -1,12 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/clean_mirror -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - clean_mirror_001_pos.ksh \ - clean_mirror_002_pos.ksh \ - clean_mirror_003_pos.ksh \ - clean_mirror_004_pos.ksh - -dist_pkgdata_DATA = \ - clean_mirror_common.kshlib \ - default.cfg diff --git a/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_001_pos.ksh b/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_001_pos.ksh index 8dd337a6efb9..f644b8f81325 100755 --- a/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_002_pos.ksh b/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_002_pos.ksh index a156368d0c89..6246b0c09f07 100755 --- a/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_003_pos.ksh b/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_003_pos.ksh index 144dcf1c0b72..3a64001eb5ac 100755 --- a/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_004_pos.ksh b/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_004_pos.ksh index ce1aac56d85a..cfbd81941023 100755 --- a/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_common.kshlib b/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_common.kshlib index cf1d77d9ccb5..1a75816f2c03 100644 --- a/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_common.kshlib +++ b/tests/zfs-tests/tests/functional/clean_mirror/clean_mirror_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/clean_mirror/cleanup.ksh b/tests/zfs-tests/tests/functional/clean_mirror/cleanup.ksh index d5614bef0420..3b54cff0e4c6 100755 --- a/tests/zfs-tests/tests/functional/clean_mirror/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/clean_mirror/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/clean_mirror/default.cfg b/tests/zfs-tests/tests/functional/clean_mirror/default.cfg index dfe7f1d0ba33..83ec31f29930 100644 --- a/tests/zfs-tests/tests/functional/clean_mirror/default.cfg +++ b/tests/zfs-tests/tests/functional/clean_mirror/default.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/clean_mirror/setup.ksh b/tests/zfs-tests/tests/functional/clean_mirror/setup.ksh index 0024f284d8f5..8d14f2d96e2d 100755 --- a/tests/zfs-tests/tests/functional/clean_mirror/setup.ksh +++ b/tests/zfs-tests/tests/functional/clean_mirror/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/Makefile.am deleted file mode 100644 index 9951f96f31ef..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/Makefile.am +++ /dev/null @@ -1,67 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root -dist_pkgdata_DATA = \ - cli_common.kshlib - -SUBDIRS = \ - zdb \ - zfs \ - zfs_bookmark \ - zfs_change-key \ - zfs_clone \ - zfs_copies \ - zfs_create \ - zfs_destroy \ - zfs_diff \ - zfs_get \ - zfs_ids_to_path \ - zfs_inherit \ - zfs_jail \ - zfs_load-key \ - zfs_mount \ - zfs_program \ - zfs_promote \ - zfs_property \ - zfs_receive \ - zfs_rename \ - zfs_reservation \ - zfs_rollback \ - zfs_send \ - zfs_set \ - zfs_share \ - zfs_snapshot \ - zfs_sysfs \ - zfs_unload-key \ - zfs_unmount \ - zfs_unshare \ - zfs_upgrade \ - zfs_wait \ - zhack \ - zpool \ - zpool_add \ - zpool_attach \ - zpool_clear \ - zpool_create \ - zpool_destroy \ - zpool_detach \ - zpool_events \ - zpool_expand \ - zpool_export \ - zpool_get \ - zpool_history \ - zpool_import \ - zpool_initialize \ - zpool_labelclear \ - zpool_offline \ - zpool_online \ - zpool_remove \ - zpool_reopen \ - zpool_replace \ - zpool_resilver \ - zpool_scrub \ - zpool_set \ - zpool_split \ - zpool_status \ - zpool_sync \ - zpool_trim \ - zpool_upgrade \ - zpool_wait diff --git a/tests/zfs-tests/tests/functional/cli_root/cli_common.kshlib b/tests/zfs-tests/tests/functional/cli_root/cli_common.kshlib index 4788de598663..bdb2d86f0380 100644 --- a/tests/zfs-tests/tests/functional/cli_root/cli_common.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/cli_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -31,21 +31,13 @@ . $STF_SUITE/include/libtest.shlib # -# Get the checksum and size of the file. -# -function get_cksum # -{ - return $(cksum $1 | awk '{print $1 $2}') -} - -# -# Compare the check sum of target files with the original file +# Compare the checksum of target files with the original file # function compare_cksum # ... { typeset orig_data=$1 - typeset orig_sum=$(get_cksum $orig_data) + typeset orig_sum=$(cksum < $orig_data) typeset target_sum="" typeset bad_data_list="" typeset -i bad_count=0 @@ -58,7 +50,7 @@ function compare_cksum # ... continue fi - target_sum=$(get_cksum $data) + target_sum=$(cksum < $data) if [[ $target_sum != $orig_sum ]]; then bad_data_list="$bad_data_list $data" (( bad_count +=1 )) diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zdb/Makefile.am deleted file mode 100644 index c1d4bf5a47ba..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zdb/Makefile.am +++ /dev/null @@ -1,20 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zdb -dist_pkgdata_SCRIPTS = \ - zdb_002_pos.ksh \ - zdb_003_pos.ksh \ - zdb_004_pos.ksh \ - zdb_005_pos.ksh \ - zdb_006_pos.ksh \ - zdb_args_neg.ksh \ - zdb_args_pos.ksh \ - zdb_block_size_histogram.ksh \ - zdb_checksum.ksh \ - zdb_decompress.ksh \ - zdb_decompress_zstd.ksh \ - zdb_object_range_neg.ksh \ - zdb_object_range_pos.ksh \ - zdb_display_block.ksh \ - zdb_label_checksum.ksh \ - zdb_objset_id.ksh \ - zdb_recover.ksh \ - zdb_recover_2.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_003_pos.ksh index 36f1929dd193..0e4dcf8e6f1e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_003_pos.ksh @@ -60,8 +60,8 @@ log_note "$DEVS" log_must dd if=/dev/${DISK[0]} of=/dev/${DISK[1]} bs=1K count=256 conv=notrunc for x in 0 1 ; do - config_count=$(zdb -l $DEV_RDSKDIR/${DISK[$x]} | grep -c features_for_read) - (( $? != 0)) && log_fail "failed to get config_count from DISK[$x]" + config_count=$(zdb -l $DEV_RDSKDIR/${DISK[$x]} | grep -c features_for_read) || + log_fail "failed to get config_count from DISK[$x]" log_note "vdev $x: message_count $config_count" [ $config_count -ne ${config_count[$x]} ] && \ log_fail "zdb produces an incorrect number of configuration dumps." diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_004_pos.ksh index 2c6e6e9be070..52bcc0311375 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_004_pos.ksh @@ -70,12 +70,14 @@ log_must dd if=$DEV_RDSKDIR/${DISK[0]} of=$DEV_RDSKDIR/${DISK[1]} bs=1K count=25 ubs=$(zdb -lu ${DISK[1]} | grep -e LABEL -e Uberblock -e 'labels = ') log_note "vdev 1: ubs $ubs" +set -o pipefail ub_dump_counts=$(zdb -lu ${DISK[1]} | \ awk ' /LABEL/ {label=$NF; blocks[label]=0}; /Uberblock/ {blocks[label]++}; - END {print blocks[0],blocks[1],blocks[2],blocks[3]}') -(( $? != 0)) && log_fail "failed to get ub_dump_counts from DISK[1]" + END {print blocks[0],blocks[1],blocks[2],blocks[3]}') || + log_fail "failed to get ub_dump_counts from DISK[1]" log_note "vdev 1: ub_dump_counts $ub_dump_counts" +set +o pipefail set -A dump_count $ub_dump_counts for label in 0 1 2 3; do diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_args_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_args_neg.ksh index cb88def7cc3e..5b34151e7588 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_args_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_args_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_args_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_args_pos.ksh index 4c2fc15ec025..37f45eae55f1 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_args_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_args_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -59,7 +59,7 @@ function test_imported_pool { typeset -a args=("-A" "-b" "-C" "-c" "-d" "-D" "-G" "-h" "-i" "-L" \ "-M" "-P" "-s" "-v" "-Y" "-y") - for i in ${args[@]}; do + for i in ${args[@]}; do log_must eval "zdb $i $TESTPOOL >/dev/null" done } @@ -69,7 +69,7 @@ function test_exported_pool log_must zpool export $TESTPOOL typeset -a args=("-A" "-b" "-C" "-c" "-d" "-D" "-F" "-G" "-h" "-i" "-L" "-M" \ "-P" "-s" "-v" "-X" "-Y" "-y") - for i in ${args[@]}; do + for i in ${args[@]}; do log_must eval "zdb -e $i $TESTPOOL >/dev/null" done log_must zpool import $TESTPOOL @@ -81,7 +81,7 @@ function test_vdev VDEVS=$(get_pool_devices ${TESTPOOL} ${DEV_RDSKDIR}) log_note $VDEVS set -A VDEV_ARRAY $VDEVS - for i in ${args[@]}; do + for i in ${args[@]}; do log_must eval "zdb -l $i ${VDEV_ARRAY[0]} >/dev/null" done } @@ -89,7 +89,7 @@ function test_vdev function test_metaslab { typeset -a args=("-A" "-L" "-P" "-Y") - for i in ${args[@]}; do + for i in ${args[@]}; do log_must eval "zdb -m $i $TESTPOOL >/dev/null" done } diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_block_size_histogram.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_block_size_histogram.ksh index 6ad93d87ca9a..0a4d24fa695a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_block_size_histogram.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_block_size_histogram.ksh @@ -115,8 +115,7 @@ function histo_populate_test_pool of=/${pool}/B_${this_rs}/file_${filenum} \ bs=${this_rs} count=${thiscount} \ iflag=fullblock 2>&1 | \ - egrep -v -e "records in" -e "records out" \ - -e "bytes.*copied" + grep -ve "records in" -e "records out" -e "bytes.*copied" ((filenum+=1)) done done @@ -146,8 +145,6 @@ function histo_check_test_pool typeset -i this_rs typeset -i this_ri typeset -i sum_filesizes=0 - typeset dumped - typeset stripped let histo_check_pool_size=$(get_pool_prop size ${pool}) if [[ ! ${histo_check_pool_size} =~ ${re_number} ]]; then @@ -158,11 +155,9 @@ function histo_check_test_pool log_fail "hctp: max_pool_record_size is not numeric ${max_pool_record_size}" fi - dumped="${TEST_BASE_DIR}/${pool}_dump.txt" stripped="${TEST_BASE_DIR}/${pool}_stripped.txt" zdb -Pbbb ${pool} | \ - tee ${dumped} | \ sed -e '1,/^block[ ][ ]*psize[ ][ ]*lsize.*$/d' \ -e '/^size[ ]*Count/d' -e '/^$/,$d' \ > ${stripped} @@ -247,6 +242,8 @@ function histo_check_test_pool fi fi done < ${stripped} + rm "${stripped}" + if [ ${fail_value} -gt 0 ]; then if [ ${error_count} -eq 1 ]; then log_note "hctp: There was ${error_count} error" diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_checksum.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_checksum.ksh index 4f661262a72d..d79933fd5b59 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_checksum.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_checksum.ksh @@ -53,13 +53,13 @@ log_note "file $init_data has object number $obj" sync_pool $TESTPOOL output=$(zdb -ddddddbbbbbb $TESTPOOL/$TESTFS $obj 2> /dev/null \ - |grep -m 1 "L0 DVA" |head -n1) + | grep -m 1 "L0 DVA") dva=$(sed -Ene 's/^.+DVA\[0\]=<([^>]+)>.*$/\1/p' <<< "$output") log_note "block 0 of $init_data has a DVA of $dva" cksum_expected=$(sed -Ene 's/^.+ cksum=([a-z0-9:]+)$/\1/p' <<< "$output") log_note "expecting cksum $cksum_expected" output=$(zdb -R $TESTPOOL $dva:c 2> /dev/null) -result=$(grep $cksum_expected <<< "$output") -(( $? != 0 )) && log_fail "zdb -R failed to print the correct checksum" +grep -q $cksum_expected <<<"$output" || + log_fail "zdb -R failed to print the correct checksum" log_pass "zdb -R generates the correct checksum" diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_decompress.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_decompress.ksh index 1ebcbfb44953..dffed48908f5 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_decompress.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_decompress.ksh @@ -73,7 +73,7 @@ obj=${array[0]} log_note "file $init_data has object number $obj" output=$(zdb -ddddddbbbbbb $TESTPOOL/$TESTFS $obj 2> /dev/null \ - |grep -m 1 "L0 DVA" |head -n1) + |grep -m 1 "L0 DVA") dva=$(sed -Ene 's/^.+DVA\[0\]=<([^>]+)>.*$/\1/p' <<< "$output") log_note "block 0 of $init_data has a DVA of $dva" @@ -81,32 +81,28 @@ log_note "block 0 of $init_data has a DVA of $dva" size_str=$(sed -Ene 's/^.+ size=([^ ]+) .*$/\1/p' <<< "$output") log_note "block size $size_str" -vdev=$(echo "$dva" |awk '{split($0,array,":")} END{print array[1]}') -offset=$(echo "$dva" |awk '{split($0,array,":")} END{print array[2]}') -output=$(zdb -R $TESTPOOL $vdev:$offset:$size_str:d 2> /dev/null) -echo $output |grep $pattern > /dev/null -(( $? != 0 )) && log_fail "zdb -R :d failed to decompress the data properly" +IFS=: read -r vdev offset _ <<< "$dva" +output=$(zdb -R $TESTPOOL $vdev:$offset:$size_str:d) +echo $output | grep -q $pattern || log_fail "zdb -R :d failed to decompress the data properly" -output=$(zdb -R $TESTPOOL $vdev:$offset:$size_str:dr 2> /dev/null) -echo $output |grep $four_k > /dev/null -(( $? != 0 )) && log_fail "zdb -R :dr failed to decompress the data properly" +output=$(zdb -R $TESTPOOL $vdev:$offset:$size_str:dr) +echo $output | grep -q $four_k || log_fail "zdb -R :dr failed to decompress the data properly" -output=$(zdb -R $TESTPOOL $vdev:$offset:$size_str:dr 2> /dev/null) +output=$(zdb -R $TESTPOOL $vdev:$offset:$size_str:dr) result=${#output} (( $result != $blksize)) && log_fail \ "zdb -R failed to decompress the data to the length (${#output} != $size_str)" # decompress using lsize -lsize=$(echo $size_str |awk '{split($0,array,"/")} END{print array[1]}') -psize=$(echo $size_str |awk '{split($0,array,"/")} END{print array[2]}') -output=$(zdb -R $TESTPOOL $vdev:$offset:$lsize:dr 2> /dev/null) +IFS=/ read -r lsize psize _ <<< "$size_str" +output=$(zdb -R $TESTPOOL $vdev:$offset:$lsize:dr) result=${#output} (( $result != $blksize)) && log_fail \ "zdb -R failed to decompress the data (length ${#output} != $blksize)" # Specifying psize will decompress successfully , but not always to full # lsize since zdb has to guess lsize incrementally. -output=$(zdb -R $TESTPOOL $vdev:$offset:$psize:dr 2> /dev/null) +output=$(zdb -R $TESTPOOL $vdev:$offset:$psize:dr) result=${#output} # convert psize to decimal psize_orig=$psize diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_decompress_zstd.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_decompress_zstd.ksh index 238d49560461..c7e10bac4570 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_decompress_zstd.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_decompress_zstd.ksh @@ -63,52 +63,44 @@ done sync_pool $TESTPOOL true # get object number of file -listing=$(ls -i $init_data) -set -A array $listing -obj=${array[0]} +read -r obj _ < <(ls -i $init_data) log_note "file $init_data has object number $obj" output=$(zdb -Zddddddbbbbbb $TESTPOOL/$TESTFS $obj 2> /dev/null \ - |grep -m 1 "L0 DVA" |head -n1) + | grep -m 1 "L0 DVA") dva=$(sed -Ene 's/^.+DVA\[0\]=<([^>]+)>.*$/\1/p' <<< "$output") log_note "block 0 of $init_data has a DVA of $dva" # use the length reported by zdb -ddddddbbbbbb size_str=$(sed -Ene 's/^.+ size=([^ ]+) .*$/\1/p' <<< "$output") # convert sizes to decimal -lsize=$(echo $size_str |awk '{split($0,array,"/")} END{print array[1]}') +IFS='/' read -r lsize psize _ <<<"$size_str" lsize_orig=$lsize -lsize=${lsize%?} -lsize_bytes=$((16#$lsize)) -psize=$(echo $size_str |awk '{split($0,array,"/")} END{print array[2]}') psize_orig=$psize +lsize=${lsize%?} psize=${psize%?} +lsize_bytes=$((16#$lsize)) psize_bytes=$((16#$psize)) log_note "block size $size_str" # Get the ZSTD header reported by zdb -Z -zstd_str=$(sed -Ene 's/^.+ ZSTD:size=([^:]+):version=([^:]+):level=([^:]+):.*$/\1:\2:\3/p' <<< "$output") -zstd_size=$(echo "$zstd_str" |awk '{split($0,array,":")} END{print array[1]}') +read -r zstd_size zstd_version zstd_level < <(sed -Ene 's/^.+ ZSTD:size=([^:]+):version=([^:]+):level=([^:]+):.*$/\1 \2 \3/p' <<<"$output") log_note "ZSTD compressed size $zstd_size" (( $psize_bytes < $zstd_size )) && log_fail \ "zdb -Z failed: physical block size was less than header content length ($psize_bytes < $zstd_size)" -zstd_version=$(echo "$zstd_str" |awk '{split($0,array,":")} END{print array[2]}') log_note "ZSTD version $zstd_version" -zstd_level=$(echo "$zstd_str" |awk '{split($0,array,":")} END{print array[3]}') log_note "ZSTD level $zstd_level" (( $zstd_level != $random_level )) && log_fail \ "zdb -Z failed: compression level did not match header level ($zstd_level < $random_level)" -vdev=$(echo "$dva" |awk '{split($0,array,":")} END{print array[1]}') -offset=$(echo "$dva" |awk '{split($0,array,":")} END{print array[2]}') +IFS=':' read -r vdev offset _ <<<"$dva" # Check the first 1024 bytes output=$(ZDB_NO_ZLE="true" zdb -R $TESTPOOL $vdev:$offset:$size_str:dr 2> /dev/null) -outsize=$(wc -c <<< "$output") -(( $outsize != $blksize )) && log_fail \ -"zdb -Z failed to decompress the data to the expected length ($outsize != $lsize_bytes)" -cmp $init_data - <<< "$output" -(( $? != 0 )) && log_fail "zdb -R :dr failed to decompress the data properly" +(( ${#output} + 1 != $blksize )) && log_fail \ +"zdb -Z failed to decompress the data to the expected length (${#output} != $lsize_bytes)" +cmp $init_data - <<< "$output" || + log_fail "zdb -R :dr failed to decompress the data properly" log_pass "zdb -Z flag (ZSTD compression header) works as expected" diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_display_block.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_display_block.ksh index 5cc4575851f7..bab0dea2969b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_display_block.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_display_block.ksh @@ -69,16 +69,16 @@ obj=${array[0]} log_note "file $init_data has object number $obj" output=$(zdb -ddddddbbbbbb $TESTPOOL/$TESTFS $obj 2> /dev/null \ - |grep -m 1 "L1 DVA" |head -n1) + |grep -m 1 "L1 DVA" ) dva=$(sed -Ene 's/^.+DVA\[0\]=<([^>]+)>.*/\1/p' <<< "$output") log_note "first L1 block $init_data has a DVA of $dva" output=$(zdb -ddddddbbbbbb $TESTPOOL/$TESTFS $obj 2> /dev/null \ - |grep -m 1 "L0 DVA" |head -n1) + |grep -m 1 "L0 DVA" ) blk_out0=${output##*>} blk_out0=${blk_out0##+([[:space:]])} output=$(zdb -ddddddbbbbbb $TESTPOOL/$TESTFS $obj 2> /dev/null \ - |grep -m 1 "1000 L0 DVA" |head -n1) + |grep -m 1 "1000 L0 DVA" ) blk_out1=${output##*>} blk_out1=${blk_out1##+([[:space:]])} @@ -106,11 +106,11 @@ if [ "$output" != "$blk_out1" ]; then log_fail "zdb -R :b80d (block 1 display/decompress) failed" fi -vdev=$(echo "$dva" |awk '{split($0,array,":")} END{print array[1]}') -offset=$(echo "$dva" |awk '{split($0,array,":")} END{print array[2]}') +vdev=$(echo "$dva" | cut -d: -f1) +offset=$(echo "$dva" | cut -d: -f2) output=$(export ZDB_NO_ZLE=\"true\";\ zdb -R $TESTPOOL $vdev:$offset:$l1_read_size:id 2> /dev/null) -block_cnt=$(echo "$output" | grep 'L0' | wc -l) +block_cnt=$(echo "$output" | grep -c 'L0') if [ $block_cnt -ne $write_count ]; then log_fail "zdb -R :id (indirect block display) failed" fi @@ -120,7 +120,7 @@ vdev="$vdev.0" log_note "Reading from DVA $vdev:$offset:$l1_read_size" output=$(export ZDB_NO_ZLE=\"true\";\ zdb -R $TESTPOOL $vdev:$offset:$l1_read_size:id 2> /dev/null) -block_cnt=$(echo "$output" | grep 'L0' | wc -l) +block_cnt=$(echo "$output" | grep -c 'L0') if [ $block_cnt -ne $write_count ]; then log_fail "zdb -R 0.0:offset:length:id (indirect block display) failed" fi diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_object_range_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_object_range_neg.ksh index e2014405853d..e4664b52eb94 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_object_range_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_object_range_neg.ksh @@ -65,7 +65,7 @@ done # Specifying a non-existent object identifier returns an error obj_id_highest=$(zdb -P -dd $TESTPOOL/$TESTFS 2>/dev/null | - egrep "^ +-?([0-9]+ +){7}" | sort -n | tail -n 1 | awk '{print $1}') + grep -E "^ +-?([0-9]+ +){7}" | sort -n | awk 'END {print $1}') obj_id_invalid=$(( $obj_id_highest + 1 )) log_mustnot zdb -dd $TESTPOOL/$TESTFS $obj_id_invalid diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_object_range_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_object_range_pos.ksh index 1e63ac7d2f4e..2c85e6e932a6 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_object_range_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_object_range_pos.ksh @@ -42,8 +42,8 @@ function get_object_list_range begin=$2 end=$3 get_object_list $dataset | - while read line; do - obj=$(echo $line | awk '{print $1}') + while read -r line; do + read -r obj _ <<<"$line" if [[ $obj -ge $begin && $obj -le $end ]] ; then echo "$line" elif [[ $obj -gt $end ]] ; then @@ -60,8 +60,7 @@ function get_object_list_range function get_object_list { zdb -P -dd $@ 2>/dev/null | - egrep "^ +-?([0-9]+ +){7}" | - sed 's/^[[:space:]]*//' | + sed -E '/^ +-?([0-9]+ +){7}/!d;s/^[[:space:]]*//' | sort -n } @@ -142,7 +141,7 @@ log_must test "\n$actual\n" == "\n$expected\n" # Specifying individual object IDs works objects="$start1 $end1 $start2 $end2" expected="$objects" -actual=$(get_object_list $TESTPOOL/$TESTFS $objects | awk '{print $1}' | tr '\n' ' ') +actual=$(get_object_list $TESTPOOL/$TESTFS $objects | awk '{printf("%s ", $1)}' | tr '\n' ' ') log_must test "${actual% }" == "$expected" # Get all objects in the meta-objset to test m (spacemap) and z (zap) flags diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_objset_id.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_objset_id.ksh index accb125280f0..fdda9ba22638 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_objset_id.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_objset_id.ksh @@ -67,9 +67,7 @@ obj=${array[0]} log_note "file $init_data has object number $obj" sync_pool $TESTPOOL -output=$(zdb -d $TESTPOOL/$TESTFS) -objset_id=$(echo $output | awk '{split($0,array,",")} END{print array[2]}' | - awk '{split($0,array," ")} END{print array[2]}') +IFS=", " read -r _ _ _ _ objset_id _ < <(zdb -d $TESTPOOL/$TESTFS) objset_hex=$(printf "0x%X" $objset_id) log_note "objset $TESTPOOL/$TESTFS has objset ID $objset_id ($objset_hex)" @@ -77,54 +75,32 @@ for id in "$objset_id" "$objset_hex" do log_note "zdb -dddddd $TESTPOOL/$id $obj" output=$(zdb -dddddd $TESTPOOL/$id $obj) - reason="($TESTPOOL/$TESTFS not in zdb output)" - echo $output |grep "$TESTPOOL/$TESTFS" > /dev/null - (( $? != 0 )) && log_fail \ - "zdb -dddddd $TESTPOOL/$id $obj failed $reason" - reason="(file1 not in zdb output)" - echo $output |grep "file1" > /dev/null - (( $? != 0 )) && log_fail \ - "zdb -dddddd $TESTPOOL/$id $obj failed $reason" - obj=$(printf "0x%X" $obj) + echo $output | grep -q "$TESTPOOL/$TESTFS" || + log_fail "zdb -dddddd $TESTPOOL/$id $obj failed ($TESTPOOL/$TESTFS not in zdb output)" + echo $output | grep -q "file1" || + log_fail "zdb -dddddd $TESTPOOL/$id $obj failed (file1 not in zdb output)" + obj=$(printf "0x%X" $obj) log_note "zdb -NNNNNN $TESTPOOL/$id $obj" - output=$(zdb -NNNNNN $TESTPOOL/$id $obj) - reason="($TESTPOOL/$TESTFS not in zdb output)" - echo $output |grep "$TESTPOOL/$TESTFS" > /dev/null - (( $? != 0 )) && log_fail \ - "zdb -NNNNNN $TESTPOOL/$id $obj failed $reason" - reason="(file1 not in zdb output)" - echo $output |grep "file1" > /dev/null - (( $? != 0 )) && log_fail \ - "zdb -NNNNNN $TESTPOOL/$id $obj failed $reason" + output=$(zdb -NNNNNN $TESTPOOL/$id $obj) + echo $output | grep -q "$TESTPOOL/$TESTFS" || + log_fail "zdb -NNNNNN $TESTPOOL/$id $obj failed ($TESTPOOL/$TESTFS not in zdb output)" + echo $output | grep -q "file1" || + log_fail "zdb -NNNNNN $TESTPOOL/$id $obj failed (file1 not in zdb output)" done if is_linux; then - output=$(ls -1 /proc/spl/kstat/zfs/$TESTPOOL |grep objset- |tail -1) + output=$(ls -1 /proc/spl/kstat/zfs/$TESTPOOL | grep objset- | tail -1) objset_hex=${output#*-} - name_from_proc=$(cat /proc/spl/kstat/zfs/$TESTPOOL/$output | - grep dataset_name | awk '{split($0,array," ")} END{print array[3]}') + name_from_proc=$(grep dataset_name /proc/spl/kstat/zfs/$TESTPOOL/$output | cut -d' ' -f3) log_note "checking zdb output for $name_from_proc" - reason="(name $name_from_proc from proc not in zdb output)" - log_note "zdb -dddddd $TESTPOOL/$objset_hex" - output=$(zdb -dddddd $TESTPOOL/$objset_hex) - echo $output |grep "$name_from_proc" > /dev/null - (( $? != 0 )) && log_fail \ - "zdb -dddddd $TESTPOOL/$objset_hex failed $reason" + log_must eval "zdb -dddddd $TESTPOOL/$objset_hex | grep -q \"$name_from_proc\"" fi log_must zfs create $hex_ds log_must zfs create $num_ds -output=$(zdb -d $hex_ds) -reason="($TESTPOOL/0x400 not in zdb output)" -echo $output |grep "$hex_ds" > /dev/null -(( $? != 0 )) && log_fail \ - "zdb -d $hex_ds failed $reason" -output=$(zdb -d $num_ds) -reason="($num_ds not in zdb output)" -echo $output |grep "$num_ds" > /dev/null -(( $? != 0 )) && log_fail \ - "zdb -d $num_ds failed $reason" +log_must eval "zdb -d $hex_ds | grep -q \"$hex_ds\"" +log_must eval "zdb -d $num_ds | grep -q \"$num_ds\"" # force numeric interpretation, expect fail log_mustnot zdb -N $hex_ds diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_recover_2.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_recover_2.ksh index 6470327a1765..d4529ff01105 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_recover_2.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_recover_2.ksh @@ -48,7 +48,7 @@ verify_disk_count "$DISKS" 2 default_mirror_setup_noexit $DISKS file_write -o create -w -f $init_data -b $blksize -c $write_count -log_must echo "zfs" >> $init_data +echo "zfs" >> $init_data sync_pool $TESTPOOL output=$(zdb -r $TESTPOOL/$TESTFS file1 $tmpfile) diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs/Makefile.am deleted file mode 100644 index 8b0ee276a3b0..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_001_neg.ksh \ - zfs_002_pos.ksh \ - zfs_003_neg.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs/setup.ksh index 6a9af3bc28c3..4c719075a741 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs/zfs_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs/zfs_001_neg.ksh index c6e45c80dd30..24ffd4751688 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs/zfs_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs/zfs_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs/zfs_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs/zfs_002_pos.ksh index 51a7ce1d96d7..de434cc47b31 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs/zfs_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs/zfs_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -48,33 +48,23 @@ function cleanup { unset ZFS_ABORT - if is_freebsd && [[ -n $savedcorefile ]]; then - sysctl kern.corefile=$savedcorefile - fi + log_must pop_coredump_pattern "$coresavepath" - if [[ -d $corepath ]]; then - rm -rf $corepath - fi for ds in $fs1 $fs $ctr; do datasetexists $ds && destroy_dataset $ds -rRf done } -log_assert "With ZFS_ABORT set, all zfs commands can abort and generate a " \ - "core file." +log_assert "With ZFS_ABORT set, all zfs commands can abort and generate a core file." log_onexit cleanup +ctr=$TESTPOOL/$TESTCTR +log_must zfs create -p $ctr + # Preparation work for testing -savedcorefile="" -corepath=$TESTDIR/core +corepath=/$ctr corefile=$corepath/core.zfs -if [[ -d $corepath ]]; then - rm -rf $corepath -fi -log_must mkdir $corepath - -ctr=$TESTPOOL/$TESTCTR -log_must zfs create $ctr +coresavepath=$corepath/save fs=$ctr/$TESTFS fs1=$ctr/$TESTFS1 @@ -93,28 +83,12 @@ typeset badparams=("" "create" "destroy" "snapshot" "rollback" "clone" \ "promote" "rename" "list -*" "set" "get -*" "inherit" "mount -A" \ "unmount" "share" "unshare" "send" "receive") -if is_linux; then - ulimit -c unlimited - echo "$corefile" >/proc/sys/kernel/core_pattern - echo 0 >/proc/sys/kernel/core_uses_pid -elif is_freebsd; then - ulimit -c unlimited - savedcorefile=$(sysctl -n kern.corefile) - log_must sysctl kern.corefile=$corepath/core.%N -else - log_must coreadm -p ${corepath}/core.%f -fi - +log_must eval "push_coredump_pattern \"$corepath\" > \"$coresavepath\"" log_must export ZFS_ABORT=yes for subcmd in "${cmds[@]}" "${badparams[@]}"; do - zfs $subcmd >/dev/null 2>&1 && log_fail "$subcmd passed incorrectly." - if [[ ! -e $corefile ]]; then - log_fail "zfs $subcmd cannot generate core file with " \ - "ZFS_ABORT set." - fi - log_must rm -f $corefile + log_mustnot eval "zfs $subcmd" + log_must rm "$corefile" done -log_pass "With ZFS_ABORT set, zfs command can abort and generate core file " \ - "as expected." +log_pass "With ZFS_ABORT set, zfs command can abort and generate core file as expected." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs/zfs_003_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs/zfs_003_neg.ksh index 0438bae8f6ce..2f592642c3ba 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs/zfs_003_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs/zfs_003_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -45,7 +45,15 @@ verify_runnable "global" +function cleanup +{ + for file in $ZFS_DEV $MNTTAB; do + log_must eval "[ -e ${file} ] || mv ${file}.bak $file" + done +} + log_assert "zfs fails with unexpected scenario." +log_onexit cleanup #verify zfs failed if ZFS_DEV cannot be opened ZFS_DEV=/dev/zfs @@ -56,13 +64,11 @@ if is_linux; then fi for file in $ZFS_DEV $MNTTAB; do - if [[ -e $file ]]; then - mv $file ${file}.bak - fi + log_must mv $file ${file}.bak for cmd in "" "list" "get all" "mount"; do log_mustnot eval "zfs $cmd >/dev/null 2>&1" done - mv ${file}.bak $file + log_must mv ${file}.bak $file done log_pass "zfs fails with unexpected scenario as expected." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/Makefile.am deleted file mode 100644 index e71fbc85ca84..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_bookmark -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_bookmark_cliargs.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/cleanup.ksh index f84ac43e67c9..eed1c9ccf54f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/setup.ksh index 40953415c6b9..8815d150bbdb 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/zfs_bookmark_cliargs.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/zfs_bookmark_cliargs.ksh index 3a1cddb5c64a..b6cb0c60034f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/zfs_bookmark_cliargs.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/zfs_bookmark_cliargs.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -62,6 +62,8 @@ function cleanup bkmarkexists "$DATASET#$TESTBMCOPY" && \ destroy_dataset "$DATASET#$TESTBMCOPY" + + log_must rm -f "$TEST_BASE_DIR/zfstest_datastream.$$" } log_assert "'zfs bookmark' should work only when passed valid arguments." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_change-key/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_change-key/Makefile.am deleted file mode 100644 index 72d6e4700e17..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_change-key/Makefile.am +++ /dev/null @@ -1,12 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_change-key -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_change-key.ksh \ - zfs_change-key_child.ksh \ - zfs_change-key_clones.ksh \ - zfs_change-key_inherit.ksh \ - zfs_change-key_format.ksh \ - zfs_change-key_load.ksh \ - zfs_change-key_location.ksh \ - zfs_change-key_pbkdf2iters.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_change-key/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_change-key/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_change-key/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_change-key/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_change-key/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_change-key/setup.ksh index 6a9af3bc28c3..4c719075a741 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_change-key/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_change-key/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/Makefile.am deleted file mode 100644 index 06099c0c2b79..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/Makefile.am +++ /dev/null @@ -1,17 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_clone -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_clone_001_neg.ksh \ - zfs_clone_002_pos.ksh \ - zfs_clone_003_pos.ksh \ - zfs_clone_004_pos.ksh \ - zfs_clone_005_pos.ksh \ - zfs_clone_006_pos.ksh \ - zfs_clone_007_pos.ksh \ - zfs_clone_008_neg.ksh \ - zfs_clone_009_neg.ksh \ - zfs_clone_010_pos.ksh \ - zfs_clone_encrypted.ksh \ - zfs_clone_deeply_nested.ksh \ - zfs_clone_rm_nested.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/cleanup.ksh index d247991fbbdb..d9f935900f90 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/setup.ksh index 985554f8cd2e..0a74ee3a879d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_001_neg.ksh index e6ffa26c0208..e8d5d73dd841 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_002_pos.ksh index 96eb3ea48d64..755079ed57df 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_003_pos.ksh index 6484de9c91a8..c113fb27aaf9 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -49,6 +49,7 @@ verify_runnable "both" function cleanup { snapexists $SNAPFS && destroy_dataset $SNAPFS -Rf + log_must rm -df "/tmp/mnt$$" } log_onexit cleanup diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_004_pos.ksh index 1c4c579f26bd..993b279411d0 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_005_pos.ksh index 6f17b176734a..e965ec1251b3 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_006_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_006_pos.ksh index f2f7a5bcd077..30fafca90054 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_007_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_007_pos.ksh index 4bfb3d5f78ab..a615392d70dc 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -40,8 +40,7 @@ # 2. Verify it succeed while upgrade, but fails while the version downgraded. # -ZFS_VERSION=$(zfs upgrade | head -1 | awk '{print $NF}' \ - | sed -e 's/\.//g') +ZFS_VERSION=$(zfs upgrade | grep -wom1 '[[:digit:]]*') verify_runnable "both" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_008_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_008_neg.ksh index 2f2b0ca18d64..96f0c16a45f4 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_008_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_008_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_009_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_009_neg.ksh index 6cdf5717fa92..e328d5b64158 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_009_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_009_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_010_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_010_pos.ksh index 13f5418d4bf5..643bf1cf28e7 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_010_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_010_pos.ksh @@ -221,7 +221,7 @@ for (( i = 1; i <= (ZFS_MAXPROPLEN / 200 + 1); i++ )); do log_must zfs clone ${fs}@snap ${fs}/${TESTCLONE}${xs}.${i} done clone_list=$(zfs list -o clones $fs@snap) -char_count=$(echo "$clone_list" | tail -1 | wc | awk '{print $3}') +char_count=$(echo "$clone_list" | tail -1 | wc -c) [[ $char_count -eq $ZFS_MAXPROPLEN ]] || \ log_fail "Clone list not truncated correctly. Unexpected character count" \ "$char_count" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/Makefile.am deleted file mode 100644 index 4ac103a8ec6a..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/Makefile.am +++ /dev/null @@ -1,14 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_copies -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_copies_001_pos.ksh \ - zfs_copies_002_pos.ksh \ - zfs_copies_003_pos.ksh \ - zfs_copies_004_neg.ksh \ - zfs_copies_005_neg.ksh \ - zfs_copies_006_pos.ksh - -dist_pkgdata_DATA = \ - zfs_copies.cfg \ - zfs_copies.kshlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/cleanup.ksh index f5e862593e1e..8c8d37276d1d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/setup.ksh index d4cf2095012b..f76b49af706f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies.cfg b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies.cfg index eb85e65c790e..ec3244c7a804 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies.kshlib b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies.kshlib index 1273ed59df30..5617bd01ba42 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_001_pos.ksh index 672692b59e42..1f8d770fc99f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_002_pos.ksh index 5774fb873f33..5b10189e8f57 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -59,7 +59,7 @@ log_assert "Verify that the space used by multiple copies is charged correctly." log_onexit cleanup for val in 1 2 3; do - log_must zfs create -o copies=$val $TESTPOOL/fs_$val + log_must zfs create -o compression=off -o copies=$val $TESTPOOL/fs_$val log_must mkfile $FILESIZE /$TESTPOOL/fs_$val/$FILE done @@ -94,11 +94,9 @@ done log_note "Verify df(1) can correctly display the space charged." for val in 1 2 3; do if is_freebsd; then - used=`df -m /$TESTPOOL/fs_$val | grep $TESTPOOL/fs_$val \ - | awk -v fs=fs_$val '$4 ~ fs {print $3}'` + used=`df -m /$TESTPOOL/fs_$val | awk -v pa=$TESTPOOL/fs_$val -v fs=fs_$val '$0 ~ pa && $4 ~ fs {print $3}'` else - used=`df -F zfs -k /$TESTPOOL/fs_$val/$FILE | grep $TESTPOOL/fs_$val \ - | awk '{print $3}'` + used=`df -F zfs -k /$TESTPOOL/fs_$val/$FILE | awk -v pa=$TESTPOOL/fs_$val '$0 ~ pa {print $3}'` (( used = used * 1024 )) # kb -> bytes fi check_used $used $val diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_003_pos.ksh index 94e72bce4e67..cfb3d3308db8 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_004_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_004_neg.ksh index dc007b11160d..ab250d4f2782 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_004_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_004_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_005_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_005_neg.ksh index 6a8b81ac5b87..d15629fb2bd0 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_005_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_005_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_006_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_006_pos.ksh index 6dc9306b33ef..28017b59a00f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_copies/zfs_copies_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_create/Makefile.am deleted file mode 100644 index 7515753c1bc2..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/Makefile.am +++ /dev/null @@ -1,28 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_create -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_create_001_pos.ksh \ - zfs_create_002_pos.ksh \ - zfs_create_003_pos.ksh \ - zfs_create_004_pos.ksh \ - zfs_create_005_pos.ksh \ - zfs_create_006_pos.ksh \ - zfs_create_007_pos.ksh \ - zfs_create_008_neg.ksh \ - zfs_create_009_neg.ksh \ - zfs_create_010_neg.ksh \ - zfs_create_011_pos.ksh \ - zfs_create_012_pos.ksh \ - zfs_create_013_pos.ksh \ - zfs_create_014_pos.ksh \ - zfs_create_encrypted.ksh \ - zfs_create_crypt_combos.ksh \ - zfs_create_dryrun.ksh \ - zfs_create_nomount.ksh \ - zfs_create_verbose.ksh - -dist_pkgdata_DATA = \ - properties.kshlib \ - zfs_create_common.kshlib \ - zfs_create.cfg diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/properties.kshlib b/tests/zfs-tests/tests/functional/cli_root/zfs_create/properties.kshlib index 4130ba446334..16d19bdbd8aa 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/properties.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/properties.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/setup.ksh index 6a9af3bc28c3..4c719075a741 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create.cfg b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create.cfg index 785d5a001603..807285ea6c73 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_001_pos.ksh index f74b2c9816f3..7b34ec33f3bf 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_002_pos.ksh index 0218e2e16b68..74b39cdaf983 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -65,8 +65,7 @@ while (( $j < ${#size[*]} )); do typeset cmdline="zfs create -s -V ${size[j]} \ $TESTPOOL/${TESTVOL}${size[j]}" - str=$(eval $cmdline 2>&1) - if (( $? == 0 )); then + if str=$(eval $cmdline 2>&1); then log_note "SUCCESS: $cmdline" log_must datasetexists $TESTPOOL/${TESTVOL}${size[j]} elif [[ $str == *${VOL_LIMIT_KEYWORD1}* || \ diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_003_pos.ksh index 120de10281db..798354db954b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_004_pos.ksh index 9e69366c8793..cc429419e67a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -49,24 +49,23 @@ verify_runnable "both" function cleanup { datasetexists $TESTPOOL/$TESTFS1 && destroy_dataset $TESTPOOL/$TESTFS1 -f + log_must rm -df "/tmp/mnt$$" } log_onexit cleanup -log_assert "'zfs create -o property=value filesystem' can successfully create \ - a ZFS filesystem with correct property set." +log_assert "'zfs create -o property=value filesystem' can successfully create" \ + "a ZFS filesystem with correct property set." typeset -i i=0 while (( $i < ${#RW_FS_PROP[*]} )); do log_must zfs create -o ${RW_FS_PROP[$i]} $TESTPOOL/$TESTFS1 - datasetexists $TESTPOOL/$TESTFS1 || \ - log_fail "zfs create $TESTPOOL/$TESTFS1 fail." - propertycheck $TESTPOOL/$TESTFS1 ${RW_FS_PROP[i]} || \ - log_fail "${RW_FS_PROP[i]} is failed to set." + log_must datasetexists $TESTPOOL/$TESTFS1 + log_must propertycheck $TESTPOOL/$TESTFS1 ${RW_FS_PROP[i]} log_must_busy zfs destroy -f $TESTPOOL/$TESTFS1 (( i = i + 1 )) done -log_pass "'zfs create -o property=value filesystem' can successfully create \ - a ZFS filesystem with correct property set." +log_pass "'zfs create -o property=value filesystem' can successfully create" \ + "a ZFS filesystem with correct property set." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_005_pos.ksh index 98cf70938e87..22f154400a1b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -48,15 +48,16 @@ verify_runnable "both" function cleanup { - datasetexists $TESTPOOL/$TESTFS1 && \ + datasetexists $TESTPOOL/$TESTFS1 && destroy_dataset $TESTPOOL/$TESTFS1 -f + log_must rm -df "/tmp/mnt$$" } log_onexit cleanup -log_assert "'zfs create -o property=value filesystem' can successfully create \ - a ZFS filesystem with multiple properties set." +log_assert "'zfs create -o property=value filesystem' can successfully create" \ + "a ZFS filesystem with multiple properties set." typeset -i i=0 typeset opts="" @@ -69,17 +70,15 @@ while (( $i < ${#RW_FS_PROP[*]} )); do done log_must zfs create $opts $TESTPOOL/$TESTFS1 -datasetexists $TESTPOOL/$TESTFS1 || \ - log_fail "zfs create $TESTPOOL/$TESTFS1 fail." +log_must datasetexists $TESTPOOL/$TESTFS1 i=0 while (( $i < ${#RW_FS_PROP[*]} )); do if [[ ${RW_FS_PROP[$i]} != *"checksum"* ]]; then - propertycheck $TESTPOOL/$TESTFS1 ${RW_FS_PROP[i]} || \ - log_fail "${RW_FS_PROP[i]} is failed to set." + log_must propertycheck $TESTPOOL/$TESTFS1 ${RW_FS_PROP[i]} fi (( i = i + 1 )) done -log_pass "'zfs create -o property=value filesystem' can successfully create \ - a ZFS filesystem with multiple properties set." +log_pass "'zfs create -o property=value filesystem' can successfully create" \ + "a ZFS filesystem with multiple properties set." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_006_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_006_pos.ksh index 551ae78cd239..733402fed39a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_007_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_007_pos.ksh index c49c77703409..bb6f9a21db13 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_008_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_008_neg.ksh index a905e50dfaa2..650d7dbcc3bd 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_008_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_008_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -56,7 +56,7 @@ set -A args "ab" "-?" "-cV" "-Vc" "-c -V" "c" "V" "--c" "-e" "-s" \ "-blah" "-cV 12k" "-s -cV 1P" "-sc" "-Vs 5g" "-o" "--o" "-O" "--O" \ "-o QuOta=none" "-o quota=non" "-o quota=abcd" "-o quota=0" "-o quota=" \ "-o ResErVaTi0n=none" "-o reserV=none" "-o reservation=abcd" "-o reserv=" \ - "-o recorDSize=64k" "-o recordsize=2048K" "-o recordsize=2M" \ + "-o recorDSize=64k" "-o recordsize=32768K" "-o recordsize=32M" \ "-o recordsize=256" "-o recsize=" "-o recsize=zero" "-o recordsize=0" \ "-o mountPoint=/tmp/tmpfile$$" "-o mountpoint=non0" "-o mountpoint=" \ "-o mountpoint=LEGACY" "-o mounpoint=none" \ diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_009_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_009_neg.ksh index 63f5e595ea38..f23f0a969df8 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_009_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_009_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -62,8 +62,7 @@ function cleanup # check to see if there is any new fs created during the test # if so destroy it. # - for dset in $(zfs list -H | \ - awk '{print $1}' | grep / ); do + for dset in $(zfs list -H | awk '$1 ~ /\/ {print $1}'); do found=false i=0 while (( $i < ${#existed_fs[*]} )); do @@ -99,7 +98,7 @@ log_assert "Verify 'zfs create ' fails with bad argumen datasetexists $TESTPOOL/$TESTFS || \ log_must zfs create $TESTPOOL/$TESTFS -set -A existed_fs $(zfs list -H | awk '{print $1}' | grep / ) +set -A existed_fs $(zfs list -H | awk '$1 ~ /\// {print $1}') log_mustnot zfs create $TESTPOOL log_mustnot zfs create $TESTPOOL/$TESTFS diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_010_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_010_neg.ksh index 4b1401d8649b..a872881b5494 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_010_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_010_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -62,8 +62,7 @@ function cleanup # check to see if there is any new fs created during the test # if so destroy it. # - for dset in $(zfs list -H | \ - awk '{print $1}' | grep / ); do + for dset in $(zfs list -H | awk '$1 ~ /\// {print $1}'); do found=false i=0 while (( $i < ${#existed_fs[*]} )); do @@ -108,7 +107,7 @@ set -A options "" "-s" datasetexists $TESTPOOL/$TESTVOL || \ log_must zfs create -V $VOLSIZE $TESTPOOL/$TESTVOL -set -A existed_fs $(zfs list -H | awk '{print $1}' | grep / ) +set -A existed_fs $(zfs list -H | awk '$1 ~ /\// {print $1}') log_mustnot zfs create -V $VOLSIZE $TESTPOOL/$TESTVOL log_mustnot zfs create -s -V $VOLSIZE $TESTPOOL/$TESTVOL diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_011_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_011_pos.ksh index c5012d4f34a8..2bbe6d84815f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_011_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_011_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_012_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_012_pos.ksh index a0b8d52f0c43..71706d4b0f65 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_012_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_012_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -41,8 +41,7 @@ # 2. Verify only the leaf filesystem to be version=1, others use the current version # -ZFS_VERSION=$(zfs upgrade | head -1 | awk '{print $NF}' \ - | sed -e 's/\.//g') +ZFS_VERSION=$(zfs upgrade | grep -wom1 '[[:digit:]]*') verify_runnable "both" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_013_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_013_pos.ksh index 993c6436a7b1..3fa5e246d8e4 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_013_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_013_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -63,8 +63,7 @@ while (( $j < ${#size[*]} )); do typeset cmdline="zfs create -s -V ${size[j]} \ $TESTPOOL/${LONGFSNAME}${size[j]}" - str=$(eval $cmdline 2>&1) - if (( $? == 0 )); then + if str=$(eval $cmdline 2>&1); then log_note "SUCCESS: $cmdline" log_must datasetexists $TESTPOOL/${LONGFSNAME}${size[j]} elif [[ $str == *${VOL_LIMIT_KEYWORD1}* || \ diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_014_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_014_pos.ksh index 2482a68dc089..8010c6df4586 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_014_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_014_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_common.kshlib b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_common.kshlib index 4c3f8b908c10..76a63a094807 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_common.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -40,15 +40,12 @@ function propertycheck { typeset dtst=$1 typeset propstr=$2 + typeset prop expect_value - typeset prop=$(echo $propstr | awk -F= '{print $1}') - typeset expect_value=$(echo $propstr | awk -F= '{print $2}') - typeset value=$(zfs get -H -p -o value $prop $dtst) + IFS='=' read -r prop expect_value <<<"$propstr" + typeset value=$(get_prop $prop $dtst) - if [[ "$expect_value" == "$value" ]]; then - return 0 - else - return 1 - fi + + [ "$expect_value" = "$value" ] } diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_dryrun.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_dryrun.ksh index 703ae8043d48..1e22da0045c7 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_dryrun.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_dryrun.ksh @@ -87,12 +87,11 @@ function dry_create_parseable typeset found_create=false log_note "$0: ${cmd[@]}" - out=$("${cmd[@]}") - (( $? == 0 )) || + out=$("${cmd[@]}") || log_fail "unexpected failure getting stdout from '${cmd[@]}'" datasetexists "$TESTPOOL/$TESTFS1" && log_fail "$TESTPOOL/$TESTFS1 unexpectedly created by '${cmd[@]}'" - echo "$out" | while IFS=$'\t' read -A toks; do + while IFS=$'\t' read -A toks; do log_note "verifying ${toks[@]}" case ${toks[0]} in create) @@ -118,7 +117,7 @@ function dry_create_parseable log_fail "Unexpected line ${toks[@]}" ;; esac - done + done <<<"$out" log_must test "$found_create" == "yes, I found create" log_must test "extra props: ${!exp[@]}" == "extra props: " diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_nomount.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_nomount.ksh index e1fbbe63ad31..5b0478e855cf 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_nomount.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_nomount.ksh @@ -30,7 +30,7 @@ verify_runnable "both" function cleanup { - local ds + typeset ds for ds in "$fs" "$vol"; do datasetexists "$ds" && destroy_dataset "$ds" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_verbose.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_verbose.ksh index acab500062ca..b18ffa404c42 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_verbose.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_create/zfs_create_verbose.ksh @@ -58,12 +58,11 @@ function dry_create_parseable done log_note "$0: ${cmd[@]}" - out=$("${cmd[@]}") - (( $? == 0 )) || + out=$("${cmd[@]}") || log_fail "unexpected failure getting stdout from '${cmd[@]}'" datasetexists "$TESTPOOL/$TESTFS1" || log_fail "$TESTPOOL/$TESTFS1 unexpectedly created by '${cmd[@]}'" - echo "$out" | while IFS=$'\t' read -A toks; do + while IFS=$'\t' read -A toks; do log_note "verifying ${toks[@]}" case ${toks[0]} in create_ancestors) @@ -107,7 +106,7 @@ function dry_create_parseable log_fail "Unexpected line ${toks[@]}" ;; esac - done + done <<<"$out" log_must test "$found_create" == "yes, I found create" log_must test "extra props: ${!exp[@]}" == "extra props: " diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/Makefile.am deleted file mode 100644 index 664f3d81aea6..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/Makefile.am +++ /dev/null @@ -1,30 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_destroy -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_clone_livelist_condense_and_disable.ksh \ - zfs_clone_livelist_condense_races.ksh \ - zfs_clone_livelist_dedup.ksh \ - zfs_destroy_001_pos.ksh \ - zfs_destroy_002_pos.ksh \ - zfs_destroy_003_pos.ksh \ - zfs_destroy_004_pos.ksh \ - zfs_destroy_005_neg.ksh \ - zfs_destroy_006_neg.ksh \ - zfs_destroy_007_neg.ksh \ - zfs_destroy_008_pos.ksh \ - zfs_destroy_009_pos.ksh \ - zfs_destroy_010_pos.ksh \ - zfs_destroy_011_pos.ksh \ - zfs_destroy_012_pos.ksh \ - zfs_destroy_013_neg.ksh \ - zfs_destroy_014_pos.ksh \ - zfs_destroy_015_pos.ksh \ - zfs_destroy_016_pos.ksh \ - zfs_destroy_clone_livelist.ksh \ - zfs_destroy_dev_removal.ksh \ - zfs_destroy_dev_removal_condense.ksh - -dist_pkgdata_DATA = \ - zfs_destroy_common.kshlib \ - zfs_destroy.cfg diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/cleanup.ksh index e838ba8a5947..8f25922aa7ac 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/setup.ksh index 6a9af3bc28c3..479ee930f27f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -29,4 +29,6 @@ DISK=${DISKS%% *} -default_setup $DISK +default_setup_noexit $DISK +log_must zfs set compression=off $TESTPOOL/$TESTFS +log_pass diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_clone_livelist_condense_and_disable.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_clone_livelist_condense_and_disable.ksh index 2f328ceac4ae..b9d6ad1e5dcf 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_clone_livelist_condense_and_disable.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_clone_livelist_condense_and_disable.ksh @@ -39,6 +39,7 @@ function cleanup set_tunable64 LIVELIST_MAX_ENTRIES $ORIGINAL_MAX # reset the minimum percent shared to 75 set_tunable32 LIVELIST_MIN_PERCENT_SHARED $ORIGINAL_MIN + log_must zfs inherit compression $TESTPOOL } function check_ll_len @@ -116,6 +117,11 @@ ORIGINAL_MAX=$(get_tunable LIVELIST_MAX_ENTRIES) ORIGINAL_MIN=$(get_tunable LIVELIST_MIN_PERCENT_SHARED) log_onexit cleanup +# You might think that setting compression=off for $TESTFS1 would be +# sufficient. You would be mistaken. +# You need compression=off for whatever the parent of $TESTFS1 is, +# and $TESTFS1. +log_must zfs set compression=off $TESTPOOL log_must zfs create $TESTPOOL/$TESTFS1 log_must mkfile 5m /$TESTPOOL/$TESTFS1/atestfile log_must zfs snapshot $TESTPOOL/$TESTFS1@snap diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_clone_livelist_condense_races.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_clone_livelist_condense_races.ksh index d83280e32dea..cf248bb87f94 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_clone_livelist_condense_races.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_clone_livelist_condense_races.ksh @@ -42,6 +42,7 @@ function cleanup # reset the condense tests to 0 set_tunable32 LIVELIST_CONDENSE_ZTHR_PAUSE 0 set_tunable32 LIVELIST_CONDENSE_SYNC_PAUSE 0 + log_must zfs inherit compression $TESTPOOL } function delete_race @@ -93,6 +94,11 @@ ORIGINAL_MAX=$(get_tunable LIVELIST_MAX_ENTRIES) log_onexit cleanup +# You might think that setting compression=off for $TESTFS1 would be +# sufficient. You would be mistaken. +# You need compression=off for whatever the parent of $TESTFS1 is, +# and $TESTFS1. +log_must zfs set compression=off $TESTPOOL log_must zfs create $TESTPOOL/$TESTFS1 log_must mkfile 100m /$TESTPOOL/$TESTFS1/atestfile sync_pool $TESTPOOL diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_clone_livelist_dedup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_clone_livelist_dedup.ksh index 00583402db89..9632cf01b3b6 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_clone_livelist_dedup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_clone_livelist_dedup.ksh @@ -80,9 +80,16 @@ function test_dedup ORIGINAL_MIN_SHARED=$(get_tunable LIVELIST_MIN_PERCENT_SHARED) log_onexit cleanup +# You might think that setting compression=off for $TESTFS1 would be +# sufficient. You would be mistaken. +# You need compression=off for whatever the parent of $TESTFS1 is, +# and $TESTFS1. +log_must zfs set compression=off $TESTPOOL log_must zfs create $TESTPOOL/$TESTFS1 log_must mkfile 5m /$TESTPOOL/$TESTFS1/atestfile log_must zfs snapshot $TESTPOOL/$TESTFS1@snap test_dedup +log_must zfs inherit compression $TESTPOOL + log_pass "Clone's livelist processes dedup blocks as expected." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy.cfg b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy.cfg index a62739b07ec8..5bb720bf1d6b 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_001_pos.ksh index 11157e93c78d..1f977628295e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_002_pos.ksh index 128921226a73..2eff646ea76a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_003_pos.ksh index 8b7e59b412b4..2c61bb7ecebb 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_004_pos.ksh index 9a2ff6bea36d..fedc30d550a4 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -98,8 +98,7 @@ log_must zfs set mountpoint=$mntp1 $fs1 log_must zfs set mountpoint=$mntp2 $clone for arg in "$fs1 $mntp1" "$clone $mntp2"; do - fs=`echo $arg | awk '{print $1}'` - mntp=`echo $arg | awk '{print $2}'` + read -r fs mntp <<<"$arg" log_note "Verify that 'zfs destroy' fails to" \ "destroy filesystem when it is busy." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_005_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_005_neg.ksh index 1c5b2cf1c741..f91909c2bc56 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_005_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_005_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_006_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_006_neg.ksh index 7dbcb242041d..2a9381cab5fa 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_006_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_006_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_007_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_007_neg.ksh index 57eb736fd88b..cafb87a42368 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_007_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_007_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_008_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_008_pos.ksh index f8b8f9005514..29e1e5e95cf6 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_009_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_009_pos.ksh index 57e5761bc064..aa841151603c 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_009_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_009_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_010_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_010_pos.ksh index 369029775f4b..91e6dfd96e1d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_010_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_010_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_clone_livelist.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_clone_livelist.ksh index 9165b03a1647..54d6096890b3 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_clone_livelist.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_clone_livelist.ksh @@ -42,6 +42,7 @@ function cleanup datasetexists $TESTPOOL/$TESTFS1 && destroy_dataset $TESTPOOL/$TESTFS1 -R # reset the livelist sublist size to its original value set_tunable64 LIVELIST_MAX_ENTRIES $ORIGINAL_MAX + log_must zfs inherit compression $TESTPOOL } function clone_write_file @@ -146,6 +147,11 @@ function test_clone_clone_promote ORIGINAL_MAX=$(get_tunable LIVELIST_MAX_ENTRIES) log_onexit cleanup +# You might think that setting compression=off for $TESTFS1 would be +# sufficient. You would be mistaken. +# You need compression=off for whatever the parent of $TESTFS1 is, +# and $TESTFS1. +log_must zfs set compression=off $TESTPOOL log_must zfs create $TESTPOOL/$TESTFS1 log_must mkfile 20m /$TESTPOOL/$TESTFS1/atestfile log_must zfs snapshot $TESTPOOL/$TESTFS1@snap diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_common.kshlib b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_common.kshlib index 51b3d2e513cc..bcc24efa6515 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_common.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_dev_removal_condense.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_dev_removal_condense.ksh index b4f2740c7aa2..fa5ebb6cce98 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_dev_removal_condense.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_destroy/zfs_destroy_dev_removal_condense.ksh @@ -53,7 +53,7 @@ VIRTUAL_DISK2=$TEST_BASE_DIR/disk2 log_must truncate -s $(($MINVDEVSIZE * 8)) $VIRTUAL_DISK1 log_must truncate -s $(($MINVDEVSIZE * 16)) $VIRTUAL_DISK2 -log_must zpool create $TESTPOOL2 $VIRTUAL_DISK1 +log_must zpool create -O compression=off $TESTPOOL2 $VIRTUAL_DISK1 log_must poolexists $TESTPOOL2 log_must zfs create $TESTPOOL2/$TESTFS diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_diff/.gitignore b/tests/zfs-tests/tests/functional/cli_root/zfs_diff/.gitignore deleted file mode 100644 index 7fa74c3575bd..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_diff/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/socket diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_diff/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_diff/Makefile.am deleted file mode 100644 index bfb01dcb8f86..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_diff/Makefile.am +++ /dev/null @@ -1,18 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_diff - -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - zfs_diff_changes.ksh \ - zfs_diff_cliargs.ksh \ - zfs_diff_encrypted.ksh \ - zfs_diff_mangle.ksh \ - zfs_diff_timestamp.ksh \ - zfs_diff_types.ksh - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_diff - -pkgexec_PROGRAMS = socket -socket_SOURCES = socket.c diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_diff/zfs_diff_changes.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_diff/zfs_diff_changes.ksh index 51a1b4aa1199..ca8df6dab910 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_diff/zfs_diff_changes.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_diff/zfs_diff_changes.ksh @@ -47,7 +47,7 @@ function verify_object_change # change="$2" log_must eval "zfs diff -F $TESTSNAP1 $TESTSNAP2 > $FILEDIFF" - diffchg="$(awk -v path="$path" '$NF == path { print $1 }' < $FILEDIFF)" + diffchg="$(awk -v path="$path" '$NF == path { print $1 }' $FILEDIFF)" if [[ "$diffchg" != "$change" ]]; then log_fail "Unexpected change for $path ('$diffchg' != '$change')" else diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_diff/zfs_diff_timestamp.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_diff/zfs_diff_timestamp.ksh index 0d08cf629572..81ee87f8aee9 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_diff/zfs_diff_timestamp.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_diff/zfs_diff_timestamp.ksh @@ -73,10 +73,8 @@ log_must zfs snapshot "$TESTSNAP2" # 3. Verify 'zfs diff -t' correctly display timestamps typeset -i count=0 log_must eval "zfs diff -t $TESTSNAP1 $TESTSNAP2 > $FILEDIFF" -awk '{print substr($1,1,index($1,".")-1)" "$NF}' < "$FILEDIFF" | while read line +awk '{print substr($1,1,index($1,".")-1) " " $NF}' "$FILEDIFF" | while read -r ctime file do - read ctime file <<< "$line" - # If path from 'zfs diff' is not a file (could be xattr object) skip it if [[ ! -f "$file" ]]; then continue; diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_diff/zfs_diff_types.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_diff/zfs_diff_types.ksh index 8e521b9f5a1e..414fde336c78 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_diff/zfs_diff_types.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_diff/zfs_diff_types.ksh @@ -52,7 +52,7 @@ function verify_object_class # symbol="$2" log_must eval "zfs diff -F $TESTSNAP1 $TESTSNAP2 > $FILEDIFF" - diffsym="$(awk -v path="$path" '$NF == path { print $2 }' < $FILEDIFF)" + diffsym="$(awk -v path="$path" '$NF == path { print $2 }' $FILEDIFF)" if [[ "$diffsym" != "$symbol" ]]; then log_fail "Unexpected type for $path ('$diffsym' != '$symbol')" else @@ -111,11 +111,7 @@ verify_object_class "$MNTPOINT/cdev" "C" # 2. | (Named pipe) log_must zfs snapshot "$TESTSNAP1" -if is_freebsd; then - log_must mkfifo "$MNTPOINT/fifo" -else - log_must mknod "$MNTPOINT/fifo" p -fi +log_must mkfifo "$MNTPOINT/fifo" log_must zfs snapshot "$TESTSNAP2" verify_object_class "$MNTPOINT/fifo" "|" @@ -127,7 +123,7 @@ verify_object_class "$MNTPOINT/dir" "/" # 2. = (Socket) log_must zfs snapshot "$TESTSNAP1" -log_must $STF_SUITE/tests/functional/cli_root/zfs_diff/socket "$MNTPOINT/sock" +log_must zfs_diff-socket "$MNTPOINT/sock" log_must zfs snapshot "$TESTSNAP2" verify_object_class "$MNTPOINT/sock" "=" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_get/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_get/Makefile.am deleted file mode 100644 index 22e95907675d..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_get/Makefile.am +++ /dev/null @@ -1,18 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_get -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_get_001_pos.ksh \ - zfs_get_002_pos.ksh \ - zfs_get_003_pos.ksh \ - zfs_get_004_pos.ksh \ - zfs_get_005_neg.ksh \ - zfs_get_006_neg.ksh \ - zfs_get_007_neg.ksh \ - zfs_get_008_pos.ksh \ - zfs_get_009_pos.ksh \ - zfs_get_010_neg.ksh - -dist_pkgdata_DATA = \ - zfs_get_common.kshlib \ - zfs_get_list_d.kshlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_get/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_get/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_get/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_get/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_get/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_get/setup.ksh index 9692385996d1..2a909eba4f87 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_get/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_get/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_001_pos.ksh index deb501698a9e..935a85285e8e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -102,16 +102,15 @@ function check_return_value found=0 while read line; do - typeset item - typeset value + typeset item value _ - item=$(echo $line | awk '{print $2}' 2>&1) + read -r _ item _ <<<"$line" if [[ $item == $p ]]; then ((found += 1)) cols=$(echo $line | awk '{print NF}') fi - value=$(echo $line | awk '{print $3}' 2>&1) + read -r _ _ value _ <<<"$line" if [[ $value == $uint64_max ]]; then log_fail "'zfs get $opt $props $dst' return " \ "UINT64_MAX constant." @@ -154,12 +153,7 @@ typeset -i i=0 while ((i < ${#dataset[@]})); do for opt in "${options[@]}"; do for prop in ${all_props[@]}; do - eval "zfs get $opt $prop ${dataset[i]} > \ - $TESTDIR/$TESTFILE0" - ret=$? - if [[ $ret != 0 ]]; then - log_fail "zfs get returned: $ret" - fi + log_must eval "zfs get $opt $prop ${dataset[i]} > $TESTDIR/$TESTFILE0" check_return_value ${dataset[i]} "$prop" "$opt" done done @@ -170,12 +164,7 @@ i=0 while ((i < ${#bookmark[@]})); do for opt in "${options[@]}"; do for prop in ${bookmark_props[@]}; do - eval "zfs get $opt $prop ${bookmark[i]} > \ - $TESTDIR/$TESTFILE0" - ret=$? - if [[ $ret != 0 ]]; then - log_fail "zfs get returned: $ret" - fi + log_must eval "zfs get $opt $prop ${bookmark[i]} > $TESTDIR/$TESTFILE0" check_return_value ${bookmark[i]} "$prop" "$opt" done done diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_002_pos.ksh index c3746514eae0..65986f6bd1a2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -85,11 +85,7 @@ for dst in ${dataset[@]}; do for opt in "" $(gen_option_str "${options[*]}" "-" "" $opt_numb); do for prop in $(gen_option_str "${props[*]}" "" "," $prop_numb) do - zfs get $opt $prop $dst > /dev/null 2>&1 - ret=$? - if [[ $ret != 0 ]]; then - log_fail "zfs get $opt $prop $dst (Code: $ret)" - fi + log_must eval "zfs get $opt $prop $dst > /dev/null 2>&1" done done done diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_003_pos.ksh index 2ea5aa0cb4cf..db80ccf1883f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -55,9 +55,8 @@ log_onexit cleanup log_must zfs set atime=on $TESTPOOL/$TESTFS log_must zfs mount -o remount,noatime $TESTPOOL/$TESTFS -value1=$(zfs get -H atime $TESTPOOL/$TESTFS | awk '{print $3}') -value2=$(zfs get -H all $TESTPOOL/$TESTFS | awk '{print $2 " " $3}' | \ - grep ^atime | awk '{print $2}') +read -r _ _ value1 _ < <(zfs get -H atime $TESTPOOL/$TESTFS) +read -r _ value2 < <(zfs get -H all $TESTPOOL/$TESTFS | cut -f2,3 | grep ^atime) if [[ $value1 != $value2 ]]; then log_fail "value1($value1) != value2($value2)" fi diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_004_pos.ksh index 3bc4c6240ed3..6101bae032c8 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -163,15 +163,12 @@ while (( i < ${#opts[*]} )); do log_must eval "zfs get ${opts[i]} all >$propfile" for ds in $allds; do - grep $ds $propfile >/dev/null 2>&1 - (( $? != 0 )) && \ + grep -q $ds $propfile || \ log_fail "There is no property for" \ "dataset $ds in 'get all' output." - propnum=`cat $propfile | awk '{print $1}' | \ - grep "${ds}$" | wc -l` - ds_type=`zfs get -H -o value type $ds` - case $ds_type in + propnum=$(awk -v ds="${ds}$" '$1 ~ ds {++cnt} END {print cnt}' $propfile) + case $(zfs get -H -o value type $ds) in filesystem ) (( propnum < fspropnum )) && \ (( failflag += 1 )) diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_005_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_005_neg.ksh index 510c54506d0f..d9be907909db 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_005_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_005_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -93,12 +93,7 @@ function test_options for dst in ${dataset[@]}; do for opt in $opts; do for prop in $props; do - zfs get $opt -- $prop $dst > /dev/null 2>&1 - ret=$? - if [[ $ret == 0 ]]; then - log_fail "zfs get $opt -- $prop " \ - "$dst unexpectedly succeeded." - fi + log_mustnot eval "zfs get $opt -- $prop $dst > /dev/null 2>&1" done done done @@ -118,12 +113,7 @@ function test_options_bookmarks for dst in ${bookmark[@]}; do for opt in $opts; do for prop in $props; do - zfs get $opt -- $prop $dst > /dev/null 2>&1 - ret=$? - if [[ $ret == 0 ]]; then - log_fail "zfs get $opt -- $prop " \ - "$dst unexpectedly succeeded." - fi + log_mustnot eval "zfs get $opt -- $prop $dst > /dev/null 2>&1" done done done diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_006_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_006_neg.ksh index a91685849069..feed960b17e4 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_006_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_006_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -47,7 +47,8 @@ log_assert "Verify 'zfs get all' fails with invalid combination scenarios." set -f # Force ksh ignore '?' and '*' set -A bad_combine "ALL" "\-R all" "-P all" "-h all" "-rph all" "-RpH all" "-PrH all" \ - "-o all" "-s all" "-? all" "-* all" "-?* all" "all -r" "all -p" \ + "-o all" "-s all" "-s none=getsubopt" "-t filesystem=getsubopt" \ + "-? all" "-* all" "-?* all" "all -r" "all -p" \ "all -H" "all -rp" "all -rH" "all -ph" "all -rpH" "all -r $TESTPOOL" \ "all -H $TESTPOOL" "all -p $TESTPOOL" "all -r -p -H $TESTPOOL" \ "all -rph $TESTPOOL" "all,available,reservation $TESTPOOL" \ diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_007_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_007_neg.ksh index 9bb00807f1b0..08a6bb7529b5 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_007_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_007_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -51,7 +51,7 @@ set -A badargs "o name,property,value,resource" "o name" \ "-o" "-o ,,,,," "-o -o -o -o" "-o NAME,PROPERTY,VALUE,SOURCE" \ "-o name,properTy,value,source" "-o name, property, value,source" \ "-o name:property:value:source" "-o name,property:value,source" \ - "-o name;property;value;source" + "-o name;property;value;source" "-o name=getsubopt" typeset -i i=0 while (( i < ${#badargs[*]} )) diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_008_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_008_pos.ksh index 296fe99968c8..6be432036845 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -53,7 +53,7 @@ set -A options " " "-r" "-H" "-p" "-rHp" "-o name" \ set -A props type used available creation volsize referenced compressratio \ mounted origin recordsize quota reservation mountpoint sharenfs \ checksum compression atime devices exec readonly setuid snapdir \ - aclinherit canmount primarycache secondarycache \ + aclinherit canmount primarycache secondarycache version \ usedbychildren usedbydataset usedbyrefreservation usedbysnapshots \ userquota@root groupquota@root userused@root groupused@root if is_freebsd; then @@ -62,11 +62,6 @@ else set -A props ${props[*]} zoned acltype fi -zfs upgrade -v > /dev/null 2>&1 -if [[ $? -eq 0 ]]; then - set -A props ${props[*]} version -fi - set -A dataset $TESTPOOL/$TESTCTR $TESTPOOL/$TESTFS $TESTPOOL/$TESTVOL \ $TESTPOOL/$TESTFS@$TESTSNAP $TESTPOOL/$TESTVOL@$TESTSNAP diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_009_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_009_pos.ksh index 7fd6918b43db..5686b63b41cc 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_009_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_009_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -39,7 +39,7 @@ # STRATEGY: # 1. Create a multiple depth filesystem. # 2. 'zfs get -d ' to get the output. -# 3. 'zfs get -r|egrep' to get the expected output. +# 3. 'zfs get -r|grep' to get the expected output. # 4. Compare the two outputs, they should be same. # @@ -55,7 +55,7 @@ log_onexit depth_fs_cleanup set -A all_props type used available creation volsize referenced \ compressratio mounted origin recordsize quota reservation mountpoint \ sharenfs checksum compression atime devices exec readonly setuid \ - snapdir aclinherit canmount primarycache secondarycache \ + snapdir aclinherit canmount primarycache secondarycache version \ usedbychildren usedbydataset usedbyrefreservation usedbysnapshots \ userquota@root groupquota@root userused@root groupused@root if is_freebsd; then @@ -64,11 +64,6 @@ else set -A all_props ${all_props[*]} zoned acltype fi -zfs upgrade -v > /dev/null 2>&1 -if [[ $? -eq 0 ]]; then - set -A all_props ${all_props[*]} version -fi - depth_fs_setup mntpnt=$(get_prop mountpoint $DEPTH_FS) @@ -86,7 +81,7 @@ for dp in ${depth_array[@]}; do done for prop in $(gen_option_str "${all_props[*]}" "" "," $prop_numb); do log_must eval "zfs get -H -d $dp -o name $prop $DEPTH_FS > $DEPTH_OUTPUT" - log_must eval "zfs get -rH -o name $prop $DEPTH_FS | egrep -e '$eg_opt' > $EXPECT_OUTPUT" + log_must eval "zfs get -rH -o name $prop $DEPTH_FS | grep -E '$eg_opt' > $EXPECT_OUTPUT" log_must diff $DEPTH_OUTPUT $EXPECT_OUTPUT done (( old_val=dp )) diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_010_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_010_neg.ksh index e1f53845fd9c..1b6e211e0681 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_010_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_010_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_common.kshlib b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_common.kshlib index 9b4eecf37127..12f157dedf19 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_common.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_list_d.kshlib b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_list_d.kshlib index d5388e6ef2ae..2084e0c18b10 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_list_d.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_get/zfs_get_list_d.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/Makefile.am deleted file mode 100644 index 5f5e3858787e..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_ids_to_path -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_ids_to_path_001_pos.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/cleanup.ksh index b5ff02217237..3b16891a6b18 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/setup.ksh index fd6f8f8bb078..f69103b7c373 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/zfs_ids_to_path_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/zfs_ids_to_path_001_pos.ksh index 563b3e00dd89..ff05a75725ad 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/zfs_ids_to_path_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/zfs_ids_to_path_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/Makefile.am deleted file mode 100644 index 95a51ec757ea..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_inherit -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - zfs_inherit_001_neg.ksh \ - zfs_inherit_002_neg.ksh \ - zfs_inherit_003_pos.ksh \ - zfs_inherit_mountpoint.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/setup.ksh index 9692385996d1..2a909eba4f87 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/zfs_inherit_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/zfs_inherit_001_neg.ksh index 62f255ca38b7..f1bd3d6e1fd3 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/zfs_inherit_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/zfs_inherit_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -47,14 +47,7 @@ verify_runnable "both" # Define uninherited properties and their short name. typeset props_str="type used available avail creation referenced refer \ compressratio ratio mounted origin quota reservation \ - reserv volsize volblocksize volblock" - -zfs upgrade -v > /dev/null 2>&1 -if [[ $? -eq 0 ]]; then - props_str="$props_str version" -fi - -set -A prop $props_str canmount + reserv volsize volblocksize volblock version canmount" log_assert "'zfs inherit' should return an error when attempting to inherit" \ diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/zfs_inherit_002_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/zfs_inherit_002_neg.ksh index 8e37e8dbcae0..0bb93a0789bf 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/zfs_inherit_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/zfs_inherit_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/zfs_inherit_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/zfs_inherit_003_pos.ksh index 3f7e4ff972ed..d27fa84c5d14 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/zfs_inherit_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_inherit/zfs_inherit_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_jail/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_jail/Makefile.am deleted file mode 100644 index b6dd7721e63e..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_jail/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_jail -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - jail.conf \ - zfs_jail_001_pos.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_jail/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_jail/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_jail/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_jail/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_jail/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_jail/setup.ksh index 6a9af3bc28c3..4c719075a741 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_jail/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_jail/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_jail/zfs_jail_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_jail/zfs_jail_001_pos.ksh index 2c0808110276..fa33bdb02ffa 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_jail/zfs_jail_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_jail/zfs_jail_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/Makefile.am deleted file mode 100644 index 7dfec435ce7f..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/Makefile.am +++ /dev/null @@ -1,18 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_load-key -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_load-key.ksh \ - zfs_load-key_all.ksh \ - zfs_load-key_file.ksh \ - zfs_load-key_https.ksh \ - zfs_load-key_location.ksh \ - zfs_load-key_noop.ksh \ - zfs_load-key_recursive.ksh - -dist_pkgdata_DATA = \ - zfs_load-key.cfg \ - zfs_load-key_common.kshlib \ - PASSPHRASE \ - HEXKEY \ - RAWKEY diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/cleanup.ksh index d397bcf4e9f0..60a1450f4c06 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/setup.ksh index 6cc5528ce5d7..43e4db5b09bf 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/zfs_load-key.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/zfs_load-key.ksh index 8af9f80cfbd7..11a97a83e25e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/zfs_load-key.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/zfs_load-key.ksh @@ -70,7 +70,8 @@ log_must key_available $TESTPOOL/$TESTFS1 log_mustnot eval "echo $PASSPHRASE | zfs load-key $TESTPOOL/$TESTFS1" -typeset DISK2="$(echo $DISKS | awk '{ print $2 }')" +typeset DISK2 _ +read -r _ DISK2 _ <<<"$DISKS" log_must eval "echo $PASSPHRASE | zpool create -O encryption=on" \ "-O keyformat=passphrase -O keylocation=prompt $TESTPOOL1 $DISK2" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/zfs_load-key_all.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/zfs_load-key_all.ksh index 3c18e4538d34..515753722d20 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/zfs_load-key_all.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/zfs_load-key_all.ksh @@ -56,7 +56,8 @@ log_must zfs create -o encryption=on -o keyformat=passphrase \ log_must zfs create -V 64M -o encryption=on -o keyformat=passphrase \ -o keylocation=file:///$TESTPOOL/pkey $TESTPOOL/zvol -typeset DISK2="$(echo $DISKS | awk '{ print $2}')" +typeset DISK2 _ +read -r _ DISK2 _ <<<"$DISKS" log_must zpool create -O encryption=on -O keyformat=passphrase \ -O keylocation=file:///$TESTPOOL/pkey $TESTPOOL1 $DISK2 diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/zfs_load-key_common.kshlib b/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/zfs_load-key_common.kshlib index f7461437c615..4a85999b4ab8 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/zfs_load-key_common.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_load-key/zfs_load-key_common.kshlib @@ -42,8 +42,7 @@ function key_available function key_unavailable { - key_available $1 && return 1 - return 0 + ! key_available $1 } function verify_keyformat diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/Makefile.am deleted file mode 100644 index 8c90b2e75e5a..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/Makefile.am +++ /dev/null @@ -1,29 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_mount -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_mount_001_pos.ksh \ - zfs_mount_002_pos.ksh \ - zfs_mount_003_pos.ksh \ - zfs_mount_004_pos.ksh \ - zfs_mount_005_pos.ksh \ - zfs_mount_006_pos.ksh \ - zfs_mount_007_pos.ksh \ - zfs_mount_008_pos.ksh \ - zfs_mount_009_neg.ksh \ - zfs_mount_010_neg.ksh \ - zfs_mount_011_neg.ksh \ - zfs_mount_012_pos.ksh \ - zfs_mount_013_pos.ksh \ - zfs_mount_014_neg.ksh \ - zfs_mount_all_001_pos.ksh \ - zfs_mount_all_fail.ksh \ - zfs_mount_all_mountpoints.ksh \ - zfs_mount_encrypted.ksh \ - zfs_mount_remount.ksh \ - zfs_mount_test_race.ksh \ - zfs_multi_mount.ksh - -dist_pkgdata_DATA = \ - zfs_mount.cfg \ - zfs_mount.kshlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/setup.ksh index 6a9af3bc28c3..4c719075a741 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount.cfg b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount.cfg index c8e46895d2a6..06d25faf0356 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount.kshlib b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount.kshlib index 85566e565319..8d06a32e7f52 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -35,10 +35,7 @@ function force_unmount #dev { typeset dev=$1 - ismounted $dev - if (( $? == 0 )); then - log_must zfs $unmountforce $dev - fi + ismounted $dev && log_must zfs $unmountforce $dev return 0 } diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_001_pos.ksh index c46a35695492..68ec78478571 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_002_pos.ksh index 8ee3e029fb4a..50161bc0a45d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_003_pos.ksh index 59d8c235f8fe..1beb501f5b64 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_004_pos.ksh index 4874c6d6530a..9dcd8af0bb35 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_005_pos.ksh index c0cb693f6ce6..b94d83f45119 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -77,14 +77,13 @@ cd $TESTDIR || \ zfs $mountcmd $TESTPOOL/$TESTFS ret=$? if is_linux || is_freebsd; then - (( ret == 0 )) || \ - log_fail "'zfs $mountcmd $TESTPOOL/$TESTFS' " \ - "unexpected return code of $ret." + expected=0 else - (( ret == 1 )) || \ - log_fail "'zfs $mountcmd $TESTPOOL/$TESTFS' " \ - "unexpected return code of $ret." + expected=1 fi +(( ret == expected )) || \ + log_fail "'zfs $mountcmd $TESTPOOL/$TESTFS' " \ + "unexpected return code of $ret." log_note "Make sure the filesystem $TESTPOOL/$TESTFS is unmounted" if is_linux || is_freebsd; then diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_006_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_006_pos.ksh index 5edce35c721e..2a2466f65c02 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_007_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_007_pos.ksh index 409dd06d7f88..84f478c938bd 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -112,7 +112,7 @@ function get_reverse_option typeset val typeset -i ind=0 - val=$(get_prop $prop $fs) || log_fail "get_prop $prop $fs" + val=$(get_prop $prop $fs) if [[ $val == "on" ]]; then (( ind = i * 2 )) else @@ -127,7 +127,6 @@ cleanup for property in ${properties[@]}; do orig_val=$(get_prop $property $fs) - (($? != 0)) && log_fail "get_prop $property $fs" # Set filesystem property temporarily reverse_opt=$(get_reverse_option $fs $property) @@ -135,7 +134,6 @@ for property in ${properties[@]}; do log_must zfs mount -o $reverse_opt $fs cur_val=$(get_prop $property $fs) - (($? != 0)) && log_fail "get_prop $property $fs" # In LZ, a user with all zone privileges can never with "devices" if ! is_global_zone && [[ $property == devices ]] ; then @@ -153,7 +151,6 @@ for property in ${properties[@]}; do log_must zfs mount $fs cur_val=$(get_prop $property $fs) - (($? != 0)) && log_fail "get_prop $property $fs" if [[ $orig_val != $cur_val ]]; then log_fail "zfs mount -o $reverse_opt " \ "change the property that is stored on disks" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_008_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_008_pos.ksh index 6a251330f62d..2c1029d551cf 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_009_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_009_neg.ksh index 978a5a4f2eea..02b3477a4452 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_009_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_009_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_010_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_010_neg.ksh index 53ebf1f26248..2c26157be835 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_010_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_010_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_011_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_011_neg.ksh index 95e2bc39727a..0e0879823619 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_011_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_011_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_all_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_all_001_pos.ksh index 765e242796b8..5fd963c03997 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_all_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_all_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_remount.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_remount.ksh index 259f0e99b65c..ed35b6e831a6 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_remount.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_remount.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_multi_mount.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_multi_mount.ksh index bd86eaa16bb4..c4b4f0773137 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_multi_mount.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_multi_mount.ksh @@ -56,7 +56,7 @@ log_must zfs create $TESTDS MNTPFS="$(get_prop mountpoint $TESTDS)" FILENAME="$MNTPFS/file" log_must mkfile 128k $FILENAME -log_must exec 9<> $FILENAME # open file +log_must eval "exec 9<> $FILENAME" # open file # 3. Lazy umount if is_freebsd; then @@ -74,7 +74,7 @@ log_must zfs mount $TESTDS if [ ! -f $FILENAME ]; then log_fail "Lazy remount failed" fi -log_must exec 9>&- # close fd +log_must eval "exec 9>&-" # close fd # 5. Verify multiple mounts of the same dataset are possible MNTPFS2="$MNTPFS-second" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_program/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_program/Makefile.am deleted file mode 100644 index d797a636b347..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_program/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_program -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_program_json.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_program/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_program/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_program/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_program/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_program/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_program/setup.ksh index 6a9af3bc28c3..4c719075a741 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_program/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_program/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/Makefile.am deleted file mode 100644 index 8a8c56a093d2..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/Makefile.am +++ /dev/null @@ -1,16 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_promote -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_promote_001_pos.ksh \ - zfs_promote_002_pos.ksh \ - zfs_promote_003_pos.ksh \ - zfs_promote_004_pos.ksh \ - zfs_promote_005_pos.ksh \ - zfs_promote_006_neg.ksh \ - zfs_promote_007_neg.ksh \ - zfs_promote_008_pos.ksh \ - zfs_promote_encryptionroot.ksh - -dist_pkgdata_DATA = \ - zfs_promote.cfg diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/setup.ksh index 3e2e2ce319a6..13a5febb25c1 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote.cfg b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote.cfg index 6407328efd91..02d7c097a519 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_001_pos.ksh index dc3ffd65ed38..173c46d76449 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_002_pos.ksh index 1a774da2c15c..fa7c82ee87ab 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_003_pos.ksh index 7dedaf91bed6..8d1ba8238a10 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_004_pos.ksh index b8a5ab9c1707..653b21139f54 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_005_pos.ksh index 289ddc6713bc..aa2deb9c3efd 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_006_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_006_neg.ksh index 7f08f28a9379..1aca734f46a1 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_006_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_006_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_007_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_007_neg.ksh index 95db7d9e6b4e..26740dcc4f6b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_007_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_007_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_008_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_008_pos.ksh index c37c9a6269e7..33dc403aefcc 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_promote/zfs_promote_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_property/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_property/Makefile.am deleted file mode 100644 index d8a5687b1ea2..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_property/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_property -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_written_property_001_pos.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_property/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_property/cleanup.ksh index e4998138f701..7bb00caed5a3 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_property/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_property/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_property/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_property/setup.ksh index 746a05015f8c..75bee687f5b7 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_property/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_property/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_property/zfs_written_property_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_property/zfs_written_property_001_pos.ksh index f31ff48099e9..f53a4ac71b68 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_property/zfs_written_property_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_property/zfs_written_property_001_pos.ksh @@ -217,12 +217,11 @@ for ds in $datasets; do sync_pool done recursive_output=$(zfs get -p -r written@current $TESTPOOL | \ - grep -v $TESTFS1@ | grep -v $TESTFS2@ | grep -v $TESTFS3@ | \ - grep -v "VALUE" | grep -v "-") + grep -ve $TESTFS1@ -e $TESTFS2@ -e $TESTFS3@ -e "VALUE" | grep -v "-") expected="$((20 * mb_block))" for ds in $datasets; do writtenat=$(echo "$recursive_output" | grep -v $ds/) - writtenat=$(echo "$writtenat" | grep $ds | awk '{print $3}') + writtenat=$(echo "$writtenat" | awk -v ds="$ds" '$0 ~ ds {print $3}') within_percent $writtenat $expected 99.5 || \ log_fail "Unexpected written@ value on $ds" done diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/Makefile.am deleted file mode 100644 index cf9490de4e81..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/Makefile.am +++ /dev/null @@ -1,34 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_receive -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_receive_001_pos.ksh \ - zfs_receive_002_pos.ksh \ - zfs_receive_003_pos.ksh \ - zfs_receive_004_neg.ksh \ - zfs_receive_005_neg.ksh \ - zfs_receive_006_pos.ksh \ - zfs_receive_007_neg.ksh \ - zfs_receive_008_pos.ksh \ - zfs_receive_009_neg.ksh \ - zfs_receive_010_pos.ksh \ - zfs_receive_011_pos.ksh \ - zfs_receive_012_pos.ksh \ - zfs_receive_013_pos.ksh \ - zfs_receive_014_pos.ksh \ - zfs_receive_015_pos.ksh \ - zfs_receive_016_pos.ksh \ - receive-o-x_props_override.ksh \ - receive-o-x_props_aliases.ksh \ - zfs_receive_from_encrypted.ksh \ - zfs_receive_from_zstd.ksh \ - zfs_receive_new_props.ksh \ - zfs_receive_to_encrypted.ksh \ - zfs_receive_raw.ksh \ - zfs_receive_raw_incremental.ksh \ - zfs_receive_raw_-d.ksh \ - zfs_receive_-e.ksh \ - zfs_receive_-wR-encrypted-mix.ksh - -dist_pkgdata_DATA = \ - zstd_test_data.txt diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/cleanup.ksh index cec69d4709f4..e3a213302614 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/receive-o-x_props_aliases.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/receive-o-x_props_aliases.ksh index d4b0aa2341fc..6e5b48ffe272 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/receive-o-x_props_aliases.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/receive-o-x_props_aliases.ksh @@ -129,7 +129,7 @@ log_must eval "check_prop_inherit $destsub copies $dest" log_must eval "check_prop_inherit $destsub atime $dest" log_must eval "check_prop_inherit $destsub checksum $dest" log_must eval "check_prop_source $destsub quota 0 default" -log_must eval "check_prop_source $destsub compression off default" +log_must eval "check_prop_source $destsub compression on default" # Cleanup log_must zfs destroy -r -f $orig log_must zfs destroy -r -f $dest diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/receive-o-x_props_override.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/receive-o-x_props_override.ksh index 2d3c15c62fc9..44fe60463b2f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/receive-o-x_props_override.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/receive-o-x_props_override.ksh @@ -135,7 +135,7 @@ log_must eval "check_prop_inherit $destsub atime $dest" log_must eval "check_prop_inherit $destsub checksum $dest" log_must eval "check_prop_inherit $destsub '$userprop:dest2' $dest" log_must eval "check_prop_source $destsub quota 0 default" -log_must eval "check_prop_source $destsub compression off default" +log_must eval "check_prop_source $destsub compression on default" log_must eval "check_prop_missing $dest '$userprop:orig'" log_must eval "check_prop_missing $destsub '$userprop:orig'" log_must eval "check_prop_source " \ diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/setup.ksh index 878189bafa2f..5fd3ff953096 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_-wR-encrypted-mix.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_-wR-encrypted-mix.ksh index 6e27130e0248..dc1653b698ee 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_-wR-encrypted-mix.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_-wR-encrypted-mix.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_001_pos.ksh index 8a6cd8c409be..d8051c919f4d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -105,10 +105,7 @@ for orig_fs in $datasets ; do typeset -i i=0 while (( i < ${#orig_snap[*]} )); do - file_write -o create -f ${orig_data[$i]} -b 512 \ - -c 8 >/dev/null 2>&1 - (( $? != 0 )) && \ - log_fail "Writing data into zfs filesystem fails." + log_must eval "file_write -o create -f ${orig_data[$i]} -b 512 -c 8 >/dev/null 2>&1" log_must zfs snapshot ${orig_snap[$i]} if (( i < 1 )); then log_must eval "zfs send ${orig_snap[$i]} > ${bkup[$i]}" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_002_pos.ksh index ba3fc49bd84c..9fb25a3e42b4 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_003_pos.ksh index cce387615315..9cac9f54df16 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -62,7 +62,7 @@ log_onexit cleanup ibackup=$TEST_BASE_DIR/ibackup.$$ fs=$TESTPOOL/$TESTFS; snap1=$fs@snap1; snap2=$fs@snap2 -mntpnt=$(get_prop mountpoint $fs) || log_fail "get_prop mountpoint $fs" +mntpnt=$(get_prop mountpoint $fs) log_must mkfile 10m $mntpnt/file1 log_must zfs snapshot $snap1 log_must mkfile 10m $mntpnt/file2 diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_004_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_004_neg.ksh index 2860cab2b679..9f4b32ade733 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_004_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_004_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_005_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_005_neg.ksh index d8c71f2c2877..1841e07bdb00 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_005_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_005_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -80,8 +80,8 @@ log_must zfs snapshot $init_snap log_must eval "zfs send $init_snap > $full_bkup" log_note "'zfs receive' fails with invalid send streams." -log_mustnot eval "cat /dev/null 2>&1 - if (( $? != 0 )); then - log_mustnot eval "zfs recv ${validopts[i]} $fs2 $fs3 < $bkup" - else + if echo ${validopts[i]} | grep -q "d"; then log_mustnot eval "zfs recv ${validopts[i]} $ctr1 $ctr2 < $bkup" + else + log_mustnot eval "zfs recv ${validopts[i]} $fs2 $fs3 < $bkup" fi (( i += 1 )) diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_010_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_010_pos.ksh index 84485977aa23..3e24d3e2e3b7 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_010_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_010_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -134,8 +134,7 @@ dd if=/dev/urandom of=$mntpnt/f18 bs=128k count=64 touch $mntpnt2/f18 # Remove objects that are intended to be missing. -rm $mntpnt/h17 -rm $mntpnt2/h* +rm $mntpnt/h17 $mntpnt2/h* # Add empty objects to $fs to exercise dmu_traverse code for i in {1..100}; do @@ -145,15 +144,15 @@ done log_must zfs snapshot $fs@s1 log_must zfs snapshot $fs2@s1 -log_must zfs send $fs@s1 > $TESTDIR/zr010p -log_must zfs send $fs2@s1 > $TESTDIR/zr010p2 +log_must eval "zfs send $fs@s1 > $TESTDIR/zr010p" +log_must eval "zfs send $fs2@s1 > $TESTDIR/zr010p2" # # Test that, when we receive a full send as a clone of itself, # nop-write saves us all the space used by data blocks. # -cat $TESTDIR/zr010p | log_must zfs receive -o origin=$fs@s1 $rfs +log_must eval "zfs receive -o origin=$fs@s1 $rfs < $TESTDIR/zr010p" size=$(get_prop used $rfs) size2=$(get_prop used $fs) if [[ $size -ge $(($size2 / 10)) ]] then @@ -163,15 +162,15 @@ fi log_must zfs destroy -fr $rfs # Correctness testing: receive each full send as a clone of the other fiesystem. -cat $TESTDIR/zr010p | log_must zfs receive -o origin=$fs2@s1 $rfs +log_must eval "zfs receive -o origin=$fs2@s1 $rfs < $TESTDIR/zr010p" mntpnt_old=$(get_prop mountpoint $fs) mntpnt_new=$(get_prop mountpoint $rfs) -log_must diff -r $mntpnt_old $mntpnt_new +log_must directory_diff $mntpnt_old $mntpnt_new log_must zfs destroy -r $rfs -cat $TESTDIR/zr010p2 | log_must zfs receive -o origin=$fs@s1 $rfs +log_must eval "zfs receive -o origin=$fs@s1 $rfs < $TESTDIR/zr010p2" mntpnt_old=$(get_prop mountpoint $fs2) mntpnt_new=$(get_prop mountpoint $rfs) -log_must diff -r $mntpnt_old $mntpnt_new +log_must directory_diff $mntpnt_old $mntpnt_new log_pass "zfs receive of full send as clone works" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_013_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_013_pos.ksh index e1e93e9d2a3b..87bb63b36e52 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_013_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_013_pos.ksh @@ -52,7 +52,7 @@ log_onexit cleanup truncate -s 100M $tpoolfile log_must zpool create $temppool $tpoolfile log_must zfs create $src_fs -src_mnt=$(get_prop mountpoint $src_fs) || log_fail "get_prop mountpoint $src_fs" +src_mnt=$(get_prop mountpoint $src_fs) echo blah > $src_mnt/blah zfs snapshot $src_fs@base diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_compressed_corrective.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_compressed_corrective.ksh new file mode 100755 index 000000000000..7f8eb0b138ee --- /dev/null +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_compressed_corrective.ksh @@ -0,0 +1,193 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# This file and its contents are supplied under the terms of the +# Common Development and Distribution License ("CDDL"), version 1.0. +# You may only use this file in accordance with the terms of version +# 1.0 of the CDDL. +# +# A full copy of the text of the CDDL should have accompanied this +# source. A copy of the CDDL is also available via the Internet at +# http://www.illumos.org/license/CDDL. +# +# CDDL HEADER END +# + +# +# Copyright (c) 2019 Datto, Inc. All rights reserved. +# Copyright (c) 2022 Axcient. +# + +. $STF_SUITE/include/libtest.shlib + +# +# DESCRIPTION: +# OpenZFS should be able to heal data using corrective recv when the send file +# was generated with the --compressed flag +# +# STRATEGY: +# 0. Create a file, checksum the file to be corrupted then compare it's checksum +# with the one obtained after healing under different testing scenarios: +# 1. Test healing (aka corrective) recv from a full send file +# 2. Test healing recv (aka heal recv) from an incremental send file +# 3. Test healing recv when compression on-disk is off but source was compressed +# 4. Test heal recv when compression on-disk is on but source was uncompressed +# 5. Test heal recv when compression doesn't match between send file and on-disk +# 6. Test healing recv of an encrypted dataset using an unencrypted send file +# 7. Test healing recv (on an encrypted dataset) using a raw send file +# 8. Test healing when specifying destination filesystem only (no snapshot) +# 9. Test incremental recv aftear healing recv +# + +verify_runnable "both" + +DISK=${DISKS%% *} + +backup=$TEST_BASE_DIR/backup +raw_backup=$TEST_BASE_DIR/raw_backup +ibackup=$TEST_BASE_DIR/ibackup +unc_backup=$TEST_BASE_DIR/unc_backup + +function cleanup +{ + log_must rm -f $backup $raw_backup $ibackup $unc_backup + + poolexists $TESTPOOL && destroy_pool $TESTPOOL + log_must zpool create -f $TESTPOOL $DISK +} + +function test_corrective_recv +{ + log_must zpool scrub -w $TESTPOOL + log_must zpool status -v $TESTPOOL + log_must eval "zpool status -v $TESTPOOL | \ + grep \"Permanent errors have been detected\"" + + # make sure we will read the corruption from disk by flushing the ARC + log_must zinject -a + + log_must eval "zfs recv -c $1 < $2" + + log_must zpool scrub -w $TESTPOOL + log_must zpool status -v $TESTPOOL + log_mustnot eval "zpool status -v $TESTPOOL | \ + grep \"Permanent errors have been detected\"" + typeset cksum=$(md5digest $file) + [[ "$cksum" == "$checksum" ]] || \ + log_fail "Checksums differ ($cksum != $checksum)" +} + +log_onexit cleanup + +log_assert "ZFS corrective receive should be able to heal data corruption" + +typeset passphrase="password" +typeset file="/$TESTPOOL/$TESTFS1/$TESTFILE0" + +log_must eval "poolexists $TESTPOOL && destroy_pool $TESTPOOL" +log_must zpool create -f -o feature@head_errlog=disabled $TESTPOOL $DISK + +log_must eval "echo $passphrase > /$TESTPOOL/pwd" + +log_must zfs create -o primarycache=none \ + -o atime=off -o compression=lz4 $TESTPOOL/$TESTFS1 + +log_must dd if=/dev/urandom of=$file bs=1024 count=1024 oflag=sync +log_must eval "echo 'aaaaaaaa' >> "$file +typeset checksum=$(md5digest $file) + +log_must zfs snapshot $TESTPOOL/$TESTFS1@snap1 + +# create full send file +log_must eval "zfs send --compressed $TESTPOOL/$TESTFS1@snap1 > $backup" + +log_must dd if=/dev/urandom of=$file"1" bs=1024 count=1024 oflag=sync +log_must eval "echo 'bbbbbbbb' >> "$file"1" +log_must zfs snapshot $TESTPOOL/$TESTFS1@snap2 +# create incremental send file +log_must eval "zfs send -c -i $TESTPOOL/$TESTFS1@snap1 \ + $TESTPOOL/$TESTFS1@snap2 > $ibackup" + +corrupt_blocks_at_level $file 0 +# test healing recv from a full send file +test_corrective_recv $TESTPOOL/$TESTFS1@snap1 $backup + +corrupt_blocks_at_level $file"1" 0 +# test healing recv from an incremental send file +test_corrective_recv $TESTPOOL/$TESTFS1@snap2 $ibackup + +# create new uncompressed dataset using our send file +log_must eval "zfs recv -o compression=off -o primarycache=none \ + $TESTPOOL/$TESTFS2 < $backup" +typeset compr=$(get_prop compression $TESTPOOL/$TESTFS2) +[[ "$compr" == "off" ]] || \ + log_fail "Unexpected compression $compr in recved dataset" +corrupt_blocks_at_level "/$TESTPOOL/$TESTFS2/$TESTFILE0" 0 +# test healing recv when compression on-disk is off but source was compressed +test_corrective_recv "$TESTPOOL/$TESTFS2@snap1" $backup + +# create a full sendfile from an uncompressed source +log_must eval "zfs send --compressed $TESTPOOL/$TESTFS2@snap1 > $unc_backup" +log_must eval "zfs recv -o compression=gzip -o primarycache=none \ + $TESTPOOL/testfs3 < $unc_backup" +typeset compr=$(get_prop compression $TESTPOOL/testfs3) +[[ "$compr" == "gzip" ]] || \ + log_fail "Unexpected compression $compr in recved dataset" +corrupt_blocks_at_level "/$TESTPOOL/testfs3/$TESTFILE0" 0 +# test healing recv when compression on-disk is on but source was uncompressed +test_corrective_recv "$TESTPOOL/testfs3@snap1" $unc_backup + +# create new compressed dataset using our send file +log_must eval "zfs recv -o compression=gzip -o primarycache=none \ + $TESTPOOL/testfs4 < $backup" +typeset compr=$(get_prop compression $TESTPOOL/testfs4) +[[ "$compr" == "gzip" ]] || \ + log_fail "Unexpected compression $compr in recved dataset" +corrupt_blocks_at_level "/$TESTPOOL/testfs4/$TESTFILE0" 0 +# test healing recv when compression doesn't match between send file and on-disk +test_corrective_recv "$TESTPOOL/testfs4@snap1" $backup + +# create new encrypted (and compressed) dataset using our send file +log_must eval "zfs recv -o encryption=aes-256-ccm -o keyformat=passphrase \ + -o keylocation=file:///$TESTPOOL/pwd -o primarycache=none \ + $TESTPOOL/testfs5 < $backup" +typeset encr=$(get_prop encryption $TESTPOOL/testfs5) +[[ "$encr" == "aes-256-ccm" ]] || \ + log_fail "Unexpected encryption $encr in recved dataset" +log_must eval "zfs send --raw $TESTPOOL/testfs5@snap1 > $raw_backup" +log_must eval "zfs send --compressed $TESTPOOL/testfs5@snap1 > $backup" +corrupt_blocks_at_level "/$TESTPOOL/testfs5/$TESTFILE0" 0 +# test healing recv of an encrypted dataset using an unencrypted send file +test_corrective_recv "$TESTPOOL/testfs5@snap1" $backup +corrupt_blocks_at_level "/$TESTPOOL/testfs5/$TESTFILE0" 0 +log_must zfs unmount $TESTPOOL/testfs5 +log_must zfs unload-key $TESTPOOL/testfs5 +# test healing recv (on an encrypted dataset) using a raw send file +test_corrective_recv "$TESTPOOL/testfs5@snap1" $raw_backup +# non raw send file healing an encrypted dataset with an unloaded key will fail +log_mustnot eval "zfs recv -c $TESTPOOL/testfs5@snap1 < $backup" + +log_must zfs rollback -r $TESTPOOL/$TESTFS1@snap1 +corrupt_blocks_at_level $file 0 +# test healing when specifying destination filesystem only (no snapshot) +test_corrective_recv $TESTPOOL/$TESTFS1 $backup +# test incremental recv aftear healing recv +log_must eval "zfs recv $TESTPOOL/$TESTFS1 < $ibackup" + +# test that healing recv can not be combined with incompatible recv options +log_mustnot eval "zfs recv -h -c $TESTPOOL/$TESTFS1@snap1 < $backup" +log_mustnot eval "zfs recv -F -c $TESTPOOL/$TESTFS1@snap1 < $backup" +log_mustnot eval "zfs recv -s -c $TESTPOOL/$TESTFS1@snap1 < $backup" +log_mustnot eval "zfs recv -u -c $TESTPOOL/$TESTFS1@snap1 < $backup" +log_mustnot eval "zfs recv -d -c $TESTPOOL/$TESTFS1@snap1 < $backup" +log_mustnot eval "zfs recv -e -c $TESTPOOL/$TESTFS1@snap1 < $backup" + +# ensure healing recv doesn't work when snap GUIDS don't match +log_mustnot eval "zfs recv -c $TESTPOOL/testfs5@snap2 < $backup" +log_mustnot eval "zfs recv -c $TESTPOOL/testfs5 < $backup" + +# test that healing recv doesn't work on non-existing snapshots +log_mustnot eval "zfs recv -c $TESTPOOL/$TESTFS1@missing < $backup" + +log_pass "OpenZFS corrective recv works for data healing" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_corrective.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_corrective.ksh new file mode 100755 index 000000000000..b2bbdf2a7bec --- /dev/null +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_corrective.ksh @@ -0,0 +1,192 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# This file and its contents are supplied under the terms of the +# Common Development and Distribution License ("CDDL"), version 1.0. +# You may only use this file in accordance with the terms of version +# 1.0 of the CDDL. +# +# A full copy of the text of the CDDL should have accompanied this +# source. A copy of the CDDL is also available via the Internet at +# http://www.illumos.org/license/CDDL. +# +# CDDL HEADER END +# + +# +# Copyright (c) 2019 Datto, Inc. All rights reserved. +# Copyright (c) 2022 Axcient. +# + +. $STF_SUITE/include/libtest.shlib + +# +# DESCRIPTION: +# OpenZFS should be able to heal data using corrective recv +# +# STRATEGY: +# 0. Create a file, checksum the file to be corrupted then compare it's checksum +# with the one obtained after healing under different testing scenarios: +# 1. Test healing (aka corrective) recv from a full send file +# 2. Test healing recv (aka heal recv) from an incremental send file +# 3. Test healing recv when compression on-disk is off but source was compressed +# 4. Test heal recv when compression on-disk is on but source was uncompressed +# 5. Test heal recv when compression doesn't match between send file and on-disk +# 6. Test healing recv of an encrypted dataset using an unencrypted send file +# 7. Test healing recv (on an encrypted dataset) using a raw send file +# 8. Test healing when specifying destination filesystem only (no snapshot) +# 9. Test incremental recv aftear healing recv +# + +verify_runnable "both" + +DISK=${DISKS%% *} + +backup=$TEST_BASE_DIR/backup +raw_backup=$TEST_BASE_DIR/raw_backup +ibackup=$TEST_BASE_DIR/ibackup +unc_backup=$TEST_BASE_DIR/unc_backup + +function cleanup +{ + log_must rm -f $backup $raw_backup $ibackup $unc_backup + + poolexists $TESTPOOL && destroy_pool $TESTPOOL + log_must zpool create -f $TESTPOOL $DISK +} + +function test_corrective_recv +{ + log_must zpool scrub -w $TESTPOOL + log_must zpool status -v $TESTPOOL + log_must eval "zpool status -v $TESTPOOL | \ + grep \"Permanent errors have been detected\"" + + # make sure we will read the corruption from disk by flushing the ARC + log_must zinject -a + + log_must eval "zfs recv -c $1 < $2" + + log_must zpool scrub -w $TESTPOOL + log_must zpool status -v $TESTPOOL + log_mustnot eval "zpool status -v $TESTPOOL | \ + grep \"Permanent errors have been detected\"" + typeset cksum=$(md5digest $file) + [[ "$cksum" == "$checksum" ]] || \ + log_fail "Checksums differ ($cksum != $checksum)" +} + +log_onexit cleanup + +log_assert "ZFS corrective receive should be able to heal data corruption" + +typeset passphrase="password" +typeset file="/$TESTPOOL/$TESTFS1/$TESTFILE0" + +log_must eval "poolexists $TESTPOOL && destroy_pool $TESTPOOL" +log_must zpool create -f -o feature@head_errlog=disabled $TESTPOOL $DISK + +log_must eval "echo $passphrase > /$TESTPOOL/pwd" + +log_must zfs create -o primarycache=none \ + -o atime=off -o compression=lz4 $TESTPOOL/$TESTFS1 + +log_must dd if=/dev/urandom of=$file bs=1024 count=1024 oflag=sync +log_must eval "echo 'aaaaaaaa' >> "$file +typeset checksum=$(md5digest $file) + +log_must zfs snapshot $TESTPOOL/$TESTFS1@snap1 + +# create full send file +log_must eval "zfs send $TESTPOOL/$TESTFS1@snap1 > $backup" + +log_must dd if=/dev/urandom of=$file"1" bs=1024 count=1024 oflag=sync +log_must eval "echo 'bbbbbbbb' >> "$file"1" +log_must zfs snapshot $TESTPOOL/$TESTFS1@snap2 +# create incremental send file +log_must eval "zfs send -i $TESTPOOL/$TESTFS1@snap1 \ + $TESTPOOL/$TESTFS1@snap2 > $ibackup" + +corrupt_blocks_at_level $file 0 +# test healing recv from a full send file +test_corrective_recv $TESTPOOL/$TESTFS1@snap1 $backup + +corrupt_blocks_at_level $file"1" 0 +# test healing recv from an incremental send file +test_corrective_recv $TESTPOOL/$TESTFS1@snap2 $ibackup + +# create new uncompressed dataset using our send file +log_must eval "zfs recv -o compression=off -o primarycache=none \ + $TESTPOOL/$TESTFS2 < $backup" +typeset compr=$(get_prop compression $TESTPOOL/$TESTFS2) +[[ "$compr" == "off" ]] || \ + log_fail "Unexpected compression $compr in recved dataset" +corrupt_blocks_at_level "/$TESTPOOL/$TESTFS2/$TESTFILE0" 0 +# test healing recv when compression on-disk is off but source was compressed +test_corrective_recv "$TESTPOOL/$TESTFS2@snap1" $backup + +# create a full sendfile from an uncompressed source +log_must eval "zfs send $TESTPOOL/$TESTFS2@snap1 > $unc_backup" +log_must eval "zfs recv -o compression=gzip -o primarycache=none \ + $TESTPOOL/testfs3 < $unc_backup" +typeset compr=$(get_prop compression $TESTPOOL/testfs3) +[[ "$compr" == "gzip" ]] || \ + log_fail "Unexpected compression $compr in recved dataset" +corrupt_blocks_at_level "/$TESTPOOL/testfs3/$TESTFILE0" 0 +# test healing recv when compression on-disk is on but source was uncompressed +test_corrective_recv "$TESTPOOL/testfs3@snap1" $unc_backup + +# create new compressed dataset using our send file +log_must eval "zfs recv -o compression=gzip -o primarycache=none \ + $TESTPOOL/testfs4 < $backup" +typeset compr=$(get_prop compression $TESTPOOL/testfs4) +[[ "$compr" == "gzip" ]] || \ + log_fail "Unexpected compression $compr in recved dataset" +corrupt_blocks_at_level "/$TESTPOOL/testfs4/$TESTFILE0" 0 +# test healing recv when compression doesn't match between send file and on-disk +test_corrective_recv "$TESTPOOL/testfs4@snap1" $backup + +# create new encrypted (and compressed) dataset using our send file +log_must eval "zfs recv -o encryption=aes-256-ccm -o keyformat=passphrase \ + -o keylocation=file:///$TESTPOOL/pwd -o primarycache=none \ + $TESTPOOL/testfs5 < $backup" +typeset encr=$(get_prop encryption $TESTPOOL/testfs5) +[[ "$encr" == "aes-256-ccm" ]] || \ + log_fail "Unexpected encryption $encr in recved dataset" +log_must eval "zfs send --raw $TESTPOOL/testfs5@snap1 > $raw_backup" +log_must eval "zfs send $TESTPOOL/testfs5@snap1 > $backup" +corrupt_blocks_at_level "/$TESTPOOL/testfs5/$TESTFILE0" 0 +# test healing recv of an encrypted dataset using an unencrypted send file +test_corrective_recv "$TESTPOOL/testfs5@snap1" $backup +corrupt_blocks_at_level "/$TESTPOOL/testfs5/$TESTFILE0" 0 +log_must zfs unmount $TESTPOOL/testfs5 +log_must zfs unload-key $TESTPOOL/testfs5 +# test healing recv (on an encrypted dataset) using a raw send file +test_corrective_recv "$TESTPOOL/testfs5@snap1" $raw_backup +# non raw send file healing an encrypted dataset with an unloaded key will fail +log_mustnot eval "zfs recv -c $TESTPOOL/testfs5@snap1 < $backup" + +log_must zfs rollback -r $TESTPOOL/$TESTFS1@snap1 +corrupt_blocks_at_level $file 0 +# test healing when specifying destination filesystem only (no snapshot) +test_corrective_recv $TESTPOOL/$TESTFS1 $backup +# test incremental recv aftear healing recv +log_must eval "zfs recv $TESTPOOL/$TESTFS1 < $ibackup" + +# test that healing recv can not be combined with incompatible recv options +log_mustnot eval "zfs recv -h -c $TESTPOOL/$TESTFS1@snap1 < $backup" +log_mustnot eval "zfs recv -F -c $TESTPOOL/$TESTFS1@snap1 < $backup" +log_mustnot eval "zfs recv -s -c $TESTPOOL/$TESTFS1@snap1 < $backup" +log_mustnot eval "zfs recv -u -c $TESTPOOL/$TESTFS1@snap1 < $backup" +log_mustnot eval "zfs recv -d -c $TESTPOOL/$TESTFS1@snap1 < $backup" +log_mustnot eval "zfs recv -e -c $TESTPOOL/$TESTFS1@snap1 < $backup" + +# ensure healing recv doesn't work when snap GUIDS don't match +log_mustnot eval "zfs recv -c $TESTPOOL/testfs5@snap2 < $backup" +log_mustnot eval "zfs recv -c $TESTPOOL/testfs5 < $backup" + +# test that healing recv doesn't work on non-existing snapshots +log_mustnot eval "zfs recv -c $TESTPOOL/$TESTFS1@missing < $backup" + +log_pass "OpenZFS corrective recv works for data healing" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_from_zstd.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_from_zstd.ksh index 72eebb4f9321..05c2ece4654f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_from_zstd.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_from_zstd.ksh @@ -58,27 +58,22 @@ log_note "Randomly selected ZSTD level: $random_level" log_must zfs create -o compress=zstd-$random_level $TESTPOOL/$TESTFS1 # Make a 5kb compressible file -log_must cat $src_data $src_data $src_data $src_data $src_data \ - > /$TESTPOOL/$TESTFS1/$TESTFILE0 +log_must eval cat $src_data $src_data $src_data $src_data $src_data \ + "> /$TESTPOOL/$TESTFS1/$TESTFILE0" typeset checksum=$(md5digest /$TESTPOOL/$TESTFS1/$TESTFILE0) log_must zfs snapshot $snap # get object number of file -listing=$(ls -i /$TESTPOOL/$TESTFS1/$TESTFILE0) -set -A array $listing -obj=${array[0]} +read -r obj _ < <(ls -i /$TESTPOOL/$TESTFS1/$TESTFILE0) log_note "file /$TESTPOOL/$TESTFS1/$TESTFILE0 has object number $obj" output=$(zdb -Zddddddbbbbbb $TESTPOOL/$TESTFS1 $obj 2> /dev/null \ - |grep -m 1 "L0 DVA" |head -n1) + | grep -m 1 "L0 DVA") dva=$(sed -Ene 's/^.+DVA\[0\]=<([^>]+)>.*$/\1/p' <<< "$output") log_note "block 0 of /$TESTPOOL/$TESTFS1/$TESTFILE0 has a DVA of $dva" -zstd_str=$(sed -Ene 's/^.+ ZSTD:size=([^:]+):version=([^:]+):level=([^:]+):.*$/\1:\2:\3/p' <<< "$output") -zstd_size1=$(echo "$zstd_str" |awk '{split($0,array,":")} END{print array[1]}') -zstd_version1=$(echo "$zstd_str" |awk '{split($0,array,":")} END{print array[2]}') -zstd_level1=$(echo "$zstd_str" |awk '{split($0,array,":")} END{print array[3]}') +read -r zstd_size1 zstd_version1 zstd_level1 < <(sed -Ene 's/^.+ ZSTD:size=([^:]+):version=([^:]+):level=([^:]+):.*$/\1 \2 \3/p' <<< "$output") log_note "ZSTD src: size=$zstd_size1 version=$zstd_version1 level=$zstd_level1" log_note "Verify ZFS can receive the ZSTD compressed stream" @@ -89,23 +84,18 @@ typeset cksum1=$(md5digest /$TESTPOOL/$TESTFS2/$TESTFILE0) log_fail "Checksums differ ($cksum1 != $checksum)" # get object number of file -listing=$(ls -i /$TESTPOOL/$TESTFS2/$TESTFILE0) -set -A array $listing -obj=${array[0]} +read -r obj _ < <(ls -i /$TESTPOOL/$TESTFS2/$TESTFILE0) log_note "file /$TESTPOOL/$TESTFS2/$TESTFILE0 has object number $obj" output=$(zdb -Zddddddbbbbbb $TESTPOOL/$TESTFS2 $obj 2> /dev/null \ - |grep -m 1 "L0 DVA" |head -n1) + | grep -m 1 "L0 DVA") dva=$(sed -Ene 's/^.+DVA\[0\]=<([^>]+)>.*$/\1/p' <<< "$output") log_note "block 0 of /$TESTPOOL/$TESTFS2/$TESTFILE0 has a DVA of $dva" -zstd_str=$(sed -Ene 's/^.+ ZSTD:size=([^:]+):version=([^:]+):level=([^:]+):.*$/\1:\2:\3/p' <<< "$output") -zstd_size2=$(echo "$zstd_str" |awk '{split($0,array,":")} END{print array[1]}') +read -r zstd_size2 zstd_version2 zstd_level2 < <(sed -Ene 's/^.+ ZSTD:size=([^:]+):version=([^:]+):level=([^:]+):.*$/\1 \2 \3/p' <<< "$output") +log_note "ZSTD dest: size=$zstd_size2 version=$zstd_version2 level=$zstd_level2" (( $zstd_size2 != $zstd_size1 )) && log_fail \ "ZFS recv failed: compressed size differs ($zstd_size2 != $zstd_size1)" -zstd_version2=$(echo "$zstd_str" |awk '{split($0,array,":")} END{print array[2]}') -zstd_level2=$(echo "$zstd_str" |awk '{split($0,array,":")} END{print array[3]}') -log_note "ZSTD dest: size=$zstd_size2 version=$zstd_version2 level=$zstd_level2" (( $zstd_level2 != $zstd_level1 )) && log_fail \ "ZFS recv failed: compression level did not match header level ($zstd_level2 != $zstd_level1)" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_new_props.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_new_props.ksh index 54f13355f5e8..ae63f5c6ac1d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_new_props.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_new_props.ksh @@ -70,8 +70,8 @@ log_must zpool set feature@filesystem_limits=enabled "$rpoolname" log_must zfs create -o filesystem_limit=100 "$sendfs" log_must zfs snapshot "$sendfs@a" -log_must zfs send -R "$sendfs@a" >"$streamfile" -log_must eval "zfs recv -svuF $recvfs <$streamfile" +log_must eval "zfs send -R \"$sendfs@a\" >\"$streamfile\"" +log_must eval "zfs recv -svuF \"$recvfs\" <\"$streamfile\"" log_pass "ZFS can handle receiving streams with filesystem limits on \ pools where the feature was recently enabled" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/Makefile.am deleted file mode 100644 index f8273d72c569..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/Makefile.am +++ /dev/null @@ -1,26 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_rename -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_rename_001_pos.ksh \ - zfs_rename_002_pos.ksh \ - zfs_rename_003_pos.ksh \ - zfs_rename_004_neg.ksh \ - zfs_rename_005_neg.ksh \ - zfs_rename_006_pos.ksh \ - zfs_rename_007_pos.ksh \ - zfs_rename_008_pos.ksh \ - zfs_rename_009_neg.ksh \ - zfs_rename_010_neg.ksh \ - zfs_rename_011_pos.ksh \ - zfs_rename_012_neg.ksh \ - zfs_rename_013_pos.ksh \ - zfs_rename_014_neg.ksh \ - zfs_rename_encrypted_child.ksh \ - zfs_rename_to_encrypted.ksh \ - zfs_rename_mountpoint.ksh \ - zfs_rename_nounmount.ksh - -dist_pkgdata_DATA = \ - zfs_rename.cfg \ - zfs_rename.kshlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/cleanup.ksh index 4638b63d72ea..f8f253a3b615 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -33,11 +33,7 @@ default_cleanup_noexit -if [[ -d $TESTDIR2 ]]; then - rm -rf $TESTDIR2 - if (( $? != 0 )); then - log_unresolved Could not remove $TESTDIR2 - fi -fi +rm -rf $TESTDIR2 || + log_unresolved Could not remove $TESTDIR2 log_pass diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/setup.ksh index 60acbe07acbe..0f22179024bc 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -36,13 +36,11 @@ DISK=${DISKS%% *} default_setup_noexit "$DISK" "true" "true" -if [[ -d $TESTDIR2 ]]; then - rm -rf $TESTDIR2 - if (( $? != 0 )); then - log_unresolved Could not remove $TESTDIR2 - fi -fi -log_must zfs create $TESTPOOL/$DATAFS +rm -rf $TESTDIR2 || + log_unresolved Could not remove $TESTDIR2 + +log_must zfs set compression=off $TESTPOOL/$TESTFS +log_must zfs create -o compression=off $TESTPOOL/$DATAFS log_must zfs set mountpoint=$TESTDIR2 $TESTPOOL/$DATAFS log_must eval "dd if=$IF of=$OF bs=$BS count=$CNT >/dev/null 2>&1" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename.cfg b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename.cfg index 9a8f38dc2443..8d3da7cdeaa0 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename.kshlib b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename.kshlib index af1c2f7bedaf..a3d735756fad 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -92,8 +92,7 @@ function cleanup fi if [[ ${dataset[i]}-new != *@* ]] ; then - zfs rename ${dataset[i]}-new ${dataset[i]} - if [[ $? -ne 0 ]]; then + if ! zfs rename ${dataset[i]}-new ${dataset[i]}; then typeset newfs=${dataset[i]}-new typeset oldfs=${dataset[i]} typeset mntp=$(get_prop mountpoint $newfs) @@ -120,8 +119,6 @@ function cmp_data #<$1 src data, $2 tgt data> typeset src=$1 typeset tgt=$2 - cmp $src $tgt >/dev/null 2>&1 - - return $? + cmp $src $tgt >/dev/null } diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_001_pos.ksh index c76b5a2fe5c3..a4fb95c8d10e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_002_pos.ksh index 6d51f612b311..05f1474b1854 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_003_pos.ksh index 0bd4aca3a7fd..26ee9cb923df 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_004_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_004_neg.ksh index b1438e8663e3..224fceaddd08 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_004_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_004_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_005_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_005_neg.ksh index 5ef77de0c751..3717d1d9dad2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_005_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_005_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_006_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_006_pos.ksh index 4d1605152201..5d67a4ed9547 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_007_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_007_pos.ksh index 2a3f8a8ccd7b..5b8d7c78dfc1 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_008_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_008_pos.ksh index 229163839058..6f5974cc235e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_009_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_009_neg.ksh index 7e8119766bf7..7eef546380b2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_009_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_009_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_010_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_010_neg.ksh index 171770fa32a1..b631ea886823 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_010_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_010_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_011_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_011_pos.ksh index 71d72619d646..43632c7d34ba 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_011_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_011_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_012_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_012_neg.ksh index 6daf001797e6..16c93e9f4e14 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_012_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_012_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_013_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_013_pos.ksh index 73790f58cd45..f33a971729ed 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_013_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_013_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_014_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_014_neg.ksh index 1c962608d784..d0b6984d21cb 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_014_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_014_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -81,7 +81,7 @@ function nesting_cleanup # before resetting it, it will be left at the modified # value for the remaining tests. That's the reason # we reset it again here just in case. - log_must set_tunable_impl MAX_DATASET_NESTING 50 Z zcommon + log_must set_tunable64 MAX_DATASET_NESTING 50 Z } log_onexit nesting_cleanup @@ -93,13 +93,13 @@ log_must zfs create -p $TESTPOOL/$dsC16 log_mustnot zfs rename $TESTPOOL/$dsA02 $TESTPOOL/$dsB15A # extend limit -log_must set_tunable_impl MAX_DATASET_NESTING 64 Z zcommon +log_must set_tunable64 MAX_DATASET_NESTING 64 Z log_mustnot zfs rename $TESTPOOL/$dsA02 $TESTPOOL/$dsB16A log_must zfs rename $TESTPOOL/$dsA02 $TESTPOOL/$dsB15A # bring back old limit -log_must set_tunable_impl MAX_DATASET_NESTING 50 Z zcommon +log_must set_tunable64 MAX_DATASET_NESTING 50 Z log_mustnot zfs rename $TESTPOOL/$dsC01 $TESTPOOL/$dsB15A47C log_must zfs rename $TESTPOOL/$dsB15A47A $TESTPOOL/$dsB15A47B diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_nounmount.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_nounmount.ksh index 1c707762a72c..96826d814f18 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_nounmount.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rename/zfs_rename_nounmount.ksh @@ -39,7 +39,7 @@ function rename_cleanup zfs destroy -fR $TESTPOOL/renamed } -back=$(pwd) +back=$PWD log_onexit rename_cleanup log_must zfs create $TESTPOOL/rename_test @@ -72,14 +72,15 @@ log_must zfs list $TESTPOOL/renamed log_must zfs list $TESTPOOL/renamed/child log_must zfs list $TESTPOOL/renamed/child/grandchild -missing=$(zfs mount | awk -v pat=$TESTPOOL/renamed '$1 ~ pat' | awk \ +missing=$(zfs mount | awk \ + -v genpat=$TESTPOOL/renamed \ -v mntp_p=$mntp_p \ -v mntp_c=$mntp_c \ -v mntp_g=$mntp_g ' BEGIN { p = c = g = 0 } - $2 == mntp_p { p = 1 } - $2 == mntp_c { c = 1 } - $2 == mntp_g { g = 1 } + $1 ~ genpat && $2 == mntp_p { p = 1 } + $1 ~ genpat && $2 == mntp_c { c = 1 } + $1 ~ genpat && $2 == mntp_g { g = 1 } END { if (p != 1) print mntp_p diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/Makefile.am deleted file mode 100644 index 5b7b758a056a..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_reservation -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_reservation_001_pos.ksh \ - zfs_reservation_002_pos.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/setup.ksh index 6a9af3bc28c3..4c719075a741 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/zfs_reservation_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/zfs_reservation_001_pos.ksh index 7ec55ce0f3d2..416139f7aa3b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/zfs_reservation_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/zfs_reservation_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/zfs_reservation_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/zfs_reservation_002_pos.ksh index c482f9c3c7df..49eabfeb7223 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/zfs_reservation_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_reservation/zfs_reservation_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -73,16 +73,8 @@ log_must zfs set reservation=none $TESTPOOL/$RESERVATION2 for FS in $TESTPOOL/$RESERVATION $TESTPOOL/$RESERVATION2 do - - reserve=`zfs get -pH reservation $FS | awk '{print $3}'` - if [[ $reserve -ne 0 ]]; then - log_fail "ZFS get -p reservation did not return 0" - fi - - reserve=`zfs get -H reservation $FS | awk '{print $3}'` - if [[ $reserve != "none" ]]; then - log_fail "ZFS get reservation did not return 'none'" - fi + log_must [ $(zfs get -pHo value reservation $FS) -eq 0 ] + log_must [ $(zfs get -Ho value reservation $FS) = none ] done log_pass "Successfully set reservation to 0 and 'none'" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/Makefile.am deleted file mode 100644 index 4d278f724527..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/Makefile.am +++ /dev/null @@ -1,12 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_rollback -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_rollback_001_pos.ksh \ - zfs_rollback_002_pos.ksh \ - zfs_rollback_003_neg.ksh \ - zfs_rollback_004_neg.ksh - -dist_pkgdata_DATA = \ - zfs_rollback.cfg \ - zfs_rollback_common.kshlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/cleanup.ksh index 707e8f121ddb..93abeb1338ce 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/setup.ksh index d3341b1c27fc..562e5242d3c8 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback.cfg b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback.cfg index d6c8934d9753..215f3c350785 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_001_pos.ksh index 607bbf06eafd..5290f06205b2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_002_pos.ksh index 659660dc0fe7..3b03063d0c1c 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_003_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_003_neg.ksh index 1e3109108bae..e798a7bec5c2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_003_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_003_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_004_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_004_neg.ksh index 9537d5077b1a..be59742c9051 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_004_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_004_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_common.kshlib b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_common.kshlib index 2eadb68c372d..29eb82a22cd6 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_common.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_rollback/zfs_rollback_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -36,9 +36,9 @@ # $1 full file name function getsum #fname { - (( ${#1} == 0 )) && \ - log_fail "Need give file name." - return $(sum $1 | awk '{print $1}') + typeset sum + read -r sum _ < <(cksum "$1") + echo $sum } # Define global variable checksum, get the original file sum. @@ -270,11 +270,7 @@ function check_files typeset sum0=$(getsum $file0) typeset sum1=$(getsum $file1) typeset sum2=$(getsum $file2) - if [[ $sum0 != $origsum || \ - $sum1 != $origsum || sum2 != $origsum ]] - then - log_fail "After rollback, file sum is changed." - fi + log_must [ $sum0 = $origsum \&\& $sum1 = $origsum \&\& sum2 = $origsum ] ;; $TESTSNAP1) log_must files_exist $file0 $file1 @@ -282,19 +278,14 @@ function check_files typeset sum0=$(getsum $file0) typeset sum1=$(getsum $file1) - if [[ $sum0 != $origsum || $sum1 != $origsum ]] - then - log_fail "After rollback, file sum is changed." - fi + log_must [ $sum0 = $origsum \&\& $sum1 = $origsum ] ;; $TESTSNAP) log_must files_exist $file0 log_must files_nonexist $file1 $file2 typeset sum0=$(getsum $file0) - if [[ $sum0 != $origsum ]]; then - log_fail "After rollback, file sum is changed." - fi + log_must [ $sum0 = $origsum ] ;; esac } diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_send/Makefile.am deleted file mode 100644 index 25c7065670f2..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/Makefile.am +++ /dev/null @@ -1,20 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_send -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_send_001_pos.ksh \ - zfs_send_002_pos.ksh \ - zfs_send_003_pos.ksh \ - zfs_send_004_neg.ksh \ - zfs_send_005_pos.ksh \ - zfs_send_006_pos.ksh \ - zfs_send_007_pos.ksh \ - zfs_send_encrypted.ksh \ - zfs_send_encrypted_unloaded.ksh \ - zfs_send_raw.ksh \ - zfs_send_sparse.ksh \ - zfs_send-b.ksh \ - zfs_send_skip_missing.ksh - -dist_pkgdata_DATA = \ - zfs_send.cfg diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_send/cleanup.ksh index cec69d4709f4..e3a213302614 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_send/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_send/setup.ksh index 6a9af3bc28c3..4c719075a741 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_send/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send-b.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send-b.ksh index f019c2215ecd..7e841c10ceac 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send-b.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send-b.ksh @@ -58,13 +58,13 @@ log_must zfs set "org.openzfs:snapprop=val" "$SENDFS@s1" # 2. Verify command line options interact with '-b' correctly typeset opts=("" "p" "Rp" "cew" "nv" "D" "DLPRcenpvw") for opt in ${opts[@]}; do - log_must eval "zfs send -b$opt $SENDFS@s1 >$TEST_BASE_DIR/devnull" - log_must eval "zfs send -b$opt -i $SENDFS@s1 $SENDFS@s2 >$TEST_BASE_DIR/devnull" - log_must eval "zfs send -b$opt -I $SENDFS@s1 $SENDFS@s2 >$TEST_BASE_DIR/devnull" + log_must eval "zfs send -b$opt $SENDFS@s1 > /dev/null" + log_must eval "zfs send -b$opt -i $SENDFS@s1 $SENDFS@s2 > /dev/null" + log_must eval "zfs send -b$opt -I $SENDFS@s1 $SENDFS@s2 > /dev/null" done for opt in ${opts[@]}; do - log_mustnot eval "zfs send -b$opt $SENDFS >$TEST_BASE_DIR/devnull" - log_mustnot eval "zfs send -b$opt $SENDFS#bm >$TEST_BASE_DIR/devnull" + log_mustnot eval "zfs send -b$opt $SENDFS > /dev/null" + log_mustnot eval "zfs send -b$opt $SENDFS#bm > /dev/null" done # Do 3..6 in a loop to verify various combination of "zfs send" options diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send.cfg b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send.cfg index bbc4d3ac833e..49f7245f9c58 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_001_pos.ksh index b18433085ed5..a8b22756a6d5 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -96,9 +96,7 @@ log_must zfs set mountpoint=$TESTDIR1 $rst_root file_write -o create -f $init_data -b $BLOCK_SIZE -c $WRITE_COUNT log_must zfs snapshot $init_snap -zfs send $init_snap > $full_bkup -(( $? != 0 )) && \ - log_fail "'zfs send' fails to create full send" +log_must eval "zfs send $init_snap > $full_bkup" log_note "Verify the send stream is valid to receive." @@ -111,9 +109,7 @@ log_note "Verify 'zfs send -i' can create incremental send stream." file_write -o create -f $inc_data -b $BLOCK_SIZE -c $WRITE_COUNT -d 0 log_must zfs snapshot $inc_snap -zfs send -i $init_snap $inc_snap > $inc_bkup -(( $? != 0 )) && \ - log_fail "'zfs send -i' fails to create incremental send" +log_must eval "zfs send -i $init_snap $inc_snap > $inc_bkup" log_note "Verify the incremental send stream is valid to receive." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_002_pos.ksh index 42bdddd2cc18..f575c0cc8da7 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -66,12 +66,8 @@ function do_testing # log_must zfs set $property=$prop_val $fs file_write -o create -f $origfile -b $BLOCK_SIZE -c $WRITE_COUNT log_must zfs snapshot $snap - zfs send $snap > $stream - (( $? != 0 )) && \ - log_fail "'zfs send' fails to create send streams." - zfs receive -d $ctr <$stream - (( $? != 0 )) && \ - log_fail "'zfs receive' fails to receive send streams." + log_must eval "zfs send $snap > $stream" + log_must eval "zfs receive -d $ctr <$stream" #verify receive result ! datasetexists $rstfs && \ diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_003_pos.ksh index caa84886fa12..211225dd6fbe 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -61,7 +61,7 @@ log_must zfs snapshot $snap2 typeset -i i=0 while (( i < ${#args[*]} )); do - log_must eval "zfs send -i ${args[i]} >$TEST_BASE_DIR/devnull" + log_must eval "zfs send -i ${args[i]} > /dev/null" (( i += 1 )) done diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_004_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_004_neg.ksh index af10e3a11faf..ea3a759ddadc 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_004_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_004_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -95,7 +95,7 @@ log_must zfs snapshot $snap3 typeset -i i=0 while (( i < ${#badargs[*]} )) do - log_mustnot eval "zfs send ${badargs[i]} >$TEST_BASE_DIR/devnull" + log_mustnot eval "zfs send ${badargs[i]} > /dev/null" (( i = i + 1 )) done diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_005_pos.ksh index c4ab7a6212bc..174b21feb23e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -61,6 +61,6 @@ log_must zfs snapshot -r $TESTPOOL@snap log_must zpool export $TESTPOOL log_must zpool import -o readonly=on $TESTPOOL -log_must eval "zfs send -R $TESTPOOL@snap >$TEST_BASE_DIR/devnull" +log_must eval "zfs send -R $TESTPOOL@snap > /dev/null" log_pass "'zfs send -R' can send from read-only pools" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_006_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_006_pos.ksh index 42628a0512e9..c5dfb89394f4 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_006_pos.ksh @@ -76,10 +76,10 @@ function verify_size_estimates typeset file_size=$2 typeset refer_diff=$(echo "$refer_size - $estimate_size" | bc) refer_diff=$(echo "$refer_diff / 1" | bc) - refer_diff=$(echo "$refer_diff" | nawk '{print ($1 < 0) ? ($1 * -1): $1'}) + refer_diff=$(echo "$refer_diff" | awk '{print ($1 < 0) ? ($1 * -1): $1'}) typeset file_diff=$(echo "$file_size - $estimate_size" | bc) file_diff=$(echo "$file_diff / 1" | bc) - file_diff=$(echo "$file_diff" | nawk '{print ($1 < 0) ? ($1 * -1):$1'}) + file_diff=$(echo "$file_diff" | awk '{print ($1 < 0) ? ($1 * -1):$1'}) typeset expected_diff=$(cal_percentage $refer_size) [[ -z $refer_diff && -z $file_diff && -z $expected_diff ]] && \ diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_007_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_007_pos.ksh index 675afa72f5af..15760398127c 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_007_pos.ksh @@ -52,12 +52,12 @@ streamfile=$(mktemp $TESTDIR/file.XXXXXX) vdev=$(mktemp $TEST_BASE_DIR/file.XXXXXX) -test_pool () +function test_pool { POOL=$1 log_must zfs create -o recordsize=512 $POOL/fs mntpnt=$(get_prop mountpoint "$POOL/fs") - log_must dd if=/dev/urandom of=${mntpnt}/file bs=512 count=1 2>/dev/null + log_must eval "dd if=/dev/urandom of=${mntpnt}/file bs=512 count=1 2>/dev/null" object=$(ls -i $mntpnt | awk '{print $1}') log_must zfs snapshot $POOL/fs@a while true; do @@ -67,10 +67,7 @@ test_pool () sync_all_pools # check if we started reusing objects object=$(ls -i $mntpnt | sort -n | awk -v object=$object \ - '{if ($1 <= object) {exit 1}} END {print $1}') - if [[ $? -ne 0 ]]; then - break - fi + '{if ($1 <= object) {exit 1}} END {print $1}') || break done dd if=/dev/urandom of=${mntpnt}/$FILE bs=512 count=1 seek=1 2>/dev/null @@ -83,7 +80,7 @@ test_pool () cat $streamfile | log_must zfs receive $POOL/recvfs recv_mntpnt=$(get_prop mountpoint "$POOL/recvfs") - log_must diff -r $mntpnt $recv_mntpnt + log_must directory_diff $mntpnt $recv_mntpnt log_must zfs destroy -rf $POOL/fs log_must zfs destroy -rf $POOL/recvfs } diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_encrypted.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_encrypted.ksh index a4c332d47dfc..b0f10028f472 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_encrypted.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_encrypted.ksh @@ -62,15 +62,15 @@ log_must eval "echo $passphrase1 | zfs create -o encryption=on" \ log_must zfs snapshot -r $snap -log_must eval "zfs send $snap >$TEST_BASE_DIR/devnull" -log_mustnot eval "zfs send -p $snap >$TEST_BASE_DIR/devnull" -log_mustnot eval "zfs send -R $snap >$TEST_BASE_DIR/devnull" +log_must eval "zfs send $snap > /dev/null" +log_mustnot eval "zfs send -p $snap > /dev/null" +log_mustnot eval "zfs send -R $snap > /dev/null" log_must zfs unmount $TESTPOOL/$TESTFS1 log_must zfs unload-key $TESTPOOL/$TESTFS1 -log_mustnot eval "zfs send $snap >$TEST_BASE_DIR/devnull" -log_must eval "zfs send $TESTPOOL/$TESTFS1/child@snap >$TEST_BASE_DIR/devnull" +log_mustnot eval "zfs send $snap > /dev/null" +log_must eval "zfs send $TESTPOOL/$TESTFS1/child@snap > /dev/null" log_pass "ZFS performs unencrypted sends of encrypted datasets, unless the" \ "'-p' or '-R' options are specified" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_encrypted_unloaded.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_encrypted_unloaded.ksh index f268f7b38d4d..bf9a2be8db53 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_encrypted_unloaded.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_encrypted_unloaded.ksh @@ -53,7 +53,7 @@ log_must eval "echo $passphrase | zfs create -o encryption=on" \ log_must zfs snapshot $snap log_must zfs unmount $TESTPOOL/$TESTFS1 log_must zfs unload-key $TESTPOOL/$TESTFS1 -log_mustnot eval "zfs send $snap >$TEST_BASE_DIR/devnull" +log_mustnot eval "zfs send $snap > /dev/null" log_pass "ZFS does not perform unencrypted sends from encrypted datasets" \ "with unloaded keys." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_raw.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_raw.ksh index 03c2e78673d8..b221d8280df2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_raw.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_raw.ksh @@ -57,21 +57,21 @@ log_must eval "echo $passphrase | zfs create -o encryption=on" \ log_must zfs snapshot $snap log_must zfs snapshot $snap1 -log_must eval "zfs send -w $snap >$TEST_BASE_DIR/devnull" -log_must eval "zfs send -w $snap1 >$TEST_BASE_DIR/devnull" +log_must eval "zfs send -w $snap > /dev/null" +log_must eval "zfs send -w $snap1 > /dev/null" log_note "Verify ZFS can perform raw sends with properties" -log_must eval "zfs send -wp $snap >$TEST_BASE_DIR/devnull" -log_must eval "zfs send -wp $snap1 >$TEST_BASE_DIR/devnull" +log_must eval "zfs send -wp $snap > /dev/null" +log_must eval "zfs send -wp $snap1 > /dev/null" log_note "Verify ZFS can perform raw replication sends" -log_must eval "zfs send -wR $snap >$TEST_BASE_DIR/devnull" -log_must eval "zfs send -wR $snap1 >$TEST_BASE_DIR/devnull" +log_must eval "zfs send -wR $snap > /dev/null" +log_must eval "zfs send -wR $snap1 > /dev/null" log_note "Verify ZFS can perform a raw send of an encrypted datasets with" \ "its key unloaded" log_must zfs unmount $TESTPOOL/$TESTFS1 log_must zfs unload-key $TESTPOOL/$TESTFS1 -log_must eval "zfs send -w $snap1 >$TEST_BASE_DIR/devnull" +log_must eval "zfs send -w $snap1 > /dev/null" log_pass "ZFS performs raw sends of datasets" diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_skip_missing.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_skip_missing.ksh index 2e12d2534412..06a53fcddd5e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_skip_missing.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_skip_missing.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -49,7 +49,6 @@ function cleanup datasetexists $PARENT && destroy_dataset $PARENT -rf [[ -e $WARNF ]] && log_must rm -f $WARNF - rm -f $TEST_BASE_DIR/devnull } log_assert "Verify 'zfs send -Rs' works as expected." @@ -66,12 +65,12 @@ log_note "Verify 'zfs send -R' fails to generate replication stream"\ log_must zfs create $PARENT log_must zfs create $CHILD log_must zfs snapshot $SNAP -log_mustnot eval "zfs send -R $SNAP >$TEST_BASE_DIR/devnull" +log_mustnot eval "zfs send -R $SNAP > /dev/null" log_note "Verify 'zfs send -Rs' warns about missing snapshots, "\ "but still succeeds" -log_must eval "zfs send -Rs $SNAP 2> $WARNF >$TEST_BASE_DIR/devnull" +log_must eval "zfs send -Rs $SNAP 2> $WARNF > /dev/null" log_must eval "[[ -s $WARNF ]]" log_pass "Verify 'zfs send -Rs' works as expected." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_set/Makefile.am deleted file mode 100644 index f7362ff2556f..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/Makefile.am +++ /dev/null @@ -1,35 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_set -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - cache_001_pos.ksh \ - cache_002_neg.ksh \ - canmount_001_pos.ksh \ - canmount_002_pos.ksh \ - canmount_003_pos.ksh \ - canmount_004_pos.ksh \ - checksum_001_pos.ksh \ - compression_001_pos.ksh \ - mountpoint_001_pos.ksh \ - mountpoint_002_pos.ksh \ - mountpoint_003_pos.ksh \ - onoffs_001_pos.ksh \ - property_alias_001_pos.ksh \ - readonly_001_pos.ksh \ - reservation_001_neg.ksh \ - ro_props_001_pos.ksh \ - share_mount_001_neg.ksh \ - snapdir_001_pos.ksh \ - user_property_001_pos.ksh \ - user_property_002_pos.ksh \ - user_property_003_neg.ksh \ - user_property_004_pos.ksh \ - version_001_neg.ksh \ - zfs_set_001_neg.ksh \ - zfs_set_002_neg.ksh \ - zfs_set_003_neg.ksh \ - zfs_set_keylocation.ksh \ - zfs_set_feature_activation.ksh - -dist_pkgdata_DATA = \ - zfs_set_common.kshlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/cache_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/cache_001_pos.ksh index bffc47f817dd..de073bf1132f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/cache_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/cache_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/cache_002_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/cache_002_neg.ksh index caad211bcf65..b4f24f5adf19 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/cache_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/cache_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_001_pos.ksh index ac5fc8188f5a..4eb11423d9e4 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -81,14 +81,8 @@ function cleanup log_assert "Setting a valid property of canmount to file system, it must be successful." log_onexit cleanup -typeset old_fs_canmount="" old_ctr_canmount="" - -old_fs_canmount=$(get_prop canmount $TESTPOOL/$TESTFS) -[[ $? != 0 ]] && \ - log_fail "Get the $TESTPOOL/$TESTFS canmount error." -old_ctr_canmount=$(get_prop canmount $TESTPOOL/$TESTCTR) -[[ $? != 0 ]] && \ - log_fail "Get the $TESTPOOL/$TESTCTR canmount error." +typeset old_fs_canmount=$(get_prop canmount $TESTPOOL/$TESTFS) +typeset old_ctr_canmount=$(get_prop canmount $TESTPOOL/$TESTCTR) log_must zfs snapshot $TESTPOOL/$TESTFS@$TESTSNAP log_must zfs snapshot $TESTPOOL/$TESTVOL@$TESTSNAP diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_002_pos.ksh index 55c71f6ca33a..d628d6fc6ab1 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -77,9 +77,7 @@ function cleanup if datasetexists $ds; then mntp=$(get_prop mountpoint $ds) destroy_dataset $ds - if [[ -d $mntp ]]; then - rm -fr $mntp - fi + [ -d $mntp ] && rm -fr $mntp fi snapexists $TESTPOOL/$TESTFS@$TESTSNAP && \ @@ -91,9 +89,7 @@ function cleanup zfs unmount -a > /dev/null 2>&1 log_must zfs mount -a - if [[ -d $tmpmnt ]]; then - rm -fr $tmpmnt - fi + [ -d $tmpmnt ] && rm -fr $tmpmnt } log_assert "Setting canmount=noauto to file system, it must be successful." @@ -125,7 +121,7 @@ while (( i < ${#dataset_pos[*]} )) ; do set_n_check_prop "noauto" "canmount" "$dataset" log_must zfs set mountpoint=$tmpmnt $dataset log_must zfs set sharenfs=on $dataset - if ismounted $dataset; then + if ismounted $dataset; then zfs unmount -a > /dev/null 2>&1 log_must mounted $dataset log_must zfs unmount $dataset diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_003_pos.ksh index e4664d03b43c..fa9948445d9f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -97,9 +97,9 @@ while (( i < ${#dataset_pos[*]} )); do done i=0 -while (( i < ${#dataset_pos[*]} )) ; do +while (( i < ${#dataset_pos[*]} )); do dataset=${dataset_pos[i]} - if ismounted $dataset; then + if ismounted $dataset; then log_must cd ${old_mnt[i]} set_n_check_prop "noauto" "canmount" "$dataset" log_must mounted $dataset diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_004_pos.ksh index e75114efdec0..b3f1afc63ab2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/canmount_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/checksum_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/checksum_001_pos.ksh index 27003b21b556..91b08a838bd1 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/checksum_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/checksum_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -46,7 +46,7 @@ verify_runnable "both" set -A dataset "$TESTPOOL" "$TESTPOOL/$TESTFS" "$TESTPOOL/$TESTVOL" -set -A values "on" "off" "fletcher2" "fletcher4" "sha256" "sha512" "skein" "edonr" "noparity" +set -A values "on" "off" "fletcher2" "fletcher4" "sha256" "sha512" "skein" "edonr" "blake3" "noparity" log_assert "Setting a valid checksum on a file system, volume," \ "it should be successful." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/cleanup.ksh index fad025cac246..b6c8bb2f47e9 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/compression_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/compression_001_pos.ksh index 06da5f2f3ce8..ebeed4a7dab7 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/compression_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/compression_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/mountpoint_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/mountpoint_001_pos.ksh index 1255ae5f5666..c1f5affa3414 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/mountpoint_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/mountpoint_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -69,11 +69,7 @@ log_assert "Setting a valid mountpoint to file system, it must be successful." log_onexit cleanup old_fs_mpt=$(get_prop mountpoint $TESTPOOL/$TESTFS) -[[ $? != 0 ]] && \ - log_fail "Get the $TESTPOOL/$TESTFS mountpoint error." old_ctr_mpt=$(get_prop mountpoint $TESTPOOL/$TESTCTR) -[[ $? != 0 ]] && \ - log_fail "Get the $TESTPOOL/$TESTCTR mountpoint error." if [[ ! -d $TESTDIR2 ]]; then log_must mkdir $TESTDIR2 diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/mountpoint_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/mountpoint_002_pos.ksh index 48580cafdb31..a5785226e02e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/mountpoint_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/mountpoint_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -66,11 +66,7 @@ log_assert "Setting a valid mountpoint for an unmounted file system, \ log_onexit cleanup old_fs_mpt=$(get_prop mountpoint $TESTPOOL/$TESTFS) -[[ $? != 0 ]] && \ - log_fail "Unable to get the mountpoint property for $TESTPOOL/$TESTFS" old_ctr_mpt=$(get_prop mountpoint $TESTPOOL/$TESTCTR) -[[ $? != 0 ]] && \ - log_fail "Unable to get the mountpoint property for $TESTPOOL/$TESTCTR" if [[ ! -d $TESTDIR2 ]]; then log_must mkdir $TESTDIR2 diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/mountpoint_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/mountpoint_003_pos.ksh index 4d86100c03ef..01b505e4ab1a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/mountpoint_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/mountpoint_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -109,10 +109,8 @@ while ((i < ${#args[@]})); do msg=$(mount | grep "$tmpmnt ") - echo $msg | grep "${args[((i))]}" > /dev/null 2>&1 - if (($? != 0)) ; then - echo $msg | grep "${args[((i-1))]}" > /dev/null 2>&1 - if (($? == 0)) ; then + if ! echo $msg | grep -q "${args[((i))]}"; then + if echo $msg | grep -q "${args[((i-1))]}"; then log_fail "Expected option: ${args[((i))]} \n" \ "Real option: $msg" fi @@ -130,8 +128,7 @@ while ((i < ${#args[@]})); do args[((i+1))]="/nodevices/" fi - echo $msg | grep "${args[((i+1))]}" > /dev/null 2>&1 - if (($? != 0)) ; then + if ! echo $msg | grep -q "${args[((i+1))]}"; then log_fail "Expected option: ${args[((i+1))]} \n" \ "Real option: $msg" fi diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/onoffs_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/onoffs_001_pos.ksh index 7ba6d7fb659d..d6daf0e1b74e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/onoffs_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/onoffs_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/property_alias_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/property_alias_001_pos.ksh index f1befe60c373..b75143ed61bd 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/property_alias_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/property_alias_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -91,9 +91,7 @@ typeset -i i=0 for ds in $pool $fs $vol; do for propname in ${ro_prop[*]}; do - zfs get -pH -o value $propname $ds >/dev/null 2>&1 - (( $? != 0 )) && \ - log_fail "Get the property $proname of $ds failed." + log_must eval "zfs get -pH -o value $propname $ds >/dev/null 2>&1" done i=0 while (( i < ${#rw_prop[*]} )); do @@ -120,9 +118,7 @@ for ds in $pool $fs $vol; do done if [[ $ds == $vol ]]; then for propname in "volblocksize" "volblock" ; do - zfs get -pH -o value $propname $ds >/dev/null 2>&1 - (( $? != 0 )) && \ - log_fail "Get the property $propname of $ds failed." + log_must eval "zfs get -pH -o value $propname $ds >/dev/null 2>&1" done fi done diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/readonly_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/readonly_001_pos.ksh index 4adac420f748..20d6b634429a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/readonly_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/readonly_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/reservation_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/reservation_001_neg.ksh index de6d83d2270c..9430c1ac4070 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/reservation_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/reservation_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -70,15 +70,7 @@ function set_n_check # data-set j=0 while (( $j < ${#suffix[*]} )); do - zfs set \ - reservation=${values[$i]}${suffix[$j]} $obj \ - > /dev/null 2>&1 - if [ $? -eq 0 ] - then - log_note "zfs set \ - reservation=${values[$i]}${suffix[$j]} $obj" - log_fail "The above reservation set returned 0!" - fi + log_mustnot zfs set reservation=${values[$i]}${suffix[$j]} $obj new_resv_val=$(get_prop reservation $obj) diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/ro_props_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/ro_props_001_pos.ksh index 7177fac202d0..d3d041f3395d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/ro_props_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/ro_props_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -55,18 +55,13 @@ typeset ro_props="type used creation referenced refer compressratio \ mounted origin" typeset snap_ro_props="volsize recordsize recsize quota reservation reserv mountpoint \ sharenfs checksum compression compress atime devices exec readonly rdonly \ - setuid" + setuid version" if is_freebsd; then snap_ro_props+=" jailed" else snap_ro_props+=" zoned" fi -zfs upgrade -v > /dev/null 2>&1 -if [[ $? -eq 0 ]]; then - snap_ro_props="$snap_ro_props version" -fi - function cleanup { datasetexists $TESTPOOL/$TESTVOL@$TESTSNAP && \ diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/setup.ksh index 9692385996d1..2a909eba4f87 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/share_mount_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/share_mount_001_neg.ksh index fa32c2774f8b..3097d931f683 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/share_mount_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/share_mount_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/snapdir_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/snapdir_001_pos.ksh index 083a6b1f464f..a41fa6c251b3 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/snapdir_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/snapdir_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -57,22 +57,12 @@ function verify_snapdir_visible # $1 dataset, $2 hidden|visible typeset dataset=$1 typeset value=$2 typeset mtpt=$(get_prop mountpoint $dataset) - typeset name - - for name in `ls -a $mtpt`; do - if [[ $name == ".zfs" ]]; then - if [[ $value == "visible" ]]; then - return 0 - else - return 1 - fi - fi - done - if [[ $value == "visible" ]]; then - return 1 + # $mtpt/.zfs always actually exists so [ -d $mtpt/.zfs ] is always true + if ls -a $mtpt | grep -xFq .zfs; then + [ $value = "visible" ] else - return 0 + [ $value != "visible" ] fi } @@ -95,15 +85,14 @@ log_assert "Setting a valid snapdir property on a dataset succeeds." for dataset in $all_datasets; do for value in hidden visible; do - if [[ $dataset == "$TESTPOOL/$TESTVOL" ]] ; then + if [ "$dataset" = "$TESTPOOL/$TESTVOL" ]; then set_n_check_prop "$value" "snapdir" \ "$dataset" "false" else set_n_check_prop "$value" "snapdir" \ "$dataset" - verify_snapdir_visible $dataset $value - [[ $? -eq 0 ]] || \ - log_fail "$dataset/.zfs is not $value as expect." + verify_snapdir_visible $dataset $value || + log_fail "$dataset/.zfs is not $value as expected." fi done done diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_001_pos.ksh index 16b9638787e2..b897dffcbd27 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_002_pos.ksh index 12cff78ea4ca..868abc82f394 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_003_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_003_neg.ksh index 3d8c1e7be18d..80f279cbe992 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_003_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_003_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_004_pos.ksh index bd11ea088333..bcd367773722 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/user_property_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -68,32 +68,19 @@ function nonexist_user_prop log_assert "User property has no effect to snapshot until 'Snapshot properties' supported." log_onexit cleanup -typeset snap_property= - -zpool upgrade -v | grep "Snapshot properties" > /dev/null 2>&1 -if (( $? == 0 )) ; then - snap_property="true" -fi - for fs in $TESTPOOL/$TESTFS $TESTPOOL/$TESTVOL $TESTPOOL ; do typeset fssnap=$fs@snap prop_name=$(valid_user_property 10) value=$(user_property_value 16) - log_must eval "zfs set $prop_name='$value' $fs" - log_must eval "check_user_prop $fs $prop_name '$value'" + log_must zfs set $prop_name="$value" $fs + log_must check_user_prop $fs $prop_name "$value" log_must zfs snapshot $fssnap - if [[ -n $snap_property ]] ; then - log_mustnot nonexist_user_prop $prop_name $fssnap + log_mustnot nonexist_user_prop $prop_name $fssnap - log_must eval "zfs set $prop_name='$value' $fssnap" - log_mustnot nonexist_user_prop $prop_name $fssnap - else - log_must nonexist_user_prop $prop_name $fssnap - log_mustnot eval "zfs set $prop_name='$value' $fssnap" - log_must nonexist_user_prop $prop_name $fssnap - fi + log_must zfs set $prop_name="$value" $fssnap + log_mustnot nonexist_user_prop $prop_name $fssnap done log_pass "User properties has effect upon snapshot." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/version_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/version_001_neg.ksh index cf5ef5116e90..0443b63376f1 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/version_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/version_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -66,11 +66,7 @@ function set_n_check # data-set orig_val=$(get_prop version $obj) while (($i < ${#values[*]})); do - zfs set version=${values[$i]} $obj > /dev/null 2>&1 - if [[ $? -eq 0 ]]; then - log_note "zfs set version=${values[$i]} $obj" - log_fail "The above version set returned 0!" - fi + log_mustnot eval "zfs set version=${values[$i]} $obj > /dev/null 2>&1" new_val=$(get_prop version $obj) diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_001_neg.ksh index c9bc7565abd6..0701c9ef0f28 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -77,9 +77,9 @@ while (( i < ${#dataset[@]} )); do done (( j += 1 )) done - # Additional recordsize - set_n_check_prop "recordsize" "2048K" "${dataset[i]}" false - set_n_check_prop "recordsize" "128B" "${dataset[i]}" false + # Additional recordsize + set_n_check_prop "32768K" "recordsize" "${dataset[i]}" false + set_n_check_prop "128B" "recordsize" "${dataset[i]}" false (( i += 1 )) done diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_002_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_002_neg.ksh index 2178175cd5b6..946f6983b161 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_003_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_003_neg.ksh index fd5f7f285f5b..3afb0eb7010e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_003_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_003_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_common.kshlib b/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_common.kshlib index 12082076322c..484c7d862276 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_common.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -73,17 +73,17 @@ function set_n_check_prop reservation|reserv|quota ) if [[ $expect_value == "none" ]]; then [[ $cur_value != "0" ]] && \ - log_fail "The '$dataset' '$prop' value \ - '$cur_value' is not expected." + log_fail "The '$dataset' '$prop' value" \ + "'$cur_value' is not expected." elif [[ $cur_value != $expect_value ]]; then - log_fail "The '$dataset' '$prop' value '$cur_value' \ - does not equal the expected value '$expect_value'." + log_fail "The '$dataset' '$prop' value '$cur_value'" \ + "does not equal the expected value '$expect_value'." fi ;; * ) if [[ $cur_value != $expect_value ]]; then - log_fail "The '$dataset' '$prop' value '$cur_value' \ - does not equal the expected value '$expect_value'." + log_fail "The '$dataset' '$prop' value '$cur_value'" \ + "does not equal the expected value '$expect_value'." fi ;; esac @@ -97,8 +97,8 @@ function set_n_check_prop if [[ "$expect_value" != "" && "$cur_value" != "$old_value" ]]; then - log_fail "The '$dataset' '$prop' value '$cur_value' \ - should equal with '$old_value'." + log_fail "The '$dataset' '$prop' value '$cur_value'" \ + "should equal '$old_value'." fi fi } @@ -119,8 +119,8 @@ function cleanup_user_prop typeset prop for prop in $user_prop; do - zfs inherit $prop $dt - (($? != 0)) && log_must zfs inherit $prop $dt + zfs inherit $prop $dt || + log_must zfs inherit $prop $dt done done } @@ -225,9 +225,7 @@ function user_property_value { typeset -i len=${1:-100} - typeset value=$(random_string ALL_CHAR $len) - - echo "$value" + random_string ALL_CHAR $len } # @@ -244,11 +242,7 @@ function check_user_prop typeset expect_value="$3" typeset value=$(zfs get -p -H -o value "$user_prop" $dtst 2>&1) - if [[ "$expect_value" == "$value" ]]; then - return 0 - else - return 1 - fi + [ "$expect_value" = "$value" ] } # @@ -258,14 +252,9 @@ function get_source { typeset prop=$1 typeset dataset=$2 - typeset source - source=$(zfs get -H -o source $prop $dataset) - if (($? != 0)); then + zfs get -H -o source $prop $dataset || log_fail "Unable to get $prop source for dataset $dataset" - fi - - echo "$source" } # @@ -289,8 +278,8 @@ function check_prop_source if [[ "$chk_value" != "$value" || "$chk_source" != "$source" ]] then - log_note "expected (value '$value', source '$source'), got \ - (value '$chk_value', source '$chk_source')" + log_note "expected (value '$value', source '$source')," \ + "got (value '$chk_value', source '$chk_source')" return 1 else return 0 @@ -315,13 +304,8 @@ function check_prop_inherit typeset value=$(get_prop "$prop" "$checked_dtst") typeset source=$(get_source "$prop" "$checked_dtst") - if [[ "$value" != "$inherited_value" || \ - "$source" != "inherited from $inherited_dtst" ]] - then - return 1 - else - return 0 - fi + [ "$value" = "$inherited_value" ] && + [ "$source" = "inherited from $inherited_dtst" ] } # @@ -339,17 +323,9 @@ function check_prop_received typeset prop="$2" typeset value="$3" - received=$(zfs get -H -o received "$prop" "$dataset") - if (($? != 0)); then - log_fail "Unable to get $prop received value for dataset " \ - "$dataset" - fi - if [[ "$received" == "$value" ]] - then - return 0 - else - return 1 - fi + received=$(zfs get -H -o received "$prop" "$dataset") || + log_fail "Unable to get $prop received value for dataset $dataset" + [ "$received" = "$value" ] } # @@ -365,14 +341,7 @@ function check_prop_missing typeset dataset="$1" typeset prop="$2" - value=$(zfs get -H -o value "$prop" "$dataset") - if (($? != 0)); then + value=$(zfs get -H -o value "$prop" "$dataset") || log_fail "Unable to get $prop value for dataset $dataset" - fi - if [[ "-" == "$value" ]] - then - return 0 - else - return 1 - fi + [ "$value" = "-" ] } diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_feature_activation.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_feature_activation.ksh index c5e6fb9c1140..760301c11758 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_feature_activation.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_set/zfs_set_feature_activation.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_share/Makefile.am deleted file mode 100644 index 35332f822e6c..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/Makefile.am +++ /dev/null @@ -1,21 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_share -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_share_001_pos.ksh \ - zfs_share_002_pos.ksh \ - zfs_share_003_pos.ksh \ - zfs_share_004_pos.ksh \ - zfs_share_005_pos.ksh \ - zfs_share_006_pos.ksh \ - zfs_share_007_neg.ksh \ - zfs_share_008_neg.ksh \ - zfs_share_009_neg.ksh \ - zfs_share_010_neg.ksh \ - zfs_share_011_pos.ksh \ - zfs_share_012_pos.ksh \ - zfs_share_013_pos.ksh \ - zfs_share_concurrent_shares.ksh - -dist_pkgdata_DATA = \ - zfs_share.cfg diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/setup.ksh index 1601087f712c..4310985e6624 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share.cfg b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share.cfg index 5c31c8fbb6e7..4c52ddef529d 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_001_pos.ksh index 6d4396aa1912..b8e9407e756e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_002_pos.ksh index 588a30f46de9..65cbe3f2a89d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_003_pos.ksh index b99c72ca14e5..b2cb8f5c470f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_004_pos.ksh index 6c48875f5268..a00a3ddcc8ac 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_005_pos.ksh index 013e04ba311f..85854085f560 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -72,20 +72,17 @@ while (( i < ${#shareopts[*]} )) do log_must zfs set sharenfs="${shareopts[i]}" $TESTPOOL/$TESTFS - option=`get_prop sharenfs $TESTPOOL/$TESTFS` + option=$(get_prop sharenfs $TESTPOOL/$TESTFS) if [[ $option != ${shareopts[i]} ]]; then log_fail "get sharenfs failed. ($option != ${shareopts[i]})" fi # Verify the single option after the leading 'ro' or 'rw'. if is_linux; then - option=`echo "$option" | cut -f2 -d','` + IFS=',' read -r _ option _ <<<"$option" fi - showshares_nfs | grep $option > /dev/null 2>&1 - if (( $? != 0 )); then - log_fail "The '$option' option was not found in share output." - fi + log_must eval "showshares_nfs | grep -q \"$option\"" ((i = i + 1)) done diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_006_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_006_pos.ksh index d5394017d780..64d6e9515549 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_007_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_007_neg.ksh index c64157cee601..6a21dd22b93c 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_007_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_007_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -63,10 +63,7 @@ do log_note "Setting sharenfs=${badopts[i]} $i " log_mustnot zfs set sharenfs="${badopts[i]}" $TESTPOOL/$TESTFS - showshares_nfs | grep $option > /dev/null 2>&1 - if (( $? == 0 )); then - log_fail "An invalid setting '$option' was propagated." - fi + log_mustnot eval "showshares_nfs | grep -q ${badopts[i]}" # # To global zone, sharenfs must be set 'off' before malformed testing. diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_008_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_008_neg.ksh index 34fbd389949d..3ac845510b24 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_008_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_008_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_009_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_009_neg.ksh index cf8c84cfe301..07aa52afd201 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_009_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_009_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -63,8 +63,7 @@ if [[ $sharenfs_val == off ]]; then log_must zfs set sharenfs=on $fs fi -showshares_nfs | grep $mpt >/dev/null 2>&1 -if (( $? != 0 )); then +if ! showshares_nfs | grep -q $mpt; then log_must zfs share $fs fi diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_010_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_010_neg.ksh index 238a5862920b..946fa36f1ffc 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_010_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_010_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_011_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_011_pos.ksh index 131b039e1cd2..7bd50e7514a8 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_011_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_011_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -67,20 +67,14 @@ typeset origdir=$PWD # unmount fails will not unshare the shared filesystem log_must zfs set sharenfs=on $TESTPOOL/$TESTFS log_must is_shared $TESTDIR -if cd $TESTDIR ; then - log_mustnot zfs umount $TESTPOOL/$TESTFS -else - log_fail "cd $TESTDIR fails" -fi +log_must cd $TESTDIR +log_mustnot zfs umount $TESTPOOL/$TESTFS log_must is_shared $TESTDIR # destroy fails will not unshare the shared filesystem log_must zfs create $TESTPOOL/$TESTFS/fs2 -if cd $TESTDIR/fs2 ; then - log_mustnot zfs destroy $TESTPOOL/$TESTFS/fs2 -else - log_fail "cd $TESTDIR/fs2 fails" -fi +log_must cd $TESTDIR/fs2 +log_mustnot zfs destroy $TESTPOOL/$TESTFS/fs2 log_must is_shared $TESTDIR/fs2 log_pass "Verify that umount and destroy fail, and do not unshare the shared" \ diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_012_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_012_pos.ksh index fe38d5559542..70eb5abd02fb 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_012_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_012_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_013_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_013_pos.ksh index 150eddac0ebb..1aacebbe44d2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_013_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_013_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -46,35 +46,35 @@ cleanup log_must zfs set sharenfs="rw=[::1]" $TESTPOOL/$TESTFS output=$(showshares_nfs 2>&1) -log_must grep "::1(" <<< "$output" > /dev/null +log_must grep -q "::1(" <<< "$output" log_must zfs set sharenfs="rw=[2::3]" $TESTPOOL/$TESTFS output=$(showshares_nfs 2>&1) -log_must grep "2::3(" <<< "$output" > /dev/null +log_must grep -q "2::3(" <<< "$output" log_must zfs set sharenfs="rw=[::1]:[2::3]" $TESTPOOL/$TESTFS output=$(showshares_nfs 2>&1) -log_must grep "::1(" <<< "$output" > /dev/null -log_must grep "2::3(" <<< "$output" > /dev/null +log_must grep -q "::1(" <<< "$output" +log_must grep -q "2::3(" <<< "$output" log_must zfs set sharenfs="rw=[::1]/64" $TESTPOOL/$TESTFS output=$(showshares_nfs 2>&1) -log_must grep "::1/64(" <<< "$output" > /dev/null +log_must grep -q "::1/64(" <<< "$output" log_must zfs set sharenfs="rw=[2::3]/128" $TESTPOOL/$TESTFS output=$(showshares_nfs 2>&1) -log_must grep "2::3/128(" <<< "$output" > /dev/null +log_must grep -q "2::3/128(" <<< "$output" log_must zfs set sharenfs="rw=[::1]/32:[2::3]/128" $TESTPOOL/$TESTFS output=$(showshares_nfs 2>&1) -log_must grep "::1/32(" <<< "$output" > /dev/null -log_must grep "2::3/128(" <<< "$output" > /dev/null +log_must grep -q "::1/32(" <<< "$output" +log_must grep -q "2::3/128(" <<< "$output" log_must zfs set sharenfs="rw=[::1]:[2::3]/64:[2a01:1234:1234:1234:aa34:234:1234:1234]:1.2.3.4/24" $TESTPOOL/$TESTFS output=$(showshares_nfs 2>&1) -log_must grep "::1(" <<< "$output" > /dev/null -log_must grep "2::3/64(" <<< "$output" > /dev/null -log_must grep "2a01:1234:1234:1234:aa34:234:1234:1234(" <<< "$output" > /dev/null -log_must grep "1\\.2\\.3\\.4/24(" <<< "$output" > /dev/null +log_must grep -q "::1(" <<< "$output" +log_must grep -q "2::3/64(" <<< "$output" +log_must grep -q "2a01:1234:1234:1234:aa34:234:1234:1234(" <<< "$output" +log_must grep -q "1\\.2\\.3\\.4/24(" <<< "$output" log_pass "NFS share ip address propagated correctly." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_concurrent_shares.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_concurrent_shares.ksh index dbaaf39b65d4..c226f56e3dcb 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_concurrent_shares.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_share/zfs_share_concurrent_shares.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -47,31 +47,20 @@ verify_runnable "global" function cleanup { wait - for fs in $(seq 0 50) + for fs in {0..50} do - log_must zfs set sharenfs=off $TESTPOOL/$TESTFS1/$fs - log_must zfs set sharenfs=off $TESTPOOL/$TESTFS2/$fs - log_must zfs set sharenfs=off $TESTPOOL/$TESTFS3/$fs - unshare_fs $TESTPOOL/$TESTFS1/$fs - unshare_fs $TESTPOOL/$TESTFS2/$fs - unshare_fs $TESTPOOL/$TESTFS3/$fs - - if mounted $TESTPOOL/$TESTFS1/$fs; then - log_must zfs unmount $TESTPOOL/$TESTFS1/$fs - fi - if mounted $TESTPOOL/$TESTFS2/$fs; then - log_must zfs unmount $TESTPOOL/$TESTFS2/$fs - fi - if mounted $TESTPOOL/$TESTFS3/$fs; then - log_must zfs unmount $TESTPOOL/$TESTFS3/$fs - fi - - datasetexists $TESTPOOL/$TESTFS1/$fs && \ - destroy_dataset $TESTPOOL/$TESTFS1/$fs -f - datasetexists $TESTPOOL/$TESTFS2/$fs && \ - destroy_dataset $TESTPOOL/$TESTFS2/$fs -f - datasetexists $TESTPOOL/$TESTFS3/$fs && \ - destroy_dataset $TESTPOOL/$TESTFS3/$fs -f + for pfs in $TESTFS1 $TESTFS2 $TESTFS3 + do + log_must zfs set sharenfs=off $TESTPOOL/$pfs/$fs + unshare_fs $TESTPOOL/$pfs/$fs + + if mounted $TESTPOOL/$pfs/$fs; then + log_must zfs unmount $TESTPOOL/$pfs/$fs + fi + + datasetexists $TESTPOOL/$pfs/$fs && \ + destroy_dataset $TESTPOOL/$pfs/$fs -f + done done log_must zfs share -a @@ -79,7 +68,7 @@ function cleanup function create_filesystems { - for fs in $(seq 0 50) + for fs in {0..50} do log_must zfs create -p $TESTPOOL/$TESTFS1/$fs log_must zfs create -p $TESTPOOL/$TESTFS2/$fs @@ -87,6 +76,12 @@ function create_filesystems done } +function sub_fail +{ + log_note $$: "$@" + exit 1 +} + # # Main test routine. # @@ -99,32 +94,45 @@ function test_share # filesystem typeset mntp=$(get_prop mountpoint $filesystem) not_shared $mntp || \ - log_fail "File system $filesystem is already shared." + sub_fail "File system $filesystem is already shared." zfs set sharenfs=on $filesystem || \ - log_fail "zfs set sharenfs=on $filesystem failed." + sub_fail "zfs set sharenfs=on $filesystem failed." is_shared $mntp || \ - log_fail "File system $filesystem is not shared (set sharenfs)." + sub_fail "File system $filesystem is not shared (set sharenfs)." # # Verify 'zfs share' works as well. # zfs unshare $filesystem || \ - log_fail "zfs unshare $filesystem failed." + sub_fail "zfs unshare $filesystem failed." is_shared $mntp && \ - log_fail "File system $filesystem is still shared." + sub_fail "File system $filesystem is still shared." + zfs share $filesystem || \ - log_fail "zfs share $filesystem failed." + sub_fail "zfs share $filesystem failed." is_shared $mntp || \ - log_fail "file system $filesystem is not shared (zfs share)." + sub_fail "file system $filesystem is not shared (zfs share)." + #log_note "Sharing a shared file system fails." zfs share $filesystem && \ - log_fail "zfs share $filesystem did not fail" + sub_fail "zfs share $filesystem did not fail" + return 0 } +function unshare_fs_nolog +{ + typeset fs=$1 + + if is_shared $fs || is_shared_smb $fs; then + zfs unshare $fs || + sub_fail "zfs unshare $fs: $?" + fi +} + # # Set the main process id so that we know to capture # failures from child processes and allow the parent process @@ -137,20 +145,16 @@ log_onexit cleanup create_filesystems child_pids=() -for fs in $(seq 0 50) +for fs in {0..50} do - test_share $TESTPOOL/$TESTFS1/$fs & - child_pids+=($!) - log_note "$TESTPOOL/$TESTFS1/$fs ==> $!" - test_share $TESTPOOL/$TESTFS2/$fs & - child_pids+=($!) - log_note "$TESTPOOL/$TESTFS2/$fs ==> $!" - test_share $TESTPOOL/$TESTFS3/$fs & - child_pids+=($!) - log_note "$TESTPOOL/$TESTFS3/$fs ==> $!" + for pfs in $TESTFS1 $TESTFS2 $TESTFS3 + do + test_share $TESTPOOL/$pfs/$fs & + child_pids+=($!) + log_note "$TESTPOOL/$pfs/$fs ==> $!" + done done -wait_for_children "${child_pids[@]}" || - log_fail "multithreaded share test failed" +log_must wait_for_children "${child_pids[@]}" log_note "Verify 'zfs share -a' succeeds." @@ -158,17 +162,16 @@ log_note "Verify 'zfs share -a' succeeds." # Unshare each of the file systems. # child_pids=() -for fs in $(seq 0 50) +for fs in {0..50} do - unshare_fs $TESTPOOL/$TESTFS1/$fs & - child_pids+=($!) - unshare_fs $TESTPOOL/$TESTFS2/$fs & - child_pids+=($!) - unshare_fs $TESTPOOL/$TESTFS3/$fs & - child_pids+=($!) + for pfs in $TESTFS1 $TESTFS2 $TESTFS3 + do + unshare_fs_nolog $TESTPOOL/$pfs/$fs & + child_pids+=($!) + log_note "$TESTPOOL/$pfs/$fs (unshare) ==> $!" + done done -wait_for_children "${child_pids[@]}" || - log_fail "multithreaded unshare failed" +log_must wait_for_children "${child_pids[@]}" # # Try a zfs share -a and verify all file systems are shared. @@ -181,21 +184,13 @@ log_must zfs share -a # unset __ZFS_POOL_EXCLUDE -for fs in $(seq 0 50) +for fs in {0..50} do - is_shared $TESTPOOL/$TESTFS1/$fs || \ - log_fail "File system $TESTPOOL/$TESTFS1/$fs is not shared" - is_shared $TESTPOOL/$TESTFS2/$fs || \ - log_fail "File system $TESTPOOL/$TESTFS2/$fs is not shared" - is_shared $TESTPOOL/$TESTFS3/$fs || \ - log_fail "File system $TESTPOOL/$TESTFS3/$fs is not shared" - - is_exported $TESTPOOL/$TESTFS1/$fs || \ - log_fail "File system $TESTPOOL/$TESTFS1/$fs is not exported" - is_exported $TESTPOOL/$TESTFS2/$fs || \ - log_fail "File system $TESTPOOL/$TESTFS2/$fs is not exported" - is_exported $TESTPOOL/$TESTFS3/$fs || \ - log_fail "File system $TESTPOOL/$TESTFS3/$fs is not exported" + for pfs in $TESTFS1 $TESTFS2 $TESTFS3 + do + log_must is_shared $TESTPOOL/$pfs/$fs + log_must is_exported $TESTPOOL/$pfs/$fs + done done -log_pass "'zfs share [ -a ] ' succeeds as root." +log_pass "'zfs share [-a] ' succeeds as root." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/Makefile.am deleted file mode 100644 index bc5b801259c8..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/Makefile.am +++ /dev/null @@ -1,16 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_snapshot -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_snapshot_001_neg.ksh \ - zfs_snapshot_002_neg.ksh \ - zfs_snapshot_003_neg.ksh \ - zfs_snapshot_004_neg.ksh \ - zfs_snapshot_005_neg.ksh \ - zfs_snapshot_006_pos.ksh \ - zfs_snapshot_007_neg.ksh \ - zfs_snapshot_008_neg.ksh \ - zfs_snapshot_009_pos.ksh - -dist_pkgdata_DATA = \ - zfs_snapshot.cfg diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/cleanup.ksh index d247991fbbdb..d9f935900f90 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/setup.ksh index b3f2b0262687..b76d7c5f2560 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot.cfg b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot.cfg index fb22367be869..cbad8a572067 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_001_neg.ksh index 2b89af9e5a43..2b883e4943ef 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_002_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_002_neg.ksh index 4ae68d411dd0..11904554a49c 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_003_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_003_neg.ksh index ca8ff8ca4357..7b6b01da07cb 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_003_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_003_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_004_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_004_neg.ksh index 16926a48ddc4..53f3015393eb 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_004_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_004_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -59,18 +59,18 @@ while ((ret == 0)); do ret=$? if ((ret != 0)); then - len=$(echo $basefs| wc -c) + len=${#basefs} log_note "The deeply-nested filesystem len: $len" # - # Make sure there are at lease 2 characters left + # Make sure there are at least 2 characters left # for snapshot name space, otherwise snapshot name # is incorrect # if ((len >= 255)); then datasetexists $basefs && destroy_dataset $basefs -r basefs=${basefs%/*} - len=$(echo $basefs| wc -c) + len=${#basefs} fi break fi diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_005_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_005_neg.ksh index c133403ac84a..c9f9803c673e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_005_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_005_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -60,7 +60,7 @@ while ((ret == 0)); do ret=$? if ((ret != 0)); then - len=$(echo $basefs | wc -c) + len=$(( ${#basefs} + 1 )) # +1 for NUL log_note "The deeply-nested filesystem len: $len" # @@ -71,7 +71,7 @@ while ((ret == 0)); do if ((len >= 255)); then datasetexists $basefs && destroy_dataset $basefs -r basefs=${basefs%/*} - len=$(echo $basefs| wc -c) + len=$(( ${#basefs} + 1 )) fi break fi diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_006_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_006_pos.ksh index 6b711286c68d..785ea8659705 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_007_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_007_neg.ksh index 9499dca21e78..5d1840447fc4 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_007_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_007_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -72,18 +72,13 @@ typeset ro_props="type used available avail creation referenced refer compressra mounted origin" typeset snap_ro_props="volsize recordsize recsize quota reservation reserv mountpoint \ sharenfs checksum compression compress atime devices exec readonly rdonly \ - setuid" + setuid version" if is_freebsd; then snap_ro_props+=" jailed" else snap_ro_props+=" zoned" fi -zfs upgrade -v > /dev/null 2>&1 -if [[ $? -eq 0 ]]; then - snap_ro_props="$snap_ro_props version" -fi - for fs in $TESTPOOL/$TESTFS $TESTPOOL/$TESTVOL $TESTPOOL/$TESTCTR $TESTPOOL ; do typeset fssnap=$fs@snap diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_008_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_008_neg.ksh index 627910abd6ed..a2f890f15f32 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_008_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_008_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_009_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_009_pos.ksh index 6fedba9e5b27..a81e82ddf546 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_009_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/zfs_snapshot_009_pos.ksh @@ -84,20 +84,17 @@ while (( i < ${#invalid_args[*]} )); do ((i = i + 1)) done log_note "verify multiple snapshot transaction group" -txg_group=$(zdb -Pd $TESTPOOL | grep snap | awk '{print $7}') +txg_group=$(zdb -Pd $TESTPOOL | awk '/snap/ {print $7}') for i in 1 2 3; do - txg_tag=$(echo "$txg_group" | nawk -v j=$i 'FNR == j {print}') + txg_tag=$(echo "$txg_group" | awk -v j=$i 'FNR == j {print}') [[ $txg_tag != $(echo "$txg_group" | \ - nawk -v j=$i 'FNR == j {print}') ]] \ + awk -v j=$i 'FNR == j {print}') ]] \ && log_fail "snapshots belong to different transaction groups" done log_note "verify snapshot contents" for ds in $datasets; do - diff -q -r /$ds /$ds/.zfs/snapshot/snap > /dev/null 2>&1 - if [[ $? -eq 1 ]]; then - log_fail "snapshot contents are different from" \ - "the filesystem" - fi + [ -d "/$ds/.zfs/snapshot/snap" ] && \ + log_must directory_diff /$ds /$ds/.zfs/snapshot/snap done # We subtract 3 + 7 + 7 + 1 = 18 for three slashes (/), strlen("TESTFSA") == 7, @@ -123,7 +120,7 @@ for x in {1..$ITERATIONS}; do for y in {1..$NUM_SNAPS}; do log_must zfs snapshot $TESTPOOL/$MYTEST@$y done; - n=$(ls -1 /$TESTPOOL/$MYTEST/.zfs/snapshot | wc -l) + n=$(ls /$TESTPOOL/$MYTEST/.zfs/snapshot | wc -l) verify_eq $n $NUM_SNAPS "count" zfs destroy -r $TESTPOOL/$MYTEST; done; diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/Makefile.am deleted file mode 100644 index 6a83edf2a443..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/Makefile.am +++ /dev/null @@ -1,10 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_sysfs -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfeature_set_unsupported.ksh \ - zfs_get_unsupported.ksh \ - zfs_set_unsupported.ksh \ - zfs_sysfs_live.ksh \ - zpool_get_unsupported.ksh \ - zpool_set_unsupported.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/cleanup.ksh index 7d6a7e13db22..a2ee99c24171 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/setup.ksh index 261bce4386e9..f40ac985e80c 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfeature_set_unsupported.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfeature_set_unsupported.ksh index c9d24224337f..523b30ff9bfe 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfeature_set_unsupported.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfeature_set_unsupported.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfs_get_unsupported.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfs_get_unsupported.ksh index 59ed7e9500dd..1d52498de0dc 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfs_get_unsupported.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfs_get_unsupported.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfs_set_unsupported.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfs_set_unsupported.ksh index 5a0b88a0b14d..5bb46ec973ad 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfs_set_unsupported.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfs_set_unsupported.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfs_sysfs_live.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfs_sysfs_live.ksh index 4bb5cc2e05ba..08a9aeaa9bad 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfs_sysfs_live.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zfs_sysfs_live.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zpool_get_unsupported.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zpool_get_unsupported.ksh index 3ab1d941e860..3624a7e8de5b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zpool_get_unsupported.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zpool_get_unsupported.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zpool_set_unsupported.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zpool_set_unsupported.ksh index 03eb2aea3fdd..a2f0c7a26405 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zpool_set_unsupported.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/zpool_set_unsupported.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/Makefile.am deleted file mode 100644 index 74cdf5c2b558..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_unload-key -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_unload-key.ksh \ - zfs_unload-key_all.ksh \ - zfs_unload-key_recursive.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/setup.ksh index 6a9af3bc28c3..4c719075a741 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/zfs_unload-key_all.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/zfs_unload-key_all.ksh index 55da68262019..6d3d37bd618a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/zfs_unload-key_all.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/zfs_unload-key_all.ksh @@ -54,7 +54,8 @@ log_must zfs create $TESTPOOL/$TESTFS1/child log_must zfs create -V 64M -o encryption=on -o keyformat=passphrase \ -o keylocation=file:///$TESTPOOL/pkey $TESTPOOL/zvol -typeset DISK2="$(echo $DISKS | awk '{ print $2}')" +typeset DISK2 _ +read -r _ DISK2 _ <<<"$DISKS" log_must zpool create -O encryption=on -O keyformat=passphrase \ -O keylocation=file:///$TESTPOOL/pkey $TESTPOOL1 $DISK2 diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/Makefile.am deleted file mode 100644 index 6507b094df47..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/Makefile.am +++ /dev/null @@ -1,20 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_unmount -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_unmount_001_pos.ksh \ - zfs_unmount_002_pos.ksh \ - zfs_unmount_003_pos.ksh \ - zfs_unmount_004_pos.ksh \ - zfs_unmount_005_pos.ksh \ - zfs_unmount_006_pos.ksh \ - zfs_unmount_007_neg.ksh \ - zfs_unmount_008_neg.ksh \ - zfs_unmount_009_pos.ksh \ - zfs_unmount_all_001_pos.ksh \ - zfs_unmount_nested.ksh \ - zfs_unmount_unload_keys.ksh - -dist_pkgdata_DATA = \ - zfs_unmount.cfg \ - zfs_unmount.kshlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/setup.ksh index 6a9af3bc28c3..4c719075a741 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount.cfg b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount.cfg index c8e46895d2a6..06d25faf0356 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount.kshlib b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount.kshlib index 525dfd162957..929452403b2e 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -53,7 +53,7 @@ function do_unmount #cmd #opt #mnt #expect zfs $cmd $opt $mnt ret=$? - if (( ret != expect)); then + if (( ret != expect )); then log_fail "'zfs $cmd $opt $mnt' " \ "unexpected return code of $ret." fi diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_001_pos.ksh index 6036eb27a062..6395a1b64363 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_002_pos.ksh index b8a50785868f..c5cabbde1642 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_003_pos.ksh index 985c3d2b1fe2..e4b292436a29 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_004_pos.ksh index bcc25ccd0f80..e139321e04ab 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_005_pos.ksh index eaea4696dfb4..23963ceaa82c 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_006_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_006_pos.ksh index 8b70e8868a02..aca84ed39856 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -56,7 +56,6 @@ log_onexit cleanup # Call cleanup to make sure the file system are mounted. cleanup mntpnt=$(get_prop mountpoint $TESTPOOL/$TESTFS) -(($? != 0)) && log_fail "get_prop mountpoint $TESTPOOL/$TESTFS" typeset -i i=0 while (( i < 10000 )); do diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_007_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_007_neg.ksh index cdf4a5a26661..1a8665b86ea8 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_007_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_007_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_008_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_008_neg.ksh index e85a0f3cbf68..3524efcd076d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_008_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_008_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_009_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_009_pos.ksh index 814d603db5e0..ece19ca1bbdf 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_009_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_009_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_all_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_all_001_pos.ksh index b5a01b53aeab..f98d48ad92d3 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_all_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_all_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_unload_keys.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_unload_keys.ksh index c92287ad75e4..d296118e1924 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_unload_keys.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_unload_keys.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/Makefile.am deleted file mode 100644 index 0845f1e2f9f6..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/Makefile.am +++ /dev/null @@ -1,11 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_unshare -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_unshare_001_pos.ksh \ - zfs_unshare_002_pos.ksh \ - zfs_unshare_003_pos.ksh \ - zfs_unshare_004_neg.ksh \ - zfs_unshare_005_neg.ksh \ - zfs_unshare_006_pos.ksh \ - zfs_unshare_007_pos.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/setup.ksh deleted file mode 100755 index 29f38e802c57..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/setup.ksh +++ /dev/null @@ -1,39 +0,0 @@ -#!/bin/ksh -p -# -# CDDL HEADER START -# -# The contents of this file are subject to the terms of the -# Common Development and Distribution License (the "License"). -# You may not use this file except in compliance with the License. -# -# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. -# See the License for the specific language governing permissions -# and limitations under the License. -# -# When distributing Covered Code, include this CDDL HEADER in each -# file and include the License file at usr/src/OPENSOLARIS.LICENSE. -# If applicable, add the following below this CDDL HEADER, with the -# fields enclosed by brackets "[]" replaced with your own identifying -# information: Portions Copyright [yyyy] [name of copyright owner] -# -# CDDL HEADER END -# - -# -# Copyright 2007 Sun Microsystems, Inc. All rights reserved. -# Use is subject to license terms. -# - -. $STF_SUITE/include/libtest.shlib - -share -s -if [ $? -ne 0 ]; then - log_unsupported "The NFS utilities are not installed" -fi - -# Make sure NFS server is running before testing. -setup_nfs_server - -DISK=${DISKS%% *} -default_container_volume_setup $DISK diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/setup.ksh new file mode 120000 index 000000000000..b5208415d45c --- /dev/null +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/setup.ksh @@ -0,0 +1 @@ +../zfs_share/setup.ksh \ No newline at end of file diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_001_pos.ksh index ac16fe97b928..4f0ef6bd389f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -149,7 +149,7 @@ while (( i < ${#mntp_fs[*]} )); do else log_must zfs set sharenfs=on ${mntp_fs[((i+1))]} is_shared ${mntp_fs[i]} || \ - log_fail "'zfs set sharenfs=on' fails to share filesystem." + log_fail "'zfs set sharenfs=on' fails to share filesystem: ${mntp_fs[i]} not shared." fi ((i = i + 2)) @@ -166,7 +166,7 @@ log_must zfs unshare -a i=0 while (( i < ${#mntp_fs[*]} )); do not_shared ${mntp_fs[i]} || \ - log_fail "'zfs unshare -a' fails to unshare all shared zfs filesystems." + log_fail "'zfs unshare -a' fails to unshare all shared zfs filesystems: ${mntp_fs[i]} still shared." ((i = i + 2)) done diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_002_pos.ksh index 1ded1b42c7ec..e5f786e0ae28 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -91,7 +91,7 @@ function test_legacy_unshare # log_fail "'zfs set sharenfs=off' fails to make ZFS " \ "filesystem $filesystem unshared." - log_must eval "share_nfs $mntp" + log_must share_nfs $mntp is_shared $mntp || \ log_fail "'share' command fails to share ZFS file system." # @@ -179,4 +179,3 @@ while (( i < ${#mntp_fs[*]} )); do done log_pass "'zfs unshare [-a]' succeeds to be aware of legacy share." - diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_003_pos.ksh index 6e66deda9bee..6d081ab6ee89 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_004_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_004_neg.ksh index fd916040b1bc..bae5e335158b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_004_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_004_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_005_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_005_neg.ksh index 7df3136dd6b4..0404e0d6ede5 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_005_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_005_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_006_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_006_pos.ksh index b4318020cc7f..ab8bfa3791b7 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -41,9 +41,7 @@ verify_runnable "global" -if is_linux; then - log_unsupported "some distros come with Samba "user shares" disabled" -fi +[ -d "/var/lib/samba/usershares" ] || log_unsupported "Samba usershares disabled" function cleanup { @@ -51,9 +49,10 @@ function cleanup log_must zfs destroy -f $TESTPOOL/$TESTFS/shared1 log_must zfs destroy -f $TESTPOOL/$TESTFS/shared2 log_must zfs destroy -f $TESTPOOL/$TESTFS/shared3 + log_must rm -f /var/lib/samba/usershares/testpool_testfs_shared{2,3} } -log_assert "Verify 'zfs unshare [nfs|smb] -a' only works on the specified "\ +log_assert "Verify 'zfs unshare [nfs|smb] -a' only works on the specified" \ "protocol." log_onexit cleanup @@ -74,19 +73,19 @@ log_must zfs share -a log_must zfs unshare nfs -a # 3. Verify that only nfs filesystems are unshared. -log_must eval "not_shared $TESTPOOL/$TESTFS/shared1" -log_must eval "not_shared $TESTPOOL/$TESTFS/shared2" -log_must eval "is_shared_smb $TESTPOOL/$TESTFS/shared2" -log_must eval "is_shared_smb $TESTPOOL/$TESTFS/shared3" +log_must not_shared $TESTPOOL/$TESTFS/shared1 +log_must not_shared $TESTPOOL/$TESTFS/shared2 +log_must is_shared_smb $TESTPOOL/$TESTFS/shared2 +log_must is_shared_smb $TESTPOOL/$TESTFS/shared3 # 4. Share all filesystems again. log_must zfs share -a # 5. Invoke 'zfs unshare smb -a' and verify only smb filesystems are unshared. log_must zfs unshare smb -a -log_must eval "is_shared $TESTPOOL/$TESTFS/shared1" -log_must eval "is_shared $TESTPOOL/$TESTFS/shared2" -log_must eval "not_shared_smb $TESTPOOL/$TESTFS/shared2" -log_must eval "not_shared_smb $TESTPOOL/$TESTFS/shared3" +log_must is_shared $TESTPOOL/$TESTFS/shared1 +log_must is_shared $TESTPOOL/$TESTFS/shared2 +log_must not_shared_smb $TESTPOOL/$TESTFS/shared2 +log_must not_shared_smb $TESTPOOL/$TESTFS/shared3 log_pass "'zfs unshare [nfs|smb] -a' only works on the specified protocol." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_007_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_007_pos.ksh index 36817a092099..0fc5bbaf3f6a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_008_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_008_pos.ksh new file mode 100755 index 000000000000..1ade9c92691e --- /dev/null +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unshare/zfs_unshare_008_pos.ksh @@ -0,0 +1,58 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +. $STF_SUITE/include/libtest.shlib + +# +# DESCRIPTION: +# Verify that datasets mounted at directories with whitespace are properly escaped +# both going in (for mountd consumption) and going out (for removing from export list) +# +# STRATEGY: +# 1. Create and share a dataset with spaces, tabs, and newlines +# 2. Verify it's shared +# 3. Unshare it +# 4. Verify it's not shared +# + +shares_can_have_whitespace || log_unsupported "Platform doesn't permit whitespace in NFS shares" +basename='a + b + c d' +escname='a\040+\040b\012\040c\011d' + +verify_runnable "global" + +function cleanup +{ + datasetexists "$TESTPOOL/$TESTFS/shared1" && \ + destroy_dataset "$TESTPOOL/$TESTFS/shared1" -f +} + +log_assert "Datasets with spaces are properly shared and unshared." +log_onexit cleanup + +log_must zfs create -o sharenfs=on -o mountpoint="$TESTDIR/$basename" "$TESTPOOL/$TESTFS/shared1" +log_must is_shared "$TESTDIR/$escname" +log_must zfs unshare "$TESTPOOL/$TESTFS/shared1" +log_mustnot is_shared "$TESTDIR/$escname" + +log_pass "Datasets with spaces are properly shared and unshared." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/Makefile.am deleted file mode 100644 index 7a71bae4bb42..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/Makefile.am +++ /dev/null @@ -1,14 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_upgrade -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_upgrade_001_pos.ksh \ - zfs_upgrade_002_pos.ksh \ - zfs_upgrade_003_pos.ksh \ - zfs_upgrade_004_pos.ksh \ - zfs_upgrade_005_pos.ksh \ - zfs_upgrade_006_neg.ksh \ - zfs_upgrade_007_neg.ksh - -dist_pkgdata_DATA = \ - zfs_upgrade.kshlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/cleanup.ksh index e2d0604fded2..6b9dbd6b6dfa 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/setup.ksh index c9f36017df9a..ab5ca7b27e6f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade.kshlib b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade.kshlib index 9c7d273b385c..948123e81157 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_001_pos.ksh index ab76461638b9..7f5f65978ebf 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -71,8 +71,8 @@ typeset expect_str2="All filesystems are formatted with the current version" typeset expect_str3="The following filesystems are out of date, and can be upgraded" typeset -i COUNT OLDCOUNT -zfs upgrade | nawk '$1 ~ "^[0-9]+$" {print $2}'> $oldoutput -OLDCOUNT=$( wc -l $oldoutput | awk '{print $1}' ) +zfs upgrade | awk '$1 ~ "^[0-9]+$" {print $2}'> $oldoutput +OLDCOUNT=$(wc -l < $oldoutput) old_datasets="" for version in $ZFS_ALL_VERSIONS ; do @@ -98,9 +98,9 @@ log_must eval 'zfs upgrade > $output 2>&1' # we also check that the usage message contains at least a description # of the current ZFS version. -log_must eval 'grep "${expect_str1} $ZFS_VERSION" $output > /dev/null 2>&1' -zfs upgrade | nawk '$1 ~ "^[0-9]+$" {print $2}'> $output -COUNT=$( wc -l $output | awk '{print $1}' ) +log_must grep -q "${expect_str1} $ZFS_VERSION" $output +zfs upgrade | awk '$1 ~ "^[0-9]+$" {print $2}'> $output +COUNT=$(wc -l < $output) typeset -i i=0 for fs in ${old_datasets}; do @@ -118,14 +118,14 @@ for fs in $old_datasets ; do done log_must eval 'zfs upgrade > $output 2>&1' -log_must eval 'grep "${expect_str1} $ZFS_VERSION" $output > /dev/null 2>&1' +log_must grep -q "${expect_str1} $ZFS_VERSION" $output if (( OLDCOUNT == 0 )); then - log_must eval 'grep "${expect_str2}" $output > /dev/null 2>&1' + log_must grep -q "${expect_str2}" $output else - log_must eval 'grep "${expect_str3}" $output > /dev/null 2>&1' + log_must grep -q "${expect_str3}" $output fi -zfs upgrade | nawk '$1 ~ "^[0-9]+$" {print $2}'> $output -COUNT=$( wc -l $output | awk '{print $1}' ) +zfs upgrade | awk '$1 ~ "^[0-9]+$" {print $2}'> $output +COUNT=$(wc -l < $output) if (( COUNT != OLDCOUNT )); then cat $output diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_002_pos.ksh index ba6e7c483edd..ea5baf8e5483 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -60,8 +60,8 @@ typeset expect_str2="Enhanced directory entries" log_must eval 'zfs upgrade -v > /dev/null 2>&1' -zfs upgrade -v | nawk '$1 ~ "^[0-9]+$" {print $0}'> $output -log_must eval 'grep "${expect_str1}" $output > /dev/null 2>&1' -log_must eval 'grep "${expect_str2}" $output > /dev/null 2>&1' +zfs upgrade -v | awk '$1 ~ "^[0-9]+$" {print $0}'> $output +log_must grep -q "${expect_str1}" $output +log_must grep -q "${expect_str2}" $output log_pass "Executing 'zfs upgrade -v' command succeeds." diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_003_pos.ksh index 57f74ca28513..e2a85ada96aa 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_004_pos.ksh index 0b8fef5cd043..5367503f270d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_005_pos.ksh index 5fcdc6e26852..ab200674b098 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_006_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_006_neg.ksh index f5eef319330b..9c4eab3ff79e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_006_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_006_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_007_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_007_neg.ksh index 6c6f94754d3c..a7d366e650ea 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_007_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/zfs_upgrade_007_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_wait/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zfs_wait/Makefile.am deleted file mode 100644 index d401fe68b1e8..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_wait/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zfs_wait -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_wait_deleteq.ksh - -dist_pkgdata_DATA = \ - zfs_wait.kshlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_wait/zfs_wait_getsubopt.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_wait/zfs_wait_getsubopt.ksh new file mode 100755 index 000000000000..d60aec96b7fe --- /dev/null +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_wait/zfs_wait_getsubopt.ksh @@ -0,0 +1,14 @@ +#!/bin/ksh -p +# SPDX-License-Identifier: 0BSD + +. $STF_SUITE/include/libtest.shlib + +# +# DESCRIPTION: +# zfs wait -t used to accept getsubopt(3)-style deleteq=whatever; +# it doesn't anymore +# + +log_mustnot zfs wait -t deleteq=getsubopt $TESTPOOL + +log_pass "'zfs wait -t' doesn't accept =getsubopt suffixes." diff --git a/tests/zfs-tests/tests/functional/cli_root/zhack/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zhack/Makefile.am deleted file mode 100644 index 931dacde6beb..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zhack/Makefile.am +++ /dev/null @@ -1,3 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zhack -dist_pkgdata_SCRIPTS = \ - zhack_label_checksum.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool/Makefile.am deleted file mode 100644 index 327f23621158..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_001_neg.ksh \ - zpool_002_pos.ksh \ - zpool_003_pos.ksh \ - zpool_colors.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool/setup.ksh index 4e3b6b0e9f47..200e914f3f5b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_001_neg.ksh index 25decd78863b..de4ca0c4ea2c 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_002_pos.ksh index e4ffa2a6c028..bd564cc6d8c3 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -47,33 +47,27 @@ function cleanup { unset ZFS_ABORT - if is_freebsd && [ -n "$old_corefile" ]; then - sysctl kern.corefile=$old_corefile - fi + log_must pop_coredump_pattern "$coresavepath" + log_must rm -rf $corepath $vdev1 $vdev2 $vdev3 # Clean up the pool created if we failed to abort. poolexists $pool && destroy_pool $pool - - rm -rf $corepath $vdev1 $vdev2 $vdev3 } log_assert "With ZFS_ABORT set, all zpool commands can abort and generate a core file." log_onexit cleanup corepath=$TESTDIR/core -corefile=$corepath/zpool.core -if [[ -d $corepath ]]; then - log_must rm -rf $corepath -fi +corefile=$corepath/core.zpool +coresavepath=$corepath/save +log_must rm -rf $corepath log_must mkdir $corepath pool=pool.$$ vdev1=$TESTDIR/file1 vdev2=$TESTDIR/file2 vdev3=$TESTDIR/file3 -for vdev in $vdev1 $vdev2 $vdev3; do - log_must mkfile $MINVDEVSIZE $vdev -done +log_must mkfile $MINVDEVSIZE $vdev1 $vdev2 $vdev3 set -A cmds "create $pool mirror $vdev1 $vdev2" "list $pool" "iostat $pool" \ "status $pool" "upgrade $pool" "get delegation $pool" "set delegation=off $pool" \ @@ -86,25 +80,12 @@ set -A badparams "" "create" "destroy" "add" "remove" "list *" "iostat" "status" "online" "offline" "clear" "attach" "detach" "replace" "scrub" \ "import" "export" "upgrade" "history -?" "get" "set" -if is_linux; then - echo $corefile >/proc/sys/kernel/core_pattern - echo 0 >/proc/sys/kernel/core_uses_pid -elif is_freebsd; then - old_corefile=$(sysctl -n kern.corefile) - log_must sysctl kern.corefile=$corefile -fi -ulimit -c unlimited - -export ZFS_ABORT=yes +log_must eval "push_coredump_pattern \"$corepath\" > \"$coresavepath\"" +log_must export ZFS_ABORT=yes for subcmd in "${cmds[@]}" "${badparams[@]}"; do - zpool $subcmd >/dev/null 2>&1 - if [[ ! -e $corefile ]]; then - log_fail "zpool $subcmd cannot generate core file with ZFS_ABORT set." - fi - rm -f $corefile + log_mustnot eval "zpool $subcmd" + log_must rm "$corefile" done -unset ZFS_ABORT - log_pass "With ZFS_ABORT set, zpool command can abort and generate core file as expected." diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_003_pos.ksh index 6f15c09a5229..215af6188420 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -46,15 +46,12 @@ function cleanup { unset ZFS_ABORT - if is_freebsd && [ -n "$old_corefile" ]; then - sysctl kern.corefile=$old_corefile - fi - - rm -rf $corepath + log_must pop_coredump_pattern "$coresavepath" + log_must rm -rf $corepath # Don't leave the pool frozen. - destroy_pool $TESTPOOL - default_mirror_setup $DISKS + log_must destroy_pool $TESTPOOL + log_must default_mirror_setup $DISKS } verify_runnable "both" @@ -63,15 +60,14 @@ log_assert "Debugging features of zpool should succeed." log_onexit cleanup corepath=$TESTDIR/core -corefile=$corepath/zpool.core -if [[ -d $corepath ]]; then - log_must rm -rf $corepath -fi +corefile=$corepath/core.zpool +coresavepath=$corepath/save +log_must rm -rf $corepath log_must mkdir $corepath log_must eval "zpool -? >/dev/null 2>&1" -if is_global_zone ; then +if is_global_zone; then log_must zpool freeze $TESTPOOL else log_mustnot zpool freeze $TESTPOOL @@ -80,21 +76,10 @@ fi log_mustnot zpool freeze fakepool -if is_linux; then - echo $corefile >/proc/sys/kernel/core_pattern - echo 0 >/proc/sys/kernel/core_uses_pid -elif is_freebsd; then - old_corefile=$(sysctl -n kern.corefile) - log_must sysctl kern.corefile=$corefile -fi -ulimit -c unlimited - -export ZFS_ABORT=yes - -zpool >/dev/null 2>&1 - -unset ZFS_ABORT +log_must eval "push_coredump_pattern \"$corepath\" > \"$coresavepath\"" +log_must export ZFS_ABORT=yes -[[ -f $corefile ]] || log_fail "zpool did not dump core by request." +log_mustnot eval "zpool >/dev/null 2>&1" +log_must [ -f "$corefile" ] log_pass "Debugging features of zpool succeed." diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_colors.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_colors.ksh index 8c7f40ba9c0b..b131bc708dbc 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_colors.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool/zpool_colors.ksh @@ -39,8 +39,7 @@ log_onexit cleanup log_assert "Test colorized zpool status output" -DISK2="$(echo $DISKS | cut -d' ' -f2)" -DISK3="$(echo $DISKS | cut -d' ' -f3)" +read -r _ DISK2 DISK3 _ <<<"$DISKS" log_must dd if=/dev/urandom of=/$TESTDIR/testfile bs=10M count=1 @@ -62,16 +61,15 @@ log_note "$(faketty TERM=xterm-256color ZFS_COLOR=1 zpool status)" # Replace the escape codes with "ESC" so they're easier to grep out="$(faketty TERM=xterm-256color ZFS_COLOR=1 zpool status | \ - grep -E 'pool:|DEGRADED' | \ - sed -r 's/[[:space:]]+//g;'$(echo -e 's/\033/ESC/g'))" + sed -E '/pool:|DEGRADED/!d;s/[[:space:]]+//g;'$(printf 's/\033/ESC/g'))" log_note "$(echo $out)" log_note "Look for 'pool:' in bold" -log_must eval "echo \"$out\" | grep -q 'ESC\[1mpool:ESC\[0m' " +log_must grep -q 'ESC\[1mpool:ESC\[0m' <<<"$out" log_note "Look for 'DEGRADED' in yellow" -log_must eval "echo \"$out\" | grep -q 'ESC\[0;33mDEGRADEDESC\[0m'" +log_must grep -q 'ESC\[0;33mDEGRADEDESC\[0m' <<<"$out" # # The escape code for 'FAULTED' is a little more tricky. The line starts like @@ -83,9 +81,11 @@ log_must eval "echo \"$out\" | grep -q 'ESC\[0;33mDEGRADEDESC\[0m'" # we can easily remove the vdev field to get what we want. # out="$(faketty TERM=xterm-256color ZFS_COLOR=1 zpool status \ - | awk '/FAULTED/{print $1$3$4}' | sed -r $(echo -e 's/\033/ESC/g'))" + | awk '/FAULTED/ {print $1$3$4}' | sed -E $(printf 's/\033/ESC/g'))" + +log_note "$(echo $out)" log_note "Look for 'FAULTED' in red" -log_must eval "echo \"$out\" | grep -q 'ESC\[0;31mFAULTEDESC\[0m'" +log_must grep -q 'ESC\[0;31mFAULTEDESC\[0m' <<<"$out" log_pass "zpool status displayed colors" diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_add/Makefile.am deleted file mode 100644 index 8d54d13f7207..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/Makefile.am +++ /dev/null @@ -1,22 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_add -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_add_001_pos.ksh \ - zpool_add_002_pos.ksh \ - zpool_add_003_pos.ksh \ - zpool_add_004_pos.ksh \ - zpool_add_005_pos.ksh \ - zpool_add_006_pos.ksh \ - zpool_add_007_neg.ksh \ - zpool_add_008_neg.ksh \ - zpool_add_009_neg.ksh \ - zpool_add_010_pos.ksh \ - add-o_ashift.ksh \ - add_prop_ashift.ksh \ - add_nested_replacing_spare.ksh \ - zpool_add_dryrun_output.ksh - -dist_pkgdata_DATA = \ - zpool_add.cfg \ - zpool_add.kshlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/add-o_ashift.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/add-o_ashift.ksh index 89cc4b0d3082..8d5ce5efa528 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/add-o_ashift.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/add-o_ashift.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -64,12 +64,8 @@ for ashift in ${ashifts[@]} do log_must zpool create $TESTPOOL $disk1 log_must zpool add -o ashift=$ashift $TESTPOOL $disk2 - verify_ashift $disk2 $ashift - if [[ $? -ne 0 ]] - then - log_fail "Device was added without setting ashift value to "\ - "$ashift" - fi + log_must verify_ashift $disk2 $ashift + # clean things for the next run log_must zpool destroy $TESTPOOL log_must zpool labelclear $disk1 @@ -81,12 +77,8 @@ do log_must zpool create $TESTPOOL $disk1 log_must set_tunable64 VDEV_FILE_PHYSICAL_ASHIFT $ashift log_must zpool add $TESTPOOL $disk2 - verify_ashift $disk2 $ashift - if [[ $? -ne 0 ]] - then - log_fail "Device was added without setting ashift value to "\ - "$ashift" - fi + log_must verify_ashift $disk2 $ashift + # clean things for the next run log_must set_tunable64 VDEV_FILE_PHYSICAL_ASHIFT $orig_ashift log_must zpool destroy $TESTPOOL diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/add_nested_replacing_spare.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/add_nested_replacing_spare.ksh index 61f5f6d1ceed..e86de0deae62 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/add_nested_replacing_spare.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/add_nested_replacing_spare.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -90,10 +90,8 @@ do log_must wait_vdev_state $TESTPOOL $REPLACE_DEV "ONLINE" 60 zpool status | awk -v poolname="$TESTPOOL" -v type="$type" 'BEGIN {s=""} $1 ~ poolname {c=4}; (c && c--) { s=s$1":" } - END { if (s != poolname":"type"-0:spare-0:replacing-0:") exit 1; }' - if [[ $? -ne 0 ]]; then + END { if (s != poolname":"type"-0:spare-0:replacing-0:") exit 1; }' || log_fail "Pool does not contain nested replacing/spare vdevs" - fi # 3. Verify 'zpool add' is able to add new devices log_must zpool add $TESTPOOL spare $SPARE_DEV2 diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/add_prop_ashift.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/add_prop_ashift.ksh index 4637fe0d84a3..964cfaa525e0 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/add_prop_ashift.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/add_prop_ashift.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -70,12 +70,8 @@ for ashift in ${ashifts[@]} do log_must zpool create -o ashift=$ashift $TESTPOOL $disk1 log_must zpool add $TESTPOOL $disk2 - verify_ashift $disk2 $ashift - if [[ $? -ne 0 ]] - then - log_fail "Device was added without setting ashift value to "\ - "$ashift" - fi + log_must verify_ashift $disk2 $ashift + # clean things for the next run log_must zpool destroy $TESTPOOL log_must zpool labelclear $disk1 @@ -88,12 +84,8 @@ do do log_must zpool create -o ashift=$ashift $TESTPOOL $disk1 log_must zpool add -o ashift=$cmdval $TESTPOOL $disk2 - verify_ashift $disk2 $cmdval - if [[ $? -ne 0 ]] - then - log_fail "Device was added without setting ashift " \ - "value to $cmdval" - fi + log_must verify_ashift $disk2 $cmdval + # clean things for the next run log_must zpool destroy $TESTPOOL log_must zpool labelclear $disk1 diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/cleanup.ksh index 33bd94fdc4f9..f3691e9207a9 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/setup.ksh index 13bd33ee42aa..ef19ba998678 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add.cfg b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add.cfg index a634b8b3c617..1831e6563e8c 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add.kshlib b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add.kshlib index c64b4a35aa03..cc850e3e451f 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -50,9 +50,8 @@ function find_vfstab_dev # function find_mnttab_dev { - typeset mnttabdev + typeset mnttabdev _ typeset mnttabdevs="" - typeset line if is_freebsd; then # FreeBSD doesn't have a mnttab file. @@ -61,21 +60,16 @@ function find_mnttab_dev return 0 elif is_linux; then typeset mnttab="/etc/mtab" - typeset tmpfile="$TEST_BASE_DIR/mtab.tmp" else typeset mnttab="/etc/mnttab" - typeset tmpfile="$TEST_BASE_DIR/mnttab.tmp" fi - cat $mnttab | grep "^${DEV_DSKDIR}" >$tmpfile - while read -r line + while read -r mnttabdev _ do - mnttabdev=`echo "$line" | awk '{print $1}'` mnttabdev=${mnttabdev%%:} mnttabdevs="$mnttabdev $mnttabdevs" - done <$tmpfile + done < <(grep "^${DEV_DSKDIR}" $mnttab) - rm -f $tmpfile echo $mnttabdevs } diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_001_pos.ksh index 191ec839a955..9ef96f554744 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_002_pos.ksh index 67810bbf985b..c5c06f76340b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_003_pos.ksh index a6b03ff3257f..62eddff7f6b6 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -90,8 +90,8 @@ done log_must zpool add -f $TESTPOOL $config zpool status $TESTPOOL | awk 'NR == 1, /NAME/ { next } /^$/ {exit} {print $1}' > "$TMPFILE_PREFIX-vdevtree" -cat "$TMPFILE_PREFIX-dryrun" | awk 'NR == 1, /would/ {next} - /^$/ {next} {print $1}' > "$TMPFILE_PREFIX-vdevtree-n" -log_must eval "diff $TMPFILE_PREFIX-vdevtree-n $TMPFILE_PREFIX-vdevtree" +awk 'NR == 1, /would/ {next} + /^$/ {next} {print $1}' "$TMPFILE_PREFIX-dryrun" > "$TMPFILE_PREFIX-vdevtree-n" +log_must diff $TMPFILE_PREFIX-vdevtree-n $TMPFILE_PREFIX-vdevtree log_pass "'zpool add -n ...' executes successfully." diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_004_pos.ksh index 64e52960d3d6..fbaed2af13c5 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_005_pos.ksh index c40f8db6f026..4990ef9d29b0 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_006_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_006_pos.ksh index 2c3f488ea267..836d6d21c746 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_007_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_007_neg.ksh index 4e9535c1c6b5..a7df42fc788f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_007_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_007_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_008_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_008_neg.ksh index 77a899f70c83..c43f90792a9d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_008_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_008_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_009_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_009_neg.ksh index 0a9fa868cb81..d7f3a900e8fd 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_009_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_009_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_010_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_010_pos.ksh index 771b689c93fd..b8b25db1b9f9 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_010_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_010_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_dryrun_output.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_dryrun_output.ksh index 73dec9240326..fa1ce9c64d33 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_dryrun_output.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_add/zpool_add_dryrun_output.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_attach/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_attach/Makefile.am deleted file mode 100644 index cc742f33d2aa..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_attach/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_attach -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_attach_001_neg.ksh \ - attach-o_ashift.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_attach/attach-o_ashift.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_attach/attach-o_ashift.ksh index 618c6992edb4..a96bc16761f7 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_attach/attach-o_ashift.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_attach/attach-o_ashift.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -69,23 +69,14 @@ do for cmdval in ${ashifts[@]} do log_must zpool create -o ashift=$ashift $TESTPOOL1 $disk1 - verify_ashift $disk1 $ashift - if [[ $? -ne 0 ]] - then - log_fail "Pool was created without setting ashift " \ - "value to $ashift" - fi + log_must verify_ashift $disk1 $ashift + # ashift_of(attached_disk) <= ashift_of(existing_vdev) if [[ $cmdval -le $ashift ]] then log_must zpool attach -o ashift=$cmdval $TESTPOOL1 \ $disk1 $disk2 - verify_ashift $disk2 $ashift - if [[ $? -ne 0 ]] - then - log_fail "Device was attached without " \ - "setting ashift value to $ashift" - fi + log_must verify_ashift $disk2 $ashift else log_mustnot zpool attach -o ashift=$cmdval $TESTPOOL1 \ $disk1 $disk2 diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_attach/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_attach/cleanup.ksh index 89c146249e71..3167a5097b5a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_attach/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_attach/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_attach/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_attach/setup.ksh index 2229f87e6208..23dc57ed3679 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_attach/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_attach/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_attach/zpool_attach_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_attach/zpool_attach_001_neg.ksh index 01b8e41ee439..50993a52a5e2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_attach/zpool_attach_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_attach/zpool_attach_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_clear/Makefile.am deleted file mode 100644 index 6986d38efc9d..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/Makefile.am +++ /dev/null @@ -1,11 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_clear -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_clear_001_pos.ksh \ - zpool_clear_002_neg.ksh \ - zpool_clear_003_neg.ksh \ - zpool_clear_readonly.ksh - -dist_pkgdata_DATA = \ - zpool_clear.cfg diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_clear/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_clear/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_clear/setup.ksh index 6a9af3bc28c3..4c719075a741 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_clear/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear.cfg b/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear.cfg index e6977350af90..4b4b05c8993b 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_001_pos.ksh index 1188ca10d14d..b1a295cd2337 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -50,9 +50,7 @@ function cleanup poolexists $TESTPOOL1 && \ log_must zpool destroy -f $TESTPOOL1 - for file in `ls $TEST_BASE_DIR/file.*`; do - log_must rm -f $file - done + log_must rm -f $fbase.{0..2} } @@ -60,14 +58,8 @@ log_assert "Verify 'zpool clear' can clear errors of a storage pool." log_onexit cleanup #make raw files to create various configuration pools -typeset -i i=0 -while (( i < 3 )); do - log_must truncate -s $FILESIZE $TEST_BASE_DIR/file.$i - - (( i = i + 1 )) -done - fbase=$TEST_BASE_DIR/file +log_must truncate -s $FILESIZE $fbase.{0..2} set -A poolconf "mirror $fbase.0 $fbase.1 $fbase.2" \ "raidz1 $fbase.0 $fbase.1 $fbase.2" \ "raidz2 $fbase.0 $fbase.1 $fbase.2" @@ -75,59 +67,24 @@ set -A poolconf "mirror $fbase.0 $fbase.1 $fbase.2" \ function check_err # [] { typeset pool=$1 - shift - if (( $# > 0 )); then - typeset checkvdev=$1 - else - typeset checkvdev="" - fi - typeset -i errnum=0 - typeset c_read=0 - typeset c_write=0 - typeset c_cksum=0 - typeset tmpfile=$TEST_BASE_DIR/file.$$ - typeset healthstr="pool '$pool' is healthy" - typeset output="`zpool status -x $pool`" - - [[ "$output" == "$healthstr" ]] && return $errnum - - zpool status -x $pool | grep -v "^$" | grep -v "pool:" \ - | grep -v "state:" | grep -v "config:" \ - | grep -v "errors:" > $tmpfile - typeset line - typeset -i fetchbegin=1 - while read line; do - if (( $fetchbegin != 0 )); then - echo $line | grep "NAME" >/dev/null 2>&1 - (( $? == 0 )) && (( fetchbegin = 0 )) - continue - fi + typeset checkvdev=$2 - if [[ -n $checkvdev ]]; then - echo $line | grep $checkvdev >/dev/null 2>&1 - (( $? != 0 )) && continue - c_read=`echo $line | awk '{print $3}'` - c_write=`echo $line | awk '{print $4}'` - c_cksum=`echo $line | awk '{print $5}'` - if [ $c_read != 0 ] || [ $c_write != 0 ] || \ - [ $c_cksum != 0 ] - then - (( errnum = errnum + 1 )) - fi - break - fi + [ "$(zpool status -x $pool)" = "pool '$pool' is healthy" ] && return + + typeset -i skipstart=1 + typeset vdev _ c_read c_write c_cksum rest + while read -r vdev _ c_read c_write c_cksum rest; do + if [ $skipstart -ne 0 ]; then + [ "$vdev" = "NAME" ] && skipstart=0 + continue + fi - c_read=`echo $line | awk '{print $3}'` - c_write=`echo $line | awk '{print $4}'` - c_cksum=`echo $line | awk '{print $5}'` - if [ $c_read != 0 ] || [ $c_write != 0 ] || \ - [ $c_cksum != 0 ] - then - (( errnum = errnum + 1 )) + if [ -n "$checkvdev" ]; then + [ "$vdev" = "$checkvdev" ] || continue fi - done <$tmpfile - return $errnum + [ $c_read$c_write$c_cksum = 000 ] || return + done < <(zpool status -x $pool | grep -ve "^$" -e "pool:" -e "state:" -e "config:" -e "errors:") } function do_testing # @@ -137,6 +94,7 @@ function do_testing # typeset type=$1 shift typeset vdev="$@" + (( i = $RANDOM % 3 )) log_must zpool create -f $TESTPOOL1 $vdev log_must zfs create $FS @@ -146,14 +104,13 @@ function do_testing # # avail=$(get_prop available $FS) fill_mb=$(((avail / 1024 / 1024) * 25 / 100)) - log_must dd if=/dev/urandom of=$file.$i bs=$BLOCKSZ count=$fill_mb + log_must dd if=/dev/urandom of=$file bs=$BLOCKSZ count=$fill_mb # # Make errors to the testing pool by overwrite the vdev device with # dd command. We do not want to have a full overwrite. That # may cause the system panic. So, we should skip the vdev label space. # - (( i = $RANDOM % 3 )) typeset -i wcount=0 typeset -i size case $FILESIZE in @@ -173,25 +130,19 @@ function do_testing # (( wcount = FILESIZE/1024 - 512 )) ;; esac - dd if=/dev/zero of=$fbase.$i seek=512 bs=1024 count=$wcount conv=notrunc \ - > /dev/null 2>&1 + dd if=/dev/zero of=$fbase.$i seek=512 bs=1024 count=$wcount conv=notrunc 2>/dev/null sync_all_pools log_must sync #ensure the vdev files are written out log_must zpool scrub -w $TESTPOOL1 - check_err $TESTPOOL1 && \ - log_fail "No error generated." - if [[ $type == "device" ]]; then - log_must zpool clear $TESTPOOL1 $fbase.$i - ! check_err $TESTPOOL1 $fbase.$i && \ - log_fail "'zpool clear' fails to clear error for $fbase.$i device." + log_mustnot check_err $TESTPOOL1 + typeset dev= + if [ "$type" = "device" ]; then + dev=$fbase.$i fi - if [[ $type == "pool" ]]; then - log_must zpool clear $TESTPOOL1 - ! check_err $TESTPOOL1 && \ - log_fail "'zpool clear' fails to clear error for pool $TESTPOOL1." - fi + log_must zpool clear $TESTPOOL1 $dev + log_must check_err $TESTPOOL1 $dev log_must zpool destroy $TESTPOOL1 } diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_002_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_002_neg.ksh index 91a637b3377b..e8ba4e66420b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_003_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_003_neg.ksh index 637fdd84288b..46c07bf56076 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_003_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_003_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_readonly.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_readonly.ksh index 9eb2a3608f17..e3d089d0469b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_readonly.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_clear/zpool_clear_readonly.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_create/Makefile.am deleted file mode 100644 index 5ffaae5b152c..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/Makefile.am +++ /dev/null @@ -1,50 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_create -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_create_001_pos.ksh \ - zpool_create_002_pos.ksh \ - zpool_create_003_pos.ksh \ - zpool_create_004_pos.ksh \ - zpool_create_005_pos.ksh \ - zpool_create_006_pos.ksh \ - zpool_create_007_neg.ksh \ - zpool_create_008_pos.ksh \ - zpool_create_009_neg.ksh \ - zpool_create_010_neg.ksh \ - zpool_create_011_neg.ksh \ - zpool_create_012_neg.ksh \ - zpool_create_014_neg.ksh \ - zpool_create_015_neg.ksh \ - zpool_create_016_pos.ksh \ - zpool_create_017_neg.ksh \ - zpool_create_018_pos.ksh \ - zpool_create_019_pos.ksh \ - zpool_create_020_pos.ksh \ - zpool_create_021_pos.ksh \ - zpool_create_022_pos.ksh \ - zpool_create_023_neg.ksh \ - zpool_create_024_pos.ksh \ - zpool_create_encrypted.ksh \ - zpool_create_crypt_combos.ksh \ - zpool_create_draid_001_pos.ksh \ - zpool_create_draid_002_pos.ksh \ - zpool_create_draid_003_pos.ksh \ - zpool_create_draid_004_pos.ksh \ - zpool_create_features_001_pos.ksh \ - zpool_create_features_002_pos.ksh \ - zpool_create_features_003_pos.ksh \ - zpool_create_features_004_neg.ksh \ - zpool_create_features_005_pos.ksh \ - zpool_create_features_006_pos.ksh \ - zpool_create_features_007_pos.ksh \ - zpool_create_features_008_pos.ksh \ - zpool_create_features_009_pos.ksh \ - create-o_ashift.ksh \ - zpool_create_tempname.ksh \ - zpool_create_dryrun_output.ksh - -dist_pkgdata_DATA = \ - draidcfg.gz \ - zpool_create.cfg \ - zpool_create.shlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/cleanup.ksh index a50487754053..2bce044319c4 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/create-o_ashift.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/create-o_ashift.ksh index 2c1f6e0ca659..f6f46cb98e90 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/create-o_ashift.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/create-o_ashift.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -59,8 +59,7 @@ function write_device_uberblocks # typeset device=$1 typeset pool=$2 - while [ "$(zdb -quuul $device | grep -c 'invalid')" -ne 0 ] - do + while zdb -quuul $device | grep -q 'invalid'; do sync_pool $pool true done } @@ -89,8 +88,6 @@ function verify_device_uberblocks # exit 1 } }' - - return $? } log_assert "zpool create -o ashift=' works with different ashift values" @@ -114,11 +111,8 @@ do "$ashift (current = $pprop)" fi write_device_uberblocks $disk $TESTPOOL - verify_device_uberblocks $disk ${ubcount[$i]} - if [[ $? -ne 0 ]] - then - log_fail "Pool was created with unexpected number of uberblocks" - fi + log_must verify_device_uberblocks $disk ${ubcount[$i]} + # clean things for the next run log_must zpool destroy $TESTPOOL log_must zpool labelclear $disk diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/setup.ksh index 115126b1ac5b..f9e3add1ed68 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create.cfg b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create.cfg index 976570d621b7..3f22136e7460 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create.shlib b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create.shlib index 005cf979befa..4987bc3a9dac 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create.shlib +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create.shlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -83,14 +83,9 @@ function find_vfstab_dev # function save_dump_dev { - typeset dumpdev="" - if is_illumos; then - typeset fnd="Dump device" - dumpdev=`dumpadm | grep "$fnd" | cut -f2 -d : | \ - awk '{print $1}'` + dumpadm | grep "Dump device" | cut -f2 -d : | awk '{print $1}' fi - echo $dumpdev } # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_001_pos.ksh index 42f57beae2a3..f24a9c705622 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_002_pos.ksh index 2f709086f751..21df8eb2b1ff 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_003_pos.ksh index dd8d0107aed2..ae81828b3e8c 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_004_pos.ksh index 835cd1f547bd..9e73c29b3b00 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_005_pos.ksh index e1d8cc474545..26c0119242a1 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -90,8 +90,8 @@ do $file.1 $file.2 $file.3 $file.4 ! poolexists $TESTPOOL && \ log_fail "Creating pool with $opt fails." - mpt=`zfs mount | egrep "^$TESTPOOL[^/]" | awk '{print $2}'` - (( ${#mpt} == 0 )) && \ + mpt=`zfs mount | awk -v pat="^$TESTPOOL[^/]" '$0 ~ pat {print $2}'` + [ -z "$mpt" ] && \ log_fail "$TESTPOOL created with $opt is not mounted." mpt_val=$(get_prop "mountpoint" $TESTPOOL) [[ "$mpt" != "$mpt_val" ]] && \ diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_006_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_006_pos.ksh index 79b41fdaec90..37790c03fee3 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_007_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_007_neg.ksh index 2873202cce91..ad0a282f731d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_007_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_007_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_008_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_008_pos.ksh index 56bb64c64051..f33fb8a2281e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_009_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_009_neg.ksh index e2f38990314c..bb74c9cdda5f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_009_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_009_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_010_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_010_neg.ksh index 36bbaa7de33a..4bb9a6d8bb49 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_010_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_010_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_011_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_011_neg.ksh index 9437033ae547..1dcbaff19eca 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_011_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_011_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_012_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_012_neg.ksh index 36888e497369..e3ed3f850958 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_012_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_012_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -51,12 +51,11 @@ function cleanup } if is_freebsd; then - typeset swap_disks=$(swapinfo -l | grep "/dev" | awk '{print $1}') + typeset swap_disks=$(swapinfo -l | awk '/\/dev/ {print $1}') elif is_linux; then - typeset swap_disks=`swapon -s | grep "/dev" | awk '{print $1}'` + typeset swap_disks=$(swapon -s | awk '/\/dev/ {print $1}') else - typeset swap_disks=`swap -l | grep "c[0-9].*d[0-9].*s[0-9]" | \ - awk '{print $1}'` + typeset swap_disks=$(swap -l | awk '/c[0-9].*d[0-9].*s[0-9]/ {print $1}') fi log_assert "'zpool create' should fail with disk slice in swap." diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_014_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_014_neg.ksh index 44ed950f7870..df14bde6b178 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_014_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_014_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_015_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_015_neg.ksh index babf5ca9c66d..6e02b585a916 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_015_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_015_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_016_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_016_pos.ksh index 1fa205b0f253..eee869820f58 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_016_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_016_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -67,19 +67,17 @@ function cleanup fi } -typeset swap_disks=$(swap -l | grep -v "swapfile" | awk '{print $1}') -typeset dump_device=$(dumpadm | grep "Dump device" | awk '{print $3}') +typeset swap_disks=$(swap -l | awk '!/swapfile/ {print $1}') +typeset dump_device=$(dumpadm | awk '/Dump device/ {print $3}') log_assert "'zpool create' should success with no device in swap." log_onexit cleanup for sdisk in $swap_disks; do log_note "Executing: swap -d $sdisk" - swap -d $sdisk >/dev/null 2>&1; - if [[ $? != 0 ]]; then + swap -d $sdisk >/dev/null 2>&1 || log_untested "Unable to delete swap device $sdisk because of" \ "insufficient RAM" - fi done log_must zpool create $TESTPOOL $DISK0 diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_017_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_017_neg.ksh index ded1e3c3966b..3892fbad165c 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_017_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_017_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_018_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_018_pos.ksh index 6ad662f95011..6da1709071ba 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_018_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_018_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_019_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_019_pos.ksh index 694ea2163c94..deddb8ba58a5 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_019_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_019_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_020_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_020_pos.ksh index 104b5ec9868a..5711d9f2686a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_020_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_020_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -75,25 +75,16 @@ log_must zpool get all $TESTPOOL zpool get all $TESTPOOL > $values # check for the cachefile property, verifying that it's set to 'none' -grep "$TESTPOOL[ ]*cachefile[ ]*none" $values > /dev/null 2>&1 -if [ $? -ne 0 ] -then - log_fail "zpool property \'cachefile\' was not set to \'none\'." -fi +log_must grep -q "$TESTPOOL[ ]*cachefile[ ]*none" $values # check that the root = /mountpoint property is set correctly -grep "$TESTPOOL[ ]*altroot[ ]*/${TESTPOOL}.root" $values > /dev/null 2>&1 -if [ $? -ne 0 ] -then - log_fail "zpool property root was not found in pool output." -fi +log_must grep -q "$TESTPOOL[ ]*altroot[ ]*/${TESTPOOL}.root" $values rm $values # finally, check that the pool has no reference in /etc/zfs/zpool.cache if [[ -f /etc/zfs/zpool.cache ]] ; then - REF=$(strings /etc/zfs/zpool.cache | grep ${TESTPOOL}) - if [ ! -z "$REF" ] + if strings /etc/zfs/zpool.cache | grep -q ${TESTPOOL} then strings /etc/zfs/zpool.cache log_fail "/etc/zfs/zpool.cache appears to have a reference to $TESTPOOL" diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_021_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_021_pos.ksh index 655f887b60ad..00e347897206 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_021_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_021_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -49,12 +49,13 @@ verify_runnable "global" function cleanup { datasetexists $TESTPOOL && destroy_pool $TESTPOOL + log_must rm -df "/tmp/mnt$$" } log_onexit cleanup -log_assert "'zpool create -O property=value pool' can successfully create a pool \ - with correct filesystem property set." +log_assert "'zpool create -O property=value pool' can successfully create a pool" \ + "with correct filesystem property set." set -A RW_FS_PROP "quota=536870912" \ "reservation=536870912" \ @@ -80,14 +81,11 @@ fi typeset -i i=0 while (( $i < ${#RW_FS_PROP[*]} )); do log_must zpool create -O ${RW_FS_PROP[$i]} -f $TESTPOOL $DISKS - datasetexists $TESTPOOL || \ - log_fail "zpool create $TESTPOOL fail." - propertycheck $TESTPOOL ${RW_FS_PROP[i]} || \ - log_fail "${RW_FS_PROP[i]} is failed to set." + log_must datasetexists $TESTPOOL + log_must propertycheck $TESTPOOL ${RW_FS_PROP[i]} log_must zpool destroy $TESTPOOL (( i = i + 1 )) done -log_pass "'zpool create -O property=value pool' can successfully create a pool \ - with correct filesystem property set." - +log_pass "'zpool create -O property=value pool' can successfully create a pool" \ + "with correct filesystem property set." diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_022_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_022_pos.ksh index 4a918c0a683a..0e9da881261e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_022_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_022_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -49,12 +49,13 @@ verify_runnable "global" function cleanup { poolexists $TESTPOOL && destroy_pool $TESTPOOL + log_must rm -df "/tmp/mnt$$" } log_onexit cleanup -log_assert "'zpool create -O property=value pool' can successfully create a pool \ - with multiple filesystem properties set." +log_assert "'zpool create -O property=value pool' can successfully create a pool" \ + "with multiple filesystem properties set." set -A RW_FS_PROP "quota=536870912" \ "reservation=536870912" \ @@ -81,15 +82,13 @@ while (( $i < ${#RW_FS_PROP[*]} )); do done log_must zpool create $opts -f $TESTPOOL $DISKS -datasetexists $TESTPOOL || log_fail "zpool create $TESTPOOL fail." +log_must datasetexists $TESTPOOL i=0 while (( $i < ${#RW_FS_PROP[*]} )); do - propertycheck $TESTPOOL ${RW_FS_PROP[i]} || \ - log_fail "${RW_FS_PROP[i]} is failed to set." + log_must propertycheck $TESTPOOL ${RW_FS_PROP[i]} (( i = i + 1 )) done -log_pass "'zpool create -O property=value pool' can successfully create a pool \ - with multiple filesystem properties set." - +log_pass "'zpool create -O property=value pool' can successfully create a pool" \ + "with multiple filesystem properties set." diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_023_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_023_neg.ksh index f101521bd3e8..d2ae03eee687 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_023_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_023_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -52,7 +52,7 @@ log_onexit cleanup set -A args "QuOta=none" "quota=non" "quota=abcd" "quota=0" "quota=" \ "ResErVaTi0n=none" "reserV=none" "reservation=abcd" "reserv=" \ - "recorDSize=64k" "recordsize=2M" "recordsize=2048K" \ + "recorDSize=64k" "recordsize=32M" "recordsize=32768K" \ "recordsize=256" "recsize=" "recsize=zero" "recordsize=0" \ "mountPoint=/tmp/tmpfile$$" "mountpoint=non0" "mountpoint=" \ "mountpoint=LEGACY" "mounpoint=none" \ diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_024_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_024_pos.ksh index 5b464c3c248b..a09ae6abc972 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_024_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_024_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -74,24 +74,20 @@ child_pools="" function zpool_stress { - typeset pool=$1 + typeset pool="$1-$$" typeset vdev0="$TEST_BASE_DIR/$pool-vdev0.img" typeset vdev1="$TEST_BASE_DIR/$pool-vdev1.img" typeset -i iters=$2 typeset retry=10 typeset j=0 - truncate -s $FILESIZE $vdev0 - truncate -s $FILESIZE $vdev1 + truncate -s $FILESIZE $vdev0 $vdev1 while [[ $j -lt $iters ]]; do ((j = j + 1)) sleep 1 - zpool create $pool $vdev0 $vdev1 - if [ $? -ne 0 ]; then - return 1; - fi + zpool create $pool $vdev0 $vdev1 || return 1 # The 'zfs destroy' command is retried because it can # transiently return EBUSY when blkid is concurrently @@ -100,13 +96,8 @@ function zpool_stress while [[ $k -lt $retry ]]; do ((k = k + 1)) - zpool destroy $pool - if [ $? -eq 0 ]; then - break; - elif [ $k -eq $retry ]; then - return 1; - fi - + zpool destroy $pool && break + [ $k -eq $retry ] && return 1 sleep 3 done done @@ -118,13 +109,11 @@ function zpool_stress # 1. Create 128 process each of which create/destroy a pool 5 times. typeset i=0 while [[ $i -lt 128 ]]; do - typeset uuid=$(uuidgen | cut -c1-13) - - zpool_stress $TESTPOOL-$uuid 5 & + zpool_stress $TESTPOOL-$i 5 & typeset pid=$! child_pids="$child_pids $pid" - child_pools="$child_pools $TESTPOOL-$uuid" + child_pools="$child_pools $TESTPOOL-$i-$pid" ((i = i + 1)) done diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_001_pos.ksh index 9717af505267..9e3e22db7cad 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_002_pos.ksh index 2e1ff39311ab..acc3c06bacb0 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_003_pos.ksh index 52cd00cf4ee4..8058691fcb12 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_004_pos.ksh index 6b700fa362a4..dbcb6e5ce737 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_draid_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_dryrun_output.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_dryrun_output.ksh index 1e4db20cfedf..485891c945a3 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_dryrun_output.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_dryrun_output.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_001_pos.ksh index 67eff8edcf3b..38a7ec06076b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_002_pos.ksh index 7ad4ae805565..db79b31a91ca 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_003_pos.ksh index c72edffde67d..032878dcb12b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_004_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_004_neg.ksh index 5c6b29c4939e..3dfaf22c15a1 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_004_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_004_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_005_pos.ksh index a8ed95109f2a..4b0618017e38 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_006_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_006_pos.ksh index fe98434d1bb6..292346ad96de 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_007_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_007_pos.ksh index 8c812911b3b3..c35ca8e8c92c 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_008_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_008_pos.ksh index 0580d444e724..99db3d31a088 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_009_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_009_pos.ksh index 052c18dcee2b..cd3111496461 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_009_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_features_009_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_tempname.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_tempname.ksh index 8fd1cea36e28..a75bdecff475 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_tempname.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_create/zpool_create_tempname.ksh @@ -55,11 +55,9 @@ for poolprop in "${poolprops[@]}"; do # 2. Verify the pool is created with the specified temporary name log_must poolexists $TEMPPOOL log_mustnot poolexists $TESTPOOL - propname="$(awk -F= '{print $1}' <<< $fsprop)" - propval="$(awk -F= '{print $2}' <<< $fsprop)" + IFS='=' read -r propname propval <<<"$fsprop" log_must test "$(get_prop $propname $TEMPPOOL)" == "$propval" - propname="$(awk -F= '{print $1}' <<< $poolprop)" - propval="$(awk -F= '{print $2}' <<< $poolprop)" + IFS='=' read -r propname propval <<<"$poolprop" log_must test "$(get_pool_prop $propname $TEMPPOOL)" == "$propval" # Cleanup destroy_pool $TEMPPOOL diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/Makefile.am deleted file mode 100644 index 7f9e00d5826f..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_destroy -dist_pkgdata_SCRIPTS = \ - zpool_destroy_001_pos.ksh \ - zpool_destroy_002_pos.ksh \ - zpool_destroy_003_neg.ksh - -dist_pkgdata_DATA = \ - zpool_destroy.cfg diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy.cfg b/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy.cfg index bf6026747f9a..c67212ce14af 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -28,7 +28,7 @@ # Copyright (c) 2012 by Delphix. All rights reserved. # -export DISK_ARRAY_NUM=$(echo ${DISKS} | nawk '{print NF}') +export DISK_ARRAY_NUM=$(echo ${DISKS} | awk '{print NF}') export DISKSARRAY=$DISKS echo $DISKS | read DISK0 DISK1 diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy_001_pos.ksh index c25b6c9230a7..e5248a1bad9a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy_002_pos.ksh index a634f10f1114..258c64754790 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy_003_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy_003_neg.ksh index 75b77b4bde72..12718eb8d987 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy_003_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_destroy/zpool_destroy_003_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_detach/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_detach/Makefile.am deleted file mode 100644 index b22018c1d550..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_detach/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_detach -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_detach_001_neg.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_detach/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_detach/cleanup.ksh index 89c146249e71..3167a5097b5a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_detach/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_detach/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_detach/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_detach/setup.ksh index 2229f87e6208..23dc57ed3679 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_detach/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_detach/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_detach/zpool_detach_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_detach/zpool_detach_001_neg.ksh index 74f8e1e664fa..5d1523fa0f92 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_detach/zpool_detach_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_detach/zpool_detach_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_events/.gitignore b/tests/zfs-tests/tests/functional/cli_root/zpool_events/.gitignore deleted file mode 100644 index a1f8c14838fa..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_events/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/ereports diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_events/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_events/Makefile.am deleted file mode 100644 index 765df102229d..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_events/Makefile.am +++ /dev/null @@ -1,26 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_events -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_events - -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_events_clear.ksh \ - zpool_events_cliargs.ksh \ - zpool_events_follow.ksh \ - zpool_events_poolname.ksh \ - zpool_events_errors.ksh \ - zpool_events_duplicates.ksh \ - zpool_events_clear_retained.ksh - -dist_pkgdata_DATA = \ - zpool_events.cfg \ - zpool_events.kshlib - -ereports_LDADD = \ - $(abs_top_builddir)/lib/libnvpair/libnvpair.la \ - $(abs_top_builddir)/lib/libzfs/libzfs.la - -pkgexec_PROGRAMS = ereports -ereports_SOURCES = ereports.c diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_clear_retained.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_clear_retained.ksh index 22212a8f50a5..0eda2de1f6cb 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_clear_retained.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_clear_retained.ksh @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -55,8 +55,6 @@ OLD_LEN_MAX=$(get_tunable ZEVENT_LEN_MAX) RETAIN_MAX=$(get_tunable ZEVENT_RETAIN_MAX) OLD_CHECKSUMS=$(get_tunable CHECKSUM_EVENTS_PER_SECOND) -EREPORTS="$STF_SUITE/tests/functional/cli_root/zpool_events/ereports" - function cleanup { log_must set_tunable64 CHECKSUM_EVENTS_PER_SECOND $OLD_CHECKSUMS @@ -66,7 +64,7 @@ function cleanup if poolexists $POOL ; then zpool export $POOL fi - log_must rm -f $VDEV1 $VDEV2 $VDEV3 + log_must rm -fd $VDEV1 $VDEV2 $VDEV3 $SUPPLY $MOUNTDIR } function damage_and_repair @@ -78,9 +76,9 @@ function damage_and_repair log_must dd conv=notrunc if=$SUPPLY of=$VDEV1 bs=1M seek=4 count=$DAMAGEBLKS log_must zpool scrub $POOL log_must zpool wait -t scrub $POOL - log_note "pass $1 observed $($EREPORTS | grep -c checksum) checksum ereports" + log_note "pass $1 observed $(ereports | grep -c checksum) checksum ereports" - repaired=$(zpool status $POOL | grep "scan: scrub repaired" | awk '{print $4}') + repaired=$(zpool status $POOL | awk '/scan: scrub repaired/ {print $4}') if [ "$repaired" == "0B" ]; then log_fail "INVALID TEST -- expected scrub to repair some blocks" else @@ -90,7 +88,7 @@ function damage_and_repair function checksum_error_count { - zpool status -p $POOL | grep $VDEV1 | awk '{print $5}' + zpool status -p $POOL | awk -v dev=$VDEV1 '$0 ~ dev {print $5}' } assertion="Damage to recently repaired blocks should be reported/counted" @@ -132,4 +130,3 @@ else log_note observed $errcnt new checksum errors after a scrub log_pass "$assertion" fi - diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_cliargs.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_cliargs.ksh index 1623a18e4740..27e2cf0ba472 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_cliargs.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_cliargs.ksh @@ -37,7 +37,7 @@ function log_must_follow # sleep 3 kill $pid if [[ $? -ne 0 ]]; then - log_fail "'$command' does not work as expected." + log_fail "'$command' exited early." else log_note "'$command' works successfully." fi diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_duplicates.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_duplicates.ksh index 7023c49e51f2..5bc369480d90 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_duplicates.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_duplicates.ksh @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -46,8 +46,6 @@ FILESIZE="10M" OLD_LEN_MAX=$(get_tunable ZEVENT_LEN_MAX) RETAIN_MAX=$(get_tunable ZEVENT_RETAIN_MAX) -EREPORTS="$STF_SUITE/tests/functional/cli_root/zpool_events/ereports" - duplicates=false function cleanup @@ -58,7 +56,7 @@ function cleanup if poolexists $POOL ; then destroy_pool $POOL fi - log_must rm -f $VDEV1 $VDEV2 + log_must rm -fd $VDEV1 $VDEV2 $MOUNTDIR } log_assert "Duplicate I/O ereport errors are not posted" @@ -103,7 +101,7 @@ function do_dup_test # Read the file a few times to generate some # duplicate errors of the same blocks for _ in {1..15}; do - dd if=$FILEPATH of=/dev/null bs=128K > /dev/null 2>&1 + dd if=$FILEPATH of=/dev/null bs=128K 2>/dev/null done log_must zinject -c all fi @@ -117,7 +115,7 @@ function do_dup_test log_must zinject -c all - ereports="$($EREPORTS | sort)" + ereports="$(ereports | sort)" actual=$(echo "$ereports" | wc -l) unique=$(echo "$ereports" | uniq | wc -l) log_note "$actual total $ERR $RW ereports where $unique were unique" @@ -140,4 +138,3 @@ if $duplicates; then else log_pass "Duplicate I/O ereport errors are not posted" fi - diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_errors.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_errors.ksh index f26c65f9db2c..2e98cdfc6f4a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_errors.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_errors.ksh @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -59,7 +59,7 @@ function cleanup if poolexists $POOL ; then log_must destroy_pool $POOL fi - log_must rm -f $VDEV1 $VDEV2 $VDEV3 + log_must rm -fd $VDEV1 $VDEV2 $VDEV3 $MOUNTDIR } log_assert "Check that the number of zpool errors match the number of events" @@ -115,7 +115,7 @@ function do_test out="$(zpool status -p | grep $VDEV1)" if [ "$ERR" == "corrupt" ] ; then - events=$(zpool events | grep checksum | wc -l) + events=$(zpool events | grep -c checksum) val=$(echo "$out" | awk '{print $5}') str="checksum" elif [ "$ERR" == "io" ] ; then diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_follow.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_follow.ksh index 258de033b86c..3311eb546676 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_follow.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_follow.ksh @@ -49,14 +49,14 @@ log_must eval "zpool events -H -f > $EVENTS_FILE &" pid=$! # 3. Generate some ZFS events -for i in `seq 1 $EVENTS_NUM`; do +for i in {1..$EVENTS_NUM}; do log_must zpool clear $TESTPOOL done # wait a bit to allow the kernel module to process new events zpool_events_settle # 4. Verify 'zpool events -f' successfully recorded these new events -EVENTS_LOG=$(cat $EVENTS_FILE | wc -l) +EVENTS_LOG=$(wc -l < $EVENTS_FILE) if [[ $EVENTS_LOG -ne $EVENTS_NUM ]]; then log_fail "Unexpected number of events: $EVENTS_LOG != $EVENTS_NUM" fi diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_poolname.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_poolname.ksh index 42c46712f3d9..0e3829fcc8d8 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_poolname.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events_poolname.ksh @@ -50,10 +50,8 @@ log_must zpool create $NEWPOOL $DISK log_must zpool events -c # 3. Generate some ZFS events on both pools -for i in `seq 1 $EVENTS_NUM`; do +for i in {1..$EVENTS_NUM}; do log_must zpool clear $TESTPOOL -done -for i in `seq 1 $EVENTS_NUM`; do log_must zpool clear $NEWPOOL done # wait a bit to allow the kernel module to process new events @@ -61,14 +59,11 @@ zpool_events_settle # 4. Verify 'zpool events poolname' successfully display events zpool events -v $TESTPOOL | - awk -v POOL=$TESTPOOL '/pool = / {if ($3 != "\""POOL"\"") exit 1}' -if [[ $? -ne 0 ]]; then + awk -v POOL=$TESTPOOL '/pool = / && $3 != "\""POOL"\"" {exit 1}' || log_fail "Unexpected events for pools other than $TESTPOOL" -fi + zpool events -v $NEWPOOL | - awk -v POOL=$NEWPOOL '/pool = / {if ($3 != "\""POOL"\"") exit 1}' -if [[ $? -ne 0 ]]; then + awk -v POOL=$NEWPOOL '/pool = / && $3 != "\""POOL"\"" {exit 1}' || log_fail "Unexpected events for pools other than $NEWPOOL" -fi log_pass "'zpool events poolname' display events only from the chosen pool." diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/Makefile.am deleted file mode 100644 index beaa411e37cb..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/Makefile.am +++ /dev/null @@ -1,12 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_expand -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_expand_001_pos.ksh \ - zpool_expand_002_pos.ksh \ - zpool_expand_003_neg.ksh \ - zpool_expand_004_pos.ksh \ - zpool_expand_005_pos.ksh - -dist_pkgdata_DATA = \ - zpool_expand.cfg diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/cleanup.ksh index 059c3839050b..2b21521f867d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/setup.ksh index 9832a441c20b..ff052d23f659 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand.cfg b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand.cfg index bec5fb1638aa..0a9c1954f123 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_001_pos.ksh index 6bbd46289f7c..b8bdfcfe78c2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -128,7 +128,7 @@ for type in " " mirror raidz draid:1s; do typeset size_addition=$(zpool history -il $TESTPOOL1 |\ grep "pool '$TESTPOOL1' size:" | \ grep "vdev online" | \ - grep "(+${expansion_size}" | wc -l) + grep -c "(+${expansion_size}") if [[ $size_addition -ne 3 ]]; then log_fail "pool $TESTPOOL1 has not expanded, " \ @@ -139,31 +139,22 @@ for type in " " mirror raidz draid:1s; do zpool history -il $TESTPOOL1 | \ grep "pool '$TESTPOOL1' size:" | \ grep "vdev online" | \ - grep "(+${expansion_size})" >/dev/null 2>&1 - - if [[ $? -ne 0 ]] ; then + grep -q "(+${expansion_size})" || log_fail "pool $TESTPOOL1 has not expanded" - fi elif [[ $type == "draid:1s" ]]; then typeset expansion_size=$((2*($exp_size-$org_size))) zpool history -il $TESTPOOL1 | \ grep "pool '$TESTPOOL1' size:" | \ grep "vdev online" | \ - grep "(+${expansion_size})" >/dev/null 2>&1 - - if [[ $? -ne 0 ]]; then - log_fail "pool $TESTPOOL has not expanded" - fi + grep -q "(+${expansion_size})" || + log_fail "pool $TESTPOOL has not expanded" else typeset expansion_size=$((3*($exp_size-$org_size))) zpool history -il $TESTPOOL1 | \ grep "pool '$TESTPOOL1' size:" | \ grep "vdev online" | \ - grep "(+${expansion_size})" >/dev/null 2>&1 - - if [[ $? -ne 0 ]]; then - log_fail "pool $TESTPOOL has not expanded" - fi + grep -q "(+${expansion_size})" || + log_fail "pool $TESTPOOL has not expanded" fi else log_fail "pool $TESTPOOL1 is not autoexpanded after vdev " \ diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_002_pos.ksh index 62843b062291..34357ca070b5 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -131,7 +131,7 @@ for type in " " mirror raidz draid:1s; do typeset size_addition=$(zpool history -il $TESTPOOL1 \ | grep "pool '$TESTPOOL1' size:" | \ grep "vdev online" | \ - grep "(+${expansion_size}" | wc -l) + grep -c "(+${expansion_size}") if [[ $size_addition -ne $i ]]; then log_fail "pool $TESTPOOL1 has not expanded " \ @@ -143,34 +143,25 @@ for type in " " mirror raidz draid:1s; do zpool history -il $TESTPOOL1 | \ grep "pool '$TESTPOOL1' size:" | \ grep "vdev online" | \ - grep "(+${expansion_size})" >/dev/null 2>&1 - - if [[ $? -ne 0 ]]; then - log_fail "pool $TESTPOOL1 has not expanded " \ - "after zpool online -e" - fi + grep -q "(+${expansion_size})" || + log_fail "pool $TESTPOOL1 has not expanded " \ + "after zpool online -e" elif [[ $type == "draid:1s" ]]; then typeset expansion_size=$((2*($exp_size-$org_size))) zpool history -il $TESTPOOL1 | \ grep "pool '$TESTPOOL1' size:" | \ grep "vdev online" | \ - grep "(+${expansion_size})" >/dev/null 2>&1 - - if [[ $? -ne 0 ]] ; then - log_fail "pool $TESTPOOL1 has not expanded " \ - "after zpool online -e" - fi + grep -q "(+${expansion_size})" || + log_fail "pool $TESTPOOL1 has not expanded " \ + "after zpool online -e" else typeset expansion_size=$((3*($exp_size-$org_size))) zpool history -il $TESTPOOL1 | \ grep "pool '$TESTPOOL1' size:" | \ grep "vdev online" | \ - grep "(+${expansion_size})" >/dev/null 2>&1 - - if [[ $? -ne 0 ]] ; then - log_fail "pool $TESTPOOL1 has not expanded " \ - "after zpool online -e" - fi + grep -q "(+${expansion_size})" || + log_fail "pool $TESTPOOL1 has not expanded " \ + "after zpool online -e" fi else log_fail "pool $TESTPOOL1 did not expand after vdev " \ diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_003_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_003_neg.ksh index b3c71b666a59..68b17592e2f8 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_003_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_003_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -39,7 +39,6 @@ # Once set zpool autoexpand=off, zpool can *NOT* autoexpand by # Dynamic VDEV Expansion # -# # STRATEGY: # 1) Create three vdevs (loopback, scsi_debug, and file) # 2) Create pool by using the different devices and set autoexpand=off @@ -73,7 +72,7 @@ log_onexit cleanup log_assert "zpool can not expand if set autoexpand=off after vdev expansion" -for type in " " mirror raidz draid; do +for type in "" mirror raidz draid; do log_note "Setting up loopback, scsi_debug, and file vdevs" log_must truncate -s $org_size $FILE_LO DEV1=$(losetup -f) @@ -89,11 +88,7 @@ for type in " " mirror raidz draid; do # The -f is required since we're mixing disk and file vdevs. log_must zpool create -f $TESTPOOL1 $type $DEV1 $DEV2 $DEV3 - typeset autoexp=$(get_pool_prop autoexpand $TESTPOOL1) - if [[ $autoexp != "off" ]]; then - log_fail "zpool $TESTPOOL1 autoexpand should be off but is " \ - "$autoexp" - fi + log_must [ "$(get_pool_prop autoexpand $TESTPOOL1)" = "off" ] typeset prev_size=$(get_pool_prop size $TESTPOOL1) @@ -107,8 +102,8 @@ for type in " " mirror raidz draid; do log_must losetup -c $DEV1 sleep 3 - echo "2" > /sys/bus/pseudo/drivers/scsi_debug/virtual_gb - echo "1" > /sys/class/block/$DEV2/device/rescan + log_must eval "echo 2 > /sys/bus/pseudo/drivers/scsi_debug/virtual_gb" + log_must eval "echo 1 > /sys/class/block/$DEV2/device/rescan" block_device_wait sleep 3 @@ -118,19 +113,11 @@ for type in " " mirror raidz draid; do sleep 5 # check for zpool history for the pool size expansion - zpool history -il $TESTPOOL1 | grep "pool '$TESTPOOL1' size:" | \ - grep "vdev online" >/dev/null 2>&1 - - if [[ $? -eq 0 ]]; then - log_fail "pool $TESTPOOL1 is not autoexpand after vdev " \ - "expansion" - fi - - typeset expand_size=$(get_pool_prop size $TESTPOOL1) + zpool history -il $TESTPOOL1 | grep "pool '$TESTPOOL1' size:" | + grep "vdev online" && + log_fail "pool $TESTPOOL1 is not autoexpand after vdev expansion" - if [[ "$prev_size" != "$expand_size" ]]; then - log_fail "pool $TESTPOOL1 size changed after vdev expansion" - fi + log_must [ "$(get_pool_prop size $TESTPOOL1)" = "$prev_size" ] cleanup done diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_004_pos.ksh index 09e2b6da2148..e50c1f8b1606 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_005_pos.ksh index 54ec73b67b26..5e8f1d7053e8 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_expand/zpool_expand_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_export/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_export/Makefile.am deleted file mode 100644 index 1c06d5b59e9b..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_export/Makefile.am +++ /dev/null @@ -1,12 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_export -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_export_001_pos.ksh \ - zpool_export_002_pos.ksh \ - zpool_export_003_neg.ksh \ - zpool_export_004_pos.ksh - -dist_pkgdata_DATA = \ - zpool_export.cfg \ - zpool_export.kshlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_export/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_export/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_export/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_export/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_export/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_export/setup.ksh index 023920dae1e9..57bbf32711e3 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_export/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_export/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export.cfg b/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export.cfg index 8bfb067c7aac..0cedb61c25d9 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -30,9 +30,9 @@ . $STF_SUITE/include/libtest.shlib -export DISK_ARRAY_NUM=$(echo ${DISKS} | nawk '{print NF}') -export DISK1=$(echo $DISKS | awk '{print $1}') -export DISK2=$(echo $DISKS | awk '{print $3}') +export DISK_ARRAY_NUM=$(echo ${DISKS} | awk '{print NF}') +read -r DISK1 _ DISK2 _ <<<"$DISKS" +export DISK1 DISK2 if is_linux; then set_slice_prefix diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export.kshlib b/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export.kshlib index 5484f20674d5..54f805ea71a4 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_001_pos.ksh index 111453c7a161..f084ac023997 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_002_pos.ksh index 8040d12b92d2..8bdaddc43da4 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -45,9 +45,7 @@ verify_runnable "global" function cleanup { - cd $olddir || \ - log_fail "Couldn't cd back to $olddir" - + log_must cd $olddir zpool_export_cleanup } @@ -57,16 +55,9 @@ log_onexit cleanup log_assert "Verify a busy ZPOOL cannot be exported." -ismounted "$TESTPOOL/$TESTFS" -(( $? != 0 )) && \ - log_fail "$TESTDIR not mounted. Unable to continue." - -cd $TESTDIR || \ - log_fail "Couldn't cd to $TESTDIR" - +log_must ismounted "$TESTPOOL/$TESTFS" +log_must cd $TESTDIR log_mustnot zpool export $TESTPOOL - -poolexists $TESTPOOL || \ - log_fail "$TESTPOOL not found in 'zpool list' output." +log_must poolexists $TESTPOOL log_pass "Unable to export a busy ZPOOL as expected." diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_003_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_003_neg.ksh index a2ee7fbdf908..ec4eae3d5685 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_003_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_003_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_004_pos.ksh index 9be3f23c4fda..4b4038bb8bfd 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_get/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_get/Makefile.am deleted file mode 100644 index 0c87c9b37763..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_get/Makefile.am +++ /dev/null @@ -1,12 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_get -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_get_001_pos.ksh \ - zpool_get_002_pos.ksh \ - zpool_get_003_pos.ksh \ - zpool_get_004_neg.ksh \ - zpool_get_005_pos.ksh - -dist_pkgdata_DATA = \ - zpool_get.cfg zpool_get_parsable.cfg diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_get/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_get/cleanup.ksh index 6c992b5f02ae..dde462e66cf9 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_get/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_get/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_get/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_get/setup.ksh index 063ad743b7a8..03d149cd967a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_get/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_get/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get.cfg b/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get.cfg index accbf69cf9c8..99a70fa2c04d 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -97,5 +97,8 @@ if is_linux || is_freebsd; then "feature@bookmark_v2" "feature@livelist" "feature@zstd_compress" + "feature@zilsaxattr" + "feature@head_errlog" + "feature@blake3" ) -fi \ No newline at end of file +fi diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_001_pos.ksh index d3b21d663f16..49017fb5eef4 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_002_pos.ksh index ba83fadb06f1..fef573c8dd7d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -66,29 +66,20 @@ log_must zpool get all $TESTPOOL zpool get all $TESTPOOL > $values log_note "Checking zpool get all output for a header." -grep ^"NAME " $values > /dev/null 2>&1 -if [ $? -ne 0 ] -then - log_fail "The header was not printed from zpool get all" -fi +log_must grep -q ^"NAME " $values while [ $i -lt "${#properties[@]}" ] do log_note "Checking for ${properties[$i]} property" - grep "$TESTPOOL *${properties[$i]}" $values > /dev/null 2>&1 - if [ $? -ne 0 ] - then - log_fail "zpool property ${properties[$i]} was not found\ - in pool output." - fi + log_must grep -q "$TESTPOOL *${properties[$i]}" $values i=$(( $i + 1 )) done # increment the counter to include the header line i=$(( $i + 1 )) -COUNT=$(wc $values | awk '{print $1}') +COUNT=$(wc -l < $values) if [ $i -ne $COUNT ] then log_fail "Found zpool features not in the zpool_get test config $i/$COUNT." diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_003_pos.ksh index 89fca9cbd485..2a4c98b61c65 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -60,14 +60,10 @@ while [ $i -lt "${#properties[@]}" ] do log_note "Checking for ${properties[$i]} property" log_must eval "zpool get ${properties[$i]} $TESTPOOL > $values" - grep "${properties[$i]}" $values > /dev/null 2>&1 - if [ $? -ne 0 ] - then - log_fail "${properties[$i]} not seen in output" - fi - grep "^NAME " $values > /dev/null 2>&1 + log_must grep -q "${properties[$i]}" $values + # only need to check this once. - if [ $i -eq 0 ] && [ $? -ne 0 ] + if [ $i -eq 0 ] && ! grep -q "^NAME " $values then log_fail "Header not seen in zpool get output" fi diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_004_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_004_neg.ksh index 1edd2ba79af9..873f58ee7f31 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_004_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_004_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -51,7 +51,8 @@ set -A arguments "$TESTPOOL $TESTPOOL" "$TESTPOOL rubbish" "-v $TESTPOOL" \ "nosuchproperty $TESTPOOL" "--$TESTPOOL" "all all" \ "type $TESTPOOL" "usage: $TESTPOOL" "bootfs $TESTPOOL@" \ "bootfs,bootfs $TESTPOOL" "name $TESTPOOL" "t%d%s" \ - "bootfs,delegation $TESTPOOL" "delegation $TESTPOOL@" + "bootfs,delegation $TESTPOOL" "delegation $TESTPOOL@" \ + "-o name=getsubopt allocated $TESTPOOL" for arg in $arguments do diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_005_pos.ksh index ad27d180fdb1..e00690fd94a7 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -50,14 +50,9 @@ typeset -i i=0 while [[ $i -lt "${#properties[@]}" ]]; do log_note "Checking for parsable ${properties[$i]} property" log_must eval "zpool get -p ${properties[$i]} $TESTPOOL >/tmp/value.$$" - grep "${properties[$i]}" /tmp/value.$$ >/dev/null 2>&1 - if [[ $? -ne 0 ]]; then - log_fail "${properties[$i]} not seen in output" - fi - - typeset v=$(grep "${properties[$i]}" /tmp/value.$$ | awk '{print $3}') + log_must grep -q "${properties[$i]}" /tmp/value.$$ - log_note "${properties[$i]} has a value of $v" + typeset v=$(awk -v p="${properties[$i]}" '$0 ~ p {print $3}' /tmp/value.$$) # Determine if this value is a valid number, result in return code log_must test -n "$v" diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_parsable.cfg b/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_parsable.cfg index e7b95a47223b..22778b41177d 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_parsable.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get_parsable.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_history/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_history/Makefile.am deleted file mode 100644 index 8755f8f6c8a9..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_history/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_history -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_history_001_neg.ksh \ - zpool_history_002_pos.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_history/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_history/cleanup.ksh index e4998138f701..7bb00caed5a3 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_history/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_history/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_history/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_history/setup.ksh index b2a852a4c9b5..06829b1da5a2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_history/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_history/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_history/zpool_history_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_history/zpool_history_001_neg.ksh index b5cd8d529ed0..052108479079 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_history/zpool_history_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_history/zpool_history_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_history/zpool_history_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_history/zpool_history_002_pos.ksh index 0146aa5bc1dd..5796c60a3772 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_history/zpool_history_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_history/zpool_history_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_import/Makefile.am deleted file mode 100644 index a8c9a31dcfdc..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/Makefile.am +++ /dev/null @@ -1,51 +0,0 @@ -SUBDIRS = blockfiles - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_import -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - import_cachefile_device_added.ksh \ - import_cachefile_device_removed.ksh \ - import_cachefile_device_replaced.ksh \ - import_cachefile_mirror_attached.ksh \ - import_cachefile_mirror_detached.ksh \ - import_cachefile_paths_changed.ksh \ - import_cachefile_shared_device.ksh \ - import_devices_missing.ksh \ - import_paths_changed.ksh \ - import_rewind_config_changed.ksh \ - import_rewind_device_replaced.ksh \ - zpool_import_001_pos.ksh \ - zpool_import_002_pos.ksh \ - zpool_import_003_pos.ksh \ - zpool_import_004_pos.ksh \ - zpool_import_005_pos.ksh \ - zpool_import_006_pos.ksh \ - zpool_import_007_pos.ksh \ - zpool_import_008_pos.ksh \ - zpool_import_009_neg.ksh \ - zpool_import_010_pos.ksh \ - zpool_import_011_neg.ksh \ - zpool_import_012_pos.ksh \ - zpool_import_013_neg.ksh \ - zpool_import_014_pos.ksh \ - zpool_import_015_pos.ksh \ - zpool_import_016_pos.ksh \ - zpool_import_017_pos.ksh \ - zpool_import_all_001_pos.ksh \ - zpool_import_features_001_pos.ksh \ - zpool_import_features_002_neg.ksh \ - zpool_import_features_003_pos.ksh \ - zpool_import_missing_001_pos.ksh \ - zpool_import_missing_002_pos.ksh \ - zpool_import_missing_003_pos.ksh \ - zpool_import_rename_001_pos.ksh \ - zpool_import_encrypted.ksh \ - zpool_import_encrypted_load.ksh \ - zpool_import_errata3.ksh \ - zpool_import_errata4.ksh - -dist_pkgdata_DATA = \ - zpool_import.cfg \ - zpool_import.kshlib - diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/blockfiles/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_import/blockfiles/Makefile.am deleted file mode 100644 index dc3685e4b9cc..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/blockfiles/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_import/blockfiles -dist_pkgdata_DATA = \ - unclean_export.dat.bz2 \ - cryptv0.dat.bz2 \ - missing_ivset.dat.bz2 diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/cleanup.ksh index bee0e11a4ff4..a1f0284c97ed 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/import_cachefile_shared_device.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/import_cachefile_shared_device.ksh index 87942b4a52e4..ce9885904b01 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/import_cachefile_shared_device.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/import_cachefile_shared_device.ksh @@ -50,14 +50,8 @@ function dev_checksum log_note "Compute checksum of '$dev'" - checksum=$(md5digest $dev) - if [[ $? -ne 0 ]]; then + md5digest $dev || log_fail "Failed to compute checksum of '$dev'" - return 1 - fi - - echo "$checksum" - return 0 } function test_shared_device diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/import_devices_missing.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/import_devices_missing.ksh index 53828c912ca3..af6ac8d78e4e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/import_devices_missing.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/import_devices_missing.ksh @@ -68,7 +68,7 @@ function test_devices_missing log_must generate_data $TESTPOOL1 $MD5FILE2 "second" - log_must zpool export $TESTPOOL1 + log_must_busy zpool export $TESTPOOL1 log_must mv $missingvdevs $BACKUP_DEVICE_DIR @@ -85,7 +85,7 @@ function test_devices_missing "get suspended." verify_data_md5sums $MD5FILE >/dev/null 2>&1 - log_must zpool export $TESTPOOL1 + log_must_busy zpool export $TESTPOOL1 typeset newpaths=$(echo "$missingvdevs" | \ sed "s:$DEVICE_DIR:$BACKUP_DEVICE_DIR:g") diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/import_rewind_config_changed.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/import_rewind_config_changed.ksh index 3ac8c104f1ca..4b6fcbd80af1 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/import_rewind_config_changed.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/import_rewind_config_changed.ksh @@ -61,6 +61,7 @@ function test_common typeset detachvdev="${4:-}" typeset removevdev="${5:-}" typeset finalpool="${6:-}" + typeset retval=1 typeset poolcheck="$poolcreate" @@ -120,19 +121,26 @@ function test_common # while having a checkpoint, we take it after the # operation that changes the config. # + # However, it is possible the MOS data was overwritten + # in which case the pool will either be unimportable, or + # may have been rewound prior to the data being written. + # In which case an error is returned and test_common() + # is retried by the caller to minimize false positives. + # log_must zpool checkpoint $TESTPOOL1 log_must overwrite_data $TESTPOOL1 "" log_must zpool export $TESTPOOL1 - log_must zpool import -d $DEVICE_DIR -T $txg $TESTPOOL1 - log_must check_pool_config $TESTPOOL1 "$poolcheck" + if zpool import -d $DEVICE_DIR -T $txg $TESTPOOL1; then + verify_data_md5sums $MD5FILE && retval=0 - log_must verify_data_md5sums $MD5FILE + log_must check_pool_config $TESTPOOL1 "$poolcheck" + log_must zpool destroy $TESTPOOL1 + fi # Cleanup - log_must zpool destroy $TESTPOOL1 if [[ -n $pathstochange ]]; then for dev in $pathstochange; do log_must mv "${dev}_new" $dev @@ -143,6 +151,7 @@ function test_common log_must zpool destroy $TESTPOOL2 log_note "" + return $retval } function test_add_vdevs @@ -152,7 +161,12 @@ function test_add_vdevs log_note "$0: pool '$poolcreate', add $addvdevs." - test_common "$poolcreate" "$addvdevs" + for retry in $(seq 1 5); do + test_common "$poolcreate" "$addvdevs" && return + log_note "Retry $retry / 5 for test_add_vdevs()" + done + + log_fail "Exhausted all 5 retries for test_add_vdevs()" } function test_attach_vdev @@ -163,7 +177,12 @@ function test_attach_vdev log_note "$0: pool '$poolcreate', attach $attachvdev to $attachto." - test_common "$poolcreate" "" "$attachto $attachvdev" + for retry in $(seq 1 5); do + test_common "$poolcreate" "" "$attachto $attachvdev" && return + log_note "Retry $retry / 5 for test_attach_vdev()" + done + + log_fail "Exhausted all 5 retries for test_attach_vdev()" } function test_detach_vdev @@ -173,7 +192,12 @@ function test_detach_vdev log_note "$0: pool '$poolcreate', detach $detachvdev." - test_common "$poolcreate" "" "" "$detachvdev" + for retry in $(seq 1 5); do + test_common "$poolcreate" "" "" "$detachvdev" && return + log_note "Retry $retry / 5 for test_detach_vdev()" + done + + log_fail "Exhausted all 5 retries for test_detach_vdev()" } function test_attach_detach_vdev @@ -186,7 +210,13 @@ function test_attach_detach_vdev log_note "$0: pool '$poolcreate', attach $attachvdev to $attachto," \ "then detach $detachvdev." - test_common "$poolcreate" "" "$attachto $attachvdev" "$detachvdev" + for retry in $(seq 1 5); do + test_common "$poolcreate" "" "$attachto $attachvdev" \ + "$detachvdev" && return + log_note "Retry $retry / 5 for test_attach_detach_vdev()" + done + + log_fail "Exhausted all 5 retries for test_attach_detach_vdev()" } function test_remove_vdev @@ -197,7 +227,13 @@ function test_remove_vdev log_note "$0: pool '$poolcreate', remove $removevdev." - test_common "$poolcreate" "" "" "" "$removevdev" "$finalpool" + for retry in $(seq 1 5); do + test_common "$poolcreate" "" "" "" "$removevdev" \ + "$finalpool" && return + log_note "Retry $retry / 5 for test_remove_vdev()" + done + + log_fail "Exhausted all 5 retries for test_remove_vdev()" } # Record txg history diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/setup.ksh index 22e619d7411b..47f85fc493c1 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import.cfg b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import.cfg index 25f541ebf185..4a9fb5e7489a 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import.kshlib b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import.kshlib index 8bbd668a9317..559810ff0e30 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import.kshlib @@ -22,7 +22,7 @@ function cleanup { # clear any remaining zinjections - log_must zinject -c all > /dev/null + log_must eval "zinject -c all > /dev/null" destroy_pool $TESTPOOL1 @@ -64,17 +64,13 @@ function write_some_data typeset files10mb=${2:-10} typeset ds="$pool/fillerds" - zfs create $ds - [[ $? -ne 0 ]] && return 1 + zfs create $ds || return 1 # Create 100 MB of data typeset file="/$ds/fillerfile" for i in {1..$files10mb}; do - dd if=/dev/urandom of=$file.$i bs=128k count=80 - [[ $? -ne 0 ]] && return 1 + dd if=/dev/urandom of=$file.$i bs=128k count=80 || return 1 done - - return 0 } # @@ -143,13 +139,12 @@ function verify_data_md5sums return 1 fi - cat $md5file | \ - while read digest file; do + while read -r digest file; do typeset digest1=$(md5digest $file) if [[ "$digest1" != "$digest" ]]; then return 1 fi - done + done < $md5file return 0 } @@ -181,20 +176,20 @@ function _translate_vdev # typeset keywords="mirror replacing raidz1 raidz2 raidz3 indirect draid1 draid2 draid3" for word in $keywords; do - echo $vdev | egrep -qE \ - "^${word}-[0-9]+\$|^${word}:[0-9]+d:[0-9]c:[0-9]+s-[0-9]+\$" - if [[ $? -eq 0 ]]; then + if echo $vdev | + grep -qE "^${word}-[0-9]+\$|^${word}:[0-9]+d:[0-9]c:[0-9]+s-[0-9]+\$" + then vdev=$word break fi done - [[ $vdev == "logs" ]] && echo "log" && return 0 - [[ $vdev == "raidz1" ]] && echo "raidz" && return 0 - [[ $vdev == "draid1" ]] && echo "draid" && return 0 - - echo $vdev - return 0 + case "$vdev" in + logs) echo "log" ;; + raidz1) echo "raidz" ;; + draid1) echo "draid" ;; + *) echo $vdev ;; + esac } # @@ -217,9 +212,8 @@ function check_pool_config typeset expected=$2 typeset status - status=$(zpool status $poolname 2>&1) - if [[ $? -ne 0 ]]; then - if ( $logfailure ); then + if ! status=$(zpool status $poolname 2>&1); then + if $logfailure; then log_note "zpool status $poolname failed: $status" fi return 1 @@ -227,8 +221,7 @@ function check_pool_config typeset actual="" typeset began=false - printf "$status\n" | while read line; do - typeset vdev=$(echo "$line" | awk '{printf $1}') + while read -r vdev _; do if ( ! $began ) && [[ $vdev == NAME ]]; then began=true continue @@ -240,12 +233,12 @@ function check_pool_config vdev=$(_translate_vdev $vdev) actual="$actual $vdev" fi - done + done <<<"$status" expected="$poolname $expected" if [[ "$actual" != "$expected" ]]; then - if ( $logfailure ); then + if $logfailure; then log_note "expected pool vdevs:" log_note "> '$expected'" log_note "actual pool vdevs:" @@ -272,13 +265,11 @@ function wait_for_pool_config timeout=$(( $timeout + $(date +%s) )) while (( $(date +%s) < $timeout )); do - check_pool_config -q $poolname "$expectedconfig" - [[ $? -eq 0 ]] && return 0 + check_pool_config -q $poolname "$expectedconfig" && return 0 sleep 3 done check_pool_config $poolname "$expectedconfig" - return $? } # @@ -287,16 +278,14 @@ function wait_for_pool_config function check_pool_healthy { typeset pool=$1 - typeset status - status=$(zpool status $pool 2>&1) - if [[ $? -ne 0 ]]; then + + if ! status=$(zpool status $pool 2>&1); then log_note "zpool status $pool failed: $status" return 1 fi - status=$(echo "$status" | grep "$pool" | grep -v "pool:" | \ - awk '{print $2}') + status=$(echo "$status" | awk -v p="$pool" '!/pool:/ && $0 ~ p {print $2}') if [[ $status != "ONLINE" ]]; then log_note "Invalid zpool status for '$pool': '$status'" \ @@ -314,9 +303,7 @@ function pool_is_replacing { typeset pool=$1 - zpool status $pool | grep "replacing" | grep "ONLINE" > /dev/null - - return $? + zpool status $pool | grep "replacing" | grep -q "ONLINE" } function set_vdev_validate_skip diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_001_pos.ksh index 6369a297c17d..4d061eff21f4 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -79,7 +79,7 @@ function cleanup cleanup_filesystem $TESTPOOL1 $TESTFS - destroy_pool $TESTPOOL1 + destroy_pool $TESTPOOL1 [[ -d $ALTER_ROOT ]] && \ log_must rm -rf $ALTER_ROOT @@ -91,7 +91,7 @@ log_assert "Verify that an exported pool can be imported." setup_filesystem "$DEVICE_FILES" $TESTPOOL1 $TESTFS $TESTDIR1 -checksum1=$(sum $MYTESTFILE | awk '{print $1}') +read -r checksum1 _ < <(cksum $MYTESTFILE) typeset -i i=0 typeset -i j=0 @@ -125,9 +125,8 @@ while (( i < ${#pools[*]} )); do [[ ! -e $basedir/$TESTFILE0 ]] && \ log_fail "$basedir/$TESTFILE0 missing after import." - checksum2=$(sum $basedir/$TESTFILE0 | awk '{print $1}') - [[ "$checksum1" != "$checksum2" ]] && \ - log_fail "Checksums differ ($checksum1 != $checksum2)" + read -r checksum2 _ < <(cksum $basedir/$TESTFILE0) + log_must [ "$checksum1" = "$checksum2" ] ((j = j + 1)) done diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_002_pos.ksh index 898f93cc9062..d677f8fb4916 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -90,7 +90,7 @@ log_assert "Verify that an exported pool cannot be imported more than once." setup_filesystem "$DEVICE_FILES" $TESTPOOL1 $TESTFS $TESTDIR1 -checksum1=$(sum $MYTESTFILE | awk '{print $1}') +read -r checksum1 _ < <(cksum $MYTESTFILE) typeset -i i=0 typeset -i j=0 @@ -126,9 +126,8 @@ while (( i < ${#pools[*]} )); do [[ ! -e $basedir/$TESTFILE0 ]] && \ log_fail "$basedir/$TESTFILE0 missing after import." - checksum2=$(sum $basedir/$TESTFILE0 | awk '{print $1}') - [[ "$checksum1" != "$checksum2" ]] && \ - log_fail "Checksums differ ($checksum1 != $checksum2)" + read -r checksum2 _ < <(cksum $basedir/$TESTFILE0) + log_must [ "$checksum1" = "$checksum2" ] log_mustnot zpool import ${devs[i]} $target diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_003_pos.ksh index dfda989deab9..56914dd4a0f9 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_004_pos.ksh index 657e28be2563..494542d59a99 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_005_pos.ksh index 79f026ede55f..b8805cd5803b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_006_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_006_pos.ksh index 2353bb07d040..d648b3262d8a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_007_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_007_pos.ksh index 928efebdd2d5..d7967eb5ef39 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_008_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_008_pos.ksh index f8da584aad1c..9489ce1460a4 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_009_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_009_neg.ksh index e7886ef3c859..be18907f6c12 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_009_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_009_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_010_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_010_pos.ksh index 212024dfcb34..107f0d88282d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_010_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_010_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_011_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_011_neg.ksh index b54168941722..19d4f56709da 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_011_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_011_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_012_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_012_pos.ksh index ec387b225665..bf499808eb3e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_012_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_012_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -165,10 +165,9 @@ for option in "" "-Df"; do fi log_note "Import with $nfs_flag and " \ "$guid_flag" - zpool import $option ${devs[i]} \ - ${options[j]} $target - #import by GUID if import by pool name fails - if [[ $? != 0 ]]; then + if ! zpool import $option ${devs[i]} \ + ${options[j]} $target; then + # import by GUID if import by pool name fails log_note "Possible pool name" \ "duplicates. Try GUID import" target=$guid diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_013_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_013_neg.ksh index 7fef6254fa68..956f851c536f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_013_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_013_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_014_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_014_pos.ksh index d70bced265f3..6e5e06515458 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_014_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_014_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_016_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_016_pos.ksh index 5434625cb985..7c1dd10c5c08 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_016_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_016_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_017_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_017_pos.ksh index 2e6cef265c4f..b8547852d01f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_017_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_017_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_all_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_all_001_pos.ksh index b384ec9b1fce..c16a37620a02 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_all_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_all_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -76,11 +76,11 @@ function cleanup_all # # Try import individually if 'import -a' failed. # - for pool in `zpool import | grep "pool:" | awk '{print $2}'`; do + for pool in $(zpool import | awk '/pool:/ {print $2}'); do zpool import -f $pool done - for pool in `zpool import -d $DEVICE_DIR | grep "pool:" | awk '{print $2}'`; do + for pool in $(zpool import -d $DEVICE_DIR | awk '/pool:/ {print $2}'); do log_must zpool import -d $DEVICE_DIR -f $pool done @@ -108,9 +108,8 @@ function checksum_all #alter_root [[ ! -e $file ]] && \ log_fail "$file missing after import." - checksum2=$(sum $file | awk '{print $1}') - [[ "$checksum1" != "$checksum2" ]] && \ - log_fail "Checksums differ ($checksum1 != $checksum2)" + read -r checksum2 _ < <(cksum $file) + log_must [ "$checksum1" = "$checksum2" ] (( id = id + 1 )) done @@ -122,7 +121,7 @@ function checksum_all #alter_root log_assert "Verify that 'zpool import -a' succeeds as root." log_onexit cleanup_all -checksum1=$(sum $MYTESTFILE | awk '{print $1}') +read -r checksum1 _ < <(cksum $MYTESTFILE) number=1 # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_errata3.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_errata3.ksh index 40b6ca1c1897..c5e578d79edb 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_errata3.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_errata3.ksh @@ -44,17 +44,15 @@ POOL_FILE=cryptv0.dat function uncompress_pool { log_note "Creating pool from $POOL_FILE" - log_must bzcat \ + log_must eval bzcat \ $STF_SUITE/tests/functional/cli_root/zpool_import/blockfiles/$POOL_FILE.bz2 \ - > /$TESTPOOL/$POOL_FILE - return 0 + "> /$TESTPOOL/$POOL_FILE" } function cleanup { poolexists $POOL_NAME && log_must zpool destroy $POOL_NAME - [[ -e /$TESTPOOL/$POOL_FILE ]] && rm /$TESTPOOL/$POOL_FILE - return 0 + log_must rm -f /$TESTPOOL/$POOL_FILE } log_onexit cleanup diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_errata4.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_errata4.ksh index a0f063a8dc83..e450d9a6222c 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_errata4.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_errata4.ksh @@ -43,18 +43,16 @@ POOL_FILE=missing_ivset.dat function uncompress_pool { log_note "Creating pool from $POOL_FILE" - log_must bzcat \ + log_must eval bzcat \ $STF_SUITE/tests/functional/cli_root/zpool_import/blockfiles/$POOL_FILE.bz2 \ - > /$TESTPOOL/$POOL_FILE - return 0 + "> /$TESTPOOL/$POOL_FILE" } function cleanup { log_must set_tunable32 DISABLE_IVSET_GUID_CHECK 0 poolexists $POOL_NAME && log_must zpool destroy $POOL_NAME - [[ -e /$TESTPOOL/$POOL_FILE ]] && rm /$TESTPOOL/$POOL_FILE - return 0 + log_must rm -rf /$TESTPOOL/$POOL_FILE } log_onexit cleanup @@ -65,11 +63,7 @@ function has_ivset_guid # dataset ds="$1" ivset_guid=$(get_prop ivsetguid $ds) - if [ "$ivset_guid" == "-" ]; then - return 1 - else - return 0 - fi + [ "$ivset_guid" != "-" ] } # 1. Import a pre-packaged pool with Errata #4 and verify its state @@ -78,9 +72,7 @@ log_must zpool import -d /$TESTPOOL/ $POOL_NAME log_must eval "zpool status $POOL_NAME | grep -q 'Errata #4'" log_must eval "zpool status $POOL_NAME | grep -q ZFS-8000-ER" bm2_value=$(zpool get -H -o value feature@bookmark_v2 $POOL_NAME) -if [ "$bm2_value" != "disabled" ]; then - log_fail "initial pool's bookmark_v2 feature is not disabled" -fi +log_must [ "$bm2_value" = "disabled" ] log_mustnot has_ivset_guid $POOL_NAME/testfs@snap1 log_mustnot has_ivset_guid $POOL_NAME/testfs@snap2 @@ -122,7 +114,7 @@ block_device_wait old_mntpnt=$(get_prop mountpoint $POOL_NAME/testfs) new_mntpnt=$(get_prop mountpoint $POOL_NAME/fixed/testfs) -log_must diff -r "$old_mntpnt" "$new_mntpnt" +log_must directory_diff "$old_mntpnt" "$new_mntpnt" log_must diff /dev/zvol/$POOL_NAME/testvol /dev/zvol/$POOL_NAME/fixed/testvol log_must has_ivset_guid $POOL_NAME/fixed/testfs@snap1 diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_features_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_features_001_pos.ksh index 973efb7acf3c..7c31229afcdc 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_features_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_features_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_features_002_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_features_002_neg.ksh index d16ef217a444..be4d41596211 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_features_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_features_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -69,18 +69,18 @@ done log_mustnot zpool import -d $DEVICE_DIR $TESTPOOL1 # error message should not mention "readonly" -log_mustnot eval "zpool import -d $DEVICE_DIR $TESTPOOL1 | grep readonly" +log_mustnot eval "zpool import -d $DEVICE_DIR $TESTPOOL1 | grep -q readonly" log_mustnot poolexists $TESTPOOL1 for feature in $active_features; do log_must eval "zpool import -d $DEVICE_DIR $TESTPOOL1 \ - | grep $feature" + | grep -q $feature" log_mustnot poolexists $TESTPOOL1 done for feature in $enabled_features; do log_mustnot eval "zpool import -d $DEVICE_DIR $TESTPOOL1 \ - | grep $feature" + | grep -q $feature" log_mustnot poolexists $TESTPOOL1 done diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_features_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_features_003_pos.ksh index e92c95f54c52..31828a47a8c1 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_features_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_features_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_missing_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_missing_001_pos.ksh index 3b5167ff0374..488024cfd0e1 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_missing_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_missing_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -111,7 +111,7 @@ log_assert "Verify that import could handle damaged or missing device." CWD=$PWD cd $DEVICE_DIR || log_fail "Unable change directory to $DEVICE_DIR" -checksum1=$(sum $MYTESTFILE | awk '{print $1}') +read -r checksum1 _ < <(cksum $MYTESTFILE) typeset -i i=0 typeset -i j=0 @@ -199,10 +199,8 @@ while (( i < ${#vdevs[*]} )); do [[ ! -e $basedir/$TESTFILE0 ]] && \ log_fail "$basedir/$TESTFILE0 missing after import." - checksum2=$(sum $basedir/$TESTFILE0 | awk '{print $1}') - [[ "$checksum1" != "$checksum2" ]] && \ - log_fail "Checksums differ ($checksum1 != $checksum2)" - + read -r checksum2 _ < <(cksum $basedir/$TESTFILE0) + log_must [ "$checksum1" = "$checksum2" ] done ((j = j + 1)) diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_missing_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_missing_002_pos.ksh index 60af3f321947..7f0cf79a4ad9 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_missing_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_missing_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_missing_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_missing_003_pos.ksh index 9d4629a77912..964b38f9e860 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_missing_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_missing_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -97,9 +97,8 @@ function verify [[ ! -e $mtpt/$file ]] && \ log_fail "$mtpt/$file missing after import." - checksum2=$(sum $mymtpt/$file | awk '{print $1}') - [[ "$checksum1" != "$checksum2" ]] && \ - log_fail "Checksums differ ($checksum1 != $checksum2)" + read -r checksum2 _ < <(cksum $mymtpt/$file) + log_must [ "$checksum1" = "$checksum2" ] return 0 @@ -107,7 +106,7 @@ function verify function cleanup { - cd $DEVICE_DIR || log_fail "Unable change directory to $DEVICE_DIR" + log_must cd $DEVICE_DIR for pool in $TESTPOOL1 $TESTPOOL2; do if poolexists "$pool" ; then @@ -136,7 +135,7 @@ function cleanup_all done log_must rm -f $DEVICE_DIR/$DEVICE_ARCHIVE - cd $CWD || log_fail "Unable change directory to $CWD" + log_must cd $CWD } @@ -146,10 +145,10 @@ log_assert "Verify that import could handle device overlapped." CWD=$PWD -cd $DEVICE_DIR || log_fail "Unable change directory to $DEVICE_DIR" +log_must cd $DEVICE_DIR log_must tar cf $DEVICE_DIR/$DEVICE_ARCHIVE ${DEVICE_FILE}* -checksum1=$(sum $MYTESTFILE | awk '{print $1}') +read -r checksum1 < <(cksum $MYTESTFILE) typeset -i i=0 typeset -i j=0 diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_rename_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_rename_001_pos.ksh index bb6bf86d7881..4d6005d2cb02 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_rename_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_rename_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -101,7 +101,7 @@ log_onexit cleanup log_assert "Verify that an imported pool can be renamed." setup_filesystem "$DEVICE_FILES" $TESTPOOL1 $TESTFS $TESTDIR1 -checksum1=$(sum $MYTESTFILE | awk '{print $1}') +read -r checksum1 _ < <(cksum $MYTESTFILE) typeset -i i=0 typeset -i j=0 @@ -140,9 +140,8 @@ while (( i < ${#pools[*]} )); do [[ ! -e $basedir/$TESTFILE0 ]] && \ log_fail "$basedir/$TESTFILE0 missing after import." - checksum2=$(sum $basedir/$TESTFILE0 | awk '{print $1}') - [[ "$checksum1" != "$checksum2" ]] && \ - log_fail "Checksums differ ($checksum1 != $checksum2)" + read -r checksum2 _ < <(cksum $basedir/$TESTFILE0) + log_must [ "$checksum1" = "$checksum2" ] log_must zpool export "${pools[i]}-new" diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/Makefile.am deleted file mode 100644 index 3968902ec36d..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/Makefile.am +++ /dev/null @@ -1,18 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_initialize -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - zpool_initialize_attach_detach_add_remove.ksh \ - zpool_initialize_fault_export_import_online.ksh \ - zpool_initialize_import_export.ksh \ - zpool_initialize_offline_export_import_online.ksh \ - zpool_initialize_online_offline.ksh \ - zpool_initialize_split.ksh \ - zpool_initialize_start_and_cancel_neg.ksh \ - zpool_initialize_start_and_cancel_pos.ksh \ - zpool_initialize_suspend_resume.ksh \ - zpool_initialize_unsupported_vdevs.ksh \ - zpool_initialize_verify_checksums.ksh \ - zpool_initialize_verify_initialized.ksh - -dist_pkgdata_DATA = \ - zpool_initialize.kshlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/cleanup.ksh index 3c7dbd31705d..92005b4efd7f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_attach_detach_add_remove.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_attach_detach_add_remove.ksh index 2a695025d214..56733758c633 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_attach_detach_add_remove.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_attach_detach_add_remove.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_import_export.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_import_export.ksh index 386d2a5dc237..c29c4b07e524 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_import_export.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_import_export.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_offline_export_import_online.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_offline_export_import_online.ksh index dedd466e4e6e..feca7f9af721 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_offline_export_import_online.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_offline_export_import_online.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_online_offline.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_online_offline.ksh index 55bd3188c966..79f47f0f8ba2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_online_offline.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_online_offline.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_split.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_split.ksh index 69b27c26c9bc..e5d528d6cef8 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_split.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_split.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_start_and_cancel_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_start_and_cancel_neg.ksh index 59b266d321c3..7de3f20223fd 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_start_and_cancel_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_start_and_cancel_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_start_and_cancel_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_start_and_cancel_pos.ksh index 5003b5f10bdb..facbd0efda59 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_start_and_cancel_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_start_and_cancel_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_suspend_resume.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_suspend_resume.ksh index bce3da5267cd..2f030c78a01b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_suspend_resume.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_suspend_resume.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_unsupported_vdevs.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_unsupported_vdevs.ksh index bd4ca069c49b..3055478841fc 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_unsupported_vdevs.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_unsupported_vdevs.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_verify_checksums.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_verify_checksums.ksh index 2ccc57b475b7..794d61c4e13f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_verify_checksums.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_verify_checksums.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_verify_initialized.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_verify_initialized.ksh index f774970a71be..dc44727d8757 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_verify_initialized.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_initialize/zpool_initialize_verify_initialized.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -65,10 +65,7 @@ log_must zpool export $TESTPOOL metaslabs=0 bs=512 zdb -p $TESTDIR -Pme $TESTPOOL | awk '/metaslab[ ]+[0-9]+/ { print $4, $8 }' | -while read -r offset_size; do - typeset offset=$(echo $offset_size | cut -d ' ' -f1) - typeset size=$(echo $offset_size | cut -d ' ' -f2) - +while read -r offset size; do log_note "offset: '$offset'" log_note "size: '$size'" @@ -79,7 +76,7 @@ while read -r offset_size; do # Note we use '-t x4' instead of '-t x8' here because x8 is not # a supported format on FreeBSD. dd if=$SMALLFILE skip=$((offset / bs)) count=$((size / bs)) bs=$bs | - od -t x4 -Ad | egrep -q "deadbeef +deadbeef +deadbeef +deadbeef" || + od -t x4 -Ad | grep -qE "deadbeef +deadbeef +deadbeef +deadbeef" || log_fail "Pattern not found in metaslab free space" done diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_labelclear/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_labelclear/Makefile.am deleted file mode 100644 index c258f0c929d4..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_labelclear/Makefile.am +++ /dev/null @@ -1,9 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_labelclear -dist_pkgdata_SCRIPTS = \ - zpool_labelclear_active.ksh \ - zpool_labelclear_exported.ksh \ - zpool_labelclear_removed.ksh \ - zpool_labelclear_valid.ksh - -dist_pkgdata_DATA = \ - labelclear.cfg diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_offline/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_offline/Makefile.am deleted file mode 100644 index 33fbb18d66f3..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_offline/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_offline -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_offline_001_pos.ksh \ - zpool_offline_002_neg.ksh \ - zpool_offline_003_pos.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_offline/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_offline/cleanup.ksh index 89c146249e71..3167a5097b5a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_offline/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_offline/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_offline/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_offline/setup.ksh index 2229f87e6208..23dc57ed3679 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_offline/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_offline/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_offline/zpool_offline_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_offline/zpool_offline_001_pos.ksh index 6f4c2e3182d1..23bb47545e81 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_offline/zpool_offline_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_offline/zpool_offline_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -55,10 +55,7 @@ function cleanup # for disk in $DISKLIST; do log_must zpool online $TESTPOOL $disk - check_state $TESTPOOL $disk "online" - if [[ $? != 0 ]]; then - log_fail "Unable to online $disk" - fi + log_must check_state $TESTPOOL $disk "online" done } @@ -79,16 +76,10 @@ for disk in $DISKLIST; do while [[ $i -lt ${#args[*]} ]]; do if (( j < num )) ; then log_must zpool offline ${args[$i]} $TESTPOOL $disk - check_state $TESTPOOL $disk "offline" - if [[ $? != 0 ]]; then - log_fail "$disk of $TESTPOOL did not match offline state" - fi + log_must check_state $TESTPOOL $disk "offline" else log_mustnot zpool offline ${args[$i]} $TESTPOOL $disk - check_state $TESTPOOL $disk "online" - if [[ $? != 0 ]]; then - log_fail "$disk of $TESTPOOL did not match online state" - fi + log_must check_state $TESTPOOL $disk "online" fi (( i = i + 1 )) @@ -106,19 +97,13 @@ for disk in $DISKLIST; do while [[ $i -lt $iters ]]; do index=`expr $RANDOM % ${#args[*]}` log_must zpool offline ${args[$index]} $TESTPOOL $disk - check_state $TESTPOOL $disk "offline" - if [[ $? != 0 ]]; then - log_fail "$disk of $TESTPOOL is not offline." - fi + log_must check_state $TESTPOOL $disk "offline" (( i = i + 1 )) done log_must zpool online $TESTPOOL $disk - check_state $TESTPOOL $disk "online" - if [[ $? != 0 ]]; then - log_fail "$disk of $TESTPOOL did not match online state" - fi + log_must check_state $TESTPOOL $disk "online" done log_pass "'zpool offline -f' succeeded" diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_offline/zpool_offline_002_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_offline/zpool_offline_002_neg.ksh index 14e0ab3fa75b..be31eb69c4f2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_offline/zpool_offline_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_offline/zpool_offline_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_offline/zpool_offline_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_offline/zpool_offline_003_pos.ksh index 7b5d21cba208..a881c7622de2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_offline/zpool_offline_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_offline/zpool_offline_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -54,10 +54,7 @@ function cleanup # Ensure we don't leave disks in the offline state for disk in $DISKLIST; do log_must zpool online $TESTPOOL $disk - check_state $TESTPOOL $disk "online" - if [[ $? != 0 ]]; then - log_fail "Unable to online $disk" - fi + log_must check_state $TESTPOOL $disk "online" done } diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_online/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_online/Makefile.am deleted file mode 100644 index 12681e3c4141..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_online/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_online -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_online_001_pos.ksh \ - zpool_online_002_neg.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_online/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_online/cleanup.ksh index 89c146249e71..3167a5097b5a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_online/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_online/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_online/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_online/setup.ksh index 2229f87e6208..23dc57ed3679 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_online/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_online/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_online/zpool_online_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_online/zpool_online_001_pos.ksh index 8489fddb4109..8cf64dfab713 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_online/zpool_online_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_online/zpool_online_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -54,10 +54,7 @@ function cleanup # for disk in $DISKLIST; do log_must zpool online $TESTPOOL $disk - check_state $TESTPOOL $disk "online" - if [[ $? != 0 ]]; then - log_fail "Unable to online $disk" - fi + log_must check_state $TESTPOOL $disk "online" done } @@ -79,16 +76,10 @@ for disk in $DISKLIST; do sync_pool $TESTPOOL log_must zpool offline $TESTPOOL $disk - check_state $TESTPOOL $disk "offline" - if [[ $? != 0 ]]; then - log_fail "$disk of $TESTPOOL did not match offline state" - fi + log_must check_state $TESTPOOL $disk "offline" log_must zpool online ${args[$i]} $TESTPOOL $disk - check_state $TESTPOOL $disk "online" - if [[ $? != 0 ]]; then - log_fail "$disk of $TESTPOOL did not match online state" - fi + log_must check_state $TESTPOOL $disk "online" while [[ $j -lt 20 ]]; do is_pool_resilvered $TESTPOOL && break @@ -112,10 +103,7 @@ for disk in $DISKLIST; do while [[ $i -lt $iters ]]; do index=`expr $RANDOM % ${#args[*]}` log_must zpool online ${args[$index]} $TESTPOOL $disk - check_state $TESTPOOL $disk "online" - if [[ $? != 0 ]]; then - log_fail "$disk of $TESTPOOL did not match online state" - fi + log_must check_state $TESTPOOL $disk "online" (( i = i + 1 )) done diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_online/zpool_online_002_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_online/zpool_online_002_neg.ksh index 79855e64a809..a0e7f21072fe 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_online/zpool_online_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_online/zpool_online_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_remove/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_remove/Makefile.am deleted file mode 100644 index e30e3f60c2d6..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_remove/Makefile.am +++ /dev/null @@ -1,10 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_remove -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_remove_001_neg.ksh \ - zpool_remove_002_pos.ksh \ - zpool_remove_003_pos.ksh - -dist_pkgdata_DATA = \ - zpool_remove.cfg diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_remove/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_remove/cleanup.ksh index 307fae7196d2..af286ff9047f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_remove/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_remove/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_remove/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_remove/setup.ksh index 8ce094bcc701..405f6fa1da8e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_remove/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_remove/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove.cfg b/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove.cfg index 1b8312e99372..8baaeb0b3299 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove_001_neg.ksh index 0c098a61e238..e87336ed00b3 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove_002_pos.ksh index 4ab7ac659b83..ddaed845e0db 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove_003_pos.ksh index 4e132d9d00b5..dc1745bf130e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_remove/zpool_remove_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_reopen/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_reopen/Makefile.am deleted file mode 100644 index 01ad68c817f2..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_reopen/Makefile.am +++ /dev/null @@ -1,15 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_reopen -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_reopen_001_pos.ksh \ - zpool_reopen_002_pos.ksh \ - zpool_reopen_003_pos.ksh \ - zpool_reopen_004_pos.ksh \ - zpool_reopen_005_pos.ksh \ - zpool_reopen_006_neg.ksh \ - zpool_reopen_007_pos.ksh - -dist_pkgdata_DATA = \ - zpool_reopen.cfg \ - zpool_reopen.shlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_reopen/zpool_reopen.cfg b/tests/zfs-tests/tests/functional/cli_root/zpool_reopen/zpool_reopen.cfg index 7451ffd8c53f..35beb568cb40 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_reopen/zpool_reopen.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_reopen/zpool_reopen.cfg @@ -18,7 +18,7 @@ verify_runnable "global" -export DISK_ARRAY_NUM=$(echo ${DISKS} | nawk '{print NF}') +export DISK_ARRAY_NUM=$(echo ${DISKS} | awk '{print NF}') export DISKSARRAY=$DISKS export SMALL_FILE_SIZE=10 export LARGE_FILE_SIZE=80 @@ -29,9 +29,8 @@ export SDHOSTS=1 export SDTGTS=1 export SDLUNS=1 -export DISK1=$(echo $DISKS | nawk '{print $1}') -export DISK2=$(echo $DISKS | nawk '{print $2}') -export DISK3=$(echo $DISKS | nawk '{print $3}') +read -r DISK1 DISK2 DISK3 _ <<<"$DISKS" +export DISK1 DISK2 DISK3 if is_linux; then set_slice_prefix diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_reopen/zpool_reopen.shlib b/tests/zfs-tests/tests/functional/cli_root/zpool_reopen/zpool_reopen.shlib index 3d142fdf70ca..9b3bc1432270 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_reopen/zpool_reopen.shlib +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_reopen/zpool_reopen.shlib @@ -20,7 +20,7 @@ function clear_labels #disks { for disk in $@; do - if ( is_loop_device $disk ) || ( is_mpath_device $disk ); then + if is_loop_device $disk || is_mpath_device $disk; then zpool labelclear -f /dev/$disk else zpool labelclear -f /dev/${disk}1 @@ -90,19 +90,16 @@ function wait_for_action #pool timeout function function wait_for_resilver_start #pool timeout { wait_for_action $1 $2 is_pool_resilvering - return $? } function wait_for_resilver_end #pool timeout { wait_for_action $1 $2 is_pool_resilvered - return $? } function wait_for_scrub_end #pool timeout { wait_for_action $1 $2 is_pool_scrubbed - return $? } # @@ -111,14 +108,10 @@ function wait_for_scrub_end #pool timeout function is_scan_restarted #pool { - typeset pool=$1 - zpool history -i $pool | grep -q "scan aborted, restarting" - return $? + zpool history -i $1 | grep -q "scan aborted, restarting" } function is_deferred_scan_started #pool { - typeset pool=$1 - zpool history -i $pool | grep -q "starting deferred resilver" - return $? + zpool history -i $1 | grep -q "starting deferred resilver" } diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_replace/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_replace/Makefile.am deleted file mode 100644 index 2e3ea69f2ca9..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_replace/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_replace -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_replace_001_neg.ksh \ - replace-o_ashift.ksh \ - replace_prop_ashift.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_replace/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_replace/cleanup.ksh index 89c146249e71..3167a5097b5a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_replace/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_replace/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_replace/replace-o_ashift.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_replace/replace-o_ashift.ksh index 1b18b1297a78..7610f2855c03 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_replace/replace-o_ashift.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_replace/replace-o_ashift.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -69,23 +69,13 @@ do for cmdval in ${ashifts[@]} do log_must zpool create -o ashift=$ashift $TESTPOOL1 $disk1 - verify_ashift $disk1 $ashift - if [[ $? -ne 0 ]] - then - log_fail "Pool was created without setting ashift " \ - "value to $ashift" - fi + log_must verify_ashift $disk1 $ashift # ashift_of(replacing_disk) <= ashift_of(existing_vdev) if [[ $cmdval -le $ashift ]] then log_must zpool replace -o ashift=$cmdval $TESTPOOL1 \ $disk1 $disk2 - verify_ashift $disk2 $ashift - if [[ $? -ne 0 ]] - then - log_fail "Device was replaced without " \ - "setting ashift value to $ashift" - fi + log_must verify_ashift $disk2 $ashift wait_replacing $TESTPOOL1 else log_mustnot zpool replace -o ashift=$cmdval $TESTPOOL1 \ diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_replace/replace_prop_ashift.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_replace/replace_prop_ashift.ksh index f076f26818eb..313b388b2ba4 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_replace/replace_prop_ashift.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_replace/replace_prop_ashift.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -77,12 +77,7 @@ do then log_must zpool replace $TESTPOOL1 $disk1 $disk2 wait_replacing $TESTPOOL1 - verify_ashift $disk2 $ashift - if [[ $? -ne 0 ]] - then - log_fail "Device was replaced without " \ - "setting ashift value to $ashift" - fi + log_must verify_ashift $disk2 $ashift else # cannot replace if pool prop ashift > vdev ashift log_mustnot zpool replace $TESTPOOL1 $disk1 $disk2 @@ -90,12 +85,7 @@ do log_must zpool replace -o ashift=$ashift $TESTPOOL1 \ $disk1 $disk2 wait_replacing $TESTPOOL1 - verify_ashift $disk2 $ashift - if [[ $? -ne 0 ]] - then - log_fail "Device was replaced without " \ - "setting ashift value to $ashift" - fi + log_must verify_ashift $disk2 $ashift fi # clean things for the next run log_must zpool destroy $TESTPOOL1 diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_replace/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_replace/setup.ksh index 2229f87e6208..23dc57ed3679 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_replace/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_replace/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_replace/zpool_replace_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_replace/zpool_replace_001_neg.ksh index 725e495ebb89..6049c1a3697c 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_replace/zpool_replace_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_replace/zpool_replace_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/Makefile.am deleted file mode 100644 index 2cec5335fbae..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/Makefile.am +++ /dev/null @@ -1,9 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_resilver -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_resilver_bad_args.ksh \ - zpool_resilver_restart.ksh - -dist_pkgdata_DATA = \ - zpool_resilver.cfg diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/cleanup.ksh index c74e23919cd1..066eb36b6f36 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/setup.ksh index 48ceecdf9eb7..02ab31958a15 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/zpool_resilver.cfg b/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/zpool_resilver.cfg index 2a942d69f6d0..16984432ee27 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/zpool_resilver.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/zpool_resilver.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -23,8 +23,7 @@ # Copyright (c) 2018 by Datto. All rights reserved. # -export DISK1=$(echo $DISKS | nawk '{print $1}') -export DISK2=$(echo $DISKS | nawk '{print $2}') -export DISK3=$(echo $DISKS | nawk '{print $3}') +read -r DISK1 DISK2 DISK3 _ <<<"$DISKS" +export DISK1 DISK2 DISK3 export MAXTIMEOUT=300 diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/zpool_resilver_bad_args.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/zpool_resilver_bad_args.ksh index abd514086630..3f4115aff089 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/zpool_resilver_bad_args.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/zpool_resilver_bad_args.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/zpool_resilver_restart.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/zpool_resilver_restart.ksh index e487afd8ae4d..8b5b5975774d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/zpool_resilver_restart.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_resilver/zpool_resilver_restart.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/Makefile.am deleted file mode 100644 index e2dfd9d64c40..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/Makefile.am +++ /dev/null @@ -1,16 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_scrub -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_scrub_001_neg.ksh \ - zpool_scrub_002_pos.ksh \ - zpool_scrub_003_pos.ksh \ - zpool_scrub_004_pos.ksh \ - zpool_scrub_005_pos.ksh \ - zpool_scrub_encrypted_unloaded.ksh \ - zpool_scrub_offline_device.ksh \ - zpool_scrub_print_repairing.ksh \ - zpool_scrub_multiple_copies.ksh - -dist_pkgdata_DATA = \ - zpool_scrub.cfg diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/cleanup.ksh index 03eb9901cb3e..c725664380f2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/setup.ksh index 936fd798e9dd..a0652219ae7d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub.cfg b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub.cfg index fdf2f428477f..d95c1decd5de 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -28,8 +28,8 @@ # Copyright (c) 2012, 2016 by Delphix. All rights reserved. # -export DISK1=${DISKS%% *} -export DISK2=$(echo $DISKS | awk '{print $2}') +read -r DISK1 DISK2 _ <<<"$DISKS" +export DISK1 DISK2 export ZFS_SCAN_VDEV_LIMIT_SLOW=$((128*1024)) export ZFS_SCAN_VDEV_LIMIT_DEFAULT=$((4*1024*1024)) diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_001_neg.ksh index 96bc185dd40c..0368c5809995 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_002_pos.ksh index 116d622960f9..e11825f91589 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_003_pos.ksh index 12dc044e9e08..1e1412ee0da9 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_004_pos.ksh index a7ae7f16b1af..846473be9736 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_005_pos.ksh index 69a33983d379..62d529486509 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_offline_device.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_offline_device.ksh index 7a07e643343d..815f26179132 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_offline_device.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_offline_device.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_print_repairing.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_print_repairing.ksh index 4b51cd9625ec..be06d6147af7 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_print_repairing.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_scrub/zpool_scrub_print_repairing.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_set/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_set/Makefile.am deleted file mode 100644 index 916e8bb8d20e..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_set/Makefile.am +++ /dev/null @@ -1,9 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_set -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_set_001_pos.ksh \ - zpool_set_002_neg.ksh \ - zpool_set_003_neg.ksh \ - zpool_set_ashift.ksh \ - zpool_set_features.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_set/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_set/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_set/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_set/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_set/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_set/setup.ksh index 6a9af3bc28c3..4c719075a741 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_set/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_set/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_001_pos.ksh index f08fdfab7fd6..3eb1411c8c86 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -51,7 +51,7 @@ then log_fail "\"zpool set\" exit status $RET should be equal to 2." fi -OUTPUT=$(zpool set 2>&1 | grep -i usage) +zpool set 2>&1 | grep -qi usage if [ $? != 0 ] then log_fail "Usage message for zpool set did not contain the word 'usage'." diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_002_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_002_neg.ksh index 34d7fd821464..ab592d433a80 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_003_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_003_neg.ksh index 1d637b77af00..7e93c8725355 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_003_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_003_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_ashift.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_ashift.ksh index 09b5f50d5e18..58119e37cc67 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_ashift.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_ashift.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_split/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_split/Makefile.am deleted file mode 100644 index aac5e0d6e7b1..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_split/Makefile.am +++ /dev/null @@ -1,19 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_split - -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - zpool_split_cliargs.ksh \ - zpool_split_devices.ksh \ - zpool_split_encryption.ksh \ - zpool_split_props.ksh \ - zpool_split_vdevs.ksh \ - zpool_split_resilver.ksh \ - zpool_split_wholedisk.ksh \ - zpool_split_indirect.ksh \ - zpool_split_dryrun_output.ksh - -dist_pkgdata_DATA = \ - zpool_split.cfg diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split.cfg b/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split.cfg index 5833a420c87c..a441a7dff0d6 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split.cfg @@ -14,5 +14,5 @@ # export DISKSARRAY=$DISKS -export DISK_ARRAY_NUM=$(echo ${DISKS} | nawk '{print NF}') +export DISK_ARRAY_NUM=$(echo ${DISKS} | awk '{print NF}') set_device_dir diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_devices.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_devices.ksh index d64c30d5c561..83b12ab784cd 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_devices.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_devices.ksh @@ -33,7 +33,7 @@ function cleanup { destroy_pool $TESTPOOL destroy_pool $TESTPOOL2 - rm -f $FILEDEV_PREFIX* + rm -fd $FILEDEV_PREFIX* $altroot } function setup_mirror # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_dryrun_output.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_dryrun_output.ksh index 2267ea7bd895..410b1fe7a03e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_dryrun_output.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_dryrun_output.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_indirect.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_indirect.ksh index 13f0d08b7f20..e6e5d245793f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_indirect.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_indirect.ksh @@ -43,7 +43,7 @@ function cleanup if poolexists $TESTPOOL2 ; then destroy_pool $TESTPOOL2 fi - rm -f $VDEV_TEMP $VDEV_M1 $VDEV_M2 + rm -fd $VDEV_TEMP $VDEV_M1 $VDEV_M2 $altroot } log_onexit cleanup @@ -64,6 +64,6 @@ log_must zpool remove $TESTPOOL $VDEV_TEMP log_must wait_for_removal $TESTPOOL log_must zpool split -R $altroot $TESTPOOL $TESTPOOL2 log_must poolexists $TESTPOOL2 -log_must test "$(get_pool_prop 'altroot' $TESTPOOL2)" == "$altroot" +log_must test "$(get_pool_prop 'altroot' $TESTPOOL2)" = "$altroot" log_pass "'zpool split' works on pools with indirect VDEVs." diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_props.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_props.ksh index 1aff8d31d91c..39ae1c9d56f9 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_props.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_props.ksh @@ -71,12 +71,11 @@ fi # Verify we can set a combination of valid property values on the new pool for prop in "${good_props[@]}" do - propname="$(awk -F= '{print $1}' <<< $prop)" - propval="$(awk -F= '{print $2}' <<< $prop)" + IFS='=' read -r propname propval <<<"$prop" setup_mirror log_must zpool split -o $prop $TESTPOOL $TESTPOOL2 log_must zpool import -N -d $TEST_BASE_DIR $TESTPOOL2 - log_must test "$(get_pool_prop $propname $TESTPOOL2)" == "$propval" + log_must test "$(get_pool_prop $propname $TESTPOOL2)" = "$propval" destroy_pool $TESTPOOL destroy_pool $TESTPOOL2 diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_vdevs.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_vdevs.ksh index 9866cf7a5a58..fff98cf83e84 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_vdevs.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_vdevs.ksh @@ -33,7 +33,7 @@ function cleanup { destroy_pool $TESTPOOL destroy_pool $TESTPOOL2 - rm -f $FILEDEV_PREFIX* + rm -fd $FILEDEV_PREFIX* $altroot } # @@ -122,7 +122,7 @@ typeset altroot="$TESTDIR/altroot-$TESTPOOL2" for config in "${goodconfs[@]}" do create_config="${config%% *}" - add_config="$(awk '{$1= "";print $0}' <<< $config)" + add_config="$(awk '{$1=""; print $0}' <<< $config)" log_must zpool create $TESTPOOL $(pool_config $create_config) for vdev in $add_config; do log_must zpool add -f $TESTPOOL $(pool_config $vdev) @@ -137,7 +137,7 @@ done for config in "${badconfs[@]}" do create_config="${config%% *}" - add_config="$(awk '{$1= "";print $0}' <<< $config)" + add_config="$(awk '{$1=""; print $0}' <<< $config)" log_must zpool create $TESTPOOL $(pool_config $create_config) for vdev in $add_config; do log_must zpool add -f $TESTPOOL $(pool_config $vdev) diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_wholedisk.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_wholedisk.ksh index 085856c428ad..0725b77a3db2 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_wholedisk.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_split/zpool_split_wholedisk.ksh @@ -44,7 +44,7 @@ function cleanup destroy_pool $TESTPOOL destroy_pool $TESTPOOL2 unload_scsi_debug - rm -f "$FILE_DEVICE" + rm -fd "$FILE_DEVICE" "$ALTROOT" } function setup_mirror diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_status/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_status/Makefile.am deleted file mode 100644 index 5553061c67b3..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_status/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_status -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_status_001_pos.ksh \ - zpool_status_002_pos.ksh \ - zpool_status_features_001_pos.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_status/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_status/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_status/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_status/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_status/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_status/setup.ksh index 6a9af3bc28c3..4c719075a741 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_status/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_status/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_001_pos.ksh index 88c0601032a5..076a08802fbe 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_002_pos.ksh index e2751b112597..3bdd7db649f9 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_003_pos.ksh new file mode 100755 index 000000000000..231c46d4fddf --- /dev/null +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_003_pos.ksh @@ -0,0 +1,70 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright (c) 2019, Delphix. All rights reserved. +# Copyright (c) 2021, George Amanakis. All rights reserved. +# + +. $STF_SUITE/include/libtest.shlib + +# +# DESCRIPTION: +# Verify correct output with 'zpool status -v' after corrupting a file +# +# STRATEGY: +# 1. Create a pool and a file +# 2. zinject checksum errors +# 3. Read the file +# 4. Take a snapshot and make a clone +# 5. Verify we see "snapshot, clone and filesystem" output in 'zpool status -v' + +function cleanup +{ + log_must zinject -c all + datasetexists $TESTPOOL2 && log_must zpool destroy $TESTPOOL2 + rm -f $TESTDIR/vdev_a +} + +verify_runnable "both" + +log_assert "Verify correct 'zpool status -v' output with a corrupted file" +log_onexit cleanup + +truncate -s $MINVDEVSIZE $TESTDIR/vdev_a +log_must zpool create -f $TESTPOOL2 $TESTDIR/vdev_a + +log_must fio --rw=write --name=job --size=10M --filename=/$TESTPOOL2/10m_file +log_must zinject -t data -e checksum -f 100 -am /$TESTPOOL2/10m_file + +# Try to read the 2nd megabyte of 10m_file +dd if=/$TESTPOOL2/10m_file bs=1M || true + +log_must zfs snapshot $TESTPOOL2@snap +log_must zfs clone $TESTPOOL2@snap $TESTPOOL2/clone + +# Look to see that snapshot, clone and filesystem our files report errors +log_must eval "zpool status -v | grep '$TESTPOOL2@snap:/10m_file'" +log_must eval "zpool status -v | grep '$TESTPOOL2/clone/10m_file'" +log_must eval "zpool status -v | grep '$TESTPOOL2/10m_file'" + +log_pass "'zpool status -v' outputs affected filesystem, snapshot & clone" diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_004_pos.ksh new file mode 100755 index 000000000000..1ac3e03b710d --- /dev/null +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_004_pos.ksh @@ -0,0 +1,81 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright (c) 2019, by Delphix. All rights reserved. +# Copyright (c) 2021, George Amanakis. All rights reserved. +# + +. $STF_SUITE/include/libtest.shlib + +# +# DESCRIPTION: +# Verify feature@head_errlog=disabled works. +# +# STRATEGY: +# 1. Create a pool with feature@head_errlog=disabled and a file +# 2. zinject checksum errors +# 3. Read the file +# 4. Take a snapshot and make a clone +# 5. Verify that zpool status displays the old behaviour. + +function cleanup +{ + log_must zinject -c all + datasetexists $TESTPOOL2 && log_must zpool destroy $TESTPOOL2 + rm -f $TESTDIR/vdev_a +} + +verify_runnable "both" + +log_assert "Verify 'zpool status -v' with feature@head_errlog=disabled works" +log_onexit cleanup + +truncate -s $MINVDEVSIZE $TESTDIR/vdev_a +log_must zpool create -f -o feature@head_errlog=disabled $TESTPOOL2 $TESTDIR/vdev_a + +state=$(zpool list -Ho feature@head_errlog $TESTPOOL2) +if [[ "$state" != "disabled" ]]; then + log_fail "head_errlog has state $state" +fi + +log_must fio --rw=write --name=job --size=10M --filename=/$TESTPOOL2/10m_file +log_must zinject -t data -e checksum -f 100 -am /$TESTPOOL2/10m_file + +# Try to read the file +dd if=/$TESTPOOL2/10m_file bs=1M || true + +log_must zfs snapshot $TESTPOOL2@snap +log_must zfs clone $TESTPOOL2@snap $TESTPOOL2/clone + +# Check that snapshot and clone do not report the error. +log_mustnot eval "zpool status -v | grep '$TESTPOOL2@snap:/10m_file'" +log_mustnot eval "zpool status -v | grep '$TESTPOOL2/clone/10m_file'" +log_must eval "zpool status -v | grep '$TESTPOOL2/10m_file'" + +# Check that enabling the feature reports the error properly. +log_must zpool set feature@head_errlog=enabled $TESTPOOL2 +log_must eval "zpool status -v | grep '$TESTPOOL2@snap:/10m_file'" +log_must eval "zpool status -v | grep '$TESTPOOL2/clone/10m_file'" +log_must eval "zpool status -v | grep '$TESTPOOL2/10m_file'" + +log_pass "'zpool status -v' with feature@head_errlog=disabled works" diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_features_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_features_001_pos.ksh index 635125fc0d1e..f5c68aca53e0 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_features_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_status/zpool_status_features_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_sync/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_sync/Makefile.am deleted file mode 100644 index 7f5f67d1f343..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_sync/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_sync -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - zpool_sync_001_pos.ksh \ - zpool_sync_002_neg.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_sync/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_sync/cleanup.ksh index 89c146249e71..3167a5097b5a 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_sync/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_sync/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_sync/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_sync/setup.ksh index 181e62b113c1..3d866cfd9f20 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_sync/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_sync/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_trim/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_trim/Makefile.am deleted file mode 100644 index 0411ab4e0070..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_trim/Makefile.am +++ /dev/null @@ -1,25 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_trim -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_trim_attach_detach_add_remove.ksh \ - zpool_trim_fault_export_import_online.ksh \ - zpool_trim_import_export.ksh \ - zpool_trim_multiple.ksh \ - zpool_trim_neg.ksh \ - zpool_trim_offline_export_import_online.ksh \ - zpool_trim_online_offline.ksh \ - zpool_trim_partial.ksh \ - zpool_trim_rate.ksh \ - zpool_trim_rate_neg.ksh \ - zpool_trim_secure.ksh \ - zpool_trim_split.ksh \ - zpool_trim_start_and_cancel_neg.ksh \ - zpool_trim_start_and_cancel_pos.ksh \ - zpool_trim_suspend_resume.ksh \ - zpool_trim_unsupported_vdevs.ksh \ - zpool_trim_verify_checksums.ksh \ - zpool_trim_verify_trimmed.ksh - -dist_pkgdata_DATA = \ - zpool_trim.kshlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_trim/zpool_trim_partial.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_trim/zpool_trim_partial.ksh index f3b3b0f272a7..bdbf3db53336 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_trim/zpool_trim_partial.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_trim/zpool_trim_partial.ksh @@ -64,7 +64,7 @@ log_must set_tunable64 TRIM_EXTENT_BYTES_MIN 4096 log_must mkdir "$TESTDIR" log_must truncate -s $LARGESIZE "$LARGEFILE" -log_must zpool create $TESTPOOL "$LARGEFILE" +log_must zpool create -O compression=off $TESTPOOL "$LARGEFILE" log_must mkfile $(( floor(LARGESIZE * 0.80) )) /$TESTPOOL/file sync_all_pools diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/Makefile.am deleted file mode 100644 index c7f321a2f61d..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/Makefile.am +++ /dev/null @@ -1,20 +0,0 @@ -SUBDIRS = blockfiles - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_upgrade -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_upgrade_001_pos.ksh \ - zpool_upgrade_002_pos.ksh \ - zpool_upgrade_003_pos.ksh \ - zpool_upgrade_004_pos.ksh \ - zpool_upgrade_005_neg.ksh \ - zpool_upgrade_006_neg.ksh \ - zpool_upgrade_007_pos.ksh \ - zpool_upgrade_008_pos.ksh \ - zpool_upgrade_009_neg.ksh \ - zpool_upgrade_features_001_pos.ksh - -dist_pkgdata_DATA = \ - zpool_upgrade.cfg \ - zpool_upgrade.kshlib diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/blockfiles/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/blockfiles/Makefile.am deleted file mode 100644 index ce8da278e9f0..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/blockfiles/Makefile.am +++ /dev/null @@ -1,54 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_upgrade/blockfiles -dist_pkgdata_DATA = \ - zfs-broken-mirror1.dat.bz2 \ - zfs-broken-mirror2.dat.bz2 \ - zfs-pool-v1.dat.bz2 \ - zfs-pool-v10.dat.bz2 \ - zfs-pool-v11.dat.bz2 \ - zfs-pool-v12.dat.bz2 \ - zfs-pool-v13.dat.bz2 \ - zfs-pool-v14.dat.bz2 \ - zfs-pool-v15.dat.bz2 \ - zfs-pool-v1mirror1.dat.bz2 \ - zfs-pool-v1mirror2.dat.bz2 \ - zfs-pool-v1mirror3.dat.bz2 \ - zfs-pool-v1raidz1.dat.bz2 \ - zfs-pool-v1raidz2.dat.bz2 \ - zfs-pool-v1raidz3.dat.bz2 \ - zfs-pool-v1stripe1.dat.bz2 \ - zfs-pool-v1stripe2.dat.bz2 \ - zfs-pool-v1stripe3.dat.bz2 \ - zfs-pool-v2.dat.bz2 \ - zfs-pool-v2mirror1.dat.bz2 \ - zfs-pool-v2mirror2.dat.bz2 \ - zfs-pool-v2mirror3.dat.bz2 \ - zfs-pool-v2raidz1.dat.bz2 \ - zfs-pool-v2raidz2.dat.bz2 \ - zfs-pool-v2raidz3.dat.bz2 \ - zfs-pool-v2stripe1.dat.bz2 \ - zfs-pool-v2stripe2.dat.bz2 \ - zfs-pool-v2stripe3.dat.bz2 \ - zfs-pool-v3.dat.bz2 \ - zfs-pool-v3hotspare1.dat.bz2 \ - zfs-pool-v3hotspare2.dat.bz2 \ - zfs-pool-v3hotspare3.dat.bz2 \ - zfs-pool-v3mirror1.dat.bz2 \ - zfs-pool-v3mirror2.dat.bz2 \ - zfs-pool-v3mirror3.dat.bz2 \ - zfs-pool-v3raidz1.dat.bz2 \ - zfs-pool-v3raidz2.dat.bz2 \ - zfs-pool-v3raidz21.dat.bz2 \ - zfs-pool-v3raidz22.dat.bz2 \ - zfs-pool-v3raidz23.dat.bz2 \ - zfs-pool-v3raidz3.dat.bz2 \ - zfs-pool-v3stripe1.dat.bz2 \ - zfs-pool-v3stripe2.dat.bz2 \ - zfs-pool-v3stripe3.dat.bz2 \ - zfs-pool-v4.dat.bz2 \ - zfs-pool-v5.dat.bz2 \ - zfs-pool-v6.dat.bz2 \ - zfs-pool-v7.dat.bz2 \ - zfs-pool-v8.dat.bz2 \ - zfs-pool-v9.dat.bz2 \ - zfs-pool-v999.dat.bz2 \ - zfs-pool-vBROKEN.dat.bz2 diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/cleanup.ksh index 1fab5305adc0..3b6f28dd7970 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/setup.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/setup.ksh index 860ba0f061a6..94301e865bc8 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade.cfg b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade.cfg index 993fafc0324d..977aca621278 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade.cfg +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade.kshlib b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade.kshlib index 783ae54e717b..0889946e8f27 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -132,8 +132,7 @@ function check_poolversion fi # check version using zpool upgrade - actual=$(zpool upgrade | grep $pool$ | \ - awk '{print $1}' | sed -e 's/ //g') + actual=$(zpool upgrade | awk -v p="$pool$" '$0 ~ p {gsub(/ /, "", $1); print $1}') if [[ $actual != $vers ]] ; then log_fail "$pool: zpool reported version $actual, expected $vers" fi diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_001_pos.ksh index b49c19c5551c..bd44db9eb6be 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_002_pos.ksh index 16ab14907055..8e5f3dd3a9b1 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_003_pos.ksh index 0ac3470dedfe..559fdf2b0980 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_004_pos.ksh index 0e278c81ccb4..bbadfc1f1d1d 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_005_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_005_neg.ksh index b294af06cf46..46c93119561f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_005_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_005_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_006_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_006_neg.ksh index 4c16f81082ca..c8280086e2e7 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_006_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_006_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_007_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_007_pos.ksh index 696c8c66cc1c..018d9c1739fe 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_008_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_008_pos.ksh index d930919652bf..aae7120bd69f 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -71,7 +71,7 @@ for ver_old in $VERSIONS; do typeset -i ver_new=$(random_int_between $ver_old $MAX_VER) create_old_pool $ver_old - log_must zpool upgrade -V $ver_new $pool_name > /dev/null + log_must eval 'zpool upgrade -V $ver_new $pool_name > /dev/null' check_poolversion $pool_name $ver_new destroy_upgraded_pool $ver_old done diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_009_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_009_neg.ksh index 22ffda2114a5..f32eac17bce9 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_009_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_009_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_features_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_features_001_pos.ksh index 5170d31b46da..448de3e4041e 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_features_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/zpool_upgrade_features_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_wait/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_wait/Makefile.am deleted file mode 100644 index 45ab8e3d4f18..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_wait/Makefile.am +++ /dev/null @@ -1,22 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_wait -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_wait_discard.ksh \ - zpool_wait_freeing.ksh \ - zpool_wait_initialize_basic.ksh \ - zpool_wait_initialize_cancel.ksh \ - zpool_wait_initialize_flag.ksh \ - zpool_wait_multiple.ksh \ - zpool_wait_no_activity.ksh \ - zpool_wait_remove.ksh \ - zpool_wait_remove_cancel.ksh \ - zpool_wait_trim_basic.ksh \ - zpool_wait_trim_cancel.ksh \ - zpool_wait_trim_flag.ksh \ - zpool_wait_usage.ksh - -dist_pkgdata_DATA = \ - zpool_wait.kshlib - -SUBDIRS = scan diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_wait/scan/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zpool_wait/scan/Makefile.am deleted file mode 100644 index 451d83a79aa6..000000000000 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_wait/scan/Makefile.am +++ /dev/null @@ -1,11 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_wait/scan -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_wait_replace.ksh \ - zpool_wait_replace_cancel.ksh \ - zpool_wait_rebuild.ksh \ - zpool_wait_resilver.ksh \ - zpool_wait_scrub_basic.ksh \ - zpool_wait_scrub_cancel.ksh \ - zpool_wait_scrub_flag.ksh diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_wait/zpool_wait.kshlib b/tests/zfs-tests/tests/functional/cli_root/zpool_wait/zpool_wait.kshlib index b413f6e9f98d..ccb97914968a 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_wait/zpool_wait.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_wait/zpool_wait.kshlib @@ -120,5 +120,5 @@ function check_while_waiting # Whether any vdev in the given pool is initializing function is_vdev_initializing # pool { - zpool status -i "$1" | grep 'initialized, started' >/dev/null + zpool status -i "$1" | grep -q 'initialized, started' } diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_wait/zpool_wait_usage.ksh b/tests/zfs-tests/tests/functional/cli_root/zpool_wait/zpool_wait_usage.ksh index 2d6f897092fe..05cbfce5741c 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_wait/zpool_wait_usage.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_wait/zpool_wait_usage.ksh @@ -43,5 +43,7 @@ zpool wait -t scrub fakepool 2>&1 | grep -i 'no such pool' || \ log_fail "Error message did not contain phrase 'no such pool'." zpool wait -t foo $TESTPOOL 2>&1 | grep -i 'invalid activity' || \ log_fail "Error message did not contain phrase 'invalid activity'." +zpool wait -t scrub=getsubopt $TESTPOOL 2>&1 | grep -i 'invalid activity' || \ + log_fail "getsubopt(3) error message did not contain phrase 'invalid activity'." log_pass "'zpool wait' behaves sensibly when invoked incorrectly." diff --git a/tests/zfs-tests/tests/functional/cli_user/Makefile.am b/tests/zfs-tests/tests/functional/cli_user/Makefile.am deleted file mode 100644 index 119f8ee187f6..000000000000 --- a/tests/zfs-tests/tests/functional/cli_user/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -SUBDIRS = \ - misc \ - zfs_list \ - zpool_iostat \ - zpool_list \ - zpool_status diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/Makefile.am b/tests/zfs-tests/tests/functional/cli_user/misc/Makefile.am deleted file mode 100644 index 2d38e65777f7..000000000000 --- a/tests/zfs-tests/tests/functional/cli_user/misc/Makefile.am +++ /dev/null @@ -1,52 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_user/misc -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zdb_001_neg.ksh \ - zfs_001_neg.ksh \ - zfs_allow_001_neg.ksh \ - zfs_clone_001_neg.ksh \ - zfs_create_001_neg.ksh \ - zfs_destroy_001_neg.ksh \ - zfs_get_001_neg.ksh \ - zfs_inherit_001_neg.ksh \ - zfs_mount_001_neg.ksh \ - zfs_promote_001_neg.ksh \ - zfs_receive_001_neg.ksh \ - zfs_rename_001_neg.ksh \ - zfs_rollback_001_neg.ksh \ - zfs_send_001_neg.ksh \ - zfs_set_001_neg.ksh \ - zfs_share_001_neg.ksh \ - zfs_snapshot_001_neg.ksh \ - zfs_unallow_001_neg.ksh \ - zfs_unmount_001_neg.ksh \ - zfs_unshare_001_neg.ksh \ - zfs_upgrade_001_neg.ksh \ - zpool_001_neg.ksh \ - zpool_add_001_neg.ksh \ - zpool_attach_001_neg.ksh \ - zpool_clear_001_neg.ksh \ - zpool_create_001_neg.ksh \ - zpool_destroy_001_neg.ksh \ - zpool_detach_001_neg.ksh \ - zpool_export_001_neg.ksh \ - zpool_get_001_neg.ksh \ - zpool_history_001_neg.ksh \ - zpool_import_001_neg.ksh \ - zpool_import_002_neg.ksh \ - zpool_offline_001_neg.ksh \ - zpool_online_001_neg.ksh \ - zpool_remove_001_neg.ksh \ - zpool_replace_001_neg.ksh \ - zpool_scrub_001_neg.ksh \ - zpool_set_001_neg.ksh \ - zpool_status_001_neg.ksh \ - zpool_upgrade_001_neg.ksh \ - arcstat_001_pos.ksh \ - arc_summary_001_pos.ksh \ - arc_summary_002_neg.ksh \ - zpool_wait_privilege.ksh - -dist_pkgdata_DATA = \ - misc.cfg diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/arc_summary_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/arc_summary_001_pos.ksh index befbea986e1b..853d40e0355b 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/arc_summary_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/arc_summary_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -27,34 +27,15 @@ . $STF_SUITE/include/libtest.shlib -log_assert "arc_summary generates output and doesn't return an error code" +is_freebsd && ! python3 -c 'import sysctl' 2>/dev/null && log_unsupported "python3 sysctl module missing" -# Depending on which version of arc_summary is installed some command -# line options may not be available. The python3 version includes -# several additional flags. -python3 -V 2>&1 > /dev/null -if (( $? )); then - # Some systems have Python 3 installed, but only older versions - # that don't have the subprocess.run() functionality. We catch - # these with a separate test. Remove this when all systems have - # reached 3.5 or greater - VERSIONPYTEST=$(python3 -V) - if [[ ${VERSIONPYTEST:9:1} -lt 5 ]]; then - set -A args "" "-a" "-d" "-p 1" - else - set -A args "" "-a" "-d" "-p 1" "-g" "-s arc" "-r" - fi -else - set -A args "" "-a" "-d" "-p 1" -fi +log_assert "arc_summary generates output and doesn't return an error code" # Without this, the below checks aren't going to work the way we hope... set -o pipefail -typeset -i i=0 -while [[ $i -lt ${#args[*]} ]]; do - log_must eval "arc_summary ${args[i]} > /dev/null" - ((i = i + 1)) +for arg in "" "-a" "-d" "-p 1" "-g" "-s arc" "-r"; do + log_must eval "arc_summary $arg > /dev/null" done log_must eval "arc_summary | head > /dev/null" diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/arc_summary_002_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/arc_summary_002_neg.ksh index de747fba89d1..1edc9fa82047 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/arc_summary_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/arc_summary_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -27,11 +27,11 @@ . $STF_SUITE/include/libtest.shlib -typeset args=("-x" "-5" "-p 7" "--err" "-@") +is_freebsd && ! python3 -c 'import sysctl' 2>/dev/null && log_unsupported "python3 sysctl module missing" log_assert "arc_summary generates an error code with invalid options" -for arg in "${args[@]}"; do +for arg in "-x" "-5" "-p 7" "--err" "-@"; do log_mustnot eval "arc_summary $arg > /dev/null" done diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/arcstat_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/arcstat_001_pos.ksh index ab574731fed9..d48aae19c746 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/arcstat_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/arcstat_001_pos.ksh @@ -1,4 +1,4 @@ -#! /bin/ksh -p +#!/bin/ksh -p # # CDDL HEADER START # @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -27,6 +27,8 @@ . $STF_SUITE/include/libtest.shlib +is_freebsd && ! python3 -c 'import sysctl' 2>/dev/null && log_unsupported "python3 sysctl module missing" + set -A args "" "-s \",\"" "-x" "-v" \ "-f time,hit%,dh%,ph%,mh%" @@ -38,4 +40,3 @@ while [[ $i -lt ${#args[*]} ]]; do ((i = i + 1)) done log_pass "arcstat generates output and doesn't return an error code" - diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/cleanup.ksh index e3dc8c179c0a..bbc2565f3b36 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/misc.cfg b/tests/zfs-tests/tests/functional/cli_user/misc/misc.cfg index 1a96ff5d93fa..e98b5e8b2214 100644 --- a/tests/zfs-tests/tests/functional/cli_user/misc/misc.cfg +++ b/tests/zfs-tests/tests/functional/cli_user/misc/misc.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/setup.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/setup.ksh index fc0ebde10025..9e49c7188ce5 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zdb_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zdb_001_neg.ksh index 3adfc59f51ce..ee889dfc2682 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zdb_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zdb_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -44,19 +44,13 @@ function check_zdb { - $@ > $TEST_BASE_DIR/zdb.$$ - grep "Dataset mos" $TEST_BASE_DIR/zdb.$$ - if [ $? -eq 0 ] - then - log_fail "$@ exited 0 when run as a non root user!" - fi - rm $TEST_BASE_DIR/zdb.$$ + log_mustnot eval "$* | grep -q 'Dataset mos'" } function cleanup { - rm -f $TEST_BASE_DIR/zdb_001_neg.$$.txt $TEST_BASE_DIR/zdb.$$ + rm -f $TEST_BASE_DIR/zdb_001_neg.$$.txt } verify_runnable "global" @@ -66,7 +60,7 @@ log_onexit cleanup log_must eval "zdb > $TEST_BASE_DIR/zdb_001_neg.$$.txt" # verify the output looks okay -log_must grep pool_guid $TEST_BASE_DIR/zdb_001_neg.$$.txt +log_must grep -q pool_guid $TEST_BASE_DIR/zdb_001_neg.$$.txt log_must rm $TEST_BASE_DIR/zdb_001_neg.$$.txt # we shouldn't able to run it on any dataset diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_001_neg.ksh index bfe8cf4bb29a..a70dc47fcc95 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -55,6 +55,6 @@ TEMPFILE="$TEST_BASE_DIR/zfs_001_neg.$$.txt" zfs > $TEMPFILE 2>&1 log_must grep "usage: zfs command args" "$TEMPFILE" -log_must eval "awk '{if (length(\$0) > 80) exit 1}' < $TEMPFILE" +log_must awk 'length($0) > 80 {print; ++err} END {exit err}' $TEMPFILE log_pass "zfs shows a usage message when run as a user" diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_allow_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_allow_001_neg.ksh index 56a74e4ae427..74dc1956105d 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_allow_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_allow_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -44,22 +44,15 @@ # # -# check to see if we have zfs allow -zfs 2>&1 | grep "allow" > /dev/null -if (($? != 0)) then - log_unsupported "ZFS allow not supported on this machine." -fi - log_assert "zfs allow returns an error when run as a user" log_must zfs allow $TESTPOOL/$TESTFS -log_mustnot zfs allow $(logname) create $TESTPOOL/$TESTFS +log_mustnot zfs allow $(id -un) create $TESTPOOL/$TESTFS # now verify that the above command actually did nothing by # checking for any allow output. ( if no allows are granted, # nothing should be output ) -OUTPUT=$(zfs allow $TESTPOOL/$TESTFS | grep "Local+Descendent" ) -if [ -n "$OUTPUT" ] +if zfs allow $TESTPOOL/$TESTFS | grep -q "Local+Descendent" then log_fail "zfs allow permissions were granted on $TESTPOOL/$TESTFS" fi diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_clone_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_clone_001_neg.ksh index 7497553ed8a5..4cb615b07cd2 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_clone_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_clone_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_create_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_create_001_neg.ksh index aa74bcf40c1f..c77641ad8d7a 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_create_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_create_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_destroy_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_destroy_001_neg.ksh index fabba4f13e9c..c1bdf6a83cea 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_destroy_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_destroy_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_get_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_get_001_neg.ksh index 0657b9143670..352dcae6301b 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_get_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_get_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_inherit_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_inherit_001_neg.ksh index 284c9f76dff1..c52c7bbfd170 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_inherit_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_inherit_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_mount_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_mount_001_neg.ksh index 8e4c39bfce75..b5779ec8b56a 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_mount_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_mount_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_promote_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_promote_001_neg.ksh index dd92f534d3a1..64cbbd57cc58 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_promote_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_promote_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_receive_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_receive_001_neg.ksh index 38c11a051028..3eec6306a60d 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_receive_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_receive_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_rename_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_rename_001_neg.ksh index ac275b01e5f9..a7c0d78e4a8f 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_rename_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_rename_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_rollback_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_rollback_001_neg.ksh index 51b5162decfc..bda0702c7581 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_rollback_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_rollback_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_send_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_send_001_neg.ksh index 6ecb66a3584c..9bdcd519716a 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_send_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_send_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_set_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_set_001_neg.ksh index 9cfb79210b11..1ec198415705 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_set_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_set_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_share_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_share_001_neg.ksh index 14c35b3da664..4ef9ac28af51 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_share_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_share_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -41,7 +41,6 @@ # 1. Attempt to share a dataset # 2. Verify the dataset was not shared. # -# verify_runnable "global" @@ -51,17 +50,11 @@ fi log_assert "zfs share returns an error when run as a user" -if is_shared $TESTDIR/unshared -then - log_fail "$TESTPOOL/$TESTFS/unshared was incorrectly shared initially!" -fi +log_mustnot is_shared $TESTDIR/unshared log_mustnot zfs share $TESTPOOL/$TESTFS/unshared # Now verify that the above command didn't actually do anything -if is_shared $TESTDIR/unshared -then - log_fail "$TESTPOOL/$TESTFS/unshared was actually shared!" -fi +log_mustnot is_shared $TESTDIR/unshared log_pass "zfs share returns an error when run as a user" diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_snapshot_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_snapshot_001_neg.ksh index 232d9574b51c..8c770ae4cc22 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_snapshot_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_snapshot_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_unallow_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_unallow_001_neg.ksh index 98383a08762e..e382141393c1 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_unallow_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_unallow_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -43,19 +43,12 @@ # # -# check to see if we have zfs unallow -zfs 2>&1 | grep "unallow" > /dev/null -if (($? != 0)) then - log_unsupported "ZFS unallow not supported on this machine." -fi - log_assert "zfs unallow returns an error when run as a user" log_mustnot zfs unallow everyone $TESTPOOL/$TESTFS/allowed # now check with zfs allow to see if the permissions are still there -OUTPUT=$(zfs allow $TESTPOOL/$TESTFS/allowed | grep "Local+Descendent" ) -if [ -z "$OUTPUT" ] +if ! zfs allow $TESTPOOL/$TESTFS/allowed | grep -q "Local+Descendent" then log_fail "Error - create permissions were unallowed on \ $TESTPOOL/$TESTFS/allowed" diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_unmount_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_unmount_001_neg.ksh index 5b0a773f15a9..d421643314cf 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_unmount_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_unmount_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_unshare_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_unshare_001_neg.ksh index 7ae86fc4ec76..f6f5b13556c9 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_unshare_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_unshare_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -52,17 +52,13 @@ fi log_assert "zfs unshare returns an error when run as a user" # verify that the filesystem was shared initially -if not_shared $TESTDIR/shared -then - log_fail "$TESTPOOL/$TESTFS/shared was not shared initially at all!" -fi +log_mustnot not_shared $TESTDIR/shared +log_fail "$TESTPOOL/$TESTFS/shared was not shared initially at all!" log_mustnot zfs unshare $TESTPOOL/$TESTFS/shared # now verify that the above command didn't do anything -if not_shared $TESTDIR/shared -then - log_fail "$TESTPOOL/$TESTFS/shared was actually unshared!" -fi +log_mustnot not_shared $TESTDIR/shared +log_fail "$TESTPOOL/$TESTFS/shared was actually unshared!" log_pass "zfs unshare returns an error when run as a user" diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_upgrade_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_upgrade_001_neg.ksh index fd65feb8c001..d03907293501 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zfs_upgrade_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zfs_upgrade_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_001_neg.ksh index 0fddc08b25db..13c4ad0f1362 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -59,6 +59,6 @@ log_assert "zpool shows a usage message when run as a user" eval "zpool > $TEMPFILE 2>&1" log_must grep "usage: zpool command args" "$TEMPFILE" -log_must eval "awk '{if (length(\$0) > 80) exit 1}' < $TEMPFILE" +log_must awk '{if (length($0) > 80) exit 1}' $TEMPFILE log_pass "zpool shows a usage message when run as a user" diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_add_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_add_001_neg.ksh index 7bcd911543ed..b68a3eae9920 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_add_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_add_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_attach_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_attach_001_neg.ksh index b0fadca8c9fe..959f2aea991d 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_attach_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_attach_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_clear_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_clear_001_neg.ksh index 5d6ef5ff3e4d..f73169ac5b75 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_clear_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_clear_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_create_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_create_001_neg.ksh index 77d08ee39009..15a44abd4dda 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_create_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_create_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_destroy_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_destroy_001_neg.ksh index acc93fe74826..fabac4dbcddb 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_destroy_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_destroy_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_detach_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_detach_001_neg.ksh index dbc540cebbe7..35f200148fc8 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_detach_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_detach_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_export_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_export_001_neg.ksh index 9bca4668d3d1..9f5c7c224f07 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_export_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_export_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_get_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_get_001_neg.ksh index 7415cebf236b..d3179be3970d 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_get_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_get_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -54,12 +54,12 @@ while [[ $i -lt ${#args[*]} ]] do PROP=${props[$i]} EXPECTED=${prop_vals[$i]} - ACTUAL=$( zpool get $PROP $TESTPOOL | grep $PROP | awk '{print $1}' ) + ACTUAL=$( zpool get $PROP $TESTPOOL | awk -v p=$PROP '$0 ~ p {print $1}' ) if [ "$ACTUAL" != "$EXPECTED" ] then log_fail "Property $PROP value was $ACTUAL, expected $EXPECTED" fi - i=$(( $i + 1 )) + i=$(( $i + 1 )) done log_must zpool get all $TESTPOOL diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_history_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_history_001_neg.ksh index 47082e3524e3..916905e24ec1 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_history_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_history_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_import_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_import_001_neg.ksh index 09fd775ff4cc..66dd71507abd 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_import_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_import_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_import_002_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_import_002_neg.ksh index f3460186725c..3eb091ef2573 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_import_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_import_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_offline_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_offline_001_neg.ksh index 523883c6f0dd..1f8b5dc94c9e 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_offline_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_offline_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_online_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_online_001_neg.ksh index cd290515357f..e3c0699e096c 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_online_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_online_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_remove_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_remove_001_neg.ksh index 1168e6bdac9f..91632c539e66 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_remove_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_remove_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_replace_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_replace_001_neg.ksh index 6211e6575540..fb6f5847c0fd 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_replace_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_replace_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_scrub_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_scrub_001_neg.ksh index bf9ee4cfadb0..730c18e085ba 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_scrub_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_scrub_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_set_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_set_001_neg.ksh index 941e20c0010e..8c89379f5690 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_set_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_set_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -59,7 +59,7 @@ do log_mustnot $POOL set $PROP=$NEW $TESTPOOL # Now verify that the above command did nothing - ACTUAL=$( zpool get $PROP $TESTPOOL | grep $PROP | awk '{print $1}' ) + ACTUAL=$( zpool get $PROP $TESTPOOL | awk -v p=$PROP '$0 ~ p {print $1}' ) if [ "$ACTUAL" != "$EXPECTED" ] then log_fail "Property $PROP was set to $ACTUAL, expected $EXPECTED" diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_status_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_status_001_neg.ksh index 9cd811a5487c..a5a44be812f0 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_status_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_status_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_upgrade_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_upgrade_001_neg.ksh index aa380010574f..eebe13ad30b3 100755 --- a/tests/zfs-tests/tests/functional/cli_user/misc/zpool_upgrade_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/misc/zpool_upgrade_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zfs_list/Makefile.am b/tests/zfs-tests/tests/functional/cli_user/zfs_list/Makefile.am deleted file mode 100644 index 8393072af225..000000000000 --- a/tests/zfs-tests/tests/functional/cli_user/zfs_list/Makefile.am +++ /dev/null @@ -1,14 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_user/zfs_list -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zfs_list_001_pos.ksh \ - zfs_list_002_pos.ksh \ - zfs_list_003_pos.ksh \ - zfs_list_004_neg.ksh \ - zfs_list_007_pos.ksh \ - zfs_list_008_neg.ksh - -dist_pkgdata_DATA = \ - zfs_list.cfg \ - zfs_list.kshlib diff --git a/tests/zfs-tests/tests/functional/cli_user/zfs_list/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_user/zfs_list/cleanup.ksh index 115e00c47be5..635042bc11bc 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zfs_list/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zfs_list/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zfs_list/setup.ksh b/tests/zfs-tests/tests/functional/cli_user/zfs_list/setup.ksh index b4c588282ff7..0196b795395d 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zfs_list/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zfs_list/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list.cfg b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list.cfg index 59f367cfa544..de343510186d 100644 --- a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list.cfg +++ b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list.kshlib b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list.kshlib index 889ae46fb932..4c0d9b5b4606 100644 --- a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list.kshlib +++ b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -120,7 +120,7 @@ function verify_reverse_sort { # command list name function is_fs_type_zfs { typeset dirname=$1 - typeset fs="$(df $dirname | tail -1 | awk '{print $NF}')" + typeset fs="$(df $dirname | awk 'END {print $NF}')" if is_freebsd; then fs_type=$(mount | awk -v fs=$fs '{if ($3 == fs) print $4}' \ diff --git a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_001_pos.ksh index 5d995a79ba33..318b42e86bdf 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_002_pos.ksh index 4951097aca51..cc0659f0761e 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_003_pos.ksh index 43cfd0cf2101..896f629a260a 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -66,10 +66,7 @@ cd /tmp for path in $TESTPOOL/$TESTFS $TESTDIR ./../$TESTDIR ; do zfs list -rH -o name $path > $tmpfile for fs in $children ; do - grep "^${fs}$" $tmpfile > /dev/null 2>&1 - if (( $? != 0 )); then - log_fail "$fs not shown in the output list." - fi + log_must grep -qxF "$fs" $tmpfile done done diff --git a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_004_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_004_neg.ksh index 727c0aef0883..cc3f2084a8b0 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_004_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_004_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_005_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_005_neg.ksh new file mode 100755 index 000000000000..ae374a6f6c14 --- /dev/null +++ b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_005_neg.ksh @@ -0,0 +1,14 @@ +#!/bin/ksh -p +# SPDX-License-Identifier: 0BSD + +. $STF_SUITE/include/libtest.shlib + +# +# DESCRIPTION: +# zfs list -t used to accept getsubopt(3)-style filesystem=whatever; +# it doesn't anymore +# + +log_mustnot zfs list -t filesystem=getsubopt + +log_pass "'zfs list -t' doesn't accept =getsubopt suffixes." diff --git a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_007_pos.ksh b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_007_pos.ksh index 8e9009bd5500..1049ba9dbe1b 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -37,7 +37,7 @@ # # STRATEGY: # 1. 'zfs list -d ' to get the output. -# 2. 'zfs list -r|egrep' to get the expected output. +# 2. 'zfs list -r|grep' to get the expected output. # 3. Compare the two outputs, they should be same. # @@ -50,8 +50,7 @@ fi function cleanup { - log_must rm -f $DEPTH_OUTPUT - log_must rm -f $EXPECT_OUTPUT + log_must rm -f $DEPTH_OUTPUT $EXPECT_OUTPUT } log_onexit cleanup @@ -76,10 +75,10 @@ for dp in ${depth_array[@]}; do log_must eval "zfs list -H -d $dp -o name -t ${fs_type[$fs]} $DEPTH_FS > $DEPTH_OUTPUT" [[ -s "$DEPTH_OUTPUT" ]] && \ log_fail "$DEPTH_OUTPUT should be null." - log_mustnot zfs list -rH -o name -t ${fs_type[$fs]} $DEPTH_FS | egrep -e '$eg_opt' + log_mustnot zfs list -rH -o name -t ${fs_type[$fs]} $DEPTH_FS | grep -E "$eg_opt" else log_must eval "zfs list -H -d $dp -o name -t ${fs_type[$fs]} $DEPTH_FS > $DEPTH_OUTPUT" - log_must eval "zfs list -rH -o name -t ${fs_type[$fs]} $DEPTH_FS | egrep -e '$eg_opt' > $EXPECT_OUTPUT" + log_must eval "zfs list -rH -o name -t ${fs_type[$fs]} $DEPTH_FS | grep -E '$eg_opt' > $EXPECT_OUTPUT" log_must diff $DEPTH_OUTPUT $EXPECT_OUTPUT fi (( fs+=1 )) diff --git a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_008_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_008_neg.ksh index 4f3504ac41ba..dda1c4f18ca0 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_008_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zfs_list/zfs_list_008_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/Makefile.am b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/Makefile.am deleted file mode 100644 index 5ee30eafc358..000000000000 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/Makefile.am +++ /dev/null @@ -1,12 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_user/zpool_iostat -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_iostat_001_neg.ksh \ - zpool_iostat_002_pos.ksh \ - zpool_iostat_003_neg.ksh \ - zpool_iostat_004_pos.ksh \ - zpool_iostat_005_pos.ksh \ - zpool_iostat_-c_disable.ksh \ - zpool_iostat_-c_searchpath.ksh \ - zpool_iostat_-c_homedir.ksh diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/setup.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/setup.ksh index 77eb6bd34f40..009d931c100d 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_-c_disable.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_-c_disable.ksh index 0d64eb0ec624..5fea262a4f61 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_-c_disable.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_-c_disable.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_-c_homedir.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_-c_homedir.ksh index 22450d89dfd2..60d350573a15 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_-c_homedir.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_-c_homedir.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_-c_searchpath.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_-c_searchpath.ksh index 11f51350af56..efd279faed1a 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_-c_searchpath.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_-c_searchpath.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_001_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_001_neg.ksh index d99d42984780..eeb2c47dcb27 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_002_pos.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_002_pos.ksh index 49be7d0701c2..72c5367522ff 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -62,7 +62,7 @@ fi zpool iostat $TESTPOOL 1 4 > $tmpfile 2>&1 & sleep 4 -stat_count=$(grep $TESTPOOL $tmpfile | wc -l) +stat_count=$(grep -c $TESTPOOL $tmpfile) if [[ $stat_count -ne 4 ]]; then log_fail "zpool iostat [pool_name] [interval] [count] failed" diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_003_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_003_neg.ksh index b6bcf71bffff..5f490b54a795 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_003_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_003_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_004_pos.ksh index a20321950113..3ff871b0d2d9 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_005_pos.ksh index 53652ec11b5a..10b4761fe233 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_iostat/zpool_iostat_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_list/Makefile.am b/tests/zfs-tests/tests/functional/cli_user/zpool_list/Makefile.am deleted file mode 100644 index de8cb366924d..000000000000 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_list/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_user/zpool_list -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_list_001_pos.ksh \ - zpool_list_002_neg.ksh diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_list/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_list/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_list/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_list/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_list/setup.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_list/setup.ksh index d275e063b13a..76aa23035dd9 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_list/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_list/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_list/zpool_list_001_pos.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_list/zpool_list_001_pos.ksh index 4c57413f1464..a54124a58e5c 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_list/zpool_list_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_list/zpool_list_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_list/zpool_list_002_neg.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_list/zpool_list_002_neg.ksh index 4ed38d475cd7..001c7292bed2 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_list/zpool_list_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_list/zpool_list_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_status/Makefile.am b/tests/zfs-tests/tests/functional/cli_user/zpool_status/Makefile.am deleted file mode 100644 index e1b339657749..000000000000 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_status/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_user/zpool_status -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - zpool_status_003_pos.ksh \ - zpool_status_-c_disable.ksh \ - zpool_status_-c_homedir.ksh \ - zpool_status_-c_searchpath.ksh diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_status/cleanup.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_status/cleanup.ksh index 79cd6e9f908e..17d29ee4f548 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_status/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_status/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_status/setup.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_status/setup.ksh index 6a9af3bc28c3..4c719075a741 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_status/setup.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_status/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_-c_disable.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_-c_disable.ksh index c8105fb4a50a..4cac07371660 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_-c_disable.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_-c_disable.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_-c_homedir.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_-c_homedir.ksh index 5363043a8307..a0d17ba7c94c 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_-c_homedir.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_-c_homedir.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_-c_searchpath.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_-c_searchpath.ksh index 3f64fdf1a708..e3c9187355ae 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_-c_searchpath.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_-c_searchpath.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_003_pos.ksh index fa7d3f3f2d56..69b5448b458a 100755 --- a/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/cli_user/zpool_status/zpool_status_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/compression/Makefile.am b/tests/zfs-tests/tests/functional/compression/Makefile.am deleted file mode 100644 index 817bd41e8075..000000000000 --- a/tests/zfs-tests/tests/functional/compression/Makefile.am +++ /dev/null @@ -1,17 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/compression -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - compress_001_pos.ksh \ - compress_002_pos.ksh \ - compress_003_pos.ksh \ - compress_004_pos.ksh \ - compress_zstd_bswap.ksh \ - l2arc_compressed_arc.ksh \ - l2arc_compressed_arc_disabled.ksh \ - l2arc_encrypted.ksh \ - l2arc_encrypted_no_compressed_arc.ksh - -dist_pkgdata_DATA = \ - compress.cfg \ - testpool_zstd.tar.gz diff --git a/tests/zfs-tests/tests/functional/compression/cleanup.ksh b/tests/zfs-tests/tests/functional/compression/cleanup.ksh index 0573003b6af5..f3647ea2bbee 100755 --- a/tests/zfs-tests/tests/functional/compression/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/compression/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/compression/compress.cfg b/tests/zfs-tests/tests/functional/compression/compress.cfg index b2373064505f..6c955807c482 100644 --- a/tests/zfs-tests/tests/functional/compression/compress.cfg +++ b/tests/zfs-tests/tests/functional/compression/compress.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/compression/compress_001_pos.ksh b/tests/zfs-tests/tests/functional/compression/compress_001_pos.ksh index fe3a3acacc04..cddd45a628f4 100755 --- a/tests/zfs-tests/tests/functional/compression/compress_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/compression/compress_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -60,8 +60,8 @@ log_must file_write -o $OP -f $TESTDIR/$TESTFILE1 -b $BLOCKSZ \ sleep 60 -FILE0_BLKS=`du -k $TESTDIR/$TESTFILE0 | awk '{ print $1}'` -FILE1_BLKS=`du -k $TESTDIR/$TESTFILE1 | awk '{ print $1}'` +FILE0_BLKS=`du -k $TESTDIR/$TESTFILE0 | awk '{print $1}'` +FILE1_BLKS=`du -k $TESTDIR/$TESTFILE1 | awk '{print $1}'` if [[ $FILE0_BLKS -le $FILE1_BLKS ]]; then log_fail "$TESTFILE0 is smaller than $TESTFILE1" \ diff --git a/tests/zfs-tests/tests/functional/compression/compress_002_pos.ksh b/tests/zfs-tests/tests/functional/compression/compress_002_pos.ksh index a07d70824042..1f7d234c10bf 100755 --- a/tests/zfs-tests/tests/functional/compression/compress_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/compression/compress_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -65,8 +65,8 @@ log_must file_write -o $OP -f $TESTDIR1/$TESTFILE1 -b $BLOCKSZ \ sleep 60 -FILE0_BLKS=`du -k $TESTDIR1/$TESTFILE0 | awk '{ print $1}'` -FILE1_BLKS=`du -k $TESTDIR1/$TESTFILE1 | awk '{ print $1}'` +FILE0_BLKS=`du -k $TESTDIR1/$TESTFILE0 | awk '{print $1}'` +FILE1_BLKS=`du -k $TESTDIR1/$TESTFILE1 | awk '{print $1}'` if [[ $FILE0_BLKS -le $FILE1_BLKS ]]; then log_fail "$TESTFILE0 is smaller than $TESTFILE1" \ diff --git a/tests/zfs-tests/tests/functional/compression/compress_003_pos.ksh b/tests/zfs-tests/tests/functional/compression/compress_003_pos.ksh index d5b7256b52d6..f8028ad218a1 100755 --- a/tests/zfs-tests/tests/functional/compression/compress_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/compression/compress_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/compression/compress_004_pos.ksh b/tests/zfs-tests/tests/functional/compression/compress_004_pos.ksh index b924bcd0ba62..ba851343712f 100755 --- a/tests/zfs-tests/tests/functional/compression/compress_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/compression/compress_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/compression/compress_zstd_bswap.ksh b/tests/zfs-tests/tests/functional/compression/compress_zstd_bswap.ksh index 9726cf0dd5a6..73428e8af510 100755 --- a/tests/zfs-tests/tests/functional/compression/compress_zstd_bswap.ksh +++ b/tests/zfs-tests/tests/functional/compression/compress_zstd_bswap.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/compression/l2arc_compressed_arc.ksh b/tests/zfs-tests/tests/functional/compression/l2arc_compressed_arc.ksh index 5980ce156934..57f6b6a0242b 100755 --- a/tests/zfs-tests/tests/functional/compression/l2arc_compressed_arc.ksh +++ b/tests/zfs-tests/tests/functional/compression/l2arc_compressed_arc.ksh @@ -52,6 +52,8 @@ export DIRECT=1 verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + log_assert "L2ARC with compressed_arc enabled succeeds." origin_carc_setting=$(get_tunable COMPRESSED_ARC_ENABLED) diff --git a/tests/zfs-tests/tests/functional/compression/l2arc_compressed_arc_disabled.ksh b/tests/zfs-tests/tests/functional/compression/l2arc_compressed_arc_disabled.ksh index 4c3b6a61c25f..c8f4111744eb 100755 --- a/tests/zfs-tests/tests/functional/compression/l2arc_compressed_arc_disabled.ksh +++ b/tests/zfs-tests/tests/functional/compression/l2arc_compressed_arc_disabled.ksh @@ -52,6 +52,8 @@ export DIRECT=1 verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + log_assert "L2ARC with compressed_arc disabled succeeds." origin_carc_setting=$(get_tunable COMPRESSED_ARC_ENABLED) diff --git a/tests/zfs-tests/tests/functional/compression/l2arc_encrypted.ksh b/tests/zfs-tests/tests/functional/compression/l2arc_encrypted.ksh index fb460daf6837..f7b8a4b950d5 100755 --- a/tests/zfs-tests/tests/functional/compression/l2arc_encrypted.ksh +++ b/tests/zfs-tests/tests/functional/compression/l2arc_encrypted.ksh @@ -53,6 +53,8 @@ export DIRECT=1 verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + log_assert "L2ARC with encryption enabled succeeds." origin_carc_setting=$(get_tunable COMPRESSED_ARC_ENABLED) diff --git a/tests/zfs-tests/tests/functional/compression/l2arc_encrypted_no_compressed_arc.ksh b/tests/zfs-tests/tests/functional/compression/l2arc_encrypted_no_compressed_arc.ksh index 45ef489c3145..0838b2c93e68 100755 --- a/tests/zfs-tests/tests/functional/compression/l2arc_encrypted_no_compressed_arc.ksh +++ b/tests/zfs-tests/tests/functional/compression/l2arc_encrypted_no_compressed_arc.ksh @@ -53,6 +53,8 @@ export DIRECT=1 verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + log_assert "L2ARC with compressed_arc disabled succeeds." origin_carc_setting=$(get_tunable COMPRESSED_ARC_ENABLED) diff --git a/tests/zfs-tests/tests/functional/compression/setup.ksh b/tests/zfs-tests/tests/functional/compression/setup.ksh index 677cb12d79ac..1eeefeaa18c1 100755 --- a/tests/zfs-tests/tests/functional/compression/setup.ksh +++ b/tests/zfs-tests/tests/functional/compression/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cp_files/.gitignore b/tests/zfs-tests/tests/functional/cp_files/.gitignore deleted file mode 100644 index eac05e155378..000000000000 --- a/tests/zfs-tests/tests/functional/cp_files/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/cp_files diff --git a/tests/zfs-tests/tests/functional/cp_files/Makefile.am b/tests/zfs-tests/tests/functional/cp_files/Makefile.am deleted file mode 100644 index 06c31f5f3f92..000000000000 --- a/tests/zfs-tests/tests/functional/cp_files/Makefile.am +++ /dev/null @@ -1,13 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cp_files - -dist_pkgdata_SCRIPTS = \ - cp_files_001_pos.ksh \ - cleanup.ksh \ - setup.ksh - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cp_files - -pkgexec_PROGRAMS = cp_files -cp_files_SOURCES= cp_files.c diff --git a/tests/zfs-tests/tests/functional/cp_files/cleanup.ksh b/tests/zfs-tests/tests/functional/cp_files/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/cp_files/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/cp_files/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/cp_files/cp_files_001_pos.ksh b/tests/zfs-tests/tests/functional/cp_files/cp_files_001_pos.ksh index 3e138cfc9f72..5a48f0fbc522 100755 --- a/tests/zfs-tests/tests/functional/cp_files/cp_files_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/cp_files/cp_files_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -55,10 +55,9 @@ log_onexit cleanup NR_FILES=60000 BATCH=1000 -log_must mkdir $TESTDIR/src -log_must mkdir $TESTDIR/dst +log_must mkdir $TESTDIR/src $TESTDIR/dst -WD=$(pwd) +WD=$PWD cd $TESTDIR/src # create NR_FILES in BATCH at a time to prevent overflowing argument buffer for i in $(seq $(($NR_FILES/$BATCH))); do touch $(seq $((($i-1)*$BATCH+1)) $(($i*$BATCH))); done @@ -67,7 +66,7 @@ cd $WD log_must test $NR_FILES -eq $(ls -U $TESTDIR/src | wc -l) # copy files from src to dst, use cp_files to make sure we copy in readdir order -log_must $STF_SUITE/tests/functional/cp_files/cp_files $TESTDIR/src $TESTDIR/dst +log_must cp_files $TESTDIR/src $TESTDIR/dst log_must test $NR_FILES -eq $(ls -U $TESTDIR/dst | wc -l) diff --git a/tests/zfs-tests/tests/functional/cp_files/setup.ksh b/tests/zfs-tests/tests/functional/cp_files/setup.ksh index fc5cec3063a6..b756d4e76c83 100755 --- a/tests/zfs-tests/tests/functional/cp_files/setup.ksh +++ b/tests/zfs-tests/tests/functional/cp_files/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/crtime/Makefile.am b/tests/zfs-tests/tests/functional/crtime/Makefile.am deleted file mode 100644 index 13e1c2dde31b..000000000000 --- a/tests/zfs-tests/tests/functional/crtime/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/crtime -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - crtime_001_pos.ksh diff --git a/tests/zfs-tests/tests/functional/crtime/cleanup.ksh b/tests/zfs-tests/tests/functional/crtime/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/crtime/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/crtime/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/crtime/crtime_001_pos.ksh b/tests/zfs-tests/tests/functional/crtime/crtime_001_pos.ksh index 4f9810553fa6..b2d0e7378652 100755 --- a/tests/zfs-tests/tests/functional/crtime/crtime_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/crtime/crtime_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/crtime/setup.ksh b/tests/zfs-tests/tests/functional/crtime/setup.ksh index fc5cec3063a6..b756d4e76c83 100755 --- a/tests/zfs-tests/tests/functional/crtime/setup.ksh +++ b/tests/zfs-tests/tests/functional/crtime/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/ctime/.gitignore b/tests/zfs-tests/tests/functional/ctime/.gitignore deleted file mode 100644 index 9e4539d5fee0..000000000000 --- a/tests/zfs-tests/tests/functional/ctime/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/ctime diff --git a/tests/zfs-tests/tests/functional/ctime/Makefile.am b/tests/zfs-tests/tests/functional/ctime/Makefile.am deleted file mode 100644 index e7479ae81056..000000000000 --- a/tests/zfs-tests/tests/functional/ctime/Makefile.am +++ /dev/null @@ -1,13 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/ctime - -dist_pkgdata_SCRIPTS = \ - ctime_001_pos.ksh \ - cleanup.ksh \ - setup.ksh - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/ctime - -pkgexec_PROGRAMS = ctime -ctime_SOURCES = ctime.c diff --git a/tests/zfs-tests/tests/functional/ctime/cleanup.ksh b/tests/zfs-tests/tests/functional/ctime/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/ctime/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/ctime/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/ctime/ctime_001_pos.ksh b/tests/zfs-tests/tests/functional/ctime/ctime_001_pos.ksh index de12efe46bc0..6791067ff72f 100755 --- a/tests/zfs-tests/tests/functional/ctime/ctime_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/ctime/ctime_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -38,7 +38,7 @@ log_note "Verify [acm]time is modified appropriately." for arg in ${args[*]}; do log_note "Testing with xattr set to $arg" log_must zfs set xattr=$arg $TESTPOOL - log_must $STF_SUITE/tests/functional/ctime/ctime + log_must ctime done log_pass "PASS" diff --git a/tests/zfs-tests/tests/functional/ctime/setup.ksh b/tests/zfs-tests/tests/functional/ctime/setup.ksh index fc5cec3063a6..b756d4e76c83 100755 --- a/tests/zfs-tests/tests/functional/ctime/setup.ksh +++ b/tests/zfs-tests/tests/functional/ctime/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/deadman/Makefile.am b/tests/zfs-tests/tests/functional/deadman/Makefile.am deleted file mode 100644 index 097f23e88404..000000000000 --- a/tests/zfs-tests/tests/functional/deadman/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/deadman -dist_pkgdata_SCRIPTS = \ - deadman_ratelimit.ksh \ - deadman_sync.ksh \ - deadman_zio.ksh - -dist_pkgdata_DATA = \ - deadman.cfg diff --git a/tests/zfs-tests/tests/functional/deadman/deadman.cfg b/tests/zfs-tests/tests/functional/deadman/deadman.cfg index e767f3dd5a98..96dc06b78675 100644 --- a/tests/zfs-tests/tests/functional/deadman/deadman.cfg +++ b/tests/zfs-tests/tests/functional/deadman/deadman.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/deadman/deadman_ratelimit.ksh b/tests/zfs-tests/tests/functional/deadman/deadman_ratelimit.ksh index 469117a56cc0..4dd4c5b9a76c 100755 --- a/tests/zfs-tests/tests/functional/deadman/deadman_ratelimit.ksh +++ b/tests/zfs-tests/tests/functional/deadman/deadman_ratelimit.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -64,7 +64,7 @@ log_must file_write -b 1048576 -c 8 -o create -d 0 -f $mntpnt/file log_must zpool export $TESTPOOL log_must zpool import $TESTPOOL log_must zinject -d $DISK1 -D 5:1 $TESTPOOL -log_must dd if=$mntpnt/file of=$TEST_BASE_DIR/devnull oflag=sync +log_must dd if=$mntpnt/file of=/dev/null oflag=sync events=$(zpool events $TESTPOOL | grep -c ereport.fs.zfs.deadman) log_note "events=$events" diff --git a/tests/zfs-tests/tests/functional/deadman/deadman_sync.ksh b/tests/zfs-tests/tests/functional/deadman/deadman_sync.ksh index 7b18ebdcb859..34a8821725a6 100755 --- a/tests/zfs-tests/tests/functional/deadman/deadman_sync.ksh +++ b/tests/zfs-tests/tests/functional/deadman/deadman_sync.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/deadman/deadman_zio.ksh b/tests/zfs-tests/tests/functional/deadman/deadman_zio.ksh index f0774c4b29d9..592399be0e80 100755 --- a/tests/zfs-tests/tests/functional/deadman/deadman_zio.ksh +++ b/tests/zfs-tests/tests/functional/deadman/deadman_zio.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/Makefile.am b/tests/zfs-tests/tests/functional/delegate/Makefile.am deleted file mode 100644 index c33da3374d88..000000000000 --- a/tests/zfs-tests/tests/functional/delegate/Makefile.am +++ /dev/null @@ -1,28 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/delegate -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - zfs_allow_001_pos.ksh \ - zfs_allow_002_pos.ksh \ - zfs_allow_003_pos.ksh \ - zfs_allow_004_pos.ksh \ - zfs_allow_005_pos.ksh \ - zfs_allow_006_pos.ksh \ - zfs_allow_007_pos.ksh \ - zfs_allow_008_pos.ksh \ - zfs_allow_009_neg.ksh \ - zfs_allow_010_pos.ksh \ - zfs_allow_011_neg.ksh \ - zfs_allow_012_neg.ksh \ - zfs_unallow_001_pos.ksh \ - zfs_unallow_002_pos.ksh \ - zfs_unallow_003_pos.ksh \ - zfs_unallow_004_pos.ksh \ - zfs_unallow_005_pos.ksh \ - zfs_unallow_006_pos.ksh \ - zfs_unallow_007_neg.ksh \ - zfs_unallow_008_neg.ksh - -dist_pkgdata_DATA = \ - delegate.cfg \ - delegate_common.kshlib diff --git a/tests/zfs-tests/tests/functional/delegate/cleanup.ksh b/tests/zfs-tests/tests/functional/delegate/cleanup.ksh index 1951c00e2cf3..173a3ef2e136 100755 --- a/tests/zfs-tests/tests/functional/delegate/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/delegate/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/delegate.cfg b/tests/zfs-tests/tests/functional/delegate/delegate.cfg index a9a46281b185..d78896fb0174 100644 --- a/tests/zfs-tests/tests/functional/delegate/delegate.cfg +++ b/tests/zfs-tests/tests/functional/delegate/delegate.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/delegate_common.kshlib b/tests/zfs-tests/tests/functional/delegate/delegate_common.kshlib index a78b390aa18e..3f2f089e8171 100644 --- a/tests/zfs-tests/tests/functional/delegate/delegate_common.kshlib +++ b/tests/zfs-tests/tests/functional/delegate/delegate_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -161,38 +161,29 @@ function common_perm typeset perm=$2 typeset dtst=$3 - typeset -i ret=1 case $perm in send) verify_send $user $perm $dtst - ret=$? ;; allow) verify_allow $user $perm $dtst - ret=$? ;; userprop) verify_userprop $user $perm $dtst - ret=$? ;; compression|checksum|readonly) verify_ccr $user $perm $dtst - ret=$? ;; copies) verify_copies $user $perm $dtst - ret=$? ;; reservation) verify_reservation $user $perm $dtst - ret=$? ;; *) - ret=1 + return 1 ;; esac - - return $ret } function check_fs_perm @@ -201,99 +192,74 @@ function check_fs_perm typeset perm=$2 typeset fs=$3 - typeset -i ret=1 case $perm in create) verify_fs_create $user $perm $fs - ret=$? ;; destroy) verify_fs_destroy $user $perm $fs - ret=$? ;; snapshot) verify_fs_snapshot $user $perm $fs - ret=$? ;; rollback) verify_fs_rollback $user $perm $fs - ret=$? ;; clone) verify_fs_clone $user $perm $fs - ret=$? ;; rename) verify_fs_rename $user $perm $fs - ret=$? ;; mount) verify_fs_mount $user $perm $fs - ret=$? ;; share) verify_fs_share $user $perm $fs - ret=$? ;; mountpoint) verify_fs_mountpoint $user $perm $fs - ret=$? ;; promote) verify_fs_promote $user $perm $fs - ret=$? ;; canmount) verify_fs_canmount $user $perm $fs - ret=$? ;; dnodesize) verify_fs_dnodesize $user $perm $fs - ret=$? ;; recordsize) verify_fs_recordsize $user $perm $fs - ret=$? ;; quota) verify_fs_quota $user $perm $fs - ret=$? ;; aclmode) verify_fs_aclmode $user $perm $fs - ret=$? ;; aclinherit) verify_fs_aclinherit $user $perm $fs - ret=$? ;; snapdir) verify_fs_snapdir $user $perm $fs - ret=$? ;; atime|exec|devices|setuid|xattr) verify_fs_aedsx $user $perm $fs - ret=$? ;; zoned) verify_fs_zoned $user $perm $fs - ret=$? ;; sharenfs) verify_fs_sharenfs $user $perm $fs - ret=$? ;; receive) verify_fs_receive $user $perm $fs - ret=$? ;; *) common_perm $user $perm $fs - ret=$? ;; esac - - return $ret } function check_vol_perm @@ -302,43 +268,32 @@ function check_vol_perm typeset perm=$2 typeset vol=$3 - typeset -i ret=1 case $perm in destroy) verify_vol_destroy $user $perm $vol - ret=$? ;; snapshot) verify_vol_snapshot $user $perm $vol - ret=$? ;; rollback) verify_vol_rollback $user $perm $vol - ret=$? ;; clone) verify_vol_clone $user $perm $vol - ret=$? ;; rename) verify_vol_rename $user $perm $vol - ret=$? ;; promote) verify_vol_promote $user $perm $vol - ret=$? ;; volsize) verify_vol_volsize $user $perm $vol - ret=$? ;; *) common_perm $user $perm $vol - ret=$? ;; esac - - return $ret } function setup_unallow_testenv @@ -362,8 +317,6 @@ function setup_unallow_testenv log_must verify_perm $SUBFS $LOCAL_DESC_SET $OTHER2 fi done - - return 0 } # @@ -391,12 +344,11 @@ function verify_send user_run $user eval "zfs send $snap > $bak_user" log_must eval "zfs send $snap > $bak_root" - if [[ $(checksum $bak_user) == $(checksum $bak_root) ]]; then + if [ "$(cksum < $bak_user)" = "$(cksum < $bak_root)" ]; then ret=0 fi - rm -rf $bak_user > /dev/null - rm -rf $bak_root > /dev/null + rm -rf $bak_user $bak_root return $ret } @@ -462,12 +414,11 @@ function verify_fs_receive log_must eval "zfs receive $dtst < $bak_root" log_must eval "zfs send $dtstsnap > $bak_root" log_must_busy zfs destroy -rf $dtst - if [[ $(checksum $bak_user) != $(checksum $bak_root) ]]; then + if [ "$(cksum < $bak_user)" != "$(cksum < $bak_root)" ]; then return 1 fi - rm -rf $bak_user > /dev/null - rm -rf $bak_root > /dev/null + rm -rf $bak_user $bak_root done @@ -1694,20 +1645,12 @@ function verify_allow typeset -i ret - user_run $user zfs allow $user allow $dtst - ret=$? - if [[ $ret -eq 0 ]]; then - return 1 - fi + user_run $user zfs allow $user allow $dtst && return 1 log_must zfs allow $user copies $dtst user_run $user zfs allow $user copies $dtst ret=$? log_must zfs unallow $user copies $dtst - if [[ $ret -eq 1 ]]; then - return 1 - fi - - return 0 + [ $ret -ne 1 ] } diff --git a/tests/zfs-tests/tests/functional/delegate/setup.ksh b/tests/zfs-tests/tests/functional/delegate/setup.ksh index 2f13da750436..dd2822442bd1 100755 --- a/tests/zfs-tests/tests/functional/delegate/setup.ksh +++ b/tests/zfs-tests/tests/functional/delegate/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -68,10 +68,8 @@ log_must add_user $OTHER_GROUP $OTHER2 # # chmod 0750 $HOME # -user_run $STAFF1 zfs list -if [ $? -ne 0 ]; then +user_run $STAFF1 zfs list || log_unsupported "Test user $STAFF1 cannot execute zfs utilities" -fi DISK=${DISKS%% *} diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_allow_001_pos.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_allow_001_pos.ksh index 1e0ed80d3203..4324e3f4cbf3 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_allow_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_allow_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -57,7 +57,7 @@ function cleanup restore_root_datasets } -log_assert "everyone' is interpreted as a keyword even if a user " \ +log_assert "'everyone' is interpreted as a keyword even if a user " \ "or group named 'everyone' exists." log_onexit cleanup @@ -79,9 +79,7 @@ if [[ $user_added == "TRUE" ]]; then fi log_note "Created a group called 'everyone'." -if ! cat /etc/group | awk -F: '{print $1}' | \ - grep -w 'everyone' > /dev/null 2>&1 -then +if ! grep -q '^everyone:' /etc/group; then group_added="TRUE" log_must add_group everyone fi diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_allow_002_pos.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_allow_002_pos.ksh index fc603eae19dc..9045f316ca81 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_allow_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_allow_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_allow_003_pos.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_allow_003_pos.ksh index d8eb58ae1d65..34a26cfc0ca4 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_allow_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_allow_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_allow_004_pos.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_allow_004_pos.ksh index 1462f5ebe984..f90cdd8aa5a9 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_allow_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_allow_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_allow_005_pos.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_allow_005_pos.ksh index 3feb598c4ced..d8e7b56021f7 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_allow_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_allow_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_allow_006_pos.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_allow_006_pos.ksh index 2fc05dee907a..a4e96adf088e 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_allow_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_allow_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_allow_007_pos.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_allow_007_pos.ksh index f3213254b97a..8855fa48e3a4 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_allow_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_allow_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_allow_008_pos.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_allow_008_pos.ksh index b0e1df32a64d..be187475e937 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_allow_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_allow_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_allow_009_neg.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_allow_009_neg.ksh index a6f12244ce2f..9b7995f651df 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_allow_009_neg.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_allow_009_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_allow_010_pos.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_allow_010_pos.ksh index 3a8ef5e6251b..549928697edd 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_allow_010_pos.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_allow_010_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_allow_011_neg.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_allow_011_neg.ksh index d4203a814341..44a98dde5ba2 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_allow_011_neg.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_allow_011_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_allow_012_neg.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_allow_012_neg.ksh index fd95db92e049..dac66c0318bd 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_allow_012_neg.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_allow_012_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_001_pos.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_001_pos.ksh index 178d250ca529..5d64880d030a 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_002_pos.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_002_pos.ksh index ea1de97614f3..471c5645bced 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_003_pos.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_003_pos.ksh index 35eb9a961ebc..98af7fba226f 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_004_pos.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_004_pos.ksh index d41d76bd0950..7e35d4cc38ec 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_005_pos.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_005_pos.ksh index a97f500ea98e..59bc6cc7d6de 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_006_pos.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_006_pos.ksh index 2bba43de9d14..5521eccebfd6 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_007_neg.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_007_neg.ksh index b1a221be0bd9..a25f60b41d0b 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_007_neg.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_007_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_008_neg.ksh b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_008_neg.ksh index a8bd5ebcb6b4..1cda9cef9fb1 100755 --- a/tests/zfs-tests/tests/functional/delegate/zfs_unallow_008_neg.ksh +++ b/tests/zfs-tests/tests/functional/delegate/zfs_unallow_008_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/devices/Makefile.am b/tests/zfs-tests/tests/functional/devices/Makefile.am deleted file mode 100644 index 42f59428d4ca..000000000000 --- a/tests/zfs-tests/tests/functional/devices/Makefile.am +++ /dev/null @@ -1,11 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/devices -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - devices_001_pos.ksh \ - devices_002_neg.ksh \ - devices_003_pos.ksh - -dist_pkgdata_DATA = \ - devices.cfg \ - devices_common.kshlib diff --git a/tests/zfs-tests/tests/functional/devices/cleanup.ksh b/tests/zfs-tests/tests/functional/devices/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/devices/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/devices/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/devices/devices.cfg b/tests/zfs-tests/tests/functional/devices/devices.cfg index ba3c36ae8c4a..6024834aa3f6 100644 --- a/tests/zfs-tests/tests/functional/devices/devices.cfg +++ b/tests/zfs-tests/tests/functional/devices/devices.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/devices/devices_001_pos.ksh b/tests/zfs-tests/tests/functional/devices/devices_001_pos.ksh index 2f2802bc65a3..d130a3d0362d 100755 --- a/tests/zfs-tests/tests/functional/devices/devices_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/devices/devices_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/devices/devices_002_neg.ksh b/tests/zfs-tests/tests/functional/devices/devices_002_neg.ksh index a768c4aa6b34..1cc7b2bd26eb 100755 --- a/tests/zfs-tests/tests/functional/devices/devices_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/devices/devices_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/devices/devices_003_pos.ksh b/tests/zfs-tests/tests/functional/devices/devices_003_pos.ksh index 1fcd67409c11..909ca9c80d4b 100755 --- a/tests/zfs-tests/tests/functional/devices/devices_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/devices/devices_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/devices/devices_common.kshlib b/tests/zfs-tests/tests/functional/devices/devices_common.kshlib index fa7fdbecf5fd..800c565bfe67 100644 --- a/tests/zfs-tests/tests/functional/devices/devices_common.kshlib +++ b/tests/zfs-tests/tests/functional/devices/devices_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/devices/setup.ksh b/tests/zfs-tests/tests/functional/devices/setup.ksh index ee6cf83acb9e..c464156ceb60 100755 --- a/tests/zfs-tests/tests/functional/devices/setup.ksh +++ b/tests/zfs-tests/tests/functional/devices/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/threadsappend/cleanup.ksh b/tests/zfs-tests/tests/functional/dos_attributes/cleanup.ksh similarity index 95% rename from tests/zfs-tests/tests/functional/threadsappend/cleanup.ksh rename to tests/zfs-tests/tests/functional/dos_attributes/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/threadsappend/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/dos_attributes/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/dos_attributes/read_dos_attrs_001.ksh b/tests/zfs-tests/tests/functional/dos_attributes/read_dos_attrs_001.ksh new file mode 100755 index 000000000000..2c99a4049c67 --- /dev/null +++ b/tests/zfs-tests/tests/functional/dos_attributes/read_dos_attrs_001.ksh @@ -0,0 +1,60 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright 2009 Sun Microsystems, Inc. All rights reserved. +# Use is subject to license terms. +# + +# +# Copyright (c) 2013, 2016 by Delphix. All rights reserved. +# + +. $STF_SUITE/include/libtest.shlib + +# +# DESCRIPTION: +# +# Read additional file level attributes stored in upper half of z_pflags +# +# STARTEGY: +# 1) Create a file +# 2) Execute read_dos_attributes on the file we created +# 3) Verify that read_dos_attributes exited successfully +# + +verify_runnable "global" + +FILETOTEST="$TESTDIR/test_read_dos_attrs.txt" + +function cleanup +{ + rm -f $FILETOTEST +} + +log_onexit cleanup + +log_must chmod 777 $TESTDIR +log_must eval "echo 'This is a test file.' > $FILETOTEST" +log_must read_dos_attributes $FILETOTEST + +log_pass "reading DOS attributes succeeded." diff --git a/tests/zfs-tests/tests/functional/threadsappend/setup.ksh b/tests/zfs-tests/tests/functional/dos_attributes/setup.ksh similarity index 93% rename from tests/zfs-tests/tests/functional/threadsappend/setup.ksh rename to tests/zfs-tests/tests/functional/dos_attributes/setup.ksh index 4fc55cd47803..b756d4e76c83 100755 --- a/tests/zfs-tests/tests/functional/threadsappend/setup.ksh +++ b/tests/zfs-tests/tests/functional/dos_attributes/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -32,5 +32,4 @@ . $STF_SUITE/include/libtest.shlib DISK=${DISKS%% *} - -default_setup ${DISK} +default_setup $DISK diff --git a/tests/zfs-tests/tests/functional/dos_attributes/write_dos_attrs_001.ksh b/tests/zfs-tests/tests/functional/dos_attributes/write_dos_attrs_001.ksh new file mode 100755 index 000000000000..4a7fb747f930 --- /dev/null +++ b/tests/zfs-tests/tests/functional/dos_attributes/write_dos_attrs_001.ksh @@ -0,0 +1,61 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright 2009 Sun Microsystems, Inc. All rights reserved. +# Use is subject to license terms. +# + +# +# Copyright (c) 2013, 2016 by Delphix. All rights reserved. +# + +. $STF_SUITE/include/libtest.shlib + +# +# DESCRIPTION: +# +# Write additional file level attributes stored in upper half of z_pflags +# +# STARTEGY: +# 1) Create a file +# 2) Execute write_dos_attributes on the file we created +# 3) Verify that write_dos_attributes exited successfully +# + +verify_runnable "global" + +FILETOTEST="$TESTDIR/test_write_dos_attrs.txt" + +function cleanup +{ + rm -f $FILETOTEST +} + +log_onexit cleanup + +log_must chmod 777 $TESTDIR +log_must eval "echo 'This is a test file.' > $FILETOTEST" +log_must write_dos_attributes offline $FILETOTEST +log_must write_dos_attributes nooffline $FILETOTEST + +log_pass "writing DOS attributes succeeded." diff --git a/tests/zfs-tests/tests/functional/events/.gitignore b/tests/zfs-tests/tests/functional/events/.gitignore deleted file mode 100644 index ed5af03a1095..000000000000 --- a/tests/zfs-tests/tests/functional/events/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/zed_fd_spill-zedlet diff --git a/tests/zfs-tests/tests/functional/events/Makefile.am b/tests/zfs-tests/tests/functional/events/Makefile.am deleted file mode 100644 index 92ce5dbc3825..000000000000 --- a/tests/zfs-tests/tests/functional/events/Makefile.am +++ /dev/null @@ -1,18 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/events -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - events_001_pos.ksh \ - events_002_pos.ksh \ - zed_rc_filter.ksh \ - zed_fd_spill.ksh - -dist_pkgdata_DATA = \ - events.cfg \ - events_common.kshlib - -pkgexecdir = $(pkgdatadir) -pkgexec_PROGRAMS = zed_fd_spill-zedlet -zed_fd_spill_zedlet_SOURCES = zed_fd_spill-zedlet.c diff --git a/tests/zfs-tests/tests/functional/events/cleanup.ksh b/tests/zfs-tests/tests/functional/events/cleanup.ksh index 699bc2823343..ef6e098cf42a 100755 --- a/tests/zfs-tests/tests/functional/events/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/events/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -28,4 +28,6 @@ zed_cleanup all-debug.sh all-syslog.sh all-dumpfds +zed_stop + default_cleanup diff --git a/tests/zfs-tests/tests/functional/events/events.cfg b/tests/zfs-tests/tests/functional/events/events.cfg index 1405dab6f989..5dbdb3074f31 100644 --- a/tests/zfs-tests/tests/functional/events/events.cfg +++ b/tests/zfs-tests/tests/functional/events/events.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -33,6 +33,4 @@ VDEV4=$TEST_BASE_DIR/vdev4 export TMP_EVENTS=$TEST_BASE_DIR/tmp_events.$$ export TMP_EVENTS_FULL=$TEST_BASE_DIR/tmp_events_full.$$ -export TMP_EVENT_FULL=$TEST_BASE_DIR/tmp_event_full.$$ export TMP_EVENTS_ZED=$TEST_BASE_DIR/tmp_events_zed.$$ -export TMP_EVENT_ZED=$TEST_BASE_DIR/tmp_event_zed.$$ diff --git a/tests/zfs-tests/tests/functional/events/events_001_pos.ksh b/tests/zfs-tests/tests/functional/events/events_001_pos.ksh index 189cf435e88e..b509a44606f0 100755 --- a/tests/zfs-tests/tests/functional/events/events_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/events/events_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/events/events_002_pos.ksh b/tests/zfs-tests/tests/functional/events/events_002_pos.ksh index af2be33dbc73..e355b8977378 100755 --- a/tests/zfs-tests/tests/functional/events/events_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/events/events_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -42,15 +42,8 @@ verify_runnable "both" function cleanup { - if poolexists $MPOOL; then - destroy_pool $MPOOL - fi - - for file in $VDEV1 $VDEV2; do - [[ -f $file ]] && rm -f $file - done - - log_must rm -f $TMP_EVENTS_ZED + poolexists $MPOOL && log_must destroy_pool $MPOOL + log_must rm -f $VDEV1 $VDEV2 $TMP_EVENTS_ZED log_must zed_stop } @@ -62,17 +55,16 @@ log_must truncate -s $MINVDEVSIZE $VDEV1 $VDEV2 # 1. Create a pool and generate some events. log_must truncate -s 0 $ZED_DEBUG_LOG log_must zpool events -c -log_must zpool create $MPOOL mirror $VDEV1 $VDEV2 +log_must zpool create -O compression=off $MPOOL mirror $VDEV1 $VDEV2 # 2. Start the ZED and verify it handles missed events. log_must zed_start log_must file_wait_event $ZED_DEBUG_LOG 'sysevent\.fs\.zfs\.config_sync' 150 log_must cp $ZED_DEBUG_LOG $TMP_EVENTS_ZED -awk -v event="sysevent.fs.zfs.pool_create" \ - 'BEGIN{FS="\n"; RS=""} $0 ~ event { print $0 }' \ - $TMP_EVENTS_ZED >$TMP_EVENT_ZED -log_must grep -q "^ZEVENT_POOL=$MPOOL" $TMP_EVENT_ZED +log_mustnot awk -v event="sysevent.fs.zfs.pool_create" -v crit="\\nZEVENT_POOL=$MPOOL" \ + 'BEGIN{FS="\n"; RS=""} $0 ~ event && $0 ~ crit { exit 1 }' \ + $TMP_EVENTS_ZED # 3. Stop the ZED zed_stop diff --git a/tests/zfs-tests/tests/functional/events/events_common.kshlib b/tests/zfs-tests/tests/functional/events/events_common.kshlib index cc600c4ed510..7a0b032c3b2f 100644 --- a/tests/zfs-tests/tests/functional/events/events_common.kshlib +++ b/tests/zfs-tests/tests/functional/events/events_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -94,7 +94,7 @@ function run_and_verify pool=${pool:-$TESTPOOL} fullcmd="$1" - cmd=$(echo $fullcmd | awk '{print $1}') + read -r cmd _ <<<"$fullcmd" # If we aren't running zpool or zfs, something is wrong [[ $cmd == "zpool" || $cmd == "zfs" ]] || \ @@ -147,23 +147,20 @@ function run_and_verify log_must grep -q "$event" $TMP_EVENTS # Verify the event is in the verbose output with pool name. - awk -v event="$event" \ - 'BEGIN{FS="\n"; RS=""} $0 ~ event { print $0 }' \ - $TMP_EVENTS_FULL >$TMP_EVENT_FULL - log_must grep -q "pool = \"$pool\"" $TMP_EVENT_FULL + log_mustnot awk -v event="$event" -v crit="pool = \"$pool\"" \ + 'BEGIN{FS="\n"; RS=""} $0 ~ event && $0 ~ crit { exit 1 }' \ + $TMP_EVENTS_FULL # all-debug.sh filters history events (seen in ZED_DEBUG_LOG) - if [[ "$event" == "sysevent.fs.zfs.history_event" ]]; then + if [ "$event" = "sysevent.fs.zfs.history_event" ]; then continue fi # Verify the event was received by the ZED and logged. - awk -v event="$event" \ - 'BEGIN{FS="\n"; RS=""} $0 ~ event { print $0 }' \ - $TMP_EVENTS_ZED >$TMP_EVENT_ZED - log_must grep -q "^ZEVENT_POOL=$pool" $TMP_EVENT_ZED + log_mustnot awk -v event="$event" -v crit="\\nZEVENT_POOL=$pool" \ + 'BEGIN{FS="\n"; RS=""} $0 ~ event && $0 ~ crit { exit 1 }' \ + $TMP_EVENTS_ZED done - rm -f $TMP_EVENTS $TMP_EVENTS_FULL $TMP_EVENT_FULL \ - $TMP_EVENTS_ZED $TMP_EVENT_ZED + rm -f $TMP_EVENTS $TMP_EVENTS_FULL $TMP_EVENTS_ZED } diff --git a/tests/zfs-tests/tests/functional/events/setup.ksh b/tests/zfs-tests/tests/functional/events/setup.ksh index 2f81d16b1814..0f058b8fe18c 100755 --- a/tests/zfs-tests/tests/functional/events/setup.ksh +++ b/tests/zfs-tests/tests/functional/events/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/events/zed_fd_spill.ksh b/tests/zfs-tests/tests/functional/events/zed_fd_spill.ksh index 8736a7fdf7e6..4803f0f3de7c 100755 --- a/tests/zfs-tests/tests/functional/events/zed_fd_spill.ksh +++ b/tests/zfs-tests/tests/functional/events/zed_fd_spill.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -47,8 +47,13 @@ log_onexit cleanup logdir="$(mktemp -d)" log_must ln -s "$logdir" /tmp/zts-zed_fd_spill-logdir -self="$(readlink -f "$0")" -log_must ln -s "${self%/*}/zed_fd_spill-zedlet" "${ZEDLET_DIR}/all-dumpfds" + +zedlet="$(command -v zed_fd_spill-zedlet)" +log_must ln -s "$zedlet" "${ZEDLET_DIR}/all-dumpfds" + +# zed will cry foul and refuse to run it if this isn't true +sudo chown root "$zedlet" +sudo chmod 700 "$zedlet" log_must zpool events -c log_must zed_stop diff --git a/tests/zfs-tests/tests/functional/events/zed_rc_filter.ksh b/tests/zfs-tests/tests/functional/events/zed_rc_filter.ksh index 0bef0ef1f96b..766468b5660d 100755 --- a/tests/zfs-tests/tests/functional/events/zed_rc_filter.ksh +++ b/tests/zfs-tests/tests/functional/events/zed_rc_filter.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/exec/Makefile.am b/tests/zfs-tests/tests/functional/exec/Makefile.am deleted file mode 100644 index 524bf2734f73..000000000000 --- a/tests/zfs-tests/tests/functional/exec/Makefile.am +++ /dev/null @@ -1,9 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/exec - -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - exec_001_pos.ksh \ - exec_002_neg.ksh diff --git a/tests/zfs-tests/tests/functional/exec/cleanup.ksh b/tests/zfs-tests/tests/functional/exec/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/exec/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/exec/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/exec/exec_001_pos.ksh b/tests/zfs-tests/tests/functional/exec/exec_001_pos.ksh index 5de8a79ea30f..097ecd5f49dd 100755 --- a/tests/zfs-tests/tests/functional/exec/exec_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/exec/exec_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/exec/exec_002_neg.ksh b/tests/zfs-tests/tests/functional/exec/exec_002_neg.ksh index c11bf8442bcd..5cda647ef14f 100755 --- a/tests/zfs-tests/tests/functional/exec/exec_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/exec/exec_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -60,18 +60,12 @@ function cleanup function exec_n_check { typeset expect_value=$1 - shift - $@ - ret=$? - if [[ $ret != $expect_value ]]; then - log_fail "Unexpected return code: '$ret'" - fi - - return 0 + "$@" + log_must [ $? = $expect_value ] } -log_assert "Setting exec=off on a filesystem, processes can not be executed " \ +log_assert "Setting exec=off on a filesystem, processes can not be executed" \ "from this file system." log_onexit cleanup @@ -79,11 +73,11 @@ log_must cp $STF_PATH/ls $TESTDIR/myls log_must zfs set exec=off $TESTPOOL/$TESTFS if is_linux; then - log_must exec_n_check 126 $TESTDIR/myls - log_must exec_n_check 1 mmap_exec $TESTDIR/myls # EPERM + exp=1 # EPERM else - log_must exec_n_check 126 $TESTDIR/myls - log_must exec_n_check 13 mmap_exec $TESTDIR/myls # EACCES + exp=13 # EACCES fi +log_must exec_n_check 126 $TESTDIR/myls +log_must exec_n_check $exp mmap_exec $TESTDIR/myls log_pass "Setting exec=off on filesystem testing passed." diff --git a/tests/zfs-tests/tests/functional/exec/setup.ksh b/tests/zfs-tests/tests/functional/exec/setup.ksh index fc5cec3063a6..b756d4e76c83 100755 --- a/tests/zfs-tests/tests/functional/exec/setup.ksh +++ b/tests/zfs-tests/tests/functional/exec/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/fallocate/Makefile.am b/tests/zfs-tests/tests/functional/fallocate/Makefile.am deleted file mode 100644 index 5ff366d2482c..000000000000 --- a/tests/zfs-tests/tests/functional/fallocate/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/fallocate -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - fallocate_prealloc.ksh \ - fallocate_punch-hole.ksh diff --git a/tests/zfs-tests/tests/functional/fallocate/cleanup.ksh b/tests/zfs-tests/tests/functional/fallocate/cleanup.ksh index bdfa6147117b..c99f11a0e87c 100755 --- a/tests/zfs-tests/tests/functional/fallocate/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/fallocate/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/fallocate/fallocate_prealloc.ksh b/tests/zfs-tests/tests/functional/fallocate/fallocate_prealloc.ksh index 7bb020fe5c19..35eb828d6ba6 100755 --- a/tests/zfs-tests/tests/functional/fallocate/fallocate_prealloc.ksh +++ b/tests/zfs-tests/tests/functional/fallocate/fallocate_prealloc.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/fallocate/fallocate_punch-hole.ksh b/tests/zfs-tests/tests/functional/fallocate/fallocate_punch-hole.ksh index ed83561bd556..ca037fdff777 100755 --- a/tests/zfs-tests/tests/functional/fallocate/fallocate_punch-hole.ksh +++ b/tests/zfs-tests/tests/functional/fallocate/fallocate_punch-hole.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -60,13 +60,17 @@ function cleanup [[ -e $TESTDIR ]] && log_must rm -f $FILE } -function check_disk_size +function check_reported_size { typeset expected_size=$1 - disk_size=$(du $TESTDIR/file | awk '{print $1}') - if [ $disk_size -ne $expected_size ]; then - log_fail "Incorrect size: $disk_size != $expected_size" + if ! [ -e "${FILE}" ]; then + log_fail "$FILE does not exist" + fi + + reported_size=$(du "${FILE}" | awk '{print $1}') + if [ "$reported_size" != "$expected_size" ]; then + log_fail "Incorrect reported size: $reported_size != $expected_size" fi } @@ -74,9 +78,9 @@ function check_apparent_size { typeset expected_size=$1 - apparent_size=$(stat_size) - if [ $apparent_size -ne $expected_size ]; then - log_fail "Incorrect size: $apparent_size != $expected_size" + apparent_size=$(stat_size "${FILE}") + if [ "$apparent_size" != "$expected_size" ]; then + log_fail "Incorrect apparent size: $apparent_size != $expected_size" fi } @@ -86,25 +90,30 @@ log_onexit cleanup # Create a dense file and check it is the correct size. log_must file_write -o create -f $FILE -b $BLKSZ -c 8 -log_must check_disk_size $((131072 * 8)) +sync_pool $TESTPOOL +log_must check_reported_size 1027 # Punch a hole for the first full block. log_must punch_hole 0 $BLKSZ $FILE -log_must check_disk_size $((131072 * 7)) +sync_pool $TESTPOOL +log_must check_reported_size 899 # Partially punch a hole in the second block. log_must punch_hole $BLKSZ $((BLKSZ / 2)) $FILE -log_must check_disk_size $((131072 * 7)) +sync_pool $TESTPOOL +log_must check_reported_size 899 -# Punch a hole which overlaps the third and forth block. +# Punch a hole which overlaps the third and fourth block. log_must punch_hole $(((BLKSZ * 2) + (BLKSZ / 2))) $((BLKSZ)) $FILE -log_must check_disk_size $((131072 * 7)) +sync_pool $TESTPOOL +log_must check_reported_size 899 # Punch a hole from the fifth block past the end of file. The apparent # file size should not change since --keep-size is implied. apparent_size=$(stat_size $FILE) log_must punch_hole $((BLKSZ * 4)) $((BLKSZ * 10)) $FILE -log_must check_disk_size $((131072 * 4)) +sync_pool $TESTPOOL +log_must check_reported_size 387 log_must check_apparent_size $apparent_size log_pass "Ensure holes can be punched in files making them sparse" diff --git a/tests/zfs-tests/tests/functional/fallocate/fallocate_zero-range.ksh b/tests/zfs-tests/tests/functional/fallocate/fallocate_zero-range.ksh new file mode 100755 index 000000000000..d8f14273ca1f --- /dev/null +++ b/tests/zfs-tests/tests/functional/fallocate/fallocate_zero-range.ksh @@ -0,0 +1,119 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright (c) 2020 by Lawrence Livermore National Security, LLC. +# Copyright (c) 2021 by The FreeBSD Foundation. +# + +. $STF_SUITE/include/libtest.shlib + +# +# DESCRIPTION: +# Test FALLOC_FL_ZERO_RANGE functionality +# +# STRATEGY: +# 1. Create a dense file +# 2. Zero various ranges in the file and verify the result. +# + +verify_runnable "global" + +if is_freebsd; then + log_unsupported "FreeBSD does not implement an analogue to ZERO_RANGE." +fi + +FILE=$TESTDIR/$TESTFILE0 +BLKSZ=$(get_prop recordsize $TESTPOOL) + +function cleanup +{ + [[ -e $TESTDIR ]] && log_must rm -f $FILE +} + +# Helpfully, this function expects kilobytes, and check_apparent_size expects bytes. +function check_reported_size +{ + typeset expected_size=$1 + + if ! [ -e "${FILE}" ]; then + log_fail "$FILE does not exist" + fi + + reported_size=$(du "${FILE}" | awk '{print $1}') + if [ "$reported_size" != "$expected_size" ]; then + log_fail "Incorrect reported size: $reported_size != $expected_size" + fi +} + +function check_apparent_size +{ + typeset expected_size=$1 + + apparent_size=$(stat_size "${FILE}") + if [ "$apparent_size" != "$expected_size" ]; then + log_fail "Incorrect apparent size: $apparent_size != $expected_size" + fi +} + +log_assert "Ensure ranges can be zeroed in files" + +log_onexit cleanup + +# Create a dense file and check it is the correct size. +log_must file_write -o create -f $FILE -b $BLKSZ -c 8 +sync_pool $TESTPOOL +log_must check_reported_size 1027 + +# Zero a range covering the first full block. +log_must zero_range 0 $BLKSZ $FILE +sync_pool $TESTPOOL +log_must check_reported_size 899 + +# Partially zero a range in the second block. +log_must zero_range $BLKSZ $((BLKSZ / 2)) $FILE +sync_pool $TESTPOOL +log_must check_reported_size 899 + +# Zero range which overlaps the third and fourth block. +log_must zero_range $(((BLKSZ * 2) + (BLKSZ / 2))) $((BLKSZ)) $FILE +sync_pool $TESTPOOL +log_must check_reported_size 899 + +# Zero range from the fifth block past the end of file, with --keep-size. +# The apparent file size must not change, since we did specify --keep-size. +apparent_size=$(stat_size $FILE) +log_must fallocate --keep-size --zero-range --offset $((BLKSZ * 4)) --length $((BLKSZ * 10)) "$FILE" +sync_pool $TESTPOOL +log_must check_reported_size 387 +log_must check_apparent_size $apparent_size + +# Zero range from the fifth block past the end of file. The apparent +# file size should change since --keep-size is not implied, unlike +# with PUNCH_HOLE. +apparent_size=$(stat_size $FILE) +log_must zero_range $((BLKSZ * 4)) $((BLKSZ * 10)) $FILE +sync_pool $TESTPOOL +log_must check_reported_size 387 +log_must check_apparent_size $((BLKSZ * 14)) + +log_pass "Ensure ranges can be zeroed in files" diff --git a/tests/zfs-tests/tests/functional/fallocate/setup.ksh b/tests/zfs-tests/tests/functional/fallocate/setup.ksh index 32334d396865..51fa056fd36f 100755 --- a/tests/zfs-tests/tests/functional/fallocate/setup.ksh +++ b/tests/zfs-tests/tests/functional/fallocate/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -26,4 +26,7 @@ . $STF_SUITE/include/libtest.shlib DISK=${DISKS%% *} -default_setup $DISK +default_setup_noexit $DISK +log_must zfs set compression=off $TESTPOOL +log_pass + diff --git a/tests/zfs-tests/tests/functional/fault/Makefile.am b/tests/zfs-tests/tests/functional/fault/Makefile.am deleted file mode 100644 index ba0d7d6992c6..000000000000 --- a/tests/zfs-tests/tests/functional/fault/Makefile.am +++ /dev/null @@ -1,20 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/fault -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - auto_offline_001_pos.ksh \ - auto_online_001_pos.ksh \ - auto_online_002_pos.ksh \ - auto_replace_001_pos.ksh \ - auto_spare_001_pos.ksh \ - auto_spare_002_pos.ksh \ - auto_spare_ashift.ksh \ - auto_spare_multiple.ksh \ - auto_spare_shared.ksh \ - decrypt_fault.ksh \ - decompress_fault.ksh \ - scrub_after_resilver.ksh \ - zpool_status_-s.ksh - -dist_pkgdata_DATA = \ - fault.cfg diff --git a/tests/zfs-tests/tests/functional/fault/auto_offline_001_pos.ksh b/tests/zfs-tests/tests/functional/fault/auto_offline_001_pos.ksh index ef2ce24e097b..17bde9a70636 100755 --- a/tests/zfs-tests/tests/functional/fault/auto_offline_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/fault/auto_offline_001_pos.ksh @@ -96,8 +96,7 @@ do log_must zpool create -f $TESTPOOL $conf block_device_wait ${DEV_DSKDIR}/${removedev} - mntpnt=$(get_prop mountpoint /$TESTPOOL) || - log_fail "get_prop mountpoint /$TESTPOOL" + mntpnt=$(get_prop mountpoint /$TESTPOOL) # 2. Simulate physical removal of one device remove_disk $removedev @@ -128,8 +127,7 @@ do block_device_wait ${DEV_DSKDIR}/${removedev} log_must zpool add $TESTPOOL spare $sparedev - mntpnt=$(get_prop mountpoint /$TESTPOOL) || - log_fail "get_prop mountpoint /$TESTPOOL" + mntpnt=$(get_prop mountpoint /$TESTPOOL) # 2. Simulate physical removal of one device remove_disk $removedev @@ -161,8 +159,7 @@ do block_device_wait ${DEV_DSKDIR}/${removedev} log_must zpool add $TESTPOOL spare $sparedev - mntpnt=$(get_prop mountpoint /$TESTPOOL) || - log_fail "get_prop mountpoint /$TESTPOOL" + mntpnt=$(get_prop mountpoint /$TESTPOOL) # 2. Fault the spare device making it unavailable log_must zpool offline -f $TESTPOOL $sparedev diff --git a/tests/zfs-tests/tests/functional/fault/auto_online_001_pos.ksh b/tests/zfs-tests/tests/functional/fault/auto_online_001_pos.ksh index 03fc15a8a7cb..950e80dfc186 100755 --- a/tests/zfs-tests/tests/functional/fault/auto_online_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/fault/auto_online_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -114,10 +114,7 @@ do # Reimport pool with drive missing log_must zpool import $TESTPOOL - check_state $TESTPOOL "" "degraded" - if (($? != 0)); then - log_fail "$TESTPOOL is not degraded" - fi + log_must check_state $TESTPOOL "" "degraded" # Clear zpool events log_must zpool events -c @@ -134,9 +131,8 @@ do ((timeout++)) sleep 1 - zpool events $TESTPOOL \ - | egrep sysevent.fs.zfs.resilver_finish > /dev/null - if (($? == 0)); then + if zpool events $TESTPOOL \ + | grep -qF sysevent.fs.zfs.resilver_finish; then log_note "Auto-online of $offline_disk is complete" sleep 1 break @@ -144,10 +140,7 @@ do done # Validate auto-online was successful - check_state $TESTPOOL "" "online" - if (($? != 0)); then - log_fail "$TESTPOOL is not back online" - fi + log_must check_state $TESTPOOL "" "online" sleep 2 done log_must zpool destroy $TESTPOOL diff --git a/tests/zfs-tests/tests/functional/fault/auto_online_002_pos.ksh b/tests/zfs-tests/tests/functional/fault/auto_online_002_pos.ksh index 60185ace34bb..09285b5108ee 100755 --- a/tests/zfs-tests/tests/functional/fault/auto_online_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/fault/auto_online_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/fault/auto_replace_001_pos.ksh b/tests/zfs-tests/tests/functional/fault/auto_replace_001_pos.ksh index 0302c45373fa..2846192d08eb 100755 --- a/tests/zfs-tests/tests/functional/fault/auto_replace_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/fault/auto_replace_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -72,8 +72,8 @@ echo "alias scsidebug /dev/disk/by-id/$SD_DEVICE_ID" >>$VDEVID_CONF block_device_wait SD_DEVICE=$(udevadm info -q all -n $DEV_DSKDIR/$SD | \ - awk -F'=' '/ID_VDEV=/{print $2; exit}') -[[ -z $SD_DEVICE ]] && log_fail "vdev rule was not registered properly" + awk -F'=' '/ID_VDEV=/ {print $2; exit}') +[ -z $SD_DEVICE ] && log_fail "vdev rule was not registered properly" log_must zpool events -c log_must zpool create -f $TESTPOOL raidz1 $SD_DEVICE $DISK1 $DISK2 $DISK3 diff --git a/tests/zfs-tests/tests/functional/fault/auto_spare_shared.ksh b/tests/zfs-tests/tests/functional/fault/auto_spare_shared.ksh index 4229537b3953..1cfa84a930bc 100755 --- a/tests/zfs-tests/tests/functional/fault/auto_spare_shared.ksh +++ b/tests/zfs-tests/tests/functional/fault/auto_spare_shared.ksh @@ -70,10 +70,7 @@ FAIL_FILEDEVPOOL2="$TEST_BASE_DIR/file-fail-dev2" SPARE_FILEDEV="$TEST_BASE_DIR/file-spare-dev" SPARE_DISKDEV="$(get_debug_device)" -for vdev in $SAFE_FILEDEVPOOL1 $SAFE_FILEDEVPOOL2 $FAIL_FILEDEVPOOL1 \ - $FAIL_FILEDEVPOOL2 $SPARE_FILEDEV; do - log_must truncate -s $MINVDEVSIZE $vdev -done +log_must truncate -s $MINVDEVSIZE $SAFE_FILEDEVPOOL1 $SAFE_FILEDEVPOOL2 $FAIL_FILEDEVPOOL1 $FAIL_FILEDEVPOOL2 $SPARE_FILEDEV for spare in $SPARE_FILEDEV $SPARE_DISKDEV; do # 1. Create two pools diff --git a/tests/zfs-tests/tests/functional/fault/cleanup.ksh b/tests/zfs-tests/tests/functional/fault/cleanup.ksh index 45b94723a543..654343c0cf00 100755 --- a/tests/zfs-tests/tests/functional/fault/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/fault/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/fault/decrypt_fault.ksh b/tests/zfs-tests/tests/functional/fault/decrypt_fault.ksh index d81c4b2bd2f2..a4cf06144e58 100755 --- a/tests/zfs-tests/tests/functional/fault/decrypt_fault.ksh +++ b/tests/zfs-tests/tests/functional/fault/decrypt_fault.ksh @@ -40,6 +40,7 @@ function cleanup log_onexit cleanup default_mirror_setup_noexit $DISK1 $DISK2 +log_must zfs set compression=off $TESTPOOL log_must eval "echo 'password' | zfs create -o encryption=on \ -o keyformat=passphrase -o keylocation=prompt $TESTPOOL/fs" mntpt=$(get_prop mountpoint $TESTPOOL/fs) diff --git a/tests/zfs-tests/tests/functional/fault/fault.cfg b/tests/zfs-tests/tests/functional/fault/fault.cfg index 839330ed47dd..0cf2b05cce44 100644 --- a/tests/zfs-tests/tests/functional/fault/fault.cfg +++ b/tests/zfs-tests/tests/functional/fault/fault.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -26,7 +26,7 @@ verify_runnable "global" -export DISK_ARRAY_NUM=$(echo ${DISKS} | nawk '{print NF}') +export DISK_ARRAY_NUM=$(echo ${DISKS} | awk '{print NF}') export DISKSARRAY=$DISKS export FSIZE=10M export MAXTIMEOUT=30 @@ -36,9 +36,8 @@ export SDHOSTS=1 export SDTGTS=1 export SDLUNS=1 -export DISK1=$(echo $DISKS | nawk '{print $1}') -export DISK2=$(echo $DISKS | nawk '{print $2}') -export DISK3=$(echo $DISKS | nawk '{print $3}') +read -r DISK1 DISK2 DISK3 _ <<<"$DISKS" +export DISK1 DISK2 DISK3 if is_linux; then set_slice_prefix diff --git a/tests/zfs-tests/tests/functional/fault/setup.ksh b/tests/zfs-tests/tests/functional/fault/setup.ksh index b78ee8ccdc98..62f1c8ab56cb 100755 --- a/tests/zfs-tests/tests/functional/fault/setup.ksh +++ b/tests/zfs-tests/tests/functional/fault/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/fault/zpool_status_-s.ksh b/tests/zfs-tests/tests/functional/fault/zpool_status_-s.ksh index a290053fd269..57911aa05ec9 100755 --- a/tests/zfs-tests/tests/functional/fault/zpool_status_-s.ksh +++ b/tests/zfs-tests/tests/functional/fault/zpool_status_-s.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -67,11 +67,10 @@ log_must mkfile 1048576 /$TESTPOOL/testfile sync_pool $TESTPOOL log_must zinject -c all -SLOW_IOS=$(zpool status -sp | grep "$DISK" | awk '{print $6}') -DELAY_EVENTS=$(zpool events | grep delay | wc -l) +SLOW_IOS=$(zpool status -sp | awk -v d="$DISK" '$0 ~ d {print $6}') +DELAY_EVENTS=$(zpool events | grep -c delay) -if [ $SLOW_IOS -gt 0 ] && [ $DELAY_EVENTS -gt 0 ] ; then - log_pass "Correctly saw $SLOW_IOS slow IOs and $DELAY_EVENTS delay events" -else - log_fail "Only saw $SLOW_IOS slow IOs and $DELAY_EVENTS delay events" -fi +log_must [ $SLOW_IOS -gt 0 ] +log_must [ $DELAY_EVENTS -gt 0 ] + +log_pass "Correctly saw $SLOW_IOS slow IOs and $DELAY_EVENTS delay events" diff --git a/tests/zfs-tests/tests/functional/features/Makefile.am b/tests/zfs-tests/tests/functional/features/Makefile.am deleted file mode 100644 index 3657461e6604..000000000000 --- a/tests/zfs-tests/tests/functional/features/Makefile.am +++ /dev/null @@ -1,3 +0,0 @@ -SUBDIRS = \ - async_destroy \ - large_dnode diff --git a/tests/zfs-tests/tests/functional/features/async_destroy/Makefile.am b/tests/zfs-tests/tests/functional/features/async_destroy/Makefile.am deleted file mode 100644 index 4c777878d689..000000000000 --- a/tests/zfs-tests/tests/functional/features/async_destroy/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/features/async_destroy -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - async_destroy_001_pos.ksh diff --git a/tests/zfs-tests/tests/functional/features/async_destroy/async_destroy_001_pos.ksh b/tests/zfs-tests/tests/functional/features/async_destroy/async_destroy_001_pos.ksh index e0617961ba15..d5c930ecfbb2 100755 --- a/tests/zfs-tests/tests/functional/features/async_destroy/async_destroy_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/features/async_destroy/async_destroy_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/features/async_destroy/cleanup.ksh b/tests/zfs-tests/tests/functional/features/async_destroy/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/features/async_destroy/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/features/async_destroy/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/features/async_destroy/setup.ksh b/tests/zfs-tests/tests/functional/features/async_destroy/setup.ksh index d275e063b13a..76aa23035dd9 100755 --- a/tests/zfs-tests/tests/functional/features/async_destroy/setup.ksh +++ b/tests/zfs-tests/tests/functional/features/async_destroy/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/features/large_dnode/Makefile.am b/tests/zfs-tests/tests/functional/features/large_dnode/Makefile.am deleted file mode 100644 index 13ba3ab33d9e..000000000000 --- a/tests/zfs-tests/tests/functional/features/large_dnode/Makefile.am +++ /dev/null @@ -1,13 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/features/large_dnode -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - large_dnode_001_pos.ksh \ - large_dnode_002_pos.ksh \ - large_dnode_003_pos.ksh \ - large_dnode_004_neg.ksh \ - large_dnode_005_pos.ksh \ - large_dnode_006_pos.ksh \ - large_dnode_007_neg.ksh \ - large_dnode_008_pos.ksh \ - large_dnode_009_pos.ksh diff --git a/tests/zfs-tests/tests/functional/features/large_dnode/cleanup.ksh b/tests/zfs-tests/tests/functional/features/large_dnode/cleanup.ksh index 60e481d998a2..9fa5ba6e13c4 100755 --- a/tests/zfs-tests/tests/functional/features/large_dnode/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/features/large_dnode/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_001_pos.ksh b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_001_pos.ksh index cb1e940a7d73..e4aea2750669 100755 --- a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -73,7 +73,7 @@ log_must zfs umount $TEST_FS for ((i=0; i < ${#dnsizes[*]}; i++)) ; do dnsize=$(zdb -dddd $TEST_FS ${inodes[$i]} | - awk '/ZFS plain file/ {print $6}' | tr K k) + awk '/ZFS plain file/ {gsub(/K/, "k", $6); print $6}') if [[ "$dnsize" != "${dnsizes[$i]}" ]]; then log_fail "dnode size is $dnsize (expected ${dnsizes[$i]})" fi diff --git a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_002_pos.ksh b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_002_pos.ksh index 9a00ceeb3cef..c0df76624f56 100755 --- a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_003_pos.ksh b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_003_pos.ksh index 20989e1d7741..867382165235 100755 --- a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_004_neg.ksh b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_004_neg.ksh index 2cc587b47879..a77fe10ccf31 100755 --- a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_004_neg.ksh +++ b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_004_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_005_pos.ksh b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_005_pos.ksh index 2be98942634f..8910f375b1c5 100755 --- a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -54,11 +54,11 @@ log_assert "zfs send stream with large dnodes accepted by new pool" log_must zfs create -o dnodesize=1k $TEST_SEND_FS log_must touch /$TEST_SEND_FS/$TEST_FILE log_must zfs snap $TEST_SNAP -log_must zfs send $TEST_SNAP > $TEST_STREAM +log_must eval "zfs send $TEST_SNAP > $TEST_STREAM" log_must rm -f /$TEST_SEND_FS/$TEST_FILE log_must touch /$TEST_SEND_FS/$TEST_FILEINCR log_must zfs snap $TEST_SNAPINCR -log_must zfs send -i $TEST_SNAP $TEST_SNAPINCR > $TEST_STREAMINCR +log_must eval "zfs send -i $TEST_SNAP $TEST_SNAPINCR > $TEST_STREAMINCR" log_must eval "zfs recv $TEST_RECV_FS < $TEST_STREAM" inode=$(ls -li /$TEST_RECV_FS/$TEST_FILE | awk '{print $1}') @@ -68,7 +68,7 @@ if [[ "$dnsize" != "1K" ]]; then fi log_must eval "zfs recv -F $TEST_RECV_FS < $TEST_STREAMINCR" -log_must diff -r /$TEST_SEND_FS /$TEST_RECV_FS +log_must directory_diff /$TEST_SEND_FS /$TEST_RECV_FS log_must zfs umount $TEST_SEND_FS log_must zfs umount $TEST_RECV_FS diff --git a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_006_pos.ksh b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_006_pos.ksh index 3727bd5c11d2..a84e38aee87a 100755 --- a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_007_neg.ksh b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_007_neg.ksh index 59364574b165..ac755a338d55 100755 --- a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_007_neg.ksh +++ b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_007_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_008_pos.ksh b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_008_pos.ksh index 71e175171322..00a0d5784cdb 100755 --- a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_009_pos.ksh b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_009_pos.ksh index 1e42202069eb..457c100ed989 100755 --- a/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_009_pos.ksh +++ b/tests/zfs-tests/tests/functional/features/large_dnode/large_dnode_009_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -66,6 +66,6 @@ log_must wait log_must_busy zpool export $TESTPOOL log_must zpool import $TESTPOOL -log_must ls -lR "/$TEST_FS/" >/dev/null 2>&1 +log_must eval "ls -lR /$TEST_FS/ >/dev/null 2>&1" log_must zdb -d $TESTPOOL log_pass diff --git a/tests/zfs-tests/tests/functional/features/large_dnode/setup.ksh b/tests/zfs-tests/tests/functional/features/large_dnode/setup.ksh index a9425cca9857..02735d765426 100755 --- a/tests/zfs-tests/tests/functional/features/large_dnode/setup.ksh +++ b/tests/zfs-tests/tests/functional/features/large_dnode/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/grow/Makefile.am b/tests/zfs-tests/tests/functional/grow/Makefile.am deleted file mode 100644 index 807610a067d8..000000000000 --- a/tests/zfs-tests/tests/functional/grow/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/grow -dist_pkgdata_SCRIPTS = \ - grow_pool_001_pos.ksh \ - grow_replicas_001_pos.ksh - -dist_pkgdata_DATA = \ - grow.cfg diff --git a/tests/zfs-tests/tests/functional/grow/grow.cfg b/tests/zfs-tests/tests/functional/grow/grow.cfg index 10d15d7546c1..0d78416317c0 100644 --- a/tests/zfs-tests/tests/functional/grow/grow.cfg +++ b/tests/zfs-tests/tests/functional/grow/grow.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/grow/grow_pool_001_pos.ksh b/tests/zfs-tests/tests/functional/grow/grow_pool_001_pos.ksh index 9c047223ab39..ae2b261f4efa 100755 --- a/tests/zfs-tests/tests/functional/grow/grow_pool_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/grow/grow_pool_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/grow/grow_replicas_001_pos.ksh b/tests/zfs-tests/tests/functional/grow/grow_replicas_001_pos.ksh index 44b38291f0a9..1d0215550145 100755 --- a/tests/zfs-tests/tests/functional/grow/grow_replicas_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/grow/grow_replicas_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/history/Makefile.am b/tests/zfs-tests/tests/functional/history/Makefile.am deleted file mode 100644 index b865a319a430..000000000000 --- a/tests/zfs-tests/tests/functional/history/Makefile.am +++ /dev/null @@ -1,23 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/history -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - history_001_pos.ksh \ - history_002_pos.ksh \ - history_003_pos.ksh \ - history_004_pos.ksh \ - history_005_neg.ksh \ - history_006_neg.ksh \ - history_007_pos.ksh \ - history_008_pos.ksh \ - history_009_pos.ksh \ - history_010_pos.ksh - -dist_pkgdata_DATA = \ - history.cfg \ - history_common.kshlib \ - i386.migratedpool.DAT.Z \ - i386.orig_history.txt \ - sparc.migratedpool.DAT.Z \ - sparc.orig_history.txt \ - zfs-pool-v4.dat.Z diff --git a/tests/zfs-tests/tests/functional/history/cleanup.ksh b/tests/zfs-tests/tests/functional/history/cleanup.ksh index 32ed921ffe76..36e0311a34cb 100755 --- a/tests/zfs-tests/tests/functional/history/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/history/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/history/history.cfg b/tests/zfs-tests/tests/functional/history/history.cfg index e9200a2b5098..a508a7935684 100644 --- a/tests/zfs-tests/tests/functional/history/history.cfg +++ b/tests/zfs-tests/tests/functional/history/history.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/history/history_001_pos.ksh b/tests/zfs-tests/tests/functional/history/history_001_pos.ksh index f33265185d5c..7180e9af767d 100755 --- a/tests/zfs-tests/tests/functional/history/history_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/history/history_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -59,7 +59,6 @@ log_assert "Verify zpool sub-commands which modify state are logged." log_onexit cleanup mntpnt=$(get_prop mountpoint $TESTPOOL) -(( $? != 0)) && log_fail "get_prop($TESTPOOL mountpoint)" VDEV1=$mntpnt/vdev1; VDEV2=$mntpnt/vdev2; VDEV3=$mntpnt/vdev3; VDEV4=$mntpnt/vdev4; diff --git a/tests/zfs-tests/tests/functional/history/history_002_pos.ksh b/tests/zfs-tests/tests/functional/history/history_002_pos.ksh index b431cdc5f144..81f2ffb21731 100755 --- a/tests/zfs-tests/tests/functional/history/history_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/history/history_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/history/history_003_pos.ksh b/tests/zfs-tests/tests/functional/history/history_003_pos.ksh index 46af53f8af90..9f27785d64f9 100755 --- a/tests/zfs-tests/tests/functional/history/history_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/history/history_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -55,7 +55,6 @@ log_assert "zpool history limitation test." log_onexit cleanup mntpnt=$(get_prop mountpoint $TESTPOOL) -(( $? != 0 )) && log_fail "get_prop mountpoint $TESTPOOL" VDEV0=$mntpnt/vdev0 log_must mkfile $MINVDEVSIZE $VDEV0 @@ -79,16 +78,16 @@ done TMPFILE=$TEST_BASE_DIR/spool.$$ zpool history $spool >$TMPFILE -typeset -i entry_count=$(wc -l $TMPFILE | awk '{print $1}') +typeset -i entry_count=$(wc -l < $TMPFILE) typeset final_md5=$(head -2 $TMPFILE | md5digest) -grep 'zpool create' $TMPFILE >/dev/null 2>&1 || +grep -q 'zpool create' $TMPFILE || log_fail "'zpool create' was not found in pool history" -grep 'zfs create' $TMPFILE >/dev/null 2>&1 && +grep -q 'zfs create' $TMPFILE && log_fail "'zfs create' was found in pool history" -grep 'zfs set compress' $TMPFILE >/dev/null 2>&1 || +grep -q 'zfs set compress' $TMPFILE || log_fail "'zfs set compress' was found in pool history" # Verify that the creation of the pool was preserved in the history. diff --git a/tests/zfs-tests/tests/functional/history/history_004_pos.ksh b/tests/zfs-tests/tests/functional/history/history_004_pos.ksh index 1b8e7dfe02ec..f7ec5df6fd31 100755 --- a/tests/zfs-tests/tests/functional/history/history_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/history/history_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -46,7 +46,7 @@ verify_runnable "global" log_assert "'zpool history' can cope with simultaneous commands." -typeset -i orig_count=$(zpool history $spool | wc -l | awk '{print $1}') +typeset -i orig_count=$(zpool history $spool | wc -l) typeset -i i=0 while ((i < 10)); do @@ -90,7 +90,7 @@ while ((i < 10)); do ((i += 1)) done -typeset -i entry_count=$(zpool history $spool | wc -l | awk '{print $1}') +typeset -i entry_count=$(zpool history $spool | wc -l) if ((entry_count - orig_count != 200)); then log_fail "The entries count error: entry_count=$entry_count " \ diff --git a/tests/zfs-tests/tests/functional/history/history_005_neg.ksh b/tests/zfs-tests/tests/functional/history/history_005_neg.ksh index 297a701cc567..c4d699f39d6c 100755 --- a/tests/zfs-tests/tests/functional/history/history_005_neg.ksh +++ b/tests/zfs-tests/tests/functional/history/history_005_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -54,10 +54,10 @@ log_assert "Verify 'zpool get|history|list|status|iostat' will not be logged." # Save initial TESTPOOL history log_must eval "zpool history $TESTPOOL >$OLD_HISTORY" -log_must zpool get all $TESTPOOL >/dev/null -log_must zpool list $TESTPOOL >/dev/null -log_must zpool status $TESTPOOL >/dev/null -log_must zpool iostat $TESTPOOL >/dev/null +log_must eval "zpool get all $TESTPOOL >/dev/null" +log_must eval "zpool list $TESTPOOL >/dev/null" +log_must eval "zpool status $TESTPOOL >/dev/null" +log_must eval "zpool iostat $TESTPOOL >/dev/null" log_must eval "zpool history $TESTPOOL >$NEW_HISTORY" log_must diff $OLD_HISTORY $NEW_HISTORY diff --git a/tests/zfs-tests/tests/functional/history/history_006_neg.ksh b/tests/zfs-tests/tests/functional/history/history_006_neg.ksh index c3a5e092d02d..c8b6404b8c35 100755 --- a/tests/zfs-tests/tests/functional/history/history_006_neg.ksh +++ b/tests/zfs-tests/tests/functional/history/history_006_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -67,17 +67,15 @@ log_must zfs snapshot $snap2 # Save initial TESTPOOL history log_must eval "zpool history $TESTPOOL > $OLD_HISTORY" -log_must zfs list $fs > /dev/null -log_must zfs get mountpoint $fs > /dev/null +log_must eval "zfs list $fs > /dev/null" +log_must eval "zfs get mountpoint $fs > /dev/null" log_must zfs unmount $fs log_must zfs mount $fs if ! is_linux; then log_must zfs share $fs log_must zfs unshare $fs fi -# https://github.com/openzfs/zfs/issues/11445 -set -o pipefail -log_must zfs send -i $snap1 $snap2 | cat > /dev/null +log_must eval "zfs send -i $snap1 $snap2 > /dev/null" log_must zfs holds $snap1 log_must eval "zpool history $TESTPOOL > $NEW_HISTORY" diff --git a/tests/zfs-tests/tests/functional/history/history_007_pos.ksh b/tests/zfs-tests/tests/functional/history/history_007_pos.ksh index 591d5b85e885..714db4772aa3 100755 --- a/tests/zfs-tests/tests/functional/history/history_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/history/history_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -49,7 +49,7 @@ function cleanup poolexists $migratedpoolname && \ log_must zpool destroy -f $migratedpoolname - [[ -d $import_dir ]] && rm -rf $import_dir + rm -rf $import_dir } log_assert "Verify command history moves with migrated pool." @@ -70,7 +70,7 @@ for arch in "i386" "sparc"; do orig_cmds_f=$import_dir/${arch}.orig_history.txt # remove blank line orig_cmds_f1=$import_dir/${arch}.orig_history_1.txt - cat $orig_cmds_f | grep -v "^$" > $orig_cmds_f1 + grep -v "^$" $orig_cmds_f > $orig_cmds_f1 log_must cp $tst_dir/${arch}.migratedpool.DAT.Z $import_dir log_must uncompress -f $import_dir/${arch}.migratedpool.DAT.Z @@ -80,10 +80,7 @@ for arch in "i386" "sparc"; do log_must zpool destroy -f $migratedpoolname log_must zpool import -d $import_dir $migratedpoolname - TZ=$TIMEZONE zpool history $migratedpoolname | grep -v "^$" \ - >$migrated_cmds_f - RET=$? - (( $RET != 0 )) && log_fail "zpool history $migratedpoolname fails." + log_must eval "TZ=$TIMEZONE zpool history $migratedpoolname | grep -v \"^\$\" >$migrated_cmds_f" # The migrated history file should differ with original history file on # two commands -- 'export' and 'import', which are included in migrated @@ -92,21 +89,20 @@ for arch in "i386" "sparc"; do # then compare this filtered file with the original history file. They # should be identical at this time. for subcmd in "export" "import"; do - grep "$subcmd" $migrated_cmds_f >/dev/null 2>&1 - RET=$? - (( $RET != 0 )) && log_fail "zpool $subcmd is not logged for" \ - "the imported pool $migratedpoolname." + grep -q "$subcmd" $migrated_cmds_f || + log_fail "zpool $subcmd is not logged for" \ + "the imported pool $migratedpoolname." done tmpfile=$import_dir/cmds_tmp.$$ - linenum=`cat $migrated_cmds_f | wc -l` + linenum=$(wc -l < $migrated_cmds_f) (( linenum = linenum - 2 )) head -n $linenum $migrated_cmds_f > $tmpfile log_must diff $tmpfile $orig_cmds_f1 # cleanup for next loop testing log_must zpool destroy -f $migratedpoolname - log_must rm -f `ls $import_dir` + log_must rm -f $(ls $import_dir) done log_pass "Verify command history moves with migrated pool." diff --git a/tests/zfs-tests/tests/functional/history/history_008_pos.ksh b/tests/zfs-tests/tests/functional/history/history_008_pos.ksh index 8e174dcb7ebf..eff313a31aef 100755 --- a/tests/zfs-tests/tests/functional/history/history_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/history/history_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -55,8 +55,7 @@ log_assert "Pool history records all recursive operations." log_onexit cleanup root_testfs=$TESTPOOL/$TESTFS -fs1=$root_testfs/fs1; fs2=$root_testfs/fs2; fs3=$root_testfs/fs3 -for fs in $fs1 $fs2 $fs3; do +for fs in $root_testfs/fs{1..3}; do log_must zfs create $fs done diff --git a/tests/zfs-tests/tests/functional/history/history_009_pos.ksh b/tests/zfs-tests/tests/functional/history/history_009_pos.ksh index cf40df84ec83..be43193dc426 100755 --- a/tests/zfs-tests/tests/functional/history/history_009_pos.ksh +++ b/tests/zfs-tests/tests/functional/history/history_009_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/history/history_010_pos.ksh b/tests/zfs-tests/tests/functional/history/history_010_pos.ksh index 2c32b1b6cefa..d972ba66d17b 100755 --- a/tests/zfs-tests/tests/functional/history/history_010_pos.ksh +++ b/tests/zfs-tests/tests/functional/history/history_010_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -66,10 +66,8 @@ add_user $HIST_GROUP $HIST_USER # # chmod 0750 $HOME # -user_run $HIST_USER zfs list -if [ $? -ne 0 ]; then - log_unsupported "Test user $HIST_USER cannot execute zfs utilities" -fi +user_run $HIST_USER zfs list || + log_unsupported "Test user $HIST_USER cannot execute zfs utilities" run_and_verify "zfs create $root_testfs" "-l" run_and_verify "zfs allow $HIST_GROUP snapshot,mount $root_testfs" "-l" diff --git a/tests/zfs-tests/tests/functional/history/history_common.kshlib b/tests/zfs-tests/tests/functional/history/history_common.kshlib index ff3260f3c0f2..2ee761cea93a 100644 --- a/tests/zfs-tests/tests/functional/history/history_common.kshlib +++ b/tests/zfs-tests/tests/functional/history/history_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -52,14 +52,12 @@ function run_and_verify flags="$2" if is_illumos; then - histcmd=$(echo $fullcmd | sed 's/\/usr\/sbin\///g') + histcmd=$(echo $fullcmd | sed 's=/usr/sbin/==g') else - histcmd=$(echo $fullcmd | sed 's/^.*\/\(zpool .*\).*$/\1/') - histcmd=$(echo $histcmd | sed 's/^.*\/\(zfs .*\).*$/\1/') + histcmd=$(echo $fullcmd | sed -E 's=^.*/(zpool|zfs)$=\1=') fi - cmd=$(echo $histcmd | awk '{print $1}') - subcmd=$(echo $histcmd | awk '{print $2}') + read -r cmd subcmd _ <<<"$histcmd" # If we aren't running zpool or zfs, something is wrong [[ $cmd == "zpool" || $cmd == "zfs" ]] || \ @@ -77,11 +75,10 @@ function run_and_verify log_must_busy user_run $user "$fullcmd" fi zpool history $flags $pool > $TMP_HISTORY 2>/dev/null - diff $OLD_HISTORY $TMP_HISTORY | grep "^> " | sed 's/^> //g' \ - > $NEW_HISTORY + diff $OLD_HISTORY $TMP_HISTORY | sed -n 's/^> //gp' > $NEW_HISTORY # Verify what's common to every case, regardless of zpool history flags. - grep "$histcmd" $NEW_HISTORY >/dev/null 2>&1 || \ + grep -q "$histcmd" $NEW_HISTORY || \ log_fail "Didn't find \"$histcmd\" in pool history" # If 'zpool history' was called without any flags, then we're done. @@ -116,8 +113,7 @@ function verify_long suffix=":freebsd" fi - grep -q "$cmd \[user $uid ($user) on $hname$suffix\]" $NEW_HISTORY - if [[ $? != 0 ]]; then + if ! grep -q "$cmd \[user $uid ($user) on $hname$suffix\]" $NEW_HISTORY; then log_note "Couldn't find long information for \"$cmd\"" return 1 fi @@ -133,7 +129,8 @@ function verify_hold [[ $flags =~ "i" ]] || return 1 - typeset tag=$(echo $cmd | awk '{print $4}') + typeset tag _ + read -r _ _ _ tag _ <<<"$cmd" typeset fullname=${cmd##* } typeset dsname=${fullname%%@*} typeset snapname=${fullname##*@} @@ -141,9 +138,7 @@ function verify_hold # This works whether or not the hold was recursive for ds in $(zfs list -r -Ho name -t snapshot $dsname | \ grep "@$snapname"); do - grep "$subcmd $ds ([0-9]*) tag=$tag" $NEW_HISTORY \ - >/dev/null 2>&1 - if [[ $? != 0 ]]; then + if ! grep -q "$subcmd $ds ([0-9]*) tag=$tag" $NEW_HISTORY; then log_note "Didn't find hold on $ds with $tag" return 1 fi @@ -172,15 +167,12 @@ function verify_rollback typeset rb_fs=${dsname}/%rollback typeset snapname=${fullname##*@} - grep "clone swap $rb_fs ([0-9]*) parent=$parent_fs" $NEW_HISTORY \ - >/dev/null 2>&1 - if [[ $? != 0 ]]; then + if ! grep -q "clone swap $rb_fs ([0-9]*) parent=$parent_fs" $NEW_HISTORY ; then log_note "Didn't find rollback clone swap in pool history" return 1 fi - grep "destroy $rb_fs" $NEW_HISTORY >/dev/null 2>&1 - if [[ $? != 0 ]]; then + if ! grep -q "destroy $rb_fs" $NEW_HISTORY; then log_note "Didn't find rollback destroy in pool history" return 1 fi @@ -201,9 +193,7 @@ function verify_inherit # This works whether or not the inherit was recursive for ds in $(zfs list -r -Ho name -t filesystem $dsname); do - grep "$subcmd $ds ([0-9]*) ${prop}=" $NEW_HISTORY >/dev/null \ - 2>&1 - if [[ $? != 0 ]]; then + if ! grep -q "$subcmd $ds ([0-9]*) ${prop}=" $NEW_HISTORY; then log_note "Didn't find inherit history for $ds" return 1 fi @@ -231,8 +221,7 @@ function verify_allow # - Whether the operation applies locally or to descendent datasets (or # both) # - echo $cmd | awk '{i = NF - 1; print $i}' | grep '@' >/dev/null \ - 2>&1 && is_set=1 + echo $cmd | awk '$(NF - 1) ~ /@/ {exit 1}' || is_set=1 dsname=${cmd##* } [[ $cmd =~ "-l " ]] && lflag=1 [[ $cmd =~ "-d " ]] && dflag=1 @@ -254,9 +243,7 @@ function verify_allow [[ -n $is_set ]] && str="S-\$@" tmp=${cmd#*@} code="$str${tmp% *}" - grep "permission $subcmd $dsname ([0-9]*) $code" \ - $NEW_HISTORY >/dev/null 2>&1 - if [[ $? != 0 ]]; then + if ! grep -q "permission $subcmd $dsname ([0-9]*) $code" $NEW_HISTORY; then log_note "Couldn't find $code in $NEW_HISTORY" return 1 fi @@ -265,9 +252,7 @@ function verify_allow [[ -n $is_set ]] && str="C-\$" tmp=${cmd#*-c} code="$str${tmp% *}" - grep "permission $subcmd $dsname ([0-9]*) $code" \ - $NEW_HISTORY >/dev/null 2>&1 - if [ $? != 0 ]]; then + if ! grep "permission $subcmd $dsname ([0-9]*) $code" $NEW_HISTORY; then log_note "Couldn't find $code in $NEW_HISTORY" return 1 fi @@ -275,22 +260,18 @@ function verify_allow str="u" [[ -n $is_set ]] && str="U" tmp=${cmd##*-u } - opt=$(echo $tmp | awk '{print $2}') + read -r _ opt _ <<<"$opt" uid=$(id -u ${tmp%% *}) if [[ -n $lflag ]]; then code="${str}l\$$uid $opt" - grep "permission $subcmd $dsname ([0-9]*) $code" \ - $NEW_HISTORY >/dev/null 2>&1 - if [ $? != 0 ]]; then + if grep -q "permission $subcmd $dsname ([0-9]*) $code" $NEW_HISTORY]; then log_note "Couldn't find $code in $NEW_HISTORY" return 1 fi fi if [[ -n $dflag ]]; then code="${str}d\$$uid $opt" - grep "permission $subcmd $dsname ([0-9]*) $code" \ - $NEW_HISTORY >/dev/null 2>&1 - if [ $? != 0 ]]; then + if grep -q "permission $subcmd $dsname ([0-9]*) $code" $NEW_HISTORY]; then log_note "Couldn't find $code in $NEW_HISTORY" return 1 fi @@ -299,22 +280,18 @@ function verify_allow str="g" [[ -n $is_set ]] && str="G" tmp=${cmd##*-g } - opt=$(echo $tmp | awk '{print $2}') + read -r _ opt _ <<<"$opt" gid=$(awk -F: "/^${tmp%% *}:/ {print \$3}" /etc/group) if [[ -n $lflag ]]; then code="${str}l\$$gid $opt" - grep "permission $subcmd $dsname ([0-9]*) $code" \ - $NEW_HISTORY >/dev/null 2>&1 - if [ $? != 0 ]]; then + if ! grep -q "permission $subcmd $dsname ([0-9]*) $code" $NEW_HISTORY; then log_note "Couldn't find $code in $NEW_HISTORY" return 1 fi fi if [[ -n $dflag ]]; then code="${str}d\$$gid $opt" - grep "permission $subcmd $dsname ([0-9]*) $code" \ - $NEW_HISTORY >/dev/null 2>&1 - if [ $? != 0 ]]; then + if ! grep -q "permission $subcmd $dsname ([0-9]*) $code" $NEW_HISTORY; then log_note "Couldn't find $code in $NEW_HISTORY" return 1 fi @@ -326,18 +303,14 @@ function verify_allow opt=${opt%% *} if [[ -n $lflag ]]; then code="${str}l\$ $opt" - grep "permission $subcmd $dsname ([0-9]*) $code" \ - $NEW_HISTORY >/dev/null 2>&1 - if [ $? != 0 ]]; then + if ! grep -q "permission $subcmd $dsname ([0-9]*) $code" $NEW_HISTORY; then log_note "Couldn't find $code in $NEW_HISTORY" return 1 fi fi if [[ -n $dflag ]]; then code="${str}d\$ $opt" - grep "permission $subcmd $dsname ([0-9]*) $code" \ - $NEW_HISTORY >/dev/null 2>&1 - if [ $? != 0 ]]; then + if ! grep -q "permission $subcmd $dsname ([0-9]*) $code" $NEW_HISTORY; then log_note "Couldn't find $code in $NEW_HISTORY" return 1 fi @@ -373,16 +346,14 @@ function verify_destroy [[ $dsname =~ "@" ]] && typeset is_snap=1 if [[ -n $is_snap ]]; then - grep "ioctl destroy_snaps" $NEW_HISTORY >/dev/null 2>&1 - if [[ $? != 0 ]]; then + if ! grep -q "ioctl destroy_snaps" $NEW_HISTORY; then log_note "Didn't find ioctl while destroying $dsname" return 1 fi fi # This should be present for datasets and snapshots alike - grep "destroy $dsname" $NEW_HISTORY >/dev/null 2>&1 - if [[ $? != 0 ]]; then + if ! grep -q "destroy $dsname" $NEW_HISTORY; then log_note "Didn't find \"destroy\" for $dsname" return 1 fi @@ -401,9 +372,7 @@ function verify_snapshot typeset dsname=${fullname%%@*} typeset snapname=${fullname##*@} - grep "\[txg:[0-9]*\] $subcmd $fullname ([0-9]*)" $NEW_HISTORY \ - >/dev/null 2>&1 - if [[ $? != 0 ]]; then + if ! grep -q "\[txg:[0-9]*\] $subcmd $fullname ([0-9]*)" $NEW_HISTORY; then log_note "Didn't find snapshot command for $fullname" return 1 fi @@ -411,8 +380,7 @@ function verify_snapshot # This works whether or not the snapshot was recursive for ds in $(zfs list -r -Ho name -t snapshot $dsname | \ grep "@$snapname"); do - grep "^[ ]* $ds$" $NEW_HISTORY >/dev/null 2>&1 - if [[ $? != 0 ]]; then + if ! grep -q "^[ ]* $ds$" $NEW_HISTORY; then log_note "Didn't find \"ioctl snapshot\" for $ds" return 1 fi diff --git a/tests/zfs-tests/tests/functional/history/setup.ksh b/tests/zfs-tests/tests/functional/history/setup.ksh index fc5cec3063a6..b756d4e76c83 100755 --- a/tests/zfs-tests/tests/functional/history/setup.ksh +++ b/tests/zfs-tests/tests/functional/history/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/hkdf/Makefile.am b/tests/zfs-tests/tests/functional/hkdf/Makefile.am deleted file mode 100644 index 8ac9053223a4..000000000000 --- a/tests/zfs-tests/tests/functional/hkdf/Makefile.am +++ /dev/null @@ -1,17 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/hkdf - -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - run_hkdf_test.ksh - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/hkdf - -pkgexec_PROGRAMS = \ - hkdf_test - -hkdf_test_SOURCES = hkdf_test.c -hkdf_test_LDADD = \ - $(abs_top_builddir)/lib/libzpool/libzpool.la diff --git a/tests/zfs-tests/tests/functional/hkdf/hkdf_test.c b/tests/zfs-tests/tests/functional/hkdf/hkdf_test.c index 5fff3e09ed04..24aeb0b224a7 100644 --- a/tests/zfs-tests/tests/functional/hkdf/hkdf_test.c +++ b/tests/zfs-tests/tests/functional/hkdf/hkdf_test.c @@ -18,29 +18,27 @@ */ #include -#include +#include #include #include #include -#define NELEMS(x) (sizeof (x) / sizeof ((x)[0])) - /* * Byte arrays are given as char pointers so that they * can be specified as strings. */ typedef struct hkdf_tv { /* test vector input values */ - char *ikm; + const char *ikm; uint_t ikm_len; - char *salt; + const char *salt; uint_t salt_len; - char *info; + const char *info; uint_t info_len; uint_t okm_len; /* expected output */ - char *okm; + const char *okm; } hkdf_tv_t; /* @@ -50,7 +48,7 @@ typedef struct hkdf_tv { * The current vectors were taken from: * https://www.kullo.net/blog/hkdf-sha-512-test-vectors/ */ -static hkdf_tv_t test_vectors[] = { +static const hkdf_tv_t test_vectors[] = { { .ikm = "\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b" "\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b" @@ -172,36 +170,33 @@ static hkdf_tv_t test_vectors[] = { }; static void -hexdump(char *str, uint8_t *src, uint_t len) +hexdump(const char *str, uint8_t *src, uint_t len) { - int i; - printf("\t%s\t", str); - for (i = 0; i < len; i++) { - printf("%02x", src[i] & 0xff); - } + for (int i = 0; i < len; i++) + printf("%02hhx", src[i]); printf("\n"); } static int -run_test(int i, hkdf_tv_t *tv) +run_test(int i, const hkdf_tv_t *tv) { int ret; - uint8_t okey[SHA512_DIGEST_LENGTH]; + uint8_t good[SHA512_DIGEST_LENGTH]; printf("TEST %d:\t", i); ret = hkdf_sha512((uint8_t *)tv->ikm, tv->ikm_len, (uint8_t *)tv->salt, - tv->salt_len, (uint8_t *)tv->info, tv->info_len, okey, tv->okm_len); + tv->salt_len, (uint8_t *)tv->info, tv->info_len, good, tv->okm_len); if (ret != 0) { printf("HKDF failed with error code %d\n", ret); return (ret); } - if (bcmp(okey, tv->okm, tv->okm_len) != 0) { + if (memcmp(good, tv->okm, tv->okm_len) != 0) { printf("Output Mismatch\n"); hexdump("Expected:", (uint8_t *)tv->okm, tv->okm_len); - hexdump("Actual: ", okey, tv->okm_len); + hexdump("Actual: ", good, tv->okm_len); return (1); } @@ -217,7 +212,7 @@ main(void) icp_init(); - for (i = 0; i < NELEMS(test_vectors); i++) { + for (i = 0; i < ARRAY_SIZE(test_vectors); i++) { ret = run_test(i, &test_vectors[i]); if (ret != 0) break; diff --git a/tests/zfs-tests/tests/functional/hkdf/run_hkdf_test.ksh b/tests/zfs-tests/tests/functional/hkdf/run_hkdf_test.ksh deleted file mode 100755 index 5fde0b837d0f..000000000000 --- a/tests/zfs-tests/tests/functional/hkdf/run_hkdf_test.ksh +++ /dev/null @@ -1,30 +0,0 @@ -#!/bin/ksh - -# -# This file and its contents are supplied under the terms of the -# Common Development and Distribution License ("CDDL"), version 1.0. -# You may only use this file in accordance with the terms of version -# 1.0 of the CDDL. -# -# A full copy of the text of the CDDL should have accompanied this -# source. A copy of the CDDL is also available via the Internet at -# http://www.illumos.org/license/CDDL. -# - -# -# Copyright (c) 2017 by Datto Inc. All rights reserved. -# - -. $STF_SUITE/include/libtest.shlib - -# -# DESCRIPTION: -# Call the hkdf_test tool to test ZFS's HKDF implementation against -# a few test vectors. -# - -log_assert "Run the tests for the HKDF algorithm." - -log_must $STF_SUITE/tests/functional/hkdf/hkdf_test - -log_pass "HKDF tests pass." diff --git a/tests/zfs-tests/tests/functional/hkdf/setup.ksh b/tests/zfs-tests/tests/functional/hkdf/setup.ksh deleted file mode 100755 index 2bdca1950d37..000000000000 --- a/tests/zfs-tests/tests/functional/hkdf/setup.ksh +++ /dev/null @@ -1,22 +0,0 @@ -#!/bin/ksh - -# -# This file and its contents are supplied under the terms of the -# Common Development and Distribution License ("CDDL"), version 1.0. -# You may only use this file in accordance with the terms of version -# 1.0 of the CDDL. -# -# A full copy of the text of the CDDL should have accompanied this -# source. A copy of the CDDL is also available via the Internet at -# http://www.illumos.org/license/CDDL. -# - -# -# Copyright (c) 2017 by Datto Inc. All rights reserved. -# - -. $STF_SUITE/include/libtest.shlib - -verify_runnable "global" - -log_pass diff --git a/tests/zfs-tests/tests/functional/inheritance/Makefile.am b/tests/zfs-tests/tests/functional/inheritance/Makefile.am deleted file mode 100644 index 3c624621f28a..000000000000 --- a/tests/zfs-tests/tests/functional/inheritance/Makefile.am +++ /dev/null @@ -1,57 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/inheritance -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - inherit_001_pos.ksh - -dist_pkgdata_DATA = \ - README.config \ - README.state \ - inherit.kshlib \ - config001.cfg \ - config002.cfg \ - config003.cfg \ - config004.cfg \ - config005.cfg \ - config006.cfg \ - config007.cfg \ - config008.cfg \ - config009.cfg \ - config010.cfg \ - config011.cfg \ - config012.cfg \ - config013.cfg \ - config014.cfg \ - config015.cfg \ - config016.cfg \ - config017.cfg \ - config018.cfg \ - config019.cfg \ - config020.cfg \ - config021.cfg \ - config022.cfg \ - config023.cfg \ - config024.cfg \ - state001.cfg \ - state002.cfg \ - state003.cfg \ - state004.cfg \ - state005.cfg \ - state006.cfg \ - state007.cfg \ - state008.cfg \ - state009.cfg \ - state010.cfg \ - state011.cfg \ - state012.cfg \ - state013.cfg \ - state014.cfg \ - state015.cfg \ - state016.cfg \ - state017.cfg \ - state018.cfg \ - state019.cfg \ - state020.cfg \ - state021.cfg \ - state022.cfg \ - state023.cfg \ - state024.cfg diff --git a/tests/zfs-tests/tests/functional/inheritance/README.config b/tests/zfs-tests/tests/functional/inheritance/README.config index 607f10a554ec..95fd363bac4f 100644 --- a/tests/zfs-tests/tests/functional/inheritance/README.config +++ b/tests/zfs-tests/tests/functional/inheritance/README.config @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/README.state b/tests/zfs-tests/tests/functional/inheritance/README.state index c1d3cc05d603..2fbd103f178e 100644 --- a/tests/zfs-tests/tests/functional/inheritance/README.state +++ b/tests/zfs-tests/tests/functional/inheritance/README.state @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/cleanup.ksh b/tests/zfs-tests/tests/functional/inheritance/cleanup.ksh index daac5c18a345..7fa851fc0ef6 100755 --- a/tests/zfs-tests/tests/functional/inheritance/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/inheritance/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config001.cfg b/tests/zfs-tests/tests/functional/inheritance/config001.cfg index 23616cbf3b7f..23819eb7ff8b 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config001.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config001.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config002.cfg b/tests/zfs-tests/tests/functional/inheritance/config002.cfg index 31294110f6ad..09f6bb3535cb 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config002.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config002.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config003.cfg b/tests/zfs-tests/tests/functional/inheritance/config003.cfg index 57a73b95d433..99c5f9f3e0d4 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config003.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config003.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config004.cfg b/tests/zfs-tests/tests/functional/inheritance/config004.cfg index 1052526379f0..196683d53d53 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config004.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config004.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config005.cfg b/tests/zfs-tests/tests/functional/inheritance/config005.cfg index d15beb2c4ca8..27f49c54e77f 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config005.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config005.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config006.cfg b/tests/zfs-tests/tests/functional/inheritance/config006.cfg index a059d17978f7..3cb630b931bb 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config006.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config006.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config007.cfg b/tests/zfs-tests/tests/functional/inheritance/config007.cfg index b416c9f17d20..8a7b47c1a93a 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config007.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config007.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config008.cfg b/tests/zfs-tests/tests/functional/inheritance/config008.cfg index 4f7e7b0ef7f8..c8105f9e47a6 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config008.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config008.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config009.cfg b/tests/zfs-tests/tests/functional/inheritance/config009.cfg index 4f7e7b0ef7f8..c8105f9e47a6 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config009.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config009.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config010.cfg b/tests/zfs-tests/tests/functional/inheritance/config010.cfg index 4f7e7b0ef7f8..c8105f9e47a6 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config010.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config010.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config011.cfg b/tests/zfs-tests/tests/functional/inheritance/config011.cfg index 57a73b95d433..99c5f9f3e0d4 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config011.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config011.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config012.cfg b/tests/zfs-tests/tests/functional/inheritance/config012.cfg index 57a73b95d433..99c5f9f3e0d4 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config012.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config012.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config013.cfg b/tests/zfs-tests/tests/functional/inheritance/config013.cfg index 1052526379f0..196683d53d53 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config013.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config013.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config014.cfg b/tests/zfs-tests/tests/functional/inheritance/config014.cfg index 1052526379f0..196683d53d53 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config014.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config014.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config015.cfg b/tests/zfs-tests/tests/functional/inheritance/config015.cfg index 31294110f6ad..09f6bb3535cb 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config015.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config015.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config016.cfg b/tests/zfs-tests/tests/functional/inheritance/config016.cfg index 31294110f6ad..09f6bb3535cb 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config016.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config016.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config017.cfg b/tests/zfs-tests/tests/functional/inheritance/config017.cfg index d15beb2c4ca8..27f49c54e77f 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config017.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config017.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config018.cfg b/tests/zfs-tests/tests/functional/inheritance/config018.cfg index d15beb2c4ca8..27f49c54e77f 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config018.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config018.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config019.cfg b/tests/zfs-tests/tests/functional/inheritance/config019.cfg index 23616cbf3b7f..23819eb7ff8b 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config019.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config019.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config020.cfg b/tests/zfs-tests/tests/functional/inheritance/config020.cfg index 23616cbf3b7f..23819eb7ff8b 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config020.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config020.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config021.cfg b/tests/zfs-tests/tests/functional/inheritance/config021.cfg index a059d17978f7..3cb630b931bb 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config021.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config021.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config022.cfg b/tests/zfs-tests/tests/functional/inheritance/config022.cfg index a059d17978f7..3cb630b931bb 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config022.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config022.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config023.cfg b/tests/zfs-tests/tests/functional/inheritance/config023.cfg index b416c9f17d20..8a7b47c1a93a 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config023.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config023.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/config024.cfg b/tests/zfs-tests/tests/functional/inheritance/config024.cfg index b416c9f17d20..8a7b47c1a93a 100644 --- a/tests/zfs-tests/tests/functional/inheritance/config024.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/config024.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/inherit.kshlib b/tests/zfs-tests/tests/functional/inheritance/inherit.kshlib index cd6bd5c027ca..8967a0fbc128 100644 --- a/tests/zfs-tests/tests/functional/inheritance/inherit.kshlib +++ b/tests/zfs-tests/tests/functional/inheritance/inherit.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -34,18 +34,12 @@ # function get_prop_src # property dataset { - typeset prop_val typeset prop=$1 typeset dataset=$2 - prop_val=`zfs get -H -o source $prop $dataset` - - if [[ $? -ne 0 ]]; then - log_fail "Unable to determine the source of $prop " \ + zfs get -H -o source $prop $dataset || + log_fail "Unable to determine the source of $prop" \ "property for dataset $dataset" - else - echo $prop_val - fi } # @@ -63,7 +57,7 @@ function verify_prop_src # child_dataset property expected_src typeset prop=$2 typeset expected=$3 - prop_src=`get_prop_src $prop $target` + prop_src=$(get_prop_src $prop $target) # # Rather than just checking if $prop_src == $expected @@ -105,7 +99,7 @@ function set_n_verify_prop #property value dataset typeset dataset=$3 zfs set $prop=$prop_val $dataset - check_val=`get_prop $prop $dataset` + check_val=$(get_prop $prop $dataset) if [[ $check_val != $prop_val ]]; then log_fail "Property $prop of $dataset has value $check_val"\ diff --git a/tests/zfs-tests/tests/functional/inheritance/inherit_001_pos.ksh b/tests/zfs-tests/tests/functional/inheritance/inherit_001_pos.ksh index 7c5b81287736..e525c51344ad 100755 --- a/tests/zfs-tests/tests/functional/inheritance/inherit_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/inheritance/inherit_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -332,14 +332,12 @@ function scan_state { #state-file log_note "No operation specified" else export __ZFS_POOL_RESTRICT="TESTPOOL" - log_must zfs unmount -a + log_must_busy zfs unmount -a unset __ZFS_POOL_RESTRICT for p in ${prop[i]} ${prop[((i+1))]}; do zfs $op $p $target - ret=$? - check_failure $ret "zfs $op $p \ - $target" + check_failure $? "zfs $op $p $target" done fi for check_obj in $list; do @@ -349,16 +347,14 @@ function scan_state { #state-file # check_failure to keep journal small verify_prop_src $check_obj $p \ $final_src - ret=$? - check_failure $ret "verify" \ + check_failure $? "verify" \ "_prop_src $check_obj $p" \ "$final_src" # Again, to keep journal size down. verify_prop_val $p $check_obj \ $final_src $j - ret=$? - check_failure $ret "verify" \ + check_failure $? "verify" \ "_prop_val $check_obj $p" \ "$final_src" done @@ -388,12 +384,12 @@ set -A prop "checksum" "" \ # above must have a corresponding entry in the two arrays below. # -set -A def_val "on" "off" "on" \ +set -A def_val "on" "on" "on" \ "off" "" \ "hidden" \ "off" -set -A local_val "off" "on" "off" \ +set -A local_val "off" "off" "off" \ "on" "" \ "visible" \ "off" diff --git a/tests/zfs-tests/tests/functional/inheritance/state001.cfg b/tests/zfs-tests/tests/functional/inheritance/state001.cfg index afaf3cccfe14..4fbdcd6e212b 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state001.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state001.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state002.cfg b/tests/zfs-tests/tests/functional/inheritance/state002.cfg index 5b2f2d2dd98a..f368be370c24 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state002.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state002.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state003.cfg b/tests/zfs-tests/tests/functional/inheritance/state003.cfg index 5780065ace88..06b092dd3800 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state003.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state003.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state004.cfg b/tests/zfs-tests/tests/functional/inheritance/state004.cfg index e62979a04b7d..e679e449d850 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state004.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state004.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state005.cfg b/tests/zfs-tests/tests/functional/inheritance/state005.cfg index 097459ffedb9..da7a4f9d4e86 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state005.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state005.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state006.cfg b/tests/zfs-tests/tests/functional/inheritance/state006.cfg index 9863b298fe35..93d90c86cbbb 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state006.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state006.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state007.cfg b/tests/zfs-tests/tests/functional/inheritance/state007.cfg index 731ec95a250b..871287b15878 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state007.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state007.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state008.cfg b/tests/zfs-tests/tests/functional/inheritance/state008.cfg index 7385c98c3c07..4b1cfbd09cbb 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state008.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state008.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state009.cfg b/tests/zfs-tests/tests/functional/inheritance/state009.cfg index 3fb2841a2cc1..19ee507ef80b 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state009.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state009.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state010.cfg b/tests/zfs-tests/tests/functional/inheritance/state010.cfg index 4c51e808b1fa..57c29d6e3e04 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state010.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state010.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state011.cfg b/tests/zfs-tests/tests/functional/inheritance/state011.cfg index f6e791ee73c1..5eb33af00b31 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state011.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state011.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state012.cfg b/tests/zfs-tests/tests/functional/inheritance/state012.cfg index 47898244b025..afe1d0d947fe 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state012.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state012.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state013.cfg b/tests/zfs-tests/tests/functional/inheritance/state013.cfg index ae5919292623..49f2c242cb76 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state013.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state013.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state014.cfg b/tests/zfs-tests/tests/functional/inheritance/state014.cfg index 3770c5093e52..e6418e3e5a80 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state014.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state014.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state015.cfg b/tests/zfs-tests/tests/functional/inheritance/state015.cfg index fb1fc3907b76..c8228f080925 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state015.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state015.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state016.cfg b/tests/zfs-tests/tests/functional/inheritance/state016.cfg index f17154c74519..c830287d2a76 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state016.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state016.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state017.cfg b/tests/zfs-tests/tests/functional/inheritance/state017.cfg index a993eed5f78c..09d061340af3 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state017.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state017.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state018.cfg b/tests/zfs-tests/tests/functional/inheritance/state018.cfg index 6fd08c6fb8b1..f718c83395d6 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state018.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state018.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state019.cfg b/tests/zfs-tests/tests/functional/inheritance/state019.cfg index 67b488f84a09..484e9e02c680 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state019.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state019.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state020.cfg b/tests/zfs-tests/tests/functional/inheritance/state020.cfg index df8ba7dd5246..5a4ec6a6f7e6 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state020.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state020.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state021.cfg b/tests/zfs-tests/tests/functional/inheritance/state021.cfg index f61472eff197..231530a14776 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state021.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state021.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state022.cfg b/tests/zfs-tests/tests/functional/inheritance/state022.cfg index 22462f97839a..990215766aad 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state022.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state022.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state023.cfg b/tests/zfs-tests/tests/functional/inheritance/state023.cfg index 8219a2abf3c3..4f9091c67908 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state023.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state023.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inheritance/state024.cfg b/tests/zfs-tests/tests/functional/inheritance/state024.cfg index 848ae1437d06..cd67aaf6728f 100644 --- a/tests/zfs-tests/tests/functional/inheritance/state024.cfg +++ b/tests/zfs-tests/tests/functional/inheritance/state024.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inuse/Makefile.am b/tests/zfs-tests/tests/functional/inuse/Makefile.am deleted file mode 100644 index c541e57eaa31..000000000000 --- a/tests/zfs-tests/tests/functional/inuse/Makefile.am +++ /dev/null @@ -1,14 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/inuse -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - inuse_001_pos.ksh \ - inuse_003_pos.ksh \ - inuse_004_pos.ksh \ - inuse_005_pos.ksh \ - inuse_006_pos.ksh \ - inuse_007_pos.ksh \ - inuse_008_pos.ksh \ - inuse_009_pos.ksh - -dist_pkgdata_DATA = \ - inuse.cfg diff --git a/tests/zfs-tests/tests/functional/inuse/inuse.cfg b/tests/zfs-tests/tests/functional/inuse/inuse.cfg index 631ace7ab795..2514f89e78e5 100644 --- a/tests/zfs-tests/tests/functional/inuse/inuse.cfg +++ b/tests/zfs-tests/tests/functional/inuse/inuse.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inuse/inuse_001_pos.ksh b/tests/zfs-tests/tests/functional/inuse/inuse_001_pos.ksh index f824661c0067..17e1169175e7 100755 --- a/tests/zfs-tests/tests/functional/inuse/inuse_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/inuse/inuse_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -49,7 +49,7 @@ function cleanup # Remove dump device. # if [[ -n $PREVDUMPDEV ]]; then - log_must dumpadm -u -d $PREVDUMPDEV > /dev/null + log_must eval "dumpadm -u -d $PREVDUMPDEV > /dev/null" fi destroy_pool $TESTPOOL @@ -61,16 +61,16 @@ log_onexit cleanup typeset dumpdev="" -PREVDUMPDEV=`dumpadm | grep "Dump device" | awk '{print $3}'` +PREVDUMPDEV=`dumpadm | awk '/Dump device/ {print $3}'` log_note "Zero $FS_DISK0" log_must cleanup_devices $FS_DISK0 log_note "Configuring $rawdisk0 as dump device" -log_must dumpadm -d $rawdisk0 > /dev/null +log_must eval "dumpadm -d $rawdisk0 > /dev/null" log_note "Confirm that dump device has been setup" -dumpdev=`dumpadm | grep "Dump device" | awk '{print $3}'` +dumpdev=`dumpadm | awk '/Dump device/ {print $3}'` [[ -z "$dumpdev" ]] && log_untested "No dump device has been configured" [[ "$dumpdev" != "$rawdisk0" ]] && \ diff --git a/tests/zfs-tests/tests/functional/inuse/inuse_003_pos.ksh b/tests/zfs-tests/tests/functional/inuse/inuse_003_pos.ksh index 07d6ac17557c..385b88bdc6b6 100755 --- a/tests/zfs-tests/tests/functional/inuse/inuse_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/inuse/inuse_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -58,25 +58,21 @@ function cleanup log_note "Kill off ufsdump process if still running" kill -0 $PIDUFSDUMP > /dev/null 2>&1 && \ - log_must kill -9 $PIDUFSDUMP > /dev/null 2>&1 + log_must eval "kill -9 $PIDUFSDUMP" # # Note: It would appear that ufsdump spawns a number of processes # which are not killed when the $PIDUFSDUMP is whacked. So best bet # is to find the rest of the them and deal with them individually. # - for all in `pgrep ufsdump` - do - kill -9 $all > /dev/null 2>&1 - done + kill -9 `pgrep ufsdump` > /dev/null 2>&1 log_note "Kill off ufsrestore process if still running" kill -0 $PIDUFSRESTORE > /dev/null 2>&1 && \ - log_must kill -9 $PIDUFSRESTORE > /dev/null 2>&1 + log_must eval "kill -9 $PIDUFSRESTORE" ismounted $UFSMP ufs && log_must umount $UFSMP - rm -rf $UFSMP - rm -rf $TESTDIR + rm -rf $UFSMP $TESTDIR # # Tidy up the disks we used. @@ -96,8 +92,8 @@ typeset -i filenum=0 typeset cwd="" log_note "Make a ufs filesystem on source $rawdisk1" -new_fs $rawdisk1 > /dev/null 2>&1 -(($? != 0)) && log_untested "Unable to create ufs filesystem on $rawdisk1" +new_fs $rawdisk1 > /dev/null 2>&1 || + log_untested "Unable to create ufs filesystem on $rawdisk1" log_must mkdir -p $UFSMP @@ -108,9 +104,9 @@ log_note "Now create some directories and files to be ufsdump'ed" while (($dirnum <= 2)); do log_must mkdir $bigdir${dirnum} while (( $filenum <= 2 )); do - file_write -o create -f $bigdir${dirnum}/file${filenum} \ + if ! file_write -o create -f $bigdir${dirnum}/file${filenum} \ -b $BLOCK_SIZE -c $BLOCK_COUNT - if [[ $? -ne 0 ]]; then + then if [[ $dirnum -lt 3 ]]; then log_fail "file_write only wrote" \ "<(( $dirnum * 3 + $filenum ))>" \ @@ -139,9 +135,7 @@ log_note "Attempt to take the source device in use by ufsdump as spare device" log_mustnot zpool create $TESTPOOL1 "$FS_DISK2" spare "$disk1" log_mustnot poolexists $TESTPOOL1 -wait $PIDUFSDUMP -typeset -i retval=$? -(($retval != 0)) && log_fail "ufsdump failed with error code $ret_val" +wait $PIDUFSDUMP || log_fail "ufsdump failed with error code $?" log_must mount $disk1 $UFSMP diff --git a/tests/zfs-tests/tests/functional/inuse/inuse_004_pos.ksh b/tests/zfs-tests/tests/functional/inuse/inuse_004_pos.ksh index a9725e06dcf0..054c6e5ac8bd 100755 --- a/tests/zfs-tests/tests/functional/inuse/inuse_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/inuse/inuse_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -52,10 +52,10 @@ function cleanup # to work correctly. So its reproduced below. Still need to fully # understand why default_cleanup does not work correctly from here. # - log_must zfs umount $TESTPOOL/$TESTFS + log_must zfs umount $TESTPOOL/$TESTFS - rm -rf $TESTDIR || \ - log_unresolved Could not remove $TESTDIR + rm -rf $TESTDIR || + log_unresolved Could not remove $TESTDIR log_must zfs destroy $TESTPOOL/$TESTFS destroy_pool $TESTPOOL @@ -71,21 +71,11 @@ function mini_format if is_linux; then parted $disk -s -- mklabel gpt - typeset -i retval=$? elif is_freebsd; then gpart create -s gpt $disk - typeset -i retval=$? else - typeset format_file=$TEST_BASE_DIR/format_in.$$.1 - echo "partition" > $format_file - echo "modify" >> $format_file - - format -e -s -d $disk -f $format_file - typeset -i retval=$? - - rm -rf $format_file + format -e -s -d $disk -f <(printf '%s\n' partition modify) fi - return $retval } log_assert "format will disallow modification of a mounted zfs disk partition"\ diff --git a/tests/zfs-tests/tests/functional/inuse/inuse_005_pos.ksh b/tests/zfs-tests/tests/functional/inuse/inuse_005_pos.ksh index afe30d059997..f50fd2ff91de 100755 --- a/tests/zfs-tests/tests/functional/inuse/inuse_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/inuse/inuse_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inuse/inuse_006_pos.ksh b/tests/zfs-tests/tests/functional/inuse/inuse_006_pos.ksh index 9657322526e7..0d2c2ded0260 100755 --- a/tests/zfs-tests/tests/functional/inuse/inuse_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/inuse/inuse_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -77,7 +77,7 @@ set -A vdevs "" "mirror" "raidz" "raidz1" "raidz2" typeset -i i=0 -PREVDUMPDEV=`dumpadm | grep "Dump device" | awk '{print $3}'` +PREVDUMPDEV=`dumpadm | awk '/Dump device/ {print $3}'` unset NOINUSE_CHECK while (( i < ${#vdevs[*]} )); do diff --git a/tests/zfs-tests/tests/functional/inuse/inuse_007_pos.ksh b/tests/zfs-tests/tests/functional/inuse/inuse_007_pos.ksh index b96b80890ed8..d3e580a6f533 100755 --- a/tests/zfs-tests/tests/functional/inuse/inuse_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/inuse/inuse_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -82,7 +82,7 @@ set -A vdevs "" "mirror" "raidz" "raidz1" "raidz2" typeset -i i=0 -PREVDUMPDEV=`dumpadm | grep "Dump device" | awk '{print $3}'` +PREVDUMPDEV=`dumpadm | awk '/Dump device/ {print $3}'` while (( i < ${#vdevs[*]} )); do typeset spare="spare $sdisks" diff --git a/tests/zfs-tests/tests/functional/inuse/inuse_008_pos.ksh b/tests/zfs-tests/tests/functional/inuse/inuse_008_pos.ksh index d60ebcee1554..68bf19f0abce 100755 --- a/tests/zfs-tests/tests/functional/inuse/inuse_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/inuse/inuse_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inuse/inuse_009_pos.ksh b/tests/zfs-tests/tests/functional/inuse/inuse_009_pos.ksh index 54d201ad6291..64fe05b422dd 100755 --- a/tests/zfs-tests/tests/functional/inuse/inuse_009_pos.ksh +++ b/tests/zfs-tests/tests/functional/inuse/inuse_009_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/inuse/setup.ksh b/tests/zfs-tests/tests/functional/inuse/setup.ksh index 962a0c98dc7e..14cdb6798a9f 100755 --- a/tests/zfs-tests/tests/functional/inuse/setup.ksh +++ b/tests/zfs-tests/tests/functional/inuse/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/io/Makefile.am b/tests/zfs-tests/tests/functional/io/Makefile.am deleted file mode 100644 index 44c0d02d6efe..000000000000 --- a/tests/zfs-tests/tests/functional/io/Makefile.am +++ /dev/null @@ -1,13 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/io -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - sync.ksh \ - psync.ksh \ - libaio.ksh \ - io_uring.ksh \ - posixaio.ksh \ - mmap.ksh - -dist_pkgdata_DATA = \ - io.cfg diff --git a/tests/zfs-tests/tests/functional/io/cleanup.ksh b/tests/zfs-tests/tests/functional/io/cleanup.ksh index 0031a26c3cec..5bd79c0070fa 100755 --- a/tests/zfs-tests/tests/functional/io/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/io/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/io/io_uring.ksh b/tests/zfs-tests/tests/functional/io/io_uring.ksh index 2d2b18f8bb5b..47e439d0f4d5 100755 --- a/tests/zfs-tests/tests/functional/io/io_uring.ksh +++ b/tests/zfs-tests/tests/functional/io/io_uring.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -40,11 +40,11 @@ verify_runnable "global" -if [[ $(linux_version) -lt $(linux_version "5.1") ]]; then +if ! $(grep -q "CONFIG_IO_URING=y" /boot/config-$(uname -r)); then log_unsupported "Requires io_uring support" fi -fio --ioengine=io_uring --parse-only || log_unsupported "io_uring support required" +fio --ioengine=io_uring --parse-only || log_unsupported "fio io_uring support required" function cleanup { diff --git a/tests/zfs-tests/tests/functional/io/libaio.ksh b/tests/zfs-tests/tests/functional/io/libaio.ksh index c434ad90ddd7..242b2ca8de30 100755 --- a/tests/zfs-tests/tests/functional/io/libaio.ksh +++ b/tests/zfs-tests/tests/functional/io/libaio.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -39,6 +39,8 @@ verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + function cleanup { log_must rm -f "$mntpnt/rw*" diff --git a/tests/zfs-tests/tests/functional/io/mmap.ksh b/tests/zfs-tests/tests/functional/io/mmap.ksh index e9600787a8bc..112267ee20d8 100755 --- a/tests/zfs-tests/tests/functional/io/mmap.ksh +++ b/tests/zfs-tests/tests/functional/io/mmap.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -37,7 +37,7 @@ # 2. Repeat the test with additional fio(1) options. # -if ! compare_version_gte $(fio --version) "fio-2.3"; then +if ! compare_version_gte "$(fio --version)" "fio-2.3"; then log_unsupported "Requires fio-2.3 or newer" fi diff --git a/tests/zfs-tests/tests/functional/io/posixaio.ksh b/tests/zfs-tests/tests/functional/io/posixaio.ksh index 0758164c1673..52331027cdd2 100755 --- a/tests/zfs-tests/tests/functional/io/posixaio.ksh +++ b/tests/zfs-tests/tests/functional/io/posixaio.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -39,6 +39,8 @@ verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + function cleanup { log_must rm -f "$mntpnt/rw*" diff --git a/tests/zfs-tests/tests/functional/io/psync.ksh b/tests/zfs-tests/tests/functional/io/psync.ksh index efeb1103d8bb..5227fcbdd449 100755 --- a/tests/zfs-tests/tests/functional/io/psync.ksh +++ b/tests/zfs-tests/tests/functional/io/psync.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -39,6 +39,8 @@ verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + function cleanup { log_must rm -f "/$TESTPOOL/rw*" diff --git a/tests/zfs-tests/tests/functional/io/setup.ksh b/tests/zfs-tests/tests/functional/io/setup.ksh index ff72fc3db7c0..82aaf5bc91b5 100755 --- a/tests/zfs-tests/tests/functional/io/setup.ksh +++ b/tests/zfs-tests/tests/functional/io/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/io/sync.ksh b/tests/zfs-tests/tests/functional/io/sync.ksh index 83f346c6972a..76248a38a519 100755 --- a/tests/zfs-tests/tests/functional/io/sync.ksh +++ b/tests/zfs-tests/tests/functional/io/sync.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -39,6 +39,8 @@ verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + function cleanup { log_must rm -f "$mntpnt/rw*" diff --git a/tests/zfs-tests/tests/functional/l2arc/Makefile.am b/tests/zfs-tests/tests/functional/l2arc/Makefile.am deleted file mode 100644 index 09f4c1d0d74f..000000000000 --- a/tests/zfs-tests/tests/functional/l2arc/Makefile.am +++ /dev/null @@ -1,15 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/l2arc -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - l2arc_arcstats_pos.ksh \ - l2arc_l2miss_pos.ksh \ - l2arc_mfuonly_pos.ksh \ - persist_l2arc_001_pos.ksh \ - persist_l2arc_002_pos.ksh \ - persist_l2arc_003_neg.ksh \ - persist_l2arc_004_pos.ksh \ - persist_l2arc_005_pos.ksh - -dist_pkgdata_DATA = \ - l2arc.cfg diff --git a/tests/zfs-tests/tests/functional/l2arc/l2arc_arcstats_pos.ksh b/tests/zfs-tests/tests/functional/l2arc/l2arc_arcstats_pos.ksh index 3e76347b029a..69d60ab8bb90 100755 --- a/tests/zfs-tests/tests/functional/l2arc/l2arc_arcstats_pos.ksh +++ b/tests/zfs-tests/tests/functional/l2arc/l2arc_arcstats_pos.ksh @@ -40,6 +40,8 @@ verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + log_assert "L2ARC MFU/MRU arcstats do not leak." function cleanup diff --git a/tests/zfs-tests/tests/functional/l2arc/l2arc_l2miss_pos.ksh b/tests/zfs-tests/tests/functional/l2arc/l2arc_l2miss_pos.ksh index 783484f52c13..c9d5d7ffe1f1 100755 --- a/tests/zfs-tests/tests/functional/l2arc/l2arc_l2miss_pos.ksh +++ b/tests/zfs-tests/tests/functional/l2arc/l2arc_l2miss_pos.ksh @@ -38,6 +38,8 @@ verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + log_assert "l2arc_misses does not increment upon reads from a pool without l2arc." function cleanup diff --git a/tests/zfs-tests/tests/functional/l2arc/l2arc_mfuonly_pos.ksh b/tests/zfs-tests/tests/functional/l2arc/l2arc_mfuonly_pos.ksh index 5d0198c90c16..f2bada0ebbec 100755 --- a/tests/zfs-tests/tests/functional/l2arc/l2arc_mfuonly_pos.ksh +++ b/tests/zfs-tests/tests/functional/l2arc/l2arc_mfuonly_pos.ksh @@ -39,6 +39,8 @@ verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + log_assert "l2arc_mfuonly does not cache MRU buffers." function cleanup diff --git a/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_001_pos.ksh b/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_001_pos.ksh index 0a9049490c71..6f7b9aff7c38 100755 --- a/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_001_pos.ksh @@ -49,6 +49,8 @@ verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + log_assert "Persistent L2ARC with an unencrypted ZFS file system succeeds." function cleanup @@ -87,8 +89,7 @@ arcstat_quiescence_noecho l2_size log_must zpool export $TESTPOOL arcstat_quiescence_noecho l2_feeds -typeset l2_dh_log_blk=$(zdb -l $VDEV_CACHE | grep log_blk_count | \ - awk '{print $2}') +typeset l2_dh_log_blk=$(zdb -l $VDEV_CACHE | awk '/log_blk_count/ {print $2}') typeset l2_rebuild_log_blk_start=$(get_arcstat l2_rebuild_log_blks) diff --git a/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_002_pos.ksh b/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_002_pos.ksh index 93982e6c605b..3b893d28da6a 100755 --- a/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_002_pos.ksh @@ -52,6 +52,8 @@ verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + log_assert "Persistent L2ARC with an encrypted ZFS file system succeeds." function cleanup @@ -90,8 +92,7 @@ arcstat_quiescence_noecho l2_size log_must zpool export $TESTPOOL arcstat_quiescence_noecho l2_feeds -typeset l2_dh_log_blk=$(zdb -l $VDEV_CACHE | grep log_blk_count | \ - awk '{print $2}') +typeset l2_dh_log_blk=$(zdb -l $VDEV_CACHE | awk '/log_blk_count/ {print $2}') typeset l2_rebuild_log_blk_start=$(get_arcstat l2_rebuild_log_blks) diff --git a/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_003_neg.ksh b/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_003_neg.ksh index fe35c8fc4500..f8dc2b108f0d 100755 --- a/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_003_neg.ksh +++ b/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_003_neg.ksh @@ -38,6 +38,8 @@ verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + log_assert "Persistent L2ARC fails as expected when L2ARC_REBUILD_ENABLED = 0." function cleanup diff --git a/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_004_pos.ksh b/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_004_pos.ksh index b40703180687..8a572c26469c 100755 --- a/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_004_pos.ksh @@ -40,6 +40,8 @@ verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + log_assert "Off/onlining an L2ARC device results in rebuilding L2ARC, vdev not present." function cleanup @@ -78,8 +80,7 @@ log_must zpool export $TESTPOOL arcstat_quiescence_noecho l2_feeds typeset l2_rebuild_log_blk_start=$(get_arcstat l2_rebuild_log_blks) -typeset l2_dh_log_blk=$(zdb -l $VDEV_CACHE | grep log_blk_count | \ - awk '{print $2}') +typeset l2_dh_log_blk=$(zdb -l $VDEV_CACHE | awk '/log_blk_count/ {print $2}') log_must zpool import -d $VDIR $TESTPOOL log_must zpool online $TESTPOOL $VDEV_CACHE diff --git a/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_005_pos.ksh b/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_005_pos.ksh index 8ad648519f5c..9663437c6597 100755 --- a/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/l2arc/persist_l2arc_005_pos.ksh @@ -39,6 +39,8 @@ verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + log_assert "Off/onlining an L2ARC device results in rebuilding L2ARC, vdev present." function cleanup @@ -75,8 +77,7 @@ log_must zpool offline $TESTPOOL $VDEV_CACHE arcstat_quiescence_noecho l2_size typeset l2_rebuild_log_blk_start=$(get_arcstat l2_rebuild_log_blks) -typeset l2_dh_log_blk=$(zdb -l $VDEV_CACHE | grep log_blk_count | \ - awk '{print $2}') +typeset l2_dh_log_blk=$(zdb -l $VDEV_CACHE | awk '/log_blk_count/ {print $2}') log_must zpool online $TESTPOOL $VDEV_CACHE arcstat_quiescence_noecho l2_size diff --git a/tests/zfs-tests/tests/functional/large_files/Makefile.am b/tests/zfs-tests/tests/functional/large_files/Makefile.am deleted file mode 100644 index 0e471533090c..000000000000 --- a/tests/zfs-tests/tests/functional/large_files/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/large_files -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - large_files_001_pos.ksh \ - large_files_002_pos.ksh diff --git a/tests/zfs-tests/tests/functional/large_files/cleanup.ksh b/tests/zfs-tests/tests/functional/large_files/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/large_files/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/large_files/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/large_files/large_files_001_pos.ksh b/tests/zfs-tests/tests/functional/large_files/large_files_001_pos.ksh index f59603724e76..aebc8b2647d5 100755 --- a/tests/zfs-tests/tests/functional/large_files/large_files_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/large_files/large_files_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/large_files/large_files_002_pos.ksh b/tests/zfs-tests/tests/functional/large_files/large_files_002_pos.ksh index 255a8f8b5ad6..e0014d64350c 100755 --- a/tests/zfs-tests/tests/functional/large_files/large_files_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/large_files/large_files_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/large_files/setup.ksh b/tests/zfs-tests/tests/functional/large_files/setup.ksh index 3a45ec8d510d..d617f361b24d 100755 --- a/tests/zfs-tests/tests/functional/large_files/setup.ksh +++ b/tests/zfs-tests/tests/functional/large_files/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/largest_pool/Makefile.am b/tests/zfs-tests/tests/functional/largest_pool/Makefile.am deleted file mode 100644 index 5f1473dede70..000000000000 --- a/tests/zfs-tests/tests/functional/largest_pool/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/largest_pool -dist_pkgdata_SCRIPTS = \ - largest_pool_001_pos.ksh - -dist_pkgdata_DATA = \ - largest_pool.cfg diff --git a/tests/zfs-tests/tests/functional/largest_pool/largest_pool.cfg b/tests/zfs-tests/tests/functional/largest_pool/largest_pool.cfg index 13c157789dd7..9306c799ec54 100644 --- a/tests/zfs-tests/tests/functional/largest_pool/largest_pool.cfg +++ b/tests/zfs-tests/tests/functional/largest_pool/largest_pool.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/largest_pool/largest_pool_001_pos.ksh b/tests/zfs-tests/tests/functional/largest_pool/largest_pool_001_pos.ksh index 6b51598d7cca..85f25c7618c6 100755 --- a/tests/zfs-tests/tests/functional/largest_pool/largest_pool_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/largest_pool/largest_pool_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -117,15 +117,12 @@ log_onexit cleanup # units for 'df'. It must be greater than one. # ----------------------------------------------------------------------- typeset str -typeset -i ret for volsize in $VOLSIZES; do log_note "Create a pool which will contain a volume device" - create_pool $TESTPOOL2 "$DISKS" + log_must create_pool $TESTPOOL2 "$DISKS" log_note "Create a volume device of desired sizes: $volsize" - str=$(zfs create -sV $volsize $TESTPOOL2/$TESTVOL 2>&1) - ret=$? - if (( ret != 0 )); then + if ! str=$(zfs create -sV $volsize $TESTPOOL2/$TESTVOL 2>&1); then if [[ is_32bit && \ $str == *${VOL_LIMIT_KEYWORD1}* || \ $str == *${VOL_LIMIT_KEYWORD2}* || \ @@ -140,7 +137,7 @@ for volsize in $VOLSIZES; do block_device_wait log_note "Create the largest pool allowed using the volume vdev" - create_pool $TESTPOOL "$VOL_PATH" + log_must create_pool $TESTPOOL "$VOL_PATH" log_note "Create a zfs file system in the largest pool" log_must zfs create $TESTPOOL/$TESTFS diff --git a/tests/zfs-tests/tests/functional/libzfs/Makefile.am b/tests/zfs-tests/tests/functional/libzfs/Makefile.am deleted file mode 100644 index 53cb635444ab..000000000000 --- a/tests/zfs-tests/tests/functional/libzfs/Makefile.am +++ /dev/null @@ -1,17 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/libzfs - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/libzfs - -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - libzfs_input.ksh - -many_fds_LDADD = \ - $(abs_top_builddir)/lib/libzfs/libzfs.la - -pkgexec_PROGRAMS = many_fds -many_fds_SOURCES = many_fds.c - diff --git a/tests/zfs-tests/tests/functional/libzfs/cleanup.ksh b/tests/zfs-tests/tests/functional/libzfs/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/libzfs/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/libzfs/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/libzfs/many_fds.c b/tests/zfs-tests/tests/functional/libzfs/many_fds.c index 6def57c5a51a..6e0b1fd4b4aa 100644 --- a/tests/zfs-tests/tests/functional/libzfs/many_fds.c +++ b/tests/zfs-tests/tests/functional/libzfs/many_fds.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -22,15 +22,13 @@ /* * Copyright (C) 2015 STRATO AG. */ -#include -#include -#include -#include -#include +#include #include #include #include -#include +#include +#include +#include /* * Check if libzfs works with more than 255 held file handles. @@ -38,35 +36,23 @@ int main(void) { - int i; - struct rlimit limit; - libzfs_handle_t *h; - - limit.rlim_cur = 65535; - limit.rlim_max = 65535; - - if (setrlimit(RLIMIT_NOFILE, &limit) != 0) { - (void) printf("many_fds: setrlimit() failed with errno=%d\n", - errno); - exit(1); - } + struct rlimit limit = { + .rlim_cur = 64 * 1024, + .rlim_max = 64 * 1024, + }; + if (setrlimit(RLIMIT_NOFILE, &limit) != 0) + err(1, "setrlimit()"); - for (i = 0; i < 255; ++i) { - int fd = open("/dev/null", O_RDONLY); - if (fd == -1) { - (void) printf("open failed with errno=%d\n", errno); - return (1); - } - } + int fd = open("/dev/null", O_RDONLY); + if (fd == -1) + err(1, "open()"); + for (int i = 0; i < limit.rlim_cur / 2; ++i) + if (dup(fd) == -1) + err(1, "dup()"); - h = libzfs_init(); + libzfs_handle_t *h = libzfs_init(); + if (h == NULL) + err(1, "libzfs_init()"); - if (h != NULL) { - libzfs_fini(h); - return (0); - } else { - (void) printf("many_fds: libzfs_init() failed with errno=%d\n", - errno); - return (1); - } + libzfs_fini(h); } diff --git a/tests/zfs-tests/tests/functional/libzfs/setup.ksh b/tests/zfs-tests/tests/functional/libzfs/setup.ksh index fc5cec3063a6..b756d4e76c83 100755 --- a/tests/zfs-tests/tests/functional/libzfs/setup.ksh +++ b/tests/zfs-tests/tests/functional/libzfs/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/limits/Makefile.am b/tests/zfs-tests/tests/functional/limits/Makefile.am deleted file mode 100644 index 72455922485d..000000000000 --- a/tests/zfs-tests/tests/functional/limits/Makefile.am +++ /dev/null @@ -1,9 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/limits -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - filesystem_count.ksh \ - filesystem_limit.ksh \ - snapshot_count.ksh \ - snapshot_limit.ksh - diff --git a/tests/zfs-tests/tests/functional/link_count/Makefile.am b/tests/zfs-tests/tests/functional/link_count/Makefile.am deleted file mode 100644 index bfb7154a6518..000000000000 --- a/tests/zfs-tests/tests/functional/link_count/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/link_count -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - link_count_001.ksh \ - link_count_root_inode.ksh diff --git a/tests/zfs-tests/tests/functional/link_count/cleanup.ksh b/tests/zfs-tests/tests/functional/link_count/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/link_count/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/link_count/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/link_count/link_count_001.ksh b/tests/zfs-tests/tests/functional/link_count/link_count_001.ksh index 3ab3fbef8df8..d3020c99c95c 100755 --- a/tests/zfs-tests/tests/functional/link_count/link_count_001.ksh +++ b/tests/zfs-tests/tests/functional/link_count/link_count_001.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/link_count/setup.ksh b/tests/zfs-tests/tests/functional/link_count/setup.ksh index d275e063b13a..76aa23035dd9 100755 --- a/tests/zfs-tests/tests/functional/link_count/setup.ksh +++ b/tests/zfs-tests/tests/functional/link_count/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/log_spacemap/Makefile.am b/tests/zfs-tests/tests/functional/log_spacemap/Makefile.am deleted file mode 100644 index a1e523426c6a..000000000000 --- a/tests/zfs-tests/tests/functional/log_spacemap/Makefile.am +++ /dev/null @@ -1,2 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/log_spacemap -dist_pkgdata_SCRIPTS = log_spacemap_import_logs.ksh diff --git a/tests/zfs-tests/tests/functional/log_spacemap/log_spacemap_import_logs.ksh b/tests/zfs-tests/tests/functional/log_spacemap/log_spacemap_import_logs.ksh index d1d283376bbb..d16574d3a8f7 100755 --- a/tests/zfs-tests/tests/functional/log_spacemap/log_spacemap_import_logs.ksh +++ b/tests/zfs-tests/tests/functional/log_spacemap/log_spacemap_import_logs.ksh @@ -57,7 +57,7 @@ function cleanup log_onexit cleanup LOGSM_POOL="logsm_import" -TESTDISK="$(echo $DISKS | cut -d' ' -f1)" +read -r TESTDISK _ <<<"$DISKS" log_must zpool create -o cachefile=none -f $LOGSM_POOL $TESTDISK log_must zfs create $LOGSM_POOL/fs @@ -70,10 +70,7 @@ sync_all_pools log_must set_tunable64 KEEP_LOG_SPACEMAPS_AT_EXPORT 1 log_must zpool export $LOGSM_POOL -LOGSM_COUNT=$(zdb -m -e $LOGSM_POOL | grep "Log Spacemap object" | wc -l) -if (( LOGSM_COUNT == 0 )); then - log_fail "Pool does not have any log spacemaps after being exported" -fi +log_must eval "zdb -m -e $LOGSM_POOL | grep -q \"Log Spacemap object\"" log_must set_tunable64 METASLAB_DEBUG_LOAD 1 log_must zpool import $LOGSM_POOL diff --git a/tests/zfs-tests/tests/functional/migration/Makefile.am b/tests/zfs-tests/tests/functional/migration/Makefile.am deleted file mode 100644 index 9c4f244156ce..000000000000 --- a/tests/zfs-tests/tests/functional/migration/Makefile.am +++ /dev/null @@ -1,20 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/migration -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - migration_001_pos.ksh \ - migration_002_pos.ksh \ - migration_003_pos.ksh \ - migration_004_pos.ksh \ - migration_005_pos.ksh \ - migration_006_pos.ksh \ - migration_007_pos.ksh \ - migration_008_pos.ksh \ - migration_009_pos.ksh \ - migration_010_pos.ksh \ - migration_011_pos.ksh \ - migration_012_pos.ksh - -dist_pkgdata_DATA = \ - migration.cfg \ - migration.kshlib diff --git a/tests/zfs-tests/tests/functional/migration/cleanup.ksh b/tests/zfs-tests/tests/functional/migration/cleanup.ksh index 1a1f3f42731e..c9f1833be10c 100755 --- a/tests/zfs-tests/tests/functional/migration/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/migration/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -34,11 +34,9 @@ verify_runnable "global" -ismounted $NONZFS_TESTDIR $NEWFS_DEFAULT_FS -(( $? == 0 )) && log_must umount -f $NONZFS_TESTDIR +ismounted $NONZFS_TESTDIR $NEWFS_DEFAULT_FS && log_must umount -f $NONZFS_TESTDIR -ismounted $TESTPOOL/$TESTFS -[[ $? == 0 ]] && log_must zfs umount -f $TESTDIR +ismounted $TESTPOOL/$TESTFS && log_must zfs umount -f $TESTDIR destroy_pool $TESTPOOL DISK=${DISKS%% *} diff --git a/tests/zfs-tests/tests/functional/migration/migration.cfg b/tests/zfs-tests/tests/functional/migration/migration.cfg index 12a5a7799b7a..7d606f6d9a12 100644 --- a/tests/zfs-tests/tests/functional/migration/migration.cfg +++ b/tests/zfs-tests/tests/functional/migration/migration.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -31,7 +31,7 @@ . $STF_SUITE/include/libtest.shlib export DISKSARRAY=$DISKS -export DISK_ARRAY_NUM=$(echo ${DISKS} | nawk '{print NF}') +export DISK_ARRAY_NUM=$(echo ${DISKS} | awk '{print NF}') set -A disk_array $(find_disks $DISKS) case "${#disk_array[*]}" in 0) @@ -102,9 +102,8 @@ export DISK_COUNT ZFS_DISK NONZFS_DISK SINGLE_DISK ZFSSIDE_DISK NONZFSSIDE_DISK export TESTFILE=/etc/passwd export NONZFS_TESTDIR=$TESTDIR/nonzfstestdir -tmp=`sum $TESTFILE` -export SUMA=`echo $tmp | awk '{print $1}'` -export SUMB=`echo $tmp | awk '{print $2}'` +read -r SUMA SUMB _ < <(cksum $TESTFILE) +export SUMA SUMB export FS_SIZE=1g export BNAME=`basename $TESTFILE` export DNAME=`dirname $TESTFILE` diff --git a/tests/zfs-tests/tests/functional/migration/migration.kshlib b/tests/zfs-tests/tests/functional/migration/migration.kshlib index a2b4ed99b11e..7078f93a559b 100644 --- a/tests/zfs-tests/tests/functional/migration/migration.kshlib +++ b/tests/zfs-tests/tests/functional/migration/migration.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -49,19 +49,11 @@ function prepare #srcdir cmd { typeset srcdir=$1 typeset cmd=$2 - typeset -i retval=0 cwd=$PWD - cd $srcdir - (( $? != 0 )) && return 1 - - $cmd - (( $? != 0 )) && return 1 - - cd $cwd - (( $? != 0 )) && return 1 - - return 0 + cd $srcdir || return 1 + $cmd || return 1 + cd $cwd || return 1 } # @@ -90,32 +82,22 @@ function migrate #destdir oldsuma oldsumb cmd typeset oldsuma=$2 typeset oldsumb=$3 typeset cmd=$4 - typeset -i retval=0 cwd=$PWD - cd $destdir - (( $? != 0 )) && return 1 - - $cmd - (( $? != 0 )) && return 1 - - sumy=`sum ./$BNAME` - suma=`echo $sumy | awk '{print $1}'` - sumb=`echo $sumy | awk '{print $2}'` + cd $destdir || return 1 + $cmd || return 1 + read -r suma sumb _ < <(cksum ./$BNAME) + cd $cwd || return 1 if (( $oldsuma != $suma )); then log_note "sum values are not the same" - retval=1 + return 1 fi if (( $oldsumb != $sumb )); then log_note "sum values are not the same" - retval=1 + return 1 fi - - cd $cwd - (( $? != 0 )) && return 1 - return $retval } function migrate_cpio @@ -124,30 +106,20 @@ function migrate_cpio typeset archive=$2 typeset oldsuma=$3 typeset oldsumb=$4 - typeset -i retval=0 cwd=$PWD - cd $destdir - (( $? != 0 )) && return 1 - - cpio -iv < $archive - (( $? != 0 )) && return 1 - - sumy=`sum ./$BNAME` - suma=`echo $sumy | awk '{print $1}'` - sumb=`echo $sumy | awk '{print $2}'` + cd $destdir || return 1 + cpio -iv < $archive || return 1 + read -r suma sumb _ < <(cksum ./$BNAME) + cd $cwd if (( $oldsuma != $suma )); then log_note "sum values are not the same" - retval=1 + return 1 fi if (( $oldsumb != $sumb )); then log_note "sum values are not the same" - retval=1 + return 1 fi - - cd $cwd - (( $? != 0 )) && return 1 - return $retval } diff --git a/tests/zfs-tests/tests/functional/migration/migration_001_pos.ksh b/tests/zfs-tests/tests/functional/migration/migration_001_pos.ksh index 875d2f7c78be..8e0cdf8034d9 100755 --- a/tests/zfs-tests/tests/functional/migration/migration_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/migration/migration_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -48,19 +48,14 @@ verify_runnable "both" function cleanup { - rm -rf $TESTDIR/tar$$.tar - rm -rf $TESTDIR/$BNAME + rm -rf $TESTDIR/tar$$.tar $TESTDIR/$BNAME } log_assert "Migrating test file from ZFS fs to ZFS fs using tar" log_onexit cleanup -prepare $DNAME "tar cf $TESTDIR/tar$$.tar $BNAME" -(( $? != 0 )) && log_fail "Unable to create src archive" - -migrate $TESTDIR $SUMA $SUMB "tar xf $TESTDIR/tar$$.tar" -(( $? != 0 )) && log_fail "Unable to successfully migrate test file from" \ - "ZFS fs to ZFS fs" +log_must prepare $DNAME "tar cf $TESTDIR/tar$$.tar $BNAME" +log_must migrate $TESTDIR $SUMA $SUMB "tar xf $TESTDIR/tar$$.tar" log_pass "Successfully migrated test file from ZFS fs to ZFS fs". diff --git a/tests/zfs-tests/tests/functional/migration/migration_002_pos.ksh b/tests/zfs-tests/tests/functional/migration/migration_002_pos.ksh index 6b97e2a4071b..fe8aee004b93 100755 --- a/tests/zfs-tests/tests/functional/migration/migration_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/migration/migration_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -34,12 +34,12 @@ # # DESCRIPTION: -# Migrating test file from ZFS fs to UFS fs using tar. +# Migrating test file from ZFS fs to platform native fs using tar. # # STRATEGY: # 1. Calculate chksum of testfile # 2. Tar up test file and place on a ZFS filesystem -# 3. Extract tar contents to a UFS file system +# 3. Extract tar contents to a platform native file system # 4. Calculate chksum of extracted file # 5. Compare old and new chksums. # @@ -48,19 +48,14 @@ verify_runnable "both" function cleanup { - rm -rf $TESTDIR/tar$$.tar - rm -rf $NONZFS_TESTDIR/$BNAME + rm -rf $TESTDIR/tar$$.tar $NONZFS_TESTDIR/$BNAME } -log_assert "Migrating test file from ZFS fs to UFS fs using tar" +log_assert "Migrating test file from ZFS fs to $NEWFS_DEFAULT_FS fs using tar" log_onexit cleanup -prepare $DNAME "tar cf $TESTDIR/tar$$.tar $BNAME" -(( $? != 0 )) && log_fail "Unable to create src archive" +log_must prepare $DNAME "tar cf $TESTDIR/tar$$.tar $BNAME" +log_must migrate $NONZFS_TESTDIR $SUMA $SUMB "tar xf $TESTDIR/tar$$.tar" -migrate $NONZFS_TESTDIR $SUMA $SUMB "tar xf $TESTDIR/tar$$.tar" -(( $? != 0 )) && log_fail "Unable to successfully migrate test file from" \ - "ZFS fs to UFS fs" - -log_pass "Successfully migrated test file from ZFS fs to UFS fs". +log_pass "Successfully migrated test file from ZFS fs to $NEWFS_DEFAULT_FS fs". diff --git a/tests/zfs-tests/tests/functional/migration/migration_003_pos.ksh b/tests/zfs-tests/tests/functional/migration/migration_003_pos.ksh index dd0baeaa9b78..3d9ba5691d59 100755 --- a/tests/zfs-tests/tests/functional/migration/migration_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/migration/migration_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -34,11 +34,11 @@ # # DESCRIPTION: -# Migrating test file from UFS fs to ZFS fs using tar. +# Migrating test file from platform native fs to ZFS fs using tar. # # STRATEGY: # 1. Calculate chksum of testfile -# 2. Tar up test file and place on a UFS filesystem +# 2. Tar up test file and place on a platform native filesystem # 3. Extract tar contents to a ZFS file system # 4. Calculate chksum of extracted file # 5. Compare old and new chksums. @@ -48,19 +48,14 @@ verify_runnable "both" function cleanup { - rm -rf $NONZFS_TESTDIR/tar$$.tar - rm -rf $TESTDIR/$BNAME + rm -rf $NONZFS_TESTDIR/tar$$.tar $TESTDIR/$BNAME } -log_assert "Migrating test file from UFS fs to ZFS fs using tar" +log_assert "Migrating test file from $NEWFS_DEFAULT_FS fs to ZFS fs using tar" log_onexit cleanup -prepare $DNAME "tar cf $NONZFS_TESTDIR/tar$$.tar $BNAME" -(( $? != 0 )) && log_fail "Unable to create src archive" +log_must prepare $DNAME "tar cf $NONZFS_TESTDIR/tar$$.tar $BNAME" +log_must migrate $TESTDIR $SUMA $SUMB "tar xvf $NONZFS_TESTDIR/tar$$.tar" -migrate $TESTDIR $SUMA $SUMB "tar xvf $NONZFS_TESTDIR/tar$$.tar" -(( $? != 0 )) && log_fail "Unable to successfully migrate test file from" \ - "UFS fs to ZFS fs" - -log_pass "Successfully migrated test file from UFS fs to ZFS fs". +log_pass "Successfully migrated test file from $NEWFS_DEFAULT_FS fs to ZFS fs". diff --git a/tests/zfs-tests/tests/functional/migration/migration_004_pos.ksh b/tests/zfs-tests/tests/functional/migration/migration_004_pos.ksh index 00a6cc172ab6..c3b7b73b7e50 100755 --- a/tests/zfs-tests/tests/functional/migration/migration_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/migration/migration_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -48,8 +48,7 @@ verify_runnable "both" function cleanup { - rm -rf $TESTDIR/cpio$$.cpio - rm -rf $TESTDIR/$BNAME + rm -rf $TESTDIR/cpio$$.cpio $TESTDIR/$BNAME } log_assert "Migrating test file from ZFS fs to ZFS fs using cpio" @@ -57,17 +56,9 @@ log_assert "Migrating test file from ZFS fs to ZFS fs using cpio" log_onexit cleanup cwd=$PWD -cd $DNAME -(( $? != 0 )) && log_untested "Could not change directory to $DNAME" - -ls $BNAME | cpio -oc > $TESTDIR/cpio$$.cpio -(( $? != 0 )) && log_fail "Unable to create cpio archive" - -cd $cwd -(( $? != 0 )) && log_untested "Could not change directory to $cwd" - -migrate_cpio $TESTDIR "$TESTDIR/cpio$$.cpio" $SUMA $SUMB -(( $? != 0 )) && log_fail "Unable to successfully migrate test file from" \ - "ZFS fs to ZFS fs" +log_must cd $DNAME +log_must eval "find $BNAME | cpio -oc > $TESTDIR/cpio$$.cpio" +log_must cd $cwd +log_must migrate_cpio $TESTDIR "$TESTDIR/cpio$$.cpio" $SUMA $SUMB log_pass "Successfully migrated test file from ZFS fs to ZFS fs". diff --git a/tests/zfs-tests/tests/functional/migration/migration_005_pos.ksh b/tests/zfs-tests/tests/functional/migration/migration_005_pos.ksh index 4386596f777d..c48c8d6bc57f 100755 --- a/tests/zfs-tests/tests/functional/migration/migration_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/migration/migration_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -34,12 +34,12 @@ # # DESCRIPTION: -# Migrating test file from ZFS fs to UFS fs using cpio +# Migrating test file from ZFS fs to platform native fs using cpio # # STRATEGY: # 1. Calculate chksum of testfile # 2. Cpio up test file and place on a ZFS filesystem -# 3. Extract cpio contents to a UFS file system +# 3. Extract cpio contents to a platform native file system # 4. Calculate chksum of extracted file # 5. Compare old and new chksums. # @@ -48,26 +48,17 @@ verify_runnable "both" function cleanup { - rm -rf $TESTDIR/cpio$$.cpio - rm -rf $NONZFS_TESTDIR/$BNAME + rm -rf $TESTDIR/cpio$$.cpio $NONZFS_TESTDIR/$BNAME } -log_assert "Migrating test file from ZFS fs to uFS fs using cpio" +log_assert "Migrating test file from ZFS fs to $NEWFS_DEFAULT_FS fs using cpio" log_onexit cleanup cwd=$PWD -cd $DNAME -(( $? != 0 )) && log_untested "Could not change directory to $DNAME" +log_must cd $DNAME +log_must eval "find $BNAME | cpio -oc > $TESTDIR/cpio$$.cpio" +log_must cd $cwd +log_must migrate_cpio $NONZFS_TESTDIR "$TESTDIR/cpio$$.cpio" $SUMA $SUMB -ls $BNAME | cpio -oc > $TESTDIR/cpio$$.cpio -(( $? != 0 )) && log_fail "Unable to create cpio archive" - -cd $cwd -(( $? != 0 )) && log_untested "Could not change directory to $cwd" - -migrate_cpio $NONZFS_TESTDIR "$TESTDIR/cpio$$.cpio" $SUMA $SUMB -(( $? != 0 )) && log_fail "Unable to successfully migrate test file from" \ - "ZFS fs to UFS fs" - -log_pass "Successfully migrated test file from ZFS fs to UFS fs". +log_pass "Successfully migrated test file from ZFS fs to $NEWFS_DEFAULT_FS fs". diff --git a/tests/zfs-tests/tests/functional/migration/migration_006_pos.ksh b/tests/zfs-tests/tests/functional/migration/migration_006_pos.ksh index 9b5c9166ed97..893a9bc50fc5 100755 --- a/tests/zfs-tests/tests/functional/migration/migration_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/migration/migration_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -48,8 +48,7 @@ verify_runnable "both" function cleanup { - rm -rf $NONZFS_TESTDIR/cpio$$.cpio - rm -rf $TESTDIR/$BNAME + rm -rf $NONZFS_TESTDIR/cpio$$.cpio $TESTDIR/$BNAME } log_assert "Migrating test file from UFS fs to ZFS fs using cpio" @@ -57,17 +56,9 @@ log_assert "Migrating test file from UFS fs to ZFS fs using cpio" log_onexit cleanup cwd=$PWD -cd $DNAME -(( $? != 0 )) && log_untested "Could not change directory to $DNAME" - -ls $BNAME | cpio -oc > $NONZFS_TESTDIR/cpio$$.cpio -(( $? != 0 )) && log_fail "Unable to create cpio archive" - -cd $cwd -(( $? != 0 )) && log_untested "Could not change directory to $cwd" - -migrate_cpio $TESTDIR "$NONZFS_TESTDIR/cpio$$.cpio" $SUMA $SUMB -(( $? != 0 )) && log_fail "Unable to successfully migrate test file from" \ - "ZFS fs to ZFS fs" +log_must cd $DNAME +log_must eval "find $BNAME | cpio -oc > $NONZFS_TESTDIR/cpio$$.cpio" +log_must cd $cwd +log_must migrate_cpio $TESTDIR "$NONZFS_TESTDIR/cpio$$.cpio" $SUMA $SUMB log_pass "Successfully migrated test file from UFS fs to ZFS fs". diff --git a/tests/zfs-tests/tests/functional/migration/migration_007_pos.ksh b/tests/zfs-tests/tests/functional/migration/migration_007_pos.ksh index 0d136550f740..80b7749d6805 100755 --- a/tests/zfs-tests/tests/functional/migration/migration_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/migration/migration_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -48,19 +48,14 @@ verify_runnable "both" function cleanup { - rm -rf $TESTDIR/dd$$.dd - rm -rf $TESTDIR/$BNAME + rm -rf $TESTDIR/dd$$.dd $TESTDIR/$BNAME } log_assert "Migrating test file from ZFS fs to ZFS fs using dd" log_onexit cleanup -prepare $DNAME "dd if=$BNAME obs=128k of=$TESTDIR/dd$$.dd" -(( $? != 0 )) && log_fail "Unable to create src archive" - -migrate $TESTDIR $SUMA $SUMB "dd if=$TESTDIR/dd$$.dd obs=128k of=$BNAME" -(( $? != 0 )) && log_fail "Unable to successfully migrate test file from" \ - "ZFS fs to ZFS fs" +log_must prepare $DNAME "dd if=$BNAME obs=128k of=$TESTDIR/dd$$.dd" +log_must migrate $TESTDIR $SUMA $SUMB "dd if=$TESTDIR/dd$$.dd obs=128k of=$BNAME" log_pass "Successfully migrated test file from ZFS fs to ZFS fs". diff --git a/tests/zfs-tests/tests/functional/migration/migration_008_pos.ksh b/tests/zfs-tests/tests/functional/migration/migration_008_pos.ksh index f62b1f33a3e5..5fdcc5f90c55 100755 --- a/tests/zfs-tests/tests/functional/migration/migration_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/migration/migration_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -34,12 +34,12 @@ # # DESCRIPTION: -# Migrating test file from ZFS fs to UFS fs using dd. +# Migrating test file from ZFS fs to platform native fs using dd. # # STRATEGY: # 1. Calculate chksum of testfile # 2. Dd up test file and place on a ZFS filesystem -# 3. Extract dd contents to a UFS file system +# 3. Extract dd contents to a platform native file system # 4. Calculate chksum of extracted file # 5. Compare old and new chksums. # @@ -48,19 +48,14 @@ verify_runnable "both" function cleanup { - rm -rf $TESTDIR/dd$$.dd - rm -rf $NONZFS_TESTDIR/$BNAME + rm -rf $TESTDIR/dd$$.dd $NONZFS_TESTDIR/$BNAME } -log_assert "Migrating test file from ZFS fs to UFS fs using dd" +log_assert "Migrating test file from ZFS fs to $NEWFS_DEFAULT_FS fs using dd" log_onexit cleanup -prepare $DNAME "dd if=$BNAME obs=128k of=$TESTDIR/dd$$.dd" -(( $? != 0 )) && log_fail "Unable to create src archive" +log_must prepare $DNAME "dd if=$BNAME obs=128k of=$TESTDIR/dd$$.dd" +log_must migrate $NONZFS_TESTDIR $SUMA $SUMB "dd if=$TESTDIR/dd$$.dd obs=128k of=$BNAME" -migrate $NONZFS_TESTDIR $SUMA $SUMB "dd if=$TESTDIR/dd$$.dd obs=128k of=$BNAME" -(( $? != 0 )) && log_fail "Unable to successfully migrate test file from" \ - "ZFS fs to ZFS fs" - -log_pass "Successfully migrated test file from ZFS fs to UFS fs". +log_pass "Successfully migrated test file from ZFS fs to $NEWFS_DEFAULT_FS fs". diff --git a/tests/zfs-tests/tests/functional/migration/migration_009_pos.ksh b/tests/zfs-tests/tests/functional/migration/migration_009_pos.ksh index 907be39eb4dd..c91946067ef2 100755 --- a/tests/zfs-tests/tests/functional/migration/migration_009_pos.ksh +++ b/tests/zfs-tests/tests/functional/migration/migration_009_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -34,11 +34,11 @@ # # DESCRIPTION: -# Migrating test file from UFS fs to ZFS fs using dd. +# Migrating test file from platform native fs to ZFS fs using dd. # # STRATEGY: # 1. Calculate chksum of testfile -# 2. Dd up test file and place on a UFS filesystem +# 2. Dd up test file and place on a platform native filesystem # 3. Extract dd contents to a ZFS file system # 4. Calculate chksum of extracted file # 5. Compare old and new chksums. @@ -48,19 +48,14 @@ verify_runnable "both" function cleanup { - rm -rf $TESTDIR/dd$$.dd - rm -rf $NONZFS_TESTDIR/$BNAME + rm -rf $TESTDIR/dd$$.dd $NONZFS_TESTDIR/$BNAME } -log_assert "Migrating test file from UFS fs to ZFS fs using dd" +log_assert "Migrating test file from $NEWFS_DEFAULT_FS fs to ZFS fs using dd" log_onexit cleanup -prepare $DNAME "dd if=$BNAME obs=128k of=$NONZFS_TESTDIR/dd$$.dd" -(( $? != 0 )) && log_fail "Unable to create src archive" +log_must prepare $DNAME "dd if=$BNAME obs=128k of=$NONZFS_TESTDIR/dd$$.dd" +log_must migrate $TESTDIR $SUMA $SUMB "dd if=$NONZFS_TESTDIR/dd$$.dd obs=128k of=$BNAME" -migrate $TESTDIR $SUMA $SUMB "dd if=$NONZFS_TESTDIR/dd$$.dd obs=128k of=$BNAME" -(( $? != 0 )) && log_fail "Unable to successfully migrate test file from" \ - "ZFS fs to ZFS fs" - -log_pass "Successfully migrated test file from UFS fs to ZFS fs". +log_pass "Successfully migrated test file from $NEWFS_DEFAULT_FS fs to ZFS fs". diff --git a/tests/zfs-tests/tests/functional/migration/migration_010_pos.ksh b/tests/zfs-tests/tests/functional/migration/migration_010_pos.ksh index e80dd67cdc21..62ccccb1613c 100755 --- a/tests/zfs-tests/tests/functional/migration/migration_010_pos.ksh +++ b/tests/zfs-tests/tests/functional/migration/migration_010_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -48,19 +48,14 @@ verify_runnable "both" function cleanup { - rm -rf $TESTDIR/cp$$.cp - rm -rf $TESTDIR/$BNAME + rm -rf $TESTDIR/cp$$.cp $TESTDIR/$BNAME } log_assert "Migrating test file from ZFS fs to ZFS fs using cp" log_onexit cleanup -prepare $DNAME "cp $BNAME $TESTDIR/cp$$.cp" -(( $? != 0 )) && log_fail "Unable to create src archive" - -migrate $TESTDIR $SUMA $SUMB "cp $TESTDIR/cp$$.cp $BNAME" -(( $? != 0 )) && log_fail "Unable to successfully migrate test file from" \ - "ZFS fs to ZFS fs" +log_must prepare $DNAME "cp $BNAME $TESTDIR/cp$$.cp" +log_must migrate $TESTDIR $SUMA $SUMB "cp $TESTDIR/cp$$.cp $BNAME" log_pass "Successfully migrated test file from ZFS fs to ZFS fs". diff --git a/tests/zfs-tests/tests/functional/migration/migration_011_pos.ksh b/tests/zfs-tests/tests/functional/migration/migration_011_pos.ksh index 2d7ecb45eadb..9718fefb406d 100755 --- a/tests/zfs-tests/tests/functional/migration/migration_011_pos.ksh +++ b/tests/zfs-tests/tests/functional/migration/migration_011_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -34,12 +34,12 @@ # # DESCRIPTION: -# Migrating test file from ZFS fs to UFS fs using cp +# Migrating test file from ZFS fs to platform native fs using cp # # STRATEGY: # 1. Calculate chksum of testfile # 2. CP up test file and place on a ZFS filesystem -# 3. Extract cp contents to a UFS file system +# 3. Extract cp contents to a platform native file system # 4. Calculate chksum of extracted file # 5. Compare old and new chksums. # @@ -48,19 +48,14 @@ verify_runnable "both" function cleanup { - rm -rf $NONZFS_TESTDIR/cp$$.cp - rm -rf $TESTDIR/$BNAME + rm -rf $NONZFS_TESTDIR/cp$$.cp $TESTDIR/$BNAME } -log_assert "Migrating test file from ZFS fs to UFS fs using cp" +log_assert "Migrating test file from ZFS fs to $NEWFS_DEFAULT_FS fs using cp" log_onexit cleanup -prepare $DNAME "cp $BNAME $TESTDIR/cp$$.cp" -(( $? != 0 )) && log_fail "Unable to create src archive" +log_must prepare $DNAME "cp $BNAME $TESTDIR/cp$$.cp" +log_must migrate $NONZFS_TESTDIR $SUMA $SUMB "cp $TESTDIR/cp$$.cp $BNAME" -migrate $NONZFS_TESTDIR $SUMA $SUMB "cp $TESTDIR/cp$$.cp $BNAME" -(( $? != 0 )) && log_fail "Unable to successfully migrate test file from" \ - "ZFS fs to UFS fs" - -log_pass "Successfully migrated test file from ZFS fs to UFS fs". +log_pass "Successfully migrated test file from ZFS fs to $NEWFS_DEFAULT_FS fs". diff --git a/tests/zfs-tests/tests/functional/migration/migration_012_pos.ksh b/tests/zfs-tests/tests/functional/migration/migration_012_pos.ksh index fd9c4549164a..a92d11181d69 100755 --- a/tests/zfs-tests/tests/functional/migration/migration_012_pos.ksh +++ b/tests/zfs-tests/tests/functional/migration/migration_012_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -34,11 +34,11 @@ # # DESCRIPTION: -# Migrating test file from UFS fs to ZFS fs using cp +# Migrating test file from platform native fs to ZFS fs using cp # # STRATEGY: # 1. Calculate chksum of testfile -# 2. CP up test file and place on a UFS filesystem +# 2. CP up test file and place on a platform native filesystem # 3. Extract cp contents to a ZFS file system # 4. Calculate chksum of extracted file # 5. Compare old and new chksums. @@ -48,19 +48,14 @@ verify_runnable "both" function cleanup { - rm -rf $TESTDIR/cp$$.cp - rm -rf $NONZFS_TESTDIR/$BNAME + rm -rf $TESTDIR/cp$$.cp $NONZFS_TESTDIR/$BNAME } -log_assert "Migrating test file from UFS fs to ZFS fs using cp" +log_assert "Migrating test file from $NEWFS_DEFAULT_FS fs to ZFS fs using cp" log_onexit cleanup -prepare $DNAME "cp $BNAME $NONZFS_TESTDIR/cp$$.cp" -(( $? != 0 )) && log_fail "Unable to create src archive" +log_mustprepare $DNAME "cp $BNAME $NONZFS_TESTDIR/cp$$.cp" +log_mustmigrate $TESTDIR $SUMA $SUMB "cp $NONZFS_TESTDIR/cp$$.cp $BNAME" -migrate $TESTDIR $SUMA $SUMB "cp $NONZFS_TESTDIR/cp$$.cp $BNAME" -(( $? != 0 )) && log_fail "Unable to successfully migrate test file from" \ - "UFS fs to ZFS fs" - -log_pass "Successfully migrated test file from UFS fs to ZFS fs". +log_pass "Successfully migrated test file from $NEWFS_DEFAULT_FS fs to ZFS fs". diff --git a/tests/zfs-tests/tests/functional/migration/setup.ksh b/tests/zfs-tests/tests/functional/migration/setup.ksh index 58edc0a9291d..5aa4d058ce5d 100755 --- a/tests/zfs-tests/tests/functional/migration/setup.ksh +++ b/tests/zfs-tests/tests/functional/migration/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -57,8 +57,7 @@ log_must zfs set mountpoint=$TESTDIR $TESTPOOL/$TESTFS rm -rf $NONZFS_TESTDIR || log_unresolved Could not remove $NONZFS_TESTDIR mkdir -p $NONZFS_TESTDIR || log_unresolved Could not create $NONZFS_TESTDIR -new_fs ${DEV_DSKDIR}/$NONZFS_DISK -(( $? != 0 )) && +new_fs ${DEV_DSKDIR}/$NONZFS_DISK || log_untested "Unable to setup a $NEWFS_DEFAULT_FS file system" log_must mount ${DEV_DSKDIR}/$NONZFS_DISK $NONZFS_TESTDIR diff --git a/tests/zfs-tests/tests/functional/mmap/Makefile.am b/tests/zfs-tests/tests/functional/mmap/Makefile.am deleted file mode 100644 index b26791ee7ce0..000000000000 --- a/tests/zfs-tests/tests/functional/mmap/Makefile.am +++ /dev/null @@ -1,11 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/mmap -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - mmap_read_001_pos.ksh \ - mmap_write_001_pos.ksh \ - mmap_libaio_001_pos.ksh \ - mmap_seek_001_pos.ksh - -dist_pkgdata_DATA = \ - mmap.cfg diff --git a/tests/zfs-tests/tests/functional/mmap/cleanup.ksh b/tests/zfs-tests/tests/functional/mmap/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/mmap/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/mmap/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/mmap/mmap.cfg b/tests/zfs-tests/tests/functional/mmap/mmap.cfg index 1501c0463015..a331c5b2aa4f 100644 --- a/tests/zfs-tests/tests/functional/mmap/mmap.cfg +++ b/tests/zfs-tests/tests/functional/mmap/mmap.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/mmap/mmap_libaio_001_pos.ksh b/tests/zfs-tests/tests/functional/mmap/mmap_libaio_001_pos.ksh index 36a7e76f9f3d..b9e9a8a3c138 100755 --- a/tests/zfs-tests/tests/functional/mmap/mmap_libaio_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/mmap/mmap_libaio_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -42,9 +42,7 @@ verify_runnable "global" log_assert "verify mmap'd pages work with libaio" # mmap_libaio is built when the libaio-devel package is installed. -if ! which mmap_libaio; then - log_unsupported "This test requires mmap_libaio." -fi +command -v mmap_libaio > /dev/null || log_unsupported "This test requires mmap_libaio." log_must chmod 777 $TESTDIR diff --git a/tests/zfs-tests/tests/functional/mmap/mmap_read_001_pos.ksh b/tests/zfs-tests/tests/functional/mmap/mmap_read_001_pos.ksh index 470f10d937bc..acef0676f5cb 100755 --- a/tests/zfs-tests/tests/functional/mmap/mmap_read_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/mmap/mmap_read_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/mmap/mmap_seek_001_pos.ksh b/tests/zfs-tests/tests/functional/mmap/mmap_seek_001_pos.ksh index 6188549ad8d2..c09746b4b66a 100755 --- a/tests/zfs-tests/tests/functional/mmap/mmap_seek_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/mmap/mmap_seek_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/mmap/mmap_sync_001_pos.ksh b/tests/zfs-tests/tests/functional/mmap/mmap_sync_001_pos.ksh new file mode 100755 index 000000000000..b764d6607ba6 --- /dev/null +++ b/tests/zfs-tests/tests/functional/mmap/mmap_sync_001_pos.ksh @@ -0,0 +1,63 @@ +#!/bin/ksh -p + +# +# This file and its contents are supplied under the terms of the +# Common Development and Distribution License ("CDDL"), version 1.0. +# You may only use this file in accordance with the terms of version +# 1.0 of the CDDL. +# +# A full copy of the text of the CDDL should have accompanied this +# source. A copy of the CDDL is also available via the Internet at +# http://www.illumos.org/license/CDDL. +# + +# +# Copyright (c) 2015, 2016 by Delphix. All rights reserved. +# + +. $STF_SUITE/include/libtest.shlib + +# +# DESCRIPTION: +# msync()s of mmap()'ed file should complete quickly during +# background dirty page writebacks by the kernel. +# + +function cleanup +{ + log_must eval "echo $saved_vm_dirty_expire_centisecs > /proc/sys/vm/dirty_expire_centisecs" + log_must eval "echo $saved_vm_dirty_background_ratio > /proc/sys/vm/dirty_background_ratio" + log_must eval "echo $saved_vm_dirty_writeback_centisecs > /proc/sys/vm/dirty_writeback_centisecs" + + # revert to some sensible defaults if the values we saved + # were incorrect due to a previous run being interrupted + if [ $( /proc/sys/vm/dirty_expire_centisecs" + fi + + if [ $( /proc/sys/vm/dirty_background_ratio" + fi + + if [ $( /proc/sys/vm/dirty_writeback_centisecs" + fi +} + +if ! is_linux; then + log_unsupported "Only supported on Linux, requires /proc/sys/vm/ tunables" +fi + +log_onexit cleanup +log_assert "Run the tests for mmap_sync" + +read -r saved_vm_dirty_expire_centisecs < /proc/sys/vm/dirty_expire_centisecs +read -r saved_vm_dirty_background_ratio < /proc/sys/vm/dirty_background_ratio +read -r saved_vm_dirty_writeback_centisecs < /proc/sys/vm/dirty_writeback_centisecs + +log_must eval "echo 1 > /proc/sys/vm/dirty_expire_centisecs" +log_must eval "echo 1 > /proc/sys/vm/dirty_background_bytes" +log_must eval "echo 1 > /proc/sys/vm/dirty_writeback_centisecs" + +log_must mmap_sync +log_pass "mmap_sync tests passed." diff --git a/tests/zfs-tests/tests/functional/mmap/mmap_write_001_pos.ksh b/tests/zfs-tests/tests/functional/mmap/mmap_write_001_pos.ksh index 2f4257993d4a..9de0471a9b69 100755 --- a/tests/zfs-tests/tests/functional/mmap/mmap_write_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/mmap/mmap_write_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/mmap/setup.ksh b/tests/zfs-tests/tests/functional/mmap/setup.ksh index d275e063b13a..76aa23035dd9 100755 --- a/tests/zfs-tests/tests/functional/mmap/setup.ksh +++ b/tests/zfs-tests/tests/functional/mmap/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/mmp/Makefile.am b/tests/zfs-tests/tests/functional/mmp/Makefile.am deleted file mode 100644 index 2848fd4ce692..000000000000 --- a/tests/zfs-tests/tests/functional/mmp/Makefile.am +++ /dev/null @@ -1,21 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/mmp -dist_pkgdata_SCRIPTS = \ - multihost_history.ksh \ - mmp_on_thread.ksh \ - mmp_on_uberblocks.ksh \ - mmp_on_off.ksh \ - mmp_interval.ksh \ - mmp_active_import.ksh \ - mmp_inactive_import.ksh \ - mmp_exported_import.ksh \ - mmp_write_uberblocks.ksh \ - mmp_reset_interval.ksh \ - mmp_on_zdb.ksh \ - mmp_write_distribution.ksh \ - mmp_hostid.ksh \ - setup.ksh \ - cleanup.ksh - -dist_pkgdata_DATA = \ - mmp.kshlib \ - mmp.cfg diff --git a/tests/zfs-tests/tests/functional/mmp/mmp.kshlib b/tests/zfs-tests/tests/functional/mmp/mmp.kshlib index 661cbf3a52e7..5071830c489a 100644 --- a/tests/zfs-tests/tests/functional/mmp/mmp.kshlib +++ b/tests/zfs-tests/tests/functional/mmp/mmp.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -36,10 +36,8 @@ function check_pool_import # pool opts token keyword typeset keyword=$4 zpool import $opts 2>&1 | \ - nawk -v token="$token:" '($1==token) {print $0}' | \ - grep -i "$keyword" > /dev/null 2>&1 - - return $? + awk -v token="$token:" '($1==token) {print}' | \ + grep -iq "$keyword" } function is_pool_imported # pool opts @@ -49,7 +47,6 @@ function is_pool_imported # pool opts check_pool_import "$pool" "$opts" "status" \ "The pool is currently imported" - return $? } function wait_pool_imported # pool opts @@ -60,8 +57,6 @@ function wait_pool_imported # pool opts while is_pool_imported "$pool" "$opts"; do log_must sleep 5 done - - return 0 } function try_pool_import # pool opts message @@ -71,8 +66,6 @@ function try_pool_import # pool opts message typeset msg=$3 zpool import $opts $pool 2>&1 | grep -i "$msg" - - return $? } function mmp_set_hostid @@ -81,11 +74,7 @@ function mmp_set_hostid zgenhostid $1 - if [ $(hostid) != "$hostid" ]; then - return 1 - fi - - return 0 + [ $(hostid) = "$hostid" ] } function mmp_clear_hostid @@ -162,45 +151,6 @@ function mmp_pool_set_hostid # pool hostid return 0 } -# Return the number of seconds the activity check portion of the import process -# will take. Does not include the time to find devices and assemble a config. -# Note that the activity check may be skipped, e.g. if the pool and host -# hostid's match, but this will return non-zero because mmp_* are populated. -function seconds_mmp_waits_for_activity -{ - typeset pool=$1 - typeset devpath=$2 - - typeset seconds=0 - typeset devices=${#DISK[@]} - typeset import_intervals=$(get_tunable MULTIHOST_IMPORT_INTERVALS) - typeset import_interval=$(get_tunable MULTIHOST_INTERVAL) - typeset tmpfile=$(mktemp) - typeset mmp_fail - typeset mmp_write - typeset mmp_delay - - log_must zdb -e -p $devpath $pool >$tmpfile 2>/dev/null - mmp_fail=$(awk '/mmp_fail/ {print $NF}' $tmpfile) - mmp_write=$(awk '/mmp_write/ {print $NF}' $tmpfile) - mmp_delay=$(awk '/mmp_delay/ {print $NF}' $tmpfile) - if [ -f $tmpfile ]; then - rm $tmpfile - fi - - # In order of preference: - if [ -n $mmp_fail -a -n $mmp_write ]; then - seconds=$((2*mmp_fail*mmp_write/1000)) - elif [ -n $mmp_delay ]; then - # MMP V0: Based on mmp_delay from the best Uberblock - seconds=$((import_intervals*devices*mmp_delay/1000000000)) - else - # Non-MMP aware: Based on zfs_multihost_interval and import_intervals - seconds=$((import_intervals*import_interval/1000)) - fi - - echo $seconds -} function import_no_activity_check # pool opts { diff --git a/tests/zfs-tests/tests/functional/mmp/mmp_on_off.ksh b/tests/zfs-tests/tests/functional/mmp/mmp_on_off.ksh index 29d771de8f8b..513b7d31f9cd 100755 --- a/tests/zfs-tests/tests/functional/mmp/mmp_on_off.ksh +++ b/tests/zfs-tests/tests/functional/mmp/mmp_on_off.ksh @@ -60,9 +60,9 @@ log_must mmp_set_hostid $HOSTID1 default_setup_noexit $DISK log_must zpool set multihost=off $TESTPOOL -log_must zdb -u $TESTPOOL > $PREV_UBER +log_must eval "zdb -u $TESTPOOL > $PREV_UBER" log_must sleep 5 -log_must zdb -u $TESTPOOL > $CURR_UBER +log_must eval "zdb -u $TESTPOOL > $CURR_UBER" if ! diff "$CURR_UBER" "$PREV_UBER"; then log_fail "mmp thread has updated an uberblock" @@ -70,7 +70,7 @@ fi log_must zpool set multihost=on $TESTPOOL log_must sleep 5 -log_must zdb -u $TESTPOOL > $CURR_UBER +log_must eval "zdb -u $TESTPOOL > $CURR_UBER" if diff "$CURR_UBER" "$PREV_UBER"; then log_fail "mmp failed to update uberblocks" diff --git a/tests/zfs-tests/tests/functional/mmp/mmp_on_thread.ksh b/tests/zfs-tests/tests/functional/mmp/mmp_on_thread.ksh index 01cca61c3c3e..cd82fa47e23f 100755 --- a/tests/zfs-tests/tests/functional/mmp/mmp_on_thread.ksh +++ b/tests/zfs-tests/tests/functional/mmp/mmp_on_thread.ksh @@ -53,9 +53,9 @@ log_must mmp_set_hostid $HOSTID1 default_setup_noexit $DISK log_must zpool set multihost=on $TESTPOOL -log_must zdb -u $TESTPOOL > $PREV_UBER +log_must eval "zdb -u $TESTPOOL > $PREV_UBER" log_must sleep 5 -log_must zdb -u $TESTPOOL > $CURR_UBER +log_must eval "zdb -u $TESTPOOL > $CURR_UBER" if diff -u "$CURR_UBER" "$PREV_UBER"; then log_fail "mmp failed to update uberblocks" diff --git a/tests/zfs-tests/tests/functional/mmp/mmp_write_distribution.ksh b/tests/zfs-tests/tests/functional/mmp/mmp_write_distribution.ksh index b6bdc6811634..1ac254aa1dab 100755 --- a/tests/zfs-tests/tests/functional/mmp/mmp_write_distribution.ksh +++ b/tests/zfs-tests/tests/functional/mmp/mmp_write_distribution.ksh @@ -37,7 +37,7 @@ verify_runnable "both" function cleanup { log_must zpool destroy $MMP_POOL - log_must rm $MMP_DIR/file.{0,1,2,3,4,5,6,7} + log_must rm $MMP_DIR/file.{0..7} log_must rm $MMP_HISTORY_TMP log_must rmdir $MMP_DIR log_must mmp_clear_hostid @@ -51,8 +51,8 @@ MMP_HISTORY=/proc/spl/kstat/zfs/$MMP_POOL/multihost # Step 1 log_must mkdir -p $MMP_DIR -log_must truncate -s 128M $MMP_DIR/file.{0,1,2,3,4,5,6,7} -log_must zpool create -f $MMP_POOL mirror $MMP_DIR/file.{0,1} mirror $MMP_DIR/file.{2,3,4,5,6,7} +log_must truncate -s 128M $MMP_DIR/file.{0..7} +log_must zpool create -f $MMP_POOL mirror $MMP_DIR/file.{0..1} mirror $MMP_DIR/file.{2..7} # Step 2 log_must mmp_set_hostid $HOSTID1 @@ -69,8 +69,8 @@ typeset -i min_writes=999 typeset -i max_writes=0 typeset -i write_count # copy to get as close to a consistent view as possible -cat $MMP_HISTORY > $MMP_HISTORY_TMP -for x in $(seq 0 7); do +cp $MMP_HISTORY $MMP_HISTORY_TMP +for x in {0..7}; do write_count=$(grep -c file.${x} $MMP_HISTORY_TMP) if [ $write_count -lt $min_writes ]; then min_writes=$write_count diff --git a/tests/zfs-tests/tests/functional/mount/Makefile.am b/tests/zfs-tests/tests/functional/mount/Makefile.am deleted file mode 100644 index bdafa69badd8..000000000000 --- a/tests/zfs-tests/tests/functional/mount/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/mount -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - umount_001.ksh \ - umount_unlinked_drain.ksh \ - umountall_001.ksh diff --git a/tests/zfs-tests/tests/functional/mount/cleanup.ksh b/tests/zfs-tests/tests/functional/mount/cleanup.ksh index f20c72841f42..bd6b0e435ed1 100755 --- a/tests/zfs-tests/tests/functional/mount/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/mount/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/mount/setup.ksh b/tests/zfs-tests/tests/functional/mount/setup.ksh index 8e2edcd24f4b..9b3bc37ce6ae 100755 --- a/tests/zfs-tests/tests/functional/mount/setup.ksh +++ b/tests/zfs-tests/tests/functional/mount/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/mount/umount_001.ksh b/tests/zfs-tests/tests/functional/mount/umount_001.ksh index ec4954569961..9e350de66fa3 100755 --- a/tests/zfs-tests/tests/functional/mount/umount_001.ksh +++ b/tests/zfs-tests/tests/functional/mount/umount_001.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/mount/umount_unlinked_drain.ksh b/tests/zfs-tests/tests/functional/mount/umount_unlinked_drain.ksh index 6130e2c82893..40045a7a96b5 100755 --- a/tests/zfs-tests/tests/functional/mount/umount_unlinked_drain.ksh +++ b/tests/zfs-tests/tests/functional/mount/umount_unlinked_drain.ksh @@ -47,8 +47,8 @@ function unlinked_size_is last_usize=0 while [[ $iters -le $MAX_ITERS ]]; do kstat_file=$(grep -nrwl /proc/spl/kstat/zfs/$2/objset-0x* -e $3) - nunlinks=`cat $kstat_file | grep nunlinks | awk '{print $3}'` - nunlinked=`cat $kstat_file | grep nunlinked | awk '{print $3}'` + nunlinks=$(awk '/nunlinks/ {print $3}' $kstat_file) + nunlinked=$(awk '/nunlinked/ {print $3}' $kstat_file) usize=$(($nunlinks - $nunlinked)) if [[ $iters == $MAX_ITERS && $usize == $1 ]]; then return 0 diff --git a/tests/zfs-tests/tests/functional/mount/umountall_001.ksh b/tests/zfs-tests/tests/functional/mount/umountall_001.ksh index 814c831e408c..40c94593ba2d 100755 --- a/tests/zfs-tests/tests/functional/mount/umountall_001.ksh +++ b/tests/zfs-tests/tests/functional/mount/umountall_001.ksh @@ -44,11 +44,11 @@ zfs_list="/ /lib /sbin /tmp /usr /var /var/adm /var/run" # Append our ZFS filesystems to the list, not worrying about duplicates. if is_linux; then - typeset mounts=$(mount | awk '{if ($5 == "zfs") print $3}') + typeset mounts=$(mount | awk '$5 == "zfs" {print $3}') elif is_freebsd; then - typeset mounts=$(mount -p | awk '{if ($3 == "zfs") print $2}') + typeset mounts=$(mount -p | awk '$3 == "zfs" {print $2}') else - typeset mounts=$(mount -p | awk '{if ($4 == "zfs") print $3}') + typeset mounts=$(mount -p | awk '$4 == "zfs" {print $3}') fi for fs in $mounts; do @@ -56,8 +56,7 @@ for fs in $mounts; do done if is_linux; then - mounts=$(umount --fake -av -t zfs 2>&1 | \ - grep "successfully umounted" | awk '{print $1}') + mounts=$(umount --fake -av -t zfs 2>&1 | awk '/successfully umounted/ {print $1}') # Fallback to /proc/mounts for umount(8) (util-linux-ng 2.17.2) if [[ -z $mounts ]]; then mounts=$(awk '/zfs/ { print $2 }' /proc/mounts) diff --git a/tests/zfs-tests/tests/functional/mv_files/Makefile.am b/tests/zfs-tests/tests/functional/mv_files/Makefile.am deleted file mode 100644 index cec02140e307..000000000000 --- a/tests/zfs-tests/tests/functional/mv_files/Makefile.am +++ /dev/null @@ -1,11 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/mv_files -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - mv_files_001_pos.ksh \ - mv_files_002_pos.ksh \ - random_creation.ksh - -dist_pkgdata_DATA = \ - mv_files.cfg \ - mv_files_common.kshlib diff --git a/tests/zfs-tests/tests/functional/mv_files/cleanup.ksh b/tests/zfs-tests/tests/functional/mv_files/cleanup.ksh index a664433743af..882f9955a4c2 100755 --- a/tests/zfs-tests/tests/functional/mv_files/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/mv_files/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -34,13 +34,12 @@ verify_runnable "global" -[[ -f $TEST_BASE_DIR/exitsZero.ksh ]] && \ +[[ -f $TEST_BASE_DIR/exitsZero.ksh ]] && log_must rm -f $TEST_BASE_DIR/exitsZero.ksh -[[ -f $TEST_BASE_DIR/testbackgprocs.ksh ]] && \ +[[ -f $TEST_BASE_DIR/testbackgprocs.ksh ]] && log_must rm -f $TEST_BASE_DIR/testbackgprocs.ksh -ismounted $TESTPOOL/$TESTFS_TGT -(( $? == 0 )) && log_must zfs umount $TESTPOOL/$TESTFS_TGT +ismounted $TESTPOOL/$TESTFS_TGT ||log_must zfs umount $TESTPOOL/$TESTFS_TGT log_must zfs destroy $TESTPOOL/$TESTFS_TGT [[ -d $TESTDIR_TGT ]] && log_must rm -rf $TESTDIR_TGT diff --git a/tests/zfs-tests/tests/functional/mv_files/mv_files.cfg b/tests/zfs-tests/tests/functional/mv_files/mv_files.cfg index ed28ff1efd27..2ee7c08d3c43 100644 --- a/tests/zfs-tests/tests/functional/mv_files/mv_files.cfg +++ b/tests/zfs-tests/tests/functional/mv_files/mv_files.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -30,7 +30,7 @@ export DISK=${DISKS%% *} export DISKSARRAY=$DISKS -export DISK_ARRAY_NUM=$(echo ${DISKS} | nawk '{print NF}') +export DISK_ARRAY_NUM=$(echo ${DISKS} | awk '{print NF}') set_device_dir export TESTFILE=testfile diff --git a/tests/zfs-tests/tests/functional/mv_files/mv_files_001_pos.ksh b/tests/zfs-tests/tests/functional/mv_files/mv_files_001_pos.ksh index c49b19c6c3a3..fffe8d285654 100755 --- a/tests/zfs-tests/tests/functional/mv_files/mv_files_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/mv_files/mv_files_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -53,8 +53,7 @@ verify_runnable "global" function cleanup { - rm -f $OLDDIR/* >/dev/null 2>&1 - rm -f $NEWDIR_IN_FS/* >/dev/null 2>&1 + rm -f $OLDDIR/* $NEWDIR_IN_FS/* >/dev/null 2>&1 } log_assert "Doing a 'mv' of a large amount of files within a zfs filesystem" \ @@ -62,7 +61,6 @@ log_assert "Doing a 'mv' of a large amount of files within a zfs filesystem" \ log_onexit cleanup -mv_test $OLDDIR $NEWDIR_IN_FS -(($? != 0 )) && log_fail "'mv' test failed to complete." +log_must mv_test $OLDDIR $NEWDIR_IN_FS log_pass diff --git a/tests/zfs-tests/tests/functional/mv_files/mv_files_002_pos.ksh b/tests/zfs-tests/tests/functional/mv_files/mv_files_002_pos.ksh index fdadac32d59b..7eea345d150e 100755 --- a/tests/zfs-tests/tests/functional/mv_files/mv_files_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/mv_files/mv_files_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -54,8 +54,7 @@ verify_runnable "global" function cleanup { - rm -f $OLDDIR/* >/dev/null 2>&1 - rm -f $NEWDIR_ACROSS_FS/* >/dev/null 2>&1 + rm -f $OLDDIR/* $NEWDIR_ACROSS_FS/* >/dev/null 2>&1 } log_assert "Doing a 'mv' of a large amount of files across two zfs filesystems" \ @@ -63,8 +62,6 @@ log_assert "Doing a 'mv' of a large amount of files across two zfs filesystems" log_onexit cleanup -mv_test $OLDDIR $NEWDIR_ACROSS_FS -(($? != 0 )) && \ - log_fail "'mv' test failed to complete." +log_must mv_test $OLDDIR $NEWDIR_ACROSS_FS log_pass diff --git a/tests/zfs-tests/tests/functional/mv_files/mv_files_common.kshlib b/tests/zfs-tests/tests/functional/mv_files/mv_files_common.kshlib index 3ddd8f113a3b..c3827198a6ef 100644 --- a/tests/zfs-tests/tests/functional/mv_files/mv_files_common.kshlib +++ b/tests/zfs-tests/tests/functional/mv_files/mv_files_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -114,9 +114,7 @@ function mv_files # function count_files { - typeset -i file_num - file_num=$(find $1 -type f -print | wc -l | tr -d ' ') - (( file_num != $2 )) && \ + (( $(find $1 -type f -print | wc -l) != $2 )) && \ log_fail "The file number of target directory"\ "$2 is not equal to that of the source "\ "directory $1" diff --git a/tests/zfs-tests/tests/functional/mv_files/setup.ksh b/tests/zfs-tests/tests/functional/mv_files/setup.ksh index d459c51e4e57..669a930c6323 100755 --- a/tests/zfs-tests/tests/functional/mv_files/setup.ksh +++ b/tests/zfs-tests/tests/functional/mv_files/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/nestedfs/Makefile.am b/tests/zfs-tests/tests/functional/nestedfs/Makefile.am deleted file mode 100644 index 6a5ecf2d9a7f..000000000000 --- a/tests/zfs-tests/tests/functional/nestedfs/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/nestedfs -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - nestedfs_001_pos.ksh diff --git a/tests/zfs-tests/tests/functional/nestedfs/cleanup.ksh b/tests/zfs-tests/tests/functional/nestedfs/cleanup.ksh index c4c369145dfc..8a6521193e66 100755 --- a/tests/zfs-tests/tests/functional/nestedfs/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/nestedfs/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/nestedfs/nestedfs_001_pos.ksh b/tests/zfs-tests/tests/functional/nestedfs/nestedfs_001_pos.ksh index b576b78c8aa5..84749d60227d 100755 --- a/tests/zfs-tests/tests/functional/nestedfs/nestedfs_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/nestedfs/nestedfs_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/nestedfs/setup.ksh b/tests/zfs-tests/tests/functional/nestedfs/setup.ksh index 677cb12d79ac..1eeefeaa18c1 100755 --- a/tests/zfs-tests/tests/functional/nestedfs/setup.ksh +++ b/tests/zfs-tests/tests/functional/nestedfs/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/no_space/Makefile.am b/tests/zfs-tests/tests/functional/no_space/Makefile.am deleted file mode 100644 index c2e42bc2ada4..000000000000 --- a/tests/zfs-tests/tests/functional/no_space/Makefile.am +++ /dev/null @@ -1,11 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/no_space -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - enospc_001_pos.ksh \ - enospc_002_pos.ksh \ - enospc_003_pos.ksh \ - enospc_df.ksh - -dist_pkgdata_DATA = \ - enospc.cfg diff --git a/tests/zfs-tests/tests/functional/no_space/cleanup.ksh b/tests/zfs-tests/tests/functional/no_space/cleanup.ksh index 62b28df8ef5a..d1a731f18372 100755 --- a/tests/zfs-tests/tests/functional/no_space/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/no_space/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/no_space/enospc.cfg b/tests/zfs-tests/tests/functional/no_space/enospc.cfg index 0b8298c03165..e8e6065bbc81 100644 --- a/tests/zfs-tests/tests/functional/no_space/enospc.cfg +++ b/tests/zfs-tests/tests/functional/no_space/enospc.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/no_space/enospc_001_pos.ksh b/tests/zfs-tests/tests/functional/no_space/enospc_001_pos.ksh index 56beb4e71e17..c6d374635c5b 100755 --- a/tests/zfs-tests/tests/functional/no_space/enospc_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/no_space/enospc_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/no_space/enospc_002_pos.ksh b/tests/zfs-tests/tests/functional/no_space/enospc_002_pos.ksh index 2fb3fb46c44c..0aecbea7eb27 100755 --- a/tests/zfs-tests/tests/functional/no_space/enospc_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/no_space/enospc_002_pos.ksh @@ -62,8 +62,8 @@ done log_mustnot_expect space zfs create $TESTPOOL/$TESTFS/subfs log_mustnot_expect space zfs clone $TESTPOOL/$TESTFS@snap $TESTPOOL/clone -log_must zfs send $TESTPOOL/$TESTFS@snap > $TEST_BASE_DIR/stream.$$ -log_mustnot_expect space zfs receive $TESTPOOL/$TESTFS/recvd < $TEST_BASE_DIR/stream.$$ +log_must eval "zfs send $TESTPOOL/$TESTFS@snap > $TEST_BASE_DIR/stream.$$" +log_mustnot_expect space eval "zfs receive $TESTPOOL/$TESTFS/recvd < $TEST_BASE_DIR/stream.$$" log_must rm $TEST_BASE_DIR/stream.$$ log_must zfs rename $TESTPOOL/$TESTFS@snap $TESTPOOL/$TESTFS@snap_newname diff --git a/tests/zfs-tests/tests/functional/no_space/enospc_003_pos.ksh b/tests/zfs-tests/tests/functional/no_space/enospc_003_pos.ksh index 496e2a029c5c..5ac96962546f 100755 --- a/tests/zfs-tests/tests/functional/no_space/enospc_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/no_space/enospc_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/no_space/enospc_df.ksh b/tests/zfs-tests/tests/functional/no_space/enospc_df.ksh index b1eeaf2cc569..cf788ca7fdf0 100755 --- a/tests/zfs-tests/tests/functional/no_space/enospc_df.ksh +++ b/tests/zfs-tests/tests/functional/no_space/enospc_df.ksh @@ -64,8 +64,8 @@ log_must zfs umount $TESTPOOL/$TESTFS log_must eval "df -h | grep $TESTPOOL" # Confirm df size and used are non-zero. -size=$(df -h /$TESTPOOL | grep $TESTPOOL | awk '{print $2}') -used=$(df -h /$TESTPOOL | grep $TESTPOOL | awk '{print $3}') +size=$(df -h /$TESTPOOL | awk -v p=$TESTPOOL '$0 ~ p {print $2}') +used=$(df -h /$TESTPOOL | awk -v p=$TESTPOOL '$0 ~ p {print $3}') if [[ "$size" = "0" ]] || [[ "$used" = "0" ]] then log_fail "df failed with size $size and used $used." diff --git a/tests/zfs-tests/tests/functional/no_space/enospc_rm.ksh b/tests/zfs-tests/tests/functional/no_space/enospc_rm.ksh new file mode 100755 index 000000000000..d0f4ff4a08fe --- /dev/null +++ b/tests/zfs-tests/tests/functional/no_space/enospc_rm.ksh @@ -0,0 +1,62 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# This file and its contents are supplied under the terms of the +# Common Development and Distribution License ("CDDL"), version 1.0. +# You may only use this file in accordance with the terms of version +# 1.0 of the CDDL. +# +# A full copy of the text of the CDDL should have accompanied this +# source. A copy of the CDDL is also available via the Internet at +# http://www.illumos.org/license/CDDL. +# +# CDDL HEADER END +# + +# +# Copyright (c) 2014, 2016 by Delphix. All rights reserved. +# Copyright (c) 2022 by Lawrence Livermore National Security, LLC. +# + +. $STF_SUITE/include/libtest.shlib +. $STF_SUITE/tests/functional/no_space/enospc.cfg + +# +# DESCRIPTION: +# After filling a filesystem, verify the contents can be removed +# without encountering an ENOSPC error. +# + +verify_runnable "both" + +command -v fio > /dev/null || log_unsupported "fio missing" + +function cleanup +{ + destroy_pool $TESTPOOL + log_must rm -f $all_vdevs +} + +log_onexit cleanup + +log_assert "Files can be removed from full file system." + +all_vdevs=$(echo $TEST_BASE_DIR/file.{01..12}) + +log_must truncate -s $MINVDEVSIZE $all_vdevs + +log_must zpool create -f $TESTPOOL draid2:8d:2s $all_vdevs +log_must zfs create $TESTPOOL/$TESTFS +log_must zfs set mountpoint=$TESTDIR $TESTPOOL/$TESTFS +log_must zfs set compression=off $TESTPOOL/$TESTFS + +log_note "Writing files until ENOSPC." +log_mustnot_expect "No space left on device" fio --name=test \ + --fallocate=none --rw=write --bs=1M --size=1G --numjobs=4 \ + --sync=1 --directory=$TESTDIR/ --group_reporting + +log_must rm $TESTDIR/test.* +log_must test -z "$(ls -A $TESTDIR)" + +log_pass "All files removed without error" diff --git a/tests/zfs-tests/tests/functional/no_space/setup.ksh b/tests/zfs-tests/tests/functional/no_space/setup.ksh index 7ab8f1473d66..458a575ed3c0 100755 --- a/tests/zfs-tests/tests/functional/no_space/setup.ksh +++ b/tests/zfs-tests/tests/functional/no_space/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/nopwrite/Makefile.am b/tests/zfs-tests/tests/functional/nopwrite/Makefile.am deleted file mode 100644 index 3f893c6382e6..000000000000 --- a/tests/zfs-tests/tests/functional/nopwrite/Makefile.am +++ /dev/null @@ -1,15 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/nopwrite -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - nopwrite_copies.ksh \ - nopwrite_mtime.ksh \ - nopwrite_negative.ksh \ - nopwrite_promoted_clone.ksh \ - nopwrite_recsize.ksh \ - nopwrite_sync.ksh \ - nopwrite_varying_compression.ksh \ - nopwrite_volume.ksh - -dist_pkgdata_DATA = \ - nopwrite.shlib diff --git a/tests/zfs-tests/tests/functional/online_offline/Makefile.am b/tests/zfs-tests/tests/functional/online_offline/Makefile.am deleted file mode 100644 index bd23f89e538a..000000000000 --- a/tests/zfs-tests/tests/functional/online_offline/Makefile.am +++ /dev/null @@ -1,10 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/online_offline -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - online_offline_001_pos.ksh \ - online_offline_002_neg.ksh \ - online_offline_003_neg.ksh - -dist_pkgdata_DATA = \ - online_offline.cfg diff --git a/tests/zfs-tests/tests/functional/online_offline/cleanup.ksh b/tests/zfs-tests/tests/functional/online_offline/cleanup.ksh index b81a372638e3..a0555aff06c2 100755 --- a/tests/zfs-tests/tests/functional/online_offline/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/online_offline/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/online_offline/online_offline.cfg b/tests/zfs-tests/tests/functional/online_offline/online_offline.cfg index ae4cb69adcb0..18f197b54d4a 100644 --- a/tests/zfs-tests/tests/functional/online_offline/online_offline.cfg +++ b/tests/zfs-tests/tests/functional/online_offline/online_offline.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -38,5 +38,5 @@ export HOLES_COUNT=${HOLES_COUNT-"16384"} # FILESIZE/BLKSIZE/8 export STF_TIMEOUT=3600 export DISKSARRAY=$DISKS -export DISK_ARRAY_NUM=$(echo ${DISKS} | nawk '{print NF}') +export DISK_ARRAY_NUM=$(echo ${DISKS} | awk '{print NF}') set_device_dir diff --git a/tests/zfs-tests/tests/functional/online_offline/online_offline_001_pos.ksh b/tests/zfs-tests/tests/functional/online_offline/online_offline_001_pos.ksh index 5050447c000a..d77103330a30 100755 --- a/tests/zfs-tests/tests/functional/online_offline/online_offline_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/online_offline/online_offline_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -54,10 +54,7 @@ function cleanup # for disk in $DISKLIST; do log_must zpool online $TESTPOOL $disk - check_state $TESTPOOL $disk "online" - if [[ $? != 0 ]]; then - log_fail "Unable to online $disk" - fi + log_must check_state $TESTPOOL $disk "online" done @@ -73,17 +70,11 @@ typeset killpid="$! " for disk in $DISKLIST; do for i in 'do_offline' 'do_offline_while_already_offline'; do log_must zpool offline $TESTPOOL $disk - check_state $TESTPOOL $disk "offline" - if [[ $? != 0 ]]; then - log_fail "$disk of $TESTPOOL is not offline." - fi + log_must check_state $TESTPOOL $disk "offline" done log_must zpool online $TESTPOOL $disk - check_state $TESTPOOL $disk "online" - if [[ $? != 0 ]]; then - log_fail "$disk of $TESTPOOL did not match online state" - fi + log_must check_state $TESTPOOL $disk "online" # Delay for resilver to complete sleep 3 diff --git a/tests/zfs-tests/tests/functional/online_offline/online_offline_002_neg.ksh b/tests/zfs-tests/tests/functional/online_offline/online_offline_002_neg.ksh index e66e7e10fedc..33244024284a 100755 --- a/tests/zfs-tests/tests/functional/online_offline/online_offline_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/online_offline/online_offline_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -55,11 +55,7 @@ function cleanup # for disk in $DISKLIST; do log_must zpool online $TESTPOOL $disk - check_state $TESTPOOL $disk "online" - if [[ $? != 0 ]]; then - log_fail "Unable to online $disk" - fi - + log_must check_state $TESTPOOL $disk "online" done kill $killpid >/dev/null 2>&1 diff --git a/tests/zfs-tests/tests/functional/online_offline/online_offline_003_neg.ksh b/tests/zfs-tests/tests/functional/online_offline/online_offline_003_neg.ksh index e33db28f8ddb..1ff5b544b62c 100755 --- a/tests/zfs-tests/tests/functional/online_offline/online_offline_003_neg.ksh +++ b/tests/zfs-tests/tests/functional/online_offline/online_offline_003_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/online_offline/setup.ksh b/tests/zfs-tests/tests/functional/online_offline/setup.ksh index 4132392d80a9..0327c20a73cc 100755 --- a/tests/zfs-tests/tests/functional/online_offline/setup.ksh +++ b/tests/zfs-tests/tests/functional/online_offline/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -43,5 +43,3 @@ case $index in default_raidz_setup $DISKS ;; esac - -log_pass diff --git a/tests/zfs-tests/tests/functional/pam/.gitignore b/tests/zfs-tests/tests/functional/pam/.gitignore new file mode 100644 index 000000000000..ae55292b08d6 --- /dev/null +++ b/tests/zfs-tests/tests/functional/pam/.gitignore @@ -0,0 +1 @@ +/utilities.kshlib diff --git a/tests/zfs-tests/tests/functional/pam/Makefile.am b/tests/zfs-tests/tests/functional/pam/Makefile.am deleted file mode 100644 index be881faccf3b..000000000000 --- a/tests/zfs-tests/tests/functional/pam/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/pam -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - pam_basic.ksh \ - pam_nounmount.ksh \ - pam_short_password.ksh \ - utilities.kshlib diff --git a/tests/zfs-tests/tests/functional/pam/cleanup.ksh b/tests/zfs-tests/tests/functional/pam/cleanup.ksh index e41622d771b4..971c7fce64e5 100755 --- a/tests/zfs-tests/tests/functional/pam/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/pam/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -26,8 +26,4 @@ rmconfig destroy_pool $TESTPOOL del_user ${username} del_group pamtestgroup - -rm -rf "$runstatedir" -for dir in $TESTDIRS; do - rm -rf $dir -done +log_must rm -rf "$runstatedir" $TESTDIRS diff --git a/tests/zfs-tests/tests/functional/pam/pam_basic.ksh b/tests/zfs-tests/tests/functional/pam/pam_basic.ksh index eab819ab8d24..dc54b2d253f8 100755 --- a/tests/zfs-tests/tests/functional/pam/pam_basic.ksh +++ b/tests/zfs-tests/tests/functional/pam/pam_basic.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/pam/pam_nounmount.ksh b/tests/zfs-tests/tests/functional/pam/pam_nounmount.ksh index 29ce437b40ee..011a873e5bfd 100755 --- a/tests/zfs-tests/tests/functional/pam/pam_nounmount.ksh +++ b/tests/zfs-tests/tests/functional/pam/pam_nounmount.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/pam/pam_short_password.ksh b/tests/zfs-tests/tests/functional/pam/pam_short_password.ksh index 100f279fffed..443e07d7f003 100755 --- a/tests/zfs-tests/tests/functional/pam/pam_short_password.ksh +++ b/tests/zfs-tests/tests/functional/pam/pam_short_password.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/pam/setup.ksh b/tests/zfs-tests/tests/functional/pam/setup.ksh index 23515a598e72..f08758d2f2dc 100755 --- a/tests/zfs-tests/tests/functional/pam/setup.ksh +++ b/tests/zfs-tests/tests/functional/pam/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -22,9 +22,8 @@ . $STF_SUITE/tests/functional/pam/utilities.kshlib -if ! which pamtester; then - log_unsupported "pam tests require the pamtester utility to be installed" -fi +command -v pamtester > /dev/null || log_unsupported "pam tests require the pamtester utility to be installed" +[ -f "$pammodule" ] || log_unsupported "$pammodule missing" DISK=${DISKS%% *} create_pool $TESTPOOL "$DISK" diff --git a/tests/zfs-tests/tests/functional/pam/utilities.kshlib b/tests/zfs-tests/tests/functional/pam/utilities.kshlib.in similarity index 67% rename from tests/zfs-tests/tests/functional/pam/utilities.kshlib rename to tests/zfs-tests/tests/functional/pam/utilities.kshlib.in index d328034300b1..e118255dbd22 100644 --- a/tests/zfs-tests/tests/functional/pam/utilities.kshlib +++ b/tests/zfs-tests/tests/functional/pam/utilities.kshlib.in @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -24,24 +24,25 @@ username="pamtestuser" runstatedir="${TESTDIR}_run" +pammodule="@pammoduledir@/pam_zfs_key.so" pamservice="pam_zfs_key_test" pamconfig="/etc/pam.d/${pamservice}" function keystatus { - log_must [ "$(zfs list -Ho keystatus "$TESTPOOL/pam/${username}")" == "$1" ] + log_must [ "$(get_prop keystatus "$TESTPOOL/pam/${username}")" = "$1" ] } function genconfig { - for i in password auth session; do - printf "%s\trequired\tpam_permit.so\n%s\toptional\tpam_zfs_key.so\t%s\n" "$i" "$i" "$1" - done > "${pamconfig}" + printf '%s\trequired\tpam_permit.so\n%s\toptional\t%s\t%s\n' \ + password password "$pammodule" "$1" \ + auth auth "$pammodule" "$1" \ + session session "$pammodule" "$1" > "${pamconfig}" } function rmconfig { - log_must rm "${pamconfig}" + log_must rm -f "${pamconfig}" } function references { - log_must [ "$(cat "${runstatedir}/$(id -u ${username})")" == "$1" ] + log_must [ "$(<"${runstatedir}/$(id -u ${username})")" = "$1" ] } - diff --git a/tests/zfs-tests/tests/functional/pool_checkpoint/Makefile.am b/tests/zfs-tests/tests/functional/pool_checkpoint/Makefile.am deleted file mode 100644 index cc1c1183db79..000000000000 --- a/tests/zfs-tests/tests/functional/pool_checkpoint/Makefile.am +++ /dev/null @@ -1,26 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/pool_checkpoint -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - checkpoint_after_rewind.ksh \ - checkpoint_big_rewind.ksh \ - checkpoint_capacity.ksh \ - checkpoint_conf_change.ksh \ - checkpoint_discard_busy.ksh \ - checkpoint_discard.ksh \ - checkpoint_discard_many.ksh \ - checkpoint_indirect.ksh \ - checkpoint_invalid.ksh \ - checkpoint_lun_expsz.ksh \ - checkpoint_open.ksh \ - checkpoint_removal.ksh \ - checkpoint_rewind.ksh \ - checkpoint_ro_rewind.ksh \ - checkpoint_sm_scale.ksh \ - checkpoint_twice.ksh \ - checkpoint_vdev_add.ksh \ - checkpoint_zdb.ksh \ - checkpoint_zhack_feat.ksh - -dist_pkgdata_DATA = \ - pool_checkpoint.kshlib diff --git a/tests/zfs-tests/tests/functional/pool_checkpoint/checkpoint_lun_expsz.ksh b/tests/zfs-tests/tests/functional/pool_checkpoint/checkpoint_lun_expsz.ksh index 59f64081a977..7cfc3b1829bc 100755 --- a/tests/zfs-tests/tests/functional/pool_checkpoint/checkpoint_lun_expsz.ksh +++ b/tests/zfs-tests/tests/functional/pool_checkpoint/checkpoint_lun_expsz.ksh @@ -42,20 +42,24 @@ setup_nested_pools log_onexit cleanup_nested_pools populate_nested_pool -INITSZ=$(zpool list -v | grep "$FILEDISK1" | awk '{print $2}') +INITSZ=$(zpool list -v | awk -v d="$FILEDISK1" '$0 ~ d {print $2}') log_must zpool checkpoint $NESTEDPOOL log_must truncate -s $EXPSZ $FILEDISK1 log_must zpool online -e $NESTEDPOOL $FILEDISK1 -NEWSZ=$(zpool list -v | grep "$FILEDISK1" | awk '{print $2}') +NEWSZ=$(zpool list -v | awk -v d="$FILEDISK1" '$0 ~ d {print $2}') +DEXPSZ=$(zpool list -v | awk -v d="$FILEDISK1" '$0 ~ d {print $6}') nested_change_state_after_checkpoint log_mustnot [ "$INITSZ" = "$NEWSZ" ] +log_must [ "$DEXPSZ" = "-" ] log_must zpool export $NESTEDPOOL log_must zpool import -d $FILEDISKDIR --rewind-to-checkpoint $NESTEDPOOL nested_verify_pre_checkpoint_state -FINSZ=$(zpool list -v | grep "$FILEDISK1" | awk '{print $2}') -log_must [ "$INITSZ" = "$FINSZ" ] +FINSZ=$(zpool list -v | awk -v d="$FILEDISK1" '$0 ~ d {print $2}') +DEXPSZ=$(zpool list -v | awk -v d="$FILEDISK1" '$0 ~ d {print $6}') +log_must [ "$EXPSZ" = "$FINSZ" ] +log_must [ "$DEXPSZ" != "-" ] log_pass "LUN expansion rewinded correctly." diff --git a/tests/zfs-tests/tests/functional/pool_checkpoint/checkpoint_zdb.ksh b/tests/zfs-tests/tests/functional/pool_checkpoint/checkpoint_zdb.ksh index fdefc0c3bdbb..18ef2309c605 100755 --- a/tests/zfs-tests/tests/functional/pool_checkpoint/checkpoint_zdb.ksh +++ b/tests/zfs-tests/tests/functional/pool_checkpoint/checkpoint_zdb.ksh @@ -47,8 +47,8 @@ verify_runnable "global" # with the current state. The name of this temporary pool is the # name of the actual pool with the suffix below appended to it. # -CHECKPOINT_SUFFIX="_CHECKPOINTED_UNIVERSE" -CHECKPOINTED_FS1=$TESTPOOL$CHECKPOINT_SUFFIX/$TESTFS1 +BOGUS_SUFFIX="_CHECKPOINTED_UNIVERSE" +CHECKPOINTED_FS1=$TESTPOOL$BOGUS_SUFFIX/$TESTFS1 setup_test_pool log_onexit cleanup_test_pool @@ -58,41 +58,23 @@ log_must zpool checkpoint $TESTPOOL test_change_state_after_checkpoint -zdb $TESTPOOL | grep "Checkpointed uberblock found" || \ - log_fail "zdb could not find checkpointed uberblock" - -zdb -k $TESTPOOL | grep "Checkpointed uberblock found" && \ - log_fail "zdb found checkpointed uberblock in checkpointed state" - -zdb $TESTPOOL | grep "Dataset $FS1" && \ - log_fail "zdb found destroyed dataset in current state" - -zdb -k $TESTPOOL | grep "Dataset $CHECKPOINTED_FS1" || \ - log_fail "zdb could not find destroyed dataset in checkpoint" +log_must eval "zdb $TESTPOOL | grep -q \"Checkpointed uberblock found\"" +log_mustnot eval "zdb -k $TESTPOOL | grep -q \"Checkpointed uberblock found\"" +log_mustnot eval "zdb $TESTPOOL | grep \"Dataset $FS1\"" +log_must eval "zdb -k $TESTPOOL | grep \"Dataset $CHECKPOINTED_FS1\"" log_must zpool export $TESTPOOL -zdb -e $TESTPOOL | grep "Checkpointed uberblock found" || \ - log_fail "zdb could not find checkpointed uberblock" - -zdb -k -e $TESTPOOL | grep "Checkpointed uberblock found" && \ - log_fail "zdb found checkpointed uberblock in checkpointed state" - -zdb -e $TESTPOOL | grep "Dataset $FS1" && \ - log_fail "zdb found destroyed dataset in current state" - -zdb -k -e $TESTPOOL | grep "Dataset $CHECKPOINTED_FS1" || \ - log_fail "zdb could not find destroyed dataset in checkpoint" +log_must eval "zdb -e $TESTPOOL | grep \"Checkpointed uberblock found\"" +log_mustnot eval "zdb -k -e $TESTPOOL | grep \"Checkpointed uberblock found\"" +log_mustnot eval "zdb -e $TESTPOOL | grep \"Dataset $FS1\"" +log_must eval "zdb -k -e $TESTPOOL | grep \"Dataset $CHECKPOINTED_FS1\"" log_must zpool import $TESTPOOL log_must zpool checkpoint -d $TESTPOOL -zdb $TESTPOOL | grep "Checkpointed uberblock found" && \ - log_fail "zdb found checkpointed uberblock after discarding " \ - "the checkpoint" - -zdb -k $TESTPOOL && \ - log_fail "zdb opened checkpointed state that was discarded" +log_mustnot eval "zdb $TESTPOOL | grep \"Checkpointed uberblock found\"" +log_mustnot eval "zdb -k $TESTPOOL" log_pass "zdb can analyze checkpointed pools." diff --git a/tests/zfs-tests/tests/functional/pool_names/Makefile.am b/tests/zfs-tests/tests/functional/pool_names/Makefile.am deleted file mode 100644 index cd874861f0ac..000000000000 --- a/tests/zfs-tests/tests/functional/pool_names/Makefile.am +++ /dev/null @@ -1,4 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/pool_names -dist_pkgdata_SCRIPTS = \ - pool_names_001_pos.ksh \ - pool_names_002_neg.ksh diff --git a/tests/zfs-tests/tests/functional/pool_names/pool_names_001_pos.ksh b/tests/zfs-tests/tests/functional/pool_names/pool_names_001_pos.ksh index 45ef2fd67a96..3695feb94931 100755 --- a/tests/zfs-tests/tests/functional/pool_names/pool_names_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/pool_names/pool_names_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/pool_names/pool_names_002_neg.ksh b/tests/zfs-tests/tests/functional/pool_names/pool_names_002_neg.ksh index 4b6744563d07..2a688b6bfcee 100755 --- a/tests/zfs-tests/tests/functional/pool_names/pool_names_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/pool_names/pool_names_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/poolversion/Makefile.am b/tests/zfs-tests/tests/functional/poolversion/Makefile.am deleted file mode 100644 index 51c2046457df..000000000000 --- a/tests/zfs-tests/tests/functional/poolversion/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/poolversion -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - poolversion_001_pos.ksh \ - poolversion_002_pos.ksh diff --git a/tests/zfs-tests/tests/functional/poolversion/cleanup.ksh b/tests/zfs-tests/tests/functional/poolversion/cleanup.ksh index c9777b27e406..05d691313b40 100755 --- a/tests/zfs-tests/tests/functional/poolversion/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/poolversion/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/poolversion/poolversion_001_pos.ksh b/tests/zfs-tests/tests/functional/poolversion/poolversion_001_pos.ksh index 048cb29e3217..f759a1099cbc 100755 --- a/tests/zfs-tests/tests/functional/poolversion/poolversion_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/poolversion/poolversion_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -47,8 +47,7 @@ log_assert "zpool set version can upgrade a pool" for version in 1 2 3 4 5 6 7 8 do log_must zpool set version=$version $TESTPOOL - ACTUAL=$(zpool get version $TESTPOOL | grep version \ - | awk '{print $3}') + ACTUAL=$(get_pool_prop version $TESTPOOL) if [ "$ACTUAL" != "$version" ] then log_fail "v. $ACTUAL set for $TESTPOOL, expected v. $version!" diff --git a/tests/zfs-tests/tests/functional/poolversion/poolversion_002_pos.ksh b/tests/zfs-tests/tests/functional/poolversion/poolversion_002_pos.ksh index 69586473eaab..1d7e42fcf394 100755 --- a/tests/zfs-tests/tests/functional/poolversion/poolversion_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/poolversion/poolversion_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -50,7 +50,7 @@ log_must zpool set version=6 $TESTPOOL2 # verify it's actually that version - by checking the version property # and also by trying to set bootfs (which should fail if it is not version 6) -VERSION=$(zpool get version $TESTPOOL2| grep version | awk '{print $3}') +VERSION=$(get_pool_prop version $TESTPOOL2) if [ "$VERSION" != "6" ] then log_fail "Version $VERSION set for $TESTPOOL2 expected version 6!" @@ -62,7 +62,7 @@ log_mustnot zpool set version=5 $TESTPOOL2 log_mustnot zpool set version=-1 $TESTPOOL2 # verify the version is still 6 -VERSION=$(zpool get version $TESTPOOL2 | grep version | awk '{print $3}') +VERSION=$(get_pool_prop version $TESTPOOL2) if [ "$VERSION" != "6" ] then log_fail "Version $VERSION set for $TESTPOOL2, expected version 6!" diff --git a/tests/zfs-tests/tests/functional/poolversion/setup.ksh b/tests/zfs-tests/tests/functional/poolversion/setup.ksh index 660083b9fc00..9532404423d1 100755 --- a/tests/zfs-tests/tests/functional/poolversion/setup.ksh +++ b/tests/zfs-tests/tests/functional/poolversion/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/privilege/Makefile.am b/tests/zfs-tests/tests/functional/privilege/Makefile.am deleted file mode 100644 index ef26a750bb00..000000000000 --- a/tests/zfs-tests/tests/functional/privilege/Makefile.am +++ /dev/null @@ -1,6 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/privilege -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - privilege_001_pos.ksh \ - privilege_002_pos.ksh diff --git a/tests/zfs-tests/tests/functional/privilege/cleanup.ksh b/tests/zfs-tests/tests/functional/privilege/cleanup.ksh index 99985c670f2f..5acce6dc6555 100755 --- a/tests/zfs-tests/tests/functional/privilege/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/privilege/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/privilege/privilege_001_pos.ksh b/tests/zfs-tests/tests/functional/privilege/privilege_001_pos.ksh index af4f705679a5..5e8b0d401966 100755 --- a/tests/zfs-tests/tests/functional/privilege/privilege_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/privilege/privilege_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/privilege/privilege_002_pos.ksh b/tests/zfs-tests/tests/functional/privilege/privilege_002_pos.ksh index ab00e3295250..705c1a60e266 100755 --- a/tests/zfs-tests/tests/functional/privilege/privilege_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/privilege/privilege_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/privilege/setup.ksh b/tests/zfs-tests/tests/functional/privilege/setup.ksh index 4eb069394475..92b08b209a65 100755 --- a/tests/zfs-tests/tests/functional/privilege/setup.ksh +++ b/tests/zfs-tests/tests/functional/privilege/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -36,8 +36,7 @@ USES_NIS=false # if we're running NIS, turn it off until we clean up # (it can cause useradd to take a long time, hitting our TIMEOUT) -svcs svc:/network/nis/client:default | grep online > /dev/null -if [ $? -eq 0 ] +if svcs svc:/network/nis/client:default | grep -q online then svcadm disable svc:/network/nis/client:default USES_NIS=true diff --git a/tests/zfs-tests/tests/functional/procfs/Makefile.am b/tests/zfs-tests/tests/functional/procfs/Makefile.am deleted file mode 100644 index a7f022d9f210..000000000000 --- a/tests/zfs-tests/tests/functional/procfs/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/procfs -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - procfs_list_basic.ksh \ - procfs_list_concurrent_readers.ksh \ - procfs_list_stale_read.ksh \ - pool_state.ksh diff --git a/tests/zfs-tests/tests/functional/procfs/cleanup.ksh b/tests/zfs-tests/tests/functional/procfs/cleanup.ksh index 8fe46577e409..b3fb51695a2b 100755 --- a/tests/zfs-tests/tests/functional/procfs/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/procfs/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/procfs/pool_state.ksh b/tests/zfs-tests/tests/functional/procfs/pool_state.ksh index 080fdddb2d8f..7a02eb68abda 100755 --- a/tests/zfs-tests/tests/functional/procfs/pool_state.ksh +++ b/tests/zfs-tests/tests/functional/procfs/pool_state.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -106,7 +106,7 @@ log_assert "Testing /proc/spl/kstat/zfs//state kstat" check_all $TESTPOOL "ONLINE" # Fault one of the disks, and check that pool is degraded -DISK1=$(echo "$DISKS" | awk '{print $2}') +read -r DISK1 _ <<<"$DISKS" log_must zpool offline -tf $TESTPOOL $DISK1 check_all $TESTPOOL "DEGRADED" log_must zpool online $TESTPOOL $DISK1 diff --git a/tests/zfs-tests/tests/functional/procfs/procfs_list_basic.ksh b/tests/zfs-tests/tests/functional/procfs/procfs_list_basic.ksh index 9104e4ba2ac3..8eb35f85da32 100755 --- a/tests/zfs-tests/tests/functional/procfs/procfs_list_basic.ksh +++ b/tests/zfs-tests/tests/functional/procfs/procfs_list_basic.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -48,8 +48,8 @@ function cleanup function count_snap_cmds { typeset expected_count=$1 - count=$(grep -E "command: (lt-)?zfs snapshot $FS@testsnapshot" | wc -l) - log_must eval "[[ $count -eq $expected_count ]]" + count=$(grep -cE "command: (lt-)?zfs snapshot $FS@testsnapshot") + log_must [ "$count" -eq "$expected_count" ] } typeset -r ZFS_DBGMSG=/proc/spl/kstat/zfs/dbgmsg @@ -85,7 +85,7 @@ done # Clear out old messages and check that they really are gone echo 0 >$ZFS_DBGMSG || log_fail "failed to write to $ZFS_DBGMSG" -cat $ZFS_DBGMSG | count_snap_cmds 0 +count_snap_cmds 0 < $ZFS_DBGMSG # # Even though we don't expect any messages in the file, reading should still # succeed. diff --git a/tests/zfs-tests/tests/functional/procfs/procfs_list_concurrent_readers.ksh b/tests/zfs-tests/tests/functional/procfs/procfs_list_concurrent_readers.ksh index a24452ed5892..59052ae05c0d 100755 --- a/tests/zfs-tests/tests/functional/procfs/procfs_list_concurrent_readers.ksh +++ b/tests/zfs-tests/tests/functional/procfs/procfs_list_concurrent_readers.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -41,8 +41,7 @@ function cleanup { - [[ -z $msgs1 ]] || log_must rm $msgs1 - [[ -z $msgs2 ]] || log_must rm $msgs2 + log_must rm -f $msgs1 $msgs2 datasetexists $FS && destroy_dataset $FS -r } @@ -69,7 +68,7 @@ msgs2=$(mktemp) || log_fail # Start reading file, pause and read it from another process, and then finish # reading. # -{ dd bs=512 count=4; cat $ZFS_DBGMSG >$msgs1; cat; } <$ZFS_DBGMSG >$msgs2 +{ dd bs=512 count=4; cp $ZFS_DBGMSG $msgs1; cat; } <$ZFS_DBGMSG >$msgs2 # # Truncate the result of the read that completed second in case it picked up an diff --git a/tests/zfs-tests/tests/functional/procfs/procfs_list_stale_read.ksh b/tests/zfs-tests/tests/functional/procfs/procfs_list_stale_read.ksh index b3958b345d2a..be89e3b8c548 100755 --- a/tests/zfs-tests/tests/functional/procfs/procfs_list_stale_read.ksh +++ b/tests/zfs-tests/tests/functional/procfs/procfs_list_stale_read.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -74,9 +74,9 @@ function do_test # finish reading. # { - log_must dd bs=512 count=4 >/dev/null + log_must eval "dd bs=512 count=4 >/dev/null" log_must eval "$cmd" - cat 2>&1 >/dev/null | log_must grep "Input/output error" + log_must eval 'cat 2>&1 >/dev/null | grep "Input/output error"' } <$TXG_HIST } diff --git a/tests/zfs-tests/tests/functional/procfs/setup.ksh b/tests/zfs-tests/tests/functional/procfs/setup.ksh index 79fa28f4f1ac..f7d07d5d7211 100755 --- a/tests/zfs-tests/tests/functional/procfs/setup.ksh +++ b/tests/zfs-tests/tests/functional/procfs/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/Makefile.am b/tests/zfs-tests/tests/functional/projectquota/Makefile.am deleted file mode 100644 index e98c8672802e..000000000000 --- a/tests/zfs-tests/tests/functional/projectquota/Makefile.am +++ /dev/null @@ -1,27 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/projectquota -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - projectid_001_pos.ksh \ - projectid_002_pos.ksh \ - projectid_003_pos.ksh \ - projectquota_001_pos.ksh \ - projectquota_002_pos.ksh \ - projectquota_003_pos.ksh \ - projectquota_004_neg.ksh \ - projectquota_005_pos.ksh \ - projectquota_006_pos.ksh \ - projectquota_007_pos.ksh \ - projectquota_008_pos.ksh \ - projectquota_009_pos.ksh \ - projectspace_001_pos.ksh \ - projectspace_002_pos.ksh \ - projectspace_003_pos.ksh \ - projectspace_004_pos.ksh \ - projecttree_001_pos.ksh \ - projecttree_002_pos.ksh \ - projecttree_003_neg.ksh - -dist_pkgdata_DATA = \ - projectquota.cfg \ - projectquota_common.kshlib diff --git a/tests/zfs-tests/tests/functional/projectquota/cleanup.ksh b/tests/zfs-tests/tests/functional/projectquota/cleanup.ksh index 0440e3d8af8c..fd04aaabc566 100755 --- a/tests/zfs-tests/tests/functional/projectquota/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectid_001_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projectid_001_pos.ksh index 46e79062a0e2..ac5524b49d04 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectid_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectid_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectid_002_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projectid_002_pos.ksh index e382f464046b..ef87a1cb774b 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectid_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectid_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectid_003_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projectid_003_pos.ksh index d6dbaafc217a..93474b53709a 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectid_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectid_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectquota.cfg b/tests/zfs-tests/tests/functional/projectquota/projectquota.cfg index 564ab3ef9698..b8a37dbd05bb 100644 --- a/tests/zfs-tests/tests/functional/projectquota/projectquota.cfg +++ b/tests/zfs-tests/tests/functional/projectquota/projectquota.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectquota_001_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projectquota_001_pos.ksh index 3f8c3d68ce3e..f365bb9df617 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectquota_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectquota_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectquota_002_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projectquota_002_pos.ksh index 66cd1fb9ef71..7a92649ab4ad 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectquota_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectquota_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectquota_003_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projectquota_003_pos.ksh index 06f360d30b2b..5468759f7e12 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectquota_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectquota_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -54,7 +54,7 @@ log_onexit cleanup log_assert "Check the basic function of project{obj}used" sync_pool -typeset project_used=$(get_value "projectused@$PRJID1" $QFS) +typeset project_used=$(get_prop "projectused@$PRJID1" $QFS) typeset file_size='10m' if [[ $project_used != 0 ]]; then @@ -66,8 +66,8 @@ log_must user_run $PUSER mkdir $PRJDIR log_must chattr +P -p $PRJID1 $PRJDIR log_must user_run $PUSER mkfile $file_size $PRJDIR/qf sync_pool -project_used=$(get_value "projectused@$PRJID1" $QFS) -# get_value() reads the exact byte value which is slightly more than 10m +project_used=$(get_prop "projectused@$PRJID1" $QFS) +# get_prop() reads the exact byte value which is slightly more than 10m if [[ "$(($project_used/1024/1024))m" != "$file_size" ]]; then log_note "project $PRJID1 used is $project_used" log_fail "projectused for project $PRJID1 expected to be $file_size, " \ @@ -75,7 +75,7 @@ if [[ "$(($project_used/1024/1024))m" != "$file_size" ]]; then fi log_must rm -rf $PRJDIR -typeset project_obj_used=$(get_value "projectobjused@$PRJID2" $QFS) +typeset project_obj_used=$(get_prop "projectobjused@$PRJID2" $QFS) typeset file_count=100 if [[ $project_obj_used != 0 ]]; then @@ -88,7 +88,7 @@ log_must chattr +P -p $PRJID2 $PRJDIR # $PRJDIR has already used one object with the $PRJID2 log_must user_run $PUSER mkfiles $PRJDIR/qf_ $((file_count - 1)) sync_pool -project_obj_used=$(get_value "projectobjused@$PRJID2" $QFS) +project_obj_used=$(get_prop "projectobjused@$PRJID2" $QFS) if [[ $project_obj_used != $file_count ]]; then log_note "project $PRJID2 used is $project_obj_used" log_fail "projectobjused for project $PRJID2 expected to be " \ diff --git a/tests/zfs-tests/tests/functional/projectquota/projectquota_004_neg.ksh b/tests/zfs-tests/tests/functional/projectquota/projectquota_004_neg.ksh index 7ca81c3fbc81..a7a39f8fe7f7 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectquota_004_neg.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectquota_004_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectquota_005_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projectquota_005_pos.ksh index 0736648f1ed6..ecd2bb278561 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectquota_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectquota_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectquota_006_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projectquota_006_pos.ksh index 73554df6a42a..984f4340d922 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectquota_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectquota_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectquota_007_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projectquota_007_pos.ksh index 3572e0118f45..a06551b56c96 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectquota_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectquota_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectquota_008_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projectquota_008_pos.ksh index b045b2c5fce3..20b64995f64a 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectquota_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectquota_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectquota_009_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projectquota_009_pos.ksh index da44e731a92d..52d896577a98 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectquota_009_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectquota_009_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectquota_common.kshlib b/tests/zfs-tests/tests/functional/projectquota/projectquota_common.kshlib index 0582164f9d6c..eea5b5100501 100644 --- a/tests/zfs-tests/tests/functional/projectquota/projectquota_common.kshlib +++ b/tests/zfs-tests/tests/functional/projectquota/projectquota_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -75,21 +75,6 @@ function check_quota fi } -function get_value -{ - typeset prop_val - typeset prop=$1 - typeset dataset=$2 - - prop_val=$(zfs get -H -p -o value $prop $dataset 2>/dev/null) - if [[ $? -ne 0 ]]; then - log_note "Unable to get $prop property for dataset $dataset" - return 1 - fi - - echo $prop_val -} - function project_obj_count { typeset fs=$1 diff --git a/tests/zfs-tests/tests/functional/projectquota/projectspace_001_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projectspace_001_pos.ksh index 4005c2a408c6..97cb13b2343a 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectspace_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectspace_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectspace_002_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projectspace_002_pos.ksh index 1d48eccf22d0..6ea68ff32188 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectspace_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectspace_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectspace_003_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projectspace_003_pos.ksh index 8db5d0d89970..07b899c2ff12 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectspace_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectspace_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projectspace_004_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projectspace_004_pos.ksh index fc4a93f04463..c928080a36c7 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projectspace_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projectspace_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -65,10 +65,10 @@ log_must chattr +P -p $PRJID1 $PRJDIR log_must user_run $PUSER mkfile 50m $PRJDIR/qf sync_pool -total=$(df $PRJDIR | tail -n 1 | awk '{ print $2 }') +total=$(df $PRJDIR | awk 'END { print $2 }') [[ $total -eq 102400 ]] || log_fail "expect '102400' resource, but got '$total'" -used=$(df -i $PRJDIR | tail -n 1 | awk '{ print $5 }') +used=$(df -i $PRJDIR | awk 'END { print $5 }') [[ "$used" == "2%" ]] || log_fail "expect '2%' used, but got '$used'" log_pass "'df' on the directory with inherit project ID flag pass as expect" diff --git a/tests/zfs-tests/tests/functional/projectquota/projecttree_001_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projecttree_001_pos.ksh index 0402e345df2a..2e048868fec6 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projecttree_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projecttree_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projecttree_002_pos.ksh b/tests/zfs-tests/tests/functional/projectquota/projecttree_002_pos.ksh index 9942a88cb347..2494b8f7ef49 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projecttree_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projecttree_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/projecttree_003_neg.ksh b/tests/zfs-tests/tests/functional/projectquota/projecttree_003_neg.ksh index cbc45857f779..ec122301255a 100755 --- a/tests/zfs-tests/tests/functional/projectquota/projecttree_003_neg.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/projecttree_003_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/projectquota/setup.ksh b/tests/zfs-tests/tests/functional/projectquota/setup.ksh index c81b300e5e96..873d706afc01 100755 --- a/tests/zfs-tests/tests/functional/projectquota/setup.ksh +++ b/tests/zfs-tests/tests/functional/projectquota/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -45,12 +45,12 @@ log_must add_user $PGROUP $PUSER # # chmod 0750 $HOME # -user_run $PUSER zfs list -if [ $? -ne 0 ]; then +user_run $PUSER zfs list || log_unsupported "Test user $PUSER cannot execute zfs utilities" -fi DISK=${DISKS%% *} default_setup_noexit $DISK +zfs set compression=off $TESTPOOL + log_pass diff --git a/tests/zfs-tests/tests/functional/pyzfs/Makefile.am b/tests/zfs-tests/tests/functional/pyzfs/Makefile.am deleted file mode 100644 index 26c5ac595a5d..000000000000 --- a/tests/zfs-tests/tests/functional/pyzfs/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -include $(top_srcdir)/config/Substfiles.am - -pkgpyzfsdir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/pyzfs -pkgpyzfs_SCRIPTS = \ - pyzfs_unittest.ksh - -SUBSTFILES += $(pkgpyzfs_SCRIPTS) diff --git a/tests/zfs-tests/tests/functional/pyzfs/pyzfs_unittest.ksh.in b/tests/zfs-tests/tests/functional/pyzfs/pyzfs_unittest.ksh.in index 84e20e2e7eba..d55c4f3270ae 100755 --- a/tests/zfs-tests/tests/functional/pyzfs/pyzfs_unittest.ksh.in +++ b/tests/zfs-tests/tests/functional/pyzfs/pyzfs_unittest.ksh.in @@ -32,10 +32,8 @@ fi verify_runnable "global" # Verify that the required dependencies for testing are installed. -@PYTHON@ -c "import cffi" 2>/dev/null -if [ $? -eq 1 ]; then +@PYTHON@ -c "import cffi" 2>/dev/null || log_unsupported "python3-cffi not found by Python" -fi # We don't just try to "import libzfs_core" because we want to skip these tests # only if pyzfs was not installed due to missing, build-time, dependencies; if @@ -43,19 +41,16 @@ fi # mismatch, we want to report it. @PYTHON@ -c ' import pkgutil, sys -sys.exit(pkgutil.find_loader("libzfs_core") is None)' -if [ $? -eq 1 ]; then +sys.exit(pkgutil.find_loader("libzfs_core") is None)' || log_unsupported "libzfs_core not found by Python" -fi log_assert "Verify the nvlist and libzfs_core Python unittest run successfully" -# NOTE: don't use log_must() here because it makes output unreadable +# log_must buffers stderr, which interacts badly with +# no-output timeouts on CI runners @PYTHON@ -m unittest --verbose \ libzfs_core.test.test_nvlist.TestNVList \ - libzfs_core.test.test_libzfs_core.ZFSTest -if [ $? -ne 0 ]; then + libzfs_core.test.test_libzfs_core.ZFSTest || log_fail "Python unittest completed with errors" -fi log_pass "Python unittest completed without errors" diff --git a/tests/zfs-tests/tests/functional/quota/Makefile.am b/tests/zfs-tests/tests/functional/quota/Makefile.am deleted file mode 100644 index ba18bff7ecb7..000000000000 --- a/tests/zfs-tests/tests/functional/quota/Makefile.am +++ /dev/null @@ -1,14 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/quota -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - quota_001_pos.ksh \ - quota_002_pos.ksh \ - quota_003_pos.ksh \ - quota_004_pos.ksh \ - quota_005_pos.ksh \ - quota_006_neg.ksh - -dist_pkgdata_DATA = \ - quota.cfg \ - quota.kshlib diff --git a/tests/zfs-tests/tests/functional/quota/cleanup.ksh b/tests/zfs-tests/tests/functional/quota/cleanup.ksh index c4c369145dfc..8a6521193e66 100755 --- a/tests/zfs-tests/tests/functional/quota/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/quota/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/quota/quota.cfg b/tests/zfs-tests/tests/functional/quota/quota.cfg index 5b19f40163de..5cb6ae002dbd 100644 --- a/tests/zfs-tests/tests/functional/quota/quota.cfg +++ b/tests/zfs-tests/tests/functional/quota/quota.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/quota/quota.kshlib b/tests/zfs-tests/tests/functional/quota/quota.kshlib index 0ffe6394b54a..5083415c8956 100644 --- a/tests/zfs-tests/tests/functional/quota/quota.kshlib +++ b/tests/zfs-tests/tests/functional/quota/quota.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -59,8 +59,8 @@ function fill_quota file_write -o create -f $MNTPT/$TESTFILE1 -b $BLOCK_SIZE \ -c $write_size -d 0 zret=$? - [[ $zret -ne $EDQUOT ]] && \ - log_fail "Returned error code: $zret. Expected: $EDQUOT." + [[ $zret -ne $EDQUOT ]] && \ + log_fail "Returned error code: $zret. Expected: $EDQUOT." typeset -i file_size=`ls -lsk $MNTPT/$TESTFILE1 | awk '{ print $1 }'` typeset -i limit=0 diff --git a/tests/zfs-tests/tests/functional/quota/quota_001_pos.ksh b/tests/zfs-tests/tests/functional/quota/quota_001_pos.ksh index d6783e9a43ac..d124cb26ae98 100755 --- a/tests/zfs-tests/tests/functional/quota/quota_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/quota/quota_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/quota/quota_002_pos.ksh b/tests/zfs-tests/tests/functional/quota/quota_002_pos.ksh index 2f34072dd151..3af005e874e9 100755 --- a/tests/zfs-tests/tests/functional/quota/quota_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/quota/quota_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/quota/quota_003_pos.ksh b/tests/zfs-tests/tests/functional/quota/quota_003_pos.ksh index 6ab25cf2d463..de265813d55b 100755 --- a/tests/zfs-tests/tests/functional/quota/quota_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/quota/quota_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/quota/quota_004_pos.ksh b/tests/zfs-tests/tests/functional/quota/quota_004_pos.ksh index 373354438925..8f20b533da68 100755 --- a/tests/zfs-tests/tests/functional/quota/quota_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/quota/quota_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/quota/quota_005_pos.ksh b/tests/zfs-tests/tests/functional/quota/quota_005_pos.ksh index e87139a58f69..98ee4edae650 100755 --- a/tests/zfs-tests/tests/functional/quota/quota_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/quota/quota_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/quota/quota_006_neg.ksh b/tests/zfs-tests/tests/functional/quota/quota_006_neg.ksh index ca5ea5f82f5a..12105162c5b5 100755 --- a/tests/zfs-tests/tests/functional/quota/quota_006_neg.ksh +++ b/tests/zfs-tests/tests/functional/quota/quota_006_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/quota/setup.ksh b/tests/zfs-tests/tests/functional/quota/setup.ksh index 677cb12d79ac..8ff0592c7632 100755 --- a/tests/zfs-tests/tests/functional/quota/setup.ksh +++ b/tests/zfs-tests/tests/functional/quota/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -33,4 +33,6 @@ DISK=${DISKS%% *} -default_container_setup $DISK +default_setup_noexit $DISK "true" +log_must zfs set compression=off $TESTPOOL +log_pass diff --git a/tests/zfs-tests/tests/functional/raidz/Makefile.am b/tests/zfs-tests/tests/functional/raidz/Makefile.am deleted file mode 100644 index d93eb73cf832..000000000000 --- a/tests/zfs-tests/tests/functional/raidz/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/raidz -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - raidz_001_neg.ksh \ - raidz_002_pos.ksh \ - raidz_003_pos.ksh \ - raidz_004_pos.ksh diff --git a/tests/zfs-tests/tests/functional/raidz/cleanup.ksh b/tests/zfs-tests/tests/functional/raidz/cleanup.ksh index c92c54c270a7..c5bb8f9b4fa2 100755 --- a/tests/zfs-tests/tests/functional/raidz/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/raidz/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/raidz/raidz_001_neg.ksh b/tests/zfs-tests/tests/functional/raidz/raidz_001_neg.ksh index 0f88a1a51468..9bd616f3a257 100755 --- a/tests/zfs-tests/tests/functional/raidz/raidz_001_neg.ksh +++ b/tests/zfs-tests/tests/functional/raidz/raidz_001_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/raidz/raidz_002_pos.ksh b/tests/zfs-tests/tests/functional/raidz/raidz_002_pos.ksh index e238a881b054..746718ad9a58 100755 --- a/tests/zfs-tests/tests/functional/raidz/raidz_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/raidz/raidz_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/raidz/raidz_003_pos.ksh b/tests/zfs-tests/tests/functional/raidz/raidz_003_pos.ksh index bf22632c7eff..ce44906d5a4f 100755 --- a/tests/zfs-tests/tests/functional/raidz/raidz_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/raidz/raidz_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/raidz/raidz_004_pos.ksh b/tests/zfs-tests/tests/functional/raidz/raidz_004_pos.ksh index 6cd2bf7c9f60..0e3affd5143c 100755 --- a/tests/zfs-tests/tests/functional/raidz/raidz_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/raidz/raidz_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/raidz/setup.ksh b/tests/zfs-tests/tests/functional/raidz/setup.ksh index 4e155d24d544..47821e42915e 100755 --- a/tests/zfs-tests/tests/functional/raidz/setup.ksh +++ b/tests/zfs-tests/tests/functional/raidz/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/redacted_send/Makefile.am b/tests/zfs-tests/tests/functional/redacted_send/Makefile.am deleted file mode 100644 index 61d0ea21356e..000000000000 --- a/tests/zfs-tests/tests/functional/redacted_send/Makefile.am +++ /dev/null @@ -1,26 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/redacted_send -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - redacted_compressed.ksh \ - redacted_contents.ksh \ - redacted_deleted.ksh \ - redacted_disabled_feature.ksh \ - redacted_embedded.ksh \ - redacted_holes.ksh \ - redacted_incrementals.ksh \ - redacted_largeblocks.ksh \ - redacted_many_clones.ksh \ - redacted_mixed_recsize.ksh \ - redacted_mounts.ksh \ - redacted_negative.ksh \ - redacted_origin.ksh \ - redacted_panic.ksh \ - redacted_props.ksh \ - redacted_resume.ksh \ - redacted_size.ksh \ - redacted_volume.ksh - -dist_pkgdata_DATA = \ - redacted.cfg \ - redacted.kshlib diff --git a/tests/zfs-tests/tests/functional/redacted_send/cleanup.ksh b/tests/zfs-tests/tests/functional/redacted_send/cleanup.ksh index 1a7c142b8551..2c344f0ed3d8 100755 --- a/tests/zfs-tests/tests/functional/redacted_send/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/redacted_send/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/redacted_send/redacted.cfg b/tests/zfs-tests/tests/functional/redacted_send/redacted.cfg index f964b37bad3b..9608b87f107b 100644 --- a/tests/zfs-tests/tests/functional/redacted_send/redacted.cfg +++ b/tests/zfs-tests/tests/functional/redacted_send/redacted.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -23,8 +23,8 @@ # Copyright (c) 2018 by Delphix. All rights reserved. # -export DISK1=$(echo $DISKS | awk '{print $1}') -export DISK2=$(echo $DISKS | awk '{print $2}') +read -r DISK1 DISK2 _ <<<"$DISKS" +export DISK1 DISK2 export POOL=$TESTPOOL export POOL2=$TESTPOOL2 @@ -83,4 +83,4 @@ typeset RANGE12="0,2097152" typeset RANGE13="0,16384" typeset RANGE14="" typeset RANGE15="0,4194304" -typeset RANGE16="0,6291456" \ No newline at end of file +typeset RANGE16="0,6291456" diff --git a/tests/zfs-tests/tests/functional/redacted_send/redacted.kshlib b/tests/zfs-tests/tests/functional/redacted_send/redacted.kshlib index 30101939db64..cc1cf0411027 100644 --- a/tests/zfs-tests/tests/functional/redacted_send/redacted.kshlib +++ b/tests/zfs-tests/tests/functional/redacted_send/redacted.kshlib @@ -8,7 +8,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -29,6 +29,10 @@ . $STF_SUITE/tests/functional/rsend/rsend.kshlib . $STF_SUITE/tests/functional/redacted_send/redacted.cfg +if ! is_linux; then + alias udevadm=: +fi + function setup_dataset { typeset ds_name=$1 @@ -227,7 +231,7 @@ function compare_files [[ -f $file2 ]] || log_fail "File $file2 does not exist." log_must eval "get_diff $file1 $file2 >$tmpfile" - typeset range="$(cat $tmpfile)" + typeset range="$(<$tmpfile)" log_must unmount_redacted $recvfs [[ "$expected" = "$range" ]] || log_fail "Unexpected range: $range" } diff --git a/tests/zfs-tests/tests/functional/redacted_send/redacted_embedded.ksh b/tests/zfs-tests/tests/functional/redacted_send/redacted_embedded.ksh index 1c5b503a9be5..35faf038ad8f 100755 --- a/tests/zfs-tests/tests/functional/redacted_send/redacted_embedded.ksh +++ b/tests/zfs-tests/tests/functional/redacted_send/redacted_embedded.ksh @@ -60,13 +60,10 @@ for recsize in 512 1024 2048 4096 8192 16384; do log_must eval "zdb -ddddd $sendfs $send_obj >$tmpdir/send.zdb" log_must eval "zdb -ddddd $recvfs $recv_obj >$tmpdir/recv.zdb" - grep -q "EMBEDDED" $tmpdir/send.zdb || \ - log_fail "Obj $send_obj not embedded in $sendfs" - grep -q "EMBEDDED" $tmpdir/recv.zdb || \ - log_fail "Obj $recv_obj not embedded in $recvfs" + log_must grep -q "EMBEDDED" $tmpdir/send.zdb + log_must grep -q "EMBEDDED" $tmpdir/recv.zdb - cat $stream | zstream dump -v | log_must grep -q \ - "WRITE_EMBEDDED object = $send_obj offset = 0" + log_must eval "zstream dump -v $stream | grep -q \"WRITE_EMBEDDED object = $send_obj offset = 0\"" done log_must zfs destroy -R $recvfs @@ -91,13 +88,10 @@ for recsize in 1024 4096 16384; do log_must eval "zdb -ddddd $sendfs $send_obj >$tmpdir/send.zdb" log_must eval "zdb -ddddd $recvfs $recv_obj >$tmpdir/recv.zdb" - grep -q "EMBEDDED" $tmpdir/send.zdb || \ - log_fail "Obj $send_obj not embedded in $sendfs" - grep -q "EMBEDDED" $tmpdir/recv.zdb || \ - log_fail "Obj $recv_obj not embedded in $recvfs" + log_must grep -q "EMBEDDED" $tmpdir/send.zdb + log_must grep -q "EMBEDDED" $tmpdir/recv.zdb - cat $stream | zstream dump -v | log_must grep -q \ - "WRITE_EMBEDDED object = $send_obj offset = 0" + log_must eval "zstream dump -v $stream | log_must grep -q \"WRITE_EMBEDDED object = $send_obj offset = 0\"" done log_pass "Embedded blocks and redacted send work correctly together." diff --git a/tests/zfs-tests/tests/functional/redacted_send/redacted_incrementals.ksh b/tests/zfs-tests/tests/functional/redacted_send/redacted_incrementals.ksh index 1d2ed3a687be..18ea5d68fcf8 100755 --- a/tests/zfs-tests/tests/functional/redacted_send/redacted_incrementals.ksh +++ b/tests/zfs-tests/tests/functional/redacted_send/redacted_incrementals.ksh @@ -51,10 +51,10 @@ log_must eval "zfs receive $POOL2/rfs <$stream" # Verify receipt of normal incrementals to redaction list members. log_must eval "zfs send -i $sendfs@snap0 $POOL/stride3@snap >$stream" log_must eval "zfs recv $POOL2/rstride3 <$stream" -log_must diff -r /$POOL/stride3 /$POOL2/rstride3 +log_must directory_diff /$POOL/stride3 /$POOL2/rstride3 log_must eval "zfs send -i $sendfs@snap0 $POOL/stride5@snap >$stream" log_must eval "zfs recv $POOL2/rstride5 <$stream" -log_must diff -r /$POOL/stride5 /$POOL2/rstride5 +log_must directory_diff /$POOL/stride5 /$POOL2/rstride5 # But not a normal child that we weren't redacted with respect to. log_must eval "zfs send -i $sendfs@snap0 $POOL/hole@snap >$stream" @@ -73,7 +73,7 @@ log_must mount_redacted -f $POOL2/rint # Verify we can receive grandchildren on the child. log_must eval "zfs send -i $POOL/int@snap $POOL/rm@snap >$stream" log_must eval "zfs receive $POOL2/rrm <$stream" -log_must diff -r /$POOL/rm /$POOL2/rrm +log_must directory_diff /$POOL/rm /$POOL2/rrm # But not a grandchild that the received child wasn't redacted with respect to. log_must eval "zfs send -i $POOL/int@snap $POOL/write@snap >$stream" @@ -92,13 +92,13 @@ log_mustnot zfs redact $POOL/int@snap book6 $POOL/hole@snap # Verify we can receive a full clone of the grandchild on the child. log_must eval "zfs send $POOL/write@snap >$stream" log_must eval "zfs recv -o origin=$POOL2/rint@snap $POOL2/rwrite <$stream" -log_must diff -r /$POOL/write /$POOL2/rwrite +log_must directory_diff /$POOL/write /$POOL2/rwrite # Along with other origins. log_must eval "zfs recv -o origin=$POOL2/rfs@snap0 $POOL2/rwrite1 <$stream" -log_must diff -r /$POOL/write /$POOL2/rwrite1 +log_must directory_diff /$POOL/write /$POOL2/rwrite1 log_must eval "zfs recv -o origin=$POOL2@init $POOL2/rwrite2 <$stream" -log_must diff -r /$POOL/write /$POOL2/rwrite2 +log_must directory_diff /$POOL/write /$POOL2/rwrite2 log_must zfs destroy -R $POOL2/rwrite2 log_must zfs destroy -R $POOL2/rfs @@ -140,7 +140,7 @@ unmount_redacted $POOL2/rfs # sending from the bookmark. log_must eval "zfs send -i $sendfs#book7 $POOL/hole1@snap >$stream" log_must eval "zfs recv $POOL2/rhole1 <$stream" -log_must diff -r /$POOL/hole1 /$POOL2/rhole1 +log_must directory_diff /$POOL/hole1 /$POOL2/rhole1 # Verify we can receive an intermediate clone redacted with respect to a # non-subset if we send from the bookmark. diff --git a/tests/zfs-tests/tests/functional/redacted_send/redacted_largeblocks.ksh b/tests/zfs-tests/tests/functional/redacted_send/redacted_largeblocks.ksh index caccdd360061..ef85bb31dcea 100755 --- a/tests/zfs-tests/tests/functional/redacted_send/redacted_largeblocks.ksh +++ b/tests/zfs-tests/tests/functional/redacted_send/redacted_largeblocks.ksh @@ -58,6 +58,6 @@ unmount_redacted $recvfs log_must eval "zfs send -L -i $sendfs@snap $clone@snap1 >$stream" log_must stream_has_features $stream large_blocks log_must eval "zfs recv $recvfs/new <$stream" -log_must diff -r $clone_mnt $recv_mnt/new +log_must directory_diff $clone_mnt $recv_mnt/new log_pass "Large blocks and redacted send work correctly together." diff --git a/tests/zfs-tests/tests/functional/redacted_send/redacted_negative.ksh b/tests/zfs-tests/tests/functional/redacted_send/redacted_negative.ksh index e591cca0bbde..ea5e08c3cb86 100755 --- a/tests/zfs-tests/tests/functional/redacted_send/redacted_negative.ksh +++ b/tests/zfs-tests/tests/functional/redacted_send/redacted_negative.ksh @@ -45,11 +45,11 @@ log_must zfs snapshot $clone2@snap # Incompatible flags log_must zfs redact $sendfs@snap2 book $clone1@snap -log_mustnot eval "zfs send -R --redact book $sendfs@snap2 >$TEST_BASE_DIR/devnull" +log_mustnot eval "zfs send -R --redact book $sendfs@snap2 > /dev/null" typeset arg for arg in "$sendfs" "$clone1#book"; do - log_mustnot eval "zfs send --redact book $arg >$TEST_BASE_DIR/devnull" + log_mustnot eval "zfs send --redact book $arg > /dev/null" done # Bad redaction list arguments @@ -58,7 +58,7 @@ log_mustnot zfs redact $sendfs@snap1 book log_mustnot zfs redact $sendfs#book1 book4 $clone1 log_mustnot zfs redact $sendfs@snap1 book snap2 snap3 log_mustnot zfs redact $sendfs@snap1 book @snap2 @snap3 -log_mustnot eval "zfs send --redact $sendfs#book $sendfs@snap >$TEST_BASE_DIR/devnull" +log_mustnot eval "zfs send --redact $sendfs#book $sendfs@snap > /dev/null" # Redaction snapshots not a descendant of tosnap log_mustnot zfs redact $sendfs@snap2 book $sendfs@snap2 @@ -66,7 +66,7 @@ log_must zfs redact $sendfs@snap2 book2 $clone1@snap $clone2@snap log_must eval "zfs send --redact book2 $sendfs@snap2 >$stream" log_must zfs redact $sendfs@snap2 book3 $clone1@snap $clone2@snap log_must eval "zfs send -i $sendfs@snap1 --redact book3 $sendfs@snap2 \ - >$TEST_BASE_DIR/devnull" + > /dev/null" log_mustnot zfs redact $sendfs@snap3 $sendfs@snap3 $clone1@snap # Full redacted sends of redacted datasets are not allowed. diff --git a/tests/zfs-tests/tests/functional/redacted_send/redacted_props.ksh b/tests/zfs-tests/tests/functional/redacted_send/redacted_props.ksh index e4163c4ef8da..74b71cdf6954 100755 --- a/tests/zfs-tests/tests/functional/redacted_send/redacted_props.ksh +++ b/tests/zfs-tests/tests/functional/redacted_send/redacted_props.ksh @@ -66,8 +66,8 @@ get_guid_list $tmpdir/prop_list $sendfs#book1 get_guid_list $tmpdir/zdb_list $sendfs#book1 true get_guid_list $tmpdir/recvd_prop_list $recvfs@snap -count=$(wc -l $tmpdir/prop_list | awk '{print $1}') -[[ $count -eq 16 ]] || log_fail "Found incorrect number of redaction snapshots." +count=$(wc -l < $tmpdir/prop_list) +[ $count -eq 16 ] || log_fail "Found incorrect number of redaction snapshots." diff $tmpdir/prop_list $tmpdir/zdb_list || \ log_fail "Property list differed from zdb output" diff --git a/tests/zfs-tests/tests/functional/redacted_send/redacted_resume.ksh b/tests/zfs-tests/tests/functional/redacted_send/redacted_resume.ksh index 4ab04a0e5730..31f7b0ada82c 100755 --- a/tests/zfs-tests/tests/functional/redacted_send/redacted_resume.ksh +++ b/tests/zfs-tests/tests/functional/redacted_send/redacted_resume.ksh @@ -53,7 +53,7 @@ log_must mount_redacted -f $recvfs log_must set_tunable32 ALLOW_REDACTED_DATASET_MOUNT 1 log_must diff $send_mnt/f1 $recv_mnt/f1 log_must eval "get_diff $send_mnt/f2 $recv_mnt/f2 >$tmpdir/get_diff.out" -typeset range=$(cat $tmpdir/get_diff.out) +typeset range=$(<$tmpdir/get_diff.out) [[ "$RANGE9" = "$range" ]] || log_fail "Unexpected range: $range" log_must dd if=/dev/urandom of=$send_mnt/f3 bs=1024k count=3 @@ -70,7 +70,7 @@ resume_test "zfs send --redact book2 -i $sendfs#book1 $sendfs@snap2" \ log_must diff $send_mnt/f1 $recv_mnt/f1 log_must diff $send_mnt/f2 $recv_mnt/f2 log_must eval "get_diff $send_mnt/f3 $recv_mnt/f3 >$tmpdir/get_diff.out" -range=$(cat $tmpdir/get_diff.out) +range=$(<$tmpdir/get_diff.out) [[ "$RANGE10" = "$range" ]] || log_fail "Unexpected range: $range" # Test recv -A works properly and verify saved sends are not allowed @@ -81,7 +81,7 @@ log_must eval "zfs send --redact book1 $sendfs@snap >$stream" dd if=$stream bs=64k count=1 | log_mustnot zfs receive -s $recvfs [[ "-" = $(get_prop receive_resume_token $recvfs) ]] && \ log_fail "Receive token not found." -log_mustnot eval "zfs send --saved --redact book1 $recvfs >$TEST_BASE_DIR/devnull" +log_mustnot eval "zfs send --saved --redact book1 $recvfs > /dev/null" log_must zfs recv -A $recvfs log_must datasetnonexists $recvfs diff --git a/tests/zfs-tests/tests/functional/redacted_send/redacted_size.ksh b/tests/zfs-tests/tests/functional/redacted_send/redacted_size.ksh index 7456084b04bc..1634ffe06463 100755 --- a/tests/zfs-tests/tests/functional/redacted_send/redacted_size.ksh +++ b/tests/zfs-tests/tests/functional/redacted_send/redacted_size.ksh @@ -43,22 +43,19 @@ typeset clone_mnt="$(get_prop mountpoint $clone)" log_must rm -rf $clone_mnt/* log_must zfs snapshot $clone@snap log_must zfs redact $sendfs@snap book $clone@snap -log_must eval "zfs send -nvP --redact book $sendfs@snap | \ - grep '^size' | awk '{print \$2}' >$size" -log_must eval "zfs send --redact book $sendfs@snap | wc -c \ - >$size2" -bytes1=$(cat $size | tr -d '[[:space:]]') -bytes2=$(cat $size2 | tr -d '[[:space:]]') -[[ "$bytes1" -eq "$bytes2" ]] || \ +log_must eval "zfs send -nvP --redact book $sendfs@snap | awk '/^size/ {print \$2}' >$size" +log_must eval "zfs send --redact book $sendfs@snap | wc -c >$size2" +read -r bytes1 < $size +read -r bytes2 < $size2 +[ "$bytes1" -eq "$bytes2" ] || \ log_fail "Full sizes differ: estimate $bytes1 and actual $bytes2" log_must zfs snapshot $sendfs@snap2 -log_must eval "zfs send -nvP -i $sendfs#book $sendfs@snap2 | \ - grep '^size' | awk '{print \$2}' >$size" +log_must eval "zfs send -nvP -i $sendfs#book $sendfs@snap2 | awk '/^size/ {print \$2}' >$size" log_must eval "zfs send -i $sendfs#book $sendfs@snap2 | wc -c >$size2" -bytes1=$(cat $size | tr -d '[[:space:]]') -bytes2=$(cat $size2 | tr -d '[[:space:]]') -[[ "$bytes1" -eq "$bytes2" ]] || \ +read -r bytes1 < $size +read -r bytes2 < $size2 +[ "$bytes1" -eq "$bytes2" ] || \ log_fail "Incremental sizes differ: estimate $bytes1 and actual $bytes2" log_pass "Size estimates of redacted sends estimate accurately." diff --git a/tests/zfs-tests/tests/functional/redacted_send/setup.ksh b/tests/zfs-tests/tests/functional/redacted_send/setup.ksh index 3f537f813db0..6e08bec2e822 100755 --- a/tests/zfs-tests/tests/functional/redacted_send/setup.ksh +++ b/tests/zfs-tests/tests/functional/redacted_send/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/redundancy/Makefile.am b/tests/zfs-tests/tests/functional/redundancy/Makefile.am deleted file mode 100644 index 42c11c4aa957..000000000000 --- a/tests/zfs-tests/tests/functional/redundancy/Makefile.am +++ /dev/null @@ -1,22 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/redundancy -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - redundancy_draid.ksh \ - redundancy_draid1.ksh \ - redundancy_draid2.ksh \ - redundancy_draid3.ksh \ - redundancy_draid_damaged.ksh \ - redundancy_draid_spare1.ksh \ - redundancy_draid_spare2.ksh \ - redundancy_draid_spare3.ksh \ - redundancy_mirror.ksh \ - redundancy_raidz.ksh \ - redundancy_raidz1.ksh \ - redundancy_raidz2.ksh \ - redundancy_raidz3.ksh \ - redundancy_stripe.ksh - -dist_pkgdata_DATA = \ - redundancy.cfg \ - redundancy.kshlib diff --git a/tests/zfs-tests/tests/functional/redundancy/cleanup.ksh b/tests/zfs-tests/tests/functional/redundancy/cleanup.ksh index ba8d980710ee..1c95d752c5a0 100755 --- a/tests/zfs-tests/tests/functional/redundancy/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/redundancy/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/redundancy/redundancy.cfg b/tests/zfs-tests/tests/functional/redundancy/redundancy.cfg index 37b93820bc1d..3e7916257ff4 100644 --- a/tests/zfs-tests/tests/functional/redundancy/redundancy.cfg +++ b/tests/zfs-tests/tests/functional/redundancy/redundancy.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/redundancy/redundancy.kshlib b/tests/zfs-tests/tests/functional/redundancy/redundancy.kshlib index 9888034667a7..30818050a07a 100644 --- a/tests/zfs-tests/tests/functional/redundancy/redundancy.kshlib +++ b/tests/zfs-tests/tests/functional/redundancy/redundancy.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -139,10 +139,9 @@ function setup_test_env log_must truncate -s $MINVDEVSIZE $vdevs - log_must zpool create -f -m $TESTDIR $pool $keyword $vdevs + log_must zpool create -O compression=off -f -m $TESTDIR $pool $keyword $vdevs log_note "Filling up the filesystem ..." - typeset -i ret=0 typeset -i i=0 typeset file=$TESTDIR/file typeset -i limit @@ -150,9 +149,7 @@ function setup_test_env while true ; do [[ $(get_prop available $pool) -lt $limit ]] && break - file_write -o create -f $file.$i -b $BLOCKSZ -c $NUM_WRITES - ret=$? - (( $ret != 0 )) && break + file_write -o create -f $file.$i -b $BLOCKSZ -c $NUM_WRITES || break (( i = i + 1 )) done @@ -163,16 +160,13 @@ function refill_test_env { log_note "Re-filling the filesystem ..." typeset pool=$1 - typeset -i ret=0 typeset -i i=0 typeset mntpnt mntpnt=$(get_prop mountpoint $pool) typeset file=$mntpnt/file while [[ -e $file.$i ]]; do log_must rm -f $file.$i - file_write -o create -f $file.$i -b $BLOCKSZ -c $NUM_WRITES - ret=$? - (( $ret != 0 )) && break + file_write -o create -f $file.$i -b $BLOCKSZ -c $NUM_WRITES || break (( i = i + 1 )) done @@ -195,15 +189,12 @@ function is_healthy return 0 else typeset -i ret - zpool status -x $pool | grep "state:" | \ - grep "FAULTED" >/dev/null 2>&1 - ret=$? - (( $ret == 0 )) && return 1 + zpool status -x $pool | grep "state:" | grep -q "FAULTED" && return 1 typeset l_scan - typeset errnum + typeset errnum _ l_scan=$(zpool status -x $pool | grep "scan:") l_scan=${l_scan##*"with"} - errnum=$(echo $l_scan | awk '{print $1}') + read -r errnum _ <<<"$l_scan" return $errnum fi @@ -221,7 +212,7 @@ function is_data_valid log_must zpool scrub -w $pool record_data $pool $PST_RECORD_FILE - if ! diff $PRE_RECORD_FILE $PST_RECORD_FILE > /dev/null 2>&1; then + if ! cmp $PRE_RECORD_FILE $PST_RECORD_FILE > /dev/null; then log_must cat $PRE_RECORD_FILE log_must cat $PST_RECORD_FILE diff -u $PRE_RECORD_FILE $PST_RECORD_FILE @@ -242,13 +233,14 @@ function get_vdevs #pool cnt typeset pool=$1 typeset -i cnt=$2 - typeset all_devs=$(zpool iostat -v $pool | awk '{print $1}'| \ - egrep -v "^pool$|^capacity$|^mirror\-[0-9]$|^raidz[1-3]\-[0-9]$|^draid[1-3].*\-[0-9]$|---" | \ - egrep -v "/old$|^$pool$") + typeset all_devs=$(zpool iostat -v $pool | awk '{print $1}' | \ + grep -vEe "^pool$|^capacity$|^mirror\-[0-9]$|^raidz[1-3]\-[0-9]$|^draid[1-3].*\-[0-9]$|---" \ + -e "/old$|^$pool$") typeset -i i=0 typeset vdevs while ((i < cnt)); do - typeset dev=$(echo $all_devs | awk '{print $1}') + typeset dev _ + read -r dev _ <<<"$all_devs" eval all_devs=\${all_devs##*$dev} vdevs="$dev $vdevs" diff --git a/tests/zfs-tests/tests/functional/redundancy/redundancy_draid.ksh b/tests/zfs-tests/tests/functional/redundancy/redundancy_draid.ksh index 8015e682c892..8208d2b4a398 100755 --- a/tests/zfs-tests/tests/functional/redundancy/redundancy_draid.ksh +++ b/tests/zfs-tests/tests/functional/redundancy/redundancy_draid.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -77,7 +77,7 @@ function test_selfheal # log_must zpool import -o cachefile=none -d $dir $pool typeset mntpnt=$(get_prop mountpoint $pool/fs) - log_must find $mntpnt -type f -exec cksum {} + >> /dev/null 2>&1 + log_must eval "find $mntpnt -type f -exec cksum {} + >> /dev/null 2>&1" log_must check_pool_status $pool "errors" "No known data errors" # @@ -100,7 +100,7 @@ function test_selfheal # log_must zpool import -o cachefile=none -d $dir $pool typeset mntpnt=$(get_prop mountpoint $pool/fs) - log_must find $mntpnt -type f -exec cksum {} + >> /dev/null 2>&1 + log_must eval "find $mntpnt -type f -exec cksum {} + >> /dev/null 2>&1" log_must check_pool_status $pool "errors" "No known data errors" log_must zpool scrub -w $pool @@ -219,7 +219,7 @@ for nparity in 1 2 3; do raid=draid$nparity dir=$TEST_BASE_DIR - log_must zpool create -f -o cachefile=none $TESTPOOL $raid ${disks[@]} + log_must zpool create -O compression=off -f -o cachefile=none $TESTPOOL $raid ${disks[@]} log_must zfs set primarycache=metadata $TESTPOOL log_must zfs create $TESTPOOL/fs diff --git a/tests/zfs-tests/tests/functional/redundancy/redundancy_draid1.ksh b/tests/zfs-tests/tests/functional/redundancy/redundancy_draid1.ksh index 85d420ab0d3a..a59743592eca 100755 --- a/tests/zfs-tests/tests/functional/redundancy/redundancy_draid1.ksh +++ b/tests/zfs-tests/tests/functional/redundancy/redundancy_draid1.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/redundancy/redundancy_draid2.ksh b/tests/zfs-tests/tests/functional/redundancy/redundancy_draid2.ksh index 04f1fdfb150d..48c4a24cb496 100755 --- a/tests/zfs-tests/tests/functional/redundancy/redundancy_draid2.ksh +++ b/tests/zfs-tests/tests/functional/redundancy/redundancy_draid2.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/redundancy/redundancy_draid3.ksh b/tests/zfs-tests/tests/functional/redundancy/redundancy_draid3.ksh index d4c823ed9b37..8e1ad237b5a2 100755 --- a/tests/zfs-tests/tests/functional/redundancy/redundancy_draid3.ksh +++ b/tests/zfs-tests/tests/functional/redundancy/redundancy_draid3.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/redundancy/redundancy_draid_damaged.ksh b/tests/zfs-tests/tests/functional/redundancy/redundancy_draid_damaged1.ksh similarity index 83% rename from tests/zfs-tests/tests/functional/redundancy/redundancy_draid_damaged.ksh rename to tests/zfs-tests/tests/functional/redundancy/redundancy_draid_damaged1.ksh index 6796cc78a1bd..110c69159eb1 100755 --- a/tests/zfs-tests/tests/functional/redundancy/redundancy_draid_damaged.ksh +++ b/tests/zfs-tests/tests/functional/redundancy/redundancy_draid_damaged1.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -89,22 +89,9 @@ function test_sequential_resilver # done log_must zpool scrub -w $pool + log_must zpool status $pool - # When only a single child was overwritten the sequential resilver - # can fully repair the damange from parity and the scrub will have - # nothing to repair. When multiple children are silently damaged - # the sequential resilver will calculate the wrong data since only - # the parity information is used and it cannot be verified with - # the checksum. However, since only the resilvering devices are - # written to with the bad data a subsequent scrub will be able to - # fully repair the pool. - # - if [[ $nparity == 1 ]]; then - log_must check_pool_status $pool "scan" "repaired 0B" - else - log_mustnot check_pool_status $pool "scan" "repaired 0B" - fi - + log_mustnot check_pool_status $pool "scan" "repaired 0B" log_must check_pool_status $pool "errors" "No known data errors" log_must check_pool_status $pool "scan" "with 0 errors" } @@ -128,7 +115,7 @@ for nparity in 1 2 3; do raid=draid${nparity}:${nparity}s dir=$TEST_BASE_DIR - log_must zpool create -f -o cachefile=none $TESTPOOL $raid ${disks[@]} + log_must zpool create -O compression=off -f -o cachefile=none $TESTPOOL $raid ${disks[@]} log_must zfs set primarycache=metadata $TESTPOOL log_must zfs create $TESTPOOL/fs diff --git a/tests/zfs-tests/tests/functional/redundancy/redundancy_draid_damaged2.ksh b/tests/zfs-tests/tests/functional/redundancy/redundancy_draid_damaged2.ksh new file mode 100755 index 000000000000..b0bb4ef84129 --- /dev/null +++ b/tests/zfs-tests/tests/functional/redundancy/redundancy_draid_damaged2.ksh @@ -0,0 +1,157 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright (c) 2022 by Lawrence Livermore National Security, LLC. +# + +. $STF_SUITE/include/libtest.shlib +. $STF_SUITE/tests/functional/redundancy/redundancy.kshlib + +# +# DESCRIPTION: +# When sequentially resilvering a dRAID pool to a distributed spare +# silent damage to an online vdev in a replacing or spare mirror vdev +# is not expected to be repaired. Not only does the rebuild have no +# reason to suspect the silent damage but even if it did there's no +# checksum available to determine the correct copy and make the repair. +# However, the subsequent scrub should detect and repair any damage. +# +# STRATEGY: +# 1. Create block device files for the test draid pool +# 2. For each parity value [1..3] +# a. Create a draid pool +# b. Fill it with some directories/files +# c. Systematically damage and replace three devices by: +# - Overwrite the device +# - Replace the damaged vdev with a distributed spare +# - Scrub the pool and verify repair IO is issued +# d. Detach the distributed spares +# e. Scrub the pool and verify there was nothing to repair +# f. Destroy the draid pool +# + +typeset -r devs=7 +typeset -r dev_size_mb=512 +typeset -a disks + +prefetch_disable=$(get_tunable PREFETCH_DISABLE) +rebuild_scrub_enabled=$(get_tunable REBUILD_SCRUB_ENABLED) + +function cleanup +{ + poolexists "$TESTPOOL" && destroy_pool "$TESTPOOL" + + for i in {0..$devs}; do + rm -f "$TEST_BASE_DIR/dev-$i" + done + + set_tunable32 PREFETCH_DISABLE $prefetch_disable + set_tunable32 REBUILD_SCRUB_ENABLED $rebuild_scrub_enabled +} + +log_onexit cleanup + +log_must set_tunable32 PREFETCH_DISABLE 1 +log_must set_tunable32 REBUILD_SCRUB_ENABLED 0 + +# Disk files which will be used by pool +for i in {0..$(($devs - 1))}; do + device=$TEST_BASE_DIR/dev-$i + log_must truncate -s ${dev_size_mb}M $device + disks[${#disks[*]}+1]=$device +done + +# Disk file which will be attached +log_must truncate -s 512M $TEST_BASE_DIR/dev-$devs + +dir=$TEST_BASE_DIR + +for nparity in 1 2 3; do + raid=draid${nparity}:3s + + log_must zpool create -f -O compression=off -o cachefile=none \ + $TESTPOOL $raid ${disks[@]} + # log_must zfs set primarycache=metadata $TESTPOOL + + log_must zfs create $TESTPOOL/fs + log_must fill_fs /$TESTPOOL/fs 1 256 10 1024 R + + log_must zfs create -o compress=on $TESTPOOL/fs2 + log_must fill_fs /$TESTPOOL/fs2 1 256 10 1024 R + + log_must zfs create -o compress=on -o recordsize=8k $TESTPOOL/fs3 + log_must fill_fs /$TESTPOOL/fs3 1 256 10 1024 R + + log_must zpool export $TESTPOOL + log_must zpool import -o cachefile=none -d $dir $TESTPOOL + + log_must check_pool_status $TESTPOOL "errors" "No known data errors" + + for nspare in 0 1 2; do + damaged=$dir/dev-${nspare} + spare=draid${nparity}-0-${nspare} + + log_must zpool export $TESTPOOL + log_must dd conv=notrunc if=/dev/zero of=$damaged \ + bs=1M seek=4 count=$(($dev_size_mb-4)) + log_must zpool import -o cachefile=none -d $dir $TESTPOOL + + log_must zpool replace -fsw $TESTPOOL $damaged $spare + + # Scrub the pool after the sequential resilver and verify + # that the silent damage was repaired by the scrub. + log_must zpool scrub -w $TESTPOOL + log_must zpool status $TESTPOOL + log_must check_pool_status $TESTPOOL "errors" \ + "No known data errors" + log_must check_pool_status $TESTPOOL "scan" "with 0 errors" + log_mustnot check_pool_status $TESTPOOL "scan" "repaired 0B" + done + + for nspare in 0 1 2; do + log_must check_vdev_state $TESTPOOL \ + spare-${nspare} "ONLINE" + log_must check_vdev_state $TESTPOOL \ + ${dir}/dev-${nspare} "ONLINE" + log_must check_vdev_state $TESTPOOL \ + draid${nparity}-0-${nspare} "ONLINE" + done + + # Detach the distributed spares and scrub the pool again to + # verify no damage remained on the originally corrupted vdevs. + for nspare in 0 1 2; do + log_must zpool detach $TESTPOOL draid${nparity}-0-${nspare} + done + + log_must zpool clear $TESTPOOL + log_must zpool scrub -w $TESTPOOL + log_must zpool status $TESTPOOL + + log_must check_pool_status $TESTPOOL "errors" "No known data errors" + log_must check_pool_status $TESTPOOL "scan" "with 0 errors" + log_must check_pool_status $TESTPOOL "scan" "repaired 0B" + + log_must zpool destroy "$TESTPOOL" +done + +log_pass "draid damaged device scrub test succeeded." diff --git a/tests/zfs-tests/tests/functional/redundancy/redundancy_draid_spare3.ksh b/tests/zfs-tests/tests/functional/redundancy/redundancy_draid_spare3.ksh index 28e8e3c6d707..dd4bca2795b9 100755 --- a/tests/zfs-tests/tests/functional/redundancy/redundancy_draid_spare3.ksh +++ b/tests/zfs-tests/tests/functional/redundancy/redundancy_draid_spare3.ksh @@ -48,10 +48,6 @@ function cleanup_tunable log_onexit cleanup_tunable -if is_kmemleak; then - log_unsupported "Test case runs slowly when kmemleak is enabled" -fi - # # Disable scrubbing after a sequential resilver to verify the resilver # alone is able to reconstruct the data without the help of a scrub. diff --git a/tests/zfs-tests/tests/functional/redundancy/redundancy_mirror.ksh b/tests/zfs-tests/tests/functional/redundancy/redundancy_mirror.ksh index b7b791b248ee..3410ac018b83 100755 --- a/tests/zfs-tests/tests/functional/redundancy/redundancy_mirror.ksh +++ b/tests/zfs-tests/tests/functional/redundancy/redundancy_mirror.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz.ksh b/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz.ksh index d73688391624..83cacda84b09 100755 --- a/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz.ksh +++ b/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -77,7 +77,7 @@ function test_selfheal # log_must zpool import -o cachefile=none -d $dir $pool typeset mntpnt=$(get_prop mountpoint $pool/fs) - log_must find $mntpnt -type f -exec cksum {} + >> /dev/null 2>&1 + log_must eval "find $mntpnt -type f -exec cksum {} + >> /dev/null 2>&1" log_must check_pool_status $pool "errors" "No known data errors" # @@ -100,7 +100,7 @@ function test_selfheal # log_must zpool import -o cachefile=none -d $dir $pool typeset mntpnt=$(get_prop mountpoint $pool/fs) - log_must find $mntpnt -type f -exec cksum {} + >> /dev/null 2>&1 + log_must eval "find $mntpnt -type f -exec cksum {} + >> /dev/null 2>&1" log_must check_pool_status $pool "errors" "No known data errors" log_must zpool scrub -w $pool @@ -219,7 +219,7 @@ for nparity in 1 2 3; do raid=raidz$nparity dir=$TEST_BASE_DIR - log_must zpool create -f -o cachefile=none $TESTPOOL $raid ${disks[@]} + log_must zpool create -O compression=off -f -o cachefile=none $TESTPOOL $raid ${disks[@]} log_must zfs set primarycache=metadata $TESTPOOL log_must zfs create $TESTPOOL/fs diff --git a/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz1.ksh b/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz1.ksh index a73890e4cc05..5700aa440606 100755 --- a/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz1.ksh +++ b/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz1.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz2.ksh b/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz2.ksh index 94b9b8825154..d73d80114b6b 100755 --- a/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz2.ksh +++ b/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz2.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz3.ksh b/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz3.ksh index 0a01c47106b3..9839d9b63145 100755 --- a/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz3.ksh +++ b/tests/zfs-tests/tests/functional/redundancy/redundancy_raidz3.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/redundancy/redundancy_stripe.ksh b/tests/zfs-tests/tests/functional/redundancy/redundancy_stripe.ksh index b2c4a85febef..774226acda96 100755 --- a/tests/zfs-tests/tests/functional/redundancy/redundancy_stripe.ksh +++ b/tests/zfs-tests/tests/functional/redundancy/redundancy_stripe.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/redundancy/setup.ksh b/tests/zfs-tests/tests/functional/redundancy/setup.ksh index e2c04fe5cfbd..0ac48c82a810 100755 --- a/tests/zfs-tests/tests/functional/redundancy/setup.ksh +++ b/tests/zfs-tests/tests/functional/redundancy/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/refquota/Makefile.am b/tests/zfs-tests/tests/functional/refquota/Makefile.am deleted file mode 100644 index 1d8418fbbec5..000000000000 --- a/tests/zfs-tests/tests/functional/refquota/Makefile.am +++ /dev/null @@ -1,12 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/refquota -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - refquota_001_pos.ksh \ - refquota_002_pos.ksh \ - refquota_003_pos.ksh \ - refquota_004_pos.ksh \ - refquota_005_pos.ksh \ - refquota_006_neg.ksh \ - refquota_007_neg.ksh \ - refquota_008_neg.ksh diff --git a/tests/zfs-tests/tests/functional/refquota/cleanup.ksh b/tests/zfs-tests/tests/functional/refquota/cleanup.ksh index ea139ae0395b..2f0a02ed8dd3 100755 --- a/tests/zfs-tests/tests/functional/refquota/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/refquota/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/refquota/refquota_001_pos.ksh b/tests/zfs-tests/tests/functional/refquota/refquota_001_pos.ksh index 411fab8459af..6583210ca3ec 100755 --- a/tests/zfs-tests/tests/functional/refquota/refquota_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/refquota/refquota_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/refquota/refquota_002_pos.ksh b/tests/zfs-tests/tests/functional/refquota/refquota_002_pos.ksh index e1d3aa82033c..e2c42529c9d4 100755 --- a/tests/zfs-tests/tests/functional/refquota/refquota_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/refquota/refquota_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/refquota/refquota_003_pos.ksh b/tests/zfs-tests/tests/functional/refquota/refquota_003_pos.ksh index e4def1a0a7b4..7f7c2ba683f1 100755 --- a/tests/zfs-tests/tests/functional/refquota/refquota_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/refquota/refquota_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/refquota/refquota_004_pos.ksh b/tests/zfs-tests/tests/functional/refquota/refquota_004_pos.ksh index 0691c3630931..8051f9af833c 100755 --- a/tests/zfs-tests/tests/functional/refquota/refquota_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/refquota/refquota_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/refquota/refquota_005_pos.ksh b/tests/zfs-tests/tests/functional/refquota/refquota_005_pos.ksh index 13977bc3b48f..cfccff5f1af5 100755 --- a/tests/zfs-tests/tests/functional/refquota/refquota_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/refquota/refquota_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/refquota/refquota_006_neg.ksh b/tests/zfs-tests/tests/functional/refquota/refquota_006_neg.ksh index 39b317512df2..d0d9d12b0813 100755 --- a/tests/zfs-tests/tests/functional/refquota/refquota_006_neg.ksh +++ b/tests/zfs-tests/tests/functional/refquota/refquota_006_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/refquota/setup.ksh b/tests/zfs-tests/tests/functional/refquota/setup.ksh index a34453bd3892..aae97e80f7b3 100755 --- a/tests/zfs-tests/tests/functional/refquota/setup.ksh +++ b/tests/zfs-tests/tests/functional/refquota/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -33,4 +33,6 @@ verify_runnable "both" DISK=${DISKS%% *} -default_setup $DISK +default_setup_noexit $DISK +log_must zfs set compression=off $TESTPOOL +log_pass diff --git a/tests/zfs-tests/tests/functional/refreserv/Makefile.am b/tests/zfs-tests/tests/functional/refreserv/Makefile.am deleted file mode 100644 index bd760a1f0697..000000000000 --- a/tests/zfs-tests/tests/functional/refreserv/Makefile.am +++ /dev/null @@ -1,14 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/refreserv -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - refreserv_001_pos.ksh \ - refreserv_002_pos.ksh \ - refreserv_003_pos.ksh \ - refreserv_004_pos.ksh \ - refreserv_005_pos.ksh \ - refreserv_multi_raidz.ksh \ - refreserv_raidz.ksh - -dist_pkgdata_DATA = \ - refreserv.cfg diff --git a/tests/zfs-tests/tests/functional/refreserv/cleanup.ksh b/tests/zfs-tests/tests/functional/refreserv/cleanup.ksh index ea139ae0395b..2f0a02ed8dd3 100755 --- a/tests/zfs-tests/tests/functional/refreserv/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/refreserv/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/refreserv/refreserv.cfg b/tests/zfs-tests/tests/functional/refreserv/refreserv.cfg index 8c892db4a3a2..818e7e8283a2 100644 --- a/tests/zfs-tests/tests/functional/refreserv/refreserv.cfg +++ b/tests/zfs-tests/tests/functional/refreserv/refreserv.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/refreserv/refreserv_001_pos.ksh b/tests/zfs-tests/tests/functional/refreserv/refreserv_001_pos.ksh index 527949f27c10..82904663b668 100755 --- a/tests/zfs-tests/tests/functional/refreserv/refreserv_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/refreserv/refreserv_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/refreserv/refreserv_002_pos.ksh b/tests/zfs-tests/tests/functional/refreserv/refreserv_002_pos.ksh index a8f58631f7b6..ed45eb782e12 100755 --- a/tests/zfs-tests/tests/functional/refreserv/refreserv_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/refreserv/refreserv_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -68,8 +68,7 @@ function max_refreserv log_must zfs set refreserv=$rr $ds while :; do - zfs set refreserv=$((rr + incsize)) $ds >/dev/null 2>&1 - if [[ $? == 0 ]]; then + if zfs set refreserv=$((rr + incsize)) $ds >/dev/null 2>&1; then ((rr += incsize)) continue else diff --git a/tests/zfs-tests/tests/functional/refreserv/refreserv_003_pos.ksh b/tests/zfs-tests/tests/functional/refreserv/refreserv_003_pos.ksh index 3e5a78cf944f..aa910facb984 100755 --- a/tests/zfs-tests/tests/functional/refreserv/refreserv_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/refreserv/refreserv_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/refreserv/refreserv_004_pos.ksh b/tests/zfs-tests/tests/functional/refreserv/refreserv_004_pos.ksh index 529d918c3695..ec67ccf44057 100755 --- a/tests/zfs-tests/tests/functional/refreserv/refreserv_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/refreserv/refreserv_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/refreserv/refreserv_005_pos.ksh b/tests/zfs-tests/tests/functional/refreserv/refreserv_005_pos.ksh index 1ccc9828d4f7..e078e76ad698 100755 --- a/tests/zfs-tests/tests/functional/refreserv/refreserv_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/refreserv/refreserv_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/refreserv/refreserv_raidz.ksh b/tests/zfs-tests/tests/functional/refreserv/refreserv_raidz.ksh index 056c7916d990..0c8de79b3f9f 100755 --- a/tests/zfs-tests/tests/functional/refreserv/refreserv_raidz.ksh +++ b/tests/zfs-tests/tests/functional/refreserv/refreserv_raidz.ksh @@ -98,7 +98,7 @@ for parity in 1 2 3; do continue fi - log_must zpool create "$TESTPOOL" "$raid" "${disks[@]}" + log_must zpool create -O compression=off "$TESTPOOL" "$raid" "${disks[@]}" for bits in "${allshifts[@]}"; do vbs=$((1 << bits)) diff --git a/tests/zfs-tests/tests/functional/refreserv/setup.ksh b/tests/zfs-tests/tests/functional/refreserv/setup.ksh index a34453bd3892..aae97e80f7b3 100755 --- a/tests/zfs-tests/tests/functional/refreserv/setup.ksh +++ b/tests/zfs-tests/tests/functional/refreserv/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -33,4 +33,6 @@ verify_runnable "both" DISK=${DISKS%% *} -default_setup $DISK +default_setup_noexit $DISK +log_must zfs set compression=off $TESTPOOL +log_pass diff --git a/tests/zfs-tests/tests/functional/removal/Makefile.am b/tests/zfs-tests/tests/functional/removal/Makefile.am deleted file mode 100644 index 878935b96d3c..000000000000 --- a/tests/zfs-tests/tests/functional/removal/Makefile.am +++ /dev/null @@ -1,38 +0,0 @@ -# -# This file and its contents are supplied under the terms of the -# Common Development and Distribution License ("CDDL"), version 1.0. -# You may only use this file in accordance with the terms of version -# 1.0 of the CDDL. -# -# A full copy of the text of the CDDL should have accompanied this -# source. A copy of the CDDL is also available via the Internet at -# http://www.illumos.org/license/CDDL. -# - -# -# Copyright (c) 2014, 2019 by Delphix. All rights reserved. -# - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/removal - -dist_pkgdata_SCRIPTS = \ - cleanup.ksh removal_all_vdev.ksh removal_cancel.ksh \ - removal_check_space.ksh removal_condense_export.ksh \ - removal_multiple_indirection.ksh \ - removal_nopwrite.ksh removal_remap_deadlists.ksh \ - removal_reservation.ksh removal_resume_export.ksh \ - removal_sanity.ksh removal_with_add.ksh removal_with_create_fs.ksh \ - removal_with_dedup.ksh removal_with_errors.ksh \ - removal_with_export.ksh removal_with_faulted.ksh \ - removal_with_ganging.ksh \ - removal_with_remove.ksh removal_with_scrub.ksh \ - removal_with_send.ksh removal_with_send_recv.ksh \ - removal_with_snapshot.ksh removal_with_write.ksh \ - removal_with_zdb.ksh remove_mirror.ksh remove_mirror_sanity.ksh \ - remove_raidz.ksh remove_expanded.ksh remove_indirect.ksh \ - remove_attach_mirror.ksh - -dist_pkgdata_DATA = \ - removal.kshlib - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/removal diff --git a/tests/zfs-tests/tests/functional/removal/removal.kshlib b/tests/zfs-tests/tests/functional/removal/removal.kshlib index 5752575a8bb1..664cafd9f39e 100644 --- a/tests/zfs-tests/tests/functional/removal/removal.kshlib +++ b/tests/zfs-tests/tests/functional/removal/removal.kshlib @@ -88,13 +88,6 @@ function attempt_during_removal # pool disk callback [args] return 0 } -function indirect_vdev_mapping_size # pool -{ - typeset pool=$1 - zdb -P $pool | grep 'indirect vdev' | \ - sed -E 's/.*\(([0-9]+) in memory\).*/\1/g' -} - function random_write # file write_size { typeset file=$1 diff --git a/tests/zfs-tests/tests/functional/removal/removal_with_errors.ksh b/tests/zfs-tests/tests/functional/removal/removal_with_errors.ksh index 9d5143ef8b17..be7364eb2b0d 100755 --- a/tests/zfs-tests/tests/functional/removal/removal_with_errors.ksh +++ b/tests/zfs-tests/tests/functional/removal/removal_with_errors.ksh @@ -78,6 +78,7 @@ function wait_for_removing_cancel default_setup_noexit "mirror $DISK0 $DISK1 mirror $DISK2 $DISK3" log_onexit cleanup +log_must zfs set compression=off $TESTPOOL FILE_CONTENTS="Leeloo Dallas mul-ti-pass." diff --git a/tests/zfs-tests/tests/functional/removal/removal_with_export.ksh b/tests/zfs-tests/tests/functional/removal/removal_with_export.ksh index f76f76d34f5b..098a52cb9f9a 100755 --- a/tests/zfs-tests/tests/functional/removal/removal_with_export.ksh +++ b/tests/zfs-tests/tests/functional/removal/removal_with_export.ksh @@ -22,6 +22,7 @@ . $STF_SUITE/tests/functional/removal/removal.kshlib default_setup_noexit "$DISKS" +log_must zfs set compression=off $TESTPOOL log_onexit default_cleanup_noexit function callback diff --git a/tests/zfs-tests/tests/functional/removal/removal_with_send.ksh b/tests/zfs-tests/tests/functional/removal/removal_with_send.ksh index a08247838105..735578104740 100755 --- a/tests/zfs-tests/tests/functional/removal/removal_with_send.ksh +++ b/tests/zfs-tests/tests/functional/removal/removal_with_send.ksh @@ -28,7 +28,7 @@ function callback { create_snapshot $TESTPOOL/$TESTFS $TESTSNAP log_must ksh -c \ - "zfs send $TESTPOOL/$TESTFS@$TESTSNAP >$TEST_BASE_DIR/devnull" + "zfs send $TESTPOOL/$TESTFS@$TESTSNAP > /dev/null" return 0 } diff --git a/tests/zfs-tests/tests/functional/removal/remove_attach_mirror.ksh b/tests/zfs-tests/tests/functional/removal/remove_attach_mirror.ksh index 9bbb07cd9419..cdbd962025cf 100755 --- a/tests/zfs-tests/tests/functional/removal/remove_attach_mirror.ksh +++ b/tests/zfs-tests/tests/functional/removal/remove_attach_mirror.ksh @@ -32,6 +32,8 @@ # 4. Reattach it to make a mirror # +command -v fio > /dev/null || log_unsupported "fio missing" + TMPDIR=${TMPDIR:-$TEST_BASE_DIR} DISK1="$TMPDIR/dsk1" diff --git a/tests/zfs-tests/tests/functional/removal/remove_mirror_sanity.ksh b/tests/zfs-tests/tests/functional/removal/remove_mirror_sanity.ksh index 4473771521ba..21af3965ae79 100755 --- a/tests/zfs-tests/tests/functional/removal/remove_mirror_sanity.ksh +++ b/tests/zfs-tests/tests/functional/removal/remove_mirror_sanity.ksh @@ -21,9 +21,7 @@ . $STF_SUITE/include/libtest.shlib . $STF_SUITE/tests/functional/removal/removal.kshlib -DISK1=$(echo $DISKS | awk '{print $1}') -DISK2=$(echo $DISKS | awk '{print $2}') -DISK3=$(echo $DISKS | awk '{print $3}') +read -r DISK1 DISK2 DISK3 _ <<<"$DISKS" DISKS="$DISK1 $DISK2 $DISK3" log_must default_setup_noexit "$DISK1 mirror $DISK2 $DISK3" diff --git a/tests/zfs-tests/tests/functional/rename_dirs/Makefile.am b/tests/zfs-tests/tests/functional/rename_dirs/Makefile.am deleted file mode 100644 index 029daf1f564b..000000000000 --- a/tests/zfs-tests/tests/functional/rename_dirs/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/rename_dirs -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - rename_dirs_001_pos.ksh diff --git a/tests/zfs-tests/tests/functional/rename_dirs/cleanup.ksh b/tests/zfs-tests/tests/functional/rename_dirs/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/rename_dirs/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/rename_dirs/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rename_dirs/rename_dirs_001_pos.ksh b/tests/zfs-tests/tests/functional/rename_dirs/rename_dirs_001_pos.ksh index d7b6de1a2e01..10e4be0e2581 100755 --- a/tests/zfs-tests/tests/functional/rename_dirs/rename_dirs_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/rename_dirs/rename_dirs_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -61,10 +61,7 @@ mkdir -p 1/2/3/4/5 a/b/c/d/e rename_dir & sleep 10 -typeset -i retval=1 -pgrep -x rename_dir >/dev/null 2>&1 -retval=$? -if (( $retval == 0 )); then +if pgrep -x rename_dir >/dev/null 2>&1; then pkill -9 -x rename_dir >/dev/null 2>&1 fi diff --git a/tests/zfs-tests/tests/functional/rename_dirs/setup.ksh b/tests/zfs-tests/tests/functional/rename_dirs/setup.ksh index fc5cec3063a6..b756d4e76c83 100755 --- a/tests/zfs-tests/tests/functional/rename_dirs/setup.ksh +++ b/tests/zfs-tests/tests/functional/rename_dirs/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/replacement/Makefile.am b/tests/zfs-tests/tests/functional/replacement/Makefile.am deleted file mode 100644 index fe6e4912198d..000000000000 --- a/tests/zfs-tests/tests/functional/replacement/Makefile.am +++ /dev/null @@ -1,21 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/replacement -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - attach_import.ksh \ - attach_multiple.ksh \ - attach_rebuild.ksh \ - attach_resilver.ksh \ - detach.ksh \ - rebuild_disabled_feature.ksh \ - rebuild_multiple.ksh \ - rebuild_raidz.ksh \ - replace_import.ksh \ - replace_rebuild.ksh \ - replace_resilver.ksh \ - resilver_restart_001.ksh \ - resilver_restart_002.ksh \ - scrub_cancel.ksh - -dist_pkgdata_DATA = \ - replacement.cfg diff --git a/tests/zfs-tests/tests/functional/replacement/attach_rebuild.ksh b/tests/zfs-tests/tests/functional/replacement/attach_rebuild.ksh index 998d3eec7c71..8b4cb9da2d23 100755 --- a/tests/zfs-tests/tests/functional/replacement/attach_rebuild.ksh +++ b/tests/zfs-tests/tests/functional/replacement/attach_rebuild.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -142,10 +142,7 @@ for op in "" "-f"; do attach_test "$opt" $TESTDIR/$TESTFILE1.1 $TESTDIR/$REPLACEFILE - zpool iostat -v $TESTPOOL1 | grep "$REPLACEFILE" - if [[ $? -ne 0 ]]; then - log_fail "$REPLACEFILE is not present." - fi + log_must eval "zpool iostat -v $TESTPOOL1 | grep \"$REPLACEFILE\"" destroy_pool $TESTPOOL1 done @@ -161,10 +158,7 @@ for type in "" "raidz" "raidz1" "draid" "draid1"; do log_mustnot zpool attach -s "$opt" $TESTDIR/$TESTFILE1.1 \ $TESTDIR/$REPLACEFILE - zpool iostat -v $TESTPOOL1 | grep "$REPLACEFILE" - if [[ $? -eq 0 ]]; then - log_fail "$REPLACEFILE should not be present." - fi + log_mustnot eval "zpool iostat -v $TESTPOOL1 | grep \"$REPLACEFILE\"" destroy_pool $TESTPOOL1 done diff --git a/tests/zfs-tests/tests/functional/replacement/attach_resilver.ksh b/tests/zfs-tests/tests/functional/replacement/attach_resilver.ksh index e99d681bb21d..724a7865fe69 100755 --- a/tests/zfs-tests/tests/functional/replacement/attach_resilver.ksh +++ b/tests/zfs-tests/tests/functional/replacement/attach_resilver.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -141,10 +141,7 @@ for op in "" "-f"; do attach_test "$opt" $TESTDIR/$TESTFILE1.1 $TESTDIR/$REPLACEFILE - zpool iostat -v $TESTPOOL1 | grep "$REPLACEFILE" - if [[ $? -ne 0 ]]; then - log_fail "$REPLACEFILE is not present." - fi + log_must eval "zpool iostat -v $TESTPOOL1 | grep \"$REPLACEFILE\"" destroy_pool $TESTPOOL1 done @@ -160,10 +157,7 @@ for type in "" "raidz" "raidz1" "draid"; do log_mustnot zpool attach "$opt" $TESTDIR/$TESTFILE1.1 \ $TESTDIR/$REPLACEFILE - zpool iostat -v $TESTPOOL1 | grep "$REPLACEFILE" - if [[ $? -eq 0 ]]; then - log_fail "$REPLACEFILE should not be present." - fi + log_mustnot eval "zpool iostat -v $TESTPOOL1 | grep \"$REPLACEFILE\"" destroy_pool $TESTPOOL1 done diff --git a/tests/zfs-tests/tests/functional/replacement/cleanup.ksh b/tests/zfs-tests/tests/functional/replacement/cleanup.ksh index b81a372638e3..a0555aff06c2 100755 --- a/tests/zfs-tests/tests/functional/replacement/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/replacement/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/replacement/detach.ksh b/tests/zfs-tests/tests/functional/replacement/detach.ksh index f049c639d8a6..c563a015ad9a 100755 --- a/tests/zfs-tests/tests/functional/replacement/detach.ksh +++ b/tests/zfs-tests/tests/functional/replacement/detach.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -134,10 +134,7 @@ log_must zfs set mountpoint=$TESTDIR1 $TESTPOOL1/$TESTFS1 detach_test $TESTDIR/$TESTFILE1.1 -zpool iostat -v $TESTPOOL1 | grep "$TESTFILE1.1" -if [[ $? -eq 0 ]]; then - log_fail "$TESTFILE1.1 should no longer be present." -fi +log_mustnot eval "zpool iostat -v $TESTPOOL1 | grep \"$TESTFILE1.1\"" destroy_pool $TESTPOOL1 @@ -150,10 +147,7 @@ for type in "" "raidz" "raidz1" "draid"; do log_mustnot zpool detach $TESTDIR/$TESTFILE1.1 - zpool iostat -v $TESTPOOL1 | grep "$TESTFILE1.1" - if [[ $? -ne 0 ]]; then - log_fail "$TESTFILE1.1 is not present." - fi + log_must eval "zpool iostat -v $TESTPOOL1 | grep \"$TESTFILE1.1\"" destroy_pool $TESTPOOL1 done diff --git a/tests/zfs-tests/tests/functional/replacement/rebuild_disabled_feature.ksh b/tests/zfs-tests/tests/functional/replacement/rebuild_disabled_feature.ksh index d17d83b78333..5e86a8ccb8ac 100755 --- a/tests/zfs-tests/tests/functional/replacement/rebuild_disabled_feature.ksh +++ b/tests/zfs-tests/tests/functional/replacement/rebuild_disabled_feature.ksh @@ -45,8 +45,7 @@ function check_feature_flag pool=$2 expected_value=$3 - value="$(zpool get -H -o property,value all $pool | \ - egrep "$feature" | awk '{print $2}')" + value="$(zpool get -H -o property,value all $pool | awk -v f="$feature" '$0 ~ f {print $2}')" if [ "$value" = "$expected_value" ]; then log_note "$feature verified to be $value" else diff --git a/tests/zfs-tests/tests/functional/replacement/replace_rebuild.ksh b/tests/zfs-tests/tests/functional/replacement/replace_rebuild.ksh index b3c7995fd62a..ea1fabdbbeb9 100755 --- a/tests/zfs-tests/tests/functional/replacement/replace_rebuild.ksh +++ b/tests/zfs-tests/tests/functional/replacement/replace_rebuild.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -145,10 +145,7 @@ for type in "" "mirror" "draid"; do replace_test "$opt" $TESTDIR/$TESTFILE1.1 $TESTDIR/$REPLACEFILE - zpool iostat -v $TESTPOOL1 | grep "$REPLACEFILE" - if [[ $? -ne 0 ]]; then - log_fail "$REPLACEFILE is not present." - fi + log_must eval "zpool iostat -v $TESTPOOL1 | grep \"$REPLACEFILE\"" destroy_pool $TESTPOOL1 log_must rm -rf /$TESTPOOL1 diff --git a/tests/zfs-tests/tests/functional/replacement/replace_resilver.ksh b/tests/zfs-tests/tests/functional/replacement/replace_resilver.ksh index 2585397bba88..bdca3c772584 100755 --- a/tests/zfs-tests/tests/functional/replacement/replace_resilver.ksh +++ b/tests/zfs-tests/tests/functional/replacement/replace_resilver.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -142,10 +142,7 @@ for type in "" "raidz" "mirror" "draid"; do replace_test "$opt" $TESTDIR/$TESTFILE1.1 $TESTDIR/$REPLACEFILE - zpool iostat -v $TESTPOOL1 | grep "$REPLACEFILE" - if [[ $? -ne 0 ]]; then - log_fail "$REPLACEFILE is not present." - fi + log_must eval "zpool iostat -v $TESTPOOL1 | grep \"$REPLACEFILE\"" destroy_pool $TESTPOOL1 log_must rm -rf /$TESTPOOL1 diff --git a/tests/zfs-tests/tests/functional/replacement/replacement.cfg b/tests/zfs-tests/tests/functional/replacement/replacement.cfg index 271317b1c970..9e82a4ccd502 100644 --- a/tests/zfs-tests/tests/functional/replacement/replacement.cfg +++ b/tests/zfs-tests/tests/functional/replacement/replacement.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/replacement/resilver_restart_001.ksh b/tests/zfs-tests/tests/functional/replacement/resilver_restart_001.ksh index 269d31bf8905..b498ba4af730 100755 --- a/tests/zfs-tests/tests/functional/replacement/resilver_restart_001.ksh +++ b/tests/zfs-tests/tests/functional/replacement/resilver_restart_001.ksh @@ -162,7 +162,7 @@ do # inject read io errors on vdev and verify resilver does not restart log_must zinject -a -d ${VDEV_FILES[2]} -e io -T read -f 0.25 $TESTPOOL1 - log_must cat ${DATAPATHS[1]} > /dev/null + log_must cp ${DATAPATHS[1]} /dev/null log_must zinject -c all # there should still be 2 resilver starts w/o defer, 1 with defer diff --git a/tests/zfs-tests/tests/functional/replacement/resilver_restart_002.ksh b/tests/zfs-tests/tests/functional/replacement/resilver_restart_002.ksh index 4f2707693d13..bc5bc017768c 100755 --- a/tests/zfs-tests/tests/functional/replacement/resilver_restart_002.ksh +++ b/tests/zfs-tests/tests/functional/replacement/resilver_restart_002.ksh @@ -57,14 +57,14 @@ log_must set_tunable32 SCAN_LEGACY 1 # create the pool and a 32M file (32k blocks) log_must truncate -s $VDEV_FILE_SIZE ${VDEV_FILES[0]} $SPARE_VDEV_FILE log_must zpool create -f -O recordsize=1k $TESTPOOL1 ${VDEV_FILES[0]} -log_must dd if=/dev/urandom of=/$TESTPOOL1/file bs=1M count=32 > /dev/null 2>&1 +log_must eval "dd if=/dev/urandom of=/$TESTPOOL1/file bs=1M count=32 2>/dev/null" # determine objset/object objset=$(zdb -d $TESTPOOL1/ | sed -ne 's/.*ID \([0-9]*\).*/\1/p') object=$(ls -i /$TESTPOOL1/file | awk '{print $1}') # inject event to cause error during resilver -log_must zinject -b `printf "%x:%x:0:3fff" $objset $object` $TESTPOOL1 +log_must zinject -b $(printf "%x:%x:0:3fff" $objset $object) $TESTPOOL1 # clear events and start resilver log_must zpool events -c @@ -74,7 +74,7 @@ log_note "waiting for read errors to start showing up" for iter in {0..59} do sync_pool $TESTPOOL1 - err=$(zpool status $TESTPOOL1 | grep ${VDEV_FILES[0]} | awk '{print $3}') + err=$(zpool status $TESTPOOL1 | awk -v dev=${VDEV_FILES[0]} '$0 ~ dev {print $3}') (( $err > 0 )) && break sleep 1 done @@ -84,7 +84,7 @@ done log_note "waiting for resilver to finish" for iter in {0..59} do - finish=$(zpool events | grep "sysevent.fs.zfs.resilver_finish" | wc -l) + finish=$(zpool events | grep -cF "sysevent.fs.zfs.resilver_finish") (( $finish > 0 )) && break sleep 1 done @@ -96,7 +96,7 @@ sync_pool $TESTPOOL1 sync_pool $TESTPOOL1 # check if resilver was restarted -start=$(zpool events | grep "sysevent.fs.zfs.resilver_start" | wc -l) +start=$(zpool events | grep -cF "sysevent.fs.zfs.resilver_start") (( $start != 1 )) && log_fail "resilver restarted unnecessarily" log_pass "Resilver did not restart unnecessarily from scan errors" diff --git a/tests/zfs-tests/tests/functional/replacement/setup.ksh b/tests/zfs-tests/tests/functional/replacement/setup.ksh index 4132392d80a9..0327c20a73cc 100755 --- a/tests/zfs-tests/tests/functional/replacement/setup.ksh +++ b/tests/zfs-tests/tests/functional/replacement/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -43,5 +43,3 @@ case $index in default_raidz_setup $DISKS ;; esac - -log_pass diff --git a/tests/zfs-tests/tests/functional/reservation/Makefile.am b/tests/zfs-tests/tests/functional/reservation/Makefile.am deleted file mode 100644 index 8eaf09861551..000000000000 --- a/tests/zfs-tests/tests/functional/reservation/Makefile.am +++ /dev/null @@ -1,30 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/reservation -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - reservation_001_pos.ksh \ - reservation_002_pos.ksh \ - reservation_003_pos.ksh \ - reservation_004_pos.ksh \ - reservation_005_pos.ksh \ - reservation_006_pos.ksh \ - reservation_007_pos.ksh \ - reservation_008_pos.ksh \ - reservation_009_pos.ksh \ - reservation_010_pos.ksh \ - reservation_011_pos.ksh \ - reservation_012_pos.ksh \ - reservation_013_pos.ksh \ - reservation_014_pos.ksh \ - reservation_015_pos.ksh \ - reservation_016_pos.ksh \ - reservation_017_pos.ksh \ - reservation_018_pos.ksh \ - reservation_019_pos.ksh \ - reservation_020_pos.ksh \ - reservation_021_neg.ksh \ - reservation_022_pos.ksh - -dist_pkgdata_DATA = \ - reservation.cfg \ - reservation.shlib diff --git a/tests/zfs-tests/tests/functional/reservation/cleanup.ksh b/tests/zfs-tests/tests/functional/reservation/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/reservation/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/reservation/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation.cfg b/tests/zfs-tests/tests/functional/reservation/reservation.cfg index b4f9af938623..18d0ead30d64 100644 --- a/tests/zfs-tests/tests/functional/reservation/reservation.cfg +++ b/tests/zfs-tests/tests/functional/reservation/reservation.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation.shlib b/tests/zfs-tests/tests/functional/reservation/reservation.shlib index 47bd70f7cbcc..911c3a908a17 100644 --- a/tests/zfs-tests/tests/functional/reservation/reservation.shlib +++ b/tests/zfs-tests/tests/functional/reservation/reservation.shlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -38,25 +38,15 @@ # function zero_reservation { - typeset resv_val dataset=$1 log_must zfs set reservation=none $dataset - resv_val=`zfs get -H reservation $dataset | awk '{print $3}'` - if [[ $? -ne 0 ]]; then - log_fail "Unable to get reservation prop on $dataset" - elif [[ $resv_val != "none" ]]; then - log_fail "Reservation not 'none' ($resv_val) as expected" - fi - + log_must eval 'resv_val="$(zfs get -Ho value reservation $dataset)"' + log_must [ $resv_val = "none" ] - resv_val=`zfs get -pH reservation $dataset | awk '{print $3}'` - if [[ $? -ne 0 ]]; then - log_fail "Unable to get reservation prop on $dataset" - elif [[ $resv_val -ne 0 ]]; then - log_fail "Reservation not 0 ($resv_val) as expected" - fi + log_must eval 'resv_val="$(zfs get -pHo value reservation $dataset)"' + log_must [ $resv_val -eq 0 ] return 0 } diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_001_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_001_pos.ksh index b8220791f1d4..dbe4890be86c 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -60,10 +60,10 @@ function cleanup log_onexit cleanup -log_assert "Verify that to set a reservation on a filesystem or volume must " \ +log_assert "Verify that to set a reservation on a filesystem or volume must" \ "use value smaller than space available property of pool" -space_avail=`get_prop available $TESTPOOL` +space_avail=$(get_prop available $TESTPOOL) if ! is_global_zone ; then OBJ_LIST="" @@ -103,7 +103,7 @@ for obj in $TESTPOOL/$TESTFS $OBJ_LIST; do log_must zfs set reservation=$resv_size_set $obj - resv_size_get=`get_prop reservation $obj` + resv_size_get=$(get_prop reservation $obj) if [[ $resv_size_set != $resv_size_get ]]; then log_fail "Reservation not the expected value " \ "($resv_size_set != $resv_size_get)" @@ -111,7 +111,7 @@ for obj in $TESTPOOL/$TESTFS $OBJ_LIST; do log_must zero_reservation $obj - new_space_avail=`get_prop available $obj` + new_space_avail=$(get_prop available $obj) # # Due to the way space is consumed and released by metadata we diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_002_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_002_pos.ksh index e0fed6389c35..eca3f5936580 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_003_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_003_pos.ksh index ee303b53bedd..a6e701dcf381 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_004_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_004_pos.ksh index eb606a762432..2b52cd6a07d9 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_005_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_005_pos.ksh index 535d652dafab..e6cfa88990ac 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_006_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_006_pos.ksh index da0d36a35d3c..ef7b554de1ef 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_007_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_007_pos.ksh index a1fffd362427..d3db1c7cfbeb 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_008_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_008_pos.ksh index cfc30f47421b..2c7c0c04014b 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_009_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_009_pos.ksh index a639abf8960a..7fd19ef50b12 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_009_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_009_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_010_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_010_pos.ksh index f3a64a0bea8c..c575dc7c3156 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_010_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_010_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_011_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_011_pos.ksh index a6abe9db6a13..d3fcc24bc49d 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_011_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_011_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_012_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_012_pos.ksh index 5ab0952b1910..77bf8beb0b92 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_012_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_012_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_013_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_013_pos.ksh index bf0955223490..7ef2b61e3c8b 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_013_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_013_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -72,8 +72,6 @@ log_must zfs create $TESTPOOL/$TESTFS1 log_must zfs create $TESTPOOL/$TESTFS1/$TESTFS2 space_avail=$(get_prop available $TESTPOOL) -[[ $? -ne 0 ]] && \ - log_fail "Unable to get space available property for $TESTPOOL" typeset -il resv_set=space_avail/5 resv_set=$(floor_volsize $resv_set) diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_014_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_014_pos.ksh index 3b7f384da365..eeefe182798a 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_014_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_014_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_015_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_015_pos.ksh index 7067a7810590..2b9d3f98694b 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_015_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_015_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_016_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_016_pos.ksh index 82bbcde4a3b7..84a80e34323e 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_016_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_016_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_017_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_017_pos.ksh index bb3af67d24d3..486ad397ca13 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_017_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_017_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_018_pos.ksh b/tests/zfs-tests/tests/functional/reservation/reservation_018_pos.ksh index 1f92c8898562..717c8371ff19 100755 --- a/tests/zfs-tests/tests/functional/reservation/reservation_018_pos.ksh +++ b/tests/zfs-tests/tests/functional/reservation/reservation_018_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/reservation/setup.ksh b/tests/zfs-tests/tests/functional/reservation/setup.ksh index c39034783555..6b93b4a7e310 100755 --- a/tests/zfs-tests/tests/functional/reservation/setup.ksh +++ b/tests/zfs-tests/tests/functional/reservation/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -32,4 +32,5 @@ . $STF_SUITE/include/libtest.shlib default_setup_noexit ${DISKS%% *} +log_must zfs set compression=off $TESTPOOL log_pass diff --git a/tests/zfs-tests/tests/functional/rootpool/Makefile.am b/tests/zfs-tests/tests/functional/rootpool/Makefile.am deleted file mode 100644 index ca5ad14fdbb1..000000000000 --- a/tests/zfs-tests/tests/functional/rootpool/Makefile.am +++ /dev/null @@ -1,7 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/rootpool -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - rootpool_002_neg.ksh \ - rootpool_003_neg.ksh \ - rootpool_007_pos.ksh diff --git a/tests/zfs-tests/tests/functional/rootpool/cleanup.ksh b/tests/zfs-tests/tests/functional/rootpool/cleanup.ksh index 8f043259fb91..c51a3f27bddc 100755 --- a/tests/zfs-tests/tests/functional/rootpool/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/rootpool/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rootpool/rootpool_002_neg.ksh b/tests/zfs-tests/tests/functional/rootpool/rootpool_002_neg.ksh index 32bacf753a0e..c6c3bf8a52b1 100755 --- a/tests/zfs-tests/tests/functional/rootpool/rootpool_002_neg.ksh +++ b/tests/zfs-tests/tests/functional/rootpool/rootpool_002_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -54,7 +54,7 @@ typeset tmpfile="$TEST_BASE_DIR/mounted-datasets.$$" # damage done by the attempted pool destroy. The destroy itself should fail, # but some filesystems can become unmounted in the process, and aren't # automatically remounted. -mount -p | awk '{if ($4 == "zfs") print $1}' > $tmpfile +mount -p | awk '$4 == "zfs" {print $1}' > $tmpfile log_mustnot zpool destroy $rootpool diff --git a/tests/zfs-tests/tests/functional/rootpool/rootpool_003_neg.ksh b/tests/zfs-tests/tests/functional/rootpool/rootpool_003_neg.ksh index 3bb5d12edddf..6dbbc7da39ef 100755 --- a/tests/zfs-tests/tests/functional/rootpool/rootpool_003_neg.ksh +++ b/tests/zfs-tests/tests/functional/rootpool/rootpool_003_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rootpool/rootpool_007_pos.ksh b/tests/zfs-tests/tests/functional/rootpool/rootpool_007_pos.ksh index e4d4268cc08e..a336e6bbb33f 100755 --- a/tests/zfs-tests/tests/functional/rootpool/rootpool_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/rootpool/rootpool_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rootpool/setup.ksh b/tests/zfs-tests/tests/functional/rootpool/setup.ksh index 8d8097108190..049bdf066aeb 100755 --- a/tests/zfs-tests/tests/functional/rootpool/setup.ksh +++ b/tests/zfs-tests/tests/functional/rootpool/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -35,7 +35,7 @@ verify_runnable "global" # # This functionality is supported under Linux, but these test cases -# are disabled by default since they manipulate the systems root pool. +# are disabled by default since they manipulate the system's root pool. # if is_linux || is_freebsd; then log_unsupported "Supported but disabled by default" diff --git a/tests/zfs-tests/tests/functional/rsend/Makefile.am b/tests/zfs-tests/tests/functional/rsend/Makefile.am deleted file mode 100644 index ba1bcf24a501..000000000000 --- a/tests/zfs-tests/tests/functional/rsend/Makefile.am +++ /dev/null @@ -1,67 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/rsend -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - recv_dedup.ksh \ - recv_dedup_encrypted_zvol.ksh \ - rsend_001_pos.ksh \ - rsend_002_pos.ksh \ - rsend_003_pos.ksh \ - rsend_004_pos.ksh \ - rsend_005_pos.ksh \ - rsend_006_pos.ksh \ - rsend_007_pos.ksh \ - rsend_008_pos.ksh \ - rsend_009_pos.ksh \ - rsend_010_pos.ksh \ - rsend_011_pos.ksh \ - rsend_012_pos.ksh \ - rsend_013_pos.ksh \ - rsend_014_pos.ksh \ - rsend_016_neg.ksh \ - rsend_019_pos.ksh \ - rsend_020_pos.ksh \ - rsend_021_pos.ksh \ - rsend_022_pos.ksh \ - rsend_024_pos.ksh \ - send_encrypted_files.ksh \ - send_encrypted_hierarchy.ksh \ - send_encrypted_props.ksh \ - send_encrypted_truncated_files.ksh \ - send-c_embedded_blocks.ksh \ - send-c_incremental.ksh \ - send-c_lz4_disabled.ksh \ - send-c_mixed_compression.ksh \ - send-c_props.ksh \ - send-c_recv_dedup.ksh \ - send-c_recv_lz4_disabled.ksh \ - send-c_resume.ksh \ - send-c_stream_size_estimate.ksh \ - send-c_verify_contents.ksh \ - send-c_verify_ratio.ksh \ - send-c_volume.ksh \ - send-c_zstreamdump.ksh \ - send-cpL_varied_recsize.ksh \ - send-L_toggle.ksh \ - send_freeobjects.ksh \ - send_partial_dataset.ksh \ - send_realloc_dnode_size.ksh \ - send_realloc_files.ksh \ - send_realloc_encrypted_files.ksh \ - send_spill_block.ksh \ - send_raw_spill_block.ksh \ - send_holds.ksh \ - send_hole_birth.ksh \ - send_invalid.ksh \ - send_mixed_raw.ksh \ - send-wR_encrypted_zvol.ksh \ - send_doall.ksh - -dist_pkgdata_DATA = \ - dedup.zsend.bz2 \ - dedup_encrypted_zvol.bz2 \ - dedup_encrypted_zvol.zsend.bz2 \ - fs.tar.gz \ - rsend.cfg \ - rsend.kshlib - diff --git a/tests/zfs-tests/tests/functional/rsend/cleanup.ksh b/tests/zfs-tests/tests/functional/rsend/cleanup.ksh index 063fe9dd1f61..9a6ad98b3fde 100755 --- a/tests/zfs-tests/tests/functional/rsend/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/rsend/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rsend/recv_dedup.ksh b/tests/zfs-tests/tests/functional/rsend/recv_dedup.ksh index e6e282a1c6f8..ba23c4f6aa3c 100755 --- a/tests/zfs-tests/tests/functional/rsend/recv_dedup.ksh +++ b/tests/zfs-tests/tests/functional/rsend/recv_dedup.ksh @@ -48,6 +48,7 @@ log_must eval "zstream redup $sendfile | zfs recv -d $TESTPOOL/recv" log_must mkdir /$TESTPOOL/tar log_must tar --directory /$TESTPOOL/tar -xzf $tarfile -log_must diff -r /$TESTPOOL/tar /$TESTPOOL/recv +# The recv'd filesystem is called "/fs", so only compare that subdirectory. +log_must directory_diff /$TESTPOOL/tar/fs /$TESTPOOL/recv/fs log_pass "zfs can receive dedup send streams with 'zstream redup'" diff --git a/tests/zfs-tests/tests/functional/rsend/rsend.cfg b/tests/zfs-tests/tests/functional/rsend/rsend.cfg index 8400ecfe35b4..9e3422ecad67 100644 --- a/tests/zfs-tests/tests/functional/rsend/rsend.cfg +++ b/tests/zfs-tests/tests/functional/rsend/rsend.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -29,9 +29,8 @@ export BACKDIR=${TEST_BASE_DIR%%/}/backdir-rsend -export DISK1=$(echo $DISKS | awk '{print $1}') -export DISK2=$(echo $DISKS | awk '{print $2}') -export DISK3=$(echo $DISKS | awk '{print $3}') +read -r DISK1 DISK2 DISK3 _ <<<"$DISKS" +export DISK1 DISK2 DISK3 export POOL=$TESTPOOL export POOL2=$TESTPOOL2 diff --git a/tests/zfs-tests/tests/functional/rsend/rsend.kshlib b/tests/zfs-tests/tests/functional/rsend/rsend.kshlib index 516d41263294..26e7c2cc25bc 100644 --- a/tests/zfs-tests/tests/functional/rsend/rsend.kshlib +++ b/tests/zfs-tests/tests/functional/rsend/rsend.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -128,7 +128,7 @@ function cleanup_pool # # https://github.com/openzfs/zfs/issues/6143 # - log_must df >/dev/null + log_must eval "df >/dev/null" log_must_busy zfs destroy -Rf $pool else typeset list=$(zfs list -H -r -t all -o name $pool) @@ -153,23 +153,13 @@ function cleanup_pool if [[ -d $mntpnt ]]; then rm -rf $mntpnt/* fi - - return 0 -} - -function cleanup_pools -{ - cleanup_pool $POOL2 - destroy_pool $POOL3 } function cmp_md5s { typeset file1=$1 typeset file2=$2 - typeset sum1=$(md5digest $file1) - typeset sum2=$(md5digest $file2) - test "$sum1" = "$sum2" + [ "$(md5digest $file1)" = "$(md5digest $file2)" ] } # @@ -183,18 +173,9 @@ function cmp_ds_subs typeset src_fs=$1 typeset dst_fs=$2 - zfs list -r -H -t all -o name $src_fs > $BACKDIR/src1 - zfs list -r -H -t all -o name $dst_fs > $BACKDIR/dst1 - - eval sed -e 's:^$src_fs:PREFIX:g' < $BACKDIR/src1 > $BACKDIR/src - eval sed -e 's:^$dst_fs:PREFIX:g' < $BACKDIR/dst1 > $BACKDIR/dst - - diff $BACKDIR/src $BACKDIR/dst - typeset -i ret=$? - - rm -f $BACKDIR/src $BACKDIR/dst $BACKDIR/src1 $BACKDIR/dst1 - - return $ret + diff \ + <(zfs list -rHt all -o name $src_fs | sed "s:^$src_fs:PREFIX:g") \ + <(zfs list -rHt all -o name $dst_fs | sed "s:^$dst_fs:PREFIX:g") } # @@ -212,8 +193,7 @@ function cmp_ds_cont srcdir=$(get_prop mountpoint $src_fs) dstdir=$(get_prop mountpoint $dst_fs) - diff -r $srcdir $dstdir > /dev/null 2>&1 - return $? + replay_directory_diff $srcdir $dstdir } # @@ -221,39 +201,30 @@ function cmp_ds_cont # # $1 dataset 1 # $2 dataset 2 +# $3 -n == don't track property source +# $4 -n == don't track the origin property # function cmp_ds_prop { typeset dtst1=$1 typeset dtst2=$2 - typeset -a props=("type" "origin" "volblocksize" "acltype" "dnodesize" \ - "atime" "canmount" "checksum" "compression" "copies" "devices" \ - "exec" "quota" "readonly" "recordsize" "reservation" "setuid" \ - "snapdir" "version" "volsize" "xattr" "mountpoint"); + typeset nosource=$3 + typeset noorigin=$4 + typeset source=",source"; [ -n "$nosource" ] && source= + typeset origin=",origin"; [ -n "$noorigin" ] && origin= + typeset props="type$origin,volblocksize,acltype,dnodesize" + props+=",atime,canmount,checksum,compression,copies,devices" + props+=",exec,quota,readonly,recordsize,reservation,setuid" + props+=",snapdir,version,volsize,xattr,mountpoint" if is_freebsd; then - props+=("jailed") + props+=",jailed" else - props+=("zoned") + props+=",zoned" fi - for prop in $props; - do - zfs get -H -o property,value,source $prop $dtst1 >> \ - $BACKDIR/dtst1 - zfs get -H -o property,value,source $prop $dtst2 >> \ - $BACKDIR/dtst2 - done - - eval sed -e 's:$dtst1:PREFIX:g' < $BACKDIR/dtst1 > $BACKDIR/dtst1 - eval sed -e 's:$dtst2:PREFIX:g' < $BACKDIR/dtst2 > $BACKDIR/dtst2 - - diff $BACKDIR/dtst1 $BACKDIR/dtst2 - typeset -i ret=$? - - rm -f $BACKDIR/dtst1 $BACKDIR/dtst2 - - return $ret - + diff \ + <(zfs get -Ho property,value$source $props $dtst1 | sed -e "s:$dtst1:PREFIX:g" -e 's/^origin [^@]*/origin POOL/' -e 's/ inherited from [^/]*/ inherited from POOL/') \ + <(zfs get -Ho property,value$source $props $dtst2 | sed -e "s:$dtst2:PREFIX:g" -e 's/^origin [^@]*/origin POOL/' -e 's/ inherited from [^/]*/ inherited from POOL/') } # @@ -295,7 +266,6 @@ function snapshot_tree typeset -i ret=0 if [[ $type == "filesystem" ]]; then typeset mntpnt=$(get_prop mountpoint $ds) - ((ret |= $?)) if ((ret == 0)) ; then eval random_tree $mntpnt/${snap##$ds} @@ -347,9 +317,7 @@ function getds_with_suffix typeset ds=$1 typeset suffix=$2 - typeset list=$(zfs list -r -H -t all -o name $ds | grep "$suffix$") - - echo $list + zfs list -rHt all -o name $ds | grep "$suffix$" } # @@ -366,8 +334,7 @@ function fs_inherit_prop fi else fs_prop=$(zfs inherit 2>&1 | \ - awk '$2=="YES" && $3=="YES" {print $1}'| - egrep -v "devices|mlslabel|sharenfs|sharesmb|zoned") + awk '$2=="YES" && $3=="YES" && !/devices|mlslabel|sharenfs|sharesmb|zoned/ {print $1}') fi echo $fs_prop @@ -378,7 +345,7 @@ function fs_inherit_prop # function vol_inherit_prop { - echo "checksum readonly" + echo checksum readonly } # @@ -583,24 +550,22 @@ function churn_files # function mess_send_file { + typeset -i minsize=2072 file=$1 filesize=$(stat_size $file) + if [ $filesize -lt $minsize ]; then + log_fail "Send file too small: $filesize < $minsize" + fi - offset=$(($RANDOM * $RANDOM % $filesize)) - - # The random offset might truncate the send stream to be - # smaller than the DRR_BEGIN record. If this happens, then - # the receiving system won't have enough info to create the - # partial dataset at all. We use zstream dump to check for - # this and retry in this case. + # Truncate the send stream at a random offset after the DRR_BEGIN + # record (beyond 2072 bytes), any smaller than this and the receiving + # system won't have enough info to create the partial dataset at all. + # We use zstream dump to verify there is an intact DRR_BEGIN record. + offset=$(((($RANDOM * $RANDOM) % ($filesize - $minsize)) + $minsize)) nr_begins=$(head -c $offset $file | zstream dump | \ - grep DRR_BEGIN | awk '{ print $5 }') - while [ "$nr_begins" -eq 0 ]; do - offset=$(($RANDOM * $RANDOM % $filesize)) - nr_begins=$(head -c $offset $file | zstream dump | \ - grep DRR_BEGIN | awk '{ print $5 }') - done + awk '/DRR_BEGIN/ { print $5 }') + log_must [ "$nr_begins" -eq 1 ] if (($RANDOM % 7 <= 1)); then # @@ -627,13 +592,11 @@ function file_check if [[ -d /$recvfs/.zfs/snapshot/a && -d \ /$sendfs/.zfs/snapshot/a ]]; then - diff -r /$recvfs/.zfs/snapshot/a /$sendfs/.zfs/snapshot/a - [[ $? -eq 0 ]] || log_fail "Differences found in snap a" + log_must directory_diff /$recvfs/.zfs/snapshot/a /$sendfs/.zfs/snapshot/a fi if [[ -d /$recvfs/.zfs/snapshot/b && -d \ /$sendfs/.zfs/snapshot/b ]]; then - diff -r /$recvfs/.zfs/snapshot/b /$sendfs/.zfs/snapshot/b - [[ $? -eq 0 ]] || log_fail "Differences found in snap b" + log_must directory_diff /$recvfs/.zfs/snapshot/b /$sendfs/.zfs/snapshot/b fi } @@ -657,20 +620,18 @@ function resume_test for ((i=0; i<2; i=i+1)); do mess_send_file /$streamfs/$stream_num - log_mustnot zfs recv -suv $recvfs /dev/null" log_must eval "zfs send -t $token >/$streamfs/$stream_num" - [[ -f /$streamfs/$stream_num ]] || \ - log_fail "NO FILE /$streamfs/$stream_num" done - log_must zfs recv -suv $recvfs /$streamfs/1" mess_send_file /$streamfs/1 - log_mustnot zfs recv -suv $recvfs < /$streamfs/1 2>&1 - token=$(zfs get -Hp -o value receive_resume_token $recvfs) - echo "$token" > /$streamfs/resume_token - - return 0 + log_mustnot eval "zfs recv -suv $recvfs < /$streamfs/1 2>&1" + get_prop receive_resume_token $recvfs > /$streamfs/resume_token } # @@ -749,7 +707,7 @@ function stream_has_features shift [[ -f $file ]] || log_fail "Couldn't find file: $file" - typeset flags=$(cat $file | zstream dump | \ + typeset flags=$(zstream dump $file | \ awk '/features =/ {features = $3} END {print features}') typeset -A feature feature[dedup]="1" @@ -799,7 +757,7 @@ function verify_stream_size [[ -f $stream ]] || log_fail "No such file: $stream" datasetexists $ds || log_fail "No such dataset: $ds" - typeset stream_size=$(cat $stream | zstream dump | sed -n \ + typeset stream_size=$(zstream dump $stream | sed -n \ 's/ Total payload size = \(.*\) (0x.*)/\1/p') typeset inc_size=0 @@ -817,8 +775,7 @@ function verify_stream_size fi ds_size=$((ds_size - inc_size)) - within_percent $stream_size $ds_size $percent || log_fail \ - "$stream_size $ds_size differed by too much" + log_must within_percent $stream_size $ds_size $percent } # Cleanup function for tests involving resumable send diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_001_pos.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_001_pos.ksh index 3864d8a7d1bd..b73c27656437 100755 --- a/tests/zfs-tests/tests/functional/rsend/rsend_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/rsend/rsend_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_002_pos.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_002_pos.ksh index f2c1e03937e2..111d00eeb63a 100755 --- a/tests/zfs-tests/tests/functional/rsend/rsend_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/rsend/rsend_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_003_pos.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_003_pos.ksh index 2ace6737b14e..fa4479c666d9 100755 --- a/tests/zfs-tests/tests/functional/rsend/rsend_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/rsend/rsend_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_004_pos.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_004_pos.ksh index c0b36b2329e7..522662076d2c 100755 --- a/tests/zfs-tests/tests/functional/rsend/rsend_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/rsend/rsend_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_005_pos.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_005_pos.ksh index 4d156690e38a..2c1ba0c1644a 100755 --- a/tests/zfs-tests/tests/functional/rsend/rsend_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/rsend/rsend_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_006_pos.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_006_pos.ksh index 8c598f4ec29c..5216016e6b71 100755 --- a/tests/zfs-tests/tests/functional/rsend/rsend_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/rsend/rsend_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_007_pos.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_007_pos.ksh index 29e9f1859579..499caff9a71a 100755 --- a/tests/zfs-tests/tests/functional/rsend/rsend_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/rsend/rsend_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_008_pos.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_008_pos.ksh index 8e1821d88a68..fab0a7d9ec7c 100755 --- a/tests/zfs-tests/tests/functional/rsend/rsend_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/rsend/rsend_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_009_pos.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_009_pos.ksh index 7d858d92d511..ba60afe9acd3 100755 --- a/tests/zfs-tests/tests/functional/rsend/rsend_009_pos.ksh +++ b/tests/zfs-tests/tests/functional/rsend/rsend_009_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -60,8 +60,8 @@ log_onexit cleanup log_must mkfile $MINVDEVSIZE $TESTDIR/bfile log_must mkfile $SPA_MINDEVSIZE $TESTDIR/sfile -log_must zpool create bpool $TESTDIR/bfile -log_must zpool create spool $TESTDIR/sfile +log_must zpool create -O compression=off bpool $TESTDIR/bfile +log_must zpool create -O compression=off spool $TESTDIR/sfile # # Test out of space on sub-filesystem diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_010_pos.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_010_pos.ksh index e76f5c5c8620..a9a7d53819d8 100755 --- a/tests/zfs-tests/tests/functional/rsend/rsend_010_pos.ksh +++ b/tests/zfs-tests/tests/functional/rsend/rsend_010_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_011_pos.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_011_pos.ksh index 68f0e13927dc..113ef4f6dc85 100755 --- a/tests/zfs-tests/tests/functional/rsend/rsend_011_pos.ksh +++ b/tests/zfs-tests/tests/functional/rsend/rsend_011_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -65,20 +65,14 @@ done # # Inherit properties in sub-datasets # -for ds in "$POOL/$FS/fs1" "$POOL/$FS/fs1/fs2" "$POOL/$FS/fs1/fclone" ; do - for prop in $(fs_inherit_prop) ; do - zfs inherit $prop $ds - if (($? !=0 )); then - log_fail "zfs inherit $prop $ds" - fi +for ds in "$POOL/$FS/fs1" "$POOL/$FS/fs1/fs2" "$POOL/$FS/fs1/fclone"; do + for prop in $(fs_inherit_prop); do + log_must zfs inherit $prop $ds done done -if is_global_zone ; then - for prop in $(vol_inherit_prop) ; do - zfs inherit $prop $POOL/$FS/vol - if (($? !=0 )); then - log_fail "zfs inherit $prop $POOL/$FS/vol" - fi +if is_global_zone; then + for prop in $(vol_inherit_prop); do + log_must zfs inherit $prop $POOL/$FS/vol done fi diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_012_pos.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_012_pos.ksh index 594357dc4b7a..0fa40f40bed5 100755 --- a/tests/zfs-tests/tests/functional/rsend/rsend_012_pos.ksh +++ b/tests/zfs-tests/tests/functional/rsend/rsend_012_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -47,37 +47,21 @@ function edited_prop typeset behaviour=$1 typeset ds=$2 typeset backfile=$TESTDIR/edited_prop_$ds + typeset te=0 case $behaviour in "get") + is_te_enabled && te=1 typeset props=$(zfs inherit 2>&1 | \ - awk '$2=="YES" {print $1}' | \ - egrep -v "^vol|\.\.\.$") - for item in $props ; do - if [[ $item == "mlslabel" ]] && \ - ! is_te_enabled ; then - continue - fi - zfs get -H -o property,value $item $ds >> \ - $backfile - if (($? != 0)); then - log_fail "zfs get -H -o property,value"\ - "$item $ds > $backfile" - fi - done + awk -v te=$te '$2=="YES" && $1 !~ /^vol|\.\.\.$/ && (te || $1 != "mlslabel") {printf("%s,", $1)}') + log_must eval "zfs get -Ho property,value ${props%,} $ds >> $backfile" ;; "set") if [[ ! -f $backfile ]] ; then log_fail "$ds need backup properties firstly." fi - typeset prop value - while read prop value ; do - eval zfs set $prop='$value' $ds - if (($? != 0)); then - log_fail "zfs set $prop=$value $ds" - fi - done < $backfile + log_must zfs set $(tr '\t' '=' < $backfile) "$ds" ;; *) log_fail "Unrecognized behaviour: $behaviour" @@ -167,20 +151,16 @@ set -A pair "$POOL" "$POOL2" \ typeset -i i=0 while ((i < ${#pair[@]})); do - log_must cmp_ds_prop ${pair[$i]} ${pair[((i+1))]} - + log_must cmp_ds_prop ${pair[$i]} ${pair[((i+1))]} nosource ((i += 2)) done -zpool upgrade -v | grep "Snapshot properties" > /dev/null 2>&1 -if (( $? == 0 )) ; then - i=0 - while ((i < ${#pair[@]})); do - log_must cmp_ds_prop ${pair[$i]}@final ${pair[((i+1))]}@final - ((i += 2)) - done -fi +i=0 +while ((i < ${#pair[@]})); do + log_must cmp_ds_prop ${pair[$i]}@final ${pair[((i+1))]}@final + ((i += 2)) +done -log_pass "Verify zfs send -R will backup all the filesystem properties " \ +log_pass "Verify zfs send -R will backup all the filesystem properties" \ "correctly." diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_013_pos.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_013_pos.ksh index 110e56144674..e06e0f3e5e53 100755 --- a/tests/zfs-tests/tests/functional/rsend/rsend_013_pos.ksh +++ b/tests/zfs-tests/tests/functional/rsend/rsend_013_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_016_neg.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_016_neg.ksh index 26573bfb594d..2741cd20ae0a 100755 --- a/tests/zfs-tests/tests/functional/rsend/rsend_016_neg.ksh +++ b/tests/zfs-tests/tests/functional/rsend/rsend_016_neg.ksh @@ -30,16 +30,9 @@ verify_runnable "both" -function cleanup -{ - rm -f $TEST_BASE_DIR/devnull -} - -log_onexit cleanup - -log_mustnot eval "zfs send -i \#bla $POOl/$FS@final > $TEST_BASE_DIR/devnull" +log_mustnot eval "zfs send -i \#bla $POOl/$FS@final > /dev/null" log_must eval "zfs send -R -i snapA $POOL/vol@snapA 2>&1 " \ - "> $TEST_BASE_DIR/devnull | grep -q WARNING" + "> /dev/null | grep -q WARNING" log_pass "Ensure that error conditions cause appropriate failures." diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_025_pos.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_025_pos.ksh new file mode 100755 index 000000000000..95ec9875737e --- /dev/null +++ b/tests/zfs-tests/tests/functional/rsend/rsend_025_pos.ksh @@ -0,0 +1,90 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright 2009 Sun Microsystems, Inc. All rights reserved. +# Use is subject to license terms. +# + +# +# Copyright (c) 2013, 2016 by Delphix. All rights reserved. +# + +. $STF_SUITE/tests/functional/rsend/rsend.kshlib + +# DESCRIPTION: +# zfs send -exclude will exclude the given hierarchy +# and only that given hierarchy. +# +# STRATEGY: +# 1. Setup test model +# 2. Create several datasets on pool. +# 3. Send -R -X pool/dataset +# 4. Verify receive does not have the excluded dataset(s). + +verify_runnable "both" + +function cleanup +{ + cleanup_pool $POOL2 + cleanup_pool $POOL + log_must setup_test_model $POOL +} + +log_assert "zfs send -R -X will skip excluded dataset(s)" +log_onexit cleanup + +cleanup + +# +# Create some datasets +log_must zfs create -p $POOL/ds1/second/third +log_must zfs create -p $POOL/ds2/second + +log_must zfs snapshot -r $POOL@presend + +log_must eval "zfs send -R $POOL@presend > $BACKDIR/presend" +log_must eval "zfs receive -d -F $POOL2 < $BACKDIR/presend" + +for ds in ds1 ds1/second ds1/second/third \ + ds2 ds2/second +do + log_must datasetexists $POOL2/$ds +done + +log_must_busy zfs destroy -r $POOL2 + +log_must eval "zfs send -R -X $POOL/ds1/second $POOL@presend > $BACKDIR/presend" +log_must eval "zfs receive -d -F $POOL2 < $BACKDIR/presend" + +for ds in ds1 ds2 ds2/second +do + log_must datasetexists $POOL2/$ds +done + +for ds in ds1/second ds1/second/third +do + log_must datasetnonexists $POOL2/$ds +done + +log_pass "zfs send -X excluded datasets" + diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_026_neg.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_026_neg.ksh new file mode 100755 index 000000000000..504e5fba811d --- /dev/null +++ b/tests/zfs-tests/tests/functional/rsend/rsend_026_neg.ksh @@ -0,0 +1,58 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright 2009 Sun Microsystems, Inc. All rights reserved. +# Use is subject to license terms. +# + +# +# Copyright (c) 2013, 2016 by Delphix. All rights reserved. +# + +. $STF_SUITE/include/libtest.shlib +. $STF_SUITE/tests/functional/rsend/rsend.kshlib + +# DESCRIPTION: +# zfs send -X without -R will fail. +# +# STRATEGY: +# 1. Setup test model +# 2. Run "zfs send -X random $POOL" and check for failure. + +verify_runnable "both" + +function cleanup +{ + cleanup_pool $POOL2 + cleanup_pool $POOL + log_must setup_test_model $POOL +} + +log_assert "zfs send -X without -R will fail" +log_onexit cleanup + +cleanup + +log_mustnot eval "zfs send -X $POOL/foobar $POOL@final" + +log_pass "Ensure that zfs send -X without -R will fail" diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_027_pos.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_027_pos.ksh new file mode 100755 index 000000000000..d575ac2676bb --- /dev/null +++ b/tests/zfs-tests/tests/functional/rsend/rsend_027_pos.ksh @@ -0,0 +1,92 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright 2009 Sun Microsystems, Inc. All rights reserved. +# Use is subject to license terms. +# + +# +# Copyright (c) 2013, 2016 by Delphix. All rights reserved. +# + +. $STF_SUITE/tests/functional/rsend/rsend.kshlib + +# DESCRIPTION: +# zfs send with multiple -X/--exclude options will +# exclude all of them. +# +# STRATEGY: +# 1. Setup test model +# 2. Create several datasets on pool. +# 3. Send -R -X pool/dataset +# 4. Verify receive does not have the excluded dataset(s). + +verify_runnable "both" + +function cleanup +{ + cleanup_pool $POOL2 + cleanup_pool $POOL + log_must setup_test_model $POOL +} + +log_assert "zfs send with multiple -X options will skip excluded dataset" +log_onexit cleanup + +cleanup + +# +# Create some datasets +log_must zfs create -p $POOL/ds1/second/third +log_must zfs create -p $POOL/ds2/second +log_must zfs create -p $POOL/ds3/first/second/third + +log_must zfs snapshot -r $POOL@presend + +log_must eval "zfs send -R $POOL@presend > $BACKDIR/presend" +log_must eval "zfs receive -d -F $POOL2 < $BACKDIR/presend" + +for ds in ds1 ds1/second ds1/second/third \ + ds2 ds2/second \ + ds3 ds3/first ds3/first/second ds3/first/second/third +do + log_must datasetexists $POOL2/$ds +done + +log_must_busy zfs destroy -r $POOL2 + +log_must eval "zfs send -R -X $POOL/ds1/second --exclude $POOL/ds3/first/second $POOL@presend > $BACKDIR/presend" +log_must eval "zfs receive -d -F $POOL2 < $BACKDIR/presend" + +for ds in ds1 ds2 ds2/second ds3 ds3/first +do + log_must datasetexists $POOL2/$ds +done + +for ds in ds1/second ds1/second/third ds3/first/second ds3/first/second/third +do + log_must datasetnonexists $POOL2/$ds +done + +log_pass "zfs send with multiple -X options excluded datasets" + diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_028_neg.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_028_neg.ksh new file mode 100755 index 000000000000..a6a1cbfc8ee8 --- /dev/null +++ b/tests/zfs-tests/tests/functional/rsend/rsend_028_neg.ksh @@ -0,0 +1,58 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright 2009 Sun Microsystems, Inc. All rights reserved. +# Use is subject to license terms. +# + +# +# Copyright (c) 2013, 2016 by Delphix. All rights reserved. +# + +. $STF_SUITE/include/libtest.shlib +. $STF_SUITE/tests/functional/rsend/rsend.kshlib + +# DESCRIPTION: +# zfs send -X with invalid dataset name will fail. +# +# STRATEGY: +# 1. Setup test model +# 2. Run "zfs send -X $POOL $POOL" and check for failure. + +verify_runnable "both" + +function cleanup +{ + cleanup_pool $POOL2 + cleanup_pool $POOL + log_must setup_test_model $POOL +} + +log_assert "zfs send -X $POOL will fail" +log_onexit cleanup + +cleanup + +log_mustnot eval "zfs send -X $POOL $POOL@final" + +log_pass "Ensure that zfs send -X $POOL will fail" diff --git a/tests/zfs-tests/tests/functional/rsend/rsend_029_neg.ksh b/tests/zfs-tests/tests/functional/rsend/rsend_029_neg.ksh new file mode 100755 index 000000000000..28d3826cef60 --- /dev/null +++ b/tests/zfs-tests/tests/functional/rsend/rsend_029_neg.ksh @@ -0,0 +1,58 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright 2009 Sun Microsystems, Inc. All rights reserved. +# Use is subject to license terms. +# + +# +# Copyright (c) 2013, 2016 by Delphix. All rights reserved. +# + +. $STF_SUITE/include/libtest.shlib +. $STF_SUITE/tests/functional/rsend/rsend.kshlib + +# DESCRIPTION: +# zfs send -X with invalid dataset name will fail. +# +# STRATEGY: +# 1. Setup test model +# 2. Run "zfs send -X $POOL/da%set $POOL" and check for failure. + +verify_runnable "both" + +function cleanup +{ + cleanup_pool $POOL2 + cleanup_pool $POOL + log_must setup_test_model $POOL +} + +log_assert "zfs send -X $POOL/da%set will fail" +log_onexit cleanup + +cleanup + +log_mustnot eval "zfs send -X $POOL/da%set $POOL@final" + +log_pass "Ensure that zfs send -X with invalid dataset name will fail" diff --git a/tests/zfs-tests/tests/functional/rsend/send-c_props.ksh b/tests/zfs-tests/tests/functional/rsend/send-c_props.ksh index 6e95c2c30b67..82a2eb91e825 100755 --- a/tests/zfs-tests/tests/functional/rsend/send-c_props.ksh +++ b/tests/zfs-tests/tests/functional/rsend/send-c_props.ksh @@ -52,16 +52,27 @@ for opt in "-p" "-R"; do randomize_ds_props $POOL$ds done - log_must eval "zfs send -c $opt $POOL@final > $BACKDIR/pool-final$opt" - log_must eval "zfs receive -d -F $POOL2 < $BACKDIR/pool-final$opt" + if [ $opt = "-p" ]; then + for ds in ${datasets[@]}; do + log_must eval "zfs send -c $opt $POOL$ds@final > $BACKDIR/pool-final$opt" + log_must eval "zfs receive -dF $POOL2 < $BACKDIR/pool-final$opt" + done + else + log_must eval "zfs send -c $opt $POOL@final > $BACKDIR/pool-final$opt" + log_must eval "zfs receive -dF $POOL2 < $BACKDIR/pool-final$opt" + fi for ds in ${datasets[@]}; do - log_must cmp_ds_prop $POOL$ds $POOL2$ds + typeset origin= + if [ $opt = "-p" ] && [ ${ds/clone//} != $ds ]; then + origin=noorigin + fi + log_must cmp_ds_prop $POOL$ds $POOL2$ds nosource $origin log_must cmp_ds_prop $POOL$ds@final $POOL2$ds@final done # Don't cleanup the second time, since we do that on exit anyway. - [[ $opt = "-p" ]] && cleanup + [ $opt = "-p" ] && cleanup done log_pass "Compressed send doesn't interfere with preservation of properties" diff --git a/tests/zfs-tests/tests/functional/rsend/send-c_stream_size_estimate.ksh b/tests/zfs-tests/tests/functional/rsend/send-c_stream_size_estimate.ksh index 056fc2cc2584..5d308d8f6574 100755 --- a/tests/zfs-tests/tests/functional/rsend/send-c_stream_size_estimate.ksh +++ b/tests/zfs-tests/tests/functional/rsend/send-c_stream_size_estimate.ksh @@ -40,19 +40,11 @@ function get_estimated_size { typeset cmd=$1 typeset ds=${cmd##* } - if is_freebsd; then - mkdir -p $BACKDIR - typeset tmpfile=$(TMPDIR=$BACKDIR mktemp) - else - typeset tmpfile=$(mktemp -p $BACKDIR) - fi - - eval "$cmd >$tmpfile" - [[ $? -eq 0 ]] || log_fail "get_estimated_size: $cmd" - typeset size=$(eval "awk '\$2 == \"$ds\" {print \$3}' $tmpfile") - rm -f $tmpfile + typeset tmpfile=$(mktemp $BACKDIR/size_estimate.XXXXXXXX) - echo $size + eval "$cmd >$tmpfile" || log_fail "$cmd: $?" + awk -v ds="$ds" '$2 == ds {print $3}' $tmpfile + rm -f $tmpfile } log_assert "Verify the stream size given by -P accounts for compressed send." @@ -75,23 +67,19 @@ for compress in "${compress_prop_vals[@]}"; do typeset ds_size=$(get_estimated_size "zfs send -nP $send_ds@snap") typeset ds_lrefer=$(get_prop lrefer $send_ds) - within_percent $ds_size $ds_lrefer 90 || log_fail \ - "$ds_size and $ds_lrefer differed by too much" + log_must within_percent $ds_size $ds_lrefer 90 typeset vol_size=$(get_estimated_size "zfs send -nP $send_vol@snap") typeset vol_lrefer=$(get_prop lrefer $send_vol) - within_percent $vol_size $vol_lrefer 90 || log_fail \ - "$vol_size and $vol_lrefer differed by too much" + log_must within_percent $vol_size $vol_lrefer 90 typeset ds_csize=$(get_estimated_size "zfs send -nP -c $send_ds@snap") typeset ds_refer=$(get_prop refer $send_ds) - within_percent $ds_csize $ds_refer 90 || log_fail \ - "$ds_csize and $ds_refer differed by too much" + log_must within_percent $ds_csize $ds_refer 90 typeset vol_csize=$(get_estimated_size "zfs send -nP -c $send_vol@snap") typeset vol_refer=$(get_prop refer $send_vol) - within_percent $vol_csize $vol_refer 90 || log_fail \ - "$vol_csize and $vol_refer differed by too much" + log_must within_percent $vol_csize $vol_refer 90 done log_pass "The stream size given by -P accounts for compressed send." diff --git a/tests/zfs-tests/tests/functional/rsend/send-c_zstreamdump.ksh b/tests/zfs-tests/tests/functional/rsend/send-c_zstreamdump.ksh index 5b9939c6a64c..82d15c68ec47 100755 --- a/tests/zfs-tests/tests/functional/rsend/send-c_zstreamdump.ksh +++ b/tests/zfs-tests/tests/functional/rsend/send-c_zstreamdump.ksh @@ -49,7 +49,7 @@ log_must zfs snapshot $sendfs@full log_must eval "zfs send -c $sendfs@full >$BACKDIR/full" log_must stream_has_features $BACKDIR/full lz4 compressed -cat $BACKDIR/full | zstream dump -v > $BACKDIR/dump.out +zstream dump -v $BACKDIR/full > $BACKDIR/dump.out lsize=$(awk '/^WRITE [^0]/ {lsize += $24} END {printf("%d", lsize)}' \ $BACKDIR/dump.out) @@ -63,8 +63,8 @@ csize_prop=$(get_prop used $sendfs) within_percent $csize $csize_prop 90 || log_fail \ "$csize and $csize_prop differed by too much" -x=$(get_resume_token "zfs send -c $sendfs@full" $streamfs $recvfs) -resume_token=$(cat /$streamfs/resume_token) +get_resume_token "zfs send -c $sendfs@full" $streamfs $recvfs +resume_token=$($stream" $verify eval "zfs recv $recv_ds <$stream" - typeset stream_size=$(cat $stream | zstream dump | sed -n \ + typeset stream_size=$(zstream dump $stream | sed -n \ 's/ Total write size = \(.*\) (0x.*)/\1/p') # diff --git a/tests/zfs-tests/tests/functional/rsend/send_doall.ksh b/tests/zfs-tests/tests/functional/rsend/send_doall.ksh index e5c3490b32cd..55b8c002e61a 100755 --- a/tests/zfs-tests/tests/functional/rsend/send_doall.ksh +++ b/tests/zfs-tests/tests/functional/rsend/send_doall.ksh @@ -46,12 +46,7 @@ log_must zfs create $POOL/fs/child # Create 3 files and a snapshot between each file creation. for i in {1..3}; do - file="/$POOL/fs/file$i" - log_must mkfile 16384 $file - - file="/$POOL/fs/child/file$i" - log_must mkfile 16384 $file - + log_must mkfile 16384 "/$POOL/fs/file$i" "/$POOL/fs/child/file$i" log_must zfs snapshot -r $POOL/fs@snap$i done @@ -59,9 +54,6 @@ done log_must eval "send_doall $POOL/fs@snap3 >$BACKDIR/fs@snap3" log_must eval "zfs recv $POOL/newfs < $BACKDIR/fs@snap3" -zfs list $POOL/newfs/child -if [[ $? -eq 0 ]]; then - log_fail "Children dataset should not have been received" -fi +log_mustnot datasetexists $POOL/newfs/child log_pass "Verify send_doall stream is correct" diff --git a/tests/zfs-tests/tests/functional/rsend/send_encrypted_truncated_files.ksh b/tests/zfs-tests/tests/functional/rsend/send_encrypted_truncated_files.ksh index aa19847e0695..aff54e3a7d69 100755 --- a/tests/zfs-tests/tests/functional/rsend/send_encrypted_truncated_files.ksh +++ b/tests/zfs-tests/tests/functional/rsend/send_encrypted_truncated_files.ksh @@ -54,11 +54,11 @@ function recursive_cksum { case "$(uname)" in FreeBSD) - find $1 -type f -exec sha256 -q {} \; | \ + find $1 -type f -exec sha256 -q {} + | \ sort | sha256digest ;; *) - find $1 -type f -exec sha256sum {} \; | \ + find $1 -type f -exec sha256sum {} + | \ sort -k 2 | awk '{ print $1 }' | sha256digest ;; esac diff --git a/tests/zfs-tests/tests/functional/rsend/send_hole_birth.ksh b/tests/zfs-tests/tests/functional/rsend/send_hole_birth.ksh index 1dfa97e77358..6754fb0f44bb 100755 --- a/tests/zfs-tests/tests/functional/rsend/send_hole_birth.ksh +++ b/tests/zfs-tests/tests/functional/rsend/send_hole_birth.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/rsend/send_invalid.ksh b/tests/zfs-tests/tests/functional/rsend/send_invalid.ksh index 2ce7ee4a082f..ab3b1d73f850 100755 --- a/tests/zfs-tests/tests/functional/rsend/send_invalid.ksh +++ b/tests/zfs-tests/tests/functional/rsend/send_invalid.ksh @@ -44,7 +44,7 @@ log_must zfs snap $testfs@snap0 log_must zfs snap $testfs@snap1 # Test bad send with the CLI -log_mustnot eval "zfs send -i $testfs@snap1 $testfs@snap0 >$TEST_BASE_DIR/devnull" +log_mustnot eval "zfs send -i $testfs@snap1 $testfs@snap0 > /dev/null" # Test bad send with libzfs/libzfs_core log_must badsend $testfs@snap0 $testfs@snap1 diff --git a/tests/zfs-tests/tests/functional/rsend/send_partial_dataset.ksh b/tests/zfs-tests/tests/functional/rsend/send_partial_dataset.ksh index c390327a5b57..c36b0c7c3ece 100755 --- a/tests/zfs-tests/tests/functional/rsend/send_partial_dataset.ksh +++ b/tests/zfs-tests/tests/functional/rsend/send_partial_dataset.ksh @@ -46,7 +46,7 @@ function cleanup } log_onexit cleanup -log_must zfs create $POOL/testfs2 +log_must zfs create -o compression=off $POOL/testfs2 log_must zfs create $POOL/stream mntpnt=$(get_prop mountpoint $POOL/testfs2) @@ -103,7 +103,7 @@ set -A badargs \ while (( i < ${#badargs[*]} )) do - log_mustnot eval "zfs send --saved ${badargs[i]} >$TEST_BASE_DIR/devnull" + log_mustnot eval "zfs send --saved ${badargs[i]} > /dev/null" (( i = i + 1 )) done diff --git a/tests/zfs-tests/tests/functional/rsend/send_raw_ashift.ksh b/tests/zfs-tests/tests/functional/rsend/send_raw_ashift.ksh new file mode 100755 index 000000000000..3cea334495d9 --- /dev/null +++ b/tests/zfs-tests/tests/functional/rsend/send_raw_ashift.ksh @@ -0,0 +1,193 @@ +#!/bin/ksh + +# +# This file and its contents are supplied under the terms of the +# Common Development and Distribution License ("CDDL"), version 1.0. +# You may only use this file in accordance with the terms of version +# 1.0 of the CDDL. +# +# A full copy of the text of the CDDL should have accompanied this +# source. A copy of the CDDL is also available via the Internet at +# http://www.illumos.org/license/CDDL. +# + +# +# Copyright (c) 2019, Lawrence Livermore National Security, LLC. +# Copyright (c) 2021, George Amanakis. All rights reserved. +# + +. $STF_SUITE/include/libtest.shlib +. $STF_SUITE/include/properties.shlib +. $STF_SUITE/tests/functional/rsend/rsend.kshlib + +# +# Description: +# Verify encrypted raw sending to pools with greater ashift succeeds. +# +# Strategy: +# 1) Create a set of files each containing some file data in an +# encrypted filesystem. +# 2) Snapshot and raw send these files to a pool with greater ashift +# 3) Verify that all the xattrs (and thus the spill block) were +# preserved when receiving the incremental stream. +# 4) Repeat the test for a non-encrypted filesystem using raw send +# + +verify_runnable "both" + +log_assert "Verify raw sending to pools with greater ashift succeeds" + +function cleanup +{ + rm -f $BACKDIR/fs@* + poolexists pool9 && destroy_pool pool9 + poolexists pool12 && destroy_pool pool12 + log_must rm -f $TESTDIR/vdev_a $TESTDIR/vdev_b +} + +function xattr_test +{ + log_must zfs set xattr=sa pool9/$1 + log_must zfs set dnodesize=legacy pool9/$1 + log_must zfs set recordsize=128k pool9/$1 + rand_set_prop pool9/$1 compression "${compress_prop_vals[@]}" + + # Create 40 files each with a spill block containing xattrs. Each file + # will be modified in a different way to validate the incremental receive. + for i in {1..40}; do + file="/pool9/$1/file$i" + + log_must mkfile 16384 $file + for j in {1..20}; do + log_must set_xattr "testattr$j" "$attrvalue" $file + done + done + + # Snapshot the pool and send it to the new dataset. + log_must zfs snapshot pool9/$1@snap1 + log_must eval "zfs send -w pool9/$1@snap1 >$BACKDIR/$1@snap1" + log_must eval "zfs recv pool12/$1 < $BACKDIR/$1@snap1" + + # + # Modify file[1-6]'s contents but not the spill blocks. + # + # file1 - Increase record size; single block + # file2 - Increase record size; multiple blocks + # file3 - Truncate file to zero size; single block + # file4 - Truncate file to smaller size; single block + # file5 - Truncate file to much larger size; add holes + # file6 - Truncate file to embedded size; embedded data + # + log_must mkfile 32768 /pool9/$1/file1 + log_must mkfile 1048576 /pool9/$1/file2 + log_must truncate -s 0 /pool9/$1/file3 + log_must truncate -s 8192 /pool9/$1/file4 + log_must truncate -s 1073741824 /pool9/$1/file5 + log_must truncate -s 50 /pool9/$1/file6 + + # + # Modify file[11-16]'s contents and their spill blocks. + # + # file11 - Increase record size; single block + # file12 - Increase record size; multiple blocks + # file13 - Truncate file to zero size; single block + # file14 - Truncate file to smaller size; single block + # file15 - Truncate file to much larger size; add holes + # file16 - Truncate file to embedded size; embedded data + # + log_must mkfile 32768 /pool9/$1/file11 + log_must mkfile 1048576 /pool9/$1/file12 + log_must truncate -s 0 /pool9/$1/file13 + log_must truncate -s 8192 /pool9/$1/file14 + log_must truncate -s 1073741824 /pool9/$1/file15 + log_must truncate -s 50 /pool9/$1/file16 + + for i in {11..20}; do + log_must rm_xattr testattr1 /pool9/$1/file$i + done + + # + # Modify file[21-26]'s contents and remove their spill blocks. + # + # file21 - Increase record size; single block + # file22 - Increase record size; multiple blocks + # file23 - Truncate file to zero size; single block + # file24 - Truncate file to smaller size; single block + # file25 - Truncate file to much larger size; add holes + # file26 - Truncate file to embedded size; embedded data + # + log_must mkfile 32768 /pool9/$1/file21 + log_must mkfile 1048576 /pool9/$1/file22 + log_must truncate -s 0 /pool9/$1/file23 + log_must truncate -s 8192 /pool9/$1/file24 + log_must truncate -s 1073741824 /pool9/$1/file25 + log_must truncate -s 50 /pool9/$1/file26 + + for i in {21..30}; do + for j in {1..20}; do + log_must rm_xattr testattr$j /pool9/$1/file$i + done + done + + # + # Modify file[31-40]'s spill blocks but not the file contents. + # + for i in {31..40}; do + file="/pool9/$1/file$i" + log_must rm_xattr testattr$(((RANDOM % 20) + 1)) $file + log_must set_xattr testattr$(((RANDOM % 20) + 1)) "$attrvalue" $file + done + + # Snapshot the pool and send the incremental snapshot. + log_must zfs snapshot pool9/$1@snap2 + log_must eval "zfs send -w -i pool9/$1@snap1 pool9/$1@snap2 >$BACKDIR/$1@snap2" + log_must eval "zfs recv pool12/$1 < $BACKDIR/$1@snap2" +} + +attrvalue="abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz" + +log_onexit cleanup + +# Create pools +truncate -s $MINVDEVSIZE $TESTDIR/vdev_a +truncate -s $MINVDEVSIZE $TESTDIR/vdev_b +log_must zpool create -f -o ashift=9 pool9 $TESTDIR/vdev_a +log_must zpool create -f -o ashift=12 pool12 $TESTDIR/vdev_b + +# Create encrypted fs +log_must eval "echo 'password' | zfs create -o encryption=on" \ + "-o keyformat=passphrase -o keylocation=prompt " \ + "pool9/encfs" + +# Run xattr tests for encrypted fs +xattr_test encfs + +# Calculate the expected recursive checksum for source encrypted fs +expected_cksum=$(recursive_cksum /pool9/encfs) + +# Mount target encrypted fs +log_must eval "echo 'password' | zfs load-key pool12/encfs" +log_must zfs mount pool12/encfs + +# Validate the received copy using the received recursive checksum +actual_cksum=$(recursive_cksum /pool12/encfs) +if [[ "$expected_cksum" != "$actual_cksum" ]]; then + log_fail "Checksums differ ($expected_cksum != $actual_cksum)" +fi + +# Perform the same test but without encryption (send -w) +log_must zfs create pool9/fs + +# Run xattr tests for non-encrypted fs +xattr_test fs + +# Calculate the expected recursive checksum for source non-encrypted fs +expected_cksum=$(recursive_cksum /pool9/fs) + +# Validate the received copy using the received recursive checksum +actual_cksum=$(recursive_cksum /pool12/fs) +if [[ "$expected_cksum" != "$actual_cksum" ]]; then + log_fail "Checksums differ ($expected_cksum != $actual_cksum)" +fi + +log_pass "Verify raw sending to pools with greater ashift succeeds" diff --git a/tests/zfs-tests/tests/functional/rsend/setup.ksh b/tests/zfs-tests/tests/functional/rsend/setup.ksh index dbf4eecede88..5632df3da7f0 100755 --- a/tests/zfs-tests/tests/functional/rsend/setup.ksh +++ b/tests/zfs-tests/tests/functional/rsend/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/scrub_mirror/Makefile.am b/tests/zfs-tests/tests/functional/scrub_mirror/Makefile.am deleted file mode 100644 index bc657fbf2ad9..000000000000 --- a/tests/zfs-tests/tests/functional/scrub_mirror/Makefile.am +++ /dev/null @@ -1,12 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/scrub_mirror -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - scrub_mirror_001_pos.ksh \ - scrub_mirror_002_pos.ksh \ - scrub_mirror_003_pos.ksh \ - scrub_mirror_004_pos.ksh - -dist_pkgdata_DATA = \ - default.cfg \ - scrub_mirror_common.kshlib diff --git a/tests/zfs-tests/tests/functional/scrub_mirror/cleanup.ksh b/tests/zfs-tests/tests/functional/scrub_mirror/cleanup.ksh index 92e58996ab62..4854a318dc08 100755 --- a/tests/zfs-tests/tests/functional/scrub_mirror/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/scrub_mirror/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/scrub_mirror/default.cfg b/tests/zfs-tests/tests/functional/scrub_mirror/default.cfg index 01c90e58f51b..9c027fd2013f 100644 --- a/tests/zfs-tests/tests/functional/scrub_mirror/default.cfg +++ b/tests/zfs-tests/tests/functional/scrub_mirror/default.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_001_pos.ksh b/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_001_pos.ksh index cce59737255c..a65375022c78 100755 --- a/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_002_pos.ksh b/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_002_pos.ksh index d1fb9d2f174c..a2e32e7ca03f 100755 --- a/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_003_pos.ksh b/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_003_pos.ksh index fa6c3c2f575f..b68e354a3fd2 100755 --- a/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_004_pos.ksh b/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_004_pos.ksh index 3139b047bb6f..c21c226b9c0c 100755 --- a/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_common.kshlib b/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_common.kshlib index 4e434ae0b600..72a9dbd4f1ac 100644 --- a/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_common.kshlib +++ b/tests/zfs-tests/tests/functional/scrub_mirror/scrub_mirror_common.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/scrub_mirror/setup.ksh b/tests/zfs-tests/tests/functional/scrub_mirror/setup.ksh index 94b995db53ed..0bed5df6fd7e 100755 --- a/tests/zfs-tests/tests/functional/scrub_mirror/setup.ksh +++ b/tests/zfs-tests/tests/functional/scrub_mirror/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/simd/Makefile.am b/tests/zfs-tests/tests/functional/simd/Makefile.am deleted file mode 100644 index bfc28868024a..000000000000 --- a/tests/zfs-tests/tests/functional/simd/Makefile.am +++ /dev/null @@ -1,2 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/simd -dist_pkgdata_SCRIPTS = simd_supported.ksh diff --git a/tests/zfs-tests/tests/functional/simd/simd_supported.ksh b/tests/zfs-tests/tests/functional/simd/simd_supported.ksh index d88bc582bf08..79bacc2b8d1e 100755 --- a/tests/zfs-tests/tests/functional/simd/simd_supported.ksh +++ b/tests/zfs-tests/tests/functional/simd/simd_supported.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -32,20 +32,20 @@ # # STRATEGY: # 1. Test if we are running on a Linux x86 system with SSE support -# 2. If so, check if the zfs_fletcher_4_impl module parameter contains +# 2. If so, check if the zfs_fletcher_4_impl module parameter contains # a sse implementation # 3. If not fail the test, otherwise pass it log_note "Testing if we support SIMD instructions (Linux x86 only)" -if !is_linux; then +if ! is_linux; then log_unsupported "Not a Linux System" fi case "$(uname -m)" in -i386|i686|x86_64) - typeset -R modparam="/sys/module/zcommon/parameters/zfs_fletcher_4_impl" - if cat /proc/cpuinfo | awk '/^flags/ {print; exit;}' | grep -q sse; then +i?86|x86_64) + typeset -R modparam="/sys/module/zfs/parameters/zfs_fletcher_4_impl" + if awk '/^flags/ {exit !/sse/}' /proc/cpuinfo; then log_must grep -q sse "$modparam" log_pass "SIMD instructions supported" else diff --git a/tests/zfs-tests/tests/functional/slog/Makefile.am b/tests/zfs-tests/tests/functional/slog/Makefile.am deleted file mode 100644 index 33e3a6d3a496..000000000000 --- a/tests/zfs-tests/tests/functional/slog/Makefile.am +++ /dev/null @@ -1,26 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/slog -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - slog_001_pos.ksh \ - slog_002_pos.ksh \ - slog_003_pos.ksh \ - slog_004_pos.ksh \ - slog_005_pos.ksh \ - slog_006_pos.ksh \ - slog_007_pos.ksh \ - slog_008_neg.ksh \ - slog_009_neg.ksh \ - slog_010_neg.ksh \ - slog_011_neg.ksh \ - slog_012_neg.ksh \ - slog_013_pos.ksh \ - slog_014_pos.ksh \ - slog_015_neg.ksh \ - slog_replay_fs_001.ksh \ - slog_replay_fs_002.ksh \ - slog_replay_volume.ksh - -dist_pkgdata_DATA = \ - slog.cfg \ - slog.kshlib diff --git a/tests/zfs-tests/tests/functional/slog/cleanup.ksh b/tests/zfs-tests/tests/functional/slog/cleanup.ksh index 92bc4aa59d8f..53b5f4c4ab87 100755 --- a/tests/zfs-tests/tests/functional/slog/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/slog/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/setup.ksh b/tests/zfs-tests/tests/functional/slog/setup.ksh index 4278fc69786d..02f05cc568ec 100755 --- a/tests/zfs-tests/tests/functional/slog/setup.ksh +++ b/tests/zfs-tests/tests/functional/slog/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/slog.cfg b/tests/zfs-tests/tests/functional/slog/slog.cfg index d0d25fde151e..ef32d2fb57ff 100644 --- a/tests/zfs-tests/tests/functional/slog/slog.cfg +++ b/tests/zfs-tests/tests/functional/slog/slog.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/slog.kshlib b/tests/zfs-tests/tests/functional/slog/slog.kshlib index 75cfec2d832d..98863d6c9882 100644 --- a/tests/zfs-tests/tests/functional/slog/slog.kshlib +++ b/tests/zfs-tests/tests/functional/slog/slog.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -101,7 +101,7 @@ function verify_slog_device # # mirror:/disks/d ONLINE mirror:/disks/e ONLINE stripe:/disks/f ONLINE # - set -A dev_stat_tab $(zpool status -v $pool | nawk 'BEGIN {start=0} \ + set -A dev_stat_tab $(zpool status -v $pool | awk 'BEGIN {start=0} \ /\tlogs/ {start=1} /\tmirror/ || /\tspares/ || /^$/ {start=0} (start==1) && /\t (\/|[a-zA-Z])/ \ diff --git a/tests/zfs-tests/tests/functional/slog/slog_001_pos.ksh b/tests/zfs-tests/tests/functional/slog/slog_001_pos.ksh index a4c35ed9e98e..112d0192f136 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/slog_002_pos.ksh b/tests/zfs-tests/tests/functional/slog/slog_002_pos.ksh index 91904aa612d1..17fb1f6a4841 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/slog_003_pos.ksh b/tests/zfs-tests/tests/functional/slog/slog_003_pos.ksh index 0b4d6ede3e13..678802e99ad2 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/slog_004_pos.ksh b/tests/zfs-tests/tests/functional/slog/slog_004_pos.ksh index 10f28dcc000b..7c29e2c05ce0 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/slog_005_pos.ksh b/tests/zfs-tests/tests/functional/slog/slog_005_pos.ksh index 4836f6f27937..2dad806033fc 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/slog_006_pos.ksh b/tests/zfs-tests/tests/functional/slog/slog_006_pos.ksh index 24143196fd2e..3804c3896da7 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/slog_007_pos.ksh b/tests/zfs-tests/tests/functional/slog/slog_007_pos.ksh index 27ac38606c29..ff54cd781781 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/slog_008_neg.ksh b/tests/zfs-tests/tests/functional/slog/slog_008_neg.ksh index 54587a0c61a7..8f95c6fe3cae 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_008_neg.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_008_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/slog_009_neg.ksh b/tests/zfs-tests/tests/functional/slog/slog_009_neg.ksh index 222f71a99928..395706b58519 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_009_neg.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_009_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/slog_010_neg.ksh b/tests/zfs-tests/tests/functional/slog/slog_010_neg.ksh index edd9abea0930..800e6cc054cf 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_010_neg.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_010_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/slog_011_neg.ksh b/tests/zfs-tests/tests/functional/slog/slog_011_neg.ksh index 3bebc8201713..9953fa55d2d4 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_011_neg.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_011_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/slog_012_neg.ksh b/tests/zfs-tests/tests/functional/slog/slog_012_neg.ksh index 8d6fb2bffb7f..f1434e10c357 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_012_neg.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_012_neg.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/slog_013_pos.ksh b/tests/zfs-tests/tests/functional/slog/slog_013_pos.ksh index 89b3aeb4034c..b412b310f5b2 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_013_pos.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_013_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/slog/slog_014_pos.ksh b/tests/zfs-tests/tests/functional/slog/slog_014_pos.ksh index dbdf1f1ce527..8f1502c71564 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_014_pos.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_014_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -76,11 +76,7 @@ for type in "mirror" "raidz" "raidz2"; do log_must zpool offline $TESTPOOL $VDIR/a log_must wait_for_degraded $TESTPOOL - zpool status -v $TESTPOOL | grep logs | \ - grep "DEGRADED" 2>&1 >/dev/null - if (( $? == 0 )); then - log_fail "log device should display correct status" - fi + log_mustnot eval "zpool status -v $TESTPOOL | grep logs | grep -q \"DEGRADED\"" log_must zpool online $TESTPOOL $VDIR/a log_must zpool destroy -f $TESTPOOL diff --git a/tests/zfs-tests/tests/functional/slog/slog_015_neg.ksh b/tests/zfs-tests/tests/functional/slog/slog_015_neg.ksh index 04fb225ed4ae..464be019dac3 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_015_neg.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_015_neg.ksh @@ -31,6 +31,8 @@ verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + function cleanup { # diff --git a/tests/zfs-tests/tests/functional/slog/slog_016_pos.ksh b/tests/zfs-tests/tests/functional/slog/slog_016_pos.ksh new file mode 100755 index 000000000000..b69c0c58e38d --- /dev/null +++ b/tests/zfs-tests/tests/functional/slog/slog_016_pos.ksh @@ -0,0 +1,157 @@ +#!/bin/ksh -p +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or https://opensource.org/licenses/CDDL-1.0. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright (c) 2021 by Nutanix. All rights reserved. +# + +. $STF_SUITE/tests/functional/slog/slog.kshlib + +# +# DESCRIPTION: +# Verify saxattr logging in to ZIL works +# +# STRATEGY: +# 1. Create an empty file system (TESTFS) +# 2. Freeze TESTFS +# 3. Create Xattrs. +# 4. Unmount filesystem +# +# 5. Remount TESTFS +# 6. Check xattrs. +# + +verify_runnable "global" + +function cleanup_testenv +{ + cleanup + log_must set_tunable32 ZIL_SAXATTR $orig_zil_saxattr +} + +log_assert "Verify saxattr logging in to ZIL works" + +orig_zil_saxattr=$(get_tunable ZIL_SAXATTR) + +log_onexit cleanup_testenv +log_must setup + +NFILES=10 +function validate_zil_saxattr +{ + saxattrzil=$1 + if [ "$2" == "disabled" ]; then + zilsaxattr_feature_disabled=1 + zpoolcreateflags="-ofeature@zilsaxattr=disabled" + else + zilsaxattr_feature_disabled=0 + zpoolcreateflags="" + fi + + log_must set_tunable32 ZIL_SAXATTR $saxattrzil + + # + # 1. Create an empty file system (TESTFS) + # + log_must zpool create $zpoolcreateflags $TESTPOOL $VDEV log mirror $LDEV + log_must zfs set compression=on $TESTPOOL + log_must zfs create -o xattr=sa $TESTPOOL/$TESTFS + log_must mkdir -p $TESTDIR + + # + # This dd command works around an issue where ZIL records aren't created + # after freezing the pool unless a ZIL header already exists. Create a + # file synchronously to force ZFS to write one out. + # + log_must dd if=/dev/zero of=/$TESTPOOL/$TESTFS/sync \ + conv=fdatasync,fsync bs=1 count=1 + + # + # 2. Freeze TESTFS + # + log_must zpool freeze $TESTPOOL + + rm /$TESTPOOL/$TESTFS/sync + # + # 3. Create xattrs + # + for i in $(seq $NFILES); do + log_must mkdir /$TESTPOOL/$TESTFS/xattr.d.$i + log_must set_xattr test test /$TESTPOOL/$TESTFS/xattr.d.$i + + log_must touch /$TESTPOOL/$TESTFS/xattr.f.$i + log_must set_xattr test test /$TESTPOOL/$TESTFS/xattr.f.$i + done + + # + # 4. Unmount filesystem and export the pool + # + # At this stage TESTFS is empty again and unfrozen, and the + # intent log contains a complete set of deltas to replay it. + # + log_must zfs unmount /$TESTPOOL/$TESTFS + + log_note "Verify transactions to replay:" + log_must zdb -iv $TESTPOOL/$TESTFS + + log_must zpool export $TESTPOOL + + # + # 5. Remount TESTFS + # + # Import the pool to unfreeze it and claim log blocks. It has to be + # `zpool import -f` because we can't write a frozen pool's labels! + # + log_must zpool import -f -d $VDIR $TESTPOOL + + # + # 6. Verify Xattr + # If zilsaxattr_feature_disabled=1 or saxattrzil=0, then xattr=sa + # logging in ZIL is not enabled, So, xattrs would be lost. + # If zilsaxattr_feature_disabled=0 and saxattrzil=1, then xattr=sa + # logging in ZIL is enabled, So, xattrs shouldn't be lost. + # + for i in $(seq $NFILES); do + if [ $zilsaxattr_feature_disabled -eq 1 -o \ + $saxattrzil -eq 0 ]; then + log_mustnot get_xattr test /$TESTPOOL/$TESTFS/xattr.d.$i + log_mustnot get_xattr test /$TESTPOOL/$TESTFS/xattr.f.$i + else + log_must get_xattr test /$TESTPOOL/$TESTFS/xattr.d.$i + log_must get_xattr test /$TESTPOOL/$TESTFS/xattr.f.$i + fi + done + + cleanup + log_must setup +} + + +#Validate zilsaxattr feature enabled. +validate_zil_saxattr 0 +validate_zil_saxattr 1 +#Validate zilsaxattr feature disabled. +validate_zil_saxattr 0 disabled +validate_zil_saxattr 1 disabled + +log_pass "Verify saxattr logging in to ZIL works" diff --git a/tests/zfs-tests/tests/functional/slog/slog_replay_fs_001.ksh b/tests/zfs-tests/tests/functional/slog/slog_replay_fs_001.ksh index 0b78a099f0b9..eddecbc2db7e 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_replay_fs_001.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_replay_fs_001.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -178,8 +178,8 @@ log_must rm /$TESTPOOL/$TESTFS/link_and_unlink.link # # 4. Copy TESTFS to temporary location (TESTDIR/copy) # -log_must mkdir -p $TESTDIR/copy -log_must cp -a /$TESTPOOL/$TESTFS/* $TESTDIR/copy/ +log_must mkdir -p $TESTDIR +log_must rsync -aHAX /$TESTPOOL/$TESTFS/ $TESTDIR/copy # # 5. Unmount filesystem and export the pool @@ -213,7 +213,7 @@ log_must ls_xattr /$TESTPOOL/$TESTFS/xattr.dir log_must ls_xattr /$TESTPOOL/$TESTFS/xattr.file log_note "Verify working set diff:" -log_must diff -r /$TESTPOOL/$TESTFS $TESTDIR/copy +log_must replay_directory_diff $TESTDIR/copy /$TESTPOOL/$TESTFS log_note "Verify file checksum:" typeset checksum1=$(sha256digest /$TESTPOOL/$TESTFS/payload) diff --git a/tests/zfs-tests/tests/functional/slog/slog_replay_fs_002.ksh b/tests/zfs-tests/tests/functional/slog/slog_replay_fs_002.ksh index 3c3ccdf4ad23..a62d229d5057 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_replay_fs_002.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_replay_fs_002.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -98,8 +98,8 @@ log_must eval 'for i in $(seq $NFILES); do zfs set dnodesize=${dnsize[$RANDOM % # # 4. Copy TESTFS to temporary location (TESTDIR/copy) # -log_must mkdir -p $TESTDIR/copy -log_must cp -a /$TESTPOOL/$TESTFS/* $TESTDIR/copy/ +log_must mkdir -p $TESTDIR +log_must rsync -aHAX /$TESTPOOL/$TESTFS/ $TESTDIR/copy # # 5. Unmount filesystem and export the pool @@ -132,6 +132,6 @@ log_note "Verify number of files" log_must test "$(ls /$TESTPOOL/$TESTFS/dir0 | wc -l)" -eq $NFILES log_note "Verify working set diff:" -log_must diff -r /$TESTPOOL/$TESTFS $TESTDIR/copy +log_must replay_directory_diff $TESTDIR/copy /$TESTPOOL/$TESTFS log_pass "Replay of intent log succeeds." diff --git a/tests/zfs-tests/tests/functional/slog/slog_replay_volume.ksh b/tests/zfs-tests/tests/functional/slog/slog_replay_volume.ksh index 35cb4b696965..e99d40570c39 100755 --- a/tests/zfs-tests/tests/functional/slog/slog_replay_volume.ksh +++ b/tests/zfs-tests/tests/functional/slog/slog_replay_volume.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapshot/Makefile.am b/tests/zfs-tests/tests/functional/snapshot/Makefile.am deleted file mode 100644 index 783133a643a1..000000000000 --- a/tests/zfs-tests/tests/functional/snapshot/Makefile.am +++ /dev/null @@ -1,28 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/snapshot -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - clone_001_pos.ksh \ - rollback_001_pos.ksh \ - rollback_002_pos.ksh \ - rollback_003_pos.ksh \ - snapshot_001_pos.ksh \ - snapshot_002_pos.ksh \ - snapshot_003_pos.ksh \ - snapshot_004_pos.ksh \ - snapshot_005_pos.ksh \ - snapshot_006_pos.ksh \ - snapshot_007_pos.ksh \ - snapshot_008_pos.ksh \ - snapshot_009_pos.ksh \ - snapshot_010_pos.ksh \ - snapshot_011_pos.ksh \ - snapshot_012_pos.ksh \ - snapshot_013_pos.ksh \ - snapshot_014_pos.ksh \ - snapshot_015_pos.ksh \ - snapshot_016_pos.ksh \ - snapshot_017_pos.ksh - -dist_pkgdata_DATA = \ - snapshot.cfg diff --git a/tests/zfs-tests/tests/functional/snapshot/cleanup.ksh b/tests/zfs-tests/tests/functional/snapshot/cleanup.ksh index 530a785330dc..4ff6739098a8 100755 --- a/tests/zfs-tests/tests/functional/snapshot/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapshot/clone_001_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/clone_001_pos.ksh index 1c8a3b2a6c20..697f8f83aa09 100755 --- a/tests/zfs-tests/tests/functional/snapshot/clone_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/clone_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -113,8 +113,7 @@ log_onexit cleanup_all setup_all -[[ -n $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 +[ -n $TESTDIR ] && log_must rm -rf $TESTDIR/* typeset -i COUNT=10 typeset -i i=0 @@ -143,8 +142,7 @@ while (( i < ${#args[*]} )); do if [[ -n ${args[i+3]} ]] ; then log_must zfs set mountpoint=${args[i+3]} ${args[i+2]} - FILE_COUNT=`ls -Al ${args[i+3]} | grep -v "total" \ - | grep -v "\.zfs" | wc -l` + FILE_COUNT=$(ls -A ${args[i+3]} | grep -cvF ".zfs") if [[ $FILE_COUNT -ne $COUNT ]]; then ls -Al ${args[i+3]} log_fail "AFTER: ${args[i+3]} contains $FILE_COUNT files(s)." @@ -158,7 +156,7 @@ while (( i < ${#args[*]} )); do (( j = j + 1 )) done - FILE_COUNT=`ls -Al ${args[i+3]}/after* | grep -v "total" | wc -l` + FILE_COUNT=$(ls -A ${args[i+3]}/after* | wc -l) if [[ $FILE_COUNT -ne $COUNT ]]; then ls -Al ${args[i+3]} log_fail "${args[i+3]} contains $FILE_COUNT after* files(s)." diff --git a/tests/zfs-tests/tests/functional/snapshot/rollback_001_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/rollback_001_pos.ksh index 97194f4fe4a0..79b00bd3c1b5 100755 --- a/tests/zfs-tests/tests/functional/snapshot/rollback_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/rollback_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -51,20 +51,17 @@ verify_runnable "both" function cleanup { - snapexists $SNAPFS - [[ $? -eq 0 ]] && \ + snapexists $SNAPFS && log_must zfs destroy $SNAPFS - [[ -e $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 + [ -e $TESTDIR ] && log_must rm -rf $TESTDIR/* } log_assert "Verify that a rollback to a previous snapshot succeeds." log_onexit cleanup -[[ -n $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 +[ -n $TESTDIR ] && log_must rm -rf $TESTDIR/* typeset -i COUNT=10 @@ -79,7 +76,7 @@ done log_must zfs snapshot $SNAPFS -FILE_COUNT=`ls -Al $SNAPDIR | grep -v "total" | wc -l` +FILE_COUNT=$(ls -A $SNAPDIR | wc -l) if [[ $FILE_COUNT -ne $COUNT ]]; then ls -Al $SNAPDIR log_fail "AFTER: $SNAPFS contains $FILE_COUNT files(s)." @@ -100,14 +97,13 @@ sync_pool $TESTPOOL # log_must zfs rollback $SNAPFS -FILE_COUNT=`ls -Al $TESTDIR/after* 2> /dev/null | grep -v "total" | wc -l` +FILE_COUNT=$(ls -A $TESTDIR/after* 2> /dev/null | wc -l) if [[ $FILE_COUNT -ne 0 ]]; then ls -Al $TESTDIR log_fail "$TESTDIR contains $FILE_COUNT after* files(s)." fi -FILE_COUNT=`ls -Al $TESTDIR/before* 2> /dev/null \ - | grep -v "total" | wc -l` +FILE_COUNT=$(ls -A $TESTDIR/before* 2> /dev/null | wc -l) if [[ $FILE_COUNT -ne $COUNT ]]; then ls -Al $TESTDIR log_fail "$TESTDIR contains $FILE_COUNT before* files(s)." diff --git a/tests/zfs-tests/tests/functional/snapshot/rollback_002_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/rollback_002_pos.ksh index c424a69f0a7e..842d469695e3 100755 --- a/tests/zfs-tests/tests/functional/snapshot/rollback_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/rollback_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -51,24 +51,20 @@ verify_runnable "both" function cleanup { - snapexists $SNAPFS.1 - [[ $? -eq 0 ]] && \ + snapexists $SNAPFS.1 && log_must zfs destroy $SNAPFS.1 - snapexists $SNAPFS - [[ $? -eq 0 ]] && \ + snapexists $SNAPFS && log_must zfs destroy $SNAPFS - [[ -e $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 + [ -e $TESTDIR ] && log_must rm -rf $TESTDIR/* } log_assert "Verify rollback is with respect to latest snapshot." log_onexit cleanup -[[ -n $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 +[ -n $TESTDIR ] && log_must rm -rf $TESTDIR/* typeset -i COUNT=10 @@ -83,7 +79,7 @@ done log_must zfs snapshot $SNAPFS -FILE_COUNT=`ls -Al $SNAPDIR | grep -v "total" | wc -l` +FILE_COUNT=$(ls -A $SNAPDIR | wc -l) if [[ $FILE_COUNT -ne $COUNT ]]; then ls -Al $SNAPDIR log_fail "AFTER: $SNAPFS contains $FILE_COUNT files(s)." @@ -109,22 +105,20 @@ while [[ $i -le $COUNT ]]; do (( i = i + 1 )) done -[[ -n $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/original_file* > /dev/null 2>&1 +[ -n $TESTDIR ] && log_must rm -f $TESTDIR/original_file* # # Now rollback to latest snapshot # log_must zfs rollback $SNAPFS.1 -FILE_COUNT=`ls -Al $TESTDIR/aftersecond* 2> /dev/null \ - | grep -v "total" | wc -l` +FILE_COUNT=$(ls -A $TESTDIR/aftersecond* 2> /dev/null | wc -l) if [[ $FILE_COUNT -ne 0 ]]; then ls -Al $TESTDIR log_fail "$TESTDIR contains $FILE_COUNT aftersecond* files(s)." fi -FILE_COUNT=`ls -Al $TESTDIR/original* $TESTDIR/afterfirst*| grep -v "total" | wc -l` +FILE_COUNT=$(ls -A $TESTDIR/original* $TESTDIR/afterfirst* | wc -l) if [[ $FILE_COUNT -ne 20 ]]; then ls -Al $TESTDIR log_fail "$TESTDIR contains $FILE_COUNT original* files(s)." diff --git a/tests/zfs-tests/tests/functional/snapshot/rollback_003_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/rollback_003_pos.ksh index 766de990ecdb..3cbc6694ee0c 100755 --- a/tests/zfs-tests/tests/functional/snapshot/rollback_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/rollback_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -100,7 +100,7 @@ log_must zfs snapshot $SNAPPOOL.1 # # https://github.com/openzfs/zfs/issues/6143 # -log_must df >/dev/null +log_must eval "df >/dev/null" export __ZFS_POOL_RESTRICT="$TESTPOOL" log_must zfs unmount -a @@ -110,6 +110,6 @@ unset __ZFS_POOL_RESTRICT log_must touch /$TESTPOOL/$TESTFILE/$TESTFILE.1 log_must zfs rollback $SNAPPOOL.1 -log_must df >/dev/null +log_must eval "df >/dev/null" log_pass "Rollbacks succeed when nested file systems are present." diff --git a/tests/zfs-tests/tests/functional/snapshot/setup.ksh b/tests/zfs-tests/tests/functional/snapshot/setup.ksh index a73d1aff3c53..17d09fa9c196 100755 --- a/tests/zfs-tests/tests/functional/snapshot/setup.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot.cfg b/tests/zfs-tests/tests/functional/snapshot/snapshot.cfg index 265903fe1469..69b4f3a06fd5 100644 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot.cfg +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_001_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_001_pos.ksh index 8b8c118d9dfb..0724216ee561 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -35,8 +35,8 @@ # # DESCRIPTION: # A zfs file system snapshot is identical to -# the originally snapshot'd file system, after the file -# system has been changed. Uses 'sum -r'. +# the originally snapshotted file system, after the file +# system has been changed. Uses 'cksum'. # # STRATEGY: # 1. Create a file in the zfs file system @@ -50,18 +50,11 @@ verify_runnable "both" function cleanup { - snapexists $SNAPFS - if [[ $? -eq 0 ]]; then + if snapexists $SNAPFS; then log_must zfs destroy $SNAPFS fi - if [[ -e $SNAPDIR ]]; then - log_must rm -rf $SNAPDIR > /dev/null 2>&1 - fi - - if [[ -e $TESTDIR ]]; then - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 - fi + log_must rm -rf $SNAPDIR $TESTDIR/* } log_assert "Verify a file system snapshot is identical to original." @@ -73,7 +66,7 @@ log_must file_write -o create -f $TESTDIR/$TESTFILE -b $BLOCKSZ \ -c $NUM_WRITES -d $DATA log_note "Sum the file, save for later comparison..." -FILE_SUM=`sum -r $TESTDIR/$TESTFILE | awk '{ print $1 }'` +read -r FILE_SUM _ < <(cksum $TESTDIR/$TESTFILE) log_note "FILE_SUM = $FILE_SUM" log_note "Create a snapshot and mount it..." @@ -83,7 +76,7 @@ log_note "Append to the original file..." log_must file_write -o append -f $TESTDIR/$TESTFILE -b $BLOCKSZ \ -c $NUM_WRITES -d $DATA -SNAP_FILE_SUM=`sum -r $SNAPDIR/$TESTFILE | awk '{ print $1 }'` +read -r SNAP_FILE_SUM _ < <(cksum $SNAPDIR/$TESTFILE) if [[ $SNAP_FILE_SUM -ne $FILE_SUM ]]; then log_fail "Sums do not match, aborting!! ($SNAP_FILE_SUM != $FILE_SUM)" fi diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_002_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_002_pos.ksh index 124a7db9c6e6..d9e44d332e05 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -51,35 +51,25 @@ verify_runnable "both" function cleanup { - if [[ -d $CWD ]]; then - cd $CWD || log_fail "Could not cd $CWD" - fi + [ -d $CWD ] && log_must cd $CWD - snapexists $SNAPFS - if [[ $? -eq 0 ]]; then - log_must zfs destroy $SNAPFS - fi - - if [[ -e $SNAPDIR ]]; then - log_must rm -rf $SNAPDIR > /dev/null 2>&1 - fi - - if [[ -e $TESTDIR ]]; then - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 - fi + snapexists $SNAPFS && log_must zfs destroy $SNAPFS + [ -e $SNAPDIR ] && log_must rm -rf $SNAPDIR + [ -e $TESTDIR ] && log_must rm -rf $TESTDIR/* + [ -d "$SNAPSHOT_TARDIR" ] && log_must rm -rf $SNAPSHOT_TARDIR } log_assert "Verify an archive of a file system is identical to " \ "an archive of its snapshot." +SNAPSHOT_TARDIR="$(mktemp -d /tmp/zfstests_snapshot_002.XXXXXX)" log_onexit cleanup typeset -i COUNT=21 typeset OP=create -[[ -n $TESTDIR ]] && \ - rm -rf $TESTDIR/* > /dev/null 2>&1 +[ -n $TESTDIR ] && rm -rf $TESTDIR/* log_note "Create files in the zfs filesystem..." @@ -87,43 +77,37 @@ typeset i=1 while [ $i -lt $COUNT ]; do log_must file_write -o $OP -f $TESTDIR/file$i \ -b $BLOCKSZ -c $NUM_WRITES -d $DATA - (( i = i + 1 )) done log_note "Create a tarball from $TESTDIR contents..." CWD=$PWD -cd $TESTDIR || log_fail "Could not cd $TESTDIR" -log_must tar cf $TESTDIR/tarball.original.tar file* -cd $CWD || log_fail "Could not cd $CWD" +log_must cd $TESTDIR +log_must tar cf $SNAPSHOT_TARDIR/original.tar . +log_must cd $CWD log_note "Create a snapshot and mount it..." log_must zfs snapshot $SNAPFS log_note "Remove all of the original files..." -log_must rm -f $TESTDIR/file* > /dev/null 2>&1 +log_must rm -f $TESTDIR/file* log_note "Create tarball of snapshot..." CWD=$PWD -cd $SNAPDIR || log_fail "Could not cd $SNAPDIR" -log_must tar cf $TESTDIR/tarball.snapshot.tar file* -cd $CWD || log_fail "Could not cd $CWD" +log_must cd $SNAPDIR +log_must tar cf $SNAPSHOT_TARDIR/snapshot.tar . +log_must cd $CWD -log_must mkdir $TESTDIR/original -log_must mkdir $TESTDIR/snapshot +log_must mkdir $TESTDIR/original $TESTDIR/snapshot CWD=$PWD -cd $TESTDIR/original || log_fail "Could not cd $TESTDIR/original" -log_must tar xf $TESTDIR/tarball.original.tar - -cd $TESTDIR/snapshot || log_fail "Could not cd $TESTDIR/snapshot" -log_must tar xf $TESTDIR/tarball.snapshot.tar +log_must cd $TESTDIR/original +log_must tar xf $SNAPSHOT_TARDIR/original.tar -cd $CWD || log_fail "Could not cd $CWD" +log_must cd $TESTDIR/snapshot +log_must tar xf $SNAPSHOT_TARDIR/snapshot.tar -diff -q -r $TESTDIR/original $TESTDIR/snapshot > /dev/null 2>&1 -if [[ $? -eq 1 ]]; then - log_fail "Directory structures differ." -fi +log_must cd $CWD +log_must directory_diff $TESTDIR/original $TESTDIR/snapshot log_pass "Directory structures match." diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_003_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_003_pos.ksh index 054b58cd5590..96f7dac0beec 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -49,20 +49,17 @@ function cleanup { typeset -i i=1 while [ $i -lt $COUNT ]; do - snapexists $SNAPFS.$i - if [[ $? -eq 0 ]]; then - log_must zfs destroy $SNAPFS.$i - fi + snapexists $SNAPFS.$i && log_must zfs destroy $SNAPFS.$i - if [[ -e $SNAPDIR.$i ]]; then - log_must rm -rf $SNAPDIR.$i > /dev/null 2>&1 + if [ -e $SNAPDIR.$i ]; then + log_must rm -rf $SNAPDIR.$i fi (( i = i + 1 )) done - if [[ -e $TESTDIR ]]; then - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 + if [ -e $TESTDIR ]; then + log_must rm -rf $TESTDIR/* fi } @@ -70,8 +67,7 @@ log_assert "Verify many snapshots of a file system can be taken." log_onexit cleanup -[[ -n $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 +[ -n $TESTDIR ] && log_must rm -rf $TESTDIR/* typeset -i COUNT=10 @@ -86,12 +82,11 @@ while [[ $i -lt $COUNT ]]; do done log_note "Remove all of the original files" -[[ -n $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/file* > /dev/null 2>&1 +[ -n $TESTDIR ] && log_must rm -rf $TESTDIR/file* i=1 while [[ $i -lt $COUNT ]]; do - FILECOUNT=`ls $SNAPDIR.$i/file* | wc -l` + FILECOUNT=$(ls $SNAPDIR.$i/file* | wc -l) typeset j=1 while [ $j -lt $FILECOUNT ]; do log_must file_check $SNAPDIR.$i/file$j $j diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_004_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_004_pos.ksh index 9d3b3de1adc0..3725de0c1ec6 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -48,23 +48,19 @@ verify_runnable "both" function cleanup { - snapexists $SNAPFS - [[ $? -eq 0 ]] && \ - log_must zfs destroy $SNAPFS + snapexists $SNAPFS && log_must zfs destroy $SNAPFS - [[ -e $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 + [ -e $TESTDIR ] && log_must rm -rf $TESTDIR/* } log_assert "Verify that a snapshot of an empty file system remains empty." log_onexit cleanup -[[ -n $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 +[ -n $TESTDIR ] && log_must rm -rf $TESTDIR/* log_must zfs snapshot $SNAPFS -FILE_COUNT=`ls -Al $SNAPDIR | grep -v "total 0" | wc -l` +FILE_COUNT=$(ls -A $SNAPDIR | wc -l) if [[ $FILE_COUNT -ne 0 ]]; then ls $SNAPDIR log_fail "BEFORE: $SNAPDIR contains $FILE_COUNT files(s)." @@ -81,7 +77,7 @@ while [[ $i -lt $COUNT ]]; do (( i = i + 1 )) done -FILE_COUNT=`ls -Al $SNAPDIR | grep -v "total 0" | wc -l` +FILE_COUNT=$(ls -A $SNAPDIR | wc -l) if [[ $FILE_COUNT -ne 0 ]]; then ls $SNAPDIR log_fail "AFTER: $SNAPDIR contains $FILE_COUNT files(s)." diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_005_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_005_pos.ksh index c1917dff1299..127b73143893 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -35,7 +35,7 @@ # # DESCRIPTION: # to the originally snapshot'd file system, after the file -# system has been changed. Uses 'sum -r'. +# system has been changed. Uses 'cksum'. # # STRATEGY: # 1) Create a file in the zfs dataset @@ -49,17 +49,14 @@ verify_runnable "both" function cleanup { - snapexists $SNAPCTR - if [[ $? -eq 0 ]]; then - log_must zfs destroy $SNAPCTR - fi + snapexists $SNAPCTR && log_must zfs destroy $SNAPCTR - if [[ -e $SNAPDIR1 ]]; then - log_must rm -rf $SNAPDIR1 > /dev/null 2>&1 + if [ -e $SNAPDIR1 ]; then + log_must rm -rf $SNAPDIR1 fi - if [[ -e $TESTDIR ]]; then - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 + if [ -e $TESTDIR ]; then + log_must rm -rf $TESTDIR/* fi } @@ -72,7 +69,7 @@ log_must file_write -o create -f $TESTDIR1/$TESTFILE -b $BLOCKSZ \ -c $NUM_WRITES -d $DATA log_note "Sum the file, save for later comparison..." -FILE_SUM=`sum -r $TESTDIR1/$TESTFILE | awk '{ print $1 }'` +read -r FILE_SUM _ < <(cksum $TESTDIR1/$TESTFILE) log_note "FILE_SUM = $FILE_SUM" log_note "Create a snapshot and mount it..." @@ -82,8 +79,8 @@ log_note "Append to the original file..." log_must file_write -o append -f $TESTDIR1/$TESTFILE -b $BLOCKSZ \ -c $NUM_WRITES -d $DATA -SNAP_FILE_SUM=`sum -r $SNAPDIR1/$TESTFILE | awk '{ print $1 }'` -if [[ $SNAP_FILE_SUM -ne $FILE_SUM ]]; then +read -r SNAP_FILE_SUM _ < <(cksum $SNAPDIR1/$TESTFILE) +if [ $SNAP_FILE_SUM -ne $FILE_SUM ]; then log_fail "Sums do not match, aborting!! ($SNAP_FILE_SUM != $FILE_SUM)" fi diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_006_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_006_pos.ksh index 68a616c02a6c..e34a50941470 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_006_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_006_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -51,33 +51,34 @@ verify_runnable "both" function cleanup { if [[ -d $CWD ]]; then - cd $CWD || log_fail "Could not cd $CWD" + log_must cd $CWD fi - snapexists $SNAPCTR - if [[ $? -eq 0 ]]; then - log_must zfs destroy $SNAPCTR - fi + snapexists $SNAPCTR && log_must zfs destroy $SNAPCTR - if [[ -e $SNAPDIR1 ]]; then - log_must rm -rf $SNAPDIR1 > /dev/null 2>&1 - fi + if [ -e $SNAPDIR1 ]; then + log_must rm -rf $SNAPDIR1 + fi - if [[ -e $TESTDIR1 ]]; then - log_must rm -rf $TESTDIR1/* > /dev/null 2>&1 - fi + if [ -e $TESTDIR1 ]; then + log_must rm -rf $TESTDIR1/* + fi + if [ -d "$SNAPSHOT_TARDIR" ]; then + log_must rm -rf $SNAPSHOT_TARDIR + fi } log_assert "Verify that an archive of a dataset is identical to " \ "an archive of the dataset's snapshot." +SNAPSHOT_TARDIR="$(mktemp -d /tmp/zfstests_snapshot_006.XXXXXX)" log_onexit cleanup typeset -i COUNT=21 typeset OP=create -[[ -n $TESTDIR1 ]] && rm -rf $TESTDIR1/* > /dev/null 2>&1 +[ -n $TESTDIR1 ] && rm -rf $TESTDIR1/* log_note "Create files in the zfs dataset ..." @@ -85,43 +86,37 @@ typeset i=1 while [ $i -lt $COUNT ]; do log_must file_write -o $OP -f $TESTDIR1/file$i \ -b $BLOCKSZ -c $NUM_WRITES -d $DATA - (( i = i + 1 )) done log_note "Create a tarball from $TESTDIR1 contents..." CWD=$PWD -cd $TESTDIR1 || log_fail "Could not cd $TESTDIR1" -log_must tar cf $TESTDIR1/tarball.original.tar file* -cd $CWD || log_fail "Could not cd $CWD" +log_must cd $TESTDIR1 +log_must tar cf $SNAPSHOT_TARDIR/original.tar . +log_must cd $CWD log_note "Create a snapshot and mount it..." log_must zfs snapshot $SNAPCTR log_note "Remove all of the original files..." -log_must rm -f $TESTDIR1/file* > /dev/null 2>&1 +log_must rm -f $TESTDIR1/file* log_note "Create tarball of snapshot..." CWD=$PWD -cd $SNAPDIR1 || log_fail "Could not cd $SNAPDIR1" -log_must tar cf $TESTDIR1/tarball.snapshot.tar file* -cd $CWD || log_fail "Could not cd $CWD" +log_must cd $SNAPDIR1 +log_must tar cf $SNAPSHOT_TARDIR/snapshot.tar . +log_must cd $CWD -log_must mkdir $TESTDIR1/original -log_must mkdir $TESTDIR1/snapshot +log_must mkdir $TESTDIR1/original mkdir $TESTDIR1/snapshot CWD=$PWD -cd $TESTDIR1/original || log_fail "Could not cd $TESTDIR1/original" -log_must tar xf $TESTDIR1/tarball.original.tar - -cd $TESTDIR1/snapshot || log_fail "Could not cd $TESTDIR1/snapshot" -log_must tar xf $TESTDIR1/tarball.snapshot.tar +log_must cd $TESTDIR1/original +log_must tar xf $SNAPSHOT_TARDIR/original.tar -cd $CWD || log_fail "Could not cd $CWD" +log_must cd $TESTDIR1/snapshot +log_must tar xf $SNAPSHOT_TARDIR/snapshot.tar -diff -q -r $TESTDIR1/original $TESTDIR1/snapshot > /dev/null 2>&1 -if [[ $? -eq 1 ]]; then - log_fail "Directory structures differ." -fi +log_must cd $CWD +log_must directory_diff $TESTDIR1/original $TESTDIR1/snapshot log_pass "Directory structures match." diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_007_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_007_pos.ksh index 7ed1fdb6e7ad..3f7029f581e7 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_007_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_007_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -49,24 +49,21 @@ function cleanup { typeset -i i=1 while [ $i -lt $COUNT ]; do - snapexists $SNAPCTR.$i - if [[ $? -eq 0 ]]; then - log_must zfs destroy $SNAPCTR.$i - fi + snapexists $SNAPCTR.$i && log_must zfs destroy $SNAPCTR.$i - if [[ -e $SNAPDIR.$i ]]; then - log_must rm -rf $SNAPDIR1.$i > /dev/null 2>&1 + if [ -e $SNAPDIR.$i ]; then + log_must rm -rf $SNAPDIR1.$i fi (( i = i + 1 )) done - if [[ -e $SNAPDIR1 ]]; then - log_must rm -rf $SNAPDIR1 > /dev/null 2>&1 + if [ -e $SNAPDIR1 ]; then + log_must rm -rf $SNAPDIR1 fi - if [[ -e $TESTDIR ]]; then - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 + if [ -e $TESTDIR ]; then + log_must rm -rf $TESTDIR/* fi } @@ -74,8 +71,7 @@ log_assert "Verify that many snapshots can be made on a zfs dataset." log_onexit cleanup -[[ -n $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 +[ -n $TESTDIR ] && log_must rm -rf $TESTDIR/* typeset -i COUNT=10 @@ -90,12 +86,11 @@ while [[ $i -lt $COUNT ]]; do done log_note "Remove all of the original files" -[[ -n $TESTDIR ]] && \ - log_must rm -rf $TESTDIR1/file* > /dev/null 2>&1 +[ -n $TESTDIR ] && log_must rm -f $TESTDIR1/file* i=1 while [[ $i -lt $COUNT ]]; do - FILECOUNT=`ls $SNAPDIR1.$i/file* | wc -l` + FILECOUNT=$(ls $SNAPDIR1.$i/file* 2>/dev/null | wc -l) typeset j=1 while [ $j -lt $FILECOUNT ]; do log_must file_check $SNAPDIR1.$i/file$j $j diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_008_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_008_pos.ksh index d0ecb77fe0a0..299424fc9573 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_008_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_008_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -48,23 +48,20 @@ function cleanup { typeset -i i=1 while [[ $i -lt $COUNT ]]; do - snapexists $SNAPFS.$i - [[ $? -eq 0 ]] && \ + snapexists $SNAPFS.$i && log_must zfs destroy $SNAPFS.$i (( i = i + 1 )) done - [[ -e $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 + [ -e $TESTDIR ] && log_must rm -rf $TESTDIR/* } log_assert "Verify that destroying snapshots returns space to the pool." log_onexit cleanup -[[ -n $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 +[ -n $TESTDIR ] && log_must rm -rf $TESTDIR/* typeset -i COUNT=10 @@ -82,7 +79,7 @@ done typeset -i i=1 while [[ $i -lt $COUNT ]]; do - log_must rm -rf $TESTDIR/file$i > /dev/null 2>&1 + log_must rm -f $TESTDIR/file$i log_must zfs destroy $SNAPFS.$i (( i = i + 1 )) diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_009_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_009_pos.ksh index 1ee7e33c2ac2..ccc0857b077b 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_009_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_009_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_010_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_010_pos.ksh index 128b443c6fc9..4aca2854a7d9 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_010_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_010_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_011_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_011_pos.ksh index 7e0a7f4ce1d8..2d5f63b6237c 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_011_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_011_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -53,15 +53,13 @@ function cleanup { snapexists $SNAPPOOL && destroy_dataset $SNAPPOOL -r - [[ -e $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 + [ -e $TESTDIR ] && log_must rm -rf $TESTDIR/* } log_assert "Verify that rollback to a snapshot created by snapshot -r succeeds." log_onexit cleanup -[[ -n $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 +[ -n $TESTDIR ] && log_must rm -rf $TESTDIR/* typeset -i COUNT=10 @@ -76,7 +74,7 @@ done log_must zfs snapshot -r $SNAPPOOL -FILE_COUNT=`ls -Al $SNAPDIR | grep -v "total" | wc -l` +FILE_COUNT=$(ls -A $SNAPDIR | wc -l) if (( FILE_COUNT != COUNT )); then ls -Al $SNAPDIR log_fail "AFTER: $SNAPFS contains $FILE_COUNT files(s)." @@ -96,14 +94,13 @@ done # log_must zfs rollback $SNAPFS -FILE_COUNT=`ls -Al $TESTDIR/after* 2> /dev/null | grep -v "total" | wc -l` +FILE_COUNT=$(ls -A $TESTDIR/after* 2> /dev/null | wc -l) if (( FILE_COUNT != 0 )); then ls -Al $TESTDIR log_fail "$TESTDIR contains $FILE_COUNT after* files(s)." fi -FILE_COUNT=`ls -Al $TESTDIR/before* 2> /dev/null \ - | grep -v "total" | wc -l` +FILE_COUNT=$(ls -A $TESTDIR/before* 2> /dev/null | wc -l) if (( FILE_COUNT != $COUNT )); then ls -Al $TESTDIR log_fail "$TESTDIR contains $FILE_COUNT before* files(s)." diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_012_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_012_pos.ksh index 92db9b53a7b8..d2671787975b 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_012_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_012_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_013_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_013_pos.ksh index e02f6eb30042..a071759beca6 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_013_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_013_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -51,8 +51,7 @@ function cleanup datasetexists $ctrfs && destroy_dataset $ctrfs -r snapexists $snappool && destroy_dataset $snappool -r - [[ -e $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 + [ -e $TESTDIR ] && log_must rm -rf $TESTDIR/* } log_assert "Verify snapshots from 'snapshot -r' can be used for zfs send/recv" @@ -67,8 +66,7 @@ snapctrfs=$ctrfs@$TESTSNAP fsdir=/$ctrfs snapdir=$fsdir/.zfs/snapshot/$TESTSNAP -[[ -n $TESTDIR ]] && \ - log_must rm -rf $TESTDIR/* > /dev/null 2>&1 +[ -n $TESTDIR ] && log_must rm -rf $TESTDIR/* typeset -i COUNT=10 @@ -89,7 +87,7 @@ if ! datasetexists $ctrfs || ! snapexists $snapctrfs; then fi for dir in $fsdir $snapdir; do - FILE_COUNT=`ls -Al $dir | grep -v "total" | wc -l` + FILE_COUNT=$(ls -A $dir | wc -l) (( FILE_COUNT != COUNT )) && log_fail "Got $FILE_COUNT expected $COUNT" done diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_014_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_014_pos.ksh index d48d404b6d14..9c7e64ab6cf0 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_014_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_014_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -48,19 +48,21 @@ verify_runnable "both" function cleanup { - [[ -e $TESTDIR1 ]] && \ - log_must rm -rf $TESTDIR1/* > /dev/null 2>&1 + [ -e $TESTDIR1 ] && log_must rm -rf $TESTDIR1/* snapexists $SNAPCTR && destroy_dataset $SNAPCTR datasetexists $TESTPOOL/$TESTCTR/$TESTFS1 && \ log_must zfs set quota=none $TESTPOOL/$TESTCTR/$TESTFS1 + zfs inherit compression $TESTPOOL } log_assert "Verify creating/destroying snapshots do things clean" log_onexit cleanup +log_must zfs set compression=off $TESTPOOL + log_must zfs set quota=$FSQUOTA $TESTPOOL/$TESTCTR/$TESTFS1 log_must mkfile $FILESIZE $TESTDIR1/$TESTFILE diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_015_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_015_pos.ksh index 5a4d2ccaf62e..4fdbfce55063 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_015_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_015_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_016_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_016_pos.ksh index b66023cc85e6..6058ec1cfe70 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_016_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_016_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapshot/snapshot_017_pos.ksh b/tests/zfs-tests/tests/functional/snapshot/snapshot_017_pos.ksh index 6e5b8973cf4f..dc234636ec6a 100755 --- a/tests/zfs-tests/tests/functional/snapshot/snapshot_017_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapshot/snapshot_017_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapused/Makefile.am b/tests/zfs-tests/tests/functional/snapused/Makefile.am deleted file mode 100644 index d6551b7c41b1..000000000000 --- a/tests/zfs-tests/tests/functional/snapused/Makefile.am +++ /dev/null @@ -1,12 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/snapused -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - snapused_001_pos.ksh \ - snapused_002_pos.ksh \ - snapused_003_pos.ksh \ - snapused_004_pos.ksh \ - snapused_005_pos.ksh - -dist_pkgdata_DATA = \ - snapused.kshlib diff --git a/tests/zfs-tests/tests/functional/snapused/cleanup.ksh b/tests/zfs-tests/tests/functional/snapused/cleanup.ksh index 2f536ca588ed..557bfeb213dd 100755 --- a/tests/zfs-tests/tests/functional/snapused/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/snapused/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapused/setup.ksh b/tests/zfs-tests/tests/functional/snapused/setup.ksh index dfe8696add67..c1e02d843a32 100755 --- a/tests/zfs-tests/tests/functional/snapused/setup.ksh +++ b/tests/zfs-tests/tests/functional/snapused/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapused/snapused.kshlib b/tests/zfs-tests/tests/functional/snapused/snapused.kshlib index d7d05bfeef1f..043f54d54fd2 100644 --- a/tests/zfs-tests/tests/functional/snapused/snapused.kshlib +++ b/tests/zfs-tests/tests/functional/snapused/snapused.kshlib @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapused/snapused_001_pos.ksh b/tests/zfs-tests/tests/functional/snapused/snapused_001_pos.ksh index c1277f2b4e2f..6d33f32d6875 100755 --- a/tests/zfs-tests/tests/functional/snapused/snapused_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapused/snapused_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapused/snapused_002_pos.ksh b/tests/zfs-tests/tests/functional/snapused/snapused_002_pos.ksh index a41ca1d70f19..c78e7cde39b7 100755 --- a/tests/zfs-tests/tests/functional/snapused/snapused_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapused/snapused_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapused/snapused_003_pos.ksh b/tests/zfs-tests/tests/functional/snapused/snapused_003_pos.ksh index ff54cbaa1aec..940dc0e7bfc5 100755 --- a/tests/zfs-tests/tests/functional/snapused/snapused_003_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapused/snapused_003_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapused/snapused_004_pos.ksh b/tests/zfs-tests/tests/functional/snapused/snapused_004_pos.ksh index 8fb8b6be5b47..d5f97168bd3c 100755 --- a/tests/zfs-tests/tests/functional/snapused/snapused_004_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapused/snapused_004_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/snapused/snapused_005_pos.ksh b/tests/zfs-tests/tests/functional/snapused/snapused_005_pos.ksh index 9d21e1d23d87..fbf7165e7734 100755 --- a/tests/zfs-tests/tests/functional/snapused/snapused_005_pos.ksh +++ b/tests/zfs-tests/tests/functional/snapused/snapused_005_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/sparse/Makefile.am b/tests/zfs-tests/tests/functional/sparse/Makefile.am deleted file mode 100644 index f1b9e04dcefe..000000000000 --- a/tests/zfs-tests/tests/functional/sparse/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/sparse -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - sparse_001_pos.ksh - -dist_pkgdata_DATA = \ - sparse.cfg diff --git a/tests/zfs-tests/tests/functional/sparse/cleanup.ksh b/tests/zfs-tests/tests/functional/sparse/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/sparse/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/sparse/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/sparse/setup.ksh b/tests/zfs-tests/tests/functional/sparse/setup.ksh index 3a45ec8d510d..d617f361b24d 100755 --- a/tests/zfs-tests/tests/functional/sparse/setup.ksh +++ b/tests/zfs-tests/tests/functional/sparse/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/sparse/sparse.cfg b/tests/zfs-tests/tests/functional/sparse/sparse.cfg index 0fc669148aa1..970dd25177ea 100644 --- a/tests/zfs-tests/tests/functional/sparse/sparse.cfg +++ b/tests/zfs-tests/tests/functional/sparse/sparse.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -39,5 +39,5 @@ export HOLES_COUNT=${HOLES_COUNT-"16384"} # FILESIZE/BLKSIZE/8 export STF_TIMEOUT=3600 export DISKSARRAY=$DISKS -export DISK_ARRAY_NUM=$(echo ${DISKS} | nawk '{print NF}') +export DISK_ARRAY_NUM=$(echo ${DISKS} | awk '{print NF}') set_device_dir diff --git a/tests/zfs-tests/tests/functional/sparse/sparse_001_pos.ksh b/tests/zfs-tests/tests/functional/sparse/sparse_001_pos.ksh index 6ad252adf325..3e25af39c08a 100755 --- a/tests/zfs-tests/tests/functional/sparse/sparse_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/sparse/sparse_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/stat/Makefile.am b/tests/zfs-tests/tests/functional/stat/Makefile.am deleted file mode 100644 index 1a861a655cb1..000000000000 --- a/tests/zfs-tests/tests/functional/stat/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/stat - -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - stat_001_pos.ksh diff --git a/tests/zfs-tests/tests/functional/stat/cleanup.ksh b/tests/zfs-tests/tests/functional/stat/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/stat/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/stat/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/stat/setup.ksh b/tests/zfs-tests/tests/functional/stat/setup.ksh index 4fc55cd47803..bc74ee62e075 100755 --- a/tests/zfs-tests/tests/functional/stat/setup.ksh +++ b/tests/zfs-tests/tests/functional/stat/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/stat/stat_001_pos.ksh b/tests/zfs-tests/tests/functional/stat/stat_001_pos.ksh index e6f9775f4b8e..e167526f4c3d 100755 --- a/tests/zfs-tests/tests/functional/stat/stat_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/stat/stat_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/suid/.gitignore b/tests/zfs-tests/tests/functional/suid/.gitignore deleted file mode 100644 index a9a3db79ba44..000000000000 --- a/tests/zfs-tests/tests/functional/suid/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/suid_write_to_file diff --git a/tests/zfs-tests/tests/functional/suid/Makefile.am b/tests/zfs-tests/tests/functional/suid/Makefile.am deleted file mode 100644 index 0145c1205fb3..000000000000 --- a/tests/zfs-tests/tests/functional/suid/Makefile.am +++ /dev/null @@ -1,17 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/suid - -dist_pkgdata_SCRIPTS = \ - suid_write_to_suid.ksh \ - suid_write_to_sgid.ksh \ - suid_write_to_suid_sgid.ksh \ - suid_write_to_none.ksh \ - suid_write_zil_replay.ksh \ - cleanup.ksh \ - setup.ksh - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/suid - -pkgexec_PROGRAMS = suid_write_to_file -suid_write_to_file_SOURCES = suid_write_to_file.c diff --git a/tests/zfs-tests/tests/functional/suid/cleanup.ksh b/tests/zfs-tests/tests/functional/suid/cleanup.ksh index 6e41e02faf58..b1aa351587f1 100755 --- a/tests/zfs-tests/tests/functional/suid/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/suid/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/suid/setup.ksh b/tests/zfs-tests/tests/functional/suid/setup.ksh index d04d5568c003..96e891a5ba1d 100755 --- a/tests/zfs-tests/tests/functional/suid/setup.ksh +++ b/tests/zfs-tests/tests/functional/suid/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/suid/suid_write_to_none.ksh b/tests/zfs-tests/tests/functional/suid/suid_write_to_none.ksh index 470350f960cf..984dda6a0208 100755 --- a/tests/zfs-tests/tests/functional/suid/suid_write_to_none.ksh +++ b/tests/zfs-tests/tests/functional/suid/suid_write_to_none.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -47,6 +47,6 @@ function cleanup log_onexit cleanup log_note "Verify write(2) to regular file by non-owner" -log_must $STF_SUITE/tests/functional/suid/suid_write_to_file "NONE" "PRECRASH" +log_must suid_write_to_file "NONE" "PRECRASH" log_pass "Verify write(2) to regular file by non-owner passed" diff --git a/tests/zfs-tests/tests/functional/suid/suid_write_to_sgid.ksh b/tests/zfs-tests/tests/functional/suid/suid_write_to_sgid.ksh index 3c95a402658e..41367e0b136d 100755 --- a/tests/zfs-tests/tests/functional/suid/suid_write_to_sgid.ksh +++ b/tests/zfs-tests/tests/functional/suid/suid_write_to_sgid.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -47,6 +47,6 @@ function cleanup log_onexit cleanup log_note "Verify write(2) to SGID file by non-owner" -log_must $STF_SUITE/tests/functional/suid/suid_write_to_file "SGID" "PRECRASH" +log_must suid_write_to_file "SGID" "PRECRASH" log_pass "Verify write(2) to SGID file by non-owner passed" diff --git a/tests/zfs-tests/tests/functional/suid/suid_write_to_suid.ksh b/tests/zfs-tests/tests/functional/suid/suid_write_to_suid.ksh index 4183cbeefc20..77d092227ab1 100755 --- a/tests/zfs-tests/tests/functional/suid/suid_write_to_suid.ksh +++ b/tests/zfs-tests/tests/functional/suid/suid_write_to_suid.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -47,6 +47,6 @@ function cleanup log_onexit cleanup log_note "Verify write(2) to SUID file by non-owner" -log_must $STF_SUITE/tests/functional/suid/suid_write_to_file "SUID" "PRECRASH" +log_must suid_write_to_file "SUID" "PRECRASH" log_pass "Verify write(2) to SUID file by non-owner passed" diff --git a/tests/zfs-tests/tests/functional/suid/suid_write_to_suid_sgid.ksh b/tests/zfs-tests/tests/functional/suid/suid_write_to_suid_sgid.ksh index f7a08a55fc4b..51f6775c003f 100755 --- a/tests/zfs-tests/tests/functional/suid/suid_write_to_suid_sgid.ksh +++ b/tests/zfs-tests/tests/functional/suid/suid_write_to_suid_sgid.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -47,6 +47,6 @@ function cleanup log_onexit cleanup log_note "Verify write(2) to SUID/SGID file by non-owner" -log_must $STF_SUITE/tests/functional/suid/suid_write_to_file "SUID_SGID" "PRECRASH" +log_must suid_write_to_file "SUID_SGID" "PRECRASH" log_pass "Verify write(2) to SUID/SGID file by non-owner passed" diff --git a/tests/zfs-tests/tests/functional/suid/suid_write_zil_replay.ksh b/tests/zfs-tests/tests/functional/suid/suid_write_zil_replay.ksh index 81f431f6b68b..1d7407000ebf 100755 --- a/tests/zfs-tests/tests/functional/suid/suid_write_zil_replay.ksh +++ b/tests/zfs-tests/tests/functional/suid/suid_write_zil_replay.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -65,10 +65,10 @@ log_must zpool freeze $TESTPOOL # # 3. Unprivileged write to a setuid file # -log_must $STF_SUITE/tests/functional/suid/suid_write_to_file "NONE" "PRECRASH" -log_must $STF_SUITE/tests/functional/suid/suid_write_to_file "SUID" "PRECRASH" -log_must $STF_SUITE/tests/functional/suid/suid_write_to_file "SGID" "PRECRASH" -log_must $STF_SUITE/tests/functional/suid/suid_write_to_file "SUID_SGID" "PRECRASH" +log_must suid_write_to_file "NONE" "PRECRASH" +log_must suid_write_to_file "SUID" "PRECRASH" +log_must suid_write_to_file "SGID" "PRECRASH" +log_must suid_write_to_file "SUID_SGID" "PRECRASH" # # 4. Unmount filesystem and export the pool @@ -91,9 +91,9 @@ log_must zpool export $TESTPOOL # log_must zpool import -f -d $VDIR $TESTPOOL -log_must $STF_SUITE/tests/functional/suid/suid_write_to_file "NONE" "REPLAY" -log_must $STF_SUITE/tests/functional/suid/suid_write_to_file "SUID" "REPLAY" -log_must $STF_SUITE/tests/functional/suid/suid_write_to_file "SGID" "REPLAY" -log_must $STF_SUITE/tests/functional/suid/suid_write_to_file "SUID_SGID" "REPLAY" +log_must suid_write_to_file "NONE" "REPLAY" +log_must suid_write_to_file "SUID" "REPLAY" +log_must suid_write_to_file "SGID" "REPLAY" +log_must suid_write_to_file "SUID_SGID" "REPLAY" log_pass diff --git a/tests/zfs-tests/tests/functional/threadsappend/.gitignore b/tests/zfs-tests/tests/functional/threadsappend/.gitignore deleted file mode 100644 index 4c8c8cdf34c1..000000000000 --- a/tests/zfs-tests/tests/functional/threadsappend/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/threadsappend diff --git a/tests/zfs-tests/tests/functional/threadsappend/Makefile.am b/tests/zfs-tests/tests/functional/threadsappend/Makefile.am deleted file mode 100644 index 80f7788c8d86..000000000000 --- a/tests/zfs-tests/tests/functional/threadsappend/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/threadsappend - -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh \ - threadsappend_001_pos.ksh diff --git a/tests/zfs-tests/tests/functional/tmpfile/Makefile.am b/tests/zfs-tests/tests/functional/tmpfile/Makefile.am deleted file mode 100644 index 35a1f44c1693..000000000000 --- a/tests/zfs-tests/tests/functional/tmpfile/Makefile.am +++ /dev/null @@ -1,16 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/tmpfile - -dist_pkgdata_SCRIPTS = \ - cleanup.ksh \ - setup.ksh - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/tmpfile - -pkgexec_PROGRAMS = tmpfile_test tmpfile_001_pos tmpfile_002_pos \ - tmpfile_003_pos tmpfile_stat_mode -tmpfile_test_SOURCES= tmpfile_test.c -tmpfile_001_pos_SOURCES = tmpfile_001_pos.c -tmpfile_002_pos_SOURCES = tmpfile_002_pos.c -tmpfile_003_pos_SOURCES = tmpfile_003_pos.c diff --git a/tests/zfs-tests/tests/functional/tmpfile/cleanup.ksh b/tests/zfs-tests/tests/functional/tmpfile/cleanup.ksh index 3166bd6ec16e..42fe70042d6a 100755 --- a/tests/zfs-tests/tests/functional/tmpfile/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/tmpfile/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/tmpfile/setup.ksh b/tests/zfs-tests/tests/functional/tmpfile/setup.ksh index bc00a2a22c5a..2b38556c9a78 100755 --- a/tests/zfs-tests/tests/functional/tmpfile/setup.ksh +++ b/tests/zfs-tests/tests/functional/tmpfile/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/tmpfile/tmpfile_001_pos.c b/tests/zfs-tests/tests/functional/tmpfile/tmpfile_001_pos.c index d40da0d2ba62..503861656367 100644 --- a/tests/zfs-tests/tests/functional/tmpfile/tmpfile_001_pos.c +++ b/tests/zfs-tests/tests/functional/tmpfile/tmpfile_001_pos.c @@ -8,6 +8,7 @@ #include #include #include +#include /* backward compat in case it's not defined */ #ifndef O_TMPFILE @@ -33,75 +34,45 @@ fill_random(char *buf, int len) { srand(time(NULL)); for (int i = 0; i < len; i++) - buf[i] = (char)rand(); + buf[i] = (char)(rand() % 0xFF); } int main(void) { - int i, fd; - char buf1[BSZ], buf2[BSZ] = {}; - char *penv[] = {"TESTDIR"}; + char buf1[BSZ], buf2[BSZ] = {0}; (void) fprintf(stdout, "Verify O_TMPFILE is working properly.\n"); - /* - * Get the environment variable values. - */ - for (i = 0; i < sizeof (penv) / sizeof (char *); i++) { - if ((penv[i] = getenv(penv[i])) == NULL) { - (void) fprintf(stderr, "getenv(penv[%d])\n", i); - exit(1); - } - } + const char *testdir = getenv("TESTDIR"); + if (testdir == NULL) + errx(1, "getenv(\"TESTDIR\")"); fill_random(buf1, BSZ); - fd = open(penv[0], O_RDWR|O_TMPFILE, 0666); - if (fd < 0) { - perror("open"); - exit(2); - } - - if (write(fd, buf1, BSZ) < 0) { - perror("write"); - close(fd); - exit(3); - } - - if (pread(fd, buf2, BSZ, 0) < 0) { - perror("pread"); - close(fd); - exit(4); - } - - if (memcmp(buf1, buf2, BSZ) != 0) { - fprintf(stderr, "data corrupted\n"); - close(fd); - exit(5); - } + int fd = open(testdir, O_RDWR|O_TMPFILE, 0666); + if (fd < 0) + err(2, "open(%s)", testdir); + + if (write(fd, buf1, BSZ) < 0) + err(3, "write"); + + if (pread(fd, buf2, BSZ, 0) < 0) + err(4, "pread"); + + if (memcmp(buf1, buf2, BSZ) != 0) + errx(5, "data corrupted"); memset(buf2, 0, BSZ); - if (fsetxattr(fd, "user.test", buf1, BSZ, 0) < 0) { - perror("fsetxattr"); - close(fd); - exit(6); - } - - if (fgetxattr(fd, "user.test", buf2, BSZ) < 0) { - perror("fgetxattr"); - close(fd); - exit(7); - } - - if (memcmp(buf1, buf2, BSZ) != 0) { - fprintf(stderr, "xattr corrupted\n"); - close(fd); - exit(8); - } - - close(fd); + if (fsetxattr(fd, "user.test", buf1, BSZ, 0) < 0) + err(6, "pread"); + + if (fgetxattr(fd, "user.test", buf2, BSZ) < 0) + err(7, "fgetxattr"); + + if (memcmp(buf1, buf2, BSZ) != 0) + errx(8, "xattr corrupted\n"); return (0); } diff --git a/tests/zfs-tests/tests/functional/tmpfile/tmpfile_002_pos.c b/tests/zfs-tests/tests/functional/tmpfile/tmpfile_002_pos.c index 39a72c22ab84..424231d112b2 100644 --- a/tests/zfs-tests/tests/functional/tmpfile/tmpfile_002_pos.c +++ b/tests/zfs-tests/tests/functional/tmpfile/tmpfile_002_pos.c @@ -6,6 +6,7 @@ #include #include #include +#include /* backward compat in case it's not defined */ #ifndef O_TMPFILE @@ -24,12 +25,27 @@ * */ +static void +run(const char *op) +{ + int ret; + char buf[50]; + sprintf(buf, "sudo -E zpool %s $TESTPOOL", op); + if ((ret = system(buf)) != 0) { + if (ret == -1) + err(4, "system \"zpool %s\"", op); + else + errx(4, "zpool %s exited %d\n", + op, WEXITSTATUS(ret)); + } +} + int main(void) { - int i, fd, ret; + int i, fd; char spath[1024], dpath[1024]; - char *penv[] = {"TESTDIR", "TESTFILE0"}; + const char *penv[] = {"TESTDIR", "TESTFILE0"}; struct stat sbuf; (void) fprintf(stdout, "Verify O_TMPFILE file can be linked.\n"); @@ -37,55 +53,25 @@ main(void) /* * Get the environment variable values. */ - for (i = 0; i < sizeof (penv) / sizeof (char *); i++) { - if ((penv[i] = getenv(penv[i])) == NULL) { - (void) fprintf(stderr, "getenv(penv[%d])\n", i); - exit(1); - } - } + for (i = 0; i < ARRAY_SIZE(penv); i++) + if ((penv[i] = getenv(penv[i])) == NULL) + errx(1, "getenv(penv[%d])", i); fd = open(penv[0], O_RDWR|O_TMPFILE, 0666); - if (fd < 0) { - perror("open"); - exit(2); - } + if (fd < 0) + err(2, "open(%s)", penv[0]); snprintf(spath, 1024, "/proc/self/fd/%d", fd); snprintf(dpath, 1024, "%s/%s", penv[0], penv[1]); - if (linkat(AT_FDCWD, spath, AT_FDCWD, dpath, AT_SYMLINK_FOLLOW) < 0) { - perror("linkat"); - close(fd); - exit(3); - } + if (linkat(AT_FDCWD, spath, AT_FDCWD, dpath, AT_SYMLINK_FOLLOW) < 0) + err(3, "linkat"); - if ((ret = system("sudo -E zpool freeze $TESTPOOL"))) { - if (ret == -1) - perror("system \"zpool freeze\""); - else - fprintf(stderr, "zpool freeze exits with %d\n", - WEXITSTATUS(ret)); - exit(4); - } + run("freeze"); close(fd); - if ((ret = system("sudo -E zpool export $TESTPOOL"))) { - if (ret == -1) - perror("system \"zpool export\""); - else - fprintf(stderr, "zpool export exits with %d\n", - WEXITSTATUS(ret)); - exit(4); - } - - if ((ret = system("sudo -E zpool import $TESTPOOL"))) { - if (ret == -1) - perror("system \"zpool import\""); - else - fprintf(stderr, "zpool import exits with %d\n", - WEXITSTATUS(ret)); - exit(4); - } + run("export"); + run("import"); if (stat(dpath, &sbuf) < 0) { perror("stat"); diff --git a/tests/zfs-tests/tests/functional/tmpfile/tmpfile_003_pos.c b/tests/zfs-tests/tests/functional/tmpfile/tmpfile_003_pos.c index 58aa9051215e..463b96c992ff 100644 --- a/tests/zfs-tests/tests/functional/tmpfile/tmpfile_003_pos.c +++ b/tests/zfs-tests/tests/functional/tmpfile/tmpfile_003_pos.c @@ -6,6 +6,7 @@ #include #include #include +#include /* backward compat in case it's not defined */ #ifndef O_TMPFILE @@ -28,7 +29,7 @@ main(void) { int i, fd; char spath[1024], dpath[1024]; - char *penv[] = {"TESTDIR", "TESTFILE0"}; + const char *penv[] = {"TESTDIR", "TESTFILE0"}; struct stat sbuf; (void) fprintf(stdout, "Verify O_EXCL tmpfile cannot be linked.\n"); @@ -36,33 +37,21 @@ main(void) /* * Get the environment variable values. */ - for (i = 0; i < sizeof (penv) / sizeof (char *); i++) { - if ((penv[i] = getenv(penv[i])) == NULL) { - (void) fprintf(stderr, "getenv(penv[%d])\n", i); - exit(1); - } - } + for (i = 0; i < ARRAY_SIZE(penv); i++) + if ((penv[i] = getenv(penv[i])) == NULL) + errx(1, "getenv(penv[%d])", i); fd = open(penv[0], O_RDWR|O_TMPFILE|O_EXCL, 0666); - if (fd < 0) { - perror("open"); - exit(2); - } + if (fd < 0) + err(2, "open(%s)", penv[0]); snprintf(spath, 1024, "/proc/self/fd/%d", fd); snprintf(dpath, 1024, "%s/%s", penv[0], penv[1]); - if (linkat(AT_FDCWD, spath, AT_FDCWD, dpath, AT_SYMLINK_FOLLOW) == 0) { - fprintf(stderr, "linkat returns successfully\n"); - close(fd); - exit(3); - } + if (linkat(AT_FDCWD, spath, AT_FDCWD, dpath, AT_SYMLINK_FOLLOW) == 0) + errx(3, "linkat returned successfully\n"); - if (stat(dpath, &sbuf) == 0) { - fprintf(stderr, "stat returns successfully\n"); - close(fd); - exit(4); - } - close(fd); + if (stat(dpath, &sbuf) == 0) + errx(4, "stat returned successfully\n"); return (0); } diff --git a/tests/zfs-tests/tests/functional/tmpfile/tmpfile_stat_mode.c b/tests/zfs-tests/tests/functional/tmpfile/tmpfile_stat_mode.c index c72ea2bb6ad7..4c34aec8bdb4 100644 --- a/tests/zfs-tests/tests/functional/tmpfile/tmpfile_stat_mode.c +++ b/tests/zfs-tests/tests/functional/tmpfile/tmpfile_stat_mode.c @@ -6,7 +6,7 @@ * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE - * or http://www.opensolaris.org/os/licensing. + * or https://opensource.org/licenses/CDDL-1.0. * See the License for the specific language governing permissions * and limitations under the License. * @@ -28,6 +28,7 @@ #include #include #include +#include /* backward compat in case it's not defined */ #ifndef O_TMPFILE @@ -50,63 +51,46 @@ test_stat_mode(mode_t mask) struct stat st, fst; int i, fd; char spath[1024], dpath[1024]; - char *penv[] = {"TESTDIR", "TESTFILE0"}; + const char *penv[] = {"TESTDIR", "TESTFILE0"}; mode_t masked = 0777 & ~mask; mode_t mode; /* * Get the environment variable values. */ - for (i = 0; i < sizeof (penv) / sizeof (char *); i++) { - if ((penv[i] = getenv(penv[i])) == NULL) { - fprintf(stderr, "getenv(penv[%d])\n", i); - exit(1); - } - } + for (i = 0; i < ARRAY_SIZE(penv); i++) + if ((penv[i] = getenv(penv[i])) == NULL) + errx(1, "getenv(penv[%d])", i); umask(mask); fd = open(penv[0], O_RDWR|O_TMPFILE, 0777); - if (fd == -1) { - perror("open"); - exit(2); - } + if (fd == -1) + err(2, "open(%s)", penv[0]); - if (fstat(fd, &fst) == -1) { - perror("fstat"); - close(fd); - exit(3); - } + if (fstat(fd, &fst) == -1) + err(3, "open"); snprintf(spath, sizeof (spath), "/proc/self/fd/%d", fd); snprintf(dpath, sizeof (dpath), "%s/%s", penv[0], penv[1]); unlink(dpath); - if (linkat(AT_FDCWD, spath, AT_FDCWD, dpath, AT_SYMLINK_FOLLOW) == -1) { - perror("linkat"); - close(fd); - exit(4); - } + if (linkat(AT_FDCWD, spath, AT_FDCWD, dpath, AT_SYMLINK_FOLLOW) == -1) + err(4, "linkat"); close(fd); - if (stat(dpath, &st) == -1) { - perror("stat"); - exit(5); - } + if (stat(dpath, &st) == -1) + err(5, "stat"); unlink(dpath); /* Verify fstat(2) result */ mode = fst.st_mode & 0777; - if (mode != masked) { - fprintf(stderr, "fstat(2) %o != %o\n", mode, masked); - exit(6); - } + if (mode != masked) + errx(6, "fstat(2) %o != %o\n", mode, masked); /* Verify stat(2) result */ mode = st.st_mode & 0777; - if (mode != masked) { - fprintf(stderr, "stat(2) %o != %o\n", mode, masked); - exit(7); - } + if (mode != masked) + errx(7, "stat(2) %o != %o\n", mode, masked); } int diff --git a/tests/zfs-tests/tests/functional/trim/Makefile.am b/tests/zfs-tests/tests/functional/trim/Makefile.am deleted file mode 100644 index 8917ed726e90..000000000000 --- a/tests/zfs-tests/tests/functional/trim/Makefile.am +++ /dev/null @@ -1,12 +0,0 @@ -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/trim -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - trim.kshlib \ - trim.cfg \ - autotrim_integrity.ksh \ - autotrim_config.ksh \ - autotrim_trim_integrity.ksh \ - trim_integrity.ksh \ - trim_config.ksh \ - trim_l2arc.ksh diff --git a/tests/zfs-tests/tests/functional/trim/trim.cfg b/tests/zfs-tests/tests/functional/trim/trim.cfg index 91adb7624326..02dc77ab57e7 100644 --- a/tests/zfs-tests/tests/functional/trim/trim.cfg +++ b/tests/zfs-tests/tests/functional/trim/trim.cfg @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/trim/trim.kshlib b/tests/zfs-tests/tests/functional/trim/trim.kshlib index dc1a60a5ee9d..f36f3870fc4e 100644 --- a/tests/zfs-tests/tests/functional/trim/trim.kshlib +++ b/tests/zfs-tests/tests/functional/trim/trim.kshlib @@ -22,8 +22,7 @@ # function get_size_mb { - typeset rval=$(du --block-size 1048576 -s "$1" | awk '{print $1}') - echo -n "$rval" + du --block-size 1048576 -s "$1" | cut -f1 } # @@ -39,14 +38,12 @@ function get_trim_io # Sum the ind or agg columns of the trim request size histogram. case "$type" in "ind") - rval=$(zpool iostat -pr $pool $vdev | awk \ - '$1 ~ /[0-9].*/ { sum += $12 } END { print sum }') - echo -n "$rval" + zpool iostat -pr $pool $vdev | + awk '$1 ~ /[0-9].*/ { sum += $12 } END { print sum }' ;; "agg") - rval=$(zpool iostat -pr $pool $vdev | awk \ - '$1 ~ /[0-9].*/ { sum += $13 } END { print sum }') - echo -n "$rval" + zpool iostat -pr $pool $vdev | + awk '$1 ~ /[0-9].*/ { sum += $13 } END { print sum }' ;; *) log_fail "Type must be 'ind' or 'agg'" diff --git a/tests/zfs-tests/tests/functional/trim/trim_l2arc.ksh b/tests/zfs-tests/tests/functional/trim/trim_l2arc.ksh index ecf9f3424eb5..0bbd08acdd3f 100755 --- a/tests/zfs-tests/tests/functional/trim/trim_l2arc.ksh +++ b/tests/zfs-tests/tests/functional/trim/trim_l2arc.ksh @@ -36,6 +36,8 @@ verify_runnable "global" +command -v fio > /dev/null || log_unsupported "fio missing" + log_assert "Trim of L2ARC succeeds." function cleanup @@ -95,8 +97,8 @@ done verify_trim_io $TESTPOOL "ind" 5 $TRIM_VDEV2 -typeset cache_size=$(zpool list -vp | grep $TRIM_VDEV2 | awk '{print $2}') -typeset cache_alloc=$(zpool list -vp | grep $TRIM_VDEV2 | awk '{print $3}') +typeset cache_size cache_alloc _ +read -r _ cache_size cache_alloc _ < <(zpool list -vp | grep $TRIM_VDEV2) log_must test $cache_alloc -lt $cache_size diff --git a/tests/zfs-tests/tests/functional/truncate/.gitignore b/tests/zfs-tests/tests/functional/truncate/.gitignore deleted file mode 100644 index f28d93573c51..000000000000 --- a/tests/zfs-tests/tests/functional/truncate/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/truncate_test diff --git a/tests/zfs-tests/tests/functional/truncate/Makefile.am b/tests/zfs-tests/tests/functional/truncate/Makefile.am deleted file mode 100644 index b2d804b5d4c2..000000000000 --- a/tests/zfs-tests/tests/functional/truncate/Makefile.am +++ /dev/null @@ -1,18 +0,0 @@ -include $(top_srcdir)/config/Rules.am - -pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/truncate - -dist_pkgdata_SCRIPTS = \ - setup.ksh \ - cleanup.ksh \ - truncate_001_pos.ksh \ - truncate_002_pos.ksh \ - truncate_timestamps.ksh - -dist_pkgdata_DATA = \ - truncate.cfg - -pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/truncate - -pkgexec_PROGRAMS = truncate_test -truncate_test_SOURCES = truncate_test.c diff --git a/tests/zfs-tests/tests/functional/truncate/cleanup.ksh b/tests/zfs-tests/tests/functional/truncate/cleanup.ksh index 9756c0fab805..68b56287ed2b 100755 --- a/tests/zfs-tests/tests/functional/truncate/cleanup.ksh +++ b/tests/zfs-tests/tests/functional/truncate/cleanup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/truncate/setup.ksh b/tests/zfs-tests/tests/functional/truncate/setup.ksh index 863492db531d..570c94570ef9 100755 --- a/tests/zfs-tests/tests/functional/truncate/setup.ksh +++ b/tests/zfs-tests/tests/functional/truncate/setup.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/truncate/truncate.cfg b/tests/zfs-tests/tests/functional/truncate/truncate.cfg index 13cdafab133b..aca91cda9c64 100644 --- a/tests/zfs-tests/tests/functional/truncate/truncate.cfg +++ b/tests/zfs-tests/tests/functional/truncate/truncate.cfg @@ -6,7 +6,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # @@ -34,6 +34,6 @@ export TRUNC_FILEOFFSET=${TRUNC_FILEOFFSET-""} export TRUNC_COUNT=${TRUNC_COUNT-"16384"} # FILESIZE/BLKSIZE/8 export DISKSARRAY=$DISKS -export DISK_ARRAY_NUM=$(echo ${DISKS} | nawk '{print NF}') +export DISK_ARRAY_NUM=$(echo ${DISKS} | awk '{print NF}') set_device_dir diff --git a/tests/zfs-tests/tests/functional/truncate/truncate_001_pos.ksh b/tests/zfs-tests/tests/functional/truncate/truncate_001_pos.ksh index 1ae2d74e0d4e..19bf0343e32d 100755 --- a/tests/zfs-tests/tests/functional/truncate/truncate_001_pos.ksh +++ b/tests/zfs-tests/tests/functional/truncate/truncate_001_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/truncate/truncate_002_pos.ksh b/tests/zfs-tests/tests/functional/truncate/truncate_002_pos.ksh index cff740e5fbba..17cde7b698ba 100755 --- a/tests/zfs-tests/tests/functional/truncate/truncate_002_pos.ksh +++ b/tests/zfs-tests/tests/functional/truncate/truncate_002_pos.ksh @@ -7,7 +7,7 @@ # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE -# or http://www.opensolaris.org/os/licensing. +# or https://opensource.org/licenses/CDDL-1.0. # See the License for the specific language governing permissions # and limitations under the License. # diff --git a/tests/zfs-tests/tests/functional/truncate/truncate_timestamps.ksh b/tests/zfs-tests/tests/functional/truncate/truncate_timestamps.ksh index 27b28e82eb5b..3793178701d8 100755 --- a/tests/zfs-tests/tests/functional/truncate/truncate_timestamps.ksh +++ b/tests/zfs-tests/tests/functional/truncate/truncate_timestamps.ksh @@ -42,7 +42,7 @@ function verify_truncate #