Don't activate metaslabs with weight 0

We return ENOSPC in metaslab_activate if the metaslab has weight 0, 
to avoid activating a metaslab with no space available.  For sanity 
checking, we also assert that there is no free space in the range 
tree in that case.

Reviewed-by: Igor Kozhukhov <igor@dilos.org>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes openzfs#8968

module/zfs/metaslab.c

@@ -2485,6 +2485,18 @@ metaslab_activate(metaslab_t *msp, int allocator, uint64_t activation_weight)
 		return (0);
 	}
 
+	/*
+	 * If the metaslab has literally 0 space, it will have weight 0. In
+	 * that case, don't bother activating it. This can happen if the
+	 * metaslab had space during find_valid_metaslab, but another thread
+	 * loaded it and used all that space while we were waiting to grab the
+	 * lock.
+	 */
+	if (msp->ms_weight == 0) {
+		ASSERT0(range_tree_space(msp->ms_allocatable));
+		return (SET_ERROR(ENOSPC));
+	}
+
 	if ((error = metaslab_activate_allocator(msp->ms_group, msp,
 	    allocator, activation_weight)) != 0) {
 		return (error);
@@ -3735,8 +3747,8 @@ metaslab_group_alloc_normal(metaslab_group_t *mg, zio_alloc_list_t *zal,
 		 * worse metaslab (we waited for that metaslab to be loaded
 		 * after all).
 		 *
-		 * If the activation failed due to an I/O error we skip to
-		 * the next metaslab.
+		 * If the activation failed due to an I/O error or ENOSPC we
+		 * skip to the next metaslab.
 		 */
 		boolean_t activated;
 		if (activation_error == 0) {