Skip bad DVAs during free by setting zfs_recover=1

When a bad DVA is encountered in metaslab_free_dva() the system
should treat it as fatal.  This indicates that somehow a damaged
DVA was written to disk and that should be impossible.

However, we have seen a handful of reports over the years of pools
somehow being damaged in this way.  Since this damage can render
otherwise intact pools unimportable, and the consequence of skipping
the bad DVA is only leaked free space, it makes sense to provide
a mechanism to ignore the bad DVA.  Setting the zfs_recover=1 module
option will cause the DVA to be ignored which may allow the pool to
be imported.

Since zfs_recover=0 by default any pool attempting to free a bad DVA
will treat it as a fatal error preserving the current behavior.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3099
Issue #3090
Issue #2720

module/zfs/metaslab.c

@@ -2426,16 +2426,14 @@ metaslab_free_dva(spa_t *spa, const dva_t *dva, uint64_t txg, boolean_t now)
 	vdev_t *vd;
 	metaslab_t *msp;
 
-	ASSERT(DVA_IS_VALID(dva));
-
 	if (txg > spa_freeze_txg(spa))
 		return;
 
-	if ((vd = vdev_lookup_top(spa, vdev)) == NULL ||
+	if ((vd = vdev_lookup_top(spa, vdev)) == NULL || !DVA_IS_VALID(dva) ||
 	    (offset >> vd->vdev_ms_shift) >= vd->vdev_ms_count) {
-		cmn_err(CE_WARN, "metaslab_free_dva(): bad DVA %llu:%llu",
-		    (u_longlong_t)vdev, (u_longlong_t)offset);
-		ASSERT(0);
+		zfs_panic_recover("metaslab_free_dva(): bad DVA %llu:%llu:%llu",
+		    (u_longlong_t)vdev, (u_longlong_t)offset,
+		    (u_longlong_t)size);
 		return;
 	}