fix: support directory listings even if a 404 page is present

fixes ipfs#4233 (comment)

Basically, there's a trade-off here:

1. We can support directory listings while supporting 404 pages (this PR).
2. If a 404 page is present, directory listings don't work.

Given that option 1 is more flexible and users shouldn't be _too_ confused if
they land on a directory with no index.html page, I've gone with that option.

core/corehttp/gateway_handler.go

@@ -342,10 +342,6 @@ func (i *gatewayHandler) getOrHeadHandler(w http.ResponseWriter, r *http.Request
 		return
 	}
 
-	if i.servePretty404IfPresent(w, r, parsedPath) {
-		return
-	}
-
 	// storage for directory listing
 	var dirListing []directoryItem
 	dirit := dir.Entries()