Documentation site not indexed by search engines

[colinodell]

TL;DR: Content from https://kubectl.docs.kubernetes.io/ cannot be found using popular search engines.

I expected to be able to find appropriate documentation pages like https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/ when searching for things like kustomization reference on Google. However, I'm unable to find results for this site when I run searches like:

• https://www.google.com/search?q=kustomization+reference
• https://www.google.com/search?q=site%3Akubectl.docs.kubernetes.io
• https://www.bing.com/search?q=site%3Akubectl.docs.kubernetes.io (XML files only; no HTML results)

I believe this is happening because of this meta tag in the HTML:

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

It looks like this is only supposed to appear when HUGO_ENV != production:

cli-experimental/site/themes/docsy/layouts/partials/head.html

Lines 4 to 8 in 3445a58

	{{ if eq (getenv "HUGO_ENV") "production" }}
	<META NAME="ROBOTS" CONTENT="INDEX, FOLLOW">
	{{ else }}
	<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
	{{ end }}

So maybe the build environment is failing to set this env var correctly?

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[colinodell]

This issue is still present and worth fixing

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten