SIG-Docs: Documentation User Survey 2019

[aimeeu]

This is a Feature Request

What would you like to be added

Put together a survey asking (random) site visitors for their pain points.

The CNCF has a Survey Monkey account. We could work with the owner to create a survey, which then could be linked to from the Docs home page. The link could also go out via social media and Google groups. We would have to be very careful how we word the questions and what the answer choices are so we collect meaningful data. What if we get 5K responses? Nobody is going to slog through 5K free form answers.

Why is this needed
Results would be useful for the Sept 13 board meeting and for project planning.

Comments
The questions below are a simple starting point. Please add your ideas.

1. How often do you access the Kubernetes documentation? (single select)
   • Every day
   • Once a week…
2. How do you usually access the Kubernetes documentation site? (single select)
   • Mobile phone
   • Tablet
   • Desktop web browser
3. Which sections do you access most often? (multi-select)
   • Getting Started
   • Concepts
   • Tasks
   • Tutorials
   • Reference
   • Contribute
4. Which sections need more detailed content?
   • Getting Started
   • Concepts
   • Tasks
   • Tutorials
   • Reference
   • Contribute

Question 4 could be broken down into several questions, with each question focusing on a section. That would be quite detailed and we might at that point lose end users taking the survey. We could focus on sections perceived as lacking detailed content: concepts, tasks, contributor guides

Also take a look at Brad's StackOverflow filter

[sftim]

I would ask people to score a list of topics by how well the documentation explains them.
Eg:

• Deploying a cluster for learning
• Deploying a cluster for production use
• Securing your cluster
• Kubernetes architecture
• Running workloads on Kubernetes
• Container networking
• Services & load balancing
• Stateful workloads & storage
• Observability & logging
• Kubernetes terminology

If doing this, it might be an idea to take the topic list as the union of the topics for the CKA and CKAD exams(?)

[aimeeu]

/assign

[aimeeu]

CKA/CKAD curriculum here

Combined Topic List (CKA/CKAD)

• Application Lifestyle Management (includes CKAD Configuration)
  • Understand Deployments and how to perform rolling updates and rollbacks
  • Know various ways to configure applications
  • Know how to scale applications
  • Understand the primitives neccesary to create a self-healing application
  • Understand Config Maps
  • Understand Security Contexts
  • Define an application’s resource requirements
  • Create & consume Secrets
  • Understand Service Accounts
• Cluster
  • Understand k8s cluster upgrade process
  • Facilitate OS upgrades
  • Implement backup and restore methodologies
• Core Concepts
  • Understand the Kubernetes API primitives.
  • Understand the Kubernetes cluster architecture.
  • Understand Services and other network primitives
  • Create and configure basic pods
• Installation, Configuration, & Validation
  • Design a Kubernetes cluster.
  • Install Kubernetes masters and nodes.
  • Configure secure cluster communications.
  • Configure a Highly-Available Kubernetes cluster.
  • Know where to get Kubernetes release binaries
  • Provision underlying infrastructure to deploy a Kubernetes cluster.
  • Choose a network solution.
  • Choose your Kubernetes infrastructure configuration.
  • Run end-to-end tests on your cluster.
  • Run Node end-to-end tests.
  • Install and use kubeadm to install,configure, and manage Kubernetes clusters.
• Logging/Monitoring (includes CKAD Observability)
  • Understand how to monitor all cluster components.
  • Understand how to monitor applications.
  • Manage cluster component logs.
  • Manage application logs.
  • Understand LivenessProbes and ReadinessProbes
  • Understand container logging
  • Understand how to monitor applications in Kubernetes
  • Understand debugging in Kubernetes
• Multi-Container Pods (design patterns, eg ambassador, adapter, sidecar)
  • Understand Multi-Container Pod design patterns (e.g. ambassador, adapter, sidecar)
• Pod Design (combine this with ALM??)
  • Understand how to use Labels, Selectors, and Annotations
  • Understand Deployments and how to perform rolling updates
  • Understand Deployments and how to perform rollbacks
  • Understand Jobs and CronJobs
• Networking (includes CKAD Services and Networking)
  • Understand the networking configuration on the cluster nodes.
  • Understand Pod networking concepts.
  • Understand service networking.
  • Deploy and configure network load balancer.
  • Know how to use Ingress rules.
  • Know how to configure and use the cluster DNS
  • Understand CNI
  • Understand Services
  • Demonstrate basic understanding of Network Policies
• Scheduling
  • Use label selectors to schedule Pods.
  • Understand the role of DaemonSets.
  • Understand how resource limits can affect Pod scheduling.
  • Understand how to run multiple schedulers and how to configure Pods to use them
  • Manually schedule a Pod without a scheduler.
  • Display scheduler events.
  • Know how to configure the Kubernetes scheduler
• Security
  • Know how to configure authentication and authorization.
  • Understand Kubernetes security primitives.
  • Know to configure network policies
  • Create and manage TLS certificates for cluster components
  • Work with images securely.
  • Define security contexts.
  • Secure persistent key-value store.
• Storage (includes CKAD State Persistence)
  • Understand persistent volumes and know how to create them.
  • Understand access modes for volumes.
  • Understand persistent volume claims primitive.
  • Understand Kubernetes storage objects.
  • Know how to configure applications with persistent storage
  • Understand Persistent Volume Claims for storage
• Troubleshooting
  • Troubleshoot application failure
  • Troubleshoot control plane failure
  • Troubleshoot worker node failure
  • Troubleshoot networking

[aimeeu]

Stats for k8s.io are published monthly to kubernetes-sig-docs@googlegroups.com. Most recent stats: https://groups.google.com/forum/#!topic/kubernetes-sig-docs/Wllo-nmklg4

Top page hits for July:

1. /docs/reference/kubectl/cheatsheet/
2. /docs/home
3. /docs/tasks/tools/install-kubectl/ - no direct link on home page
4. /docs/concepts/services-networking/service/ - no direct link on home page
5. /docs/tutorials/kubernetes-basics/ - no direct link on home page
6. /docs/concepts/workloads/controllers/deployment/ - no direct link from home page
7. /docs/concepts/overview/what-is-kubernetes/
8. /docs/tasks/tools/install-minikube/ - no direct link on home page; minikube installation docs shouldn't be on k8s site but rather link to minikube site
9. /docs/concepts/services-networking/ingress/ - no direct link on home page

And the StackOverflow kubernetes questions with the most votes: https://stackoverflow.com/questions/tagged/kubernetes?sort=MostVotes&edited=true

[zacharysarah]

@sftim The more I consider it, the less on board I am with asking site visitors to rank a list of topics.

1. It's a steep cognitive load/time investment to ask from a random site visitor.

2. We need to measure only what we intend to act on.

@aimeeu, I think your original proposal is solid and encourage you to proceed with it.

[sftim]

OK, cool. That makes a lot of sense.

[aimeeu]

@zacharysarah thanks for the "measure" article link. One actionable item I see already is modifying the Docs home page to include links to top pages so users don't have to dig. I will ping Jared about getting stats for more than just the top 10 pages.

[aimeeu]

Demographics

1. What is your level of experience using Kubernetes?
   • Beginner
   • Experienced
   • Expert
2. How do you use Kubernetes?
   • Developer
   • Administrator
   • Other
3. How do you usually access the Kubernetes documentation site? (single select)
   • Phone
   • Tablet
   • Desktop
4. How long have you been using the Kubernetes documentation?
   • Less than 6 months
   • 6-12 months
   • More than 12 months
5. In which languages do you read the Kubernetes documentation?
   • German (de)
   • English (en)
   • Spanish (es)
   • French (fr)
   • Japanese (ja)
   • Korean (ko)
   • Portuguese (pt)
   • Chinese (zh)
     (Hindi, Italian, and Norwegian are disabled in config.toml).

Content

1. Is the Kubernetes documentation the first place you look for information about Kubernetes?
   • Yes
   • No
2. How easy is it to find content using the site’s navigation (top and left)?
   • 1 (not easy)
   • 2
   • 3
   • 4
   • 5 (very easy)
3. How often do you use the search functionality to find content?
   • 1 (rarely)
   • 2
   • 3
   • 4
   • 5 (every visit)
4. Why do you access the Kubernetes documentation? Check all that apply.
   • Installation for learning
   • Installation for production
   • Architecture
   • Concepts
   • Terminology (glossary)
   • API reference
   • kubectl CLI reference
   • Tasks (administering a cluster, configuring pods, securing Kubernetes, upgrading, etc)
   • Troubleshooting guidance
   • Tutorials
5. How often do you find what you are looking for?
   • 1 (rarely)
   • 2
   • 3
   • 4
   • 5 (every visit)
6. Concepts: How satisfied are you with the level of detail in the Concepts section?
   • 1 (not satisfied)
   • 2
   • 3
   • 4
   • 5 (very satisfied)
7. Concepts: How often do you find outdated content in the Concepts section?
   • 1 (rarely)
   • 2
   • 3
   • 4
   • 5 (every visit)
8. Concepts: How can the Concepts section be improved?
   • More diagrams
   • More example code
   • More detailed content
   • Update outdated FEATURE STATE
9. Tasks: How satisfied are you with the level of detail in the Tasks section?
   • 1 (not satisfied)
   • 2
   • 3
   • 4
   • 5 (very satisfied)
10. Tasks: How often do you find outdated content in the Tasks section?
    • 1 (rarely)
    • 2
    • 3
    • 4
    • 5 (every visit)
11. Tasks: How can the Tasks section be improved?
    • More diagrams
    • More example code
    • More detailed content
12. What other areas do you use for Kubernetes documentation? (this question is poorly worded)
    - Kubernetes Slack
    - Kubernetes mailing lists
    - Stack Overflow
    - Kubernetes forum (https://discuss.kubernetes.io/)
    - Internet search engine
    - Books
    - Blogs

[zacharysarah]

@aimeeu I recommend leaving off the last question, otherwise this content LGTM. 👍

[aimeeu]

Remove #12 - adding questions on Reference and Tutorials sections

12. Reference: How satisfied are you with the level of detail in the Reference section?
    • 1 (not satisfied)
    • 2
    • 3
    • 4
    • 5 (very satisfied)
13. Reference: How often do you find outdated content in the Reference section?
    • 1 (rarely)
    • 2
    • 3
    • 4
    • 5 (every visit)
14. Reference: How can the Reference section be improved?
    • More example code
    • More detailed content
15. Tutorials: How satisfied are you with the level of detail in the Tutorials section?
    • 1 (not satisfied)
    • 2
    • 3
    • 4
    • 5 (very satisfied)
16. Tutorials: How often do you find outdated content in the Tutorials section?
    • 1 (rarely)
    • 2
    • 3
    • 4
    • 5 (every visit)
17. Tutorials: How can the Tutorials section be improved?
    • Advanced tutorials (security, networking, configuration, troubleshooting, etc)
    • Developer-oriented content
18. How can the Kubernetes documentation be improved? (free form answer - yikes!)

[aimeeu]

Some feedback for next year: it might be a good idea to add a question that's related to something like "how often do you use the docs (daily/weekly/monthly)" & "whens the last time you used the docs" in case there was a big doc push that more experienced people might not be aware of that could skew the answers (feedback from Mike Johnson via Jim Angel)

[sftim]

/priority important-soon

Is this OK to close?

[aimeeu]

/close

[k8s-ci-robot]

@aimeeu: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.