Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] Streamline beats getting started guides #17007

Merged
merged 42 commits into from
Jun 18, 2020

Conversation

dedemorton
Copy link
Contributor

@dedemorton dedemorton commented Mar 14, 2020

This PR moves extraneous content out of the Beats "getting started" topics and adds a tabbed widget for displaying platform-specific info (see NOTES below).

Preview: https://beats_17007.docs-preview.app.elstc.co/diff

This is a very big PR. To make it easier for you, I've collected a list of topics that you should view in the HTML preview (rather than the diff). Each team should review topics for the areas they own. Also look at Files Changed for other small changes that affect your area.

Rules of engagement:

  • Please check off the item below and add your initials when you are done reviewing the topic. You can skip reviewing files that have Javascript and HTML--we have someone lined up to help us improve the code.
  • I still think that some of our command examples that show sudo could be wrong. Please add a comment if you something that's wrong.

AUDITBEAT

FILEBEAT

HEARTBEAT

JOURNALBEAT

METRICBEAT

PACKETBEAT

WINLOGBEAT

FUNCTIONBEAT

SHARED TOPICS

NOTES:

This PR adds a new widget for showing platform-specific content. It is a prototype and up for discussion. The widget is based on some code that @bmorelli25 came up with for an APM prototype plus some CSS changes that I made. Still needs work.

image

@dedemorton dedemorton self-assigned this Mar 14, 2020
@dedemorton dedemorton changed the title [WIP][docs] Streamline beats getting started guildes [WIP][docs] Streamline beats getting started guides Mar 14, 2020
@dedemorton dedemorton marked this pull request as ready for review April 23, 2020 01:12
@dedemorton dedemorton added the in progress Pull request is currently in progress. label Apr 23, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented May 7, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

Test stats 🧪

Test Results
Failed 0
Passed 9214
Skipped 1542
Total 10756

@bmorelli25
Copy link
Member

This is fantastic! You've drastically improved my very simple prototype, and turned it into something that we can actually use!

I pushed some CSS changes. A few small changes, and one big change that removes the red border highlighting.

Unfortunately, I was unable to make tab changes apply to all tab boxes on the page. It appears to be a bit beyond my skill level.

@bmorelli25
Copy link
Member

I wonder if we could get a front-end dev to look at the CSS/HTML/JS. I'm sure there are still a lot of improvements that can be made, and they might even be able to help us sync the tabs.

@dedemorton
Copy link
Contributor Author

dedemorton commented May 11, 2020

Adding a to-do list because there are a lot of moving parts to track:

To-do list:

  • Make the getting started (GS) guide a single topic that describes the easiest getting started experience (for Filebeat, Auditbeat, and Metricbeat, this should be modules).
    • Packetbeat
    • Filebeat
    • Winlogbeat
    • Metricbeat
    • Heartbeat
    • Auditbeat
    • Functionbeat
    • Journalbeat
  • Check the shared commands against the original version.
  • Make sure Logstash is not part of the main getting started flow.
    • Winlogbeat
    • All other Beats.
  • Move content that doesn't belong in the GS the how-to section:
    • Configure the input manually
    • Changing index name
    • Loading index templates
  • Simplify conditional coding and content reuse, where feasible (focus on changed files)
  • Delete extraneous files when all references are cleaned up:
    • shared/download-and-install.asciidoc
    • step-configure-credentials.asciidoc
    • step-configure-kibana-endpoint.asciidoc
    • step-condigure-output.asciidoc
    • shared-template-load.asciidoc
    • shared-configuring.asciidoc
    • shared-getting-started-intro.asciidoc
  • Consolidate duplicated content:
    • In Filebeat docs, delete the modules quick start (since the new getting started will cover modules) and move any remaining (non-GS content) to the topic about running modules:
    • Load ingest pipelines manually.
  • Update dashboards because they are really old.
  • Make changes to improve SEO for getting started topic (based on input from Anne B):
    • Rename topics to Beatname quick start: installation and configuration
    • Change url to beatname-installation-configuration.asciidoc
    • Remove short titles from intro topics to make context clearer.

Work that requires Javascript/CSS help

  • Add widget for displaying multi-platform commands:
    • Add Javascript that synchronizes tabs so that user won't have to keep changing the OS [need dev help]

Copy link

@EamonnTP EamonnTP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dedemorton @bmorelli25 LGTM My only comment relates to the naming of the titles. Should we use "quick start" in the titles? Some users, especially if they choose to self-manage the stack, may not find the process that quick. And also, the title "Auditbeat quick start: installation and configuration" seems a little lengthy. Maybe to be consistent with the other headings, such as "Set up and run", we could go with "Auditbeat: install and configure" or just "Install and configure"?

@dedemorton
Copy link
Contributor Author

dedemorton commented Jun 15, 2020

@EamonnTP Thanks for your feedback

@dedemorton @bmorelli25 LGTM My only comment relates to the naming of the titles. Should we use "quick start" in the titles? Some users, especially if they choose to self-manage the stack, may not find the process that quick. And also, the title "Auditbeat quick start: installation and configuration" seems a little lengthy. Maybe to be consistent with the other headings, such as "Set up and run", we could go with "Auditbeat: install and configure" or just "Install and configure"?

Re titles: I'll check google analytics (and look back at my notes from my meeting with Anne B). I came up with that title after meeting with Anne B to discuss the best title + URL to use for SEO. "Get started" didn't rank very highly, so we decided to drop it. I can't remember if "quick start" was necessary. I do think it's good to have some kind of signpost to tell users "start here".

Copy link
Contributor

@sayden sayden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incredible work!

@elasticmachine
Copy link
Collaborator

elasticmachine commented Jun 17, 2020

❕ Build Aborted

There is a new build on-going so the previous on-going builds have been aborted.

Pipeline View Test View Changes Artifacts

Expand to view the summary

Build stats

  • Build Cause: [Pull request #17007 updated]

  • Reason: Aborted from #25

  • Start Time: 2020-06-18T20:41:07.631+0000

  • Duration: 32 min 9 sec

  • Commit: 0827608

Test stats 🧪

Test Results
Failed 0
Passed 773
Skipped 156
Total 929

Steps errors

Expand to view the steps failures

  • Name: Error signal

    • Description: untar: step failed with error null

    • Duration: 0 min 0 sec

    • Start Time: 2020-06-18T21:05:30.345+0000

    • log

  • Name: Notifies GitHub of the status of a Pull Request

    • Description: untar: step failed with error null

    • Duration: 0 min 1 sec

    • Start Time: 2020-06-18T21:05:34.214+0000

    • log

Log output

Expand to view the last 100 lines of log output

[2020-06-18T21:12:33.936Z] [success] 0.05% test_keystore.TestKeystore.test_keystore_with_key_not_present: 0.0992s
[2020-06-18T21:12:33.936Z] [success] 0.00% test_modules.load_fileset_test_cases: 0.0054s
[2020-06-18T21:12:33.936Z] ----------------------------------------------------------------------
[2020-06-18T21:12:33.936Z] Ran 314 tests in 204.287s
[2020-06-18T21:12:33.936Z] 
[2020-06-18T21:12:33.936Z] OK (SKIP=132)
[2020-06-18T21:12:33.936Z] >> python test: Unit Testing Complete
[2020-06-18T21:12:34.221Z] Recording test results
[2020-06-18T21:12:35.479Z] Stashed 2 file(s)
[2020-06-18T21:12:35.541Z] Archiving artifacts
[2020-06-18T21:12:37.067Z] + curl -sSLo codecov https://codecov.io/bash
[2020-06-18T21:12:37.067Z] + for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat
[2020-06-18T21:12:37.067Z] + FILE=auditbeat/build/coverage/full.cov
[2020-06-18T21:12:37.067Z] + '[' -f auditbeat/build/coverage/full.cov ']'
[2020-06-18T21:12:37.067Z] + for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat
[2020-06-18T21:12:37.067Z] + FILE=filebeat/build/coverage/full.cov
[2020-06-18T21:12:37.067Z] + '[' -f filebeat/build/coverage/full.cov ']'
[2020-06-18T21:12:37.067Z] + for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat
[2020-06-18T21:12:37.067Z] + FILE=heartbeat/build/coverage/full.cov
[2020-06-18T21:12:37.067Z] + '[' -f heartbeat/build/coverage/full.cov ']'
[2020-06-18T21:12:37.067Z] + for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat
[2020-06-18T21:12:37.067Z] + FILE=libbeat/build/coverage/full.cov
[2020-06-18T21:12:37.067Z] + '[' -f libbeat/build/coverage/full.cov ']'
[2020-06-18T21:12:37.067Z] + for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat
[2020-06-18T21:12:37.067Z] + FILE=metricbeat/build/coverage/full.cov
[2020-06-18T21:12:37.067Z] + '[' -f metricbeat/build/coverage/full.cov ']'
[2020-06-18T21:12:37.067Z] + for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat
[2020-06-18T21:12:37.067Z] + FILE=packetbeat/build/coverage/full.cov
[2020-06-18T21:12:37.067Z] + '[' -f packetbeat/build/coverage/full.cov ']'
[2020-06-18T21:12:37.067Z] + for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat
[2020-06-18T21:12:37.067Z] + FILE=winlogbeat/build/coverage/full.cov
[2020-06-18T21:12:37.067Z] + '[' -f winlogbeat/build/coverage/full.cov ']'
[2020-06-18T21:12:37.067Z] + for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat
[2020-06-18T21:12:37.067Z] + FILE=journalbeat/build/coverage/full.cov
[2020-06-18T21:12:37.067Z] + '[' -f journalbeat/build/coverage/full.cov ']'
[2020-06-18T21:12:38.428Z] Post stage
[2020-06-18T21:12:38.436Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-17007/src/github.com/elastic/beats
[2020-06-18T21:12:39.385Z] Starting "default"...
[2020-06-18T21:12:39.385Z] Machine "default" is already running.
[2020-06-18T21:12:41.620Z] Error checking TLS connection: Error checking and/or regenerating the certs: There was an error validating certificates for host "192.168.99.101:2376": dial tcp 192.168.99.101:2376: connect: connection refused
[2020-06-18T21:12:41.620Z] You can attempt to regenerate them using 'docker-machine regenerate-certs [name]'.
[2020-06-18T21:12:41.620Z] Be advised that this will trigger a Docker daemon restart which might stop running containers.
[2020-06-18T21:12:41.620Z] 
[2020-06-18T21:12:41.620Z] Client: Docker Engine - Community
[2020-06-18T21:12:41.620Z]  Version:           19.03.1
[2020-06-18T21:12:41.620Z]  API version:       1.40
[2020-06-18T21:12:41.620Z]  Go version:        go1.12.5
[2020-06-18T21:12:41.620Z]  Git commit:        74b1e89
[2020-06-18T21:12:41.620Z]  Built:             Thu Jul 25 21:18:17 2019
[2020-06-18T21:12:41.620Z]  OS/Arch:           darwin/amd64
[2020-06-18T21:12:41.620Z]  Experimental:      false
[2020-06-18T21:12:41.621Z] Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
[2020-06-18T21:12:41.621Z] It requires Docker daemon to be installed and running
[2020-06-18T21:12:53.753Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-17007/src/github.com/elastic/beats
[2020-06-18T21:12:54.059Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-06-18T21:12:54.070Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-17007/src/github.com/elastic/beats/Lint
[2020-06-18T21:12:54.152Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-17007/src/github.com/elastic/beats/Auditbeat-oss-Mac-OS-X
[2020-06-18T21:12:54.223Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-17007/src/github.com/elastic/beats/Auditbeat-x-pack-Mac-OS-X
[2020-06-18T21:12:54.295Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-17007/src/github.com/elastic/beats/Filebeat-Mac-OS-X
[2020-06-18T21:12:54.651Z] + cat
[2020-06-18T21:12:54.651Z] + /usr/local/bin/runbld ./runbld-script
[2020-06-18T21:12:54.651Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-06-18T21:13:01.261Z] runbld>>> runbld started
[2020-06-18T21:13:01.261Z] runbld>>> 1.6.12/f45d832f2ba0aa2722ab4ec1fda8ad140f027f8b
[2020-06-18T21:13:03.188Z] runbld>>> The following profiles matched the job 'Beats/beats-beats-mbp/PR-17007' in order of occurrence in the config (last value wins).
[2020-06-18T21:13:04.577Z] runbld>>> Debug logging enabled.
[2020-06-18T21:13:04.577Z] runbld>>> Storing result
[2020-06-18T21:13:04.839Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-06-18T21:13:04.839Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200618211304-412DE8B4
[2020-06-18T21:13:04.839Z] runbld>>> Adding system facts.
[2020-06-18T21:13:05.790Z] runbld>>> Adding vcs info for the latest commit:  58db5bcb79e44ef27abb67079ebed5a66c1e1da4
[2020-06-18T21:13:05.790Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-06-18T21:13:05.790Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-06-18T21:13:06.052Z] + echo 'Processing JUnit reports with runbld...'
[2020-06-18T21:13:06.052Z] Processing JUnit reports with runbld...
[2020-06-18T21:13:06.314Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-06-18T21:13:06.314Z] runbld>>> DURATION: 26ms
[2020-06-18T21:13:06.314Z] runbld>>> STDOUT: 40 bytes
[2020-06-18T21:13:06.314Z] runbld>>> STDERR: 49 bytes
[2020-06-18T21:13:06.314Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-06-18T21:13:06.314Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-17007/src/github.com/elastic/beats
[2020-06-18T21:13:08.232Z] runbld>>> Storing build metadata: 
[2020-06-18T21:13:08.232Z] runbld>>> Adding test report.
[2020-06-18T21:13:08.232Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-17007/src/github.com/elastic/beats
[2020-06-18T21:13:09.174Z] runbld>>> Found 6 test output files
[2020-06-18T21:13:09.748Z] runbld>>> Test output logs contained: Errors: 0 Failures: 0 Tests: 929 Skipped: 153
[2020-06-18T21:13:09.748Z] runbld>>> Storing result
[2020-06-18T21:13:09.748Z] runbld>>> FAILURES: 0
[2020-06-18T21:13:10.010Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-06-18T21:13:10.010Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200618211304-412DE8B4
[2020-06-18T21:13:10.272Z] runbld>>> Email notification disabled by environment variable.
[2020-06-18T21:13:10.272Z] runbld>>> Slack notification disabled by environment variable.
[2020-06-18T21:13:16.026Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-17007
[2020-06-18T21:13:16.249Z] [INFO] getVaultSecret: Getting secrets
[2020-06-18T21:13:16.303Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-06-18T21:13:17.064Z] + chmod 755 generate-build-data.sh
[2020-06-18T21:13:17.064Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-17007/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-17007/runs/24 ABORTED 1929174
[2020-06-18T21:13:17.064Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-17007/runs/24/steps/?limit=10000 -o steps-info.json
[2020-06-18T21:13:17.975Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-17007/runs/24/tests/?status=FAILED -o tests-errors.json
[2020-06-18T21:13:18.225Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-17007/runs/24/log/ -o pipeline-log.txt

Copy link
Contributor

@andresrc andresrc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@dedemorton dedemorton merged commit c5a4d92 into elastic:master Jun 18, 2020
@dedemorton dedemorton deleted the issue#58898 branch June 18, 2020 21:15
dedemorton added a commit to dedemorton/beats that referenced this pull request Jun 18, 2020
* Move detail out of getting started guides

* Changes in progress

* Add widgets

* Change tag width to 100%

* remove unused css and red border

* breathing room below tab content

* Remove inexplicable tab

Co-authored-by: Brandon Morelli <bmorelli25@gmail.com>

* Clean up wording

* Cleanup the getting started and move some stuff

* Git auditbeat building

* Get all books building

* More small changes

* Fix build errors

* Finish adding widgets and cleaning up gs guides

* Fix widget filenames

* More cleanup work

* Add keystore tip to cloud example

* Another round of improvements

* Delete redundant files

* Replace dashboards in quick start topics

* Change short titles for quick start guides

* Make shared what's next list easier to include

* Remove modules quick start

* More cleanup

* Remove redundant files

* Remove Logstash from the Winlogbeat quick start

* Remove old comments

* More cleanup work

* Delete unused graphics

* Use correct spelling for GitHub

Co-authored-by: Brandon Morelli <bmorelli25@gmail.com>

* Add formatting

Co-authored-by: Brandon Morelli <bmorelli25@gmail.com>

* Use correct capitalization for GitHub

Co-authored-by: Brandon Morelli <bmorelli25@gmail.com>

* Use time filter instead of date range

Co-authored-by: Brandon Morelli <bmorelli25@gmail.com>

* Change wording of headings as suggested during review

* Add more fixes from the review

* Synchronize widget tab selection and fix bugs

* Include javascript only once in a file

* Add script and stylesheet to support tabbed panel

* Fix step numbering

* Include code for displaying tabbed panel

* Add javascript and css for tabbed panel

* Fix code include

Co-authored-by: Brandon Morelli <brandon.morelli@elastic.co>
Co-authored-by: Brandon Morelli <bmorelli25@gmail.com>
@dedemorton dedemorton removed the needs_backport PR is waiting to be backported to other branches. label Jun 27, 2020
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020
* Move detail out of getting started guides

* Changes in progress

* Add widgets

* Change tag width to 100%

* remove unused css and red border

* breathing room below tab content

* Remove inexplicable tab

Co-authored-by: Brandon Morelli <bmorelli25@gmail.com>

* Clean up wording

* Cleanup the getting started and move some stuff

* Git auditbeat building

* Get all books building

* More small changes

* Fix build errors

* Finish adding widgets and cleaning up gs guides

* Fix widget filenames

* More cleanup work

* Add keystore tip to cloud example

* Another round of improvements

* Delete redundant files

* Replace dashboards in quick start topics

* Change short titles for quick start guides

* Make shared what's next list easier to include

* Remove modules quick start

* More cleanup

* Remove redundant files

* Remove Logstash from the Winlogbeat quick start

* Remove old comments

* More cleanup work

* Delete unused graphics

* Use correct spelling for GitHub

Co-authored-by: Brandon Morelli <bmorelli25@gmail.com>

* Add formatting

Co-authored-by: Brandon Morelli <bmorelli25@gmail.com>

* Use correct capitalization for GitHub

Co-authored-by: Brandon Morelli <bmorelli25@gmail.com>

* Use time filter instead of date range

Co-authored-by: Brandon Morelli <bmorelli25@gmail.com>

* Change wording of headings as suggested during review

* Add more fixes from the review

* Synchronize widget tab selection and fix bugs

* Include javascript only once in a file

* Add script and stylesheet to support tabbed panel

* Fix step numbering

* Include code for displaying tabbed panel

* Add javascript and css for tabbed panel

* Fix code include

Co-authored-by: Brandon Morelli <brandon.morelli@elastic.co>
Co-authored-by: Brandon Morelli <bmorelli25@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs review Team:Integrations Label for the Integrations team Team:obs-ds-hosted-services Label for the Observability Hosted Services team v7.9.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants