-
Notifications
You must be signed in to change notification settings - Fork 522
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug] Adjust build-release
CLI and fix links when generating security docs
#3434
[Bug] Adjust build-release
CLI and fix links when generating security docs
#3434
Conversation
@@ -71,6 +71,7 @@ jobs: | |||
token: ${{ secrets.READ_WRITE_RELEASE_FLEET }} | |||
repository: ${{github.event.inputs.target_repo}} | |||
path: integrations | |||
fetch-depth: 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Had an issue with checking out backport branches, so pulling all history.
Tested: elastic/integrations#9114
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In general looks good.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you give this a shot?
…:elastic/detection-rules into bug-security-doc-generation-broken-links
…ty docs (#3434) * removed historical argument; added setup string; fixed links * fixing flake errors * added types for command arguments * adjusted get_release_diff to append strings for release tags * set fetch-depth to 0 for integrations checkout in workflow * changed the name of the workflow * removed TODOs * adjusted release docs workflow to remove prefix for release tags * adjusted URL replacement only if pointed to docs site * added elastic website to regex pattern * add docstrings; adjusted regex; add note for stopgap * added a note about the regex pattern for elastic URLs (cherry picked from commit 06b97ec)
…ty docs (#3434) * removed historical argument; added setup string; fixed links * fixing flake errors * added types for command arguments * adjusted get_release_diff to append strings for release tags * set fetch-depth to 0 for integrations checkout in workflow * changed the name of the workflow * removed TODOs * adjusted release docs workflow to remove prefix for release tags * adjusted URL replacement only if pointed to docs site * added elastic website to regex pattern * add docstrings; adjusted regex; add note for stopgap * added a note about the regex pattern for elastic URLs (cherry picked from commit 06b97ec)
…ty docs (#3434) * removed historical argument; added setup string; fixed links * fixing flake errors * added types for command arguments * adjusted get_release_diff to append strings for release tags * set fetch-depth to 0 for integrations checkout in workflow * changed the name of the workflow * removed TODOs * adjusted release docs workflow to remove prefix for release tags * adjusted URL replacement only if pointed to docs site * added elastic website to regex pattern * add docstrings; adjusted regex; add note for stopgap * added a note about the regex pattern for elastic URLs (cherry picked from commit 06b97ec)
…ty docs (#3434) * removed historical argument; added setup string; fixed links * fixing flake errors * added types for command arguments * adjusted get_release_diff to append strings for release tags * set fetch-depth to 0 for integrations checkout in workflow * changed the name of the workflow * removed TODOs * adjusted release docs workflow to remove prefix for release tags * adjusted URL replacement only if pointed to docs site * added elastic website to regex pattern * add docstrings; adjusted regex; add note for stopgap * added a note about the regex pattern for elastic URLs (cherry picked from commit 06b97ec)
…ty docs (#3434) * removed historical argument; added setup string; fixed links * fixing flake errors * added types for command arguments * adjusted get_release_diff to append strings for release tags * set fetch-depth to 0 for integrations checkout in workflow * changed the name of the workflow * removed TODOs * adjusted release docs workflow to remove prefix for release tags * adjusted URL replacement only if pointed to docs site * added elastic website to regex pattern * add docstrings; adjusted regex; add note for stopgap * added a note about the regex pattern for elastic URLs (cherry picked from commit 06b97ec)
…ty docs (#3434) * removed historical argument; added setup string; fixed links * fixing flake errors * added types for command arguments * adjusted get_release_diff to append strings for release tags * set fetch-depth to 0 for integrations checkout in workflow * changed the name of the workflow * removed TODOs * adjusted release docs workflow to remove prefix for release tags * adjusted URL replacement only if pointed to docs site * added elastic website to regex pattern * add docstrings; adjusted regex; add note for stopgap * added a note about the regex pattern for elastic URLs (cherry picked from commit 06b97ec)
…ty docs (#3434) * removed historical argument; added setup string; fixed links * fixing flake errors * added types for command arguments * adjusted get_release_diff to append strings for release tags * set fetch-depth to 0 for integrations checkout in workflow * changed the name of the workflow * removed TODOs * adjusted release docs workflow to remove prefix for release tags * adjusted URL replacement only if pointed to docs site * added elastic website to regex pattern * add docstrings; adjusted regex; add note for stopgap * added a note about the regex pattern for elastic URLs (cherry picked from commit 06b97ec)
…ty docs (#3434) * removed historical argument; added setup string; fixed links * fixing flake errors * added types for command arguments * adjusted get_release_diff to append strings for release tags * set fetch-depth to 0 for integrations checkout in workflow * changed the name of the workflow * removed TODOs * adjusted release docs workflow to remove prefix for release tags * adjusted URL replacement only if pointed to docs site * added elastic website to regex pattern * add docstrings; adjusted regex; add note for stopgap * added a note about the regex pattern for elastic URLs (cherry picked from commit 06b97ec)
…ty docs (#3434) * removed historical argument; added setup string; fixed links * fixing flake errors * added types for command arguments * adjusted get_release_diff to append strings for release tags * set fetch-depth to 0 for integrations checkout in workflow * changed the name of the workflow * removed TODOs * adjusted release docs workflow to remove prefix for release tags * adjusted URL replacement only if pointed to docs site * added elastic website to regex pattern * add docstrings; adjusted regex; add note for stopgap * added a note about the regex pattern for elastic URLs (cherry picked from commit 06b97ec)
…ty docs (#3434) * removed historical argument; added setup string; fixed links * fixing flake errors * added types for command arguments * adjusted get_release_diff to append strings for release tags * set fetch-depth to 0 for integrations checkout in workflow * changed the name of the workflow * removed TODOs * adjusted release docs workflow to remove prefix for release tags * adjusted URL replacement only if pointed to docs site * added elastic website to regex pattern * add docstrings; adjusted regex; add note for stopgap * added a note about the regex pattern for elastic URLs (cherry picked from commit 06b97ec)
Issues
Summary
This pull request adjusts the
build-release
CLI command and theIntegrationRuleDetail.generate()
method to account for link formatting issues when converted to asciidoc. Below is a more thorough breakdown of what has changed and why:--add-historical
was removed as an argument frombuild-release
command. Since 8.7, historical rules are required for all packages. Since we release current-3 and are at 8.12-8.9, we can always add historical rules by default. Note - this will not effect backporting negatively as we have historical packages for 8.3+.--generate-docs
argument has been added tobuild-release
. This is to optionally generate the MDX security docs which was built for a future integration into the improved docs site. It is default to no for now as it is not necessary when running this command at this time.IntegrationRuleDetail.generate()
we have logic to build out the sections of a rule's security doc. These are the attributes of the rule itself. We had not accounted for thesetup
field prior to this, thusIntegrationRuleDetail.setup_str()
was added to add this.setup_str()
andnote_str()
methods have been updated to replace[title](url)
withurl
if found. This will format it correctly in the docs so it does not break. Note that investigation guide (note) and setup are code blocks in each rule detail page, thus they do not render Elastic security doc formatting and thus are not compatible with{security-guide}/URI
link, which is why we are just giving the URL. - We can focus on reconciliation with doc migration strategy.Testing
Testing for this involves the detection-rules repository and security-docs repository. Images and links related to successful runs will be found below.
bug-security-doc-generation-broken-links
8.12
Successful Results
Successful Release Docs Workflow Job: https://github.com/elastic/detection-rules/actions/runs/7848748147/job/21420563267
CI-Passing PR to Security Docs without needing to manually fix links: elastic/security-docs#4787
![Screenshot 2024-02-09 at 2 16 41 PM](https://private-user-images.githubusercontent.com/99630311/303763143-a8e4ddae-22ee-4e10-a9eb-61a7f96072bd.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkxNzYwMTEsIm5iZiI6MTczOTE3NTcxMSwicGF0aCI6Ii85OTYzMDMxMS8zMDM3NjMxNDMtYThlNGRkYWUtMjJlZS00ZTEwLWE5ZWItNjFhN2Y5NjA3MmJkLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjEwVDA4MjE1MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWFkNGQ5OWVlZGE5MzQ0ZjA3YWYwODYyMjQ3OTUwODUyNWNjNTMwMzgwN2EzZDc4ZjFkZDc0MGI5Y2RhNjY3NDYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.YL7pgBMQI6rw6Lkd6LVVY6xGttJm5Y9QA23Trp5_9k8)
![Screenshot 2024-02-09 at 2 59 07 PM](https://private-user-images.githubusercontent.com/99630311/303763175-2af5a071-ba9f-4414-9372-1fd51cca56f4.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkxNzYwMTEsIm5iZiI6MTczOTE3NTcxMSwicGF0aCI6Ii85OTYzMDMxMS8zMDM3NjMxNzUtMmFmNWEwNzEtYmE5Zi00NDE0LTkzNzItMWZkNTFjY2E1NmY0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjEwVDA4MjE1MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTIzZGJkMmI1ODkzOGMzOGUwMzE5MDAxNmZmYmZiMzIxNTc3NmRjZWU1OGI3ZmRiNDdkYmZkYzBjMzMzMTg2YmEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.basZZjxANZnbOsUKWxkx2y7Tz8q-MdVkUkLD3CxVcmE)