Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate forge command outputs in docs #2687

Open
wants to merge 89 commits into
base: master
Choose a base branch
from

Conversation

franciszekjob
Copy link
Collaborator

@franciszekjob franciszekjob commented Nov 19, 2024

Relates #2479
Closes #2684

Introduced changes

  • Automatically run snforge commands which are present in the docc; if the output of respective output is present in the docs too, it's compared with the actual output from executed command
  • Allow ignoring commands (so they will not be executed)

Checklist

  • Linked relevant issue
  • Updated relevant documentation
  • Added relevant tests
  • Performed self-review of the code
  • Added changes to CHANGELOG.md

franciszekjob and others added 30 commits November 8, 2024 17:38
@franciszekjob franciszekjob changed the title Franciszekjob/2479 3 shell commands running validation Validate forge command outputs in docs Nov 21, 2024
@@ -1,4 +1,4 @@
#!/bin/bash
set -e

for d in ./docs/listings/*; do (cd "$d" && scarb test); done
for d in ./docs/listings/*; do (cd "$d" && scarb check); done
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: Instead of calling scarb test we should call scarb check, some tests in examples are failing on purpose.

@franciszekjob
Copy link
Collaborator Author

Validation of sncast command outputs will be added in the next PR.

@franciszekjob
Copy link
Collaborator Author

franciszekjob commented Nov 21, 2024

For some reason, Test Building Docs fails on CI (example here). It complains about opening scarb plugin file which doesn't exist 🤔 .

@franciszekjob franciszekjob marked this pull request as ready for review November 21, 2024 11:09
}

fn list_packages_in_directory(dir_path: &Path) -> Vec<String> {
let crates_path = dir_path.join("");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let crates_path = dir_path.join("");
let crates_path = dir_path.to_owned();

Also why you need this at all?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's used to retrieve packages located in docs listing directory.

crates/docs/src/validation.rs Outdated Show resolved Hide resolved
Comment on lines 70 to 91
let temp = resolve_temp_dir(&package_name, &listings_to_packages_mapping);
let output = test_runner(&temp).args(args).assert();

assert_stdout_contains(output, snippet_output);
}
}

print_success_message(snippets.len(), "snforge");
print_success_message(snippets.len(), snippet_type.as_str());
}

fn resolve_temp_dir(
package_name: &str,
listings_to_packages_mapping: &HashMap<String, Vec<String>>,
) -> TempDir {
if is_package_from_docs_listings(package_name, listings_to_packages_mapping) {
setup_package_from_docs_listings(package_name, listings_to_packages_mapping)
} else if ["addition", "fibonacci"].contains(&package_name) {
setup_hello_workspace()
} else {
setup_package(package_name)
}
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: Currently we use hello_workpaces package (from crates/forge/tests/data) in one example in docs. In future we can simplify this logic and remove is_package_from_docs_listings, once we use only packages from docs listings. Ref: #2698

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve skipping docs snippets
2 participants