Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Merged by Bors] - Adding VM conformance output to PR checks #1685

Closed
wants to merge 9 commits into from
Closed

[Merged by Bors] - Adding VM conformance output to PR checks #1685

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
7882f81
Adding VM conformance output to PR checks
Razican Oct 24, 2021
249246b
chore: don't fail if ../results exists
RageKnify Oct 24, 2021
0d6fae0
refactor: stop using json path in boa_tester
RageKnify Oct 26, 2021
684248c
style: respect clippy
RageKnify Oct 26, 2021
18c2293
fix: try to fix comment
RageKnify Oct 26, 2021
bd731a1
style: use ' for string
RageKnify Oct 26, 2021
be1648b
fix: try to fix comment v2
RageKnify Oct 26, 2021
ae844a5
fix: try to fix comment with multi-line string
RageKnify Oct 26, 2021
4433cc3
fix: try to fix comment v3
RageKnify Oct 26, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 23 additions & 3 deletions .github/workflows/test262.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ jobs:
# Run the results comparison
- name: Compare results
if: github.event_name == 'pull_request'
id: compare
id: compare-non-vm
shell: bash
run: |
cd boa
Expand All @@ -61,6 +61,26 @@ jobs:
comment="${comment//$'\r'/'%0D'}"
echo "::set-output name=comment::$comment"

- name: Run the test262 test suite for VM
run: |
cd boa
mkdir ../results
cargo run --release --features vm --bin boa_tester -- run -v -o ../results/test262/vm
RageKnify marked this conversation as resolved.
Show resolved Hide resolved
cd ..

# Run the results comparison for VM
- name: Compare results for VM
if: github.event_name == 'pull_request'
id: compare-vm
shell: bash
run: |
cd boa
comment="$(./target/release/boa_tester compare ../gh-pages/test262/refs/heads/main/vm-latest.json ../results/test262/vm/pull/latest.json -m)"
comment="${comment//'%'/'%25'}"
comment="${comment//$'\n'/'%0A'}"
comment="${comment//$'\r'/'%0D'}"
echo "::set-output name=comment::$comment"

- name: Get the PR number
if: github.event_name == 'pull_request'
id: pr-number
Expand All @@ -80,7 +100,7 @@ jobs:
continue-on-error: true
with:
comment-id: ${{ steps.previous-comment.outputs.comment-id }}
body: ${{ steps.compare.outputs.comment }}
body: \### Test262 conformance changes%0A%0A${{ steps.compare-non-vm.outputs.comment }}%0A${{ steps.compare-vm.outputs.comment }}
edit-mode: replace

- name: Write a new comment
Expand All @@ -89,7 +109,7 @@ jobs:
continue-on-error: true
with:
issue-number: ${{ steps.pr-number.outputs.pr }}
body: ${{ steps.compare.outputs.comment }}
body: \### Test262 conformance changes%0A%0A${{ steps.compare-non-vm.outputs.comment }}%0A%0A${{ steps.compare-vm.outputs.comment }}

# Commit changes to GitHub pages.
- name: Commit files
Expand Down
3 changes: 3 additions & 0 deletions boa_tester/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ exclude = ["../.vscode/*", "../Dockerfile", "../Makefile", "../.editorConfig"]
edition = "2021"
rust-version = "1.56"

[features]
vm = ["Boa/vm"]

[dependencies]
Boa = { path = "../boa" }
structopt = "0.3.25"
Expand Down
6 changes: 5 additions & 1 deletion boa_tester/src/results.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,11 @@ pub(crate) fn compare_results(base: &Path, new: &Path, markdown: bool) {
)
}

println!("### Test262 conformance changes:");
#[cfg(feature = "vm")]
println!("#### VM implementation");
#[cfg(not(feature = "vm"))]
println!("#### Non-VM implementation");

println!("| Test result | main count | PR count | difference |");
println!("| :---------: | :----------: | :------: | :--------: |");
println!(
Expand Down