Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stage2 test harness: declarative format #8828

Closed
andrewrk opened this issue May 18, 2021 · 2 comments
Closed

stage2 test harness: declarative format #8828

andrewrk opened this issue May 18, 2021 · 2 comments
Labels
enhancement Solving this issue will likely involve adding new logic or components to the codebase. frontend Tokenization, parsing, AstGen, Sema, and Liveness.
Milestone

Comments

@andrewrk
Copy link
Member

It takes a long time to run ./zig build test-stage2 because it builds the full compiler. This issue is to introduce into the test harness, ability to load test cases from a declarative file format such as JSON, which references source files on disk. For example, here is the first test case from test/stage2/test.zig converted to declarative form:

manifest.json

{
  kind: "exe",
  target: "x86_64-linux",
  updates: {
    "empty file": {
      kind: "error",
      sources: {"test.zig": "empty.zig"},
      errors: [
        "test.zig:84:9: error: struct 'test_case.test_case' has no member named 'main'",
      ],
    },
    "incorrect return type": {
      kind: "error",
      sources: {"test.zig": "bad-start-ret-type.zig"},
      errors: [
        "test.zig:2:1: error: expected noreturn, found void",
      ],
    },
    "hello world bare bones": {
      kind: "compareOutput",
      sources: {"test.zig": "x86-asm-hello.zig"},
      stdout: "Hello, World!\n",
    },
    "convert to pub fn main": {
      kind: "compareOutput",
      sources: {"test.zig": "x86-asm-hello-2.zig"},
      stdout: "Hello, World!\n",
    },
    "change the message only": {
      kind: "compareOutput",
      sources: {"test.zig": "x86-asm-hello-3.zig"},
      stdout: "What is up? This is a longer message that will force the data to be relocated in virtual address space.\n",
    },
    "print it twice": {
      kind: "compareOutput",
      sources: {"test.zig": "x86-asm-hello-4.zig"},
      stdout: "What is up? This is a longer message that will force the data to be relocated in virtual address space.\nWhat is up? This is a longer message that will force the data to be relocated in virtual address space.\n",
    },
  }
}

file structure would look something like:

test/stage2/cases/hello world with updates/manifest.json
test/stage2/cases/hello world with updates/empty.zig
test/stage2/cases/hello world with updates/bad-start-ret-type.zig
test/stage2/cases/hello world with updates/x86-asm-hello.zig
test/stage2/cases/hello world with updates/x86-asm-hello-2.zig
test/stage2/cases/hello world with updates/x86-asm-hello-3.zig

On each update, the test harness scans over sources and copies the files listed there to a temporary directory, which acts as the source directory for the test compilation.

@andrewrk andrewrk added enhancement Solving this issue will likely involve adding new logic or components to the codebase. frontend Tokenization, parsing, AstGen, Sema, and Liveness. labels May 18, 2021
@andrewrk andrewrk added this to the 0.8.0 milestone May 18, 2021
@andrewrk andrewrk modified the milestones: 0.8.0, 0.9.0 Jun 4, 2021
@andrewrk andrewrk modified the milestones: 0.9.0, 0.10.0 Nov 22, 2021
@nektro
Copy link
Contributor

nektro commented Jan 30, 2022

i think a rework of the implementation of TestContext could also achieve this

@andrewrk
Copy link
Member Author

Superseded by #11288 (cc @mitchellh)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Solving this issue will likely involve adding new logic or components to the codebase. frontend Tokenization, parsing, AstGen, Sema, and Liveness.
Projects
None yet
Development

No branches or pull requests

2 participants