From 7b1b03dac1f8db9906b32bb908b09a14e66c377d Mon Sep 17 00:00:00 2001 From: Chris Cummins Date: Tue, 28 Sep 2021 15:58:01 +0100 Subject: [PATCH 1/5] Release v0.2.0. This release adds two new compiler optimization problems to CompilerGym: GCC command line flag optimization and CUDA loop nest optimization. - [GCC] A new `gcc-v0` environment, authored by @hughleat, exposes the command line flags of GCC as a reinforcement learning environment. GCC is a production-grade compiler for C and C++ used throughout industry. The environment provides several datasets and a large, high dimensional action space that works on several GCC versions. For further details check out the reference documentation: https://facebookresearch.github.io/CompilerGym/envs/gcc.html - [loop_tool] A new `loop_tool-v0` environment, authored by @bwasti, provides an experimental intermediate representation of *n*-dimensional data computation that can be lowered to both CPU and GPU backends. This provides a reinforcement learning environment for manipulating nests of loop computations to maximize throughput. For further details check out the reference documentation: https://facebookresearch.github.io/CompilerGym/envs/loop_tool.html Other highlights of this release include: - [Docker] Published a chriscummins/compiler_gym docker image that can be used to run CompilerGym services in standalone isolated containers. - [LLVM] Fixed a bug in the experimental `Runtime` observation space that caused observations to slow down over time. - [LLVM] Added a new utility module to compute observations from bitcodes. - Overhauled the continuous integration services to reduce computational requirements by 59.4% while increasing test coverage. - Improved error reporting if computing an observation fails. - Changed the return type of compiler_gym.random_search() to a CompilerEnv. - Numerous other bug fixes and improvements. Many thanks to code contributors: @thecoblack, @bwasti, @hughleat, and @sahirgomez1! --- CHANGELOG.md | 44 ++++++++++++++++++++++++++++++++++++++++++++ CITATION.cff | 4 ++-- VERSION | 2 +- 3 files changed, 47 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c7849c851..acafa886c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,47 @@ +## Release 0.2.0 (2021-09-28) + +This release adds two new compiler optimization problems to CompilerGym: GCC +command line flag optimization and CUDA loop nest optimization. + +- **[GCC]** A new `gcc-v0` environment, authored by @hughleat, exposes the + command line flags of [GCC](https://gcc.gnu.org/) as a reinforcement learning + environment. GCC is a production-grade compiler for C and C++ used throughout + industry. The environment provides several datasets and a large, high + dimensional action space that works on several GCC versions. For further + details check out the [reference + documentation](https://facebookresearch.github.io/CompilerGym/envs/gcc.html). +- **[loop_tool]** A new `loop_tool-v0` environment, authored by @bwasti, + provides an experimental intermediate representation of *n*-dimensional data + computation that can be lowered to both CPU and GPU backends. This provides a + reinforcement learning environment for manipulating nests of loop computations + to maximize throughput. For further details check out the [reference + documentation](https://facebookresearch.github.io/CompilerGym/envs/loop_tool.html). + +Other highlights of this release include: + +- **[Docker]** Published a + [chriscummins/compiler_gym](https://hub.docker.com/repository/docker/chriscummins/compiler_gym) + docker image that can be used to run CompilerGym services in standalone + isolated containers + ([#424](https://github.com/facebookresearch/CompilerGym/pull/424)). +- **[LLVM]** Fixed a bug in the experimental `Runtime` observation space that + caused observations to slow down over time + ([#398](https://github.com/facebookresearch/CompilerGym/pull/398)). +- **[LLVM]** Added a new utility module to compute observations from bitcodes + ([#405](https://github.com/facebookresearch/CompilerGym/pull/405)). +- Overhauled the continuous integration services to reduce computational + requirements by 59.4% while increasing test coverage + ([#392](https://github.com/facebookresearch/CompilerGym/pull/392)). +- Improved error reporting if computing an observation fails + ([#380](https://github.com/facebookresearch/CompilerGym/pull/380)). +- Changed the return type of `compiler_gym.random_search()` to a `CompilerEnv` + ([#387](https://github.com/facebookresearch/CompilerGym/pull/387)). +- Numerous other bug fixes and improvements. + +Many thanks to code contributors: @thecoblack, @bwasti, @hughleat, and +@sahirgomez1! + + ## Release 0.1.10 (2021-09-08) This release lays the foundation for several new exciting additions to diff --git a/CITATION.cff b/CITATION.cff index 007468167..3cbeaa34c 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -31,8 +31,8 @@ authors: - family-names: "Leather" given-names: "Hugh" title: "CompilerGym" -version: 0.1.10 -date-released: 2021-09-08 +version: 0.2.0 +date-released: 2021-09-28 url: "https://github.com/facebookresearch/CompilerGym" preferred-citation: type: article diff --git a/VERSION b/VERSION index 9767cc98e..0ea3a944b 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.1.10 +0.2.0 From b0a075f12c0ca4a93bbaa333c0909589c7f7e37d Mon Sep 17 00:00:00 2001 From: Chris Cummins Date: Wed, 29 Sep 2021 12:04:47 +0100 Subject: [PATCH 2/5] [ci] Automatically build release artifacts on push to stable. --- .github/workflows/build.yaml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/workflows/build.yaml b/.github/workflows/build.yaml index cccd1c299..86709b2b6 100644 --- a/.github/workflows/build.yaml +++ b/.github/workflows/build.yaml @@ -7,6 +7,10 @@ name: Build Binaries on: workflow_dispatch: + pull_request: + branches: [stable] + push: + branches: [stable] jobs: build: From ed5ced8f19e0a7dc12b424d04f3e75573c41d07c Mon Sep 17 00:00:00 2001 From: Chris Cummins Date: Wed, 29 Sep 2021 12:06:58 +0100 Subject: [PATCH 3/5] [Makefile] Split example tests into a separate target. --- .github/workflows/ci.yaml | 14 ++++---------- Makefile | 17 ++++++++++++----- 2 files changed, 16 insertions(+), 15 deletions(-) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 0b65e62f3..4b6a4b556 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -97,7 +97,7 @@ jobs: run: python -m pip install -r tests/requirements.txt - name: Run the test suite - run: make install-test-cov TEST_TARGET="tests" PYTEST_ARGS="--ignore tests/llvm --ignore tests/gcc --ignore tests/loop_tool" + run: make install-test-cov PYTEST_ARGS="--ignore tests/llvm --ignore tests/gcc --ignore tests/loop_tool" - name: Upload coverage report to Codecov uses: codecov/codecov-action@v2 @@ -134,7 +134,7 @@ jobs: run: python -m pip install -r tests/requirements.txt - name: Run the test suite - run: make install-test-cov TEST_TARGET="tests" PYTEST_ARGS="--ignore tests/llvm --ignore tests/gcc" + run: make install-test-cov PYTEST_ARGS="--ignore tests/llvm --ignore tests/gcc" - name: Upload coverage report to Codecov uses: codecov/codecov-action@v2 @@ -468,7 +468,7 @@ jobs: run: python -m pip install -r examples/requirements.txt -r tests/requirements.txt - name: Run the test suite - run: make install-test-cov TEST_TARGET="examples" + run: make examples-test - name: Upload coverage report to Codecov uses: codecov/codecov-action@v2 @@ -476,9 +476,6 @@ jobs: fail_ci_if_error: true verbose: true - - name: Run the tests in the source tree - run: cd examples && pytest . - test-examples-macos: needs: build-macos runs-on: macos-latest @@ -509,7 +506,7 @@ jobs: run: python -m pip install -r examples/requirements.txt -r tests/requirements.txt - name: Run the test suite - run: make install-test-cov TEST_TARGET="examples" + run: make examples-test - name: Upload coverage report to Codecov uses: codecov/codecov-action@v2 @@ -517,9 +514,6 @@ jobs: fail_ci_if_error: true verbose: true - - name: Run the tests in the source tree - run: cd examples && pytest . - build-asan-llvm-service: runs-on: ubuntu-latest steps: diff --git a/Makefile b/Makefile index 65ff46ce1..f08fc520b 100644 --- a/Makefile +++ b/Makefile @@ -51,6 +51,10 @@ Post-installation Tests the fuzz tests for a minimum of one minute. This requires that the CompilerGym package has been installed (`make install`). + make examples-test + Run pytest in the examples directory. This requires that the CompilerGym + package has been installed (`make install`). + Documentation ------------- @@ -302,18 +306,21 @@ itest: bazel-fetch # symlinking the test directory into it so that pytest can be invoked. install-test-setup: mkdir -p "$(INSTALL_TEST_ROOT)" - rm -f "$(INSTALL_TEST_ROOT)/examples" "$(INSTALL_TEST_ROOT)/tests" "$(INSTALL_TEST_ROOT)/tox.ini" - ln -s "$(ROOT)/examples" "$(INSTALL_TEST_ROOT)" + rm -f "$(INSTALL_TEST_ROOT)/tests" "$(INSTALL_TEST_ROOT)/tox.ini" + ln -s "$(INSTALL_TEST_ROOT)" ln -s "$(ROOT)/tests" "$(INSTALL_TEST_ROOT)" ln -s "$(ROOT)/tox.ini" "$(INSTALL_TEST_ROOT)" define pytest - cd "$(INSTALL_TEST_ROOT)" && pytest $(if $(TEST_TARGET),$(TEST_TARGET),examples tests) $(1) $(PYTEST_ARGS) + cd "$(INSTALL_TEST_ROOT)" && pytest $(if $(TEST_TARGET),$(TEST_TARGET),tests) $(1) $(PYTEST_ARGS) endef -install-test: install-test-setup +install-test: | install-test-setup $(call pytest,--no-success-flaky-report --benchmark-disable -n auto -k "not fuzz" --durations=5) +examples-test: + cd examples && pytest --no-success-flaky-report --benchmark-disable -n auto --durations=5 . --cov=compiler_gym --cov-report=xml:$(COV_REPORT) $(PYTEST_ARGS) + # Note we export $CI=1 so that the tests always run as if within the CI # environement. This is to ensure that the reported coverage matches that of # the value on: https://codecov.io/gh/facebookresearch/CompilerGym @@ -331,7 +338,7 @@ post-install-test: $(MAKE) -C examples/makefile_integration clean SEARCH_TIME=3 $(MAKE) -C examples/makefile_integration test -.PHONY: test post-install-test +.PHONY: test post-install-test examples-test ################ From 4b3d220d86bd5599d63f99a2b2730fed8e42aadb Mon Sep 17 00:00:00 2001 From: Chris Cummins Date: Wed, 29 Sep 2021 12:13:29 +0100 Subject: [PATCH 4/5] [ci] Only run examples tests as post-build verification. We will be running the full CI workflow on every push / PR, so repeating every test on the build artifacts seems wasteful. Instead just run the examples tests. --- .github/workflows/build.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/build.yaml b/.github/workflows/build.yaml index 86709b2b6..b42c7a34b 100644 --- a/.github/workflows/build.yaml +++ b/.github/workflows/build.yaml @@ -53,7 +53,7 @@ jobs: if-no-files-found: error retention-days: 14 - test: + examples-test: needs: build runs-on: ${{ matrix.os }} strategy: @@ -93,5 +93,5 @@ jobs: - name: Install test dependencies run: python -m pip install -r examples/requirements.txt -r tests/requirements.txt - - name: Run the test suite - run: make install-test + - name: Run the examples tests + run: make examples-test From f51194191261e6ecb1a624bf921f7827864d0e08 Mon Sep 17 00:00:00 2001 From: Chris Cummins Date: Wed, 29 Sep 2021 12:18:56 +0100 Subject: [PATCH 5/5] [git] Don't track in-tree .coverage files. --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 3d24ca5d2..62536927a 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,5 @@ __pycache__ +.coverage* .DS_Store .env /.act