Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bazel 1.1 #3249

Merged
merged 19 commits into from
Nov 11, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,10 @@ build:darwin --host_platform=@rules_haskell//haskell/platforms:darwin_x86_64_nix
# and GHC's gcc on Windows
build:windows --crosstool_top=@rules_haskell_ghc_windows_amd64//:cc_toolchain

# Bazel 1.0 disabled the bash test-runner on Windows. However, some of our
# test-cases are implemented as bash scripts and rely on the bash test-runner.
build:windows --noincompatible_windows_native_test_wrapper
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not for this PR but could we work around this by having some wrapper binary that then calls bash internally?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One way to address this is turn things like client_server_test into a macro that generates a script file and then calls sh_test|binary on that. Another is to use cc_toolchain.link|compile to generate a binary that calls the shell script. I think the former is the better approach. I've experimented with both and the latter caused some issues with the test-environment not matching what the test-cases expected.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, thanks for the explanation!


# Caching is currently disabled on Windows.
# See: https://github.com/tweag/rules_haskell/issues/744 for details.
build:windows --noremote_accept_cached
Expand Down
2 changes: 1 addition & 1 deletion BAZEL.md
Original file line number Diff line number Diff line change
Expand Up @@ -723,7 +723,7 @@ Attributes:

The output of `da_doc_package` with name `"foo"` is a bundle
`sources.tar.gzip` in the path
`//bazel-genfiles/daml-foundations/daml-tools/docs/foo`. The bundle for `"foo"` would be produced with the command:
`//bazel-bin/daml-foundations/daml-tools/docs/foo`. The bundle for `"foo"` would be produced with the command:
```
bazel build //daml-foundations/daml-tools/docs/foo:foo
```
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ On Windows:

```
bazel build //release:sdk-release-tarball
tar -vxf .\bazel-genfiles\release\sdk-release-tarball.tar.gz
tar -vxf .\bazel-bin\release\sdk-release-tarball.tar.gz
cd sdk-*
daml\daml.exe install . --activate
```
Expand Down
2 changes: 2 additions & 0 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -642,6 +642,7 @@ hazel_repositories(
hazel_custom_package_hackage(
package_name = "ghc-paths",
build_file = "@ai_formation_hazel//third_party/haskell:BUILD.ghc-paths",
sha256 = "afa68fb86123004c37c1dc354286af2d87a9dcfb12ddcb80e8bd0cd55bc87945",
version = "0.1.0.9",
)

Expand Down Expand Up @@ -889,6 +890,7 @@ buildifier_dependencies()
nixpkgs_package(
name = "python3_nix",
attribute_path = "python3",
nix_file = "//nix:bazel.nix",
nix_file_deps = common_nix_file_deps,
repositories = dev_env_nix_repos,
)
Expand Down
2 changes: 2 additions & 0 deletions bazel_tools/dev_env_tool/dev_env_tool.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -137,4 +137,6 @@ dev_env_tool = repository_rule(
),
"prefix": attr.string(),
},
configure = True,
local = True,
)
1 change: 1 addition & 0 deletions bazel_tools/grpc-bazel-mingw.patch
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,4 @@ index 0349e31bb3..d2e7408dbf 100644
+#define _WIN32_WINNT 0x0A00

#include <windows.h>

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bazel now uses an internal implementation instead of shelling out to patch. This internal implementation is a bit less lenient than patch and complains about this missing newline.

13 changes: 13 additions & 0 deletions bazel_tools/haskell-bazel-1.1.0.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
diff --git a/haskell/private/path_utils.bzl b/haskell/private/path_utils.bzl
index 22bd50e8..89d1b52b 100644
--- a/haskell/private/path_utils.bzl
+++ b/haskell/private/path_utils.bzl
@@ -580,7 +580,7 @@ def parse_pattern(ctx, pattern_str):
pattern_str = "//{package}{target}".format(package = ctx.label.package, target = pattern_str)

# Separate package and target (if present).
- package_target = pattern_str[2:].split(":", maxsplit = 2)
+ package_target = pattern_str[2:].split(":", 2)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll update rules_haskell in a separate PR, at which point this patch will be unnecessary..

package_str = package_target[0]
target_str = None
if len(package_target) == 2:
8 changes: 4 additions & 4 deletions bazel_tools/proto.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ def get_plugin_runfiles(tool, plugin_runfiles):
return files

def _proto_gen_impl(ctx):
src_descs = [src.proto.direct_descriptor_set for src in ctx.attr.srcs]
dep_descs = [dep.proto.direct_descriptor_set for dep in ctx.attr.deps]
src_descs = [src[ProtoInfo].direct_descriptor_set for src in ctx.attr.srcs]
dep_descs = [dep[ProtoInfo].direct_descriptor_set for dep in ctx.attr.deps]
descriptors = src_descs + dep_descs

sources_out = ctx.actions.declare_directory(ctx.attr.name + "-sources")
Expand All @@ -63,8 +63,8 @@ def _proto_gen_impl(ctx):
inputs = []

for src in ctx.attr.srcs:
src_root = src.proto.proto_source_root
for direct_source in src.proto.direct_sources:
src_root = src[ProtoInfo].proto_source_root
for direct_source in src[ProtoInfo].direct_sources:
path = ""

# in some cases the paths of src_root and direct_source are only partially
Expand Down
4 changes: 4 additions & 0 deletions build.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ function bazel() {
# which is a workaround for this problem.
bazel shutdown

# Manually fetch a Windows dev-env tool to avoid the following error:
# ERROR loading ~/.scoop: The process cannot access the file 'C:\Users\VssAdministrator\.scoop' because it is being used by another process.
bazel fetch @makensis_dev_env//...

bazel build `-`-experimental_execution_log_file ${ARTIFACT_DIRS}/build_execution_windows.log //...

bazel shutdown
Expand Down
2 changes: 1 addition & 1 deletion ci/build-unix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ steps:
- bash: |
set -euo pipefail
ARTIFACT=daml-sdk-$(release_tag)-${{ parameters.name }}.tar.gz
cp bazel-genfiles/release/sdk-release-tarball.tar.gz $(Build.StagingDirectory)/$ARTIFACT
cp bazel-bin/release/sdk-release-tarball.tar.gz $(Build.StagingDirectory)/$ARTIFACT
echo "##vso[task.setvariable variable=artifact;isOutput=true]$ARTIFACT"
name: publish
condition: eq(variables['release.has_released'], 'true')
Expand Down
4 changes: 2 additions & 2 deletions ci/build-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,10 @@ steps:
- bash: |
set -euo pipefail
ARTIFACT=daml-sdk-$(release_tag)-windows.tar.gz
cp bazel-genfiles/release/sdk-release-tarball.tar.gz '$(Build.StagingDirectory)'/$ARTIFACT
cp bazel-bin/release/sdk-release-tarball.tar.gz '$(Build.StagingDirectory)'/$ARTIFACT
echo "##vso[task.setvariable variable=artifact;isOutput=true]$ARTIFACT"
WINDOWS_INSTALLER=daml-sdk-$(release_tag)-windows-unsigned.exe
cp bazel-genfiles/release/windows-installer/daml-sdk-installer.exe "$(Build.StagingDirectory)/$WINDOWS_INSTALLER"
cp bazel-bin/release/windows-installer/daml-sdk-installer.exe "$(Build.StagingDirectory)/$WINDOWS_INSTALLER"
echo "##vso[task.setvariable variable=artifact-unsigned-windows-installer;isOutput=true]$WINDOWS_INSTALLER"
name: publish
condition: eq(variables['release.has_released'], 'true')
Expand Down
6 changes: 3 additions & 3 deletions ci/cron/src/Main.hs
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ build_docs_folder path versions = do
shell_ $ "git checkout v" <> latest
robustly_download_nix_packages
shell_ "bazel build //docs:docs"
shell_ $ "tar xzf bazel-genfiles/docs/html.tar.gz --strip-components=1 -C " <> path
shell_ $ "tar xzf bazel-bin/docs/html.tar.gz --strip-components=1 -C " <> path
-- Not going through Aeson because it represents JSON objects as unordered
-- maps, and here order matters.
let versions_json = versions
Expand All @@ -139,14 +139,14 @@ build_docs_folder path versions = do
& \s -> "{" <> s <> "}"
writeFile (path <> "/versions.json") versions_json
shell_ $ "mkdir -p " <> path <> "/" <> latest
shell_ $ "tar xzf bazel-genfiles/docs/html.tar.gz --strip-components=1 -C " <> path <> "/" <> latest
shell_ $ "tar xzf bazel-bin/docs/html.tar.gz --strip-components=1 -C " <> path <> "/" <> latest
Foldable.for_ (tail versions) $ \version -> do
putStrLn $ "Building older docs: " <> version
shell_ $ "git checkout v" <> version
robustly_download_nix_packages
shell_ "bazel build //docs:docs"
shell_ $ "mkdir -p " <> path <> "/" <> version
shell_ $ "tar xzf bazel-genfiles/docs/html.tar.gz --strip-components=1 -C" <> path <> "/" <> version
shell_ $ "tar xzf bazel-bin/docs/html.tar.gz --strip-components=1 -C" <> path <> "/" <> version
shell_ $ "git checkout " <> cur_sha

check_s3_versions :: Set.Set String -> IO Bool
Expand Down
8 changes: 7 additions & 1 deletion compiler/daml-extension/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -95,11 +95,17 @@ genrule(
"//:VERSION",
],
outs = ["daml-bundled.vsix"],
# rm -rf can fail with "directory not empty" on Windows.
# As a workaround we add `|| return`.
cmd = """
set -euo pipefail
TMP_DIR=$$(mktemp -d)
cleanup () { rm -rf $$TMP_DIR || return; }
trap cleanup EXIT
DIR=$$PWD
VERSION=$$(cat $(location //:VERSION))
cd compiler/daml-extension
cp -r compiler/daml-extension $$TMP_DIR
cd $$TMP_DIR/daml-extension
tar xzf $$DIR/$(location :node_deps_cache)
sed -i "s/__VERSION__/$$VERSION/" package.json
sed -i 's/"name": "daml"/"name": "daml-bundled"/' package.json
Expand Down
4 changes: 2 additions & 2 deletions daml-lf/archive/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -155,13 +155,13 @@ da_scala_test_suite(
data = [
":DarReaderTest.dalf",
":DarReaderTest.dar",
":daml_lf_1.6_archive_proto_srcs",
":daml_lf_1.7_archive_proto_srcs",
],
scalacopts = lf_scalacopts,
deps = [
":daml_lf_1.6_archive_java_proto",
":daml_lf_1.6_archive_proto_srcs",
":daml_lf_1.7_archive_java_proto",
":daml_lf_1.7_archive_proto_srcs",
":daml_lf_archive_reader",
":daml_lf_dev_archive_java_proto",
"//bazel_tools/runfiles:scala_runfiles",
Expand Down
10 changes: 6 additions & 4 deletions deps.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive", "http_file")
load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")

rules_scala_version = "8092d5f6165a8d9c4797d5f089c1ba4eee3326b1"
rules_scala_version = "0f89c210ade8f4320017daf718a61de3c1ac4773"
rules_haskell_version = "74d91c116c59f0a3ad41e376d3307d85ddcc3253"
rules_haskell_sha256 = "5f2423d4707c5601f465d7343c68ff4e8f271c1269e79af4dc2d156cb8a0c17d"
rules_nixpkgs_version = "5ffb8a4ee9a52bc6bc12f95cd64ecbd82a79bc82"
rules_nixpkgs_version = "2980a2f75a178bb3c521d46380a52fe72c656239"

def daml_deps():
if "rules_haskell" not in native.existing_rules():
Expand All @@ -41,6 +41,8 @@ def daml_deps():
strip_prefix = "rules_haskell-%s" % rules_haskell_version,
urls = ["https://github.com/tweag/rules_haskell/archive/%s.tar.gz" % rules_haskell_version],
patches = [
# Bazel 1.1.0 compatibility. Remove once rules_haskell has been updated.
"@com_github_digital_asset_daml//bazel_tools:haskell-bazel-1.1.0.patch",
# Remove once https://github.com/tweag/rules_haskell/pull/1039 is merged.
"@com_github_digital_asset_daml//bazel_tools:haskell-cc-wrapper.patch",
# Upstream once https://github.com/tweag/rules_haskell/pull/1039 is merged.
Expand All @@ -62,7 +64,7 @@ def daml_deps():
name = "io_tweag_rules_nixpkgs",
strip_prefix = "rules_nixpkgs-%s" % rules_nixpkgs_version,
urls = ["https://github.com/tweag/rules_nixpkgs/archive/%s.tar.gz" % rules_nixpkgs_version],
sha256 = "085d480232c0bada20c0d0b8b1b4ba8c62fcc006d9dc826aa0e4205e4dca6cb3",
sha256 = "a37917d9cb535466d94cbedab5a3d7365fc333843b2265cfb0a5211647769b88",
)

if "ai_formation_hazel" not in native.existing_rules():
Expand Down Expand Up @@ -112,7 +114,7 @@ def daml_deps():
url = "https://github.com/bazelbuild/rules_scala/archive/%s.zip" % rules_scala_version,
type = "zip",
strip_prefix = "rules_scala-%s" % rules_scala_version,
sha256 = "db536b9db36b5aa737db9d08fa05d1fa5531c9cf213b04bed4e9b9fc34cc2390",
sha256 = "37eb013ea3e6a940da70df43fe2dd6f423d1ac0849042aa586f9ac157321018d",
patches = [
"@com_github_digital_asset_daml//bazel_tools:scala-escape-jvmflags.patch",
],
Expand Down
2 changes: 1 addition & 1 deletion dev-env/bin/daml-sdk-head
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ trap cleanup EXIT
# Building here separately so the user can see the build process which could take a while
bazel build $BAZEL_MODE_FLAG //release:sdk-head-tarball.tar.gz

readonly TARBALL=$(bazel info bazel-genfiles $BAZEL_MODE_FLAG)/release/sdk-head-tarball.tar.gz
readonly TARBALL=$(bazel info bazel-bin $BAZEL_MODE_FLAG)/release/sdk-head-tarball.tar.gz
readonly TMPDIR=$(mktemp -d)
mkdir -p $TMPDIR/sdk-head

Expand Down
6 changes: 3 additions & 3 deletions dev-env/windows/manifests/bazel.json
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
{
"homepage": "https://bazel.build",
"version": "0.28.1",
"version": "1.1.0",
"license": "Apache-2.0",
"bin": "bazel.exe",
"architecture": {
"64bit": {
"url": "https://github.com/bazelbuild/bazel/releases/download/0.28.1/bazel-0.28.1-windows-x86_64.zip",
"hash": "2cb949887f0c8aa5039bb39f52a13a9a13cfcd019d42f65ee16dd134d41c0f72"
"url": "https://github.com/bazelbuild/bazel/releases/download/1.1.0/bazel-1.1.0-windows-x86_64.zip",
"hash": "6ef3b5756ba9d8b26fc599884800aa9c55948478f9574540acba7112e0329db2"
}
},
"depends": [
Expand Down
12 changes: 6 additions & 6 deletions docs/scripts/live-preview.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ ln -s ../source $BUILD_DIR
ln -s ../configs $BUILD_DIR
mkdir $BUILD_DIR/theme
bazel build //docs:theme
tar -zxf ../../bazel-genfiles/docs/da_theme.tar.gz -C $BUILD_DIR/theme
tar -zxf ../../bazel-bin/docs/da_theme.tar.gz -C $BUILD_DIR/theme

# License and Notices
cp ../../LICENSE ../source
Expand All @@ -41,26 +41,26 @@ do
if [ "$arg" = "--pdf" ]; then
bazel build //docs:pdf-docs
mkdir -p $BUILD_DIR/gen/_downloads
cp -L ../../bazel-genfiles/docs/DigitalAssetSDK.pdf $BUILD_DIR/gen/_downloads
cp -L ../../bazel-bin/docs/DigitalAssetSDK.pdf $BUILD_DIR/gen/_downloads
fi
if [ "$arg" = "--gen" ]; then
# Hoogle
bazel build //compiler/damlc:daml-base-hoogle-docs
mkdir -p $BUILD_DIR/gen/hoogle_db
cp -L ../../bazel-genfiles/compiler/damlc/daml-base-hoogle.txt $BUILD_DIR/gen/hoogle_db/base.txt
cp -L ../../bazel-bin/compiler/damlc/daml-base-hoogle.txt $BUILD_DIR/gen/hoogle_db/base.txt

# Javadoc
bazel build //language-support/java:javadocs
mkdir -p $BUILD_DIR/gen/app-dev/bindings-java
tar -zxf ../../bazel-genfiles/language-support/java/javadocs.tar.gz -C $BUILD_DIR/gen/app-dev/bindings-java
tar -zxf ../../bazel-bin/language-support/java/javadocs.tar.gz -C $BUILD_DIR/gen/app-dev/bindings-java

# Proto-docs
bazel build //ledger-api/grpc-definitions:docs
cp -L ../../bazel-genfiles/ledger-api/grpc-definitions/proto-docs.rst ../source/app-dev/grpc/
cp -L ../../bazel-bin/ledger-api/grpc-definitions/proto-docs.rst ../source/app-dev/grpc/

#StdLib
bazel build //compiler/damlc:daml-base-rst-docs
cp -L ../../bazel-genfiles/compiler/damlc/daml-base.rst ../source/daml/reference/base.rst
cp -L ../../bazel-bin/compiler/damlc/daml-base.rst ../source/daml/reference/base.rst
fi
done

Expand Down
2 changes: 1 addition & 1 deletion docs/scripts/preview.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,6 @@ rm -rf $BUILD_DIR
mkdir $BUILD_DIR

bazel build //docs:docs
tar -zxf ../../bazel-genfiles/docs/html.tar.gz -C $BUILD_DIR
tar -zxf ../../bazel-bin/docs/html.tar.gz -C $BUILD_DIR
cd $BUILD_DIR/html
python -m http.server 8000
2 changes: 1 addition & 1 deletion ledger-api/grpc-definitions/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ ledger_api_proto_source_root = "ledger-api/grpc-definitions"
proto_library(
name = "protos",
srcs = glob(["**/*.proto"]),
proto_source_root = ledger_api_proto_source_root,
strip_import_prefix = "/" + ledger_api_proto_source_root,
visibility = [
"//visibility:public",
],
Expand Down
12 changes: 7 additions & 5 deletions ledger-service/http-json/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,13 @@ da_scala_test(
name = "tests",
size = "medium",
srcs = glob(["src/test/scala/**/*.scala"]),
data = ["//docs:quickstart-model.dar"],
data = [
"//docs:quickstart-model.dar",
"@postgresql_dev_env//:all",
"@postgresql_dev_env//:createdb",
"@postgresql_dev_env//:initdb",
"@postgresql_dev_env//:pg_ctl",
],
resources = glob(["src/test/resources/**/*"]),
scalacopts = hj_scalacopts,
deps = [
Expand All @@ -82,10 +88,6 @@ da_scala_test(
"//ledger/sandbox:sandbox-scala-tests-lib",
"//ledger/participant-state",
"//bazel_tools/runfiles:scala_runfiles",
"@postgresql_dev_env//:all",
"@postgresql_dev_env//:createdb",
"@postgresql_dev_env//:initdb",
"@postgresql_dev_env//:pg_ctl",
] + http_json_deps,
)

Expand Down
Loading