Skip to content

Enable autodiff in ci for all major os #152768

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rust-bors merged 3 commits into from
Feb 24, 2026
Merged

Enable autodiff in ci for all major os #152768

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7daabfd
adjust libdir for windows, use lld during enzyme build when needed
ZuseZ4 Feb 18, 2026
4bf4ff1
Enable autodiff in CI for mingw-llvm,linux
ZuseZ4 Feb 19, 2026
c033de9
Move aarch64-apple dist builder to dynamic llvm linking and enable au…
ZuseZ4 Feb 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions src/bootstrap/src/core/build_steps/llvm.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ use crate::core::config::{Config, TargetSelection};
use crate::utils::build_stamp::{BuildStamp, generate_smart_stamp_hash};
use crate::utils::exec::command;
use crate::utils::helpers::{
self, exe, get_clang_cl_resource_dir, t, unhashed_basename, up_to_date,
self, exe, get_clang_cl_resource_dir, libdir, t, unhashed_basename, up_to_date,
};
use crate::{CLang, GitRepo, Kind, trace};

Expand Down Expand Up @@ -561,7 +561,6 @@ impl Step for Llvm {
}
};

// FIXME(ZuseZ4): Do we need that for Enzyme too?
// When building LLVM with LLVM_LINK_LLVM_DYLIB for macOS, an unversioned
// libLLVM.dylib will be built. However, llvm-config will still look
// for a versioned path like libLLVM-14.dylib. Manually create a symbolic
Expand Down Expand Up @@ -1159,7 +1158,7 @@ impl Step for Enzyme {
let llvm_version_major = llvm::get_llvm_version_major(builder, &host_llvm_config);
let lib_ext = std::env::consts::DLL_EXTENSION;
let libenzyme = format!("libEnzyme-{llvm_version_major}");
let build_dir = out_dir.join("lib");
let build_dir = out_dir.join(libdir(target));
let dylib = build_dir.join(&libenzyme).with_extension(lib_ext);

trace!("checking build stamp to see if we need to rebuild enzyme artifacts");
Expand Down Expand Up @@ -1197,7 +1196,16 @@ impl Step for Enzyme {
// hard to spot more relevant issues.
let mut cflags = CcFlags::default();
cflags.push_all("-Wno-deprecated");
configure_cmake(builder, target, &mut cfg, true, LdFlags::default(), cflags, &[]);

// Logic copied from `configure_llvm`
// ThinLTO is only available when building with LLVM, enabling LLD is required.
// Apple's linker ld64 supports ThinLTO out of the box though, so don't use LLD on Darwin.
let mut ldflags = LdFlags::default();
if builder.config.llvm_thin_lto && !target.contains("apple") {
ldflags.push_all("-fuse-ld=lld");
}

configure_cmake(builder, target, &mut cfg, true, ldflags, cflags, &[]);

// Re-use the same flags as llvm to control the level of debug information
// generated by Enzyme.
Expand Down
2 changes: 1 addition & 1 deletion src/ci/docker/host-aarch64/dist-aarch64-linux/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ ENV RUST_CONFIGURE_ARGS \

ENV SCRIPT python3 ../x.py build --set rust.debug=true opt-dist && \
./build/$HOSTS/stage1-tools-bin/opt-dist linux-ci -- python3 ../x.py dist \
--host $HOSTS --target $HOSTS --include-default-paths build-manifest bootstrap
--host $HOSTS --target $HOSTS --include-default-paths build-manifest bootstrap enzyme

ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=clang
ENV LIBCURL_NO_PKG_CONFIG 1
Expand Down
1 change: 1 addition & 0 deletions src/ci/docker/host-x86_64/dist-x86_64-linux/dist.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ python3 ../x.py build --set rust.debug=true opt-dist
--include-default-paths \
build-manifest \
bootstrap \
enzyme \
rustc_codegen_gcc

# Use GCC for building GCC components, as it seems to behave badly when built with Clang
Expand Down
7 changes: 4 additions & 3 deletions src/ci/github-actions/jobs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -504,7 +504,7 @@ auto:
- name: dist-aarch64-apple
env:
SCRIPT: >-
./x.py dist bootstrap
./x.py dist bootstrap enzyme
--include-default-paths
--host=aarch64-apple-darwin
--target=aarch64-apple-darwin
Expand All @@ -513,6 +513,7 @@ auto:
--enable-sanitizers
--enable-profiler
--set rust.jemalloc
--set llvm.link-shared=true
Copy link
Member

@RalfJung RalfJung Feb 25, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a highly non-trivial change with a risk of all sorts of unexpected side-effects. It's odd that this isn't even mentioned in the PR description, and it definitely should only have landed in a rollup=never PR to facilitate bisecting. I think this is the cause for #153077 but this PR became part of a gigantic rollup so I am not sure.

Copy link
Contributor

@madsmtm madsmtm Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that this shouldn't be done for several reasons, including also performance.

At the very least, it should only be done when building enzyme (e.g. in a separate build step).

Copy link
Member

@bjorn3 bjorn3 Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the very least, it should only be done when building enzyme (e.g. in a separate build step).

Enzyme needs to be linked against the same LLVM as rustc given that it is an LLVM plugin, yet we we don't want to statically link it into rustc, such that we can put it in a separate rustup component that isn't installed by default.

--set rust.lto=thin
--set rust.codegen-units=1
# Aarch64 tooling only needs to support macOS 11.0 and up as nothing else
Expand Down Expand Up @@ -706,7 +707,7 @@ auto:
# i686 has no dedicated job, build it here because this job is fast
- name: dist-aarch64-llvm-mingw
env:
SCRIPT: python x.py dist bootstrap --include-default-paths
SCRIPT: python x.py dist bootstrap enzyme --include-default-paths
RUST_CONFIGURE_ARGS: >-
--build=aarch64-pc-windows-gnullvm
--target=aarch64-pc-windows-gnullvm,i686-pc-windows-gnullvm
Expand All @@ -720,7 +721,7 @@ auto:

- name: dist-x86_64-llvm-mingw
env:
SCRIPT: python x.py dist bootstrap --include-default-paths
SCRIPT: python x.py dist bootstrap enzyme --include-default-paths
RUST_CONFIGURE_ARGS: >-
--build=x86_64-pc-windows-gnullvm
--enable-full-tools
Expand Down
Loading