-
Notifications
You must be signed in to change notification settings - Fork 200
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix some logger issues #1739
Fix some logger issues #1739
Changes from all commits
470865f
9c59c06
ecf455f
79d3424
2e77da3
59fe89a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -26,13 +26,13 @@ requirements: | |
- {{ stdlib("c") }} | ||
host: | ||
- cuda-version ={{ cuda_version }} | ||
# We require spdlog and fmt (which was de-vendored from spdlog | ||
# conda-forge packages in 1.11.0) so that the spdlog headers are not | ||
# pulled by CPM and installed as a part of the rmm packages. However, | ||
# building against librmm still requires these headers. They are also | ||
# added as a run requirement via the packages' run_exports. | ||
# We need fmt here for now because the conda spdlog package is hard-coded | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. But we removed the spdlog package dependencies…? So this is an “ABI constraint” that is meant to align downstream packages that depend on spdlog, even though fmt is not used/linked as a shared dependency in librmm? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Your comment isn't correct, but I'm afraid I don't fully understand your explanation so I may not be able to address your confusion correctly. Basically what is happening is that (because rmm is an interface target) it becomes part of downstream packages build to find spdlog. When that package, say cudf, finds a conda-provided spdlog in the environment, that version of spdlog says to use the fmt shared library even if you use the spdlog_header_only target. Therefore, we actually do have an fmt shared library dependency. That is what I tried to explain in the PR description. Does that help? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh I'm sorry I just realized that there is a typo here. The first sentence should read "The conda package for spdlog is hard-coded to assume that we use a preexisting fmt library" There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. But the spdlog conda package already has a dependency on fmt. Why not just let that dependency suffice? Why add it to rmm? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. And spdlog run-exports itself so two different consumers of spdlog will only be usable at runtime with compatible spdlog and fmt versions — and that’s enforced by the solver / dependency graph. https://github.com/conda-forge/spdlog-feedstock/blob/97ea289e6733b60e4c619e9cb1e0ebc329358936/recipe/meta.yaml#L18 I’m not convinced we need a fmt dependency here. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I realized I never answered this question. The problem is that spdlog could be in the build environment as a transitive dependency even though we no longer have it as a dependency at all, resulting in linkage to fmt without spdlog or fmt being part of the runtime dependency list of our conda package (run exports aren't transitive). But yes, you could also fix this problem by simply putting spdlog back into the dependency list right now. |
||
# to use fmt as a compiled library, not header-only, so we must ensure that | ||
# the library is present for now so that if a downstream library tries to | ||
# build against rmm and some other package in its build environment uses | ||
# fmt (or spdlog) that the default rmm build is consistent with such | ||
# environments. | ||
- fmt {{ fmt_version }} | ||
- spdlog {{ spdlog_version }} | ||
|
||
build: | ||
script_env: | ||
|
@@ -77,8 +77,8 @@ outputs: | |
{% if cuda_major == "11" %} | ||
- cudatoolkit | ||
{% endif %} | ||
# See comment about fmt in the build section above. | ||
- fmt {{ fmt_version }} | ||
- spdlog {{ spdlog_version }} | ||
test: | ||
commands: | ||
- test -d "${PREFIX}/include/rmm" | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it viable to remove conda package dependencies and always get spdlog/fmt as header-only deps via CPM? That way we should never interact with a local spdlog or local/shared fmt.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No. The problem is that the logger is compiled not by rmm itself but by packages that build against rmm, and we cannot guarantee that spdlog or fmt will never exist in the environment in which that build is taking place. For example, if cuml has some other dependency that requires spdlog, then during cuml's conda build spdlog will be in the environment when its copy of the rmm logger is built. You can think of how the new build works as similar to what we do with gtest/gmock, with the additional complexity that the build is done by the user of rmm rather than rmm itself.
Yes, with caveats. This is what I was alluding to in my message above where I said
The big caveat is that we do not want to require this all the time, otherwise it will make offline builds of any package built against rmm impossible because you will have CPMAddPackage calls embedded in a dependency (the only workarounds would be to manually modify CPM-cloned code inside the build directory). We need to write the CMake so as to make this behavior possible to opt out of.