[AIX] change system dynamic library format #132362

mustartt · 2024-10-30T14:32:37Z

Historically on AIX, almost all dynamic libraries are distributed in .a Big Archive Format which can consists of both static and shared objects in the same archive (e.g. libc++abi.a(libc++abi.so.1)). During the initial porting process, the dynamic libraries are kept as .a to simplify the migration, but semantically having an XCOFF object under the archive extension is wrong. For crate type cdylib we want to be able to distribute the libraries as archives as well.

We are migrating to archives with the following format:

$ ar -t lib<name>.a
lib<name>.so

where each archive contains a single member that is a shared XCOFF object that can be loaded.

rustbot · 2024-10-30T14:32:46Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @jieyouxu (or someone else) some time within the next two weeks.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

@rustbot author: the review is finished, PR author should check the comments and take action accordingly
@rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

jieyouxu · 2024-10-31T09:01:23Z

Hi @mustartt, is this PR blocked on something?

src/bootstrap/src/utils/helpers.rs

daltenty · 2024-10-31T17:01:23Z

@rustbot author

mustartt · 2024-11-05T17:42:57Z

Keeping this as draft for now. Need to investigate if dylibs should be archived into .a and how to properly detect between static archives and archives that contains shared objects.

mustartt · 2024-11-19T14:04:11Z

@rustbot review

gilamn5tr

Looks good in terms of the AIX implementation.

jieyouxu

Thanks, this LGTM overall, left a few nits/questions.

compiler/rustc_codegen_ssa/src/back/link.rs

compiler/rustc_metadata/src/creader.rs

jieyouxu · 2024-11-20T09:06:55Z

src/bootstrap/src/utils/helpers.rs

+    // todo: reading the entire file as &[u8] into memory seems excessive
+    // look into either mmap it or use the &CacheReader


Suggestion: can you convert this to a FIXME and open a C-cleanup issue then link to it FIXME(#nnnnnn): ...?

jieyouxu · 2024-11-20T09:09:49Z

@rustbot author

rustbot · 2024-11-21T01:03:58Z

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

mustartt · 2024-11-21T01:08:29Z

Cargo.lock

@@ -1965,9 +1965,9 @@ checksum = "baff4b617f7df3d896f97fe922b64817f6cd9a756bb81d40f8883f2f66dcb401"

 [[package]]
 name = "libc"
-version = "0.2.161"
+version = "0.2.164"


libc 0.2.163 contains the new RTLD_MEMBER

mustartt · 2024-11-21T02:44:28Z

Thanks for the comments. Rebase on newer master to avoid llvm assertion for f128. Bumped up libc to 0.2.164 that contains the new aix RTLD_MEMBER.

@rustbot review

fmt fix cfg for windows remove unused imports address comments update libc to 0.2.164 fmt remove unused imports

jieyouxu

Thanks. My only reservation is having to load the whole shared archive in bootstrap for dylib detection, but since that only happens for .a I suppose it's not too bad, and has an associated cleanup issue #133268, so I think that's acceptable.

jieyouxu · 2024-11-21T15:52:52Z

@bors r+ rollup=never (this PR contains a libc bump)

bors · 2024-11-21T15:52:55Z

📌 Commit 0db9059 has been approved by jieyouxu

It is now in the queue for this repository.

mustartt · 2024-11-21T16:11:00Z

Thanks. My only reservation is having to load the whole shared archive in bootstrap for dylib detection, but since that only happens for .a I suppose it's not too bad, and has an associated cleanup issue #133268, so I think that's acceptable.

It have a pretty small impact on the bootstrap time maybe an additional minute at most. But I'm wondering is the bootstrap process okay with including other dependencies such as memmap2? I was a bit hesitant to include it in the initial patch?

jieyouxu · 2024-11-21T16:26:12Z

But I'm wondering is the bootstrap process okay with including other dependencies such as memmap2? I was a bit hesitant to include it in the initial patch?

Ideally bootstrap needs to build fast with minimal deps, so I'm inclined to say no, not for just this one purpose.

bors · 2024-11-21T17:17:49Z

⌛ Testing commit 0db9059 with merge f39baca...

…ouxu [AIX] change system dynamic library format Historically on AIX, almost all dynamic libraries are distributed in `.a` Big Archive Format which can consists of both static and shared objects in the same archive (e.g. `libc++abi.a(libc++abi.so.1)`). During the initial porting process, the dynamic libraries are kept as `.a` to simplify the migration, but semantically having an XCOFF object under the archive extension is wrong. For crate type `cdylib` we want to be able to distribute the libraries as archives as well. We are migrating to archives with the following format: ``` $ ar -t lib<name>.a lib<name>.so ``` where each archive contains a single member that is a shared XCOFF object that can be loaded.

rust-log-analyzer · 2024-11-21T18:45:11Z

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

bors · 2024-11-21T18:45:45Z

💔 Test failed - checks-actions

jieyouxu · 2024-11-21T19:02:00Z

@bors retry (not even trying)

bors · 2024-11-21T21:36:50Z

⌛ Testing commit 0db9059 with merge 5d3c6ee9b34989595d2a72b79e61ca37e949d757...

…ouxu [AIX] change system dynamic library format Historically on AIX, almost all dynamic libraries are distributed in `.a` Big Archive Format which can consists of both static and shared objects in the same archive (e.g. `libc++abi.a(libc++abi.so.1)`). During the initial porting process, the dynamic libraries are kept as `.a` to simplify the migration, but semantically having an XCOFF object under the archive extension is wrong. For crate type `cdylib` we want to be able to distribute the libraries as archives as well. We are migrating to archives with the following format: ``` $ ar -t lib<name>.a lib<name>.so ``` where each archive contains a single member that is a shared XCOFF object that can be loaded.

rustbot assigned jieyouxu Oct 30, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Oct 30, 2024

This comment has been minimized.

Sign in to view

daltenty suggested changes Oct 31, 2024

View reviewed changes

src/bootstrap/src/utils/helpers.rs Outdated Show resolved Hide resolved

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 31, 2024

jieyouxu added the O-aix OS: Big Blue's Advanced Interactive eXecutive.. label Oct 31, 2024

mustartt force-pushed the aix-dylib-detection branch from 45354fd to 7af8f2e Compare November 18, 2024 03:15

mustartt changed the title ~~AIX: extend bootstrap is_dylib to support XCOFF format~~ [AIX] change system dynamic library format Nov 18, 2024

This comment has been minimized.

Sign in to view

mustartt requested a review from daltenty November 18, 2024 03:26

This comment has been minimized.

Sign in to view

mustartt marked this pull request as ready for review November 18, 2024 20:03

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Nov 19, 2024

gilamn5tr reviewed Nov 19, 2024

View reviewed changes

jieyouxu approved these changes Nov 20, 2024

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 20, 2024

mustartt force-pushed the aix-dylib-detection branch from 3058de9 to d999999 Compare November 21, 2024 01:03

mustartt commented Nov 21, 2024

View reviewed changes

This comment has been minimized.

Sign in to view

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Nov 21, 2024

aix: fix archive format

0db9059

fmt fix cfg for windows remove unused imports address comments update libc to 0.2.164 fmt remove unused imports

mustartt force-pushed the aix-dylib-detection branch from 25da642 to 0db9059 Compare November 21, 2024 15:33

jieyouxu approved these changes Nov 21, 2024

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 21, 2024

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Nov 21, 2024

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[AIX] change system dynamic library format #132362

[AIX] change system dynamic library format #132362

mustartt commented Oct 30, 2024 •

edited

Loading

rustbot commented Oct 30, 2024

This comment has been minimized.

jieyouxu commented Oct 31, 2024

daltenty commented Oct 31, 2024

mustartt commented Nov 5, 2024

This comment has been minimized.

This comment has been minimized.

mustartt commented Nov 19, 2024

gilamn5tr left a comment

jieyouxu left a comment

jieyouxu Nov 20, 2024 •

edited

Loading

jieyouxu commented Nov 20, 2024

rustbot commented Nov 21, 2024

mustartt Nov 21, 2024

This comment has been minimized.

mustartt commented Nov 21, 2024 •

edited

Loading

jieyouxu left a comment •

edited

Loading

jieyouxu commented Nov 21, 2024

bors commented Nov 21, 2024

mustartt commented Nov 21, 2024 •

edited

Loading

jieyouxu commented Nov 21, 2024

bors commented Nov 21, 2024

rust-log-analyzer commented Nov 21, 2024

bors commented Nov 21, 2024

jieyouxu commented Nov 21, 2024

bors commented Nov 21, 2024

		// todo: reading the entire file as &[u8] into memory seems excessive
		// look into either mmap it or use the &CacheReader

[AIX] change system dynamic library format #132362

Are you sure you want to change the base?

[AIX] change system dynamic library format #132362

Conversation

mustartt commented Oct 30, 2024 • edited Loading

rustbot commented Oct 30, 2024

This comment has been minimized.

jieyouxu commented Oct 31, 2024

daltenty commented Oct 31, 2024

mustartt commented Nov 5, 2024

This comment has been minimized.

This comment has been minimized.

mustartt commented Nov 19, 2024

gilamn5tr left a comment

Choose a reason for hiding this comment

jieyouxu left a comment

Choose a reason for hiding this comment

jieyouxu Nov 20, 2024 • edited Loading

Choose a reason for hiding this comment

jieyouxu commented Nov 20, 2024

rustbot commented Nov 21, 2024

mustartt Nov 21, 2024

Choose a reason for hiding this comment

This comment has been minimized.

mustartt commented Nov 21, 2024 • edited Loading

jieyouxu left a comment • edited Loading

Choose a reason for hiding this comment

jieyouxu commented Nov 21, 2024

bors commented Nov 21, 2024

mustartt commented Nov 21, 2024 • edited Loading

jieyouxu commented Nov 21, 2024

bors commented Nov 21, 2024

rust-log-analyzer commented Nov 21, 2024

bors commented Nov 21, 2024

jieyouxu commented Nov 21, 2024

bors commented Nov 21, 2024

mustartt commented Oct 30, 2024 •

edited

Loading

jieyouxu Nov 20, 2024 •

edited

Loading

mustartt commented Nov 21, 2024 •

edited

Loading

jieyouxu left a comment •

edited

Loading

mustartt commented Nov 21, 2024 •

edited

Loading