Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Illumos support #31078

Merged
merged 6 commits into from
Feb 4, 2016
Merged

Add Illumos support #31078

merged 6 commits into from
Feb 4, 2016

Conversation

nbaksalyar
Copy link
Contributor

This pull request adds support for Illumos-based operating systems: SmartOS, OpenIndiana, and others. For now it's x86-64 only, as I'm not sure if 32-bit installations are widespread. This PR is based on #28589 by @potatosalad, and also closes #21000, #25845, and #25846.

Required changes in libc are already merged: rust-lang/libc#138

Here's a snapshot required to build a stage0 compiler:
https://s3-eu-west-1.amazonaws.com/nbaksalyar/rustc-sunos-snapshot.tar.gz
It passes all checks from make check.

There are some changes I'm not quite sure about, e.g. macro usage in src/libstd/num/f64.rs and DirEntry structure in src/libstd/sys/unix/fs.rs, so any comments on how to rewrite it better would be greatly appreciated.

Also, LLVM configure script might need to be patched to build it successfully, or a pre-built libLLVM should be used. Some details can be found here: https://llvm.org/bugs/show_bug.cgi?id=25409

Thanks!

r? @brson

@rust-highfive
Copy link
Collaborator

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @brson (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

let mut base = super::sunos_base::opts();
base.pre_link_args.push("-m64".to_string());
base.pre_link_args.push("-lsocket".to_string());
base.pre_link_args.push("-lposix4".to_string());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sorts of "default libraries" are typically linked where they're needed rather than by the compiler unconditionally. Do you know what symbols these are pulling in? I suspect they're only needed by the standard library so could the linkage directives go there instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, these are mostly for socket functions and e.g. fork.
Could you please point out to the correct place to move it in libstd?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing, they'll probably reside in rtdeps.rs.

Out of curiosity, is sunos posix? Or would a "true port" of libstd contain an entirely different implementation of std::sys basically (e.g. how Windows is entirely separate from Unix)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, completely missed it, thank you!

is sunos posix?

Yes, absolutely. I believe historically it's derived from the Unix SVR4 codebase, and mostly compatible with BSD-like OSes. And, as I understand it, at least Illumos (the open source Solaris fork) strives to be maximally compatible with other *nixes, and it includes e.g. GCC toolkit by default instead of some proprietary compilers. Actually, I'm not even sure that this port would work on the original Sun/Oracle Solaris - there are much more diversions from standards.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok cool, sounds good to me!

@alexcrichton
Copy link
Member

Quite exciting, thanks @nbaksalyar!

@nbaksalyar
Copy link
Contributor Author

Thanks for the review, Alex! :)

@@ -1,5 +1,13 @@
#!/bin/sh

# /bin/sh on Solaris is not a POSIX compatible shell, but /usr/bin/ksh is.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ksh -> bash

@brson
Copy link
Contributor

brson commented Jan 22, 2016

lgtm r? @alexcrichton

@rust-highfive rust-highfive assigned alexcrichton and unassigned brson Jan 22, 2016
self // log2(Inf) = Inf
} else {
NAN // log2(-Inf) = NaN
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sorry I was just indicating that the macro could probably be removed in favor of generic functions and/or otherwise non-meta-programming pieces. Could the duplication between the functions here be reduced?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, sorry, got it - I think it should be possible to just pass a reference to a corresponding intrinsic function. Will give it a try in a moment.

EDIT: Actually, it turns out I totally missed your comment on this regard - sorry, will fix it. Thanks! :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It hasn't worked with a generic signature fn foo<F: Fn(f64) -> f64>(val: f64, f: F) -> f64, because the intrinsic functions are unsafe & extern - they don't implement the Fn trait. However, it worked this way: fn foo(self, log_fn: unsafe extern "rust-intrinsic" fn(f64) -> f64) -> f64 - i.e., with a completely defined type in a function reference.

Would this work as a solution, or should I find a better way?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah you may want to all it like:

foo(val, |f| unsafe { intrinsics::doit(f) })

(or something like that). I think there are other bugs for taking intrinsics by-value unfortunately

@bors
Copy link
Contributor

bors commented Jan 27, 2016

☔ The latest upstream changes (presumably #31120) made this pull request unmergeable. Please resolve the merge conflicts.

@alexcrichton
Copy link
Member

Ok cool, I think this is getting pretty close to being good-to-go. I've got a few final questions about this:

  • The target being passed down to LLVM is x86_64-pc-solaris2.11, but could you elaborate on what the 2.11 is?
  • The rustc target is being called x86_64-sun-solaris, but is this the standard name? This does seem to follow the "arch-vendor-os" pattern, right? I'm not sure if something like x86_64-unknown-solaris is really a thing
  • The target_os #[cfg] is currently called "sunos", but that string doesn't appear in the LLVM or rustc target, and going strictly by the way "target triples" are laid out this should technically be solaris, so I'm curious what led you to "sunos"?

@potatosalad
Copy link

@alexcrichton I might be able to add a little explanation to your questions:

  • The target being passed down to LLVM is x86_64-pc-solaris2.11, but could you elaborate on what the 2.11 is?

The 2.* prefix was historically used as an internal representation of the "compatibility" with the underlying version of solaris. The 5.* prefix, on the other hand, is typically the one returned by things like uname -a. So 2.10 was the same as 5.10 which also meant "Solaris 10" and 2.11, 5.11, and "Solaris 11" have the same meaning.

So, it's essentially specifying that it can run on a "Solaris 11" compatible system. The same binary built for a current illumos system in many cases will run just fine on an Oracle Solaris 11 system without an issue.

  • The rustc target is being called x86_64-sun-solaris, but is this the standard name? This does seem to follow the "arch-vendor-os" pattern, right? I'm not sure if something like x86_64-unknown-solaris is really a thing

I don't know about anywhere that it's an official standard, but based on Google search results alone (which I'm sure can always be trusted as a metric 😉):

  • The target_os #[cfg] is currently called "sunos", but that string doesn't appear in the LLVM or rustc target, and going strictly by the way "target triples" are laid out this should technically be solaris, so I'm curious what led you to "sunos"?

The difference in terms of "solaris" and "sunos" might be more useful in the future, but are really no different than "darwin" and "macos" or "mingw32" and "windows".

The benefit is that if there is ever a future target triple for something like x86_64-unknown-illumos mapping "illumos" to "sunos" makes more sense than mapping it to "solaris". Historically, "sunos" has been used to describe the "type" of operating system (like "bsd"), rather than the actual operating system (like "solaris" or "freebsd").

However, beyond that I don't know if there is any explicit reason to use "sunos" over "solaris" for the target_os #[cfg].

@nbaksalyar
Copy link
Contributor Author

I've got a few final questions about this:

@potatosalad has already answered them all, but I would add my 2c:

The rustc target is being called x86_64-sun-solaris, but is this the standard name?

Other than Google results, that's the standard build target for GCC on Illumos (only with addition of version 2.11). E.g. compiler tools are called like x86_64-sun-solaris2.11-gcc. I'm not sure that the version would be needed in this case, though - I believe it's optional.

going strictly by the way "target triples" are laid out this should technically be solaris, so I'm curious what led you to "sunos"?

@potatosalad is correct, it's more like "darwin". Solaris and Illumos self-identify with uname as "SunOS", which is for compatibility reasons (as SunOS is a completely different branch of operating systems, based on BSD rather than SVR4 Unix).

Technically, I think solaris would be more appropriate here, but now Solaris is a proprietary operating system that belongs to Oracle. And this port is intended to work on its open source derivatives too (SmartOS, OmniOS, OpenIndiana, etc.), so I was a bit reluctant to choose solaris as a tag.

But any would work here, so let me know if you want me to change it to some better name.

Hopefully that explains it a bit.
Thanks!

@alexcrichton
Copy link
Member

@potatosalad, @nbaksalyar

Thanks for the info! That's certainly quite enlightenting.

Would it be possible to not pass down 2.11 to LLVM then? I don't think we do this for other platforms, and then LLVM would pick the "default level" I guess in theory. Or is solaris different where the OS version has major changes in ABI for things like thread locals each revision?

Also after some discussion with @brson, we think that the target_os should probably follow the os portion of the target triple itself, so in this case perhaps we can stick to solaris as the triple name is x86_64-sun-solaris?

@nbaksalyar
Copy link
Contributor Author

@alexcrichton

so in this case perhaps we can stick to solaris as the triple name is x86_64-sun-solaris?

Sure! I will make required changes and resolve the merge conflict then.

@bors
Copy link
Contributor

bors commented Jan 31, 2016

☔ The latest upstream changes (presumably #31298) made this pull request unmergeable. Please resolve the merge conflicts.

@nbaksalyar
Copy link
Contributor Author

I've rebased the commits - should be ready to go!
Please let me know if something has to be improved or changed.

@@ -25,6 +24,10 @@ use sys::platform::raw;
use sys::{cvt, cvt_r};
use sys_common::{AsInner, FromInner};
use vec::Vec;
#[cfg(target_os = "solaris")]
use core_collections::borrow::ToOwned;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can remove the #[cfg] and the one below by just adding use prelude::v1::* to the top of this file (these are both in the prelude), and then you don't need the #[cfg] as well I believe.

@alexcrichton
Copy link
Member

Looks good to me, thanks @nbaksalyar! I had just one tiny nit but it's not very relevant, so I'm gonna send this to bors:

@bors: r+ bb6e646

@bors
Copy link
Contributor

bors commented Jan 31, 2016

⌛ Testing commit bb6e646 with merge 78d0184...

bors added a commit that referenced this pull request Jan 31, 2016
This pull request adds support for [Illumos](http://illumos.org/)-based operating systems: SmartOS, OpenIndiana, and others. For now it's x86-64 only, as I'm not sure if 32-bit installations are widespread. This PR is based on #28589 by @potatosalad, and also closes #21000, #25845, and #25846.

Required changes in libc are already merged: rust-lang/libc#138

Here's a snapshot required to build a stage0 compiler:
https://s3-eu-west-1.amazonaws.com/nbaksalyar/rustc-sunos-snapshot.tar.gz
It passes all checks from `make check`.

There are some changes I'm not quite sure about, e.g. macro usage in `src/libstd/num/f64.rs` and `DirEntry` structure in `src/libstd/sys/unix/fs.rs`, so any comments on how to rewrite it better would be greatly appreciated.

Also, LLVM configure script might need to be patched to build it successfully, or a pre-built libLLVM should be used. Some details can be found here: https://llvm.org/bugs/show_bug.cgi?id=25409

Thanks!

r? @brson
@bors
Copy link
Contributor

bors commented Jan 31, 2016

💔 Test failed - auto-mac-ios-opt

@nbaksalyar
Copy link
Contributor Author

The issue should be fixed by now. Sorry it took so long - it turned out that I was building the bootstrap compiler from an incorrect base (the issue is described in #31142). So actually to run make check with the new compiler version I have to rebase my branch upon the same commit that other snapshots use.

I've yet to make the new stage0 work correctly (it fails with SIGILL for now, for some reason). I guess the reason is that the codebase with the latest commits, on which I based my work, has diverged substantially. I'll try to find & fix the reason, but actually it might be better to wait for a snapshot update before rebuilding the bootstrapping compiler and adding it to the list of snapshots.

Other than that, it should work correctly, as the only major change from my previous snapshot is renaming "sunos" to "solaris" (and that's why it wouldn't work).

@alexcrichton
Copy link
Member

@bors: r+ fae883c

Ok, sounds good to me!

bors added a commit that referenced this pull request Feb 3, 2016
This pull request adds support for [Illumos](http://illumos.org/)-based operating systems: SmartOS, OpenIndiana, and others. For now it's x86-64 only, as I'm not sure if 32-bit installations are widespread. This PR is based on #28589 by @potatosalad, and also closes #21000, #25845, and #25846.

Required changes in libc are already merged: rust-lang/libc#138

Here's a snapshot required to build a stage0 compiler:
https://s3-eu-west-1.amazonaws.com/nbaksalyar/rustc-sunos-snapshot.tar.gz
It passes all checks from `make check`.

There are some changes I'm not quite sure about, e.g. macro usage in `src/libstd/num/f64.rs` and `DirEntry` structure in `src/libstd/sys/unix/fs.rs`, so any comments on how to rewrite it better would be greatly appreciated.

Also, LLVM configure script might need to be patched to build it successfully, or a pre-built libLLVM should be used. Some details can be found here: https://llvm.org/bugs/show_bug.cgi?id=25409

Thanks!

r? @brson
@bors
Copy link
Contributor

bors commented Feb 3, 2016

⌛ Testing commit fae883c with merge e3bcddb...

@nbaksalyar
Copy link
Contributor Author

Awesome! @alexcrichton, @potatosalad, and everyone - thanks for your help! :)
Next step - a working compiler snapshot and Cargo.

@dhuseby
Copy link

dhuseby commented Feb 9, 2016

@nbaksalyar have you successfully bootstrapped rustc on illumos yet?

@nbaksalyar
Copy link
Contributor Author

@dhuseby
Not for the recent versions of Rust. There's a working compiler (the link can be found in the first comment to this PR), but it expects target_os to be sunos, which is changed to solaris now. Besides, it was built from an incorrect source base - as it turns out, it has to be the same commit as the other snapshots use. Current WIP branch can be found here: https://github.com/nbaksalyar/rust/tree/illumos-stage0
It segfaults for now, but I think I'll have a working compiler this week.

@nbaksalyar nbaksalyar deleted the illumos branch February 21, 2016 22:59
@dhuseby
Copy link

dhuseby commented Mar 21, 2016

@nbaksalyar any update on the WIP branch?

@nbaksalyar
Copy link
Contributor Author

@dhuseby
Yes! @jperkin has built a Rust package for SmartOS, it's available in pkgsrc.
I plan to add it to the official list of snaphots soon & I'm working on a Cargo port for Illumos/Solaris.

@alexcrichton
Copy link
Member

@nbaksalyar oh I should also mention that we've recently got cross-compiled nightlies from Linux to FreeBSD/NetBSD up and running, so we have nightly builds of rustc and the standard library for FreeBSD and NetBSD. If there's a cross compiler from Linux to Illumos then I'd be game to set one up as well for Illumos!

@cneira
Copy link

cneira commented Jan 12, 2018

@alexcrichton
Maybe this could help https://gist.github.com/AlainODea/6679613

@jperkin
Copy link

jperkin commented Jan 12, 2018

FWIW we've been shipping rust/cargo packages in SmartOS for a while now, currently 1.22.1 but 1.23 should be ready soon. Our binaries should work on any illumos platform built after 20141030.

@despair86
Copy link

despair86 commented Jan 21, 2019

...as I'm not sure if 32-bit installations are widespread.

not since 2.11, but a big chunk of userspace is still 32-bit (GNU CC always builds a 32-bit multilib-enabled compiler, for instance)

@nbaksalyar: do you still have the target spec JSON files originally used to bootstrap this port? Looking into an i386-sun-solaris2.11 build (and finally rein in memory usage of the net's oldest browser at last: locking 6GB+1 segment at startup is in poor taste on an 8GB laptop)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot compile on SunOS
10 participants