[WIP] 64-bit support for native board / cpu

[x3ro]

Contribution description

Disclaimer: this is still in the early stages. I'm creating a PR now so that people can try this out themselves.

64-bit support for RIOT native, since some operating systems (read: macOS) and some Linux distros are moving away from easily supporting 32-bit applications.

Testing procedure

cd examples/hello-world && make && make term

Issues/PRs references

Fixes #6603

[x3ro]

So after having gotten this to work experimentally and thinking about how to clean up the implementation, I've reached and impasse, since I feel like the inclusion of 64 bit raises some questions in how native should be structured. Right now there's only native, which is used for Linux on ARM and x86 as well as macOS and FreeBSD. Adding 64 bit support to this would require more ingredients to the #ifdef soup.

I'm wondering if a structure like the following would not make more sense for the future:

•
├── boards
│   ├── common
│   │   └── unix
│   │       ├── vfs
│   │       └── mtd
│   ├── linux
│   │   ├── (includes common/unix)
│   │   ├── can
│   │   └── [...]
│   ├── macos
│   │   └── (includes common/unix)
│   └── [...]
└── cpu
    ├── i386
    └── amd64

This would also remove native as an outlier in how it works in comparison to all other boards/architectures, since e.g. cpu/native currently supports multiple CPUs.

@RIOT-OS/maintainers

[x3ro]

@RIOT-OS/maintainers sorry for spamming all of you, but if I need some input here in order to go forward :)

[kaspar030]

@RIOT-OS/maintainers sorry for spamming all of you, but if I need some input here in order to go forward :)

hmm. It might depend on whether we want to be "native" to always compile for the host system (no matter what it is, that's what happens now, but with only 32bit support) or if we want to freely cross compile all possible combinations. Or even if we want to just identify them individually.

I'll put some thought into it...

[miri64]

Why not have 3 boards, as I originally proposed: native, native32 and native64, with native being an alias dependent on the host platform (which I would prefer for usability reasons). Having the CPU split up in i386/amd64 gives the wrong impression IMHO, as it looks like you could also have an i386-based board (which you couldn't with just the native wrapper). Also: where did the armv7 support go. I'd still like to run native on a Raspberry Pi (maybe even on armv8 (= RPi3/RPi4) if 64-bit support is provided anyway).

[kaspar030]

Having the CPU split up in i386/amd64 gives the wrong impression IMHO, as it looks like you could also have an i386-based board (which you couldn't with just the native wrapper).

hopefully the differences are mostly in the threading code (which has assembly versions), in cpu/native/native_cpu.c and cpu/native/tramp.S. Are the 64bit versions fundamentally different to what is already there, or would they just be an addition?

I propose to keep one cpu/native and boards/native.

[miri64]

I propose to keep one cpu/native and boards/native.

And what if if you want to run a 32-bit native (because closer to a typical board e.g.) on a 64-bit machine?

[kaspar030]

And what if if you want to run a 32-bit native (because closer to a typical board e.g.) on a 64-bit machine?

Make it a compile time option? if ($(uname -m) == x86_64) NATIVE_ARCH ?= x86_64 -> NATIVE_ARCH=i386 make ...?

Why not have 3 boards, as I originally proposed: native, native32 and native64, with native being an alias dependent on the host platform (which I would prefer for usability reasons).

Maybe that makes more sense from the usability perspective. But that would also mean that we'd always have to ask which native was actually used... Could be preferable.

[miri64]

Maybe that makes more sense from the usability perspective. But that would also mean that we'd always have to ask which native was actually used... Could be preferable.

As we would need to ask on which host platform they compiled it... or which compile-time option they used.

[kaspar030]

As we would need to ask on which host platform they compiled it... or which compile-time option they used.

true that...

[x3ro]

If I work on this I'd like to split up the CPUs because it's very confusing to work on it right now, since stuff is littered with ifdefs, and adding arm64 and x64 support would at least add additional ifdefs for

• darwin + arm64
• darwin + x64
• linux + arm64
• [...] you get the idea

It's hard enough to work with (and document) as it is :D While reading this I was wondering if the following would make sense:

1. Have a unix_common that is included by e.g. darwin or macos and linux and which include
2. CPUs called unix_386, unix_arm, unix_arm64 or unix_x64 depending on the desired architecture

🤔

[x3ro]

Ping @kaspar030 @miri64

[miri64]

• Have a unix_common that is included by e.g. darwin or macos and linux and which include

• CPUs called unix_386, unix_arm, unix_arm64 or unix_x64 depending on the desired architecture

Why not keep the name/prefix native_?

[x3ro]

I think it's a misnomer, since native is usually used differently. Although Wikipedia is of course not authorative:

Native (computing) In computing, Native software or data-formats are those that were designed to run on that operating system. For example, a Game Boy receives its software through a cartridge and on this cartridge, is the code that runs natively on the Game Boy.

This is also my understanding of native, and thus essentially we run native code on all of the platforms, not just on Unix. This would be a good time to fix it, I believe.

[x3ro]

Whoops sorry I edited your comment, I meant to edit my comment before that v_v I didn't even know I could edit random people's comments :D

[miri64]

Then UNIX is a misnomer too, as ideally it would run on any host machine and makes the operating system (or family of operating systems) the board somehow ;-).

[miri64]

• this PR should be on providing 64-bit support for native not fixing a (possible) misnomer in addition to that. If you think native should be renamed, we should do this in a separate PR.

[x3ro]

No it's not, since all of the common functionality provided by native is unix specific.

In terms of "this PR is about adding 64-bit": yeah, and I think the way that native is currently structured should be changed in order to do so. It'll be more work to first add more stuff onto the existing structure and then refactor. Also it's never gonna get done because nobody will be around to do the refactoring. I'm offering to put some work into this now, but I don't feel like doing it in a quick-and-dirty way, because that's what has gotten us to the current structure of ifdefs.

Edit: For the record, we can also call it native-common, native-macos, native-linux, etc. I don't really care about the unix part specifically. I just think it's confusing.

[miri64]

Edit: For the record, we can also call it native-common, native-macos, native-linux, etc. I don't really care about the unix part specifically. I just think it's confusing.

And I think it is confusing (for users) to effectively just remove a popular testing platform and replace it with a different one in an architecture update ;-).

[smlng]

s/handler handler/handler/

[smlng]

#ifndef __MACH__?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

[MrKevinWeiss]

@x3ro Can someone else take this over, I think people still want this!

[x3ro]

Hey folks,

TLDR: @MrKevinWeiss Sure, I'd be happy to hand this over to someone else.

I took another peek at this, and here's where I think this PR currently stands:

• The proof-of-concept I wrote works for Ubuntu, but hasn't been tested anywhere else.
• I looked at how my proof-of-concept could translate to macOS and learned that some of the functionality we're using (eg getcontext / setcontext) has been deprecated in macOS since 2009. Try running cat $(xcrun --show-sdk-path)/usr/include/ucontext.h. Basing a new implementation on this seems like a brittle foundation at best.
• The advent of Apple's M1 ARM chip will make this even more complicated (assuming this will also fall under the umbrella of native), since now we'll have to deal with macOS ARM and macOS x64.
• As far as I understand, native doesn't currently work on macOS, since macOS Catalina (released October 2019) dropped support for 32-bit applications altogether.

As far as CPUs go, native is a special case. It currently contains support for macOS, BSD and Linux, the latter on Intel and ARM CPUs, for a total of 4 possible combinations. Adding 64-bit would increase this to 7 possible combinations, 8 if we assume ARM on macOS will also be an option. As I hinted at above, I also think that especially the macOS implementation will likely have to differ in non-trivial ways from the Linux/BSD implementation.

I don't think that jamming all of this into native is a good idea, as

• it will make the implementation more difficult (lots of ifdefs and Makefile shenanigans, several flavors of assembly in the same file - this is already the case)
• it will be more difficult to read, since there'll be multiple interleaved implementations

I understand and generally agree with the argument that individual PRs should concern themselves with a single change, and I also agree that breaking changes aren't great. In this particular case, I think that both are warranted because it'd make this endeavor much easier to pull off. Adding 64-bit support to the current state would certainly be possible, but doing that and then ripping it apart again in a subsequent PR would be much more difficult and, lets admit it, unlikely to happen, since nobody seems super eager to touch this code at all.

From where I'm standing, we should start with

1. Splitting the code into chunks that are linux specific (board?) and those that are arm/x64 specific (cpu?).
2. Throwing away the macOS implementation for now and adding it back once 64-bit is merged and working well for Linux (ARM+x64), and we have a better idea of how an implementation could look that works well for both macOS on Intel and macOS on ARM.

But I'll leave the decision / discussion on this to whoever ends up picking this up.

[miri64]

2\. Throwing away the macOS implementation for now and adding it back once 64-bit is merged and working well for Linux (ARM+x64), and we have a better idea of how an implementation could look that works well for both macOS on Intel and macOS on ARM.

Maybe that could be a good way for now. I think the current state of the MacOS support is suspect anyway, since, while trying to clean-up tapsetup, I noticed that Apple also through out kernel extensions (or at least made them very hard to use for end users), which basically makes it next to impossible to start a TAP or TUN interface, making networking with native not compatible with the Linux approach anymore.

[miri64]

To say it short: Providing 64-bit should be in the first place about future-proofing RIOT for now. 1. Ubuntu already gave a warning shot against 32-bit systems, that though it saw backlash might return in the future, thus is not a MacOS-only problem, and 2. maybe will not only be about native/Linux support, as we have dual-core microprocessors now, so who knows what the future holds with regards to 64-bit support.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

[chrysn]

There is now native64 support since #20315, closing this. Thanks for making the case, and for kicking off work that made that PR easier to complete.