Allow AOT executables to be cross-compiled

[nex3]

Currently application snapshots (as described here) can only be compiled for the architecture of the machine doing the compilation. This makes releasing cross-platform apps using these snapshots much more difficult, requiring multiple machines (including at least one physical machine, since OS X isn't virtualizable) to compile for all platforms.

It seems that the compilation process could do the initial run on the current platform, but then preserve the profiling information and use it to cross-compile the result to other platforms.

[rmacnak-google]

App snapshots are tied to the target ABI of the VM, not the host OS and not the host ABI. If you create an app snapshot using an IA32 VM, you can run the same snapshot on Windows, Mac or Linux. If you create an app snapshot using an X64 VM on Mac or Linux, you run it on the other (they share the sysv ABI), but not Windows (which has its own ABI). If you create an app snapshot using a SIMARM VM on your desktop, you can run it with an ARM VM on, say, an Android device (this is how Flutter works).

We will not produce VMs that target multiple ABIs. The assumption of a single target ABI as a build-time decision is quite deep in the VM.

[nex3]

I want to push back strongly on this. Startup performance is very important for providing my users with a good experience, and the Dart VM's startup performance without using application snapshots compares poorly to just about any other language (including Ruby, whose Sass implementation we're in the process of deprecating because of performance issues).

Other languages are able to do this. Easy cross-compilation is a major reason that Go is so popular for writing command-line tools, and if Dart provided it with similar ease it would make the server-side Dart world very compelling. But without it, Dart users are forced to deal with sub-par performance when they could easily get performance and portability with another language.

[nex3]

Ping... efficient startup speeds across all platforms are a very important customer feature.

[nex3]

I haven't received any response, so I'm re-opening this. It's a pressing customer issue.

[passsy]

One workaround is https://github.com/filiph/dartbin which creates a go wrapper. Not Dart2 compatible though and it requires go as dependency.

[nex3]

Updating to AOT executables, since that's not the best way to distribute CLI applications.

[nex3]

@mit-mit has told me offline that "getting to full cross-compilation is going to be a long journey", but "the journey has already started; we just don't have a timeline for when it will end".

[shinayser]

Looking forward to see this feature! Loving to work with dart ❤

[Schwusch]

This feature is a show stopper for us in order to adopt Dart in AWS Lambda. Since the AWS CLI runs the compiled executable in a Docker container, offline development is only possible in a Linux environment. Manual deployment without CI/CD is also impossible in Mac and Windows.

[t0mmar]

Also a cross-compile option for linux ARM architectures please!

[mit-mit]

Potential work-around that may work for some: https://blog.dantup.com/2019/11/easily-compiling-dart-to-native-executables-for-windows-linux-macos-with-github-actions/

[DrSensor]

Just found this wiki: https://github.com/dart-lang/sdk/wiki/Building-Dart-SDK-for-ARM-processors#building

I wonder if dart2native and the produced executable/snapshot works on RaspberryPi 🤔?

[ifree92]

Ah, that would be awesome to have a cross compilation !
I just was looking for this functionality, but found this issue.
If my vote for this topic can change the priorities for dart2native tools improvement - I would be really glad.

I'm trying to use Dart for several projects for Raspberry, but that's hard to easily win this battle without ability to get ARM binary using my host OS (MacOS).

What's about plan? Will this feature be included once?

[ThomasJaeger]

Is there an update to this? Really interested in cross-compilation!

[jpnurmi]

Would something like this be an interesting mid-term solution?

 $ dart2native foo.dart \
    --gen-snapshot out/ProductSIMARM/dart-sdk/bin/utils/gen_snapshot \
    --aot-runtime out/ProductXARM/dart-sdk/bin/dartaotruntime

jpnurmi@9625310

[mraleph]

Would something like this be an interesting mid-term solution?

That's pretty close to how it would be implemented in the long term too (if we had resources or if we had somebody willing to contribute external implementation). Few missing pieces to take it all the way:

• We need to create X-configurations for producing gen_snapshot for all configurations of host-target OSes and architectures.
• We need to figure out packaging and distribution for these binary artifacts. I don't think including them into SDK is scalable - we need to figure out "out of band" delivery mechanism (which might somewhat complicate Dart SDK release process).

[dvc94ch]

@maks well, I don't think we want to include gen_snapshot and dart_precompiled_runtime pair for each configuration (which is actually a combination of CPU and OS - something that @jpnurmi did not tackle at all) into an SDK archive.

If I understand correctly the dart_precompiled_runtime is per target while gen_snapshot is per (host, target) tuple. I'm wondering why this approach was chosen, as new compilers (clang,rust,etc) tend to be retargetable. I believe gcc only supported a single host,target tuple because in 1987 computers had a very limited amount of memory and disk space.

[mraleph]

@dvc94ch Note that gen_snapshot does not actually support arbitrary combinations of host and target.

The root of this is in Dart's origins. Dart is originally a JITed language and JITs tend to have host == target. Additionally there is a tight coupling between compilation toolchain and the runtime system (again because it was first and foremost a JITed language). So most of the code was written like that and properly disentangling this is quite an adventure. We managed to disentangle some of it when we were forced to support generating 32-bit ARM code on 64-bit hosts (due to Mac OS X removing support for 32-bit binaries), but there are still more things to refactor.

[a-eniou-pvotal]

I used this GitHub action workflow to get cross-platform builds

# This is a basic workflow to help you get started with Actions

name: CI Building native server

# Controls when the workflow will run
on:
  # Triggers the workflow on push or pull request events but only for the main branch
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

jobs:
  build:
    runs-on: ${{ matrix.os }}

    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest, macOS-latest]
        include:
          - os: ubuntu-latest
            output-name: server-linux
          - os: macOS-latest
            output-name: server-mac
          - os: windows-latest
            output-name: server-windows.exe
    steps:
          - uses: actions/checkout@v2
          - uses: dart-lang/setup-dart@v1.3
          - name: Install dependencies
            run: dart pub get
          - run: mkdir build
          - name: Install Dependencies
            run: dart pub get
          - run: dart compile exe ./bin/server.dart -v -o build/${{ matrix.output-name }}
          - uses: actions/upload-artifact@v1
            with:
                name: native-executables
                path: build

and here is the result

[dvc94ch]

I think people are more concerned with cross compiling debug builds on a development machine and testing it on a target machine than creating cross compiled releases. There are a few reasons why it's hard even if dart could cross compile or why it's not a great idea.

• dmg used for macos packaging and squashfs used in appimages or snaps use file systems not commonly found on other platforms, I expect it to be quite a bit of work to get it working.
• when building something like an appimage you want to link against old libraries to make the artifact usable on as many systems as possible

It is already possible today by using the kernel_blob.bin (cross compiled windows apps work already) and I'm working on a tool that exposes this (among other things like packaging, signing and publishing) to flutter developers.

[dvc94ch]

Is there any way to build the aot assembly for iOS from Linux/windows? kernel_blob.bin won't work on iOS right? Got other platforms working, still figuring out iOS.

[dvc94ch]

So https://github.com/cloudpeers/xbuild would really benefit from being able to create aot snapshots for ios from linux/windows. It's the only reason you still need a mac for developing ios apps on non jailbroken ios devices. Of course you still need an apple developer subscription.

[dvc94ch]

xbuild now supports signing/notarizing/stapling the appbundle, creating a dmg and signing/notarizing/stapling the dmg from linux and windows. This makes it really easy to provide users with a bugfix for testing.

[maks]

Coming back to this because it occurred to me to ask: how does Flutter already manage to do this?

Flutter is able to both:

1. compile Dart code on my x86 laptop and deploy it (with hot reload) to a Flutter app running on a Arm emulator or device
2. compile Dart code and package it into a release apk that can run a Arm emulator or device

In both the above, my assumption is that cross compilation is happening for the Flutter apps Dart code and if so is this all Flutter specific code and is there a reason it cannot be used for Dart cli as well as Flutter apps?

My current use case for cross compiling is trying to develop for a RISC-V device where the CPU is relatively slow and also has a modest (512MB) amount of RAM so running Dart on the cli (which of course requires compiling) is very slow and so on device development is very slow and painful.

[mraleph]

@maks Flutter does it in the same way which is described in #28617 (comment) and other comments on the thread.

They build a bunch of gen_snapshot binaries for each host -> target combination they support as well as building a bunch of libflutter.so for all target configurations they support. flutter tool then dynamically downloads necessary pieces.

Dart's dart compile exe could do the same. For combinations that we already support the biggest open question is packaging this all together, rather than anything else. For combinations that we don't support (e.g. building Windows exe on Linux) more work needs to be done in VM innards.

[maks]

Thanks @mraleph 👍🏻 that gives me a good place to go start looking into the Flutter code to learn how they build the separate gen_snapshot binaries for each target.

In regards to the packaging and distribution for doing this in Dart, is there the possibility of reusing the infrastructure that Flutter has setup or do the Dart & Flutter projects prefer to keep these separate?

[mraleph]

Thanks @mraleph 👍🏻 that gives me a good place to go start looking into the Flutter code to learn how they build the separate gen_snapshot binaries for each target.

You will have to read their buildbot recipes for that (https://flutter.googlesource.com/recipes). It's all done on the CI and uploaded to the cloud storage bucket.

Ultimately it is just about configuring the engine build with the right flags.

In regards to the packaging and distribution for doing this in Dart, is there the possibility of reusing the infrastructure

We will probably have some variation of the same infrastructure. We had a brief chat with @mit-mit and we might lay some ground work for this next quarter.

[yingshaoxo]

I think "dart can't cross compile" is simply a lie !!!

Because apparently, flutter can compile an arm apk file directly on window, linux amd64 machine.

[NBSgamesAT]

With Mac moving over to arm. Being able to cross-compile because even more important. Because people on mac will not be able to run a windows vm and do the compiling there. So we will need 2 seperated machines now just to get our executables for all platforms...

But I'll need to ask then: Why does the dart compile command mention the --target-os option to begin with? Why does it not print a warning that --target-os is currently ignored when entered?

[TarasMazepa]

With MacOS having two different architectures arm and intell I need to have two Mac machines to compile all the variants. Note that not all CI platforms, as GitHub offer Arm Mac machines. So you would need to search to cover your Mac compilation needs.

[SittiphanSittisak]

I used this GitHub action workflow to get cross-platform builds

# This is a basic workflow to help you get started with Actions

name: CI Building native server

# Controls when the workflow will run
on:
  # Triggers the workflow on push or pull request events but only for the main branch
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

jobs:
  build:
    runs-on: ${{ matrix.os }}

    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest, macOS-latest]
        include:
          - os: ubuntu-latest
            output-name: server-linux
          - os: macOS-latest
            output-name: server-mac
          - os: windows-latest
            output-name: server-windows.exe
    steps:
          - uses: actions/checkout@v2
          - uses: dart-lang/setup-dart@v1.3
          - name: Install dependencies
            run: dart pub get
          - run: mkdir build
          - name: Install Dependencies
            run: dart pub get
          - run: dart compile exe ./bin/server.dart -v -o build/${{ matrix.output-name }}
          - uses: actions/upload-artifact@v1
            with:
                name: native-executables
                path: build

and here is the result

Hi, thank you for this solution. I am new to git workflows/actions!?
I wonder about the Install Dependencies step. Why have the install dependencies 2 times?

          - name: Install dependencies   *1
            run: dart pub get
          - run: mkdir build
          - name: Install Dependencies   *2

My root folder seems like this.

root
     - .github
          - workflows
               - github-actions-demo.yml
               
     - dart_server
          - bin
               - dart_server.dart
          - pubspec.yaml

     - dart_share                       //include the object, and function these were used in dart_share
          - pubspec.yaml

So I should change some steps in github-actions-demo.yml, right?

UPDATE
I am using this.

# This is a basic workflow to help you get started with Actions

name: CI Building native server

# Controls when the workflow will run
on:
  # Triggers the workflow on push or pull request events but only for the main branch
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

jobs:
  build:
    runs-on: ${{ matrix.os }}

    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest, macOS-latest]
        include:
          - os: ubuntu-latest
            output-name: server-linux
          - os: macOS-latest
            output-name: server-mac
          - os: windows-latest
            output-name: server-windows.exe
    steps:
      - uses: actions/checkout@v2
      - uses: dart-lang/setup-dart@v1.3
      - name: Install dependencies for share project
        working-directory: ./share_dart
        run: dart pub get
      - name: Install dependencies for server project
        working-directory: ./server
        run: dart pub get
      - run: mkdir build
      - name: Install Dependencies for share project
        working-directory: ./share_dart
        run: dart pub get
      - name: Install Dependencies for server project
        working-directory: ./server
        run: dart pub get
      - run: dart compile exe ./server/bin/server.dart -v -o build/${{ matrix.output-name }}
      - uses: actions/upload-artifact@v1
        with:
          name: native-executables
          path: build

[geocode]

Alternative approach Using node.js to create executables

I have a work around that I would like to share as it does not require any GitHub services, etc. It can all be done from command line.

I am compiling to JS, then using node.js tools to convert to executable. I wrote a bash script to simplify the process. This is certainly not going to be the most efficient and probably has gaps (that I haven't found), but for the command line tools I've been building it has worked well so far.

Until it is supported natively...

Example Steps

Create the js files

dart compile js bin/hello.dart -o js/hello.js

Command line arguments don’t work in generated js

It turns out that for some reason the command line arguments passed to the program are not picked up by the generated node program, unless you do a special little trick. I found out about the trick from this site: https://japhr.blogspot.com/2014/07/command-line-arguments-with-nodejs-and.html

Basically the trick is to add a function to the generated node.js file:

function dartMainRunner(main, args) {
  main(process.argv.slice(2)); 
}

Create the executable using node.js tools

Create the package.json file in js directory

{
    "name": "hello",
    "version": "0.0.1",
    "description": "Say hello.",
    "main": "hello.js",
    "bin": "hello.js", 
    "author": "anybody",
    "license": "MIT"
}

Then build the executable

pkg .

Script to do all of this

In order to automate this I created a script dart2js in ~/local/bin that does all of these steps.

#!/bin/bash

if [[ "$1" == "" ]]; then
  echo -e "Usage:\n\t$0 [basename]"
  exit 1
fi

if [[ ! -f bin/$1.dart ]]; then
  echo "bin/$1.dart does not exist"
  exit 2
fi

dart compile js bin/$1.dart -o js/$1.js
if [ $? -ne 0 ]; then
  echo "Compiling js failed, exiting build script"
  exit 3
fi

wrapper='function dartMainRunner(main, args) { main(process.argv.slice(2)); }';
echo $wrapper >> js/$1.js

cat <<EOT > js/package.json
{
    "name": "$1",
    "version": "0.0.1",
    "description": "Do amazing $1 stuff.",
    "main": "$1.js",
    "bin": "$1.js", 
    "author": "Geoff Ruscoe",
    "license": "MIT"
}
EOT

cd js
pkg .

[liudonghua123]

Alternative approach Using node.js to create executables

I have a work around that I would like to share as it does not require any GitHub services, etc. It can all be done from command line.

I am compiling to JS, then using node.js tools to convert to executable. I wrote a bash script to simplify the process. This is certainly not going to be the most efficient and probably has gaps (that I haven't found), but for the command line tools I've been building it has worked well so far.

Until it is supported natively...

Example Steps

Create the js files

dart compile js bin/hello.dart -o js/hello.js

Command line arguments don’t work in generated js

It turns out that for some reason the command line arguments passed to the program are not picked up by the generated node program, unless you do a special little trick. I found out about the trick from this site: https://japhr.blogspot.com/2014/07/command-line-arguments-with-nodejs-and.html

Basically the trick is to add a function to the generated node.js file:

function dartMainRunner(main, args) {
  main(process.argv.slice(2)); 
}

Create the executable using node.js tools

Create the package.json file in js directory

{
    "name": "hello",
    "version": "0.0.1",
    "description": "Say hello.",
    "main": "hello.js",
    "bin": "hello.js", 
    "author": "anybody",
    "license": "MIT"
}

Then build the executable

pkg .

Script to do all of this

In order to automate this I created a script dart2js in ~/local/bin that does all of these steps.

#!/bin/bash

if [[ "$1" == "" ]]; then
  echo -e "Usage:\n\t$0 [basename]"
  exit 1
fi

if [[ ! -f bin/$1.dart ]]; then
  echo "bin/$1.dart does not exist"
  exit 2
fi

dart compile js bin/$1.dart -o js/$1.js
if [ $? -ne 0 ]; then
  echo "Compiling js failed, exiting build script"
  exit 3
fi

wrapper='function dartMainRunner(main, args) { main(process.argv.slice(2)); }';
echo $wrapper >> js/$1.js

cat <<EOT > js/package.json
{
    "name": "$1",
    "version": "0.0.1",
    "description": "Do amazing $1 stuff.",
    "main": "$1.js",
    "bin": "$1.js", 
    "author": "Geoff Ruscoe",
    "license": "MIT"
}
EOT

cd js
pkg .

this method produces much big sized binary. For simple hello world example, the binary maybe almost 100MB.

I would prefer to use github action using matrix to build the aot binary.