Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PennAI image for raspberrypi #302

Open
weixuanfu opened this issue Dec 3, 2020 · 5 comments
Open

PennAI image for raspberrypi #302

weixuanfu opened this issue Dec 3, 2020 · 5 comments

Comments

@weixuanfu
Copy link
Contributor

I pushed a branch called raspberrypi for trying building docker images of PennAI in raspberrypi.

This issue is for discussion of solution for transporting PennAI based on x86 (most Intel and AMD CPU) to ARM (raspberrypi's CPU and Apple M1)

Below are things I have tested:

  • A 64bit Linux OS, like Raspberry Pi OS (64 bit) beta test version is needed for installation of dbmongo docker images.
  • Docker and docker-compose can be installed in the Raspberry Pi OS (64 bit) beta test version (see this instruction)
  • dbmongo docker image can be built with the commit 7ad961e
  • python and node can be installed into lab docker image with the commit c8e51b3

But some python packages (like numpy and pandas) were failed to installed due to ARM6 architecture, we may find a way to re-complile them from source codes.

@JDRomano2
Copy link
Contributor

All of the individual dependencies can be installed on the Raspberry Pi 4 (I have a 2GB model - the 4GB or 8GB model would definitely be better). The main issue is that most (all?) of the Python packages need to be compiled from source. pip/pip3 work just fine, but piwheels doesn't track arm64 wheels yet.

Therefore, in most configurations I've tested, 1 of 2 things seems to happen:

  1. After several hours of building dependency packages (predominantly scipy and scikit-learn) a small bug in the Dockerfile means the build has to start from square 1.
  2. The build process is killed by the kernel due to memory usage.

@weixuanfu
Copy link
Contributor Author

We need a 8Gb one. 1 or 2Gb is not reality for building those packages or running PennAI.

Or compiling from another powerful arm64 machine may be alternative solution.

@JDRomano2
Copy link
Contributor

Agreed.

I'm currently working on building the dependencies on an x86_64 version of Ubuntu using aarch64 cross-compiling tools. If this works, we can try to distribute the compiled Python packages as wheel files.

However, that won't solve the issue of needing more resources to actually run all of the components of PennAI once it's installed.

@hjwilli
Copy link
Collaborator

hjwilli commented Dec 23, 2020

Thanks @weixuanfu @JDRomano2!

Please make new Dockerfiles for the raspberry pi builds (perhaps */docker/Dockerfile_pi or */docker/Dockerfile_x86_64) rather then overriding the existing ones so that we can eventually merge this to the main branch.

@JDRomano2
Copy link
Contributor

PR #305 provides initial support for building on Raspberry Pi. As requested above, there are separate Dockerfiles and a separate docker-compose configuration file for building arm64-compatible PennAI.

The Docker application can be built by running docker-compose -f docker-compose-raspberrypi.yml build in the root directory.

I think we should hold off on closing this issue until #306 is resolved and we can verify that the AutoML features of PennAI work on the Raspberry Pi 400.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants