tensorflow{,-cpu,-gpu} differences?

[bjarthur]

Comment:

i can guess how tensorflow-cpu differs from tensorflow-gpu, but what is tensorflow?

[hmaarrfk]

We should document this better, but they are mostly holdover names from previous naming attempts.

Today, i would depend on tensorflow and allow conda (or mamba) decide on which one is better. CUDA should have a preference if you have a CUDA GPU detected.

[bjarthur]

thanks for the info!

relatedly, if i depend on tensorflow, i see that there is an osx-arm64 build for apple silicon. does that install all the deps needed for the metal framework, as described here? that is, tensorflow-deps, tensorflow-macos, and tensorflow-metal? the last two are pip, i was hoping to use solely conda.

[ngam]

thanks for the info!

relatedly, if i depend on tensorflow, i see that there is an osx-arm64 build for apple silicon. does that install all the deps needed for the metal framework, as described here? that is, tensorflow-deps, tensorflow-macos, and tensorflow-metal? the last two are pip, i was hoping to use solely conda.

Nope, the tensorflow-macos and tensorflow-metal builds are closed-source. You can only get them via pypi (pip install).

This osx-arm64 builds we have are basically CPU-only builds --- actually, they are pretty good in general. In my experience the metal versions (via pypi) only get you ~3x performance boost at most, mostly much lower than that.

(We do however have full support for the metal framework with pytorch)

[ngam]

copying @kulinseth

If you'd like us to help you build and distribute tensorflow-metal and tensorflow-macos through conda-forge, please do not hesitate to ask us. We are more than happy to facilitate that :)

[bjarthur]

what is the workaround until a proper tensorflow conda recipe exists for metal on osx-arm64?

should i, following the apple installations directions, put tensorflow-deps in meta.yaml and then in post-link.sh put python -m pip install tensorflow-macos tensorflow-metal?? i know pip dependencies in conda recipes are highly discouraged, but i don't see another option here.

sorry for what might be a dumb question but i am still quite new to conda, both as a user and a recipe developer. thanks!

[hmaarrfk]

Hey Ben,

I'm not sure there really are "best practices" for these new processors. :/ You are, somewhat, beholden to following instructions piecemeal.

You should, in general, be careful about mixing instructions, but given your experience with various software package, I don't think I need to tell you that.

I thin generally, the idea of:

1. Installing all the dependencies with conda.
2. Installing the one thing you need from pip
3. Praying.
4. Never calling mamba install again.

Works. This seems to be in line with the instructions from Apple. Their meta package, tensorflow-deps, seems to depend on:

  "depends": [
    "grpcio >=1.37.0,<2.0",
    "h5py >=3.6.0,<3.7",
    "numpy >=1.22.3,<1.23.0",
    "python"
  ],

(You can download the tar file, and open the info/index.json file to look at things.

Generally, pip and conda seem to clash on detecting software installed by the other.

[ngam]

Since I use m1 machines... I personally advise you choose one of the following:

1. Just get everything from conda-forge (everything is completely fine but we don't have metal support). Simply mamba install tensorflow -c conda-forge or conda install tensorflow -c conda-forge on your machine.

2. Just get everything from pip (they used to have those instructions to mix conda and pip because h5py wasn't available as a wheel for m1, but it is now available, so you no longer really need to install tensorflow-deps). On a brand-new M1 macbook, you can get the command-line tools and then use the python3.8 that comes with them to do python3 -m pip install tensorflow-macos tensorflow-metal and you will be set. If you're using python from conda-forge, you might as well get the tensorflow-deps though unless you want to also get the compilers.

I can't imagine you'd be getting all that good of a performance with the tensorflow-metal version anyway, so option 1 is probably good enough. Note: you can always have both setups (like I personally do)

(Btw, M1 or M2, both are the same for all practical purposes here)

[ngam]

should i, following the apple installations directions, put tensorflow-deps in meta.yaml and then in post-link.sh put python -m pip install tensorflow-macos tensorflow-metal?? i know pip dependencies in conda recipes are highly discouraged, but i don't see another option here.

Also, I am not sure what you're talking about here. Why do you need a "meta.yaml" file for your own use? These files are only important for building. If you just want an environment.yaml file, then you can list tensorflow-deps under dependencies and tensorflow-macos and tensorlfow-metal under pip like this:

name: myenv
channels:
  - conda-forge
  - apple
dependencies:
  - python=3.*
  - tensorflow-deps
  - pip
  - pip:
    - tensorflow-macos
    - tensorflow-metal

[stuarteberg]

@ngam:

Also, I am not sure what you're talking about here. Why do you need a "meta.yaml" file for your own use?

He wants to package his own software application for conda users, and his software depends on tensorflow.

[bjarthur]

precisely. thanks stuart. i meant to reply days ago.

@ngam i've got a working beta conda recipe for my application here. once this is vetted by my users, and by more experienced conda developers, the plan is to stage it to conda forge. any comments would be appreciated.

[ngam]

Ah, makes sense!

@stuarteberg @bjarthur I will have a look and provide feedback soon. Please feel free to tag me when you end up submitting it to conda-forge.