-
Notifications
You must be signed in to change notification settings - Fork 309
Instructions to install TensorFlow in a Conda Environment #153
Comments
I couldn't install the wheels without renaming them to read "...-macosx_10_16_..." instead of "...-macosx_11_0_..." |
@Tioz90 I didn't have that problem. Could you tell me what your Mac's version is? That's Apple (in the Menu Bar) > About my Mac. Mine is Version 11.2. |
@mwidjaja1 MacOs 11.1 |
@Tioz90 You may want to create a new issue then about this. I'm not on the team working on this, I just wrote up these instructions as a 'side contributor' with their help. |
I think it's a known issue with |
That's an interesting thought @Tioz90 -- My pip version is 20.2.4 which I checked by running |
Mine is |
@Tioz90 I cannot reproduce your issue. I've upgraded to the same pip version you had, trashed my environment, and was able to successfully rebuild it. I have a hunch somewhere in your Python installation, you're accidentally linking to some other Python that you did not intend to use but I wouldn't know how to begin figuring that out. Glad you got it to work by renaming it though |
There's a few things that could make this issue a mess. I haven't found versions of pip later than 20.2.4 to work here; version 20.3 and later actually broke some of the logic. Also, the logic inside of pip is duplicated across setuptools and packaging, both of which are baked into the core python distribution; upgrading these might help, but your mileage may vary. |
Does it mean that installing TensorFlow in a Conda Environment allows m1 chip to use tensorflow with SciPy and dependent packages? |
Yes, SciPy works for me
Thomas Tiotto
…On 4 Feb 2021, 10:09 +0100, Stanfording ***@***.***>, wrote:
Does it mean that installing TensorFlow in a Conda Environment allows m1 chip to use tensorflow with scipy?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
I got this Error
I'm Confused... Run Indeed:
Python 3.9 installed with homebrew:
Python 3.8 from a conda-forge env:
So conda-forge's Python is indeed pretending to be installed on a fake 10.16 version of macOS to bypass this problem with pip 20.2.4. |
@Tioz90, I think @gyuannnn figured it out. I get your exact error on Anaconda on Intel too. Interestingly enough on an identical macOS version on an Apple Silicon Mac, it actually is 11_0. So it looks like it's all Conda Pythons (since I tried on Anaconda and you tried Miniforge) on Big Sur Intel, the platform is being saved as 10_16 rather than 11_0. Not sure how to report this to the Miniforge folks but I can poke around unless you have an idea where the problem might be. In the meanwhile, can you try downloading the wheel files and renaming it so instead of having @hoytak tagging you for situational awareness that we figured out the issue. EDIT: Nevermind I see you reported it already. https://github.com/conda-forge/miniforge/issues/105. Not sure if this is the right place admittedly (it could be Conda's fault) but let's see if they know where to direct us to. |
Downloaded and renamed it to 10_16_ works fine. |
@mwidjaja1 |
Yes, renaming the files installs them without an issue |
@gyuannnn Thanks for the heads up, I updated the documentation to reflect this temporary workaround. Python 3.8.8 sounds like it'll fix this issue fortunately. Conversely, renaming the files may crash your Python @Tioz90 if you ever update your Python to 3.8.8 which is planned to fix this issue. https://www.python.org/downloads/release/python-387/ @hoytak A heads up for your team's awareness that this is an issue... and actually might explain some of the crazy pip commands I noticed you guys pulling off. I'm starting to wonder if some of the weird installation issues I saw, while trying to use Conda installs to setup TensorFlow's pre-reqs, were related to this whole |
How can I "point" python and pip to use their respective miniforge versions? I have my conda environment working fine, but when I enter I tried installing as above but got the wheel error: |
+1 Ran into the exact same issue today. |
@icenando & @mathieuboudreau this means Miniforge is not added to your system path. This is usually a Googling task just because everybody's terminals shell prompts are different, it'd be way beyond the scope of this post to cover every permutation. In theory running |
@mwidjaja1 Thank you for that. I notice that the python version installed with miniforge is 3.9. So I tried changing the pip install line to python3.9 instead of 3.8, but I got the same wheel error. When I try using Apple's .sh file, Any help to sort this out would be appreciated. |
Ok, so removing the evironment and re-creating it, specifying |
@icenando I'm glad you got it working though just to get feedback, I'm not sure why that article would work whereas these notes wouldn't. "Installing TensorFlow" section of this guide tells you to download environment.yml and then activate said environment. That environment is designed to give you Python 3.8 and all of the prerequisites you need. With all due respect, your first comment implies you didn't read my instructions, because you're right miniforge installs Python 3.9 by default but my instructions literally guide you away from using it. But if you have any error messages while setting up that environment or activating it, please let me know, I'd be glad to rephrase the guide. Thanks. |
@mwidjaja1 Sorry, I didn't mean to imply that your instructions are incorrect. I'm sure I'm making a lot of mistakes as I go: I'm not that experienced, and there are a lot of instructions that I follow blindly because I don't understand what they do. I don't remember getting any error messages prior to the wheel one. But regardless, that meant that I was unable to finish the install. Not that I know what I'm talking about, but wouldn't it be good to replace The troubleshooting section of that article also covered many of the errors that I was getting, as well as how to install libraries that I was unable to before (I did have a working environment in place before things went pear shaped). Apologies again if my previous message implied that there was something wrong with your instructions. |
@icenando No worries! I've been there too -- I just have been getting more DMs about this and I'm just a 'volunteer' so I've been running around heh. If you wouldn't mind, could you give the original command another whirl? So long as you name that Conda environment something different than what you did with your working tensorflow env, they shouldn't stomp each other out. Specifically the commands I'm asking you to run are:
The YML file you're downloading is the 'recipe' that Conda will use to download everything and set Python up. Because of that, there SHOULD be no need to specify |
@Eunchan24, the base Tensorflow 2 distribution currently supports Python 3.6-3.8. |
I find the instructions to install tensorflow on the Mac absolutely useless
…On Thu, Apr 15, 2021, 10:05 AM Kevin Dalton ***@***.***> wrote:
@Eunchan24 <https://github.com/Eunchan24>, the base Tensorflow 2
distribution currently supports <https://www.tensorflow.org/install>
Python 3.6-3.8.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#153 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AOJSP42FSANHYGI4Y2OSCPTTI3XDDANCNFSM4XAB6KEA>
.
|
@mwidjaja1 thank you so much for your answer and sorry for the late reply. It's strange because when I do:
This is what is inside of .bash_profile : # >>> conda initialize >>> This is what is inside of .zshrc : # If you come from bash you might have to change your $PATH. # Path to your oh-my-zsh installation. source ~/.bash_profile There is nothing in .bashrc I don't see anything strange except 'source ~/.bash_profile' that is something that I added myself in .zshrc to make oh-my-zsh work. |
@RomainBsb without access to your Mac, it'd be hard for me to guess every possible outcome. This sounds like an issue with something in how you configured your computer and there's only so much I can help you with then. Other things to jot your mind:
Sorry for the unsatisfying response but again, there's nothing you're showing me that's outright wrong. There's just something, somewhere, incorrectly directing your pip to site-packages rather than the bin folder. Best of luck, sorry I can't help more from a far. |
Hi @mwidjaja1, I set up my env as you described. I encountered some nonblocking problems.
Ain't this a problem? Futhrer my setup seems to ignore the GPU. How can I make tensorflow use the GPU instead of the CPU? This is the cifar10 script to test tensor flow which also reveals that tensorflow does not see the GPU at all. import tensorflow as tf
print(tf.__version__)
tf.compat.v1.disable_eager_execution()
print(tf.config.list_physical_devices())
from tensorflow.python.client import device_lib
print([x.name for x in device_lib.list_local_devices() if x.device_type == 'GPU'])
from tensorflow.python.compiler.mlcompute import mlcompute
print("is_apple_mlc_enabled %s" % mlcompute.is_apple_mlc_enabled())
print("is_tf_compiled_with_apple_mlc %s" % mlcompute.is_tf_compiled_with_apple_mlc())
from tensorflow.keras import datasets, layers, models
(train_images, train_labels), (test_images, test_labels) = datasets.cifar10.load_data()
train_images, test_images = train_images / 255.0, test_images / 255.0
class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer',
'dog', 'frog', 'horse', 'ship', 'truck']
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.Flatten())
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(10))
model.summary()
model.compile(optimizer='adam',
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
history = model.fit(train_images, train_labels, epochs=10,
validation_data=(test_images, test_labels)) What am I doing wrong? Regards |
@hoytak have the requirements for Apple's Tensorflow changed? @ManuelSchneid3r for your second issue, please make a new GitLab issue (or search the issues list... I'd be surprised if you're the only one having this problem) for the CPU vs. GPU question. This Gitlab issue is specifically for conda installation. |
@mwidjaja1 indeed there are open issues concerning the GPU. I followed your instructions for conda. Does you setup use the CPU? I mean I should get GPU acceleration if you do right? |
If you get a mistake: you need to uninstall |
Python 3.9 is native with Big Sur and TensorFlow v2.5.0rc2 supports python 3.9... |
@HiLiHeYi I think your issue might be better as its own GitLab Issue. I'm just a 'volunteer' monitoring this issue just for Conda installation issues, and this guide has no reference to a requirement.txt file. It sounds like you're having 'other' issues using TensorFlow for your specific Pythonic use case, as opposed to installing TensorFlow, which I see you got working. However, the version of TF Apple Built, is every so slightly different than what your requirements.txt file wants. Your text file does NOT want a 'beta' release candidate, whereas Apple's is because... well... it is a beta release candidate. You may want to modify your requirements.txt file so it doesn't even mention tensorflow because you already had it installed anyway. I think that should do the trick. If this doesn't do the trick or doesn't meet your needs, please create a new GitLab issue and tag me so we can discuss there, and keep this clear for Conda install issues. Thanks! |
tensorflow v2.5.0 is out. Is it compatible with m1? |
Failing with installing Miniforge on Mac M1 using brew
My installation was failing on this part. It turned out that Miniforge had been installed to |
Hey @JakobLS, while I'm glad you figured it out, I'm super curious how your Homebrew got installed to caskroom. I... don't think that was possible. Do you recall manually requiring miniforge to save there or did it just happen? Glad you figured it out though, that is the absolute right step! Thanks for sharing |
Hi @mwidjaja1. Thank you so much for supporting the community. I don't recall the exact details, but I'm pretty sure I didn't manually specify miniforge to be installed at a certain location, nor with Homebrew. I followed your steps in the top, but chose to install miniforge using Homebrew purely for convenience (rather than downloading it first). And this tutorial was very useful to install Homebrew with. I had to add Running on MacOS Big Sur 11.3.1 |
This is not so much an issue as opposed to a 'How To' if you'd like to install this version of Tensorflow in Conda.
Prerequisites: You must be on macOS Big Sur
If you have an Apple Silicon Mac, this is a freebie, you're already on Big Sur. If you're on an Intel Mac, the Intel versions of TensorFlow are Big Sur only.
Sanity Check before Proceeding: To ensure you're on the right version of macOS, run
sw_vers -productVersion
in your terminal. If it's not version 11.##, you're not on Big Sur and must upgrade to it from the macOS App Store.Prerequisites: Install XCode Command Line Tools
Install Xcode Command Line tools if you haven't. To do so, run this in your terminal:
xcode-select --install
Sanity Check before Proceeding: To ensure installation worked, run
which xcrun
in your terminal and you should get a path like/usr/bin/xcrun
. If you haven't, you did not install it correctly.Prerequisites: Install Miniforge
Where to download Miniforge from
Miniforge, is a 'lightweight' Python interpreter that has full access to the Conda ecosystem. You can download Miniforge from https://github.com/conda-forge/miniforge#miniforge3. You can use Anaconda if you're on Intel, but note that this guide will be written from the perspective of using miniforge.
Sanity Check before Proceeding:
file $(which python)
in your terminal (thanks to @lebigot for this shortcut!). Please make sure that you got:miniforge
version of Python. It'll probably be<your home dir>/miniforge3/bin/python
.Mach-O 64-bit executable arm64
. If you have an Intel Mac, it should also sayMach-O 64-bit executable x86_64
.which pip
in your terminal and it too should resolve to some path that implies you're using miniforge3.If any of those sanity checks failed, you must redo this section. Please ensure that you downloaded the correct Miniforge for your system architecture and installed it. If you did all that, set your environment paths to Miniforge's Python Installation. To do that, you need to figure out where conda was installed to (it's probably
~/miniforge3/condabin/conda
) and then run~/miniforge3/condabin/conda init
in your terminal.Apple Silicon Only Warning: You CANNOT use Anaconda
This warning only applies to Apple Silicon Macs. Anaconda comes with many Python packages included, some of which are not Apple Silicon (i.e. ARM) compatible and thus Anaconda is not ARM compatible. You can use Anaconda if you're using an Intel Mac though.
If you were planning to use Anaconda on ARM, please scroll back up and install Miniforge. Miniforge has Conda, which means you can install many of the packages you want such as Pandas, Scipy, and Numpy -- unlike Anaconda, you just have to do the install manually by running
conda install mypackagenamehere
.Intel Only Warning: Python Bugs in Big Sur
This warning only apply to Intel Macs. For Intel, both Anaconda and MiniForge have a Python Bug which prevents you from running Python correctly in some instances on macOS Big Sur. Until the Python community fixes this, each time prior to loading Python, you must run
export SYSTEM_VERSION_COMPAT=0
. You could also add this to your.bash_profile
or other shell environment file if you have one, to do this automatically for you.Installing TensorFlow
Attached to this Issue is a YAML file which will help you create a Conda Environment with TensorFlow, along with all the prerequisites you need from the ARM conda-forge channel.
.txt
to the end of the file -- do not let your browser do that. [thanks to @isuruf for streamlining this file to be all Conda]conda env create --file=PATH_TO_ENVIRONMENT.YML --name=YOUR_ENV_NAME_HERE
.conda activate YOUR_ENV_NAME_HERE
For X86 as of 03/11/2021:
Thanks to @edwin-yan for the updated commands
For Apple Silicon as of 03/11/2021:
python
and try importingtensorflow
.Example Commands
In this below example, I'm installing & running the ARM version of tensorflow from an environment I've named
test
. The yml file is placed in the same directory I'm running this command from, which is my home directory (i.e.~
)Troubleshooting for importing TensorFlow
which python
and thenwhich pip
in your terminal. Both paths should point to a Python that is inside the environment you created in Step 2. If it doesn't, you may not have installed Miniforge correctly, ran Step 2 correctly, and/or may not have ran Step 3.python --version
and it should be version 3.8. If it isn't, you most likely did not create or activate your environment correctly, as per Steps 2 & 3. Do those again.not a supported wheel on this platform
error, runexport SYSTEM_VERSION_COMPAT=0
in your terminal and try again. If this works, you'll need to do this everytime you use Python until a Python Bug is resolved.pip
with a-v
flag for additional logging. Remember I'm just a volunteer -- I'll try to help but there's only so much I can help with.Troubleshooting for setting up TensorFlow
failed to decode
error: Try downgrading to h5py to the 2.10.0 wheel file that was packaged with this alpha release (pip install ~/path to h5py.whl
). Thanks to @ramicaza.The text was updated successfully, but these errors were encountered: