Installing Caiman - Keras/VS issue

[Haennio]

For better support, please use the template below to submit your issue. When your issue gets resolved please remember to close it.

Sometimes errors while running CNMF occur during parallel processing which prevents the log to provide a meaningful error message. Please reproduce your error with setting dview=None.

If you need to upgrade CaImAn follow the instructions given in the documentation.

• Tell us a bit about your setup:
  1. Operating system (Linux/macOS/Windows): Windows 10
  2. Python version (3.x): Python 3.6.7
  3. Working environment (Python IDE/Jupyter Notebook/other): Anaconda prompt
  4. Which of the demo scripts you're using for your analysis (if applicable):
  5. CaImAn version:
  6. CaImAn installation process (pip install ./pip install -e ./conda): Package-based Process, anaconda

*You can get the CaImAn version by creating a params object and then typing params.data['caiman_version']. If the field doesn't exist, type N/A and consider upgrading)

• Describe the issue that you are experiencing
  Hello,
  I tried to install Caiman today and got an error related to "keras", I think? It is the first time I am using anaconda on this computer and installing Caiman. I tried deleting the environment and installing again, which resulted in the same error.
  Keras installation get error that the system was unable to find the specified registry key or value  keras-team/keras#13209 is giving some suggestions, but I was wondering, what will work best within caiman?
  Thank you for your help!
  Hannah

• Copy error log below
  Preparing transaction: done
  Verifying transaction: done
  Executing transaction: / b'Enabling notebook extension jupyter-js-widgets/extension...\n - Validating: ok\n'
  / DEBUG menuinst_win32:init(199): Menu: name: 'Anaconda${PY_VER} ${PLATFORM}', prefix: 'C:\Users\Hannah\Anaconda3\envs\caiman', env_name: 'caiman', mode: 'user', used_mode: 'user'
  DEBUG menuinst_win32:create(323): Shortcut cmd is C:\Users\Hannah\Anaconda3\pythonw.exe, args are ['C:\Users\Hannah\Anaconda3\cwp.py', 'C:\Users\Hannah\Anaconda3\envs\caiman', 'C:\Users\Hannah\Anaconda3\envs\caiman\pythonw.exe', 'C:\Users\Hannah\Anaconda3\envs\caiman\Scripts\spyder-script.py']
  DEBUG menuinst_win32:create(323): Shortcut cmd is C:\Users\Hannah\Anaconda3\python.exe, args are ['C:\Users\Hannah\Anaconda3\cwp.py', 'C:\Users\Hannah\Anaconda3\envs\caiman', 'C:\Users\Hannah\Anaconda3\envs\caiman\python.exe', 'C:\Users\Hannah\Anaconda3\envs\caiman\Scripts\spyder-script.py', '--reset']
  done

C:\Users\Hannah>set "KERAS_BACKEND="

C:\Users\Hannah>set "MKL_NUM_THREADS="

C:\Users\Hannah>set "OPENBLAS_NUM_THREADS="

C:\Users\Hannah>set MKL_NUM_THREADS=1

C:\Users\Hannah>set OPENBLAS_NUM_THREADS=1

C:\Users\Hannah>python C:\Users\Hannah\Anaconda3\envs\caiman\etc\keras\load_config.py 1>temp.txt

C:\Users\Hannah>set /p KERAS_BACKEND= 0<temp.txt

C:\Users\Hannah>del temp.txt

C:\Users\Hannah>python -c "import keras" 1>nul 2>&1

C:\Users\Hannah>if errorlevel 1 (
ver 1>nul
set "KERAS_BACKEND=theano"
python -c "import keras" 1>nul 2>&1
)

C:\Users\Hannah>SET DISTUTILS_USE_SDK=1

C:\Users\Hannah>SET MSSdk=1

C:\Users\Hannah>SET platform=

C:\Users\Hannah>IF /I [AMD64] == [amd64] set "platform=true"

C:\Users\Hannah>IF /I [] == [amd64] set "platform=true"

C:\Users\Hannah>if defined platform (set "VSREGKEY=HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\14.0" ) ELSE (set "VSREGKEY=HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\14.0" )

C:\Users\Hannah>for /F "skip=2 tokens=2,*" %A in ('reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\14.0" /v InstallDir') do SET "VSINSTALLDIR=%B"
ERROR: The system was unable to find the specified registry key or value.

C:\Users\Hannah>if "" == "" (set "VSINSTALLDIR=" )

C:\Users\Hannah>if "" == "" (
ECHO "WARNING: Did not find VS in registry or in VS140COMNTOOLS env var - your compiler may not work"
GOTO End
)
"WARNING: Did not find VS in registry or in VS140COMNTOOLS env var - your compiler may not work"
The system cannot find the batch label specified - End

• If you're not reporting an error, type your message below

[pgunn]

Hi,
This looks more like an issue with your Visual Studio install than one with Keras. What version of Visual Studio do you have? We have some fairly detailed instructions on what components/version you need (although if you can live with a package-based install, that's usually much easier on Windows).

[epnev]

@Haennio The discussion on the keras issue suggests that python is usable even after that error. Is that the case for you?

[Haennio]

@epnev yes, it is still usable.
@pgunn , yes that could be. It is vs2015, I think? I am definitely fine with a package-based install. But I am a bit confused, as I thought the package-based installation of caiman would not give this problem?

Thank you very much for your help!
Hannah

[pgunn]

Hi,
If things are usable for you now, the only thing to deal with is that noise every time you activate an environment. Provided that's livable, you're probably fine now.

If you were to switch to a package-based install (removing your current caiman environment and building a new one that uses the precompiled package), these messages would disappear and future upgrades would probably be easier. It's up to you if your current setup is working for you.

[epnev]

@Haennio The only place that keras is used is within the VolPy module for analyzing voltage imaging data. If that's not what you need CaImAn for, then you might be fine. Provided you finished the installation (i.e., installed caimanmanager), try running some of the demos to see if that's the case.

[Haennio]

@pgunn, but I used the package based installation?
@epnev I also cannot see the demos, unfortunately... (in jupyter notebook, right?)

Thank you for you effort,
Hannah

[pgunn]

@Haennio Sorry, I misunderstood and thought you were doing a source-based install. I am surprised that the package-based install pulled in the Visual Studio packages.

Please use caimanmanager to install the caiman_data directory. You will find the demos under there.

[epnev]

v1.8.1 removed the keras dependency so it should not give these problems (although it is very likely already have a working installation).

[Haennio]

@pgunn and @epnev thank you very much for your help already! I will try to figure out what the exact problem is and will share it here then.

[pgunn]

Please remember to share it - we try not to leave github issues open for too long when inactive.

[Haennio]

Hello, so indeed, the keras/VS issue didn't appear after updating the installation to v1.8.1. I also had problems with the installation of the demos. I needed to navigate to the folder where the caimanmanager.py file is located ( C:\Users\user\Anaconda3\envs\caiman\bin), otherwise the file was not found ( '[Errno 2] No such file or directory') and then also used 'python caimanmanager.py install', otherwise I got the error 'no app to open this file'. Did I do something wrong in the first place or did I misread the manual? Thank you for your support @pgunn and @epnev

[pgunn]

Hi,
For some reason, Conda on Windows doesn't always have the ability to manage paths and associations like I expect; your experiences are not unusual, but not all Windows users have that experience either. I don't think it indicates a mistake on your part.