Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Instructions could use a bit more detail for basic Jupyter use case #10

Open
TashaSkyUp opened this issue Nov 15, 2019 · 20 comments
Open

Comments

@TashaSkyUp
Copy link

Currently using the provided instructions results in

'Traceback (most recent call last):
File "/usr/local/bin/jupyter-notebook", line 10, in
sys.exit(main())
File "/usr/local/lib/python3.6/dist-packages/jupyter_core/application.py", line 267, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/traitlets/config/application.py", line 657, in launch_instance
app.initialize(argv)
File "</usr/local/lib/python3.6/dist-packages/decorator.py:decorator-gen-7>", line 2, in initialize
File "/usr/local/lib/python3.6/dist-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/notebook/notebookapp.py", line 1679, in initialize
self.init_webapp()
File "/usr/local/lib/python3.6/dist-packages/notebook/notebookapp.py", line 1442, in init_webapp
self.http_server.listen(port, self.ip)
File "/usr/local/lib/python3.6/dist-packages/tornado/tcpserver.py", line 151, in listen
sockets = bind_sockets(port, address=address)
File "/usr/local/lib/python3.6/dist-packages/tornado/netutil.py", line 174, in bind_sockets
sock.bind(sockaddr)
OSError: [Errno 99] Cannot assign requested address'

using the --ip option attempts to actually work but it is not possible to connect to the notebook from the desktop.
tried --ip options:
localhost
127.0.0.1
0.0.0.0

the command even lists a file to access the notebook by, however it does not work either.

@TashaSkyUp
Copy link
Author

Creating 2 .toml files.. adding my user to the docker group and rebooting my machine solved the problem..

However this issue still persists better instructions are needed.

@mmstick
Copy link
Member

mmstick commented Nov 15, 2019

When launching a container, be sure to supply the port mappings argument to map the port from the container to the host.

tensorman run -p 8888:8888 --gpu --python3 --jupyter  bash

Within the container, you may start the Jupyter notebook with

jupyter notebook --ip=0.0.0.0 --no-browser

Which will give you the address and token to use for accessing it from your host OS, like so: http://127.0.0.1:8888/?token=bb826e8887076ecf3b1912a5e8f848e8029773bb04ad4990

@twosky2000
Copy link

I would like a Wiki in GitHub. This project is really nice to play around with tensorflow. But it took me ~6h to geht object recognition started.

@mmstick
Copy link
Member

mmstick commented Dec 19, 2019

@twosky2000 The README.md documents all the behavior of Tensorman. Which areas did you feel weren't explained well enough?

@twosky2000
Copy link

I niemals usw python, so i had Problems installing packets. I then das the custom Image Part and got it to work. -> start tensorman with --root and --name, Install packages, tensorman save name Image.
Then I exited the bash tensorman shell(cause i don't know if there ist something different from normal run) and Trier tensorman =opencv run --gpu ./webcam.py and got eine missing libs. Update Image again and now i'm at the point that i need to add the --device option to docker(should be no problem with a local toml).

That's a quick recap, i will hopfully get to it tomorrow and open respective issues

@mmstick
Copy link
Member

mmstick commented Dec 20, 2019

For the most part, you can install Python libraries locally in your project workspace, that way it doesn't have to be in the Docker image. If you do need some C libraries in the system image, you can do that by creating a new image. If you have any ideas for how to make this easier, I'm all ears.

@MikeyBeez
Copy link

MikeyBeez commented Feb 12, 2020

Here's what I did. I'm not certain this is right, but so far, it's what worked for me:

  • cd
  • mkdir project
  • cd project
  • tensorman pull latest
  • tensorman run --gpu --python3 --jupyter bash -p 8888:8888

That brings up a container running the Docker image. Then from the new "I have no name" prompt

  • cd ~/project
  • enter jupyter notebook --ip 0.0.0.0 --NotebookApp.token='' --NotebookApp.password=''

Then in your browser, go to 127.0.0.1:8888

Put your notebooks in ~/project/ on your own system -- not the Docker image.

@MikeyBeez
Copy link

@mmstick I'm currently trying to get fastai to work. I have it cloned inside my project directory. Should I make a virtual environment on my host and pip install fastai? Is it that easy? BTW, thank you for creating such an awesome distro!

@MikeyBeez
Copy link

I tested the Tensorflow inside the container and it doesn't work.
tensorman run --gpu --python3 --jupyter bash -p 8888:8888
python

import tensorflow as tf

hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()

print(sess.run(hello))

Traceback (most recent call last):
File "", line 1, in
AttributeError: module 'tensorflow' has no attribute 'Session'

@chriswininger
Copy link

When launching a container, be sure to supply the port mappings argument to map the port from the container to the host.

tensorman run -p 8888:8888 --gpu --python3 --jupyter  bash

Within the container, you may start the Jupyter notebook with

jupyter notebook --ip=0.0.0.0 --no-browser

Which will give you the address and token to use for accessing it from your host OS, like so: http://127.0.0.1:8888/?token=bb826e8887076ecf3b1912a5e8f848e8029773bb04ad4990

That bit about passing through the port and how to fire up the notebook is very useful. I wonder if that should not be included in the official docs somewhere. To me the -jupyter flag feels useless without knowing how to actual get to the notebook.

I also can't help but wonder if there is not some way to have it spin up the docker environment and execute this command in one go.

@MikeyBeez
Copy link

@chriswininger Thank you. I've moved on, however. I'm running Ubuntu 18.04 with Lamda Labs' Lambda stack. That works fine for me. I have installed the pop theme and the pop shop, and I've installed Popsicle. I'll look at Pop!_OS again when 20.04 comes out. I think it's a terrific distro.

@Cattleman
Copy link

I tested the Tensorflow inside the container and it doesn't work.
tensorman run --gpu --python3 --jupyter bash -p 8888:8888
python

import tensorflow as tf

hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()

print(sess.run(hello))

Traceback (most recent call last):
File "", line 1, in
AttributeError: module 'tensorflow' has no attribute 'Session'

I think you have TF 2.0 which replaces Session with tf.funtion() I think. See: https://www.tensorflow.org/guide/migrate

@MikeyBeez
Copy link

@Cattleman Thank you for the info. Everything changes all the time. It's hard to do actual work when one has to spend so much time keeping up with multiple APIs. Cheers.

@esthor
Copy link

esthor commented Apr 24, 2020

PLEASE add this to the instructions.

@florianbaer
Copy link

florianbaer commented May 3, 2020

I would be interestet how a simple project setup is.
Like the following one:

  1. Set up a jupyter notebook with the following dep: e.g. matplotlib, numpy, pandas and OpenCV (are they now installed in the container or on the physical system - and whats about pipenv's?)
  2. Share these tensorman configurations with other team mates
  3. Add additional dependencies later on...
  4. What are the config-possibilities for Tensorman.toml..? (I did not find it.)
  5. AAAAAAAAAAND a litte bit more detailed information how tensorman works.

It would make the use of tensorman much easier.

@Cattleman
Copy link

Hi all,
I wrote up a quick and dirty walkthrough of my workflow here:
https://www.notion.so/ionatomic/Getting-Started-with-Tensorman-2c8f7b8eb2fb4f00a40859c99a7c40e1

Feedback welcome - hope this helps folks. I'll add Tensorman.toml at some point too.

@MikeyBeez
Copy link

Nice job, Ian! Thank you.

@AAnzel
Copy link

AAnzel commented Aug 5, 2020

Is there a way to combine Tensorman with Anaconda (or Miniconda)? What would the workflow look like? I am currently doing everything through Miniconda (lots of Python libraries and Jupyter-Lab for coding).
And if there is a way, it would be nice to have a step-by-step instructions on the website. :)

@Cattleman
Copy link

Cattleman commented Aug 12, 2020

Hi @AAnzel,
You should be able to use miniconda inside the docker container. Tensorman is just an easier way to manage the docker container and cuda stuff.

@AAnzel
Copy link

AAnzel commented Aug 12, 2020

Thanks for the answer @Cattleman! I used Google Colab until now, so this is pretty new to me. I will try to set it up and see what happens.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants