SageNB package contains many packages

[sagetrac-felixs]

This ticket changes SageNB from a distribution to a normal Python package. This means unbundling everything which used to be part of the SageNB distribution.

The package webassets is simply removed, since it doesn't seem to be used by anything in Sage or SageNB. This thread indicates that webassets is used by the newui branch of SageNB but not in vanilla SageNB. So it has no reason being in Sage currently.

New upstream tarballs:

• http://sage.ugent.be/www/jdemeyer/sage/sagenb-0.12.tar.bz2
• ​https://pypi.python.org/packages/source/B/Babel/Babel-2.2.0.tar.gz
• https://pypi.python.org/packages/source/F/Flask/Flask-0.10.1.tar.gz
• http://sage.ugent.be/www/jdemeyer/sage/Flask_AutoIndex-0.5.tar.gz (renamed - -> _)
• http://sage.ugent.be/www/jdemeyer/sage/Flask_Babel-0.9.tar.gz (renamed - -> _)
• http://sage.ugent.be/www/jdemeyer/sage/Flask_OldSessions-0.10.tar.gz (renamed - -> _)
• http://sage.ugent.be/www/jdemeyer/sage/Flask_OpenID-1.2.5.tar.gz (renamed - -> _)
• http://sage.ugent.be/www/jdemeyer/sage/Flask_Silk-0.2.tar.gz (renamed - -> _)
• https://pypi.python.org/packages/source/i/itsdangerous/itsdangerous-0.24.tar.gz
• http://sage.ugent.be/www/jdemeyer/sage/python_openid-2.2.5.tar.gz (renamed - -> _)
• ​http://sage.ugent.be/www/jdemeyer/sage/pytz-2016.3.tar.bz2 (upstream version with sanitized permissions)
• https://pypi.python.org/packages/source/s/speaklater/speaklater-1.3.tar.gz
• https://pypi.python.org/packages/source/T/Twisted/Twisted-15.5.0.tar.bz2
• https://pypi.python.org/packages/source/W/Werkzeug/Werkzeug-0.11.5.tar.gz
• https://pypi.python.org/packages/source/z/zope.interface/zope.interface-4.1.3.tar.gz

Upstream: Completely fixed; Fix reported upstream

CC: @jondo @kcrisman

Component: packages: standard

Keywords: notebook, days77

Author: Jeroen Demeyer

Branch/Commit: b5fb38b

Reviewer: Salvatore Stella, Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/14840

[jdemeyer]

Changed author from Felix Salfelder to Jeroen Demeyer

[jdemeyer]

Changed keywords from toplevel build system notebook to notebook

[jdemeyer]

Changed keywords from notebook to notebook, days77

[dimpase]

comment:59

IMHO sagenb should not drag all of its dependencies into sage. Why can't the installation script simply run
sage --pip?

[jhpalmieri]

comment:60

Sage policy is to not require internet access to build Sage: you should be able to download a complete tarball and build from that. So we have to include tarballs for all of these dependencies, at which point why not make them Sage packages? Then we know they are available if they are needed elsewhere in Sage, and we can make any other dependencies explicit.

[dimpase]

comment:61

Replying to @jhpalmieri:

Sage policy is to not require internet access to build Sage: you should be able to download a complete tarball and build from that.

perhaps it's time for it to change; after all we tell users to install ssl support via pip, creating a lot of pain and confusion along the way, as we don't package the corresponding script thanks to exactly this prohibition. If we changed this policy we would only improve the nb-related procedures here.

So we have to include tarballs for all of these dependencies, at which point why not make them Sage packages? Then we know they are available if they are needed elsewhere in Sage, and we can make any other dependencies explicit.

well, in this case they are not needed anywhere else in sage.

[vbraun]

comment:62

I tend to agree that we should just exclude web-based UIs (SageNB + Jupyter) from the "self-contained tarball" and make them pip-type optional packages instead. If you don't have internet access then how come you have a browser installed? Also the whole OpenSSL can of worms if you really want to run a server.

But this should be changed at a major release, not now.

[jhpalmieri]

comment:63

Replying to @vbraun:

I tend to agree that we should just exclude web-based UIs (SageNB + Jupyter) from the "self-contained tarball" and make them pip-type optional packages instead. If you don't have internet access then how come you have a browser installed?

Because you live in a third world country and have inconsistent access? Because you're traveling and want to build Sage while on an airplane (and without paying the wifi fees)? You can come up with plenty of reasons.

[dimpase]

comment:64

Replying to @jhpalmieri:

Replying to @vbraun:

I tend to agree that we should just exclude web-based UIs (SageNB + Jupyter) from the "self-contained tarball" and make them pip-type optional packages instead. If you don't have internet access then how come you have a browser installed?

Because you live in a third world country and have inconsistent access? Because you're traveling and want to build Sage while on an airplane (and without paying the wifi fees)? You can come up with plenty of reasons.

Inconsistent access is a nightmare if you need to get a large file (e.g. current Sage source tarball), as opposed to several small files.

Besides, mind you, much more popular than Sage software distribution systems, e.g. ones for Python, Haskell, OCaml do require internet access, by default. Convenience for the majority (and for developers) ought to come before exotic usage cases.

[vbraun]

comment:65

IMHO those use cases are better served by having a script that downloads all standard+optional packages. I already have a script where you just have to

sage-package download :standard: :optional:

to grab them all in one go.

[jhpalmieri]

comment:66

Replying to @dimpase:

Replying to @jhpalmieri:

Replying to @vbraun:

I tend to agree that we should just exclude web-based UIs (SageNB + Jupyter) from the "self-contained tarball" and make them pip-type optional packages instead. If you don't have internet access then how come you have a browser installed?

Because you live in a third world country and have inconsistent access? Because you're traveling and want to build Sage while on an airplane (and without paying the wifi fees)? You can come up with plenty of reasons.

Inconsistent access is a nightmare if you need to get a large file (e.g. current Sage source tarball), as opposed to several small files.

Hence tarballs on thumb drives.

Besides, mind you, much more popular than Sage software distribution systems, e.g. ones for Python, Haskell, OCaml do require internet access, by default. Convenience for the majority (and for developers) ought to come before exotic usage cases.

If/when we make sagenb into an optional or experimental package, I have no objections to doing the same with these dependencies. But until then, in what way is including a dozen packages an inconvenience? It doesn't take up any extra disk space or extra time building Sage since we're already including them, it just adds a few directories in build/pkgs. How is this more of an inconvenience than requiring internet access during compilation of Sage?

Replying to @vbraun:

IMHO those use cases are better served by having a script that downloads all standard+optional packages. I already have a script where you just have to

sage-package download :standard: :optional:

to grab them all in one go.

And how are Sage developers supposed to know about this, if they are going to be traveling and without good internet access for a while? It sounds like a good idea, but I still don't see what it saves vs. making these standard Sage packages (for as long as sagenb is a standard Sage package, but no longer).

[vbraun]

comment:67

As I said above, I do agree that we should make the packages standard now. But in the future we should probably reevaluate the wisdom for why we are making source tarballs with two separate web-based notebooks.

[kcrisman]

comment:68

As I said above, I do agree that we should make the packages standard now. But in the future we should probably reevaluate the wisdom for why we are making source tarballs with two separate web-based notebooks.

I'm not quite sure why the "web-based" is relevant here; why not say "browser-based" instead? I use my browser all the time without being on the internet - primarily as a Sage GUI, but most browsers now save things for later reading, I often save pages to show to students in situations where wireless isn't working and it opens in the browser, etc. After all, most people aren't running their local sagenb or Jupyter as a web service.

Anyway, this solution seems to me like a good intermediate solution.

[dimpase]

comment:69

OK, I'll be happy to review this; I just merged your pull request. Could you perhaps make an official tarball, based off https://github.com/sagemath/sagenb ?

[jdemeyer]

comment:70

No need for a new tarball, just define the one already on this ticket as the official one.

[sagetrac-git]

Changed commit from 161ac3b to b5fb38b

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

b5fb38b

Use "setup.py install" to install Python packages

[jdemeyer]

comment:72

Volker just reminded me that we cannot use pip install . currently, since that needs SSL.

[dimpase]

comment:73

Replying to @jdemeyer:

Volker just reminded me that we cannot use pip install . currently, since that needs SSL.

pip is a standard Sage package. Are you saying that it is broken, generally speaking?
Probably if you build everything without ssl then it would work too, no?

[jdemeyer]

comment:74

Replying to @dimpase:

Replying to @jdemeyer:

Volker just reminded me that we cannot use pip install . currently, since that needs SSL.

pip is a standard Sage package. Are you saying that it is broken, generally speaking?

Sort of: it always requires SSL even if you don't make any internet connection.

Probably if you build everything without ssl then it would work too, no?

Apparently not.

[dimpase]

comment:75

looks OK on linux. let me see if it's also OK on OSX, just in case.

[dimpase]

Changed upstream from Reported upstream. No feedback yet. to None of the above - read trac for reasoning.

[dimpase]

Changed reviewer from Salvatore Stella to Salvatore Stella, Dima Pasechnik

[dimpase]

comment:76

OK.lightly tested on few random worksheets, old and new, on OSX too.

[jdemeyer]

comment:77

Thanks!

[jdemeyer]

Changed upstream from None of the above - read trac for reasoning. to Completely fixed; Fix reported upstream

[vbraun]

Changed branch from u/jdemeyer/ticket/14840 to b5fb38b