Possible legal problems with vendor/patched libraries #1961
Comments
|
Before I agree (though I can say I’ve been thinking about this now that our bigger problems are addressed) I’d want to hear @ncoghlan’s thoughts. My initial instinct is that this seems like a valid concern and we would appreciate help. Thanks for documenting it! |
|
Aye, we should definitely fix this, and I think @hroncok's suggested resolution is a good one. |
|
Awesome. Let’s sort this out |
|
Regarding strict-rfc3339 replacement, unfortunately rfc3339 does not do string parsing, only formatting. utcdatetime might be a viable alternative. It wouldn’t be astromical to build a helper library from the ground up based on |
|
OK. I'll start with the license files. If anyone is able to do the strict-rfc3339 thing, that would be great. If not, I'll look into it after the first thing is done. |
|
Note that pipenv bundles pip as notpip and that bundles a lot as well and might have the very same problem. |
|
But since pip itself is licensed under MIT I assume there shouldn’t be a GPL problem? |
|
I haven't checked all the bundles libraries in pip for GPL. Yet there is a licensing problem. |
|
There is, indeed. What I meant is that pip (notpip) wouldn’t be a source of licensing problems unless pip itself does, since pip itself is licensed under MIT, and Pipenv is MIT as well. |
|
Pip itself is a source of licensing problem now, because pipenv violates pip's license terms. However, the license of pip (MIT) is compatible with license of pipenv (also MIT). |
contoml: MIT from https://github.com/jumpscale7/python-consistent-toml/blob/master/LICENSE crayons: has the same copyright note and license as pipenv, so it's fine without notpip: MIT from https://github.com/pypa/pip/blob/master/LICENSE.txt (bundles internally a lot, I haven't yet addressed that) pew: MIT from https://github.com/berdario/pew/blob/master/LICENSE no copyright notice, filled pew-org/pew#190 pipfile: BSD+Apache from: https://github.com/pypa/pipfile/blob/master/LICENSE https://github.com/pypa/pipfile/blob/master/LICENSE.APACHE https://github.com/pypa/pipfile/blob/master/LICENSE.BSD piptools: BSD from https://github.com/jazzband/pip-tools/blob/master/LICENSE prettytoml: MIT from https://pypi.python.org/pypi/prettytoml/0.3 sdist safety: MIT already included in zip as LICENSE See pypa#1961
|
Fun fact, pipenv bundles requests and colorama 3 times: $ find -name requests
./vendor/pip9/_vendor/requests
./vendor/requests
./patched/notpip/_vendor/requests
$ find -name colorama
./vendor/pip9/_vendor/colorama
./vendor/colorama
./patched/notpip/_vendor/coloramaThe copies are identical. |
|
Problems:
|
appdirs: MIT from https://github.com/ActiveState/appdirs/blob/master/LICENSE.txt backports: Python from https://github.com/python/cpython/blob/master/LICENSE click: BSD from https://github.com/pallets/click/blob/master/LICENSE click_completion: MIT from https://github.com/click-contrib/click-completion/blob/master/LICENSE click_didyoumean: MIT from https://github.com/click-contrib/click-didyoumean/blob/master/LICENSE colorama: BSD from https://github.com/tartley/colorama/blob/master/LICENSE.txt docopt: MIT from https://github.com/docopt/docopt/blob/master/LICENSE-MIT dotenv: BSD from https://github.com/theskumar/python-dotenv/blob/master/LICENSE first: MIT from https://github.com/hynek/first/blob/master/LICENSE iso8601: MIT from https://bitbucket.org/micktwomey/pyiso8601/src/default/LICENSE jinja2: BSD from https://github.com/pallets/jinja/blob/master/LICENSE and AUTHORS markupsafe: BSD from https://github.com/pallets/markupsafe/blob/master/LICENSE and AUTHORS pathlib2: MIT from https://github.com/mcmtroffaes/pathlib2/blob/develop/LICENSE.rst pexpect: ISC from https://github.com/pexpect/pexpect/blob/master/LICENSE pip9: MIT from https://github.com/pypa/pip/blob/master/LICENSE.txt (bundles internally a lot, I haven't yet addressed that) pipdeptree: MIT from https://github.com/naiquevin/pipdeptree/blob/master/LICENSE pipreqs: Apache from https://github.com/bndr/pipreqs/blob/master/LICENSE ptyprocess: ISC from https://github.com/pexpect/ptyprocess/blob/master/LICENSE pytoml: No-notice MIT License from https://github.com/avakar/pytoml/blob/master/LICENSE (doesn't technically need to be added, yet I added it) requests: Apache notice from https://github.com/requests/requests/blob/master/LICENSE (bundles chardet and urllib3 internally, I haven't yet addressed that) semver: MIT from https://github.com/k-bx/python-semver/blob/master/LICENSE.txt shutilwhich: Python from https://github.com/mbr/shutilwhich/blob/master/LICENSE six: MIT from https://github.com/benjaminp/six/blob/master/LICENSE toml: MIT from https://github.com/uiri/toml/blob/0.9.2/LICENSE yarg: MIT from https://github.com/kura/yarg/blob/master/LICENSE Soem vendor licenses are still missing, see pypa#1961 (comment)
contoml: MIT from https://github.com/jumpscale7/python-consistent-toml/blob/master/LICENSE crayons: has the same copyright note and license as pipenv, so it's fine without notpip: MIT from https://github.com/pypa/pip/blob/master/LICENSE.txt (bundles internally a lot, I haven't yet addressed that) pew: MIT from https://github.com/berdario/pew/blob/master/LICENSE no copyright notice, filled pew-org/pew#190 pipfile: BSD+Apache from: https://github.com/pypa/pipfile/blob/master/LICENSE https://github.com/pypa/pipfile/blob/master/LICENSE.APACHE https://github.com/pypa/pipfile/blob/master/LICENSE.BSD piptools: BSD from https://github.com/jazzband/pip-tools/blob/master/LICENSE prettytoml: MIT from https://pypi.python.org/pypi/prettytoml/0.3 sdist safety: MIT already included in zip as LICENSE See pypa#1961
appdirs: MIT from https://github.com/ActiveState/appdirs/blob/master/LICENSE.txt backports: Python from https://github.com/python/cpython/blob/master/LICENSE click: BSD from https://github.com/pallets/click/blob/master/LICENSE click_completion: MIT from https://github.com/click-contrib/click-completion/blob/master/LICENSE click_didyoumean: MIT from https://github.com/click-contrib/click-didyoumean/blob/master/LICENSE colorama: BSD from https://github.com/tartley/colorama/blob/master/LICENSE.txt docopt: MIT from https://github.com/docopt/docopt/blob/master/LICENSE-MIT dotenv: BSD from https://github.com/theskumar/python-dotenv/blob/master/LICENSE first: MIT from https://github.com/hynek/first/blob/master/LICENSE iso8601: MIT from https://bitbucket.org/micktwomey/pyiso8601/src/default/LICENSE jinja2: BSD from https://github.com/pallets/jinja/blob/master/LICENSE and AUTHORS markupsafe: BSD from https://github.com/pallets/markupsafe/blob/master/LICENSE and AUTHORS pathlib2: MIT from https://github.com/mcmtroffaes/pathlib2/blob/develop/LICENSE.rst pexpect: ISC from https://github.com/pexpect/pexpect/blob/master/LICENSE pip9: MIT from https://github.com/pypa/pip/blob/master/LICENSE.txt (bundles internally a lot, I haven't yet addressed that) pipdeptree: MIT from https://github.com/naiquevin/pipdeptree/blob/master/LICENSE pipreqs: Apache from https://github.com/bndr/pipreqs/blob/master/LICENSE ptyprocess: ISC from https://github.com/pexpect/ptyprocess/blob/master/LICENSE pytoml: No-notice MIT License from https://github.com/avakar/pytoml/blob/master/LICENSE (doesn't technically need to be added, yet I added it) requests: Apache notice from https://github.com/requests/requests/blob/master/LICENSE (bundles chardet and urllib3 internally, I haven't yet addressed that) semver: MIT from https://github.com/k-bx/python-semver/blob/master/LICENSE.txt shutilwhich: Python from https://github.com/mbr/shutilwhich/blob/master/LICENSE six: MIT from https://github.com/benjaminp/six/blob/master/LICENSE toml: MIT from https://github.com/uiri/toml/blob/0.9.2/LICENSE yarg: MIT from https://github.com/kura/yarg/blob/master/LICENSE Soem vendor licenses are still missing, see pypa#1961 (comment)
|
I need help here. Should pipenv be GPLv3 or should Levenshtein and strict_rfc3339 go away? Also timestamp is a big no no. |
|
Bundled libraries inside bundled libraries need to be addressed as well, however, maybe on pip level mostly? |
|
Note that I cannot find a place where Levenshtein is used. It has been added in here: aeaabf4 but AFAIK it is not used at all. |
|
aeaabf4 was an old feature that Pipenv suggests package names during |
|
Gosh timestamp is virtually a one-liner. Pipenv’s pad-left. |
|
@hroncok I believe that we stopped using Levenshtein recently and can remove it (I thought about doing that yesterday even) -- and agree we do need to un-bundle the pip stuff most likely although that's more a time issue (we don't have any) timestamp -- where are we using this? I'll have to look strict_rfc3339 -- we need an alternative, @uranusjr was looking at this? requirements -- https://github.com/davidfischer/requirements-parser/blob/master/LICENSE.rst (BSD) |
|
GPL stuff needs to be axed |
|
@hroncok sorry for the delay on this but it's safe to say if we can find replacements for GPL'ed libraries or unlicensed code we can definitely axe it. Also I'm sure we can find something we currently have to convert to timestamps if we really need it |
appdirs: MIT from https://github.com/ActiveState/appdirs/blob/master/LICENSE.txt backports: Python from https://github.com/python/cpython/blob/master/LICENSE click: BSD from https://github.com/pallets/click/blob/master/LICENSE click_completion: MIT from https://github.com/click-contrib/click-completion/blob/master/LICENSE click_didyoumean: MIT from https://github.com/click-contrib/click-didyoumean/blob/master/LICENSE colorama: BSD from https://github.com/tartley/colorama/blob/master/LICENSE.txt docopt: MIT from https://github.com/docopt/docopt/blob/master/LICENSE-MIT dotenv: BSD from https://github.com/theskumar/python-dotenv/blob/master/LICENSE first: MIT from https://github.com/hynek/first/blob/master/LICENSE iso8601: MIT from https://bitbucket.org/micktwomey/pyiso8601/src/default/LICENSE jinja2: BSD from https://github.com/pallets/jinja/blob/master/LICENSE and AUTHORS markupsafe: BSD from https://github.com/pallets/markupsafe/blob/master/LICENSE and AUTHORS pathlib2: MIT from https://github.com/mcmtroffaes/pathlib2/blob/develop/LICENSE.rst pexpect: ISC from https://github.com/pexpect/pexpect/blob/master/LICENSE pip9: MIT from https://github.com/pypa/pip/blob/master/LICENSE.txt (bundles internally a lot, I haven't yet addressed that) pipdeptree: MIT from https://github.com/naiquevin/pipdeptree/blob/master/LICENSE pipreqs: Apache from https://github.com/bndr/pipreqs/blob/master/LICENSE ptyprocess: ISC from https://github.com/pexpect/ptyprocess/blob/master/LICENSE pytoml: No-notice MIT License from https://github.com/avakar/pytoml/blob/master/LICENSE (doesn't technically need to be added, yet I added it) requests: Apache notice from https://github.com/requests/requests/blob/master/LICENSE (bundles chardet and urllib3 internally, I haven't yet addressed that) semver: MIT from https://github.com/k-bx/python-semver/blob/master/LICENSE.txt shutilwhich: Python from https://github.com/mbr/shutilwhich/blob/master/LICENSE six: MIT from https://github.com/benjaminp/six/blob/master/LICENSE toml: MIT from https://github.com/uiri/toml/blob/0.9.2/LICENSE yarg: MIT from https://github.com/kura/yarg/blob/master/LICENSE Soem vendor licenses are still missing, see #1961 (comment)
requests.chardet: LGPLv2 from https://github.com/chardet/chardet/blob/master/LICENSE requests.urllib3: MIT from https://github.com/urllib3/urllib3/blob/master/LICENSE.txt See #1961
|
We can upstream issues to the mentioned libraries but you'll have more success just PR'ing them I'd guess, they are marked MIT and I assume that's how Kenneth wants them. I think we covered everything else, but let me know. I began working on some logic to automatically rebuild the license files if we ever lose them, loosely based on pip 10's new vendoring code. |
|
Also see pypa/pip#5213 the code is there. |
|
oh your implementation is much cleaner, if it's not too much trouble can you PR it back this way? |
|
Once it's landed in pip, I'll do what I can to make that work here. It bas been easy with pip given that the vendored libs are tracked there (I recommend adapting a similar workflow here and possibly only have one layer of bundled libs). |
|
@hroncok already began that with my hacked implementation which uses the same approach you took |
|
See |
techalchemy
added
Type: Vendored Dependencies
|
OK, AFAIK the only thing that misses licensefiles now is pip9's and notpip's _vendor. Any idea what is bundled there? The upstream vendor.txt file is missing, so the tracking is lost. I'll strat with https://github.com/pypa/pip/blob/9.0.3/pip/_vendor/vendor.txt and see where that leads me. |
Used the following: (modified from https://github.com/pypa/pip/blob/9.0.3/pip/_vendor/vendor.txt) setuptools==39.1.0 appdirs==1.4.0 distlib==0.2.4 distro==1.0.1 html5lib==1.0b10 six==1.10.0 colorama==0.3.7 requests==2.11.1 chardet==3.0.4 idna==2.6 urllib3==1.22 certifi==2018.1.18 CacheControl==0.11.7 lockfile==0.12.2 ordereddict==1.1 progress==1.2 ipaddress==1.0.17 packaging==16.8 pyparsing==2.1.10 retrying==1.3.3 webencodings==0.5 And the vendoring script from pypa/pip#5213 Fixes pypa#1961
|
#2094 should be the last needed batch. I suggest to adapt pypa/pip#5213 as much as possible later not to have to deal with this manually. Sharing the deps between pip9 and notpip would also be nice, if possible. |
|
@hroncok I already adapted and merged that code and used it to re-vendor everything. |
|
@hroncok one additional note -- we only have both versions of pip in teh first place in order to accommodate pip-tools which has a hard dependency on pip 9 and which handles our dependency resolution (and which we have made modifications to, which are now visible in the This will no longer be necessary after they accept my PR over at jazzband/pip-tools#657 which adds a compatibility layer so that pip-tools can use pip 8, 9 or 10. As a sidenote I believe these licenses will get picked up whenever we re-vendor pip assuming they get bundled in a release at some point, and if we don't automate them they will get wiped next time we re-vendor. |
Assuming they get installed with pip install. I've never tried that.
right, however i have no idea when you'll switch to pip 10, so I'd rather see this in before manually for 9, so we can finally package this for Fedora, as was our primary motivation for this. |
|
@techalchemy Thank you very much! Could I please have a release? |
|
Yes we have 11.10.2 planned so we have a few things left |
tl;dr
Pipenv probably violates couple of free software licenses by not shipping their text. It is also probably "infected" by GPL.
Details
Hello, we (@mcyprian, me and the Fedora Python SIG @fedora-python) are trying to finally package pipenv for Fedora, so user can just do
sudo dnf install pipenv. The Fedora package review request is in Red Hat Bugzilla 1564500.While we are trying to unbundle (unvendor?) most of the 3rd party libraries shipped with pipenv, we are in a bit of hurry so we decided to leave the libraries that are
bundled for now. The first category is a TODO for future, the second will probably remain bundled forever.
As part of the review of the Fedora package, the reviewer is obligated to check whether the package is licensed with approved free software/content license and whether the licensing information for the package is correct.
Missing licenses
This is where I found out that all the vendored 3rd party libraries are shipped without their LICENSE/COPYING/etc. files and the NOTICES file is shipped instead.
This is unacceptable for Fedora* and IMHO should not be acceptable for @pypa either. Most of the libraries are licensed with licenses that require the license text to be shipped. See MIT:
Or BSD:
This is similar for most of the permissive licenses. You should not strip the license file, in fact you need to ship it with the code unconditionally. You may put the license text and copyrights inside another file, such as NOTICES, but a link to pypi is IMHO not enough.
* This is currently only my opinion. It has not yet been reviewed by the Fedora legal team.
Copyleft
I also found out that strict_rfc3339 is shipped under the terms of the GNU General Public License version 3 or later. Needless to say that GPL is a copyleft license. By bundling this part of code inside pipenv, pipenv is "infected" with this license and shall be GPLv3 as well (which I think is undesired).
IANAL, however I'm quite confident that pipenv now violates couple of free software licenses including the GPL. This currently blocks us from inclusion into Fedora. Since pipenv is the recommended tool I think this shall be brought to @pypa.
Conclusion
What I believe shall be done:
I offer my help with collecting the licenses back, if that's agreed upon by pipenv maintainers. I can also try to replace strict_rfc3339 with rfc3339, however I haven't looked into it yet. There might also be other copylefted files (without a header that makes it obvious).
The text was updated successfully, but these errors were encountered: