Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(license): reorder logic of how python package licenses are acquired #6220

Merged
merged 4 commits into from
Mar 8, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
fix(license): reorder logic of how python package licenses are acquired
dus7eh committed Feb 28, 2024
commit d82a8164f7f77ef030c754b0111ebfa6e2212bb6
48 changes: 29 additions & 19 deletions pkg/dependency/parser/python/packaging/parse.go
Original file line number Diff line number Diff line change
@@ -20,8 +20,6 @@ func NewParser() types.Parser {
return &Parser{}
}

// Parse parses egg and wheel metadata.
// e.g. .egg-info/PKG-INFO and dist-info/METADATA
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you have reason to remove this comment?

func (*Parser) Parse(r xio.ReadSeekerAt) ([]types.Library, []types.Dependency, error) {
rd := textproto.NewReader(bufio.NewReader(r))
h, err := rd.ReadMIMEHeader()
@@ -40,31 +38,43 @@ func (*Parser) Parse(r xio.ReadSeekerAt) ([]types.Library, []types.Dependency, e
return nil, nil, xerrors.New("name or version is empty")
}

// "License-Expression" takes precedence as "License" is deprecated.
// cf. https://peps.python.org/pep-0639/#deprecate-license-field
var license string
// "License-Expression" takes precedence in accordance with https://peps.python.org/pep-0639/#deprecate-license-field
// Although keep in mind that pep-0639 is still in draft.
var licenses types.Libraries
if l := h.Get("License-Expression"); l != "" {
license = l
} else if l := h.Get("License"); l != "" {
license = l
licenses = types.Libraries{{Name: name, Version: version, License: l}}
} else {
// Get possible multiple occurrences of licenses from "Classifier: License" field
// When present it should define the license whereas "License" would define any additional exceptions or modifications
// ref. https://packaging.python.org/en/latest/specifications/core-metadata/#license
for _, classifier := range h.Values("Classifier") {
if strings.HasPrefix(classifier, "License :: ") {
values := strings.Split(classifier, " :: ")
license = values[len(values)-1]
break
licensename := values[len(values)-1]
// According to the classifier list https://pypi.org/classifiers/ there is one classifier which seems more like a grouping
// It has no specific license definition (Classifier: License :: OSI Approved) - it is skipped
if licensename != "OSI Approved" {
licenses = append(licenses, types.Library{Name: name, Version: version, License: values[len(values)-1]})
}
}
}

if l := h.Get("License"); l != "" {
if len(licenses) == 0 {
licenses = types.Libraries{{Name: name, Version: version, License: l}}
} else {
// Instead of warning would be probably better to pass the contents of "License" field to results
log.Logger.Warnf("License acquired from METADATA classifiers may be subject to additional terms for [%s]", name)
}
}
}
if license == "" && h.Get("License-File") != "" {
license = "file://" + h.Get("License-File")
if len(licenses) == 0 {
if lf := h.Get("License-File"); lf != "" {
licenses = types.Libraries{{Name: name, Version: version, License: "file://" + lf}}
} else {
licenses = append(licenses, types.Library{Name: name, Version: version})
}
}

return []types.Library{
{
Name: name,
Version: version,
License: license,
},
}, nil, nil
return licenses, nil, nil
}
20 changes: 19 additions & 1 deletion pkg/dependency/parser/python/packaging/parse_test.go
Original file line number Diff line number Diff line change
@@ -76,7 +76,25 @@ func TestParse(t *testing.T) {
// for single METADATA file with known name
// cat "{{ libname }}.METADATA | grep -e "^Name:" -e "^Version:" -e "^License:" | cut -d" " -f2- | tr "\n" "\t" | awk -F "\t" '{printf("\{\""$1"\", \""$2"\", \""$3"\"\}\n")}'
input: "testdata/distlib-0.3.1.METADATA",
want: []types.Library{{Name: "distlib", Version: "0.3.1", License: "Python license"}},
want: []types.Library{{Name: "distlib", Version: "0.3.1", License: "Python Software Foundation License"}},
},
{
name: "wheel METADATA",
// Input defines "Classifier: License" but it ends at "OSI Approved" which doesn't define any specific license, thus "License" field is added to results
input: "testdata/asyncssh-2.14.2.METADATA",

want: []types.Library{{Name: "asyncssh", Version: "2.14.2", License: "Eclipse Public License v2.0"}},
},
{
name: "wheel METADATA",
// Input defines multiple "Classifier: License"
input: "testdata/pyphen-0.14.0.METADATA",

want: []types.Library{
{Name: "pyphen", Version: "0.14.0", License: "GNU General Public License v2 or later (GPLv2+)"},
{Name: "pyphen", Version: "0.14.0", License: "GNU Lesser General Public License v2 or later (LGPLv2+)"},
{Name: "pyphen", Version: "0.14.0", License: "Mozilla Public License 1.1 (MPL 1.1)"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your solution creates duplicate packages. It is not right.

},
},
{
name: "invalid",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can reduce the length of file by removing lines that are unnecessary for tests.

Original file line number Diff line number Diff line change
@@ -0,0 +1,277 @@
Metadata-Version: 2.1
Name: asyncssh
Version: 2.14.2
Summary: AsyncSSH: Asynchronous SSHv2 client and server library
Home-page: http://asyncssh.timeheart.net
Author: Ron Frederick
Author-email: ronf@timeheart.net
License: Eclipse Public License v2.0
Project-URL: Documentation, https://asyncssh.readthedocs.io
Project-URL: Source, https://github.com/ronf/asyncssh
Project-URL: Tracker, https://github.com/ronf/asyncssh/issues
Platform: Any
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: POSIX
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Internet
Classifier: Topic :: Security :: Cryptography
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Networking
Requires-Python: >= 3.6
License-File: LICENSE
Requires-Dist: cryptography (>=39.0)
Requires-Dist: typing-extensions (>=3.6)
Provides-Extra: bcrypt
Requires-Dist: bcrypt (>=3.1.3) ; extra == 'bcrypt'
Provides-Extra: fido2
Requires-Dist: fido2 (>=0.9.2) ; extra == 'fido2'
Provides-Extra: gssapi
Requires-Dist: gssapi (>=1.2.0) ; extra == 'gssapi'
Provides-Extra: libnacl
Requires-Dist: libnacl (>=1.4.2) ; extra == 'libnacl'
Provides-Extra: pkcs11
Requires-Dist: python-pkcs11 (>=0.7.0) ; extra == 'pkcs11'
Provides-Extra: pyopenssl
Requires-Dist: pyOpenSSL (>=23.0.0) ; extra == 'pyopenssl'
Provides-Extra: pywin32
Requires-Dist: pywin32 (>=227) ; extra == 'pywin32'

.. image:: https://readthedocs.org/projects/asyncssh/badge/?version=latest
:target: https://asyncssh.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status

.. image:: https://img.shields.io/pypi/v/asyncssh.svg
:target: https://pypi.python.org/pypi/asyncssh/
:alt: AsyncSSH PyPI Project


AsyncSSH: Asynchronous SSH for Python
=====================================

AsyncSSH is a Python package which provides an asynchronous client and
server implementation of the SSHv2 protocol on top of the Python 3.6+
asyncio framework.

.. code:: python

import asyncio, asyncssh, sys

async def run_client():
async with asyncssh.connect('localhost') as conn:
result = await conn.run('echo "Hello!"', check=True)
print(result.stdout, end='')

try:
asyncio.get_event_loop().run_until_complete(run_client())
except (OSError, asyncssh.Error) as exc:
sys.exit('SSH connection failed: ' + str(exc))

Check out the `examples`__ to get started!

__ http://asyncssh.readthedocs.io/en/stable/#client-examples

Features
--------

* Full support for SSHv2, SFTP, and SCP client and server functions

* Shell, command, and subsystem channels
* Environment variables, terminal type, and window size
* Direct and forwarded TCP/IP channels
* OpenSSH-compatible direct and forwarded UNIX domain socket channels
* Local and remote TCP/IP port forwarding
* Local and remote UNIX domain socket forwarding
* Dynamic TCP/IP port forwarding via SOCKS
* X11 forwarding support on both the client and the server
* SFTP protocol version 3 with OpenSSH extensions

* Experimental support for SFTP versions 4-6, when requested

* SCP protocol support, including third-party remote to remote copies

* Multiple simultaneous sessions on a single SSH connection
* Multiple SSH connections in a single event loop
* Byte and string based I/O with settable encoding
* A variety of `key exchange`__, `encryption`__, and `MAC`__ algorithms

* Including OpenSSH post-quantum kex algorithm
sntrup761x25519-sha512\@openssh.com

* Support for `gzip compression`__

* Including OpenSSH variant to delay compression until after auth

* User and host-based public key, password, and keyboard-interactive
authentication methods

* Many types and formats of `public keys and certificates`__

* Including OpenSSH-compatible support for U2F and FIDO2 security keys
* Including PKCS#11 support for accessing PIV security tokens
* Including support for X.509 certificates as defined in RFC 6187

* Support for accessing keys managed by `ssh-agent`__ on UNIX systems

* Including agent forwarding support on both the client and the server

* Support for accessing keys managed by PuTTY's Pageant agent on Windows
* Support for accessing host keys via OpenSSH's ssh-keysign
* OpenSSH-style `known_hosts file`__ support
* OpenSSH-style `authorized_keys file`__ support
* Partial support for `OpenSSH-style configuration files`__
* Compatibility with OpenSSH "Encrypt then MAC" option for better security
* Time and byte-count based session key renegotiation
* Designed to be easy to extend to support new forms of key exchange,
authentication, encryption, and compression algorithms

__ http://asyncssh.readthedocs.io/en/stable/api.html#key-exchange-algorithms
__ http://asyncssh.readthedocs.io/en/stable/api.html#encryption-algorithms
__ http://asyncssh.readthedocs.io/en/stable/api.html#mac-algorithms
__ http://asyncssh.readthedocs.io/en/stable/api.html#compression-algorithms
__ http://asyncssh.readthedocs.io/en/stable/api.html#public-key-support
__ http://asyncssh.readthedocs.io/en/stable/api.html#ssh-agent-support
__ http://asyncssh.readthedocs.io/en/stable/api.html#known-hosts
__ http://asyncssh.readthedocs.io/en/stable/api.html#authorized-keys
__ http://asyncssh.readthedocs.io/en/stable/api.html#config-file-support

License
-------

This package is released under the following terms:

Copyright (c) 2013-2022 by Ron Frederick <ronf@timeheart.net> and others.

This program and the accompanying materials are made available under
the terms of the Eclipse Public License v2.0 which accompanies this
distribution and is available at:

http://www.eclipse.org/legal/epl-2.0/

This program may also be made available under the following secondary
licenses when the conditions for such availability set forth in the
Eclipse Public License v2.0 are satisfied:

GNU General Public License, Version 2.0, or any later versions of
that license

SPDX-License-Identifier: EPL-2.0 OR GPL-2.0-or-later

For more information about this license, please see the `Eclipse
Public License FAQ <https://www.eclipse.org/legal/epl-2.0/faq.php>`_.

Prerequisites
-------------

To use AsyncSSH 2.0 or later, you need the following:

* Python 3.6 or later
* cryptography (PyCA) 3.1 or later

Installation
------------

Install AsyncSSH by running:

::

pip install asyncssh

Optional Extras
^^^^^^^^^^^^^^^

There are some optional modules you can install to enable additional
functionality:

* Install bcrypt from https://pypi.python.org/pypi/bcrypt
if you want support for OpenSSH private key encryption.

* Install fido2 from https://pypi.org/project/fido2 if you want support
for key exchange and authentication with U2F/FIDO2 security keys.

* Install python-pkcs11 from https://pypi.org/project/python-pkcs11 if
you want support for accessing PIV keys on PKCS#11 security tokens.

* Install gssapi from https://pypi.python.org/pypi/gssapi if you
want support for GSSAPI key exchange and authentication on UNIX.

* Install liboqs from https://github.com/open-quantum-safe/liboqs
if you want support for the OpenSSH post-quantum key exchange
algorithm sntrup761x25519-sha512\@openssh.com.

* Install libsodium from https://github.com/jedisct1/libsodium
and libnacl from https://pypi.python.org/pypi/libnacl if you have
a version of OpenSSL older than 1.1.1b installed and you want
support for Curve25519 key exchange, Ed25519 keys and certificates,
or the Chacha20-Poly1305 cipher.

* Install libnettle from http://www.lysator.liu.se/~nisse/nettle/
if you want support for UMAC cryptographic hashes.

* Install pyOpenSSL from https://pypi.python.org/pypi/pyOpenSSL
if you want support for X.509 certificate authentication.

* Install pywin32 from https://pypi.python.org/pypi/pywin32 if you
want support for using the Pageant agent or support for GSSAPI
key exchange and authentication on Windows.

AsyncSSH defines the following optional PyPI extra packages to make it
easy to install any or all of these dependencies:

| bcrypt
| fido2
| gssapi
| libnacl
| pkcs11
| pyOpenSSL
| pywin32

For example, to install bcrypt, fido2, gssapi, libnacl, pkcs11, and
pyOpenSSL on UNIX, you can run:

::

pip install 'asyncssh[bcrypt,fido2,gssapi,libnacl,pkcs11,pyOpenSSL]'

To install bcrypt, fido2, libnacl, pkcs11, pyOpenSSL, and pywin32 on
Windows, you can run:

::

pip install 'asyncssh[bcrypt,fido2,libnacl,pkcs11,pyOpenSSL,pywin32]'

Note that you will still need to manually install the libsodium library
listed above for libnacl to work correctly and/or libnettle for UMAC
support. Unfortunately, since liboqs, libsodium, and libnettle are not
Python packages, they cannot be directly installed using pip.

Installing the development branch
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If you would like to install the development branch of asyncssh directly
from Github, you can use the following command to do this:

::

pip install git+https://github.com/ronf/asyncssh@develop

Mailing Lists
-------------

Three mailing lists are available for AsyncSSH:

* `asyncssh-announce@googlegroups.com`__: Project announcements
* `asyncssh-dev@googlegroups.com`__: Development discussions
* `asyncssh-users@googlegroups.com`__: End-user discussions

__ http://groups.google.com/d/forum/asyncssh-announce
__ http://groups.google.com/d/forum/asyncssh-dev
__ http://groups.google.com/d/forum/asyncssh-users
Loading