Development libraries in the release version, including GPL-2.0 licensed mysqlclient

[martinkonopka]

Dockerfile defines installation of the packages for development from requirements/development.txt also to lean version of Docker image. This results in inclusion of packages for development in a Docker image not tagged for development, i.e. apache/superset:latest (not latest-dev). I would like to verify, if it is desired.

I am asking because I noticed the mysqlclient package between the packages in requirements/development.txt which is licensed under GPL-2.0. Wouldn't this impose the requirement to license the whole project under GPL-2.0?

mysqlclient package license: https://github.com/PyMySQL/mysqlclient/blob/main/LICENSE

How to reproduce the bug

1. Run pip freeze in Docker image with release version of Superset:
   docker run --rm apache/superset:latest pip freeze
2. Output contains mysqlclient library which is installed from requirements/development.txt:
   

   superset/requirements/development.txt

   Line 57 in bd3166b

   mysqlclient==2.1.0

3. The library is licensed under GPL-2:
   docker run --rm apache/superset:latest pip show -v mysqlclient

Expected results

I did not expect to find GPL-2.0 licensed package installed if the project is licensed under Apache-2.0.

Actual results

Name: mysqlclient
Version: 2.1.0
Summary: Python interface to MySQL
Home-page: https://github.com/PyMySQL/mysqlclient
Author: Inada Naoki
Author-email: [songofacandy@gmail.com](mailto:songofacandy@gmail.com)
License: GPL
Location: /usr/local/lib/python3.8/site-packages
Requires:
Required-by:
Metadata-Version: 2.1
Installer: pip
Classifiers:
  Development Status :: 5 - Production/Stable
  Environment :: Other Environment
  License :: OSI Approved :: GNU General Public License (GPL)
  Operating System :: MacOS :: MacOS X
  Operating System :: Microsoft :: Windows :: Windows NT/2000
  Operating System :: OS Independent
  Operating System :: POSIX
  Operating System :: POSIX :: Linux
  Operating System :: Unix
  Programming Language :: C
  Programming Language :: Python
  Programming Language :: Python :: 3
  Programming Language :: Python :: 3.6
  Programming Language :: Python :: 3.7
  Programming Language :: Python :: 3.8
  Programming Language :: Python :: 3.9
  Programming Language :: Python :: 3.10
  Topic :: Database
  Topic :: Database :: Database Engines/Servers
Entry-points:

Environment

(please complete the following information):

• superset version: noticed in 1.3.0, still present in the latest version

Checklist

Make sure to follow these steps before submitting your issue - thank you!

• I have checked the superset logs for python stacktraces and included it here as text if there are any.
• I have reproduced the issue with at least the latest released version of superset.
• I have checked the issue tracker for the same issue and I haven't found one similar.

[rusackas]

I looked into this once, but the licensing of that package was super ambiguous, and a lot of people were wrestling with the same ambiguity. I'm not sure how big of a problem this is... maybe @mistercrunch or @dpgaspar can let us know how big of a concern this is?

[mistercrunch]

I'd say flush it out of development.in

[rusackas]

I think that would kill our Python-Integration / test-mysql action. If we can live without that, I can open a PR. I seem to recall that the license for this package was fairly ambiguous, but I'll check in and see if this situation has become any clearer.

[mistercrunch]

Can we move it to testing.in?

[mistercrunch]

I'd do it quickly, but pip-compile-multi has been broken - re: #26944

[rusackas]

Assuming we're in a good place to tackle this now... while I see a development.txt file that we can delete it from, I don't see a testing.txt file to add it to. This seems like it'd take some setup if I'm not mistaken, and we'll probably want to move several other dependencies along with it?

[mistercrunch]

Yes when I took on python dependency management originally I wanted to simplify from 5-6 (base, development, ci, docker, testing) bundles down to 2. We could bring back some of that complexity if we wanted to (back to 3 total bundles).

I think we might want to create a new requirements/ci.in build that would be used for CI (would be base.in + mysqlclient, hive/presto + whatever is used in CI), and remove mysqlclient from development.in.

[mistercrunch]

I think this is a non-issue now. mysqlclient is no longer in the lean image AFAIK. lean installs these requirements which does not include mysqlclient. Note that it's still installed in the dev image which should be ok.