-
Notifications
You must be signed in to change notification settings - Fork 14.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docker image does not have mysqlclient #29727
Comments
Yes, the removal of the To include the necessary dependencies in the Docker image by default, you can create a
If you are running a customized Docker image, rebuild your local image with the new driver baked in: docker compose build --force-rm After the rebuild is complete, relaunch Superset by running: docker compose up This approach ensures that the |
Yes, that's intentional, we try not to bloat our docker images as there's an infinity of drivers that you may or may not want as part of your Superset environment, and there's a maintenance burden associated with that too. MySQL is pretty well supported as a metadata backend for Superset (we run some of our CI against it), and we may want to improve the documentation to clarify how to do this. In your typical k8s setup you'll need a dockerfile that inherits from ours, add the apt-get packages and pip installs mysqlclient. |
I wrote the comment above and then I was thinking "but wait, how we're able to build the mysql-related matrices in our CI?". Turns out we do have the os-level mysql dep in our Dockerfile here https://github.com/apache/superset/blob/master/Dockerfile#L78, but arguably maybe we shouldn't. Then our "developement" python requirements, indirectly installs mysqlclient, and that happens in our CI https://github.com/apache/superset/blob/master/requirements/development.in So I'm unclear why your pip install failed. Are you using one of our dockers? If so I'm pretty sure gcc should be on there and mysqlclient installable at this time. |
Might this little quagmire be resolved by removing it? |
Related: #29771 |
Using the official image, simply run it and try to install. You will see it fails. Because it needs some deps which is not included to image as I mentioned in issue.
|
Oh after digging in the Dockerfile I understand why it's not the case. We're playing tricks to install apt-get's relevant, the RUN statement that plays that trick: https://github.com/apache/superset/blob/master/Dockerfile#L93-L97 Also note that in #29771 I'm fully removing all mysql-related packages from the lean build as we really shouldn't ship a binary with GPL in it, though unclear whether a docker image IS a binary under GPL, but cleaning it out nonetheless. As a workaround, you should be able to install the mysql deps on top lean by install build-essentials, the mysql apt-get client package, and then running pip install mysqlclient. It'd be great if someone could add this to the docs, somewhere under the installation section |
Bug description
I'm testing 4.1.0rc1 using a Docker image and noticed it lacks the mysqlclient package that previous versions included. Was this removal intentional? Attempting to install it results in the following error.
I resolved the issue by installing the necessary packages. A better solution would be to include it as a built-in image.
apt install -y python3-dev default-libmysqlclient-dev build-essential pkg-config
How to reproduce the bug
Screenshots/recordings
No response
Superset version
master / latest-dev
Python version
3.9
Node version
16
Browser
Chrome
Additional context
No response
Checklist
The text was updated successfully, but these errors were encountered: