Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: ubuntu-latest now has MySQL 8.0.26, let us override it with latest 8.0.x #9368

Merged
merged 3 commits into from
Dec 14, 2021

Conversation

deepthi
Copy link
Member

@deepthi deepthi commented Dec 13, 2021

Description

Actions workflows that run on ubuntu-latest have been failing for ~12 hours now.
This is because of changes made to the images
actions/runner-images#4708
actions/runner-images#4674

Since the image already includes a mysql install, perhaps we can stop trying to install it ourselves.
Let's try to install the latest 8.0.x version ourselves.

Related Issue(s)

Checklist

  • Should this PR be backported?
  • Tests were added or are not required
  • Documentation was added or is not required

Deployment Notes

Signed-off-by: deepthi <deepthi@planetscale.com>
@mattlord
Copy link
Contributor

mattlord commented Dec 13, 2021

I'm OK with this as a temporary workaround but it breaks a reasonable assumption here, that is: you install the latest official MySQL version with apt-get install. That's what we normally want to do as we try and regularly upgrade to stay up to date. We do also in some cases alter behavior based on the patch version.

And even if it caused no immediate issues, not being able to rely on apt being able to manage the dependencies is likely to crop up in unexpected ways moving forward.

This is one potential short-term option though, for sure. Others may be:

  • Specifying a specific image that is fine, e.g. 18.04 or the specific version latest pointed to before this change
  • Explicitly adding the upstream MySQL APT repository (potential long-term solution if we need it)
  • Pinning and managing usage of a specific MySQL version on our side and then installing via dpkg directly (potential long-term solution if we need it)
  • ...

@deepthi
Copy link
Member Author

deepthi commented Dec 13, 2021

This is one potential short-term option though, for sure. Others may be:

  • Specifying a specific image that is fine, e.g. 18.04 or the specific version latest pointed to before this change
  • Explicitly adding the upstream MySQL APT repository (potential long-term solution if we need it)
  • Pinning and managing usage of a specific MySQL version on our side and then installing via dpkg directly (potential long-term solution if we need it)

I'm reluctant to move everything back to 18.04, we had tried to actually move everything forward to 20.04 at some point but didn't complete that work, see #7850
The others are good options, this was just the fastest, smallest change to get us unblocked.
I can try the second option and see if I can do it correctly :)

Signed-off-by: deepthi <deepthi@planetscale.com>
@deepthi deepthi changed the title CI: ubuntu-latest already has MySQL 8.0, don't try to install it CI: ubuntu-latest now has MySQL 8.0.26, let us override it with latest 8.0.x Dec 13, 2021
…the latest

Signed-off-by: deepthi <deepthi@planetscale.com>
Comment on lines +38 to +48
sudo DEBIAN_FRONTEND="noninteractive" apt-get update

# Uninstall any previously installed MySQL first
sudo systemctl stop apparmor
sudo DEBIAN_FRONTEND="noninteractive" apt-get remove -y --purge mysql-server mysql-client mysql-common
sudo apt-get -y autoremove
sudo apt-get -y autoclean
sudo deluser mysql
sudo rm -rf /var/lib/mysql
sudo rm -rf /etc/mysql

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't any of this is necessary, is it? Any previous installation and its files don't matter as we use non-default values anyway and this will slow down the CI jobs further.

sudo service mysql stop
sudo service etcd stop
sudo bash -c "echo '/usr/sbin/mysqld { }' > /etc/apparmor.d/usr.sbin.mysqld" # https://bugs.launchpad.net/ubuntu/+source/mariadb-10.1/+bug/1806263
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove this. This was about running MariaDB after MySQL was previously installed -- which is not relevant for us here.

sudo apt-get update
sudo apt-get install -y mysql-server mysql-client make unzip g++ etcd curl git wget eatmydata
sudo DEBIAN_FRONTEND="noninteractive" apt-get install -y mysql-server mysql-client
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove this and just keep the original apt-get install for all of our packages.

sudo ln -s /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/
sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.mysqld
sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.mysqld || echo "could not remove mysqld profile"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this step failing is not fatal, then why do it at all?

Copy link
Contributor

@mattlord mattlord left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a few comments/nits, but want to unblock you and thus everyone else. :-)

I can make another commit, or another PR, if you want.

@deepthi
Copy link
Member Author

deepthi commented Dec 14, 2021

I'm going to merge this as-is to unblock everyone, and @mattlord will do a follow up PR cleaning up how we are installing mysql in some of the tests.

@deepthi deepthi merged commit 4dd92c3 into vitessio:main Dec 14, 2021
@deepthi deepthi deleted the ds-fix-ci branch December 14, 2021 00:28
Comment on lines +38 to +47
sudo DEBIAN_FRONTEND="noninteractive" apt-get update

# Uninstall any previously installed MySQL first
sudo systemctl stop apparmor
sudo DEBIAN_FRONTEND="noninteractive" apt-get remove -y --purge mysql-server mysql-client mysql-common
sudo apt-get -y autoremove
sudo apt-get -y autoclean
sudo deluser mysql
sudo rm -rf /var/lib/mysql
sudo rm -rf /etc/mysql
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a generated file, so we should be changing the code generation and not the file directly. The file that generates this is ci_workflow_gen.go

@doeg
Copy link
Contributor

doeg commented Dec 14, 2021

Thank you both for looking into this! ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants