You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We (Arrikto) have been deploying multi-user KFP as part of KF with Istio sidecars. We have noticed that some Istio connection resets made the KFP API server <-> MySQL db communication break. And this was happening a lot.
We were essentially bumping into this issue: go-sql-driver/mysql#674
So after some research, we found out that we are currently using an old version of the Go SQL driver for MySQL.
We have lots of reasons to upgrade it, see here for a great article on this. It contains a summary of important bugs which got fixed in v1.5.0.
After upgrading it for our deployments, this issue was greatly eliminated to the point that it almost never happens [or on the very rare occasion that it may occur, it is totally unnoticeable].
The text was updated successfully, but these errors were encountered:
elikatsis
added a commit
to arrikto/kubeflow-pipelines
that referenced
this issue
Dec 17, 2020
We (Arrikto) have been deploying multi-user KFP as part of KF with Istio sidecars. We have noticed that some Istio connection resets made the KFP API server <-> MySQL db communication break. And this was happening a lot.
We were essentially bumping into this issue: go-sql-driver/mysql#674
So after some research, we found out that we are currently using an old version of the Go SQL driver for MySQL.
We have lots of reasons to upgrade it, see here for a great article on this. It contains a summary of important bugs which got fixed in v1.5.0.
After upgrading it for our deployments, this issue was greatly eliminated to the point that it almost never happens [or on the very rare occasion that it may occur, it is totally unnoticeable].
The text was updated successfully, but these errors were encountered: