Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refine fullbackup #570

Merged
merged 14 commits into from
Jun 18, 2019
Merged

refine fullbackup #570

merged 14 commits into from
Jun 18, 2019

Conversation

weekface
Copy link
Contributor

@weekface weekface commented Jun 11, 2019

What problem does this PR solve?

What is changed and how it works?

This pr passed the stability tests on cluster 1.

Check List

Tests

  • E2E test
  • Stability test

Code changes

  • Has Helm charts change
  • Has Go code change

Side effects

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation

Does this PR introduce a user-facing change?:

get the TS first and increase the TiKV GC life time to 3 hours before the full backup

@weekface weekface marked this pull request as ready for review June 11, 2019 11:00
{{ .Values.backupOptions }}

echo "Reset TiKV GC life time to 10m"
/usr/bin/mysql -h${host} -P4000 -u${TIDB_USER} -p${TIDB_PASSWORD} -Nse "update mysql.tidb set variable_value='10m' where variable_name='tikv_gc_life_time';"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this assumes 10 minutes was previously set, but I think we need to get the original value and set it back to that original value.

@weekface weekface requested a review from LinuxGit June 12, 2019 06:58
@weekface
Copy link
Contributor Author

/run-e2e-tests

1 similar comment
@weekface
Copy link
Contributor Author

/run-e2e-tests

- -pd-urls=http://{{ .Values.clusterName }}-pd:2379
- -cmd=generate_meta
- -data-dir=/savepoint-dir
command: ["/bin/sh", "-c", "cp /binlogctl /shared-dir/"]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The initContainer only copy binlogctl to the backup container

cofyc
cofyc previously approved these changes Jun 13, 2019
Copy link
Contributor

@cofyc cofyc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM on TikV failover check bugfix

Copy link
Contributor

@xiaojingchen xiaojingchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cofyc
Copy link
Contributor

cofyc commented Jun 13, 2019

/run-e2e-tests

@cofyc
Copy link
Contributor

cofyc commented Jun 14, 2019

failed to watch file "/var/log/pods/e804fae3-8df2-11e9-8cad-0242aae97541/tidb-operator-e2e/0.log": no space left on device++ kubectl get po tidb-operator-e2e -n tidb-operator-e2e '-ojsonpath={.status.phase}'

no enough space left?

@cofyc
Copy link
Contributor

cofyc commented Jun 14, 2019

/run-e2e-tests
try again

@weekface
Copy link
Contributor Author

/run-e2e-tests

xiaojingchen
xiaojingchen previously approved these changes Jun 14, 2019
@@ -1,26 +1,46 @@
set -euo pipefail

host=`echo {{ .Values.clusterName }}_TIDB_SERVICE_HOST | tr '[a-z]' '[A-Z]' | tr '-' '_'`
host_env=`echo {{ .Values.clusterName }}_TIDB_SERVICE_HOST | tr '[a-z]' '[A-Z]' | tr '-' '_'`
host=`eval echo '${'$host_env'}'`
Copy link
Member

@tennix tennix Jun 14, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

host=$(getent hosts {{ .Values.clusterName }}-tidb | head | awk '{print $1}')

@weekface weekface dismissed stale reviews from xiaojingchen and cofyc via 19edd9b June 14, 2019 12:09
@weekface weekface changed the title refine fullbackup [DNM] refine fullbackup Jun 14, 2019
@weekface
Copy link
Contributor Author

/run-e2e-tests

@weekface weekface changed the title [DNM] refine fullbackup refine fullbackup Jun 17, 2019
@weekface
Copy link
Contributor Author

/run-e2e-tests

@weekface
Copy link
Contributor Author

@tennix ptal

Copy link
Member

@tennix tennix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@weekface weekface merged commit ddcb27c into pingcap:master Jun 18, 2019
@weekface weekface deleted the refine-fullbackup branch June 18, 2019 07:01
yahonda pushed a commit that referenced this pull request Dec 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

scheduled backup mydumper failed to start: snapshot is older than GC safe point
5 participants