Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dm: fix binlog 999999 test #11726

Merged
merged 26 commits into from
Nov 26, 2024
Merged
Changes from 25 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 33 additions & 11 deletions .github/workflows/dm_binlog_999999.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,11 @@ jobs:
test-binlog-999999:
name: Test binlog 999999
runs-on: ubuntu-20.04
services:
docker:
image: docker:19.03.12
options: >-
--privileged

steps:
- name: Set up Go env
Expand All @@ -30,6 +35,14 @@ jobs:
with:
ref: refs/pull/${{ github.event.inputs.pr }}/head

- name: Set DOCKER_HOST
run: echo "export DOCKER_HOST=unix:///var/run/docker.sock" >> $GITHUB_ENV

- name: Install docker-compose
run: |
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

- name: Cache go modules
uses: actions/cache@v2
with:
Expand All @@ -48,7 +61,9 @@ jobs:

- name: Setup CI environment
run: |
docker-compose -f ./dm/tests/binlog_999999/docker-compose.yml up -d
sudo apt-get update
sudo apt-get install -y curl
sudo docker-compose -f ./dm/tests/binlog_999999/docker-compose.yml up -d
curl http://download.pingcap.org/tidb-enterprise-tools-nightly-linux-amd64.tar.gz | tar xz
mv tidb-enterprise-tools-nightly-linux-amd64/bin/sync_diff_inspector bin/
curl http://download.pingcap.org/tidb-nightly-linux-amd64.tar.gz | tar xz
Expand All @@ -57,7 +72,7 @@ jobs:
chmod +x minio
mv minio bin/

- name: change binlog sequence number to 999998
- name: Change binlog sequence number to 999999
run: |
while ! mysqladmin -h127.0.0.1 -P3306 -p123456 -uroot ping --connect-timeout=1 > /dev/null 2>&1 ; do
echo "wait mysql"
Expand Down Expand Up @@ -87,11 +102,16 @@ jobs:
sleep 1
done

echo "show binary logs;" | mysql -uroot -h127.0.0.1 -P3306 -p123456
echo "show binary logs;" | mysql -uroot -h127.0.0.1 -P3307 -p123456
Comment on lines +105 to +106
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait mysql
Warning: arning] Using a password on the command line interface can be insecure.
Log_name	File_size
mysql-bin.999998	177
mysql-bin.999999	1[54](https://github.com/pingcap/tiflow/actions/runs/12004028356/job/33458364347#step:12:55)
Warning: arning] Using a password on the command line interface can be insecure.
Log_name	File_size	Encrypted
mysql-bin.999998	179	No
mysql-bin.999999	1[56](https://github.com/pingcap/tiflow/actions/runs/12004028356/job/33458364347#step:12:57)	No


- name: Run test cases
run: |
RESET_MASTER=false make dm_integration_test CASE=incremental_mode
echo "show binary logs;" | mysql -uroot -h127.0.0.1 -P3306 -p123456 | grep -q "mysql-bin.1000000"
echo "show binary logs;" | mysql -uroot -h127.0.0.1 -P3307 -p123456 | grep -q "mysql-bin.1000000"
echo "show binary logs;" | mysql -uroot -h127.0.0.1 -P3306 -p123456
echo "show binary logs;" | mysql -uroot -h127.0.0.1 -P3307 -p123456
echo "show binary logs;" | mysql -uroot -h127.0.0.1 -P3306 -p123456 | grep -q "mysql-bin.1000003"
echo "show binary logs;" | mysql -uroot -h127.0.0.1 -P3307 -p123456 | grep -q "mysql-bin.1000002"
Comment on lines +113 to +114
Copy link
Contributor Author

@River2000i River2000i Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mysql 3306 trigger 4 times flush logs in case and restart 1 time. Mysql 3307 trigger 3 times flush logs in case and restart 1 time. The origin binlog file is mysql-bin.999998.


- name: Copy logs to hack permission
if: ${{ always() }}
Expand All @@ -112,12 +132,14 @@ jobs:
path: |
./logs

# send Slack notify if failed.
# NOTE: With the exception of `GITHUB_TOKEN`, secrets are not passed to the runner when a workflow is triggered from a forked repository.
- name: Slack notification
# Send feishu notification if failed.
- name: Feishu notification
continue-on-error: true
uses: foxundermoon/feishu-action@v2
if: ${{ failure() }}
env:
SLACK_WEBHOOK: ${{ secrets.SLACK_NOTIFY }}
uses: Ilshidur/action-slack@2.1.0
with:
args: "binlog 999999 failed, see https://github.com/pingcap/tiflow/actions/runs/{{ GITHUB_RUN_ID }}"
url: ${{ secrets.ENGINE_FEISHU_NOTIFY_URL }}
msg_type: text
content: |
text: |
dataflow engine chaos job failed, see https://github.com/pingcap/tiflow/actions/runs/${{ github.run_id }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
dataflow engine chaos job failed, see https://github.com/pingcap/tiflow/actions/runs/${{ github.run_id }}
DM chaos job failed, see https://github.com/pingcap/tiflow/actions/runs/${{ github.run_id }}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Loading