Skip to content

Latest commit

 

History

History
68 lines (44 loc) · 4.28 KB

README.md

File metadata and controls

68 lines (44 loc) · 4.28 KB
Raw

Preparations

  1. The following executables must be copied or generated or linked into these locations, mydumper and sync_diff_inspector can be downloaded from tidb-enterprise-tools-latest-linux-amd64:

    • bin/tidb-server
    • bin/sync_diff_inspector
    • bin/mydumper
    • bin/dm-master.test # generated by make dm_integration_test_build
    • bin/dm-worker.test # generated by make dm_integration_test_build
    • gh-ost # must be added to path, or you can export GHOST_BINARY=/path/to/gh-ost-binary
    • pt-online-schema-change # must be added to path, or you can export PTOSC_BINARY=/path/to/pt-osc-binary

  2. The following programs must be installed:

    • mysql (the CLI client)
    • python2.7 or python3.x

  3. The user executing the tests must have permission to create the folder /tmp/dm_test. All test artifacts will be written into this folder.

Running

Unit Test

  1. Setup a MySQL server with binlog enabled first, export proper environment variable MYSQL_HOST, MYSQL_PORT, MYSQL_USER, MYSQL_PSWD, default is 127.0.0.1, 3306, root and empty password.

  2. Run make test to run unit test

Integration Test

  1. Run make dm_integration_test_build to generate DM related binary for integration test

  2. Run make integration_test to execute the integration tests. This command will

    1. Check that all required executables exist.
    2. Execute tests/run.sh

    If want to run one integration test case only, just pass the CASE parameter, such as make integration_test CASE=sharding.

    There exists some environment variables that you can set by yourself, including MYSQL_HOST1, MYSQL_PORT1, MYSQL_HOST2, MYSQL_PORT2, RESET_MASTER. If RESET_MASTER is not set or set to true, RESET MASTER will be executed at upstream MySQL before each case.

    The online DDL test using pt-osc doesn't work if the upstream MySQL has different connect port and bind port (often caused by port forwarding via NAT). In this case, you must specify the real IP and port of MySQL. Otherwise you can skip online DDL test by export ONLINE_DDL_ENABLE=false.

  3. After executing the tests, run make coverage to get a coverage report at /tmp/dm_test/all_cov.html.

Writing new tests

New integration tests can be written as shell scripts in tests/TEST_NAME/run.sh. The script should exit with a nonzero error code on failure.

Note the integration test runs in parallel, and new added test case is not listed in CI, so remember to add the TEST_NAME of new added test case to file others_integration.txt in a newline.

Several convenient commands are provided:

  • run_dm_master <WORKDIR> <PORT> <CONFIG> — Starts dm-master using config provided, on given port, running in workdir.
  • run_dm_worker <WORKDIR> <PORT> <CONFIG> — Starts dm-worker using config provided, on given port, running in workdir.
  • run_dm_ctl <WORKDIR> <MASTER_ADDR> <DMCTL_COMMAND> - Runs dmctl with given command in non-interactive mode.
  • run_sql <SQL> <PORT> — Executes an SQL query in database based on port provided
  • run_sql_file <path_to_SQL_file> <HOST> <PORT> — Executes all SQLs in given file to the database on port provided
  • run_sql_file_online_ddl <path_to_SQL_file> <HOST> <PORT> <DB> <ONLINE DDL TOOL> — Executes all SQLs in given file, will auto switch DDL to online DDL command.
  • check_contains <TEXT> — Checks if the previous run_sql/run_sql_file result contains the given text (in -E format)
  • check_sync_diff <WORKDIR> <CONFIG> - Runs sync_diff_inspector to check diff between databases, using config file provided
  • check_rpc_alive <CHECK_TOOL> <ADDR> - Wrapper to check a rpc address is available, at most 10 times.
  • check_port_alive <PORT> - Wrapper to check a port is alive, at most 20 times.
  • check_port <HOST> <PORT> - Checks a host:port is alive.
  • wait_process_exit <process_name> - Wait for one or more processes to exit by given process name.
  • check_metric <PORT> <METRIC_NAME> <RETRY_COUNT> <VALUE PATTERN LIST>... - check metric value from prometheus.
  • truncate_trace_events <PORT> - truncate trace server events records.