Flink Connector TDengine

English | 简体中文

Table of Contents

• 1. Introduction
• 2. Documentation
• 3. Prerequisites
• 4. Build
• 5. Testing
  • 5.1 Test Execution
  • 5.2 Test Case Addition
  • 5.3 Performance Testing
• 6. CI/CD
• 7. Submitting Issues
• 8. Submitting PRs
• 9. References
• 10. License

1. Introduction

flink-connector-tdengine is a connector developed by TDengine's official team for Flink. Developers can use it to develop application software for accessing the TDengine database. flink-connector-tdengine supports features related to Source, CDC, Sink, and Table SQL.

2. Documentation

• To use TDengine Flink Connector, please refer to the Developer Guide, which includes instructions on how to integrate flink-connector-tdengine into an application, along with examples of Source, CDC, Sink, and Table SQL.
• This quick guide is primarily for developers who wish to contribute, build, and test the Flink connector on their own. To learn about TDengine, you can visit the official documentation.

3. Prerequisites

• Java 1.8 or above runtime environment and Maven 3.6 or above installed, with environment variables correctly set.
• TDengine has been deployed locally. For specific steps, please refer to Deploy TDengine. Please make sure taosd and taosAdapter have been started.
• Apache Flink version 1.19.0 or above has been installed. For the specific installation method, please refer to the Apache Flink.

4. Build

Execute mvn clean package in the project directory to build the project.

5. Testing

5.1 Test Execution

Execute mvn test in the project directory to run the tests. The test cases will connect to the local TDengine server and taosAdapter for testing. After running the tests, the result similar to the following will be printed eventually. If all test cases pass, both Failures and Errors will be 0.

[INFO] Results:
[INFO] 
[INFO] Tests run: 31, Failures: 0, Errors: 0, Skipped: 0

5.2 Test Case Addition

All tests are located in the src/test/java/com/taosdata/flink directory of the project. The directory is divided according to the functions being tested. You can add new test files or add test cases in existing test files. The test cases use the JUnit framework. Generally, a connection is established and a database is created in the beforeEach method, and the database is droped and the connection is released in the afterEach method.

5.3 Performance Testing

Performance testing is in progress.

6. CI/CD

• Build Workflow
• Code Coverage

7. Submitting Issues

We welcome the submission of GitHub Issue. When submitting, please provide the following information:

• Problem description, whether it always occurs, and it's best to include a detailed call stack.
• Flink version.
• Flink Connector version.
• Flink Connector connection parameters (username and password not required).
• TDengine server version.

8. Submitting PRs

We welcome developers to contribute to this project. When submitting PRs, please follow these steps:

1. Fork this project, refer to (how to fork a repo).
2. Create a new branch from the main branch with a meaningful branch name (git checkout -b my_branch). Do not modify the main branch directly.
3. Modify the code, ensure all unit tests pass, and add new unit tests to verify the changes.
4. Push the changes to the remote branch (git push origin my_branch).
5. Create a Pull Request on GitHub (how to create a pull request).
6. After submitting the PR, you can find your PR through the Pull Request. Click on the corresponding link to see if the CI for your PR has passed. If it has passed, it will display "All checks have passed". Regardless of whether the CI passes or not, you can click "Show all checks" -> "Details" to view the detailed test case logs.
7. After submitting the PR, if CI passes, you can find your PR on the codecov page to check the test coverage.

9. References

• TDengine Official Website
• TDengine GitHub

10. License

MIT License