Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pipe: Ensure at-least-once semantic by forcibly starting PipeHistoricalDataRegionTsFileExtractor after pipe restart #12010

Merged
merged 7 commits into from
Feb 1, 2024
Merged

Pipe: Ensure at-least-once semantic by forcibly starting PipeHistoricalDataRegionTsFileExtractor after pipe restart #12010

merged 7 commits into from
Feb 1, 2024

Conversation

Caideyipi
Copy link
Collaborator

Description

As the title said.

This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.
Key changed/added classes (or packages if there are too many classes) in this PR

@Caideyipi Caideyipi changed the title Pipe: Enable restart pipe correctness by deactivating real-time mode when pipe is restarted Pipe: Assure restart pipe correctness by deactivating real-time mode when pipe is restarted Jan 30, 2024
@Caideyipi Caideyipi changed the title Pipe: Assure restart pipe correctness by deactivating real-time mode when pipe is restarted Pipe: Enable restart pipe correctness by deactivating real-time mode when pipe is restarted Jan 30, 2024
@SteveYurongSu SteveYurongSu self-requested a review January 31, 2024 01:48
@SteveYurongSu SteveYurongSu self-assigned this Jan 31, 2024
@SteveYurongSu SteveYurongSu changed the title Pipe: Enable restart pipe correctness by deactivating real-time mode when pipe is restarted Pipe: Ensure at-least-once semantic by forcibly starting PipeHistoricalDataRegionTsFileExtractor after pipe restart Feb 1, 2024
@SteveYurongSu SteveYurongSu merged commit c8419ec into apache:master Feb 1, 2024
36 checks passed
Caideyipi added a commit to Caideyipi/iotdb that referenced this pull request Feb 5, 2024
@Caideyipi @SteveYurongSu
Pipe: Ensure at-least-once semantic by forcibly starting PipeHistoric…
831ea92 
…alDataRegionTsFileExtractor after pipe restart (apache#12010)

Co-authored-by: Steve Yurong Su <rong@apache.org>
SteveYurongSu added a commit that referenced this pull request Feb 5, 2024
@Caideyipi @SteveYurongSu
Pipe Meta: Support deletion and air gap sink sync in meta sync (#12027)
21ec650 
* Update IoTDBPipeNullValueIT.java

* Support air gap

* code style

* Update IoTDBPipeProtocolIT.java

* Support schema deletion

* Update TsFileProcessor.java

* main logic

* Update IoTDBPipeInclusionIT.java

* partial commit

* partial commit

* partial commit

* partial commit

* partial commit

* main logic

* remain logic

* fix null

* fix handshake

* Pipe: Ensure at-least-once semantic by forcibly starting PipeHistoricalDataRegionTsFileExtractor after pipe restart (#12010)

Co-authored-by: Steve Yurong Su <rong@apache.org>

* Pipe: Allow pipe times configured by raw timestamp (#12004)

* Pipe: fixed the bug that region follower tsfiles can not record the IoTProgressIndex from region leader (#11980)

* bug fix

---------

Co-authored-by: Steve Yurong Su <rong@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SteveYurongSu SteveYurongSu SteveYurongSu approved these changes

Assignees

@SteveYurongSu SteveYurongSu

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Caideyipi @SteveYurongSu