Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Source-mysql] : Implement WASS algo #38240

Merged
merged 40 commits into from
Jul 11, 2024
Merged
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
9c36109
Initial Commit
akashkulk May 15, 2024
5ed8d39
Merge branch 'master' into akash/wass-algo
akashkulk Jun 3, 2024
4816c58
Fix config error
akashkulk May 21, 2024
22d4674
Revert integration runner
akashkulk Jun 3, 2024
2eaadf3
commit
akashkulk Jun 4, 2024
45b4b57
fix
akashkulk Jun 4, 2024
4d38d55
duration fix
akashkulk Jun 5, 2024
a7ab494
timing
akashkulk Jun 6, 2024
e45ec75
Include partially completed streams
akashkulk Jun 10, 2024
83b9aab
Merge branch 'master' into akash/wass-algo
akashkulk Jun 11, 2024
777f7bc
Merge branch 'master' into akash/wass-algo
akashkulk Jun 11, 2024
6912e30
Merge branch 'master' into akash/wass-algo
akashkulk Jun 17, 2024
aab6ea2
revert
akashkulk Jun 17, 2024
5b83880
Fixes
akashkulk Jun 17, 2024
9bdd2a0
Merge branch 'master' into akash/wass-algo
akashkulk Jun 17, 2024
7fbbbd6
Filter out streams
akashkulk Jun 17, 2024
5231b60
Merge branch 'master' into akash/wass-algo
akashkulk Jun 29, 2024
b20700f
Add stream status emissions + config
akashkulk Jun 30, 2024
f8de8af
Fix unit tests
akashkulk Jul 2, 2024
78d0377
Merge branch 'master' into akash/wass-algo
akashkulk Jul 2, 2024
134d395
Bump versions
akashkulk Jul 2, 2024
aa045b1
Merge branch 'master' into akash/wass-algo
akashkulk Jul 2, 2024
4059b17
Add cdc config for initial load timeout
akashkulk Jul 2, 2024
e698fa8
FIx unit tests
akashkulk Jul 2, 2024
16b349b
Merge branch 'master' into akash/wass-algo
akashkulk Jul 2, 2024
6a33ce5
Update airbyte-integrations/connectors/source-mysql/src/main/java/io/…
akashkulk Jul 3, 2024
0a40758
Address comments
akashkulk Jul 3, 2024
894aa6c
Merge branch 'master' into akash/wass-algo
akashkulk Jul 3, 2024
f11dc87
remove comment
akashkulk Jul 3, 2024
6364f9e
Fix excessive logging
akashkulk Jul 3, 2024
c1aef67
Fix
akashkulk Jul 3, 2024
ccd7f6b
Merge branch 'master' into akash/wass-algo
akashkulk Jul 8, 2024
1fd46cc
Fixes
akashkulk Jul 9, 2024
16ce774
Merge branch 'master' into akash/wass-algo
akashkulk Jul 9, 2024
9781923
Address comments
akashkulk Jul 10, 2024
9bacf66
Merge branch 'master' into akash/wass-algo
akashkulk Jul 10, 2024
44093a0
Update changelog
akashkulk Jul 11, 2024
67ac347
Merge branch 'master' into akash/wass-algo
akashkulk Jul 11, 2024
37a9c41
Update readme
akashkulk Jul 11, 2024
bf3aecc
Merge branch 'master' into akash/wass-algo
akashkulk Jul 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Filter out streams
akashkulk committed Jun 17, 2024
commit 7fbbbd62425c7850609712ad77500a07c60faab0
Original file line number Diff line number Diff line change
@@ -245,6 +245,7 @@ public static List<AutoCloseableIterator<AirbyteMessage>> getCdcReadIterators(fi
false);
final var cdcStreamList = catalog.getStreams().stream()
.filter(stream -> stream.getSyncMode() == SyncMode.INCREMENTAL)
.filter(stream -> isStreamPartiallyOrFullyCompleted(stream, initialLoadStreams))
.map(stream -> stream.getStream().getNamespace() + "." + stream.getStream().getName()).toList();
final var propertiesManager = new RelationalDbDebeziumPropertiesManager(
MySqlCdcProperties.getDebeziumProperties(database), sourceConfig, catalog, cdcStreamList);