Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CDK: Emit control message on config mutation #19428

Merged
merged 30 commits into from
Nov 29, 2022
Merged
Changes from 1 commit
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
f3d4b5a
wip
alafanechere Nov 14, 2022
24d24dd
implementation
alafanechere Nov 15, 2022
355b90b
format
alafanechere Nov 15, 2022
61e9d4b
bump version
alafanechere Nov 15, 2022
1101ccc
Merge branch 'master' into augustin/cdk/emit-updated-configs
alafanechere Nov 15, 2022
22f75c7
always update by default, even if same value
alafanechere Nov 16, 2022
d562117
rename split_config to filter_internal_keywords
alafanechere Nov 16, 2022
733b89a
bing ads example
alafanechere Nov 16, 2022
a59a1e6
wrap around AirbyteMessage
alafanechere Nov 16, 2022
616256e
exclude unset
alafanechere Nov 16, 2022
3ef545d
observer does not write config to disk
alafanechere Nov 21, 2022
0ff7318
revert global changes
alafanechere Nov 21, 2022
9e1483c
revert global changes
alafanechere Nov 21, 2022
2bbff33
revert global changes
alafanechere Nov 21, 2022
1f02860
observe from Oauth2Authenticator
alafanechere Nov 21, 2022
f40d99c
Merge branch 'master' into augustin/cdk/emit-updated-configs
alafanechere Nov 21, 2022
04041d8
ref
alafanechere Nov 21, 2022
12db1b7
handle list of dicts
alafanechere Nov 21, 2022
63d129c
implement SingleUseRefreshTokenOauth2Authenticator
alafanechere Nov 23, 2022
62ef553
test SingleUseRefreshTokenOauth2Authenticator
alafanechere Nov 23, 2022
bbe802a
call copy in ObservedDict
alafanechere Nov 23, 2022
8fd52f9
add docstring
alafanechere Nov 23, 2022
c5a0b6d
source harvest example
alafanechere Nov 23, 2022
803b70a
use dpath
alafanechere Nov 25, 2022
37f474f
better doc string
alafanechere Nov 25, 2022
08f64e4
update changelog
alafanechere Nov 25, 2022
518a36f
use sequence instead of string path for dpath declaration
alafanechere Nov 29, 2022
64bbc48
Merge branch 'master' into augustin/cdk/emit-updated-configs
alafanechere Nov 29, 2022
4da6581
revert connector changes
alafanechere Nov 29, 2022
380e175
format
alafanechere Nov 29, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
wrap around AirbyteMessage
  • Loading branch information
alafanechere committed Nov 16, 2022
commit a59a1e6ed5efa7204b2a1e8476782462ba39a8d8
5 changes: 3 additions & 2 deletions airbyte-cdk/python/airbyte_cdk/config_observation.py
Original file line number Diff line number Diff line change
@@ -6,7 +6,7 @@
from abc import ABC, abstractmethod
from typing import Any, Callable, MutableMapping

from airbyte_cdk.models import AirbyteControlConnectorConfigMessage, AirbyteControlMessage, OrchestratorType
from airbyte_cdk.models import AirbyteControlConnectorConfigMessage, AirbyteControlMessage, AirbyteMessage, OrchestratorType, Type


class BaseObserver(ABC):
alafanechere marked this conversation as resolved.
Show resolved Hide resolved
@@ -59,4 +59,5 @@ def _emit_airbyte_control_message(self) -> None:
emitted_at=time.time() * 1000,
connectorConfig=AirbyteControlConnectorConfigMessage(config=self.config),
)
print(control_message.json())
airbyte_message = AirbyteMessage(type=Type.CONTROL, control=control_message)
print(airbyte_message.json())
5 changes: 4 additions & 1 deletion airbyte-cdk/python/unit_tests/test_config_observation.py
Original file line number Diff line number Diff line change
@@ -37,7 +37,10 @@ def test_update(self, mocker, capsys):
config_observer.update()
after_time = time.time() * 1000
captured = capsys.readouterr()
raw_control_message = json.loads(captured.out)
airbyte_message = json.loads(captured.out)
assert airbyte_message["type"] == "CONTROL"
assert "control" in airbyte_message
raw_control_message = airbyte_message["control"]
mock_write_config_fn.assert_called_with(config_observer.config, mock_config_path)
assert raw_control_message["type"] == "CONNECTOR_CONFIG"
assert raw_control_message["connectorConfig"] == {"config": dict(config_observer.config)}