🐛 🚨🚨 Source LinkedIn Pages: fixed failing connection checks #18967

tinomerl · 2022-11-04T14:26:53Z

What

How

The problem was that the org_id was defined as a secret and a integer. By setting it to string in the spec.json The error mentioned in #15986 got fixed. But by changing the type of the org_id the acceptance tests for the backwards compatibility fail. Therefore i tested the connector with a local deployment. Everything could be replicated and worked fine.

Fixed an additional bug where two streams where defined two times in the streams method. This led to an error making it impossible to repliace data and broke the airbyte ui when checking the connection.

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub by running the /publish command described here
After the connector is published, connector added to connector index as described here
Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub and connector version bumped by running the /publish command described here

Connector Generator

Issue acceptance criteria met
PR name follows PR naming conventions
If adding a new generator, add it to the list of scaffold modules being tested
The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
Documentation which references the generator is updated as needed

Tests

Unit

None were defined

Integration

Put your integration tests output here.

Acceptance

Test session starts (platform: linux, Python 3.9.15, pytest 6.2.5, pytest-sugar 0.9.5) cachedir: .pytest_cache hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/mnt/c/Users/Tino.Merl/Documents/repos/airbyte_fork/airbyte-integrations/connectors/source-linkedin-pages/.hypothesis/examples') rootdir: /mnt/c/Users/Tino.Merl/Documents/repos/airbyte_fork, configfile: pytest.ini plugins: hypothesis-6.54.6, cov-3.0.0, mock-3.6.1, sugar-0.9.5, timeout-1.4.2, requests-mock-1.9.3 collecting ... {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}} {"type": "LOG", "log": {"level": "WARN", "message": "The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format."}}

airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestSpec.test_config_match_spec[inputs0] ✓ 4% ▍
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestSpec.test_match_expected[inputs0] ✓ 8% ▊
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestSpec.test_docker_env[inputs0] ✓ 12% █▎
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestSpec.test_oneof_usage[inputs0] ✓ 15% █▋
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestSpec.test_required[inputs0] ✓ 19% █▉
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestSpec.test_optional[inputs0] ✓ 23% ██▍
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestSpec.test_has_secret[inputs0] ✓ 27% ██▊
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestSpec.test_secret_never_in_the_output[inputs0] ✓ 31% ███▏
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestSpec.test_defined_refs_exist_in_json_spec_file[inputs0] ✓ 35% ███▌
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestSpec.test_oauth_flow_parameters[inputs0] ✓ 38% ███▉

――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestSpec.test_backward_compatibility[inputs0] ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <source_acceptance_test.tests.test_core.TestSpec object at 0x7fd502c65d90>, skip_backward_compatibility_tests = False
actual_connector_spec = ConnectorSpecification(documentationUrl=AnyUrl('https://docs.airbyte.com/integrations/sources/linkedin-pages/', scheme...t_id'], ['client_secret']], oauthFlowOutputParameters=[['refresh_token']])), advanced_auth=None, protocol_version=None)
previous_connector_spec = ConnectorSpecification(documentationUrl=AnyUrl('https://docs.airbyte.com/integrations/sources/linkedin-pages/', scheme...t_id'], ['client_secret']], oauthFlowOutputParameters=[['refresh_token']])), advanced_auth=None, protocol_version=None)
number_of_configs_to_generate = 100

@pytest.mark.default_timeout(60)
@pytest.mark.backward_compatibility
def test_backward_compatibility(
    self,
    skip_backward_compatibility_tests: bool,
    actual_connector_spec: ConnectorSpecification,
    previous_connector_spec: ConnectorSpecification,
    number_of_configs_to_generate: int = 100,
):
    """Check if the current spec is backward_compatible with the previous one"""
    assert isinstance(actual_connector_spec, ConnectorSpecification) and isinstance(previous_connector_spec, ConnectorSpecification)
    checker = SpecDiffChecker(previous=previous_connector_spec.dict(), current=actual_connector_spec.dict())

  checker.assert_is_backward_compatible()

../../bases/source-acceptance-test/source_acceptance_test/tests/test_core.py:181:

../../bases/source-acceptance-test/source_acceptance_test/utils/backward_compatibility.py:131: in assert_is_backward_compatible
self.check_if_value_of_type_field_changed(self.connection_specification_diff)
../../bases/source-acceptance-test/source_acceptance_test/utils/backward_compatibility.py:60: in check_if_value_of_type_field_changed
self._raise_error("The'type' field value was changed.", diff)

self = <source_acceptance_test.utils.backward_compatibility.SpecDiffChecker object at 0x7fd502be9a60>, message = "The'type' field value was changed."
diff = {'values_changed': [<root['properties']['org_id']['type'] t1:'integer', t2:'string'>, <root['properties']['credentials']['title'] t1:'Authenticat...', t2:'Authenticat...'>]}

def _raise_error(self, message: str, diff: DeepDiff):

  raise NonBackwardCompatibleError(f"{message}. Diff: {diff.pretty()}", self.context)

E source_acceptance_test.utils.backward_compatibility.NonBackwardCompatibleError: BackwardIncompatibilityContext.SPEC - The'type' field value was changed.. Diff: Value of root['properties']['org_id']['type'] changed from "integer" to "string".
E Value of root['properties']['credentials']['title'] changed from "Authentication *" to "Authentication".

../../bases/source-acceptance-test/source_acceptance_test/utils/backward_compatibility.py:38: NonBackwardCompatibleError

airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestSpec.test_backward_compatibility[inputs0] ⨯ 42% ████▎
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestSpec.test_additional_properties_is_true[inputs0] ✓ 46% ████▋
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestConnection.test_check[inputs0] ✓ 50% █████
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestConnection.test_check[inputs1] ✓ 54% █████▍
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestDiscovery.test_discover[inputs0] ✓ 58% █████▊
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestDiscovery.test_defined_cursors_exist_in_schema[inputs0] ✓ 62% ██████▎
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestDiscovery.test_defined_refs_exist_in_schema[inputs0] ✓ 65% ██████▋
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestDiscovery.test_defined_keyword_exist_in_schema[inputs0-allOf] ✓ 69% ██████▉
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestDiscovery.test_defined_keyword_exist_in_schema[inputs0-not] ✓ 73% ███████▍
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestDiscovery.test_primary_keys_exist_in_schema[inputs0] ✓ 77% ███████▊
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestDiscovery.test_streams_has_sync_modes[inputs0] ✓ 81% ████████▏
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestDiscovery.test_additional_properties_is_true[inputs0] ✓ 85% ████████▌ {"type": "LOG", "log": {"level": "ERROR", "message": "Docker container failed, code 1, error:\n{"type": "TRACE", "trace": {"type": "ERROR", "emitted_at": 1667571281562.35, "error": {"message": "Something went wrong in the connector. See the logs for more details.", "internal_message": "Config validation error: '' is not of type 'integer'", "stack_trace": "Traceback (most recent call last):\n File \"/airbyte/integration_code/main.py\", line 13, in \n launch(source, sys.argv[1:])\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 123, in launch\n for message in source_entrypoint.run(parsed_args):\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 96, in run\n check_config_against_spec_or_exit(connector_config, source_spec)\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/utils/schema_helpers.py\", line 160, in check_config_against_spec_or_exit\n raise Exception(\"Config validation error: \" + validation_error.message) from None\nException: Config validation error: '' is not of type 'integer'\n", "failure_type": "system_error"}}}\n"}}

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ERROR at setup of TestDiscovery.test_backward_compatibility[inputs0] ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

connector_config = SecretDict(******), previous_connector_docker_runner = <source_acceptance_test.utils.connector_runner.ConnectorRunner object at 0x7fd502a9faf0>, previous_cached_schemas = {}

@pytest.fixture(name="previous_discovered_catalog")
def previous_discovered_catalog_fixture(
    connector_config, previous_connector_docker_runner: ConnectorRunner, previous_cached_schemas
) -> MutableMapping[str, AirbyteStream]:
    """JSON schemas for each stream"""
    if not previous_cached_schemas:

      output = previous_connector_docker_runner.call_discover(config=connector_config)

../../bases/source-acceptance-test/source_acceptance_test/conftest.py:259:

../../bases/source-acceptance-test/source_acceptance_test/utils/connector_runner.py:82: in call_discover
output = list(self.run(cmd=cmd, config=config, **kwargs))
../../bases/source-acceptance-test/source_acceptance_test/utils/connector_runner.py:109: in run
for line in self.read(container, command=cmd, with_ext=raise_container_error):

cls = <class 'source_acceptance_test.utils.connector_runner.ConnectorRunner'>, container = <Container: fa83990fdf>, command = 'discover --config /data/tap_config.json', with_ext = True

@classmethod
def read(cls, container: Container, command: str = None, with_ext: bool = True) -> Iterable[str]:
    """Reads connector's logs per line"""
    buffer = b""
    exception = ""
    line = ""
    for chunk in container.logs(stdout=True, stderr=True, stream=True, follow=True):

        buffer += chunk
        while True:
            # every chunk can include several lines
            found = buffer.find(b"\n")
            if found <= -1:
                break

            line = buffer[: found + 1].decode("utf-8")
            if len(exception) > 0 or line.startswith("Traceback (most recent call last)"):
                exception += line
            else:
                yield line
            buffer = buffer[found + 1 :]

    if buffer:
        # send the latest chunk if exists
        line = buffer.decode("utf-8")
        if exception:
            exception += line
        else:
            yield line
    try:
        exit_status = container.wait()
        container.remove()
    except NotFound as err:
        logging.error(f"Waiting error: {err}, logs: {exception or line}")
        raise
    if exit_status["StatusCode"]:
        error = exit_status["Error"] or exception or line
        logging.error(f"Docker container failed, " f'code {exit_status["StatusCode"]}, error:\n{error}')
        if with_ext:

          raise ContainerError(

                container=container,
                exit_status=exit_status["StatusCode"],
                command=command,
                image=container.image,
                stderr=error,
            )

E docker.errors.ContainerError: Command 'discover --config /data/tap_config.json' in image '<Image: 'airbyte/source-linkedin-pages:latest'>' returned non-zero exit status 1: {"type": "TRACE", "trace": {"type": "ERROR", "emitted_at": 1667571281562.35, "error": {"message": "Something went wrong in the connector. See the logs for more details.", "internal_message": "Config validation error: '' is not of type 'integer'", "stack_trace": "Traceback (most recent call last):\n File "/airbyte/integration_code/main.py", line 13, in \n launch(source, sys.argv[1:])\n File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 123, in launch\n for message in source_entrypoint.run(parsed_args):\n File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 96, in run\n check_config_against_spec_or_exit(connector_config, source_spec)\n File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/utils/schema_helpers.py", line 160, in check_config_against_spec_or_exit\n raise Exception("Config validation error: " + validation_error.message) from None\nException: Config validation error: '' is not of type 'integer'\n", "failure_type": "system_error"}}}

../../bases/source-acceptance-test/source_acceptance_test/utils/connector_runner.py:155: ContainerError
------------------------------------------------------------------------------------------- Captured log setup -------------------------------------------------------------------------------------------
ERROR root:connector_runner.py:153 Docker container failed, code 1, error:
{"type": "TRACE", "trace": {"type": "ERROR", "emitted_at": 1667571281562.35, "error": {"message": "Something went wrong in the connector. See the logs for more details.", "internal_message": "Config validation error: '' is not of type 'integer'", "stack_trace": "Traceback (most recent call last):\n File "/airbyte/integration_code/main.py", line 13, in \n launch(source, sys.argv[1:])\n File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 123, in launch\n for message in source_entrypoint.run(parsed_args):\n File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 96, in run\n check_config_against_spec_or_exit(connector_config, source_spec)\n File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/utils/schema_helpers.py", line 160, in check_config_against_spec_or_exit\n raise Exception("Config validation error: " + validation_error.message) from None\nException: Config validation error: '' is not of type 'integer'\n", "failure_type": "system_error"}}}
88% ████████▉
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestBasicRead.test_read[inputs0] ✓ 92% █████████▎{"type": "LOG", "log": {"level": "ERROR", "message": "Docker container failed, code 1, error:\n{"type": "TRACE", "trace": {"type": "ERROR", "emitted_at": 1667571293712.079, "error": {"message": "Something went wrong in the connector. See the logs for more details.", "internal_message": "2 validation errors for ConfiguredAirbyteCatalog\nstreams -> 0 -> sync_mode\n value is not a valid enumeration member; permitted: 'full_refresh', 'incremental' (type=type_error.enum; enum_values=[<SyncMode.full_refresh: 'full_refresh'>, <SyncMode.incremental: 'incremental'>])\nstreams -> 0 -> destination_sync_mode\n value is not a valid enumeration member; permitted: 'append', 'overwrite', 'append_dedup' (type=type_error.enum; enum_values=[<DestinationSyncMode.append: 'append'>, <DestinationSyncMode.overwrite: 'overwrite'>, <DestinationSyncMode.append_dedup: 'append_dedup'>])", "stack_trace": "Traceback (most recent call last):\n File \"/airbyte/integration_code/main.py\", line 13, in \n launch(source, sys.argv[1:])\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 131, in launch\n for message in source_entrypoint.run(parsed_args):\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 119, in run\n config_catalog = self.source.read_catalog(parsed_args.catalog)\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/source.py\", line 90, in read_catalog\n return ConfiguredAirbyteCatalog.parse_obj(self.read_config(catalog_path))\n File \"pydantic/main.py\", line 521, in pydantic.main.BaseModel.parse_obj\n File \"pydantic/main.py\", line 341, in pydantic.main.BaseModel.init\npydantic.error_wrappers.ValidationError: 2 validation errors for ConfiguredAirbyteCatalog\nstreams -> 0 -> sync_mode\n value is not a valid enumeration member; permitted: 'full_refresh', 'incremental' (type=type_error.enum; enum_values=[<SyncMode.full_refresh: 'full_refresh'>, <SyncMode.incremental: 'incremental'>])\nstreams -> 0 -> destination_sync_mode\n value is not a valid enumeration member; permitted: 'append', 'overwrite', 'append_dedup' (type=type_error.enum; enum_values=[<DestinationSyncMode.append: 'append'>, <DestinationSyncMode.overwrite: 'overwrite'>, <DestinationSyncMode.append_dedup: 'append_dedup'>])\n", "failure_type": "system_error"}}}\n"}}

airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestBasicRead.test_airbyte_trace_message_on_failure[inputs0] ✓ 96% █████████▋
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_full_refresh.py::TestFullRefresh.test_sequential_reads[inputs0] ✓ 100% ██████████
{"type": "LOG", "log": {"level": "INFO", "message": "/mnt/c/Users/Tino.Merl/Documents/repos/airbyte_fork/airbyte-integrations/connectors/source-linkedin-pages - SAT run - 7dad80e - FAILED"}}

============================================================================================ warnings summary ============================================================================================
../../bases/source-acceptance-test/source_acceptance_test/config.py:246: 26 warnings
airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py: 1 warning
/mnt/c/Users/Tino.Merl/Documents/repos/airbyte_fork/airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/config.py:246: DeprecationWarning: The 'warn' function is deprecated, use 'warning' instead
logging.warn("The acceptance-test-config.yml file is in a legacy format. Please migrate to the latest format.")

-- Docs: https://docs.pytest.org/en/stable/warnings.html
======================================================================================== short test summary info =========================================================================================
SKIPPED [1] ../../bases/source-acceptance-test/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
FAILED ../../bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestSpec::test_backward_compatibility[inputs0] - source_acceptance_test.utils.backward_compatibility.NonBackwardCo...
FAILED ../../bases/source-acceptance-test/source_acceptance_test/tests/test_core.py::TestDiscovery::test_backward_compatibility[inputs0] - docker.errors.ContainerError: Command 'discover --config /da...

Results (119.29s):
24 passed
1 failed
- airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py:169 TestSpec.test_backward_compatibility[inputs0]
1 error

…e streams

CLAassistant · 2022-11-04T14:27:08Z

All committers have signed the CLA.

…ests

vincentkoc · 2022-11-05T03:16:21Z

@tinomerl thanks for your contribution, I’m one of the Airbyte maintainers and will take a quick look at your submission.

unit test issues, will recheck

…ests

vincentkoc · 2022-11-06T23:30:31Z

/test connector=connectors/source-linkedin-pages

🕑 connectors/source-linkedin-pages https://github.com/airbytehq/airbyte/actions/runs/3406469631
❌ connectors/source-linkedin-pages https://github.com/airbytehq/airbyte/actions/runs/3406469631
🐛 https://gradle.com/s/twfl6l4ixheoe

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestSpec::test_config_match_spec[inputs0] - Failed: Conf...
FAILED test_core.py::TestConnection::test_check[inputs0] - AssertionError: as...
FAILED test_core.py::TestDiscovery::test_discover[inputs0] - docker.errors.Co...
ERROR test_core.py::TestDiscovery::test_defined_cursors_exist_in_schema[inputs0]
ERROR test_core.py::TestDiscovery::test_defined_refs_exist_in_schema[inputs0]
ERROR test_core.py::TestDiscovery::test_defined_keyword_exist_in_schema[inputs0-allOf]
ERROR test_core.py::TestDiscovery::test_defined_keyword_exist_in_schema[inputs0-not]
ERROR test_core.py::TestDiscovery::test_primary_keys_exist_in_schema[inputs0]
ERROR test_core.py::TestDiscovery::test_streams_has_sync_modes[inputs0] - doc...
ERROR test_core.py::TestDiscovery::test_additional_properties_is_true[inputs0]
ERROR test_core.py::TestDiscovery::test_backward_compatibility[inputs0] - doc...
ERROR test_core.py::TestBasicRead::test_read[inputs0] - docker.errors.Contain...
ERROR test_full_refresh.py::TestFullRefresh::test_sequential_reads[inputs0]
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:57: Backward compatibility tests are disabled for version 0.1.0.
======= 3 failed, 12 passed, 2 skipped, 27 warnings, 10 errors in 25.83s =======

> Task :airbyte-integrations:connectors:source-linkedin-pages:sourceAcceptanceTest FAILED

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
46 actionable tasks: 33 executed, 13 up-to-date

Publishing build scan...
https://gradle.com/s/twfl6l4ixheoe

tinomerl · 2022-11-07T07:58:08Z

Hey @koconder, thanks for reviewing. Let me know if i can be of any help.

vincentkoc · 2022-11-07T22:27:42Z

@sajarin @marcosmarxm i think we need to update the config in github actions to match the changes in the PR. Can we please make the org id in the test config a string.

…ests

sajarin · 2022-11-08T15:44:04Z

/test connector=connectors/source-linkedin-pages

🕑 connectors/source-linkedin-pages https://github.com/airbytehq/airbyte/actions/runs/3420947566
❌ connectors/source-linkedin-pages https://github.com/airbytehq/airbyte/actions/runs/3420947566
🐛 https://gradle.com/s/aaj6rmsanbql4

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestConnection::test_check[inputs0] - AssertionError: as...
FAILED test_core.py::TestBasicRead::test_read[inputs0] - docker.errors.Contai...
FAILED test_full_refresh.py::TestFullRefresh::test_sequential_reads[inputs0]
ERROR test_core.py::TestDiscovery::test_backward_compatibility[inputs0] - doc...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:71: Backward compatibility tests are disabled for version 0.1.0.
======== 3 failed, 21 passed, 2 skipped, 27 warnings, 1 error in 22.59s ========

leminhds · 2022-11-11T11:37:38Z

hi, any plan to merge this soon?

vincentkoc · 2022-11-11T12:23:26Z

@sajarin @marcosmarxm CI is returning the following error:

"error": ***"message": "The token used in the request has expired",

We need to update the sandbox credentials to pass this change for LinkedIn pages

vincentkoc · 2022-11-11T12:24:41Z

@leminhds our automated unit testing is failing, we will need to correct this before we can finalise the PR. Please hold whilst we review the PR and finalise the tests.

tinomerl · 2022-11-18T07:59:59Z

Hey @koconder @marcosmarxm @sajarin any updates on this pr? can i help out in any way?

marcosmarxm · 2022-12-20T19:51:26Z

Hello 👋:skin-tone-2: and thank you for your contribution!

Airbyte has instituted a code freeze between 19 and 30 December, to make sure there are no disruptions during the holidays.
Because of this, reviewing and merging your contribution may take longer than usual.
We apologize for the delay, but we want everyone to have a quiet and happy holiday.

If you have any questions or need further clarification, please don't hesitate to ping via Slack.

firehist · 2023-01-12T06:49:10Z

Thanks a lot @tinomerl and Airbyte Team.

As holidays (hopefully) was nice for everyone regarding airbyte usage, is there any plan on the review of this request?

Thanks a lot.

…ests

natalyjazzviolin · 2023-01-18T14:05:53Z

@koconder I've put in a request for new credentials to run the integration tests!

…ests

vincentkoc · 2023-01-21T10:17:39Z

/test connector=connectors/source-linkedin-pages

🕑 connectors/source-linkedin-pages https://github.com/airbytehq/airbyte/actions/runs/3974371912
❌ connectors/source-linkedin-pages https://github.com/airbytehq/airbyte/actions/runs/3974371912
🐛 https://gradle.com/s/rbopqpigaoylq

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestConnection::test_check[inputs0] - AssertionError: as...
FAILED test_core.py::TestBasicRead::test_read[inputs0] - docker.errors.Contai...
FAILED test_full_refresh.py::TestFullRefresh::test_sequential_reads[inputs0]
ERROR test_core.py::TestDiscovery::test_backward_compatibility[inputs0] - doc...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:103: Backward compatibility tests are disabled for version 0.1.0.
======== 3 failed, 23 passed, 2 skipped, 29 warnings, 1 error in 23.16s ========

sh4sh · 2023-02-07T20:05:22Z

Sorry for the delay here, just need to sign off on the last step and the creds will be updated.

LuliRib · 2023-02-27T10:40:03Z

Hi, any update? Can I help in any way?

tinomerl · 2023-03-13T13:30:20Z

Hey @marcosmarxm, @sh4sh, @koconder

can this pr be merged soon? it's almost 4 months now for what should've been a small fix.

marcosmarxm · 2023-03-13T17:34:28Z

/test connector=connectors/source-linkedin-pages

🕑 connectors/source-linkedin-pages https://github.com/airbytehq/airbyte/actions/runs/4407855048
❌ connectors/source-linkedin-pages https://github.com/airbytehq/airbyte/actions/runs/4407855048
🐛 https://gradle.com/s/pjzxp4n2nvxqy

Build Failed

Test summary info:

=========================== short test summary info ============================
ERROR test_core.py::TestDiscovery::test_backward_compatibility[inputs0] - doc...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:109: Backward compatibility tests are disabled for version 0.1.0.
=================== 32 passed, 2 skipped, 1 error in 33.14s ====================

marcosmarxm · 2023-03-15T19:42:55Z

@tinomerl I'm having some problems with our CI to make the final test pass. Waiting some return from the engineer team about the topic.

tinomerl · 2023-03-16T14:40:45Z

@marcosmarxm thanks for the info and the tests. let me know if i can be of any help.

…ests

marcosmarxm · 2023-03-16T16:45:03Z

/test connector=connectors/source-linkedin-pages

🕑 connectors/source-linkedin-pages https://github.com/airbytehq/airbyte/actions/runs/4439505517
❌ connectors/source-linkedin-pages https://github.com/airbytehq/airbyte/actions/runs/4439505517
🐛

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:109: Backward compatibility tests are disabled for version 0.1.0.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:518: Backward compatibility tests are disabled for version 0.1.0.
======================== 32 passed, 3 skipped in 34.00s ========================

marcosmarxm · 2023-03-16T18:27:51Z

/publish connector=connectors/source-linkedin-pages

🕑 Publishing the following connectors:
connectors/source-linkedin-pages
https://github.com/airbytehq/airbyte/actions/runs/4440420317

Connector	Did it publish?	Were definitions generated?
connectors/source-linkedin-pages	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

marcosmarxm

Thanks @tinomerl

…ests

marcosmarxm · 2023-03-16T19:21:42Z

@tinomerl I'm trying to get the final approval from our connector team because of the backward compatibility issue.

marcosmarxm · 2023-03-16T19:43:31Z

Merged code at #24161

tinomerl added 2 commits November 4, 2022 13:39

fixed issue with the org id not being an integer and deleted duplicat…

68b16eb

…e streams

bumped version to 0.1.1

7dad80e

github-actions bot added the area/connectors Connector related issues label Nov 4, 2022

octavia-squidington-iv added bounty community connectors/source/linkedin-pages labels Nov 4, 2022

Merge branch 'master' into tinomerl/15986-linkedin-pages-connection-t…

8f0038b

…ests

vincentkoc and others added 2 commits November 5, 2022 03:18

Update source_definitions.yaml

8d90b4d

Update acceptance-test-config.yml

bf1450b

vincentkoc previously approved these changes Nov 5, 2022

View reviewed changes

Update linkedin-pages.md

4518d71

github-actions bot added the area/documentation Improvements or additions to documentation label Nov 5, 2022

vincentkoc self-assigned this Nov 5, 2022

black formatting for python

49d1ac5

Merge branch 'master' into tinomerl/15986-linkedin-pages-connection-t…

6011c72

…ests

sajarin added the bounty-L Maintainer program: claimable large bounty PR label Nov 7, 2022

vincentkoc changed the title ~~🐛 Source LinkedIn Pages: fixed failing connection checks~~ 🐛 🚨🚨 Source LinkedIn Pages: fixed failing connection checks Nov 7, 2022

vincentkoc and others added 2 commits November 8, 2022 09:27

Merge branch 'master' into tinomerl/15986-linkedin-pages-connection-t…

1f5f95f

…ests

Merge branch 'master' into tinomerl/15986-linkedin-pages-connection-t…

1706dd2

…ests

Merge branch 'master' into tinomerl/15986-linkedin-pages-connection-t…

6d7ee56

…ests

vincentkoc added 2 commits January 19, 2023 21:17

Merge branch 'master' into tinomerl/15986-linkedin-pages-connection-t…

26c4d18

…ests

Merge branch 'master' into tinomerl/15986-linkedin-pages-connection-t…

9b3154b

…ests

marcosmarxm added 3 commits March 13, 2023 13:58

fix tests

ddddf4b

merge master

363e024

update acceptance test file

462f2f3

marcosmarxm and others added 2 commits March 16, 2023 13:24

Merge branch 'master' into tinomerl/15986-linkedin-pages-connection-t…

7ecdadd

…ests

change break spec - backward compt.

d5b5eaf

fix changelog

fa37036

auto-bump connector version

aedafc3

marcosmarxm approved these changes Mar 16, 2023

View reviewed changes

marcosmarxm enabled auto-merge (squash) March 16, 2023 19:04

marcosmarxm disabled auto-merge March 16, 2023 19:04

Merge branch 'master' into tinomerl/15986-linkedin-pages-connection-t…

ad8f3a5

…ests

marcosmarxm closed this Mar 16, 2023

tinomerl mentioned this pull request Mar 21, 2023

Source LinkedIn Pages: isn't able to pass connection tests #15986

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛 🚨🚨 Source LinkedIn Pages: fixed failing connection checks #18967

🐛 🚨🚨 Source LinkedIn Pages: fixed failing connection checks #18967

tinomerl commented Nov 4, 2022 •

edited

Loading

CLAassistant commented Nov 4, 2022 •

edited

Loading

vincentkoc commented Nov 5, 2022

vincentkoc commented Nov 6, 2022 •

edited by github-actions bot

Loading

tinomerl commented Nov 7, 2022

vincentkoc commented Nov 7, 2022

sajarin commented Nov 8, 2022 •

edited by github-actions bot

Loading

leminhds commented Nov 11, 2022

vincentkoc commented Nov 11, 2022 •

edited

Loading

vincentkoc commented Nov 11, 2022

tinomerl commented Nov 18, 2022

marcosmarxm commented Dec 20, 2022

firehist commented Jan 12, 2023

natalyjazzviolin commented Jan 18, 2023

vincentkoc commented Jan 21, 2023 •

edited by github-actions bot

Loading

sh4sh commented Feb 7, 2023

LuliRib commented Feb 27, 2023

tinomerl commented Mar 13, 2023

marcosmarxm commented Mar 13, 2023 •

edited by github-actions bot

Loading

marcosmarxm commented Mar 15, 2023

tinomerl commented Mar 16, 2023

marcosmarxm commented Mar 16, 2023 •

edited by github-actions bot

Loading

marcosmarxm commented Mar 16, 2023 •

edited by github-actions bot

Loading

marcosmarxm left a comment

marcosmarxm commented Mar 16, 2023

marcosmarxm commented Mar 16, 2023

🐛 🚨🚨 Source LinkedIn Pages: fixed failing connection checks #18967

🐛 🚨🚨 Source LinkedIn Pages: fixed failing connection checks #18967

Conversation

tinomerl commented Nov 4, 2022 • edited Loading

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

Airbyter

Community member or Airbyter

Airbyter

Tests

CLAassistant commented Nov 4, 2022 • edited Loading

vincentkoc commented Nov 5, 2022

vincentkoc commented Nov 6, 2022 • edited by github-actions bot Loading

Build Failed

tinomerl commented Nov 7, 2022

vincentkoc commented Nov 7, 2022

sajarin commented Nov 8, 2022 • edited by github-actions bot Loading

Build Failed

leminhds commented Nov 11, 2022

vincentkoc commented Nov 11, 2022 • edited Loading

vincentkoc commented Nov 11, 2022

tinomerl commented Nov 18, 2022

marcosmarxm commented Dec 20, 2022

firehist commented Jan 12, 2023

natalyjazzviolin commented Jan 18, 2023

vincentkoc commented Jan 21, 2023 • edited by github-actions bot Loading

Build Failed

sh4sh commented Feb 7, 2023

LuliRib commented Feb 27, 2023

tinomerl commented Mar 13, 2023

marcosmarxm commented Mar 13, 2023 • edited by github-actions bot Loading

Build Failed

marcosmarxm commented Mar 15, 2023

tinomerl commented Mar 16, 2023

marcosmarxm commented Mar 16, 2023 • edited by github-actions bot Loading

Build Passed

marcosmarxm commented Mar 16, 2023 • edited by github-actions bot Loading

marcosmarxm left a comment

Choose a reason for hiding this comment

marcosmarxm commented Mar 16, 2023

marcosmarxm commented Mar 16, 2023

tinomerl commented Nov 4, 2022 •

edited

Loading

CLAassistant commented Nov 4, 2022 •

edited

Loading

vincentkoc commented Nov 6, 2022 •

edited by github-actions bot

Loading

sajarin commented Nov 8, 2022 •

edited by github-actions bot

Loading

vincentkoc commented Nov 11, 2022 •

edited

Loading

vincentkoc commented Jan 21, 2023 •

edited by github-actions bot

Loading

marcosmarxm commented Mar 13, 2023 •

edited by github-actions bot

Loading

marcosmarxm commented Mar 16, 2023 •

edited by github-actions bot

Loading

marcosmarxm commented Mar 16, 2023 •

edited by github-actions bot

Loading