Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source CommerceTools: JSON Schema start_date is self-conflicting; impossible to fill out. #34133

Open
1 task done
theomarshanti opened this issue Jan 10, 2024 · 1 comment · May be fixed by #48564
Open
1 task done

Comments

@theomarshanti
Copy link

theomarshanti commented Jan 10, 2024

Connector Name

source-commercetools

Connector Version

0.2.0

What step the error happened?

Configuring a new connector

Relevant information

Right now it is impossible to register a new CommerceTools source. This is because the json-schema for start-date specifies two mutually incompatible conditions. It declares the type be a start-date, which requires HHMMSS and it declares the type to be a date format. This causes JSON-Schema failures like the following:

Relevant log output

POST http://localhost:8000/api/v1/sources/create

{
    "name": "Commercetools",
    "sourceDefinitionId": "008b2e26-11a3-11ec-82a8-0242ac130003",
    "workspaceId": "4853dbc7-d665-4c32-844e-a48156633689",
    "connectionConfiguration": {
        "start_date": "2024-01-01",
        "client_id": "<>",
        "client_secret": "<>",
        "project_key": "<>",
        "region": "<>",
        "host": "<>"
    }
}
{
    "message": "The provided configuration does not fulfill the specification. Errors: json schema validation failed when comparing the data to the json schema. \nErrors: $.start_date: is missing but it is required \nSchema: \n{\n  \"type\" : \"object\",\n  \"$schema\" : \"http://json-schema.org/draft-07/schema#\",\n  \"required\" : [ \"start_date\", \"client_id\", \"client_secret\", \"host\", \"project_key\", \"region\" ],\n  \"properties\" : {\n    \"host\" : {\n      \"enum\" : [ \"gcp\", \"aws\" ],\n      \"type\" : \"string\",\n      \"order\" : 3,\n      \"title\" : \"Host\",\n      \"description\" : \"The cloud provider your shop is hosted. See: https://docs.commercetools.com/api/authorization\"\n    },\n    \"region\" : {\n      \"type\" : \"string\",\n      \"order\" : 5,\n      \"title\" : \"Region\",\n      \"examples\" : [ \"us-central1\", \"australia-southeast1\" ],\n      \"description\" : \"The region of the platform.\"\n    },\n    \"client_id\" : {\n      \"type\" : \"string\",\n      \"order\" : 1,\n      \"title\" : \"Client ID\",\n      \"description\" : \"Id of API Client.\",\n      \"airbyte_secret\" : true\n    },\n    \"start_date\" : {\n      \"type\" : \"string\",\n      \"order\" : 0,\n      \"title\" : \"Start date\",\n      \"format\" : \"date-time\",\n      \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}$\"\n    },\n    \"project_key\" : {\n      \"type\" : \"string\",\n      \"order\" : 4,\n      \"title\" : \"Project key\",\n      \"description\" : \"The project key\"\n    },\n    \"client_secret\" : {\n      \"type\" : \"string\",\n      \"order\" : 2,\n      \"title\" : \"Client secret\",\n      \"description\" : \"The password of secret of API Client.\",\n      \"airbyte_secret\" : true\n    }\n  },\n  \"additionalProperties\" : true\n}",
    "exceptionClassName": "io.airbyte.commons.server.errors.BadObjectSchemaKnownException",
    "exceptionStack": [
        "io.airbyte.commons.server.errors.BadObjectSchemaKnownException: The provided configuration does not fulfill the specification. Errors: json schema validation failed when comparing the data to the json schema. ",
        "Errors: $.start_date: is missing but it is required ",
        "Schema: ",
        "{",
        "  \"type\" : \"object\",",
        "  \"$schema\" : \"http://json-schema.org/draft-07/schema#\",",
        "  \"required\" : [ \"start_date\", \"client_id\", \"client_secret\", \"host\", \"project_key\", \"region\" ],",
        "  \"properties\" : {",
        "    \"host\" : {",
        "      \"enum\" : [ \"gcp\", \"aws\" ],",
        "      \"type\" : \"string\",",
        "      \"order\" : 3,",
        "      \"title\" : \"Host\",",
        "      \"description\" : \"The cloud provider your shop is hosted. See: https://docs.commercetools.com/api/authorization\"",
        "    },",
        "    \"region\" : {",
        "      \"type\" : \"string\",",
        "      \"order\" : 5,",
        "      \"title\" : \"Region\",",
        "      \"examples\" : [ \"us-central1\", \"australia-southeast1\" ],",
        "      \"description\" : \"The region of the platform.\"",
        "    },",
        "    \"client_id\" : {",
        "      \"type\" : \"string\",",
        "      \"order\" : 1,",
        "      \"title\" : \"Client ID\",",
        "      \"description\" : \"Id of API Client.\",",
        "      \"airbyte_secret\" : true",
        "    },",
        "    \"start_date\" : {",
        "      \"type\" : \"string\",",
        "      \"order\" : 0,",
        "      \"title\" : \"Start date\",",
        "      \"format\" : \"date-time\",",
        "      \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}$\"",
        "    },",
        "    \"project_key\" : {",
        "      \"type\" : \"string\",",
        "      \"order\" : 4,",
        "      \"title\" : \"Project key\",",
        "      \"description\" : \"The project key\"",
        "    },",
        "    \"client_secret\" : {",
        "      \"type\" : \"string\",",
        "      \"order\" : 2,",
        "      \"title\" : \"Client secret\",",
        "      \"description\" : \"The password of secret of API Client.\",",
        "      \"airbyte_secret\" : true",
        "    }",
        "  },",
        "  \"additionalProperties\" : true",
        "}",
        "\tat io.airbyte.server.apis.ApiHelper.execute(ApiHelper.java:40)",
        "\tat io.airbyte.server.apis.SourceApiController.createSource(SourceApiController.java:101)",
        "\tat io.airbyte.server.apis.$SourceApiController$Definition$Exec.dispatch(Unknown Source)",
        "\tat io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invoke(AbstractExecutableMethodsDefinition.java:371)",
        "\tat io.micronaut.context.DefaultBeanContext$4.invoke(DefaultBeanContext.java:594)",
        "\tat io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:303)",
        "\tat io.micronaut.web.router.RouteMatch.execute(RouteMatch.java:111)",
        "\tat io.micronaut.http.context.ServerRequestContext.with(ServerRequestContext.java:103)",
        "\tat io.micronaut.http.server.RouteExecutor.lambda$executeRoute$14(RouteExecutor.java:659)",
        "\tat reactor.core.publisher.FluxDeferContextual.subscribe(FluxDeferContextual.java:49)",
        "\tat reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:62)",
        "\tat reactor.core.publisher.FluxSubscribeOn$SubscribeOnSubscriber.run(FluxSubscribeOn.java:194)",
        "\tat io.micronaut.reactive.reactor.instrument.ReactorInstrumentation.lambda$init$0(ReactorInstrumentation.java:62)",
        "\tat reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)",
        "\tat reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)",
        "\tat io.micronaut.scheduling.instrument.InvocationInstrumenterWrappedCallable.call(InvocationInstrumenterWrappedCallable.java:53)",
        "\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)",
        "\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)",
        "\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)",
        "\tat java.base/java.lang.Thread.run(Thread.java:1583)",
        "Caused by: io.airbyte.validation.json.JsonValidationException: json schema validation failed when comparing the data to the json schema. ",
        "Errors: $.start_date: is missing but it is required ",
        "Schema: ",
        "{",
        "  \"type\" : \"object\",",
        "  \"$schema\" : \"http://json-schema.org/draft-07/schema#\",",
        "  \"required\" : [ \"start_date\", \"client_id\", \"client_secret\", \"host\", \"project_key\", \"region\" ],",
        "  \"properties\" : {",
        "    \"host\" : {",
        "      \"enum\" : [ \"gcp\", \"aws\" ],",
        "      \"type\" : \"string\",",
        "      \"order\" : 3,",
        "      \"title\" : \"Host\",",
        "      \"description\" : \"The cloud provider your shop is hosted. See: https://docs.commercetools.com/api/authorization\"",
        "    },",
        "    \"region\" : {",
        "      \"type\" : \"string\",",
        "      \"order\" : 5,",
        "      \"title\" : \"Region\",",
        "      \"examples\" : [ \"us-central1\", \"australia-southeast1\" ],",
        "      \"description\" : \"The region of the platform.\"",
        "    },",
        "    \"client_id\" : {",
        "      \"type\" : \"string\",",
        "      \"order\" : 1,",
        "      \"title\" : \"Client ID\",",
        "      \"description\" : \"Id of API Client.\",",
        "      \"airbyte_secret\" : true",
        "    },",
        "    \"start_date\" : {",
        "      \"type\" : \"string\",",
        "      \"order\" : 0,",
        "      \"title\" : \"Start date\",",
        "      \"format\" : \"date-time\",",
        "      \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}$\"",
        "    },",
        "    \"project_key\" : {",
        "      \"type\" : \"string\",",
        "      \"order\" : 4,",
        "      \"title\" : \"Project key\",",
        "      \"description\" : \"The project key\"",
        "    },",
        "    \"client_secret\" : {",
        "      \"type\" : \"string\",",
        "      \"order\" : 2,",
        "      \"title\" : \"Client secret\",",
        "      \"description\" : \"The password of secret of API Client.\",",
        "      \"airbyte_secret\" : true",
        "    }",
        "  },",
        "  \"additionalProperties\" : true",
        "}",
        "\tat io.airbyte.validation.json.JsonSchemaValidator.ensure(JsonSchemaValidator.java:157)",
        "\tat io.airbyte.commons.server.handlers.SourceHandler.validateSource(SourceHandler.java:444)",
        "\tat io.airbyte.commons.server.handlers.SourceHandler.createSource(SourceHandler.java:162)",
        "\tat io.airbyte.commons.server.handlers.SourceHandler.createSourceWithOptionalSecret(SourceHandler.java:129)",
        "\tat io.airbyte.server.apis.SourceApiController.lambda$createSource$4(SourceApiController.java:101)",
        "\tat io.airbyte.server.apis.ApiHelper.execute(ApiHelper.java:28)",
        "\t... 19 more"
    ],
    "rootCauseExceptionClassName": "java.lang.Class",
    "rootCauseExceptionStack": [
        "io.airbyte.validation.json.JsonValidationException: json schema validation failed when comparing the data to the json schema. ",
        "Errors: $.start_date: is missing but it is required ",
        "Schema: ",
        "{",
        "  \"type\" : \"object\",",
        "  \"$schema\" : \"http://json-schema.org/draft-07/schema#\",",
        "  \"required\" : [ \"start_date\", \"client_id\", \"client_secret\", \"host\", \"project_key\", \"region\" ],",
        "  \"properties\" : {",
        "    \"host\" : {",
        "      \"enum\" : [ \"gcp\", \"aws\" ],",
        "      \"type\" : \"string\",",
        "      \"order\" : 3,",
        "      \"title\" : \"Host\",",
        "      \"description\" : \"The cloud provider your shop is hosted. See: https://docs.commercetools.com/api/authorization\"",
        "    },",
        "    \"region\" : {",
        "      \"type\" : \"string\",",
        "      \"order\" : 5,",
        "      \"title\" : \"Region\",",
        "      \"examples\" : [ \"us-central1\", \"australia-southeast1\" ],",
        "      \"description\" : \"The region of the platform.\"",
        "    },",
        "    \"client_id\" : {",
        "      \"type\" : \"string\",",
        "      \"order\" : 1,",
        "      \"title\" : \"Client ID\",",
        "      \"description\" : \"Id of API Client.\",",
        "      \"airbyte_secret\" : true",
        "    },",
        "    \"start_date\" : {",
        "      \"type\" : \"string\",",
        "      \"order\" : 0,",
        "      \"title\" : \"Start date\",",
        "      \"format\" : \"date-time\",",
        "      \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}$\"",
        "    },",
        "    \"project_key\" : {",
        "      \"type\" : \"string\",",
        "      \"order\" : 4,",
        "      \"title\" : \"Project key\",",
        "      \"description\" : \"The project key\"",
        "    },",
        "    \"client_secret\" : {",
        "      \"type\" : \"string\",",
        "      \"order\" : 2,",
        "      \"title\" : \"Client secret\",",
        "      \"description\" : \"The password of secret of API Client.\",",
        "      \"airbyte_secret\" : true",
        "    }",
        "  },",
        "  \"additionalProperties\" : true",
        "}",
        "\tat io.airbyte.validation.json.JsonSchemaValidator.ensure(JsonSchemaValidator.java:157)",
        "\tat io.airbyte.commons.server.handlers.SourceHandler.validateSource(SourceHandler.java:444)",
        "\tat io.airbyte.commons.server.handlers.SourceHandler.createSource(SourceHandler.java:162)",
        "\tat io.airbyte.commons.server.handlers.SourceHandler.createSourceWithOptionalSecret(SourceHandler.java:129)",
        "\tat io.airbyte.server.apis.SourceApiController.lambda$createSource$4(SourceApiController.java:101)",
        "\tat io.airbyte.server.apis.ApiHelper.execute(ApiHelper.java:28)",
        "\tat io.airbyte.server.apis.SourceApiController.createSource(SourceApiController.java:101)",
        "\tat io.airbyte.server.apis.$SourceApiController$Definition$Exec.dispatch(Unknown Source)",
        "\tat io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invoke(AbstractExecutableMethodsDefinition.java:371)",
        "\tat io.micronaut.context.DefaultBeanContext$4.invoke(DefaultBeanContext.java:594)",
        "\tat io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:303)",
        "\tat io.micronaut.web.router.RouteMatch.execute(RouteMatch.java:111)",
        "\tat io.micronaut.http.context.ServerRequestContext.with(ServerRequestContext.java:103)",
        "\tat io.micronaut.http.server.RouteExecutor.lambda$executeRoute$14(RouteExecutor.java:659)",
        "\tat reactor.core.publisher.FluxDeferContextual.subscribe(FluxDeferContextual.java:49)",
        "\tat reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:62)",
        "\tat reactor.core.publisher.FluxSubscribeOn$SubscribeOnSubscriber.run(FluxSubscribeOn.java:194)",
        "\tat io.micronaut.reactive.reactor.instrument.ReactorInstrumentation.lambda$init$0(ReactorInstrumentation.java:62)",
        "\tat reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)",
        "\tat reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)",
        "\tat io.micronaut.scheduling.instrument.InvocationInstrumenterWrappedCallable.call(InvocationInstrumenterWrappedCallable.java:53)",
        "\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)",
        "\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)",
        "\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)",
        "\tat java.base/java.lang.Thread.run(Thread.java:1583)"
    ]
}

Contribute

  • Yes, I want to contribute
@theomarshanti
Copy link
Author

I have a fix here:
#34130

@DanyloGL DanyloGL changed the title Source CommerceTools JSON Schema start_date is self-conflicting; impossible to fill out. Source CommerceTools: JSON Schema start_date is self-conflicting; impossible to fill out. Jan 31, 2024
@girarda girarda added the Icebox label Jun 19, 2024
@mombrea mombrea linked a pull request Nov 19, 2024 that will close this issue
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants