CDK: Make consts required in Pydantic generated json schemas

[flash1293]

This PR fixes a problem @chandlerprall noticed.

For specs that are built using pydantic models, each used oneOf requires a constant property that can be used to clearly determine which option was chosen. This property needs the const setting.

In pydantic, this can be enabled via Field("val", const=True), there is a catch however: If you do this, then the field will not be required (because a default value is set, and in pydantic a field with a default value can not be required at the same time).

This works fine for the UI because it has a special handling for this case and always sets the constant property correctly, however for the terraform resource generation, this shows up as optional even though the connector won't work this way:

The same problem also applies to the format selection in file based sources.

This PR fixes the json schema generation for both vector db destinations and file sources.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	⬜️ Ignored (Inspect)	Visit Preview		Nov 7, 2023 5:10pm

[girarda]

do we also need to update connectors that might be using oneOfs in their pydantic spec?

[flash1293]

@girarda after this is merged we need to roll out the change to the connectors consuming it, that's right. However we need to release the CDK before doing this.

[clnoll]

@flash1293 the changes look good to me, but do you have a PR for the connector updates and/or can you confirm that you tested this manually locally? Do you expect there to be any UI changes (e.g. fields marked as required)?

[flash1293]

@clnoll I didn't create the PR yet, but I tested locally it works as expected. There won't be UI changes as only the hidden const field is set to required (tested this with the connector form component here: https://components.airbyte.dev/?path=/story/connector-connectorform--common )

[clnoll]

Awesome, thanks for the info. In that case LGTM!