Make DatetimeBasedCursor.end_datetime optional

[maxi297]

What area the feature impact?

Connectors

Revelant Information

Tell us about the problem you're trying to solve

Today, users wanting to configure an incremental sync using DatetimeBasedCursor need to provide a end_datetime. However, the vast majority of users user a combination of today_utc() when the granularity is longer than a day, now_utc() when the granularity is smaller than a day or config["end_datetime"].

Describe the solution you’d like

The default behavior should be to use now_utc() as the value of the end_datetime. Therefore, following

    incremental_sync:
      type: "DatetimeBasedCursor"
      start_datetime:
        datetime: "{{ config['start_date'] }}"
        datetime_format: "%Y-%m"
      step: "P1M"
      datetime_format: "%Y-%m-%dT%H:%M:%S%z"
      cursor_granularity: "PT1S"
      cursor_field: "pub_date"

... is the equivalent of:

    incremental_sync:
      type: "DatetimeBasedCursor"
      start_datetime:
        datetime: "{{ config['start_date'] }}"
        datetime_format: "%Y-%m"
      end_datetime:
        datetime: "{{ now_utc() }}"
        datetime_format: <format for now_utc()>
      step: "P1M"
      datetime_format: "%Y-%m-%dT%H:%M:%S%z"
      cursor_granularity: "PT1S"
      cursor_field: "pub_date"

Note that the end_datetime.datetime_format is using the same as start_datetime.datetime_format. If this is not the case, the user will have the explicitly define end_datetime.datetime_format.
If the connector developer wants to allow the end-user to use config, he will have t

Note that by default, we don't allow use `"{{ config['end_datetime'] or now_utc() }}". This is to avoid unexpected behavior where the connector developer defines a variable that clash with this one.

Acceptance Criteria

• Implement the proposed solution in the CDK
• Update the Connector Builder (UI is fine but it's about state management)
• Update the documentation