Low code AddFields does not work as expected

[alexander-marquardt]

Environment

• Airbyte version: v0.40.14
• OS Version / Instance: macOS
• Deployment: Docker
• Source Connector and version: Custom - low code connector
• Step where error happened: Python / Read

Current Behavior

I have setup a low code connector following instructions in our docs. I then attempt to make use of the AddFields tranformation functionality but the "path" appears to be taken from the wrong field -- looks like it is trying to use the $options.path value rather than the path defined within the transformations.

My lowcode.yaml file looks as follows:

version: "0.1.0"

definitions:
  selector:
    extractor:
      field_pointer: [ ]
  requester:
    url_base: "https://api.apilayer.com"
    http_method: "GET"
    authenticator:
      type: ApiKeyAuthenticator
      header: "apikey"
      api_token: "{{ config['access_key'] }}"
    request_options_provider:
      request_parameters:
        base: "{{ config['base'] }}"
  retriever:
    record_selector:
      $ref: "*ref(definitions.selector)"
    paginator:
      type: NoPagination
    requester:
      $ref: "*ref(definitions.requester)"
  base_stream:
    retriever:
      $ref: "*ref(definitions.retriever)"
  rates_stream:
    $ref: "*ref(definitions.base_stream)"
    $options:
      name: "rates"
      primary_key: "date"
      path: "/exchangerates_data/latest"
    transformations:
      - type: RemoveFields
        field_pointers:
          - ["rates", "AED"]
      - type: AddFields
        fields:
          - path: ["some_new_path"]
            value: "some new string foobar"

streams:
  - "*ref(definitions.rates_stream)"
check:
  stream_names:
    - "rates"

And the output is :

{
  "type": "RECORD",
  "record": {
    "stream": "rates",
    "data": {
      "success": true,
      "timestamp": 1665675123,
      "base": "USD",
      "date": "2022-10-13",
      "rates": {
        "AFN": 85.433018,
         .... etc ...
      },
,
      "exchangerates_data": {
        "latest": "some new string foobar"
      }
    },

Expected Behavior

I would expect a new field called some_new_path to be added but it looks like the path that is used for creating the new field is taken from the URL path instead.

Steps to Reproduce

1. Follow the lowcode tutorial
2. Add additional Yaml to try to add a new field

Are you willing to submit a PR?

Possibly, but not likely as I am not totally sure how this code works.

[alexander-marquardt]

@girarda - the merged code appears to fix this. I'm not sure what the correct process is. Should I close this issue?