Run ci checks for ✨ Source Insightly : Migrate Python CDK to Low-code CDK

airbyte-integrations/connectors/source-insightly/Dockerfile

@@ -1,4 +1,4 @@
-FROM python:3.9.13-alpine3.15 as base
+FROM python:3.9.11-alpine3.15 as base
 
 # build and load all requirements
 FROM base as builder
@@ -34,5 +34,5 @@ COPY source_insightly ./source_insightly
 ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
 ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
 
-LABEL io.airbyte.version=0.1.3
+LABEL io.airbyte.version=0.2.0
 LABEL io.airbyte.name=airbyte/source-insightly

airbyte-integrations/connectors/source-insightly/README.md

@@ -1,70 +1,52 @@
 # Insightly Source
 
-This is the repository for the Insightly source connector, written in Python.
-For information about how to use this connector within Airbyte, see [the documentation](https://docs.airbyte.io/integrations/sources/insightly).
+This is the repository for the Insightly configuration based source connector.
+For information about how to use this connector within Airbyte, see [the documentation](https://docs.airbyte.com/integrations/sources/insightly).
 
 ## Local development
 
-### Prerequisites
-**To iterate on this connector, make sure to complete this prerequisites section.**
+#### Building via Gradle
 
-#### Minimum Python version required `= 3.9.0`
+You can also build the connector in Gradle. This is typically used in CI and not needed for your development workflow.
 
-#### Build & Activate Virtual Environment and install dependencies
-From this connector directory, create a virtual environment:
-```
-python -m venv .venv
-```
+To build using Gradle, from the Airbyte repository root, run:
 
-This will generate a virtualenv for this module in `.venv/`. Make sure this venv is active in your
-development environment of choice. To activate it from the terminal, run:
 ```
-source .venv/bin/activate
-pip install -r requirements.txt
-pip install '.[tests]'
+./gradlew :airbyte-integrations:connectors:source-insightly:build
 ```
-If you are in an IDE, follow your IDE's instructions to activate the virtualenv.
-
-Note that while we are installing dependencies from `requirements.txt`, you should only edit `setup.py` for your dependencies. `requirements.txt` is
-used for editable installs (`pip install -e`) to pull in Python dependencies from the monorepo and will call `setup.py`.
-If this is mumbo jumbo to you, don't worry about it, just put your deps in `setup.py` but install using `pip install -r requirements.txt` and everything
-should work as you expect.
 
 #### Create credentials
-**If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.io/integrations/sources/insightly)
+
+**If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.com/integrations/sources/insightly)
 to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_insightly/spec.yaml` file.
 Note that any directory named `secrets` is gitignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information.
 See `integration_tests/sample_config.json` for a sample config file.
 
 **If you are an Airbyte core member**, copy the credentials in Lastpass under the secret name `source insightly test creds`
 and place them into `secrets/config.json`.
 
-### Locally running the connector
-```
-python main.py spec
-python main.py check --config secrets/config.json
-python main.py discover --config secrets/config.json
-python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json
-```
-
 ### Locally running the connector docker image
 
-
 #### Build
+
 **Via [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md) (recommended):**
+
 ```bash
 airbyte-ci connectors --name=source-insightly build
 ```
 
 An image will be built with the tag `airbyte/source-insightly:dev`.
 
 **Via `docker build`:**
+
 ```bash
 docker build -t airbyte/source-insightly:dev .
 ```
 
 #### Run
+
 Then run any of the connector commands as follows:
+
 ```
 docker run --rm airbyte/source-insightly:dev spec
 docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-insightly:dev check --config /secrets/config.json
@@ -73,28 +55,65 @@ docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integrat
 ```
 
 ## Testing
+
+<<<<<<< HEAD
+
+#### Acceptance Tests
+
+Customize `acceptance-test-config.yml` file to configure tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
+If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py.
+
+To run your integration tests with Docker, run:
+
+```
+./acceptance-test-docker.sh
+```
+
+### Using gradle to run tests
+
+All commands should be run from airbyte project root.
+To run unit tests:
+
+```
+./gradlew :airbyte-integrations:connectors:source-insightly:unitTest
+```
+
+To run acceptance and custom integration tests:
+
+```
+./gradlew :airbyte-integrations:connectors:source-insightly:integrationTest
+```
+
+=======
 You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):
+
 ```bash
 airbyte-ci connectors --name=source-insightly test
 ```
 
 ### Customizing acceptance Tests
+
 Customize `acceptance-test-config.yml` file to configure tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
 If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py.
 
+> > > > > > > master
+
 ## Dependency Management
+
 All of your dependencies should go in `setup.py`, NOT `requirements.txt`. The requirements file is only used to connect internal Airbyte dependencies in the monorepo for local development.
 We split dependencies between two groups, dependencies that are:
-* required for your connector to work need to go to `MAIN_REQUIREMENTS` list.
-* required for the testing need to go to `TEST_REQUIREMENTS` list
+
+- required for your connector to work need to go to `MAIN_REQUIREMENTS` list.
+- required for the testing need to go to `TEST_REQUIREMENTS` list
 
 ### Publishing a new version of the connector
+
 You've checked out the repo, implemented a million dollar feature, and you're ready to share your changes with the world. Now what?
+
 1. Make sure your changes are passing our test suite: `airbyte-ci connectors --name=source-insightly test`
 2. Bump the connector version in `metadata.yaml`: increment the `dockerImageTag` value. Please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors).
 3. Make sure the `metadata.yaml` content is up to date.
 4. Make the connector documentation and its changelog is up to date (`docs/integrations/sources/insightly.md`).
 5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention).
 6. Pat yourself on the back for being an awesome contributor.
 7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.
-

airbyte-integrations/connectors/source-insightly/acceptance-test-config.yml

@@ -18,10 +18,46 @@ acceptance_tests:
     tests:
       - config_path: "secrets/config.json"
         configured_catalog_path: "integration_tests/configured_catalog.json"
+        empty_streams:
+          - name: emails
+            bypass_reason: "no data for this stream in our sandbox account"
+          - name: events
+            bypass_reason: "no data for this stream in our sandbox account"
+          - name: milestones
+            bypass_reason: "no data for this stream in our sandbox account"
+          - name: notes
+            bypass_reason: "no data for this stream in our sandbox account"
+          - name: opportunity_categories
+            bypass_reason: "no data for this stream in our sandbox account"
+          - name: project_categories
+            bypass_reason: "no data for this stream in our sandbox account"
+          - name: knowledge_article_categories
+            bypass_reason: "current sandbox account does not have permissions to access this stream (403 error)"
+          - name: knowledge_article_folders
+            bypass_reason: "current sandbox account does not have permissions to access this stream (403 error)"
+          - name: knowledge_articles
+            bypass_reason: "current sandbox account does not have permissions to access this stream (403 error)"
+          - name: lead_sources
+            bypass_reason: "current sandbox account does not have permissions to access this stream (403 error)"
+          - name: lead_statuses
+            bypass_reason: "current sandbox account does not have permissions to access this stream (403 error)"
+          - name: prospects
+            bypass_reason: "current sandbox account does not have permissions to access this stream (403 error)"
+
   full_refresh:
     tests:
       - config_path: "secrets/config.json"
         configured_catalog_path: "integration_tests/configured_catalog.json"
+        ignored_fields:
+          contacts:
+            - name: "IMAGE_URL"
+              bypass_reason: "image url is a dynamic s3 url with a changing expiration date in the query parameter"
+          organisations:
+            - name: "IMAGE_URL"
+              bypass_reason: "image url is a dynamic s3 url with a changing expiration date in the query parameter"
+          users:
+            - name: "USER_CURRENCY"
+              bypass_reason: "this can change between sequential reads"
   incremental:
     tests:
       - config_path: "secrets/config.json"

airbyte-integrations/connectors/source-insightly/integration_tests/abnormal_state.json

@@ -1,12 +1,144 @@
 [
+  {
+    "type": "STREAM",
+    "stream": {
+      "stream_descriptor": {
+        "name": "contacts"
+      },
+      "stream_state": {
+        "DATE_UPDATED_UTC": "2122-10-17 19:10:14"
+      }
+    }
+  },
+  {
+    "type": "STREAM",
+    "stream": {
+      "stream_descriptor": {
+        "name": "events"
+      },
+      "stream_state": {
+        "DATE_UPDATED_UTC": "2122-10-17 19:10:14"
+      }
+    }
+  },
+  {
+    "type": "STREAM",
+    "stream": {
+      "stream_descriptor": {
+        "name": "knowledge_article_categories"
+      },
+      "stream_state": {
+        "DATE_UPDATED_UTC": "2122-10-17 19:10:14"
+      }
+    }
+  },
+  {
+    "type": "STREAM",
+    "stream": {
+      "stream_descriptor": {
+        "name": "knowledge_article_folders"
+      },
+      "stream_state": {
+        "DATE_UPDATED_UTC": "2122-10-17 19:10:14"
+      }
+    }
+  },
+  {
+    "type": "STREAM",
+    "stream": {
+      "stream_descriptor": {
+        "name": "knowledge_articles"
+      },
+      "stream_state": {
+        "DATE_UPDATED_UTC": "2122-10-17 19:10:14"
+      }
+    }
+  },
+  {
+    "type": "STREAM",
+    "stream": {
+      "stream_descriptor": {
+        "name": "milestones"
+      },
+      "stream_state": {
+        "DATE_UPDATED_UTC": "2122-10-17 19:10:14"
+      }
+    }
+  },
+  {
+    "type": "STREAM",
+    "stream": {
+      "stream_descriptor": {
+        "name": "notes"
+      },
+      "stream_state": {
+        "DATE_UPDATED_UTC": "2122-10-17 19:10:14"
+      }
+    }
+  },
+  {
+    "type": "STREAM",
+    "stream": {
+      "stream_descriptor": {
+        "name": "opportunities"
+      },
+      "stream_state": {
+        "DATE_UPDATED_UTC": "2122-10-17 19:10:14"
+      }
+    }
+  },
+  {
+    "type": "STREAM",
+    "stream": {
+      "stream_descriptor": {
+        "name": "organisations"
+      },
+      "stream_state": {
+        "DATE_UPDATED_UTC": "2122-10-17 19:10:14"
+      }
+    }
+  },
+  {
+    "type": "STREAM",
+    "stream": {
+      "stream_descriptor": {
+        "name": "projects"
+      },
+      "stream_state": {
+        "DATE_UPDATED_UTC": "2122-10-17 19:10:14"
+      }
+    }
+  },
+  {
+    "type": "STREAM",
+    "stream": {
+      "stream_descriptor": {
+        "name": "prospects"
+      },
+      "stream_state": {
+        "DATE_UPDATED_UTC": "2122-10-17 19:10:14"
+      }
+    }
+  },
+  {
+    "type": "STREAM",
+    "stream": {
+      "stream_descriptor": {
+        "name": "tasks"
+      },
+      "stream_state": {
+        "DATE_UPDATED_UTC": "2122-10-17 19:10:14"
+      }
+    }
+  },
   {
     "type": "STREAM",
     "stream": {
       "stream_descriptor": {
         "name": "users"
       },
       "stream_state": {
-        "DATE_UPDATED_UTC": "2122-10-17T19:10:14+00:00"
+        "DATE_UPDATED_UTC": "2122-10-17 19:10:14"
       }
     }
   }