Skip to content

Commit

Permalink
Adds only markdown file changes
Browse files Browse the repository at this point in the history
  • Loading branch information
supertopher committed Feb 24, 2023
1 parent ca16502 commit 6e157e9
Show file tree
Hide file tree
Showing 490 changed files with 1,214 additions and 1,214 deletions.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@ It can be as pedantic as a typo or as titanic as an entire tutorial request.


### If applicable, add links to the relevant docs that should be updated
- [Link to relevant docs page](https://docs.airbyte.io)
- [Link to relevant docs page](https://docs.airbyte.com)
20 changes: 10 additions & 10 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Expand the relevant checklist and delete the others.
- [ ] Documentation updated
- [ ] Connector's `README.md`
- [ ] Connector's `bootstrap.md`. See [description and examples](https://docs.google.com/document/d/1ypdgmwmEHWv-TrO4_YOQ7pAJGVrMp5BOkEVh831N260/edit?usp=sharing)
- [ ] `docs/integrations/<source or destination>/<name>.md` including changelog. See changelog [example](https://docs.airbyte.io/integrations/sources/stripe#changelog)
- [ ] `docs/integrations/<source or destination>/<name>.md` including changelog. See changelog [example](https://docs.airbyte.com/integrations/sources/stripe#changelog)
- [ ] `docs/integrations/README.md`
- [ ] `airbyte-integrations/builds.md`
- [ ] PR name follows [PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/issues-and-pull-requests)
Expand All @@ -37,11 +37,11 @@ If this is a community PR, the Airbyte engineer reviewing this PR is responsible

- [ ] Create a non-forked branch based on this PR and test the below items on it
- [ ] Build is successful
- [ ] If new credentials are required for use in CI, add them to GSM. [Instructions](https://docs.airbyte.io/connector-development#using-credentials-in-ci).
- [ ] [`/test connector=connectors/<name>` command](https://docs.airbyte.io/connector-development#updating-an-existing-connector) is passing
- [ ] New Connector version released on Dockerhub by running the `/publish` command described [here](https://docs.airbyte.io/connector-development#updating-an-existing-connector)
- [ ] After the connector is published, connector added to connector index as described [here](https://docs.airbyte.io/connector-development#publishing-a-connector)
- [ ] Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described [here](https://docs.airbyte.io/connector-development#publishing-a-connector)
- [ ] If new credentials are required for use in CI, add them to GSM. [Instructions](https://docs.airbyte.com/connector-development#using-credentials-in-ci).
- [ ] [`/test connector=connectors/<name>` command](https://docs.airbyte.com/connector-development#updating-an-existing-connector) is passing
- [ ] New Connector version released on Dockerhub by running the `/publish` command described [here](https://docs.airbyte.com/connector-development#updating-an-existing-connector)
- [ ] After the connector is published, connector added to connector index as described [here](https://docs.airbyte.com/connector-development#publishing-a-connector)
- [ ] Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described [here](https://docs.airbyte.com/connector-development#publishing-a-connector)

</details>

Expand All @@ -56,7 +56,7 @@ If this is a community PR, the Airbyte engineer reviewing this PR is responsible
- [ ] Documentation updated
- [ ] Connector's `README.md`
- [ ] Connector's `bootstrap.md`. See [description and examples](https://docs.google.com/document/d/1ypdgmwmEHWv-TrO4_YOQ7pAJGVrMp5BOkEVh831N260/edit?usp=sharing)
- [ ] Changelog updated in `docs/integrations/<source or destination>/<name>.md` including changelog. See changelog [example](https://docs.airbyte.io/integrations/sources/stripe#changelog)
- [ ] Changelog updated in `docs/integrations/<source or destination>/<name>.md` including changelog. See changelog [example](https://docs.airbyte.com/integrations/sources/stripe#changelog)
- [ ] PR name follows [PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/issues-and-pull-requests)

### Airbyter
Expand All @@ -65,9 +65,9 @@ If this is a community PR, the Airbyte engineer reviewing this PR is responsible

- [ ] Create a non-forked branch based on this PR and test the below items on it
- [ ] Build is successful
- [ ] If new credentials are required for use in CI, add them to GSM. [Instructions](https://docs.airbyte.io/connector-development#using-credentials-in-ci).
- [ ] [`/test connector=connectors/<name>` command](https://docs.airbyte.io/connector-development#updating-an-existing-connector) is passing
- [ ] New Connector version released on Dockerhub and connector version bumped by running the `/publish` command described [here](https://docs.airbyte.io/connector-development#updating-an-existing-connector)
- [ ] If new credentials are required for use in CI, add them to GSM. [Instructions](https://docs.airbyte.com/connector-development#using-credentials-in-ci).
- [ ] [`/test connector=connectors/<name>` command](https://docs.airbyte.com/connector-development#updating-an-existing-connector) is passing
- [ ] New Connector version released on Dockerhub and connector version bumped by running the `/publish` command described [here](https://docs.airbyte.com/connector-development#updating-an-existing-connector)

</details>

Expand Down
2 changes: 1 addition & 1 deletion CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
# Code of conduct
View in [docs.airbyte.io](https://docs.airbyte.io/contributing-to-airbyte/code-of-conduct)
View in [docs.airbyte.com](https://docs.airbyte.com/contributing-to-airbyte/code-of-conduct)
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
# Contributing
View on [docs.airbyte.io](https://docs.airbyte.io/contributing-to-airbyte)
View on [docs.airbyte.com](https://docs.airbyte.com/contributing-to-airbyte)
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ Deployment options: [Docker](https://docs.airbyte.com/deploying-airbyte/local-de

Airbyte Cloud is the fastest and most reliable way to run Airbyte. You can get started with free credits in minutes.

Sign up for [Airbyte Cloud](https://cloud.airbyte.io/signup).
Sign up for [Airbyte Cloud](https://cloud.airbyte.com/signup).

## Contributing

Expand All @@ -99,7 +99,7 @@ See the [LICENSE](docs/project-overview/licenses/) file for licensing informatio

## Resources

- [Weekly office hours](https://airbyte.io/weekly-office-hours/) for live informal sessions with the Airbyte team
- [Weekly office hours](https://airbyte.com/weekly-office-hours/) for live informal sessions with the Airbyte team
- [Slack](https://slack.airbyte.io) for quick discussion with the Community and Airbyte team
- [Discourse](https://discuss.airbyte.io/) for deeper conversations about features, connectors, and problems
- [GitHub](https://github.com/airbytehq/airbyte) for code, issues and pull requests
Expand Down
2 changes: 1 addition & 1 deletion airbyte-cdk/python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ cd airbyte-integrations/connector-templates/generator

then follow the interactive prompt. Next, find all `TODO`s in the generated project directory -- they're accompanied by lots of comments explaining what you'll need to do in order to implement your connector. Upon completing all TODOs properly, you should have a functioning connector.

Additionally, you can follow [this tutorial](https://docs.airbyte.io/connector-development/tutorials/cdk-tutorial-python-http) for a complete walkthrough of creating an HTTP connector using the Airbyte CDK.
Additionally, you can follow [this tutorial](https://docs.airbyte.com/connector-development/tutorials/cdk-tutorial-python-http) for a complete walkthrough of creating an HTTP connector using the Airbyte CDK.

### Concepts & Documentation

Expand Down
2 changes: 1 addition & 1 deletion airbyte-cdk/python/docs/concepts/incremental-stream.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Several new pieces are essential to understand how incrementality works with the

### `AirbyteStateMessage`

The `AirbyteStateMessage` persists state between syncs, and allows a new sync to pick up from where the previous sync last finished. See the [incremental sync guide](https://docs.airbyte.io/understanding-airbyte/connections/incremental-append) for more information.
The `AirbyteStateMessage` persists state between syncs, and allows a new sync to pick up from where the previous sync last finished. See the [incremental sync guide](https://docs.airbyte.com/understanding-airbyte/connections/incremental-append) for more information.

### Cursor fields

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# CDK Speedrun (HTTP API Source Creation [Any%](https://en.wikipedia.org/wiki/Speedrun#:~:text=Any%25%2C%20or%20fastest%20completion%2C,the%20game%20to%20its%20fullest.&text=Specific%20requirements%20for%20a%20100,different%20depending%20on%20the%20game.) Route)

This is a blazing fast guide to building an HTTP source connector. Think of it as the TL;DR version of [this tutorial](https://docs.airbyte.io/connector-development/tutorials/cdk-tutorial-python-http).
This is a blazing fast guide to building an HTTP source connector. Think of it as the TL;DR version of [this tutorial](https://docs.airbyte.com/connector-development/tutorials/cdk-tutorial-python-http).

# Dependencies

Expand Down Expand Up @@ -36,7 +36,7 @@ We're working with the Exchange Rates API, so we need to define our input schema

```json
{
"documentationUrl": "https://docs.airbyte.io/integrations/sources/exchangeratesapi",
"documentationUrl": "https://docs.airbyte.com/integrations/sources/exchangeratesapi",
"connectionSpecification": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Python Http Tutorial Spec",
Expand Down Expand Up @@ -217,4 +217,4 @@ If all goes well, containerize it so you can use it in the UI:
docker build . -t airbyte/source-python-http-example:dev
```

You're done. Stop the clock :)
You're done. Stop the clock :)
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ Each connector declares the inputs it needs to read data from the underlying dat

The simplest way to implement this is by creating a `spec.json` file in `source_<name>/spec.json` which describes your connector's inputs according to the [ConnectorSpecification](https://github.com/airbytehq/airbyte/blob/master/airbyte-protocol/models/src/main/resources/airbyte_protocol/airbyte_protocol.yaml#L211) schema. This is a good place to start when developing your source. Using JsonSchema, define what the inputs are \(e.g. username and password\). Here's [an example](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-freshdesk/source_freshdesk/spec.json) of what the `spec.json` looks like for the Freshdesk API source.

For more details on what the spec is, you can read about the Airbyte Protocol [here](https://docs.airbyte.io/understanding-airbyte/airbyte-protocol).
For more details on what the spec is, you can read about the Airbyte Protocol [here](https://docs.airbyte.com/understanding-airbyte/airbyte-protocol).

The generated code that Airbyte provides, handles implementing the `spec` method for you. It assumes that there will be a file called `spec.json` in the same directory as `source.py`. If you have declared the necessary JsonSchema in `spec.json` you should be done with this step.

Given that we'll pulling currency data for our example source, we'll define the following `spec.json`:

```text
{
"documentationUrl": "https://docs.airbyte.io/integrations/sources/exchangeratesapi",
"documentationUrl": "https://docs.airbyte.com/integrations/sources/exchangeratesapi",
"connectionSpecification": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Python Http Tutorial Spec",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Step 5: Declare the Schema

The `discover` method of the Airbyte Protocol returns an `AirbyteCatalog`: an object which declares all the streams output by a connector and their schemas. It also declares the sync modes supported by the stream \(full refresh or incremental\). See the [catalog tutorial](https://docs.airbyte.io/understanding-airbyte/beginners-guide-to-catalog) for more information.
The `discover` method of the Airbyte Protocol returns an `AirbyteCatalog`: an object which declares all the streams output by a connector and their schemas. It also declares the sync modes supported by the stream \(full refresh or incremental\). See the [catalog tutorial](https://docs.airbyte.com/understanding-airbyte/beginners-guide-to-catalog) for more information.

This is a simple task with the Airbyte CDK. For each stream in our connector we'll need to: 1. Create a python `class` in `source.py` which extends `HttpStream` 2. Place a `<stream_name>.json` file in the `source_<name>/schemas/` directory. The name of the file should be the snake\_case name of the stream whose schema it describes, and its contents should be the JsonSchema describing the output from that stream.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ Let's also pass the config specified by the user to the stream class:

We're now ready to query the API!

To do this, we'll need a [ConfiguredCatalog](https://docs.airbyte.io/understanding-airbyte/beginners-guide-to-catalog#configuredairbytecatalog). We've prepared one [here](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/python/docs/tutorials/http_api_source_assets/configured_catalog.json) -- download this and place it in `sample_files/configured_catalog.json`. Then run:
To do this, we'll need a [ConfiguredCatalog](https://docs.airbyte.com/understanding-airbyte/beginners-guide-to-catalog#configuredairbytecatalog). We've prepared one [here](https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/python/docs/tutorials/http_api_source_assets/configured_catalog.json) -- download this and place it in `sample_files/configured_catalog.json`. Then run:

```text
python main.py read --config sample_files/config.json --catalog sample_files/configured_catalog.json
Expand All @@ -132,7 +132,7 @@ To add incremental sync, we'll do a few things:
6. Update the `path` method to specify the date to pull exchange rates for.
7. Update the configured catalog to use `incremental` sync when we're testing the stream.

We'll describe what each of these methods do below. Before we begin, it may help to familiarize yourself with how incremental sync works in Airbyte by reading the [docs on incremental](https://docs.airbyte.io/architecture/connections/incremental-append).
We'll describe what each of these methods do below. Before we begin, it may help to familiarize yourself with how incremental sync works in Airbyte by reading the [docs on incremental](https://docs.airbyte.com/architecture/connections/incremental-append).

To keep things concise, we'll only show functions as we edit them one by one.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Step 7: Use the Connector in Airbyte

To use your connector in your own installation of Airbyte, build the docker image for your container by running `docker build . -t airbyte/source-python-http-example:dev`. Then, follow the instructions from the [building a python source tutorial](https://docs.airbyte.io/connector-development/tutorials/building-a-python-source) for using the connector in the Airbyte UI, replacing the name as appropriate.
To use your connector in your own installation of Airbyte, build the docker image for your container by running `docker build . -t airbyte/source-python-http-example:dev`. Then, follow the instructions from the [building a python source tutorial](https://docs.airbyte.com/connector-development/tutorials/building-a-python-source) for using the connector in the Airbyte UI, replacing the name as appropriate.

Note: your built docker image must be accessible to the `docker` daemon running on the Airbyte node. If you're doing this tutorial locally, these instructions are sufficient. Otherwise you may need to push your Docker image to Dockerhub.
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,6 @@ Place any integration tests in the `integration_tests` directory such that they

## Standard Tests

Standard tests are a fixed set of tests Airbyte provides that every Airbyte source connector must pass. While they're only required if you intend to submit your connector to Airbyte, you might find them helpful in any case. See [Testing your connectors](https://docs.airbyte.io/connector-development/testing-connectors)
Standard tests are a fixed set of tests Airbyte provides that every Airbyte source connector must pass. While they're only required if you intend to submit your connector to Airbyte, you might find them helpful in any case. See [Testing your connectors](https://docs.airbyte.com/connector-development/testing-connectors)

If you want to submit this connector to become a default connector within Airbyte, follow steps 8 onwards from the [Python source checklist](https://docs.airbyte.io/connector-development/tutorials/building-a-python-source#step-8-set-up-standard-tests)
If you want to submit this connector to become a default connector within Airbyte, follow steps 8 onwards from the [Python source checklist](https://docs.airbyte.com/connector-development/tutorials/building-a-python-source#step-8-set-up-standard-tests)
Loading

0 comments on commit 6e157e9

Please sign in to comment.