Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleaned up PyAibyte docs (PR # 35603) #35603

Merged
merged 5 commits into from
Feb 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/assets/docs/airbyte-lib-high-level-architecture.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
23 changes: 16 additions & 7 deletions docs/using-airbyte/airbyte-lib/getting-started.mdx
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
import AirbyteLibConnectors from '@site/src/components/AirbyteLibConnectors';

# Getting Started with AirbyteLib (Beta)
# Getting Started with PyAirbyte (Beta)

AirbyteLib is a library that provides a set of utilities to use Airbyte connectors in Python. It is meant to be used in situations where setting up an Airbyte server or cloud account is not possible or desirable, for example in a Jupyter notebook or when iterating on early prototypes on a developer's workstation.
PyAirbyte is a library that provides a set of utilities to use Airbyte connectors in Python. It is meant to be used in situations where setting up an Airbyte server or cloud account is not possible or desirable, for example in a Jupyter notebook or when iterating on early prototypes on a developer's workstation.

## Installation

```bash
pip install airbyte-lib
pip install airbyte
```

Or during the beta, you may want to install the latest from from source with:

```bash
pip install 'git+https://www.github.com/airbytehq/airbyte.git@master#egg=airbyte-lib&subdirectory=airbyte-lib'
pip install 'git+https://github.com/airbytehq/PyAirbyte.git'
```

## Usage
Expand All @@ -23,7 +23,7 @@ Data can be extracted from sources and loaded into caches:
<a href="https://colab.research.google.com/github/airbytehq/quickstarts/blob/master/airbyte_lib_notebooks/AirbyteLib_Basic_Features_Demo.ipynb" target="_parent"><img src="https://img.shields.io/badge/-Try%20with%20Colab-grey?logo=googlecolab" alt="Try with Colab"/></a>

```python
import airbyte_lib as ab
import airbyte as ab

source = ab.get_source(
"source-faker",
Expand All @@ -38,17 +38,26 @@ for name, records in result.streams.items():
print(f"Stream {name}: {len(list(records))} records")
```

## Quickstarts

* [CoinAPI](https://github.com/airbytehq/quickstarts/blob/main/airbyte_lib_notebooks/AirbyteLib_CoinAPI_Demo.ipynb)
* [GA4](https://github.com/airbytehq/quickstarts/blob/main/airbyte_lib_notebooks/AirbyteLib_GA4_Demo.ipynb)
* [Shopify](https://github.com/airbytehq/quickstarts/blob/main/airbyte_lib_notebooks/PyAirbyte_Shopify_Demo.ipynb)
* [GitHub](https://github.com/airbytehq/quickstarts/blob/main/airbyte_lib_notebooks/AirbyteLib_Github_Incremental_Demo.ipynb)
* [Postgres (cache)](https://github.com/airbytehq/quickstarts/blob/main/airbyte_lib_notebooks/PyAirbyte_Postgres_Custom_Cache_Demo.ipynb)


## API Reference

For details on specific classes and methods, please refer to our [AirbyteLib API Reference](./reference).
For details on specific classes and methods, please refer to our [PyAirbyte API Reference](https://airbytehq.github.io/PyAirbyte/index.html).

## Architecture

[comment]: <> (Edit under https://docs.google.com/drawings/d/1M7ti2D4ha6cEtPnk04RLp1SSh3au4dRJsLupnGPigHQ/edit?usp=sharing)

![Architecture](../../assets/docs/airbyte-lib-high-level-architecture.svg)

airbyte-lib is a python library that can be run in any context that supports Python >=3.9. It contains the following main components:
PyAirbyte is a python library that can be run in any context that supports Python >=3.9. It contains the following main components:
* **Source**: A source object is using a Python connector and includes a configuration object. The configuration object is a dictionary that contains the configuration of the connector, like authentication or connection modalities. The source object is used to read data from the connector.
* **Cache**: Data can be read directly from the source object. However, it is recommended to use a cache object to store the data. The cache object allows to temporarily store records from the source in a SQL database like a local DuckDB file or a Postgres or Snowflake instance.
* **Result**: An object holding the records from a read operation on a source. It allows quick access to the records of each synced stream via the used cache object. Data can be accessed as a list of records, a Pandas DataFrame or via SQLAlchemy queries.
Expand Down
15 changes: 0 additions & 15 deletions docs/using-airbyte/airbyte-lib/reference.mdx

This file was deleted.

3 changes: 3 additions & 0 deletions docusaurus/redirects.yml
Original file line number Diff line number Diff line change
Expand Up @@ -93,3 +93,6 @@
- /enterprise-setup/self-managed/sso
- /enterprise-setup/sso
to: /access-management/sso
- from:
- /pyairbyte
to: /using-airbyte/airbyte-lib/getting-started
Binary file added docusaurus/static/img/pyairbyte-logo-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading