Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source Faker: export json files #23468

Merged
merged 2 commits into from
Feb 28, 2023
Merged

Conversation

sherifnada
Copy link
Contributor

What

This bundles all the JSONs in the schemas/ directory with the wheel/pip package which gets built when someone runs pip install against this package.

The motivation is that I'm working on a project which directly imports connectors such as this one from another python package e.g from source_faker import SourceFaker. But read and discover don't run properly since this these JSON files are needed for the connector to work but are not being packaged with the pip package produced by this connector.

Notes

This is not meaningful enough to require publishing a new connector version so I won't do it. But I'll run /test anyways

@sherifnada
Copy link
Contributor Author

sherifnada commented Feb 27, 2023

/test connector=source-faker

🕑 source-faker https://github.com/airbytehq/airbyte/actions/runs/4278908181
✅ source-faker https://github.com/airbytehq/airbyte/actions/runs/4278908181
Python tests coverage:

Name                                               Stmts   Miss  Cover
----------------------------------------------------------------------
source_faker/__init__.py                               2      0   100%
source_faker/streams.py                              126      1    99%
source_faker/source.py                                17      3    82%
source_faker/utils.py                                 18      6    67%
source_faker/airbyte_message_with_cached_json.py       8      4    50%
source_faker/user_generator.py                        28     16    43%
source_faker/purchase_generator.py                    55     41    25%
----------------------------------------------------------------------
TOTAL                                                254     71    72%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:98: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:507: The previous and actual discovered catalogs are identical.
======================== 35 passed, 2 skipped in 39.35s ========================

@sherifnada sherifnada changed the title Source Faker: export schema json files Source Faker: export json files Feb 27, 2023
Copy link
Contributor

@evantahler evantahler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet.

package_data={"": ["*.json"]},
package_data={"": ["*.json", "schemas/*.json", "record_data/*.json"]},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For all connectors which might one day need to do this... is there a globpattern we should add to bundle all data files like ["**/*.json", "**/*.yaml"]?

@sherifnada sherifnada merged commit a595bb4 into master Feb 28, 2023
@sherifnada sherifnada deleted the sherif/faker-export-schema-json branch February 28, 2023 17:55
danielduckworth pushed a commit to danielduckworth/airbyte that referenced this pull request Mar 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants