Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parse dev mode argument #158

Merged
merged 4 commits into from
Nov 3, 2022
Merged

Conversation

RushiT0122
Copy link
Contributor

@RushiT0122 RushiT0122 commented Sep 21, 2022

Description of change

This implementation adds support to parse dev mode arguments -D or --dev in cmdline.

Manual QA steps

  • Manually updated tap singer-python library and from orchestrator tested the discovery with -D and --dev on an existing connection.

Risks

Rollback steps

  • revert this branch

@RushiT0122 RushiT0122 marked this pull request as ready for review September 22, 2022 12:59
@RushiT0122 RushiT0122 force-pushed the TDL-20723-parse-dev-mode-argument branch from bd70722 to b470f8a Compare September 23, 2022 10:32
singer/utils.py Outdated
@@ -134,6 +134,7 @@ def parse_args(required_config_keys):
-d,--discover Run in discover mode
-p,--properties Properties file: DEPRECATED, please use --catalog instead
--catalog Catalog file
-dev, --dev Runs the tap in dev mode
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there really any value to having -dev? If we want a short flag, can we pick some other letter?

In some circles, this would read as "I'm passing -d, -e, and -v and POSIX convention says I can chain them together". And this is not something I want us to introduce to this CLI

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah, I missed that one. I really to like the convention of having the short flag be only one letter. I'd even be good with -D.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the short flag -D for dev mode as suggested.

Replace `-dev` to `-D`
@RushiT0122 RushiT0122 requested a review from luandy64 September 27, 2022 16:41
singer/utils.py Outdated
@@ -163,6 +164,11 @@ def parse_args(required_config_keys):
action='store_true',
help='Do schema discovery')

parser.add_argument(
'-dev', '--dev',

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it needs to be changed here too

singer/utils.py Outdated
@@ -40,7 +40,7 @@ def strptime(dtime):
...
ValueError: time data '2018-01-01T00:00:00' does not match format '%Y-%m-%dT%H:%M:%SZ'

Can't parse non-UTC DTs
Can't parse non-UTC DTsx
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like an errant x committed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleted errant x.

singer/utils.py Outdated
@@ -134,6 +134,7 @@ def parse_args(required_config_keys):
-d,--discover Run in discover mode
-p,--properties Properties file: DEPRECATED, please use --catalog instead
--catalog Catalog file
-D, --dev Runs the tap in dev mode
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest that having a shortcode for this isn't useful since it reduces readability at the savings of 3 characters, maybe if it was --developer-mode or something, but this feels like it would just lead to folks not knowing what the -D means and copy/pasting.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On that note, maybe not critical right now, but the docstring above the changed lines mentions the SPEC and BEST_PRACTICES documents in the getting started repo, so it might be good to eventually update those.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the short flag -D.

@RushiT0122 RushiT0122 requested a review from dmosorast October 17, 2022 15:46
@RushiT0122
Copy link
Contributor Author

@dmosorast please re-review.

@RushiT0122
Copy link
Contributor Author

@luandy64 @kethan1122 please re-review.

@RushiT0122 RushiT0122 merged commit bc3d942 into master Nov 3, 2022
@RushiT0122 RushiT0122 deleted the TDL-20723-parse-dev-mode-argument branch November 3, 2022 08:48
jesuejunior added a commit to sixcodes/singer-python that referenced this pull request Oct 5, 2024
* parse dev mode argument (singer-io#158)

* parse dev mode argument

* update the short flag for dev mode

Replace `-dev` to `-D`

* Updated the short flag for dev mode

* - remove dev mode short flag

Co-authored-by: RushiT0122 <rtodkar@stitchdata-talend.com>

* Bump version 5.13.0 (singer-io#160)

* Bump version 5.13.0

* update changelog for dev mode argument parsing

* change versioning to minor version bump

Co-authored-by: RushiT0122 <rtodkar@stitchdata-talend.com>

* Bump backoff to be compatible with newer python versions (singer-io#165)

* bump backoff for pyton 3.11 compatibility

* update pip version

* try new circleci yml

* remove 'make'

* make pylint happy

* make pylint happy

* make pylint happy again

* backoff version is a breaking change for old python versions

* Changelog update

* Relax dependency version requirements (singer-io#167)

* Relax dependency constraints

* Bump version to `6.0.1`

* pin backoff and simplejson to major version

* Don't allow older versions

* Update changelog

* Pin minumum and major versions

* Make `ensure_ascii` Dynamic with Default Set to `True` in JSON Serialization (singer-io#168)

* add parameter - ensure_ascii to load non ascii characters when set to false

* add unit test for ensuring ascii characters while loading

* update setup and changelog

* Enable copilot usage in PR template according to Qlik policy

* removing simplejson dep and improve dep version

* Adding

* bumping version

* Removing use_decimal param for json.dumps

* Adding a consig to be able to send message to pubsub

* Extracting to a F

* pubsub doesnt expect None as attr

* Parsing state to be used as attr in pubsub

* Set state as stream name when type is state

* feat: supporting a simple entry from lakehouse lib to send messages to a queue

* chore(deps) make jsonschema version flexible

---------

Co-authored-by: Rushikesh Todkar <98420315+RushiT0122@users.noreply.github.com>
Co-authored-by: RushiT0122 <rtodkar@stitchdata-talend.com>
Co-authored-by: Leslie VanDeMark <38043390+leslievandemark@users.noreply.github.com>
Co-authored-by: Bryant Gray <bryant.gray@qlik.com>
Co-authored-by: Sourabh Gandhi <105213416+sgandhi1311@users.noreply.github.com>
Co-authored-by: Eivin Giske Skaaren <eivin.skaaren@qlik.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants