Not getting data | JSONDecodeError | {"currently_syncing": null}

[kumarvinod002]

Hi I am using tap-jira and target-csv to export jira data hosted locally to csv
I have configured config.json for jira and tried both discovery mode and sync mode
tap-jira -c config-jira.json -d > catalog.json
tap-jira -c config-jira.json -p catalog.json

There are few issues to be noticed here:

1. Even if i run tap-jira in discovery mode leaving all values for each key in config empty
   { "start_date": "", "username": "", "password": "", "base_url": "", "user_agent": "" }
   or I run with actual correct values, in both cases I get same catalog.json file and I get no exceptions.
2. when run in sync mode on catalog file (generated with correct values in config) I am getting JSONDecodeError exception:

Traceback (most recent call last):
File "C:\Singer\tap-jira\Scripts\tap-jira-script.py", line 33, in
sys.exit(load_entry_point('tap-jira==2.0.0', 'console_scripts', 'tap-jira')())
File "c:\singer\tap-jira\lib\site-packages\tap_jira_init_.py", line 135, in main
raise exc
File "c:\singer\tap-jira\lib\site-packages\tap_jira_init_.py", line 132, in main
main_impl()
File "c:\singer\tap-jira\lib\site-packages\tap_jira_init_.py", line 109, in main_impl
args = get_args()
File "c:\singer\tap-jira\lib\site-packages\tap_jira_init_.py", line 28, in get_args
unchecked_args = utils.parse_args([])
File "c:\singer\tap-jira\lib\site-packages\singer\utils.py", line 174, in parse_args
args.properties = load_json(args.properties)
File "c:\singer\tap-jira\lib\site-packages\singer\utils.py", line 109, in load_json
return json.load(fil)
File "C:\Python39\lib\json_init_.py", line 293, in load
return loads(fp.read(),
File "C:\Python39\lib\json_init_.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Python39\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python39\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

3. I can see syncing null {"currently_syncing": null} when trying to run command for sending data to target-csv

INFO Using Basic Auth API authentication
{"currently_syncing": null}

Q1. Is singer compatible with powershell and windows ?
Q2. Is there something that I am missing in execution ?

[luandy64]

Even if i run tap-jira in discovery mode leaving all values for each key in config empty
{ "start_date": "", "username": "", "password": "", "base_url": "", "user_agent": "" }
or I run with actual correct values, in both cases I get same catalog.json file and I get no exceptions.

The catalog for jira is built completely from the schema files I believe, so this is not surprising to me

File "c:\singer\tap-jira\lib\site-packages\singer\utils.py", line 174, in parse_args
args.properties = load_json(args.properties)

Seems like you have invalid JSON in the catalog made by the tap / in your edits with the metadata

Is singer compatible with powershell and windows ?

No clue. The taps are only written with a *nix environment in mind

[kumarvinod002]

@luandy64 thanks for your response

File "c:\singer\tap-jira\lib\site-packages\singer\utils.py", line 174, in parse_args
args.properties = load_json(args.properties)

Seems like you have invalid JSON in the catalog made by the tap / in your edits with the metadata
classmethod in catalog.py was

    def load(cls, filename):
        with open(filename) as fp:  # pylint: disable=invalid-name
            return Catalog.from_dict(json.load(fp))

which wasnt working for me in python 3.9
I had to update to

    def load(cls, filename):
        with open(filename,**encoding="utf-16"**) as fp:  # pylint: disable=invalid-name
            return Catalog.from_dict(json.load(fp))

I just edited tap_config.json with my JIRA credentials, urls, start_date and useragent.
Then I executed

tap-jira -c config-jira.json -d > catalog.json
tap-jira -c config-jira.json -p catalog.json

noticed in code that -p is depreciated
kindly update README.md for same
moving ahead I also tried
tap-jira -c config-jira.json --catalog catalog.json
In that case I am not getting nay RECORD in data.

I didn't edited anything else. Executed with catalog file generated by discovery mode and set

"selected": true

for project type schema in catalog.json

Am I missing something in execution ? Is there something else I need to do ? I am new user It would be kind of you if you can guide me a little here please.

[kumarvinod002]

@luandy64
Context.is_selected(stream.tap_stream_id)
is not returning true for any of my streams
even if I have "selected": true, added to catalog file

[luandy64]

You should read this guide, I'm not sure you edited the catalog correctly https://github.com/singer-io/getting-started/blob/master/docs/DISCOVERY_MODE.md#example-2

[luandy64]

open(filename,encoding="utf-16")

Seems like this is what is needed to get things working on Windows? I'm not sure what to think here

[kumarvinod002]

@luandy64 thanks for the info on metadata, but in order to get my catalog.json read by tap-jira for sync
I had to change tap-jira/init.py
from

catalog = Catalog.from_dict(args.properties)
if args.properties else discover()

to

catalog = args.catalog
if args.catalog else discover()

Other wise it was expecting -p only and not even accepting -- catalog
As per documentation property option is now stale.

also send method in http.py for request needed update for handling failing SSL_VERIFICATION

If its feasible for you could you please check and modify code in your repo also please?

I have got it working with target-csv for stream: projects
I am exploring other streams e.g. version, issues etc for which I am seeing some exceptions. I will share the relevant information if required.

Thanks in advance.