Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow skipping bad records instead of raising exceptions #58

Open
lbeltrame opened this issue May 14, 2019 · 3 comments
Open

Allow skipping bad records instead of raising exceptions #58

lbeltrame opened this issue May 14, 2019 · 3 comments

Comments

@lbeltrame
Copy link

Due to the way bad records are handled currently it is really hard to identify what is the cause of an error (long SQLAlchemy backtraces aren't helpful) and makes it impossible to actually generate a database (normally I'd fall back to GEMINI, but I need hg38).

There should be an option of skipping bad records (off by default) when loading VCFs: not optimal, but at least would allow a database to be generated.

@brentp
Copy link
Member

brentp commented May 14, 2019

I think it would be better to improve the error message and/or provide a tool that will find these bad records.
If you have a single-line VCF that generates an sqlalchemy error, then I'll use it to work on improving the error messages.

@lbeltrame
Copy link
Author

Examples are records without gnomad_af data, which cause a sqlalchemy error:

sqlalchemy.exc.StatementError: (sqlalchemy.exc.InvalidRequestError) A value is required for bind parameter 'gnomad_nfe_af', in parameter group 1

The parameter changes every time, because dictionaries are non ordered and the record is (excerpt):

 u'gnomad_afr_af': u'', u'gnomad_nfe_af': u''

@lbeltrame
Copy link
Author

On second thought, what would be useful would be printing the actual VCF record, so people can at least locate the offending line easier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants