Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix batched_advisories method #338

Closed
sbs2001 opened this issue Feb 7, 2021 · 1 comment
Closed

Fix batched_advisories method #338

sbs2001 opened this issue Feb 7, 2021 · 1 comment
Labels
bug

Comments

@sbs2001
Copy link
Collaborator

sbs2001 commented Feb 7, 2021 •
edited
Loading

The current implementation of the batched_advisories at https://github.com/nexB/vulnerablecode/blob/6609bcfdebb4c28bd30dfd9d42011323d238ee4c/vulnerabilities/data_source.py#L195 needs to be simplified

  1. Don't use sets, the hash method for the advisory dataclass is broken.
  2. Simplify the logic.

Eventually we want to move to non-batch proccessing so this low priority
@sbs2001 sbs2001 mentioned this issue Mar 25, 2021
Merged
6 tasks
Hritik14 added a commit to Hritik14/vulnerablecode that referenced this issue Mar 25, 2021
@Hritik14
Refactor according to first review
3a08cd8 
Use batch_advisories for now. It has it's own problems and there's aboutcode-org#338 for that.
The generator thing won't do much, since we are importing like 10-20 MBs of data.

The codebase already has overuse of methods starting with _ , I'd say avoid them.
They don't help with readability nor are they trivial in this case

_parse_md and _parse_yml:
The name is misleading. The function is parsing + enriching the data.
converted to get_advisories_from_yml and get_advisories_from_md

Remove `"branch": None` in importer_yielder

Group imports

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
Hritik14 added a commit to Hritik14/vulnerablecode that referenced this issue Mar 29, 2021
@Hritik14
Refactor according to first review
923197d 
Use batch_advisories for now. It has it's own problems and there's aboutcode-org#338 for that.
The generator thing won't do much, since we are importing like 10-20 MBs of data.

The codebase already has overuse of methods starting with _ , I'd say avoid them.
They don't help with readability nor are they trivial in this case

_parse_md and _parse_yml:
The name is misleading. The function is parsing + enriching the data.
converted to get_advisories_from_yml and get_advisories_from_md

Remove `"branch": None` in importer_yielder

Group imports

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
Hritik14 added a commit to Hritik14/vulnerablecode that referenced this issue Mar 30, 2021
@Hritik14
Refactor according to first review
ecb3f13 
Use batch_advisories for now. It has it's own problems and there's aboutcode-org#338 for that.
The generator thing won't do much, since we are importing like 10-20 MBs of data.

The codebase already has overuse of methods starting with _ , I'd say avoid them.
They don't help with readability nor are they trivial in this case

_parse_md and _parse_yml:
The name is misleading. The function is parsing + enriching the data.
converted to get_advisories_from_yml and get_advisories_from_md

Remove `"branch": None` in importer_yielder

Group imports

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
Hritik14 added a commit to Hritik14/vulnerablecode that referenced this issue Apr 1, 2021
@Hritik14
Refactor according to first review
f545012 
Use batch_advisories for now. It has it's own problems and there's aboutcode-org#338 for that.
The generator thing won't do much, since we are importing like 10-20 MBs of data.

The codebase already has overuse of methods starting with _ , I'd say avoid them.
They don't help with readability nor are they trivial in this case

_parse_md and _parse_yml:
The name is misleading. The function is parsing + enriching the data.
converted to get_advisories_from_yml and get_advisories_from_md

Remove `"branch": None` in importer_yielder

Group imports

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
@Hritik14 Hritik14 mentioned this issue Apr 1, 2021
Open
Hritik14 added a commit to Hritik14/vulnerablecode that referenced this issue Apr 1, 2021
@Hritik14
Refactor according to first review
701d7c6 
Use batch_advisories for now. It has it's own problems and there's aboutcode-org#338 for that.
The generator thing won't do much, since we are importing like 10-20 MBs of data.

The codebase already has overuse of methods starting with _ , I'd say avoid them.
They don't help with readability nor are they trivial in this case

_parse_md and _parse_yml:
The name is misleading. The function is parsing + enriching the data.
converted to get_advisories_from_yml and get_advisories_from_md

Remove `"branch": None` in importer_yielder

Group imports

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
Hritik14 added a commit to Hritik14/vulnerablecode that referenced this issue Apr 2, 2021
@Hritik14
Refactor according to first review
7aad267 
Use batch_advisories for now. It has it's own problems and there's aboutcode-org#338 for that.
The generator thing won't do much, since we are importing like 10-20 MBs of data.

The codebase already has overuse of methods starting with _ , I'd say avoid them.
They don't help with readability nor are they trivial in this case

_parse_md and _parse_yml:
The name is misleading. The function is parsing + enriching the data.
converted to get_advisories_from_yml and get_advisories_from_md

Remove `"branch": None` in importer_yielder

Group imports

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
@Hritik14 Hritik14 mentioned this issue Apr 15, 2021
Closed
@Hritik14 Hritik14 mentioned this issue Jun 25, 2021
Merged
@pombredanne
Copy link
Member

This is all fixed and merged now.... closing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pombredanne @sbs2001