Collect xen #464

Pushpit07 · 2021-05-27T14:02:47Z

Fixes #103
Added code to import xen

Tried to make it run by modifying the importer_yielder.py and some other files but couldn't. The code is somewhat similar to the ubuntu_usn importer. Can't figure out why I'm getting an unexpected keyword argument

Pulling changes

Signed-off-by: Pushpit <pushpit07@gmail.com>

pombredanne

Thanks! this is looking great. One question though: this would only add new references to existing vulnerabilities and this based on the CVE?
Also we need to find out about the license for the data?

Also what could we do about XSA-377 released in a few days/weeks on 2021-06-08 17:00 per https://xenbits.xen.org/xsa/ ? (I guess just waiting may be enough?)

pombredanne · 2021-05-27T15:35:40Z

vulnerabilities/fixtures/openssl.json

+        "pk": 17,
+        "fields": {
+            "name": "xen",
+            "license": "",


Do you know what the license would be?

@Pushpit07

re: GPLv2

if this is it, then state it. Do you have a link to the license info?

https://wiki.xenproject.org/wiki/Xen_FAQ_General

Pushpit07 · 2021-05-28T11:18:09Z

Also what could we do about XSA-377 released in a few days/weeks on 2021-06-08 17:00 per https://xenbits.xen.org/xsa/ ? (I guess just waiting may be enough?)

Yes, we can wait

pombredanne · 2021-07-15T14:05:10Z

@Pushpit07 do think you want to finish this?
Other I can close for now and we can reopen later.

Pushpit07 · 2021-07-17T04:30:20Z

@Pushpit07 do think you want to finish this?
Other I can close for now and we can reopen later.

Sorry for leaving in between. Will surely finish this. Give me a few days time

pombredanne · 2021-07-21T13:40:44Z

Sorry for leaving in between. Will surely finish this. Give me a few days time

That's very nice of you! thank you ++

Signed-off-by: Pushpit <pushpit07@gmail.com>

pombredanne · 2021-07-26T17:03:28Z

@Pushpit07 Thanks! Think you missed a signoff on a few commits... do you mind to amend these?

helper for istio and mozilla importers Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

also, sort imports Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Better documentation and more readable function structrue review: aboutcode-org#443 (review) Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Internally, the difference between both has faded and updated_advisories is preferred. Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Earlier, one batch of advisories was requested from updated_advisories method of the respective importers. This was inefficient as not all importers respect batching internally. Eventually, we wish to eliminate batches as well ( # 338 ). Now, the updated_advisories method of each importer is expected to create at least one Advisory object. If it does so, the importer is marked working. This brings major performance improvement. It is a necessity to improve this test as GitHub only allows 6 hrs of workflow time. Before: ~6hrs, now ~9 minutes Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Signed-off-by: GitHub <noreply@github.com>

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Signed-off-by: Shivam Sandbhor <shivam@crowdsec.net>

Use uniform names and prototypes for `fetch` function. Remove unnecessary checks. Re-use sessions Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

The code for `load_api` was repetitive and used in all subclasses of VersionAPI, it is better suited in the parent class. `fetch` method is also now consistent and defined as an abstract method in the base class. Python ABC is not used as different implementations of `load_api` and `fetch` are allowed as done in DebianVersionAPI Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Signed-off-by: Shivam Sandbhor <shivam@crowdsec.net>

A minor bug was introduced by 775aa1d which was crashing the nginx importer. Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Recent time travel heuristics need published date of versions, thus Version dataclass was created. Some of the importers got bugged by this new system and were crashing. This fix is a part of # 467 Importers bugged: - nginx - debian_oval - ubuntu Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Signed-off-by: Pushpit <pushpit07@gmail.com>

…code into collect_xen Pulling changes Signed-off-by: Pushpit <pushpit07@gmail.com> Signing-off

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

pombredanne · 2022-02-08T10:35:19Z

@Pushpit07 before we can merge, please just confirm here that your DCO signoff "Signed-off-by: Pushpit pushpit07@gmail.com" applies to all your commits. Thank you!

Pushpit07 · 2022-02-08T10:38:02Z

Yes, it does

pombredanne · 2022-02-08T12:21:40Z

Yes, it does

@Pushpit07 Thank you ++ , you rock ❤️

pombredanne

@Hritik14 I think we should merge and this can be completed afterwards.

Hritik14 · 2022-02-08T12:24:24Z

@pombredanne Will do after fixing tests here

This is required to merge the PR to main for a clean slate for new contributions for importer-improver migration Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Pushpit07 added 3 commits April 28, 2021 20:19

Merge pull request #2 from nexB/main

c585f02

Pulling changes

Merge branch 'nexB:main' into main

a2e4487

Added code to import Xen

46aacf1

Signed-off-by: Pushpit <pushpit07@gmail.com>

pombredanne reviewed May 27, 2021

View reviewed changes

Merge branch 'main' into collect_xen

c5118c9

Pushpit07 added 3 commits July 25, 2021 21:17

Merge branch 'main' into collect_xen

3f776ff

Update importer_yielder.py

307ead3

Added xen license info

7b828ea

Signed-off-by: Pushpit <pushpit07@gmail.com>

Hritik14 and others added 17 commits July 27, 2021 12:02

helper: split_markdown_front_matter

73a884f

helper for istio and mozilla importers Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

black -l 100

a1752bd

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Use split_markdown_front_matter helper in istio

614e0f3

also, sort imports Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Use a more obvious version and reorder imports

d872763

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Update according to first review

f4ca861

Better documentation and more readable function structrue review: aboutcode-org#443 (review) Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Comment regarding raw docstring and sort imports

617700a

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Remove added_advisories for updated_advisories

0f6eece

Internally, the difference between both has faded and updated_advisories is preferred. Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

🙈 Ignore legacy pypi package versions in github importer

1f2cb59

Signed-off-by: GitHub <noreply@github.com>

Add basic implementation for time travel in imports

1f8ef6b

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Use web scraping to obtain github tags

e847303

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Time travel for githubtags, maven version api importers

ba9051d

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Fix tests

9842367

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Use class for returning versions partitioned by cutoff time

0dccf22

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Make review changes

f7dc3e6

Signed-off-by: Shivam Sandbhor <shivam@crowdsec.net>

Refactor package_manangers

72a84bc

Use uniform names and prototypes for `fetch` function. Remove unnecessary checks. Re-use sessions Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

sbs2001 and others added 7 commits July 27, 2021 12:02

Remove misleading frozen dataclass

80e3cf0

Signed-off-by: Shivam Sandbhor <shivam@crowdsec.net>

Fix npm importer date manipulation

9d359f2

Signed-off-by: Shivam Sandbhor <shivam@crowdsec.net>

Fix recursion in GitHubTagsAPI

047398e

A minor bug was introduced by 775aa1d which was crashing the nginx importer. Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Update importer_yielder.py

1f3fdef

Signed-off-by: Pushpit <pushpit07@gmail.com>

Added xen license info

5774649

Signed-off-by: Pushpit <pushpit07@gmail.com>

Merge branch 'collect_xen' of https://github.com/Pushpit07/vulnerable…

62c2b74

…code into collect_xen Pulling changes Signed-off-by: Pushpit <pushpit07@gmail.com> Signing-off

pombredanne added this to the v30.0 milestone Feb 2, 2022

Merge branch 'main' into collect_xen

bd18abe

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

pombredanne approved these changes Feb 8, 2022

View reviewed changes

Hritik14 added 2 commits February 9, 2022 01:13

Merge branch 'main' of github.com:nexb/vulnerablecode into collect_xen

8298b4c

Rename DataSource to Improver, blackify, ignore tests

6ef62fa

This is required to merge the PR to main for a clean slate for new contributions for importer-improver migration Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>

Hritik14 merged commit 8d66f64 into aboutcode-org:main Feb 8, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Collect xen #464

Collect xen #464

Pushpit07 commented May 27, 2021

pombredanne left a comment

pombredanne May 27, 2021

Pushpit07 May 28, 2021

pombredanne Jun 20, 2021

Pushpit07 Jul 25, 2021

Pushpit07 commented May 28, 2021

pombredanne commented Jul 15, 2021

Pushpit07 commented Jul 17, 2021

pombredanne commented Jul 21, 2021

pombredanne commented Jul 26, 2021

pombredanne commented Feb 8, 2022

Pushpit07 commented Feb 8, 2022

pombredanne commented Feb 8, 2022

pombredanne left a comment

Hritik14 commented Feb 8, 2022

Collect xen #464

Collect xen #464

Conversation

Pushpit07 commented May 27, 2021

pombredanne left a comment

Choose a reason for hiding this comment

pombredanne May 27, 2021

Choose a reason for hiding this comment

Pushpit07 May 28, 2021

Choose a reason for hiding this comment

pombredanne Jun 20, 2021

Choose a reason for hiding this comment

Pushpit07 Jul 25, 2021

Choose a reason for hiding this comment

Pushpit07 commented May 28, 2021

pombredanne commented Jul 15, 2021

Pushpit07 commented Jul 17, 2021

pombredanne commented Jul 21, 2021

pombredanne commented Jul 26, 2021

pombredanne commented Feb 8, 2022

Pushpit07 commented Feb 8, 2022

pombredanne commented Feb 8, 2022

pombredanne left a comment

Choose a reason for hiding this comment

Hritik14 commented Feb 8, 2022