Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add management command to watch all PURLs #268

Merged
merged 5 commits into from
Feb 5, 2024

Conversation

keshav-space
Copy link
Member

  • Management command to watch and index latest version of all the PURLs

- Management command to watch and index latest version of all the PURLs

Signed-off-by: Keshav Priyadarshi <git@keshav.space>
Copy link
Member

@pombredanne pombredanne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fine, but this is difficult to test as it looks at the whole DB at once.
If you add an option to focus the command on a PURL: this will make it easier to test even if we do not have the models yet.
See also a few minor nits.

packagedb/management/commands/watch_packages.py Outdated Show resolved Hide resolved
continue

version_class = VERSION_CLASS_BY_PACKAGE_TYPE.get(package.type)
latest_local = package.get_latest_version()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a next iteration, you should consider using some groupby and get all versions at once, or have a queryset that gets you the latest version automatically.

But really you likely want the missing versions rather than the latest versions.
It could be a refinement in a next iteration.
The cases I can fathom could be:

  1. get all the missing versions
  2. get the latest version (but we could have multiple ranges? ... so may be get all the latest versions in each "range"?)
  3. get all the newest versions after some version

try:
index_of_local_version = sorted_versions.index(latest_local_version)
except ValueError:
self.stdout.write(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have some history log entry to write in a next iteration?

packagedb/management/commands/watch_packages.py Outdated Show resolved Hide resolved
packagedb/management/commands/watch_packages.py Outdated Show resolved Hide resolved
Signed-off-by: Keshav Priyadarshi <git@keshav.space>
Signed-off-by: Keshav Priyadarshi <git@keshav.space>
@JonoYang
Copy link
Member

@pombredanne @keshav-space

Given that we mention watching purls, do we want this command to use the purl watch code from #271 ?

@pombredanne
Copy link
Member

@pombredanne @keshav-space

Given that we mention watching purls, do we want this command to use the purl watch code from #271 ?

@JonoYang I thought this was the case?

@JonoYang
Copy link
Member

@pombredanne @keshav-space
Given that we mention watching purls, do we want this command to use the purl watch code from #271 ?

@JonoYang I thought this was the case?

The logic in the management command looks similar to the function here: https://github.com/nexB/purldb/blob/244-watch-model/packagedb/tasks.py#L26

@keshav-space I'm just wondering if you were going to to update this management command to use the same code from the purl watch PR in #271 or after #271 has been merged

@keshav-space
Copy link
Member Author

@keshav-space I'm just wondering if you were going to to update this management command to use the same code from the purl watch PR in #271 or after #271 has been merged

@JonoYang This management command in it's current implementation will watch all the PURLs irrespective of whether or not the PURL is present in PackageWatch.
We can reuse the code from #271. But before that, we need to decide whether or not this management command should depend on PackageWatch model if it depends on PackageWatch model then this command will not watch all the packages.

Signed-off-by: Keshav Priyadarshi <git@keshav.space>
@keshav-space keshav-space force-pushed the 244-watch-all-management-command branch from 4012876 to a4b1484 Compare February 2, 2024 17:17
@JonoYang JonoYang merged commit 1ad3a01 into main Feb 5, 2024
7 checks passed
@JonoYang
Copy link
Member

JonoYang commented Feb 5, 2024

@keshav-space Thanks!

@JonoYang JonoYang deleted the 244-watch-all-management-command branch February 5, 2024 23:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants