Files rescan and pivot #121
Conversation
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Reviewer's Guide by SourceryThis pull request implements a rescan functionality for jobs in the ThreatMatrix project. It adds a new API endpoint for rescanning jobs, updates the frontend to support this feature, and includes corresponding test cases. The changes span across backend (Django) and frontend (React) components, with modifications to API views, serializers, React components, and test files. Sequence diagram for job rescan processsequenceDiagram
actor User
participant Frontend
participant Backend
participant Database
User->>Frontend: Click 'Rescan' button
Frontend->>Backend: POST /api/jobs/{jobId}/rescan
Backend->>Database: Retrieve existing job details
alt Job is observable
Backend->>Database: Create new observable job
else Job is sample
Backend->>Database: Create new sample job
end
Database-->>Backend: Return new job ID
Backend-->>Frontend: Respond with new job ID
Frontend-->>User: Display success message with new job ID
Updated class diagram for Job and related serializersclassDiagram
class Job {
+String tlp
+String observable_name
+String observable_classification
+String status
+DateTime finished_analysis_time
+Map runtime_configuration
+Boolean is_sample
+File file
+String file_name
+PlaybookConfig playbook_requested
+Set analyzers_requested
+Set connectors_requested
}
class ObservableAnalysisSerializer {
+validate()
+save()
}
class FileJobSerializer {
+validate()
+save()
}
Job <|-- ObservableAnalysisSerializer
Job <|-- FileJobSerializer
Job : +rescan()
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
Here's the code health analysis summary for commits Analysis Summary
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Explore these optional code suggestions:
|
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
![qodo-merge-pro[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
| @action(detail=True, methods=["post"]) | ||
| def rescan(self, request, pk=None): | ||
| logger.info(f"rescan request for job: {pk}") | ||
| existing_job: Job = self.get_object() |
There was a problem hiding this comment.
Suggestion: Add a permission check before allowing job rescan [Security, importance: 8]
| @action(detail=True, methods=["post"]) | |
| def rescan(self, request, pk=None): | |
| logger.info(f"rescan request for job: {pk}") | |
| existing_job: Job = self.get_object() | |
| @action(detail=True, methods=["post"]) | |
| def rescan(self, request, pk=None): | |
| logger.info(f"rescan request for job: {pk}") | |
| existing_job: Job = self.get_object() | |
| if not request.user.has_perm('api_app.rescan_job', existing_job): | |
| raise PermissionDenied("You do not have permission to rescan this job.") |
![codiumai-pr-agent-free[bot]](https://avatars.githubusercontent.com/in/354216?s=60&v=4){kind=small}
| @action(detail=True, methods=["post"]) | ||
| def rescan(self, request, pk=None): | ||
| logger.info(f"rescan request for job: {pk}") | ||
| existing_job: Job = self.get_object() |
There was a problem hiding this comment.
Suggestion: Add a permission check before allowing job rescan [Security, importance: 8]
| @action(detail=True, methods=["post"]) | |
| def rescan(self, request, pk=None): | |
| logger.info(f"rescan request for job: {pk}") | |
| existing_job: Job = self.get_object() | |
| @action(detail=True, methods=["post"]) | |
| def rescan(self, request, pk=None): | |
| logger.info(f"rescan request for job: {pk}") | |
| existing_job: Job = self.get_object() | |
| if not request.user.has_perm('api_app.rescan_job', existing_job): | |
| raise PermissionDenied("You do not have permission to rescan this job.") |
| job_serializer.is_valid(raise_exception=True) | ||
| new_job = job_serializer.save(send_task=True) |
There was a problem hiding this comment.
Suggestion: Add error handling for job serializer operations [Error handling, importance: 7]
| job_serializer.is_valid(raise_exception=True) | |
| new_job = job_serializer.save(send_task=True) | |
| try: | |
| job_serializer.is_valid(raise_exception=True) | |
| new_job = job_serializer.save(send_task=True) | |
| except serializers.ValidationError as e: | |
| logger.error(f"Validation error during job rescan: {e}") | |
| raise ValidationError(f"Failed to create new job: {e}") | |
| except Exception as e: | |
| logger.error(f"Unexpected error during job rescan: {e}") | |
| raise ValidationError("An unexpected error occurred while creating the new job.") |
| job_serializer.is_valid(raise_exception=True) | ||
| new_job = job_serializer.save(send_task=True) |
There was a problem hiding this comment.
Suggestion: Add error handling for job serializer operations [Error handling, importance: 7]
| job_serializer.is_valid(raise_exception=True) | |
| new_job = job_serializer.save(send_task=True) | |
| try: | |
| job_serializer.is_valid(raise_exception=True) | |
| new_job = job_serializer.save(send_task=True) | |
| except serializers.ValidationError as e: | |
| logger.error(f"Validation error during job rescan: {e}") | |
| raise ValidationError(f"Failed to create new job: {e}") | |
| except Exception as e: | |
| logger.error(f"Unexpected error during job rescan: {e}") | |
| raise ValidationError("An unexpected error occurred while creating the new job.") |
User description
(Please add to the PR name the issue/s that this PR would close if merged by using a Github keyword. Example:
<feature name>. Closes #999. If your PR is made by a single commit, please add that clause in the commit too. This is all required to automate the closure of related issues.)Description
Please include a summary of the change and link to the related issue.
Type of change
Please delete options that are not relevant.
Checklist
developdumpplugincommand and added it in the project as a data migration. ("How to share a plugin with the community")test_files.zipand you added the default tests for that mimetype in test_classes.py.FREE_TO_USE_ANALYZERSplaybook by following this guide.urlthat contains this information. This is required for Health Checks._monkeypatch()was used in its class to apply the necessary decorators.MockUpResponseof the_monkeypatch()method. This serves us to provide a valid sample for testing.Black,Flake,Isort) gave 0 errors. If you have correctly installed pre-commit, it does these checks and adjustments on your behalf.testsfolder). All the tests (new and old ones) gave 0 errors.DeepSource,Django Doctorsor other third-party linters have triggered any alerts during the CI checks, I have solved those alerts.Important Rules
PR Type
enhancement, tests
Description
rescanfeature in the Job viewset, allowing jobs to be rescanned with updated configurations.rescanfeature, including UI updates and API integration.rescanfeature, covering various scenarios and permissions..prettierignorefile to exclude coverage artifacts from formatting checks.Changes walkthrough 📝
7 files
job.py
Add `is_sample` field to Job serializerapi_app/serializers/job.py
is_samplefield to the Job serializer.views.py
Implement rescan action in Job viewsetapi_app/views.py
ScanModefromapi_app.choices.rescanaction to Job viewset.rescanaction.utils.js
Include `is_sample` property in job nodefrontend/src/components/investigations/flow/utils.js
is_sampleproperty to job node.CustomJobNode.jsx
Update pivot button for sample analysisfrontend/src/components/investigations/flow/CustomJobNode.jsx
isSampleparameter.JobActionBar.jsx
Refactor retry logic to use rescanJobfrontend/src/components/jobs/result/bar/JobActionBar.jsx
rescanJobfunction.jobApi.jsx
Add rescanJob function for job rescanningfrontend/src/components/jobs/result/jobApi.jsx
rescanJobfunction to handle job rescanning.ScanForm.jsx
Handle isSample parameter and refactor observable selectionfrontend/src/components/scan/ScanForm.jsx
isSampleparameter in URL.5 files
test_api.py
Add tests for Job rescan featuretests/api_app/test_api.py
rescanfeature.observables.test.js
Extend invalid domain test casesfrontend/tests/utils/observables.test.js
InvestigationFlow.test.jsx
Add tests for sample job nodes in InvestigationFlowfrontend/tests/components/investigations/flow/InvestigationFlow.test.jsx
is_sampleproperty in job nodes.JobActionBar.test.jsx
Update tests for rescanJob API callsfrontend/tests/components/jobs/result/utils/JobActionBar.test.jsx
rescanJobAPI calls.ScanForm.advanced.test.jsx
Test handling of isSample parameter in ScanFormfrontend/tests/components/scan/ScanForm/ScanForm.advanced.test.jsx
isSampleparameter in ScanForm.1 files
.prettierignore
Add .prettierignore file for coverage artifactsfrontend/.prettierignore
.prettierignorefile to ignore coverage artifacts.Summary by Sourcery
Implement a 'rescan' feature for jobs, allowing users to re-analyze jobs with the same configuration. Update the frontend to support this feature and add corresponding tests to ensure functionality.
New Features:
Enhancements:
Tests: