Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add client for SystemLink jobs API #85

Open
wants to merge 23 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
c2278f1
feat: add systems client code for get jobs API.
ganesh-nithin Nov 27, 2024
cbba5c7
feat: add systems client code for create job API.
ganesh-nithin Nov 27, 2024
258f373
feat: add systems client code for get jobs summary API.
ganesh-nithin Nov 27, 2024
4e22c28
feat: add systems client code for query jobs API.
ganesh-nithin Nov 27, 2024
6070c71
feat: add systems client code for cancel jobs API.
ganesh-nithin Nov 27, 2024
55684d8
refactor: modify docs strings for systems client.
ganesh-nithin Nov 27, 2024
412c8ef
feat: add unit tests for jobs in systems client/
ganesh-nithin Nov 29, 2024
2f91a6c
refactor: change name systems into system.
ganesh-nithin Nov 29, 2024
63083f5
feat: add example documentation for system client.
ganesh-nithin Nov 29, 2024
5a4f0ae
feat: add api references for system client API.
ganesh-nithin Nov 29, 2024
1e17b89
refactor: format code using black & flake8.
ganesh-nithin Dec 1, 2024
13accda
refactor: address system client PR comments.
ganesh-nithin Dec 3, 2024
5dd2381
refactor: modify system client tests.
ganesh-nithin Dec 3, 2024
4a4b977
refactor: fix lint issues in CI.
ganesh-nithin Dec 3, 2024
814f1fb
refactor: modify model names.
ganesh-nithin Dec 4, 2024
85cc103
refactor: modify test cases for system client.
ganesh-nithin Dec 4, 2024
3ef21ec
refactor: modify field names to be pythonic.
ganesh-nithin Dec 6, 2024
6167da2
refactor: modify tests.
ganesh-nithin Dec 9, 2024
0e33a87
refactor: add wrapping for job client.
ganesh-nithin Dec 10, 2024
5575f20
refactor: modify clients & tests.
ganesh-nithin Dec 11, 2024
ee1d908
refactor: format system client code.
ganesh-nithin Dec 11, 2024
3e727c8
refactor: modify union operator.
ganesh-nithin Dec 11, 2024
6ebb8f1
refactor: format system client code.
ganesh-nithin Dec 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/api_reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ API Reference
api_reference/dataframe
api_reference/spec
api_reference/file
api_reference/system

Indices and tables
------------------
Expand Down
18 changes: 18 additions & 0 deletions docs/api_reference/system.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
.. _api_tag_page:

nisystemlink.clients.system
======================

.. autoclass:: nisystemlink.clients.system.SystemClient
:exclude-members: __init__

.. automethod:: __init__
.. automethod:: list_jobs
.. automethod:: create_job
.. automethod:: get_job_summary
.. automethod:: query_jobs
.. automethod:: cancel_jobs

.. automodule:: nisystemlink.clients.system.models
:members:
:imported-members:
28 changes: 28 additions & 0 deletions docs/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -210,5 +210,33 @@ Examples
Get the metadata of a File using its Id and download it.

.. literalinclude:: ../examples/file/download_file.py
:language: python
:linenos:


System API
-------

Overview
~~~~~~~~

The :class:`.SystemClient` class is the primary entry point of the System API.

When constructing a :class:`.SystemClient`, you can pass an
:class:`.HttpConfiguration` (like one retrieved from the
:class:`.HttpConfigurationManager`), or let :class:`.SystemClient` use the
default connection. The default connection depends on your environment.

With a :class:`.SystemClient` object, you can:

* Create, get, query, and cancel jobs
* Get the summary of a jobs

Examples
~~~~~~~~

Create, get, query, and cancel jobs

.. literalinclude:: ../examples/system/system.py
:language: python
:linenos:
63 changes: 63 additions & 0 deletions examples/system/job.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
from nisystemlink.clients.core import HttpConfiguration
from nisystemlink.clients.system import SystemClient
from nisystemlink.clients.system.models import (
CancelJobRequest,
CreateJobRequest,
JobField,
JobState,
QueryJobsRequest,
)

# Setup the server configuration to point to your instance of SystemLink Enterprise
server_configuration = HttpConfiguration(
server_uri="https://yourserver.yourcompany.com",
api_key="YourAPIKeyGeneratedFromSystemLink",
)
client = SystemClient(configuration=server_configuration)

# Get all jobs that have succeeded
jobs = client.list_jobs(
system_id="system_id",
job_id="jid",
state=JobState.SUCCEEDED,
function="function",
skip=0,
take=10,
)

# Create a job
arguments = [["A description"]]
target_systems = [
"HVM_domU--SN-ec200972-eeca-062e-5bf5-017a25451b39--MAC-0A-E1-20-D6-96-2B"
]
functions = ["system.set_computer_desc"]
metadata = {"queued": True, "refresh_minion_cache": {"grains": True}}
job = CreateJobRequest(
arguments=arguments,
target_systems=target_systems,
functions=functions,
metadata=metadata,
)

create_job_response = client.create_job(job)

# Get job summary
job_summary = client.get_job_summary()

# Query jobs
query_job = QueryJobsRequest(
skip=0,
take=1000,
filter="result.success.Contains(false)",
projection=[JobField.ID, JobField.SYSTEM_ID, "metadata.queued"],
orderBy=JobField.CREATED_TIMESTAMP,
descending=True,
)
query_jobs_response = client.query_jobs(query_job)


# Cancel a job
cancel_job_request = CancelJobRequest(
id=create_job_response.id, system_id=target_systems[0]
)
cancel_job_response = client.cancel_jobs([cancel_job_request])
1 change: 1 addition & 0 deletions nisystemlink/clients/core/_uplink/_methods.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ def get(path: str, args: Optional[Sequence[Any]] = None) -> Callable[[F], F]:
"""Annotation for a GET request."""

def decorator(func: F) -> F:
print("get decorator", func)
return commands.get(path, args=args)(func) # type: ignore

return decorator
Expand Down
3 changes: 3 additions & 0 deletions nisystemlink/clients/system/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
from ._system_client import SystemClient

# flake8: noqa
194 changes: 194 additions & 0 deletions nisystemlink/clients/system/_system_client.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,194 @@
import json
from typing import List, Optional, Union

from nisystemlink.clients import core
from nisystemlink.clients.core import ApiError
from nisystemlink.clients.core._uplink._base_client import BaseClient
from nisystemlink.clients.core._uplink._methods import get, post, response_handler
from requests.models import Response
from uplink import Query

from . import models


def _cancel_job_response_handler(response: Response) -> Union[ApiError, None]:
"""Response handler for Cancel Job response."""
if response is None:
return None

try:
cancel_response = response.json()
except json.JSONDecodeError:
return None

return cancel_response.get("error")


def _list_jobs_response_handler(response: Response) -> List[models.Job]:
"""Response handler for List Jobs response."""
if response is None:
return []

jobs = response.json()

return jobs


class SystemClient(BaseClient):
def __init__(self, configuration: Optional[core.HttpConfiguration] = None):
"""Initialize an instance.

Args:
configuration: Defines the web server to connect to and information about
how to connect. If not provided, the
:class:`HttpConfigurationManager <nisystemlink.clients.core.HttpConfigurationManager>`
is used to obtain the configuration.

Raises:
ApiException: if unable to communicate with the ``/nisysmgmt`` Service.
"""
if configuration is None:
configuration = core.HttpConfigurationManager.get_configuration()

super().__init__(configuration, "/nisysmgmt/v1/")

@response_handler(_list_jobs_response_handler)
@get(
"jobs",
args=[
Query("systemId"),
Query("jid"),
Query("state"),
Query("function"),
Query("skip"),
Query("take"),
],
)
def list_jobs(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def list_jobs(
def list_jobs_paged(

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skipping the suggestion as list jobs are not paged as it don't have continuation token property.

self,
system_id: Optional[str] = None,
job_id: Optional[str] = None,
state: Optional[models.JobState] = None,
function: Optional[str] = None,
skip: Optional[int] = None,
take: Optional[int] = None,
) -> List[models.Job]:
"""List the jobs that matched the criteria.

Args:
system_id: The ID of the system to that the jobs belong.
jid: The ID of the job.
state: The state of the jobs.
function: The salt function to run on the client.
skip: The number of jobs to skip.
take: The number of jobs to return.

Returns:
The list of jobs that matched the criteria.

Raises:
ApiException: if unable to communicate with the ``/nisysmgmt`` Service
or provided an invalid argument.
"""
...

@post("jobs")
def create_job(self, job: models.CreateJobRequest) -> models.CreateJobResponse:
"""Create a job.

Args:
job: The request to create the job.

Returns:
The job that was created.

Raises:
ApiException: if unable to communicate with the ``/nisysmgmt`` Service
or provided an invalid argument.
"""
...

@get("get-jobs-summary")
def get_job_summary(self) -> models.JobSummaryResponse:
"""Get a summary of the jobs.

Returns:
An instance of a JobsSummaryResponse.

Raises:
ApiException: if unable to communicate with the ``/nisysmgmt`` Service
or provided an invalid argument.
"""
...

@post("query-jobs")
def _query_jobs(self, query: models._QueryJobsRequest) -> models.QueryJobsResponse:
"""Query the jobs.

Args:
query: The request to query the jobs.

Returns:
An instance of QueryJobsRequest.

Raises:
ApiException: if unable to communicate with the ``/nisysmgmt`` Service
or provided an invalid argument.
"""
...

def query_jobs(self, query: models.QueryJobsRequest) -> models.QueryJobsResponse:
"""Query the jobs.

Args:
query: The request to query the jobs.

Returns:
An instance of QueryJobsRequest.

Raises:
ApiException: if unable to communicate with the ``/nisysmgmt`` Service
or provided an invalid argument.
"""
projection = f"new({','.join(query.projection)})" if query.projection else None

order_by = (
f"{query.order_by.value} {'descending' if query.descending else 'ascending'}"
if query.order_by
else None
)

query_params = {
"skip": query.skip,
"take": query.take,
"filter": query.filter,
"projection": projection,
"order_by": order_by,
}

# Clean the query_params to remove any keys with None values
query_params = {k: v for k, v in query_params.items() if v is not None}

# Create the query request with the cleaned parameters
query_request = models._QueryJobsRequest(**query_params)

return self._query_jobs(query_request)

@response_handler(_cancel_job_response_handler)
@post("cancel-jobs")
def cancel_jobs(
self, job_ids: List[models.CancelJobRequest]
) -> Union[ApiError, None]:
"""Cancel the jobs.

Args:
job_ids: List of CancelJobRequest.

Returns:
The errors that appear while attempting the operation.

Raises:
ApiException: if unable to communicate with the ``/nisysmgmt`` Service
or provided an invalid argument.
"""
...
9 changes: 9 additions & 0 deletions nisystemlink/clients/system/models/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
from ._job import Job, JobState, JobConfig, JobResult
from ._create_job_request import CreateJobRequest
from ._create_job_response import CreateJobResponse
from ._job_summary_response import JobSummaryResponse
from ._query_jobs_request import QueryJobsRequest, _QueryJobsRequest, JobField
from ._query_jobs_response import QueryJobsResponse, QueryJob, QueryJobConfig
from ._cancel_job_request import CancelJobRequest

# flake8: noqa
12 changes: 12 additions & 0 deletions nisystemlink/clients/system/models/_cancel_job_request.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
from nisystemlink.clients.core._uplink._json_model import JsonModel
from pydantic import Field


class CancelJobRequest(JsonModel):
"""Model for cancel job request."""

id: str = Field(alias="jid")
"""The ID of the job to cancel."""

system_id: str
"""The system ID that the job to cancel targets."""
20 changes: 20 additions & 0 deletions nisystemlink/clients/system/models/_create_job_request.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
from typing import Any, Dict, List, Optional

from nisystemlink.clients.core._uplink._json_model import JsonModel
from pydantic import Field


class CreateJobRequest(JsonModel):
"""Model for create job request."""

arguments: Optional[List[List[Any]]] = Field(None, alias="arg")
"""List of arguments to the functions specified in the "fun" property."""

target_systems: List[str] = Field(alias="tgt")
"""List of system IDs on which to run the job."""

functions: List[str] = Field(alias="fun")
"""Functions contained in the job."""

metadata: Optional[Dict[str, Any]] = None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update it with actual properties..

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated it. As CreateJobResponse is extended from CreateJobRequest, properties will be updated in it.

"""Additional information of the job."""
16 changes: 16 additions & 0 deletions nisystemlink/clients/system/models/_create_job_response.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
from typing import Optional

from nisystemlink.clients.core import ApiError
from pydantic import Field

from ._create_job_request import CreateJobRequest


class CreateJobResponse(CreateJobRequest):
"""Model for response of create job request."""

error: Optional[ApiError] = None
"""Represents the standard error structure."""

id: str = Field(alias="jid")
"""The job ID."""
Loading
Loading