Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PC-9915: Change SDK objects API #90

Merged
merged 57 commits into from
Aug 11, 2023
Merged

Conversation

nieomylnieja
Copy link
Collaborator

@nieomylnieja nieomylnieja commented Aug 2, 2023

Premise

Operating on a generic map[string]interface{} doesn't lend to a great devx, If you want to define your objects in code and then use sdk.Client to apply them, you'd need to convert them somehow to sdk.AnyJSONObject (most likely through a round trip for marshaling).
Furthermore the intermediate representation of GenericObject along with APIObjects model was initially created to serve Ingest's purposes, and does not fit well into standard API user workflow.
Objects often shared code, like Metadata and only used parts of its definition (like only name and project), which was introducing ambiguity as to what exactly does the object definition permit to be defined.

What's changed?

Manifest

  • manifest package is now only defining the absolute core of our schema, things we never expect to change and take for granted. It defines Object interface which must be implemented by all objects across all API versions. It's the building block we can operate on at the highest level of abstraction.

v1alpha

  • Objects no longer share or inline commonly used structs like Metadata or ObjectHeader, instead they explicitly list the fields and define their own flavours of metadata
  • Objects implementation of the manifest.Object, manifest.ProjectScopedObject and v1alpha.ObjectContext interfaces is auto generated to reduce boilerplate work, the generator script is in-house and can be adjusted over time (scripts/generate-object-impl.go. The auto generated code lives under <object_filename>_object.go.
  • APIObjects has been moved to Ingest and will be eventually removed entirely in favour of []manifest.Object while genericToXXX methods were removed. The parsing function located at parser.go now uses generics to decode each objects into its concrete implementation
  • OrganizationInformation definition has been moved to Ingest

Client

  • sdk.Client operates on manifest.Object instead of the old map[string]interface{}, this means it both accepts and returns parsed manifest.Object
  • sdk.Client now sets organization automatically for all the objects it accepts in apply and delete methods

Definitions

  • definitions package now handles the decoding process on its own, it no longer relies on a bloated k8s yaml import. It is able to decode both yaml and json formats into specific manifest.Object. I used https://github.com/goccy/go-yaml as it offers much greater flexibility in decoding then https://github.com/go-yaml/yaml.
  • definitions package no longer requires implicit MetdataAnnotations and it only annotates the objects with manifest source, the Project is set by the SDK users (like sloctl) and Organization is set by sdk.Client

What's to come

  • switch to manifest.Version enum in objects definitions
  • remove deprecated objects, like Metadata
  • move unnecessarily exported things back to n9
  • further polish the API
  • split validator for each object (and eventually move away from it entirely)
  • move objects to separate packages (TBD?)

@nieomylnieja nieomylnieja force-pushed the PC-9915-change-sdk-objects-api branch from 229127d to 022c632 Compare August 2, 2023 14:38
Copy link
Contributor

@kskitek kskitek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As agreed, I only took a general look without testing or trying to use the client. But I like the new shape! It is simple to understand and use.

I left a couple of minor comments and a few questions mainly to think about / discuss rather than change.

.github/workflows/vulns.yml Outdated Show resolved Hide resolved
manifest/v1alpha/agent.go Show resolved Hide resolved
manifest/v1alpha/agent.go Show resolved Hide resolved
manifest/v1alpha/agent.go Show resolved Hide resolved
manifest/v1alpha/alert_method.go Show resolved Hide resolved
sdk/definitions/parser.go Show resolved Hide resolved
sdk/definitions/parser.go Show resolved Hide resolved
sdk/definitions/reader.go Show resolved Hide resolved
manifest/v1alpha/reports.go Show resolved Hide resolved
manifest/v1alpha/ts.go Show resolved Hide resolved
@nieomylnieja nieomylnieja force-pushed the PC-9915-change-sdk-objects-api branch from 3c7e821 to acbea63 Compare August 7, 2023 14:30
@nieomylnieja nieomylnieja force-pushed the PC-9915-change-sdk-objects-api branch from acbea63 to 08a490a Compare August 7, 2023 14:35
@nieomylnieja nieomylnieja force-pushed the PC-9915-change-sdk-objects-api branch from 5592074 to 786ff36 Compare August 8, 2023 10:37
@nieomylnieja nieomylnieja mentioned this pull request Aug 8, 2023
manifest/object.go Outdated Show resolved Hide resolved
manifest/v1alpha/alert.go Show resolved Hide resolved
manifest/v1alpha/models.go Outdated Show resolved Hide resolved
nieomylnieja and others added 2 commits August 9, 2023 09:48
Co-authored-by: krolik <szymon@nobl9.com>
Copy link
Member

@skrolikiewicz skrolikiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🍊

@nieomylnieja nieomylnieja merged commit dae5329 into main Aug 11, 2023
3 checks passed
@nieomylnieja nieomylnieja deleted the PC-9915-change-sdk-objects-api branch August 11, 2023 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants