feat: OBS-398 - netbox-diode-plugin: create api endpoint for applying change sets

[Julio-Oliveira-Encora]

Add an endpoint to apply changes (create and update) objects in the Netbox.

[linear]

OBS-398 Create API endpoint for applying change sets

In the NetBox diode plugin, we need to create an API endpoint for change sets to process by the plugin.

POST /api/plugins/netbox_diode_plugin/apply-change-set

Diode's reconciler service will make change sets based on ingested data and existing matching NetBox object states and send a request to this API endpoint.

Payload will contain a list of changes to action with following fields:

• change_id - [required] string containing change ID generated by diode backend during data ingestion
• change_type - [required] string containing operation type: create, update
• object_type - [required] string containing NetBox object type, i.e. device
• object_id - integer containing NetBox object ID for change types other than create (i.e. we need object ID to update or delete [in the future] existing objects)
• object_version - integer containing NetBox object's objectchange ID (see https://docs.netbox.dev/en/stable/features/change-logging/), similarly to object_id required for change types other than create
• data - [required] an object/dict containing properties corresponding to object_type NetBox model

Example payload:

{
    "change_set_id": "<UUID-0>",
    "change_set": [
        {"change_id": "<UUID-1>", "change_type": "create", "object_version": null, "object_type": "dcim.device", "object_id": null, "data": {"name": "router01", "device_type": "ISR4321", "device_role": "undefined", "site": "undefined", "platform": "IOS 15.6", "status": "Active", ...}},
    	{"change_id": "<UUID-2>", "change_type": "update", "object_version": 1, "object_type": "dcim.device", "object_id": 123, "data": {"name": "router01", "primary_ipv4": "mgmt", ...}},
	    {"change_id": "<UUID-3>", "change_type": "create", "object_version": null, "object_type": "dcim.site", "object_id": null, "data": {"name": "undefined", ...}},
    	{"change_id": "<UUID-4>", "change_type": "create", "object_version": null, "object_type": "dcim.device_role", "object_id": null, "data": {"name": "undefined", ...}},
	    {"change_id": "<UUID-5>", "change_type": "create", "object_version": null, "object_type": "dcim.platform", "object_id": null, "data": {"name": "IOS 15.6", ...}},
     	{"change_id": "<UUID-6>", "change_type": "create", "object_version": null, "object_type": "dcim.manufacturer", "object_id": null, "data": {"name": "Cisco", ...}},
	    {"change_id": "<UUID-7>", "change_type": "create", "object_version": null, "object_type": "dcim.device_type", "object_id": null, "data": {"model": "ISR4321", "manufacturer": "Cisco", ...}}
    ]
}

Once we receive the request, we should action given change sets based on the change_type, i.e:

{"change_id": "<UUID-7>", "change_type": "create", "object_version": null, "object_type": "dcim.device", "object_id": null, "data": {"name": "router01", "device_type": "ISR4321", "device_role": "undefined", "site": "undefined", "platform": "IOS 15.6", "status": "Active", ...}}

should create a new Device NetBox object using provided data.

We should use atomic transactions and attempt to commit all changes in a set or rollback on any failure. Example use of Django's built-in transaction.atomic functionality (docs):

• https://github.com/netbox-community/netbox/blob/develop/netbox/dcim/views.py#L115-L151

These change sets should be applied utilising existing corresponding NetBox models.

Note that each NetBox object type (model) might have some requirements, i.e. Device requires Site, DeviceTypeID and RoleID which we should create first as a placeholders prior to creation of Device object itself. It's not specified yet, but might provide these placeholders to create in the change set's data while being generated by reconciler service (TBC).

API endpoint should respond with with a success response or return an error.

Success:

{"change_set_id": "<UUID-0>", "result": "success"}

Error:

{"change_set_id": "<UUID-0>", "result": "failed", "errors": ["failed to create device..."]}

(response structure may change as we find it appropriate during development and testing).

This API should be unit tested appropriately.

[mfiedorowicz]

Thanks @Julio-Oliveira-Encora, a few comments to think about.

[mfiedorowicz]

Atomic transactions are working 🎉

A couple of comments on error handling/responses.

[mfiedorowicz]

Thanks @Julio-Oliveira-Encora, a few more suggestions/points to think and discuss if needed.