List of NetAPP SDK calls and functionalty for the DB On Demand use case

[icot]

I will mention both the SDK object/method used and the context in which they are used currently (middle level methods as currently used in the current version of the code).

The currently used version of the Netapp SDK is 5.3.1

[icot]

Each section maps to the high level methods of our internal ZAPI call module.
Inside each section you can find the internal method calls for the SOAP objects (NaServer, NaElement) provided by the SDK.
If you want to just take a look at the code, please check the source here. I have not looked at the Python SDK but I'm guessing the calls should transalte 1-1.

[icot]

Snapshot related functionality

I'm writing the SDK calls in a direct Python translation of the original Perl SDK calls, assuming the interface would be the same or similar ...
Each high level method should be exposed as an API endpoint.

snap_create

NaElement.invoke('snapshot-create', 'volume', volume_name, 'snapshot', snapshot_name)

snap_prepare

This method should be integrated in the creation action and actually transparent to the end-user. The objective is to check whether or not it is possible to create a new snapshot depending on the maximum number of snapshots permitted for a volume (254) and if the answer is negative
because the maximum has been reached; to delete the oldest snapshot to make space for the new one before attempting the actual creation call

snap_delete

NaElement.invoke('snapshot-delete', 'volume', volume_name, 'snapshot', snapshot_name)

snap_restore

NaElement.invoke('snapshot-restore-volume', 'volume', volume_name, 'snapshot', snapshot_name)

snap_list

buf = NaElement.invoke('snapshot-list-info', 'volume', volume_name, 'snapshot', snapshot_name)

snapshot_list = buf.child_get('snapshots')

for snapshot in snapshot_list:
    snapshot.child_get_int('access-time', 0)
    snapshot.child_get_int('cumulative-total', 0)    
    snapshot.child_get_string(busy')
    snapshot.child_get_string('dependency')

snap_clone

nae = NaElement.new('volume-clone-create')
nae.child_add_string('junction-active', 'true')
nae.child_add_string('junction-path', newjunction)
nae.child_add_string('parent-snapshot', snapshot)
nae.child_add_string('parent-volume', volume)
nae.child_add_string('volume', new_volume)
NaServer.invoke_elem(nae)

[icot]

The following functionality is currently not implemented as part of the DB On Demand framework, but exploited through external projects (DFM) and should be made available through the API. I will group them as volume related methods:

vol_create

Creates a volume. The current implementation allows for profiles or packs, where volumes for a specific type of a database can be requested requiring a certain size for the data volume and the size of the WAL logs volume will be computed accordingly to the database size. That functionality should
be left on the application side (in my opinion)

vol_restrict

Restrict access to the volume prior to supervised deletion

vol_break_locks

Removes locks on volume

vol_info

Gets volume information such as filer ip address, mounting path, etc.

[icot]

ACL modification functionality

This is not currently available anywhere. It would be useful to be able to add/remove hosts from the ACL definitions

[icot]

@AlbinS Do you need additional information to get going?

[amandasystems]

@icot I think I'm good for now, but I'm certain more questions will pop up as I go. The ACL bit might need a bit of clarification, but I'd say that's probably not part of a minimal implementation, but rather a future extension, given that you don't even need it right now. Baby steps!

Also, authentication -- we spoke about not using SSO, but if not -- what should we be using in stead? Also, should/must there be granular access control, or is this a case of "access to the API -> access to everything"?

[amandasystems]

@icot Ok, so the API uses volume names and snapshot names to identify a snapshot. Is that what we want to be using in the API as well? Also, what types of forms do they take -- arbitrary strings, paths, ...?

[icot]

The ACL is truly the least priority item on the list yes.

Regarding authentication, in the end it is a matter of how much you want to implement access control at the API level versus how much you can delegate to the filers. What I mean by this is the following:

• If the filers support for instance creating an account which you can then limit to have access only to a subset of resources/functionality then this account credentials could be used as basic auth credentials for the API
• If you only can have full access admin accounts on the filers, then we would need to implement authentication restrictions at the API level, and use of SSO (Oauth) would be probably unavoidable as you would also want the e-groups information to limit the access

Regarding volume and snapshot names, AFAIK both can be arbitrary strings
which in our case follow some conventions (linked to database name, volume intended use and
mounting path for the volume name and to the timestamp and database version for the snapshots).
I think the API should be as generic as possible and leave convention to the application/user.
If you would prefer to use ids instead of names, we would adapt to that.

[gasparuben]

Hi guys, just to comment, the ACL functionality is implemented in the RulesREST.py endpoint. If you refer by that to the possibility to add/remove IPs to an existing access policy. The creation is part of the vol create. I put it there and not as a added functionality of the RulesREST because a policy should be always part of a volume or set of volumes.

[icot]

What @gasparuben mentioned is exactly what we need.

[amandasystems]

I would prefer not to use IDs, as that would probably require the API to keep an internal mapping of id->resources, and that is unnecessary added complexity if it can be avoided.

That said, I'm leaning more and more towards keeping the existing authentication implementation for the API itself. SSO may be messy and inconvenient for client/server applications, but hopefully I should be able to provide a nice Python wrapper (and a Perl one already exists, right?), so most pain points should be covered.

[amandasystems]

Speaking of IDs, @gasparuben, what was the reason behind the desicion to base64-encode resources/arguments?