Skip to content

Commit

Permalink
Final VSDK 3.0.0 docs and release changes delphix#311
Browse files Browse the repository at this point in the history
  • Loading branch information
jeffngo committed Jan 19, 2021
1 parent 6acfa72 commit 2b97795
Show file tree
Hide file tree
Showing 5 changed files with 22 additions and 15 deletions.
6 changes: 3 additions & 3 deletions docs/docs/Building_Your_First_Plugin/Data_Ingestion.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ quite limiting.

For our first plugin, we will be using the more flexible [staging](/References/Glossary.md#staged-linkingsyncing) strategy. With this strategy, the Delphix Engine uses NFS for Unix environments (or iSCSI on Windows environments) to mount storage onto a [staging environment](/References/Glossary.md#staging-environment). Our plugin will then be in full control of how to get data from the source environment onto this storage mount.

With the staging strategy, there are two types of syncs: sync and resync. A `sync` is used to ingestion incremental changes while a `resync` is used to re-ingest all the data for the dSource. For databases, this could mean re-ingesting from a full database backup to reset the dSource. A `sync` and a `resync` execute the same plugin operations and are differentiated by a boolean flag in the [snapshot_parameters](/References/Classes.md#snapshotparametersdefinition) argument passed into [linked.pre_snapshot](/References/Plugin_Operations.md#staged-linked-source-pre-snapshot) and [linked.post_snapshot](/References/Plugin_Operations.md#staged-linked-source-post-snapshot).
With the staging strategy, there are two types of syncs: sync and resync. A `sync` is used to ingestion incremental changes while a `resync` is used to re-ingest all the data for the dSource. For databases, this could mean re-ingesting from a full database backup to reset the dSource. A `sync` and a `resync` execute the same plugin operations and are differentiated by a boolean flag in the [snapshot parameters](/References/Glossary.md#snapshot-parameters) argument passed into [linked.pre_snapshot](/References/Plugin_Operations.md#staged-linked-source-pre-snapshot) and [linked.post_snapshot](/References/Plugin_Operations.md#staged-linked-source-post-snapshot).

A regular `sync` is the default and is executed as part of policy driven syncs. A `resync` is only executed during initial ingestion or if the Delphix user manually starts one. The customer can manually trigger a `resync` via the UI by selecting the dSource, going to more options and selecting **Resynchronize dSource**. ![Screenshot](images/Resync.png)

Expand Down Expand Up @@ -168,7 +168,7 @@ Next, we'll add a new function:

```python
@plugin.linked.pre_snapshot()
def copy_data_from_source(staged_source, repository, source_config, snapshot_parameters):
def copy_data_from_source(staged_source, repository, source_config, optional_snapshot_parameters):
stage_mount_path = staged_source.mount.mount_path
data_location = "{}@{}:{}".format(staged_source.parameters.username,
staged_source.parameters.source_address,
Expand Down Expand Up @@ -249,7 +249,7 @@ In fact, the default implementation that was generated by `dvp init` will work j
def linked_post_snapshot(staged_source,
repository,
source_config,
snapshot_parameters):
optional_snapshot_parameters):
return SnapshotDefinition()
```

Expand Down
6 changes: 3 additions & 3 deletions docs/docs/References/Classes.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,9 +154,9 @@ from dlpx.virtualization.platform import Plugin
plugin = Plugin()

@plugin.linked.pre_snapshot()
def linked_pre_snapshot(staged_source, repository, source_config, snapshot_parameters):
if snapshot_parameter.resync:
print(snapshot_parameter.resync)
def linked_pre_snapshot(staged_source, repository, source_config, optional_snapshot_parameters):
if optional_snapshot_parameters is not None and optional_snapshot_parameters.resync:
print(optional_snapshot_parameters.resync)
```

> This class will be generated during build and is located with the autogenerated classes. As it is passed into the operation, importing it is not neccessary.
Expand Down
14 changes: 7 additions & 7 deletions docs/docs/References/Plugin_Operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ Sets up a [dSource](Glossary.md#dsource) to ingest data. Only applies when using

### Signature

`def linked_pre_snapshot(staged_source, repository, source_config, snapshot_parameters)`
`def linked_pre_snapshot(staged_source, repository, source_config, optional_snapshot_parameters)`

### Decorator

Expand All @@ -290,7 +290,7 @@ Argument | Type | Description
staged_source | [StagedSource](Classes.md#stagedsource) | The source associated with this operation.
repository | [RepositoryDefinition](Schemas_and_Autogenerated_Classes.md#repositorydefinition-class) | The repository associated with this source.
source_config | [SourceConfigDefinition](Schemas_and_Autogenerated_Classes.md#sourceconfigdefinition-class) | The source config associated with this source.
snapshot_parameters | [SnapshotParametersDefinition](Classes.md#snapshotparametersdefinition) | The snapshot parameters.
optional_snapshot_parameters | [SnapshotParametersDefinition](Classes.md#snapshotparametersdefinition) | **Optional.** The snapshot parameters. The value is `None` when executed during a snapshot policy.

### Returns
None
Expand All @@ -303,7 +303,7 @@ from dlpx.virtualization.platform import Plugin
plugin = Plugin()

@plugin.linked.pre_snapshot()
def linked_pre_snapshot(staged_source, repository, source_config, snapshot_parameters):
def linked_pre_snapshot(staged_source, repository, source_config, optional_snapshot_parameters):
pass
```

Expand All @@ -320,7 +320,7 @@ Captures metadata from a [dSource](Glossary.md#dsource) once data has been inges

### Signature

`def linked_post_snapshot(staged_source, repository, source_config, snapshot_parameters)`
`def linked_post_snapshot(staged_source, repository, source_config, optional_snapshot_parameters)`

### Decorator

Expand All @@ -333,7 +333,7 @@ Argument | Type | Description
staged_source | [StagedSource](Classes.md#stagedsource) | The source associated with this operation.
repository | [RepositoryDefinition](Schemas_and_Autogenerated_Classes.md#repositorydefinition-class) | The repository associated with this source.
source_config | [SourceConfigDefinition](Schemas_and_Autogenerated_Classes.md#sourceconfigdefinition-class) | The source config associated with this source.
snapshot_parameters | [SnapshotParametersDefinition](Classes.md#snapshotparametersdefinition) | The snapshot parameters.
optional_snapshot_parameters | [SnapshotParametersDefinition](Classes.md#snapshotparametersdefinition) | **Optional.** The snapshot parameters. The value is `None` when executed during a snapshot policy.

### Returns
[SnapshotDefinition](Schemas_and_Autogenerated_Classes.md#snapshotdefinition-class)
Expand All @@ -347,9 +347,9 @@ from generated.definitions import SnapshotDefinition
plugin = Plugin()

@plugin.linked.post_snapshot()
def linked_post_snapshot(staged_source, repository, source_config, snapshot_parameters):
def linked_post_snapshot(staged_source, repository, source_config, optional_snapshot_parameters):
snapshot = SnapshotDefinition()
if snapshot_parameters.resync:
if optional_snapshot_parameters is not None and optional_snapshot_parameters.resync:
snapshot.transaction_id = 1000
else:
snapshot.transaction_id = 10
Expand Down
9 changes: 8 additions & 1 deletion docs/docs/Release_Notes/3.0.0/3.0.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,14 @@
To install or upgrade the SDK, refer to instructions [here](/Getting_Started.md#installation).

## New & Improved
* Added a `scratch_path` property on the [RemoteHost](/References/Classes/#remotehost) object for storage and debugging purposes.
* Added the ability to define snapshot parameters in a [Snapshot Parameters Definition](/References/Schemas_and_Autogenerated_Classes.md#snapshotparametersdefinition-schema).
* Provide end-users with configurable options prior to taking a snapshot.
* The options selected are provided as input to pre/post-snapshot functions.

* Added a `scratch_path` property on the [RemoteHost](/References/Classes/#remotehost) object which can be used to:
* Store small amounts of persistent data.
* Mount VDB data.
* Keep temporarily logs for debugging.
More details about `scratch_path` can be found [here](/Best_Practices/Scratch_Paths.md)

## Breaking Changes
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/Release_Notes/3.0.0/3.0.0_Breaking_Changes.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Breaking Changes - v.3.0.0

## New required schema
[Snapshot Parameters Definition](/References/Schemas_and_Autogenerated_Classes.md#snapshotparametersdefinition-schema) allows plugin authors to define [Snapshot Parameters] which can be displayed to an end-user whenever a linked source snapshot is taken.
[Snapshot Parameters Definition](/References/Schemas_and_Autogenerated_Classes.md#snapshotparametersdefinition-schema) allows plugin authors to define [Snapshot Parameters](/References/Glossary.md#snapshot-parameters) which can be displayed to an end-user whenever a linked source snapshot is taken.

### What is affected
All plugins built with v2.0.0 or below will be affected. The [Schema](/References/Schemas) must contain a `snapshotParametersDefinition`.
Expand Down

0 comments on commit 2b97795

Please sign in to comment.