Skip to content
This repository has been archived by the owner on Jan 28, 2021. It is now read-only.

Commit

Permalink
#103 - support multiple properties, and also blob and clob
Browse files Browse the repository at this point in the history
  • Loading branch information
danhaywood committed Jan 16, 2019
1 parent 7fab8f8 commit 59ccefc
Show file tree
Hide file tree
Showing 37 changed files with 953 additions and 476 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
114 changes: 47 additions & 67 deletions adocs/documentation/src/main/asciidoc/modules/lib/minio/lib-minio.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -28,39 +28,39 @@ image::{_imagesdir}/design-sketch.png[width="800px",link="{_imagesdir}/design-sk
The central algorithm is implemented by `MinioArchiver` (in the `incode-module-minio-minioarchlib` submodule).
This:

* calls `DocBlobClient` to obtain a list of all documents that need to be archived.
* calls `DomainObjectPropertyClient` to obtain a list of all documents that need to be archived.
+
The `DocBlobClient` is the `incode-module-minio-docclient` submodule.
The `DomainObjectPropertyClient` is the `incode-module-minio-dopclient` submodule.

* The `DocBlobClient` in turn makes a REST call to `DocBlobService`, hosted by the Apache Isis webapp.
* The `DomainObjectClient` in turn makes a REST call to `DomainObjectPropertyService`, hosted by the Apache Isis webapp.
+
The `DocBlobService` is in the `incode-module-minio-docserver` submodule, and so this is the one dependency that an Apache Isis webapp has on the Minio library.
The `DomainObjectPropertyService` is in the `incode-module-minio-dopserver` submodule, and so this is the one dependency that an Apache Isis webapp has on the Minio library.

* within the Apache Isis webapp, the `DocBlobService` delegates to the `DocBlobServiceBridge`.
* within the Apache Isis webapp, the `DomainObjectPropertyService` delegates to the `DomainObjectPropertyServiceBridge`.
+
This is an SPI that the consuming application is required to implement.
Typically this is likely to delegate to some repository service to obtain entity/ies that have blobs, eg documents.
Typically this is likely to delegate to some repository service to obtain entity/ies that have blobs/clobs, eg documents.

* for each of the returned blobs, the `MinioArchiver` then calls the `MinioBlobClient` to upload the blob into Minio itself.
This returns back a URL.
+
The `MinioBlobClient` is in the `incode-module-minio-minioclient` submodule.

* for each blob and its corresponding URI, the `MinioArchiver` then calls the `DocBlobClient` once more, this time to indicate that the blob has been archived.
* for each blob and its corresponding URI, the `MinioArchiver` then calls the `DomainObjectPropertyClient` once more, this time to indicate that the blob has been archived.

* the `DocBlobClient` in turn again makes a REST call up to `DocBlobService`, hosted on the Apache Isis webapp.
* the `DomainObjectPropertyClient` in turn again makes a REST call up to `DomainObjectPropertyService`, hosted on the Apache Isis webapp.

* once more the `DocBlobService` delegates to the `DocBlobServiceBridge` SPI.
* once more the `DomainObjectPropertyService` delegates to the `DomainObjectPropertyServiceBridge` SPI.
+
The implementation of this (provided by the consuming application) will typically delete the blob from the original entity, and in its place store the URI as a pointer to Minio.
The implementation of this (provided by the consuming application) will typically delete the blob/clob from the original entity, and in its place store the provided URL as a pointer to the content now stored in Minio.
+
[NOTE]
====
It's the responsibility of the consuming application to download the content from Minio (using the URL); this library does not (currently) provide any utility services for this.
====

The `MinioArchiver` described above is packaged as a library, so will typically be called periodically from some sort of integration solution.
As the diagram above suggests, this could be accompllished using Apache Camel, whereby a `MinioArchiverProcessor` (not part of this library) acts as a simple wrapper that calls the `MinioArchiver`.
As the diagram above suggests, this could be accomplished using Apache Camel, whereby a `MinioArchiverProcessor` (not part of this library) acts as a simple wrapper that calls the `MinioArchiver`.
That processer in turn might be scheduled to run periodically, say once an hour.

When first deploying this solution, there will probably be a need to archive historical blobs.
Expand Down Expand Up @@ -88,7 +88,7 @@ The `Main` utility in the `incode-module-minio-minioarchtool` is a standalone ut
The format of the URLs created by `MinioBlobClient is:
`http://minioserver/prod/myapp/cust.Customer/1234`
`http://minioserver/prod/myapp/cust.Customer/1234/photo`
where:
Expand All @@ -98,14 +98,9 @@ where:
This represents the original app from which the Blob was obtained, and therefore how to interpret the remainder of the URL
* "cust.Customer/1234" is the identifier of (in this case) a customer.
It corresponds to the bookmark of the Apache Isis application (having replaced '/' with ':').
* "photo" is the property containing the blob
[NOTE]
====
The above scheme means that only one blob can be persisted per object instance.
A future enhancement would be to allow multiple blobs to be persisted (corresponding to different properties of the original entity in the Apache Isis webapp).
====
== Integration solution (eg Apache Camel)
Expand All @@ -120,15 +115,7 @@ Update your classpath by adding these dependencies to your `pom.xml`:
----
<dependency>
<groupId>org.incode.module.minio</groupId>
<artifactId>incode-module-minio-minioclient</artifactId>
</dependency>
<dependency>
<groupId>org.incode.module.minio</groupId>
<artifactId>incode-module-minio-archlib</artifactId>
</dependency>
<dependency>
<groupId>org.incode.module.minio</groupId>
<artifactId>incode-module-minio-docclient</artifactId>
<artifactId>incode-module-minio-minioarchlib</artifactId>
</dependency>
----
Expand Down Expand Up @@ -197,12 +184,12 @@ If running inside of Apache Camel and using Spring to configure the components:

<bean id="minioArchiver"
class="org.incode.module.minio.minioarchlib.MinioArchiver">
<property name="docBlobClient" ref="docBlobClient"/>
<property name="minioBlobClient" ref="minioBlobClient"/>
<property name="dopClient" ref="domainObjectPropertyClient"/>
<property name="minioUploadClient" ref="minioUploadClient"/>
</bean>

<bean id="minioBlobClient"
class="org.incode.module.minio.minioclient.MinioBlobClient"
<bean id="minioUploadClient"
class="org.incode.module.minio.minioclient.MinioUploadClient"
init-method="init">
<property name="url" value="${minio.baseUrl}"/>
<property name="accessKey" value="${minio.accessKey}"/>
Expand All @@ -211,14 +198,15 @@ If running inside of Apache Camel and using Spring to configure the components:
<property name="prefix" value="${minio.prefix}"/>
</bean>

<bean id="docBlobClient"
class="org.incode.module.minio.docclient.DocBlobClient"
<bean id="domainObjectPropertyClient"
class="org.incode.module.minio.dopclient.DomainObjectPropertyClient"
init-method="init">
<property name="base" value="${apacheIsisWebapp.baseUrl}"/>
<property name="username" value="${apacheIsisWebapp.username}"/>
<property name="password" value="${apacheIsisWebapp.password}"/>
</bean>

...
</beans>
----
Expand Down Expand Up @@ -274,56 +262,49 @@ This section describes the responsibilities of the Apache Isis webapp that has d
The consuming Apache Isis application is required to implement the `DocBlobServiceBridge` SPI:
[source,java]
.DocBlobServiceBridge.java
.DomainObjectPropertyServiceBridge.java
----
public interface DocBlobServiceBridge {
public interface DomainObjectPropertyServiceBridge {

@Programmatic
List<DocBlob> findToArchive(String caller);
List<DomainObjectProperty> findToArchive(String caller);

@Programmatic
Blob blobFor(DocBlob docBlob);
Blob blobFor(DomainObjectProperty dop);

@Programmatic
void archive(String docBookmark, String externalUrl);
Clob clobFor(DomainObjectProperty dop);

void archive(DomainObjectProperty domainObjectProperty, String externalUrl);
}
----
where `DocBlob` is a view model that in effect just wraps the identity of the source entity which holds the blob:
where `DomainObjectProperty` is a DTO that identifies the source entity and property which holds the blob or clob, also indicating which type it is:
[source,java]
.DocBlob.java
.DomainObjectProperty.java
----
@DomainObject(
nature = Nature.VIEW_MODEL,
objectType = "incodeMinio.DocBlob"
)
public class DocBlob implements ViewModel {

public DocBlob(final String docBookmark) {
this.docBookmark = docBookmark;
}
@Data
@AllArgsConstructor
public class DomainObjectProperty {

@Getter
private String docBookmark; // <1>
public enum Type { BLOB,CLOB }

...
private final String bookmark;
private final String property;
private final Type type;

public DomainObjectProperty(final Bookmark bookmark, final String property, final Type type) {
this(bookmark.toString(), property, type);
}
}
----
<1> Bookmark of the persisted entity which holds the blob to be archived (or may have been archived).
In the implementation of this SPI, the application can create `DocBlob` instances simply using the regular `BookmarkService`:
In the implementation of this SPI, the application can create `DomainObjectProperty` instances simply using the regular `BookmarkService`:
[source,java]
----
final Bookmark bookmark = bookmarkService.bookmarkFor(entity);
return new DocBlob(bookmark.toString());
return new DomainObjectProperty(bookmark, "photo", Type.BLOB);
----
[NOTE]
====
The `DocBlobServiceBridge` SPI is slightly inconsistent; the `archive(...)` method ought to take a `DocBlob` rather than a `docBookmark`.
====
=== Classpath
Expand All @@ -333,11 +314,11 @@ Update your classpath by adding these dependencies to your `pom.xml`:
----
<dependency>
<groupId>org.incode.module.minio</groupId>
<artifactId>incode-module-minio-docserver</artifactId>
<artifactId>incode-module-minio-dopserver</artifactId>
</dependency>
----
Check for later releases by searching http://search.maven.org/#search|ga|1|incode-module-minio-docserver[Maven Central Repo].
Check for later releases by searching http://search.maven.org/#search|ga|1|incode-module-minio-dopserver[Maven Central Repo].
=== Bootstrapping
Expand All @@ -348,9 +329,8 @@ Typically this is done by defining an owning `Module` and then including that mo
== Known issues / Limitations
* This implementation only supports one blob/clob property per domain type.
* It also doesn't distinguish between different domain types which may require archiving.
* Ths implementation doesn't distinguish between different domain types which may require archiving.
+
This makes it the responsibility of the SPI to "assemble" the lists of all domain entities which may require archival (eg ``Document``s, ``Command``s, ``PublishedEvent``s) rather than each being archived separately.
* The library also doesn't provide any support for the consuming application to retrieving blobs from Minio.
Expand Down
Loading

0 comments on commit 59ccefc

Please sign in to comment.