v3.3 Custom Storage Driver

Common Steps

1. Please review the Storage Driver Inheritance Hierarchy

2. Implement the custom storage driver by coding the abstract methods (see details below).

3. Build the storage driver implementation jar file.

4. Put the storage driver implementation jar file into the Mongoose's directory among with other Mongoose jar files.

5. Modify the config/defaults.json file by adding the implementation info.

   To do this, find the section storage->driver->impl. This is a list of nodes. Each node reflects the particular storage driver implementation metainfo and contains the following required fields:

   • type: the identifier of the storage driver implementation
   • file: the path to a jar file containing the storage driver implementation
   • fqcn: storage driver implementation FQCN (fully qualified class name)

   Append the node describing the custom storage driver implementation to the list.

   Note:

   When using in the distributed mode, the custom storage driver configuration and implementation jar file are required to be located on the storage driver server host.

6. Run Mongoose with an argument --storage-driver-type=X where X is the custom storage driver identifier.

NIO Storage Driver Implementation

This abstract implementation uses few I/O threads to execute a lot if I/O tasks in parallel. Actual I/O work should be executed in the method invokeNio(ioTask) in the reentrant and non-blocking manner.

• Reference implementations:

  com.emc.mongoose.storage.driver.nio.fs.BasicFileStorageDriver

• Extend the class com.emc.mongoose.storage.driver.nio.base.NioStorageDriverBase

  Methods to implement:

  1. void invokeNio(O ioTask) Invoked to perform the actual I/O. May not complete the specified I/O task (reentrancy).

  2. List<I> list(ItemFactory<I> itemFactory, String path, String prefix, int idRadix, I lastPrevItem, int count) throws IOException Invoked to get a list of the items located at the specified path.

  3. void adjustIoBuffers(SizeInBytes avgDataItemSize, IoType ioType) Invoked to notify the storage driver about the expected amount of data to be transferred.

  4. String requestNewPath(final String path) Create the specified path on the storage. Invoked if the specified path is not existing yet.

  5. String requestNewAuthToken(final Credential credential) Create the specified auth token. Invoked if the auth token for the specified credential is not existing yet.

• Implement interface com.emc.mongoose.model.storage.StorageDriver

Netty-Based Storage Driver Implementation

This abstract implementation is intended to work with distributed storage with multiple endpoint nodes accessible via the network. Provides high-performance connection pool, simple endpoint node balancing, SSL/TLS functionality.

• Reference implementations:

  N/A

• Extend the class com.emc.mongoose.storage.driver.net.base.NetStorageDriverBase

  Methods to implement:

  1. ChannelFuture sendRequest(Channel channel, O ioTask) Invoked when a connection ready for work is obtained from the connection pool. The method should build a request from the specified I/O task and write it to a specified channel.

  2. List<I> list(ItemFactory<I> itemFactory, String path, String prefix, int idRadix, I lastPrevItem, int count) throws IOException Invoked to get a list of the items located at the specified path.

  3. String requestNewPath(final String path) Create the specified path on the storage. Invoked if the specified path is not existing yet.

  4. String requestNewAuthToken(final Credential credential) Create the specified auth token. Invoked if the auth token for the specified credential is not existing yet.

• Extend the class com.emc.mongoose.storage.driver.net.base.ResponseHandlerBase

  Methods to implement:

  1. void handle(Channel channel, O ioTask, N msg) throws IOException

• Implement the interface:

  com.emc.mongoose.storage.driver.net.base.NetStorageDriver

HTTP Storage Driver Implementation

This abstract implementation inherits the Netty-Based one and adds the HTTP-related specific functionality.

• Reference implementations:

  1. com.emc.mongoose.storage.driver.net.http.atmos.AtmosStorageDriver
  2. com.emc.mongoose.storage.driver.net.http.s3.S3StorageDriver
  3. com.emc.mongoose.storage.driver.net.http.swift.SwiftStorageDriver

• Extend the class com.emc.mongoose.storage.driver.net.http.base.HttpStorageDriverBase

  Methods to implement:

  1. List<I> list(ItemFactory<I> itemFactory, String path, String prefix, int idRadix, I lastPrevItem, int count)

  2. String requestNewPath(final String path)

  3. String requestNewAuthToken(final Credential credential)

  4. HttpMethod getTokenHttpMethod(IoType ioType) Convert the internal I/O type to the corresponding HTTP method for token items.

  5. HttpMethod getPathHttpMethod(IoType ioType) Convert the internal I/O type to the corresponding HTTP method for path items.

  6. String getTokenUriPath(I item, String srcPath, String dstPath, IoType ioType) Construct the URI path for the given token item.

  7. String getPathUriPath(I item, String srcPath, String dstPath, IoType ioType) Construct the URI path for the given path item.

  8. void applyMetaDataHeaders(final HttpHeaders httpHeaders) Add additional headers if necessary.

  9. void applyAuthHeaders(HttpMethod httpMethod, String dstUriPath, HttpHeaders httpHeaders) Add authentication headers if necessary.

  10. void applyCopyHeaders(HttpHeaders httpHeaders, String srcPath) Add the headers to perform a server side copy operation.

• Extend class com.emc.mongoose.storage.driver.net.http.base.HttpResponseHandlerBase

  Methods to implement:

  1. void handleResponseHeaders(O ioTask, HttpHeaders responseHeaders)

• Implement the interface:

  com.emc.mongoose.storage.driver.net.http.base.HttpStorageDriver