Ability to connect to multiple instances of content stores within the same application

[kendclark]

Currently it is possible only to connect to a single instance of each content store type, for instance a single S3 account. It would be preferable to be able to have multiple S3 accounts and to be able to switch among them during content store or retrieval, even from within the same ContentStore. Likewise, it may be necessary to use entirely different contentstore types (i.e. filesystem vs S3 etc.) within the same entity.

This would allow easier management of multiple tenants who each may have (or require) a separate content store.

[paulcwarren]

Hi @kendclark, thanks for raising this issue.

Spring Data has a very similar capability as described at Baeldung here. Usually I would implement the same, or similar, approach assuming that the Spring guys thought pretty carefully about how to implement this in the first place and so that it is familiar to Spring developers in general.

So, if we added a s3ClientRef (and so on for each Store) would this work for you?

Thanks

[kendclark]

I think so, but a few things to consider:

• at least in my case I would not be able to isolate entities against a single s3ClientRef - rather the same entity would use different s3ClientRefs depending on the tenant who owns each entity instance.
• the existing hook is the ClientId converter, which allows me to choose the bucket based on the current entity. I don't know if this could continue to work as the place to select the appropriate s3ClientRef for each entity instance, but if so it would need a good amount of rework (so probably not)
• I would not want to be restricted to configuration by property file or annotated class - I need to set up new s3ClientRefs on the fly so I can add new tenant configurations by storing the configuration in the database and wiring it up - for instance how Spring Security allows the creation of the ClientRegistrationRepository as a Bean, and so I can control that and therefore add and manage as many ClientRegistration instances as I need, and accordingly can add new authentication providers for new tenants without changing any code

[paulcwarren]

I had figured you needed more dynamic behavior. So what you need is much more like setting the multi-tenancy properties in JPA.

I also had a google to see if the amazon s3 sdk provided any multi-tenancy capability and found nothing. It would have been nice to leverage an existing capability.

So, in lieu of that I am thinking that we could define a new AmazonS3Provider interface allowing you to contribute a bean that know how to get hold of the AmazonS3 object for the current request/thread. Does that sound more like what you need?

[kendclark]

Yes, that is what I was thinking.

However, it is possible that it is not always going to be S3. Our app will have a single entity that stores all document metadata, and then the content will be held outside the app database.

S3 does have access management that would allow separation of content (by bucket) to different clients, so at least for the short term the current functionality may be acceptable. However, it may not satisfy all CISOs, and probably would increase the effort on our side as well.

[paulcwarren]

Thanks for the feedback.

So, are we talking about something like this then?

@Entity
@Data
public class SomeEntity {

   @Id
   @GeneratedValue(...)
   private Long id;

   @ContentId
   private String contentId;
} 

public interface SomeEntityS3ContentStore extends S3ContentStore<SomeEntity, String>
public interface SomeEntityJPAContentStore extends JPAContentStore<SomeEntity, String>
public interface SomeEntityFSContentStore extends FilesystemContentStore<SomeEntity, String>

And depending on the tenant and their configuration the service would (1) resolve to the correct content store and (2) resolve to the correct connection for that content store.

So, a tenant specifying S3 would resolve to (1) SomeEntityS3ContentStore and to (2) an AmazonS3 bean with the credentials as specified in their tenant configuration.

Likewise though, a tenant using an external database (RDS perhaps) would resolve to (1) SomeEntityJPAContentStore and to (2) a Datasource bean with the credentials specified in their tenant configuration.

And so on.

Two questions come to mind:

• would you want to use Spring Content REST on top of this content service too?
• are their any other storage modules that you would need besides those already offered?

[paulcwarren]

Hi @kendclark I have made a few commits on 1.1.x branch (Spring Boot 2.3) and I would now like to get your feedback before proceeding further.

In brief:

• added the ability to configure a StoreResolver for Spring Content REST so that you can resolve a request to a specific store for each tenant. docs.
• added the ability to configure a MultiTenantAmazonS3Provider so that you can resolve each s3 store content operation to a specific AmazonS3 client for each tenant. docs

Is this looking like it is headed in a direction that will solve you requirements?

[paulcwarren]

Ping @kendclark . is this still a requirement for you?

[kendclark]

It is, though this project is on hold at the moment and I am full-time on another. However, I will look at this as soon as I get a few hours.

It is not immediately clear to me how to use these. I assume I need both?

It looks like the MultiTenantAmazonS3Provider bean method replaces the AmazonS3 bean, but I don't understand the difference. Do I just build the AmazonS3 exactly as before? For the StoreResolver/StoreInfo, a couple things. The first is the path ("examples" in your example) - where does that come from? Also, as for StoreInfo, it may be helpful to see an example as to how to configure that.

[paulcwarren]

Thanks for getting back to me @kendclark .

Yes, from what you described above you would need both.

StoreResolver

Having multiple stores typed to the same domain object (in the absence of a @StoreRestResource path declaration the domain type class determines the URI path) will cause store conflicts in the REST layer that need to be resolved somehow. The StoreResolver allows you to plug code in that tells the REST layer which store to use, depending on context. Under the covers we use a more generic "named" StoreFilter. When the REST layer needs uses that filter to resolve a request conflict it will use the "path" as the "name". "examples" in my docs example. I have updated this to try and make that more clear. That said, I wasn't sold on this so I am happy to investigate/improve the intuitiveness in this area.

MultiTenantAmazonS3Provider

Yes, the MultiTenantAmazonS3Provider bean replaces the AmazonS3 bean. The MultiTenantAmazonS3Provider can provide different AmazonS3 bean depending on (most probably thread local) context.

[paulcwarren]

For Information:

Spring Cloud AWS recently modified the protocol resolver that plugs into the Spring Resource Loader. They removed access to the constructor that accepted an AmazonS3 that we relied on to make the S3 Store multi-tenanted. Looking at the implementation it looks like it is now assumes there will only ever be one AmazonS3 bean in the application context. We are going to have to work with this Spring team to figure out how they see multi-tenancy working with this current implementation.

I have raised this issue to investigate.

[marcellodesales]

t looks like it is now assumes there will only ever be one AmazonS3 bean in the application context. We are going to have to work with this Spring team to figure out how they see multi-tenancy working with this current implementation.

Any update on this?

[paulcwarren]

Hi @marcellodesales, as far as I am aware this works as described in the multi-tenancy section here against Spring Cloud Hoxton.SR12.

That said, that version of Spring Cloud is actually meant for Spring Boot 2.3, not 2.4. But it continues to work for our use cases and as a result we have not need to upgrade. Upgrading to Spring Cloud 2020.0.4 is a major upgrade as they have made backward incompatible changes to their API (by the looks).

What version of Spring Boot and Spring Cloud are you using?

[marcellodesales]

Hi @marcellodesales, as far as I am aware this works as described in the multi-tenancy section here against Spring Cloud Hoxton.SR12.

That said, that version of Spring Cloud is actually meant for Spring Boot 2.3, not 2.4. But it continues to work for our use cases and as a result we have not need to upgrade. Upgrading to Spring Cloud 2020.0.4 is a major upgrade as they have made backward incompatible changes to their API (by the looks).

What version of Spring Boot and Spring Cloud are you using?

@paulcwarren I'm on Spring 2.3 and so it would be working properly... Thank you!

[paulcwarren]

As a side note. I have now upgraded to Spring Cloud 2021.0.0 @marcellodesales. Thanks, also, for the update.