Blob storage for all content on web site

[slacto]

I see Composite.Forms.FormBuilder has an option for using blobConnectionString in \App_Data\Composite\Configuration\Composite.Forms.FormBuilder.xml

We’re thinking about implementing this in C1 to remove all content from the website to make it more like a "live editing" WebApp.
The content (i.e the files that are not in our source code) could be located here:
• \App_Data*.config
• \App_Data\certificates
• \App_Data\Composite\LogFiles
• \App_Data\Composite\Versioning
• \App_Data\DataStores (maybe just selected files)
• \App_Data\Email
• \App_Data\FormBuilderData
• \App_Data\Media
• \App_Data\Search
• \robots.txt

Is this something you have considered/tried?
Do you have recommendations/warnings/alternative solutions?

[burningice2866]

While not impossible its a pretty daunting task.

Most of it could be achieved by writing new Providers - ie. mediafiles are loaded and saved using the MediaFileProvider which could be re-implented loading and saving files from a Blob Storage.

Its also worth noticing that C1 already tries to abstract the underlying filesystem away by having its own C1Directory and C1File classes, and as long as developers remember to use those, it should be possible to direct regular filehandling towards a Blob Storage by introducing a custom IIOProvider.

This is of course all speculations - it would require somewhat of a big investment in doing the proof of concept work which might end up in a dead end.

[burningice2866]

For the Xmlbased Datastore i'm afraid you're out of luck. While it looks like the C1 Team did a proper job of using C1File for loading the xml data store files, they unfortunately rely on local file handling for saving the files again

[burningice2866]

Have you considered simply mounting your Blob Storage as a Network Drive and fool C1 to thinking its saving all its files on disk, while they in reality go to a Blob Storage?

https://stackoverflow.com/questions/56010675/how-can-we-mount-azure-blob-storage-as-a-network-drive-in-windows

[napernik]

That is a big question, can I ask which problem you're trying to address. Is it for:

• For an easier deployment (the continuous deployment of new features to an environment, without without overriding the content, will be easier if they are stored separately)
• As a way to extract/backup website content
• For scale out - the content, uploaded to the blob storage to be shared across multiple instances.
• For something else entirely?

[slacto]

We want to have a "live editing" C1 website on Azure. That means the "content" (like pages, media and form submissions) will be created/administrated and presented on the same website (Azure App Service).
Our "source code" will be C1 source code, custom DLLs, some DataSources like page types, functions, js and css. We'll deploy the source code artifact from GitHub.

I'm not an Azure expert but my understanding is that "content" can disappar on Azure. If something goes wrong Azure will create a new instance based on the source code and the old instance with the error (and the content) will disappear.

Since we have one instance in one region at the moment we might not have this problem. But if the customer wants multiple instances or regions we will have problems with the "live editing content".

We could add SQL Server for the "content" data types as people would normally do, if there are no other options, and Composite.Forms.FormBuilder has a BlobStorage option, but we would still have problems with media, server logs and maybe other files.

[slacto]

It does not have to be a blob storage. WordPress has the wp-content folder for all content files.

[burningice2866]

It does not have to be a blob storage. WordPress has the wp-content folder for all content files.

Have you considered mounting the App_Data folder of your C1 website as a Network Drive pointing to Azure Blob Storage?

[napernik]

We want to have a "live editing" C1 website on Azure. That means the "content" (like pages, media and form submissions) will be created/administrated and presented on the same website (Azure App Service). Our "source code" will be C1 source code, custom DLLs, some DataSources like page types, functions, js and css. We'll deploy the source code artifact from GitHub.

I'm not an Azure expert but my understanding is that "content" can disappar on Azure. If something goes wrong Azure will create a new instance based on the source code and the old instance with the error (and the content) will disappear.

Since we have one instance in one region at the moment we might not have this problem. But if the customer wants multiple instances or regions we will have problems with the "live editing content".

We could add SQL Server for the "content" data types as people would normally do, if there are no other options, and Composite.Forms.FormBuilder has a BlobStorage option, but we would still have problems with media, server logs and maybe other files.

@slacto My current understanding of the App Services is the following:

• For each App Service, a network drive is created, that holds related files.
• The files fill not be removed/lost unless App Service itself is deleted
• When, according to the "App Service plan", website instances are created, those instances are mounted to the same shared network drive ( https://docs.microsoft.com/en-us/azure/app-service/operating-system-functionality#file-access-across-multiple-instances )
• The drive can be accessed through SFTP, for deployment or backups ( https://docs.microsoft.com/en-us/azure/app-service/deploy-ftp?tabs=portal )

When it relates to C1 CMS: C1 console doesn't support "multiple instances + shared network drive" setup.

Current approach that we have, for scalable commerce sites:

• The website is deployed into 2 App Services:
  • Content Management (CM) - a single instance, where the website editors/administrator edit new content
  • Content Delivery (CD) - a multi instance deployment for the public access to the website. Doesn't have the console, and is "stateless", meaning that the content (C1 pages/media) is static and all of the user data is fetched/updated though API calls to a separately deployed service(s).

The editor have an ability to publish from CM to CD via https://c1.orckestra.com/Add-ons/All/Orckestra.Azure.CloudPublisher
package.

• For continuous deployment, we use Azure DevOps pipelines. Where the pipeline does the following:

• Builds custom solution, which generates C1 packages
• The build takes an empty C1 CMS website, runs a console application that installs all of the C1 packages on it ( https://github.com/Orckestra/BetterRetail/blob/dev/Build/install.cake#L261,%20L266 ).
• Packages the result website
• Deploys and overrides the website files to the necessary environment, with the exclusion of a few folders (C1 data files, media, etc.)

It has a few limitations, in particular:

• Manual publish from CM to CD can be an inconvenience to the users
• CloudPublisher uses GIT as an intermediary, and currently the underlying libs don't support SSH, so will not work for GitHub, (does work for Azure DevOps, where HTTPS/password authentication can be used).
• C1 data files aren't cleanly separated between: content, configuration, and custom code (f.e. inline functions are defined in a data type as well.

Another option -> some of the data types can be moved to SQL Data Provider, and the same SQL database can be referenced by CM and CD instances.

For future, currently we consider implementing API based approach, where CD instances can fetch the data from the CM instance through REST API.

Answering your question. There's no straight forward way for multi-instance, the easiest being CM + CD + CloudPublisher + SQL Data Provider.

As a backup strategy, you can backup the website files by reading from SFTP.

[slacto]

Hi Dmitry Thanks for the thorough explanation. It sounds like we can host the website on Azure without any problems as long as we stick to one instance