Consolidate development of Docker images

[bittner]

I'm working with colleagues on operating ModSecurity+CRS on modern infrastructure for some months, and we've seen a few challenges that make it hard to do sustainable software development.

What's wrong?

The industry needs a set of official images that are both trustworthy and easy to work with. Modern software development requires us to react quickly, hence we need sane defaults and convenient ways to override the default configuration. Container security is a growing concern, hence we must ensure the smallest attack surface possible -- without compromising development and operation convenience.

We need to:

• simplify and standardize the current development efforts (e.g. Collapse branches into single branch #24 et al.)
• separate Docker image development from CRS development (see owasp-modsecurity-crs)
• provide fully functional convenience configurations (e.g. vshn/modsecurity-docker)
• ensure we have automated builds to remove development burden from maintainers (related to Things Needed For Docker #15, Post Images to GCR #22)
• ensure that we can react in a timely fashion on issues and PRs (e.g. Purge Git history in Docker image (saves > 3 MiB) SpiderLabs/owasp-modsecurity-crs#1345 (comment))

How can we help?

We have some capacity (and need) to work on consolidating the existing container image efforts. You can see our efforts in the related images on Docker Hub (e.g. vshn/modsecurity, vshn/modsecurity-crs).

My colleagues and I have extensive experience with crafting and running container images. If we could help maintain the Docker images in https://github.com/CRS-support (e.g. modsecurity-docker and modsecurity-crs-docker) that may free some of your resources for maintaining the CRS rules.

Can we help maintaining? Would that be possible?

[chaimsanders-okta]

0. Mock Create separate CRS-docker in CRS-suppport.
1. Consolidate structure (similar to that https://github.com/appuio/container-oc)
2. Auto-rebuilld images
   2.1) push image to multiple hubs (quay.io)
3. Add TLS and Proxy and other flags - make it agnostic of tools on OS
4. Add FTW
5. Adding ModSecurity Testing to Docker Image
6. Attack surface - reduce usage of third party tools

[bittner]

Related: SpiderLabs/owasp-modsecurity-crs#1600

[bittner]

Planned tasks to refactor modsecurity-docker

• Switch from branch-based versioning to folder-based versioning, on a single main branch (e.g. master)
• Clean up Dockerfile implementations for all supported combinations of ModSecurity and Apache/Nginx versions (inherit from existing, stable images as much as makes sense)
• Automate building images for all supported combinations of ModSecurity and Apache/Nginx versions

[dune73]

This is a very sane approach and I think the CRS projects can agree to this. Formal decision (and we think it takes a formal decision) will be on Monday during the CRS chat.

[bittner]

For the sake of transparency, a detail that guides us now in implementing this change:

@csanders-git explained his motivation that led to the change of the CRS setup directory structure in SpiderLabs/owasp-modsecurity-crs#1420 (comment). It is essentially, to get rid of distro based images (ubuntu, fedora, alpine) and rely solely on the official upstream images of the web proxies.

In alignment with that we will create a repository structure and automated builds for the following combinations:

• v2-apache
• v2-nginx
• v3-apache
• v3-nginx

... with v2/v3 referring to the latest ModSecurity version 2 and 3
... apache to the httpd and
... nginx to the nginx Docker image.

As by the default of the upstream images those will be debian:buster-slim based. It should be easy to extend the build matrix later if any other specific combinations are needed.

[bittner]

@zimmerle from SpiderLabs recommends using ModSecurity v3 with Nginx and using v2 only with Apache. Does that mean that we can assume those as defaults? i.e.

• owasp/modsecurity:v2 --> Apache 2.4 and ModSecurity v2 (latest stable)
• owasp/modsecurity:v3 --> Nginx 1.17 and ModSecurity v3 (latest stable)

This would simplify the build matrix for the Docker images. But is this a good idea?

Any thoughts?

[lifeforms]

I think it makes perfect sense.

[csanders-git]

Agreed, this was the route that we wanted to take.

[bittner]

@csanders-git and I just clarified this on the OWASP Slack in addition. We will configure automatic builds for the tags

• 2, 2.9, 2.9.3 (with Apache) and
• 3, 3.0, 3.0.3 (with Nginx).

The builds for all other tags will remain unaffected for the moment, they will continue to run separately.

Afterwards, everyone should take a look at how they like the result. Later in 2020, we can look into how to allow v3 to be built with any supported Web proxy other than Nginx in addition. Apache will remain the only supported Web proxy for ModSecurity v2.

[bittner]

Automatic builds of the owasp/modsecurity Docker image are now working.

They are not configured as Docker Hub Automated Builds but implemented via GitHub Actions, which provides good integration with the repository, transparency, more flexibility (e.g. we can put several tags onto one image being built) and ultimately more build speed.

The README, seemingly maintained manually on Docker Hub, is now also under version control in the master branch. The new image tags are also mentioned there now.

Your feedback, please!

I would like to invite everyone now to revisit the achievemements on this repository.

• master branch, which should become the main (and only permanent) branch in the long run.
• Automatic builds, implemented using GitHub Actions.
• New images tags 2, ... and 3, ...

If everything looks good we may start to deprecate the other branches and their related image tags.

[bittner]

A quick update:

We are continuing our consolidation efforts. In PR #34 we've tried to make the inclusion of configuration related to ModSecurity more clearly isolated. Yet, there are some issues we still left unaddressed in the "ModSecurity 2 + Apache" image:

1. the absolute path used in the Include /etc/modsecurity.d/include.conf statement (this was probably added for technical reasons, to make the include outside of the Apache conf folder work)
2. the file name of /etc/modsecurity.d/include.conf as such (it should probably convey "use me to include the ModSecurity configuration from Apache", but the pattern of "include" is broken here)

Any ideas that help to make this cleaner or more elegant are very welcome!

[simu]

We are considering using the modsecurity v3 docker images built from this repository to implement a setup for a customer where the customer provides a small set of custom modsecurity rules and we load those rules into the modsecurity docker image at runtime. The customer is not interested in using the core rule set at this time, so we're looking for a plain modsecurity v3 image without any rules preconfigured.

However, the images built from this repository are not flexible enough for our use case (running modsecurity v3 on OpenShift in proxy mode, listening on a custom port), as they don't allow any runtime configuration and very limited build time configuration. Even if there was enough build time configuration options, we'd still have to build our own plain modsecurity v3 docker image to configure proxy mode and the a listen port which is compatible with running the image on OpenShift.

I've seen that there is some work to make the CRS docker image expose more configuration options at runtime (coreruleset/modsecurity-crs-docker#7), and I'm wondering whether it wouldn't make more sense to expose the configuration options related to modsecurity itself and the webserver (Apache in the mentioned PR, or nginx for our use case) in the image built from this repository, rather than the image which includes the core rule set since those options may be interesting also for users who don't want to use the core rule set.

[bittner]

That totally makes sense! -- The underlying idea of the current architecture was based on the understanding that no-one would or should ever use plain ModSecurity w/o the CRS. Hence the configuration options would have been fine in the image that adds the CRS. Your use case changes that view.

It should be fairly easy to move the work that we have started in coreruleset/modsecurity-crs-docker#7 to this repository, which would also improve the focus on adding "just CRS" in the former.

Any other thoughts? @csanders-git @dune73 @lifeforms @srueg @zugao

[srueg]

I totally agree. Ideally the base image (non CRS) provides all the config options for Apache/nginx & ModSecurity and the CRS image really only adds the core rule set.

[bittner]

PR #38 is now ready for review. Please, take a look!

[fzipi]

@bittner Do you think we can close this one?

[bittner]

This is up to @coreruleset/vshn-ag to decide. I don't work for them anymore.

[fzipi]

I'm deciding then. Closing.