Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update "select a storage driver" with current status #7669

Merged
merged 6 commits into from
Nov 13, 2018
Merged

Update "select a storage driver" with current status #7669

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3b24959
Update "select a storage driver" with current status
thaJeztah Nov 12, 2018
48f9a7e
Update select-storage-driver.md
ahh-docker Nov 13, 2018
f0c93ff
Update select-storage-driver.md
ahh-docker Nov 13, 2018
3d7a90a
Minor edits
bermudezmt Nov 13, 2018
e7cbb50
Update select-storage-driver.md
ahh-docker Nov 13, 2018
3805b23
Update select-storage-driver.md
ahh-docker Nov 13, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
154 changes: 81 additions & 73 deletions storage/storagedriver/select-storage-driver.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,48 +21,53 @@ After you have read the [storage driver overview](index.md), the
next step is to choose the best storage driver for your workloads. In making
this decision, there are three high-level factors to consider:

- If multiple storage drivers are supported in your kernel, Docker has a
prioritized list of which storage driver to use if no storage driver is
explicitly configured, assuming that the prerequisites for that storage driver
are met:

- If possible, the storage driver with the least amount of configuration is
used, such as `btrfs` or `zfs`. Each of these relies on the backing
filesystem being configured correctly.

- Otherwise, try to use the storage driver with the best overall performance
and stability in the most usual scenarios.

- `overlay2` is preferred, followed by `overlay`. Neither of these requires
extra configuration. `overlay2` is the default choice for Docker CE.

- `devicemapper` is next, but requires `direct-lvm` for production
environments, because `loopback-lvm`, while zero-configuration, has very
poor performance.

The selection order is defined in Docker's source code. You can see the order
by looking at
[the source code for Docker CE {{ site.docker_ce_stable_version }}](https://github.com/docker/docker-ce/blob/{{ site.docker_ce_stable_version }}/components/engine/daemon/graphdriver/driver_linux.go#L50)
You can use the branch selector at the top of the file viewer to choose a
different branch, if you run a different version of Docker.
{: id="storage-driver-order" }

- Your choice may be limited by your Docker edition, operating system, and
distribution. For instance, `aufs` is only supported on Ubuntu and Debian, and
may require extra packages to be installed,
while `btrfs` is only supported on SLES, which is only supported with Docker
EE. See
[Support storage drivers per Linux distribution](#supported-storage-drivers-per-linux-distribution).

- Some storage drivers require you to use a specific format for the backing
filesystem. If you have external requirements to use a specific backing
filesystem, this may limit your choices. See
[Supported backing filesystems](#supported-backing-filesystems).

- After you have narrowed down which storage drivers you can choose from, your
choice are determined by the characteristics of your workload and the
level of stability you need. See [Other considerations](#other-considerations)
for help making the final decision.
If multiple storage drivers are supported in your kernel, Docker has a prioritized
list of which storage driver to use if no storage driver is explicitly configured,
assuming that the storage driver meets the prerequisites.

Use the storage driver with the best overall performance and stability in the most
usual scenarios.

Docker supports the following storage drivers:

* `overlay2` is the preferred storage driver, for all currently supported
Linux distributions, and requires no extra configuration.
* `aufs` is the preferred storage driver for Docker 18.06 and older, when
running on Ubuntu 14.04 on kernel 3.13 which has no support for `overlay2`.
* `devicemapper` is supported, but requires `direct-lvm` for production
environments, because `loopback-lvm`, while zero-configuration, has very
poor performance. `devicemapper` was the recommended storage driver for
CentOS and RHEL, as their kernel version did not support `overlay2`. However,
current versions of CentOS and RHEL now have support for `overlay2`,
which is now the recommended driver.
* The `btrfs` and `zfs` storage drivers are used if they are the backing
filesystem (the filesystem of the host on which Docker is installed).
These filesystems allow for advanced options, such as creating "snapshots",
but require more maintenance and setup. Each of these relies on the backing
filesystem being configured correctly.
* The `vfs` storage driver is intended for testing purposes, and for situations
where no copy-on-write filesystem can be used. Performance of this storage
driver is poor, and is not generally recommended for production use.

Docker's source code defines the selection order. You can see the order at
[the source code for Docker Engine - Community {{ site.docker_ce_stable_version }}](https://github.com/docker/docker-ce/blob/{{ site.docker_ce_stable_version }}/components/engine/daemon/graphdriver/driver_linux.go#L50)

If you run a different version of Docker, you can use the branch selector at the top of the file viewer to choose a different branch.
{: id="storage-driver-order" }

Some storage drivers require you to use a specific format for the backing filesystem. If you have external
requirements to use a specific backing filesystem, this may limit your choices. See [Supported backing filesystems](#supported-backing-filesystems).

After you have narrowed down which storage drivers you can choose from, your choice is determined by the
characteristics of your workload and the level of stability you need. See [Other considerations](#other-considerations)
for help in making the final decision.

> ***NOTE***: Your choice may be limited by your Docker edition, operating system, and distribution.
> For instance, `aufs` is only supported on Ubuntu and Debian, and may require extra packages
> to be installed, while `btrfs` is only supported on SLES, which is only supported with Docker
> Enterprise. See [Support storage drivers per Linux distribution](#supported-storage-drivers-per-linux-distribution)
> for more information.


## Supported storage drivers per Linux distribution

Expand All @@ -73,25 +78,34 @@ In addition, Docker does not recommend any configuration that requires you to
disable security features of your operating system, such as the need to disable
`selinux` if you use the `overlay` or `overlay2` driver on CentOS.

### Docker EE and CS-Engine
### Docker Engine - Enterprise and Docker Enterprise

For Docker EE and CS-Engine, the definitive resource for which storage drivers
are supported is the
For Docker Engine - Enterprise and Docker Enterprise, the definitive resource for which
storage drivers are supported is the
[Product compatibility matrix](https://success.docker.com/Policies/Compatibility_Matrix).
To get commercial support from Docker, you must use a supported configuration.

### Docker CE
### Docker Engine - Community

For Docker CE, only some configurations are tested, and your operating system's
kernel may not support every storage driver. In general, the following
For Docker Engine - Community, only some configurations are tested, and your operating
system's kernel may not support every storage driver. In general, the following
configurations work on recent versions of the Linux distribution:

| Linux distribution | Recommended storage drivers |
|:--------------------|:------------------------------------------------------------------------------------------------------|
| Docker CE on Ubuntu | `aufs`, `devicemapper`, `overlay2` (Ubuntu 14.04.4 or later, 16.04 or later), `overlay`, `zfs`, `vfs` |
| Docker CE on Debian | `aufs`, `devicemapper`, `overlay2` (Debian Stretch), `overlay`, `vfs` |
| Docker CE on CentOS | `devicemapper`, `vfs` |
| Docker CE on Fedora | `devicemapper`, `overlay2` (Fedora 26 or later, experimental), `overlay` (experimental), `vfs` |
| Linux distribution | Recommended storage drivers | Alternative drivers |
|:--------------------|:-----------------------------------------------------------------------|:--------------------------------------------------|
| Docker Engine - Community on Ubuntu | `overlay2` or `aufs` (for Ubuntu 14.04 running on kernel 3.13) | `overlay`¹, `devicemapper`², `zfs`, `vfs` |
| Docker Engine - Community on Debian | `overlay2` (Debian Stretch), `aufs` or `devicemapper` (older versions) | `overlay`¹, `vfs` |
| Docker Engine - Community on CentOS | `overlay2` | `overlay`¹, `devicemapper`², `zfs`, `vfs` |
| Docker Engine - Community on Fedora | `overlay2` | `overlay`¹, `devicemapper`², `zfs`, `vfs` |

¹) The `overlay` storage driver is deprecated in Docker Engine - Enterprise 18.09, and will be
removed in a future release. It is recommended that users of the `overlay` storage driver
migrate to `overlay2`.

²) The `devicemapper` storage driver is deprecated in Docker Engine 18.09, and will be
removed in a future release. It is recommended that users of the `overlay` storage driver
migrate to `overlay2`.


When possible, `overlay2` is the recommended storage driver. When installing
Docker for the first time, `overlay2` is used by default. Previously, `aufs` was
Expand All @@ -112,10 +126,10 @@ storage driver, be sure to read about
[its performance and storage characteristics and limitations](vfs-driver.md).

> **Expectations for non-recommended storage drivers**: Commercial support is
> not available for Docker CE, and you can technically use any storage driver
> not available for Docker Engine - Community, and you can technically use any storage driver
> that is available for your platform. For instance, you can use `btrfs` with
> Docker CE, even though it is not recommended on any platform for Docker CE,
> and you do so at your own risk.
> Docker Engine - Community, even though it is not recommended on any platform for
> Docker Engine - Community, and you do so at your own risk.
>
> The recommendations in the table above are based on automated regression
> testing and the configurations that are known to work for a large number of
Expand All @@ -140,12 +154,12 @@ backing filesystems.

| Storage driver | Supported backing filesystems |
|:----------------------|:------------------------------|
| `overlay`, `overlay2` | `ext4`, `xfs` |
| `aufs` | `ext4`, `xfs` |
| `overlay2`, `overlay` | `xfs` with fstype=1, `ext4` |
| `aufs` | `xfs`, `ext4` |
| `devicemapper` | `direct-lvm` |
| `btrfs` | `btrfs` |
| `zfs` | `zfs` |

| `vfs` | any filesystem |

## Other considerations

Expand All @@ -155,13 +169,14 @@ Among other things, each storage driver has its own performance characteristics
that make it more or less suitable for different workloads. Consider the
following generalizations:

- `aufs`, `overlay`, and `overlay2` all operate at the file level rather than
- `overlay2`, `aufs`, and `overlay` all operate at the file level rather than
the block level. This uses memory more efficiently, but the container's
writable layer may grow quite large in write-heavy workloads.
- Block-level storage drivers such as `devicemapper`, `btrfs`, and `zfs` perform
better for write-heavy workloads (though not as well as Docker volumes).
- For lots of small writes or containers with many layers or deep filesystems,
`overlay` may perform better than `overlay2`.
`overlay` may perform better than `overlay2`, but consumes more inodes, which
can lead to inode exhaustion.
- `btrfs` and `zfs` require a lot of memory.
- `zfs` is a good choice for high-density workloads such as PaaS.

Expand All @@ -186,15 +201,8 @@ specific shared storage system.

For some users, stability is more important than performance. Though Docker
considers all of the storage drivers mentioned here to be stable, some are newer
and are still under active development. In general, `aufs`, `overlay`, and
`devicemapper` are the choices with the highest stability.

### Experience and expertise

Choose a storage driver that your organization is comfortable maintaining. For
example, if you use RHEL or one of its downstream forks, you may already have
experience with LVM and Device Mapper. If so, the `devicemapper` driver might
be the best choice.
and are still under active development. In general, `overlay2`, `aufs`, `overlay`,
and `devicemapper` are the choices with the highest stability.

### Test with your own workloads

Expand All @@ -216,8 +224,8 @@ $ docker info

Containers: 0
Images: 0
Storage Driver: overlay
Backing Filesystem: extfs
Storage Driver: overlay2
Backing Filesystem: xfs
<output truncated>
```

Expand Down