Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Overlay deprecation in documentation #7803

Closed
wants to merge 4 commits into from
Closed

Overlay deprecation in documentation #7803

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
05ee64f
Initial overlay deprecation cleanup. There might be more files, but n…
ahh-docker Dec 6, 2018
58c2e7f
Update overlayfs-driver.md
ahh-docker Dec 18, 2018
3509dd9
Update index.md
ahh-docker Dec 18, 2018
111ae39
Update index.md
ahh-docker Dec 18, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions storage/storagedriver/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
description: Learn the technologies that support storage drivers.
keywords: container, storage, driver, AUFS, btfs, devicemapper,zvfs
keywords: container, storage, driver, AUFS, btfs, devicemapper,overlay2,overlay,overlayfs
title: About storage drivers
redirect_from:
- /en/latest/terms/layer/
Expand All @@ -9,15 +9,16 @@ redirect_from:
- /storage/storagedriver/imagesandcontainers/
---

Storage drivers create data in the writable layer of your container. The files won't
be persisted after the container is deleted, and both read and write speeds are low.

Docker recommends Overlay2 for the storage driver.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to have it Capitalized, i.e.
Overlay2 -> overlay2


To use storage drivers effectively, it's important to know how Docker builds and
stores images, and how these images are used by containers. You can use this
information to make informed choices about the best way to persist data from
your applications and avoid performance problems along the way.

Storage drivers allow you to create data in the writable layer of your container.
The files won't be persisted after the container is deleted, and both read and
write speeds are low.

[Learn how to use volumes](../volumes.md) to persist data and improve performance.

## Images and layers
Expand Down Expand Up @@ -292,8 +293,8 @@ layer. This means that the writable layer is as small as possible.

When an existing file in a container is modified, the storage driver performs a
copy-on-write operation. The specifics steps involved depend on the specific
storage driver. For the `aufs`, `overlay`, and `overlay2` drivers, the
copy-on-write operation follows this rough sequence:
storage driver. For the `aufs` and `overlay2` drivers, the copy-on-write operation
follows this rough sequence:

* Search through the image layers for the file to update. The process starts
at the newest layer and works down to the base layer one layer at a time.
Expand Down
157 changes: 52 additions & 105 deletions storage/storagedriver/overlayfs-driver.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
description: Learn how to optimize your use of OverlayFS driver.
keywords: container, storage, driver, OverlayFS
keywords: container, storage, driver, OverlayFS,overlay2
title: Use the OverlayFS storage driver
redirect_from:
- /engine/userguide/storagedriver/overlayfs-driver/
Expand All @@ -11,26 +11,32 @@ and with a simpler implementation. Docker provides two storage drivers for
OverlayFS: the original `overlay`, and the newer and more stable `overlay2`.

This topic refers to the Linux kernel driver as `OverlayFS` and to the Docker
storage driver as `overlay` or `overlay2`.

> **Note**: If you use OverlayFS, use the `overlay2` driver rather than the
> `overlay` driver, because it is more efficient in terms of inode utilization.
> To use the new driver, you need version 4.0 or higher of the Linux kernel,
> unless you are a Docker EE user on RHEL or CentOS, in which case you need
> version 3.10.0-514 or higher of the kernel and to follow some extra steps.
>
storage driver as `overlay2`.

> **Note**: The `overlay` driver is deprecated in Docker 18.09 and later.
> Docker recommends `overlay2` for its storage driver.
>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The information about what kernel is required for overlay2 ("you need version 4.0 or higher of the Linux kernel"...) is useful and should not be removed.

> For more information about differences between `overlay` vs `overlay2`, check
> [Docker storage drivers](select-storage-driver.md).

The `overlay2` driver only works with two layers. This means that multi-layered
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is about the overlay driver (which only can work with a single "lowerdir". The overlay2 driver does not have this limitation, and doesn't have to do the hardlink trick to workaround the "single lowerdir" limitation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thaJeztah can you please explain to me what edits need to be done? thanks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What @thaJeztah means is this paragraph describes the details of overlay and is not applicable to overlay2.

images cannot be implemented as multiple OverlayFS layers. Instead, each image
layer is implemented as its own directory under `/var/lib/docker/overlay2`. Hard
links are then used as a space-efficient way to reference data shared with lower
layers. As of Docker 1.10, image layer IDs no longer correspond to directory
names in `/var/lib/docker/`.

To create a container, the `overlay` driver combines the directory representing
the image's top layer plus a new directory for the container. The image's top
layer is the `lowerdir` in the overlay and is read-only. The new directory for
the container is the `upperdir` and is writable.
## Prerequisites

OverlayFS is supported if you meet the following prerequisites:

- The `overlay2` driver is supported for Docker EE 17.06.02-ee5 and later and
recommended for Docker CE.

- The `overlay` driver is allowed but not recommended for Docker CE.

- Version 4.0 or higher of the Linux kernel, or RHEL or CentOS using
version 3.10.0-514 of the kernel or higher. Docker EE users using kernels older
than 4.0 need to follow some extra steps, outlined below.
Expand All @@ -56,16 +62,12 @@ OverlayFS is supported if you meet the following prerequisites:
re-create them later.


## Configure Docker with the `overlay` or `overlay2` storage driver
## Configure Docker with the `overlay2` storage driver

It is highly recommended that you use the `overlay2` driver if possible, rather
than the `overlay` driver. The `overlay` driver is **not** supported for
Docker EE.
For the `overlay2` driver support, use the following:

To configure Docker to use the `overlay` storage driver your Docker host must be
running version 3.18 of the Linux kernel (preferably newer) with the overlay
kernel module loaded. For the `overlay2` driver, the version of your kernel must
be 4.0 or newer.
* Kernel 4.x (any distro, except for RHEL/CentOS, which don't have a 4.x kernel)
* The RHEL/CentOS 3.10.x kernel

Before following this procedure, you must first meet all the
[prerequisites](#prerequisites).
Expand Down Expand Up @@ -116,8 +118,6 @@ Before following this procedure, you must first meet all the
> later enable `overlay2` by default and do not require
> `override_kernel_check`.

If you need to use the legacy `overlay` driver, specify it instead.

More storage options are available. See all storage options for each storage
driver:

Expand All @@ -132,7 +132,7 @@ Before following this procedure, you must first meet all the
$ sudo systemctl start docker
```

4. Verify that the daemon is using the `overlay`/`overlay2` storage driver.
4. Verify that the daemon is using the `overlay2` storage driver.
Use the `docker info` command and look for `Storage Driver` and
`Backing filesystem`.

Expand All @@ -156,18 +156,13 @@ its compatibility with different backing filesystems.

## How the `overlay2` driver works

If you are still using the `overlay` driver rather than `overlay2`, see
[How the overlay driver works](#how-the-overlay-driver-works) instead.

OverlayFS layers two directories on a single Linux host and presents them as
a single directory. These directories are called _layers_ and the unification
process is referred to as a _union mount_. OverlayFS refers to the lower directory
as `lowerdir` and the upper directory a `upperdir`. The unified view is exposed
through its own directory called `merged`.

While the `overlay` driver only works with a single lower OverlayFS layer and
hence requires hard links for implementation of multi-layered images, the
`overlay2` driver natively supports up to 128 lower OverlayFS layers. This
The `overlay2` driver natively supports up to 128 lower OverlayFS layers. This
capability provides better performance for layer-related Docker commands such
as `docker build` and `docker commit`, and consumes fewer inodes on the backing
filesystem.
Expand Down Expand Up @@ -245,57 +240,21 @@ $ ls /var/lib/docker/overlay2/223c2864175491657d238e2664251df13b63adb8d050924fd1
etc sbin usr var
```

To view the mounts which exist when you use the `overlay` storage driver with
To view the mounts which exist when you use the `overlay2` storage driver with
Docker, use the `mount` command. The output below is truncated for readability.

```bash
$ mount | grep overlay
$ mount | grep overlay2

overlay on /var/lib/docker/overlay2/9186877cdf386d0a3b016149cf30c208f326dca307529e646afce5b3f83f5304/merged
type overlay (rw,relatime,
type overlay2 (rw,relatime,
lowerdir=l/DJA75GUWHWG7EWICFYX54FIOVT:l/B3WWEFKBG3PLLV737KZFIASSW7:l/JEYMODZYFCZFYSDABYXD5MF6YO:l/UL2MW33MSE3Q5VYIKBRN4ZAGQP:l/NFYKDW6APBCCUCTOUSYDH4DXAT:l/6Y5IM2XC7TSNIJZZFLJCS6I4I4,
upperdir=9186877cdf386d0a3b016149cf30c208f326dca307529e646afce5b3f83f5304/diff,
workdir=9186877cdf386d0a3b016149cf30c208f326dca307529e646afce5b3f83f5304/work)
```

The `rw` on the second line shows that the `overlay` mount is read-write.

## How the `overlay` driver works
The `rw` on the second line shows that the `overlay2` mount is read-write.

This content applies to the `overlay` driver only. Docker recommends using the
`overlay2` driver, which works differently. See
[How the overlay2 driver works](#how-the-overlay2-driver-works)
for `overlay2`.

OverlayFS layers two directories on a single Linux host and presents them as
a single directory. These directories are called _layers_ and the unification
process is referred to as a _union mount_. OverlayFS refers to the lower directory
as `lowerdir` and the upper directory a `upperdir`. The unified view is exposed
through its own directory called `merged`.

The diagram below shows how a Docker image and a Docker container are layered.
The image layer is the `lowerdir` and the container layer is the `upperdir`.
The unified view is exposed through a directory called `merged` which is
effectively the containers mount point. The diagram shows how Docker constructs
map to OverlayFS constructs.

![overlayfs lowerdir, upperdir, merged](images/overlay_constructs.jpg)

Where the image layer and the container layer contain the same files, the
container layer "wins" and obscures the existence of the same files in the image
layer.

The `overlay` driver only works with two layers. This means that multi-layered
images cannot be implemented as multiple OverlayFS layers. Instead, each image
layer is implemented as its own directory under `/var/lib/docker/overlay`. Hard
links are then used as a space-efficient way to reference data shared with lower
layers. As of Docker 1.10, image layer IDs no longer correspond to directory
names in `/var/lib/docker/`.

To create a container, the `overlay` driver combines the directory representing
the image's top layer plus a new directory for the container. The image's top
layer is the `lowerdir` in the overlay and is read-only. The new directory for
the container is the `upperdir` and is writable.

### Image and container layers on-disk

Expand All @@ -319,15 +278,15 @@ Status: Downloaded newer image for ubuntu:latest

#### The image layers

Each image layer has its own directory within `/var/lib/docker/overlay/`, which
Each image layer has its own directory within `/var/lib/docker/overlay2/`, which
contains its contents, as shown below. The image layer IDs do not correspond to
the directory IDs.

> **Warning**: Do not directly manipulate any files or directories within
> `/var/lib/docker/`. These files and directories are managed by Docker.

```bash
$ ls -l /var/lib/docker/overlay/
$ ls -l /var/lib/docker/overlay2/

total 20
drwx------ 3 root root 4096 Jun 20 16:11 38f3ed2eac129654acef11c32670b534670c3a06e483fce313d72e3e0a15baa8
Expand All @@ -342,23 +301,23 @@ hard links to the data that is shared with lower layers. This allows for
efficient use of disk space.

```bash
$ ls -i /var/lib/docker/overlay/38f3ed2eac129654acef11c32670b534670c3a06e483fce313d72e3e0a15baa8/root/bin/ls
$ ls -i /var/lib/docker/overlay2/38f3ed2eac129654acef11c32670b534670c3a06e483fce313d72e3e0a15baa8/root/bin/ls
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll have to revisit this section (haven't had time yet to check if it's still current)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thaJeztah please let me know. Thanks.


19793696 /var/lib/docker/overlay/38f3ed2eac129654acef11c32670b534670c3a06e483fce313d72e3e0a15baa8/root/bin/ls
19793696 /var/lib/docker/overlay2/38f3ed2eac129654acef11c32670b534670c3a06e483fce313d72e3e0a15baa8/root/bin/ls

$ ls -i /var/lib/docker/overlay/55f1e14c361b90570df46371b20ce6d480c434981cbda5fd68c6ff61aa0a5358/root/bin/ls
$ ls -i /var/lib/docker/overlay2/55f1e14c361b90570df46371b20ce6d480c434981cbda5fd68c6ff61aa0a5358/root/bin/ls

19793696 /var/lib/docker/overlay/55f1e14c361b90570df46371b20ce6d480c434981cbda5fd68c6ff61aa0a5358/root/bin/ls
19793696 /var/lib/docker/overlay2/55f1e14c361b90570df46371b20ce6d480c434981cbda5fd68c6ff61aa0a5358/root/bin/ls
```

#### The container layer

Containers also exist on-disk in the Docker host's filesystem under
`/var/lib/docker/overlay/`. If you list a running container's subdirectory
`/var/lib/docker/overlay2/`. If you list a running container's subdirectory
using the `ls -l` command, three directories and one file exist:

```bash
$ ls -l /var/lib/docker/overlay/<directory-of-running-container>
$ ls -l /var/lib/docker/overlay2/<directory-of-running-container>

total 16
-rw-r--r-- 1 root root 64 Jun 20 16:39 lower-id
Expand All @@ -371,7 +330,7 @@ The `lower-id` file contains the ID of the top layer of the image the container
is based on, which is the OverlayFS `lowerdir`.

```bash
$ cat /var/lib/docker/overlay/ec444863a55a9f1ca2df72223d459c5d940a721b2288ff86a3f27be28b53be6c/lower-id
$ cat /var/lib/docker/overlay2/ec444863a55a9f1ca2df72223d459c5d940a721b2288ff86a3f27be28b53be6c/lower-id

55f1e14c361b90570df46371b20ce6d480c434981cbda5fd68c6ff61aa0a5358
```
Expand All @@ -384,27 +343,26 @@ comprises the view of the filesystem from within the running container.

The `work` directory is internal to OverlayFS.

To view the mounts which exist when you use the `overlay` storage driver with
To view the mounts which exist when you use the `overlay2` storage driver with
Docker, use the `mount` command. The output below is truncated for readability.

```bash
$ mount | grep overlay
$ mount | grep overlay2

overlay on /var/lib/docker/overlay/ec444863a55a.../merged
type overlay (rw,relatime,lowerdir=/var/lib/docker/overlay/55f1e14c361b.../root,
upperdir=/var/lib/docker/overlay/ec444863a55a.../upper,
workdir=/var/lib/docker/overlay/ec444863a55a.../work)
overlay on /var/lib/docker/overlay2/ec444863a55a.../merged
type overlay2 (rw,relatime,lowerdir=/var/lib/docker/overlay/55f1e14c361b.../root,
upperdir=/var/lib/docker/overlay2/ec444863a55a.../upper,
workdir=/var/lib/docker/overlay2/ec444863a55a.../work)
```

The `rw` on the second line shows that the `overlay` mount is read-write.


## How container reads and writes work with `overlay` or `overlay2`
## How container reads and writes work with `overlay2`

### Reading files

Consider three scenarios where a container opens a file for read access with
overlay.
OverlayFS.

- **The file does not exist in the container layer**: If a container opens a
file for read access and the file does not already exist in the container
Expand All @@ -427,7 +385,7 @@ Consider some scenarios where files in a container are modified.

- **Writing to a file for the first time**: The first time a container writes
to an existing file, that file does not exist in the container (`upperdir`).
The `overlay`/`overlay2` driver performs a *copy_up* operation to copy the file
The `overlay2` driver performs a *copy_up* operation to copy the file
from the image (`lowerdir`) to the container (`upperdir`). The container then
writes the changes to the new copy of the file in the container layer.

Expand All @@ -443,11 +401,8 @@ Consider some scenarios where files in a container are modified.

- OverlayFS only works with two layers. This means that performance should
be better than AUFS, which can suffer noticeable latencies when searching
for files in images with many layers. This advantage applies to both
`overlay` and `overlay2` drivers. `overlayfs2` is slightly
less performant than `overlayfs` on initial read, because it must look
through more layers, but it caches the results so this is only a small
penalty.
for files in images with many layers. This advantage applies to the `overlay2`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This advantage actually was for the overlay driver (as explained in the removed part); overlay2 has a performance penalty on first read (as it has multiple lowerdirs), but once cached, that performance penalty no longer applies.

driver.

- **Deleting files and directories**:

Expand All @@ -470,14 +425,14 @@ Consider some scenarios where files in a container are modified.

## OverlayFS and Docker Performance

Both `overlay2` and `overlay` drivers are more performant than `aufs` and
`devicemapper`. In certain circumstances, `overlay2` may perform better than
`btrfs` as well. However, be aware of the following details.
The `overlay2` driver is more performant than `aufs`. In certain circumstances,
`overlay2` may perform better than `btrfs` as well. However, be aware of the
following details.

- **Page Caching**. OverlayFS supports page cache sharing. Multiple containers
accessing the same file share a single page cache entry for that file. This
makes the `overlay` and `overlay2` drivers efficient with memory and a good
option for high-density use cases such as PaaS.
makes the `overlay2` drivers efficient with memory and a good option for
high-density use cases such as PaaS.

- **copy_up**. As with AUFS, OverlayFS performs copy-up operations
whenever a container writes to a file for the first time. This can add latency
Expand All @@ -490,14 +445,6 @@ Both `overlay2` and `overlay` drivers are more performant than `aufs` and
far larger latencies if searching through many AUFS layers. `overlay2` supports
multiple layers as well, but mitigates any performance hit with caching.

- **Inode limits**. Use of the `overlay` storage driver can cause excessive
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if we should add a page about the overlay driver and capture this information there.

This issue has been the major pain-point for the legacy overlay driver; people ran out of inodes (resulting in "no space on device") whereas actually there was enough space left, only not enough inodes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thaJeztah Since we're deprecating the overlay driver, why should we keep documentation on the older driver? If you want to have something on why we are deprecating it, I'm for it.

@andrewhsu What are your thoughts?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should have that info because even if the driver is deprecated, there are still people using it (and will be, for at least a few years). Having a doc to help them out is good, especially since we already have that info written -- it just need to be moved into a separate doc having a preamble/disclaimer like "Note overlay is deprecated and should not be used; the information below is kept here mostly for historical purposes" or so.

inode consumption. This is especially true in the presence of a large number
of images and containers on the Docker host. The only way to increase the
number of inodes available to a filesystem is to reformat it. To avoid running
into this issue, it is highly recommended that you use `overlay2` if at all
possible.


### Performance best practices

The following generic performance best practices also apply to OverlayFS.
Expand Down
2 changes: 1 addition & 1 deletion storage/storagedriver/select-storage-driver.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ the Docker edition you use.

In addition, Docker does not recommend any configuration that requires you to
disable security features of your operating system, such as the need to disable
`selinux` if you use the `overlay` or `overlay2` driver on CentOS.
`selinux` if you use the `overlay2` driver on CentOS.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to double check selinux; IIRC, SELinux is now supported with overlay/overlay2, but there may be some limitations still

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thaJeztah Can you please let me know what the limitations are?


### Docker Engine - Enterprise and Docker Enterprise

Expand Down