Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Details on how to change data dir when running Docker on Windows #18

Merged
merged 2 commits into from
Mar 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 38 additions & 1 deletion storage-provider-guide/storage-provider-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,4 +110,41 @@ Save blobs error err="missing trie node c4b8803bb72e5841894b5348db64a818f2e6481e
```
If you encounter the `missing trie node` error as described, it is likely that your es-node attempted to access the L1 state older than 256 blocks without success. This may be due to your execution layer endpoint not being an archive node.

If you are using a `BlockPI` endpoint, please navigate to the dashboard, access the detailed page of your API endpoint, and enable the `Archive Mode` within the `Advanced Features` section.
If you are using a `BlockPI` endpoint, please navigate to the dashboard, access the detailed page of your API endpoint, and enable the `Archive Mode` within the `Advanced Features` section.


### Why should I use WSL to run Docker instead of Windows cmd?

If you are using a Windows system, make sure to use WSL to run the docker command instead of running it directly from the command prompt (cmd). The reason for this is that in Windows cmd, it seems that the volume option (-v) for a relative path is not taking effect, which causes the data to be stored inside the Docker container. This can lead to data loss when the container needs to be upgraded. If an absolute path is provided, an "operation not supported" error occurs when creating the data file, which is caused by the program cannot allocate space for the file accross operating system.

### When running es-node in Docker on Windows, I want to store data files on a disk other than `C:`. How can I achieve this?

First of all, make sure you are running `docker run` command on WSL 2 (Ubuntu distro by default) instead of Windows cmd.

Secondly, when start es-node with Docker image, we use the volumes option (-v) for storing data files outside containers to achive persistent storage. However the following attempt to mount an absolute path (e.g., `D:\es-data`) to the container would failed with "operation not supported" error.

```sh
# does not work!
docker run --name es -d \
-v /mnt/d/es-data:/es-node/es-data \
...
```

The following approach will relocate the Ubuntu distro to the target volume (e.g. `D:`) so that `-v ./es-data:/es-node/es-data` can work properly.

By default the WSL system files are mapped to a specific directory on the `C` drive. To relocate the system to another volume (e.g., `D:`), execute the following command in `PowerShell`:

```sh
# create new location in the target volume
> mkdir D:\wsl-ubuntu
> wsl --shutdown
# verify Ubuntu is stopped
> wsl -l -v
> wsl --export Ubuntu ubuntu.tar
> wsl --unregister Ubuntu
> wsl --import Ubuntu D:\wsl-ubuntu\ .\ubuntu.tar --version 2
```
and reboot the computer at the end.


Now you can open a WSL window (Ubuntu on Windows) and start Docker container with the command [here](/storage-provider-guide/tutorials.md#from-a-docker-image).
20 changes: 16 additions & 4 deletions storage-provider-guide/tutorials.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ This guide provides practical steps for the storage providers to start an es-nod

### System Environment

* MacOS Version 14+, Ubuntu 20.04+ or Windows with WSL
* MacOS Version 14+, Ubuntu 20.04+, or Windows with WSL (Windows Subsystem for Linux) version 2
* (Optional) Docker 24.0.5+ (would simplify the process)
* (Optional) Go 1.20+ and Node.js 16+ (can be installed following the [steps](tutorials.md#install-dependencies))

Expand Down Expand Up @@ -79,7 +79,7 @@ Before running es-node from the pre-built executables, ensure that you have inst

Download the pre-built package suitable for your platform:

Linux x86-64 or WSL (Windows Subsystem for Linux):
Linux x86-64 or WSL:

```sh
curl -L https://github.com/ethstorage/es-node/releases/download/v0.1.10/es-node.v0.1.10.linux-amd64.tar.gz | tar -xz
Expand All @@ -106,7 +106,7 @@ env ES_NODE_STORAGE_MINER=<miner> ES_NODE_SIGNER_PRIVATE_KEY=<private_key> ./run

### From a Docker image

Run an es-node container in one step:
Run an es-node container with a single command like following. (If you are using Windows, execute the command in WSL):

```sh
docker run --name es -d \
Expand All @@ -122,11 +122,23 @@ docker run --name es -d \
--l1.beacon <cl_rpc>
```

You can check docker logs using the following command:
After launch, you can check docker logs using the following command:

```sh
docker logs -f es
```
#### Mount data location using Docker volume option

Docker volumes (-v) are a mechanism for storing data outside containers.
In the above `docker run` command , you have the flexibility to modify the data file location on your host machine, ensuring that the disk space requirements are fulfilled. For example:

```sh
docker run --name es -d \
-v /host/path/with/large/space:/es-node/es-data \
...
```
> ℹ️ **_Note:_** The absolute host path does not function well on Windows, for more details please refer [here](/storage-provider-guide/storage-provider-faq.md#when-running-es-node-in-docker-on-windows-i-want-to-store-data-files-on-a-disk-other-than-c-how-can-i-achieve-this)


### From source code

Expand Down