unable to install benchbot, issac-sim docker pull fails

[cjpurackal]

The issac-sim docker pull fails at round 1.74GB. I've tried this multiple times at different times. What could possible be the issue here?
PS: I'm running this on a 90 GB partition.

`
sha256:059811a8ca6017b38b060975cacf598979aa4326921f1e6f7584da1278935a68 1.74GB / 3.74GB
FROM nvcr.io/nvidia/isaac-sim:2021.2.1@sha256:4a9a1cfadc276c72dc15b64fde4b08c1030ca92005f480605950e54c324c0708
error: failed to solve: failed commit on ref "layer-sha256:059811a8ca6017b38b060975cacf598979aa4326921f1e6f7584da1278935a68": "layer-sha256:059811a8ca6017b38b060975cacf598979aa4326921f1e6f7584da1278935a68" failed size validation: 1742733312 != 3743189127: failed precondition

ERROR: Building BenchBot simulator 'sim_omni' returned a non-zero error code: 1

Ensure that Docker has been installed correctly AND that you can run Docker
WITHOUT root access (there is no need to ever run Docker with root). See
https://docs.docker.com/install/linux/linux-postinstall/ for details on how to
fix this.

If the error is more generic, please contact us so that we can update our
pre-install host system checks.
`

[btalb]

Thanks for sharing this @cjpurackal .

This error appears to be coming from further down, at the Docker level.

This issue (containerd/containerd#2306) seems to suggest it can be caused by an incomplete download of a layer.

Can you try doing the following:

1. Check the space on your partition, how much free space do you have remaining?
2. Delete any downloaded images so the download can start again from scratch. A system prune (docker system prune) should be sufficient (double check the images are gone from your docker images list afterwards).

[cjpurackal]

When I'm installing benchbot, I have around 71 GB of free memory. I did try docker system prune (removed all the images) but still getting the same error. Do I need to have more memory ?

[btalb]

I tried reproducing this locally but haven't had any success.

After an installation has failed can you show me how much free space you have remaining by posting the output of df -h please.

[cjpurackal]

Seems like it worked when I tried today but there seems to be an issue when I'm running it (benchbot_run --robot carter_omni --env miniroom:1 --task semantic_slam:active:ground_truth). The issue seems similar to #18

[btalb]

I'm going to close this issue off @cjpurackal .

If the remedies in #18 don't help resolve your issue, please open another issue with some logs and I'll be happy to help.