support download files for Offline deployment

[willzhang]

What would you like to be added:
support download files for Offline deployment.

Why is this needed:
Offline deployment too hard now.

ref here : https://kubespray.io/#/docs/offline-environment

it say i need a HTTP reverse proxy/cache/mirror to serve some static files (zips and binaries),

so what i do:

1、step 1 : generate files.list by generate_list.sh

https://github.com/kubernetes-sigs/kubespray/tree/master/contrib/offline#generate_listsh

$ cd contrib/offline
$ bash generate_list.sh
$ tree temp
temp
├── files.list
├── generate.sh
└── images.list

2、step 2 : download files

wget -P temp/files -i temp/files.list

and the temp/files will be

root@ubuntu:~/kubespray-2.19.0/contrib/offline# tree temp/files
temp/files
└── kubeadm
└── kubelet
└── kubectl

3、step 3 : copy files to my nginx file server

apt install -y nginx
cp -r  temp/files /usr/share/nginx/html/

then the problems is i must change env one by one in offline.yml
https://github.com/kubernetes-sigs/kubespray/blob/master/inventory/sample/group_vars/all/offline.yml

# files_repo: "http://192.168.93.23"
# kubeadm_download_url: "{{ files_repo }}/kubernetes/{{ kube_version }}/kubeadm"

must change all to ---->

# files_repo: "http://192.168.93.23"
kubeadm_download_url: "{{ files_repo }}/files/kubeadm"

what need

 bash generate_list.sh --download --file-list temp/files.list --save-dir temp/files

and the temp/files will be

root@ubuntu:~/kubespray-2.19.0/contrib/offline# tree temp/files
temp/files
└── kubernetes
    └── v1.23.7
        └── kubeadm

and offline.yml default is follow , and no need to change anything

# files_repo: "http://192.168.93.23"
# kubeadm_download_url: "{{ files_repo }}/kubernetes/{{ kube_version }}/kubeadm"

[ErikJiang]

that's exactly what I need, so leave it to me, I will handle it😁

[willzhang]

or wget with -x option

wget -x -P temp/files -i temp/files.list

and the temp/files will be

root@ubuntu:~/kubespray-2.19.0/contrib/offline# tree temp/files
temp/files
├── get.helm.sh
│   └── helm-v3.8.2-linux-amd64.tar.gz
├── github.com
│   ├── containerd
│   │   ├── containerd
│   │   │   └── releases
│   │   │       └── download
│   │   │           └── v1.6.4
│   │   │               └── containerd-1.6.4-linux-amd64.tar.gz
│   │   └── nerdctl
│   │       └── releases
│   │           └── download
│   │               └── v0.19.0
│   │                   └── nerdctl-0.19.0-linux-amd64.tar.gz
│   ├── containernetworking
│   │   └── plugins
│   │       └── releases
│   │           └── download
│   │               └── v1.1.1
│   │                   └── cni-plugins-linux-amd64-v1.1.1.tgz
│   ├── containers
│   │   ├── crun
│   │   │   └── releases
│   │   │       └── download
│   │   │           └── 1.4.4
│   │   │               └── crun-1.4.4-linux-amd64
│   │   └── youki
│   │       └── releases
│   │           └── download
│   │               └── v0.0.1
│   │                   └── youki_v0_0_1_linux.tar.gz
│   ├── etcd-io
│   │   └── etcd
│   │       └── releases
│   │           └── download
│   │               └── v3.5.3
│   │                   └── etcd-v3.5.3-linux-amd64.tar.gz
│   ├── kata-containers
│   │   └── kata-containers
│   │       └── releases
│   │           └── download
│   │               └── 2.4.1
│   │                   └── kata-static-2.4.1-x86_64.tar.xz
│   ├── kubernetes-sigs
│   │   ├── cri-tools
│   │   │   └── releases
│   │   │       └── download
│   │   │           └── v1.23.0
│   │   │               └── crictl-v1.23.0-linux-amd64.tar.gz
│   │   └── krew
│   │       └── releases
│   │           └── download
│   │               └── v0.4.2
│   │                   └── krew-linux_amd64.tar.gz
│   ├── Mirantis
│   │   └── cri-dockerd
│   │       └── releases
│   │           └── download
│   │               └── v0.2.0
│   │                   └── cri-dockerd-v0.2.0-linux-amd64.tar.gz
│   ├── opencontainers
│   │   └── runc
│   │       └── releases
│   │           └── download
│   │               └── v1.1.1
│   │                   └── runc.amd64
│   └── projectcalico
│       └── calico
│           ├── archive
│           │   └── v3.22.3.tar.gz
│           └── releases
│               └── download
│                   └── v3.22.3
│                       └── calicoctl-linux-amd64
└── storage.googleapis.com
    ├── gvisor
    │   └── releases
    │       └── release
    │           └── 20210921
    │               └── x86_64
    │                   ├── containerd-shim-runsc-v1
    │                   └── runsc
    └── kubernetes-release
        └── release
            └── v1.23.7
                └── bin
                    └── linux
                        └── amd64
                            ├── kubeadm
                            ├── kubectl
                            └── kubelet

72 directories, 19 files

then

root@ubuntu:~/kubespray-2.19.0/contrib/offline# cat temp/files
......
https://storage.googleapis.com/kubernetes-release/release/v1.23.7/bin/linux/amd64/kubeadm

change this in offline.yml

# kubeadm_download_url: "{{ files_repo }}/kubernetes/{{ kube_version }}/kubeadm"

let it default to be:

# kubeadm_download_url: "{{ files_repo }}/storage.googleapis.com/kubernetes-release/release/{{ kube_version }}/bin/linux/amd64/kubeadm"

[ErikJiang]

or wget with -x option

wget -x -P temp/files -i temp/files.list

and the temp/files will be

root@ubuntu:~/kubespray-2.19.0/contrib/offline# tree temp/files
temp/files
├── get.helm.sh
│   └── helm-v3.8.2-linux-amd64.tar.gz
├── github.com
│   ├── containerd
│   │   ├── containerd
│   │   │   └── releases
│   │   │       └── download
│   │   │           └── v1.6.4
│   │   │               └── containerd-1.6.4-linux-amd64.tar.gz
│   │   └── nerdctl
│   │       └── releases
│   │           └── download
│   │               └── v0.19.0
│   │                   └── nerdctl-0.19.0-linux-amd64.tar.gz
│   ├── containernetworking
│   │   └── plugins
│   │       └── releases
│   │           └── download
│   │               └── v1.1.1
│   │                   └── cni-plugins-linux-amd64-v1.1.1.tgz
│   ├── containers
│   │   ├── crun
│   │   │   └── releases
│   │   │       └── download
│   │   │           └── 1.4.4
│   │   │               └── crun-1.4.4-linux-amd64
│   │   └── youki
│   │       └── releases
│   │           └── download
│   │               └── v0.0.1
│   │                   └── youki_v0_0_1_linux.tar.gz
│   ├── etcd-io
│   │   └── etcd
│   │       └── releases
│   │           └── download
│   │               └── v3.5.3
│   │                   └── etcd-v3.5.3-linux-amd64.tar.gz
│   ├── kata-containers
│   │   └── kata-containers
│   │       └── releases
│   │           └── download
│   │               └── 2.4.1
│   │                   └── kata-static-2.4.1-x86_64.tar.xz
│   ├── kubernetes-sigs
│   │   ├── cri-tools
│   │   │   └── releases
│   │   │       └── download
│   │   │           └── v1.23.0
│   │   │               └── crictl-v1.23.0-linux-amd64.tar.gz
│   │   └── krew
│   │       └── releases
│   │           └── download
│   │               └── v0.4.2
│   │                   └── krew-linux_amd64.tar.gz
│   ├── Mirantis
│   │   └── cri-dockerd
│   │       └── releases
│   │           └── download
│   │               └── v0.2.0
│   │                   └── cri-dockerd-v0.2.0-linux-amd64.tar.gz
│   ├── opencontainers
│   │   └── runc
│   │       └── releases
│   │           └── download
│   │               └── v1.1.1
│   │                   └── runc.amd64
│   └── projectcalico
│       └── calico
│           ├── archive
│           │   └── v3.22.3.tar.gz
│           └── releases
│               └── download
│                   └── v3.22.3
│                       └── calicoctl-linux-amd64
└── storage.googleapis.com
    ├── gvisor
    │   └── releases
    │       └── release
    │           └── 20210921
    │               └── x86_64
    │                   ├── containerd-shim-runsc-v1
    │                   └── runsc
    └── kubernetes-release
        └── release
            └── v1.23.7
                └── bin
                    └── linux
                        └── amd64
                            ├── kubeadm
                            ├── kubectl
                            └── kubelet

72 directories, 19 files

then

root@ubuntu:~/kubespray-2.19.0/contrib/offline# cat temp/files
......
https://storage.googleapis.com/kubernetes-release/release/v1.23.7/bin/linux/amd64/kubeadm

change this in offline.yml

# kubeadm_download_url: "{{ files_repo }}/kubernetes/{{ kube_version }}/kubeadm"

let it default to be:

# kubeadm_download_url: "{{ files_repo }}/storage.googleapis.com/kubernetes-release/release/{{ kube_version }}/bin/linux/amd64/kubeadm"

@willzhang this might work, but the directory hierarchy is too deep.😀

[willzhang]

greate job, thanks.