Cleanup dockerfiles, remove unused ones

[jngrad]

• update ROCm to 3.3 (closes Updated ROCm to latest #166)
• pin Fedora to 31
• cleanup non-Ubuntu dockerfiles, add GDB and pip
• remove unused Ubuntu dockerfiles

[jngrad]

Oh, we actually don't deploy built images to the registry from a PR. I'll update the CI docs.

[mkuron]

• update ROCm to 3.3 (closes Updated ROCm to latest #166)
• pin Fedora to 31

What do we want to do to keep up with latest on these? Both release significantly more often than Ubuntu LTS, so we need some kind of procedure (e.g. automated weekly tests against latest, someone who manually tests every month, etc.).

[KaiSzuttor]

What part of the rocm 3.3 gets updated frequently and is relevant to us? Since we not have any user for amd gpus yet, IIRC we decided to keep the maintainance effort as low as possible.

[jngrad]

It's about following up on rocm 3.4, 3.5, and so on. They release a new minor version every 3 months. Fedora also has a fast release cycle, they even test packages (including ESPResSo) on unstable to get feedback very early in their release cycle. Our new workflow generates static docker images: one tag is one image. To be notified of breaking changes in future releases of ROCm and Fedora, we could set up a monthly build on GitLab-CI that uses a sed expression to replace the version tag with latest before building and testing the dockerfiles.

[KaiSzuttor]

we do not want to increase the complexity again by introducing a second place for building images. Is it really important for us to keep up with such a fast release cycle, especially for the case of the rocm project that has shown to release poorly tested code. we can probably save some dev time here by choosing a more “conservative” approach.

[mkuron]

To be notified of breaking changes in future releases of ROCm and Fedora, we could set up a monthly build on GitLab-CI that uses a sed expression

Sounds good. Could you also do that for CUDA? That tends to break with every release too.

Is it really important for us to keep up with such a fast release cycle

If you pin a dependency to a specific version forever, you might not support it at all because it makes it difficult to use for the average user if the current version is not tested and thus likely incompatible. Also, it‘s not just rocm. Fedora shows us problems with new versions of libraries, before they hit more common distributions. CUDA breaks with almost every release and we are often way behind in testing/supporting the latest (or even the one before).

[jngrad]

I'd also personally prefer to pin ROCm/CUDA/Fedora and state in the docs we only support specific versions. Testing on the latest version is really exhausting. Even if the sed solution has minimal overhead, what happens if we end up with a ROCm version that requires us to change our CMake logic? We'll have to choose between supporting the newer version, or freezing the older one. Similar logic for Fedora and CUDA: if we were ready to test on latest, we wouldn't need to get ESPResSo packaged by Christoph, we could do it ourselves for Fedora and open a PPA for Ubuntu + CUDA.

[RudolfWeeber]

>update ROCm to 3.3 (closes <#166> #166) >pin Fedora to 31 What do we want to do to keep up with latest on these? Both release significantly more often than Ubuntu LTS, so we need some kind of procedure (e.g. >automated weekly tests against latest, someone who manually tests every month, etc.).

I’d say we do nothing except ahead of Espresso releases. At that point, we have to make final decisions about what to test, in any case. People using the dev branch can cope and report issues, if something arises in between releases.

[mkuron]

state in the docs we only support specific versions

Users don‘t always control the library versions. Sometimes it‘s a reluctant sysadmin, a Linux distributor, or a hardware incompatibility that dictates a certain version. Specifying a maximum supported version in the docs sends a terrible (and false) message about Espresso‘s support state.

do nothing except ahead of Espresso releases.

Still needs to be (semi-)automated. Otherwise it won‘t happen.

[KaiSzuttor]

we can produce a latest tagged image on github and a manual job in the CI

[KaiSzuttor]

why is the image for min-boost not needed anymore?

[jngrad]

we can produce a latest tagged image on github and a manual job in the CI

Nobody's going to click on the manual job in the CI pipeline. See the recent issue with name collision that went un-noticed for weeks because the manual job installation is never triggered. If we want to be notified about breaking updates, it needs to be automatized.

why is the image for min-boost not needed anymore?

Maybe I've misunderstood the core team consensus on boost. I thought we were moving away from older boost versions and only supported 1.65 (or superior) from Ubuntu 18.04 until 2021 (3 years on every Ubuntu LTS). Images currently used in CI have a spectrum of boost versions: 1.65, 1.66, 1.67, 1.69, 1.71. I'm not including here 1.58 from clang:6.0 because we want to drop it soonish.

[KaiSzuttor]

Nobody's going to click on the manual job in the CI pipeline

This has to be part of the release checklist. If you want to be a tactical tornado you could even automate pressing the button on release...

I just wanted to make sure we still test the minimal required boost version.