Server migration of plaza.deegree.org

[tfr42]

The server plaza.deegree.org will be upgraded to a new machine (from EX-41 to EX-42 sponsored by FOSSGIS e.V.).

List of tasks:

• create a private repository on https://github.com/deegree/infrastructure for all infrastructure code (docker compose)
• decide on host name of new server
• new server is created (requires support by FOSSGIS admins)
• basic installation of Ubuntu 20 LTS @stephanr
• setup accounts and login via SSH (fail2ban) @stephanr
• setup Hetzner firewall and fw on local machine @stephanr
• installation of official docker from docker.com @stephanr
• create initial docker compose file with the infrastructure/services @copierrj
• setup apache httpd/nginx/traeffic
• setup the SSL certificate (lets encrypt) @stephanr
• migrate api, documentation, schemas, the old wiki pages @stephanr
• setup nexus using docker @copierrj
• migrate nexus content @copierrj
• migrate nexus from v2 to v3 @copierrj
• setup jenkins using docker @tfr42
• migrate jenkins content @tfr42
• migrate deegree reference implementation (CITE, OGCRI) @dstenger, @stephanr
• update reference implementation workspace (CITE, OGCRI) @dstenger, @stephanr
• migrate deegree demo @tfr42
• fix the remaining issues in the demo workspaces
• complete backup of files from plaza and transfer to central
• change DNS entries (requires OSGeo SAC team, see https://trac.osgeo.org/osgeo/ticket/2568)

see more information about the infrastructure in wiki https://github.com/deegree/deegree3/wiki/Infrastructure

[tfr42]

Opened ticket at OSGeo SAC team to inform about upcoming change, https://trac.osgeo.org/osgeo/ticket/2568.

[stephanr]

In reference to the existing name plaza, I have given the server the name central for the time being.
However, this can also be changed later if another name is selected.

[stephanr]

• The ssh service has been setup for tmc and supporters.
• Fail2ban has been activated in aggressive mode (3 attemps incl. key based) will be banned for a day
• Firewall has been setup to only allow ssh, http(s) and icmp (software and hetzner)

[stephanr]

Webserver-Configurations on plaza prepared for letsencrypt move:

• 000-default.conf - Redirect to www.deegree.org (http)
• 001-default-ssl.conf - Redirect to www.deegree.org (https)
• apidoc.deegree.org.conf - apidoc.deegree.org
• archived_wiki.deegree.org.conf - wiki.deegree.org
• buildserver.deegree.org.conf - buildserver.deegree.org
• demo.deegree.org.conf - demo.deegree.org deegree3-demo.deegree.org deegree3-testing.deegree.org
• download.deegree.org.conf - download.deegree.org
• ogcri.deegree.org.conf - ogcri.deegree.org cite.deegree.org
• repo.deegree.org.conf - repo.deegree.org nexus.deegree.org
• schemas.deegree.org.conf - schemas.deegree.org

[dstenger]

One addition: There is also a PostgreSQL database running on plaza which is used by the CITE reference implementation workspace.

[stephanr]

One addition: There is also a PostgreSQL database running on plaza which is used by the CITE reference implementation workspace.

I looked onto plaza and exported the cite db, but the exports contained the complete postgis part (seems to be a non-extension install).
@dstenger Do you have the source sql/dump for the initial content of the ogcri database as i think it would be better as a dump form the previous db.

P.S. i prepared a pr as template for the database in pr 7 on infrastructure

[dstenger]

Unfortunately, I also do not know how this database was created and do not have any scripts. Thus, I also had the idea of creating a dump.

[robe2]

FWIW if the database was installed using script way instead of extension way, using the backup should work just fine if you

1. Create the database beforehand and install postgis using CREATE EXTENSION in the same schema as the old install.
2. Then restore the custom database backup using pg_restore on top of this new database.
3. Then run SELECT postgis_extensions_upgrade();

As I recall if you functions are tied to an old .so file, they won't install anyway from the backup. So for example if you just install postgis extension and not postgis_raster, all the raster functions will naturally fail on install.

Alternatively you can
You can use postgis_restore to definitely leave out the croft.

Anyway let me know if you need any help on that.

[tfr42]

Migrated old HUDSON_HOME to new JENKINS_HOME including

• scripts into $JENKINS_HOME/bin/update-schemas.sh
• .gitconfig into $JENKINS_HOME
• .ssh into $JENKINS_HOME

[tfr42]

Issued switch of DNS at https://trac.osgeo.org/osgeo/ticket/2568#comment:2

[robe2]

DNS switched - @tfr42 I sent you the zone file to make sure I didn't miss anything. Let me know if more is needed

I forgot to mention I think your @ I still have pointing at the old plaza IP but I don't think anything references it so probably doesn't matter much aside from cleanup

[tfr42]

@robe2 Thanks for the zone file. We will get back to you about the outdated/unused names in trac issue https://trac.osgeo.org/osgeo/ticket/2568.

[tfr42]

Findings after DNS change:

• integration tests fail due to malformed url such as "'http://deegree3-demo.deegree.org:80:80/utah-workspace/services?request=DTD", see http://buildserver.deegree.org/job/deegree-3.4-openjdk8/lastCompletedBuild/org.deegree$deegree-protocol-wms/testReport/org.deegree.protocol.wms.client/WMSClientTest/testWMS111InstantiationFromUrl/
• HTTPS for https://nexus3.deegree.org/ not working
• HTTPS for https://artefacts.deegree.org/ not working
• https://download.deegree.org/documentation/3.4.16/html fails with 403 Forbidden, https://download.deegree.org/documentation/3.4.16/html/ is OK

[tfr42]

Final clean up

• remove https://schemas.deegree.org/central-test.tmp and https://schemas.deegree.org/central-test.dir/

[stephanr]

Findings after DNS change:

• HTTPS for https://artefacts.deegree.org/ not working

As for artefacts.deegree.org, I suggest deleting this entry from the DNS.
This entry was already inactive on plaza (grep -i artefacts /etc/apache2/sites-enabled/*) and previously consisted only of redirects to repo.deegree.org:

Redirect 301 /libs-releases-local http://repo.deegree.org/content/repositories/releases
Redirect 301 /libs-snapshots-local http://repo.deegree.org/content/repositories/snapshots
Redirect 301 /repo http://repo.deegree.org/content/groups/public
Redirect 301 /index.html http://repo.deegree.org/index.html

[stephanr]

• remove https://schemas.deegree.org/central-test.tmp and https://schemas.deegree.org/central-test.dir/

I removed the empty file and directory

• HTTPS for https://nexus3.deegree.org/ not working

After restarting traefik (docker-compose restart traefik), the certificate generation was restarted, a certificate successfully issued and is now active.

[stephanr]

Findings after the DNS change / zone file review

• We have failed to request the change for deegree.org (@ IN A) to the address of central. I would also suggest to request a change of @ IN A to the address of central and also let an IPv6 equivalent @ IN AAAA be added
• We should vote on whether to have aliases drupal and drupal-test removed from the DNS as well.

[tfr42]

• Finding regarding download of workspaces within deegree admin console: https://sourceforge.net/p/deegree/mailman/message/37270584/

[tfr42]

Finding regarding download of workspaces within deegree admin console

The REST API of Nexus changed in v3, download links such as https://repo.deegree.org/repository/releases/org/deegree/deegree-workspace-csw/3.4.16/deegree-workspace-csw-3.4.16.deegree-workspace still work, but the url as in https://github.com/deegree/deegree3/blob/master/deegree-services/deegree-webservices/src/main/java/org/deegree/console/workspace/WorkspaceBean.java#L374 won't work.

[tfr42]

The old server plaza.deegree.org is offline and shutdown.

[tfr42]

Proposal to remove artefacts, drupal and drupal-test from DNS entries

[tfr42]

Issue for SAC team reopened: https://trac.osgeo.org/osgeo/ticket/2568#comment:6

[stephanr]

Finding regarding download of workspaces within deegree admin console

The REST API of Nexus changed in v3, download links such as https://repo.deegree.org/repository/releases/org/deegree/deegree-workspace-csw/3.4.16/deegree-workspace-csw-3.4.16.deegree-workspace still work, but the url as in https://github.com/deegree/deegree3/blob/master/deegree-services/deegree-webservices/src/main/java/org/deegree/console/workspace/WorkspaceBean.java#L374 won't work.

We have provided a workaround that redirects all old requests, so that the current users are not affected. (https://github.com/deegree/infrastructure/pull/19)

[tfr42]

https://nexus.deegree.org shall forward to https://nexus3.deegree.org. Nexus 2.x services can be stopped.

[tfr42]

enable protocol switch for deegree.org and www.deegree.org

[stephanr]

enable protocol switch for deegree.org and www.deegree.org

The redirect now works as intended

[stephanr]

• https://download.deegree.org/documentation/3.4.16/html fails with 403 Forbidden, https://download.deegree.org/documentation/3.4.16/html/ is OK

/var/www/download.deegree.org/documentation/3.4.15/:
total 41292
drwxr-xr-x  3 ...     4096 Jan 30 16:20 html

/var/www/download.deegree.org/documentation/3.4.16/:
total 41292
drwxr-x--x  3 ...     4096 Mar 12 19:42 html

We seem to have a permission problem here, we may have to adjust the release script again.
For the folder 3.4.16 I have adjusted it manually for now.

[tfr42]

All services are up and running. All tasks done. The server migration is completed.

[tfr42]

Announcements to the mailing list:

• https://sourceforge.net/p/deegree/mailman/message/37284464/
• https://sourceforge.net/p/deegree/mailman/message/37286858/